Rotten at the source

The Big Story

Supply and demand

Some of the scariest security problems are the ones lying in plain sight.

This week, security researcher Alex Birsan described how he was able to take advantage of the way some popular open-source package registries handle dependencies, a key cog in the software development process. Birsan hacked into some of the world's most valuable tech companies, including Apple and Microsoft, using a supply-chain method he disclosed responsibly through several corporate bug bounty programs before explaining it in public.

The explanation for the vulnerability is both simple and complicated, but somewhat terrifying considering how much access some of these tools (often managed by overworked and under-appreciated open-source communities) have to trusted machines inside valuable corporate networks.

First: What are software packages and dependencies? Here's a refresher.

• Packages contain a combination of executable files and others that describe what the software does, as well as the information that the software depends on in order to run properly.
• A huge number of basic software functions have already been written, tested and verified, and in the interest of saving future developers from having to do the same work, that metadata is stored in registries that new applications depend on to work properly.
• One of the most popular package registries is npm, acquired by Microsoft's GitHub in 2020, which manages software packages for the widely used Node.js development tool.
• Other programming languages and tools have their own registries, such as Python and Ruby, while companies like JFrog and GitHub also offer registry services.
• There are tons of publicly available open-source software packages containing dependencies on the most popular registries, alongside private packages intended only for internal corporate use.

So what did Birsan actually do? Normally, a developer inside a company starting a new project would pull any required packages from their private registries via a basic command, and without much thought. But Birsan discovered a very simple way to trick developers into downloading malware disguised as a corporate software package onto their development machines, giving him access to anything a developer could access on their company's network.

• After finding code written by PayPal that was mistakenly posted in public on GitHub, Birsan realized that the code used a combination of public and private software packages on the npm registry.
• He then realized that if he created a public package with the same name as a private package, the registry was set up so that it would default to the public package when that name was requested by a developer.
• That meant he could put any code he wanted in that public registry, and the developer would have no idea they were getting malicious code along with the files they wanted when requesting the usual private package.
• Through a mixture of old-fashioned sleuthing and understanding the way Javascript packages work, Birsan was able to find "hundreds" of names for private packages belonging to enormous companies that were not being used for public packages.
• "Squatting valid internal package names was a nearly sure-fire method to get into the networks of some of the biggest tech companies out there, gaining remote code execution, and possibly allowing attackers to add backdoors during builds," he wrote.

So far Birsan has earned a total of $130,000 from various companies that he showed were vulnerable to this type of attack through their bug bounty programs, which reward security researchers for responsibly finding and disclosing security issues that could lead to far more costly attacks in the wrong hands.

• Several of the companies affected have changed the way they handle software packages, and registries are also updating their policies.
• But there are lots of companies that might not realize how their software package dependencies are vulnerable to this type of attack, ones who can't afford lucrative bug bounty programs.
• Microsoft recommended that companies change the way their developers access package registries to prioritize downloading their own private packages first.

Supply-chain hacks are going to be one of the most prominent security threats facing enterprise companies over the next decade, as open-source and private software continue to mix in ever-more complex systems. And as it turns out, you don't need a sophisticated bunch of government-backed hackers to cause real damage.

— Tom Krazit

This Week On Protocol

Another setback: Salesforce kicked off Black History Month with the prominent resignation of Cynthia Perry, a Black senior manager. In a scathing letter, she said she had "been gaslit, manipulated, bullied, neglected, and mostly unsupported … the entire time I've been here." Protocol's Joe Williams reported that it's just another example of how Salesforce has struggled to improve diversity, equity and inclusion inside the company.

Hybrid lyfe:This week's Protocol Braintrust question asked a number of prominent cloud leaders a simple question with a complex answer: What's the most critical step in creating hybrid cloud infrastructure that can scale? Choice, automation and flexibility were some of the most popular answers from the panel, as tallied by Kevin McAllister.

RIP NDAs? California is considering a bill that would prohibit Silicon Valley companies from forcing employees to sign non-disclosure agreements prohibiting them from talking about negative experiences at a former employer, such as racism. Protocol's Emily Birnbaum reported that California law already prohibits NDAs from covering allegations of sexism or sexual harassment, but doesn't extend that protection to other forms of discrimination.

Five Questions For...

Justin Borgman, CEO of Starburst

What was your first tech job?

Working for my high school in the 1990s, installing ethernet cabling in every classroom. It was a great experience. I felt like I was laying the infrastructure for the future, both literally and figuratively.

What was the first computer that made you realize the power of computing and connectivity?

My 486DX2 when I was roughly 12 years old. It wasn't my first computer, but it was the one that made me really fall in love with computing. I learned how to program in Basic, C and C++ on that machine. I learned how to upgrade the hardware components like the network card, memory and graphics card. I also learned how to partition the hard drive and installed RedHat Linux for the first time, which was an eye-opening experience.

What was the biggest reason for the success of cloud computing over the past decade?

The speed with which you can move in the cloud. There is no delay associated with procuring hardware for your project, and that allows for nearly instantaneous experimentation. In an increasingly competitive environment that businesses operate in, one where incumbents can be displaced by startups seemingly overnight, speed really matters. The one who can adapt and iterate the fastest usually wins, and the cloud really enables that.

What will be the biggest challenge for cloud computing over the coming decade?

Costs. The speed and ease with which you can get going can also lead to surprisingly high cloud bills. Customers will need to spend more time and energy on the economics of cloud computing. This means creating leverage with cloud vendors by going multi-cloud as well as maintaining on-prem as an option for some workloads.

Who is your mentor?

I've been lucky enough to have many mentors over my career, but if I had to pick just one it would be my mother. She was the embodiment of integrity, hard work and an unbreakable will; all of which are essential ingredients for a successful entrepreneur.

Around the Enterprise

• File this under "baffling arguments": Fresh off declaring that Microsoft was the "world's largest and most comprehensive cloud platform" two weeks ago, CEO Satya Nadella essentially told Bloomberg on Wednesday that Slack should be happy Microsoft allowed it to exist on Windows. I imagine a lawyer somewhere on the east side of Lake Washington now has a minor headache.
• CBS All Access crashed harder than Kansas City's offensive line Sunday during Super Bowl LV, going down right at kickoff. The service ran on Google Cloud as of 2019, but ViacomCBS and AWS signed a big partnership deal in late 2020.
• A report last week that IBM was shuttering its blockchain team appears to have been premature, according to Ledger Insights. Though it's still not clear what non-cryptocurrency blockchain efforts actually hope to accomplish.
• AWS, Microsoft and Google don't partner on a lot of things, but they're all inaugural members of the Rust Foundation, set up this week to steer development of the intriguing programming language.
• A former Intel engineer has been accused of taking trade secrets related to Intel's Xeon processors over to Microsoft, which reportedly harbors chip design ambitions of its own.
• Google Cloud signed a 10-year deal with Telus, a Canadian wireless carrier. Telus will use Google's Anthos hybrid cloud tool as part of its 5G rollout.
• Oracle rolled out a ruggedized edge-computing device for customers willing to spend $160 a day for computing power and connectivity in difficult-to-reach places.
• Use Slack on Android? It's time to reset your password after Slack inadvertently introduced a software bug that stored those passwords in plaintext.
• Remember JEDI? It's taken so long for the overhyped cloud contract to make its way through procurement and the court system that the Pentagon is starting to wonder if it should even bother, and start fresh.