Protocol | Enterprise
Your guide to the future of enterprise computing, every Monday and Thursday.

Software supply chain security scares spread

​A new backdoor into software production has emerged.

Welcome to Protocol | Enterprise, your comprehensive roundup of everything you need to know about the week in cloud and enterprise software. This Thursday: another supply-chain hack hits an enterprise software vendor, Google's army of data-center contractors, and Linux maintainers invoke the nuclear option against Minnesota.

(Was this email forwarded to you? Sign up here to get it in your inbox every week.)

The Big Story

Cover up

A new software supply-chain hack surfaced this week, making it clear that the massive SolarWinds hack was just a preview of what could await companies that depend on countless vendors for the software they need to operate in the 21st century.

The nature of the product involved in this attack — software development testing tools from a startup called Codecov — is especially troubling because by definition it requires access to sensitive assets on a company's network in order to accomplish its tasks. Codecov is a 35-person seed-stage company whose products are used by a number of high-profile companies, including IBM, HPE and RBC.

In the complex world of modern software development, there are dozens of moving parts, inside even smaller companies. And under pressure to continuously ship software, companies rely on tools that if compromised, could cause more problems than they solve.

  • Codecov's tool allows customers to analyze code running on testing servers in an attempt to find potential errors when that new code is pushed to production.
  • Everybody (well, almost everybody) tests their software before pushing it live, but it can be tricky to know when you've run enough tests to feel confident that code will work.
  • But you can't run indefinite numbers of tests on new code, otherwise you'll never ship anything.
  • So tools like Codecov run tests against new code and generate a result measured against internal performance standards that tells you how much of that code actually executed; the more code that executes, the more likely it is that the software will work.

That means Codecov needs access to its customers' software development assembly lines in order to run those tests. And someone determined to gain access to the internal networks of Codecov customers managed to infiltrate its tool.

  • Codecov notified customers last week that it discovered its software had been compromised with a backdoor as early as January, but it wasn't clear until this week how widespread the damage might be.
  • It's also not entirely clear how hackers gained access to Codecov, but once they did they "put extra effort into using Codecov to get inside other makers of software development programs, as well as companies that themselves provide many customers with technology services, including IBM," Reuters reported.
  • IBM said it had not found any evidence of its own code being altered, but it's not clear if the hackers had used their access to IBM to breach customer networks.
  • Still, the backdoor could have allowed the attackers to siphon loads of login credentials related to key enterprise tech services such as AWS or GitHub, because Codecov's tools would need to know where code was stored and running.

Software supply-chain hacks aren't entirely new, but the SolarWinds breach discovered last year underscored that big compromises at small vendors can have enormous effects.

  • Startups like Codecov have thrived over the past several years because there's just no way even huge companies like IBM can make all the software tools they need themselves.
  • And these attacks are especially damaging, because in order to use tools that sit at the heart of software development pipelines, customers have to trust them with so much sensitive information.

Codecov removed the backdoor from the latest edition of its tool and urged customers to upgrade and change their login credentials. The story of this supply chain hack is just beginning, and it won't be the last such attempt to infiltrate critical infrastructure.

— Tom Krazit

A MESSAGE FROM HEWLETT PACKARD ENTERPRISE

Global lockdown orders and the ongoing pandemic have disrupted supply chains, underscoring the need for greater agility and resilience. On the season premiere of The Element Podcast, leading experts from MIT and Hewlett Packard Enterprise explore how companies are rapidly transforming their supply chains and adapting for a more dynamic and digital future.

Learn more

This Week On Protocol

Data-center gigs: Don't miss this story about life for Google's data-center employees. Protocol's Anna Kramer talked to several current and former employees who, for the most part, like doing the work required to keep servers up and running — but they want a path to full-time employment.

Cloudy gamers: Last year looked like it was going to be the year of cloud gaming thanks to technical advances and the pandemic, but it hasn't quite worked out that way. This week's Protocol Gaming takes a look at how different cloud gaming models fared in 2020, and what might come next.

On the path: UiPath went public Wednesday, raising $1.35 billion for its robotic-process automation software services and ending the day valued at around $36 billion. Protocol's Joe Williams spoke with CFO Ashim Gupta, who acknowledged the intense interest in enterprise cloud stocks right now but said "the best way to combat hype is by progress."

Five Questions For...

David Friend, CEO, Wasabi Technologies

What was your first tech job?

I built the electronic music studio at Yale University as an undergraduate bursary job. This was an incredible experience for me as I'd later go on to co-found ARP Instruments which developed synthesizers used by Stevie Wonder, David Bowie and Led Zeppelin, and even helped Steven Spielberg communicate with aliens providing that legendary five-note communication in "Close Encounters of the Third Kind."

What was the first computer that made you realize the power of computing and connectivity?

My first software company, Computer Pictures Inc., was an early player in computer graphics. I built the Computer Pictures software on an Apple II computer. This was one of the first times I really experienced the extensive power of computing and is what really got me into the business of software.

What was the biggest reason for the success of cloud computing over the past decade?

Although I can only speak to cloud storage specifically, what I have noticed is that no one wants to own generic storage servers anymore. It is cheaper and more simple to store data in the cloud and it obviously comes with less capacity restrictions than on-premises storage. As more data moves to the cloud over time, we'll see the benefits of greater data accessibility on cloud computing as well.

What will be the biggest challenge for cloud computing over the coming decade?

There is a rapidly growing need for large amounts of cloud data storage. One of the challenges we notice, that should be avoided, is not getting locked into a single vendor. To get real value from the cloud, the cloud has to be a richer environment with multiple vendors focusing on parts of the ecosystem that they can do better than anyone else. The hardware world evolved this way, with dozens or hundreds of specialty vendors, and the cloud environment should do the same.

Will the pandemic usher in a new era of remote working, or will we all come back together when it is safe to do so?

I believe workplace operations will go back to 75% of the way they were pre-pandemic with employees going into the office three to four times a week. Although it is possible to complete the majority of their work from home, people still need some amount of face-to-face contact. It energizes us, makes us more creative and ultimately drives greater innovation.

Around the Enterprise

A MESSAGE FROM HEWLETT PACKARD ENTERPRISE

Global lockdown orders and the ongoing pandemic have disrupted supply chains, underscoring the need for greater agility and resilience. On the season premiere of The Element Podcast, leading experts from MIT and Hewlett Packard Enterprise explore how companies are rapidly transforming their supply chains and adapting for a more dynamic and digital future.

Learn more

Thanks for reading — see you Monday.

Recent Issues