Cloud software will be built in new ways. Pulumi CEO Joe Duffy thinks he has the right tool kit

The cloud engineering startup, which counts its namesake infrastructure-as-a-code tool as its flagship technology, has plans to build a whole set of tools for developers who want to work on the cloud.

Photograph of Pulumi CEO Joe Duffy

"[L]ong-term, we're a cloud engineering platform, and we're solving for so much more of the space of what these enterprises have to build internally," said Joe Duffy, Pulumi's co-founder and CEO.

Photo: Pulumi

Pulumi CEO and co-founder Joe Duffy’s vision for his open-source cloud engineering startup is to offer developers and infrastructure teams the tools they need to “make the most out of the cloud.”

With its universal infrastructure-as-code software tool as its flagship technology, the company – which marked its fifth year in March – is approaching 1,000 commercial customers and growing rapidly, according to Duffy, a Microsoft veteran who ran the cloud provider’s developer strategy and managed its programming languages teams.

“The cloud is really transforming everything about how we build software, enabling new capabilities,” Duffy said. “What we're really trying to do is break down the silos and enable developers, infrastructure teams and security experts to just collaborate and build cloud-powered software. Software is changing everything about modern business.”

Duffy talked to Protocol about Pulumi’s start and where it’s headed — including plans for an initial public offering — as well as its competition with HashiCorp’s Terraform and why the company uses AWS as its internal cloud provider.

The interview has been edited and condensed for clarity.

Pulumi has been in business for five-plus years now. How’s it going and can you give any metrics to support that?

In short, it's going really well. We just hit our five-year anniversary in March this year. We came from a developer heritage, and our bet was the future is really all about empowering developers and bringing software engineering to infrastructure teams. What we found was, frankly, what folks are using in this space is not very good. People thought about giving care and love to developers in a fundamentally different way than infrastructure teams.

So we sort of made the big bet, and it took a while to build the initial platform. It took about a year to build the first version. We open-sourced it. We didn't even launch the commercial edition for another year after that, so really we've been in market commercially for three years. Within the last year, things have really just picked up steam — really, the last six months even. I think the market has really changed in sort of the way that we've predicted. Honestly, COVID actually accelerated this. You see a lot more people accelerating their cloud efforts. The idea that developers should be in the driver's seat has picked up steam as well. They're the people building business value, and the more you can empower them with guardrails — hence, security is important — the faster the business can go, and we see that's part of the dialogue for every enterprise we work with.

We're open source first, and so we always think about the open-source community. In terms of metrics … [we’re in the] ballpark [of] 100,000 community members and coming up on almost cracking the 1,000-commercial-customer mark. That's growing very rapidly. I think we're kind of where HashiCorp was three, four years ago, but growing much faster than they are these days, so I think we'll catch up pretty soon. I feel very fortunate, especially in this current macroeconomic climate, that things are going so well.

You compete against HashiCorp’s Terraform. What's your argument for going with Pulumi over Terraform?

There are two things we find. One is really just the level of productivity is much greater with Pulumi. Atlassian is a customer of ours, and they said, “Hey, we can get twice as much done now with Pulumi than we could before.” Mercedes-Benz, Snowflake, they moved from Terraform to Pulumi and found similar results. Snowflake's story was, on their road to IPO they had to ship their data cloud platform, and I think they had some intense deadlines set by the board. Based on their past experience in Terraform, they said, “Hey, we just can't get it done in time with Terraform. We're going to pick Pulumi, and by picking Pulumi, we can empower our entire team.”

Often we find that Terraform silos are formed. There's “Oh, those five people over there in the corner know Terraform, but nobody else does.” Pulumi allows you to use your favorite language — any great programming language like JavaScript or Python or Go, Java. It meets people where they are rather than you having to learn a proprietary language like with Terraform. So that's the first: It's productivity, it's familiarity.

The second is scalability. We often find with increasingly complex cloud architectures, there's a lot more moving pieces. That's exactly the sort of problem that we invented these great programming languages to solve — it's just in a different domain. Over the last 30 years, we built these great programming languages and tools and ecosystems around them to solve these problems for application development. What we're finding is now those same challenges are emerging in infrastructure management, and so we were able to apply those same techniques. And again, because Terraform [has] a proprietary language that doesn't handle a lot of these things, it's sort of a siloed ecosystem, and it doesn't scale. Those are the two things we hear the most from customers.

Do you see a lot of customers making that switch from Terraform, or do you have more customers that just start with Pulumi?

It's a mix. Terraform had a multiyear head start on us, and so there's definitely a lot of Terraform in the market. In some sense, those folks have already felt the pain of those two things that I just mentioned, and so it's a lot easier for them to see “Oh, infrastructure as code is important to me, and yet this thing I'm using didn't satisfy my needs.” Pulumi clearly solves that, so it's an easier path for them. I'd say it's about half and half: Fifty percent are coming from Terraform or another [infrastructure-as-code] tool.

But one thing has changed since we started: We now hear infrastructure as code is “table stakes.” We hear from people, “Oh, I'm going to cloud, so I need infrastructure as code.” What that means is literally anybody who's doing anything in the cloud is a potential customer, and so there's a lot of greenfield opportunity as well.

"[L]ong-term, we're a cloud engineering platform, and we're solving for so much more of the space of what these enterprises have to build internally."

Within larger enterprises, I think our biggest competitor is actually homegrown. A lot of folks, especially on the journey to multicloud, have had to build a lot of custom internal tools and platforms — not because they wanted to, but because they had to. And so really long-term, Pulumi's not really competing with Terraform; long-term, we're a cloud engineering platform, and we're solving for so much more of the space of what these enterprises have to build internally. It just so happens infrastructure as code is our flagship technology. It's where we started, it's where we are today. But over time, we'll grow into more of a management suite for the entire enterprise, and that's where homegrown actually becomes much more of the challenge. But today, purely on infrastructure as code, definitely Terraform is the current market standard.

Can you expand on where you want to go in the future?

I can use sort of an analogy: GitLab … started with source control management, but to compete against GitHub, they really expanded into being a DevOps platform, and source control management is one piece of that. It's a very similar thing with Pulumi.

We started with infrastructure as code, but when it comes time to adopt cloud engineering in an organization, it goes well beyond infrastructure as code. You think of policy as code, you think of “How do I empower my developers with self-serve portals.” You think of observability, you think of all the management tasks — so compliance and auditing and security, enforcing best practices. How do we enable the cloud engineering team — that's developers, infrastructure teams and security experts — to just collaborate in the most seamless way?

What we're really trying to do is break down the silos and enable developers, infrastructure teams and security experts to just collaborate and build cloud-powered software. And that's the vision, that's the future. Infrastructure as code is sort of the first entrée toward that.

You mentioned moving upmarket customerwise. Who are your customers today?

We have offerings all the way from free … to what we call our business-critical tier. We have an enterprise product, and we have a team edition. On the high end, the typical story is we … work with a platform team. A lot of these enterprises are starting to create these platform teams. They basically are responsible for leading the charge for adopting the cloud within the organization. They sort of sit between the infrastructure and operations team and the developers. And their goal is “Hey, figure out how to adopt AWS, Azure, Kubernetes — make the choices on the platforms.”

That includes something like Pulumi. That includes CI/CD systems. They're trying to turn around and empower developers to be in control of their own destinies, but they want to do that with guardrails in place, so developers don't do the wrong thing accidentally. Then they turn around and give patterns and practices to the infrastructure team, so things are repeatable.

Our biggest customer – not one I can name unfortunately, but it's a super exciting partnership – is using us within over 100 teams within the company and close to 10,000 engineers on a daily basis. And really at that scale, what they're trying to do is tame the chaos of adopting the public cloud. Pulumi can scale to very large-scale complexity in a way that most other kinds of technologies can't, and so that is a very common story with larger companies like Snowflake, Univision and Mercedes-Benz … DocuSign — folks like that.

We have a strong contingency in the mid-market as well, which is folks who recently IPO-ed or are about to IPO and people for which the cloud is a competitive advantage in a fundamental way.

You mentioned in your blog that when you were exploring starting Pulumi in 2016, the capabilities of the cloud were incredible but far too difficult to use, which held true in 2018, when the company launched. Is that still the case?

I think we've definitely made a huge dent in making it easier. My customers were developers when I was at Microsoft, and they really weren't excited about the cloud. They didn't want to touch it. They would write their code like the cloud didn't exist, and they’d throw it over the wall to the infrastructure team, and it was very inefficient.

I thought, why is that? One reason is because I don't want to go learn some weird, proprietary language; I want to use what's comfortable to me. I want to use my favorite editor. I want to use all the practices that I know already. So we solved that problem. We've really brought the cloud a lot closer to developers and brought great software engineering tools to infrastructure teams, but there's so much more we can do. The shift to the cloud is really a shift to building distributed applications, and the most innovative companies out there are the ones that have figured this out and have figured out how to do it. What we want to do is bring that to everybody, and we've got a great foundation to start from, but so much more to go from here.

Are cloud providers improving on their side?

They are. It's in their best interest to make it as easy to use as possible. For many large enterprises, they're still doing multicloud, and I think that's one thing that really sets us apart even if the cloud providers continue to improve their offerings. We work with folks who still have on-prem, virtual-machine data centers, and Pulumi gives them a consistent model they can standardize on across the team, even for private cloud and hybrid cloud. That's really special for especially large enterprises.

"The shift to the cloud is really a shift to building distributed applications, and the most innovative companies out there are the ones that have figured this out and have figured out how to do it."

I understand Pulumi uses mostly AWS internally. You’re a Microsoft veteran. Why AWS?

We're customers of all the clouds, but most of our service runs on AWS. If you go to Pulumi.com, that is hosted in AWS. My CTO [Luke Hoban] came from Amazon. He was on the EC2 team prior to coming to Pulumi. Amazon's philosophy resonates with us, which is they give you these powerful building blocks, all these services. It's up to you to stitch them together. That's the hard part. But Microsoft — and Google to some extent — try to give you more of a platform-as-a-service-like approach, where they're more monolithic and chunkier building blocks. We found that, especially with the way Pulumi allows you to program the cloud and stitch things together and compose things, the AWS model really just works perfectly.

What’s next for Pulumi?

There are really three main areas of focus for us. One is just being the market leader for infrastructure as code. We know we're growing faster than Terraform … but we still know they have a lot more market share. Priority No. 1 is really just infrastructure as code and being the market leader.

The second is doing better with developers. We figured out over time, as we were trying to find the product-market fit that “Oh wow, there's some really gnarly issues for people that we can solve in the infrastructure management space.” But really we're still far away from empowering every developer to tap into the cloud — not just infrastructure experts, not just developers who want to do infrastructure as code, but all developers, really allowing them to really embrace these new kinds of distributed cloud architectures. There's a huge opportunity that's going to totally reshape the way that developers write code.

The third is really that cloud engineering platform vision … going from infrastructure as code to a complete product suite that solves for this entire space.

You mentioned the economic climate. How is that from a startup's point of view? Do you have enough funding right now, and is that drying up at all?

It's changed the dynamics a bit. We don't worry too much about it. I don't say that to sound arrogant or anything, but we've always focused on business fundamentals. We wanted to build for the long-term. Our dream is to IPO and build a sustainable business that can eventually become profitable. That's very important to us. But unfortunately, I think a number of folks out there were raising well beyond where they were and maybe not as focused on business fundamentals. We feel good about where we're at right now. We raised a Series B [funding round] led by NEA about a year and a half ago … and that was a preemptive round. We raised about a year before we thought we'd need to, so we're still in a very strong position cashwise.

Is there a time frame envisioned for an IPO?

It's hard to predict the future, but I would say in the three- to five-year ballpark would be sort of where we're at as a company. You look at other comparable businesses like HashiCorp, GitLab, Datadog, and I think we're right around where they were three to four, sometimes five years before the IPO. A lot of it depends on the market and how it evolves, but given current growth rates, the opportunity is huge, and we're still very early in our growth curve within the market.


Judge Zia Faruqui is trying to teach you crypto, one ‘SNL’ reference at a time

His decisions on major cryptocurrency cases have quoted "The Big Lebowski," "SNL," and "Dr. Strangelove." That’s because he wants you — yes, you — to read them.

The ways Zia Faruqui (right) has weighed on cases that have come before him can give lawyers clues as to what legal frameworks will pass muster.

Photo: Carolyn Van Houten/The Washington Post via Getty Images

“Cryptocurrency and related software analytics tools are ‘The wave of the future, Dude. One hundred percent electronic.’”

That’s not a quote from "The Big Lebowski" — at least, not directly. It’s a quote from a Washington, D.C., district court memorandum opinion on the role cryptocurrency analytics tools can play in government investigations. The author is Magistrate Judge Zia Faruqui.

Keep ReadingShow less
Veronica Irwin

Veronica Irwin (@vronirwin) is a San Francisco-based reporter at Protocol covering fintech. Previously she was at the San Francisco Examiner, covering tech from a hyper-local angle. Before that, her byline was featured in SF Weekly, The Nation, Techworker, Ms. Magazine and The Frisc.

The financial technology transformation is driving competition, creating consumer choice, and shaping the future of finance. Hear from seven fintech leaders who are reshaping the future of finance, and join the inaugural Financial Technology Association Fintech Summit to learn more.

Keep ReadingShow less
The Financial Technology Association (FTA) represents industry leaders shaping the future of finance. We champion the power of technology-centered financial services and advocate for the modernization of financial regulation to support inclusion and responsible innovation.

AWS CEO: The cloud isn’t just about technology

As AWS preps for its annual re:Invent conference, Adam Selipsky talks product strategy, support for hybrid environments, and the value of the cloud in uncertain economic times.

Photo: Noah Berger/Getty Images for Amazon Web Services

AWS is gearing up for re:Invent, its annual cloud computing conference where announcements this year are expected to focus on its end-to-end data strategy and delivering new industry-specific services.

It will be the second re:Invent with CEO Adam Selipsky as leader of the industry’s largest cloud provider after his return last year to AWS from data visualization company Tableau Software.

Keep ReadingShow less
Donna Goodison

Donna Goodison (@dgoodison) is Protocol's senior reporter focusing on enterprise infrastructure technology, from the 'Big 3' cloud computing providers to data centers. She previously covered the public cloud at CRN after 15 years as a business reporter for the Boston Herald. Based in Massachusetts, she also has worked as a Boston Globe freelancer, business reporter at the Boston Business Journal and real estate reporter at Banker & Tradesman after toiling at weekly newspapers.

Image: Protocol

We launched Protocol in February 2020 to cover the evolving power center of tech. It is with deep sadness that just under three years later, we are winding down the publication.

As of today, we will not publish any more stories. All of our newsletters, apart from our flagship, Source Code, will no longer be sent. Source Code will be published and sent for the next few weeks, but it will also close down in December.

Keep ReadingShow less
Bennett Richardson

Bennett Richardson ( @bennettrich) is the president of Protocol. Prior to joining Protocol in 2019, Bennett was executive director of global strategic partnerships at POLITICO, where he led strategic growth efforts including POLITICO's European expansion in Brussels and POLITICO's creative agency POLITICO Focus during his six years with the company. Prior to POLITICO, Bennett was co-founder and CMO of Hinge, the mobile dating company recently acquired by Match Group. Bennett began his career in digital and social brand marketing working with major brands across tech, energy, and health care at leading marketing and communications agencies including Edelman and GMMB. Bennett is originally from Portland, Maine, and received his bachelor's degree from Colgate University.


Why large enterprises struggle to find suitable platforms for MLops

As companies expand their use of AI beyond running just a few machine learning models, and as larger enterprises go from deploying hundreds of models to thousands and even millions of models, ML practitioners say that they have yet to find what they need from prepackaged MLops systems.

As companies expand their use of AI beyond running just a few machine learning models, ML practitioners say that they have yet to find what they need from prepackaged MLops systems.

Photo: artpartner-images via Getty Images

On any given day, Lily AI runs hundreds of machine learning models using computer vision and natural language processing that are customized for its retail and ecommerce clients to make website product recommendations, forecast demand, and plan merchandising. But this spring when the company was in the market for a machine learning operations platform to manage its expanding model roster, it wasn’t easy to find a suitable off-the-shelf system that could handle such a large number of models in deployment while also meeting other criteria.

Some MLops platforms are not well-suited for maintaining even more than 10 machine learning models when it comes to keeping track of data, navigating their user interfaces, or reporting capabilities, Matthew Nokleby, machine learning manager for Lily AI’s product intelligence team, told Protocol earlier this year. “The duct tape starts to show,” he said.

Keep ReadingShow less
Kate Kaye

Kate Kaye is an award-winning multimedia reporter digging deep and telling print, digital and audio stories. She covers AI and data for Protocol. Her reporting on AI and tech ethics issues has been published in OneZero, Fast Company, MIT Technology Review, CityLab, Ad Age and Digiday and heard on NPR. Kate is the creator of RedTailMedia.org and is the author of "Campaign '08: A Turning Point for Digital Media," a book about how the 2008 presidential campaigns used digital media and data.

Latest Stories