Over the last six years, Target CIO Mike McNamara has guided one of the largest retail operations in the U.S. through a modernization plan that prized independence over convenience.
Target was once all-in on Amazon, hiring it to run its entire ecommerce operation in 2001. That alliance wound down as the cloud heated up, and Target used multiple providers alongside home-built data centers for several years. Then Amazon bought Whole Foods, putting it in direct competition with Target's grocery business. That conflict pushed Target off the leading cloud infrastructure player for good.
Now, Target's operations run on an application layer built around the vanilla cloud-computing services offered by Google Cloud and Microsoft Azure, as well as its own data centers. Composed of several different popular open-source projects like Kubernetes, Docker and Spinnaker, this architecture allows McNamara to run ecommerce and in-store workloads on three different platforms as needed.
In a recent interview with Protocol, McNamara outlined Target's approach to modern infrastructure as well as the importance of open-source software to enterprise computing.
This interview has been edited and condensed for clarity.
What does your current setup look like?
I came here in 2015. And when I came here, I was in a reasonably traditional corporate IT department. We made the decision that technology was important, that we were going to build ourselves all of the things that really matter to a retailer.
I wasn't interested in building a new financial ledger or anything, but all of those things that actually make a retailer different from other retailers. So anything that touches the guest experience — we call our customers our guests — and anything that touches our store team members, because we've got a lot of them, we were going to build those things ourselves, as well as build a lot of the technology that supports our supply chain.
That was a big decision to make: bring all engineering back in-house, and start the journey on building out a new application architecture and a new set of applications to run the business.
We did adopt an enterprise microservices strategy. So we started to decompose our own applications into microservices. We built an API layer between all the legacy stuff and the new stuff that we're building.
Part of that whole journey was around infrastructure as well. The way I describe it is to abstract the infrastructure away from application developers. So instead of an application developer worrying about how many cores they need, how much memory they need, how much heap space they need, anything like that, I wanted them to focus on writing features and functions. Because at the end of the day, it's features and functions that actually make money for Target, either by improving our sales or by improving the productivity of our operations.
We set about creating an infrastructure so that was abstracted away from application engineering. We relied very, very heavily on open-source software to create the whole CI/CD pipeline, which included all the things you would expect: GitHub, Artifactory, Jira, Docker. We actually have run Kubernetes, the open-source version of Kubernetes, everywhere. And then we adapted Spinnaker to give us a single pane of glass through which our application teams could manage all of their applications, manage the whole CI/CD pipeline, and then crucially, be able to manage the deployment pipeline.
Unlike many organizations, a lot of our technology is actually deployed in our stores. We've got about 1,900 stores plus 40 distribution centers. So being able to automate the deployment down to the stores, really, really mattered to us.
At the same time, we moved a lot — certainly all of our digital assets and websites and software — to the cloud. We use Google predominantly for that, and we are multicloud; we do use [Microsoft] Azure as well. The majority of what you would experience as a guest through our website or through the app actually resides for most of the time in Google's cloud, although we do deploy the same binaries in our own data centers as well.
Target is run by a single commerce platform. There is no difference between the technology that sits underneath our website and sits underneath our stores. If you scan a box of cereal in our stores on a cash register, you're hitting the same exact binaries as if you had clicked on it to put it into your online basket.
We achieved that through this application platform that we built ourselves. That gives us huge flexibility in terms of cloud. So I can take a workload and I can choose where I want to deploy it. I can choose to deploy it in [Google Cloud Platform]. I can choose to deploy it on Azure, or as I say, I can choose to deploy it on Target assets.
Just to confirm, are you completely off of AWS at this point?
Yes, we are.
How much of that was a result of the fact that Amazon is Amazon?
All of it was a result of Amazon.
I talk to a lot of people who express interest in doing multicloud and hybrid as well, and then in practice, it tends to not work out so well. Can you explain how that went for Target, in terms of migrating off of one cloud onto another and maintaining your own assets as well?
We've created this thing — TAP, or Target Application Platform — and that is our deployment toolset.
We are using both Google and Microsoft [just] as infrastructure as a service. And I think that that's important, because that means that we can maintain that flexibility of placing workloads where we want.
If you actually use the toolsets provided by the provider — you know, their proprietary databases, their proprietary cluster management — then you do get locked in and it becomes much more difficult to move from cloud to cloud. For some companies, that's a reasonable strategy to pursue. We just chose to pursue a more independent strategy.
A lot of the people I've talked to understand the lock-in problem, but prefer the managed service just for ease of use. [Your strategy] requires you to build all that expertise in house. How long did that take? And how did that go?
This is a scale thing. At Target we've got over 4,000 engineers. We are a scale operation. If I was a much, much smaller operation, there's no way I could build all of these tools.
Are you finding it easier to hire those folks? For a long time, it seemed like a lot of that great engineering capability was sort of constrained within the tech industry itself, in that it was harder to find those folks to work on retail or insurance or consumer goods or whatever. Has that changed a lot or are you just finding it easier because these skills are a little bit more widespread?
We have hired a lot from the tech industry. We've got people on this team who came from, you know, name that great West Coast company, right?
What's attractive about retail is that the problems you get to solve are fun, and they're interesting. And they're immediate; you're not a million miles removed from the guests. It's very, very easy to understand the connection between the work you do and what happens when one of our guests walks into our stores.
Now we're reasonably well known for our technology prowess and for the kind of work that you get to do if you come here. I remember back in the day when I used to do programming, that was the thing that got me the most excited, the quality of the work that I had to do. Once you have great quality work for people to do, then recruitment becomes less of an issue.
I wanted to circle back to the data center question. A lot of people who have been moving to the cloud are doing it as a way to get rid of data centers, but it sounds like you still want to keep that flexibility for yourself, at least to some extent. Can you give me a sense of what you use data centers for? And how you think about that going forward?
If we didn't have them, I don't know if I ever would have built them, but we have them. And they are a sunk cost. They are a sophisticated asset that costs a lot of money, so given that we have them, we're going to use them.
The reality is, when you look at one of the great advantages of cloud to us as a retailer is the elasticity of the infrastructure. Most of the year long, our workload is reasonably predictable. And we can run that very, very efficiently within our own data centers.
But it is very seasonal, particularly when you get into the period around Thanksgiving and Cyber Monday. Typically you can be looking at workloads 20 times the normal volume. That elasticity in the cloud is an enormously useful tool for us to manage those periods.
And they're not the only peaks within the retail year. So the infrastructure elasticity is really, really valuable for us. But we have our own existing assets and want to continue to use them because we can run them very, very efficiently.
Is there anything that you're looking at pretty closely that you think will be interesting for you and your needs?
I've always been excited by open source, I think it is probably the most underrated technology advancement in my lifetime.
I won't say how old I am, but you go back through the course of my life, there's the 8086 chip from Intel, then there was the PC, and give [Bill] Gates his due with the creation of DOS. Then there was the internet, and then probably the iPhone, but then in the last decade, it's been open source.
It's been engineers creating technology for engineers. The speed of delivery today in comparison to what I faced throughout the most of my career is extraordinary. I've been in retail a long, long time. And for most of that time, you would plan to upgrade the software on your cash registers four times a year. We always failed; you usually made it about three times, by the fourth time you're getting too close to the holiday season so you call that one off.
Now we upgrade that software on a weekly basis. It's amazing the speed with which we can produce and deploy software. And that is because we have the tools to do that. And those tools are generally known open-source tools.
So I think it is very much underrated. And the more people who can embrace open source, the larger the community gets it, the more contributors, I think the better you do.
It's a very interesting time for open source, with respect to some of the vendors and the way that they have been thinking about licensing and monetizing these open source projects. As a user, and maybe as a customer of some of these things, how do you think about that evolution?
Well, I think people have to make money, right? So I'm OK with, you know, with Docker making money out of their invention. I actually quite liked the way they've gone about it, in that mostly they are still taking a lot of the ideas in from the open-source community to incorporate into their supported products.
Target is in a very fortunate position that we have the engineering capability to support ourselves. If I were in a much smaller organization, I might choose to purchase the enterprise support because I don't have that engineering capability. So actually, it's not a bad compromise. These companies get to make money out of their inventions, which I think is fair. But also they keep alive the open-source versions, and the open-source community can continue to contribute to them, and use them and use those free versions.
