Target CIO Mike McNamara makes a cloud declaration of independence

Staying on Amazon Web Services wasn't an option, so the retailer built a multicloud architecture that gives it flexibility across vendors.

​Target CIO Mike McNamara speaks at the Google Cloud Next conference in 2018.

Target CIO Mike McNamara speaks at the Google Cloud Next conference in 2018.

Photo: David Paul Morris/Getty Images

Over the last six years, Target CIO Mike McNamara has guided one of the largest retail operations in the U.S. through a modernization plan that prized independence over convenience.

Target was once all-in on Amazon, hiring it to run its entire ecommerce operation in 2001. That alliance wound down as the cloud heated up, and Target used multiple providers alongside home-built data centers for several years. Then Amazon bought Whole Foods, putting it in direct competition with Target's grocery business. That conflict pushed Target off the leading cloud infrastructure player for good.

Now, Target's operations run on an application layer built around the vanilla cloud-computing services offered by Google Cloud and Microsoft Azure, as well as its own data centers. Composed of several different popular open-source projects like Kubernetes, Docker and Spinnaker, this architecture allows McNamara to run ecommerce and in-store workloads on three different platforms as needed.

In a recent interview with Protocol, McNamara outlined Target's approach to modern infrastructure as well as the importance of open-source software to enterprise computing.

This interview has been edited and condensed for clarity.

What does your current setup look like?

I came here in 2015. And when I came here, I was in a reasonably traditional corporate IT department. We made the decision that technology was important, that we were going to build ourselves all of the things that really matter to a retailer.

I wasn't interested in building a new financial ledger or anything, but all of those things that actually make a retailer different from other retailers. So anything that touches the guest experience — we call our customers our guests — and anything that touches our store team members, because we've got a lot of them, we were going to build those things ourselves, as well as build a lot of the technology that supports our supply chain.

That was a big decision to make: bring all engineering back in-house, and start the journey on building out a new application architecture and a new set of applications to run the business.

We did adopt an enterprise microservices strategy. So we started to decompose our own applications into microservices. We built an API layer between all the legacy stuff and the new stuff that we're building.

Part of that whole journey was around infrastructure as well. The way I describe it is to abstract the infrastructure away from application developers. So instead of an application developer worrying about how many cores they need, how much memory they need, how much heap space they need, anything like that, I wanted them to focus on writing features and functions. Because at the end of the day, it's features and functions that actually make money for Target, either by improving our sales or by improving the productivity of our operations.

We set about creating an infrastructure so that was abstracted away from application engineering. We relied very, very heavily on open-source software to create the whole CI/CD pipeline, which included all the things you would expect: GitHub, Artifactory, Jira, Docker. We actually have run Kubernetes, the open-source version of Kubernetes, everywhere. And then we adapted Spinnaker to give us a single pane of glass through which our application teams could manage all of their applications, manage the whole CI/CD pipeline, and then crucially, be able to manage the deployment pipeline.

Unlike many organizations, a lot of our technology is actually deployed in our stores. We've got about 1,900 stores plus 40 distribution centers. So being able to automate the deployment down to the stores, really, really mattered to us.

At the same time, we moved a lot — certainly all of our digital assets and websites and software — to the cloud. We use Google predominantly for that, and we are multicloud; we do use [Microsoft] Azure as well. The majority of what you would experience as a guest through our website or through the app actually resides for most of the time in Google's cloud, although we do deploy the same binaries in our own data centers as well.

Target is run by a single commerce platform. There is no difference between the technology that sits underneath our website and sits underneath our stores. If you scan a box of cereal in our stores on a cash register, you're hitting the same exact binaries as if you had clicked on it to put it into your online basket.

We achieved that through this application platform that we built ourselves. That gives us huge flexibility in terms of cloud. So I can take a workload and I can choose where I want to deploy it. I can choose to deploy it in [Google Cloud Platform]. I can choose to deploy it on Azure, or as I say, I can choose to deploy it on Target assets.

Just to confirm, are you completely off of AWS at this point?

Yes, we are.

How much of that was a result of the fact that Amazon is Amazon?

All of it was a result of Amazon.

I talk to a lot of people who express interest in doing multicloud and hybrid as well, and then in practice, it tends to not work out so well. Can you explain how that went for Target, in terms of migrating off of one cloud onto another and maintaining your own assets as well?

We've created this thing — TAP, or Target Application Platform — and that is our deployment toolset.

We are using both Google and Microsoft [just] as infrastructure as a service. And I think that that's important, because that means that we can maintain that flexibility of placing workloads where we want.

If you actually use the toolsets provided by the provider — you know, their proprietary databases, their proprietary cluster management — then you do get locked in and it becomes much more difficult to move from cloud to cloud. For some companies, that's a reasonable strategy to pursue. We just chose to pursue a more independent strategy.

A lot of the people I've talked to understand the lock-in problem, but prefer the managed service just for ease of use. [Your strategy] requires you to build all that expertise in house. How long did that take? And how did that go?

This is a scale thing. At Target we've got over 4,000 engineers. We are a scale operation. If I was a much, much smaller operation, there's no way I could build all of these tools.

Are you finding it easier to hire those folks? For a long time, it seemed like a lot of that great engineering capability was sort of constrained within the tech industry itself, in that it was harder to find those folks to work on retail or insurance or consumer goods or whatever. Has that changed a lot or are you just finding it easier because these skills are a little bit more widespread?

We have hired a lot from the tech industry. We've got people on this team who came from, you know, name that great West Coast company, right?

What's attractive about retail is that the problems you get to solve are fun, and they're interesting. And they're immediate; you're not a million miles removed from the guests. It's very, very easy to understand the connection between the work you do and what happens when one of our guests walks into our stores.

Now we're reasonably well known for our technology prowess and for the kind of work that you get to do if you come here. I remember back in the day when I used to do programming, that was the thing that got me the most excited, the quality of the work that I had to do. Once you have great quality work for people to do, then recruitment becomes less of an issue.

I wanted to circle back to the data center question. A lot of people who have been moving to the cloud are doing it as a way to get rid of data centers, but it sounds like you still want to keep that flexibility for yourself, at least to some extent. Can you give me a sense of what you use data centers for? And how you think about that going forward?

If we didn't have them, I don't know if I ever would have built them, but we have them. And they are a sunk cost. They are a sophisticated asset that costs a lot of money, so given that we have them, we're going to use them.

The reality is, when you look at one of the great advantages of cloud to us as a retailer is the elasticity of the infrastructure. Most of the year long, our workload is reasonably predictable. And we can run that very, very efficiently within our own data centers.

But it is very seasonal, particularly when you get into the period around Thanksgiving and Cyber Monday. Typically you can be looking at workloads 20 times the normal volume. That elasticity in the cloud is an enormously useful tool for us to manage those periods.

And they're not the only peaks within the retail year. So the infrastructure elasticity is really, really valuable for us. But we have our own existing assets and want to continue to use them because we can run them very, very efficiently.

Is there anything that you're looking at pretty closely that you think will be interesting for you and your needs?

I've always been excited by open source, I think it is probably the most underrated technology advancement in my lifetime.

I won't say how old I am, but you go back through the course of my life, there's the 8086 chip from Intel, then there was the PC, and give [Bill] Gates his due with the creation of DOS. Then there was the internet, and then probably the iPhone, but then in the last decade, it's been open source.

It's been engineers creating technology for engineers. The speed of delivery today in comparison to what I faced throughout the most of my career is extraordinary. I've been in retail a long, long time. And for most of that time, you would plan to upgrade the software on your cash registers four times a year. We always failed; you usually made it about three times, by the fourth time you're getting too close to the holiday season so you call that one off.

Now we upgrade that software on a weekly basis. It's amazing the speed with which we can produce and deploy software. And that is because we have the tools to do that. And those tools are generally known open-source tools.

So I think it is very much underrated. And the more people who can embrace open source, the larger the community gets it, the more contributors, I think the better you do.

It's a very interesting time for open source, with respect to some of the vendors and the way that they have been thinking about licensing and monetizing these open source projects. As a user, and maybe as a customer of some of these things, how do you think about that evolution?

Well, I think people have to make money, right? So I'm OK with, you know, with Docker making money out of their invention. I actually quite liked the way they've gone about it, in that mostly they are still taking a lot of the ideas in from the open-source community to incorporate into their supported products.

Target is in a very fortunate position that we have the engineering capability to support ourselves. If I were in a much smaller organization, I might choose to purchase the enterprise support because I don't have that engineering capability. So actually, it's not a bad compromise. These companies get to make money out of their inventions, which I think is fair. But also they keep alive the open-source versions, and the open-source community can continue to contribute to them, and use them and use those free versions.


Judge Zia Faruqui is trying to teach you crypto, one ‘SNL’ reference at a time

His decisions on major cryptocurrency cases have quoted "The Big Lebowski," "SNL," and "Dr. Strangelove." That’s because he wants you — yes, you — to read them.

The ways Zia Faruqui (right) has weighed on cases that have come before him can give lawyers clues as to what legal frameworks will pass muster.

Photo: Carolyn Van Houten/The Washington Post via Getty Images

“Cryptocurrency and related software analytics tools are ‘The wave of the future, Dude. One hundred percent electronic.’”

That’s not a quote from "The Big Lebowski" — at least, not directly. It’s a quote from a Washington, D.C., district court memorandum opinion on the role cryptocurrency analytics tools can play in government investigations. The author is Magistrate Judge Zia Faruqui.

Keep ReadingShow less
Veronica Irwin

Veronica Irwin (@vronirwin) is a San Francisco-based reporter at Protocol covering fintech. Previously she was at the San Francisco Examiner, covering tech from a hyper-local angle. Before that, her byline was featured in SF Weekly, The Nation, Techworker, Ms. Magazine and The Frisc.

The financial technology transformation is driving competition, creating consumer choice, and shaping the future of finance. Hear from seven fintech leaders who are reshaping the future of finance, and join the inaugural Financial Technology Association Fintech Summit to learn more.

Keep ReadingShow less
The Financial Technology Association (FTA) represents industry leaders shaping the future of finance. We champion the power of technology-centered financial services and advocate for the modernization of financial regulation to support inclusion and responsible innovation.

AWS CEO: The cloud isn’t just about technology

As AWS preps for its annual re:Invent conference, Adam Selipsky talks product strategy, support for hybrid environments, and the value of the cloud in uncertain economic times.

Photo: Noah Berger/Getty Images for Amazon Web Services

AWS is gearing up for re:Invent, its annual cloud computing conference where announcements this year are expected to focus on its end-to-end data strategy and delivering new industry-specific services.

It will be the second re:Invent with CEO Adam Selipsky as leader of the industry’s largest cloud provider after his return last year to AWS from data visualization company Tableau Software.

Keep ReadingShow less
Donna Goodison

Donna Goodison (@dgoodison) is Protocol's senior reporter focusing on enterprise infrastructure technology, from the 'Big 3' cloud computing providers to data centers. She previously covered the public cloud at CRN after 15 years as a business reporter for the Boston Herald. Based in Massachusetts, she also has worked as a Boston Globe freelancer, business reporter at the Boston Business Journal and real estate reporter at Banker & Tradesman after toiling at weekly newspapers.

Image: Protocol

We launched Protocol in February 2020 to cover the evolving power center of tech. It is with deep sadness that just under three years later, we are winding down the publication.

As of today, we will not publish any more stories. All of our newsletters, apart from our flagship, Source Code, will no longer be sent. Source Code will be published and sent for the next few weeks, but it will also close down in December.

Keep ReadingShow less
Bennett Richardson

Bennett Richardson ( @bennettrich) is the president of Protocol. Prior to joining Protocol in 2019, Bennett was executive director of global strategic partnerships at POLITICO, where he led strategic growth efforts including POLITICO's European expansion in Brussels and POLITICO's creative agency POLITICO Focus during his six years with the company. Prior to POLITICO, Bennett was co-founder and CMO of Hinge, the mobile dating company recently acquired by Match Group. Bennett began his career in digital and social brand marketing working with major brands across tech, energy, and health care at leading marketing and communications agencies including Edelman and GMMB. Bennett is originally from Portland, Maine, and received his bachelor's degree from Colgate University.


Why large enterprises struggle to find suitable platforms for MLops

As companies expand their use of AI beyond running just a few machine learning models, and as larger enterprises go from deploying hundreds of models to thousands and even millions of models, ML practitioners say that they have yet to find what they need from prepackaged MLops systems.

As companies expand their use of AI beyond running just a few machine learning models, ML practitioners say that they have yet to find what they need from prepackaged MLops systems.

Photo: artpartner-images via Getty Images

On any given day, Lily AI runs hundreds of machine learning models using computer vision and natural language processing that are customized for its retail and ecommerce clients to make website product recommendations, forecast demand, and plan merchandising. But this spring when the company was in the market for a machine learning operations platform to manage its expanding model roster, it wasn’t easy to find a suitable off-the-shelf system that could handle such a large number of models in deployment while also meeting other criteria.

Some MLops platforms are not well-suited for maintaining even more than 10 machine learning models when it comes to keeping track of data, navigating their user interfaces, or reporting capabilities, Matthew Nokleby, machine learning manager for Lily AI’s product intelligence team, told Protocol earlier this year. “The duct tape starts to show,” he said.

Keep ReadingShow less
Kate Kaye

Kate Kaye is an award-winning multimedia reporter digging deep and telling print, digital and audio stories. She covers AI and data for Protocol. Her reporting on AI and tech ethics issues has been published in OneZero, Fast Company, MIT Technology Review, CityLab, Ad Age and Digiday and heard on NPR. Kate is the creator of RedTailMedia.org and is the author of "Campaign '08: A Turning Point for Digital Media," a book about how the 2008 presidential campaigns used digital media and data.

Latest Stories