He complained about Democrats’ tech ‘morass’ — then built the Iowa app

The CEO of the tech startup at the center of the Iowa caucus debacle complained last year that critical parts of the Democrats' election technology were a "shitshow" and a "tangled morass." He said he wanted to fix things.

"These systems, they're not designed super intentionally from day one, which frankly, nothing in the political tech ecosystem has been," Gerard Niemira, now the CEO of Shadow Inc., told me last March. "You just accrue layers of plaque over time."

Niemira was talking specifically about Vertica, a legacy system the Democrats used to house their data. I was writing a story about it for Wired when I called him to talk. At the time, he was working for Acronym, an organization that works on digital strategy for Democrats. Most of our interview wound up on the cutting-room floor, but in one choice quote that made it in, Niemira said Vertica — which he'd encountered working as the director of product for Hillary Clinton's 2016 presidential run — had been "a shitshow from the moment I started there." He then eagerly told me how the tools he was working on at the time would help modernize campaigns and Democratic organizations.

Get what matters in tech, in your inbox every morning. Sign up for Source Code.

What he couldn't have known then was that, nearly a year later, one of the tools he created would be mired in a morass of its own, when his app caused a delay in the caucus results. But what he should have known is that a hasty rollout of new technology can be just as damaging as relying on the antiquated systems he was trying to improve.

In a statement Tuesday, Iowa Democratic Party Chair Troy Price chalked the caucus night chaos up to a "coding issue." "While the app was recording data accurately, it was reporting out only partial data," the statement read. Later in the day, Shadow confirmed as much on Twitter.

But that's hardly the whole story. While it's easy to point the finger at a single app developer or a single glitch, election security experts and campaign tech veterans say the meltdown in Iowa represents a systemic failure within the Democratic Party, and a study in how not to roll out technology in future elections.

"Bugs happen," says Michael Slaby, former chief innovation officer for President Barack Obama's campaign and currently a chief strategist for the startup Harmony Labs. "It's not that easy to build something new that works perfectly in this scenario. So why are we building something new a couple months before the primary?"

It's a classic mistake. They threw untested software into a real-world situation, and it blew up in their face. — Josh Greenbaum

"It's very clear from what went on in Iowa that the Iowa Democratic Party failed on all accounts," says Josh Greenbaum, chief technology officer for the U.S. Vote Foundation. "They didn't understand their technology, and they didn't understand how to educate their users. It's a classic mistake. They threw untested software into a real-world situation, and it blew up in their face."

On Monday night and throughout the day Tuesday, stories poured forth from school gymnasiums across the state, where precinct captains were having trouble logging into the app or entering their security codes to download and access it. Motherboard reported that Shadow was pushing a new version of the app to precinct leaders just two days before the caucus.

"They didn't train people to use this thing properly. They didn't plan for it not to work," one former Democratic tech operative told Protocol of the Iowa Democratic Party. "You can't simply be like, 'We're going to deploy an app and it's going to fix everything.' There's training, design, testing. This is clearly a pendulum that has swung too far."

One irony of all of this is that, when we spoke last year, Niemira was keenly aware of how crucial it is for the party to build technology that non-technical people can use.

"Our data ecosystem is so crazy and overcomplicated and requires so much expertise to run even the simplest parts of these tools," he said then. Yet, he explained, campaigns, committees and state parties lack the resources to hire many top-shelf engineers, who could easily make more money working for Facebook or Google. "Engineer hires are expensive. Even engineers who are ready to take a passion project are usually shocked by the sticker price difference," Niemira said.

Besides, he added, "Our world is so specific and complicated that it took me four months of being on the [Clinton] campaign to finally understand exactly what was going on."

It was that experience that drove him to work on political tech. One of Shadow's core products is a tool called Lightrail. Niemira began developing it while he was working at Acronym. (On Monday night, Acronym rushed to distance itself from the mess.) As Niemira described it, Lightrail is effectively a technical pipeline to connect disparate Democratic technology tools, like, say, the fundraising platform ActBlue and the voter contact file. Lightrail would enable campaigns to merge datasets without a ton of manual labor.

At the time, he used South Bend Indiana Mayor Pete Buttigieg's still-small presidential campaign as a hypothetical example to explain how Lightrail worked. "A guy like Mayor Pete, the best thing he can do to recruit volunteers is reach out to people who have donated to him," Niemira said. "The path to doing that today is like a 15-step process with two different CSV downloads."

Federal Election Commission reports show that the Buttigieg campaign, as well as the campaigns of Vice President Joe Biden and Sen. Kirsten Gillibrand, all hired Shadow at various points. The Buttigieg campaign told CNN it only used Shadow's text messaging services.

Just how Shadow became a vendor for the Iowa caucuses, selling an app that is not among its core services, is still unclear. Niemira didn't respond to Protocol's request for comment, and the Iowa Democratic Party didn't respond to a question about how and when Shadow was awarded the contract.

"I'm really disappointed that some of our technology created an issue that made the caucus difficult," Niemira told Bloomberg late Tuesday night. "We exist to help campaigns. It really pains me that we did the opposite."

He confirmed that a bug in the app's code caused the problems. Earlier Tuesday, Shadow Inc.'s Twitter account posted a statement saying it had corrected the underlying issue and would "apply the lessons learned in the future."

But Democratic technologists say the lessons need to be absorbed not just by the company, but by all election operators and political parties. Even the best tech requires a careful rollout. And when introducing new systems, redundancies need to be built in to keep the system resilient to failure. In the case of Iowa, reports have suggested that when people began having trouble with the app, the state party's phone lines were overwhelmed.

"We believe the failure of the Iowa caucus reporting system underscores the importance of not only early investments in technology, but also — indeed, most importantly — undertaking rigorous testing and thorough training for dependable results," read a statement from Betsy Hoover and Shomik Dutta, co-founders of the Democratic investing firm Higher Ground Labs, which invested in one of Niemira's earlier companies. Dutta and Hoover called Niemira a "talented and capable technologist."

"Campaign strategists must commit to technology early," they wrote. "New apps should not be built in the final stretch, and volunteers should not be asked to make a download on Caucus Day."

Nevada's Democratic Party, which spent $58,000 on Shadow's services according to FEC records, scrapped plans to use the app for its own caucus later this month.

The good news is that the Iowa caucuses are still relatively lo-fi, with votes being tallied manually and publicly and recorded in each precinct on pen and paper. "This is far from the worst that could have happened," tweeted Ed Felten, a Princeton University computer science professor and former deputy chief technology officer of the United States under President Obama. "Results will be tabulated correctly, if a bit more slowly than news junkies preferred. The key to securing elections is resilience."

By Tuesday afternoon, the Iowa Democratic Party released preliminary results for 62% of precincts, showing Buttigieg and Bernie Sanders leading. Tabulating these results by hand is a tedious process, but at least it's relatively reliable. Recovering from this kind of glitch would be far more fraught in counties across the country that use voting technology with no paper trail. According to Verified Voting, an advocacy group that monitors voting systems, that includes counties in Texas, Louisiana, Kansas, Kentucky, Mississippi, Tennessee, Indiana and New Jersey.

"There needs to be a low-tech solution in order to recover from technological failures — no matter the cause," wrote Marian K. Schneider, president of Verified Voting, in a statement. "It's clear that mobile apps are not ready for prime time, but thankfully Iowa has paper records of their vote totals and will be able to release results from those records."

Charles Levinson contributed reporting.