Policy

They left Facebook’s integrity team. Now they want the world to know how it works.

Without breaking their NDAs.

A black and white image of Lady Jjustice

The Integrity Institute's goal is to build a network of integrity professionals and work toward a public consensus about the nitty gritty scientific and philosophical questions that integrity teams have mostly tried to answer behind closed doors.

Photo: Oleksandr Berezko / EyeEm / Getty Images

Shortly before he left Facebook in October 2019, Jeff Allen published his last report as a data scientist for the company's integrity team — the team Facebook Papers whistleblower Frances Haugen has recently made famous.

The report revealed, as Allen put it at the time, some "genuinely horrifying" findings. Namely, three years after the 2016 election, troll farms in Kosovo and Macedonia were continuing to operate vast networks of Facebook pages filled with mostly plagiarized content targeting Black Americans and Christian Americans on Facebook. Combined, the troll farms' pages reached 140 million Facebook users a month, dwarfing the reach of even Walmart's Facebook presence.

Not only had Facebook failed to stop its spread, Allen wrote in a 20-page report that was recently leaked to and published by MIT Tech Review, but the vast majority of the network's reach came from Facebook's ranking algorithms.

"I have no problem with Macedonians reaching US audiences," Allen wrote. "But if you just want to write python scripts that scrape social media and anonymously regurgitate content into communities while siphoning off some monetary or influence reward for yourself… well you can fuck right off."

A Facebook spokesperson said the company has taken "aggressive enforcement actions against these kinds of foreign and domestic inauthentic groups" and was doing so even when Allen wrote his report.

Allen left Facebook soon after that for a data scientist job at the Democratic National Committee. He ended his report with a goodbye, saying he was sad about leaving the company because he wouldn't be able "to work on this problem with the level of impact that FB provides."

Two years later, Allen is hoping to prove himself wrong about that last part.

For the last 10 months, he and Sahar Massachi, another former Facebook staffer who helped build the company's election and civic integrity teams from their earliest days, have been quietly creating an independent organization they hope could help not just Facebook, but also the entire tech industry and lawmakers alike navigate some of the trickiest questions facing integrity workers today.

Now, Allen and Massachi, who spoke exclusively with Protocol, are publicly launching the group at a time when those questions are at the white hot core of the international scandal, spurred by Haugen's disclosures.

Called the Integrity Institute, its goal is to build a network of integrity professionals who are currently employed or were previously employed by tech companies and work toward some kind of public consensus about the nitty gritty scientific and philosophical questions that integrity teams have mostly tried to answer behind closed doors.

The group plans to advise policy makers, regulators and the media about how social media platforms work, publish its own research and serve as a sort of open-source resource for small platforms that lack massive integrity teams of their own. Already, Allen and Massachi have briefed Congress and recruited 11 members, who have collectively worked for nine different tech platforms, to the organization. While they don't define themselves as whistleblowers and are "resolutely not breaking our NDAs," Massachi said, they say Haugen's disclosures have sparked a growing interest in the emerging field of integrity work — interest they're eager to capitalize on.

"Frances is exposing a lot of the knobs in the machine that is a modern social media platform," Massachi said. "Now that people know that those knobs exist, we can start having a conversation about what is the science of how these work, what these knobs do and why you would want to turn them in which direction."

Another benefit of the Facebook Papers? "My mom finally kind of understands the kind of work I was doing," Massachi said.

'Emerging science'

Integrity work only really got its name a few years ago when companies like Facebook began assigning teams of researchers to study all the ways their platforms could be misused. Massachi joined Facebook in 2016 and was one of the earliest members of its election integrity team, working on a dashboard to monitor the Alabama special election in 2017 and, later, on Facebook's election war room. He describes his time working on that team and Facebook's civic integrity team as a career highlight, but says the job of an integrity worker was also full of frustration.

"I thought that integrity was heroic, virtuous work that was not getting enough power and attention inside the company," he said. The people whose job it was to make the platform more moral and less messy had to "jump through a lot more hoops than normal product work, and that felt really wrong to me," Massachi said, though he's careful not to bash his former employer as others have. "Our membership has a wide variety of views," he said.

In January, Massachi began reaching out to former Facebook integrity workers, including Allen, to see if they were interested in forming something of a super group outside of the company. He sent potential members a memo laying out the problems on social media platforms, as he sees them, and the potential fixes he would propose.

"Then people would say, 'Well, I agree with 80% of this. I really disagree with the other 20%.' Then I would say, 'Great. The world deserves to hear our disagreements. Let's hash it out in public," Massachi, who also recently completed a fellowship at Harvard's Berkman Klein Center, said. "That was a recruiting tactic."

For Allen, now chief research officer of the institute, the concept of an independent integrity body was something he'd been considering himself for years. "I think this is an idea that crosses a lot of integrity workers' minds," he said. "What does it look like to do this outside the company?"

Jeff Allen and Sahar Massachi Jeff Allen (left) and Sahar Massachi (right)Photos: Courtesy Jeff Allen and Sahar Massachi

In July, Massachi and Allen gathered the group's 11 members for a retreat in Washington, D.C. They bunked up together at Airbnbs and spent their days in a coworking space, trying to come to a consensus on questions only a geek could love, but which matter deeply to billions of people's experience of the internet.

For instance: If a user doesn't follow a specific account, should they ever see posts from that account in their main feed? On Facebook, that happens all the time. On Instagram, it doesn't. Allen's own research at Facebook had shown how troll farms exploit Facebook's ranking algorithm to get mass exposure in the News Feeds of people who didn't even follow them. At the same time, some in the group argued that, in the best case scenario, this can be a way for platforms to share more high-quality content with users who naturally follow low-quality content.

The group compiled the answers they agreed on — that, as it turns out, was not one of them — into a running list it expects to build upon as the community grows. "When I was working on an integrity team, it's really important work, it's under tight constraints, you're working long hours, and there isn't much time to step back and think," Massachi said. "So this is just a venue for integrity workers across all different companies to take a step back and think about this emerging science."

The Institute has won the support of some leaders in the integrity space, including Samidh Chakrabarti, who led the civic integrity team at Facebook before it was dissolved and has since become an outspoken critic of Facebook. "Given the unprecedented influence that digital platforms have on our lives, it is paramount that we bring together the industry's brightest minds to tackle challenges at the intersection of technology and society," Chakrabarti said in a statement to Protocol. "I encourage all those who work in trust and safety and who take their responsibility to the world seriously to join them in this endeavor and I look forward to supporting this work."

Strength in numbers

The Institute also includes members who were, at times, on the opposite side of the table from integrity workers at large tech companies. One such member is Katie Harbath, Facebook's former public policy director, who worked at the company for a decade and who Massachi called a "good faith partner in stress testing solutions from the Integrity team."

"I do think there's some valid questions about how much what they were saying was being prioritized over other things in some of the conversations," Harbath said of integrity workers inside Facebook.

Harbath saw firsthand how difficult it was to balance the integrity team's priorities against the company's larger goals, including growth. "It can be challenging when you have people constantly coming to you, particularly if you weren't on integrity and not working on this, and being told: Your product's going to cause X,Y and Z, and we want to do these things, but that's going to hurt your goals," she said. "It can certainly cause frustration within teams who are like: Wait, leadership told us we had to do these goals, and the integrity team was like: But we were told we have to do these goals."

Coming to any kind of consensus on actual decisions, Harbath said, was often a laborious, time consuming process that required buy-in from the very top. "You would have to get pretty high up into senior leadership before those two teams crossed into somebody that could say: I want to prioritize this thing over that thing," she said.

In a statement to Protocol, Facebook spokesperson Andy Stone said. "No single employee should be able to decide as serious an issue as what content is or isn't allowed on our site or what products get shipped. There's a reason we make our Community Standards public and have a deliberative process for developing our policies, involving teams from across the company as well as consultation with external stakeholders, to make those determinations."

One goal of the institute is to navigate these tricky ethical and technical questions without having to simultaneously navigate a corporate org chart.

Until now, the organization has been funded "out of pocket," Massachi said, "specifically Jeff's pocket." But the group is looking for institutional funding — as long as it doesn't come directly from the companies. Still, Allen and Massachi say they haven't ruled out taking money from funds that are backed by tech executives.

While lawmakers and the media may well be interested in what the group's members have to say, the bigger question is what impact, if any, this group can have inside the companies themselves. Allen and Massachi know better than almost anyone how integrity workers' recommendations can be thwarted or ignored in the face of competing priorities like growth. And through her disclosures, Haugen has also tried to make that point abundantly clear.

If integrity workers are struggling to break through inside the companies that are paying them to do this work, will they really have any better luck working from the outside in?

Allen and Massachi believe they might. For all the attention it's gotten recently, integrity work is still an amorphous and emerging field that's mostly been developed in private. There's no set of minimum standards to gauge a company's success or failure in integrity work the way there is in software engineering or cybersecurity. Allen and Massachi want to turn integrity work into an established discipline of its own.

"You put out the arguments for how to build responsibly or what the actual integrity professional viewpoint is out there, so that people at the companies can point to it internally," Allen said. "So they're not fighting the battle on their own."

A MESSAGE FROM NASDAQ

As we come out of the pandemic, consumers will be even more accustomed to highly flexible, customer-centric digital experiences that let them engage with companies on their own terms. Businesses must consider the impact of an even more digital economy with dynamic buyer behaviors and determine how next-generation customers want to consume products and services and interact with them.

LEARN MORE

Fintech

Judge Zia Faruqui is trying to teach you crypto, one ‘SNL’ reference at a time

His decisions on major cryptocurrency cases have quoted "The Big Lebowski," "SNL," and "Dr. Strangelove." That’s because he wants you — yes, you — to read them.

The ways Zia Faruqui (right) has weighed on cases that have come before him can give lawyers clues as to what legal frameworks will pass muster.

Photo: Carolyn Van Houten/The Washington Post via Getty Images

“Cryptocurrency and related software analytics tools are ‘The wave of the future, Dude. One hundred percent electronic.’”

That’s not a quote from "The Big Lebowski" — at least, not directly. It’s a quote from a Washington, D.C., district court memorandum opinion on the role cryptocurrency analytics tools can play in government investigations. The author is Magistrate Judge Zia Faruqui.

Keep Reading Show less
Veronica Irwin

Veronica Irwin (@vronirwin) is a San Francisco-based reporter at Protocol covering fintech. Previously she was at the San Francisco Examiner, covering tech from a hyper-local angle. Before that, her byline was featured in SF Weekly, The Nation, Techworker, Ms. Magazine and The Frisc.

The financial technology transformation is driving competition, creating consumer choice, and shaping the future of finance. Hear from seven fintech leaders who are reshaping the future of finance, and join the inaugural Financial Technology Association Fintech Summit to learn more.

Keep Reading Show less
FTA
The Financial Technology Association (FTA) represents industry leaders shaping the future of finance. We champion the power of technology-centered financial services and advocate for the modernization of financial regulation to support inclusion and responsible innovation.
Enterprise

AWS CEO: The cloud isn’t just about technology

As AWS preps for its annual re:Invent conference, Adam Selipsky talks product strategy, support for hybrid environments, and the value of the cloud in uncertain economic times.

Photo: Noah Berger/Getty Images for Amazon Web Services

AWS is gearing up for re:Invent, its annual cloud computing conference where announcements this year are expected to focus on its end-to-end data strategy and delivering new industry-specific services.

It will be the second re:Invent with CEO Adam Selipsky as leader of the industry’s largest cloud provider after his return last year to AWS from data visualization company Tableau Software.

Keep Reading Show less
Donna Goodison

Donna Goodison (@dgoodison) is Protocol's senior reporter focusing on enterprise infrastructure technology, from the 'Big 3' cloud computing providers to data centers. She previously covered the public cloud at CRN after 15 years as a business reporter for the Boston Herald. Based in Massachusetts, she also has worked as a Boston Globe freelancer, business reporter at the Boston Business Journal and real estate reporter at Banker & Tradesman after toiling at weekly newspapers.

Image: Protocol

We launched Protocol in February 2020 to cover the evolving power center of tech. It is with deep sadness that just under three years later, we are winding down the publication.

As of today, we will not publish any more stories. All of our newsletters, apart from our flagship, Source Code, will no longer be sent. Source Code will be published and sent for the next few weeks, but it will also close down in December.

Keep Reading Show less
Bennett Richardson

Bennett Richardson ( @bennettrich) is the president of Protocol. Prior to joining Protocol in 2019, Bennett was executive director of global strategic partnerships at POLITICO, where he led strategic growth efforts including POLITICO's European expansion in Brussels and POLITICO's creative agency POLITICO Focus during his six years with the company. Prior to POLITICO, Bennett was co-founder and CMO of Hinge, the mobile dating company recently acquired by Match Group. Bennett began his career in digital and social brand marketing working with major brands across tech, energy, and health care at leading marketing and communications agencies including Edelman and GMMB. Bennett is originally from Portland, Maine, and received his bachelor's degree from Colgate University.

Enterprise

Why large enterprises struggle to find suitable platforms for MLops

As companies expand their use of AI beyond running just a few machine learning models, and as larger enterprises go from deploying hundreds of models to thousands and even millions of models, ML practitioners say that they have yet to find what they need from prepackaged MLops systems.

As companies expand their use of AI beyond running just a few machine learning models, ML practitioners say that they have yet to find what they need from prepackaged MLops systems.

Photo: artpartner-images via Getty Images

On any given day, Lily AI runs hundreds of machine learning models using computer vision and natural language processing that are customized for its retail and ecommerce clients to make website product recommendations, forecast demand, and plan merchandising. But this spring when the company was in the market for a machine learning operations platform to manage its expanding model roster, it wasn’t easy to find a suitable off-the-shelf system that could handle such a large number of models in deployment while also meeting other criteria.

Some MLops platforms are not well-suited for maintaining even more than 10 machine learning models when it comes to keeping track of data, navigating their user interfaces, or reporting capabilities, Matthew Nokleby, machine learning manager for Lily AI’s product intelligence team, told Protocol earlier this year. “The duct tape starts to show,” he said.

Keep Reading Show less
Kate Kaye

Kate Kaye is an award-winning multimedia reporter digging deep and telling print, digital and audio stories. She covers AI and data for Protocol. Her reporting on AI and tech ethics issues has been published in OneZero, Fast Company, MIT Technology Review, CityLab, Ad Age and Digiday and heard on NPR. Kate is the creator of RedTailMedia.org and is the author of "Campaign '08: A Turning Point for Digital Media," a book about how the 2008 presidential campaigns used digital media and data.

Latest Stories
Bulletins