Policy

They left Facebook’s integrity team. Now they want the world to know how it works.

Without breaking their NDAs.

A black and white image of Lady Jjustice

The Integrity Institute's goal is to build a network of integrity professionals and work toward a public consensus about the nitty gritty scientific and philosophical questions that integrity teams have mostly tried to answer behind closed doors.

Photo: Oleksandr Berezko / EyeEm / Getty Images

Shortly before he left Facebook in October 2019, Jeff Allen published his last report as a data scientist for the company's integrity team — the team Facebook Papers whistleblower Frances Haugen has recently made famous.

The report revealed, as Allen put it at the time, some "genuinely horrifying" findings. Namely, three years after the 2016 election, troll farms in Kosovo and Macedonia were continuing to operate vast networks of Facebook pages filled with mostly plagiarized content targeting Black Americans and Christian Americans on Facebook. Combined, the troll farms' pages reached 140 million Facebook users a month, dwarfing the reach of even Walmart's Facebook presence.

Not only had Facebook failed to stop its spread, Allen wrote in a 20-page report that was recently leaked to and published by MIT Tech Review, but the vast majority of the network's reach came from Facebook's ranking algorithms.

"I have no problem with Macedonians reaching US audiences," Allen wrote. "But if you just want to write python scripts that scrape social media and anonymously regurgitate content into communities while siphoning off some monetary or influence reward for yourself… well you can fuck right off."

A Facebook spokesperson said the company has taken "aggressive enforcement actions against these kinds of foreign and domestic inauthentic groups" and was doing so even when Allen wrote his report.

Allen left Facebook soon after that for a data scientist job at the Democratic National Committee. He ended his report with a goodbye, saying he was sad about leaving the company because he wouldn't be able "to work on this problem with the level of impact that FB provides."

Two years later, Allen is hoping to prove himself wrong about that last part.

For the last 10 months, he and Sahar Massachi, another former Facebook staffer who helped build the company's election and civic integrity teams from their earliest days, have been quietly creating an independent organization they hope could help not just Facebook, but also the entire tech industry and lawmakers alike navigate some of the trickiest questions facing integrity workers today.

Now, Allen and Massachi, who spoke exclusively with Protocol, are publicly launching the group at a time when those questions are at the white hot core of the international scandal, spurred by Haugen's disclosures.

Called the Integrity Institute, its goal is to build a network of integrity professionals who are currently employed or were previously employed by tech companies and work toward some kind of public consensus about the nitty gritty scientific and philosophical questions that integrity teams have mostly tried to answer behind closed doors.

The group plans to advise policy makers, regulators and the media about how social media platforms work, publish its own research and serve as a sort of open-source resource for small platforms that lack massive integrity teams of their own. Already, Allen and Massachi have briefed Congress and recruited 11 members, who have collectively worked for nine different tech platforms, to the organization. While they don't define themselves as whistleblowers and are "resolutely not breaking our NDAs," Massachi said, they say Haugen's disclosures have sparked a growing interest in the emerging field of integrity work — interest they're eager to capitalize on.

"Frances is exposing a lot of the knobs in the machine that is a modern social media platform," Massachi said. "Now that people know that those knobs exist, we can start having a conversation about what is the science of how these work, what these knobs do and why you would want to turn them in which direction."

Another benefit of the Facebook Papers? "My mom finally kind of understands the kind of work I was doing," Massachi said.

'Emerging science'

Integrity work only really got its name a few years ago when companies like Facebook began assigning teams of researchers to study all the ways their platforms could be misused. Massachi joined Facebook in 2016 and was one of the earliest members of its election integrity team, working on a dashboard to monitor the Alabama special election in 2017 and, later, on Facebook's election war room. He describes his time working on that team and Facebook's civic integrity team as a career highlight, but says the job of an integrity worker was also full of frustration.

"I thought that integrity was heroic, virtuous work that was not getting enough power and attention inside the company," he said. The people whose job it was to make the platform more moral and less messy had to "jump through a lot more hoops than normal product work, and that felt really wrong to me," Massachi said, though he's careful not to bash his former employer as others have. "Our membership has a wide variety of views," he said.

In January, Massachi began reaching out to former Facebook integrity workers, including Allen, to see if they were interested in forming something of a super group outside of the company. He sent potential members a memo laying out the problems on social media platforms, as he sees them, and the potential fixes he would propose.

"Then people would say, 'Well, I agree with 80% of this. I really disagree with the other 20%.' Then I would say, 'Great. The world deserves to hear our disagreements. Let's hash it out in public," Massachi, who also recently completed a fellowship at Harvard's Berkman Klein Center, said. "That was a recruiting tactic."

For Allen, now chief research officer of the institute, the concept of an independent integrity body was something he'd been considering himself for years. "I think this is an idea that crosses a lot of integrity workers' minds," he said. "What does it look like to do this outside the company?"

Jeff Allen and Sahar Massachi Jeff Allen (left) and Sahar Massachi (right)Photos: Courtesy Jeff Allen and Sahar Massachi

In July, Massachi and Allen gathered the group's 11 members for a retreat in Washington, D.C. They bunked up together at Airbnbs and spent their days in a coworking space, trying to come to a consensus on questions only a geek could love, but which matter deeply to billions of people's experience of the internet.

For instance: If a user doesn't follow a specific account, should they ever see posts from that account in their main feed? On Facebook, that happens all the time. On Instagram, it doesn't. Allen's own research at Facebook had shown how troll farms exploit Facebook's ranking algorithm to get mass exposure in the News Feeds of people who didn't even follow them. At the same time, some in the group argued that, in the best case scenario, this can be a way for platforms to share more high-quality content with users who naturally follow low-quality content.

The group compiled the answers they agreed on — that, as it turns out, was not one of them — into a running list it expects to build upon as the community grows. "When I was working on an integrity team, it's really important work, it's under tight constraints, you're working long hours, and there isn't much time to step back and think," Massachi said. "So this is just a venue for integrity workers across all different companies to take a step back and think about this emerging science."

The Institute has won the support of some leaders in the integrity space, including Samidh Chakrabarti, who led the civic integrity team at Facebook before it was dissolved and has since become an outspoken critic of Facebook. "Given the unprecedented influence that digital platforms have on our lives, it is paramount that we bring together the industry's brightest minds to tackle challenges at the intersection of technology and society," Chakrabarti said in a statement to Protocol. "I encourage all those who work in trust and safety and who take their responsibility to the world seriously to join them in this endeavor and I look forward to supporting this work."

Strength in numbers

The Institute also includes members who were, at times, on the opposite side of the table from integrity workers at large tech companies. One such member is Katie Harbath, Facebook's former public policy director, who worked at the company for a decade and who Massachi called a "good faith partner in stress testing solutions from the Integrity team."

"I do think there's some valid questions about how much what they were saying was being prioritized over other things in some of the conversations," Harbath said of integrity workers inside Facebook.

Harbath saw firsthand how difficult it was to balance the integrity team's priorities against the company's larger goals, including growth. "It can be challenging when you have people constantly coming to you, particularly if you weren't on integrity and not working on this, and being told: Your product's going to cause X,Y and Z, and we want to do these things, but that's going to hurt your goals," she said. "It can certainly cause frustration within teams who are like: Wait, leadership told us we had to do these goals, and the integrity team was like: But we were told we have to do these goals."

Coming to any kind of consensus on actual decisions, Harbath said, was often a laborious, time consuming process that required buy-in from the very top. "You would have to get pretty high up into senior leadership before those two teams crossed into somebody that could say: I want to prioritize this thing over that thing," she said.

In a statement to Protocol, Facebook spokesperson Andy Stone said. "No single employee should be able to decide as serious an issue as what content is or isn't allowed on our site or what products get shipped. There's a reason we make our Community Standards public and have a deliberative process for developing our policies, involving teams from across the company as well as consultation with external stakeholders, to make those determinations."

One goal of the institute is to navigate these tricky ethical and technical questions without having to simultaneously navigate a corporate org chart.

Until now, the organization has been funded "out of pocket," Massachi said, "specifically Jeff's pocket." But the group is looking for institutional funding — as long as it doesn't come directly from the companies. Still, Allen and Massachi say they haven't ruled out taking money from funds that are backed by tech executives.

While lawmakers and the media may well be interested in what the group's members have to say, the bigger question is what impact, if any, this group can have inside the companies themselves. Allen and Massachi know better than almost anyone how integrity workers' recommendations can be thwarted or ignored in the face of competing priorities like growth. And through her disclosures, Haugen has also tried to make that point abundantly clear.

If integrity workers are struggling to break through inside the companies that are paying them to do this work, will they really have any better luck working from the outside in?

Allen and Massachi believe they might. For all the attention it's gotten recently, integrity work is still an amorphous and emerging field that's mostly been developed in private. There's no set of minimum standards to gauge a company's success or failure in integrity work the way there is in software engineering or cybersecurity. Allen and Massachi want to turn integrity work into an established discipline of its own.

"You put out the arguments for how to build responsibly or what the actual integrity professional viewpoint is out there, so that people at the companies can point to it internally," Allen said. "So they're not fighting the battle on their own."

A MESSAGE FROM NASDAQ

As we come out of the pandemic, consumers will be even more accustomed to highly flexible, customer-centric digital experiences that let them engage with companies on their own terms. Businesses must consider the impact of an even more digital economy with dynamic buyer behaviors and determine how next-generation customers want to consume products and services and interact with them.

LEARN MORE

Workplace

Everything you need to know about tech layoffs and hiring slowdowns

Will tech companies and startups continue to have layoffs?

It’s not just early-stage startups that are feeling the burn.

Photo: Kirsty O'Connor/PA Images via Getty Images

What goes up must come down.

High-flying startups with record valuations, huge hiring goals and ambitious expansion plans are now announcing hiring slowdowns, freezes and in some cases widespread layoffs. It’s the dot-com bust all over again — this time, without the cute sock puppet and in the midst of a global pandemic we just can’t seem to shake.

Keep Reading Show less
Nat Rubio-Licht

Nat Rubio-Licht is a Los Angeles-based news writer at Protocol. They graduated from Syracuse University with a degree in newspaper and online journalism in May 2020. Prior to joining the team, they worked at the Los Angeles Business Journal as a technology and aerospace reporter.

Sustainability. It can be a charged word in the context of blockchain and crypto – whether from outsiders with a limited view of the technology or from insiders using it for competitive advantage. But as a CEO in the industry, I don’t think either of those approaches helps us move forward. We should all be able to agree that using less energy to get a task done is a good thing and that there is room for improvement in the amount of energy that is consumed to power different blockchain technologies.

So, what if we put the enormous industry talent and minds that have created and developed blockchain to the task of building in a more energy-efficient manner? Can we not just solve the issues but also set the standard for other industries to develop technology in a future-proof way?

Keep Reading Show less
Denelle Dixon, CEO of SDF

Denelle Dixon is CEO and Executive Director of the Stellar Development Foundation, a non-profit using blockchain to unlock economic potential by making money more fluid, markets more open, and people more empowered. Previously, Dixon served as COO of Mozilla. Leading the business, revenue and policy teams, she fought for Net Neutrality and consumer privacy protections and was responsible for commercial partnerships. Denelle also served as general counsel and legal advisor in private equity and technology.

Entertainment

Sink into ‘Love, Death & Robots’ and more weekend recs

Don’t know what to do this weekend? We’ve got you covered.

Our favorite picks for your weekend pleasure.

Image: A24; 11 bit studios; Getty Images

We could all use a bit of a break. This weekend we’re diving into Netflix’s beautifully animated sci-fi “Love, Death & Robots,” losing ourselves in surreal “Men” and loving Zelda-like Moonlighter.

Keep Reading Show less
Nick Statt

Nick Statt is Protocol's video game reporter. Prior to joining Protocol, he was news editor at The Verge covering the gaming industry, mobile apps and antitrust out of San Francisco, in addition to managing coverage of Silicon Valley tech giants and startups. He now resides in Rochester, New York, home of the garbage plate and, completely coincidentally, the World Video Game Hall of Fame. He can be reached at nstatt@protocol.com.

Workplace

This machine would like to interview you for a job

Companies are embracing automated video interviews to filter through floods of job applicants. But interviews with a computer screen raise big ethical questions and might scare off candidates.

Although automated interview companies claim to reduce bias in hiring, the researchers and advocates who study AI bias are these companies’ most frequent critics.

Photo: Johner Images via Getty Images

Applying for a job these days is starting to feel a lot like online dating. Job-seekers send their resume into portal after portal and a silent abyss waits on the other side.

That abyss is silent for a reason and it has little to do with the still-tight job market or the quality of your particular resume. On the other side of the portal, hiring managers watch the hundreds and even thousands of resumes pile up. It’s an infinite mountain of digital profiles, most of them from people completely unqualified. Going through them all would be a virtually fruitless task.

Keep Reading Show less
Anna Kramer

Anna Kramer is a reporter at Protocol (Twitter: @ anna_c_kramer, email: akramer@protocol.com), where she writes about labor and workplace issues. Prior to joining the team, she covered tech and small business for the San Francisco Chronicle and privacy for Bloomberg Law. She is a recent graduate of Brown University, where she studied International Relations and Arabic and wrote her senior thesis about surveillance tools and technological development in the Middle East.

Fintech

A crypto advocate’s plea: Cool the Twitter trash talk

A top blockchain advocate says the SEC is wrong in its efforts to regulate crypto, but crypto advocates’ personal attacks aren’t helping.

Chamber of Digital Commerce founder Perianne Boring spoke with Protocol about how crypto can strike a better tone.

Photo: Chamber of Digital Commerce

Chamber of Digital Commerce founder Perianne Boring cites a Bible verse to sum up her philosophy about how the crypto trade group should take on the industry’s many critics.

Her Twitter page refers to Ephesians 4:29, which says — oh, let’s use the King James version, it’s more fun: “Let no corrupt communication proceed out of your mouth.” For her, it’s a reminder that trash talk in defense of crypto is unacceptable.

Keep Reading Show less
Benjamin Pimentel

Benjamin Pimentel ( @benpimentel) covers crypto and fintech from San Francisco. He has reported on many of the biggest tech stories over the past 20 years for the San Francisco Chronicle, Dow Jones MarketWatch and Business Insider, from the dot-com crash, the rise of cloud computing, social networking and AI to the impact of the Great Recession and the COVID crisis on Silicon Valley and beyond. He can be reached at bpimentel@protocol.com or via Google Voice at (925) 307-9342.

Latest Stories
Bulletins