Protocol | Policy

They left Facebook’s integrity team. Now they want the world to know how it works.

Without breaking their NDAs.

A black and white image of Lady Jjustice

The Integrity Institute's goal is to build a network of integrity professionals and work toward a public consensus about the nitty gritty scientific and philosophical questions that integrity teams have mostly tried to answer behind closed doors.

Photo: Oleksandr Berezko / EyeEm / Getty Images

Shortly before he left Facebook in October 2019, Jeff Allen published his last report as a data scientist for the company's integrity team — the team Facebook Papers whistleblower Frances Haugen has recently made famous.

The report revealed, as Allen put it at the time, some "genuinely horrifying" findings. Namely, three years after the 2016 election, troll farms in Kosovo and Macedonia were continuing to operate vast networks of Facebook pages filled with mostly plagiarized content targeting Black Americans and Christian Americans on Facebook. Combined, the troll farms' pages reached 140 million Facebook users a month, dwarfing the reach of even Walmart's Facebook presence.

Not only had Facebook failed to stop its spread, Allen wrote in a 20-page report that was recently leaked to and published by MIT Tech Review, but the vast majority of the network's reach came from Facebook's ranking algorithms.

"I have no problem with Macedonians reaching US audiences," Allen wrote. "But if you just want to write python scripts that scrape social media and anonymously regurgitate content into communities while siphoning off some monetary or influence reward for yourself… well you can fuck right off."

A Facebook spokesperson said the company has taken "aggressive enforcement actions against these kinds of foreign and domestic inauthentic groups" and was doing so even when Allen wrote his report.

Allen left Facebook soon after that for a data scientist job at the Democratic National Committee. He ended his report with a goodbye, saying he was sad about leaving the company because he wouldn't be able "to work on this problem with the level of impact that FB provides."

Two years later, Allen is hoping to prove himself wrong about that last part.

For the last 10 months, he and Sahar Massachi, another former Facebook staffer who helped build the company's election and civic integrity teams from their earliest days, have been quietly creating an independent organization they hope could help not just Facebook, but also the entire tech industry and lawmakers alike navigate some of the trickiest questions facing integrity workers today.

Now, Allen and Massachi, who spoke exclusively with Protocol, are publicly launching the group at a time when those questions are at the white hot core of the international scandal, spurred by Haugen's disclosures.

Called the Integrity Institute, its goal is to build a network of integrity professionals who are currently employed or were previously employed by tech companies and work toward some kind of public consensus about the nitty gritty scientific and philosophical questions that integrity teams have mostly tried to answer behind closed doors.

The group plans to advise policy makers, regulators and the media about how social media platforms work, publish its own research and serve as a sort of open-source resource for small platforms that lack massive integrity teams of their own. Already, Allen and Massachi have briefed Congress and recruited 11 members, who have collectively worked for nine different tech platforms, to the organization. While they don't define themselves as whistleblowers and are "resolutely not breaking our NDAs," Massachi said, they say Haugen's disclosures have sparked a growing interest in the emerging field of integrity work — interest they're eager to capitalize on.

"Frances is exposing a lot of the knobs in the machine that is a modern social media platform," Massachi said. "Now that people know that those knobs exist, we can start having a conversation about what is the science of how these work, what these knobs do and why you would want to turn them in which direction."

Another benefit of the Facebook Papers? "My mom finally kind of understands the kind of work I was doing," Massachi said.

'Emerging science'

Integrity work only really got its name a few years ago when companies like Facebook began assigning teams of researchers to study all the ways their platforms could be misused. Massachi joined Facebook in 2016 and was one of the earliest members of its election integrity team, working on a dashboard to monitor the Alabama special election in 2017 and, later, on Facebook's election war room. He describes his time working on that team and Facebook's civic integrity team as a career highlight, but says the job of an integrity worker was also full of frustration.

"I thought that integrity was heroic, virtuous work that was not getting enough power and attention inside the company," he said. The people whose job it was to make the platform more moral and less messy had to "jump through a lot more hoops than normal product work, and that felt really wrong to me," Massachi said, though he's careful not to bash his former employer as others have. "Our membership has a wide variety of views," he said.

In January, Massachi began reaching out to former Facebook integrity workers, including Allen, to see if they were interested in forming something of a super group outside of the company. He sent potential members a memo laying out the problems on social media platforms, as he sees them, and the potential fixes he would propose.

"Then people would say, 'Well, I agree with 80% of this. I really disagree with the other 20%.' Then I would say, 'Great. The world deserves to hear our disagreements. Let's hash it out in public," Massachi, who also recently completed a fellowship at Harvard's Berkman Klein Center, said. "That was a recruiting tactic."

For Allen, now chief research officer of the institute, the concept of an independent integrity body was something he'd been considering himself for years. "I think this is an idea that crosses a lot of integrity workers' minds," he said. "What does it look like to do this outside the company?"

Jeff Allen and Sahar Massachi Jeff Allen (left) and Sahar Massachi (right)Photos: Courtesy Jeff Allen and Sahar Massachi

In July, Massachi and Allen gathered the group's 11 members for a retreat in Washington, D.C. They bunked up together at Airbnbs and spent their days in a coworking space, trying to come to a consensus on questions only a geek could love, but which matter deeply to billions of people's experience of the internet.

For instance: If a user doesn't follow a specific account, should they ever see posts from that account in their main feed? On Facebook, that happens all the time. On Instagram, it doesn't. Allen's own research at Facebook had shown how troll farms exploit Facebook's ranking algorithm to get mass exposure in the News Feeds of people who didn't even follow them. At the same time, some in the group argued that, in the best case scenario, this can be a way for platforms to share more high-quality content with users who naturally follow low-quality content.

The group compiled the answers they agreed on — that, as it turns out, was not one of them — into a running list it expects to build upon as the community grows. "When I was working on an integrity team, it's really important work, it's under tight constraints, you're working long hours, and there isn't much time to step back and think," Massachi said. "So this is just a venue for integrity workers across all different companies to take a step back and think about this emerging science."

The Institute has won the support of some leaders in the integrity space, including Samidh Chakrabarti, who led the civic integrity team at Facebook before it was dissolved and has since become an outspoken critic of Facebook. "Given the unprecedented influence that digital platforms have on our lives, it is paramount that we bring together the industry's brightest minds to tackle challenges at the intersection of technology and society," Chakrabarti said in a statement to Protocol. "I encourage all those who work in trust and safety and who take their responsibility to the world seriously to join them in this endeavor and I look forward to supporting this work."

Strength in numbers

The Institute also includes members who were, at times, on the opposite side of the table from integrity workers at large tech companies. One such member is Katie Harbath, Facebook's former public policy director, who worked at the company for a decade and who Massachi called a "good faith partner in stress testing solutions from the Integrity team."

"I do think there's some valid questions about how much what they were saying was being prioritized over other things in some of the conversations," Harbath said of integrity workers inside Facebook.

Harbath saw firsthand how difficult it was to balance the integrity team's priorities against the company's larger goals, including growth. "It can be challenging when you have people constantly coming to you, particularly if you weren't on integrity and not working on this, and being told: Your product's going to cause X,Y and Z, and we want to do these things, but that's going to hurt your goals," she said. "It can certainly cause frustration within teams who are like: Wait, leadership told us we had to do these goals, and the integrity team was like: But we were told we have to do these goals."

Coming to any kind of consensus on actual decisions, Harbath said, was often a laborious, time consuming process that required buy-in from the very top. "You would have to get pretty high up into senior leadership before those two teams crossed into somebody that could say: I want to prioritize this thing over that thing," she said.

In a statement to Protocol, Facebook spokesperson Andy Stone said. "No single employee should be able to decide as serious an issue as what content is or isn't allowed on our site or what products get shipped. There's a reason we make our Community Standards public and have a deliberative process for developing our policies, involving teams from across the company as well as consultation with external stakeholders, to make those determinations."

One goal of the institute is to navigate these tricky ethical and technical questions without having to simultaneously navigate a corporate org chart.

Until now, the organization has been funded "out of pocket," Massachi said, "specifically Jeff's pocket." But the group is looking for institutional funding — as long as it doesn't come directly from the companies. Still, Allen and Massachi say they haven't ruled out taking money from funds that are backed by tech executives.

While lawmakers and the media may well be interested in what the group's members have to say, the bigger question is what impact, if any, this group can have inside the companies themselves. Allen and Massachi know better than almost anyone how integrity workers' recommendations can be thwarted or ignored in the face of competing priorities like growth. And through her disclosures, Haugen has also tried to make that point abundantly clear.

If integrity workers are struggling to break through inside the companies that are paying them to do this work, will they really have any better luck working from the outside in?

Allen and Massachi believe they might. For all the attention it's gotten recently, integrity work is still an amorphous and emerging field that's mostly been developed in private. There's no set of minimum standards to gauge a company's success or failure in integrity work the way there is in software engineering or cybersecurity. Allen and Massachi want to turn integrity work into an established discipline of its own.

"You put out the arguments for how to build responsibly or what the actual integrity professional viewpoint is out there, so that people at the companies can point to it internally," Allen said. "So they're not fighting the battle on their own."

A MESSAGE FROM NASDAQ

As we come out of the pandemic, consumers will be even more accustomed to highly flexible, customer-centric digital experiences that let them engage with companies on their own terms. Businesses must consider the impact of an even more digital economy with dynamic buyer behaviors and determine how next-generation customers want to consume products and services and interact with them.

LEARN MORE

Protocol | Fintech

Crypto wallet maker Ledger gears up for battle with Dorsey’s Block

CEO Pascal Gauthier wishes Block’s CEO were still distracted with Twitter, but he’s still gunning for the big opportunity in securely stashing customers’ coins.

Ledger CEO Pascal Gauthier talked about Ledger’s strategy in an interview with Protocol.

Photo: Ledger

Ledger CEO Pascal Gauthier reacted with an odd mix of excitement and fear to news that Jack Dorsey was leaving Twitter to focus full-time on Square.

“Oh, shit,” was his immediate thought, he told Protocol. “I would have preferred him to stay with more companies and not focus on anything.”

Keep Reading Show less
Benjamin Pimentel

Benjamin Pimentel ( @benpimentel) covers fintech from San Francisco. He has reported on many of the biggest tech stories over the past 20 years for the San Francisco Chronicle, Dow Jones MarketWatch and Business Insider, from the dot-com crash, the rise of cloud computing, social networking and AI to the impact of the Great Recession and the COVID crisis on Silicon Valley and beyond. He can be reached at bpimentel@protocol.com or via Signal at (510)731-8429.

In a tight labor market, businesses are competing for top talent, even as employees leave in droves. A record 4.4 million Americans resigned in September 2021 — the highest on record for nearly 20 years — ushering in what some call the Great Resignation. That same month, 65% of U.S. workers said they were looking for a new job.

Business leaders have to respond to mitigate the negative impacts of this disruptive churn, with 36% of CFOs saying they're very concerned about turnover remaining high indefinitely and weighing on revenue growth. The answers to this challenge should be informed by the root causes of employee dissatisfaction as well as retention drivers.

Keep Reading Show less
Suneet Dua, PwC
As PwC’s US Products & Technology Chief Revenue and Growth Officer, Suneet Dua is responsible for driving more than $1 billion in product revenue and executing PwC’s product revenue strategy. He’s focused on driving innovation, delivering world-class, forward-thinking products and digitally upskilling the workforce and society at large. With 20+ years of technology, media and entertainment industry experience, he’s positioned as a catalyst for organizational transformation and delivers on the firm’s promise to solve the world’s most important problems. Additionally, he launched Salesforce and client-focused centers of excellence, such as our Cybersecurity centers in Israel, Singapore and India––all to improve the way PwC serves its clients. During his tenure as US Chief Product Leader, Suneet, and his team, played a critical role in designing and implementing digital tools that upskilled more than 55,000 of its US employees, which led to the development of PwC’s digital learning platform, ProEdge, that addresses the digital skills gap crisis facing today’s workforce. He also serves as a board member of PwC’s Trifecta Consulting (US, China, Japan and Mexico). Previously, Suneet served on PwC’s US leadership team and was Global Client Market Leader for PwC’s Global Network.
Protocol | Fintech

A legal brawl failed to uncover bitcoin’s fabled creator

Is Craig Wright Satoshi Nakamoto? A trial didn’t lead to an answer.

Craig Wright has claimed to be the creator of bitcoin.

Photo: Eugene Gologursky/Getty Images for CoinGeek

A legal battle was supposed to answer the biggest question in crypto: Who is Satoshi Nakamoto?

Well, that didn’t exactly happen. The identity of bitcoin’s fabled creator remains a mystery, despite high hopes that an unusual civil suit would lead to Nakamoto’s unmasking.

Keep Reading Show less
Benjamin Pimentel

Benjamin Pimentel ( @benpimentel) covers fintech from San Francisco. He has reported on many of the biggest tech stories over the past 20 years for the San Francisco Chronicle, Dow Jones MarketWatch and Business Insider, from the dot-com crash, the rise of cloud computing, social networking and AI to the impact of the Great Recession and the COVID crisis on Silicon Valley and beyond. He can be reached at bpimentel@protocol.com or via Signal at (510)731-8429.

Snap CTO Bobby Murphy on embracing Apple’s AR glasses

Snap is building its own AR Spectacles, but the company also wants to embrace third-party devices.

Bobby Murphy wants Snap’s AR lenses to run everywhere — even on hardware made by competitors.

Photo: Getty Images for Snap Inc

Snap is all in on AR: The Snapchat maker has been building its own AR glasses, and is currently testing an early version with a small group of creators. Snap has also signed up 250,000 creators to build mobile-centric AR experiences through its Lens Studio platform, whose lenses have collectively been viewed over 3.5 trillion times.

Snap celebrated those milestones at its Lens Fest Tuesday, which the company also used to release a number of updates for both mobile and headworn AR. Snap CTO Bobby Murphy recently put that work in context in an interview with Protocol, in which he talked about the company’s progress in building AR Spectacles, why it isn’t focused on non-AR wearables anymore and why it ultimately also wants to build apps and experiences for AR devices made by its competitors.

Keep Reading Show less
Janko Roettgers

Janko Roettgers (@jank0) is a senior reporter at Protocol, reporting on the shifting power dynamics between tech, media, and entertainment, including the impact of new technologies. Previously, Janko was Variety's first-ever technology writer in San Francisco, where he covered big tech and emerging technologies. He has reported for Gigaom, Frankfurter Rundschau, Berliner Zeitung, and ORF, among others. He has written three books on consumer cord-cutting and online music and co-edited an anthology on internet subcultures. He lives with his family in Oakland.

Discord launches paid channel memberships

The company’s new subscription tiers effectively broaden the creator economy to include people managing communities.

Select Discord server creators can start charging membership fees as part of a new pilot program.

Image: Discord

Creating and managing successful communities can be a lot of work. Now, Discord wants to make sure that the people doing this on its platform can also reap some rewards: The company launched a pilot program for premium memberships Tuesday that allows community creators to put parts or all of their servers behind a paywall.

“We want to make sure that running communities on Discord is more sustainable,” said Discord Engineering Director Sumeet Vaidya in an interview with Protocol.

Keep Reading Show less
Janko Roettgers

Janko Roettgers (@jank0) is a senior reporter at Protocol, reporting on the shifting power dynamics between tech, media, and entertainment, including the impact of new technologies. Previously, Janko was Variety's first-ever technology writer in San Francisco, where he covered big tech and emerging technologies. He has reported for Gigaom, Frankfurter Rundschau, Berliner Zeitung, and ORF, among others. He has written three books on consumer cord-cutting and online music and co-edited an anthology on internet subcultures. He lives with his family in Oakland.

Latest Stories
Bulletins