Bulletins

TikTok moderators sue over 'trauma,' proving there’s no easy fix to stop horrible content

Relying on AI to moderate content is an imperfect solution. But companies also have to address the trauma that human moderators face.

TikTok on an iPhone

TikTok is facing a class-action lawsuit over content moderation "trauma."

Photo: Solen Feyissa/Unsplash

Moderating content on social media platforms is key to mitigating the spread of everything from conspiracy theories to horrific photos and videos of mass violence. Companies like Facebook use a combination of tools to search out and remove posts that violate their rules, including AI and human content moderators.

TikTok, like Facebook, employs humans to view and take down problematic posts. On Thursday, two former content moderators filed a federal lawsuit against the company for negligence and a lack of protection for workers against emotional trauma. They are seeking class-action status.


The two women behind the suit, Ashley Velez and Reece Young, both worked as TikTok moderators on contract through third-party companies, but claimed TikTok controlled the rules of their work day-to-day. They also alleged that the videos and photos they were forced to review, combined with strict schedules, consistent 12-hour days and aggressive quotas, left them traumatized.

The allegations echo claims in a lawsuit that Facebook (now Meta) settled for $52 million in May 2020. In that case, 11,250 moderators accused the company of similar abuses. The experiences these workers describe sound ghastly: One of the women suing TikTok recalled watching videos of children being abused, for example. Moderators in the Facebook suit referenced watching videos of suicide.

But relying solely on AI to censor content is also an imperfect solution. AI models are more likely to censor tweets authored by Black people, one study showed. Indigenous activists say they’ve been censored by racially biased algorithms, too. The known potential for bias also allows content-moderating algorithms to be more easily politicized, inflamed by national polarization so that different social media platforms become known as left- or right-wing. TikTok creators complain of getting their content removed for the wrong reasons so often, entire how-to videos and articles are dedicated to learning ways to appeal the decision.

TikTok is known for providing its content moderators better pay and benefits than many of its competitors, and has been on a hiring spree to bring content moderation in-house. The lawsuit filed by contractors indicates that perhaps that move was a good one. But TikTok must both figure out how to better protect its content moderators and keep its moderation strategy agile, proactive and accurate to succeed.

Latest Bulletins

Mobile game revenue will decline for the first time in history this year, market research firm Newzoo now says in a revised outlook for the 2022 global games market. While the whole game industry is expected to contract by 4.3% — another first since Newzoo began tracking the market in 2007 — the company is predicting a 6.4% decline in mobile game spending on top of a 4.2% decline in console game spending.

Keep Reading Show less

Amazon is planning to lay off thousands of employees, Protocol has learned, ahead of what the company has cautioned will be a slow holiday shopping season.

Keep Reading Show less

Google agreed to pay $391.5 million and make changes to its user privacy controls as part of a settlement with a coalition of 40 state attorneys general. The coalition accused Google of misleading customers about location-tracking practices that informed ad targeting.

Keep Reading Show less

FTX has filed for bankruptcy and the crypto company also announced that founder Sam Bankman-Fried has resigned as CEO.

Keep Reading Show less

Salesforce recently updated its internal policies to make it easier for managers to terminate employees for performance issues without HR involvement, Protocol has learned, a move that comes as the software giant looks to shed as many as 2,500 jobs.

Keep Reading Show less

The Consumer Financial Protection Bureau said fraud and scam reports comprise the top complaint it receives about virtual currencies — and that customers are finding little help from companies when it happens.

Keep Reading Show less

Elon Musk sent his first email to Twitter staff late Wednesday, warning of a difficult economic road ahead and telling employees they need to be in office for a minimum of 40 hours per week. "Sorry that this is my first email to the whole company, but there is no way to sugarcoat the message," he began, ominously.

Keep Reading Show less

Binance isn’t buying FTX after all. The crypto giant said Wednesday it has decided that it “will not pursue the potential acquisition” based on a “corporate due diligence” review.

Keep Reading Show less

On Wednesday, John Kerry unveiled a plan for a new carbon credit program aimed at mobilizing private capital to help middle-income countries transition away from coal and move toward renewable energy.

Keep Reading Show less

Meta announced it was laying off more than 11,000 employees Wednesday morning, slashing jobs in its recruiting department and refocusing its remaining team on AI discovery, ads, and its investment in the metaverse.

"I want to take accountability for these decisions and for how we got here," Mark Zuckerberg wrote in a message to employees that was also posted online. "I know this is tough for everyone, and I’m especially sorry to those impacted."

Keep Reading Show less

Al Gore has one mission this week at COP27, and that’s to give climate negotiators what he hopes will be a critical tool to address the crisis at hand: an independent, global inventory of greenhouse gas emissions, down to the individual facility.

The Climate TRACE coalition just released the world’s most detailed inventory of global greenhouse gas emissions, which Gore, a founding member, is unveiling on Wednesday at the United Nations climate summit in Egypt.

Keep Reading Show less

Way back in March, your friendly Protocol Climate team offered you some tips for writing a climate plan that doesn’t suck. Surely you took that advice. But if for some reason you didn’t, the United Nations has your back.

Keep Reading Show less

Binance CEO Changpeng “CZ” Zhao said Tuesday the crypto powerhouse signed a deal to acquire rival FTX.

Keep Reading Show less

Salesforce is preparing for a major round of layoffs that could affect as many as 2,500 workers across the software vendor, Protocol has learned, in a bid to cut costs amid a new activist investor challenge and harsh economic conditions.

Keep Reading Show less

BlockFi has introduced a new digital assets interest product for accredited investors, after previously agreeing to shut down a yield-paying crypto product that the SEC said was illegal.

Keep Reading Show less

The Justice Department said Monday it seized $3.4 billion worth of bitcoin stolen in the 2012 hack of the Silk Road dark web marketplace.

Keep Reading Show less

U.S. election infrastructure is exceedingly secure, and voter fraud here is so rare it’s comparable to your annual chances of getting struck by lightning. Despite this, former President Donald Trump and a long list of allies in the Republican Party have spent the last two years questioning the overall integrity of the U.S. election system. Many of those allies are now candidates themselves, and their coordinated attack on the country’s status as a democracy is not a relic of 2020. Some have already started repeating these “Big Lie” charges ahead of next week’s midterms. And the social platforms that help them spread their message have prepared few measures to stop it.

Keep Reading Show less

The White House just laid out its climate tech priorities to reach net zero by 2050.

Keep Reading Show less

Coinbase said Thursday that it lost more users in the third quarter. But the decline wasn’t the disastrous drop that Wall Street was expecting, and that sparked a rally in the crypto company’s shares after-hours.

Keep Reading Show less

The Biden administration announced $9 billion in funding Wednesday to improve home efficiency, which could help support the installation of up to 500,000 heat pumps. With winter approaching and utilities warning of gas shortages, there are some major challenges facing the technology that money can be used to tackle.

Keep Reading Show less

Block beat earnings expectations, with strong growth largely fueled by its Cash App business. Traders sent shares up more than 12% after-hours Thursday.

Keep Reading Show less

Stripe is laying off 14% of its staff, its co-founders said Thursday, as the fintech startup must start "building differently for leaner times."

Keep Reading Show less

Roku saw its revenue growth slow in Q3, and warned investors Wednesday that things are about to get worse: “A lot of Q4 ad campaigns are being canceled,” said Roku CEO Anthony Wood during the company’s Q4 earnings call. “We’re seeing lots of big categories pull back. Telecom, insurance … even toy marketers are planning on reducing their spending.”

Keep Reading Show less

Green jobs and corporate climate pledges abound, but skilled sustainability professionals are scarce.

Keep Reading Show less

Robinhood reported a drop in third-quarter revenue but also a narrower loss on Wednesday, in a sign that it might be stabilizing its business as it attempts to recover from a staggering drop in the stock and crypto trading activity that fueled its growth.

Keep Reading Show less
Bulletins