Protocol | Policy

Why Twitch’s 'hate raid' lawsuit isn’t just about Twitch

When is it OK for tech companies to unmask their anonymous users? And when should a violation of terms of service get someone sued?

Phone with Twitch app

The case Twitch is bringing against two hate raiders is hardly black and white.

Photo: Caspar Camille Rubin/Unsplash

It isn't hard to figure out who the bad guys are in Twitch's latest lawsuit against two of its users. On one side are two anonymous "hate raiders" who have been allegedly bombarding the gaming platform with abhorrent attacks on Black and LGBTQ+ users, using armies of bots to do it. On the other side is Twitch, a company that, for all the lumps it's taken for ignoring harassment on its platform, is finally standing up to protect its users against persistent violators whom it's been unable to stop any other way.

But the case Twitch is bringing against these hate raiders is hardly black and white. For starters, the plaintiff here isn't an aggrieved user suing another user for defamation on the platform. The plaintiff is the platform itself. Complicating matters more is the fact that, according to a spokesperson, at least part of Twitch's goal in the case is to "shed light on the identity of the individuals behind these attacks," raising complicated questions about when tech companies should be able to use the courts to unmask their own anonymous users and, just as critically, when they should be able to actually sue them for violating their speech policies.

"Normally, what happens when someone violates your terms of service is you boot them off the service," said Aaron Mackey, senior staff attorney for the Electronic Frontier Foundation. "This lawsuit, while ostensibly good because it's aimed at bad actors who have done pretty awful things, is an escalation."

A Twitch spokesperson emphasized that the company "respect[s] anonymity" and that Twitch is using "all of our tools in the tool kit to stop the attack — including a legal claim."

"Our intent is not to reveal the actors' names but to identify these individuals for law enforcement or to compel them to cease attacking our community," the spokesperson said.

Tech companies sue their own users all the time for violating rules against spam, selling counterfeit goods and even data scraping. The Twitch suit has some of that, taking aim at the two anonymous users — known as CruzzControl and CreatineOverdose — for allegedly using bots to assault users with slurs and other offensive speech. But it's also coming after these users for the offensive speech itself, which Twitch argues violates its policies against "hateful conduct," an important but ever-evolving category of violation that companies themselves still struggle to define.

"It's not new for online service providers such as Twitch to sue their own users," said Riana Pfefferkorn, a research scholar at the Stanford Internet Observatory, who herself brought a case against spammers on behalf of Twitter while working in private practice. "What I find novel is that the purpose of the large-scale, bot-enabled violation of the platform's policies was spewing hate at other users, rather than spammy behavior or phishing."

That signals an important recognition by Twitch that hateful behavior on its platform can be as damaging to users' experience as spam or other bad behavior, if not more. And trying new ways to punish that behavior is laudable.

And yet, taking users to court over violations of terms of service has been particularly contentious in other contexts. Academics and civil rights groups, for instance, have fought back against tech platforms' crackdowns on scraping, arguing that data scraping, which lots of companies consider a privacy violation, is actually an important research method. The Supreme Court recently voted against the Department of Justice in a case where the DOJ argued that breaching terms of service constituted a violation of the Computer Fraud and Abuse Act.

Suing over violations of terms of service can seem virtuous when the defendants are so — well, not. But, Mackey said, "I think there could be larger implications to what's happening here."

Twitch's spokesperson noted that "anyone is subject to a lawsuit under a Terms of Service given that it is treated as a contract." And that's true. It's just that when it comes to hate speech issues, those lawsuits by companies are vanishingly rare.

Asking a court to intervene in identifying an anonymous person is also not a trivial thing. The defendants' actions in this case may be detestable, but there are any number of other good reasons for people's anonymity to be protected both online and off. That's why courts have long recognized that the First Amendment establishes a right to anonymity.

But while that protection is strong, it's "not absolute," said Jeff Kosseff, associate professor of cybersecurity law in the United States Naval Academy, who is writing a book on anonymity. "It comes down to looking at the strength of the case," Kosseff said.

Courts have developed a framework to determine when it is reasonable to issue subpoenas to unmask anonymous figures. That framework depends on things like whether the plaintiffs have a legitimate claim and whether they can show a necessity for the information. Other tests courts have used include assessing whether the information can be attained anywhere else or whether the harm of unmasking outweighs the need to unmask.

"The test is designed to not be absolute, but to be flexible, to allow for certain situations where perhaps someone doesn't have to be publicly identified in a court filing, but you could use their information," Mackey said.

Typically the plaintiffs in these cases are users seeking subpoenas that could compel online service providers to give up other users' identities. "The twist here is that the service provider is also the plaintiff," Pfefferkorn said.

Of course, there's also a case to be made that the speech these users have engaged in, which included "racial slurs and descriptions of violent acts against racial minorities and members of the LGBTQIA+ community," according to the complaint, is harassment, which is not protected by the First Amendment. "I think those are going to be tricky questions," Mackey said. "Do these folks have a First Amendment right to engage in their expression?"

Twitch argues they do not. "In this case, the actors are not using anonymity to conduct speech or expression activity or to participate in our community, but rather to deliberately attack our community, break our terms of service, and obfuscate their methods," the spokesperson said.

According to what information Twitch does have about the users, they appear to be outside of the United States. Foreign defendants are entitled to First Amendment protections in U.S. courts, but as Pfefferkorn put it, "the wheels of justice against overseas defendants move even more slowly than they do when everyone involved is located in the U.S."

The big question is what kind of precedent this case could set for other tech platforms that are also trying to figure out how to stop targeted harassment from their users. Hate raids may be a phenomenon associated with Twitch, but targeted hate speech is a problem all over the internet. Kosseff is dubious the case will open the floodgates to mass litigation, if it even goes forward. For one thing, it's expensive, and for another, it's not a great look for platforms to sue average users for slight offenses. The defendants in this case, Kosseff argues, happen to be particularly odious.

There's also no guarantee these defendants' identities can even be found, if they were really diligent about covering their tracks. "[Twitch] presumably has some sort of IP address, it could be a Tor exit node, in which case, it's useless," Kosseff said. He also pointed out Twitch filed another lawsuit against anonymous trolls in 2019, which it voluntarily dismissed a year later.

Even if the case doesn't go forward, though, the threat of this type of suit is, on its own, a signaling exercise. "Twitch is ready, willing and able to expend significant resources against hate-spewing abusers on its service," Pfefferkorn said. "If having an account terminated for terms of service violations doesn't scare the people who harass and victimize others on Twitch, perhaps the specter of being sued in federal court will."

Protocol | Policy

5 things to know about FCC nominee Gigi Sohn

The veteran of some of the earliest tech policy fights is a longtime consumer champion and net-neutrality advocate.

Gigi Sohn, who President Joe Biden nominated to serve on the FCC, is a longtime net-neutrality advocate.

Photo: Alex Wong/Getty Images

President Joe Biden on Tuesday nominated Gigi Sohn to serve as a Federal Communications Commissioner, teeing up a Democratic majority at the agency that oversees broadband issues after months of delay.

Like Lina Khan, who Biden picked in June to head up the Federal Trade Commission, Sohn is a progressive favorite. And if confirmed, she'll take up a position in an agency trying to pull policy levers on net neutrality, privacy and broadband access even as Congress is stalled.

Keep Reading Show less
Ben Brody

Ben Brody (@ BenBrodyDC) is a senior reporter at Protocol focusing on how Congress, courts and agencies affect the online world we live in. He formerly covered tech policy and lobbying (including antitrust, Section 230 and privacy) at Bloomberg News, where he previously reported on the influence industry, government ethics and the 2016 presidential election. Before that, Ben covered business news at CNNMoney and AdAge, and all manner of stories in and around New York. He still loves appearing on the New York news radio he grew up with.

If you've ever tried to pick up a new fitness routine like running, chances are you may have fallen into the "motivation vs. habit" trap once or twice. You go for a run when the sun is shining, only to quickly fall off the wagon when the weather turns sour.

Similarly, for many businesses, 2020 acted as the storm cloud that disrupted their plans for innovation. With leaders busy grappling with the pandemic, innovation frequently got pushed to the backburner. In fact, according to McKinsey, the majority of organizations shifted their focus mainly to maintaining business continuity throughout the pandemic.

Keep Reading Show less
Gaurav Kataria
Group Product Manager, Trello at Atlassian
Protocol | Workplace

Adobe wants a more authentic NFT world

Adobe's Content Credentials feature will allow Creative Cloud subscribers to attach edit-tracking information to Photoshop files. The goal is to create a more trustworthy NFT market and digital landscape.

Adobe's Content Credentials will allow users to attach their identities to an image

Image: Adobe

Remember the viral, fake photo of Kurt Cobain and Biggie Smalls that duped and delighted the internet in 2017? Doctored images manipulate people and erode trust and we're not great at spotting them. The entire point of the emerging NFT art market is to create valuable and scarce digital files and when there isn't an easy way to check for an image's origin and edits, there's a problem. What if someone steals an NFT creator's image and pawns it off as their own? As a hub for all kinds of multimedia, Adobe feels a responsibility to combat misinformation and provide a safe space for NFT creators. That's why it's rolling out Content Credentials, a record that can be attached to a Photoshop file of a creator's identity and includes any edits they made.

Users can connect their social media addresses and crypto wallet addresses to images in Photoshop. This further proves the image creator's identity, but it's also helpful in determining the creators of NFTs. Adobe has partnered with NFT marketplaces KnownOrigin, OpenSea, Rarible and SuperRare in this effort. "Today there's not a way to know that the NFT you're buying was actually created by a true creator," said Adobe General Counsel Dana Rao. "We're allowing the creator to show their identity and attach it to the image."

Keep Reading Show less
Lizzy Lawrence

Lizzy Lawrence ( @LizzyLaw_) is a reporter at Protocol, covering tools and productivity in the workplace. She's a recent graduate of the University of Michigan, where she studied sociology and international studies. She served as editor in chief of The Michigan Daily, her school's independent newspaper. She's based in D.C., and can be reached at llawrence@protocol.com.

Protocol | China

Why another Chinese lesbian dating app just shut down

With neither political support nor a profitable business model, lesbian dating apps are finding it hard to survive in China.

Operating a dating app for LGBTQ+ communities in China is like walking a tightrope.

Photo: Nicolas Asfouri/AFP via Getty Images

When Lesdo, a Chinese dating app designed for lesbian women, announced it was closing down, it didn't come as a surprise to the LGBTQ+ community.

It's unclear what directly caused this decision. 2021 hasn't been kind to China's queer communities; WeChat has deactivated queer groups' public accounts and Beijing has pressured charity organizations not to work with queer activists.

Keep Reading Show less
Zeyi Yang
Zeyi Yang is a reporter with Protocol | China. Previously, he worked as a reporting fellow for the digital magazine Rest of World, covering the intersection of technology and culture in China and neighboring countries. He has also contributed to the South China Morning Post, Nikkei Asia, Columbia Journalism Review, among other publications. In his spare time, Zeyi co-founded a Mandarin podcast that tells LGBTQ stories in China. He has been playing Pokemon for 14 years and has a weird favorite pick.

The Oura Ring was a sleep-tracking hit. Can the next one be even more?

Oura wants to be a media company, an activity tracker and even a way to know you're sick before you feel sick.

Over the last few years, the Oura Ring has become one of the most recognizable wearables this side of the Apple Watch.

Photo: Oura

Oura CEO Harpreet Rai swears he didn't know Kim Kardashian was a fan. He was as surprised as anyone when she started posting screenshots from the Oura app to her Instagram story, and got into a sleep battle with fellow Oura user Gwyneth Paltrow. Or when Jennifer Aniston revealed that Jimmy Kimmel got her hooked on Oura … and how her ring fell off in a salad. "I am addicted to it," Aniston said, "and it's ruining my life" by shaming her about her lack of sleep. "I think we're definitely seeing traction outside of tech," Rai said. "Which is cool."

Over the last couple of years, Oura's ring (imaginatively named the Oura Ring) has become one of the most recognizable wearables this side of the Apple Watch. The company started with a Kickstarter campaign in 2015, but really started to find traction with its second-generation model in 2018. It's not exactly a mainstream device — Oura said it has sold more than 500,000 rings, up from 150,000 in March 2020 but still not exactly Apple Watch levels — but it has reached some of the most successful, influential and probably sleep-deprived people in the industry. Jack Dorsey is a professed fan, as is Marc Benioff.

Keep Reading Show less
David Pierce

David Pierce ( @pierce) is Protocol's editorial director. Prior to joining Protocol, he was a columnist at The Wall Street Journal, a senior writer with Wired, and deputy editor at The Verge. He owns all the phones.

Latest Stories