Policy

Why Twitch’s 'hate raid' lawsuit isn’t just about Twitch

When is it OK for tech companies to unmask their anonymous users? And when should a violation of terms of service get someone sued?

Phone with Twitch app

The case Twitch is bringing against two hate raiders is hardly black and white.

Photo: Caspar Camille Rubin/Unsplash

It isn't hard to figure out who the bad guys are in Twitch's latest lawsuit against two of its users. On one side are two anonymous "hate raiders" who have been allegedly bombarding the gaming platform with abhorrent attacks on Black and LGBTQ+ users, using armies of bots to do it. On the other side is Twitch, a company that, for all the lumps it's taken for ignoring harassment on its platform, is finally standing up to protect its users against persistent violators whom it's been unable to stop any other way.

But the case Twitch is bringing against these hate raiders is hardly black and white. For starters, the plaintiff here isn't an aggrieved user suing another user for defamation on the platform. The plaintiff is the platform itself. Complicating matters more is the fact that, according to a spokesperson, at least part of Twitch's goal in the case is to "shed light on the identity of the individuals behind these attacks," raising complicated questions about when tech companies should be able to use the courts to unmask their own anonymous users and, just as critically, when they should be able to actually sue them for violating their speech policies.

"Normally, what happens when someone violates your terms of service is you boot them off the service," said Aaron Mackey, senior staff attorney for the Electronic Frontier Foundation. "This lawsuit, while ostensibly good because it's aimed at bad actors who have done pretty awful things, is an escalation."

A Twitch spokesperson emphasized that the company "respect[s] anonymity" and that Twitch is using "all of our tools in the tool kit to stop the attack — including a legal claim."

"Our intent is not to reveal the actors' names but to identify these individuals for law enforcement or to compel them to cease attacking our community," the spokesperson said.

Tech companies sue their own users all the time for violating rules against spam, selling counterfeit goods and even data scraping. The Twitch suit has some of that, taking aim at the two anonymous users — known as CruzzControl and CreatineOverdose — for allegedly using bots to assault users with slurs and other offensive speech. But it's also coming after these users for the offensive speech itself, which Twitch argues violates its policies against "hateful conduct," an important but ever-evolving category of violation that companies themselves still struggle to define.

"It's not new for online service providers such as Twitch to sue their own users," said Riana Pfefferkorn, a research scholar at the Stanford Internet Observatory, who herself brought a case against spammers on behalf of Twitter while working in private practice. "What I find novel is that the purpose of the large-scale, bot-enabled violation of the platform's policies was spewing hate at other users, rather than spammy behavior or phishing."

That signals an important recognition by Twitch that hateful behavior on its platform can be as damaging to users' experience as spam or other bad behavior, if not more. And trying new ways to punish that behavior is laudable.

And yet, taking users to court over violations of terms of service has been particularly contentious in other contexts. Academics and civil rights groups, for instance, have fought back against tech platforms' crackdowns on scraping, arguing that data scraping, which lots of companies consider a privacy violation, is actually an important research method. The Supreme Court recently voted against the Department of Justice in a case where the DOJ argued that breaching terms of service constituted a violation of the Computer Fraud and Abuse Act.

Suing over violations of terms of service can seem virtuous when the defendants are so — well, not. But, Mackey said, "I think there could be larger implications to what's happening here."

Twitch's spokesperson noted that "anyone is subject to a lawsuit under a Terms of Service given that it is treated as a contract." And that's true. It's just that when it comes to hate speech issues, those lawsuits by companies are vanishingly rare.

Asking a court to intervene in identifying an anonymous person is also not a trivial thing. The defendants' actions in this case may be detestable, but there are any number of other good reasons for people's anonymity to be protected both online and off. That's why courts have long recognized that the First Amendment establishes a right to anonymity.

But while that protection is strong, it's "not absolute," said Jeff Kosseff, associate professor of cybersecurity law in the United States Naval Academy, who is writing a book on anonymity. "It comes down to looking at the strength of the case," Kosseff said.

Courts have developed a framework to determine when it is reasonable to issue subpoenas to unmask anonymous figures. That framework depends on things like whether the plaintiffs have a legitimate claim and whether they can show a necessity for the information. Other tests courts have used include assessing whether the information can be attained anywhere else or whether the harm of unmasking outweighs the need to unmask.

"The test is designed to not be absolute, but to be flexible, to allow for certain situations where perhaps someone doesn't have to be publicly identified in a court filing, but you could use their information," Mackey said.

Typically the plaintiffs in these cases are users seeking subpoenas that could compel online service providers to give up other users' identities. "The twist here is that the service provider is also the plaintiff," Pfefferkorn said.

Of course, there's also a case to be made that the speech these users have engaged in, which included "racial slurs and descriptions of violent acts against racial minorities and members of the LGBTQIA+ community," according to the complaint, is harassment, which is not protected by the First Amendment. "I think those are going to be tricky questions," Mackey said. "Do these folks have a First Amendment right to engage in their expression?"

Twitch argues they do not. "In this case, the actors are not using anonymity to conduct speech or expression activity or to participate in our community, but rather to deliberately attack our community, break our terms of service, and obfuscate their methods," the spokesperson said.

According to what information Twitch does have about the users, they appear to be outside of the United States. Foreign defendants are entitled to First Amendment protections in U.S. courts, but as Pfefferkorn put it, "the wheels of justice against overseas defendants move even more slowly than they do when everyone involved is located in the U.S."

The big question is what kind of precedent this case could set for other tech platforms that are also trying to figure out how to stop targeted harassment from their users. Hate raids may be a phenomenon associated with Twitch, but targeted hate speech is a problem all over the internet. Kosseff is dubious the case will open the floodgates to mass litigation, if it even goes forward. For one thing, it's expensive, and for another, it's not a great look for platforms to sue average users for slight offenses. The defendants in this case, Kosseff argues, happen to be particularly odious.

There's also no guarantee these defendants' identities can even be found, if they were really diligent about covering their tracks. "[Twitch] presumably has some sort of IP address, it could be a Tor exit node, in which case, it's useless," Kosseff said. He also pointed out Twitch filed another lawsuit against anonymous trolls in 2019, which it voluntarily dismissed a year later.

Even if the case doesn't go forward, though, the threat of this type of suit is, on its own, a signaling exercise. "Twitch is ready, willing and able to expend significant resources against hate-spewing abusers on its service," Pfefferkorn said. "If having an account terminated for terms of service violations doesn't scare the people who harass and victimize others on Twitch, perhaps the specter of being sued in federal court will."

Fintech

Binance CEO wrestles with the 'Chinese company' label

Changpeng "CZ" Zhao, who leads crypto’s largest marketplace, is pushing back on attempts to link Binance to Beijing.

Despite Binance having to abandon its country of origin shortly after its founding, critics have portrayed the exchange as a tool of the Chinese government.

Photo: Akio Kon/Bloomberg via Getty Images

In crypto, he is known simply as CZ, head of one of the industry’s most dominant players.

It took only five years for Binance CEO and co-founder Changpeng Zhao to build his company, which launched in 2017, into the world’s biggest crypto exchange, with 90 million customers and roughly $76 billion in daily trading volume, outpacing the U.S. crypto powerhouse Coinbase.

Keep Reading Show less
Benjamin Pimentel

Benjamin Pimentel ( @benpimentel) covers crypto and fintech from San Francisco. He has reported on many of the biggest tech stories over the past 20 years for the San Francisco Chronicle, Dow Jones MarketWatch and Business Insider, from the dot-com crash, the rise of cloud computing, social networking and AI to the impact of the Great Recession and the COVID crisis on Silicon Valley and beyond. He can be reached at bpimentel@protocol.com or via Google Voice at (925) 307-9342.

Sponsored Content

Great products are built on strong patents

Experts say robust intellectual property protection is essential to ensure the long-term R&D required to innovate and maintain America's technology leadership.

Every great tech product that you rely on each day, from the smartphone in your pocket to your music streaming service and navigational system in the car, shares one important thing: part of its innovative design is protected by intellectual property (IP) laws.

From 5G to artificial intelligence, IP protection offers a powerful incentive for researchers to create ground-breaking products, and governmental leaders say its protection is an essential part of maintaining US technology leadership. To quote Secretary of Commerce Gina Raimondo: "intellectual property protection is vital for American innovation and entrepreneurship.”

Keep Reading Show less
James Daly
James Daly has a deep knowledge of creating brand voice identity, including understanding various audiences and targeting messaging accordingly. He enjoys commissioning, editing, writing, and business development, particularly in launching new ventures and building passionate audiences. Daly has led teams large and small to multiple awards and quantifiable success through a strategy built on teamwork, passion, fact-checking, intelligence, analytics, and audience growth while meeting budget goals and production deadlines in fast-paced environments. Daly is the Editorial Director of 2030 Media and a contributor at Wired.
Enterprise

How I decided to leave the US and pursue a tech career in Europe

Melissa Di Donato moved to Europe to broaden her technology experience with a different market perspective. She planned to stay two years. Seventeen years later, she remains in London as CEO of Suse.

“It was a hard go for me in the beginning. I was entering inside of a company that had been very traditional in a sense.”

Photo: Suse

Click banner image for more How I decided seriesA native New Yorker, Melissa Di Donato made a life-changing decision back in 2005 when she packed up for Europe to further her career in technology. Then with IBM, she made London her new home base.

Today, Di Donato is CEO of Germany’s Suse, now a 30-year-old, open-source enterprise software company that specializes in Linux operating systems, container management, storage, and edge computing. As the company’s first female leader, she has led Suse through the coronavirus pandemic, a 2021 IPO on the Frankfurt Stock Exchange, and the acquisitions of Kubernetes management startup Rancher Labs and container security company NeuVector.

Keep Reading Show less
Donna Goodison

Donna Goodison (@dgoodison) is Protocol's senior reporter focusing on enterprise infrastructure technology, from the 'Big 3' cloud computing providers to data centers. She previously covered the public cloud at CRN after 15 years as a business reporter for the Boston Herald. Based in Massachusetts, she also has worked as a Boston Globe freelancer, business reporter at the Boston Business Journal and real estate reporter at Banker & Tradesman after toiling at weekly newspapers.

Enterprise

UiPath had a rocky few years. Rob Enslin wants to turn it around.

Protocol caught up with Enslin, named earlier this year as UiPath’s co-CEO, to discuss why he left Google Cloud, the untapped potential of robotic-process automation, and how he plans to lead alongside founder Daniel Dines.

Rob Enslin, UiPath's co-CEO, chats with Protocol about the company's future.

Photo: UiPath

UiPath has had a shaky history.

The company, which helps companies automate business processes, went public in 2021 at a valuation of more than $30 billion, but now the company’s market capitalization is only around $7 billion. To add insult to injury, UiPath laid off 5% of its staff in June and then lowered its full-year guidance for fiscal year 2023 just months later, tanking its stock by 15%.

Keep Reading Show less
Aisha Counts

Aisha Counts (@aishacounts) is a reporter at Protocol covering enterprise software. Formerly, she was a management consultant for EY. She's based in Los Angeles and can be reached at acounts@protocol.com.

Workplace

Figma CPO: We can do more with Adobe

Yuhki Yamashita thinks Figma might tackle video or 3D objects someday.

Figman CPO Yuhki Yamashita told Protocol about Adobe's acquisition of the company.

Photo: Figma

Figma CPO Yuhki Yamashita’s first design gig was at The Harvard Crimson, waiting for writers to file their stories so he could lay them out in Adobe InDesign. Given his interest in computer science, pursuing UX design became the clear move. He worked on Outlook at Microsoft, YouTube at Google, and user experience at Uber, where he was a very early user of Figma. In 2019, he became a VP of product at Figma; this past June, he became CPO.

“Design has been really near and dear to my heart, which is why when this opportunity came along to join Figma and rethink design, it was such an obvious opportunity,” Yamashita said.

Keep Reading Show less
Lizzy Lawrence

Lizzy Lawrence ( @LizzyLaw_) is a reporter at Protocol, covering tools and productivity in the workplace. She's a recent graduate of the University of Michigan, where she studied sociology and international studies. She served as editor in chief of The Michigan Daily, her school's independent newspaper. She's based in D.C., and can be reached at llawrence@protocol.com.

Latest Stories
Bulletins