Protocol | Policy

Why Twitch’s 'hate raid' lawsuit isn’t just about Twitch

When is it OK for tech companies to unmask their anonymous users? And when should a violation of terms of service get someone sued?

Phone with Twitch app

The case Twitch is bringing against two hate raiders is hardly black and white.

Photo: Caspar Camille Rubin/Unsplash

It isn't hard to figure out who the bad guys are in Twitch's latest lawsuit against two of its users. On one side are two anonymous "hate raiders" who have been allegedly bombarding the gaming platform with abhorrent attacks on Black and LGBTQ+ users, using armies of bots to do it. On the other side is Twitch, a company that, for all the lumps it's taken for ignoring harassment on its platform, is finally standing up to protect its users against persistent violators whom it's been unable to stop any other way.

But the case Twitch is bringing against these hate raiders is hardly black and white. For starters, the plaintiff here isn't an aggrieved user suing another user for defamation on the platform. The plaintiff is the platform itself. Complicating matters more is the fact that, according to a spokesperson, at least part of Twitch's goal in the case is to "shed light on the identity of the individuals behind these attacks," raising complicated questions about when tech companies should be able to use the courts to unmask their own anonymous users and, just as critically, when they should be able to actually sue them for violating their speech policies.

"Normally, what happens when someone violates your terms of service is you boot them off the service," said Aaron Mackey, senior staff attorney for the Electronic Frontier Foundation. "This lawsuit, while ostensibly good because it's aimed at bad actors who have done pretty awful things, is an escalation."

A Twitch spokesperson emphasized that the company "respect[s] anonymity" and that Twitch is using "all of our tools in the tool kit to stop the attack — including a legal claim."

"Our intent is not to reveal the actors' names but to identify these individuals for law enforcement or to compel them to cease attacking our community," the spokesperson said.

Tech companies sue their own users all the time for violating rules against spam, selling counterfeit goods and even data scraping. The Twitch suit has some of that, taking aim at the two anonymous users — known as CruzzControl and CreatineOverdose — for allegedly using bots to assault users with slurs and other offensive speech. But it's also coming after these users for the offensive speech itself, which Twitch argues violates its policies against "hateful conduct," an important but ever-evolving category of violation that companies themselves still struggle to define.

"It's not new for online service providers such as Twitch to sue their own users," said Riana Pfefferkorn, a research scholar at the Stanford Internet Observatory, who herself brought a case against spammers on behalf of Twitter while working in private practice. "What I find novel is that the purpose of the large-scale, bot-enabled violation of the platform's policies was spewing hate at other users, rather than spammy behavior or phishing."

That signals an important recognition by Twitch that hateful behavior on its platform can be as damaging to users' experience as spam or other bad behavior, if not more. And trying new ways to punish that behavior is laudable.

And yet, taking users to court over violations of terms of service has been particularly contentious in other contexts. Academics and civil rights groups, for instance, have fought back against tech platforms' crackdowns on scraping, arguing that data scraping, which lots of companies consider a privacy violation, is actually an important research method. The Supreme Court recently voted against the Department of Justice in a case where the DOJ argued that breaching terms of service constituted a violation of the Computer Fraud and Abuse Act.

Suing over violations of terms of service can seem virtuous when the defendants are so — well, not. But, Mackey said, "I think there could be larger implications to what's happening here."

Twitch's spokesperson noted that "anyone is subject to a lawsuit under a Terms of Service given that it is treated as a contract." And that's true. It's just that when it comes to hate speech issues, those lawsuits by companies are vanishingly rare.

Asking a court to intervene in identifying an anonymous person is also not a trivial thing. The defendants' actions in this case may be detestable, but there are any number of other good reasons for people's anonymity to be protected both online and off. That's why courts have long recognized that the First Amendment establishes a right to anonymity.

But while that protection is strong, it's "not absolute," said Jeff Kosseff, associate professor of cybersecurity law in the United States Naval Academy, who is writing a book on anonymity. "It comes down to looking at the strength of the case," Kosseff said.

Courts have developed a framework to determine when it is reasonable to issue subpoenas to unmask anonymous figures. That framework depends on things like whether the plaintiffs have a legitimate claim and whether they can show a necessity for the information. Other tests courts have used include assessing whether the information can be attained anywhere else or whether the harm of unmasking outweighs the need to unmask.

"The test is designed to not be absolute, but to be flexible, to allow for certain situations where perhaps someone doesn't have to be publicly identified in a court filing, but you could use their information," Mackey said.

Typically the plaintiffs in these cases are users seeking subpoenas that could compel online service providers to give up other users' identities. "The twist here is that the service provider is also the plaintiff," Pfefferkorn said.

Of course, there's also a case to be made that the speech these users have engaged in, which included "racial slurs and descriptions of violent acts against racial minorities and members of the LGBTQIA+ community," according to the complaint, is harassment, which is not protected by the First Amendment. "I think those are going to be tricky questions," Mackey said. "Do these folks have a First Amendment right to engage in their expression?"

Twitch argues they do not. "In this case, the actors are not using anonymity to conduct speech or expression activity or to participate in our community, but rather to deliberately attack our community, break our terms of service, and obfuscate their methods," the spokesperson said.

According to what information Twitch does have about the users, they appear to be outside of the United States. Foreign defendants are entitled to First Amendment protections in U.S. courts, but as Pfefferkorn put it, "the wheels of justice against overseas defendants move even more slowly than they do when everyone involved is located in the U.S."

The big question is what kind of precedent this case could set for other tech platforms that are also trying to figure out how to stop targeted harassment from their users. Hate raids may be a phenomenon associated with Twitch, but targeted hate speech is a problem all over the internet. Kosseff is dubious the case will open the floodgates to mass litigation, if it even goes forward. For one thing, it's expensive, and for another, it's not a great look for platforms to sue average users for slight offenses. The defendants in this case, Kosseff argues, happen to be particularly odious.

There's also no guarantee these defendants' identities can even be found, if they were really diligent about covering their tracks. "[Twitch] presumably has some sort of IP address, it could be a Tor exit node, in which case, it's useless," Kosseff said. He also pointed out Twitch filed another lawsuit against anonymous trolls in 2019, which it voluntarily dismissed a year later.

Even if the case doesn't go forward, though, the threat of this type of suit is, on its own, a signaling exercise. "Twitch is ready, willing and able to expend significant resources against hate-spewing abusers on its service," Pfefferkorn said. "If having an account terminated for terms of service violations doesn't scare the people who harass and victimize others on Twitch, perhaps the specter of being sued in federal court will."

Protocol | Enterprise

Why Segment is central to Twilio’s path to enterprise software stardom

Given Apple's recent changes to third-party tracking technology and Google's looming changes, the customer data platform provider is poised to play a central role in Twilio's product vision moving forward.

The launch of Engage points to the critical role Segment will play in Twilio's future.

Photo: David Paul Morris/Bloomberg via Getty Images

This week at Twilio's annual conference, it's Segment, the company it acquired last year for $3.2 billion, that's poised to take center stage.

Signal kicks off on Wednesday. Alongside Michelle Obama, one of the highlights of the two-day event is bound to be Twilio Engage, a new product that showcases the combined capabilities of the cloud communications provider and Segment, the customer data platform vendor that Twilio bought last November.

Keep Reading Show less
Joe Williams

Joe Williams is a senior reporter at Protocol covering enterprise software, including industry giants like Salesforce, Microsoft, IBM and Oracle. He previously covered emerging technology for Business Insider. Joe can be reached at JWilliams@Protocol.com. To share information confidentially, he can also be contacted on a non-work device via Signal (+1-309-265-6120) or JPW53189@protonmail.com.

The way we work has fundamentally changed. COVID-19 upended business dealings and office work processes, putting into hyperdrive a move towards digital collaboration platforms that allow teams to streamline processes and communicate from anywhere. According to the International Data Corporation, the revenue for worldwide collaboration applications increased 32.9 percent from 2019 to 2020, reaching $22.6 billion; it's expected to become a $50.7 billion industry by 2025.

"While consumers and early adopter businesses had widely embraced collaborative applications prior to the pandemic, the market saw five years' worth of new users in the first six months of 2020," said Wayne Kurtzman, research director of social and collaboration at IDC. "This has cemented collaboration, at least to some extent, for every business, large and small."

Keep Reading Show less
Kate Silver

Kate Silver is an award-winning reporter and editor with 15-plus years of journalism experience. Based in Chicago, she specializes in feature and business reporting. Kate's reporting has appeared in the Washington Post, The Chicago Tribune, The Atlantic's CityLab, Atlas Obscura, The Telegraph and many other outlets.

Protocol | Workplace

The whiteboard wars: Miro and Figma want to make meetings better

Miro and Figma separately launched features on Tuesday aimed at improving collaboration on their platforms.

Whiteboard rivals Miro and Figma each released collaboration improvements.

Logos: Figma and Miro

We expect a lot from our productivity tools these days. You can't just stroll over to your team members' desks and show them what you're working on anymore. Most of those interactions need to happen online, and it's even better if the work and the communication can happen in one place. Miro and Figma — competitors in the collaborative whiteboard space — understand how critical remote collaboration is, and are both working to up their meeting game.

This week, both platforms announced features aimed at improving the collaboration experience, each vying to be the home base for teams to work and hang out together. Figma announced updates to its multiplayer whiteboard FigJam, and Miro announced a new set of tools that it's calling Miro Smart Meetings. Figma's goal is to make FigJam more customizable and accessible for everyone; Miro wants to be the best place for content-centered, professional meetings. They both want to be the go-to hub for teams looking to get stuff done.

Keep Reading Show less
Lizzy Lawrence

Lizzy Lawrence ( @LizzyLaw_) is a reporter at Protocol, covering tools and productivity in the workplace. She's a recent graduate of the University of Michigan, where she studied sociology and international studies. She served as editor in chief of The Michigan Daily, her school's independent newspaper. She's based in D.C., and can be reached at llawrence@protocol.com.

Protocol | Workplace

Hybrid work is here to stay. Here’s how to do it better.

We've recovered from the COVID-19 digital collaboration whiplash. Now we must build a more intentional model for hybrid work.

This is a call to managers to understand the mundane or unwanted projects their employees face, and what work excites them.

Photo: Adobe

Ashley Still is Adobe's Senior Vice President of Digital Media – Marketing, Strategy & Global Partnerships.

When COVID-19 hit, we were forced into a fully digital mode of business operation. Overnight, we adopted available remote work tools — even if imperfect, they were the best tools for the job.

Keep Reading Show less
Ashley Still
As Senior Vice President, Digital Media – Marketing, Strategy & Global Partnerships, Ashley Still leads product marketing and business development for Adobe's flagship Creative Cloud and Document Cloud offerings. This includes iconic software brands such as Photoshop, Lightroom, Illustrator, InDesign and Acrobat. Her expanded remit now includes Adobe's strategic partnership work with technology companies globally, including Apple, Microsoft and Google; and driving Adobe's fast-growing mobile app business. Her team is also responsible for the demand generation marketing campaigns that makes Adobe the market-leader, across creative and document productivity segments. Previously she was Vice President and General Manager, Adobe Creative Cloud for Enterprise. Here her team delivered an integrated content creation, collaboration and publishing solution that securely enables brands to create exceptional design and content. Prior to this, Ashley was Senior Director of Product & Marketing for Adobe Primetime, an Internet television platform used by Comcast, Turner, NBC Sports and other global media companies to deliver TV content and dynamic advertising to any Internet device. Under Ashley's leadership, Adobe Primetime won an Emmy Award for the Adobe Pass TV-Everywhere service. Ashley joined Adobe in 2004 following her internship with the company and held several product management positions for Adobe Photoshop. Still earned her Bachelor of Arts degree from Yale University and her Masters degree from Stanford Graduate School of Business.
Protocol | Workplace

Meet the productivity app influencers

Within the realm of productivity influencing, there is a somewhat surprising sect: Creators who center their content around a specific productivity app.

People are making content and building courses based off of their favorite productivity apps.

Photos: Courtesy

This is the creators' internet. The rest of us are just living in it. We're accustomed to the scores of comedy TikTokers, beauty YouTubers and lifestyle Instagram influencers gracing our feeds. A significant portion of these creators are productivity gurus, advising their followers on how they organize their lives.

Within the realm of productivity influencing, there's a surprising sect: Creators who center their content around a specific productivity app. They're a powerful part of these apps' ecosystems, drawing users to the platform and offering helpful tips and tricks. Notion in particular has a huge influencer family, with #notion gaining millions of views on TikTok.

Keep Reading Show less
Lizzy Lawrence

Lizzy Lawrence ( @LizzyLaw_) is a reporter at Protocol, covering tools and productivity in the workplace. She's a recent graduate of the University of Michigan, where she studied sociology and international studies. She served as editor in chief of The Michigan Daily, her school's independent newspaper. She's based in D.C., and can be reached at llawrence@protocol.com.

Latest Stories