a sidewalk chalk mural at the scene of a mass shooting in Buffalo
Photo: Kent Nishimura / Los Angeles Times via Getty Images

The infinite loop of online extremism

Source Code

Good morning. Americans are still reeling from a trio of mass shootings that took place over the weekend, and despite all the thoughts and prayers, it seems the solutions to prevent massacres like the one in Buffalo, New York, are still out of reach. Are tech companies and legislators incapable of solving a problem as out of control as online terrorism?

Is online hate a solvable problem?

Tech platforms' patchwork approach to content moderation has made them a hotbed for hate speech that can turn deadly, as it did this past weekend in Buffalo. The alleged shooter that killed 10 in a historically Black neighborhood used Discord to plan his rampage for months and livestreamed it on Twitch. We’ve seen this all before.

In 2019, a white supremacist murdered 51 people in two mosques in Christchurch, New Zealand. He viewed the killings as a meme. To disseminate that meme, he turned to the same place more than 1 billion other users do: Facebook. The Buffalo shooter was inspired by that livestream.

The platforms change, but the pattern is destined to repeat. Tech companies are playing defense instead of offense against online hate and fail to work together. Online terrorism experts fear it’s an endless loop.

  • “They’ll take the hits when someone uses their platform to post terrorism and they will take the the bad op-eds about them, because they know that we've been here before and we'll be here again,” Jon Lewis, a research fellow at The George Washington University’s Program on Extremism, told me.
  • But even if one company quickly addresses violence on its platform, another may be slow to act or not act at all. Twitch took down the Buffalo shooter’s livestream within two minutes after it began, but the video was still screen-recorded and posted across other social media platforms, where it was viewed millions of times.
  • Gab, an alt-right site that hosts neo-Nazi groups, and 4chan, a popular site among the far- and alt-right, are likely landing spots if, say, Twitch came up with policies making it harder for white nationalists to share hate on its platform.

Tech companies have done some work to prevent the spread of mass violence. Twitch, Facebook, Discord and Twitter are members of an anti-terrorism industry group that shares digital signatures of violent content to identify and take it down more quickly.

  • Smaller sites like 4chan have not joined the anti-terrorism groups that the bigger companies are part of, and the Buffalo gunman reportedly spent much of his time on the latter two platforms.
  • In addition to stopping the spread of videos, there are some ways Twitch and other platforms can prevent a livestream from beginning in the first place, such as making it more difficult to start accounts and requiring a user to have a certain number of followers to be able to go live.

But without legislation, we’re at an impasse. In the aftermath of the Christchurch shooting, Australia passed a law that will fine social media platforms and jail executives if they don’t quickly remove violent content. It seems unlikely that will happen in the U.S. anytime soon, if ever, given that Congress has neither the time nor the inclination to even begin to tackle the issue.

  • “The legislation lacks teeth, law enforcement have their hands tied behind their backs and lack resources, the platforms lack the will (hate, afterall is profitable), and users are far too often bystanders who simply scroll on by,” Cardiff University professor Matthew Williams, who heads hate crime data hub HateLab, told me.

The question now becomes: How many more mass shootings can we take before something is done? At the moment, it seems there is no limit.

Sarah Roach (email | twitter)

A MESSAGE FROM FOURSQUARE

Data-visualization platforms make complex information and insights easier to understand and ultimately react to. You’ll see companies that adopt data visualization are empowered and can spot emerging trends and speed reaction time.

Learn more

People are talking

Twitter’s gonna close this deal whether Elon Musk likes it or not:

  • “We intend to close the transaction and enforce the merger agreement.”

Raghu Raghuram said VMware is shifting its product delivery to meet customers where they are:

  • "The best delivery model for them is to increasingly deliver our products not as licensed software, but as subscription and SaaS offerings."

Phil Venables said Google Cloud’s built-in security capability is its key strength:

  • “Our whole approach to default security across the platform is important.”

Mandiant’s Kevin Mandia said many cybersecurity firms aren’t needed, but they’ll still do well:

  • “I feel like probably 80% of them shouldn't be there ... [but] there's probably a higher percentage of them that are good investments.”

Making moves

Ian Goodfellow just joined Google’s DeepMind AI division. Goodfellow recently left Apple, where he was a machine-learning director, over the company’s return-to-office policy. Guess DeepMind will let him work from home!

Hasso Plattner is running for a final two-year term on SAP’s supervisory board today. He's the only person to been with SAP over its entire 50-year history, so his departure — even in two years — could leave a leadership vacuum.

Three more top employees left Twitter: VP of product management Ilya Brown, VP of Twitter service Katrina Lane and head of data science Max Schmeiser. All three reportedly chose to leave on their own, but the timing is obviously interesting.

Netflix laid off 150 workers across all departments. The cuts mostly affected U.S. employees, and represent a little over 1% of the company's workforce.

CrowdStrike joined BSA | The Software Alliance, which includes members like Dropbox, Microsoft, Salesforce and others.

Terraform Labs’ legal team resigned, and outside counsel stepped in on legal operations, a source told The Block. The departures come just after Terra’s UST collapsed.

In other news

Jack Dorsey and Elon Musk are buddies, according to a filing sharing details of Musk's deal to buy Twitter. The filing also states that Musk met with Twitter's top leadership for three days just before announcing his intent to buy the company. Just imagine three days of meetings with Musk.

Disney+ wants to keep its ads family-friendly. It won’t allow ads with alcohol or political content, and ads from other streaming services aren’t welcome either.

Apple delayed its return-to-office plans again. The company won’t require employees to head back to the office three days a week just yet, although many were already going a couple times a week.

Tiger Global sold stakes in a bunch of companies including Bumble, Airbnb, Affirm, PayPal and DiDi earlier this year. It also cut its stake in Meta, Amazon and several others.

Robinhood is working on a new appthat will let users hold their crypto and NFTs. People will also be able to use it to scroll through NFT marketplaces and access crypto assets on other exchanges. Good thing crypto is so strong right now, eh.

When was the last time you heard LeBron James tout crypto? The basketball star and other celebrities have appeared in commercials praising crypto, but they've been conspicuously quiet about its flaws in recent weeks.

Blackmail yourself into productivity

If you put money down to get yourself to complete a task, will the risk of losing your hard-earned cash make you more likely to get the work done? Beeminder is betting it will.

Commitment contracts like Beeminder's aren't for everyone. Some people are more motivated to get work done when their money's on the line, while others feel that it could prompt unhealthy behaviors. But for those who use them, companies like Beeminder are testing the limits of what motivates us. Will you give it a shot?

A MESSAGE FROM FOURSQUARE

In order to be successful in Flatiron, a restaurant will need to draw a weekday lunch crowd with healthy offerings and a work-friendly setting for professionals; to stand out among nearly double the restaurants in SoHo, a new restaurant should lean into arts and culture with a design-forward setting.

Learn more

Thoughts, questions, tips? Send them to sourcecode@protocol.com, or our tips line, tips@protocol.com. Enjoy your day, see you tomorrow.

Update: This story was updated to include the anti-terrorism groups Discord is part of.

Recent Issues

The best of Protocol

The confessions of SBF

Your holiday book list

A tale of two FTXs