Power

How COVID-19 helped — and hurt — Facebook’s fight against bad content

The amount of child sexual abuse material Instagram caught and removed fell dramatically, while hate speech removals on Facebook and Instagram grew.

Mark Zuckerberg looking sad

Facebook's data shows that the pandemic made content-moderation systems both better and worse.

Photo: Drew Angerer/Getty Images

When Facebook sent home its content moderators in March due to the COVID-19 pandemic, announcing it would rely on automation to at least temporarily do their job, the company predicted the decision would have a major impact on its ability to find and remove content that violates its policies. Now, according to newly released data from Facebook, we know just how big an impact it had.

During the second quarter of 2020 the company removed less than half of the child sexual abuse material from Instagram that it did the quarter before — not because there was less of it, but because the company was less equipped to catch it. And on both Facebook and Instagram, the amount of suicide and self-injury content it removed dropped precipitously too. On Instagram, it fell from 1.3 million pieces of suicide content removed last quarter to just 275,000 pieces this quarter.

But in other categories, like hate speech, Facebook's new reliance on automated systems actually led to a drastic increase in removals, from just 9.6 million pieces of hate speech removed from Facebook in the beginning of 2020 to 22.5 million pieces removed between April and June.

The drop in the removal of child sexual abuse material from Instagram wasn't due to a decrease in the amount of it on the platform. Neither was the decrease in takedowns of suicide related content. It was due to the limited number of human beings who were available to look at those posts, since, initially at least, they were all working from home. "The reason this content is challenging is because it's graphic content that, at home, is very hard for people to moderate," said Guy Rosen, Facebook's vice president of integrity. "We want to be very careful with the environment that people have in order to look at that content."

It's not that the human reviewers are required to spot all child sexual abuse material. Automated systems are already responsible for removing 97.5% of those types of posts that appear on Facebook. But according to Facebook spokesperson Emily Cain, human reviewers are critical when it comes to "banking" child sexual abuse material. That is, taking known images and logging them so that Facebook's AI systems can then go find and remove them.

"Without humans banking this content then our machines can't find it at scale," Cain said. "And this compounds after a while, so our content-actioned numbers decreased."

"Overall, this pandemic and this situation really reinforced to us that it is always people and technology working together," Rosen said on a call with reporters Tuesday. "We always need people who look and measure and help tune our automation to ensure that we're always up to speed and always up to date with how content is evolving."

The decrease in content removal is a blow to Facebook's ongoing efforts to fight the spread of child sexual abuse material on the platform at a time when the National Center for Missing and Exploited Children says that it's seeing an exponential increase in the number of reports about child exploitation. That said, the company did manage to remove more pieces of child sexual abuse material from the Facebook app than it did last quarter. And yet, overall, in 2020, removals in that category are down significantly from where they were at the end of last year.

During the COVID-19 crisis, Rosen said Facebook has developed a ranking system to prioritize the most critical content in these sensitive categories. That might include anything from a live video to a post in which someone indicates they plan to harm themselves imminently. This ranking system was already in the works before COVID-19, but Rosen said the company expedited its development in response to the crisis.

"This enables our teams to spend their time on the cases where we need their expertise the most, and it means there will be a shift towards more content being initially actioned by our automated systems," Rosen said.

As for the sharp increase in the amount of hate speech being removed from the platform, Rosen attributed that, too, to the ongoing development of Facebook's AI systems. Because hate speech is less graphic than, say, a video of child abuse, moderators are more able to handle that content remotely. As Facebook's chief technology officer Mike Schroepfer told Protocol in a tweet, "The more … sensitive and nuanced the content the more we need help from people."

Of course, the perennial question about hate speech, child sexual abuse material and other types of problematic content is not just how much Facebook is taking down and how fast, but how prevalent that content is to begin with. On the subject of hate speech, that's a question that Facebook hasn't been able to answer yet. Turns out, measuring prevalence requires a lot of human input, too.

Enterprise

Why software releases should be quick but 'palatable and realistic'

Modern software developers release updates much more quickly than in the past, which is great for security and adding new capabilities. But Edith Harbaugh thinks business leaders need a little control of that schedule.

LaunchDarkly was founded in 2014 to help companies manage the software release cycle.

Photo: LaunchDarkly

Gone are the days of quarterly or monthly software update release cycles; today’s software development organizations release updates and fixes on a much more frequent basis. Edith Harbaugh just wants to give business leaders a modicum of control over the process.

The CEO of LaunchDarkly, which was founded in 2014 to help companies manage the software release cycle, is trying to reach customers who want to move fast but understand that moving fast and breaking things won’t work for them. Companies that specialize in continuous integration and continuous delivery services have thrived over the last few years as customers look for help shipping at speed, and LaunchDarkly extends those capabilities to smaller features of existing software.

Keep Reading Show less
Tom Krazit

Tom Krazit ( @tomkrazit) is Protocol's enterprise editor, covering cloud computing and enterprise technology out of the Pacific Northwest. He has written and edited stories about the technology industry for almost two decades for publications such as IDG, CNET, paidContent, and GeekWire, and served as executive editor of Gigaom and Structure.

COVID-19 accelerated what many CEOs and CTOs have struggled to do for the past decade: It forced organizations to be agile and adjust quickly to change. For all the talk about digital transformation over the past decade, when push came to shove, many organizations realized they had made far less progress than they thought.

Now with the genie of rapid change out of the bottle, we will never go back to accepting slow and steady progress from our organizations. To survive and thrive in times of disruption, you need to build a resilient, adaptable business with systems and processes that will keep you nimble for years to come. An essential part of business agility is responding to change by quickly developing new applications and adapting old ones. IT faces an unprecedented demand for new applications. According to IDC, by 2023, more than 500 million digital applications and services will be developed and deployed — the same number of apps that were developed in the last 40 years.[1]

Keep Reading Show less
Denise Broady, CMO, Appian
Denise oversees the Marketing and Communications organization where she is responsible for accelerating the marketing strategy and brand recognition across the globe. Denise has over 24+ years of experience as a change agent scaling businesses from startups, turnarounds and complex software companies. Prior to Appian, Denise worked at SAP, WorkForce Software, TopTier and Clarkston Group. She is also a two-time published author of “GRC for Dummies” and “Driven to Perform.” Denise holds a double degree in marketing and production and operations from Virginia Tech.
Workplace

Building an antiracist company: From idea to practice

Twilio’s chief diversity, inclusion and belonging officer says it’s time for a new approach to DEI.

“The most impactful way to prioritize DEI and enable antiracism is to structure your company accordingly,” says Twilio’s head of DEI Lybra Clemons.

Photo: Twilio

Lybra Clemons is responsible for guiding and scaling inclusion strategy and diversity initiatives at Twilio.

I’ve been in the corporate diversity, equity and inclusion space for over 15 years. In that time, I’ve seen the field evolve slowly from a “nice-to-have” function of Human Resources to a rising company-wide priority. June 2020 was different. Suddenly my and my peers’ phones started ringing off the hook and DEI leaders became the most sought-after professionals. With so many DEI roles being created and corporate willingness to invest, for a split second it looked like there might be real change on the horizon.

Keep Reading Show less
Lybra Clemons
Lybra S. Clemons is a seasoned C-suite executive with over 15 years of Human Resources, Talent and Diversity & Inclusion experience at Fortune 500 companies. She is responsible for guiding and scaling inclusion strategy and diversity initiatives across Twilio's global workforce. Prior to Twilio, Lybra was global head of Diversity & Inclusion at PayPal, where she managed and oversaw all global diversity initiatives. Lybra has held critical roles in Diversity & Inclusion with Morgan Stanley, The Brunswick Group and American Express. She serves on the board of directors of Makers and How Women Lead Silicon Valley Executive Board of Advisers, and has been recognized by Black Enterprise as one of the Top Corporate Women in Diversity.
Boost 2

Can Matt Mullenweg save the internet?

He's turning Automattic into a different kind of tech giant. But can he take on the trillion-dollar walled gardens and give the internet back to the people?

Matt Mullenweg, CEO of Automattic and founder of WordPress, poses for Protocol at his home in Houston, Texas.
Photo: Arturo Olmos for Protocol

In the early days of the pandemic, Matt Mullenweg didn't move to a compound in Hawaii, bug out to a bunker in New Zealand or head to Miami and start shilling for crypto. No, in the early days of the pandemic, Mullenweg bought an RV. He drove it all over the country, bouncing between Houston and San Francisco and Jackson Hole with plenty of stops in national parks. In between, he started doing some tinkering.

The tinkering is a part-time gig: Most of Mullenweg’s time is spent as CEO of Automattic, one of the web’s largest platforms. It’s best known as the company that runs WordPress.com, the hosted version of the blogging platform that powers about 43% of the websites on the internet. Since WordPress is open-source software, no company technically owns it, but Automattic provides tools and services and oversees most of the WordPress-powered internet. It’s also the owner of the booming ecommerce platform WooCommerce, Day One, the analytics tool Parse.ly and the podcast app Pocket Casts. Oh, and Tumblr. And Simplenote. And many others. That makes Mullenweg one of the most powerful CEOs in tech, and one of the most important voices in the debate over the future of the internet.

Keep Reading Show less
David Pierce

David Pierce ( @pierce) is Protocol's editorial director. Prior to joining Protocol, he was a columnist at The Wall Street Journal, a senior writer with Wired, and deputy editor at The Verge. He owns all the phones.

China

Why China is outselling the US in EVs 5 to 1

Electric cars made up 14.8% of Chinese car sales in 2021, compared with 4.1% in the U.S.

Passenger EV sales in China in 2021 jumped 169.1% to nearly 3.3 million from a year ago.

Photo: VCG/VCG via Getty Images

When Tesla entered China in 2014, the country’s EV market was going through a reset. The Austin, Texas-based automaker created a catfish effect — a strong competitor that compels weaker peers to up their game — in China’s EV market for the past few years. Now, Tesla’s sardine-sized Chinese competitors have grown into big fishes in the tank, gradually weakening Tesla’s own prominence in the field.

2021 was a banner year for China’s EV industry. The latest data from the China Passenger Car Association shows that total passenger EV sales in China in 2021 jumped 169.1% from a year ago to nearly 2.99 million: about half of all EVs sold globally. Out of every 100 passenger cars sold in China last year, almost 15 were so-called "new energy vehicles" (NEVs) — a mix of battery-electric vehicles and hybrids.

Keep Reading Show less
Shen Lu

Shen Lu covers China's tech industry.

SKOREA-ENTERTAINMENT-GAMING-MICROSOFT-XBOX
A visitor plays a game using Microsoft's Xbox controller at a flagship store of SK Telecom in Seoul on November 10, 2020. (Photo by Jung Yeon-je / AFP) (Photo by JUNG YEON-JE/AFP via Getty Images)

On this episode of the Source Code podcast: Nick Statt joins the show to discuss Microsoft’s $68.7 billion acquisition of Activision Blizzard, and what it means for the tech and game industries. Then, Issie Lapowsky talks about a big week in antitrust reform, and whether real progress is being made in the U.S. Finally, Hirsh Chitkara explains why AT&T, Verizon, the FAA and airlines have been fighting for months about 5G coverage.

For more on the topics in this episode:

Keep Reading Show less
David Pierce

David Pierce ( @pierce) is Protocol's editorial director. Prior to joining Protocol, he was a columnist at The Wall Street Journal, a senior writer with Wired, and deputy editor at The Verge. He owns all the phones.

Latest Stories
Bulletins