Source Code: Your daily look at what matters in tech.

source-codesource codeauthorIssie LapowskyNoneWant your finger on the pulse of everything that's happening in tech? Sign up to get David Pierce's daily newsletter.64fd3cbe9f
×

Get access to Protocol

Your information will be used in accordance with our Privacy Policy

I’m already a subscriber
Power

How COVID-19 helped — and hurt — Facebook’s fight against bad content

The amount of child sexual abuse material Instagram caught and removed fell dramatically, while hate speech removals on Facebook and Instagram grew.

Mark Zuckerberg looking sad

Facebook's data shows that the pandemic made content-moderation systems both better and worse.

Photo: Drew Angerer/Getty Images

When Facebook sent home its content moderators in March due to the COVID-19 pandemic, announcing it would rely on automation to at least temporarily do their job, the company predicted the decision would have a major impact on its ability to find and remove content that violates its policies. Now, according to newly released data from Facebook, we know just how big an impact it had.

During the second quarter of 2020 the company removed less than half of the child sexual abuse material from Instagram that it did the quarter before — not because there was less of it, but because the company was less equipped to catch it. And on both Facebook and Instagram, the amount of suicide and self-injury content it removed dropped precipitously too. On Instagram, it fell from 1.3 million pieces of suicide content removed last quarter to just 275,000 pieces this quarter.

But in other categories, like hate speech, Facebook's new reliance on automated systems actually led to a drastic increase in removals, from just 9.6 million pieces of hate speech removed from Facebook in the beginning of 2020 to 22.5 million pieces removed between April and June.

The drop in the removal of child sexual abuse material from Instagram wasn't due to a decrease in the amount of it on the platform. Neither was the decrease in takedowns of suicide related content. It was due to the limited number of human beings who were available to look at those posts, since, initially at least, they were all working from home. "The reason this content is challenging is because it's graphic content that, at home, is very hard for people to moderate," said Guy Rosen, Facebook's vice president of integrity. "We want to be very careful with the environment that people have in order to look at that content."

It's not that the human reviewers are required to spot all child sexual abuse material. Automated systems are already responsible for removing 97.5% of those types of posts that appear on Facebook. But according to Facebook spokesperson Emily Cain, human reviewers are critical when it comes to "banking" child sexual abuse material. That is, taking known images and logging them so that Facebook's AI systems can then go find and remove them.

"Without humans banking this content then our machines can't find it at scale," Cain said. "And this compounds after a while, so our content-actioned numbers decreased."

"Overall, this pandemic and this situation really reinforced to us that it is always people and technology working together," Rosen said on a call with reporters Tuesday. "We always need people who look and measure and help tune our automation to ensure that we're always up to speed and always up to date with how content is evolving."

The decrease in content removal is a blow to Facebook's ongoing efforts to fight the spread of child sexual abuse material on the platform at a time when the National Center for Missing and Exploited Children says that it's seeing an exponential increase in the number of reports about child exploitation. That said, the company did manage to remove more pieces of child sexual abuse material from the Facebook app than it did last quarter. And yet, overall, in 2020, removals in that category are down significantly from where they were at the end of last year.

During the COVID-19 crisis, Rosen said Facebook has developed a ranking system to prioritize the most critical content in these sensitive categories. That might include anything from a live video to a post in which someone indicates they plan to harm themselves imminently. This ranking system was already in the works before COVID-19, but Rosen said the company expedited its development in response to the crisis.

"This enables our teams to spend their time on the cases where we need their expertise the most, and it means there will be a shift towards more content being initially actioned by our automated systems," Rosen said.

As for the sharp increase in the amount of hate speech being removed from the platform, Rosen attributed that, too, to the ongoing development of Facebook's AI systems. Because hate speech is less graphic than, say, a video of child abuse, moderators are more able to handle that content remotely. As Facebook's chief technology officer Mike Schroepfer told Protocol in a tweet, "The more … sensitive and nuanced the content the more we need help from people."

Of course, the perennial question about hate speech, child sexual abuse material and other types of problematic content is not just how much Facebook is taking down and how fast, but how prevalent that content is to begin with. On the subject of hate speech, that's a question that Facebook hasn't been able to answer yet. Turns out, measuring prevalence requires a lot of human input, too.

Protocol | Workplace

The pay gap persists for Black women

"The pay gap is a multifaceted problem and any time you have a complex problem, there's not a single solution that's going to solve it."

For every dollar paid to white, non-Hispanic men, Black women are paid just 63 cents, according to the American Community Survey Census data.

Photo: Christine/Unsplash

Last year's racial reckoning following the murder of George Floyd led many tech companies to commit to promoting equity within their organizations, including working toward pay equity. But despite efforts, the wage gap for Black women still persists. For every dollar paid to white, non-Hispanic men, Black women are paid just 63 cents, according to the American Community Survey Census data.

Black Women's Equal Pay Day on Tuesday represents the estimated number of days into the year it would take for Black women to make what their white, non-Hispanic male counterparts made at the end of the previous year, according to the organization Equal Pay Today. And while the responsibility to fix the pay gap falls mostly on companies to rectify, some female employees have taken matters into their own hands and held companies to their asserted values by negotiating higher pay.

Keep Reading Show less
Amber Burton

Amber Burton (@amberbburton) is a reporter at Protocol. Previously, she covered personal finance and diversity in business at The Wall Street Journal. She earned an M.S. in Strategic Communications from Columbia University and B.A. in English and Journalism from Wake Forest University. She lives in North Carolina.

pay

What comes to mind when you think of AI? In the past, it might have been the Turing test, a sci-fi character or IBM's Deep Blue-defeating chess champion Garry Kasparov. Today, instead of copying human intelligence, we're seeing immense progress made in using AI to unobtrusively simplify and enrich our own intelligence and experiences. Natural language processing, modern encrypted security solutions, advanced perception and imaging capabilities, next-generation data management and logistics, and automotive assistance are some of the many ways AI is quietly yet unmistakably driving some of the latest advancements inside our phones, PCs, cars and other crucial 21st century devices. And the combination of 5G and AI is enabling a world with distributed intelligence where AI processing is happening on devices and in the cloud.

Keep Reading Show less
Alex Katouzian
Alex Katouzian currently serves as senior vice president and general manager of the Mobile, Compute and Infrastructure (MCI) Business Unit at Qualcomm Technologies, Inc. In this role, Katouzian is responsible for the profit, loss and strategy of the MCI BU, which includes business lines for Mobile Handset Products and Application Processor Technologies, 4G and 5G Mobile Broadband for embedded applications, Small and Macro Cells, Modem Technologies, Compute products across multiple OS’, eXtended Reality and AI Edge Cloud products.
Protocol | Workplace

Tech company hybrid work policies are becoming more flexible, not less

Twitter, LinkedIn and Asana are already changing their hybrid policies to allow for more flexibility.

Photo: FG Trade/Getty Images

Twitter, LinkedIn and Asana are all loosening up their strategies around hybrid work, allowing for more flexibility before even fully reopening their offices.

In the last week and a half, Twitter announced it's adopting an asynchronous-first approach, and both Asana and LinkedIn said they would increase the amount of time their employees can work remotely.

Keep Reading Show less
Allison Levitsky
Allison Levitsky is a reporter at Protocol covering workplace issues in tech. She previously covered big tech companies and the tech workforce for the Silicon Valley Business Journal. Allison grew up in the Bay Area and graduated from UC Berkeley.
Power

Activision Blizzard scrambles to repair its toxic image

Blizzard President J. Allen Brack is the first executive to depart amid the sexual harassment crisis.

Activision Blizzard doesn't seem committed to lasting change.

Photo: Allen J. Schaben/Getty Images

As Activision Blizzard's workplace crisis rages on into its third week, the company is taking measures to try to calm the storm — to little avail. On Tuesday, Blizzard President J. Allen Brack, who took the reins at the developer responsible for World of Warcraft back in 2018, resigned. He's to be replaced by executives Jen Oneal and Mike Ybarra, who will co-lead the studio in a power-sharing agreement some believe further solidifies CEO Bobby Kotick's control over the subsidiary.

Nowhere in Blizzard's statement about Brack's departure does it mention California's explosive sexual harassment and discrimination lawsuit at the heart of the saga. The lawsuit, filed last month, resulted last week in a 500-person walkout at Blizzard's headquarters in Irvine. (Among the attendees was none other than Ybarra, the new studio co-head.)

Keep Reading Show less
Nick Statt
Nick Statt is Protocol's video game reporter. Prior to joining Protocol, he was news editor at The Verge covering the gaming industry, mobile apps and antitrust out of San Francisco, in addition to managing coverage of Silicon Valley tech giants and startups. He now resides in Rochester, New York, home of the garbage plate and, completely coincidentally, the World Video Game Hall of Fame. He can be reached at nstatt@protocol.com.
Protocol | Workplace

Alabama Amazon workers will likely get a second union vote

An NLRB judge said that Amazon "usurped" the NLRB by pushing for a mailbox to be installed in front of its facility, and also that the company violated laws that protect workers from monitoring of their behavior during union elections.

An NLRB judge ruled that Amazon has violated union election rules

Image: Amazon

Bessemer, Alabama warehouse workers will likely get a second union vote because of Amazon's efforts to have a USPS ballot box installed just outside of the Bessemer warehouse facility during the mail-in vote, as well as other violations of union vote rules, according to an NLRB ruling published Tuesday morning.

While union organizers, represented by the Retail, Wholesale, and Department Store Union, lost the first vote by more than a 2:1 margin, a second election will be scheduled and held unless Amazon successfully appeals the ruling. Though Amazon is the country's second-largest private employer, no unionization effort at the company has ever been successful.

Keep Reading Show less
Anna Kramer

Anna Kramer is a reporter at Protocol (Twitter: @ anna_c_kramer, email: akramer@protocol.com), where she writes about labor and workplace issues. Prior to joining the team, she covered tech and small business for the San Francisco Chronicle and privacy for Bloomberg Law. She is a recent graduate of Brown University, where she studied International Relations and Arabic and wrote her senior thesis about surveillance tools and technological development in the Middle East.

Latest Stories