Power

Twitter's image cropping was biased, so it dumped the algorithm

The META team found the Twitter cropping algorithm occasionally discriminated based on race and gender.

Twitter

Twitter is abandoning its image cropping algorithm after finding racial and gender-based bias.

Photo: Joshua Hoehne/Unsplash

In one of the first instances of total algorithmic transparency from a social platform, Twitter has removed its image cropping algorithm and created new controls for users over how images appear, based on newly published findings of gender and skin color biases.

Members of the Twitter Machine Learning, Ethics and Algorithmic Transparency team conducted a research experiment to review how the image-cropping algorithm chose points of focus in pictures (called "saliency") and found occasional instances of bias in cropping based on the gender and skin color of people in photos. The researchers found that there was an 8% difference in favor of women from demographic parity, a 4% difference in favor of white individuals, and a 7% difference in favor of white women in comparisons of black and white women, META lead Rumman Chowdhury wrote in a blog post Wednesday afternoon. The research found other biases as well. (The company's analysis used subjective applications of "Black," "white", "male", and "female", not the total spectrum of skin tone and racial and ethnic identities).

After analyzing the tradeoffs between the consistency of the algorithm and its potential for harm, Twitter opted away from speed — an unusual, if unheard of, policy change for social networks that usually obsess with reducing friction — and instead designed a new way to share vertical, uncropped images. "This update also includes a true preview of the image in the Tweet composer field, so Tweet authors know how their Tweets will look before they publish. This release reduces our dependency on ML for a function that we agree is best performed by people using our products," Chowdhury wrote.

Twitter first introduced the "saliency" cropping algorithm in 2018 in order to make timeline photos appear more consistent, training the algorithm to crop based on how the human eye sees pictures. The cropping almost immediately drew criticism by people who felt that from their own anecdotal experience, the crops often focused on people with light skin and on women while cutting out others, which could perpetuate both demographic biases and the objectifying "male gaze" toward women. While Twitter at first said the algorithm had been tested for biases before it was released, the company then apologized in September 2020 to the users who were sharing their own experiences with bias on the platform and committed to further assessing the algorithm.

Over the last year, Twitter has heavily emphasized user choice in its public statements about conversational health and safety and in its descriptions of how the company is designing future products. The team did the same for its analysis of the cropping algorithm, asking whether the cropping algorithm caused harm by taking away people's ability to make their own decisions. "Not everything on Twitter is a good candidate for an algorithm, and in this case, how to crop an image is a decision best made by people. They should be able to decide for themselves what part of an image is the most important and what part is the focal point of their Tweet," Chowdhury wrote.

The company also shared a paper published today detailing how the researchers found the biases within the algorithms and how they analyzed the possible harm that could be caused by them, written by Kyra Yee and Tao Tantipongpipat from the META team and Shubhanshu Mishra from the Content Understanding Research team. "The use of this model poses concerns that Twitter's cropping system favors cropping light-skinned over dark-skinned individuals and favors cropping women's bodies over their heads," they wrote in the paper's conclusion.

Chowdhury, a widely-respected leader in the field of algorithmic accountability, was hired in February to lead the new META team after months of widespread criticism toward large tech companies and social platforms for their failures to analyze possible biases and discrimination in their algorithms. She is just one of Twitter's many recent hires of prominent and well-respected leaders in tech ethics and accountability.

Can we bring malls into the metaverse?

Soon you might buy digital sneakers to wear on your digital date in a digital world.

Combined with the hype around digital goods and cryptocurrency, companies and futurists are starting to imagine what shopping in the metaverse might look like.

Photo illustration: Mark Abramson/Bloomberg via Getty Images; The Fabricant; Protocol

Click banner image for more Shopping Week coverage

Before the internet, the mall was the spot for watching movies, hanging out, listening to music, finding love — and an embodiment of all-American consumerism. "The shopping center was Amazon, it was Facebook, it was Tinder, it was Spotify, it was Netflix," said retail futurist Doug Stephens. "It was the gathering point in the community."

Keep Reading Show less
Lizzy Lawrence

Lizzy Lawrence ( @LizzyLaw_) is a reporter at Protocol, covering tools and productivity in the workplace. She's a recent graduate of the University of Michigan, where she studied sociology and international studies. She served as editor in chief of The Michigan Daily, her school's independent newspaper. She's based in D.C., and can be reached at llawrence@protocol.com.

The Bureau of Labor Statistics indicates that by 2026, the shortage of engineers in the U.S. will exceed 1.2 million, while 545,000 software developers will have left the market by that time. Meanwhile, business is becoming increasingly more digital-first, and teams need the tools in place to keep distributed teams aligned and able to respond quickly to changing business needs. That means businesses need to build powerful workplace applications without relying on developers.

In fact, according to Gartner, by 2025, 70% of new applications developed by enterprises will use low-code or no-code technologies and, by 2023, there will be at least four times as many active citizen developers as professional developers at large enterprises. We're on the cusp of a big shift in how businesses operate and how organization wide innovation happens.

Keep Reading Show less
Andrew Ofstad
As Airtable’s co-founder, Andrew spearheads Airtable’s long-term product bets and represents the voice of the customer in major product decisions. After co-founding the company, he helped scale Airtable’s original product and engineering teams. He previously led the redesign of Google's flagship Maps product, and before that was a product manager for Android.

Can we bring malls into the metaverse?

Soon you might buy digital sneakers to wear on your digital date in a digital world.

Combined with the hype around digital goods and cryptocurrency, companies and futurists are starting to imagine what shopping in the metaverse might look like.

Photo illustration: Mark Abramson/Bloomberg via Getty Images; The Fabricant; Protocol

Click banner image for more Shopping Week coverage

Before the internet, the mall was the spot for watching movies, hanging out, listening to music, finding love — and an embodiment of all-American consumerism. "The shopping center was Amazon, it was Facebook, it was Tinder, it was Spotify, it was Netflix," said retail futurist Doug Stephens. "It was the gathering point in the community."

Keep Reading Show less
Lizzy Lawrence

Lizzy Lawrence ( @LizzyLaw_) is a reporter at Protocol, covering tools and productivity in the workplace. She's a recent graduate of the University of Michigan, where she studied sociology and international studies. She served as editor in chief of The Michigan Daily, her school's independent newspaper. She's based in D.C., and can be reached at llawrence@protocol.com.

It’s still too hard to give crypto this holiday

Here's how the industry Scrooged itself out of a big opportunity.

If only giving bitcoin were this simple.

Photo illustration: gpointstudio/Getty Images Plus; Protocol

Click banner image for more Shopping Week coverage

Crypto took huge steps toward the mainstream this year. Bitcoin soared in value, Coinbase went public and VCs poured even more money into the industry. In case consumers didn't get the message, they'll surely notice when the Staples Center turns into Crypto.com Arena next month and FTX airs its first Super Bowl ad in February.

Keep Reading Show less
Benjamin Pimentel

Benjamin Pimentel ( @benpimentel) covers fintech from San Francisco. He has reported on many of the biggest tech stories over the past 20 years for the San Francisco Chronicle, Dow Jones MarketWatch and Business Insider, from the dot-com crash, the rise of cloud computing, social networking and AI to the impact of the Great Recession and the COVID crisis on Silicon Valley and beyond. He can be reached at bpimentel@protocol.com or via Signal at (510)731-8429.

It’s time to rethink Black Friday

The pandemic didn't end Black Friday, but it'll never look the same again.

We can expect Black Friday to stick around but lose relevance as retailers effectively dilute its meaning and purpose.

Illustration: Christopher T. Fong/Protocol

Click banner image for more Shopping Week coverage

"I'm selling meditation, so I shouldn't be stressed," said Charlie Rousset, the co-founder of sleep and relaxation gadget-maker Morphée. But even deep breathing can't help Rousset feel less on edge this Black Friday.

Keep Reading Show less
Janko Roettgers

Janko Roettgers (@jank0) is a senior reporter at Protocol, reporting on the shifting power dynamics between tech, media, and entertainment, including the impact of new technologies. Previously, Janko was Variety's first-ever technology writer in San Francisco, where he covered big tech and emerging technologies. He has reported for Gigaom, Frankfurter Rundschau, Berliner Zeitung, and ORF, among others. He has written three books on consumer cord-cutting and online music and co-edited an anthology on internet subcultures. He lives with his family in Oakland.

Latest Stories