People

What Tracy Chou learned about online harassment while building an app to solve it

Her new app, Block Party, aims to give people control over harassing content.

What Tracy Chou learned about online harassment while building an app to solve it

Block Party founder and diversity activist Tracy Chou became the target of a Reddit harassment campaign while trying to promote the importance of anti-harassment tools.

Photo: Tracy Chou

When Tracy Chou decided to host a Reddit "Ask Me Anything" about online harassment over the summer, she knew it probably wouldn't be the easiest experience, but she'd been dealing with trolls for most of her career. How bad could it really be?

A vitriol-filled nightmare, it turns out. The woman hosting a forum on why she was building an app to protect against online harassment was the target of one of the biggest harassment campaigns of her life. Reddit users mocked her inability to answer their questions (she had, but due to a system error, her comments were disappearing before anyone could read them). They ridiculed her appearance and motivations. Someone created a campaign to say nasty things about her on Substack, attaching her name and photo to some of the posts. They moved to Twitter, and then to 4chan, where they organized a group that flooded her site with a denial of service attack until it went down.

It took her weeks to get everything back under control. "I was getting harassed by trying to tell people about this anti-harassment software I'm building. It was awful," she said.

Chou made a name for herself as an outspoken diversity activist making common-sense arguments about why the tech industry needs serious culture change. Now, she's building an app to protect people from online abuse and harassment, inspired by her own experience engendering hate that grew along with her public profile. Her startup, called Block Party, provides a tool to filter trolling and harassing posts, and it launched for public signups last week.

People have asked Chou why she just doesn't stay off Twitter. But instead of allowing harassment to push her off the internet, she wants to feel safe and comfortable online, and that's what Block Party is supposed to help do. For Chou, it seems obvious that people shouldn't be afraid to avail themselves of social platforms just because they have "controversial" ideas or an underrepresented background.

The premise of Block Party is simple. Based on your needs, you set up filters that take messages and comments out of your Twitter experience and into a folder on Block Party, which you can then review when you feel comfortable and prepared. It also has a function for people who need to compile evidence of harassment or stalking (say, for a police report) which allows users to have a friend go through that evidence for them (a tool based on Chou's personal experience; she's filed several police reports in the U.S. and the U.K. about dedicated stalkers who escalated from online harassment).

Chou has big-time ambitions for Block Party. She wants to increase the types of filters people can apply to their accounts. She hopes the app will one day be able to interface with every social media account, not just Twitter, becoming a portable safety net people can take with them to have more control over their online experience. She'd like to eventually build Block Party APIs for new social platforms as they come online, so that they can integrate the tool into their own platforms as a "trust and preferences layer." "This stuff is cross-platform. People move around to harass, but platforms don't collaborate on this stuff," she said.

The idea for Block Party grew out of Chou's early experiences as an engineer. She was among the first 10 employees at Quora and then at Pinterest, and at both places her own need to protect herself from harassing users shaped the products she built. At Quora, one of the first products she shipped was a block button, designed because she needed a way to protect herself from someone targeting her with angry comments on the site. "If you don't have representation or these perspectives in a room, when people are deciding what to work, it gets very influenced by the people who are there. There's no ill intent," she explained.

The typical approach to content moderation is a machine-learning black box that filters content, and human moderators then do the same for everything missed by the system. Because machine learning is not as effective as it needs to be, social media companies rely on huge teams of content moderators that have to make quick decisions about a flood of often traumatizing content. "The whole approach seems pretty flawed to me," Chou told me (she has a master's degree in artificial intelligence). "Machine learning is not going to be the magical solution. People who think ML is going to solve everything are generally men who are enamored with this technology. They don't experience the problem and see how much basic stuff you can do with simple products."

Her approach with Block Party instead considers the very human aspect of what people who are dealing with abuse actually need. "There's a lot of really unpleasant content that a lot of people get, but it's not that people need to be deplatformed. There's a huge margin between bad enough to get deplatformed and stuff I actually want to see. Your Twitter account doesn't need to be taken away because you called me an ugly, skinny Chinese girl, but I don't want to see it," she said.

Her tool gives the consumer control over that content. Rather than delete the comment or deplatform the person who made it, settings could allow a user to filter that comment out of their Twitter experience and into a folder in the Block Party app, which they can then revisit later and review on their own terms. It's about allowing people to have a chance to prepare themselves to deal with whatever someone may or may not say, according to Chou.

Alex Stamos, an early Block Party investor and the director of the Stanford Internet Observatory, used the app recently for just that purpose. After a television appearance discussing the right-wing media ecosystem in late January, a QAnon leader suggested Stamos was a pedophile to his followers, leading to a massive Twitter attack and series of death threats. "I ended up using Block Party, and I blocked over 2,000 people over the weekend, and it was incredibly successful. I had a very tense situation there, and it's the only thing that kept Twitter usable for me. And it made me feel pretty darn good about investing in it," Stamos said.

Though Twitter has its own plans to change and invest in content moderation and user experience, Chou's relationship with the company has actually been fruitful. "It's been interesting to see how many different people within Twitter have reached out to say it's great," she said. "They have acknowledged that as much as they are going to do, they aren't going to be able to solve this problem themselves." And she really believes they are serious about doing it: "There's been a bit of a sea change within Twitter over the last few years."

"There's been a lot of discussion about the broad trust and safety problem at companies, and it's great for the companies to do what they can to improve, but the truth is that there are a number of people who have very intense safety issues and are also willing to take charge of it themselves," Stamos said. He's hoping that over time, Block Party's success will encourage companies to build their APIs for easy interface with tools like Chou's.

While Chou has convinced Twitter of the need for her work, persuading investors has been a far more difficult battle, especially when she encounters groups of all-male or mostly male panels, which tend to be heavily skeptical of any actual demand for Block Party's product.

Along with Stamos, her first investors were people who inherently understood the need for the app: Alexia Bonatsos, the former co-editor of TechCrunch and part of Block Party's target audience, and Charles Hudson, the managing partner and founder of Precursor Ventures, one of the few Black-founded VC firms. "Hudson got it immediately," she said.

Block Party has escaped serious criticism from moderation foes so far because it's "content-neutral," Stamos explained. Because each user decides for themselves what to filter and block, there are no debates about the types of appropriate or "allowed" content like those that plague social media companies. "But there will always be a tension between allowing for open debate and the tools that allow people to block, and that includes tools like Block Party," Stamos said.

While it's been difficult to persuade investors of the need for a product they personally can't relate to, the launch of Block Party's open signups might change that. To participate in the open registration and skip the app's traditional waitlist, people had to pay $8. And they did, increasing the app's number of users by 50% in one week and proving both interest and potential for monetization far beyond Chou's expectations.

"My production intuition around this is pretty good. Sometimes you worry that your own experience doesn't generalize, but it turns out that my experience isn't that unusual," she said. And demand for the app isn't coming just from prominent women on the platform in positions like Chou's, illustrating the ubiquity of online harassment; many of Block Party's recent new users are scientists and doctors facing online hate for their posts about coronavirus.

When I asked Chou what kept her determined to do this work despite the fear of abuse and harassment, she seemed almost confused. "It takes a toll, yes," she said. "It also feels like the only thing I can do to solve it is work on this."

Protocol | Fintech

How European fintech startup N26 is preparing for U.S. regulations

"There's a lot more scrutiny being placed on fintech. We are definitely mindful of it."

Photo: N26

N26's monster $900 million funding round announced Monday underlined the German startup's momentum in the digital banking market.

Stephanie Balint, N26's U.S. general manager, said the funding will be used for expansion and also to improve "our core offering to make this the most reliable bank that our customers can trust," she told Protocol.

Keep Reading Show less
Benjamin Pimentel

Benjamin Pimentel ( @benpimentel) covers fintech from San Francisco. He has reported on many of the biggest tech stories over the past 20 years for the San Francisco Chronicle, Dow Jones MarketWatch and Business Insider, from the dot-com crash, the rise of cloud computing, social networking and AI to the impact of the Great Recession and the COVID crisis on Silicon Valley and beyond. He can be reached at bpimentel@protocol.com or via Signal at (510)731-8429.

The way we work has fundamentally changed. COVID-19 upended business dealings and office work processes, putting into hyperdrive a move towards digital collaboration platforms that allow teams to streamline processes and communicate from anywhere. According to the International Data Corporation, the revenue for worldwide collaboration applications increased 32.9 percent from 2019 to 2020, reaching $22.6 billion; it's expected to become a $50.7 billion industry by 2025.

"While consumers and early adopter businesses had widely embraced collaborative applications prior to the pandemic, the market saw five years' worth of new users in the first six months of 2020," said Wayne Kurtzman, research director of social and collaboration at IDC. "This has cemented collaboration, at least to some extent, for every business, large and small."

Keep Reading Show less
Kate Silver

Kate Silver is an award-winning reporter and editor with 15-plus years of journalism experience. Based in Chicago, she specializes in feature and business reporting. Kate's reporting has appeared in the Washington Post, The Chicago Tribune, The Atlantic's CityLab, Atlas Obscura, The Telegraph and many other outlets.

Apple’s new MacBooks are the future — and the past

After years of reinventing the wheel, Apple's back to just building really good ones.

Apple brought back the ports.

Photo: Apple

The 2015 Pro was, by most accounts, one of the best laptops Apple ever made. It was fast and functional, and it had a great screen, a MagSafe charger, plenty of ports, a great keyboard and solid battery life. If you walked around practically any office in Silicon Valley, you'd see Pros everywhere.

Many of those users have been holding on to their increasingly old and dusty 2015 Pros, too, because right about when that computer came out was when Apple seemed to lose its way in the laptop market. It released the 12-inch MacBook, an incredibly thin and light computer that made a bunch of changes — a new keyboard and trackpad design chief among them — that eventually made their way around the rest of the MacBook lineup. Then came the Touch Bar, Apple's attempt to build an entirely new user interface into a laptop.

Keep Reading Show less
David Pierce

David Pierce ( @pierce) is Protocol's editorial director. Prior to joining Protocol, he was a columnist at The Wall Street Journal, a senior writer with Wired, and deputy editor at The Verge. He owns all the phones.

Protocol spoke to founders and tech execs who've embraced async and have tips on how to get started.

Image: Christopher T. Fong/Protocol

Imagine a company where there are no meetings — just time for deep, focused work punctuated by short conversations on Slack and project updates on Trello.

Now imagine a company where the no-meeting ethos is so ingrained that it's possible to work there for 10 years without ever speaking face-to-face with a single coworker, and for your boss to not even recognize the sound of your voice.

Keep Reading Show less
Michelle Ma
Michelle Ma (@himichellema) is a reporter at Protocol, where she writes about management, leadership and workplace issues in tech. Previously, she was a news editor of live journalism and special coverage for The Wall Street Journal. Prior to that, she worked as a staff writer at Wirecutter. She can be reached at mma@protocol.com.
Protocol | Workplace

#AppleToo activist says Apple fired her for deleting apps from her devices

Janneke Parrish says she was dismissed after deleting Robinhood, Pokemon Go and Google Drive from her work devices during an investigation inside the company.

The Apple Too movement is trying to organize Apple workers into a collective movement.
Photo: Bloomberg via Getty

Unlike most other companies, Apple asks that its employees use their work phones like personal ones — and for five years, Apple program manager Janneke Parrish did as she was told. But last week, when Apple asked Parrish for her devices in an internal investigation, she was afraid Apple would see her personal and private information. She disobeyed orders and deleted apps like Robinhood, Pokemon Go and Google Drive. Then Apple fired her.

Keep Reading Show less
Anna Kramer

Anna Kramer is a reporter at Protocol (Twitter: @ anna_c_kramer, email: akramer@protocol.com), where she writes about labor and workplace issues. Prior to joining the team, she covered tech and small business for the San Francisco Chronicle and privacy for Bloomberg Law. She is a recent graduate of Brown University, where she studied International Relations and Arabic and wrote her senior thesis about surveillance tools and technological development in the Middle East.

Latest Stories