When Tracy Chou decided to host a Reddit "Ask Me Anything" about online harassment over the summer, she knew it probably wouldn't be the easiest experience, but she'd been dealing with trolls for most of her career. How bad could it really be?
A vitriol-filled nightmare, it turns out. The woman hosting a forum on why she was building an app to protect against online harassment was the target of one of the biggest harassment campaigns of her life. Reddit users mocked her inability to answer their questions (she had, but due to a system error, her comments were disappearing before anyone could read them). They ridiculed her appearance and motivations. Someone created a campaign to say nasty things about her on Substack, attaching her name and photo to some of the posts. They moved to Twitter, and then to 4chan, where they organized a group that flooded her site with a denial of service attack until it went down.
It took her weeks to get everything back under control. "I was getting harassed by trying to tell people about this anti-harassment software I'm building. It was awful," she said.
Chou made a name for herself as an outspoken diversity activist making common-sense arguments about why the tech industry needs serious culture change. Now, she's building an app to protect people from online abuse and harassment, inspired by her own experience engendering hate that grew along with her public profile. Her startup, called Block Party, provides a tool to filter trolling and harassing posts, and it launched for public signups last week.
People have asked Chou why she just doesn't stay off Twitter. But instead of allowing harassment to push her off the internet, she wants to feel safe and comfortable online, and that's what Block Party is supposed to help do. For Chou, it seems obvious that people shouldn't be afraid to avail themselves of social platforms just because they have "controversial" ideas or an underrepresented background.
The premise of Block Party is simple. Based on your needs, you set up filters that take messages and comments out of your Twitter experience and into a folder on Block Party, which you can then review when you feel comfortable and prepared. It also has a function for people who need to compile evidence of harassment or stalking (say, for a police report) which allows users to have a friend go through that evidence for them (a tool based on Chou's personal experience; she's filed several police reports in the U.S. and the U.K. about dedicated stalkers who escalated from online harassment).
Chou has big-time ambitions for Block Party. She wants to increase the types of filters people can apply to their accounts. She hopes the app will one day be able to interface with every social media account, not just Twitter, becoming a portable safety net people can take with them to have more control over their online experience. She'd like to eventually build Block Party APIs for new social platforms as they come online, so that they can integrate the tool into their own platforms as a "trust and preferences layer." "This stuff is cross-platform. People move around to harass, but platforms don't collaborate on this stuff," she said.
The idea for Block Party grew out of Chou's early experiences as an engineer. She was among the first 10 employees at Quora and then at Pinterest, and at both places her own need to protect herself from harassing users shaped the products she built. At Quora, one of the first products she shipped was a block button, designed because she needed a way to protect herself from someone targeting her with angry comments on the site. "If you don't have representation or these perspectives in a room, when people are deciding what to work, it gets very influenced by the people who are there. There's no ill intent," she explained.
The typical approach to content moderation is a machine-learning black box that filters content, and human moderators then do the same for everything missed by the system. Because machine learning is not as effective as it needs to be, social media companies rely on huge teams of content moderators that have to make quick decisions about a flood of often traumatizing content. "The whole approach seems pretty flawed to me," Chou told me (she has a master's degree in artificial intelligence). "Machine learning is not going to be the magical solution. People who think ML is going to solve everything are generally men who are enamored with this technology. They don't experience the problem and see how much basic stuff you can do with simple products."
Her approach with Block Party instead considers the very human aspect of what people who are dealing with abuse actually need. "There's a lot of really unpleasant content that a lot of people get, but it's not that people need to be deplatformed. There's a huge margin between bad enough to get deplatformed and stuff I actually want to see. Your Twitter account doesn't need to be taken away because you called me an ugly, skinny Chinese girl, but I don't want to see it," she said.
Her tool gives the consumer control over that content. Rather than delete the comment or deplatform the person who made it, settings could allow a user to filter that comment out of their Twitter experience and into a folder in the Block Party app, which they can then revisit later and review on their own terms. It's about allowing people to have a chance to prepare themselves to deal with whatever someone may or may not say, according to Chou.
Alex Stamos, an early Block Party investor and the director of the Stanford Internet Observatory, used the app recently for just that purpose. After a television appearance discussing the right-wing media ecosystem in late January, a QAnon leader suggested Stamos was a pedophile to his followers, leading to a massive Twitter attack and series of death threats. "I ended up using Block Party, and I blocked over 2,000 people over the weekend, and it was incredibly successful. I had a very tense situation there, and it's the only thing that kept Twitter usable for me. And it made me feel pretty darn good about investing in it," Stamos said.
Though Twitter has its own plans to change and invest in content moderation and user experience, Chou's relationship with the company has actually been fruitful. "It's been interesting to see how many different people within Twitter have reached out to say it's great," she said. "They have acknowledged that as much as they are going to do, they aren't going to be able to solve this problem themselves." And she really believes they are serious about doing it: "There's been a bit of a sea change within Twitter over the last few years."
"There's been a lot of discussion about the broad trust and safety problem at companies, and it's great for the companies to do what they can to improve, but the truth is that there are a number of people who have very intense safety issues and are also willing to take charge of it themselves," Stamos said. He's hoping that over time, Block Party's success will encourage companies to build their APIs for easy interface with tools like Chou's.
While Chou has convinced Twitter of the need for her work, persuading investors has been a far more difficult battle, especially when she encounters groups of all-male or mostly male panels, which tend to be heavily skeptical of any actual demand for Block Party's product.
Along with Stamos, her first investors were people who inherently understood the need for the app: Alexia Bonatsos, the former co-editor of TechCrunch and part of Block Party's target audience, and Charles Hudson, the managing partner and founder of Precursor Ventures, one of the few Black-founded VC firms. "Hudson got it immediately," she said.
Block Party has escaped serious criticism from moderation foes so far because it's "content-neutral," Stamos explained. Because each user decides for themselves what to filter and block, there are no debates about the types of appropriate or "allowed" content like those that plague social media companies. "But there will always be a tension between allowing for open debate and the tools that allow people to block, and that includes tools like Block Party," Stamos said.
While it's been difficult to persuade investors of the need for a product they personally can't relate to, the launch of Block Party's open signups might change that. To participate in the open registration and skip the app's traditional waitlist, people had to pay $8. And they did, increasing the app's number of users by 50% in one week and proving both interest and potential for monetization far beyond Chou's expectations.
"My production intuition around this is pretty good. Sometimes you worry that your own experience doesn't generalize, but it turns out that my experience isn't that unusual," she said. And demand for the app isn't coming just from prominent women on the platform in positions like Chou's, illustrating the ubiquity of online harassment; many of Block Party's recent new users are scientists and doctors facing online hate for their posts about coronavirus.
When I asked Chou what kept her determined to do this work despite the fear of abuse and harassment, she seemed almost confused. "It takes a toll, yes," she said. "It also feels like the only thing I can do to solve it is work on this."
