People

What Tracy Chou learned about online harassment while building an app to solve it

Her new app, Block Party, aims to give people control over harassing content.

What Tracy Chou learned about online harassment while building an app to solve it

Block Party founder and diversity activist Tracy Chou became the target of a Reddit harassment campaign while trying to promote the importance of anti-harassment tools.

Photo: Tracy Chou

When Tracy Chou decided to host a Reddit "Ask Me Anything" about online harassment over the summer, she knew it probably wouldn't be the easiest experience, but she'd been dealing with trolls for most of her career. How bad could it really be?

A vitriol-filled nightmare, it turns out. The woman hosting a forum on why she was building an app to protect against online harassment was the target of one of the biggest harassment campaigns of her life. Reddit users mocked her inability to answer their questions (she had, but due to a system error, her comments were disappearing before anyone could read them). They ridiculed her appearance and motivations. Someone created a campaign to say nasty things about her on Substack, attaching her name and photo to some of the posts. They moved to Twitter, and then to 4chan, where they organized a group that flooded her site with a denial of service attack until it went down.

It took her weeks to get everything back under control. "I was getting harassed by trying to tell people about this anti-harassment software I'm building. It was awful," she said.

Chou made a name for herself as an outspoken diversity activist making common-sense arguments about why the tech industry needs serious culture change. Now, she's building an app to protect people from online abuse and harassment, inspired by her own experience engendering hate that grew along with her public profile. Her startup, called Block Party, provides a tool to filter trolling and harassing posts, and it launched for public signups last week.

People have asked Chou why she just doesn't stay off Twitter. But instead of allowing harassment to push her off the internet, she wants to feel safe and comfortable online, and that's what Block Party is supposed to help do. For Chou, it seems obvious that people shouldn't be afraid to avail themselves of social platforms just because they have "controversial" ideas or an underrepresented background.

The premise of Block Party is simple. Based on your needs, you set up filters that take messages and comments out of your Twitter experience and into a folder on Block Party, which you can then review when you feel comfortable and prepared. It also has a function for people who need to compile evidence of harassment or stalking (say, for a police report) which allows users to have a friend go through that evidence for them (a tool based on Chou's personal experience; she's filed several police reports in the U.S. and the U.K. about dedicated stalkers who escalated from online harassment).

Chou has big-time ambitions for Block Party. She wants to increase the types of filters people can apply to their accounts. She hopes the app will one day be able to interface with every social media account, not just Twitter, becoming a portable safety net people can take with them to have more control over their online experience. She'd like to eventually build Block Party APIs for new social platforms as they come online, so that they can integrate the tool into their own platforms as a "trust and preferences layer." "This stuff is cross-platform. People move around to harass, but platforms don't collaborate on this stuff," she said.

The idea for Block Party grew out of Chou's early experiences as an engineer. She was among the first 10 employees at Quora and then at Pinterest, and at both places her own need to protect herself from harassing users shaped the products she built. At Quora, one of the first products she shipped was a block button, designed because she needed a way to protect herself from someone targeting her with angry comments on the site. "If you don't have representation or these perspectives in a room, when people are deciding what to work, it gets very influenced by the people who are there. There's no ill intent," she explained.

The typical approach to content moderation is a machine-learning black box that filters content, and human moderators then do the same for everything missed by the system. Because machine learning is not as effective as it needs to be, social media companies rely on huge teams of content moderators that have to make quick decisions about a flood of often traumatizing content. "The whole approach seems pretty flawed to me," Chou told me (she has a master's degree in artificial intelligence). "Machine learning is not going to be the magical solution. People who think ML is going to solve everything are generally men who are enamored with this technology. They don't experience the problem and see how much basic stuff you can do with simple products."

Her approach with Block Party instead considers the very human aspect of what people who are dealing with abuse actually need. "There's a lot of really unpleasant content that a lot of people get, but it's not that people need to be deplatformed. There's a huge margin between bad enough to get deplatformed and stuff I actually want to see. Your Twitter account doesn't need to be taken away because you called me an ugly, skinny Chinese girl, but I don't want to see it," she said.

Her tool gives the consumer control over that content. Rather than delete the comment or deplatform the person who made it, settings could allow a user to filter that comment out of their Twitter experience and into a folder in the Block Party app, which they can then revisit later and review on their own terms. It's about allowing people to have a chance to prepare themselves to deal with whatever someone may or may not say, according to Chou.

Alex Stamos, an early Block Party investor and the director of the Stanford Internet Observatory, used the app recently for just that purpose. After a television appearance discussing the right-wing media ecosystem in late January, a QAnon leader suggested Stamos was a pedophile to his followers, leading to a massive Twitter attack and series of death threats. "I ended up using Block Party, and I blocked over 2,000 people over the weekend, and it was incredibly successful. I had a very tense situation there, and it's the only thing that kept Twitter usable for me. And it made me feel pretty darn good about investing in it," Stamos said.

Though Twitter has its own plans to change and invest in content moderation and user experience, Chou's relationship with the company has actually been fruitful. "It's been interesting to see how many different people within Twitter have reached out to say it's great," she said. "They have acknowledged that as much as they are going to do, they aren't going to be able to solve this problem themselves." And she really believes they are serious about doing it: "There's been a bit of a sea change within Twitter over the last few years."

"There's been a lot of discussion about the broad trust and safety problem at companies, and it's great for the companies to do what they can to improve, but the truth is that there are a number of people who have very intense safety issues and are also willing to take charge of it themselves," Stamos said. He's hoping that over time, Block Party's success will encourage companies to build their APIs for easy interface with tools like Chou's.

While Chou has convinced Twitter of the need for her work, persuading investors has been a far more difficult battle, especially when she encounters groups of all-male or mostly male panels, which tend to be heavily skeptical of any actual demand for Block Party's product.

Along with Stamos, her first investors were people who inherently understood the need for the app: Alexia Bonatsos, the former co-editor of TechCrunch and part of Block Party's target audience, and Charles Hudson, the managing partner and founder of Precursor Ventures, one of the few Black-founded VC firms. "Hudson got it immediately," she said.

Block Party has escaped serious criticism from moderation foes so far because it's "content-neutral," Stamos explained. Because each user decides for themselves what to filter and block, there are no debates about the types of appropriate or "allowed" content like those that plague social media companies. "But there will always be a tension between allowing for open debate and the tools that allow people to block, and that includes tools like Block Party," Stamos said.

While it's been difficult to persuade investors of the need for a product they personally can't relate to, the launch of Block Party's open signups might change that. To participate in the open registration and skip the app's traditional waitlist, people had to pay $8. And they did, increasing the app's number of users by 50% in one week and proving both interest and potential for monetization far beyond Chou's expectations.

"My production intuition around this is pretty good. Sometimes you worry that your own experience doesn't generalize, but it turns out that my experience isn't that unusual," she said. And demand for the app isn't coming just from prominent women on the platform in positions like Chou's, illustrating the ubiquity of online harassment; many of Block Party's recent new users are scientists and doctors facing online hate for their posts about coronavirus.

When I asked Chou what kept her determined to do this work despite the fear of abuse and harassment, she seemed almost confused. "It takes a toll, yes," she said. "It also feels like the only thing I can do to solve it is work on this."

Fintech

Judge Zia Faruqui is trying to teach you crypto, one ‘SNL’ reference at a time

His decisions on major cryptocurrency cases have quoted "The Big Lebowski," "SNL," and "Dr. Strangelove." That’s because he wants you — yes, you — to read them.

The ways Zia Faruqui (right) has weighed on cases that have come before him can give lawyers clues as to what legal frameworks will pass muster.

Photo: Carolyn Van Houten/The Washington Post via Getty Images

“Cryptocurrency and related software analytics tools are ‘The wave of the future, Dude. One hundred percent electronic.’”

That’s not a quote from "The Big Lebowski" — at least, not directly. It’s a quote from a Washington, D.C., district court memorandum opinion on the role cryptocurrency analytics tools can play in government investigations. The author is Magistrate Judge Zia Faruqui.

Keep ReadingShow less
Veronica Irwin

Veronica Irwin (@vronirwin) is a San Francisco-based reporter at Protocol covering fintech. Previously she was at the San Francisco Examiner, covering tech from a hyper-local angle. Before that, her byline was featured in SF Weekly, The Nation, Techworker, Ms. Magazine and The Frisc.

The financial technology transformation is driving competition, creating consumer choice, and shaping the future of finance. Hear from seven fintech leaders who are reshaping the future of finance, and join the inaugural Financial Technology Association Fintech Summit to learn more.

Keep ReadingShow less
FTA
The Financial Technology Association (FTA) represents industry leaders shaping the future of finance. We champion the power of technology-centered financial services and advocate for the modernization of financial regulation to support inclusion and responsible innovation.
Enterprise

AWS CEO: The cloud isn’t just about technology

As AWS preps for its annual re:Invent conference, Adam Selipsky talks product strategy, support for hybrid environments, and the value of the cloud in uncertain economic times.

Photo: Noah Berger/Getty Images for Amazon Web Services

AWS is gearing up for re:Invent, its annual cloud computing conference where announcements this year are expected to focus on its end-to-end data strategy and delivering new industry-specific services.

It will be the second re:Invent with CEO Adam Selipsky as leader of the industry’s largest cloud provider after his return last year to AWS from data visualization company Tableau Software.

Keep ReadingShow less
Donna Goodison

Donna Goodison (@dgoodison) is Protocol's senior reporter focusing on enterprise infrastructure technology, from the 'Big 3' cloud computing providers to data centers. She previously covered the public cloud at CRN after 15 years as a business reporter for the Boston Herald. Based in Massachusetts, she also has worked as a Boston Globe freelancer, business reporter at the Boston Business Journal and real estate reporter at Banker & Tradesman after toiling at weekly newspapers.

Image: Protocol

We launched Protocol in February 2020 to cover the evolving power center of tech. It is with deep sadness that just under three years later, we are winding down the publication.

As of today, we will not publish any more stories. All of our newsletters, apart from our flagship, Source Code, will no longer be sent. Source Code will be published and sent for the next few weeks, but it will also close down in December.

Keep ReadingShow less
Bennett Richardson

Bennett Richardson ( @bennettrich) is the president of Protocol. Prior to joining Protocol in 2019, Bennett was executive director of global strategic partnerships at POLITICO, where he led strategic growth efforts including POLITICO's European expansion in Brussels and POLITICO's creative agency POLITICO Focus during his six years with the company. Prior to POLITICO, Bennett was co-founder and CMO of Hinge, the mobile dating company recently acquired by Match Group. Bennett began his career in digital and social brand marketing working with major brands across tech, energy, and health care at leading marketing and communications agencies including Edelman and GMMB. Bennett is originally from Portland, Maine, and received his bachelor's degree from Colgate University.

Enterprise

Why large enterprises struggle to find suitable platforms for MLops

As companies expand their use of AI beyond running just a few machine learning models, and as larger enterprises go from deploying hundreds of models to thousands and even millions of models, ML practitioners say that they have yet to find what they need from prepackaged MLops systems.

As companies expand their use of AI beyond running just a few machine learning models, ML practitioners say that they have yet to find what they need from prepackaged MLops systems.

Photo: artpartner-images via Getty Images

On any given day, Lily AI runs hundreds of machine learning models using computer vision and natural language processing that are customized for its retail and ecommerce clients to make website product recommendations, forecast demand, and plan merchandising. But this spring when the company was in the market for a machine learning operations platform to manage its expanding model roster, it wasn’t easy to find a suitable off-the-shelf system that could handle such a large number of models in deployment while also meeting other criteria.

Some MLops platforms are not well-suited for maintaining even more than 10 machine learning models when it comes to keeping track of data, navigating their user interfaces, or reporting capabilities, Matthew Nokleby, machine learning manager for Lily AI’s product intelligence team, told Protocol earlier this year. “The duct tape starts to show,” he said.

Keep ReadingShow less
Kate Kaye

Kate Kaye is an award-winning multimedia reporter digging deep and telling print, digital and audio stories. She covers AI and data for Protocol. Her reporting on AI and tech ethics issues has been published in OneZero, Fast Company, MIT Technology Review, CityLab, Ad Age and Digiday and heard on NPR. Kate is the creator of RedTailMedia.org and is the author of "Campaign '08: A Turning Point for Digital Media," a book about how the 2008 presidential campaigns used digital media and data.

Latest Stories
Bulletins