yesAnna KramerNone
×

Get access to Protocol

I’ve already subscribed

Will be used in accordance with our Privacy Policy

People

What Tracy Chou learned about online harassment while building an app to solve it

Her new app, Block Party, aims to give people control over harassing content.

What Tracy Chou learned about online harassment while building an app to solve it

Block Party founder and diversity activist Tracy Chou became the target of a Reddit harassment campaign while trying to promote the importance of anti-harassment tools.

Photo: Tracy Chou

When Tracy Chou decided to host a Reddit "Ask Me Anything" about online harassment over the summer, she knew it probably wouldn't be the easiest experience, but she'd been dealing with trolls for most of her career. How bad could it really be?

A vitriol-filled nightmare, it turns out. The woman hosting a forum on why she was building an app to protect against online harassment was the target of one of the biggest harassment campaigns of her life. Reddit users mocked her inability to answer their questions (she had, but due to a system error, her comments were disappearing before anyone could read them). They ridiculed her appearance and motivations. Someone created a campaign to say nasty things about her on Substack, attaching her name and photo to some of the posts. They moved to Twitter, and then to 4chan, where they organized a group that flooded her site with a denial of service attack until it went down.

It took her weeks to get everything back under control. "I was getting harassed by trying to tell people about this anti-harassment software I'm building. It was awful," she said.

Chou made a name for herself as an outspoken diversity activist making common-sense arguments about why the tech industry needs serious culture change. Now, she's building an app to protect people from online abuse and harassment, inspired by her own experience engendering hate that grew along with her public profile. Her startup, called Block Party, provides a tool to filter trolling and harassing posts, and it launched for public signups last week.

People have asked Chou why she just doesn't stay off Twitter. But instead of allowing harassment to push her off the internet, she wants to feel safe and comfortable online, and that's what Block Party is supposed to help do. For Chou, it seems obvious that people shouldn't be afraid to avail themselves of social platforms just because they have "controversial" ideas or an underrepresented background.

The premise of Block Party is simple. Based on your needs, you set up filters that take messages and comments out of your Twitter experience and into a folder on Block Party, which you can then review when you feel comfortable and prepared. It also has a function for people who need to compile evidence of harassment or stalking (say, for a police report) which allows users to have a friend go through that evidence for them (a tool based on Chou's personal experience; she's filed several police reports in the U.S. and the U.K. about dedicated stalkers who escalated from online harassment).

Chou has big-time ambitions for Block Party. She wants to increase the types of filters people can apply to their accounts. She hopes the app will one day be able to interface with every social media account, not just Twitter, becoming a portable safety net people can take with them to have more control over their online experience. She'd like to eventually build Block Party APIs for new social platforms as they come online, so that they can integrate the tool into their own platforms as a "trust and preferences layer." "This stuff is cross-platform. People move around to harass, but platforms don't collaborate on this stuff," she said.

The idea for Block Party grew out of Chou's early experiences as an engineer. She was among the first 10 employees at Quora and then at Pinterest, and at both places her own need to protect herself from harassing users shaped the products she built. At Quora, one of the first products she shipped was a block button, designed because she needed a way to protect herself from someone targeting her with angry comments on the site. "If you don't have representation or these perspectives in a room, when people are deciding what to work, it gets very influenced by the people who are there. There's no ill intent," she explained.

The typical approach to content moderation is a machine-learning black box that filters content, and human moderators then do the same for everything missed by the system. Because machine learning is not as effective as it needs to be, social media companies rely on huge teams of content moderators that have to make quick decisions about a flood of often traumatizing content. "The whole approach seems pretty flawed to me," Chou told me (she has a master's degree in artificial intelligence). "Machine learning is not going to be the magical solution. People who think ML is going to solve everything are generally men who are enamored with this technology. They don't experience the problem and see how much basic stuff you can do with simple products."

Her approach with Block Party instead considers the very human aspect of what people who are dealing with abuse actually need. "There's a lot of really unpleasant content that a lot of people get, but it's not that people need to be deplatformed. There's a huge margin between bad enough to get deplatformed and stuff I actually want to see. Your Twitter account doesn't need to be taken away because you called me an ugly, skinny Chinese girl, but I don't want to see it," she said.

Her tool gives the consumer control over that content. Rather than delete the comment or deplatform the person who made it, settings could allow a user to filter that comment out of their Twitter experience and into a folder in the Block Party app, which they can then revisit later and review on their own terms. It's about allowing people to have a chance to prepare themselves to deal with whatever someone may or may not say, according to Chou.

Alex Stamos, an early Block Party investor and the director of the Stanford Internet Observatory, used the app recently for just that purpose. After a television appearance discussing the right-wing media ecosystem in late January, a QAnon leader suggested Stamos was a pedophile to his followers, leading to a massive Twitter attack and series of death threats. "I ended up using Block Party, and I blocked over 2,000 people over the weekend, and it was incredibly successful. I had a very tense situation there, and it's the only thing that kept Twitter usable for me. And it made me feel pretty darn good about investing in it," Stamos said.

Though Twitter has its own plans to change and invest in content moderation and user experience, Chou's relationship with the company has actually been fruitful. "It's been interesting to see how many different people within Twitter have reached out to say it's great," she said. "They have acknowledged that as much as they are going to do, they aren't going to be able to solve this problem themselves." And she really believes they are serious about doing it: "There's been a bit of a sea change within Twitter over the last few years."

"There's been a lot of discussion about the broad trust and safety problem at companies, and it's great for the companies to do what they can to improve, but the truth is that there are a number of people who have very intense safety issues and are also willing to take charge of it themselves," Stamos said. He's hoping that over time, Block Party's success will encourage companies to build their APIs for easy interface with tools like Chou's.

While Chou has convinced Twitter of the need for her work, persuading investors has been a far more difficult battle, especially when she encounters groups of all-male or mostly male panels, which tend to be heavily skeptical of any actual demand for Block Party's product.

Along with Stamos, her first investors were people who inherently understood the need for the app: Alexia Bonatsos, the former co-editor of TechCrunch and part of Block Party's target audience, and Charles Hudson, the managing partner and founder of Precursor Ventures, one of the few Black-founded VC firms. "Hudson got it immediately," she said.

Block Party has escaped serious criticism from moderation foes so far because it's "content-neutral," Stamos explained. Because each user decides for themselves what to filter and block, there are no debates about the types of appropriate or "allowed" content like those that plague social media companies. "But there will always be a tension between allowing for open debate and the tools that allow people to block, and that includes tools like Block Party," Stamos said.

While it's been difficult to persuade investors of the need for a product they personally can't relate to, the launch of Block Party's open signups might change that. To participate in the open registration and skip the app's traditional waitlist, people had to pay $8. And they did, increasing the app's number of users by 50% in one week and proving both interest and potential for monetization far beyond Chou's expectations.

"My production intuition around this is pretty good. Sometimes you worry that your own experience doesn't generalize, but it turns out that my experience isn't that unusual," she said. And demand for the app isn't coming just from prominent women on the platform in positions like Chou's, illustrating the ubiquity of online harassment; many of Block Party's recent new users are scientists and doctors facing online hate for their posts about coronavirus.

When I asked Chou what kept her determined to do this work despite the fear of abuse and harassment, she seemed almost confused. "It takes a toll, yes," she said. "It also feels like the only thing I can do to solve it is work on this."

People

No editing, no hashtags: Dispo wants you to live in the moment

David Dobrik's new photography app harkens back to the days of the disposable camera.

Dispo turns the concept of a photography app into something altogether different.

Image: Katya Sapozhnina, Diana Morgan, Amanda Luke

Instagram was once a place to share Starbucks cups and high-contrast pet photos. After Facebook acquired it in 2012, it has turned into a competition of getting as many likes as possible (using the same formula over and over: post the best highly-curated, edited photos with the funniest captions). More recently, it's essentially become a shopping mall, with brands falling over themselves to be heard through the noise. Doing something "for the gram" — scaling buildings, posting the same cringe picture over and over — became the norm. Pop-up museums litter cities with photo ops for posts; "camera eats first"; everything can be a cute Instagram story; everything is content.

And to be clear, Dispo — a buzzy new photography app that just came out of beta — is still a place for content. It probably isn't going to fix our collective online brains and their inclination to share everything about our private lives with others online. It's still an app, and it's still social media, and it encourages documenting your life. But it runs pretty differently than any other image-sharing app out there. And that might be what helps it stand out in an oversaturated market of social networking apps.

Keep Reading Show less
Jane Seidel

Jane Seidel is Protocol's social media manager. She was previously a platform producer at The Wall Street Journal, creating mobile content and crafting alert strategy. Prior to that, she worked in audience development at WSJ and on digital editorial at NBC Universal. She lives in Brooklyn.

Sponsored Content

Building better relationships in the age of all-remote work

How Stripe, Xero and ModSquad work with external partners and customers in Slack channels to build stronger, lasting relationships.

Image: Original by Damian Zaleski

Every business leader knows you can learn the most about your customers and partners by meeting them face-to-face. But in the wake of Covid-19, the kinds of conversations that were taking place over coffee, meals and in company halls are now relegated to video conferences—which can be less effective for nurturing relationships—and email.

Email inboxes, with hard-to-search threads and siloed messages, not only slow down communication but are also an easy target for scammers. Earlier this year, Google reported more than 18 million daily malware and phishing emails related to Covid-19 scams in just one week and more than 240 million daily spam messages.

Keep Reading Show less

Tech companies promised to do better on diversity. Here's how they can.

Holler CEO Travis Montaque calls for an agile approach to make systemic, not surgical, changes, both internally and externally.

"When you're looking to solve a big, complex problem, you can't expect to achieve it in one fell swoop," says Holler CEO Travis Montaque.

Image: Spencer Platt/Getty Images

Travis Montaque is the founder and CEO of Holler, a messaging technology company.

In 2020, protests against police brutality and racism erupted across the country. Issues of racial injustice were brought to the forefront of our national consciousness. In response, business leaders from just about every industry came out with statements acknowledging the work that needs to be done, and committing to making a change.

Keep Reading Show less
Travis Montaque

As founder and CEO of Holler, Travis Montaque is building the ultimate content engine for conversations, starting with animated, expressive and branded content for messaging environments. He's opened up a massive new opportunity for brands to engage with audiences via messaging, the most popular digital space to connect, earning him a spot in Forbes 30 Under 30 and Entrepreneur Magazine's Most Daring Entrepreneurs. Most recently, he led Holler to a spot on Fast Company's 2020 Most Innovative Companies List in Social Media. Travis is also an inaugural member of the University of Miami Marketing Advisory Board.

Transforming 2021

Blockchain, QR codes and your phone: the race to build vaccine passports

Digital verification systems could give people the freedom to work and travel. Here's how they could actually happen.

One day, you might not need to carry that physical passport around, either.

Photo: CommonPass

There will come a time, hopefully in the near future, when you'll feel comfortable getting on a plane again. You might even stop at the lounge at the airport, head to the regional office when you land and maybe even see a concert that evening. This seemingly distant reality will depend upon vaccine rollouts continuing on schedule, an open-sourced digital verification system and, amazingly, the blockchain.

Several countries around the world have begun to prepare for what comes after vaccinations. Swaths of the population will be vaccinated before others, but that hasn't stopped industries decimated by the pandemic from pioneering ways to get some people back to work and play. One of the most promising efforts is the idea of a "vaccine passport," which would allow individuals to show proof that they've been vaccinated against COVID-19 in a way that could be verified by businesses to allow them to travel, work or relax in public without a great fear of spreading the virus.

Keep Reading Show less
Mike Murphy

Mike Murphy ( @mcwm) is the director of special projects at Protocol, focusing on the industries being rapidly upended by technology and the companies disrupting incumbents. Previously, Mike was the technology editor at Quartz, where he frequently wrote on robotics, artificial intelligence, and consumer electronics.

Protocol | Enterprise

Salesforce’s equality struggles burst into the public

Former Salesforce senior manager Cynthia Perry tore into the company and its treatment of Black workers in a recent resignation letter.

Benioff has publicly championed Salesforce's diversity efforts, but employees tell a different story.

Photo: Jason Alden/Getty Images

Salesforce is facing fresh pressure over its diversity and inclusion efforts after a senior manager quit amid what she described as consistent discriminatory behavior.

In a resignation letter posted to LinkedIn earlier this month, Cynthia Perry wrote a searing take-down of the company's racial equality efforts, specifically the treatment of Black employees, at the massive software provider.

Keep Reading Show less
Joe Williams

Joe Williams is a senior reporter at Protocol covering enterprise software, including industry giants like Salesforce, Microsoft, IBM and Oracle. He previously covered emerging technology for Business Insider. Joe can be reached at JWilliams@Protocol.com. To share information confidentially, he can also be contacted on a non-work device via Signal (+1-309-265-6120) or JPW53189@protonmail.com.

Latest Stories