People

What went wrong with free speech online — and how Big Tech can fix it

Jillian C. York on free speech, how platforms should manage moderation and free expression, and her new book.

Jillian York's book

Jillian C. York's new book explores the history of online expression from YouTube to Facebook and from Europe to Myanmar.

Image: Jillian C. York

Jillian C. York has been thinking about online expression longer than most. As director for international freedom of expression at the Electronic Frontier Foundation, she's a longtime writer and advocate when it comes to preserving freedom of speech and expression on social platforms, and the consequences — particularly outside the U.S. — when that freedom is taken away.

In York's most recent book, "Silicon Values: The Future of Free Speech Under Surveillance Capitalism," she explores the history of online expression from YouTube to Facebook and from Europe to Myanmar. She depicts a complicated puzzle without a lot of easy answers, and worries that because things are complicated, too many companies have decided to do nothing. Or, even worse, to let governments around the world tell them what to do.

York joined the Source Code podcast to discuss the themes of her book, plus what platforms got right about the pandemic, why global companies need a more global perspective on expression and what other platforms can learn from Reddit.

Subscribe to the show: Apple Podcasts | Spotify | Google Podcasts | RSS

You can listen to the entire conversation with Jillian C. York on this episode of the Source Code podcast. The below excerpts have been edited for length and clarity.

In the course of reading your book, I had this same question and answered it like 50 different ways. Basically: Is it totally crazy for tech companies to say, "We operate in these countries, different countries have different rules and laws and norms, and we exist to operate by those? China is the most obvious example, because what China wants is very different from what people in San Francisco mostly want. But they're like, "Well, those are the rules, we have to play by the rules."

It's a tricky one. Let's break it down. On the one hand, it's not unreasonable to adhere to the rules of a democratic nation. So if we're talking about Germany, as much as I don't agree with all of the laws here around hate speech, they were for the most part decided in a democratic fashion. This is an electoral democracy.

When we're talking about China or Saudi Arabia or Turkey, we're talking about countries that do not have the same level of democracy. And so a U.S. company going ahead and saying, "We're going to side with the government?" I think that's really the important part here: These governments are not representative of the people. And so what you have is American capitalism siding with power with governments that were not chosen. These are not governments vying for the people.

At this stage in the internet, it's kind of too late to put it back in the box. People around the world know more than they did a generation ago. They know what the other options are, immigration is on the rise in every direction. It's not just people coming to the U.S. I'm an immigrant to Germany! And we've got a global playing field when it comes to all sorts of things, including the job market. And so to say that it's OK to censor certain information in another country is essentially putting people at a disadvantage.

I'm obviously making a kind of a capitalist argument here. But that's the one that I think might end up winning. And so we have to think about it from that perspective, as well as from a perspective of cultural knowledge.

Would that question be different if networks like YouTube and Facebook were deliberately walled off in some of these places? If Facebook Saudi Arabia was its own thing within the borders of Saudi Arabia and didn't touch the rest of the world, would we be having a different conversation about it?

Yes. And I think we'd also be having a different conversation about it if these companies hadn't started off from a very different perspective.

This debate has been going on for more than a decade. When I was talking to Facebook back around 2010, Google had just pulled out of China. This was an era where Yahoo had handed over information about a Chinese dissident to the Chinese government, putting him in prison. So this is a very different era, these companies were making their decisions based on user safety, and free expression to some degree, but not from an absolutist perspective.

And then the technology starts to grow and change. We have the introduction of SSL, which enabled censorship in a different way. Before that, a government could make a more granular decision about what to block, so they could block a specific YouTube video. SSL comes along, makes it harder for a government entity to identify the specific piece of content. And so the decision becomes much more binary. At that point, the companies start doing things differently, based on profit.

If we had these companies in individual countries — if their servers were based there, all of that — it would be much different. I do want to also note that it's not an easy decision for them, because they are risking their employees on the ground in a lot of these cases. But my argument is, and always has been, you shouldn't put offices in a country if that country is going to threaten your employees' lives.

I don't know if he originated this, but Siva Vaidhyanathan likes to say that "the problem with Facebook is Facebook." Is this just a totally intractable problem, where we're either going to have to tear down and rebuild the structures of these giant companies and the technology that powers the internet, or just learn to live with the side effects?

I'm of two minds here, because on the one hand, I don't think we can easily put it back in the box. I also think that that argument does a disservice to people in other parts of the world who may not have access to the broader internet. And again, that's the fault of these companies. They've chosen to go in and give free access to their platforms, and maybe throw in some Wikipedia on the side. So I do worry about the privilege in that argument.

But at the same time, I don't think that we should view these platforms as inevitable. I do think that we can create a better world. But it's going to come from a number of different sectors. It has to come from entrepreneurship, it also has to come from political science and sociology. And so we have to have a united force in designing what we want the next iteration of the internet to look like.

There's a really interesting transition in that, which you talked about a couple of times. Knitting communities, you point out —

Ravelry!

Ravelry! These same rules don't have to apply to Ravelry, or JDate, or some of these other things. The rules are different when you're huge. And I feel like Facebook has spent the last decade resolutely denying the fact that it's huge, at least from a policy perspective. The part of me that's sympathetic to that is the part of me that realizes that figuring out when you're huge, and what to do about it, is a really challenging thing.

It's definitely tricky. But with Facebook in particular, I don't think they've even tried. Part of the issue with Facebook is that a lot of the top-level executives have been there since the inception of the company, or since the very early years. They're out of touch with society. They're very privileged. They're living in their walled gardens, literally. And then the hires that they have made over the years have come primarily from government, like Nick Clegg, and law enforcement, like Monika Bickert. And so it's a very, very narrow view of the world.

With some of the other platforms, I think it's a little bit different. I mean, we've seen Twitter rethink itself, we've seen Reddit really rethink itself. As these platforms grow, they do have to go back and say, does this still make sense for 2021? And if I look at Facebook's rules, they've been just piling rule after rule on top of each other without really doing some kind of assessment or audit of what makes sense in this current era. And I think that's what they have to go back and do. And we are seeing other companies do that, which is great.

If I'm Ravelry, should I be thinking about that stuff right now? Do they need to be having deep conversations about their place in the expression world?

Nah. Ravelry, they're there for one reason. They're a knitting platform, and they've decided that political speech just isn't that important to their bottom line and to their M.O. If Ravelry started saying "you can't say anything but knitting," they might lose some users, right? Like, if you can't even just share what happened that day. But I think the thing that's so interesting about the political speech example, the fact that they chose to just be like "you know what, we're not even just going to ban conversation about Trump or U.S. politics, we're just gonna say no politics at all," it was a really stark reminder for me that political speech is not the most important speech.

A lot of these platforms, and U.S. culture in general, treats it that way. But it's not the be-all end-all, there's a lot of other types of expression that are absolutely vital. Cultural artistic expression, for example. And so I think with Ravelry, it's fine for them to do that.

I think the problem is when you try to be a platform for everything — and in Twitter and Facebook's case in particular, when you decide that you're going to be a conduit for the expression of elected and other public officials — that's where I think it gets really tricky. Because if you're going to be the place where politicians are talking to each other, there is some degree of transparency needed there. It's kind of a weird thing to just say, "no, we're not going to allow political expression," or "we're going to kick off this politician, but not this one."

These companies talk pretty freely about how much of their audience is outside of the U.S. It seems like you could make a pretty simple capitalistic case that they should really stop paying so much attention to people in the United States. Why hasn't that happened? What would it take to actually make them think more proactively about global issues?

The cynical answer to that is that only certain countries and regions are profitable. They do pay a lot of attention to Saudi Arabia, to Turkey, because those are big markets for them. They don't pay a lot of attention in Myanmar, because it's a poor country. It's that simple.

I also think that we have to bring this back to universal human rights frameworks and values. And so there is some stuff that the majority of the world is OK with having taken down. The problem is, you have to do it really carefully. So you can't just apply automation to hate speech and hope that it works, and hope that it doesn't catch counter speech and satire and human rights documentation in the mix, because that is what's going to happen if we're not being cautious and gentle in our approach to this.

That said, I used to be more of an absolutist. I've come around to the fact that we can't just allow hate to flourish on these platforms. But Article 19 doesn't allow that, either. There are restrictions that most of the world is signed onto and accepted that the U.S. (and Japan, also, for some reason) is really the big outlier on all of this. If we were using these frameworks, it would not allow for a lot of the stuff that Facebook takes down. We would have nudity, we would have some documentation of violence, we would have counter-speech against violence. But the way that these platforms do it is the lazy way. They apply the laziest, cheapest method. And that's really why we're in this mess.

I'm gonna make the bull case for why AI is going to solve all of our problems. And I want you to tell me why I'm wrong. Because I also think I'm wrong, but I'm gonna make the bull case anyway.

Ooh, all right.

The bull case says that AI doesn't solve all problems right now. But eventually, machine translation is going to get good enough that we can reasonably understand things that are going on everywhere. And machines are truly the only way that this can happen at scale. The goal is to do it quickly and proactively, so that people are not relied on to report on these things. The only solution, then, is to turn to sufficiently intelligent machines — which we don't have yet but we'll have someday — that can solve these problems. At the limit of scale, that's the only way this will ever actually work. So even if it will never be perfect, that's where we should be investing.

What a robotic lack of imagination that is! OK, so I speak Arabic badly. And I've been relying on Google Translate to fill in the blanks for me for many, many years, and I don't think Arabic translation has improved one iota.

Now, automation is good for some things. What it's good at is stuff that can be put in box A or box B. And what I would like to see happen is an increased use of automation for that, but with the decision of what happens after it's placed in boxes A or B in the hands of the user instead of in the hands of the centralized platform. So when it comes to nudity, you can use automation to detect nudity, but give me the choice whether to flip that switch on or off. If it's Saudi Arabia, maybe the government gets the choice. If it's a parent, maybe they get the choice. But there's so many different ways we can do this. And instead, these companies are demanding that we trust them with automation. And even the best automation, I don't trust Mark Zuckerberg with it. Why should I? He's given me zero reason to.

If I were to put it at the broadest possible level, your solution to a lot of these problems is just resources. More people, more diverse groups, more money, just paying more attention to these issues in more places, and in more cases. Is that a fair characterization?

Yeah! Stop spending the money on acquisitions and start spending it on genuine inclusivity. They speak about diversity, but diversity to them is, to put it bluntly, people of different races in the U.S. And that's important, don't get me wrong, but inclusivity is different. Inclusivity is bringing those people, both in the U.S. and abroad, up to the highest levels of governance.

If you look at the executive team of Facebook, they're almost entirely white. Many of them are men. And those who are not men, not white, are still Ivy-league educated. It's a very elite bubble. And so inclusivity is different from diversity in the sense that it's really looking at the broad intersectional backgrounds of different people, and bringing them not just into the room for consultations, but bringing them to the table, bringing them to the boardroom. It's not Sheryl Sandberg "Lean In." It's really more like, "Let's lean out and look at who's missing."

Correction: An earlier version of this story misspelled Monika Bickert's name. This story was updated on April 14, 2021.

Hybrid work has some distinct advantages when it comes to onboarding.

Photo: LogMeIn

Jo Deal is the chief human resources officer at LogMeIn. She is responsible for leading global people strategy with a focus on attracting, developing and engaging talent.

The desire for change that sprung up during the pandemic resulted in the highest attrition levels in decades and a fierce war for talent playing out in the market. The Great Resignation forced managers to suddenly make hiring their top priority, and recruitment partners became everyone’s best friend as leaders scrambled to replace key roles within their teams.

Keep Reading Show less
Jo Deal
Jo Deal serves as LogMeIn’s Chief Human Resources Officer. She is responsible for leading global people strategy with a focus on attracting, developing and engaging world class talent by expanding LogMeIn’s reputation as one of tech’s most desirable career destinations, and by providing a collaborative learning environment where employees can grow their careers.
Sponsored Content

A CCO’s viewpoint on top enterprise priorities in 2022

The 2022 non-predictions guide to what your enterprise is working on starting this week

As Honeywell’s global chief commercial officer, I am privileged to have the vantage point of seeing the demands, challenges and dynamics that customers across the many sectors we cater to are experiencing and sharing.

This past year has brought upon all businesses and enterprises an unparalleled change and challenge. This was the case at Honeywell, for example, a company with a legacy in innovation and technology for over a century. When I joined the company just months before the pandemic hit we were already in the midst of an intense transformation under the leadership of CEO Darius Adamczyk. This transformation spanned our portfolio and business units. We were already actively working on products and solutions in advanced phases of rollouts that the world has shown a need and demand for pre-pandemic. Those included solutions in edge intelligence, remote operations, quantum computing, warehouse automation, building technologies, safety and health monitoring and of course ESG and climate tech which was based on our exceptional success over the previous decade.

Keep Reading Show less
Jeff Kimbell
Jeff Kimbell is Senior Vice President and Chief Commercial Officer at Honeywell. In this role, he has broad responsibilities to drive organic growth by enhancing global sales and marketing capabilities. Jeff has nearly three decades of leadership experience. Prior to joining Honeywell in 2019, Jeff served as a Partner in the Transformation Practice at McKinsey & Company, where he worked with companies facing operational and financial challenges and undergoing “good to great” transformations. Before that, he was an Operating Partner at Silver Lake Partners, a global leader in technology and held a similar position at Cerberus Capital LP. Jeff started his career as a Manufacturing Team Manager and Engineering Project Manager at Procter & Gamble before becoming a strategy consultant at Bain & Company and holding executive roles at Dell EMC and Transamerica Corporation. Jeff earned a B.S. in electrical engineering at Kansas State University and an M.B.A. at Dartmouth College.
Entertainment

Peloton’s terrible, horrible, no good, very bad year

2022 just started, and Peloton has already halted bike production and is talking about mass layoffs. How did the pandemic darling get here?

How did Peloton go from pandemic star to sinking ship? One answer is the classic problem of supply and demand.

Image: Peloton; Protocol

It’s been a hell of a ride for Peloton. The headlines have been practically nonstop, from 2019’s cringey wife ad to 2021’s series of unfortunate “Sex and The City” events. But in 2020, Peloton could do no wrong. The at-home fitness company saw a 172% spike in sales over the course of that year, buoyed by the pandemic forcing wealthy gym-goers to stay home.

But nothing is ever easy or certain when it comes to Peloton. In the past week, Business Insider reported that Peloton is considering laying off 41% of its sales and marketing staff and closing down stores. CNBC learned that the company has hired McKinsey & Co. to help cut costs. And yesterday, CNBC reported that Peloton is temporarily halting production of its bikes. Peloton shares promptly plunged 24%.

Keep Reading Show less
Lizzy Lawrence

Lizzy Lawrence ( @LizzyLaw_) is a reporter at Protocol, covering tools and productivity in the workplace. She's a recent graduate of the University of Michigan, where she studied sociology and international studies. She served as editor in chief of The Michigan Daily, her school's independent newspaper. She's based in D.C., and can be reached at llawrence@protocol.com.

Boost 2

Can Matt Mullenweg save the internet?

He's turning Automattic into a different kind of tech giant. But can he take on the trillion-dollar walled gardens and give the internet back to the people?

Matt Mullenweg, CEO of Automattic and founder of WordPress, poses for Protocol at his home in Houston, Texas.
Photo: Arturo Olmos for Protocol

In the early days of the pandemic, Matt Mullenweg didn't move to a compound in Hawaii, bug out to a bunker in New Zealand or head to Miami and start shilling for crypto. No, in the early days of the pandemic, Mullenweg bought an RV. He drove it all over the country, bouncing between Houston and San Francisco and Jackson Hole with plenty of stops in national parks. In between, he started doing some tinkering.

The tinkering is a part-time gig: Most of Mullenweg’s time is spent as CEO of Automattic, one of the web’s largest platforms. It’s best known as the company that runs WordPress.com, the hosted version of the blogging platform that powers about 43% of the websites on the internet. Since WordPress is open-source software, no company technically owns it, but Automattic provides tools and services and oversees most of the WordPress-powered internet. It’s also the owner of the booming ecommerce platform WooCommerce, Day One, the analytics tool Parse.ly and the podcast app Pocket Casts. Oh, and Tumblr. And Simplenote. And many others. That makes Mullenweg one of the most powerful CEOs in tech, and one of the most important voices in the debate over the future of the internet.

Keep Reading Show less
David Pierce

David Pierce ( @pierce) is Protocol's editorial director. Prior to joining Protocol, he was a columnist at The Wall Street Journal, a senior writer with Wired, and deputy editor at The Verge. He owns all the phones.

Entertainment

Netflix looks to expand gaming with major IP deals, Fortnite-like updates

Remarks made to investors and recent job postings hint at big ambitions for Netflix’s nascent gaming efforts.

Netflix may be taking some cues from games like Fortnite and Apex: Legends for its own video game initiative.

Photo: Cameron Venti/Unsplash

Two months after launching mobile games to all of its members, Netflix is looking to double down on gaming: The company told investors Thursday that it wants to expand its portfolio of games “across both casual and core gaming genres.” Recent job offers suggest that this could include both live services games as well as an expansion to PC and console gaming, and the company's COO hinted at major licensing deals ahead.

Keep Reading Show less
Janko Roettgers

Janko Roettgers (@jank0) is a senior reporter at Protocol, reporting on the shifting power dynamics between tech, media, and entertainment, including the impact of new technologies. Previously, Janko was Variety's first-ever technology writer in San Francisco, where he covered big tech and emerging technologies. He has reported for Gigaom, Frankfurter Rundschau, Berliner Zeitung, and ORF, among others. He has written three books on consumer cord-cutting and online music and co-edited an anthology on internet subcultures. He lives with his family in Oakland.

Policy

Tim Cook, Ted Cruz and the strange politics of tech antitrust

Democrats and Republicans have found the tech reform debate scrambles traditional party politics — and the Apple CEO and Texas senator have found themselves chatting.

The Senate Judiciary Committee advanced a bill on Thursday that could remake the tech industry.

Photo: PartTime Portraits/Unsplash

Strange alliances formed ahead of Thursday's vote to advance a key antitrust bill to the Senate floor, with frequent foes like Sens. Amy Klobuchar and Ted Cruz supporting the measure, and prominent Democrats including California Sen. Dianne Feinstein pushing back against it.

Ultimately the bill moved out of the Senate Judiciary Committee by a vote of 16-6 after a surprisingly speedy debate (at least, speedy for the Senate). Even some of the lawmakers who called for further changes agreed to move the bill forward — a sign that the itch to finally regulate Big Tech after years of congressional inaction is intensifying, even as the issue scrambles traditional party politics in a way that could threaten its final passage.

Keep Reading Show less
Ben Brody

Ben Brody (@ BenBrodyDC) is a senior reporter at Protocol focusing on how Congress, courts and agencies affect the online world we live in. He formerly covered tech policy and lobbying (including antitrust, Section 230 and privacy) at Bloomberg News, where he previously reported on the influence industry, government ethics and the 2016 presidential election. Before that, Ben covered business news at CNNMoney and AdAge, and all manner of stories in and around New York. He still loves appearing on the New York news radio he grew up with.

Latest Stories
Bulletins