next-upnext upauthorJanko RoettgersNoneDo you know what's coming next up in the world of tech and entertainment? Get Janko Roettgers' newsletter every Thursday.9147dfd6b1
×

Get access to Protocol

I’ve already subscribed

Will be used in accordance with our Privacy Policy

People

How tech is packing empty stadiums with (fake) raucous crowds

Computer-generated fans could allow smaller sports events to make it big on TV.

A baseball game with fake fans

Real game, fake fans: Fox Sports is filling empty stadiums with computer-generated fans for MLB games during COVID.

Photo: Courtesy of Fox Sports

When Fox Sports aired the first MLB games of the new, shortened season this past weekend, it filled parts of the stadium with digital people, as the pandemic is keeping the real ones at home. These virtual crowds were a first for U.S. sports, but we'll likely see a lot more examples in the coming months. CGI animators have seen a flood of incoming requests to generate crowds for sports events and concerts alike, and insiders say that the trend could provide the industry with new opportunities even after COVID subsides.

Fox Sports only added CGI crowds to some of its camera feeds, leading to a somewhat dissonant mixture of shots of empty seats and shoulder-to-shoulder crowds. The results were nonetheless real enough to confuse some viewers. "Fox fooled me for a second with the digital fans," wrote one viewer on Twitter. Another asked: "Wait there are fans there? Are they all wearing masks?"

Movies, commercials and video games have long used virtual crowds as stand-ins for real humans, but up until now, the craft of generating those digital crowds has gotten very little attention. "There is really no difference between a crowd and a tree," said Dan Warom, who has been producing digital crowds for visual-effects company The Mill around 15 years.

Crowds provide important visual cues to viewers, allowing them to get a sense of scale in a scene. But if CGI artists do their work well, crowds simply blend into the background. "Everyone needs it. But it is not sexy," Warom acknowledged about his line of work.

That changed dramatically over the past several months, as sports leagues and broadcasters tried to figure out how to safely bring sports back to TV. Leagues initially used cardboard cutouts, stuffed animals and even sex dolls to fill the stands, often while augmenting games with prerecorded crowd noise, which itself has come from video games in some cases.

At the same time, the industry began to look for technology that could help make games look and feel more familiar. "Most of the big broadcasters started to seek out a solution to this," Epic Games product manager Andy Blondin said. Epic is best known for Fortnite, but the company also makes the Unreal Engine, a key production tool for video games that is increasingly being embraced by the entertainment industry as well.

Over the past few years, Hollywood has been moving away from adding visual effects in post-processing and toward generating them in real time while actors are being recorded playing in front of green screens. This approach gives directors of feature films a way to watch an approximation of the final shot live on set, an approach that Lucasfilm's visual effects unit ILM used extensively for the latest "Star Wars" movies.

The same technology also makes it possible to add virtual fans to sports games in real time, but there are some unique challenges. For one thing, sports games are being broadcast live, and crowds have to be overlaid over the actual camera feed without noticeable delays. As a result, broadcasters may have to compromise on visual complexity — something that some viewers complained about. The digital assets to create these crowds also have to be customized for each game, "to dress them properly, to put the jerseys in there," Blondin said. "All the nuances that make them feel real."

Adding to the illusion are AI algorithms that allow these virtual fans to respond to cues from the game. Fan behavior in a baseball stadium is very predictable, with viewers sitting on their chairs for much of the game, only to jump up and cheer when someone hits a home run. "The actual AI requirements are very, very low," Warom said, adding that AI can also be used to lighten things up a bit: "You can have the wave propagate through the crowd."

The technology that is now being tested by Fox Sports and others could come in handy even in a post-COVID world, after fans are allowed back into stadiums. For instance, it could give smaller regional sports teams a chance to shine. "For smaller stadiums, you may be able to augment with a second deck," Blondin said.

Broadcasters could potentially also augment and air games that traditionally happen without sizable crowds, like training matches. A team could have its training match on a regular field, and a broadcaster could then add a whole virtual stadium to the TV feed. "It's the difference between playing for the Premier League and playing [on] Sunday morning," Warom said. Adding virtual crowds could also give smaller leagues new revenue opportunities, for instance through commercials. Warom even mused that one day, parents may be able to add virtual crowds to their kids' Little League games.

In the more immediate future, the technology is being adopted by music festivals and others looking to put on believable digital events without real crowds during COVID. Having broadcasters like Fox Sports pilot digital crowds has demonstrated the possibility of the technology, according to business-development manager at Epic Games, Craig Laliberte. "It's opening up a lot of doors," he said.

People

Beeper built the universal messaging app the world needed

It's an app for all your social apps. And part of an entirely new way to think about chat.

Beeper is an app for all your messaging apps, including the hard-to-access ones.

Image: Beeper

Eric Migicovsky likes to tinker. And the former CEO of Pebble — he's now a partner at Y Combinator — knows a thing or two about messaging. "You remember on the Pebble," he asked me, "how we had this microphone, and on Android you could reply to all kinds of messages?" Migicovsky liked that feature, and he especially liked that it didn't care which app you used. Android-using Pebble wearers could speak their replies to texts, Messenger chats, almost any notification that popped up.

That kind of universal, non-siloed approach to messaging appealed to Migicovsky, and it didn't really exist anywhere else. "Remember Trillian from back in the day?" he asked, somewhat wistfully. "Or Adium?" They were the gold-standard of universal messaging apps; users could log in to their AIM, MSN, GChat and Yahoo accounts, and chat with everyone in one place.

Keep Reading Show less
David Pierce

David Pierce ( @pierce) is Protocol's editor at large. Prior to joining Protocol, he was a columnist at The Wall Street Journal, a senior writer with Wired, and deputy editor at The Verge. He owns all the phones.

Politics

In 2020, COVID-19 derailed the privacy debate

From biometric monitoring to unregulated contact tracing, the crisis opened up new privacy vulnerabilities that regulators did little to address.

Albert Fox Cahn, executive director of the Surveillance Technology Oversight Project, says the COVID-19 pandemic has become a "cash grab" for surveillance tech companies.

Photo: Lianhao Qu/Unsplash

As the coronavirus began its inexorable spread across the United States last spring, Adam Schwartz, senior staff attorney at the Electronic Frontier Foundation, worried the virus would bring with it another scourge: mass surveillance.

"A lot of really bad ideas were being advanced here in the U.S. and a lot of really bad ideas were being actually implemented in foreign countries," Schwartz said.

Keep Reading Show less
Issie Lapowsky
Issie Lapowsky (@issielapowsky) is a senior reporter at Protocol, covering the intersection of technology, politics, and national affairs. Previously, she was a senior writer at Wired, where she covered the 2016 election and the Facebook beat in its aftermath. Prior to that, Issie worked as a staff writer for Inc. magazine, writing about small business and entrepreneurship. She has also worked as an on-air contributor for CBS News and taught a graduate-level course at New York University’s Center for Publishing on how tech giants have affected publishing. Email Issie.
People

The year our personal lives took center stage at work

2020's blurring of professional and personal boundaries exacerbated disparities, humanized leaders and put personal values front and center.

In 2020, the personal and the professional became inextricable at work.

Photo: Tom Werner/Getty Images

For those of us lucky enough to keep our jobs and privileged enough to be able to work from home, our whole selves were bared at work this year. Our homes and faces were blown up for virtual inspection. Our children's demands and crises filled our working hours, and our working mothers became schoolteachers and housewives, whether they wanted to or not. Our illnesses became vital public information, and our tragedies shared. Our work lives ate into our social lives until there was no boundary between them.

In 2020, the personal and the professional became inextricable at work. Remote work might be the most sexy 2020 trend, but for the CEOs and leaders I spoke with, the de-professionalization of work could be the most important effect on a personal level. It's the one that has caused the most harm to women in the workplace and destroyed work-life balance for basically everyone. It's also what has contributed to the majority of work-from-home Americans being more satisfied with their work lives than they were before, mostly because they feel more connected to their families, they're able to set their own schedules and they're more comfortable at home, according to a Morning Consult poll. While we can't know exactly how many and who will be going back to the office just yet, as long as there is some kind of flexible work schedule, people's personal lives will be part of their work lives and vice versa.

Keep Reading Show less
Anna Kramer

Anna Kramer is a reporter at Protocol (@ anna_c_kramer), where she helps write and produce Source Code, Protocol's daily newsletter. Prior to joining the team, she covered tech and small business for the San Francisco Chronicle and privacy for Bloomberg Law. She is a recent graduate of Brown University, where she studied International Relations and Arabic and wrote her senior thesis about surveillance tools and technological development in the Middle East.

Protocol | Enterprise

How Christian Klein’s reboot of SAP’s strategy is working out

The pandemic wasn't kind to the company. But the way it's working with the major COVID-19 vaccine makers is a model for what comes next.

Christian Klein became SAP's sole CEO in April.

Photo: Picture Alliance/Getty Images

Christian Klein took over as SAP's sole CEO in April. It wasn't an ideal time to take the helm of an organization that sells expensive enterprise software.

As the spread of COVID-19 forced corporations everywhere to cut costs, one of the first places they looked was IT budgets. Specifically, companies around the world trimmed spending on back-end products, such as those offered by SAP, many of which still run via on-premise data centers.

Keep Reading Show less
Joe Williams

Joe Williams is a senior reporter at Protocol covering enterprise software, including industry giants like Salesforce, Microsoft, IBM and Oracle. He previously covered emerging technology for Business Insider. Joe can be reached at JWilliams@Protocol.com. To share information confidentially, he can also be contacted on a non-work device via Signal (+1-309-265-6120) or JPW53189@protonmail.com.

Politics

Congress is unleashing the FTC on COVID-19 scammers

The second stimulus package will expand the FTC's authority to penalize companies promoting fake information and faulty products.

The FTC will finally have the authority to immediately fine scammers touting fake COVID-19 treatments and lies about pandemic government benefits.

Photo: Al Drago/Getty Images

The FTC will finally have the authority to fine scammers touting fake COVID-19 treatments and lies about pandemic government benefits, thanks to new language in the economic stimulus package set to pass on Monday.

COVID-related scams proliferating online have overwhelmed government agencies and cost Americans more than $145 million since the beginning of the pandemic — and the FTC has been almost powerless to stop them, thanks to limitations on the agency's authority.

Keep Reading Show less
Emily Birnbaum

Emily Birnbaum ( @birnbaum_e) is a tech policy reporter with Protocol. Her coverage focuses on the U.S. government's attempts to regulate one of the most powerful industries in the world, with a focus on antitrust, privacy and politics. Previously, she worked as a tech policy reporter with The Hill after spending several months as a breaking news reporter. She is a Bethesda, Maryland native and proud Kenyon College alumna.

Latest Stories