The Big Story

Bruce Willis' latest gig: A Russian deepfake ad

John McClane is back. Russian mobile-phone company MegaFon recently began running a series of ads starring Bruce Willis, playing a character that looks awfully similar to the grumpy cop in everyone's favorite Christmas movie. Only, Willis didn't actually participate in the filming of the ads. Instead, the footage features an AI-rendered deepfake version of the actor produced by Russian synthetic media startup Deepcake.

The key to generating a believable action hero? Training data, according to Deepcake CEO Maria Chmir. "We used footage from 'Die Hard' and 'The Fifth Element' because it was important to us to create Bruce from that period," she told me. "We used computer vision to 'watch' these movies, finding the right scenes in them."

• Deepcake's algorithms scoured the two movies for scenes that showed Willis in the right angle, with the right lighting and even with a certain range of emotions.
• Then, Deepcake filtered out blurry frames, and ultimately used the remaining 34,000 images to train its neural network.
• The company also used AI to cast the actor playing Willis' part in the ad, looking for faces with similar features to the original McClane — a kind of understudy on to which the deepfake was imposed, if you will.
• The whole process of producing the first ad took five weeks, but Chmir told me that the company can now generate follow-up ads featuring deepfake Willis in three to five days.

Willis isn't the first celebrity to appear in a deepfake ad. Last year, Hulu used the same approach to create ad spots featuring athletes like Damian Lillard and Skylar Diggins-Smith when COVID-19 restrictions got in the way of a regular shoot. However, those athletes still filmed some of the training data for their deepfakes on Zoom calls; Willis' involvement ended with signing the contract for the project.

• Inevitably, this could lead to some eerie déjà vus. "Soon it will be possible to 'hire' actresses from the Golden Age of Hollywood, or young clones of living celebrities," Chmir said. "Who doesn't want to see a little Macaulay Culkin or Humphrey Bogart on screen?"

But Deepcake has some grand plans ahead, despite the fact it launched about a year ago and has so far only produced a few ads for the Russian market.

• "We don't position ourselves as a deepfake company," Chmir told me. "We are more of an AI-driven talent management agency. We do the whole cycle from buying the rights to use the image of a celebrity to generating the desired person inside the video."
• Chmir also teased plans for a feature film starring a deepfake character, but also said that the company was looking to help curtail abuse of deepfake tech. To that end, Deepcake wants to release its own deepfake detector early next year.
• "Our goal is [...] technology which doesn't mislead people," she said. "Face-swapping should become as commonplace as computer graphics, makeup or decorations."

A MESSAGE FROM CHECKOUT.COM

Over the last two years, many retailers have seen the benefit of investing in new, flexible payments. Despite the low-hanging fruit this opportunity presents, our research shows 60% of ecommerce merchants globally do not feel they receive enough payment insight to allow them to innovate their models. So how can businesses turn their ships around before it's too late?

Learn more

Watch Out

Facebook has acquired Daqri's AR patent portfolio

Remember Daqri? The Bay Area startup debuted a high-tech hard hat a few years ago that was supposed to revolutionize manufacturing with the help of AR. Daqri went belly-up in late 2019, and key members of its team have since joined Snap to help with the company's own AR efforts, as I first reported in early 2020.

Now, I've got an update to share on Daqri's final chapter: The company's patent portfolio has been acquired by Facebook this summer, according to U.S. Patent and Trademark Office filings.

• Hilco Streambank, a company specializing in the sale of IP assets, had begun shopping around Daqri's patents and patent applications in early 2020.
• In June of that year, the Daqri IP got sold to RPX Corporation, which then resold those rights to Facebook this summer.
• The rights acquired by Facebook include over 100 patents and patent applications covering everything from holographic lenses to AR content-creation tools.
• "Applications for the various portions of the portfolio include AR glasses in the industrial space and extend to autonomous vehicles, gaming, artificial intelligence, consumer electronics, digital content creation, 3D digital mapping and more," according to Hilco Streambank CEO Gabe Fried.

I expect we'll see more deals like this as tech giants like Apple and Facebook gear up to make their own AR glasses.

• After investing billions of dollars into this technology, the last thing these companies want is for some patent troll to snap up patent rights on the cheap, and then cash in on every future AR device sold.
• Ironically, this could mean that some of the pioneering AR startups will be more valuable to the industry after their demise.
• Daqri in particular always seemed better at marketing its own technology than actually selling it. Hillco Streambank's marketing material for Daqri's IP sale reveals that the startup had sold "over 700 units" of its headset. Daqri had raised a total of $300 million.

Fast Forward

• On Protocol:LG launched the NOVA startup competition. The consumer electronics giant wants to work with startups on IoT and metaverse products, among other things.
• Meredith doesn't want ads from TV antenna makers. The broadcast company allegedly blocked a seller of over-the-air antennas from advertising his products.
• Voices of VR turned 1,000. Kent Bye's podcast has been chronicling the VR industry for seven years and a whopping 1,000 episodes. Now, he's released a three-hour retrospective.
• On Protocol:Ian Rogers on his journey from Beats to crypto. Rogers played a key role in making Apple Music a success story — and now bemoans the dominance of a handful of streaming services.
• How did the Tokyo Olympic Games help Peacock? Here's an interesting analysis on the effect Tokyo had on sign-ups for NBCUniversal's streaming service.
• Clubhouse added spatial audio. Now you can listen to VCs complimenting each other in 3D!
• Apple has acquired Primephonic, a streaming service specializing in classical music. Primephonic is shutting down, but its playlists and recommendations will come to Apple Music soon.
• On Protocol: Virtual Burning Man is a blueprint for VR events. The festival, fully digital for the second year in a row, will run through Sept. 7 in Altspace.

Auf Wiedersehen

The other day, my daughter asked me for our HBO Max password. After getting it via text message, she responded: "Doesn't work." And thus, the troubleshooting began. Had I given her the correct username? (I had.) Did she by accident download the HBO Go app or the HBO Now app? (She didn't.) Do those apps even exist anymore? (They do not, at least not in the United States.) A few more wild guesses later, she finally got back to me. Turns out the password was correct all along. However, it happened to end with an exclamation mark, which she chose to ignore. "I thought you were just really excited," she said.