Meta’s next headset has a secret superpower

Meta is getting ready to make VR feel a lot more real: The company is announcing the general availability of its Presence Platform today, which allows developers to incorporate better hand-tracking, voice interaction and video passthrough into their apps.

Passthrough essentially turns a VR headset into a mixed-reality device by incorporating both VR and live video images of the real world. For now, these efforts are very much focused on the Quest 2, whose inside-out tracking cameras only offer a washed-out, grayscale view of the world. But Meta’s push into mixed reality is clearly also part of its future device road map, which includes a high-end VR headset code-named Project Cambria that the company will release later this year.

I got to briefly try a pre-release version of Cambria this week after getting a more extensive demo of mixed reality on a current-generation Quest 2. I didn’t get a chance to check out Cambria in depth, and was only able to try one mixed-reality demo (The World Beyond, which is being released for Quest 2 next week), but even those few minutes were enough to convince me that mixed reality on Cambria is a kind of secret superpower.

• Cambria’s improved image sensors allow you to see the real world in color, which is a massive improvement over the Quest’s grayscale look. It’s still not a photorealistic image, but it’s starting to feel a lot less jarring. Think more decent-quality home video, less “Blair Witch Project.”
• The new image sensors also have a much higher resolution than those of the Quest 2. Meta has yet to share complete specs for Cambria, but Mark Zuckerberg told me during a chat following the demo that the sensors have three times the resolution of those used for the Quest 2.
• “There's a good road map to get that to be even higher over time,” Zuckerberg said. “We'll keep on pushing on that.”
• Color and higher resolution isn’t just vastly improving the passthrough experience for Cambria wearers; it also helps the device itself make sense of the world.
• This includes separating individual objects from another, something that’s key to incorporating our real-world surroundings into mixed-reality experiences. While the Quest may have seen three objects lying on top of each other as one big blob, Cambria may actually be able to detect clearer boundaries.
• “Once you bring in color, you can start separating things,” said Reality Labs product manager Prabhu Parthasarathy.

Cambria will be all about work use cases at launch, and adding a better passthrough experience is a big part of making work in VR feel more natural, Zuckerberg told me.

• Cambria is the first in a series of devices that Meta is looking to sell to companies and knowledge workers. “I think there's going to be a work-grade device, which is going to eventually [be] a laptop or workstation replacement,” Zuckerberg said.
• By the end of the decade, a headset like that could be the main device we work with every day, and combining the real world with virtual screens and objects will be key. “You'll be able to see your desk, snap your fingers, bring up your screens,” Zuckerberg said.
• In this mixed-reality future, remote work will feel more like being in the same room with your co-workers, thanks to the ability to bring 3D avatars into a passthrough view of your home office. “This feeling of presence is something that you can't really get through any other technology today,” Zuckerberg said.

Meta wants to make VR feel more natural, and color passthrough is part of a broader strategy that also includes incorporating your hands and voice. All of this hinges on hardware improvements for Cambria and future headsets.

• The Quest didn’t have a dedicated depth sensor, and instead used the onboard camera sensors to compute depth sensing. “It's kind of just hacked,” Zuckerberg admitted. “On Cambria, we actually have a depth sensor.” Cambria will use an IR projector for active depth sensing.

• The demo I got to try used hand-tracking instead of controllers, and allowed me to “reel in” objects simply by making a grabbing motion.
• Zuckerberg said hand-tracking on Quest outperformed the company’s expectations, and that Meta is doubling down on this input method for future headsets.
• “With Cambria and the devices going forward, we now have this whole sensor architecture that is going to be more optimized towards hands. So you'll just have much better hardware support for that,” Zuckerberg said.

— Janko Roettgers

Sonos is launching its own voice assistant

Sonos is getting ready to take on Alexa and Google Assistant: The company announced its very own voice assistant Wednesday. Sonos Voice Control will be available on the company’s speakers in the U.S. at the beginning of next month, with plans to launch in France later this year.

Sonos Voice Control will be primarily focused on music search and playback, as well as control of Sonos speaker systems. The assistant also differs from Alexa and Google Assistant in that it never uploads any audio to the cloud, but instead processes everything on the device.

• That’s key to winning over people who have thus far steered clear of voice assistants, said Sonos senior sound experience manager Greg McAllister.
• Almost half of the voice-capable Sonos speakers it sold aren’t being used for that purpose, according to McAllister.
• “Time and time again, when we speak to our customers, they express that they have concerns over privacy,” he said.

Sonos is also putting a new spotlight on interoperability issues that have long plagued the industry. At launch, consumers will be able to run Alexa and the Sonos assistant on the same device and invoke them with a specific wake word.

• Google has long resisted this kind of voice interoperability, so people won’t be able to use Sonos Voice Control and the Google Assistant on the same speaker.
• At the same time, Sonos also has to stay clear of playing favorites with voice commands, especially since the company is also running its own music services.
• To do so, Sonos is asking people to set their favorite music service in its app; if none is set, it will default to the most-used service, and consider other services as fallback options.
• That’s especially noteworthy because Sonos Voice Control won’t work with every service at launch: The voice assistant is capable of sending queries to Sonos Radio, Apple Music, Amazon Music, Deezer and Pandora, but isn’t working with Spotify just yet.

“We are working towards having them part of Sonos Voice Control,” promised Sonos voice experience director David Leroy, without providing additional details.

— Janko Roettgers

A version of this story first appeared on Protocol.com. Read it here.

A MESSAGE FROM ROKU

Ahoy, media professionals! Ready to dive into the entire streaming journey? A better TV storytelling experience starts with Roku Brand Studio. Chart a course and take streamers on a journey they’ll love today.

Learn more

In other news

The console industry’s chip shortage woes. Console makers bemoaned the ongoing semiconductor supply crunch, attributing lower-than-expected hardware sales in earnings this week to the ongoing component issues. “No end in sight,” were the words of Nintendo President Shuntaro Furukawa.

EA and FIFA break up. Electronic Arts and FIFA aren’t renewing their licensing contract after an extension expires next year, bringing an end to their three-decade gaming partnership. EA will rename the series EA Sports FC.

Ubisoft CEO tries to dispel buyout rumors. Ubisoft CEO Yves Guillemot told investors on Wednesday his company “has everything it needs to remain independent,” regarding recent rumors the company was attracting takeover bids from private equity firms.

Nintendo’s plans for a Switch successor. Nintendo thinks it has at least a couple more years of life in the Switch handheld, but the company said transitioning to a new generation of hardware is a “major concern,” given past mistakes surrounding the Wii and Nintendo DS.

TelevisaUnivision acquires Spanish-language streaming service. The U.S.-based streaming service Pantaya will help Univision build out its own ViX+ service, which is supposed to launch later this year.

Apple may restructure its services business. The company is considering reorganizing the business with a bigger focus on streaming, according to Business Insider.

Another game studio union, in Croatia. Croatian developer Gamechuck is the latest studio to have its own union after employees signed an agreement late last month, Game Developer reported. Similar efforts to unionize QA testers in the U.S. at Activision Blizzard and EA’s BioWare have been met with resistance.

Google is making a tablet again. The company teased a new Pixel tablet at its I/O developer conference, but people won’t be able to buy it until 2023.

Napster is back, again

You know what they say about cats: No, not that they’re smarter than dogs, but the whole multiple lives thing. That apparently also seems to extend to cat-themed startups, as Napster is proving this week. The former file-sharing network is slated to reemerge as a Web3-focused music service in the coming months.

Prior Napster iterations include everyone’s favorite source of MP3s; a never-launched legal music service owned by Bertelsmann; a streaming service owned by CD software maker Roxio; the same service, but owned by Best Buy; a European offshoot of the U.S. music service Rhapsody; a rebranded version of Rhapsody that was available to U.S. consumers as well; and most recently, a part of a U.K. startup looking to build immersive live music experiences.

If you do the math, this means that the Web3 Napster will actually be the seventh and perhaps final iteration before the brand finally fades into oblivion. Watch out before crossing the road, little cat!

— Janko Roettgers

A MESSAGE FROM ROKU

Want stellar performance from your media buy? Discover out-of-this-world reach with The Roku Channel and engage over 80 million people – more streamers than on any other platform – or planet. A better TV advertising experience starts with Roku, America’s #1 TV streaming platform.

Learn more