Four ways mass-market consumer AR could become reality

The Big Story

The future of AR & VR, according to Qualcomm

Apple, Facebook, Snap, Google, Microsoft, Magic Leap and numerous other hardware makers: There is no shortage of companies working on AR headsets and glasses these days. Some have embraced the enterprise as an on-ramp for future consumer products. Others are embracing consumer headgear, including VR headsets and smart glasses that may not offer true AR experiences for years to come.

As different as these products are, there's one thing that many have in common: Qualcomm chips. The company's Snapdragon processors are already powering some 45 headsets from a wide variety of manufacturers across enterprise and consumer AR and VR, giving the company unique insights into the evolution of this nascent market.

That's why I caught up with Qualcomm VP and GM of XR Hugo Swart, who told me that he sees the road to mass-market consumer AR developing on four parallel tracks:

• VR headsets with video pass-through: Facebook's Quest already offers a rudimentary version of this, allowing users to see a grainy version of the real world in VR. That image quality is going to get better, and deliver high-fidelity video for immersive AR — but you won't be wearing these headsets on the street. "We will have really cool experiences, but it will be indoors," Swart said.
• Enterprise headsets like Microsoft's HoloLens: These devices are still far too bulky for the average consumer, but that's likely going to change over time. "Even for enterprise, you need to make them lighter," Swart told me.
• Smart glasses: Glasses like the Snap's early Spectacles, Bose's audio sunglasses or Facebook's upcoming collaboration with Ray-Ban don't offer true AR immersion, but they do foreshadow where things are going. "It's a first step in that direction," Swart said.
• Wireless viewers: Wearables like Nreal's smart glasses have been striking a compromise between immersion and comfort by outsourcing compute to phones or dedicated processing units. Future iterations could use Wi-Fi 6E to connect wirelessly to phones or PCs, or tap directly into the cloud via 5G. "Within the next couple of years, we will see a lot of progress" on this front, Swart predicted.

Eventually, a number of these paths could converge for a true consumer AR device. Swart didn't want to predict when that will be, but he told me that Qualcomm sees AR/VR as a massive opportunity. "Over the next few years, it's going to grow exponentially," he said, adding that AR devices could ultimately become as ubiquitous as phones are today. "Every person on the planet is likely going to have one," he said.

Overheard

"I would love to have a health insurance program. We don't have it yet." —Patreon CEO Jack Conte about things his company could offer creatives in the future.

"Maybe. That sounds right." —Roku CEO Anthony Wood, when asked in an interview to confirm that he is 56.

A MESSAGE FROM MICRON

Recently, Micron announced new memory and storage innovations across its portfolio based on its industry-leading 176-layer NAND and 1α (1-alpha) DRAM technology. But what does "1α" mean, and just how amazing is it?

Learn more

Watch Out

Adobe's new 3D tools help creatives generate virtual photos

Add photography to the long list of things that may never be the same after the pandemic: When photo studios and agencies closed their doors last year, a number of companies turned to 3D graphics and virtual photography to create photorealistic assets and designs.

Adobe is betting that this trend will continue even as pandemic restrictions are lifted. The company released a new suite of 3D tools Wednesday that aims to help creatives in a wide variety of industries embrace virtual photography and 3D design.

Adobe VP of 3D and Immersive Sébastien Deguy told Protocol that the software could change how companies market and design products, and perhaps ultimately even lead to the creation of entirely new types of products.

• Adobe Substance 3D consists of four applications. With these individual tools, designers can create 3D objects, apply textures and materials to these objects, create materials based on real-world images and stage objects in what amounts to virtual versions of real-world studio sets.
• Substance 3D also taps into the company's online 3D asset library, and each tool is closely interconnected. Designers can, for instance, create an object in Substance 3D Painter, and then send it with one click to 3D Stager to incorporate it into a virtual environment.
• There's a bunch of cool technology working under the hood. 3D Sampler, for instance, uses AI to create 3D objects based on 2D photos. That way, a flat photo of a brick wall can be transformed into a 3D-rendered asset with depth and shadows that responds to virtual lights, ready to be incorporated into a digital 3D scene.

The pandemic definitely increased demand for these kinds of tools by forcing designers to change their process. "They work from home," Deguy said. "They have to work more collaboratively."

• Even before the pandemic, virtual photography emerged as a way for companies to create photorealistic digital assets at scale. Ikea, Ben & Jerry's and Lowe's have all embraced it.
• And those product photos released each year by big tech companies for their new gadgets? They're almost certainly created digitally as well. "It is indiscernible" from photos taken in studios, Deguy said about virtual photography.

But there's more in store for digital 3D designs. Companies will also be able to use these tools during the product design process, Deguy explained, and they will be able to use the resulting imagery to market products differently. One example: 3D objects can more easily be turned into augmented-reality assets, which consumers can then place into their own living rooms, and perhaps one day explore with dedicated AR glasses.

"The demand for 3D tools is only going to grow," Deguy said. "Life is in 3D, and now, creativity is in 3D, too."

A version of this story first appeared on Protocol.com.

Fast Forward

• Nielsen started publishing new streaming metrics. Interestingly, the company got some buy-in from Netflix, which traditionally has been shy about sharing data.
• Nvidia's Shield streamer got a UI update. It's a minor tweak for now, but part of a bigger transition for Android TV toward a content-first experience.
• On Protocol: TikTok is launching TV apps to conquer the living room. Experts think adapting the service to the TV screen won't be easy.
• Is Roku the Netflix of streaming players? Netflix's DNA may have rubbed off on the hardware maker, which recently started to dabble in original content.
• Univision will launch a paid streaming service next year. The new service will be available worldwide, and there will also be a free tier.
• A gaming startup quit Facebook's VR ad test. Blaston was supposed to be the first game on Quest featuring Facebook ads. Then, the inevitable backlash happened. But Resolution Games said it might trial the ads on another game.
• On Protocol: Can China's Kuaishou succeed where Quibi failed? Its short-form dramas are already attracting 210 million viewers a day, so maybe.
• The FTC is looking at Amazon's proposed MGM acquisition. Maybe Jeff Bezos won't star in the next Bond film after all…

Auf Wiedersehen

The other day, a Roku PR rep sent me a list of Prime Day deals, but every link in her email pointed to the websites of Target or Best Buy. I found this puzzling — but then I remembered that "Prime Day" has become a universal shopping holiday not limited to Amazon. Some of the big chains call it "Deals Days" so as not to offend Jeff Bezos and his lawyers, but smaller online stores seem to have no such concerns, and proudly advertise their own Prime deals. I kind of like that, and hope that people will start to reclaim other holidays as well, corporate or not. Student Body Presidents Day, anyone? Block Friday, a day to celebrate all things Lego? I for one can't wait for National Donot Day, where we all just get to lay around and not move a finger.