Microsoft’s master plan for consumer AR: Start with the plumbing

The company's new Mesh platform allows developers to share holograms and avatars across mixed-reality devices.

Microsoft’s master plan for consumer AR: Start with the plumbing

Ever since launching the original HoloLens in 2016, Microsoft executives have hinted at plans to eventually build consumer AR hardware and services.

Image: Microsoft

Microsoft unveiled an ambitious new AR/VR platform called Mesh on Tuesday that allows developers to add avatar synchronization, spatial audio, shared holograms and persistent virtual spaces to any app. At the company's Ignite conference, executives demonstrated Mesh with a virtual keynote held in AltSpace, Microsoft's social VR world that is being powered by the platform going forward. Developers will also be able to build Mesh-powered apps for VR headsets, including Facebook's Oculus Quest, and desktop PCs. Support for Android and iOS will be added in the near future.

Ultimately, Microsoft plans to turn Mesh into something like the company's Xbox Live service: the plumbing that allows developers to add multi-user functionality to their own AR and VR apps at scale. And while Microsoft's own bets may be focused on the enterprise at first, the company clearly hopes that third-party developers will extend its reach to millions of consumers. "We do think this is the future of computing, beyond just enterprise customers," Microsoft's mixed reality director, Greg Sullivan, said.

Some of that expanded focus was apparent during Tuesday's keynote, which included guest appearances by James Cameron and Niantic CEO John Hanke, who demonstrated a proof-of-concept demo of a Mesh-powered Pokemon Go game using HoloLens.

Microsoft's own first test case for Mesh is a little closer to heart: The company unveiled a new HoloLens collaboration app, internally code-named Fenix but ultimately released under the Microsoft Mesh moniker, that allows teams to meet for design reviews in AR. During a demo given to a small group of journalists ahead of Ignite, Sullivan used the app to place shared holograms on a virtual table that participants could then hand back and forth, mark up, and discuss with spatial audio, all while being represented with AltSpace-like avatars.

At one point during the demo, Sullivan opened a simulated Teams call in a 2D window to show how cross-platform support with participants joining from desktop devices may look. Mesh's built-in 3D tracking allowed him to treat that 2D window as a kind of virtual camera, and move it around a hologram to show the desktop participant all angles of the virtual object.

For Microsoft, Mesh (Mesh the app, not to be confused with Mesh the platform) is more than just a demo; it's a hint of things to come. "The future default experience of HoloLens is multi-user collaboration," Sullivan said, suggesting that the home screen of the device itself could become a Mesh-like collaboration environment. And the company's plans don't stop there. "We will Mesh-enable our first-party Microsoft apps," he said. One prime candidate is Teams, the company's Slack-like collaboration service. Company executives didn't share a timeline for bringing AR meetings to Teams just yet, but Sullivan made it clear that it is part of the roadmap. "That will happen," he said.

Bringing Teams calls to AR could pit Microsoft against startups like Spatial, which has developed its own AR collaboration software. However, Sullivan singled out Spatial as an example for a company that could ultimately benefit from Mesh as well. "The Spatial guys spent a non-trivial percentage of their resources building the plumbing," he said. "We can relieve them of the burden of the plumbing and help them differentiate on the things that really are special about their offering."

While Microsoft may initially focus on Spatial-like services for HoloLens, Mesh is clearly meant for much broader use cases. Developers are getting access to Mesh for free, and cross-platform support is supposed to further accelerate adoption. "It's not locked, and I don't have to buy one company's hardware, or one company's OS," Sullivan said. Part of that cross-platform approach has been the decision not to reinvent the wheel on mobile. "We're not asking people to kind of rip out their native APIs and replace them with ours," he said. "We'll use ARKit and ARCore for anchoring and mapping."

Sullivan suggested that developers could use Mesh to create immersive social apps like Rec Room, and he likened the platform to Xbox Live, Microsoft's multiplayer game service. "You could do multiplayer network gaming before, but you had to build all the plumbing yourself. Quake and Doom, they did it, but that doesn't scale," he said. "Now, any application that wants to, will be able to have that capability."

Ever since launching the original HoloLens in 2016, Microsoft executives have hinted at plans to eventually build consumer AR hardware and services as well. "We think this is the future of computing," Sullivan said. "And implicit in that statement is, yeah, a consumer thing."

How Microsoft would get to that future has long been less clear. The HoloLens 2, which was unveiled in 2019, is an impressive piece of hardware with hand- and eye-tracking. It's also still a pretty bulky headset, with a hefty $3,500 price tag that is clearly made for the enterprise.

At the same time, the company has struggled to establish itself as a player in the consumer AR/VR space. Partnerships with companies like Lenovo, Asus, Dell and Samsung to build VR headsets powered by Microsoft's mixed-reality software largely fizzled, with only HP sticking around to iterate on compatible hardware. Now, Microsoft is now adding an interesting twist: a cloud-based platform that could power the AR apps of the future, whether they run on AR glasses made by Microsoft, Facebook or even Apple.

"One of the discussions that we've had internally has been: How much hinting, how much teasing do we do about how we think about this?" Sullivan said.

When it comes to Microsoft's vision for the future of AR beyond the enterprise, Mesh may have been the clearest hint yet.

Clarification: This story was updated on March 2 to clarify that Fenix was the internal code name for Mesh.

LA is a growing tech hub. But not everyone may fit.

LA has a housing crisis similar to Silicon Valley’s. And single-family-zoning laws are mostly to blame.

As the number of tech companies in the region grows, so does the number of tech workers, whose high salaries put them at an advantage in both LA's renting and buying markets.

Photo: Nat Rubio-Licht/Protocol

LA’s tech scene is on the rise. The number of unicorn companies in Los Angeles is growing, and the city has become the third-largest startup ecosystem nationally behind the Bay Area and New York with more than 4,000 VC-backed startups in industries ranging from aerospace to creators. As the number of tech companies in the region grows, so does the number of tech workers. The city is quickly becoming more and more like Silicon Valley — a new startup and a dozen tech workers on every corner and companies like Google, Netflix, and Twitter setting up offices there.

But with growth comes growing pains. Los Angeles, especially the burgeoning Silicon Beach area — which includes Santa Monica, Venice, and Marina del Rey — shares something in common with its namesake Silicon Valley: a severe lack of housing.

Keep Reading Show less
Nat Rubio-Licht

Nat Rubio-Licht is a Los Angeles-based news writer at Protocol. They graduated from Syracuse University with a degree in newspaper and online journalism in May 2020. Prior to joining the team, they worked at the Los Angeles Business Journal as a technology and aerospace reporter.

While there remains debate among economists about whether we are officially in a full-blown recession, the signs are certainly there. Like most executives right now, the outlook concerns me.

In any case, businesses aren’t waiting for the official pronouncement. They’re already bracing for impact as U.S. inflation and interest rates soar. Inflation peaked at 9.1% in June 2022 — the highest increase since November 1981 — and the Federal Reserve is targeting an interest rate of 3% by the end of this year.

Keep Reading Show less
Nancy Sansom

Nancy Sansom is the Chief Marketing Officer for Versapay, the leader in Collaborative AR. In this role, she leads marketing, demand generation, product marketing, partner marketing, events, brand, content marketing and communications. She has more than 20 years of experience running successful product and marketing organizations in high-growth software companies focused on HCM and financial technology. Prior to joining Versapay, Nancy served on the senior leadership teams at PlanSource, Benefitfocus and PeopleMatter.

Policy

SFPD can now surveil a private camera network funded by Ripple chair

The San Francisco Board of Supervisors approved a policy that the ACLU and EFF argue will further criminalize marginalized groups.

SFPD will be able to temporarily tap into private surveillance networks in certain circumstances.

Photo: Justin Sullivan/Getty Images

Ripple chairman and co-founder Chris Larsen has been funding a network of security cameras throughout San Francisco for a decade. Now, the city has given its police department the green light to monitor the feeds from those cameras — and any other private surveillance devices in the city — in real time, whether or not a crime has been committed.

This week, San Francisco’s Board of Supervisors approved a controversial plan to allow SFPD to temporarily tap into private surveillance networks during life-threatening emergencies, large events, and in the course of criminal investigations, including investigations of misdemeanors. The decision came despite fervent opposition from groups, including the ACLU of Northern California and the Electronic Frontier Foundation, which say the police department’s new authority will be misused against protesters and marginalized groups in a city that has been a bastion for both.

Keep Reading Show less
Issie Lapowsky

Issie Lapowsky ( @issielapowsky) is Protocol's chief correspondent, covering the intersection of technology, politics, and national affairs. She also oversees Protocol's fellowship program. Previously, she was a senior writer at Wired, where she covered the 2016 election and the Facebook beat in its aftermath. Prior to that, Issie worked as a staff writer for Inc. magazine, writing about small business and entrepreneurship. She has also worked as an on-air contributor for CBS News and taught a graduate-level course at New York University's Center for Publishing on how tech giants have affected publishing.

Enterprise

These two AWS vets think they can finally solve enterprise blockchain

Vendia, founded by Tim Wagner and Shruthi Rao, wants to help companies build real-time, decentralized data applications. Its product allows enterprises to more easily share code and data across clouds, regions, companies, accounts, and technology stacks.

“We have this thesis here: Cloud was always the missing ingredient in blockchain, and Vendia added it in,” Wagner (right) told Protocol of his and Shruthi Rao's company.

Photo: Vendia

The promise of an enterprise blockchain was not lost on CIOs — the idea that a database or an API could keep corporate data consistent with their business partners, be it their upstream supply chains, downstream logistics, or financial partners.

But while it was one of the most anticipated and hyped technologies in recent memory, blockchain also has been one of the most failed technologies in terms of enterprise pilots and implementations, according to Vendia CEO Tim Wagner.

Keep Reading Show less
Donna Goodison

Donna Goodison (@dgoodison) is Protocol's senior reporter focusing on enterprise infrastructure technology, from the 'Big 3' cloud computing providers to data centers. She previously covered the public cloud at CRN after 15 years as a business reporter for the Boston Herald. Based in Massachusetts, she also has worked as a Boston Globe freelancer, business reporter at the Boston Business Journal and real estate reporter at Banker & Tradesman after toiling at weekly newspapers.

Fintech

Kraken's CEO got tired of being in finance

Jesse Powell tells Protocol the bureaucratic obligations of running a financial services business contributed to his decision to step back from his role as CEO of one of the world’s largest crypto exchanges.

Photo: David Paul Morris/Bloomberg via Getty Images

Kraken is going through a major leadership change after what has been a tough year for the crypto powerhouse, and for departing CEO Jesse Powell.

The crypto market is still struggling to recover from a major crash, although Kraken appears to have navigated the crisis better than other rivals. Despite his exchange’s apparent success, Powell found himself in the hot seat over allegations published in The New York Times that he made insensitive comments on gender and race that sparked heated conversations within the company.

Keep Reading Show less
Benjamin Pimentel

Benjamin Pimentel ( @benpimentel) covers crypto and fintech from San Francisco. He has reported on many of the biggest tech stories over the past 20 years for the San Francisco Chronicle, Dow Jones MarketWatch and Business Insider, from the dot-com crash, the rise of cloud computing, social networking and AI to the impact of the Great Recession and the COVID crisis on Silicon Valley and beyond. He can be reached at bpimentel@protocol.com or via Google Voice at (925) 307-9342.

Latest Stories
Bulletins