A Unity demo of Facebook’s new VR features seats players from different locations at the same real-world table.
Image: Unity

How Facebook is merging VR with the real world

Protocol Next Up

Good morning, and welcome to Protocol Next Up. This week: Facebook's plans to make us feel more at home in VR, and Roku's first, timid embrace of IoT. Also: MySpace is alive and well in VR!

Facebook wants to make VR feel more real

It's a metaverse lovefest: As Facebook's Connect developer conference gets underway with its remote programming, everyone is eager for updates on the company's big metaverse plans. Facebook obliged by making a video to show off its vision for the metaverse.

Mark Zuckerberg previewed some of that thinking during an earnings call earlier this week, during which he laid out the ambitious goal of having a billion people participate in the metaverse before the end of the decade, and he reiterated some of those ideas during a call with reporters Tuesday. "The goal isn't that we're spending more time with technology, it's that the time that we do spend is more natural," Zuckerberg said during that call.

Much of that metaverse rhetoric still sounds like science fiction, and many of the actual product updates shared at Connect were a lot more mundane. It's a developer conference after all, and that crowd cares just as much about multiplayer SDKs and dev tools as it does about a 10-year vision. But one theme stood out among the many announcements: On its way to a metaverse that feels natural, Facebook is first looking to make VR more real.

  • To achieve this, Facebook is making pass-through video available as a standard feature to all developers. This allows people to build Quest apps that incorporate a video-view of the real world, and mix it with VR elements.
  • This new Insight SDK allows developers to use Spatial Anchors, which means that they can add persistent virtual objects to a real space. Put a virtual TV into your real living room, and it will be at the exact same spot the next time you put on your headset.
  • Developers can even prompt people to capture some aspects of their room, and then incorporate those objects into gameplay — kind of the VR version of AR, if you will.
  • Facebook built its own demo app, called The World Beyond, to show off what's possible when you mix VR with the real world. The app isn't available to end users just yet, but a video showed a person in their living room, playing fetch with an adorable little monster that was aware of and avoided any of the actual furniture, only to turn the walls of the room into portals into a 3D VR world.
  • Unity has developed its own mixed reality demo for the Quest. In Unity Slices: Table, as the demo is a bit awkwardly called, multiple players from remote locations can seat their avatars around a real table in a player's home, and then play chess against each other. The app will be made available to Unity developers soon, and should eventually make its way to ordinary people as well (you can get a glimpse of it in the image above).

Granted, pass-through is still hampered by the Quest's cameras, which make everything look like a fuzzy monochrome dream. But it's not hard to imagine that this will improve dramatically with future device generations. In the meantime, the company is doing a few more things to further blur the line between VR and the existing internet.

  • Facebook will soon allow Quest owners to make Messenger video calls with their non-VR buddies.
  • The company will allow developers to distribute progressive web apps to Quest users as a way to get them to use 2D apps and services. This will make it possible to check Instagram, Dropbox or Slack in VR, just like you would on your desktop PC or phone.
  • And because Facebook really wants working in VR to become a thing, the company is also relaunching its Quest for Business program. Instead of having to rely on dedicated business versions of the headset, companies will now be able to use off-the-shelf Quests, and allow employees to log in with their work accounts.

All of this isn't quite the metaverse yet, but anything that makes VR feel more real and natural could help get more people interested in the medium — which in turn can help Facebook justify the more than $10 billion it spent on these technologies in 2021 alone.


"One of the most effective competitors we have ever faced." — Mark Zuckerberg with a shout-out to TikTok during Facebook's earnings call this week.

"Minecraft and Fortnite are closer to the metaverse than anything Facebook has built, or frankly, is likely to build." — Facebook Reality Labs Consulting CTO John Carmack, being as frank as ever.


Meet less but collaborate more! Confluence is the alternative to endless meetings. Kickstart projects with templates, and stay in the loop with real time collaboration features. Clear meetings off your plate with Confluence. Get started for free!

Learn more

Roku takes baby steps toward an IoT future

IoT has become a major focus for smart TV platforms in recent years. Samsung, Google, Amazon and Apple have been busy integrating their smart TV platforms with internet-connected devices, allowing people to check and control light bulbs, thermostats and other things directly from the biggest screen in their home.

For Roku, IoT has been a bit of a blind spot. The company's streaming devices and smart TVs don't integrate with any of the existing smart home platforms, and even something as simple as viewing a security camera currently requires a third-party app.

Now, Roku is taking steps to bring IoT to its platform with the launch of a new Independent Developer Kit that allows developers to build apps with features currently not supported by the company's BrightScript scripting language.

  • Roku's IDK was announced at the company's annual developer conference this week, where Partner Management Director Sharana Math specifically called out "IoT apps to monitor and control smart devices" as one use case.
  • IDK apps can be written in C/C++, or other programming languages if developers include an interpreter in their apps.
  • IDK apps support real-time 3D graphics, standard OSS shared libraries and more. There are also some tie-ins into Roku media APIs, allowing app developers to, for instance, play DRM-protected media content.

However, there are some major caveats. The IDK is being rolled out as a non-commercial framework for developers and enthusiasts looking to tinker, not as something to build any actual products with.

  • IDK apps won't be distributed via the Roku channel store. Instead, they have to be side-loaded, which requires people to enable developer mode on their Roku devices first.
  • IDK apps also don't run on all Roku hardware out there. Most notably, developers won't be able to run these apps on Roku TVs.
  • Roku is pretty much telling developers that they're on their own with this: "While Roku will continue to support a robust ecosystem for video and music streaming channels, the IDK will be completely community-driven," the company said in a blog post.

Still, it's a first step in the right direction, and if recent hires are any indication, we might see a lot more support for IoT on the Roku platform soon. Until that happens, the IDK may be the next best thing for developers looking to explore what they may be able to offer millions of people who own Roku devices in the future.

Fast forward

On Protocol: Google threatens to pull YouTube from Roku devices. New Roku devices may lose access to YouTube's apps in December if the two companies can't strike a new distribution agreement.

Leaked setup videos show a possible Oculus Quest Pro headset. The videos are animated, so we don't quite know what the device will really look like or when it will come out.

Black consumers join the cord-cutting crowd. Black households held onto pay TV longer than others, but now they're giving up on massive cable bills as well.

On Protocol: Facebook wants to be a metaverse company. What does that mean for Facebook.com?Video didn't kill the radio star, and the metaverse might not kill Facebook's legacy apps, either.

Vice shuts down its TV network in France. The company will instead distribute programming via a free, ad-supported streaming channel on Samsung's TV Plus service.

Amazon is working on a "Project Mic" live audio app. The app will reportedly mix live discussions with tunes from Amazon Prime Music.

HBO Max expands in Europe. The streaming service is now available in Spain as well as the Nordics.

Spotify claims to be the biggest podcast service. The music service has more U.S.-based listeners than Apple Podcasts now, executives claimed this week.

Auf wiedersehen

The other day, while working on a story about Facebook's metaverse plans, I briefly went down memory lane, trying to remember all of Facebook's short-lived predecessors. You know, Friendster, MySpace, Bebo and the like. And ever since, I've had this weird idea: What if those guys, and not Zuckerberg, had built the metaverse? I imagine a MySpace VR world would have been loud, messy and a complete sensory overload, like living in a GIF. And then it dawned on me: MySpace VR exists! Only, it's called VRChat. Now, someone go and build Friendster VR please …


Meet less but collaborate more! Confluence is the alternative to endless meetings. Kickstart projects with templates, and stay in the loop with real time collaboration features. Clear meetings off your plate with Confluence. Get started for free!

Learn more

Thanks for reading — see you next week!

Recent Issues