New Facebook research suggests audio will be key to its AR glasses

When Facebook eventually launches its own AR glasses, advanced audio features are likely going to be a major selling point. That's the main takeaway from new research coming out of Facebook Reality Labs, the company's recently renamed AR/VR division.

The division's research arm gave a group of journalists a sneak peek at their work this week, highlighting how it aims to both improve consumers' sense of presence in AR and VR through audio and give them the ability to block out background noise and amplify voices in bars and other loud social spaces. And while the researchers stressed that their work was in early stages, they also said that it was all about laying the groundwork for future AR products.

Sign up for Protocol newsletters

"Once you experience them, it's sort of like the first time you put on a VR headset and you said, 'Wow, this is just different from anything I've ever experienced,'" FRL Research Chief Scientist Michael Abrash said about the two features.

FRL Research Auditory Perception Lead Owen Brimijoin said it "feels like nothing short of magic." Part of that "magic" is making audio that is delivered over headphones indistinguishable from something happening in the room you're in, which could, for instance, make telepresence a lot more lifelike. To achieve this, audio needs to be spatial, meaning that it can be placed in a 3D space, as opposed to just mono or stereo.

Facebook has been using spatial audio to improve immersion for its VR headsets already, but the company's researchers stressed this week that a true sense of audio presence requires a lot more work. For instance, everyone's ear is shaped differently, which is why the company is working on using photographs and videos of people's ears to personalize spatial audio.

Facebook's researchers also outlined their work on "perceptual superpowers," which could turn AR glasses into a kind of situationally aware hearing aid. In essence, Facebook wants to use an array of microphones on its AR glasses to amplify the voice of someone you may be talking to in a crowded bar, while simultaneously blocking out any background noise.

Abrash acknowledged that this feature in particular may raise privacy concerns. "It is something that is very much on our minds as we develop it," he said, adding that Facebook could include safeguards like limiting the reach of such microphones.

However, there is also a potential flip side: By explicitly focusing on social situations, Facebook may have found a novel way to drive the adoption of AR glasses. Previous iterations, from Google Glass to Magic Leap's headset, have been very much focused on delivering features to the wearer of the device alone, prompting the backlash against "glassholes."

Giving wearers of Facebook's upcoming AR glasses the ability to hear others better, and in turn participate more in social settings, might take away a lot of that stigma, and ultimately turn them into more of a utilitarian product that could have a much larger potential customer base.