Power

This is how Facebook’s Project Aria AR glasses work

Facebook's experimental smart glasses use the same camera sensor as the Oculus Quest 2.

Facebook’s Project Aria smart glasses, Gemini

Facebook's Project Aria smart glasses are called Gemini.

Photo: Facebook/FCC

It's been close to a year since Facebook first announced Project Aria, a research effort to test smart glasses with a small group of employees. The idea behind the project was to have employees use these devices in the wild and collect a bunch of data to help the company figure out what kind of sensors it needs to add to future devices, executives said at the time.

There haven't been many public updates on the project in recent months, aside from a redacted FCC filing for the Project Aria glasses earlier this year. Now, we are getting a first look at how the Project Aria device actually works, thanks to newly published regulatory documents that include the device user manual.

  • Project Aria's hardware is known as Gemini. The device is being called "Gemini EVT," with EVT (engineering valuation test) being a common acronym in the hardware industry for small product test runs of a few dozen units, meant to test both the design and functionality of a product before it is put into actual production.
  • There is no visual AR component. The filing confirmed what Facebook executives have been telling us for some time: Project Aria is all about collecting data, not displaying it. In addition to camera sensors, the manual also mentions a proximity sensor on the inner temple.
  • Gemini Glasses are available with prescription lenses. "If your glasses have prescription lenses, they are intended only for use by the prescribed user," the user manual warns. "The Glasses are not intended for users whose vision cannot be corrected with standard corrective lenses."
  • There's a mobile companion app called Ariane that can be used to set up the device, connect it to a Wi-Fi network, check its battery status and upload collected sensor data. There's also mention of alerts through the mobile app, but it's unclear what these alerts may look like. The manual shows an iOS version of Ariane, and there's no word on whether Facebook has been testing the glasses with an Android companion app as well.
  • The glasses are equipped with four cameras, capable of recording both photos and videos. Photos of the individual device components show that Facebook used the same camera sensors as those that power the Oculus Quest 2. Videos are being recorded in the VRS file format, which contains captures of all four camera streams simultaneously.
  • The hardware interface is fairly pared-down. There's a shutter button, a power button and a mute switch, which "toggles privacy mode on/off," according to the manual. It also has multiple LEDs to signal recording both to the wearer as well as any bystanders.
  • Gemini glasses are using a Qualcomm chipset, and run a customized version of Android that is being called Oculus OS. The USB charging cable, which attaches to the glasses via a magnetic port, is also being used for Android ADB functionality.

Project Aria's Gemini glasses are clearly still an experimental device, and not meant for consumer use. However, the existence of a privacy mode, a relatively simple UI and the companion mobile app are all things that may find their way into future consumer products, including the smart glasses that Facebook is building in partnership with Ray-Ban.

The biggest surprise is perhaps that Facebook used the Quest's camera sensors for these glasses. This could have been a choice of convenience, as the company has experience with these cameras, and may have been able to use a lot of the data it has gained from developing the Quest to get Project Aria off the ground. However, using four of these sensors could also be a sign that Facebook is already testing visual SLAM for future device iterations that will enable full 3D AR experiences.

Climate

New Jersey could become an ocean energy hub

A first-in-the-nation bill would support wave and tidal energy as a way to meet the Garden State's climate goals.

Technological challenges mean wave and tidal power remain generally more expensive than their other renewable counterparts. But government support could help spur more innovation that brings down cost.

Photo: Jeremy Bishop via Unsplash

Move over, solar and wind. There’s a new kid on the renewable energy block: waves and tides.

Harnessing the ocean’s power is still in its early stages, but the industry is poised for a big legislative boost, with the potential for real investment down the line.

Keep Reading Show less
Lisa Martine Jenkins

Lisa Martine Jenkins is a senior reporter at Protocol covering climate. Lisa previously wrote for Morning Consult, Chemical Watch and the Associated Press. Lisa is currently based in Brooklyn, and is originally from the Bay Area. Find her on Twitter ( @l_m_j_) or reach out via email (ljenkins@protocol.com).

Every day, millions of us press the “order” button on our favorite coffee store's mobile application: Our chosen brew will be on the counter when we arrive. It’s a personalized, seamless experience that we have all come to expect. What we don’t know is what’s happening behind the scenes. The mobile application is sourcing data from a database that stores information about each customer and what their favorite coffee drinks are. It is also leveraging event-streaming data in real time to ensure the ingredients for your personal coffee are in supply at your local store.

Applications like this power our daily lives, and if they can’t access massive amounts of data stored in a database as well as stream data “in motion” instantaneously, you — and millions of customers — won’t have these in-the-moment experiences.

Keep Reading Show less
Jennifer Goforth Gregory
Jennifer Goforth Gregory has worked in the B2B technology industry for over 20 years. As a freelance writer she writes for top technology brands, including IBM, HPE, Adobe, AT&T, Verizon, Epson, Oracle, Intel and Square. She specializes in a wide range of technology, such as AI, IoT, cloud, cybersecurity, and CX. Jennifer also wrote a bestselling book The Freelance Content Marketing Writer to help other writers launch a high earning freelance business.
Entertainment

Watch 'Stranger Things,' play Neon White and more weekend recs

Don’t know what to do this weekend? We’ve got you covered.

Here are our picks for your long weekend.

Image: Annapurna Interactive; Wizard of the Coast; Netflix

Kick off your long weekend with an extra-long two-part “Stranger Things” finale; a deep dive into the deckbuilding games like Magic: The Gathering; and Neon White, which mashes up several genres, including a dating sim.

Keep Reading Show less
Nick Statt

Nick Statt is Protocol's video game reporter. Prior to joining Protocol, he was news editor at The Verge covering the gaming industry, mobile apps and antitrust out of San Francisco, in addition to managing coverage of Silicon Valley tech giants and startups. He now resides in Rochester, New York, home of the garbage plate and, completely coincidentally, the World Video Game Hall of Fame. He can be reached at nstatt@protocol.com.

Fintech

Debt fueled crypto mining’s boom — and now, its bust

Leverage helped mining operations expand as they borrowed against their hardware or the crypto it generated.

Dropping crypto prices have upended the economics of mining.

Photo: Lars Hagberg/AFP via Getty Images

As bitcoin boomed, crypto mining seemed almost like printing money. But in reality, miners have always had to juggle the cost of hardware, electricity and operations against the tokens their work yielded. Often miners held onto their crypto, betting it would appreciate, or borrowed against it to buy more mining rigs. Now all those bills are coming due: The industry has accumulated as much as $4 billion in debt, according to some estimates.

The crypto boom encouraged excess. “The approach was get rich quick, build it big, build it fast, use leverage. Do it now,” said Andrew Webber, founder and CEO at crypto mining service provider Digital Power Optimization.

Keep Reading Show less
Tomio Geron

Tomio Geron ( @tomiogeron) is a San Francisco-based reporter covering fintech. He was previously a reporter and editor at The Wall Street Journal, covering venture capital and startups. Before that, he worked as a staff writer at Forbes, covering social media and venture capital, and also edited the Midas List of top tech investors. He has also worked at newspapers covering crime, courts, health and other topics. He can be reached at tgeron@protocol.com or tgeron@protonmail.com.

Policy

How lax social media policies help fuel a prescription drug boom

Prescription drug ads are all over TikTok, Facebook and Instagram. As the potential harms become clear, why haven’t the companies updated their advertising policies?

Even as providers like Cerebral draw federal attention, Meta’s and TikTok’s advertising policies still allow telehealth providers to turbocharge their marketing efforts.

Illustration: Overearth/iStock/Getty Images Plus

In the United States, prescription drug advertisements are as commonplace as drive-thru lanes and Pete Davidson relationship updates. We’re told every day — often multiple times a day — to ask our doctor if some new medication is right for us. Saturday Night Live has for decades parodied the breathless parade of side effect warnings tacked onto drug commercials. Here in New York, even our subway swipes are subsidized by advertisements that deliver the good news: We can last longer in bed and keep our hair, if only we turn to the latest VC-backed telehealth service.

The U.S. is almost alone in embracing direct-to-consumer prescription drug advertisements. Nations as disparate as Saudi Arabia, France and China all find common ground in banning such ads. In fact, of all developed nations, only New Zealand joins the U.S. in giving pharmaceutical companies a direct line to consumers.

Keep Reading Show less
Hirsh Chitkara

Hirsh Chitkara ( @HirshChitkara) is a reporter at Protocol focused on the intersection of politics, technology and society. Before joining Protocol, he helped write a daily newsletter at Insider that covered all things Big Tech. He's based in New York and can be reached at hchitkara@protocol.com.

Latest Stories
Bulletins