Power

Microsoft wants to record your emotions

Facebook IoT controllers, Apple triangle tablets, Amazon tracking you in IRL and other patents from Big Tech.

Microsoft wants to record your emotions

Your memories in AR.

Image: Microsoft/USPTO

Welcome to a special Monday edition of the Big Tech patent roundup! Hopefully you're reading this while waiting for your grill to heat up or socially distancing yourself at the beach. If not, go outside and enjoy the day off, then come back and learn about what Big Tech has in store for our future. Up this week, safer noise-canceling headphones, AR maps, flexible tablets and emotion-recording AR glasses. Maybe just stay at the beach, honestly.

And remember: The big tech companies file all kinds of crazy patents for things, and though most never amount to anything, some end up defining the future.

Alphabet

Staying safe with noise-canceling headphones

If you've ever worn a pair of noise-canceling headphones while out in the world, you'll know how easy it is to get lost in the music and … almost get hit by a truck while crossing the road. Google's new patent aims to make wearing headphones a little safer. The concept in the patent is to have sensors on the outside of headphones that can help glean what you're doing while wearing them: If you're out running in a busy downtown area with lots of traffic, the headphones could automatically switch between noise-canceling and a transparency mode so you're not hit with an unexpected surprise.

Identifying songs

The concept of using software to figure out what song is playing isn't new — even Google's own ability to listen to the world around a smartphone and pick up songs playing has been around for a couple years — but I have to give props to the patent artist here for the song they chose to illustrate this idea. If you somehow haven't heard "Sandstorm" by Darude, you must listen to it right now.

Amazon

Tracking the stores you go to

Sites like Amazon already track you around the web to show you ads and see where else you're shopping, but what if they did that in real life? A new patent from the retailer explores how devices could be used to track the movements of potential customers to greet them when they show up at a store, based on how they're traveling. This could be useful for Amazon's subsidiary Whole Foods, which offers grocery pickups (which are especially useful during this pandemic), but could also be used to offer targeted discounts or messages to shoppers as they show up to shop in the store.

Sending Ring videos to the cops

Ring's ties to law enforcement have been well-documented at this point, but this new concept seems to close the loop between doorbell owners and the cops. Amazon's patent is for a function in the Ring app where anyone picked up by a Ring doorbell could be recorded and have that video sent by the owner to the police with one button, or even call the cops right from the app. How convenient! Also: If someone who looks like this stick figure ever tried to rob your house, I wouldn't worry too much.

Apple

Fabric hardware and 'Power Glove'

Apple has had a few patents recently around interweaving electric circuitry into fabrics, and this week it was awarded two more. One of them deals with how to actually put circuits into individual fabrics, and the other is something that reminds me of the single greatest gaming peripheral of all time: the Power Glove. It's basically a mesh fabric glove that could be used to control things like VR in a way that would feel more natural than the controls on the market today. I can't wait to use it to control a 2D-racing game with a pair Wayfarers on.

Pyramid tablets

This is a pretty neat idea from Apple, for a device with a wraparound, flexible display that can be folded into any shape you want. This is an objectively useful concept, and I could see folding up a tablet or phone so it can prop itself up for video calls or for use as a laptop, and then unfolding it to read a book. But with one of the concepts in the patent, for the tablet folded into a triangular prism, I couldn't help but think of "The Office." Ten years ago, the show parodied Apple's iPad with The Pyramid, a triangular-shaped tablet that made very little sense, but is, after all, "the strongest shape ever constructed, a shape that fits all other shapes inside of it."

Augmented reality maps

Apple is trying hard to shed the early negative reactions people have had to its Maps program, recently adding in a ton of new features. It also seems it's trying to catch up with a feature that Google Maps has had for a while: augmented reality maps. This patent outlines overlaying directions onto the real world using an iPhone camera, where you could also search for locations from the AR function. Will this be the thing that gets Apple users to switch back from using Google Maps? Perhaps it'll take off when Apple does eventually release some sort of AR glasses that it's been working on for so long.

Facebook

Controlling IoT devices through Facebook

Can you imagine a world where you give Facebook access to the cameras, lights and other smart devices in your home? Facebook can. Its new patent explores what it would be like for the social network to have a function for controlling IoT devices. There could be some value in using a social network's graph to connect people together to see if they know each other and allow them to share IoT permissions via a social network, which the patent discusses, but it still seems like a stretch for Facebook to be the place where you control your home.

Microsoft

Recording the world, based on your emotions

This feels right out of "Black Mirror." This new Microsoft patent suggests using sensors to glean the emotional state of someone wearing AR glasses and using that to trigger video recording. This could have some very lovely outcomes — like parents managing to capture their baby's first steps before they could possibly reach for a phone — but it also opens up some more troubling possibilities, assuming it could even work. The patent outlines some other moods that could trigger video recording, including "focused, engaged, distracted, bored, sleepy, confused or frustrated." It's not too much of a stretch to see this being used to monitor students' or employees' attention, or other situations where perhaps you don't exactly want someone recording — like the bathroom, perhaps.

Measuring soil with computers

Microsoft loves to talk about AI and big data in its commercials (especially if Common is in them), but it seems those ads are more than just bluster. This patent deals with precision agriculture, the concept of using computer-monitoring hardware and software to understand more about the land you're trying to farm, getting more out of the same amount of land. It outlines a soil-surveying device, with a radio antenna, that could connect to software to monitor things like soil moisture and electrical conductivity, both of which are important to tell you about the health of the land you're trying to farm. In the future, your farm could well be running Windows.

Entertainment

'The Wilds' is a must-watch guilty pleasure and more weekend recs

Don’t know what to do this weekend? We’ve got you covered.

Our favorite things this week.

Illustration: Protocol

The East Coast is getting a little preview of summer this weekend. If you want to stay indoors and beat the heat, we have a few suggestions this week to keep you entertained, like a new season of Amazon Prime’s guilty-pleasure show, “The Wilds,” a new game from Horizon Worlds that’s fun for everyone and a sneak peek from Adam Mosseri into what Instagram is thinking about Web3.

Keep Reading Show less
Janko Roettgers

Janko Roettgers (@jank0) is a senior reporter at Protocol, reporting on the shifting power dynamics between tech, media, and entertainment, including the impact of new technologies. Previously, Janko was Variety's first-ever technology writer in San Francisco, where he covered big tech and emerging technologies. He has reported for Gigaom, Frankfurter Rundschau, Berliner Zeitung, and ORF, among others. He has written three books on consumer cord-cutting and online music and co-edited an anthology on internet subcultures. He lives with his family in Oakland.

Sponsored Content

Why the digital transformation of industries is creating a more sustainable future

Qualcomm’s chief sustainability officer Angela Baker on how companies can view going “digital” as a way not only toward growth, as laid out in a recent report, but also toward establishing and meeting environmental, social and governance goals.

Three letters dominate business practice at present: ESG, or environmental, social and governance goals. The number of mentions of the environment in financial earnings has doubled in the last five years, according to GlobalData: 600,000 companies mentioned the term in their annual or quarterly results last year.

But meeting those ESG goals can be a challenge — one that businesses can’t and shouldn’t take lightly. Ahead of an exclusive fireside chat at Davos, Angela Baker, chief sustainability officer at Qualcomm, sat down with Protocol to speak about how best to achieve those targets and how Qualcomm thinks about its own sustainability strategy, net zero commitment, other ESG targets and more.

Keep Reading Show less
Chris Stokel-Walker

Chris Stokel-Walker is a freelance technology and culture journalist and author of "YouTubers: How YouTube Shook Up TV and Created a New Generation of Stars." His work has been published in The New York Times, The Guardian and Wired.

Workplace

Work expands to fill the time – but only if you let it

The former Todoist productivity expert drops time-blocking tips, lofi beats playlists for concentrating and other knowledge bombs.

“I do hope the productivity space as a whole is more intentional about pushing narratives that are about life versus just work.”

Photo: Courtesy of Fadeke Adegbuyi

Fadeke Adegbuyi knows how to dole out productivity advice. When she was a marketing manager at Doist, she taught users via blogs and newsletters about how to better organize their lives. Doist, the company behind to-do-list app Todoist and messaging app Twist, has pushed remote and asynchronous work for years. Adegbuyi’s job was to translate these ideas to the masses.

“We were thinking about asynchronous communication from a work point of view, of like: What is most effective for doing ambitious and awesome work, and also, what is most advantageous for living a life that feels balanced?” Adegbuyi said.

Keep Reading Show less
Lizzy Lawrence

Lizzy Lawrence ( @LizzyLaw_) is a reporter at Protocol, covering tools and productivity in the workplace. She's a recent graduate of the University of Michigan, where she studied sociology and international studies. She served as editor in chief of The Michigan Daily, her school's independent newspaper. She's based in D.C., and can be reached at llawrence@protocol.com.

Workplace

It's OK to cry at work

Our comfort with crying at work has changed drastically over the past couple years. But experts said the hard part is helping workers get through the underlying mental health challenges.

Tech workers and workplace mental health experts said discussing emotions at work has become less taboo over the past couple years, but we’re still a ways away from completely normalizing the conversation — and adjusting policies accordingly.

Photo: Teerasak Ainkeaw / EyeEm via Getty Images

Everyone seems to be ugly crying on the internet these days. A new Snapchat filter makes people look like they’re breaking down on television, crying at celebratory occasions or crying when it sounds like they’re laughing. But one of the ways it's been used is weirdly cathartic: the workplace.

In one video, a creator posted a video of their co-worker merely sitting at a desk, presumably giggling or smiling, but the Snapchat tool gave them a pained look on their face. The video was captioned: “When you still have two hours left of your working day.” Another video showed someone asking their co-workers if they enjoy their job. Everyone said yes, but the filter indicated otherwise.

Keep Reading Show less
Sarah Roach

Sarah Roach is a news writer at Protocol (@sarahroach_) and contributes to Source Code. She is a recent graduate of George Washington University, where she studied journalism and mass communication and criminal justice. She previously worked for two years as editor in chief of her school's independent newspaper, The GW Hatchet.

Enterprise

Arm’s new CEO is planning the IPO it sought to avoid last year

Arm CEO Rene Haas told Protocol that Arm will be fine as a standalone company, as it focuses on efficient computing and giving customers a more finished product than a basic chip core design.

Rene Haas is taking Arm on a fresh trajectory.

Photo: Arm

The new path for Arm is beginning to come into focus.

Weeks after Nvidia’s $40 bid to acquire Arm from SoftBank collapsed, the appointment of Rene Haas to replace longtime chief executive Simon Segars has set the business on a fresh trajectory. Haas appears determined to shake up the company, with plans to lay off as much as 15% of the staff ahead of plans to take the company public once again by the end of March next year.

Keep Reading Show less
Max A. Cherney

Max A. Cherney is a senior reporter at Protocol covering the semiconductor industry. He has worked for Barron's magazine as a Technology Reporter, and its sister site MarketWatch. He is based in San Francisco.

Latest Stories
Bulletins