Apple wants to look under your skin

Plus medical AI stenographers, private collaborative docs, modular drones and other patents from Big Tech.

Apple wants to look under your skin

Apple's new patent outlines using "subepidermal features" — like the shape of your veins — to figure out if it's really you trying to unlock your phone.

Image: Apple/USPTO

The weekly Protocol patent roundup is here, which means it's time to kick back, relax and read about the zany things Big Tech now holds patents for.

There are a few trends that emerged this week that reflect some of Big Tech's ambitions: Google is interested in making voice assistants more useful, Amazon is trying to revolutionize shipping, Apple has some well-crafted gadgets, and Microsoft wants to make computing a little easier. And then there's Facebook, which was light on patents, but wants us all to see movie trailers customized to what we love.

And remember: The big tech companies file all kinds of crazy patents for things, and though most never amount to anything, some end up defining the future.


An AI stenographer for doctors

If you've ever had the misfortune of having to try to read your doctors' handwritten notes, you'll know that anything that can take the pen out of their hands and onto a screen would be super helpful. That's basically what Google is envisioning in this patent: an AI assistant that could listen in to doctors' appointments, taking down notes on everything the doctor and patient say. The system could generate a transcript in real time, and it could also pull out important keywords; if the patient said they felt feverish and had a sore leg, it would collate those as symptoms for the doctor to follow up on. Other startups are working on similar ideas, but they don't have the weight of Google's AI prowess behind them — or the stigmas, for that matter.

Making talking to voice assistants less awkward

It's pretty amazing that there are tiny devices that I can wear in my ears that allow me to talk to a virtual assistant that can call me a car, respond to my texts or answer my random questions. That being said, it's still extremely awkward to talk to virtual assistants in public — if you're wearing something like a pair of AirPods and start talking to Siri while you're on the subway, you look like you're talking to yourself. But this new patent might help, with the idea being that you could pull out your phone and talk to your assistant like you were taking a call, rather than yelling into the ether. The phone would recognize the specific motion you did to invoke the assistant while pulling it up, and trigger it right away. It's a neat idea, but I think my favorite part of this patent is that the diagram seems to suggest this suited man is in an infinite loop or looking for coffee. I guess on a long enough scale, though, many of us are.


Recyclable mailer envelopes

It might not be the most grandiose invention of all time, but it could have a big impact on the environment, given how many packages Amazon ships a day. The idea is simple: Many mailer envelopes are plastic on the inside and paper on the outside, which many recycling facilities can't process, sending them to the junk heap. Amazon's patent is for a flexible, all-cardboard model that uses various interconnected slats of cardboard to provide the cushioning on the inside of the package. Another green-friendly alternative: getting off your butt and going to the store for that dishwasher detergent.

Modular delivery drones

The future of package delivery looks a lot like high-tech Lego. Amazon's patent outlines a completely modular drone system, where every single aspect of the drone can be removed as needed. The system could act as a sort-of checklist to ensure a drone is ready before being deployed for a delivery, and it wouldn't take too much of a imagination jump to see an automated process in which a package makes its way down a conveyor belt, where it's attached to the drone's avionics unit, then its battery, then its propellor system, and then sent off on its way to your house. It could also cut down on repair costs: Instead of having to pull a whole drone out of operation if something's broken, you could just swap out the necessary part. Like I used to do when one of the Lego bricks went missing.


Vein-matching biometrics

Apple has been leading the charge of replacing passwords with biometric information (mainly, the details of your face) for a few years now, but this patent would be taking things deeper — literally. It outlines using "subepidermal features" — like the shape of your veins — to figure out if it's really you trying to unlock your phone. The patent suggests that this would be useful for more easily discerning truly identical twins, but I could see it also being helpful for the time we're all in now. The number of times I've recently tried to use Apple Pay while wearing a mask and not understanding why it wasn't working is … far too high. Maybe tech like this could help my phone realize it's me without my having to take my mask off. I also wanted to share this patent for the terrifying imagery it contains:

An Apple Kinect

This patent from Apple seems a lot like what it would look like if the company had released its own version of a Microsoft's Kinect motion-tracking sensor. The device in the patent tracks the gesticulations of the user in front of it, translating them into actions on the computer screen. It just so happens that Apple actually bought PrimeSense, the company originally behind the Kinect technology, back in 2013. It's reportedly been using the technology in its Face ID software (without the aforementioned veins), but it's possible that it could be working on some new ways of interacting with its other devices. And flailing wildly at my Apple TV would be a big improvement over the Apple TV remote, to be honest.

A cabinet

This is just a design patent for the cabinets Apple has started using in its retail stores, but it's very nice. I think it would look nice in my living room with my TV on top of it.


Individualized movie trailers

In a world — where Mike Murphy wants to go see this specific movie, only one man is up for the task. Hopefully the final product would be a little better than this, but Facebook is exploring the possibility of creating personalized trailers for movies and TV shows, playing up the parts of the content that the viewer of the trailer might be more interested in. If it's a buddy-cop race car movie, perhaps they would show one person more explosions, another person fast cars, and another some witty banter between partners where one is just one day away from retirement. Whatever the producers — and Facebook's algorithm — would think most appropriate.


A mouse stylus

Have you ever wished your pointy device were more like another pointy device? Your (extremely specific) prayers may be answered soon. Microsoft is apparently exploring what it would be like to put some sort of mouse cursor on the end of a stylus, seemingly rather like the red nub found on ThinkPad laptops. You could hold the stylus in your hand and use the mouse with your thumb, which might be useful if you're giving a presentation and need to point to something onscreen. Though it doesn't seem like this would be easier than, you know, just bringing a mouse along with you as well.

Private mode for collaborative documents

If you're anything like me, you often start another Google Doc or Word file because you don't want to your co-workers to see you go through the extremely awkward and arduous process of writing your dumb sentences until they absolutely have to, and then copying that text in a big block back into the collaborative doc you're all working on. But Microsoft's new patent could make things a little easier — and not leave you with hundreds of abandoned documents in your cloud folder: It envisions building in a privacy mode for each collaborator so they can all work in the same document, without anyone else having to see. It's a self-deprecator's dream!

Another blood pressure monitor

Following last week's patent, Microsoft is back with another patent for a blood pressure monitor built into a wearable. Is there a product on the horizon, or is this just securing old research? Either way, it's making my chest tight.


How I decided to leave the US and pursue a tech career in Europe

Melissa Di Donato moved to Europe to broaden her technology experience with a different market perspective. She planned to stay two years. Seventeen years later, she remains in London as CEO of Suse.

“It was a hard go for me in the beginning. I was entering inside of a company that had been very traditional in a sense.”

Photo: Suse

Click banner image for more How I decided seriesA native New Yorker, Melissa Di Donato made a life-changing decision back in 2005 when she packed up for Europe to further her career in technology. Then with IBM, she made London her new home base.

Today, Di Donato is CEO of Germany’s Suse, now a 30-year-old, open-source enterprise software company that specializes in Linux operating systems, container management, storage, and edge computing. As the company’s first female leader, she has led Suse through the coronavirus pandemic, a 2021 IPO on the Frankfurt Stock Exchange, and the acquisitions of Kubernetes management startup Rancher Labs and container security company NeuVector.

Keep Reading Show less
Donna Goodison

Donna Goodison (@dgoodison) is Protocol's senior reporter focusing on enterprise infrastructure technology, from the 'Big 3' cloud computing providers to data centers. She previously covered the public cloud at CRN after 15 years as a business reporter for the Boston Herald. Based in Massachusetts, she also has worked as a Boston Globe freelancer, business reporter at the Boston Business Journal and real estate reporter at Banker & Tradesman after toiling at weekly newspapers.

Sponsored Content

Great products are built on strong patents

Experts say robust intellectual property protection is essential to ensure the long-term R&D required to innovate and maintain America's technology leadership.

Every great tech product that you rely on each day, from the smartphone in your pocket to your music streaming service and navigational system in the car, shares one important thing: part of its innovative design is protected by intellectual property (IP) laws.

From 5G to artificial intelligence, IP protection offers a powerful incentive for researchers to create ground-breaking products, and governmental leaders say its protection is an essential part of maintaining US technology leadership. To quote Secretary of Commerce Gina Raimondo: "intellectual property protection is vital for American innovation and entrepreneurship.”

Keep Reading Show less
James Daly
James Daly has a deep knowledge of creating brand voice identity, including understanding various audiences and targeting messaging accordingly. He enjoys commissioning, editing, writing, and business development, particularly in launching new ventures and building passionate audiences. Daly has led teams large and small to multiple awards and quantifiable success through a strategy built on teamwork, passion, fact-checking, intelligence, analytics, and audience growth while meeting budget goals and production deadlines in fast-paced environments. Daly is the Editorial Director of 2030 Media and a contributor at Wired.

UiPath had a rocky few years. Rob Enslin wants to turn it around.

Protocol caught up with Enslin, named earlier this year as UiPath’s co-CEO, to discuss why he left Google Cloud, the untapped potential of robotic-process automation, and how he plans to lead alongside founder Daniel Dines.

Rob Enslin, UiPath's co-CEO, chats with Protocol about the company's future.

Photo: UiPath

UiPath has had a shaky history.

The company, which helps companies automate business processes, went public in 2021 at a valuation of more than $30 billion, but now the company’s market capitalization is only around $7 billion. To add insult to injury, UiPath laid off 5% of its staff in June and then lowered its full-year guidance for fiscal year 2023 just months later, tanking its stock by 15%.

Keep Reading Show less
Aisha Counts

Aisha Counts (@aishacounts) is a reporter at Protocol covering enterprise software. Formerly, she was a management consultant for EY. She's based in Los Angeles and can be reached at acounts@protocol.com.


Figma CPO: We can do more with Adobe

Yuhki Yamashita thinks Figma might tackle video or 3D objects someday.

Figman CPO Yuhki Yamashita told Protocol about Adobe's acquisition of the company.

Photo: Figma

Figma CPO Yuhki Yamashita’s first design gig was at The Harvard Crimson, waiting for writers to file their stories so he could lay them out in Adobe InDesign. Given his interest in computer science, pursuing UX design became the clear move. He worked on Outlook at Microsoft, YouTube at Google, and user experience at Uber, where he was a very early user of Figma. In 2019, he became a VP of product at Figma; this past June, he became CPO.

“Design has been really near and dear to my heart, which is why when this opportunity came along to join Figma and rethink design, it was such an obvious opportunity,” Yamashita said.

Keep Reading Show less
Lizzy Lawrence

Lizzy Lawrence ( @LizzyLaw_) is a reporter at Protocol, covering tools and productivity in the workplace. She's a recent graduate of the University of Michigan, where she studied sociology and international studies. She served as editor in chief of The Michigan Daily, her school's independent newspaper. She's based in D.C., and can be reached at llawrence@protocol.com.


Microsoft lays out its climate advocacy goals

The tech giant has staked out exactly what kind of policies it will support to decarbonize the world and clean up the grid.

Microsoft published two briefs explaining what new climate policies it will advocate for.

Photo by Jeremy Bezanger on Unsplash

The tech industry has no shortage of climate goals, but they’ll be very hard to achieve without the help of sound public policy.

Microsoft published two new briefs on Sept. 22 explaining what policies it will advocate for in the realm of reducing carbon and cleaning up the grid. With policymakers in the U.S. and around the world beginning to weigh more stringent climate policies (or in the U.S.’s case, any serious climate policies at all), the briefs will offer a measuring stick for whether Microsoft is living up to its ideals.

Keep Reading Show less
Brian Kahn

Brian ( @blkahn) is Protocol's climate editor. Previously, he was the managing editor and founding senior writer at Earther, Gizmodo's climate site, where he covered everything from the weather to Big Oil's influence on politics. He also reported for Climate Central and the Wall Street Journal. In the even more distant past, he led sleigh rides to visit a herd of 7,000 elk and boat tours on the deepest lake in the U.S.

Latest Stories