Power

Microsoft wants to outsource your memories to AR

Plus autolocking doors, an Apple AR headset and other patents from Big Tech.

Microsoft wants to outsource your memories to AR

Apple's new AR headset patent design.

Photo: Courtesy of USPTO

Life is … strange … these days, so anything that can make it a little easier to get through the day would be very welcome. Coincidentally, many of the patents from Big Tech this week do just that. Google wants to make sure you don't forget to lock your doors or miss your appointments, Amazon wants to make sure your microphone works wherever you are, and Microsoft wants to make your camera easier to use. For these minor improvements to my future life, I thank you, Big Tech.

And remember: Companies file all kinds of crazy patents for things, and though most never amount to anything, some end up defining the future.

Alphabet

Automatically booking you an Uber

Sometimes, when you're running late and trying to juggle a million different things, you completely forget that you should've called for a cab five minutes ago, and now you're definitely going to be late for your new puppy's first vet appointment. (Purely hypothetical, definitely not my exact current situation.) Google's new patent could be a lifesaver for people who've entrusted the company with their calendar and whereabouts. It suggests that the assistant would be able to automatically book a ride for you when it's getting close to the time you need to leave, based on traffic. Where was this feature last week when I was sitting in unexpected traffic on the Brooklyn Bridge with a whining pup?!

Holding down the fort with Google

Apparently, in the world beyond big cities, people leave their doors unlocked. Sometimes even when they're not home. Google seems to want to add a little bit of security to small neighborhoods, though: Its new patent outlines a system for controlling smart locks based on your location. If you're home alone, you can have the door always automatically lock behind you. The patent also imagines a scenario where it's late at night and someone is approaching the front door while people are home — the lock could send a notification to the owner's phone asking whether they'd like to lock the door. Sounds handy, but also a lot more complicated than what we do in cities: keep your door locked all the time.

Amazon

A robot that sucks

Amazon received considerably fewer patents than usual this week, but there were still a few gems in the mix, including this one that at first glance looks like a lightsaber. It's in fact a robotic arm with a suction-cup attachment at the end. The suction cup is actuated, so it can pick up objects at different angles. Amazon has been working on figuring out how to automate the process of packing boxes at its warehouse for years — perhaps this is a step along that process.

Figuring out if it's too loud to use a microphone

As I write this, there is a van beeping as it backs out of a warehouse, a power saw going across the street, and a line of cars honking that do not seem to realize there's a big crane in the middle of my street. It's not the best environment for constant Zoom calls, if I'm honest. Perhaps in the future this patent could help me out: Using sensors built into devices, such as microphones, the patent suggests a future where devices would be able to tell us if the environment is too loud for us to be heard without annoying the people we're speaking to, or whether a voice assistant product like an Amazon Echo would be able to hear us. But hopefully all this working from home will encourage device manufacturers to actually start putting decent microphones ( and cameras!) into our products.

Apple

A whole building

In my time looking into Apple, it's won patents for all sorts of random things you see in its stores — the planters, the weird wooden boxes they have instead of seats, the Apple Watch display tables — but I can't remember ever seeing one for an entire Apple store before. But that's just what this patent is for, and it appears to be the very snazzy store built by Foster + Partners in 2015 in Hangzhou, China. I'm not sure if this means the company wants to take this design and replicate it in other locations, or if it's just so unique that it felt it needed the USPTO to have a record of its beauty for some reason.

Using an iPhone to power an AR headset

Apple is, at some point, likely going to release an augmented-reality headset. Most reports have suggested that it would be a standalone device that links up to the iPhone, much like the Apple Watch, but this new patent implies that Apple is at least considering making a headset that's powered by the phone itself. In the drawings, the phone would slot into the headset, and the screen would become the headset display. There would also be a small cutout for the camera on the front of the headset, allowing the wearer to see the world in front of them. Comparing the device to other phone-powered headsets on the market, it seems like this could be a way for Apple to keep the price down on its own headset, even if it would likely be sacrificing a fair bit in terms of weight and aesthetics.

Facebook

A pop-up over your News Feed

On the rare occasions that I log on to Facebook anymore, I almost always miss whatever important life events my friends have been going through, and instead see the last thing they posted about the election, Trump or some other argument I really don't want to jump into. In this patent, a user is shown a pop-up covering their News Feed for a post from a friend asking if anyone wants to get dinner soon. It's more timely and pertinent than the post it's covering ("what do you think of the nice people in the Star Gazing movie?" is an amazing made-up post, though), and so is likely more useful to the user at that exact moment. After they've looked at it, then they could get back to their diatribe about how Star Gazing is in fact propaganda or a global conspiracy.

Microsoft

Changing camera modes automatically

This new patent from Microsoft envisions making the process a little simpler with computer vision. In the patent, the camera system would be able to "see" what's in the photo and help determine what camera settings the user likely wants to use. For example, in the situation below, the camera recognizes the page of the book, and highlights it, and would then ask the user if they wanted to turn on document-scanning mode. This could be extremely useful for all those times you've wanted to capture a quick-moving scenario and randomly selected panoramic or slow-motion or some other unhelpful mode on your camera, and then completely missed the moment as you swipe over to the right setting.

Generating event details from messages

If you've ever tried to organize an event with people who use several different apps for their primary mode of communication, you've invariably lost the thread with someone at some point. This patent aims to make keeping track of the details a little easier. It envisions software that would live inside a phone or computer that could collect information from multiple different apps to create "event cards" to help you keep the details straight. You'd then be able to share that card with anyone else who was interested in joining. It also suggests a screen that would collect all your upcoming events, which in this example includes something called a "mud puddle," which I both do and do not want to know what that entails.

Saving an augmented reality

This patent is truly from the realm of fantasy — not because of the tech it outlines, but because it envisions the Milwaukee Brewers in Game 7 of the World Series. That being said, the idea in the patent is pretty neat: It suggests being able to save experiences watched in augmented reality so you could watch them again. The case in the patent shows someone watching this unrealistic baseball game with two other people, and recording the experience so they could go back and experience it exactly as they did when it happened. No longer will you have to ask the question "where were you?" when something momentous happens — you'll be able to play back the exact moment and re-create it in your glasses. Who needs memories anymore?

Fintech

Judge Zia Faruqui is trying to teach you crypto, one ‘SNL’ reference at a time

His decisions on major cryptocurrency cases have quoted "The Big Lebowski," "SNL," and "Dr. Strangelove." That’s because he wants you — yes, you — to read them.

The ways Zia Faruqui (right) has weighed on cases that have come before him can give lawyers clues as to what legal frameworks will pass muster.

Photo: Carolyn Van Houten/The Washington Post via Getty Images

“Cryptocurrency and related software analytics tools are ‘The wave of the future, Dude. One hundred percent electronic.’”

That’s not a quote from "The Big Lebowski" — at least, not directly. It’s a quote from a Washington, D.C., district court memorandum opinion on the role cryptocurrency analytics tools can play in government investigations. The author is Magistrate Judge Zia Faruqui.

Keep ReadingShow less
Veronica Irwin

Veronica Irwin (@vronirwin) is a San Francisco-based reporter at Protocol covering fintech. Previously she was at the San Francisco Examiner, covering tech from a hyper-local angle. Before that, her byline was featured in SF Weekly, The Nation, Techworker, Ms. Magazine and The Frisc.

The financial technology transformation is driving competition, creating consumer choice, and shaping the future of finance. Hear from seven fintech leaders who are reshaping the future of finance, and join the inaugural Financial Technology Association Fintech Summit to learn more.

Keep ReadingShow less
FTA
The Financial Technology Association (FTA) represents industry leaders shaping the future of finance. We champion the power of technology-centered financial services and advocate for the modernization of financial regulation to support inclusion and responsible innovation.
Enterprise

AWS CEO: The cloud isn’t just about technology

As AWS preps for its annual re:Invent conference, Adam Selipsky talks product strategy, support for hybrid environments, and the value of the cloud in uncertain economic times.

Photo: Noah Berger/Getty Images for Amazon Web Services

AWS is gearing up for re:Invent, its annual cloud computing conference where announcements this year are expected to focus on its end-to-end data strategy and delivering new industry-specific services.

It will be the second re:Invent with CEO Adam Selipsky as leader of the industry’s largest cloud provider after his return last year to AWS from data visualization company Tableau Software.

Keep ReadingShow less
Donna Goodison

Donna Goodison (@dgoodison) is Protocol's senior reporter focusing on enterprise infrastructure technology, from the 'Big 3' cloud computing providers to data centers. She previously covered the public cloud at CRN after 15 years as a business reporter for the Boston Herald. Based in Massachusetts, she also has worked as a Boston Globe freelancer, business reporter at the Boston Business Journal and real estate reporter at Banker & Tradesman after toiling at weekly newspapers.

Image: Protocol

We launched Protocol in February 2020 to cover the evolving power center of tech. It is with deep sadness that just under three years later, we are winding down the publication.

As of today, we will not publish any more stories. All of our newsletters, apart from our flagship, Source Code, will no longer be sent. Source Code will be published and sent for the next few weeks, but it will also close down in December.

Keep ReadingShow less
Bennett Richardson

Bennett Richardson ( @bennettrich) is the president of Protocol. Prior to joining Protocol in 2019, Bennett was executive director of global strategic partnerships at POLITICO, where he led strategic growth efforts including POLITICO's European expansion in Brussels and POLITICO's creative agency POLITICO Focus during his six years with the company. Prior to POLITICO, Bennett was co-founder and CMO of Hinge, the mobile dating company recently acquired by Match Group. Bennett began his career in digital and social brand marketing working with major brands across tech, energy, and health care at leading marketing and communications agencies including Edelman and GMMB. Bennett is originally from Portland, Maine, and received his bachelor's degree from Colgate University.

Enterprise

Why large enterprises struggle to find suitable platforms for MLops

As companies expand their use of AI beyond running just a few machine learning models, and as larger enterprises go from deploying hundreds of models to thousands and even millions of models, ML practitioners say that they have yet to find what they need from prepackaged MLops systems.

As companies expand their use of AI beyond running just a few machine learning models, ML practitioners say that they have yet to find what they need from prepackaged MLops systems.

Photo: artpartner-images via Getty Images

On any given day, Lily AI runs hundreds of machine learning models using computer vision and natural language processing that are customized for its retail and ecommerce clients to make website product recommendations, forecast demand, and plan merchandising. But this spring when the company was in the market for a machine learning operations platform to manage its expanding model roster, it wasn’t easy to find a suitable off-the-shelf system that could handle such a large number of models in deployment while also meeting other criteria.

Some MLops platforms are not well-suited for maintaining even more than 10 machine learning models when it comes to keeping track of data, navigating their user interfaces, or reporting capabilities, Matthew Nokleby, machine learning manager for Lily AI’s product intelligence team, told Protocol earlier this year. “The duct tape starts to show,” he said.

Keep ReadingShow less
Kate Kaye

Kate Kaye is an award-winning multimedia reporter digging deep and telling print, digital and audio stories. She covers AI and data for Protocol. Her reporting on AI and tech ethics issues has been published in OneZero, Fast Company, MIT Technology Review, CityLab, Ad Age and Digiday and heard on NPR. Kate is the creator of RedTailMedia.org and is the author of "Campaign '08: A Turning Point for Digital Media," a book about how the 2008 presidential campaigns used digital media and data.

Latest Stories
Bulletins