Get access to Protocol
Launching on June 23.
Another week in self-isolation, and another week of solid patents from big tech. Amazon is getting serious about automating the last leg of delivery. Alphabet is designing a seat-scrambling system for self-driving cars. A surprising number of patents discussed styluses, which may be coming back in a big way — and so might AOL Instant Messenger, too. A bunch of patents also dive into new uses for VR, which is a technology that seemed absolutely ripe to take off during this time when no one can go outside, but still hasn't. Maybe what it needed can be found in the files of the U.S. Patent and Trademark Office.
And remember: The big tech companies file all kinds of crazy patents for things, and though most never amount to anything, some end up defining the future.
Most companies working on autonomous vehicles like to talk about how they can rearrange the layout of the seats in the vehicle to make them more comfortable for the passengers, and more conducive to chatting or working than a regular car. But Waymo's new patent has another goal in mind: safety. The patent suggests a system in self-driving cars that would be able to dynamically change the seat layout. If the car sensed that a crash was imminent, it could quickly arrange seats to protect all the human passengers inside as much as possible. If the car has time to do this, one might wonder, why can't it just get out of the way of the crash?
If you've ever done anything in VR, you know that holding a device to replicate your hands works only OK. Google's new patent suggests a better alternative: your actual hands. It's not the first time someone has suggested using sensors to capture the motion of your hands to control virtual worlds — startup Leap Motion tried to turn that into a reality — but existing systems have had issues with latency and fidelity. Google's idea would apparently track each individual section of each finger to allow for more accurate tracking. Perhaps in the near future, using VR won't feel as clunky as trying to lead a blind bull through a china shop.
This is one of the most novel uses I've seen for a stylus in a long time. My handwriting is terrible, and I can't draw, so I have little use for styluses personally, but this Google patent might change my mind. It envisions being able to call on Google Assistant by drawing a circle around something on your screen in a specific way. In one of the examples in the patent, a user is chatting with someone who asks him for his flight number; instead of leaving the app and going to search for it, the person circles the question, which triggers the Assistant (and it being Google, it's already connected up to his Gmail), and it pulls up the answer. It seems a lot simpler than typing that into Google or asking the same question out loud, which always feels rather awkward, even when you're alone.
With everything that's going on right now, a lot of people probably wish a drone could deliver them groceries instead of having to battle it out in the real world or wait for a delivery slot to open up. But landing autonomous drones in someone's backyard without killing anyone or crashing into anything is a fair bit harder than it sounds. One solution Amazon appears to be looking at is placing radar devices at landing sites, which could bounce signals to a drone to let it know where to land or things to avoid. In one drawing in the patent, that just looks like a sheet with a massive "X" in the middle of it. Hopefully when the drone comes in for a landing, it'll say something like, "Ahoy, me mateys, here's yer treasure!"
Amazon was also awarded another patent this week for using image-based codes to direct drones around, which would probably be helpful for the aircraft when they're maneuvering their way out of an Amazon facility and off to deliver their booty.
Have you ever gone to Amazon's website and thought, "I really wish this was blown out into a series of modules across my entire field of view?" No? Weird. Well, a new patent from Amazon gives an indication of what shopping on the site might look like in VR. And much like every '90s skeuomorphic vision of future computer systems, the patent envisions virtually re-creating things like coffee shops or parks to do our shopping in. Then as you move your head around, you'd see all sorts of items floating in front of you to buy — it's a lot like that famous scene in "Minority Report," except instead of trying to prevent murders, you're stocking back up on toilet paper.
Modern Amazon warehouses are filled with hundreds of tiny robots that move giant racks of products around in silence so that humans can sort everything into boxes to send to you. Amazon's new patent imagines cutting out a few stages, namely: getting rid of those pesky humans. Instead, the patent suggests using robots to load shelves of products onto trucks, and then autonomously dropping them off at your driveway for you to take the products. Hopefully you're fine with sorting your own packages and having a rack sit in your driveway for a while.
One way to help self-driving cars navigate the world they're driving in is to provide them with maps of what they're going to see and let them plan their trip from there. Amazon's patent takes that idea a little further, using one autonomous vehicle to lay down a map of the route for a second vehicle to follow, like a set of train tracks. It would almost be like racing your ghost in Mario Kart, but for convoys of autonomous vehicles.
One of the great things about robots is that you can tell them to do multiple things at once. Amazon's patent suggests that while drones are out delivering you your packages, they can also be scanning the world below them to make better topographical maps. The patent also suggests that other types of aircraft, from personal drones to helicopters, could be doing the same thing while they're up in the sky. This probably won't do much to allay the worries of privacy advocates who aren't fans of the idea of drones constantly surveilling the world below them as they pass, though.
Apple was famously against styluses for the longest time — until other companies started making devices with styluses that people seemed to like. But the Apple Pencil, like just about every other stylus on the market today, suffers from the fact that it doesn't really feel like you're drawing with a pencil; you're rubbing a piece of plastic against a piece of glass. This new patent from Apple explores how you could change that with haptic feedback, either using small vibrations or magnets. That doesn't mean Steve Jobs wouldn't still hate it, though.
Coincidentally, Apple has another patent out this week that could replicate the sensation of real drawing in a totally different way: by just writing on whatever surface you like. In the patent, Apple outlines an input device (like a stylus) that can gauge things like pressure, when the stylus is being used to draw and when it's just sitting there. This could actually be really useful with a computer rather than a tablet. Most designers today use something like a Wacom tablet to digitize their drawings, but it'd be a lot simpler if you could take the same stylus you use on an iPad, pair it with your Mac, and start digitally drawing on a physical piece of paper in front of you.
A lanyard worthy of Jony Ive
Some of my favorite Apple patents — of which there are many! — are the design patents the company files for extremely basic stuff that it refuses to just buy from a store like anyone else. Why buy plant pots (like we saw last week) when you can get some of the best product designers in the world to use their time to do it instead? Same goes for a lanyard and part of a gift card this week. The lanyard has 17 people listed as inventors, including former Apple design chief Jony Ive. I would love to have seen what the meeting to design this thing was like.
Facebook got a patent this week that would provide crucial context about who people and organizations are on the platform. It suggests creating information snippets for search results. The example it gives is someone searching for "oculus new headset." The site returns a bunch of results that mention Facebook and Mark Zuckerberg, and you could get some helpful information if you scrolled over highlighted names. You'd learn that Mark Zuckberg is the CEO of the parent company of Oculus, which is indeed Facebook. This could be helpful in addressing some of the spread of false or misleading information on the platform. But I think my favorite part of this patent is all the references in the example it uses to employees of Oculus who are no longer at the company, like Brendan Iribe and Palmer Luckey, who left under … very different circumstances.
This is a funny one: It seems that at some point, Microsoft acquired a patent from AOL that's only been published this week. The listed inventor, Patrick Blattner, worked in product management for AOL at the start of the 2000s, and all the drawings in the patent clearly show AIM, rather than any Microsoft product, like MSN Messenger. The patent comes from a simpler time in internet communication, when we would use emoji on AIM to describe our moods — here, it outlines a system where emoji avatars can be changed based on the users' perceived moods. It's not clear why Microsoft would want to push ahead with this patent (other than just to have it), but some small part of me hopes that a future version of Microsoft Teams brings back away messages where we *~aLL tAlK LikE tHiS~*
Mike Murphy ( @mcwm) is the director of special projects at Protocol, focusing on the industries being rapidly upended by technology and the companies disrupting incumbents. Previously, Mike was the technology editor at Quartz, where he frequently wrote on robotics, artificial intelligence, and consumer electronics.