Big tech’s big patents: The smartest home of all?

This week we found patents and applications from the big tech companies that suggest they all have a few similar things on their minds. Apple and Microsoft are thinking about new wearable devices, while Amazon and Apple are both thinking about how the internet of things affects where we live and shop. Then there's Microsoft, which is patenting a balloon on a string. Alphabet has surprisingly been absent from the U.S. Patent and Trademark Office's new filings for the last few weeks, but there are more than enough zany ideas from the other tech giants this week.

Get what matters in tech, in your inbox every morning. Sign up for Source Code.

Apple

Projecting directions onto the real world

Everyone's struggled to follow directions while driving, and it seems Apple wants to do something about it. The company was awarded a patent for a system that could use a vehicle's headlights to project direction signals (like a big right-hand-turn arrow) onto the road ahead of the driver. It's kind of like a bigger version of heads-up displays in some vehicles, only the info is projected on the road, not the windshield. This tech could also be useful for pedestrians unsure of what a car is about to do, both for drivers who forget to use their blinkers, but also for self-driving cars telegraphing their intentions when there's no driver aboard. Speaking of Apple's work in autonomy …

Optics systems for Lidar

Apple's self-driving car ambitions are reportedly focused on the software that controls autonomous vehicles, but it's also patented a proprietary system for the sensors these vehicles need to perceive the world. Apple received a patent this week for an optical system setup that could be used in a Lidar sensor. Alexander Shpunt is listed as an inventor on the patent — he was the former CTO of PrimeSense, the Israeli company that designed the Microsoft Kinect, which Apple bought in 2013 and likely used to develop its FaceID technology. Another inventor listed on the patent is Chandra Kakani, who is now an optical engineer at Waymo.

Making the Apple Watch much easier to use

This is a somewhat surprising patent, as it straight up admits that interacting with small touch screens — just like the Apple Watch shown in the patent's drawings — is "generally cumbersome and inefficient." The patent suggests incorporating gesture controls that use built-in motion sensors, such as tilting your wrist left or right to answer or ignore a call, or flicking your wrist to hang up. This would be great news, because as Apple suggests in the patent, "some existing techniques use a complex and time-consuming user interface, which can include multiple button presses or finger touches."

The all-knowing home

Apple's patent application imagines a word where every smart-home device can talk to each other, regardless of manufacturer or ecosystem. The system it envisions, built into a user's house, would be able to detect or infer every new type of smart gadget plugged into a socket, and try to automatically add it to the smart-home system. Plug in a smart light switch in the living room for example, and the system will know to let that device control the lights in that room. The system could also guess which inanimate objects (like couches or chairs) and people are in a room. One example it spells out for the brave new smart world is the system detecting a ding in the kitchen, inferring that it's popcorn in the microwave that's just finished, and detecting that "movie night" is on the homeowner's calendar, so the system automatically turns on the TV in the living room, dims the lights, closes the blinds, and even silences phones in the room. Just don't let it have control over the pod bay doors.

Alphabet

Google and its sister companies have been pretty quiet on the patent front the last few weeks. Google itself wasn't awarded a single patent that was published this week — or last.

Amazon

Making augmented reality feel a bit more … real

Amazon launched its AR viewer for hundreds of products it sells over two years ago, so shoppers can see how stuff would look in their houses. But the feature leaves a lot to be desired. This week, the company was awarded a patent that might be able to help: It outlines using infrared sensors to figure out the reflectiveness of surfaces an AR-capable device is looking at, which would give the AR objects better shadowing or shading, making them look more like they're part of the world. Amazon doesn't make a phone (anymore!), and most phones and tablets rely on traditional cameras for AR programs, but the patent suggests this could work on "smart glasses or contacts, AR goggles, and the like." Maybe this is what comes next after the connected frames it launched last year?

Facebook

A more complete Facebook Messenger app?

Facebook rolled out a redesign to Messenger over the course of late 2018 and 2019, which freshened up the design and added in stories and consolidated a lot of the disparate functions into three tabs. A series of design patents it was awarded this week suggests there might be more changes ahead. One patent shows how photo albums could be shared in group chats, another shows how a shared calendar in a group chat might look, and another appears to be for a video chat with the calendar function sitting on top of it. This seems perfect for anyone who wants Facebook to know everything that they and all their friends are doing at all time.

Microsoft

A massive wearable

Microsoft hasn't had a lot of luck in wearables — it killed off its Band line last year — but that hasn't stopped the company from patenting a new one. The new wearable idea puts a massive, curved display on your wrist, where tilting the display up or down could reveal info from notifications (wait, this sounds familiar!). It also suggests a strap made of "detachable modular segments" that you could add or remove as you wanted. They list all sorts of things the segments could contain, including extra sensors, a SIM card, batteries, a microphone, even "an auxiliary display." It's reminiscent of the failed mobile concept Project Ara but on your wrist.

Affordable aerial photography

Who needs a drone when you've got a camera on a balloon tied to a string? This is one of the more bizarre patents I've ever seen, but in fairness, it does live up to its title: "Low-cost, long-term aerial imagery." Just fill a balloon up with helium, attach it to a tether on your camera (or have a man dressed like Mr. Rogers just hold it), and there you go. Stunning. In reality, these would likely be lightweight balloons that could be sent far up into the atmosphere to take high-resolution photos of the world below, like Google Earth meets Loon. Microsoft notes that aerial photography like this has a number of uses in research and industry, including disaster assessment and agricultural analytics.

Clean up on aisle four!

Spills and broken containers are big issues for any grocery store, and some have even started employing cumbersome robots to be on the lookout for liquids on the floor. Microsoft has applied for a patent for a simpler solution: a computer-vision algorithm that lives in ceiling-mounted cameras. The application suggests using UV lights and regular cameras to detect variations in the expected surface of a store's floor. Microsoft partnered with Kroger early last year to enhance the shopping experience, and press at the time suggested cameras could be used to see whether items were out of stock (sounds familiar), but these cameras could also run double duty to make sure no one slips.