Power

Microsoft wants to use AR to see through fog and smoke

Apple autonomous cars, AI coffee orders, emailing help and other patents from Big Tech.

Microsoft wants to use AR to see through fog and smoke

See what isn't there.

Image: Microsoft/USPTO

It's beyond dark out at 5:30 p.m. these days, so perhaps, as you're stuck at home with nowhere to go, you're tempted to log off your bad screen and onto your good screen a little earlier than you should. Perhaps that's what happened over at the U.S. Patent and Trademark Office, as this was a bit of a fallow week for patents from Big Tech.

That being said, there were still a few neat ones out there: Microsoft is looking into using AR to actually augment what you see; Apple is hard at work on autonomous vehicles; and Facebook, for some reason, is very concerned about the longevity of magnetic tapes.

And remember: The big tech companies file all kinds of crazy patents for things, and though most never amount to anything, some end up defining the future.

Alphabet

Determining what's newsworthy

There have been absolutely no problems with using algorithms to program the content that people see on their social feeds, so I can't see any issue with this patent idea. Essentially, the patent outlines crawling the web for new videos posted to sites and blogs online, and using a corpus of knowledge about the topic at hand, determining whether someone interested in that topic would find this new video interesting, and then alert them to it. It's not entirely clear how this system would moderate what it's finding and showing to people, but I can't imagine how that would be an issue.



Encasing passengers in airbags

This reminds me of the inflatable bubble thing James Bond used to save his life in "The World Is Not Enough." Waymo's patent outlines several different ways of automatically encasing passengers in something safe in the event of a crash, ranging from a netting and curtain design that would just hold the passengers in place to a full-on squishy safety bubble. If a self-driving car with me in it has to crash (I hear a lot of talk about Trolley Problems or some such), I hope I'm in the squishy kind.

Amazon

Making sure you're emailing the right person

Have you ever had one of those mortifying moments when you meant to email John Smith at Microsoft and you accidentally emailed John Smith at Apple? It's awkward, to say the least. But Amazon is apparently looking into helping you pick the right email addresses. Using a combination of understanding what the email writer is saying and who they've picked as the recipient would help Amazon's system determine whether they meant to choose who they have. As the patent itself says, "'Stephen is a terrible manager' is probably acceptable to send a friend in a personal interaction, but sending 'Stephen is a terrible manager' to Stephen is probably not okay."

Apple

Autonomously finding barriers

Apple reportedly reorganized its vehicle project to report to the company's AI chief, John Giannandrea, and this sounds like the sort of work his team would be useful for. The patent covers the sorts of detections that autonomous vehicles will be doing as they venture out into the world, mapping static objects like barriers, fences, walls and the like, and determining that unlike people, vehicles or animals, they likely won't be moving into their path — and if they do, there's probably a bigger issue at play.

Making dragging and dropping on iPad a little easier

Apple is slowly acknowledging the fact that, on the iPad, doing complicated stuff with a mouse and keyboard is often easier than with your fingers. But dragging and dropping content from one app to another still leaves a bit to be desired. This patent outlines a future iPad operating system where users could use multiple fingers to drag multiple images (or other files) from one program to another. It also seems to suggest a future where you can have more than two or three programs onscreen at once — rather like windows on a computer. But what really is a computer?

Facebook

Making magnetic tapes last longer

This probably isn't the first area of innovation that you'd expect a young tech giant to be researching. Facebook's new patent revolves around magnetic-tape data storage systems for archival purposes. Physics and entropy being what they are, these tapes tend to degrade over time, especially with continual use. This patent outlines ways to check on the health of the tapes, how often they're read and written, and to flag to administrators if they're failing. It also includes, according to the patent, a "predictive analysis method used to predict a future health status of the archival tape." Or you could, you know, just stop using tape.

Microsoft

Using AI to get your pumpkin spice latte

Microsoft has a new patent about designing a conversational AI assistant that would actually be somewhat enjoyable to talk to; think the assistant in "Her," but more about selling you more stuff than achieving singularity. In one of the examples given, the user wants to know how long pumpkin spice latte season will last at their favorite cafe, BigCoffeeChain. The AI tells them it lasts well into the new year (those must be some resilient pumpkins they over there!), and uses context later in their interaction with the user to provide them with a coupon for a PSL, and later, a few questions about one of the chain's locations. It's a smart way to get more data out of customers in a rather unassuming fashion. At least there's cheap PSLs involved.


AR glasses that can see through fog

A lot of work in augmented reality seems to focus more on adding things on top of reality rather than actually … augmenting it. But this patent has some interesting ideas on how to augment the real world: It outlines using AR glasses to, among other things, see through things that impede the vision of our mere mortal eyes. Small particulates, like those in smoke, dust or fog, could be filtered out by a digital vision system, and the clearer picture could be overlaid onto the wearer's glasses. This could be useful for your average jogger or commuter, but could be life-saving for emergency services personnel.

Auto-detecting out of office

Sometimes, when it's late on a Friday and a much-needed vacation is staring you in the face, the last thing you remember to do before leaving the office is turn on your out-of-office message. Who wants another moment in Outlook when the beach is calling? Microsoft seems to be working on a solution to this with its patent. It describes a system built into your email that could detect whether you're likely working that day or not. It would see if your usage matches how much you check your email on the weekend, and if it's about the same, it could flag to colleagues who email you that you're not likely to respond. It could also do the same thing if you forgot to turn off our OOO when you're back at work, tan and rested and still thinking about the daiquiris from last week.

The pandemic permanently changed Black Friday. Here’s how.

Here are the five biggest trends that will affect Black Friday and the holiday shopping season.

Here are the five biggest trends that will affect Black Friday and the holiday shopping season.

Photo: Jewel Samad/AFP via Getty Images

Click banner image for more Shopping Week coverage

Shopping is changing. It's not just the influence of COVID-19 altering what products we buy and how we buy them. It's also the many shifts in consumer behavior and retailer strategy — from the steady rise of ecommerce to the boom of on-demand delivery — years in the making, which have all been accelerated by the pandemic.

Keep Reading Show less
Nick Statt
Nick Statt is Protocol's video game reporter. Prior to joining Protocol, he was news editor at The Verge covering the gaming industry, mobile apps and antitrust out of San Francisco, in addition to managing coverage of Silicon Valley tech giants and startups. He now resides in Rochester, New York, home of the garbage plate and, completely coincidentally, the World Video Game Hall of Fame. He can be reached at nstatt@protocol.com.

The Bureau of Labor Statistics indicates that by 2026, the shortage of engineers in the U.S. will exceed 1.2 million, while 545,000 software developers will have left the market by that time. Meanwhile, business is becoming increasingly more digital-first, and teams need the tools in place to keep distributed teams aligned and able to respond quickly to changing business needs. That means businesses need to build powerful workplace applications without relying on developers.

In fact, according to Gartner, by 2025, 70% of new applications developed by enterprises will use low-code or no-code technologies and, by 2023, there will be at least four times as many active citizen developers as professional developers at large enterprises. We're on the cusp of a big shift in how businesses operate and how organization wide innovation happens.

Keep Reading Show less
Andrew Ofstad
As Airtable’s co-founder, Andrew spearheads Airtable’s long-term product bets and represents the voice of the customer in major product decisions. After co-founding the company, he helped scale Airtable’s original product and engineering teams. He previously led the redesign of Google's flagship Maps product, and before that was a product manager for Android.

It’s time to rethink Black Friday

The pandemic didn't end Black Friday, but it'll never look the same again.

We can expect Black Friday to stick around but lose relevance as retailers effectively dilute its meaning and purpose.

Illustration: Christopher T. Fong/Protocol

Click banner image for more Shopping Week coverage

"I'm selling meditation, so I shouldn't be stressed," said Charlie Rousset, the co-founder of sleep and relaxation gadget-maker Morphée. But even deep breathing can't help Rousset feel less on edge this Black Friday.

Keep Reading Show less
Janko Roettgers

Janko Roettgers (@jank0) is a senior reporter at Protocol, reporting on the shifting power dynamics between tech, media, and entertainment, including the impact of new technologies. Previously, Janko was Variety's first-ever technology writer in San Francisco, where he covered big tech and emerging technologies. He has reported for Gigaom, Frankfurter Rundschau, Berliner Zeitung, and ORF, among others. He has written three books on consumer cord-cutting and online music and co-edited an anthology on internet subcultures. He lives with his family in Oakland.

The pandemic permanently changed Black Friday. Here’s how.

Here are the five biggest trends that will affect Black Friday and the holiday shopping season.

Here are the five biggest trends that will affect Black Friday and the holiday shopping season.

Photo: Jewel Samad/AFP via Getty Images

Click banner image for more Shopping Week coverage

Shopping is changing. It's not just the influence of COVID-19 altering what products we buy and how we buy them. It's also the many shifts in consumer behavior and retailer strategy — from the steady rise of ecommerce to the boom of on-demand delivery — years in the making, which have all been accelerated by the pandemic.

Keep Reading Show less
Nick Statt
Nick Statt is Protocol's video game reporter. Prior to joining Protocol, he was news editor at The Verge covering the gaming industry, mobile apps and antitrust out of San Francisco, in addition to managing coverage of Silicon Valley tech giants and startups. He now resides in Rochester, New York, home of the garbage plate and, completely coincidentally, the World Video Game Hall of Fame. He can be reached at nstatt@protocol.com.
Protocol | Fintech

The pandemic keeps changing ecommerce. That makes fraud harder to fight.

As the second holiday season under COVID-19 gets underway, fraud finds new forms.

Online fraud is frustrating consumers and merchants.

Photo: fizkes/iStock/Getty Images Plus

Click banner image for more Shopping Week coverage

The second pandemic holiday shopping season is underway. That means cybersecurity experts get another chance to figure out how fraudsters operate in the COVID era.

Keep Reading Show less
Benjamin Pimentel

Benjamin Pimentel ( @benpimentel) covers fintech from San Francisco. He has reported on many of the biggest tech stories over the past 20 years for the San Francisco Chronicle, Dow Jones MarketWatch and Business Insider, from the dot-com crash, the rise of cloud computing, social networking and AI to the impact of the Great Recession and the COVID crisis on Silicon Valley and beyond. He can be reached at bpimentel@protocol.com or via Signal at (510)731-8429.

Latest Stories