Power

Microsoft wants to use AR to see through fog and smoke

Apple autonomous cars, AI coffee orders, emailing help and other patents from Big Tech.

Microsoft wants to use AR to see through fog and smoke

See what isn't there.

Image: Microsoft/USPTO

It's beyond dark out at 5:30 p.m. these days, so perhaps, as you're stuck at home with nowhere to go, you're tempted to log off your bad screen and onto your good screen a little earlier than you should. Perhaps that's what happened over at the U.S. Patent and Trademark Office, as this was a bit of a fallow week for patents from Big Tech.

That being said, there were still a few neat ones out there: Microsoft is looking into using AR to actually augment what you see; Apple is hard at work on autonomous vehicles; and Facebook, for some reason, is very concerned about the longevity of magnetic tapes.

And remember: The big tech companies file all kinds of crazy patents for things, and though most never amount to anything, some end up defining the future.

Alphabet

Determining what's newsworthy

There have been absolutely no problems with using algorithms to program the content that people see on their social feeds, so I can't see any issue with this patent idea. Essentially, the patent outlines crawling the web for new videos posted to sites and blogs online, and using a corpus of knowledge about the topic at hand, determining whether someone interested in that topic would find this new video interesting, and then alert them to it. It's not entirely clear how this system would moderate what it's finding and showing to people, but I can't imagine how that would be an issue.



Encasing passengers in airbags

This reminds me of the inflatable bubble thing James Bond used to save his life in "The World Is Not Enough." Waymo's patent outlines several different ways of automatically encasing passengers in something safe in the event of a crash, ranging from a netting and curtain design that would just hold the passengers in place to a full-on squishy safety bubble. If a self-driving car with me in it has to crash (I hear a lot of talk about Trolley Problems or some such), I hope I'm in the squishy kind.

Amazon

Making sure you're emailing the right person

Have you ever had one of those mortifying moments when you meant to email John Smith at Microsoft and you accidentally emailed John Smith at Apple? It's awkward, to say the least. But Amazon is apparently looking into helping you pick the right email addresses. Using a combination of understanding what the email writer is saying and who they've picked as the recipient would help Amazon's system determine whether they meant to choose who they have. As the patent itself says, "'Stephen is a terrible manager' is probably acceptable to send a friend in a personal interaction, but sending 'Stephen is a terrible manager' to Stephen is probably not okay."

Apple

Autonomously finding barriers

Apple reportedly reorganized its vehicle project to report to the company's AI chief, John Giannandrea, and this sounds like the sort of work his team would be useful for. The patent covers the sorts of detections that autonomous vehicles will be doing as they venture out into the world, mapping static objects like barriers, fences, walls and the like, and determining that unlike people, vehicles or animals, they likely won't be moving into their path — and if they do, there's probably a bigger issue at play.

Making dragging and dropping on iPad a little easier

Apple is slowly acknowledging the fact that, on the iPad, doing complicated stuff with a mouse and keyboard is often easier than with your fingers. But dragging and dropping content from one app to another still leaves a bit to be desired. This patent outlines a future iPad operating system where users could use multiple fingers to drag multiple images (or other files) from one program to another. It also seems to suggest a future where you can have more than two or three programs onscreen at once — rather like windows on a computer. But what really is a computer?

Facebook

Making magnetic tapes last longer

This probably isn't the first area of innovation that you'd expect a young tech giant to be researching. Facebook's new patent revolves around magnetic-tape data storage systems for archival purposes. Physics and entropy being what they are, these tapes tend to degrade over time, especially with continual use. This patent outlines ways to check on the health of the tapes, how often they're read and written, and to flag to administrators if they're failing. It also includes, according to the patent, a "predictive analysis method used to predict a future health status of the archival tape." Or you could, you know, just stop using tape.

Microsoft

Using AI to get your pumpkin spice latte

Microsoft has a new patent about designing a conversational AI assistant that would actually be somewhat enjoyable to talk to; think the assistant in "Her," but more about selling you more stuff than achieving singularity. In one of the examples given, the user wants to know how long pumpkin spice latte season will last at their favorite cafe, BigCoffeeChain. The AI tells them it lasts well into the new year (those must be some resilient pumpkins they over there!), and uses context later in their interaction with the user to provide them with a coupon for a PSL, and later, a few questions about one of the chain's locations. It's a smart way to get more data out of customers in a rather unassuming fashion. At least there's cheap PSLs involved.


AR glasses that can see through fog

A lot of work in augmented reality seems to focus more on adding things on top of reality rather than actually … augmenting it. But this patent has some interesting ideas on how to augment the real world: It outlines using AR glasses to, among other things, see through things that impede the vision of our mere mortal eyes. Small particulates, like those in smoke, dust or fog, could be filtered out by a digital vision system, and the clearer picture could be overlaid onto the wearer's glasses. This could be useful for your average jogger or commuter, but could be life-saving for emergency services personnel.

Auto-detecting out of office

Sometimes, when it's late on a Friday and a much-needed vacation is staring you in the face, the last thing you remember to do before leaving the office is turn on your out-of-office message. Who wants another moment in Outlook when the beach is calling? Microsoft seems to be working on a solution to this with its patent. It describes a system built into your email that could detect whether you're likely working that day or not. It would see if your usage matches how much you check your email on the weekend, and if it's about the same, it could flag to colleagues who email you that you're not likely to respond. It could also do the same thing if you forgot to turn off our OOO when you're back at work, tan and rested and still thinking about the daiquiris from last week.

Entertainment

Niantic is building an AR map of the world

The company’s Visual Positioning System will help developers build location-based AR games and experiences; a new social app aims to help with AR content discovery.

VPS will allow developers to build location-based AR experiences for tens of thousands of public spaces.

Image: Niantic

Pokémon Go maker Niantic has quietly been building a 3D AR map of the world. Now, the company is getting ready to share the fruits of its labor with third-party developers: Niantic announced the launch of its Lightship Visual Positioning System at its developer summit in San Francisco on Tuesday. VPS will allow developers to build location-based AR experiences for tens of thousands of public spaces, Niantic said.

Niantic also announced a new service called Campfire that adds a social discovery layer to AR, starting with Niantic’s own games. Both announcements show that Niantic wants to be much more than a game developer with just one or two hit apps (and a couple of flops). Instead, it aims to play a key role in the future of AR — and it’s relying on millions of Ingress and Pokémon Go players to help build that future.

Keep Reading Show less
Janko Roettgers

Janko Roettgers (@jank0) is a senior reporter at Protocol, reporting on the shifting power dynamics between tech, media, and entertainment, including the impact of new technologies. Previously, Janko was Variety's first-ever technology writer in San Francisco, where he covered big tech and emerging technologies. He has reported for Gigaom, Frankfurter Rundschau, Berliner Zeitung, and ORF, among others. He has written three books on consumer cord-cutting and online music and co-edited an anthology on internet subcultures. He lives with his family in Oakland.

Sponsored Content

Why the digital transformation of industries is creating a more sustainable future

Qualcomm’s chief sustainability officer Angela Baker on how companies can view going “digital” as a way not only toward growth, as laid out in a recent report, but also toward establishing and meeting environmental, social and governance goals.

Three letters dominate business practice at present: ESG, or environmental, social and governance goals. The number of mentions of the environment in financial earnings has doubled in the last five years, according to GlobalData: 600,000 companies mentioned the term in their annual or quarterly results last year.

But meeting those ESG goals can be a challenge — one that businesses can’t and shouldn’t take lightly. Ahead of an exclusive fireside chat at Davos, Angela Baker, chief sustainability officer at Qualcomm, sat down with Protocol to speak about how best to achieve those targets and how Qualcomm thinks about its own sustainability strategy, net zero commitment, other ESG targets and more.

Keep Reading Show less
Chris Stokel-Walker

Chris Stokel-Walker is a freelance technology and culture journalist and author of "YouTubers: How YouTube Shook Up TV and Created a New Generation of Stars." His work has been published in The New York Times, The Guardian and Wired.

Workplace

Why it's time to give all your employees executive coaching

In an effort to boost retention and engagement, companies are rolling out access to executive coaching to all of their employees.

Coaching is among personalized and exclusive benefits employers chose to offer their workforce during the pandemic.

Image: Christopher T. Fong/Protocol

Executive coaching has long been a quiet force behind leaders in the tech industry, but that premium benefit, often only offered to the top executives, is changing. A new wave of executive coaching services are hitting the market aimed at workers who would have traditionally been excluded from access.

Tech companies know that in order to stay competitive in today’s still-hot job market, it pays to offer more personalized and exclusive benefits. Chief People Officer Annette Reavis says Envoy, a workplace tech company, offers all employees access to a broad range of opportunities. “We offer everyone an L&D credit that they can spend on outside learning, whether it's executive coaching or learning a new coding language. We do this so that people can have access to and learn skills specific to their job.”

Keep Reading Show less
Amber Burton

Amber Burton (@amberbburton) is a reporter at Protocol. Previously, she covered personal finance and diversity in business at The Wall Street Journal. She earned an M.S. in Strategic Communications from Columbia University and B.A. in English and Journalism from Wake Forest University. She lives in North Carolina.

Enterprise

Microsoft thinks Windows developers are ready for virtual workstations

The new Microsoft Dev Box service, coupled with Azure Deployment Environments, lets developers go from code to the cloud faster than ever.

Microsoft hopes a new cloud service will address one of developers' biggest challenges.

Photo: Grant Hindsley/Bloomberg via Getty Images

Microsoft hopes a new cloud service will address one of the biggest challenges that developers have raised with the technology giant over the last several years: managing developer workstations.

Microsoft Dev Box, now in private preview, creates virtual developer workstations running its Windows operating system in the cloud, allowing development teams to standardize how those fundamental tools are initialized, set up and managed.

Keep Reading Show less
Donna Goodison

Donna Goodison (@dgoodison) is Protocol's senior reporter focusing on enterprise infrastructure technology, from the 'Big 3' cloud computing providers to data centers. She previously covered the public cloud at CRN after 15 years as a business reporter for the Boston Herald. Based in Massachusetts, she also has worked as a Boston Globe freelancer, business reporter at the Boston Business Journal and real estate reporter at Banker & Tradesman after toiling at weekly newspapers.

Enterprise

Okta CEO: 'We should have done a better job' with the Lapsus$ breach

In an interview with Protocol, Okta CEO Todd McKinnon said the cybersecurity firm could’ve done a lot of things better after the Lapsus$ breach of a third-party support provider earlier this year.

From talking to hundreds of customers, “I've had a good sense of the sentiment and the frustrations,” McKinnon said.

Photo: David Paul Morris via Getty Images

Okta co-founder and CEO Todd McKinnon agrees with you: Disclosing a breach that impacts customer data should not take months.

“If that happens in January, customers can't be finding out about it in March,” McKinnon said in an interview with Protocol.

Keep Reading Show less
Kyle Alspach

Kyle Alspach ( @KyleAlspach) is a senior reporter at Protocol, focused on cybersecurity. He has covered the tech industry since 2010 for outlets including VentureBeat, CRN and the Boston Globe. He lives in Portland, Oregon, and can be reached at kalspach@procotol.com.

Latest Stories
Bulletins