Power

Microsoft wants to use AR to see through fog and smoke

Apple autonomous cars, AI coffee orders, emailing help and other patents from Big Tech.

Microsoft wants to use AR to see through fog and smoke

See what isn't there.

Image: Microsoft/USPTO

It's beyond dark out at 5:30 p.m. these days, so perhaps, as you're stuck at home with nowhere to go, you're tempted to log off your bad screen and onto your good screen a little earlier than you should. Perhaps that's what happened over at the U.S. Patent and Trademark Office, as this was a bit of a fallow week for patents from Big Tech.

That being said, there were still a few neat ones out there: Microsoft is looking into using AR to actually augment what you see; Apple is hard at work on autonomous vehicles; and Facebook, for some reason, is very concerned about the longevity of magnetic tapes.

And remember: The big tech companies file all kinds of crazy patents for things, and though most never amount to anything, some end up defining the future.

Alphabet

Determining what's newsworthy

There have been absolutely no problems with using algorithms to program the content that people see on their social feeds, so I can't see any issue with this patent idea. Essentially, the patent outlines crawling the web for new videos posted to sites and blogs online, and using a corpus of knowledge about the topic at hand, determining whether someone interested in that topic would find this new video interesting, and then alert them to it. It's not entirely clear how this system would moderate what it's finding and showing to people, but I can't imagine how that would be an issue.



Encasing passengers in airbags

This reminds me of the inflatable bubble thing James Bond used to save his life in "The World Is Not Enough." Waymo's patent outlines several different ways of automatically encasing passengers in something safe in the event of a crash, ranging from a netting and curtain design that would just hold the passengers in place to a full-on squishy safety bubble. If a self-driving car with me in it has to crash (I hear a lot of talk about Trolley Problems or some such), I hope I'm in the squishy kind.

Amazon

Making sure you're emailing the right person

Have you ever had one of those mortifying moments when you meant to email John Smith at Microsoft and you accidentally emailed John Smith at Apple? It's awkward, to say the least. But Amazon is apparently looking into helping you pick the right email addresses. Using a combination of understanding what the email writer is saying and who they've picked as the recipient would help Amazon's system determine whether they meant to choose who they have. As the patent itself says, "'Stephen is a terrible manager' is probably acceptable to send a friend in a personal interaction, but sending 'Stephen is a terrible manager' to Stephen is probably not okay."

Apple

Autonomously finding barriers

Apple reportedly reorganized its vehicle project to report to the company's AI chief, John Giannandrea, and this sounds like the sort of work his team would be useful for. The patent covers the sorts of detections that autonomous vehicles will be doing as they venture out into the world, mapping static objects like barriers, fences, walls and the like, and determining that unlike people, vehicles or animals, they likely won't be moving into their path — and if they do, there's probably a bigger issue at play.

Making dragging and dropping on iPad a little easier

Apple is slowly acknowledging the fact that, on the iPad, doing complicated stuff with a mouse and keyboard is often easier than with your fingers. But dragging and dropping content from one app to another still leaves a bit to be desired. This patent outlines a future iPad operating system where users could use multiple fingers to drag multiple images (or other files) from one program to another. It also seems to suggest a future where you can have more than two or three programs onscreen at once — rather like windows on a computer. But what really is a computer?

Facebook

Making magnetic tapes last longer

This probably isn't the first area of innovation that you'd expect a young tech giant to be researching. Facebook's new patent revolves around magnetic-tape data storage systems for archival purposes. Physics and entropy being what they are, these tapes tend to degrade over time, especially with continual use. This patent outlines ways to check on the health of the tapes, how often they're read and written, and to flag to administrators if they're failing. It also includes, according to the patent, a "predictive analysis method used to predict a future health status of the archival tape." Or you could, you know, just stop using tape.

Microsoft

Using AI to get your pumpkin spice latte

Microsoft has a new patent about designing a conversational AI assistant that would actually be somewhat enjoyable to talk to; think the assistant in "Her," but more about selling you more stuff than achieving singularity. In one of the examples given, the user wants to know how long pumpkin spice latte season will last at their favorite cafe, BigCoffeeChain. The AI tells them it lasts well into the new year (those must be some resilient pumpkins they over there!), and uses context later in their interaction with the user to provide them with a coupon for a PSL, and later, a few questions about one of the chain's locations. It's a smart way to get more data out of customers in a rather unassuming fashion. At least there's cheap PSLs involved.


AR glasses that can see through fog

A lot of work in augmented reality seems to focus more on adding things on top of reality rather than actually … augmenting it. But this patent has some interesting ideas on how to augment the real world: It outlines using AR glasses to, among other things, see through things that impede the vision of our mere mortal eyes. Small particulates, like those in smoke, dust or fog, could be filtered out by a digital vision system, and the clearer picture could be overlaid onto the wearer's glasses. This could be useful for your average jogger or commuter, but could be life-saving for emergency services personnel.

Auto-detecting out of office

Sometimes, when it's late on a Friday and a much-needed vacation is staring you in the face, the last thing you remember to do before leaving the office is turn on your out-of-office message. Who wants another moment in Outlook when the beach is calling? Microsoft seems to be working on a solution to this with its patent. It describes a system built into your email that could detect whether you're likely working that day or not. It would see if your usage matches how much you check your email on the weekend, and if it's about the same, it could flag to colleagues who email you that you're not likely to respond. It could also do the same thing if you forgot to turn off our OOO when you're back at work, tan and rested and still thinking about the daiquiris from last week.

Fintech

Judge Zia Faruqui is trying to teach you crypto, one ‘SNL’ reference at a time

His decisions on major cryptocurrency cases have quoted "The Big Lebowski," "SNL," and "Dr. Strangelove." That’s because he wants you — yes, you — to read them.

The ways Zia Faruqui (right) has weighed on cases that have come before him can give lawyers clues as to what legal frameworks will pass muster.

Photo: Carolyn Van Houten/The Washington Post via Getty Images

“Cryptocurrency and related software analytics tools are ‘The wave of the future, Dude. One hundred percent electronic.’”

That’s not a quote from "The Big Lebowski" — at least, not directly. It’s a quote from a Washington, D.C., district court memorandum opinion on the role cryptocurrency analytics tools can play in government investigations. The author is Magistrate Judge Zia Faruqui.

Keep Reading Show less
Veronica Irwin

Veronica Irwin (@vronirwin) is a San Francisco-based reporter at Protocol covering fintech. Previously she was at the San Francisco Examiner, covering tech from a hyper-local angle. Before that, her byline was featured in SF Weekly, The Nation, Techworker, Ms. Magazine and The Frisc.

The financial technology transformation is driving competition, creating consumer choice, and shaping the future of finance. Hear from seven fintech leaders who are reshaping the future of finance, and join the inaugural Financial Technology Association Fintech Summit to learn more.

Keep Reading Show less
FTA
The Financial Technology Association (FTA) represents industry leaders shaping the future of finance. We champion the power of technology-centered financial services and advocate for the modernization of financial regulation to support inclusion and responsible innovation.
Enterprise

AWS CEO: The cloud isn’t just about technology

As AWS preps for its annual re:Invent conference, Adam Selipsky talks product strategy, support for hybrid environments, and the value of the cloud in uncertain economic times.

Photo: Noah Berger/Getty Images for Amazon Web Services

AWS is gearing up for re:Invent, its annual cloud computing conference where announcements this year are expected to focus on its end-to-end data strategy and delivering new industry-specific services.

It will be the second re:Invent with CEO Adam Selipsky as leader of the industry’s largest cloud provider after his return last year to AWS from data visualization company Tableau Software.

Keep Reading Show less
Donna Goodison

Donna Goodison (@dgoodison) is Protocol's senior reporter focusing on enterprise infrastructure technology, from the 'Big 3' cloud computing providers to data centers. She previously covered the public cloud at CRN after 15 years as a business reporter for the Boston Herald. Based in Massachusetts, she also has worked as a Boston Globe freelancer, business reporter at the Boston Business Journal and real estate reporter at Banker & Tradesman after toiling at weekly newspapers.

Image: Protocol

We launched Protocol in February 2020 to cover the evolving power center of tech. It is with deep sadness that just under three years later, we are winding down the publication.

As of today, we will not publish any more stories. All of our newsletters, apart from our flagship, Source Code, will no longer be sent. Source Code will be published and sent for the next few weeks, but it will also close down in December.

Keep Reading Show less
Bennett Richardson

Bennett Richardson ( @bennettrich) is the president of Protocol. Prior to joining Protocol in 2019, Bennett was executive director of global strategic partnerships at POLITICO, where he led strategic growth efforts including POLITICO's European expansion in Brussels and POLITICO's creative agency POLITICO Focus during his six years with the company. Prior to POLITICO, Bennett was co-founder and CMO of Hinge, the mobile dating company recently acquired by Match Group. Bennett began his career in digital and social brand marketing working with major brands across tech, energy, and health care at leading marketing and communications agencies including Edelman and GMMB. Bennett is originally from Portland, Maine, and received his bachelor's degree from Colgate University.

Enterprise

Why large enterprises struggle to find suitable platforms for MLops

As companies expand their use of AI beyond running just a few machine learning models, and as larger enterprises go from deploying hundreds of models to thousands and even millions of models, ML practitioners say that they have yet to find what they need from prepackaged MLops systems.

As companies expand their use of AI beyond running just a few machine learning models, ML practitioners say that they have yet to find what they need from prepackaged MLops systems.

Photo: artpartner-images via Getty Images

On any given day, Lily AI runs hundreds of machine learning models using computer vision and natural language processing that are customized for its retail and ecommerce clients to make website product recommendations, forecast demand, and plan merchandising. But this spring when the company was in the market for a machine learning operations platform to manage its expanding model roster, it wasn’t easy to find a suitable off-the-shelf system that could handle such a large number of models in deployment while also meeting other criteria.

Some MLops platforms are not well-suited for maintaining even more than 10 machine learning models when it comes to keeping track of data, navigating their user interfaces, or reporting capabilities, Matthew Nokleby, machine learning manager for Lily AI’s product intelligence team, told Protocol earlier this year. “The duct tape starts to show,” he said.

Keep Reading Show less
Kate Kaye

Kate Kaye is an award-winning multimedia reporter digging deep and telling print, digital and audio stories. She covers AI and data for Protocol. Her reporting on AI and tech ethics issues has been published in OneZero, Fast Company, MIT Technology Review, CityLab, Ad Age and Digiday and heard on NPR. Kate is the creator of RedTailMedia.org and is the author of "Campaign '08: A Turning Point for Digital Media," a book about how the 2008 presidential campaigns used digital media and data.

Latest Stories
Bulletins