Power

Microsoft wants to use AR to see through fog and smoke

Apple autonomous cars, AI coffee orders, emailing help and other patents from Big Tech.

Microsoft wants to use AR to see through fog and smoke

See what isn't there.

Image: Microsoft/USPTO

It's beyond dark out at 5:30 p.m. these days, so perhaps, as you're stuck at home with nowhere to go, you're tempted to log off your bad screen and onto your good screen a little earlier than you should. Perhaps that's what happened over at the U.S. Patent and Trademark Office, as this was a bit of a fallow week for patents from Big Tech.

That being said, there were still a few neat ones out there: Microsoft is looking into using AR to actually augment what you see; Apple is hard at work on autonomous vehicles; and Facebook, for some reason, is very concerned about the longevity of magnetic tapes.

And remember: The big tech companies file all kinds of crazy patents for things, and though most never amount to anything, some end up defining the future.

Alphabet

Determining what's newsworthy

There have been absolutely no problems with using algorithms to program the content that people see on their social feeds, so I can't see any issue with this patent idea. Essentially, the patent outlines crawling the web for new videos posted to sites and blogs online, and using a corpus of knowledge about the topic at hand, determining whether someone interested in that topic would find this new video interesting, and then alert them to it. It's not entirely clear how this system would moderate what it's finding and showing to people, but I can't imagine how that would be an issue.



Encasing passengers in airbags

This reminds me of the inflatable bubble thing James Bond used to save his life in "The World Is Not Enough." Waymo's patent outlines several different ways of automatically encasing passengers in something safe in the event of a crash, ranging from a netting and curtain design that would just hold the passengers in place to a full-on squishy safety bubble. If a self-driving car with me in it has to crash (I hear a lot of talk about Trolley Problems or some such), I hope I'm in the squishy kind.

Amazon

Making sure you're emailing the right person

Have you ever had one of those mortifying moments when you meant to email John Smith at Microsoft and you accidentally emailed John Smith at Apple? It's awkward, to say the least. But Amazon is apparently looking into helping you pick the right email addresses. Using a combination of understanding what the email writer is saying and who they've picked as the recipient would help Amazon's system determine whether they meant to choose who they have. As the patent itself says, "'Stephen is a terrible manager' is probably acceptable to send a friend in a personal interaction, but sending 'Stephen is a terrible manager' to Stephen is probably not okay."

Apple

Autonomously finding barriers

Apple reportedly reorganized its vehicle project to report to the company's AI chief, John Giannandrea, and this sounds like the sort of work his team would be useful for. The patent covers the sorts of detections that autonomous vehicles will be doing as they venture out into the world, mapping static objects like barriers, fences, walls and the like, and determining that unlike people, vehicles or animals, they likely won't be moving into their path — and if they do, there's probably a bigger issue at play.

Making dragging and dropping on iPad a little easier

Apple is slowly acknowledging the fact that, on the iPad, doing complicated stuff with a mouse and keyboard is often easier than with your fingers. But dragging and dropping content from one app to another still leaves a bit to be desired. This patent outlines a future iPad operating system where users could use multiple fingers to drag multiple images (or other files) from one program to another. It also seems to suggest a future where you can have more than two or three programs onscreen at once — rather like windows on a computer. But what really is a computer?

Facebook

Making magnetic tapes last longer

This probably isn't the first area of innovation that you'd expect a young tech giant to be researching. Facebook's new patent revolves around magnetic-tape data storage systems for archival purposes. Physics and entropy being what they are, these tapes tend to degrade over time, especially with continual use. This patent outlines ways to check on the health of the tapes, how often they're read and written, and to flag to administrators if they're failing. It also includes, according to the patent, a "predictive analysis method used to predict a future health status of the archival tape." Or you could, you know, just stop using tape.

Microsoft

Using AI to get your pumpkin spice latte

Microsoft has a new patent about designing a conversational AI assistant that would actually be somewhat enjoyable to talk to; think the assistant in "Her," but more about selling you more stuff than achieving singularity. In one of the examples given, the user wants to know how long pumpkin spice latte season will last at their favorite cafe, BigCoffeeChain. The AI tells them it lasts well into the new year (those must be some resilient pumpkins they over there!), and uses context later in their interaction with the user to provide them with a coupon for a PSL, and later, a few questions about one of the chain's locations. It's a smart way to get more data out of customers in a rather unassuming fashion. At least there's cheap PSLs involved.


AR glasses that can see through fog

A lot of work in augmented reality seems to focus more on adding things on top of reality rather than actually … augmenting it. But this patent has some interesting ideas on how to augment the real world: It outlines using AR glasses to, among other things, see through things that impede the vision of our mere mortal eyes. Small particulates, like those in smoke, dust or fog, could be filtered out by a digital vision system, and the clearer picture could be overlaid onto the wearer's glasses. This could be useful for your average jogger or commuter, but could be life-saving for emergency services personnel.

Auto-detecting out of office

Sometimes, when it's late on a Friday and a much-needed vacation is staring you in the face, the last thing you remember to do before leaving the office is turn on your out-of-office message. Who wants another moment in Outlook when the beach is calling? Microsoft seems to be working on a solution to this with its patent. It describes a system built into your email that could detect whether you're likely working that day or not. It would see if your usage matches how much you check your email on the weekend, and if it's about the same, it could flag to colleagues who email you that you're not likely to respond. It could also do the same thing if you forgot to turn off our OOO when you're back at work, tan and rested and still thinking about the daiquiris from last week.

Policy

Musk’s texts reveal what tech’s most powerful people really want

From Jack Dorsey to Joe Rogan, Musk’s texts are chock-full of überpowerful people, bending a knee to Twitter’s once and (still maybe?) future king.

“Maybe Oprah would be interested in joining the Twitter board if my bid succeeds,” one text reads.

Photo illustration: Patrick Pleul/picture alliance via Getty Images; Protocol

Elon Musk’s text inbox is a rarefied space. It’s a place where tech’s wealthiest casually commit to spending billions of dollars with little more than a thumbs-up emoji and trade tips on how to rewrite the rules for how hundreds of millions of people around the world communicate.

Now, Musk’s ongoing legal battle with Twitter is giving the rest of us a fleeting glimpse into that world. The collection of Musk’s private texts that was made public this week is chock-full of tech power brokers. While the messages are meant to reveal something about Musk’s motivations — and they do — they also say a lot about how things get done and deals get made among some of the most powerful people in the world.

Keep Reading Show less
Issie Lapowsky

Issie Lapowsky ( @issielapowsky) is Protocol's chief correspondent, covering the intersection of technology, politics, and national affairs. She also oversees Protocol's fellowship program. Previously, she was a senior writer at Wired, where she covered the 2016 election and the Facebook beat in its aftermath. Prior to that, Issie worked as a staff writer for Inc. magazine, writing about small business and entrepreneurship. She has also worked as an on-air contributor for CBS News and taught a graduate-level course at New York University's Center for Publishing on how tech giants have affected publishing.

Sponsored Content

Great products are built on strong patents

Experts say robust intellectual property protection is essential to ensure the long-term R&D required to innovate and maintain America's technology leadership.

Every great tech product that you rely on each day, from the smartphone in your pocket to your music streaming service and navigational system in the car, shares one important thing: part of its innovative design is protected by intellectual property (IP) laws.

From 5G to artificial intelligence, IP protection offers a powerful incentive for researchers to create ground-breaking products, and governmental leaders say its protection is an essential part of maintaining US technology leadership. To quote Secretary of Commerce Gina Raimondo: "intellectual property protection is vital for American innovation and entrepreneurship.”

Keep Reading Show less
James Daly
James Daly has a deep knowledge of creating brand voice identity, including understanding various audiences and targeting messaging accordingly. He enjoys commissioning, editing, writing, and business development, particularly in launching new ventures and building passionate audiences. Daly has led teams large and small to multiple awards and quantifiable success through a strategy built on teamwork, passion, fact-checking, intelligence, analytics, and audience growth while meeting budget goals and production deadlines in fast-paced environments. Daly is the Editorial Director of 2030 Media and a contributor at Wired.
Fintech

Circle’s CEO: This is not the time to ‘go crazy’

Jeremy Allaire is leading the stablecoin powerhouse in a time of heightened regulation.

“It’s a complex environment. So every CEO and every board has to be a little bit cautious, because there’s a lot of uncertainty,” Circle CEO Jeremy Allaire told Protocol at Converge22.

Photo: Circle

Sitting solo on a San Francisco stage, Circle CEO Jeremy Allaire asked tennis superstar Serena Williams what it’s like to face “unrelenting skepticism.”

“What do you do when someone says you can’t do this?” Allaire asked the athlete turned VC, who was beaming into Circle’s Converge22 convention by video.

Keep Reading Show less
Benjamin Pimentel

Benjamin Pimentel ( @benpimentel) covers crypto and fintech from San Francisco. He has reported on many of the biggest tech stories over the past 20 years for the San Francisco Chronicle, Dow Jones MarketWatch and Business Insider, from the dot-com crash, the rise of cloud computing, social networking and AI to the impact of the Great Recession and the COVID crisis on Silicon Valley and beyond. He can be reached at bpimentel@protocol.com or via Google Voice at (925) 307-9342.

Enterprise

Is Salesforce still a growth company? Investors are skeptical

Salesforce is betting that customer data platform Genie and new Slack features can push the company to $50 billion in revenue by 2026. But investors are skeptical about the company’s ability to deliver.

Photo: Marlena Sloss/Bloomberg via Getty Images

Salesforce has long been enterprise tech’s golden child. The company said everything customers wanted to hear and did everything investors wanted to see: It produced robust, consistent growth from groundbreaking products combined with an aggressive M&A strategy and a cherished culture, all operating under the helm of a bombastic, but respected, CEO and team of well-coiffed executives.

Dreamforce is the embodiment of that success. Every year, alongside frustrating San Francisco residents, the over-the-top celebration serves as a battle cry to the enterprise software industry, reminding everyone that Marc Benioff’s mighty fiefdom is poised to expand even deeper into your corporate IT stack.

Keep Reading Show less
Joe Williams

Joe Williams is a writer-at-large at Protocol. He previously covered enterprise software for Protocol, Bloomberg and Business Insider. Joe can be reached at JoeWilliams@Protocol.com. To share information confidentially, he can also be contacted on a non-work device via Signal (+1-309-265-6120) or JPW53189@protonmail.com.

Policy

The US and EU are splitting on tech policy. That’s putting the web at risk.

A conversation with Cédric O, the former French minister of state for digital.

“With the difficulty of the U.S. in finding political agreement or political basis to legislate more, we are facing a risk of decoupling in the long term between the EU and the U.S.”

Photo: David Paul Morris/Bloomberg via Getty Images

Cédric O, France’s former minister of state for digital, has been an advocate of Europe’s approach to tech and at the forefront of the continent’s relations with U.S. giants. Protocol caught up with O last week at a conference in New York focusing on social media’s negative effects on society and the possibilities of blockchain-based protocols for alternative networks.

O said watching the U.S. lag in tech policy — even as some states pass their own measures and federal bills gain momentum — has made him worry about the EU and U.S. decoupling. While not as drastic as a disentangling of economic fortunes between the West and China, such a divergence, as O describes it, could still make it functionally impossible for companies to serve users on both sides of the Atlantic with the same product.

Keep Reading Show less
Ben Brody

Ben Brody (@ BenBrodyDC) is a senior reporter at Protocol focusing on how Congress, courts and agencies affect the online world we live in. He formerly covered tech policy and lobbying (including antitrust, Section 230 and privacy) at Bloomberg News, where he previously reported on the influence industry, government ethics and the 2016 presidential election. Before that, Ben covered business news at CNNMoney and AdAge, and all manner of stories in and around New York. He still loves appearing on the New York news radio he grew up with.

Latest Stories
Bulletins