Power

Why Microsoft's new Flight Simulator should make Google and Amazon nervous

Stunning visuals and world-building show Microsoft is far ahead of its rivals in bringing different teams and different technologies together.

A screenshot from Flight Simulator showing a plane over the British Virgin Islands

Cruising over the British Virgin Islands in Microsoft's new Flight Simulator.

Screenshot: Seth Schiesel/Protocol

Quick, what's Microsoft's most venerable franchise?

Nope, it's not Office (1990), Windows (1985) or even Word (1983). It's actually Microsoft Flight Simulator, first released in November 1982.

Microsoft, still then an upstart in the fledgling personal computer business, commissioned and acquired Flight during Reagan's first term as a showcase to demonstrate the power of "modern" technology to everyday consumers and to the company's corporate rivals.

Thirty-eight years later, on Tuesday, Microsoft is releasing a stunning, groundbreaking new version of Flight — the first since 2006 — for exactly the same reason. Microsoft is no longer an upstart, but Flight may provide the most vivid, consumer-friendly demonstration in years of how the bleeding edge of modern technology can provide fundamentally new human experiences and (quite literally) change how we see our world. In that sense, Flight may become Microsoft's most effective marketing vehicle in years.

What makes the new Flight so revolutionary is that it's simply the most detailed and realistic model of Earth yet created. Call it a digital twin. In Flight, being released initially for powerful Windows PCs, you can go anywhere on the planet almost instantaneously, and it looks like that place on the planet (with the general exception of restricted military areas). The buildings, the hills, the vegetation, the water, the sky, the clouds — it's all there, everywhere. The degree of freedom produces an intellectual effect that is at first a bit disorienting — perhaps akin to the first time you realized you could just type a question into Google and get an answer.

Flight planning in Microsoft's new Flight Simulator.Screenshot: Seth Schiesel/Protocol

With Flight, it is superficially easy to focus merely on the graphics, which are amazing in the sense that they are realistic. But the real leap forward — what makes Flight so powerful for Microsoft and so potentially concerning to its Big Tech brethren — is how adroitly the company has integrated many different layers of advanced technology to create a fresh, easy-to-use consumer experience that truly has no competition.

In February, Phil Spencer, Microsoft's head of Xbox, made some waves when he said that his company's true competitors in interactive entertainment were no longer traditional gaming rivals Sony and Nintendo, but rather Big Tech rivals Amazon and Google. For those two companies, Flight should be a sobering reminder of just how far behind Microsoft they are in gaming in particular, but also perhaps more generally in their ability to bring different internal groups together for a common cause.

Above Manhattan in the new Flight Simulator.Screenshot: Seth Schiesel/Protocol

How to Flight

For Jörg Neumann, a longtime Microsoft executive who is in charge of the team, the new Flight got its start at Machu Picchu.

Around six years ago, he was working on a product called HoloTour for Microsoft's HoloLens augmented reality headset. "And that's basically what inspired Flight Sim because you can't build Machu Picchu, right?" he told Protocol. "You can build all the buildings and roam on a street level, but Machu Picchu is this sprawling mountainside and endless vistas. So I ended up talking to the Bing Maps team and they had the entire planet. They have the height field and also aerial photography so we basically used that, put it into the engine and added a few things — like the actual ruins of Machu Picchu — and I swear the first time I put on the headset, the kernel of the idea was right there: 'Man, if we could do this planet-wide wouldn't that be amazing?'"

For HoloTour, Neumann was working with Asobo Studio in Bordeaux, France, which had developed a graphics engine specialized for huge virtual environments. Neumann discovered that the Bing team had detailed photogrammetry data down to 5-centimeter resolution for more than 400 cities. Microsoft contracts with various providers around the world, such as Vexcel Imaging, to fly all over the world using technologies including lidar (a laser version of radar), to generate imagery and models of the real world.

Neumann got the Bing photogrammetry for Seattle ("30, 40 gigs or something") and reached back out to the French studio in early 2017. "Asobo built themselves a quick Cessna 172 or something, and we flew over it, and it looked exactly like the city of Seattle at daytime," he said. Neumann showed a video to Spencer and got the green light to resurrect Flight.

Near Sedona, Arizona, in Flight Simulator. Screenshot: Seth Schiesel/Protocol

Microsoft didn't have detailed photogrammetry for the whole planet, though, so it had to use proprietary satellite imagery in addition to traditional aerial photography. That's where a company called Blackshark.ai in Graz, Austria, came in. It just so happens that Bing's camera technology had been developed at a university there, and the town is home to a community of vision scientists.

Blackshark initially approached Neumann with a pitch to make a game about drone racing in the Alps. Neuman wasn't interested in a drone game but was definitely interested in the Alps. "They had things figured out like analyzing buildings and the color of the rooftops, and it was clear they had done a lot of work on machine learning, so I told them the vision: 'Hey, I just want to be able to go anywhere on Earth, and it has to look exactly right,'" he said. Blackshark became Asobo's co-developer on Flight.

Waxahachie, Texas, in Flight Simulator. Waxahachie, Texas, in Flight Simulator. Screenshot: Seth Schiesel/Protocol

In the meantime, Neumann went to Switzerland to meet with some hardcore weather nerds about incorporating their models into the product. "They can tell us stuff like the altitude ice condensates at under various conditions," he said. The team also leveraged technology from Microsoft's discontinued Photosynth project, which generates 3D models from 2D photos.

The algorithms and data — including OpenStreetMap — were then fed into Microsoft's vast Azure computing cloud to generate Flight's 2.5-petabyte model, which includes 2 trillion trees, 1.5 billion buildings (you can probably find your house), 117 million lakes and just about every road, mountain, city and airport on the planet.

Azure is used not just to generate the data but also to deliver it to players. No personal computer could hold all that information, so Flight streams real-world geography, weather and air traffic information to users as they play. (There is also a less-detailed offline mode.)

"I sat in a meeting this morning with the Bing Maps guys, and we were looking at a grid of the world that is being updated because there are photographs being taken right this moment, and they will be in the product in two months," Neumann said. "Flight Simulator is a living, breathing thing that has never been possible before."

The true 'next generation'

Every company that could be considered Big Tech is investing billions of dollars into gaming. This fall, Sony and Microsoft will release major new game machines, their first in seven years.

These waves are often referred to as generations; gamers (and marketers) love to talk about "next-generation" games and "next-generation" experiences. But for most of the last 15 years or so, "next generation" has meant little more than prettier explosions and faster load times. Even as processing power, storage and bandwidth — the basic constraints of electronics (along with power consumption) — have exploded in capacity, the types of entertainment and education experiences they are used to provide have remained essentially static. Even if they are delivered differently, these new conduits are usually being used to provide access to the same old stuff.

A summer rain shower in the Alps in Flight Simulator. Screenshot: Seth Schiesel/Protocol

For example, Google's Stadia cloud gaming service is amazing in the sense that it actually provides a gaming experience in a Chrome browser on potato-power systems that is, let's say, 90% as good as playing on local hardware. But it is providing access to games that generally make zero use of advanced computing in their basic design. When Google announced Stadia last year, it teased fundamentally new sorts of interactive experiences like jumping into a game with a YouTube streamer in real time. But it has not delivered on those yet.

Amazon is having related difficulties with its own gaming efforts. Amazon wants to make games that exploit the vast power of the Amazon Web Services platform but is struggling to harness that power in the service of experiences that are actually fun ("fun" being perhaps the least quantifiable metric out there). Amazon hopes to make that leap in its coming online fantasy adventure New World by letting players fight in scrums of hundreds (though still far smaller than the multi-thousand-player space battles in Eve Online), but we will just have to see.

Sony and Nintendo, the traditional Japanese gaming giants, each routinely churns out games of the highest quality. But they are almost always incremental, rather than truly revolutionary, in their progression: prettier explosions, faster load times. That's because Sony and Nintendo just don't have the vast technical and financial capabilities of the Big Tech cabal.

And so it's easy to see Microsoft's new Flight Simulator as the first truly "next-generation" digital experience in a long time. It evokes the childlike wonder of spinning a globe, except now you can actually zoom all the way in and check it out for yourself.

The 100,000-foot view

Microsoft created its own gaming operation more than 20 years ago in part because the company recognized that gaming presented an opportunity to bring together various esoteric technologies in a way that everyday people could understand.

Even now, most normal folks have no real idea what "cloud computing" or "machine learning" or "photogrammetry" mean. As they soar over the Kalahari, Hong Kong, Paris, Mumbai or Brooklyn (as they actually sit at home in quarantine), they won't have to. What they will understand is that Microsoft made it possible. The appeal of flying over Earth is so fundamentally human (I'm looking at you, Icarus) and extends so far beyond the stereotypical "gamer" that Flight also presents a powerful marketing opportunity for Microsoft.

For instance, it's easy to imagine Microsoft bundling a three-month trial of Flight into every Office 365 subscription to appeal to students and white-collar professionals who dream of owning their own real airplane. It is even easier to imagine Microsoft using Flight to launch a future desktop version of its xCloud game streaming service so people don't have to own a powerful PC in order to take to the skies. And of course Flight is definitely going to be part of the Xbox Game Pass subscription service that Microsoft is depending upon so heavily for its gaming ambitions.

Goat Rock Beach Goat Rock Beach, on the Pacific in Sonoma County, via Microsoft's Flight Simulator. Screenshot: Seth Schiesel/Protocol

As for Neumann, he believes the version of Flight the world will see on Tuesday is just the beginning.

"We can spin up as many virtual machines as we want," he said with no small bit of pride. "If we want to have a million-animal caribou herd where every caribou is running its own AI, we can do that. The boundary of the local machine is broken. That is no longer a barrier for us to do things. Now it really just comes down to, 'What do you want to simulate?' We dream about these things a lot. Now, the dream is no longer some vapor that goes away. It is all totally possible."

Fintech

Judge Zia Faruqui is trying to teach you crypto, one ‘SNL’ reference at a time

His decisions on major cryptocurrency cases have quoted "The Big Lebowski," "SNL," and "Dr. Strangelove." That’s because he wants you — yes, you — to read them.

The ways Zia Faruqui (right) has weighed on cases that have come before him can give lawyers clues as to what legal frameworks will pass muster.

Photo: Carolyn Van Houten/The Washington Post via Getty Images

“Cryptocurrency and related software analytics tools are ‘The wave of the future, Dude. One hundred percent electronic.’”

That’s not a quote from "The Big Lebowski" — at least, not directly. It’s a quote from a Washington, D.C., district court memorandum opinion on the role cryptocurrency analytics tools can play in government investigations. The author is Magistrate Judge Zia Faruqui.

Keep Reading Show less
Veronica Irwin

Veronica Irwin (@vronirwin) is a San Francisco-based reporter at Protocol covering fintech. Previously she was at the San Francisco Examiner, covering tech from a hyper-local angle. Before that, her byline was featured in SF Weekly, The Nation, Techworker, Ms. Magazine and The Frisc.

The financial technology transformation is driving competition, creating consumer choice, and shaping the future of finance. Hear from seven fintech leaders who are reshaping the future of finance, and join the inaugural Financial Technology Association Fintech Summit to learn more.

Keep Reading Show less
FTA
The Financial Technology Association (FTA) represents industry leaders shaping the future of finance. We champion the power of technology-centered financial services and advocate for the modernization of financial regulation to support inclusion and responsible innovation.
Enterprise

AWS CEO: The cloud isn’t just about technology

As AWS preps for its annual re:Invent conference, Adam Selipsky talks product strategy, support for hybrid environments, and the value of the cloud in uncertain economic times.

Photo: Noah Berger/Getty Images for Amazon Web Services

AWS is gearing up for re:Invent, its annual cloud computing conference where announcements this year are expected to focus on its end-to-end data strategy and delivering new industry-specific services.

It will be the second re:Invent with CEO Adam Selipsky as leader of the industry’s largest cloud provider after his return last year to AWS from data visualization company Tableau Software.

Keep Reading Show less
Donna Goodison

Donna Goodison (@dgoodison) is Protocol's senior reporter focusing on enterprise infrastructure technology, from the 'Big 3' cloud computing providers to data centers. She previously covered the public cloud at CRN after 15 years as a business reporter for the Boston Herald. Based in Massachusetts, she also has worked as a Boston Globe freelancer, business reporter at the Boston Business Journal and real estate reporter at Banker & Tradesman after toiling at weekly newspapers.

Image: Protocol

We launched Protocol in February 2020 to cover the evolving power center of tech. It is with deep sadness that just under three years later, we are winding down the publication.

As of today, we will not publish any more stories. All of our newsletters, apart from our flagship, Source Code, will no longer be sent. Source Code will be published and sent for the next few weeks, but it will also close down in December.

Keep Reading Show less
Bennett Richardson

Bennett Richardson ( @bennettrich) is the president of Protocol. Prior to joining Protocol in 2019, Bennett was executive director of global strategic partnerships at POLITICO, where he led strategic growth efforts including POLITICO's European expansion in Brussels and POLITICO's creative agency POLITICO Focus during his six years with the company. Prior to POLITICO, Bennett was co-founder and CMO of Hinge, the mobile dating company recently acquired by Match Group. Bennett began his career in digital and social brand marketing working with major brands across tech, energy, and health care at leading marketing and communications agencies including Edelman and GMMB. Bennett is originally from Portland, Maine, and received his bachelor's degree from Colgate University.

Enterprise

Why large enterprises struggle to find suitable platforms for MLops

As companies expand their use of AI beyond running just a few machine learning models, and as larger enterprises go from deploying hundreds of models to thousands and even millions of models, ML practitioners say that they have yet to find what they need from prepackaged MLops systems.

As companies expand their use of AI beyond running just a few machine learning models, ML practitioners say that they have yet to find what they need from prepackaged MLops systems.

Photo: artpartner-images via Getty Images

On any given day, Lily AI runs hundreds of machine learning models using computer vision and natural language processing that are customized for its retail and ecommerce clients to make website product recommendations, forecast demand, and plan merchandising. But this spring when the company was in the market for a machine learning operations platform to manage its expanding model roster, it wasn’t easy to find a suitable off-the-shelf system that could handle such a large number of models in deployment while also meeting other criteria.

Some MLops platforms are not well-suited for maintaining even more than 10 machine learning models when it comes to keeping track of data, navigating their user interfaces, or reporting capabilities, Matthew Nokleby, machine learning manager for Lily AI’s product intelligence team, told Protocol earlier this year. “The duct tape starts to show,” he said.

Keep Reading Show less
Kate Kaye

Kate Kaye is an award-winning multimedia reporter digging deep and telling print, digital and audio stories. She covers AI and data for Protocol. Her reporting on AI and tech ethics issues has been published in OneZero, Fast Company, MIT Technology Review, CityLab, Ad Age and Digiday and heard on NPR. Kate is the creator of RedTailMedia.org and is the author of "Campaign '08: A Turning Point for Digital Media," a book about how the 2008 presidential campaigns used digital media and data.

Latest Stories
Bulletins