Quick, what's Microsoft's most venerable franchise?
Nope, it's not Office (1990), Windows (1985) or even Word (1983). It's actually Microsoft Flight Simulator, first released in November 1982.
Microsoft, still then an upstart in the fledgling personal computer business, commissioned and acquired Flight during Reagan's first term as a showcase to demonstrate the power of "modern" technology to everyday consumers and to the company's corporate rivals.
Thirty-eight years later, on Tuesday, Microsoft is releasing a stunning, groundbreaking new version of Flight — the first since 2006 — for exactly the same reason. Microsoft is no longer an upstart, but Flight may provide the most vivid, consumer-friendly demonstration in years of how the bleeding edge of modern technology can provide fundamentally new human experiences and (quite literally) change how we see our world. In that sense, Flight may become Microsoft's most effective marketing vehicle in years.
What makes the new Flight so revolutionary is that it's simply the most detailed and realistic model of Earth yet created. Call it a digital twin. In Flight, being released initially for powerful Windows PCs, you can go anywhere on the planet almost instantaneously, and it looks like that place on the planet (with the general exception of restricted military areas). The buildings, the hills, the vegetation, the water, the sky, the clouds — it's all there, everywhere. The degree of freedom produces an intellectual effect that is at first a bit disorienting — perhaps akin to the first time you realized you could just type a question into Google and get an answer.
Flight planning in Microsoft's new Flight Simulator.Screenshot: Seth Schiesel/Protocol
With Flight, it is superficially easy to focus merely on the graphics, which are amazing in the sense that they are realistic. But the real leap forward — what makes Flight so powerful for Microsoft and so potentially concerning to its Big Tech brethren — is how adroitly the company has integrated many different layers of advanced technology to create a fresh, easy-to-use consumer experience that truly has no competition.
In February, Phil Spencer, Microsoft's head of Xbox, made some waves when he said that his company's true competitors in interactive entertainment were no longer traditional gaming rivals Sony and Nintendo, but rather Big Tech rivals Amazon and Google. For those two companies, Flight should be a sobering reminder of just how far behind Microsoft they are in gaming in particular, but also perhaps more generally in their ability to bring different internal groups together for a common cause.
Above Manhattan in the new Flight Simulator.Screenshot: Seth Schiesel/Protocol
How to Flight
For Jörg Neumann, a longtime Microsoft executive who is in charge of the team, the new Flight got its start at Machu Picchu.
Around six years ago, he was working on a product called HoloTour for Microsoft's HoloLens augmented reality headset. "And that's basically what inspired Flight Sim because you can't build Machu Picchu, right?" he told Protocol. "You can build all the buildings and roam on a street level, but Machu Picchu is this sprawling mountainside and endless vistas. So I ended up talking to the Bing Maps team and they had the entire planet. They have the height field and also aerial photography so we basically used that, put it into the engine and added a few things — like the actual ruins of Machu Picchu — and I swear the first time I put on the headset, the kernel of the idea was right there: 'Man, if we could do this planet-wide wouldn't that be amazing?'"
For HoloTour, Neumann was working with Asobo Studio in Bordeaux, France, which had developed a graphics engine specialized for huge virtual environments. Neumann discovered that the Bing team had detailed photogrammetry data down to 5-centimeter resolution for more than 400 cities. Microsoft contracts with various providers around the world, such as Vexcel Imaging, to fly all over the world using technologies including lidar (a laser version of radar), to generate imagery and models of the real world.
Neumann got the Bing photogrammetry for Seattle ("30, 40 gigs or something") and reached back out to the French studio in early 2017. "Asobo built themselves a quick Cessna 172 or something, and we flew over it, and it looked exactly like the city of Seattle at daytime," he said. Neumann showed a video to Spencer and got the green light to resurrect Flight.
Near Sedona, Arizona, in Flight Simulator. Screenshot: Seth Schiesel/Protocol
Microsoft didn't have detailed photogrammetry for the whole planet, though, so it had to use proprietary satellite imagery in addition to traditional aerial photography. That's where a company called Blackshark.ai in Graz, Austria, came in. It just so happens that Bing's camera technology had been developed at a university there, and the town is home to a community of vision scientists.
Blackshark initially approached Neumann with a pitch to make a game about drone racing in the Alps. Neuman wasn't interested in a drone game but was definitely interested in the Alps. "They had things figured out like analyzing buildings and the color of the rooftops, and it was clear they had done a lot of work on machine learning, so I told them the vision: 'Hey, I just want to be able to go anywhere on Earth, and it has to look exactly right,'" he said. Blackshark became Asobo's co-developer on Flight.
Waxahachie, Texas, in Flight Simulator. Screenshot: Seth Schiesel/Protocol
In the meantime, Neumann went to Switzerland to meet with some hardcore weather nerds about incorporating their models into the product. "They can tell us stuff like the altitude ice condensates at under various conditions," he said. The team also leveraged technology from Microsoft's discontinued Photosynth project, which generates 3D models from 2D photos.
The algorithms and data — including OpenStreetMap — were then fed into Microsoft's vast Azure computing cloud to generate Flight's 2.5-petabyte model, which includes 2 trillion trees, 1.5 billion buildings (you can probably find your house), 117 million lakes and just about every road, mountain, city and airport on the planet.
Azure is used not just to generate the data but also to deliver it to players. No personal computer could hold all that information, so Flight streams real-world geography, weather and air traffic information to users as they play. (There is also a less-detailed offline mode.)
"I sat in a meeting this morning with the Bing Maps guys, and we were looking at a grid of the world that is being updated because there are photographs being taken right this moment, and they will be in the product in two months," Neumann said. "Flight Simulator is a living, breathing thing that has never been possible before."
The true 'next generation'
Every company that could be considered Big Tech is investing billions of dollars into gaming. This fall, Sony and Microsoft will release major new game machines, their first in seven years.
These waves are often referred to as generations; gamers (and marketers) love to talk about "next-generation" games and "next-generation" experiences. But for most of the last 15 years or so, "next generation" has meant little more than prettier explosions and faster load times. Even as processing power, storage and bandwidth — the basic constraints of electronics (along with power consumption) — have exploded in capacity, the types of entertainment and education experiences they are used to provide have remained essentially static. Even if they are delivered differently, these new conduits are usually being used to provide access to the same old stuff.
A summer rain shower in the Alps in Flight Simulator. Screenshot: Seth Schiesel/Protocol
For example, Google's Stadia cloud gaming service is amazing in the sense that it actually provides a gaming experience in a Chrome browser on potato-power systems that is, let's say, 90% as good as playing on local hardware. But it is providing access to games that generally make zero use of advanced computing in their basic design. When Google announced Stadia last year, it teased fundamentally new sorts of interactive experiences like jumping into a game with a YouTube streamer in real time. But it has not delivered on those yet.
Amazon is having related difficulties with its own gaming efforts. Amazon wants to make games that exploit the vast power of the Amazon Web Services platform but is struggling to harness that power in the service of experiences that are actually fun ("fun" being perhaps the least quantifiable metric out there). Amazon hopes to make that leap in its coming online fantasy adventure New World by letting players fight in scrums of hundreds (though still far smaller than the multi-thousand-player space battles in Eve Online), but we will just have to see.
Sony and Nintendo, the traditional Japanese gaming giants, each routinely churns out games of the highest quality. But they are almost always incremental, rather than truly revolutionary, in their progression: prettier explosions, faster load times. That's because Sony and Nintendo just don't have the vast technical and financial capabilities of the Big Tech cabal.
And so it's easy to see Microsoft's new Flight Simulator as the first truly "next-generation" digital experience in a long time. It evokes the childlike wonder of spinning a globe, except now you can actually zoom all the way in and check it out for yourself.
The 100,000-foot view
Microsoft created its own gaming operation more than 20 years ago in part because the company recognized that gaming presented an opportunity to bring together various esoteric technologies in a way that everyday people could understand.
Even now, most normal folks have no real idea what "cloud computing" or "machine learning" or "photogrammetry" mean. As they soar over the Kalahari, Hong Kong, Paris, Mumbai or Brooklyn (as they actually sit at home in quarantine), they won't have to. What they will understand is that Microsoft made it possible. The appeal of flying over Earth is so fundamentally human (I'm looking at you, Icarus) and extends so far beyond the stereotypical "gamer" that Flight also presents a powerful marketing opportunity for Microsoft.
For instance, it's easy to imagine Microsoft bundling a three-month trial of Flight into every Office 365 subscription to appeal to students and white-collar professionals who dream of owning their own real airplane. It is even easier to imagine Microsoft using Flight to launch a future desktop version of its xCloud game streaming service so people don't have to own a powerful PC in order to take to the skies. And of course Flight is definitely going to be part of the Xbox Game Pass subscription service that Microsoft is depending upon so heavily for its gaming ambitions.
Goat Rock Beach, on the Pacific in Sonoma County, via Microsoft's Flight Simulator. Screenshot: Seth Schiesel/Protocol
As for Neumann, he believes the version of Flight the world will see on Tuesday is just the beginning.
"We can spin up as many virtual machines as we want," he said with no small bit of pride. "If we want to have a million-animal caribou herd where every caribou is running its own AI, we can do that. The boundary of the local machine is broken. That is no longer a barrier for us to do things. Now it really just comes down to, 'What do you want to simulate?' We dream about these things a lot. Now, the dream is no longer some vapor that goes away. It is all totally possible."
