Power

Why Microsoft's new Flight Simulator should make Google and Amazon nervous

Stunning visuals and world-building show Microsoft is far ahead of its rivals in bringing different teams and different technologies together.

A screenshot from Flight Simulator showing a plane over the British Virgin Islands

Cruising over the British Virgin Islands in Microsoft's new Flight Simulator.

Screenshot: Seth Schiesel/Protocol

Quick, what's Microsoft's most venerable franchise?

Nope, it's not Office (1990), Windows (1985) or even Word (1983). It's actually Microsoft Flight Simulator, first released in November 1982.

Microsoft, still then an upstart in the fledgling personal computer business, commissioned and acquired Flight during Reagan's first term as a showcase to demonstrate the power of "modern" technology to everyday consumers and to the company's corporate rivals.

Thirty-eight years later, on Tuesday, Microsoft is releasing a stunning, groundbreaking new version of Flight — the first since 2006 — for exactly the same reason. Microsoft is no longer an upstart, but Flight may provide the most vivid, consumer-friendly demonstration in years of how the bleeding edge of modern technology can provide fundamentally new human experiences and (quite literally) change how we see our world. In that sense, Flight may become Microsoft's most effective marketing vehicle in years.

What makes the new Flight so revolutionary is that it's simply the most detailed and realistic model of Earth yet created. Call it a digital twin. In Flight, being released initially for powerful Windows PCs, you can go anywhere on the planet almost instantaneously, and it looks like that place on the planet (with the general exception of restricted military areas). The buildings, the hills, the vegetation, the water, the sky, the clouds — it's all there, everywhere. The degree of freedom produces an intellectual effect that is at first a bit disorienting — perhaps akin to the first time you realized you could just type a question into Google and get an answer.

Flight planning in Microsoft's new Flight Simulator.Screenshot: Seth Schiesel/Protocol

With Flight, it is superficially easy to focus merely on the graphics, which are amazing in the sense that they are realistic. But the real leap forward — what makes Flight so powerful for Microsoft and so potentially concerning to its Big Tech brethren — is how adroitly the company has integrated many different layers of advanced technology to create a fresh, easy-to-use consumer experience that truly has no competition.

In February, Phil Spencer, Microsoft's head of Xbox, made some waves when he said that his company's true competitors in interactive entertainment were no longer traditional gaming rivals Sony and Nintendo, but rather Big Tech rivals Amazon and Google. For those two companies, Flight should be a sobering reminder of just how far behind Microsoft they are in gaming in particular, but also perhaps more generally in their ability to bring different internal groups together for a common cause.

Above Manhattan in the new Flight Simulator.Screenshot: Seth Schiesel/Protocol

How to Flight

For Jörg Neumann, a longtime Microsoft executive who is in charge of the team, the new Flight got its start at Machu Picchu.

Around six years ago, he was working on a product called HoloTour for Microsoft's HoloLens augmented reality headset. "And that's basically what inspired Flight Sim because you can't build Machu Picchu, right?" he told Protocol. "You can build all the buildings and roam on a street level, but Machu Picchu is this sprawling mountainside and endless vistas. So I ended up talking to the Bing Maps team and they had the entire planet. They have the height field and also aerial photography so we basically used that, put it into the engine and added a few things — like the actual ruins of Machu Picchu — and I swear the first time I put on the headset, the kernel of the idea was right there: 'Man, if we could do this planet-wide wouldn't that be amazing?'"

For HoloTour, Neumann was working with Asobo Studio in Bordeaux, France, which had developed a graphics engine specialized for huge virtual environments. Neumann discovered that the Bing team had detailed photogrammetry data down to 5-centimeter resolution for more than 400 cities. Microsoft contracts with various providers around the world, such as Vexcel Imaging, to fly all over the world using technologies including lidar (a laser version of radar), to generate imagery and models of the real world.

Neumann got the Bing photogrammetry for Seattle ("30, 40 gigs or something") and reached back out to the French studio in early 2017. "Asobo built themselves a quick Cessna 172 or something, and we flew over it, and it looked exactly like the city of Seattle at daytime," he said. Neumann showed a video to Spencer and got the green light to resurrect Flight.

Near Sedona, Arizona, in Flight Simulator. Screenshot: Seth Schiesel/Protocol

Microsoft didn't have detailed photogrammetry for the whole planet, though, so it had to use proprietary satellite imagery in addition to traditional aerial photography. That's where a company called Blackshark.ai in Graz, Austria, came in. It just so happens that Bing's camera technology had been developed at a university there, and the town is home to a community of vision scientists.

Blackshark initially approached Neumann with a pitch to make a game about drone racing in the Alps. Neuman wasn't interested in a drone game but was definitely interested in the Alps. "They had things figured out like analyzing buildings and the color of the rooftops, and it was clear they had done a lot of work on machine learning, so I told them the vision: 'Hey, I just want to be able to go anywhere on Earth, and it has to look exactly right,'" he said. Blackshark became Asobo's co-developer on Flight.

Waxahachie, Texas, in Flight Simulator. Waxahachie, Texas, in Flight Simulator. Screenshot: Seth Schiesel/Protocol

In the meantime, Neumann went to Switzerland to meet with some hardcore weather nerds about incorporating their models into the product. "They can tell us stuff like the altitude ice condensates at under various conditions," he said. The team also leveraged technology from Microsoft's discontinued Photosynth project, which generates 3D models from 2D photos.

The algorithms and data — including OpenStreetMap — were then fed into Microsoft's vast Azure computing cloud to generate Flight's 2.5-petabyte model, which includes 2 trillion trees, 1.5 billion buildings (you can probably find your house), 117 million lakes and just about every road, mountain, city and airport on the planet.

Azure is used not just to generate the data but also to deliver it to players. No personal computer could hold all that information, so Flight streams real-world geography, weather and air traffic information to users as they play. (There is also a less-detailed offline mode.)

"I sat in a meeting this morning with the Bing Maps guys, and we were looking at a grid of the world that is being updated because there are photographs being taken right this moment, and they will be in the product in two months," Neumann said. "Flight Simulator is a living, breathing thing that has never been possible before."

The true 'next generation'

Every company that could be considered Big Tech is investing billions of dollars into gaming. This fall, Sony and Microsoft will release major new game machines, their first in seven years.

These waves are often referred to as generations; gamers (and marketers) love to talk about "next-generation" games and "next-generation" experiences. But for most of the last 15 years or so, "next generation" has meant little more than prettier explosions and faster load times. Even as processing power, storage and bandwidth — the basic constraints of electronics (along with power consumption) — have exploded in capacity, the types of entertainment and education experiences they are used to provide have remained essentially static. Even if they are delivered differently, these new conduits are usually being used to provide access to the same old stuff.

A summer rain shower in the Alps in Flight Simulator. Screenshot: Seth Schiesel/Protocol

For example, Google's Stadia cloud gaming service is amazing in the sense that it actually provides a gaming experience in a Chrome browser on potato-power systems that is, let's say, 90% as good as playing on local hardware. But it is providing access to games that generally make zero use of advanced computing in their basic design. When Google announced Stadia last year, it teased fundamentally new sorts of interactive experiences like jumping into a game with a YouTube streamer in real time. But it has not delivered on those yet.

Amazon is having related difficulties with its own gaming efforts. Amazon wants to make games that exploit the vast power of the Amazon Web Services platform but is struggling to harness that power in the service of experiences that are actually fun ("fun" being perhaps the least quantifiable metric out there). Amazon hopes to make that leap in its coming online fantasy adventure New World by letting players fight in scrums of hundreds (though still far smaller than the multi-thousand-player space battles in Eve Online), but we will just have to see.

Sony and Nintendo, the traditional Japanese gaming giants, each routinely churns out games of the highest quality. But they are almost always incremental, rather than truly revolutionary, in their progression: prettier explosions, faster load times. That's because Sony and Nintendo just don't have the vast technical and financial capabilities of the Big Tech cabal.

And so it's easy to see Microsoft's new Flight Simulator as the first truly "next-generation" digital experience in a long time. It evokes the childlike wonder of spinning a globe, except now you can actually zoom all the way in and check it out for yourself.

The 100,000-foot view

Microsoft created its own gaming operation more than 20 years ago in part because the company recognized that gaming presented an opportunity to bring together various esoteric technologies in a way that everyday people could understand.

Even now, most normal folks have no real idea what "cloud computing" or "machine learning" or "photogrammetry" mean. As they soar over the Kalahari, Hong Kong, Paris, Mumbai or Brooklyn (as they actually sit at home in quarantine), they won't have to. What they will understand is that Microsoft made it possible. The appeal of flying over Earth is so fundamentally human (I'm looking at you, Icarus) and extends so far beyond the stereotypical "gamer" that Flight also presents a powerful marketing opportunity for Microsoft.

For instance, it's easy to imagine Microsoft bundling a three-month trial of Flight into every Office 365 subscription to appeal to students and white-collar professionals who dream of owning their own real airplane. It is even easier to imagine Microsoft using Flight to launch a future desktop version of its xCloud game streaming service so people don't have to own a powerful PC in order to take to the skies. And of course Flight is definitely going to be part of the Xbox Game Pass subscription service that Microsoft is depending upon so heavily for its gaming ambitions.

Goat Rock Beach Goat Rock Beach, on the Pacific in Sonoma County, via Microsoft's Flight Simulator. Screenshot: Seth Schiesel/Protocol

As for Neumann, he believes the version of Flight the world will see on Tuesday is just the beginning.

"We can spin up as many virtual machines as we want," he said with no small bit of pride. "If we want to have a million-animal caribou herd where every caribou is running its own AI, we can do that. The boundary of the local machine is broken. That is no longer a barrier for us to do things. Now it really just comes down to, 'What do you want to simulate?' We dream about these things a lot. Now, the dream is no longer some vapor that goes away. It is all totally possible."

Protocol | Workplace

Productivity apps can’t stop making money

ClickUp had one of the biggest Series C funding rounds ever. Here's how it matches up to the other productivity unicorns.

ClickUp made $400 million in its series C funding round.

Photo: ClickUp

Productivity platform ClickUp announced a milestone today. The company raised $400 million, which is one of the biggest series C funding rounds in the workplace productivity market ever. The round, led by Andreessen Horowitz and Tiger Global, put the private company at a $4 billion valuation post-money.

In case it's not clear: This is a massive amount of money. It shows how hot the productivity space is right now, with some predicting the market size could reach almost $120 billion by 2028. In a world of hybrid workers, all-in-one tool platforms are all the rage among both startups and productivity stalwarts. Companies everywhere want to escape tool overwhelm, where work is spread across dozens of apps.

Keep Reading Show less
Lizzy Lawrence

Lizzy Lawrence ( @LizzyLaw_) is a reporter at Protocol, covering tools and productivity in the workplace. She's a recent graduate of the University of Michigan, where she studied sociology and international studies. She served as editor in chief of The Michigan Daily, her school's independent newspaper. She's based in D.C., and can be reached at llawrence@protocol.com.

If you've ever tried to pick up a new fitness routine like running, chances are you may have fallen into the "motivation vs. habit" trap once or twice. You go for a run when the sun is shining, only to quickly fall off the wagon when the weather turns sour.

Similarly, for many businesses, 2020 acted as the storm cloud that disrupted their plans for innovation. With leaders busy grappling with the pandemic, innovation frequently got pushed to the backburner. In fact, according to McKinsey, the majority of organizations shifted their focus mainly to maintaining business continuity throughout the pandemic.

Keep Reading Show less
Gaurav Kataria
Group Product Manager, Trello at Atlassian
The Supreme Court of the United States
Photo: Angel Xavier Viera-Vargas

If a company resolved a data breach in the past, does it need to disclose the potential negative fallout of that breach as a risk to investors later on? In a new petition asking the Supreme Court to take up the question, Alphabet is arguing emphatically: no. And it's using the ol' "the past is history, tomorrow's a mystery" defense.

Keep Reading Show less
Issie Lapowsky

Issie Lapowsky ( @issielapowsky) is Protocol's chief correspondent, covering the intersection of technology, politics, and national affairs. She also oversees Protocol's fellowship program. Previously, she was a senior writer at Wired, where she covered the 2016 election and the Facebook beat in its aftermath. Prior to that, Issie worked as a staff writer for Inc. magazine, writing about small business and entrepreneurship. She has also worked as an on-air contributor for CBS News and taught a graduate-level course at New York University's Center for Publishing on how tech giants have affected publishing.

Protocol | Workplace

Facebook’s hiring crisis: Engineers are turning down offers

"All of you are now starting to experience that major imbalance between supply and demand — and it doesn't feel good," a recruiting leader wrote in an internal memo.

Here are all the Facebook Papers stories
Image: Getty Images, Protocol

Facebook cannot find enough candidates to meet engineering demand, especially in the Bay Area, and has struggled and failed to meet early 2021 recruiting goals, according to a detailed internal memo outlining recruitment strategy and hiring pains.

The company also failed to meet hiring goals in 2019, which frustrated CEO Mark Zuckerberg, and it built an ad-hoc team of leaders to create an emergency plan to address the painful shortage, according to disclosures made to the Securities and Exchange Commission and provided to Congress in redacted form by Frances Haugen's legal counsel. A consortium of news organizations, including Protocol, has reviewed the redacted versions received by Congress.

Keep Reading Show less
Anna Kramer

Anna Kramer is a reporter at Protocol (Twitter: @ anna_c_kramer, email: akramer@protocol.com), where she writes about labor and workplace issues. Prior to joining the team, she covered tech and small business for the San Francisco Chronicle and privacy for Bloomberg Law. She is a recent graduate of Brown University, where she studied International Relations and Arabic and wrote her senior thesis about surveillance tools and technological development in the Middle East.

Theranos trial reveals DeVos family invested $100 million

The family committed "on the spot" to double its investment, an investment adviser said. Meanwhile, the jury lost another two members, with two alternates left.

Betsy DeVos' family invested $100 million in Theranos, an investment adviser said.

Photo: Alex Wong/Getty Images

Lisa Peterson, a wealth manager for the DeVos family, testified in Elizabeth Holmes's criminal fraud trial Tuesday, as prosecutors continued to highlight allegations about how the Theranos CEO courted investors in the once-high-flying blood-testing startup.

An email presented by the defense revealed that the family committed to doubling their investment in Theranos to $100 million "on the spot" during a 2014 visit to company headquarters.

Keep Reading Show less
Michelle Ma
Michelle Ma (@himichellema) is a reporter at Protocol, where she writes about management, leadership and workplace issues in tech. Previously, she was a news editor of live journalism and special coverage for The Wall Street Journal. Prior to that, she worked as a staff writer at Wirecutter. She can be reached at mma@protocol.com.
Latest Stories