gaminggamingauthorSeth Schiesel and Shakeel HashimNoneWant to better understand the $150 billion gaming industry? Get Shakeel Hashim's newsletter every Tuesday.03807ace1f
×

Get access to Protocol

I’ve already subscribed

Will be used in accordance with our Privacy Policy

Power

Why Microsoft's new Flight Simulator should make Google and Amazon nervous

Stunning visuals and world-building show Microsoft is far ahead of its rivals in bringing different teams and different technologies together.

A screenshot from Flight Simulator showing a plane over the British Virgin Islands

Cruising over the British Virgin Islands in Microsoft's new Flight Simulator.

Screenshot: Seth Schiesel/Protocol

Quick, what's Microsoft's most venerable franchise?

Nope, it's not Office (1990), Windows (1985) or even Word (1983). It's actually Microsoft Flight Simulator, first released in November 1982.

Microsoft, still then an upstart in the fledgling personal computer business, commissioned and acquired Flight during Reagan's first term as a showcase to demonstrate the power of "modern" technology to everyday consumers and to the company's corporate rivals.

Thirty-eight years later, on Tuesday, Microsoft is releasing a stunning, groundbreaking new version of Flight — the first since 2006 — for exactly the same reason. Microsoft is no longer an upstart, but Flight may provide the most vivid, consumer-friendly demonstration in years of how the bleeding edge of modern technology can provide fundamentally new human experiences and (quite literally) change how we see our world. In that sense, Flight may become Microsoft's most effective marketing vehicle in years.

What makes the new Flight so revolutionary is that it's simply the most detailed and realistic model of Earth yet created. Call it a digital twin. In Flight, being released initially for powerful Windows PCs, you can go anywhere on the planet almost instantaneously, and it looks like that place on the planet (with the general exception of restricted military areas). The buildings, the hills, the vegetation, the water, the sky, the clouds — it's all there, everywhere. The degree of freedom produces an intellectual effect that is at first a bit disorienting — perhaps akin to the first time you realized you could just type a question into Google and get an answer.

Flight planning in Microsoft's new Flight Simulator.Screenshot: Seth Schiesel/Protocol

With Flight, it is superficially easy to focus merely on the graphics, which are amazing in the sense that they are realistic. But the real leap forward — what makes Flight so powerful for Microsoft and so potentially concerning to its Big Tech brethren — is how adroitly the company has integrated many different layers of advanced technology to create a fresh, easy-to-use consumer experience that truly has no competition.

In February, Phil Spencer, Microsoft's head of Xbox, made some waves when he said that his company's true competitors in interactive entertainment were no longer traditional gaming rivals Sony and Nintendo, but rather Big Tech rivals Amazon and Google. For those two companies, Flight should be a sobering reminder of just how far behind Microsoft they are in gaming in particular, but also perhaps more generally in their ability to bring different internal groups together for a common cause.

Above Manhattan in the new Flight Simulator.Screenshot: Seth Schiesel/Protocol

How to Flight

For Jörg Neumann, a longtime Microsoft executive who is in charge of the team, the new Flight got its start at Machu Picchu.

Around six years ago, he was working on a product called HoloTour for Microsoft's HoloLens augmented reality headset. "And that's basically what inspired Flight Sim because you can't build Machu Picchu, right?" he told Protocol. "You can build all the buildings and roam on a street level, but Machu Picchu is this sprawling mountainside and endless vistas. So I ended up talking to the Bing Maps team and they had the entire planet. They have the height field and also aerial photography so we basically used that, put it into the engine and added a few things — like the actual ruins of Machu Picchu — and I swear the first time I put on the headset, the kernel of the idea was right there: 'Man, if we could do this planet-wide wouldn't that be amazing?'"

For HoloTour, Neumann was working with Asobo Studio in Bordeaux, France, which had developed a graphics engine specialized for huge virtual environments. Neumann discovered that the Bing team had detailed photogrammetry data down to 5-centimeter resolution for more than 400 cities. Microsoft contracts with various providers around the world, such as Vexcel Imaging, to fly all over the world using technologies including lidar (a laser version of radar), to generate imagery and models of the real world.

Neumann got the Bing photogrammetry for Seattle ("30, 40 gigs or something") and reached back out to the French studio in early 2017. "Asobo built themselves a quick Cessna 172 or something, and we flew over it, and it looked exactly like the city of Seattle at daytime," he said. Neumann showed a video to Spencer and got the green light to resurrect Flight.

Near Sedona, Arizona, in Flight Simulator. Screenshot: Seth Schiesel/Protocol

Microsoft didn't have detailed photogrammetry for the whole planet, though, so it had to use proprietary satellite imagery in addition to traditional aerial photography. That's where a company called Blackshark.ai in Graz, Austria, came in. It just so happens that Bing's camera technology had been developed at a university there, and the town is home to a community of vision scientists.

Blackshark initially approached Neumann with a pitch to make a game about drone racing in the Alps. Neuman wasn't interested in a drone game but was definitely interested in the Alps. "They had things figured out like analyzing buildings and the color of the rooftops, and it was clear they had done a lot of work on machine learning, so I told them the vision: 'Hey, I just want to be able to go anywhere on Earth, and it has to look exactly right,'" he said. Blackshark became Asobo's co-developer on Flight.

Waxahachie, Texas, in Flight Simulator. Waxahachie, Texas, in Flight Simulator. Screenshot: Seth Schiesel/Protocol

In the meantime, Neumann went to Switzerland to meet with some hardcore weather nerds about incorporating their models into the product. "They can tell us stuff like the altitude ice condensates at under various conditions," he said. The team also leveraged technology from Microsoft's discontinued Photosynth project, which generates 3D models from 2D photos.

The algorithms and data — including OpenStreetMap — were then fed into Microsoft's vast Azure computing cloud to generate Flight's 2.5-petabyte model, which includes 2 trillion trees, 1.5 billion buildings (you can probably find your house), 117 million lakes and just about every road, mountain, city and airport on the planet.

Azure is used not just to generate the data but also to deliver it to players. No personal computer could hold all that information, so Flight streams real-world geography, weather and air traffic information to users as they play. (There is also a less-detailed offline mode.)

"I sat in a meeting this morning with the Bing Maps guys, and we were looking at a grid of the world that is being updated because there are photographs being taken right this moment, and they will be in the product in two months," Neumann said. "Flight Simulator is a living, breathing thing that has never been possible before."

The true 'next generation'

Every company that could be considered Big Tech is investing billions of dollars into gaming. This fall, Sony and Microsoft will release major new game machines, their first in seven years.

These waves are often referred to as generations; gamers (and marketers) love to talk about "next-generation" games and "next-generation" experiences. But for most of the last 15 years or so, "next generation" has meant little more than prettier explosions and faster load times. Even as processing power, storage and bandwidth — the basic constraints of electronics (along with power consumption) — have exploded in capacity, the types of entertainment and education experiences they are used to provide have remained essentially static. Even if they are delivered differently, these new conduits are usually being used to provide access to the same old stuff.

A summer rain shower in the Alps in Flight Simulator. Screenshot: Seth Schiesel/Protocol

For example, Google's Stadia cloud gaming service is amazing in the sense that it actually provides a gaming experience in a Chrome browser on potato-power systems that is, let's say, 90% as good as playing on local hardware. But it is providing access to games that generally make zero use of advanced computing in their basic design. When Google announced Stadia last year, it teased fundamentally new sorts of interactive experiences like jumping into a game with a YouTube streamer in real time. But it has not delivered on those yet.

Amazon is having related difficulties with its own gaming efforts. Amazon wants to make games that exploit the vast power of the Amazon Web Services platform but is struggling to harness that power in the service of experiences that are actually fun ("fun" being perhaps the least quantifiable metric out there). Amazon hopes to make that leap in its coming online fantasy adventure New World by letting players fight in scrums of hundreds (though still far smaller than the multi-thousand-player space battles in Eve Online), but we will just have to see.

Sony and Nintendo, the traditional Japanese gaming giants, each routinely churns out games of the highest quality. But they are almost always incremental, rather than truly revolutionary, in their progression: prettier explosions, faster load times. That's because Sony and Nintendo just don't have the vast technical and financial capabilities of the Big Tech cabal.

And so it's easy to see Microsoft's new Flight Simulator as the first truly "next-generation" digital experience in a long time. It evokes the childlike wonder of spinning a globe, except now you can actually zoom all the way in and check it out for yourself.

The 100,000-foot view

Microsoft created its own gaming operation more than 20 years ago in part because the company recognized that gaming presented an opportunity to bring together various esoteric technologies in a way that everyday people could understand.

Even now, most normal folks have no real idea what "cloud computing" or "machine learning" or "photogrammetry" mean. As they soar over the Kalahari, Hong Kong, Paris, Mumbai or Brooklyn (as they actually sit at home in quarantine), they won't have to. What they will understand is that Microsoft made it possible. The appeal of flying over Earth is so fundamentally human (I'm looking at you, Icarus) and extends so far beyond the stereotypical "gamer" that Flight also presents a powerful marketing opportunity for Microsoft.

For instance, it's easy to imagine Microsoft bundling a three-month trial of Flight into every Office 365 subscription to appeal to students and white-collar professionals who dream of owning their own real airplane. It is even easier to imagine Microsoft using Flight to launch a future desktop version of its xCloud game streaming service so people don't have to own a powerful PC in order to take to the skies. And of course Flight is definitely going to be part of the Xbox Game Pass subscription service that Microsoft is depending upon so heavily for its gaming ambitions.

Goat Rock Beach Goat Rock Beach, on the Pacific in Sonoma County, via Microsoft's Flight Simulator. Screenshot: Seth Schiesel/Protocol

As for Neumann, he believes the version of Flight the world will see on Tuesday is just the beginning.

"We can spin up as many virtual machines as we want," he said with no small bit of pride. "If we want to have a million-animal caribou herd where every caribou is running its own AI, we can do that. The boundary of the local machine is broken. That is no longer a barrier for us to do things. Now it really just comes down to, 'What do you want to simulate?' We dream about these things a lot. Now, the dream is no longer some vapor that goes away. It is all totally possible."

Microsoft wants to replace artists with AI

Better Zoom calls, simpler email attachments, smart iPhone cases and other patents from Big Tech.

Turning your stories into images.

Image: USPTO/Microsoft

Hello and welcome to 2021! The Big Tech patent roundup is back, after a short vacation and … all the things … that happened between the start of the year and now. It seems the tradition of tech companies filing weird and wonderful patents has carried into the new year; there are some real gems from the last few weeks. Microsoft is trying to outsource all creative endeavors to AI; Apple wants to make seat belts less annoying; and Amazon wants to cut down on some of the recyclable waste that its own success has inevitably created.

And remember: The big tech companies file all kinds of crazy patents for things, and though most never amount to anything, some end up defining the future.

Keep Reading Show less
Mike Murphy

Mike Murphy ( @mcwm) is the director of special projects at Protocol, focusing on the industries being rapidly upended by technology and the companies disrupting incumbents. Previously, Mike was the technology editor at Quartz, where he frequently wrote on robotics, artificial intelligence, and consumer electronics.

People

Google's union has big goals — and big roadblocks

Absence of dues, retaliation fears and small numbers could pose problems for the union's dream of collective bargaining, but Googlers are undeterred.

Recruiting union members beyond the early adopters has had its challenges.

Photo: David Paul Morris/Getty Images

When the Alphabet Workers Union launched with more than 200 Googlers at the beginning of the year, it saw a quick flood of new sign-ups, nearly quadrupling membership over a few weeks. But even with the more than 710 members it now represents, the union still stands for just a tiny fraction of Google's more than 200,000 North American employees and contractors. The broader Alphabet workforce could prove difficult to win over, which is a hurdle that could stand in the way of the group's long-term ambitions for substantive culture change and even collective bargaining.

The initial boom of interest from Googlers was thrilling for Alex Peterson, a software engineer and union spokesperson. "It's really reinvigorating what it means to actually be a community of Googlers, which is something that's been eroding over the past four or five years, or even longer."

Keep Reading Show less
Anna Kramer

Anna Kramer is a reporter at Protocol (@ anna_c_kramer), where she helps write and produce Source Code, Protocol's daily newsletter. Prior to joining the team, she covered tech and small business for the San Francisco Chronicle and privacy for Bloomberg Law. She is a recent graduate of Brown University, where she studied International Relations and Arabic and wrote her senior thesis about surveillance tools and technological development in the Middle East.

About Protocol | Enterprise

‘It’s not OK’: Elastic takes aim at AWS, at the risk of major collateral damage

Elastic's long-running dispute with AWS entered a new chapter last week with big changes to two of its open-source projects. AWS now plans to take those projects under its wing.

"I don't know why this is surprising to people," Elastic CEO Shay Banon said in an interview with Protocol.

Photo: Michael Nagle/Getty Images

Fed up with what he sees as unfair competition from AWS, Elastic CEO Shay Banon felt he had no choice but to restrict the way third parties can use two important open-source projects developed by his company. Yet much of enterprise tech thinks he just threw the baby out with the bathwater.

Last Thursday, Elastic published a blog post — curiously titled "Doubling down on open, Part II" — announcing that Elasticsearch and Kibana, two widely used open-source projects in enterprise tech, would no longer be available under the permissive Apache 2.0 license. Instead, all subsequent releases to those projects will only be available under either a controversial new license known as the SSPL, or the Elastic License, both of which were designed to make it difficult for cloud companies to sell managed versions of the open-source projects they're applied to.

Keep Reading Show less
Tom Krazit

Tom Krazit ( @tomkrazit) is a senior reporter at Protocol, covering cloud computing and enterprise technology out of the Pacific Northwest. He has written and edited stories about the technology industry for almost two decades for publications such as IDG, CNET, paidContent, and GeekWire. He served as executive editor of Gigaom and Structure, and most recently produced a leading cloud computing newsletter called Mostly Cloudy.

Power

Everything you need to know about the Roblox direct listing

The company is expected to go public via direct listing on the New York Stock Exchange in February.

Roblox CEO David Baszucki is taking the company public.

Photo: Ian Tuttle/Getty Images

Roblox is a video game platform, though it describes itself alternatively as a "metaverse," "human co-experience platform" and "new category of human interaction." It's expected to go public via direct listing on the New York Stock Exchange in February.

In simpler terms, Roblox enables developers to build games within the Roblox virtual world, which looks like a crossover between Minecraft and Lego. Developers publish and distribute their games through Roblox to an audience of some 31.1 million daily active users.

Keep Reading Show less
Hirsh Chitkara
Hirsh Chitkara (@ChitkaraHirsh) is a researcher at Protocol, based out of New York City. Before joining Protocol, he worked for Business Insider Intelligence, where he wrote about Big Tech, telecoms, workplace privacy, smart cities, and geopolitics. He also worked on the Strategy & Analytics team at the Cleveland Indians.

Big Tech gets a win from Biden’s sweeping immigration actions

Tim Cook and Sundar Pichai praised President Biden's immigration actions, which read like a tech industry wishlist.

Newly-inaugurated President Joe Biden signed two immigration-related executive orders on Wednesday.

Photo: Chip Somodevilla/Getty Images

Immediately after being sworn in as president Wednesday, Joe Biden signed two pro-immigration executive orders and delivered an immigration bill to Congress that reads like a tech industry wishlist. The move drew enthusiastic praise from tech leaders, including Apple CEO Tim Cook and Alphabet CEO Sundar Pichai.

President Biden nullified several of former-President Trump's most hawkish immigration policies. His executive orders reversed the so-called "Muslim ban" and instructed the attorney general and the secretary of Homeland Security to preserve the Deferred Action for Childhood Arrivals, or DACA, program, which the Trump administration had sought to end. He also sent an expansive immigration reform bill to Congress that would provide a pathway to citizenship for undocumented individuals and make it easier for foreign U.S. graduates with STEM degrees to stay in the United States, among other provisions.

Keep Reading Show less
Emily Birnbaum

Emily Birnbaum ( @birnbaum_e) is a tech policy reporter with Protocol. Her coverage focuses on the U.S. government's attempts to regulate one of the most powerful industries in the world, with a focus on antitrust, privacy and politics. Previously, she worked as a tech policy reporter with The Hill after spending several months as a breaking news reporter. She is a Bethesda, Maryland native and proud Kenyon College alumna.

Latest Stories