Power

Inside Amazon’s quest to build more lively gadgets, starting with making the Echo Show 10 move

The company's latest smart device features a rotating display, and Amazon is allowing developers to tap into the feature to make their Alexa skills multimodal.

Amazon Echo Show 10 with motion

Amazon's Echo Show 10, performing the "mild expressive shake" motion

Courtesy of Amazon

Get ready for Alexa to wave at you.

Well, not literally. Amazon hasn't built a humanoid smart home robot just yet, but the company's latest Echo Show 10 smart display is a small step in that direction. Equipped with a display that can rotate 360 degrees around its speaker base, the Echo Show 10 is the first device of its kind that can physically move. When you say "Alexa, goodbye," it not only responds with audible and visual feedback, but also performs a kind of sweeping motion with its display, resembling a slow farewell wave.

Originally conceived as a way to keep the display in view while you move around the kitchen, Amazon's hardware team has turned this motion feature into a novel way to interact with the device. For instance, if you profess your love to Alexa, the Echo Show 10 does a little happy dance, with the display bobbing back and forth. "It seemed appropriate for Alexa to look cheerful when you say you love her," Amazon's Alexa devices and developer technologies VP, Nedim Fresko, said.

For Fresko and his team, adding motion to the Echo Show 10 was just the latest step toward a more responsive and multimodal future of ambient computing. The company also opened the feature up to developers, and began to choreograph a set of predefined motions to make Alexa's skills more expressive. In an exclusive conversation with Protocol, Fresko explained why Amazon is betting on motion, how his team conceives new features for smart home devices and why Amazon executives regularly spend hours reading customer reviews.

Working backward from an idea

Out of all the things Amazon could have added to its smart displays, giving developers the ability to make it wave and shake may not have been the most obvious choice. Fresko said that it all started with an idea for solving a common customer problem with regular smart displays. "Every year, we have new ideas for working-backwards documents, as we call them, where we consider the customer need, and then we try to look at which technologies can achieve that customer need," he explained.

"Motion was described in exactly the same way a couple of years ago, where we described how it would behave before any hardware or software was built," Fresko said. "We turned that around until we really loved it. Then we built the technology."

That early idea was all about making screens a key part of the ambient computing experience. "Sometimes, when you're multitasking in the kitchen, or walking around the home, you have to face a choice: You're either facing the screen or you have to make do with voice," Fresko said. "So we thought: What if we could build a system where the screen always faces you? No matter where you are, you'll find the screen facing you."

A screen that moves with you, aided by the built-in camera and its ability to track your movement across the room, seemed like a good solution for cooking in the kitchen. It also lent itself naturally to Amazon's push into video chat. "Person-to-person communication, where you're always in the frame no matter where you are, was kind of consistent with the overall vision," Fresko said.

Opening up the feature to developers

Once Amazon's hardware team began working on the technology, it had to keep it under the lid for a while. "It's hard to go out there and do a survey about motion," he said. "That kind of thing is an internal idea." However, the company did reach out to a few trusted developers before it officially announced the Echo Show 10 with motion last October. "If we find that the experience is useful for consumers, we automatically assume that developers will also find it useful for similar reasons," Fresko said.

Some of the third-party developers to make use of motion include Comedy Central, Sony and Universal Games. One of the few smaller companies that got early access is Volley, a San Francisco-based startup that builds casual games for smart speakers. "It's fun," Volley CEO Max Child said. "It's like a cute little robot from a movie."

Volley has added the Echo 10's ability to have the display follow a user to its Song Quiz Alexa skill, and Child is already thinking about additional applications that could add to the company's multiplayer games. "Secret information is interesting, and also just cues around whose turn it is, or who should be engaging, that kind of thing," he said.

In addition to giving developers access to the basic motion that keeps the display in view, Amazon has also begun to release a few choreographed motions that can be used to make skills more dynamic. "We added a few preprogrammed choreos to express emotion, and we also added something called 'entity sensing' that allows you to act as input yourself," Fresko said. "As you move, the UI could react with updates based on your motion."

"We'll add more of these personality and reaction features," Child said. "There's a lot of opportunity to make these devices more friendly and more endearing."

On Wednesday, Amazon released three new such "choreos," including one aptly called "MildExpressiveShakes." To develop these and other choreographed motions, Amazon's designers spitballed a bunch of different ideas on what Fresko described as a "mild to wild" scale. "They basically disregarded all constraints and tried to see what would feel delightful, what would evoke emotion," he said. "Then they thought about what experiences they could attach those motions to, and eventually narrowed the focus to a few that we felt were in good taste, delightful, useful."

As part of that process, they also had to throw out a bunch of different motions that were simply too exhausting or dizzying. "It's very easy to go overboard with these things," Fresko said. "Sometimes, you just have to restrain yourself."

Getting it wrong to ultimately get it right

It's too early to say whether consumers are willing to pay a premium for motion. The Echo Show 10 became available for $250 at the end of February; early reviews on Amazon.com are generally positive, but some reviewers have taken issue with the feature. "The first thing I turned off was motion," one customer noted, who complained about a lack of accuracy when the display follows you. A few others had similar gripes, with another consumer expressing hope that things could improve: "The follow-mode feels more gimmicky than useful right now but that could change as they update."

"Once we launch, our customers publicly grade us," Fresko said. "Very publicly grade us, and tell us exactly what they like and don't like." Amazon keeps close tabs on those comments, and has developed automated ways to pay attention to thousands of reviews. But Fresko is also going through them manually. "In the few months that are after a device's launch, I read every single review, without exception," he said. "They're very interesting. We're very grateful to our customers for being frank about that feedback."

One reason these reviews are so valuable is that ambient computing is still so novel. The first Echo smart speaker was released in 2014, and the company's first smart display only became available in 2017. Both products ultimately laid the foundation for entire product categories, and turned Amazon into the market leader in the smart speaker space. But they began as unproven bets, just as the Echo Show 10's motion feature.

"We keep on placing bets in areas where we have a really good theory that customers will find it useful," Fresko said. "But there's only one way to find out, which is to build it."

Some bets ultimately don't pan out, as Fresko knows all too well. Before working on Alexa devices, he was director of engineering for Amazon's Fire phone. Just like the Echo Show 10, that device had a very unique and interesting feature that set it apart from the competition: a 3D display that gave consumers a perception of depth by tracking their viewing angle. The device nonetheless failed miserably, and Amazon pulled the plug on the project after just one year.

"You imagine customer utility, you write about it, you convince yourself, and then you realize the only way to find out if it's going to work is to actually build it," Fresko said. "Sometimes there is awesome customer interest, and sometimes there isn't. But there's no way to find out unless you try. So I think of the phone as exactly that. There was a very good theory behind its utility. It didn't find the reception we expected, and therefore we moved to the next thing. And guess what? The next thing was incredibly successful."

Fintech

Election markets are far from a sure bet

Kalshi has big-name backing for its plan to offer futures contracts tied to election results. Will that win over a long-skeptical regulator?

Whether Kalshi’s election contracts could be considered gaming or whether they serve a true risk-hedging purpose is one of the top questions the CFTC is weighing in its review.

Photo illustration: Getty Images; Protocol

Crypto isn’t the only emerging issue on the CFTC’s plate. The futures regulator is also weighing a fintech sector that has similarly tricky political implications: election bets.

The Commodity Futures Trading Commission has set Oct. 28 as a date by which it hopes to decide whether the New York-based startup Kalshi can offer a form of wagering up to $25,000 on which party will control the House of Representatives and Senate after the midterms. PredictIt, another online market for election trading, has also sued the regulator over its decision to cancel a no-action letter.

Keep Reading Show less
Ryan Deffenbaugh
Ryan Deffenbaugh is a reporter at Protocol focused on fintech. Before joining Protocol, he reported on New York's technology industry for Crain's New York Business. He is based in New York and can be reached at rdeffenbaugh@protocol.com.
Sponsored Content

Great products are built on strong patents

Experts say robust intellectual property protection is essential to ensure the long-term R&D required to innovate and maintain America's technology leadership.

Every great tech product that you rely on each day, from the smartphone in your pocket to your music streaming service and navigational system in the car, shares one important thing: part of its innovative design is protected by intellectual property (IP) laws.

From 5G to artificial intelligence, IP protection offers a powerful incentive for researchers to create ground-breaking products, and governmental leaders say its protection is an essential part of maintaining US technology leadership. To quote Secretary of Commerce Gina Raimondo: "intellectual property protection is vital for American innovation and entrepreneurship.”

Keep Reading Show less
James Daly
James Daly has a deep knowledge of creating brand voice identity, including understanding various audiences and targeting messaging accordingly. He enjoys commissioning, editing, writing, and business development, particularly in launching new ventures and building passionate audiences. Daly has led teams large and small to multiple awards and quantifiable success through a strategy built on teamwork, passion, fact-checking, intelligence, analytics, and audience growth while meeting budget goals and production deadlines in fast-paced environments. Daly is the Editorial Director of 2030 Media and a contributor at Wired.
Enterprise

The Uber verdict shows why mandatory disclosure isn't such a bad idea

The conviction of Uber's former chief security officer, Joe Sullivan, seems likely to change some minds in the debate over proposed cyber incident reporting regulations.

Executives and boards will now be "a whole lot less likely to cover things up," said one information security veteran.

Photo: Al Drago/Bloomberg via Getty Images

If nothing else, the guilty verdict delivered Wednesday in a case involving Uber's former security head will have this effect on how breaches are handled in the future: Executives and boards, according to information security veteran Michael Hamilton, will be "a whole lot less likely to cover things up."

Following the conviction of former Uber chief security officer Joe Sullivan, "we likely will get better voluntary reporting" of cyber incidents, said Hamilton, formerly the chief information security officer of the City of Seattle, and currently the founder and CISO at cybersecurity vendor Critical Insight.

Keep Reading Show less
Kyle Alspach

Kyle Alspach ( @KyleAlspach) is a senior reporter at Protocol, focused on cybersecurity. He has covered the tech industry since 2010 for outlets including VentureBeat, CRN and the Boston Globe. He lives in Portland, Oregon, and can be reached at kalspach@protocol.com.

Climate

Delta and MIT are running flight tests to fix contrails

The research team and airline are running flight tests to determine if it’s possible to avoid the climate-warming effects of contrails.

Delta and MIT just announced a partnership to test how to mitigate persistent contrails.

Photo: Gabriela Natiello/Unsplash

Contrails could be responsible for up to 2% of all global warming, and yet how they’re formed and how to mitigate them is barely understood by major airlines.

That may be changing.

Keep Reading Show less
Michelle Ma

Michelle Ma (@himichellema) is a reporter at Protocol covering climate. Previously, she was a news editor of live journalism and special coverage for The Wall Street Journal. Prior to that, she worked as a staff writer at Wirecutter. She can be reached at mma@protocol.com.

Entertainment

Inside Amazon’s free video strategy

Amazon has been doubling down on original content for Freevee, its ad-supported video service, which has seen a lot of growth thanks to a deep integration with other Amazon properties.

Freevee’s investment into original programming like 'Bosch: Legacy' has increased by 70%.

Photo: Tyler Golden/Amazon Freevee

Amazon’s streaming efforts have long been all about Prime Video. So the company caught pundits by surprise when, in early 2019, it launched a stand-alone ad-supported streaming service called IMDb Freedive, with Techcrunch calling the move “a bit odd.”

Nearly four years and two rebrandings later, Amazon’s ad-supported video efforts appear to be flourishing. Viewership of the service grew by 138% from 2020 to 2021, according to Amazon. The company declined to share any updated performance data on the service, which is now called Freevee, but a spokesperson told Protocol the performance of originals in particular “exceeded expectations,” leading Amazon to increase investments into original content by 70% year-over-year.

Keep Reading Show less
Janko Roettgers

Janko Roettgers (@jank0) is a senior reporter at Protocol, reporting on the shifting power dynamics between tech, media, and entertainment, including the impact of new technologies. Previously, Janko was Variety's first-ever technology writer in San Francisco, where he covered big tech and emerging technologies. He has reported for Gigaom, Frankfurter Rundschau, Berliner Zeitung, and ORF, among others. He has written three books on consumer cord-cutting and online music and co-edited an anthology on internet subcultures. He lives with his family in Oakland.

Latest Stories
Bulletins