Power

Inside Amazon’s quest to build more lively gadgets, starting with making the Echo Show 10 move

The company's latest smart device features a rotating display, and Amazon is allowing developers to tap into the feature to make their Alexa skills multimodal.

Amazon Echo Show 10 with motion

Amazon's Echo Show 10, performing the "mild expressive shake" motion

Courtesy of Amazon

Get ready for Alexa to wave at you.

Well, not literally. Amazon hasn't built a humanoid smart home robot just yet, but the company's latest Echo Show 10 smart display is a small step in that direction. Equipped with a display that can rotate 360 degrees around its speaker base, the Echo Show 10 is the first device of its kind that can physically move. When you say "Alexa, goodbye," it not only responds with audible and visual feedback, but also performs a kind of sweeping motion with its display, resembling a slow farewell wave.

Originally conceived as a way to keep the display in view while you move around the kitchen, Amazon's hardware team has turned this motion feature into a novel way to interact with the device. For instance, if you profess your love to Alexa, the Echo Show 10 does a little happy dance, with the display bobbing back and forth. "It seemed appropriate for Alexa to look cheerful when you say you love her," Amazon's Alexa devices and developer technologies VP, Nedim Fresko, said.

For Fresko and his team, adding motion to the Echo Show 10 was just the latest step toward a more responsive and multimodal future of ambient computing. The company also opened the feature up to developers, and began to choreograph a set of predefined motions to make Alexa's skills more expressive. In an exclusive conversation with Protocol, Fresko explained why Amazon is betting on motion, how his team conceives new features for smart home devices and why Amazon executives regularly spend hours reading customer reviews.

Working backward from an idea

Out of all the things Amazon could have added to its smart displays, giving developers the ability to make it wave and shake may not have been the most obvious choice. Fresko said that it all started with an idea for solving a common customer problem with regular smart displays. "Every year, we have new ideas for working-backwards documents, as we call them, where we consider the customer need, and then we try to look at which technologies can achieve that customer need," he explained.

"Motion was described in exactly the same way a couple of years ago, where we described how it would behave before any hardware or software was built," Fresko said. "We turned that around until we really loved it. Then we built the technology."

That early idea was all about making screens a key part of the ambient computing experience. "Sometimes, when you're multitasking in the kitchen, or walking around the home, you have to face a choice: You're either facing the screen or you have to make do with voice," Fresko said. "So we thought: What if we could build a system where the screen always faces you? No matter where you are, you'll find the screen facing you."

A screen that moves with you, aided by the built-in camera and its ability to track your movement across the room, seemed like a good solution for cooking in the kitchen. It also lent itself naturally to Amazon's push into video chat. "Person-to-person communication, where you're always in the frame no matter where you are, was kind of consistent with the overall vision," Fresko said.

Opening up the feature to developers

Once Amazon's hardware team began working on the technology, it had to keep it under the lid for a while. "It's hard to go out there and do a survey about motion," he said. "That kind of thing is an internal idea." However, the company did reach out to a few trusted developers before it officially announced the Echo Show 10 with motion last October. "If we find that the experience is useful for consumers, we automatically assume that developers will also find it useful for similar reasons," Fresko said.

Some of the third-party developers to make use of motion include Comedy Central, Sony and Universal Games. One of the few smaller companies that got early access is Volley, a San Francisco-based startup that builds casual games for smart speakers. "It's fun," Volley CEO Max Child said. "It's like a cute little robot from a movie."

Volley has added the Echo 10's ability to have the display follow a user to its Song Quiz Alexa skill, and Child is already thinking about additional applications that could add to the company's multiplayer games. "Secret information is interesting, and also just cues around whose turn it is, or who should be engaging, that kind of thing," he said.

In addition to giving developers access to the basic motion that keeps the display in view, Amazon has also begun to release a few choreographed motions that can be used to make skills more dynamic. "We added a few preprogrammed choreos to express emotion, and we also added something called 'entity sensing' that allows you to act as input yourself," Fresko said. "As you move, the UI could react with updates based on your motion."

"We'll add more of these personality and reaction features," Child said. "There's a lot of opportunity to make these devices more friendly and more endearing."

On Wednesday, Amazon released three new such "choreos," including one aptly called "MildExpressiveShakes." To develop these and other choreographed motions, Amazon's designers spitballed a bunch of different ideas on what Fresko described as a "mild to wild" scale. "They basically disregarded all constraints and tried to see what would feel delightful, what would evoke emotion," he said. "Then they thought about what experiences they could attach those motions to, and eventually narrowed the focus to a few that we felt were in good taste, delightful, useful."

As part of that process, they also had to throw out a bunch of different motions that were simply too exhausting or dizzying. "It's very easy to go overboard with these things," Fresko said. "Sometimes, you just have to restrain yourself."

Getting it wrong to ultimately get it right

It's too early to say whether consumers are willing to pay a premium for motion. The Echo Show 10 became available for $250 at the end of February; early reviews on Amazon.com are generally positive, but some reviewers have taken issue with the feature. "The first thing I turned off was motion," one customer noted, who complained about a lack of accuracy when the display follows you. A few others had similar gripes, with another consumer expressing hope that things could improve: "The follow-mode feels more gimmicky than useful right now but that could change as they update."

"Once we launch, our customers publicly grade us," Fresko said. "Very publicly grade us, and tell us exactly what they like and don't like." Amazon keeps close tabs on those comments, and has developed automated ways to pay attention to thousands of reviews. But Fresko is also going through them manually. "In the few months that are after a device's launch, I read every single review, without exception," he said. "They're very interesting. We're very grateful to our customers for being frank about that feedback."

One reason these reviews are so valuable is that ambient computing is still so novel. The first Echo smart speaker was released in 2014, and the company's first smart display only became available in 2017. Both products ultimately laid the foundation for entire product categories, and turned Amazon into the market leader in the smart speaker space. But they began as unproven bets, just as the Echo Show 10's motion feature.

"We keep on placing bets in areas where we have a really good theory that customers will find it useful," Fresko said. "But there's only one way to find out, which is to build it."

Some bets ultimately don't pan out, as Fresko knows all too well. Before working on Alexa devices, he was director of engineering for Amazon's Fire phone. Just like the Echo Show 10, that device had a very unique and interesting feature that set it apart from the competition: a 3D display that gave consumers a perception of depth by tracking their viewing angle. The device nonetheless failed miserably, and Amazon pulled the plug on the project after just one year.

"You imagine customer utility, you write about it, you convince yourself, and then you realize the only way to find out if it's going to work is to actually build it," Fresko said. "Sometimes there is awesome customer interest, and sometimes there isn't. But there's no way to find out unless you try. So I think of the phone as exactly that. There was a very good theory behind its utility. It didn't find the reception we expected, and therefore we moved to the next thing. And guess what? The next thing was incredibly successful."

Climate

2- and 3-wheelers dominate oil displacement by EVs

Increasingly widespread EV adoption is starting to displace the use of oil, but there's still a lot of work to do.

More electric mopeds on the road could be an oil demand game-changer.

Photo: Humphrey Muleba/Unsplash

Electric vehicles are starting to make a serious dent in oil use.

Last year, EVs displaced roughly 1.5 million barrels per day, according to a new analysis from BloombergNEF. That is more than double the share EVs displaced in 2015. The majority of the displacement is coming from an unlikely source.

Keep Reading Show less
Lisa Martine Jenkins

Lisa Martine Jenkins is a senior reporter at Protocol covering climate. Lisa previously wrote for Morning Consult, Chemical Watch and the Associated Press. Lisa is currently based in Brooklyn, and is originally from the Bay Area. Find her on Twitter ( @l_m_j_) or reach out via email (ljenkins@protocol.com).

Sponsored Content

Foursquare data story: leveraging location data for site selection

We take a closer look at points of interest and foot traffic patterns to demonstrate how location data can be leveraged to inform better site selecti­on strategies.

Imagine: You’re the leader of a real estate team at a restaurant brand looking to open a new location in Manhattan. You have two options you’re evaluating: one site in SoHo, and another site in the Flatiron neighborhood. Which do you choose?

Keep Reading Show less
Enterprise

The limits of AI and automation for digital accessibility

AI and automated software that promises to make the web more accessible abounds, but people with disabilities and those who regularly test for digital accessibility problems say it can only go so far.

The everyday obstacles blocking people with disabilities from a satisfying digital experience are immense.

Image: alexsl/Getty Images

“It’s a lot to listen to a robot all day long,” said Tina Pinedo, communications director at Disability Rights Oregon, a group that works to promote and defend the rights of people with disabilities.

But listening to a machine is exactly what many people with visual impairments do while using screen reading tools to accomplish everyday online tasks such as paying bills or ordering groceries from an ecommerce site.

Keep Reading Show less
Kate Kaye

Kate Kaye is an award-winning multimedia reporter digging deep and telling print, digital and audio stories. She covers AI and data for Protocol. Her reporting on AI and tech ethics issues has been published in OneZero, Fast Company, MIT Technology Review, CityLab, Ad Age and Digiday and heard on NPR. Kate is the creator of RedTailMedia.org and is the author of "Campaign '08: A Turning Point for Digital Media," a book about how the 2008 presidential campaigns used digital media and data.

Fintech

The crypto crash's violence shocked Circle's CEO

Jeremy Allaire remains upbeat about stablecoins despite the UST wipeout, he told Protocol in an interview.

Allaire said what really caught him by surprise was “how fast the death spiral happened and how violent of a value destruction it was.”

Photo: Heidi Gutman/CNBC/NBCU Photo Bank/NBCUniversal via Getty Images

Circle CEO Jeremy Allaire said he saw the UST meltdown coming about six months ago, long before the stablecoin crash rocked the crypto world.

“This was a house of cards,” he told Protocol. “It was very clear that it was unsustainable and that there would be a very high risk of a death spiral.”

Keep Reading Show less
Benjamin Pimentel

Benjamin Pimentel ( @benpimentel) covers crypto and fintech from San Francisco. He has reported on many of the biggest tech stories over the past 20 years for the San Francisco Chronicle, Dow Jones MarketWatch and Business Insider, from the dot-com crash, the rise of cloud computing, social networking and AI to the impact of the Great Recession and the COVID crisis on Silicon Valley and beyond. He can be reached at bpimentel@protocol.com or via Google Voice at (925) 307-9342.

A DTC baby formula startup is caught in the center of a supply chain crisis

After weeks of “unprecedented growth,” Bobbie co-founder Laura Modi made a hard decision: to not accept any more new customers.

Parents unable to track down formula in stores have been turning to Facebook groups, homemade formula recipes and Bobbie, a 4-year-old subscription baby formula company.

Photo: JIM WATSON/AFP via Getty Images

The ongoing baby formula shortage has taken a toll on parents throughout the U.S. Laura Modi, co-founder of formula startup Bobbie, said she’s been “wearing the hat of a mom way more than that of a CEO” in recent weeks.

“It's scary to be a parent right now, with the uncertainty of knowing you can’t find your formula,” Modi told Protocol.

Keep Reading Show less
Nat Rubio-Licht

Nat Rubio-Licht is a Los Angeles-based news writer at Protocol. They graduated from Syracuse University with a degree in newspaper and online journalism in May 2020. Prior to joining the team, they worked at the Los Angeles Business Journal as a technology and aerospace reporter.

Latest Stories
Bulletins