Power

Inside Amazon’s quest to build more lively gadgets, starting with making the Echo Show 10 move

The company's latest smart device features a rotating display, and Amazon is allowing developers to tap into the feature to make their Alexa skills multimodal.

Amazon Echo Show 10 with motion

Amazon's Echo Show 10, performing the "mild expressive shake" motion

Courtesy of Amazon

Get ready for Alexa to wave at you.

Well, not literally. Amazon hasn't built a humanoid smart home robot just yet, but the company's latest Echo Show 10 smart display is a small step in that direction. Equipped with a display that can rotate 360 degrees around its speaker base, the Echo Show 10 is the first device of its kind that can physically move. When you say "Alexa, goodbye," it not only responds with audible and visual feedback, but also performs a kind of sweeping motion with its display, resembling a slow farewell wave.

Originally conceived as a way to keep the display in view while you move around the kitchen, Amazon's hardware team has turned this motion feature into a novel way to interact with the device. For instance, if you profess your love to Alexa, the Echo Show 10 does a little happy dance, with the display bobbing back and forth. "It seemed appropriate for Alexa to look cheerful when you say you love her," Amazon's Alexa devices and developer technologies VP, Nedim Fresko, said.

For Fresko and his team, adding motion to the Echo Show 10 was just the latest step toward a more responsive and multimodal future of ambient computing. The company also opened the feature up to developers, and began to choreograph a set of predefined motions to make Alexa's skills more expressive. In an exclusive conversation with Protocol, Fresko explained why Amazon is betting on motion, how his team conceives new features for smart home devices and why Amazon executives regularly spend hours reading customer reviews.

Working backward from an idea

Out of all the things Amazon could have added to its smart displays, giving developers the ability to make it wave and shake may not have been the most obvious choice. Fresko said that it all started with an idea for solving a common customer problem with regular smart displays. "Every year, we have new ideas for working-backwards documents, as we call them, where we consider the customer need, and then we try to look at which technologies can achieve that customer need," he explained.

"Motion was described in exactly the same way a couple of years ago, where we described how it would behave before any hardware or software was built," Fresko said. "We turned that around until we really loved it. Then we built the technology."

That early idea was all about making screens a key part of the ambient computing experience. "Sometimes, when you're multitasking in the kitchen, or walking around the home, you have to face a choice: You're either facing the screen or you have to make do with voice," Fresko said. "So we thought: What if we could build a system where the screen always faces you? No matter where you are, you'll find the screen facing you."

A screen that moves with you, aided by the built-in camera and its ability to track your movement across the room, seemed like a good solution for cooking in the kitchen. It also lent itself naturally to Amazon's push into video chat. "Person-to-person communication, where you're always in the frame no matter where you are, was kind of consistent with the overall vision," Fresko said.

Opening up the feature to developers

Once Amazon's hardware team began working on the technology, it had to keep it under the lid for a while. "It's hard to go out there and do a survey about motion," he said. "That kind of thing is an internal idea." However, the company did reach out to a few trusted developers before it officially announced the Echo Show 10 with motion last October. "If we find that the experience is useful for consumers, we automatically assume that developers will also find it useful for similar reasons," Fresko said.

Some of the third-party developers to make use of motion include Comedy Central, Sony and Universal Games. One of the few smaller companies that got early access is Volley, a San Francisco-based startup that builds casual games for smart speakers. "It's fun," Volley CEO Max Child said. "It's like a cute little robot from a movie."

Volley has added the Echo 10's ability to have the display follow a user to its Song Quiz Alexa skill, and Child is already thinking about additional applications that could add to the company's multiplayer games. "Secret information is interesting, and also just cues around whose turn it is, or who should be engaging, that kind of thing," he said.

In addition to giving developers access to the basic motion that keeps the display in view, Amazon has also begun to release a few choreographed motions that can be used to make skills more dynamic. "We added a few preprogrammed choreos to express emotion, and we also added something called 'entity sensing' that allows you to act as input yourself," Fresko said. "As you move, the UI could react with updates based on your motion."

"We'll add more of these personality and reaction features," Child said. "There's a lot of opportunity to make these devices more friendly and more endearing."

On Wednesday, Amazon released three new such "choreos," including one aptly called "MildExpressiveShakes." To develop these and other choreographed motions, Amazon's designers spitballed a bunch of different ideas on what Fresko described as a "mild to wild" scale. "They basically disregarded all constraints and tried to see what would feel delightful, what would evoke emotion," he said. "Then they thought about what experiences they could attach those motions to, and eventually narrowed the focus to a few that we felt were in good taste, delightful, useful."

As part of that process, they also had to throw out a bunch of different motions that were simply too exhausting or dizzying. "It's very easy to go overboard with these things," Fresko said. "Sometimes, you just have to restrain yourself."

Getting it wrong to ultimately get it right

It's too early to say whether consumers are willing to pay a premium for motion. The Echo Show 10 became available for $250 at the end of February; early reviews on Amazon.com are generally positive, but some reviewers have taken issue with the feature. "The first thing I turned off was motion," one customer noted, who complained about a lack of accuracy when the display follows you. A few others had similar gripes, with another consumer expressing hope that things could improve: "The follow-mode feels more gimmicky than useful right now but that could change as they update."

"Once we launch, our customers publicly grade us," Fresko said. "Very publicly grade us, and tell us exactly what they like and don't like." Amazon keeps close tabs on those comments, and has developed automated ways to pay attention to thousands of reviews. But Fresko is also going through them manually. "In the few months that are after a device's launch, I read every single review, without exception," he said. "They're very interesting. We're very grateful to our customers for being frank about that feedback."

One reason these reviews are so valuable is that ambient computing is still so novel. The first Echo smart speaker was released in 2014, and the company's first smart display only became available in 2017. Both products ultimately laid the foundation for entire product categories, and turned Amazon into the market leader in the smart speaker space. But they began as unproven bets, just as the Echo Show 10's motion feature.

"We keep on placing bets in areas where we have a really good theory that customers will find it useful," Fresko said. "But there's only one way to find out, which is to build it."

Some bets ultimately don't pan out, as Fresko knows all too well. Before working on Alexa devices, he was director of engineering for Amazon's Fire phone. Just like the Echo Show 10, that device had a very unique and interesting feature that set it apart from the competition: a 3D display that gave consumers a perception of depth by tracking their viewing angle. The device nonetheless failed miserably, and Amazon pulled the plug on the project after just one year.

"You imagine customer utility, you write about it, you convince yourself, and then you realize the only way to find out if it's going to work is to actually build it," Fresko said. "Sometimes there is awesome customer interest, and sometimes there isn't. But there's no way to find out unless you try. So I think of the phone as exactly that. There was a very good theory behind its utility. It didn't find the reception we expected, and therefore we moved to the next thing. And guess what? The next thing was incredibly successful."

Fintech

Judge Zia Faruqui is trying to teach you crypto, one ‘SNL’ reference at a time

His decisions on major cryptocurrency cases have quoted "The Big Lebowski," "SNL," and "Dr. Strangelove." That’s because he wants you — yes, you — to read them.

The ways Zia Faruqui (right) has weighed on cases that have come before him can give lawyers clues as to what legal frameworks will pass muster.

Photo: Carolyn Van Houten/The Washington Post via Getty Images

“Cryptocurrency and related software analytics tools are ‘The wave of the future, Dude. One hundred percent electronic.’”

That’s not a quote from "The Big Lebowski" — at least, not directly. It’s a quote from a Washington, D.C., district court memorandum opinion on the role cryptocurrency analytics tools can play in government investigations. The author is Magistrate Judge Zia Faruqui.

Keep Reading Show less
Veronica Irwin

Veronica Irwin (@vronirwin) is a San Francisco-based reporter at Protocol covering fintech. Previously she was at the San Francisco Examiner, covering tech from a hyper-local angle. Before that, her byline was featured in SF Weekly, The Nation, Techworker, Ms. Magazine and The Frisc.

The financial technology transformation is driving competition, creating consumer choice, and shaping the future of finance. Hear from seven fintech leaders who are reshaping the future of finance, and join the inaugural Financial Technology Association Fintech Summit to learn more.

Keep Reading Show less
FTA
The Financial Technology Association (FTA) represents industry leaders shaping the future of finance. We champion the power of technology-centered financial services and advocate for the modernization of financial regulation to support inclusion and responsible innovation.
Enterprise

AWS CEO: The cloud isn’t just about technology

As AWS preps for its annual re:Invent conference, Adam Selipsky talks product strategy, support for hybrid environments, and the value of the cloud in uncertain economic times.

Photo: Noah Berger/Getty Images for Amazon Web Services

AWS is gearing up for re:Invent, its annual cloud computing conference where announcements this year are expected to focus on its end-to-end data strategy and delivering new industry-specific services.

It will be the second re:Invent with CEO Adam Selipsky as leader of the industry’s largest cloud provider after his return last year to AWS from data visualization company Tableau Software.

Keep Reading Show less
Donna Goodison

Donna Goodison (@dgoodison) is Protocol's senior reporter focusing on enterprise infrastructure technology, from the 'Big 3' cloud computing providers to data centers. She previously covered the public cloud at CRN after 15 years as a business reporter for the Boston Herald. Based in Massachusetts, she also has worked as a Boston Globe freelancer, business reporter at the Boston Business Journal and real estate reporter at Banker & Tradesman after toiling at weekly newspapers.

Image: Protocol

We launched Protocol in February 2020 to cover the evolving power center of tech. It is with deep sadness that just under three years later, we are winding down the publication.

As of today, we will not publish any more stories. All of our newsletters, apart from our flagship, Source Code, will no longer be sent. Source Code will be published and sent for the next few weeks, but it will also close down in December.

Keep Reading Show less
Bennett Richardson

Bennett Richardson ( @bennettrich) is the president of Protocol. Prior to joining Protocol in 2019, Bennett was executive director of global strategic partnerships at POLITICO, where he led strategic growth efforts including POLITICO's European expansion in Brussels and POLITICO's creative agency POLITICO Focus during his six years with the company. Prior to POLITICO, Bennett was co-founder and CMO of Hinge, the mobile dating company recently acquired by Match Group. Bennett began his career in digital and social brand marketing working with major brands across tech, energy, and health care at leading marketing and communications agencies including Edelman and GMMB. Bennett is originally from Portland, Maine, and received his bachelor's degree from Colgate University.

Enterprise

Why large enterprises struggle to find suitable platforms for MLops

As companies expand their use of AI beyond running just a few machine learning models, and as larger enterprises go from deploying hundreds of models to thousands and even millions of models, ML practitioners say that they have yet to find what they need from prepackaged MLops systems.

As companies expand their use of AI beyond running just a few machine learning models, ML practitioners say that they have yet to find what they need from prepackaged MLops systems.

Photo: artpartner-images via Getty Images

On any given day, Lily AI runs hundreds of machine learning models using computer vision and natural language processing that are customized for its retail and ecommerce clients to make website product recommendations, forecast demand, and plan merchandising. But this spring when the company was in the market for a machine learning operations platform to manage its expanding model roster, it wasn’t easy to find a suitable off-the-shelf system that could handle such a large number of models in deployment while also meeting other criteria.

Some MLops platforms are not well-suited for maintaining even more than 10 machine learning models when it comes to keeping track of data, navigating their user interfaces, or reporting capabilities, Matthew Nokleby, machine learning manager for Lily AI’s product intelligence team, told Protocol earlier this year. “The duct tape starts to show,” he said.

Keep Reading Show less
Kate Kaye

Kate Kaye is an award-winning multimedia reporter digging deep and telling print, digital and audio stories. She covers AI and data for Protocol. Her reporting on AI and tech ethics issues has been published in OneZero, Fast Company, MIT Technology Review, CityLab, Ad Age and Digiday and heard on NPR. Kate is the creator of RedTailMedia.org and is the author of "Campaign '08: A Turning Point for Digital Media," a book about how the 2008 presidential campaigns used digital media and data.

Latest Stories
Bulletins