How Amazon built its kid-focused Glow video calling projector

Robots, laser pointers, talking stuffies: Amazon’s devices team went through many iterations while developing its very first interactive projection device.

Illustration of an Amazon Glow device on a blueprint.

The Amazon Glow is the first interactive projection device sold by Amazon, and it could be a stepping stone for the company to use the technology in other areas.

Illustration: Christopher T. Fong/Protocol

Cats love chasing laser pointers. So why not have kids do the same?

When a small team within Amazon’s devices group began exploring the idea of a kid-focused video calling device nearly five years ago, they toyed with a lot of far-out ideas, a laser pointer controlled by an adult calling from afar being one of them. The suggestion was quickly dismissed over eye safety concerns, but it did lead the team down a path exploring projection technologies.

This ultimately resulted in the Amazon Glow, a unique device for kids that combines an 8-inch LCD screen designed for a traditional video calling experience with an additional 19.2" projected touch screen that lets children manipulate objects, incorporate physical play pieces and even “scan” their own toys. The Glow allows kids to read books, play games and create art together with grandparents and other trusted adults participating from afar via mobile devices.

Amazon GlowThe Amazon Glow combines an LCD screen with an interactive projector.Image: Amazon.

The Amazon Glow launched as an invite-only early access product last year and became widely available in the U.S. at the end of March. It’s the first interactive projection device sold by Amazon, and it could be a stepping stone for the company to use the technology in other areas, including next-generation smart home devices, or even the enterprise.

It’s also a fascinating story of trial-and-error development, and small, scrappy teams ideating inside of giant corporations. “We knew what we wanted to do, but we didn't really have an idea how to do it,” said Amazon Senior Hardware Engineer Martin Aalund, a founding member of the Glow team.

Kids are terrible camera operators

The Amazon Glow was first announced at the company’s hardware event last fall. Amazon also used the annual showcase to announce countless other new and upgraded devices, but the Glow stood out, both for its technology as well as its intended purpose: Amazon built the Glow to give grandparents and other remote family members and caregivers a chance to be more involved in the lives of children ages 3 to 9. It’s a need that became very obvious during the pandemic — a time that also demonstrated how traditional video calling solutions weren’t up to the task in a more remote world.

“They're designed for conversation between adults, not for playing with children,” said Behrang Assadi, head of Marketing for Amazon Glow. “Kids are terrible cameramen. They tend to not sit still. They get easily distracted by other features of phones, and sometimes they just put it down and walk away.” Aalund agreed. “You get good pictures of the floor and the ceiling and up the child's nose, but very rarely of what you're interested in,” he said.

Sketch of a robot that would follow the child around during video calls.Sketch of a first prototype: a robot that would follow the child around during video calls.Image: Amazon

Aalund’s team wanted to overcome those challenges and allow remote callers to play a more active role, despite potentially being thousands of miles apart. “We wanted the adult to be able to reach into that child's space and have a meaningful interaction [as if] they're in the same room,” Aalund said.

After formulating that end goal in a white paper, the Glow team began with weekly design sprints, toying with a range of ideas and building prototypes to test them out. “We looked at using cranes and robotic arms to grab objects and move them around,” Aalund said. Then, the team had a first of many lightbulb moments: Why build robot arms if you already have a pair of human arms at your disposal? “We got some early feedback [telling us that] we can use the child as a manipulator,” he said.

Still, the team wasn’t ready to give up on the idea of building a robot altogether. The thinking was: If kids weren’t able to stay in one place, then why not follow them around? “I went out and bought a bunch of VEX Robotics kits, and we built up little prototypes and drove them around,” Aalund said. “We created a little mobile robot platform and drove that around with a camera on it.”

From lasers to flashlights to projectors

The team soon discovered, however, that a robot, especially one that followed a child around, came with its own set of issues. “We realized that there's challenges with a remote person navigating [the robot], especially in a real home that may have stairs, obstacles and clutter on the floor,” Aalund said.

Of course, that’s not a new problem to anyone looking to build consumer robots — Amazon included. “We later discovered there was another team working to solve a lot of those problems,” Aalund said. “But that's the Astro story.”

With robots out of the question, the team began looking at alternatives. One idea involved adding a camera that could follow the child around the room with a pan and tilt mechanism while keeping the device itself in place. Add a laser pointer, and you could even play I-Spy with the child. “We thought that was a pretty cool concept, but we were concerned with the laser and eye safety,” Aalund recalled.

Another Glow prototype: a giant projector.Another Glow prototype: a giant projector.Image: Amazon

The team briefly considered a flashlight as an eye-safe alternative, but struggled to get it to work with its existing prototype hardware. “The little handheld mechanism we had wasn't really strong enough to move a flashlight around,” Aalund said. So he decided to fake it. “We can simulate this,” he recalled thinking. “Let's just grab a projector, shining down on the floor where we're playing games, and let's just create a flashlight and move it around the playspace to highlight things.”

The projector was initially just meant to be a quick hack to save a prescheduled demo — but the results were so compelling that the approach quickly became a centerpiece of the Glow hardware. “It set off a lightbulb,” Aalund said. “We could read books, we could actually play games where the remote person has virtual pieces and the child has physical pieces.”

Testing it with kids, getting approval from parents

Developing the Glow included a lot of tests with children and their caregivers, which helped refine the projection technology. At first, the Glow team was thinking about just using a traditional projector. “Initially, it was primarily for that remote person to reach in and augment the child's environment, to move light around, to move virtual pieces,” Aalund said. “[When] we put it in front of some kids, the first thing they did was try [to] pinch and zoom and touch and interact.”

That’s how the team realized it had to make the projector interactive, and effectively turn the projected space into a touch screen. “This was actually fairly far along in our development process,” Aalund said. The team ultimately settled on using an IR camera for hand tracking that could be used in a variety of lighting conditions, including the light emitted by the projector.

A lot of the testing of early Glow prototypes involved make-believe, according to Aalund. “We did a lot of ‘Wizard of Oz,’ man-behind-the-curtain things,” he said. “Rather than actually detecting what the child was doing, we would sit in another room. When we saw the kid touch a spot, we'd click a key and have it change what was displayed.”

A prototype for the Glow device that looks like a penguin.Early on, the Glow team was playing with the idea to make the device look like a stuffed animal. Parents were not amused.Image: Amazon

Testing the device with kids required ingenuity. Getting buy-in from parents was just as challenging, albeit for very different reasons. Early on, the Glow team was playing with the idea of making the device look like a toy itself. “[For] our very first prototype, we actually took an Amazon [Echo] Show and put it inside a stuffed animal,” Aalund said.

There were some thermal issues with that approach, and having a stuffie that doubles as a projector also didn’t seem the best idea for good sleep habits. However, the biggest pushback came from parents. Anything that would look like a toy would have to live in the kid’s room because it clashed with their living room design, some of them argued. Yet others objected to that approach, because they didn’t like the idea of getting judged by their relatives if their kid’s room was messy. In the end, the Glow team settled on a more neutral hardware design.

Next up: Music, coding and … elder care?

While a lot of effort went into building the physical product, the team also worked hard on the Glow’s apps and games, which had to be optimized for both the projector and remote interaction. “We weren't able to just take apps and games that live on existing mobile app stores,” Assadi said. “We ended up building most of it from scratch.”

The result includes games like Go Fish, chess and whack-a-mole, some of which can only be played together with a remote participant. That focus on making the games part of a call was intentional, according to Aalund. “We didn't want this to be another game console or content consumption device,” he said. The Glow also includes access to picture books and chapter books, some of which offer some interactivity.

One of the more unique experiences gives children the ability to scan real-life objects, and then incorporate them into their digital art. Amazon also sells Tangram puzzle pieces that can be used to solve puzzles, and the company has plans to launch additional add-on packs that further allow users to merge physical with digital play, starting with coding bits. Also on the horizon: music games. And eventually, developers may even be able to build their own apps for the platform.

Two children playing a game on the Amazon Glow.The Amazon Glow is optimized for children interacting with remote caregivers, but its technology could also be used for many other purposes.Image: Amazon.

This also opens up the opportunity to use the Glow for other target audiences. “There are some really interesting applications,” Assadi said. “My mind goes to senior assisted living homes, where senior adults want to interact with a caregiver, whether it's local or remote, and play games like Bridge.”

Ultimately, Amazon may even decide to take some of the lessons it learned from developing the Glow and incorporate projection into devices with other form factors. “We've had people within the company ask us about our hardware and software stack for all sorts of different things,” said Aalund. “From office environments to video conferencing, where this could make a great whiteboard, to tutoring. All sorts of different things.”

Amazon did explore using projection mapping for AR-like experiences in the past, and may be getting ready to revive some of those ideas, if recent job listings are any indication. Chances are that those ideas never amount to anything. Or perhaps they’ll take on a life of their own, and turn into something completely different.

That’s perhaps the biggest lesson from the development of the Amazon Glow: Even in a gigantic company like Amazon, there are small, startup-like teams that just throw ideas against the wall and see what sticks. Experiment with DIY robotics kits, wing a few demos, until finally a lightbulb goes off and something compelling emerges.

“We were very scrappy,” Aalund said.


Judge Zia Faruqui is trying to teach you crypto, one ‘SNL’ reference at a time

His decisions on major cryptocurrency cases have quoted "The Big Lebowski," "SNL," and "Dr. Strangelove." That’s because he wants you — yes, you — to read them.

The ways Zia Faruqui (right) has weighed on cases that have come before him can give lawyers clues as to what legal frameworks will pass muster.

Photo: Carolyn Van Houten/The Washington Post via Getty Images

“Cryptocurrency and related software analytics tools are ‘The wave of the future, Dude. One hundred percent electronic.’”

That’s not a quote from "The Big Lebowski" — at least, not directly. It’s a quote from a Washington, D.C., district court memorandum opinion on the role cryptocurrency analytics tools can play in government investigations. The author is Magistrate Judge Zia Faruqui.

Keep ReadingShow less
Veronica Irwin

Veronica Irwin (@vronirwin) is a San Francisco-based reporter at Protocol covering fintech. Previously she was at the San Francisco Examiner, covering tech from a hyper-local angle. Before that, her byline was featured in SF Weekly, The Nation, Techworker, Ms. Magazine and The Frisc.

The financial technology transformation is driving competition, creating consumer choice, and shaping the future of finance. Hear from seven fintech leaders who are reshaping the future of finance, and join the inaugural Financial Technology Association Fintech Summit to learn more.

Keep ReadingShow less
The Financial Technology Association (FTA) represents industry leaders shaping the future of finance. We champion the power of technology-centered financial services and advocate for the modernization of financial regulation to support inclusion and responsible innovation.

AWS CEO: The cloud isn’t just about technology

As AWS preps for its annual re:Invent conference, Adam Selipsky talks product strategy, support for hybrid environments, and the value of the cloud in uncertain economic times.

Photo: Noah Berger/Getty Images for Amazon Web Services

AWS is gearing up for re:Invent, its annual cloud computing conference where announcements this year are expected to focus on its end-to-end data strategy and delivering new industry-specific services.

It will be the second re:Invent with CEO Adam Selipsky as leader of the industry’s largest cloud provider after his return last year to AWS from data visualization company Tableau Software.

Keep ReadingShow less
Donna Goodison

Donna Goodison (@dgoodison) is Protocol's senior reporter focusing on enterprise infrastructure technology, from the 'Big 3' cloud computing providers to data centers. She previously covered the public cloud at CRN after 15 years as a business reporter for the Boston Herald. Based in Massachusetts, she also has worked as a Boston Globe freelancer, business reporter at the Boston Business Journal and real estate reporter at Banker & Tradesman after toiling at weekly newspapers.

Image: Protocol

We launched Protocol in February 2020 to cover the evolving power center of tech. It is with deep sadness that just under three years later, we are winding down the publication.

As of today, we will not publish any more stories. All of our newsletters, apart from our flagship, Source Code, will no longer be sent. Source Code will be published and sent for the next few weeks, but it will also close down in December.

Keep ReadingShow less
Bennett Richardson

Bennett Richardson ( @bennettrich) is the president of Protocol. Prior to joining Protocol in 2019, Bennett was executive director of global strategic partnerships at POLITICO, where he led strategic growth efforts including POLITICO's European expansion in Brussels and POLITICO's creative agency POLITICO Focus during his six years with the company. Prior to POLITICO, Bennett was co-founder and CMO of Hinge, the mobile dating company recently acquired by Match Group. Bennett began his career in digital and social brand marketing working with major brands across tech, energy, and health care at leading marketing and communications agencies including Edelman and GMMB. Bennett is originally from Portland, Maine, and received his bachelor's degree from Colgate University.


Why large enterprises struggle to find suitable platforms for MLops

As companies expand their use of AI beyond running just a few machine learning models, and as larger enterprises go from deploying hundreds of models to thousands and even millions of models, ML practitioners say that they have yet to find what they need from prepackaged MLops systems.

As companies expand their use of AI beyond running just a few machine learning models, ML practitioners say that they have yet to find what they need from prepackaged MLops systems.

Photo: artpartner-images via Getty Images

On any given day, Lily AI runs hundreds of machine learning models using computer vision and natural language processing that are customized for its retail and ecommerce clients to make website product recommendations, forecast demand, and plan merchandising. But this spring when the company was in the market for a machine learning operations platform to manage its expanding model roster, it wasn’t easy to find a suitable off-the-shelf system that could handle such a large number of models in deployment while also meeting other criteria.

Some MLops platforms are not well-suited for maintaining even more than 10 machine learning models when it comes to keeping track of data, navigating their user interfaces, or reporting capabilities, Matthew Nokleby, machine learning manager for Lily AI’s product intelligence team, told Protocol earlier this year. “The duct tape starts to show,” he said.

Keep ReadingShow less
Kate Kaye

Kate Kaye is an award-winning multimedia reporter digging deep and telling print, digital and audio stories. She covers AI and data for Protocol. Her reporting on AI and tech ethics issues has been published in OneZero, Fast Company, MIT Technology Review, CityLab, Ad Age and Digiday and heard on NPR. Kate is the creator of RedTailMedia.org and is the author of "Campaign '08: A Turning Point for Digital Media," a book about how the 2008 presidential campaigns used digital media and data.

Latest Stories