Entertainment

How Amazon built its kid-focused Glow video calling projector

Robots, laser pointers, talking stuffies: Amazon’s devices team went through many iterations while developing its very first interactive projection device.

Illustration of an Amazon Glow device on a blueprint.

The Amazon Glow is the first interactive projection device sold by Amazon, and it could be a stepping stone for the company to use the technology in other areas.

Illustration: Christopher T. Fong/Protocol

Cats love chasing laser pointers. So why not have kids do the same?

When a small team within Amazon’s devices group began exploring the idea of a kid-focused video calling device nearly five years ago, they toyed with a lot of far-out ideas, a laser pointer controlled by an adult calling from afar being one of them. The suggestion was quickly dismissed over eye safety concerns, but it did lead the team down a path exploring projection technologies.

This ultimately resulted in the Amazon Glow, a unique device for kids that combines an 8-inch LCD screen designed for a traditional video calling experience with an additional 19.2" projected touch screen that lets children manipulate objects, incorporate physical play pieces and even “scan” their own toys. The Glow allows kids to read books, play games and create art together with grandparents and other trusted adults participating from afar via mobile devices.

Amazon Glow The Amazon Glow combines an LCD screen with an interactive projector.Image: Amazon.

The Amazon Glow launched as an invite-only early access product last year and became widely available in the U.S. at the end of March. It’s the first interactive projection device sold by Amazon, and it could be a stepping stone for the company to use the technology in other areas, including next-generation smart home devices, or even the enterprise.

It’s also a fascinating story of trial-and-error development, and small, scrappy teams ideating inside of giant corporations. “We knew what we wanted to do, but we didn't really have an idea how to do it,” said Amazon Senior Hardware Engineer Martin Aalund, a founding member of the Glow team.

Kids are terrible camera operators

The Amazon Glow was first announced at the company’s hardware event last fall. Amazon also used the annual showcase to announce countless other new and upgraded devices, but the Glow stood out, both for its technology as well as its intended purpose: Amazon built the Glow to give grandparents and other remote family members and caregivers a chance to be more involved in the lives of children ages 3 to 9. It’s a need that became very obvious during the pandemic — a time that also demonstrated how traditional video calling solutions weren’t up to the task in a more remote world.

“They're designed for conversation between adults, not for playing with children,” said Behrang Assadi, head of Marketing for Amazon Glow. “Kids are terrible cameramen. They tend to not sit still. They get easily distracted by other features of phones, and sometimes they just put it down and walk away.” Aalund agreed. “You get good pictures of the floor and the ceiling and up the child's nose, but very rarely of what you're interested in,” he said.

Sketch of a robot that would follow the child around during video calls. Sketch of a first prototype: a robot that would follow the child around during video calls.Image: Amazon

Aalund’s team wanted to overcome those challenges and allow remote callers to play a more active role, despite potentially being thousands of miles apart. “We wanted the adult to be able to reach into that child's space and have a meaningful interaction [as if] they're in the same room,” Aalund said.

After formulating that end goal in a white paper, the Glow team began with weekly design sprints, toying with a range of ideas and building prototypes to test them out. “We looked at using cranes and robotic arms to grab objects and move them around,” Aalund said. Then, the team had a first of many lightbulb moments: Why build robot arms if you already have a pair of human arms at your disposal? “We got some early feedback [telling us that] we can use the child as a manipulator,” he said.

Still, the team wasn’t ready to give up on the idea of building a robot altogether. The thinking was: If kids weren’t able to stay in one place, then why not follow them around? “I went out and bought a bunch of VEX Robotics kits, and we built up little prototypes and drove them around,” Aalund said. “We created a little mobile robot platform and drove that around with a camera on it.”

From lasers to flashlights to projectors

The team soon discovered, however, that a robot, especially one that followed a child around, came with its own set of issues. “We realized that there's challenges with a remote person navigating [the robot], especially in a real home that may have stairs, obstacles and clutter on the floor,” Aalund said.

Of course, that’s not a new problem to anyone looking to build consumer robots — Amazon included. “We later discovered there was another team working to solve a lot of those problems,” Aalund said. “But that's the Astro story.”

With robots out of the question, the team began looking at alternatives. One idea involved adding a camera that could follow the child around the room with a pan and tilt mechanism while keeping the device itself in place. Add a laser pointer, and you could even play I-Spy with the child. “We thought that was a pretty cool concept, but we were concerned with the laser and eye safety,” Aalund recalled.

Another Glow prototype: a giant projector. Another Glow prototype: a giant projector.Image: Amazon

The team briefly considered a flashlight as an eye-safe alternative, but struggled to get it to work with its existing prototype hardware. “The little handheld mechanism we had wasn't really strong enough to move a flashlight around,” Aalund said. So he decided to fake it. “We can simulate this,” he recalled thinking. “Let's just grab a projector, shining down on the floor where we're playing games, and let's just create a flashlight and move it around the playspace to highlight things.”

The projector was initially just meant to be a quick hack to save a prescheduled demo — but the results were so compelling that the approach quickly became a centerpiece of the Glow hardware. “It set off a lightbulb,” Aalund said. “We could read books, we could actually play games where the remote person has virtual pieces and the child has physical pieces.”

Testing it with kids, getting approval from parents

Developing the Glow included a lot of tests with children and their caregivers, which helped refine the projection technology. At first, the Glow team was thinking about just using a traditional projector. “Initially, it was primarily for that remote person to reach in and augment the child's environment, to move light around, to move virtual pieces,” Aalund said. “[When] we put it in front of some kids, the first thing they did was try [to] pinch and zoom and touch and interact.”

That’s how the team realized it had to make the projector interactive, and effectively turn the projected space into a touch screen. “This was actually fairly far along in our development process,” Aalund said. The team ultimately settled on using an IR camera for hand tracking that could be used in a variety of lighting conditions, including the light emitted by the projector.

A lot of the testing of early Glow prototypes involved make-believe, according to Aalund. “We did a lot of ‘Wizard of Oz,’ man-behind-the-curtain things,” he said. “Rather than actually detecting what the child was doing, we would sit in another room. When we saw the kid touch a spot, we'd click a key and have it change what was displayed.”

A prototype for the Glow device that looks like a penguin. Early on, the Glow team was playing with the idea to make the device look like a stuffed animal. Parents were not amused.Image: Amazon

Testing the device with kids required ingenuity. Getting buy-in from parents was just as challenging, albeit for very different reasons. Early on, the Glow team was playing with the idea of making the device look like a toy itself. “[For] our very first prototype, we actually took an Amazon [Echo] Show and put it inside a stuffed animal,” Aalund said.

There were some thermal issues with that approach, and having a stuffie that doubles as a projector also didn’t seem the best idea for good sleep habits. However, the biggest pushback came from parents. Anything that would look like a toy would have to live in the kid’s room because it clashed with their living room design, some of them argued. Yet others objected to that approach, because they didn’t like the idea of getting judged by their relatives if their kid’s room was messy. In the end, the Glow team settled on a more neutral hardware design.

Next up: Music, coding and … elder care?

While a lot of effort went into building the physical product, the team also worked hard on the Glow’s apps and games, which had to be optimized for both the projector and remote interaction. “We weren't able to just take apps and games that live on existing mobile app stores,” Assadi said. “We ended up building most of it from scratch.”

The result includes games like Go Fish, chess and whack-a-mole, some of which can only be played together with a remote participant. That focus on making the games part of a call was intentional, according to Aalund. “We didn't want this to be another game console or content consumption device,” he said. The Glow also includes access to picture books and chapter books, some of which offer some interactivity.

One of the more unique experiences gives children the ability to scan real-life objects, and then incorporate them into their digital art. Amazon also sells Tangram puzzle pieces that can be used to solve puzzles, and the company has plans to launch additional add-on packs that further allow users to merge physical with digital play, starting with coding bits. Also on the horizon: music games. And eventually, developers may even be able to build their own apps for the platform.

Two children playing a game on the Amazon Glow. The Amazon Glow is optimized for children interacting with remote caregivers, but its technology could also be used for many other purposes.Image: Amazon.

This also opens up the opportunity to use the Glow for other target audiences. “There are some really interesting applications,” Assadi said. “My mind goes to senior assisted living homes, where senior adults want to interact with a caregiver, whether it's local or remote, and play games like Bridge.”

Ultimately, Amazon may even decide to take some of the lessons it learned from developing the Glow and incorporate projection into devices with other form factors. “We've had people within the company ask us about our hardware and software stack for all sorts of different things,” said Aalund. “From office environments to video conferencing, where this could make a great whiteboard, to tutoring. All sorts of different things.”

Amazon did explore using projection mapping for AR-like experiences in the past, and may be getting ready to revive some of those ideas, if recent job listings are any indication. Chances are that those ideas never amount to anything. Or perhaps they’ll take on a life of their own, and turn into something completely different.

That’s perhaps the biggest lesson from the development of the Amazon Glow: Even in a gigantic company like Amazon, there are small, startup-like teams that just throw ideas against the wall and see what sticks. Experiment with DIY robotics kits, wing a few demos, until finally a lightbulb goes off and something compelling emerges.

“We were very scrappy,” Aalund said.

Every day, millions of us press the “order” button on our favorite coffee mobile application. When we arrive at the coffee shop, we expect that our chosen brew will be on the counter a few minutes later. It’s a personalized, seamless experience that we have all come to expect. What we don’t know is what’s happening behind the scenes. The mobile application is sourcing data from a database that stores information about each customer and what their favorite coffee drinks are. It is also leveraging event-streaming data in real time to ensure the ingredients for your personal coffee are in supply at your local store.

Applications like this power our daily lives, and if they can’t access massive amounts of data stored in a database as well as streaming data “in motion” instantaneously, you, and millions of customers, won’t have the in-the-moment experiences we all expect.

Keep Reading Show less
Jennifer Goforth Gregory
Jennifer Goforth Gregory has worked in the B2B technology industry for over 20 years. As a freelance writer she writes for top technology brands, including IBM, HPE, Adobe, AT&T, Verizon, Epson, Oracle, Intel and Square. She specializes in a wide range of technology, such as AI, IoT, cloud, cybersecurity, and CX. Jennifer also wrote a bestselling book The Freelance Content Marketing Writer to help other writers launch a high earning freelance business.
Policy

How the internet got privatized and how the government could fix it

Author Ben Tarnoff discusses municipal broadband, Web3 and why closing the “digital divide” isn’t enough.

The Biden administration’s Internet for All initiative, which kicked off in May, will roll out grant programs to expand and improve broadband infrastructure, teach digital skills and improve internet access for “everyone in America by the end of the decade.”

Decisions about who is eligible for these grants will be made based on the Federal Communications Commission’s broken, outdated and incorrect broadband maps — maps the FCC plans to update only after funding has been allocated. Inaccurate broadband maps are just one of many barriers to getting everyone in the country successfully online. Internet service providers that use government funds to connect rural and low-income areas have historically provided those regions with slow speeds and poor service, forcing community residents to find reliable internet outside of their homes.

Keep Reading Show less
Aditi Mukund
Aditi Mukund is Protocol’s Data Analyst. Prior to joining Protocol, she was an analyst at The Daily Beast and NPR where she wrangled data into actionable insights for editorial, audience, commerce, subscription, and product teams. She holds a B.S in Cognitive Science, Human Computer Interaction from The University of California, San Diego.
Fintech

How I decided to exit my startup’s original business

Bluevine got its start in factoring invoices for small businesses. CEO Eyal Lifshitz explains why it dropped that business in favor of “end-to-end banking.”

"[I]t was a realization that we can't be successful at both at the same time: You've got to choose."

Photo: Bluevine

Click banner image for more How I decided series

Bluevine got its start in fintech by offering a modern version of invoice factoring, the centuries-old practice where businesses sell off their accounts receivable for up-front cash. It’s raised $767 million in venture capital since its founding in 2013 by serving small businesses. But along the way, it realized it was better to focus on the checking accounts and lines of credit it provided customers than its original product. It now manages some $500 million in checking-account deposits.

Keep Reading Show less
Ryan Deffenbaugh
Ryan Deffenbaugh is a reporter at Protocol focused on fintech. Before joining Protocol, he reported on New York's technology industry for Crain's New York Business. He is based in New York and can be reached at rdeffenbaugh@protocol.com.
Enterprise

The Roe decision could change how advertisers use location data

Over the years, the digital ad industry has been resistant to restricting use of location data. But that may be changing.

Over the years, the digital ad industry has been resistant to restrictions on the use of location data. But that may be changing.

Illustration: Christopher T. Fong/Protocol

When the Supreme Court overturned Roe v. Wade on Friday, the likelihood for location data to be used against people suddenly shifted from a mostly hypothetical scenario to a realistic threat. Although location data has a variety of purposes — from helping municipalities assess how people move around cities to giving reliable driving directions — it’s the voracious appetite of digital advertisers for location information that has fueled the creation and growth of a sector selling data showing who visited specific points on the map, when, what places they came from and where they went afterwards.

Over the years, the digital ad industry has been resistant to restrictions on the use of location data. But that may be changing. The overturning of Roe not only puts the wide availability of location data for advertising in the spotlight, it could serve as a turning point compelling the digital ad industry to take action to limit data associated with sensitive places before the government does.

Keep Reading Show less
Kate Kaye

Kate Kaye is an award-winning multimedia reporter digging deep and telling print, digital and audio stories. She covers AI and data for Protocol. Her reporting on AI and tech ethics issues has been published in OneZero, Fast Company, MIT Technology Review, CityLab, Ad Age and Digiday and heard on NPR. Kate is the creator of RedTailMedia.org and is the author of "Campaign '08: A Turning Point for Digital Media," a book about how the 2008 presidential campaigns used digital media and data.

Latest Stories
Bulletins