yesKevin McAllisterNone
×

Get access to Protocol

I’ve already subscribed

Will be used in accordance with our Privacy Policy

People

Amazon Halo’s principal medical officer talks privacy, shortcomings with competitors, and why it’s now or never for the device

The company's new health-monitoring device has received plenty of criticism since it was announced. Dr. Maulik Majmudar hopes science can win over the skeptics.

Amazon Halo’s principal medical officer talks privacy, shortcomings with competitors, and why it’s now or never for the device

Halo, Amazon's entrance into health and wellness wearables, is ramping up to general availability.

Image: Amazon

Amazon Halo's Principal Medical Officer Dr. Maulik Majmudar says he hears your concerns about the company's newly unveiled Halo device — but not in a creepy way.

Halo, Amazon's entrance into the health and wellness wearables space, launched on Aug. 27 to the company's usual fanfare. But it immediately faced skepticism, centered primarily around the product's new Tone feature, which provides insights about how users sound to other people throughout the day by analyzing audio samples from their conversations.

Currently, Halo is only available through an early-access program. But as the company ramps up to general availability, Majmudar says his focus has been to "add clinical credibility to the features within Halo, making sure that the product experience is rooted in science and then validating all those features to make sure we can earn and maintain customer trust."

In an interview with Protocol, Majmudar described how Amazon has tried to prioritize privacy and avoid algorithmic bias in Halo's design, explained why the device needs to be a success from the get-go, and raised doubts about sophisticated cardiology features in competing devices.

This interview has been edited and condensed for clarity.

Wearables is a space that's getting crowded. Why did you choose now to launch the Halo?

Whenever we think about entering a new category at Amazon, there are two key questions we always ask ourselves. First and foremost: Does this matter to customers? Is there an unmet customer need here? And the second is: Can we as Amazon meaningfully differentiate and add innovation and value of the customer experience in the space?

In the case of health and wellness, it was very clear to us that there's an obvious customer need. If you look at the proliferation of digital health devices and services in the last decade or so, unfortunately there hasn't been an equally meaningful change in the health of the population. One data point that I always refer to is the prevalence of obesity. The latest data from the CDC from February 2020 suggested that the prevalence of obesity in the U.S. is up to 40% now, and it's been going up steadily for over decade now.

The second part is my more critical one: Why Amazon, and why now? We really believe that we could leverage our expertise and experience in artificial intelligence, including machine learning and computer vision, to invent new metrics and new features for our customers to give them a much more holistic view of health than traditionally is done. The traditional focus has always been around physical health: activity, nutrition, sleep. Can we go beyond that to the social and emotional aspects of health? There are lots of products and services focusing on information, but very few of them actually get to the next step of insights and then actions. And one of the core components of Halo is giving customers tools to actually help them improve their own health and wellness.

Is the distribution of the device intended to be consumer-led, or are you exploring other channels like hospitals and clinics to have this be used as a medical tool in those environments?

Given [this is] Amazon … I think it makes a lot of sense to go direct-to-consumer. The purpose of the product and the features is to improve individual health and wellness. I strongly believe that the product's design and the features have incredible value for our enterprise customers, whether it be employers who are prioritizing the employee health and wellness, whether it be hospitals or insurance companies [that] want to optimize their members' health and wellness. I believe that the features and the product and the service really tie to that, and it would help in all those different corporate channels.

But I want to be clear on one point: Halo is not a medical device. It has a lot of uses in those environments, but at the end of the day, the device is not a medical device. Even these really valid, accurate features like Body and Tone are incredibly helpful for improving customers' lives, but they're not diagnostic.

There's incredible value in those enterprise customers; we will just see as we launch and go to [general availability] how we can have dialogues with those types of customers to make sure we can deploy Halo within those enterprises.

Analyzing how you sound to others with the Tone feature has been a focus of attention so far. From a medical perspective, do you see the Tone data having medical benefits beyond being an interesting self-evaluation?

Absolutely. I strongly believe that your tone of voice and the perception of that tone of voice to others has a lot of implications on two specific things: social relationships and your interpersonal communication. There's a lot of really solid science on the consequences of poor social interactions and poor social relationships on not just emotional mental health, [but] actually physical health. There's a really fantastic meta-analysis by Julianne Holt-Lunstad out of Brigham Young University that looked at 25 years of research and showed that poor social relationships actually lead to premature death and chronic diseases.

The question is, how do you evolve over time from taking the Tone feature as a perception of tone of voice into getting more clinical benefit in those environments? And I think we're going to continue to do that over time.

Dr. Maulik Majmudar joined Amazon in 2018 following his tenure as the associate director of Massachusetts General Hospital's Healthcare Transformation Lab.Photo: Amazon

With the understanding that Tone is evolving, can you walk me through the data that was used to train the insights?

Tone leverages machine learning and was trained on thousands of hours of beta trials data. Between the band — where there are microphones and sensors —and then the app, we use machine learning models: audio algorithms that remove the background noise, process the speech, and then analyze the acoustic characteristics of that tone of voice. So we are analyzing the tempo, intensity, pitch and rhythm to then help customers understand how they sound to others. You can imagine that the whole chain of data is incredibly sensitive, and the type of information is sensitive, so there are a number of privacy controls and specific machine learning models we built to do this. First and foremost, the audio data never goes to the cloud. It is all processed locally on the phone and automatically deleted. So no one, including me, can hear the actual audio ever.

The output itself is [made up of] three things. We output something called positivity and [another called] energy. So positivity is how happy or sad you sound, and then energy is how excited or tired you sound. The third thing is we actually provide descriptors or words that describe how you sound to others. It can say "Maulik sounded confident," or compassionate or irritated. And we do this throughout the day. There are a couple of modes for tone, but the traditional mode is that it passively and intermittently collects samples of speech throughout the day, about 30 to 60 seconds of speech every 10 minutes or so.

You get this entire day's view of how you communicated and how you sound to others, and then we highlight notable moments. And the privacy controls are in place to make it super easy to use for customers and also to drive transparency and trust in that feature.

One concern, specifically among women I've spoken to, is the idea of tone policing in a workplace environment. Were there controls put in effect to handle the differences in the way different people communicate?

It's a very relevant question in this day and age, where algorithmic bias is a hot topic, and there have been studies coming out in clinical literature about concerns around that. This was top of mind for our machine learning scientists and the product team, privacy team, everybody else. When we look at thousands of hours of data collected that we talked about, we really ensured that [when] we had a blinded panel of annotators annotate these samples of speech and label them, we didn't introduce any bias as part of that development and validation pathway.

It was very important to us that we analyzed our own models to make sure that for gender diversity, age or ethnicity, there's not any bias and introduced by the algorithms to say, "Hey, women always sound mad or angry." That is obviously a very fair concern.

But as you know, with all machine learning models, there's always room for improvement over time. And the goal is to get this out of early access and hopefully to general availability, and then continue to learn over time and keep iterating. But at least as far as all the testing has been done, we made sure we weren't introducing bias.

Getting back to privacy and security, you mentioned the data is all local, but I want to focus on third-party security specifically. Halo has third-party integration opportunities, how did privacy factor into that development?

We announced a few third-party partnerships through what's called account linking. The fundamental principle behind all of this was [twofold]. One was [that] privacy was foundational. Two was customer transparency. We wanted to provide customers full control of not just who they link accounts with, but also what data is actually being shared. If you go to the settings page of the app today, [we have] the opt in feature; you have to link the accounts voluntarily. We surface to customers specifically which individual data points will be shared, and we don't share all of them by design. For each partner that we announced, we want to figure out what metrics are necessary for them to deliver the right customer experience and we only share those data points.

[For] Cerner, it was only body fat percentage, nothing else. For WW, it was [the] activity score. We wanted to be very specific and deliberate about what we share and why we're sharing it and explain that to our customers. Beyond that, customers can request to delete the data at any point in time. So you sign up voluntarily, it's an opt-in feature, you know exactly what you're sharing at the time you sign up, you read the policies and procedures and accept them, and then you can request to delete the data later on. And, of course, [you can] delete the account link as well.

Unveiled at the Apple event a few weeks ago, the new Apple Watch has wellness-focused features: increased ECG functionality, a blood oximeter. Was there any feature envy on your part? And should we expect a future iteration of Halo to have something similar?

As a cardiologist, you're always dreaming up metrics, and you're like, I wish we could measure this all the time. You're trying to innovate about what could be interesting. The challenge is, when you get out of the clinical environment and apply the same metrics that doctors use in the hospital setting to the general population, you open up a Pandora's box. This idea of — to get nerdy for a second here — prevalence of disease and the false positive rate is happening with COVID, so people are somewhat familiar with this now, but it's what happens in the medical world all the time.

Just [last week], a paper came out from the Mayo Clinic that looked at Apple's AFib feature that was FDA-cleared a few years ago and the amount of excessive health care testing and costs that it resulted in. You have a disease where prevalence is 1% or 2% at best in the general population. Even a product that has 99% accuracy is going to have thousands and thousands of false positives. So any time we think about a feature, we want to be very mindful about the value to customers balanced with the anxiety generation of having false positives.

Of course you want to deliver more valuable features to customers over time, but we want to be very mindful of what those features are and what the level of accuracy is and how clinicians and the health care system would digest that data and what the downstream actions will be. Because if [we're] just saying, "Hey, here's the feature, go use it." And then we don't really care what happens next, that's not great.

In the next six to 12 months, as usage and engagement data comes in, what are you hoping to learn about Halo?

The No. 1 thing we're all looking forward to is [learning] what aspects of the product and the service customers really engage with and doubling down there. The other one is, what are the features they're looking for that we may have overlooked or incorrectly assumed customers are not interested in? Once we get customer engagement and the use of the feature and the service on a regular basis, it will give us the opportunity to build more features and more metrics and deliver [them]. But if [we] lose the customer now, I think we don't have a chance.

We decided to launch as a membership service. One of the critical reasons we did that is that it actually makes us accountable to deliver value. Traditionally, you buy a device, you paid for it, and it's all done. But if [we] don't deliver value — as Amazon, as Halo — customers will stop purchasing the memberships. I think that is an onus on us to make sure we continue to add value.

Martin Cooper with his original DynaTAC cell phone.

Photo: Ted Soqui/Getty Images

Martin Cooper helped invent one of the most consequential and successful products in history: the cell phone. And almost five decades after he made the first public cell phone call, on a 2-pound brick of a device called the DynaTAC, he's written a book about his career called "Cutting the Cord: The Cell Phone has Transformed Humanity." In it he tells the story of the cell phone's invention, and looks at how it has changed the world and will continue to do so.

Cooper came on the Source Code Podcast to talk about his time at Motorola, the process of designing the first-ever cell phone, whether today's tech giants are monopolies, and why he's bullish on the future of AI.

Keep Reading Show less
David Pierce

David Pierce ( @pierce) is Protocol's editor at large. Prior to joining Protocol, he was a columnist at The Wall Street Journal, a senior writer with Wired, and deputy editor at The Verge. He owns all the phones.

People

What $9 billion would do for the Technology Modernization Fund

The Alliance for Digital Innovation's Matthew T. Cornelius looks at how a new administration's big investment could alter the fund he helped set up.

The funding itself is only half the battle.

Photo: Joshua Sukoff/Unsplash

The Biden administration wants to give the Technology Modernization Fund a $9 billion payday. In doing so, they could change what the fund actually does.

Matthew T. Cornelius, now the Alliance for Digital Innovation's executive director, was instrumental in getting the fund off the ground back in 2018. As a senior adviser for technology and cybersecurity policy at the White House's Office of Management and Budget, he helped make some of the fund's first investments in government IT modernization. At the time, though, there was only about $100 million in the fund.

Keep Reading Show less
Kevin McAllister

Kevin McAllister ( @k__mcallister) is an associate editor at Protocol, leading the development of Braintrust. Prior to joining the team, he was a rankings data reporter at The Wall Street Journal, where he oversaw structured data projects for the Journal's strategy team.

People

Amazon’s head of Alexa Trust on how Big Tech should talk about data

Anne Toth, Amazon's director of Alexa Trust, explains what it takes to get people to feel comfortable using your product — and why that is work worth doing.

Anne Toth, Amazon's director of Alexa Trust, has been working on tech privacy for decades.

Photo: Amazon

Anne Toth has had a long career in the tech industry, thinking about privacy and security at companies like Yahoo, Google and Slack, working with the World Economic Forum and advising companies around Silicon Valley.

Last August she took on a new job as the director of Alexa Trust, leading a big team tackling a big question: How do you make people feel good using a product like Alexa, which is designed to be deeply ingrained in their lives? "Alexa in your home is probably the closest sort of consumer experience or manifestation of AI in your life," she said. That comes with data questions, privacy questions, ethical questions and lots more.

Keep Reading Show less
David Pierce

David Pierce ( @pierce) is Protocol's editor at large. Prior to joining Protocol, he was a columnist at The Wall Street Journal, a senior writer with Wired, and deputy editor at The Verge. He owns all the phones.

People

Poshmark made ecommerce social. Wall Street is on board.

"When we go social, we're not going back," says co-founder Tracy Sun.

Tracy Sun is Poshmark's co-founder and SVP of new markets.

Photo: Poshmark/Ken Jay

Investors were keen to buy into Poshmark's vision for the future of retail — one that is social, online and secondhand. The company's stock price more than doubled within a few minutes of its Nasdaq debut this morning, rising from $42 to $103.

Poshmark is anything but an overnight success. The California-based company, founded in 2011, has steadily attracted a community of 31.7 million active users to its marketplace for secondhand apparel, accessories, footwear, home and beauty products. In 2019, these users spent an average of 27 minutes per day on the platform, placing it in the same realm as some of the most popular social media services. This is likely why Poshmark points out in its S-1 that it isn't just an ecommerce platform, but a "social marketplace." Users can like, comment, share and follow other buyers and sellers on the platform.

Keep Reading Show less
Hirsh Chitkara
Hirsh Chitkara (@ChitkaraHirsh) is a researcher at Protocol, based out of New York City. Before joining Protocol, he worked for Business Insider Intelligence, where he wrote about Big Tech, telecoms, workplace privacy, smart cities, and geopolitics. He also worked on the Strategy & Analytics team at the Cleveland Indians.
People

Affirm CEO Max Levchin: ‘I see an ocean of opportunities’

The fintech startup's stock soared more than 90% in its IPO debut today.

It was a blockbuster debut for Affirm. The fintech startup's shares soared more than 90% when it went public on Wednesday.

The day itself began quietly for CEO Max Levchin: He kicked it off with a Zoom call with his kids, made a latte for his wife and joined a group chat with some high school friends, one of whom is recovering from COVID-19. "We were very happy to hear that he's doing well," he told Protocol shortly after his startup began trading on the Nasdaq Global Exchange.

Keep Reading Show less
Benjamin Pimentel

Benjamin Pimentel ( @benpimentel) covers fintech from San Francisco. He has reported on many of the biggest tech stories over the past 20 years for the San Francisco Chronicle, Dow Jones MarketWatch and Business Insider, from the dot-com crash, the rise of cloud computing, social networking and AI to the impact of the Great Recession and the COVID crisis on Silicon Valley and beyond. He can be reached at bpimentel@protocol.com or via Signal at (510)731-8429.

Latest Stories