Enterprise

Intel calls its AI that detects student emotions a teaching tool. Others call it 'morally reprehensible.'

Virtual school software startup Classroom Technologies will test the controversial “emotion AI” technology.

Zoom grid with emojis

The system can detect whether students are bored, distracted or confused.

Illustration: Christopher T. Fong/Protocol

When college instructor Angela Dancey wants to decipher whether her first-year English students comprehend what she’s trying to get across in class, their facial expressions and body language don’t reveal much.

"Even in an in-person class, students can be difficult to read. Typically, undergraduates don't communicate much through their faces, especially a lack of understanding,” said Dancey, a senior lecturer at the University of Illinois Chicago.

Dancey uses tried-and-true methods such as asking students to identify their "muddiest point" — a concept or idea she said students still struggle with — following a lecture or discussion. "I ask them to write it down, share it and we address it as a class for everyone's benefit," she said.

But Intel and Classroom Technologies, which sells virtual school software called Class , think there might be a better way. The companies have partnered to integrate an AI-based technology developed by Intel with Class, which runs on top of Zoom. Intel claims its system can detect whether students are bored, distracted or confused by assessing their facial expressions and how they’re interacting with educational content.

“We can give the teacher additional insights to allow them to better communicate,” said Michael Chasen, co-founder and CEO of Classroom Technologies, who said teachers have had trouble engaging with students in virtual classroom environments throughout the pandemic.

His company plans to test Intel’s student engagement analytics technology, which captures images of students’ faces with a computer camera and computer vision technology and combines it with contextual information about what a student is working on at that moment to assess a student’s state of understanding. Intel's partnership with Class is a research proof-of-concept undertaking, said Sinem Aslan, a research scientist at Intel, who helped develop the technology.

“We are trying to enable one-on-one tutoring at scale,” said Aslan, adding that the system is intended to help teachers recognize when students need help and to inform how they might alter educational materials based on how students interact with the educational content. “High levels of boredom will lead [students to] completely zone out of educational content,” said Aslan.

But critics argue that it is not possible to accurately determine whether someone is feeling bored, confused, happy or sad based on their facial expressions or other external signals.

Some researchers have found that because people express themselves through tens or hundreds of subtle and complex facial expressions, bodily gestures or physiological signals, categorizing their state with a single label is an ill-suited approach . Other research indicates that people communicate emotions such as anger, fear and surprise in ways that vary across cultures and situations, and how they express emotion can fluctuate on an individual level.

“Students have different ways of presenting what’s going on inside of them,” said Todd Richmond, a longtime educator and the director of the Tech and Narrative Lab and a professor at the Pardee RAND Graduate School. “That student being distracted at that moment in time may be the appropriate and necessary state for them in that moment in their life,” he said, if they’re dealing with personal issues, for example.

Controversial emotion AI seeps into everyday tech

The classroom is just one arena where controversial “emotion AI” is finding its way into everyday tech products and generating investor interest. It’s also seeping into delivery and passenger vehicles and virtual sales and customer service software. After Protocol's report last week on the use of this technology on sales calls, Fight for the Future launched a campaign urging Zoom not to adopt the technology in its near-ubiquitous video-conferencing software.

At this early stage, it’s not clear how Intel’s technology will be integrated with the Class software, said Chasen, who said he expects the company will partner with one of the colleges it already works with to evaluate the Intel system. Chasen told Protocol that Classroom Technologies is not paying Intel to test the technology. Class is backed by investors including NFL quarterback Tom Brady, AOL co-founder Steve Case and Salesforce Ventures.

Intel has established partnerships to help distribute other nascent forms of AI it has built. For example, in the hopes of productizing a system that turns data visualizing joints and skeletal movements into analytics to monitor and improve athletic performance, the company has partnered with Purdue University and soccer scouting app AiScout.



Educators
and advocacy groups have raised alarms regarding excessive student surveillance and privacy invasions associated with facial recognition deployed in schools for identification and security purposes. Those concerns have accelerated as AI-based software has been used more often than ever during the pandemic, including technologies that monitor student behavior in the hopes of preventing cheating during virtual testing and systems that track content that students view on their laptops in an effort to detect whether they are at risk of self-harm .

Class already tracks how often students raise their hands during a session, and offers a “proctor view” feature that lets teachers monitor what students are viewing on their computers if the students agree to share their desktop screen with instructors.

“I think we have to be very sensitive about people’s personal rights and not being overly intrusive with these systems,” said Chasen.

Cameras as a social-justice issue

As virtual class became the norm in the past couple years, a debate emerged among educators over whether or not to require students to turn on their cameras during class. Today in Dancey’s English program, cameras are optional, in part because in virtual settings students can communicate with instructors through their microphones or via chat.

But in order to capture students’ facial expressions, Intel’s technology would need those cameras turned on.

“The thing about turning cameras on, it became almost like a social-justice issue,” Dancey said. Not only are some students concerned about others seeing where or how they live, but enabling the cameras drains power, which can be a problem for students using a mobile hotspot to connect for class, she said.

“It’s kind of an invasion of privacy, and there are accessibility issues, because having your camera on uses up a huge amount of bandwidth. That could literally be costing them money to do that,” Dancey said.

We don’t want this technology to be a surveillance system.

“Students shouldn’t have to police how they look in the classroom,” said Nandita Sampath, a policy analyst with Consumer Reports focused on algorithmic bias and accountability issues, who said she wondered whether students would have the ability to contest inaccurate results if Intel’s system leads to negative consequences. “What cognitive and emotional states do these companies claim they are able to assess or predict, and what is the accountability?” she said.

Aslan said the goal of Intel’s technology is not to surveil or penalize students, but rather to coach teachers and provide additional information so they can better understand when students need help. “We did not start this technology as a surveillance system. In fact, we don’t want this technology to be a surveillance system,” Aslan said.

Sampath said Intel’s technology could be used to judge or penalize students even if that is not the intent. “Maybe they might not intend for this to be the ultimate decision-maker, but this doesn’t mean the teacher or administrator can’t use it in that way,” she said.

Dancey said teachers worry about surveillance being used against them, too. “Often surveillance is used against instructors really unfairly,” she said. “I don’t think it would be paranoid to say, especially if it’s going to measure ‘student engagement’ — TM, in quotes — that If I go up for promotion or tenure, is that going to be part of my evaluation? Could they say, ‘So-and-so had a low comprehension quotient?’”

When Intel tested the system in a physical classroom setting, some teachers who participated in the study suggested it provided useful information. “I was able to witness how I could catch some emotional challenges of the students that I could not have anticipated [before],” said one teacher, according to a document provided by Intel.

But while some teachers may have found it helpful, Dancey said she would not want to use the Intel system. “I think most teachers, especially at the university level, would find this technology morally reprehensible, like the panopticon . Frankly, if my institution offered it to me, I would reject it, and if we were required to use it, I would think twice about continuing to work here,” she said.

AI data prep by psychologists


Four days after this story was published, Intel said it wanted to offer an additional statement about its emotion AI work and partnership with Classroom Technologies.

"Intel’s partnership to test the technology in the Class software at this stage is a research proof of concept. We have no near-term plans to productize the technology and are currently collaborating with Class to conduct further research to identify any socio-technical challenges or outcomes to refine and iterate on this proof of concept," said Mindy Nelson, an Intel representative, in an email.

At this early stage, Intel aims to find the best ways to implement the technology so it is most useful for teachers, Aslan said: “How do we make it in a way that it is aligned with what the teacher does on a daily basis?”

I think most teachers, especially at the university level, would find this technology morally reprehensible.

Intel developed its adaptive learning analytics system by incorporating data gathered from students in real-life classroom sessions using laptops with 3D cameras. To label the ground truth data used to train its algorithmic models, the researchers hired psychologists who viewed videos of the students and categorized the emotions they detected in their expressions.

“We don’t want to start with any assumptions. That’s why we hired the subject matter experts to label the data,” said Nese Alyuz Civitci, a machine-learning researcher at Intel. The researchers only used data when at least two of three labelers agreed how a student’s expressions should be categorized.

“It was really interesting to see those emotions — the states are really subtle, they are really tiny differences,” Civitci said. “It was really hard for me to identify those differences.”

Rather than assessing Intel’s AI models on whether they accurately reflected the actual emotions of students, the researchers “positioned it as how instrumental or how much a teacher can trust the models,” Aslan said.

“I don’t think it’s tech that’s fully reached its maturity yet,” Chasen said regarding Intel’s system. “We need to see if the results are relevant to the performance of the students and see if we can’t get useful data for the instructors out of it. This is what we’re testing to find out.”

Ultimately, he said the Intel system will provide one piece of data that Classroom Technologies and its customers will combine with other signals to form a holistic assessment of students.

“There’s never one piece of data,” he said. He also suggested that the information revealed by the Intel technology should not be used on its own without context to judge a student’s performance, such as, “if the AI says they’re not paying attention and they have all straight As.”

This story was updated to clarify the status of Intel's work with Class Technologies and include an additional statement.

Fintech

Judge Zia Faruqui is trying to teach you crypto, one ‘SNL’ reference at a time

His decisions on major cryptocurrency cases have quoted "The Big Lebowski," "SNL," and "Dr. Strangelove." That’s because he wants you — yes, you — to read them.

The ways Zia Faruqui (right) has weighed on cases that have come before him can give lawyers clues as to what legal frameworks will pass muster.

Photo: Carolyn Van Houten/The Washington Post via Getty Images

“Cryptocurrency and related software analytics tools are ‘The wave of the future, Dude. One hundred percent electronic.’”

That’s not a quote from "The Big Lebowski" — at least, not directly. It’s a quote from a Washington, D.C., district court memorandum opinion on the role cryptocurrency analytics tools can play in government investigations. The author is Magistrate Judge Zia Faruqui.

Keep Reading Show less
Veronica Irwin

Veronica Irwin (@vronirwin) is a San Francisco-based reporter at Protocol covering fintech. Previously she was at the San Francisco Examiner, covering tech from a hyper-local angle. Before that, her byline was featured in SF Weekly, The Nation, Techworker, Ms. Magazine and The Frisc.

The financial technology transformation is driving competition, creating consumer choice, and shaping the future of finance. Hear from seven fintech leaders who are reshaping the future of finance, and join the inaugural Financial Technology Association Fintech Summit to learn more .

Keep Reading Show less
FTA
The Financial Technology Association (FTA) represents industry leaders shaping the future of finance. We champion the power of technology-centered financial services and advocate for the modernization of financial regulation to support inclusion and responsible innovation.
Enterprise

AWS CEO: The cloud isn’t just about technology

As AWS preps for its annual re:Invent conference, Adam Selipsky talks product strategy, support for hybrid environments, and the value of the cloud in uncertain economic times.

Photo: Noah Berger/Getty Images for Amazon Web Services

AWS is gearing up for re:Invent, its annual cloud computing conference where announcements this year are expected to focus on its end-to-end data strategy and delivering new industry-specific services.

It will be the second re:Invent with CEO Adam Selipsky as leader of the industry’s largest cloud provider after his return last year to AWS from data visualization company Tableau Software.

Keep Reading Show less
Donna Goodison

Donna Goodison ( @dgoodison ) is Protocol's senior reporter focusing on enterprise infrastructure technology, from the 'Big 3' cloud computing providers to data centers. She previously covered the public cloud at CRN after 15 years as a business reporter for the Boston Herald. Based in Massachusetts, she also has worked as a Boston Globe freelancer, business reporter at the Boston Business Journal and real estate reporter at Banker & Tradesman after toiling at weekly newspapers.

Image: Protocol

We launched Protocol in February 2020 to cover the evolving power center of tech. It is with deep sadness that just under three years later, we are winding down the publication.

As of today, we will not publish any more stories. All of our newsletters, apart from our flagship, Source Code, will no longer be sent. Source Code will be published and sent for the next few weeks, but it will also close down in December.

Keep Reading Show less
Bennett Richardson

Bennett Richardson ( @bennettrich ) is the president of Protocol. Prior to joining Protocol in 2019, Bennett was executive director of global strategic partnerships at POLITICO, where he led strategic growth efforts including POLITICO's European expansion in Brussels and POLITICO's creative agency POLITICO Focus during his six years with the company. Prior to POLITICO, Bennett was co-founder and CMO of Hinge, the mobile dating company recently acquired by Match Group. Bennett began his career in digital and social brand marketing working with major brands across tech, energy, and health care at leading marketing and communications agencies including Edelman and GMMB. Bennett is originally from Portland, Maine, and received his bachelor's degree from Colgate University.

Enterprise

Why large enterprises struggle to find suitable platforms for MLops

As companies expand their use of AI beyond running just a few machine learning models, and as larger enterprises go from deploying hundreds of models to thousands and even millions of models, ML practitioners say that they have yet to find what they need from prepackaged MLops systems.

As companies expand their use of AI beyond running just a few machine learning models, ML practitioners say that they have yet to find what they need from prepackaged MLops systems.

Photo: artpartner-images via Getty Images

On any given day, Lily AI runs hundreds of machine learning models using computer vision and natural language processing that are customized for its retail and ecommerce clients to make website product recommendations, forecast demand, and plan merchandising. But this spring when the company was in the market for a machine learning operations platform to manage its expanding model roster, it wasn’t easy to find a suitable off-the-shelf system that could handle such a large number of models in deployment while also meeting other criteria.

Some MLops platforms are not well-suited for maintaining even more than 10 machine learning models when it comes to keeping track of data, navigating their user interfaces, or reporting capabilities, Matthew Nokleby, machine learning manager for Lily AI’s product intelligence team, told Protocol earlier this year. “The duct tape starts to show,” he said.

Keep Reading Show less
Kate Kaye

Kate Kaye is an award-winning multimedia reporter digging deep and telling print, digital and audio stories. She covers AI and data for Protocol. Her reporting on AI and tech ethics issues has been published in OneZero, Fast Company, MIT Technology Review, CityLab, Ad Age and Digiday and heard on NPR. Kate is the creator of RedTailMedia.org and is the author of "Campaign '08: A Turning Point for Digital Media," a book about how the 2008 presidential campaigns used digital media and data.

Latest Stories
Bulletins