Enterprise

Intel calls its AI that detects student emotions a teaching tool. Others call it 'morally reprehensible.'

Virtual school software startup Classroom Technologies will test the controversial “emotion AI” technology.

Zoom grid with emojis

The system can detect whether students are bored, distracted or confused.

Illustration: Christopher T. Fong/Protocol

When college instructor Angela Dancey wants to decipher whether her first-year English students comprehend what she’s trying to get across in class, their facial expressions and body language don’t reveal much.

"Even in an in-person class, students can be difficult to read. Typically, undergraduates don't communicate much through their faces, especially a lack of understanding,” said Dancey, a senior lecturer at the University of Illinois Chicago.

Dancey uses tried-and-true methods such as asking students to identify their "muddiest point" — a concept or idea she said students still struggle with — following a lecture or discussion. "I ask them to write it down, share it and we address it as a class for everyone's benefit," she said.

But Intel and Classroom Technologies, which sells virtual school software called Class, think there might be a better way. The companies have partnered to integrate an AI-based technology developed by Intel with Class, which runs on top of Zoom. Intel claims its system can detect whether students are bored, distracted or confused by assessing their facial expressions and how they’re interacting with educational content.

“We can give the teacher additional insights to allow them to better communicate,” said Michael Chasen, co-founder and CEO of Classroom Technologies, who said teachers have had trouble engaging with students in virtual classroom environments throughout the pandemic.

His company plans to test Intel’s student engagement analytics technology, which captures images of students’ faces with a computer camera and computer vision technology and combines it with contextual information about what a student is working on at that moment to assess a student’s state of understanding. Intel's partnership with Class is a research proof-of-concept undertaking, said Sinem Aslan, a research scientist at Intel, who helped develop the technology.

“We are trying to enable one-on-one tutoring at scale,” said Aslan, adding that the system is intended to help teachers recognize when students need help and to inform how they might alter educational materials based on how students interact with the educational content. “High levels of boredom will lead [students to] completely zone out of educational content,” said Aslan.

But critics argue that it is not possible to accurately determine whether someone is feeling bored, confused, happy or sad based on their facial expressions or other external signals.

Some researchers have found that because people express themselves through tens or hundreds of subtle and complex facial expressions, bodily gestures or physiological signals, categorizing their state with a single label is an ill-suited approach. Other research indicates that people communicate emotions such as anger, fear and surprise in ways that vary across cultures and situations, and how they express emotion can fluctuate on an individual level.

“Students have different ways of presenting what’s going on inside of them,” said Todd Richmond, a longtime educator and the director of the Tech and Narrative Lab and a professor at the Pardee RAND Graduate School. “That student being distracted at that moment in time may be the appropriate and necessary state for them in that moment in their life,” he said, if they’re dealing with personal issues, for example.

Controversial emotion AI seeps into everyday tech

The classroom is just one arena where controversial “emotion AI” is finding its way into everyday tech products and generating investor interest. It’s also seeping into delivery and passenger vehicles and virtual sales and customer service software. After Protocol's report last week on the use of this technology on sales calls, Fight for the Future launched a campaign urging Zoom not to adopt the technology in its near-ubiquitous video-conferencing software.

At this early stage, it’s not clear how Intel’s technology will be integrated with the Class software, said Chasen, who said he expects the company will partner with one of the colleges it already works with to evaluate the Intel system. Chasen told Protocol that Classroom Technologies is not paying Intel to test the technology. Class is backed by investors including NFL quarterback Tom Brady, AOL co-founder Steve Case and Salesforce Ventures.

Intel has established partnerships to help distribute other nascent forms of AI it has built. For example, in the hopes of productizing a system that turns data visualizing joints and skeletal movements into analytics to monitor and improve athletic performance, the company has partnered with Purdue University and soccer scouting app AiScout.



Educators
and advocacy groups have raised alarms regarding excessive student surveillance and privacy invasions associated with facial recognition deployed in schools for identification and security purposes. Those concerns have accelerated as AI-based software has been used more often than ever during the pandemic, including technologies that monitor student behavior in the hopes of preventing cheating during virtual testing and systems that track content that students view on their laptops in an effort to detect whether they are at risk of self-harm.

Class already tracks how often students raise their hands during a session, and offers a “proctor view” feature that lets teachers monitor what students are viewing on their computers if the students agree to share their desktop screen with instructors.

“I think we have to be very sensitive about people’s personal rights and not being overly intrusive with these systems,” said Chasen.

Cameras as a social-justice issue

As virtual class became the norm in the past couple years, a debate emerged among educators over whether or not to require students to turn on their cameras during class. Today in Dancey’s English program, cameras are optional, in part because in virtual settings students can communicate with instructors through their microphones or via chat.

But in order to capture students’ facial expressions, Intel’s technology would need those cameras turned on.

“The thing about turning cameras on, it became almost like a social-justice issue,” Dancey said. Not only are some students concerned about others seeing where or how they live, but enabling the cameras drains power, which can be a problem for students using a mobile hotspot to connect for class, she said.

“It’s kind of an invasion of privacy, and there are accessibility issues, because having your camera on uses up a huge amount of bandwidth. That could literally be costing them money to do that,” Dancey said.

We don’t want this technology to be a surveillance system.

“Students shouldn’t have to police how they look in the classroom,” said Nandita Sampath, a policy analyst with Consumer Reports focused on algorithmic bias and accountability issues, who said she wondered whether students would have the ability to contest inaccurate results if Intel’s system leads to negative consequences. “What cognitive and emotional states do these companies claim they are able to assess or predict, and what is the accountability?” she said.

Aslan said the goal of Intel’s technology is not to surveil or penalize students, but rather to coach teachers and provide additional information so they can better understand when students need help. “We did not start this technology as a surveillance system. In fact, we don’t want this technology to be a surveillance system,” Aslan said.

Sampath said Intel’s technology could be used to judge or penalize students even if that is not the intent. “Maybe they might not intend for this to be the ultimate decision-maker, but this doesn’t mean the teacher or administrator can’t use it in that way,” she said.

Dancey said teachers worry about surveillance being used against them, too. “Often surveillance is used against instructors really unfairly,” she said. “I don’t think it would be paranoid to say, especially if it’s going to measure ‘student engagement’ — TM, in quotes — that If I go up for promotion or tenure, is that going to be part of my evaluation? Could they say, ‘So-and-so had a low comprehension quotient?’”

When Intel tested the system in a physical classroom setting, some teachers who participated in the study suggested it provided useful information. “I was able to witness how I could catch some emotional challenges of the students that I could not have anticipated [before],” said one teacher, according to a document provided by Intel.

But while some teachers may have found it helpful, Dancey said she would not want to use the Intel system. “I think most teachers, especially at the university level, would find this technology morally reprehensible, like the panopticon. Frankly, if my institution offered it to me, I would reject it, and if we were required to use it, I would think twice about continuing to work here,” she said.

AI data prep by psychologists


Four days after this story was published, Intel said it wanted to offer an additional statement about its emotion AI work and partnership with Classroom Technologies.

"Intel’s partnership to test the technology in the Class software at this stage is a research proof of concept. We have no near-term plans to productize the technology and are currently collaborating with Class to conduct further research to identify any socio-technical challenges or outcomes to refine and iterate on this proof of concept," said Mindy Nelson, an Intel representative, in an email.

At this early stage, Intel aims to find the best ways to implement the technology so it is most useful for teachers, Aslan said: “How do we make it in a way that it is aligned with what the teacher does on a daily basis?”

I think most teachers, especially at the university level, would find this technology morally reprehensible.

Intel developed its adaptive learning analytics system by incorporating data gathered from students in real-life classroom sessions using laptops with 3D cameras. To label the ground truth data used to train its algorithmic models, the researchers hired psychologists who viewed videos of the students and categorized the emotions they detected in their expressions.

“We don’t want to start with any assumptions. That’s why we hired the subject matter experts to label the data,” said Nese Alyuz Civitci, a machine-learning researcher at Intel. The researchers only used data when at least two of three labelers agreed how a student’s expressions should be categorized.

“It was really interesting to see those emotions — the states are really subtle, they are really tiny differences,” Civitci said. “It was really hard for me to identify those differences.”

Rather than assessing Intel’s AI models on whether they accurately reflected the actual emotions of students, the researchers “positioned it as how instrumental or how much a teacher can trust the models,” Aslan said.

“I don’t think it’s tech that’s fully reached its maturity yet,” Chasen said regarding Intel’s system. “We need to see if the results are relevant to the performance of the students and see if we can’t get useful data for the instructors out of it. This is what we’re testing to find out.”

Ultimately, he said the Intel system will provide one piece of data that Classroom Technologies and its customers will combine with other signals to form a holistic assessment of students.

“There’s never one piece of data,” he said. He also suggested that the information revealed by the Intel technology should not be used on its own without context to judge a student’s performance, such as, “if the AI says they’re not paying attention and they have all straight As.”

This story was updated to clarify the status of Intel's work with Class Technologies and include an additional statement.

Enterprise

How I decided to leave the US and pursue a tech career in Europe

Melissa Di Donato moved to Europe to broaden her technology experience with a different market perspective. She planned to stay two years. Seventeen years later, she remains in London as CEO of Suse.

“It was a hard go for me in the beginning. I was entering inside of a company that had been very traditional in a sense.”

Photo: Suse

Click banner image for more How I decided seriesA native New Yorker, Melissa Di Donato made a life-changing decision back in 2005 when she packed up for Europe to further her career in technology. Then with IBM, she made London her new home base.

Today, Di Donato is CEO of Germany’s Suse, now a 30-year-old, open-source enterprise software company that specializes in Linux operating systems, container management, storage, and edge computing. As the company’s first female leader, she has led Suse through the coronavirus pandemic, a 2021 IPO on the Frankfurt Stock Exchange, and the acquisitions of Kubernetes management startup Rancher Labs and container security company NeuVector.

Keep Reading Show less
Donna Goodison

Donna Goodison (@dgoodison) is Protocol's senior reporter focusing on enterprise infrastructure technology, from the 'Big 3' cloud computing providers to data centers. She previously covered the public cloud at CRN after 15 years as a business reporter for the Boston Herald. Based in Massachusetts, she also has worked as a Boston Globe freelancer, business reporter at the Boston Business Journal and real estate reporter at Banker & Tradesman after toiling at weekly newspapers.

Sponsored Content

Great products are built on strong patents

Experts say robust intellectual property protection is essential to ensure the long-term R&D required to innovate and maintain America's technology leadership.

Every great tech product that you rely on each day, from the smartphone in your pocket to your music streaming service and navigational system in the car, shares one important thing: part of its innovative design is protected by intellectual property (IP) laws.

From 5G to artificial intelligence, IP protection offers a powerful incentive for researchers to create ground-breaking products, and governmental leaders say its protection is an essential part of maintaining US technology leadership. To quote Secretary of Commerce Gina Raimondo: "intellectual property protection is vital for American innovation and entrepreneurship.”

Keep Reading Show less
James Daly
James Daly has a deep knowledge of creating brand voice identity, including understanding various audiences and targeting messaging accordingly. He enjoys commissioning, editing, writing, and business development, particularly in launching new ventures and building passionate audiences. Daly has led teams large and small to multiple awards and quantifiable success through a strategy built on teamwork, passion, fact-checking, intelligence, analytics, and audience growth while meeting budget goals and production deadlines in fast-paced environments. Daly is the Editorial Director of 2030 Media and a contributor at Wired.
Enterprise

UiPath had a rocky few years. Rob Enslin wants to turn it around.

Protocol caught up with Enslin, named earlier this year as UiPath’s co-CEO, to discuss why he left Google Cloud, the untapped potential of robotic-process automation, and how he plans to lead alongside founder Daniel Dines.

Rob Enslin, UiPath's co-CEO, chats with Protocol about the company's future.

Photo: UiPath

UiPath has had a shaky history.

The company, which helps companies automate business processes, went public in 2021 at a valuation of more than $30 billion, but now the company’s market capitalization is only around $7 billion. To add insult to injury, UiPath laid off 5% of its staff in June and then lowered its full-year guidance for fiscal year 2023 just months later, tanking its stock by 15%.

Keep Reading Show less
Aisha Counts

Aisha Counts (@aishacounts) is a reporter at Protocol covering enterprise software. Formerly, she was a management consultant for EY. She's based in Los Angeles and can be reached at acounts@protocol.com.

Workplace

Figma CPO: We can do more with Adobe

Yuhki Yamashita thinks Figma might tackle video or 3D objects someday.

Figman CPO Yuhki Yamashita told Protocol about Adobe's acquisition of the company.

Photo: Figma

Figma CPO Yuhki Yamashita’s first design gig was at The Harvard Crimson, waiting for writers to file their stories so he could lay them out in Adobe InDesign. Given his interest in computer science, pursuing UX design became the clear move. He worked on Outlook at Microsoft, YouTube at Google, and user experience at Uber, where he was a very early user of Figma. In 2019, he became a VP of product at Figma; this past June, he became CPO.

“Design has been really near and dear to my heart, which is why when this opportunity came along to join Figma and rethink design, it was such an obvious opportunity,” Yamashita said.

Keep Reading Show less
Lizzy Lawrence

Lizzy Lawrence ( @LizzyLaw_) is a reporter at Protocol, covering tools and productivity in the workplace. She's a recent graduate of the University of Michigan, where she studied sociology and international studies. She served as editor in chief of The Michigan Daily, her school's independent newspaper. She's based in D.C., and can be reached at llawrence@protocol.com.

Climate

Microsoft lays out its climate advocacy goals

The tech giant has staked out exactly what kind of policies it will support to decarbonize the world and clean up the grid.

Microsoft published two briefs explaining what new climate policies it will advocate for.

Photo by Jeremy Bezanger on Unsplash

The tech industry has no shortage of climate goals, but they’ll be very hard to achieve without the help of sound public policy.

Microsoft published two new briefs on Sept. 22 explaining what policies it will advocate for in the realm of reducing carbon and cleaning up the grid. With policymakers in the U.S. and around the world beginning to weigh more stringent climate policies (or in the U.S.’s case, any serious climate policies at all), the briefs will offer a measuring stick for whether Microsoft is living up to its ideals.

Keep Reading Show less
Brian Kahn

Brian ( @blkahn) is Protocol's climate editor. Previously, he was the managing editor and founding senior writer at Earther, Gizmodo's climate site, where he covered everything from the weather to Big Oil's influence on politics. He also reported for Climate Central and the Wall Street Journal. In the even more distant past, he led sleigh rides to visit a herd of 7,000 elk and boat tours on the deepest lake in the U.S.

Latest Stories
Bulletins