When psychologist Dr. Paul Ekman visited the Fore tribe in the highlands of Papua New Guinea in 1967, he probably didn’t imagine that his work would become the foundation for some of the latest developments in artificial intelligence (AI).
After studying the tribe, which was still living in the preliterate state it had been in since the Stone Age, Ekman believed he had found the blueprint for a set of universal human emotions and related expressions that crossed cultures and were present in all humans. A decade later he created the Facial Action Coding System, a comprehensive tool for objectively measuring facial movement. Ekman’s work has been used by the FBI and police departments to identify the seeds of violent behavior in nonverbal expressions of sentiment. He has also developed the online Atlas of Emotions at the behest of the Dalai Lama.
And today his research is being used to teach computer systems how to feel.
Facial expressions are just one set of data that’s fueling the rapid advancement of a subset of AI called “affective computing.” Researchers and developers are creating algorithms that try to determine the emotional state of the human on the other side of the machine based on input such as gestures, facial expressions, text, and tone of voice.
More importantly, they’re using machine-learning techniques to develop increasingly emotional-intelligent interfaces that can not only accurately detect a person’s mood but also respond to it appropriately. A number of startups have already amassed databases of millions of human facial reactions and libraries of written communication and are actively hunting for patterns to predict human emotion—and resulting behavior—on a large scale.
Just as once-novel voice recognition technology is now a ubiquitous part of human–machine relationships, so too could this kind of mood recognition technology soon pervade digital interactions—and help businesses peer into our inner feelings.
“Once you are able to analyze a person’s affective state, you can also respond to it and influence it,” says Stacy Marsella, a professor in Northeastern University’s College of Computer and Information Science with a joint appointment in psychology.
The customer experience is the most obvious sweet spot for affective computing capabilities. Forrester analyzed its customer experience data from 2014 and 2015 and found that emotion was the number-one factor in determining customer loyalty in 17 out of the 18 industries surveyed—far more important than the ease or effectiveness of their interactions with a company. Yet most businesses have focused more on the functional experience their customers have with them than on the emotional one, in large part because, until now, there has been no easy way to assess or address the latter.
But the potential benefits of affective computing go beyond building a better customer service bot. Low-cost, wearable sensors could enable companies to measure how environment and experiences affect employee mood. Organizations could use this knowledge to design more effective work settings and processes to increase productivity and employee satisfaction. Empathy could be built into enterprise software systems to improve the user experience by, for example, sensing when employees become frustrated with a task and offering feedback or suggestions for help.
Indeed, emotion is already big business and is expected to become much bigger. The global affective computing market is estimated to grow from just over US$9.3 billion a year in 2015 to more than $42.5 billion by 2020, according to market research firm Research and Markets. In fact, the firm predicts that affective computing will “revolutionize the way organizations, especially across the retail, healthcare, government and defense, and academia sectors, gather, organize, collaborate, and deliver information.