(DEEP) Deep Emotion Processing for Social Agents Combining Social Signal Interpretation
and Computationally Modeling User Emotions

• Applicants: Professorin Dr. Elisabeth André; Dr.-Ing. Patrick Gebhard
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: from 2018 to 2024
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 392401413

The DEEP project tackles the challenge of connecting the outside world with internal symbolic situational representations that are related to individual human emotions. Therefore, the project creates and evaluates a unique combination of a real-time interpretation of human social signals and a real-time computational model of emotions in a dyadic communication setup between a human and a Social Agent.The combination relies on a sophisticated representation of communicative emotions, and internal emotions, possible emotion elicitors and emotion targets, suitable emotion regulation strategies, and related sequences of social signals and its directions. At runtime, based on the interpretation of the social signals of a human dialog partner, a dynamic theory of mind representation of user emotions is created. It holds all possible internal user emotions with related mental states and cognitive strategies. As a result, this approach allows for a first time a real-time computationally disambiguation of emotion elicitors, emotion targets, and the recognition of possible emotion regulation strategies based on the interpretation of social signals.Overall, the DEEP project realizes a computational real-time model that describes, on a symbolic level, how social cues can be linked to emotional appraisal context and internal emotional states. The model takes the personality, role, status, relation(s), and other individual values of a user into account. The DEEP model is evaluated in dyadic dialogs between a human and a Social Agent. In the future, the DEEP model can be exploited for the creation and investigation of next generation Social Agent applications by extending the user model of such systems by a real-time model of internal feelings and emotion regulation strategies. This allows a more empathic adaptation to the current situation of a human user.

• DFG Programme: Research Grants
• International Connection: Austria, Switzerland
• Cooperation Partners: Professorin Dr. Eva Bänninger-Huber; Professor Dr. Cornelius König; Professor Dr. Ulrich Moser