Organic Computing Techniques for Run-Time Self-Adaptation of Ubiquitous, Multi-Modal Activity Recognition Systems

• Applicants: Professor Dr. Paul Lukowicz; Professor Dr. Bernhard Sick
• Subject Area: Computer Architecture, Embedded and Massively Parallel Systems
• Term: from 2015 to 2019
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 276698135

Ubiquitous activity and context recognition (AR) aims at translating information provided by simple sensors into high level knowledge about human activities and the situation in the environment. Over the last decade researchers have shown the principle feasibility of recognizing activities and situations ranging from the steps of a maintenance task, through every day activities at home, to sport and social interactions. A major limitation of today's state of the art approaches is that they mostly assume system configurations exactly defined at the system's design-time that remain fixed at run-time. Thus, for each application, the user needs to place specific sensors at certain well-defined locations in the environment and on his body. All stages of the signal processing chain (from signal conditioning through feature selection to classification) are then custom-designed for the concrete configuration and task. While such static runtime setups can be guaranteed under controlled laboratory conditions, the possibility of sensors dropping out and new sensors appearing must be taken into account in real world settings. In this proposal we will develop new Organic Computing (OC) techniques to facilitate self-healing (when a sensor drops out) and self-improvement (when a new sensor appears) for ubiquitous activity and context recognition systems. Specifically, we will develop a layered Observer/Controller architecture where the System under Observation and Control (SuOC) is (are) human(s) in an intelligent, sensor enabled environment. The bottom layer (reaction layer) can be seen as a blueprint of standard AR based context sensitive systems. The adaptation layer enables the system to improve autonomously -- or semi-autonomously with sporadic human feedback -- the classifier at the reaction layer using the new sensor information or to adapt it a sensor drops out. In general, autonomous adaptation methods cannot guarantee to always lead to an improvement and, in special cases, they can even result in performance degradation. Thus, the potential gains and the risks of a possible adaptation are estimated and considered not only at the adaptation layer, but also at the reflection layer (top layer) that models the long term system evolution to ensure that continuous modifications of the system configuration lead to long term improvement and not to un-bounded performance degradation of the overall system. In our approach we develop new OC techniques for AR by combining and extending methods from Machine Leaning, Pattern Recognition, and related fields (in particular generative and discriminative modeling, semi-supervised learning, active learning, and nonlinear dynamic systems theory). We will evaluate our methods on existing large scale AR data sets.

• DFG Programme: Research Grants