Long-Term Activity Recognition with Wearable Sensors
Final Report Abstract
• A novel open-source hardware platform was designed that allows effcient and precise logging of activity data. Its embedded algorithms abstract the incoming sensor data so that only the most essential patterns are stored, allowing the hardware to run for approximately three weeks on a miniature rechargeable battery. To date, several hundreds of prototypes have been manufactured, with enclosures and in batches of 100, and used in experiments across Europe. • A set of recognition methods have been designed for automatically analyzing the logged data for sleeping behavior (times and amount of sleep, poses, presence of twitches), leisure activities and fitness training, as well as short sporadic activities. The methods match state-of-the-art activity recognition methods while scaling to real-world deployments, and foresee using existing data such as time-use surveys to increase performance. • A high variety of deployments with patients from a range of medical fields, such as sleep study, psychology, and psychiatry, have been performed to evaluate the research for feasibility in long-term trials and medical monitoring tasks. Further collaborations with psychologists from leading German institutes have led to plans for a 1-year trial with 25 bipolar patients to integrate the outcomes from all previous deployments and assess the activity recognition system in a truly long-term setting for its applicability as a psychological instrument. Data sets and scripts from the above-mentioned studies have been published online (http://www.ess.tu-darmstadt.de/datasets) and have been used extensively by other research. Two changes to the project plan occurred, mostly as a result of the project’s deployment-driven approach and a widening of the medical scenarios benefitting of long-term activity recognition: More attention to the embedded platform was required, including the request to make the prototypes look and function like a wrist watch. Far more time and effort were needed than originally allocated to find good constellations of sensor functionality and design. Two types of activities suggested by the psychiatrists led to medical application scenarios on their own, and were integrated as collaborations with sleep researchers and psychologists studying smoking cessation, alongside the core long-term activity recognition for psychiatry. These were overcome with additional funding and delaying the extensive trials with domain experts. Research of this project has been presented at leading venues in embedded sensing and ubiquitous computing, but also through invited talks at psychology venues and mainstream articles (such as MIT TechReview 10.7.2013 or Stuttgarter Zeitung 21.1.2015).
Publications
- ”Coming to Grips with the Objects We Grasp: Detecting Interactions with E cient Wrist-Worn Sensors”, TEI 2010, Cambridge MA, USA, ACM Press, pp. 57-64, 01/2010
Eugen Berlin, Jun Liu, Kristof Van Laerhoven and Bernt Schiele
- ”How to Log Sleeping Trends? A Case Study on the Long-Term Capturing of User Data”, The 5th European Conference on Smart Sensing and Context 2010 (EuroSSC 2010), vol. 6446, Passau, Germany, Springer Verlag, pp. 15-27, 2010
Holger Becker, Marko Borazio and Kristof Van Laerhoven
- ”myHealthAssistant: A Phone-based Body Sensor Network that Captures the Wearer’s Exercises throughout the Day”, The 6th International Conference on Body Area Networks, Beijing, China, ACM Press, 11/2011
Christian Seeger, Alejandro Buchmann and Kristof Van Laerhoven
- ”Predicting Sleeping Behaviors in Long-Term Studies with Wrist-Worn Sensor Data”, International Joint Conference on Ambient Intelligence (AmI-11), vol. LNCS 7040, Amsterdam, Springer Verlag, pp. 151-156, 11/2011
Marko Borazio and Kristof Van Laerhoven
- ”Combining Wearable and Environmental Sensing into an Unobtrusive Tool for Long-Term Sleep Studies”, 2nd ACM SIGHIT International Health Informatics Symposium (IHI 2012), Miami, Florida, USA, ACM Press, 01/2012
Marko Borazio and Kristof Van Laerhoven
- ”Detecting Leisure Activities with Dense Motif Discovery”, 14th ACM International Conference on Ubiquitous Computing (UbiComp 2012), Pittsburgh, PA, USA, ACM Press, pp. 250-259, 09/2012
Eugen Berlin and Kristof Van Laerhoven
- ”A Publish/Subscribe Middleware for Body and Ambient Sensor Networks that Mediates between Sensors and Applications”, IEEE International Conference on Healthcare Informatics (ICHI 2013), Philadelphia, PA, IEEE Press, 09/2013
Christian Seeger, Kristof Van Laerhoven, Jens Sauer and Alejandro Buchmann
- ”Already Up? Using Mobile Phones to Track % Share Sleep Behavior”, International Journal of Human-Computer Studies: Elsevier, 2013
Alireza Sahami Shirazi, James Clawson, Yashar Hassanpour, Mohammad J Tourian, Ed Chi, Marko Borazio, Albrecht Schmidt and Kristof Van Laerhoven
- "Towards Benchmarked Sleep Detection with Inertial Wrist-worn Sensing Units”, ICHI 2014, Verona, Italy, IEEE Press, 09/2014
Marko Borazio, Eugen Berlin, Nagihan Kücükyildiz, Philipp M Scholl and Kristof Van Laerhoven
(See online at https://doi.org/10.1109/ICHI.2014.24) - ”MyHealthAssistant: An Eventdriven Middleware for Multiple Medical Applications on a Smartphone-mediated Body Sensor Network”, IEEE Journal of Biomedical and Health Informatics (J-BHI), vol. PP, no. 9, 05/2014
Christian Seeger, Kristof Van Laerhoven and Alejandro Buchmann
(See online at https://doi.org/10.1109/JBHI.2014.2326604) - ”Low-power Lessons from Designing a Wearable Logger for Long-term Deployments”, 2015 IEEE Sensors Applications Symposium (SAS 2015), Zadar, Croatia, IEEE, 2015
Eugen Berlin, Martin Zittel, Michael Bräunlein and Kristof Van Laerhoven
(See online at https://doi.org/10.1109/SAS.2015.7133581)
