Your Smartphone Knows you Better than you May Think: Emotional Assessment ‘on the Go’ Via TapSense

AbstractQuality of life (QoL) is a subjective term often determined by various aspects of living, such as personal well-being, health, family, and safety. QoL is challenging to capture objectively but can be anticipated through a person’s emotional state; especially positive emotions indicate an increased QoL and may be a potential indicator for other QoL aspects (such as health, safety). Affective computing is the study of technologies that can quantitatively assess human emotions from external clues. It can leverage different modalities including facial expression, physiological responses, or smartphone usage patterns and correlate them with the person’s life quality assessments. Smartphones are emerging as a main modality, mostly because of their ubiquitous availability and use throughout daily life activities. They include a plethora of onboard sensors (e.g., accelerometer, gyroscope, GPS) and can sense different user activities passively (e.g., mobility, app usage history). This chapter presents a research study (here referred to as the TapSense study) that focuses on assessing the individual’s emotional state from the smartphone usage patterns. In this TapSense study, the keyboard interaction of n = 22 participants was unobtrusively monitored for 3 weeks to determine the users’ emotional state (i.e., happy, sad, stressed, relaxed) using a personalized machine learning model. TapSense can assess emotions with an average AUCROC of 78%(±7% std). We summarize the findings and reflect upon these in the context of the potential developments within affective computing at large, in the long term, indicating a person’s quality of life.