ALOE: Active Learning based Opportunistic Experience Sampling for Smartphone Keyboard driven Emotion Self-report Collection

Smartphone keyboard interaction based emotion detection systems are used widely to provide value-added services such as mental health monitoring, keyboard layout optimization, guided response generation. At the core of these services lie a machine learning model, which automatically infers emotion based on keyboard interaction pattern. To train these models, the emotion ground truth labels are typically collected as emotion self-report by conducting an Experience Sampling Method (ESM) based study. However, as responding to repetitive self-report probes is time-consuming and fatigue-inducing, efficient self-report collection approaches are essential that avoid probing at inopportune moments and reduce survey fatigue. To address this problem, we propose an active learning based framework, ALOE (Active Learning based Opportunistic Experience Sampling for Emotion Self-report Collection) that automatically decides to avoid probing at the unfavorable moments based on the typing signatures captured from smartphone keyboard interaction sessions. We bootstrap the framework with a few labeled instances (typing session) and allow the learner to probe (or query) the user only when it is least confident about an instance (typing session) and retrain accordingly. This way, we reduce the number of probes required (and therefore user engagement) and yet probe at the opportune moments. We evaluate ALOE in a 3-week in-the-wild study involving 18 participants, who record their smartphone keyboard interaction patterns and emotion self-reports during this period. The experimental results demonstrate that ALOE requires 56% less inopportune self-reports to train the probing moment detection learning model and yet detects the probing moments accurately with an average F-score of 93%.