Beyond-Voice: Towards Continuous 3D Hand Pose Tracking on Commercial Home Assistant Devices
arxiv(2023)
摘要
The surging popularity of home assistants and their voice user interface
(VUI) have made them an ideal central control hub for smart home devices.
However, current form factors heavily rely on VUI, which poses accessibility
and usability issues; some latest ones are equipped with additional cameras and
displays, which are costly and raise privacy concerns. These concerns jointly
motivate Beyond-Voice, a novel high-fidelity acoustic sensing system that
allows commodity home assistant devices to track and reconstruct hand poses
continuously. It transforms the home assistant into an active sonar system
using its existing onboard microphones and speakers. We feed a high-resolution
range profile to the deep learning model that can analyze the motions of
multiple body parts and predict the 3D positions of 21 finger joints, bringing
the granularity for acoustic hand tracking to the next level. It operates
across different environments and users without the need for personalized
training data. A user study with 11 participants in 3 different environments
shows that Beyond-Voice can track joints with an average mean absolute error of
16.47mm without any training data provided by the testing subject.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要