LLMSense: Harnessing LLMs for High-level Reasoning Over Spatiotemporal Sensor Traces
CoRR(2024)
摘要
Most studies on machine learning in sensing systems focus on low-level
perception tasks that process raw sensory data within a short time window.
However, many practical applications, such as human routine modeling and
occupancy tracking, require high-level reasoning abilities to comprehend
concepts and make inferences based on long-term sensor traces. Existing machine
learning-based approaches for handling such complex tasks struggle to
generalize due to the limited training samples and the high dimensionality of
sensor traces, necessitating the integration of human knowledge for designing
first-principle models or logic reasoning methods. We pose a fundamental
question: Can we harness the reasoning capabilities and world knowledge of
Large Language Models (LLMs) to recognize complex events from long-term
spatiotemporal sensor traces? To answer this question, we design an effective
prompting framework for LLMs on high-level reasoning tasks, which can handle
traces from the raw sensor data as well as the low-level perception results. We
also design two strategies to enhance performance with long sensor traces,
including summarization before reasoning and selective inclusion of historical
traces. Our framework can be implemented in an edge-cloud setup, running small
LLMs on the edge for data summarization and performing high-level reasoning on
the cloud for privacy preservation. The results show that LLMSense can achieve
over 80% accuracy on two high-level reasoning tasks such as dementia diagnosis
with behavior traces and occupancy tracking with environmental sensor traces.
This paper provides a few insights and guidelines for leveraging LLM for
high-level reasoning on sensor traces and highlights several directions for
future work.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要