Constrained Robotic Navigation on Preferred Terrains Using LLMs and Speech Instruction: Exploiting the Power of Adverbs
CoRR(2024)
摘要
This paper explores leveraging large language models for map-free off-road
navigation using generative AI, reducing the need for traditional data
collection and annotation. We propose a method where a robot receives verbal
instructions, converted to text through Whisper, and a large language model
(LLM) model extracts landmarks, preferred terrains, and crucial adverbs
translated into speed settings for constrained navigation. A language-driven
semantic segmentation model generates text-based masks for identifying
landmarks and terrain types in images. By translating 2D image points to the
vehicle's motion plane using camera parameters, an MPC controller can guides
the vehicle towards the desired terrain. This approach enhances adaptation to
diverse environments and facilitates the use of high-level instructions for
navigating complex and challenging terrains.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要