Bi-KVIL: Keypoints-based Visual Imitation Learning of Bimanual Manipulation Tasks
CoRR(2024)
摘要
Visual imitation learning has achieved impressive progress in learning
unimanual manipulation tasks from a small set of visual observations, thanks to
the latest advances in computer vision. However, learning bimanual coordination
strategies and complex object relations from bimanual visual demonstrations, as
well as generalizing them to categorical objects in novel cluttered scenes
remain unsolved challenges. In this paper, we extend our previous work on
keypoints-based visual imitation learning ()
to bimanual manipulation tasks. The proposed Bi-KVIL jointly extracts so-called
Hybrid Master-Slave Relationships (HMSR) among objects and hands,
bimanual coordination strategies, and sub-symbolic task representations. Our
bimanual task representation is object-centric, embodiment-independent, and
viewpoint-invariant, thus generalizing well to categorical objects in novel
scenes. We evaluate our approach in various real-world applications, showcasing
its ability to learn fine-grained bimanual manipulation tasks from a small
number of human demonstration videos. Videos and source code are available at
https://sites.google.com/view/bi-kvil.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要