Geometric Fabrics: a Safe Guiding Medium for Policy Learning
arxiv(2024)
摘要
Robotics policies are always subjected to complex, second order dynamics that
entangle their actions with resulting states. In reinforcement learning (RL)
contexts, policies have the burden of deciphering these complicated
interactions over massive amounts of experience and complex reward functions to
learn how to accomplish tasks. Moreover, policies typically issue actions
directly to controllers like Operational Space Control (OSC) or joint PD
control, which induces straightline motion towards these action targets in task
or joint space. However, straightline motion in these spaces for the most part
do not capture the rich, nonlinear behavior our robots need to exhibit,
shifting the burden of discovering these behaviors more completely to the
agent. Unlike these simpler controllers, geometric fabrics capture a much
richer and desirable set of behaviors via artificial, second order dynamics
grounded in nonlinear geometry. These artificial dynamics shift the
uncontrolled dynamics of a robot via an appropriate control law to form
behavioral dynamics. Behavioral dynamics unlock a new action space and safe,
guiding behavior over which RL policies are trained. Behavioral dynamics enable
bang-bang-like RL policy actions that are still safe for real robots, simplify
reward engineering, and help sequence real-world, high-performance policies. We
describe the framework more generally and create a specific instantiation for
the problem of dexterous, in-hand reorientation of a cube by a highly actuated
robot hand.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要