There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.
Abstract
<p class="first" id="d37255483e118">Similar to real snakes in nature, the flexible
trunks of snake-like robots enhance
their movement capabilities and adaptabilities in diverse environments. However, this
flexibility corresponds to a complex control task involving highly redundant degrees
of freedom, where traditional model-based methods usually fail to propel the robots
energy-efficiently and adaptively to unforeseeable joint damage. In this work, we
present an approach for designing an energy-efficient and damage-recovery slithering
gait for a snake-like robot using the reinforcement learning (RL) algorithm and the
inverse reinforcement learning (IRL) algorithm. Specifically, we first present an
RL-based controller for generating locomotion gaits at a wide range of velocities,
which is trained using the proximal policy optimization (PPO) algorithm. Then, by
taking the RL-based controller as an expert and collecting trajectories from it, we
train an IRL-based controller using the adversarial inverse reinforcement learning
(AIRL) algorithm. For the purpose of comparison, a traditional parameterized gait
controller is presented as the baseline and the parameter sets are optimized using
the grid search and Bayesian optimization algorithm. Based on the analysis of the
simulation results, we first demonstrate that this RL-based controller exhibits very
natural and adaptive movements, which are also substantially more energy-efficient
than the gaits generated by the parameterized controller. We then demonstrate that
the IRL-based controller cannot only exhibit similar performances as the RL-based
controller, but can also recover from the unpredictable damage body joints and still
outperform the model-based controller, which has an undamaged body, in terms of energy
efficiency. Videos can be viewed at https://videoviewsite.wixsite.com/rlsnake.
</p>