A Survey on Policy Search for Robotics

Abstract

<p>Policy search is a subfield in reinforcement learning which focuses onfinding good parameters for a given policy parametrization. It is wellsuited for robotics as it can cope with high-dimensional state and actionspaces, one of the main challenges in robot learning. We review recentsuccesses of both model-free and model-based policy search in robotlearning.Model-free policy search is a general approach to learn policiesbased on sampled trajectories. We classify model-free methods based ontheir policy evaluation strategy, policy update strategy, and explorationstrategy and present a unified view on existing algorithms. Learning apolicy is often easier than learning an accurate forward model, and,hence, model-free methods are more frequently used in practice. However,for each sampled trajectory, it is necessary to interact with the* Both authors contributed equally.robot, which can be time consuming and challenging in practice. Modelbasedpolicy search addresses this problem by first learning a simulatorof the robot’s dynamics from data. Subsequently, the simulator generatestrajectories that are used for policy learning. For both modelfreeand model-based policy search methods, we review their respectiveproperties and their applicability to robotic systems.</p>

Keywords

Affiliated Institutions

• Technical University of Darmstadt DE

Related Publications