A Fast Hierarchical Learning Approach for Autonomous Robots

In this article we present an approach that enables robots to learn how to act and react robustly in continuous and noisy environments while not loosing track of the overall feasibility, i.e. minimising the execution time in order to keep up continuous learning. We do so by combining reinforcement learning mechanisms with techniques belonging to the field of multivariate statistics on three different levels of abstraction: the motivation layer and the two simultaneously learning strategy and skill layers. The motivation layer allows for modelling occasionally contradicting goals in terms of drives in a very intuitive fashion. A drive represents one single goal, that a robot wants to be satisfied, like charging its battery, when it is nearly exhausted, or transporting an object to a target position. The strategy layer encapsulates the main reinforcement learning algorithm based on an abstracted and dynamically adjusted Markovian state space. By means of state abstraction, we minimise the overall state space size in order to ensure feasibility of the learning process in a dynamically changing environment. The skill layer finally realises a generalised learning method for learning reactive low-level behaviours, that enable a robot to interact with the environment.