In this project we set up the AMARSi Oncilla Simulator1 and used Dynamic movement primitivies (DMPs) as movement representation and optimized their parameters in a reinforcement learning framework to adapt the robot’s behaviour to new problems. After some experiments on toy examples we applied an open-loop control scheme to the Oncilla Simulator. In the end we want to apply this approach to a real robot, the AMARSi2 Oncilla quadroped and evaluate its performance.

Motor planning algorithms are essential for the development of robust autonomous robot systems. Various approaches exist to compute movement trajectories efficiently by applying quadratic control costs. However, with quadratic costs hard constraints cannot be adequately modelled. In this thesis I choose the Monte Carlo (MC) sampling approach to investigate how dynamic motor planning tasks, considering hard constraints can be solved efficiently. For efficient sampling, Gibbs sampling, rejection sampling, and importance sampling are combined. Two different sampling methods are investigated. The first and simpler method does not consider the dynamic state transition model of a robot. The second method is more sophisticated and considers a linearised approximation of this dynamic model. The experiments range from simple tasks on a 2-link robot arm to tasks using a more complex 4-link robot arm. To enhance the performance of the investigated methods, they are extended by a via point approach. Finally, in a novel trajectory mixing approach complex planning scenarios are solved by mixing multiple trajectories, which are computed in parallel.

In this thesis I investigate three probabilistic regression models from the literature and try to use them for learning the dynamics model of robots, i.e. the function that describes the transition between two states when executing a certain action. I add regularization terms to the error function where necessary to address the problem of overfitting and numerical instability and derive the corresponding learning rules. Then I evaluate these models on four simulated robotic tasks using my MATLAB implementation of the models. A simple one-dimensional toy task is used to analyze and visualize the characteristics of the approaches. In a second and third experiment I test these models on multi-link arms and analyze their robustness to noise. Finally, in the most complex experiment I use an existing simplified model of a humanoid robot in a balancing task. The learned dynamics model is used in combination with a well-known movement planning algorithm to solve a balancing problem, where the robot gets pushed with different forces. The results demonstrate that it is possible to predict and plan movement using these models. The Hierarchical Mixtures of Experts model shows the worst performance, against the expectation to work best on high dimensional data because it has the least number of model parameters. However, this can be improved using different initialization approaches. The Gated Linear Regression with Gaussian Noise Model and Conditioned Mixture of Gaussian show good performance on small datasets. Additionally, the Conditioned Mixture of Gaussian has the advantage of training both the kinematics and inverse kinematic model at the same time, without overhead compared to the other models.

Stochastic optimal control methods are often applied to motor control problems but have disadvantages in special situations. This paper evaluates the efficiency of sample based methods instead of stochastic optimal control methods solving motor planning problems. Standard methods such as Gibbs sampling, rejection sampling and importance sampling are applied on simple tasks. The experiments reveal specific advantages and disadvantages depending on the applied methods. Importance sampling guarantees a smooth state transition trajectory but probably violates defined boundaries. By contrast, rejection sampling generates a more scattered trajectory, however defined boundaries are not exceeded.

Viele Probleme in der Robotik, wie zum Beispiel das Generieren von Bewegungen, die Navigation durch ein Gebäude oder das logische Planen von Aktionen können als Parametersuchprobleme formuliert werden. Dazu definiert man für diese Aktionen eine Reihe von Parametern, wobei die Suche nach einem guten Parameterset zur Aufgabe von Suchalgorithmen gemacht werden kann. Dies ist eine komplexe Aufgabe und entspricht typischerweise einer Suche im hochdimensionalen Raum (z.B.: 10 Parameter = 10¹⁰ dimensionalen Raum). Derzeit gibt es verschiedene Algorithmen um Parameter im hochdimensionalen Raum zu suchen und zu optimieren.
Die hier vorliegende Arbeit soll einen Vergleich dreier Algorithmen, CMA-ES, Particle Swarm Optimization sowie SIMPLEX geben. Ziel dieser Arbeit ist es vorhandene Algorithmen in Hinsicht auf Lerngeschwindigkeit, Qualität der Lösung und Anwendbarkeit zu prüfen. Die Algorithmen sollen Parameter finden, sodass ein simuliertes Modell eines Schwimmers möglichst schnell von einem Punkt zu einem anderen schwimmt. Wir werden hier zuerst näher auf die Aufgabenstellung eingehen und werden zeigen mit welchen Problemen man konfrontiert ist, wenn die Suche sich über einen hochdimensionalen Raum erstreckt. Weiters wird dann verdeutlicht, dass es mithilfe der Algorithmen verschieden-gut und verschieden-schnell möglich ist, solche Parameter zu finden.