image_pdfimage_print

B.Sc. Thesis: Phillip Overlöpper on An exploration scheme based on the state-action novelty in continuos state-action space

Supervisors: Elmar Rückert, Honghu Xue

Finished: 11. November.2019

Abstract

Exploration in step-based reinforcement learning is a challenging and open problem. If it is applied in a continuous search space, the naive exploration strategy could result in an explored space which is only explored in the neighbourhood of an initial state, leaving a vast amount of entire space unexplored. Visiting states only once leads to poor performance, where the reinforcement learning algorithm gets stuck in a local minimum. This thesis presents a novel exploration scheme for continuous state-action space reinforcement learning, based on the novelty of state-action pairs, where the novelty is measured via the density of the compressed state-action pair. Furthermore, this thesis presents a method to interpolate the action to reach a smooth trajectory in a Markov Decision Process, which can be applied to any robot. The experiment was performed in the CoppeliaSim simulator on a robot with seven degrees of freedom. The results of the new approach show a more effective exploration than the baseline exploration.

Thesis

B.Sc. Thesis: Robin Denz on Complete coverage path planning for low cost robots

Supervisors: Elmar Rückert, Nils Rottmann

Finished: 11. November.2019

Abstract

The demand among the population for household robots continues to rise. These include in particular mobile cleaning and lawn mowing robots. These are usually very expensive and still very inefficient. Especially for lawn mowing robots, it is essential to have visited the entire working space in order to perform their task correctly. However, the current state of the art is still random walk algorithms, which are very unreliable and inefficient. The present bachelor thesis therefore presents a method for intelligent path planning for mobile ”low cost”robots using a lawn mower robot. The robot is only equipped with binary sensors to detect its position in its working space, which is fraught with high uncertainties. By an intelligent representation of the already visited working space as well as the path planning inspired by neural networks, the lawn mower robot manages to achieve a decisive improvement in efficiency compared to the random walk.

Thesis

M.Sc. Thesis: Franz Johannes Michael Werner on HIBO: Hierachical Acquisition Functions for Bayesian Optimization

Supervisors: Elmar Rückert, Nils Rottmann

Finished: 17.Juni.2019

Abstract

Bayesian Optimization is a powerful method to optimize black-box derivative-free functions, with high evaluation costs. For instance, applications can be found in the context of robotics, animation design or molecular design. However, Bayesian Optimization is not able to scale into higher dimensions, equivalent to optimizing more than 20 parameters. This thesis introduces HIBO, a new hierarchical algorithm in the context of high dimensional Bayesian Optimization. The algorithm uses an automatic feature generation. The features are used to condition the parameters, to enable faster optimization. The performance of HIBO is compared to existing high dimensional extensions of Bayesian Optimization on three common benchmark functions. Additionally, an air hockey simulation is used to examine the capability in a task-oriented setting. The conducted experiments show that HIBO performs similar to the basic Bayesian Optimization algorithm, independent from the dimensionality of the given problem. Hence, the proposed HIBO algorithm does not scale Bayesian Optimization to higher dimensions.

 

Thesis

M.Sc. Thesis: Viktor Daibert on Automated Real-time 3D Reconstruction on Mobile Devices

Supervisors: Elmar Rückert

Finished: 5. November.2019

Abstract

In this master thesis, a prototype is being developed that uses existing frameworks to make a Smartphone to a pocket 3D scanner. The resulting prototype consists of a smartphone-, server-application, and a database in between. On the server application, point clouds from photos are created in real time. The point clouds and user support, should be used to assist during the photo creation and reconstruction of 3D-models from photos. After an introductory consideration of the theoretical background, common algorithms of 3D reconstruction are examined for their advantages and disadvantages with regard to the computing time and efficiency with the target platform. Based on the results, a combination of MeshLab, OpenMVG, OpenMVE, and SMVS is used to construct the 3D models. On the basis of fault tolerance with unsorted images, AKAZE and DAISY have been selected, which is why they are being supplemented in the OpenMVG application in this work. This work answers the question of How a technological can be implemente.

Thesis

B.Sc. Thesis: Alexander Walter on Machine Learning for plant classification based on chlorophyll detection

Supervisors: Elmar Rückert, Nils Rottmann

Finished: 17.Juni.2019

Abstract

Based on the intention to build an autonomous lawn mower robot, this work examines the viability of a sensor and microprocessor for onboard plant classification using machine learning. Usually, some sort of fencing is required to keep the robot in its intended processing area, so such a sensor would allow the robot to differentiate between grass and e.g. flowers. Also, the drive and blade speed can be adjusted for certain species or plant densities, etc.
For this, a data set was collected utilizing a specific method called chlorophyll fluorescence induction. A series of narrowband LEDs are used to drive this process, while a spectrometer measures the spectral intensity. The plant will fluoresce in specific wavelengths when sufficiantly illuminated. With this data, machine learning algorithms are trained to explore if they are capable to classify these plants without further information.
With accuracies up to 98% for three plants commonly present on a lawn and up to 86% for eight plants, the results show that chlorophyll fluorescence is a viable method for classification, even under sunlight using the random forest machine learning algorithm

Thesis

B.Sc. Project: Jan Uhlenberg on Development of a High-performance and low-cost fraction collector

Supervisors: Elmar Rückert, Wolfgang Risler, Nils Rottmann

Finished: 2019

Abstract

Preparative high-performance liquid chromatography is a chemical procedure in which a components mixture is separated into its components. In this procedure, fraction collectors are used to fill the separated components of a mixture into their dedicated vials. Fraction collectors available on the marked are usually made of a multiplicity of custom-made parts, which results in high manufacturing costs. A wide used drive concept for fraction collectors is the spindle drive, which is slow due to its gear ration. In this work, we propose an approach of building a fraction collector using a fast timing belt drive, which has performed well for 3D-Printers in practice. The result of a first prototype showed, that this drive-concept can be adapted also for fraction collectors. Thereby, the material and manufacturing costs can be kept low using a manufacturing aware part design.

Paper

B.Sc. Thesis: Tolga-Can Çallar on Design of a Simulation Framework for the Exploration of Kinematic Structures for Robotic Ultrasound Imaging

Supervisors: Elmar Rückert

Finished: 11.November.2018

Abstract

As an versatile, fast and innocuous imaging technique, medical ultrasound is wellestablished in clinical practise. In comparison to other imaging techniques, diagnostic success is highly user-dependent. Due to limited access to qualified medical personnel, the potential of medical ultrasound has not been exploited fully. Over the last three decades, this problem has been addressed by several research groups, that established the field of robotic ultrasound imaging. Although numerous findings have been made, to this day there is no robotic system fit for widespread clinical use. A significant reason for this has been an insuficcient kinematic design. Therefore kinematic optimization is one of the main tasks in the field of robotic ultrasound imaging. In this thesis we present a simulation framework, that can be utilized to explore the fitness of a certain kinematic structure concerning ultrasound imaging

Thesis

M.Sc. Thesis: Denny Dittmar on Distributed Reinforcement Learning with Neural Networks for Robotics

Supervisors: Elmar Rückert, Prof. Dr. Jan Peters

Finished: 2.Januar.2018

Abstract

We propose and investigate a novel type of parameterized policy based on a two-layered stochastic spiking neural network consisting of multiple populations of stochastic spiking neurons. We show that the proposed policy type is a spatially distributed generalization of a discrete basic policy with lookup table parameterization. Our policy reveals remarkable capabilities but also crucial limitations. In particular, our policy is able to deal with high-dimensional state and action spaces but loses expressive power such that it cannot represent certain functions like XOR.
Furthermore, we propose corresponding reinforcement learning methods to train the policy. These methods are based on value function methods and generalize these to train our distributed policy type. We compare these methods to state-ofthe-art methods including black-box approaches and likelihood ratio approaches. It turns out that our proposed methods outperform these methods significantly. In our experiments we demonstrate that our policy can be trained effectively from rewards to guide a 10-link robot-arm in a toy task through a grid world to reach a specified target without hitting obstacles.

Thesis

B.Sc. Thesis: Viktor Nikolas Pfanschilling on Self-Programming Mutation and Crossover in Genetic Programming for Code Generation

Supervisors: Elmar Rückert, Prof. Dr. Jan Peters

Finished: 22.Oktober.2017

Abstract

In Genetic programming, the genetic operators mutation and crossover are defined by the developer in order to produce a system that automatically generates computer code, usually incorporating information about the target language as well as about the problem. The genetic operators are a critical design decision because they control the evolution of genomes. This thesis outlines a way of eliminating that design decision by parameterizing genetic operators as part of the genome. In doing so, we discovered that several measures are necessary to accommodate the lack of assumptions that can be made about genomes. Among these measures are the introduction of an ancestry tree of genomes as additional bookkeeping to enable backtracking, the employment of a method for selection of candidates from that structure and the manual implementation of the first genome.
In our experiments, we discovered that -when uncompensated for- our approach will tend to bloat the genome extremely and distort our metrics. The cause of this was that individuals changed their own code to better fit the way their code generator works. We were able to solve this though, by employing only crossover, thus shifting the responsibility of improvement solely on the code generator’s behavior. Our variant of crossover passes one genome’s code to another genomes mutation function. We also discovered that the exact implementation of the initial genome is extremely important and seemingly inconsequential changes like removing a few lines of dead code make a huge difference. The results show that our current setup is still insufficient but demonstrates that our measures are effective.

Thesis

B.Sc. Thesis: Simon-Konstantin Thiem on Simulation of the Underactuated Sake Robotics Gripper in V-REP and ROS

Supervisors: Elmar Rückert, Prof. Dr. Jan Peters

Finished: 28.September.2017

Abstract

The Sake Robotics Gripper is a cheap, robust and versatile gripper that has not been simulated yet. It is an underactuated gripper and therefore needs a workaround to achieve realistic behavior in simulation. The model has to be able to interpret the exact same ROS messages the gripper receives. This paper proposes a reproduction of the Sake Robotics Gripper in V-REP. We analyze the tools provided by V-REP to develop an algorithm to achieve underactuation in simulation. This contributes to the kitchen cleaning task of the GOAL-Robots project with a platform to do machine learning on. All models are available open source. Our model can be used as a foundation for research in complex grasping and manipulation tasks with the Sake Robotics Gripper

Thesis