image_pdfimage_print

M.Sc. Thesis: Rui Song on Solving Visual Navigation Tasks for Pedestrian Trajectory Generation Using Distributional Reinforcement Learning and Automatic Curriculum Learning in CARLA

Supervisors: Honghu Xue, Elmar Rückert

Finished: 22.April 2022

Abstract

In this thesis, we propose an approach that combines reinforcement learning and automatic curriculum learning to solve a visual navigation task. A pedestrian agent is expected to learn a policy from scratch in a street-crossing scenario in a realistic traffic simulator CARLA. For this, the pedestrian is restricted to its first-person perspective as sensory input. The pedestrian cannot obtain full knowledge of the environment, which raises a partial observability challenge. To achieve this, an improved version of the Distributional Soft Actor-Critic algorithm is implemented. The algorithm adopts a newly proposed 3D dilated convolutional architecture to deal with the partial observability problem. To further improve its performance, we develop an automatic curriculum learning algorithm called NavACL+ on top of NavACL. As suggested in the results and ablation studies, our approach outperforms the original NavACL by 23.1%. Additionally, the convergence speed of NavACL+ is also observed to be 37.5% quicker. Moverover, the validation results show that the trained policies of NavACL+ are much more generalizable and robust than other variants in terms of different initial starting poses. NavACL+ policies perform 28.3% better than other policies training from a fixed start.

Thesis

B.Sc. Thesis: Fritze Clemens on Mobile Robot Teleoperation in ROS for Basic SLAM Application

Supervisor: Linus Nwankwo, M.Sc.;
Univ.-Prof. Dr Elmar Rückert
Start date: 10th January 2022

 

Theoretical difficulty: mid
Practical difficulty: mid

Abstract

Nowadays, robots used for survey of indoor and outdoor environments are either teleoperated in fully autonomous mode where the robot makes a decision by itself and have complete control of its actions; semi-autonomous mode where the robot’s decisions and actions are both manually (by a human) and autonomously (by the robot) controlled; and in full manual mode where the robot actions and decisions are manually controlled by humans. In full manual mode, the robot can be operated using a teach pendant, computer keyboard, joystick, mobile device, etc.

Although the Robot Operating System (ROS) has provided roboticists with easy and efficient tools to teleoperate or command robots with both hardware and software compatibility on the ROS framework, however, there is a need to provide an alternative approach to encourage a non-robotic expert to interact with a robot. The human hand-gesture approach does not only enables the robot users to teleoperate the robot by demonstration but also enhances user-friendly interaction between robots and humans.

In the context of this thesis, the application of human hand gestures is proposed to teleoperate our mobile robot using embedded computers and inertial measurement sensors. First, we will teleoperate the robot on the ROS platform and then via hand gestures, leveraging on the framework developed by [1] and [2].

Tentative Work Plan

To achieve our aim, the following concrete tasks will be focused on:

  • Design and generate the Unified Robot Description Format (URDF) for the mobile robot
  • Simulate the mobile robot within the ROS framework (Gazebo, Rviz)
  • Set up the interfaces and serial connection between ROS and the robot control devices
  • Develop an algorithm to teleoperate the robot in the ROS using hand gesture
  • Use the algorithm to perform Simultaneous Localization and Mapping (SLAM) for indoor applications (simulation only)

References

[1] Nils Rottmann et al,  https://github.com/ROS-Mobile/ROS-Mobile-Android, 2020

[2] Wei Zhang, Hongtai Cheng, Liang Zhao, Lina Hao, Manli Tao and ChaoqunXiang, “A gesture-based Teleoperation System for Compliant Robot Motion“, Appl. Sci. 20199(24), 5290; https://doi.org/10.3390/app9245290

Thesis Document

B.Sc. Thesis: Tolga-Can Callar on Learning of Inverse Dynamics for Proprioceptive Force Estimation during Irregular Fine-Scale Robot Motion

Supervisors: Sven Böttger, Elmar Rückert

Finished: 21.September 2021

Abstract

The applicability of robotic automation has transcended the industrial domain through the emergence of collaborative robotics and is increasingly entering the realm of applications with high levels of physical human-robot interactions. This is concomitant with a paradigm shift towards higher force control sensitivity to accomplish functional and safety requirements concerning the regulation of contact forces between robots and humans. A fundamental challenge in this regard is the observability and estimation of interaction forces. Utilizing the availability of joint position and torque sensors in recent collaborative robot models that yield a larger perceptive field for interaction forces than local force sensors, a proprioceptive approach is taken in this thesis to develop inverse dynamic models to estimate dynamic disturbances and determine external interaction forces during fine-scale motion. A series of state-of-the-art techniques are implemented and evaluated on the KUKA LBR iiwa 14, including dynamic parameter identification, neural-network based single-step, and time-series models, and a novel hybrid architecture combining a rigid body dynamics model with downstream neural networks and joint rotational displacement encodings. The results indicate that significant improvements in torque and force estimation accuracy can be obtained by the proposed method when compared with conventional rigid body dynamics models or neural networks alone.

Thesis

M.Sc. Thesis: Nikolaus Feith on A Motor Control Learning Framework for Cyber Physical Systems

Supervisor: Univ.-Prof. Dr Elmar Rückert
Start date: 1st of July 2021

Theoretical difficulty: mid
Practical difficulty: mid

Abstract

 A central problem in robotics is the description of the movement of a robot. This task is complex, especially for robots with high degrees of freedom. In the case of complex movements, they can no longer be programmed manually. Instead, they are taught to the robot utilizing machine learning. The Motor Control Learning framework presents an easy-to-use method for generating complex trajectories. Dynamic Movement Primitives is a method for describing movements as a non-linear dynamic system. Here, the trajectories are modelled by weighted basis functions, whereby the machine learning algorithms must determine only the respective weights. Thus, it is possible for complex movements to be defined by a few parameters. As a result, two motion learning methods were implemented. When imitating motion demonstrations, the weights are determined using regression methods. A reinforcement learning algorithm is used for policy optimization to generate waypoint trajectories. For this purpose, the weights are improved iteratively through a cost function using the covariance matrix adaptation evolution strategy. The generated trajectories were evaluated in experiments. 

Thesis Document

B.Sc. Thesis: Leander Busch on Learning Motion Models for Local Path Planning Strategies

Supervisors: Elmar Rückert, Nils Rottmann

Finished: 15.Februar.2021

Abstract

The Segway Loomo is a self-balancing segway robot, which is constantly balanced by an internal control system. A local path planning strategy was developed in advance for this robot. For local path planning, a motion model of the robot is needed to determine the effect of velocity commands on the robot’s pose. In the implemented local path planner, a simple motion model of the robot is used, which does not model the effect of the segway robot’s internal control on its motion. In this work, it was investigated whether a more accurate motion model for the Segway Loomo robot can be learned by using artificial neural networks to improve the local path planning for this robot. For this purpose, different architectures of feedforward networks were tested. The neural networks were trained and evaluated using recorded motion data of the segway robot. The best learned model was validated by using a standard differential drive motion model as a reference. For the validation of the learned model, the accuracy of both motion models was examined on the recorded motion data. On average, the learned model is 59.48 % more accurate in determining the position of the robot at the next time step and 24.61 % more accurate in determining the new orientation of the robot than the differential drive motion model.

Thesis

B.Sc. Thesis: Phillip Overlöpper on An exploration scheme based on the state-action novelty in continuos state-action space

Supervisors: Elmar Rückert, Honghu Xue

Finished: 11. November.2019

Abstract

Exploration in step-based reinforcement learning is a challenging and open problem. If it is applied in a continuous search space, the naive exploration strategy could result in an explored space which is only explored in the neighbourhood of an initial state, leaving a vast amount of entire space unexplored. Visiting states only once leads to poor performance, where the reinforcement learning algorithm gets stuck in a local minimum. This thesis presents a novel exploration scheme for continuous state-action space reinforcement learning, based on the novelty of state-action pairs, where the novelty is measured via the density of the compressed state-action pair. Furthermore, this thesis presents a method to interpolate the action to reach a smooth trajectory in a Markov Decision Process, which can be applied to any robot. The experiment was performed in the CoppeliaSim simulator on a robot with seven degrees of freedom. The results of the new approach show a more effective exploration than the baseline exploration.

Thesis

B.Sc. Thesis: Robin Denz on Complete coverage path planning for low cost robots

Supervisors: Elmar Rückert, Nils Rottmann

Finished: 11. November.2019

Abstract

The demand among the population for household robots continues to rise. These include in particular mobile cleaning and lawn mowing robots. These are usually very expensive and still very inefficient. Especially for lawn mowing robots, it is essential to have visited the entire working space in order to perform their task correctly. However, the current state of the art is still random walk algorithms, which are very unreliable and inefficient. The present bachelor thesis therefore presents a method for intelligent path planning for mobile ”low cost”robots using a lawn mower robot. The robot is only equipped with binary sensors to detect its position in its working space, which is fraught with high uncertainties. By an intelligent representation of the already visited working space as well as the path planning inspired by neural networks, the lawn mower robot manages to achieve a decisive improvement in efficiency compared to the random walk.

Thesis

M.Sc. Thesis: Franz Johannes Michael Werner on HIBO: Hierachical Acquisition Functions for Bayesian Optimization

Supervisors: Elmar Rückert, Nils Rottmann

Finished: 17.Juni.2019

Abstract

Bayesian Optimization is a powerful method to optimize black-box derivative-free functions, with high evaluation costs. For instance, applications can be found in the context of robotics, animation design or molecular design. However, Bayesian Optimization is not able to scale into higher dimensions, equivalent to optimizing more than 20 parameters. This thesis introduces HIBO, a new hierarchical algorithm in the context of high dimensional Bayesian Optimization. The algorithm uses an automatic feature generation. The features are used to condition the parameters, to enable faster optimization. The performance of HIBO is compared to existing high dimensional extensions of Bayesian Optimization on three common benchmark functions. Additionally, an air hockey simulation is used to examine the capability in a task-oriented setting. The conducted experiments show that HIBO performs similar to the basic Bayesian Optimization algorithm, independent from the dimensionality of the given problem. Hence, the proposed HIBO algorithm does not scale Bayesian Optimization to higher dimensions.

 

Thesis

M.Sc. Thesis: Viktor Daibert on Automated Real-time 3D Reconstruction on Mobile Devices

Supervisors: Elmar Rückert

Finished: 5. November.2019

Abstract

In this master thesis, a prototype is being developed that uses existing frameworks to make a Smartphone to a pocket 3D scanner. The resulting prototype consists of a smartphone-, server-application, and a database in between. On the server application, point clouds from photos are created in real time. The point clouds and user support, should be used to assist during the photo creation and reconstruction of 3D-models from photos. After an introductory consideration of the theoretical background, common algorithms of 3D reconstruction are examined for their advantages and disadvantages with regard to the computing time and efficiency with the target platform. Based on the results, a combination of MeshLab, OpenMVG, OpenMVE, and SMVS is used to construct the 3D models. On the basis of fault tolerance with unsorted images, AKAZE and DAISY have been selected, which is why they are being supplemented in the OpenMVG application in this work. This work answers the question of How a technological can be implemente.

Thesis

B.Sc. Thesis: Alexander Walter on Machine Learning for plant classification based on chlorophyll detection

Supervisors: Elmar Rückert, Nils Rottmann

Finished: 17.Juni.2019

Abstract

Based on the intention to build an autonomous lawn mower robot, this work examines the viability of a sensor and microprocessor for onboard plant classification using machine learning. Usually, some sort of fencing is required to keep the robot in its intended processing area, so such a sensor would allow the robot to differentiate between grass and e.g. flowers. Also, the drive and blade speed can be adjusted for certain species or plant densities, etc.
For this, a data set was collected utilizing a specific method called chlorophyll fluorescence induction. A series of narrowband LEDs are used to drive this process, while a spectrometer measures the spectral intensity. The plant will fluoresce in specific wavelengths when sufficiantly illuminated. With this data, machine learning algorithms are trained to explore if they are capable to classify these plants without further information.
With accuracies up to 98% for three plants commonly present on a lawn and up to 86% for eight plants, the results show that chlorophyll fluorescence is a viable method for classification, even under sunlight using the random forest machine learning algorithm

Thesis