Supervisor: Linus Nwankwo, M.Sc.;
Univ.-Prof. Dr Elmar Rückert
Start date: As soon as possible

Theoretical difficulty: mid
Practical difficulty: high

Abstract

Unlike the traditional mine inspection approach, which is inefficient in terms of time, terrain, and coverage, this project/thesis aims to investigate novel 3D perception and SLAM using geometric and semantic information for real-time mine inspection.

We propose to develop a SLAM approach that takes into account the terrain of the mining site and the sensor characteristics to ensure complete coverage of the environment while minimizing traversal time.

Tentative Work Plan

To achieve our objective, the following concrete tasks will be focused on:

• Study the concept of 3D perception and SLAM for mine inspection, as well as algorithm development, system integration and real-world demonstration using Unitree Go1 quadrupedal robot.

• Setup and familiarize with the simulation environment:
  • Build the robot model (URDF) for the simulation (optional if you wish to use the existing one)
  • Setup the ROS framework for the simulation (Gazebo, Rviz)
  • Recommended programming tools: C++, Python, Matlab
• Develop a novel SLAM system for the quadrupedal robot to navigate, map and interact with challenging real-world environments:

  • 2D/3D mapping in complex indoor/outdoor environments

  • Localization using either Monte Carlo or extended Kalman filter

  • Complete coverage path-planning

• Intermediate presentation:
  • Presenting the results of the literature study
  • Possibility to ask questions about the theoretical background
  • Detailed planning of the next steps

• Implementation:

  • Simulate the achieved results in a virtual environment (Gazebo, Rviz, etc.)

  • Real-time testing on Unitree Go1 quadrupedal robot.

• Evaluate the performance in various challenging real-world environments, including outdoor terrains, urban environments, and indoor environments with complex structures.
• M.Sc. thesis or research paper writing (optional)

Related Work

[1]  Wolfram Burgard, Cyrill Stachniss, Kai Arras, and Maren Bennewitz , ‘SLAM: Simultaneous
Localization and Mapping’,  http://ais.informatik.uni-freiburg.de/teaching/ss12/robotics/slides/12-slam.pdf

[2]  V.Barrile, G. Candela, A. Fotia, ‘Point cloud segmentation using image processing techniques for structural analysis’, The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2/W11, 2019 

[3]  Łukasz Sobczak , Katarzyna Filus , Adam Domanski and Joanna Domanska, ‘LiDAR Point Cloud Generation for SLAM Algorithm Evaluation’, Sensors 2021, 21, 3313. https://doi.org/10.3390/ s21103313.

Supervisors: Univ. -Prof. Dr. Elmar Rückert  and Nwankwo Linus M.Sc.

Theoretical difficulty: mid
Practical difficulty: mid

Naturally, humans have the ability to give directions (go front, back, right, left etc) by merely pointing fingers towards the direction in question. This can be done effortlessly without saying a word. However, mimicking or training a mobile robot to understand such gestures is still today an open problem to solve.
In the context of this thesis, we propose finger-pose based mobile robot navigation to maximize natural human-robot interaction. This could be achieved by observing the human fingers’ Cartesian  pose from an

RGB-D camera and translating it to the robot’s linear and angular velocity commands. For this, we will leverage computer vision algorithms and the ROS framework to achieve the objectives.
The prerequisite for this project are basic understanding of Python or C++ programming, OpenCV and ROS.

Tentative work plan

In the course of this thesis, the following concrete tasks will be focused on:

• study the concept of visual navigation of mobile robots
• develop a hand detection and tracking algorithm in Python or C++
• apply the developed algorithm to navigate a simulated mobile robot
• real-time experimentation
• thesis writing

References

1. Shuang Li, Jiaxi Jiang, Philipp Ruppel, Hongzhuo Liang, Xiaojian Ma,
   Norman Hendrich, Fuchun Sun, Jianwei Zhang,  “A Mobile Robot Hand-Arm Teleoperation System by Vision and IMU“,  IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS),  October 25-29, 2020, Las Vegas, NV, USA.
2.  

Supervisor: Linus Nwankwo, M.Sc.;
Vedant Dave M.Sc.;
Univ.-Prof. Dr Elmar Rückert
Start date: 1st June 2023

Theoretical difficulty: mid
Practical difficulty: mid

Abstract

For more than two decades now, the technique of simultaneous localization and mapping (SLAM) has served as a cornerstone in achieving goals related to autonomous navigation.

The core essence of the SLAM problem lies in the creation of an environmental map while concurrently estimating the robot’s relative position to this map. This task is undertaken with the aid of sensor observations and control data, both of which are subject to noise.

In recent times, a shift towards a mapless-based approach employing deep reinforcement learning has emerged. In this innovative methodology, the agent, in this case a robot, learns the navigation policy. This learning process is driven solely by sensor data and control data, effectively bypassing the need for a prior map of the task environment. In the scope of this dissertation, we will conduct a comprehensive performance evaluation of both the traditional SLAM and the emerging mapless-based approach. We’ll utilize a dynamic, crowded environment as our test bed, and the open-source open-shuttle mobile robot with a differential drive will serve as our experimental subject.

Tentative Work Plan

To achieve our objective, the following concrete tasks will be focused on:

• Literature research and field familiarization
  • Mobile robotics and industrial use cases
  • Overview of map-based autonomous navigation (SLAM & Path planning)
  • Overview of mapless-based autonomous navigation approach with deep reinforcement learning
• Setup and familiarize with the simulation environment
  • Build the robot model (URDF) for the simulation (optional if you wish to use the existing one)
  • Setup the ROS framework for the simulation (Gazebo, Rviz)
  • Recommended programming tools: C++, Python, Matlab
• Intermediate presentation:
  • Presenting the results of the literature study
  • Possibility to ask questions about the theoretical background
  • Detailed planning of the next steps
• Define key performance/quality metrics for evaluation:
  • Time to reach the desired goal
  • Average/mean speed
  • Path smoothness
  • Obstacle avoidance/distance to obstacles
  • Computational requirement
  • Success rate
  • Other relevant parameters
• Assessment and execution:
  • Compare the results from both map-based and map-less approaches on the above-defined evaluation metrics.
• Validation:
  • Validate both approaches in a real-world scenario using our open-source open-shuttle mobile robot.
• Furthermore, the following optional goals are planned:
  • Develop a hybrid approach combining both the map-based and the map-less methods.
• M.Sc. thesis writing
• Research paper writing (optional)

Related Work

[1] Xue, Honghu; Hein, Benedikt; Bakr, Mohamed; Schildbach, Georg; Abel, Bengt; Rueckert, Elmar, “Using Deep Reinforcement Learning with Automatic Curriculum Learning for Mapless Navigation in Intralogistics“, In: Applied Sciences (MDPI), Special Issue on Intelligent Robotics, 2022.

[2] Han Hu; Kaicheng Zhang; Aaron Hao Tan; Michael Ruan; Christopher Agia; Goldie Nejat “Sim-to-Real Pipeline for Deep Reinforcement Learning for Autonomous Robot Navigation in Cluttered Rough Terrain”,  IEEE Robotics and Automation Letters ( Volume: 6, Issue: 4, October 2021).

[3] Md. A. K. Niloy; Anika Shama; Ripon K. Chakrabortty; Michael J. Ryan; Faisal R. Badal; Z. Tasneem; Md H. Ahamed; S. I. Mo, “Critical Design and Control Issues of Indoor Autonomous Mobile Robots: A Review”, IEEE Access ( Volume: 9), February 2021.

[4]  Ning Wang, Yabiao Wang, Yuming Zhao, Yong Wang and Zhigang Li , “Sim-to-Real: Mapless Navigation for USVs Using Deep Reinforcement Learning”, Journal of Marine Science and Engineering, 2022, 10, 895. https://doi.org/10.3390/jmse10070895

Abstract

The need for efficient and compact representations of sensory data such as visual and textual has grown significantly due to the exponential growth in the size and complexity of the data. Self-supervised learning techniques, such as autoencoders, contrastive learning, and transformer, have shown significant promise in learning such representations from large unlabeled datasets. This research aims to develop novel self-supervised learning techniques inspired by these approaches to improve the quality and efficiency of unsupervised representation learning.

Description

The study will begin by reviewing the state-of-the-art self-supervised learning techniques and their applications in various domains, including computer vision and natural language processing. Next, a set of experiments will be conducted to develop and evaluate the proposed techniques on standard datasets in these domains.

The experiments will focus on learning compact and efficient representations of sensory data using autoencoder-based techniques, contrastive learning, and transformer-based approaches. The performance of the proposed techniques will be evaluated based on their ability to improve the accuracy and efficiency of unsupervised representation learning tasks.

The research will also investigate the impact of different factors such as the choice of loss functions, model architecture, and hyperparameters on the performance of the proposed techniques. The insights gained from this study will help in developing guidelines for selecting appropriate self-supervised learning techniques for efficient and compact representation learning.

Overall, this research will contribute to the development of novel self-supervised learning techniques for efficient and compact representation learning of sensory data. The proposed techniques will have potential applications in various domains, including computer vision, natural language processing, and other sensory data analysis tasks.

Qualifications

• Currently pursuing a Bachelor’s or Master’s degree in Computer Science,
  Electrical Engineering, Mechanical Engineering, Mathematics, or related
  fields.
• Strong programming skills in Python
• Experience with deep learning frameworks such as PyTorch or TensorFlow.
• Good written and verbal communication skills in English.
• (optional) Familiarity with unsupervised learning techniques such as contrastive learning, self-supervised learning, and generative models

Interested?

If this topic excites you you, please contact Fotios Lygerakis by email at fotios.lygerakis@unileoben.ac.at or simple visit us at our chair in the Metallurgie building, 1st floor.

Abstract

Human-Robot Interaction using Sign Language is a project that aims to revolutionize the way we communicate with machines. With the increasing use of robots in our daily lives, it is important to create a more natural and intuitive way for humans to communicate with them.

Sign language is a unique and powerful form of communication that is widely used by the deaf and hard-of-hearing community. By incorporating sign language into robot interaction, we can create a more inclusive and accessible technology for everyone.

Moreover, sign language will provide a new and innovative way to interact with robots, making it possible for people to control and communicate with them in a way that is both non-verbal and non-intrusive.

Note: This project is also offered as Internship position.

Thesis Description

The implementation of sign language in human-robot interaction will not only improve the user experience but will also advance the field of robotics and artificial intelligence. This project has the potential to bring about a new era of human-robot interaction, where machines and humans can communicate in a more natural and human-like way. Therefore, the Human-Robot Interaction using Sign Language project is a crucial step toward creating a more accessible and user-friendly technology for everyone.

This thesis will encompass three crucial elements. The first part will focus on recognizing human gestures in sign language through the development of deep learning methods utilizing a camera. The second part will involve programming a robotic hand to translate text back into gestures. Finally, the third part will bring together the first two components to create a seamless human-robot interaction framework using sign language.

Qualifications

• Currently pursuing a Bachelor’s or Master’s degree in Computer Science, Electrical Engineering, Mechanical Engineering, Mathematics or related fields.
• Strong programming skills in Python
• Experience with deep learning frameworks such as PyTorch or TensorFlow.
• Experience working with robotics hardware
• Knowledge of computer vision and image processing techniques
• Good written and verbal communication skills in English.

Interested?

If this project sounds like fun to you, please contact Fotios Lygerakis by email at fotios.lygerakis@unileoben.ac.at or simple visit us at our chair in the Metallurgie building, 1st floor.

Supervisors:

• Linus Nwankwo
  
• Konrad Bartsch
• Univ. Prof. Dr Elmar Rückert

Start date: October 2022

Qualifications

• Interest in controlling and simulating mobile robotics
• Interest in Programming in Python and ROS or ROS2

 Keywords: Mobile robot control, robot operating system (ROS), ESP32

Description

The goal of this project or thesis is to develop a control and sensing interface for our mobile robot “RMP220“. The RMP220 has two powerful brush-less motors equipped with two magnetic encoders.

Learn in this project how to read the sensor values and how to control the motors via micro-ros on a ESP32 controller.

Links:

• https://micro.ros.org/blog/2020/08/27/esp32/
• https://answers.ros.org/question/379267/micro-ros-on-esp32/
• https://github.com/micro-ROS/micro_ros_espidf_component
• https://micro.ros.org/docs/tutorials/core/first_application_rtos/

Note: This project is also offered as Internship position.

Single Person Project or Team Work

You may work on the project alone or in teams of up to 4 persons.

For a team work task, the goals will be extended to control the robot via ROS 2 and to simulate it in Gazebo or RViz.

Interested?

If this project sounds like fun to you, please contact Linus Nwankwo or Elmar Rueckert or simply visit us at our chair in the Metallurgie building, 1st floor.

Description

Mixed Reality (AR) interface based on Unity 3D for intuitive programming of robotic manipulators (UR3). The interface will be implemented within on the ROS 2 robotic framework.

Note: This project is also offered as Internship position.

Abstract

Robots will become a necessity for every business in the near future. Especially companies that rely heavily on the constant manipulation of objects will need to be able to constantly repurpose their robots to meet the ever changing demands. Furthermore, with the rise of Machine Learning, human collaborators or ” robot teachers” will need a more intuitive interface to communicate with them, either when interacting with them or when teaching them.

In this project we will develop a novel Mixed (Augmented) Reality Interface for teleoperating the UR3 robotic manipulator. For this purpose we will use AR glasses to augment the user’s reality with information about the robot and enable intuitive programming of the robot. The interface will be implemented on a ROS 2 framework for enhanced scalability and better integration potential to other devices.

Outcomes

This thesis will result to an innovative graphical interface that enables non-experts to program a robotic manipulator.

The student will get valuable experience in the Robot Operating System (ROS) framework and developing graphical interfaces on Unity. The student will also get a good understanding of robotic manipulators (like UR3) and develop a complete engineering project.

Qualifications

• Currently pursuing a Bachelor’s or Master’s degree in Computer Science, Electrical Engineering, Mechanical Engineering or related fields.
• Good programming skills in C# and Unity 3D
• Familiarity with ROS or other robotic frameworks
• Basic knowledge of 3D modeling and animation
• Good written and verbal communication skills in English.
• (optional) Experience with mixed reality development and programming

Interested?

If this project sounds like fun to you, please contact Fotios Lygerakis by email at fotios.lygerakis@unileoben.ac.at or simple visit us at our chair in the Metallurgie building, 1st floor.

Supervisors: Univ.-Prof. Dr Elmar Rückert, LUPA Electronics GmbH
Start date: ASAP from June 2022

Theoretical difficulty: high
Practical difficulty: low

Abstract

Wir Menschen sind in der Lage unter widrigen Bedingungen z.B. bei eingeschränkter Sicht, oder bei Störungen komplexe Vorgänge wahrzunehmen, vorherzusagen und innerhalb von wenigen Millisekunden zusammenhängende Entscheidungen zu treffen. Mit dem zunehmenden Grad der Automatisierung steigen auch die Anforderungen an künstliche Systeme. Immer komplexere und größere Datenmengen müssen verarbeitet werden um autonome Entscheidungen zu treffen. Mit gängigen KI Ansätzen stoßen wir aufgrund der konvergierenden Miniaturisierung an Grenzen, die z.B. im Bereich des autonomen Fahrens nicht ausreichen, um ein sicheres autonomes System zu entwickeln.

Ziel dieser Forschung ist es probabilistische Vorhersagemodelle in massiv parallelisierbaren neuronalen Netzen zu implementieren und mit diesen komplexe Entscheidungen Aufgrund erlernter interner Vorhersagemodelle zu treffen. Die neuronalen Modelle verarbeiten hoch dimensionale Daten moderner und innovativer taktiler und visueller Sensoren. Wir testen die neuronalen Vorhersage und Entscheidungsmodelle in humanoiden Roboteranwendungen in dynamischen Umgebungen.

Unser Ansatz beruht auf der Theorie der probabilistischen Informationsverarbeitung in neuronalen Netzen und unterscheidet sich somit grundlegend von den gängigen Methoden tiefer neuronaler Netze. Die zugrundeliegende Theorie ermöglicht weitreichende Modelleinblicke und erlaubt neben den Vorhersagen von Mittelwerten auch Unsicherheiten und Korrelationen. Diese zusätzlichen Vorhersagen sind entscheidend für verlässliche, erklärbare und robuste künstliche Systeme und sind eines der größten offenen Probleme in der künstlichen Intelligenz Forschung.

Tentative Work Plan

To achieve our aim, the following concrete tasks will be focused on:

• Literature research on graphical model inference of motion plans.
• Toy Task implementation in Python. 
• Implementation of  GMMs, PTSMs and combinations in Python.
• Visualization and analysis of the prediction performance. Definition of suitable evaluation criteria.
• (Optional) Implementation in a realistic driving simulator.
• Analysis and evaluation of the generated data.

Supervisors: Univ.-Prof. Dr Elmar Rückert,
Assoc. Prof. Dr. Susanne Michelic & Assoc. Prof. Dr. Christian Bernhard (Chair of Ferrous Metallurgy),
Markus Brummayer (voestalpine Stahl GmbH)
Start date: ASAP from December 2021

Theoretical difficulty: mid
Practical difficulty: mid

Abstract

In this thesis, the student has the unique opportunity to investigate meniscus level fluctuations in the mold using deep learning approaches at the voestalpine Stahl GmbH in Linz. 

The mold, illustrated in the image above, is equipped with electromagnetic mold level sensors and with temperature image cameras that measure the surface temperature of the casting powder. 

The goal of this thesis is to understand and model the underlying dynamic processes of the meniscus level fluctuations in the mold.

In the thesis black box models as well as gray box models that combine analytic dynamic models with learned  models will be investigated. 

Tentative Work Plan

To achieve our aim, the following concrete tasks will be focused on:

• Literature research on meniscus level fluctuations in the mold.
• Data analysis, filtering, preprocessing, visualization of meniscus level fluctuations data. 
• Implementation of  deep convolutional neural networks (CNN) as low-dimensional feature extractors. Visualization and analysis of the dynamic processes.
• (Optional) Implementation of neural time-series models like LSTMs trained with the computed CNN features.
• Analysis and evaluation of the provided data.

Supervisor: Univ.-Prof. Dr Elmar Rückert, Dr. Petra Spörk-Erdely (Chair of Physical Metallurgy and Metallic Materials)
Start date: ASAP from December 2021

Theoretical difficulty: mid
Practical difficulty: low

Abstract

In the context of this thesis, we propose to apply modern machine learning approaches such as variational autoencoder to visualize and reduce the complexity of X-ray diffraction (XRD) data collected on advanced γ-TiAl based alloys. By classifying XRD data collected during in situ experiments into known phases, we aim at disclosing phase transformation temperatures and selected properties of the individual phases, which are of interest with regard to the current alloy development. Furthermore, the capabilities of the applied machine learning approaches going beyond basic XRD data analysis will be explored.

Sketch of a synchron from synchrotron.org.au, illustrating the process of accelerating electrons at almost the speed of light.   

Illustration of a collected data sample which is a 2D X-ray diffraction of a nominal Ti-44Al-7Mo (in at.%) alloy collected.

Topic and Motivation

Intermetallic γ-TiAl based alloys are a promising class of structural materials for lightweight high-temperature applications. Following intensive research activities, they have recently entered service in the automotive and aircraft engine industries, e.g. as low-pressure turbine blades in environmentally-friendly combustion engine options [1].

During the past decades, the development of these complex multi-phase alloys has been strongly driven by the application of diffraction and scattering techniques [2]. These characterization techniques offer access to the atomic structure of materials and provide insights into a variety of microstructural parameters. High-energy X-rays, such as available at modern synchrotron radiation sources (i.e. large-scale research facilities for X-ray experiments), and recent advances in hardware technology nowadays allow to conduct so-called in situ experiments that reveal at a high temporal resolution the relationship between selected external conditions (e.g. thermal or mechanical load) and structural changes in the material. Various specimen environments can be adjusted to emulate technologically relevant or real-life conditions, addressing a multitude of research topics ranging from fundamental questions in the primary alloy design over process-related to application-related issues. Modern setups at synchrotron sources even allow the investigation of elaborate manufacturing, joining and repair processes in an in situ manner, producing insights that have been inaccessible by means of conventional methods so far.

Advanced materials characterization techniques such as described above are often characterized by an ever-growing data acquisition speed and storage capabilities. While enabling novel insights, they, thus, also pose a serious challenge to modern materials science. In situ synchrotron X-ray diffraction (XRD) experiments usually bring about large sets of two-dimensional diffraction data such as those shown in Figure 1. New procedures are needed to quickly assess and analyze this type of datasets.

Tentative Work Plan

To achieve our aim, the following concrete tasks will be focused on:

• Literature research on state-of-the-art materials characterization methods.
• Implementation of  deep convolutional neural networks (CNN) and Variational Autoencoder in Python/Tensorflow.
• Application and evaluation of variational autoencoder on the CNN features.
• Analysis and Evaluation of the provided synchrotron data.

References

[1] Clemens, S. Mayer, Design, processing, microstructure, properties, and applications of advanced intermetallic TiAl alloys, Advanced Engineering Materials 15 (2013) 191-215, doi: 10.1002/adem.201200231.

[2] Spoerk-Erdely, P. Staron, J. Liu, N. Kashaev, A. Stark, K. Hauschildt, E. Maawad, S. Mayer, H. Clemens, Exploring structural changes, manufacturing, joining, and repair of intermetallic γ-TiAl-based alloys: Recent progress enabled by in situ synchrotron X-ray techniques, Advanced Engineering Materials (2020) 2000947, doi: 10.1002/adem.202000947.