Supervisor: Linus Nwankwo, M.S.c;
Univ.-Prof. Dr Elmar Rückert
Start date: ASAP, e.g., 1st of October 2021

Theoretical difficulty: low
Practical difficulty: high

Introduction

The SLAM problem as described in [3] is the problem of building a map of the environment while simultaneously estimating the robot’s position relative to the map given noisy sensor observations. Probabilistically, the problem is often approached by leveraging the Bayes formulation due to the uncertainties in the robot’s motions and observations. 

SLAM has found many applications not only in navigation, augmented reality, and autonomous vehicles e.g. self-driving cars, and drones but also in indoor & outdoor delivery robots, intelligent warehousing etc. While many possible solutions have been presented in the literature to solve the SLAM problem, in challenging real-world scenarios with features or geometrically constrained characteristics, the reality is far different.

Some of the most common challenges with SLAM are the accumulation of errors over time due to inaccurate pose estimation (localization errors) while the robot moves from the start location to the goal location; the high computational cost for image, point cloud processing and optimization [1]. These challenges can cause a significant deviation from the actual values and at the same time leads to inaccurate localization if the image and cloud processing is not processed at a very high frequency [2]. This would also impair the frequency with which the map is updated and hence the overall efficiency of the SLAM algorithm will be affected.

For this thesis, we propose to investigate in-depth the visual or LiDAR SLAM approach using our state-of-the-art Intel Realsense cameras and light detection and ranging sensors (LiDAR). For this, the following concrete tasks will be focused on:

Tentative Work Plan

• study the concept of visual or LiDAR-based SLAM as well as its application in the survey of an unknown environment.
• 2D/3D mapping in both static and dynamic environments.
• localise the robot in the environment using the adaptive Monte Carlo localization (AMCL) approach.
•  write a path planning algorithm to navigate the robot from starting point to the destination avoiding collision with obstacles.
• real-time experimentation, simulation (MATLAB, ROS & Gazebo, Rviz, C/C++, Python etc.) and validation.

About the Laboratory

Robotics & AI-Lab of the Chair of Cyber-Physical Systems is a research innovative lab focusing on robotics, artificial intelligence, machine and deep learning, embedded smart sensing systems and computational models. To support its research and training activities, the laboratory currently has:

• additive manufacturing unit (3D and laser printing technologies).
• metallic production workshop.
• robotics unit (mobile robots, robotic manipulators, robotic hands, unmanned aerial vehicles (UAV))
• sensors unit (Intel Realsense (LiDAR, depth and tracking cameras), Inertial Measurement Unit (IMU), OptiTrack cameras etc.)
• electronics and embedded systems unit (Arduino, Raspberry Pi, e.t.c)

Expression of Interest

Students interested in carrying out their Master of Science (M.Sc.) or Bachelor of Science (B.Sc.) thesis on the above topic should immediately contact or visit the Chair of Cyber Physical Systems.

Phone: +43 3842 402 – 1901 

Map: click here

References

[1]  V.Barrile, G. Candela, A. Fotia, ‘Point cloud segmentation using image processing techniques for structural analysis’, The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2/W11, 2019 

[2]  Łukasz Sobczak , Katarzyna Filus , Adam Domanski and Joanna Domanska, ‘LiDAR Point Cloud Generation for SLAM Algorithm Evaluation’, Sensors 2021, 21, 3313. https://doi.org/10.3390/ s21103313.

[3]  Wolfram Burgard, Cyrill Stachniss, Kai Arras, and Maren Bennewitz , ‘SLAM: Simultaneous
Localization and Mapping’,  http://ais.informatik.uni-freiburg.de/teaching/ss12/robotics/slides/12-slam.pdf

Master Thesis

The final master thesis document can be downloaded here. 

Supervisor: Linus Nwankwo, M.Sc.;
Univ.-Prof. Dr Elmar Rückert
Start date:  As soon as possible

Theoretical difficulty: mid
Practical difficulty: High

Abstract

The goal of this thesis is to enhance the method proposed in [1] to enable autonomous robots to effectively interpret open-ended language commands, plan actions, and adapt to dynamic environments.

The scope is limited to grounding the semantic understanding of large-scale pre-trained language and multimodal vision

language models to physical sensor data that enables autonomous agents to execute complex, long-horizon tasks without task-specific programming. The expected outcomes include a unified framework for language-driven autonomy, a method for cross-modal alignment, and real-world validation.

Tentative Work Plan

To achieve the objectives, the following concrete tasks will be focused on:

• Backgrounds and Setup:
  • Study LLM-for-robotics papers (e.g., ReLI [1], Code-as-Policies [2], ProgPrompt [3]), vision-language models (CLIP, LLaVA).
  • Set up a ROS/Isaac Sim simulation environment and build a robot model (URDF) for the simulation (optional if you wish to use an existing one).
  • Familiarise with how LLMs and VLMs can be grounded for short-horizon robotic tasks (e.g., “Move towards the {color} block near the {object}”), in static environments.
  • Recommended programming tools: C++, Python, Matlab.
• Modular Pipeline Design:
  • Speech/Text (Task Instruction) ⇾ LLM (Task Planning) ⇾ CLIP (Object Grounding) ⇾  Motion Planner (e.g., move towards the {colour} block near the {object}) ⇾ Execution (In simulation or real-world environment).
  •  
• Intermediate Presentation:
  • Present the results of your background study or what you must have done so far.
  • Detailed planning of the next steps.
  •  
• Implementation & Real-World Testing (If Possible):
  • Test the implemented pipeline with a Gazebo-simulated quadruped or differential drive robot.
  • Perform real-world testing of the developed framework with our Unitree Go1 quadruped robot or with our Segway RMP 220 Lite robot.
  • Analyse and compare the model’s performance in real-world scenarios versus simulations with the different LLMs and VLMs pipelines.
  • Validate with 50+ language commands in both simulation and the real world.
  •  
• Optimise the Pipeline for Optimal Performance and Efficiency (Optional):
  • Validate the model to identify bottlenecks within the robot’s task environment.
  •  
• Documentation and Thesis Writing:
  • Document the entire process, methodologies, and tools used.
  • Analyse and interpret the results.
  • Draft the thesis, ensuring that the primary objectives are achieved.
    • Chapters: Introduction, Background (LLMs/VLMs in robotics), Methodology, Results, Conclusion.
  • Deliverables: Code repository, simulation demo video, thesis document.
  •  
• Research Paper Writing (optional)
  •  

References

[1] Nwankwo L, Ellensohn B, Özdenizci O, Rueckert E. ReLI: A Language-Agnostic Approach to Human-Robot Interaction. arXiv preprint arXiv:2505.01862. 2025 May 3.

[2] Liang J, Huang W, Xia F, Xu P, Hausman K, Ichter B, Florence P, Zeng A. Code as policies: Language model programs for embodied control. In2023 IEEE International Conference on Robotics and Automation (ICRA) 2023 May 29 (pp. 9493-9500). IEEE.

[3] Singh I, Blukis V, Mousavian A, Goyal A, Xu D, Tremblay J, Fox D, Thomason J, Garg A. Progprompt: Generating situated robot task plans using large language models. In2023 IEEE International Conference on Robotics and Automation (ICRA) 2023 May 29 (pp. 11523-11530). IEEE.

[4] Nwankwo L, Rueckert E. The Conversation is the Command: Interacting with Real-World Autonomous Robots Through Natural Language. In Companion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction 2024 Mar 11 (pp. 808-812).

Supervisor: Vedant Dave, M.Sc.
Univ.-Prof. Dr Elmar Rückert
Start date:  3rd April 2025

Theoretical difficulty: mid
Practical difficulty: low

Abstract

Due to the tendency of reinforcement learning models to overfit to training data, data augmentation has become a widely adopted technique for visual reinforcement learning tasks for its capability of enhancing the performance and generalization of agents by increasing the diversity of training data. Often, different tasks benefit from different types of augmentations, and selecting them requires prior knowledge of the environment. This thesis aims to explore how various augmentation strategies can impact the performance and generalization of agents in visual environments, including visual augmentations and context-aware augmentations.

Tentative Work Plan

• Literature research.
• Understanding of concepts of visual RL models (SVEA).
• Implementing and testing different augmentations.
• Observation and documentation of results.
• Thesis writing.

Related Work

[1] N. Hansen and X. Wang, “Generalization in Reinforcement Learning by Soft Data Augmentation,” 2021 IEEE International Conference on Robotics and Automation (ICRA), Xi’an, China, 2021, pp. 13611-13617, doi: 10.1109/ICRA48506.2021.9561103

[2] Hansen, Nicklas, Hao Su, and Xiaolong Wang. “Stabilizing deep q-learning with convnets and vision transformers under data augmentation.” Advances in neural information processing systems 34 (2021): 3680-3693.

[3] Almuzairee, Abdulaziz, Nicklas Hansen, and Henrik I. Christensen. “A Recipe for Unbounded Data Augmentation in Visual Reinforcement Learning.” Reinforcement Learning Conference.

Supervisor: M.Eng Fotios Lygerakis, Univ.-Prof. Dr. Elmar Rückert

Theoretical difficulty: mid
Practical difficulty: hard

Abstract

Robotic manipulation in dynamic environments requires systems that can adapt
to uncertainties and learn from limited human input. This thesis presents a dexterous
multi-finger robotic framework that integrates intuitive teleoperation with
self-supervised visuotactile representation learning to enable contact-rich imitation
learning. Central to the system is a Franka Emika Panda robotic arm paired with a
multi-fingered LEAP Hand equipped with high-resolution GelSight Mini tactile sensors.
A Meta Quest 3 teleoperation interface captures natural human demonstrations while
collecting multimodal data, including visual, tactile, and joint-state inputs, to train
the self-supervised encoders.

The study evaluates two representation learning methods, BYOL and MViTac, under
low-data conditions. Extensive experiments on complex manipulation tasks — such as
pick-and-place, battery insertion, and book opening—demonstrate that BYOL-trained
encoders consistently outperform both MViTac and a ResNet18 baseline, achieving
a 60% success rate on the challenging spiked cylinder task. Key findings highlight
the critical role of tactile feedback quality, with GelSight sensors delivering robust
tactile impressions compared to lower-resolution alternatives. Furthermore, parameter
studies reveal how system settings (e.g., reject buffers, movement thresholds) and
demonstration selection critically influence task performance.

Despite challenges in scenarios requiring precise visual-tactile coordination, the
results validate the potential of self-supervised learning to reduce human annotation
effort and facilitate a smooth transition from teleoperated control to autonomous
execution. This work provides valuable insights into the integration of hardware and
software components, as well as control strategies, demonstrating BYOL’s potential as
a promising approach for tactile representation learning in advancing autonomous
robotic manipulation.

Milestones

Teleoperation test of the LEAP Hand:

https://cps.unileoben.ac.at/wp/LeapHandTest.mp4

Visual encoder test:

https://cps.unileoben.ac.at/wp/VisualEncoderTest.mp4

First version of the FrankaArm-control test:

https://cps.unileoben.ac.at/wp/FrankaArmTest.mp4

Dataset collection / teleoperation of the whole setup:

https://cps.unileoben.ac.at/wp/CompleteSetup.mp4

Fully autonomous task execution:

https://cps.unileoben.ac.at/wp/AutonomousTaskExecution.mp4

The Chair of Cyber-Physical Systems at Montanuniversität Leoben is offering a fully funded PhD position (100% employment) starting as soon as possible.

• Employment Type: Full-time doctoral student (40 hours/week)

• Salary: €3,714.80/month (14 times per year), Salary Group B1 according to Uni-KV

• Duration: The position includes the opportunity to complete a PhD

About the Position

We are at the forefront of developing cutting-edge machine learning algorithms for detecting, tracking, and classifying material flows using various advanced sensing technologies, including:

• RGB cameras
• 3D imaging
• LiDAR
• Hyperspectral cameras
• RAMAN devices
• Tactile sensors

The resulting model predictions are used for automated data labeling, real-time process monitoring, and autonomous object manipulation.

This PhD research will focus on multiple aspects of these topics, with a special emphasis on multimodal sensing and robotic grasping. The goal is to enhance robotic perception and interaction by integrating machine learning with tactile sensing technologies.

What we offer

•A dynamic and collaborative research environment in artificial intelligence and robotics

•The opportunity to develop your own research ideas and work on cutting-edge projects

• Access to state-of-the-art lab facilities

•International research collaborations and conference travel opportunities

•Targeted career guidance for a successful academic and research career

Plus a great lab space shown in this image.

Requirements

• Master’s degree in Computer Science, Physics, Telematics, Statistics, Mathematics, Electrical Engineering, Mechanics, Robotics, or a related field

• Strong motivation for scientific research and publications

• Ability to work independently and collaboratively in an interdisciplinary team

• Interest in writing a PhD dissertation

Desired additional qualifications

• Programming experience in C, C++, C#, Java, MATLAB, Python, or a similar language

• Familiarity with AI libraries and frameworks (e.g., TensorFlow, PyTorch)

• Strong English communication skills (written and spoken)

• Willingness to travel for research collaborations and technical presentations

Application & Materials

A complete application includes:

1. Curriculum Vitae (CV) (detailed)

2. Letter of Motivation

3. Master’s Thesis (PDF or link)

4. Academic Certificates (Bachelor’s and Master’s degrees)

Optional but beneficial:

5. Letter(s) of Recommendation

6. Contact Information for References (name, email, phone)

7. Previous Publications (PDFs or links)

Application deadline: Open until the position is filled.

Online Application via Email: Please send your application files to rueckert@unileoben.ac.at

The Montanuniversität Leoben intends to increase the number of women on its faculty and therefore specifically invites applications by women. Among equally qualified applicants women will receive preferential consideration.

Forschungsprojekt Nr. 991 “#command21 – Joint Environmental Denied Interface (JEDI)”

Das Projekt ist im Januar 2025 gestartet und befasst sich mit der Erforschung von KI-Modellen zur Gestenerkennung und Steuerung von autonomen Systemen. 

Für die Mensch-Maschine-Interaktion kommen multiple Systeme, wie zum Beispiel VR/AR-Brillen, zum Einsatz.  

Laufende Projekte, Bachelor- und Masterarbeiten

• Offene Bachelor- und Masterarbeiten zur “Sensorfusion für multimodale Gestenerkennung”
• Offene Bachelor- und Masterarbeiten zur “Prozess- und Workflow-Modellierung für die Interaktion mit autonomen Systemen”

Update (as of 5 Feb 2025)

The university has an insurance for all employees for business trips. Thus, whenever you officially applied for a business trip (via MUOnline) and after the trip is granted, you are insured. The insurance includes many aspects, including medical treatment, retrieval, lost bags, etc.  Note: Thus, even for trips to the USA, you do not need a private insurance.  

Dear CPS team, thank you all very much for this successful year. We had a very nice Christmas party in our new building – the center of digitalization (CeDi), or in german – Das Haus der Digitalisierung. 

Schlüsseltechnologien im produktionsnahen Umfeld 2024

FFG Projekt 03/2025-02/2028

Unsere Forschung entwickelt intelligente Roboterhände, die fühlen können – ähnlich wie menschliche Hände. Mit innovativen Sensoren auf Basis von nachhaltigen Materialien wie Cellulose und Nanocellulose schaffen wir Lösungen, die nicht nur Druck und Temperatur spüren, sondern auch Schadstoffe und biologische Gefahren erkennen können.

Diese Technologie wird speziell für schwierige Aufgaben in der Industrie entwickelt, z. B. beim Sortieren von Abfällen oder empfindlichen Materialien, die bisher nur Menschen bewältigen konnten. Durch den Einsatz von Robotern in gefährlichen oder anstrengenden Arbeitsumfeldern schützen wir Menschen und verbessern die Arbeitsbedingungen.

Mit unseren nachhaltigen, langlebigen und präzisen Sensoren setzen wir neue Maßstäbe in der Automatisierung. Das stärkt nicht nur die heimische Wirtschaft, sondern bringt auch Innovation und Sicherheit für die Gesellschaft.

Projekt Consortium

• Montanuniversität Leoben
  • Forschungs- und Innovationsservice Montanuniversität (Administrativer Koordinator)
  • Lehrstuhl für Automation und Messtechnik
  • Lehrstuhl für Cyber-Physical-Systems
• FreyZein GmbH
• Saubermacher Dienstleistungs AG

Fördergeber

• Österreichische Forschungsförderungsgesellschaft mbH (FFG)

Project Summary

Our research is developing intelligent robotic hands that can feel – just like human hands. Using innovative sensors based on sustainable materials such as cellulose and nanocellulose, we are creating solutions that not only sense pressure and temperature, but can also detect pollutants and biological hazards.

This technology is being developed specifically for difficult tasks in industry, such as sorting waste or sensitive materials that previously only humans could handle. By using robots in hazardous or strenuous working environments, we protect people and improve working conditions.

With our sustainable, durable and precise sensors, we are setting new standards in automation. This not only strengthens the domestic economy, but also brings innovation and safety to society.

Poster

TBA

Other

TBA

Secretary

Short bio: Ms. Eva Anderson joined the CPS team in February 2025 and is in charge of organizational and administrative matters. She earned her degree in commerce at the Vienna university of economics and business.

Research Interests

• Cyber-Physical-Systems 
• Modern Technologies 
• Learning Machines and Robotics

Contact

Mag. Eva Anderson
Sekretariat des Lehrstuhls für Cyber-Physical-Systems
Montanuniversität Leoben
Franz-Josef-Straße 18, 
8700 Leoben, Austria 

Phone:  +43 3842 402 – 1907
Email:   eva.anderson@unileoben.ac.at 
Web:  https://cps.unileoben.ac.at