You are interested in working with modern robots or want to understand how such machines ‘learn’?

If so, this bachelor seminar will enable you to dig into the fascinating world of robot learning. You will implement and apply modern machine learning algorithms in Python, Matlab or C++/ROS. 

Your learning or control algorithm will be evaluated in cyber-physical-systems. Find out which theses are currently supervised and offered. 

Links and Resources

• Winter Semester: 140.186 MUOnline and 560.150 MUOnline .
• Sommer Semester: 140.186 MUOnline and 560.150 MUOnline .
• Latex Template for the thesis document.
• Instructions on how to write a thesis with us.

Location & Time

• Location: Laboratories at the Chair of Cyber-Physical-Systems.
• Dates: Regular, typ. weekly meetings, with your supervisor(s). 

Learning objectives / qualifications

• Students will work on controlling, modeling and simulating Cyber-Physical-Systems and autonomously learning systems.
• Students understand and can apply advanced model learning and reinforcement  techniques to real world problems.
• Students learn how to write scientific reports.

Literature

• The Probabilistic Machine Learning book by Univ.-Prof. Dr. Elmar Rueckert. 
• Bishop 2006. Pattern Recognition and Machine Learning, Springer. 
• Barber 2007. Bayesian Reasoning and Machine Learning, Cambridge University Press. 
• Murray, Li and Sastry 1994. A mathematical introduction to robotic manipulation, CRC Press. 
• B. Siciliano, L. Sciavicco 2009. Robotics: Modelling,Planning and Control, Springer.
• Kevin M. Lynch and Frank C. Park 2017. MODERN ROBOTICS, MECHANICS, PLANNING, AND CONTROL, Cambridge University Press.

You are interested in working with modern robots or want to understand how such machines ‘learn’?

If so, this bachelor seminar will enable you to dig into the fascinating world of robot learning. You will implement and apply modern machine learning algorithms in Python, Matlab or C++/ROS. 

Your learning or control algorithm will be evaluated in cyber-physical-systems. Find out which theses are currently supervised and offered. 

Links and Resources

• Winter Semester: 140.185 MUOnline  and 560.100 MUOnline .
• Sommer Semester: 140.185 MUOnline  and 560.100 MUOnline .
• Latex Templates for the thesis document.
• Instructions on how to write a thesis with us.

Location & Time

• Location: Laboratories at the Chair of Cyber-Physical-Systems.
• Dates: Regular, typ. weekly meetings, with your supervisor(s). 

Learning objectives / qualifications

• Students will work on controlling, modeling and simulating Cyber-Physical-Systems and autonomously learning systems.
• Students understand and can apply advanced model learning and reinforcement  techniques to real world problems.
• Students learn how to write scientific reports.

Literature

• The Probabilistic Machine Learning book by Univ.-Prof. Dr. Elmar Rueckert. 
• Bishop 2006. Pattern Recognition and Machine Learning, Springer. 
• Barber 2007. Bayesian Reasoning and Machine Learning, Cambridge University Press. 
• Murray, Li and Sastry 1994. A mathematical introduction to robotic manipulation, CRC Press. 
• B. Siciliano, L. Sciavicco 2009. Robotics: Modelling,Planning and Control, Springer.
• Kevin M. Lynch and Frank C. Park 2017. MODERN ROBOTICS, MECHANICS, PLANNING, AND CONTROL, Cambridge University Press.

Supervisor: Linus Nwankwo, M.Sc.;
Univ.-Prof. Dr Elmar Rückert
Start date: 10th January 2022

Theoretical difficulty: mid
Practical difficulty: mid

Abstract

Nowadays, robots used for survey of indoor and outdoor environments are either teleoperated in fully autonomous mode where the robot makes a decision by itself and have complete control of its actions; semi-autonomous mode where the robot’s decisions and actions are both manually (by a human) and autonomously (by the robot) controlled; and in full manual mode where the robot actions and decisions are manually controlled by humans. In full manual mode, the robot can be operated using a teach pendant, computer keyboard, joystick, mobile device, etc.

Although the Robot Operating System (ROS) has provided roboticists with easy and efficient tools to teleoperate or command robots with both hardware and software compatibility on the ROS framework, however, there is a need to provide an alternative approach to encourage a non-robotic expert to interact with a robot. The human hand-gesture approach does not only enables the robot users to teleoperate the robot by demonstration but also enhances user-friendly interaction between robots and humans.

In the context of this thesis, the application of human hand gestures is proposed to teleoperate our mobile robot using embedded computers and inertial measurement sensors. First, we will teleoperate the robot on the ROS platform and then via hand gestures, leveraging on the framework developed by [1] and [2].

Tentative Work Plan

To achieve our aim, the following concrete tasks will be focused on:

• Design and generate the Unified Robot Description Format (URDF) for the mobile robot
• Simulate the mobile robot within the ROS framework (Gazebo, Rviz)
• Set up the interfaces and serial connection between ROS and the robot control devices
• Develop an algorithm to teleoperate the robot in the ROS using hand gesture
• Use the algorithm to perform Simultaneous Localization and Mapping (SLAM) for indoor applications (simulation only)

References

[1] Nils Rottmann et al,  https://github.com/ROS-Mobile/ROS-Mobile-Android, 2020

[2] Wei Zhang, Hongtai Cheng, Liang Zhao, Lina Hao, Manli Tao and ChaoqunXiang, “A gesture-based Teleoperation System for Compliant Robot Motion“, Appl. Sci. 2019, 9(24), 5290; https://doi.org/10.3390/app9245290

Thesis Document

❯ Gesture Based Mobile Robot Teleoperation for Navigation Application

Demonstration Video

https://cps.unileoben.ac.at/wp/Steuerung.mp4

Signs of an Overheat Error

Two signs occur when the overheating error is present:

1) Only the red lamp on top of the robot hand is light up continuously.
2) The ROS Terminal (running the roslaunch rh8d start_rh8d.launch programm) displays “Overheat Error”. In the picture below the error message for servo #38 is displayed.

Needed Resources

1.  Download the TSBAdvanced Loader binary from github.
2.  Download the RH8D Fix folder (so far we have only the fixes for Servo #33, #35 and #38, if you need a different one please talk to Prof. Rückert).

How the fix works

1. Connect the robot hand via the micro USB connector on the side of the hand to a Windows computer.
2. Plug in the power supply of the robot hand.
3. Open a Terminal on the Windows computer.
4. Move to the directory were you unziped the “RH8D Fix.zip” files.
5. Check the COM connection to the robot hand in the devices manager (i.e. COM4).
6. Now enter the bridge mode with the terminal command : “tsbloader_adv.exe -port=[the Virtual Serial port number/id you determined in point 1] -seederos=bron” (replace the [the Virtual Serial port number/id you determined in point 1] with i.e. COM4.
7.  Depending on the servo motor, the corresponding pwd must now be used. Servo #31 = A; Servo #32 = B; … Servo #38 = H.
8. To reset the faulty temperature sensor of the corresponding servo motor use the following command: “tsbloader_adv.exe -port=[YOUR COMM PORT] -prewait=2000 -pwd=[SERVO-PWD] -eop=ewv -efile=[RESET FILE FOR THE SERVO] -baud=33333”
9. If there are more than one errors, fix only one at a time!
10.  After completion use the command: “tsbloader_adv.exe -port=[the Virtual Serial port number/id you determined in point 1] -seederos=broff” to leave the bridge mode.
11. Afterwards reboot the hand (replug the power supply and disconnect the micro USB cable). After booting all three lamps should light again. Otherwise contact the support of seed robotics via email: support@seedrobotics.com

Sources

• EROS Manual(How to connect to RH8D)
• Tinysafeboot github
• How to use Sensor Glove with Robot Hand

Meeting on the 9th of December 2021

Location: Chair of CPS

Date & Time: 9th Dec. 2021, 1pm to 1.30pm

Participants: Univ.-Prof. Dr. Elmar Rueckert, Linus Nwankwo, M.Sc.

Agenda

1. Discussion of the research progress
2. Discussion of related literature for robot intention communication and human robot interaction

Topic 1: Robot Intention Communication and Human Robot Interaction

1. Study the given literature to identify the areas of possible improvement
2. Investigate the integration of LED projector for robot intention communication to humans
3. Implement a visual signal algorithm to communicate the robot’s intention and direction of motion to humans

Topic 2: Probabilistic SLAM

1.  Review the  AMCL approach to track the pose of our robot against the map of the environment while mapping.
2.  Design a SLAM algorithm based on the probabilistic framework and evaluate the performance in terms of loop closure detection, accurate pose estimation and computational cost.

Future Steps

Integrate deep Learning approach with Faster R-CNN model to:

• detect for example if the person interacting with robot is wearing a face mask
• classify the face mask worn based on the recommended or not

Next Meeting

Yet to be scheduled

Install a Virtual Machine that supports Mac M1 Architectures

Well, the options are limited.

• I went for UTM.

Install ubuntu server 20.04

• Download the ubuntu server iso and install it.
• Install Ubuntu Budgie as Desktop or
• Install your favorite desktop following instructions like here.

Coding Environment

• sudo apt-get install openjdk-17-jre-headless (or something like that)
• PyCharm from JetBrains.
• CoppeliaSim.
• GitHubDesktop.
• Terminator.
• HTop.

Repository Clone

• You need to clone this repository.

Additional Requirements

• Download and install the Arduino IDE.

• Follow this page to configure IDE and install additional libraries.

• Install Rosserial Python.

Connection with PC

First step is to make sure that the Sensor glove is connected to the USB0 and Robot Hand is connected to USB1. If this is not in order, we might have to change it inside the files and aslo in rosserial_python library. 

Connecting with ROS

• Initiate Roscore with command: roscore
• Run the Rosserial Python command to initiate the Serial connection between with the hand through Python:

          rosrun rosserial_python serial_node.py tcp

           You will see something like this:

• After this, run the Arduino file to initiate the calibration. If the Serial connection is finished, you will see something like this:

Connecting with the Robot Hand

In order to connect with the hand, just run this file:

roslaunch rh8d start_rh8d.launch

Ablauf

• Spätestens alle 3 Monate sollen Mittel von der DFG abgerufen werden.
• Das erfolgt durch den Lehrstuhl.
• Mittelabruf mit dem Formular DFG-Vordruck 41.039 (41_039_deMUL_Template.doc).
• Offene Kostenpositionen können in SAP abgerufen werden, siehe Bilder unterhalb.

Kreditkartenabrechnungen

Kreditkartenabrechnungen oder Banküberweisungen in Vorleistung werden per Email an Frau Anja Schweinzger geschickt.Das ausgefüllte Refundierungsformular und die Belege müssen beigefügt werden.

Beschaffungen von Sachmittel

• Anlagen werden wie gewohnt per Formular angelegt. SAP Bestellungen per Kostentyp Anlage. Als Abrechnugnskonte das AP26019001 angeben.
• GWG ???

Get Lotus Notes for Linux or IOS

• Download the relevant files from the ZID.
• Or contact the ZID, Christoph Frimmel, christoph.frimmel@unileoben.ac.at and ask for the latest installation files.
• Install Lotus Notes.
• Get a connection file or ask the ZID to setup the connection.

Details to the MAC OS Installation (M1)

• Run the installer called, Notes1101FP4_DE_IT_Notarized_Mac.dmg.
• Start the package: HCL Notes Installer-nl1d.pkg
• Note that the installer ‘Notes_1101_Mac_German_Italian.dmg‘ is not working on an M1.
• Enter your Name.
• Enter the Server: nmail1.unileoben.ac.at

• Enter Lotus Notes Password that you got from the ZID.
• Otherwise, you need to get a notes account.

Setup the Notes Client

• Option 1: Call ZID and let them do it for you.
• Option 2: Copy your old ‘desktop8.ndk‘ file from /Library/Application Support/HCL Notes Data/ into the same folder on your new MAC OS system.

Transponder

Contact Helga Winklmayr, see our post on responsibilities.

Arbeitssicherheit

• Univ.-Prof. Dr. Elmar Rückert is responsiple for complying with the regulations.
• All new employees get a safety briefing.