Author: Elmar Rueckert

image_pdfimage_print

B.Sc. or M.Sc. Thesis/Project: Running ROS-Mobile on a EV3

Elmar Rueckert teaching_openthesestopics

Supervisor: Linus Nwankwo, M.Sc.;
Univ.-Prof. Dr Elmar Rückert
Start date: ASAP from October 2021

 

Theoretical difficulty: low
Practical difficulty: mid

Abstract

Nowadays, robots used for survey of indoor and outdoor environments are either teleoperated in fully autonomous mode where the robot makes a decision by itself and have complete control of its actions; semi-autonomous mode where the robot’s decisions and actions are both manually (by a human) and autonomously (by the robot) controlled; and in full manual mode where the robot actions and decisions are manually controlled by humans. In full manual mode, the robot can be operated using a teach pendant, computer keyboard, joystick, mobile device, etc.

Recently, the Robot Operating System (ROS) has provided roboticists easy and efficient tools to visualize and debug robot data; teleoperate or control robots with both hardware and software compatibility on the ROS framework. Unfortunately, the Lego Mindstorms EV3 is not yet strongly supported on the ROS platform since the ROS master is too heavy on the EV3 RAM [2]. This limits our chances of exploring the full possibilities of the bricks.

However, in the context of this project, we aim to get ROS to run on the EV3 Mindstorms to enable us to teleoperate or control it on the ROS platform using a mobile device and leveraging on the framework developed by [1].

Tentative Work Plan

To achieve our aim, the following concrete tasks will be focused on:

  • Configure and run ROS on the Lego EV3 Mindstorms
  • Set up a network connection between the ROS-Mobile device and the EV3 robot
  • Teleoperate the EV3 robot on the ROS-Mobile platform
  • Perform Simultaneous Localization and Mapping (SLAM) for indoor applications with the EV3 robot

References

[1] Nils Rottmann et al,  https://github.com/ROS-Mobile/ROS-Mobile-Android, 2020

[2] ROS.org, http://wiki.ros.org/Robots/EV3

Meeting Notes Frontiers, Vacation, Cuda

Elmar Rueckert notes_honghu

Meeting Details

Date : 18th August 2021

Time : 15:00 – 15:20

Location : Web-ex Meeting

Participants: Univ.-Prof. Dr. Elmar Rueckert, Honghu Xue

Agenda

Vacation and Future Plan

Topic 1: Frontiers Journal

  1. Both reviewers endorsed publication of the paper, waiting for the final response of editor
  2. Print the Invoice from Frontiers, take to Conni and deduct from TRAIN Project.

Topic 2: Update Information on website

  1. Provide some tutorials on using Cuda Pytorch
    •  
  2. Update personal information
    • Add publication with Tuluhan.
    • Integrate it with the transfer learning model. 

Next Steps

  1. Data journal submission.
  2. Benedikt Hein’s Master thesis as conference submission.
  3. Benedikt Hein’s Master Internship as journal submission.

Next Meeting

Honghu will write you an email on when his flexible vacation ends.

GPU Cluster with eight A100

Elmar Rueckert wiki_phds

Getting started with Pytorch using Cuda acceleration

This tutorial gives an instruction on installing Cuda and enabling Cuda acceleration using Pytorch in Win10. Installation in Linux or Mac systems are all possible. An additional .py file will verify whether the current computer configuration uses the Cuda or not. The following instruction assumes that you have already installed Python IDE, e.g., Anaconda, Pycharm, Visual Studio…

Step 1: Check which Cuda version is supported by your current GPUs under this website. From the left figure, we can see that A100 supports Cuda 11.0. It is also reported from other blogs/ forums that A100 can support Cuda 11.1. In this post, we install Cuda 11.1.

Step 2: Download Nvidia Cuda Toolkit 11.1 (the same version as Cuda in Step 1) from the website. In Win10, for instance, we follow up the choice as shown right. The size of exe(local) is around 3.1GB. After downloading, run the .exe and perform installation. It may take some minutes to complete installation.

Step 3: On the homepage of Pytorch, choose the appropriate options as shown in the left figure. IMPORTANT: The cuda version must be the same as in Step 1. It is also recommended to use Stable version. After finishing the , copy the command into Anaconda Powershell Prompt or other command prompt where you install packages for Python. Waiting for the installation, which may require larger than 1GB disk space and takes some minutes for installation. You could also find historical version of Pytorch in that homepage.

Verify your installation with .py file

You could download a cuda-test.py file and run it. If the result shows ‘cuda’, then you can enjoy the Cuda acceleration for training neural networks!

Using Multiple GPUs for further acceleration

Running Pytorch with Multiple GPUs can further increase the efficiency. We have 8 GPU cards and can be used parallely for training. Please refer to (1) (2) (3) for details. 

Meeting Notes Template

Elmar Rueckert wiki_phds

As PhD you have to attach a meeting note to your personal profile post, e.g., Linus Nwankwo, M.Sc.. The notes have to be uploaded right after the meeting!

 

Clone this template and change the category to ‘notes_yourname’ (you may need to create this category first). You will need this category to display only your notes on your personal post. 

Make sure that the post you create is only accessible by the administrator and PhDs. In special case you may restrict the access only to you and your supervisor which is the administrator. 

 

The title of the meeting notes should contain the date and some keywords of the discussed topics. 

To attach the notes to your personal post, look at the example here. An image of the relevant section in your post is shown on top.

Meeting on the 13th of August 2021

Location: Chair of CPS

Date & Time: 13th Aug. 2021, 1pm to 2pm

Participants: Univ.-Prof. Dr. Elmar Rueckert, Linus Nwankwo, M.Sc.

 

Agenda

  1. Organisatorial progress update by Linus.
  2. Feedback to the research talk presentation by Elmar.
  3. Topics of promising future research direction.
  4. Next steps of Linus.
  5. Date of the next meeting.

Top 1: Organisational Update

add you text here. 

Getting started with LEGO MINDSTORMS Education EV3

Elmar Rueckert wiki_schueler, wiki_undergradsgrads

What can you do with the LEGO robot sets?

The LEGO Mindstorms Education EV3 sets can be used in different scenarios. They offer a quick and easy introduction into robot control, motion planning and visual navigation from depth images with Python. One can assemble the robots in various ways with different sensors and motors depending on the desired task. 

For more information go to Robot LEGO Robotics EV3 Dev and to https://pypi.org/project/python-ev3dev2/ .

Prerequisites

First of all a development set is necessary. At the chair of Cyber-Physical Systems we have five sets available for students. The implementation of the python code and connection to the EV3 can be done with Visual Studio Code and the extension LEGO MINDSTORMS EV3 MicroPython. The EV3 bricks are equipped with a micro-SD card on which the Micropython Image is installed. A more detailed installation guide is provided on GitHub.

Example - Motor control

In the following is an example python code to control a motor with the EV3. At the beginning the motor has to be initialized with the corresponding port (line 8). There are two different ways to control a motor. First, one can set a desired acceleration and target position to run the motor (line 11). Or one can set the desired acceleration and let the motor run until it is stopped by a command (line 17-23). 

Demo

If you want to get the python code or if you are interested in other example codes go to our GitHub repository or to this repository: https://github.com/bittner/lego-mindstorms-ev3-comparison#inspiration-for-lego-ev3-robots. 

Simulation Tools

You may also build your LEGO robot model in a simulation tool and test your Python algorithms. Here is a list of projects:

Here is a list of 3D Modelling Tools for LEGO systems:

Building a Cyber-Physical-System (CPS)

A CPS combines the predictions or commands of computer simulations (see the section on Simulation Tools) and offers a real-time visualization of the real system and the environment. 

Such a CPS can also be developed with our LEGO Ev3 robots. Current sensor measurements can be communicated in real-time to a simulation and visualization tool via bluethooth or wifi connections. Here is a collection of relevant resources:

PhD Thesis Roadmap & Goals

Elmar Rueckert wiki_road_to_thesis

Defining concrete measurable goals is of major importance for PhD students. 

These goals should be well crafted based on the research focus and the strengths of the student.

The goals form an agreement between the supervisor and the student. They can be used to evaluate the progress at any time. 

Your Thesis Topic in One Sentence

Every student should be able to explain their research topic and the scientific contribution in one sentence! 

Start thinking about it now. You will be asked dozens of times during your PhD at meetings or conferences.

What is your research about; and why is that important; and what is new; in one sentence. 

Thesis Topics

Ideally, your thesis consists of three parts or research topics. These parts might be adapted during the progress of the thesis. 

Each part should result an a journal publication and/or two conference publications. A good strategy is to first publish two short stories (= conference papers of 6-8 pages). Thereafter, the short stories are combined in a larger approach and published as journal paper. 

  1. Plan: planning future actions.
    Open research question: How can biologically inspired neural networks plan complex actions in few milliseconds?
  2. Act: Executing the actions. 
    Open research question: How can closed-loop controller be learned or generated from high-level plans?
  3. Sense: Sensing the environment and monitoring the execution. 
    Open research question: How to monitor the execution of neural closed-loop controller and to apply corrective actions in real-time?

Defining a Short Research Story

  1. Start by collecting the relevant related work to your research question. 
  2. Define an open unsolved problem and why it is relevant to the community. 
  3. Create measurable scores or goals how a solution to this open problem could be validated. 
  4. Also define 1-3 baseline algorithms or approaches which you will (re-)implement first. 
  5. Think about a toy task to demonstrate the principles. Typically, toy task simulations of point masses, simple 2D games or robot simulations in V-Rep are useful first case studies.
  6. Develop a relevant experimental setting to evaluate your approach. 
  7. Compute statistically significant statistics. Print Tables and Figures.
  8. Start by writing the results section of you paper. Describe the Tables and Figures first. Is your research question answered? 
  9. Continue with the methods section. Did you define a problem statement? Are your baseline approaches explained?
  10. Structure the collected related work and add it as subsection to the introduction.
  11. For the introduction, first write 1-3 sentences on the topic and the relevance. Next talk about the open problem and discuss the related work. Explicitly discuss your idea, own prior work and your contribution. 
  12. Write an abstract and a conclusion. 
 

Here are some examples.

PhD Thesis Goals and Achievements

.tg {border-collapse:collapse;border-color:#9ABAD9;border-spacing:0;} .tg td{background-color:#EBF5FF;border-color:#9ABAD9;border-style:solid;border-width:1px;color:#444; font-family:Arial, sans-serif;font-size:14px;overflow:hidden;padding:10px 5px;word-break:normal;} .tg th{background-color:#409cff;border-color:#9ABAD9;border-style:solid;border-width:1px;color:#fff; font-family:Arial, sans-serif;font-size:14px;font-weight:normal;overflow:hidden;padding:10px 5px;word-break:normal;} .tg .tg-0lax{text-align:left;vertical-align:top}
Item Goal Accepted or
Achieved		 Under review or
in Progress		 Note
Journal Paper 2 - 3 min 1 accepted & 1 submitted
Conference Paper 4 - 8 min 2 accepted & 2 submitted
Workshop Paper or
Abstracts		 2 - 4
Live Demo 1 - 2 min 1 own demo
Patents 0 - 2 a patent can replace a journal paper!
Conference Paper Review 8 - 24 number of reviewed papers, not conferences.
Journal Paper Review 2 - 6
B.Sc. Thesis 4 - 12
B.Sc. Project 4 - 12
M.Sc. Thesis / M.Sc. Project 3 - 6
Teaching assistant of exercises 2 semester. It can be the same exercise.
Guest lectures 0 - 4
Conference and Lab Talks 6 - 12
Research or
Industrial Collaborations		 1 - 3
Summer Schools 1 - 2
Lab Stays at other Universities
for 1 - 3 weeks		 0 - 2
Internship of 2 - 3 months 0 - 1
Soft Skill Training 2 - 6
Outreach (lab demos,
school events, fairs, etc.)		 4 - 12
Contribution to a research
grant proposal		 0 - 3 typically few paragraphs only.
Student or hiring committee
member		 0 - 2

Checklist New Employee

Elmar Rueckert wiki_phds

Congratulations! You are now part of the CPS team. 

This post will help you setting up your work environment, getting ready to start as scientist and provides helpful instructions at the beginning. 

Organisation Things to Do

If you have not looked at our organisational instructions, do it right now! 

These steps are of utter importance. Your official start and when you get your first salary depend on these steps.

First Technical Steps


  • Read the How to Add Posts Instructions.
  • Update your People Post on our CPS wordpress site.
  • Create a Gmail account if you do not have one already. Send your gmail email to the Chair.
  • Create a Google Calendar with the name: CPS First Name Last Name, e.g., CPS Elmar Rueckert

First Scientific Steps

Clone the PhD Roadmap post and add it as link to your people page post. Follow the instructions there, define topics and research questions. Remove unnecessary parts like the section on ‘Defining a Short Research Story’.

Checklist New Employee for Admins

Elmar Rueckert wiki_administrative

When a new employee starts at the Chair of CPS, a number of things needs to be done by the Chair, the secretary and the technician. 

Just before the new employee starts

  • Add People Post.
  • Prepare a Workstation.
  • Prepare an office / desk.
  • Get a Transponder from MRs. Winklmayr.

Items to be done at the first day at work

Items to be done after the Work Contract was signed

Welcome to Leoben for Researcher

Elmar Rueckert wiki_phds

Welcome Guide

Have a look at our detailed welcome guide.

Please prepare all necessary documents (e.g., Bachelor & Master certificates) to enrole as student at the MU Leoben.

Note that as PhD you need to attend some courses, seminars, etc. which requires this official registration as student.

All information below is relevant for new PhD students and post docs.

For guest researcher, please visit the site of our international department.

Organisational Steps before you arrive

For EU & Swiss Citizens: 

  • Get familiar with our Webpage and with the team.
  • Get in contact with our secretary.

For Non-EU or Swiss Citizens: 

  • Get a Visa D for gainful employment (germ. VISUM D Erwerb) at an Austrian embassy. 
    Note that without that visa, it may take up to two months until you can sign your work contract, get an health insurance and, most importantly, get your first salary. 
  • With the Visa D for gainful employment, cou can immediately start to work and will have up to six month to get the Austrian residence permit at the local headquarters district (see below).
  • Also all points like for EU & Swiss citizens.   

Organisational Steps after your Arrival

There are a few very important things you need to do after you arrived. Please update the information if it is outdated. 

1. Get a Certificate of Registration from your Landlord


Once you signed the contract for your room or flat in Austria, you get a Certificate of Registration (germ. Meldebestätigung). An example is shown in the Figure on the left. If you did not get such a certificate, contact your landlord. 

 

2. Get a Registration Form from the City Hall

You need to go to the city hall (germ. Stadtgemeinde Leoben) to register your new main residence.  You will receive a Registration form (germ. Meldebestätigung). An example is shown in the left panel.

3. VISA or Residence Permit for Researcher

  • As European, you need to contact the district headquarters within 4 months after your arrival. Her is some description.
  • As Non-European researcher you can apply for a Residence Permit for Researcher. Sadly, the government’s information page is only in german. Below we outline the necessary steps for Non-Europeans. 

4. Get in contact via Email with the District Headquarters

To get a residence permit, you need to get in contact with the district headquarters Leoben (DH). Send them an Email or call them. The district headquarters will need from you: 

  1. A copy of your passport.
  2. Your Work contract.
  3. Your Registration form.

Once you send all your files to the DH, they will send you an appointment date via email. 

5. Documents required from Non-EU applicants

Find all important information on the page of the government. 

In order to obtain a residence permit, you need to contact the Bezirkshauptmannschaft Leoben. Send them an email or call them. The best way is to call them and inform them about your visa status. They will send you and appointment and you will need to submit the following documents:

  1. Completed and signed application form
  2. Copy of valid travel document
  3. Birth certificate (with translation)
  4. Proof of criminal record from the place of your residence (with translation). Note in some cases also records from your home country are needed!
  5. Current passport photo in the size of 3,5 x 4,5 cm (not older than 6 months)
  6. City Registration certificate
  7. Proof of legal entitlement to accommodation in Austria: e.g. rental contract, housing rights agreement.
  8. Proof of health insurance valid in Austria (probably only possible after starting work)
  9. Original admission agreement with the University of Leoben
  10. Original employment contract with the University of Leoben

6. Send all your Files to Mrs. Hotter

Mrs. Hotter of the Montanuniversität Leoben is responsible for new employees. Send her your files asap.