Making Virtual Lectures Interactive

Giving virtual lectures can be exciting. Inspired by numerous blog posts of colleagues all over the world (e.g., [1], [2]), I decided to turned an ordinary glass desk into a light board. The total costs were less than 100 EUR.

Below you can see some snapshots of the individual steps.

Details to the Lightboard Construction

The light board construction is based on

• A glas pane, 8mm thick. Hint: do not use acrylic glass or glas panes thinner than 8mm. I got an used glass/metal desk for 20EUR.
• LED stripes from YUNBO 4mm width, e.g. from [4] for 13EUR. Hint: Larger LED strips, which you can typically get at DIY markets have width of 10mm. These strips do not fit into the transparent u profile.
• Glass clamps for 8mm glass, e.g., from onpira-sales [5] for 12EUR.
• Transparent U profiles from a DIY store, e.g., the 4005011040225 from HORNBACH [6] for 14EUR.
• 4 castor wheels with breaks, e.g. from HORNBACH no. 4002350510587 for 21EUR.

Details to the Markers, the Background and the Lighting

Some remarks are given below on the background, the lighting and the markers.

• I got well suited flourescent markers, e.g., from [6] for 12EUR. Hint: Compared to liquid chalk, these markers do not produce any noise during the writing and are far more visible.
• The background blind is of major importance. I used an old white roller blind from [7] and turned it into a black blind using 0.5l of black paint. Hint: In the future, I will use a larger blind with a width of 3m. A larger background blind is required to build larger lightboards (mine is 140x70mm). Additionally, the distance between the glass pane and the blind could be increased (in my current setting I have a distance of 55cm).
• Lighting is important to illuminate the presenter. I currently use two small LED spots. However, in the future I will use professional LED studio panels with blinds, e.g. [8]. Hint: The blinds are important to prevent illuminating the black background.
• The LED stripes run at 12Volts. However, my old glass pane had many scratches, which become fully visible at the maximum power. To avoid these distracting effects, I found an optimal setting with 8Volts worked best for my old glass pane.

Details to the Software and to the Microphone

At the University, we are using CISCO’s tool WEBEX for our virtual lectures. The tool is suboptimal for interactive lightboard lectures, however, with some additional tools, I converged to a working solution.

• Camera streaming app, e.g., EPOCCAM for the iphones or IRIUN for android phones. Hint: the smartphone is mounted on a tripod using a smartphone mount.
• On the client side, a driver software is required. Details can be found when running the smartphone app.
• On my mac, I am running the app Quick Camera to get a real time view of the recording. The viewer is shown in a screen mounted to the ceiling. Hint: The screen has to be placed such that no reflections are shown in the recordings.
• In the WEBEX application, I select the IRIUN (virtual) webcam as source and share the screen with the quick camera viewer app.
• To ensure an undamped audio signal, I am using a lavalier microphone like that one [9].
• For offline recordings, apple’s quicktime does a decent job. Video and audio sources can be selected correctly. Hint: I also tested VLC, however, the lag of 2-3 seconds was perceived suboptimal by the students (a workaround with proper command line arguments was not tested).

An Example Lecture

And that’s how it looks …

Laufende Projekte, Bachelor- und Masterarbeiten

• Automatisierungslösungen in der Produktion

One vacant position for a full-time Senior Scientist (m/f/d) at the Chair of Cyber-Physical-Systems in the Department Product Engineering from the earliest possible date in a 3-year fixed-term employment contract; with the option of a qualification position after positive evaluation according to §99 para. 5 (UG 2002), equivalent to a tenure track assistant professorship.

Salary Group B1 according to the Uni-KV, monthly minimum salary excl. Szlg.: € 4.752,30 for 40 hours per week (14 x per year).

We are looking for a researcher with high personal motivation for scientific excellence and integrity, with the ability to solve problems and enjoy working in research teams in an interdisciplinary and internationally oriented environment.

Core Responsibilities

• Independent teaching in the areas of digitalization / data modelling / programming, artificial intelligence/machine learning and robotics.

• Extension and active participation in the group’s research on topics such as Computer Vision, Deep Learning, Large Language Models, Machine Learning, Reinforcement Learning, Smart Devices, Tactile Learning, Autonomous Systems, Mixed Reality, Autonomous Navigation, Human-Robot Interaction, Interactive Learning, Manipulation, Probabilistic Inference, Robot Learning, Simulation, Spiking Neural Networks and Transfer Learning, demonstrated by publications at international conferences (e.g. AAAI, AISTATS, CoRL, ICML, ICRA, IJCAI, IROS, NeurIPS, RSS) and in renowned journals (e.g. AURO, IJRR, JMLR, MLJ, Neural Computation, SciReps, TRo).

• Leading project management tasks and applying for funding to support and further develop the chair’s research priorities.

• Promotion of interdisciplinary collaboration and publication of research results in high-ranking scientific journals and conferences.

Requirements

• Degree in computer science, physics, telematics, electrical engineering, mechanics, robotics,
  mathematics or related studies with a doctorate in a relevant subject
• Experience in at least one of the following areas or related topics: Machine Learning, Neural Networks,
  Robot Learning or Learning Sensor Systems.
• Willingness and ability to co-supervise scientific work and to publish research results.
• Programming experience in one of the following languages: C, C++, C#, JAVA, Python or similar.
• Ability to work in a team, sociability, self-motivation and reliability in a growing team.
• Good command of English and willingness to travel for research and technical presentations.

Desired Additional Qualifications

Several years of professional experience in the above-mentioned subject areas or experience as a postdoc or research group leader in international teams.

Application Documents Required:

A complete application includes a (1) detailed curriculum vitae, (2) a letter of motivation with a reference to the desired research and teaching field from the above-mentioned subject areas, (3) two letters of recommendation, (4) all relevant certificates of previous education for bachelor’s, master’s and doctoral studies, (5) name, email and telephone number of two further references for contact, (6) previously published or submitted publications, the doctoral thesis and potential patents in the form of links integrated into a PDF file.

We Offer:

The Senior Scientist will be employed for a fixed term of three years. After one year, the position will be evaluated with the possibility of extending the position or extending the position to a career position in accordance with §99 para. 5 (UG 2002). 

We offer a varied and independent job. A team-oriented working atmosphere, intensive cooperation with project partners and involvement in teaching offer ideal opportunities for professional and personal development. The Montanuniversität promotes career paths and offers excellent conditions for social diversity in a contemporary working environment.

For further information, please contact cps@unileoben.ac.at.

Reference ID: 2406WPE Job portal of the university
Application deadline: 08.08.2024

The Montanuniversitaet Leoben intends to increase the number of women on its faculty and therefore specifically invites applications by women. Among equally qualified applicants women will receive preferential consideration. Scientific experience demonstrated through publications in international conferences and journals on machine learning, neural networks, robotics, or embedded systems. Good team-leading skills and the ambition to supervise doctoral students. Experience in obtaining external funding and in collaborating with industrial partners is advantageous but not a requirement. Excellent English skills and willingness to travel for research and to give technical presentations are required.

Note, this course is held in German, for English-speaking students there is an exercise group (Group E) in which the lecture part is repeated at the beginning of each unit.  

A course description in English can be found via MUOnline. 

Kursbeschreibung

Die Datenmodellierung spielt eine wesentliche Rolle in modernen Unternehmen. Unzählige Prozessdaten werden gespeichert und zur Prozessoptimierung, zur Qualitätssicherung oder zum Verbessern der Arbeitssicherheit verwendet.

Aber um welche Daten handelt es sich dabei, wie werden die Daten gespeichert und verarbeitet, und wie können damit die obigen Ziele, z.b. durch KI-Methoden erreicht werden? Diese Fragen werden in dieser Lehrveranstaltung behandelt. 

Ziele der Lehrveranstaltung (LV)

Das Ziel der LV ist die Vermittlung von Kompetenzen zur Datenmodellierung und nicht die Anwendung spezieller Tools.

Um das zu erreichen werden Konzepte (z.B. was sind Datenstörungen, wie erkennt man sie und wie können sie behoben werden) vermittelt und an Beispielen im HS interaktiv und eigenständig erprobt. Es ist keine Datenbankenvorlesung.  

Erworbene Kompetenzen

• Grundlagen der Datenmodellierung kennen. Dazu gehören die unterschiedlichen Daten (quantitative und kategoriale Variablen) und Aufnahme/Modellierungsarten (z.B. Exceltabellen, Datenbanken, Arrays, etc.), Diskretisierungsgrundlagen (Nyquist-Shannon Theorem) und Imformationstheorie-Grundlagen (Entropie der Daten).
• Die Notwendigkeit von Kenntnissen zur Datenmodellierung durch Expertenvorträge aus der deutschsprachigen Industrie oder von Experten der MUL erkennen. Eingeladene Experten stellen ihre Daten, Modellierungsverfahren und die nötigen Kenntnisse der Mitarbeiter vor.  
• Korrelationen erkennen und für Vorhersagen nützen.
• Daten visuell aufbereiten können, z.b. mit online Tools wie Jupyter Notebooks, https://www.chartle.com,https://plotdb.com, oder unserem zukünftigen Research Data Management Repository (https://inveniordm.web.cern.ch).
• Probleme wie Ausreißer, Störungen, fehlende Messwerte in Daten erkennen können und Lösungsstrategien anwenden können.
• Unterschiedliche Datensatzformate und Zugriffsmöglichkeiten kennen, z.B.: online Datenbanken kennen und verarbeiten können (https://www.statista.com, https://trends.google.com/trends/, https://ourworldindata.org).
• Eigene Datenmanagementsysteme erstellen können (SQLLite, mariadb, CSV, Excel) und mit Visualisierungstools verknüpfen können (graphana, Jupyter NB, etc).
• Grundlagen der Statistik auf Daten anwenden können (Mittelwert, Median, Standardabweichung, Quantiles, Tests auf Normalverteilung der Daten, Korrelationen visualisieren).
• Das Funktionsprinzip von maschinellen Lernmethoden auf Daten beschreiben können: Begriffe wie Vorhersagen, abhängige Variablen, erklärende Variablen erklären können.
• Onlinetools (lineare- und nichtlineare Regressionen) zur Vorhersage anwenden können.

Unterrichtsformat

Die LV baut auf vier Säulen auf:

1. Grundlagenvermittlung per Frontalunterricht im Hörsaal mit interaktiven Elementen.
2. Expertenvorträge zur Datenmodellierung aus Unternehmenssicht (was ist der Stand der Technik im Unternehmen, was müssen Studierende beherrschen, wenn sie bei diesen Unternehmen arbeiten wollen).
3. Expertenvorträge zu weiterführenden Inhalten an der Montanuniversität (Wo und wie wende ich im Laufe des Studiums die Kenntnisse an, z.B. Machine Learning, IoT, etc.).
4. Praktische Übungen in Gruppen. 

Links and Ressourcen

• • MUOnline
  • MOODLE

Ort und Zeit

• Vorlesungen und Expertenvorträge: HS 1 Studienzentrum
  • jede Woche am Montag (11:00-13:00), ab dem 11.11.2024
  • jede Woche am Donnerstag (11:00-12:00), ab dem 14.11.2024
• Übungen: Es gibt 10 Gruppen zur Auswahl mit unterschiedlichen Zeiten. Gruppe E wird in Englisch abgehalten. Aufgrund von Ausnahmen, bitte im MUOnline die genauen Termine beachten.   
  • Gruppe 1: Dienstag (14:00-16:00) CR Hilbert
  • Gruppe 2: Dienstag (16:00-18:00) CR Hilbert
  • Gruppe 3: Dienstag (18:00-20:00) CR Hilbert
  • Gruppe 4: Mittwoch (14:00-16:00) CR Hilbert
  • Gruppe 5: Mittwoch (16:00-18:00) CR Hilbert
  • Gruppe 6: Mittwoch (18:00-20:00) CR Hilbert
  • Gruppe 7: Montag (16:00-18:00) CR Hilbert
  • Gruppe 8: Dienstag (16:00-18:00) CR IZR
  • Gruppe 9: Mittwoch (14:00-16:00) CR IZR
  • Gruppe E: Mittwoch (16:00-18:30) CR IZR

Notwendiges Vorwissen

Keine. 

Folien und Unterlagen

Folgende Termine sind für die LV vorgesehen. Jedoch gilt die Liste als vorläufig und nicht alle Termine werden benötigt.  

• 11.11.2024 (Montag)
  • Einführungsvorlesung und Organisation
• 14.11.2024 (Donnerstag)
  • Grundlagen der Datenmodellierung (Prozesse -> Sensoren -> Daten -> Variablen) 
• 18.11.2024 (Montag)
  • Grundlagen der Datenspeicherung (Diskretisierung, Nyquist-Shannon Theorem, Datenspeicherungsarten)
• 21.11.2024 (Donnerstag)
  •  Imformationstheorie-Grundlagen (Entropie der Daten)
• 25.11.2024 (Montag)
  • Grundlagen der Datenanalyse (Ausreißer, Störungen, fehlende Messwerte)
• 28.11.2024 (Donnerstag)
  • Grundlagen Statistik zur Datenanalyse (Ziele, Werkzeuge, Auswertungsbeispiele)
• 02.12.2024 (Montag)
  • Ausgewählte Beispiele zur Datenmodellierung (lineare/nicht-lineare Regression, neuronale Netze)
• 05.12.2024 (Donnerstag)
  • TBA
• 09.12.2024 (Montag)
  • [Expertenvortrag] DI Clemens Friedl, Systemtechnik / Teamleiter Software Engineering Elektronik, Wacker Neuson Linz GmbH, Titel: KI Anwendungsbeispiele in Baumaschinen.
• 12.12.2024 (Donnerstag)
  • TBA
• 16.12.2024 (Montag)
  • [Expertenvortrag] DI Daniel Valtiner, B.Sc. MBA, Infineon Technologies Austria AG, Titel: Chancen und Herausforderungen beim Einsatz von Large Language Models in der Fertigung von Halbleitertechnologie.
• 20.12.2024 (Donnerstag)
  • TBA
• 13.01.2025 (Montag)
  • TBA
• 16.01.2025 (Donnerstag)
  • TBA
• 20.01.2025 (Montag)
  • Prüfungsvorbereitung
• 23.01.2025 (Donnerstag)
  • [Expertenvortrag] Dr. Nils Rottmann, HAKO GmbH, Titel: Datenmodellierungskonzepte und Umsetzungsbeispiele von autonomen Reinigungsfahrzeugen im industriellen Umfeld.
• 27.01.2025 (Montag)
  • Q&A Prüfungsvorbereitung
• 30.01.2025 (Donnerstag)
  • Schriftliche Prüfung
• TBA
  • [Expertenvortrag] Dr. Christoph Sorger
    Forschung & Innovation R&I Manager Stahl- und Walzwerk Marienhütte GmbH, Titel: Digitalisierungstechnologien in der Stahlindustrie, Potentiale und Herausforderungen.
  •  

Benotung

Die Benotung erfolgt immanent. Insgesamt können 100 Punkte durch aktives Mitarbeiten, Übungsblätter und durch Prüfungen erworben werden. 

Details zur Benotung werden in der ersten Vorlesungseinheit vorgestellt, d.h. am 11.11.2024. 

Benotungsschema: 0-49,9 Punkte (5), 50-65,9 Punkte (4), 66-79 Punkte (3), 80-91 Punkte (2), 92-100 Punkte (1).

Bei einer Gesamtpunktzahl von bis zu 79 % KANN (!) auch eine zusätzliche mündliche Leistungsüberprüfung erforderlich sein, wenn der positive Leistungsnachweis nicht eindeutig ist. In diesem Fall werden Sie informiert, sobald die Noten bekannt gegeben werden. Wenn Sie bereits eine Note über MU online erhalten haben, werden Sie nicht zu einer weiteren mündlichen Leistungskontrolle eingeladen.

Literatur

Grundlagen zur Datenmodellierung

– Karsten Berns, Bernd Schürmann, Mario Trapp 2010. Eingebettete Systeme: Systemgrundlagen und Entwicklung eingebetteter Software. Vieweg+Teubner Verlag.

Maschine Learning and Data-modelling:

– Rueckert Elmar 2022. An Introduction to Probabilistic Machine Learning, https://cloud.cps.unileoben.ac.at/index.php/s/iDztK2ByLCLxWZA

– James-A. Goulet 2020. Probabilistic Machine Learning for Civil Engineers. MIT Press.

– Bishop 2006. Pattern Recognition and Machine Learning, Springer.

Learning method Programming in Python:

– Sebastian Raschka, YuxiH. Liu and Vahid Mirjalili 2022. Machine Learning with PyTorch and Scikit- Learn. Packt Publishing Ltd, UK.

– Matthieu Deru and Alassane Ndiaye 2020. Deep Learning mit TensorFlow, Keras und TensorFlow.js., Rheinwerk-verlag, DE. 

Problemspecific Litheratur:

– B. Siciliano, L.Sciavicco 2009. Robotics: Modelling, Planning and Control, Springer.

– Kevin M. Lynch and FrankC. Park 2017. MODERN ROBOTICS, MECHANICS, PLANNING, AND CONTROL, Cambridge University Press.

– E.T. Turkogan 1996. Fundamentals of Steelmaking. Maney Publishing,UK.

Course Content & Topics

Theoretical and practical aspects of computing and learning with neural networks. Investigation of the most commonly used algorithms for deep learning. From the practical perspective, various learning algorithms and types of neural networks will be implemented and applied to artificial and real-world problems. A list of the topics that will be covered is as follows:

• Theoretical background on machine learning
• Feedforward neural networks
  • Training methods, optimization algorithms
  • Regularization, generalization
  • Convolutional neural networks
• Recurrent neural networks (LSTMs, GRUs, etc.)
• Deep generative models
  • Variational autoencoders
  • Generative adversarial networks
  • Denoising diffusion probabilistic models
• Attention & Transformers

Learning Objectives

After positive completion of the course, students will be able to:

• Understand and apply the fundamental concepts of learning principles to implement commonly used architectures in deep learning and to develop novel architectures.
• Design and train complex deep neural networks in supervised and unsupervised learning scenarios which requires a thorough theoretical and practical understanding of the algorithms.
• Identify relevant and important features, benefits and limitations of different neural network models and algorithms, e.g., with respect to their practical, generalization- and regularization abilities.
• Apply state-of-the-art deep learning methods to different problems and to analyze, monitor and visualize the models’ performance and limitations.

Location & Time

Location: Seminarraum (84DIEG007), Haus der Digitalisierung, Roseggerstraße 11
Time: Tuesdays & Thursdays 09:30 – 12:00. Detailed plan can be found here.

Grading

* Continuous assessment and written exam: Details will be presented in the first lecture unit.

* Task assignments: Several practical assignments have to be implemented. For each assignment a written report and/or slides have to be submitted. Details will be presented in the first lecture unit.

* Grading scheme: 0-49.9 Points (5), 50-62.4 Points (4), 62.5-74.9 Points (3), 75-87.4 Points (2), 87.5-100 Points (1)

(With an overall score of up to 75%, an additional oral performance review MAY (!) also be required if the positive performance record is not clear. In this case, you will be informed as soon as the grades are released. If you have already received a grade via MU online, you will not be invited to another oral performance review.)

Prerequisites

• Formal prerequisite: Introduction to Machine Learning VU (“190.018”) or L+P (“190.012” and “190.013”).
• Recommended prerequisites: Good Python programming skills, Fundamentals of Probability Theory, Basic Algebra & Vector Calculus.

Literature

– Ian Goodfellow, Yoshua Bengio and Aaron Courville, “Deep Learning”, 2016.
– Christopher M Bishop, “Pattern Recognition and Machine Learning”, 2006.

Supervisor: Linus Nwankwo, M.Sc.;
Univ.-Prof. Dr Elmar Rückert
Start date:  As soon as possible

Theoretical difficulty: mid
Practical difficulty: High

Abstract

In this project or thesis, we aim to enhance the method proposed in [1] for robust natural human-autonomous agent interaction through verbal and textual conversations. 

The primary focus would be to develop a system that can enhance the natural language conversations, understand the 

semantic  context of the robot’s task environment, and abstract this information into actionable commands or queries. This will be achieved by leveraging the capabilities of pre-trained large language models (LLMs) – GPT-4, visual language models (VLMs) – CLIP, and audio language models (ALMs) – AudioLM.

Tentative Work Plan

To achieve the objectives, the following concrete tasks will be focused on:

• Initialisation and Background:
  • Study the concept of LLMs, VLMs, and ALMs.
  • How LLMs, VLMs, and ALMs can be grounded for autonomous robotic tasks.
  • Familiarise yourself with the methods at the project website – https://linusnep.github.io/MTCC-IRoNL/.
  •  
• Setup and Familiarity with the Simulation Environment
  • Build a robot model (URDF) for the simulation (optional if you wish to use the existing one).
  • Set up the ROS framework for the simulation (Gazebo, Rviz).
  • Recommended programming tools: C++, Python, Matlab.
  •  
• Coding
  • Improve the existing code of the method proposed in [1] to incorporate the aforementioned modalities—the code to be provided to the student.
  • Integrate other LLMs e.g., LLaMA and VLMs e.g., GLIP modalities into the framework and compare their performance with the baseline (GPT-4 and CLIP).
  •  
• Intermediate Presentation:
  • Present the results of your background study or what you must have done so far.
  • Detailed planning of the next steps.
  •  
• Simulation & Real-World Testing (If Possible):
  • Test your implemented model with a Gazebo-simulated quadruped or differential drive robot.
  • Perform the real-world testing of the developed framework with our Unitree Go1 quadruped robot or with our Segway RMP 220 Lite robot.
  • Analyse and compare the model’s performance in real-world scenarios versus simulations with the different LLMs and VLMs pipelines.
  •  
• Optimize the Framework for Optimal Performance and Efficiency (Optional):
  • Validate the model to identify bottlenecks within the robot’s task environment.
  •  
• Documentation and Thesis Writing:
  • Document the entire process, methodologies, and tools used.
  • Analyse and interpret the results.
  • Draft the project report or thesis, ensuring that the primary objectives are achieved.
  •  
• Research Paper Writing (optional)
  •  

Related Work

[1]  Linus Nwankwo and Elmar Rueckert. 2024. The Conversation is the Command: Interacting with Real-World Autonomous Robots Through Natural Language. In Companion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction (HRI ’24). Association for Computing Machinery, New York, NY, USA, 808–812. https://doi.org/10.1145/3610978.3640723.

[2]  Nwankwo, L., & Rueckert, E. (2024). Multimodal Human-Autonomous Agents Interaction Using Pre-Trained Language and Visual Foundation Models. arXiv preprint arXiv:2403.12273.

New Obligation: Submit along comparitive offers

(This starts from 30 July 2024)

You are required to submit a comparative analysis of the prices for your trip along with other documents for claims.

Documents to submit and print in hard copy:

• Conference/summer school schedule
• Transport ticket (flight/intercity train/city train/bus)
• Registration fee
• Spesenabrechnung/Reisekostennachweis (from SAP)
• Accomodation
• Comparative analysis of the prices (only be paid 50% of the flight costs, if this document is not submitted along)

New Obligation: Monthly data entry for öbb tickets

(This starts from 26 Jun 2024)

You can find the entry form at here: https://cloud.cps.unileoben.ac.at/index.php/s/GTFTrT8btK7mMtW

Procedure to submit paperwork to Financial Department

Published on 21 May 2024

Update 1 on 26 Jun 2024

Update 2 on 30 July 2024

1. Login into SAP

At your SAP, click on “Meine Reisen und Spesen”.

2. Click on your desired Trip

In my case, I will show example in Austria.

Click “Welter” to proceed.

3. At the Main Page with 4 steps

Step 1: Verify every information especially Kontierung (Your project number)

Next, click on checkbox with * and then proceed with “Belege erfassen”

Step 2: Add all related claims

Step 3: Validate

There are two options: Save it for future or Sent it to financial department

4. Final step

• Prepare all the original receipts and keep a copy with you.
• Print out the above from system
• Put the documents at “Dienstreisen Folder” at Regina’s place
• Bring the folder to Uni Post Office at 1st floor of old building.

Kindly ask for permission before proceed to poster printing.

To print a poster, you can either go for:

Option 1: Mail Boxes Leoben

Price list:

A0: ~20.00 euro

CPS account: KST 101900

Email them, and they will record at CPS account.

Option 2: ÖH Leoben

Fill the form at : https://www.oeh-leoben.at/de/plotauftrag

Price list:

A0: ~6.63 euro

A1: ~3.35 euro

Only cash payment, and pay it when obtaining.

Unser Lehrling Kosmo Obermayer berichtet über seinen Lehrberuf als Informationstechnologe mit Betriebstechnik und gibt spannnende Einblicke in seinen Alltag am Lehrstuhl für Cyber-Physical-Systems.

Der Artikel ist unter diesem Link erreichbar.

Are you fascinated by the intricate dance of robots and objects? Do you dream of pushing the boundaries of robotic manipulation? If so, this internship is your chance to dive into the heart of robotic innovation!

You can work on this project either by doing a B.Sc or M.Sc. thesis or an internship.

Job Description

This internship offers a unique opportunity to explore the exciting world of robotic learning. You’ll join our team, working alongside cutting-edge robots like the UR3 and Franka Emika cobots, and advanced grippers like the 2F Adaptive Gripper (Robotiq), the dexterous RH8D Seed Robotics Hand and the LEAP hand. Equipped with tactile sensors, you’ll delve into the world of grasping, manipulation, and interaction with diverse objects using Deep Learning methods.

Start date: Open

Location: Leoben

Duration: 3-6 months

Supervisors:

• Fotios Lygerakis
  
• Univ. Prof. Dr Elmar Rückert

Keywords:

• Robot learning
• Robotic manipulation
• Reinforcement Learning
• Sim2Real
• Robot Teleoperation
• Imitation Learning
• Deep learning
• Research

Responsibilities

• Collaborate with researchers to develop and implement novel robotic manipulation learning algorithms in simulation and in real-world.
• Gain hands-on experience programming and controlling robots like the UR3 and Franka Emika cobots.
• Experiment with various grippers like the 2F Adaptive Gripper, the RH8D Seed Robotics Hand and the LEAP hand, exploring their functionalities.
• Develop data fusion methods for vision and tactile sensing.
• Participate in research activities, including data collection, analysis, and documentation.
• Contribute to the development of presentations and reports to effectively communicate research findings.

Qualifications

• Currently pursuing a Bachelor’s or Master’s degree in Computer Science,
  Electrical Engineering, Mechanical Engineering, Mathematics or related
  fields.
• Solid foundation in robotics fundamentals (kinematics, dynamics, control theory).
• Solid foundation in machine learning concepts (e.g., supervised learning, unsupervised learning, reinforcement learning, neural networks, etc)
• Strong programming skills in Python and experience with deep learning frameworks such as PyTorch or TensorFlow.
• Excellent analytical and problem-solving skills.
• Effective communication and collaboration skills to work seamlessly within the research team.
• Good written and verbal communication skills in English.
• (optional) Prior experience in robot systems and published work.

Opportunities and Benefits of the Internship

• Gain invaluable hands-on experience with state-of-the-art robots and grippers.
• Work alongside other researchers at the forefront of robot learning.
• Develop your skills in representation learning, reinforcement learning, robot learning and robotics.
• Contribute to novel research that advances the capabilities of robotic manipulation.
• Build your resume and gain experience in a dynamic and exciting field.
   

Application

Send us your CV accompanied by a letter of motivation at fotios.lygerakis@unileoben.ac.at with the subject: “Internship Application | Robot Learning”

Related Work

• MViTac: Self-Supervised Visual-Tactile Representation Learning via Multimodal Contrastive Training
• M2CURL: Sample-Efficient Multimodal Reinforcement Learning via Self-Supervised Representation Learning for Robotic Manipulation

Funding

We will support you during your application for an internship grant. Below we list some relevant grant application details.

CEEPUS grant (European for undergrads and graduates)

Find details on the Central European Exchange Program for University Studies program at https://grants.at/en/ or at https://www.ceepus.info.

In principle, you can apply at any time for a scholarship. However, also your country of origin matters and there exist networks of several countries that have their own contingent.

Ernst Mach Grant (Worldwide for PhDs and Seniors)

Find details on the program at https://grants.at/en/ or at https://oead.at/en/to-austria/grants-and-scholarships/ernst-mach-grant.

Rest Funding Resourses

Apply online at http://www.scholarships.at/

Supervisor: Univ.-Prof. Dr Elmar Rückert

Project: K1-MET P3.4
Start date: 1st of May 2024

Theoretical difficulty: high
Practical difficulty: mid

Topic

The the steel production, the steel quality heavily depends on the dynamic processes of the meniscus level fluctuations in the mold. These complex dynamic  processes can be observed using IR cameras observing the surface level and the casting powder temperature. 

The goal of this thesis is to develop and compare deep learning approaches (CNNs, transformers) for predicting fluid dynamics in lab prototype environment. 

Tasks

• Literature research of state of the art, see references
• Lab prototype environment for generating complex (structured and chaotic) fluid dynamics
• Dataset recording, visualization and annotation
• Deep Learning algorithm implementation (CNNs & Transformers)
• Evaluation on different datasets (predictable dynamics, complex dynamics, synchronous and async. surface level dynamics, chaotic dynamics).
• Thesis writing.

References

• Enhancing computational fluid dynamics with machine learning R Vinuesa, SL Brunton, Nature Computational Science, 2022.
• Current and emerging deep–learning methods for the simulation of fluid dynamics M Lino, S Fotiadis, AA Bharath… – Proceedings of the …, 2023.
• Unsupervised deep learning of incompressible fluid dynamics, N Wandel, M Weinmann, R Klein, arXiv preprint arXiv:2006.08762, 2020.