Course Content

In the first week, advanced machine and deep learning methods like multi-layer-perceptrons, convolutional neural networks, variational autoencoder, transformers, simultaneous navigation and mapping approaches, and more will be presented.

These methods can be tested using interactive tools like for example using   https://playground.tensorflow.org. To deepen the knowledge, students will answer well-crafted scientific questions using latex handouts alone or in teams of two students in the lecture room. 

Additionally, Jupyter notebook files were prepared to implement advanced machine and deep learning approaches without installing any software. For all participants of the course user accounts will be created using our JupyterHub at https://jupyter.cps.unileoben.ac.at. The accounts will remain active till the end of the semester. 

Prerequisites & If you Miss Course Contents

During the first week, a laptop or tablet will be needed to use the interactive tools and the Jupyter notebooks. 

Webex Online Sessions of the 1st Week

Find here the link to the online stream during the first week in October, 2023: https://unileoben.webex.com/unileoben/j.php?MTID=m5492385776dd885ca5dde72e52563c61

When you miss some course contens

If you miss some course contents due to overlapping events, you can watch recordings of the sessions online. All recordings will be hosted via Moodle at https://moodle.unileoben.ac.at/course/view.php?id=3082.

Course Description

Modern machine learning methods and in particular deep learning methods are entering almost all areas of engineering. 

The integrated course enables the students to apply these methods in the application domains of their study.

For this purpose, current problems from the industry are investigated and the possibilities of machine and deep learning methods are tested.

Students gain a deep understanding of method implementations, how data must be prepared, which criteria are relevant for selecting learning methods, and how evaluations must be performed in order to interpret the results in a meaningful way.

Initially, the basics of learning methods are developed in 5-6 lecture units. Then, students select one of the listed industrial problems and work on it alone or as a team (with extended assignments). The project work is accompanied by weekly tutorials with tips and tricks. Finally, the project results are discussed in a written report and presented for a final 10-15min.

Grading is based on the quality of the code, the report, and the final short presentation.

Among others, one of the following industry problems can be chosen:

1. Application and comparison of deep neural networks for steel quality prediction in continuous casting plants with data from the ‘Stahl- und Walzwerk Marienhütte GmbH Graz’.

2. Predictive maintenance of bearing shells using frequency analysis in decision trees and deep neural networks based on acoustic measurement data.

3. Motion analysis and path planning for human-machine interaction in logistics tasks with mobile robots of the Chair of CPS.

4. Autonomous navigation and mapping with RGB-D cameras of the four-legged robot Unitree Go1 for excavation inspection in mining.

The project list is continuously extended.

Links and Resources

• MUOnline
• Link to the ML Book by Univ.-Prof. Dr. Elmar Rueckert

Location & Time

• Location: HS 3 Studienzentrum
• Dates: 02.10.2023 – 06.10.2023, see the course schedule above. Weekly online meetings on Wednesdays 17:00-18:00 via
  • https://unileoben.webex.com/unileoben/j.php?MTID=m5e17e864e5784737dffd2fa1d27d161c
  • Meeting number (access code): 2789 858 4770
  • Meeting password: vTHYP5QMj77

   

Previous Knowledge Expected

Formal Prerequisite: Basic Python programming skills, Fundamentals of Statistics.

Recommended Prerequisites: Introduction to Machine Learning (“190.012” and “190.013”).

Slides

• 02.10.2023
  • Introduction to Machine Learning
  • Introduction to Statistics, Model Validation & Evaluations 
  • Course Organization & Grading
  • Jupyter Notebook on Python Basics and Plotting Tricks
• 03.10.2023
  • Introduction to Multi-Layer-Perceptrons
  • Handout on Neural Networks
  • Time Series Models & Databases
  • Jupyter Notebook on Databases
• 04.10.2023
  • Introduction to Deep Representation Learning [slides] [code] – Fotios Lygerakis
  • Curiosity(MLPs), Imagination(Dreamer) and Information(Empowerment) – Vedant Dave
  • Projects Teaser & Questionary
• 05.10.2023
  • Introduction to Robot Learning
  • Handout on Robot Learning
  • Jupyter Notebook on Path Planning
• 06.10.2023
  • Projekt Tasks & Workload
  • Project Presentations 
  • Team Assignments & Kickoff Meetings
• 31.01.2024 Project Results & Best Practices – Evaluation Results

Learning objectives / qualifications

• Implement or independently adapt modern machine learning methods and in particular deep learning methods in Python.
• Analyze data of complex industrial problems, process (filter) the data, and divide it into training- and test data sets such that a meaningful interpretation is possible.
• Define criteria and metrics to evaluate evaluations and predictions and generate statistics.
• Develop, evaluate, and discuss meaningful experiments and evaluations.
• Identify and describe assumptions, problems, and ideas for improvement of practical learning problems.

Grading

Continuous assessment: During the lectures and the tutorials 0-20 bonus points can be collected through active participation.

Project assignments: Alone or in small groups (2-3 students) one of the listed projects has to be implemented. A written report in form of a git repository wiki page have to be submitted.
– For the implementation (Python Code) 0-40 Points can be obtained.
– For the wiki page report, 0-60 Points will be given.

Grading scheme: 0-49,9 Points (5), 50-65,9 Points (4), 66-79 Points (3), 80-91 Points (2), 92-100 Points (1).

With an overall score of up to 79%, an additional oral performance review MAY (!) also be required if the positive performance record is not clear. In this case, you will be informed as soon as the grades are released. If you have already received a grade via MU online, you will not be invited to another oral performance review.

Literature

Maschine Learning and Data-modelling:

– Rueckert Elmar 2022. An Introduction to Probabilistic Machine Learning, https://cloud.cps.unileoben.ac.at/index.php/s/iDztK2ByLCLxWZA

– James-A. Goulet 2020. Probabilistic Machine Learning for Civil Engineers. MIT Press.

– Bishop 2006. Pattern Recognition and Machine Learning, Springer.

Learning method Programming in Python:

– Sebastian Raschka, YuxiH. Liu and Vahid Mirjalili 2022. Machine Learning with PyTorch and Scikit- Learn. Packt Publishing Ltd, UK.

– Matthieu Deru and Alassane Ndiaye 2020. Deep Learning mit TensorFlow, Keras und TensorFlow.js., Rheinwerk-verlag, DE. 

Problemspecific Litheratur:

– B. Siciliano, L.Sciavicco 2009. Robotics: Modelling, Planning and Control, Springer.

– Kevin M. Lynch and FrankC. Park 2017. MODERN ROBOTICS, MECHANICS, PLANNING, AND CONTROL, Cambridge University Press.

– E.T. Turkogan 1996. Fundamentals of Steelmaking. Maney Publishing,UK.

Increase your Productivity

Schedule your weekly tasks, meetings, courses or activities!

Increase your Flexibility

Access your files from any computer, tablet or phone!

Work as a Team

Edit together in real-time with easy sharing, and use comments, suggestions, and action items to keep things moving. Or use @-mentions to pull relevant people, files, and events into your online files for rich collaboration.

Important Links

• Calendar
• Documents(Google’s “Word”)
• Sheets(Google’s “Excel”)
• Slides (Google’s PowerPoint)
• Forms

Why do I need CAD Software?

• Computer-Aided Design (CAD) is the cornerstone of how you design and build things. It allows the user to digitally create, visualise, and simulate 2D or 3D models of real-world products before it is being manufactured.
• CAD models allow users to iterate and optimize designs to meet design intent.
• The use of CAD software facilitates the testing of real-world conditions, loads, and constraints, which increases the quality of the product.
• CAD software helps to explore ideas and visualise the concept.
• Improve the quality, precision of the design, and communication in the design process.
• Analyse real-world scenarios by computer-aided analysis
• Create a database for product development and manufacturing.

Some Practical Applications of CAD Software

Automobile parts can be modelled, visualised, revised, and improved on the screen before being manufactured.

Electrical schematics, control circuit diagrams, PCBs, and integrated circuits (ICs)  can be designed and developed with ECAD software 

With CAD software, architects can visualise and simulate their entire project using real-world parameters, without needing to build any physical structuress or models. 

What CAD software do I need?

Something free

• FreeCAD
•  TinkerCAD
• Fusion 360
• Onshape
• Solid Edge
• Blender
• SketchUp

My design goes with me wherever I go (cloud-based)

• Onshape
• TinkerCAD
• AutoCAD Web
• SelfCAD
• Vectary
• SketchUp

Something more advanced and professional

• AutoCAD
• Autodesk Inventor
• SolidWorks
• Fusion 360
• Solid Edge
• CATIA
• Onshape
• Shapr3D
• Creo

Windows OS

• AutoCAD
• Autodesk Inventor
• Solidworks
• Fusion 360
• CATIA
• Creo
• Solid Edge
• Shapr3D
• Blender

Linux OS

• NX Advanced Designer
• Blender

MacOS

• AutoCAD
• Autodesk Inventor
• Fusion 360
• Shapr3D
• Blender
• NX Advanced Designer

iOS, Android

• AutoCAD
• Autodesk Inventor
• Shapr3D

Where can I learn CAD?

1. Coursera:  https://coursera.org/courses?query=cad
2. Udemy:  https://udemy.com/topic/autocad/
3. MyCADSite:  https://mycadsite.com/
4. Skill Share:    https://skillshare.com/search?query=solidworks
5. CAD-Tutorials.de:  https://cad-tutorials.de/
6. Youtube:  https://youtube.com/watch?v=cAgpDFTHxpY
7. CADTutor:  http://cadtutor.net/
8. PTC Training:  https://ptc.com/en/ptc-university/training-catalogs
9. Autodesk Tinkercad: https://tinkercad.com/

Slides by CPS on Python

Short introduction to Python with some first examples and a coding convention. 

Why should you consider learning Python

• Python is the most popular programming language in the world with a popularity of 28%.
• It is easier to learn than many other programming languages.
• Python is very readable due to its structure. Thus, bugs can usually be found and fixed quickly.
• Python can be both procedural and object-oriented, which makes it very versatile.
• There is a large number of software libraries. On python package index, PyPI, over 400,000 packages can be found. And the most important ones can be downloaded quickly and easily using pip.
• Python and almost all Python software libraries are available for free on the Internet.

What Python can be used for

• Data Science is one of the most popular application areas of Python. Here, not only the complete data analysis but also the visualization will be implemented in Python. Generally, software libraries such as Numpy, Matplotlib and Pandas are used for this purpose.

• Machine learning is another very popular area. Python can be used for supervised, unsupervised and reinforcement learning. Libraries such as TensorFlow, Keras, PyTorch or scikit-learn are used for this purpose.

Furthermore, Python can be used for all kinds of general purpose programs, such as app development, GUI design, web development and in many other areas. In addition, programs with Python interfaces can be automated through a Python program, for example simulation programs.

Practical example of Python in academia

Especially during the time as a student it is good to have a tool with which you can validate your hypotheses or perform the calculations by the computer. Thus, programs can be written with which problems in physics, chemistry or other tasks can be simulated and subsequently the simulation data can be plotted or even videos can be generated.

Important links

• Digital Competences Python Repository
• Overview of current online python courses (fee required)
• Overview of current online python courses (free)
• Visual Studio Code IDE
• Pycharm IDE
• JupytierLite

Important Articles

• Article on ‘What Does Google Do With My Data… and Should I Be Worried?‘.
• An article in Forbes on ‘Why Facebook’s Metaverse is a privacy nightmare’.

Companies like Google collect and process your data

Google collects your data from many different sources. Here are some examples:

• Gmail: Google can read and store information from every email you write and receive, including in the spam, draft, and trash folders.
• Google Maps: Google saves every location you search, in addition to all the places you physically visit with your devices, even if you aren’t logged in. Are you using Waze instead? Google owns that too. The ubiquity of phones and our constant use of them makes them almost like tracking devices we carry around willingly.
• Android devices: Because Android phones and tablets run on an operating system built by Google, the company can track which ads you’re shown while using your phone. Google also knows what time, down to the second, you open each app.
• Google apps: The Google Play store records all your searches and downloads, as well as any rewards cards used. Google also tracks which articles you’ve read through Google News.
• YouTube: Google acquired YouTube back in 2006. When you’re using YouTube, Google tracks your search history, your watch history, how long you spend watching videos, and all your comments and likes or dislikes.
• Google Assistant: Every request you make and every question you pose is recorded — you can even listen to the audio playback.
• G Suite: Your calendar shows where you’ll be and when, and Google Hangouts saves all of your conversations.

If you are interessted in which data Google has collected about you, test Google Takeout.

Recommendations: Browser, Search Engine & Online Docs

In our digital age, we have to be aware of the data collection strategies of all services that we use. However, often, alternatives  developed by the open-source community exist. Here are some recommendations:

• Browser & Search Engine: I personallyfavor Mozilla’s Firefox as browser and  DuckDuckGo as search engine.
• Deactivate Google’s Activity Tracker & Delete your history: Go to https://myactivity.google.com/myactivity and delte your history.

• Online Docs:
  • Install your own nextcloud server with onlyoffice.
  • Use other & paid online editing tools, like https://www.zoho.com (not sure about their privacy policy).

Recommendations: Messenger & Repositories

• I personally recommend: Nextcloud’s Talk App.
• Setup your own repo server using, e.g., Gitea or Gitey.

Final remarks: Stay sensitive to what happens to your data. Nothing is for free.

This entry course discusses major competences all students should have to study at the MUL. The Chair of CPS provides tutorials on

• Data Safety, Privacy and Content Search on the net.
• Learning Python using online tools.
• Learning to develop 3D-CAD models using online tools.
• Using powerful online team working tools including shared documents.
• Using data repositories and creating your personal webpage.

Links and Resources

• MUOnline course description.

Location & Time

• See the MUOnline link. 
• CPS Presentations are on the: 17.10.2022 at 11.00 in the HS1 Studierendenzentrum.

Posts on Digital Competencies

Introduction to Productivity, Flexibility and Team Work

Read More »

Introduction to CAD Software

Read More »

Introduction to Python

Read More »

Understanding the basics of privacy

Read More »

This course accompanies the 190.012 Introduction to Machine Learning lecture.

Enrolling for this exercise is highly recommended but not a requirement for passing the machine learning lecture. 

Format, Location & Time

• Format: The course format is physical attendance. 
• Location: HS Kuppelwieser
• Broadcast: WEBEX for the Q&A sessions
• Dates: Fridays 13:15 – 14:45 Attention: There is one exception, on the 08.03.24, we start at 12:15!

Learning objectives / qualifications

Hands-on experience with machine learning methods. 

Course Resources

Everybody needs to sign in at MUOnline for the course. Access to Moodle will be essential for all participants in the course, as it will serve as the platform where course materials will be made available. 

Additional Course Resources

• Python Programming Tutorial by Socratica [recommended]
• MUOnline course link
• Introduction to Python
• Multi-Threading Jupiter Tutorial by Vedant Dave

Python Videolectures and Tutorials

• Python Online Tutorials

Contact

Melanie Neubauer via email at melanie.neubauer@unileoben.ac.at

This course is based on the Machine Learning book by Univ.-Prof. Dr. Elmar Rueckert. 

It is written for experienced undergraduates or for first
semester graduate students.

The lecture provides the basic knowledge for the application of modern machine learning methods. It includes an introduction to the basics of data modeling and probability theory. Classical probabilistic linear and non-linear regression methods are derived and discussed using practical examples.

Links and Resources

• MUOnline 
• Strongly Recommended Exercise 
• Link to the Book

Location & Time

• Location: HS 1 Studienzentrum
• Dates: Wednesdays 13:15-15:00 Summer semester

Slides

• 06.03.2023 Introduction & Organisation
  • Slides
  • Python crash course using our Jupyter Hub & Jupyter Notebooks
• 13.03.2024 Machine Learning Fundamentals I + Python Perceptron Example
• 20.03.2024 Machine Learning Fundamentals II
• 10.04.2024 Linear Algebra for ML + Jupyter NB  Line Fitting Example + Jupyter NB Perceptron Iterative Update Example 
• 17.04.2024 Probability Theory
• 24.04.2024 KL divergence & Linear Feature Regression I
• 08.05.2024 Neural Networks & Linear Feature Regression II
• 15.05.2024 Linear Feature Regression III
• 22.05.2024 Bayesian Feature Regression
• 29.05.2024 Gaussian Processes + GPy Example + Jupyter NB GP sklearn Example + Jupyter NB Heteroscedastic GP Example
• 05.06.2024 Probabilistic Trajectory Models
• 12.06.2024 Exam Preparation and Q&A
• 19.06.2024 Written Exam
• 26.06.2024 Exam Results & Best Practices

Course Topics

1. Introduction to Machine Learning (Data and modelling fundamentals)
2. Introduction to Probability Theory (Statistics refresher, Bayes Theorem, Common Probability distributions, Gaussian Calculus).
3. Linear Probabilistic Regression (Linear models, Maximum Likelihood, Bayes & Logistic Regression).
4. Nonlinear Probabilistic Regression (Radial basis function networks, Gaussian Processes, Recent research results in Robotic Movement Primitives, Hierarchical Bayesian & Mixture Models).
5. Probabilistic Inference for Time Series (Time series data, basis function models, learning).

Learning objectives / qualifications

• Students get a comprehensive understanding of basic probability theory concepts and methods.
• Students learn to analyze the challenges in a task and to identify promising machine learning approaches.
• Students will understand the difference between deterministic and probabilistic algorithms and can define underlying assumptions and requirements.
• Students understand and can apply advanced regression, inference and optimization techniques to real world problems.
• Students know how to analyze the models’ results, improve the model parameters and can interpret the model predictions and their relevance.
• Students understand how the basic concepts are used in current state-of-the-art research in robot movement primitive learning and in neural planning.

Grading

The course will be graded based on a written exam (100 Points). 50% of all questions need to be answered correctly to be positive. The exam will take place in the classroom, or online, depending on the current university regulations.

In addition, up to 10 bonus points obtained in regular quiz sessions in the classroom, and 20% of the achieved points of the Machine Learning Lab will be added to your exam result. Note that bonus points can only be obtained when attending the lectures in person. 

Grading scheme: 0-49.9Pts (5), 50-65.9Pts (4), 66-79Pts (3), 80-91Pts (2), 92-100Pts (1).

Forthcoming exam dates are:

• 19.06.2024 at 13:15 HS 1 Studienzentrum
• 09.10.2024 at 13:15 – 14:45 HS Kupelwieser (location not fixed)
• More dates upon request via email to teaching@ai-lab.science (send your request one month in advance to the desired exam date).

Literature

• The Probabilistic Machine Learning book by Univ.-Prof. Dr. Elmar Rueckert. 
• James-A. Goulet. Probabilistic Machine Learning for Civil Engineers. ISBN 978-0-262-53870-1.
• Daphne Koller, Nir Friedman. Probabilistic Graphical Models: Principles and Techniques. ISBN 978-0-262-01319-2
• Christopher M. Bishop. Pattern Recognition and Machine Learning. Springer (2006). ISBN 978-0-387-31073-2.
• David Barber. Bayesian Reasoning and Machine Learning, Cambridge University Press (2012). ISBN 978-0-521-51814-7.
• Kevin P. Murphy. Machine Learning: A Probabilistic Perspective. ISBN 978-0-262-01802-9

Note that all books are available at our library or at the chair of CPS.