Meeting 03/11​

Done

• Virtual machine setup & running experiments
  • speed is x3 slower than RTX 3090

Started working on

• develop experiments with new dataset (Caltech101)
• assess experiments
• test transfer learning capabilities of CR-VAE

MS Students Updates

• Melanie
  • Object Detection using Resnet
• Julian
  • basic tutorials on ROS
  • Teleoperation of Turtlebot using PS5 controller
  • UR3 on ROS using MoveIt

Next on

• description of caltech
• talk with Konrad about the cluster
• Update paper with correct results and send it to CVPR
• Hyperparameter grid search experiments
• literature review on representation learning
• experiments with artificial datasets
• develop methods
  • contrastive learning for spiking neural networks.
  • mode-seeking kl divergence

Meeting 11/11​

Progress

• Updated draft of the CR-VAE paper.
• Missed author registration deadline for CVPR >_<*
• Run experiments with simple AE / CR-AE
• Test if InfoMax objective actually works -> it doesn’t.
• New implementation of the loss function
• Tried input normalization and MSE for reconstruction error

• Assess experiments
• Hyperparameter grid search experiments for CR-VAE
• Literature review on representation learning (review paper)
• Going through ROS 2 documentation
• Get acquanted with UR3

MS Students Updates

• Melanie
  • Object Detection using Resnet
  • Next on: Image Segmentation
• Julian
  • Teleoperation of Turtlebot using PS5 controller
  • Next on: UR3 on Gazebo using MoveIt

Other activities

• Discussion with Sahar and Vedant about how Sahar could frame her Reinforcement learning research problem.

Next on

• assess experiments from grid search with the new loss function
  • hopefully there will be some distinct difference of the 3 methods
• further assess the value of MI as an auxilary task for unsupervised representation learnning
  • show that MI in InfoMax actually introduce noise
• Denoising AE/CR-AE/VAE/CR-VAE with augmented images.
• develop experiments with new dataset (Caltech101)
• description of caltech
• test transfer learning capabilities of CR-VAE
• literature review on representation learning
• experiments with artificial datasets
• develop methods
  • contrastive learning for spiking neural networks.
  • mode-seeking kl divergence

   

Journal ideas

• Find the best CL method for CR-VAE
• Transfer learning

Meeting 17/11

Progress

• gpu grid setbacks
• caltech101 dataset
• ROS2 refresh
• new results

MS Students Updates

• Melanie
  • Image Segmentation on Steel Defect dataset
  • Next on: Contrastive Learning
• Julian
  • Sick
  • Next on: UR3 on Gazebo using MoveIt

Other activities

• Study abroad fair speech

Next on

• study the big performance gap in KLD between CR-VAE and VAE
• further assess the value of MI as an auxilary task for unsupervised representation learnning
  • show that MI in InfoMax actually introduce noise
• Denoising AE/CR-AE/VAE/CR-VAE with augmented images.
• develop experiments with new dataset (Caltech101)
• description of caltech
• test transfer learning capabilities of CR-VAE
• literature review on representation learning
• experiments with artificial datasets
• develop methods
  • contrastive learning for spiking neural networks.
  • mode-seeking kl divergence

   

Journal ideas

• Find the best CL method for CR-VAE
• Transfer learning
• develop a contrastive regularization layer for NN

Meeting 23/11

Updates

• AAAI paper submission update
  • received an email that the file was never uploaded even though I have a verification email. Still in the process of figuring out
• new results on smaller architecture -> more distinct results

MS Students Updates

• Melanie
  • Image Segmentation on Steel Defect dataset
  • Next on: Contrastive Learning
• Julian
  • Dropped
  • Subject was not aligned with his program
  • working at the lab did not fit his schedule

Other activities

• christmas & hololens 2 unboxing videos
• storage place or display for PS5 controler, hololense, etc?
• plan to publish the AR project as internship position
  • LinkedIn -> CPS page?
  • MUL
  • Emails

Next on

• reconstruct the paper
  • ICML
  • IJCAI
• caltech101 dataset
• literature review on representation learning
• Hololens 2 review
• seminar talk for latent space representation and explainability in neural networks, feature maps. organize meetings (1 paper per week)

Meeting 30/11​

Updates

• experiments with caltech 101
  • too small dataset. Network needs pretraining
  • too big bictures. problems with gpu memory when training and big storage space when saving models
• refactor code to better scale for more evaluation techniques
• reviewed XAI methods.
• Further literature review for representation learning

MS Student Updates

• Melanie
  • Image Segmentation on Steel Defect dataset
  • Next on: Deep Optical Flow

Other activities

• Hololens 2 review
• plan to publish the AR project as internship position
  • LinkedIn -> CPS page?
  • MUL
  • Emails
• Share Christmas video with the public relations team of MUL
• linked in account ->  page

Next on

• do experiments for interpolating latent space
• run experiments with more datasets
• reconstruct the paper for
  • ICML
  • IJCAI
• literature review on representation learning
• seminar talk for latent space representation and explainability in neural networks, feature maps. organize meetings (1 paper per week)

Meeting 21/10

Done

• experiment assessing with small custom architecture

Next on

• find a new controller
• set up computer for Melanie & Julian
• Virtual machine setup

Meeting 25/10

Done

• preliminary experiment assessing with resnet architecture
• schedule new experiments on resnet architecure
• preparation and meetings with MS studentsmeetings notes per month

Next on

• assess experiments
• literature review on representation learning
• Virtual machine setup
• experiments with artificial datasets
• develop methods
  • contrastive learning for spiking neural networks.
  • mode-seeking kl divergence

General Workflows and Agreements

Find below some general descriptions of workflows and agreements for working at our chair.

For any further question, you can always ask the whole CPS team.

Vacation Application Process

Please follow the instructions below, when you apply for any vacation days.  

1. Request my permission via email
   • Please always send me an email first, where you ask for my ok for your vacation plan. Otherwise, I will ignore any notifications from the SAP system.
   • Add Regina cc to the email
2. Apply for vacations in the online SAP portal (https://ess.unileoben.ac.at/)
   • I will receive an automatic email notification and will approve your application.
   • Without my approval, your application is not granted.
3. Add your vacation days to our “CPS Events” Calendar
   • All of you have write access to add your vacation days.  

Working from at Home

According to your work contract, you have to come to the office for work.

Exceptions based on eventually valid covid regulations will be communicated via emails from the president of the university.

Meeting notes on the 20th of October, 2022

Location: Chair of CPS

Date & Time: 20th October, 2022, 12:35 pm to 1:38 pm

Participants: Univ.-Prof. Dr. Elmar Rueckert, Linus Nwankwo, M.Sc.

Agenda

1. General Discussion
2. Update on Conference Paper
3. Do-It-Lab

General Discussion

1.  Add dates to all the meeting notes
2. Add publication to the home page
3. Contact Christopher for his presentation date and update our calendar accordingly
4. Use the Deck app to communicate updates of current, completed and yet-to-be-done tasks

Update on Conference Paper

1.  Compare map-based and map-less indoor SLAM methods.
2. Focus on indoor navigation.
3. Evaluate 1 – 3 lidar-based, 1 – 3 visual-based, and 1- 3 deep learning SLAM methods.
4. Pick up some ideas from the referenced papers in the Deck app.

Do-It-Lab

1. Organise the students into four groups
2.  Give the students the questionnaire after the lab to fill out and submit. You could generate a barcode using the web link to the form.

Location: Chair of CPS

Date & Time: 5th Oct 2022

Participants: Univ.-Prof. Dr. Elmar Rueckert, DI Nikolaus Feith, BSc

Agenda

1. Update
2. Future Steps/News

Top 1: Update

• Since October 12, two presentations were prepared and one was given to the chair on the ROS2 tutorial. The second one deals with the paper “VisuoSpatial Foresigt for Multi-Step, Multi-Task Fabric Manipulation” and will be given next Thursday.
• Furthermore, a concept for the CPS HUB for the ROS2 repositories was developed and finalized with the other PhD students. 
• Literature search on Related work to GNN in Motion Planning has been started.
•  First ROS2 submodule of RH8D robot arm was completed. Data of force sensors at fingertips can be published.

Top 2: Future Steps/News

• Usage of the desk app in nexttcloud was discussed
• ROS2: Further discussion on the shared control project, usage of the tablet with android app. Continue working on the RH8D hand.
• Topics for the Integrated Project, Bachelor’s theses and Master’s theses should be considered. What should be investigated in more detail?
• Not only on shared control should be further investigated but also GNN.

Supervisor: Vedant Dave, M.Sc; Univ.-Prof. Dr Elmar Rückert
Start date: 1st May 2022                          Finsihed: 18th October 2022

Theoretical difficulty: Mid
Practical difficulty: Mid

Abstract

Tactile perception is one of the basic senses in humans that utilize almost at every instance. We predict the touch of the object even before touching it, only through vision. If a novel object is encountered, we predict the tactile sensation even before touching. The goal of this project is to predict tactile response that would be experienced if this grasp were performed on the object. This is achieved by extracting the features of the visual data and the tactile information and then learning the mapping between those features. 

We use Intel RealSense depth camera D435i for capturing images of the objects and Seed RH8D Hand with tactile sensors to capture the tactile data in real time(15 dimensional data). The main objective is to perform well on the novel object which have some shared feature representation of the previously seen objects.

Plan

• Literature Research
• Architecture Development
• Dataset Collection from Real Robot.
• Application in Real Robot.
• Master Thesis Writing
• Research Paper Writing (Optional)

Related Work

[1] B. S. Zapata-Impata, P. Gil, Y. Mezouar and F. Torres, “Generation of Tactile Data From 3D Vision and Target Robotic Grasps,” in IEEE Transactions on Haptics, vol. 14, no. 1, pp. 57-67, 1 Jan.-March 2021, doi: 10.1109/TOH.2020.3011899.

[2] Z. Abderrahmane, G. Ganesh, A. Crosnier and A. Cherubini, “A Deep Learning Framework for Tactile Recognition of Known as Well as Novel Objects,” in IEEE Transactions on Industrial Informatics, vol. 16, no. 1, pp. 423-432, Jan. 2020, doi: 10.1109/TII.2019.2898264.

Thesis Document

A Framework for Learning Visual and Tactile Correlation

Increase your Productivity

Schedule your weekly tasks, meetings, courses or activities!

Increase your Flexibility

Access your files from any computer, tablet or phone!

Work as a Team

Edit together in real-time with easy sharing, and use comments, suggestions, and action items to keep things moving. Or use @-mentions to pull relevant people, files, and events into your online files for rich collaboration.

Important Links

• Calendar
• Documents(Google’s “Word”)
• Sheets(Google’s “Excel”)
• Slides (Google’s PowerPoint)
• Forms

Participants

Niko, Fotis, Linus, Vedant

Agenda

• First discussion on the projects structure of CPS Hub
• Initial plan & examples

Notes

• Components (e.g. UR3, RH8D_hand, Glove, Hololens2) are independent repositories
• Projects (e.g. TacProMPs, HololensTeleop) are independent repositories that use the above repos.
• No custom messages without previous team meeting
• Use Foxy ROS2

Robot Operating System 2 (ROS2) is an open source middleware for the development of robot applications. This tutorial describes the architecture and the basic components. Furthermore, a comparison to ROS1 is drawn and the application with Python is explained in more detail.

Hier finden sie den Foliensatz zum Vortrag.

Why do I need CAD Software?

• Computer-Aided Design (CAD) is the cornerstone of how you design and build things. It allows the user to digitally create, visualise, and simulate 2D or 3D models of real-world products before it is being manufactured.
• CAD models allow users to iterate and optimize designs to meet design intent.
• The use of CAD software facilitates the testing of real-world conditions, loads, and constraints, which increases the quality of the product.
• CAD software helps to explore ideas and visualise the concept.
• Improve the quality, precision of the design, and communication in the design process.
• Analyse real-world scenarios by computer-aided analysis
• Create a database for product development and manufacturing.

Some Practical Applications of CAD Software

Automobile parts can be modelled, visualised, revised, and improved on the screen before being manufactured.

Electrical schematics, control circuit diagrams, PCBs, and integrated circuits (ICs)  can be designed and developed with ECAD software 

With CAD software, architects can visualise and simulate their entire project using real-world parameters, without needing to build any physical structuress or models. 

What CAD software do I need?

Something free

• FreeCAD
•  TinkerCAD
• Fusion 360
• Onshape
• Solid Edge
• Blender
• SketchUp

My design goes with me wherever I go (cloud-based)

• Onshape
• TinkerCAD
• AutoCAD Web
• SelfCAD
• Vectary
• SketchUp

Something more advanced and professional

• AutoCAD
• Autodesk Inventor
• SolidWorks
• Fusion 360
• Solid Edge
• CATIA
• Onshape
• Shapr3D
• Creo

Windows OS

• AutoCAD
• Autodesk Inventor
• Solidworks
• Fusion 360
• CATIA
• Creo
• Solid Edge
• Shapr3D
• Blender

Linux OS

• NX Advanced Designer
• Blender

MacOS

• AutoCAD
• Autodesk Inventor
• Fusion 360
• Shapr3D
• Blender
• NX Advanced Designer

iOS, Android

• AutoCAD
• Autodesk Inventor
• Shapr3D

Where can I learn CAD?

1. Coursera:  https://coursera.org/courses?query=cad
2. Udemy:  https://udemy.com/topic/autocad/
3. MyCADSite:  https://mycadsite.com/
4. Skill Share:    https://skillshare.com/search?query=solidworks
5. CAD-Tutorials.de:  https://cad-tutorials.de/
6. Youtube:  https://youtube.com/watch?v=cAgpDFTHxpY
7. CADTutor:  http://cadtutor.net/
8. PTC Training:  https://ptc.com/en/ptc-university/training-catalogs
9. Autodesk Tinkercad: https://tinkercad.com/