Abstract

Robot learning combines the challenges of understanding, modeling and applying dynamical systems with task learning from rewards, through human robot interaction or from intrinsic motivation signals. While outstanding results using machine and deep learning have been generated in robot learning in the last years, current challenges in industrial applications are underrepresented. The goal of this workshop is to go beyond discussing potential industrial applications like in related past workshops.

These topics were discussed with Pieter Abbeel, Dileep George, Sergey Levine, Jan Peters, Freek Stulp, Marc Toussaint, Patrick van der Smagt, and Georg von Wichert.

Links

Details to the workshop, the speakers and links to slides can be found on the workshop webpage.

Print Media Article in the BILANZ magazine 2019

Links

Link to the PDF.

NeurIPS 2019 Competition Track

Open-ended learning aims to build learning machines and robots that are able to acquire skills and knowledge in an incremental fashion in a certain environment. This competition addresses autonomous open-ended learning with a focus on simulated robot systems that: (a) acquire a sensorimotor competence that allows them to interact with objects and physical environments; (b) learn in a fully autonomous way, i.e. with no human intervention (e.g., no tasks or reward functions) on the basis of mechanisms such as curiosity, intrinsic motivations, task-free reinforcement learning, self-generated goals, and any other mechanism that might support autonomous learning. The competition challenge will feature two phases: during an initial “”intrinsic phase”” the system will have a certain time to freely explore and learn in an environment containing multiple objects, and then during an “”extrinsic phase”” the quality of the autonomously acquired knowledge will be measured with tasks unknown at design time and during the intrinsic phase.

Links

Details on the competition can be found on the project webpage.

Publications

2020

Cartoni, E.; Mannella, F.; Santucci, V. G.; Triesch, J.; Rueckert, E.; Baldassarre, G. REAL-2019: Robot open-Ended Autonomous Learning competition Journal Article In: Proceedings of Machine Learning Research, vol. 123, pp. 142-152, 2020, (NeurIPS 2019 Competition and Demonstration Track). Links | BibTeX @article{Cartoni2020, title = {REAL-2019: Robot open-Ended Autonomous Learning competition}, author = {E. Cartoni and F. Mannella and V.G. Santucci and J. Triesch and E. Rueckert and G. Baldassarre}, editor = {H. J. Escalante and R. Hadsell}, url = {https://cps.unileoben.ac.at/wp/PMLR2020Cartoni.pdf, Article File}, year = {2020}, date = {2020-06-20}, journal = {Proceedings of Machine Learning Research}, volume = {123}, pages = {142-152}, note = {NeurIPS 2019 Competition and Demonstration Track}, keywords = {}, pubstate = {published}, tppubtype = {article} } Close Article File Close

This project aims to develop a new paradigm to build open-ended learning robots called `Goal-based Open ended Autonomous Learning’ (GOAL). GOAL rests upon two key insights. First, to exhibit an autonomous open-ended learning process, robots should be able to self-generate goals, and hence tasks to practice. Second, new learning algorithms can leverage self-generated goals to dramatically accelerate skill learning. The new paradigm will allow robots to acquire a large repertoire of flexible skills in conditions unforeseeable at design time with little human intervention, and then to exploit these skills to efficiently solve new user-defined tasks with no/little additional learning.

Link: http://www.goal-robots.eu

Print Media Article im FMSH Magazin 2019

Links

Link to the PDF.

This post discusses how to develop a low cost sensor glove with tactile feedback using flex sensors and small vibration motors. MATLAB and JAVA code is linked.

Note that this project is not longer maintained. Use the GitHub project instead.

Publications

2016
Weber, Paul; Rueckert, Elmar; Calandra, Roberto; Peters, Jan; Beckerle, Philipp A Low-cost Sensor Glove with Vibrotactile Feedback and Multiple Finger Joint and Hand Motion Sensing for Human-Robot Interaction Proceedings Article In: Proceedings of the IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), 2016. Links | BibTeX @inproceedings{ROMANS16_daglove, title = {A Low-cost Sensor Glove with Vibrotactile Feedback and Multiple Finger Joint and Hand Motion Sensing for Human-Robot Interaction}, author = {Paul Weber and Elmar Rueckert and Roberto Calandra and Jan Peters and Philipp Beckerle}, url = {https://cps.unileoben.ac.at/wp/ROMANS2016Weber.pdf, Article File https://cps.unileoben.ac.at/wp/resources/code/MATLAB_Arduino_SensorGlove_2015Rueckert.zip, MATLAB Code (JAVA Interface), ARDUINO Firmware https://cps.unileoben.ac.at/wp/resources/code/MATLAB_SensorGloveMexInterface_2015Rueckert.zip, MATLAB Code (MEX-Function Demo Interface)}, year = {2016}, date = {2016-01-01}, booktitle = {Proceedings of the IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN)}, crossref = {p10949}, key = {codyco}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Close Article File MATLAB Code (JAVA Interface) MATLAB Code (MEX-Function Demo Interface) Close
2015
Rueckert, Elmar; Lioutikov, Rudolf; Calandra, Roberto; Schmidt, Marius; Beckerle, Philipp; Peters, Jan Low-cost Sensor Glove with Force Feedback for Learning from Demonstrations using Probabilistic Trajectory Representations Proceedings Article In: ICRA 2015 Workshop on Tactile and force sensing for autonomous compliant intelligent robots, 2015. Links | BibTeX @inproceedings{Rueckert2015b, title = {Low-cost Sensor Glove with Force Feedback for Learning from Demonstrations using Probabilistic Trajectory Representations}, author = {Elmar Rueckert and Rudolf Lioutikov and Roberto Calandra and Marius Schmidt and Philipp Beckerle and Jan Peters}, url = {https://cps.unileoben.ac.at/wp/ICRA2015Rueckertb.pdf, Article File https://cps.unileoben.ac.at/wp/resources/code/MATLAB_Arduino_SensorGlove_2015Rueckert.zip, MATLAB Code (JAVA Interface), ARDUINO Firmware https://cps.unileoben.ac.at/wp/resources/code/MATLAB_SensorGloveMexInterface_2015Rueckert.zip, MATLAB Code (MEX-Function Demo Interface)}, year = {2015}, date = {2015-01-01}, booktitle = {ICRA 2015 Workshop on Tactile and force sensing for autonomous compliant intelligent robots}, crossref = {p10831}, key = {codyco}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Close Article File MATLAB Code (JAVA Interface) MATLAB Code (MEX-Function Demo Interface) Close

Details to the Hardware

• Arduino Mega 2560 Board
• Check which USB device is used (e.g., by running dmesg). On most of our machines it is /dev/ttyACM0
• Enable read/write permissions if necessary, e.g., run sudo chmod o+rw /dev/ttyACM0
• Serial protocoll based communication: Flex sensor readings are streamed and Vibration motor PWM values can be set between 0 and 255
• Firmware can be found here (follow the instructions in the README.txt to compile and upload the firmware)
• Features frame rates of up to 350Hz
• Five flex sensors provide continuous readings within the range [0, 1024]

Simple Matlab Serial Interface (max 100Hz)

• Download the Matlab demo code from here
• Tell Matlab which serial ports to use: copy the java.opts file to your Matlab bin folder, e.g., to /usr/local/MATLAB/R2012a/bin/glnxa64/
• Run FastComTest.m

Fast Mex-file based Matlab Interface – max 350Hz

• Install libserial-dev
• Compile the mex function with: mex SensorGloveInterface.cpp -lserial
• Run EventBasedSensorGloveDemo.m

The challenges in understanding human motor control, in brain-machine interfaces and anthropomorphic robotics are currently converging. Modern anthropomorphic robots with their compliant actuators and various types of sensors (e.g., depth and vision cameras, tactile fingertips, full-body skin, proprioception) have reached the perceptuomotor complexity faced in human motor control and learning. While outstanding robotic and prosthetic devices exist, current brain machine interfaces (BMIs) and robot learning methods have not yet reached the required autonomy and performance needed to enter daily life.

The groups vision is that four major challenges have to be addressed to develop truly autonomous learning systems. These are, (1) the decomposability of complex motor skills into basic primitives organized in complex architectures, (2) the ability to learn from partial observable noisy observations of inhomogeneous high-dimensional sensor data, (3) the learning of abstract features, generalizable models and transferable policies from human demonstrations, sparse rewards and through active learning, and (4), accurate predictions of self-motions, object dynamics and of humans movements for assisting and cooperating autonomous systems.

Neural models have incredible learning and modeling capabilities which was demonstrated in complex robot learning tasks (e.g., Martin Riedmiller’s or Sergey Levine’s work). While these results are promising we lack a theoretical understanding of the learning capabilities of such networks and it is unclear how learned features and models can be reused or exploited in other tasks.

The ai-lab investigates deep neural network implementations that are theoretical grounded in the framework of probabilistic inference and develops deep transfer learning strategies for stochastic neural networks. We evaluate our models in challenging robotics applications where the networks have to scale to high-dimensional control signals and need to generate reactive feedback command in real-time.

Our developments will enable complex online adaptation and skill learning behavior in autonomous systems and will help to gain a better understanding of the meaning and function of the learned features in large neural networks with millions of parameters.

Abstract

Modern robots are complex machines with many compliant actuators and various types of sensors including depth and vision cameras, tactile electrodes and dozens of proprioceptive sensors. The obvious challenges are to process these high dimensional input patterns, memorize low dimensional representations of them and to generate the desired motor commands to interact in dynamically changing environments. Similar challenges exist in brain machine interfaces (BMIs) where complex prostheses with perceptional feedback are controlled, or in motor neuroscience where in addition cognitive features need to be considered. Despite this broad research overlap the developments happened mainly in parallel and were not ported or exploited in the related domains. The main bottleneck for collaborative studies has been a lack of interaction between the core robotics, the machine learning and the neuroscience communities.

Link

Link to the workshop description at the nips webpage. 

Agenda and Speaker

Accapted Wokshop Papers

• Kyuhwa Lee, Ruslan Aydarkhanov, Luca Randazzo and José Millán. Neural Decoding of Continuous Gait Imagery from Brain Signals. (ID 2)
• Aviv Tamar, Garrett Thomas, Tianhao Zhang, Sergey Levine and Pieter Abbeel. Episodic MPC Improvement with the Hindsight Plan. (ID 11)
• Jim Mainprice, Arunkumar Bryavan, Daniel Kappler, Dieter Fox, Stefan Schaal and Nathan Ratliff. Functional manifold projections in Deep-LEARCH. (ID 12)
• Nutan Chen, Maximilian Karl and Patrick van der Smagt. Dynamic Movement Primitives in Latent Space of Time-Dependent Variational Autoencoders. (ID 1)
  → Alexander Gabriel, Riad Akrour and Gerhard Neumann. Empowered Skills. (ID 7)
• Dieter Buechler, Roberto Calandra and Jan Peters. Modeling Variability of Musculoskeletal Systems with Heteroscedastic Gaussian Processes. (ID 10)
• David Sharma, Daniel Tanneberg, Moritz Grosse-Wentrup, Jan Peters and Elmar Rueckert. Adapting Brain Signals with Reinforcement Learning Strategies for Brain Computer Interfaces. (ID 16)
• Dmytro Velychko, Benjamin Knopp and Dominik Endres. The Variational Coupled Gaussian Process Dynamical Model. (ID 5)
• Felix End, Riad Akrour and Gerhard Neumann. Layered Direct Policy Search for Learning Hierarchical Skills. (ID 6)
• Erwan Renaudo, Benoît Girard, Raja Chatila and Mehdi Khamassi. Bio-inspired habit learning in a robotic architecture. (ID 9)

Organizer

Elmar Rueckert is a postdoctoral scholar at the Intelligent Autonomous Systems lab headed by Jan Peters. He has a strong expertise in learning spiking neural networks, probabilistic planning and robot control. Before joining IAS in 2014, he has been with the Institute for Theoretical Computer Science at Graz University of Technology, where he received his Ph.D. under the supervision of Wolfgang Mass. His Thesis, “On Biologically inspired motor skill learning in robotics through probabilistic inference” concentrated on probabilistic inference for motor skill learning and on learning biologically inspired movement representations.

Martin Riedmiller joined Google DeepMind in 2015 as research scientist. He received a Diploma in Computer Science in 1992 and a PhD on Self-learning Neural Controllers in 1996 from University of Karlsruhe. He has been a professor at TU Dortmund (2002), University of Osnabrück (2003-2009), and University of Freiburg (2009-2015) where he headed the Machine Learning Lab. His general research interest is applying machine learning techniques to interesting real world problems. His RoboCup team Brainstormes won five international competitions in the 2D Simulation and MiddleSize leagues.