Publications
|
Publication List with Images
2024 |
|
Nwankwo, Linus; Rueckert, Elmar 2024, ( In Workshop of the 2024 ACM/IEEE International Conference on HumanRobot Interaction (HRI ’24 Workshop), March 11–14, 2024, Boulder, CO, USA. ACM, New York, NY, USA). Abstract | Links | BibTeX | Tags: Autonomous Navigation, Human-Robot Interaction, Large Language Models, mobile navigation @workshop{Nwankwo2024MultimodalHA,In this paper, we extended the method proposed in [17] to enable humans to interact naturally with autonomous agents through vocal and textual conversations. Our extended method exploits the inherent capabilities of pre-trained large language models (LLMs), multimodal visual language models (VLMs), and speech recognition (SR) models to decode the high-level natural language conversations and semantic understanding of the robot's task environment, and abstract them to the robot's actionable commands or queries. We performed a quantitative evaluation of our framework's natural vocal conversation understanding with participants from different racial backgrounds and English language accents. The participants interacted with the robot using both vocal and textual instructional commands. Based on the logged interaction data, our framework achieved 87.55% vocal commands decoding accuracy, 86.27% commands execution success, and an average latency of 0.89 seconds from receiving the participants' vocal chat commands to initiating the robot’s actual physical action. The video demonstrations of this paper can be found at https://linusnep.github.io/MTCC-IRoNL/ |  |
2023 |
|
Nwankwo, Linus; Rueckert, Elmar Understanding why SLAM algorithms fail in modern indoor environments Proceedings Article In: International Conference on Robotics in Alpe-Adria-Danube Region (RAAD). , pp. 186 - 194, Cham: Springer Nature Switzerland., 2023. Abstract | Links | BibTeX | Tags: mobile navigation, robotics, SLAM @inproceedings{Nwankwo2023,Simultaneous localization and mapping (SLAM) algorithms are essential for the autonomous navigation of mobile robots. With the increasing demand for autonomous systems, it is crucial to evaluate and compare the performance of these algorithms in real-world environments. In this paper, we provide an evaluation strategy and real-world datasets to test and evaluate SLAM algorithms in complex and challenging indoor environments. Further, we analysed state-of-the-art (SOTA) SLAM algorithms based on various metrics such as absolute trajectory error, scale drift, and map accuracy and consistency. Our results demonstrate that SOTA SLAM algorithms often fail in challenging environments, with dynamic objects, transparent and reflecting surfaces. We also found that successful loop closures had a significant impact on the algorithm’s performance. These findings highlight the need for further research to improve the robustness of the algorithms in real-world scenarios. |  |
Nwankwo, Linus; Fritze, Clemens; Bartsch, Konrad; Rueckert, Elmar ROMR: A ROS-based Open-source Mobile Robot Journal Article In: HardwareX, vol. 15, pp. 1–29, 2023. Abstract | Links | BibTeX | Tags: Autonomous Navigation, mobile navigation, SLAM @article{Nwankwo2023b,Currently, commercially available intelligent transport robots that are capable of carrying up to 90kg of load can cost $5,000 or even more. This makes real-world experimentation prohibitively expensive, and limiting the applicability of such systems to everyday home or industrial tasks. Aside from their high cost, the majority of commercially available platforms are either closed-source, platform-specific, or use difficult-to-customize hardware and firmware. In this work, we present a low-cost, open-source and modular alternative, referred to herein as ”ROS-based open-source mobile robot (ROMR)”. ROMR utilizes off-the-shelf (OTS) components, additive manufacturing technologies, aluminium profiles, and a consumer hoverboard with high-torque brushless direct current (BLDC) motors. ROMR is fully compatible with the robot operating system (ROS), has a maximum payload of 90kg, and costs less than $1500. Furthermore, ROMR offers a simple yet robust framework for contextualizing simultaneous localization and mapping (SLAM) algorithms, an essential prerequisite for autonomous robot navigation. The robustness and performance of the ROMR were validated through realworld and simulation experiments. All the design, construction and software files are freely available online under the GNU GPL v3 license at https://doi.org/10.17605/OSF.IO/K83X7. A descriptive video of ROMR can be found at https://osf.io/ku8ag. |  |
Yadav, Harsh; Xue, Honghu; Rudall, Yan; Bakr, Mohamed; Hein, Benedikt; Rueckert, Elmar; Nguyen, Thinh Deep Reinforcement Learning for Autonomous Navigation in Intralogistics Workshop 2023, (European Control Conference (ECC) Workshop, Extended Abstract.). Abstract | Links | BibTeX | Tags: Autonomous Navigation, Deep Learning, mobile navigation, SLAM @workshop{Yadav2023,Even with several advances in autonomous mobile robots, navigation in a highly dynamic environment still remains a challenge. Classical navigation systems, such as Simultaneous Localization and Mapping (SLAM), build a map of the environment and constructing maps of highly dynamic environments is impractical. Deep Reinforcement Learning (DRL) approaches have the ability to learn policies without knowledge of the maps or the transition models of the environment. The aim of our work is to investigate the potential of using DRL to control an autonomous mobile robot to dock with a load carrier. This paper presents an initial successful training result of the Soft Actor-Critic (SAC) algorithm, which can navigate a robot toward an open door only based on the 360° LiDAR observations. Ongoing work is using visual sensors for load carrier docking. |  |
2022 |
|
Xue, Honghu; Song, Rui; Petzold, Julian; Hein, Benedikt; Hamann, Heiko; Rueckert, Elmar End-To-End Deep Reinforcement Learning for First-Person Pedestrian Visual Navigation in Urban Environments Proceedings Article In: International Conference on Humanoid Robots (Humanoids 2022), 2022. Abstract | Links | BibTeX | Tags: Autonomous Navigation, Deep Learning, mobile navigation @inproceedings{Xue2022b,We solve a visual navigation problem in an urban setting via deep reinforcement learning in an end-to-end manner. A major challenge of a first-person visual navigation problem lies in severe partial observability and sparse positive experiences of reaching the goal. To address partial observability, we propose a novel 3D-temporal convolutional network to encode sequential historical visual observations, its effectiveness is verified by comparing to a commonly-used frame-stacking approach. For sparse positive samples, we propose an improved automatic curriculum learning algorithm NavACL+, which proposes meaningful curricula starting from easy tasks and gradually generalizes to challenging ones. NavACL+ is shown to facilitate the learning process, greatly improve the task success rate on difficult tasks by at least 40% and offer enhanced generalization to different initial poses compared to training from a fixed initial pose and the original NavACL algorithm. |  |
Rottmann, Nils; Studt, Nico; Ernst, Floris; Rueckert, Elmar ROS-Mobile: An Android™ application for the Robot Operating System Journal Article In: Arxiv, 2022. Links | BibTeX | Tags: Autonomous Navigation, mobile navigation, Simulation @article{Rottmann2022, |  |
Xue, Honghu; Hein, Benedikt; Bakr, Mohamed; Schildbach, Georg; Abel, Bengt; Rueckert, Elmar Using Deep Reinforcement Learning with Automatic Curriculum Learning for Mapless Navigation in Intralogistics Journal Article In: Applied Sciences (MDPI), Special Issue on Intelligent Robotics, 2022, (Supplement: https://cloud.cps.unileoben.ac.at/index.php/s/Sj68rQewnkf4ppZ). Abstract | Links | BibTeX | Tags: Autonomous Navigation, Deep Learning, mobile navigation, Reinforcement Learning @article{Xue2022,We propose a deep reinforcement learning approach for solving a mapless navigation problem in warehouse scenarios. The automatic guided vehicle is equipped with LiDAR and frontal RGB sensors and learns to reach underneath the target dolly. The challenges reside in the sparseness of positive samples for learning, multi-modal sensor perception with partial observability, the demand for accurate steering maneuvers together with long training cycles. To address these points, we proposed NavACL-Q as an automatic curriculum learning together with distributed soft actor-critic. The performance of the learning algorithm is evaluated exhaustively in a different warehouse environment to check both robustness and generalizability of the learned policy. Results in NVIDIA Isaac Sim demonstrates that our trained agent significantly outperforms the map-based navigation pipeline provided by NVIDIA Isaac Sim in terms of higher agent-goal distances and relative orientations. The ablation studies also confirmed that NavACL-Q greatly facilitates the whole learning process and a pre-trained feature extractor manifestly boosts the training speed. |  |
2021 |
|
Rottmann, N.; Denz, R.; Bruder, R.; Rueckert, E. Probabilistic Approach for Complete Coverage Path Planning with low-cost Systems Proceedings Article In: European Conference on Mobile Robots (ECMR 2021), 2021. Links | BibTeX | Tags: mobile navigation, Probabilistic Inference @inproceedings{Rottmann2021, |  |
2020 |
|
Rottmann, N.; Bruder, R.; Schweikard, A.; Rueckert, E. Exploiting Chlorophyll Fluorescense for Building Robust low-Cost Mowing Area Detectors Proceedings Article In: IEEE SENSORS , pp. 1–4, 2020. Links | BibTeX | Tags: mobile navigation, smart sensors @inproceedings{Rottmann2020b, |  |
Rottmann, N.; Bruder, R.; Xue, H.; Schweikard, A.; Rueckert, E. Parameter Optimization for Loop Closure Detection in Closed Environments Proceedings Article In: Workshop Paper at the International Conference on Intelligent Robots and Systems (IROS), pp. 1–8, 2020. Links | BibTeX | Tags: mobile navigation, Reinforcement Learning @inproceedings{Rottmann2020c, |  |
Rottmann, N.; Bruder, R.; Schweikard, A.; Rueckert, E. A novel Chlorophyll Fluorescence based approach for Mowing Area Classification Journal Article In: IEEE Sensors Journal, 2020. Links | BibTeX | Tags: mobile navigation, smart sensors @article{Rottmann2020d, |  |
2019 |
|
Rottmann, N.; Bruder, R.; Schweikard, A.; Rueckert, E. Loop Closure Detection in Closed Environments Proceedings Article In: European Conference on Mobile Robots (ECMR 2019), 2019, ISBN: 978-1-7281-3605-9. Links | BibTeX | Tags: mobile navigation @inproceedings{Rottmann2019b, |  |
Rottmann, N.; Bruder, R.; Schweikard, A.; Rueckert, E. Cataglyphis ant navigation strategies solve the global localization problem in robots with binary sensors Proceedings Article In: Proceedings of International Conference on Bio-inspired Systems and Signal Processing (BIOSIGNALS), Prague, Czech Republic , 2019, ( February 22-24, 2019). Links | BibTeX | Tags: constraint optimization, mobile navigation, Simulation @inproceedings{Rottmann2019, |  |
Compact List without Images
Journal Articles |
Nwankwo, Linus; Fritze, Clemens; Bartsch, Konrad; Rueckert, Elmar ROMR: A ROS-based Open-source Mobile Robot Journal Article In: HardwareX, vol. 15, pp. 1–29, 2023. @article{Nwankwo2023b,Currently, commercially available intelligent transport robots that are capable of carrying up to 90kg of load can cost $5,000 or even more. This makes real-world experimentation prohibitively expensive, and limiting the applicability of such systems to everyday home or industrial tasks. Aside from their high cost, the majority of commercially available platforms are either closed-source, platform-specific, or use difficult-to-customize hardware and firmware. In this work, we present a low-cost, open-source and modular alternative, referred to herein as ”ROS-based open-source mobile robot (ROMR)”. ROMR utilizes off-the-shelf (OTS) components, additive manufacturing technologies, aluminium profiles, and a consumer hoverboard with high-torque brushless direct current (BLDC) motors. ROMR is fully compatible with the robot operating system (ROS), has a maximum payload of 90kg, and costs less than $1500. Furthermore, ROMR offers a simple yet robust framework for contextualizing simultaneous localization and mapping (SLAM) algorithms, an essential prerequisite for autonomous robot navigation. The robustness and performance of the ROMR were validated through realworld and simulation experiments. All the design, construction and software files are freely available online under the GNU GPL v3 license at https://doi.org/10.17605/OSF.IO/K83X7. A descriptive video of ROMR can be found at https://osf.io/ku8ag. |
Rottmann, Nils; Studt, Nico; Ernst, Floris; Rueckert, Elmar ROS-Mobile: An Android™ application for the Robot Operating System Journal Article In: Arxiv, 2022. @article{Rottmann2022, |
Xue, Honghu; Hein, Benedikt; Bakr, Mohamed; Schildbach, Georg; Abel, Bengt; Rueckert, Elmar Using Deep Reinforcement Learning with Automatic Curriculum Learning for Mapless Navigation in Intralogistics Journal Article In: Applied Sciences (MDPI), Special Issue on Intelligent Robotics, 2022, (Supplement: https://cloud.cps.unileoben.ac.at/index.php/s/Sj68rQewnkf4ppZ). @article{Xue2022,We propose a deep reinforcement learning approach for solving a mapless navigation problem in warehouse scenarios. The automatic guided vehicle is equipped with LiDAR and frontal RGB sensors and learns to reach underneath the target dolly. The challenges reside in the sparseness of positive samples for learning, multi-modal sensor perception with partial observability, the demand for accurate steering maneuvers together with long training cycles. To address these points, we proposed NavACL-Q as an automatic curriculum learning together with distributed soft actor-critic. The performance of the learning algorithm is evaluated exhaustively in a different warehouse environment to check both robustness and generalizability of the learned policy. Results in NVIDIA Isaac Sim demonstrates that our trained agent significantly outperforms the map-based navigation pipeline provided by NVIDIA Isaac Sim in terms of higher agent-goal distances and relative orientations. The ablation studies also confirmed that NavACL-Q greatly facilitates the whole learning process and a pre-trained feature extractor manifestly boosts the training speed. |
Rottmann, N.; Bruder, R.; Schweikard, A.; Rueckert, E. A novel Chlorophyll Fluorescence based approach for Mowing Area Classification Journal Article In: IEEE Sensors Journal, 2020. @article{Rottmann2020d, |
Proceedings Articles |
Nwankwo, Linus; Rueckert, Elmar Understanding why SLAM algorithms fail in modern indoor environments Proceedings Article In: International Conference on Robotics in Alpe-Adria-Danube Region (RAAD). , pp. 186 - 194, Cham: Springer Nature Switzerland., 2023. @inproceedings{Nwankwo2023,Simultaneous localization and mapping (SLAM) algorithms are essential for the autonomous navigation of mobile robots. With the increasing demand for autonomous systems, it is crucial to evaluate and compare the performance of these algorithms in real-world environments. In this paper, we provide an evaluation strategy and real-world datasets to test and evaluate SLAM algorithms in complex and challenging indoor environments. Further, we analysed state-of-the-art (SOTA) SLAM algorithms based on various metrics such as absolute trajectory error, scale drift, and map accuracy and consistency. Our results demonstrate that SOTA SLAM algorithms often fail in challenging environments, with dynamic objects, transparent and reflecting surfaces. We also found that successful loop closures had a significant impact on the algorithm’s performance. These findings highlight the need for further research to improve the robustness of the algorithms in real-world scenarios. |
Xue, Honghu; Song, Rui; Petzold, Julian; Hein, Benedikt; Hamann, Heiko; Rueckert, Elmar End-To-End Deep Reinforcement Learning for First-Person Pedestrian Visual Navigation in Urban Environments Proceedings Article In: International Conference on Humanoid Robots (Humanoids 2022), 2022. @inproceedings{Xue2022b,We solve a visual navigation problem in an urban setting via deep reinforcement learning in an end-to-end manner. A major challenge of a first-person visual navigation problem lies in severe partial observability and sparse positive experiences of reaching the goal. To address partial observability, we propose a novel 3D-temporal convolutional network to encode sequential historical visual observations, its effectiveness is verified by comparing to a commonly-used frame-stacking approach. For sparse positive samples, we propose an improved automatic curriculum learning algorithm NavACL+, which proposes meaningful curricula starting from easy tasks and gradually generalizes to challenging ones. NavACL+ is shown to facilitate the learning process, greatly improve the task success rate on difficult tasks by at least 40% and offer enhanced generalization to different initial poses compared to training from a fixed initial pose and the original NavACL algorithm. |
Rottmann, N.; Denz, R.; Bruder, R.; Rueckert, E. Probabilistic Approach for Complete Coverage Path Planning with low-cost Systems Proceedings Article In: European Conference on Mobile Robots (ECMR 2021), 2021. @inproceedings{Rottmann2021, |
Rottmann, N.; Bruder, R.; Schweikard, A.; Rueckert, E. Exploiting Chlorophyll Fluorescense for Building Robust low-Cost Mowing Area Detectors Proceedings Article In: IEEE SENSORS , pp. 1–4, 2020. @inproceedings{Rottmann2020b, |
Rottmann, N.; Bruder, R.; Xue, H.; Schweikard, A.; Rueckert, E. Parameter Optimization for Loop Closure Detection in Closed Environments Proceedings Article In: Workshop Paper at the International Conference on Intelligent Robots and Systems (IROS), pp. 1–8, 2020. @inproceedings{Rottmann2020c, |
Rottmann, N.; Bruder, R.; Schweikard, A.; Rueckert, E. Loop Closure Detection in Closed Environments Proceedings Article In: European Conference on Mobile Robots (ECMR 2019), 2019, ISBN: 978-1-7281-3605-9. @inproceedings{Rottmann2019b, |
Rottmann, N.; Bruder, R.; Schweikard, A.; Rueckert, E. Cataglyphis ant navigation strategies solve the global localization problem in robots with binary sensors Proceedings Article In: Proceedings of International Conference on Bio-inspired Systems and Signal Processing (BIOSIGNALS), Prague, Czech Republic , 2019, ( February 22-24, 2019). @inproceedings{Rottmann2019, |
Workshops |
Nwankwo, Linus; Rueckert, Elmar 2024, ( In Workshop of the 2024 ACM/IEEE International Conference on HumanRobot Interaction (HRI ’24 Workshop), March 11–14, 2024, Boulder, CO, USA. ACM, New York, NY, USA). @workshop{Nwankwo2024MultimodalHA,In this paper, we extended the method proposed in [17] to enable humans to interact naturally with autonomous agents through vocal and textual conversations. Our extended method exploits the inherent capabilities of pre-trained large language models (LLMs), multimodal visual language models (VLMs), and speech recognition (SR) models to decode the high-level natural language conversations and semantic understanding of the robot's task environment, and abstract them to the robot's actionable commands or queries. We performed a quantitative evaluation of our framework's natural vocal conversation understanding with participants from different racial backgrounds and English language accents. The participants interacted with the robot using both vocal and textual instructional commands. Based on the logged interaction data, our framework achieved 87.55% vocal commands decoding accuracy, 86.27% commands execution success, and an average latency of 0.89 seconds from receiving the participants' vocal chat commands to initiating the robot’s actual physical action. The video demonstrations of this paper can be found at https://linusnep.github.io/MTCC-IRoNL/ |
Yadav, Harsh; Xue, Honghu; Rudall, Yan; Bakr, Mohamed; Hein, Benedikt; Rueckert, Elmar; Nguyen, Thinh Deep Reinforcement Learning for Autonomous Navigation in Intralogistics Workshop 2023, (European Control Conference (ECC) Workshop, Extended Abstract.). @workshop{Yadav2023,Even with several advances in autonomous mobile robots, navigation in a highly dynamic environment still remains a challenge. Classical navigation systems, such as Simultaneous Localization and Mapping (SLAM), build a map of the environment and constructing maps of highly dynamic environments is impractical. Deep Reinforcement Learning (DRL) approaches have the ability to learn policies without knowledge of the maps or the transition models of the environment. The aim of our work is to investigate the potential of using DRL to control an autonomous mobile robot to dock with a load carrier. This paper presents an initial successful training result of the Soft Actor-Critic (SAC) algorithm, which can navigate a robot toward an open door only based on the 360° LiDAR observations. Ongoing work is using visual sensors for load carrier docking. |