Publication List with Images
2026 |
|
Soni, Krishnam; Sehgal, Aditya; Dave*, Vedant; Rückert*, Elmar ALRA: Action-Latent Residual Adaptation for World Models Workshop NeurIPS 2026 Workshop on Towards Test-Time Continual Learning Agents, NeurIPS 2026 Workshop on Continual Learning in the Era of Foundation Models and Embodied Agents, NeurIPS 2026 Workshop on PTA: From Pretrained Representations to Acting Agents, 2026, (* equal supervision). Abstract | Links | BibTeX | Tags: Representation Learning, Test-time Adaptation, World Models @workshop{krishnamalra2026,Latent world models degrade when deployed under unseen dynamics shifts. Recent test-time adaptation methods such as AdaJEPA address this by fine-tuning selected sub-networks of the predictor and visual encoder online, updating on the order of 100M parameters within every episode. We show that such updates are unnecessary when the shift acts on the agent's own dynamics: a 210-parameter affine residual applied to the action embedding, with all encoders and the predictor left entirely frozen, recovers as much planning performance as sub-network fine-tuning. On PointMaze-Medium, this residual matches the strongest AdaJEPA baseline under all three shifts: 50x a damping increase, a 0.2x mass reduction, and a 10x mass increase, improving over the frozen baseline in each while leaving in-distribution performance unchanged, and updating four orders of magnitude fewer parameters. Our results suggest that dynamics shifts which rescale how commanded actions translate into motion can be corrected entirely in action space, leaving the learned visual and proprioceptive representation untouched. |  |
Gottam, Sai Puneeth Reddy; Rueckert*, Elmar; Dave*, Vedant InfiNoVA: Infinite Novel View Augmentation for Viewpoint Invariant Robot Policies Online 2026, visited: 23.09.2026, (* equal supervision). Abstract | Links | BibTeX | Tags: Gaussian Splatting, Manipulation, Representation Learning, VLA @online{puneethinfinova2026,Vision-Language-Action (VLA) policies often rely strongly on the camera viewpoints seen during training, causing substantial performance degradation when deployed from unseen perspectives. Collecting demonstrations from sufficiently diverse physical viewpoints is expensive and still provides only sparse coverage of the viewpoint space. We introduce InfiNoVA, a data-augmentation framework that converts synchronized multi-camera demonstrations into a dense distribution of geometrically consistent training views. InfiNoVA reconstructs each manipulation trajectory as a time-varying 3D Gaussian representation and renders novel observations from sampled camera poses while preserving the original state–action correspondence. This explicit scene representation improves frame-level fidelity and temporal consistency while reducing task-critical hallucinations observed in generative novel-view synthesis. Across four real-world manipulation tasks, policies trained with InfiNoVA achieve 5.4× higher average success under unseen randomized viewpoints than both VISTA-based augmentation and the unaugmented policy. InfiNoVA further achieves 1.7× higher success than training directly on all five physical camera views. These results show that dense, geometrically grounded viewpoint augmentation provides a practical route toward camera-robust robot policies without modifying the underlying policy architecture. |  |
Soni, Krishnam; Sehgal, Aditya; Dave*, Vedant; Rückert*, Elmar Sandwich-Residuals: Parameter-Efficient Test-time Adaptation of World Models Online 2026, visited: 18.09.2026, (* equal supervision). Abstract | Links | BibTeX | Tags: Representation Learning, Test-time Adaptation, World Models @online{krishnamresi2026,Latent world models enable planning by predicting the effects of actions in a learned representation space, but their predictions can become unreliable when test-time conditions differ from training. Existing test-time adaptation methods address this by updating parts of the pretrained model, often modifying millions of parameters and requiring a choice of which internal components to adapt. We introduce SandwichResiduals, a lightweight alternative that keeps the pretrained world model frozen and learns only small residual corrections around the predictor. The residuals are optimized online using the model’s self-supervised prediction error and require no rewards, labels, or source-domain data. Across 21 conditions on the AdaJEPA benchmark, our method achieves 1.3× the success rate of the frozen model while retaining 95% of the performance of the strongest AdaJEPA variant and adapting 97–99% fewer parameters. Under compound shifts, this advantage increases to 1.9× the success rate of the frozen model, while remaining comparable to internal block adaptation. We further demonstrate the same adaptation principle on a DINO-WM model for 3-D manipulation. These results suggest that effective test-time adaptation of world models does not necessarily require modifying their pretrained internal weights. |  |
Dave, Vedant; Rauch, Christian; Rückert, Elmar Task-Relevant Language-conditioned Segmentation for Robust Generalization in Reinforcement Learning Journal Article In: Transactions on Machine Learning Research, 2026. Links | BibTeX | Tags: machine learning, Reinforcement Learning, Representation Learning @article{dave2026learning, |  |
Lygerakis, Fotios; Özdenizci, Ozan; Rueckert, Elmar ViTaPEs: Visuotactile Position Encodings for Cross-Modal Alignment in Multimodal Transformers Journal Article In: Transactions on Machine Learning Research (TMLR), 2026. Links | BibTeX | Tags: Deep Learning, Representation Learning, Tactile Sensing @article{Lygerakis2026b, |  |
2025 |
|
Dave, Vedant; Özdenizci, Ozan; Rückert, Elmar Learning Robust Representations for Visual Reinforcement Learning via Task-Relevant Mask Sampling Journal Article In: Transactions on Machine Learning Research, 2025. Links | BibTeX | Tags: machine learning, Reinforcement Learning, Representation Learning @article{dave2025learning, |  |
Vanjani, Pankhuri; Mattes, Paul; Li, Maximilian Xiling; Dave, Vedant; Lioutikov, Rudolf DisDP: Robust Imitation Learning via Disentangled Diffusion Policies Proceedings Article In: Reinforcement Learning Conference (RLC), Reinforcement Learning Journal, 2025. Links | BibTeX | Tags: Contrastive Learning, Imitation Learning, Representation Learning @inproceedings{dave2025disdp, |  |
2024 |
|
Dave, Vedant; Rueckert, Elmar Denoised Predictive Imagination: An Information-theoretic approach for learning World Models Conference European Workshop on Reinforcement Learning (EWRL), 2024. Abstract | Links | BibTeX | Tags: computer vision, Contrastive Learning, Reinforcement Learning, Representation Learning @conference{dpidave2024,Humans excel at isolating relevant information from noisy data to predict the behavior of dynamic systems, effectively disregarding non-informative, temporally-correlated noise. In contrast, existing reinforcement learning algorithms face challenges in generating noise-free predictions within high-dimensional, noise-saturated environments, especially when trained on world models featuring realistic background noise extracted from natural video streams. We propose a novel information-theoretic approach that learn world models based on minimising the past information and retaining maximal information about the future, aiming at simultaneously learning control policies and at producing denoised predictions. Utilizing Soft Actor-Critic agents augmented with an information-theoretic auxiliary loss, we validate our method's effectiveness on complex variants of the standard DeepMind Control Suite tasks, where natural videos filled with intricate and task-irrelevant information serve as a background. Experimental results demonstrate that our model outperforms nine state-of-the-art approaches in various settings where natural videos serve as dynamic background noise. Our analysis also reveals that all these methods encounter challenges in more complex environments. |  |
Dave*, Vedant; Lygerakis*, Fotios; Rueckert, Elmar Multimodal Visual-Tactile Representation Learning through Self-Supervised Contrastive Pre-Training Proceedings Article In: IEEE International Conference on Robotics and Automation (ICRA), pp. 8013-8020, IEEE, 2024, ISBN: 979-8-3503-8457-4, (* equal contribution). Abstract | Links | BibTeX | Tags: Contrastive Learning, Representation Learning, Self-supervised Learning, Tactile Sensing @inproceedings{Dave2024b,The rapidly evolving field of robotics necessitates methods that can facilitate the fusion of multiple modalities. Specifically, when it comes to interacting with tangible objects, effectively combining visual and tactile sensory data is key to understanding and navigating the complex dynamics of the physical world, enabling a more nuanced and adaptable response to changing environments. Nevertheless, much of the earlier work in merging these two sensory modalities has relied on supervised methods utilizing datasets labeled by humans. This paper introduces MViTac, a novel methodology that leverages contrastive learning to integrate vision and touch sensations in a self-supervised fashion. By availing both sensory inputs, MViTac leverages intra and inter-modality losses for learning representations, resulting in enhanced material property classification and more adept grasping prediction. Through a series of experiments, we showcase the effectiveness of our method and its superiority over existing state-of-the-art self-supervised and supervised techniques. In evaluating our methodology, we focus on two distinct tasks: material classification and grasping success prediction. Our results indicate that MViTac facilitates the development of improved modality encoders, yielding more robust representations as evidenced by linear probing assessments. https://sites.google.com/view/mvitac/home |  |
2023 |
|
Lygerakis, Fotios; Rueckert, Elmar CR-VAE: Contrastive Regularization on Variational Autoencoders for Preventing Posterior Collapse Proceedings Article In: Asian Conference of Artificial Intelligence Technology (ACAIT)., IEEE, 2023. Links | BibTeX | Tags: Representation Learning, Self-supervised Learning @inproceedings{Lygerakis2023, |  |
Compact List without Images
Journal Articles |
Dave, Vedant; Rauch, Christian; Rückert, Elmar Task-Relevant Language-conditioned Segmentation for Robust Generalization in Reinforcement Learning Journal Article In: Transactions on Machine Learning Research, 2026. @article{dave2026learning, |
Lygerakis, Fotios; Özdenizci, Ozan; Rueckert, Elmar ViTaPEs: Visuotactile Position Encodings for Cross-Modal Alignment in Multimodal Transformers Journal Article In: Transactions on Machine Learning Research (TMLR), 2026. @article{Lygerakis2026b, |
Dave, Vedant; Özdenizci, Ozan; Rückert, Elmar Learning Robust Representations for Visual Reinforcement Learning via Task-Relevant Mask Sampling Journal Article In: Transactions on Machine Learning Research, 2025. @article{dave2025learning, |
Conferences |
Dave, Vedant; Rueckert, Elmar Denoised Predictive Imagination: An Information-theoretic approach for learning World Models Conference European Workshop on Reinforcement Learning (EWRL), 2024. @conference{dpidave2024,Humans excel at isolating relevant information from noisy data to predict the behavior of dynamic systems, effectively disregarding non-informative, temporally-correlated noise. In contrast, existing reinforcement learning algorithms face challenges in generating noise-free predictions within high-dimensional, noise-saturated environments, especially when trained on world models featuring realistic background noise extracted from natural video streams. We propose a novel information-theoretic approach that learn world models based on minimising the past information and retaining maximal information about the future, aiming at simultaneously learning control policies and at producing denoised predictions. Utilizing Soft Actor-Critic agents augmented with an information-theoretic auxiliary loss, we validate our method's effectiveness on complex variants of the standard DeepMind Control Suite tasks, where natural videos filled with intricate and task-irrelevant information serve as a background. Experimental results demonstrate that our model outperforms nine state-of-the-art approaches in various settings where natural videos serve as dynamic background noise. Our analysis also reveals that all these methods encounter challenges in more complex environments. |
Proceedings Articles |
Vanjani, Pankhuri; Mattes, Paul; Li, Maximilian Xiling; Dave, Vedant; Lioutikov, Rudolf DisDP: Robust Imitation Learning via Disentangled Diffusion Policies Proceedings Article In: Reinforcement Learning Conference (RLC), Reinforcement Learning Journal, 2025. @inproceedings{dave2025disdp, |
Dave*, Vedant; Lygerakis*, Fotios; Rueckert, Elmar Multimodal Visual-Tactile Representation Learning through Self-Supervised Contrastive Pre-Training Proceedings Article In: IEEE International Conference on Robotics and Automation (ICRA), pp. 8013-8020, IEEE, 2024, ISBN: 979-8-3503-8457-4, (* equal contribution). @inproceedings{Dave2024b,The rapidly evolving field of robotics necessitates methods that can facilitate the fusion of multiple modalities. Specifically, when it comes to interacting with tangible objects, effectively combining visual and tactile sensory data is key to understanding and navigating the complex dynamics of the physical world, enabling a more nuanced and adaptable response to changing environments. Nevertheless, much of the earlier work in merging these two sensory modalities has relied on supervised methods utilizing datasets labeled by humans. This paper introduces MViTac, a novel methodology that leverages contrastive learning to integrate vision and touch sensations in a self-supervised fashion. By availing both sensory inputs, MViTac leverages intra and inter-modality losses for learning representations, resulting in enhanced material property classification and more adept grasping prediction. Through a series of experiments, we showcase the effectiveness of our method and its superiority over existing state-of-the-art self-supervised and supervised techniques. In evaluating our methodology, we focus on two distinct tasks: material classification and grasping success prediction. Our results indicate that MViTac facilitates the development of improved modality encoders, yielding more robust representations as evidenced by linear probing assessments. https://sites.google.com/view/mvitac/home |
Lygerakis, Fotios; Rueckert, Elmar CR-VAE: Contrastive Regularization on Variational Autoencoders for Preventing Posterior Collapse Proceedings Article In: Asian Conference of Artificial Intelligence Technology (ACAIT)., IEEE, 2023. @inproceedings{Lygerakis2023, |
Online |
Gottam, Sai Puneeth Reddy; Rueckert*, Elmar; Dave*, Vedant InfiNoVA: Infinite Novel View Augmentation for Viewpoint Invariant Robot Policies Online 2026, visited: 23.09.2026, (* equal supervision). @online{puneethinfinova2026,Vision-Language-Action (VLA) policies often rely strongly on the camera viewpoints seen during training, causing substantial performance degradation when deployed from unseen perspectives. Collecting demonstrations from sufficiently diverse physical viewpoints is expensive and still provides only sparse coverage of the viewpoint space. We introduce InfiNoVA, a data-augmentation framework that converts synchronized multi-camera demonstrations into a dense distribution of geometrically consistent training views. InfiNoVA reconstructs each manipulation trajectory as a time-varying 3D Gaussian representation and renders novel observations from sampled camera poses while preserving the original state–action correspondence. This explicit scene representation improves frame-level fidelity and temporal consistency while reducing task-critical hallucinations observed in generative novel-view synthesis. Across four real-world manipulation tasks, policies trained with InfiNoVA achieve 5.4× higher average success under unseen randomized viewpoints than both VISTA-based augmentation and the unaugmented policy. InfiNoVA further achieves 1.7× higher success than training directly on all five physical camera views. These results show that dense, geometrically grounded viewpoint augmentation provides a practical route toward camera-robust robot policies without modifying the underlying policy architecture. |
Soni, Krishnam; Sehgal, Aditya; Dave*, Vedant; Rückert*, Elmar Sandwich-Residuals: Parameter-Efficient Test-time Adaptation of World Models Online 2026, visited: 18.09.2026, (* equal supervision). @online{krishnamresi2026,Latent world models enable planning by predicting the effects of actions in a learned representation space, but their predictions can become unreliable when test-time conditions differ from training. Existing test-time adaptation methods address this by updating parts of the pretrained model, often modifying millions of parameters and requiring a choice of which internal components to adapt. We introduce SandwichResiduals, a lightweight alternative that keeps the pretrained world model frozen and learns only small residual corrections around the predictor. The residuals are optimized online using the model’s self-supervised prediction error and require no rewards, labels, or source-domain data. Across 21 conditions on the AdaJEPA benchmark, our method achieves 1.3× the success rate of the frozen model while retaining 95% of the performance of the strongest AdaJEPA variant and adapting 97–99% fewer parameters. Under compound shifts, this advantage increases to 1.9× the success rate of the frozen model, while remaining comparable to internal block adaptation. We further demonstrate the same adaptation principle on a DINO-WM model for 3-D manipulation. These results suggest that effective test-time adaptation of world models does not necessarily require modifying their pretrained internal weights. |
Workshops |
Soni, Krishnam; Sehgal, Aditya; Dave*, Vedant; Rückert*, Elmar ALRA: Action-Latent Residual Adaptation for World Models Workshop NeurIPS 2026 Workshop on Towards Test-Time Continual Learning Agents, NeurIPS 2026 Workshop on Continual Learning in the Era of Foundation Models and Embodied Agents, NeurIPS 2026 Workshop on PTA: From Pretrained Representations to Acting Agents, 2026, (* equal supervision). @workshop{krishnamalra2026,Latent world models degrade when deployed under unseen dynamics shifts. Recent test-time adaptation methods such as AdaJEPA address this by fine-tuning selected sub-networks of the predictor and visual encoder online, updating on the order of 100M parameters within every episode. We show that such updates are unnecessary when the shift acts on the agent's own dynamics: a 210-parameter affine residual applied to the action embedding, with all encoders and the predictor left entirely frozen, recovers as much planning performance as sub-network fine-tuning. On PointMaze-Medium, this residual matches the strongest AdaJEPA baseline under all three shifts: 50x a damping increase, a 0.2x mass reduction, and a 10x mass increase, improving over the frozen baseline in each while leaving in-distribution performance unchanged, and updating four orders of magnitude fewer parameters. Our results suggest that dynamics shifts which rescale how commanded actions translate into motion can be corrected entirely in action space, leaving the learned visual and proprioceptive representation untouched. |