Robotics
HybridMimic: Hybrid RL-Centroidal Control for Humanoid Motion Mimicking
Ludwig Chee-Ying Tay, I-Chia Chang and Yan Gu
Motion mimicking, i.e., encouraging the control policy to mimic human motion, facilitates the learning of complex tasks via reinforcement learning (RL) for
humanoid robots. Although standard RL frameworks demonstrate impressive locomotion agility, they often bypass explicit
reasoning about
robot dynamics during deployment, which is a design choice that can lead to physically infeasible commands when the
robot encounters out-of-distribution environments. By integrating model-based principles, hybrid approaches can improve performance; however, existing methods typically rely on predefined contact timing, limiting their versatility. This paper introduces HybridMimic, a framework in which a learned policy dynamically modulates a centroidal-model-based controller by predicting continuous contact states and desired centroidal velocities. This architecture exploits the physical grounding of centroidal dynamics to generate feedforward torques that remain feasible even under domain shift. Using physics-informed rewards, the policy is trained to efficiently utilize the centroidal controller's optimization by outputting precise control targets and reference torques. Through hardware experiments on the Booster T1
humanoid, HybridMimic reduces the average base position tracking error by 13% compared to a state-of-the-art RL baseline, demonstrating the robustness of dynamics-aware deployment.
Read the full abstract ▾
Robotics
Stability-Guided Exploration for Diverse Motion Generation
Eckart Cobo-Briesewitz, Tilman Burghoff, Denis Shcherba et al. (+2 more)
Scaling up
datasets is highly effective in improving the performance of deep learning models, including in the field of
robot learning. However, data collection still proves to be a bottleneck. Approaches relying on collecting human demonstrations are labor-intensive and inherently limited: they tend to be narrow, task-specific, and fail to adequately explore the full space of feasible states. Synthetic data generation could remedy this, but current techniques mostly rely on local trajectory optimization and fail to find diverse solutions. In this work, we propose a novel method capable of finding diverse long-horizon manipulations through black-box simulation. We achieve this by combining an RRT-style search with sampling-based MPC, together with a novel sampling scheme that guides the exploration toward stable configurations. Specifically, we sample from a
manifold of stable states while growing a search tree directly through simulation, without restricting the planner to purely stable motions. We demonstrate the method's ability to discover diverse manipulation strategies, including pushing, grasping, pivoting, throwing, and tool use, across different
robot morphologies, without task-specific guidance.
Read the full abstract ▾
All authors Eckart Cobo-Briesewitz, Tilman Burghoff, Denis Shcherba, Armand Jordana, Marc Toussaint
Robotics
Neural-Surrogate Co-Design of Aerodynamic Shape and Control
Daniele Affinita, Mingda Xu, Benoît Valentin Gherardi et al. (+1 more)
Control co-design jointly optimizes a
robot's physical design and its controller. Existing techniques are limited in the number of model parameters and the complexity of the physical models they can handle. In this paper, we propose a gradient-based bilevel formulation that replaces the dynamics models of earlier techniques with a differentiable neural surrogate and differentiates through the trajectory optimizer using a structure-exploiting approach. We validate the framework on fixed-wing gliders, using NeuralFoil as the aerodynamic surrogate. The tasks are maximum-distance gliding, minimum-distance landing, and perching. Experiments on 2D airfoils and 3D wings show that our method runs about five times faster than a state-of-the-art evolutionary baseline. It also outperforms pipelines that optimize aerodynamic proxy metrics.
Read the full abstract ▾
All authors Daniele Affinita, Mingda Xu, Benoît Valentin Gherardi, Pascal Fua
Robotics
Robotic Foundation Models for Industrial Control: A Comprehensive Survey and Readiness Assessment Framework
David Kube, Simon Hadwiger and Tobias Meisen
Robotic foundation models (RFMs) are emerging as a promising route towards flexible, instruction- and demonstration-driven
robot control, however, a critical investigation of their industrial applicability is still lacking. This survey gives an extensive overview over the RFM-landscape and analyses, driven by concrete implications, how industrial domains and use cases shape the requirements of RFMs, with particular focus on collaborative
robot platforms, heterogeneous sensing and actuation, edge-computing constraints, and safety-critical operation. We synthesise industrial deployment perspectives into eleven interdependent implications and operationalise them into an assessment framework comprising a catalogue of 149 concrete criteria, spanning both model capabilities and ecosystem requirements. Using this framework, we evaluate 324 manipulation-capable RFMs via 48,276 criterion-level decisions obtained via a conservative LLM-assisted evaluation pipeline, validated against expert judgements. The results indicate that industrial maturity is limited and uneven: even the highest-rated models satisfy only a fraction of criteria and typically exhibit narrow implication-specific peaks rather than integrated coverage. We conclude that progress towards industry-grade RFMs depends less on isolated
benchmark successes than on systematic incorporation of safety, real-time feasibility, robust perception, interaction, and cost-effective system integration into auditable deployment stacks.
Read the full abstract ▾
Robotics · Cross-listed from cs.LG
Improved Constrained Generation by Bridging Pretrained Generative Models
Xiaoxuan Liang, Saeid Naderiparizi, Yunpeng Liu et al. (+2 more)
Constrained generative modeling is fundamental to applications such as robotic control and autonomous driving, where models must respect physical laws and safety-critical constraints. In real-world settings, these constraints rarely take the form of simple linear inequalities, but instead complex feasible regions that resemble road maps or other structured spatial domains. We propose a constrained generation framework that generates samples directly within such feasible regions while preserving realism. Our method fine-tunes a pretrained generative model to enforce constraints while maintaining generative fidelity. Experimentally, our method exhibits characteristics distinct from existing fine-tuning and training-free constrained baselines, revealing a new compromise between constraint satisfaction and sampling quality.
Read the full abstract ▾
All authors Xiaoxuan Liang, Saeid Naderiparizi, Yunpeng Liu, Berend Zwartsenberg, Frank Wood
Robotics · Cross-listed from cs.LG
Don't Freeze, Don't Crash: Extending the Safe Operating Range of Neural Navigation in Dense Crowds
Jiefu Zhang, Yang Xu and Vaneet Aggarwal
Navigating safely through dense crowds requires collision avoidance that generalizes beyond the densities seen during training. Learning-based crowd
navigation can break under out-of-distribution crowd sizes due to density-sensitive observation normalization and social-cost scaling, while analytical solvers often remain safe but freeze in tight interactions. We propose a reinforcement learning approach for dense, variable-density
navigation that attains zero-shot density generalization using a density-invariant observation encoding with density-randomized training and physics-informed proxemic reward shaping with density-adaptive scaling. The encoding represents the distance-sorted $K$ nearest pedestrians plus bounded crowd summaries, keeping input statistics stable as crowd size grows. Trained with $N\!\in\![11,16]$ pedestrians in a $3\mathrm{m}\times3\mathrm{m}$ arena and evaluated up to $N\!=\!21$ pedestrians ($1.3\times$ denser), our policy reaches the goal in $>99\%$ of episodes and achieves $86\%$ collision-free success in random crowds, with markedly less freezing than analytical methods and a $>\!60$-point collision-free margin over learning-based
benchmark methods. Codes are available at \href{https://github.com/jznmsl/PSS-Social}{https://github.com/jznmsl/PSS-Social}.
Read the full abstract ▾
Robotics · Cross-listed from cs.CV
BEVLM: Distilling Semantic Knowledge from LLMs into Bird's-Eye View Representations
Thomas Monninger, Shaoyuan Xie, Qi Alfred Chen et al. (+1 more)
The integration of Large
Language Models (LLMs) into autonomous driving has attracted growing interest for their strong
reasoning and semantic understanding abilities, which are essential for handling complex decision-making and long-tail scenarios. However, existing methods typically feed LLMs with
tokens from multi-view and multi-frame images independently, leading to redundant computation and limited spatial consistency. This separation in visual processing hinders accurate 3D spatial
reasoning and fails to maintain
geometric coherence across views. On the other hand, Bird's-Eye View (BEV) representations learned from geometrically annotated tasks (e.g., object detection) provide spatial structure but lack the semantic richness of foundation
vision encoders. To bridge this gap, we propose BEVLM, a framework that connects a spatially consistent and semantically distilled BEV representation with LLMs. Through extensive experiments, we show that BEVLM enables LLMs to reason more effectively in cross-view driving scenes, improving accuracy by 46.0%, by leveraging BEV features as unified inputs. Furthermore, by distilling semantic knowledge from LLMs into BEV representations, BEVLM significantly improves closed-loop end-to-end driving performance in safety-critical scenarios across UniAD and VAD, with gains of up to 28.2%.
Read the full abstract ▾
All authors Thomas Monninger, Shaoyuan Xie, Qi Alfred Chen, Sihao Ding
Robotics
Fly360: Omnidirectional Obstacle Avoidance within Drone View
Xiangkai Zhang, Dizhe Zhang, WenZhuo Cao et al. (+5 more)
Obstacle avoidance in unmanned aerial vehicles (
UAVs), as a fundamental capability, has gained increasing
attention with the growing focus on spatial intelligence. However, current obstacle-avoidance methods mainly depend on limited field-of-view sensors and are ill-suited for
UAV scenarios which require full-spatial awareness when the movement direction differs from the
UAV's heading. This limitation motivates us to explore omnidirectional obstacle avoidance for panoramic drones with full-view perception. We first study an under explored problem setting in which a
UAV must generate collision-free motion in environments with obstacles from arbitrary directions, and then construct a
benchmark that consists of three representative flight tasks. Based on such settings, we propose Fly360, a two-stage perception-decision pipeline with a fixed random-yaw training strategy. At the perception stage, panoramic RGB observations are input and converted into depth maps as a robust intermediate representation. For the policy network, it is lightweight and used to output body-frame velocity commands from depth inputs. Extensive simulation and real-world experiments demonstrate that Fly360 achieves stable omnidirectional obstacle avoidance and outperforms forward-view baselines across all tasks. Our model is available at https://zxkai.github.io/fly360/
Read the full abstract ▾
All authors Xiangkai Zhang, Dizhe Zhang, WenZhuo Cao, Zhaoliang Wan, Yingjie Niu, Lu Qi, Xu Yang, Zhiyong Liu
Robotics
Uncertainty-Aware Adaptive Dynamics For Underwater Vehicle-Manipulator Robots
Edward Morgan, Nenyi K Dadson and Corina Barbalata
Accurate and adaptive dynamic models are critical for
underwater vehicle-manipulator systems where hydrodynamic effects induce time-varying parameters. This paper introduces a novel uncertainty-aware adaptive dynamics model framework that remains linear in lumped vehicle and manipulator parameters, and embeds convex physical consistency constraints during online estimation. Moving horizon estimation is used to stack horizon regressors, enforce realizable inertia, damping, friction, and hydrostatics, and quantify uncertainty from parameter
evolution. Experiments on a BlueROV2 Heavy with a 4-DOF manipulator demonstrate rapid convergence and calibrated predictions. Manipulator fits achieve R2 = 0.88 to 0.98 with slopes near unity, while vehicle surge, heave, and roll are reproduced with good fidelity under stronger coupling and noise. Median solver time is approximately 0.023 s per update, confirming online feasibility. A comparison against a fixed parameter model shows consistent reductions in MAE and RMSE across degrees of freedom. Results indicate physically plausible parameters and confidence intervals with near 100% coverage, enabling reliable feedforward control and simulation in
underwater environments.
Read the full abstract ▾
Robotics
Unified Learning of Temporal Task Structure and Action Timing for Bimanual Robot Manipulation
Christian Dreher, Patrick Dormanns, Andre Meixner et al. (+1 more)
Bimanual manipulation requires both temporal task structure - which actions precede or overlap others - and concrete timing - when each action starts and how long it takes. Symbolic relations enable high-level
reasoning about task structure and alternative execution sequences, while concrete timing parameters are essential for coordinating two hands at the execution level. Existing approaches address these two levels in isolation, leaving a gap between task planning and movement synchronization. This work presents an approach for learning symbolic and subsymbolic temporal task constraints from human demonstrations and deriving executable, temporally parametrized plans for bimanual manipulation. Specifically, our contributions are (i) a 3-D representation of timings with methods based on multivariate Gaussian Mixture Models to represent temporal relationships between actions on a subsymbolic level, (ii) a method based on the Davis-Putnam-Logemann-Loveland
algorithm that finds and ranks all contradiction-free Allen relation assignments of a task, representing its modes, and (iii) an optimization-based planning system combining the identified symbolic and subsymbolic temporal task constraints to derive parametrized plans for
robot execution. We quantitatively evaluate our approach and show that the inferred symbolic task constraints are more accurate than those of prior heuristic approaches and that the generated plans are temporally closer to human demonstrations than the most characteristic demonstration. Additionally, we qualitatively demonstrate its applicability through orchestrated task executions in simulation and on real
robots.
Read the full abstract ▾
All authors Christian Dreher, Patrick Dormanns, Andre Meixner, Tamim Asfour
Robotics · Cross-listed from cs.CV
Spatial Calibration of Diffuse LiDARs
Nikhil Behari and Ramesh Raskar
Diffuse direct time-of-flight
LiDARs report per-pixel depth histograms formed by aggregating
photon returns over a wide instantaneous field of view, violating the single-ray assumption behind standard
LiDAR-RGB calibration. We present a simple spatial calibration procedure that estimates, for each diffuse
LiDAR pixel, its footprint (effective support region) and relative spatial sensitivity in a co-located RGB image plane. Using a scanned retroreflective patch with background subtraction, we recover per-pixel response maps that provide an explicit
LiDAR-to-RGB correspondence for cross-modal
alignment and fusion. We demonstrate the method on the ams OSRAM TMF8828.
Read the full abstract ▾
Robotics
Underactuated multimodal jumping robot for extraterrestrial exploration
Neil R. Wagner and Justin K. Yim
We present a rolling and jumping underactuated monopedal
robot designed to explore
multimodal locomotion on low-
gravity bodies. It uses only two reaction wheels to control its spatial orientation with two controllers: a balancing controller which can aim the
robot's jump direction on the ground, and an aerial reorientation controller which can aim the
robot's leg for landing after flight. We demonstrate rolling, targeted jumping and landing, and self-righting using only three actuators total, keeping system size to 0.33m and 1.25kg. Simple switching between locomotion modes enables the system to deal with differing landscapes and environmental conditions.
Read the full abstract ▾
Robotics
SG-DOR: Learning Scene Graphs with Direction-Conditioned Occlusion Reasoning for Pepper Plants
Rohit Menon, Niklas Mueller-Goldingen, Sicong Pan et al. (+2 more)
Robotic harvesting in dense crop canopies requires effective interventions that depend not only on geometry, but also on explicit, direction-conditioned relations identifying which organs obstruct a target fruit. We present SG-DOR (Scene Graphs with Direction-Conditioned Occlusion
Reasoning), a relational framework that, given instance-segmented organ point clouds, infers a scene graph encoding physical attachments and direction-conditioned occlusion. We introduce an occlusion ranking task for retrieving and ranking candidate leaves for a target fruit and approach direction, and propose a direction-aware graph neural architecture with per-fruit leaf-set
attention and union-level aggregation. Experiments on a multi-plant synthetic pepper
dataset show improved occlusion prediction (F1=0.73, NDCG@3=0.85) and attachment inference (edge F1=0.83) over strong ablations, yielding a structured relational signal for downstream intervention planning.
Read the full abstract ▾
All authors Rohit Menon, Niklas Mueller-Goldingen, Sicong Pan, Gokul Krishna Chenchani, Maren Bennewitz
Robotics
CFEAR-Teach-and-Repeat: Fast and Accurate Radar-only Localization
Maximilian Hilger, Daniel Adolfsson, Ralf Becker et al. (+2 more)
Reliable localization in prior maps is essential for autonomous
navigation, particularly under adverse weather, where optical sensors may fail. We present CFEAR-TR, a teach-and-repeat localization pipeline using a single spinning radar, which is designed for easily deployable, lightweight, and robust
navigation in adverse conditions. Our method localizes by jointly aligning live scans to both stored scans from the teach mapping pass, and to a sliding window of recent live keyframes. This ensures accurate and robust pose estimation across different seasons and weather phenomena. Radar scans are represented using a sparse set of oriented surface points, computed from Doppler-compensated measurements. The map is stored in a pose graph that is traversed during localization. Experiments on the held-out test sequences from the Boreas
dataset show that CFEAR-TR can localize with an accuracy as low as 0.117 m and 0.096{\deg}, corresponding to improvements of up to 63% over the previous state of the art, while running efficiently at 29 Hz. These results substantially narrow the gap to
lidar-level localization, particularly in heading estimation. We make the C++ implementation of our work available to the community.
Read the full abstract ▾
All authors Maximilian Hilger, Daniel Adolfsson, Ralf Becker, Henrik Andreasson, Achim J. Lilienthal
Robotics
A Unified Low-Dimensional Design Embedding for Joint Optimization of Shape, Material, and Actuation in Soft Robots
Vittorio Candiello, Manuel Mekkattu, Mike Y. Michelis et al. (+1 more)
Soft
robots achieve functionality through tight coupling among geometry, material composition, and actuation. As a result, effective design optimization requires these three aspects to be considered jointly rather than in isolation. This coupling is computationally challenging: nonlinear large-deformation mechanics increase simulation cost, while contact, collision handling, and non-smooth state transitions limit the applicability of standard gradient-based approaches. We introduce a smooth, low-dimensional design
embedding for soft
robots that unifies shape morphing, multi-material distribution, and actuation within a single structured parameter space. Shape variation is modeled through continuous deformation maps of a reference geometry, while material properties are encoded as spatial fields. Both are constructed from shared basis functions. This representation enables expressive co-design while drastically reducing the dimensionality of the search space. In our experiments, we show that design expressiveness increases with the number of basis functions, unlike comparable neural network encodings whose representational capacity does not scale predictably with parameter count. We further show that joint co-optimization of shape, material, and actuation using our unified
embedding consistently outperforms sequential strategies. All experiments are performed independently of the underlying simulator, confirming compatibility with black-box simulation pipelines. Across multiple dynamic tasks, the proposed
embedding surpasses neural network and voxel-based baseline parameterizations while using significantly fewer design parameters. Together, these findings demonstrate that structuring the design space itself enables efficient co-design of soft
robots.
Read the full abstract ▾
All authors Vittorio Candiello, Manuel Mekkattu, Mike Y. Michelis, Robert K. Katzschmann
Robotics
Control Barrier Corridors: From Safety Functions to Safe Sets
Ömür Arslan and Nikolay Atanasov
Safe autonomy is a critical requirement and a key enabler for
robots to operate safely in unstructured complex environments. Control barrier functions and safe motion corridors are two widely used but technically distinct safety methods, functional and
geometric, respectively, for safe motion planning and control. Control barrier functions are applied to the safety filtering of control inputs to limit the decay rate of system safety, whereas safe motion corridors are geometrically constructed to define a local safe zone around the system state for use in motion optimization and reference-governor design. This paper introduces a new notion of control barrier corridors, which unifies these two approaches by converting control barrier functions into local safe goal regions for reference goal selection in feedback control systems. We show, with examples on fully actuated systems, kinematic unicycles, and linear output regulation systems, that individual state safety can be extended locally over control barrier corridors for convex barrier functions, provided the control convergence rate matches the barrier decay rate, highlighting a trade-off between safety and reactiveness. Such safe control barrier corridors enable safely reachable persistent goal selection over continuously changing barrier corridors during system motion, which we demonstrate for verifiably safe and persistent path following in autonomous exploration of unknown environments.
Read the full abstract ▾
Robotics
Qitong Wang, Yijun Liang, Ming Li et al. (+2 more)
Vision-
Language Navigation (VLN) enables
robots to follow natural-
language instructions in visually grounded environments, serving as a key capability for
embodied robotic systems. Recent
Vision-
Language-Action (VLA) models have demonstrated strong
navigation performance, but their high computational cost introduces latency that limits real-time deployment. We propose a training-free spatio-temporal
vision token pruning framework tailored to VLA-based VLN. We apply spatial
token selection to the current view, alongside spatio-temporal compression for historical
memories, enabling efficient long-horizon inference while reducing redundant computation. Leveraging
attention-based
token importance and query-guided spatio-temporal filtering, the proposed approach preserves
navigation-relevant information without retraining or modifying pretrained models, allowing plug-and-play integration into existing VLA systems. Through experiments on standard VLN
benchmarks, we confirm that our method significantly outperforms existing
pruning strategies. It successfully preserves superior
navigation accuracy under extreme
pruning scenarios, all while maintaining the highly competitive inference efficiency. Real-world deployment on a Unitree Go2 quadruped
robot further validates reliable and low-latency instruction-following
navigation under practical robotic constraints. We hope this work helps bridge the gap between large-scale
multimodal modeling and efficient, real-time
embodied deployment in robotic
navigation systems. Project Page: https://wqtwjt1996.github.io/publications/2026-vln.html
Read the full abstract ▾
All authors Qitong Wang, Yijun Liang, Ming Li, Tianyi Zhou, Christopher Rasmussen
Robotics
Data Analogies Enable Efficient Cross-Embodiment Transfer
Jonathan Yang, Chelsea Finn and Dorsa Sadigh
Generalist
robot policies are trained on demonstrations collected across a wide variety of
robots, scenes, and viewpoints. Yet it remains unclear how to best organize and scale such heterogeneous data so that it genuinely improves performance in a given target setting. In this work, we ask: what form of demonstration data is most useful for enabling transfer across
robot set-ups? We conduct controlled experiments that vary end-effector morphology,
robot platform appearance, and camera perspective, and compare the effects of simply scaling the number of demonstrations against systematically broadening the diversity in different ways. Our simulated experiments show that while perceptual shifts such as viewpoint benefit most from broad diversity, morphology shifts benefit far less from unstructured diversity and instead see the largest gains from data analogies, i.e. paired demonstrations that align scenes, tasks, and/or trajectories across different embodiments. Informed by the simulation results, we improve real-world cross-embodiment transfer success by an average of $22.5\%$ over large-scale, unpaired
datasets by changing only the composition of the data.
Read the full abstract ▾
Robotics
Safe Consensus of Cooperative Manipulation with Hierarchical Event-Triggered Control Barrier Functions
Simiao Zhuang, Bingkun Huang and Zewen Yang
Cooperative transport and manipulation of heavy or bulky payloads by multiple manipulators requires coordinated formation tracking, while simultaneously enforcing strict safety constraints in varying environments with limited communication and real-time computation budgets. This paper presents a distributed control framework that achieves consensus coordination with safety guarantees via hierarchical event-triggered control barrier functions (CBFs). We first develop a consensus-based protocol that relies solely on local neighbor information to enforce both translational and rotational consistency in task space. Building on this coordination layer, we propose a three-level hierarchical event-triggered safety architecture with CBFs, which is integrated with a risk-aware leader selection and smooth switching strategy to reduce online computation. The proposed approach is validated through real-world hardware experiments using two Franka manipulators operating with static obstacles, as well as comprehensive simulations demonstrating scalable multi-arm cooperation with dynamic obstacles. Results demonstrate higher precision cooperation under strict safety constraints, achieving substantially reduced computational cost and communication frequency compared to baseline methods.
Read the full abstract ▾
Robotics
Open-Source Based and ETSI Compliant Cooperative, Connected, and Automated Mini-Cars
Lorenzo Farina, Federico Gavioli, Salvatore Iandolo et al. (+9 more)
The automotive sector is following a revolutionary path from vehicles controlled by humans to vehicles that will be fully automated, fully connected, and ultimately fully cooperative. Along this road, new cooperative
algorithms and protocols will be designed and field tested, which represents a great challenge in terms of costs. In this context, in particular, moving from simulations to practical experiments requires huge investments that are not always affordable and may become a barrier in some cases. To solve this issue and provide the community with an intermediate step, we here propose the use of 1:10 scaled cooperative, autonomous, and connected mini-cars. The mini-car is equipped with a Jetson Orin board running the open
Robot Operating System 2 (ROS2), sensors for autonomous operations, and a Raspberry Pi board for connectivity mounting the open source Open Stack for Car (OScar). A key aspect of the proposal is the use of OScar, which implements a full ETSI cooperative-intelligent transport systems (C-ITS) compliant stack. The feasibility and potential of the proposed platform is here demonstrated through the implementation of a case study where the Day-1 intersection collision warning (ICW) application is implemented and validated.
Read the full abstract ▾
All authors Lorenzo Farina, Federico Gavioli, Salvatore Iandolo, Francesco Moretti, Giuseppe Perrone, Matteo Piccoli, Francesco Raviglione, Marco Rapelli, Antonio Solida, Paolo Burgio, Carlo Augusto Grazia, Alessandro Bazzi
Robotics
SuperSuit: An Isomorphic Bimodal Interface for Scalable Mobile Manipulation
Tongqing Chen, Hang Wu, Jiasen Wang et al. (+1 more)
Long-horizon mobile manipulation needs whole-body demonstrations, and collecting them is the bottleneck. Keeping the
robot in the loop ties the
dataset to
robot uptime, while recording human motion without it instead requires a conversion step that decides the joint-space action the policy learns. SuperSuit is a single wearable that both teleoperates a wheeled bimanual
robot and, with the
robot absent, records demonstrations for it. Its arms are mechanically aligned to the
robot's, so recorded joint motion needs no inverse kinematics and no retargeting; one body-worn tracker drives the torso and the planar base through a deadband that keeps posture micro-sway out of the base command. Both roles emit the same 22-D command over the torso, planar base, two 7-DoF arms, and two grippers, with arm targets expressed as forward-looking increments of the observed joint state. On three real-
robot tasks this lets active demonstrations replace most
robot-in-the-loop collection without changing policy quality. With a 10-episode
robot-grounded seed held fixed, the remaining 100 episodes can be recorded without the
robot, whereas active-only training falls well short. SuperSuit moves the limit on
dataset size from the
robot to the operator.
Read the full abstract ▾
All authors Tongqing Chen, Hang Wu, Jiasen Wang, Xiaotao Li
Robotics · Cross-listed from cs.CV
Can we Trust Unreliable Voxels? Exploring 3D Semantic Occupancy Prediction under Label Noise
Wenxin Li, Kunyu Peng, Di Wen et al. (+6 more)
3D semantic occupancy prediction is a cornerstone of robotic perception, yet real-world voxel annotations are inherently corrupted by structural artifacts and dynamic trailing effects. This raises a critical but underexplored question: can autonomous systems safely rely on such unreliable occupancy supervision? To systematically investigate this issue, we establish OccNL, the first
benchmark dedicated to 3D occupancy under occupancy-asymmetric and dynamic trailing noise. Our analysis reveals a fundamental domain gap: state-of-the-art 2D label noise learning strategies collapse catastrophically in sparse 3D voxel spaces, exposing a critical vulnerability in existing paradigms. To address this challenge, we propose DPR-Occ, a principled label-noise-robust framework that constructs reliable supervision through dual-source partial label
reasoning. By synergizing temporal model
memory with representation-level structural affinity, DPR-Occ dynamically expands and prunes candidate label sets to preserve true semantics while suppressing noise propagation. Extensive experiments on SemanticKITTI demonstrate that DPR-Occ prevents
geometric and semantic collapse under extreme corruption. Notably, even at 90% label noise, our method achieves significant performance gains (up to 2.57% mIoU and 13.91% IoU) over existing label noise learning baselines adapted to the 3D occupancy prediction task. By bridging label noise learning and 3D perception, OccNL and DPR-Occ provide a reliable foundation for safety-critical robotic perception in dynamic environments. The
benchmark and source code will be made publicly available at https://github.com/mylwx/OccNL.
Read the full abstract ▾
All authors Wenxin Li, Kunyu Peng, Di Wen, Junwei Zheng, Jiale Wei, Mengfei Duan, Yuheng Zhang, Rui Fan, Kailun Yang
Robotics
Towards Robotic Lake Maintenance: Integrating SONAR and Satellite Data to Assist Human Operators
Ahmed H. Elsayed, Christoph Manss, Tarek A. El-Mihoub et al. (+2 more)
Artificial Water Bodies (AWBs) are human-made systems that require continuous monitoring due to their artificial biological processes. These systems demand regular maintenance to manage their ecosystems effectively. As a result of these artificial conditions,
underwater vegetation can grow rapidly and must be harvested to preserve the ecological balance. This paper proposes a two-step approach to support targeted weed harvesting for the maintenance of artificial lakes. The first step is the initial detection of Submerged Aquatic Vegetation (SAV), also referred to in this paper as areas of interest, is performed using satellite-derived indices, specifically the Aquatic Plants and Algae (APA) index, which highlights submerged vegetation in water bodies. Subsequently, an Unmanned Surface Vehicle (USV) equipped with multibeam SOund
NAvigation and Ranging (SONAR) performs high-resolution bathymetric mapping to locate and quantify aquatic vegetation precisely. This two-stage approach offers an effective human-
robot collaboration, where satellite data guides the USV missions and boat skippers leverage detailed SONAR maps for targeted harvesting. This setup narrows the search space and reduces manual workload from human operators, making the harvesting process less labour-intensive for operators. Preliminary results demonstrate the feasibility of integrating satellite imagery and
underwater acoustic sensing to improve vegetation management in artificial lakes.
Read the full abstract ▾
All authors Ahmed H. Elsayed, Christoph Manss, Tarek A. El-Mihoub, Andrej Lejman, Frederic Stahl
Robotics · Cross-listed from cs.CV
NOVA: Next-step Open-Vocabulary Autoregression for 3D Multi-Object Tracking in Autonomous Driving
Kai Luo, Xu Wang, Rui Fan et al. (+1 more)
Generalizing across unknown targets is critical for open-world perception, yet existing 3D Multi-Object Tracking (3D MOT) pipelines remain limited by closed-set assumptions and “semantic-blind” heuristics. To address this, we propose Next-step Open-Vocabulary Autoregression (NOVA), an autoregressive association formulation that shifts the data association stage from fragmented distance-based matching toward trajectory-conditioned spatio-semantic modeling. NOVA reformulates 3D trajectories as structured spatio-temporal semantic sequences, enabling the simultaneous encoding of physical motion continuity and deep linguistic priors. By leveraging the autoregressive capabilities of Large
Language Models (LLMs), we transform the tracking task into a principled process of next-step sequence completion. This mechanism allows the model to explicitly utilize the hierarchical structure of
language space to resolve fine-grained semantic ambiguities and maintain identity consistency across complex long-range sequences through high-level commonsense
reasoning. Extensive experiments on nuScenes, V2X-Seq-SPD, and KITTI demonstrate the superior performance of NOVA. Notably, on the nuScenes
dataset, NOVA achieves an AMOTA of 22.41% for Novel categories, yielding a significant 20.21% absolute improvement over the baseline. These gains are realized through a compact 0.5B autoregressive model. Code will be available at https://github.com/xifen523/NOVA.
Read the full abstract ▾
All authors Kai Luo, Xu Wang, Rui Fan, Kailun Yang
Robotics · Cross-listed from cs.CV
TaPD: Temporal-adaptive Progressive Distillation for Observation-Adaptive Trajectory Forecasting in Autonomous Driving
Mingyu Fan, Yi Liu, Hao Zhou et al. (+3 more)
Trajectory prediction is essential for autonomous driving, enabling vehicles to anticipate the motion of surrounding
agents to support safe planning. However, most existing predictors assume fixed-length histories and suffer substantial performance degradation when observations are variable or extremely short in real-world settings (e.g., due to occlusion or a limited sensing range). We propose TaPD (Temporal-adaptive Progressive
Distillation), a unified plug-and-play framework for observation-adaptive trajectory
forecasting under variable history lengths. TaPD comprises two cooperative modules: an Observation-Adaptive Forecaster (OAF) for future prediction and a Temporal Backfilling Module (TBM) for explicit reconstruction of the past. OAF is built on progressive knowledge
distillation (PKD), which transfers motion pattern knowledge from long-horizon "teachers" to short-horizon "students" via hierarchical feature regression, enabling short observations to recover richer motion context. We further introduce a cosine-annealed
distillation weighting scheme to balance
forecasting supervision and feature
alignment, improving optimization stability and cross-length consistency. For extremely short histories where implicit
alignment is insufficient, TBM backfills missing historical segments conditioned on scene
evolution, producing context-rich trajectories that strengthen PKD and thereby improve OAF. We employ a decoupled pretrain-reconstruct-finetune protocol to preserve real-motion priors while adapting to backfilled inputs. Extensive experiments on Argoverse 1 and Argoverse 2 show that TaPD consistently outperforms strong baselines across all observation lengths, delivers especially large gains under very short inputs, and improves other predictors (e.g., HiVT) in a plug-and-play manner. Code will be available at https://github.com/zhouhao94/TaPD.
Read the full abstract ▾
All authors Mingyu Fan, Yi Liu, Hao Zhou, Deheng Qian, Mohammad Haziq Khan, Matthias Raetsch
Robotics
Few-Shot Neural Differentiable Simulator: Real-to-Sim Rigid-Contact Modeling
Zhenhao Huang, Siyuan Luo, Bingyang Zhou et al. (+3 more)
Accurate physics simulation is essential for robotic learning and control, yet analytical simulators often fail to capture complex contact dynamics, while learning-based simulators typically require large amounts of costly real-world data. To bridge this gap, we propose a few-shot real-to-sim approach that combines the physical consistency of analytical formulations with the representational capacity of graph neural network (GNN)-based models. Using only a small amount of real-world data, our method calibrates analytical simulators to generate large-scale synthetic
datasets that capture diverse contact interactions. On this foundation, we introduce a mesh-based GNN that implicitly models rigid-body forward dynamics and derive surrogate gradients for collision detection, achieving full differentiability. Experimental results demonstrate that our approach enables learning-based simulators to outperform differentiable baselines in replicating real-world trajectories. In addition, the differentiable design supports gradient-based optimization, which we validate through simulation-based policy learning in multi-object interaction scenarios. Extensive experiments show that our framework not only improves simulation fidelity with minimal supervision but also increases the efficiency of policy learning. Taken together, these findings suggest that differentiable simulation with few-shot real-world grounding provides a powerful direction for advancing future robotic manipulation and control.
Read the full abstract ▾
All authors Zhenhao Huang, Siyuan Luo, Bingyang Zhou, Ziqiu Zeng, Jason Pho, Fan Shi
Robotics · Cross-listed from cs.CV
VG3S: Visual Geometry Grounded Gaussian Splatting for Semantic Occupancy Prediction
Xiaoyang Yan, Muleilan Pei and Shaojie Shen
3D semantic occupancy prediction has become a crucial perception task for comprehensive scene understanding in autonomous driving. While recent advances have explored 3D Gaussian splatting for occupancy modeling to substantially reduce computational overhead, the generation of high-quality 3D Gaussians relies heavily on accurate
geometric cues, which are often insufficient in purely
vision-centric paradigms. To bridge this gap, we advocate for injecting the strong
geometric grounding capability from
Vision Foundation Models (VFMs) into occupancy prediction. In this regard, we introduce Visual Geometry Grounded Gaussian Splatting (VG3S), a novel framework that empowers Gaussian-based occupancy prediction with cross-view 3D
geometric grounding. Specifically, to fully exploit the rich 3D
geometric priors from a frozen VFM, we propose a plug-and-play hierarchical
geometric feature adapter, which can effectively transform generic VFM
tokens via feature aggregation, task-specific
alignment, and multi-scale restructuring. Extensive experiments on the nuScenes occupancy
benchmark demonstrate that VG3S achieves remarkable improvements of 12.6% in IoU and 7.5% in mIoU over the baseline. Furthermore, we show that VG3S generalizes seamlessly across diverse VFMs, consistently enhancing occupancy prediction accuracy and firmly underscoring the immense value of integrating priors derived from powerful, pre-trained geometry-grounded VFMs.
Read the full abstract ▾
Robotics
KISS-IMU: Self-supervised Inertial Odometry with Motion-balanced Learning and Uncertainty-aware Inference
Jiwon Choi, Hogyun Kim, Geonmo Yang et al. (+2 more)
Inertial measurement units (IMUs), which provide high-frequency linear acceleration and angular velocity measurements, serve as fundamental sensing modalities in robotic systems. Recent advances in deep neural networks have led to remarkable progress in inertial odometry. However, the heavy reliance on ground truth data during training fundamentally limits scalability and generalization to unseen and diverse environments. We propose KISS-IMU, a novel self-supervised inertial odometry framework that eliminates ground truth dependency by leveraging simple
LiDAR-based ICP registration and pose graph optimization as a supervisory signal. Our approach embodies two key principles: keeping the IMU stable through motion-aware balanced training and keeping the IMU strong through uncertainty-driven adaptive weighting during inference. To evaluate performance across diverse motion patterns and scenarios, we conducted comprehensive experiments on various real-world platforms, including quadruped
robots. Importantly, we train only the IMU network in a self-supervised manner, with
LiDAR serving solely as a lightweight supervisory signal rather than requiring additional learnable processes. This design enables the framework to ensure robustness without relying on joint multi-modal learning or ground truth supervision. The supplementary materials are available at https://sparolab.github.io/research/kiss_imu.
Read the full abstract ▾
All authors Jiwon Choi, Hogyun Kim, Geonmo Yang, Juhui Lee, Younggun Cho
Robotics
DreamToNav: Generalizable Navigation for Robots via Generative Video Planning
Valerii Serpiva, Jeffrin Sam, Chidera Simon et al. (+4 more)
We present DreamToNav, a novel autonomous
robot framework that uses generative
video models to enable intuitive, human-in-the-loop control. Instead of relying on rigid waypoint
navigation, users provide natural
language prompts (e.g. “Follow the person carefully”), which the system translates into executable motion. Our pipeline first employs Qwen 2.5-VL-7B-Instruct to refine vague user instructions into precise visual descriptions. These descriptions condition NVIDIA Cosmos 2.5, a state-of-the-art
video foundation model, to synthesize a physically consistent
video sequence of the
robot performing the task. From this synthetic
video, we extract a valid kinematic path using visual pose estimation,
robot detection and trajectory recovery. By treating
video generation as a planning engine, DreamToNav allows
robots to visually "dream" complex behaviors before executing them, providing a unified framework for obstacle avoidance and goal-directed
navigation without task-specific engineering. We evaluate the approach on both a wheeled mobile
robot and a quadruped
robot in indoor
navigation tasks. DreamToNav achieves a success rate of 76.7%, with final goal errors typically within 0.05-0.10 m and trajectory tracking errors below 0.15 m. These results demonstrate that trajectories extracted from generative
video predictions can be reliably executed on physical
robots across different locomotion platforms.
Read the full abstract ▾
All authors Valerii Serpiva, Jeffrin Sam, Chidera Simon, Hajira Amjad, Iana Zhura, Artem Lykov, Dzmitry Tsetserukou
Robotics
Dual-Agent Multiple-Model Reinforcement Learning for Event-Triggered Human-Robot Co-Adaptation in Decoupled Task Spaces
Yaqi Li, Zhengqi Han, Huifang Liu et al. (+1 more)
This paper presents a shared-control rehabilitation policy for a custom 6-degree-of-freedom (6-DoF) upper-limb
robot that decomposes complex reaching tasks into decoupled spatial axes. The patient governs the primary reaching direction using binary commands, while the
robot autonomously manages orthogonal corrective motions. Because traditional fixed-frequency control often induces trajectory oscillations due to variable inverse-kinematics execution times, an event-driven progression strategy is proposed. This architecture triggers subsequent control actions only when the end-effector enters an admission sphere centred on the immediate target waypoint, and was validated in a semi-virtual setup linking a physical pressure sensor to a MuJoCo simulation. To optimise human–
robot co-adaptation safely and efficiently, this study introduces Dual
Agent Multiple Model Reinforcement Learning (DAMMRL). This framework discretises decision characteristics: the human
agent selects the admission sphere radius to reflect their inherent speed–accuracy trade-off, while the
robot agent dynamically adjusts its 3D Cartesian step magnitudes to complement the user's cognitive state. Trained in simulation and deployed across mixed environments, this event-triggered DAMMRL approach effectively suppresses waypoint chatter, balances spatial precision with temporal efficiency, and significantly improves success rates in object acquisition tasks.
Read the full abstract ▾
All authors Yaqi Li, Zhengqi Han, Huifang Liu, Steven W.Su