Robotics
Flow-based Policy Adaptation without Policy Updates
Luzhe Sun, Jingtian Ji, Haoran Chen et al. (+2 more)
Foundation models and other policies trained on large and diverse
datasets can endow
robots with semantic and task knowledge that would be difficult to acquire from task-specific demonstrations alone. Despite their broad scene understanding and
reasoning capabilities, however, these models often lack the environment- and embodiment-specific knowledge required for precise and reliable execution. We propose GLOVES, a flow-based framework that complements an existing
agent with a lightweight, goal-agnostic expert action prior that refines the action chunks proposed by the
agent without updating its parameters. The prior captures observation-conditioned expert behavior, while the
agent's proposal supplies task- and context-specific intent that the correction aims to preserve. The learned flow also provides a natural in-distribution scoring mechanism through reverse flow evaluation. We use this signal as an intervention gate: actions that appear consistent with the expert distribution are passed through unchanged, while anomalous or out-of-distribution (OOD) actions are corrected. In this way, GLOVES only provides assistance when necessary. By learning local goal-agnostic expert action patterns and using them to adapt
agent-proposed actions during execution, GLOVES provides a lightweight shared-control module for robust action adaptation across tasks and environments. Across controlled OOD-recovery studies, simulation
benchmarks, and real-
robot experiments, GLOVES consistently improves the success of
vision-
language-action models (VLAs) and Code-as-Policy
agents without retraining the underlying policy. Code and demos are available at ripl.github.io/GLOVES_web.
Read the full abstract ▾
All authors Luzhe Sun, Jingtian Ji, Haoran Chen, Jiawei Zhou, Matthew R. Walter
Robotics
RiskFlow: Fast and Faithful Safety-Critical Traffic Scenario Generation
Qi Lan, Yining Tang, Yu Shen et al. (+4 more)
Safety-critical traffic scenario generation is essential for evaluating autonomous driving systems under rare but high-risk interactions. Existing
diffusion-based methods offer strong controllability in closed-loop generation, but their iterative denoising process is computationally expensive and may accumulate sampling and guidance errors over long rollouts, causing unrealistic motion artifacts such as jitter, abnormal acceleration, and off-road behavior. To address these issues, we propose RiskFlow, a closed-loop safety-critical multi-
agent traffic generation framework that formulates future trajectory generation as transport in the action space. Instead of relying on iterative denoising, RiskFlow learns an average velocity field over a finite interval to transform Gaussian action sequences into future acceleration and yaw-rate commands with a single forward pass, using a JVP-based objective for efficient and stable training. At test time, RiskFlow applies output-space guidance to the generated actions, steering selected critical
agents toward risky interactions while regularizing off-road behavior, and reconstructs physically feasible trajectories through vehicle dynamics. Experiments on nuScenes with tbsim closed-loop evaluation show that RiskFlow achieves a strong adversariality-realism trade-off across multi-
agent and long-horizon settings. Compared with representative baselines, RiskFlow consistently improves realism while maintaining competitive safety-critical generation capability, and substantially reduces inference time for evaluation.
Read the full abstract ▾
All authors Qi Lan, Yining Tang, Yu Shen, Yi Zhou, Yuhao Wei, Jie Li, Guofa Li
Robotics
Ensuring Interaction Safety in Multitask Exoskeleton Control: A Simulation-Trained Variable Impedance Framework
Muyuan Ma, Houcheng Li, Haotian Zhai et al. (+4 more)
Wearable exoskeletons can augment human phys ical capabilities during complex activities. However, ensuring adaptation across diverse tasks while guaranteeing interaction safety remains a critical challenge. To address this, a simulation trained variable impedance control approach with stability guarantees is proposed. First, a simulation-based human exoskeleton motion data generation pipeline is established, utilizing Proximal Policy Optimization (PPO) to synthesize human muscle activations while the exoskeleton provides direct compensation for human biological joint torques. Subsequently, the generated
dataset is used to train a dual modality policy that fuses semantic instructions with proprioceptive history, enabling the prediction of reference trajectories and variable impedance gains for nine different motion tasks. To guarantee safety, the network outputs are constrained by a stability criterion derived from Lyapunov stability theory, which bounds stiffness variations to ensure the asymptotic stability of the coupled system. Experimental results indicate that the proposed framework reduces metabolic cost in real-world scenarios com pared with standard baseline methods. These findings suggest the feasibility of the proposed framework for safe, multitask exoskeleton control.
Read the full abstract ▾
All authors Muyuan Ma, Houcheng Li, Haotian Zhai, Lijun Han, Xinpan Meng, Xiuze Xia, Long Cheng
Robotics
Waypoints Matter: A Systematic Study for Sampling-Based Trajectory Planning
Josep M. Barbera, Antonio Artuñedo and Jorge Villagra
Real-time autonomous driving commonly relies on sampling-based trajectory planners that link candidate trajectories to target waypoints along the road centerline. The placement of these waypoints directly impacts both the existence and quality of feasible trajectories. Yet, its effect on planner performance remains largely unexplored. In this paper, we treat waypoint placement as a first-class design variable. We hold the trajectory primitive and candidate budget fixed, and systematically sweep three placement strategies (uniform spacing, an augmented Ramer-Douglas-Peucker variant (RDP*), and a novel
curvature-conditioned allocation) across 449 configurations and five CommonRoad maps of increasing
geometric complexity. Our results show that the nominal inter-waypoint spacing $d_s$ is the primary performance driver, with large differences in planner reliability attributed to placement alone. Uniform sampling at a well-tuned spacing matches or surpasses both RDP* and the centered
curvature variant. The
curvature variant offers a small but consistent advantage on geometrically complex roads under reliability-first and balanced weightings, while RDP* never outperforms uniform sampling. These findings suggest that $d_s$ should be treated as the dominant tuning parameter, with geometry-aware strategies reserved for
curvature-rich corridors where feasibility is the limiting factor.
Read the full abstract ▾
Robotics
VOLT: Vision and Language Trajectory Segmentation for Faster-than-Demonstration Policies
Robert Ramirez Sanchez, Daniel J. Evans, Dylan P. Losey et al. (+1 more)
Humans often take longer to demonstrate a task than a
robot would need to execute it. Rather than learning to replicate the demonstration at the same pace, many industrial and practical applications require
robots to perform tasks as quickly as possible. In this paper, we investigate several hypotheses for learning policies that operate faster-than-demonstrations. Our experiments show that the most effective strategy is to downsample recorded demonstrations and train the
robot's policy on this accelerated data. However, uniformly downsampling an entire trajectory can be problematic. Some parts of a task can be safely sped up (e.g., unconstrained motion), while others demand slower, more precise motion (e.g., object interactions or fine manipulation). To address this challenge, we introduce VOLT, a
vision-and-
language trajectory
segmentation method that reasons over
video demonstrations, and leverages contextual cues to determine when acceleration is appropriate and when careful precision is required. VOLT identifies segments where slow, deliberate motion is necessary, then selectively downsamples the remaining segments. The resulting reformatted trajectories can be used with standard imitation learning approaches, such as
diffusion policies. Our results highlight that
segmentation quality is critical—baseline methods often misidentify when acceleration is possible, leading to overly cautious or unreliable policies. Compared to state-of-the-art alternatives, VOLT allows
robots to execute tasks faster while maintaining strong performance. See our project website: https://volt2026.github.io/VOLT2026
Read the full abstract ▾
All authors Robert Ramirez Sanchez, Daniel J. Evans, Dylan P. Losey, Siddarth Jain
Robotics
Meridian: Metric-Semantic Primitive Matching for Cross-View Geo-Localization Beyond Urban Environments
Mason Peterson, Qingyuan Li, Yixuan Jia et al. (+4 more)
Successful
robot automation requires accurate global localization to support repeatability, task planning, goal specification, and safe operation. However, reliable localization in GNSS-denied environments remains an open problem. Overhead aerial imagery offers a promising solution, but existing approaches primarily target structured urban environments and have been rarely demonstrated in unstructured natural terrain. Limitations of the state-of-the-art include a reliance on models trained for specific environments, as well as difficulty handling repetitive geometries and featureless landscapes commonly found in natural outdoor areas. To overcome these challenges, we present Meridian, a method for matching high-level metric-semantic primitives across aerial images and ground
robot RGB-D camera data that achieves accurate global localization and generalizes well across diverse environments, all without any training or algorithmic fine-tuning on area-specific data. We formulate novel consistency metrics to estimate a distribution over
robot submap poses and to reject outlier hypotheses in a robust pose graph optimization step for accurate
robot trajectory estimation. We demonstrate that our
algorithm can localize a ground
robot across a wide variety of environments, including an autonomous driving
dataset, a park and campus area, and a wilderness camp, with an average optimized trajectory error of 2.4 m over 19 km of ground traversal.
Read the full abstract ▾
All authors Mason Peterson, Qingyuan Li, Yixuan Jia, Fernando Cladera, Carlos Nieto-Granda, Camillo Jose Taylor, Jonathan P. How
Robotics
Attitude-Aided Linear Calibration of Triaxial Accelerometers
Yongqiang Yu, Tian Huang and Yipeng Yang
Triaxial MEMS accelerometers are widely used for inertial sensing,
navigation, and sensor fusion, but existing calibration methods often rely on costly reference setups or nonlinear iterative optimization, limiting their efficiency and applicability to low-cost or self-calibrating systems. We present attitude-aided linear accelerometer calibration (ALAC), a method that operates on any platform providing orientation information, such as turntables, robotic arms, or inertial measurement units. ALAC constructs a combined error
matrix (CEM) to represent sensor errors in a unified calibration model and enables linear least-squares estimation. The bias and
gravity vector are jointly estimated, implicitly accounting for platform misalignment, and
matrix decomposition of the CEM recovers scale, non-orthogonality, and
alignment rotation parameters. Under static
gravity, calibration is formulated as a constrained homogeneous least-squares (CHLS) problem and solved in closed form using standard linear
algebra. Only five arbitrarily oriented measurements are required, and a recursive extension supports online or in-field calibration. Experiments on a stationary
robot-mounted accelerometer and a quasi-static public IMU trajectory show that ALAC, in both offline and online modes, outperforms reference-based and online baselines in accuracy and robustness to sensor noise. On the same
dataset, it matches iterative self-calibration under filtered conditions and surpasses all evaluated baselines on raw measurements. These results demonstrate a robust and practical calibration scheme for MEMS-based inertial platforms, especially low-cost IMUs and online calibration scenarios.
Read the full abstract ▾
Robotics · Cross-listed from cs.CV
Synthetic Data Generation and Vision-based Wrinkle and Keypoint Detection for Bimanual Cloth Manipulation
Ariel Herrera, Xueyang Kang and Atal Anil Kumar
Robotic manipulation of textiles remains challenging because continuous deformation and self-occlusions hinder the robust visual perception required to estimate the cloth's state. To address the lack of annotated real-world data, we developed a Blender-based synthetic pipeline exporting auto-annotated keypoints, and combined manually labeled renders with real-world data to train a wrinkle detector. We present a perception framework integrating a CNN for permutation-invariant keypoint detection and a YOLOv8-OpenCV pipeline to extract grasping points from structural wrinkles. A proposed bimanual
algorithm uses this system to stretch fully folded garments via wrinkles, transitioning to keypoint-based ironing once corners emerge. The keypoint model achieves a Mean Position Error (MPE) of 1.7615 pixels. The perception system transfers to physical fabrics without fine-tuning, outperforming baselines that fail in high-occlusion states or yield false positives on severe folds.
Read the full abstract ▾
Robotics
Multi-Resolution Tactile Imitation Learning for Contact-Rich Robotic Manipulation
Rickmer Krohn, Erik Helmut, Niklas Funk et al. (+3 more)
Touch sensing is beneficial for solving a wide variety of manipulation tasks. While there exists a wide range of tactile sensors with different properties, exploiting the fusion of multiple heterogeneous tactile sensors to improve manipulation learning remains underexplored. We present Multi-Resolution Tactile Sensing (MiTaS), a representation framework that leverages multiple tactile sensors operating at different temporal resolutions in order to solve complex contact-rich manipulation tasks. We propose an effective architecture using modality-specific convolutional stems and
transformer-based fusion that effectively integrates information from an RGB camera stream, a
vision-based GelSight Mini sensor and a high-frequency event-based Evetac sensor. This multisensory representation then conditions a flow-matching policy for solving downstream tasks. Our systematic study across five contact-rich manipulation tasks demonstrates the effectiveness of multi-resolution tactile features in imitation learning wherein MiTaS achieves an average success rate of 80 %, while
vision-only (35.5\,%) and visuotactile (55.5\,%, 63\,%) baselines cannot solve the task reliably. Project Page: http://mitas-touch.github.io.
Read the full abstract ▾
All authors Rickmer Krohn, Erik Helmut, Niklas Funk, Jan Peters, Vignesh Prasad, Georgia Chalvatzaki
Robotics
RadiusFPS: Efficient Farthest Point Sampling on CPUs and GPUs via Spherical Voxel Pruning
Ziyang Yu, Xiang Li, Qiong Chang et al. (+1 more)
Point clouds are a primary sensory representation for robotic perception, underpinning
LiDAR-based autonomous driving, simultaneous localization and mapping (SLAM), and
navigation. Within these pipelines, Farthest Point Sampling (FPS) is the most well-known downsampling operator, as its uniform coverage preserves the
geometric structure on which downstream perception relies. However, the large time complexity of classical FPS scales poorly with the million-point-per-second rates of modern 3D sensors, making it a dominant latency bottleneck that conflicts with the real-time and limited onboard compute budgets of robotic systems. Therefore, we propose RadiusFPS, an FPS acceleration framework based on spherical voxel
pruning that preserves the standard FPS update rule under the same initialization and tie-breaking policy. By indexing the point cloud with spherical voxels, RadiusFPS derives a conservative
geometric bound that prunes redundant distance computations in each iteration, complemented by a coordinate-wise point-skip test that removes residual updates. We further introduce RadiusFPS-G, a warp-level GPU implementation that fuses voxel selection,
pruning, and distance update into
memory-coalesced kernels, eliminating costly global-
memory round-trips. On indoor (S3DIS, ScanNet) and outdoor
LiDAR (SemanticKITTI)
benchmarks, RadiusFPS-G attains up to 2.5x speedup over GPU-based FPS and matches or exceeds QuickFPS among the evaluated methods while using roughly half its GPU
memory, with comparable
segmentation accuracy. When coupled with the learning-based FastPoint sampler, the resulting pipeline achieves the fastest End-to-End inference among all evaluated configurations. These properties make high-quality FPS-style sampling practical for latency- and
memory-constrained robotic
vision.
Read the full abstract ▾
All authors Ziyang Yu, Xiang Li, Qiong Chang, Jun Miyazaki
Robotics
Breaking Time: A Fully Gaussian Framework for Distributed and Continuous-Time SLAM
Davide Ceriola, Simone Ferrari, Luca Di Giammarino et al. (+2 more)
Continuous-time SLAM provides a principled framework for fusing heterogeneous sensors while estimating smooth trajectories, and is particularly well-suited for handling heterogeneous, asynchronous sensor streams with non-uniform readout patterns, such as rolling shutter cameras,
LiDAR scanners, radar sweeps, or event-based sensors. In this work, we introduce G-solver, a fully Gaussian and distributed framework that combines Gaussian Belief Propagation (GBP) with Gaussian Process (GP) motion priors for continuous-time trajectory estimation. Our GP model provides a probabilistic representation of the trajectory, enabling consistent interpolation and the use of data-driven hyperparameters, while GBP offers a scalable message-passing formulation well-suited for decentralized settings. The resulting solver naturally extends to multi-camera scenarios without specialized synchronization or engineering effort. We evaluate the approach on synthetic and real data, including rolling shutter and distributed multi-camera optimization, demonstrating accurate and stable estimation with runtimes comparable to existing continuous-time methods. An open-source implementation is released.
Read the full abstract ▾
All authors Davide Ceriola, Simone Ferrari, Luca Di Giammarino, Leonardo Brizi, Giorgio Grisetti
Robotics
MPCoT: Reward-Guided Multi-Path Latent Reasoning for Test-Time Scalable Vision-Language-Action
Boyang Zhang and Lianlei Shan
Vision-
Language-Action (VLA) policies remain brittle in long-horizon control, where one-pass action decoding offers limited inference-time deliberation. Explicit chain-of-thought adds
reasoning depth but incurs
token-generation latency and an indirect text-to-action interface. We propose MPCoT, a reward-guided multi-path latent
reasoning framework with configurable depth K and width M. MPCoT initializes M latent hypotheses, refines them for K weight-tied steps, and softly aggregates them before action decoding. A training-only path-preference objective combines expert-trajectory consistency, frozen Qwen3-VL progress scores, and endpoint-consistency feedback to supervise the path scorer. Under matched protocols on LIBERO and CALVIN, MPCoT improves long-horizon performance; ablations support the contributions of depth, width, soft aggregation, and reward supervision. On five real-world ALOHA Mini bimanual tasks, average success increases from 71.3% to 82.0% over the matched OpenVLA-OFT baseline. These results support latent deliberation as a means of improving execution while preserving the action interface and generating no
reasoning tokens.
Read the full abstract ▾
Robotics
Vault: One-Step Latent Generation with Positive-Anchored Rewards for Autonomous Driving
Yining Xing, Zehong Ke, Zhiyuan Liu et al. (+1 more)
End-to-end autonomous driving must balance
multimodal maneuver generation against real-time inference constraints.
Diffusion planners capture diverse behaviors, but their iterative denoising incurs considerable inference cost in safety-critical deployment, while one-step alternatives still imitate only the single expert demonstration, so neither family certifies both the diversity and the quality of its outputs. To address this, we propose Vault, a framework that couples one-step latent generation with sample-based reinforcement learning guidance. Built on a V-JEPA 2.1 visual representation pretrained on the full NAVSIM training imagery, Vault generates trajectory candidates in a single forward pass by drifting in a VAE latent space. During training, a reward-gated positive pool collects the maximum-scoring trajectories that the model itself discovers under the official evaluator and anchors the drift targets to these certified samples. This closes a reinforcement-learning loop purely through sample-based target construction, without policy gradients, learned reward models, or extra interaction: the generator imitates the frontier of its own certified successes, so quality and multimodality improve together under a single fixed sampling configuration. A learned scorer that predicts the official score and its sub-metrics then selects the executed trajectory, and all training-only modules are discarded at inference. On the NAVSIM
benchmarks, Vault achieves state-of-the-art performance at real-time inference cost, reaching 94.6 PDMS on v1 and 91.2 EPDMS on v2.
Read the full abstract ▾
All authors Yining Xing, Zehong Ke, Zhiyuan Liu, Jianqiang Wang
Robotics
TAM: Torque Adaptation Module for Robust Motion Transfer in Manipulation
Dongwon Son, Florian Shkurti, Jason Lee et al. (+3 more)
A policy tuned for one
robot often behaves differently on another, whether due to the sim-to-real gap, unknown payloads, or the differing dynamics of two instances of the same
robot. In contact-rich, dynamic manipulation, even small motion discrepancies can result in failure to track reference motion, since they disrupt the timing and modes of contact. Common remedies, such as domain randomization or system identification, either produce overly conservative task policies or require data that must be recollected for each
robot or payload. We introduce the Torque Adaptation Module (TAM), a learned module that adapts the torque commands sent to the
robot to match the behavior of an ideal
robot. TAM operates between the low-level controller that tracks the policy's actions and the
robot's torque interface. It includes a history encoder that embeds proprioceptive history into a latent state and a torque adaptor that computes residual torque corrections. Because TAM depends only on proprioceptive history and not on policy observations, or the action space, the same TAM weights can be reused to adapt policies with different action spaces (joint targets, end-effector targets, or direct torques). The policies themselves do not need to be trained with domain randomization of
robot parameters. Instead, we offload the need for domain randomization to TAM by training it entirely in randomized simulation, using multi-
robot pretraining followed by a
robot-specific fine-tuning step that still requires no real-
robot data. We evaluate TAM zero-shot on a real Franka Panda
robot across dynamic manipulation tasks that include a
vision-based box pushing policy (from RL), a flip policy (from BC), and an MPC ball-on-plate balancing. Our experiments show that TAM improves zero-shot real-
robot execution compared to online system identification and RMA baselines and enables robust dynamic manipulation performance.
Read the full abstract ▾
All authors Dongwon Son, Florian Shkurti, Jason Lee, Naman Shah, Beomjoon Kim, Dieter Fox
Robotics
ActiveMimic: Egocentric Video Pretraining with Active Perception
Xingyao Lin, Guojin Zhong, Tianyi Lu et al. (+4 more)
Egocentric human
video offers a scalable alternative to
robot data for pretraining, yet models pretrained on such
video consistently underperform those pretrained on
robot data. We attribute this gap to a missing signal, the active perception behavior in
egocentric videos, where humans continuously reposition their viewpoint during manipulation, inducing camera motion that standard pipelines treat as noise. To address this, we present ActiveMimic, a pretraining framework that recovers synchronized camera and wrist trajectories from a single body-worn RGB camera, models camera motion as a viewpoint action, and jointly learns active perception and manipulation from in-the-wild
egocentric human
video before adapting to a target
robot. Empirically, real-world experiments across tasks with diverse active perception demands show that ActiveMimic consistently surpasses baselines pretrained on human
video and matches state-of-the-art models pretrained on
robot data. Further analysis provides evidence that active perception capability originates from
egocentric human
video pretraining rather than
robot-specific fine-tuning, confirming active perception as the key to unlocking
egocentric human
video for
robot pretraining.
Read the full abstract ▾
All authors Xingyao Lin, Guojin Zhong, Tianyi Lu, Ziyi Ye, Yichen Zhu, Zuxuan Wu, Yu-Gang Jiang
Robotics
AffordanceVLA: A Vision-Language-Action Model Empowering Action Generation through Affordance-Aware Understanding
Qize Yu, Jiadi You, Yuran Wang et al. (+12 more)
Vision-
Language-Action (VLA) models leverage the rich world knowledge of pretrained
vision-
language models (VLMs) to enable instruction-following robotic manipulation. However, the structural mismatch between VLM semantic spaces and
embodied control policies often hinders the learning of precise perception-action mappings. To address this challenge, we propose AffordanceVLA, a scalable, unified framework that introduces structured affordance learning as a task-oriented and object-centric intermediate representation to establish a more precise and robust perception-action mapping. Specifically, we progressively model manipulation priors through three complementary components - Which2Act, Where2Act, and How2Act - which jointly produce spatially grounded, semantically conditioned, and action-coupled intermediate representations, thereby naturally bridging
vision,
language, and action. We integrate these modules into a Mixture-of-
Transformer (MoT) architecture and adopt a three-stage progressive training curriculum that leverages 8.8M affordance-centric grounding and pointing samples, translating scalable
vision-
language supervision into improved robotic manipulation. To overcome the scarcity of dense affordance labels in robotic
datasets, we also develop a robust automated data augmentation pipeline. Extensive experiments on ID & OOD simulation and real-world demonstrate that AffordanceVLA outperforms strong VLA baselines across diverse manipulation scenarios, validating structured affordance learning as a scalable and robust bridge that integrates multi-source
vision-
language supervision while connecting pretrained VLM semantics to
embodied robot control.
Read the full abstract ▾
All authors Qize Yu, Jiadi You, Yuran Wang, Jiaqi Liang, Bowen Ping, Yang Tian, Yue Chen, Bohan Zhou, Wenxuan Song, Minghong Cai, Zeying Gong, Ruihai Wu, Yinchuan Li, Junwei Liang, Yingcong Chen
Robotics
MotionDisco: Motion Discovery for Extreme Humanoid Loco-Manipulation
Ilyass Taouil, Michal Ciebelski, Shafeef Omar et al. (+4 more)
We present MotionDisco, a framework that discovers contact-rich, long-horizon
humanoid loco-manipulation motions from scratch, without relying on teleoperation or motion retargeting from human demonstrations. This is challenging because the space of possible contact interactions grows combinatorially with the task horizon and the number of objects in the scene. MotionDisco enables rapid discovery of novel motions by coupling a large
language model (LLM) guided evolutionary search over sequences of interactions with an efficient sequential kinodynamic trajectory optimizer and
pruning strategy, enabling the rapid discovery of novel skills. Through extensive ablation studies, we show that our LLM-guided search discovers successful whole-body trajectories across several challenging long-horizon tasks. Finally, by training reinforcement learning tracking policies on the discovered trajectories, we transfer the motions to a real
humanoid robot. This is the first work to discover and deploy long-horizon
humanoid loco-manipulation skills entirely through automated evolutionary search. Supplementary
videos of the experiments are available at: https://youtu.be/DHiVz34QYlw.
Read the full abstract ▾
All authors Ilyass Taouil, Michal Ciebelski, Shafeef Omar, Haizhou Zhao, Angela Dai, Aaron M. Johnson, Majid Khadiv
Robotics
Towards Realistic 3D Sonar Simulation
Youssef Attia, Davide Costa, Francesco Wanderlingh et al. (+2 more)
As
underwater robotics research increasingly addresses complex 3D perception and autonomous
navigation, the fidelity of sonar simulation has become a key factor in
algorithm development. Current simulation frameworks typically rely on geometry-driven rendering, approximating 3D sonar as an
underwater equivalent to
LiDAR, which fails to account for fundamental acoustic phenomena such as refraction, multi-path interference, and phase-dependent signal formation. This paper proposes a modular architecture for realistic 3D sonar simulation that integrates GPU-accelerated graphics engines with physically grounded acoustic propagation principles. We implement a volumetric 3D sonar model within the NVIDIA Isaac Sim environment, modeled after the Water Linked 3D-15 sensor, and integrate it into a comprehensive
underwater simulation framework. The system is validated through a hardware-in-the-loop configuration, where a modified FastLIO2 SLAM pipeline, executed on an NVIDIA Jetson Orin Nano, performs sensor fusion using synthetic 3D sonar, DVL, IMU, and pressure data. Finally, a qualitative comparison between simulated outputs and real-world data from harbor sheet-pile inspections is provided, characterizing the remaining sim-to-real gap and establishing a roadmap toward fully acoustics-driven volumetric sensing.
Read the full abstract ▾
All authors Youssef Attia, Davide Costa, Francesco Wanderlingh, Filippo Campagnaro, Enrico Simetti
Robotics
3D Underwater Path Planning via Generative Flow Field Surrogates
Zachary Cooper-Baldock, Paulo E. Santos, Russell S.A. Brinkworth et al. (+1 more)
Autonomous
underwater vehicle (AUV) launch and recovery (LAR) into the hull of an advancing host platform requires traversal of a complex, three-dimensional propeller wake whose hydrodynamic structure cannot be characterised by a uniform current model. High-fidelity Reynolds-Averaged Navier-Stokes (RANS) Computational Fluid Dynamics (CFD) simulations resolve this structure with sufficient accuracy for path planning, but their computational cost renders them impractical for onboard use. We address this gap by integrating two conditional generative
adversarial network (cGAN) architectures – a regularised PatchGAN and a 2D3DGAN with self-
attention – as drop-in replacements for RANS CFD data within a three-dimensional, energy-weighted A* path planning framework. Both generators are driven by a hierarchical pipeline that synthesises full $128^3$ voxel flow field volumes from scalar operating condition inputs alone, with end-to-end inference times of approximately 28-146 $\mu$s, compared to hours for a single RANS computation. We
benchmark all four environmental knowledge levels: uniform current, ground-truth CFD, PatchGAN, and 2D3DGAN~SA across 19,800 independently generated trajectories spanning 550 distinct flow conditions. Full CFD wake knowledge reduces energy expenditure by 5.7-12.5% and high-velocity wake-core encounters by up to 77.8% relative to uniform-current planning, with both benefits scaling with operating severity. The cGAN surrogates recover approximately 45-60% of the CFD energy benefit and high-velocity
cell avoidance benefit while operating at inference speeds compatible with edge device use. These results provide the first systematic quantification of the downstream path planning value of cGAN-predicted hydrodynamic fields in a three-dimensional maritime robotics application.
Read the full abstract ▾
All authors Zachary Cooper-Baldock, Paulo E. Santos, Russell S.A. Brinkworth, Karl Sammut
Robotics
A Conversational Framework for Human-Robot Collaborative Manipulation with Distributed Generative AI models
Arash Ghasemzadeh Kakroudi and Roel Pieters
This paper presents a distributed conversational framework for human-
robot collaborative manipulation that integrates local
language and
vision-
language models (VLMs) with a
Robot Operating System 2 (ROS 2)-based execution stack.
Language understanding, visual grounding, orchestration, and motion execution run as separate ROS 2 nodes, enabling flexible deployment across distributed hardware while maintaining a responsive control loop. From free-form user commands, the system generates structured action requests for pick, place, and handover. It uses a VLM to return image-space targets, which are converted into metric
robot-frame goals using depth and calibration. A web dashboard exposes intermediate intent and grounding overlays (pixel, depth, and
robot-frame) and requires explicit operator confirmation before any motion is executed. Experiments on a Franka FR3 platform evaluate end-to-end task reliability and latency under increasing working table scene ambiguity and compare alternative LLM/VLM configurations in the same pipeline. Code and full documentation are available at [github.com/cogrob-tuni/franka-llm](https://github.com/cogrob-tuni/franka-llm).
Read the full abstract ▾
Robotics
L-SDPPO: Policy Optimization of Spiking Diffusion Policy for Intra-vehicular Robotic Manipulation
Liwen Zhang, Dong Zhou, Guanghui Sun et al. (+4 more)
Intra-vehicular
robots in spacecraft help reduce astronaut workload and improve mission efficiency. Recent research focuses on using deep learning methods to achieve the acute control required for operations in these complex environments. However, objects exhibit unpredictable, unconstrained drift without gravitational damping. These factors demand robustness against complex
multimodal action distributions.
Diffusion policies (DP) can model these complex actions, but their iterative sampling process consumes too much energy for the limited power budgets of spacecraft. We therefore propose a low-energy intra-vehicular robotic manipulation framework, L-SDPPO, in which the Spiking
Diffusion Policy (SDP) is optimized with a reinforcement learning (RL)
algorithm. Furthermore, to address the insufficient perception of dynamic spatiotemporal features in microgravity, we propose the statedependent latency injection (SDLI) mechanism, which mimics biological neural delays to dynamically regulate the timing of input information. Evaluation on five representative intra-vehicular daily tasks (e.g., hatch opening and precision container capping) shows that our method consistently achieves higher success rates and lower energy consumption, compared to the state-of-the-art robotic manipulation methods. These results demonstrate our method is a viable intra-vehicular robotic manipulation method.
Read the full abstract ▾
All authors Liwen Zhang, Dong Zhou, Guanghui Sun, Yifei Zheng, Yuhui Hu, Kaihong Ouyang, Zuoquan Zhao
Robotics
Sample-efficient Low-level Motion Planning for Robotic Manipulation Tasks via Zero-shot Transfer Learning
Yuanzhi He, Victor Romero-Cano, José J. Patiño et al. (+3 more)
As robotic systems become more sophisticated, the growing complexity of their motion planning models and the longer training times pose substantial challenges. Evolutionary
algorithms such as the Sample-efficient Cross-
Entropy Method (iCEM) have recently demonstrated promising potential for low-level real-time planning by leveraging efficient knowledge reuse strategies to improve performance. Although effective in many control tasks, iCEM's performance can be constrained in more complex scenarios, particularly those requiring stacking, sliding, and shelf placement. In this work, we propose a novel iCEM+TL framework that explicitly leverages Transfer Learning (TL), where key iCEM parameters are transferred from simpler upstream tasks to guide more complex downstream tasks. Additionally, we applied Reward Redesign (RR) through task decomposition for stacking objects and shelf placement to optimize task-specific performance. Results from the simulation show that our framework achieves success rate improvements of up to 23%. The framework is further validated on a real Franka Emika
robot in a stacking task, demonstrating its practical feasibility for real-world deployment.
Read the full abstract ▾
All authors Yuanzhi He, Victor Romero-Cano, José J. Patiño, Juan David Hernández, William Sawtell, Gualtiero Colombo
Robotics
Gotta Grow Fast: Design and Benchmarking of a Tip Mount for High-Speed Vine Robots
Antonio Alvarez Valdivia, Robert Reeve, Ankush Dhawan et al. (+4 more)
Soft, growing vine
robots extend through tip eversion, a mechanism that enables
navigation through cluttered environments. However, integrating cameras and other sensors at the tip is uniquely challenging because the material forming the tip is constantly renewed as the
robot grows. This continual material turnover, combined with friction between internal layers, added tip weight, and fabric constriction, complicates sensor and tool mounting. These limitations hinder the deployment of vine
robots for inspection and search tasks, where rapid growth while carrying tip-mounted sensors is essential. In this work, we present a triangular roller tip mount that reduces internal resistance during growth by rolling rather than sliding against the
robot body. The design was refined through iterative failure analysis, enabling, for the first time, consistent eversion on a TPU-coated ripstop nylon vine
robot. To quantitatively evaluate mount performance, we introduce a custom testbed that isolates tip mounting effects by measuring tail tension during eversion. Comparative experiments across multiple mount variants, including prior designs, show that our triangular roller mount achieves the lowest tail tension and most repeatable growth performance. These results establish both a validated tip mount design and a repeatable benchmarking framework for advancing sensor and tool integration in soft growing
robots. CAD for the mount and testbed is available at: https://sprout-mitll.github.io/tip_mounts/.
Read the full abstract ▾
All authors Antonio Alvarez Valdivia, Robert Reeve, Ankush Dhawan, Ciera McFarland, Chad Council, Margaret McGuinness, Nathaniel Hanson
Robotics
RealDexUMI: A Wearable Universal Manipulation Interface for Dexterous Robot Learning
Chaoyi Xu, Yixuan Jiang, Jiahui Huan et al. (+7 more)
Learning dexterous manipulation requires demonstrations that preserve fine hand-object interactions while remaining executable at deployment. Existing pipelines either lose deployable dexterity through retargeting or embodiment conversion, or rely on
robot-specific teleoperation that is costly to scale and often lacks intuitive, contact-aware control for dexterous data collection. We present RealDexUMI, a wearable universal manipulation interface built around a shared dexterous end-effector module that integrates a lightweight dexterous hand, in-hand
vision, and fingertip tactile sensing. A palm-side isomorphic teleoperation glove maps human finger inputs to
robot-hand joint commands, enabling real-time, retargeting-free, intuitive, and precise hand control. The shared hand and sensing modules yield zero-gap end-effector data, with matched in-hand observations, tactile signals, contacts, and hand actions between collection and deployment. Across eight real-
robot tasks spanning fine-grained, contact-rich, long-horizon, and bimanual manipulation, policies trained on RealDexUMI data achieve an average success rate of 88.75%, generalize to unseen initial poses, and transfer across three embodiments. Website: https://research.beingbeyond.com/realdexumi
Read the full abstract ▾
All authors Chaoyi Xu, Yixuan Jiang, Jiahui Huan, Yuhui Fu, Haoyu Zhou, Weitian Yuan, Jiayi Yu, Wanpeng Zhang, Haoqi Yuan, Zongqing Lu
Robotics · Cross-listed from cs.AI
PLAN-S: Bridging Planning with Latent Style Dynamics for Autonomous Driving World Models
Xiaoyun Qiu, Jingtao He, Yijie Chen et al. (+4 more)
Latent world models (LWMs) have strengthened end-to-end autonomous driving by
forecasting compact scene dynamics for downstream planning. However, existing LWM-based planners usually generate trajectories directly from entangled latent representations. This compact latent-to-planner pathway lacks explicit modeling of risk, drivability, and diverse style preferences, making driving-style dynamics difficult to supervise, inspect, or modulate before a final trajectory is selected. We propose PLAN-S (PLANning with latent Style dynamics), a planner-facing bridge that addresses this compactness-controllability dilemma by decoding a style-conditioned, four-channel semantic cost map from the latent representation. The cost map is conditioned on ego state and driving style and is consumed up-stream of the planning decision through two host-side interfaces:
attention-level fusion for regression planners and reward-level fusion for anchor-score planners. We validate PLAN-S on two architecturally distinct hosts, ResWorld on nuScenes and WoTE on NAVSIM, while keeping the host backbones frozen to isolate the contribution of the proposed bridge. On nuScenes, PLAN-S reduces L2 at every horizon over the baseline, with 0.55 m average L2 and a 42% relative reduction in the 3 s collision rate. On NAVSIM, the rule-cost variant reaches 89.4 Predictive Driver Model Score (PDMS), while the learned cost variant provides complementary gains on baseline-challenging scenes. Ablations show that the cost pathway contributes most directly to safer trajectory selection. Qualitative results further show that PLAN-S can produce diverse cost maps, with spatially consistent variations aligned to different driving styles.
Read the full abstract ▾
All authors Xiaoyun Qiu, Jingtao He, Yijie Chen, Yusong Huang, Haotian Wang, Yixuan Wang, Xinhu Zheng
Robotics
Merging model-based control with multi-agent reinforcement learning for multi-agent cooperative teaming strategies
Christian Llanes, Spencer W. Jensen and Samuel Coogan
In this work, we propose a framework that combines multi-
agent reinforcement learning (MARL) with model-based control to achieve safe, dynamically feasible actions in cooperative multi-
agent tasks. Multi-
agent reinforcement learning provides the advantage of learning cooperative policies for multi-
agent teams from discrete non-differentiable rewards in a long planning horizon. Model-predictive control is robust and offers safe, dynamically feasible actions in a fast replanning framework for short horizons. We propose an
algorithm that extends actor-critic model predictive control for MARL which we refer to as multi-
agent actor-critic model predictive control (MA-AC-MPC). We demonstrate the capabilities of this
algorithm by applying it to a multi-
agent pursuit-evasion scenario. Specifically, we compare the evader team's strategy using the MA-AC-MPC model and a multi-layer perceptron model (MA-AC-MLP). The pursuer team uses augmented proportional
navigation as it is accepted as an advanced
adversarial control law. We also provide an example with a heterogeneous environment where a drone and omni-wheeled rover cooperate to achieve repeatable and successful landing with 100% success rate in hardware for MA-AC-MPC compared to 60% for MA-AC-MLP. We demonstrate the robustness of the proposed MA-AC-MPC
algorithm in hardware for both environments.
Read the full abstract ▾
Robotics
World-Language-Action Model for Unified World Modeling, Language Reasoning, and Action Synthesis
Yi Yang, Zhihong Liu, Siqi Kou et al. (+9 more)
We propose world-
language-action (WLA) models as a new class of
embodied foundation models. WLA takes textual instructions, images, and
robot states as inputs to jointly predict textual subtasks, subgoal images, and
robot actions, conjoining the world modeling interface to learn from extensive
egocentric videos as in the world-action model (WAM) and the
language reasoning capacities to solve complex long-horizon tasks as in
vision-
language-action (VLA) models. At the core of WLA lies an autoregressive (AR)
Transformer backbone, instead of a bidirectional
diffusion Transformer as in WAMs, to predict the next state, comprising the semantic-level textual intention and complementary fine-grained physical dynamics. The physical dynamics are supervised by the world modeling objective based on a dedicated World Expert, and are leveraged to ease the characterization of the state-action correlation for the Action Expert. WLA leverages meta-queries to make the world prediction implicitly impact the action generation so that the former can be disabled during inference. The world prediction can also be activated to enable test-time scaling for improved
robot control. Our WLA-0 prototype, with 2B active parameters, achieves 40 ms per inference on an NVIDIA RTX 5090. Evaluations across simulated and real-world environments demonstrate that WLA-0 achieves state-of-the-art multi-task and long-horizon learning abilities, e.g., 92.94% success rate on RoboTwin2.0 Clean and 56.5% success rate on RMBench. WLA-0 also holds the promise to learn novel tasks directly from cross-embodiment
robot videos without action annotations.
Read the full abstract ▾
All authors Yi Yang, Zhihong Liu, Siqi Kou, Yiyang Chen, Yanzhe Hu, Jianbo Zhou, Boyuan Zhao, Zhijie Wei, Xiao Xia, Xueqi Li, Pengfei Liu, Zhijie Deng
Robotics · Cross-listed from cs.CV
T-FunS3D: Task-Driven Hierarchical Open-Vocabulary 3D Functionality Segmentation
Jingkun Feng and Reza Sabzevari
Open-vocabulary 3D functionality
segmentation enables
robots to localize functional object components in 3D scenes. It is a challenging task that requires spatial understanding and task interpretation. Current open-vocabulary 3D
segmentation methods primarily focus on object-level recognition, while scene-wide part
segmentation methods attempt to segment the entire scene exhaustively, making them highly resource-intensive and time consuming. Balancing
segmentation performance in terms of granularity, accuracy, and speed remains a challenge. As one step towards alleviating this, we introduce T-FunS3D, a task-driven hierarchical open-vocabulary 3D functionality
segmentation method that provides actionable perception for robotic applications. Our method takes as input the 3D point cloud and posed RGB-D images of an indoor scene. We construct an open-vocabulary scene graph by extracting instances and their visual
embeddings in the environment. Given a task description, T-FunS3D identifies the most relevant instances in the scene graph and locates their functional components leveraging a
vision-
language model. Experiments on the SceneFun3D
dataset demonstrate that T-FunS3D is comparable to state-of-the-art in open-vocabulary 3D functionality
segmentation, while achieving faster runtime and reduced
memory usage.
Read the full abstract ▾
Robotics
Towards a Data Flywheel for Embodied Intelligence in Logistics
Anlan Yu, Zaishu Chen, Zhiqing Hong et al. (+1 more)
Embodied intelligence is moving from laboratory demonstrations toward industrial deployment, with the logistics industry serving as a key application scenario. Learning-based policies offer a promising path beyond traditional perception-planning-control pipelines, but their scalability depends on how
embodied data can be collected, organized, and reused. This research studies a data-centric framework for industrial
embodied intelligence by constructing a logistics data flywheel. Our framework converts daily operations into reusable data assets, uses World Models to generate reliable supervision for long-tail parcel manipulation, and feeds deployment feedback back into policy improvement. As an initial result, WM-DAgger introduces a World-Model-based data aggregation framework that synthesizes out-of-distribution recovery data for robust imitation learning. Building on this result, ongoing work explores how large-scale in-the-wild
multimodal data, including labeled human demonstrations, unlabeled operational
videos, and system-level
robot logs, can be aligned for policy learning and transformed into feedback for continual system improvement.
Read the full abstract ▾
All authors Anlan Yu, Zaishu Chen, Zhiqing Hong, Daqing Zhang
Robotics
Learning of Robot Safety Policies via Adversarial Synthetic Scenarios
Nikolai Dorofeev, Alexey Odinokov and Rostislav Yavorskiy
In this work, we propose an agentic gamification framework for hazard-informed learning of
robot safety policies through synthetic scenarios. We model scenario generation as an
adversarial game between two
agents: a Red Team that explores the space of potential failures by constructing hazardous situations, and a Blue Team that incrementally refines safety policies to prevent them. This iterative process enables efficient discovery of high-risk edge cases that are unlikely to be captured through random simulation or manual enumeration. By combining classical risk modeling with
adversarial scenario generation and modern learning paradigms, this work provides a scalable pathway for
embedding safety into Physical AI systems operating in complex real-world environments. The paper describes ongoing work. The contribution is a problem formulation and a proposed solution architecture.
Read the full abstract ▾