Robotics · Cross-listed from cs.CR
Property-Guided Cyber-Physical Reduction and Surrogation for Safety Analysis in Robotic Vehicles
Nazmus Shakib Sayom and Luis Garcia
We propose a methodology for falsifying safety properties in robotic vehicle systems through property-guided reduction and surrogate execution. By isolating only the control logic and physical dynamics relevant to a given specification, we construct lightweight surrogate models that preserve property-relevant behaviors while eliminating unrelated system complexity. This enables scalable falsification via trace analysis and temporal logic oracles. We demonstrate the approach on a drone control system containing a known safety flaw. The surrogate replicates failure conditions at a fraction of the simulation cost, and a property-guided fuzzer efficiently discovers semantic violations. Our results suggest that controller reduction, when coupled with logic-aware test generation, provides a practical and scalable path toward semantic verification of cyber-physical systems.
Read the full abstract ▾
Robotics
EfficientFlow: Efficient Equivariant Flow Policy Learning for Embodied AI
Jianlei Chang, Ruofeng Mei, Wei Ke et al. (+1 more)
Generative modeling has recently shown remarkable promise for visuomotor policy learning, enabling flexible and expressive control across diverse
embodied AI tasks. However, existing generative policies often struggle with data inefficiency, requiring large-scale demonstrations, and sampling inefficiency, incurring slow action generation during inference. We introduce EfficientFlow, a unified framework for efficient
embodied AI with flow-based policy learning. To enhance data efficiency, we bring equivariance into flow matching. We theoretically prove that when using an isotropic Gaussian prior and an equivariant velocity prediction network, the resulting action distribution remains equivariant, leading to improved generalization and substantially reduced data demands. To accelerate sampling, we propose a novel acceleration regularization strategy. As direct computation of acceleration is intractable for marginal flow trajectories, we derive a novel surrogate loss that enables stable and scalable training using only conditional trajectories. Across a wide range of robotic manipulation
benchmarks, the proposed
algorithm achieves competitive or superior performance under limited data while offering dramatically faster inference. These results highlight EfficientFlow as a powerful and efficient paradigm for high-performance
embodied AI.
Read the full abstract ▾
All authors Jianlei Chang, Ruofeng Mei, Wei Ke, Xiangyu Xu
Robotics · Cross-listed from cs.CV
Data-Centric Visual Development for Self-Driving Labs
Anbang Liu, Guanzhong Hu, Jiayi Wang et al. (+2 more)
Self-driving laboratories offer a promising path toward reducing the labor-intensive, time-consuming, and often irreproducible workflows in the biological sciences. Yet their stringent precision requirements demand highly robust models whose training relies on large amounts of annotated data. However, this kind of data is difficult to obtain in routine practice, especially negative samples. In this work, we focus on pipetting, the most critical and precision sensitive action in SDLs. To overcome the scarcity of training data, we build a hybrid pipeline that fuses real and virtual data generation. The real track adopts a human-in-the-loop scheme that couples automated acquisition with selective human verification to maximize accuracy with minimal effort. The virtual track augments the real data using reference-conditioned,
prompt-guided image generation, which is further screened and validated for reliability. Together, these two tracks yield a class-balanced
dataset that enables robust bubble detection training. On a held-out real test set, a model trained entirely on automatically acquired real images reaches 99.6% accuracy, and mixing real and generated data during training sustains 99.4% accuracy while reducing collection and review load. Our approach offers a scalable and cost-effective strategy for supplying visual feedback data to SDL workflows and provides a practical solution to data scarcity in rare event detection and broader
vision tasks.
Read the full abstract ▾
All authors Anbang Liu, Guanzhong Hu, Jiayi Wang, Ping Guo, Han Liu
Robotics · Cross-listed from cs.CV
Visual Sync: Multi-Camera Synchronization via Cross-View Object Motion
Shaowei Liu, David Yifan Yao, Saurabh Gupta et al. (+1 more)
Today, people can easily record memorable moments, ranging from concerts, sports events, lectures, family gatherings, and birthday parties with multiple consumer cameras. However, synchronizing these cross-camera streams remains challenging. Existing methods assume controlled settings, specific targets, manual correction, or costly hardware. We present VisualSync, an optimization framework based on multi-view dynamics that aligns unposed, unsynchronized
videos at millisecond accuracy. Our key insight is that any moving 3D point, when co-visible in two cameras, obeys epipolar constraints once properly synchronized. To exploit this, VisualSync leverages off-the-shelf 3D reconstruction, feature matching, and dense tracking to extract tracklets, relative poses, and cross-view correspondences. It then jointly minimizes the epipolar error to estimate each camera's time offset. Experiments on four diverse, challenging
datasets show that VisualSync outperforms baseline methods, achieving an median synchronization error below 50 ms.
Read the full abstract ▾
All authors Shaowei Liu, David Yifan Yao, Saurabh Gupta, Shenlong Wang
Robotics
ManualVLA: A Unified VLA Model for Chain-of-Thought Manual Generation and Robotic Manipulation
Chenyang Gu, Jiaming Liu, Hao Chen et al. (+9 more)
Vision-
Language-Action (VLA) models have recently emerged, demonstrating strong generalization in robotic scene understanding and manipulation. However, when confronted with long-horizon tasks that require defined goal states, such as LEGO assembly or object rearrangement, existing VLA models still face challenges in coordinating high-level planning with precise manipulation. Therefore, we aim to endow a VLA model with the capability to infer the "how" process from the "what" outcomes, transforming goal states into executable procedures. In this paper, we introduce ManualVLA, a unified VLA framework built upon a Mixture-of-
Transformers (MoT) architecture, enabling coherent collaboration between
multimodal manual generation and action execution. Unlike prior VLA models that directly map sensory inputs to actions, we first equip ManualVLA with a planning expert that generates intermediate manuals consisting of images, position
prompts, and textual instructions. Building upon these
multimodal manuals, we design a Manual Chain-of-Thought (ManualCoT)
reasoning process that feeds them into the action expert, where each manual step provides explicit control conditions, while its latent representation offers implicit guidance for accurate manipulation. To alleviate the burden of data collection, we develop a high-fidelity digital-twin toolkit based on 3D Gaussian Splatting, which automatically generates manual data for planning expert training. ManualVLA demonstrates strong real-world performance, achieving an average success rate 32% higher than the previous hierarchical SOTA baseline on LEGO assembly and object rearrangement tasks.
Read the full abstract ▾
All authors Chenyang Gu, Jiaming Liu, Hao Chen, Runzhong Huang, Qingpo Wuwu, Zhuoyang Liu, Xiaoqi Li, Ying Li, Renrui Zhang, Peng Jia, Pheng-Ann Heng, Shanghang Zhang
Robotics
Learning Dexterous Manipulation Skills from Imperfect Simulations
Elvis Hsieh, Wen-Han Hsieh, Yen-Jen Wang et al. (+4 more)
Reinforcement learning and sim-to-real transfer have made significant progress in dexterous manipulation. However, progress remains limited by the difficulty of simulating complex contact dynamics and multisensory signals, especially tactile feedback. In this work, we propose \ours, a sim-to-real framework that addresses these limitations and demonstrates its effectiveness on nut-bolt fastening and screwdriving with multi-fingered hands. The framework has three stages. First, we train reinforcement learning policies in simulation using simplified object models that lead to the emergence of correct finger gaits. We then use the learned policy as a skill primitive within a teleoperation system to collect real-world demonstrations that contain tactile and proprioceptive information. Finally, we train a behavior cloning policy that incorporates tactile sensing and show that it generalizes to nuts and screwdrivers with diverse geometries. Experiments across both tasks show high task progress ratios compared to direct sim-to-real transfer and robust performance even on unseen object shapes and under external perturbations.
Videos and code are available on https://dexscrew.github.io.
Read the full abstract ▾
All authors Elvis Hsieh, Wen-Han Hsieh, Yen-Jen Wang, Toru Lin, Jitendra Malik, Koushil Sreenath, Haozhi Qi
Robotics
LLM-Driven Corrective Robot Operation Code Generation with Static Text-Based Simulation
Wenhao Wang, Yi Rong, Yanyan Li et al. (+2 more)
Recent advances in Large
language models (LLMs) have demonstrated their promising capabilities of generating
robot operation code to enable LLM-driven
robots. To enhance the reliability of operation code generated by LLMs, corrective designs with feedback from the observation of executing code have been increasingly adopted in existing research. However, the code execution in these designs relies on either a physical experiment or a customized simulation environment, which limits their deployment due to the high configuration effort of the environment and the potential long execution time. In this paper, we explore the possibility of directly leveraging LLM to enable static simulation of
robot operation code, and then leverage it to design a new reliable LLM-driven corrective
robot operation code generation framework. Our framework configures the LLM as a static simulator with enhanced capabilities that reliably simulate
robot code execution by interpreting actions,
reasoning over state transitions, analyzing execution outcomes, and generating semantic observations that accurately capture trajectory dynamics. To validate the performance of our framework, we performed experiments on various operation tasks for different
robots, including
UAVs and small ground vehicles. The experiment results not only demonstrated the high accuracy of our static text-based simulation but also the reliable code generation of our LLM-driven corrective framework, which achieves a comparable performance with state-of-the-art research while does not rely on dynamic code execution using physical experiments or simulators.
Read the full abstract ▾
All authors Wenhao Wang, Yi Rong, Yanyan Li, Long Jiao, Jiawei Yuan
Robotics
Learning Sim-to-Real Humanoid Locomotion in 15 Minutes
Younggyo Seo, Carmelo Sferrazza, Juyue Chen et al. (+3 more)
Massively parallel simulation has reduced reinforcement learning (RL) training time for
robots from days to minutes. However, achieving fast and reliable sim-to-real RL for
humanoid control remains difficult due to the challenges introduced by factors such as high dimensionality and domain randomization. In this work, we introduce a simple and practical recipe based on off-policy RL
algorithms, i.e., FastSAC and FastTD3, that enables rapid training of
humanoid locomotion policies in just 15 minutes with a single RTX 4090 GPU. Our simple recipe stabilizes off-policy RL
algorithms at massive scale with thousands of parallel environments through carefully tuned design choices and minimalist reward functions. We demonstrate rapid end-to-end learning of
humanoid locomotion controllers on Unitree G1 and Booster T1
robots under strong domain randomization, e.g., randomized dynamics, rough terrain, and push perturbations, as well as fast training of whole-body human-motion tracking policies. We provide
videos and open-source implementation at: https://younggyo.me/fastsac-
humanoid.
Read the full abstract ▾
All authors Younggyo Seo, Carmelo Sferrazza, Juyue Chen, Guanya Shi, Rocky Duan, Pieter Abbeel
Robotics
RoaD: Rollouts as Demonstrations for Closed-Loop Supervised Fine-Tuning of Autonomous Driving Policies
Guillermo Garcia-Cobo, Maximilian Igl, Peter Karkus et al. (+5 more)
Autonomous driving policies are typically trained via open-loop behavior cloning of human demonstrations. However, such policies suffer from
covariate shift when deployed in closed loop, leading to compounding errors. We introduce Rollouts as Demonstrations (RoaD), a simple and efficient method to mitigate
covariate shift by leveraging the policy's own closed-loop rollouts as additional training data. During rollout generation, RoaD incorporates expert guidance to bias trajectories toward high-quality behavior, producing informative yet realistic demonstrations for fine-tuning. This approach enables robust closed-loop adaptation with orders of magnitude less data than reinforcement learning, and avoids restrictive assumptions of prior closed-loop supervised fine-tuning (CL-SFT) methods, allowing broader applications domains including end-to-end driving. We demonstrate the effectiveness of RoaD on WOSAC, a large-scale traffic simulation
benchmark, where it performs similar or better than the prior CL-SFT method; and in AlpaSim, a high-fidelity neural reconstruction-based simulator for end-to-end driving, where it improves driving score by 41% and reduces collisions by 54%.
Read the full abstract ▾
All authors Guillermo Garcia-Cobo, Maximilian Igl, Peter Karkus, Zhejun Zhang, Michael Watson, Yuxiao Chen, Boris Ivanovic, Marco Pavone
Robotics · Cross-listed from cs.LG
Forecasting in Offline Reinforcement Learning for Non-stationary Environments
Suzan Ece Ada, Georg Martius, Emre Ugur et al. (+1 more)
Offline Reinforcement Learning (RL) provides a promising avenue for training policies from pre-collected
datasets when gathering additional interaction data is infeasible. However, existing offline RL methods often assume stationarity or only consider synthetic perturbations at test time, assumptions that often fail in real-world scenarios characterized by abrupt, time-varying offsets. These offsets can lead to partial observability, causing
agents to misperceive their true state and degrade performance. To overcome this challenge, we introduce
Forecasting in Non-stationary Offline RL (FORL), a framework that unifies (i) conditional
diffusion-based candidate state generation, trained without presupposing any specific pattern of future non-stationarity, and (ii) zero-shot time-series foundation models. FORL targets environments prone to unexpected, potentially non-Markovian offsets, requiring robust
agent performance from the onset of each episode. Empirical evaluations on offline RL
benchmarks, augmented with real-world time-series data to simulate realistic non-stationarity, demonstrate that FORL consistently improves performance compared to competitive baselines. By integrating zero-shot
forecasting with the
agent's experience, we aim to bridge the gap between offline RL and the complexities of real-world, non-stationary environments.
Read the full abstract ▾
All authors Suzan Ece Ada, Georg Martius, Emre Ugur, Erhan Oztop
Robotics · Cross-listed from cs.CV
GrndCtrl: Grounding World Models via Self-Supervised Reward Alignment
Haoyang He, Jay Patrikar, Dong-Ki Kim et al. (+5 more)
Recent advances in
video world modeling have enabled large-scale generative models to simulate
embodied environments with high visual fidelity, providing strong priors for prediction, planning, and control. Yet, despite their realism, these models often lack
geometric grounding, limiting their use in
navigation tasks that require spatial coherence and stability. We introduce Reinforcement Learning with World Grounding (RLWG), a self-supervised post-training framework that aligns pretrained world models with a physically verifiable structure through
geometric and perceptual rewards. Analogous to reinforcement learning from verifiable feedback (RLVR) in
language models, RLWG can use multiple rewards that measure pose cycle-consistency, depth reprojection, and temporal coherence. We instantiate this framework with GrndCtrl, a reward-aligned adaptation method based on Group Relative Policy Optimization (GRPO), yielding world models that maintain stable trajectories, consistent geometry, and reliable rollouts for
embodied navigation. Like post-training
alignment in large
language models, GrndCtrl leverages verifiable rewards to bridge generative pretraining and grounded behavior, achieving superior spatial coherence and
navigation stability over supervised fine-tuning in outdoor environments.
Read the full abstract ▾
All authors Haoyang He, Jay Patrikar, Dong-Ki Kim, Max Smith, Daniel McGann, Ali-akbar Agha-mohammadi, Shayegan Omidshafiei, Sebastian Scherer
Robotics
Guardian: Detecting Robotic Planning and Execution Errors with Vision-Language Models
Paul Pacaud, Ricardo Garcia, Shizhe Chen et al. (+1 more)
Robust robotic manipulation requires reliable failure detection and recovery. Although recent
Vision-
Language Models (VLMs) show promise in
robot failure detection, their generalization is severely limited by the scarcity and narrow coverage of failure data. To address this bottleneck, we propose an automatic framework for generating diverse robotic planning and execution failures across both simulated and real-world environments. Our approach perturbs successful manipulation trajectories to synthesize failures that reflect realistic failure distributions, and leverages VLMs to produce structured step-by-step
reasoning traces. This yields GuardianFail-36k, a large-scale failure
reasoning dataset built upon the RLBench simulator and the BridgeDataV2 real-
robot dataset. Using GuardianFail-36k, we train Guardian, a multi-view
reasoning VLM for unified planning and execution verification. Guardian achieves state-of-the-art performance on three unseen real-world
benchmarks: RoboFail, RoboVQA, and our newly introduced UR5-Fail. When integrated with a state-of-the-art LLM-based manipulation policy, it consistently boosts task success rates in both simulation and real-world deployment. These results demonstrate that scaling high-quality failure
reasoning data is critical for improving generalization in robotic failure detection. Code, Data, and Models are available at https://www.di.ens.fr/willow/research/guardian/.
Read the full abstract ▾
All authors Paul Pacaud, Ricardo Garcia, Shizhe Chen, Cordelia Schmid
Robotics
Real-World Robot Control by Deep Active Inference With a Temporally Hierarchical World Model
Kentaro Fujii and Shingo Murata
Robots in uncertain real-world environments must perform both goal-directed and exploratory actions. However, most deep learning-based control methods neglect exploration and struggle under uncertainty. To address this, we adopt deep active inference, a framework that accounts for human goal-directed and exploratory actions. Yet, conventional deep active inference approaches face challenges due to limited environmental representation capacity and high computational cost in action selection. We propose a novel deep active inference framework that consists of a world model, an action model, and an abstract world model. The world model encodes environmental dynamics into hidden state representations at slow and fast timescales. The action model compresses action sequences into abstract actions using vector quantization, and the abstract world model predicts future slow states conditioned on the abstract action, enabling low-cost action selection. We evaluate the framework on object-manipulation tasks with a real-world
robot. Results show that it achieves high success rates across diverse manipulation tasks and switches between goal-directed and exploratory actions in uncertain settings, while making action selection computationally tractable. These findings highlight the importance of modeling multiple timescale dynamics and abstracting actions and state transitions.
Read the full abstract ▾
Robotics
NeuroHJR: Hamilton-Jacobi Reachability-based Obstacle Avoidance in Complex Environments with Physics-Informed Neural Networks
Granthik Halder, Rudrashis Majumder, Rakshith M R et al. (+2 more)
Autonomous ground vehicles (AGVs) must navigate safely in cluttered environments while accounting for complex dynamics and environmental uncertainty. Hamilton-Jacobi Reachability (HJR) offers formal safety guarantees through the computation of forward and backward reachable sets, but its application is hindered by poor scalability in environments with numerous obstacles. In this paper, we present a novel framework called NeuroHJR that leverages Physics-Informed Neural Networks (PINNs) to approximate the HJR solution for real-time obstacle avoidance. By
embedding system dynamics and safety constraints directly into the neural network loss function, our method bypasses the need for grid-based discretization and enables efficient estimation of reachable sets in continuous state spaces. We demonstrate the effectiveness of our approach through simulation results in densely cluttered scenarios, showing that it achieves safety performance comparable to that of classical HJR solvers while significantly reducing the computational cost. This work provides a new step toward real-time, scalable deployment of reachability-based obstacle avoidance in robotics.
Read the full abstract ▾
All authors Granthik Halder, Rudrashis Majumder, Rakshith M R, Rahi Shah, Suresh Sundaram
Robotics
Is Image-based Object Pose Estimation Ready to Support Grasping?
Eric C. Joyce, Qianwen Zhao, Nathaniel Burgdorfer et al. (+2 more)
We present a framework for evaluating 6-DoF instance-level object pose
estimators, focusing on those that require a single RGB (not RGB-D) image as input. Besides gaining intuition about how accurate these
estimators are, we are interested in the degree to which they can serve as the sole perception mechanism for robotic grasping. To assess this, we perform grasping trials in a physics-based simulator, using image-based pose estimates to guide a parallel gripper and an underactuated robotic hand in picking up 3D models of objects. Our experiments on a subset of the BOP (
Benchmark for 6D Object Pose Estimation)
dataset compare five open-source object pose
estimators and provide insights that were missing from the literature.
Read the full abstract ▾
All authors Eric C. Joyce, Qianwen Zhao, Nathaniel Burgdorfer, Long Wang, Philippos Mordohai
Robotics · Cross-listed from cs.CV
Register Any Point: Scaling 3D Point Cloud Registration by Flow Matching
Yue Pan, Tao Sun, Liyuan Zhu et al. (+4 more)
Point cloud registration aligns multiple unposed point clouds into a common reference frame and is a core step for 3D reconstruction and
robot localization without initial guess. In this work, we cast registration as conditional generation: a learned, continuous point-wise velocity field transports noisy points to a registered scene, from which the pose of each view is recovered. Unlike prior methods that perform correspondence matching to estimate pairwise transformations and then optimize a pose graph for multi-view registration, our model directly generates the registered point cloud, yielding both efficiency and point-level global consistency. By scaling the training data and conducting test-time
rigidity enforcement, our approach achieves state-of-the-art results on existing pairwise registration
benchmarks and on our proposed cross-domain multi-view registration
benchmark. The superior zero-shot performance on this
benchmark shows that our method generalizes across view counts, scene scales, and sensor modalities even with low overlap. Source code available at: https://github.com/PRBonn/RAP.
Read the full abstract ▾
All authors Yue Pan, Tao Sun, Liyuan Zhu, Lucas Nunes, Iro Armeni, Jens Behley, Cyrill Stachniss
Robotics
Much Ado About Noising: Dispelling the Myths of Generative Robotic Control
Chaoyi Pan, Giri Anantharaman, Nai-Chieh Huang et al. (+8 more)
Generative models, like flows and
diffusions, have recently emerged as popular and efficacious policy parameterizations in robotics. There has been much speculation as to the factors underlying their successes, ranging from capturing multi-modal action distribution to expressing more complex behaviors. In this work, we perform a comprehensive evaluation of popular generative control policies (GCPs) on common behavior cloning (BC)
benchmarks. We find that GCPs do not owe their success to their ability to capture multi-modality or to express more complex observation-to-action mappings. Instead, we find that their advantage stems from iterative computation, as long as intermediate steps are supervised during training and this supervision is paired with a suitable level of stochasticity. As a validation of our findings, we show that a minimum iterative policy (MIP), a lightweight two-step regression-based policy, essentially matches the performance of flow GCPs, and often outperforms distilled shortcut models. Our results suggest that the distribution-fitting component of GCPs is less salient than commonly believed, and point toward new design spaces focusing solely on control performance. Project page: https://simchowitzlabpublic.github.io/much-ado-about-noising-project/
Read the full abstract ▾
All authors Chaoyi Pan, Giri Anantharaman, Nai-Chieh Huang, Claire Jin, Daniel Pfrommer, Chenyang Yuan, Frank Permenter, Guannan Qu, Nicholas Boffi, Guanya Shi, Max Simchowitz
Robotics
GR-RL: Going Dexterous and Precise for Long-Horizon Robotic Manipulation
Yunfei Li, Xiao Ma, Jiafeng Xu et al. (+18 more)
We present GR-RL, a robotic learning framework that turns a generalist
vision-
language-action (VLA) policy into a highly capable specialist for long-horizon dexterous manipulation. Assuming the optimality of human demonstrations is core to existing VLA policies. However, we claim that in highly dexterous and precise manipulation tasks, human demonstrations are noisy and suboptimal. GR-RL proposes a multi-stage training pipeline that filters, augments, and reinforces the demonstrations by reinforcement learning. First, GR-RL learns a
vision-
language-conditioned task progress, filters the demonstration trajectories, and only keeps the transitions that contribute positively to the progress. Specifically, we show that by directly applying offline RL with sparse reward, the resulting $Q$-values can be treated as a robust progress function. Next, we introduce morphological
symmetry augmentation that greatly improves the generalization and performance of GR-RL. Lastly, to better align the VLA policy with its deployment behaviors for high-precision control, we perform online RL by learning a latent space noise predictor. With this pipeline, GR-RL is, to our knowledge, the first learning-based policy that can autonomously lace up a shoe by threading shoelaces through multiple eyelets with an 83.3% success rate, a task requiring long-horizon
reasoning, millimeter-level precision, and compliant soft-body interaction. We hope GR-RL provides a step toward enabling generalist
robot foundation models to specialize into reliable real-world experts.
Read the full abstract ▾
All authors Yunfei Li, Xiao Ma, Jiafeng Xu, Yu Cui, Zhongren Cui, Zhigang Han, Liqun Huang, Tao Kong, Yuxiao Liu, Hao Niu, Wanli Peng, Jingchao Qiao, Zeyu Ren, Haixin Shi, Zhi Su, Jiawen Tian, Yuyang Xiao, Shenyu Zhang, Liwei Zheng, Hang Li, Yonghui Wu
Robotics
IGen: Scalable Data Generation for Robot Learning from Open-World Images
Chenghao Gu, Haolan Kang, Junchao Lin et al. (+10 more)
The rise of generalist robotic policies has created an exponential demand for large-scale training data. However, on-
robot data collection is labor-intensive and often limited to specific environments. In contrast, open-world images capture a vast diversity of real-world scenes that naturally align with robotic manipulation tasks, offering a promising avenue for low-cost, large-scale
robot data acquisition. Despite this potential, the lack of associated
robot actions hinders the practical use of open-world images for
robot learning, leaving this rich visual resource largely unexploited. To bridge this gap, we propose IGen, a framework that scalably generates realistic visual observations and executable actions from open-world images. IGen first converts unstructured 2D pixels into structured 3D scene representations suitable for scene understanding and manipulation. It then leverages the
reasoning capabilities of
vision-
language models to transform scene-specific task instructions into high-level plans and generate low-level actions as SE(3) end-effector pose sequences. From these poses, it synthesizes dynamic scene
evolution and renders temporally coherent visual observations. Experiments validate the high quality of visuomotor data generated by IGen, and show that policies trained solely on IGen-synthesized data achieve performance comparable to those trained on real-world data. This highlights the potential of IGen to support scalable data generation from open-world images for generalist robotic policy training.
Read the full abstract ▾
All authors Chenghao Gu, Haolan Kang, Junchao Lin, Jinghe Wang, Duo Wu, Shuzhao Xie, Fanding Huang, Junchen Ge, Ziyang Gong, Letian Li, Hongying Zheng, Changwei Lv, Zhi Wang
Robotics
AgriLiRa4D: A Multi-Sensor UAV Dataset for Robust SLAM in Challenging Agricultural Fields
Zhihao Zhan, Yuhang Ming, Shaobin Li et al. (+1 more)
Multi-sensor Simultaneous Localization and Mapping (SLAM) is essential for Unmanned Aerial Vehicles (
UAVs) performing agricultural tasks such as spraying, surveying, and inspection. However, real-world, multi-modal agricultural
UAV datasets that enable research on robust operation remain scarce. To address this gap, we present AgriLiRa4D, a multi-modal
UAV dataset designed for challenging outdoor agricultural environments. AgriLiRa4D spans three representative farmland types-flat, hilly, and terraced-and includes both boundary and coverage operation modes, resulting in six flight sequence groups. The
dataset provides high-accuracy ground-truth trajectories from a Fiber Optic Inertial
Navigation System with Real-Time Kinematic capability (FINS_RTK), along with synchronized measurements from a 3D
LiDAR, a 4D Radar, and an Inertial Measurement Unit (IMU), accompanied by complete intrinsic and extrinsic calibrations. Leveraging its comprehensive sensor suite and diverse real-world scenarios, AgriLiRa4D supports diverse SLAM and localization studies and enables rigorous robustness evaluation against low-texture crops, repetitive patterns, dynamic vegetation, and other challenges of real agricultural environments. To further demonstrate its utility, we
benchmark four state-of-the-art multi-sensor SLAM
algorithms across different sensor combinations, highlighting the difficulty of the proposed sequences and the necessity of multi-modal approaches for reliable
UAV localization. By filling a critical gap in agricultural SLAM
datasets, AgriLiRa4D provides a valuable
benchmark for the research community and contributes to advancing autonomous
navigation technologies for agricultural
UAVs. The
dataset can be downloaded from: https://zhan994.github.io/AgriLiRa4D.
Read the full abstract ▾
All authors Zhihao Zhan, Yuhang Ming, Shaobin Li, Jie Yuan
Robotics
Transport Discrepancy as a Reliability Signal for Vision-Language-Action Models
Wanpeng Zhang, Ye Wang, Hao Luo et al. (+6 more)
Vision-
language-action (VLA) models that generate continuous action chunks via flow matching lack an internal signal for judging whether a given prediction is reliable. Distribution shift and long-horizon rollouts can push backbone representations away from the region the action head decodes reliably, yet the policy has no mechanism to detect or react to this drift. We observe that the cost of transporting observation features to the action representation in a shared feature space rises precisely when such drift occurs, providing a per-step reliability estimate without extra supervision. Building on this observation, we propose DiG (Discrepancy Gate), a lightweight plug-in module for flow-matching VLA policies. DiG computes a sliced Wasserstein transport cost between backbone features and the action expert's own input projection, maps it through an exponential gate, and uses the gate to modulate both a residual feature refinement and the training loss. At inference time, the gate enables DiG-Refinefine, an iterative refinement process that corrects action chunks before execution. Experiments on both simulation and real-world scenarios show that DiG consistently improves success rates, with the largest gains under distribution shift and on long-horizon tasks.
Read the full abstract ▾
All authors Wanpeng Zhang, Ye Wang, Hao Luo, Haoqi Yuan, Yicheng Feng, Chaoyi Xu, Sipeng Zheng, Qin Jin, Zongqing Lu
Robotics
Dynamic Log-Gaussian Process Control Barrier Function for Safe Robotic Navigation in Dynamic Environments
Xin Yin, Chenyang Liang, Yanning Guo et al. (+1 more)
Control Barrier Functions (CBFs) have emerged as efficient tools to address the safe
navigation problem for
robot applications. However, synthesizing informative and obstacle motion-aware CBFs online using real-time sensor data remains challenging, particularly in unknown and dynamic scenarios. Motived by this challenge, this paper aims to propose a novel Gaussian Process-based formulation of CBF, termed the Dynamic Log Gaussian Process Control Barrier Function (DLGP-CBF), to enable real-time construction of CBF which are both spatially informative and responsive to obstacle motion. Firstly, the DLGP-CBF leverages a logarithmic transformation of GP regression to generate smooth and informative barrier values and gradients, even in sparse-data regions. Secondly, by explicitly modeling the DLGP-CBF as a function of obstacle positions, the derived safety constraint integrates predicted obstacle velocities, allowing the controller to proactively respond to dynamic obstacles' motion. Simulation results demonstrate significant improvements in obstacle avoidance performance, including increased safety margins, smoother trajectories, and enhanced responsiveness compared to baseline methods.
Read the full abstract ▾
All authors Xin Yin, Chenyang Liang, Yanning Guo, Jie Mei
Robotics · Cross-listed from cs.CV
SPARK: Sim-ready Part-level Articulated Reconstruction with VLM Knowledge
Yumeng He, Ying Jiang, Jiayin Lu et al. (+2 more)
Articulated 3D objects are critical for
embodied AI, robotics, and interactive scene understanding, yet creating simulation-ready assets remains labor-intensive and requires expert modeling of part hierarchies and motion structures. We introduce SPARK, a framework for reconstructing physically consistent, kinematic part-level articulated objects from a single RGB image. Given an input image, we first leverage VLMs to extract coarse URDF parameters and generate part-level reference images. We then integrate the part-image guidance and the inferred structure graph into a generative
diffusion transformer to synthesize consistent part and complete shapes of articulated objects. To further refine the URDF parameters, we incorporate differentiable forward kinematics and differentiable rendering to optimize joint types, axes, and origins under VLM-generated open-state supervision. Extensive experiments show that SPARK produces high-quality, simulation-ready articulated assets across diverse categories, enabling downstream applications such as robotic manipulation and interaction modeling. Project page: https://heyumeng.com/SPARK/index.html.
Read the full abstract ▾
All authors Yumeng He, Ying Jiang, Jiayin Lu, Yin Yang, Chenfanfu Jiang
Robotics
Integrated YOLOP Perception and Lyapunov-based Control for Autonomous Mobile Robot Navigation on Track
Mo Chen
This work presents a real-time autonomous track
navigation framework for nonholonomic differential-drive mobile
robots by jointly integrating multi-task visual perception and a provably stable tracking controller. The perception pipeline reconstructs lane centerlines using 2D-to-3D camera projection, arc-length based uniform point resampling, and cubic
polynomial fitting solved via robust QR least-squares optimization. The controller regulates
robot linear and angular velocities through a Lyapunov-stability grounded design, ensuring bounded error dynamics and asymptotic convergence of position and heading deviations even in dynamic and partially perceived lane scenarios, without relying on HD prior maps or global satellite localization. Real-world experiments on embedded platforms verify system fidelity, real-time execution, trajectory smoothness, and closed-loop stability for reliable autonomous
navigation.
Read the full abstract ▾
Robotics
A Cross-Embodiment Gripper Benchmark for Rigid-Object Manipulation in Aerial and Industrial Robotics
Marek Vagas, Martin Varga, Jaroslav Romancik et al. (+5 more)
Robotic grippers are increasingly deployed across industrial, collaborative, and aerial platforms, where each embodiment imposes distinct mechanical, energetic, and operational constraints. Established YCB and NIST
benchmarks quantify grasp success, force, or timing on a single platform, but do not evaluate cross-embodiment transferability or energy-aware performance, capabilities essential for modern mobile and aerial manipulation. This letter introduces the Cross-Embodiment Gripper
Benchmark (CEGB), a compact and reproducible benchmarking suite extending YCB and selected NIST metrics with three additional components: a transfer-time
benchmark measuring the practical effort required to exchange embodiments, an energy-consumption
benchmark evaluating grasping and holding efficiency, and an intent-specific ideal payload assessment reflecting design-dependent operational capability. Together, these metrics characterize both grasp performance and the suitability of reusing a single gripper across heterogeneous robotic systems. A lightweight self-locking gripper prototype is implemented as a reference case. Experiments demonstrate rapid embodiment transfer (median ~= 17.6 s across user groups), low holding energy for gripper prototype (~= 1.5 J per 10 s), and consistent grasp performance with cycle times of 3.2 - 3.9 s and success rates exceeding 90%. CEGB thus provides a reproducible foundation for cross-platform, energy-aware evaluation of grippers in aerial and manipulators domains.
Read the full abstract ▾
All authors Marek Vagas, Martin Varga, Jaroslav Romancik, Ondrej Majercak, Alejandro Suarez, Anibal Ollero, Bram Vanderborght, Ivan Virgala
Robotics
L2M-Calib: One-key Calibration Method for LiDAR and Multiple Magnetic Sensors
Qiyang Lyu, Wei Wang, Zhenyu Wu et al. (+3 more)
Multimodal sensor fusion enables robust environmental perception by leveraging complementary information from heterogeneous sensing modalities. However, accurate calibration is a critical prerequisite for effective fusion. This paper proposes a novel one-key calibration framework named L2M-Calib for a fused
magnetic-
LiDAR system, jointly estimating the extrinsic transformation between the two kinds of sensors and the intrinsic distortion parameters of the
magnetic sensors.
Magnetic sensors capture ambient
magnetic field (AMF) patterns, which are invariant to geometry, texture, illumination, and weather, making them suitable for challenging environments. Nonetheless, the integration of
magnetic sensing into
multimodal systems remains underexplored due to the absence of effective calibration techniques. To address this, we optimize extrinsic parameters using an iterative Gauss-Newton scheme, coupled with the intrinsic calibration as a weighted ridge-regularized total least squares (w-RRTLS) problem, ensuring robustness against measurement noise and ill-conditioned data. Extensive evaluations on both simulated
datasets and real-world experiments, including AGV-mounted sensor configurations, demonstrate that our method achieves high calibration accuracy and robustness under various environmental and operational conditions.
Read the full abstract ▾
All authors Qiyang Lyu, Wei Wang, Zhenyu Wu, Hongming Shen, Huiqin Zhou, Danwei Wang
Robotics
NavForesee: A Unified Vision-Language World Model for Hierarchical Planning and Dual-Horizon Navigation Prediction
Fei Liu, Shichao Xie, Minghua Luo et al. (+4 more)
Embodied navigation for long-horizon tasks, guided by complex natural
language instructions, remains a formidable challenge in artificial intelligence. Existing
agents often struggle with robust long-term planning about unseen environments, leading to high failure rates. To address these limitations, we introduce NavForesee, a novel
Vision-
Language Model (VLM) that unifies high-level
language planning and predictive world model imagination within a single, unified framework. Our approach empowers a single VLM to concurrently perform planning and predictive foresight. Conditioned on the full instruction and historical observations, the model is trained to understand the
navigation instructions by decomposing the task, tracking its progress, and formulating the subsequent sub-goal. Simultaneously, it functions as a generative world model, providing crucial foresight by predicting short-term environmental dynamics and long-term
navigation milestones. The VLM's structured plan guides its targeted prediction, while the imagined future provides rich context to inform the
navigation actions, creating a powerful internal feedback loop of perception-planning/prediction-action. We demonstrate through extensive experiments on the R2R-CE and RxR-CE
benchmark that NavForesee achieves highly competitive performance in complex scenarios. Our work highlights the immense potential of fusing explicit
language planning with implicit spatiotemporal prediction, paving the way for more intelligent and capable
embodied agents.
Read the full abstract ▾
All authors Fei Liu, Shichao Xie, Minghua Luo, Zedong Chu, Junjun Hu, Xiaolong Wu, Mu Xu
Robotics
$\mathbf{M^3A}$ Policy: Mutable Material Manipulation Augmentation Policy through Photometric Re-rendering
Jiayi Li, Yuxuan Hu, Haoran Geng et al. (+4 more)
Material generalization is essential for real-world robotic manipulation, where
robots must interact with objects exhibiting diverse visual and physical properties. This challenge is particularly pronounced for objects made of glass, metal, or other materials whose transparent or reflective surfaces introduce severe out-of-distribution variations. Existing approaches either rely on simulated materials in simulators and perform sim-to-real transfer, which is hindered by substantial visual domain gaps, or depend on collecting extensive real-world demonstrations, which is costly, time-consuming, and still insufficient to cover various materials. To overcome these limitations, we resort to computational photography and introduce Mutable Material Manipulation Augmentation (M$^3$A), a unified framework that leverages the physical characteristics of materials as captured by light transport for photometric re-rendering. The core idea is simple yet powerful: given a single real-world demonstration, we photometrically re-render the scene to generate a diverse set of highly realistic demonstrations with different material properties. This augmentation effectively decouples task-specific manipulation skills from surface appearance, enabling policies to generalize across materials without additional data collection. To systematically evaluate this capability, we construct the first comprehensive multi-material manipulation
benchmark spanning both simulation and real-world environments. Extensive experiments show that the M$^3$A policy significantly enhances cross-material generalization, improving the average success rate across three real-world tasks by 58.03%, and demonstrating robust performance on previously unseen materials.
Read the full abstract ▾
All authors Jiayi Li, Yuxuan Hu, Haoran Geng, Xiangyu Chen, Chuhao Zhou, Ziteng Cui, Jianfei Yang
Robotics · Cross-listed from cs.OS
Accelerating Probabilistic Response-Time Analysis: Revised Critical Instant and Optimized Convolution
Hiroto Takahashi, Atsushi Yano and Takuya Azumi
Accurate estimation of the Worst-Case Deadline Failure Probability (WCDFP) has attracted growing
attention as a means to provide safety assurances in complex systems such as robotic platforms and autonomous vehicles. WCDFP quantifies the likelihood of deadline misses under the most pessimistic operating conditions, and safe estimation is essential for dependable real-time applications. However, achieving high accuracy in WCDFP estimation often incurs significant computational cost. Recent studies have revealed that the classical assumption of the critical instant, the activation pattern traditionally considered to trigger the worst-case behavior, can lead to underestimation of WCDFP in probabilistic settings. This observation motivates the use of a revised critical instant formulation that more faithfully captures the true worst-case scenario. This paper investigates convolution-based methods for WCDFP estimation under this revised setting and proposes an optimization technique that accelerates convolution by improving the merge order. Extensive experiments with diverse execution-time distributions demonstrate that the proposed optimized Aggregate Convolution reduces computation time by up to an order of magnitude compared to Sequential Convolution, while retaining accurate and safe-sided WCDFP estimates. These results highlight the potential of the approach to provide both efficiency and reliability in probabilistic timing analysis for safety-critical real-time applications.
Read the full abstract ▾
Robotics
Modality-Augmented Fine-Tuning of Foundation Robot Policies for Cross-Embodiment Manipulation on GR1 and G1
Junsung Park, Hogun Kee and Songhwai Oh
This paper presents a modality-augmented fine-tuning framework designed to adapt foundation
robot policies to diverse
humanoid embodiments. We validate our approach across two distinct settings: (i) the GR1 embodiment, utilizing public
datasets where we introduce post-processed modalities, including binary contact signals and ZoeDepth-generated metric depth; and (ii) the Unitree G1 embodiment, for which we contribute a novel multi-modal
dataset incorporating cuRobo motion planning, inverse kinematics, and ground-truth contact-force measurements. Our experiments demonstrate that modality augmentation consistently enhances policy performance across different embodiments. Specifically, for the GR1, integrating contact-state cues and RGB-D fusion improves online success rates from 51% to 63%. Furthermore, in the G1 "Pick Apple to Bowl" task, our contact-augmented model achieves a success rate of 94%, significantly outperforming the 48% achieved by standard fine-tuning and the 0% baseline of zero-shot transfer. These results highlight that lightweight post-processing effectively strengthens policies for GR1, while high-quality multi-modal data is crucial for reliable transfer to the Unitree G1. Consequently, this work establishes a unified, data-centric pathway for extending foundation
robot policies through targeted modality design and multi-modal fine-tuning.
Read the full abstract ▾