Robotics
R2RGEN: Real-to-Real 3D Data Generation for Spatially Generalized Manipulation
Xiuwei Xu, Angyuan Ma, Hankun Li et al. (+4 more)
Towards the aim of generalized robotic manipulation, spatial generalization is the most fundamental capability that requires the policy to work robustly under different spatial distribution of objects, environment and
agent itself. To achieve this, substantial human demonstrations need to be collected to cover different spatial configurations for training a generalized visuomotor policy via imitation learning. Prior works explore a promising direction that leverages data generation to acquire abundant spatially diverse data from minimal source demonstrations. However, most approaches face significant sim-to-real gap and are often limited to constrained settings, such as fixed-base scenarios and predefined camera viewpoints. In this paper, we propose a real-to-real 3D data generation framework (R2RGen) that directly augments the pointcloud observation-action pairs to generate real-world data. R2RGen is simulator- and rendering-free, thus being efficient and plug-and-play. Specifically, we propose a unified three-stage framework, which (1) pre-processes source demonstrations under different camera setups in a shared 3D space with scene / trajectory parsing; (2) augments objects and
robot's position with a group-wise backtracking strategy; (3) aligns the distribution of generated data with real-world 3D sensor using camera-aware post-processing. Empirically, R2RGen substantially enhances data efficiency on extensive experiments and demonstrates strong potential for scaling and application on mobile manipulation.
Read the full abstract ▾
All authors Xiuwei Xu, Angyuan Ma, Hankun Li, Bingyao Yu, Zheng Zhu, Jie Zhou, Jiwen Lu
Robotics · Cross-listed from cs.CV
Have We Scene It All? Scene Graph-Aware Deep Point Cloud Compression
Nikolaos Stathoulopoulos, Christoforos Kanellakis and George Nikolakopoulos
Efficient transmission of 3D point cloud data is critical for advanced perception in centralized and decentralized multi-
agent robotic systems, especially nowadays with the growing reliance on edge and cloud-based processing. However, the large and complex nature of point clouds creates challenges under bandwidth constraints and intermittent connectivity, often degrading system performance. We propose a deep compression framework based on semantic scene graphs. The method decomposes point clouds into semantically coherent patches and encodes them into compact latent representations with semantic-aware encoders conditioned by Feature-wise Linear Modulation (FiLM). A folding-based decoder, guided by latent features and graph node attributes, enables structurally accurate reconstruction. Experiments on the SemanticKITTI and nuScenes
datasets show that the framework achieves state-of-the-art compression rates, reducing data size by up to 98% while preserving both structural and semantic fidelity. In addition, it supports downstream applications such as multi-
robot pose graph optimization and map merging, achieving trajectory accuracy and map
alignment comparable to those obtained with raw
LiDAR scans.
Read the full abstract ▾
Robotics
DexMan: Learning Bimanual Dexterous Manipulation from Human and Generated Videos
Jhen Hsieh, Kuan-Hsun Tu, Kuo-Han Hung et al. (+1 more)
We present DexMan, an automated framework that converts human visual demonstrations into bimanual dexterous manipulation skills for
humanoid robots in simulation. Operating directly on third-person
videos of humans manipulating rigid objects, DexMan eliminates the need for camera calibration, depth sensors, scanned 3D object assets, or ground-truth hand and object motion annotations. Unlike prior approaches that consider only simplified floating hands, it directly controls a
humanoid robot and leverages novel contact-based rewards to improve policy learning from noisy hand-object poses estimated from in-the-wild
videos. DexMan achieves state-of-the-art performance in object pose estimation on the TACO
benchmark, with absolute gains of 0.08 and 0.12 in ADD-S and VSD. Meanwhile, its reinforcement learning policy surpasses previous methods by 19% in success rate on OakInk-v2. Furthermore, DexMan can generate skills from both real and synthetic
videos, without the need for manual data collection and costly motion capture, and enabling the creation of large-scale, diverse
datasets for training generalist dexterous manipulation.
Read the full abstract ▾
All authors Jhen Hsieh, Kuan-Hsun Tu, Kuo-Han Hung, Tsung-Wei Ke
Robotics
Don't Run with Scissors: Pruning Breaks VLA Models but They Can Be Recovered
Jason Jabbour, Dong-Ki Kim, Max Smith et al. (+6 more)
Vision-
Language-Action (VLA) models are a promising paradigm for field robotics, but their deployment is bottlenecked by strict edge SWaP-C constraints. Compression techniques such as quantization,
distillation, and
pruning are well-established for LLMs and VLMs.
Pruning, however, remains particularly underexplored for VLA models. We surprisingly find that
pruning VLA models yields drastic performance degradation and increased safety violations. Through spectral analysis, we identify structural properties of VLA weights that help explain this fragility. Based on this insight, we introduce GLUESTICK, a post-
pruning recovery method that restores much of the original model's capability while preserving sparsity benefits. GLUESTICK computes a one-time corrective term by interpolating between the dense and pruned models in weight space. At inference, each pruned layer applies this correction to recover lost performance with minimal overhead. Our approach requires no additional training and is agnostic to the
pruning algorithm. Across diverse VLA architectures and domains, GLUESTICK recovers up to 100% of the success rate and restores safety to near-dense levels, while saving up to 5 GB of
memory and achieving up to 1.75x speedup. Additional material can be found at: https://gluestick-vla.github.io/
Read the full abstract ▾
All authors Jason Jabbour, Dong-Ki Kim, Max Smith, Jay Patrikar, Radhika Ghosal, Youhui Wang, Ali Agha, Vijay Janapa Reddi, Shayegan Omidshafiei
Robotics · Cross-listed from cs.CV
Gaze on the Prize: Shaping Visual Attention with Return-Guided Contrastive Learning
Andrew Lee, Ian Chuang, Dechen Gao et al. (+2 more)
Visual Reinforcement Learning (RL)
agents must learn to act based on high-dimensional image data where only a small fraction of the pixels is task-relevant. This forces
agents to waste exploration and computational resources on irrelevant features, leading to sample-inefficient and unstable learning. To address this, inspired by human visual foveation, we introduce Gaze on the Prize. This framework augments visual RL with a learnable foveal
attention mechanism (Gaze), guided by a self-supervised signal derived from the
agent's experience pursuing higher returns (the Prize). Our key insight is that return differences reveal what matters most: If two similar representations produce different outcomes, their distinguishing features are likely task-relevant, and the gaze should focus on them accordingly. This is realized through return-guided contrastive learning that trains the
attention to distinguish between the features relevant to success and failure. We group similar visual representations into positives and negatives based on their return differences and use the resulting labels to construct contrastive triplets. These triplets provide the training signal that teaches the
attention mechanism to produce distinguishable representations for states associated with different outcomes. Our method achieves up to 2.52x improvement in sample efficiency and can solve challenging tasks from the ManiSkill3
benchmark that the baseline fails to learn, without modifying the underlying
algorithm or hyperparameters.
Read the full abstract ▾
All authors Andrew Lee, Ian Chuang, Dechen Gao, Kai Fukazawa, Iman Soltani
Robotics
Validation of collision-free spheres of Stewart-Gough platforms for constant orientations using the Application Programming Interface of a CAD software
Bibekananda Patra, Rajeevlochana G. Chittawadigi and Sandipan Bandyopadhyay
This paper presents a method of validation of the size of the largest collision-free sphere (CFS) of a 6-6 Stewart-Gough platform manipulator (SGPM) for a given orientation of its moving platform (MP) using the Application Programming Interface (API) of a CAD software. The position of the MP is updated via the API in an automated manner over a set of samples within a shell enclosing the surface of the CFS. For each pose of the manipulator, each pair of legs is investigated for mutual collisions. The CFS is considered safe or validated iff none of the points falling inside the CFS lead to a collision between any pair of legs. This approach can not only validate the safety of a precomputed CFS, but also estimate the same for any spatial parallel manipulator.
Read the full abstract ▾
Robotics
Reliability of Single-Level Equality-Constrained Inverse Optimal Control
Filip Bečanović, Kosta Jovanović and Vincent Bonnet
Inverse optimal control (IOC) allows the
retrieval of optimal cost function weights, or behavioral parameters, from human motion. The literature on IOC uses methods that are either based on a slow bilevel process or a fast but noise-sensitive minimization of optimality condition violation. Assuming equality-constrained optimal control models of human motion, this article presents a faster but robust approach to solving IOC using a single-level reformulation of the bilevel method and yields equivalent results. Through numerical experiments in simulation, we analyze the robustness to noise of the proposed single-level reformulation to the bilevel IOC formulation with a human-like planar reaching task that is used across recent studies. The approach shows resilience to very large levels of noise and reduces the computation time of the IOC on this task by a factor of 15 when compared to a classical bilevel implementation.
Read the full abstract ▾
Robotics
Airy: Reading Robot Intent through Height and Sky
Baoyang Chen, Xian Xu and Huamin Qu
As industrial
robots move into shared human spaces, their opaque decision making threatens safety, trust, and public oversight. This artwork, Airy, asks whether complex multi
agent AI can become intuitively understandable by staging a competition between two reinforcement trained
robot arms that snap a bedsheet skyward. Building on three design principles, competition as a clear metric (who lifts higher),
embodied familiarity (audiences recognize fabric snapping), and sensor to sense mapping (
robot cooperation or rivalry shown through forest and weather projections), the installation gives viewers a visceral way to read machine intent. Observations from five international exhibitions indicate that audiences consistently read the
robots' strategies, conflict, and cooperation in real time, with emotional reactions that mirror the system's internal state. The project shows how sensory metaphors can turn a black box into a public interface.
Read the full abstract ▾
Robotics · Cross-listed from cs.NE
Co-design is powerful and not free
Yi Zhang, Yue Xie, Tao Sun et al. (+1 more)
Robotic performance emerges from the coupling of body and controller, yet it remains unclear when morphology-control co-design is necessary. We present a unified framework that embeds morphology and control parameters within a single neural network, enabling end-to-end joint optimization. Through case studies in static-obstacle-constrained reaching, we evaluate trajectory error, success rate, and collision probability. The results show that co-design provides clear benefits when morphology is poorly matched to the task, such as near obstacles or workspace boundaries, where structural adaptation simplifies control. Conversely, when the baseline morphology already affords sufficient capability, control-only optimization often matches or exceeds co-design. By clarifying when control is enough and when it is not, this work advances the understanding of
embodied intelligence and offers practical guidance for embodiment-aware
robot design.
Read the full abstract ▾
All authors Yi Zhang, Yue Xie, Tao Sun, Fumiya Iida
Robotics · Cross-listed from cs.CV
A Multimodal Depth-Aware Method For Embodied Reference Understanding
Fevziye Irem Eyiokur, Dogucan Yaman, Haz{\i}m Kemal Ekenel et al. (+1 more)
Embodied Reference Understanding requires identifying a target object in a visual scene based on both
language instructions and pointing cues. While prior works have shown progress in open-vocabulary object detection, they often fail in ambiguous scenarios where multiple candidate objects exist in the scene. To address these challenges, we propose a novel ERU framework that jointly leverages LLM-based data augmentation, depth-map modality, and a depth-aware decision module. This design enables robust integration of linguistic and
embodied cues, improving disambiguation in complex or cluttered environments. Experimental results on two
datasets demonstrate that our approach significantly outperforms existing baselines, achieving more accurate and reliable referent detection.
Read the full abstract ▾
All authors Fevziye Irem Eyiokur, Dogucan Yaman, Haz{\i}m Kemal Ekenel, Alexander Waibel
Robotics
Evaluation of a Robust Control System in Real-World Cable-Driven Parallel Robots
Damir Nurtdinov, Aliaksei Korshuk, Alexei Kornaev et al. (+1 more)
This study evaluates the performance of classical and modern control methods for real-world Cable-Driven Parallel
Robots (CDPRs), focusing on underconstrained systems with limited time discretization. A comparative analysis is conducted between classical PID controllers and modern reinforcement learning
algorithms, including Deep Deterministic Policy Gradient (DDPG), Proximal Policy Optimization (PPO), and Trust Region Policy Optimization (TRPO). The results demonstrate that TRPO outperforms other methods, achieving the lowest root mean square (RMS) errors across various trajectories and exhibiting robustness to larger time intervals between control updates. TRPO's ability to balance exploration and exploitation enables stable control in noisy, real-world environments, reducing reliance on high-frequency sensor feedback and computational demands. These findings highlight TRPO's potential as a robust solution for complex robotic control tasks, with implications for dynamic environments and future applications in sensor fusion or hybrid control strategies.
Read the full abstract ▾
All authors Damir Nurtdinov, Aliaksei Korshuk, Alexei Kornaev, Alexander Maloletov
Robotics
NavSpace: How Navigation Agents Follow Spatial Intelligence Instructions
Haolin Yang, Yuxing Long, Zhuoyuan Yu et al. (+8 more)
Read the full abstract ▾
All authors Haolin Yang, Yuxing Long, Zhuoyuan Yu, Zihan Yang, Minghan Wang, Jiapeng Xu, Yihan Wang, Ziyan Yu, Wenzhe Cai, Lei Kang, Hao Dong
Robotics
Accurate and Noise-Tolerant Extraction of Routine Logs in Robotic Process Automation (Extended Version)
Massimiliano de Leoni, Faizan Ahmed Khan and Simone Agostinelli
Robotic Process Mining focuses on the identification of the routine types performed by human resources through a User Interface. The ultimate goal is to discover routine-type models to enable robotic process automation. The discovery of routine-type models requires the provision of a routine log. Unfortunately, the vast majority of existing works do not directly focus on enabling the model discovery, limiting themselves to extracting the set of actions that are part of the routines. They were also not evaluated in scenarios characterized by inconsistent routine execution, hereafter referred to as noise, which reflects natural variability and occasional errors in human performance. This paper presents a clustering-based technique that aims to extract routine logs. Experiments were conducted on nine UI logs from the literature with different levels of injected noise. Our technique was compared with existing techniques, most of which are not meant to discover routine logs but were adapted for the purpose. The results were evaluated through standard state-of-the-art metrics, showing that we can extract more accurate routine logs than what the state of the art could, especially in the presence of noise.
Read the full abstract ▾
Robotics
Beyond hospital reach: Autonomous lightweight ultrasound robot for liver sonography
Zihan Li, Yixiao Xu, Lei Zhang et al. (+8 more)
Liver disease is a major global health burden. While ultrasound is the first-line diagnostic tool, liver sonography requires locating multiple non-continuous planes from positions where target structures are often not visible, for biometric assessment and lesion detection, requiring significant expertise. However, expert sonographers are severely scarce in resource-limited regions. Here, we develop an autonomous lightweight ultrasound
robot comprising an AI
agent that integrates multi-modal perception with
memory attention for localization of unseen target structures, and a 588-gram 6-degrees-of-freedom cable-driven
robot. By mounting on the abdomen, the system enhances robustness against motion. Our
robot can autonomously acquire expert-level standard liver ultrasound planes and detect pathology in patients, including two from Xining, a 2261-meter-altitude city with limited medical resources. Our system performs effectively on rapid-motion individuals and in wilderness environments. This work represents the first demonstration of autonomous sonography across multiple challenging scenarios, potentially transforming access to expert-level diagnostics in underserved regions.
Read the full abstract ▾
All authors Zihan Li, Yixiao Xu, Lei Zhang, Taiyu Han, Xinshan Yang, Yingni Wang, Mingxuan Liu, Shenghai Xin, Linxun Liu, Hongen Liao, Guochen Ning
Robotics
Towards Reliable LLM-based Robot Planning via Combined Uncertainty Estimation
Shiyuan Yin, Chenjia Bai, Zihao Zhang et al. (+4 more)
Large
language models (LLMs) demonstrate advanced
reasoning abilities, enabling
robots to understand natural
language instructions and generate high-level plans with appropriate grounding. However, LLM
hallucinations present a significant challenge, often leading to overconfident yet potentially misaligned or unsafe plans. While researchers have explored uncertainty estimation to improve the reliability of LLM-based planning, existing studies have not sufficiently differentiated between epistemic and intrinsic uncertainty, limiting the effectiveness of uncertainty estimation. In this paper, we present Combined Uncertainty estimation for Reliable
Embodied planning (CURE), which decomposes the uncertainty into epistemic and intrinsic uncertainty, each estimated separately. Furthermore, epistemic uncertainty is subdivided into task clarity and task familiarity for more accurate evaluation. The overall uncertainty assessments are obtained using random network
distillation and multi-layer perceptron regression heads driven by LLM features. We validated our approach in two distinct experimental settings: kitchen manipulation and tabletop rearrangement experiments. The results show that, compared to existing methods, our approach yields uncertainty estimates that are more closely aligned with the actual execution outcomes.
Read the full abstract ▾
All authors Shiyuan Yin, Chenjia Bai, Zihao Zhang, Junwei Jin, Xinxin Zhang, Chi Zhang, Xuelong Li
Robotics
FastUMI-100K: Advancing Data-driven Robotic Manipulation with a Large-scale UMI-style Dataset
Kehui Liu, Zhongjie Jia, Yang Li et al. (+14 more)
Data-driven robotic manipulation learning depends on large-scale, high-quality expert demonstration
datasets. However, existing
datasets, which primarily rely on human teleoperated
robot collection, are limited in terms of scalability, trajectory smoothness, and applicability across different robotic embodiments in real-world environments. In this paper, we present FastUMI-100K, a large-scale UMI-style
multimodal demonstration
dataset, designed to overcome these limitations and meet the growing complexity of real-world manipulation tasks. Collected by FastUMI, a novel robotic system featuring a modular, hardware-decoupled mechanical design and an integrated lightweight tracking system, FastUMI-100K offers a more scalable, flexible, and adaptable solution to fulfill the diverse requirements of real-world
robot demonstration data. Specifically, FastUMI-100K contains over 100K+ demonstration trajectories collected across representative household environments, covering 54 tasks and hundreds of object types. Our
dataset integrates
multimodal streams, including end-effector states, multi-view wrist-mounted fisheye images and textual annotations. Each trajectory has a length ranging from 120 to 500 frames. Experimental results demonstrate that FastUMI-100K enables high policy success rates across various baseline
algorithms, confirming its robustness, adaptability, and real-world applicability for solving complex, dynamic manipulation challenges. The source code and
dataset will be released in this link https://github.com/MrKeee/FastUMI-100K.
Read the full abstract ▾
All authors Kehui Liu, Zhongjie Jia, Yang Li, Zhaxizhuoma, Pengan Chen, Song Liu, Xin Liu, Pingrui Zhang, Haoming Song, Xinyi Ye, Nieqing Cao, Zhigang Wang, Jia Zeng, Dong Wang, Yan Ding, Bin Zhao, Xuelong Li
Robotics
Orientation Learning and Adaptation towards Simultaneous Incorporation of Multiple Local Constraints
Gaofeng Li, Peisen Xu, Ruize Wang et al. (+4 more)
Orientation learning plays a pivotal role in many tasks. However, the rotation group SO(3) is a Riemannian
manifold. As a result, the distortion caused by non-Euclidean
geometric nature introduces difficulties to the incorporation of local constraints, especially for the simultaneous incorporation of multiple local constraints. To address this issue, we propose the Angle-Axis Space-based orientation representation method to solve several orientation learning problems, including orientation adaptation and minimization of angular acceleration. Specifically, we propose a weighted average mechanism in SO(3) based on the angle-axis representation method. Our main idea is to generate multiple trajectories by considering different local constraints at different basepoints. Then these multiple trajectories are fused to generate a smooth trajectory by our proposed weighted average mechanism, achieving the goal to incorporate multiple local constraints simultaneously. Compared with existing solution, ours can address the distortion issue and make the off-theshelf Euclidean learning
algorithm be re-applicable in non-Euclidean space. Simulation and Experimental evaluations validate that our solution can not only adapt orientations towards arbitrary desired via-points and cope with angular acceleration constraints, but also incorporate multiple local constraints simultaneously to achieve extra benefits, e.g., achieving smaller acceleration costs.
Read the full abstract ▾
All authors Gaofeng Li, Peisen Xu, Ruize Wang, Qi Ye, Jiming Chen, Dezhen Song, Yanlong Huang
Robotics
Executable Analytic Concepts as the Missing Link Between VLM Insight and Precise Manipulation
Mingyang Sun, Jiude Wei, Qichen He et al. (+3 more)
Enabling
robots to perform precise and generalized manipulation in unstructured environments remains a fundamental challenge in
embodied AI. While
Vision-
Language Models (VLMs) have demonstrated remarkable capabilities in semantic
reasoning and task planning, a significant gap persists between their high-level understanding and the precise physical execution required for real-world manipulation. To bridge this "semantic-to-physical" gap, we introduce GRACE, a novel framework that grounds VLM-based
reasoning through executable analytic concepts (EAC)-mathematically defined blueprints that encode object affordances,
geometric constraints, and semantics of manipulation. Our approach integrates a structured policy scaffolding pipeline that turn natural
language instructions and visual information into an instantiated EAC, from which we derive grasp poses, force directions and plan physically feasible motion trajectory for
robot execution. GRACE thus provides a unified and interpretable interface between high-level instruction understanding and low-level
robot control, effectively enabling precise and generalizable manipulation through semantic-physical grounding. Extensive experiments demonstrate that GRACE achieves strong zero-shot generalization across a variety of articulated objects in both simulated and real-world environments, without requiring task-specific training.
Read the full abstract ▾
All authors Mingyang Sun, Jiude Wei, Qichen He, Donglin Wang, Cewu Lu, Jianhua Sun
Robotics
Towards Proprioception-Aware Embodied Planning for Dual-Arm Humanoid Robots
Boyu Li, Siyuan He, Hang Xu et al. (+10 more)
In recent years,
Multimodal Large
Language Models (MLLMs) have demonstrated the ability to serve as high-level planners, enabling
robots to follow complex human instructions. However, their effectiveness, especially in long-horizon tasks involving dual-arm
humanoid robots, remains limited. This limitation arises from two main challenges: (i) the absence of simulation platforms that systematically support task evaluation and data collection for
humanoid robots, and (ii) the insufficient embodiment awareness of current MLLMs, which hinders
reasoning about dual-arm selection logic and body positions during planning. To address these issues, we present DualTHOR, a new dual-arm
humanoid simulator, with continuous transition and a contingency mechanism. Building on this platform, we propose Proprio-MLLM, a model that enhances embodiment awareness by incorporating proprioceptive information with motion-based position
embedding and a cross-spatial encoder. Experiments show that, while existing MLLMs struggle in this environment, Proprio-MLLM achieves an average improvement of 19.75% in planning performance. Our work provides both an essential simulation platform and an effective model to advance
embodied intelligence in
humanoid robotics. The code is available at https://anonymous.4open.science/r/DualTHOR-5F3B.
Read the full abstract ▾
All authors Boyu Li, Siyuan He, Hang Xu, Haoqi Yuan, Xinrun Xu, Yu Zang, Liwei Hu, Junpeng Yue, Zhenxiong Jiang, Pengbo Hu, Börje F. Karlsson, Yehui Tang, Zongqing Lu
Robotics
Learning to Navigate Socially Through Proactive Risk Perception
Erjia Xiao, Lingfeng Zhang, Yingbo Tang et al. (+7 more)
In this report, we describe the technical details of our submission to the IROS 2025 RoboSense Challenge Social
Navigation Track. This track focuses on developing RGBD-based perception and
navigation systems that enable autonomous
agents to navigate safely, efficiently, and socially compliantly in dynamic human-populated indoor environments. The challenge requires
agents to operate from an
egocentric perspective using only onboard sensors including RGB-D observations and odometry, without access to global maps or privileged information, while maintaining social norm compliance such as safe distances and collision avoidance. Building upon the Falcon model, we introduce a Proactive Risk Perception Module to enhance social
navigation performance. Our approach augments Falcon with collision risk understanding that learns to predict distance-based collision risk scores for surrounding humans, which enables the
agent to develop more robust spatial awareness and proactive collision avoidance behaviors. The evaluation on the Social-HM3D
benchmark demonstrates that our method improves the
agent's ability to maintain personal space compliance while navigating toward goals in crowded indoor scenes with dynamic human
agents, achieving 2nd place among 16 participating teams in the challenge.
Read the full abstract ▾
All authors Erjia Xiao, Lingfeng Zhang, Yingbo Tang, Hao Cheng, Renjing Xu, Wenbo Ding, Lei Zhou, Long Chen, Hangjun Ye, Xiaoshuai Hao
Robotics
Junwen Gu, Zhiheng Wu, Pengxuan Si et al. (+8 more)
Underwater environments pose unique challenges for robotic
navigation and manipulation. While existing research has primarily focused on task-specific methods, studies on general-purpose intelligence for multi-task execution remain scarce. To address this gap, we propose a unified framework for general-purpose
underwater robots that integrates perception and action driven by
language instructions. First, we develop a data synthesis pipeline to construct USIM, a simulation-based
dataset which comprises over 905K frames from 2275 trajectories, totaling approximately 25 hours of BlueROV2 interactions. Furthermore, we propose U0, a
vision-
language-action (VLA) model capable of executing various tasks from obstacle-avoidance
navigation to three-dimensional mobile manipulation. The model features a convolution-
attention-based perception (CAP) module, which incorporates target pose estimation as an auxiliary task to explicitly bolster the model's spatial awareness. For evaluation, we establish a systematic assessment framework and an automated pipeline encompassing both offline metrics and online task execution. Experimental results demonstrate that the USIM
dataset significantly empowers existing VLA models to adapt to
underwater scenarios. Notably, our U0 model achieves state-of-the-art performance: it reduces the offline mean action prediction error to 0.0359 and achieves an overall online success rate of 43.1%, marking a 5.5% improvement over existing competitive baselines (below 37.6%), with
navigation tasks reaching as high as 87.5%. These results validate the feasibility of general-purpose intelligence in
underwater robotics, providing a foundation for scalable
dataset synthesis and aquatic
embodied agents.
Read the full abstract ▾
All authors Junwen Gu, Zhiheng Wu, Pengxuan Si, Shuang Qiu, Zhentao Zhang, Yukai Feng, Luoyang Sun, Laien Luo, Lianyi Yu, Jian Wang, Zhengxing Wu
Robotics
DM1: MeanFlow with Dispersive Regularization for 1-Step Robotic Manipulation
Guowei Zou, Haitao Wang, Hejun Wu et al. (+3 more)
The ability to learn multi-modal action distributions is indispensable for robotic manipulation policies to perform precise and robust control. Flow-based generative models have recently emerged as a promising solution to learning distributions of actions, offering one-step action generation and thus achieving much higher sampling efficiency compared to
diffusion-based methods. However, existing flow-based policies suffer from representation collapse, the inability to distinguish similar visual representations, leading to failures in precise manipulation tasks. We propose DM1 (MeanFlow with Dispersive Regularization for One-Step Robotic Manipulation), a novel flow matching framework that integrates dispersive regularization into MeanFlow to prevent collapse while maintaining one-step efficiency. DM1 employs multiple dispersive regularization variants across different intermediate
embedding layers, encouraging diverse representations across training batches without introducing additional network modules or specialized training procedures. Experiments on RoboMimic
benchmarks show that DM1 achieves 20-40 times faster inference (0.07s vs. 2-3.5s) and improves success rates by 10-20 percentage points, with the Lift task reaching 99% success over 85% of the baseline. Real-
robot deployment on a Franka Panda further validates that DM1 transfers effectively from simulation to the physical world. To the best of our knowledge, this is the first work to leverage representation regularization to enable flow-based policies to achieve strong performance in robotic manipulation, establishing a simple yet powerful approach for efficient and robust manipulation.
Read the full abstract ▾
All authors Guowei Zou, Haitao Wang, Hejun Wu, Yukun Qian, Yuhang Wang, Weibing Li
Robotics
GM3: A General Physical Model for Micro-Mobility Vehicles
Grace Cai, Nithin Parepally, Laura Zheng et al. (+1 more)
Modeling the dynamics of micro-mobility vehicles (MMV) is becoming increasingly important for training autonomous vehicle systems and building urban traffic simulations. However, mainstream tools rely on variants of the Kinematic Bicycle Model (KBM) or mode-specific physics that miss tire slip, load transfer, and rider/vehicle lean. To our knowledge, no unified, physics-based model captures these dynamics across the full range of common MMVs and wheel layouts. We propose the "Generalized Micro-mobility Model" (GM3), a tire-level formulation based on the tire brush representation that supports arbitrary wheel configurations, including single/double track and multi-wheel platforms. We introduce an interactive model-agnostic simulation framework that decouples vehicle/layout specification from dynamics to compare the GM3 with the KBM and other models, consisting of fixed step RK4 integration, human-in-the-loop and scripted control, real-time trajectory traces and logging for analysis. We also empirically validate the GM3 on the Stanford Drone
Dataset's deathCircle (roundabout) scene for biker, skater, and cart classes.
Read the full abstract ▾
All authors Grace Cai, Nithin Parepally, Laura Zheng, Ming C. Lin
Robotics
IntentionVLA: Generalizable and Efficient Embodied Intention Reasoning for Human-Robot Interaction
Yandu Chen, Kefan Gu, Yuqing Wen et al. (+3 more)
Vision-
Language-Action (VLA) models leverage pretrained
vision-
language models (VLMs) to couple perception with robotic control, offering a promising path toward general-purpose
embodied intelligence. However, current SOTA VLAs are primarily pretrained on
multimodal tasks with limited relevance to
embodied scenarios, and then finetuned to map explicit instructions to actions. Consequently, due to the lack of
reasoning-intensive pretraining and
reasoning-guided manipulation, these models are unable to perform implicit human intention
reasoning required for complex, real-world interactions. To overcome these limitations, we propose IntentionVLA, a VLA framework with a curriculum training paradigm and an efficient inference mechanism. Our proposed method first leverages carefully designed
reasoning data that combine intention inference, spatial grounding, and compact
embodied reasoning, endowing the model with both
reasoning and perception capabilities. In the following finetuning stage, IntentionVLA employs the compact
reasoning outputs as contextual guidance for action generation, enabling fast inference under indirect instructions. Experimental results show that IntentionVLA substantially outperforms $\pi_0$, achieving 18% higher success rates with direct instructions and 28% higher than ECoT under intention instructions. On out-of-distribution intention tasks, IntentionVLA achieves over twice the success rate of all baselines, and further enables zero-shot human-
robot interaction with 40% success rate. These results highlight IntentionVLA as a promising paradigm for next-generation human-
robot interaction (HRI) systems.
Read the full abstract ▾
All authors Yandu Chen, Kefan Gu, Yuqing Wen, Yucheng Zhao, Tiancai Wang, Liqiang Nie
Robotics
Trajectory Conditioned Cross-embodiment Skill Transfer
YuHang Tang, Yixuan Lou, Pengfei Han et al. (+4 more)
Learning manipulation skills from human demonstration
videos presents a promising yet challenging problem, primarily due to the significant embodiment gap between human body and
robot manipulators. Existing methods rely on paired
datasets or hand-crafted rewards, which limit scalability and generalization. We propose TrajSkill, a framework for Trajectory Conditioned Cross-embodiment Skill Transfer, enabling
robots to acquire manipulation skills directly from human demonstration
videos. Our key insight is to represent human motions as sparse optical flow trajectories, which serve as embodiment-agnostic motion cues by removing morphological variations while preserving essential dynamics. Conditioned on these trajectories together with visual and textual inputs, TrajSkill jointly synthesizes temporally consistent
robot manipulation
videos and translates them into executable actions, thereby achieving cross-embodiment skill transfer. Extensive experiments are conducted, and the results on simulation data (MetaWorld) show that TrajSkill reduces FVD by 39.6% and KVD by 36.6% compared with the state-of-the-art, and improves cross-embodiment success rate by up to 16.7%. Real-
robot experiments in kitchen manipulation tasks further validate the effectiveness of our approach, demonstrating practical human-to-
robot skill transfer across embodiments.
Read the full abstract ▾
All authors YuHang Tang, Yixuan Lou, Pengfei Han, Haoming Song, Xinyi Ye, Dong Wang, Bin Zhao
Robotics
Injecting Hallucinations in Autonomous Vehicles: A Component-Agnostic Safety Evaluation Framework
Alexandre Moreira Nascimento, Gabriel Kenji Godoy Shimanuki, Lúcio Flavio Vismari et al. (+5 more)
Perception failures in autonomous vehicles (AV) remain a major safety concern because they can directly compromise decision-making and contribute to unsafe behavior. To study how these failures affect safety, researchers typically inject artificial faults into real-world or simulated components and observe the resulting outcomes. However, existing fault injection (FI) studies often target a single sensor within the machine perception (MP) module, resulting in siloed frameworks that are difficult to generalize or integrate into unified simulation environments. This work addresses that limitation by reframing perception failures as
hallucinations, that is, false perceptions that distort an AV's situational awareness and may trigger unsafe control actions. Building on this concept, we propose a configurable, component-agnostic
hallucination injection framework that induces six plausible
hallucination types in an open-source simulation environment. A total of 18,356 simulations (8,695 baseline and 9,661 with
hallucinations injected) were executed while AVs crossed an unsignalized intersection with transversal traffic. The results statistically validate the framework and quantify the impact of each
hallucination type on collisions and on the minimum distance between the AV and the nearest vehicle, used as a continuous proxy for the safety margin. Certain
hallucinations, especially missed detection, blind region, angular drift, phantom perception, and latency, significantly increase the odds of collision, while linear drift significantly reduces minimum distances without producing a statistically significant increase in collision odds in the evaluated scenario. The proposed framework provides a scalable and statistically grounded method for evaluating AV safety under perception distortions and supports future research on fault tolerance and resilient AV design.
Read the full abstract ▾
All authors Alexandre Moreira Nascimento, Gabriel Kenji Godoy Shimanuki, Lúcio Flavio Vismari, João Batista Camargo Jr, Jorge Rady de Almeida Jr, Paulo Sergio Cugnasca, Anna Carolina Muller Queiroz, Jeremy Noah Bailenson
Robotics · Cross-listed from cs.LG
DEAS: DEtached value learning with Action Sequence for Scalable Offline RL
Changyeon Kim, Haeone Lee, Younggyo Seo et al. (+2 more)
Offline reinforcement learning (RL) presents an attractive paradigm for training intelligent
agents without expensive online interactions. However, current approaches still struggle with complex, long-horizon sequential decision making. In this work, we introduce DEtached value learning with Action Sequence (DEAS), a simple yet effective offline RL framework that leverages action sequences for value learning. These temporally extended actions provide richer information than single-step actions and can be interpreted through the options framework via semi-Markov decision process Q-learning, enabling reduction of the effective planning horizon by considering longer sequences at once. However, directly adopting such sequences in actor-critic
algorithms introduces excessive value overestimation, which we address through detached value learning that steers value estimates toward in-distribution actions that achieve high return in the offline
dataset. We demonstrate that DEAS consistently outperforms baselines on complex, long-horizon tasks from OGBench and can be applied to enhance the performance of large-scale
Vision-
Language-Action models that predict action sequences, significantly boosting performance in both RoboCasa Kitchen simulation tasks and real-world manipulation tasks.
Read the full abstract ▾
All authors Changyeon Kim, Haeone Lee, Younggyo Seo, Kimin Lee, Yuke Zhu
Robotics
Probabilistically-Safe Bipedal Navigation over Uncertain Terrain via Conformal Prediction and Contraction Analysis
Kasidit Muenprasitivej, Ye Zhao and Glen Chou
We address the challenge of enabling bipedal
robots to traverse rough terrain by developing probabilistically safe planning and control strategies that ensure dynamic feasibility and centroidal robustness under terrain uncertainty. Specifically, we propose a high-level Model Predictive Control (MPC)
navigation framework for a bipedal
robot with a specified confidence level of safety that (i) enables safe traversal toward a desired goal location across a terrain map with uncertain elevations, and (ii) formally incorporates uncertainty bounds into the centroidal dynamics of locomotion control. To model the rough terrain, we employ Gaussian Process (GP) regression to estimate elevation maps and leverage Conformal Prediction (CP) to construct calibrated confidence intervals that capture the true terrain elevation. Building on this, we formulate contraction-based reachable tubes that explicitly account for terrain uncertainty, ensuring state convergence and tube invariance. In addition, we introduce a contraction-based flywheel torque control law for the reduced-order Linear Inverted Pendulum Model (LIPM), which stabilizes the angular momentum about the center-of-mass (CoM). This formulation provides both probabilistic safety and goal reachability guarantees. For a given confidence level, we establish the forward invariance of the proposed torque control law by demonstrating exponential stabilization of the actual CoM phase-space trajectory and the desired trajectory prescribed by the high-level planner. Finally, we evaluate the effectiveness of our planning framework through physics-based simulations of the Digit bipedal
robot in MuJoCo.
Read the full abstract ▾
Robotics
EB-MBD: Emerging-Barrier Model-Based Diffusion for Safe Trajectory Optimization in Highly Constrained Environments
Raghav Mishra and Ian R. Manchester
We propose enforcing constraints on Model-Based
Diffusion by introducing emerging barrier functions inspired by interior point methods. We demonstrate that the standard Model-Based
Diffusion algorithm can lead to catastrophic performance degradation in highly constrained environments, even on simple 2D systems due to sample inefficiency in the Monte Carlo approximation of the score function. We introduce Emerging-Barrier Model-Based
Diffusion (EB-MBD) which uses progressively introduced barrier constraints to avoid these problems, significantly improving solution quality, without expensive projection operations such as projections. We analyze the sampling liveliness of samples at each iteration to inform barrier parameter scheduling choice. We demonstrate results for 2D collision avoidance and a 3D
underwater manipulator system and show that our method achieves lower cost solutions than Model-Based
Diffusion, and requires orders of magnitude less computation time than projection based methods.
Read the full abstract ▾
Robotics
Differentiable Particle Optimization for Fast Sequential Manipulation
Lucas Chen, Shrutheesh Raman Iyer and Zachary Kingston
Sequential
robot manipulation tasks require finding collision-free trajectories that satisfy
geometric constraints across multiple object interactions in potentially high-dimensional configuration spaces. Solving these problems in real-time and at large scales has remained out of reach due to computational requirements. Recently, GPU-based acceleration has shown promising results, but prior methods achieve limited performance due to CPU-GPU data transfer overhead and complex logic that prevents full hardware utilization. To this end, we present SPaSM (Sampling Particle optimization for Sequential Manipulation), a fully GPU-parallelized framework that compiles constraint evaluation, sampling, and gradient-based optimization into optimized CUDA kernels for end-to-end trajectory optimization without CPU coordination. The method consists of a two-stage particle optimization strategy: first solving placement constraints through massively parallel sampling, then lifting solutions to full trajectory optimization in joint space. Unlike hierarchical approaches, SPaSM jointly optimizes object placements and
robot trajectories to handle scenarios where motion feasibility constrains placement options. Experimental evaluation on challenging
benchmarks demonstrates solution times in the realm of $\textbf{milliseconds}$ with a 100% success rate; a $4000\times$ speedup compared to existing approaches. Code and examples are available at $\href{https://commalab.org/papers/spasm}{commalab.org/papers/spasm}$.
Read the full abstract ▾