Robotics · Cross-listed from cs.GR
NaP-Control: Navigating Diffusion Prior for Versatile and Fast Character Control
Chia-Wen Chen, Yan Wu, Korrawe Karunratanakul et al. (+1 more)
Achieving precise, versatile whole-body character control in physics-based animation remains challenging. Recent
diffusion-based policies generate rich and expressive motions but typically rely on gradient-based test-time guidance to satisfy task objectives, which is slow and can reduce robustness. We introduce NaP-Control (Navigating
Diffusion Prior for Versatile and Fast Character Control), abbreviated as NaP. Our method uses reinforcement learning to manipulate the latent noise of a task-agnostic
diffusion policy prior, steering it toward task-specific behaviors for fast, robust control with high motion fidelity. In contrast to methods that rely solely on offline training, NaP interacts with the environment during training to correct motions and optimize task rewards, improving success rates and enabling adaptation to challenging scenarios. By directly predicting task-optimized
diffusion noise, NaP eliminates iterative guidance during denoising and enables efficient inference. Experiments show that NaP attains higher success rates and faster inference while preserving natural motion across diverse tasks.
Read the full abstract ▾
All authors Chia-Wen Chen, Yan Wu, Korrawe Karunratanakul, Siyu Tang
Robotics · Cross-listed from cs.LG
Positive-Only Drifting Policy Optimization
Qi Zhang
In the field of online reinforcement learning (RL), traditional Gaussian policies and flow-based methods are often constrained by their unimodal expressiveness, complex gradient clipping, or stringent trust-region requirements. Moreover, they all rely on post-hoc penalization of negative samples to correct erroneous actions. This paper introduces Positive-Only Drifting Policy Optimization (PODPO), a likelihood-free and gradient-clipping-free generative approach for online RL. By leveraging the drifting model, PODPO performs policy updates via advantage-weighted local contrastive drifting. Relying solely on positive-advantage samples, it elegantly steers actions toward high-return regions while exploiting the inherent local smoothness of the generative model to enable proactive error prevention. In doing so, PODPO opens a promising new pathway for generative policy learning in online settings.
Read the full abstract ▾
Robotics
On-Orbit Space AI: Federated, Multi-Agent, and Collaborative Algorithms for Satellite Constellations
Ziyang Wang
Satellite constellations are transforming space systems from isolated spacecraft into networked, software-defined platforms capable of on-orbit perception, decision making, and adaptation. Yet much of the existing AI studies remains centered on single-satellite inference, while constellation-scale autonomy introduces fundamentally new algorithmic requirements: learning and coordination under dynamic inter-satellite connectivity, strict SWaP-C limits, radiation-induced faults, non-IID data, concept drift, and safety-critical operational constraints. This survey consolidates the emerging field of on-orbit space AI through three complementary paradigms: (i) {
federated learning} for cross-satellite training, personalization, and secure aggregation; (ii) {multi-
agent algorithms} for cooperative planning, resource allocation, scheduling, formation control, and collision avoidance; and (iii) {collaborative sensing and distributed inference} for multi-satellite fusion, tracking, split/early-exit inference, and cross-layer co-design with constellation networking. We provide a system-level view and a taxonomy that unifies collaboration architectures, temporal mechanisms, and trust models. To support community development and keep this review actionable over time, we continuously curate relevant papers and resources at https://github.com/ziyangwang007/AI4Space.
Read the full abstract ▾
Robotics
Learning-Guided Sparsification of Dynamic Graphs in Robotic Exploration
Adithya V. Sastry, Bibek Poudel and Weizi Li
Many robotic exploration
algorithms rely on graph structures for frontier-based exploration and dynamic path planning. However, these graphs grow rapidly, accumulating redundant information and impacting performance. We present a hybrid
transformer-based framework trained with Proximal Policy Optimization which complements exploration
algorithms by
pruning these graphs during exploration, limiting their growth and reducing the accumulation of excess information. The framework was evaluated on simulations of a robotic
agent using Rapidly-Exploring Random Trees to carry out frontier-based exploration, where the learned policy reduces graph size by up to 96%. We find preliminary evidence that our framework enables effective, generalizable exploration under reduced information density, consistently outperforming the randomly pruned baseline and improving both exploration efficiency and computational efficiency beyond the unpruned baseline for complex long-horizon exploration tasks. To the best of our knowledge, these results are the first suggesting the viability of RL to prune dynamic graphs used in robotic exploration
algorithms.
Read the full abstract ▾
Robotics
Behavior Cloning Under PD Control: A Finite-Horizon Theory of Gain-Dependent Error Amplification
Junghoon Seo
Behavior cloning (BC) on position-controlled
robots is shaped by the PD loop that executes policy actions. We give a finite-horizon, nonasymptotic analysis of how controller gains affect BC failure. Independent sub-Gaussian action errors propagate through gain-dependent closed-loop dynamics into sub-Gaussian position errors. The resulting failure tail is controlled by controller amplification multiplied by validation loss and generalization slack, so validation loss alone can mis-rank gains. Under shape-preserving upper-bound assumptions, the analysis separates label difficulty, injection strength, and contraction, ranking compliant-overdamped gains as tightest and stiff-underdamped gains as loosest, with the mixed regimes system-dependent. In the canonical scalar second-order PD system, stationary position-error variance increases with stiffness and decreases with damping over the stable range, and exact zero-order-hold discretization inherits the ordering to leading order. This extends the error-attenuation explanation of bronars et al. (2026) to finite-horizon failure bounds.
Read the full abstract ▾
Robotics
CooperDrive: Enhancing Driving Decisions Through Cooperative Perception
Deyuan Qu, Qi Chen, Takayuki Shimizu et al. (+1 more)
Autonomous vehicles equipped with robust onboard perception, localization, and planning still face limitations in occlusion and non-line-of-sight (NLOS) scenarios, where delayed reactions can increase collision risk. We propose CooperDrive, a cooperative perception framework that augments situational awareness and enables earlier, safer driving decisions. CooperDrive offers two key advantages: (i) each vehicle retains its native perception, localization, and planning stack, and (ii) a lightweight object-level sharing and fusion strategy bridges perception and planning. Specifically, CooperDrive reuses detector Bird's-Eye View (BEV) features to estimate accurate vehicle poses without additional heavy encoders, thereby reconstructing BEV representations and feeding the planner with low latency. On the planning side, CooperDrive leverages the expanded object set to anticipate potential conflicts earlier and adjust speed and trajectory proactively, thereby transforming reactive behaviors into predictive and safer driving decisions. Real-world closed-loop tests at occlusion-heavy NLOS intersections demonstrate that CooperDrive increases reaction lead time, minimum time-to-collision (TTC), and stopping margin, while requiring only 90 kbps bandwidth and maintaining an average end-to-end latency of 89 ms.
Read the full abstract ▾
All authors Deyuan Qu, Qi Chen, Takayuki Shimizu, Onur Altintas
Robotics
BIEVR-LIO: Robust LiDAR-Inertial Odometry through Bump-Image-Enhanced Voxel Maps
Patrick Pfreundschuh, Turcan Tuna, Cedric Le Gentil et al. (+3 more)
Reliable odometry is essential for mobile
robots as they increasingly enter more challenging environments, which often contain little information to constrain point cloud registration, resulting in degraded
LiDAR-Inertial Odometry (LIO) accuracy or even divergence. To address this, we present BIEVR-LIO, a novel approach designed specifically to exploit subtle variations in the available geometry for improved robustness. We propose a high-resolution map representation that stores surfaces as voxel-wise oriented height images. This representation can directly be used for registration without the calculation of intermediate
geometric primitives while still supporting efficient updates. Since informative geometry is often sparsely distributed in the environment, we further propose a map-informed point sampling strategy to focus registration on geometrically informative regions, improving robustness in uninformative environments while reducing computational cost compared to global high-resolution sampling. Experiments across multiple sensors, platforms, and environments demonstrate state-of-the-art performance in well-constrained scenes and substantial improvements in challenging scenarios where baseline methods diverge. Additionally, we demonstrate that the fine-grained geometry captured by BIEVR-LIO can be used for downstream tasks such as elevation mapping for
robot locomotion.
Read the full abstract ▾
All authors Patrick Pfreundschuh, Turcan Tuna, Cedric Le Gentil, Roland Siegwart, Cesar Cadena, Helen Oleynikova
Robotics
SpaceMind: A Modular and Self-Evolving Embodied Vision-Language Agent Framework for Autonomous On-orbit Servicing
Aodi Wu, Haodong Han, Xubo Luo et al. (+3 more)
Autonomous on-orbit servicing demands
embodied agents that perceive through visual sensors, reason about 3D spatial situations, and execute multi-phase tasks over extended horizons. We present SpaceMind, a modular and self-evolving
vision-
language model (VLM)
agent framework that decomposes knowledge, tools, and
reasoning into three independently extensible dimensions: skill modules with dynamic routing, Model Context Protocol (MCP) tools with configurable profiles, and injectable
reasoning-mode skills. An MCP-Redis interface layer enables the same codebase to operate across simulation and physical hardware without modification, and a Skill Self-
Evolution mechanism distills operational experience into persistent skill files without model fine-tuning. We validate SpaceMind through 192 closed-loop runs across five satellites, three task types, and two environments, a UE5 simulation and a physical laboratory, deliberately including degraded conditions to stress-test robustness. Under nominal conditions all modes achieve 90–100%
navigation success; under degradation, the Prospective mode uniquely succeeds in search-and-approach tasks where other modes fail. A self-
evolution study shows that the
agent recovers from failure in four of six groups from a single failed episode, including complete failure to 100% success and inspection scores improving from 12 to 59 out of 100. Real-world validation confirms zero-code-modification transfer to a physical
robot with 100% rendezvous success. Code: https://github.com/wuaodi/SpaceMind
Read the full abstract ▾
All authors Aodi Wu, Haodong Han, Xubo Luo, Ruisuo Wang, Shan He, Xue Wan
Robotics
RoSLAC: Robust Simultaneous Localization and Calibration of Multiple Magnetometers
Qiyang Lyu, Zhenyu Wu, Wei Wang et al. (+2 more)
Localization of autonomous mobile
robots (AMRs) in enclosed or semi-enclosed environments such as offices, hotels, hospitals, indoor parking facilities, and underground spaces where GPS signals are weak or unavailable remains a major obstacle to the deployment of fully autonomous systems. Infrastructure-based localization approaches, such as QR codes and RFID, are constrained by high installation and maintenance costs as well as limited flexibility, while onboard sensor-based methods, including
LiDAR- and
vision-based solutions, are affected by ambiguous
geometric features and frequent occlusions caused by dynamic obstacles such as pedestrians. Ambient
magnetic field (AMF)-based localization has therefore attracted growing interest in recent years because it does not rely on external infrastructure or
geometric features, making it well-suited for AMR applications such as service
robots and security
robots. However, magnetometer measurements are often corrupted by distortions caused by
ferromagnetic materials present on the sensor platform, which bias the AMF and degrade localization reliability. As a result, accurate magnetometer calibration to estimate distortion parameters becomes essential. Conventional calibration methods that rely on rotating the magnetometer are impractical for large and heavy platforms. To address this limitation, this paper proposes a robust simultaneous localization and calibration (RoSLAC) approach based on alternating optimization, which iteratively and efficiently estimates both the platform pose and magnetometer calibration parameters. Extensive evaluations conducted in high-fidelity simulation and real-world environments demonstrate that the proposed RoSLAC method achieves high localization accuracy while maintaining low computational cost compared with state-of-the-art magnetometer calibration techniques.
Read the full abstract ▾
All authors Qiyang Lyu, Zhenyu Wu, Wei Wang, Hongming Shen, Danwei Wang
Robotics
CART: Context-Aware Terrain Adaptation using Temporal Sequence Selection for Legged Robots
Kartikeya Singh, Youngjin Kim, Yash Turkar et al. (+1 more)
Animals in nature combine multiple modalities, such as sight and feel, to perceive terrain and develop an understanding of how to walk on uneven terrain in an efficient manner. Similarly, legged
robots need to develop their ability to stably walk on complex terrains by developing an understanding of the relationship between
vision and proprioception. Most current terrain-adaptation methods remain susceptible to failure on complex off-road terrain because they do not explicitly model the context between exteroceptive terrain appearance and proprioceptive physical interaction. This experience-based learning often creates a Visual-Texture Paradox between what has been seen and how it actually feels. In this work, we introduce CART, a high-level controller built on a context-aware terrain adaptation approach that integrates proprioception and exteroception from onboard sensing to achieve a robust understanding of terrain. We evaluate our method on multiple terrains using the Unitree Go2 and ANYmal-C
robot on the IsaacSim simulator and a Boston Dynamics SPOT
robot for our real-world experiments. To evaluate whether the learned context improves locomotion behavior under the various paradox circumstances, we measure the
robot s stability, traversal success, and task completion time in both simulation and real-world experiments. We compare CART against state-of-the-art locomotion and terrain- adaptation baselines across diverse terrain conditions. CART improves the average success rate by 5% over the baselines in simulation, while improving context-conditioned locomotion behavior, including up to 41% lower base oscillation in simulation and 22% in the real world, without increasing the time required to complete the locomotion tasks.
Read the full abstract ▾
All authors Kartikeya Singh, Youngjin Kim, Yash Turkar, Karthik Dantu
Robotics · Cross-listed from cs.CV
HiVLA: A Visual-Grounded-Centric Hierarchical Embodied Manipulation System
Tianshuo Yang, Guanyu Chen, Yutian Chen et al. (+8 more)
While end-to-end
Vision-
Language-Action (VLA) models offer a promising paradigm for robotic manipulation, fine-tuning them on narrow control data often compromises the profound
reasoning capabilities inherited from their base
Vision-
Language Models (VLMs). To resolve this fundamental trade-off, we propose HiVLA, a visual-grounded-centric hierarchical framework that explicitly decouples high-level semantic planning from low-level motor control. In high-level part, a VLM planner first performs task decomposition and visual grounding to generate structured plans, comprising a subtask instruction and a precise target bounding box. Then, to translate this plan into physical actions, we introduce a flow-matching
Diffusion Transformer (DiT) action expert in low-level part equipped with a novel cascaded cross-
attention mechanism. This design sequentially fuses global context, high-resolution object-centric crops and skill semantics, enabling the DiT to focus purely on robust execution. Our decoupled architecture preserves the VLM's zero-shot
reasoning while allowing independent improvement of both components. Extensive experiments in simulation and the real world demonstrate that HiVLA significantly outperforms state-of-the-art end-to-end baselines, particularly excelling in long-horizon skill composition and the fine-grained manipulation of small objects in cluttered scenes.
Read the full abstract ▾
All authors Tianshuo Yang, Guanyu Chen, Yutian Chen, Zhixuan Liang, Yitian Liu, Zanxin Chen, Chunpu Xu, Haotian Liang, Jiangmiao Pang, Yao Mu, Ping Luo
Robotics
UMI-3D: Extending Universal Manipulation Interface from Vision-Limited to 3D Spatial Perception
Ziming Wang
We present UMI-3D, a
multimodal extension of the Universal Manipulation Interface (UMI) for robust and scalable data collection in
embodied manipulation. While UMI enables portable, wrist-mounted data acquisition, its reliance on
monocular visual SLAM makes it vulnerable to occlusions, dynamic scenes, and tracking failures, limiting its applicability in real-world environments. UMI-3D addresses these limitations by introducing a lightweight and low-cost
LiDAR sensor tightly integrated into the wrist-mounted interface, enabling
LiDAR-centric SLAM with accurate metric-scale pose estimation under challenging conditions. We further develop a hardware-synchronized
multimodal sensing pipeline and a unified spatiotemporal calibration framework that aligns visual observations with
LiDAR point clouds, producing consistent 3D representations of demonstrations. Despite maintaining the original 2D visuomotor policy formulation, UMI-3D significantly improves the quality and reliability of collected data, which directly translates into enhanced policy performance. Extensive real-world experiments demonstrate that UMI-3D not only achieves high success rates on standard manipulation tasks, but also enables learning of tasks that are challenging or infeasible for the original
vision-only UMI setup, including large deformable object manipulation and articulated object operation. The system supports an end-to-end pipeline for data acquisition,
alignment, training, and deployment, while preserving the portability and accessibility of the original UMI. All hardware and software components are open-sourced to facilitate large-scale data collection and accelerate research in
embodied intelligence: \href{https://umi-3d.github.io}{https://umi-3d.github.io}.
Read the full abstract ▾
Robotics
Scale-Invariant Sampling in Multi-Arm Bandit Motion Planning for Object Extraction
Servet B. Bayraktar, Andreas Orthey and Marc Toussaint
Object extraction tasks often occur in disassembly problems, where bolts, screws, or pins have to be removed from tight, narrow spaces. In such problems, the distance to the environment is often on the millimeter scale. Sampling-based planners can solve such problems and provide completeness guarantees. However, sampling becomes a bottleneck, since almost all motions will result in collisions with the environment. To overcome this problem, we propose a novel scale-invariant sampling strategy which explores the configuration space using a grow-shrink search to find useful, high-
entropy sampling scales. Once a useful sampling scale has been found, our framework exploits this scale by using a principal components analysis (PCA) to find useful directions for object extraction. We embed this sampler into a multi-arm bandit rapidly-exploring random tree (MAB-RRT) planner and test it on eight challenging 3D object extraction scenarios, involving bolts, gears, rods, pins, and sockets. To evaluate our framework, we compare it with classical sampling strategies like uniform sampling, obstacle-based sampling, and narrow-passage sampling, and with modern strategies like mate vectors, physics-based planning, and disassembly breadth first search. Our experiments show that scale-invariant sampling improves success rate by one order of magnitude on 7 out of 8 scenarios. This demonstrates that scale-invariant sampling is an important concept for general purpose object extraction in disassembly tasks.
Read the full abstract ▾
Robotics
Neuromorphic Spiking Ring Attractor for Proprioceptive Joint-State Estimation
Federica Ferrari, Flavia Davidhi, Bernard Maacaron et al. (+6 more)
Maintaining stable internal representations of continuous variables is fundamental for effective robotic control. Continuous attractor networks provide a biologically inspired mechanism for encoding such variables, yet neuromorphic realizations have rarely addressed proprioceptive estimation under resource constraints. This work introduces a spiking ring-attractor network representing a
robot joint angle through self-sustaining population activity. Local excitation and broad inhibition support a stable activity bump, while velocity-modulated asymmetries drive its translation and boundary conditions confine motion within mechanical limits. The network reproduces smooth trajectory tracking and remains stable near joint limits, showing reduced drift and improved accuracy compared to unbounded models. Such compact hardware-compatible implementation preserves multi-second stability demonstrating a near-linear relationship between bump velocity and synaptic modulation.
Read the full abstract ▾
All authors Federica Ferrari, Flavia Davidhi, Bernard Maacaron, Alberto Motta, Luuk van Keeken, Elisa Donati, Giacomo Indiveri, Chiara De Luca, Chiara Bartolozzi
Robotics
Towards Multi-Object-Tracking with Radar on a Fast Moving Vehicle: On the Potential of Processing Radar in the Frequency Domain
Tim Hansen, Arturo Gomez-Chavez, Ilya Shimchik et al. (+1 more)
We promote in this paper the processing of radar data in the frequency domain to achieve higher robustness against noise and structural errors, especially in comparison to feature-based methods. This holds also for high dynamics in the scene, i.e., ego-motion of the vehicle with the sensor plus the presence of an unknown number of other moving objects. In addition to the high robustness, the processing in the frequency domain has the so far neglected advantage that the underlying correlation based methods used for, e.g., registration, provide information about all moving structures in the scene. A typical automotive application case is overtaking maneuvers, which in the context of autonomous racing are used here as a motivating example. Initial experiments and results with
Fourier SOFT in 2D (FS2D) are presented that use the Boreas
dataset to demonstrate radar-only-odometry, i.e., radar-odometry without sensor-fusion, to support our arguments.
Read the full abstract ▾
All authors Tim Hansen, Arturo Gomez-Chavez, Ilya Shimchik, Andreas Birk
Robotics
Goal2Skill: Long-Horizon Manipulation with Adaptive Planning and Reflection
Zhen Liu, Xinyu Ning, Zhe Hu et al. (+8 more)
Recent
vision-
language-action (VLA) systems have demonstrated strong capabilities in
embodied manipulation. However, most existing VLA policies rely on limited observation windows and end-to-end action prediction, which makes them brittle in long-horizon,
memory-dependent tasks with partial observability, occlusions, and multi-stage dependencies. Such tasks require not only precise visuomotor control, but also persistent
memory, adaptive task decomposition, and explicit recovery from execution failures. To address these limitations, we propose a dual-system framework for long-horizon
embodied manipulation. Our framework explicitly separates high-level semantic
reasoning from low-level motor execution. A high-level planner, implemented as a VLM-based agentic module, maintains structured task
memory and performs goal decomposition, outcome verification, and error-driven correction. A low-level executor, instantiated as a VLA-based visuomotor controller, carries out each sub-task through
diffusion-based action generation conditioned on geometry-preserving filtered observations. Together, the two systems form a closed loop between planning and execution, enabling
memory-aware
reasoning, adaptive replanning, and robust online recovery. Experiments on representative RMBench tasks show that the proposed framework substantially outperforms representative baselines, achieving a 32.4% average success rate compared with 9.8% for the strongest baseline. Ablation studies further confirm the importance of structured
memory and closed-loop recovery for long-horizon manipulation.
Read the full abstract ▾
All authors Zhen Liu, Xinyu Ning, Zhe Hu, Xinxin Xie, Weize Li, Zhipeng Tang, Chongyu Wang, Zejun Yang, Hanlin Wang, Yitong Liu, Zhongzhu Pu
Robotics
Beyond Conservative Automated Driving in Multi-Agent Scenarios via Coupled Model Predictive Control and Deep Reinforcement Learning
Saeed Rahmani, Gözde Körpe, Zhenlin Xu et al. (+3 more)
Automated driving at unsignalized intersections is challenging due to complex multi-vehicle interactions and the need to balance safety and efficiency. Model Predictive Control (MPC) offers structured constraint handling through optimization but relies on hand-crafted rules that often produce overly conservative behavior. Deep Reinforcement Learning (RL) learns adaptive behaviors from experience but often struggles with safety assurance and generalization to unseen environments. In this study, we present an integrated MPC-RL framework to improve
navigation performance in multi-
agent scenarios. Experiments show that MPC-RL outperforms standalone MPC and end-to-end RL across three traffic-density levels. Collectively, MPC-RL reduces the collision rate by 21% and improves the success rate by 6.5% compared to pure MPC. We further evaluate zero-shot transfer to a highway merging scenario without retraining. Both MPC-based methods transfer substantially better than end-to-end PPO, which highlights the role of the MPC backbone in cross-scenario robustness. The framework also shows faster loss stabilization than end-to-end RL during training, which indicates a reduced learning burden. These results suggest that the integrated approach can improve the balance between safety performance and efficiency in multi-
agent intersection scenarios, while the MPC component provides a strong foundation for generalization across driving environments. The implementation code is available open-source.
Read the full abstract ▾
All authors Saeed Rahmani, Gözde Körpe, Zhenlin Xu, Bruno Brito, Simeon Craig Calvert, Bart van Arem
Robotics
Mosaic: An Extensible Framework for Composing Rule-Based and Learned Motion Planners
Nick Le Large, Marlon Steiner, Lingguang Wang et al. (+4 more)
Safe and explainable motion planning remains a central challenge in autonomous driving. While rule-based planners offer predictable and explainable behavior, they often fail to grasp the complexity and uncertainty of real-world traffic. Conversely, learned planners exhibit strong adaptability but suffer from reduced transparency and occasional safety violations. We introduce Mosaic, a framework for structured decision-making that integrates both paradigms through arbitration graphs. By decoupling trajectory verification and selection from the generation of trajectories by individual planners, every decision becomes transparent and traceable. This separation lets verification and trajectory selection contribute independently: centralized verification acts as a safety floor, reducing at-fault collisions from 25 for each standalone planner to 16. In contrast, per-step trajectory selection acts as a performance ceiling, combining the complementary strengths of a rule-based and a learned planner. In experimental evaluation on nuPlan, Mosaic achieves 95.56 CLS-NR and 94.18 CLS-R on the Val14 closed-loop
benchmark, setting a new state of the art. On the interPlan
benchmark, focused on highly interactive and out-of-distribution scenarios, Mosaic scores 54.10 CLS-R, outperforming its best constituent planner by 22.8% – all without retraining or requiring additional data. The code is available at github.com/KIT-MRT/mosaic.
Read the full abstract ▾
All authors Nick Le Large, Marlon Steiner, Lingguang Wang, Willi Poh, Jan-Hendrik Pauls, Ömer Şahin Taş, Christoph Stiller
Robotics
EmbodiedClaw: Conversational Workflow Execution for Embodied AI Development
Xueyang Zhou, Yihan Sun, Xijie Gong et al. (+4 more)
Embodied AI research is increasingly moving beyond single-task, single-environment policy learning toward multi-task, multi-scene, and multi-model settings. This shift substantially increases the engineering overhead and development time required for stages such as evaluation environment construction, trajectory collection, model training, and evaluation. To address this challenge, we propose a new paradigm for
embodied AI development in which users express goals and constraints through conversation, and the system automatically plans and executes the development workflow. We instantiate this paradigm with EmbodiedClaw, a conversational
agent that turns high-frequency, high-cost
embodied research activities, including environment creation and revision,
benchmark transformation, trajectory synthesis, model evaluation, and asset expansion, into executable skills. Experiments on end-to-end workflow tasks, capability-specific evaluations, human researcher studies, and ablations show that EmbodiedClaw reduces manual engineering effort while improving executability, consistency, and reproducibility. These results suggest a shift from manual toolchains to conversationally executable workflows for
embodied AI development.
Read the full abstract ▾
All authors Xueyang Zhou, Yihan Sun, Xijie Gong, Guiyao Tie, Pan Zhou, Lichao Sun, Yongchao Chen
Robotics
Failure Identification in Imitation Learning Via Statistical and Semantic Filtering
Quentin Rolland, Fabrice Mayran de Chamisso and Jean-Baptiste Mouret
Imitation learning (IL) policies in robotics deliver strong performance in controlled settings but remain brittle in real-world deployments: rare events such as hardware faults, defective parts, unexpected human actions, or any state that lies outside the training distribution can lead to failed executions.
Vision-based Anomaly Detection (AD) methods emerged as an appropriate solution to detect these anomalous failure states but do not distinguish failures from benign deviations. We introduce FIDeL (Failure Identification in Demonstration Learning), a policy-independent failure detection module. Leveraging recent AD methods, FIDeL builds a compact representation of nominal demonstrations and aligns incoming observations via optimal transport matching to produce anomaly scores and heatmaps. Spatio-temporal thresholds are derived with an extension of conformal prediction, and a
Vision-
Language Model (VLM) performs semantic filtering to discriminate benign anomalies from genuine failures. We also introduce BotFails, a
multimodal dataset of real-world tasks for failure detection in robotics. FIDeL consistently outperforms state-of-the-art baselines, yielding +5.30% percent AUROC in anomaly detection and +17.38% percent failure-detection accuracy on BotFails compared to existing methods.
Read the full abstract ▾
Robotics · Cross-listed from eess.SY
A Dynamic-Growing Fuzzy-Neuro Controller, Application to a 3PSP Parallel Robot
Mohsen Jalaeian-Farimani, Mohammad-R Akbarzadeh-T, Alireza Akbarzadeh et al. (+1 more)
To date, various paradigms of soft-Computing have been used to solve many modern problems. Among them, a self organizing combination of fuzzy systems and neural networks can make a powerful decision making system. Here, a Dynamic Growing Fuzzy Neural Controller (DGFNC) is combined with an adaptive strategy and applied to a 3PSP parallel
robot position control problem. Specifically, the dynamic growing mechanism is considered in more detail. In contrast to other self-organizing methods, DGFNC adds new rules more conservatively; hence the
pruning mechanism is omitted. Instead, the adaptive strategy 'adapts' the control system to parameter variation. Furthermore, a sliding mode-based nonlinear controller ensures system stability. The resulting general control strategy aims to achieve faster response with less computation while maintaining overall stability. Finally, the 3PSP is chosen due to its complex dynamics and the utility of such approaches in modern industrial systems. Several simulations support the merits of the proposed DGFNC strategy as applied to the 3PSP
robot.
Read the full abstract ▾
All authors Mohsen Jalaeian-Farimani, Mohammad-R Akbarzadeh-T, Alireza Akbarzadeh, Mostafa Ghaemi
Robotics · Cross-listed from cs.LG
Vision-Language-Action Jump-Starting for Reinforcement Learning Robotic Agents
Angelo Moroncelli, Roberto Zanetti, Marco Maccarini et al. (+1 more)
Reinforcement learning (RL) enables high-frequency, closed-loop control for robotic manipulation, but scaling to long-horizon tasks with sparse or imperfect rewards remains difficult due to inefficient exploration and poor credit assignment.
Vision-
Language-Action (VLA) models leverage large-scale
multimodal pretraining to provide generalist, task-level
reasoning, but current limitations hinder their direct use in fast and precise manipulation. In this paper, we propose
Vision-
Language-Action Jump-Starting (VLAJS), a method that bridges sparse VLA guidance with on-policy RL to improve exploration and learning efficiency. VLAJS treats VLAs as transient sources of high-level action suggestions that bias early exploration and improve credit assignment, while preserving the high-frequency, state-based control of RL. Our approach augments Proximal Policy Optimization (PPO) with a directional action-consistency regularization that softly aligns the RL
agent's actions with VLA guidance during early training, without enforcing strict imitation, requiring demonstrations, or relying on continuous teacher queries. VLA guidance is applied sparsely and annealed over time, allowing the
agent to adapt online and ultimately surpass the guiding policy. We evaluate VLAJS on six challenging manipulation tasks: lifting, pick-and-place, peg reorientation, peg insertion, poking, and pushing in simulation, and validate a subset on a real Franka Panda
robot. VLAJS consistently outperforms PPO and
distillation-style baselines in sample efficiency, reducing required environment interactions by over 50% in several tasks. Real-world experiments demonstrate zero-shot sim-to-real transfer and robust execution under clutter, object variation, and external perturbations.
Read the full abstract ▾
All authors Angelo Moroncelli, Roberto Zanetti, Marco Maccarini, Loris Roveda
Robotics
Empirical Prediction of Pedestrian Comfort in Mobile Robot Pedestrian Encounters
Alireza Jafari, Hong-Son Nguyen and Yen-Chen Liu
Mobile
robots joining public spaces like sidewalks must care for pedestrian comfort. Many studies consider pedestrians' objective safety, for example, by developing collision avoidance
algorithms, but not enough studies take the pedestrian's subjective safety or comfort into consideration. Quantifying comfort is a major challenge that hinders mobile
robots from understanding and responding to human
emotions. We empirically look into the relationship between the mobile
robot-pedestrian interaction kinematics and subjective comfort. We perform one-on-one experimental trials, each involving a mobile
robot and a volunteer. Statistical analysis of pedestrians' reported comfort versus the kinematic variables shows moderate but significant correlations for most variables. Based on these empirical findings, we design three comfort
estimators/predictors derived from the minimum distance, the minimum projected time-to-collision, and a composite
estimator. The composite
estimator employs all studied kinematic variables and reaches the highest prediction rate and classifying performance among the predictors. The composite predictor has an odds ratio of 3.67. In simple terms, when it identifies a pedestrian as comfortable, it is almost 4 times more likely that the pedestrian is comfortable rather than uncomfortable. The study provides a comfort quantifier for incorporating pedestrian feelings into path planners for more socially compliant
robots.
Read the full abstract ▾
Robotics
Vision-and-Language Navigation for UAVs: Progress, Challenges, and a Research Roadmap
Hanxuan Chen, Jie Zheng, Siqi Yang et al. (+9 more)
Vision-and-
Language Navigation for Unmanned Aerial Vehicles (
UAV-VLN) represents a pivotal challenge in
embodied artificial intelligence, focused on enabling
UAVs to interpret high-level human commands and execute long-horizon tasks in complex 3D environments. This paper provides a comprehensive and structured survey of the field, from its formal task definition to the current state of the art. We establish a methodological taxonomy that charts the technological
evolution from early modular and deep learning approaches to contemporary agentic systems driven by large foundation models, including
Vision-
Language Models (VLMs),
Vision-
Language-Action (VLA) models, and the emerging integration of generative world models with VLA architectures for physically-grounded
reasoning. The survey systematically reviews the ecosystem of essential resources simulators,
datasets, and evaluation metrics that facilitates standardized research. Furthermore, we conduct a critical analysis of the primary challenges impeding real-world deployment: the simulation-to-reality gap, robust perception in dynamic outdoor settings,
reasoning with linguistic ambiguity, and the efficient deployment of large models on resource-constrained hardware. By synthesizing current
benchmarks and limitations, this survey concludes by proposing a forward-looking research roadmap to guide future inquiry into key frontiers such as multi-
agent swarm coordination and air-ground collaborative robotics.
Read the full abstract ▾
All authors Hanxuan Chen, Jie Zheng, Siqi Yang, Tianle Zeng, Siwei Feng, Songsheng Cheng, Ruilong Ren, Hanzhong Guo, Shuai Yuan, Xiangyue Wang, Kangli Wang, Ji Pei
Robotics
A Mechanistic Analysis of Sim-and-Real Co-Training in Generative Robot Policies
Yu Lei, Minghuan Liu, Abhiram Maddukuri et al. (+2 more)
Co-training, which combines limited in-domain real-world data with abundant surrogate data such as simulation or cross-embodiment
robot data, is widely used for training generative
robot policies. Despite its empirical success, the mechanisms that determine when and why co-training is effective remain poorly understood. We investigate the mechanism of sim-and-real co-training through theoretical analysis and empirical study, and identify two intrinsic effects governing performance. The first, “structured representation
alignment", reflects a balance between cross-domain representation
alignment and domain discernibility, and plays a primary role in downstream performance. The second, the “importance reweighting effect", arises from domain-dependent modulation of action weighting and operates at a secondary level. We validate these effects with controlled experiments on a toy model and extensive sim-and-sim and sim-and-real
robot manipulation experiments. Our analysis offers a unified interpretation of recent co-training techniques and motivates a simple method that consistently improves upon prior approaches. More broadly, our aim is to examine the inner workings of co-training and to facilitate research in this direction.
Read the full abstract ▾
All authors Yu Lei, Minghuan Liu, Abhiram Maddukuri, Zhenyu Jiang, Yuke Zhu
Robotics · Cross-listed from cs.CV
ESCAPE: Episodic Spatial Memory and Adaptive Execution Policy for Long-Horizon Mobile Manipulation
Jingjing Qian, Zeyuan He, Chen Shi et al. (+2 more)
Coordinating
navigation and manipulation with robust performance is essential for
embodied AI in complex indoor environments. However, as tasks extend over long horizons, existing methods often struggle due to catastrophic forgetting, spatial inconsistency, and rigid execution. To address these issues, we propose ESCAPE (Episodic Spatial
Memory Coupled with an Adaptive Policy for Execution), operating through a tightly coupled perception-grounding-execution workflow. For robust perception, ESCAPE features a Spatio-Temporal Fusion Mapping module to autoregressively construct a depth-free, persistent 3D spatial
memory, alongside a
Memory-Driven Target Grounding module for precise interaction mask generation. To achieve flexible action, our Adaptive Execution Policy dynamically orchestrates proactive global
navigation and reactive local manipulation to seize opportunistic targets. ESCAPE achieves state-of-the-art performance on the ALFRED
benchmark, reaching 65.09% and 60.79% success rates in test seen and unseen environments with step-by-step instructions. By reducing redundant exploration, our ESCAPE attains substantial improvements in path-length-weighted metrics and maintains robust performance (61.24% / 56.04%) even without detailed guidance for long-horizon tasks.
Read the full abstract ▾
All authors Jingjing Qian, Zeyuan He, Chen Shi, Lei Xiao, Li Jiang
Robotics
UNRIO: Uncertainty-Aware Velocity Learning for Radar-Inertial Odometry
Jui-Te Huang, Tianshu Huang, Anthony Rowe et al. (+1 more)
mmWave radars are robust to darkness and occlusions such as
dust and smoke, and can directly constrain ego-velocity from a single frame via Doppler measurements, making them attractive sensors for odometry in visually denied conditions. However, almost all existing radar-inertial odometry systems rely on lossy radar point clouds that are highly sparse, generally concentrated in a narrow angular band, and aliased at high speeds. We propose to instead estimate ego-velocity directly from unfiltered mmWave I/Q signals. Taking advantage of a foundation model for 4D radar spectrum, we develop a system that integrates uncertainty-aware velocity predictions with IMU measurements using an uncertainty-weighted sliding-window pose graph to accurately compute odometry even when provided radar data with aliasing or an unfavorable field of view. Evaluated on public
benchmark datasets, our system, UNRIO, attains the lowest relative pose error on the majority of sequences across held-out environments, despite differing chirp configurations, motion patterns, and platforms with its strongest gains coming where point clouds fail most: lateral motion in IQ1M and heavily aliased Doppler in ColoRadar.
Read the full abstract ▾
All authors Jui-Te Huang, Tianshu Huang, Anthony Rowe, Michael Kaess