Robotics
DistAL: Distance-based Advantage Learning for VLA Fine-Tuning
Reece O'Mahoney and Ioannis Havoutis
Vision-
language-action models (VLAs) have trans- formed the field of robotic manipulation in recent years by combining the semantic understanding of LLMs with the precise control of flow-matching policies. Advantage conditioning is a recent technique that iteratively improves VLAs by training a value function on deployment data and using this to train an advantage-conditioned policy. Previous works have only applied simple, low-information success/failure rewards, which leave the value function unable to distinguish states of differing quality beyond how far along the task they appear. Motivated by an exploration of out-of-distribution (OOD) detection methods, we introduce Distance-based Advantage Learning (DistAL), which, by using an
embedding space distance as a reward, produces a more informative value function and subsequently a higher downstream task success rate. We validate our method on a series of simulation
benchmarks and dexterous bi-manual manipulation tasks on real hardware.
Read the full abstract ▾
Robotics
Decoupling Vision, Language, and Action for Efficient Multi-Task Robot Policies
Xiatao Sun, Chen Liang, Ziyao Zeng et al. (+5 more)
Vision-
Language-Action (VLA) policies commonly run
Vision-
Language Model (VLM) backbones with billions of parameters at every policy inference, which costs latency and energy. We revisit a decoupled alternative for multi-task manipulation: separate
vision and
language encoders whose representations condition a compact action head. We run a standardized comparison that varies the
vision encoder, the
language encoder, and the action head while holding the demonstrations, the training-step budget, the tasks, the evaluation protocol, and the measurement platform fixed, against seven VLA baselines. The resulting Decoupled Embodiment Model (DEM) combines a fine-tuned DINOv3
vision encoder, a frozen NeoBERT
language encoder, and a MeanFlow head that generates an action chunk in one forward pass. On 18 RoboCasa tasks evaluated with held-out instruction paraphrases and randomized scenes, DEM reaches 55.6% mean success against 56.9% for GR00T N1.7 and 54.6% for $\pi_{0.5}$, and on three real-
robot tasks it reaches 66.0% against 68.0% for GR00T N1.7. On the same workstation, DEM needs 6.1\,ms per policy forward pass, a maximum throughput of 162.7 policy calls per second, and draws an estimated 2.07\,J of GPU energy per call, eight to seventeen times the throughput and six to fifteen times less energy than these VLM-backbone policies. Within this trained-task regime, DEM sits on the observed success–latency–energy frontier and provides a strong, efficient baseline for
language-conditioned
robot skills.
Read the full abstract ▾
All authors Xiatao Sun, Chen Liang, Ziyao Zeng, Qian Wang, Haoyang Zhang, Yue Sun, Qiucheng Li, Daniel Rakita
Robotics
RecMorph: Topology-Guided Spatial Recurrence for Generalized Morphology Control
Quanrui Rao, Yong Liu, Xueming Xiao et al. (+4 more)
Generalized morphology control requires a single policy to transform information across limbs with different physical roles, coordinate whole-body motion, and remain efficient as body size grows. Existing communication mechanisms address these requirements only partially. We introduce RecMorph, a
topology-guided spatial recurrent architecture that uses recurrent sequence computation to jointly perform cross-limb communication and representation transformation. A depth-first traversal converts the kinematic tree into a morphology-derived sequence, along which shared bidirectional transitions progressively transform limb information before action decoding. Residual preservation, RMS normalization, and input-dependent channel modulation stabilize this repeated spatial transformation, yielding linear
token complexity at fixed model width and depth. Across five UNIMAL tasks, RecMorph achieves the strongest mean final training performance among the evaluated generalized morphology controllers and the highest measured inference throughput on FT, while generalizing to unseen variations and bodies with up to 30 limbs. We further migrate representative generalized controllers from UNIMAL
benchmarks to a four-platform quadruped setting. RecMorph achieves the best macro-averaged performance under nominal and high friction, reduces nominal velocity RMSE by 43.5% relative to specialist MLPs, and one shared policy completes 40 physical Go1/Go2 trials without falls. These results show that
topology-guided recurrent transformation provides an effective and efficient communication mechanism for Generalized Morphology Control and remains effective when transferred from procedural bodies to physical
robot platforms. Code and experimental resources are publicly available at https://github.com/quanruirao/RecMorph.
Read the full abstract ▾
All authors Quanrui Rao, Yong Liu, Xueming Xiao, Yingbo Luo, Kun Wu, Zhenyu Xu, Meibao Yao
Robotics
GraphPoint: Semantic Entity Graphs and Point Trajectories for Compositional Robot Manipulation
Kang Luo and Hesheng Wang
Robot manipulation policies often struggle to generalize beyond their demonstrations, even when new instructions involve familiar objects and behaviors. When
language and scenes are strongly correlated during training, a policy can learn a fixed visual-action mapping rather than respond to the requested behavior. We investigate compositional reuse at two levels: within a subtask, combining familiar entities, action types, and action modifiers; and across subtasks, reusing learned subtasks in unseen long-horizon tasks. We introduce CoMani, a
benchmark with controlled splits for evaluating both capabilities. Matched initial scenes and controlled changes to a single semantic factor encourage reliance on
language rather than visual shortcuts. We further propose GraphPoint, which connects semantic entity graphs to
geometric control by predicting future gripper point trajectories and converting them into actions using
robot geometry. The framework organizes the gripper and objects by semantic roles and conditions their interactions on action types and modifiers, while predicted progress guides transitions during execution. Experiments and ablations on CoMani validate the effectiveness of our method for instruction-dependent generalization at both levels. Code will be released at GraphPoint.
Read the full abstract ▾
Robotics
UAVs Meet Embodied Intelligence: Bridging Human Intents and Flying Dynamics Via Harnessing Physical-Digital AI Agents
Yonglin Tian, Weiyi Wang, Houhua Lu et al. (+13 more)
Unmanned aerial vehicles (
UAVs) extend embodied intelligence into continuous three-dimensional space, where perception,
reasoning, physical embodiment, and action are tightly coupled through flight and environmental interaction. Recent advances in foundation models, world models, and AI
agents are shifting
UAV autonomy from task-specific perception and control toward systems that can interpret human intent, understand open environments, reason about physical consequences, and organize complex behaviors under embodiment and flight-dynamic constraints. We characterize this emerging paradigm as
UAV embodied intelligence (
UAV EI) and distinguish it from its system realization, the embodied-intelligent
UAV (EI
UAV). To provide a unified view of the field, we introduce a 5+5 framework that describes
UAV EI through five capability dimensions and EI
UAVs through five architectural layers spanning physical embodiment, general cognition, embodied skills, external interaction, and system harnessing. Based on this framework, we systematically review recent progress in embodied morphology, embodied perception, world models, embodied planning,
vision-
language navigation, embodied manipulation, and embodied collaboration. We further identify long-horizon autonomy, predictive physical
reasoning, test-time skill acquisition, and autonomous capability
evolution as key challenges toward more general aerial embodied intelligence. Finally, we argue that harnessing physical-digital AI
agents, through persistent coupling of digital intelligence with physical sensing, dynamics, action, and feedback, provides a system-level pathway toward adaptive and continuously evolving
UAV autonomy. Project resources are available at our project website and GitHub repository.
Read the full abstract ▾
All authors Yonglin Tian, Weiyi Wang, Houhua Lu, Xinyi Li, Yihao Wu, Jingyang Chen, Jianli Sun, Chengxiang Li, Yinuo Chen, Fei Lin, Tengchao Zhang, Jing Yang, Deyi Ji, Jian Di, Naiqi Wu, Yisheng Lv
Robotics
RAFAIL: Relationship-Aware Failure Detection for Robotic Manipulation
Loris Schneider, Edgar Welte and Rania Rayyes
Detecting failures during execution is essential for reliable robotic manipulation.
Vision-
language models (VLMs) can assess task outcomes semantically but add runtime computation, whereas out-of-distribution (OOD) detectors may respond to harmless scene variations rather than failure-relevant deviations. We introduce RAFAIL, a framework for detecting execution failures during robotic manipulation. RAFAIL identifies failures by detecting anomalies in task-relevant relationships between entities, such as a gripper and an object or an object and its target. By focusing OOD detection on relevant parts of the observation, RAFAIL reduces sensitivity to task-irrelevant scene variation. Offline, a VLM annotates successful demonstrations with task progress and relationship importance, which are used to learn point-cloud-based relationship representations without relying on policy-internal features. At runtime, relationship-specific OOD detectors evaluate these representations while relationship importance and task progress are predicted without VLM inference. RAFAIL requires no failure data and achieves 73.4% balanced accuracy across three real-world robotic manipulation tasks, outperforming the strongest evaluated OOD- and uncertainty-based baselines.
Read the full abstract ▾
Robotics · Cross-listed from cs.CL
Rollback the World, Keep the Reflection: Rollback-Induced Reflection for Long-Horizon LLM Agents
Yi Yu, Liuyi Yao, Yaliang Li et al. (+2 more)
Large
language model (LLM)
agents increasingly tackle long-horizon tasks through multi-step environment interaction, yet a single erroneous action can alter subsequent states and observations, causing errors to compound over time. Existing methods either correct the context without repairing altered environment states or restore earlier states while discarding useful experience, making it difficult to both eliminate failure conditions and avoid repeating past mistakes. We argue that reliable recovery should instead be treated as a rollback-boundary control problem that jointly determines when to intervene, where to resume, and what information should survive recovery. Based on this view, we propose Rollback-Induced Reflection (RIR), a unified recovery framework that restores execution to a selected prior state while carrying forward reusable knowledge distilled from the abandoned trajectory to guide subsequent decisions. We further characterize recovery through a unified operator over rollback depth and retained
memory, providing a general view of state restoration and knowledge retention. Experiments on three long-horizon
benchmarks show that RIR consistently improves average task performance across multiple LLM backbones, with structured reflection
memory preserving useful experience and selective rollback enabling efficient recovery.
Read the full abstract ▾
All authors Yi Yu, Liuyi Yao, Yaliang Li, Enshu Wang, Libing Wu
Robotics
Function-Preserving Data Generation for Zero-Shot Real-to-Sim-to-Real Manipulation
Tianyi Xiang, Xupeng Xie, Jiahang Cao et al. (+3 more)
Robotic data generation is a promising paradigm for scaling
robot learning without collecting large-scale real-world data. However, generating geometrically diverse yet physically valid data for contact-rich tasks remains challenging, especially when success depends on precise
geometric interfaces. Standard shape augmentation methods often distort task-critical interfaces, resulting in invalid contact relationships, e.g., fit mismatches or interpenetration, rendering downstream interactions infeasible. To address these limitations, we propose a function-preserving Real-to-Sim-to-Real framework that generates synthetic demonstrations from reconstructed assets without teleoperated source trajectories. Our method augments task-relevant object geometries through constraint-guided mesh deformation, together with physically consistent transfer of task poses and collision proxies. Visual domain randomization is further applied during simulation rollouts, enabling robust zero-shot policy deployment without real-world fine-tuning. Extensive experiments in both real-world and simulation settings demonstrate that our method enables robust generalization across unseen object geometries and diverse visual conditions in contact-rich and long-horizon tasks. Our method provides a practical path toward scalable
robot learning for contact-rich tasks via shape deformation.
Read the full abstract ▾
All authors Tianyi Xiang, Xupeng Xie, Jiahang Cao, Andrew F. Luo, Haoang Li, Jun Ma
Robotics · Cross-listed from eess.SY
Indicators of resilience for autonomous control systems
Jasper van Beers, Da-Hwi Kim, Prashant Solanki et al. (+1 more)
As modern societies rely more on autonomous systems to facilitate daily life, assuring their safe operation is paramount. Naturally, there are many techniques available to predict and prevent system failures. However, the safety afforded by such schemes may become misaligned with the true system, which can change in unexpected ways - from partial faults to natural wear-and-tear - that subtly degrade its stability. The implications that such subtle changes have on autonomous system stability can be observed through generic indicators of resilience derived from critical slowing down, popular for anticipating catastrophic tipping points in natural systems. Here, we show how one can systematically design these generic indicators for nonlinear control systems and show how these can reflect loss of stability though simulations of canonical robotic systems wherein their proximity to instability is manipulated directly. These results are affirmed through real-world flight experiments of a quadrotor that is nudged towards instability by progressively damaging its propeller blades. Our results show that the implications of degraded resilience on closed-loop stability are evident well before they appear, for which the indicators of resilience derived here can provide an early warning.
Read the full abstract ▾
All authors Jasper van Beers, Da-Hwi Kim, Prashant Solanki, Coen de Visser
Robotics
M2Tok: Multi-head Multi-codebook Discrete Action Tokenization for Vision-Language-Action Models
Chunpu Xu, Zhixuan Liang, Yuhao Zhang et al. (+6 more)
Recent advancements have successfully adapted autoregressive
language models to process
multimodal signals, such as images and actions. Since raw action signals are continuous, effective tokenization is essential to map high-dimensional inputs into compact discrete
tokens for autoregressive processing. However, existing discrete action tokenizers often suffer from high reconstruction loss, failing to preserve the fine-grained dynamics required for precise control. This "discretization bottleneck" significantly limits the performance ceiling of downstream
Vision-
Language-Action (VLA) models. To address this, we propose ${M}^2$Tok, a Multi-head Multi-codebook Action Tokenizer designed to minimize reconstruction error and enhance policy performance. Our approach introduces two key structural innovations: (1) we decompose the latent action features into multiple heads, enabling the model to implicitly align specific heads with distinct action dimensions; (2) we assign independent codebooks to each head for quantization. By leveraging the combinatorial nature of multiple codebooks, we significantly expand the representational expressivity of the tokenizer, leading to substantially lower reconstruction loss compared to previous methods. We evaluate the ${M}^2$Tok-based VLA on the RoboTwin, Simpler-Env, and 3 zero-shot real-world tasks. Experimental results demonstrate our method not only achieves superior reconstruction fidelity but also significantly boosts the success rate of VLA models. Comprehensive ablation studies further confirm the effectiveness of the multi-head and multi-codebook mechanisms. Code is available at https://github.com/cpaaax/M2Tok.
Read the full abstract ▾
All authors Chunpu Xu, Zhixuan Liang, Yuhao Zhang, Chi-Min Chan, Jessie Wang, Yang Xiao, Mengkang Hu, Xiaokang Yang, Yao Mu
Robotics
A3P5 NEMESIS Integrated Rover Design for Environmental Reconnaissance and Robotic Sampling with Reproducible Mobility Analysis and an External Data Machine Learning Calibration Benchmark
Shafi Bin Sultan, Sabik Bin Sultan and Safwan Sadad
A3P5 NEMESIS is a four-wheel rover intended to combine remote inspection, environmental observation and lightweight manipulation within one serviceable platform. This study develops a photo-constrained
geometric reconstruction, a subsystem architecture and a reproducible analytical assessment while distinguishing physical prototype evidence from proposed functions. An exploratory search retrieved 5,000 bibliographic records across ten queries, yielding 4,897 distinct DOI records and 1,212 metadata candidates; selected primary studies and technical documents informed the design. The reconstructed configuration retains the carbon-pattern enclosure, independently steered wheel assemblies, folded manipulator, inclined camera mast and side sampling equipment. A declared 24 kg scenario predicts 3.28 newton-metres of gearbox-output torque per wheel on a 20-degree grade under equal load sharing; a separate static model shows how a 2 kg forward payload reduces the
geometric front-tipping bound from 38.1 degrees to 32.7 degrees. These are design screens, not measured operating limits. A public-data calibration
benchmark uses 7,344 eligible hourly observations, eight sensor/environmental predictors and chronological training, validation and test partitions. Validation-selected ridge regression achieves a held-out CO root-mean-square error of 0.502 milligrams per cubic metre, with a 95% daily-block bootstrap interval of 0.435-0.569 milligrams per cubic metre. This result concerns an external sensor array and cannot establish NEMESIS accuracy. The combined analysis identifies priority measurements, proposed control interfaces and mission-specific validation requirements. The contribution is a traceable engineering design study and evaluation framework for a prototype whose integrated field performance remains to be established.
Read the full abstract ▾
Robotics
Acting in Meters: Learning Metric Interactions for Precise Robotic Manipulation
Lijie Wang, Zheng Lu, Yiming Wang et al. (+12 more)
Vision-
Language-Action models and World-Action Models have advanced
language-conditioned robotic manipulation, yet often leave metric relations among actions, objects, and scene geometry implicit. Human manipulation combines semantic understanding of task-relevant objects with spatial feedback that guides hand motion relative to objects and their surroundings. Inspired by this, we introduce a metric interaction framework that models object-level and scene-level interactions in physical Cartesian space at a shared metric scale. At the object level, Interaction-Centric
Tokens (ICTs) explicitly represent end-effector pose trajectories relative to manipulated objects and are jointly denoised with actions, providing physically grounded interaction supervision. At the scene level, the Metric Action Interaction Field (MAIF) uses action and ICT queries to attend to metric scene point-cloud features and learns geometry-conditioned action corrections. Through two-stage adaptation, our framework improves diverse VLA and WAM baselines with a small number of additional parameters and training steps. Experiments demonstrate average success-rate gains of 0.80 and 3.59 percentage points on LIBERO and RoboTwin 2.0, respectively, alongside gains of 6.80 percentage points on real-world tasks and 7.45 percentage points on their out-of-distribution variants.
Read the full abstract ▾
All authors Lijie Wang, Zheng Lu, Yiming Wang, Heyang Yu, Kenghou Hoi, Bowen Hu, Di Cui, Tianyu Xin, Haoran Liao, Wanqi Zhong, Xingjie Fan, Yizhao Xu, Ziliang Wang, Fei Gao, Yiming Li
Robotics
ForceDelta-VLA: Distilling Force-Conditioned ActionCorrections for Contact-Rich Manipulation
Ju Dong, Yu Fu, Jian Chen et al. (+9 more)
Force-aware
Vision-
Language-Action (VLA) policies improve contact-rich manipulation, but typically combine task-level motion and contact-dependent adjustment in a single action prediction. Demonstrations provide no explicit labels for decomposing that prediction into a reusable reference action and a correction. We present ForceDelta-VLA, a correction-
distillation framework that constructs an explicit force-correction target using paired predictions from a frozen teacher's force-conditioned and learned force-agnostic modes. A separate delay-correction target accounts for reference-action mismatch and the change in reference state. Training uses asynchronous schedule replay with the cached task context available during execution. The resulting lightweight policy adjusts the reference actions using recent force history and
robot state, responding to contact changes between reference-action updates without regenerating complete action chunks. Across nine single-arm and bimanual contact-rich tasks, ForceDelta-VLA achieves an 82.2% mean success rate, compared with 54.4% for the original ForceVLA baseline. Direct execution of our Stage-1 Temporal Teacher achieves 70.6%. Relative to ForceVLA, the complete system reduces mean peak contact force over successful trials by approximately 26% on both platforms.
Read the full abstract ▾
All authors Ju Dong, Yu Fu, Jian Chen, Yimeng Liu, Haocheng Zhao, Lei Zhang, Kaixin Bai, Liding Zhang, Diwen Zheng, Alois Christian Knoll, Angela P. Schoellig, Jianwei Zhang
Robotics
UMI-Bridge: Action-Anchored Latent Alignment across Human and Robot Manipulation Data
Haiyi Liu, Jinming Ma, Ke Rui et al. (+12 more)
Real-
robot demonstrations are limited, motivating the use of human manipulation data collected without
robots, including egocentric
videos and handheld Universal Manipulation Interface (UMI) demonstrations. However, differences in viewpoint, embodiment, and available action supervision make it difficult to align representations across these sources according to manipulation motion rather than visual appearance. We introduce UMI-Bridge, which uses UMI as an intermediate domain to align representations according to action equivalence rather than pixel similarity. UMI action supervision anchors the latent representation to end-effector motion and gripper behavior, while synchronized head-wrist observations and paired ego-UMI clips support
alignment across views and domains. We train a dual-view latent action model (LAM) on human manipulation data without
robot demonstrations, then freeze its wrist teacher and dynamics model to regularize
vision-
language-action (VLA) post-training on UMI and
robot data. The shared wrist interface enables this training-time supervision across both domains while preserving the policy's standard inference architecture. Across three real-
robot tasks, UMI-Bridge achieves 91.7% mean success versus 73.3% for Naive Co-training with matched UMI and
robot data. On two data-efficiency tasks, it surpasses a full-data
Robot-only baseline using 25% of the
robot demonstrations together with UMI data. It also achieves 85% and 90% success on two additional tasks learned from UMI demonstrations without task-specific
robot demonstrations. These results support action-anchored latent
alignment for data-efficient
robot learning and UMI-to-
robot task transfer.
Read the full abstract ▾
All authors Haiyi Liu, Jinming Ma, Ke Rui, Yuteng Wei, Yuan Ma, Yushen Zuo, Honglong Tian, Haoran Jia, Weitao Zhou, Jiawei Wang, Shiyi Chen, Haiyan Mao, Jiaqi Zhang, Chun Zhang, Minglei Li
Robotics
CANTABILE: Learning Expressive Dynamics for Robotic Piano Performance
Woosik Kim, Wonhyeok Choi and Sunghoon Im
Robotic piano playing has emerged as a standard
benchmark for dexterous bimanual manipulation, yet progress on it has been measured almost entirely by note accuracy – which keys are pressed (pitch) and when (onset) – leaving the musical dynamics essential for expressive performance neither rewarded nor evaluated. We propose CANTABILE, a dynamics-aware framework for robotic piano performance that (i) closes the score-to-contact loop by conditioning the policy on upcoming velocity goals and mapping each key's angular velocity at onset back to MIDI velocity, (ii) couples a velocity-fidelity reward with an onset-coverage reward, so that dynamics cannot be improved by omitting difficult notes, and (iii) refines a frozen dynamics-aware base policy with an alpha-scaled, finger-only residual that localizes strike-intensity adaptation away from nominal note execution. On EXPRESSIVE-51, a dynamics-rich 51-song subset of RoboPianist, CANTABILE raises Velocity F1 – jointly measuring pitch, onset, and intensity within a +/-8 MIDI-velocity tolerance – from 0.06 to 0.34 over the RoboPianist baseline, improves all 51 songs, more than halves matched-note velocity error, and reduces log-mel distance to reference audio by 8%. Intensity-randomized training further enables runtime control of performance intensity without retraining.
Read the full abstract ▾
Robotics
Reinforcement Learning for Real-Time Vision-Language-Action Policies
Perry Dong, Kuo-Han Hung, Dorsa Sadigh et al. (+1 more)
Reinforcement learning fine-tuning on top of large, pretrained
Vision-
Language-Action (VLA) models offers promise for highly reliable
robot deployment. However, because of their scale, modern VLA models suffer from high inference latency, so the observation used to select an action is often stale by execution time, creating a distribution shift that can substantially degrade reliability and performance. Prior work has explored asynchronous policy execution to reduce the effect of latency, but these methods are mostly built on imitation learning and offer no mechanism for moving beyond the training distribution toward higher reliability. We close this gap by enabling RL fine-tuning that meets the real-time control requirements of dynamic real-world manipulation. Our approach builds on EXPO-FT, a framework for sample-efficient, reliable VLA fine-tuning with reinforcement learning, and decouples slow, expressive action generation from fast, reactive action edits: a large pretrained VLA proposes action chunks using its strong behavior prior, while a lightweight edit policy performs fast, reactive decision-making by editing actions in response to changes in state, conditioned on the latest observation. We instantiate this as Real-Time EXPO-FT, an RL framework for finetuning real-time VLA policies. On the Kinetix
benchmark, Real-Time EXPO-FT enables a delayed policy to achieve the best performance among delayed and non-delayed methods in 10 out of 10 environments. On four dynamic real-world tasks,
robot object passing, ball balancing, table soccer kicking, and dynamic object picking, with online
robot data capped at 10 minutes, Real-Time EXPO-FT improves average policy performance from 42% to 97%, all without human intervention, demonstrating rapid, sample-efficient adaptation to challenging real-world dynamics. Website: https://pd-perry.github.io/real-time-expo-ft
Read the full abstract ▾
All authors Perry Dong, Kuo-Han Hung, Dorsa Sadigh, Chelsea Finn
Robotics
WholeBodyWAM: Learning Whole-Body World Action Models with Scalable Motion Priors
Bowei Zhang, Qiyao Zhang, Shuanghao Bai et al. (+8 more)
Humanoid whole-body manipulation requires coordinated whole-body dynamics, yet large-scale trajectories from a target
robot are expensive to collect and difficult to scale. In contrast, whole-body motion from human and humanoid sources is abundantly available, although such data cannot be directly used as embodiment-specific
robot actions. This work asks whether these scalable motion resources can instead provide a transferable predictive prior for humanoid world-action modeling. We introduce WholeBodyWAM, a humanoid world-action model that learns whole-body dynamics from large-scale heterogeneous motion before target-
robot training. We curate UniMotion-4K, a motion corpus spanning more than 4K hours from human
videos, native 3D motion
datasets, and heterogeneous humanoid platforms, and canonicalize these diverse sources into a unified motion space. A
language-conditioned Motion Expert is then pretrained to predict future whole-body motion without target-
robot action supervision. During
robot post-training, the pretrained Motion Expert is integrated with
Video and Action Experts through asymmetric Mixture-of-
Transformers (MoT)
attention, enabling predictive scene dynamics and whole-body motion to jointly inform embodiment-specific action generation. Experiments show that WholeBodyWAM consistently benefits from increased motion-pretraining scale, improves future-motion prediction and downstream task performance, and transfers effectively to real-world humanoid manipulation. Moreover, the pretrained motion prior substantially improves data efficiency under limited target-
robot demonstrations.
Read the full abstract ▾
All authors Bowei Zhang, Qiyao Zhang, Shuanghao Bai, Xinhua Wang, Meng Li, Yilei Wang, Leiwang Zhang, Jian Tang, Lu Zhou, Lei Sun, Zhengping Che
Robotics
WAVE-Go: World-Model Navigation with Adaptive Execution for Wheel-Legged Robots
Mingyi Li, Ji Li, Zhihao Ouyang et al. (+2 more)
World models can anticipate the consequences of navigation actions, but predicted action sequences may become invalid during execution, especially when wheel-legged
robots encounter dynamic obstacles or change locomotion modes. We propose WAVE-Go, an image-goal navigation framework that separates world-action prediction from interruptible command execution. Its executor adaptively selects an action prefix and cancels pending commands when updated observations invalidate execution. A conditional-risk formulation specifies prefix selection under an estimated cumulative failure budget, while posture and locomotion-mode transitions require clearance, stability, and task-evidence checks. In the reported navigation evaluation, WAVE-Go achieves 74.1% in-distribution success and 63.3% dynamic out-of-distribution success, exceeding the strongest baseline by 4.7 and 7.7 percentage points, respectively, while reducing collisions from 4.4 to 2.9 per 100 m. Compared with interruptible fixed four-command execution, WAVE-Go raises success by 4.0 percentage points while reducing replanning frequency by 51.2% and collision rate by 6.5%. Execution ablations also show that runtime interruption improves success, collision rate, and reaction latency at the cost of additional replanning. These results support adaptive, interruptible execution as a means of balancing navigation performance and planning overhead. Code is available at https://github.com/vigorlee/wave-go.
Read the full abstract ▾
All authors Mingyi Li, Ji Li, Zhihao Ouyang, Yage He, Börje F. Karlsson
Robotics
OmniRisk: Omnidirectional Trajectory-Risk Learning for Agile Quadrotor Dynamic Avoidance
Yifan He, Yang Liu, Wenhao Zhao et al. (+8 more)
Agile quadrotor avoidance of fast-moving obstacles requires anticipating collisions and selecting feasible maneuvers within short reaction windows. Reliable predictive avoidance remains challenging because sparse range observations do not directly reveal obstacle motion, while online trajectory optimizers either scale poorly with obstacle count or remain efficient at the expense of reliability in dense, high-speed encounters. We present OmniRisk, an omnidirectional planning framework that learns trajectory-level risk offline for efficient onboard evasion. A fixed-dimensional tensor combines
LiDAR range panoramas, dynamic masks, and Cartesian surface velocities to represent geometry and motion jointly. We formulate an asymmetric risk field aligned with obstacle velocity that emphasizes approaching interactions and attenuates receding ones. Accumulating this risk along predicted relative trajectories provides dense supervision and discourages unnecessary hesitation after obstacles pass. A dual-branch circular convolutional network predicts terminal boundary states and dynamic risks for candidate primitives over an omnidirectional anchor
lattice in a single forward pass, followed by selection and closed-form reconstruction of the selected candidate primitive. This formulation removes online risk accumulation along trajectories and makes risk-inference cost independent of obstacle count. OmniRisk enables efficient onboard avoidance, with real-world flights demonstrating consecutive evasive maneuvers at relative encounter speeds up to 15 m/s without fine-tuning. Code is available at https://github.com/VANdexj/OmniRisk.
Read the full abstract ▾
All authors Yifan He, Yang Liu, Wenhao Zhao, Hai Lin, Deping Zhang, Mingze Ma, Xin Zhou, Fei Gao, Huan Yu, Zipeng Dai, Ziming Ding
Robotics
TacBPM: A Tactile-conditioned Behavior Prior Model for Dexterous Reorientation
Jie Yin, Wanli Xing, Zeyuan Zhao et al. (+3 more)
Dexterous in-hand manipulation requires policies that coordinate high-DoF hand joints through intermittent, contact-rich interaction. Beyond target-orientation tracking, such policies must discover finger gaits that preserve object stability while adapting to geometry, anisotropy, pose, contact, and sensing changes. We propose \method, a tactile-conditioned behavior prior model for dexterous reorientation. \method distills multi-scale sphere specialists into a latent controller and lets downstream policies reuse the fixed tactile prior through residual latent actions, reducing renewed exploration from raw joint commands. The prior conditions on tactile-proprioceptive history so latent behavior reflects the current hand-object interaction. We evaluate arbitrary-pose transfer across anisotropic objects, commanded-axis rotation, and an arm-hand Grasp-to-AnyPose task in which the
robot must grasp, lift, transport, and reach goal poses for novel tool geometries and generalized placements. Extensive experiments demonstrate that the proposed method accelerates training and enables stable policies where matched raw-action PPO remains near failure, with successful sim-to-real transfer in in-hand and arm-hand tasks.
Read the full abstract ▾
All authors Jie Yin, Wanli Xing, Zeyuan Zhao, Xuezhou Zhu, Zhijie Deng, Kaifeng Zhang
Robotics
Approximating High Dimensional Self-Motion Manifolds via Deep Generative Models
Haitao Gao, Yang Song and Liao Wu
Self-motion
manifold (SMM) characterizes the
geometric structure of the infinite inverse kinematic solutions set of a redundant manipulator at a fixed end-effector pose, and its efficient recovery underpins feasible and global optimal motion planning. Existing methods such as null-space continuation and learning-based methods are formulated around the assumption that an SMM is a curve, and do not extend to higher redundancy orders. We instead adopt a probabilistic view: SMMs are the support of the conditional posterior over configurations given a target pose, so that recovering it reduces to sampling from a learned distribution and separating its disjoint components by clustering. The formulation is independent of the
manifold dimension and requires no architectural change as the redundancy order grows. In this work, we demonstrate that our method can approximate 1-D SMMs with performance comparable to the latest null-space continuation and learning-based approach, and that it is the first method capable of approximating highly redundant 4-D SMMs in a 7R manipulator for position tasks. Project website: \href{https://github.com/accuracy-maker/high-dimenstional-self-motion-
manifold-approximation}{https://github.com/accuracy-maker/high-dimenstional-self-motion-
manifold-approximation}
Read the full abstract ▾
Robotics
Characterizing Replay Retention Under Dynamics Shift in Model-Based Reinforcement Learning
Everest Yang, Skye Thompson and George D. Konidaris
Adapting to changes in
robot dynamics requires learning from new data without discarding experience that may still be useful. In continual model-based reinforcement learning (RL), replay collected before a dynamics change can slow adaptation, while removing it unnecessarily reduces available training data and can be especially costly if earlier dynamics return. We study when recent transitions are preferable to the full replay history. Two quantities characterize this trade-off: change magnitude and age-staleness area under the curve (AUC), measuring how well transition age separates stale from fresh data. Forgetting stale data helps after large permanent shifts but hurts when dynamics recur and older data becomes useful again. Choosing a replay strategy therefore depends on predicting when older data will help or hurt. We test these effects across two locomotion morphologies, two model-based RL
algorithms, and Real-World RL
benchmark perturbations. Because ground-truth staleness labels are unavailable on deployed
robots, we evaluate whether an
estimator built from interaction data can still provide the quantities needed to choose a replay strategy after permanent changes. Our results show that replay retention depends on change magnitude and on how the dynamics evolve.
Read the full abstract ▾
Robotics
LUMO: Designing Luminous Contact Morphology for Repeatable Whole-Finger Contact Observation
Dong Ho Kang, Youngsu Ko and Luis Sentis
A low-impedance
robot finger reports through joint torque how strongly it is loaded, but the same torque can arise from a small force near the fingertip or a large force near the joint. Resolving the force therefore requires knowing where along the finger contact occurred. LUMO makes that location externally observable. Embedded LEDs illuminate a compliant silicone pad, and contact deforms the pad so that light emerging from the finger's side changes in a pattern set by where the load acts. Because the same structure also carries the contact load, we optimize its cross-section, including the pad profile, rigid carrier, and lateral void, for two behaviors at once. Mechanically, the pad conforms under low preload while the carrier increasingly restricts further deformation as load rises. Optically, different contact locations produce separated responses on the finger's side. The search uses rigid–soft contact simulation, ray tracing, and multi-objective
Bayesian optimization. Across two silicones, six contact locations, and 10- and 30-mm spherical indenters, the optimized morphologies improve neighboring-location separation relative to variation from re-establishing contact by \(15\)–\(59\%\). Estimating contact location from the optical response using the known LED spacing and combining it with joint torque gives \(1.44~\mathrm{N}\) normal-force MAE over 931 samples. In a two-finger hand, localized side responses appear on several links simultaneously during grasps.
Read the full abstract ▾
Robotics
Energy-Regularized Imitation Learning for Force- and Work-Aware Robotic Manipulation
Toshiki Otani, Hiromu Taketsugu and Norimichi Ukita
This paper studies energy-aware manipulation as a physically grounded learning problem. We define a joint-space mechanical-work proxy from joint torque and angular displacement, and train a differentiable energy predictor that estimates this work from
robot states and actions. The predictor converts a non-differentiable simulator-side physical quantity into a differentiable regularizer for fine-tuning a pretrained manipulation policy. We instantiate the framework with RVT-2 on RLBench and evaluate 12 manipulation tasks involving object contact, articulated motion, placement, pushing, and sweeping. The proposed fine-tuning reduces the average mechanical work from 208.8J to 204.4J (i.e., 2.1% reduction), while the mean task success rate also increases slightly from 86.2% to 86.9%. These results show that work-aware policy optimization can suppress physically inefficient motion without requiring an explicit differentiable dynamics model.
Read the full abstract ▾
Robotics
Weiliang Deng, Zhengyang Dang, Yao Mu et al. (+1 more)
Aerial grasping is a remarkable capability exhibited by predatory birds, allowing them to capture prey through highly coordinated maneuvers in flight. Inspired by this capability, researchers have developed various formulations to reproduce such maneuvers through trajectory optimization. However, two limitations remain in practice. First, the resulting optimization problem is highly nonconvex and sensitive to initialization, making high-quality solutions difficult to obtain under a limited computational budget. Second, prescribed numerical objectives are human-designed abstractions that describe successful grasping through a limited set of mathematically tractable quantities and may not fully capture what determines task success. We investigate how learning can address these limitations within an analytical planner. Accordingly, a trajectory prior is first learned from optimized motions and then evolved through a CEM-based process that evaluates sampled initializations with the deployed optimizer and retains favorable ones as new supervision. An Execution-Aware Critic learns from contact, lift, and completion outcomes to assess whether the optimized trajectories are likely to succeed in physical execution. Its frozen energy can further serve as a differentiable grasping cost, allowing execution data to directly shape trajectory generation. Simulation and real-world experiments demonstrate improved optimization reliability, trajectory consistency, and grasping performance.
Read the full abstract ▾
All authors Weiliang Deng, Zhengyang Dang, Yao Mu, Ximin Lyu
Robotics · Cross-listed from cs.CV
PRISM: Predictive Representation of Interaction Style and Motion for Social Robot Navigation
Bo-Han Chen, Hiromu Taketsugu and Norimichi Ukita
Humans often observe others before interacting and adjust their behavior accordingly.
Robot navigation in crowds, however, often represents pedestrians mainly by observed
geometric states, leaving individual differences in interaction tendencies implicit. We propose PRISM (Predictive Representation of Interaction Style and Motion), a framework that infers interaction traits from passive observations of human-human interactions. PRISM encodes human trajectories into a continuous ordinal latent space with a
transformer encoder trained by Rank-N-Contrast loss, and pairs each inferred trait with a temporal-stability score supplied to the navigation policy. In randomized crowd simulations, PRISM reduces collision rates over the geometry-only baseline and yields small improvements in navigation-time and path-length metrics. These results suggest the utility of passive latent-trait inference for social navigation in dynamic crowds.
Read the full abstract ▾
Robotics · Cross-listed from cs.CR
"Your Robot Was Trained on a Lie": Collision Mesh Poisoning Attacks on Robotic Manipulation
Gengyang Xu, Dongwei Xiao, Yiteng Peng et al. (+6 more)
Learning-enabled robotic manipulation increasingly relies on
robot simulators for policy training and evaluation before real-world deployment. Inside a simulator, a 3D asset contains two separate geometries: a visual mesh used for rendering and a collision mesh used for physical interaction. For computational efficiency, the collision mesh is deliberately a coarse approximation that need not have the same geometry as the visual mesh, a legitimate and pervasive discrepancy we call the Visual–Collision Gap (V–C Gap). We show that the V–C Gap opens a new and practical attack surface, and propose Collision Mesh Poisoning (CMP), the first poisoning attack against robotic manipulation delivered through the 3D asset supply chain. An attacker modifies only the collision mesh of a 3D asset, leaving the visual mesh and all other components unchanged. A policy trained and evaluated with the poisoned asset behaves normally throughout simulation, yet degrades, fails, or creates physical safety risks once deployed in the real world. Since current asset review practices cover malware, copyright, and format compliance, but not visual–collision consistency, poisoned assets can be distributed through legitimate supply chain channels. We evaluate several defenses and our results show that they are insufficient to defend against CMP, highlighting the need for new defenses.
Read the full abstract ▾
All authors Gengyang Xu, Dongwei Xiao, Yiteng Peng, Yanbo Dai, Ruochen Zhou, Shing-Chi Cheung, Xiaoyu Ji, Wenyuan Xu, Shuai Wang
Robotics
Fetch My Beer: Synthetic-to-real Hierarchical Policy for Smooth Pick-and-place
Yingyue Li, Chenyangguang Zhang, Ruida Zhang et al. (+3 more)
Many real-world robotic applications require dynamically sensitive manipulation, where success depends not only on reaching a target state but on maintaining stable object dynamics throughout execution. We study the stable transport of
liquid-filled containers, where a
robot must move objects to target locations while suppressing sloshing and preventing spillage. Unlike conventional pick-and-place, this task imposes stringent requirements on motion smoothness and trajectory-level stability, exposing clear limitations in existing systems. Specifically, fluid simulation remains too costly for online reinforcement learning; human teleoperation introduces unintended accelerations that induce sloshing during imitation learning; and current policy pipelines optimize for task completion rather than dynamic stability. We propose a synthetic-to-real framework coupling physically validated data generation with a hierarchical,
diffusion-based controller. The scalable data pipeline synthesizes grasps, filters unstable poses via a
vision-
language model, and validates transport trajectories through fluid simulation. The policy is organized with a high-level module that translates
language and visual observations into SE(3) control targets, and a latent
diffusion controller that first plans efficiently in a compact latent space and then decodes dense action chunks, enabling the high control frequency needed for smooth and stable motion. Extensive experiments show our system outperforms state-of-the-art manipulation policies in transport smoothness and dynamic stability. Our project page: https://fetch-my-beer.github.io/
Read the full abstract ▾
All authors Yingyue Li, Chenyangguang Zhang, Ruida Zhang, Bowen Fu, Guangyao Zhai, Xiangyang Ji
Robotics
OpenDexGrasp: Open-vocabulary Task-Oriented Dexterous Grasping
Jiyao Zhang, Junhan Wang, Tianyu Wang et al. (+4 more)
Dexterous grasp synthesis has advanced rapidly in generating stable and physically plausible hand poses, but real-world manipulation requires grasps that preserve the function implied by the task. We study open-vocabulary task-oriented dexterous grasp generation, where a
robot must infer functional intent from free-form
language, ground it in multi-view visual observations and object geometry, and generate an executable high-degree-of-freedom grasp. We present OpenDexGrasp, a unified data and generative modeling framework for this setting. OpenDexVerse provides dual-source supervision organized by the Coverage-to-
Alignment (C2A) Recipe: OpenDex-Scale offers large-scale semantic and
geometric coverage through automatic grasp synthesis and
vision-
language annotation, while OpenDex-Align supplies high-quality embodied
alignment through human teleoperation and category-level transfer. OpenDexGrasp learns a shared perception-action latent representation that couples open-vocabulary
vision-
language context with dexterous action generation. Affordance grounding and grasp generation provide complementary supervision over this latent space, enabling direct generation of task-consistent dexterous grasps without a separate affordance-to-pose inference stage. Extensive simulation and real-
robot experiments demonstrate improved functional
alignment, physical feasibility, generalization to unseen categories, and real-world execution success. Additional details and
videos are available at https://opendexgrasp.github.io/.
Read the full abstract ▾
All authors Jiyao Zhang, Junhan Wang, Tianyu Wang, Zeyuan Chen, Anthony Bolton, Yitong Peng, Hao Dong
Robotics
A Comprehensive Review of Generative Physical Artificial Intelligence
Satyam Gaba, Krutiksinh Rana, Siva Sai et al. (+2 more)
The integration of large-scale foundation models with physical embodiments has led to significant advancements in robotics known as Generative Physical Artificial Intelligence (GPAI). These agentic AI systems autonomously perceive, reason, and act in complex real-world situations. This survey comprehensively analyzes GPAI systems, focusing on their architectural foundations, current applications, and key limitations. We introduce a taxonomy of five distinct approaches:
Robot Foundation Models (RFMs) for cross-platform skill transfer;
Vision-
Language Action (VLA) models for end-to-end multi-modal perception and control; Large Behavior Models (LBMs) for human-like movement generation;
Diffusion Policy Models (DPMs) for
diffusion model-based temporally coherent action generation; and World Foundation Models (WFMs) for physics-compliant simulation and data generation. We examine how these approaches complement each other: WFMs generate training data for VLAs and DPMs, RFMs enable cross-platform deployment of learned policies, while LBMs provide motion priors for natural behavior. Through examples across autonomous vehicles, industrial automation, healthcare robotics, and humanoid systems, we identify significant performance improvements and summarize promising research directions in data-efficient learning, sim-to-real transfer, edge-compatible architectures, and safety frameworks. These insights advance embodied AI for IoT-connected environments where intelligent
agents interact with networked sensors, actuators, and edge devices.
Read the full abstract ▾
All authors Satyam Gaba, Krutiksinh Rana, Siva Sai, Vinay Chamola, Dusit Niyato