Robotics
Learning Dynamic Grasping
Nga Kiu Kelly Ho, Yilin Liu, Hao Xu et al. (+1 more)
Synthesizing believable 4D hand-object grasping sequences is important for character animation, extended reality, and embodied
agents. Grasping requires a hand to approach, contact, and lift or manipulate an object coherently. Existing synthesis methods often model only the final static grasp or rely on additional conditions, such as
language, target grasps, or object trajectories, to specify the interaction. We instead study the underconstrained problem of generating a complete grasping motion from a single initial hand-object state. Our framework decouples hand-motion generation from object-motion recovery. Given an object point cloud in the initial wrist frame and the initial parameterized hand configuration, conditional flow matching predicts a sequence-level contact map and a hand pose at each normalized sequence time. This time-indexed formulation avoids the error accumulation of autoregressive rollout while representing the complete motion. We then infer an attachment frame and recover rigid-object motion through training-free, contact-guided weighted registration. We evaluate on held-out DexYCB objects and unseen object geometries reconstructed from in-the-wild
videos by Open-CHOIR. Compared with LatentHOI, our method produces smoother hand and object motion in every setting, matches or improves on its contact and penetration metrics in the wild under a shared initialization, and is about 30x faster per generated frame. In a 15-participant study, our motions were preferred in roughly two thirds of (forced-choice pairwise) comparisons. Precise contact closure on novel geometry, however, remains a key open challenge.
Read the full abstract ▾
All authors Nga Kiu Kelly Ho, Yilin Liu, Hao Xu, Niloy J. Mitra
Robotics
Push the Limit: Learning High-Speed Sprinting and Aggressive Cornering on a Full-Scale Humanoid
Huaxing Huang, Ruidong Li, Jinchao Han et al. (+2 more)
We present a learning-based framework that enables a full-scale bipedal humanoid to sprint at over 7 m/s with motion-prior-guided gait and perform aggressive cornering at high speed with a pronounced trunk lean of up to $25^\circ$. Achieving this combination is challenging: above approximately 4 m/s, periodic foot impacts induce severe, high-frequency yaw-rate oscillations intrinsic to high-speed gaits. This high-frequency temporal structure corrupts the history of proprioceptive feedback that flat-MLP policies rely on, while rendering conventional yaw-rate tracking rewards counterproductive, as they fight against the
robot's natural oscillatory dynamics. Moreover, a single style prior shared across skills cannot tell which skill is commanded and inherently compromises the low-speed skill fidelity. To overcome these limitations, we adopt a causal multi-head
attention (MHA) module to explicitly model the high-frequency temporal feedback. In place of the oscillatory yaw rate, we design a heading-direction reward that supervises only the low-frequency velocity direction, allowing agile turning at high speed without suppressing natural gait oscillations. We further introduce skill-conditioned multi-
adversarial-motion-prior (MAMP) discriminators that condition the style reward on the commanded skill. Ablation experiments confirm that each mechanism is essential: replacing structured temporal encoding with a flat-history MLP caps the top speed; reverting to yaw-rate tracking prevents high-speed turning; and a single shared discriminator makes in-place turning collapse into a quasi-static stance. Our framework bridges the gap between raw sprinting speed and reference-motion fidelity, demonstrating stable high-speed sprinting and aggressive cornering in both simulation and real-
robot deployment.
Read the full abstract ▾
All authors Huaxing Huang, Ruidong Li, Jinchao Han, Ziyang Dong, Wenhao Cui
Robotics
Scaling Latent Motor Adaptation to Coordinated Bimanual Manipulation
Seung-Bin Joo, Alessandro Albini, Luca Wiehe et al. (+3 more)
Robust manipulation under uncertain object dynamics and geometry remains a core challenge in real-world robotics. Policies trained under fixed assumptions often degrade when object mass, friction, scale, or shape vary.
Embedding-based adaptation methods such as Rapid Motor Adaptation (RMA) address this by conditioning actions on latent representations of environment dynamics inferred from interaction history. We investigate whether this paradigm scales from single arm settings to coordinated bimanual manipulation, where higher dimensional control, coupled contacts, and shared object forces introduce additional complexity. We evaluate bimanual RMA on three contact-rich ManiSkill tasks with randomized YCB object properties. Bimanual RMA, using observation-action history and dual wrist depth images, retains privileged policy performance on bimanual picking and lifting, while an otherwise matched stationary PPO policy without latent conditioning achieves low success; all methods remain limited on long-horizon stacking. We further demonstrate direct sim-to-real transfer on a dual Franka lifting task without real-world finetuning. Together, these results suggest that latent conditioning provides useful structure for scaling robust control to coordinated bimanual systems, while longer horizon tasks require additional temporal structure.
Read the full abstract ▾
All authors Seung-Bin Joo, Alessandro Albini, Luca Wiehe, Perla Maiolino, Daniel Olmeda Reino, João F. Henriques
Robotics · Cross-listed from cs.CV
YOUDrive: Driving Style as a Composable Direction for End-to-End Autonomous Driving
Guantian Zheng, Lidong Yu, Tianyu Gao et al. (+3 more)
End-to-end driving policies can achieve strong task performance while offering limited control over their driving style. Driving style describes how a vehicle accelerates, brakes, and maintains distance. Policies trained without explicit preference conditioning lack a direct mechanism for adjusting these behaviors at inference. We propose YOUDrive, a
vision–
language–action planner for continuous and composable driving-style control. Each teacher's behavior is distilled into a task vector, a learned parameter update that induces a style direction in the planner's representation. These directions can be scaled and combined at inference without retraining the backbone. Conditioned on these directions, a flow-matching decoder models a
multimodal trajectory distribution and generates kinematically feasible motion alternatives. We introduce the Style
Alignment Score (SAS) to describe human-referenced kinematic behavior after safety and task gating. On NAVSIM navtest, a single coefficient monotonically controls assertiveness across six teacher-derived personas, while PDMS remains above 0.90 across the tested individual and blended configurations, reaching 0.954. Further analyses reveal systematic policy-specific kinematics after controlling for scene context, whereas the evaluated discrete style labels provide limited additional explanatory power. Code, checkpoints, and the metric will be released.
Read the full abstract ▾
All authors Guantian Zheng, Lidong Yu, Tianyu Gao, Jiaxing Chen, Jiashu Li, Chen Lv
Robotics
Beyond Fixed Visual History: Learning Motion-Adaptive Temporal Receptive Fields for Complex-Terrain Quadruped Locomotion
Hao Zhang, Yuping Yan, Yitian Hong et al. (+5 more)
Versatile quadruped locomotion depends on a tight coupling between what the
robot perceives and what its body can execute. Yet existing visual controllers typically organize depth history using fixed temporal patterns, overlooking that the same temporal span corresponds to different spatial ranges as the
robot's speed, posture, and gait phase change. Consequently, the same visual history may be insufficient in one motion state but outdated or misaligned in another. To address this limitation, we propose TempoLoco, a body-aware dynamic visual perception framework that organizes visual
memory according to the
robot's current locomotion state. TempoLoco learns multiple complementary temporal views from a long depth buffer, encodes them within a shared feature space, and dynamically aggregates them using proprioceptive history and multi-scale terrain evidence. This enables a single policy to align visual anticipation with the
robot's current motion capabilities across complex terrains, including stairs, platforms, gaps, discrete obstacles, rough ground, and slopes. From the experiment analysis, TempoLoco achieves a 78.4% success rate, compared with 58.2% for the strongest fixed-history baseline and 62.1% for uniform multi-scale fusion. In separate capability evaluations, it completes 41/45 simulated trials across nine terrain types and 23/25 real-world trials on a Unitree B2 with a fixed arm, demonstrating robust and deployable locomotion through body-aware dynamic visual perception.
Read the full abstract ▾
All authors Hao Zhang, Yuping Yan, Yitian Hong, Luying Feng, Tianhong Wang, Fengqi Dai, Wei Chen, Yaochu Jin
Robotics
BridgeWAM: Bridging Video and Action Experts via Latent Bridge Queries
Jiaxin Fang, Ruitong Liu, Zeyu Lai et al. (+6 more)
World Action Models (WAMs) commonly couple
video and action experts through dense, layer-wise Mixture-of-
Transformers (MoT)
attention, exposing action prediction to irrelevant visual features while enforcing matched expert depths. This coupling hinders selective visual-feature use and introduces redundant computation. We introduce Bridge-of-Experts (BoE), instantiated as BridgeWAM, which aggregates visual,
language, and proprioceptive priors through 32 Latent Bridge Queries (LBQs) into a shared, limited latent space. Joint
video-action supervision lets action gradients guide the
Video Expert toward representations better suited for action prediction, while the Action Expert learns to selectively retrieve task-relevant information through cross-
attention. Decoupling expert depths and reusing LBQs during action denoising yield a 5.06B model with a 30-layer
Video Expert and a two-layer Action Expert, achieving 98.25% success on LIBERO and 72.24% pooled success on LIBERO-Plus, demonstrating effective task execution and strong generalization with an ultralight action head. Ablations support joint representation adaptation and unified LBQ conditioning. Relative to Fast-WAM, BridgeWAM reduces single-sample inference FLOPs by 28.72% in simulation profiling and real-
robot latency to 77.1 ms (a 2.47-fold speedup). Qualitative experimental results further indicate preferential
attention to target objects and task destinations across perturbations.
Read the full abstract ▾
All authors Jiaxin Fang, Ruitong Liu, Zeyu Lai, Xingyu Wang, Mingyuan Jia, Pingyue Sheng, Weikang Wan, Mengdi Xu, Yuxiang Gao
Robotics
B-Human '26 - Scalable Team Play up to 11 vs. 11
Arne Hasselbring, Tim Laue, Thomas Röfer et al. (+1 more)
B-Human won multiple competitions in the Middle Division of the new Humanoid Soccer League in 2026. In addition to regular matches with three or five players per team, we ventured into uncharted territory by playing demo matches with eleven humanoid
robots, highlighting the tactical flexibility of our behavior and the overall robustness of our system. As B-Human formerly competed in the Standard Platform League with
robots half the size and less than a third the weight, we begin by outlining the adjustments needed for this transition. The central section of this paper details our behavior architecture, drawing on concepts from RoboCup leagues that have used 11 players for much longer.
Read the full abstract ▾
All authors Arne Hasselbring, Tim Laue, Thomas Röfer, Philip Reichenberg
Robotics
CAG-WAM: Causal Action-Gain World Action Models for Robot Manipulation
Peizheng Guo, Cheng Zhang, Xingyu Zhang et al. (+2 more)
World Action Models (WAMs) combine action generation with future prediction, allowing predicted futures to support action evaluation and
robot decision making. However, the future associated with a candidate action may contain action-induced and environment-driven changes. Directly evaluating the predicted future may not fully characterize the future change associated with the candidate action. For example, a candidate action that worsens a favorable environmental future may still yield a favorable outcome, while one that improves an unfavorable environmental future may still yield an unfavorable outcome. To analyze this issue, we formulate a Structural Causal Model (SCM) for action evaluation in WAMs. The SCM formalizes how action-induced and environment-driven changes jointly shape future evaluation, and reveals a backdoor path through which environmental
evolution can also affect the future evaluation associated with a candidate action. Intervention and backdoor adjustment account for this path, providing a causal basis for characterizing the future change associated with each candidate action. Based on this analysis, we propose Causal Action-Gain World Action Model (CAG-WAM). For each candidate action, CAG-WAM predicts an action-conditioned future and an environmental future under the same current observation, and defines their evaluation difference as causal action gain (CAG) for characterizing the future change associated with the candidate action. During training, we incorporate CAG into Group Relative Policy Optimization (GRPO) to guide
robot policy learning. During inference, we evaluate the initial action with CAG and, when needed, revise it using predicted future information. Experiments on multiple
benchmarks support the effectiveness of CAG-WAM.
Read the full abstract ▾
All authors Peizheng Guo, Cheng Zhang, Xingyu Zhang, Changwen Zheng, Wenwen Qiang
Robotics
C2Flight: A Closed-Loop Co-Simulation Framework for Early-Stage Evaluation of Heterogeneous UAV Computing Systems
Xizhu Chen, Shuyuan Zhang, Yixuan Li et al. (+4 more)
Early-stage design of heterogeneous
UAV computing systems requires evaluating how
algorithm, hardware, communication, and system
topology affect closed-loop flight before the complete system is available. However, design choices interact through the robotic feedback loop, making task performance difficult to predict from isolated component metrics, while existing execution-driven approaches postpone system-level design feedback until the computing system is integrated. This work presents C2Flight, a closed-loop co-simulation framework that represents target systems as parameterized dataflow graphs and integrates models of hardware under design with independently accessible physical hardware through a unified node interface. Its graph runtime captures the combined effects of buffering, activation, resource sharing, communication, and local timing on closed-loop flight behavior. Validation against onboard and wireless offboard physical computing pipelines confirms its timing fidelity and closed-loop prediction accuracy. Two case studies demonstrate early-stage exploration of deployment and accelerator bit-width choices. These results show that C2Flight exposes system-level design trade-offs before full-system integration, enabling rapid early-stage design refinement.
Read the full abstract ▾
All authors Xizhu Chen, Shuyuan Zhang, Yixuan Li, Huazhong Yang, Yu Wang, Jincheng Yu, Hongyang Jia
Robotics
PRISM: Precision Forcing for Generalist Vision-Language-Action Policies
Chenghao Yue, Peiyan Li, Hao Wang et al. (+5 more)
Coupling a pretrained
vision-
language model (VLM) with a
diffusion or flow-matching action expert has become the mainstream
vision-
language-action (VLA) paradigm for embodied policies, yet they remain unreliable once a task demands precise manipulation, e.g., submillimeter cable insertion. We attribute this weakness to a training objective that averages flow-matching errors across contexts and flow times, allocating little supervision to the few decisive moments determining task success. Along the generative path, inference integrates a locally supervised velocity field, so equally penalized errors at different flow times can affect the executed action unequally. Consequently, precision-task performance deteriorates as physical tolerance tightens by orders of magnitude. We call this supervision imbalance across contexts and flow times Flow-Time Starvation. The remedy is to jointly align supervision allocation and representation learning with context-dependent precision demand, enabling the policy to recognize when greater precision is needed. We instantiate this precision-forcing strategy as Precision-aware Reweighting and Internalization for Skilled Manipulation (PRISM), which scores precision demand offline, strengthens flow supervision where errors are consequential and action signals are visible, and internalizes that demand in the VLM condition representation. Simulation and real-
robot experiments show that PRISM preserves or improves general-task performance while raising mean end-to-end success from 38.3% to 86.7% across drive connector insertion, bimanual connector mating, and screw tightening, substantially extending the precision boundary of existing embodied policies without new sensors or observation modalities.
Read the full abstract ▾
All authors Chenghao Yue, Peiyan Li, Hao Wang, Lei Lv, Jiahao Chen, Huaping Liu, Yu Luo, Yuanlong Zhang
Robotics
BGC-Nav: Cross-Altitude Multi-UAV Collaborative Navigation via Belief-Guided Coupling
Jiuwu Hao, Yueyang Wu, Liguo Sun et al. (+3 more)
Despite recent advances in Aerial
Vision-and-
Language Navigation (VLN), single Unmanned Aerial Vehicle (
UAV) search-and-rescue in outdoor scenarios remains challenging due to partial observability, making cross-altitude collaboration a promising direction. However, existing collaborative navigation methods rely on precise target localization and rigid goal-dispatching paradigms, ignoring high-altitude observation uncertainty and suppressing low-altitude exploration autonomy. To address these issues, we introduce BGC-Nav, a training-free cross-altitude collaboration framework that enables robust navigation via Belief-Guided Coupling (BGC). Specifically, the high-altitude
UAV maintains a Global Belief
Memory encoding task-conditioned priors, which the low-altitude
agent incorporates into decision-making via Local Exhaustion
Memory (LEM), Local Attraction Augmentation (LAA), and Directional Belief Prior (DBP) as an advisory bias rather than a rigid command. Extensive experiments on ESARBench show that BGC-Nav consistently outperforms state-of-the-art single-
agent and collaborative methods. Code is available at https://github.com/uestchjw/BGC-Nav.
Read the full abstract ▾
All authors Jiuwu Hao, Yueyang Wu, Liguo Sun, Haolin Song, Zao Zhang, Pin Lv
Robotics
DexCast: Learning Real-World Dexterous Control from Human Videos
Zihan Gao, Yuhong Zhang, Mengzhe Wang et al. (+7 more)
Recent advances in 3D hand-object interaction (HOI) perception have opened new opportunities for learning dexterous robotic skills from human
videos. However, existing dexterous imitation learning methods often rely on costly and manually teleoperated
robot demonstrations, requiring substantial human effort. Therefore, we build DEXCAST, a framework that learns dexterous robotic hand control directly from monocular RGB-D
videos by leveraging joint HOI reconstructions, without requiring any teleoperation. Furthermore, how to model high-dimensional robotic hand motion and extract temporal context to inform future action generation remains a key challenge. To this end, we propose a DexHand Motion Perception Module (DMPM) to capture inter-joint coordination and temporal dependencies in compact latent representations that condition our
diffusion policy. Extensive real-world experiments demonstrate that DEXCAST achieves exceptional performance across diverse dexterous manipulation tasks.
Read the full abstract ▾
All authors Zihan Gao, Yuhong Zhang, Mengzhe Wang, Kaisheng Liu, Shengpeng Li, Siyao Xiao, Runqing Cheng, Xiao Lin, Zhifang Liu, Haoqian Wang
Robotics
Successful but Unsafe: An Analytical Benchmark for Revealing Collisions in Vision-and-Language Navigation
Hieu D. Pham, Quoc-An Luong and Dung D. Le
Vision-and-
language navigation (VLN)
agents are typically evaluated by whether they reach a target and how efficiently they follow an instructed route, yet these metrics provide little information about collision behavior. Moreover, continuous VLN evaluation commonly uses a sliding regime, where an
agent can slide along obstacles after a collision, potentially masking the effect of unsafe actions on navigation performance. We introduce VLN-Safe, a
benchmark for analyzing collision safety under two complementary evaluation regimes: sliding, reflecting common VLN evaluation practice, and blocking, where collision-inducing actions prevent forward motion and more directly expose their consequences. Across six published
agents on R2R-CE, 38.5-56.8% of successful episodes contain contact under sliding, while switching to blocking decreases success rate by 11.7-36.1 percentage points. We further analyze signals for anticipating imminent collisions and find that depth geometry and recent movement history are more informative than policy-derived signals, while safe alternatives near the policy's preferred action are rarely available. Motivated by these findings, we introduce a training-free collision guard that vetoes risky forward actions, executes an escape maneuver, and returns control to the original policy. Under blocking, the guard reduces collision steps by 62-98% and improves success rate by 6.9-29.3 percentage points across five recurrent, topological, and
vision-
language-action
agents. Under sliding, it leaves success rate unchanged while raising the share of collision-free successes by 4.1-12.4 points. These results reveal a substantial gap between task success and collision-safe navigation and show that lightweight collision-aware intervention can improve both safety and navigation performance.
Read the full abstract ▾
Robotics
Global Optimality of Pose Graph Optimization
Shuan Ding, Ruishuang Chen, Mengfan Cao et al. (+1 more)
Pose Graph Optimization (PGO) is a classical and widely applied problem in state estimation for robotics, playing a fundamental role in localization, mapping, and navigation. Yet, it is nonconvex in nature and may have many local minimizers, thus difficult to analyze. In this paper, we investigate the optimization landscape and establish a sufficient condition of global optimality for PGO. The ultimate goal of this paper is to theoretically confirm that the optimization landscape is benign and attractive around the global minimizer, if noise is smaller than an explicit and a priori bound. To reveal such an insight, this paper develops three main technical results. First, we identify a proper basin of attraction (BOA) containing global minimizer, and characterize its
geometric property by showing that its radius is controlled by noise in an explicit way. Then, by exploring the first order optimality conditions of PGO, we prove that a local minimizer is globally optimal if a noise-controlled criterion is satisfied. Finally, by combining above two results, we prove that one and only one local minimizer exists in the BOA and this minimizer is globally optimal, when the noise is smaller than an explicit bound that can be verified a priori. Numerical simulations validate the proposed theory under various random cases, showing that the obtained conditions provide an effective criterion for certifying the global optimality of PGO.
Read the full abstract ▾
All authors Shuan Ding, Ruishuang Chen, Mengfan Cao, Zaiyue Yang
Robotics · Cross-listed from cs.CV
Can Synthetic UAV Benchmarks Be Trusted? Real-Synthetic Validation for UAV Model Selection
Yuyi Zhi, Min Du, Bill Shi et al. (+4 more)
Synthetic
UAV imagery can expand pre-deployment test evidence when task-specific Real data are scarce, but visual plausibility or semantic label
alignment alone does not establish validity for model selection. We study when Synthetic evaluation preserves the model-selection conclusions obtained from Real
UAV data through two complementary cases under fixed evaluation protocols. In a requirement-specific road-debris
benchmark, four
language-conditioned perception systems obtain lower absolute F1 on Synthetic data, yet the point-estimate ordering and top-ranked model are preserved; bootstrap analysis yields no supported pairwise reversal, and size-matched resampling preserves the top-model decision. In contrast, a closed-set stress test on a separately materialized business corpus, using four VisDrone-compatible detectors over eight empirically supported classes, changes the point-estimate top-ranked detector and reverses three of six pairwise point-estimate preference directions. A complementary audit of five
UAV business scenarios further reveals substantial Real-Synthetic mismatch in class composition, object density, and target scale, including systematic under-representation of small targets. These results show that Synthetic
UAV benchmark trust is conditional: semantic coverage is insufficient, and validity must be established empirically for the task, operating regime, and model panel of interest.
Read the full abstract ▾
All authors Yuyi Zhi, Min Du, Bill Shi, Guodong Yuan, Junheng Liang, Peifan Liao, Xin Guan
Robotics
VbotNav-0: Asynchronous Cloud-Edge Dual-Policy Navigation across Indoor and Outdoor Environments
Shiqi Tan, Yujun Zhang, Keyu Li et al. (+4 more)
Continuous indoor-outdoor navigation connects destination-level
reasoning with motion decisions that must respond to current observations. We present VbotNav-0, a cloud-edge dual-policy framework that links these processes through asynchronous policy-to-policy conditioning. The cloud policy produces subgoal guidance from visual history, navigation instructions, and route
tokens; the edge policy combines available guidance with current RGB and
LiDAR observations to generate local trajectories. This formulation allows the local perception-action loop to continue between cloud updates. A common route representation connects indoor paths and outdoor map routes, and the training recipe combines navigation demonstrations, corrective data, and grounded visual question answering (VQA). In the reported closed-loop evaluation of 258 episodes across 145 scenes, VbotNav-0 achieves 55.04% success rate and 55.03% success weighted by path length.
Read the full abstract ▾
All authors Shiqi Tan, Yujun Zhang, Keyu Li, Shuaixi Pan, Siyu Zhou, Hailong Qin, Yinan Yu
Robotics
AnyRank: Reliable Anytime Policy Ranking via Sequential Testing
Apurva Badithela, David Snyder, Tenny Yin et al. (+6 more)
Due to the high cost of hardware experiments,
robot policies are predominantly evaluated by comparing their empirical performance on a limited number of hardware trials. However, finite-sample statistical uncertainty can lead to erroneous conclusions, especially when ranking multiple policies using a fixed batch of evaluation data. We propose AnyRank: a sequential method for multi-policy ranking that (i) adaptively allocates trials to hard-to-separate policies, (ii) provides (partial) rankings at any stopping time while bounding the probability of an incorrect ranking, and (iii) optionally exploits prior information to prioritize hardware evaluations more efficiently. This prior information could be informed from expert intuition, a small set of held-out hardware evaluations, or from policy rollouts in a simulator or an action-conditioned
video world model. Notably, the prior can be imperfect; while a strong prior can provide efficiency gains on hardware, any inaccuracies in prior policy rankings do not invalidate the statistical assurances of AnyRank. We achieve the aforementioned properties through novel theoretical contributions that combine sequential testing with multiple-hypothesis graphical testing approaches. Experiments in simulation and real-world policy evaluation
benchmarks show that AnyRank requires up to 50% fewer hardware evaluations compared to existing statistically rigorous approaches, while guarding against the incorrect rankings produced by heuristic methods used by state-of-the-art
benchmarks.
Read the full abstract ▾
All authors Apurva Badithela, David Snyder, Tenny Yin, Jade Sceats, George Pappas, Nikolai Matni, Masha Itkina, Haruki Nishimura, Anirudha Majumdar
Robotics
StateSkill-Agent: An Effective and Efficient Embodied Agent via State-conditioned Skill
Changxu Wei, Jingyi Zhang, Zhiyu Zheng et al. (+6 more)
In this work, we aim to develop an embodied MLLM
agent that understands and solves embodied tasks by learning, abstracting, and reusing transferrable and generalizable behavioral patterns. To this end, we propose StateSkill, a new skill representation for embodied
agents, which introduces the concept of "finite-state machines" into skill modeling for effective and efficient skill construction and learning. The core idea of StateSkill is to model embodied interaction experiences as a finite-state machine, where a skill is organized around a set of task-progress states, state transitions, and state-conditioned guidance, enabling embodied
agents to dynamically adjust their skills as the task progresses while keeping skill contexts compact and stage-relevant. We further develop StateSkill-Evolve, an experience-driven
evolution mechanism that iteratively refines StateSkill through retrospective analysis of new successful and failed state-guided trajectories. Upon this, we introduce StateGRPO, which integrates StateSkill
evolution and model optimization via state-conditioned GRPO into a unified iterative learning loop, enabling state-conditioned skills and model capabilities to co-evolve and reinforce each other. Extensive experiments demonstrate the effectiveness of our proposed methods and trained model (i.e., StateSkill-
Agent) across various embodied tasks.
Read the full abstract ▾
All authors Changxu Wei, Jingyi Zhang, Zhiyu Zheng, Tianyi Lin, Yixian Zhang, Yunpeng Qing, Jingyuan Liu, Zhi Hou, Jiaxing Huang
Robotics
Load-Aware Quadruped Navigation with Online Contact-based Geometry Estimation
Qi Yin, XiaoLi Zhang and Yangming Shi
Navigating cluttered environments while carrying objects remains a challenging problem for quadruped
robots. Compared with works that focused on load-aware locomotion adaptation to maintain gait stability, existing autonomous navigation systems generally ignore the influence of carried objects' geometries and use fixed footprint inflation, leading to overly conservative navigation. To fill this knowledge gap, we propose a two-phase load-aware navigation learning framework. In Phase 1, we train LOADNav, a load-aware navigation policy via reinforcement learning, enabling the
robot to adapt its trajectory and clearance behavior according to the
geometric size of the carried object. In Phase 2, we train CoGE, an online geometry predictor via imitation learning, which estimates unknown payload geometry from occasional contact events. We evaluate the framework in cluttered environments with loads of varying sizes and shapes. The results show that our navigation policy exhibits clear load-aware behaviors, and the online geometry predictor estimates object geometry from occasional contact events, refining its estimate and adapting the navigation behavior accordingly. Without requiring any prior knowledge of the payload, it narrows the gap to geometry-aware policies.
Read the full abstract ▾
Robotics
ToF-Wheel Odometry: Navigating Degeneration of Localization With Limited Sensing Capabilities
Xu Yang, Wenhao Li, Zhengyu Wei et al. (+3 more)
Wheel odometry provides an efficient motion prior for ground
robots but is prone to transient violations such as wheel slip or terrain-induced tilt. In filtering-based odometry, these violations frequently trigger divergence when they coincide with constrained
geometric observability (e.g., featureless corridors). We address this challenge by proposing ToF-Wheel Odometry (TWO), a tightly-coupled filtering framework that integrates wheel kinematics with
geometric constraints from Time-of-Flight (ToF) cameras. The primary contribution of this work is a degeneracy-aware, directional covariance-shaping strategy that replaces heuristic online covariance tuning with geometry-conditioned inflation. The core idea is to infer process noise from
geometric corrections with a strict reliability check. We decompose the posterior-to-prior correction into a wheel-explainable subspace and an orthogonal remainder. The process covariance is anisotropically adjusted according to the observability of the ToF update: if the geometry is degenerate in non-kinematic directions, the remainder is suppressed as sensor noise; if the geometry is distinct, it is admitted as a valid kinematic violation. Additionally, we introduce an intra-frame feedback loop that retrospectively repairs the process covariance within the current frame to reduce structural lag during bursty disturbances. Extensive experiments on challenging scenarios demonstrate that TWO improves trajectory accuracy and robustness compared to static and adaptive Gaussian filtering, robustly navigating the trade-off between kinematic reliability and limited sensing capabilities.
Read the full abstract ▾
All authors Xu Yang, Wenhao Li, Zhengyu Wei, Zening Li, Longxiang Huang, Bo Wang
Robotics
Flow-Support-Weighted Proposals for Test-Time Verified VLA Policies
Chen Zhao, Zipeng Ye, Haitao Shen et al. (+3 more)
Test-time sampling promises to turn one action prediction into a set of distinct executable candidates that a verifier can choose among. Downstream fine-tuning works against that promise. A
vision-
language-action model trained to solve one task distribution concentrates its flow-matching decoder around the dominant demonstrated action, so repeated samples differ only marginally and the verifier receives many copies of one decision. We call this proposal contraction and identify it as a key constraint on test-time scaling: repeated samples can remain highly redundant as the candidate budget grows. Action Dynamic Sampling (ADS) alleviates the constraint on the proposal side, during training. At each training step, ADS integrates $K$ additional trajectories from the current policy under fresh base noise, scores each by the flow support along its path with a Hutchinson trace
estimator, and corrects the policy's local action distribution toward this weighted set through a detached finite-set KL term. Training cost rises; deployment cost is unchanged. Across simulation and four physical-
robot tasks, ADS raises proposal spread without the success penalty that Gaussian perturbation incurs, lifts oracle best-of-8 success on RoboCasa from 57.6% to 63.0%, improves fixed verifiers by 2.4 to 4.9 points in simulation, and raises TokenPB by up to 13.0 points on real-world manipulation.
Read the full abstract ▾
All authors Chen Zhao, Zipeng Ye, Haitao Shen, Guanlin Li, Yuchen Xie, Jing Zhang
Robotics
PC7 Model: Dynamics of Expanding and Contracting Soft Robots using Piece-wise Constant 7D Deformation Based on Similarity Transformation Group
Yamato Kudo, Taiki Ishigaki, Ko Ayusawa et al. (+1 more)
This paper proposes a novel soft
robot model named piece-wise constant 7D deformation (PC7) model that can represent expanding and contracting deformation of, for example, artificial muscle actuator. In the proposed model, we newly define the 7D deformation vector based on SIM(3) - 3D similarity transformation group - whereas a 6D deformation vector was defined for Cosserat rod based on SE(3) - the special Euclid group - in the piece-wise constant strain (PCS) model proposed in the previous study. The additional dimension of the 7D deformation vector represents expanding and contracting deformation, which comes from the scaling parameter in SIM(3). Common mathematical properties between SE(3) and SIM(3) allow us to directly extend the kinematics and dynamics of the PCS model to those of the PC7 model. In particular, we derive the dynamical equation of the PC7 model based on Euler-Poincare equation in a mathematically precise manner. We validate the proposed model through dynamical simulations assuming a thin cylinder structure or McKibben-type artificial muscle actuator.
Read the full abstract ▾
All authors Yamato Kudo, Taiki Ishigaki, Ko Ayusawa, Ko Yamamoto
Robotics
Remembering What Matters for Long-Horizon Robotic World Models
Hao Wu, Tianyu Shao, Shengju Qian et al. (+10 more)
Autoregressive
robot video world models based on discrete visual
tokens remain unreliable over long rollout horizons. Retaining the entire generated history does not necessarily improve prediction quality. We investigate a possible
memory-role conflict in Full Context decoding: persistent scene information, recent visual-action states, and stale generated history compete for
attention despite their different predictive roles. This perspective motivates ReCAP (Receding Context with Anchor Preservation), a training-free context-scheduling method for long-horizon
robot video world models. ReCAP keeps the pretrained tokenizer, world model, and visual decoder frozen. It preserves the initial scene anchor and a contiguous window of recent complete visual-action blocks while evicting older generated history. The schedule preserves within-block structure and temporal order, requires no additional modules or parameter updates, and bounds active-context growth independently of rollout length. We evaluate ReCAP on RT-1, BridgeV2, LIBERO-90, and CALVIN using frozen autoregressive world models. Across evaluated environments and horizons, ReCAP improves visual fidelity, action-conditioned consistency, and
robot structure preservation relative to Full Context. On BridgeV2, it reduces LPIPS by 63.3%. In the system
benchmark at H=64, it reduces the logical KV footprint by 73.3% and p50 latency by 6.51%.
Attention analyses show increased
attention to retained recent states, while history interventions associate larger amounts of stale context with higher prediction error. These observations are consistent with the
memory-role-conflict hypothesis, although they do not isolate it from context-length and position-handling effects. Our results support inference-time context scheduling as a practical approach to improving long-horizon
robot video prediction and computational efficiency without retraining.
Read the full abstract ▾
All authors Hao Wu, Tianyu Shao, Shengju Qian, Yizhou Zhao, Fan Zhang, Penghao Zhao, Weiyan Wang, Yufei Huang, Fan Xu, Kun Wang, Xin Wang, Ming-Hsuan Yang, Yuxuan Liang
Robotics
In View or Out of View? A Vision-Aware Filter that Infers and Reacts to the Human's Field of View
Mengxue Fu, Ya-Chuan Hsu, Yinlong Dai et al. (+2 more)
During human-
robot interaction, sight plays an important role. When the human sees a
robot, they can observe the
robot's actions and better predict the
robot's future behaviors. Conversely, when the human cannot see the
robot, they become uncertain about the
robot's intent and plans. In this paper, we focus on the human's field of view. Existing human-aware planners either ignore this field of view and assume the human can observe the entire scene, or intentionally design their interactions so the
robot is always visible to the human. By contrast, we enable
robots to infer what the human has and has not seen during interaction, i.e., the human's field of view. We then apply this estimate as a
vision-aware filter on the
robot's actions. Given a nominal
robot policy and a model of the interaction, we roll out the policy over a short horizon to see how the
robot's state updates. We then reweight the actions based on how those actions impact the
robot's predicted visibility: higher weights are assigned to actions that bring the
robot into (or away from) the human's line of sight. In practice, our framework provides an online estimation and filtering process that repeatedly updates its understanding of the human's field of view and then locally perturbs
robot actions based on the human's
vision. We evaluate our
vision-aware filter on
robots that assist humans in collaborative cooking and coordinate with them in driving environments. Across simulations and a 12-participant user study, we find that our
vision-aware filter changes both performance and preference: participants completed over three times as many subtasks with in-view assistance while cooking, and had fewer collisions with cars that purposely stayed out of their view while driving.
Read the full abstract ▾
All authors Mengxue Fu, Ya-Chuan Hsu, Yinlong Dai, Erdem B{\i}y{\i}k, Dylan P. Losey
Robotics
Chunk-wise Model-based Policy Optimization
Weixi Song, Donglin Wang and Yong-Lu Li
Model-based reinforcement learning has shown superior sample efficiency by leveraging the learned latent dynamics either in the form of derivative-free planning/imagined rollout training or dynamics gradient-based policy updates. While these approaches achieve comparatively fast learning, they all suffer, to varying degrees, from issues of recurrent compounding errors, high inference-time latency, and instability in backward propagation through time (BPTT). In this work, we propose a model-based RL approach that uses the chunk world model as a differentiable training-time evaluator to optimize the chunk actor without recurrent model invocation. Through the collaborative chunk-wise prediction of the world model and policy, we can generate H-step hypothetical rollouts in a single forward pass and directly optimize value expansion, avoiding the costly and unstable BPTT. We evaluate our method on 19 challenging high-dimensional humanoid tasks from HumanoidBench and deploy our method to the real-world for quadruped bipedal walking. Project Website: https://song-wx.github.io/cmpo.github.io/
Read the full abstract ▾
Robotics
From Video to Vision: Learning Generalizable Humanoid Carrying from Human Motion
Ziluo Ding
Most existing approaches rely on
video or motion-capture data for reinforcement learning (RL) training; however, all of these methods require an upper-level module to generate reference trajectories in real time. In object carrying tasks, the humanoid can only lift objects from designated locations along the reference trajectory, resulting in poor spatial generalization. Moreover, existing policies can only be evaluated inside a motion-capture studio. To address these limitations, we propose MOVER, a multi-stage, Teacher-Assistant-Student, RL training framework for humanoid object carrying. We first collect data from
videos and employ GENMO and GMR to retarget the
video data, combining this with data augmentation techniques to generalize the data across objects of varying shapes and sizes. In the first stage, building upon the DeepMimic \cite{peng2018deepmimic} motion-training framework, we train the humanoid to transport objects from specified locations along designated trajectories. In the second stage, we relax the tracking of the reference motion and adjust the reward so that the humanoid learns to transport objects from different positions. In the third stage, we apply a Dagger-PPO
distillation method to distill away the proprioceptive position observations and reference joint position of the humanoid. This allows the deployed policy to operate without relying on global position information from a motion-capture system; instead, the humanoid can directly perceive the object's pose from its onboard RGB-D camera to complete the carrying task. We design a series of simulation experiments to compare the success rates across stages (Stages 1 vs. 2) and before versus after
distillation. We further transfer the policy to a real humanoid, successfully transporting objects at varying positions in a motion-capture-free setting.
Read the full abstract ▾
Robotics
Before We Act: Learning Action-Conditioned Team-Response Beliefs for Decentralized Multi-Robot Manipulation
Zihao Li, Yanan Zhou, Ranpeng Qiu et al. (+2 more)
In multi-
robot manipulation, an action's value depends on how teammates respond. We present \method{} (Cooperative Action–Response Belief Estimation), a framework for learning team-response beliefs from local history. A predictive belief encoder represents the interaction and conditions a nominal action. Matched reactive and replay continuations separate each candidate's direct advantage from the utility contributed by teammate adaptation. A candidate-conditioned readout learns this response distribution alongside total advantage, and calibrated selection chooses an alternative or retains the nominal action. Our analysis characterizes when response changes the preferred action and how prediction and source errors limit the recoverable gain. With the representation, proposals, and data fixed, explicit response supervision improves candidate ranking and reduces selection regret relative to learning the same information through branch utilities. The benefit is concentrated in decisions where teammate adaptation changes the preferred action. Across RoboFactory, the MARS Control Track, and physical collaboration tasks, CARE achieves the highest average success among the evaluated policies, combining a capable nominal policy with selective coordination adjustments. Together, these results show that CARE turns predictions of teammate adaptation into effective decentralized action selection. Project page: https://jeong-zju.github.io/care/.
Read the full abstract ▾
All authors Zihao Li, Yanan Zhou, Ranpeng Qiu, Guoqiang Ren, Weiming Zhi
Robotics
GG-Nav: Goal-oriented Grounding Chain-of-Thought Elicits Reasoning in Navigation Foundation Models
Shaohao Zhu, Wensi Huang, Siqi Zhang et al. (+6 more)
Emerging strong
Vision-
Language Models (VLMs) have advanced navigation foundation models. However, these models still have unsatisfactory performance on goal-oriented navigation settings. In this paper, we first conduct a motivating study revealing that the key issue lies in the insufficient utilization of streaming observations and imprecise target localization. To tackle these challenges, we propose GG-Nav, a simple yet effective navigation framework that learns a structured chain-of-thought (CoT) mechanism via supervised fine-tuning to explicitly localize goal-relevant instances, accumulate grounding evidence, and support more reliable decision-making. Specifically, the structured CoT is designed with two key stages: (i) Observation-to-Evidence, which reasons over all intermediate frames between adjacent decision steps to retain useful goal-related evidence; and (ii) Evidence-to-Decision, which integrates the current observation with accumulated evidence to ground the target and support the generation of navigation actions or clarification questions. Extensive experiments show that GG-Nav outperforms the state-of-the-art methods in terms of success rates on the HM3D ObjectNav, VL-LN, and CoIN
benchmarks by 3.1%, 16.6%, and 5.3%, respectively. Ablation studies further show that accumulating evidence from intermediate observations reduces evidence-missing failures by 11.6%, while explicit grounding reduces stopping errors by 49.4%. We also deploy GG-Nav on a Unitree Go2
robot, demonstrating its zero-shot generalization capability in unseen real-world environments.
Read the full abstract ▾
All authors Shaohao Zhu, Wensi Huang, Siqi Zhang, Yuqiang Yang, Meng Wei, Chenyang Wan, Tianle Liu, Tai Wang, Jinming Xu
Robotics
Who to Trust in Multi-Robot Systems: Joint Robot and Object Hypothesis Trust Inference via Neural-Symbolic Cross-Validation
Ruiyang Wang, Hao-Lun Hsu, Haocheng Meng et al. (+2 more)
Multi-
robot systems (MRS) offer significant advantages over single-
robot platforms by enabling information sharing and coordination for different tasks in complex environments. However, reliable cooperation critically depends on inferring trust over both information sources and their reported hypotheses, particularly under partial observability and potential
adversarial interference. Existing approaches frequently depend on pre-established trust relationships or side-channel information, which may not be available or reliable in realistic MRS deployments. This work introduces NeST-Bayes, a neural-symbolic framework that jointly infers
robot and hypothesis trust using only inter-
robot observation consistency, without requiring preassigned
robot trust, reputation information, or side-channel identity signals. NeST-Bayes infers instantaneous trust evidence scores for
robots and object hypotheses by
reasoning over ego-centric relational graphs constructed from cross-
robot observation consistency. These evidence scores are accumulated over time through
Bayesian updates of Beta-distributed trust beliefs to maintain temporally consistent trust estimates. Extensive evaluations show that NeST-Bayes achieves the highest mean object precision and
robot recall across all
benchmark settings, and the highest
robot precision and object recall in most settings, compared to trust-based methods. Under aggressive optimized attacks, it achieves 98.8%
robot precision with 93.8% recall and 98.3% object precision with 95.4% recall. In an occlusion-rich Webots warehouse, it maintains 92.5%
robot precision with 92.0% recall and 97.3% object precision with 85.0% recall. These results demonstrate joint improvements in the reliability and coverage of retained information.
Read the full abstract ▾
All authors Ruiyang Wang, Hao-Lun Hsu, Haocheng Meng, Songqiao Xie, Miroslav Pajic
Robotics
Safety-Critical Quadrotor Navigation Using Fourth-Order Control Barrier Functions with Full Nonlinear Quadrotor Dynamics
Yanze Liu, Yaşar İdikut, Dhruv Madhusudan Agrawal et al. (+2 more)
Safety-critical quadrotor navigation in cluttered environments typically relies on hierarchical architectures in which a high-level safety filter modifies position references that are then tracked by a separate attitude controller. However, this separation can create a mismatch between translational safety objectives and attitude
evolution. To address this gap, we propose a safety controller based on fourth-order control barrier functions (CBFs) that explicitly incorporates the full nonlinear quadrotor dynamics, including attitude dynamics and torque-level actuation. Furthermore, we apply the integral high-order CBF technique to resolve the mixed-relative degree issue in the derived constraint formulation. The resulting formulation enforces obstacle-avoidance constraints while optimizing control torques, granting the safety layer direct authority over attitude behavior and eliminating the hierarchical disconnect inherent in reference-tracking safety filters. We validate the approaches in both high-fidelity simulation and real-world flight experiments, demonstrating reliable constraint satisfaction and successful obstacle avoidance. These results suggest that torque-level high-order CBFs provide a practical pathway to provably safer quadrotor navigation under full rigid-body dynamics.
Read the full abstract ▾
All authors Yanze Liu, Yaşar İdikut, Dhruv Madhusudan Agrawal, Wei Xiao, Guanrui Li