Robotics
TORL-VLA: Tactile Guided Online Reinforcement Learning for Contact-Rich Manipulation
Huaihang Zheng, Yi Yang, Kai Ma et al. (+8 more)
Vision-
Language-Action (VLA) models have become a powerful framework for robotic manipulation, and recent studies have introduced tactile or force feedback into VLAs to address contact-rich tasks. However, these models are typically deployed as offline policies. When contact conditions shift from the training distribution, the policy cannot perform online adaptation, leading to problems such as inappropriate contact forces and inefficient retries. Therefore, we propose TORL-VLA, a tactile-guided online reinforcement learning framework that couples tactile feedback with policy refinement for contact-rich manipulation. Our method introduces a tactile-derived wrench-aware VLA to predict reference actions and future wrench sequences, while a lightweight online RL module is used to refine the reference actions. To stabilize learning from mixed exploratory policy-generated and human-intervention data, we introduce an intervention-censored critic that prevents post-intervention success from being wrongly credited to policy-generated actions preceding intervention. Real-
robot experiments on long-horizon contact-rich tasks, including latch manipulation, coffee-cup placement, and egg handling, show that TORL-VLA improves success rates at both subtask and full-task levels, as well as time-bounded execution efficiency over strong baselines. Project page: https://torl-vla.github.io/
Read the full abstract ▾
All authors Huaihang Zheng, Yi Yang, Kai Ma, Shenglin Xu, Tian Xie, Guozheng Li, Xiangyu Wang, Yiren Ma, Si Liu, Yinian Mao, Baoxu Liu
Robotics
KPGrasp: Scalable Keypoint Flow Matching for Dexterous Grasp Generation
Yuansen Huang, Jiayi Chen, Haoran Liu et al. (+6 more)
Generating high-quality dexterous grasps remains challenging for learning-based methods, which often depend on carefully tuned contact losses or costly contact-based test-time refinement. We present KPGrasp, a flow-matching framework that learns dexterous grasp priors from large-scale data rather than relying on contact losses or contact-based test-time refinement. KPGrasp couples an all-Euclidean 3D hand-keypoint parameterization with a simple yet scalable
Transformer flow model. The parameterization avoids the drawbacks of the conventional mixed SE(3) pose and joint-angle output space, expresses grasps in the same frame as the object point cloud, and thus enables native spatial
reasoning; the
Transformer flow model is trained with only the standard flow-matching loss and scales effectively with data, model capacity, and batch size. Experiments demonstrate state-of-the-art performance on two simulation
benchmarks. On the Dexonomy
benchmark, it reaches a 76.3% grasp success rate, improving over the strongest directly comparable baseline by 47.4% while reducing penetration depth to 2.4 mm. The same model also achieves the best average performance on the DexGrasp Anything
benchmark without fine-tuning. For batched inference, KPGrasp requires only 0.032 s per grasp. Finally, real-world experiments on 20 diverse objects demonstrate that the pipeline can be deployed in a real-world setup. Project Page: https://yuansen-huang.github.io/KPGrasp-page/
Read the full abstract ▾
All authors Yuansen Huang, Jiayi Chen, Haoran Liu, Yubin Ke, Bing Han, Jiangran Lyu, Mi Yan, Li Yi, He Wang
Robotics
Dual Quaternion-Based Unscented Kalman Filter with Visual Inertial Odometry for Navigation in GPS-Denied Environments
Mohamed Khalifa and Hashim A. Hashim
Reliable
navigation in GPS-denied environments remains a fundamental challenge in robotics, aerospace, and autonomous vehicle applications. This paper presents a Dual Quaternion-Based Unscented Kalman Filter (DQUKF) equipped with a Visual Inertial Odometry (VIO)
algorithm for accurate state estimation enabling
navigation in GPS denied locations. The proposed framework formulates the DQUKF in an error state manner, where the nominal pose is represented by a unit dual quaternion and the local pose error is represented by a 6-dimensional twistor parameterization used for sigma point generation, covariance propagation, and measurement correction. In parallel, the VIO
algorithm tracks features across image frames, synchronizes measurements between the IMU and camera, and provides visual constraints that complement inertial propagation. Simulation results on the EuRoC MAV
dataset show that the proposed DQUKF converges under high initialization uncertainty and achieves a position RMSE of 0.2584~m in the difficult flight sequence, outperforming the
benchmark filters.
Read the full abstract ▾
Robotics
VAIC: Vision-Guided Humanoid Agile Object Interaction Control via Decoupled Commands
Dongting Li, Qianyang Wu, Xingyu Chen et al. (+9 more)
Humanoid robots hold immense potential for real-world assistance, yet agile interaction with objects in unstructured environments demands tightly coupled whole-body coordination. Despite recent advancements, current controllers face a critical deployment gap. They rely heavily on dense reference trajectories and perfect state observability, which inherently limits physical generalization. We present
Vision Guided Agile Interaction Control (VAIC), a unified framework that bridges this gap by operating exclusively on onboard depth, historical proprioception, and a decoupled user command interface. VAIC employs a two-stage
distillation paradigm. First, a privileged teacher policy masters diverse interaction skills using precise object kinematics and exact environmental states. Second, a deployable student policy distills these capabilities by replacing full body tracking with velocity targets across multiple axes and an interaction indicator for each frame. The student utilizes a recurrent object adaptation module to implicitly infer unobservable object dynamics from raw depth streams and proprioception. Evaluations and real-world deployments on the
humanoid robot demonstrate that a single VAIC policy successfully executes highly diverse dynamic tasks. These tasks include box carrying, cart interaction, and skateboarding, consistently outperforming baselines and advancing autonomous
humanoid deployment.
Read the full abstract ▾
All authors Dongting Li, Qianyang Wu, Xingyu Chen, Liang Li, Yuhang Lin, Sikai Wu, Guoyao Zhang, Mingliang Zhou, Diyun Xiang, Qiang Zhang, Renjing Xu, Jianzhu Ma
Robotics
Hewei Pan, Weiye Zhu, Zekai Zhang et al. (+4 more)
Reliable robotic
navigation necessitates the seamless integration of accurate global localization and dense, metric-consistent obstacle perception. A common strategy to achieve these capabilities involves integrating diverse sensing modalities: cameras offer rich visual features for localization, while active sensors like
LiDAR provide direct metric measurements. However, such multi-sensor configurations necessitate complex spatial-temporal calibration and increase deployment overhead. Although
vision-only approaches offer a low-cost and scalable alternative, existing
monocular visual systems typically struggle to simultaneously achieve efficient, globally consistent localization and dense, metric-consistent
geometric perception. To bridge this gap, we propose VGP-Nav, a unified framework for Metric-Aware Visual
Geometric Perception that relies solely on
monocular RGB input to jointly support metric localization and obstacle perception. Our key insight is to anchor localization-grounded visual geometry to physically meaningful scale constraints derived from ground-plane geometry, thereby providing a reliable metric reference for
monocular perception. VGP-Nav resolves
monocular scale ambiguity online and produces localization-grounded, metric obstacle representations that are directly applicable to downstream planning. Extensive experiments demonstrate strong generalization across diverse environments and successful deployment on real mobile
robots, highlighting the practicality of our approach for scalable, low-cost, and safe autonomous
navigation.
Read the full abstract ▾
All authors Hewei Pan, Weiye Zhu, Zekai Zhang, Zitong Huang, Rongtao Xu, Jinbao Wang, Feng Zheng
Robotics
Back to the Familiar Future: Failure Recovery for VLA Policies via Pre-Imagined Milestone Selection
Suyeon Shin, Juwon Kim, Hyeonbin Park et al. (+4 more)
Vision-
language-action (VLA) policies can deviate from nominal trajectories during manipulation, even when tasks remain physically feasible. Recovering from these deviations is challenging, as they push the policy into unfamiliar state spaces where direct re-planning frequently destabilizes action sequences. We propose Back to the Familiar Future (B2FF), a recovery framework for foresight-driven VLAs that leverages future visual conditioning as a recovery interface. Before execution, the VLA generates a milestone bank of familiar future states conditioned on the clean initial observation. At recovery time, a recoverability-aware selector selects a recovery milestone from this bank and enforces it as a fixed visual goal. This enables the VLA to robustly map off-trajectory observations back to a familiar future. On failure-injected LIBERO, under controlled recovery timing aligned with the injected failure, B2FF increases the average success rate of a baseline VLA from 56.3% to 74.0%, demonstrating that pre-imagined milestones can guide recovery without fine-tuning the low-level action generator.
Read the full abstract ▾
All authors Suyeon Shin, Juwon Kim, Hyeonbin Park, Hyunseo Kim, Hyundo Lee, Hyung-Sin Kim, Byoung-Tak Zhang
Robotics
RPO-PDT: Demonstrating Role-Play-Based Knowledge Adaptation for Student Support Dialogue (Demonstration System)
Filip Janik, Ewa Olton, Robert Smales et al. (+4 more)
We present RPO-PDT: a
retrieval-grounded, role-play-based dialogue system for adaptive student support in higher education. RPO-PDT is: (1) able to provide institution-specific Personal Development Tutor (PDT) guidance using structured knowledge sources; (2) constrained by explicit persona, boundary, confidentiality, and safety policies; and (3) designed around a reverse-roleplay loop where unresolved interactions are replayed from the student perspective, enabling alternative tutor strategies to be generated and stored as reusable strategy
memory. RPO-PDT supports both text-based and Furhat-based
embodied interaction for demonstrating grounded, safe, and adaptive student-support dialogue.
Read the full abstract ▾
All authors Filip Janik, Ewa Olton, Robert Smales, Harris Spratt, Shea Tait, Md Zia Ullah, Yanchao Yu
Robotics
Stabilizing Fast Dynamical Systems Using Time-of-Flight Cameras
Anthony Czubarow, Antonio Terpin and Raffaello D'Andrea
Time-of-flight cameras are popular in robotics because they provide direct depth information and are compact, inexpensive, and less dependent on ambient visible light. However, their low spatial resolution and depth noise raise questions about their suitability for stabilizing fast, unstable dynamics. In this paper, we show that an inexpensive, low-resolution time-of-flight camera can provide angle feedback for reliable and precise balancing of an inverted pendulum on a cart, a canonical control
benchmark. These results open the door to using compact, inexpensive, low-resolution time-of-flight cameras for precise feedback control of fast, unstable systems.
Read the full abstract ▾
Robotics
Self-Paced Curriculum Reinforcement Learning for Autonomous Superbike Racing in Simulation
Luca Ghisi, Jacopo Essenziale, Carlo D'Eramo et al. (+1 more)
Autonomous Racing has seen remarkable progress through deep Reinforcement Learning (RL), primarily for four-wheeled vehicles. However, motorbikes introduce substantially greater complexity due to the need to manage balance and lean angle, in addition to more reactive steering and throttle control, and a smaller weight. In this work, we present a framework for training an autonomous
agent to race a superbike in VRider SBK, a physics-accurate Unity-based motorbike simulator. Our approach integrates Soft Actor-Critic (SAC) with Self-Paced curriculum Deep reinforcement Learning (SPDL), which dynamically generates progressively more challenging tasks based on the
agent's performance, without requiring manual curriculum design. The
agent's state space comprises proprioceptive features extended with lean-angle history, along with global track features via course points. The reward signal is shaped to encourage progress along the track while penalizing instability-inducing behaviors specific to two-wheeled dynamics. Preliminary experimental results demonstrate that SPDL outperforms SAC alone in training efficiency, lap time, and driving stability across multiple tracks and motorbike models, establishing a first baseline for RL-based autonomous motorbike racing.
Read the full abstract ▾
All authors Luca Ghisi, Jacopo Essenziale, Carlo D'Eramo, Matteo Luperto
Robotics
MotionWAM: Towards Foundation World Action Models for Real-Time Humanoid Loco-Manipulation
Jia Zheng, Teli Ma, Yudong Fan et al. (+3 more)
World Action Models (WAMs) couple a
video dynamics prior to the policy and have shown encouraging results on tabletop manipulation, but iterative denoising over high-dimensional
video-action latents leaves them too slow for real-time
humanoid loco-manipulation. The problem is compounded by the dominant hierarchical paradigm, in which a high-level manipulation policy controls only the upper body while a low-level controller tracks coarse base commands – placing upper and lower body in inconsistent action spaces and reducing the legs to balance-preserving locomotion. We present MotionWAM, a real-time WAM that drives autonomous
humanoid loco-manipulation from a single
egocentric camera by conditioning the policy on the intermediate denoising features of a
video world model. MotionWAM replaces the upper-lower split with a unified motion latent and predicts whole-body motion
tokens that jointly cover locomotion, torso motion, height regulation, foot interaction, and hand manipulation in a single action space. A three-stage learning framework progressively adapts the
video world model to
egocentric visual dynamics and to the target
humanoid embodiment. On nine real-world Unitree G1 tasks, MotionWAM runs in real time, substantially outperforms
Vision-
Language-Action (VLA) baselines fine-tuned on the same demonstrations by over 30% in overall success rate, and executes task-driven foot interaction that decoupled upper-lower policies cannot reach. Our results suggest that
video-pretrained WAMs can be lifted from tabletop manipulation to coordinated, human-like whole-body
humanoid control.
Read the full abstract ▾
All authors Jia Zheng, Teli Ma, Yudong Fan, Zifan Wang, Shuo Yang, Junwei Liang
Robotics
Deterministic Execution of ROS 2 Applications via Lingua Franca
Harun Teper, Shaokai Lin, Shulu Li et al. (+2 more)
The
Robot Operating System 2 (ROS 2) is a widely used middleware for robotic systems, characterized by a publish-subscribe (pub-sub) communication mechanism in which computation is structured as callbacks dispatched by ROS 2 executors. Despite its popularity, the pub-sub pattern in ROS 2 is inherently nondeterministic: the order in which these callbacks run is nondeterministic even within a single executor, and distributed deployments add further nondeterminism from the interleaving of messages across nodes and from network latency. Such nondeterminism often leads to concurrency issues and makes it virtually impossible to analyze for safeness and provide guarantees. We present a framework that is able to convert an unmodified ROS 2 application and run it under Lingua Franca (LF), a coordination
language for deterministic execution using logical time, so that the same input always produces the same deterministic execution order. We first describe which ROS 2 features can be executed deterministically under logical time. Such features enable the possibility to establish an automatic conversion framework to extract information from a ROS 2 application and directly convert it into an LF program. The rich features of LF, such as logical-time delays,
federated execution across processes, and fault handling, can then be applied to make the ROS 2 application be executed in a deterministic and timing-predictable manner without changing the ROS 2 code. We evaluate the framework on a synthetic example and on the Autoware reference system. We show that the order in which callbacks are executed differs in default ROS 2, while also having end-to-end latencies that vary across executions. In contrast, our LF-controlled ROS 2 system produces a deterministic execution order and consistent end-to-end latencies.
Read the full abstract ▾
All authors Harun Teper, Shaokai Lin, Shulu Li, Edward A. Lee, Jian-Jia Chen
Robotics
Trajectory Optimization in Single and Dual-UAV Bearing-Only Target Localization
Zhijian Xiao, Huayu Huang, Bin Li et al. (+2 more)
Bearing-only target localization is a fundamental problem in optical measurement and finds extensive applications in unmanned aerial vehicle (
UAV) technology. Effective trajectory planning establishes favorable observation geometries, thereby enhancing the target localization accuracy of bearing-only
UAV systems. This paper proposes an trajectory optimization method for unmanned aerial vehicles (
UAVs) in bearing-only target localization scenarios. By leveraging the Fisher Information
Matrix (FIM), the proposed approach dynamically integrates the
geometric configuration and vehicle maneuverability into the optimization framework. Specifically, we introduce a spectrally-weighted FIM objective function that provides better gradient dynamics near degenerate configurations, enabling the planner to rapidly escape from poor observation conditions. For dual-
UAV scenarios, an intersection angle sine term is introduced to optimize triangulation geometry by improving the sight-line intersection angle, thereby preventing trajectory aggregation. Furthermore, we propose an improved Particle Swarm Optimization (PSO)
algorithm with motion model constraints and particle normalization to ensure the physical feasibility of the trajectory and enhance the compatibility with the objective functions. Simulation results demonstrate that the proposed method reduces the median localization error by 99.21% compared to conventional FIM-based approaches in single-
UAV scenarios, and achieves a 69.70% improvement for dual-
UAV configurations, exhibits superior performance in long-duration bearing-only target localization of maneuverability targets at extended ranges.
Read the full abstract ▾
All authors Zhijian Xiao, Huayu Huang, Bin Li, Yang Shang, Banglei Guan
Robotics
Autonomous Obstacle Removal for Excavators through Policy Learning with Particle Simulation
Yuki Kadokawa, Sandro M. Alcantara Tacora, Taro Abe et al. (+4 more)
Autonomous obstacle removal from the ground is an important earthwork task, but this is difficult to automate because an excavator must adapt its excavation trajectories over repeated cycles as soil-obstacle conditions change. Learning such state-dependent behavior requires a training environment that reproduces accumulated soil-obstacle interactions, including contact states, terrain deformation, and obstacle visibility. Accordingly, particle-based simulation is suitable for the relevant policy learning. However, particle simulation is computationally expensive, and repeated excavation cycles further increase the learning cost. We observe that the burial condition of an obstacle governs both task difficulty and simulation cost: deeper burial makes obstacle removal harder while also requiring more particles for accurate simulation. This observation motivates a burial-conditioned curriculum learning strategy. We propose a time-efficient sim-to-real policy learning framework in which the policy observes terrain and obstacle information from RGB-D measurements and then outputs a parameterized excavation trajectory; in this process, the simulator reproduces in a real-world excavator the same observation-action interface it uses under controllable burial conditions. The curriculum begins with shallow burial conditions and progressively increases burial depth while adjusting particle count, thus simultaneously controlling task difficulty and simulation cost. Experiments show that the proposed framework successfully learns an effective obstacle-removal policy, whereas baseline methods fail even after a full week of training. The proposed curriculum achieves effective performance within three days and achieves successful transfer to a real 12-ton excavator operating on open ground with various steel obstacles, thus demonstrating robust obstacle removal.
Read the full abstract ▾
All authors Yuki Kadokawa, Sandro M. Alcantara Tacora, Taro Abe, Daisuke Endo, Genki Yamauchi, Takeshi Hashimoto, Takamitsu Matsubara
Robotics
Bridged SBI: Correcting Biased Low-Fidelity Posteriors for Cost-Efficient High-Fidelity Inference
Gahee Kim, Yuki Kadokawa, Sandro M. Alcantara Tacora et al. (+5 more)
Accurate calibration of particle-based simulators is crucial for robotic earthwork simulation, but analytical calibration is challenging due to this task's highly nonlinear particle dynamics and the black-box nature of conventional simulators. Although simulation-based inference (SBI) can estimate posterior distributions over simulation parameters solely from forward simulations, applying SBI directly to high-fidelity (HF) particle simulators is often computationally prohibitive. Low-fidelity (LF) simulators with coarser particles can reduce this cost, but changes in particle size and particle count shift the parameter values needed to reproduce the same observation, producing biased LF posteriors. We propose Bridged SBI, which leverages a biased but informative LF posterior to guide HF inference. This method first uses inexpensive LF simulations to identify a coarse high-density parameter region, and then it learns a local residual bridge to transport LF posterior samples toward HF-consistent regions by correcting the LF–HF discrepancy. We analyze how sequential multi-fidelity SBI (Naive-MF) can suffer from LF-induced posterior miscoverage when it directly relies on the LF posterior without discrepancy correction. We then show that Bridged SBI is designed to alleviate this issue by explicitly modeling the LF–HF discrepancy through residual correction. Experiments on both sim-to-sim particle-parameter calibration and real-to-sim calibration with real soil observation show that Bridged SBI produces more accurate and reliable HF posteriors than HF-only SBI or the Naive-MF baseline, especially under limited HF simulation costs.
Read the full abstract ▾
All authors Gahee Kim, Yuki Kadokawa, Sandro M. Alcantara Tacora, Taro Abe, Daisuke Endo, Genki Yamauchi, Takeshi Hashimoto, Takamitsu Matsubara
Robotics
From USD Scenes to Knowledge Graphs: Zero-Shot Ontology Grounding with LLMs
Jiangtao Shuai, Zongxiong Chen, Manfred Hauswirth et al. (+1 more)
Constructing knowledge graphs from 3D simulation scenes is essential for
robot task
reasoning, but the key bottleneck, grounding scene objects to formal ontology classes, still relies on manually curated dictionaries that are brittle and do not generalize across assets. We investigate whether large
language models (LLMs) can automate this grounding step for Universal Scene Description (USD) scenes as a zero-shot, training-free alternative. On a kitchen scene (125 objects) with SOMA-HOME Ontology, LLMs achieve 90-96% exact-match accuracy with descriptive names and 49-89% with abbreviated names, substantially outperforming dictionary and
embedding baselines. Under fully opaque names, context-augmented prompting recovers up to 48%. Feature ablation reveals that LLMs primarily exploit semantic cues in the scene graph (sibling names and parent paths); anonymizing these cues reduces accuracy to 0-6%, while geometry alone yields only 4-17%.
Read the full abstract ▾
All authors Jiangtao Shuai, Zongxiong Chen, Manfred Hauswirth, Sonja Schimmler
Robotics
RAM: Reachability Across Morphologies
Tim Walter, Xinyu Chen, Jonathan Külz et al. (+1 more)
Many stages of the robotic lifecycle, from morphology synthesis to operation, rely fundamentally on the reachable workspace. However, current methods for approximating workspaces are slow, imprecise, or tied to a single morphology. We introduce Reachability Across Morphologies (RAM): a morphology-conditioned, implicit neural representation that acts as a fast, differentiable surrogate for pose reachability, generalising to unseen morphologies while inherently accounting for self-collisions. RAM is trained on a large-scale
dataset of $3\cdot10^{10}$ samples generated solely from forward kinematics, which we publish alongside the model. Experiments show that RAM reaches 84% balanced accuracy at nanosecond inference, 15 percentage points higher than a generative inverse kinematics baseline that also generalises across morphologies, while running three orders of magnitude faster. This speed carries over to downstream tasks, accelerating morphology and trajectory optimisation by one to two orders of magnitude.
Read the full abstract ▾
All authors Tim Walter, Xinyu Chen, Jonathan Külz, Matthias Althoff
Robotics
LAEI: Layered Autonomous Edge Intelligence Framework for Robust UAV Swarm Operations
Changmin Park, Wooyong Jung and Hwangnam Kim
Autonomous
UAV swarms require scalable coordination mechanisms that maintain mission performance under limited communication, environmental uncertainty, and component failures. Centralized approaches provide global coordination but suffer from communication bottlenecks and single-node vulnerabilities, whereas fully decentralized methods often lack mission-level consistency. This paper presents Layered Autonomous Edge Intelligence (LAEI), a
UAV-swarm framework that combines onboard learned policies with lightweight mission-level supervision. Each
UAV performs local perception, obstacle avoidance, and action selection onboard, while the supervisory layer provides adaptive goal reassignment, fault-aware recovery, and context-dependent policy guidance without directly controlling low-level actions. LAEI further incorporates recovery strategies, including dynamic reassociation, backup supervisory support, and fallback local autonomy, to maintain mission continuity under representative failure scenarios. We evaluate LAEI in simulated
UAV-swarm scenarios using mission completion time, collision rate, and coverage efficiency. The results show that LAEI reduces mission completion time and improves operational efficiency while maintaining collision-aware distributed
UAV-level decision-making.
Read the full abstract ▾
Robotics
Autonomous FPV Flight with Translational Optical Flow and Uncertainty Mask
Yang Deng, Yu Hu, Feng Yu et al. (+2 more)
Autonomous FPV quadrotor flight in complex environments using a
monocular RGB camera as the sole exteroceptive sensor remains a fundamental challenge. Recent research has shown that using optical flow as the input of a neural network can achieve end-to-end autonomous flight in cluttered scenes. However, extracting the most relevant information from the flow estimation is the key bottleneck limiting agility and robustness. Existing methods struggle to disentangle obstacle-induced optical flow from the ego-motion background flow and suffer from low signal-to-noise ratios near the focus of expansion (FoE). To address these issues, we decompose the optical flow into translational and rotational components and utilize only the translational flow, which captures scene geometry and depth cues. In addition, we introduce an uncertainty mask derived from inconsistencies between forward and backward flow estimates. This mask highlights obstacle structures, including those within the FoE region. Both cues are fed to a control policy trained in a differentiable simulation framework, which enables efficient first-order optimization across perception and control. We validate our approach through extensive experiments in both simulated and real-world forest environments. The proposed system achieves robust flight at speeds of up to 13.91 m/s in simulation and 11.79 m/s in real-world tests, with a 93.3% success rate over 30 real-world trials, nearly doubling the previously reported 6 m/s real-world speed of the
monocular-RGB optical-flow
UAV obstacle avoidance system.
Read the full abstract ▾
All authors Yang Deng, Yu Hu, Feng Yu, Linzuo Zhang, Danping Zou
Robotics · Cross-listed from cs.CV
ATM: Why Latent World Models Can Fail to Plan
Jiaheng Chen, Tinghe Zhang, Yucheng Xiao et al. (+5 more)
Latent world models can achieve accurate latent prediction yet still differ substantially in downstream planning performance. We argue that a key source of this discrepancy lies in the structure of action-induced latent transitions. We formalize action-identifiability through Bayes inverse risk, characterizing how much uncertainty about an action remains after observing the transition it induces. Model-predicted transitions can become highly self-decodable while encoding a domain-specific action relationship that fails to transfer to real environment transitions. We characterize this mismatch through cross-domain inverse transfer, instantiated as the Action-Consistency Transfer
Matrix (ATM), a $2\times2$ inverse-risk
matrix over real and predicted transition domains. Across TwoRoom, PushT, and OGBench-Cube, true-transition action-identifiability tracks downstream planning substantially better than standard prediction loss, while the full ATM reveals highly self-decodable yet cross-domain-inconsistent predicted transitions. Controlled interventions on the two domains further produce distinct transition structures and planning outcomes, supporting this decomposition. The same diagnostics also support lightweight model screening, reaching 98.81% pairwise ranking accuracy for candidates separated by more than 5% success.
Read the full abstract ▾
All authors Jiaheng Chen, Tinghe Zhang, Yucheng Xiao, Xinyong Cai, Lan Yu, Juncheng Bu, Jiaxing Li, Yunlong Wang
Robotics
Yucheng Deng, Pingrui Lai, Xinhai Li et al. (+5 more)
Vision-and-
Language Navigation in continuous environments requires
agents to understand the spatial structure of previously unseen environments in order to follow
language instructions. Although foundation models have opened a promising path toward zero-shot
navigation without task-specific policy training, many navigators still rely on local visual cues and linear history-based
reasoning, overlooking the spatial nature of
navigation across explored regions, traversed paths, landmarks, and their spatial relations. In this paper, we propose SpaceVLN, a
navigation agent built around Spatial Cognitive
Memory and Task-Guided Spatial
Reasoning. Specifically, SpaceVLN introduces an efficient stagewise closed-loop framework where planning and execution are organized around verifiable space–landmark stages. During
navigation, the
agent progressively abstracts explored regions into Spatial Waypoints and dynamically maintains subtask-grounded landmark evidence, forming a hierarchical Spatial Cognitive
Memory for progress localization and spatial-relation understanding. Built on this
memory, Spatial-CoT integrates task-progress
reasoning with spatial perception, analysis, and prediction, enabling Task-Guided Spatial
Reasoning for
embodied navigation. The unified stage interface enables SpaceVLN to address both
Vision-and-
Language Navigation and Object-Goal
Navigation under a unified zero-shot setting, without task-specific policy training. Across R2R-CE, RxR-CE, GN-Bench, and HM3D-OVON, SpaceVLN achieves state-of-the-art zero-shot performance, and real-
robot deployment further validates its applicability. These results highlight Spatial Cognitive
Memory and Task-Guided Spatial
Reasoning as a practical foundation for stronger
embodied navigation agents.
Read the full abstract ▾
All authors Yucheng Deng, Pingrui Lai, Xinhai Li, Chenjia Bai, Xiaoheng Deng, Chengnuo Sun, Xuelong Li, Hua Yang
Robotics · Cross-listed from cs.LG
C$^3$ache: Accelerating World Action Models with Cross Inference Chunk Cache
Weisen Zhao, Lam Nguyen, Zhicong Lu et al. (+1 more)
World Action Models (WAMs) generalize better than standard
Vision-
Language-Action (VLA) policies to novel motions and environments, because a
video-modeling objective lets them learn from abundant unlabeled
video rather than scarce labeled
robot demonstrations. This generalization is computationally expensive. To complete a task, a WAM runs over multiple inference chunks, and each chunk requires a costly denoising process. Existing acceleration methods reduce this cost by caching and reusing computation within a single chunk's denoising trajectory. Our empirical analysis reveals a substantial source of redundancy they overlook: redundancy across chunks. When a
robot executes a smooth behavior, the residuals computed at a given denoising step are strongly correlated from one chunk to the next. We introduce C$^3$ache, a training-free method that caches and reuses these residuals across inference chunks at the same denoising step. Experiments on
benchmarks with a Fast-WAM backbone show that C$^3$ache achieves up to a $2.5\times$ speedup in total wall-clock inference time, with negligible degradation in task success rate.
Read the full abstract ▾
All authors Weisen Zhao, Lam Nguyen, Zhicong Lu, Yuzhang Shang
Robotics
UniReLo: Learning a Unified Humanoid Policy from Fall Recovery to Locomotion across Diverse Terrains
Xiaoyu Xu, Zhiming Chen, Yuenan Zhao et al. (+2 more)
Reliable fall recovery, which commonly aims at attaining a nominal upright posture, is essential for the autonomous operation of
humanoid robots in unstructured field environments. Although existing posture-centered methods can synthesize coordinated whole-body recovery motions from diverse fallen configurations, they may result in a dynamically fragile support state, leading to secondary loss of balance or unstable resumption of commanded locomotion, particularly under terrain-dependent contact conditions. We propose to learn a unified
humanoid policy from fall recovery to locomotion (UniReLo) across heterogeneous field terrains. UniReLo leverages continuously gated multi-scale motion priors to modulate frame-, sequence-, and gait-level
adversarial supervision according to recovery progress, preserving the distinct temporal structures of recovery and locomotion without requiring fixed-threshold switching. In addition, terrain-conditioned recovery guidance evaluates the evolving support state using a terrain-relative support representation and support-feasibility assessment. Simulation and outdoor real-world experiments demonstrate that UniReLo can deliver stable and continuous recovery-to-locomotion behaviors for
humanoids across diverse field terrains. The supplementary
video is available at https://vsislab.github.io/UniReLo/.
Read the full abstract ▾
All authors Xiaoyu Xu, Zhiming Chen, Yuenan Zhao, Ran Song, Wei Zhang