Robotics
Humanoid Loco-Manipulation With Discrete VLA Model
Wenxin Shao, Siqi Chai, Kun Li et al. (+4 more)
Vision-
language-action (VLA) models using discrete action
tokens have proven effective for controling robotic arms on manipulation tasks. For a humanoid, however, the whole-body action space – legs, torso, arms, and hands – is far higher-dimensional and heterogeneous, raising tokenization, training, and real-time inference challenges that the previous VLA models do not address. We present Holo-M, to our knowledge the first discrete VLA model for humanoid loco-manipulation that intrinsically exploits the
language model by extending its vocabulary with action
tokens. In this model, we devise a unified action tokenizer that decomposes the humanoid action space into four body-part-specific tokenizers – end-effector, body, hand, and kinematics – enabling training across drastically different embodiments and data sources, including humanoid teleoperation, ego-centric human
video, and simulation. By extending the
language model's vocabulary with these action
tokens, we avoid the knowledge-insulation problem inherent to the models that use separate continuous action experts. To meet real-time control requirements, we decode each body part's action
tokens through grouped discrete
diffusion decoding, rather than using autoregression on the action
tokens. We have conducted extensive experiments on the SIMPLE humanoid loco-manipulation
benchmark, in which Holo-M achieves the highest success rates in both the generalist and specialist evaluations, leading the second best by significant margins. We will release all the code and model weights.
Read the full abstract ▾
All authors Wenxin Shao, Siqi Chai, Kun Li, Kerou Zhang, Xinzhou Jiang, Wei Xu, Qiang Liu
Robotics
LQR-ArUco Fusion: Robust Hierarchical Control for Navigation and Asymmetric Manipulation in Two-Wheeled Robots
Anupam Chatterjee and Arpita Kumari
We propose a hierarchical control framework to address severe dynamic instabilities and navigational drift that arise when a two-wheeled inverted pendulum (TWIP)
robot attempts asymmetric object manipulation. While two-wheeled platforms are highly manoeuvrable, their constant balancing adjustments make onboard odometry highly unreliable for precise navigation. Furthermore, the addition of a side-mounted robotic arm introduces unactuated lateral roll moments when a payload is lifted, a challenge heavily compounded on uneven terrain. To solve these coupled problems, our architecture divides the workload. An offboard
vision system tracks overhead ArUco markers to provide high-latency global waypoint navigation, bypassing odometry drift. Simultaneously, a low-latency onboard control loop rejects active physical disturbances using inertial and encoder data. In our physical experiments, this dual-loop approach enabled the custom-built
robot to navigate accurately, reject transient impacts from speed bumps, adapt to a dynamic seesaw ramp, and carry a payload securely without falling over its narrow wheelbase.
Read the full abstract ▾
Robotics
Agent Priors-guided Policy Learning
Puming Jiang, Tianrun Hu, Haozhe Du et al. (+4 more)
Robots that learn from a few demonstrations often require two forms of generalization. Compositional generalization recombines skills to solve new tasks, and skill generalization lets the learned policy behind each skill work in new situations. The two depend on each other, yet information is lost between composition and the skills it calls. Where a skill works is determined by the structure its policy is trained with, while composition sees the skill only through a separate description, such as a name, an instruction, or a symbolic operator, that omits this structure. Our key idea is to use each policy's structural prior as part of the interface between composition and the skill. A structural prior states what a behavior depends on, for example that a grasp depends only on the gripper's pose relative to the object. Built into training, it shapes where the policy generalizes; stated in
language, it tells composition where the policy applies. We instantiate this idea in
Agent Priors-guided Policy Learning (APPL). A construction
agent segments complete demonstrations into reusable skills, proposes several structural priors for each skill, and trains and verifies one policy per prior. A runtime
agent then selects among these prior-specific policies and composes them toward new task goals using their interfaces. Across MetaWorld and long-horizon ManiSkill tasks, APPL improves out-of-distribution skill generalization and enables previously unseen skill compositions; ablating the interface information substantially reduces performance. These results support the use of training-time structural assumptions as a bridge between skill learning and skill composition.
Read the full abstract ▾
All authors Puming Jiang, Tianrun Hu, Haozhe Du, Yibo Li, Zhiwei Xue, Xinhu Li, Harold Soh
Robotics
MM-ABC: Towards Generalist Mobile Manipulation via Seeing, Coordinating and Imagining
Qiwei Liang, Guangyu Chen, Shaolong Zhu et al. (+6 more)
Mobile manipulation extends
robot interaction beyond a fixed kinematic workspace by making the reachable region itself controllable. This flexibility introduces two central challenges: spatially grounded perception under continuous ego-motion and coordinated control of heterogeneous arm and base actions. Existing approaches strengthen geometry through explicit 3D representations or predictive world models, and often decouple mobility and manipulation into separate action streams. We argue that effective mobile manipulation requires not only decoupling, but also representations that support efficient cross-stream collaboration. We present MM-ABC, a foundation model built around Seeing, Coordinating, and Imagining Arm-Base Collaboration. MM-ABC combines sparse multi-level VLM features for spatial perception; a training-only future branch that uses world imagination and
geometric intent as extra supervision, strengthening perception and manipulation-intent prediction and improving the overall learning signal; and MM-APT, which coordinates separate manipulation and mobility streams through masked joint
attention and clean-action x-prediction. In controlled ablations, replacing clean-action prediction with velocity prediction lowers success on RoboCasa365 composite-seen tasks from 32.8% to 29.2%, and removing future supervision or multilevel conditioning causes larger drops. We pretrain MM-ABC on 5,000+ hours of heterogeneous
robot data spanning 400K+ episodes, 12
datasets, and 17 embodiments. Experiments cover EBench, RoboCasa365, ManiSkill-HAB, LIBERO, LIBERO-Plus, and real-world mobile manipulation. MM-ABC achieves 44.71% success on EBench, 61.2% on RoboCasa365, 99.1% on LIBERO, 82.8% on LIBERO-Plus without perturbation training, and 83% mean success on five real-world tasks.
Read the full abstract ▾
All authors Qiwei Liang, Guangyu Chen, Shaolong Zhu, Zikuan Xiao, Jinxuan Lu, Yifan Xie, Renjing Xu, Wenbo Ding, Tianxing Chen
Robotics
Denoising Multi-Robot Trajectories
Yuhao Zhang, Keisuke Okumura, Ajay Shankar et al. (+1 more)
Multi-
robot trajectory planning is a fundamental problem in multi-
robot coordination but remains computationally challenging due to its nonconvex,
multimodal, and high-dimensional nature. This work builds upon D4orm, a dynamics-aware
diffusion-denoising framework, and develops a family of planning architectures for diverse operational requirements. Unlike conventional numerical optimization methods, D4orm employs sampling-based optimization to generate solution trajectories through massively parallel sampling, leveraging modern computing architectures such as GPUs. Its
diffusion-denoising structure iteratively optimizes deformations to candidate control trajectories, providing an efficient and versatile paradigm for generating kinodynamically feasible and conflict-free trajectories. Using D4orm as the building block for advanced planners, we present a decoupled planner for improved scalability, an online receding-horizon planner with feedback control, and a distributed planner for resource-constrained settings. Evaluations with differential-drive and holonomic
robots in 2D and 3D environments demonstrate that D4orm-based approaches find high-quality solutions faster and more reliably than other sampling-based optimization methods, such as MPPI, as well as a learned
diffusion-model-based method. We further demonstrate zero-shot deployment on ten real quadrotors with obstacles, large-scale deconfliction with 100 simulated
robots, and fully onboard distributed `lifelong' operation with six ground
robots. Overall, these results establish
diffusion denoising as a scalable and reliable framework for multi-
robot coordination. Code and
video: https://github.com/proroklab/d4orm
Read the full abstract ▾
All authors Yuhao Zhang, Keisuke Okumura, Ajay Shankar, Amanda Prorok
Robotics
CollisionSplatting: Collision-Aware Motion Planning in 3DGS Scenes with Image-Conditioned Objectives and Adjustable Conservatism
R. Khorrambakht, Joaquim Ortiz-Haro, Stephan Weiss et al. (+1 more)
Incorporating dense visual information into motion planning remains challenging, as
geometric planners rely on abstracted scene representations that discard visual richness, while learned visual models often lack
geometric interpretability and computational efficiency. This paper introduces CollisionSplatting, a simple, modular, GPU-accelerated, probability-inspired distance metric with tunable conservatism that operates directly on standard 3D Gaussian Splatting (3DGS) scenes. When combined with learned image-conditioned reward functions, this metric enables joint
geometric and visual planning by unifying collision-aware costs with image-space objectives. We integrate the metric into GPU-accelerated Model Predictive Path Integral (MPPI) and Rapidly-Exploring Random Tree (RRT) planners, and show on-par or better collision-classification performance compared to representative baselines while achieving substantially higher collision-checking throughput and significantly lower VRAM usage. Finally, we demonstrate the effectiveness of our metric in real-world
vision-guided navigation and manipulation tasks, highlighting 3DGS as a practical bridge between rich perception and real-time motion planning.
Read the full abstract ▾
All authors R. Khorrambakht, Joaquim Ortiz-Haro, Stephan Weiss, Ludovic Righetti
Robotics
F4R: Failure-Driven Recognition, Reconstruction, Refinement, and Redeployment for Continual Robot Self-Improvement
Zhuoyuan Yu, Jiacheng Wang, Tianle Liu et al. (+13 more)
The real-world performance of current
vision-
language-action models is fundamentally constrained by the limited coverage of expert demonstrations and their insufficient understanding of physical interactions. A common remedy is to collect additional real-world demonstrations of newly encountered failures. However, this process is costly, inefficient, potentially unsafe, and difficult to scale. To address this challenge, we propose Failure for Rising (F4R), a failure-driven real-to-sim-to-real closed-loop learning framework that converts real-world failures into targeted policy improvement. F4R first uses an
agent to automatically identify and diagnose failures from rollouts. It reconstructs each failure as an interactive, object-centric table-top environment that preserves the task-relevant spatial and physical conditions. The policy is then refined through failure-conditioned sim-real co-training followed by targeted reinforcement learning in the reconstructed environments. The improved policy is subsequently redeployed, while newly observed failures are continuously fed back into the next reconstruction and learning cycle. Real-world evaluations on four manipulation tasks show that F4R achieves 93.75% In-Distribution and 90.0% Out-of-Distribution (OOD) success, outperforming the budget-matched Targeted BC baseline by 18.75 percentage points under OOD conditions without collecting additional real-world corrective demonstrations.
Read the full abstract ▾
All authors Zhuoyuan Yu, Jiacheng Wang, Tianle Liu, Yihua Ren, Peng Yu, Chen Bai, Ziheng Zhang, Yufei Jia, Jindou Jia, Yuhang Zhang, Xinrui Zhang, Shang Yujing, Yuxiang Chen, Chuhao Zhou, Tiancai Wang, Jianfei Yang
Robotics
EdgeVLN: Runtime-Aware Deployment Ready Quantized Vision Language Navigation Model
Rithvik Jonna, Man Namgung, Aakash Gurram et al. (+1 more)
Vision-
language navigation (VLN) models perform well but target compute-rich platforms, limiting deployment on
memory- and power-constrained robotic edge devices. Compression alone does not establish whether a VLN model fits the
memory, latency, and energy budgets of an edge platform while preserving navigation behavior. We introduce EdgeVLN, a runtime-aware, deployment-ready quantized VLN model that closes this gap. EdgeVLN combines a quantized StreamVLN model with Latent Trajectory Termination Extractor (LATTE), a lightweight causal
transformer that improves real-time stopping by predicting a Stop Action verifier rank. Both execute through our llama.cpp VLN driver, which reconstructs streaming context and prunes
memory tokens on-board. We characterize a pretrained StreamVLN backbone across weight quantization from 8 to 2 bits and multiple inference runtimes to identify a feasible operating point. LATTE reuses backbone hidden states within the budget freed by quantization, requiring neither a second
vision encoder nor an additional backbone forward pass. We evaluate six backbone precisions and seven candidate stop heads on BF16 and IQ4 NL across all 1,839 R2R VLN-CE val-unseen episodes. We measure success rate (SR) in simulation and latency, energy, and resident
memory on an NVIDIA Jetson Orin NX 16 GB. LATTE achieves our highest SR, 58.02 percent on the deployed 4-bit model, exceeding the BF16 baseline with only 0.013 s additional latency per navigation step. Four-bit formats achieve nearly identical SR, but step energy varies 36.8 times by execution path. Only IQ4 NL under our VLN driver fits the board, using 11.35 GB resident
memory while running 20.8 times faster and using 13.3 times less energy than storage-streamed BF16. INT2 collapses. Runtime selection,
memory-
token pruning, and quantization are essential for efficient edge deployment.
Read the full abstract ▾
All authors Rithvik Jonna, Man Namgung, Aakash Gurram, Tinoosh Mohsenin
Robotics
Inspection-SPARS: Task-Oriented Sparse Roadmaps for Inspection Planning
Adir Morgan, Oren Salzman and Kiril Solovey
Inspection planning seeks a minimum-length collision-free
robot tour that observes a given set of points of interest (POIs). Sampling-based methods reduce this continuous problem to a graph inspection planning (GIP) problem over a discrete roadmap, which is then solved using combinatorial solvers. Dense roadmaps capture diverse inspection viewpoints and motion shortcuts, and thus admit higher-quality solutions, but they induce large combinatorial search spaces on which state-of-the-art GIP solvers struggle to find good solutions within practical time budgets. Roadmap sparsification—restructuring a dense roadmap into a compact representation that preserves connectivity and path lengths—can alleviate this burden. However, existing sparsification approaches are either agnostic to the underlying inspection task, or strive to ensure coverage of the POIs without accounting for the quality of the resulting inspection plan. We present Inspection-SPARS, which is, to our knowledge, the first inspection-roadmap sparsifier with POI coverage and path-quality guarantees relative to the dense roadmap. To this end, we generalize the SPARS framework, a popular task-agnostic sparsifier, from purely
geometric criteria to task-oriented ones, introducing an inspection-aware vertex admission mechanism that treats POI coverage as a first-class sparsification criterion alongside connectivity and path quality. Experiments in realistic 3D environments show that Inspection-SPARS reduces vertex and edge counts by 4-8x while preserving coverage, allowing the GIP solver to compute tours up to 25% shorter than with the dense roadmap or state-of-the-art inspection roadmap. More broadly, Inspection-SPARS shows that sparsification can be made task-aware without sacrificing guarantees on solution quality.
Read the full abstract ▾
Robotics
Terrain-Aware Autonomous Planetary Exploration for Exteroceptive-Proprioceptive Mapping with Quadruped Scouts
Alberto Sanchez-Delgado, João Carlos Virgolino Soares, Victor Barasuol et al. (+1 more)
Autonomous planetary exploration requires
robots to navigate unknown, uneven terrain while assessing risk, traversability, and energetic cost. Quadruped scouts are well suited for this task because they can traverse irregular surfaces and gather mobility-relevant information during locomotion. This paper presents a terrain-aware exploration framework that combines exteroceptive and proprioceptive mapping for a quadruped
robot in lunar-like environments. An onboard RGB-D camera builds
robot-centered elevation maps, estimates
geometric traversability, and derives navigation costs for autonomous planning. In parallel, proprioceptive measurements provide interaction-aware terrain cues that complement geometry-based assessment. Local maps are incrementally registered into a global multi-layer representation, which is used by an exploration module to select targets in unexplored regions of interest. The targets are reached by an autonomous navigation system that guides collision-aware motion using the available map and cost layers. Simulation results on NVIDIA Isaac Sim show autonomous exploration, map expansion, and spatial association between terrain geometry and
robot-terrain interaction. Subsequent navigation using this information exhibits lower average Cost of Transport (CoT) than initial exploration.
Read the full abstract ▾
All authors Alberto Sanchez-Delgado, João Carlos Virgolino Soares, Victor Barasuol, Claudio Semini
Robotics
ForVis: An In-Field Dataset and Benchmark for VIO Using Under-Canopy UAV Flights in Forests
Arman Kiani, Masoud Ataei, Elvis Gyaase et al. (+4 more)
Visual-inertial Simultaneous Localization and Mapping (VI-SLAM) for
UAVs remains difficult to evaluate in real forest environments, where motion, illumination changes, repetitive vegetation, and vibration can all affect estimation. We present ForVis, an in-field
dataset and
benchmark for evaluating VI-SLAM during
UAV flight in forest environments. The
dataset contains twelve flights across open meadow, above-canopy, and under-canopy conditions in each environment. In total, it provides 563.8s of flight over 1096.8m of trajectory, recorded simultaneously with an Intel RealSense D435i and an OAK-D Pro Wide together with inertial and flight-controller data. We
benchmark seven open-source VI-SLAM systems over 504 runs. The results show that sensor choice has a larger effect on trajectory error than the spread between
algorithms: all seven methods achieve lower median error on the OAK-D Pro than on the D435i. ForVis is intended to support evaluation of speed, accuracy and robustness for VI-SLAM in challenging forest flight.
Read the full abstract ▾
All authors Arman Kiani, Masoud Ataei, Elvis Gyaase, Jeffrey Eiyike, Aaron Weiskittel, Prabuddha Chakraborty, Vikas Dhiman
Robotics
Robot Tool Design from Scratch via Behavior-Aware Hierarchical Optimization
Yinghan Chen, Xiyao Tian, Yizan Dai et al. (+2 more)
The ability to design a tool for a task marks a level of intelligence beyond merely understanding, selecting, or using one. Existing methods for robotic tool design typically optimize a tool's continuous shape and action within a structure that is prescribed or generated beforehand, so the structure itself stays outside the physical optimization loop. We study task-driven tool design from scratch, where tool structure, shape, and action are all derived from the desired physical outcome. Here we show that the three elements can be designed jointly by HOT, a hierarchical optimization whose upper level searches over discrete tool structures with BASS, while lower-level physical optimization evaluates their task behavior and returns milestone progress as behavioral evidence for the search, ultimately providing jointly optimized shape and action. On four tool-use tasks with distinct physical functions, HOT discovers functional structures after evaluating only a small fraction of search spaces containing up to 56 million structures, and the subsequent refinement of their geometry lowers the task loss on all tasks while preserving success, through deformations that are functionally interpretable. Once 3D printed, the tools accomplish all tasks on a real
robot with the actions found in simulation. Designing tools from required physical effects, rather than a catalog of known tools, is a step toward the open-ended tool making seen in humans and animals.
Read the full abstract ▾
All authors Yinghan Chen, Xiyao Tian, Yizan Dai, Yuyang Li, Yixin Zhu
Robotics
CoBrush: A Hierarchical Planning Framework for Human-Robot Co-Painting
Dantong Qin, Yike Guo, Qinlin Liu et al. (+2 more)
Embodied co-painting requires a
robot to repeatedly update a shared physical canvas while human intent evolves over interaction. Existing reference-driven painters or reactive assistants are typically optimized for single-shot rendering or sketch completion, limiting their ability to sustain coherent multi-round collaboration or to construct complex, content-rich scenes over time. We present CoBrush, a hierarchical framework that formulates multi-round co-painting as a coordinated semantic, spatial, and execution process. By separating high-level intent inference from spatial grounding and stroke-level control, the system supports progressive scene development on real acrylic canvases. We evaluate the framework through real human-
robot painting sessions, stress tests, and user studies. Compared to single-turn baselines, our approach achieves stronger semantic
alignment, more stable spatial progression, and higher perceived plausibility of
robot actions. These results demonstrate that structured multi-stage
reasoning improves the coherence and robustness of interactive painting and supports the progressive development of content-rich physical artworks.
Read the full abstract ▾
All authors Dantong Qin, Yike Guo, Qinlin Liu, Alessandro Bozzon, Pan Wang
Robotics
Rethinking Causal Action Tokenization with Conditional Annealing in Flow Matching
Chenyu Zhang, Yuhang Cao, Daru Du et al. (+8 more)
Autoregressive
Vision-
Language-Action (VLA) models offer a scalable path to
robot learning, yet existing action tokenizers treat tokenization as a compression problem, producing representations that are semantically misaligned with the autoregressive backbone. We propose CATok, a causal action tokenizer that reframes tokenization as a causally structured generative process. CATok introduces a conditional annealing mechanism that extracts action
tokens by progressively annealing a flow-matching process: each
token is conditioned on all preceding
tokens and encodes the residual reconstruction signal at a specific noise level, establishing a coarse-to-fine causal
token space whose generative semantics are structurally aligned with autoregressive modeling. A
token-conditioned flow-matching decoder built on
Multimodal Diffusion Transformer (MMDiT) reconstructs continuous action chunks from these discrete
tokens with the precision of hybrid
diffusion-head architectures. This discrete bottleneck enforces knowledge insulation by design, cleanly separating high-level semantic
reasoning from low-level motor execution without requiring explicit
attention masking. Extensive evaluations across three simulation
benchmarks and real-world robotic manipulation tasks demonstrate that CATok consistently surpasses existing tokenization methods in both reconstruction fidelity-compression tradeoff and inference efficiency, while improving VLA task success rate and training efficiency, establishing a high-performance, scalable foundation for purely autoregressive VLA systems.
Read the full abstract ▾
All authors Chenyu Zhang, Yuhang Cao, Daru Du, Yingxi Lu, Jing Shao, Ruoqu Chen, Jiajun Liu, Liu Cao, Yicheng Liu, Hang Zhao, Mengdi Xu
Robotics · Cross-listed from cs.LG
Manifold-Stable Flow Matching
Amirhossein Nazerian, Ali Pezeshki and Jianguo Zhao
Flow matching (FM) learns generative dynamics through velocity regression.
Geometric FM variants commonly assume a prior supported on the data
manifold, requiring
geometric knowledge that is often unavailable. Without such knowledge, low regression error alone does not guarantee
manifold adherence. Adherence keeps generated samples within valid configurations and is empirically associated with better task performance. We introduce
manifold-stable flow matching (MSFM), which can start from an arbitrary ambient prior, not necessarily supported on the
manifold. Using tools from nonlinear dynamics, namely contraction theory, MSFM combines learned tangential transport with prescribed normal contraction. The construction uses analytical projectors for known
manifolds and local affine proxies estimated by principal component analysis for unknown data geometry. By implementing contraction theory in both cases of known and unknown
manifolds, we guarantee
manifold invariance and transverse convergence to the
manifold within a desired time window (e.g., one second). We derive a family of compatible probability paths and decompose the training loss into a learnable tangential term and a normal residual. An ellipse experiment attains a mean terminal off-
manifold error of order $10^{-6}$. In Push-T robotic experiments, MSFM raises success from $74\%$ to $82\%$. In the Robomimic Square task, success increases from $60\%$ to $72\%$, while rotation-
manifold deviation decreases from order $10^{-2}$ to $10^{-7}$. The MSFM terminal
geometric errors are controlled by the chosen numerical tolerance. These results demonstrate stronger
geometric adherence and higher observed task performance, supporting prescribed normal contraction as a complement to learned generative transport.
Read the full abstract ▾
Robotics
Uni-VLaT: Whole-Body Tactile Adaptation of VLA Policies for Humanoid Loco-Manipulation
Zihao Wang, Shutong Liu, Siqi Zheng et al. (+5 more)
Physical contact often determines how a humanoid should respond during loco-manipulation, yet
vision and proprioception alone are often insufficient to characterize physical interaction, especially when the contact region is occluded. Unlike sparse force or torque measurements at predefined regions, distributed tactile sensing preserves spatially resolved contact patterns across the
robot body. We therefore study how to integrate such whole-body tactile information into
vision-
language-action (VLA) policies for contact-rich control. Our approach, Uni-VLaT, introduces a tactile pathway whose latent state is trained not only for action generation, but also to predict future tactile, proprioceptive, and visual representations. This predictive objective builds a tactile-anchored
multimodal context, encouraging a more structured understanding of the physical world. We evaluate Uni-VLaT on five real-
robot tasks covering tactile-triggered locomotion, sustained physical interaction, human-
robot contact, and loco-manipulation. Uni-VLaT achieves a 75% average success rate, outperforming a baseline without tactile input by 43 points and a tactile-input baseline without predictive supervision by 7 points. Across two pretrained VLA backbones, our method improves Table Sweeping by 30 points on both backbones and Back-Tap Walking by 85-90 points. Ablations further show that contextualized tactile prediction and absolute future targets are critical to performance. These results indicate that predictive tactile learning provides an effective route for extending pretrained VLA policies to whole-body physical interaction.
Read the full abstract ▾
All authors Zihao Wang, Shutong Liu, Siqi Zheng, Liu Cao, Ruoqu Chen, Rundong Liu, Yanchao Yang, Mengdi Xu
Robotics
Revision, Not Restart: Revisable Visual Plans for Closed-Loop World-Action Models
Pengyiang Liu, Junbo Niu, Wenhao Zheng et al. (+5 more)
World-action models use predicted visual futures to condition
robot actions, yet execution feedback can invalidate parts of a prediction while leaving its task structure useful. We propose Revisable Temporal Planning (RTP), which maintains the visual future as a persistent action condition and revises it after feedback. Its central mechanism is a learned revision bridge: it resumes an intermediate state saved during visual generation and adapts its continuation to current observations. Visual and action supervision connect this revision to subsequent control. Time-aware history supplies observed evidence, and an adaptive policy selects retention, bridge revision, or fresh replanning from new noise before decoding the next action. On RoboMME and RMBench, RTP achieves task-averaged success rates of 48.6% and 84.8%, respectively. Matched comparisons support learned continuation; estimated checkpoint-source and action-prefix effects are positive but less precisely resolved. These results connect feedback-driven visual-plan revision to closed-loop task performance. Project Page: https://PLACEHOLDER.github.io/RTP/
Read the full abstract ▾
All authors Pengyiang Liu, Junbo Niu, Wenhao Zheng, Xinchen Chen, Canyu Li, Zhongyue Shi, Jiahao Xie, Si Liu
Robotics
Self-Evolving Coding Agents: From Digital Programs to Physical-World Intelligence
Hongcheng Gao, Jingjing Zhou, Zelin Zheng et al. (+10 more)
Vision-
language-action (VLA) and world-action (WAM) models map observations and instructions directly to
robot actions. This directness ties a policy to training: minor layout or viewpoint changes cause failure, and instructions generalize poorly. The root cause lies in representation: task requirements, conditions, progress, and failure recovery are implicitly encoded in action sequences, making them difficult to inspect or revise. Digital coding
agents offer a precedent: LLMs call tools, verify results, and revise from feedback as executable code. The same working pattern of explicit state, manageable execution, and revisable procedures underlies generalization and long-horizon execution in the physical world, letting physical experience return as reusable programs,
memory, or evidence. We propose Physical Coding, representing task state and execution as code. Code as World records objects, relations, constraints, and progress; Code as Policy organizes planning, verification, recovery, and execution. We build HexaAnything, which calls perception, planning, and control tools, including VLA/WAM policies, and makes in-the-loop decisions from external feedback. Verified traces become data and
memory, enabling
evolution from tools and Harness to model weights, architectures, and ultimately hardware and task design. On RoboCasa365, HexaAnything improves Composite-Unseen and overall success over XR-1 VLA, and its Harness-trained HexaModel beats the base on every split, indicating code traces internalize physical execution. On PhyBench and a dual-arm AgileX
robot, the
agent autonomously completes physics experiments and most tabletop tasks, often faster than published results. We observe data, model, and tool self-
evolution; future work targets weight internalization, autonomous redesign of architectures,
languages, representations, and tasks, and deployment in manufacturing and science.
Read the full abstract ▾
All authors Hongcheng Gao, Jingjing Zhou, Zelin Zheng, Shijia Ge, Jay Zhu, Yazhe Wang, Jianshu Zeng, Xuan Shangguan, Di Wu, Lingyu He, Zhiqi Jia, Sihang Wu, Xiao He
Robotics
Memory in the Sky: Low-Altitude Question Answering with Multi-Agent Memory Aggregation
Chengyang Li, Yujie Wan, Shuai Wang et al. (+6 more)
This paper studies low-altitude question answering (LAQA), in which distributed unmanned aerial vehicle (
UAV)
memories are aggregated at a ground server to answer questions about observations over a long horizon. Unlike conventional resource allocation based on sensing, communication, control, or computation metrics, LAQA requires an explicit measure of
memory value. We propose a generative
adversarial exam (GAE) that uses forward simulation to evaluate
memory retrieval and exam scores to quantify
memory quality. This enables the downstream QA value of candidate
memories to be measured and optimized without accessing the internal mechanisms of the black-box captioning,
retrieval, and
reasoning pipeline. Building on this metric, we develop a
memory-centric (MemCen) framework that jointly selects
UAVs and allocates transmit power to maximize
memory quality under communication constraints. In the noise-limited regime, we derive a QoM-aware capped water-filling law that explicitly connects task utility with physical-layer power allocation. We further develop penalty successive optimization (PSO) and learning to memorize (L2M) solvers. MemCen achieves QA accuracies of 92.4% and 84.0% in CARLA Town04 and Town05 under static and dynamic communication conditions, respectively. In real-world experiments, MemCen achieves 88.5% QA accuracy on the panoramic multi-
agent system (PMAS)
benchmark. Finally,
UAV-to-
robot-dog demonstrations further validate the practical utility of the acquired
memories for environmental understanding and navigation.
Read the full abstract ▾
All authors Chengyang Li, Yujie Wan, Shuai Wang, Kejiang Ye, Weijie Yuan, Boyu Zhou, Yik-Chung Wu, Chengzhong Xu, Huseyin Arslan
Robotics
Adaptive Safety Filtering for Frozen ACC Policies via Conformal Residual Calibration
Zhiruo Zhou, Rigaudiere Z. Li, Chen Xiwen et al. (+3 more)
Frozen adaptive cruise control (ACC) policies can violate constraints when deployment dynamics differ from their training conditions. We propose residual-aware conformal action filtering (RACF), which calibrates residuals of a fixed nominal predictor and converts their
quantile into an operating margin for finite-model action projection. Completed transitions update margins and candidate selection without retraining the policy. In a registered comparison over 2,400 controller-trial units, Adaptive RACF achieves 94.3% episode safety, improving by 19.9 percentage points over the evaluated nominal CBF-QP baseline while reducing projection frequency from 8.11% to 6.63%. A controlled study isolates a 4.54-point improvement from residual-margin injection. In a separate matched-hardware evaluation, Adaptive reduces mean amortized rollout time by 21.2% relative to Robust CBF-QP, with 161/180 versus 170/180 safe episodes. We characterize conditions linking one-step residual coverage to constraint satisfaction and quantify the observed safety-computation trade-offs.
Read the full abstract ▾
All authors Zhiruo Zhou, Rigaudiere Z. Li, Chen Xiwen, Yucheng Chen, Xiaojun Zhu, Houde Liu
Robotics
From Pixel to Poses: Object-centric Tool Manipulation Learning from Human Demonstrations
Bangjun Wang, Longyan Wu, Yukun Wei et al. (+8 more)
Scaling up robotic manipulation is primarily bottlenecked by the scarcity of real-world
robot data. While recent approaches leverage human
video demonstrations to mitigate this shortage, they remain computationally expensive and still rely on paired human-
robot data for domain
alignment. Although current state-of-the-arts excel at long-horizon tasks, they struggle with the delicate and precise control required for complex tool manipulation. To overcome these limitations, we introduce P2P-T, from Pixel to Poses for Tool Manipulation, a data-efficient, object-centric framework that learns tool use directly from human demonstrations. P2P-T bridges the cognitive and physical execution gap through a two-stage approach. First, pretraining an object-centric world model to extract stable pose priors; second, integrating these priors into an efficient, pose-aware low-level policy. By utilizing a robust automated data processing pipeline powered by modern foundation models, P2P-T completely bypasses the need for human-
robot aligned data. This reduces overall training overhead drastically. With minimal per-task fine-tuning, our framework achieves a 73% improvement over the previous state of the art in execution performance on complex, real-world tool manipulation tasks that currently remain out of reach for standard large-scale pretrained models.
Read the full abstract ▾
All authors Bangjun Wang, Longyan Wu, Yukun Wei, Shenghe Shao, Chaoyi Huang, Wenze Cui, Zetong Xu, Hanlin Wu, Long Chen, Yi Ma, Hongyang Li
Robotics
DexAgent: An Agentic Human2Sim2Robot Framework for Dexterous Manipulation with Self-Evolving Tool Library
Youhui Wang, Yunzhu Li, Li Fei-Fei et al. (+2 more)
Human
videos offer a scalable source of demonstrations for dexterous
robot manipulation. However, existing human-to-simulation-to-
robot (Human2Sim2
Robot) pipelines rely on predefined procedures that struggle to accommodate diverse object properties and interactions, particularly those involving articulated and deformable objects. We introduce DexAgent, an agentic Human2Sim2
Robot framework that converts a single egocentric human
video and a task
prompt into physically grounded
robot trajectories for policy training. It operates through four stages: semantic understanding of human
videos, property-based simulation reconstruction,
robot trajectory optimization, and
robot data generation. At each stage, DexAgent adapts its approach to the task and object properties by selecting suitable skills from its tool library or developing new ones when needed. Property-specific verifiers assess stage outcomes for physical validity and task-specific requirements and provide feedback for refinement, preventing error propagation through the workflow. This adaptive, verification-guided process allows DexAgent to process diverse objects and long-horizon tasks. In the final stage, DexAgent varies object and
robot states in simulation to generate diverse
robot trajectories from a single human
video, then retextures the rendered observations to facilitate sim-to-real transfer. Newly developed skills and verifiers are retained in its tool library, making it self-evolving to accumulate reusable capabilities. This reduces processing time as DexAgent encounters more human
videos. Across eleven real-world tasks, policies trained with DexAgent-generated data achieve a 3.5x higher success rate than competing baselines. Project website: https://dexagent1.github.io/.
Read the full abstract ▾
All authors Youhui Wang, Yunzhu Li, Li Fei-Fei, Jiajun Wu, Huang Huang
Robotics · Cross-listed from cs.CV
RoGSW4RLD: Feed-Forward 4D Gaussian Lifting for Robot World Model Rollouts
Jin Hyun Kim, Min Young Kim, Soohwan Song et al. (+1 more)
Action-conditioned
video world models predict future
robot interactions from multiple cameras, yet their outputs remain disparate
video collections rather than a shared metric scene queryable across viewpoints and time. While existing 4D reconstruction methods offer a path to spatialize these predictions, independently reconstructing and merging each camera stream fails to enforce cross-view consistency. This limitation is particularly detrimental when combining moving
robot-mounted cameras with fixed external views. To address this, we introduce RoGSW4RLD, a feed-forward framework that lifts synchronized multi-camera rollouts into a unified, time-queryable metric 4D Gaussian field. Rather than learning a separate
geometric transition model, RoGSW4RLD directly reconstructs the visual future generated by existing world models. Its core innovation is a two-stage architecture: Stage 1 jointly forms the metric 4D field by fusing cross-view evidence with
robot-specific articulated geometry and kinematics, while Stage 2 refines the field's geometry and appearance while strictly preserving the initial temporal displacements. Evaluated on 256 held-out DROID episodes, RoGSW4RLD significantly outperforms camera-wise reconstruction with calibrated merging, improving novel-view PSNR by 2.15 dB, reducing depth AbsRel by 47%, and lowering
robot displacement error by 61%. These robust gains extend to action-conditioned Cosmos 3 rollouts, demonstrating that predicted
video futures can be successfully translated into consistent, spatially queryable 4D metric representations.
Read the full abstract ▾
All authors Jin Hyun Kim, Min Young Kim, Soohwan Song, Daekyum Kim
Robotics
GuardPIBT: Counterfactually Gated Neural Guidance for Ultra-Large-Scale 3D Multi-Agent Path Finding
Yuan Zhou, Zhenyu Hou, Guangtong Xu et al. (+4 more)
Large-scale 3D multi-
agent path finding becomes increasingly difficult under dense traffic. Priority Inheritance with Backtracking (PIBT) scales well, but its one-step goal-directed ordering may become insufficient under dense interactions and large-scale congestion. We present GuardPIBT, which augments rather than replaces the PIBT executor: neural predictions only propose residual reorderings of PIBT's native candidates, while final actions remain determined by PIBT. First, local graph
attention models nearby interactions, while global source–goal transport features provide population-level coordination context for candidate reordering. Second, a counterfactual group gate filters reorderings whose closed-loop effects may degrade coordination. Third, for ultra-large populations, population-adaptive grouping preserves decision granularity, asynchronous cached inference amortizes neural computation, and selective repair resolves long-tail
agents. PIBT retains validity checking, priority inheritance, and backtracking throughout. Experiments with up to 100,000
agents demonstrate reliable completion across 2D and 3D environments, including all three 100,000-
agent warehouse runs with zero audited graph violations. The project website is available at {\color{magenta}https://guardpibt.github.io/GuardPIBT/}.
Read the full abstract ▾
All authors Yuan Zhou, Zhenyu Hou, Guangtong Xu, Xiaoqiang Ji, Yuqing Tang, Jialiang Hou, Fei Gao
Robotics
Repair Before You Fuse: Frozen-Host Adaptation for Corrupted-but-Present Sensors
Gia-Huy Thai, Quang-Thinh Ly, Anh-Minh Phan et al. (+1 more)
Camera-
LiDAR detectors can continue to consume unreliable features even when both sensors remain present, synchronized, and calibrated. We introduce Boundary Feature Repair (BFR), a frozen-host adaptation framework that learns task-supervised residual corrections at modality interfaces the detector already consumes. BFR-C repairs each camera feature level read by fusion, whereas BFR-L aligns host-conditioned
LiDAR candidates to a selected boundary and routes site-wise innovations relative to the frozen anchor. Their jointly trained composition is BFR-CL. Zero-initialized per-channel scales make every variant an exact detector-level identity before optimization; only the repair modules train, while the encoders, fusion consumer, router, detection head, and host normalization statistics remain fixed. At inference, BFR requires neither clean references, corruption metadata, temporal history, nor online updates. Across the complete 20-corruption, five-severity KITTI-C grid, BFR-C reduces RCE from $14.07$ to $11.92$ on MVX-Net and from $14.29$ to $11.00$ on Focals Conv-F relative to their reproduced frozen baselines. On the latter host, BFR-L raises AP$_{\mathrm{cor}}$ from $73.65$ to $74.48$, while BFR-CL reaches $77.01$ AP$_{\mathrm{cor}}$ and $10.46$ RCE with $86.02$ clean AP. On nuScenes-R, BFR-CL raises the reproduced MoME baseline's mAP robustness ratio from $80.1$ to $81.4$. These results establish boundary repair as a targeted retrofit for corrupted-but-present sensing without retraining the deployed detector.
Read the full abstract ▾
All authors Gia-Huy Thai, Quang-Thinh Ly, Anh-Minh Phan, Tuan Dang
Robotics
Spatial Grafting: Grounding 3D Features for Flow-Matching Robot Policies
Dingsheng Liu, Yangzheng Wu, Mahboubeh Asadi et al. (+5 more)
Pretrained
robot manipulation policies such as
vision-
language-action models (VLAs) or world-action models (WAMs) leave interaction-relevant metric geometry implicit. Recent breakthroughs in spatial reconstruction can supply the necessary geometry reliably, but their features describe local shape without stating where it lies with respect to the
robot. How best to deliver these features to a pretrained policy remains unresolved. We propose Spatial Grafting, a versatile, lightweight spatial module that binds frozen reconstruction features to metric,
robot-relative geometry. Spatial Grafting constructs metric-grounded spatial
tokens and injects them into the flow-matching action expert through cross-
attention, without modifying the host's perceptual pathway, so the host retains the full benefit of its pretraining. We evaluate it more broadly than any geometry-aware policy we compare against: one graft architecture, with no per-host redesign, on two VLAs and two WAMs, across four simulation
benchmarks that span short-horizon manipulation, visual robustness, clutter and long-horizon mobile manipulation, and on three real-
robot platforms with single- and dual-arm configurations. On RoboTwin 2.0, a dual-arm manipulation
benchmark, the graft improves every host across VLAs and WAMs. Grafted $\pi_{0.5}$ gains 11.3% and 15.6% on clean and randomized scenes, reaching 94.0% and 92.4%, above the strongest published 3D-conditioned policy, WAM4D (93.8% and 89.9%). The margin widens as the horizon lengthens: on tasks from BEHAVIOR-1K, a dual-arm mobile manipulation challenge scored by average task progress, it surpasses the 2025 challenge winner on five of six tasks,by up to 0.47 Q-score, and exceeds a map-conditioned spatial policy on average across the three tasks both report.
Read the full abstract ▾
All authors Dingsheng Liu, Yangzheng Wu, Mahboubeh Asadi, Zhiyuan Li, Jinbang Huang, Yixin Xiao, Tongtong Cao, Yingxue Zhang
Robotics
Zero-Shot Reactive Obstacle Avoidance for Generative Robot Policies
Weihang Guo and Lydia E. Kavraki
We propose NUDGE (Nudge Update via Differentiable GEometry), a training-free obstacle-avoidance procedure that can be incorporated in any
robot policy based on
diffusion or flow matching, including
diffusion policies and
vision-
language-action models. Our work injects gradients from a signed distance field, a function returning each point's distance to the nearest obstacle, into the policy at inference time to steer it away from obstacles. It supports any common action parameterization, from absolute or relative joint poses to end-effector poses, through a differentiable joint-trajectory decoder. Experiments show that NUDGE preserves the policy's task distribution and runs reactively in real time.
Read the full abstract ▾
Robotics
ReCAT: Remember, Count, and Time: Structured Recurrent Memory for Robot Manipulation
Pankhuri Vanjani, Mostafa Hatab, Can Mizrakli et al. (+4 more)
Memory-dependent manipulation requires
robots to make decisions using information that is no longer available to their current sensors, such as recalling an earlier visual cue, tracking task progress, counting repeated events, or estimating elapsed time. We present ReCAT, a
language-conditioned policy with structured recurrent
memory. An instruction-conditioned encoder forms features from the current observation. A recurrent
memory integrates the observation stream through Mamba-2 layers and one causal
attention layer. A flow-matching
Transformer decoder reads the current and the historical representation through separate cross-
attention in every block. ReCAT reaches 95.3% average success on LIBERO and 62.4% on RMBench, with the best or tied-best result on six of nine tasks. On three real-
robot tasks probing spatial recall, event counting, and interval timing, the best ReCAT variant reaches 66.7% average success, against 8.3% for the strongest short-history baseline. Controlled comparisons within ReCAT show that the observation encoder and every-block
memory conditioning are needed for this performance. They also show that update rules developed for efficient sequence modeling behave differently as
robot memory: additive updates have the highest observed success on counting and timing, and delta-rule updates on spatial recall. Project website is at https://intuitive-
robots.github.io/ReCAT
Read the full abstract ▾
All authors Pankhuri Vanjani, Mostafa Hatab, Can Mizrakli, Vaisakh Shaj, Zhuoyue Li, Moritz Reuss, Rudolf Lioutikov
Robotics · Cross-listed from cs.NI
Edge-Assisted Multi-View Localization for Low-Altitude Economy under GPS-Challenged Environments
Zhengru Fang, Huanhuan Lou, Senkang Hu et al. (+5 more)
Unmanned aerial vehicles (
UAVs) serving the low-altitude economy require reliable localization in urban canyons, indoor facilities, and other GPS-challenged environments. Visual matching with a geo-tagged database provides an alternative source for absolute positioning, but onboard computation and energy limits motivate offloading the database and matching pipeline to an edge server. The resulting localization quality depends on what visual information can reach the edge in time under varying wireless-communication and edge-computing resources. In this paper, we propose a network-adaptive edge-assisted multi-view localization framework that combines scalable orthogonality-regularized variational information bottleneck (O-VIB) encoding, value-of-information (VOI)-guided request control, and value-aware edge scheduling. We design an O-VIB model that supports nested latent prefixes from 8 to 128 dimensions and four
UAV view modes. Each
UAV requests edge assistance when the predicted localization-risk reduction exceeds the communication and service costs. On CARLA multi-view
UAV data, VOI-guided control can lower the mean and 95th-percentile (p95) route errors by 24.8% and 31.0%, respectively, relative to budgeted periodic offloading under a matched per-route traffic budget. In indoor
UAV experiments with motion-capture ground truth, our design can lower the mean position error by 28.0% relative to uncompressed all-view CLIP
retrieval while cutting the descriptor traffic by 98.6%, using a 0.145 KB semantic representation. Under high congestion, a VOI-weighted scheduler with waiting-age and deadline shaping can lower the edge-side p95 latency of the top-10% high-value requests from 137.7 ms to 32.8 ms.
Read the full abstract ▾
All authors Zhengru Fang, Huanhuan Lou, Senkang Hu, Yihang Tao, Zongdian Li, Yiqin Deng, Jingjing Wang, Yuguang Fang
Robotics
QuadHand: A Compact Quadrotor Aerial Manipulator with MRC-SDF-Based Whole-Body Motion Planning
Rui Jin, Ruiyang Liu, Xinhang Xu et al. (+4 more)
Uncrewed aerial manipulators (UAMs) integrate robotic arms with aerial platforms for three-dimensional physical interaction. However, enlarging the workspace increases arm-induced disturbances, while existing
geometric representations face a trade-off between
geometric fidelity and computational efficiency in close-proximity interaction. This paper presents QuadHand, a compact quadrotor aerial manipulator with a 3-DoF arm, gripper, and battery-assisted passive CoG compensation module to reduce dominant arm-induced disturbances. We further propose MRC-SDF, a Multi-articulated
Robot-Centric Signed Distance Field that preserves fine
geometric detail with tractable computation, and a spatiotemporal whole-body trajectory optimization framework that jointly optimizes the quadrotor and manipulator for safe and executable trajectory generation. Simulations and real-world experiments demonstrate safe and executable aerial manipulation in complex environments.
Read the full abstract ▾
All authors Rui Jin, Ruiyang Liu, Xinhang Xu, Haotian Jin, Yi Wang, Yizhuo Yang, Lihua Xie