Robotics · Cross-listed from cs.CV
VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning
Junxiang Xu, Ruisi Wang, Fanyi Pu et al. (+49 more)
Native visual
reasoning treats visual generation as the medium of
reasoning itself: visual states (i.e. images and
videos) are not merely inputs to be understood or outputs to be rendered, but first-class substrates for problem solving beyond
language. Yet progress remains bottlenecked by the lack of scalable training tasks, reliable feedback, and controlled comparisons across generative substrates. In this work, we introduce VBVR-Pro, a closed-loop testbed that makes native visual
reasoning through generation trainable, verifiable, optimizable, and experimentally controllable. 1) Task scaling. VBVR-Pro turns visual
reasoning into a controlled task space of 300 procedurally generated tasks. Models trained on VBVR-Pro show strong transfer beyond the proposed suite across seven external visual
reasoning benchmarks such as RISE-
Video, MME-CoF-Pro, and BabyVision. 2) Verifiable rewards. VBVR-Pro provides verifiable reward scorers for task-grounded evaluation. Through a systematic study of leading MLLMs as judges, we identify recurring failure modes of the prevalent VLM-as-a-judge paradigm. In contrast, the proposed scorers are grounded in deterministic, task-specific rules, achieve fine-grained
alignment with human judgments. Importantly, they serve as reliable reward signals for large-scale multi-task reinforcement learning and demonstrate stronger post-RL performance across visual
reasoning tasks. 3) Mechanism study. VBVR-Pro enables controlled modality studies across more than 30 image,
video, and interleaved generators. Our analysis shows that
video generation remains strongest for tasks requiring persistent spatiotemporal state tracking, while interleaved generation provides a compute-efficient alternative. Critically, ablations and probing suggest the presence of
vision-native trajectories that are crucial to visual
reasoning. We release all data, models, scorers, and code.
Read the full abstract ▾
All authors Junxiang Xu, Ruisi Wang, Fanyi Pu, Maijunxian Wang, Ran Ji, Tongxi Zhou, Chenyang Gu, Jing Zuo, Hongcan Xiao, Yimeng Geng, Wanqi Yin, Wei Chen, Oscar Qian, Zhengan Yan, Ziqi Huang, Haiwen Diao, Liang Pan, Bo Li, Xiangyu Fan, Dezhi Luo, Fengyuan Yu, Zehong Zhao, Qingying Gao, Tinghui Zhu, Yilan Zhang, Jingqi Tong, Pinyuan Feng, Zhengze Jiang, Letian Wang, Ziyu Guo, Renrui Zhang, Jieneng Chen, Sonia Joseph, Constantin Venhoff, Saman Motamed, Mengyue Yang, Chandra Sripada, Alan Yuille, Philip Torr, Lvmin Zhang, Vikash Kumar, Daniel Khashabi, Nikolaus Kriegeskorte, Raphaël Millière, Vincent C. Müller, Anyi Rao, Quan Wang, Ziwei Liu, Dahua Lin, Lei Yang, Hokin Deng, Zhongang Cai
Robotics
Zero-WAM: In-Context World-Action Modeling from Human Videos for Open-Ended Task Generalization
Jiaming Zhou, Qihang Zhang, Gangwei Xu et al. (+9 more)
Zero-shot cross-task generalization, where a policy must execute manipulation tasks never seen during training, remains a central challenge in
robot learning. In large
language models, a novel task can be performed simply by specifying it in the context, without any parameter update. This form of in-context learning (ICL) turns generalization into a problem of task specification. To achieve cross-task generalization, we bring this paradigm to robotic manipulation, and argue that the natural task specification for manipulation is a human
video: unlike
language, it provides rich visual cues about the intended task
evolution. We present Zero-WAM, a causal
video-action model that executes unseen tasks by following in-context human
video guidance. To address the scarcity of task-rich paired human-
robot data, we propose an automatic pipeline that converts task-sampled
robot trajectories into semantically matched human
videos, yielding HumanGen, a
dataset of 74.2K human-
robot ICL pairs across 8.6K tasks. For model training, we further introduce an in-context future chunk prediction (IFP) objective that suppresses shortcuts learned from seen tasks and forces the policy to draw task information from the
video prompt. On seven unseen tasks in RoboTwin 2.0 simulation, Zero-WAM achieves a 47.0% average success rate, an absolute improvement of 29.5 percentage points over the strongest
video-action baseline. In real-world evaluations, it follows human
video guidance to generalize to unseen task configurations involving multi-object scenes, long-horizon manipulation, and fine-grained insertion.
Read the full abstract ▾
All authors Jiaming Zhou, Qihang Zhang, Gangwei Xu, Cunxin Fan, Yujie Zhao, Ruilin Wang, Yiming Luo, Shuai Yang, Xing Zhu, Yujun Shen, Junwei Liang, Yinghao Xu
Robotics
Fast Generative Grasping via Lie Group-Constrained MeanFlow
S. Talha Bukhari, Yi Wei, Ruiqi Ni et al. (+2 more)
Grasp synthesis is a core task in robotic manipulation, for which the solution typically forms a
multimodal distribution rather than a point estimate. Generative robotic grasping aims to learn this distribution with deep generative models such as
diffusion and flow-based approaches. The iterative nature of such generative models makes them flexible and generalizable; however, multi-step sampling impedes the time-critical operation required in robotics. We devise an approach to fast generative grasping based on MeanFlow on the product Lie group $\mathcal{G} = \mathrm{SO}(3) \times \mathbb{R}^3$. The training objective couples a purely algebraic semigroup consistency condition with Riemannian Conditional Flow Matching on $\mathcal{G}$ that anchors the average velocity to the data distribution. The resulting Lie Group-constrained MeanFlow formulation samples reliable grasps in $\leq 5$ network evaluations, matching the grasp generation performance of state-of-the-art
diffusion and flow-based models on the ACRONYM
dataset at millisecond-scale inference latency (up to $39\times$ speed-up). We further demonstrate that the approach directly translates to real-world robotic grasping without additional training or domain adaptation, exhibiting robust grasp synthesis under observation noise.
Read the full abstract ▾
All authors S. Talha Bukhari, Yi Wei, Ruiqi Ni, Zachary Kingston, Aniket Bera
Robotics
Gating Before Commitment: Anticipating Intent Divergence to Prevent Post-Interaction Decision Failures in Autonomous Driving
Cong Xu and Ravi Sankar
Intent misinterpretation during vehicle interactions causes recurring planning failures. We study a decision layer in which a
language-guided intent module reads structured descriptors, computes a smoothed intent-geometry divergence score, and gates the planned maneuver before commitment, upstream of a corridor envelope. On a replayed off-road departure and four crash clips under a frozen, disclosed implementation, gating is the only layer that repairs the plan: on the main case it fires 72 ms after the drift onset but 161 ms before the corridor exit, keeping the trajectory in the corridor in all ten replays. The first calibration draws nine false triggers in 5.9 minutes, each from scoring uncertainty as half a conflict; a preregistered redesign treating uncertainty as abstention cuts this to 0.341 per minute. Two ablations bound the model's contribution: the full score detects fastest on four of five failures under the deployed eligibility, three of five against the unvetoed rule (000871 by one cycle; 000228 by a pre-onset fire on an uncertain stretch that five clips cannot classify as signal or coincidence; dropping the confidence term costs two detections), while on in-domain tracks at equal false positives the
geometric rule more than triples its detection. The evidence supports the gating mechanism; the model's demonstrated roles are the fastest detection on these failures and an uncertainty veto on the
geometric rule.
Read the full abstract ▾
Robotics
VirTooS: A ROS 2 - Unity Virtualization Toolkit for Fleet Management of Autonomous Mobile Robots
Andrea Drudi, Lorenzo Pichierri, Andrea Testa et al. (+1 more)
In this paper, we present VirTooS, a Python/C# toolkit designed to implement fleet-management tasks on teams of Autonomous Mobile
Robots (AMRs). VirTooS leverages the
Robot Operating System (ROS) 2 and Unity game engine to provide realistic, scalable virtual experiments in a mixed-reality environment. The toolbox allows users to easily generate and customize virtual scenarios for realistic simulations. Virtual and real sensors as, e.g.,
LiDARs, can be exploited to map and safely navigate in the mixed-reality environment. To enable distributed robotics experiments, we propose a set of tailored routines leveraging the ChoiRbot framework. As a motivating example, we show a set of experiments for task assignment problems in a virtual environment, allowing seamless interaction among real and virtual
robots. Moreover, the package comes with a containerized suite to easily deploy it on different machines. The source code will be made publicly available on GitHub.
Read the full abstract ▾
All authors Andrea Drudi, Lorenzo Pichierri, Andrea Testa, Giuseppe Notarstefano
Robotics
One Policy, Many Embodiments: Unified Camera-Centric Action Geometry Pre-training for Heterogeneous Embodied Manipulation
Xiaomi Embodied Intelligence Team, University of Macau: Shaoqing Xu, Fang Li et al. (+19 more)
Scaling generalist
vision-
language-action (VLA) policies is severely bottlenecked by the inherent heterogeneity of embodied data, which spans diverse
robot morphologies, camera configurations, and low-level action spaces. Existing paradigms typically address this mismatch through explicit action retargeting, human-to-
robot video synthesis, or
dataset-specific adaptation branches, fundamentally hindering the joint learning of a unified policy. We introduce UCAG-P, a camera-centric unified action formulation that structurally aligns heterogeneous embodied
datasets into a shared
geometric action space. Rather than treating
robot-specific commands as the shared policy target, UCAG-P represents manipulation through camera-observable anchor motion in image and camera-frame coordinates, treating
robot arms, humanoids, and human hands as different embodiments of a common action schema. A geometry-conditioned action translator combines predicted motion with target-embodiment kinematics to produce executable controls. The resulting decoupled architecture allows a shared VLA policy to learn transferable manipulation geometry while retaining embodiment-specific controllability. UCAG-P is trained on 4.03K hours of
robot and simulation data and 2.34K hours of human demonstrations. A single checkpoint reaches 98.3% on LIBERO, 88.7% and 89.2% on RoboTwin Easy and Hard, 82.0% zero-shot on LIBERO-Plus, and 62.0% on RoboCasa GR-1, without
benchmark-specific fine-tuning.
Read the full abstract ▾
All authors Xiaomi Embodied Intelligence Team, University of Macau: Shaoqing Xu, Fang Li, Guozhi Zhan, Zhixiang Duan, Yuhan Wang, Yuechen Luo, Shengyin Jiang, Hanbing Li, Zhiying Du, Longlong Wang, Longmei Jiang, Weixiang Liang, Ying Gong, Yong Pan, Ziping Zhao, Zhiyuan Chen, Yangwei You, Kun Ma, Qinyuan Liu, Hangjun Ye, Zhi-xin Yang
Robotics
$R^3$: Training Robots to Reason in Natural Language via Reinforcement Learning
Lehong Wu, Yuxiao Qu, Zheyuan Hu et al. (+4 more)
Reasoning in
language allows foundation models to spend more test-time compute on hard problems, such as those requiring decomposition, constraint tracking, and prediction of future consequences. Whether this mechanism can improve robotic manipulation remains unclear, where long-horizon tasks require tracking partial progress,
reasoning about object relations, recovering from mistakes, and steering noisy low-level policies. In this paper, we study whether VLMs can be trained to reason directly in natural
language to guide low-level manipulation policies. We introduce $R^3$, a simple post-training recipe that turns off-the-shelf VLMs into robotic reasoners: it first mid-trains a VLM on expert-generated
reasoning traces to initialize the desired
reasoning style, then improves the reasoner with single-step rubric-based RL from offline action data. Unlike prior robotic
reasoning methods that mostly use structured traces as auxiliary supervision, $R^3$ trains free-form
language reasoning to produce test-time guidance for action. We instantiate $R^3$ on
Language Table and simulated bimanual grocery packing, two controlled testbeds for studying robotic
reasoning and long-horizon manipulation. $R^3$ improves exploration and generalization across unseen tasks and significantly outperforms instruction-only imitation learning baselines on both
benchmarks. Our analyses suggest that free-form
language reasoning can function as a test-time compute mechanism for steering low-level policies. Our project page is available at https://robotic-reasoner.github.io/.
Read the full abstract ▾
All authors Lehong Wu, Yuxiao Qu, Zheyuan Hu, Ivan Zhang, Limin Wei, Zackory Erickson, Aviral Kumar
Robotics
When Obstacles Bend: Modeling Vegetation Deformation in the context of Field Robotics
Zaar Khizar, Tom Montagnon, Roland Lenain et al. (+2 more)
Autonomous
robots operating in natural environments must often interact with vegetation rather than simply avoid it. In this context, traversability is typically defined from the
robot's perspective, by measuring how a specific platform responds when moving through the environment. While practical, this viewpoint entangles the assessment of the environment with the
robot's own dynamics, making the resulting characterization difficult to transfer across different platforms. More importantly, it does not directly reflect the properties of the vegetation itself, which are the true source of interaction and potential damage in applications such as agriculture and environmental monitoring. To address this limitation, we propose to characterize vegetation through its intrinsic mechanical properties, independently of any specific
robot. By combining deformation measurements with contact force data, we estimate the underlying mechanical parameters and reconstruct the vegetation's response to interaction. This enables vegetation-aware navigation based on intrinsic environmental properties rather than platform-dependent metrics.
Read the full abstract ▾
All authors Zaar Khizar, Tom Montagnon, Roland Lenain, Romuald Aufrère, Johann Laconte
Robotics
Phantom Navigator: Stealthy and Precise Unmanned Aerial Vehicle Redirection with Real-Time Tracking and GPS Spoofing
Haocheng Meng, Shaocheng Luo, Songqiao Xie et al. (+1 more)
Redirecting unmanned aerial vehicles (
UAVs) from their intended mission trajectories has been an active area of research. However, existing
UAV redirection attacks lack reliability, precision, and covertness for a targeted diversion. They primarily rely on physical capture, communication hijacking, or sensor spoofing. Yet, physical interception is costly, offers only a single opportunity for success, and poses a high risk of collateral damage; network-based attacks demand deep technical expertise and access to encrypted communication channels; and sensor spoofing techniques typically fall short in achieving the accuracy and robustness required to steer a
UAV toward a specified target. Consequently, we propose Phantom Navigator, a
UAV redirection attack to mislead drones to a designated spoofing target, covertly and precisely. Our approach combines offline pre-redirection reachability analysis, which provides high-fidelity estimates of achievable redirect ranges, with an online closed-loop, stealthy execution layer that ensures successful redirection in practice. Based on this approach, we build a physical attack platform equipped with a
LiDAR–camera detection, tracking, and spoofing stack that performs real-time identification, pose estimation, and computation of targeted spoofing signals to covertly and accurately redirect victim
UAVs to a designated location. We demonstrate the effectiveness of our redirection methodology and the attack implementation in real-world case studies.
Read the full abstract ▾
All authors Haocheng Meng, Shaocheng Luo, Songqiao Xie, Miroslav Pajic
Robotics
DESCENT: Directed Edge Scene Encoding for Airport Surface Movement Prediction
Alexander Prutsch, David Schinagl and Horst Possegger
Advanced automation is a key technology for enhancing the safety of ground operations amidst the increasing density of commercial air traffic. While motion forecasting is a well-studied task in autonomous driving, its application to airport surface movements remains underexplored. To enable efficient and accurate prediction in this domain, we propose DESCENT, a
transformer-based architecture designed to handle heterogeneous dynamics and strict topological constraints. Our approach features a Potential Reachable Set (PRS) context sampling mechanism that adaptively collects airfield environment context across diverse operational phases. Combined with a detection
transformer-based decoder, DESCENT generates accurate trajectory forecasts. Extensive evaluations on the Amelia-10
benchmark demonstrate significant performance improvements over state-of-the-art baselines. These gains are especially pronounced in safety-critical scenarios, where our domain-aware sampling provides critical long-horizon context necessary for safe navigation.
Read the full abstract ▾
Robotics
A Statistical Audit of Physical AI Benchmark Redundancy
Zaruhi Navasardyan and Hrant Davtyan
Physical AI models are evaluated on suites of
benchmarks that differ across model reports, leaving the model-by-
benchmark matrix sparse and the relationship between
benchmarks unmeasured. We construct a
matrix of 51 models on 12 physical AI
benchmarks, selected from a registry of 51
benchmarks and 152 models by reporting density, combining scores from model cards and
benchmark papers with our own evaluation runs under each
benchmark's official protocol. We measure how much information the
benchmarks share and show quantitative evidence of Redundancy. Redundancy affects reported rankings: collapsing the two substitute pairs into single columns moves 22 of 51 models by three or more places under an equally weighted average. We then select
benchmarks greedily under a utility combining score dispersion with variance not explained by the already-selected set, and obtain a four-
benchmark subset retaining 78.5% of the utility of all 12, on which we fit a Bradley–Terry ranking. The procedure requires only
benchmark-level scores with sufficient overlap and is not specific to physical AI.
Read the full abstract ▾
Robotics · Cross-listed from cs.AI
Choose Your Game Wisely: Measuring Game-Theoretic Structures in Real-World Vehicle Interactions
Yueyuan Li, Rongcheng Nie, Weijie Xi et al. (+4 more)
Game-theoretic models provide principled frameworks for modeling vehicle interactions, but their underlying temporal assumptions have not been systematically examined against real-world driving behavior. In particular, it remains unclear how simultaneous, sequential, and asymmetric interaction structures can be measured from vehicle trajectories. This paper develops a trajectory-based interaction measurement framework to identify interaction events and quantify behavioral change onset, temporal organization, post-onset response dynamics, and ordering stability. The framework uses behavioral deviations to verify candidate interactions. We evaluate the framework on six real-world trajectory
datasets, including INTERACTION, highD, inD, rounD, Waymo Open Motion, and nuPlan, covering diverse road geometries, traffic environments, and interaction types. The results show that concurrent and sequential behavioral changes both constitute substantial proportions of observed following, merging, and conflicting interactions. Among sequential interactions, stable ordering is more prevalent than alternating ordering, indicating that persistent asymmetric roles are a common interaction structure. Importantly, temporal precedence does not necessarily coincide with a measurable behavioral response, indicating that temporal ordering alone may not be sufficient to characterize behavioral dependence. These findings show that real-world interactions exhibit concurrent, sequential, and persistently ordered temporal structures. Different game-theoretic formulations are therefore better regarded as complementary modeling abstractions for different interaction regimes rather than as a universal structure governing all vehicle interactions.
Read the full abstract ▾
All authors Yueyuan Li, Rongcheng Nie, Weijie Xi, Mingyang Jiang, Songan Zhang, Hanyang Zhuang, Ming Yang
Robotics
Low-Resolution Perception for Robotic Packing
Giuseppe Fabio Preziosa, Federico Vignoni, Chiara Castellano et al. (+3 more)
This work tackles the problem of scalable perception for robotic packing with low-cost, low-resolution depth sensing. We propose a framework where reconstruction cues drive next-view selection and grasp evidence updates a per-object stability estimate, jointly deciding what to acquire next and when to grasp. During the reconstruction, a low-resolution Next Best View (NBV) strategy explicitly avoids redundant views while preserving task-relevant geometry. We validate the approach in two steps: (i) an ablation study of the utility function under very low resolution, and (ii) a full end-to-end evaluation across policies, showing how low-resolution perception is a practical, scalable option for robotic packing.
Read the full abstract ▾
All authors Giuseppe Fabio Preziosa, Federico Vignoni, Chiara Castellano, Marco Faroni, Andrea Maria Zanchettin, Paolo Rocco
Robotics
VISTA: Visually Inferred Spatial ConTact Attention for Contact-Rich Manipulation
Jiayi Chen, Wenlong Dong, Yan Huang et al. (+5 more)
Contact-rich manipulation requires precise interaction feedback. While
vision-centric imitation learning is prevalent, external visual observations provide indirect and ambiguous cues about contact states, particularly under occlusion or subtle object–gripper interactions; dedicated tactile or force sensors can provide rich contact information but introduce additional hardware complexity, calibration requirements, and deployment costs. To bridge this gap, we propose VISTA-Policy, an imitation learning paradigm that utilizes the Visual Deformation Field (VDF), a 3D displacement representation of a compliant gripper, as high-dimensional visuo-physical feedback. The framework integrates: 1) a Physics-Aware Encoding Engine for real-time VDF decoding; 2) an Energy Aggregation Denoising Mechanism to isolate true interaction signals; and 3) a Deformation-Augmented Policy Network with incremental gripper actions for precise closed-loop correction. Extensive evaluations on Cross-Scale Object Grasping, Cap Unscrewing, and Calligraphy Writing demonstrate that VISTA-Policy outperforms the strong pure-
vision baseline 3D
Diffusion Policy and the tactile baseline. VISTA-Policy further demonstrates substantial out-of-distribution generalization to unseen object scales and robustness against dynamic disturbances, offering a durable and cost-effective route toward general-purpose fine-grained manipulation in unstructured environments. Project
videos and supplementary materials are available at: https://sites.google.com/view/vista-policy.
Read the full abstract ▾
All authors Jiayi Chen, Wenlong Dong, Yan Huang, Xianglin Chen, Zijian Lin, Jiaqi Yin, Yushan Liu, Wenbo Ding
Robotics
MA-VLA: Multi-Arm Vision-Language-Action Model for Collaboration and Compositional Generalization
Zaibin Zhang, Junlan Xiao, Zhongbo Zhang et al. (+11 more)
Multi-arm collaboration is becoming a core capability in embodied manipulation. Recent
vision-
language-action (VLA) models integrate perception,
language, and control, but most represent
language as a single global instruction and do not provide an explicit mechanism for assigning and composing arm-specific behaviors. This design limits transfer to collaboration patterns that differ from those observed during training. We present MA-VLA, a unified framework for multi-arm collaboration via atomic action assignment. MA-VLA decomposes cooperative behavior into mid-level atomic
prompts and allocates them to individual arms, enabling explicit subgoal specification and compositional reuse across tasks. To reduce reliance on fixed execution roles, we introduce Arm Shuffle, a training-time permutation of the observation, state, and assigned atomic
prompts for each arm. This permutation enforces role-agnostic instruction following and supports recomposition into unseen coordination patterns, which we term multi-arm compositional generalization. We also construct a
benchmark in which test-time collaboration patterns are absent in training set. Across simulation and real-world evaluations, prior state-of-the-art VLAs largely fail under these unseen collaborations, while MA-VLA consistently succeeds. These results indicate that structured, per-arm atomic action assignment offers a practical route to scalable generalization in multi-arm embodied systems. Code, models, and data are available at https://github.com/zhangzaibin/future-
robots
Read the full abstract ▾
All authors Zaibin Zhang, Junlan Xiao, Zhongbo Zhang, Yifan Wang, Li Kang, Yiran Qin, Changxing Xia, Heng Zhou, Talas Fu, Enshen Zhou, Ruimao Zhang, Zhenfei Yin, Huchuan Lu, Lijun Wang
Robotics
Anytime Global Tensor Motion Planning
Sai Coumar, An T. Le and Zachary Kingston
Global Tensor Motion Planning (GTMP) solves motion planning with batched tensor operations over a layered multipartite graph. We generalize GTMP so that adjacent-layer edges are realized by any black-box local planner (e.g., linear interpolation, splines, sampling-based planning, trajectory optimization, or generative sampling). We provide two anytime policies on top of this generalization: Anytime GTMP with random restarts at a fixed budget, which covers every homotopy class almost surely, and AO-GTMP with informed expansion with growing budgets, which converges to the optimal cost. We prove that a single sampled graph covers every endpoint-fixed homotopy class admitting a \(\delta\)-clear representative of bounded length. We also prove that additional samples per layer reduce the per-layer miss probability exponentially, whereas stronger local planners reduce the required layer count only sublinearly. On manipulation
benchmarks the method matches state-of-the-art performance, and on 2D navigation it returns batches of topologically diverse solutions, while the informed baselines concentrate on one or two classes.
Read the full abstract ▾
Robotics
AGRO-Nav: Autonomous Graph-based Orchard Navigation
Ho Young Yun, Jaemin Yu and Duksu Kim
Orchards form semi-structured environments in which parallel tree rows create natural driving corridors, yet narrow inter-row clearance and dense foliage lead geometry-agnostic grid planners to drift off the row center and risk trunk or canopy contact. We present AGRO-Nav, an automated framework for static graph-based global planning in orchards. From tree-row lines fitted to trunk clusters in a SLAM point cloud, it builds, without any manual waypoints, a sparse topological graph of intra- and inter-row connectivity; a global route is then found by Dijkstra search on this graph, connected to the start and goal by any-angle Theta* segments, and smoothed with a cubic B-spline. In real-orchard trials, AGRO-Nav follows the row center with a mean error of about 0.08 m, far below the A* (0.31 m) and Theta* (0.43 m) shortest-path baselines, while planning roughly four to five times faster. In Isaac Sim, it attains the lowest error among A*, Theta*, and a reproduced RANSAC midline baseline and remains stable as tree density drops to 70%, where the RANSAC baseline degrades. The resulting trajectories—straight row-centered segments joined by controlled turns—suit differential-drive and four-wheel-steering platforms.
Read the full abstract ▾
Robotics
TacForcing: Streaming Action Generation with Execution-Time Tactile Feedback
Jianbo Zhou, Boyuan Zhao, Yuzheng Zhang et al. (+9 more)
Contact-rich manipulation requires adapting to contact states that can evolve substantially within an action horizon. However, chunk-based
vision-
language-action models predict complete action chunks from observations collected before execution, leaving tactile conditioning stale during execution. Existing tactile-reactive approaches typically rely on separate high-frequency controllers, which increase both architectural and training complexity. In this paper, we introduce TacForcing, a streaming action-generation framework incorporating execution-time tactile feedback. TacForcing replaces the standard action expert with a streaming expert that generates action blocks sequentially while preserving intermediate states of unfinished blocks. After each block is executed, the expert resumes generation from these states using newly acquired tactile feedback. To better align tactile conditioning with action execution, we further introduce Execution-Aware Tactile
Attention (EATA), which restricts direct access to each tactile update to the next block scheduled for execution. Across six UniVTAC simulation tasks and six real-world contact-rich manipulation tasks on two
robot platforms, TacForcing achieves average success rates of 65% in simulation and 66% in the real world, outperforming the strongest baselines by 6 and 15 percentage points, respectively.
Read the full abstract ▾
All authors Jianbo Zhou, Boyuan Zhao, Yuzheng Zhang, Yiyang Chen, Wenxin Chen, Qiuyue Li, Xiangyang Gu, Yuhan Cao, Xiao Xia, Yi Yang, Yanzhe Hu, Zhijie Deng
Robotics
LM-X: Explainable Vision–Language–Action Modeling via Progress, Event, and Uncertainty Prediction
Jin Lou, Zhiyuan Jing, Xupeng Wang et al. (+21 more)
Large-scale
vision–
language–action (VLA) policies have advanced generalist
robot control, yet most remain stimulus-to-action black boxes: actions are exposed, but their explanatory state is not. They provide no native account of three explanatory signals: task progress, the next semantic transition, or local command reliability. Prior work shows that progress and event structure aid long-horizon control and that uncertainty supports monitoring; however, such capabilities are typically added or extracted only after action pretraining. The field therefore lacks a VLA foundation model whose explanatory state is jointly pretrained with control. Drawing on biological sensorimotor organization, in which outcome-sensitive, event-segmented, and probabilistic predictions structure behavior, we introduce LM-X. LM-X learns three directly supervised online signals: return-to-go (RTG) estimates visible progress and state quality; event-to-go (ETG) predicts the action sequence to the next semantic event; and heteroscedastic action-flow variance reports local command reliability. RTG conditions ETG and both condition action generation; uncertainty is estimated inside the action expert, making explanation part of control rather than a post-hoc description. We pretrain LM-X on more than 20,000 hours of heterogeneous real-
robot trajectories, including over 1,000 hours of failed rollouts. LM-X achieves 74.1% success on 50 randomized-hard RoboTwin2.0 tasks and 73.5% on seven real-
robot tasks, compared with 55.4% and 50.7% for GR00T N1.7. Its signals track progress and regression, anticipate event-scale motion, detect high-error actions, and provide advance failure warning. These results establish LM-X as an explainable VLA foundation model that couples transparent predictive state with stronger generalist control. Github: https://github.com/loongOpen/LoongWu-LM-X-VLA
Read the full abstract ▾
All authors Jin Lou, Zhiyuan Jing, Xupeng Wang, Andong Chen, Xingdong Zhu, Yuexuan Li, Yuan Xu, Zhijie Zhu, Yingwei Ji, Wenpeng Nie, Renxing Feng, Liangliang Chen, Ying Chu, Jingyi Li, Jinyan Liu, Zhiqi Song, Jingxuan Zhu, Jidong Zhang, Yufei Liu, Boyang Xing, Lei Jiang, Yan Cui, Hongming Li, Yuchen Zhu
Robotics · Cross-listed from cs.MA
Trust-Aware Sequential Decision Making and Rollout Planning for Resilient Multi-Robot Systems
Roee M. Francos, Daniel Garces, Orhan Eren Akgün et al. (+2 more)
Sequential decision-making in multi-
robot systems typically assumes that planning information is reliable and that
agents execute the actions anticipated by the planner. Compromised
agents can violate both assumptions, creating a mismatch between the planning model and physical execution. We study this problem in online multi-
robot routing under localization spoofing. We introduce a distance-constrained spoofing model for monitor-aware adversaries, together with a tiered bipartite matching strategy that maximizes assignment influence while limiting spoofing magnitude. To mitigate such attacks, we develop a trust-aware monitor that combines probabilistic localization trust, calibrated using real GPS spoofing data, with behavioral evidence from task execution to classify
agents and remove detected adversaries from subsequent planning. We further show that undetected adversaries can cause rollout to lose its expected cost-improvement behavior by violating planner-execution consistency. Trust-aware removal restores this consistency after detection, enabling stable routing and recovery of rollout's empirical advantage over the base policy. Experiments using real GPS spoofing
datasets and San Francisco taxicab demand demonstrate effective detection and resilient routing across varying spoofing capabilities,
adversarial fleet sizes, adaptive attacks, monitoring configurations, and rollout horizons.
Read the full abstract ▾
All authors Roee M. Francos, Daniel Garces, Orhan Eren Akgün, Nathaniel D. Bastian, Stephanie Gil
Robotics
Opportunities of Self Supervised Learning for GNSS: Evaluation of a Deep Learning-Enhanced PVT Algorithm
Thomas Barbero and Bertrand Ekambi
This work proposes a Deep Learning Enhanced PVT
algorithm to mitigate multipath interference in dense urban areas. A supervised objective jointly predicts range corrections and uncertainty, while a JEPA-based self-supervised pretraining stage improves representation quality. The
algorithm is evaluated over diverse driving scenarios, substantially improving PVT accuracy, particularly for unseen harsh urban conditions. These results highlight the potential of unlabelled GNSS data to improve generalization performance.
Read the full abstract ▾
Robotics
PRISM: Projection-Integrated Sampling-Based MPC with Bayesian Cost Tuning for Bimanual Manipulation
Alinjar Dan, Iryna Hurova, Karl Kruusamäe et al. (+1 more)
Bimanual manipulation in cluttered, contact-rich environments remains challenging because it requires coordinated motion generation, interaction-aware planning, and reliable execution under tight kinematic constraints. We present PRISM, a projection-integrated sampling-based Model Predictive Control (MPC) framework that uses a GPU-accelerated physics simulator as an online world model for complex dual-arm manipulation. The main algorithmic contribution is a QP-guided control sampling strategy that decouples trajectory exploration from kinematic feasibility. At each MPC step, sampled joint-velocity trajectories are projected onto the set of motions satisfying joint position, velocity, acceleration, and jerk bounds, together with an initial-velocity boundary condition, before rollout evaluation. This enables broad yet feasible exploration of coordinated bimanual behaviors. To support efficient online execution, we derive a custom ADMM/Bregman-splitting QP solver that exploits joint-wise separability and reusable
matrix factorizations. We further use
Bayesian optimization to tune task-cost weights offline, reducing manual parameter selection. We evaluate PRISM on challenging variants of PerAct$^{2}$ tasks, including obstacle-constrained ball transport, tray transport, cube handover, and box lifting. Experiments show improved robustness and task success relative to representative sampling-based baselines, while maintaining real-time or near-real-time execution. We also demonstrate successful sim-to-real transfer on dual UR5e manipulators, highlighting the practical potential of physics-based online planning for contact-rich bimanual manipulation. Project details, including code and supplementary
videos, are available at \href{https://sites.google.com/view/prismbimanual}{https://sites.google.com/view/prismbimanual}.
Read the full abstract ▾
All authors Alinjar Dan, Iryna Hurova, Karl Kruusamäe, Arun Kumar Singh
Robotics
GaussianDream++: Efficient 3D Gaussian World Modeling for Robotic Manipulation
Yuqing Jiang, Zijian Zhang, Weitao Zhou et al. (+8 more)
Vision-
Language-Action (VLA) policies have advanced
language-conditioned robotic manipulation, yet action-imitation objectives provide only weak supervision for metric 3D structure and short-horizon physical
evolution. Geometry-enhanced policies mainly improve current-scene grounding, whereas predictive policies often model future dynamics in RGB or latent spaces and may incur substantial deployment cost. GaussianDream demonstrates that training-time current Gaussian reconstruction and future Gaussian prediction provide effective 3D supervision, but its dense VGGT/TGE-based prefix jointly carries state, dynamics, and action-conditioning information. We present \methodname, a compact, policy-native extension that inserts World State
Tokens and World Prediction
Tokens directly into the VLA backbone. A training-only World Representation Head decodes these
tokens into a Current World and coupled Future Prediction over shared Gaussian primitives, while static–dynamic factorization preserves persistent structure and focuses residual motion on interaction-relevant regions. At inference, the head, renderer, auxiliary objectives, and VGGT/TGE pathway are removed, leaving only 20 world
tokens without online Gaussian decoding or rollout. \method achieves 98.6% on LIBERO and 87.8% on LIBERO-Plus, with clear gains under Camera and Layout shifts. Real-
robot experiments further improve average success from 29.2% to 52.5% over reproduced $\pi_{0.5}$ while maintaining efficient closed-loop control.
Read the full abstract ▾
All authors Yuqing Jiang, Zijian Zhang, Weitao Zhou, Jiawei Wang, Junjie He, Lei Yang, Haifang Qing, Si Liu, Ding Zhao, Ping Luo, Haibao Yu
Robotics
EgoNav: Bridging Learned Waypoints and Geometry-Aware Local Control for Robust Indoor Navigation
Jing Wang, Shiqi Zhao, Hairong Qu et al. (+1 more)
Image-goal navigation using lightweight topological maps is a practical paradigm for indoor
robot deployment: the map requires only geotagged images, and localization relies on visual matching rather than precise pose estimation. However, learned waypoint predictors can produce targets that violate
geometric constraints or deviate from the global path. Executing these waypoints safely further requires a local planner capable of collision avoidance, yet existing systems either lack one or rely on fixed parameters that cannot adapt to confined spaces. To address these limitations while retaining the navigational intuition of the learned predictor, we present EgoNav, a hierarchical system that implements this idea by generating candidates from semantically segmented traversable regions and scoring them alongside the learned waypoint for
geometric safety, directional coherence, and fidelity to the learned prior. An adaptive local path planner then executes the refined waypoint with parameters modulated based on the refinement outcome. Experiments in Habitat-sim and on a physical humanoid
robot show that EgoNav consistently outperforms contemporary baselines in both success rate and path efficiency.
Read the full abstract ▾
All authors Jing Wang, Shiqi Zhao, Hairong Qu, Peng Yin
Robotics
Leveraging Inter-object Affordances for Efficient Planning in Contact-rich Tasks
Pouya P. Niaz, Justus Piater and Alejandro Agostini
Traditional task-and-motion planning (TAMP) approaches primarily focus on defining sequences of actions along with the necessary
geometric and kinematic constraints to execute long-horizon tasks. However, their applicability in real-world settings is limited, as they typically assume simplified object models that overlook key physical properties critical for the successful execution of contact-rich tasks. Moreover, they often use sub-symbolic
reasoning during motion planning, which drastically increases planning time and decreases overall success rates. We propose a method that leverages a TAMP approach, defining object-centric abstractions of execution constraints, called Unified TAMP (U-TAMP), to execute robotic tasks involving interactions among objects with heterogeneous shapes, sizes, and materials. Using a
Vision-
Language Model (VLM), we generate abstractions of inter-object affordances for characterizing physical interaction constraints between objects in contact-rich tasks, such as grasp and support constraints. These constraints are used to enrich the U-TAMP planning domain to deal with objects with variable physical properties. We perform experiments in simulated kitchen table organization scenarios and compare our results with those of the original U-TAMP, as well as a state-of-the-art VLM-based planner that leverages common sense knowledge of objects' affordances for plan generation. Our approach achieves significantly higher planning success rates and improves planning times by one to two orders of magnitude compared to other methods.
Read the full abstract ▾
Robotics
Advantage-Driven Explicit Memory for Social Navigation
Yeonsoo Park, Mattia Racca, Guillaume Bono et al. (+4 more)
Robot policies are predominantly learned with classical parametric variants of imitation learning or RL, where training stores the
agent's behavior exclusively in the policy's network parameters, putting a heavy burden on the representation learning
algorithm. We propose a new navigation
agent equipped with non-parametric
memory which explicitly indexes prior steps leading to critical events. The advantages are twofold: first, it allows the policy to outsource some of its behavior into an explicit
memory; second, it encourages a form of continual learning by allowing an
agent to collect data from its testing episodes during deployment and therefore to better generalize to OOD situations. In the context of social navigation, we show that this improves the
agent's capability to retain sparse, high-cost failures, such as human collisions. If the policy is trained in simulation, this also naturally addresses the sim-to-real gap, partially, by basing some of the decision making on real data. We integrate the explicit
memory into a recurrent PPO architecture and use hidden states for
memory retrieval to capture continuous spatiotemporal dynamics. The goal of exploiting rare, high-impact events is achieved by leveraging the RL
agent's advantage signals. We train our
agent in simulation with a combination of photorealistic rendering and non-visual crowd simulation and show that the
agent is robust with respect to OOD social behavior.
Read the full abstract ▾
All authors Yeonsoo Park, Mattia Racca, Guillaume Bono, Steeven Janny, Gianluca Monaci, Tomi Silander, Christian Wolf
Robotics
RA-VLA: Retrieval-Augmented VLA for Test-Time Adaptation
Sanghwan Jang, Minjin Jeon, Minsoo Kim et al. (+3 more)
Vision-
Language-Action (VLA) models provide a versatile foundation for general robotic manipulation, yet they exhibit significant brittleness when confronted with novel task distributions. While In-Context Imitation Learning (ICIL) offers a training-free alternative, existing frameworks suffer from an adaptation bottleneck that hinders the effective translation of expert context to executable actions. This failure originates from superficial
retrieval mechanisms and an inherent behavioral inertia that anchors the policy to its pre-trained priors. To address these limitations, we present RA-VLA, a
retrieval-augmented VLA framework that integrates behavior-aligned context
retrieval with a grounded execution pipeline. By enforcing faithful adherence to functional cues within a scalable architecture, RA-VLA facilitates seamless task adaptation while preserving inference efficiency. Our empirical evaluations across the LIBERO
benchmark and a real-world UR5e environment demonstrate that RA-VLA achieves superior success rates and computational efficiency, establishing a robust framework for training-free robotic adaptation.
Read the full abstract ▾
All authors Sanghwan Jang, Minjin Jeon, Minsoo Kim, Seongjin Choi, Dongha Kim, Hwanjo Yu
Robotics
ConfAL-WM: Confidence-Guided Active Learning for Action-Conditioned World Models
Xiang Liu, Kunwei Wu, Miao Liu et al. (+2 more)
Action-conditioned world models have become an important foundation for embodied prediction, planning, and synthetic data generation, but their errors under new task and scene distributions are often concentrated in localized spatiotemporal regions such as
robot arms, manipulated objects, contact areas, and occluded objects. This paper presents ConfAL-WM, a confidence-guided active learning framework for post-training embodied world models. Building upon EnerVerse-AC (EVAC), we attach a lightweight confidence probe to UNet decoder features and predict dense confidence maps in the latent space. These maps are aggregated into task-, frame-, and patch-level scores, enabling data-budget allocation and localized training enhancement. Our pipeline trains the probe and warms up EVAC on a small target-domain subset. EVAC-v1 then supplies task-level acquisition and optional frame/patch weighting signals; all selected-data models are initialized from the original pretrained EVAC checkpoint for retraining. Experiments on RoboTwin2.0 at the default 40% data budget show that confidence-guided selection improves post-training quality, while dense frame and patch weighting offers complementary reconstruction and semantic gains compared with scalar reward, progress, and judge-based scoring baselines. A quick visual overview of this work is available at https://ConfAL-WM.github.io.
Read the full abstract ▾
All authors Xiang Liu, Kunwei Wu, Miao Liu, Sen Cui, Changshui Zhang
Robotics
A Tendon-Driven Five-Fingered Hand with Distributed Tactile Perception for Dexterous Manipulation
Huayang Chen and Longhui Qin
To apply the techniques of embodied artificial intelligence to human-oid
robots for complex manipulations, dexterous robotic hands are indispensable, which are restricted by the dexterity and tactile perception capability. In this work, we proposed a novel design of tendon-driven five-fingered hand with dis-tributed tactile perception. With a soft-rigid-hybrid structure employed, both compliance and operational force are endowed to the hand. Dual-modality tactile sensing elements are distributed on the distal and middle phalanges of all five fingers, enabling the simultaneous detection of static contact and dynamic force variations. Manipulation experiments, including counting gestures, finger-to-thumb pinching, object grasping, and bottle-grasp tactile recording, demonstrate the feasibility of the integrated actuation-perception system.
Read the full abstract ▾
Robotics
Dynamic Modeling of a Welding Torch Umbilical and Its Impact on Robot Dynamics
Nicolas Gautier, Yves Guillermit, Mathieu Porez et al. (+2 more)
Robotic welding is widely used in industrial manufacturing, where the welding torch is often connected to the generator through an external umbilical. With the increasing deployment of lightweight and collaborative
robots, the dynamic influence of this umbilical can significantly affect the
robot motion and the actuation forces. This paper proposes a constrained multibody dynamic model of a welding umbilical, represented as a serial chain of rigid bodies interconnected by passive joints with elastic and dissipative effects. Prescribed motions at the distal anchor point are introduced through holonomic kinematic constraints. The equations of motion are reduced by projecting the dynamics onto the subspace of admissible velocities, yielding an efficient formulation free of Lagrange multipliers. The reaction wrench exerted by the umbilical on the
robot is explicitly recovered. A planar case study illustrates the approach.
Read the full abstract ▾
All authors Nicolas Gautier, Yves Guillermit, Mathieu Porez, Fabien Rousset, Damien Chablat