Robotics
DreamWAM: Beyond RGB Future Prediction for World Action Models
Shanglin Yuan, Weiheng Zhao, Xin Shi et al. (+6 more)
World Action Models (WAMs) learn action-relevant representations by predicting how the observed world will evolve. Most existing WAMs define this future in RGB space, where task-relevant state transitions are entangled with nuisance variations in texture, illumination, background, and viewpoint. We argue that WAMs should explicitly predict action-relevant future state rather than relying on RGB prediction alone. We introduce DreamWAM, which reformulates future prediction as structured world modeling beyond RGB, representing future states through complementary views of appearance, motion, geometry, and semantics. During training, DreamWAM combines joint latent denoising of RGB and motion with lightweight gated residual branches for geometry and semantics. Shared
attention between VideoDiT and ActionDiT allows the action branch to learn from these future-state predictions, while all beyond-RGB supervision branches are disabled at inference and deployment remains RGB-only. Across both no-rollout and joint
video-action inference, DreamWAM consistently improves the matched RGB-only baselines on LIBERO, from 97.30% to 98.40% and from 98.00% to 98.90%, respectively. The gains become larger under unseen LIBERO-Plus perturbations, from 51.36% to 63.44% and from 69.16% to 75.47%. The same robustness extends to real-world manipulation, where DreamWAM attains an average success rate of 74.4% across unseen changes in lighting, background, and object layout, compared with 55.6% for Fast-WAM-Joint. These results show that robust world-action learning depends not only on predicting the future, but on representing it in a form that matters for action. The code and models are publicly released at https://github.com/hustvl/DreamWAM.
Read the full abstract ▾
All authors Shanglin Yuan, Weiheng Zhao, Xin Shi, Haoyi Jiang, Xianda Guo, Liu Liu, Wenyu Liu, Wei Sui, Xinggang Wang
Robotics
Mimir: A Neuro-Symbolic Memory System with Dynamic Grounding for Embodied Agents in Interactive Environments
Haoming Xu, Zhenlin He, Hengyi Wang et al. (+2 more)
Long-horizon embodied task requires
agents to act under partial observability while preserving both scene belief and execution progress. Flat histories or implicit policy states may contain past observations, but they do not provide an explicit interface for deciding which world facts support the currently active goal. We introduce Mimir, a neuro-symbolic
memory that separates world
memory from task
memory and dynamically grounds them before each action. World
memory maintains object locations, object states, and perceptual evidence, while task
memory maintains an ordered goal agenda, progress state, hand state, failures, and execution constraints. A grounding module binds the active goal to recalled world candidates, fills missing source locations, and attaches evidence before planning and embodiment-specific execution. Across tested backbones, Mimir consistently improves on different EB-ALFRED and EB-Habitat tasks, with maximum gains of 42.5% and average gains of 23.0%, respectively. Compared with the best results among prior
agent and
memory systems evaluated under the same backbone, Mimir improves the overall average success rate by 8.5%. Finally, on the EB-Habitat Long-horizon subset, Mimir achieves 86.0% success rate, substantially outperforming current closed-source models. Our code will be released soon.
Read the full abstract ▾
All authors Haoming Xu, Zhenlin He, Hengyi Wang, Jiafeng Xu, Hao Dong
Robotics
PRIMAL3: Pathfinding via Reinforcement and Imitation Multi-Agent Learning - Leveraging LaCAM3
Chengyang He, Tanishq Duhan, Gadiel Sznaier Camps et al. (+6 more)
We present PRIMAL3, an ultra-large-scale learning-based framework for multi-
agent pathfinding (MAPF) that integrates reinforcement learning,
topology-aware communication, LaCAM3-guided training, and PIBT-based action refinement. PRIMAL3 targets failures at topologically critical states, where
agents must coordinate decisively around bottlenecks, dead ends, and persistent conflicts. Each
agent is represented using features derived from cut vertices, dead-end regions, shortest-path distances, and blocking estimates. Two complementary graphs capture
agent interactions: a same-direction following graph propagates multihop context along compatible paths, while a different-direction conflict graph differentiates
agents competing for shared space through masked
attention and relative features. During training, we propose to let policy
entropy identify uncertain
agents, for which LaCAM3 provides confidence-triggered action interventions and label-smoothed imitation targets. During execution, a priority-aware PIBT module refines the proposed joint actions using persistent, learned, and distance-aware priorities together with policy-aware fallback preferences while maintaining collision-free execution. The resulting framework combines learned exploration with structured expert guidance without requiring LaCAM3 at inference. Experiments demonstrate that PRIMAL3 substantially outperforms state-of-the-art learning-based baselines and scales to ultra-large instances with up to city-level 100,000
agents. Real-world experiments further demonstrate the feasibility of deploying PRIMAL3 on physical robotic systems and ablation studies validate the individual contributions the components we proposed. Project page: https://marmotlab.github.io/PRIMAL3/
Read the full abstract ▾
All authors Chengyang He, Tanishq Duhan, Gadiel Sznaier Camps, Fangyuan Wang, Yuhong Cao, Jiankai Sun, Ge Sun, Mac Schwager, Guillaume Sartoretti
Robotics
RORA: Realistic Object Reconstruction with Articulation
Hyesung Lee, Youngseon Lee, Kyutae Lee et al. (+2 more)
Replicating real-world environments into simulation by realistic visual representation like NeRF and 3D Gaussian Splatting (3DGS) has emerged as an effective strategy to reduce the sim-to-real gap in
robot learning. However, implementing object articulation during the real-to-sim process is still a challenging task. Existing motion tracking or learning based articulation methods shows low success rates on complex kinematic structures having multiple joints. Furthermore, those methods require scan of dynamic motion of objects, which makes reconstruction process much complicated. In this work, we propose the first end-to-end pipeline that reconstructs simulation-ready assets with accurate articulation from a single static object
video input through suggestion based human-in-the-loop process. Our approach exports a hybrid representation combining 3DGS for photorealistic rendering and mesh-based geometry for physical interaction. In the reconstruction process, our pipeline performs convex decomposition followed by user grouping for intuitive part segmentation, subsequently binding 3D Gaussians to the corresponding mesh parts. An Automatic Joint Suggestion
Algorithm then calculates candidate joint axes from local boundary geometries and presents them to users for efficient articulated asset reconstruction. We have shown that our method achieves precise articulation results on partnet-mobility-v0
dataset and real objects. Additionally we presented a potential usage of our framework on
robot learning, deploying the reconstructed assets in Unreal Engine and NVIDIA Isaac Sim, demonstrating real-time dexterous hand manipulation tasks.
Read the full abstract ▾
All authors Hyesung Lee, Youngseon Lee, Kyutae Lee, Dongjun Lee, Yongseok Lee
Robotics
Deliberate Before You Fly: Vision-Guided Spatial Deliberation for UAV See-and-Reach Navigation
Fanfu Xue, En Yu, Bohang Liu et al. (+4 more)
UAV see-and-reach navigation requires an aerial
agent to approach a
language-specified target visible in its initial view and stop reliably near it. Existing methods typically map
vision-
language representations directly to action outputs without explicitly modeling intermediate fine-grained spatial decisions. This direct mapping causes semantic-control misalignment, leading to inconsistent maneuvers and unreliable termination. To address this issue, we propose DBFly, a
vision-
language waypoint prediction framework that introduces explicit
vision-guided spatial deliberation before waypoint generation. Specifically, DBFly introduces a spatial maneuver decision chain that progressively performs target-direction anchoring, spatial diagnosis, and maneuver decision, enabling high-level maneuver intent to explicitly guide continuous waypoint generation. DBFly further constructs an implicit flight corridor by transforming the initial target-direction prior into a persistent
geometric reference and deriving an online corridor state from the
UAV's current position, thereby providing soft
geometric guidance for spatial diagnosis and maneuver correction. In addition, DBFly develops a terminal-convergence-aware stopping strategy that characterizes terminal states through both target proximity and short-horizon motion convergence, enabling more reliable stopping near the target. Extensive experiments across seen, unseen-object, and unseen-scene test sets demonstrate that DBFly improves the success rate over the SOTA baseline by an average of 25.07 percentage points. The project homepage is available at https://xuefanfu.github.io/DBFly-Page.
Read the full abstract ▾
All authors Fanfu Xue, En Yu, Bohang Liu, Hongjun Wang, Yang Yang, Xindi Wang, Jiande Sun
Robotics
From Transparent Labware Segmentation to Collision Avoidance: A Real-Time Edge-Aware Perception Pipeline
Shijun Ding, Chen Qian, Weiwei Shang et al. (+1 more)
This paper presents an edge-aware instance segmentation framework that enables real-time robotic collision avoidance with transparent laboratory glassware using purely visual perception. Transparent vessels defy conventional segmentation due to refraction, specular reflection, and the absence of stable interior texture, yet their boundary contours remain comparatively reliable visual cues. Exploiting this observation, we augment a one-stage real-time instance segmentation backbone with a lightweight edge-detection branch, edge-guided
attention fusion, and a parameter-free SimAM module, and further construct LabGlass-IS, a 3485-image, 21-category instance segmentation
dataset of real laboratory glassware. The enhanced model achieves the highest Boundary F-score of 97.80 among compared methods, outperforming the YOLO-prompted FastSAM framework by 18.93 BF points. Furthermore, it maintains an inference speed of 7.1ms per frame and requires only 2.85% of the parameters of the closest accuracy competitor. Multi-view triangulation of mask centroids further provides 3D positions for conservative bounding-volume collision constraints. Real-
robot trials achieve a 93.3% collision avoidance success rate, indicating the feasibility of the proposed perception-to-action pipeline for
robot collision avoidance among fragile transparent objects. Our code is available at https://github.com/havishamy/TransYOLO_3D. Our
video is available at https://havishamy.github.io/paper-
videos/.
Read the full abstract ▾
All authors Shijun Ding, Chen Qian, Weiwei Shang, Junlin Xiong
Robotics
Houze Xu, Jizhong Li and Ziyi Ye
Vision-
language-action (VLA) models provide a unified paradigm for connecting visual perception,
language understanding, and robotic control. However, existing VLA models still face major challenges in long-horizon tasks: sparse expert demonstrations constrain cross-task compositional generalization; the non-Markovian nature of long-horizon tasks makes it difficult for policies conditioned only on current observations to maintain temporal consistency; limited closed-loop error correction allows execution errors to accumulate; and end-to-end action fine-tuning may weaken the high-level semantic representations of
vision-
language model (VLM) backbones. To address these issues, we propose a hierarchical long-horizon VLA architecture with an explicit
language-
memory module. The central idea is to convert discrete temporal observations into a coherent textual
memory sequence with temporal logic. The system is decoupled into a high-level VLM and a low-level VLA: the high-level VLM performs semantic
reasoning through a visual question answering training paradigm, while the low-level VLA executes precise continuous control conditioned on subtask instructions and visual observations. The high-level VLM recursively updates both
language memory and subtask instructions using the previous
memory as a contextual anchor, enabling persistent temporal tracking and dynamic correction during long-horizon execution. We evaluate the proposed method in multiple simulation environments and conduct sim-to-real experiments on a real robotic platform. The results demonstrate that explicit
language memory improves the success rate and robustness of VLA models on complex long-horizon tasks while providing an interpretable semantic account of the decision process.
Read the full abstract ▾
Robotics · Cross-listed from eess.SY
Toward Integrating Adaptive Experience Replay and Online Uncertainty Estimation in Safe Actor-Critic Optimal Control
Mahshad Rastegarmoghaddam, Davoud Nikkhouy and Shima Samadzadeh
Safe actor-critic control often treats barrier filtering, uncertainty estimation, and experience replay as separate modules, even though each changes the data used for learning and control. We develop an integrated architecture in which the uncertainty estimate updates the obstacle geometry used by a control barrier function, filter interventions and estimation residuals determine replay priority, and the critic learns from the executed rather than nominal action. We instantiate the architecture on a two-dimensional
robot-navigation task with corrupted obstacle measurements and compare six component-matched configurations under common training budgets, random seeds, sensor streams, exploration, and disturbances. Evaluation includes a moderate post-training test, an eleven-level perception-noise sweep, and an exploratory extreme-stress test at multiplier $6.0$. In the extreme test, the integrated configuration recorded no contacts and reached the goal in all five evaluation seeds. Its mean cost was $7.63\pm0.44$ and its obstacle-belief root-mean-square error was $3.52\pm0.55$ cm. The uncertainty-estimation ablation also recorded no contacts but reached the goal in four of five seeds, with mean cost $8.96\pm2.08$ and belief error $11.08\pm1.23$ cm. A finite-training bound clarifies replay exposure, and a robust barrier condition states the required estimation-error and feasibility assumptions. The results support coupling estimation, safety filtering, and replay on this
benchmark; broader safety and convergence claims require further study.
Read the full abstract ▾
Robotics
A GitOps-Driven Annotation Catalog for Fully Automatic Railway Operations
Martin Köppel, Tobias Cronauer, Zekiye Ilknur-Öz et al. (+3 more)
Automatic train operation (ATO) at grade of automation 3 and above (GoA3-GoA4) requires robust AI-based perception systems capable of reliably detecting obstacles and railway-specific objects under real-world conditions. The effectiveness of these modern artificial intelligence approaches depends heavily on large-scale, high-quality, and highly dynamic annotated
datasets. However, managing metadata, maintaining provenance, and tracking the iterative
evolution of these annotations impose significant infrastructural and regulatory requirements. Existing monolithic data catalogs often suffer from massive operational overhead, poor integration into developer workflows, and severe documentation drift. This paper introduces an innovative, lightweight GitOps-based architecture for metadata management. By leveraging Data-as-Code principles, Continuous Integration/Continuous Deployment (CI/CD) pipelines, and Static Site Generation (SSG), the proposed approach establishes a seamless, developer-centric workflow. This ensures an traceability, enforces strict regulatory compliance, and automatically generates a highly performant
dataset overview.
Read the full abstract ▾
All authors Martin Köppel, Tobias Cronauer, Zekiye Ilknur-Öz, Sebastian Dubiel, Patrick Naumann, Philipp Neumaier
Robotics
A Vision-based Control Framework for Real-time Autonomous UUV Operations
Erik Tjærand Frøland, Marco Job, Md Ether Deowan et al. (+1 more)
This paper presents a fully integrated
vision-based framework for real-time and robust localization, autonomous navigation, and mapping for unmanned underwater vehicles (UUVs) in dynamic, visually challenging environments. The proposed pipeline enables both net-relative and global localization while generating continuous 3D maps of the surroundings in real-time. The framework was validated on synthetic
datasets with ground truth and tested onboard an UUV during autonomous net-relative navigation experiments. Results demonstrate real-time performance and enhanced robustness, supporting
vision-driven autonomous navigation and enabling the field deployment of marine
robots for critical inspection and mapping tasks in complex underwater environments.
Read the full abstract ▾
All authors Erik Tjærand Frøland, Marco Job, Md Ether Deowan, Eleni Kelasidi
Robotics
Enabling Urgency-aware Robot Swarm Intralogistics using Smart IoT Tags
Youssef Alboraei, Murray Groves, Shane Wen et al. (+6 more)
Warehouse items differ in how urgently they must be moved: perishable goods, pharmaceutical shipments, and just-in-time production materials must be delivered sooner than the rest of the stock. Decentralised
robot swarms suit warehouses that cannot justify fixed automation infrastructure, but current swarm controllers treat all items alike or rely on an external scheduler to set priorities, so urgent items wait as long as ordinary ones. This paper presents a swarm logistics system in which each warehouse carrier holds an ultra-low-power Internet-of-Things (IoT) tag that broadcasts the urgency of its item over Bluetooth Low Energy (BLE).
Robots read these broadcasts directly and weigh urgency against travel distance when choosing which carrier to serve, so prioritisation happens at the item level without central scheduling. The system is evaluated in simulation and validated on real
robots and IoT-tagged carriers against a proximity-only baseline. In the physical trials, priority
alignment (i.e. proportion of urgent items served first), improved from 0.41 to 0.64, with a nonsignificant trend toward lower 95th-percentile (P95) delivery latency and throughput within 1.2% of the baseline. In simulation, the benefit grew with system size: across three larger configurations, P95 latency fell by 5.2% to 11.8% and priority
alignment improved by 41.7% to 51.6%. Attaching urgency to the items themselves therefore allows a decentralised swarm to serve time-critical stock sooner while keeping the low infrastructure requirements that make swarm systems attractive for warehouse automation.
Read the full abstract ▾
All authors Youssef Alboraei, Murray Groves, Shane Wen, Wenda Zhao, Senhui Qiu, Mohammud J. Bocus, Robert Piechocki, Sabine Hauert, Kerstin Eder
Robotics · Cross-listed from cs.CV
A Multi-Sensor Dataset for Monitoring the Operational Environment of Rail Vehicles
Claudio Diotallevi, Rodrigo Gudiño, Zaharia Pachalieva et al. (+5 more)
Reliable environment monitoring is essential for the safe and efficient operation of automated railway systems, covering all Grades of Automation (GoA), from partially automated (GoA2) to fully automated operation (GoA4). Artificial Intelligence (AI) plays a central role in enabling these systems to detect, classify, and react to potential hazards in real time. The development of such AI-based perception systems requires large volumes of accurately annotated data for training and validation. Within the Digitale Schiene Deutschland (DSD) program, DB InfraGO AG and understandAI GmbH have developed a comprehensive multi- sensor
dataset tailored to the needs of railway environment perception. This
dataset contains over 7 million high-quality annotations of both railway-specific and general perception objects, captured under varying operational scenarios. The finalized
dataset can now be requested at the DB InfraGO AG and serve as a valuable resource for advancing AI-driven environment monitoring in the railway domain.
Read the full abstract ▾
All authors Claudio Diotallevi, Rodrigo Gudiño, Zaharia Pachalieva, Philipp Neumaier, Patrick Naumann, Erik Bochinski, Volker Eiselein, Martin Köppel
Robotics
Suppression Sticks, Locality Is Fragile: A Closed-Loop Target-and-Control Audit of Task-Vector Negation in VLA Policies
Shaoguang Wang, Weiyu Guo, Rushi Dai et al. (+3 more)
Task-vector arithmetic offers a closed-form way to modify a model, yet its behavioral locality remains unclear in closed-loop
robot control. We present a target-and-control audit of per-skill task-vector subtraction from multitask
vision-
language-action (VLA) policies. Across all ten LIBERO-Goal skills, subtraction produces three qualitatively different regimes: target-control separation for five skills, resistance for three, and global collapse for two. On held-out initial states, the five suppressible targets remain at 0% success; however, mean baseline-normalized control retention is only 52%, and each target-suppressing edit materially harms at least one nominally unrelated control. Additional Goal panels show separation across tested policies with continuous-regression, discrete-
token, and flow-matching action heads, whereas we observe no clean separation on Spatial and control collapse on the tested Object and Long-horizon panels. Mean task-vector cosine does not account for this variation. A matched-norm control identifies a local sign asymmetry around one Goal anchor, while multi-vector outcomes vary with anchor and scale. Retain-aware gradient baselines provide data-dependent comparators but require removal-time data and optimization; subtraction is data- and gradient-free only at edit time, assuming precomputed expert deltas. Finally, a single-skill relearning probe is consistent with behavioral masking, not certified unlearning. These results characterize task-vector subtraction as a fast but brittle intervention and underscore the need for closed-loop target-and-control evaluation when assessing locality in embodied model editing.
Read the full abstract ▾
All authors Shaoguang Wang, Weiyu Guo, Rushi Dai, Yiren Zhao, Yandong Guo, Hui Xiong
Robotics · Cross-listed from cs.CV
Differential 6-DOF Pose Estimation with Provable First-Order Immunity to Camera Calibration Errors
Yueqiang Zhang, Liang Deng, Yi Zhang et al. (+5 more)
Accurate six-degree-of-freedom (6-DOF) motion estimation is essential for robotic manipulation, autonomous systems, and structural displacement monitoring. Conventional 3D-2D methods estimate absolute camera poses independently at each time and recover platform motion through camera-to-platform extrinsics, making them sensitive to extrinsic calibration errors, especially for micromotion. We present a differential pose estimation method that directly recovers platform motion from inter-frame image displacements and known 3D control points. By differencing perspective projection equations, using a depth-invariance approximation, and modeling motion on SE(3), the method avoids independent absolute-pose estimation and supports both monocular and multi-camera systems. We prove that translational extrinsic errors cancel exactly, while rotational errors induce a bounded perturbation determined by calibration error, motion magnitude, and observation geometry. We also derive generic observability conditions, a Cramer-Rao lower bound, and a bias-eliminated consistent
estimator, and characterize the validity limits of the approximations. Extensive synthetic and real-world experiments establish a new state of the art for 6-DOF platform micromotion estimation, outperforming representative PnP and generalized-PnP methods in accuracy, calibration robustness, and computational efficiency. With five control points and 0.5-pixel image noise, the monocular solver obtains a combined pitch-yaw rotation RMSE of 10.09 arcsec, a translation RMSE of 3.70 mm, and a runtime of 0.34 ms. The binocular solver achieves a rotation RMSE of 10.58 arcsec, a translation RMSE of 3.91 mm, and a runtime of 0.27 ms. Code will be released upon publication at https://github.com/zyoungszu/pami2026.
Read the full abstract ▾
All authors Yueqiang Zhang, Liang Deng, Yi Zhang, Baoqiong Wang, Wenjun Chen, Shuixin Pan, Yulan Guo, Qifeng Yu
Robotics · Cross-listed from cs.CV
Overcoming Statistical Bias in Action-Controllable World Models
Yuhong Shi, Zhenhao Chu, Jie Wei et al. (+3 more)
Action-conditioned world models aim to predict how visual environments evolve under an
agent's actions. Yet future frames are often highly predictable from visual inertia and recurring motion patterns alone. This creates a shortcut: models can fit the data by exploiting statistical biases without making their visible dynamics meaningfully depend on the action. As a result, different actions may produce similar futures, while motion may persist even under zero action. The key question is how to reduce reliance on statistical shortcuts from dominating action-conditioned prediction. We argue that action control requires more than injecting action features; it requires enforcing consistency under counterfactual changes to actions and observations. Based on this insight, we introduce CoCo, a Counterfactual Consistency framework to enhance action controllability through two complementary constraints. Multi-step counterfactual consistency constrains reference, inverse-action, and zero-action rollouts, while action-spatial counterfactual consistency enforces consistent predictions under mirrored scenes and transformed actions. Together, they reduce reliance on statistical shortcuts from substituting for action-dependent dynamics. We further introduce Action Response Consistency (ARC) and Drift Energy (DE) to assess action controllability, together with Mini-SSMB for same-state, multi-action counterfactual evaluation. On Mini-SSMB, our full model achieved ARC_inv of 0.412 and ARC_ref of 0.483, while reducing DE by 17.07% relative to the baseline. On VP2 visual planning, it achieves the highest average success rate among SOTA models, at 73.1%. Experiments on BAIR and RoboNet further show that these gains preserve
video prediction quality and transfer across model settings.
Read the full abstract ▾
All authors Yuhong Shi, Zhenhao Chu, Jie Wei, Jun Hao, Jianyi Liu, Jingwen Fu
Robotics
Mind-VLA: Instruction-Aware Spatial Representation Alignment for Vision-Language-Action Models
Xingyu Ding, Yuzhong Zhao, Yang Wu et al. (+4 more)
Recent
Vision-
Language-Action (VLA) methods improve generalization by aligning their representations with 3D scene geometry. However, these methods are fundamentally instruction-agnostic: the representations align the entire scene uniformly, neglecting the 3D geometry of the specific target object designated by the
language instruction. This causes failures on fine-grained manipulation and target occlusion tasks, where success depends on accurate 3D understanding of the target object rather than the entire scene. To address this, we present Mind-VLA, an instruction-aware spatial representation
alignment method for VLA models. Specifically, Mind-VLA first obtains the target object specified by the
language instruction, then prepares its canonical target views and extracts the corresponding VAE and VGGT features. Finally, the latent representation of the VLA model is aligned with these features to enable instruction-aware 3D understanding. Mind-VLA reaches 94.4% on LIBERO and 4.47 on CALVIN with a compact 345M-parameter backbone. On real-
robot tasks with target occlusion, Mind-VLA reaches 54% average success, outperforming the matched scene-VGGT control by 26 percentage points.
Read the full abstract ▾
All authors Xingyu Ding, Yuzhong Zhao, Yang Wu, Chunhai Zhao, Chaoyang Zhao, Yifan Zhang, Jian Cheng
Robotics
GASP: GPU-Accelerated Safe Planner for Real-Time Collision-Aware Motion Generation with Latent Trajectory Sampling
Colin Merk, Stefanos Charalambous, Peter Dürr et al. (+1 more)
We present GASP, a GPU-Accelerated Safe Planner for real-time, collision-aware joint-space motion generation in known environments. GASP combines a clamped B-spline trajectory parameterization with a convolutional residual neural network that predicts the free interior control points, while analytically inserted boundary control points enforce initial and final derivative constraints for collision-aware planning under non-stationary conditions. A conditional variational autoencoder samples multiple trajectory candidates, which are decoded and validated in parallel on the GPU, yielding a batched planner for collision-aware coupled joint-space motion with near-millisecond inference. We validate GASP as an online motion-generation module, where it achieves analytical-level success rates with high collision-aware feasibility and substantially reduces inference time relative to GPU-based trajectory optimization. We further deploy GASP as a reinforcement-learning reset planner in competitive robotic table tennis, matching the baseline return rate while roughly halving training-time collisions.
Read the full abstract ▾
All authors Colin Merk, Stefanos Charalambous, Peter Dürr, Farshad Khadivar
Robotics
Static Timing Orchestration for Tree-Structured Robot Control Firmware
Wang Xi, Feiran Wei, Mo Deng et al. (+3 more)
As robotic systems become increasingly complex, generating control firmware from structural description files has emerged as a promising paradigm for reducing development complexity and improving maintainability. Existing
robot description formats naturally represent robotic systems as hierarchical tree structures, where devices are recursively composed into functional subsystems and eventually into the complete
robot. However, such tree-structured organization also introduces structured data dependencies that affect perception-to-decision latency and, consequently, control performance. In this paper, we propose FineMote, a control firmware generation framework with a scheduling mechanism tailored for tree-structured device models. The framework objectifies heterogeneous low-level control logic and exposes unified scheduling units and execution entry points. Based on the resulting object hierarchy, the scheduling mechanism exploits compile-time information to statically determine execution order with minimal runtime overhead. We prove that the proposed mechanism satisfies deadline and precedence constraints, and further derive an upper bound on intra-tree decision latency. We implement the proposed framework and evaluate it on real robotic control platforms. The experimental results show improved timing behavior and runtime responsiveness, demonstrating the practical effectiveness of the proposed design.
Read the full abstract ▾
All authors Wang Xi, Feiran Wei, Mo Deng, Weiheng Lin, Pangkit Fong, Jianping He
Robotics
Retrieve in Time, Correct in Frequency
Yuze Fan, Yue Cao, Pengjie Gao et al. (+7 more)
Frozen
vision-
language-action (VLA) policies generate temporally extended action chunks, but long-horizon manipulation remains vulnerable to accumulated execution error and visual aliasing across task stages. Successful rollouts provide useful corrective evidence, yet current frame
retrieval can return progress-misaligned actions,while direct replay or time-domain fusion can overwrite the reactive structure of the policy proposal. We introduce Retrieve in Time, Correct in Frequency (RTCF), a training-free test-time correction framework that improves frozen VLA performance with low model-side overhead.RTCF separates which experience to retrieve from which part of its action to transfer. Progressive
Memory Alignment (PMA) causally aligns the growing visual execution history with complete successful trajectories through incrementally updated monotonic frontiers, jointly identifying a relevant
memory and the current aligned
memory position without stage labels. From the aligned action chunk,RTCF transfers a coefficient-wise-clipped low-frequency residual on motion channels. Higher-frequency components and gripper decisions remain inherited from the frozen policy. Across four LIBERO suites and 2,000 episodes per condition, RTCF raises aggregate success from 86.4% to 88.4% and improves LIBERO-Long from 61.6% to 68.6%.These gains require no parameter updates, repeated VLA inference, or additional GPU resources: correction can be performed on the client CPU after a single policy invocation, and the median latencies sum to only 10.99 ms per action chunk
Read the full abstract ▾
All authors Yuze Fan, Yue Cao, Pengjie Gao, Haojia Gao, Guangqiu Guo, Ziyue Zhang, Junbo Tan, Bokui Chen, Zhuo Zou, Xueqian Wang
Robotics
GUARD: Grounding Uncertainty and Ablation-Based Risk Detection for Diffusion-Based VLAs
Suhas Hegde and Jitendra Yasaswi Bharadwaj Katta
Diffusion-based
vision-
language-action (VLA) policies can generate plausible actions even when their predictions are weakly grounded in the visual and
language evidence defining the task. We introduce GUARD, a test-time failure detection method that measures this grounding without modifying the pretrained policy. GUARD estimates the influence of
token-indexed entries in the final
vision-
language model key-value (KV) cache, constructs counterfactual caches by ablating salient KV entries, and compares their denoising responses with the original conditioning. Based on the comparison, we derive GUARD diagnostic stream including sensitivity,
attention entropy, modality bias, and grounding efficiency, which are calibrated online and processed by a lightweight temporal classifier. We evaluate GUARD under task-held-out splits across five policy-
benchmark settings, using Pi0, SmolVLA, and Alpamayo-1.5 on LIBERO, SimplerEnv, MetaWorld, and PhysicalAI-AV. GUARD achieves the best ROC-AUC on four of five unseen-task settings and ranks second on the remaining setting, improving the average unseen-task ROC-AUC by 5.73 percentage points over the strongest competing runtime monitor while remaining within 0.19 points of the best seen-task average. These results show that directly probing action-head dependence on
multimodal evidence provides a transferable failure signal across policies, tasks, embodiments, and domains.
Read the full abstract ▾
Robotics
SSC: A Verifiable Structured Representation for Bimanual Manipulation Labelling
Yupu Lu, Shuang Wu, Sihan Chen et al. (+4 more)
Subtask labels decompose a long-horizon manipulation demonstration into shorter semantic segments for policy training and evaluation. Natural
language descriptions are easy to read, but their linguistic variability makes automatic verification difficult. Rigid template formats, such as BEHAVIOR-1K's skill_annotation, are linguistically over-segmented, hindering both readability and annotation consistency. We propose the Structured Subtask Chain (SSC), a state-transition representation that bridges these extremes. A demonstration is a sequence of Structured Subtask Template (SST) entries. Each SST stores core action components (subject, predicate, object), flexible conditions (adverbial modifiers such as spatial or instrumental phrases), a base-motion field separate from arm actions, and an after-state scene graph. Built on this format, SSC supports three
vision-
language assisted functions: rendering SSTs as natural
language, checking the assembled chain against four state-transition rules, and completing underspecified fields through a query resolution cascade. We instantiate the pipeline on BEHAVIOR-1K (50 tasks, 3 episodes per task, 2,357 annotated action
cells) for logic verification and content completion, evaluating 13 selected state-of-the-art VL models as candidate verifiers and reporting labelling anomalies.
Read the full abstract ▾
All authors Yupu Lu, Shuang Wu, Sihan Chen, Ruihua Han, Yichen Zhang, Marcus Kalander, Jia Pan
Robotics
SCOPE: Field-of-View-Aware Path Planning in Unknown Space via Safety-Volume Certification
Junbin Yuan, Muqing Cao, Yunwoo Lee et al. (+2 more)
Safe navigation with a body-mounted limited-field-of-view sensor requires the complete
robot-inflated volume of an intended motion to be observed and verified free before execution. We formulate this requirement as online safety-volume certification in an unknown voxel map and construct a certified graph whose vertices correspond exactly to positions with fully known-free safety volumes. Based on this representation, we propose SCOPE (Safety Certification through Observation Planning and Execution), a planning framework that decouples optimistic goal-directed guidance from certified execution. SCOPE converts the first uncertified point along an optimistic route into an explicit observation obligation, resolves it through target-centric viewpoint search, and recursively clears intermediate obligations when useful viewpoints are not yet certified-reachable. A certified preview mechanism and an observation-aware trajectory optimization backend enable smooth execution. We prove conditional completeness: under ideal monotone sensing and exhaustive finite-domain graph search, SCOPE reaches the goal whenever a finite feasible sequence of certified sensing actions exists within its planning primitives. Across 100 randomized tasks in five unknown 3D environments, SCOPE reaches every goal while maintaining near-zero entry into non-certified inflated space, and an ablation shows that the certified preview mechanism reduces mean mission time by 27%. Finally, we validate the complete system through real-
robot demonstrations in four scenarios.
Read the full abstract ▾
All authors Junbin Yuan, Muqing Cao, Yunwoo Lee, Brady Moon, Sebastian Scherer
Robotics
Approximate Multi-Objective Search Under Rulebooks
Omar Muhammetkulyyev, Oren Salzman and Tichakorn Wongpiromsarn
Robotic planning often involves multiple objectives with complex priority relationships, such as safety, efficiency, and regulatory compliance. Rulebooks formalize these relationships, allowing partial ordering of objectives that generalizes both Pareto and lexicographic dominance. Computing the full set of rulebook-optimal solutions, however, is computationally expensive. To address this challenge, we introduce the concept of epsilon-rule-dominance, a principled notion of approximate dominance under rulebooks, and propose RA*pex, a best-first search
algorithm that efficiently computes a compact set of epsilon-approximate rulebook-optimal solutions. RA*pex leverages dimensionality reduction, a technique used to speed up existing multi-objective search
algorithms, while respecting rule hierarchies by maintaining separate closed sets and performing dominance checks over truncated and residual rule sets. We provide a formal analysis of RA*pex, proving that every rulebook-optimal solution is epsilon-rule-dominated (a generalization of approximate dominance we introduce) by at least one solution in the returned set. Empirical results demonstrate that our approach achieves computation times over two orders of magnitude faster than existing methods.
Read the full abstract ▾
Robotics
Design and Flight of an Ion-propelled Micro Hovercraft Leveraging Ground Proximity Effects
C. Luke Nelson, Grant Nations, Mrinmoy Modak et al. (+1 more)
Electroaerodynamic propulsion is compelling for use in micro air vehicles due to its silent and solid-state nature, but its limited efficiency has thus far precluded a path towards power-autonomous flight. Recent work has shown that thrust density and efficiency for small-scale atmospheric ion thrusters can be vastly increased when operating close to a ground plane. Here, we explore the design space of centimeter-scale hovercraft, which can leverage this ground effect for low-altitude flight. We first perform an empirical investigation, characterizing the performance benefits and trade-offs for different geometries and configurations of passive hovercraft skirts, then use the results to fabricate a viable point design. We demonstrate a palm-sized hovercraft that, while tethered to an external power source, can fly for extended periods, withstand dozens of takeoff and landing cycles, passively stabilize to reject significant mechanical disturbances, and generate practically zero audible noise signature. The measured thrust efficiency of 16 mN/W and additional payload capacity of almost 1.5 grams above the vehicle's self mass of about 1.6 grams exceeds any similarly sized electroaerodynamically propelled
robot by an order of magnitude. This is the first time an ion-propelled micro hovercraft has been shown in the open literature, and our work points the way towards an entirely new class of
robot.
Read the full abstract ▾
All authors C. Luke Nelson, Grant Nations, Mrinmoy Modak, Daniel S. Drew
Robotics
Structured LLM Reasoning for Zero-Shot Human–Robot Coordination Under Hidden Goals
Dong Hae Mangalindan, Anand Gokhale, Francesco Bullo et al. (+1 more)
We present a structured large-
language-model (LLM) architecture for zero-shot human–
robot coordination in a cooperative construction task with private goal views. Guided by a Dec-POMDP formulation, the architecture decomposes decision-making into (i) action-conditioned Theory-of-Mind (ToM) inference, (ii) hierarchical planning, (iii) conversation interpretation, (iv) action verification, and (v) feedback-based replanning. We compare the proposed method with an ablation without ToM inference and a multi-
agent reinforcement-learning policy trained offline over many goal pairs. In human-participant experiments, the proposed method required fewer interaction steps and yielded higher post-interaction trust ratings than both baselines. These results suggest that systematically decomposing the team decision problem, using LLMs as tractable surrogates for otherwise intractable inference and planning computations, and retaining conventional verification for physical feasibility can improve both task coordination and the human experience.
Read the full abstract ▾
All authors Dong Hae Mangalindan, Anand Gokhale, Francesco Bullo, Vaibhav Srivastava