Robotics
ObjView-Bench: Rethinking Difficulty and Deployment for Object-Centric View Planning
Sicong Pan, Hao Hu, Xuying Huang et al. (+2 more)
Object-centric view planning is a core component of active
geometric 3D reconstruction in robotics, yet existing evaluations often conflate object complexity, planning difficulty, budget assumptions, and physical reachability constraints. As a result, conclusions drawn from idealized view-planning evaluations may not reliably predict performance under realistic reconstruction settings. We introduce ObjView-Bench, an evaluation framework for rethinking difficulty and deployment in object-centric view planning. First, we disentangle three quantities underlying view-planning evaluation: omnidirectional self-occlusion as an object-side attribute, observation saturation difficulty, and protocol-dependent planning difficulty defined through a set-cover formulation. This separation supports controlled
dataset construction, analysis of slow-saturation objects, and a case study showing that planning difficulty-aware sampling can improve learned view planners. Second, we design deployment-oriented evaluation protocols that reveal how budget regimes and reachable-view constraints alter method behavior. Across classical, learned, and hybrid planners, ObjView-Bench shows that difficulty, budget, and reachability constraints substantially change method rankings and failure modes.
Read the full abstract ▾
All authors Sicong Pan, Hao Hu, Xuying Huang, Benno Wingender, Maren Bennewitz
Robotics · Cross-listed from eess.SP
xApp Empowered Resource Management for Non-Terrestrial Users in 5G O-RAN Networks
Mohammed M.H. Qazzaz, Syed Ali Zaidi, Aubida A. Al-Hameed et al. (+2 more)
This paper introduces a proactive Unmanned Aerial Vehicle (
UAV) mobility management xApp for Open
Radio Access Network (O-RAN) Near Real-Time
Radio Intelligent Controller (Near-RT RIC) environments, employing Double Deep Q-Network (DDQN) reinforcement learning (RL) enhanced with transfer learning to optimise handover decisions for
UAVs operating along predetermined flight trajectories. Unlike reactive approaches that respond to signal degradation, the proposed framework anticipates network conditions and minimises both outage probability and handover frequency through predictive optimisation. The system leverages centralised weight averaging to consolidate knowledge from multiple flight scenarios into a global model capable of generalising to previously unseen operational environments without extensive retraining. A comprehensive evaluation demonstrates that the proposed framework achieves a favourable trade-off between handover frequency and connectivity reliability, reducing handover events by up to 54.6% compared to greedy approaches while maintaining outage probability at practically negligible levels. The results validate the effectiveness of intelligent learning-based approaches for
UAV mobility management in next-generation O-RAN architectures, thereby contributing to seamless integration of aerial user equipment into cellular networks.
Read the full abstract ▾
All authors Mohammed M.H. Qazzaz, Syed Ali Zaidi, Aubida A. Al-Hameed, Abdelaziz Salama, Des Mclernon
Robotics
VRA: Grounding Discrete-Time Joint Acceleration in Voltage-Constrained Actuation
Lingwei Zhang, Jiaming Wang, Tianlin Zhang et al. (+5 more)
Discrete-time joint acceleration constraints are widely used to enforce position and velocity limits. However, under voltage-constrained electric actuators, kinematically admissible accelerations may be physically unrealizable, exposing a missing execution-level abstraction. We propose Voltage-Realizable Acceleration (VRA), a joint-level acceleration interface that grounds kinematic acceleration in voltage-constrained actuator physics by restricting commanded accelerations to voltage-realizable constraints. Hardware experiments on electric actuators and a wheel-legged quadruped show that VRA removes unrealizable accelerations, restores consistent near-constraint execution, and reduces constraint-induced oscillations.
Read the full abstract ▾
All authors Lingwei Zhang, Jiaming Wang, Tianlin Zhang, Zhitao Song, Xuanqi Zeng, Weipeng Xia, Zhongyu Li, Yun-hui Liu
Robotics
Embodied AI in Action: Insights from SAE World Congress 2026 on Safety, Trust, Robotics, and Real-World Deployment
Jan-Mou Li, Paul Schmitt, Wei Tong et al. (+4 more)
Embodied artificial intelligence is rapidly moving from research into real-world systems such as autonomous vehicles, mobile
robots, and industrial machines. As these systems become more capable of perceiving, deciding, and acting in dynamic environments, they also introduce new challenges in safety, trust, governance, and operational reliability. This white paper summarizes key insights from the SAE World Congress 2026 panel session
Embodied AI in Action, which brought together experts from automotive, robotics, artificial intelligence, and safety engineering. The discussion highlighted the need to treat
embodied AI as a systems challenge requiring engineering rigor, lifecycle governance, human-centered design, and evolving standards. The paper provides practical perspectives for executives, policymakers, and technical leaders seeking to adopt
embodied AI responsibly. The panel reached broad agreement that long-term success will depend not only on advances in AI capability, but equally on safe and trustworthy deployment.
Read the full abstract ▾
All authors Jan-Mou Li, Paul Schmitt, Wei Tong, Majed Mohammed, Akshay Chalana, Arpan Kusari, Edward Griffor
Robotics · Cross-listed from cs.CV
DeepSight: Long-Horizon World Modeling via Latent States Prediction for End-to-End Autonomous Driving
Lingjun Zhang, Changjie Wu, Linzhe Shi et al. (+6 more)
End-to-end autonomous driving systems are increasingly integrating
Vision-
Language Model (VLM) architectures, incorporating text
reasoning or visual
reasoning to enhance the robustness and accuracy of driving decisions. However, the
reasoning mechanisms employed in most methods are direct adaptations from general domains, lacking in-depth exploration tailored to autonomous driving scenarios, particularly within visual
reasoning modules. In this paper, we propose a driving world model that performs parallel prediction of latent semantic features for consecutive future frames in the bird's-eye-view (BEV) space, thereby enabling long-horizon modeling of future world states. We also introduce an efficient and adaptive text
reasoning mechanism that utilizes additional social knowledge and
reasoning capabilities to further improve driving performance in challenging long-tail scenarios. We present a novel, efficient, and effective approach that achieves state-of-the-art (SOTA) results on the closed-loop Bench2drive
benchmark. Codes are available at: https://github.com/hotdogcheesewhite/DeepSight.
Read the full abstract ▾
All authors Lingjun Zhang, Changjie Wu, Linzhe Shi, Jiangyang Li, Jiaxin Liu, Lei Yang, Hang Zhang, Mu Xu, Hong Wang
Robotics
VEGA: Visual Encoder Grounding Alignment for Spatially-Aware Vision-Language-Action Models
Hao Wang, Xiaobao Wei, Jingyang He et al. (+10 more)
Precise spatial
reasoning is fundamental to robotic manipulation, yet the visual backbones of current
vision-
language-action (VLA) models are predominantly pretrained on 2D image data without explicit 3D
geometric supervision, resulting in representations that lack accurate spatial awareness. Existing implicit spatial grounding methods partially address this by aligning VLA features with those of 3D-aware foundation models, but they rely on empirical layer search and perform
alignment on LLM-level visual
tokens where spatial structure has already been entangled with linguistic semantics, limiting both generalizability and
geometric interpretability. We propose VEGA (Visual Encoder Grounding
Alignment), a simple yet effective framework that directly aligns the output of the VLA's visual encoder with spatially-aware features from DINOv2-FiT3D, a DINOv2 model fine-tuned with multi-view consistent 3D Gaussian Splatting supervision. By performing
alignment at the visual encoder output level, VEGA grounds spatial awareness before any linguistic
entanglement occurs, offering a more interpretable and principled
alignment target. The
alignment is implemented via a lightweight projector trained with a cosine similarity loss alongside the standard action prediction objective, and is discarded at inference time, introducing no additional computational overhead. Extensive experiments on simulation
benchmark and real-world manipulation tasks demonstrate that VEGA consistently outperforms existing implicit spatial grounding baselines, establishing a new state-of-the-art among implicit spatial grounding methods for VLA models.
Read the full abstract ▾
All authors Hao Wang, Xiaobao Wei, Jingyang He, Chengyu Bai, Chun-Kai Fan, Jiajun Cao, Jintao Chen, Ying Li, Shanyu Rong, Ming Lu, Xiaozhu Ju, Jian Tang, Shanghang Zhang
Robotics · Cross-listed from cs.CV
OpenSGA: Efficient 3D Scene Graph Alignment in the Open World
Gang Chen, Sebastián Barbas Laina, Stefan Leutenegger et al. (+1 more)
Scene graph
alignment establishes object correspondences between two 3D scene graphs constructed from partially overlapping observations. This enables efficient scene understanding and object-level relocalization when a
robot revisits a place, as well as global map fusion across multiple
agents. Such capabilities are essential for
robots that require long-term
memory for long-horizon tasks involving interactions with the environment. Existing approaches mainly focus on subscan-to-subscan (S2S)
alignment and depend heavily on
geometric point-cloud features, leaving frame-to-scan (F2S)
alignment and open-set
vision-
language features underexplored. In addition, existing
datasets for scene graph
alignment remain small-scale with limited object diversity, constraining systematic training and evaluation. We present a unified and efficient scene graph
alignment framework that predicts object correspondences by fusing
vision-
language, textual, and
geometric features with spatial context. The framework comprises modules such as a distance-gated spatial
attention encoder, a minimum-cost-flow-based allocator, and a global scene
embedding generator to achieve accurate
alignment even under large coordinate discrepancies. We further introduce ScanNet-SG, a large-scale
dataset generated via an automated annotation pipeline with over 700k samples, covering 509 object categories from ScanNet labels and over 3k categories from GPT-4o-based tagging. Experiments show that our method achieves the best overall performance on both F2S and S2S tasks, substantially outperforming existing scene graph
alignment methods. Our code and
dataset are released at: https://autonomousrobots.nl/paper_websites/opensga.
Read the full abstract ▾
All authors Gang Chen, Sebastián Barbas Laina, Stefan Leutenegger, Javier Alonso-Mora
Robotics · Cross-listed from eess.SY
Priority-Driven Control and Communication in Decentralized Multi-Agent Systems via Reinforcement Learning
Qingyun Guo, Junyi Shi, Tomasz Piotr Kucner et al. (+1 more)
Event-triggered control provides a mechanism for avoiding excessive use of constrained communication bandwidth in networked multi-
agent systems. However, most existing methods rely on accurate system models, which may be unavailable in practice. In this work, we propose a model-free, priority-driven reinforcement learning
algorithm that learns communication priorities and control policies jointly from data in decentralized multi-
agent systems. By learning communication priorities, we circumvent the hybrid action space typical in event-triggered control with binary communication decisions. We evaluate our
algorithm on
benchmark tasks and demonstrate that it outperforms the baseline method.
Read the full abstract ▾
All authors Qingyun Guo, Junyi Shi, Tomasz Piotr Kucner, Dominik Baumann
Robotics · Cross-listed from cs.GR
Geometrically Approximated Modeling for Emitter-Centric Ray-Triangle Filtering in Arbitrarily Dynamic LiDAR Simulation
Rabin Gajmer, Joonas Haapala and Zoltan Beck
Real-time Light Detection And Ranging (
LiDAR) simulation must find, per emitted ray, the closest intersecting triangle even in dynamic scenes containing large numbers of moving and deformable objects. Dominant acceleration-structure approaches require rebuilding each frame for dynamic geometry – a cost that compounds directly with scene dynamics and cannot be amortized regardless of how little actually changed. This paper presents the Gajmer Ray-Casting
Algorithm (GRCA), which inverts the question: instead of asking what does each ray hit? it asks which rays can each triangle possibly hit? GRCA geometrically models spinning
LiDAR emitters as rotation-traced cones or planes and uses each triangle's emitter-centric apparent area to cull, per triangle, which channels and the rays within those channels can possibly reach it – without any acceleration structure. GRCA is compute-based and vendor-agnostic by design, targeting highly dynamic, high-resolution simultaneous multi-sensor simulation. At its core, GRCA is a general-purpose ray-casting
algorithm: the emitter-centric inversion applies to any setting where rays originate from a known position, not only
LiDAR.
Benchmarks evaluate 2-8 simultaneous 128x4096-ray
LiDARs (360deg/180deg) over complex dynamic scenes – with just two sensors casting ~1M rays per frame. With range culling inactive, GRCA reaches up to 7.97x over hardware-accelerated OptiX (GPU) and 14.55x over Embree (CPU). Two independent extensions further boost performance even in the most complex scene (~22M triangles, ~9M of which are dynamic, 8
LiDARs): range culling at realistic deployment ranges (10-100m) reaches up to 7.02x GPU and 9.33x CPU; a hybrid pipeline – GRCA for dynamic geometry, OptiX/Embree for static – reaches up to 10.5x GPU and 19.2x CPU.
Read the full abstract ▾
Robotics
Learning Point Cloud Geometry as a Statistical Manifold: Theory and Practice
Jinwoo Lee, Jiwoo Kim, Woojae Shin et al. (+2 more)
Point clouds are a fundamental representation for robotic perception tasks such as localization, mapping, and object pose estimation. However,
LiDAR-acquired point clouds are inherently sparse and non-uniform, providing incomplete observations of the underlying scene geometry. This makes reliable
geometric reasoning challenging and degrades downstream perception performance. Existing approaches attempt to compensate for these limitations by estimating local geometry, but often rely on hand-crafted statistics or end-to-end supervised learning, which can suffer from limited scalability or require large amounts of accurately labeled data. To address these challenges, we explicitly model point cloud geometry under a principled mathematical formulation. We represent local geometry as a statistical
manifold induced by a family of Gaussian distributions, where each point is associated with a Gaussian capturing its local
geometric structure. Based on this formulation, we introduce Point-to-Ellipsoid (POLI), a deep neural
estimator that predicts per-point Gaussian geometry. POLI learns a mapping from point cloud observations to their underlying geometry in a self-supervised manner, removing the need for labeled data while preserving strong
geometric inductive biases. The resulting representation integrates seamlessly into existing robotic perception pipelines without architectural modifications. Extensive experiments show that POLI enables accurate and robust geometry estimation and consistently improves performance across diverse robotic perception tasks.
Read the full abstract ▾
All authors Jinwoo Lee, Jiwoo Kim, Woojae Shin, Giseop Kim, Hyondong Oh
Robotics · Cross-listed from cs.SE
VISOR: A Vision-Language Model-based Test Oracle for Testing Robots
Prasun Saurabh, Pablo Valle, Aitor Arrieta et al. (+2 more)
Testing
robots requires assessing whether they perform their intended tasks correctly, dependably, and with high quality, a challenge known as the test oracle problem in software testing. Traditionally, this assessment relies on task-specific symbolic oracles for task correctness and on human manual evaluation of
robot behavior, which is time-consuming, subjective, and error-prone. To address this, we propose VISOR, a
Vision-
Language Model (VLM)-based approach for automated test oracle assessment that eliminates the need of expensive human evaluations. VISOR performs automated evaluation of task correctness and quality, addressing the limitations of existing symbolic test oracles, which are task-specific and provide pass/fail judgments without explicitly quantifying task quality. Given the inherent uncertainty in VLMs, VISOR also explicitly quantifies its own uncertainty during test assessments. We evaluated VISOR using two VLMs, i.e., GPT and Gemini, across four robotic tasks on over 1,000
videos. Results show that Gemini achieves higher recall while GPT achieves higher precision. However, both models show low correlation between uncertainty and correctness, which prevents using uncertainty as a correctness predictor.
Read the full abstract ▾
All authors Prasun Saurabh, Pablo Valle, Aitor Arrieta, Shaukat Ali, Paolo Arcaini
Robotics · Cross-listed from cs.CV
Temporal Sampling Frequency Matters: A Capacity-Aware Study of End-to-End Driving Trajectory Prediction
Yumao Liu, Tao Liu, Xiangyu Li et al. (+2 more)
End to end (E2E) autonomous driving trajectory prediction is often trained with camera frames sampled at the highest available temporal frequency, assuming that denser sampling improves performance. We question this assumption by treating temporal sampling frequency as an explicit training set design variable. Starting from high frequency E2E driving
datasets, we construct frequency sweep training sets by temporally subsampling camera frames along each trajectory. For each model
dataset pair, we train and evaluate the same model under a fixed protocol, so the frequency response reflects how prediction performance changes with sampling frequency. We analyze this response from a capacity aware perspective. Sparse sampling may miss driving relevant cues, while dense sampling may add redundant visual content and off
manifold noise. For finite capacity models, this can create a driving irrelevant capacity burden. We evaluate three smaller E2E models and a larger VLA style AutoVLA model on Waymo, nuScenes, and PAVE. Results show model and
dataset dependent frequency responses. Smaller E2E models often show non monotonic or near plateau trends and achieve their best 3 second ADE at lower or intermediate frequencies. In contrast, AutoVLA achieves its best 3 second ADE and FDE at the highest evaluated frequency on all three
datasets. Iteration matched controls suggest that the advantage of lower or intermediate frequencies for smaller models is not explained only by unequal training update counts. These findings show that temporal sampling frequency should be reported and tuned, rather than fixed to the highest available value.
Read the full abstract ▾
All authors Yumao Liu, Tao Liu, Xiangyu Li, Jiaxiang Li, Ke Ma
Robotics · Cross-listed from cs.CV
PaMoSplat: Part-Aware Motion-Guided Gaussian Splatting for Dynamic Scene Reconstruction
Yinan Deng, Jianyu Dou, Jiahui Wang et al. (+3 more)
Dynamic scene reconstruction represents a fundamental yet demanding challenge in computer
vision and robotics. While recent progress in 3DGS-based methods has advanced dynamic scene modeling, obtaining high-fidelity rendering and accurate tracking in scenarios with substantial, intricate motions remains significantly challenging. To address these challenges, we propose PaMoSplat, a novel dynamic Gaussian splatting framework incorporating part awareness and motion priors. Our approach is grounded in two key observations: 1) Parts serve as primitives for scene deformation, and 2) Motion cues from optical flow can effectively guide part motion. Specifically, PaMoSplat initializes by lifting multi-view
segmentation masks into 3D space via graph clustering, establishing coherent Gaussian parts. For subsequent timestamps, we leverage a differential evolutionary
algorithm to estimate the rigid motion of these parts using multi-view optical flow cues, providing a robust warm-start for further optimization. Additionally, PaMoSplat introduces an adaptive iteration count mechanism, internal learnable
rigidity, and flow-supervised rendering loss to accelerate and optimize the training process. Comprehensive evaluations across diverse scenes, including real-world environments, demonstrate that PaMoSplat delivers superior rendering quality, improved tracking precision, and faster convergence compared to existing methods. Furthermore, it enables multiple part-level downstream applications, such as 4D scene editing.
Read the full abstract ▾
All authors Yinan Deng, Jianyu Dou, Jiahui Wang, Jingyu Zhao, Yi Yang, Yufeng Yue
Robotics · Cross-listed from cs.CV
Increasing the Efficiency of DETR for Maritime High-Resolution Images
Tinsae Yehuala, Hao Cheng and Ville Lehtola
Maritime object detection is critical for the safe
navigation of unmanned surface vessels (USVs), requiring accurate recognition of obstacles from small buoys to large vessels. Real-time detection is challenging due to long distances, small object sizes, large-scale variations, edge computing limitations, and the high
memory demands of high-resolution imagery. Existing solutions, such as downsampling or image splitting, often reduce accuracy or require additional processing, while
memory-efficient models typically handle only limited resolutions. To overcome these limitations, we leverage
Vision Mamba (ViM) backbones, which build on State Space Models (SSMs) to capture long-range dependencies while scaling linearly with sequence length. Images are tokenized into sequences for efficient high-resolution processing. For further computational efficiency, we design a tailored Feature Pyramid Network with successive downsampling and SSM layers, as well as
token pruning to reduce unnecessary computation on background regions. Compared to state-of-the-art methods like RT-DETR with ResNet50 backbone, our approach achieves a better balance between performance and computational efficiency in maritime object detection.
Read the full abstract ▾
Robotics
Federico Pizzolato, Francesco Pasti and Nicola Bellotto
Terrain
segmentation is a fundamental capability for autonomous mobile
robots operating in unstructured outdoor environments. However, state-of-the-art models are incompatible with the
memory and compute constraints typical of microcontrollers, limiting scalable deployment in small robotics platforms. To address this gap, we develop a complete framework for robust binary terrain
segmentation on a low-cost microcontroller. At the core of our approach we design Nano-U, a highly compact binary
segmentation network with a few thousand parameters. To compensate for the network's minimal capacity, we train Nano-U via Quantization-Aware
Distillation (QAD), combining knowledge
distillation and quantization-aware training. This allows the final quantized model to achieve excellent results on the Botanic Garden
dataset and to perform very well on TinyAgri, a custom agricultural field
dataset with more challenging scenes. We deploy the quantized Nano-U on a commodity microcontroller by extending MicroFlow, a compiler-based inference engine for TinyML implemented in Rust. By eliminating interpreter overhead and dynamic
memory allocation, the quantized model executes on an ESP32-S3 with a minimal
memory footprint and low latency. This compiler-based execution demonstrates a viable and energy-efficient solution for perception on low-cost robotic platforms.
Read the full abstract ▾
Robotics
HeteroGenManip: Generalizable Manipulation For Heterogeneous Object Interactions
Zhenhao Shen, Zeming Yang, Yue Chen et al. (+5 more)
Generalizable manipulation involving cross-type object interactions is a critical yet challenging capability in robotics. To reliably accomplish such tasks,
robots must address two fundamental challenges: "where to manipulate" (contact point localization) and "how to manipulate" (subsequent interaction trajectory planning). Existing foundation-model-based approaches often adopt end-to-end learning that obscures the distinction between these stages, exacerbating error accumulation in long-horizon tasks. Furthermore, they typically rely on a single uniform model, which fails to capture the diverse, category-specific features required for heterogeneous objects. To overcome these limitations, we propose HeteroGenManip, a task-conditioned, two-stage framework designed to decouple initial grasp from complex interaction execution. First, Foundation-Correspondence-Guided Grasp module leverages structural priors to align the initial contact state, thereby significantly reducing the pose uncertainty of grasping. Subsequently, Multi-Foundation-Model
Diffusion Policy (MFMDP) routes objects to category-specialized foundation models, integrating fine-grained
geometric information with highly-variable part features via a dual-stream cross-
attention mechanism. Experimental evaluations demonstrate that HeteroGenManip achieves robust intra-category shape and pose generalization. The framework achieves an average 31% performance improvement in simulation tasks with broad type setting, alongside a 36.7% gain across four real-world tasks with different interaction types.
Read the full abstract ▾
All authors Zhenhao Shen, Zeming Yang, Yue Chen, Yuran Wang, Shengqiang Xu, Mingleyang Li, Hao Dong, Ruihai Wu
Robotics · Cross-listed from cs.CV
MTA-RL: Robust Urban Driving via Multi-modal Transformer-based 3D Affordances and Reinforcement Learning
Guangli Chen, Dianzhao Li, Wenjian Zhong et al. (+2 more)
Robust urban autonomous driving requires reliable 3D scene understanding and stable decision-making under dense interactions. However, existing end-to-end models lack interpretability, while modular pipelines suffer from error propagation across brittle interfaces. This paper proposes MTA-RL, the first framework that bridges perception and control through Multi-modal
Transformer-based 3D Affordances and Reinforcement Learning (RL). Unlike previous fusion models that directly regress actions, RGB images and
LiDAR point clouds are fused using a
transformer architecture to predict explicit, geometry-aware affordance representations. These structured representations serve as a compact observation space, enabling the RL policy to operate purely on predicted driving semantics, which significantly improves sample efficiency and stability. Extensive evaluations in CARLA Town01-03 across varying densities (20-60 background vehicles) show that MTA-RL consistently outperforms state-of-the-art baselines. Trained solely on Town03, our method demonstrates superior zero-shot generalization in unseen towns, achieving up to a 9.0% increase in Route Completion, an 11.0% increase in Total Distance, and an 83.7% improvement in Distance Per Violation. Furthermore, ablation studies confirm that our multi-modal fusion and reward shaping are critical, significantly outperforming image-only and unshaped variants, demonstrating the effectiveness of MTA-RL for robust urban autonomous driving.
Read the full abstract ▾
All authors Guangli Chen, Dianzhao Li, Wenjian Zhong, Bangquan Xie, Ostap Okhrin
Robotics
Data-Asymmetric Latent Imagination and Reranking for 3D Robotic Imitation Learning
Lianghao Luo, Xizhou Bu, Ruyan Liu et al. (+5 more)
Robotic imitation learning typically assumes access to optimal demonstrations, yet real-world data collection often yields suboptimal, exploratory, or even failed trajectories. Discarding such data wastes valuable information about environment dynamics and failure modes, which can instead be leveraged to improve decision-making. While 3D policies reduce reliance on high-quality demonstrations through strong spatial generalization, they still require large-scale data to achieve high task success. To address this, we propose DALI-R, a Data-Asymmetric Latent Imagination and Reranking framework for 3D robotic imitation learning from mixed-quality trajectories. It learns a Latent World Model over 3D point clouds for imagined rollouts and a Task Completion Scorer that reranks candidate action chunks, improving decision-making without additional high-quality demonstrations. We instantiate DALI-R with both
diffusion and efficient flow-matching policies and evaluate it on Adroit and MetaWorld
benchmarks. Across the two evaluated 3D base policies, DALI-R achieves an average $6.8$% improvement in success rate while incurring less than $0.7\times$ additional inference overhead.
Read the full abstract ▾
All authors Lianghao Luo, Xizhou Bu, Ruyan Liu, Qingqiu Huang, Chufeng Tang, Xiaoshuai Hao, Hongbo Wang, Wei Li
Robotics
Plan in Sandbox, Navigate in Open Worlds: Learning Physics-Grounded Abstracted Experience for Embodied Navigation
Zhixuan Shen, Jiawei Du, Ziyu Guo et al. (+5 more)
Vision-
Language Models (VLMs) have demonstrated exceptional general
reasoning capabilities. However, their performance in
embodied navigation remains hindered by a scarcity of aligned open-world
vision and
robot control data. Despite simulators providing a cost-effective alternative for data collection, the inherent reliance on photorealistic simulations often limits the transferability of learned policies. To this end, we propose \textit{Sandbox-Abstracted Grounded Experience} (\textbf{SAGE}), a framework that enables
agents to learn within a physics-grounded semantic abstraction rather than a photorealistic simulation, mimicking the human capacity for mental simulation where plans are rehearsed in simplified physics abstractions before execution. SAGE system operates via three synergistic phases: (1) Genesis: constructing diverse, physics-constrained semantic environments to bootstrap experience; (2)
Evolution: distilling experiences through Reinforcement Learning (RL), utilizing a novel asymmetric adaptive clipping mechanism to stabilize updates; (3)
Navigation: bridging the abstract policy to open-world control. We demonstrate that SAGE significantly improves planner-assisted
embodied navigation, achieving a 53.21% LLM-Match Success Rate on A-EQA (+9.7% over baseline), while showing encouraging transfer to physical indoor
robot deployment.
Read the full abstract ▾
All authors Zhixuan Shen, Jiawei Du, Ziyu Guo, Han Luo, Lilan Peng, Joey Tianyi Zhou, Haonan Luo, Tianrui Li
Robotics
Retrieve-then-Steer: Online Success Memory for Test-Time Adaptation of Generative VLAs
Jianchao Zhao, Huoren Yang, Yusong Hu et al. (+6 more)
Vision-
Language-Action (VLA) models show strong potential for general-purpose robotic manipulation, yet their closed-loop reliability often degrades under local deployment conditions. Existing evaluations typically treat test episodes as independent zero-shot trials. However, real
robots often operate repeatedly in the same or slowly changing environments, where successful executions provide environment-verified evidence of reliable behavior patterns. We study this persistent-deployment setting, asking whether a partially competent frozen VLA can improve its reliability by reusing its successful test-time experience. We propose an online success-
memory guided test-time adaptation framework for generative VLAs. During deployment, the
robot stores progress-calibrated successful observation-action segments in a long-term
memory. At inference, it retrieves state-relevant action chunks, filters inconsistent candidates via trajectory-level consistency, and aggregates them into an elite action prior. To incorporate this prior into action generation, we introduce confidence-adaptive prior guidance, which injects the elite prior into an intermediate state of the flow-matching action sampler and adjusts the guidance strength based on
retrieval confidence. This design allows the frozen VLA to exploit environment-specific successful experience while preserving observation-conditioned generative refinement. This retrieve-then-steer mechanism enables lightweight, non-parametric test-time adaptation without requiring parameter updates. Simulation and real-world experiments show improved task success and closed-loop stability, especially in long-horizon and multi-stage tasks.
Read the full abstract ▾
All authors Jianchao Zhao, Huoren Yang, Yusong Hu, Yuyang Gao, Qiguan Ou, Cong Wan, SongLin Dong, Zhiheng Ma, Yihong Gong
Robotics
A cell-decomposition based path planner for 3D navigation in constrained workspaces
João P. L. Morais, Luciano C. A. Pimenta, Marcelo A. Santos et al. (+1 more)
This paper proposes a
cell decomposition
algorithm for binary occupancy grids that ensures mutual complete visibility from each
cell to at least one adjacent
cell. This decomposition establishes a simplified framework for verifying path feasibility that can be easily embedded in optimization problems. To illustrate its utility, we formulate both second-order cone programs (SOCP) and their mixed-integer variant (MISOCP) within the proposed framework. Furthermore, we propose the KSP-SOCP method, which combines Yen's k-shortest path
algorithm with the SOCP, achieving improved solutions compared to a standard SOCP approach while avoiding the computational burden of MISOCP. The
cell decomposition
algorithm, KSP-SOCP, and MISOCP approaches were evaluated in 9 city-like workspaces. The decomposition efficiently partitioned each map, enabling both optimization methods to compute feasible paths. The proposed KSP-SOCP achieved time performance comparable to the MISOCP while requiring less
memory, making it highly suitable for large-scale problems.
Read the full abstract ▾
All authors João P. L. Morais, Luciano C. A. Pimenta, Marcelo A. Santos, Guilherme V. Raffo
Robotics
EFGCL: Learning Dynamic Motion through Spotting-Inspired External Force Guided Curriculum Learning
Keita Yoneda, Kento Kawaharazuka and Kei Okada
Learning dynamic whole-body motions for legged
robots through reinforcement learning (RL) remains challenging due to the high risk of failure, which makes efficient exploration difficult and often leads to unstable learning. In this paper, we propose External Force Guided Curriculum Learning (EFGCL), a guided RL approach based on the principle of physical guidance, in which external assistive forces are introduced during training. Inspired by spotting in artistic gymnastics, EFGCL enables
agents to physically experience successful motion executions without relying on task-specific reward shaping or reference trajectories. Experiments on a quadrupedal
robot performing Jump, Backflip, and Lateral-Flip tasks demonstrate that EFGCL accelerates learning of the Jump task by approximately a factor of two and enables the acquisition of complex whole body motions that conventional RL methods fail to learn. We further show that the learned policies can be deployed on real
robot, reproducing motions consistent with those observed in simulation. These results indicate that physically guided exploration, which allows
agents to experience success early in training, is an effective and general strategy for improving learning efficiency in dynamic whole-body motion tasks.
Read the full abstract ▾
Robotics
Guided Streaming Stochastic Interpolant Policy
Puming Jiang, Meiyi Wang, Kelvin Lin et al. (+2 more)
Inference-time guidance is essential for steering generative
robot policies toward dynamic objectives without retraining, yet existing methods are largely confined to chunk-based architectures that exhibit high latency and lack the reactivity needed for test-time preference
alignment or obstacle avoidance. In this work, we formally derive the optimal guidance term for
Stochastic Interpolants (SI) by analyzing the value function's time
evolution via the Backward Kolmogorov Equation, establishing a modified drift that theoretically guarantees sampling from a target distribution. We apply this framework to real-time control through the Streaming
Stochastic Interpolant Policy (SSIP), which generalizes the deterministic Streaming Flow Policy (SFP). Unifying this guidance law with the streaming architecture enables fast and reactive control. To support diverse deployment needs, we propose two complementary mechanisms: training-free
Stochastic Trajectory Ensemble Guidance (STEG) that computes gradients on-the-fly for zero-shot adaptation, and training-based Conditional Critic Guidance (CCG) for amortized inference. Empirical evaluations demonstrate that our guided streaming approach significantly outperforms conventional chunk-based policies in reactivity and provides superior, physically valid guidance for dynamic, unstructured environments.
Read the full abstract ▾
All authors Puming Jiang, Meiyi Wang, Kelvin Lin, Ce Hao, Harold Soh
Robotics
Beyond Self-Play and Scale: A Behavior Benchmark for Generalization in Autonomous Driving
Aron Distelzweig, Faris Janjoš, Andreas Look et al. (+7 more)
Recent Autonomous Driving (AD) works such as GigaFlow and PufferDrive have unlocked Reinforcement Learning (RL) at scale as a training strategy for driving policies. Yet such policies remain disconnected from established
benchmarks, leaving the performance of large-scale RL for driving on standardized evaluations unknown. We present BehaviorBench – a comprehensive test suite that closes this gap along three axes: Evaluation, Complexity, and Behavior Diversity. In terms of Evaluation, we provide an interface connecting PufferDrive to nuPlan, which, for the first time, enables policies trained via RL at scale to be evaluated on an established planning
benchmark for autonomous driving. Complementarily, we offer an evaluation framework that allows planners to be benchmarked directly inside the PufferDrive simulation, at a fraction of the time. Regarding Complexity, we observe that today's standardized
benchmarks are so simple that near-perfect scores are achievable by straight lane following with collision checking. We extract a meaningful, interaction-rich split from the Waymo Open Motion
Dataset (WOMD) on which strong performance is impossible without multi-
agent reasoning. Lastly, we address Behavior Diversity. Existing
benchmarks commonly evaluate planners against a single rule-based traffic model, the Intelligent Driver Model (IDM). We provide a diverse suite of interactive traffic
agents to stress-test policies under heterogeneous behaviors, beyond just using IDM. Overall, our benchmarking analysis uncovers the following insight: despite learning interactive behaviors in an emergent manner, policies trained via pure self-play under standard reward functions overfit to their training opponents and fail to generalize to other traffic
agent behaviors. Building on this observation, we propose a hybrid planner that combines a PPO policy with a rule-based planner.
Read the full abstract ▾
All authors Aron Distelzweig, Faris Janjoš, Andreas Look, Anna Rothenhäusler, Daniel Jost, Oliver Scheel, Raghu Rajan, Daphne Cornelisse, Eugene Vinitsky, Joschka Boedecker
Robotics
Muninn: Your Trajectory Diffusion Model But Faster
Gokul Puthumanaillam, Hao Jiang, Ruben Hernandez et al. (+4 more)
Diffusion-based trajectory planners can synthesize rich,
multimodal robot motions, but their iterative denoising makes online planning and control prohibitively slow. Existing accelerations either modify the sampler or compress the network–sacrificing plan quality or requiring retraining without accounting for downstream control risk. We address the problem of making
diffusion-based trajectory planners fast enough for real-time
robot use without retraining the model or sacrificing trajectory quality, and in a way that works across diverse state-space
diffusion architectures. Our key insight is that
diffusion trajectory planners expose two signals we can exploit: a cheap probe of how their internal trajectory representation changes across steps, and analytic coefficients that describe how denoiser errors affect the sampler's state update. By calibrating the first signal against the second on offline runs, we obtain a per-step score that upper-bounds how far the final trajectory can deviate when we reuse a cached denoiser output, and we treat this bound as an uncertainty budget that we can spend over the denoising process. Building on this insight, we present Muninn, a training-free caching wrapper that tracks this uncertainty budget during sampling and, at each
diffusion step, chooses between reusing a cached denoiser output when the predicted deviation is small and recomputing the denoiser when it is not. Across standard
benchmarks Muninn delivers up to 4.6x wall-clock speedups across several trajectory
diffusion models by reducing denoiser evaluations, while preserving task performance and safety metrics. Muninn further certifies that cached rollouts remain within a specified distance of their full-compute counterparts, and we validate these gains in real-time closed-loop
navigation and manipulation hardware deployments. Project page: https://github.com/gokulp01/Muninn.
Read the full abstract ▾
All authors Gokul Puthumanaillam, Hao Jiang, Ruben Hernandez, Jose Fuentes, Paulo Padrao, Leonardo Bobadilla, Melkior Ornik
Robotics
StereoPolicy: Improving Robotic Manipulation Policies via Stereo Perception
Evans Han, Yunfan Jiang, Yingke Wang et al. (+6 more)
Recent advances in
robot imitation learning have produced powerful visuomotor policies that manipulate diverse objects from visual inputs. However,
monocular observations lack depth information, which is critical for precise manipulation in cluttered or geometrically complex scenes. Explicit depth maps and point clouds are often noisy and fragile in real-world manipulation. We introduce StereoPolicy, a visuomotor policy learning framework that directly leverages synchronized stereo image pairs to improve spatial perception without constructing explicit 3D representations. StereoPolicy processes each image with pretrained 2D
vision encoders and fuses left-right features through a cross-
attention-based Stereo
Transformer, learning task-conditioned binocular representations. The framework integrates with
diffusion-based and pretrained
vision-
language-action (VLA) policies, delivering consistent improvements over RGB, RGB-D, point cloud, and multi-view baselines across three simulation
benchmarks and seven real-
robot tabletop and bimanual mobile manipulation tasks. Our results support task-trained binocular fusion as an effective way to adapt 2D visual representations for manipulation.
Read the full abstract ▾
All authors Evans Han, Yunfan Jiang, Yingke Wang, Haoyue Xiao, Huang Huang, Jianwen Xie, Jiajun Wu, Li Fei-Fei, Ruohan Zhang
Robotics
HiDrive: A Closed-Loop Benchmark for High-Level Autonomous Driving
Zhongyu Xia, Guanyu Zhu, Guo Tang et al. (+2 more)
End-to-end autonomous driving has witnessed rapid progress, yet existing
benchmarks are increasingly saturated, with state-of-the-art models achieving near-perfect scores on widely used open-loop and closed-loop
benchmarks. This saturation does not mean that the problem has been solved; instead, it reveals that current
benchmarks remain limited in scenario diversity, object variety, and the breadth of driving capabilities they evaluate. In particular, they lack sufficient long-tail scenarios involving rare but safety-critical objects and fail to assess advanced decision-making such as legal compliance, ethical
reasoning, and emergency response. To address these gaps, we propose HiDrive, a new closed-loop
benchmark for end-to-end autonomous driving that emphasizes long-tail scenarios and a richer evaluation of driving capabilities. HiDrive introduces a diverse set of rare objects and uncommon traffic situations, and expands evaluation from basic driving skills to more advanced capabilities, including rule compliance, moral
reasoning, and context-dependent emergency maneuvers. Correspondingly, we extend previous collision-avoidance-centered metrics into a comprehensive evaluation system that encompasses collision and braking, traffic-rule compliance, and moral-
reasoning indicators. Built on a more advanced physics engine, HiDrive provides physically realistic lighting and high-fidelity visual rendering, offering a more challenging and realistic testbed for assessing whether autonomous driving systems can handle the complexity of real-world deployment. The HiDrive software, source code, digital assets, and documentation are available at https://github.com/VDIGPKU/HiDrive.
Read the full abstract ▾
All authors Zhongyu Xia, Guanyu Zhu, Guo Tang, Wenhao Chen, Yongtao Wang
Robotics
JODA: Composable Joint Dynamics for Articulated Objects
Tianhong Gao, Cheng Yu, Yinghao Xu et al. (+1 more)
Articulated objects used in simulation and
embodied AI are typically specified by geometry and kinematic structure, but lack the fine-grained dynamical effects that govern realistic mechanical behavior, such as frictional holding, detents, soft closing, and snap latching. Existing approaches either ignore the detailed structure of dynamics entirely, or use simple models with limited expressiveness. We introduce JODA, a framework for generating joint-level dynamics as a structured three-channel field over the joint degree of freedom, capturing conservative forces, dry friction, and damping. Instantiated using shape-constrained piecewise cubic interpolation (PCHIP), this formulation defines a compact and expressive function space that is both interpretable and compatible with differentiable simulation. Building on this representation, we develop methods for inferring and refining joint dynamics from
multimodal inputs. Given visual observations and joint context, a
vision-
language model proposes structured dynamical primitives, which are composed into a unified dynamics field. The resulting representation supports both direct manipulation and gradient-based refinement. We demonstrate that JODA enables plausible and controllable modeling of diverse joint behaviors, providing a unified interface for inference, editing, and optimization. Code and example assets with their generated profiles will be released upon publication.
Read the full abstract ▾
All authors Tianhong Gao, Cheng Yu, Yinghao Xu, Mengyu Chu
Robotics · Cross-listed from cs.AI
LoopVLA: Learning Sufficiency in Recurrent Refinement for Vision-Language-Action Models
Boyang Shen, Kaixiang Yang, Hao Wang et al. (+4 more)
Current
Vision-
Language-Action (VLA) models typically treat the deepest representation of a
vision-
language backbone as universally optimal for action prediction. However, robotic manipulation is composed of many frequent closed-loop spatial adjustments, for which excessive abstraction may waste computation and weaken low-level
geometric cues essential for precise control. Existing early-exit strategies attempt to reduce computation by stopping at predefined layers or applying heuristic rules such as action consistency, but they do not directly answer when a representation is actually sufficient for action. In this paper, we present LoopVLA, a recurrent VLA architecture that jointly learns representation refinement, action prediction, and sufficiency estimation. LoopVLA iteratively applies a shared
Transformer block to refine
multimodal tokens, and at each iteration produces both a candidate action and a sufficiency score that estimates whether further refinement is necessary. By sharing parameters across iterations, LoopVLA decouples refinement from absolute layer indices and grounds sufficiency estimation in the evolving representation itself. Since sufficiency has no direct supervision, we introduce a self-supervised distribution
alignment objective, where intermediate confidence scores are trained to match the relative action quality across refinement steps, thereby linking sufficiency learning to policy optimization signals. Experiments on LIBERO, LIBERO-Plus, and VLA-Arena show that LoopVLA pushes the efficiency-performance frontier of VLA policies, reducing parameters by 45% and improving inference throughput by up to 1.7 times while matching or outperforming strong baselines in task success.
Read the full abstract ▾
All authors Boyang Shen, Kaixiang Yang, Hao Wang, Qiuyu Yu, Qiang Xie, Qiang Li, Zhiwei Wang
Robotics
Explicit Stair Geometry Conditioning for Robust Humanoid Locomotion
Jianguo Zhang, Wentai Xu, Shusheng Ye et al. (+5 more)
Robust
humanoid stair climbing remains challenging due to
geometric discontinuities, sensitivity to step height variations, and perception uncertainty in real-world environments. Existing learning-based locomotion policies often rely on implicit terrain representations or blind proprioceptive feedback, limiting their ability to generalize across varying stair geometries and to anticipate required gait adjustments. This paper proposes an explicit stair geometry conditioning framework for robust
humanoid stair climbing. Instead of encoding terrain as high-dimensional latent features, we extract a compact set of interpretable
geometric parameters, including step height, step depth, and current yaw angle relative to the
robot heading. These explicit stair parameters directly condition a Proximal Policy Optimization (PPO)-based locomotion policy, enabling proactive modulation of swing-foot clearance and stride characteristics according to stair structure. Simulation experiments demonstrate improved generalization across unseen stair heights beyond the training distribution. Real-world experiments on the Unitree G1
humanoid validate reliable indoor and outdoor stair traversal. In challenging outdoor scenarios, the
robot successfully ascends 33 consecutive steps without failure, demonstrating robustness and practical deployability.
Read the full abstract ▾
All authors Jianguo Zhang, Wentai Xu, Shusheng Ye, Yuxiang He, Weimin Qi, Qinbo Sun, Ning Ding, Liguang Zhou