Robotics
SG-WAM: Self-Guided World Modeling in Geometry-Aware Policy Space
Ruiteng Zhao, Zhengshen Zhang, Yue Su et al. (+6 more)
World Action Models (WAMs) couple action generation with prediction of future states. Their effectiveness depends on whether future dynamics are modeled in a space that is both aligned with action generation and sufficiently geometry-aware to capture where and how actions change the scene. Existing WAMs typically satisfy only part of this requirement, relying on either perceptually heavy observation-space targets or auxiliary latent spaces that are not jointly structured for action relevance and geometry. We propose SG-WAM, a self-guided framework that learns geometry-aware action-conditioned dynamics directly in the policy-derived representation space. SG-WAM introduces learnable dynamics
tokens and a Self-Guided World Predictor that forecasts their future latent states conditioned on intervening
robot actions. Prediction targets are generated by an exponential moving average copy of the same policy backbone, providing stable supervision within the representation family used by the action expert.
Geometric supervision further structures the policy image-
token representations, providing spatially grounded context for the dynamics
tokens and yielding a future-
alignment space that is both action-relevant and geometry-aware. Latent future prediction,
geometric grounding, and flow-matching action generation are jointly optimized end-to-end in a unified framework. Built on a 0.9B model without large-scale
embodied pretraining, SG-WAM achieves 98.5% average success on LIBERO and 73% on LIBERO-Plus, while outperforming strong baselines in both in-distribution and out-of-distribution real-world evaluations.
Read the full abstract ▾
All authors Ruiteng Zhao, Zhengshen Zhang, Yue Su, Wenshuo Wang, Jiahui Li, Zhiyuan Yang, Francis E.H. Tay, Marcelo H. Ang Jr., Haiyue Zhu
Robotics
DreamTrajectory: Trajectory-Guided Action Generation with World Model Alignment for Mobile Manipulation
Zheng Yang, Wenjie Zhang, Xiangyu Chen et al. (+9 more)
Mobile manipulation requires a
robot to coordinate base and arm motion under continuously changing viewpoints and contact conditions, within an action space far larger than that of fixed-base manipulation. Existing
Vision-
Language-Action (VLA) policies are limited in two respects. (i)They map observations directly to whole-body action chunks, searching this large action space without an explicit task-space motion plan, which makes coordinated base–arm prediction imprecise. (ii)They execute the predicted chunk open-loop, without checking whether the actions can realize the motion the policy intended, so control errors and unmodeled contacts accumulate into a gap between planned and realized motion. We present DreamTrajectory, a trajectory-guided framework for
language-conditioned mobile manipulation that introduces one component for each limitation. Addressing(i), DreamTrajectory jointly predicts an intention-level end-effector trajectory and a whole-body action chunk in a single action expert, so that the trajectory explicitly guides base–arm action generation instead of remaining implicit. Addressing(ii), a lightweight trajectory world model predicts the trajectory that a candidate action chunk would induce, and a test-time search–predict–score procedure selects the candidate best aligned with the planned trajectory. On MS-HAB, trajectory guidance raises average success from 32.3% to 47.5% and test-time refinement further to 54.8%, with the largest gains on contact-rich articulated-object tasks. On three real-world mobile manipulation tasks, the corresponding average success rates are 63.3%, 81.7%, and 90.0%.
Read the full abstract ▾
All authors Zheng Yang, Wenjie Zhang, Xiangyu Chen, Wenxuan Song, Xianpeng Wang, Yihang Kang, Jiawen Wen, Wen Chen, Lujia Wang, Renjing Xu, Haoang Li, Xiaowen Chu
Robotics · Cross-listed from math.OC
Rake-Compress Riccati Recursions for Parallel Scenario-Tree Model Predictive Control
João Sousa-Pinto
Scenario-tree model predictive control (MPC) represents future information by a rooted tree and optimizes a nonanticipative policy over that tree. Numerical methods for solving the resulting nonlinear program typically compute their search directions through a sequence of branched linear-quadratic regulator (LQR) subproblems. The standard tree Riccati recursion requires linear work but has a dependency chain proportional to tree height. We present an algebraically exact parallel solver based on rake-compress tree contraction. After independent local control condensation, its two operations act on node and edge data that represent conditional quadratic functions. A rake eliminates a leaf and its parent edge, adding their reduced contribution to the parent-node data. A compress eliminates a unary node and replaces its two adjacent edges by one edge, using the same conditional-value composition as parallel Riccati methods on a chain. Together they contract an arbitrary rooted tree to its root; reversing the contraction recovers every Riccati coefficient, state, control, and multiplier. Given a reusable
topology plan, a solve with $N$ nodes and fixed state and control dimensions has $O(N)$ arithmetic work and storage and $O(\log N)$ span, independently of tree height, balance, and maximum out-degree. The formulation allows positive-semidefinite dual regularization, including the unregularized case, and an exact linear-size lifting covers the standard scenario-MPC convention of one control per information node. We prove the contraction identities and equivalence to the Karush-Kuhn-Tucker (KKT) system. Three MIT-licensed JAX packages implement the bidirectional contraction, the dual-regularized LQR solver, and a user-facing primal-dual interior-point solver for tree-structured optimal control.
Read the full abstract ▾
Robotics
Hermite Curves as Trajectory Priors for Vision-Language-Action Models
Qi Lv, Jianming Xing, Zhao Yang et al. (+5 more)
Despite recent progress in
Vision-
Language-Action (VLA) models for robotic manipulation, the action chunk remains a weakly structured interface. Existing work typically flatten each chunk into per-timestep controls, relying on implicit data learning that manifests as jagged motion and boundary discontinuities during physical execution. To address these limitations, we introduce Hermite trajectory priors, parameterizing the chunk trajectory as a piecewise cubic Hermite curve defined by endpoint positions and velocities to explicitly enforce smoothness and continuity. We instantiate this fixed operator across discrete autoregressive and continuous generative paradigms via three variants: (1) Hermite
Tokens, which predict quantized boundary variables autoregressively; (2) Hermite Scaffold, which decomposes clean actions into a base scaffold and residuals; and (3) Hermite Regularization, which applies the prior strictly as an auxiliary training objective. Across simulation
benchmarks and real-
robot platforms, Hermite Regularization achieves superior performance among these three variants, improving {\pi}0.5 baseline success rates from 95.9% to 98.7% on LIBERO, 85.7% to 90.9% on LIBERO-plus, and 63.4% to 90.0% across four real-
robot tasks without additional inference overhead. Trajectory analyses reveal that explicitly structuring trajectory priors serves most effectively as a learning inductive bias rather than a runtime constraint.
Read the full abstract ▾
All authors Qi Lv, Jianming Xing, Zhao Yang, Mingyuan Yao, Yinan Shi, Yawei Jueluo, Mike Zheng Shou, Xiang Deng
Robotics
EndoWAM: A Grounded World-Action Model for Generalizable Endoscopic Navigation
Jinsong Lin, Zikang Pan, Wanhao Liu et al. (+10 more)
Autonomous endoscopic
navigation can reduce clinicians' operational burden, yet robust control remains challenging due to tissue deformation, transient occlusions, and rapidly changing viewpoints. Existing learning-based policies typically predict actions from current observations without explicitly modeling future dynamics, limiting their robustness and reliability in safety-critical settings. World Action Models (WAMs) offer a promising alternative by coupling predictive visual dynamics with action generation, but extending them to robotic endoscopy remains challenging due to limited training data, restricted viewpoint diversity, deformable anatomy, and high inference latency. We present EndoWAM, which is, to our knowledge, the first WAM for generalizable robotic endoscopic
navigation. EndoWAM introduces future grounding, which predicts task-relevant target regions in future observations from intermediate denoising features of a
video world model. Specifically, EndoWAM couples a lightweight
diffusion transformer for future target-region prediction with a discrete action expert through a shared predictive representation. This design injects target-aware supervision into predictive dynamics modeling, improving robustness to visual degradation and viewpoint changes while enabling real-time control in a single denoising pass. We further introduce EndoMotion, a robotic endoscopic motion
dataset spanning three anatomically distinct procedures: ureteroscopy, esophagoscopy, and endoscopic retrograde cholangiopancreatography (ERCP). EndoWAM consistently outperforms all baselines and alternative grounding strategies, while demonstrating strong zero-shot generalization to unseen viewpoints, environments, and targets. These results establish EndoWAM as a predictive, target-grounded framework for accurate, generalizable, and long-horizon
navigation in visually constrained endoscopic environments.
Read the full abstract ▾
All authors Jinsong Lin, Zikang Pan, Wanhao Liu, Chi Kit Ng, Liangjing Shao, Zihang Yu, Ziyu Wang, Yin Wang, Jiaxi Wang, Jeremy Yuen-Chun Teoh, Zhiyong Xiong, Huxin Gao, Hongliang Ren
Robotics
PRISM: Privileged Probabilistic Latent Supervision for End-to-End Autonomous Driving Motion Planning
Volodymyr Havrylov, Faris Janjoš, Andreas Look et al. (+2 more)
End-to-end autonomous driving (E2E AD) systems integrate perception, prediction, and planning into a single differentiable architecture. While these models show great promise, their standard training often relies on output-only supervision, which can lead to weak gradients for the hidden layers of increasingly complex models. Recent works have integrated
vision-
language model (VLM) supervision for latent features to address this, yielding substantial empirical gains, yet leaving the underlying theoretical mechanisms poorly understood. Our investigation into this methodology reveals that the resulting performance gains stem not from VLM
reasoning capabilities, as previously assumed, but rather from the latent connections forged between the E2E AD model and ground-truth (GT) data during training. Building on this insight, we propose a probabilistic deep supervision framework that regularizes intermediate latent representations directly from GT data. By treating model latents as reparameterizable distributions, we optimize the architecture via the Evidence Lower Bound (ELBO). Our evaluations conducted on the nuScenes
dataset demonstrate that supervising trajectory-related latents with future GT paths consistently improves planning performance. Using identical training data and E2E architectures, our method achieves an 8% reduction in planning L2 error and a 3% decrease in collision rates compared to competitive vectorized baselines, all while incurring negligible computational overhead.
Read the full abstract ▾
All authors Volodymyr Havrylov, Faris Janjoš, Andreas Look, Jürgen Mathes, Andreas Geiger
Robotics
Complete Motion Planning using Workspace-Fibered Decomposition for nR-Planar Manipulator
Aayush Rath and Antony Thomas
We propose a workspace-fibered decomposition framework for motion planning in nR planar redundant manipulators operating in cluttered environments. Rather than planning directly in the full n-dimensional configuration space, the method incrementally constructs obstacle-constrained reachable workspaces of lower-dimensional non-redundant sub-chains and recursively lifts them through redundant orientation fibers. This yields a sequence of reduced planning
manifolds that preserve branch-consistent reachability structure while avoiding explicit construction of the full configuration-space obstacle geometry. We first establish that, for planar position-only manipulators, the obstacle-constrained reachable workspace induced by the minimal non-redundant sub-chain provides an exact characterization of feasibility with respect to the connected component of the start configuration, enabling early infeasibility detection prior to introducing redundant degrees of freedom (DOF). We then introduce an incremental fiber-lifting procedure that propagates reachable workspace structure through successive redundant links while enforcing local inverse-kinematic branch consistency using Jacobian determinant continuity constraints. The resulting representation admits efficient reduced-space planning directly on recursively-constructed workspace-fiber
manifolds. Experimental results on redundant nR planar manipulators demonstrate that the proposed construction preserves collision-free connectivity structure across successive lifting stages while substantially reducing collision checking complexity relative to direct configuration space
reasoning.
Read the full abstract ▾
Robotics
FeDepth: Federated Learning for Depth Estimation under Robot Heterogeneity
Ganghyeon Lee, Inha Lee, Junhee Lee et al. (+3 more)
Although recent
robot perception research emphasizes training on data from diverse environments to improve generalization, most existing methods still rely on centralized learning, which is inefficient and difficult to scale across heterogeneous
robot platforms.
Federated learning (FL) offers an alternative by enabling distributed training without raw data transfer, but it suffers from severe performance degradation under domain shifts caused by heterogeneity across clients. In real robotic deployments, data distributions often overlap across platforms, environments, and sensing conditions, making it difficult to partition clients into clearly separated domains. However, this characteristic breaks the assumption of clearly separable client domains commonly used in clustered FL. To address this gap in
robot perception, particularly in depth estimation, we introduce two realistic and unexplored non-IID scenarios that reflect heterogeneity in terms of platform, environment, and depth distribution. We then propose FeDepth, a descriptor-based clustered FL framework that models client relationships through soft clustering. Unlike hard clustering methods that assume clearly separated clusters, FeDepth allows clients to participate in multiple clusters, capturing continuous and ambiguous domain transitions commonly observed in robotic environments. Extensive experiments demonstrate that FeDepth consistently improves robustness over standard FL and clustered FL baselines across multiple depth estimation architectures, providing a practical and effective solution for
federated robot perception. Our project page is available at https://
vision3d-lab.github.io/fedepth/.
Read the full abstract ▾
All authors Ganghyeon Lee, Inha Lee, Junhee Lee, Jeongeon Lee, Sung Whan Yoon, Kyungdon Joo
Robotics
CAAT: Contact-Aware Attention Scaling and Tactile Masking for Data-Efficient Contact-Rich Manipulation
Jiaming Jiang, Yuzhe Huang, Hao Liang et al. (+7 more)
In contact-rich manipulation, visual observations primarily guide motion in free space, whereas tactile observations become particularly informative during contact. However, standard
Transformer-based visuo-tactile policies typically rely on either
token concatenation or learnable gating. These approaches lack explicit contact-aware priors, making it difficult to efficiently learn effective cross-modal representations from demonstrations. To address this limitation, we propose CAAT, a lightweight contact-aware framework that explicitly incorporates contact priors through
attention scaling and dynamic tactile masking. Specifically, CAAT emphasizes visual information before contact and tactile information during contact. It also suppresses static background
tokens by comparing the current tactile observation with a non-contact reference. CAAT can be integrated into commonly used
Transformer-based policies without modifying their action decoders. In simulation, integrating CAAT with ACT improves the average success rate by 18.0 percentage points over direct visuo-tactile fusion and by 10.0 percentage points over gated fusion. In real-world experiments using a visuo-tactile UMI platform, CAAT achieves an average success rate of 60.0% across ACT,
Diffusion Policy, and $\pi_0$, outperforming the strongest baseline by an average of 21.1 percentage points. These results demonstrate that explicit contact priors and dynamic tactile masking are effective in improving visuo-tactile policy learning and task performance of diverse policy architectures. https://mrjiangjm.github.io/caat/
Read the full abstract ▾
All authors Jiaming Jiang, Yuzhe Huang, Hao Liang, Pei Lin, Shengcheng Luo, Fanrong Dong, Jiaping Wu, Chenxi Xiao, Wanlin Li, Ziyuan Jiao
Robotics
Sparse Meets Dense: Correspondence Guided Robotic Manipulation with Rigid-Deformable Interactions
Ziyu Zhu, Yue Chen, Xirui Liang et al. (+5 more)
Manipulation involving rigid-deformable interactions, such as hanging clothes or dressing humans, is common in daily life, making it essential for household
robots. Compared to single-object manipulation or interactions between rigid bodies, these tasks are particularly challenging due to the rich multi-point contacts and the complex dynamics of the deformable bodies during interaction. Therefore, object-centric representations such as 6D poses or structural points without task-specific information become insufficient for these interactions. In this work, we propose a hybrid correspondence-based representation tailored for rigid-deformable interactions. First, to capture intricate interaction information, we introduce structure-, task-, and interaction-aware sparse keypoints. The keypoints are generated based on the global structures of both rigid and deformable objects, and filtered by their local interaction contacts. However, tracking these sparse keypoints through the interaction remains difficult due to the high-dimensional dynamics of deformable objects. Therefore, we further construct dense correspondences on the deformable objects for accurate keypoint tracking throughout the manipulation. This hybrid design combines the advantages of both representations: sparse keypoints encode rich, task-specific information for fine-grained manipulation, while dense correspondences ensure efficient tracking and generalization to novel deformations, shapes, and scenarios. Together, they enable one-shot transfer to new tasks with minimal demonstrations. Extensive experiments demonstrate the effectiveness and broad applicability of our method.
Read the full abstract ▾
All authors Ziyu Zhu, Yue Chen, Xirui Liang, Hojin Bae, Yuran Wang, Zhen Yuan, Ruihai Wu, Hao Dong
Robotics
VespaSeg: A Resource-Aware Ground-then-Segment Pipeline for Referring Expression Segmentation
Savindu Dilshan Wickramasinghe
Referring expression
segmentation requires
language conditioned localization and pixel-accurate masks, but monolithic models can be costly to deploy. We present VespaSeg, a modular pipeline that grounds a text query with a compact
vision-
language model and converts the predicted box to a mask with MobileSAM. We study Florence-2-base, Florence-2-large, and Moondream2 grounders together with targeted adaptation of the grounding and
segmentation stages. Under a repository-specific RefCOCO validation protocol containing the first expression for each of 3,811 referenced-object records, the adapted Florence-2-base pipeline obtains 73.64 mean intersection over union (mIoU) and 84.60 precision at IoU 0.5. On an NVIDIA RTX 6000 Ada GPU it processes 22.8 cached-image queries per second with 2.20 GB mean allocated GPU
memory. A matched 500-query comparison gives 73.73 mIoU for Florence-2-base and 72.82 for Florence-2-large, while the base model is 1.70 times faster and uses 1.17 GB less allocated
memory. Ablations show that ground-truth-box adaptation raises MobileSAM mIoU from 82.22 to 86.61 and that reducing the Florence-2 output-
token budget from 64 to 32 preserves accuracy. These results support compact, modular grounding and
segmentation, while also exposing the need for evaluation on the complete standard RefCOCO expression splits and deployment hardware.
Read the full abstract ▾
Robotics
OC-VLA++: Monocular Geometry-Guided Cross-View Consistency for Viewpoint-Robust Robotic Manipulation
Tianyi Zhang, Ziyang Gong, Zhenjie Yang et al. (+2 more)
We propose OC-VLA++, an extension of OC-VLA for viewpoint generalization under limited camera coverage. While OC-VLA grounds
robot actions in the camera coordinate system to align action supervision with visual observations, camera-space grounding alone can still overfit to the few viewpoints observed during training. OC-VLA++ addresses this limitation by introducing geometry-guided paired-view supervision and an explicit cross-view action-equivariance objective. Given paired observations of the same manipulation scene from geometrically related viewpoints, the model is trained such that their camera-space predictions correspond to the same
robot-frame action. This objective explicitly supervises how action predictions should transform across viewpoints, rather than relying solely on image-level augmentation. Experiments demonstrate substantial improvements in unseen-view generalization under limited camera coverage, with performance degrading more gracefully under increasing camera displacement. These results establish cross-view action equivariance as an effective complement to observation-centric action grounding for robust real-world deployment.
Read the full abstract ▾
All authors Tianyi Zhang, Ziyang Gong, Zhenjie Yang, Zhe Qian, Haonan Duan
Robotics
WAM-Diff2: Hierarchical AR-to-Diffusion Distillation for Highly Efficient Autonomous Driving VLA
Zhihao Zhu, Hanlin Shang, Mingwang Xu et al. (+6 more)
Vision-
Language-Action (VLA) models have emerged as a prominent paradigm for end-to-end autonomous driving; however, their efficient deployment is severely constrained by high computational latency and exposure bias arising from sequential autoregressive decoding. Conversely, while specialized
diffusion policies enable low-latency, parallel execution, training them from scratch typically yields narrow, single-task architectures that lack holistic visual-linguistic
reasoning. Successfully transforming pre-trained autoregressive generalists into parallel
diffusion models could combine multi-task cognitive intelligence with execution efficiency, yet this transition presents a formidable architectural challenge due to mismatched
attention patterns (causal versus bidirectional) and divergent optimization objectives. To bridge this divide, we introduce WAM-Diff2, a multi-task discrete
diffusion VLA framework powered by a three-stage hierarchical
distillation strategy. By structuring the architectural shift through progressive block-wise adaptation, block-wise
distillation, and model-wise cross-scale
distillation, WAM-Diff2 preserves the underlying semantic foundations of the base model while accelerating inference. Extensive evaluations across driving understanding, perception, and planning
benchmarks demonstrate that WAM-Diff2 effectively mitigates exposure bias and achieves performance parity with autoregressive baselines. Crucially, the autoregressive-to-
diffusion transition yields a 2.8x decoding speedup, which scales to an ultimate 15.1x acceleration when combined with system-level optimizations including FlashInfer and CUDA Graphs.
Read the full abstract ▾
All authors Zhihao Zhu, Hanlin Shang, Mingwang Xu, Feipeng Cai, Zhuolin He, Yaoyi Li, Jianhua Han, Hang Xu, Siyu Zhu
Robotics
Diffusion-Based Body Schema Learning Enabling Abnormal-State Adaptation in Musculoskeletal Robots
Kento Kawaharazuka and Shuhei Ikemoto
Musculoskeletal
robots require an internal body schema that remains consistent under a wide range of physical state changes, including abnormalities such as muscle rupture and actuator jamming. Conventional approaches based on autoencoders or variational autoencoders learn average behaviors by projecting sensor and actuator signals into a low-dimensional latent space; however, exploration within the latent space alone has limited capability to handle out-of-distribution or abnormal states that are not included in the training data. To address this limitation, this study proposes a
diffusion-based framework for body schema learning in musculoskeletal
robots. Unlike generative models that operate through low-dimensional latent spaces,
diffusion models can directly and iteratively estimate physically consistent sensor and actuator values in the high-dimensional space through a denoising process, even under partial observations and constraints, without requiring retraining. By formulating body schema adaptation as a gradient-guided denoising process, the proposed method enables adaptive estimation of appropriate muscle lengths and muscle tensions even under abnormal conditions such as muscle rupture and actuator jamming. The validity of the proposed framework is verified through simulation experiments using a musculoskeletal
robot model.
Read the full abstract ▾
Robotics
VLAGuard: A Framework for Evaluating and Mitigating Physical Attention Hijacking in Vision-Language-Action Robots within Wireless Sensor Networks
Dongfu Yin and Jinquan Zhang
Deploying
Vision-
Language-Action (VLA)
robots as mobile edge nodes within wireless sensor networks (WSNs) requires robust protection against physical
adversarial threats. We present VLAGuard, a framework to assess and mitigate a critical vulnerability: policy-critical action-to-
vision attention hijacking. We first introduce a stress-test module, Visuomotor
Attention-guided Semantic Attack (VASA), using printable patches to severely distract the
robot's action-conditioned cross-
attention. To counter this, we propose
Attention-Protective Fine-Tuning (APFT), a defense that stabilizes spatiotemporal
attention and enforces
geometric consistency with zero inference overhead. Evaluations across simulated and physical WSN-assisted smart environments demonstrate significant robustness gains. APFT reduces the OpenVLA failure rate from 100.0% to 25.9% in LIBERO simulations. Furthermore, across 2,000 real-world trials, APFT improves the average success rate from 23.0% to 67.4% under severe patch attacks. This highlights that protecting
attention pathways is important for improving the robustness of VLA-driven edge nodes in sensor networks.
Read the full abstract ▾
Robotics
Sampling-Based Visibility Task Planning
Stav Ashur and Avishai Sintov
Robot Task and Motion Planning (TAMP)
algorithms enable autonomous operation by incorporating the specific functions and constraints of end-effector tools, such as grippers or soldering irons, directly into the planning process. In this paper, we explore sampling-based TAMP
algorithms specifically designed for a critical subset of devices whose unique properties make traditional planning methods ineffective. Visibility-based instruments, such as exteroceptive sensors, cameras, flashlights and directional
antennas, are essential across a vast array of human activities. The unique properties of these devices, and particularly, their field-of-view, render many widely used heuristics and distance metrics less effective. We introduce two new sampling-based
algorithms, FOV-PRM and FOV-RRT, designed to tackle visibility-based tasks. FOV-PRM employs a hierarchical decomposition of the environment, leveraging the concept of visibility integrity, to efficiently sample configurations with a clear line-of-sight to the target. A specialized Inverse Kinematics solver enables FOV-RRT to "glance" in the direction of the target at opportune moments, facilitating the rapid discovery of key configurations. We show that FOV-PRM and FOV-RRT achieve a higher success rate and faster runtimes compared to adaptations of RRT, PRM and VIR, through both simulated and physical experiments.
Read the full abstract ▾
Robotics · Cross-listed from cs.MA
Stress-Relief Annealing: Polynomial-Time Simulation-Free Layout Optimization for Automated Warehouses
Xiangjie Luo, Yulun Zhang, Miyuki Koshimura et al. (+2 more)
We study the problem of optimizing physical layouts for automated warehouses, where hundreds to thousands of
robots are coordinated to transport packages. Previous works have shown that optimizing the warehouse layout (e.g., the physical location of the storage shelves) significantly improves throughput. However, state-of-the-art layout optimization approaches are based on evolutionary optimization methods, which treat the entire warehouse as a black box and rely on random mutation to search for high-quality layouts. While the optimization outcomes are promising, these methods require a massive number of simulations to evaluate candidate solutions, making them sample-inefficient. In this paper, we present Stress-Relief Annealing (SRA), a
polynomial-time simulation-free layout optimization
algorithm. SRA turns the task demand into a per-vertex stress field that predicts where traffic will concentrate in the warehouse; the field's peak provably caps the throughput. Our experimental results show that (1) SRA improves both the throughput and the scalability of a human-designed warehouse, roughly doubling the number of
robots it can sustain, (2) it matches or exceeds the throughput of the evolutionary baselines while taking only $19$ minutes on one CPU core, against their $25{,}000$ simulations and $25$ hours on a $64$-core machine, and (3) the gain generalizes across different Multi-
Agent Path Finding
algorithms, non-uniform task demands, and a warehouse with doubled dimensions.
Read the full abstract ▾
All authors Xiangjie Luo, Yulun Zhang, Miyuki Koshimura, Makoto Yokoo, Jiaoyang Li
Robotics
KING: Embodiment-Aware Kinematic Graph Neural Network for Unified Motion Representation of Legged and Wheeled Robots
Taku Okawara, Aoki Takanose, Kenji Koide et al. (+2 more)
Kinematic models provide reliable motion constraints for odometry estimation in featureless environments, where exteroceptive sensing degrades and IMU integration drifts. Learning-based kinematic models can achieve more accurate odometry estimation than model-based methods by capturing nonlinear effects; however, most existing learning-based models are trained on a single embodiment and generalize poorly to new embodiments. This generalization is difficult because the meanings and structures of proprioceptive measurements vary across embodiments, including the number of joints and ground-contact elements (e.g., wheels, feet). To address this challenge, we propose KING, a Graph Neural Network (GNN)-based kinematic model that explicitly incorporates
robot embodiments by representing them as a common graph. We show that wheel and leg kinematic models can be expressed by a unified representation, enabling a single model for both wheeled and legged
robots. Trained on
datasets spanning diverse embodiments, KING provides a unified representation of wheeled and legged kinematics and achieves high-accuracy odometry estimation in real environments. KING estimates accurate odometry using only an embodiment description (e.g., a URDF file) and on-board proprioception (encoders and an IMU) and can be adapted to new
robot embodiments through few-shot learning with only one minute of data, avoiding retraining from scratch on a new
dataset for each
robot. The project page is available at: https://smrg-aist.github.io/king_project_page/
Read the full abstract ▾
All authors Taku Okawara, Aoki Takanose, Kenji Koide, Shuji Oishi, Masashi Yokozuka
Robotics
RL Bootstrapping of OpenVLA-OFT for a Novel Robot Embodiment
Damir Nurtdinov, Alexei Kornaev and Alexander Maloletov
Adapting a pretrained
vision-
language-action (VLA) policy to a new
robot usually assumes embodiment-specific demonstrations. This assumption is especially restrictive for custom
robots whose morphology differs strongly from the manipulators seen in large
robot datasets. We study a harder setting: zero-demo embodiment
alignment of OpenVLA-OFT on a cable-driven parallel
robot (CDPR) with a simple gripper and a previously unseen control interface. Instead of supervised fine-tuning, we use reinforcement learning in simulation with dense
geometric rewards computed from simulator state. The training is performed in two stages: a PPO stage for directional motion primitives, followed by GRPO continuation from the PPO checkpoint with an expanded instruction space that includes object-conditioned commands. On the four shared directional instructions, the average held-out success rate improves from 34.25% after PPO to 53.50% after PPO$\rightarrow$GRPO, with especially large gains on move left and move backward. In the GRPO stage we additionally introduce move to <object> over eight target objects and obtain 39/400 = 9.75% strict success, while qualitative rollouts frequently show correct target-directed approach behavior before late-stage instability. Compared with prior OpenVLA and OpenVLA-OFT results, which rely on demonstration
datasets and mostly standard rigid-arm embodiments, our method uses no embodiment-specific
dataset at all. The results do not yet establish robust manipulation, but they provide stronger evidence that RL-only bootstrapping can create the first usable
language-conditioned controller for a genuinely novel embodiment.
Read the full abstract ▾
Robotics
FreqNav: Stage-Wise Frequency Routing for Object-Oriented Aerial Vision-Language Navigation
Yin Tang, Jiawei Ma, Jiahao Li et al. (+4 more)
Object-oriented aerial
vision-and-
language navigation (VLN) requires searching for a described target and landing on it precisely, under long-horizon and closed-loop control. Guided by a target-descriptive instruction during
navigation, perceptual priorities dynamically evolve: early-stage exploration prioritizes low-frequency spatial layout, and then shifts to high-frequency target details. Existing VLN methods model the varying perceptual requirements across
navigation stages with identical visual
tokens, leading to interference from irrelevant objects and background clutter. To this end, we therefore formulate long-horizon aerial
navigation as a frequencypreference shift from spatial structure to local detail and propose FreqNav, a lightweight frequency-routing adaptive perception framework. Under a fixed computational budget, FreqNav dynamically reallocates visual
tokens across frequency components according to the current
navigation stage. A Frequency
Token Router selects stage-relevant visual representations from dual-view observations, while a Phase-dependent Grounding Module anchors visual evidence through explicit supervision. A
Diffusion Transformer then predicts smooth trajectories for continuous control. Experiments show that FreqNav outperforms strong baselines while achieving approximately 3x faster inference. Real-world deployment further demonstrates its effectiveness, efficiency, and practical potential for long-horizon aerial autonomy.
Read the full abstract ▾
All authors Yin Tang, Jiawei Ma, Jiahao Li, Hao Zhang, Zhemin Sun, Jianqiao Sun, Deyu Zhang
Robotics · Cross-listed from cs.GT
MixedComplementarityProblems.jl: A Fast, Batched, Open-Source Interior Point Solver for Mixed Complementarity Problems
David Fridovich-Keil
Mixed complementarity problems (MCPs) arise as the first-order optimality conditions of nonlinear programs and noncooperative games, and provide a natural formulation for multi-
agent trajectory optimization problems that appear throughout robotics. The dominant solver for problems of this form is PATH, which offers strong performance on robotics problems but remains closed-source. We present MixedComplementarityProblems.jl, an open-source, pure Julia implementation of an interior point method for parametric MCPs that: (i) matches PATH's reliability on standard
benchmarks, (ii) natively supports batched, parallel processing of many parameter instances, either across CPU threads or on an NVIDIA GPU, and (iii) supports efficient automatic differentiation of solutions with respect to problem parameters. On a multi-
agent lane-change trajectory game representative of robotics planning problems, our CPU-multithreaded batched solver clears a batch of parametric instances ~100x faster than sequential calls to PATH. A GPU backend, running the same solver implementation unmodified, also clears these batches far faster than PATH, but does not outperform the multithreaded CPU on this problem; the GPU pulls ahead only once each per-instance KKT system grows large, and we characterize this regime dependence. We describe the solver's interior point formulation, the abstraction that lets a single solver implementation run unmodified across dense, batched-sparse, and single-large linear-
algebra backends, and report
benchmarks against PATH on both randomly generated quadratic programs and trajectory games.
Read the full abstract ▾
Robotics · Cross-listed from cs.CV
Swimm3R: Splatting with Medium-aware SfM for Underwater 3D Reconstruction
Minseong Kweon and Junaed Sattar
We propose Swimm3R, a unified framework that combines medium-aware structure-from-motion (SfM) with
Underwater Beta Splatting to address scattering- and attenuation-induced failures in
underwater 3D reconstruction. Swimm3R distills in-air
geometric priors into a feed-forward backbone and uses a physics head to regress
underwater image-formation parameters, camera poses, and restored point clouds. Additionally, we introduce
Underwater Beta Splatting, which extends Gaussian splatting with Beta primitives and scattering-aware
geometric gradients for stable
underwater geometry representation. We further establish the Barbados
underwater video dataset to demonstrate the effectiveness of our method in challenging
underwater environments. On this
dataset, Swimm3R robustly recovers
underwater scene structure under challenging scattering conditions, yielding coherent seafloor geometry. Using these predicted point clouds, the proposed
Underwater Beta Splatting improves average PSNR by $1.47$ dB over WaterSplatting while increasing downstream localization performance by $2.0$ and $2.4$ percentage points in RRA@15 and RTA@15, respectively.
Read the full abstract ▾
Robotics
GraRe: Grasp Candidate Re-Ranking for Frozen 6-DoF Grasp Detectors
Jibao Yuan, Yuhui Zhao, Yinzhen Lv et al. (+4 more)
Existing 6-DoF grasp detectors typically rank grasp candidates by detector confidence. However, our analysis on GraspNet-1Billion shows that detector confidence is often poorly aligned with grasp quality, leaving successful grasp candidates at low ranks. Motivated by this observation, we study whether learned re-ranking can improve candidate ordering while keeping detector parameters and grasp candidates unchanged. We propose GraRe, which estimates grasp quality from candidate attributes, shell-stratified local geometry, and object context. Candidate attributes condition the local
geometric and object-context representations, and a
Transformer fuses all three feature types. The predicted quality is combined with detector confidence to produce the final ranking. Experiments on GraspNet-1Billion with five frozen detectors show consistent improvements, with gains of up to 13.56 points in Average AP. Real-
robot experiments further demonstrate robust grasping in cluttered scenes. These results show that improving candidate ranking provides a practical way to enhance frozen 6-DoF grasp detectors. Project code is available at https://minakanmi-yuki.github.io/grare/.
Read the full abstract ▾
All authors Jibao Yuan, Yuhui Zhao, Yinzhen Lv, Chao Xu, Shun Li, Chenxi Deng, Shaofei Chen
Robotics
VertiAKD: Adaptive Off-Road Kinodynamics on Vertically Challenging Terrain
Tong Xu, Chenhui Pan, Francesco Cancelliere et al. (+1 more)
Off-road mobility requires autonomous mobile
robots to generalize across heterogeneous vehicle fleets and continuously changing terrain conditions. Existing cross-vehicle adaptation approaches generally assume flat terrain, while terrain-aware kinodynamic models often require platform-specific data collection and retraining. To this end, we propose VertiAKD, a unified framework for transferring and adapting off-road kinodynamic knowledge across diverse vehicles on geometrically and semantically complex terrain simultaneously. VertiAKD learns a shared mobility representation that jointly encodes vehicle configurations, trajectory transitions, and local elevation and semantic terrain features. Given limited data from a novel vehicle operating on unseen terrain, VertiAKD identifies the most relevant mobility descriptors and transfers their knowledge to initialize a terrain-aware kinodynamic model via function encoders, which is then periodically refined online from streaming observations without gradient-based retraining. We evaluate VertiAKD in the Verti-Bench simulator, built on the Chrono multi-physics engine, and on five physical configurations of the Verti-4-Wheeler platform. With only one minute of new trajectory data and associated terrain features, VertiAKD reduces long-horizon prediction error by up to 34.52% over direct mobility descriptor transfer across diverse unseen vehicle configurations and 94.43% over competing baselines. We further demonstrate robust closed-loop trajectory tracking in both simulation and physical experiments, highlighting the effectiveness of terrain-aware cross-vehicle knowledge transfer for accurate modeling and reliable off-road
navigation.
Read the full abstract ▾
All authors Tong Xu, Chenhui Pan, Francesco Cancelliere, Xuesu Xiao
Robotics
Stipple: Real-Time Incremental Gaussian Splatting with Visual-Inertial Tracking
Kilian Northoff, Mateo de Mayo and Daniel Cremers
3D Gaussian Splatting (3DGS) provides efficient rendering of photo-realistic scenes, but its heavy preprocessing and training steps make it a poor fit for applications that require real-time reconstruction in robotics or XR. This capability is important since it allows immediate feedback and interaction with new environments. Visual-inertial odometry (VIO) and simultaneous localization and mapping (VI-SLAM) systems, on the other hand, specifically target these real-time applications, which makes them a good choice for integration with 3DGS. We propose a new method that tracks and reconstructs simultaneously in real-time by leveraging an efficient visual-inertial tracking system based on Basalt together with a novel incremental method built on top of Brush, an efficient Rust-based GPU-vendor-agnostic implementation of 3D Gaussian Splatting. We show that many of the heavy preprocessing and training steps of 3DGS can be replaced with a more efficient incremental training strategy that has direct access to the information generated by the visual-inertial tracking system. Furthermore, we propose and combine multiple practical improvements to increase the efficiency of the training pipeline and adapt it to run in real-time, parallel to the tracking thread. This work highlights the value of exploiting the complementary nature of SLAM and 3DGS, and how that can lead to promising results for real-time 3D reconstruction.
Read the full abstract ▾