Robotics
SimFoundry: Modular and Automated Scene Generation for Policy Learning and Evaluation
Nadun Ranawaka, Josiah Wong, Wei-Lin Pai et al. (+15 more)
Training and evaluating
robot policies in the real world is costly and difficult to scale. We introduce SimFoundry, a modular and automated system for zero-shot real-to-sim scene construction from a
video. SimFoundry generates sim-ready digital twins and supports object, scene, and task editing, enabling the automated generation of diverse digital cousins: affordance-preserving variations of reconstructed real-world scenes. Policies trained on SimFoundry data transfer zero-shot to challenging real tasks involving multi-step manipulation, articulated object interaction, and bimanual interaction, and its digital cousins (variations of the original scene, objects, and tasks) facilitate generalization to new real-world conditions. Across 7 manipulation tasks and 5 policy architectures, SimFoundry simulation evaluations strongly predict real-world performance, with mean Pearson correlation 0.911 and mean maximum ranking violation 0.018. When evaluating sim-trained policies zero-shot in the real world, policies trained with object, scene, and task cousins in simulation show average task success rate improvements of 17%, 21%, and 40%, respectively. Additional details at https://research.nvidia.com/labs/gear/simfoundry/ .
Read the full abstract ▾
All authors Nadun Ranawaka, Josiah Wong, Wei-Lin Pai, Wei-Teng Chu, Tianyuan Dai, Masoud Moghani, Hang Yin, Yunfan Jiang, Wesley Durbano, Brandon Huynh, Yu Fang, Danfei Xu, Ruohan Zhang, Li Fei-Fei, Linxi Fan, Bowen Wen, Ajay Mandlekar, Yuke Zhu
Robotics
Unleashing Infinite Motion: Scaling Expressive Quadrupedal Motion via Generative Video Priors
Youzhi Liu, Li Gao, Yifei Qian et al. (+3 more)
Quadruped
robots have achieved remarkable locomotion, yet their behavioral repertoire remains confined to a few gaits–far from the expressive, companion-like presence long envisioned for them. Attempts to import the
humanoid recipe of large-scale motion data have inherited one tacit assumption: that
robot motion must first pass through an animal body, making data collection dependent on cooperative animals, reconstruction fragile across species, and retargeting ill-posed across incompatible morphologies. We propose Uni-Mo, a fully automated pipeline that removes the animal from the loop by reframing data scarcity as a generation problem: an LLM proposes motion
prompts, a
video diffusion model synthesizes the corresponding
robot behaviors, and the generated
videos are lifted into 3D reference trajectories used to train tracking policies deployed on a real Unitree Go2. To make naively-drifting generations reliably extractable, we introduce an Identity Consistency Loss that enforces appearance coherence across frames. We release Quad-Imaginarium at https://github.com/Amap-Robotics/Quad-Imaginarium, the resulting open-source
dataset of 7,488
language-annotated quadruped motions (18.5 hours) spanning acrobatic and performative behaviors. We validate 392 randomly sampled motions on a real Unitree Go2 with a 96.7% deployment success rate, complemented by a 97.6% success rate across the full
dataset in simulation.
Read the full abstract ▾
All authors Youzhi Liu, Li Gao, Yifei Qian, Liu Liu, Yang Cai, Wenbin Tang
Robotics
Learning Stable In-Grasp Manipulation in a Non-Dropping Action Space
Ha Thang Long Doan, Hikaru Arita, Kazuto Nakashima et al. (+1 more)
Traditionally, dexterous manipulation controllers are designed using analytic models constrained by strong assumptions about the hand and the objects being manipulated. Reinforcement learning (RL) has become another common approach in which skills are explored openly in an end-to-end manner but is inefficient because of unnoticeable instability and conflicts in learning objectives. This paper attempts to efficiently explore stable and accurate manipulation skills by decomposing dexterous skills into multiple simpler/analyzable components. Each skill component is subsequently learned with constraints and guidance from classical physics and control theory. Our work shows that for stable grasp, in-grasp reposition/reorientation with different objects, sensor/motor noise, latency, and frictional conditions, skill learning becomes efficient and stable with prior knowledge from theory.
Read the full abstract ▾
All authors Ha Thang Long Doan, Hikaru Arita, Kazuto Nakashima, Kenji Tahara
Robotics
PA-BiCoop: A Primary-Auxiliary Cooperative Framework for General Bimanual Manipulation
Bai Qicheng, Wang Ziru, Ma Teli et al. (+3 more)
Bimanual manipulation is essential for advanced robotic systems because it offers higher efficiency and flexibility compared to single-arm configurations. However, existing approaches either lack inter-arm interaction or ignore the need for a dynamic division of labor, treating the arms as functionally equivalent. To address these limitations, this paper draws inspiration from human bimanual manipulation where one arm handles core operations and the other provides auxiliary support, and proposes PA-BiCoop, a new single-model bimanual cooperation framework with dynamic primary-auxiliary arm differentiation. PA-BiCoop categorizes robotic arms into primary and auxiliary arms with adaptively adjustable roles across task stages, employs two specialized decoders that share a global feature encoder: the primary decoder generates the primary arm's base-coordinate pose and core-task affordance heatmaps, and the auxiliary decoder outputs the auxiliary arm's relative pose in the primary arm's coordinate system. Moreover, we design a dynamic role assignment module to automatically map roles to left/right arms without manual pre-definition. This design facilitates inter-arm knowledge sharing and coordinated manipulation. Extensive experiments demonstrate that our PA-BiCoop achieves superior performance: it outperforms state-of-the-art baselines by 48% on average in RLBench2 simulation tasks and by over 50% on average in real world tasks, thereby verifying its effectiveness and advancement in bimanual manipulation.
Read the full abstract ▾
All authors Bai Qicheng, Wang Ziru, Ma Teli, Dai Guang, Wang Jingdong, Wang Mengmeng
Robotics · Cross-listed from cs.LG
LLawCo: Learning Laws of Cooperation for Modeling Embodied Multi-Agent Behavior
Qinhong Zhou, Chuang Gan and Anoop Cherian
Embodied agents operating in decentralized and partially observable environments have attracted growing
attention in recent years. However, existing large
language model (LLM)-based
agents often exhibit behaviors that are misaligned with their partners or inconsistent with the environment state, leading to inefficient cooperation and poor task success. To address this challenge, we propose a novel framework, Learning Laws of Cooperation (LLawCo), that enables
embodied agents to autonomously align with both their partners and task objectives. Our framework allows
agents to reflect on past failures to extract misaligned behavioral patterns, which are used to derive high-level behavioral laws, such as "Talk when necessary" and "Wait for partner." These laws are explicitly incorporated into the
agents' chains of thought via supervised fine-tuning, aligning their
reasoning with task requirements and the behavior of other
agents. To evaluate our approach, we introduce PARTNR-Dialog, a large-scale multi-
agent communicative and cooperative planning
benchmark built on the PARTNR environment. Experiments on existing tasks and our new
benchmark demonstrate significant improvements in cooperative efficiency and task success rates. Across four backbone LLMs, our method achieves average success rate improvements of 4.5% on the PARTNR-Dialog
benchmark and 6.8% on the TDW-MAT
benchmark over state-of-the-art open-source communicative
agent frameworks. See the LLawCo project page for details: https://www.merl.com/research/highlights/LLawCo
Read the full abstract ▾
Robotics · Cross-listed from cs.LG
Regularized Reward-Punishment Reinforcement Learning
Jiexin Wang and Eiji Uchibe
We propose KL-Coupled Policy Regularization (KCPR), a policy coordination framework for Reward-Punishment Reinforcement Learning (RPRL). Based on KCPR, we derive KL-Coupled Soft Optimality (KCSO) and develop its deep realization, klDMP. Unlike existing RPRL approaches that optimize reward-seeking and punishment-related policies largely independently, KCPR enables direct interactions between companion policies by treating each as a dynamically learned prior for the other. KCSO yields coupled soft-optimal policies and KL-regularized Bellman operators, allowing reward and punishment information to jointly influence value propagation. To improve learning stability, we introduce a companion-prior softening mechanism and evaluate separate replay-buffer designs for balancing reward- and punishment-related experience. Experiments in grid-world and Gazebo robotic
navigation tasks demonstrate that klDMP improves safety and learning stability while maintaining competitive task performance compared with DQN, SQL and softDMP. These results suggest that policy-level coordination provides an effective mechanism for integrating multiple behavioral objectives and may serve as a useful design principle for reinforcement learning systems with interacting motivational processes.
Read the full abstract ▾
Robotics
Translation as a Bridging Action: Transferring Manipulation Skills from Humans to Robots
Sijin Chen, Kaixuan Jiang, Haixin Shi et al. (+6 more)
We study whether we can learn novel manipulation skills from human actions to a bi-manual
robot with parallel grippers. Human action data is cheap, abundant, and diverse, making it one of the most promising resources for scaling up
robot learning. Yet transferring skills from humans to
robots remains hard: most prior work treats humans as just another bi-manual 6DoF embodiment, where hand-pose estimates are noisy and the contact patterns of human fingers differ fundamentally from those of a parallel gripper. We argue that learning rotation-inclusive action signals from human data is therefore sub-optimal, and instead propose a bridging action representation: the relative wrist translation within the initial head-camera frame, an action space shared by humans and
robots. To handle the potential absence of certain action components in different embodiments, we build a $\pi_0$-like
vision-
language-action model with interleaved action
tokens and
attention masking. On a suite of novel bi-manual manipulation tasks, our bridging action transfers human manipulation knowledge to
robots far more effectively than noisy 6DoF human actions and scales with the amount of human data.
Read the full abstract ▾
All authors Sijin Chen, Kaixuan Jiang, Haixin Shi, Yanhui Wang, Weiheng Zhong, Haosheng Li, Bo Jiang, Yuxiao Liu, Xihui Liu
Robotics · Cross-listed from cs.CV
PhysisForcing: Physics Reinforced World Simulator for Robotic Manipulation
Peiwen Zhang, Yufan Deng, Shangkun Sun et al. (+11 more)
Video generation models have emerged as a promising paradigm for
embodied world simulation. However, both general-domain
video generators and
robot-specific data fine-tuned models can still produce physically implausible manipulations, including discontinuous motion trajectories and inconsistent
robot-object interactions, which limits their reliability as world simulators. Through extensive experiments, we find that such physical instability mainly arises from two factors: deformation of moving objects and implausible spatio-temporal correlations among interacting entities, particularly during contact. Building on this observation, we propose PhysisForcing, a scalable training framework that strengthens physical consistency by focusing supervision on physics-informative regions through joint optimization of pixel-level and semantic-level features. The framework consists of a pixel-level trajectory
alignment loss, which supervises DiT features using reference point trajectories, and a semantic-level relational
alignment loss, which aligns DiT features with inter-region relations extracted from a frozen
video understanding encoder. Extensive experiments on R-Bench, PAI-Bench, and EZS-Bench show that PhysisForcing consistently improves
embodied video generation over strong baselines, improving the Wan2.2-I2V-A14B and Cosmos3-Nano base models on R-Bench by 22.3% and 9.2% (7.1% and 3.7% over vanilla finetuning), with the Cosmos3-Nano variant attaining the best overall score. Beyond generation, as a world model under the WorldArena action-planner protocol it raises the closed-loop success rate from 16.0% to 24.0% and further improves downstream policy success, indicating that physically aligned
video models yield stronger representations for robotic manipulation.
Read the full abstract ▾
All authors Peiwen Zhang, Yufan Deng, Shangkun Sun, Juncheng Ma, Duomin Wang, Jonas Du, Zilin Pan, Ye Huang, Hao Liang, Songyan Huang, Ruihua Zhang, Enze Xie, Ming-Yu Liu, Daquan Zhou
Robotics · Cross-listed from cs.AI
AI-Driven Synthesis for High-Tech System Design: Automating Innovation
Luuk Oerlemans, Steven Westerhof and Theo Hofman
This article addresses the combinatorial complexity inherent in modern high-tech system design by presenting automation-in-design (AiD) as a transformative paradigm. We propose computational design synthesis (CDS), a framework utilising deep learning and generative AI to automate the creation of novel systems. Two case studies (e-drive system design and spatial dimensioning problem) serve as proof-points for this approach. The AI-driven methods used in the case studies represent a fundamental shift in engineering, advancing from simulation-based optimisation towards autonomous design with minimal human supervision.
Read the full abstract ▾
Robotics
Building a Scalable, Reproducible, Evaluatable, and Closed-Loop Simulation Environment Foundation for Embodied Intelligence
Junwu Xiong, Yongjian Guo, Mingxi Luo et al. (+17 more)
This paper presents a cloud-native simulation infrastructure framework for
embodied intelligence that supports large-scale training, standardized evaluation, and simulation-based data collection. The framework unifies simulation environment generation, task execution, trajectory collection, model evaluation, data management, and cloud services into a scalable and reproducible platform. To address the high cost, limited scalability, and poor reproducibility of real-world robotic data collection, the framework adopts cloud-native technologies including elastic resource scheduling, containerized simulation, unified data management, and service-oriented system design, enabling efficient large-scale simulation for multi-model and multi-task workloads. Built on a four-layer architecture, the framework provides standardized environment assets, automated task generation, trajectory collection,
benchmark evaluation, and closed-loop data optimization. It further integrates representative systems including D-VLA, RL-VLA3, Sword, and Pre-VLA to support scalable simulation, dynamic scheduling, visual augmentation, and real-time data filtering. We argue that cloud-native simulation infrastructure provides a unified foundation for data generation, model training, standardized evaluation, and real-world deployment, and will play a key role in the future development of
embodied intelligence.
Read the full abstract ▾
All authors Junwu Xiong, Yongjian Guo, Mingxi Luo, Ning Qiao, Lei Kang, Song Wang, Yince Gao, Chenfeng Gu, Zhen Sun, Haoran Li, Wei Lu, Yucheng Guo, Shuai Di, Xiaodong Bai, Haoran Sun, Jing Long, Jiaxuan Gao, Hui Zhang, Peng Hao, Lu Lu
Robotics
When Multi-Robot Systems Meet Agentic AI:Towards Embodied Collective Intelligence
Yuxuan Yan, Yuanyuan Jia and Qianqian Yang
Embodied AI is increasingly becoming agentic, shifting
robots from perception–control pipelines towards closed-loop systems that can retrieve context, deliberate during execution, monitor feedback, and refine future behavior. In parallel, robotics research has also moved from single-
robot autonomy towards multi-
robot systems, driven by the need for wider sensing, distributed action, heterogeneous capabilities, and fault tolerance. As AI
agents move from single-
agent use towards multi-
agent collaboration, robotics faces a parallel challenge:
robot teams must move beyond sharing maps, task assignments, and
datasets towards sharing the state produced by
embodied agent loops. This article explores
Embodied Collective Intelligence (ECI), a future multi-
robot paradigm in which a
robot team accumulates and uses world context, task progress, and skill experience as shared resources. Specifically, we first review how
embodied AI is becoming agentic and how multi-
robot cooperation has evolved. We then present
Embodied Collective Intelligence through Co-Perception, Co-Action, and Co-
Evolution. Finally, we use an illustrative
navigation study to examine one concrete component of the concept: shared world-
memory inheritance. The study shows that a newly added
robot can benefit from merged team
memory, but it is not intended as a full evaluation of the ECI framework. Taken together, the review and conceptual framework motivate
Embodied Collective Intelligence as a direction for
embodied multi-
agent intelligence, while the case study grounds one measurable part of the concept.
Read the full abstract ▾
Robotics
Drifting in the Future: Stabilizing Path Following Drifting on High-Latency Vehicle Systems
Frederik Werner, Till Heintzenberg, Markus Lienkamp et al. (+1 more)
Autonomously controlling and handling a vehicle at and beyond its stability limit is a mathematically and computationally demanding task. Prior demonstrations of automated drifting have been limited to research platforms with instantaneous torque delivery and independently actuated wheels, leaving their applicability to production vehicles with actuator latencies and mechanically coupled axles uncertain. To overcome these issues, we design a predictor to compensate for powertrain delays, develop a revised control formulation to accommodate higher actuation latencies as well as a differential coupling on the driven axle, and introduce brake-based velocity stabilization. This paper presents the controller framework, the model extensions, and real-world experimental results. We observe that our controller enables a production sports car with a combustion engine to robustly sustain circular and figure-eight drifts, limiting lateral error to 1.1 m and sideslip overshoot to 0.06 rad despite actuator delays exceeding 250 ms, while mitigating oscillations and maintaining stable path and sideslip tracking. In conclusion, our results establish that autonomous drifting is feasible on production-ready vehicles, opening pathways to advanced safety systems capable of stabilizing cars in scenarios where traditional control fails.
Read the full abstract ▾
All authors Frederik Werner, Till Heintzenberg, Markus Lienkamp, Johannes Betz
Robotics
Swarm sign language: motion-based communication between drones
Thomas Rey, Julien Moras, Alexandre Eudes et al. (+1 more)
In stealth-constrained swarm robotics, visual communication provides a critical alternative to active
radio transmissions, which might be jammed. This research investigates motion-based communication for non-active information exchange, utilizing modular, dynamically feasible planar trajectories as visual cues. On the receiver drone end, a pose
estimator tracks the transmitting drone's pose, feeding it into our custom 3DTrajDecoder. The decoder is designed to classify and segment the spatiotemporal sequence while simultaneously regressing its size and normal vector. To robustly train the decoder on both communicative and non-communicative trajectories, we developed a configurable online procedural generation pipeline. We validate our system through real-world testing and simulation to define its operating domain, supported by an extensive ablation study detailing our architectural choices and system limitations.
Read the full abstract ▾
All authors Thomas Rey, Julien Moras, Alexandre Eudes, Antoine Manzanera
Robotics
S$^2$-VLA: State-Space Guided Vision-Language-Action Models for Long-Horizon Manipulation
Zhipeng Xie, Zongyi Han, Xiangyi Wei et al. (+3 more)
Vision-
Language-Action (VLA) models have demonstrated strong capabilities in robotic manipulation, but their performance degrades significantly in long-horizon tasks due to cumulative error propagation. This limitation largely arises from static feature fusion mechanisms that rely on fixed weights to combine visual,
language, and action representations, preventing the model from adapting to different phases of task execution. To address this limitation, we propose S$^2$-VLA, a framework that introduces a State-Space Guided Adaptive
Attention (SSGAA) mechanism. SSGAA maintains a belief state that tracks task progression and generates dynamic gating weights to adaptively fuse information from three complementary sources visual features for spatial perception, task intents for high-level task planning, and temporal action sequences for execution consistency. This adaptive fusion allows the model to shift its focus throughout task execution, aligning with the evolving requirements of different task stages. Despite its compact 2B parameter size, S$^2$-VLA consistently outperforms larger 7B-scale models and achieves state-of-the-art performance on long-horizon manipulation
benchmarks, including LIBERO and SimplerEnv. highlighting the importance of adaptive feature fusion for long-horizon robotic manipulation.
Read the full abstract ▾
All authors Zhipeng Xie, Zongyi Han, Xiangyi Wei, Shiliang Sun, Yang Li, Jing Zhao
Robotics
LocalNav: Distilling Frontier VLMs and Embodied RL for On-Device Object Goal Navigation
Nicolas Baumann, Liam Boyle, Pu Deng et al. (+5 more)
Vision Language Models (VLMs) have emerged in the robotic domain as a powerful tool that enables environmental perception with
language context, serving as a catalyst for open-vocabulary tasks like ObjectNav. Yet, their computational footprint typically confines them to cloud execution, hindering low-latency inference with local deployment on resource-constrained
robots. To address this challenge, we present a
distillation strategy that transfers complex spatial-semantic
reasoning from large frontier models into a lightweight, 4B-parameter local VLM for edge execution on embedded GPU devices (e.g., Jetson Orin). We first establish a State of the Art (SotA), Scene Graph (SG)-based pipeline using Claude Sonnet 4.6, achieving a 39.7% Success Rate (SR) on the HM3D OVON
benchmark. We then demonstrate that fine-tuning Qwen3.5-4B on just 500 frontier
reasoning traces effectively enables
navigation capabilities, yielding a SR of 34.5%, narrowing the gap to the performance of large cloud models. Finally, we introduce E-RLVR with
Token Generation (TG) regularization to compress output sequence lengths for physical deployment while grounding the
agent in its task. This downstream optimization reduces TG overhead by 72.1% and latency by 71.8%. Combined with quantization, this joint strategy yields a cumulative 82.8% reduction in overall inference latency without significantly sacrificing performance, presenting a viable paradigm for local, low-latency VLM execution on mobile
robots.
Read the full abstract ▾
All authors Nicolas Baumann, Liam Boyle, Pu Deng, Edoardo Ghignone, Boyang Sun, Marc Pollefeys, Luca Benini, Michele Magno
Robotics
PPO-EAL: Exact Augmented Lagrangian Proximal Policy Optimization for Safe Robotic Control
Jiatao Ding, Songqun Gao, Andrea Del Prete et al. (+1 more)
Reinforcement learning (RL) has emerged as a promising solution to accomplish complex robotic control tasks; however, most of the current work ignores the safety requirements. Safe RL seeks to maximize task performance while satisfying explicit physical constraints, but current
algorithms struggle to learn the policy efficiently with precise constraint satisfaction. This work proposes PPO-EAL, a novel first-order constrained policy optimization framework that integrates exact augmented Lagrangian optimization into proximal policy optimization for safe robotic control. By combining clipped policy updates with exact quadratic penalty terms, PPO-EAL achieves theoretically grounded constraint enforcement without requiring impractically large penalty factors. A momentum-regulated multiplier update further improves dual-variable stability, reducing constraint oscillation and unsafe behavior while preserving task performance. We provide exactness and convergence analysis under standard
stochastic approximation assumptions. Extensive validation across diverse GPU-accelerated robotic
benchmarks-including cart-pole balancing, cart-double-pendulum stabilization, 7-DoF Franka end-effector reaching, and quadrupedal locomotion-demonstrates superior safety precision and reward performance compared with state-of-the-art first-order safe RL baselines. Finally, we demonstrate zero-shot sim-to-real deployment in a contact-rich gear assembly task, where PPO-EAL substantially improves task success, reduces peak contact force, and enhances operational robustness. These results establish PPO-EAL as a general and practically deployable safe RL framework for diverse safety-critical robotic systems.
Read the full abstract ▾
All authors Jiatao Ding, Songqun Gao, Andrea Del Prete, Matteo Saveriano
Robotics
Booster Lab: A Data-Centric Pipeline for Learning Deployable Humanoid Locomotion Policies
Penghui Chen, Tinglong Zheng, Yufeng Zhang et al. (+1 more)
Humanoid robot motion learning requires not only task-oriented control policies but also physically feasible and natural behaviors that can be transferred to real
robots. However,
robot-feasible motion data are often scarce: raw human demonstrations may be incompatible with the
robot morphology, open-source clips vary in quality, and simulation-collected
robot trajectories still require feasibility checking. To address these challenges, we propose a data-centric training and deployment pipeline that integrates motion data curation, real-to-sim model adaptation, AMP-based reinforcement learning, and sim-to-real deployment. We validate the framework on the Booster T1
robot and further provide preliminary cross-platform validation on Booster K1.
Read the full abstract ▾
All authors Penghui Chen, Tinglong Zheng, Yufeng Zhang, Mingguo Zhao
Robotics
LXD-SLAM: LiDAR+X Dense SLAM with $\sum_{i=0}^{5}C_5^i$ Configurable Sensor Combinations
Zhong Wang, Lin Zhang, Linfei Li et al. (+4 more)
Simultaneous Localization and Mapping (SLAM) is essential for autonomous systems, yet achieving reliable, globally consistent pose estimation and dense mapping in complex environments remains challenging due to
geometric degeneracy and sensor drift. While multi-sensor fusion addresses these issues, existing systems often lack the modularity to adapt to diverse platforms and rely on mathematically inconsistent fusion or suboptimal map representations. To address these limitations, we propose LXD-SLAM (
LiDAR+X Dense SLAM), a highly versatile and unified multi-sensor fusion framework. Centered around 3D
LiDAR, our system allows for the plug-and-play integration of
LiDAR, Camera, IMU, Wheel Encoder, and GNSS, supporting up to 32 distinct sensor combinations. We employ a mathematically unified Iterative Error-Sate Kalman Filter with an adaptive hierarchical prediction strategy and an update step that minimizes point-to-mesh distances and visual reprojection errors. To support this, the environment is modeled using continuous multi-layered Gaussian Process (GP) sub-meshes, which enables efficient ray-to-mesh depth recovery for visual features. For global consistency, we introduce an Extended Scan Context (ESC) descriptor derived from the GP sub-meshes alongside a Bidirectional PnP optimization for robust multi-modal loop closure within a hybrid pose graph. Extensive evaluations on public
datasets and real-world experiments demonstrate that LXD-SLAM matches or exceeds state-of-the-art specialized odometry solutions across various configurations while generating high-fidelity, globally consistent dense meshes in real-time. The relevant codes and data will be made available at https://github.com/peterWon/LXD-SLAM upon publication.
Read the full abstract ▾
All authors Zhong Wang, Lin Zhang, Linfei Li, Ying Shen, Shaoming Zhang, Pengcheng Shi, Shengjie Zhao
Robotics
SpikeVLA: Vision-Language-Action Models with Spiking Neural Networks
Ruiqi Song, Dujun Nie, Siyu Teng et al. (+7 more)
Vision-
Language-Action (VLA) models have become a dominant paradigm for
embodied intelligence. However, most existing approaches are built on large-scale
transformers, resulting in substantial inference latency and energy consumption that limit their practical deployment in low-power, real-time scenarios. We propose SpikeVLA, a spiking VLA architecture for
embodied navigation with energy-efficient inference, consisting of three key components. (i) A spiking
vision encoder, Spike-V, that replaces dense continuous layers with event-driven spiking layers to reduce the energy consumption of visual representation learning. (ii) A multi-modal spiking large
language model, Spike-L, that reformulates cross-modal
reasoning with spiking dynamics and
token-level event-driven sparsity to further lower computational cost. (iii) A spiking action policy network, Spike-A employs
Laplacian-kernel population coding with a multi-layer fully connected SNN, and decodes spiking activities into stable and robust continuous control with energy-efficient inference under low-power constraints. Experiments on
navigation and robotic control tasks show that SpikeVLA significantly reduces energy consumption and computational cost while maintaining competitive performance, highlighting its potential for low-power, real-time
embodied intelligence.
Read the full abstract ▾
All authors Ruiqi Song, Dujun Nie, Siyu Teng, Baiyong Ding, Xiaotong Zhang, Dong Li, Chenming Zhang, Yuchen Li, Hangbin Wu, Long Chen
Robotics · Cross-listed from cs.LG
RS-Diffuser: Risk-Sensitive Diffusion Planning with Distributional Value Guidance
Shiqiang Gong
Offline reinforcement learning enables policy learning from fixed
datasets without additional environment interaction, making it appealing for safety-critical applications where online exploration is costly or unsafe.
Diffusion-based decision-making methods have recently achieved strong performance in offline RL by modeling rich,
multimodal trajectory distributions. However, existing
diffusion planners are typically risk-neutral and therefore may overlook rare but catastrophic outcomes that are crucial in real-world deployment. In this work, we propose RS-Diffuser, a risk-sensitive offline
diffusion planning framework that combines
diffusion-based trajectory generation with distributional value critics. RS-Diffuser learns a
diffusion planner over future state trajectories, a separate inverse dynamics model for action decoding, and a Monte Carlo distributional critic that estimates the full return distribution of candidate plans through
quantile regression. At sampling time, we incorporate a risk-sensitive guidance signal into the denoising process, using gradients computed from tail-aware objectives such as Conditional Value at Risk to steer generation toward desired risk profiles. As a result, a single trained model can flexibly produce risk-averse, risk-neutral, or risk-seeking behaviors by changing only the inference-time risk parameter. Extensive experiments on risk-sensitive D4RL and risky
robot navigation benchmarks demonstrate that RS-Diffuser achieves state-of-the-art performance, improving both overall return and worst-case robustness while reducing safety violations.
Read the full abstract ▾
Robotics
Drop-Then-Recovery: How Redundant Are Vision-Language-Action Models?
Guoheng Sun, Kaixi Feng, Shwai He et al. (+8 more)
Vision-
Language-Action (VLA) models enable instruction-driven robotic manipulation, but they inherit oversized
language backbones from pretrained VLMs whose capacity far exceeds what is needed for short robotic instructions. This raises a basic question: how much of a VLA model is actually necessary for closed-loop control? In this work, we study architectural redundancy in VLA models by using
transformer block removal as a controlled intervention. We introduce Drop-Then-Recovery (DTR), an analysis protocol that removes selected blocks from a pretrained VLA model and then fine-tunes the resulting model to measure whether the removed capacity was necessary for downstream control. To make this intervention reliable, we propose GateProbe, a one-shot virtual-gate sensitivity metric that ranks blocks by their contribution to the downstream action loss. Across multiple VLA architectures, manipulation
benchmarks and even real-
robot industrial scenarios, we find a strong asymmetry in post-removal recoverability: \ul{
language backbones are highly redundant for standard robotic manipulation tasks, whereas
vision and action pathways are substantially less tolerant to removal}. On LIBERO, removing half of the LLM blocks even improves OpenVLA-OFT from 95.0% to 98.3% under the same downstream fine-tuning budget, and retaining only two
language blocks still recovers baseline-level performance. These results suggest that current VLA
benchmarks may exert limited pressure on deep
language grounding and compositional instruction understanding, and that future VLA architectures should allocate capacity more deliberately across
language,
vision, and action components. The code is available at https://github.com/s1ghhh/VLADrop.
Read the full abstract ▾
All authors Guoheng Sun, Kaixi Feng, Shwai He, Xiaochuan Gong, Yexiao He, Ziyao Wang, Zheyu Shen, Wanghao Ye, Ramana Rao Kompella, Gaowen Liu, Ang Li
Robotics
DIM-WAM: World-Action Modeling with Diverse Historical Event Memory
Kai Wang, Zhaopeng Gu, Yixiang Chen et al. (+7 more)
World-action models have shown promising
robot-manipulation performance by jointly predicting future visual states and actions. However, existing methods mainly rely on short-term history and short-horizon future prediction, which is insufficient for long-horizon tasks whose correct execution depends on earlier observations and task progress. Such temporally dependent tasks require effective use of complementary temporal information, including recent local context, cross-stage historical events, immediate future dynamics, and global task progress. To address long-term forgetting and poor awareness of the global task state, we introduce DiM-WAM, a
memory-augmented world-action model that integrates multi-scale historical context, local future dynamics, and global task progress. The
memory extracts compact visual event information from real observations, updates multiple
memory banks through independent similarity-based merging, and then reads the bank-identity- and time-embedded long-term context to condition
video and action denoising. A progress-supervision objective further encourages
memory tokens to encode not only completed historical events but also the current task stage and its implications for the remaining task. On RMBench, DiM-WAM raises average success from 28.4% with LingBot-VA to 69.8%, exceeding the explicit-
memory Mem-0 baseline at 42.0%. On four real-world Franka tasks, it improves average stage success from 70.7% to 91.5% and full-task success from 52.5% to 80.0%. Project page: https://wangkai-casia.github.io/dim-wam.
Read the full abstract ▾
All authors Kai Wang, Zhaopeng Gu, Yixiang Chen, Yuan Xu, Qisen Ma, Jiabing Yang, Zhaowen Li, Yan Huang, Liang Wang, Peng Su
Robotics
CWI: Composite Humanoid Whole-Body Imitation System for Loco-manipulation
Wenqi Ge, Junde Guo, Zhen Fu et al. (+3 more)
Achieving everyday tasks with
humanoid robots requires coordinating stable locomotion with versatile manipulation. However, existing whole-body controllers still face significant challenges. Methods trained solely via command sampling, without motion-capture (MoCap) data, often struggle with sparse rewards and require carefully tuned curricula to converge. This is especially problematic for upper-body control, where the resulting motions deviate from human-like statistics and degrade whole-body coordination. Conversely, approaches that imitate full-body MoCap data suffer from
dataset imbalance, as many locomotion trajectories are overly aggressive for stable-locomotion scenarios, necessitating extensive data filtering and augmentation. To address this, we present Composite Whole-Body Imitation (CWI), a framework that decouples the use of MoCap data for upper-body manipulation and lower-body locomotion. This decoupling allows us to exploit the full MoCap
dataset of diverse manipulation references, while stable, command-conditioned lower-body locomotion is guided by dual discriminators trained on curated expert-quality walking and squatting clips via an
Adversarial Motion Prior (AMP). A multi-critic architecture reduces conflicts among locomotion, manipulation, and motion-style objectives, and a teacher–student
distillation stage yields a whole-body policy conditioned only on bimanual hand poses and velocity/height commands. We evaluate CWI through simulation experiments and real-world deployment on a full-size LimX Oli
humanoid. The results show competitive loco-manipulation performance, robust whole-body coordination, and practical teleoperation without full-body motion-capture equipment. A project page with supplementary material can be found at https://cwi-ral.github.io/CWI-RAL-Webpage.
Read the full abstract ▾
All authors Wenqi Ge, Junde Guo, Zhen Fu, Shunpeng Yang, Jiayu Chen, Hua Chen
Robotics
Direct Action-Head Injection of A Grounded 3D Point Unlocks Spatial and Task Generalization
Shiang-Feng Tsai, Jin-Cheng Jhang, Yen-Ling Tai et al. (+5 more)
Vision-
Language-Action (VLA) models leverage large-scale
vision-
language pretraining for flexible
robot manipulation, yet at test time they remain brittle to changed object positions and to familiar scenes paired with different instructions. A growing family of methods addresses this brittleness by supplying the policy with grounding signals, such as 2D pixel coordinates for object localization and placement. However, we find that how the grounding signal is represented and injected matters more than the signal itself. In this work, we propose a lightweight module that represents the grounding signal in 3D and injects the resulting
embedding directly into the action head. The module is a two-layer MLP and requires no changes to the VLA backbone or pretraining pipeline, yet it yields substantially larger gains than
language- or visual-prompting alternatives. On LIBERO-PRO, our method improves the average success rate of GR00T-N1.6 from $31.2$ to $77.5$ under task perturbation and from $28.1$ to $60.2$ under position perturbation. Comparable gains are also achieved for $\pi_{0.5}$, demonstrating that the mechanism is backbone-agnostic across VLAs with
diffusion-based action heads. We further validate the practical applicability with real-world experiments. Together, these results support our central finding: lifting adequate 2D grounding into 3D and injecting it into the action head enables spatial and instance-level task generalization in VLAs.
Read the full abstract ▾
All authors Shiang-Feng Tsai, Jin-Cheng Jhang, Yen-Ling Tai, Jia-Hong Lai, Shih-Yun Wong, Kang-Tung Hsu, Yi-Ting Chen, Min Sun
Robotics · Cross-listed from eess.SY
Characterizing Driver Interactions with Autonomous Vehicles via Response Maps
Dave Broaddus, Rachel DiPirro, Chishang Yang et al. (+3 more)
Understanding human responses to autonomous vehicle (AV) behaviors is essential for socially aware interaction, which is crucial for socially compatible
navigation in shared traffic environments. We characterize human driving responses in interactions with AVs as feedback laws over the coupled state space of the human driven vehicle and the AV. We model the human driver's actions using a response map, a concept based in game theory, and employ a linear representation to capture driver behaviors as a function of AV behaviors, based on empirical data from a driving simulator study. Our results show that 1) human driver acceleration behavior can be captured using response maps, and 2) human driver responses differ significantly with respect to AV behaviors of yielding, non-yielding, and responsive to the human driver.
Read the full abstract ▾
All authors Dave Broaddus, Rachel DiPirro, Chishang Yang, Dan Calderone, Wendy Ju, Meeko Oishi
Robotics
P-ARC: Exploiting Subproblem Independence for Parallel Multi-Robot Motion Planning
James D. Motes, Marco Morales and Nancy M. Amato
This paper presents Parallel ARC (P-ARC), a parallel formulation of the Adaptive
Robot Coordination (ARC) approach to multi-
robot motion planning (MRMP) which exploits subproblem independence. ARC's adaptive (de)composition of the multi-
robot planning space exposes parallelism: single-
robot paths are solved independently and iterative conflict detection and resolution create locally coupled subproblems. While distributing single-
robot queries is trivial, not all conflicts are independent, so P-ARC proposes
robot-disjoint conflict batches which enable efficient distributed detection and concurrent repair. Additionally, OR-multi-start strategies are employed at the global and subproblem resolution levels, creating a hybrid parallel strategy OR-P-ARC. We evaluate the methods against sequential ARC, multi-start OR-ARC, and coupled and prioritized parallel baselines on controlled 2D mobile
robot and planar-manipulator problems with up to 256
robots and 3D Panda manipulator problems with up to 16
robots. On 16-
robot Panda tasks, with 16 workers, P-ARC and OR-P-ARC achieve 3.48X and 6.67X speedups, respectively.
Read the full abstract ▾
Robotics
Physics-Guided Robotic Radiation Source Localization along Arbitrary Measurement Paths in Unstructured Environments
Hojoon Son, Kai Tan and Fan Zhang
Using
robots to estimate the location of the radiation source is an effective way to improve efficiency and safety. Existing methods focus on planning the
robot's path to achieve precise estimation, typically approaching the source. However, approaching the source increases the risk of radiation damage to a
robot. In addition, a path-planning
algorithm designed solely for radiation source localization (RSL) limits the flexibility of missions that deploy
robots into radioactive environments. This study presents an automation framework for robotic RSL that leverages a physics-informed machine learning (PIML) model to precisely estimate the source location, regardless of measurement paths, in unknown environments. Physics-inspired model tensors have been designed for PIML to handle attenuated gamma-ray flux signals from unknown obstacles, and multiple models are computed in parallel to improve the robustness and precision of the RSL. The proposed method is evaluated in high-fidelity simulation environments using Monte Carlo particle transport across diverse randomized domains, including spatial scales, radiation source types, obstacle materials and geometries, and
robot trajectories. The method is also validated through physical experiments on configurations that are not included in the simulation-based evaluation. The continuous learning technique is applied in real-
robot deployment to enhance the practical applicability of the online robotic RSL system. The proposed method advances
robot radiation perception from pointwise flux detection to spatial intelligence.
Read the full abstract ▾