Robotics
World Guidance: World Modeling in Condition Space for Action Generation
Yue Su, Sijin Chen, Haixin Shi et al. (+7 more)
Leveraging future observation modeling to facilitate action generation presents a promising avenue for enhancing the capabilities of
Vision-
Language-Action (VLA) models. However, existing approaches struggle to strike a balance between maintaining efficient, predictable future representations and preserving sufficient fine-grained information to guide precise action generation. To address this limitation, we propose WoG (World Guidance), a framework that maps future observations into compact conditions by injecting them into the action inference pipeline. The VLA is then trained to simultaneously predict these compressed conditions alongside future actions, thereby achieving effective world modeling within the condition space for action inference. We demonstrate that modeling and predicting this condition space not only facilitates fine-grained action generation but also exhibits superior generalization capabilities. Moreover, it learns effectively from substantial human manipulation
videos. Extensive experiments across both simulation and real-world environments validate that our method significantly outperforms existing methods based on future prediction. Project page is available at: https://selen-suyue.github.io/WoGNet/
Read the full abstract ▾
All authors Yue Su, Sijin Chen, Haixin Shi, Mingyu Liu, Zhengshen Zhang, Ningyuan Huang, Weiheng Zhong, Zhengbang Zhu, Yuxiao Liu, Xihui Liu
Robotics
Parallel Reference-Centric Continuous-Time Relative Localization with Augmented Clamped Non-Uniform B-Splines
Jiadong Lu, Zhehan Li, Tao Han et al. (+4 more)
Accurate relative localization is critical for multi-
robot cooperation. In
robot groups, measurements from different
robots arrive asynchronously and with clock time-offsets. Although Continuous-Time (CT) formulations have proved effective for handling asynchronous measurements in single-
robot SLAM and calibration, extending CT methods to multi-
robot settings faces great challenges in achieving high-accuracy, low-latency, and high-frequency performance. In particular, existing CT methods suffer from the inherent query-time delay of unclamped B-splines and high optimization latency. This paper proposes CT-RIO, a novel Continuous-Time Relative-Inertial Odometry framework. We adopt Clamped Non-Uniform B-splines (C-NUBS) to represent states, eliminating the query-time delay. We further augment C-NUBS with closed-form extension and shrinkage operations that preserve the spline shape, making it suitable for online estimation and enabling flexible
knot management. This flexibility leads to the concept of a
knot-keyknot strategy, which supports spline extension at high frequency while retaining sparse keyknots for adaptive relative motion modeling. We then formulate a reference-centric sliding-window relative localization problem that operates purely on relative kinematics and inter-
robot constraints. To enable low-latency and high-frequency estimation, we decompose the tightly coupled optimization into
robot-wise subproblems and solve them in parallel using asynchronous block coordinate descent. Extensive experiments show that CT-RIO converges from time-offsets as large as 264 ms to sub-millisecond within 3 s, and achieves RMSEs of 0.046 m and 1.8 degree. It consistently outperforms evaluated published methods, with improvements of up to 60% under high-speed motion.
Read the full abstract ▾
All authors Jiadong Lu, Zhehan Li, Tao Han, Miao Xu, Jiajun Lv, Chao Xu, Yanjun Cao
Robotics
Are Foundation Models the Route to Full-Stack Transfer in Robotics?
Freek Stulp, Samuel Bustamante, João Silvério et al. (+3 more)
In humans and
robots alike, transfer learning occurs at different levels of abstraction, from high-level linguistic transfer to low-level transfer of motor skills. In this article, we provide an overview of the impact that foundation models and
transformer networks have had on these different levels, bringing
robots closer than ever to "full-stack transfer". Considering LLMs, VLMs and VLAs from a robotic transfer learning perspective allows us to highlight recurring concepts for transfer, beyond specific implementations. We also consider the challenges of data collection and transfer
benchmarks for robotics in the age of foundation models. Are foundation models the route to full-stack transfer in robotics? Our expectation is that they will certainly stay on this route as a key technology.
Read the full abstract ▾
All authors Freek Stulp, Samuel Bustamante, João Silvério, Alin Albu-Schäffer, Jeannette Bohg, Shuran Song
Robotics
Humanizing Robot Gaze Shifts: A Framework for Natural Gaze Shifts in Humanoid Robots
Jingchao Wei, Jingkai Qin, Yuxiao Cao et al. (+4 more)
Leveraging auditory and visual feedback for
attention reorientation is essential for natural gaze shifts in social interaction. However, enabling
humanoid robots to perform natural and context-appropriate gaze shifts in unconstrained human–
robot interaction (HRI) remains challenging, as it requires the coupling of cognitive
attention mechanisms and biomimetic motion generation. In this work, we propose the
Robot Gaze-Shift (RGS) framework, which integrates these two components into a unified pipeline. First, RGS employs a
vision–
language model (VLM)-based gaze
reasoning pipeline to infer context-appropriate gaze targets from
multimodal interaction cues, ensuring consistency with human gaze-orienting regularities. Second, RGS introduces a conditional Vector Quantized-Variational Autoencoder (VQ-VAE) model for eye–head coordinated gaze-shift motion generation, producing diverse and human-like gaze-shift behaviors. Experiments validate that RGS effectively replicates human-like target selection and generates realistic, diverse gaze-shift motions.
Read the full abstract ▾
All authors Jingchao Wei, Jingkai Qin, Yuxiao Cao, Jingcheng Huang, Xiangrui Zeng, Min Li, Zhouping Yin
Robotics
Dream-SLAM: Dreaming the Unseen for Active SLAM in Dynamic Environments
Xiangqi Meng, Pengxu Hou, Zhenjun Zhao et al. (+4 more)
In addition to the core tasks of simultaneous localization and mapping (SLAM), active SLAM additionally in- volves generating
robot actions that enable effective and efficient exploration of unknown environments. However, existing active SLAM pipelines are limited by three main factors. First, they inherit the restrictions of the underlying SLAM modules that they may be using. Second, their motion planning strategies are typically shortsighted and lack long-term
vision. Third, most approaches struggle to handle dynamic scenes. To address these limitations, we propose a novel
monocular active SLAM method, Dream-SLAM, which is based on dreaming cross-spatio-temporal images and semantically plausible structures of partially observed dynamic environments. The generated cross-spatio-temporal im- ages are fused with real observations to mitigate noise and data incompleteness, leading to more accurate camera pose estimation and a more coherent 3D scene representation. Furthermore, we integrate dreamed and observed scene structures to enable long- horizon planning, producing farsighted trajectories that promote efficient and thorough exploration. Extensive experiments on both public and self-collected
datasets demonstrate that Dream-SLAM outperforms state-of-the-art methods in localization accuracy, mapping quality, and exploration efficiency. Source code will be publicly available upon paper acceptance.
Read the full abstract ▾
All authors Xiangqi Meng, Pengxu Hou, Zhenjun Zhao, Javier Civera, Daniel Cremers, Hesheng Wang, Haoang Li
Robotics · Cross-listed from physics.soc-ph
The Swarm Intelligence Freeway-Urban Trajectories (SWIFTraj) Dataset – Part II: A Graph-Based Approach for Trajectory Connection
Xinkai Ji, Pan Liu, Ying Yang et al. (+1 more)
In Part I of this companion paper series, we introduced SWIFTraj, a new open-source vehicle trajectory
dataset collected using a unmanned aerial vehicle (
UAV) swarm. The
dataset has two distinctive features. First, by connecting trajectories across consecutive
UAV videos, it provides long-distance continuous trajectories, with the longest exceeding 4.5 km. Second, it covers an integrated traffic network consisting of both freeways and their connected urban roads. Obtaining such long-distance continuous trajectories from a
UAV swarm is challenging, due to the need for accurate time
alignment across multiple
videos and the irregular spatial distribution of
UAVs. To address these challenges, this paper proposes a novel graph-based approach for connecting vehicle trajectories captured by a
UAV swarm. An undirected graph is constructed to represent flexible
UAV layouts, and an automatic time
alignment method based on trajectory matching cost minimization is developed to estimate optimal time offsets across
videos. To associate trajectories of the same vehicle observed in different
videos, a vehicle matching table is established using the Hungarian
algorithm. The proposed approach is evaluated using both simulated and real-world data. Results from real-world experiments show that the time
alignment error is within three
video frames, corresponding to approximately 0.1 s, and that the vehicle matching achieves an F1-score of about 0.99. These results demonstrate the effectiveness of the proposed method in addressing key challenges in
UAV-based trajectory connection and highlight its potential for large-scale vehicle trajectory collection.
Read the full abstract ▾
All authors Xinkai Ji, Pan Liu, Ying Yang, Yu Han
Robotics · Cross-listed from cs.CV
UNet-Based Keypoint Regression for 3D Cone Localization in Autonomous Racing
Mariia Baidachna, James Carty, Aidan Ferguson et al. (+7 more)
Accurate cone localization in 3D space is essential in autonomous racing for precise
navigation around the track. Approaches that rely on traditional computer
vision algorithms are sensitive to environmental variations, and neural networks are often trained on limited data and are infeasible to run in real time. We present a UNet-based neural network for keypoint detection on cones, leveraging the largest custom-labeled
dataset we have assembled. Our approach enables accurate cone position estimation and the potential for color prediction. Our model achieves substantial improvements in keypoint accuracy over conventional methods. Furthermore, we leverage our predicted keypoints in the perception pipeline and evaluate the end-to-end autonomous system. Our results show high-quality performance across all metrics, highlighting the effectiveness of this approach and its potential for adoption in competitive autonomous racing systems.
Read the full abstract ▾
All authors Mariia Baidachna, James Carty, Aidan Ferguson, Joseph Agrane, Varad Kulkarni, Aubrey Agub, Michael Baxendale, Aaron David, Rachel Horton, Elliott Atkinson
Robotics
Enhancing Cellular-enabled Collaborative Robots Planning through GNSS data for SAR Scenarios
Arnau Romero, Carmen Delgado, Jana Baguer et al. (+2 more)
Cellular-enabled collaborative
robots are becoming paramount in Search-and-Rescue (SAR) and emergency response. Crucially dependent on resilient mobile network connectivity, they serve as invaluable assets for tasks like rapid victim localization and the exploration of hazardous, otherwise unreachable areas. However, their reliance on battery power and the need for persistent, low-latency communication limit operational time and mobility. To address this, and considering the evolving capabilities of 5G/6G networks, we propose a novel SAR framework that includes Mission Planning and Mission Execution phases and that optimizes
robot deployment. By considering parameters such as the exploration area size, terrain elevation,
robot fleet size, communication-influenced energy profiles, desired exploration rate, and target response time, our framework determines the minimum number of
robots required and their optimal paths to ensure effective coverage and timely data backhaul over mobile networks. Our results demonstrate the trade-offs between number of
robots, explored area, and response time for wheeled and quadruped
robots. Further, we quantify the impact of terrain elevation data on mission time and energy consumption, showing the benefits of incorporating real-world environmental factors that might also affect mobile signal propagation and connectivity into SAR planning. This framework provides critical insights for leveraging next-generation mobile networks to enhance autonomous SAR operations.
Read the full abstract ▾
All authors Arnau Romero, Carmen Delgado, Jana Baguer, Raúl Suárez, Xavier Costa-Pérez
Robotics
Self-Curriculum Model-based Reinforcement Learning for Shape Control of Deformable Linear Objects
Zhaowei Liang, Song Wang, Zhao Jin et al. (+2 more)
Precise shape control of Deformable Linear Objects (DLOs) is crucial in robotic applications such as industrial and medical fields. However, existing methods face challenges in handling complex large deformation tasks, especially those involving opposite
curvatures, and lack efficiency and precision. To address this, we propose a two-stage framework combining Reinforcement Learning (RL) and online visual servoing. In the large-deformation stage, a model-based reinforcement learning approach using an ensemble of dynamics models is introduced to significantly improve sample efficiency. Additionally, we design a self-curriculum goal generation mechanism that dynamically selects intermediate-difficulty goals with high diversity through imagined evaluations, thereby optimizing the policy learning process. In the small-deformation stage, a Jacobian-based visual servo controller is deployed to ensure high-precision convergence. Simulation results show that the proposed method enables efficient policy learning and significantly outperforms mainstream baselines in shape control success rate and precision. Furthermore, the framework effectively transfers the policy trained in simulation to real-world tasks with zero-shot adaptation. It successfully completes all 30 cases with diverse initial and target shapes across DLOs of different sizes and materials. The project website is available at: https://anonymous.4open.science/w/sc-mbrl-dlo-EB48/
Read the full abstract ▾
All authors Zhaowei Liang, Song Wang, Zhao Jin, Shirui Wu, Dan Wu
Robotics
DexRepNet++: Learning Dexterous Robotic Manipulation with Geometric and Spatial Hand-Object Representations
Qingtao Liu, Zhengnan Sun, Yu Cui et al. (+5 more)
Robotic dexterous manipulation is a challenging problem due to high degrees of freedom (DoFs) and complex contacts of multi-fingered robotic hands. Many existing deep reinforcement learning (DRL) based methods aim at improving sample efficiency in high-dimensional output action spaces. However, existing works often overlook the role of representations in achieving generalization of a manipulation policy in the complex input space during the hand-object interaction. In this paper, we propose DexRep, a novel hand-object interaction representation to capture object surface features and spatial relations between hands and objects for dexterous manipulation skill learning. Based on DexRep, policies are learned for three dexterous manipulation tasks, i.e. grasping, in-hand reorientation, bimanual handover, and extensive experiments are conducted to verify the effectiveness. In simulation, for grasping, the policy learned with 40 objects achieves a success rate of 87.9% on more than 5000 unseen objects of diverse categories, significantly surpassing existing work trained with thousands of objects; for the in-hand reorientation and handover tasks, the policies also boost the success rates and other metrics of existing hand-object representations by 20% to 40%. The grasp policies with DexRep are deployed to the real world under multi-camera and single-camera setups and demonstrate a small sim-to-real gap.
Read the full abstract ▾
All authors Qingtao Liu, Zhengnan Sun, Yu Cui, Haoming Li, Gaofeng Li, Lin Shao, Jiming Chen, Qi Ye
Robotics
Therapist-Robot-Patient Physical Interaction is Worth a Thousand Words: Enabling Intuitive Therapist Guidance via Remote Haptic Control
Beatrice Luciani, Alex van den Berg, Matti Lang et al. (+2 more)
Robotic systems can enhance the amount and repeatability of physically guided motor training. Yet their real-world adoption is limited, partly due to non-intuitive trainer/therapist-trainee/patient interactions. To address this gap, we present a haptic teleoperation system for trainers to remotely guide and monitor the movements of a trainee wearing an arm exoskeleton. The trainer can physically interact with the exoskeleton through a commercial handheld haptic device via virtual contact points at the exoskeleton's elbow and wrist, allowing intuitive guidance. Thirty-two participants tested the system in a trainer-trainee paradigm, comparing our haptic demonstration system with conventional visual demonstration in guiding trainees in executing arm poses. Quantitative analyses showed that haptic demonstration significantly reduced movement completion time and improved smoothness, while
speech analysis using large
language models for automated transcription and categorization of verbal commands revealed fewer verbal instructions. The haptic demonstration did not result in higher reported mental and physical effort by trainers compared to the visual demonstration, while trainers reported greater competence and trainees lower physical demand. These findings support the feasibility of our proposed interface for effective remote human-
robot physical interaction. Future work should assess its usability and efficacy for clinical populations in restoring clinicians' sense of agency during
robot-assisted therapy.
Read the full abstract ▾
All authors Beatrice Luciani, Alex van den Berg, Matti Lang, Alexandre L. Ratschat, Laura Marchal-Crespo
Robotics
Joint-Aligned Latent Action: Towards Scalable VLA Pretraining in the Wild
Hao Luo, Ye Wang, Wanpeng Zhang et al. (+5 more)
Despite progress,
Vision-
Language-Action models (VLAs) are limited by a scarcity of large-scale, diverse
robot data. While human manipulation
videos offer a rich alternative, existing methods are forced to choose between small, precisely-labeled
datasets and vast in-the-wild footage with unreliable hand tracking labels. We present JALA, a pretraining framework that learns Jointly-Aligned Latent Actions. JALA bypasses full visual dynamic reconstruction, instead learns a predictive action
embedding aligned with both inverse dynamics and real actions. This yields a transition-aware, behavior-centric latent space for learning from heterogeneous human data. We scale this approach with UniHand-Mix, a 7.5M
video corpus (>2,000 hours) blending laboratory and in-the-wild footage. Experiments demonstrate that JALA generates more realistic hand motions in both controlled and unconstrained scenarios, significantly improving downstream
robot manipulation performance in both simulation and real-world tasks. These results indicate that jointly-aligned latent actions offer a scalable pathway for VLA pretraining from human data.
Read the full abstract ▾
All authors Hao Luo, Ye Wang, Wanpeng Zhang, Haoqi Yuan, Yicheng Feng, Haiweng Xu, Sipeng Zheng, Zongqing Lu
Robotics
LessMimic: Long-Horizon Humanoid Interaction with Unified Distance Field Representations
Yutang Lin, Jieming Cui, Yixuan Li et al. (+3 more)
Humanoid robots that autonomously interact with physical environments over extended horizons represent a central goal of
embodied intelligence. Existing approaches rely on reference motions or task-specific rewards, tightly coupling policies to particular object geometries and precluding multi-skill generalization within a single framework. A unified interaction representation enabling reference-free inference,
geometric generalization, and long-horizon skill composition within one policy remains an open challenge. Here we show that Distance Field (DF) provides such a representation: LessMimic conditions a single whole-body policy on DF-derived
geometric cues–surface distances, gradients, and velocity decompositions–removing the need for motion references, with interaction latents encoded via a Variational Auto-Encoder (VAE) and post-trained using
Adversarial Interaction Priors (AIP) under Reinforcement Learning (RL). Through DAgger-style
distillation that aligns DF latents with
egocentric depth features, LessMimic further transfers seamlessly to
vision-only deployment without motion capture (MoCap) infrastructure. A single LessMimic policy achieves 80–100% success across object scales from 0.4x to 1.6x on PickUp and SitStand where baselines degrade sharply, attains 62.1% success on 5 task instances trajectories, and remains viable up to 40 sequentially composed tasks. By grounding interaction in local geometry rather than demonstrations, LessMimic offers a scalable path toward
humanoid robots that generalize, compose skills, and recover from failures in unstructured environments.
Read the full abstract ▾
All authors Yutang Lin, Jieming Cui, Yixuan Li, Baoxiong Jia, Yixin Zhu, Siyuan Huang
Robotics
Dual-Regime Hybrid Aerodynamic Modeling of Winged Blimps With Neural Mixing
Xiaorui Wang, Hongwu Wang, Yue Fan et al. (+2 more)
Winged blimps operate across distinct aerodynamic regimes that cannot be adequately captured by a single model. At high speeds and small angles of attack, their dynamics exhibit strong coupling between lift and attitude, resembling fixed-wing aircraft behavior. At low speeds or large angles of attack, viscous effects and flow separation dominate, leading to drag-driven and damping-dominated dynamics. Accurately representing transitions between these regimes remains a fundamental challenge. This paper presents a hybrid aerodynamic modeling framework that integrates a fixed-wing Aerodynamic Coupling Model (ACM) and a Generalized Drag Model (GDM) using a learned neural network mixer with explicit physics-based regularization. The mixer enables smooth transitions between regimes while retaining explicit, physics-based aerodynamic representation. Model parameters are identified through a structured three-phase pipeline tailored for hybrid aerodynamic modeling. The proposed approach is validated on the RGBlimp platform through a large-scale experimental campaign comprising 1,320 real-world flight trajectories across 330 thruster and moving mass configurations, spanning a wide range of speeds and angles of attack. Experimental results demonstrate that the proposed hybrid model consistently outperforms single-model and predefined-mixer baselines, establishing a practical and robust aerodynamic modeling solution for winged blimps.
Read the full abstract ▾
All authors Xiaorui Wang, Hongwu Wang, Yue Fan, Hao Cheng, Feitian Zhang
Robotics
Trajectory Generation with Endpoint Regulation and Momentum-Aware Dynamics for Visually Impaired Scenarios
Yuting Zeng, Manping Fan, You Zhou et al. (+5 more)
Trajectory generation for visually impaired scenarios requires smooth and temporally consistent state in structured, low-speed dynamic environments. However, traditional jerk-based heuristic trajectory sampling with independent segment generation and conventional smoothness penalties often lead to unstable terminal behavior and state discontinuities under frequent regenerating. This paper proposes a trajectory generation approach that integrates endpoint regulation to stabilize terminal states within each segment and momentum-aware dynamics to regularize the
evolution of velocity and acceleration for segment consistency. Endpoint regulation is incorporated into trajectory sampling to stabilize terminal behavior, while a momentum-aware dynamics enforces consistent velocity and acceleration
evolution across consecutive trajectory segments. Experimental results demonstrate reduced acceleration peaks and lower jerk levels with decreased dispersion, smoother velocity and acceleration profiles, more stable endpoint distributions, and fewer infeasible trajectory candidates compared with a baseline planner.
Read the full abstract ▾
All authors Yuting Zeng, Manping Fan, You Zhou, Yongbin Yu, Zhiwen Zheng, Jingtao Zhang, Liyong Ren, Zhenglin Yang
Robotics
Primary-Fine Decoupling for Action Generation in Robotic Imitation
Xiaohan Lei, Min Wang, Wengang Zhou et al. (+2 more)
Multi-modal distribution in robotic manipulation action sequences poses critical challenges for imitation learning. To this end, existing approaches often model the action space as either a discrete set of
tokens or a continuous, latent-variable distribution. However, both approaches present trade-offs: some methods discretize actions into
tokens and therefore lose fine-grained action variations, while others generate continuous actions in a single stage tend to produce unstable mode transitions. To address these limitations, we propose Primary-Fine Decoupling for Action Generation (PF-DAG), a two-stage framework that decouples coarse action consistency from fine-grained variations. First, we compress action chunks into a small set of discrete modes, enabling a lightweight policy to select consistent coarse modes and avoid mode bouncing. Second, a mode conditioned MeanFlow policy is learned to generate high-fidelity continuous actions. Theoretically, we prove PF-DAG's two-stage design achieves a strictly lower MSE bound than single-stage generative policies. Empirically, PF-DAG outperforms state-of-the-art baselines across 56 tasks from Adroit, DexArt, and MetaWorld
benchmarks. It further generalizes to real-world tactile dexterous manipulation tasks. Our work demonstrates that explicit mode-level decoupling enables both robust multi-modal modeling and reactive closed-loop control for robotic manipulation.
Read the full abstract ▾
All authors Xiaohan Lei, Min Wang, Wengang Zhou, Xingyu Lu, Houqiang Li
Robotics
SunnyParking: Multi-Shot Trajectory Generation and Motion State Awareness for Human-like Parking
Jishu Miao, Han Chen, Jiankun Zhai et al. (+4 more)
Autonomous parking fundamentally differs from on-road driving due to its frequent direction changes and complex maneuvering requirements. However, existing End-to-End (E2E) planning methods often simplify the parking task into a
geometric path regression problem, neglecting explicit modeling of the vehicle's kinematic state. This "dimensionality deficiency" easily leads to physically infeasible trajectories and deviates from real human driving behavior, particularly at critical gear-shift points in multi-shot parking scenarios. In this paper, we propose SunnyParking, a novel dual-branch E2E architecture that achieves motion state awareness by jointly predicting spatial trajectories and discrete motion state sequences (e.g., forward/reverse). Additionally, we introduce a
Fourier feature-based representation of target parking slots to overcome the resolution limitations of traditional bird's-eye view (BEV) approaches, enabling high-precision target interactions. Experimental results demonstrate that our framework generates more robust and human-like trajectories in complex multi-shot parking scenarios, while significantly improving gear-shift point localization accuracy compared to state-of-the-art methods. We open-source a new parking
dataset of the CARLA simulator, specifically designed to evaluate full prediction capabilities under complex maneuvers.
Read the full abstract ▾
All authors Jishu Miao, Han Chen, Jiankun Zhai, Qi Liu, Tsubasa Hirakawa, Takayoshi Yamashita, Hironobu Fujiyoshi
Robotics
Hierarchical LLM-Based Multi-Agent Framework with Prompt Optimization for Multi-Robot Task Planning
Tomoya Kawabe and Rin Takano
Multi-
robot task planning requires decomposing natural-
language instructions into executable actions for heterogeneous
robot teams. Conventional Planning Domain Definition
Language (PDDL) planners provide rigorous guarantees but struggle to handle ambiguous or long-horizon missions, while large
language models (LLMs) can interpret instructions and propose plans but may hallucinate or produce infeasible actions. We present a hierarchical multi-
agent LLM-based planner with
prompt optimization: an upper layer decomposes tasks and assigns them to lower-layer
agents, which generate PDDL problems solved by a classical planner. When plans fail, the system applies TextGrad-inspired textual-gradient updates to optimize each
agent's
prompt and thereby improve planning accuracy. In addition, meta-
prompts are learned and shared across
agents within the same layer, enabling efficient
prompt optimization in multi-
agent settings. On the MAT-THOR
benchmark, our planner achieves success rates of 0.95 on compound tasks, 0.84 on complex tasks, and 0.60 on vague tasks, improving over the previous state-of-the-art LaMMA-P by 2, 7, and 15 percentage points respectively. An ablation study shows that the hierarchical structure,
prompt optimization, and meta-
prompt sharing contribute roughly +59, +37, and +4 percentage points to the overall success rate.
Read the full abstract ▾
Robotics
Biomechanical Comparisons Reveal Divergence of Human and Humanoid Gaits
Luying Feng, Yaochu Jin, Hanze Hu et al. (+1 more)
It remains challenging to achieve human-like locomotion in legged
robots due to fundamental discrepancies between biological and mechanical structures. Although imitation learning has emerged as a promising approach for generating natural robotic movements, simply replicating joint angle trajectories fails to capture the underlying principles of human motion. This study proposes a Gait Divergence Analysis Framework (GDAF), a unified biomechanical evaluation framework that systematically quantifies kinematic and kinetic discrepancies between humans and bipedal
robots. We apply GDAF to systematically compare human and
humanoid locomotion across 28 walking speeds. To enable reproducible analysis, we collect and release a speed-continuous
humanoid locomotion
dataset from a state-of-the-art
humanoid controller. We further provide an open-source implementation of GDAF, including analysis, visualization, and MuJoCo-based tools, enabling quantitative, interpretable, and reproducible biomechanical analysis of
humanoid locomotion. Results demonstrate that despite visually human-like motion generated by modern
humanoid controllers, significant biomechanical divergence persists across speeds.
Robots exhibit systematic deviations in gait
symmetry, energy distribution, and joint coordination, indicating that substantial room remains for improving the biomechanical fidelity and energetic efficiency of
humanoid locomotion. This work provides a quantitative
benchmark for evaluating
humanoid locomotion and offers data and versatile tools to support the development of more human-like and energetically efficient locomotion controllers. The data and code will be made publicly available upon acceptance of the paper.
Read the full abstract ▾
All authors Luying Feng, Yaochu Jin, Hanze Hu, Wei Chen
Robotics
DAGS-SLAM: Dynamic-Aware 3DGS SLAM via Spatiotemporal Motion Probability and Uncertainty-Aware Scheduling
Li Zhang, Yu-An Liu, Xijia Jiang et al. (+3 more)
Mobile
robots and IoT devices demand real-time localization and dense reconstruction under tight compute and energy budgets. While 3D Gaussian Splatting (3DGS) enables efficient dense SLAM, dynamic objects and occlusions still degrade tracking and mapping. Existing dynamic 3DGS-SLAM often relies on heavy optical flow and per-frame
segmentation, which is costly for mobile deployment and brittle under challenging illumination. We present DAGS-SLAM, a dynamic-aware 3DGS-SLAM system that maintains a spatiotemporal motion probability (MP) state per Gaussian and triggers semantics on demand via an uncertainty-aware scheduler. DAGS-SLAM fuses lightweight YOLO instance priors with
geometric cues to estimate and temporally update MP, propagates MP to the front-end for dynamic-aware correspondence selection, and suppresses dynamic artifacts in the back-end via MP-guided optimization. Experiments on public dynamic RGB-D
benchmarks show improved reconstruction and robust tracking while sustaining real-time throughput on a commodity GPU, demonstrating a practical speed-accuracy tradeoff with reduced semantic invocations toward mobile deployment.
Read the full abstract ▾
All authors Li Zhang, Yu-An Liu, Xijia Jiang, Conghao Huang, Danyang Li, Yanyong Zhang
Robotics
Self-Correcting VLA: Online Action Refinement via Sparse World Imagination
Chenyv Liu, Wentao Tan, Lei Zhu et al. (+4 more)
Standard
vision-
language-action (VLA) models rely on fitting statistical data priors, limiting their robust understanding of underlying physical dynamics. Reinforcement learning enhances physical grounding through exploration yet typically relies on external reward signals that remain isolated from the
agent's internal states. World action models have emerged as a promising paradigm that integrates imagination and control to enable predictive planning. However, they rely on implicit context modeling, lacking explicit mechanisms for self-improvement. To solve these problems, we propose Self-Correcting VLA (SC-VLA), which achieve self-improvement by intrinsically guiding action refinement through sparse imagination. We first design sparse world imagination by integrating auxiliary predictive heads to forecast current task progress and future trajectory trends, thereby constraining the policy to encode short-term physical
evolution. Then we introduce the online action refinement module to reshape progress-dependent dense rewards, adjusting trajectory orientation based on the predicted sparse future states. Evaluations on challenging
robot manipulation tasks from simulation
benchmarks and real-world settings demonstrate that SC-VLA achieve state-of-the-art performance, yielding the highest task throughput with 16% fewer steps and a 9% higher success rate than the best-performing baselines, alongside a 14% gain in real-world experiments. Code is available at https://github.com/Kisaragi0/SC-VLA.
Read the full abstract ▾
All authors Chenyv Liu, Wentao Tan, Lei Zhu, Fengling Li, Jingjing Li, Guoli Yang, Heng Tao Shen
Robotics
Tacmap: Bridging the Tactile Sim-to-Real Gap via Geometry-Consistent Penetration Depth Map
Lei Su, Zhijie Peng, Renyuan Ren et al. (+4 more)
Vision-Based Tactile Sensors (VBTS) are essential for achieving dexterous robotic manipulation, yet the tactile sim-to-real gap remains a fundamental bottleneck. Current tactile simulations suffer from a persistent dilemma: simplified
geometric projections lack physical authenticity, while high-fidelity Finite Element Methods (FEM) are too computationally prohibitive for large-scale reinforcement learning. In this work, we present Tacmap, a high-fidelity, computationally efficient tactile simulation framework anchored in volumetric penetration depth. Our key insight is to bridge the tactile sim-to-real gap by unifying both domains through a shared deform map representation. Specifically, we compute 3D intersection volumes as depth maps in simulation, while in the real world, we employ an automated data-collection rig to learn a robust mapping from raw tactile images to ground-truth depth maps. By aligning simulation and real-world in this unified
geometric space, Tacmap minimizes domain shift while maintaining physical consistency. Quantitative evaluations across diverse contact scenarios demonstrate that Tacmap's deform maps closely mirror real-world measurements. Moreover, we validate the utility of Tacmap through an in-hand rotation task, where a policy trained exclusively in simulation achieves zero-shot transfer to a physical
robot.
Read the full abstract ▾
All authors Lei Su, Zhijie Peng, Renyuan Ren, Shengping Mao, Juan Du, Kaifeng Zhang, Xuezhou Zhu
Robotics
ADM-DP: Adaptive Dynamic Modality Diffusion Policy through Vision-Tactile-Graph Fusion for Multi-Agent Manipulation
Enyi Wang, Wen Fan and Dandan Zhang
Multi-
agent robotic manipulation remains challenging due to the combined demands of coordination, grasp stability, and collision avoidance in shared workspaces. To address these challenges, we propose the Adaptive Dynamic Modality
Diffusion Policy (ADM-DP), a framework that integrates
vision, tactile, and graph-based (multi-
agent pose) modalities for coordinated control. ADM-DP introduces four key innovations. First, an enhanced visual encoder merges RGB and point-cloud features via Feature-wise Linear Modulation (FiLM) modulation to enrich perception. Second, a tactile-guided grasping strategy uses Force-Sensitive Resistor (FSR) feedback to detect insufficient contact and trigger corrective grasp refinement, improving grasp stability. Third, a graph-based collision encoder leverages shared tool center point (TCP) positions of multiple
agents as structured kinematic context to maintain spatial awareness and reduce inter-
agent interference. Fourth, an Adaptive Modality
Attention Mechanism (AMAM) dynamically re-weights modalities according to task context, enabling flexible fusion. For scalability and modularity, a decoupled training paradigm is employed in which
agents learn independent policies while sharing spatial information. This maintains low interdependence between
agents while retaining collective awareness. Across seven multi-
agent tasks, ADM-DP achieves 12-25% performance gains over state-of-the-art baselines. Ablation studies show the greatest improvements in tasks requiring multiple sensory modalities, validating our adaptive fusion strategy and demonstrating its robustness for diverse manipulation scenarios.
Read the full abstract ▾
Robotics
Jumping Control for a Quadrupedal Wheeled-Legged Robot via NMPC and DE Optimization
Xuanqi Zeng, Lingwei Zhang, Linzhu Yue et al. (+4 more)
Quadrupedal wheeled-legged
robots combine the advantages of legged and wheeled locomotion to achieve superior mobility, but executing dynamic jumps remains a significant challenge due to the additional degrees of freedom introduced by wheeled legs. This paper develops a mini-sized wheeled-legged
robot for agile motion and presents a novel motion control framework that integrates the Nonlinear Model Predictive Control (NMPC) for locomotion and the Differential
Evolution (DE) based trajectory optimization for jumping in quadrupedal wheeled-legged
robots. The proposed controller utilizes wheel motion and locomotion to enhance jumping performance, achieving versatile maneuvers such as vertical jumping, forward jumping, and backflips. Extensive simulations and real-world experiments validate the effectiveness of the framework, demonstrating a forward jump over a 0.12 m obstacle and a vertical jump reaching 0.5 m.
Read the full abstract ▾
All authors Xuanqi Zeng, Lingwei Zhang, Linzhu Yue, Zhitao Song, Hongbo Zhang, Tianlin Zhang, Yun-Hui Liu
Robotics
Iterative Closed-Loop Motion Synthesis for Scaling the Capabilities of Humanoid Control
Weisheng Xu, Qiwei Wu, Jiaxi Zhang et al. (+7 more)
Physics-based
humanoid control relies on training with motion
datasets that have diverse data distributions. However, the fixed difficulty distribution of
datasets limits the performance ceiling of the trained control policies. Additionally, the method of acquiring high-quality data through professional motion capture systems is constrained by costs, making it difficult to achieve large-scale scalability. To address these issues, we propose a closed-loop automated motion data generation and iterative framework. It can generate high-quality motion data with rich action semantics, including martial arts, dance, combat, sports, gymnastics, and more. Furthermore, our framework enables difficulty iteration of policies and data through physical metrics and objective evaluations, allowing the trained tracker to break through its original difficulty limits. On the PHC single-primitive tracker, using only approximately 1/10 of the AMASS
dataset size, the average failure rate on the test set (2201 clips) is reduced by 45% compared to the baseline. Finally, we conduct comprehensive ablation and comparative experiments to highlight the rationality and advantages of our framework.
Read the full abstract ▾
All authors Weisheng Xu, Qiwei Wu, Jiaxi Zhang, Tan Jing, Yangfan Li, Yuetong Fang, Jiaqi Xiong, Kai Wu, Rong Ou, Renjing Xu
Robotics
SPOC: Safety-Aware Planning Under Partial Observability And Physical Constraints
Hyungmin Kim, Hobeom Jeon, Dohyung Kim et al. (+2 more)
Embodied Task Planning with large
language models faces safety challenges in real-world environments, where partial observability and physical constraints must be respected. Existing
benchmarks often overlook these critical factors, limiting their ability to evaluate both feasibility and safety. We introduce SPOC, a
benchmark for safety-aware
embodied task planning, which integrates strict partial observability, physical constraints, step-by-step planning, and goal-condition-based evaluation. Covering diverse household hazards such as fire, fluid, injury, object damage, and pollution, SPOC enables rigorous assessment through both state and constraint-based online metrics. Experiments with state-of-the-art LLMs reveal that current models struggle to ensure safety-aware planning, particularly under implicit constraints. Code and
dataset are available at https://github.com/khm159/SPOC
Read the full abstract ▾
All authors Hyungmin Kim, Hobeom Jeon, Dohyung Kim, Minsu Jang, Jeahong Kim
Robotics
Learning End-to-End Control for Omnidirectional Aerial Motion on Overactuated Tilt-rotor Quadrotors
Wentao Zhang, Zhaoqi Ma, Jinjie Li et al. (+8 more)
While reinforcement learning (RL) has been successfully applied to conventional quadrotors for agile and robust flight, whether actuator-level RL can be reliably deployed on tilt-rotor aerial
robots remains an open question, as the hybrid actuation coupling brushless rotors with rotational joints introduces a substantially harder sim-to-real gap. In this work, we propose an end-to-end RL framework for omnidirectional motion control on overactuated tilt-rotor quadrotors, directly mapping target poses to joint and rotor commands. The learning framework combines actuator-level simulation with an asymmetric actor-critic architecture for 6D pose-reaching. For reliable sim-to-real transfer on the hybrid actuation, we integrate system identification with minimal yet physically grounded domain randomization. The trained policy is deployed zero-shot on real hardware and evaluated across waypoint hovering, external disturbances, payload variation and trajectory tracking, together with simulated traversal of allocation-singular configurations. The policy is compared with a state-of-the-art NMPC baseline: NMPC attains lower steady-state position error, whereas the RL policy offers a more uniform orientation error across evaluations, transitions between poses faster, and requires less onboard computation.
Read the full abstract ▾
All authors Wentao Zhang, Zhaoqi Ma, Jinjie Li, Huayi Wang, Haokun Liu, Junichiro Sugihara, Chen Chen, Yicheng Chen, Cuniato Eugenio, Pantic Michael, Moju Zhao
Robotics
LiLo-VLA: Compositional Long-Horizon Manipulation via Linked Object-Centric Policies
Yue Yang, Shuo Cheng, Yu Fang et al. (+4 more)
General-purpose
robots must master long-horizon manipulation, defined as tasks involving multiple kinematic structure changes (e.g., attaching or detaching objects) in unstructured environments. While
Vision-
Language-Action (VLA) models offer the potential to master diverse atomic skills, they struggle with the combinatorial complexity of sequencing them and are prone to cascading failures due to environmental sensitivity. To address these challenges, we propose LiLo-VLA (Linked Local VLA), a modular framework capable of zero-shot generalization to novel long-horizon tasks without ever being trained on them. Our approach decouples transport from interaction: a Reaching Module handles global motion, while an Interaction Module employs an object-centric VLA to process isolated objects of interest, ensuring robustness against irrelevant visual features and invariance to spatial configurations. Crucially, this modularity facilitates robust failure recovery through dynamic replanning and skill reuse, effectively mitigating the cascading errors common in end-to-end approaches. We introduce a 21-task simulation
benchmark consisting of two challenging suites: LIBERO-Long++ and Ultra-Long. In these simulations, LiLo-VLA achieves a 69% average success rate, outperforming Pi0.5 by 41% and OpenVLA-OFT by 67%. Furthermore, real-world evaluations across 8 long-horizon tasks demonstrate an average success rate of 85%. Project page: https://yy-gx.github.io/LiLo-VLA/.
Read the full abstract ▾
All authors Yue Yang, Shuo Cheng, Yu Fang, Homanga Bharadhwaj, Mingyu Ding, Gedas Bertasius, Daniel Szafir
Robotics
Vector Fields for Path Following on Lie Groups with Application in Robot Control
Felipe Bartelt, Luciano C. A. Pimenta, Weijia Yao et al. (+1 more)
Many robotic systems allow independent control of position and orientation (pose), including omnidirectional aerial vehicles,
underwater robots, and manipulator end-effectors. In many applications, these systems must follow a continuous sequence of poses, leading to either trajectory-tracking or path following formulations. Compared to trajectory-tracking, path following offers important practical advantages. In particular, we focus on the problem of path following on Lie groups. Considering the
robots as rigid bodies moving in the 3D space, this path-following problem can be posed as a problem of designing guiding vector fields on the
matrix Lie group SE(3). In this paper, we develop a general vector-field framework for path following on connected
matrix Lie groups, of which SE(3) is a prominent special case. The proposed vector field guarantees convergence to a desired parametric curve from almost all initial conditions while ensuring continuous motion along the path. Furthermore, another interesting feature is that, as opposed to previous works, the control input is "minimal" in terms of representation and closer to the engineering application (e.g., the body twist in the case SE(3)). After establishing the general case, the framework is then specialized to SE(3), of special interest in robotics, yielding an efficient
algorithm suitable for real-time robotic control. Experiments with a robotic manipulator tracking complex pose paths demonstrate the effectiveness of the approach. An open-source implementation is also provided.
Read the full abstract ▾
All authors Felipe Bartelt, Luciano C. A. Pimenta, Weijia Yao, Vinicius M. Gonçalves