Robotics
SaPaVe: Towards Active Perception and Manipulation in Vision-Language-Action Models for Robotics
Mengzhen Liu, Enshen Zhou, Cheng Chi et al. (+6 more)
Active perception and manipulation are crucial for
robots to interact with complex scenes. Existing methods struggle to unify semantic-driven active perception with robust, viewpoint-invariant execution. We propose SaPaVe, an end-to-end framework that jointly learns these capabilities in a data-efficient manner. Our approach decouples camera and manipulation actions rather than placing them in a shared action space, and follows a bottom-up training strategy: we first train semantic camera control on a large-scale
dataset, then jointly optimize both action types using hybrid data. To support this framework, we introduce ActiveViewPose-200K, a
dataset of 200k image-
language-camera movement pairs for semantic camera movement learning, and a 3D geometry-aware module that improves execution robustness under dynamic viewpoints. We also present ActiveManip-Bench, the first
benchmark for evaluating active manipulation beyond fixed-view settings. Extensive experiments in both simulation and real-world environments show that SaPaVe outperforms recent
vision-
language-action models such as GR00T N1 and \(\pi_0\), achieving up to 31.25% higher success rates in real-world tasks. These results show that tightly coupled perception and execution, when trained with decoupled yet coordinated strategies, enable efficient and generalizable active manipulation. Project page: https://lmzpai.github.io/SaPaVe
Read the full abstract ▾
All authors Mengzhen Liu, Enshen Zhou, Cheng Chi, Yi Han, Shanyu Rong, Liming Chen, Pengwei Wang, Zhongyuan Wang, Shanghang Zhang
Robotics
ComFree-Sim: A GPU-Parallelized Analytical Contact Physics Engine for Scalable Contact-Rich Robotics Simulation and Control
Chetan Borse, Zhixian Xie, Wei-Cheng Huang et al. (+1 more)
Physics simulation for contact-rich robotics is often bottlenecked by contact resolution: mainstream engines enforce non-penetration and Coulomb friction via complementarity constraints or constrained optimization, requiring per-step iterative solves whose cost grows superlinearly with contact density. We present ComFree-Sim, a GPU-parallelized analytical contact physics engine built on complementarity-free contact modeling. ComFree-Sim computes contact impulses in closed form via an impedance-style prediction–correction update in the dual cone of Coulomb friction. Contact computation decouples across contact pairs and becomes separable across cone facets, mapping naturally to GPU kernels and yielding near-linear runtime scaling with the number of contacts. We further extend the formulation to a unified 6D contact model capturing tangential, torsional, and rolling friction, and introduce a practical dual-cone impedance heuristic. ComFree-Sim is implemented in Warp and exposed through a MuJoCo-compatible interface as a drop-in backend alternative to MuJoCo Warp (MJWarp). Experiments
benchmark penetration, friction behaviors, stability, and simulation runtime scaling against MJWarp, demonstrating near-linear scaling and 2–3 times higher throughput in dense contact scenes with comparable physical fidelity. We deploy ComFree-Sim in real-time MPC for in-hand dexterous manipulation on a real-world multi-fingered LEAP hand and in dynamics-aware motion retargeting, demonstrating that low-latency simulation yields higher closed-loop success rates and enables practical high-frequency control in contact-rich tasks.
Read the full abstract ▾
All authors Chetan Borse, Zhixian Xie, Wei-Cheng Huang, Wanxin Jin
Robotics · Cross-listed from cs.CV
O3N: Omnidirectional Open-Vocabulary Occupancy Prediction for Embodied Intelligent Robotics
Mengfei Duan, Hao Shi, Fei Teng et al. (+4 more)
The rapid
evolution of consumer electronics toward
embodied intelligence has accelerated the emergence of Consumer
Embodied Intelligent Robotics (CEIRs), where intelligent devices are expected to perceive, understand, and interact with complex real-world environments. Understanding and reconstructing the 3D world through omnidirectional perception is therefore becoming increasingly important for CEIRs operating in complex and dynamic environments. However, existing
vision-based 3D occupancy prediction methods are constrained by limited perspective inputs and a predefined training distribution, making them difficult to support
embodied intelligent systems that require comprehensive and safe perception of scenes in open-world exploration. To address this, we present O3N, the first framework for open-vocabulary occupancy prediction from a single omnidirectional RGB image. O3N embeds omnidirectional voxels in a polar-spiral
topology via the Polar-spiral Mamba (PsM) module, enabling continuous spatial representation and long-range context modeling across 360{\deg}. The Occupancy Cost Aggregation (OCA) module introduces a principled mechanism for unifying
geometric and semantic supervision within the voxel space, ensuring consistency between reconstructed geometry and underlying semantic structure. Moreover, Natural Modality
Alignment (NMA) establishes a gradient-free
alignment pathway that harmonizes visual features, voxel
embeddings, and text semantics, forming a consistent pixel-voxel-text representation triad for open-world perception. Extensive experiments on multiple models demonstrate that our method not only achieves state-of-the-art performance on QuadOcc and Human360Occ
benchmarks but also exhibits remarkable cross-scene generalization and semantic scalability. The source code will be made publicly available at https://github.com/MengfeiD/O3N.
Read the full abstract ▾
All authors Mengfei Duan, Hao Shi, Fei Teng, Guoqiang Zhao, Yuheng Zhang, Zhiyong Li, Kailun Yang
Robotics
CRAFT: A Tendon-Driven Hand with Hybrid Hard-Soft Compliance
Leo Lin, Shivansh Patel, Jay Moon et al. (+2 more)
We introduce CRAFT hand, a tendon-driven anthropomorphic hand with hybrid hard-soft compliance for contact-rich manipulation. The design is based on a simple idea: contact is not uniform across the hand. Impacts concentrate at joints, while links carry most of the load. CRAFT places soft material at joints and keeps links rigid, and uses rollingcontact joint surfaces to keep flexion on repeatable motion paths. Fifteen motors mounted on the fingers drive the hand through tendons, keeping the form factor compact and the fingers light. In structural tests, CRAFT improves strength and endurance while maintaining comparable repeatability. In teleoperation, CRAFT improves handling of fragile and low-friction items, and the hand covers 33/33 grasps in the Feix taxonomy. The full design costs under $600 and will be released open-source with visionbased teleoperation and simulation integration. Project page: http://craft-hand.github.io/
Read the full abstract ▾
All authors Leo Lin, Shivansh Patel, Jay Moon, Svetlana Lazebnik, Unnat Jain
Robotics
Dynamic Model Identification and Gravity Compensation for the dVRK-Si Patient Side Manipulator
Haoying Zhou, Hao Yang, Brendan Burkhart et al. (+6 more)
The da Vinci Research Kit (dVRK) has been widely used for research in
robot-assisted surgery over the past decade, and its latest generation, the dVRK-Si was made publicly available in 2025. This manuscript presents the first complete kinematic and dynamic modeling framework for the dVRK-Si Patient-Side Manipulator (PSM), whose redesigned mechanism differs substantially from that of the first-generation dVRK and therefore requires a new formulation to accurately capture gravitational effects and improve overall control performance. We first develop a modified DH kinematic model that represents the dVRK-Si PSM's redesigned mechanism. The equations of motion are then derived using the Euler-Lagrange formulation, and the inverse dynamics are expressed in a linear-in-parameters regressor form. The dynamic parameters are identified from data collected along a periodic excitation trajectory optimized for numerical conditioning and estimated by convex optimization subject to the physical feasibility constraints. The identified model is integrated into the dVRK control stack to enable real-time
gravity compensation and computed-torque feedforward control. Experiments on a physical dVRK-Si demonstrate that the
gravity compensation reduces steady-state joint errors by up to 84% and decreases end-effector tip drift during static holds from 4.2 mm to 0.7 mm. Computed-torque feedforward control further improves transient and trajectory-tracking accuracy. During sinusoidal trajectory tracking, computed-torque feedforward reduces position errors by 35% versus
gravity-only feedforward and by 40% versus PID-only. The proposed pipeline could support reliable control, high-fidelity simulation, and learning-based automation on the dVRK-Si.
Read the full abstract ▾
All authors Haoying Zhou, Hao Yang, Brendan Burkhart, Anton Deguet, Zhonghao Zhang, Loris Fichera, Gregory S. Fischer, Jie Ying Wu, Peter Kazanzides
Robotics · Cross-listed from cs.CV
Towards Universal Computational Aberration Correction in Photographic Cameras: A Comprehensive Benchmark Analysis
Xiaolong Qian, Qi Jiang, Yao Gao et al. (+5 more)
Prevalent Computational Aberration Correction (CAC) methods are typically tailored to specific optical systems, leading to poor generalization and labor-intensive re-training for new lenses. Developing CAC paradigms capable of generalizing across diverse photographic lenses offers a promising solution to these challenges. However, efforts to achieve such cross-lens universality within consumer photography are still in their early stages due to the lack of a comprehensive
benchmark that encompasses a sufficiently wide range of optical aberrations. Furthermore, it remains unclear which specific factors influence existing CAC methods and how these factors affect their performance. In this paper, we present comprehensive experiments and evaluations involving 24 image restoration and CAC
algorithms, utilizing our newly proposed UniCAC, a large-scale
benchmark for photographic cameras constructed via automatic optical design. The Optical Degradation Evaluator (ODE) is introduced as a novel framework to objectively assess the difficulty of CAC tasks, offering credible quantification of optical aberrations and enabling reliable evaluation. Drawing on our comparative analysis, we identify three key factors – prior utilization, network architecture, and training strategy – that most significantly influence CAC performance, and further investigate their respective effects. We believe that our
benchmark,
dataset, and observations contribute foundational insights to related areas and lay the groundwork for future investigations.
Benchmarks, codes, and Zemax files will be available at https://github.com/XiaolongQian/UniCAC.
Read the full abstract ▾
All authors Xiaolong Qian, Qi Jiang, Yao Gao, Lei Sun, Zhonghua Yi, Kailun Yang, Luc Van Gool, Kaiwei Wang
Robotics
Decentralized Cooperative Localization for Multi-Robot Systems with Asynchronous Sensor Fusion
Nivand Khosravi, Niusha Khosravi, Mohammad Bozorg et al. (+1 more)
Decentralized cooperative localization (DCL) is a promising approach for nonholonomic mobile
robots operating in GPS-denied environments with limited communication infrastructure. This paper presents a DCL framework in which each
robot performs localization locally using an Extended Kalman Filter, while sharing measurement information during update stages only when communication links are available and companion
robots are successfully detected by
LiDAR. The framework preserves cross-correlation consistency among
robot state estimates while handling asynchronous sensor data with heterogeneous sampling rates and accommodating accelerations during dynamic maneuvers. Unlike methods that require pre-aligned coordinate systems, the proposed approach allows
robots to initialize with arbitrary reference-frame orientations and achieves automatic
alignment through transformation
matrices in both the prediction and update stages. To improve robustness in feature-sparse environments, we introduce a dual-landmark evaluation framework that exploits both static environmental features and mobile
robots as dynamic landmarks. The proposed framework enables reliable detection and feature extraction during sharp turns, while prediction accuracy is improved through information sharing from mutual observations. Experimental results in both Gazebo simulation and real-world basement environments show that DCL outperforms centralized cooperative localization (CCL), achieving a 34% reduction in RMSE, while the dual-landmark variant yields an improvement of 56%. These results demonstrate the applicability of DCL to challenging domains such as enclosed spaces,
underwater environments, and feature-sparse terrains where conventional localization methods are ineffective.
Read the full abstract ▾
All authors Nivand Khosravi, Niusha Khosravi, Mohammad Bozorg, Masoud S. Bahraini
Robotics
Flight through Narrow Gaps with Morphing-Wing Drones
Julius Wanner, Hoang-Vu Phan, Charbel Toumieh et al. (+1 more)
The size of a narrow gap traversable by a fixed-wing drone is limited by its wingspan. Inspired by birds, here, we enable the traversal of a gap of sub-wingspan width and height using a morphing-wing drone capable of temporarily sweeping in its wings mid-flight. This maneuver poses control challenges due to sudden lift loss during gap-passage at low flight speeds and the need for precisely timed wing-sweep actuation ahead of the gap. To address these challenges, we first develop an aerodynamic model for general wing-sweep morphing drone flight including low flight speeds and post-stall angles of attack. We integrate longitudinal drone dynamics into an optimal reference trajectory generation and Nonlinear Model Predictive Control framework with runtime adaptive costs and constraints. Validated on a 130 g wing-sweep-morphing drone, our method achieves an average altitude error of 5 cm during narrow-gap passage at forward speeds between 5 and 7 m/s, whilst enforcing fully swept wings near the gap across variable threshold distances. Trajectory analysis shows that the drone can compensate for lift loss during gap-passage by accelerating and pitching upwards ahead of the gap to an extent that differs between reference trajectory optimization objectives. We show that our strategy also allows for accurate gap passage on hardware whilst maintaining a constant forward flight speed reference and near-constant altitude.
Read the full abstract ▾
All authors Julius Wanner, Hoang-Vu Phan, Charbel Toumieh, Dario Floreano
Robotics
Sim-to-reality adaptation for Deep Reinforcement Learning applied to an underwater docking application
Alaaeddine Chaarani, Narcis Palomeras and Pere Ridao
Deep Reinforcement Learning (DRL) offers a robust alternative to traditional control methods for autonomous
underwater docking, particularly in adapting to unpredictable environmental conditions. However, bridging the "sim-to-real" gap and managing high training latencies remain significant bottlenecks for practical deployment. This paper presents a systematic approach for autonomous docking using the Girona Autonomous
Underwater Vehicle (AUV) by leveraging a high-fidelity digital twin environment. We adapted the Stonefish simulator into a multiprocessing RL framework to significantly accelerate the learning process while incorporating realistic AUV dynamics, collision models, and sensor noise. Using the Proximal Policy Optimization (PPO)
algorithm, we developed a 6-DoF control policy trained in a headless environment with randomized starting positions to ensure generalized performance. Our reward structure accounts for distance, orientation, action smoothness, and adaptive collision penalties to facilitate soft docking. Experimental results demonstrate that the
agent achieved a success rate of over 90% in simulation. Furthermore, successful validation in a physical test tank confirmed the efficacy of the sim-to-reality adaptation, with the DRL controller exhibiting emergent behaviors such as pitch-based braking and yaw oscillations to assist in mechanical
alignment.
Read the full abstract ▾
Robotics
Learning Visuomotor Policy for Multi-Robot Laser Tag Game
Kai Li and Shiyu Zhao
In this paper, we study multi
robot laser tag, a simplified yet practical shooting-game-style task. Classic modular approaches on these tasks face challenges such as limited observability and reliance on depth mapping and inter
robot communication. To overcome these issues, we present an end-to-end visuomotor policy that maps images directly to
robot actions. We train a high performing teacher policy with multi
agent reinforcement learning and distill its knowledge into a
vision-based student policy. Technical designs, including a permutation-invariant feature extractor and depth heatmap input, improve performance over standard architectures. Our policy outperforms classic methods by 16.7% in hitting accuracy and 6% in collision avoidance, and is successfully deployed on real
robots. Code will be released publicly.
Read the full abstract ▾
Robotics
Energy Prediction on Sloping Ground for Quadruped Robots
Mohamed Ounally, Cyrille Pierre and Johann Laconte
Energy management is a fundamental challenge for legged
robots in outdoor environments. Endurance directly constrains mission success, while efficient resource use reduces ecological impact. This paper investigates how terrain slope and heading orientation influence the energetic cost of quadruped locomotion. We introduce a simple energy model that relies solely on standard onboard sensors, avoids specialized instrumentation, and remains applicable in previously unexplored environments. The model is identified from field runs on a commercial quadruped and expressed as a compact function of slope angle and heading. Field validation on natural terrain shows near-linear trends of force-equivalent cost with slope angle, consistently higher lateral costs, and additive behavior across trajectory segments, supporting path-level energy prediction for planning-oriented evaluation.
Read the full abstract ▾
Robotics
RADAR: Closed-Loop Robotic Data Generation via Semantic Planning and Autonomous Causal Environment Reset
Yongzhong Wang, Keyu Zhu, Yong Zhong et al. (+3 more)
The acquisition of large-scale physical interaction data, a critical prerequisite for modern
robot learning, is severely bottlenecked by the prohibitive cost and scalability limits of human-in-the-loop collection paradigms. To break this barrier, we introduce Robust Autonomous Data Acquisition for Robotics (RADAR), a fully autonomous, closed-loop data generation engine that completely removes human intervention from the collection cycle. RADAR elegantly divides the cognitive load into a four-module pipeline. Anchored by 2-5 3D human demonstrations as
geometric priors, a
Vision-
Language Model first orchestrates scene-relevant task generation via precise semantic object grounding and skill
retrieval. Next, a Graph Neural Network policy translates these subtasks into physical actions via in-context imitation learning. Following execution, the VLM performs automated success evaluation using a structured Visual Question Answering pipeline. Finally, to shatter the bottleneck of manual resets, a Finite State Machine orchestrates an autonomous environment reset and asymmetric data routing mechanism. Driven by simultaneous forward-reverse planning with a strict Last-In, First-Out causal sequence, the system seamlessly restores unstructured workspaces and robustly recovers from execution failures. This continuous
brain-cerebellum synergy transforms data collection into a self-sustaining process. Extensive evaluations highlight RADAR's exceptional versatility. In simulation, our framework achieves up to 90% success rates on complex, long-horizon tasks, effortlessly solving challenges where traditional baselines plummet to near-zero performance. In real-world deployments, the system reliably executes diverse, contact-rich skills (e.g., deformable object manipulation) via few-shot adaptation without domain-specific fine-tuning, providing a highly scalable paradigm for robotic data acquisition.
Read the full abstract ▾
All authors Yongzhong Wang, Keyu Zhu, Yong Zhong, Liqiong Wang, Jinyu Yang, Feng Zheng
Robotics · Cross-listed from cs.HC
HiSync: Spatio-Temporally Aligning Hand Motion from Wearable IMU and On-Robot Camera for Command Source Identification in Long-Range HRI
Chengwen Zhang, Chun Yu, Borong Zhuang et al. (+9 more)
Long-range Human-
Robot Interaction (HRI) remains underexplored. Within it, Command Source Identification (CSI) - determining who issued a command - is especially challenging due to multi-user and distance-induced sensor ambiguity. We introduce HiSync, an optical-inertial fusion framework that treats hand motion as binding cues by aligning
robot-mounted camera optical flow with hand-worn IMU signals. We first elicit a user-defined (N=12) gesture set and collect a
multimodal command gesture
dataset (N=38) in long-range multi-user HRI scenarios. Next, HiSync extracts frequency-domain hand motion features from both camera and IMU data, and a learned CSINet denoises IMU readings, temporally aligns modalities, and performs distance-aware multi-window fusion to compute cross-modal similarity of subtle, natural gestures, enabling robust CSI. In three-person scenes up to 34m, HiSync achieves 92.32% CSI accuracy, outperforming the prior SOTA by 48.44%. HiSync is also validated on real-
robot deployment. By making CSI reliable and natural, HiSync provides a practical primitive and design guidance for public-space HRI. https://github.com/OctopusWen/HiSync
Read the full abstract ▾
All authors Chengwen Zhang, Chun Yu, Borong Zhuang, Haopeng Jin, Qingyang Wan, Zhuojun Li, Zhe He, Zhoutong Ye, Yu Mei, Chang Liu, Weinan Shi, Yuanchun Shi
Robotics · Cross-listed from cs.DS
Adapting Dijkstra for Buffers and Unlimited Transfers
Denys Katkalo, Andrii Rohovyi and Toby Walsh
In recent years, RAPTOR based
algorithms have been considered the state-of-the-art for path-finding with unlimited transfers without preprocessing. However, this status largely stems from the
evolution of routing research, where Dijkstra-based solutions were superseded by timetable-based
algorithms without a systematic comparison. In this work, we revisit classical Dijkstra-based approaches for public transit routing with unlimited transfers and demonstrate that Time-Dependent Dijkstra (TD-Dijkstra) outperforms MR. However, efficient TD-Dijkstra implementations rely on filtering dominated connections during preprocessing, which assumes passengers can always switch to a faster connection. We show that this filtering is unsound when stops have buffer times, as it cannot distinguish between seated passengers who may continue without waiting and transferring passengers who must respect the buffer. To address this limitation, we introduce Transfer Aware Dijkstra (TAD), a modification that scans entire trip sequences rather than individual edges, correctly handling buffer times while maintaining performance advantages over MR. Our experiments on the London and Switzerland networks show that we can achieve more than a twofold speedup over MR while producing optimal results on both networks, with and without buffer times.
Read the full abstract ▾
Robotics
Coupling Tensor Trains with Graph of Convex Sets: Effective Compression, Exploration, and Planning in the C-Space
Gerhard Reinerth, Riddhiman Laha and Marcello Romano
We present TANGO (Tensor ANd Graph Optimization), a novel motion planning framework that integrates tensor-based compression with structured graph optimization to enable efficient and scalable trajectory generation. While optimization-based planners such as the Graph of Convex Sets (GCS) offer powerful tools for generating smooth, optimal trajectories, they typically rely on a predefined convex characterization of the high-dimensional configuration space-a requirement that is often intractable for general robotic tasks. TANGO builds further by using Tensor Train decomposition to approximate the feasible configuration space in a compressed form, enabling rapid discovery and estimation of task-relevant regions. These regions are then embedded into a GCS-like structure, allowing for geometry-aware motion planning that respects both system constraints and environmental complexity. By coupling tensor-based compression with structured graph
reasoning, TANGO enables efficient, geometry-aware motion planning and lays the groundwork for more expressive and scalable representations of configuration space in future robotic systems. Rigorous simulation studies on planar and real
robots reinforce our claims of effective compression and higher quality trajectories.
Read the full abstract ▾
Robotics
Concurrent Prehensile and Nonprehensile Manipulation: A Practical Approach to Multi-Stage Dexterous Tasks
Hao Jiang, Yue Wu, Yue Wang et al. (+2 more)
Dexterous hands enable concurrent prehensile and nonprehensile manipulation, such as holding one object while interacting with another, a capability essential for everyday tasks yet underexplored in robotics. Learning such long-horizon, contact-rich multi-stage behaviors is challenging because demonstrations are expensive to collect and end-to-end policies require substantial data to generalize across varied object geometries and placements. We present DexMulti, a sample-efficient approach for real-world dexterous multi-task manipulation that decomposes demonstrations into object-centric skills with well-defined temporal boundaries. Rather than learning monolithic policies, our method retrieves demonstrated skills based on current object geometry, aligns them to the observed object state using an uncertainty-aware
estimator that tracks centroid and yaw, and executes them via a retrieve-align-execute paradigm. We evaluate on three multi-stage tasks requiring concurrent manipulation (Grasp + Pull, Grasp + Open, and Grasp + Grasp) across two dexterous hands (Allegro and LEAP) in over 1,000 real-world trials. Our approach achieves an average success rate of 66% on training objects with only 3-4 demonstrations per object, outperforming
diffusion policy baselines by 2-3x while requiring far fewer demonstrations. Results demonstrate robust generalization to held-out objects and spatial variations up to +/-25 cm.
Read the full abstract ▾
All authors Hao Jiang, Yue Wu, Yue Wang, Gaurav S. Sukhatme, Daniel Seita
Robotics · Cross-listed from cs.LG
Simple Recipe Works: Vision-Language-Action Models are Natural Continual Learners with Reinforcement Learning
Jiaheng Hu, Jay Shim, Chen Tang et al. (+4 more)
Continual Reinforcement Learning (CRL) for
Vision-
Language-Action (VLA) models is a promising direction toward self-improving
embodied agents that can adapt in openended, evolving environments. However, conventional wisdom from continual learning suggests that naive Sequential Fine-Tuning (Seq. FT) leads to catastrophic forgetting, necessitating complex CRL strategies. In this work, we take a step back and conduct a systematic study of CRL for large pretrained VLAs across diverse lifelong RL
benchmarks. We find that, contrary to established belief, simple Seq. FT with low-rank adaptation (LoRA) is remarkably strong: it achieves high plasticity, exhibits little to no forgetting, and retains strong zero-shot generalization, frequently outperforming more sophisticated CRL methods. Through detailed analysis, we show that this robustness arises from a synergy between the large pretrained model, parameter-efficient adaptation, and on-policy RL. Together, these components reshape the stability-plasticity trade-off, making continual adaptation both stable and scalable. Our results position Sequential Fine-Tuning as a powerful method for continual RL with VLAs and provide new insights into lifelong learning in the large model era. Code is available at https://github.com/UT-Austin-RobIn/continual-vla-rl
Read the full abstract ▾
All authors Jiaheng Hu, Jay Shim, Chen Tang, Yoonchang Sung, Bo Liu, Peter Stone, Roberto Martin-Martin
Robotics
A Hybrid Neural-Assisted Unscented Kalman Filter for Unmanned Ground Vehicle Navigation
Gal Versano and Itzik Klein
Modern autonomous
navigation for unmanned ground vehicles relies on different
estimators to fuse inertial sensors and GNSS measurements. However, the constant noise covariance
matrices often struggle to account for dynamic real-world conditions. In this work we propose a hybrid estimation framework that bridges classical state estimation foundations with modern deep learning approaches. Instead of altering the fundamental unscented Kalman filter equations, a dedicated deep neural network is developed to predict the process and measurement noise uncertainty directly from raw inertial and GNSS measurements. We present a sim2real approach, with training performed only on simulative data. In this manner, we offer perfect ground truth data and relieves the burden of extensive data recordings. To evaluate our proposed approach and examine its generalization capabilities, we employed a 160-minutes test set from three
datasets each with different types of vehicles (off-road vehicle, passenger car, and mobile
robot), inertial sensors, road surface, and environmental conditions. We demonstrate across the three
datasets a position improvement of $12.7\%$ compared to the adaptive model-based approach. Thus, offering a scalable and a more robust solution for unmanned ground vehicles
navigation tasks.
Read the full abstract ▾
Robotics
Noise-Space Attribution and Control of Chunk-Boundary Artifact
Rui Wang
Action chunking is widely used in generative visuomotor policies, yet the recurring execution discontinuities at chunk boundaries still lack a mechanistic explanation. This paper treats chunk-boundary artifact as an analyzable mechanism variable. We first show that successful and failed episodes separate stably on artifact metrics. We then show that, in
stochastic action-chunked policies, fixing the observation context and changing only latent noise is sufficient to modulate artifact systematically. On the same
Diffusion Policy checkpoint, comparisons among DDPM, zero-variance DDPM, and DDIM further show that this local controllability depends on whether the information path from initial noise to action output remains intact. Finally, from controlled interventions at fixed local execution states, we find that artifact changes can carry through to final outcome, and that the preferred direction can reverse even within the same task: some contexts achieve higher success under lower artifact, whereas others achieve higher success under higher artifact. In a representative high-artifact-favoring key context selected by held-out matched-continuation validation, success rate increases from 0.033 to 0.717. These results show that chunk-boundary artifact is not a mere execution-side by-product, but a variable in noise space that can be attributed, controlled, and mechanistically linked to task outcome.
Read the full abstract ▾
Robotics
Learn Structure, Adapt on the Fly: Multi-Scale Residual Learning and Online Adaptation for Aerial Manipulators
Samaksh Ujjawal, Naveen Sudheer Nair, Shivansh Pratap Singh et al. (+3 more)
Autonomous Aerial Manipulators (AAMs) are inherently coupled, nonlinear systems that exhibit nonstationary and multiscale residual dynamics, particularly during manipulator reconfiguration and abrupt payload variations. Conventional analytical dynamic models rely on fixed parametric structures, while static data-driven model assume stationary dynamics and degrade under configuration changes and payload variations. Moreover, existing learning architectures do not explicitly factorize cross-variable coupling and multi-scale temporal effects, conflating instantaneous inertial dynamics with long-horizon regime
evolution. We propose a predictive-adaptive framework for real-time residual modeling and compensation in AAMs. The core of this framework is the Factorized Dynamics
Transformer (FDT), which treats physical variables as independent
tokens. This design enables explicit cross-variable
attention while structurally separating short-horizon inertial dependencies from long-horizon aerodynamic effects. To address deployment-time distribution shifts, a Latent Residual Adapter (LRA) performs rapid linear adaptation in the latent space via Recursive Least Squares, preserving the offline nonlinear representation without prohibitive computational overhead. The adapted residual forecast is directly integrated into a residual-compensated adaptive controller. Real-world experiments on an aerial manipulator subjected to unseen payloads demonstrate higher prediction fidelity, accelerated disturbance attenuation, and superior closed-loop tracking precision compared to state-of-the-art learning baselines, all while maintaining strict real-time feasibility.
Read the full abstract ▾
All authors Samaksh Ujjawal, Naveen Sudheer Nair, Shivansh Pratap Singh, Rishabh Dev Yadav, Wei Pan, Spandan Roy
Robotics
Diversity You Can Actually Measure: A Fast, Model-Free Diversity Metric for Robotics Datasets
Sreevardhan Sirigiri, Nathan Samuel de Lara, Christopher Agia et al. (+2 more)
Robotics
datasets for imitation learning typically consist of long-horizon trajectories of different lengths over states, actions, and high-dimensional observations (e.g., RGB
video), making it non-trivial to quantify diversity in a way that respects the underlying trajectory structure and geometry. We extend Shannon and von Neumann
entropy to this setting by defining signature transform-based
entropy on the Gram
matrix of a signature kernel over demonstrations, yielding
entropy and diversity metrics that operate directly on the demonstration
dataset. Building on these metrics, we study how
dataset diversity affects generalization performance in
robot imitation learning and propose a simple, model-free way to curate diverse demonstrations. We introduce FAKTUAL (FAst trajectory Kernel
enTropy cUration for imitation Learning), a data curation
algorithm that selects a subset of demonstrations maximizing
entropy given a subset-size budget. FAKTUAL is fully model-free, requires no access to the imitation policy or rollouts, and adds negligible overhead relative to policy training. We evaluate our approach on image and state-based RoboMimic and MetaWorld
benchmarks, as well as four real-world manipulation tasks. Across tasks and architectures, diversity-aware curation with FAKTUAL consistently improves downstream success rates over random selection, while being substantially more computationally efficient compared to recent
robot data curation methods. Our results suggest that the
entropy of demonstration
datasets is a practical tool for understanding and improving
dataset diversity in
robot imitation learning.
Read the full abstract ▾
All authors Sreevardhan Sirigiri, Nathan Samuel de Lara, Christopher Agia, Florian Shkurti, Fabio Ramos
Robotics · Cross-listed from cs.HC
From Pets to Robots: MojiKit as a Data-Informed Toolkit for Affective HRI Design
Liwen He, Pingting Chen, Ziheng Tang et al. (+4 more)
Designing affective behaviors for animal-inspired social
robots often relies on intuition and personal experience, leading to fragmented outcomes. To provide more systematic guidance, we first coded and analyzed human-pet interaction
videos, validated insights through literature and interviews, and created structured reference cards that map the design space of pet-inspired affective interactions. Building on this, we developed MojiKit, a toolkit combining reference cards, a zoomorphic
robot prototype (MomoBot), and a behavior control studio. We evaluated MojiKit in co-creation workshops with 18 participants, finding that MojiKit helped them design 35 affective interaction patterns beyond their own pet experiences, while the code-free studio lowered the technical barrier and enhanced creative agency. Our contributions include the data-informed structured resource for pet-inspired affective HRI design, an integrated toolkit that bridges reference materials with hands-on prototyping, and empirical evidence showing how MojiKit empowers users to systematically create richer, more diverse affective
robot behaviors.
Read the full abstract ▾
All authors Liwen He, Pingting Chen, Ziheng Tang, Yixiao Liu, Jihong Jeung, Teng Han, Xin Tong
Robotics
Unsupervised LiDAR-Based Multi-UAV Detection and Tracking Under Extreme Sparsity
Nivand Khosravi, Rodrigo Ventura and Meysam Basiri
Non-repetitive solid-state
LiDAR scanning leads to an extremely sparse measurement regime for detecting airborne
UAVs: a small quadrotor at 10-25 m typically produces only 1-2 returns per scan, which is far below the point densities assumed by most existing detection approaches and inadequate for robust multi-target data association. We introduce an unsupervised,
LiDAR-only pipeline that addresses both detection and tracking without the need for labeled training data. The detector integrates range-adaptive DBSCAN clustering with a three-stage temporal consistency check and is benchmarked on real-world air-to-air flight data under eight different parameter configurations. The best setup attains 0.891 precision, 0.804 recall, and 0.63 m RMSE, and a systematic minPts sweep verifies that most scans contain at most 1-2 target points, directly quantifying the sparsity regime. For multi-target tracking, we compare deterministic Hungarian assignment with joint probabilistic data association (JPDA), each coupled with Interacting Multiple Model filtering, in four simulated scenarios with increasing levels of ambiguity. JPDA cuts identity switches by 64% with negligible impact on MOTA, demonstrating that probabilistic association is advantageous when
UAV trajectories approach one another closely. A two-environment evaluation strategy, combining real-world detection with RTK-GPS ground truth and simulation-based tracking with identity-annotated ground truth, overcomes the limitations of GNSS-only evaluation at inter-
UAV distances below 2 m.
Read the full abstract ▾
Robotics · Cross-listed from cs.CV
SVLL: Staged Vision-Language Learning for Physically Grounded Embodied Task Planning
Yuyuan Yang, Junkun Hong, Hongrong Wang et al. (+11 more)
Embodied task planning demands
vision-
language models to generate action sequences that are both visually grounded and causally coherent over time. However, existing training paradigms face a critical trade-off: joint end-to-end training often leads to premature temporal binding, while standard reinforcement learning methods suffer from optimization instability. To bridge this gap, we present Staged
Vision-
Language Learning (SVLL), a unified three-stage framework for robust, physically-grounded
embodied planning. In the first two stages, SVLL decouples spatial grounding from temporal
reasoning, establishing robust visual dependency before introducing sequential action history. In the final stage, we identify a key limitation of standard Direct Preference Optimization (DPO), its purely relative nature – optimizing only the preference gap between winning and losing trajectories while neglecting absolute likelihood constraints on optimal path, often yields unsafe or hallucinated behaviors. To address this, we further introduce Bias-DPO, a novel
alignment objective that injects an inductive bias toward expert trajectories by explicitly maximizing likelihood on ground-truth actions while penalizing overconfident
hallucinations. By anchoring the policy to the expert
manifold and mitigating causal misalignment, SVLL, powered by Bias-DPO, ensures strict adherence to environmental affordances and effectively suppresses physically impossible shortcuts. Finally, extensive experiments on the interactive AI2-THOR
benchmark and real-world robotic deployments demonstrate that SVLL outperforms both state-of-the-art open-source (e.g., Qwen2.5-VL-7B) and closed-source models (e.g., GPT-4o, Gemini-2.0-flash) in task success rate, while significantly reducing physical constraint violations.
Read the full abstract ▾
All authors Yuyuan Yang, Junkun Hong, Hongrong Wang, Honghao Cai, Xunpeng Ren, Ge Wang, Mingcong Lei, Shenhao Yan, Jiahao Yang, Chengsi Yao, Xi Li, Yiming Zhao, Yatong Han, Jinke Ren
Robotics
RoboClaw: An Agentic Framework for Scalable Long-Horizon Robotic Tasks
Ruiying Li, Yunlang Zhou, YuYao Zhu et al. (+15 more)
Vision-
Language-Action (VLA) systems have shown strong potential for
language-driven robotic manipulation. However, scaling them to long-horizon tasks remains challenging. Existing pipelines typically separate data collection, policy learning, and deployment, resulting in heavy reliance on manual environment resets and brittle multi-policy execution. We present RoboClaw, an agentic robotics framework that unifies data collection, policy learning, and task execution under a single VLM-driven controller. At the policy level, RoboClaw introduces Entangled Action Pairs (EAP), which couple forward manipulation behaviors with inverse recovery actions to form self-resetting loops for autonomous data collection. This mechanism enables continuous on-policy data acquisition and iterative policy refinement with minimal human intervention. During deployment, the same
agent performs high-level
reasoning and dynamically orchestrates learned policy primitives to accomplish long-horizon tasks. By maintaining consistent contextual semantics across collection and execution, RoboClaw reduces mismatch between the two phases and improves multi-policy robustness. Experiments in real-world manipulation tasks demonstrate improved stability and scalability compared to conventional open-loop pipelines, while significantly reducing human effort throughout the
robot lifecycle, achieving a 25% improvement in success rate over baseline methods on long-horizon tasks and reducing human time investment by 53.7%.
Read the full abstract ▾
All authors Ruiying Li, Yunlang Zhou, YuYao Zhu, Kylin Chen, Jingyuan Wang, Sukai Wang, Kongtao Hu, Minhui Yu, Bowen Jiang, Zhan Su, Jiayao Ma, Xin He, Yongjian Shen, Yang Yang, Guanghui Ren, Maoqing Yao, Wenhao Wang, Yao Mu
Robotics · Cross-listed from cs.CV
MANSION: Multi-floor lANguage-to-3D Scene generatIOn for loNg-horizon tasks
Lirong Che, Shuo Wen, Shan Huang et al. (+5 more)
Real-world robotic tasks are long-horizon and often span multiple floors, demanding rich spatial
reasoning. However, existing
embodied benchmarks are largely confined to single-floor in-house environments, failing to reflect the complexity of real-world tasks. We introduce MANSION, the first
language-driven framework for generating building-scale, multi-floor 3D environments. Being aware of vertical structural constraints, MANSION generates realistic, navigable whole-building structures with diverse, human-friendly scenes, enabling the development and evaluation of cross-floor long-horizon tasks. Building on this framework, we release MansionWorld, a
dataset of over 1,000 diverse buildings ranging from hospitals to offices, alongside a Task-Semantic Scene Editing
Agent that customizes these environments using open-vocabulary commands to meet specific user needs. Benchmarking reveals that state-of-the-art
agents degrade sharply in our settings, establishing MANSION as a critical testbed for the next generation of spatial
reasoning and planning.
Read the full abstract ▾
All authors Lirong Che, Shuo Wen, Shan Huang, Chuang Wang, Yuzhe Yang, Gregory Dudek, Xueqian Wang, Jian Su
Robotics
MiNI-Q: A Miniature, Wire-Free Quadruped with Unbounded, Independently Actuated Leg Joints
Daniel Koh, Suraj Shah, Yufeng Wu et al. (+1 more)
Physical joint limits are common in legged
robots and can restrict workspace, constrain gait design, and increase the risk of hardware damage. This paper introduces MiNI-Q^2, a miniature, wire-free quadruped
robot with independently actuated, mechanically unbounded 2-DOF leg joints. We present the mechanical design, kinematic analysis, and experimental validation of the proposed
robot. The leg mechanism enables both oscillatory gaits and rotary locomotion while allowing the
robot to fold to a minimum height of 2.5 cm. Experimentally, MiNI-Q achieves speeds up to 0.46 m/s and demonstrates low-clearance crawling, stair climbing, inverted locomotion, jumping, and backflipping. The wire-free architecture extends our previous Q8bot design, improving assembly reliability at miniature scale. All mechanical and electrical design files are released open source to support reproducibility and further research.
Read the full abstract ▾
All authors Daniel Koh, Suraj Shah, Yufeng Wu, Dennis Hong
Robotics
SPARK: Skeleton-Parameter Aligned Retargeting on Humanoid Robots with Kinodynamic Trajectory Optimization
Hanwen Wang, Qiayuan Liao, Bike Zhang et al. (+3 more)
Human motion provides rich priors for training general-purpose
humanoid control policies, but raw demonstrations are often incompatible with a
robot's kinematics and dynamics, limiting their direct use. We present a two-stage pipeline for generating natural and dynamically feasible motion references from task-space human data. First, we convert human motion into a unified
robot description format (URDF)-based skeleton representation and calibrate it to the target
humanoid's dimensions. By aligning the underlying skeleton structure rather than heuristically modifying task-space targets, this step significantly reduces inverse kinematics error and tuning effort. Second, we refine the retargeted trajectories through progressive kinodynamic trajectory optimization (TO), solved in three stages: kinematic TO, inverse dynamics, and full kinodynamic TO, each warm-started from the previous solution. The final result yields dynamically consistent state trajectories and joint torque profiles, providing high-quality references for learning-based controllers. Together, skeleton calibration and kinodynamic TO enable the generation of natural, physically consistent motion references across diverse
humanoid platforms.
Read the full abstract ▾
All authors Hanwen Wang, Qiayuan Liao, Bike Zhang, Kunzhao Ren, Koushil Sreenath, Xiaobin Xiong
Robotics
NFPO: Stabilized Policy Optimization of Normalizing Flow for Robotic Policy Learning
Diyuan Shi, Yiqi Tang, Zifeng Zhuang et al. (+1 more)
Deep Reinforcement Learning (DRL) has experienced significant advancements in recent years and has been widely used in many fields. In DRL-based robotic policy learning, however, current de facto policy parameterization is still multivariate Gaussian (with diagonal covariance
matrix), which lacks the ability to model multi-modal distribution. In this work, we explore the adoption of a modern network architecture, i.e. Normalizing Flow (NF) as the policy parameterization for its ability of multi-modal modeling, closed form of log probability and low computation and
memory overhead. However, naively training NF in online Reinforcement Learning (RL) usually leads to training instability. We provide a detailed analysis for this phenomenon and successfully address it via simple but effective technique. With extensive experiments in multiple simulation environments, we show our method, NFPO could obtain robust and strong performance in widely used robotic learning tasks and successfully transfer into real-world
robots.
Read the full abstract ▾
All authors Diyuan Shi, Yiqi Tang, Zifeng Zhuang, Donglin Wang
Robotics
CoViLLM: An Adaptive Human-Robot Collaborative Assembly Framework Using Large Language Models
Jiabao Zhao, Jonghan Lim, Hongliang Li et al. (+1 more)
With increasing demand for mass customization, traditional manufacturing
robots that rely on rule-based operations lack the flexibility to accommodate customized or new product variants. Human-
Robot Collaboration has demonstrated potential to improve system adaptability by leveraging human versatility and decision-making capabilities. However, existing Human-
Robot Collaborative frameworks typically depend on predefined perception-manipulation pipelines, limiting their ability to autonomously generate task plans for new product assembly. In this work, we propose CoViLLM, an adaptive human-
robot collaborative assembly framework that supports the assembly of customized and previously unseen products. CoViLLM combines depth-camera-based localization for object position estimation, human operator classification for identifying new components, and a Large
Language Model for assembly task planning based on natural
language instructions. The framework is validated on the NIST Assembly Task Board for known, customized, and new product cases. Experimental results show that the proposed framework enables flexible collaborative assembly by extending Human-
Robot Collaboration beyond predefined product and task settings.
Read the full abstract ▾
All authors Jiabao Zhao, Jonghan Lim, Hongliang Li, Ilya Kovalenko