Robotics · Cross-listed from cs.AI
Solution Space Path Planning: A Real-Time Human-Centered Path Planning Algorithm for En-Route Air Traffic Control
Yiyuan Zou, Wenying Lyu and Clark Borst
As technology advances, various
algorithms have been proposed for air traffic management, yet their operational adoption in tactical control remains limited. This gap motivates a human-centered design emphasizing algorithmic interpretability, controller-relevant operational constraints, and real-time computation. Inspired by the interpretability and flexibility of solution-space displays, as well as by the decision logic controllers naturally apply when enforcing operational constraints, this study extends the solution-space concept to path planning and develops a fast conflict-free path-planning
algorithm for en-route Air Traffic Control (ATC), termed Solution Space Path Planning (SSPP). The
algorithm integrates three intent-based conflict detection methods—distance-based, time-interval-based, and zone-based—within the solution-space framework to identify conflict-free paths in computationally efficient ways. SSPP is developed using both vertex-based and edge-based search nodes, resulting in two variants—SSPPV and SSPPE, respectively. Empirical results show that SSPPV paired with zone-based conflict detection performs best, computing paths in 3.69 ms on average in the Dutch Delta sector using a 5 nmi grid. SSPPV remains approximately 3.77 times faster than SSPPE while offering competitive effectiveness, making it suitable for time-critical operations and interactive 'what-if' probing in real time. An extension to SSPPV and SSPPE further examines the trade-off between delay minimization and separation requirements, demonstrating the flexibility of SSPP in revising optimization objectives. This study not only proposes a novel path-planning
algorithm but also shows how such
algorithms can be designed to align with human use and operational requirements, supporting their integration into future ATC systems.
Read the full abstract ▾
Robotics
DVG-WM: Disentangled Video Generation Enables Efficient Embodied World Model for Robotic Manipulation
Ziyu Shan, Zhenyu Wu, Xiaofeng Wang et al. (+2 more)
Video-based
embodied world models provide an appealing substrate for robotic manipulation by predicting future states, yet current approaches remain limited by a fundamental
entanglement: accurately modeling dynamics typically requires low-level temporal
reasoning, while producing high-resolution frames demands expansive visual synthesis according to high-level semantics. This
entanglement results in slow inference speed for iterative planning or too coarse predictions to retain contact-rich details. To solve this dilemma, we present Disentangled
Video Generation World Model (DVG-WM), an efficient framework that explicitly decomposes world modeling into dynamics learning and visual synthesis. Conditioned on an initial observation and a
language instruction, our model first generates a plausible sequence of intermediate visual states to preview the physical interaction and refines them to obtain high-fidelity
videos. Furthermore, an efficient cascading mechanism is proposed, where DVG-WM uses flow matching to directly map the dynamics to
video latents, and introduces a latent degradation mechanism to regenerate contact-rich details. Experiments on LIBERO and real-world platforms demonstrate improved
video quality with up to 3.97 times acceleration, validating that disentangled
video generation can be an efficient
embodied world model for robotic manipulation.
Read the full abstract ▾
All authors Ziyu Shan, Zhenyu Wu, Xiaofeng Wang, Zheng Zhu, Ziwei Wang
Robotics
Freeform Preference Learning for Robotic Manipulation
Marcel Torne, Anubha Mahajan, Abhijnya Bhat et al. (+1 more)
Reward design remains a central bottleneck for autonomous
robot policy improvement, especially in long-horizon manipulation tasks where sparse success labels provide too little signal and binary preferences collapse many competing notions of quality into one ambiguous signal. We introduce Freeform Preference Learning (FPL), a method for learning
robot policies from freeform human preferences. Rather than asking annotators which of two trajectories is better overall, FPL lets them define natural-
language preference axes, such as speed, safety, quality of placement, or carefulness, and provide pairwise preferences along each axis. These annotations are used to learn a
language-conditioned reward model that maps a trajectory and preference label to an axis-specific reward. We use this model to train a reward-conditioned policy that optimizes across the multiple human-specified dimensions. Across four real-world and two simulated long-horizon manipulation tasks, FPL improves over sparse-reward and binary-preference methods by 38 percentage points. Beyond improved performance, FPL learns dense progress signals without explicit subtask
segmentation, shows compositionality of behavior not present in the data, and allows users to steer the policy towards different behaviors at test time without retraining. Blog post with
videos available at https://freeform-pl.github.io/fpl.website/
Read the full abstract ▾
All authors Marcel Torne, Anubha Mahajan, Abhijnya Bhat, Chelsea Finn
Robotics
Human-as-Humanoid: Enabling Zero-Shot Humanoid Learning from Ego-Exo Human Videos with Human-Aligned Embodiments
Xiaopeng Lin, Ruoqi Yang, Shijie Lian et al. (+14 more)
Vision-
language-action (VLA) models across
robot embodiments require high-quality observation–action supervision to learn deployable action distributions, yet scaling such
robot data remains difficult, especially for high-DoF
humanoids. Teleoperation provides controller-aligned supervision, while human
egocentric videos capture diverse bimanual manipulation but do not directly provide executable
robot actions. We introduce Human-as-
Humanoid, a human-to-
humanoid supervision framework that enables near-real-time human-centric action generation, making human demonstrations usable for high-DoF
humanoid VLA training by jointly aligning the
robot embodiment, the sensing setup, and the action-label interface. Built on PrimeU, a human-aligned 60-DoF upper-body
humanoid, Human-as-
Humanoid uses synchronized ego-exo
videos to pair deployment-aligned
egocentric observations with exocentric motion recovery, retargets the recovered human motion through staged Inverse Kinematics (IK) into controller-aligned 60-DoF action chunks, and trains the VLA model with Forward Kinematics (FK)-aware supervision to preserve wrist and fingertip task-space geometry. This converts large-scale human demonstrations from visual observations into executable observation–action supervision for the target
humanoid. Experiments validate the conversion chain at the motion-recovery,
robot-action-space, and real-
robot deployment levels. Human-as-
Humanoid yields a 4.8–7.2x raw demonstration-throughput gain over
humanoid teleoperation in our data-collection analysis, and on several downstream tasks, policies post-trained only with the converted human labels generalize to real-
robot deployment without target-task
robot demonstrations. The official project website is available at https://zgc-embodyai.github.io/Human-as-
Humanoid.
Read the full abstract ▾
All authors Xiaopeng Lin, Ruoqi Yang, Shijie Lian, Zhaolong Shen, Bin Yu, Changti Wu, Haibao Liu, Yuxiang Zhang, Hong Li, Qiyuan Su, Haochen Liu, Xuguo He, Yukun Shi, Cong Huang, Zhirui Zhang, Bojun Cheng, Kai Chen
Robotics
OopsieVerse: A Safety Benchmark with Damage-Aware Simulation for Robot Manipulation
Arnav Balaji, Arpit Bahety, Sriniket Ambatipudi et al. (+3 more)
While robotic manipulation capabilities have advanced rapidly, physical safety remains a major barrier to deploying household
robots: task success is insufficient if the
robot damages itself or its surroundings. Simulation offers a harm-free alternative to costly and dangerous real-world training and evaluation, yet existing simulators lack general mechanisms to detect, quantify, and represent damage. To address this gap, we introduce OOPSIEVERSE, a unified simulation framework and
benchmark for damage-aware household manipulation. OOPSIEVERSE provides damage as an explicit, physically-grounded, and taskagnostic signal by converting sources such as contact forces, temperature changes, and
liquid interactions into corresponding mechanical, thermal or fluid damage. OOPSIEVERSE comprises two core elements: (1) DAMAGESIM, a simulator-agnostic framework for detecting and quantifying damage during
navigation and manipulation, and (2) a suite of household tasks designed to evaluate common damage modes and distinguish between task completion and safe execution. We demonstrate the generality of our framework by instantiating DAMAGESIM in two simulators with different physics backends, OmniGibson (Nvidia Omniverse) and RoboCasa (MuJoCo). We further showcase the utility of OOPSIEVERSE across multiple use cases, including (1) guiding safer demonstration collection via real-time damage feedback, (2) learning safer manipulation policies through damage-conditioned imitation learning and reinforcement learning, (3) benchmarking the safety of state-of-the-art
Vision Language Action policies, and (4) improving real-world safety of sim-to-real transferred policies. Together, our results highlight the potential of OOPSIEVERSE as an open-source foundation for systematic, scalable research on safe
robot manipulation. For code and more information, please refer to https://robin-lab.cs.utexas.edu/oopsieverse/
Read the full abstract ▾
All authors Arnav Balaji, Arpit Bahety, Sriniket Ambatipudi, Daniel Lam, Junhong Xu, Roberto Martín-Martín
Robotics
Adapting Generalist Robot Policies with Semantic Reinforcement Learning
Jagdeep Singh Bhatia, Andrew Wagenmaker, William Chen et al. (+1 more)
Generalist
robot policies learn a diverse repertoire of behaviors from large-scale pretraining. In principle, this makes them excellent priors for downstream adaptation via reinforcement learning (RL). In practice, however, standard RL methods leveraging this prior optimize directly over
robot actions, requiring the base policy's action distribution to be close to that of a performant policy from the start. This assumption breaks down for complex or long-horizon tasks that fall outside the pretraining distribution. Our key insight is that, for sufficiently expressive generalist policies,
language prompts are an effective alternative space for learning to solve such tasks: modulating
language inputs elicits skills already within the policy's repertoire, which can be composed to solve tasks beyond its zero-shot capabilities. We propose Semantic Action Reinforcement Learning (SARL), which learns to optimize this
prompt space through online interaction, treating the generalist policy as a controllable skill prior. Importantly, leveraging pretrained skills rather than learning new ones from scratch yields structured, semantically meaningful exploration and highly efficient online improvement, and learning to modulate
prompts through experience grounds them in induced real-world behaviors for robust task-solving. Across real-world settings and simulated
benchmarks, we show SARL unlocks fundamentally new capabilities – adapting VLA behavior to solve complex, long-horizon tasks – and significantly outperforms existing approaches for improving
robot behavior in deployment.
Read the full abstract ▾
All authors Jagdeep Singh Bhatia, Andrew Wagenmaker, William Chen, Sergey Levine
Robotics
RRT-Rope: A deterministic shortening approach for fast near-optimal path planning in large-scale uncluttered 3D environments
Louis Petit and Alexis Lussier Desbiens
Many path planning
algorithms have been introduced so far, but most are costly, in path cost and in processing time, in large-scale uncluttered 3D environments such as underground mining stopes explored by an unmanned aerial vehicle (
UAV). Rapidly-exploring Random Tree (RRT)
algorithms are popular because of their probabilistic completeness and rapidity in finding a feasible path in single-query problems. Many of the
algorithms (e.g. Informed RRT*, RRT#) developed to improve RRT need considerable time to converge in large environments. Shortcutting an RRT is an old idea that has been proven to outperform RRT variants. This paper introduces a new method, RRT-Rope, that aims at finding a near-optimal solution in a drastically shorter amount of time. The proposed approach benefits from fast computation of a feasible path with an altered version of RRT-connect, and post-processes it quickly with a deterministic shortcutting technique, taking advantage of intermediate nodes added to each branch of the tree. This paper presents simulations and statistics carried out to show the efficiency of RRT-Rope, which gives better results in terms of path cost and computation time than other popular RRT variations and shortening techniques in all our simulation environments, and is up to 70% faster than the next best
algorithm in a representative stope.
Read the full abstract ▾
Robotics
LeCropFollow: Latent Space Planning for Navigation in Unstructured Crop Fields
Felipe Tommaselli, Francisco Affonso, Arthur Pompeu et al. (+4 more)
Unstructured navigational features, such as irregular planting or discontinuities, remain the primary failure mode for under-canopy agricultural
robots. Existing
geometric approaches often fail in these scenarios because they compress high-dimensional visual data into deterministic spatial references, effectively discarding the uncertainty and semantic context required to navigate ambiguous terrain. To address this, we present LeCropFollow, a visual
navigation framework that bypasses explicit
geometric modeling in favor of a learned latent representation. By integrating a self-supervised semantic heatmap extractor with TD-MPC2, a Model-Based Reinforcement Learning (MBRL) planner, our system optimizes trajectories directly within a latent
manifold. The framework operates over the uncompressed heatmap signal, preserving the semantic context that
geometric reductions discard. We demonstrate that this representational shift enables zero-shot transfer from simplified simulation to the physical world without fine-tuning. Extensive field experiments in late-stage corn fields show that LeCropFollow matches state-of-the-art baselines in unstructured rows but significantly outperforms them in plantation gaps, achieving a 2.4x reduction in semantic failures compared to keypoint-based methods. These results suggest that latent planning offers a robust alternative to
geometric estimation for operations in heterogeneous agricultural environments. Code, models, and data available: https://felipe-tommaselli.github.io/lecropfollow .
Read the full abstract ▾
All authors Felipe Tommaselli, Francisco Affonso, Arthur Pompeu, Gianluca Capezzuto, Arun Narenthiran Sivakumar, Girish Chowdhary, Marcelo Becker
Robotics
MVP-Nav: Multi-layer Value Map Planner Navigator
Wenyuan Xie, Shaokai Wu, Yijin Zhou et al. (+7 more)
Zero-shot Object Goal
Navigation (ZSON) with RGB-only perception poses a fundamental challenge for
embodied agents, as the absence of explicit depth information introduces severe physical uncertainty and semantic-physical misalignment. Existing approaches either rely on high-level semantic
reasoning without
geometric grounding or learn end-to-end policies that lack explicit physical constraints, often resulting in semantically plausible but physically unsafe behaviors. In this paper, we propose MVP-Nav, a physical-aware RGB-only
navigation framework that aligns perception, planning, and control with the real 3D world. MVP-Nav reconstructs explicit physical occupancy from
monocular observations by leveraging 3D foundation models to project 2D semantic instances into 3D oriented bounding boxes, forming a global spatial semantic representation. To unify high-level semantic
reasoning and low-level physical constraints, we introduce a Multi-layer Value Map (MVM) that integrates semantic priorities and reconstructed geometry into a shared cost space, enabling physically grounded
geometric planning. Extensive experiments on zero-shot object
navigation benchmarks demonstrate that MVP-Nav significantly outperforms existing depth-free methods, achieving state-of-the-art performance and validating that structured physical priors can effectively compensate for the absence of active depth sensors.
Read the full abstract ▾
All authors Wenyuan Xie, Shaokai Wu, Yijin Zhou, Yanbiao Ji, Guodong Zhang, Bayram Bayramli, Qiuchang Li, Xunchu Zhou, Yue Ding, Hongtao Lu
Robotics
Learning Locomotion on Discrete Terrain via Minimal Proximity Sensing
Jiale Fan, Connor Flynn, Tianao Xu et al. (+4 more)
Learning-based control has revolutionized dynamic locomotion, yet navigating unstructured terrain remains limited by a
robot's incomplete awareness of imminent ground contact. While global perception systems such as
LiDARs and depth cameras provide environmental context, they are frequently plagued by latencies, occlusions, and the high computational cost of dense
geometric reconstruction. On the other hand, proprioceptive feedback is purely reactive, initiating corrections only after impact has occurred. This work explores
embedding a minimal suite of low-cost, high-frequency infrared proximity sensors directly into the feet of a quadrupedal
robot. These sensors provide "pre-contact" feedback that is robust to self-occlusions and significantly less computationally demanding than conventional
vision-based pipelines. By integrating these localized signals into a reinforcement learning framework, we enable the
robot to anticipate terrain discontinuities such as gaps and stepping stones that are problematic for traditional perception stacks due to occlusions or state estimation drift. We demonstrate that such sparse, near-field sensing can be reliably modeled in simulation and transferred to the real world with high fidelity. Experimental results show that local proximity sensing substantially improves traversal robustness over discrete terrain and offers a low-power, low-latency alternative or complement to complex global perception suites in unpredictable environments. For more information about results and methods, please see the project website: https://sites.google.com/view/foot-tof/home.
Read the full abstract ▾
All authors Jiale Fan, Connor Flynn, Tianao Xu, Junzhe He, Andrei Cramariuc, Marco Hutter, Robert Baines
Robotics
CoDex: Learning Compositional Dexterous Functional Manipulation without Demonstrations
Bowen Jiang, William Painter Reger and Roberto Martin-Martin
In this work, we study Compositional Dexterous Functional Object Manipulation (CD-FOM): tasks such as aiming and actuating a spray bottle on a plant or a glue gun on wood, which require both actuating an object's internal mechanism and controlling its pose to apply the object's function to the environment. These tasks pose significant challenges for
robots due to the demanding integration of semantic understanding of the object's function, actuation mode, and application area with intricate physical dexterity to manage grasp stability, movement trajectory, and actuation. We introduce CoDex, a zero-demonstration framework that autonomously discovers CD-FOM manipulation strategies. CoDex uses
vision-
language models (VLMs) to infer semantic constraints from the task and scene. These constraints guide analytic constrained optimization to generate a short list of functional grasp candidates that can be efficiently refined with reinforcement learning to generate full grasp-move-actuate policies transferable from simulation to the real world. We evaluate CoDex on a 7-DoF
robot arm with a 16-DoF multi-fingered hand across six CD-FOM tasks involving previously unseen objects with internal mechanisms, including spray bottles, hot glue guns, air dusters, flashlights, and pepper grinders, and their application to unseen target objects, showcasing its ability to autonomously discover and execute complex, physically viable dexterous behaviors without human demonstrations. More information at https://robin-lab.cs.utexas.edu/CoDex/.
Read the full abstract ▾
Robotics
Improving path-tracking performance of an articulated tractor-trailer system using a non-linear kinematic model
Marina Murillo, Guido Sanchez, Nestor Deniz et al. (+2 more)
This paper presents a novel non-linear mathematical model of an articulated tractor-trailer system that can be used, in combination with receding horizon techniques, to improve the performance of path tracking tasks of articulated systems. Due to its dual steering mechanisms, this type of vehicle can be very useful in precision agriculture, particularly for seeding, spraying and harvesting in small fields. The articulated tractor-trailer system model was embedded within a non-linear model predictive controller and the trailer position was monitored. When the kinematic of the trailer was considered, the deviation of trailer's position was reduced substantially alongside not only straight paths but also in headland turns. Using the proposed mathematical model, we were able to control the trailer's position itself rather than the tractor's position. The
Robot Operating System (ROS) framework and Gazebo simulator were used to perform realistic simulations examples.
Read the full abstract ▾
All authors Marina Murillo, Guido Sanchez, Nestor Deniz, Lucas Genzelis, Leonardo Giovanini
Robotics
Z-1: Efficient Reinforcement Learning for Vision-Language-Action Models
Lang Cao, Renhong Chen, Luyi Li et al. (+3 more)
Vision-
Language-Action (VLA) models offer a promising framework for robotic manipulation by connecting
language instructions, visual observations, and continuous control. However, most existing policies remain limited by behavior cloning or supervised fine-tuning (SFT) from fixed demonstrations, which provides limited opportunity to improve from the policy's own failures. In this paper, we present Z-1, a reinforcement learning (RL) post-training framework for flow-based VLA models. Built on top of $\pi_{0.5}$, Z-1 uses only publicly released RoboCasa demonstrations for SFT and then applies a task-wise Group Relative Policy Optimization (GRPO) strategy across $24$ standard RoboCasa tasks. To improve the efficiency and stability of online optimization, Z-1 combines shared-prefix rollout construction, tree-structured trajectory branching, completion-aware reward calibration, and selective joint training of VLM and Action Expert. Across all $24$ RoboCasa tasks, Z-1 achieves an average success rate of $80.6\%$, improving over its SFT initialization by $13.2\%$ points and outperforms the published sota models. These results show that systematic GRPO post-training can substantially improve flow-based VLA policies without additional private demonstrations.
Read the full abstract ▾
All authors Lang Cao, Renhong Chen, Luyi Li, Peng Wang, Mofan Peng, Yitong Li
Robotics
Bridging Local Observation and Global Simulation in Closed-Loop Traffic Modeling
Ziyan Wang, Tan Xiang, Peng Chen et al. (+1 more)
A local-to-global context mismatch arises when autoregressive traffic simulators trained on ego-centric driving logs are deployed in globally observable closed-loop environments. In such logs, the ego vehicle has rich local observations, while surrounding
agents are only partially observed due to perception limits and occlusions. As a result, simulators may learn incomplete context–action mappings that remain hidden in log-based training but emerge during closed-loop rollouts, leading to unrealistic behaviors such as abnormal stops, unsafe interactions, and rule violations. We propose CRAFT, a Contextual pReference
Alignment Framework for Traffic Simulation, to mitigate this mismatch via self-supervised failure discovery and preference-guided test-time
alignment. CRAFT treats the base simulator as a globally observable sandbox, generating diverse what-if rollouts from logged initial states to expose context-induced failures. These failures are grounded with human-aligned driving priors and converted into preference supervision for training a Contextual Preference Evaluator (CPE). At inference time, CPE acts as a plug-in
alignment module that scores candidate actions under complete scene context and reweights autoregressive decoding toward globally coherent behaviors. CRAFT mitigates this local-to-global contextual bias, reducing collisions by 31.2% and traffic violations by 33.2% without retraining the base simulator.
Read the full abstract ▾
All authors Ziyan Wang, Tan Xiang, Peng Chen, Xintao Yan
Robotics
RoboTacDex: A Dexterous Visual-Tactile-Action Dataset for Humanoid Manipulation
Xinyi Wang, Donghan Li, Zi'Ang Chen et al. (+5 more)
In the field of
robot learning, large-scale and diverse demonstration trajectories provide the fundamental basis for enhancing robotic manipulation ability. We introduce RoboTacDex, a large, multi-modal, and diverse
dataset of dexterous manipulation behaviors performed with a
humanoid robot. Built on the publicly accessible
humanoid robot Unitree G1, RoboTacDex consists of 6k trajectories covering 19 tasks, 23 skills, and interactions with 22 objects. RoboTacDex provides comprehensive records including multi-view RGB and depth information, tactile feedback, and detailed semantic annotations. Furthermore, the
dataset features a variety of relatively challenging tasks that can only be completed by dual arms and dexterous hands, aiming to mimic human-like operational logic and simulate real-world manipulation complexity. To ensure data collection quality, we develop an improved multi-camera synchronization system to enable millisecond data synchronization and recording of modalities. In our experiments, we evaluate three representative imitation learning models on our
dataset, analyzing their performance as well as their respective strengths and limitations across different task categories. Successful trial results and a moderate level of generalization capabilities across a suite of tasks indicate the effectiveness and diversity of the collected
dataset. Our
dataset will be open-sourced soon.
Read the full abstract ▾
All authors Xinyi Wang, Donghan Li, Zi'Ang Chen, Chong Yu, Chen Xin, Peng Ye, Yingkai Sun, Tao Chen
Robotics · Cross-listed from cs.CV
PriorEye: Geospatial Visual Priors for End-to-End Autonomous Driving
Kyuhwan Yeon, Benjamin Ramtoula and Daniele De Martini
Most end-to-end autonomous driving methods rely solely on instantaneous sensor observations, limiting them to reactive behavior without the anticipatory foresight human drivers employ through prior experience. We introduce geospatial visual priors, street-level visual context anchored to the intended driving route, providing visual-spatial foresight independent of real-time sensors. We propose a
memory augmentation module featuring a dual-
memory architecture and an adaptive
memory gate, which can be easily integrated into existing end-to-end approaches. This design pairs a contextual
memory for retrieved priors with a persistent fallback
memory, and dynamically regulates the influence of
memories based on current state compatibility. Evaluated on the NAVSIM-v2
benchmark, our approach consistently improves performance across diverse end-to-end baselines. Furthermore, because these priors are independent of onboard sensors, our method inherently improves robustness against sensor corruption, while the dual-
memory design ensures safe fallback when the retrieved priors themselves become unreliable. Our project page is available at https://ori-mrg.github.io/PriorEye.
Read the full abstract ▾
Robotics
Reinforcement Learning-Based Control for an Inline Skating Humanoid Robot
Ethan Marot, Thomas Bi, Clemens Schwarke et al. (+3 more)
As
humanoid robots become increasingly dynamic, coupling them with reinforcement learning offers a promising approach to solving the complex, underactuated mechanics of passive inline skating. Equipping a
humanoid robot with passive inline skating wheels presents an opportunity to combine the versatile agility of
humanoids with the high-speed, energy-efficient locomotion strategies utilized by human skaters. In this paper, we train and deploy a reinforcement learning control policy that enables novel locomotion strategies for a
humanoid robot modified to equip consumer inline skates instead of conventional feet. Unlike previous work limited to quadrupedal
robots or actively driven wheels, our system allows for precise 6-DoF control of the skates to execute dynamic, edge-driven propulsion strategies. Our skating strategies emerge entirely from our reward structure, without reliance on human motion data, imitation learning, or kinematic priors. We overcome the inherent instability of passive wheels and simulation contact artifacts by utilizing different
geometric wheel models (spherical and ellipsoidal) during training and validation, along with a custom success-based command curriculum and a specialized rolling reward. Consequently, our policy demonstrates up to a 50% reduction in Cost of Transport (CoT) compared to standard walking gaits. The resulting policy successfully transfers zero-shot to the physical Booster T1 hardware. Real-world deployments demonstrate dynamic balance, the ability to reject active physical perturbations, and agile locomotion strategies capable of turning at speed. A
video of our results can be found at https://www.youtube.com/watch?v=-_APcOS7uFo.
Read the full abstract ▾
All authors Ethan Marot, Thomas Bi, Clemens Schwarke, Victor Klemm, Marco Hutter, Raffaello D'Andrea
Robotics
Autonomous UAV Navigation for Individual Wildlife Re-Identification
Claire Sun, Tanya Berger-Wolf and Jenna Kline
Reliable individual re-identification (re-ID) of wildlife is essential for population monitoring, behavioral tracking, and conservation policy evaluation, yet large-scale data collection remains labor-intensive, relying on manual efforts by ecologists or citizen scientists. We propose an autonomous drone
navigation system that actively optimizes image capture for downstream re-ID, moving beyond passive aerial sensing. The system combines YOLOv11 object detection with a DINOv2-based pose classifier to guide real-time flight decisions: detecting animals, orienting to expose the lateral flank (the surface of interest for pattern-based re-ID), and approaching until the subject meets a minimum bounding-box threshold. Unlike prior drone systems that optimize for group-level behavioral
video, ours targets the specific image-quality requirements of individual-identification models. We demonstrate feasibility through a case study on zebra using footage collected in Kenya, and show the approach generalizes to other species with diagnostic surface patterns, including giraffes, tigers, and elephants. Our work establishes a framework for task-aware
embodied AI for ecological data collection, in which downstream re-ID requirements drive real-time perception and control.
Read the full abstract ▾
Robotics
UniTacVLA: Unified Tactile Understanding and Prediction in Vision Language Action Models
Xidong Zhang, Yichi Zhang, Jiaxin Shi et al. (+5 more)
Vision-
language-action (VLA) models have achieved strong performance in many robotic manipulation tasks, yet remain limited in contact-rich dexterous manipulation. To overcome this limitation, recent
vision-tactile-
language-action (VTLA) methods incorporate tactile sensing into VLA models to provide direct contact information. However, they typically treat tactile signals as passive auxiliary inputs, making it difficult to model tactile semantics and future physical interactions. To this end, we propose a unified tactile learning framework for contact-rich manipulation that models tactile signals as dynamic interaction cues for both contact understanding and prediction. Specifically, we construct a unified tactile latent space and jointly model current tactile states and future contact changes through tactile chain-of-thought
reasoning and coarse-to-fine future tactile prediction, thereby forming a state-aware and dynamics-aware tactile prior. Based on this prior, we introduce a tactile-action mixed controller that combines real-time and predicted tactile feedback to refine low-frequency action chunks with high-frequency corrections. Real-world experiments on four categories of contact-rich tasks, including adjustment, insertion, wiping, and assembly, under both clean and externally perturbed settings, show that our method improves success rate, manipulation accuracy, and contact robustness over existing methods, demonstrating its effectiveness in dexterous physical interaction.
Read the full abstract ▾
All authors Xidong Zhang, Yichi Zhang, Jiaxin Shi, Fucai Zhu, Siyu Zhu, Michael Yu Wang, Xiaojun Wu, Weihao Yuan
Robotics
RCT: A Robot-Collected Touch-Vision-Language Dataset for Tactile Generalization
Jingbo He, Michael Färber and Roberto Calandra
For
robots manipulating open-world objects, tactile representations must generalize to unseen materials. We introduce RCT (Robotic Contact Tactile), a
robot-collected touch-
vision-
language dataset with 29,279 tactile frames from full
robot presses on 122 industrial reference materials in 7 categories, recorded with three DIGIT sensors at multiple contact positions. RCT preserves each press as a contact sequence, enabling held-out evaluation across materials, categories, sensors, contact positions, and contact sequences. Frames from one press are strongly correlated: frame-random splits can place near-duplicate observations of the same physical interaction in both training and test. With the encoder held fixed, removing contact-sequence overlap reduces tactile-to-text Recall@1 by 17.7 percentage points. When materials are additionally held out at training time, performance drops sharply, leaving held-out-material Recall@1 at 25.1 +/- 6.1% averaged over three held-out draws. The public TVL/HCT split shows the same structure: every test contact sequence appears in training, and raw-pixel nearest neighbors recover the correct sequence in 98.3% of cases. Uniformly sampling a press improves contrastive training, and RCT-trained
embeddings improve category probes on unseen materials. RCT makes contact-sequence-aware, held-out-material evaluation reproducible and exposes novel-material generalization as a central challenge for robotic tactile perception. The RCT
dataset is open-sourced at https://faerber-lab.github.io/RCT/
Read the full abstract ▾
Robotics
FastDSAC: Enhancing Policy Plasticity via Constrained Exploration for Scalable Humanoid Locomotion
Guanchen Lu, Yajuan Dun, Yi Zhou et al. (+4 more)
Scalable reinforcement learning has popularized high-throughput sampling architectures, which significantly compresses the training time for off-policy methods in robotic locomotion. However, the rapid increase of data volume and update frequency undermines the stability of value-based methods and diminishes the plasticity of policy networks. To address these challenges, this work presents FastDSAC, a fast and high-performance variant of the Distributional Actor-Critic
algorithm designed for parallel sampling scenarios. Specifically, we introduce a truncated Gaussian distribution to approximate the learned policy, which effectively excludes out-of-distribution actions that strain target value estimation while keeping necessary stochasticity for exploration. The proposed action constraint functions as an implicit regularization, which counteracts the plasticity loss typically caused by aggressive gradient updates. This preservation of network adaptability enhances sample efficiency, particularly in scenarios with a high update-to-data ratio, and accelerates the early training process. In contrast to prior fast reinforcement learning approaches that rely on discrete value distributions, our method utilizes a continuous Gaussian representation equipped with adaptive variance regulation, which improves value estimation accuracy by sampling confident and informative transitions. Extensive experiments on MuJoCo Playground and HumanoidBench demonstrate that FastDSAC not only stabilizes the overall training process but also achieves superior asymptotic performance and faster convergence compared to state-of-the-art baselines.
Read the full abstract ▾
All authors Guanchen Lu, Yajuan Dun, Yi Zhou, Letian Tao, Jingliang Duan, Jie Li, Guofa Li
Robotics
HABIT: Human-Aware Behavior and Interaction Training Dataset for Robot Manipulation
Jaehwi Song, Suchae Jeong, Byeongguk Jeon et al. (+4 more)
Large-scale demonstration
datasets have been central to recent progress in general-purpose
robot policies. However, existing
datasets are collected in human-absent settings, and policies trained on such data may perform tasks competently in isolation but fail to exhibit human-aware behaviors. To address this gap, we introduce HABIT, a large-scale
robot demonstration
dataset for human-present environments. We organize tasks into three roles capturing distinct modes of human-
robot interaction: Collaborator, where human and
robot jointly accomplish a task; Coworker, where they pursue separate tasks in a shared space; and Supervisor, where the human directs the
robot. The
dataset comprises over 10K episodes and over 160 hours across 60 tasks. Our experiments show that training on human-present data elicits human-aware behaviors that
robot-only data fails to produce: spatiotemporal synchronization in Collaborator tasks, yielding in Coworker tasks, and gesture grounding in Supervisor tasks. Moreover, training on HABIT enables rapid adaptation to new human-
robot interaction tasks. By introducing human presence as a new axis of
dataset diversity, HABIT extends
robot policies to environments shared with humans.
Read the full abstract ▾
All authors Jaehwi Song, Suchae Jeong, Byeongguk Jeon, Sungdong Kim, Minjoon Seo, Hyungmok Son, Kimin Lee
Robotics
DynFly: Dynamic-Aware Continuous Trajectory Generation for UAV Vision-Language Navigation in Urban Environments
Wen Jiang, Hanfang Liang, Li Wang et al. (+9 more)
Recent advances in
multimodal large models have significantly improved
UAV vision-
language navigation (
UAV-VLN) by enhancing high-level perception and
reasoning. However, existing methods mainly focus on predicting discrete actions, local targets, or sparse waypoints, while the continuous transition from
navigation intent to executable
UAV motion remains weakly modeled. This motion-interface gap limits the continuity, stability, and executability of generated
UAV trajectories. To address this gap, we propose DynFly, a dynamic-aware continuous trajectory generation framework that bridges high-level
navigation reasoning and executable
UAV motion. DynFly bridges high-level
navigation intent and continuous
UAV motion through a lightweight trajectory generation layer. Specifically, it represents expert trajectories in B-spline control-point space and employs a Spline-DiT generator to learn conditional trajectory generation via flow matching. Furthermore, we introduce
UAV-oriented dynamic-aware supervision over position, finite-difference velocity, finite-difference acceleration, heading consistency, and local target
alignment, enabling the generated trajectories to better satisfy
UAV motion characteristics. And our trajectory generation framework can also be integrated with an existing
UAV-VLN framework while preserving its original visual-
language reasoning pipeline. Extensive experiments on the OpenUAV
UAV-VLN
benchmark show that DynFly improves both
navigation performance and trajectory quality. On the Test Unseen Full split, DynFly improves the strongest baseline by 4.69 NDTW, 2.40 SDTW, 2.14 SR points and 4.87 OSR points, while reducing NE by 4.51 m.
Read the full abstract ▾
All authors Wen Jiang, Hanfang Liang, Li Wang, Kangyao Huang, Wang Xu, Wei Fan, Jinyuan Liu, Shaoyu Liu, Hongwei Duan, Bin Xu, Xiangyang Ji, Huaping Liu
Robotics · Cross-listed from cs.CV
Robust Autonomous UAV Landing on Maritime Platforms via Multimodal Agentic AI and Active Wave Compensation
Francisco S. Neves, Pedro N. Pereira, Raul D.S.G. Campilho et al. (+1 more)
Autonomous aerial inspection of marine infrastructure is frequently compromised by
stochastic sea states, introducing risks of high-kinetic impacts, post-landing toppling, and sensory occlusion. This paper proposes a decoupled, multi-vehicle landing framework synchronizing an Unmanned Surface Vehicle (USV) equipped with a 3-RPU stabilized platform with a robust Unmanned Aerial Vehicle (
UAV). The architecture utilizes two independent Deep Reinforcement Learning (DRL)
agents: a Soft Actor-Critic (SAC)
agent providing high-frequency wave-motion compensation for the landing deck, and a
multimodal RL
agent for the
UAVs final approach. Evaluated in high-fidelity maritime simulations, the system achieved a 100% landing success rate across 15 trials in wave states varying from calm to rough. Results show a mean stabilization efficacy of 87.8%, maintaining the landing surface within 1 degree of the horizontal plane for 96% of the mission duration in rough conditions, effectively contributing to safer landings.
Read the full abstract ▾
All authors Francisco S. Neves, Pedro N. Pereira, Raul D.S.G. Campilho, Andry M. Pinto
Robotics
Stabilization Learning: A Paradigm Transition Bridging Control Theory and Machine Learning
Quan Quan
Stabilization learning is an interdisciplinary paradigm that bridges control theory and machine learning. Its core idea is to enable systems to adjust their policies under perturbations or environmental changes through real-time feedback and adaptive mechanisms. It takes stability as its primary goal, distinguishing itself from certificate learning, which focuses on formal proofs, and reinforcement learning, which pursues optimality. It encompasses a range of methods, including Lyapunov-based analysis and design, deep feature extraction, and data-driven feedback synthesis, and is applicable to complex high-dimensional, nonlinear systems. This paper elaborates on the two major categories of stability in stabilization learning, as well as three typical application scenarios: control, observation, and recognition. It constructs a unified mathematical framework based on a six-tuple, and expands into two types of seven-tuple models: constrained learning with barrier spaces and tracking problems with targets. It also analyzes the roles, meanings, and implementation choices of key elements such as state space, controlled system, metrics, and policy. Through the formal reformulation of 11 types of problems, including multi-
agent cooperative tracking, visual servo
robot position stabilization, chess games, and Push-T tasks, this paper illustrates the potential applicability of the framework across multiple domains. Finally, it points out that future stabilization learning will focus on two major directions: constructing a unified problem framework and achieving efficient and robust learning, providing solutions for complex system control that combine theoretical rigor with engineering practicality.
Read the full abstract ▾
Robotics
Communication-Aware Robot Execution for Cloud Inference under Spatially Heterogeneous Connectivity
Fengkai Liu, Yuichi Ohsita, Masayuki Murata et al. (+1 more)
Cloud-hosted foundation models enable
robots to use semantic
reasoning beyond onboard computational limits. In this setting, the
robot executes a currently available primitive generated by the cloud, and uninterrupted task progress requires the next cloud result before this primitive is exhausted. This execution becomes fragile under spatially heterogeneous connectivity, because the current primitive determines when the next result is needed, whereas the wireless environment determines where the next request can be submitted and where the response can be retrieved. To address this problem, we introduce the request–response window, which characterizes the time required for the next cloud cycle, including uplink transmission, cloud inference, downlink
retrieval, and inference uncertainty. Building on this window and an available communication map, the proposed framework treats the next request point as a motion decision during ongoing primitive execution, selecting it to provide sufficient communication quality for cloud request submission while preserving progress within the finite support of the current primitive. The selected request point is incorporated into a local planner, which guides the
robot toward it before submission and then uses communication and arrival time penalties while the response is pending. Experiments in a simulated indoor wireless scenario with a measurement-based communication map and emulated cloud inference show that the proposed method achieves the highest task success and lowest request failure rates among the compared methods. The request point ablation further supports the contribution of selecting submission locations to execution continuity under spatially varying connectivity.
Read the full abstract ▾
All authors Fengkai Liu, Yuichi Ohsita, Masayuki Murata, Hideyuki Shimonishi
Robotics
Robustness of Robotic Manipulation: Foundations and Frontiers
Yifei Dong, Zhanyi Sun, Lujie Yang et al. (+5 more)
Humans and animals exhibit remarkable robustness in physical manipulation, yet
robots remain far behind. Progress toward human-level manipulation robustness is hindered by the absence of a unified and systematic understanding: different subfields frame robustness in distinct ways, often leaving the concept ambiguous and limiting deeper analysis as well as communication across research areas. This paper presents a systematic study of manipulation robustness. We begin with a formal definition, characterizing robustness as the degree to which a manipulation system can achieve its goal in the presence of uncertainty and variation. Building on this definition, we introduce general formulations of manipulation robustness from probabilistic and control-theoretic perspectives. We then synthesize the guiding principles and concrete mechanisms of manipulation robustness across perception, planning, control, policy learning, and hardware, illustrating each mechanism through representative works, including foundational and recent studies. In addition, we revisit existing metrics and evaluation methods for quantifying manipulation robustness. Finally, we distill broader lessons for designing robust manipulation systems and discuss open problems and future directions toward achieving human-level robustness in robotic manipulation.
Read the full abstract ▾
All authors Yifei Dong, Zhanyi Sun, Lujie Yang, Manuel Baum, Kei Ikemura, Shuran Song, Florian T. Pokorny, Xianyi Cheng
Robotics
ChronoFlow-Policy: Unifying Past-Current-Future Interaction Flow in Visuomotor Policy Learning
Bokai Lin, Yifu Xu, Xinyu Zhan et al. (+6 more)
Visual signals play a crucial role in policy learning by enabling models to capture object motion and interaction dynamics. Just as humans reason about actions using both past experience and anticipated outcomes, effective policies should integrate past interactions with future predictions. However, existing visuomotor policies typically model either historical context or future dynamics in isolation, lacking a unified temporal representation of interaction dynamics. In this work, we introduce ChronoFlow, a temporally unified representation that captures past, current, and future interaction dynamics through sparse 3D keypoints of both objects and the gripper. Based on this representation, we propose ChronoFlow-Policy, a
diffusion-based visuomotor policy that jointly learns ChronoFlow and action sequences through a co-training objective. Experiments on 14 simulated tasks and 5 real-world manipulation tasks demonstrate that ChronoFlow-Policy consistently outperforms strong
diffusion-policy baselines and improves robustness in long-horizon and non-Markovian manipulation scenarios. Our project page is available at https://the-kamisato-sii.github.io/ChronoFlow-Policy-project-page/.
Read the full abstract ▾
All authors Bokai Lin, Yifu Xu, Xinyu Zhan, Hongjie Fang, Jialin Tian, Fu-Cheng Zhang, Yong-Lu Li, Cewu Lu, Lixin Yang
Robotics
Energy-Optimal Spatial Iterative Learning within a Virtual Tube
Chen Min, Shuli Lv, Pengda Mao et al. (+3 more)
Due to the limited endurance of embedded energy sources such as lithium-
polymer (LiPo) batteries, the flight duration and operational range of unmanned aerial vehicles (
UAVs) are severely constrained. Although energy-efficient trajectory planning and control have been widely studied, most existing approaches rely on accurate system models and computationally expensive optimization procedures. This paper proposes a model-free online iterative learning (IL) framework to minimize energy consumption. Without requiring explicit models of
UAV dynamics or energy consumption, the proposed method improves energy efficiency while maintaining a low computational cost. The per-iteration computational complexity is O(n), where n denotes the number of path points. In the tested cases, the proposed method is approximately 50–60 times faster than the model-based IPOPT
benchmark. Simulation results and real-world flight experiments across multiple
UAV platforms validate the effectiveness, computational efficiency, and practical applicability of the proposed approach.
Read the full abstract ▾
All authors Chen Min, Shuli Lv, Pengda Mao, Huixin Cao, Li Hong, Quan Quan
Robotics
A Large-Language-Model Supported Personalized Driving Framework for Lane Change in Highway Scenarios
Dong Bi, Yongqi Zhao, Paul Kovacevic et al. (+4 more)
Personalized driving can improve the user acceptance of automated driving systems. However, existing methods still provide limited support for translating natural-
language driving preferences, especially when such preferences are expressed implicitly, into executable and distinguishable driving behaviors. This paper proposes a large
language model (LLM)-supported personalized driving framework for highway lane-change scenarios. The framework maps natural-
language driving commands to executable planning parameters in the open-source Apollo automated driving stack according to three driving styles: aggressive, normal, and conservative. To establish this mapping, candidate planning parameters are evaluated based on the resulting lane-change behaviors, and style-specific parameter sets are constructed through clustering and style-intensity ranking. For command interpretation, a
retrieval dataset is constructed to support
retrieval-augmented generation (RAG), enabling LLM-based interpretation of implicit user commands. Experimental results show that the derived parameter sets generate distinguishable personalized lane-change behaviors, while RAG consistently improves preference interpretation, particularly for implicit commands. These results indicate the potential of integrating LLM-based natural-
language interaction with Apollo to support personalized lane-change behavior generation. The source code and the relevant
datasets are available at: https://github.com/ftgTUGraz/LLM-Personalized-Driving.
Read the full abstract ▾
All authors Dong Bi, Yongqi Zhao, Paul Kovacevic, Tomislav Mihalj, Ji Zhou, Jiayuan Gong, Arno Eichberger