Robotics · Cross-listed from physics.flu-dyn
Statistical Turbulence and High-Fidelity Disturbance Fields for Quadrotor Flight Control
Xun Huang
Reinforcement-learning quadrotor controllers are usually trained under simplified wind models, yet the impact of wind-field fidelity, as opposed to magnitude, on policy robustness remains unquantified. This paper compares five disturbance-fidelity levels, from wind-free flight and discrete 1-cosine gusts through statistical
turbulence and synthetic coherent structures to large-eddy-simulation fields of the atmospheric boundary layer, in a full cross-fidelity train test evaluation of proximal policy optimization (PPO)
agents, with cascaded PID and
geometric SE(3) controllers as training-free references, over a 0-12 m/s wind sweep. Before any controller comparison is made, all disturbance data are validated: every synthetic generator is checked quantitatively against its analytical or certification-standard reference, and the large-eddy-simulation fields against the imposed log law. On a racing-class quadrotor in hover, the train test
matrix is remarkably flat, and the cheapest structured training wind, which is discrete-gust domain randomization, ranks first in every test column, a ranking replicated on a wind-sensitive 27 g platform; a once-tuned
geometric controller brackets the learned PPO policies at zero crash rate. Mechanism diagnostics show that control authority, not wind realism, bounds robustness, so wind-fidelity investment should scale with platform wind sensitivity.
Read the full abstract ▾
Robotics
Ergodic Control and Controlled Diffusion for Robot Learning: Review and Tutorial
Max Muchen Sun, Cem Bilaloglu, Ananya Rao et al. (+6 more)
Diffusion learning leverages the statistical mechanism of
diffusion processes for learning,
reasoning, and inferring complex distributions from data. Recent advances in
diffusion learning have been transformative, with
robot learning emerging as a key opportunity area, with applications spanning perception, control, and decision-making. At the same time, the statistical mechanism of
diffusion processes can be controlled to shape the temporal
evolution of the state distribution underlying
robot trajectories, inducing ergodic behavior in robotic systems. The frameworks of controlled
diffusion and ergodic control were developed around the same time as
diffusion learning, and their theories and
algorithms have increasingly converged. Ergodicity induced by controlled
diffusion has several significant implications for
robot learning, distinct from applying
diffusion learning to robotics problems: it formally enforces statistical properties required to ensure optimality of
robot learning, enables non-myopic search over uncertain information landscapes for data collection, and enables behavior specification based on spatial rather than temporal characteristics of trajectories. This survey introduces the intuition behind controlled
diffusion for
robot learning, explores its connection to
diffusion learning, presents theoretical foundations and numerical tutorials for solving controlled
diffusion and ergodic control problems, and reviews applications across robotics. Finally, we discuss key challenges and future opportunities in leveraging controlled
diffusion for
robot learning.
Read the full abstract ▾
All authors Max Muchen Sun, Cem Bilaloglu, Ananya Rao, Stefan Ivic, Guillaume Sartoretti, Kathleen Fitzsimons, Ian Abraham, Sylvain Calinon, Todd Murphey
Robotics
Breaking speed scaling in quadrupedal robots via Huygens' coupled-pendulum dynamics
Yucheng Tao, Yongbin Jin, Shaowen Cheng et al. (+8 more)
Achieving biological-level running speeds has largely been pursued through advances in control
algorithms, which improve the utilization of existing hardware. However, the ultimate speed limits remain governed by the underlying force and torque requirements of rapid locomotion, which are typically addressed through increased actuator capacity. Inspired by Huygens' coupled pendulums, we demonstrate that superior locomotion can emerge from principled exploitation of intrinsic dynamics rather than brute-force hardware scaling. Inter-limb inertial coupling redistributes energy across the gait cycle and reduces peak joint torque required for rapid periodic motion, thereby expanding the achievable speed without proportional increases in actuator capability. Incorporating hardware parameters as additional design variables further extends this analysis into a co-optimization framework, enabling the systematic utilization of inertial coupling in
robot design. Guided by this framework, a quadruped
robot achieves a running speed of 10.74 m/s (Froude number 21.4) and completes a 100-meter sprint in 12.2 seconds, representing the first legged
robot to surpass 10 m/s. These results establish inertial coupling as an underlying mechanism governing high-speed legged locomotion and highlight its role in reducing force requirements, offering new insights into the design of agile robotic systems.
Read the full abstract ▾
All authors Yucheng Tao, Yongbin Jin, Shaowen Cheng, Xianwei Liu, Yanyan Yuan, Yanhong Liang, Chengkai Su, Chaojie Fu, Guorong Lan, Wei Yang, Hongtao Wang
Robotics
Extending the Speed Limit of Quadrupedal Locomotion via Refined Actuator Modeling and Adaptive Command Scheduling
Yucheng Tao, Shaowen Cheng, Guorong Lan et al. (+3 more)
Achieving high-speed locomotion in quadrupedal
robots remains highly challenging, as actuators operate near their physical limits and exhibit pronounced nonlinearities. However, many existing methods neglect actuator nonlinearities and physical constraints during training, leading to a significant sim-to-real gap under highly dynamic motions and limiting achievable performance. To address this issue, we propose a high-speed locomotion framework that reduces sim-to-real discrepancies and stabilizes learning over a wide command distribution. A refined actuator model explicitly captures high-speed voltage coupling and
magnetic saturation, enabling a more accurate representation of the torque-speed envelope. In addition, a reinforcement learning framework incorporating a two-stage curriculum and adaptive command scheduling (ACS) ensures stable training. Experiments on the 36.5 kg quadruped BlackPanther2 (BP2) demonstrate speeds of up to 13.2 m/s on a treadmill and 11.65 m/s outdoors, establishing a new state-of-the-art and, to the best of our knowledge, a world record for quadrupedal
robot locomotion. The results further highlight the importance of accurate actuator modeling in preventing non-physical policy exploitation, and show that ACS improves robustness without sacrificing performance.
Read the full abstract ▾
All authors Yucheng Tao, Shaowen Cheng, Guorong Lan, Yanyan Yuan, Yongbin Jin, Hongtao Wang
Robotics
Planning along Differentiable Charts of Constraint Manifolds with General-Purpose IK Solvers
Thomas Cohn, Seiji Shaw, Harel Biggie et al. (+3 more)
Planning trajectories for
robot manipulators under kinematic equality constraints restricts feasible motions to a measure-zero submanifold of the configuration space, requiring special algorithmic treatment. A promising strategy is parametrizing the set of feasible configurations using analytic inverse kinematics (IK). Bespoke analytic IK functions can be written to be differentiable, a necessary property for gradient-based trajectory optimization. But the vast majority of IK functions are computed by automated meta-solvers like IKFast, and are difficult to modify for differentiability. We present a new approach for computing gradients of analytic IK parameterizations: we leverage the inverse function
theorem to recover the desired gradients from the ordinary forward kinematic Jacobian. Furthermore, we present a least-squares domain extension and an optimization-amenable description of the reachability constraint, which preserves gradient signal outside the reachable workspace. We demonstrate the efficacy of our approach through numerical experiments and downstream tasks, including a hardware demonstration of an RB-Y1 picking up a box and placing it on a table. Project website: https://cohnt.github.io/inverse-function-
theorem-parameterization/
Read the full abstract ▾
All authors Thomas Cohn, Seiji Shaw, Harel Biggie, Travis Manderson, Nicholas Roy, Russ Tedrake
Robotics
ReactHuman: A Physics-Grounded Benchmark for Human-Like Reactive Decision-Making in Embodied Multimodal LLMs
Yizhan Li, Jianxin You, Mengyang Xiong et al. (+5 more)
Reacting to sudden physical hazards (catching a slipping plate, dodging a falling knife) is both a meaningful test of embodied intelligence and a hard requirement for deploying
multimodal large
language models (MLLMs) as the decision coreof household
robots. Existing evaluations, however, probe intuitive physics passively through question answering over
videos, or target deliberate, long-horizon tasks such as navigation and rearrangement; none measure whether a model can turn physical understanding into immediate, safety-critical action. We introduce ReactHuman, the first physics-grounded
benchmark for human-like reactive decision-making, in which the evaluated MLLM acts as the
brain of a simulated humanoid facing sudden household hazards; it spans 17 event families and over 1,000 bit-for-bit reproducible scenes with exact, annotation-free ground truth derived from 240 Hz rigid-body simulation, including
adversarial objects whose appearance contradicts their physics (a foam anvil, a steel apple). We further design a five-metric suite that scores each reaction along three axes: reasonable, safe, and physically grounded. We physically execute every committed plan so that decisions have observable consequences. With this harness we evaluate seven representative MLLMs. Results show that reactive safety is far from solved: models mishandle roughly one hazard in three, act from fixed dispositions rather than the observed scene, trust appearance over motion, and miss interception points at meter scale even when the chosen action is correct; none of these failures shrink with model scale. ReactHuman thus offers both a fine-grained diagnosis and a scalable training signal toward physically grounded, safety-aware embodied
agents. The
benchmark can be found here: https://huggingface.co/
datasets/Alan123/reacthuman-
benchmark-scaled
Read the full abstract ▾
All authors Yizhan Li, Jianxin You, Mengyang Xiong, Yinhuan Chen, Zicheng Zhao, Dekun Wu, Dongqing Zhang, Bang Liu
Robotics · Cross-listed from cs.CV
Evaluation of Vision-Language Models Across Diverse Coastal Environments
Seth Knoop, Chad R. Samuelson, Gabriel R. Slade et al. (+2 more)
Vision-
language models (VLMs) enable robotic per- ception by associating visual observations with natural-
language concepts. Yet their performance in coastal environments remains largely unexplored. We introduce a densely labeled coastal
dataset containing more than 1,000 images collected across seven missions in three regions of Oahu, Hawaii, with 18 semantic classes and over 7,400 annotated instances. We evaluate seven modern VLMs through three complementary experiments mea- suring text-to-mask, mask-to-mask, and mask-to-text
alignment. Broad landscape classes are generally recognized more accurately than conventional object and coastal classes, with coastal con- cepts presenting the greatest challenge. However, comparisons of shared conventional classes across coastal and terrestrial
datasets reveal no consistent performance difference attributable solely to environmental context. Mask-to-mask matching also remains similar across conventional and coastal classes, while alternative textual labels substantially improve recognition of several coastal concepts. These results suggest that lower performance on coastal classes (at least on the objects/query categories evaluated) is heavily influenced by segmentation and linguistic representation.
Read the full abstract ▾
All authors Seth Knoop, Chad R. Samuelson, Gabriel R. Slade, Brady Moon, Joshua G. Mangelson
Robotics
Expressive Robotic Pianist: Mastering Complex Piano Repertoire with Graph-Mimic and Musical Dynamics
Yanhong Liang, Xianwei Liu, Chaojie Fu et al. (+7 more)
Enabling
robots to perform musical instruments with human-level expressivity represents a frontier in bridging the gap between mechanical precision and artistic interpretation. Despite advances in robotic dexterity, replicating the fluid finger transitions and nuanced dynamic control characteristic of human pianists remains a significant challenge. Through a reinforcement learning-based control framework, we demonstrate that a dexterous robotic hand can achieve high-fidelity performance across a diverse piano repertoire. Central to our approach is a graph-based optimization strategy that guides the
robot to generate natural pre-press and key-press fingering strategies that closely resemble human movement patterns. To achieve expressive sound production, the control system is coupled with a physics-inspired acoustic model that modulates keypress velocity to accurately reproduce the dynamic variations specified in musical scores. Quantitative evaluations demonstrate that our expressive control model significantly outperforms baseline methods in both finger morphology similarity and dynamic velocity accuracy. In a perceptual test involving participants from diverse listener groups, performances generated by our system are significantly preferred over baseline robotic performances and are indistinguishable from human performances for non-professional audiences. Furthermore, extensive experiments across multiple musical styles confirm that our method maintains high note-level accuracy while achieving expressive performance. Our approach provides a robust pathway for robotic systems to move beyond mere mechanical accuracy, elevating robotic musicianship to a level of expressive performance comparable to human pianists.
Read the full abstract ▾
All authors Yanhong Liang, Xianwei Liu, Chaojie Fu, Shaowen Cheng, Yanyan Yuan, Chengwei Zhuo, Xi Chen, Yongbin Jin, Wei Yang, Hongtao Wang
Robotics · Cross-listed from cs.CV
GRADE: Single-Frame Generative Radar Depth Estimation Under Visual Degradation
Bin Zhao, Patrick Chiou and Nakul Garg
Dense 3D depth perception fails under smoke, fog, and darkness because optical sensors cannot penetrate airborne particulates. mmWave radar remains usable and measures range accurately under these conditions, but its small aperture limits angular resolution. We present GRADE, which grounds a pretrained generative prior in single-frame radar geometry to estimate high-fidelity metric depth. GRADE first maps raw 4D radar spectra to coarse metric depth. A latent
diffusion backbone then recovers structural detail while conditioning every denoising step on this estimate. A pixel-space adapter uses residual camera cues when available and is trained across clear, smoke-degraded, and occluded inputs so the full output approaches the radar-conditioned path as visibility degrades. Trained and evaluated on ~95K frames across 12 buildings with real smoke, GRADE achieves an MAE of 0.303 m in clear scenes and 0.313 m under smoke, outperforming existing baselines. Code and
datasets are available at https://phi-lab-rice.github.io/GRADE.
Read the full abstract ▾
Robotics
Lie-Algebraic Bell Recurrences for Arbitrary-Order Twist Jets and Parallel-Mechanism Closure
Daniel Condurache
This paper develops an arbitrary-order kinematic construction that links serial propagation, parallel-mechanism closure, and rigid-platform point fields within one dual screw framework. A cylindrical joint is retained as one native physical block, with revolute and prismatic joints obtained as special cases. For each fixed joint axis, ordinary Bell
polynomials organize the derivatives of the exponential factor; across a chain, the noncommuting factors remain in their physical order. Initial-frame prefix and terminal-resolved covariant formulas then produce equivalent representations of the serial twist jet. For a parallel mechanism, repeated Leibniz differentiation, with joint-level derivatives organized by Bell
polynomials, yields an arbitrary-order triangular active-passive closure recurrence: the same passive Jacobian is solved at every derivative order at a regular configuration, while the right-hand side contains only prescribed active data and lower-order jets. The resulting platform twist jet is mapped exactly to the point-independent affine invariants of the velocity, acceleration, jerk, and snap fields. The validation is deliberately complementary: a generic 3C chain with noncoplanar axes and nonzero rotational and translational cylindrical coordinates tests ordered serial propagation, an RR+RRR spherical wrist tests active-passive closure, and a Hunt-type 6-RUS mechanism with six active revolute joints tests an independently reconstructed platform jet and its affine fields. Independent differentiation of the rigid motion, evaluation of the affine fields, and the differentiated branch closures all agree through fourth order with residuals below $10^{-12}$ in the corresponding SI units. The formulation is purely kinematic and applies at configurations where the selected active-passive partition is regular.
Read the full abstract ▾
Robotics
When Information is Worth the Risk: Behavioral Valuation for Hazardous Robotic Exploration
Alkesh K. Srivastava, Aamodh Suresh, Carlos Nieto-Granda et al. (+1 more)
Hazardous robotic exploration requires
robots to map spatial risks, such as unsafe terrain, radiation, fire, mines, or structural damage, while operating where collecting information can itself cause failure. A highly informative path may expose the
robot to hazards, terminate execution, and prevent future observations. Hazardous exploration therefore requires deciding not only where uncertainty is largest, but when reducing it is worth the risk. This paper introduces a valuation-layer view of this problem. We keep the belief update, sensor model, physical risk model, and finite-horizon informative planner fixed, and change only the scalar objective used to rank feasible paths. Within this framework, we introduce a risk-augmented Behavioral Information objective based on Prelec probability weighting, yielding an interpretable family of conservative-to-aggressive information-risk valuations. Theoretically, we show that valuation parameters create switching boundaries between high-information/high-risk and lower-information/lower-risk paths, and induce a transformed Pareto-frontier structure over feasible exploration policies. Large-scale failure-truncated grid-world experiments show that valuation alone reshapes the information-risk frontier. Shannon information planning remains a strong raw-information baseline, while risk-aware objectives can reduce hazard exposure and
robot losses by avoiding failures that truncate future sensing. Risk-augmented Behavioral valuation is Pareto-competitive with standard risk-aware baselines and provides interpretable conservative and intermediate regimes. These results support a framework in which
robots reason not only about how much uncertainty an action reduces, but whether that reduction is worth the risk required to obtain it.
Read the full abstract ▾
All authors Alkesh K. Srivastava, Aamodh Suresh, Carlos Nieto-Granda, Philip Dames
Robotics
HuRo: Robotizing Human Videos for Scalable VLA Pretraining
Jinho Jeong, Se June Joo, Jaehyun Kang et al. (+4 more)
Human
video datasets offer an abundant and diverse source of interaction data that can complement expensive real-
robot data. To bridge the human-to-
robot embodiment gap, existing approaches either robotize
videos in task-matched settings or address observation and action
alignment separately. In this work, we systematically examine whether robotized human
videos can serve as an effective and scalable source of
robot-aligned supervision. To this end, we develop a robotization pipeline that converts heterogeneous human
videos into
robot-aligned observations and action trajectories while inferring missing intermediate signals across annotation levels. Using this pipeline, we construct the HuRo
dataset, comprising about 630K robotized episodes and 142M processed frames from five human-
video sources. Across four real-world manipulation tasks, pretraining a VLA policy on increasing amounts of robotized human-
video data improves overall completion from 51.5% to 80.3% and OOD completion under spatial and visual shifts from 34.9% to 72.2%. Ablations further show that visual robotization improves OOD robustness and that end-to-end pretraining with retargeted actions outperforms visual-only transfer. Project website: https://3587jjh.github.io/HuRo.
Read the full abstract ▾
All authors Jinho Jeong, Se June Joo, Jaehyun Kang, Dongyun Kim, Yena Kim, Hanjung Kim, Seon Joo Kim
Robotics
Show-Harness: Just a VLM Agent Can Play Robots
Yanzhe Chen, Zechen Bai, Zhijun Cao et al. (+7 more)
Foundation
vision-
language models (VLMs) exhibit broad intelligence about the world, yet translating this intelligence into
robot control remains challenging. We present Show-Harness, an Embodied Harness that enables VLMs to "play"
robots through a compact semantic interface linking intent to action. Show-Harness exposes discrete semantic action units that VLMs can naturally reason over, while embodiment-specific interpreters deterministically ground them into local
robot actions, keeping the VLM directly responsible for fine-grained physical decisions. Through the same interface, Show-Harness demonstrates the feasibility of (1) directly unlocking closed-source frontier VLMs for zero-shot
robot control, and (2) adapting small-scale open-source VLMs for low-cost deployment with just a few GPU-hours of fine-tuning. We further develop GUMI (GUI Manipulation Interface), which extends the same semantic action space to GUI-based demonstration collection, allowing humans and
agents to "play"
robots across embodiments without specialized teleoperation hardware. Extensive experiments show that Show-Harness-equipped VLM
agents generalize robustly across tasks, embodiments, and environments, outperforming representative agentic and VLA paradigms. These results suggest that the right interface can unlock substantial embodied capability from foundation VLMs, without requiring additional model capacity or costly embodiment-specific pretraining.
Read the full abstract ▾
All authors Yanzhe Chen, Zechen Bai, Zhijun Cao, Wenzheng Zeng, Kevin Qinghong Lin, Yiqi Lin, Guoqiang Liang, Kevin Yuchen Ma, Qiming Huang, Mike Zheng Shou
Robotics
DUET-DINO: Simultaneous Cross-View World Modeling for Latent Planning in Robot Manipulation
Nisarga Nilavadi, Ralf Römer, Moritz Reuss et al. (+5 more)
Action-conditioned latent world models predict future visual representations, enabling zero-shot goal-conditioned
robot planning and control. However, their predictions for fine-grained spatial and rotational actions are unreliable for full 7-DoF end-effector control. To address this gap, we introduce DUET-DINO, a simultaneous cross-view latent world model that jointly learns action-conditioned predictions from static side- and wrist-camera observations through cross-view conditioning. By exploiting complementary global scene and gripper-centric information, DUET-DINO enables latent planning over the full 7-DoF action space. Across spatially diverse reach, orientation-intensive angled-reach, and multi-goal grasp-and-lift tasks, DUET-DINO consistently outperforms single-view and independent dual-view baselines, achieving 92% success on reach, 72.5% on angled-reach, and 60.0% on lift tasks. DUET-DINO is trained from scratch on DROID and RoboArena
datasets and generalizes robustly under visual distribution shifts. We further show that while V-JEPA 2 wrist-view predictions underestimate visual dynamics induced by fine-grained actions, DINOv3 predictions better capture action-conditioned scene changes, leading to stronger downstream planning. The code and model checkpoints will be open-sourced. Project page: https://utn-air.github.io/DUET-DINO
Read the full abstract ▾
All authors Nisarga Nilavadi, Ralf Römer, Moritz Reuss, Michael Krawez, Tobias Jülg, Angela P. Schoellig, Rudolf Lioutikov, Wolfram Burgard
Robotics
Coastal Environment Generation with HoloOcean
Abigail Austin, Brady Moon and Joshua G. Mangelson
Marine robotic simulation provides a safe and inexpensive method of developing and testing
algorithms for unmanned underwater vehicle (UUV) and unmanned surface vessel (USV) autonomy and perception before full field deployment. However, these simulations are often limited by the availability of simulated environments. Current marine robotics simulation suites offer manual ways to edit or create environments, but they require existing data or specialized knowledge of the environment system. To address these issues, we introduce a novel Unreal Engine 5 level generation pipeline that enables automatic creation of coastal environments for HoloOcean. Our pipeline relies on a user-provided overhead image of a coastal scene. The pipeline then uses the image to generate height map data, as well as automatically select assets and place them in the environment.
Read the full abstract ▾
Robotics
Multi-Agent Reinforcement Learning for Autonomous UAV Exploration in Wildfire Response
Caden Chandra and Jerry Ng
This study develops a deep reinforcement learning framework for training Unmanned Aerial Vehicle (
UAV)
agents to navigate and monitor simulated wildfire environments. Results show that
agents learn increasingly stable and effective behaviors over time, as demonstrated by converging loss trends, improved reward signals, and more consistent navigation patterns such as fire-boundary tracking. Overall, these findings highlight the potential of deep reinforcement learning (DRL) based
UAV systems for autonomous wildfire monitoring and suggest that environmental structure and reward design influence policy effectiveness.
Read the full abstract ▾
Robotics
Frequency-Conditioned Flow Matching for Vision-Language-Action Models
Haochen Niu, Shengye Dong, Hao Liu et al. (+2 more)
Robot actions are temporally correlated trajectories whose frequency components encode motion at different scales with highly non-uniform energy distributions. Yet Flow Matching–based
vision-
language-action (VLA) models typically generate actions in temporal coordinates, without explicitly modeling or systematically leveraging this frequency heterogeneity. We introduce FreqFM, a frequency-conditioned Flow Matching framework for VLA models. It raises action frequency from an implicit trajectory property to an explicit conditioning dimension that spans the entire generation pipeline. Concretely, in DCT frequency coordinates, FreqFM constructs a spectrum-matched source distribution, adaptively balances the objective across frequencies, and constrains per-frequency guidance residuals using the corresponding reference transport scales. FreqFM integrates into existing Flow Matching action experts without changing the VLA backbone. Across LIBERO, LIBERO-Plus, and VLA-Arena, FreqFM consistently improves performance, including a 9.3-point gain on LIBERO-Plus, and further demonstrates its effectiveness on six real-
robot tasks.
Read the full abstract ▾
All authors Haochen Niu, Shengye Dong, Hao Liu, Peiwen Lin, Wang Chuang
Robotics
A traffic management system for large and heterogeneous vehicles in narrow industrial environments
Alessandro Bonetti, Silvia Proia, Simone Guidetti et al. (+1 more)
The coordination of Automated Guided Vehicles (AGVs) in high-density industrial environments represents a critical challenge within Logistics 4.0, as traditional traffic management methods often lead to inefficiencies caused by negotiation-based priority assignment. To overcome the resulting limitations, this paper presents an innovative AGV traffic management system based on a Lifelong Multi-
Agent Path Finding (L-MAPF)
algorithm operating on roadmaps generated with Non-Uniform Rational B-Splines (NURBS) curves. The approach guarantees locally optimal coordination and ensures safe operation of large and heterogeneous AGVs. Building on this concept, the proposed framework integrates a modified version of the Bounded Horizon Conflict Based Search (CBS) technique within a Rolling Horizon Conflict Resolution strategy, utilizing an extended time horizon for each
agent to enable effective conflict resolution in corridors identified by a topological map. In contrast to state-of-the-art methods for AGV fleet traffic management, the proposed solution is designed for real-world, non-standardized (i.e., non-grid-like) industrial settings characterized by narrow bidirectional corridors and high-traffic density, where AGVs of various sizes and capabilities operate simultaneously. Key contributions include an anytime conflict resolution strategy with adaptive time horizon regulation, an execution layer for safe and standard-compliant interaction with real AGVs, and an advanced mechanism for deadlock detection and resolution. Experimental results obtained in realistic industrial environments demonstrate higher throughput, with improvements of up to 11% over a conventional rule-based traffic management system, a state-of-the-art industrial method, and a priority-based L-MAPF variant, while maintaining continuous operation and improved efficiency.
Read the full abstract ▾
All authors Alessandro Bonetti, Silvia Proia, Simone Guidetti, Lorenzo Sabattini
Robotics
Data-Driven Risk Fields for Safer End-to-End Autonomous Driving
Yuanxin Tian, Zhiyuan Liu, Jinhao Li et al. (+9 more)
Safety is a fundamental requirement for autonomous driving, yet existing end-to-end driving models still lack explicit risk-aware learning capacities. Existing rule-based risk models provide interpretable safety priors, yet their absolute risk scores depend on handcrafted functions, coefficients, and thresholds. Learning-based risk representations reduce part of this manual design, but their supervision often relies on occupancy-derived labels or heuristic cost values, which may not capture ego-conditioned planning risk. In this paper, we propose DRiF, a data-driven risk-field framework for safer end-to-end autonomous driving. DRiF learns a shared BEV feature with static map segmentation, dynamic risk prediction, and vehicle planning. For dynamic risk learning, DRiF converts rule-based safety priors into pairwise risk labels, and trains the risk field to preserve relative risk ordering instead of regressing handcrafted absolute scores. Experiments on Bench2Drive show that DRiF achieves competitive overall performance, with consistent improvements in driving score, success rate, and collision-related metrics. These results establish relative risk supervision as an effective way to connect explicit safety structure with end-to-end planning. The data and code will be publicly available.
Read the full abstract ▾
All authors Yuanxin Tian, Zhiyuan Liu, Jinhao Li, Liangfan Zhu, Shuai Wang, Heye Huang, Qingwen Meng, Fang Zhang, Liuzhu Tong, Zhenhua Xu, Wenhao Yu, Jianqiang Wang
Robotics · Cross-listed from cs.CV
PACE: Perceived-Latency-Aware Cascading Service Routing and Filler Control for QoE-Efficient Retrieval-Augmented Dialogue Serving
Lin Huang, Yujuan Tan, Weisheng Li et al. (+4 more)
We present the PACE, a framework for
retrieval-augmented dialogue serving that formalizes Perceived Time-to-First-Response (PTFR) as a QoE objective and minimizes it under quality/cost constraints. Unlike prior work on cascaded routing, semantic caching, or adaptive
retrieval, PACE jointly controls which answer source composes the response and what fills the waiting window. Deployed on a humanoid-
robot sales service, it combines three mechanisms: a load-adaptive cascading router, a joint path-filler controller, and
volatility-aware cache admission. On 75k CarQA requests, the cascade halves pure-LLM PTFR at P95 (0.29 vs 0.53s at c16). The adaptive controller reaches 0.41s P95, outperforming RAG by 2.4 times at high load with equal quality. The filler controller cuts calls by 94% with zero conflict.
Volatility-aware admission reduces stale answers from 86% to 0%. A gating rule ensures the controller never worse than the baseline, with exposure bounded by one hold period. This is the first quantification of filler-answer conflict risk in deployed services.
Read the full abstract ▾
All authors Lin Huang, Yujuan Tan, Weisheng Li, Lixiang Zeng, Kun Yang, Yongzong Wang, Suihan Xiao
Robotics
A Confidence-Aware Multimodal Fusion Framework for Industrial Human-Robot Collaboration
Xinyu Liu, Qiqi Dong, Boya Jia et al. (+2 more)
A confidence-aware
multimodal fusion framework (CAMF) is proposed to realize reliable human intention prediction for industrial human-
robot collaboration. This framework fuses four heterogeneous modalities including object 6D pose, gaze, skeletal motion and IMU-based hand motion. It embeds a confidence-trend-driven dynamic fusion mechanism into BiLSTM to adaptively balance bidirectional temporal features according to real-time modality reliability. A confidence-guided balanced learning strategy combined with a confidence freezing mechanism is further adopted to adjust network gradients dynamically, suppress noise from low-quality modalities and mitigate cross-modal learning bias. A physical platform based on the UR3 collaborative
robot is built for experimental validation. Comparative results show that the proposed method reaches an intention recognition accuracy of 91.86% and outperforms existing
multimodal fusion approaches in overall performance and stability. It also maintains satisfactory accuracy under low light and partial occlusion interference. In practical assembly tasks, the framework enables proactive and stable human-
robot cooperation with strong environmental adaptability.
Read the full abstract ▾
All authors Xinyu Liu, Qiqi Dong, Boya Jia, Yi Zhang, Binbin Lian
Robotics
Odometer-Agnostic Drift Correction Using OpenStreetMap Lane Geometry
Joaquin Caballero, Emilio Garcia-Fidalgo, Alberto Ortiz et al. (+1 more)
Despite significant progress in odometry estimation, long-term drift remains a fundamental limitation of incremental pose integration, especially in large-scale or loop-free environments. Existing map-assisted methods can reduce drift, but often depend on dense maps, sensor-specific processing, or complex matching pipelines. We propose a lightweight open-source, odometry-agnostic correction method that aligns short trajectory segments to OpenStreetMap (OSM) lane centerlines. By formulating drift correction as a direct
alignment between recent odometry and sparse lane geometry, the method enables efficient online operation without dense priors or expensive preprocessing. Experiments with
LiDAR and visual odometry backends demonstrate consistent improvements, with particularly strong gains under severe drift.
Read the full abstract ▾
All authors Joaquin Caballero, Emilio Garcia-Fidalgo, Alberto Ortiz, Jarno Ralli
Robotics
Deformable Object Manipulation under Partial Observability via Real-Time Full-Shape Estimation
Kosar Behnia, Ville Kyrki and Gokhan Alcan
Manipulating deformable objects (DOs) is challenging due to their high-dimensional state space, underactuated dynamics, and partial observability. In this paper, we propose cRVAE, a lightweight conditional recurrent variational autoencoder that estimates the full DO state from only partial corner-node observations during inference. The resulting model is used as the forward model in a receding-horizon optimal control framework for obstacle-aware collaborative DO manipulation. In simulation on rope and fabric, cRVAE estimates the full DO state from the available corner-node measurements alone, matching the accuracy of a parameter-identified XPBD model. At inference it uses no physical parameters as model inputs and performs no online parameter identification. It also runs approximately 350 times faster on the rope and over 1500 times faster on the fabric per forward pass, keeping horizon-based planning within the 100 ms control budget where XPBD exceeds it already at short horizons. Full-shape estimation from corner sensing at in-loop speed is what makes the model deployable on hardware, which we demonstrate on a Unitree Go2
robot.
Read the full abstract ▾
Robotics
GM-Loco: Terrain-Adaptive Humanoid Locomotion on Granular Media
Junnosuke Kamohara, Feiyang Wu, Andy Ningan Zong et al. (+4 more)
Humanoid locomotion on granular terrain remains a significant challenge due to its complex foot-terrain interaction dynamics that are difficult to model. Existing approaches either ignore granular contact dynamics or incorporate simplified normal force models with heuristic tangential components. In this work, we present a physics-grounded granular contact model based on three-dimensional resistive force theory (3D RFT) and efficiently simulate granular terrain for reinforcement learning (RL) training. Unlike traditional rigid contact models and simplified granular contact models with ad-hoc heuristics, our contact solver produces physically accurate granular intrusion dynamics without resorting to heuristics. It captures realistic penetration and tangential drag during training, enabling the policy to learn behaviors that transfer reliably to real-world granular terrain where rigid contact models fail. To adapt to varying terrain conditions, we train a terrain-adaptive locomotion controller via teacher-student RL, using a variational autoencoder to encode terrain information into a compact latent representation. Simulation studies using material point method (MPM) with NVIDIA Newton demonstrate that our method generalizes to unseen granular terrains, achieves a significantly higher success rate than baselines, and demonstrates zero-shot terrain identification and adaptation. We further validate our approach through extensive hardware experiments across diverse real-world granular terrains including basalt, dry sand, and beach sand. To the best of our knowledge, this is the first demonstration of agile humanoid locomotion on real-world granular terrain. Project page: https://humanoid-gm-locomotion.github.io/HUMANOID-GM/
Read the full abstract ▾
All authors Junnosuke Kamohara, Feiyang Wu, Andy Ningan Zong, Daniel I. Goldman, Yashwanth Nakka, Seth Hutchinson, Ye Zhao
Robotics
SwingBot: Learning Whole-Body Brachiation for Humanoid Robots
Yujie Xiong, Peng Zhai, Taixian Hou et al. (+6 more)
Brachiation enables primates to move across overhead supports when ground paths are blocked, suggesting a complementary locomotion mode for
robots operating in cluttered or hazardous environments. Bringing this capability to high-DoF humanoid
robots is difficult because the controller must discover a long-horizon release-swing-capture sequence, coordinate alternating contacts with whole-body momentum, and act without reliable measurements of segment-relative displacement or hook-contact state. We present SwingBot, a learning framework for continuous humanoid brachiation with passive wrist hooks. SwingBot makes the task trainable by organizing learning around the structure of brachiation: biomimetic keyframes make rare release-swing-capture transitions reachable during early exploration, and recurrent privileged-state estimation provides compact position and contact latents for deployment. Hardware experiments demonstrate continuous bar traversal and robustness to payload, external disturbances and different bar spacings, showing that this formulation offers a practical route to whole-body robotic brachiation.
Read the full abstract ▾
All authors Yujie Xiong, Peng Zhai, Taixian Hou, Quancheng Qian, Cunwang Liu, Kangmai Hu, Long Yang, Zhiyan Dong, Lihua Zhang
Robotics · Cross-listed from cs.IT
Frame-Coded Legged Locomotion over Noisy Terrain
Lav R. Varshney
Open-loop multilegged locomotion over rough terrain has been interpreted as matter transport over a noisy channel: leg-ground interactions are discrete basic active contacts, terrain deletes or perturbs those contacts, and spatial redundancy concentrates the resulting thrust and arrival time. That construction is repetition-like because every module carries the same scalar locomotion task. It consequently provides neither a positive task rate nor a decoder that changes with the surviving contact set. Here we formulate locomotion instead as a quantized finite-frame expansion with erasures. A d-dimensional body-level command is mapped into N>d heterogeneous local contact commands. Rough terrain erases or corrupts frame coefficients, while a contact-gated compliant morphology physically realizes the weighted active-subframe decoder. For a linear-Gaussian model, mechanical
equilibrium is exactly the posterior mean, tangent stiffness is posterior precision, and mechanical compliance is posterior covariance. Equal-norm Parseval frames are shown to be minimax optimal against one missing contact, two-contact robustness is governed by frame coherence, and a harmonic frame gives a directly realizable gait family. For independently surviving contacts of probability q, random Gaussian gait frames admit exact reconstruction at every analog dimension rate R<q, with a binomial reliability exponent, whereas recovery of arbitrary commands is impossible for R>q. Residual contact noise yields an asymptotic per-mode amplification 1/(q-R) and a vanishing mechanical stiffness margin at the threshold. An information-locomotion inequality and an exact incremental-redundancy rule direct the next gait component toward the softest task-relevant unresolved mode. The resulting analog frame-coding
theorem establishes a finite relative redundancy and converse as part of a fundamental limit theory of legged locomotion.
Read the full abstract ▾
Robotics
RoboFolDeX: A Physical-World Benchmark for Long-Horizon Robotic Manipulation of Deformable Objects
Chenhuan Liu, Yi Xu, Feng Wu et al. (+7 more)
Embodied AI, including
vision-
language-action and world-action models, must operate reliably in the physical world. Yet methods that perform well in simulation can degrade substantially on real
robots, especially in long-horizon deformable-object manipulation, where policies must track changing states and execute reliable multi-stage bimanual interactions. Existing real-
robot benchmarks mainly focus on short-horizon rigid-object tasks and offer limited coverage of long-horizon deformable manipulation. We introduce RoboFolDeX, a physical-world
benchmark built entirely from real-
robot data, with garment folding as its primary task. Since real-
robot data collection is costly, RoboFolDeX studies how heterogeneous physical experience can be reused efficiently. The
benchmark is organized around four research axes: leveraging human intervention and recovery data collected during deployment; transferring data across tasks, including across garment categories and from rigid to deformable-object manipulation; reusing data across scenes with changes in lighting, background, and layout; and transferring data across robotic embodiments. RoboFolDeX provides 2,000+ hours of real-
robot data spanning 20+ tasks and 10+ embodiments. We also establish a fair real-
robot evaluation platform for externally submitted policies, with standardized tasks, held-out physical objects, controlled initializations, and a unified execution protocol. The platform is publicly accessible at https://ai.midea.com/#/fold-challenge. We hope RoboFolDeX will serve as a unified testbed for heterogeneous real-
robot data reuse and reliable long-horizon deformable manipulation.
Read the full abstract ▾
All authors Chenhuan Liu, Yi Xu, Feng Wu, Hanyang Wang, Wenxiao Kuai, Weihao Ding, Shan Wang, Yang Liu, Shuyong Gao, Wenqiang Zhang
Robotics
CougarTail & CUB: A General-Purpose Mast and Central Utility Board for Cylindrical Underwater Enclosures
Ben Washburn, Clayton Smith, Eli Gaskin et al. (+4 more)
Cylindrical watertight enclosures are widely used across various underwater systems, from unmanned underwater vehicles (UUVs), to remotely operated vehicles (ROVs), to various sensor platforms. However, electronics are typically built on rectangular PCBs arranged in horizontal stacks, which inefficiently occupy the circular cross-section volume that is critical for both payload capacity and buoyancy management. This paper presents CUB (Central Utility Board) and CougarTail, a general-purpose system designed to address this gap. CUB is a circular PCB sized for 4-inch-diameter enclosures that consolidates a Raspberry Pi Compute Module 5 (CM5) and an STM32 microcontroller, while also providing power management features and auxiliary connections. Mounted coaxially, CUB reduces the electronics stack of our CougUV from 200 mm of tube length and 709 g to 25 mm and 156 g, returning that length and mass budget to payload and buoyancy trim. CougarTail is an open-source companion sensor mast that houses a GPS
antenna and two dual-band (2.4 and 5 GHz) omnidirectional PCB
antennas. Both components are validated through bench testing and integration on a CougUV platform, our small open-sourced torpedo UUVs.
Read the full abstract ▾
All authors Ben Washburn, Clayton Smith, Eli Gaskin, Brighton Anderson, Braden Meyers, Brady Moon, Joshua Mangelson
Robotics
Adaptive Shared Control with Online Bounded-Rational Human Behavior Estimation
Henry Ascencio Trejo, Roel Pieters and Gokhan Alcan
This work considers adaptive shared human-
robot control for nonlinear control-affine systems, where the assumption of a fully rational human is relaxed and the
robot adapts its assistance to observed boundedly rational human behavior. We use a level-k bounded-rationality model of the two-player game to construct a finite bank of candidate human and
robot policies through alternating best-response computations, with the associated value functions and policies approximated using adaptive dynamic programming. During the shared-control interaction, state-transition residuals compare the measured system
evolution with the trajectories predicted by the candidate human policies. The residuals are accumulated using a forgetting factor and mapped to a probabilistic human-behavior model over the finite candidate bank. Rather than selecting a single candidate or averaging stored
robot policies, the
robot computes a distribution-aware one-step best response by minimizing an expected cooperative cost over the complete estimated human behavior distribution. For a quadratic terminal-value approximation and Euler state propagation, this response admits a closed-form solution expressed in terms of the expected human input. The proposed methods are evaluated in simulations of a
benchmark nonlinear system stabilization task, and of a planar manipulator shared control setup. The reported results show decreasing Kullback-Leibler divergence between the estimated and simulated human behavior distributions, and a lower accumulated running cost for the
robot agent over the shared control interaction period, than the maximum-probability and probability-weighted alternative policies baseline.
Read the full abstract ▾
Robotics · Cross-listed from eess.SY
IMU-Centric Moving Horizon Estimation for Lateral Dynamics Estimation Across Vehicles and Grip Conditions
Seuffo Akouan ha Ngoune, Alessandro Toschi, Paolo Burgio et al. (+1 more)
Accurate estimation of lateral vehicle dynamics near the adhesion limit is important for stability control and high-performance driving, but lateral velocity is rarely measured directly because sensors such as optical sensors are costly. This paper presents an inertial measurement unit (IMU)-centric Moving Horizon Estimation framework that reconstructs lateral velocity using standard onboard signals, without relying on exteroceptive odometry or detailed tire-parameter tuning. Experimental validation on human-driven sports cars and an autonomous open-wheel race car across tracks, maneuvers, and conditions demonstrates accurate and robust lateral velocity and lateral acceleration estimates. The proposed framework is available at https://github.com/Aseuffo/IMU-Centric-MHE
Read the full abstract ▾
All authors Seuffo Akouan ha Ngoune, Alessandro Toschi, Paolo Burgio, Marko Bertogna