Robotics · Cross-listed from cs.AI
Artificial Intelligence for Modeling and Simulation of Mixed Automated and Human Traffic
Saeed Rahmani, Shiva Rasouli, Daphne Cornelisse et al. (+3 more)
Autonomous vehicles (AVs) are now operating on public roads, which makes their testing and validation more critical than ever. Simulation offers a safe and controlled environment for evaluating AV performance in varied conditions. However, existing simulation tools mainly focus on graphical realism and rely on simple rule-based models and therefore fail to accurately represent the complexity of driving behaviors and interactions. Artificial intelligence (AI) has shown strong potential to address these limitations; however, despite the rapid progress across AI methodologies, a comprehensive survey of their application to mixed autonomy traffic simulation remains lacking. Existing surveys either focus on simulation tools without examining the AI methods behind them, or cover ego-centric decision-making without addressing the broader challenge of modeling surrounding traffic. Moreover, they do not offer a unified taxonomy of AI methods covering individual behavior modeling to full scene simulation. To address these gaps, this survey provides a structured review and synthesis of AI methods for modeling AV and human driving behavior in mixed autonomy traffic simulation. We introduce a taxonomy that organizes methods into three families:
agent-level behavior models, environment-level simulation methods, and cognitive and physics-informed methods. The survey analyzes how existing simulation platforms fall short of the needs of mixed autonomy research and outlines directions to narrow this gap. It also provides a chronological overview of AI methods and reviews evaluation protocols and metrics, simulation tools, and
datasets. By covering both traffic engineering and computer science perspectives, we aim to bridge the gap between these two communities.
Read the full abstract ▾
All authors Saeed Rahmani, Shiva Rasouli, Daphne Cornelisse, Eugene Vinitsky, Bart van Arem, Simeon C. Calvert
Robotics
Evolving the Complete Muscle: Efficient Morphology-Control Co-design for Musculoskeletal Locomotion
Lidong Sun, Wentao Zhao, Ye Wang et al. (+2 more)
Musculoskeletal
robots offer intrinsic compliance and flexibility, providing a promising paradigm for versatile locomotion. However, existing research typically relies on models with fixed muscle physiological parameters. This static physical setting fails to accommodate the diverse dynamic demands of complex tasks, inherently limiting the
robot's performance upper bound. In this work, we focus on the morphology and control co-design of musculoskeletal systems. Unlike previous studies that optimize single physiological attributes such as stiffness, we introduce a Complete Musculoskeletal Morphological
Evolution Space that simultaneously evolves muscle strength, velocity, and stiffness. To overcome the exponential expansion of the exploration space caused by this comprehensive
evolution, we propose Spectral Design
Evolution (SDE), a high-efficiency co-optimization framework. By integrating a bilateral
symmetry prior with Principal Component Analysis (PCA), SDE projects complex muscle parameters onto a low-dimensional spectral
manifold, enabling efficient morphological exploration. Evaluated on the MyoSuite framework across four tasks (Walk, Stair, Hilly, and Rough terrains), our method demonstrates superior learning efficiency and locomotion stability compared to fixed-morphology and standard evolutionary baselines.
Read the full abstract ▾
All authors Lidong Sun, Wentao Zhao, Ye Wang, Huaping Liu, Fuchun Sun
Robotics
PAINT: Partner-Agnostic Intent-Aware Cooperative Transport with Legged Robots
Zhihao Cao, Tianxu An, Chenhao Li et al. (+2 more)
Collaborative transport requires
robots to infer partner intent through physical interaction while maintaining stable loco-manipulation. This becomes particularly challenging in complex environments, where interaction signals are difficult to capture and model. We present PAINT, a hierarchical learning framework for partner-agnostic intent-aware collaborative legged transport that represents partner intent as an explicit interaction wrench and recovers it from payload-coupled proprioceptive histories. PAINT decouples intent understanding from terrain-robust locomotion: A high-level policy uses the inferred interaction wrench for transport and reconstructs it through teacher-student training, while a low-level locomotion backbone ensures robust execution. This enables lightweight deployment without external force-torque sensing or payload tracking. Extensive simulation and real-world experiments demonstrate compliant cooperative transport across diverse terrains, payloads, and partners. Furthermore, we show that PAINT can reuse the same single-
agent policy for decentralized multi-
robot transport by mechanically combining team interactions into a common wrench space, and supports heterogeneous-team transport. Our results suggest that payload-coupled proprioceptive interaction provides a physically grounded interface for partner-agnostic intent-aware collaborative transport.
Read the full abstract ▾
All authors Zhihao Cao, Tianxu An, Chenhao Li, Stelian Coros, Marco Hutter
Robotics
GGD-SLAM: Monocular 3DGS SLAM Powered by Generalizable Motion Model for Dynamic Environments
Yi Liu, Haoxuan Xu, Hongbo Duan et al. (+5 more)
Visual SLAM
algorithms achieve significant improvements through the exploration of 3D Gaussian Splatting (3DGS) representations, particularly in generating high-fidelity dense maps. However, they depend on a static environment assumption and experience significant performance degradation in dynamic environments. This paper presents GGD-SLAM, a framework that employs a generalizable motion model to address the challenges of localization and dense mapping in dynamic environments - without predefined semantic annotations or depth input. Specifically, the proposed system employs a First-In-First-Out (FIFO) queue to manage incoming frames, facilitating dynamic semantic feature extraction through a sequential
attention mechanism. This is integrated with a dynamic feature enhancer to separate static and dynamic components. Additionally, to minimize dynamic distractors' impact on the static components, we devise a method to fill occluded areas via static information sampling and design a distractor-adaptive Structure Similarity Index Measure (SSIM) loss tailored for dynamic environments, significantly enhancing the system's resilience. Experiments conducted on real-world dynamic
datasets demonstrate that the proposed system achieves state-of-the-art performance in camera pose estimation and dense reconstruction in dynamic scenes.
Read the full abstract ▾
All authors Yi Liu, Haoxuan Xu, Hongbo Duan, Keyu Fan, Zhengyang Zhang, Peiyu Zhuang, Pengting Luo, Houde Liu
Robotics
VULCAN: Vision-Language-Model Enhanced Multi-Agent Cooperative Navigation for Indoor Fire-Disaster Response
Shengding Liu and Qiben Yan
Indoor fire disasters pose severe challenges to autonomous search and rescue due to dense smoke, high temperatures, and dynamically evolving indoor environments. In such time-critical scenarios, multi-
agent cooperative
navigation is particularly useful, as it enables faster and broader exploration than single-
agent approaches. However, existing multi-
agent navigation systems are primarily
vision-based and designed for benign indoor settings, leading to significant performance degradation under fire-driven dynamic conditions. In this paper, we present VULCAN, a multi-
agent cooperative
navigation framework based on multi-modal perception and
vision-
language models (VLMs), tailored for indoor fire disaster response. We extend the Habitat-Matterport3D
benchmark by simulating physically realistic fire scenarios, including smoke
diffusion, thermal hazards, and sensor degradation. We evaluate representative multi-
agent cooperative
navigation baselines under both normal and fire-driven environments. Our results reveal critical failure modes of existing methods in fire scenarios and underscore the necessity of robust perception and hazard-aware planning for reliable multi-
agent search and rescue.
Read the full abstract ▾
Robotics
Actuation space reduction to facilitate insightful shape matching in a novel reconfigurable tendon driven continuum manipulator
Sabyasachi Dash, John Golden and Girish Krishnan
In tendon driven continuum manipulators (TDCMs), reconfiguring the tendon routing enables tailored spatial deformation of the backbone. This work presents a design in which tendons can be rerouted either prior to or after actuation by actively rotating the individual spacer disks. Each disk rotation thus adds a degree of freedom to the actuation space, complicating the mapping from a desired backbone curve to the corresponding actuator inputs. However, when the backbone shape is projected into an intermediate space defined by
curvature and torsion (C-T), patterns emerge that highlight which disks are most influential in achieving a global shape. This insight enables a simplified, sequential shape-matching strategy: first, the proximal and intermediate disks are rotated to approximate the global shape; then, the distal disks are adjusted to fine-tune the end-effector position with minimal impact on the overall shape. The proposed actuation framework offers a model-free alternative to conventional control approaches, bypassing the complexities of modeling reconfigurable TDCMs.
Read the full abstract ▾
Robotics
Reliability-Guided RGB-D Sensor Fusion for Glare-Resilient Navigation Costmaps
Shang-En Tsai and Wei-Cheng Sun
Specular glare on reflective floors, glass boundaries, and glossy indoor surfaces can corrupt active-stereo RGB-D measurements, producing holes and spikes that persist as phantom obstacles in
navigation costmaps. This article presents a glare-resilient RGB-D sensor-fusion method based on explicit per-pixel depth reliability. A lightweight Depth Reliability Map network (DRM-Net) predicts sensor trustworthiness, and reliability-guided fusion (RGF) combines continuous weighting with a minimum gate before occupancy integration. Rejected measurements generate neither obstacle insertion nor free-space clearing; affected
cells remain unknown or retain prior evidence. Training targets are built from a five-frame, pose-aligned multiview buffer using independent
LiDAR/AMCL poses, occlusion-aware aggregation, and a calibrated range-dependent depth-uncertainty model. The evaluation includes tuned nvblox TSDF, Intel RealSense SDK postprocessing, high-threshold stereoconfidence filtering, TDCNet, and HDCNet baselines, together with statistical, safety, generalization, and embedded-runtime analyses. Under severe glare, Depth Reliability Map (DRM)+RGF achieves false obstacle rate (FOR) 0.056 +/- 0.012, free-space recall (FSR) 0.897 +/- 0.045, FNOR 0.018, 1.00 +/- 0.00 degraded-mode safety interventions per 10 m, and 91.4% task success while operating at 16.5 ms per frame. Across the retained reflective-scene trials (Baseline N = 98; DRM+RGF N = 105), collisions decrease from 14 to 1. These results support RGF as a favorable safety-utility tradeoff relative to aggressive filtering and dense completion for glare-affected indoor
navigation.
Read the full abstract ▾
Robotics
FeaXDrive: Feasibility-aware Trajectory-Centric Diffusion Planning for End-to-End Autonomous Driving
Baoyun Wang, Zhuoren Li, Ran Yu et al. (+6 more)
End-to-end
diffusion planning has shown strong potential for autonomous driving, but the physical feasibility of generated trajectories remains insufficiently addressed. In particular, generated trajectories may exhibit local
geometric irregularities, violate trajectory-level kinematic constraints, or deviate from the drivable area, indicating that the commonly used noise-centric formulation in
diffusion planning is not yet well aligned with the trajectory space where feasibility is more naturally characterized. To address this issue, we propose FeaXDrive, a feasibility-aware trajectory-centric
diffusion planning method for end-to-end autonomous driving. The core idea is to treat the clean trajectory as the unified object for feasibility-aware modeling throughout the
diffusion process. Built on this trajectory-centric formulation, FeaXDrive integrates adaptive
curvature-constrained training to improve intrinsic
geometric and kinematic feasibility, drivable-area guidance within reverse
diffusion sampling to enhance consistency with the drivable area, and feasibility-aware GRPO post-training to further improve planning performance while balancing trajectory-space feasibility. Experiments on the NAVSIM
benchmark show that FeaXDrive achieves strong closed-loop planning performance while substantially improving trajectory-space feasibility. These findings highlight the importance of explicitly modeling trajectory-space feasibility in end-to-end
diffusion planning and provide a step toward more reliable and physically grounded autonomous driving planners.
Read the full abstract ▾
All authors Baoyun Wang, Zhuoren Li, Ran Yu, Yu Che, Xinrui Zhang, Ming Liu, Jia Hu, Chen Lv, Bo Leng
Robotics
Contextual Multi-Task Reinforcement Learning for Autonomous Reef Monitoring
Melvin Laux, Yi-Ling Liu, Rina Alo et al. (+4 more)
Although autonomous
underwater vehicles promise the capability of marine ecosystem monitoring, their deployment is fundamentally limited by the difficulty of controlling vehicles under highly uncertain and non-stationary
underwater dynamics. To address these challenges, we employ a data-driven reinforcement learning approach to compensate for unknown dynamics and task variations. Traditional single-task reinforcement learning has a tendency to overfit the training environment, thus, limit the long-term usefulness of the learnt policy. Hence, we propose to use a contextual multi-task reinforcement learning paradigm instead, allowing us to learn controllers that can be reused for various tasks, e.g., detecting oysters in one reef and detecting corals in another. We evaluate whether contextual multi-task reinforcement learning can efficiently learn robust and generalisable control policies for autonomous
underwater reef monitoring. We train a single context-dependent policy that is able to solve multiple related monitoring tasks in a simulated reef environment in HoloOcean. In our experiments, we empirically evaluate the contextual policies regarding sample-efficiency, zero-shot generalisation to unseen tasks, and robustness to varying water currents. By utilising multi-task reinforcement learning, we aim to improve the training effectiveness, as well as the reusability of learnt policies to take a step towards more sustainable procedures in autonomous reef monitoring.
Read the full abstract ▾
All authors Melvin Laux, Yi-Ling Liu, Rina Alo, Sören Töpper, Mariela De Lucas Alvarez, Frank Kirchner, Rebecca Adam
Robotics · Cross-listed from math.OC
A Comparison of Reinforcement Learning and Optimal Control Methods for Path Planning
Qiang Le, Yaguang Yang and Isaac E. Weintraub
Path-planning for autonomous vehicles in threat-laden environments is a fundamental challenge. While traditional optimal control methods can find ideal paths, the computational time is often too slow for real-time decision-making. To solve this challenge, we propose a method based on Deep Deterministic Policy Gradient (DDPG) and model the threat as a simple, circular `no-go' zone. A mission failure is claimed if the vehicle enters this `no-go' zone at any time or does not reach a neighborhood of the destination. The DDPG
agent is trained to learn a direct mapping from its current state (position and velocity) to a series of feasible actions that guide the
agent to safely reach its goal. A reward function and two neural networks, critic and actor, are used to describe the environment and guide the control efforts. The DDPG trains the
agent to find the largest possible set of starting points (“feasible set”) wherein a safe path to the goal is guaranteed. This provides critical information for mission planning, showing beforehand whether a task is achievable from a given starting point, assisting pre-mission planning activities. The approach is validated in simulation. A comparison between the DDPG method and a traditional optimal control (pseudo-spectral) method is carried out. The results show that the learning-based
agent may produce effective paths while being significantly faster, making it a better fit for real-time applications. However, there are areas (“infeasible set”) where the DDPG
agent cannot find paths to the destination, and the paths in the feasible set may not be optimal. These preliminary results guide our future research: (1) improve the reward function to enlarge the DDPG feasible set, (2) examine the feasible set obtained by the pseudo-spectral method, and (3) investigate the arc-search IPM method for the path planning problem.
Read the full abstract ▾
Robotics
Habitat-GS: A High-Fidelity Navigation Simulator with Dynamic Gaussian Splatting
Ziyuan Xia, Jingyi Xu, Chong Cui et al. (+9 more)
Training
embodied AI
agents depends critically on the visual fidelity of simulation environments and the ability to model dynamic humans. Current simulators predominantly rely on mesh-based rasterization, for which photorealistic assets are costly to author at scale, and their support for dynamic human avatars is largely constrained to mesh representations, hindering
agent generalization to human-populated real-world scenarios. We present Habitat-GS, a
navigation-centric
embodied AI simulator extended from Habitat-Sim that integrates 3D Gaussian Splatting scene rendering and drivable gaussian avatars while maintaining full compatibility with the Habitat ecosystem. Our system implements a 3DGS renderer for real-time photorealistic rendering and supports scalable 3DGS asset import from diverse sources. For dynamic human modeling, we introduce a gaussian avatar module that enables each avatar to simultaneously serve as a photorealistic visual entity and an effective
navigation obstacle, allowing
agents to learn human-aware behaviors in realistic settings. Experiments on point-goal
navigation demonstrate that
agents trained on 3DGS scenes achieve stronger cross-domain generalization. Evaluations on avatar-aware
navigation further confirm that gaussian avatars enable effective human-aware
navigation, while performance
benchmarks validate the system's scalability. Code is available at https://github.com/zju3dv/habitat-gs.
Read the full abstract ▾
All authors Ziyuan Xia, Jingyi Xu, Chong Cui, Yuanhong Yu, Jiazhao Zhang, Qingsong Yan, Tao Ni, Junbo Chen, Xiaowei Zhou, Hujun Bao, Ruizhen Hu, Sida Peng
Robotics
Machine Learning-Based Real-Time Detection of Compensatory Trunk Movements Using Trunk-Wrist Inertial Measurement Units
Jannis Gabler, Clément Lhoste, Max Quast et al. (+5 more)
Compensatory trunk movements (CTMs) are commonly observed after stroke and can lead to maladaptive movement patterns, limiting targeted training of affected structures. Objective, continuous detection of CTMs during therapy and activities of daily living remains challenging due to the typically complex measurements setups required, as well as limited applicability for real-time use. This study investigates whether a two-inertial measurement unit configuration enables reliable, real-time CTM detection using machine learning. Data were collected from ten able-bodied participants performing activities of daily living under simulated impairment conditions (elbow brace restricting flexion-extension, resistance band inducing flexor-synergy-like patterns), with synchronized optical motion capture (OMC) and manually annotated
video recordings serving as reference. A systematic location-reduction analysis using OMC identified wrist and trunk kinematics as a minimal yet sufficient set of anatomical sensing locations. Using an extreme gradient boosting classifier (XGBoost) evaluated with leave-one-subject-out cross-validation, our two-IMU model achieved strong discriminative performance (macro-F1 = 0.80 +/- 0.07, MCC = 0.73 +/- 0.08; ROC-AUC > 0.93), with performance comparable to an OMC-based model and prediction timing suitable for real-time applications. Explainability analysis revealed dominant contributions from trunk dynamics and wrist-trunk interaction features. In preliminary evaluation using recordings from four participants with neurological conditions, the model retained good discriminative capability (ROC-AUC ~ 0.78), but showed reduced and variable threshold-dependent performance, highlighting challenges in clinical generalization. These results support sparse wearable sensing as a viable pathway toward scalable, real-time monitoring of CTMs during therapy and daily living.
Read the full abstract ▾
All authors Jannis Gabler, Clément Lhoste, Max Quast, Laura Mayrhuber, Andrea Ronco, Olivier Lambercy, Paulius Viskaitis, Dane Donegan
Robotics
Scalable Trajectory Generation for Whole-Body Mobile Manipulation
Yida Niu, Xinhai Chang, Xin Liu et al. (+2 more)
Robots deployed in unstructured environments must coordinate whole-body motion – simultaneously moving a mobile base and arm – to interact with the physical world. This coupled mobility and dexterity yields a state space that grows combinatorially with scene and object diversity, demanding
datasets far larger than those sufficient for fixed-base manipulation. Yet existing acquisition methods, including teleoperation and planning, are either labor-intensive or computationally prohibitive at scale. The core bottleneck is the lack of a scalable pipeline for generating large-scale, physically valid, coordinated trajectory data across diverse embodiments and environments. Here we introduce AutoMoMa, a GPU-accelerated framework that unifies AKR modeling, which consolidates base, arm, and object kinematics into a single chain, with parallelized trajectory optimization. AutoMoMa achieves 5,000 episodes per GPU-hour (over $80\times$ faster than CPU-based baselines), producing a
dataset of over 500k physically valid trajectories spanning 330 scenes, diverse articulated objects, and multiple
robot embodiments. Prior
datasets were forced to compromise on scale, diversity, or kinematic fidelity; AutoMoMa addresses all three simultaneously. Training downstream IL policies further reveals that even a single articulated-object task requires tens of thousands of demonstrations for SOTA methods to reach $\approx 80\%$ success, confirming that data scarcity – not algorithmic limitations – has been the binding constraint. AutoMoMa thus bridges high-performance planning and reliable IL-based control, providing the infrastructure previously missing for coordinated mobile manipulation research. By making large-scale, kinematically valid training data practical, AutoMoMa showcases generalizable whole-body
robot policies capable of operating in the diverse, unstructured settings of the real world.
Read the full abstract ▾
All authors Yida Niu, Xinhai Chang, Xin Liu, Ziyuan Jiao, Yixin Zhu
Robotics
Whole-Body Mobile Manipulation using Offline Reinforcement Learning on Sub-optimal Controllers
Snehal Jauhri, Vignesh Prasad and Georgia Chalvatzaki
Mobile Manipulation (MoMa) of articulated objects, such as opening doors, drawers, and cupboards, demands simultaneous, whole-body coordination between a
robot's base and arms. Classical whole-body controllers (WBCs) can solve such problems via hierarchical optimization, but require extensive hand-tuned optimization and remain brittle. Learning-based methods, on the other hand, show strong generalization capabilities but typically rely on expensive whole-body teleoperation data or heavy reward engineering. We observe that even a sub-optimal WBC is a powerful structural prior: it can be used to collect data in a constrained, task-relevant region of the state-action space, and its behavior can still be improved upon using offline reinforcement learning. Building on this, we propose WHOLE-MoMa, a two-stage pipeline that first generates diverse demonstrations by randomizing a lightweight WBC, and then applies offline RL to identify and stitch together improved behaviors via a reward signal. To support the expressive action-chunked
diffusion policies needed for complex coordination tasks, we extend offline implicit Q-learning with Q-chunking for chunk-level critic evaluation and advantage-weighted policy extraction. On three tasks of increasing difficulty using a TIAGo++ mobile manipulator in simulation, WHOLE-MoMa significantly outperforms WBC, behavior cloning, and several offline RL baselines. Policies transfer directly to the real
robot without finetuning, achieving 80% success in bimanual drawer manipulation and 68% in simultaneous cupboard opening and object placement, all without any teleoperated or real-world training data.
Read the full abstract ▾
Robotics
DeCoNav: Dialog enhanced Long-Horizon Collaborative Vision-Language Navigation
Sunyao Zhou, Yunzi Wu, Tianhang Wang et al. (+5 more)
Long-horizon collaborative
vision-
language navigation (VLN) is critical for multi-
robot systems to accomplish complex tasks beyond the capability of a single
agent. CoNavBench takes a first step by introducing the first collaborative long-horizon VLN
benchmark with relay-style multi-
robot tasks, a collaboration taxonomy, along with graph-grounded generation and evaluation to model handoffs and rendezvous in shared environments. However, existing
benchmarks and evaluations often do not enforce strictly synchronized dual-
robot rollout on a shared world timeline, and they typically rely on static coordination policies that cannot adapt when new cross-
agent evidence emerges. We present Dialog enhanced Long-Horizon Collaborative
Vision-
Language Navigation (DeCoNav), a decentralized framework that couples event-triggered dialogue with dynamic task allocation and replanning for real-time, adaptive coordination. In DeCoNav,
robots exchange compact semantic states via dialogue without a central controller. When informative events such as new evidence, uncertainty, or conflicts arise, dialogue is triggered to dynamically reassign subgoals and replan under synchronized execution. Implemented in DeCoNavBench with 1,213 tasks across 176 HM3D scenes, DeCoNav improves the both-success rate (BSR) by 69.2%, demonstrating the effectiveness of dialogue-driven, dynamically reallocated planning for multi-
robot collaboration.
Read the full abstract ▾
All authors Sunyao Zhou, Yunzi Wu, Tianhang Wang, Xinhai Li, Guang Chen, Lizheng Liu, Chenjia Bai, Xuelong Li
Robotics
Social Learning Strategies for Evolved Virtual Soft Robots
K. Ege de Bruin, Kyrre Glette, Kai Olav Ellefsen et al. (+2 more)
Optimizing the body and
brain of a
robot is a coupled challenge: the morphology determines what control strategies are effective, while the control parameters influence how well the morphology performs. This joint optimization can be done through nested loops of evolutionary and learning processes, where the control parameters of each
robot are learned independently. However, the control parameters learned by one
robot may contain valuable information for others. Thus, we introduce a social learning approach in which
robots can exploit optimized parameters from their peers to accelerate their own
brain optimization. Within this framework, we systematically investigate how the selection of teachers, deciding which and how many
robots to learn from, affects performance, experimenting with virtual soft
robots in four tasks and environments. In particular, we study the effect of inheriting experience from morphologically similar
robots due to the tightly coupled body and
brain in
robot optimization. Our results confirm the effectiveness of building on others' experience, as social learning clearly outperforms learning from scratch under equivalent computational budgets. In addition, while the optimal teacher selection strategy remains open, our findings suggest that incorporating knowledge from multiple teachers can yield more consistent and robust improvements.
Read the full abstract ▾
All authors K. Ege de Bruin, Kyrre Glette, Kai Olav Ellefsen, Giorgia Nadizar, Eric Medvet
Robotics
From Kinematics to Dynamics: Learning to Refine Hybrid Plans for Physically Feasible Execution
Lidor Erez, Shahaf S. Shperberg and Ayal Taitler
In many robotic tasks,
agents must traverse a sequence of spatial regions to complete a mission. Such problems are inherently mixed discrete-continuous: a high-level action sequence and a physically feasible continuous trajectory. The resulting trajectory and action sequence must also satisfy problem constraints such as deadlines, time windows, and velocity or acceleration limits. While hybrid temporal planners attempt to address this challenge, they typically model motion using linear (first-order) dynamics, which cannot guarantee that the resulting plan respects the
robot's true physical constraints. Consequently, even when the high-level action sequence is fixed, producing a dynamically feasible trajectory becomes a bi-level optimization problem. We address this problem via reinforcement learning in continuous space. We define a Markov Decision Process that explicitly incorporates analytical second-order constraints and use it to refine first-order plans generated by a hybrid planner. Our results show that this approach can reliably recover physical feasibility and effectively bridge the gap between a planner's initial first-order trajectory and the dynamics required for real execution.
Read the full abstract ▾
Robotics
Designing for Error Recovery in Human-Robot Interaction
Christopher D. Wallbridge and Erwin Jose Lopez Pulgarin
This position paper looks briefly at the way we attempt to program robotic AI systems. Many AI systems are based on the idea of trying to improve the performance of one individual system to beyond so-called human baselines. However, these systems often look at one shot and one-way decisions, whereas the real world is more continuous and interactive. Humans, however, are often able to recover from and learn from errors - enabling a much higher rate of success. We look at the challenges of building a system that can detect/recover from its own errors, using the example of robotic
nuclear gloveboxes as a use case to help illustrate examples. We then go on to talk about simple starting designs.
Read the full abstract ▾
Robotics
HazardArena: Evaluating Semantic Safety in Vision-Language-Action Models
Zixing Chen, Yifeng Gao, Li Wang et al. (+8 more)
Vision-
Language-Action (VLA) models inherit rich world knowledge from
vision-
language backbones and acquire executable skills from action demonstrations. Yet current evaluations primarily measure task completion, leaving the semantic safety of learned action policies underexplored. This gap creates a critical vulnerability: a policy may execute the intended action correctly while producing unsafe outcomes when the surrounding visual-linguistic context changes. We present HazardArena, a
benchmark for stress-testing semantic safety in VLA systems. Its core design is a set of safe/unsafe twin scenarios: paired environments with matched objects, layouts, and action requirements, but different semantic risk contexts. This controlled contrast isolates safety judgment from motor capability and directly tests whether a VLA can recognize when an otherwise valid action becomes hazardous. HazardArena includes over 2,000 assets and 51 risk-sensitive tasks across seven safety categories grounded in robotic safety standards. Across four representative VLA backbones, we observe a consistent and alarming pattern: safe-only fine-tuning improves benign task success while also increasing hazardous execution on matched unsafe scenarios. Physical-world experiments confirm that this failure transfers beyond simulation. These results show that stronger action execution does not imply safer behavior, and motivate semantic-risk-aware evaluation and enforcement as first-class requirements for real-world VLA deployment. Code released at https://github.com/HazardArena-Team/HazardArena ; updated code availability information.
Read the full abstract ▾
All authors Zixing Chen, Yifeng Gao, Li Wang, Yunhan Zhao, Yi Liu, Jiayu Li, Xiang Zheng, Zuxuan Wu, Cong Wang, Xingjun Ma, Yu-Gang Jiang
Robotics
D-BDM: A Direct and Efficient Boundary-Based Occupancy Grid Mapping Framework for LiDARs
Benxu Tang, Yixi Cai, Fanze Kong et al. (+2 more)
Efficient and scalable 3D occupancy mapping is essential for autonomous
robot applications in unknown environments. However, traditional occupancy grid representations suffer from two fundamental limitations. First, explicitly storing all voxels in three-dimensional space leads to prohibitive
memory consumption. Second, exhaustive ray casting incurs high update latency. A recent representation alleviate
memory demands by maintaining only the voxels on the two-dimensional boundary, yet they still rely on full ray casting updates. This work advances the boundary-based framework with a highly efficient update scheme. We introduce a truncated ray casting strategy that restricts voxel traversal to the exterior of the boundary, which dramatically reduces the number of updated voxels. In addition, we propose a direct boundary update mechanism that removes the need for an auxiliary local 3D occupancy grid, further reducing
memory usage and simplifying the map update pipeline. We name our framework as D-BDM. Extensive evaluations on public
datasets demonstrate that our approach achieves significantly lower update time and reduced
memory consumption compared with the baseline methods, as well as the prior boundary-based approach.
Read the full abstract ▾
All authors Benxu Tang, Yixi Cai, Fanze Kong, Longji Yin, Fu Zhang
Robotics
RACF: A Resilient Autonomous Car Framework with Object Distance Correction
Chieh Tsai, Hossein Rastgoftar and Salim Hariri
Autonomous vehicles are increasingly deployed in safety-critical applications, where sensing failures or cyberphysical attacks can lead to unsafe operations resulting in human loss and/or severe physical damages. Reliable real-time perception is therefore critically important for their safe operations and acceptability. For example,
vision-based distance estimation is vulnerable to environmental degradation and
adversarial perturbations, and existing defenses are often reactive and too slow to promptly mitigate their impacts on safe operations. We present a Resilient Autonomous Car Framework (RACF) that incorporates an Object Distance Correction
Algorithm (ODCA) to improve perception-layer robustness through redundancy and diversity across a depth camera,
LiDAR, and physics-based kinematics. Within this framework, when obstacle distance estimation produced by depth camera is inconsistent, a cross-sensor gate activates the correction
algorithm to fix the detected inconsistency. We have experiment with the proposed resilient car framework and evaluate its performance on a testbed implemented using the Quanser QCar 2 platform. The presented framework achieved up to 35% RMSE reduction under strong corruption and improves stop compliance and braking latency, while operating in real time. These results demonstrate a practical and lightweight approach to resilient perception for safety-critical autonomous driving
Read the full abstract ▾
Robotics · Cross-listed from physics.flu-dyn
Learning step-level dynamic soaring in shear flow
Lunbing Chen, Jixin Lu, Yufei Yin et al. (+3 more)
Dynamic soaring enables sustained flight by extracting energy from wind shear, yet it is commonly understood as a cycle-level maneuver that assumes stable flow conditions. In realistic unsteady environments, however, such assumptions are often violated, raising the question of whether explicit cycle-level planning is necessary. Here, we show that dynamic soaring can emerge from step-level, state-feedback control using only local sensing, without explicit trajectory planning. Using deep reinforcement learning as a tool, we obtain policies that achieve robust omnidirectional
navigation across diverse shear-flow conditions. The learned behavior organizes into a structured control law that coordinates turning and vertical motion, giving rise to a two-phase strategy governed by a trade-off between energy extraction and directional progress. The resulting policy generalizes across varying conditions and reproduces key features observed in biological flight and optimal-control solutions. These findings identify a feedback-based control structure underlying dynamic soaring, demonstrating that efficient energy-harvesting flight can emerge from local interactions with the flow without explicit planning, and providing insights for biological flight and autonomous systems in complex, flow-coupled environments.
Read the full abstract ▾
All authors Lunbing Chen, Jixin Lu, Yufei Yin, Jinpeng Huang, Yang Xiang, Hong Liu
Robotics
Defining an Evaluation Method for External Human-Machine Interfaces
Jose Gonzalez-Belmonte and Jaerock Kwon
As the number of fatalities involving Autonomous Vehicles increase, the need for a universal method of communicating between vehicles and other
agents on the road has also increased. Over the past decade, numerous proposals of external Human-Machine Interfaces (eHMIs) have been brought forward with the purpose of bridging this communication gap, with none yet to be determined as the ideal one. This work proposes a universal evaluation method conformed of 223 questions to objectively evaluate and compare different proposals and arrive at a conclusion. The questionnaire is divided into 7 categories that evaluate different aspects of any given proposal that uses eHMIs: ease of standardization, cost effectiveness, accessibility, ease of understanding, multifacetedness in communication, positioning, and readability. In order to test the method it was used on four existing proposals, plus a baseline using only kinematic motions, in order to both exemplify the application of the evaluation method and offer a baseline score for future comparison. The result of this testing suggests that the ideal method of machine-human communication is a combination of intentionally-designed vehicle kinematics and distributed well-placed text-based displays, but it also reveals knowledge gaps in the readability of eHMIs and the speed at which different observers may learn their meaning. This paper proposes future work related to these uncertainties, along with future testing with the proposed method.
Read the full abstract ▾
Robotics
Asymptotically Stable Gait Generation and Instantaneous Walkability Determination for Planar Almost Linear Biped with Knees
Fumihiko Asano, Ning Lei and Taiki Sedoguchi
A class of planar bipedal
robots with unique mechanical properties has been proposed, where all links are balanced around the hip joint, preventing natural swinging motion due to
gravity. A common property of their equations of motion is that the inertia
matrix is a constant
matrix, there are no nonlinear velocity terms, and the
gravity term contains simple nonlinear terms. By performing a Taylor expansion of the
gravity term and making a linear approximation, it is easy to derive a linearized model, and calculations for future states or walkability determination can be performed instantaneously without the need for numerical integration. This paper extends the method to a planar biped
robot model with knees. First, we derive the equations of motion, constraint conditions, and inelastic collisions for a planar 6-DOF biped
robot, design its control system, and numerically generate a stable bipedal gait on a horizontal plane. Next, we reduce the equations of motion to a 3-DOF model, and derive a linearized model by approximating the
gravity term as linear around the expansion point for the thigh frame angle. Through numerical simulations, we demonstrate that calculations for future states and walkability determination can be completed in negligible time. By applying control inputs to the obtained model, performing state-space realization, and then discretizing it, instantaneous walkability determination through iterative calculation becomes possible. Through detailed gait analysis, we discuss how the knee joint flexion angle and the expansion point affect the accuracy of the linear approximation, and the issues that arise when descending a small step.
Read the full abstract ▾
Robotics
Unveiling the Surprising Efficacy of Navigation Understanding in End-to-End Autonomous Driving
Zhihua Hua, Junli Wang, Pengfei LI et al. (+6 more)
Global
navigation information and local scene understanding are two crucial components of autonomous driving systems. However, our experimental results indicate that many end-to-end autonomous driving systems tend to over-rely on local scene understanding while failing to utilize global
navigation information. These systems exhibit weak correlation between their planning capabilities and
navigation input, and struggle to perform
navigation-following in complex scenarios. To overcome this limitation, we propose the Sequential
Navigation Guidance (SNG) framework, an efficient representation of global
navigation information based on real-world
navigation patterns. The SNG encompasses both
navigation paths for constraining long-term trajectories and turn-by-turn (TBT) information for real-time decision-making logic. We constructed the SNG-QA
dataset, a visual question answering (VQA)
dataset based on SNG that aligns global and local planning. Additionally, we introduce an efficient model SNG-VLA that fuses local planning with global planning. The SNG-VLA achieves state-of-the-art performance through precise
navigation information modeling without requiring auxiliary loss functions from perception tasks. Project page: SNG-VLA
Read the full abstract ▾
All authors Zhihua Hua, Junli Wang, Pengfei LI, Qihao Jin, Bo Zhang, Kehua Sheng, Yilun Chen, Zhongxue Gan, Wenchao Ding
Robotics
Robotic Nanoparticle Synthesis via Solution-based Processes
Dasharadhan Mahalingam, Michael Gallagher, Nilanjan Chakraborty et al. (+1 more)
We present a screw geometry-based manipulation planning framework for the robotic automation of solution-based synthesis, exemplified through the preparation of gold and magnetite nanoparticles. The synthesis protocols are inherently long-horizon, multi-step tasks, requiring skills such as pick-and-place, pouring, turning a knob, and periodic visual inspection to detect reaction completion. A central challenge is that some skills, notably pouring, transferring containers with solutions, and turning a knob, impose
geometric and kinematic constraints on the end-effector motion. To address this, we use a programming by demonstration paradigm where the constraints can be extracted from a single demonstration. This combination of screw-based motion representation and demonstration-driven specification enables domain experts, such as chemists, to readily adapt and reprogram the system for new experimental protocols and laboratory setups without requiring expertise in robotics or motion planning. We extract sequences of constant screws from demonstrations, which compactly encode the motion constraints while remaining coordinate-invariant. This representation enables robust generalization across variations in grasp placement and allows parameterized reuse of a skill learned from a single example. By composing these screw-parameterized primitives according to the synthesis protocol, the
robot autonomously generates motion plans that execute the complete experiment over repeated runs. Our results highlight that screw-theoretic planning, combined with programming by demonstration, provides a rigorous and generalizable foundation for long-horizon laboratory automation, thereby enabling fundamental kinematics to have a translational impact on the use of
robots in developing scalable solution-based synthesis protocols.
Read the full abstract ▾
All authors Dasharadhan Mahalingam, Michael Gallagher, Nilanjan Chakraborty, Stanislaus S. Wong