Robotics
Inchworm-Inspired Soft Robot with Groove-Guided Locomotion
Hari Prakash Thanabalan, Lars Bengtsson, Ugo Lafont et al. (+1 more)
Soft
robots require directional control to navigate complex terrains. However, achieving such control often requires multiple actuators, which increases mechanical complexity, complicates control systems, and raises energy consumption. Here, we introduce an inchworm-inspired soft
robot whose locomotion direction is controlled passively by patterned substrates. The
robot employs a single rolled dielectric elastomer actuator, while groove patterns on a 3D-printed substrate guide its
alignment and trajectory. Through systematic experiments, we demonstrate that varying groove angles enables precise control of locomotion direction without the need for complex actuation strategies. This groove-guided approach reduces energy consumption, simplifies
robot design, and expands the applicability of bio-inspired soft
robots in fields such as search and rescue, pipe inspection, and planetary exploration.
Read the full abstract ▾
All authors Hari Prakash Thanabalan, Lars Bengtsson, Ugo Lafont, Giovanni Volpe
Robotics
David Thorne, Nathan Chan, Christa S. Robison et al. (+2 more)
Autonomous
robots rely on
geometric maps to inform a diverse set of perception and decision-making
algorithms. As autonomy requires
reasoning and planning on multiple scales, each
algorithm may require a different map for optimal performance.
LiDAR sensors generate an abundance of
geometric data (up to 50 MB per second) to satisfy these diverse requirements. However, the point-based operations required to process perception data are both
memory and computationally expensive. Such operations can be bypassed via learned representations that encode similarity, but selecting informative, size-constrained maps remains an NP-hard combinatorial problem. In this work we present OptMap: a
geometric map
distillation algorithm which achieves online, application-specific map generation via multiple theoretical and algorithmic innovations. A central feature is the maximization of set functions that exhibit diminishing returns, i.e., submodularity, using
polynomial-time
algorithms with provably near-optimal solutions. We formulate a novel submodular reward function which quantifies informativeness, reduces input set sizes, and minimizes solution bias. Further, we propose a dynamically reordered streaming submodular
algorithm which improves empirical solution quality and addresses input order bias via an online approximation of the value of all scans. Testing was conducted on open-source and custom
datasets with an emphasis on long-duration mapping sessions, highlighting OptMap's minimal computation requirements. OptMap's practical value is then illustrated through its application to online
geometric change detection. Open-source ROS1 and ROS2 packages are available and can be used alongside any
LiDAR odometry
algorithm.
Read the full abstract ▾
All authors David Thorne, Nathan Chan, Christa S. Robison, Philip R. Osteen, Brett T. Lopez
Robotics
Toward Seamless Physical Human-Humanoid Interaction: Insights from Control, Intent, and Modeling with a Vision for What Comes Next
Gustavo A. Cardona, Shubham S. Kumbhar and Panagiotis Artemiadis
Physical Human-
Humanoid Interaction (pHHI) is a rapidly advancing field with significant implications for deploying
robots in unstructured, human-centric environments. In this review, we examine the current state of the art in pHHI through three core pillars: (i)
humanoid modeling and control, (ii) human intent estimation, and (iii) computational human models. For each pillar, we survey representative approaches, identify open challenges, and analyze current limitations that hinder robust, scalable, and adaptive interaction. These include the need for whole-body control strategies capable of handling uncertain human dynamics, real-time intent inference under limited sensing, and modeling techniques that account for variability in human physical states. Although significant progress has been made within each domain, integration across pillars remains limited. We propose pathways for unifying methods across these areas to enable cohesive interaction frameworks. This structure enables us not only to map the current landscape but also to propose concrete directions for future research that aim to bridge these domains. Additionally, we introduce a unified taxonomy of interaction types based on modality, distinguishing between direct interactions (e.g., physical contact) and indirect interactions (e.g., object-mediated), and on the level of
robot engagement, ranging from assistance to cooperation and collaboration. For each category in this taxonomy, we provide the three core pillars that highlight opportunities for cross-pillar unification. Our goal is to suggest avenues to advance robust, safe, and intuitive physical interaction, providing a roadmap for future research that will allow
humanoid systems to effectively understand, anticipate, and collaborate with human partners in diverse real-world settings.
Read the full abstract ▾
Robotics · Cross-listed from cs.CV
UltrasODM: A Dual Stream Optical Flow Mamba Network for 3D Freehand Ultrasound Reconstruction
Mayank Anand, Ujair Alam, Surya Prakash et al. (+3 more)
Clinical ultrasound acquisition is highly operator-dependent, where rapid probe motion and brightness fluctuations often lead to reconstruction errors that reduce trust and clinical utility. We present UltrasODM, a dual-stream framework that assists sonographers during acquisition through calibrated per-frame uncertainty, saliency-based diagnostics, and actionable
prompts. UltrasODM integrates (i) a contrastive ranking module that groups frames by motion similarity, (ii) an optical-flow stream fused with Dual-Mamba temporal modules for robust 6-DoF pose estimation, and (iii) a Human-in-the-Loop (HITL) layer combining
Bayesian uncertainty, clinician-calibrated thresholds, and saliency maps highlighting regions of low confidence. When uncertainty exceeds the threshold, the system issues unobtrusive alerts suggesting corrective actions such as re-scanning highlighted regions or slowing the sweep. Evaluated on a clinical freehand ultrasound
dataset, UltrasODM reduces drift by 15.2%, distance error by 12.1%, and Hausdorff distance by 10.1% relative to UltrasOM, while producing per-frame uncertainty and saliency outputs. By emphasizing transparency and clinician feedback, UltrasODM improves reconstruction reliability and supports safer, more trustworthy clinical workflows. Our code is publicly available at https://github.com/AnandMayank/UltrasODM.
Read the full abstract ▾
All authors Mayank Anand, Ujair Alam, Surya Prakash, Priya Shukla, Gora Chand Nandi, Domenec Puig
Robotics · Cross-listed from cs.CV
sim2art: Accurate Articulated Object Modeling from a Single Video using Synthetic Training Data Only
Arslan Artykov, Tom Ravaud, Corentin Sautier et al. (+1 more)
Understanding articulated objects from
monocular video is a crucial yet challenging task in robotics and digital twin creation. Existing methods often rely on complex multi-view setups, high-fidelity object scans, or fragile long-term point tracks that frequently fail in casual real-world captures. In this paper, we present sim2art, a data-driven framework that recovers the 3D part
segmentation and joint parameters of articulated objects from a single
monocular video captured by a freely moving camera. Our core insight is a robust representation based on per-frame surface point sampling, which we augment with short-term scene flow and DINOv3 semantic features. Unlike previous works that depend on error-prone long-term correspondences, our representation is easy to obtain and exhibits a negligible difference between simulation and reality without requiring domain adaptation. Also, by construction, our method relies on single-viewpoint visibility, ensuring that the
geometric representation remains consistent across synthetic and real data despite noise and occlusions. Leveraging a suitable
Transformer-based architecture, sim2art is trained exclusively on synthetic data yet generalizes strongly to real-world sequences. To address the lack of standardized
benchmarks in the field, we introduce two
datasets featuring a significantly higher diversity of object categories and instances than prior work. Our evaluations show that sim2art effectively handles large camera motions and complex articulations, outperforming state-of-the-art optimization-based and tracking-dependent methods. sim2art offers a scalable solution that can be easily extended to new object categories without the need for cumbersome real-world annotations. Project webpage: https://aartykov.github.io/sim2art/
Read the full abstract ▾
All authors Arslan Artykov, Tom Ravaud, Corentin Sautier, Vincent Lepetit
Robotics
Delay-Aware Diffusion Policy: Bridging the Observation-Execution Gap in Dynamic Tasks
Aileen Liao, Dong-Ki Kim, Max Olan Smith et al. (+2 more)
As a
robot senses and selects actions, the world keeps changing. This inference delay creates a gap of tens to hundreds of milliseconds between the observed state and the state at execution. In this work, we take the natural generalization from zero delay to measured delay during training and inference. We introduce Delay-Aware
Diffusion Policy (DA-DP), a framework for explicitly incorporating inference delays into policy learning. DA-DP corrects zero-delay trajectories to their delay-compensated counterparts, and augments the policy with delay conditioning. We empirically validate DA-DP on a variety of tasks,
robots, and delays and find its success rate more robust to delay than delay-unaware methods. DA-DP is architecture agnostic and transfers beyond
diffusion policies, offering a general pattern for delay-aware imitation learning. More broadly, DA-DP encourages evaluation protocols that report performance as a function of measured latency, not just task difficulty.
Read the full abstract ▾
All authors Aileen Liao, Dong-Ki Kim, Max Olan Smith, Ali-akbar Agha-mohammadi, Shayegan Omidshafiei
Robotics
AMBER: A tether-deployable gripping crawler with compliant microspines for canopy manipulation
P. A. Wigner, L. Romanello, A. Hammad et al. (+5 more)
This paper presents an aerially deployable crawler designed for adaptive locomotion and manipulation within tree canopies. The system combines compliant microspine-based tracks, a dual-track rotary gripper, and an elastic tail, enabling secure attachment and stable traversal across branches of varying
curvature and inclination. Experiments demonstrate reliable gripping up to 90$^\circ$ body roll and inclination, while effective climbing on branches inclined up to 67.5$^\circ$, achieving a maximum speed of 0.55 body lengths per second on horizontal branches. The compliant tracks allow yaw steering of up to 10$^\circ$, enhancing maneuverability on irregular surfaces. Power measurements show efficient operation with a dimensionless cost of transport over an order of magnitude lower than typical hovering power consumption in aerial
robots. The crawler provides a robust, low-power platform for environmental sampling and in-canopy sensing. The aerial deployment is demonstrated at a conceptual and feasibility level, while full drone-crawler integration is left as future work.
Read the full abstract ▾
All authors P. A. Wigner, L. Romanello, A. Hammad, P. H. Nguyen, T. Lan, S. F. Armanini, B. B. Kocer, M. Kovac
Robotics
Multi-Domain Motion Embedding: Expressive Real-Time Mimicry for Legged Robots
Matthias Heyrman, Chenhao Li, Victor Klemm et al. (+3 more)
Effective motion representation is crucial for enabling
robots to imitate expressive behaviors in real time, yet existing motion controllers often ignore inherent patterns in motion. Previous efforts in representation learning do not attempt to jointly capture human and animal movements through structured periodic patterns and variational aperiodic descriptions. To address this, we present Multi-Domain Motion
Embedding (MDME), a motion representation that unifies the complementary
embedding of structured and unstructured features using a wavelet-based encoder and a probabilistic
embedding in parallel. This produces a rich representation of reference motions from a minimal input set that generalizes across diverse motion styles. We evaluate MDME on retargeting-free motion imitation at deployment by conditioning
robot control policies on the learned
embeddings to reconstruct ideal retargeted states on the
robot, demonstrating accurate reproduction of long-horizon trajectories on both
humanoid and quadruped platforms. Our comparative studies confirm that MDME outperforms prior approaches in motion reproduction and generalization to unseen motions. Furthermore, we demonstrate real-time zero-shot deployment on unseen motions, removing per-motion tuning and online retargeting. These results show that MDME provides a generalizable and structure-aware foundation for scalable real-time
robot imitation.
Read the full abstract ▾
All authors Matthias Heyrman, Chenhao Li, Victor Klemm, Dongho Kang, Stelian Coros, Marco Hutter
Robotics · Cross-listed from eess.SY
Obstacle Avoidance of UAV in Dynamic Environments Using Direction and Velocity-Adaptive Artificial Potential Field
Nikita Vaibhav Pavle, Rakesh Kumar Sahoo and Manoranjan Sinha
The conventional Artificial Potential Field (APF) is fundamentally limited by the local minima issue and its inability to account for the kinematics of moving obstacles. This paper addresses the critical challenge of autonomous collision avoidance for Unmanned Aerial Vehicles (
UAVs) operating in dynamic and cluttered airspace by proposing a novel Direction and Relative Velocity Weighted Artificial Potential Field (APF). In this approach, a bounded weighting function, $\omega(\theta,v_{e})$, is introduced to dynamically scale the repulsive potential based on the direction and velocity of the obstacle relative to the
UAV. This robust APF formulation is integrated within a Model Predictive Control (MPC) framework to generate collision-free trajectories while adhering to kinematic constraints. Simulation results demonstrate that the proposed method effectively resolves local minima and significantly enhances safety by enabling smooth, predictive avoidance maneuvers. The system ensures superior path integrity and reliable performance, confirming its viability for autonomous
navigation in complex environments.
Read the full abstract ▾
Robotics · Cross-listed from cs.CV
More than Segmentation: Benchmarking SAM 3 for Segmentation, 3D Perception, and Reconstruction in Robotic Surgery
Wenzhen Dong, Jieming Yu, Yiming Huang et al. (+5 more)
The recent SAM 3 and SAM 3D have introduced significant advancements over the predecessor, SAM 2, particularly with the integration of
language-based
segmentation and enhanced 3D perception capabilities. SAM 3 supports zero-shot
segmentation across a wide range of
prompts, including point, bounding box, and
language-based
prompts, allowing for more flexible and intuitive interactions with the model. In this empirical evaluation, we assess the performance of SAM 3 in
robot-assisted surgery, benchmarking its zero-shot
segmentation with point and bounding box
prompts and exploring its effectiveness in dynamic
video tracking, alongside its newly introduced
language prompt segmentation. While
language prompts show potential, their performance in the
surgical domain is currently suboptimal, highlighting the need for further domain-specific training. Additionally, we investigate SAM 3D's depth reconstruction abilities, demonstrating its capacity to process
surgical scene data and reconstruct 3D anatomical structures from 2D images. Through comprehensive testing on the MICCAI EndoVis 2017 and EndoVis 2018
benchmarks, SAM 3 shows clear improvements over SAM and SAM 2 in both image and
video segmentation under spatial
prompts, while the zero-shot evaluations of SAM 3D on SCARED, StereoMIS, and EndoNeRF indicate strong
monocular depth estimation and realistic 3D instrument reconstruction, yet also reveal remaining limitations in complex, highly dynamic
surgical scenes.
Read the full abstract ▾
All authors Wenzhen Dong, Jieming Yu, Yiming Huang, Hongqiu Wang, Lei Zhu, Albert C. S. Chung, Hongliang Ren, Long Bai
Robotics
See Once, Then Act: Vision-Language-Action Model with Task Learning from One-Shot Video Demonstrations
Guangyan Chen, Meiling Wang, Qi Shao et al. (+10 more)
Developing robust and general-purpose manipulation policies represents a fundamental objective in robotics research. While
Vision-
Language-Action (VLA) models have demonstrated promising capabilities for end-to-end
robot control, existing approaches still exhibit limited generalization to tasks beyond their training distributions. In contrast, humans possess remarkable proficiency in acquiring novel skills by simply observing others performing them once. Inspired by this capability, we propose ViVLA, a generalist robotic manipulation policy that achieves efficient task learning from a single expert demonstration
video at test time. Our approach jointly processes an expert demonstration
video alongside the
robot's visual observations to predict both the demonstrated action sequences and subsequent
robot actions, effectively distilling fine-grained manipulation knowledge from expert behavior and transferring it seamlessly to the
agent. To enhance the performance of ViVLA, we develop a scalable expert-
agent pair data generation pipeline capable of synthesizing paired trajectories from easily accessible human
videos, further augmented by curated pairs from publicly available
datasets. This pipeline produces a total of 892,911 expert-
agent samples for training ViVLA. Experimental results demonstrate that our ViVLA is able to acquire novel manipulation skills from only a single expert demonstration
video at test time. Our approach achieves over 30% improvement on unseen LIBERO tasks and maintains above 35% gains with cross-embodiment
videos. Real-world experiments demonstrate effective learning from human
videos, yielding more than 38% improvement on unseen tasks.
Read the full abstract ▾
All authors Guangyan Chen, Meiling Wang, Qi Shao, Zichen Zhou, Weixin Mao, Te Cui, Minzhao Zhu, Yinan Deng, Luojie Yang, Zhanqi Zhang, Yi Yang, Hua Chen, Yufeng Yue
Robotics
VP-AutoTest: A Virtual-Physical Fusion Autonomous Driving Testing Platform
Yiming Cui, Shiyu Fang, Jiarui Zhang et al. (+6 more)
The rapid development of autonomous vehicles has led to a surge in testing demand. Traditional testing methods, such as virtual simulation, closed-course, and public road testing, face several challenges, including unrealistic vehicle states, limited testing capabilities, and high costs. These issues have prompted increasing interest in virtual-physical fusion testing. However, despite its potential, virtual-physical fusion testing still faces challenges, such as limited element types, narrow testing scope, and fixed evaluation metrics. To address these challenges, we propose the Virtual-Physical Testing Platform for Autonomous Vehicles (VP-AutoTest), which integrates over ten types of virtual and physical elements, including vehicles, pedestrians, and roadside infrastructure, to replicate the diversity of real-world traffic participants. The platform also supports both single-vehicle interaction and multi-vehicle cooperation testing, employing
adversarial testing and parallel deduction to accelerate fault detection and explore algorithmic limits, while OBU and Redis communication enable seamless vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) cooperation across all levels of cooperative automation. Furthermore, VP-AutoTest incorporates a multidimensional evaluation framework and AI-driven expert systems to conduct comprehensive performance assessment and defect diagnosis. Finally, by comparing virtual-physical fusion test results with real-world experiments, the platform performs credibility self-evaluation to ensure both the fidelity and efficiency of autonomous driving testing. Please refer to the website for the full testing functionalities on the autonomous driving public service platform OnSite:https://www.onsite.com.cn.
Read the full abstract ▾
All authors Yiming Cui, Shiyu Fang, Jiarui Zhang, Yan Huang, Chengkai Xu, Bing Zhu, Hao Zhang, Peng Hang, Jian Sun
Robotics
From Real-World Traffic Data to Relevant Critical Scenarios
Florian Lüttner, Nicole Neis, Daniel Stadler et al. (+5 more)
The reliable operation of autonomous vehicles, automated driving functions, and advanced driver assistance systems across a wide range of relevant scenarios is critical for their development and deployment. Identifying a near-complete set of relevant driving scenarios for such functionalities is challenging due to numerous degrees of freedom involved, each affecting the outcomes of the driving scenario differently. Moreover, with increasing technical complexity of new functionalities, the number of potentially relevant, particularly "unknown unsafe" scenarios is increasing. To enhance validation efficiency, it is essential to identify relevant scenarios in advance, starting with simpler domains like highways before moving to more complex environments such as urban traffic. To address this, this paper focuses on analyzing lane change scenarios in highway traffic, which involve multiple degrees of freedom and present numerous safetyrelevant scenarios. We describe the process of data acquisition and processing of real-world data from public highway traffic, followed by the application of criticality measures on trajectory data to evaluate scenarios, as conducted within the AVEAS project (www.aveas.org). By linking the calculated measures to specific lane change driving scenarios and the conditions under which the data was collected, we facilitate the identification of safetyrelevant driving scenarios for various applications. Further, to tackle the extensive range of "unknown unsafe" scenarios, we propose a way to generate relevant scenarios by creating synthetic scenarios based on recorded ones. Consequently, we demonstrate and evaluate a processing chain that enables the identification of safety-relevant scenarios, the development of data-driven methods for extracting these scenarios, and the generation of synthetic critical scenarios via sampling on highways.
Read the full abstract ▾
All authors Florian Lüttner, Nicole Neis, Daniel Stadler, Robin Moss, Mirjam Fehling-Kaschek, Matthias Pfriem, Alexander Stolz, Jens Ziehn
Robotics
Affordance Field Intervention: Enabling VLAs to Escape Memory Traps in Robotic Manipulation
Siyu Xu, Zijian Wang, Yunke Wang et al. (+3 more)
Vision-
Language-Action (VLA) models have shown great performance in robotic manipulation by mapping visual observations and
language instructions directly to actions. However, they remain brittle under distribution shifts: when test scenarios change, VLAs often reproduce memorized trajectories instead of adapting to the updated scene, which is a failure mode we refer to as the "
Memory Trap". This limitation stems from the end-to-end design, which lacks explicit 3D spatial
reasoning and prevents reliable identification of actionable regions in unfamiliar environments. To compensate for this missing spatial understanding, 3D Spatial Affordance Fields (SAFs) can provide a
geometric representation that highlights where interactions are physically feasible, offering explicit cues about regions the
robot should approach or avoid. We therefore introduce Affordance Field Intervention (AFI), a lightweight hybrid framework that uses SAFs as an on-demand plug-in to guide VLA behavior. Our system detects
memory traps through proprioception, repositions the
robot to recent high-affordance regions, and proposes affordance-driven waypoints that anchor VLA-generated actions. A SAF-based scorer then selects trajectories with the highest cumulative affordance. Extensive experiments demonstrate that our method achieves an average improvement of 23.5% across different VLA backbones ($\pi_{0}$ and $\pi_{0.5}$) under out-of-distribution scenarios on real-world robotic platforms, and 20.2% on the LIBERO-Pro
benchmark, validating its effectiveness in enhancing VLA robustness to distribution shifts.
Read the full abstract ▾
All authors Siyu Xu, Zijian Wang, Yunke Wang, Chenghao Xia, Tao Huang, Chang Xu
Robotics
Gait-Adaptive Perceptive Humanoid Locomotion with Real-Time Under-Base Terrain Reconstruction
Haolin Song, Hongbo Zhu, Tao Yu et al. (+5 more)
For full-size
humanoid robots, even with recent advances in reinforcement learning-based control, achieving reliable locomotion on complex terrains, such as long staircases, remains challenging. In such settings, limited perception, ambiguous terrain cues, and insufficient adaptation of gait timing can cause even a single misplaced or mistimed step to result in rapid loss of balance. We introduce a perceptive locomotion framework that merges terrain sensing, gait regulation, and whole-body control into a single reinforcement learning policy. A downward-facing depth camera mounted under the base observes the support region around the feet, and a compact U-Net reconstructs a dense
egocentric height map from each frame in real time, operating at the same frequency as the control loop. The perceptual height map, together with proprioceptive observations, is processed by a unified policy that produces joint commands and a global stepping-phase signal, allowing gait timing and whole-body posture to be adapted jointly to the commanded motion and local terrain geometry. We further adopt a single-stage successive teacher-student training scheme for efficient policy learning and knowledge transfer. Experiments conducted on a 31-DoF, 1.65 m
humanoid robot demonstrate robust locomotion in both simulation and real-world settings, including forward and backward stair ascent and descent, as well as crossing a 46 cm gap. Project Page:https://ga-phl.github.io/
Read the full abstract ▾
All authors Haolin Song, Hongbo Zhu, Tao Yu, Yan Liu, Mingqi Yuan, Wengang Zhou, Hua Chen, Houqiang Li
Robotics · Cross-listed from cs.LG
KAN-Dreamer: Benchmarking Kolmogorov-Arnold Networks as Function Approximators in World Models
Chenwei Shi and Xueyu Luan
DreamerV3 is a state-of-the-art online model-based reinforcement learning (MBRL)
algorithm known for remarkable sample efficiency. Concurrently, Kolmogorov-Arnold Networks (KANs) have emerged as a promising alternative to Multi-Layer Perceptrons (MLPs), offering superior parameter efficiency and interpretability. To mitigate KANs' computational overhead, variants like FastKAN leverage Radial Basis Functions (RBFs) to accelerate inference. In this work, we investigate integrating KAN architectures into the DreamerV3 framework. We introduce KAN-Dreamer, replacing specific MLP and convolutional components of DreamerV3 with KAN and FastKAN layers. To ensure efficiency within the JAX-based World Model, we implement a tailored, fully vectorized version with simplified grid management. We structure our investigation into three subsystems: Visual Perception, Latent Prediction, and Behavior Learning. Empirical evaluations on the DeepMind Control Suite (walker_walk) analyze sample efficiency, training time, and asymptotic performance. Experimental results demonstrate that utilizing our adapted FastKAN as a drop-in replacement for the Reward and Continue predictors yields performance on par with the original MLP-based architecture, maintaining parity in both sample efficiency and training speed. This report serves as a preliminary study for future developments in KAN-based world models.
Read the full abstract ▾
Robotics
ESPADA: Execution Speedup via Semantics Aware Demonstration Data Downsampling for Imitation Learning
Byung-ju Kim, Jinu Pahk, Chungwoo Lee et al. (+6 more)
Behavior-cloning based visuomotor policies enable precise manipulation but often inherit the slow, cautious tempo of human demonstrations, limiting practical deployment. However, prior studies on acceleration methods mainly rely on statistical or heuristic cues that ignore task semantics and can fail across diverse manipulation settings. We present ESPADA, a semantic and spatially aware framework that segments demonstrations using a VLM-LLM pipeline with 3D gripper-object relations, enabling aggressive downsampling only in non-critical segments while preserving precision-critical phases, without requiring extra data or architectural modifications, or any form of retraining. To scale from a single annotated episode to the full
dataset, ESPADA propagates segment labels via Dynamic Time Warping (DTW) on dynamics-only features. Across both simulation and real-world experiments with ACT and DP baselines, ESPADA achieves approximately a 2x speed-up while maintaining success rates, narrowing the gap between human demonstrations and efficient
robot control.
Read the full abstract ▾
All authors Byung-ju Kim, Jinu Pahk, Chungwoo Lee, Jaejoon Kim, Jangha Lee, Theo Taeyeong Kim, Kyuhwan Shim, Jun Ki Lee, Byoung-Tak Zhang
Robotics
Multi-Rigid-Body Approximation of Human Hands with Application to Digital Twin
Bin Zhao, Yiwen Lu, Haohua Zhu et al. (+2 more)
Human hand simulation plays a critical role in digital twin applications, requiring models that balance anatomical fidelity with computational efficiency. We present a complete pipeline for constructing multi-rigid-body approximations of human hands that preserve realistic appearance while enabling real-time physics simulation. Starting from optical motion capture of a specific human hand, we construct a personalized MANO (Multi-Abstracted hand model with Neural Operations) model and convert it to a URDF (Unified
Robot Description Format) representation with anatomically consistent joint axes. The key technical challenge is projecting MANO's unconstrained SO(3) joint rotations onto the kinematically constrained joints of the rigid-body model. We derive closed-form solutions for single degree-of-freedom joints and introduce a Baker-Campbell-Hausdorff (BCH)-corrected iterative method for two degree-of-freedom joints that properly handles the non-commutativity of rotations. We validate our approach through digital twin experiments where reinforcement learning policies control the multi-rigid-body hand to replay captured human demonstrations. Quantitative evaluation shows sub-centimeter reconstruction error and successful grasp execution across diverse manipulation tasks.
Read the full abstract ▾
All authors Bin Zhao, Yiwen Lu, Haohua Zhu, Xiao Li, Sheng Yi
Robotics
Model Predictive Control for Cooperative Docking Between Autonomous Surface Vehicles with Disturbance Rejection
Gianpietro Battocletti, Dimitris Boskos and Bart De Schutter
Uncrewed Surface Vehicles (USVs) are a popular and efficient type of marine craft that find application in a large number of water-based tasks. When multiple USVs operate in the same area, they may be required to dock to each other to perform a shared task. Existing approaches for the docking between autonomous USVs generally consider one USV as a stationary target, while the second one is tasked to reach the required docking pose. In this work, we propose a cooperative approach for USV-USV docking, where two USVs work together to dock at an agreed location. We use a centralized Model Predictive Control (MPC) approach to solve the control problem, obtaining feasible trajectories that also guarantee constraint satisfaction. Owing to its model-based nature, this approach allows the rejection of disturbances, inclusive of exogenous inputs, by anticipating their effect on the USVs through the MPC prediction model. This is particularly effective in case of almost-stationary disturbances such as water currents. In simulations, we demonstrate how the proposed approach allows for a faster and more efficient docking with respect to existing approaches.
Read the full abstract ▾
Robotics
Efficient Computation of a Continuous Topological Model of the Configuration Space of Tethered Mobile Robots
Gianpietro Battocletti, Dimitris Boskos and Bart De Schutter
Despite the
attention that the problem of path planning for tethered
robots has garnered in the past few decades, the approaches proposed to solve it typically rely on a discrete representation of the configuration space and do not exploit a model that can simultaneously capture the topological information of the tether and the continuous location of the
robot. In this work, we explicitly build a topological model of the configuration space of a tethered
robot starting from a polygonal representation of the workspace where the
robot moves. To do so, we first establish a link between the configuration space of the tethered
robot and the universal covering space of the workspace, and then we exploit this link to develop an
algorithm to compute a simplicial complex model of the configuration space. We show how this approach improves the performances of existing
algorithms that build other types of representations of the configuration space. The proposed model can be computed in a fraction of the time required to build traditional homotopy-augmented graphs, and is continuous, allowing to solve the path planning task for tethered
robots using a broad set of path planning
algorithms.
Read the full abstract ▾
Robotics
SINRL: Socially Integrated Navigation with Reinforcement Learning using Spiking Neural Networks
Florian Tretter, Daniel Flögel, Alexandru Vasilache et al. (+3 more)
Integrating autonomous mobile
robots into human environments requires human-like decision-making and energy-efficient, event-based computation. Despite progress, neuromorphic methods are rarely applied to Deep Reinforcement Learning (DRL)
navigation approaches due to unstable training. We address this gap with a hybrid socially integrated DRL actor-critic approach that combines Spiking Neural Networks (SNNs) in the actor with Artificial Neural Networks (ANNs) in the critic and a neuromorphic feature extractor to capture temporal crowd dynamics and human-
robot interactions. Our approach enhances social
navigation performance and reduces estimated energy consumption by approximately 1.69 orders of magnitude.
Read the full abstract ▾
All authors Florian Tretter, Daniel Flögel, Alexandru Vasilache, Max Grobbel, Jürgen Becker, Sören Hohmann
Robotics
Spatiotemporal Calibration and Ground Truth Estimation for High-Precision SLAM Benchmarking in Extended Reality
Zichao Shu, Shitao Bei, Lijun Li et al. (+1 more)
Simultaneous localization and mapping (SLAM) plays a fundamental role in extended reality (XR) applications. As the standards for immersion in XR continue to increase, the demands for SLAM benchmarking have become more stringent. Trajectory accuracy is the key metric, and marker-based optical motion capture (MoCap) systems are widely used to generate ground truth (GT) because of their drift-free and relatively accurate measurements. However, the precision of MoCap-based GT is limited by two factors: the spatiotemporal calibration with the device under test (DUT) and the inherent jitter in the MoCap measurements. These limitations hinder accurate SLAM benchmarking, particularly for key metrics like rotation error and inter-frame jitter, which are critical for immersive XR experiences. This paper presents a novel continuous-time maximum likelihood
estimator to address these challenges. The proposed method integrates auxiliary inertial measurement unit (IMU) data to compensate for MoCap jitter. Additionally, a variable time synchronization method and a pose residual based on screw congruence constraints are proposed, enabling precise spatiotemporal calibration across multiple sensors and the DUT. Experimental results demonstrate that our approach outperforms existing methods, achieving the precision necessary for comprehensive benchmarking of state-of-the-art SLAM
algorithms in XR applications. Furthermore, we thoroughly validate the practicality of our method by benchmarking several leading XR devices and open-source SLAM
algorithms. The code is publicly available at https://github.com/ylab-xrpg/xr-hpgt.
Read the full abstract ▾
All authors Zichao Shu, Shitao Bei, Lijun Li, Zetao Chen
Robotics · Cross-listed from cs.MA
Characterizing Lane-Changing Behavior in Mixed Traffic
Sungyong Chung, Alireza Talebpour and Samer H. Hamdar
Characterizing and understanding lane-changing behavior in the presence of automated vehicles (AVs) is crucial to ensuring safety and efficiency in mixed traffic. Accordingly, this study aims to characterize the interactions between the lane-changing vehicle (active vehicle) and the vehicle directly impacted by the maneuver in the target lane (passive vehicle). Utilizing real-world trajectory data from the Waymo Open Motion
Dataset (WOMD), this study explores patterns in lane-changing behavior and provides insight into how these behaviors evolve under different AV
market penetration rates (MPRs). In particular, we propose a game-theoretic framework to analyze cooperative and defective behaviors in mixed traffic, applied to the 7,636 observed lane-changing events in the WOMD. First, we utilize k-means clustering to classify vehicles as cooperative or defective, revealing that the proportions of cooperative AVs are higher than those of HDVs in both active and passive roles. Next, we jointly estimate the utilities of active and passive vehicles to model their behaviors using the quantal response
equilibrium framework. Empirical payoff tables are then constructed based on these utilities. Using these payoffs, we analyze the presence of social dilemmas and examine the
evolution of cooperative behaviors using evolutionary game theory. Our results reveal the presence of social dilemmas in approximately 4% and 11% of lane-changing events for active and passive vehicles, respectively, with most classified as Stag Hunt or Prisoner's Dilemma (Chicken Game rarely observed). Moreover, the Monte Carlo simulation results show that repeated lane-changing interactions consistently lead to increased cooperative behavior over time, regardless of the AV penetration rate.
Read the full abstract ▾
Robotics
Using Vision-Language Models as Proxies for Social Intelligence in Human-Robot Interaction
Fanjun Bu, Melina Tsai, Audrey Tjokro et al. (+3 more)
Robots operating in everyday environments must often decide when and whether to engage with people, yet such decisions often hinge on subtle nonverbal cues that unfold over time and are difficult to model explicitly. Drawing on a five-day Wizard-of-Oz deployment of a mobile service
robot in a university cafe, we analyze how people signal interaction readiness through nonverbal behaviors and how expert wizards use these cues to guide engagement. Motivated by these observations, we propose a two-stage pipeline in which lightweight perceptual detectors (gaze shifts and proxemics) are used to selectively trigger heavier
video-based
vision-
language model (VLM) queries at socially meaningful moments. We evaluate this pipeline on replayed field interactions and compare two prompting strategies. Our findings suggest that selectively using VLMs as proxies for social
reasoning enables socially responsive
robot behavior, allowing
robots to act appropriately by attending to the cues people naturally provide in real-world interactions.
Read the full abstract ▾
All authors Fanjun Bu, Melina Tsai, Audrey Tjokro, Tapomayukh Bhattacharjee, Jorge Ortiz, Wendy Ju
Robotics
Time-Varying Formation Tracking Control of Wheeled Mobile Robots With Region Constraint: A Generalized Udwadia-Kalaba Framework
Yijie Kang, Yuqing Hao, Qingyun Wang et al. (+1 more)
In this article, the time-varying formation tracking control of wheeled mobile
robots with region constraint is investigated from a generalized Udwadia-Kalaba framework. The communication network is modeled as a directed and weighted graph that has a spanning tree with the leader being the root. By reformulating the time-varying formation tracking control objective as an equality constrained equation and transforming the region constraint by a diffeomorphism, the time-varying formation tracking controller with the region constraint is designed under the generalized Udwadia-Kalaba framework. Compared with the existing works on time-varying formation tracking control, the region constraint is taken into account in this paper, which ensures the safety of the
robots. Finally, the feasibility of the proposed control strategy is illustrated through some numerical simulations.
Read the full abstract ▾
All authors Yijie Kang, Yuqing Hao, Qingyun Wang, Guanrong Chen
Robotics
Mimir: Hierarchical Goal-Driven Diffusion with Uncertainty Propagation for End-to-End Autonomous Driving
Zebin Xing, Yupeng Zheng, Qichao Zhang et al. (+5 more)
End-to-end autonomous driving has emerged as a pivotal direction in the field of autonomous systems. Recent works have demonstrated impressive performance by incorporating high-level guidance signals to steer low-level trajectory planners. However, their potential is often constrained by inaccurate high-level guidance and the computational overhead of complex guidance modules. To address these limitations, we propose Mimir, a novel hierarchical dual-system framework capable of generating robust trajectories relying on goal points with uncertainty estimation: (1) Unlike previous approaches that deterministically model, we estimate goal point uncertainty with a Laplace distribution to enhance robustness; (2) To overcome the slow inference speed of the guidance system, we introduce a multi-rate guidance mechanism that predicts extended goal points in advance. Validated on challenging Navhard and Navtest
benchmarks, Mimir surpasses previous state-of-the-art methods with a 20% improvement in the driving score EPDMS, while achieving 1.6 times improvement in high-level module inference speed without compromising accuracy. The code and models will be released soon to promote reproducibility and further development. The code is available at https://github.com/ZebinX/Mimir-Uncertainty-Driving
Read the full abstract ▾
All authors Zebin Xing, Yupeng Zheng, Qichao Zhang, Zhixing Ding, Pengxuan Yang, Songen Gu, Zhongpu Xia, Dongbin Zhao
Robotics
Surrogate compliance modeling enables reinforcement learned locomotion gaits for soft robots
Jue Wang, Mingsong Jiang, Luis A.Ramirez et al. (+5 more)
Adaptive morphogenetic
robots adapt their morphology and control policies to meet changing tasks and environmental conditions. Many such systems leverage soft components, which enable shape morphing but also introduce simulation and control challenges. Soft-body simulators remain limited in accuracy and computational tractability, while rigid-body simulators cannot capture soft-material dynamics. Here, we present a surrogate compliance modeling approach: rather than explicitly modeling soft-body physics, we introduce indirect variables representing soft-material deformation within a rigid-body simulator. We validate this approach using our amphibious robotic turtle, a quadruped with soft morphing limbs designed for multi-environment locomotion. By capturing deformation effects as changes in effective limb length and limb center of mass, and by applying reinforcement learning with extensive randomization of these indirect variables, we achieve reliable policy learning entirely in a rigid-body simulation. The resulting gaits transfer directly to hardware, demonstrating high-fidelity sim-to-real performance on hard, flat substrates and robust, though lower-fidelity, transfer on rheologically complex terrains. The learned closed-loop gaits exhibit unprecedented terrestrial maneuverability and achieve an order-of-magnitude reduction in cost of transport compared to open-loop baselines. Field experiments with the
robot further demonstrate stable, multi-gait locomotion across diverse natural terrains, including gravel, grass, and mud.
Read the full abstract ▾
All authors Jue Wang, Mingsong Jiang, Luis A.Ramirez, Bilige Yang, Mujun Zhang, Esteban Figueroa, Wenzhong Yan, Rebecca Kramer-Bottiglio
Robotics
SCU-Hand with Integrated Single-Sheet Valve: A Funnel-Shaped Robotic Hand for Milligram-Scale Powder Handling
Tomoya Takahashi, Yusaku Nakajima, Cristian Camilo Beltran-Hernandez et al. (+5 more)
Laboratory Automation (LA) has the potential to accelerate solid-state materials discovery by enabling continuous robotic operation without human intervention. While robotic systems have been developed for tasks such as powder grinding and X-ray diffraction (XRD) analysis, fully automating powder handling at the milligram scale remains a significant challenge due to the complex flow dynamics of powders and the diversity of laboratory tasks. To address this challenge, this study proposes the SCU-Hand-SV (Soft Conical Universal Robotic Hand with Single-sheet Valve), which preserves the softness and conical sheet designs in prior work while incorporating a controllable valve at the cone apex to enable precise, incremental dispensing of milligram-scale powder quantities. The SCU-Hand-SV is integrated with an external balance through a feedback control system based on a model of powder flow and online parameter identification. Experimental evaluations with glass beads, monosodium glutamate, and titanium dioxide demonstrated that 80% of the trials achieved an error within -2 mg to +2 mg, and the maximum error observed was approximately 20 mg across a target range of 20 mg to 3 g. In addition, by incorporating flow prediction models commonly used for hoppers and performing online parameter identification, the system is able to adapt to variations in powder dynamics. Compared to direct PID control, the proposed model-based control significantly improved both accuracy and convergence speed. These results highlight the potential of the proposed system to enable efficient and flexible powder weighing, with scalability toward larger quantities and applicability to a broad range of laboratory automation tasks.
Read the full abstract ▾
All authors Tomoya Takahashi, Yusaku Nakajima, Cristian Camilo Beltran-Hernandez, Yuki Kuroda, Kazutoshi Tanaka, Masashi Hamaya, Kanta Ono, Yoshitaka Ushiku