Robotics
Real-Time Robot Execution with Masked Action Chunking
Haoxuan Wang, Gengyu Zhang, Yan Yan et al. (+3 more)
Real-time execution is essential for cyber-physical systems such as
robots. These systems operate in dynamic real-world environments where even small delays can undermine responsiveness and compromise performance. Asynchronous inference has recently emerged as a system-level paradigm for real-time
robot manipulation, enabling the next action chunk to be predicted while the current one is being executed. While this approach achieves real-time responsiveness, naive integration often results in execution failure. Previous methods attributed this failure to inter-chunk discontinuity and developed test-time
algorithms to smooth chunk boundaries. In contrast, we identify another critical yet overlooked factor: intra-chunk inconsistency, where the
robot's executed action chunk partially misaligns with its current perception. To address this, we propose REMAC, which learns corrective adjustments on the pretrained policy through masked action chunking, enabling the policy to remain resilient under mismatches between intended actions and actual execution during asynchronous inference. In addition, we introduce a prefix-preserved sampling procedure to reinforce inter-chunk continuity. Overall, our method delivers more reliable policies without incurring additional latency. Extensive experiments in both simulation and real-world settings demonstrate that our method enables faster task execution, maintains robustness across varying delays, and consistently achieves higher completion rates.
Read the full abstract ▾
All authors Haoxuan Wang, Gengyu Zhang, Yan Yan, Yuzhang Shang, Ramana Rao Kompella, Gaowen Liu
Robotics · Cross-listed from cs.MA
Game-Theoretic Autonomous Driving: A Graphs of Convex Sets Approach
Nikolaj Käfer, Ahmed Khalil, Edward Huynh et al. (+2 more)
Multi-vehicle autonomous driving couples strategic interaction with hybrid (discrete-continuous) maneuver planning under shared safety constraints. We introduce IBR-GCS, an Iterative Best Response (IBR) planning approach based on the Graphs of Convex Sets (GCS) framework that models highway driving as a generalized noncooperative game. IBR-GCS integrates combinatorial maneuver
reasoning, trajectory planning, and game-theoretic interaction within a unified framework. The key novelty is a vehicle-specific, strategy-dependent GCS construction. Specifically, at each best-response update, each vehicle builds its own graph conditioned on the current strategies of the other vehicles, with vertices representing lane-specific, time-varying, convex, collision-free regions and edges encoding dynamically feasible transitions. This yields a shortest-path problem in GCS for each best-response step, which admits an efficient convex relaxation that can be solved using convex optimization tools without exhaustive discrete tree search. We then apply an iterative best-response scheme in which vehicles update their trajectories sequentially and provide conditions under which the resulting inexact updates converge to an approximate generalized Nash
equilibrium. Simulation results across multi-lane, multi-vehicle scenarios demonstrate that IBR-GCS produces safe trajectories and strategically consistent interactive behaviors.
Read the full abstract ▾
All authors Nikolaj Käfer, Ahmed Khalil, Edward Huynh, Efstathios Bakolas, David Fridovich-Keil
Robotics
Just in time Informed Trees: Manipulability-Aware Asymptotically Optimized Motion Planning
Kuanqi Cai, Liding Zhang, Xinwen Su et al. (+6 more)
In high-dimensional robotic path planning, traditional sampling-based methods often struggle to efficiently identify both feasible and optimal paths in complex, multi-obstacle environments. This challenge is intensified in robotic manipulators, where the risk of kinematic singularities and self-collisions further complicates motion efficiency and safety. To address these issues, we introduce the Just-in-Time Informed Trees (JIT*)
algorithm, an enhancement over Effort Informed Trees (EIT*), designed to improve path planning through two core modules: the Just-in-Time module and the Motion Performance module. The Just-in-Time module includes "Just-in-Time Edge," which dynamically refines edge connectivity, and "Just-in-Time Sample," which adjusts sampling density in bottleneck areas to enable faster initial path discovery. The Motion Performance module balances manipulability and trajectory cost through dynamic switching, optimizing motion control while reducing the risk of singularities. Comparative analysis shows that JIT* consistently outperforms traditional sampling-based planners across $\mathbb{R}^4$ to $\mathbb{R}^{16}$ dimensions. Its effectiveness is further demonstrated in single-arm and dual-arm manipulation tasks, with experimental results available in a
video at https://youtu.be/nL1BMHpMR7c.
Read the full abstract ▾
All authors Kuanqi Cai, Liding Zhang, Xinwen Su, Kejia Chen, Chaoqun Wang, Sami Haddadin, Alois Knoll, Arash Ajoudani, Luis Figueredo
Robotics
E2HiL: Entropy-Guided Sample Selection for Efficient Real-World Human-in-the-Loop Reinforcement Learning
Haoyuan Deng, Yudong Lin, Yuanjiang Xue et al. (+5 more)
Human-in-the-loop guidance has emerged as an effective approach for accelerating online reinforcement learning (RL) in real-world manipulation. However, existing human-in-the-loop RL (HiL-RL) frameworks often suffer from low sample efficiency, requiring substantial human interventions to achieve convergence and thereby leading to high labor costs. To address this, we propose a sample-efficient real-world human-in-the-loop RL framework named E2HiL, which requires fewer human interventions by actively selecting informative samples. Specifically, stable reduction of policy
entropy enables improved trade-off between exploration and exploitation with higher sample efficiency. We first build influence functions of different samples on the policy
entropy, which is efficiently estimated by the covariance of action probabilities and soft advantages of policies. Then we select samples with moderate values of influence functions, where shortcut samples that induce sharp
entropy drops and noisy samples with negligible effect are pruned. Extensive experiments across 10 real-world manipulation tasks, spanning multiple embodiments and learning frameworks, demonstrate that E2HiL improves success rates by 24.9% while reducing human interventions by 9.3% compared to state-of-the-art HiL-RL baselines. These results validate its effectiveness as a policy- and embodiment-agnostic plug-and-play module for efficient real-world RL. The project page can be found at https://e2hil.github.io/.
Read the full abstract ▾
All authors Haoyuan Deng, Yudong Lin, Yuanjiang Xue, Haoyang Du, Qianzhun Wang, Boyang Zhou, Zhenyu Wu, Ziwei Wang
Robotics · Cross-listed from cs.CV
VGGT-SLAM 2.0: Real-time Dense Feed-forward Scene Reconstruction
Dominic Maggio and Luca Carlone
We present VGGT-SLAM 2.0, a real-time RGB feed-forward SLAM system which substantially improves upon VGGT-SLAM for incrementally aligning submaps created from VGGT. Firstly, we remove high-dimensional 15-degree-of-freedom drift and planar degeneracy from VGGT-SLAM by creating a new factor graph design while still addressing the reconstruction ambiguity of VGGT given unknown camera intrinsics. Secondly, by studying the
attention layers of VGGT, we show that one of the layers is well suited to assist in image
retrieval verification for free without additional training, which enables both rejecting false positive matches and allows for completing more loop closures. Finally, we conduct a suite of experiments which includes showing VGGT-SLAM 2.0 can easily be adapted for open-set object detection and demonstrating real-time performance while running online onboard a ground
robot using a Jetson Thor. We test in environments ranging from cluttered indoor apartments and office scenes to a 4,200 square foot barn, and we also demonstrate VGGT-SLAM 2.0 achieves the highest accuracy on the TUM
dataset with about 23 percent less pose error than VGGT-SLAM. Code will be released upon publication.
Read the full abstract ▾
Robotics
Estimating Trust in Human-Robot Collaboration through Behavioral Indicators and Explainability
Giulio Campagna, Marta Lagomarsino, Marta Lorenzini et al. (+3 more)
Industry 5.0 focuses on human-centric collaboration between humans and
robots, prioritizing safety, comfort, and trust. This study introduces a data-driven framework to assess trust using behavioral indicators. The framework employs a Preference-Based Optimization
algorithm to generate trust-enhancing trajectories based on operator feedback. This feedback serves as ground truth for training machine learning models to predict trust levels from behavioral indicators. The framework was tested in a chemical industry scenario where a
robot assisted a human operator in mixing chemicals. Machine learning models classified trust with over 80% accuracy, with the Voting Classifier achieving 84.07% accuracy and an AUC-ROC score of 0.90. These findings underscore the effectiveness of data-driven methods in assessing trust within human-
robot collaboration, emphasizing the valuable role behavioral indicators play in predicting the dynamics of human trust.
Read the full abstract ▾
All authors Giulio Campagna, Marta Lagomarsino, Marta Lorenzini, Dimitrios Chrysostomou, Matthias Rehm, Arash Ajoudani
Robotics · Cross-listed from cs.HC
How Does Delegation in Social Interaction Evolve Over Time? Navigation with a Robot for Blind People
Rayna Hata, Masaki Kuribayashi, Allan Wang et al. (+2 more)
Autonomy and independent
navigation are vital to daily life but remain challenging for individuals with blindness. Robotic systems can enhance mobility and confidence by providing intelligent
navigation assistance. However, fully autonomous systems may reduce users' sense of control, even when they wish to remain actively involved. Although collaboration between user and
robot has been recognized as important, little is known about how perceptions of this relationship change with repeated use. We present a repeated exposure study with six blind participants who interacted with a
navigation-assistive
robot in a real-world museum. Participants completed tasks such as navigating crowds, approaching lines, and encountering obstacles. Findings show that participants refined their strategies over time, developing clearer preferences about when to rely on the
robot versus act independently. This work provides insights into how strategies and preferences evolve with repeated interaction and offers design implications for
robots that adapt to user needs over time.
Read the full abstract ▾
All authors Rayna Hata, Masaki Kuribayashi, Allan Wang, Hironobu Takagi, Chieko Asakawa
Robotics
HARMONI: Multimodal Personalization of Multi-User Human-Robot Interactions with LLMs
Jeanne Malécot, Hamed Rahimi, Jeanne Cattoni et al. (+5 more)
Existing human-
robot interaction systems often lack mechanisms for sustained personalization and dynamic adaptation in multi-user environments, limiting their effectiveness in real-world deployments. We present HARMONI, a
multimodal personalization framework that leverages large
language models to enable socially assistive
robots to manage long-term multi-user interactions. The framework integrates four key modules: (i) a perception module that identifies active speakers and extracts
multimodal input; (ii) a world modeling module that maintains representations of the environment and short-term conversational context; (iii) a user modeling module that updates long-term speaker-specific profiles; and (iv) a generation module that produces contextually grounded and ethically informed responses. Through extensive evaluation and ablation studies on four
datasets, as well as a real-world scenario-driven user-study in a nursing home environment, we demonstrate that HARMONI supports robust speaker identification, online
memory updating, and ethically aligned personalization, outperforming baseline LLM-driven approaches in user modeling accuracy, personalization quality, and user satisfaction.
Read the full abstract ▾
All authors Jeanne Malécot, Hamed Rahimi, Jeanne Cattoni, Marie Samson, Mouad Abrini, Mahdi Khoramshahi, Maribel Pino, Mohamed Chetouani
Robotics
Information-Theoretic Detection of Bimanual Interactions for Dual-Arm Robot Plan Generation
Elena Merlo, Marta Lagomarsino and Arash Ajoudani
Programming by demonstration is a strategy to simplify the
robot programming process for non-experts via human demonstrations. However, its adoption for bimanual tasks is an underexplored problem due to the complexity of hand coordination, which also hinders data recording. This paper presents a novel one-shot method for processing a single RGB
video of a bimanual task demonstration to generate an execution plan for a dual-arm robotic system. To detect hand coordination policies, we apply Shannon's information theory to analyze the information flow between scene elements and leverage scene graph properties. The generated plan is a modular behavior tree that assumes different structures based on the desired arms coordination. We validated the effectiveness of this framework through multiple subject
video demonstrations, which we collected and made open-source, and exploiting data from an external, publicly available
dataset. Comparisons with existing methods revealed significant improvements in generating a centralized execution plan for coordinating two-arm systems.
Read the full abstract ▾
Robotics
Whether We Care, How We Reason: The Dual Role of Anthropomorphism and Moral Foundations in Robot Abuse
Fan Yang, Renkai Ma, Yaxin Hu et al. (+1 more)
As
robots become increasingly integrated into daily life, understanding responses to
robot mistreatment carries important ethical and design implications. This mixed-methods study (N = 201) examined how anthropomorphic levels and moral foundations shape reactions to
robot abuse. Participants viewed
videos depicting physical mistreatment of
robots varying in humanness (Spider, Twofoot,
Humanoid) and completed measures assessing moral foundations, anger, and social distance. Results revealed that anthropomorphism determines whether people extend moral consideration to
robots, while moral foundations shape how they reason about such consideration. Qualitative analysis revealed distinct
reasoning patterns: low-progressivism individuals employed character-based judgments, while high-progressivism individuals engaged in future-oriented moral deliberation. Findings offer implications for
robot design and policy communication.
Read the full abstract ▾
All authors Fan Yang, Renkai Ma, Yaxin Hu, Lingyao Li
Robotics · Cross-listed from cs.LG
Unsupervised Learning of Efficient Exploration: Pre-training Adaptive Policies via Self-Imposed Goals
Octavio Pappalardo
Unsupervised pre-training can equip reinforcement learning
agents with prior knowledge and accelerate learning in downstream tasks. A promising direction, grounded in human development, investigates
agents that learn by setting and pursuing their own goals. The core challenge lies in how to effectively generate, select, and learn from such goals. Our focus is on broad distributions of downstream tasks where solving every task zero-shot is infeasible. Such settings naturally arise when the target tasks lie outside of the pre-training distribution or when their identities are unknown to the
agent. In this work, we (i) optimize for efficient multi-episode exploration and adaptation within a meta-learning framework, and (ii) guide the training curriculum with evolving estimates of the
agent's post-adaptation performance. We present ULEE, an unsupervised meta-learning method that combines an in-context learner with an
adversarial goal-generation strategy that maintains training at the frontier of the
agent's capabilities. On XLand-MiniGrid
benchmarks, ULEE pre-training yields improved exploration and adaptation abilities that generalize to novel objectives, environment dynamics, and map structures. The resulting policy attains improved zero-shot and few-shot performance, and provides a strong initialization for longer fine-tuning processes. It outperforms learning from scratch, DIAYN pre-training, and alternative curricula. Code is available at: https://github.com/Octavio-Pappalardo/ulee-jax
Read the full abstract ▾
Robotics
Reimagining Social Robots as Recommender Systems: Foundations, Framework, and Applications
Jin Huang, Fethiye Irmak Doğan and Hatice Gunes
Personalization in social
robots refers to the ability of the
robot to meet the needs and/or preferences of an individual user. Existing approaches typically rely on large
language models (LLMs) to generate context-aware responses based on user metadata and historical interactions or on adaptive methods such as reinforcement learning (RL) to learn from users' immediate reactions in real time. However, these approaches fall short of comprehensively capturing user preferences-including long-term, short-term, and fine-grained aspects-, and of using them to rank and select actions, proactively personalize interactions, and ensure ethically responsible adaptations. To address the limitations, we propose drawing on recommender systems (RSs), which specialize in modeling user preferences and providing personalized recommendations. To ensure the integration of RS techniques is well-grounded and seamless throughout the social
robot pipeline, we (i) align the paradigms underlying social
robots and RSs, (ii) identify key techniques that can enhance personalization in social
robots, and (iii) design them as modular, plug-and-play components. This work not only establishes a framework for integrating RS techniques into social
robots but also opens a pathway for deep collaboration between the RS and HRI communities, accelerating innovation in both fields.
Read the full abstract ▾
Robotics
SCOPE: Smooth Convex Optimization for Planned Evolution of Deformable Linear Objects
Ali Jnadi, Hadi Salloum, Yaroslav Kholodov et al. (+2 more)
We present SCOPE, a fast and efficient framework for modeling and manipulating deformable linear objects (DLOs). Unlike conventional energy-based approaches, SCOPE leverages convex approximations to significantly reduce computational cost while maintaining smooth and physically plausible deformations. This trade-off between speed and accuracy makes the method particularly suitable for applications requiring real-time or near-real-time response. The effectiveness of the proposed framework is demonstrated through comprehensive simulation experiments, highlighting its ability to generate smooth shape trajectories under
geometric and length constraints.
Read the full abstract ▾
All authors Ali Jnadi, Hadi Salloum, Yaroslav Kholodov, Alexander Gasnikov, Karam Almaghout
Robotics · Cross-listed from cs.LG
Scalable Exploration for High-Dimensional Continuous Control via Value-Guided Flow
Yunyue Wei, Chenhui Zuo and Yanan Sui
Controlling high-dimensional systems in biological and robotic applications is challenging due to expansive state-action spaces, where effective exploration is critical. Commonly used exploration strategies in reinforcement learning are largely undirected with sharp degradation as action dimensionality grows. Many existing methods resort to dimensionality reduction, which constrains policy expressiveness and forfeits system flexibility. We introduce Q-guided Flow Exploration (Qflex), a scalable reinforcement learning method that conducts exploration directly in the native high-dimensional action space. During training, Qflex traverses actions from a learnable source distribution along a probability flow induced by the learned value function, aligning exploration with task-relevant gradients rather than isotropic noise. Our proposed method substantially outperforms representative online reinforcement learning baselines across diverse high-dimensional continuous-control
benchmarks. Qflex also successfully controls a full-body human musculoskeletal model to perform agile, complex movements, demonstrating superior scalability and sample efficiency in very high-dimensional settings. Our results indicate that value-guided flows offer a principled and practical route to exploration at scale.
Read the full abstract ▾
Robotics
Enhancing Worker Safety in Harbors Using Quadruped Robots
Zoe Betta, Davide Corongiu, Carmine Tommaso Recchiuto et al. (+1 more)
Infrastructure inspection is becoming increasingly relevant in the field of robotics due to its significant impact on ensuring workers' safety. The harbor environment presents various challenges in designing a robotic solution for inspection, given the complexity of daily operations. This work introduces an initial phase to identify critical areas within the port environment. Following this, a preliminary solution using a quadruped
robot for inspecting these critical areas is analyzed.
Read the full abstract ▾
All authors Zoe Betta, Davide Corongiu, Carmine Tommaso Recchiuto, Antonio Sgorbissa
Robotics
AC^2-VLA: Action-Context-Aware Adaptive Computation in Vision-Language-Action Models for Efficient Robotic Manipulation
Wenda Yu, Tianshi Wang, Fengling Li et al. (+2 more)
Vision-
Language-Action (VLA) models have demonstrated strong performance in robotic manipulation, yet their closed-loop deployment is hindered by the high latency and compute cost of repeatedly running large
vision-
language backbones at every timestep. We observe that VLA inference exhibits structured redundancies across temporal, spatial, and depth dimensions, and that most existing efficiency methods ignore action context, despite its central role in
embodied tasks. To address this gap, we propose Action-Context-aware Adaptive Computation for VLA models (AC^2-VLA), a unified framework that conditions computation on current visual observations,
language instructions, and previous action states. Based on this action-centric context, AC^2-VLA adaptively performs cognition reuse across timesteps,
token pruning, and selective execution of model components within a unified mechanism. To train the adaptive policy, we introduce an action-guided self-
distillation scheme that preserves the behavior of the dense VLA policy while enabling structured sparsification that transfers across tasks and settings. Extensive experiments on robotic manipulation
benchmarks show that AC^2-VLA achieves up to a 1.79\times speedup while reducing FLOPs to 29.4% of the dense baseline, with comparable task success.
Read the full abstract ▾
All authors Wenda Yu, Tianshi Wang, Fengling Li, Jingjing Li, Lei Zhu
Robotics · Cross-listed from cs.LG
Safe Exploration via Policy Priors
Manuel Wendl, Yarden As, Manish Prajapat et al. (+3 more)
Safe exploration is a key requirement for reinforcement learning (RL)
agents to learn and adapt online, beyond controlled (e.g. simulated) environments. In this work, we tackle this challenge by utilizing suboptimal yet conservative policies (e.g., obtained from offline data or simulators) as priors. Our approach, SOOPER, uses probabilistic dynamics models to optimistically explore, yet pessimistically fall back to the conservative policy prior if needed. We prove that SOOPER guarantees safety throughout learning, and establish convergence to an optimal policy by bounding its cumulative regret. Extensive experiments on key safe RL
benchmarks and real-world hardware demonstrate that SOOPER is scalable, outperforms the state-of-the-art and validate our theoretical guarantees in practice.
Read the full abstract ▾
All authors Manuel Wendl, Yarden As, Manish Prajapat, Anton Pollak, Stelian Coros, Andreas Krause
Robotics · Cross-listed from cs.CV
The S3LI Vulcano Dataset: A Dataset for Multi-Modal SLAM in Unstructured Planetary Environments
Riccardo Giubilato, Marcus Gerhard Müller, Marco Sewtz et al. (+3 more)
We release the S3LI Vulcano
dataset, a multi-modal
dataset towards development and benchmarking of Simultaneous Localization and Mapping (SLAM) and place recognition
algorithms that rely on visual and
LiDAR modalities. Several sequences are recorded on the volcanic island of Vulcano, from the Aeolian Islands in Sicily, Italy. The sequences provide users with data from a variety of environments, textures and terrains, including basaltic or iron-rich rocks, geological formations from old lava channels, as well as dry vegetation and water. The data (rmc.dlr.de/s3li_
dataset) is accompanied by an open source toolkit (github.com/DLR-RM/s3li-toolkit) providing tools for generating ground truth poses as well as preparation of labelled samples for place recognition tasks.
Read the full abstract ▾
All authors Riccardo Giubilato, Marcus Gerhard Müller, Marco Sewtz, Laura Alejandra Encinar Gonzalez, John Folkesson, Rudolph Triebel
Robotics
Enhancing Inverse Perspective Mapping for Automatic Vectorized Road Map Generation
Hongji Liu, Linwei Zheng, Yongjian Li et al. (+4 more)
In this study, we present a low-cost and unified framework for vectorized road mapping leveraging enhanced inverse perspective mapping (IPM). In this framework, Catmull-Rom splines are utilized to characterize lane lines, and all the other ground markings are depicted using polygons uniformly. The results from instance
segmentation serve as references to refine the three-dimensional position of spline control points and polygon corner points. In conjunction with this process, the homography
matrix of IPM and vehicle poses are optimized simultaneously. Our proposed framework significantly reduces the mapping errors associated with IPM. It also improves the accuracy of the initial IPM homography
matrix and the predicted vehicle poses. Furthermore, it addresses the limitations imposed by the coplanarity assumption in IPM. These enhancements enable IPM to be effectively applied to vectorized road mapping, which serves a cost-effective solution with enhanced accuracy. In addition, our framework generalizes road map elements to include all common ground markings and lane lines. The proposed framework is evaluated in two different practical scenarios, and the test results show that our method can automatically generate high-precision maps with near-centimeter-level accuracy. Importantly, the optimized IPM
matrix achieves an accuracy comparable to that of manual calibration, while the accuracy of vehicle poses is also significantly improved.
Read the full abstract ▾
All authors Hongji Liu, Linwei Zheng, Yongjian Li, Mingkai Tang, Xiaoyang Yan, Ming Liu, Jun Ma
Robotics
RhoMorph: Rhombus-shaped Deformable Modular Robots for Stable, Medium-Independent Reconfiguration Motion
Jie Gu, Yirui Sun, Zhihao Xia et al. (+3 more)
In this paper, we present RhoMorph, a novel deformable planar
lattice modular self-reconfigurable
robot (MSRR) with a rhombus shaped module. Each module consists of a parallelogram skeleton with a single centrally mounted actuator that enables folding and unfolding along its diagonal. The core design philosophy is to achieve essential MSRR functionalities such as morphing, docking, and locomotion with minimal control complexity. This enables a continuous and stable reconfiguration process that is independent of the surrounding medium, allowing the system to reliably form various configurations in diverse environments. To leverage the unique kinematics of RhoMorph, we introduce morphpivoting, a novel motion primitive for reconfiguration that differs from advanced MSRR systems, and propose a strategy for its continuous execution. Finally, a series of physical experiments validate the module's stable reconfiguration ability, as well as its positional and docking accuracy.
Read the full abstract ▾
All authors Jie Gu, Yirui Sun, Zhihao Xia, Tin Lun Lam, Chunxu Tian, Dan Zhang
Robotics
PALM: Enhanced Generalizability for Local Visuomotor Policies via Perception Alignment
Ruiyu Wang, Zheyu Zhuang, Danica Kragic et al. (+1 more)
Generalizing beyond the training domain in image-based behavior cloning remains challenging. Existing methods address individual axes of generalization, workspace shifts, viewpoint changes, and cross-embodiment transfer, yet they are typically developed in isolation and often rely on complex pipelines. We introduce PALM (Perception
Alignment for Local Manipulation), which leverages the invariance of local action distributions between out-of-distribution (OOD) and demonstrated domains to address these OOD shifts concurrently, without additional input modalities, model changes, or data collection. PALM modularizes the manipulation policy into coarse global components and a local policy for fine-grained actions. We reduce the discrepancy between in-domain and OOD inputs at the local policy level by enforcing local visual focus and consistent proprioceptive representation, allowing the policy to retrieve invariant local actions under OOD conditions. Experiments show that PALM limits OOD performance drops to 8% in simulation and 24% in the real world, compared to 45% and 77% for baselines.
Read the full abstract ▾
All authors Ruiyu Wang, Zheyu Zhuang, Danica Kragic, Florian T. Pokorny
Robotics
ALRM: Agentic LLM for Robotic Manipulation
Vitor Gaboardi dos Santos, Ibrahim Khadraoui, Ibrahim Farhat et al. (+3 more)
Large
Language Models (LLMs) have recently empowered agentic frameworks to exhibit advanced
reasoning and planning capabilities. However, their integration in robotic control pipelines remains limited in two aspects: (1) prior \ac{llm}-based approaches often lack modular, agentic execution mechanisms, limiting their ability to plan, reflect on outcomes, and revise actions in a closed-loop manner; and (2) existing
benchmarks for manipulation tasks focus on low-level control and do not systematically evaluate multistep
reasoning and linguistic variation. In this paper, we propose Agentic LLM for
Robot Manipulation (ALRM), an LLM-driven agentic framework for robotic manipulation. ALRM integrates policy generation with agentic execution through a ReAct-style
reasoning loop, supporting two complementary modes: Code-asPolicy (CaP) for direct executable control code generation, and Tool-as-Policy (TaP) for iterative planning and tool-based action execution. To enable systematic evaluation, we also introduce a novel simulation
benchmark comprising 56 tasks across multiple environments, capturing linguistically diverse instructions. Experiments with ten LLMs demonstrate that ALRM provides a scalable, interpretable, and modular approach for bridging natural
language reasoning with reliable robotic execution. Results reveal Claude-4.1-Opus as the top closed-source model and Falcon-H1-7B as the top open-source model under CaP.
Read the full abstract ▾
All authors Vitor Gaboardi dos Santos, Ibrahim Khadraoui, Ibrahim Farhat, Hamza Yous, Samy Teffahi, Hakim Hacid
Robotics
Analytical Covariance Propagation for DVL-Aided Loosely Coupled SINS Under Attitude Uncertainty
Jin Huang, Zichen Liu, Haoda Li et al. (+5 more)
In loosely coupled strapdown inertial
navigation system/Doppler velocity log (SINS/DVL) integration, the body-frame DVL velocity is projected into the
navigation frame using the SINS-computed attitude. Attitude uncertainty consequently affects both the projected velocity observation and its associated measurement covariance. This paper develops an analytical covariance propagation (ACP) method that treats these two effects consistently. The attitude-error-aware observation model represents perturbations in the projected DVL velocity, whereas the covariance construction propagates the body-frame DVL covariance at the
matrix level and adds a closed-form term derived from the predicted attitude-error covariance, rather than directly rotating component-wise standard deviations. Simulation and a controlled surface-water field experiment compare ACP with conventional SINS/DVL and adaptive-covariance
benchmarks. For the evaluated data sets and parameterizations, ACP yields the lowest position-error metrics among the compared methods. Relative to the variational
Bayesian adaptive Kalman filter (VBAKF) in the field experiment, ACP reduces the northward, eastward, and downward position root-mean-square errors (RMSEs) by 48.5%, 55.0%, and 74.8%, respectively. Beyond DVL aiding, ACP provides a transferable covariance-construction principle for attitude-dependent vector-measurement fusion and a physically interpretable alternative to heuristic frame-dependent covariance assignment.
Read the full abstract ▾
All authors Jin Huang, Zichen Liu, Haoda Li, Zhikun Wang, Yuan Zhao, Xianyu Peng, Yongqun Yu, Ying Chen
Robotics
Reversible Simplex Supervision with Post-Action Debt Accounting for Goal-Reaching RL
Mehdi Heydari Shahna, Joongheon Kim and Jouni Mattila
Deploying reinforcement learning (RL) on multi-tonne
robots calls for supervisory mechanisms that address both operational safety and progress toward task completion. However, repeated switching need not preserve task progress when a learned action increases storage before recovery takes control. We introduce reversible Simplex supervision with post-action debt accounting for a frozen finite-state policy and robust-adaptive recovery. Under exact sampled-state information and stated model and certificate conditions, we prove that recovery repayment exceeding a uniform triggering-edge debt bound guarantees finite switching and finite-sample goal entry. We formulate reachability-based certificate constructions for establishing these sufficient conditions. In 20 matched simulations, goal-entry counts are 20 with debt gating and 18 without it; the two remaining runs terminate under the supervisor's admissibility stopping rule. On an experimental 6000 kg
robot, 24 asphalt and soft-terrain trials evaluate 50 ms supervision above a 1 kHz actuator stack; all eight triggered recoveries complete debt-gated re-entry. The experiments demonstrate the supervisory mechanism in the tested trials.
Read the full abstract ▾
Robotics
Self-Reconfiguration Planning for Deformable Quadrilateral Modular Robots
Jie Gu, Hongrun Gao, Zhihao Xia et al. (+3 more)
For
lattice modular self-reconfigurable
robots (MSRRs), maintaining stable connections during reconfiguration is crucial for physical feasibility and deployability. This letter presents a novel self-reconfiguration planning
algorithm for deformable quadrilateral MSRRs that guarantees stable connection. The method first constructs feasible connect/disconnect actions using a virtual graph representation, and then organizes these actions into a valid execution sequence through a Dependence-based Reverse Tree (DRTree) that resolves interdependencies. We also prove that reconfiguration sequences satisfying motion characteristics exist for any pair of configurations with seven or more modules (excluding linear
topologies). Finally, comparisons with a modified BiRRT
algorithm highlight the superior efficiency and stability of our approach, while deployment on a physical robotic platform confirms its practical feasibility.
Read the full abstract ▾
All authors Jie Gu, Hongrun Gao, Zhihao Xia, Yirun Sun, Chunxu Tian, Dan Zhang
Robotics · Cross-listed from eess.SY
Physical Human-Robot Interaction: A Critical Review of Safety Constraints
Riccardo Zanella, Federico Califano and Stefano Stramigioli
This paper aims to provide a clear and rigorous understanding of commonly recognized safety constraints in physical human-
robot interaction, particularly regarding ISO/TS 15066. We investigate the derivation of these constraints, critically examine the underlying assumptions, and evaluate their practical implications for system-level safety and performance in industrially relevant scenarios. Key design parameters within safety-critical control architectures are identified, and numerical examples are provided to quantify performance degradation arising from typical approximations and design decisions in manufacturing environments. Within this analysis, the fundamental role of energy in safety assessment is emphasized, providing focused insights into energy-based safety methodologies for collaborative industrial
robot systems.
Read the full abstract ▾
Robotics · Cross-listed from cs.CV
Towards Gold-Standard Depth Estimation for Tree Branches in UAV Forestry: Benchmarking Deep Stereo Matching Methods
Yida Lin, Bing Xue, Mengjie Zhang et al. (+2 more)
Autonomous
UAV forestry operations require robust depth estimation with strong cross-domain generalization, yet existing evaluations focus on urban and indoor scenarios, leaving a critical gap for vegetation-dense environments. We present the first systematic zero-shot evaluation of eight stereo methods spanning iterative refinement, foundation model,
diffusion-based, and 3D CNN paradigms. All methods use officially released pretrained weights (trained on Scene Flow) and are evaluated on four standard
benchmarks (ETH3D, KITTI 2012/2015, Middlebury) plus a novel 5,313-pair Canterbury Tree Branches
dataset ($1920 \times 1080$). Results reveal scene-dependent patterns: foundation models excel on structured scenes (BridgeDepth: 0.23 px on ETH3D; DEFOM: 4.65 px on Middlebury), while iterative methods show variable cross-
benchmark performance (IGEV++: 0.36 px on ETH3D but 6.77 px on Middlebury; IGEV: 0.33 px on ETH3D but 4.99 px on Middlebury). Qualitative evaluation on the Tree Branches
dataset establishes DEFOM as the gold-standard baseline for vegetation depth estimation, with superior cross-domain consistency (consistently ranking 1st-2nd across
benchmarks, average rank 1.75). DEFOM predictions will serve as pseudo-ground-truth for future benchmarking.
Read the full abstract ▾
All authors Yida Lin, Bing Xue, Mengjie Zhang, Sam Schofield, Richard Green
Robotics
Task-Centric Policy Optimization from Misaligned Motion Priors
Ziang Zheng, Kai Feng, Yi Nie et al. (+1 more)
Humanoid control often leverages motion priors from human demonstrations to encourage natural behaviors. However, such demonstrations are frequently suboptimal or misaligned with robotic tasks due to embodiment differences, retargeting errors, and task-irrelevant variations, causing naïve imitation to degrade task performance. Conversely, task-only reinforcement learning admits many task-optimal solutions, often resulting in unnatural or unstable motions. This exposes a fundamental limitation of linear reward mixing in
adversarial imitation learning. We propose Task-Centric Motion Priors (TCMP), a task-priority
adversarial imitation framework that treats imitation as a conditional regularizer rather than a co-equal objective. TCMP maximizes task improvement while incorporating imitation signals only when they are compatible with task progress, yielding an adaptive, geometry-aware update that preserves task-feasible descent and suppresses harmful imitation under misalignment. We provide theoretical analysis of gradient conflict and task-priority stationary points, and validate our claims through
humanoid control experiments demonstrating robust task performance with consistent motion style under noisy demonstrations.
Read the full abstract ▾
All authors Ziang Zheng, Kai Feng, Yi Nie, Shentao Qin
Robotics
Sim-and-Human Co-training for Data-Efficient and Scene-Generalizable Bimanual Manipulation
Kaipeng Fang, Weiqing Liang, Yuyang Li et al. (+5 more)
Real-
robot demonstrations are prohibitively expensive, while simulation data and real-world human demonstrations are both scalable but each leaves a distinct gap: simulation suffers from a sim-to-real visual gap, and human data suffers from a human-to-
robot embodiment gap. In this work, we identify a natural yet underexplored complementarity between these sources: simulation contributes
robot-valid actions absent in human data, while human data provides real-world observations that simulation struggles to render. Building on this insight, we present SimHum, a co-training recipe that extracts kinematic priors from simulation and visual priors from human observations, then fine-tunes on a small real-
robot dataset. SimHum exhibits strong scene-generalizable and data-efficient capabilities. With only 80 real-
robot episodes per task, it achieves 62.5% success on held-out OOD scenes across four bimanual tabletop tasks, 53.7% higher than Real only in absolute success rate. Moreover, in a controlled data-collection study with matched collection time, SimHum improves over the best single-source pre-training baseline by 35.0% in absolute success rate. Project page: https://kaipengfang.github.io/sim-and-human/
Read the full abstract ▾
All authors Kaipeng Fang, Weiqing Liang, Yuyang Li, Ji Zhang, Pengpeng Zeng, Heng Tao Shen, Jingkuan Song, Lianli Gao
Robotics
Judgelight: Trajectory-Level Post-Optimization for Multi-Agent Path Finding via Closed-Subwalk Collapsing
Yimin Tang, Sven Koenig and Erdem B{\i}y{\i}k
Multi-
Agent Path Finding (MAPF) is an NP-hard problem with applications in warehouse automation and multi-
robot coordination. Learning-based MAPF solvers offer fast and scalable planning but often produce feasible trajectories that contain unnecessary or oscillatory movements. We propose Judgelight, a post-optimization layer that improves trajectory quality after a MAPF solver generates a feasible schedule. Judgelight collapses closed subwalks in
agents' trajectories to remove redundant movements while preserving all feasibility constraints. We formalize this process as MAPF-Collapse, prove that it is NP-hard, and present an exact optimization approach by formulating it as integer linear programming (ILP) problem. Experimental results show Judgelight consistently reduces solution cost by around 20%, particularly for learning-based solvers, producing trajectories that are better suited for real-world deployment.
Read the full abstract ▾