Robotics
Tim Windecker, Manthan Patel, Moritz Reuss et al. (+5 more)
Vision-
language models demonstrate unprecedented performance and generalization across a wide range of tasks and scenarios. Integrating these foundation models into robotic
navigation systems opens pathways toward building general-purpose
robots. Yet, evaluating these models'
navigation capabilities remains constrained by costly real-world trials, overly simplified simulations, and limited
benchmarks. We introduce NaviTrace, a high-quality Visual Question Answering
benchmark where a model receives an instruction and embodiment type (human, legged
robot, wheeled
robot, bicycle) and must output a 2D
navigation trace in image space. Across 1000 scenarios and more than 3000 expert traces, we systematically evaluate eight state-of-the-art VLMs using a newly introduced semantic-aware trace score. This metric combines Dynamic Time Warping distance, goal endpoint error, and embodiment-conditioned penalties derived from per-pixel semantics and correlates with human preferences. Our evaluation reveals consistent gap to human performance caused by poor spatial grounding and goal localization. NaviTrace establishes a scalable and reproducible
benchmark for real-world robotic
navigation. The
benchmark and leaderboard can be found at https://leggedrobotics.github.io/navitrace_webpage/.
Read the full abstract ▾
All authors Tim Windecker, Manthan Patel, Moritz Reuss, Richard Schwarzkopf, Cesar Cadena, Rudolf Lioutikov, Marco Hutter, Jonas Frey
Robotics
Design for One, Deploy for Many: Navigating Tree Mazes with Multiple Agents
Jahir Argote-Gerald, Genki Miyauchi, Julian Rau et al. (+2 more)
Maze-like environments, such as cave and pipe networks, pose unique challenges for multiple
robots to coordinate, including communication constraints and congestion. To address these challenges, we propose a distributed multi-
agent maze traversal
algorithm for environments that can be represented by acyclic graphs. It uses a leader-switching mechanism where one
agent, assuming a head role, employs any single-
agent maze solver while the other
agents each choose an
agent to follow. The head role gets transferred to neighboring
agents where necessary, ensuring it follows the same path as a single
agent would. The multi-
agent maze traversal
algorithm is evaluated in simulations with groups of up to 300
agents, various maze sizes, and multiple single-
agent maze solvers. It is compared against strategies that are naïve, or assume either global communication or full knowledge of the environment. The
algorithm outperforms the naïve strategy in terms of makespan and sum-of-fuel. It is superior to the global-communication strategy in terms of makespan but is inferior to it in terms of sum-of-fuel. The findings suggest it is asymptotically equivalent to the full-knowledge strategy with respect to either metric. Moreover, real-world experiments with up to 20 Pi-puck
robots confirm the feasibility of the approach.
Read the full abstract ▾
All authors Jahir Argote-Gerald, Genki Miyauchi, Julian Rau, Paul Trodden, Roderich Gross
Robotics
Leveraging Foundation Models for Enhancing Robot Perception and Action
Reihaneh Mirjalili
This thesis investigates how foundation models can be systematically leveraged to enhance robotic capabilities, enabling more effective localization, interaction, and manipulation in unstructured environments. The work is structured around four core lines of inquiry, each addressing a fundamental challenge in robotics while collectively contributing to a cohesive framework for semantics-aware robotic intelligence.
Read the full abstract ▾
Robotics
Running VLAs at Real-time Speed
Yunchao Ma, Yizhuang Zhou, Yunhuan Yang et al. (+2 more)
In this paper, we show how to run pi0-level multi-view VLA at 30Hz frame rate and at most 480Hz trajectory frequency using a single consumer GPU. This enables dynamic and real-time tasks that were previously believed to be unattainable by large VLA models. To achieve it, we introduce a bag of strategies to eliminate the overheads in model inference. The real-world experiment shows that the pi0 policy with our strategy achieves a 100% success rate in grasping a falling pen task. Based on the results, we further propose a full streaming inference framework for real-time
robot control of VLA. Code is available at https://github.com/Dexmal/realtime-vla.
Read the full abstract ▾
All authors Yunchao Ma, Yizhuang Zhou, Yunhuan Yang, Tiancai Wang, Haoqiang Fan
Robotics
Hybrid Consistency Policy: Decoupling Multi-Modal Diversity and Real-Time Efficiency in Robotic Manipulation
Qianyou Zhao, Yuliang Shen, Xuanran Zhai et al. (+5 more)
In visuomotor policy learning,
diffusion-based imitation learning has become widely adopted for its ability to capture diverse behaviors. However, approaches built on ordinary and
stochastic denoising processes struggle to jointly achieve fast sampling and strong multi-modality. To address these challenges, we propose the Hybrid Consistency Policy (HCP). HCP runs a short
stochastic prefix up to an adaptive switch time, and then applies a one-step consistency jump to produce the final action. To align this one-jump generation, HCP performs time-varying consistency
distillation that combines a trajectory-consistency objective to keep neighboring predictions coherent and a denoising-matching objective to improve local fidelity. In both simulation and on a real
robot, HCP with 25 SDE steps plus one jump approaches the 80-step DDPM teacher in accuracy and mode coverage while significantly reducing latency. These results show that multi-modality does not require slow inference, and a switch time decouples mode retention from speed. It yields a practical accuracy efficiency trade-off for
robot policies.
Read the full abstract ▾
All authors Qianyou Zhao, Yuliang Shen, Xuanran Zhai, Ce Hao, Duidi Wu, Jin Qi, Jie Hu, Qiaojun Yu
Robotics
Posterior-driven Heuristic Support Adaptation in a Probabilistic Treatment of Real2Sim2Real for Vision-Driven Deformable Linear Object Manipulation
Georgios Kamaras, Craig Innes and Subramanian Ramamoorthy
Likelihood-free inference (LFI) enables system identification in complex tasks via black-box modelling, abstracting nonlinearity and stochasticity, and infers a domain distribution for adapting
agents to parametric deployment conditions. LFI assumes an arbitrary support for sampling, which remains fixed as the initial generic prior is refined to increasingly descriptive posteriors. Misspecified support can therefore yield suboptimal yet overconfident posteriors. We address this issue by using the posterior of an inference step to guide the adaptation of the support using three illustrative heuristics: EDGE, MODE, and CENTRE. Each heuristic interprets the updated belief and enables support adaptation alongside posterior inference. For illustrative purposes, we first study misspecified support in LFI and evaluate the utility of our heuristics using
stochastic dynamical
benchmarks. We then evaluate posterior-driven heuristic support adaptation for parameter inference and policy learning in a dynamic deformable linear object (DLO) manipulation task. Inference results in a finer length and stiffness classification for a parametric set of DLOs. When the resulting posteriors are used as domain distributions for sim-based policy learning, they lead to more robust object-centric
agent performance.
Read the full abstract ▾
Robotics
Hybrid DQN-TD3 Reinforcement Learning for Autonomous Navigation in Dynamic Environments
Xiaoyi He, Danggui Chen, Zhenshuo Zhang et al. (+1 more)
This paper presents a hierarchical path-planning and control framework that combines a high-level Deep Q-Network (DQN) for discrete sub-goal selection with a low-level Twin Delayed Deep Deterministic Policy Gradient (TD3) controller for continuous actuation. The high-level module selects behaviors and sub-goals; the low-level module executes smooth velocity commands. We design a practical reward shaping scheme (direction, distance, obstacle avoidance, action smoothness, collision penalty, time penalty, and progress), together with a
LiDAR-based safety gate that prevents unsafe motions. The system is implemented in ROS + Gazebo (TurtleBot3) and evaluated with PathBench metrics, including success rate, collision rate, path efficiency, and re-planning efficiency, in dynamic and partially observable environments. Experiments show improved success rate and sample efficiency over single-
algorithm baselines (DQN or TD3 alone) and rule-based planners, with better generalization to unseen obstacle configurations and reduced abrupt control changes. Code and evaluation scripts are available at the project repository.
Read the full abstract ▾
All authors Xiaoyi He, Danggui Chen, Zhenshuo Zhang, Zimeng Bai
Robotics
REALMS2 – Resilient Exploration And Lunar Mapping System 2 – A Comprehensive Approach
Dave van der Meer, Loïck P. Chovet, Gabriel M. Garcia et al. (+2 more)
The European Space Agency (ESA) and the European Space Resources Innovation Centre (ESRIC) created the Space Resources Challenge to invite researchers and companies to propose innovative solutions for Multi-
Robot Systems (MRS) space prospection. This paper proposes the Resilient Exploration And Lunar Mapping System 2 (REALMS2), a MRS framework for planetary prospection and mapping. Based on
Robot Operating System version 2 (ROS 2) and enhanced with Visual Simultaneous Localisation And Mapping (vSLAM) for map generation, REALMS2 uses a mesh network for a robust ad hoc network. A single graphical user interface (GUI) controls all the rovers, providing a simple overview of the robotic mission. This system is designed for heterogeneous multi-
robot exploratory missions, tackling the challenges presented by extraterrestrial environments. REALMS2 was used during the second field test of the ESA-ESRIC Challenge and allowed to map around 60% of the area, using three homogeneous rovers while handling communication delays and blackouts.
Read the full abstract ▾
All authors Dave van der Meer, Loïck P. Chovet, Gabriel M. Garcia, Abhishek Bera, Miguel A. Olivares-Mendez
Robotics
A Sliding-Window Filter for Online Continuous-Time Continuum Robot State Estimation
Spencer Teetaert, Sven Lilge, Jessica Burgner-Kahrs et al. (+1 more)
Stochastic state estimation methods for continuum
robots (CRs) often struggle to balance accuracy and computational efficiency. While several recent works have explored sliding-window formulations for CRs, these methods are limited to simplified, discrete-time approximations and do not provide
stochastic representations. In contrast, current
stochastic filter methods must run at the speed of measurements, limiting their full potential. Recent works in continuous-time estimation techniques for CRs show a principled approach to addressing this runtime constraint, but are currently restricted to offline operation. In this work, we present a sliding-window filter (SWF) for continuous-time state estimation of CRs that improves upon the accuracy of a filter approach while enabling continuous-time methods to operate online, all while running at faster-than-real-time speeds. This represents the first
stochastic SWF specifically designed for CRs, providing a promising direction for future research in this area.
Read the full abstract ▾
All authors Spencer Teetaert, Sven Lilge, Jessica Burgner-Kahrs, Timothy D. Barfoot
Robotics · Cross-listed from cs.CV
Spiking Patches: Asynchronous, Sparse, and Efficient Tokens for Event Cameras
Christoffer Koo Øhrstrøm, Ronja Güldenring and Lazaros Nalpantidis
We propose tokenization of events and present a tokenizer, Spiking Patches, specifically designed for event cameras. Given a stream of asynchronous and spatially sparse events, our goal is to discover an event representation that preserves these properties. Prior works have represented events as frames or as voxels. However, while these representations yield high accuracy, both frames and voxels are synchronous and decrease the spatial sparsity. Spiking Patches gives the means to preserve the unique properties of event cameras and we show in our experiments that this comes without sacrificing accuracy. We evaluate our tokenizer using a GNN, PCN, and a
Transformer on gesture recognition and object detection.
Tokens from Spiking Patches yield inference times that are up to 3.4x faster than voxel-based
tokens and up to 10.4x faster than frames. We achieve this while matching their accuracy and even surpassing in some cases with absolute improvements up to 3.8 for gesture recognition and up to 1.4 for object detection. Thus, tokenization constitutes a novel direction in event-based
vision and marks a step towards methods that preserve the properties of event cameras.
Read the full abstract ▾
Robotics
FLYINGTRUST: A Benchmark for Quadrotor Navigation Across Scenarios and Vehicles
Gang Li, Chunlei Zhai, Teng Wang et al. (+3 more)
Visual
navigation algorithms for quadrotors often exhibit a large variation in performance when transferred across different vehicle platforms and scene geometries, which increases the cost and risk of field deployment. To support systematic early-stage evaluation, we introduce FLYINGTRUST, a high-fidelity, configurable benchmarking framework that measures how platform kinodynamics and scenario structure jointly affect
navigation robustness. FLYINGTRUST models vehicle capability with two compact, physically interpretable indicators: maximum thrust-to-weight ratio and axis-wise maximum angular acceleration. The
benchmark pairs a diverse scenario library with a heterogeneous set of real and virtual platforms and prescribes a standardized evaluation protocol together with a composite scoring method that balances scenario importance, platform importance and performance stability. We use FLYINGTRUST to compare representative optimization-based and learning-based
navigation approaches under identical conditions, performing repeated trials per platform-scenario combination and reporting uncertainty-aware metrics. The results reveal systematic patterns:
navigation success depends predictably on platform capability and scene geometry, and different
algorithms exhibit distinct preferences and failure modes across the evaluated conditions. These observations highlight the practical necessity of incorporating both platform capability and scenario structure into
algorithm design, evaluation, and selection, and they motivate future work on methods that remain robust across diverse platforms and scenarios.
Read the full abstract ▾
All authors Gang Li, Chunlei Zhai, Teng Wang, Shaun Li, Shangsong Jiang, Xiangwei Zhu
Robotics · Cross-listed from physics.soc-ph
Proxemics and Permeability of the Pedestrian Group
Saleh Albeaik, Faisal Alsallum and Mohamad Alrished
People tend to walk in groups, and interactions with those groups have a significant impact on crowd behavior and pedestrian traffic dynamics. Social norms can be seen as unwritten rules regulating people interactions in social settings. This article studies people interactions with groups and the emergence of group proxemics. Group zones, zone occupancy counts and people clearance from the group are studied using naturalistic data. Analysis indicate potential presence of three different zones in addition to the public zone. People tend to remain in the public zone and only progressively get closer to groups, and those closer approaches happen in a low frequency and for brief periods of time.
Read the full abstract ▾
Robotics
Adaptive Inverse Kinematics Framework for Learning Variable-Length Tool Manipulation in Robotics
Prathamesh Kothavale and Sravani Boddepalli
Conventional
robots possess a limited understanding of their kinematics and are confined to preprogrammed tasks, hindering their ability to leverage tools efficiently. Driven by the essential components of tool usage - grasping the desired outcome, selecting the most suitable tool, determining optimal tool orientation, and executing precise manipulations - we introduce a pioneering framework. Our novel approach expands the capabilities of the
robot's inverse kinematics solver, empowering it to acquire a sequential repertoire of actions using tools of varying lengths. By integrating a simulation-learned action trajectory with the tool, we showcase the practicality of transferring acquired skills from simulation to real-world scenarios through comprehensive experimentation. Remarkably, our extended inverse kinematics solver demonstrates an impressive error rate of less than 1 cm. Furthermore, our trained policy achieves a mean error of 8 cm in simulation. Noteworthy, our model achieves virtually indistinguishable performance when employing two distinct tools of different lengths. This research provides an indication of potential advances in the exploration of all four fundamental aspects of tool usage, enabling
robots to master the intricate art of tool manipulation across diverse tasks.
Read the full abstract ▾
Robotics
RoboOS-NeXT: A Unified Memory-based Framework for Lifelong, Scalable, and Robust Multi-Robot Collaboration
Huajie Tan, Cheng Chi, Xiansheng Chen et al. (+21 more)
The proliferation of collaborative
robots across diverse tasks and embodiments presents a central challenge: achieving lifelong adaptability, scalable coordination, and robust scheduling in multi-
agent systems. Existing approaches, from
vision-
language-action (VLA) models to hierarchical frameworks, fall short due to their reliance on limited or dividual-
agent memory. This fundamentally constrains their ability to learn over long horizons, scale to heterogeneous teams, or recover from failures, highlighting the need for a unified
memory representation. To address these limitations, we introduce RoboOS-NeXT, a unified
memory-based framework for lifelong, scalable, and robust multi-
robot collaboration. At the core of RoboOS-NeXT is the novel Spatio-Temporal-Embodiment
Memory (STEM), which integrates spatial scene geometry, temporal event history, and embodiment profiles into a shared representation. This
memory-centric design is integrated into a
brain-cerebellum framework, where a high-level
brain model performs global planning by retrieving and updating STEM, while low-level controllers execute actions locally. This closed loop between cognition,
memory, and execution enables dynamic task allocation, fault-tolerant collaboration, and consistent state synchronization. We conduct extensive experiments spanning complex coordination tasks in restaurants, supermarkets, and households. Our results demonstrate that RoboOS-NeXT achieves superior performance across heterogeneous embodiments, validating its effectiveness in enabling lifelong, scalable, and robust multi-
robot collaboration. Project website: https://flagopen.github.io/RoboOS/
Read the full abstract ▾
All authors Huajie Tan, Cheng Chi, Xiansheng Chen, Yuheng Ji, Zhongxia Zhao, Xiaoshuai Hao, Yaoxu Lyu, Mingyu Cao, Junkai Zhao, Huaihai Lyu, Enshen Zhou, Ning Chen, Yankai Fu, Cheng Peng, Wei Guo, Dong Liang, Zhuo Chen, Mengsi Lyu, Chenrui He, Yulong Ao, Yonghua Lin, Pengwei Wang, Zhongyuan Wang, Shanghang Zhang
Robotics · Cross-listed from eess.SY
Efficient Collision-Avoidance Constraints for Ellipsoidal Obstacles in Optimal Control: Application to Path-Following MPC and UAVs
David Leprich, Mario Rosenfelder, Markus Herrmann-Wicklmayr et al. (+3 more)
This article proposes a modular optimal control framework for local three-dimensional ellipsoidal obstacle avoidance, exemplarily applied to model predictive path-following control. Static as well as moving obstacles are considered. Central to the approach is a computationally efficient and continuously differentiable condition for detecting collisions with ellipsoidal obstacles. A novel two-stage optimization approach mitigates numerical issues arising from the structure of the resulting optimal control problem. The effectiveness of the approach is demonstrated through simulations and real-world experiments with the Crazyflie quadrotor. This represents the first hardware demonstration of an MPC controller of this kind for
UAVs in a three-dimensional task.
Read the full abstract ▾
All authors David Leprich, Mario Rosenfelder, Markus Herrmann-Wicklmayr, Kathrin Flaßkamp, Peter Eberhard, Henrik Ebel
Robotics
Human-in-the-loop Online Rejection Sampling for Robotic Manipulation
Guanxing Lu, Rui Zhao, Haitao Lin et al. (+2 more)
Reinforcement learning (RL) is widely used to produce robust robotic manipulation policies, but fine-tuning
vision-
language-action (VLA) models with RL can be unstable due to inaccurate value estimates and sparse supervision at intermediate steps. In contrast, imitation learning (IL) is easy to train but often underperforms due to its offline nature. In this paper, we propose Hi-ORS, a simple yet effective post-training method that utilizes rejection sampling to achieve both training stability and high robustness. Hi-ORS stabilizes value estimation by filtering out negatively rewarded samples during online fine-tuning, and adopts a reward-weighted supervised training objective to provide dense intermediate-step supervision. For systematic study, we develop an asynchronous inference-training framework that supports flexible online human-in-the-loop corrections, which serve as explicit guidance for learning error-recovery behaviors. Across three real-world tasks and two embodiments, Hi-ORS fine-tunes a pi-base policy to master contact-rich manipulation in just 1.5 hours of real-world training, outperforming RL and IL baselines by a substantial margin in both effectiveness and efficiency. Notably, the fine-tuned policy exhibits strong test-time scalability by reliably executing complex error-recovery behaviors to achieve better performance.
Read the full abstract ▾
All authors Guanxing Lu, Rui Zhao, Haitao Lin, He Zhang, Yansong Tang
Robotics · Cross-listed from cs.LG
CorVS+: Correspondence-Driven Association of Video Trajectories and Sensors for Identity-Aware Person Localization in Warehouses
Kazuma Kano, Yuki Mori, Shin Katayama et al. (+3 more)
Logistics warehouses have struggled with labor shortages, but the inbound processes remain particularly human-powered. Worker location data is a key to higher productivity in such cases. Fixed cameras are a promising tool for localization, as they also offer valuable environmental information such as package status. However, identifying individuals from visual data alone is often impractical. To enable identity-aware localization, prior studies have attempted to identify people in
videos by associating their trajectories with wearable sensor measurements. Although this appearance-independent approach has several advantages, existing methods may fail under real-world conditions. Therefore, we propose CorVS+, a novel data-driven person identification framework based on the correspondence between visual tracking trajectories and sensor measurements. Firstly, our deep learning model predicts the correspondence probabilities and reliabilities for every pair of a trajectory and sensor measurements. Secondly, our
algorithm matches the pairs over time based on the model predictions. We developed a
dataset comprising 27 hours of sensor measurements and 38 km of trajectories in a warehouse. This
dataset covers actual activities and challenging situations, such as multiple stationary workers inspecting items. The evaluation indicated the superiority of CorVS+ over existing methods and the effectiveness of its unique designs for industrial-scale settings. The model and
dataset will be available at https://doi.org/10.5281/zenodo.17745683.
Read the full abstract ▾
All authors Kazuma Kano, Yuki Mori, Shin Katayama, Kenta Urano, Takuro Yonezawa, Nobuo Kawaguchi
Robotics
Towards Reinforcement Learning Based Log Loading Automation
Ilya Kurinov, Miroslav Ivanov, Grzegorz Orzechowski et al. (+1 more)
Forestry forwarders play a central role in mechanized timber harvesting by picking up and moving logs from the felling site to a processing area or a secondary transport vehicle. Forwarder operation is challenging and physically and mentally exhausting for the operator who must control the machine in remote areas for prolonged periods of time. Therefore, even partial automation of the process may reduce stress on the operator. This study focuses on continuing previous research efforts in application of reinforcement learning
agents in automating log handling process, extending the task from grasping which was studied in previous research to full log loading operation. The resulting
agent will be capable to automate a full loading procedure from locating and grappling to transporting and delivering the log to a forestry forwarder bed. To train the
agent, a trailer type forestry forwarder simulation model in NVIDIA's Isaac Gym and a virtual environment for a typical log loading scenario were developed. With reinforcement learning
agents and a curriculum learning approach, the trained
agent may be a stepping stone towards application of reinforcement learning
agents in automation of the forestry forwarder. The
agent learnt grasping a log in a random position from grapple's random position and transport it to the bed with 94% success rate of the best performing
agent.
Read the full abstract ▾
All authors Ilya Kurinov, Miroslav Ivanov, Grzegorz Orzechowski, Aki Mikkola
Robotics
Cooperative Task Spaces for Multi-Arm Manipulation Control based on Similarity Transformations
Tobias Löw, Cem Bilaloglu and Sylvain Calinon
Many tasks in human environments require collaborative behavior between multiple kinematic chains, either to provide additional support for carrying big and bulky objects or to enable the dexterity that is required for in-hand manipulation. Since these complex systems often have a very high number of degrees of freedom coordinating their movements is notoriously difficult to model. In this article, we present the derivation of the theoretical foundations for cooperative task spaces of multi-arm robotic systems based on
geometric primitives defined using conformal
geometric algebra. Based on the similarity transformations of these cooperative
geometric primitives, we derive an abstraction of complex robotic systems that enables representing these systems in a way that directly corresponds to single-arm systems. By deriving the associated analytic and
geometric Jacobian
matrices, we then show the straightforward integration of our approach into classical control techniques rooted in operational space control. We demonstrate this using bimanual manipulators,
humanoids and multi-fingered hands in optimal control experiments for reaching desired
geometric primitives and in teleoperation experiments using differential kinematics control. We then discuss how the
geometric primitives naturally embed nullspace structures into the controllers that can be exploited for introducing secondary control objectives. This work, represents the theoretical foundations of this cooperative manipulation control framework, and thus the experiments are presented in an abstract way, while giving pointers towards potential future applications.
Read the full abstract ▾
Robotics
AgriGS-SLAM: Orchard Mapping Across Seasons via Multi-View Gaussian Splatting SLAM
Mirko Usuelli, David Rapado-Rincon, Gert Kootstra et al. (+1 more)
Autonomous
robots in orchards require real-time 3D scene understanding despite repetitive row geometry, seasonal appearance changes, and wind-driven foliage motion. We present AgriGS-SLAM, a Visual–
LiDAR SLAM framework that couples direct
LiDAR odometry and loop closures with multi-camera 3D Gaussian Splatting (3DGS) rendering. Batch rasterization across complementary viewpoints recovers orchard structure under occlusions, while a unified gradient-driven map lifecycle executed between keyframes preserves fine details and bounds
memory. Pose refinement is guided by a probabilistic
LiDAR-based depth consistency term, back-propagated through the camera projection to tighten geometry-appearance coupling. We deploy the system on a field platform in apple and pear orchards across dormancy, flowering, and harvesting, using a standardized trajectory protocol that evaluates both training-view and novel-view synthesis to reduce 3DGS overfitting in evaluation. Across seasons and sites, AgriGS-SLAM delivers sharper, more stable reconstructions and steadier trajectories than recent state-of-the-art 3DGS-SLAM baselines while maintaining real-time performance on-tractor. While demonstrated in orchard monitoring, the approach can be applied to other outdoor domains requiring robust
multimodal perception.
Read the full abstract ▾
All authors Mirko Usuelli, David Rapado-Rincon, Gert Kootstra, Matteo Matteucci
Robotics
Thor: Towards Human-Inspired Whole-Body Reactions for Intense Contact-Rich Environments
Gangyang Li, Hongzhe Shi, Qing Shi et al. (+5 more)
Maintaining whole-body stability and motion tracking under large interaction forces remains challenging for
humanoids. We present Thor, a reinforcement learning framework for forceful
humanoid loco-manipulation. Thor jointly trains lower-body, waist, and upper-body policies with shared whole-body observations and body-specific rewards to coordinate locomotion and force adaptation, waist posture regulation, and upper-body motion tracking. We further introduce a force-adaptive torso-tilt (FAT2) objective that derives a load-dependent horizontal center-of-mass offset reference from quasi-static moment balance. Capacity-matched simulation blations show that the three-policy architecture improves tracking under large external force disturbances, while real-world ablations demonstrate that FAT2 increases peak pulling capability. On the Unitree G1, Thor achieves mean peak dual-hand pulling forces of 167.7 N and 145.5 N during backward and forward locomotion, exceeding the best-performing baseline by 68.9% and 74.7%, respectively. Real-world demonstrations include opening a fire door with one hand using approximately 60 N of pulling force and towing a 1.7-ton passenger car.
Read the full abstract ▾
All authors Gangyang Li, Hongzhe Shi, Qing Shi, Youhao Hu, Cong Ma, Zhongyuan Wang, Xinlong Wang, Shaqi Luo
Robotics
PHUMA: Physically Reliable Humanoid Locomotion Dataset
Kyungmin Lee, Sibeen Kim, Youngdo Lee et al. (+6 more)
Motion imitation is a promising approach for
humanoid locomotion, enabling
agents to acquire humanlike behaviors. Existing methods typically rely on high-quality motion capture
datasets such as AMASS, but these are scarce and expensive, limiting scalability and diversity. Recent studies attempt to scale data collection by converting large-scale internet
videos, exemplified by
Humanoid-X. However, they often suffer from physical artifacts such as floating, penetration, and foot skating, which hinder stable imitation. To address this, we introduce PHUMA, a Physically Reliable
HUMAnoid locomotion
dataset produced by a two-stage pipeline combining physics-aware curation and physics-constrained retargeting, aggregating both motion capture and internet
video into a physically reliable, 73-hour corpus. On motion tracking
benchmarks, PHUMA-trained policies achieve higher success rates than those trained on AMASS and
Humanoid-X, and successfully transfer zero-shot to a real Unitree G1. The code is available at https://davian-robotics.github.io/PHUMA.
Read the full abstract ▾
All authors Kyungmin Lee, Sibeen Kim, Youngdo Lee, Minho Park, Hyunseung Kim, Dongyoon Hwang, Donghu Kim, Hojoon Lee, Jaegul Choo
Robotics
Self-localization on a 3D map by fusing global and local features from a monocular camera
Satoshi Kikuchi, Masaya Kato and Tsuyoshi Tasaki
Self-localization on a 3D map by using an inexpensive
monocular camera is required to realize autonomous driving. Self-localization based on a camera often uses a convolutional neural network (CNN) that can extract local features that are calculated by nearby pixels. However, when dynamic obstacles, such as people, are present, CNN does not work well. This study proposes a new method combining CNN with
Vision Transformer, which excels at extracting global features that show the relationship of patches on whole image. Experimental results showed that, compared to the state-of-the-art method (SOTA), the accuracy improvement rate in a CG
dataset with dynamic obstacles is 1.5 times higher than that without dynamic obstacles. Moreover, the self-localization error of our method is 20.1% smaller than that of SOTA on public
datasets. Additionally, our
robot using our method can localize itself with 7.51cm error on average, which is more accurate than SOTA.
Read the full abstract ▾
Robotics
ART-TEB: Adaptive Trajectory Planning for Mobile Robots in Cluttered Environments
Hahjin Lee and Young J. Kim
Trajectory planning for mobile
robots remains a major challenge, particularly in cluttered environments, where existing planning
algorithms often fail or produce suboptimal paths. To address this issue, we propose an adaptive trajectory refinement
algorithm, ART-TEB, comprising two main stages. First, to ensure safety at the path-segment level, a segment-wise conservative collision test is applied, recursively subdividing risky path segments until the collision risk is eliminated. Second, to guarantee pose-level safety, pose correction based on separation direction and line search is applied, ensuring that each pose in the trajectory is collision-free and maximally clear from obstacles. Simulation results demonstrate that the ART-TEB achieves up to 3.12x higher success rates and up to 23.7x faster average planning times than state-of-the-art approaches. Furthermore, real-world experiments confirm that the
robot can safely pass through highly constrained environments while maintaining rapid planning performance.
Read the full abstract ▾
Robotics
Kinodynamic Task and Motion Planning using VLM-guided and Interleaved Sampling
Minseo Kwon and Young J. Kim
Task and Motion Planning (TAMP) integrates high-level task planning with low-level motion feasibility, but existing methods are costly in long-horizon problems due to excessive motion sampling. While LLMs provide commonsense priors, they lack 3D spatial
reasoning and cannot ensure
geometric or dynamic feasibility. We propose a kinodynamic TAMP planner based on a hybrid state tree that uniformly represents symbolic and numeric states during planning, enabling task and motion decisions to be jointly decided. Kinodynamic constraints embedded in the TAMP problem are verified by an off-the-shelf motion planner and physics simulator, and a VLM guides exploring a TAMP solution and backtracks the search based on visual rendering of the states. Experiments on the simulated domains and in the real world show 32.14% - 1166.67% increased average success rates compared to traditional and LLM-based TAMP planners and reduced planning time on complex problems, with ablations further highlighting the benefits of VLM backtracking. More details are available at https://graphics.ewha.ac.kr/kinodynamicTAMP/.
Read the full abstract ▾
Robotics
Embodied Intelligence for Advanced Bioinspired Microrobotics: Examples and Insights
Nestor O. Perez-Arancibia
The term
embodied intelligence (EI) conveys the notion that body morphology, material properties, interaction with the environment, and control strategies can be purposefully integrated into the process of robotic design to generate intelligent behavior; in particular, locomotion and
navigation. In this paper, we discuss EI as a design principle for advanced microrobotics, with a particular focus on co-design – the simultaneous and interdependent development of physical structure and behavioral function. To illustrate the contrast between EI-inspired systems and traditional architectures that decouple sensing, computation, and actuation, we present and discuss a collection of
robots developed by the author and his team at the Autonomous Microrobotic Systems Laboratory (AMSL). These
robots exhibit intelligent behavior that emerges from their structural dynamics and the physical interaction between their components and with the environment. Platforms such as the Bee++, RoBeetle, SMALLBug, SMARTI, WaterStrider, VLEIBot+, and FRISSHBot exemplify how feedback loops, decision logics, sensing mechanisms, and smart actuation strategies can be embedded into the physical properties of the robotic system itself. Along these lines, we contend that co-design is not only a method for empirical optimization under constraints, but also an enabler of EI, offering a scalable and robust alternative to classical control for robotics at the mm-to-cm-scale.
Read the full abstract ▾
Robotics · Cross-listed from cs.CV
Exploring Object-Aware Attention Guided Frame Association for RGB-D SLAM
Ali Caglayan, Nevrez Imamoglu, Oguzhan Guclu et al. (+3 more)
Attention models have recently emerged as a powerful approach, demonstrating significant progress in various fields. Visualization techniques, such as class activation mapping, provide visual insights into the
reasoning of convolutional neural networks (CNNs). Using network gradients, it is possible to identify regions where the network pays
attention during image recognition tasks. Furthermore, these gradients can be combined with CNN features to localize more generalizable, task-specific attentive (salient) regions within scenes. However, explicit use of this gradient-based
attention information integrated directly into CNN representations for semantic object understanding remains limited. Such integration is particularly beneficial for visual tasks like simultaneous localization and mapping (SLAM), where CNN representations enriched with spatially attentive object locations can enhance performance. In this work, we propose utilizing task-specific network
attention for RGB-D indoor SLAM. Specifically, we integrate layer-wise
attention information derived from network gradients with CNN feature representations to improve frame association performance. Experimental results indicate improved performance compared to baseline methods, particularly for large environments.
Read the full abstract ▾
All authors Ali Caglayan, Nevrez Imamoglu, Oguzhan Guclu, Ali Osman Serhatoglu, Ahmet Burak Can, Ryosuke Nakamura
Robotics
Beyond the Uncanny Valley: A Mixed-Method Investigation of Anthropomorphism in Protective Responses to Robot Abuse
Fan Yang, Lingyao Li, Yaxin Hu et al. (+2 more)
Robots with anthropomorphic features are increasingly shaping how humans perceive and morally engage with them. Our research investigates how different levels of anthropomorphism influence protective responses to
robot abuse, extending the Computers as Social Actors (CASA) and uncanny valley theories into a moral domain. In an experiment, we invite 201 participants to view
videos depicting abuse toward a
robot with low (Spider), moderate (Two-Foot), or high (
Humanoid) anthropomorphism. To provide a comprehensive analysis, we triangulate three modalities: self-report surveys measuring
emotions and uncanniness, physiological data from automated facial expression analysis, and qualitative reflections. Findings indicate that protective responses are not linear. The moderately anthropomorphic Two-Foot
robot, rated highest in eeriness and "spine-tingling" sensations consistent with the uncanny valley, elicited the strongest physiological anger expressions. Self-reported anger and guilt are significantly higher for both the Two-Foot and
Humanoid robots compared to the Spider. Qualitative findings further reveal that as anthropomorphism increases, moral
reasoning shifts from technical assessments of property damage to condemnation of the abuser's character, while governance proposals expand from property law to calls for quasi-animal rights and broader societal responsibility. These results suggest that the uncanny valley does not dampen moral concern but paradoxically heightens protective impulses, offering critical implications for
robot design, policy, and future legal frameworks.
Read the full abstract ▾
All authors Fan Yang, Lingyao Li, Yaxin Hu, Michael Rodgers, Renkai Ma
Robotics
I don't Want You to Die: A Shared Responsibility Framework for Safeguarding Child-Robot Companionship
Fan Yang, Renkai Ma, Yaxin Hu et al. (+2 more)
Social
robots like Moxie are designed to form strong emotional bonds with children, but their abrupt discontinuation can cause significant struggles and distress to children. When these services end, the resulting harm raises complex questions of who bears responsibility when children's emotional bonds are broken. Using the Moxie shutdown as a case study through a qualitative survey of 72 U.S. participants, our findings show that the responsibility is viewed as a shared duty across the
robot company, parents, developers, and government. However, these attributions varied by political ideology and parental status of whether they have children. Participants' perceptions of whether the
robot service should continue are highly polarized; supporters propose technical, financial, and governmental pathways for continuity, while opponents cite business realities and risks of unhealthy emotional dependency. Ultimately, this research contributes an empirically grounded shared responsibility framework for safeguarding child-
robot companionship by detailing how accountability is distributed and contested, informing concrete design and policy implications to mitigate the emotional harm of
robot discontinuation.
Read the full abstract ▾
All authors Fan Yang, Renkai Ma, Yaxin Hu, Michael Rodgers, Lingyao Li
Robotics
Morphology-Aware Graph Reinforcement Learning for Tensegrity Robot Locomotion
Chi Zhang, Mingrui Li, Wenzhe Tong et al. (+1 more)
Tensegrity
robots combine rigid rods and elastic cables, offering high resilience and deployability but at the same time posing major challenges for locomotion control due to their underactuated and highly coupled dynamics. This paper introduces a morphology-aware reinforcement learning framework that integrates a graph neural network (GNN) into the Soft Actor-Critic (SAC)
algorithm. By representing the
robot's physical
topology as a graph, the proposed GNN-based policy captures coupling among components, enabling faster and more stable learning than conventional multilayer perceptron (MLP) policies. The method is validated on a physical 3-bar tensegrity
robot across three locomotion primitives, including straight-line tracking and bidirectional turning. It shows superior sample efficiency, robustness to noise and stiffness variations, and improved trajectory accuracy. Additionally, the learned policies transfer directly from simulation to hardware without fine-tuning, achieving stable real-world locomotion. These results demonstrate the advantages of incorporating structural priors into reinforcement learning for tensegrity
robot control.
Read the full abstract ▾
All authors Chi Zhang, Mingrui Li, Wenzhe Tong, Xiaonan Huang