Robotics
The SLAM Confidence Trap
Sebastian Sansoni and Santiago Ramón Tosetti Sanz
The SLAM community has fallen into a "Confidence Trap" by prioritizing
benchmark scores over principled uncertainty estimation. This yields systems that are geometrically accurate but probabilitistically inconsistent and brittle. We advocate for a paradigm shift where the consistent, real-time computation of uncertainty becomes a primary metric of success.
Read the full abstract ▾
Robotics
PACT-WAM: Predicting Actions and Visual Foresight with Compact Temporal Encoding for Robot Manipulation
Yushan Liu, Jingjing Fan, Shoujie Li et al. (+3 more)
Robot manipulation uses temporal context to select actions and visual foresight to assess their consequences, yet dense representations of past and future observations incur substantial processing costs. We introduce PACT-WAM, a world-action model that jointly generates a 16-step action trajectory and its temporally corresponding visual forecast through conditional flow sampling. Hierarchical history encoding assigns coarse spatial representations to earlier observations and finer representations to recent ones, retaining 16 observations with 256
tokens per view, 75% fewer than dense encoding of the same frames. A shared flow module jointly updates continuous action and visual states through two modality-specific heads under transition-wise causal
attention, and a TiTok-VAE decoder reconstructs multi-view future images from the visual latents. Decoded forecasts also support Proposal Review (PR), a
vision-
language model component for execution-prefix selection and proposal rejection. Without PR, PACT-WAM achieves average success rates of 98.6%, 92.3%, and 78.0% on LIBERO, RoboTwin 2.0, and real-world Piper tasks, respectively. PR provides a test-time enhancement, raising these rates to 99.5%, 93.4%, and 86.7%. Ablations show that hierarchical history allocation and joint action-visual generation improve control success, while analyses of visual capacity and forecast-guided execution characterize the trade-offs between success and proposal-generation cost.
Read the full abstract ▾
All authors Yushan Liu, Jingjing Fan, Shoujie Li, Yifan Xie, Xiao-Ping Zhang, Wenbo Ding
Robotics
MORPH Wheel: A Passive Variable-Radius Wheel Embedding Mechanical Behavior Logic for Input-Responsive Transformation
JaeHyung Jang, JuYeong Seo, Dae-Young Lee et al. (+1 more)
This paper introduces the Mechacnially prOgrammed Radius-adjustable PHysical (MORPH) wheel, a fully passive variable-radius wheel that embeds mechanical behavior logic for torque-responsive transformation. Unlike conventional variable transmission systems relying on actuators, sensors, and active control, the MORPH wheel achieves passive adaptation solely through its geometry and compliant structure. The design integrates a torque-response coupler and spring-loaded connecting struts to mechanically adjust the wheel radius between 80 mm and 45 mm in response to input torque, without any electrical components. The MORPH wheel provides three unique capabilities rarely achieved simultaneously in previous passive designs: (1) bidirectional operation with unlimited rotation through a symmetric coupler; (2) high torque capacity exceeding 10 N with rigid power transmission in drive mode; and (3) precise and repeatable transmission ratio control governed by deterministic kinematics. A comprehensive analytical model was developed to describe the wheel's mechanical behavior logic, establishing threshold conditions for mode switching between direct drive and radius transformation. Experimental validation confirmed that the measured torque-radius and force-displacement characteristics closely follow theoretical predictions across wheel weights of 1.8-2.8kg.
Robot-level demonstrations on varying loads (0-25kg), slopes, and unstructured terrains further verified that the MORPH wheel passively adjusts its radius to provide optimal transmission ratio. The MORPH wheel exemplifies a mechanically programmed structure,
embedding intelligent, context-dependent behavior directly into its physical design. This approach offers a new paradigm for passive variable transmission and mechanical intelligence in robotic mobility systems operating in unpredictable or control-limited environments.
Read the full abstract ▾
All authors JaeHyung Jang, JuYeong Seo, Dae-Young Lee, Jee-Hwan Ryu
Robotics
A Dialogue-Based Human-Robot Interaction Protocol for Wheelchair and Robotic Arm Integrated Control
Guangping Liu, Nicholas Hawkins, Billy Madden et al. (+2 more)
People with lower and upper body disabilities can benefit from wheelchairs and robotic arms to improve mobility and independence. Prior assistive interfaces, such as touchscreens and voice-driven predefined commands, often remain unintuitive and struggle to capture complex user intent. We propose a natural, dialogue based human
robot interaction protocol that simulates an intelligent
agent capable of communicating with users to understand intent and execute assistive actions. In a pilot study, five participants completed five assistive tasks (cleaning, drinking, feeding, drawer opening, and door opening) through dialogue-based interaction with a wheelchair and robotic arm. As a baseline, participants were required to open a door using the manual control (a wheelchair joystick and a game controller for the arm) and complete a questionnaire to gather their feedback. By analyzing the post-study questionnaires, we found that most participants enjoyed the dialogue-based interaction and assistive
robot autonomy.
Read the full abstract ▾
All authors Guangping Liu, Nicholas Hawkins, Billy Madden, Tipu Sultan, Madi Babaiasl
Robotics
Coupled Local and Global World Models for Efficient First Order RL
Joseph Amigo, Rooholla Khorrambakht, Nicolas Mansard et al. (+1 more)
World models offer a promising avenue for more faithfully capturing complex dynamics, including contacts and non-
rigidity, as well as complex sensory information, such as visual perception, in situations where standard simulators struggle. However, these models are computationally complex to evaluate, posing a challenge for popular RL approaches that have been successfully used with simulators to solve complex locomotion tasks but yet struggle with manipulation. This paper introduces a method that bypasses simulators entirely, training RL policies inside world models learned from
robots' interactions with real environments. At its core, our approach enables policy training with large-scale
diffusion models via a novel decoupled first-order gradient (FoG) method: a full-scale world model generates accurate forward trajectories, while a lightweight latent-space surrogate approximates its local dynamics for efficient gradient computation. This coupling of a local and global world model ensures high-fidelity unrolling alongside computationally tractable differentiation. We demonstrate the efficacy of our method on the Push-T manipulation task, where it significantly outperforms PPO in sample efficiency. We further evaluate our approach through an ego-centric object manipulation task with a quadruped. Together, these results demonstrate that learning inside data-driven world models is a promising pathway for solving hard-to-model RL tasks in image space without reliance on hand-crafted physics simulators.
Read the full abstract ▾
All authors Joseph Amigo, Rooholla Khorrambakht, Nicolas Mansard, Ludovic Righetti
Robotics · Cross-listed from cs.CV
Addressing the Waypoint-Action Gap in End-to-End Autonomous Driving via Vehicle Motion Models
Jorge Daniel Rodríguez-Vidal, Gabriel Villalonga, Diego Porres et al. (+1 more)
End-to-End Autonomous Driving (E2E-AD) systems are typically grouped by the nature of their outputs: (i) waypoint-based models that predict a future trajectory, and (ii) action-based models that directly output throttle, steer and brake. Most recent
benchmark protocols and training pipelines are waypoint-based, which makes action-based policies harder to train and compare, slowing their progress. To bridge this waypoint-action gap, we propose a novel, differentiable vehicle-model framework that rolls out predicted action sequences to their corresponding ego-frame waypoint trajectories while supervising in waypoint space. Our approach enables action-based architectures to be trained and evaluated, for the first time, within waypoint-based
benchmarks without modifying the underlying evaluation protocol. We extensively evaluate our framework across multiple challenging
benchmarks and observe consistent improvements over the baselines. In particular, on NAVSIM navhard our approach achieves state-of-the-art performance. Our code will be made publicly available upon acceptance.
Read the full abstract ▾
All authors Jorge Daniel Rodríguez-Vidal, Gabriel Villalonga, Diego Porres, Antonio M. López Peña
Robotics
Bioinspired Kirigami Capsule Robot for Minimally Invasive Gastrointestinal Biopsy
Ruizhou Zhao, Yichen Chu, Shuwei Zhao et al. (+3 more)
Wireless capsule endoscopy (WCE) has transformed gastrointestinal (GI) diagnostics by enabling noninvasive visualization of the digestive tract, yet its diagnostic yield remains constrained by the absence of biopsy capability, as histological analysis is still the gold standard for confirming disease. Conventional biopsy using forceps, needles, or rotating blades is invasive, limited in reach, and carries risks of perforation or mucosal trauma, while fluid- or microbiota-sampling capsules cannot provide structured tissue for pathology, leaving a critical gap in swallowable biopsy solutions. Here we present the Kiri-Capsule, a kirigami-inspired capsule
robot that integrates deployable PI-film flaps actuated by a compact dual-cam mechanism to achieve minimally invasive and repeatable tissue collection. The kirigami surface remains flat during locomotion but transforms into sharp protrusions upon cam-driven stretching, enabling controlled penetration followed by rotary scraping, with specimens retained in internal fan-shaped cavities. Bench tests confirmed that PI films exhibit a Young's modulus of approximately 20 MPa and stable deployment angles (about 34$^\circ$ at 15% strain), while ex vivo porcine studies demonstrated shallow penetration depths (median $\sim$0.61 mm, range 0.46–0.66 mm) and biopsy yields comparable to standard forceps (mean $\sim$10.9 mg for stomach and $\sim$18.9 mg for intestine), with forces within safe ranges reported for GI biopsy. These findings demonstrate that the Kiri-Capsule bridges passive imaging and functional biopsy, providing a swallowable, depth-controlled, and histology-ready solution that advances capsule-based diagnostics toward safe and effective clinical application.
Read the full abstract ▾
All authors Ruizhou Zhao, Yichen Chu, Shuwei Zhao, Wenchao Yue, Raymond Shing-Yan Tang, Hongliang Ren
Robotics · Cross-listed from cs.CV
AnyThermal: Towards Learning Universal Representations for Thermal Perception
Parv Maheshwari, Jay Karhade, Yogesh Chawla et al. (+8 more)
We present AnyThermal, a thermal backbone that captures robust task-agnostic thermal features suitable for a variety of tasks such as cross-modal place recognition, thermal
segmentation, and
monocular depth estimation using thermal images. Existing thermal backbones that follow task-specific training from small-scale data result in utility limited to a specific environment and task. Unlike prior methods, AnyThermal can be used for a wide range of environments (indoor, aerial, off-road, urban) and tasks, all without task-specific training. Our key insight is to distill the feature representations from visual foundation models such as DINOv2 into a thermal encoder using thermal data from these multiple environments. To bridge the diversity gap of the existing RGB-Thermal
datasets, we introduce the TartanRGBT platform, the first open-source data collection platform with synced RGB-Thermal image acquisition. We use this payload to collect the TartanRGBT
dataset - a diverse and balanced
dataset collected in 4 environments. We demonstrate the efficacy of AnyThermal and TartanRGBT, achieving state-of-the-art results with improvements of up to 36% across diverse environments and downstream tasks on existing
datasets.
Read the full abstract ▾
All authors Parv Maheshwari, Jay Karhade, Yogesh Chawla, Isaiah Adu, Florian Heisen, Andrew Porco, Andrew Jong, Yifei Liu, Santosh Pitla, Sebastian Scherer, Wenshan Wang
Robotics
Active Localization of Unstable Systems with Coarse Information
Ege Yuceel, Daniel Liberzon and Sayan Mitra
We study localization and control for unstable systems under coarse, single-bit sensing. Motivated by understanding the fundamental limitations imposed by such minimal feedback, we identify sufficient conditions under which the initial state can be recovered despite instability and extremely sparse measurements. Building on these conditions, we develop an active localization
algorithm that integrates a set-based
estimator with a control strategy derived from Voronoi partitions, which provably estimates the initial state while ensuring the
agent remains in informative regions. Under the derived conditions, the proposed approach guarantees exponential contraction of the initial-state uncertainty, and the result is further supported by numerical experiments. These findings can offer theoretical insight into localization in robotics, where sensing is often limited to coarse abstractions such as keyframes,
segmentations, or line-based features.
Read the full abstract ▾
Robotics
Dynamic Modeling, Parameter Identification and Numerical Analysis of Flexible Cables in Flexibly Connected Dual-AUV Systems
Kuo Chen, Minghao Dou, Qianqi Liu et al. (+8 more)
This research presents a dynamic modeling framework and parameter identification methods for describing the highly nonlinear behaviors of flexibly connected dual-AUV systems. The modeling framework is established based on the lumped mass method, integrating axial elasticity, bending stiffness, added mass and hydrodynamic forces, thereby accurately capturing the time-varying response of the forces and cable configurations. To address the difficulty of directly measuring material-related and hydrodynamic coefficients, this research proposes a parameter identification method that combines the physical model with experimental data. High-precision inversion of the equivalent Youngs modulus and hydrodynamic coefficients is performed through tension experiments under multiple configurations, effectively demonstrating that the identified model maintains predictive consistency in various operational conditions. Further numerical analysis indicates that the dynamic properties of flexible cable exhibit significant nonlinear characteristics, which are highly dependent on material property variations and AUV motion conditions. This nonlinear dynamic behavior results in two typical response states, slack and taut, which are jointly determined by boundary conditions and hydrodynamic effects, significantly affecting the cable configuration and endpoint loads. In this research, the dynamics of flexible cables under complex boundary conditions is revealed, providing a theoretical foundation for the design, optimization and further control research of similar systems.
Read the full abstract ▾
All authors Kuo Chen, Minghao Dou, Qianqi Liu, Yang An, Kai Ren, Zeming WU, Yu Tian, Jie Sun, Xinping Wang, Zhier Chen, Jiancheng Yu
Robotics
CommCP: Efficient Multi-Agent Coordination via LLM-Based Communication with Conformal Prediction
Xiaopan Zhang, Zejin Wang, Zhixu Li et al. (+2 more)
To complete assignments provided by humans in natural
language,
robots must interpret commands, generate and answer relevant questions for scene understanding, and manipulate target objects. Real-world deployments often require multiple heterogeneous
robots with different manipulation capabilities to handle different assignments cooperatively. Beyond the need for specialized manipulation skills, effective information gathering is important in completing these assignments. To address this component of the problem, we formalize the information-gathering process in a fully cooperative setting as an underexplored multi-
agent multi-task
Embodied Question Answering (MM-EQA) problem, which is a novel extension of canonical
Embodied Question Answering (EQA), where effective communication is crucial for coordinating efforts without redundancy. To address this problem, we propose CommCP, a novel LLM-based decentralized communication framework designed for MM-EQA. Our framework employs conformal prediction to calibrate the generated messages, thereby minimizing receiver distractions and enhancing communication reliability. To evaluate our framework, we introduce an MM-EQA
benchmark featuring diverse, photo-realistic household scenarios with
embodied questions. Experimental results demonstrate that CommCP significantly enhances the task success rate and exploration efficiency over baselines. The experiment
videos, code, and
dataset are available on our project website: https://comm-cp.github.io.
Read the full abstract ▾
All authors Xiaopan Zhang, Zejin Wang, Zhixu Li, Jianpeng Yao, Jiachen Li
Robotics · Cross-listed from cs.CV
InterPrior: Scaling Generative Control for Physics-Based Human-Object Interactions
Sirui Xu, Samuel Schulter, Morteza Ziyadi et al. (+4 more)
Humans rarely plan whole-body interactions with objects at the level of explicit whole-body movements. High-level intentions, such as affordance, define the goal, while coordinated balance, contact, and manipulation can emerge naturally from underlying physical and motor priors. Scaling such priors is key to enabling
humanoids to compose and generalize loco-manipulation skills across diverse contexts while maintaining physically coherent whole-body coordination. To this end, we introduce InterPrior, a scalable framework that learns a unified generative controller through large-scale imitation pretraining and post-training by reinforcement learning. InterPrior first distills a full-reference imitation expert into a versatile, goal-conditioned variational policy that reconstructs motion from
multimodal observations and high-level intent. While the distilled policy reconstructs training behaviors, it does not generalize reliably due to the vast configuration space of large-scale human-object interactions. To address this, we apply data augmentation with physical perturbations, and then perform reinforcement learning finetuning to improve competence on unseen goals and initializations. Together, these steps consolidate the reconstructed latent skills into a valid
manifold, yielding a motion prior that generalizes beyond the training data, e.g., it can incorporate new behaviors such as interactions with unseen objects. We further demonstrate its effectiveness for user-interactive control and its potential for real
robot deployment.
Read the full abstract ▾
All authors Sirui Xu, Samuel Schulter, Morteza Ziyadi, Xialin He, Xiaohan Fei, Yu-Xiong Wang, Liangyan Gui
Robotics · Cross-listed from cs.AI
Developing a Discrete-Event Simulator of School Shooter Behavior from VR Data
Christopher A. McClurg and Alan R. Wagner
Virtual reality (VR) has emerged as a powerful tool for evaluating school security measures in high-risk scenarios such as school shootings, offering experimental control and high behavioral fidelity. However, assessing new interventions in VR requires recruiting new participant cohorts for each condition, making large-scale or iterative evaluation difficult. These limitations are especially restrictive when attempting to learn effective intervention strategies, which typically require many training episodes. To address this challenge, we develop a data-driven discrete-event simulator (DES) that models shooter movement and in-region actions as
stochastic processes learned from participant behavior in VR studies. We use the simulator to examine the impact of a
robot-based shooter intervention strategy. Once shown to reproduce key empirical patterns, the DES enables scalable evaluation and learning of intervention strategies that are infeasible to train directly with human subjects. Overall, this work demonstrates a high-to-mid fidelity simulation workflow that provides a scalable surrogate for developing and evaluating autonomous school-security interventions.
Read the full abstract ▾
Robotics
Visuo-Tactile World Models
Carolina Higuera, Sergio Arnaud, Byron Boots et al. (+3 more)
We introduce multi-task Visuo-Tactile World Models (VT-WM), which capture the physics of contact through touch
reasoning. By complementing
vision with tactile sensing, VT-WM better understands
robot-object interactions in contact-rich tasks, avoiding common failure modes of
vision-only models under occlusion or ambiguous contact states, such as objects disappearing, teleporting, or moving in ways that violate basic physics. Trained across a set of contact-rich manipulation tasks, VT-WM improves physical fidelity in imagination, achieving 33% better performance at maintaining object permanence and 29% better compliance with the laws of motion in autoregressive rollouts. Moreover, experiments show that grounding in contact dynamics also translates to planning. In zero-shot real-
robot experiments, VT-WM achieves up to 35% higher success rates, with the largest gains in multi-step, contact-rich tasks. Finally, VT-WM demonstrates significant downstream versatility, effectively adapting its learned contact dynamics to a novel task and achieving reliable planning success with only a limited set of demonstrations.
Read the full abstract ▾
All authors Carolina Higuera, Sergio Arnaud, Byron Boots, Mustafa Mukadam, Francois Robert Hogan, Franziska Meier
Robotics · Cross-listed from cs.DC
Location-Aware Dispersion on Anonymous Graphs
Himani, Supantha Pandit and Gokarna Sharma
The well-studied DISPERSION problem is a fundamental coordination problem in distributed robotics, where a set of mobile
robots must relocate so that each occupies a distinct node of a network. DISPERSION assumes that a
robot can settle at any node as long as no other
robot settles on that node. In this work, we introduce LOCATION-AWARE DISPERSION, a novel generalization of DISPERSION that incorporates location awareness: Let $G = (V, E)$ be an anonymous, connected, undirected graph with $n = |V|$ nodes, each labeled with a color $\sf{col}(v) \in C = \{c_1, \dots, c_t\}, t\leq n$. A set $R = \{r_1, \dots, r_k\}$ of $k \leq n$ mobile
robots is given, where each
robot $r_i$ has an associated color $\mathsf{col}(r_i) \in C$. Initially placed arbitrarily on the graph, the goal is to relocate the
robots so that each occupies a distinct node of the same color. When $|C|=1$, LOCATION-AWARE DISPERSION reduces to DISPERSION. There is a solution to DISPERSION in graphs with any $k\leq n$ without knowing $k,n$. Like DISPERSION, the goal is to solve LOCATION-AWARE DISPERSION minimizing both time and
memory requirement at each
agent. We develop several deterministic
algorithms with guaranteed bounds on both time and
memory requirement. We also give an impossibility and a lower bound for any deterministic
algorithm for LOCATION-AWARE DISPERSION. To the best of our knowledge, the presented results collectively establish the algorithmic feasibility of LOCATION-AWARE DISPERSION in anonymous networks and also highlight the challenges on getting an efficient solution compared to the solutions for DISPERSION.
Read the full abstract ▾
Robotics
From Bench to Flight: Translating Drone Impact Tests into Operational Safety Limits
Aziz Mohamed Mili, Louis Catar, Paul Gérard et al. (+2 more)
Indoor micro-aerial vehicles (MAVs) are increasingly used for tasks that require close proximity to people, yet practitioners lack practical methods to tune motion limits based on measured impact risk. We present an end-to-end, open toolchain that converts benchtop impact tests into deployable safety governors for drones. First, we describe a compact and replicable impact rig and protocol for capturing force-time profiles across drone classes and contact surfaces. Second, we provide data-driven models that map pre-impact speed to impulse and contact duration, enabling direct computation of speed bounds for a target force limit. Third, we release scripts and a ROS2 node that enforce these bounds online and log compliance, with support for facility-specific policies. We validate the workflow on multiple commercial off-the-shelf quadrotors and representative indoor assets, demonstrating that the derived governors preserve task throughput while meeting force constraints specified by safety stakeholders. Our contribution is a practical bridge from measured impacts to runtime limits, with shareable
datasets, code, and a repeatable process that teams can adopt to certify indoor MAV operations near humans.
Read the full abstract ▾
All authors Aziz Mohamed Mili, Louis Catar, Paul Gérard, Ilyass Tabiai, David St-Onge
Robotics
Residual Reinforcement Learning for Waste-Container Lifting Using Large-Scale Cranes with Underactuated Tools
Qi Li and Karsten Berns
This paper studies the container lifting phase of a waste-container recycling task in urban environments, performed by a hydraulic loader crane equipped with an underactuated discharge unit, and proposes a residual reinforcement learning (RRL) approach that combines a nominal Cartesian controller with a learned residual policy. All experiments are conducted in simulation, where the task is characterized by tight
geometric tolerances between the discharge-unit hooks and the container rings relative to the overall crane scale, making precise trajectory tracking and swing suppression essential. The nominal controller uses admittance control for trajectory tracking and pendulum-aware swing damping, followed by damped least-squares inverse kinematics with a nullspace posture term to generate joint velocity commands. A PPO-trained residual policy in Isaac Lab compensates for unmodeled dynamics and parameter variations, improving precision and robustness without requiring end-to-end learning from scratch. We further employ randomized episode initialization and domain randomization over payload properties, actuator gains, and passive joint parameters to enhance generalization. Simulation results demonstrate improved tracking accuracy, reduced oscillations, and higher lifting success rates compared to the nominal controller alone.
Read the full abstract ▾
Robotics · Cross-listed from cs.LG
Constrained Group Relative Policy Optimization
Roger Girgis, Rodrigue de Schaetzen, Luke Rowe et al. (+3 more)
Group Relative Policy Optimization (GRPO) remains the dominant critic-free approach for fine-tuning LLMs and VLMs, but its compatibility with constrained policy optimization (e.g. for safety-critical domains) has not been carefully examined. In this work, we introduce Constrained GRPO, a Lagrangian-based extension of GRPO for constrained policy optimization. We show that the standard practice of scalarizing rewards before normalization introduces a critical Lagrangian-specific failure mode: GRPO's within-group normalization makes constrained optimization highly sensitive to how multi-component learning signals are aggregated. We show that scalarizing rewards before normalization introduces shared-denominator coupling, so that changing one multiplier alters not only the emphasis on its corresponding constraint, but also the relative weighting of the reward and other constraints. We address this with a simple but crucial modification: scalarizing standardized advantages rather than rewards. This yields a better-conditioned update by addressing the coupling induced by reward scalarization, resulting in better-behaved multiplier dynamics and more stable constraint enforcement in practice. Empirically, across a controlled gridworld, a real-world autonomous driving
benchmark, and a mathematical
reasoning task, Constrained GRPO consistently achieves better adherence to specified constraints while maintaining or improving task performance.
Read the full abstract ▾
All authors Roger Girgis, Rodrigue de Schaetzen, Luke Rowe, Azalée Robitaille, Christopher Pal, Liam Paull
Robotics
A Hybrid Autoencoder for Robust Heightmap Generation from Fused Lidar and Depth Data for Humanoid Robot Locomotion
Dennis Bank, Joost Cordes, Thomas Seel et al. (+1 more)
Reliable terrain perception is a critical prerequisite for the deployment of
humanoid robots in unstructured, human-centric environments. While traditional systems often rely on manually engineered, single-sensor pipelines, this paper presents a learning-based framework that uses an intermediate,
robot-centric heightmap representation. A hybrid Encoder-Decoder Structure (EDS) is introduced, utilizing a Convolutional Neural Network (CNN) for spatial feature extraction fused with a Gated Recurrent Unit (GRU) core for temporal consistency. The architecture integrates
multimodal data from an Intel RealSense depth camera, a LIVOX MID-360
LiDAR processed via efficient spherical projection, and an onboard IMU. Quantitative results demonstrate that
multimodal fusion improves reconstruction accuracy by 7.2% over depth-only and 9.9% over
LiDAR-only configurations. Furthermore, the integration of a 3.2 s temporal context reduces mapping drift.
Read the full abstract ▾
All authors Dennis Bank, Joost Cordes, Thomas Seel, Simon F.G. Ehlers
Robotics · Cross-listed from cs.CV
Hai Zhang, Siqi Liang, Li Chen et al. (+5 more)
Why must
vision-
language navigation be bound to detailed and verbose
language instructions? While such details ease decision-making, they fundamentally contradict the goal for
navigation in the real-world. Ideally,
agents should possess the autonomy to navigate in unknown environments guided solely by simple and high-level intents. Realizing this ambition introduces a formidable challenge: Beyond-the-View
Navigation (BVN), where
agents must locate distant, unseen targets without dense and step-by-step guidance. Existing large
language model (LLM)-based methods, though adept at following dense instructions, often suffer from short-sighted behaviors due to their reliance on short-horimzon supervision. Simply extending the supervision horizon, however, destabilizes LLM training. In this work, we identify that
video generation models inherently benefit from long-horizon supervision to align with
language instructions, rendering them uniquely suitable for BVN tasks. Capitalizing on this insight, we propose introducing the
video generation model into this field for the first time. Yet, the prohibitive latency for generating
videos spanning tens of seconds makes real-world deployment impractical. To bridge this gap, we propose SparseVideoNav, achieving sub-second trajectory inference guided by a generated sparse future spanning a 20-second horizon. This yields a remarkable 27x speed-up compared to the unoptimized counterpart. Extensive real-world zero-shot experiments demonstrate that SparseVideoNav achieves 2.5x the success rate of state-of-the-art LLM baselines on BVN tasks and marks the first realization of such capability in challenging night scenes.
Read the full abstract ▾
All authors Hai Zhang, Siqi Liang, Li Chen, Yuxian Li, Yukuan Xu, Yichao Zhong, Fu Zhang, Hongyang Li
Robotics
Scalable and General Whole-Body Control for Cross-Humanoid Locomotion
Yufei Xue, YunFeng Lin, Wentao Dong et al. (+6 more)
Learning-based whole-body controllers have become a key driver for
humanoid robots, yet most existing approaches require
robot-specific training. In this paper, we study the problem of cross-embodiment
humanoid control and show that a single policy can robustly generalize across a wide range of
humanoid robot designs with one-time training. We introduce XHugWBC, a novel cross-embodiment training framework that enables generalist
humanoid control through: (1) physics-consistent morphological randomization, (2) semantically aligned observation and action spaces across diverse
humanoid robots, and (3) effective policy architectures modeling morphological and dynamical properties. XHugWBC is not tied to any specific
robot. Instead, it internalizes a broad distribution of morphological and dynamical characteristics during training. By learning motion priors from diverse randomized embodiments, the policy acquires a strong structural bias that supports zero-shot transfer to previously unseen
robots. Experiments on twelve simulated
humanoids and seven real-world
robots demonstrate the strong generalization and robustness of the resulting universal controller.
Read the full abstract ▾
All authors Yufei Xue, YunFeng Lin, Wentao Dong, Yang Tang, Jingbo Wang, Jiangmiao Pang, Ming Zhou, Minghuan Liu, Weinan Zhang
Robotics
Task-Oriented Robot-Human Handovers on Legged Manipulators
Andreea Tulbure, Carmen Scheidemann, Elias Steiner et al. (+1 more)
Task-oriented handovers (TOH) are fundamental to effective human-
robot collaboration, requiring
robots to present objects in a way that supports the human's intended post-handover use. Existing approaches are typically based on object- or task-specific affordances, but their ability to generalize to novel scenarios is limited. To address this gap, we present AFT-Handover, a framework that integrates large
language model (LLM)-driven affordance
reasoning with efficient texture-based affordance transfer to achieve zero-shot, generalizable TOH. Given a novel object-task pair, the method retrieves a proxy exemplar from a database, establishes part-level correspondences via LLM
reasoning, and texturizes affordances for feature-based point cloud transfer. We evaluate AFT-Handover across diverse task-object pairs, showing improved handover success rates and stronger generalization compared to baselines. In a comparative user study, our framework is significantly preferred over the current state-of-the-art, effectively reducing human regrasping before tool use. Finally, we demonstrate TOH on legged manipulators, highlighting the potential of our framework for real-world
robot-human handovers.
Read the full abstract ▾
All authors Andreea Tulbure, Carmen Scheidemann, Elias Steiner, Marco Hutter
Robotics
From Vision to Decision: Neuromorphic Control for Autonomous Navigation and Tracking
Chuwei Wang, Eduardo Sebastián, Amanda Prorok et al. (+1 more)
Robotic
navigation has historically struggled to reconcile reactive, sensor-based control with the decisive capabilities of model-based planners. This duality becomes critical when the absence of a predominant option among goals leads to indecision, challenging reactive systems to break
symmetries without computationally-intense planners. We propose a parsimonious neuromorphic control framework that bridges this gap for
vision-guided
navigation and tracking. Image pixels from an onboard camera are encoded as inputs to dynamic neuronal populations that directly transform visual target excitation into
egocentric motion commands. A dynamic bifurcation mechanism resolves indecision by delaying commitment until a critical point induced by the environmental geometry. Inspired by recently proposed mechanistic models of animal cognition and opinion dynamics, the neuromorphic controller provides real-time autonomy with a minimal computational burden, a small number of interpretable parameters, and can be seamlessly integrated with application-specific image processing pipelines. We validate our approach in simulation environments as well as on an experimental quadrotor platform.
Read the full abstract ▾
All authors Chuwei Wang, Eduardo Sebastián, Amanda Prorok, Anastasia Bizyaeva
Robotics
HiCrowd: Hierarchical Crowd Flow Alignment for Dense Human Environments
Yufei Zhu, Shih-Min Yang, Martin Magnusson et al. (+1 more)
Navigating through dense human crowds remains a significant challenge for mobile
robots. A key issue is the freezing
robot problem, where the
robot struggles to find safe motions and becomes stuck within the crowd. To address this, we propose HiCrowd, a hierarchical framework that integrates reinforcement learning (RL) with model predictive control (MPC). HiCrowd leverages surrounding pedestrian motion as guidance, enabling the
robot to align with compatible crowd flows. A high-level RL policy generates a follow point to align the
robot with a suitable pedestrian group, while a low-level MPC safely tracks this guidance with short horizon planning. The method combines long-term crowd aware decision making with safe short-term execution. We evaluate HiCrowd against reactive and learning-based baselines in offline setting (replaying recorded human trajectories) and online setting (human trajectories are updated to react to the
robot in simulation). Experiments on a real-world
dataset and a synthetic crowd
dataset show that our method outperforms in
navigation efficiency and safety, while reducing freezing behaviors. We further validate through real-world deployment in a public museum and Expo 2025 Osaka, where it navigates dense pedestrian flows without retraining, demonstrating robust and socially aware behavior. Our results suggest that leveraging human motion as guidance, rather than treating humans solely as dynamic obstacles, provides a powerful principle for safe and efficient
robot navigation in crowds. Project code and demos are available at https://github.com/test-bai-cpu/HiCrowd.
Read the full abstract ▾
All authors Yufei Zhu, Shih-Min Yang, Martin Magnusson, Allan Wang
Robotics
TOLEBI: Learning Fault-Tolerant Bipedal Locomotion via Online Status Estimation and Fallibility Rewards
Hokyun Lee, Woo-Jeong Baek, Junhyeok Cha et al. (+1 more)
With the growing employment of learning
algorithms in robotic applications, research on reinforcement learning for bipedal locomotion has become a central topic for
humanoid robotics. While recently published contributions achieve high success rates in locomotion tasks, scarce
attention has been devoted to the development of methods that enable to handle hardware faults that may occur during the locomotion process. However, in real-world settings, environmental disturbances or sudden occurrences of hardware faults might yield severe consequences. To address these issues, this paper presents TOLEBI (A faulT-tOlerant Learning framEwork for Bipedal locomotIon) that handles faults on the
robot during operation. Specifically, joint locking, power loss and external disturbances are injected in simulation to learn fault-tolerant locomotion strategies. In addition to transferring the learned policy to the real
robot via sim-to-real transfer, an online joint status module incorporated. This module enables to classify joint conditions by referring to the actual observations at runtime under real-world conditions. The validation experiments conducted both in real-world and simulation with the
humanoid robot TOCABI highlight the applicability of the proposed approach. To our knowledge, this manuscript provides the first learning-based fault-tolerant framework for bipedal locomotion, thereby fostering the development of efficient learning methods in this field.
Read the full abstract ▾
All authors Hokyun Lee, Woo-Jeong Baek, Junhyeok Cha, Jaeheung Park
Robotics · Cross-listed from cs.CV
PIRATR: Parametric Object Inference for Robotic Applications with Transformers in 3D Point Clouds
Michael Schwingshackl, Fabio F. Oberweger, Mario Niedermeyer et al. (+2 more)
We present PIRATR, an end-to-end 3D object detection framework for robotic use cases in point clouds. Extending PI3DETR, our method streamlines parametric 3D object detection by jointly estimating multi-class 6-DoF poses and class-specific parametric attributes directly from occlusion-affected point cloud data. This formulation enables not only
geometric localization but also the estimation of task-relevant properties for parametric objects, such as a gripper's opening, where the 3D model is adjusted according to simple, predefined rules. The architecture employs modular, class-specific heads, making it straightforward to extend to novel object types without re-designing the pipeline. We validate PIRATR on an automated forklift platform, focusing on three structurally and functionally diverse categories: crane grippers, loading platforms, and pallets. Trained entirely in a synthetic environment, PIRATR generalizes effectively to real outdoor
LiDAR scans, achieving a detection mAP of 0.919 without additional fine-tuning. PIRATR establishes a new paradigm of pose-aware, parameterized perception. This bridges the gap between low-level
geometric reasoning and actionable world models, paving the way for scalable, simulation-trained perception systems that can be deployed in dynamic robotic environments. Code available at https://github.com/swingaxe/piratr.
Read the full abstract ▾
All authors Michael Schwingshackl, Fabio F. Oberweger, Mario Niedermeyer, Huemer Johannes, Markus Murschitz