Robotics
High-fidelity 3D reconstruction for planetary exploration
Alfonso Martínez-Petersen, Levin Gerdes, David Rodríguez-Martínez et al. (+1 more)
Planetary exploration increasingly relies on autonomous robotic systems capable of perceiving, interpreting, and reconstructing their surroundings in the absence of global positioning or real-time communication with Earth. Rovers operating on planetary surfaces must navigate under sever environmental constraints, limited visual redundancy, and communication delays, making onboard spatial awareness and visual localization key components for mission success. Traditional techniques based on Structure-from-Motion (SfM) and Simultaneous Localization and Mapping (SLAM) provide
geometric consistency but struggle to capture radiometric detail or to scale efficiently in unstructured, low-texture terrains typical of extraterrestrial environments. This work explores the integration of radiance field-based methods - specifically Neural Radiance Fields (NeRF) and Gaussian Splatting - into a unified, automated environment reconstruction pipeline for planetary robotics. Our system combines the Nerfstudio and COLMAP frameworks with a ROS2-compatible workflow capable of processing raw rover data directly from rosbag recordings. This approach enables the generation of dense, photorealistic, and metrically consistent 3D representations from minimal visual input, supporting improved perception and planning for autonomous systems operating in planetary-like conditions. The resulting pipeline established a foundation for future research in radiance field-based mapping, bridging the gap between
geometric and neural representations in planetary exploration.
Read the full abstract ▾
All authors Alfonso Martínez-Petersen, Levin Gerdes, David Rodríguez-Martínez, C. J. Pérez-del-Pulgar
Robotics · Cross-listed from cs.CV
RPGD: RANSAC-P3P Gradient Descent for Extrinsic Calibration in 3D Human Pose Estimation
Zhanyu Tuo
In this paper, we propose RPGD (RANSAC-P3P Gradient Descent), a human-pose-driven extrinsic calibration framework that robustly aligns MoCap-based 3D skeletal data with
monocular or multi-view RGB cameras using only natural human motion. RPGD formulates extrinsic calibration as a coarse-to-fine problem tailored to human poses, combining the global robustness of RANSAC-P3P with Gradient-Descent-based refinement. We evaluate RPGD on three large-scale public 3D HPE
datasets as well as on a self-collected in-the-wild
dataset. Experimental results demonstrate that RPGD consistently recovers extrinsic parameters with accuracy comparable to the provided ground truth, achieving sub-pixel MPJPE reprojection error even in challenging, noisy settings. These results indicate that RPGD provides a practical and automatic solution for reliable extrinsic calibration of large-scale 3D HPE
dataset collection.
Read the full abstract ▾
Robotics
UAV-SEAD: State Estimation Anomaly Dataset for UAVs
Aykut Kabaoglu and Sanem Sariel
Accurate state estimation in Unmanned Aerial Vehicles (
UAVs) is crucial for ensuring reliable and safe operation, as anomalies occurring during mission execution may induce discrepancies between expected and observed system behaviors, thereby compromising mission success or posing potential safety hazards. It is essential to continuously monitor and detect such conditions in order to ensure a timely response and maintain system reliability. In this work, we focus on
UAV state estimation anomalies and provide a large-scale real-world
UAV dataset to facilitate research aimed at improving the development of anomaly detection. Unlike existing
datasets that primarily rely on injected faults into simulated data, this
dataset comprises 1396 real flight logs totaling over 52 hours of flight time, collected across diverse indoor and outdoor environments using a collection of PX4-based
UAVs equipped with a variety of sensor configurations. The
dataset comprises both normal and anomalous flights without synthetic manipulation, making it uniquely suitable for realistic anomaly detection tasks. A structured classification is proposed that categorizes
UAV state estimation anomalies into four classes: mechanical and electrical, external position, global position, and altitude anomalies. These classifications reflect collective, contextual, and outlier anomalies observed in multivariate sensor data streams, including IMU, GPS, barometer, magnetometer, distance sensors, visual odometry, and optical flow, that can be found in the PX4 logging mechanism. It is anticipated that this
dataset will play a key role in the development, training, and evaluation of anomaly detection and isolation systems to address the critical gap in
UAV reliability research.
Read the full abstract ▾
Robotics
Modeling and Optimizing the Provisioning of Exhaustible Capabilities for Simultaneous Task Allocation and Scheduling
Jinwoo Park, Harish Ravichandar and Seth Hutchinson
Deploying heterogeneous
robot teams to accomplish multiple tasks over extended time horizons presents significant computational challenges for task allocation and planning. In this paper, we present a comprehensive, time-extended, offline heterogeneous multi-
robot task allocation framework, TRAITS, which we believe to be the first that can cope with the provisioning of exhaustible traits under battery and temporal constraints. Specifically, we introduce a nonlinear programming-based trait distribution module that can optimize the trait-provisioning rate of coalitions to yield feasible and time-efficient solutions. TRAITS provides a more accurate feasibility assessment and estimation of task execution times and makespan by leveraging trait-provisioning rates while optimizing battery consumption – an advantage that state-of-the-art frameworks lack. We evaluate TRAITS against two state-of-the-art frameworks, with results demonstrating its advantage in satisfying complex trait and battery requirements while remaining computationally tractable.
Read the full abstract ▾
Robotics · Cross-listed from cs.AI
Enabling Option Learning in Sparse Rewards with Hindsight Experience Replay
Gabriel Romio, Mateus Begnini Melchiades, Bruno Castro da Silva et al. (+1 more)
Hierarchical Reinforcement Learning (HRL) frameworks like Option-Critic (OC) and Multi-updates Option Critic (MOC) have introduced significant advancements in learning reusable options. However, these methods underperform in multi-goal environments with sparse rewards, where actions must be linked to temporally distant outcomes. To address this limitation, we first propose MOC-HER, which integrates the Hindsight Experience Replay (HER) mechanism into the MOC framework. By relabeling goals from achieved outcomes, MOC-HER can solve sparse reward environments that are intractable for the original MOC. However, this approach is insufficient for object manipulation tasks, where the reward depends on the object reaching the goal rather than on the
agent's direct interaction. This makes it extremely difficult for HRL
agents to discover how to interact with these objects. To overcome this issue, we introduce Dual Objectives Hindsight Experience Replay (2HER), a novel extension that creates two sets of virtual goals. In addition to relabeling goals based on the object's final state (standard HER), 2HER also generates goals from the
agent's effector positions, rewarding the
agent for both interacting with the object and completing the task. Experimental results in robotic manipulation environments show that MOC-2HER achieves success rates of up to 90%, compared to less than 11% for both MOC and MOC-HER. These results highlight the effectiveness of our dual objective relabeling strategy in sparse reward, multi-goal tasks.
Read the full abstract ▾
All authors Gabriel Romio, Mateus Begnini Melchiades, Bruno Castro da Silva, Gabriel de Oliveira Ramos
Robotics
Humanoid Hanoi: Investigating Shared Whole-Body Control for Skill-Based Box Rearrangement
Minku Kim, Kuan-Chia Chen, Aayam Shrestha et al. (+3 more)
We introduce
Humanoid Hanoi, a challenge problem for long-horizon
humanoid skill composition that requires a
robot to repeatedly sequence pickup, locomotion, and placement skills to rearrange stacked boxes in simulation and on hardware. This symbolically simple task exposes a weakness in a common modular
humanoid architecture, where reusable high-level skills are learned on top of a shared, task-agnostic whole-body controller (WBC). While such a WBC provides a unified low-level interface for skill composition, it may not be sufficiently hardened to execute the full collection of learned skills under domain randomization and long-horizon closed-loop execution. We compare strategies for addressing this issue, including skill-specific WBC fine-tuning, residual low-level adaptation, and task-driven data aggregation that expands the shared-WBC training distribution with closed-loop trajectories induced by learned skills while preserving its original tracking objective. In our evaluated system, the shared-WBC approach achieves the strongest long-horizon robustness. A direct transition-smoothing intervention further shows that low-level controller handoffs account for a substantial part of the composition loss observed with non-shared controllers. Project page: https://osudrl.github.io/
Humanoid_Hanoi/
Read the full abstract ▾
All authors Minku Kim, Kuan-Chia Chen, Aayam Shrestha, Li Fuxin, Stefan Lee, Alan Fern
Robotics
Push-Placement: A Hybrid Approach Integrating Prehensile and Non-Prehensile Manipulation for Object Rearrangement
Majid Sadeghinejad, Arman Barghi, Hamed Hosseini et al. (+2 more)
Efficient tabletop rearrangement remains challenging due to collisions and the need for temporary buffering when target poses are obstructed. Prehensile pick-and-place provides precise control but often requires extra moves, whereas non-prehensile pushing can be more efficient but suffers from complex, imprecise dynamics. This paper proposes push-placement, a hybrid action primitive that uses the grasped object to displace obstructing items while being placed, thereby reducing explicit buffering. The method is integrated into a physics-in-the-loop Monte Carlo Tree Search (MCTS) planner and evaluated in the PyBullet simulator. Empirical results show push-placement reduces the manipulator travel cost by up to 11.12% versus a baseline MCTS planner and 8.56% versus dynamic stacking. These findings indicate that hybrid prehensile/non-prehensile action primitives can substantially improve efficiency in long-horizon rearrangement tasks.
Read the full abstract ▾
All authors Majid Sadeghinejad, Arman Barghi, Hamed Hosseini, Mehdi Tale Masouleh, Ahmad Kalhor
Robotics
Semantic-Contact Fields for Category-Level Generalizable Tactile Tool Manipulation
Kevin Yuchen Ma, Heng Zhang, Weisi Lin et al. (+2 more)
Generalizing tool manipulation requires both semantic planning and precise physical control. Modern generalist
robot policies, such as
Vision-
Language-Action (VLA) models, often lack the physical grounding required for contact-rich tool manipulation. Conversely, existing contact-aware policies that leverage tactile or haptic sensing are typically instance-specific and fail to generalize across diverse tool geometries. Bridging this gap requires learning representations that are both semantically transferable and physically grounded, yet a fundamental barrier remains: diverse real-world tactile data are prohibitive to collect at scale, while direct zero-shot sim-to-real transfer is challenging due to the complex nonlinear deformation of soft tactile sensors. To address this, we propose Semantic-Contact Fields (SCFields), a unified 3D representation that fuses visual semantics with dense extrinsic contact estimates, including contact probability and force. SCFields is learned through a two-stage Sim-to-Real Contact Learning Pipeline: we first pre-train on large-scale simulation to learn geometry-aware contact priors, then fine-tune on a small set of real data pseudo-labeled via
geometric heuristics and force optimization to align real tactile signals. The resulting force-aware representation serves as the dense observation input to a
diffusion policy, enabling physical generalization to unseen tool instances. Experiments on scraping, crayon drawing, and peeling demonstrate robust category-level generalization, significantly outperforming
vision-only and raw-tactile baselines. Project page: https://kevinskwk.github.io/SCFields/.
Read the full abstract ▾
All authors Kevin Yuchen Ma, Heng Zhang, Weisi Lin, Mike Zheng Shou, Yan Wu
Robotics · Cross-listed from cs.CV
Gaussian Sequences with Multi-Scale Dynamics for 4D Reconstruction from Monocular Casual Videos
Can Li, Jie Gu, Jingmin Chen et al. (+2 more)
Understanding dynamic scenes from casual
videos is critical for scalable
robot learning, yet four-dimensional (4D) reconstruction under strictly
monocular settings remains highly ill-posed. To address this challenge, our key insight is that real-world dynamics exhibits a multi-scale regularity from object to particle level. To this end, we design the multi-scale dynamics mechanism that factorizes complex motion fields. Within this formulation, we propose Gaussian sequences with multi-scale dynamics, a novel representation for dynamic 3D Gaussians derived through compositions of multi-level motion. This layered structure substantially alleviates ambiguity of reconstruction and promotes physically plausible dynamics. We further incorporate multi-modal priors from
vision foundation models to establish complementary supervision, constraining the solution space and improving the reconstruction fidelity. Our approach enables accurate and globally consistent 4D reconstruction from
monocular casual
videos. Experiments of dynamic novel-view synthesis (NVS) on
benchmark and real-world manipulation
datasets demonstrate considerable improvements over existing methods.
Read the full abstract ▾
All authors Can Li, Jie Gu, Jingmin Chen, Fangzhou Qiu, Lei Sun
Robotics
Ontological grounding for sound and natural robot explanations via large language models
Alberto Olivares-Alarcos, Muhammad Ahsan, Satrio Sanjaya et al. (+2 more)
Building effective human-
robot interaction requires
robots to derive conclusions from their experiences that are both logically sound and communicated in ways aligned with human expectations. This paper presents a hybrid framework that blends ontology-based
reasoning with large
language models (LLMs) to produce semantically grounded and natural
robot explanations. Ontologies ensure logical consistency and domain grounding, while LLMs provide fluent, context-aware and adaptive
language generation. The proposed method grounds data from human-
robot experiences, enabling
robots to reason about whether events are typical or atypical based on their properties. We integrate a state-of-the-art
algorithm for retrieving and constructing static contrastive ontology-based narratives with an LLM
agent that uses them to produce concise, clear, interactive explanations. The approach is validated through a laboratory study replicating an industrial collaborative task. Empirical results show significant improvements in the clarity and brevity of ontology-based narratives while preserving their semantic accuracy. Initial evaluations further demonstrate the system's ability to adapt explanations to user feedback. Overall, this work highlights the potential of ontology-LLM integration to advance explainable agency, and promote more transparent human-
robot collaboration.
Read the full abstract ▾
All authors Alberto Olivares-Alarcos, Muhammad Ahsan, Satrio Sanjaya, Hsien-I Lin, Guillem Alenyà
Robotics
MOTIF: Learning Action Motifs for Few-shot Cross-Embodiment Transfer
Heng Zhi, Wentao Tan, Lei Zhu et al. (+4 more)
While
vision-
language-action (VLA) models have advanced generalist robotic learning, cross-embodiment transfer remains challenging due to kinematic heterogeneity and the high cost of collecting sufficient real-world demonstrations to support fine-tuning. Existing cross-embodiment policies typically rely on shared-private architectures, which suffer from limited capacity of private parameters and lack explicit adaptation mechanisms. To address these limitations, we introduce MOTIF for efficient few-shot cross-embodiment transfer that decouples embodiment-agnostic spatiotemporal patterns, termed action motifs, from heterogeneous action data. Specifically, MOTIF first learns unified motifs via vector quantization with progress-aware
alignment and embodiment
adversarial constraints to ensure temporal and cross-embodiment consistency. We then design a lightweight predictor that predicts these motifs from real-time inputs to guide a flow-matching policy, fusing them with
robot-specific states to enable action generation on new embodiments. Evaluations across both simulation and real-world environments validate the superiority of MOTIF, which significantly outperforms strong baselines in few-shot transfer scenarios by 6.5% in simulation and 43.7% in real-world settings. Code is available at https://github.com/buduz/MOTIF.
Read the full abstract ▾
All authors Heng Zhi, Wentao Tan, Lei Zhu, Fengling Li, Jingjing Li, Guoli Yang, Heng Tao Shen
Robotics
Impact-Robust Posture Optimization for Aerial Manipulation
Amr Afifi, Ahmad Gazar, Javier Alonso-Mora et al. (+2 more)
We present a novel method for optimizing the posture of kinematically redundant torque-controlled
robots to improve robustness during impacts. A rigid impact model is used as the basis for a configuration-dependent metric that quantifies the variation between pre- and post-impact velocities. By finding configurations (postures) that minimize the aforementioned metric, spikes in the
robot's state and input commands can be significantly reduced during impacts, improving safety and robustness. The problem of identifying impact-robust postures is posed as a min-max optimization of the aforementioned metric. To overcome the real-time intractability of the problem, we reformulate it as a gradient-based motion task that iteratively guides the
robot towards configurations that minimize the proposed metric. This task is embedded within a task-space inverse dynamics (TSID) whole-body controller, enabling seamless integration with other control objectives. The method is applied to a kinematically redundant aerial manipulator performing repeated point contact tasks. We test our method inside a realistic physics simulator and compare it with the nominal TSID. Our method leads to a reduction (up to 51% w.r.t. standard TSID) of post-impact spikes in the
robot's configuration and successfully avoids actuator saturation. Moreover, we demonstrate the importance of kinematic redundancy for impact robustness using additional numerical simulations on a quadruped and a
humanoid robot, resulting in up to 45% reduction of post-impact spikes in the
robot's state w.r.t. nominal TSID.
Read the full abstract ▾
All authors Amr Afifi, Ahmad Gazar, Javier Alonso-Mora, Paolo Robuffo Giordano, Antonio Franchi
Robotics
The More the Merrier: Running Multiple Neuromorphic Components On-Chip for Robotic Control
Evan Eames, Priyadarshini Kannan, Ronan Sangouard et al. (+9 more)
It has long been realized that neuromorphic hardware offers benefits for the domain of robotics such as low energy, low latency, as well as unique methods of learning. In aiming for more complex tasks, especially those incorporating
multimodal data, one hurdle continuing to prevent their realization is an inability to orchestrate multiple networks on neuromorphic hardware without resorting to off-chip process management logic. To address this, we show a first example of a pipeline for
vision-based
robot control in which numerous complex networks can be run entirely on hardware via the use of a spiking neural state machine for process orchestration. The pipeline is validated on the Intel Loihi 2 research chip. We show that all components can run concurrently on-chip in the milli Watt regime at latencies competitive with the state-of-the-art. An equivalent network on simulated hardware is shown to accomplish robotic arm plug insertion in simulation, and the core elements of the pipeline are additionally tested on a real robotic arm.
Read the full abstract ▾
All authors Evan Eames, Priyadarshini Kannan, Ronan Sangouard, Philipp Plank, Elvin Hajizada, Gintautas Palinauskas, Lana Amaya, Michael Neumeier, Sai Thejeshwar Sharma, Marcella Toth, Prottush Sarkar, Axel von Arnim
Robotics
XIT: Exploration and Exploitation Informed Trees for Active Gas Distribution Mapping in Unknown Environments
Mal Fazliu, Matthew Coombes, Sen Wang et al. (+1 more)
Mobile robotic gas distribution mapping (GDM) provides critical situational awareness during emergency responses to hazardous gas releases. However, most systems still rely on teleoperation, limiting scalability and response speed. Autonomous active GDM is challenging in unknown and cluttered environments, because the
robot must simultaneously explore traversable space, map the environment, and infer the gas distribution belief from sparse chemical measurements. We address this by formulating active GDM as a next-best-trajectory informative path planning (IPP) problem and propose XIT (Exploration and Exploitation Informed Trees), a sampling-based planner that balances exploration and exploitation by generating concurrent trajectories toward exploration-rich goals while collecting informative gas measurements en route. XIT draws a batch of samples from an Upper Confidence Bound (UCB) information field derived from the current gas posterior and expands trees using a cost that trades off travel effort against information acquisition. To enable plume-aware exploration, we introduce the gas frontier concept, defined as unobserved regions adjacent to high gas concentrations, and propose the Wavefront Gas Frontier Detection (WGFD)
algorithm for their identification. High-fidelity simulations and a real-world experiment demonstrate the benefits of XIT in terms of GDM quality and efficiency. Although developed for active GDM, XIT is readily applicable to other robotic information-gathering tasks in unknown environments that face the exploration and exploitation trade-off.
Read the full abstract ▾
All authors Mal Fazliu, Matthew Coombes, Sen Wang, Cunjia Liu
Robotics
Improving Driver Satisfaction with a Driving Function Learning from Implicit Human Feedback – a Test Group Study
Robin Schwager, Andrea Anastasio, Simon Hartmann et al. (+6 more)
During the use of advanced driver assistance systems, drivers frequently intervene into the active driving function and adjust the system's behavior to their personal wishes. These active driver-initiated takeovers contain feedback about deviations in the driving function's behavior from the drivers' personal preferences. This feedback should be utilized to optimize and personalize the driving function's behavior. In this work, the adjustment of the speed profile of a Predictive Longitudinal Driving Function (PLDF) on a pre-defined route is highlighted. An
algorithm is introduced which iteratively adjusts the PLDF's speed profile by taking into account both the original speed profile of the PLDF and the driver demonstration. This approach allows for personalization in a traded control scenario during active use of the PLDF. The applicability of the proposed
algorithm is tested in a driving simulator-based test group study with 43 participants. The study finds a significant increase in driver satisfaction and a significant reduction in the intervention frequency when using the proposed adaptive PLDF. Additionally, feedback by the participants was gathered to identify further optimization potentials of the proposed system.
Read the full abstract ▾
All authors Robin Schwager, Andrea Anastasio, Simon Hartmann, Andreas Ronellenfitsch, Michael Grimm, Tim Brühl, Tin Stribor Sohn, Tim Dieter Eberhardt, Sören Hohmann
Robotics
FC-Vision: Real-Time Visibility-Aware Replanning for Occlusion-Free Aerial Target Structure Scanning in Unknown Environments
Chen Feng, Yang Xu and Shaojie Shen
Autonomous aerial scanning of target structures is crucial for practical applications, requiring online adaptation to unknown obstacles during flight. Existing methods largely emphasize collision avoidance and efficiency, but overlook occlusion-induced visibility degradation, severely compromising scanning quality. This study proposes FC-
Vision, an on-the-fly visibility-aware replanning framework that proactively and safely prevents target occlusions while preserving full target coverage and efficiency of the original plan. Our approach explicitly enforces dense surface-visibility constraints to regularize replanning behavior in real-time via an efficient two-level decomposition: occlusion-free viewpoint repair that maintains coverage with minimal deviation from the nominal scan, followed by segment-wise clean-sensing connection in 5-DoF space. A plug-in integration strategy is also presented to seamlessly interface FC-
Vision with existing
UAV scanning systems without architectural changes. Comprehensive simulation and real-world evaluations show that FC-
Vision consistently improves scanning quality under unexpected occluders, delivering a maximum coverage gain of 55.32% and a 73.17% reduction in the occlusion ratio, while achieving real-time performance with a moderate increase in flight time. The code has been released at https://github.com/FC-Family/FC-
Vision.
Read the full abstract ▾
Robotics
HybridFlow: A 2-NFE Generative Policy for Real-Time Robotic Manipulation
Zhenchen Dong, Fulin Chen, Jinna Fu et al. (+5 more)
Generative policies for robotic manipulation must balance action accuracy with inference latency. We present HybridFlow, a three-stage policy inference procedure requiring two network function evaluations (2-NFE). A Global Jump uses the MeanFlow average velocity to generate a coarse action trajectory; a parameter-free ReNoise interpolation constructs a state at a nonzero refinement time; and a Local Refine queries the instantaneous-velocity limit of the same network at that time. This construction reuses a unified model without
distillation. Our analysis characterizes interval-composition errors and the attenuation of endpoint error under ReNoise interpolation. Controlled RoboMimic ablations support the MeanFlow proposal and intermediate-state construction, achieving 95% average success with reused noise and 95.5% with fresh noise, versus 78% for one-step MeanFlow. Across five real-
robot settings with all policies running on the same Jetson AGX Thor, HybridFlow improves normalized task performance over 16-step
Diffusion Policy by 13-68 points with approximately eightfold lower action-generation latency. Additional experiments demonstrate its compatibility as an action expert in a
vision-
language-action framework. Project page: https://hybridflow-anonymous.pages.dev/
Read the full abstract ▾
All authors Zhenchen Dong, Fulin Chen, Jinna Fu, Jiaming Wu, Qingran Wu, Shengyuan Yu, Hongyu Yu, Yide Liu
Robotics
Symmetry-Aware Fusion of Vision and Tactile Sensing via Bilateral Force Priors for Robotic Manipulation
Wonju Lee, Matteo Grimaldi and Tao Yu
Insertion tasks in robotic manipulation demand precise, contact-rich interactions that
vision alone cannot resolve. While tactile feedback is intuitively valuable, existing studies have shown that naïve visuo-tactile fusion often fails to deliver consistent improvements. In this work, we propose a Cross-Modal
Transformer (CMT) for visuo-tactile fusion that integrates wrist-camera observations with tactile signals through structured self- and cross-
attention. To stabilize tactile
embeddings, we further introduce a physics-informed regularization that encourages bilateral force balance, reflecting principles of human motor control. Experiments on the TacSL
benchmark show that CMT with
symmetry regularization achieves a 96.59% insertion success rate, surpassing naïve and gated fusion baselines and closely matching the privileged "wrist + contact force" configuration (96.09%). These results highlight two central insights: (i) tactile sensing is indispensable for precise
alignment, and (ii) principled
multimodal fusion, further strengthened by physics-informed regularization, unlocks complementary strengths of
vision and touch, approaching privileged performance under realistic sensing.
Read the full abstract ▾
Robotics
KungfuAthleteBot: Learning High-Dynamic Motion from Videos with Unified Robust Recovery
Zhongxiang Lei, Lulu Cao, Xuyang Wang et al. (+3 more)
Learning highly dynamic motions from
video provides a scalable alternative to expensive data collection for
humanoid robots, yet it is challenged by height floating, missing kinematic and force information, and limited robustness. We present KungfuAthleteBot (KAB), a unified framework that systematically addresses these issues. We first build a high-dynamic motion
dataset in collaboration with professional martial artists, alleviating data scarcity while improving motion quality. To eliminate height drift and global jitter during both aerial and landing phases, we introduce a physics-guided
parabolic trajectory correction that enforces physical plausibility. To bridge the discrepancy between human and
robot actuation in highly dynamic behaviors, we design a three-stage curriculum that progressively enables stable acquisition of such motions. In addition, we propose a direct, teacher-free training paradigm that jointly learns disturbance rejection and fall recovery within a single policy. To the best of our knowledge, KAB achieves the fastest recovery time reported for a unified policy ($\approx 0.7$s). Experiments on a real
humanoid robot demonstrate successful learning of highly dynamic motions from
video and strong robustness, including rapid recovery under severe disturbances.
Read the full abstract ▾
All authors Zhongxiang Lei, Lulu Cao, Xuyang Wang, Tianyi Qian, Jinyan Liu, Xuesong Li
Robotics
SPLIT: Sparse Incremental Learning of Error Dynamics for Control-Oriented Modeling in Autonomous Vehicles
Yaoyu Li, Chaosheng Huang and Jun Li
Accurate, computationally efficient, and adaptive vehicle models are essential for autonomous vehicle control. Hybrid models that combine a nominal model with a Gaussian Process (GP)-based residual model have emerged as a promising approach. However, the GP-based residual model suffers from the curse of dimensionality, high evaluation complexity, and the inefficiency of online learning, which impede the deployment in real-time vehicle controllers. To address these challenges, we propose SPLIT, a sparse incremental learning framework for control-oriented vehicle dynamics modeling. SPLIT integrates three key innovations: (i) Model Decomposition. We decompose the vehicle model into invariant elements calibrated by experiments, and variant elements compensated by the residual model to reduce feature dimensionality. (ii) Local Incremental Learning. We define the valid region in the feature space and partition it into subregions, enabling efficient online learning from streaming data. (iii) GP Sparsification. We use
bayesian committee machine to ensure scalable online evaluation. Integrated into model-based controllers, SPLIT is evaluated in aggressive simulations and real-vehicle experiments. Results demonstrate that SPLIT improves model accuracy and control performance online. Moreover, it enables rapid adaptation to vehicle dynamics deviations and exhibits robust generalization to previously unseen scenarios.
Read the full abstract ▾
Robotics
Hierarchical Audio-Visual-Proprioceptive Fusion for Precise Robotic Manipulation
Siyuan Li, Jiani Lu, Yu Song et al. (+3 more)
Existing robotic manipulation methods primarily rely on visual and proprioceptive observations, which may struggle to infer contact-related interaction states in partially observable real-world environments. Acoustic cues, by contrast, naturally encode rich interaction dynamics during contact, yet remain underexploited in current
multimodal fusion literature. Most
multimodal fusion approaches implicitly assume homogeneous roles across modalities, and thus design flat and symmetric fusion structures. However, this assumption is ill-suited for acoustic signals, which are inherently sparse and contact-driven. To achieve precise robotic manipulation through acoustic-informed perception, we propose a hierarchical representation fusion framework that progressively integrates audio,
vision, and proprioception. Our approach first conditions visual and proprioceptive representations on acoustic cues, and then explicitly models higher-order cross-modal interactions to capture complementary dependencies among modalities. The fused representation is leveraged by a
diffusion-based policy to directly generate continuous
robot actions from
multimodal observations. The combination of end-to-end learning and hierarchical fusion structure enables the policy to exploit task-relevant acoustic information while mitigating interference from less informative modalities. The proposed method has been evaluated on real-world robotic manipulation tasks, including
liquid pouring and cabinet opening. Extensive experiment results demonstrate that our approach consistently outperforms state-of-the-art
multimodal fusion frameworks, particularly in scenarios where acoustic cues provide task-relevant information not readily available from visual observations alone. Furthermore, a mutual information analysis is conducted to interpret the effect of audio cues in robotic manipulation via
multimodal fusion.
Read the full abstract ▾
All authors Siyuan Li, Jiani Lu, Yu Song, Xianren Li, Bo An, Peng Liu
Robotics
AgentRob: From Virtual Forum Agents to Hijacked Physical Robots
Wenrui Liu, Yaxuan Wang, Xun Zhang et al. (+13 more)
Large
Language Model (LLM)-powered autonomous
agents have demonstrated significant capabilities in virtual environments, yet their integration with the physical world remains narrowly confined to direct control interfaces. We present AgentRob, a framework that bridges online community forums, LLM-powered
agents, and physical
robots through the Model Context Protocol (MCP). AgentRob enables a novel paradigm where autonomous
agents participate in online forums–reading posts, extracting natural
language commands, dispatching physical
robot actions, and reporting results back to the community. The system comprises three layers: a Forum Layer providing asynchronous, persistent, multi-
agent interaction; an
Agent Layer with forum
agents that poll for @mention-targeted commands; and a
Robot Layer with VLM-driven controllers and Unitree Go2/G1 hardware that translate commands into
robot primitives via iterative tool calling. The framework supports multiple concurrent
agents with distinct identities and physical embodiments coexisting in the same forum, establishing the feasibility of forum-mediated multi-
agent robot orchestration.
Read the full abstract ▾
All authors Wenrui Liu, Yaxuan Wang, Xun Zhang, Yanshu Wang, Jiashen Wei, Yifan Xiang, Yuhang Wang, Mingshen Ye, Elsie Dai, Zhiqi Liu, Yingjie Xu, Xinyang Chen, Hengzhe Sun, Jiyu Shen, Jingjing He, Tong Yang
Robotics
TactAlign: Human-to-Robot Policy Transfer via Tactile Alignment
Youngsun Wi, Jessica Yin, Elvis Xiang et al. (+5 more)
Human demonstrations collected by wearable devices (e.g., tactile gloves) provide fast and dexterous supervision for policy learning, and are guided by rich, natural tactile feedback. However, a key challenge is how to transfer human-collected tactile signals to
robots despite the differences in sensing modalities and embodiment. Existing human-to-
robot (H2R) approaches that incorporate touch often assume identical tactile sensors, require paired data, and involve little to no embodiment gap between human demonstrator and the
robots, limiting scalability and generality. We propose TactAlign, a cross-embodiment tactile
alignment method that transfers human-collected tactile signals to a
robot with different embodiment. TactAlign transforms human and
robot tactile observations into a shared latent representation using a rectified flow, without paired
datasets, manual labels, or privileged information. Our method enables low-cost latent transport guided by hand-object interaction-derived pseudo-pairs. We demonstrate that TactAlign improves H2R policy transfer across multiple contact-rich tasks (pivoting, insertion, lid closing), generalizes to unseen objects and tasks with human data (less than 5 minutes), and enables zero-shot H2R transfer on a highly dexterous tasks (light bulb screwing).
Read the full abstract ▾
All authors Youngsun Wi, Jessica Yin, Elvis Xiang, Akash Sharma, Jitendra Malik, Mustafa Mukadam, Nima Fazeli, Tess Hellebrekers