Robotics
Stage-Aware and Roughness-Constrained Diffusion Policy for Multi-Stage Robotic Polishing
Shuai Ke, Jiexin Zhang, Huan Zhao et al. (+7 more)
Polishing is a critical finishing process in high-end manufacturing fields such as aerospace, where surface quality directly affects the service performance and reliability of components. Robotic imitation learning provides a flexible solution for such tasks, but current methods remain limited in industrial polishing because of long-horizon dependencies, uncertain stage transitions, and the difficulty of modeling and regulating coupled process parameters. To address these issues, this paper proposes a Stage-Aware and Roughness-Constrained
Diffusion Policy (SRDP) for robotic polishing. SRDP infers the process-stage posterior from
multimodal observation histories and uses it to condition the shared reverse denoising process, enabling stage-consistent action generation without external stage labels during execution. Furthermore, a roughness-oriented process-constrained
diffusion sampling method is incorporated to generate constrained feed speed and normal contact force under stage-wise preset spindle speeds, thereby improving process consistency and physical feasibility. Systematic experiments are conducted on two representative scenarios, namely spacecraft cabin coating-surface polishing and inner-cavity structural surface finishing. Comparisons with advanced baselines, ablation studies, and real-
robot validations comprehensively evaluate the proposed method. The results show that SRD improves stage-transition stability, process-parameter consistency, and final surface quality across different polishing scenarios.
Read the full abstract ▾
All authors Shuai Ke, Jiexin Zhang, Huan Zhao, Zhiao Wei, Yikun Guo, Tiange Wu, Guoqiang Guo, Haoyuan Zhou, Jie Pan, Han Ding
Robotics
Learning Asynchronous Upper-body Task-space Trajectory Tracking Policy for Humanoid Robots
Yumeng Liu, Dongqi Wang, Jiyu Yu et al. (+3 more)
High-level
humanoid planners often output sparse task-space, low-rate trajectories, whereas whole-body controllers run at high frequency. This creates temporal asynchrony between the planning and execution, and structural incompleteness for full-body control. We propose an asynchronous upper body task-space tracking framework for
humanoids. A student policy is initialized by teacher-student
distillation, conditioned on the full cached future trajectory and an execution-time index, and trained with a sliding-window global reward to reduce frame drift without explicit frame estimation. For task-specific post-training, an MPC module completes sparse references into floating-base and upper-body guidance, while action- and FK level self-guidance constrain policy drift. Simulation and Unitree G1 hardware experiments show improved tracking under low update rates, stronger performance than synchronous and decoupled baselines, and safer adaptation to out-of-distribution motions.
Read the full abstract ▾
All authors Yumeng Liu, Dongqi Wang, Jiyu Yu, Yijun Fan, Rong Xiong, Yue Wang
Robotics · Cross-listed from cs.LG
Memory-Efficient Policy Libraries with Low-Rank Adaptation in Reinforcement Learning
Samuel Valland Lyngset, Tor Viljen Raanaas, Gard Sveipe et al. (+4 more)
When fine-tuning Large
Language Models (LLMs), there has been success in minimizing both
memory usage and computation with Parameter-Efficient Fine-Tuning (PEFT), like Low Rank Adaptation (LoRA). In this article, we have explored whether this approach is transferable to the world of robotics and Reinforcement Learning (RL), allowing learning with reduced
memory usage and improved computational performance. Specifically, we focused on a version of multi-task robotics, where a library of specialist policies are created. In such a library
memory efficiency is especially important. We used a Proximal Policy Optimization (PPO)
algorithm and fine-tuned a baseline model to different tasks using LoRA. Our results demonstrate that, depending on the hyperparameters, LoRA can minimize
memory usage by a factor of 20-160 compared to full fine-tuning of all layers. This implies a 90-95% storage saving when deploying a library of many (10-50) specialized policies, which can be the differentiating factor between being able to store the entire library in
memory or having to use swap-
memory in an applied robotics setting. At the same time, our results indicate that there is no significant difference in the success-rate between full fine-tuning and LoRA fine-tuning for the selected tasks.
Read the full abstract ▾
All authors Samuel Valland Lyngset, Tor Viljen Raanaas, Gard Sveipe, Eirik Møller Nilsen, Jim Torresen, Kai Olav Ellefsen, Tobias Lømo
Robotics
SA-LIVO: Efficient LiDAR-Inertial-Visual Odometry with Subspace-Aware Degeneracy Handling
Yinong Cao, Xin He, Yijie Wang et al. (+11 more)
Light detection and ranging (
LiDAR)-inertial-visual odometry fuses
geometric depth with visual measurements, but its sensors fail independently:
LiDAR in under-constrained geometry,
vision under poor illumination or texture. Weak information is harmless, since the filter then relies on the other sensor or the inertial prediction; over-confident information is not. Existing countermeasures act on weak information or switch
vision by hand-set thresholds. This article presents SA-LIVO, which fuses
vision through explicit, online-estimated error models. A frame-level noise scale, estimated by covariance matching, absorbs whole-image errors, and Schmidt filtering carries the correlation of reused observations. Subspace-aware information fusion models the error shared by a frame's observations as a pose offset capping the visual information per direction; the exact fusion attenuates
vision only above that cap and reduces to the information sum when the visual model is consistent.
LiDAR and visual residuals share one invariant extended Kalman filter loop. On 24 handheld and aerial sequences, SA-LIVO is the most accurate or tied on 22 and stays bounded where others diverge; on the HILTI 2022
benchmark it needs 13.4 ms per frame on a laptop processor and 2.0-6.2x less
memory than FAST-LIVO2, R3LIVE, and SR-LIVO.
Read the full abstract ▾
All authors Yinong Cao, Xin He, Yijie Wang, Tianzhen Ma, Pingfeng He, Chenyang Zhang, Tingwei Wang, Shouzheng Zhu, Senyuan Wang, Changhui Jiang, Yuwei Chen, Chunlai Li, Genghua Huang, Jianyu Wang
Robotics
Average-Power-Budgeted Underwater Vehicle Control via Constrained Reinforcement Learning
Yinuo Wang, Gavin Tao, Yuze Liu et al. (+1 more)
Underwater vehicles operate from a fixed onboard energy budget that propulsion rapidly depletes, so a controller that completes its task while drawing less thruster power directly extends mission range and endurance. Reinforcement learning yields capable model-free controllers for station-keeping and trajectory tracking, but optimizing task accuracy alone drives the policy toward oscillatory, energy-wasting actuation. The established remedy subtracts an energy penalty from the reward, yet this sets the task-power trade-off through a single weight with no physical units: a target power level cannot be specified, the weight must be re-tuned for every vehicle and task, and a mismatched weight can even raise power. This paper instead formulates energy-efficient
underwater control as a constrained Markov decision process in which average thruster power is subject to an explicit budget, solved with a PPO-Lagrangian
algorithm. The power level is set by declaring a budget in physical units, and a single dual variable is updated online to meet it for each vehicle and task, without manual weight search. Across three vehicles and four tasks in the MarineGym simulator, the energy-constrained policy draws the least power in all twelve settings, reducing it by 14–65% (up to 64.9%) over a task-only baseline and below an energy-reward baseline everywhere, while remaining the smoothest in ten settings and preserving task accuracy except in one deliberately power-limited regime. Imposing energy as an explicit constraint thus offers a tuning-free route to energy-efficient
underwater control that needs no per-vehicle, per-task weight search.
Read the full abstract ▾
All authors Yinuo Wang, Gavin Tao, Yuze Liu, John V. Ringwood
Robotics
Learning to Adapt: Reptile-D-Learning for Robust and Efficient Control Under Parametric Uncertainty
Haipeng Cao, Zhaolong Shen and Quan Quan
Learning-based Lyapunov Control (LLC) provides formal stability guarantees for nonlinear systems, but its validity relies heavily on accurate system models. Parameter variations and uncertainties may invalidate stability constraints, leading to costly retraining. Although D-learning can estimate Lyapunov derivatives without relying on explicit dynamics models, it remains limited by single-task dynamics and degrades under large parameter shifts. We propose Reptile-D-learning, a framework that leverages the Reptile meta-learning
algorithm to capture shared dynamical structures across systems with different parameters, thereby learning a generalizable Lyapunov network initialization and a high-performance controller. Experiments on multiple nonlinear control systems demonstrate that Reptile-D-learning significantly improves both generalization and rapid adaptation to unseen parameter configurations.
Read the full abstract ▾
Robotics
Calousel: Extrinsic Calibration of Non-overlapping Multi-camera Systems from Pure Rotation
Gwanhyeong Song, Chaehyeon Song and Ayoung Kim
Extrinsic calibration of multi-camera systems with non-overlapping FOVs has been a challenging problem in the robotics literature. Conventional target-based methods impose substantial target setup overhead, either deploying large calibration targets or requiring pre-measured multi-target poses. Motion-based approaches instead suffer from drift error, scale ambiguity, and motion degeneracy. Securing both accuracy and usability, we propose a novel calibration method that leverages pure rotational motion, requiring only a single static calibration board. The key idea is to make all cameras sequentially observe the same target under a shared
geometric reference, even without overlapping views. To integrate these time-separated observations, we formulate the problem using a latent turntable frame and a 3D error on SE(3) within a global optimization framework. We validate the proposed method on both a controlled camera rig and a full-scale vehicle platform with heterogeneous cameras, and analyze robustness under non-ideal turntable motion. Extensive experiments show that our approach maintains competitive accuracy without specialized precision hardware, proving its strong suitability for realistic on-site deployments. Our code is publicly available here.
Read the full abstract ▾
Robotics
Event-Adaptive Motion Planning with Distilled Vision-Language Model in Safety-Critical Situations
Zhenwei Huang, Changsheng You, Shuai Wang et al. (+3 more)
Robot navigation in safety-critical scenarios faces significant challenges from unforeseen semantic events, where collisions arise primarily from the unpredictable behaviors of dynamic
agents rather than unseen objects. While large
vision-
language models (VLMs) offer remarkable capabilities in commonsense
reasoning, frequently invoking them within the continuous control loop introduces severe computational latency, fundamentally destabilizing physical execution. To address these challenges, we propose event-adaptive motion planning (EAMP), an efficient framework for VLM-based
robot navigation. Specifically, a
prompt-configurable semantic event trigger (PC-SET) selectively activates semantic intervention by continuously monitoring short temporal clips for behavioral anomalies. Upon triggering, an event-triggered distilled SemNav-VLM, fine-tuned via physically verified semantic
distillation, maps detected anomalies into discrete strategy-level decisions. Subsequently, a semantic model predictive control (SMPC) module translates these strategies into dynamic reconfigurations of optimization objectives and
geometric references. Extensive experiments in safety-critical logistics scenarios demonstrate that EAMP effectively aligns high-level
reasoning with low-level control, significantly improving dynamic safety margins over existing baselines while preserving real-time efficiency.
Read the full abstract ▾
All authors Zhenwei Huang, Changsheng You, Shuai Wang, Chao Zhou, Wei Xu, Yi Gong
Robotics · Cross-listed from cs.AI
Reasonable Motion: A General ASP Foundation for Environment Constrained Movement Trajectory Computation
Julius Monsen, Jakob Suchan, Mehul Bhatt et al. (+1 more)
We present a general answer set programming based hybrid quantitative-qualitative method for computing constrained branching trajectory modes for moving objects in real-world settings. The method performs constrained traversal of an environment graph, enumerating geometrically admissible motion behaviours as stable models, each constituting a distinct trajectory mode characterised by both domain-dependent and independent factors such as derived event sequence, map
topology, and domain norms. The hybrid trajectory computation method is generally applicable across motion characteristics typically encountered in diverse dynamic domains with moving objects, e.g., autonomous driving. We demonstrate applicability and highlight how computed trajectories are traceable to their underlying stable model, thereby affording verifiable interpretability that purely learned approaches cannot provide. We also perform an empirical evaluation with Argoverse 2, a large-scale real-world autonomous driving
benchmark representative of the class of dynamic domains within the scope of the proposed method.
Read the full abstract ▾
All authors Julius Monsen, Jakob Suchan, Mehul Bhatt, Lars Karlsson
Robotics
1000 Rallies: An Event-Camera Dataset and Real-Time Learned Ball-State Estimation for Robotic Table Tennis
Raphaela Kreiser, Asude Aydin, Yin Bi et al. (+3 more)
Robotic table tennis has emerged as a compelling
benchmark for real-time robotic perception due to its fast ball dynamics and stringent timing requirements. Accurate, high-frequency, and low-latency ball state estimation is critical for reliable trajectory prediction and timely control. Traditional frame-based cameras face an inherent trade-off: low frame rates leave temporal blind spots that miss fast-moving objects and high frame rates raise data and computational cost. Event cameras instead offer microsecond temporal resolution and, under sufficient illumination, remain largely free of motion blur even at high ball speeds. However, the community lacks large-scale
datasets to develop and
benchmark event-based perception in realistic sports scenarios. We address this gap by introducing the first large-scale event-camera
dataset for table tennis, comprising over 1000 rallies from a diverse group of players ranging from amateurs to elite-level athletes. Each recording captures the event stream alongside 14 synchronized high-speed frame-based cameras at 200 FPS, which we use to produce 1 kHz pseudo ground-truth labels for ball position, velocity, and
spin. Building on this
dataset, we train a convolutional neural network robust to background player motion that jointly estimates the ball's position and velocity in the image-plane from events. Treating the predicted velocity as an additional measurement in the Kalman filter reduces bounce-point prediction error by 36% relative to a position-only baseline. Finally, we close the perception-action loop by integrating the event-based system with a Stäubli robotic arm, enabling the first real-time human-
robot table tennis rallies driven by event-based perception.
Read the full abstract ▾
All authors Raphaela Kreiser, Asude Aydin, Yin Bi, Claudio Fanconi, Peter Dürr, Naoya Takahashi
Robotics
WOLF-VLA: Whole-Body Humanoid Optimal Locomotion Framework for Vision-Language-Action Learning
Melya Boukheddimi, Omar Adjali, Daniel Sontag et al. (+1 more)
Vision-
Language-Action (VLA) models have recently demonstrated strong generalization in robotic manipulation, yet their applicability to whole-body, contact-rich
humanoid locomotion remains severely underexplored due to data scarcity, the absence of dynamically consistent demonstrations, and the difficulty of encoding optimality and safety in learning-based pipelines. This work introduces a unified framework WOLF-VLA that integrates whole-body optimal-control (OC) motion synthesis with large-scale multi-modal
dataset to train VLAs capable of generating
humanoid locomotion policies directly from natural-
language instructions. We construct a comprehensive
dataset of dynamically feasible
humanoid trajectories across six locomotion-related task families, each parameterized by environmental variations, object colors, placements, and visual distractors. We train a VLA model using the collected joint trajectories, ego-centric visual observations and natural
language instruction, yielding a policy that exhibits strong
reasoning and robustness to initial-condition variability, and competitive performance across several tasks and environment settings. A systematic ablation study demonstrates the impact of each modality on the model performance. The full
dataset, model checkpoints, and benchmarking simulation suite will be openly released, establishing a reproducible dynamically consistent
benchmark for whole-body
humanoid locomotion rich VLA control and enabling future research in scalable transfer of instruction-driven locomotion policies.
Read the full abstract ▾
All authors Melya Boukheddimi, Omar Adjali, Daniel Sontag, Frank Kirchner
Robotics
One Body, Two Minds: Variable Autonomy Approach for a Co-embodied Robotic Hand
Piotr Koczy, Yuchong Zhang, Danica Kragic et al. (+1 more)
Assistive robotic systems face a fundamental trade-off: fully autonomous systems lack user agency, while fully user-controlled systems demand continuous cognitive effort. Existing shared autonomy approaches blend human and
robot commands but are mostly deployed in separate physical bodies. We introduce co-embodiment with variable autonomy, where human and
robot share a single physical body and operate at different autonomy levels across task phases, from mutual autonomy during object search and grasping to human-dominant control during actuation. We present a co-
embodied, wearable robotic hand that has its own “mind” and operates with variable autonomy levels. A learning-from-demonstration visuomotor
diffusion policy enables autonomous grasping when the user positions the hand near known objects. Once grasped, the system signals completion and the human can actuate the grasped tool (drill, spray bottle, infrared thermometer, lighter, and ice-cream scoop) via hands-free head gestures. The human retains veto authority at all times through a release gesture that returns the system to the initial phase. Unlike blended autonomy, where control is continuously negotiated, our co-
embodied approach consists of variable autonomy from full human control to full independent actions while maintaining physical coupling, realizing a one body, two minds paradigm. In a user study with 44 participants performing five bimanual tasks, users rapidly adapted to this “two minds” paradigm: completion times improved by 23.3% across trials ($p < 0.001$, Cohen's $d = 0.94$), the best-performing policy variant reached a 93.6% task success rate, and acceptance ratings were high (5.70/7 overall impression, 5.52/7 daily use willingness). This work establishes co-embodiment with variable autonomy as a viable approach for assistive robotics, enabling human-
robot collaboration through co-embodiment.
Read the full abstract ▾
All authors Piotr Koczy, Yuchong Zhang, Danica Kragic, Michael C. Welle
Robotics
ASSCG: Just-Right Gating over Chattering for Fast-Slow LLM Planning in Autonomous Driving
Sining Ang, Yuan Chen, Liu Haiyan et al. (+5 more)
Large
language models (LLMs) can improve autonomous driving planning but are costly to query online, and existing fast-slow planners often rely on hand-designed triggering rules that either over-call the slow system or call it at the wrong times. We formulate slow-system invocation as a resource-aware sequential decision problem and propose the Adaptive Slow-System Control Gate (ASSCG), which makes frame-level Query/Cache/Drop decisions to refresh, reuse, or suppress slow guidance. ASSCG uses an RWKV backbone for efficient long-horizon gating and is trained with supervised fine-tuning followed by GRPO-style compute-aware reinforcement fine-tuning. We apply ASSCG to two different fast-slow architectures: (i) AsyncDriver on nuPlan Hard20 closed-loop evaluation, where ASSCG improves score to 67.28 (+2.28) while reducing average end-to-end inference latency by 60%; and (ii) a RecogDrive-based dual system that we build by replacing its original VLM-2B module with a lightweight ViT-based fast planner and adding an LLM slow planner, evaluated on NAVSIM, where ASSCG achieves 91.4 PDMS (+0.6) and increases average speed by 25%. The project page, including
video visualizations and additional results, is available at https://williamxuanyu.github.io/asscg/.
Read the full abstract ▾
All authors Sining Ang, Yuan Chen, Liu Haiyan, Xuanyao Mao, Jason Bao, Xuliang, Bingchuan Sun, Yan Wang
Robotics
GROVE: Grounded Pedestrian Simulation via Natural Language for Interactive Social Robot Navigation
Duc Tai Nguyen, Volodymyr Shcherbyna, Anh Do Duc et al. (+3 more)
Pedestrian simulation is a critical component for training and deploying social
robot navigation approaches, yet it remains a largely rigid system that repeatedly requires manual data generation to define even simple scenarios. We propose GROVE, a text-to-scenario pedestrian simulation framework that combines state-of-the-art approaches to produce realistic, socially challenging scenarios for social
robot navigation. Our framework allows users to customize one of several common presets (emergency, queuing, normal) or even enter a fully independent
prompt to generate a highly customizable pedestrian simulation. Multiple modules separately ensure the realism and soundness of long-horizon human behavior, medium-horizon pedestrian
navigation, and short-horizon
robot/social interactions. Each module is tuned by the
prompt in a way that reflects the user intent across all aspects of pedestrian simulation. By dynamically selecting one of several state-of-the-art (SotA) approaches in our modules based on the scenario, we capture many situational nuances of pedestrian behavior in order to narrow the simulation-to-real (sim2real) gap. The human simulation is directly integrated into Isaac Sim, Gazebo, and RViz simulators for
robot deployment in highly social environments. We validate our approach through qualitative comparison against existing pedestrian simulation baselines across scenarios of varying complexity in residential, hospital, and office environments. The result is a high-fidelity pedestrian simulation that challenges social
robot navigation with complex, diverse, realistic human behaviors.
Read the full abstract ▾
All authors Duc Tai Nguyen, Volodymyr Shcherbyna, Anh Do Duc, Zhengcheng Shen, Teham Buiyan, Linh Kästner
Robotics
AISPO: Enhancing Depth Reliability for Robotic Manipulation of Non-Lambertian Objects via Affine-Invariant Shape Prior
Zhiming Chen, Linfang Zheng, Kun Zhang et al. (+4 more)
Reliable depth perception is critical for robotic manipulation, especially for non-Lambertian objects such as transparent or highly specular surfaces, where raw depth measurements are often corrupted or missing. These failures frequently propagate to motion planning, resulting in invalid grasp poses and execution errors. We propose AISPO, a depth completion framework that improves depth reliability for manipulation in challenging sensing conditions. AISPO combines multi-scale RGB-D feature fusion with an affine-invariant shape prior to enforce
geometric consistency and mitigate catastrophic depth failures. Unlike methods that focus primarily on average depth accuracy, our approach emphasizes physical plausibility and structural integrity of the predicted depth maps. Extensive
benchmark evaluations demonstrate competitive performance and strong generalization to unseen objects and novel scenes. Real-world grasping experiments further show that enhanced depth reliability significantly improves manipulation success rates, particularly for transparent objects where many existing methods fail to produce physically usable depth estimates.
Read the full abstract ▾
All authors Zhiming Chen, Linfang Zheng, Kun Zhang, Hyung Jin Chang, Wei Zhang, Hongyu Yu, Hua Chen
Robotics
SAGE-Nav: Leveraging LLM Planning and Alignment Fusion for Hierarchical Scene Graph-Guided Navigation
Hao Su, Yuehao Huang, Yukai Ma et al. (+2 more)
Object-Goal
Navigation (ObjNav) requires
embodied agents to autonomously locate specified targets using only
egocentric visual observations. Existing monolithic methods struggle with long-horizon
reasoning and generalize poorly to novel environments. To address these limitations, we propose SAGE-Nav, a novel hierarchical framework that integrates the
reasoning capabilities of Large
Language Models (LLMs) with dynamic scene graphs. Crucially, it decouples asynchronous global semantic planning from the high-frequency reactive control loop. The LLM serves as a global planner, decomposing abstract instructions into a sequence of semantically grounded waypoints. To translate these plans into dense multi-modal guidance, we design a Hierarchical Scene Graph Encoder (HSGE) that leverages relational graph convolutions to produce structure-aware
embeddings preserving both semantic and spatial
topology. Furthermore, we develop the Goal-aware
Alignment-Fusion Network (GAFN) to dynamically fuse real-time perception with these structural priors. Using an adaptive gating mechanism with an explicit inductive bias, GAFN ensures robust visual-topological
alignment for the low-level policy. Extensive evaluations in the i-THOR and RoboTHOR environments demonstrate that SAGE-Nav achieves state-of-the-art performance, delivering substantial gains in
navigation efficiency and zero-shot generalization while maintaining the low control latency required for physical robotic deployment.
Read the full abstract ▾
All authors Hao Su, Yuehao Huang, Yukai Ma, Yong Liu, Jiajun Lv
Robotics
Generative AI for Safe and Photorealistic Drone Light Shows
Pascal Reinhold, Alexander Gräfe and Sebastian Trimpe
Drone light shows are redefining aerial entertainment, yet their widespread adoption is bottlenecked by labor-intensive, manual animation. While generative AI promises an automated alternative, current frameworks fail to provide photorealism with fluid, dynamic motion. To address this limitation, we introduce SWAN, an end-to-end pipeline that synthesizes photorealistic, large-scale, and collision-free drone choreographies directly from text
prompts. SWAN converts text into realistic reference
videos and translates these pixel-space dynamics into physical swarm kinematics using a novel, adaptive point-tracking
algorithm. Unlike existing trackers, this method maintains spatial coherence through severe occlusions and rapid topological shifts. A dedicated planner then allocates these trajectories to individual drones, while a subsequent safety filter ensures collision-free execution. We demonstrate scalability by safely orchestrating simulated 2,000-drone formations and validate physical feasibility on a dense real-world swarm of 49 quadcopters, operating everything entirely on standard consumer hardware. Combined, this work demonstrates how generative AI can be leveraged to automate multi-
robot choreography design, providing an accessible new framework for drone light shows.
Read the full abstract ▾
Robotics
Delta-Position Estimation-Based IMU Odometry: A Comparison of MLP and Kolmogorov-Arnold Networks
Osman Tokluoglu and Emin Keresteci
In this study, the learning-based inertial odometry problem is investigated using raw IMU measurements obtained from the EuRoC MAV
benchmark dataset. Instead of absolute position regression-a formulation that may lead to large constant errors-the models are trained to estimate the incremental displacement ({\Delta}p) over a fixed 50 ms sliding window, and the full trajectory is reconstructed through numerical integration. A standard Multi-Layer Perceptron (MLP) is compared with a Kolmogorov-Arnold Network (KAN) equipped with learnable B-spline activations. Although KAN has 6.9 times fewer parameters than MLP (8,444 versus 57,859), it produces a 44% lower error in terms of final cumulative drift on the test trajectory (9.61 m versus 17.23 m). In addition, KAN exhibits more stable behavior in terms of long-term error accumulation, with lower P_50 and P_90 cumulative drift values. These findings indicate that learnable B-spline-based activations have the potential to reduce error accumulation in the inertial odometry problem.
Read the full abstract ▾
Robotics
HEART: Coordination of Heterogeneous Expert Agents for Physically Grounded Robotic Task Planning
Junho Lee, Seabin Lee, Wonjong Lee et al. (+3 more)
Large
Language Models (LLMs) can reason over complex instructions but often fail to satisfy the physical and spatial constraints required for robotic task planning. Recent LLM-based planners directly translate text into action sequences, yet they lack structured
reasoning about feasibility, reachability, and logical order, resulting in invalid or incomplete plans. We present a heterogeneous multi-LLM framework that decomposes instructions into atomic
reasoning tasks and allocates them to role-specialized expert
agents under a
token budget for real-world computational and communicational constraints. By combining role-oriented
reasoning from heterogeneous
agents followed by constraint-driven plan synthesis, HEART validates capability, reachability, and constraint conditions before planning and helps produce physically executable plans while maintaining efficiency. Experiments across different household
benchmarks show that HEART consistently improves plan success compared to single-LLM and rule-based planners, demonstrating that heterogeneous LLM collaboration enables robust and scalable robotic task planning under resource constraints.
Read the full abstract ▾
All authors Junho Lee, Seabin Lee, Wonjong Lee, Nayoung Kim, Moonjeong Kang, Changjoo Nam
Robotics
MAPL: Multi-Objective Preference Learning for Robot Locomotion
Xiyue Chen, Muhan Lin, Shuyang Shi et al. (+1 more)
Reward design remains a major bottleneck in reinforcement learning for
robot locomotion, where successful policies often depend on carefully tuned, task-specific reward functions. Preference-based reinforcement learning offers an alternative, but existing LLM-based methods typically ask for a single overall judgment between behaviors, making it difficult to capture the multiple competing objectives that underlie high-quality locomotion. We present Multi-Objective AI-Informed Preference Learning (MAPL), a framework that learns locomotion rewards from high-level natural
language objectives rather than manually engineered reward equations. MAPL
prompts a large
language model to compare trajectories independently along semantically meaningful criteria, using generic
language descriptions that are terrain-invariant and require little domain expertise. These objective-wise preferences are used to train a multi-head preference scoring model, whose outputs are aggregated to form a scalar reward for policy optimization. Across four quadruped locomotion environments, MAPL trains policies using only LLM-generated preferences and achieves performance comparable to or better than expert-designed rewards, while eliminating task-specific reward engineering.
Read the full abstract ▾
All authors Xiyue Chen, Muhan Lin, Shuyang Shi, Joseph Campbell
Robotics
Large-Scale Tunnel Air-Ground Collaboration With FLISP: Fast LiDAR-IMU Synchronized Path Planner
Fenghe Guo, Runjie Shen, Chenyang Sun et al. (+4 more)
Hydropower tunnel inspection is critical for infrastructure integrity yet remains inefficient and hazardous using manual methods. We propose FLISP (Fast
LiDAR-IMU Synchronized Path Planner), a mapless planning framework for cooperative UGV-
UAV inspection. Unlike traditional map-based paradigms, FLISP features three core contributions: (1) a unified architecture where a single UGV-mounted
LiDAR-IMU suite drives synchronized path generation for both platforms; (2) platform-specific solvers utilizing an enhanced Firefly
Algorithm for UGV obstacle avoidance and a dynamic iterative optimizer for
UAV flight; and (3) a hierarchical refinement strategy ensuring kinematic feasibility without state estimation drift.
Benchmarks in a 1.2 km operational tunnel demonstrate that FLISP circumvents structural bottlenecks of map-based methods, eliminating map rasterization overhead (Fast-LIO2 + A*) and sampling instability (LIO-SAM + RRT*). FLISP achieves a 100% success rate with 7 ms latency, representing a 7-fold speedup over grid-based and a three-order-of-magnitude improvement over sampling-based baselines. Validated in operational hydropower tunnels, this approach offers a scalable solution for robotic inspection in feature-degraded linear infrastructure. A demonstration
video is available at https://youtu.be/Y_ezs1PfLJ4, and the code at https://github.com/ArchibaldGuo/FLISP.git.
Read the full abstract ▾
All authors Fenghe Guo, Runjie Shen, Chenyang Sun, Junrui Zhang, Quanxi Zhan, Yongchun Wang, Junjie Zhang
Robotics
Commerge: Communication-Efficient, Robust, and Fast LiDAR Map Merging Framework for Multi-Robot Coordination in Resource-Constrained Scenarios
Hogyun Kim, Jiwon Choi, Juwon Kim et al. (+4 more)
By maintaining global consistency across
robot teams, multi-
robot LiDAR map merging enables faster exploration and efficient area coverage. However, map merging requires exchanging massive sensor data between the server and
robots, making communication the bottleneck, especially in communication-constrained environments. Therefore, we present Commerge, a communication-efficient map merging framework that achieves bandwidth reduction through graph-theoretic selective data exchange. By doing so, our Commerge reduces inter-
robot communication by up to 5,000x while maintaining
alignment accuracy. Our key insight is that only a small subset of carefully selected scans is sufficient for robust map merging. We formulate this as a three-stage cascaded optimization problem on an exchange graph, where vertices represent
robot keyframes and edges denote candidate inter-
robot loops. Through three cascade stages, we select a sequentially overlapped, balanced-transmission-cost, and geometrically-perceptually optimal scan subset that preserves
alignment quality while reducing communication. Unlike existing approaches that either transmit whole scans, which require GB-scale data exchange, or employ naive downsampling, our approach exchanges only MB-scale data while achieving comparable
alignment accuracy. Extensive evaluation on five public
datasets and four in-house
datasets covering cave, planetary-analog, indoor, and outdoor campus environments shows up to 99.98% reduction in data exchange (e.g., from 7,000MB to 1.3MB on the HeLiPR
dataset), while maintaining
alignment performance across embedded to desktop platforms. The supplementary materials are available at https://sparolab.github.io/research/commerge.
Read the full abstract ▾
All authors Hogyun Kim, Jiwon Choi, Juwon Kim, Geonmo Yang, Seokhwan Jeong, Hyungtae Lim, Younggun Cho
Robotics
Reliability-Asymmetric Spacecraft Autonomy: Co-Designing a Capable Learned GNC Stack with a Verified, Adaptation-Aware Runtime Shield
Alireza Shojaei
Deep-space missions need onboard autonomy that is both capable and certifiable. Rule-based autonomy is certifiable but brittle, while learned autonomy is capable but hard to verify. We present AMPLE-GNC, a three-tier guidance,
navigation, and control stack. Its capability path combines a small foundation-model commander that maps natural
language to PDDL+, a constraint-screening verifier, and a fault-adaptive controller. All three are bounded by a runtime shield with nine linear-temporal-logic invariants whose predictor soundness is machine-checked by the Kind 2 model checker. On a 6-DOF Basilisk testbed, we make three contributions. First, we deploy an edge commander. Fine-tuning a pretrained 360M model with grammar-constrained decoding gives a hard output-validity guarantee and 84% planner-executable actions. On a de-leaked test, novel-phrasing generalization is 38% exact and 51% action, rising to 48% exact after phrasing-diversity re-finetuning; we separate syntactic validity from semantic accuracy. Second, we introduce a fault-adaptive controller. Rapid Motor Adaptation infers latent actuator faults online and recovers 97.8% of actuator-sign faults and 94.4% of continuous-gain faults within the training randomization envelope. Fault-unaware PD and from-scratch end-to-end RL both score 0%, while the strongest classical-adaptive baseline reaches 55% on continuous gain. Beyond the envelope, a split-conformant retrain scores 57-67%, and adding 4x more in-regime data worsens performance, showing that randomization breadth, not data volume, drives generalization. Robustness is flat under
star-tracker noise to 0.005. Third, we show that a latching safe-hold shield can suppress even a capable controller. A split-conformal recovery-deadline certificate with adaptation-aware engagement reconciles safety and recovery, keeping the controller 94.5% autonomous while still catching non-recovery.
Read the full abstract ▾
Robotics
ResCue: Residual Spatial Cueing for Language-Conditioned Imitation Learning
Yanzhe Tang, Xinyu Shao, Yuxuan Hu et al. (+8 more)
Visual imitation policies must learn which image regions matter for action, whether applying the same operation to different objects or selecting a
language-specified target. We introduce ResCue (Residual Spatial Cueing), which converts
language-derived target information into a spatial condition for visual policy learning. Pretrained grounding produces a target mask; an independent cue encoder adds its features to spatial RGB features before further visual processing and pooling. This retains scene context while making the target explicit. With
Diffusion Policy (DP), ResCue achieves 56.5% average success across 15 simulated tasks, versus 33.1% for DP and 40.% for the strongest baseline, $\pi_0$, with gains in both ordinary manipulation and instruction-dependent object selection. On three real-world bimanual tasks, it reaches 60.0%, compared with 31.7% for $\pi_0$. Matched-mask ablations favor residual fusion over input-level and pooled alternatives, while test-time interventions show that target-correct guidance matters for execution.
Read the full abstract ▾
All authors Yanzhe Tang, Xinyu Shao, Yuxuan Hu, Siyu Chen, Chenxi Zhai, Xi Cheng, Bowen Yang, Yajun Gao, Tongtong Cao, Xiu Li, Long Zeng
Robotics
Self Capacitive Tactile Sensor System designed for Companion Robots
Mohsin Ali, Hidenobu Sumioka and Shuhei Ikemoto
Tactile sensing is essential for
humanoid robots to achieve safe physical interaction, dexterous manipulation, and truly human-like responsiveness. However, the design of such systems remains challenging. Conventional approaches often suffer from complex multilayer structures, intricate wiring, high cost, and poor scalability, making it difficult to realize full-body tactile sensing with real-time, low-latency detection while maintaining minimal computational load on the
robot's main processor. In this work, we present a simple, scalable and hardware friendly tactile sensing system for a companion
humanoid robot based on the self-capacitance principle. The proposed sensor system employs a single conductive fabric layer with a conductive fabric wire architecture and does not require intricate electrode patterning. Scalability was demonstrated by fabricating a 100-point sensor array on a flexible printed
circuit (FPC). Evaluation across sampling frequencies showed that 10 Hz is insufficient and misses transient events, whereas 100 Hz and 1000 Hz reliably capture and clearly distinguish all interaction types: gentle touch, slow tapping, fast tapping, and hitting. A decision-tree classifier was implemented directly on the FPGA, offloading real-time inference from the Raspberry Pi 4 with minimal latency and negligible power overhead. This design fully meets the tactile sensing requirements of the HIRO-chan
robot and is well-suited for full-body tactile sensing in HIRO-chan and other companion
robots.
Read the full abstract ▾
Robotics
AI Coaching for Accelerating Human Skill Development with Reinforcement Learning
Wei Wang, Enlin Gu, Antonio Loquercio et al. (+2 more)
AI copilots can substantially boost human performance through shared control, but excessive assistance can induce over-reliance and skill atrophy. This paper studies how an
embodied AI
agent can act as a coach that accelerates human motor-skill development. We argue that effective coaching requires strategic scaffolding and stepping back that are aligned with the learner's capability, allowing productive failures that drive learning. We formalize the interactive AI coaching process as a non-cooperative dynamic game in which the learner optimizes task performance while the coach targets the learner's independent competence. Building on this formalism, we develop a reinforcement learning framework combining adaptive shared control with probabilistic models of the coach's causal influence on skill
evolution, enabling tractable training of coaching policies. A comprehensive user study (N=33) on first-person-view drone racing shows significant gains in human learning outcomes over state-of-the-art AI coaching baselines.
Read the full abstract ▾
All authors Wei Wang, Enlin Gu, Antonio Loquercio, Haimin Hu, Rahul Mangharam
Robotics
WaveForward: An Omnidirectional Passive Wheeled Quadruped Robot with Casters
Chuanlin Zhao, Qifeng Zheng, Shuhan Wang et al. (+3 more)
Wheeled-legged
robots possess both agile mobility for traversing complex terrains and high efficiency, making them suitable for long-distance transportation applications. Conventional actuated wheeled
robots require specialized hardware and electrical design due to the incorporation of wheel components. We propose a novel and low-cost passive wheeled legged
robot equipped with standard casters on each leg to obtain omnidirectional mobility. The control method employs an asymmetric actor-critic structure, enabling the utilization of the privileged information of the passive caster's angles and velocities. We develop a caster base posture adjustment strategy based on velocity commands, utilizing actuated joints to modify the caster base joint axis posture and thereby adjust the propulsion direction of the casters. Moreover, we implemented multiple propulsion modes to achieve varying degrees of caster twisting oscillation, converting these into propulsive force. We conducted a slalom test and mode switch experience, which shows the passive wheeled quadruped could achieve omnidirectional movement versatility, and reduce the cost of transport (COT) by up to 89.1% with respect to legged motion.
Read the full abstract ▾
All authors Chuanlin Zhao, Qifeng Zheng, Shuhan Wang, Tiancheng Ma, Weixian Lin, Xin Luo
Robotics
DynaMOMA: Instantaneous Prediction of Grasp Poses for Mobile Manipulation of Dynamic Objects
Zhinan Yu, Junyan Xu, Jiazhao Zhang et al. (+9 more)
Mobile manipulation is a fundamental robotics task and has advanced rapidly in recent years, enabling
robots to navigate, reach, and interact with objects in complex environments. However, mobile manipulation of dynamic objects remains highly challenging, as
robots must coordinate the mobile base and arm while adapting to continuously evolving target poses. A key challenge lies in predicting temporally consistent short-horizon grasp trajectories from dynamic observations. In this work, we propose \ours{}, a dynamic mobile manipulation framework that couples instantaneous grasp trajectory prediction with whole-body control policy. Our predictor uses an anchor-based
diffusion model to generate temporally consistent short-horizon grasp trajectories conditioned on historical observations. The predicted trajectories are then encoded as compact features and fed to a whole-body reinforcement learning policy, which controls the mobile manipulator for dynamic grasping. We further introduce a anticipation-guided reward that equips the policy with an anticipatory grasping horizon by adaptively shifting the target from the current grasp observation to the instantaneously predicted grasp trajectory. Through extensive experiments in Isaac Gym simulation, we show that our method achieves strong performance in mobile manipulation of dynamic objects across diverse settings and grasping metrics. Furthermore, our predictor and policy demonstrate strong generalizability in real-world experiments.
Read the full abstract ▾
All authors Zhinan Yu, Junyan Xu, Jiazhao Zhang, Zheng Qin, Yijie Tang, Yuhang Huang, Yihan Cao, Zhiyuan Yu, Yongjun Wang, Renjiao Yi, Chenyang Zhu, Kai Xu
Robotics
An Integrated Hardware-Software Design for Low-Data Spatial Defect Detection in Robotic Visual Inspection with Hybrid Optoelectronic Neural Networks
Chaoqing Tang, Jiaxuan Li, Huanze Zhuang et al. (+4 more)
To address data overload and inefficient shape-level annotation in robotic visual inspection, this paper proposes a hardware-software integrated optoelectronic architecture. A non-imaging, low-data paradigm is established to minimize annotation dependency. First, a sensor-in-the-loop strategy reconfigures a Digital Micromirror Device (DMD) as a physical optical convolutional layer, enabling photonic-domain feature extraction that unifies sensing hardware and processing software. To suppress data volume at the source, a block-based compressed sensing strategy encodes spatial information into low-dimensional temporal signals, drastically reducing redundancy. Subsequently, to bypass laborious manual defect shape annotation, natural
language descriptions guide the network to align with highly generalizable features from Contrastive
Language-Image Pre-training (CLIP), steering the
attention maps of the optoelectronic neural network toward defect shapes. Furthermore, a Localization Accuracy for
Attention (LAA) metric is proposed to quantify shape-level defect localization performance. Experiments on transparent material defect detection validate the system's effectiveness. Parametric analysis reveals how measurement
matrices, compression ratios, and block sizes affect accuracy. Results show that, compared to traditional imaging, the proposed architecture maintains equivalent accuracy while reducing data volume by 90% for
Vision Transformers and computational workload by 60% for Convolutional Neural Networks. This low-data paradigm offers an efficient solution for industrial automation scenarios involving massive data streams, high acquisition costs, or constrained edge resources.
Read the full abstract ▾
All authors Chaoqing Tang, Jiaxuan Li, Huanze Zhuang, Guiyun Tian, Chao Wang, Yihao Ouyang, Wenzhong Liu