Robotics · Cross-listed from cs.CV
SCC-Loc: A Unified Semantic Cascade Consensus Framework for UAV Thermal Geo-Localization
Xiaoran Zhang, Yu Liu, Jinyu Liang et al. (+3 more)
Cross-modal Thermal Geo-localization (TG) provides a robust, all-weather solution for Unmanned Aerial Vehicles (
UAVs) in Global
Navigation Satellite System (GNSS)-denied environments. However, profound thermal-visible modality gaps introduce severe feature ambiguity, systematically corrupting conventional coarse-to-fine registration. To dismantle this bottleneck, we propose SCC-Loc, a unified Semantic-Cascade-Consensus localization framework. By sharing a single DINOv2 backbone across global
retrieval and MINIMA$_{\text{RoMa}}$ matching, it minimizes
memory footprint and achieves zero-shot, highly accurate absolute position estimation. Specifically, we tackle modality ambiguity by introducing three cohesive components. First, we design the Semantic-Guided Viewport
Alignment (SGVA) module to adaptively optimize satellite crop regions, effectively correcting initial spatial deviations. Second, we develop the Cascaded Spatial-Adaptive Texture-Structure Filtering (C-SATSF) mechanism to explicitly enforce
geometric consistency, thereby eradicating dense cross-modal outliers. Finally, we propose the Consensus-Driven Reliability-Aware Position Selection (CD-RAPS) strategy to derive the optimal solution through a synergy of physically constrained pose optimization. To address data scarcity, we construct Thermal-
UAV, a comprehensive
dataset providing 11,890 diverse thermal queries referenced against a large-scale satellite ortho-photo and corresponding spatially aligned Digital Surface Model (DSM). Extensive experiments demonstrate that SCC-Loc establishes a new state-of-the-art, suppressing the mean localization error to 9.37 m and providing a 7.6-fold accuracy improvement within a strict 5-m threshold over the strongest baseline. Code and
dataset are available at https://github.com/FloralHercules/SCC-Loc.
Read the full abstract ▾
All authors Xiaoran Zhang, Yu Liu, Jinyu Liang, Kangqiushi Li, Zhiwei Huang, Huaxin Xiao
Robotics
Rémi Marsal, Quentin Picard, Adrien Poiré et al. (+5 more)
Off-road autonomous
navigation demands reliable 3D perception for robust obstacle detection in challenging unstructured terrain. While
LiDAR is accurate, it is costly and power-intensive.
Monocular depth estimation using foundation models offers a lightweight alternative, but its integration into outdoor
navigation stacks remains underexplored. We present an open-source off-road
navigation stack supporting both
LiDAR and
monocular 3D perception without task-specific training. For the
monocular setup, we combine zero-shot depth prediction (Depth Anything V2) with metric depth rescaling using sparse SLAM measurements (VINS-Mono). Two key enhancements improve robustness: edge-masking to reduce obstacle
hallucination and temporal smoothing to mitigate the impact of SLAM instability. The resulting point cloud is used to generate a
robot-centric 2.5D elevation map for costmap-based planning. Evaluated in photorealistic simulations (Isaac Sim) and real-world unstructured environments, the
monocular configuration matches high-resolution
LiDAR performance in most scenarios, demonstrating that foundation-model-based
monocular depth estimation is a viable
LiDAR alternative for robust off-road
navigation. By open-sourcing the
navigation stack and the simulation environment, we provide a complete pipeline for off-road
navigation as well as a reproducible
benchmark. Code available at https://github.com/LARIAD/Offroad-Nav.
Read the full abstract ▾
All authors Rémi Marsal, Quentin Picard, Adrien Poiré, Sébastien Kerbourc'h, Thibault Toralba, Clément Yver, Alexandre Chapoutot, David Filliat
Robotics
Flash-Mono: Feed-Forward Accelerated Gaussian Splatting Monocular SLAM
Zicheng Zhang, Ke Wu, Xiangting Meng et al. (+3 more)
Monocular 3D Gaussian Splatting SLAM suffers from critical limitations in time efficiency,
geometric accuracy, and multi-view consistency. These issues stem from the time-consuming $\textit{Train-from-Scratch}$ optimization and the lack of inter-frame scale consistency from single-frame geometry priors. We contend that a feed-forward paradigm, leveraging multi-frame context to predict Gaussian attributes directly, is crucial for addressing these challenges. We present Flash-Mono, a system composed of three core modules: a feed-forward prediction frontend, a 2D Gaussian Splatting mapping backend, and an efficient hidden-state-based loop closure module. We trained a recurrent feed-forward frontend model that progressively aggregates multi-frame visual features into a hidden state via cross
attention and jointly predicts camera poses and per-pixel Gaussian properties. By directly predicting Gaussian attributes, our method bypasses the burdensome per-frame optimization required in optimization-based GS-SLAM, achieving a $\textbf{10x}$ speedup while ensuring high-quality rendering. The power of our recurrent architecture extends beyond efficient prediction. The hidden states act as compact submap descriptors, facilitating efficient loop closure and global $\mathrm{Sim}(3)$ optimization to mitigate the long-standing challenge of drift. For enhanced
geometric fidelity, we replace conventional 3D Gaussian ellipsoids with 2D Gaussian surfels. Extensive experiments demonstrate that Flash-Mono achieves state-of-the-art performance in both tracking and mapping quality, highlighting its potential for
embodied perception and real-time reconstruction applications. Project page: https://victkk.github.io/flash-mono.
Read the full abstract ▾
All authors Zicheng Zhang, Ke Wu, Xiangting Meng, Keyu Liu, Jieru Zhao, Wenchao Ding
Robotics
Joint Prediction of Human Motions and Actions in Human-Robot Collaboration
Alessandra Bulanti, Alessandro Carfì and Fulvio Mastrogiovanni
Fluent human–
robot collaboration requires
robots to continuously estimate human behaviour and anticipate future intentions. This entails
reasoning jointly about continuous movements and discrete actions, which are still largely modelled in isolation. In this paper, we introduce MA-HERP, a hierarchical and recursive probabilistic framework for the joint estimation and prediction of human movements and actions. The model combines: (i) a hierarchical representation in which movements compose into actions through admissible Allen interval relations, (ii) a unified probabilistic factorisation coupling continuous dynamics, discrete labels, and durations, and (iii) a recursive inference scheme inspired by
Bayesian filtering, alternating top-down action prediction with bottom-up sensory evidence. We present a preliminary experimental evaluation based on neural models trained on musculoskeletal simulations of reaching movements, showing accurate motion prediction, robust action inference under noise, and computational performance compatible with on-line human–
robot collaboration.
Read the full abstract ▾
Robotics
Enhancing Multi-Robot Exploration Using Probabilistic Frontier Prioritization with Dirichlet Process Gaussian Mixtures
John Lewis Devassy, Meysam Basiri, Mário A. T. Figueiredo et al. (+1 more)
Multi-
agent autonomous exploration is essential for applications such as environmental monitoring, search and rescue, and industrial-scale surveillance. However, effective coordination under communication constraints remains a significant challenge. Frontier exploration
algorithms analyze the boundary between the known and unknown regions to determine the next-best view that maximizes exploratory gain. This article proposes an enhancement to existing frontier-based exploration
algorithms by introducing a probabilistic approach to frontier prioritization. By leveraging Dirichlet process Gaussian mixture model (DP-GMM) and a probabilistic formulation of information gain, the method improves the quality of frontier prioritization. The proposed enhancement, integrated into two state-of-the-art multi-
agent exploration
algorithms, consistently improves performance across environments of varying clutter, communication constraints, and team sizes. Simulations showcase an average gain of $10\%$ and $14\%$ for the two
algorithms across all combinations. Successful deployment in real-world experiments with a dual-drone system further corroborates these findings.
Read the full abstract ▾
All authors John Lewis Devassy, Meysam Basiri, Mário A. T. Figueiredo, Pedro U. Lima
Robotics
ARM: Advantage Reward Modeling for Long-Horizon Manipulation
Yiming Mao, Zixi Yu, Weixin Mao et al. (+5 more)
Long-horizon robotic manipulation remains challenging for reinforcement learning (RL) because sparse rewards provide limited guidance for credit assignment. Practical policy improvement thus relies on richer intermediate supervision, such as dense progress rewards, which are costly to obtain and ill-suited to non-monotonic behaviors such as backtracking and recovery. To address this, we propose Advantage Reward Modeling (ARM), a framework that shifts from hard-to-quantify absolute progress to estimating relative advantage. We introduce a cost-effective tri-state labeling strategy – Progressive, Regressive, and Stagnant – that reduces human cognitive overhead while ensuring high cross-annotator consistency. By training on these intuitive signals, ARM enables automated progress annotation for both complete demonstrations and fragmented DAgger-style data. Integrating ARM into an offline RL pipeline allows for adaptive action-reward reweighting, effectively filtering suboptimal samples. Our approach achieves a 99.4% success rate on a challenging long-horizon towel-folding task, demonstrating improved stability and data efficiency over current VLA baselines with near-zero human intervention during policy training.
Read the full abstract ▾
All authors Yiming Mao, Zixi Yu, Weixin Mao, Yinhao Li, Qirui Hu, Zihan Lan, Minzhao Zhu, Hua Chen
Robotics
Behavior-Constrained Reinforcement Learning with Receding-Horizon Credit Assignment for High-Performance Control
Siwei Ju, Jan Tauberschmidt, Oleg Arenz et al. (+2 more)
Learning high-performance control policies that remain consistent with expert behavior is a fundamental challenge in robotics. Reinforcement learning can discover high-performing strategies but often departs from desirable human behavior, whereas imitation learning is limited by demonstration quality and struggles to improve beyond expert data. We propose a behavior-constrained reinforcement learning framework that improves beyond demonstrations while explicitly controlling deviation from expert behavior. Because expert-consistent behavior in dynamic control is inherently trajectory-level, we introduce a receding-horizon predictive mechanism that models short-term future trajectories and provides look-ahead rewards during training. To account for the natural variability of human behavior under disturbances and changing conditions, we further condition the policy on reference trajectories, allowing it to represent a distribution of expert-consistent behaviors rather than a single deterministic target. Empirically, we evaluate the approach in high-fidelity race car simulation using data from professional drivers, a domain characterized by extreme dynamics and narrow performance margins. The learned policies achieve competitive lap times while maintaining close
alignment with expert driving behavior, outperforming baseline methods in both performance and imitation quality. Beyond standard
benchmarks, we conduct human-grounded evaluation in a driver-in-the-loop simulator and show that the learned policies reproduce setup-dependent driving characteristics consistent with the feedback of top-class professional race drivers. These results demonstrate that our method enables learning high-performance control policies that are both optimal and behavior-consistent, and can serve as reliable surrogates for human decision-making in complex control systems.
Read the full abstract ▾
All authors Siwei Ju, Jan Tauberschmidt, Oleg Arenz, Peter van Vliet, Jan Peters
Robotics
Asymptotically-Bounded 3D Frontier Exploration enhanced with Bayesian Information Gain
John Lewis, Meysam Basiri and Pedro U. Lima
Robotic exploration in large-scale environments is computationally demanding due to the high overhead of processing extensive frontiers. This article presents an OctoMap-based frontier exploration
algorithm with predictable, asymptotically bounded performance. Unlike conventional methods whose complexity scales with environment size, our approach maintains a complexity of $\mathcal{O}(|\mathcal{F}|)$, where $|\mathcal{F}|$ is the number of frontiers. This is achieved through strategic forward and inverse sensor modeling, which enables approximate yet efficient frontier detection and maintenance. To further enhance performance, we integrate a
Bayesian regressor to estimate information gain, circumventing the need to explicitly count unknown voxels when prioritizing viewpoints. Simulations show the proposed method is more computationally efficient than the existing OctoMap-based methods and achieves computational efficiency comparable to baselines that are independent of OctoMap. Specifically, the
Bayesian-enhanced framework achieves up to a $54\%$ improvement in total exploration time compared to standard deterministic frontier-based baselines across varying spatial scales, while guaranteeing task completion. Real-world experiments confirm the computational bounds as well as the effectiveness of the proposed enhancement.
Read the full abstract ▾
Robotics
A Flow Matching Framework for Soft-Robot Inverse Dynamics
Hang Yang, Fangju Yang, Yangming Zhang et al. (+5 more)
Learning the inverse dynamics of soft continuum
robots remains challenging due to high-dimensional nonlinearities and complex actuation coupling. Conventional feedback-based controllers often suffer from control chattering due to corrective oscillations, while deterministic regression-based learners struggle to capture the complex nonlinear mappings required for accurate dynamic tracking. Motivated by these limitations, we propose an inverse-dynamics framework for open-loop feedforward control that learns the system's differential dynamics as a generative transport map. Specifically, inverse dynamics is reformulated as a conditional flow-matching problem, and Rectified Flow (RF) is adopted as a lightweight instance to generate physically consistent control inputs rather than conditional averages. Two variants are introduced to further enhance physical consistency: RF-Physical, utilizing a physics-based prior for residual modeling; and RF-FWD, integrating a forward-dynamics consistency loss during flow matching. Extensive evaluations demonstrate that our framework reduces trajectory tracking RMSE by over 50% compared to standard regression baselines (MLP, LSTM,
Transformer). The system sustains stable open-loop execution at a peak end-effector velocity of 1.14 m/s with sub-millisecond inference latency (0.995 ms). This work demonstrates flow matching as a robust, high-performance paradigm for learning differential inverse dynamics in soft robotic systems.
Read the full abstract ▾
All authors Hang Yang, Fangju Yang, Yangming Zhang, Ibrahim Alsarraj, Yuhao Wang, Zhenye Luo, Zixi Chen, Ke Wu
Robotics
Open-Loop Planning, Closed-Loop Verification: Speculative Verification for VLA
Zihua Wang, Zhitao Lin, Ruibo Li et al. (+4 more)
Vision-
Language-Action (VLA) models, as large foundation models for
embodied control, have shown strong performance in manipulation tasks. However, their performance comes at high inference cost. To improve efficiency, recent methods adopt action chunking, which predicts a sequence of future actions for open-loop execution. Although effective for reducing computation, open-loop execution is sensitive to environmental changes and prone to error accumulation due to the lack of close-loop feedback. To address this limitation, we propose Speculative Verification for VLA Control (SV-VLA), a framework that combines efficient open-loop long-horizon planning with lightweight closed-loop online verification. Specifically, SV-VLA uses a heavy VLA as a low-frequency macro-planner to generate an action chunk together with a planning context, while a lightweight verifier continuously monitors execution based on the latest observations. Conditioned on both the current observation and the planning context, the verifier compares the planned action against a closed-loop reference action and triggers replanning only when necessary. Experiments demonstrate that SV-VLA combines the efficiency of chunked prediction with the robustness of closed-loop control, enabling efficient and reliable VLA-based control in dynamic environments. Code is available: https://github.com/edsad122/SV-VLA.
Read the full abstract ▾
All authors Zihua Wang, Zhitao Lin, Ruibo Li, Yu Zhang, Xu Yang, Siya Mi, Xiu-Shen Wei
Robotics
Learning Task-Invariant Properties via Dreamer: Enabling Efficient Policy Transfer for Quadruped Robots
Junyang Liang, Yuxuan Liu, Yabin Chang et al. (+5 more)
Achieving quadruped
robot locomotion across diverse and dynamic terrains presents significant challenges, primarily due to the discrepancies between simulation environments and real-world conditions. Traditional sim-to-real transfer methods often rely on manual feature design or costly real-world fine-tuning. To address these limitations, this paper proposes the DreamTIP framework, which incorporates Task-Invariant Properties learning within the Dreamer world model architecture to enhance sim-to-real transfer capabilities. Guided by large
language models, DreamTIP identifies and leverages Task-Invariant Properties, such as contact stability and terrain clearance, which exhibit robustness to dynamic variations and strong transferability across tasks. These properties are integrated into the world model as auxiliary prediction targets, enabling the policy to learn representations that are insensitive to underlying dynamic changes. Furthermore, an efficient adaptation strategy is designed, employing a mixed replay buffer and regularization constraints to rapidly calibrate to real-world dynamics while effectively mitigating representation collapse and catastrophic forgetting. Extensive experiments on complex terrains, including Stair, Climb, Tilt, and Crawl, demonstrate that DreamTIP significantly outperforms state-of-the-art baselines in both simulated and real-world environments. Our method achieves an average performance improvement of 28.1% across eight distinct simulated transfer tasks. In the real-world Climb task, the baseline method achieved only a 10\ success rate, whereas our method attained a 100% success rate. These results indicate that incorporating Task-Invariant Properties into Dreamer learning offers a novel solution for achieving robust and transferable
robot locomotion.
Read the full abstract ▾
All authors Junyang Liang, Yuxuan Liu, Yabin Chang, Junfan Lin, Junkai Ji, Hui Li, Changxin Huang, Jianqiang Li
Robotics
RAGE: A Tightly Coupled Radar-Aided Grip Estimator For Autonomous Race Cars
Davide Malvezzi, Nicola Musiu, Eugenio Mascaro et al. (+2 more)
Real-time estimation of vehicle-tire-road friction is critical for allowing autonomous race cars to safely and effectively operate at their physical limits. Traditional approaches to measure tire grip often depend on costly, specialized sensors that require custom installation, limiting scalability and deployment. In this work, we introduce RAGE, a novel real-time
estimator that simultaneously infers the vehicle velocity, slip angles of the tires and the lateral forces that act on them, using only standard sensors, such as IMUs and RADARs, which are commonly available on most of modern autonomous platforms. We validate our approach through both high-fidelity simulations and real-world experiments conducted on the EAV-24 autonomous race car, demonstrating the accuracy and effectiveness of our method in estimating the vehicle lateral dynamics.
Read the full abstract ▾
All authors Davide Malvezzi, Nicola Musiu, Eugenio Mascaro, Francesco Iacovacci, Marko Bertogna
Robotics
An Asynchronous Two-Speed Kalman Filter for Real-Time UUV Cooperative Navigation Under Acoustic Delays
Shuyue Li, Miguel López-Benítez, Eng Gee Lim et al. (+5 more)
In Global
Navigation Satellite System (GNSS)-denied
underwater environments, individual unmanned
underwater vehicles (UUVs) suffer from unbounded dead-reckoning drift, making collaborative
navigation (CN) crucial for accurate state estimation. However, the severe communication delay inherent in
underwater acoustic channels poses serious challenges to real-time state estimation. Traditional filters, such as Extended Kalman Filters (EKFs) or Unscented Kalman Filters (UKFs), usually block the main control loop while waiting for delayed data, or effectively discard Out-of-Sequence Measurements (OOSMs), resulting in serious drift. To address this, we propose an Asynchronous Two-Speed Kalman Filter (TSKF) enhanced by a novel projection mechanism, which we term Variational History
Distillation (VHD). The proposed architecture decouples the estimation process into two parallel threads: a fast-rate thread that utilizes Gaussian Process (GP) compensated dead reckoning to guarantee high-frequency real-time control, and a slow-rate thread dedicated to processing asynchronously delayed collaborative information. By introducing a Finite-Length Circular State Buffer (FLCSB), the
algorithm applies delayed measurements to their corresponding historical states, and utilizes a VHD-based projection to fast-forward the correction to the current time without computationally heavy recalculations. Simulation results demonstrate that the proposed TSKF maintains a trajectory error comparable to computationally intensive batch-optimization methods under severe delays (up to 30\,s). Executing in sub-millisecond time, it significantly outperforms standard EKF/UKF. The results demonstrate an effective control, communication, and computing (3C) co-design that significantly enhances the resilience of autonomous marine automation systems.
Read the full abstract ▾
All authors Shuyue Li, Miguel López-Benítez, Eng Gee Lim, Fei Ma, Qian Dong, Mengze Cao, Limin Yu, Xiaohui Qin
Robotics · Cross-listed from cs.CV
STRNet: Visual Navigation with Spatio-Temporal Representation through Dynamic Graph Aggregation
Hao Ren, Zetong Bi, Yiming Zeng et al. (+3 more)
Visual
navigation requires the
robot to reach a specified goal such as an image, based on a sequence of first-person visual observations. While recent learning-based approaches have made significant progress, they often focus on improving policy heads or decision strategies while relying on simplistic feature encoders and temporal pooling to represent visual input. This leads to the loss of fine-grained spatial and temporal structure, ultimately limiting accurate action prediction and progress estimation. In this paper, we propose a unified spatio-temporal representation framework that enhances visual encoding for robotic
navigation. Our approach extracts features from both image sequences and goal observations, and fuses them using the designed spatio-temporal fusion module. This module performs spatial graph
reasoning within each frame and models temporal dynamics using a hybrid temporal shift module combined with multi-resolution difference-aware convolution. Experimental results demonstrate that our approach consistently improves
navigation performance and offers a generalizable visual backbone for goal-conditioned control. Code is available at \href{https://github.com/hren20/STRNet}{https://github.com/hren20/STRNet}.
Read the full abstract ▾
All authors Hao Ren, Zetong Bi, Yiming Zeng, Zhaoliang Wan, Lu Qi, Hui Cheng
Robotics
Full 3D Radiation Pattern Approximation in Multi-Rotor UAVs
Martin Zoula, Daniel Bonilla Licea, Jan Faigl et al. (+2 more)
In-flight outdoor approximation of full 3D Radiation Patterns (RPs) in non-holonomic multi-rotor Uncrewed Aerial Vehicles (
UAVs) enables efficient communication-aware robotic missions. Our method identifies the Spherical Harmonics series coefficients, decoupling the observed signal power gains into respective RPs. The proposed calibration trajectory provides training and testing samples of signal strength at an obstacle-free altitude. Simulation verifies the proposed novel approximation of the RPs, and real-world validation achieves an RMS prediction error under 5dB.
Read the full abstract ▾
All authors Martin Zoula, Daniel Bonilla Licea, Jan Faigl, Václav Navrátil, Martin Saska
Robotics
Goal-Conditioned Neural ODEs with Guaranteed Safety and Stability for Learning-Based All-Pairs Motion Planning
Dechuan Liu, Ruigang Wang and Ian R. Manchester
This paper presents a learning-based approach for all-pairs motion planning, where the initial and goal states are allowed to be arbitrary points in a safe set. We construct smooth goal-conditioned neural ordinary differential equations (neural ODEs) via bi-Lipschitz diffeomorphisms. Theoretical results show that the proposed model can provide guarantees of global exponential stability and safety (safe set forward invariance) regardless of goal location. Moreover, explicit bounds on convergence rate, tracking error, and vector field magnitude are established. Our approach admits a tractable learning implementation using bi-Lipschitz neural networks and can incorporate demonstration data. We illustrate the effectiveness of the proposed method on a 2D corridor
navigation task.
Read the full abstract ▾
Robotics
MFE: A Multimodal Hand Exoskeleton with Interactive Force, Pressure and Thermo-haptic Feedback
Ziyuan Tang, Yitian Guo and Chenxi Xiao
Recent advancements in virtual reality and robotic teleoperation have greatly increased the variety of haptic information that must be conveyed to users. While existing haptic devices typically provide unimodal feedback to enhance situational awareness, a gap remains in their ability to deliver rich,
multimodal sensory feedback encompassing force, pressure, and thermal sensations. To address this limitation, we present the
Multimodal Feedback Exoskeleton (MFE), a hand exoskeleton designed to deliver hybrid haptic feedback. The MFE features 20 degrees of freedom for capturing hand pose. For force feedback, it employs an active mechanism capable of generating 3.5-8.1 N of pushing and pulling forces at the fingers' resting pose, enabling realistic interaction with deformable objects. The fingertips are equipped with flat actuators based on the electro-osmotic principle, providing pressure and vibration stimuli and achieving up to 2.47 kPa of contact pressure to render tactile sensations. For thermal feedback, the MFE integrates thermoelectric heat pumps capable of rendering temperatures from 10 to 55 degrees Celsius. We validated the MFE by integrating it into a robotic teleoperation system using the X-Arm 6 and Inspire Hand manipulator. In user studies, participants successfully recognized and manipulated deformable objects and differentiated remote objects with varying temperatures. These results demonstrate that the MFE enhances situational awareness, as well as the usability and transparency of robotic teleoperation systems.
Read the full abstract ▾
Robotics
Learning Structured Robot Policies from Vision-Language Models via Synthetic Neuro-Symbolic Supervision
Alessandro Adami, Tommaso Tubaldo, Marco Todescato et al. (+2 more)
Vision-
Language Models (VLMs) have recently demonstrated strong capabilities in mapping
multimodal observations to
robot behaviors. However, most current approaches rely on end-to-end visuomotor policies that remain opaque and difficult to analyze, limiting their use in real-world robotic applications. In contrast, classical robotic systems often rely on structured policy representations that provide interpretability, modularity, and reactive execution. This work investigates how foundation models can be specialized to generate structured
robot policies grounded in
multimodal perception, bridging high-dimensional learning and symbolic control. We propose a neuro-symbolic approach in which a VLM synthesizes executable Behavior Tree policies from visual observations, natural
language instructions, and structured system specifications. To enable scalable supervision without manual annotation, we introduce an automated pipeline that generates a synthetic
multimodal dataset of domain-randomized scenes paired with instruction-policy examples produced by a foundation model. By decoupling structured task decomposition under constrained symbolic grammars from hardware-specific motor control, we demonstrate that a 12B-parameter model can learn structured spatial-symbolic mappings required for executable BT synthesis, solely through in-silico supervision. Real-world physical experiments on two heterogeneous robotic manipulators confirm that these structurally constrained policies achieve zero-shot transfer to real-world environments. The results emphasize that the data bottleneck in robotic planning can be bypassed by procedurally synthesizing high-fidelity, neuro-symbolic training data.
Read the full abstract ▾
All authors Alessandro Adami, Tommaso Tubaldo, Marco Todescato, Ruggero Carli, Pietro Falco
Robotics
QuadAgent: A Responsive Agent System for Vision-Language Guided Quadrotor Agile Flight
Ao Zhuang, Feng Yu, Tianbao Zhang et al. (+2 more)
We present QuadAgent, a training-free
agent system for agile quadrotor flight guided by
vision-
language inputs. Unlike prior end-to-end or serial
agent approaches, QuadAgent decouples high-level
reasoning from low-level control using an asynchronous multi-
agent architecture: Foreground Workflow
Agents handle active tasks and user commands, while Background
Agents perform look-ahead
reasoning. The system maintains scene
memory via the Impression Graph, a lightweight topological map built from sparse keyframes, and ensures safe flight with a
vision-based obstacle avoidance network. Simulation results show that QuadAgent outperforms baseline methods in efficiency and responsiveness. Real-world experiments demonstrate that it can interpret complex instructions, reason about its surroundings, and navigate cluttered indoor spaces at speeds up to 5 m/s.
Read the full abstract ▾
All authors Ao Zhuang, Feng Yu, Tianbao Zhang, Linzuo Zhang, Danping Zou
Robotics
Vision-Based End-to-End Learning for UAV Traversal of Irregular Gaps via Differentiable Simulation
Linzuo Zhang, Yu Hu, Feng Yu et al. (+3 more)
-
Navigation through narrow and irregular gaps is an essential skill in autonomous drones for applications such as inspection, search-and-rescue, and disaster response. However, traditional planning and control methods rely on explicit gap extraction and measurement, while recent end-to-end approaches often assume regularly shaped gaps, leading to poor generalization and limited practicality. In this work, we present a fully
vision-based, end-to-end framework that maps depth images directly to control commands, enabling drones to traverse complex gaps within unseen environments. Operating in the Special Euclidean group SE(3), where position and orientation are tightly coupled, the framework leverages differentiable simulation, a Stop-Gradient operator, and a Bimodal Initialization Distribution to achieve stable traversal through consecutive gaps. Two auxiliary prediction modules-a gap-crossing success classifier and a traversability predictor-further enhance continuous
navigation and safety. Extensive simulation and real-world experiments demonstrate the approach's effectiveness, generalization capability, and practical robustness.
Read the full abstract ▾
All authors Linzuo Zhang, Yu Hu, Feng Yu, Yang Deng, Wenxian Yu, Danping Zou
Robotics
OMNI-PoseX: A Fast Vision Model for 6D Object Pose Estimation in Embodied Tasks
Michael Zhang, Wei Ying, Fangwen Chen et al. (+2 more)
Accurate 6D object pose estimation is a fundamental capability for
embodied agents, yet remains highly challenging in open-world environments. Many existing methods often rely on closed-set assumptions or geometry-agnostic regression schemes, limiting their generalization, stability, and real-time applicability in robotic systems. We present OMNI-PoseX, a
vision foundation model that introduces a novel network architecture unifying open-vocabulary perception with an SO(3)-aware reflected flow matching pose predictor. The architecture decouples object-level understanding from geometry-consistent rotation inference, and employs a lightweight multi-modal fusion strategy that conditions rotation-sensitive
geometric features on compact semantic
embeddings, enabling efficient and stable 6D pose estimation. To enhance robustness and generalization, the model is trained on large-scale 6D pose
datasets, leveraging broad object diversity, viewpoint variation, and scene complexity to build a scalable open-world pose backbone. Comprehensive evaluations across
benchmark pose estimation, ablation studies, zero-shot generalization, and system-level robotic grasping integration demonstrate the effectiveness of OMNI-PoseX. The OMNI-PoseX achieves SOTA pose accuracy and real-time efficiency, while delivering geometrically consistent predictions that enable reliable grasping of diverse, previously unseen objects.
Read the full abstract ▾
All authors Michael Zhang, Wei Ying, Fangwen Chen, Shifeng Bai, Hanwen Kang
Robotics
Geometrically-Constrained Radar-Inertial Odometry via Continuous Point-Pose Uncertainty Modeling
Wooseong Yang, Dongjae Lee, Minwoo Jung et al. (+1 more)
Radar odometry is crucial for robust localization in challenging environments; however, the sparsity of reliable returns and distinctive noise characteristics impede its performance. This paper introduces geometrically-constrained radar-inertial odometry and mapping that jointly consolidates point and pose uncertainty. We employ the continuous trajectory model to estimate the pose uncertainty at any arbitrary timestamp by propagating uncertainties of the control points. These pose uncertainties are continuously integrated with heteroscedastic measurement uncertainty during point projection, thereby enabling dynamic evaluation of observation confidence and adaptive down-weighting of uninformative radar points. By leveraging quantified uncertainties in radar mapping, we construct a high-fidelity map that improves odometry accuracy under imprecise radar measurements. Moreover, we reveal the effectiveness of explicit geometrical constraints in radar-inertial odometry when incorporated with the proposed uncertainty-aware mapping framework. Extensive experiments on diverse real-world
datasets demonstrate the superiority of our method, yielding substantial performance improvements in both accuracy and efficiency compared to existing baselines. We release our open-source implementation at https://github.com/wooseongY/GeoRIO.
Read the full abstract ▾
All authors Wooseong Yang, Dongjae Lee, Minwoo Jung, Ayoung Kim
Robotics
Learning Locomotion on Complex Terrain for Quadrupedal Robots with Foot Position Maps and Stability Rewards
Matthew Hwang, Yubin Liu, Ryo Hakoda et al. (+1 more)
Quadrupedal locomotion over complex terrain has been a long-standing research topic in robotics. While recent reinforcement learning-based locomotion methods improve generalizability and foot-placement precision, they rely on implicit inference of foot positions from joint angles, lacking the explicit precision and stability guarantees of optimization-based approaches. To address this, we introduce a foot position map integrated into the heightmap, and a dynamic locomotion-stability reward within an
attention-based framework to achieve locomotion on complex terrain. We validate our method extensively on terrains seen during training as well as out-of-domain (OOD) terrains. Our results demonstrate that the proposed method enables precise and stable movement, resulting in improved locomotion success rates on both in-domain and OOD terrains.
Read the full abstract ▾
All authors Matthew Hwang, Yubin Liu, Ryo Hakoda, Takeshi Oishi
Robotics
V2X-QA: A Comprehensive Reasoning Dataset and Benchmark for Multimodal Large Language Models in Autonomous Driving Across Ego, Infrastructure, and Cooperative Views
Junwei You, Pei Li, Zhuoyu Jiang et al. (+7 more)
Multimodal large
language models (MLLMs) have shown strong potential for autonomous driving, yet existing
benchmarks remain largely ego-centric and therefore cannot systematically assess model performance in infrastructure-centric and cooperative driving conditions. In this work, we introduce V2X-QA, a real-world
dataset and
benchmark for evaluating MLLMs across vehicle-side, infrastructure-side, and cooperative viewpoints. V2X-QA is built around a view-decoupled evaluation protocol that enables controlled comparison under vehicle-only, infrastructure-only, and cooperative driving conditions within a unified multiple-choice question answering (MCQA) framework. The
benchmark is organized into a twelve-task taxonomy spanning perception, prediction, and
reasoning and planning, and is constructed through expert-verified MCQA annotation to enable fine-grained diagnosis of viewpoint-dependent capabilities.
Benchmark results across ten representative state-of-the-art proprietary and open-source models show that viewpoint accessibility substantially affects performance, and infrastructure-side
reasoning supports meaningful macroscopic traffic understanding. Results also indicate that cooperative
reasoning remains challenging since it requires cross-view
alignment and evidence integration rather than simply additional visual input. To address these challenges, we introduce V2X-MoE, a
benchmark-aligned baseline with explicit view routing and viewpoint-specific LoRA experts. The strong performance of V2X-MoE further suggests that explicit viewpoint specialization is a promising direction for multi-view
reasoning in autonomous driving. Overall, V2X-QA provides a foundation for studying multi-perspective
reasoning, reliability, and cooperative physical intelligence in connected autonomous driving. The
dataset and V2X-MoE resources are publicly available at: https://github.com/junwei0001/V2X-QA.
Read the full abstract ▾
All authors Junwei You, Pei Li, Zhuoyu Jiang, Weizhe Tang, Zilin Huang, Rui Gan, Jiaxi Liu, Yan Zhao, Sikai Chen, Bin Ran
Robotics
A Rapid Instrument Exchange System for Humanoid Robots in Minimally Invasive Surgery
Bingcong Zhang, Yihang Lyv, Lianbo Ma et al. (+6 more)
Humanoid robot technologies have demonstrated immense potential for minimally invasive surgery (MIS). Unlike dedicated multi-arm
surgical platforms, the inherent dual-arm configuration of
humanoid robots necessitates an efficient instrument exchange capability to perform complex procedures, mimicking the natural workflow where surgeons manually switch instruments. To address this, this paper proposes an immersive teleoperated rapid instrument exchange system. The system utilizes a low-latency mechanism based on single-axis compliant docking and environmental constraint release. Integrated with real-time first-person view (FPV) perception via a head-mounted display (HMD), this framework significantly reduces operational complexity and cognitive load during the docking process. Comparative evaluations between experts and novices demonstrate high operational robustness and a rapidly converging learning curve; novice performance in instrument attachment and detachment improved substantially after brief training. While long-distance spatial
alignment still presents challenges in time cost and collaborative stability, this study successfully validates the technical feasibility of
humanoid robots executing stable instrument exchanges within constrained clinical environments.
Read the full abstract ▾
All authors Bingcong Zhang, Yihang Lyv, Lianbo Ma, Yushi He, Pengfei Wei, Xingchi Liu, Jinhua Li, Jianchang Zhao, Lizhi Pan
Robotics
ALIVE-LIO: Degeneracy-Aware Learning of Inertial Velocity for Enhancing ESKF-Based LiDAR-Inertial Odometry
Seongjun Kim, Daehan Lee, Junwoo Hong et al. (+3 more)
Odometry estimation using light detection and ranging (
LiDAR) and an inertial measurement unit (IMU), known as
LiDAR-inertial odometry (LIO), often suffers from performance degradation in degenerate environments, such as long corridors or single-wall scenarios with narrow field-of-view
LiDAR. To address this limitation, we propose ALIVE-LIO, a degeneracy-aware
LiDAR-inertial odometry framework that explicitly enhances state estimation in degenerate directions. The key contribution of ALIVE-LIO is the strategic integration of a deep neural network into a classical error-state Kalman filter (ESKF) to compensate for the loss of
LiDAR observability. Specifically, ALIVE-LIO employs a neural network to predict the body-frame velocity and selectively fuses this prediction into the ESKF only when degeneracy is detected, providing effective state updates along degenerate directions. This design enables ALIVE-LIO to utilize the probabilistic structure and consistency of the ESKF while benefiting from learning-based motion estimation. The proposed method was evaluated on publicly available
datasets exhibiting degeneracy, as well as on our own collected data. Experimental results demonstrate that ALIVE-LIO substantially reduces pose drift in degenerate environments, yielding the top-ranked result in 22 out of 32 sequences. Our code is available at https://github.com/cocel-postech/alive-lio.
Read the full abstract ▾
All authors Seongjun Kim, Daehan Lee, Junwoo Hong, Sanghyun Park, Hyunyoung Jo, Soohee Han
Robotics · Cross-listed from cs.CV
VBGS-SLAM: Variational Bayesian Gaussian Splatting Simultaneous Localization and Mapping
Yuhan Zhu, Yanyu Zhang, Jie Xu et al. (+1 more)
3D Gaussian Splatting (3DGS) has shown promising results for 3D scene modeling using mixtures of Gaussians, yet its existing simultaneous localization and mapping (SLAM) variants typically rely on direct, deterministic pose optimization against the splat map, making them sensitive to initialization and susceptible to catastrophic forgetting as map evolves. We propose Variational
Bayesian Gaussian Splatting SLAM (VBGS-SLAM), a novel framework that couples the splat map refinement and camera pose tracking in a generative probabilistic form. By leveraging conjugate properties of multivariate Gaussians and variational inference, our method admits efficient closed-form updates and explicitly maintains posterior uncertainty over both poses and scene parameters. This uncertainty-aware method mitigates drift and enhances robustness in challenging conditions, while preserving the efficiency and rendering quality of existing 3DGS. Our experiments demonstrate superior tracking performance and robustness in long sequence prediction, alongside efficient, high-quality novel view synthesis across diverse synthetic and real-world scenes.
Read the full abstract ▾
All authors Yuhan Zhu, Yanyu Zhang, Jie Xu, Wei Ren
Robotics
Differentiable SpaTiaL: Symbolic Learning and Reasoning with Geometric Temporal Logic for Manipulation Tasks
Licheng Luo, Kaier Liang, Cristian-Ioan Vasile et al. (+1 more)
Executing complex manipulation in cluttered environments requires satisfying coupled
geometric and temporal constraints. Although Spatio-Temporal Logic (SpaTiaL) offers a principled specification framework, its use in gradient-based optimization is limited by non-differentiable
geometric operations. Existing differentiable temporal logics focus on the
robot's internal state and neglect interactive object-environment relations, while spatial logic approaches that capture such interactions rely on discrete geometry engines that break the computational graph and preclude exact gradient propagation. To overcome this limitation, we propose Differentiable SpaTiaL, a fully tensorized toolbox that constructs smooth, autograd-compatible
geometric primitives directly over polygonal sets. To the best of our knowledge, this is the first end-to-end differentiable symbolic spatio-temporal logic toolbox. By analytically deriving differentiable relaxations of key spatial predicates–including signed distance, intersection, containment, and directional relations–we enable an end-to-end differentiable mapping from high-level semantic specifications to low-level
geometric configurations, without invoking external discrete solvers. This fully differentiable formulation unlocks two core capabilities: (i) massively parallel trajectory optimization under rigorous spatio-temporal constraints, and (ii) direct learning of spatial logic parameters from demonstrations via backpropagation. Experimental results validate the effectiveness and scalability of the proposed framework.
Read the full abstract ▾
All authors Licheng Luo, Kaier Liang, Cristian-Ioan Vasile, Mingyu Cai
Robotics
Elastomeric Strain Limitation for Design of Soft Pneumatic Actuators
Gregory M. Campbell
Modern
robots embody power and precision control. Yet, as
robots undertake tasks that apply forces on humans, this power brings risk of injury. Soft robotic actuators use deformation to produce smooth, continuous motions and conform to delicate objects while imparting forces capable of safely pushing humans. This thesis presents strategies for the design, modeling, and strain-based control of human-safe elastomeric soft pneumatic actuators (SPA) for force generation, focusing on
embodied mechanical response to simple pressure inputs. We investigate electroadhesive (EA) strain limiters for variable shape generation, rapid force application, and targeted inflation trajectories. We attach EA clutches to a concentrically strain-limited elastomeric membrane to alter the inflation trajectory and rapidly reorient the inflated shape. We expand the capabilities of EA for soft
robots by encasing them in elastomeric sheaths and varying their activation in real time, demonstrating applications in variable trajectory inflation under identical pressure sweeps. We then address the problem of trajectory control in the presence of external forces by modeling the pressure-trajectory relationship for a concentrically strain-limited class of silicone actuators. We validate theoretical models based on material properties and energy minimization using active learning and automated testing. We apply our ensemble of neural networks for inverse membrane design, specifying quasi-static mass lift trajectories from a simple pressure sweep. Finally, we demonstrate the power of multiple pressure-linked actuators in a proof-of-concept mannequin leg lift.
Read the full abstract ▾