Robotics
How IMU Drift Influences Multi-Radar Inertial Odometry for Ground Robots in Subterranean Terrains
Moumita Mukherjee, Magnus Norén, Anton Koval et al. (+2 more)
Reliable radar inertial odometry (RIO) requires mitigating IMU bias drift, a challenge that intensifies in subterranean environments due to extreme temperatures and
gravity-induced accelerations. Cost-effective IMUs such as the Pixhawk, when paired with FMCW TI IWR6843AOP EVM radars, suffer from drift-induced degradation compounded by sparse, noisy, and flickering radar returns, making fusion less stable than
LiDAR-based odometry. Yet,
LiDAR fails under smoke,
dust, and aerosols, whereas FMCW radars remain compact, lightweight, cost-effective, and robust in these situations. To address these challenges, we propose a two-stage MRIO framework that combines an IMU bias
estimator for resilient localization and mapping in GPS-denied subterranean environments affected by smoke. Radar-based ego-velocity estimation is formulated through a least-squares approach and incorporated into an EKF for online IMU bias correction; the corrected IMU accelerations are fused with heterogeneous measurements from multiple radars and an IMU to refine odometry. The proposed framework further supports radar-only mapping by exploiting the
robot's estimated translational and rotational displacements. In subterranean field trials, MRIO delivers robust localization and mapping, outperforming EKF-RIO. It maintains accuracy across cost-efficient FMCW radar setups and different IMUs, showing resilience with Pixhawk and higher-grade units such as VectorNav. The implementation will be provided as an open-source resource to the community (code available at https://github.com/LTU-RAI/MRIO
Read the full abstract ▾
All authors Moumita Mukherjee, Magnus Norén, Anton Koval, Avijit Banerjee, George Nikolakopoulos
Robotics
Humanoid Robots as First Assistants in Endoscopic Surgery
Sue Min Cho, Jan Emily Mangulabnan, Han Zhang et al. (+7 more)
Humanoid robots have become a focal point of technological ambition, with claims of
surgical capability within years in mainstream discourse. These projections are aspirational yet lack empirical grounding. To date, no
humanoid has assisted a surgeon through an actual procedure, let alone performed one. The work described here breaks this new ground. Here we report a proof of concept in which a teleoperated Unitree G1 provided endoscopic visualization while an attending otolaryngologist performed a cadaveric sphenoidectomy. The procedure was completed successfully, with stable visualization maintained throughout. Teleoperation allowed assessment of whether the
humanoid form factor could meet the physical demands of
surgical assistance in terms of sustenance and precision; the cognitive demands were satisfied – for now – by the operator. Post-procedure analysis identified engineering targets for clinical translation, alongside near-term opportunities such as autonomous diagnostic scoping. This work establishes form-factor feasibility for
humanoid surgical assistance while identifying challenges for continued development.
Read the full abstract ▾
All authors Sue Min Cho, Jan Emily Mangulabnan, Han Zhang, Zhekai Mao, Yufan He, Pengfei Guo, Daguang Xu, Gregory Hager, Masaru Ishii, Mathias Unberath
Robotics
Robust Skills, Brittle Grounding: Diagnosing Restricted Generalization in Vision-Language Action Policies via Multi-Object Picking
David Emukpere, Romain Deffayet and Jean-Michel Renders
Vision-
language action (VLA) policies often report strong manipulation
benchmark performance with relatively few demonstrations, but it remains unclear whether this reflects robust
language-to-object grounding or reliance on object–location correlations that do not transfer beyond the training distribution. We present a controlled multi-object picking study that progressively increases object placement variability up to full workspace randomization and evaluates held-out object–location pairings that break familiar associations without increasing spatial difficulty. Across these stress tests and data scaling, we find that for representative VLA policies, including SmolVLA and $\pi_{0.5}$, execution of the manipulation primitive remains substantially more reliable than instruction-conditioned task success in harder regimes, suggesting that manipulation skill acquisition is decoupled from instruction following. We recommend augmenting manipulation
benchmarks with task ladders and decomposed metrics that separately measure primitive execution and instruction-conditioned success to better diagnose instruction-grounded generalization.
Read the full abstract ▾
Robotics
Online World Modeling Enables Real-World Inverse Reinforcement Learning from Observation
Tyler Han, Bat Nemekhbold, Siyang Shen et al. (+6 more)
Current methods in
robot learning are fundamentally bottlenecked by one or more of: hand-designed rewards, simulation modeling, or action supervision (e.g. teleoperation) each requiring significant domain expertise, engineering effort, and
robot-operator labor. Towards reducing these bottlenecks, this work pursues observational learning via Inverse Reinforcement Learning from Observation (IRLfO) in which only access to task observations (e.g.
video) is assumed. Due to the challenging setting and limitations of RL methods, IRLfO has thus far remained impractical for real-world
robot learning. Here, we present the first IRL method to learn visual manipulation in the real world from scratch, and the first real-world demonstration of positive online transfer across visual manipulation tasks from scratch. In under 40 minutes, MPAIL2 learns pick-and-place from scratch to 82%, where RL and BC with equal interaction and demonstration budgets reach only 0% and 12% despite their reward and action supervision. Interactive project page with training
videos: https://uwrobotlearning.github.io/mpail2/
Read the full abstract ▾
All authors Tyler Han, Bat Nemekhbold, Siyang Shen, Rohan Baijal, Richard Ebock, Harine Ravichandiran, Sanghun Jung, Kevin Huang, Byron Boots
Robotics
Geometry-based pneumatic actuators for soft robotics
Rui Chen, Daniele Leonardis, Domenico Chiaradia et al. (+1 more)
Soft pneumatic actuators enable safe human-machine interaction with lightweight and powerful applied parts. On the other side, they suffer design limitations as regards complex actuation patterns, including minimum bending radii, multi-states capabilities and structural stability. We present geometry-based pneumatic actuators (GPAs), a design and implementation approach that introduces constraint layers with configurable CNC heat-sealed chambers. The approach achieves predictable deformation, near-zero bending radii, multi-states actuation, and enables customizable and repeatable complex actuated geometries. Mathematical modeling reveals predictable linear angle transformations and validates nonlinear torque-angle relationships across diverse configurations. We demonstrate versatility of the GPAs approach through three applications: a 49 g wrist exoskeleton reducing muscle activity by up to 51%, a 30.8 g haptic interface delivering 8 N force feedback with fast response, and a 208 g bipedal
robot achieving multi-gait locomotion. GPAs establish a configurable platform for next-generation wearable robotics, haptic systems, and soft locomotion devices.
Read the full abstract ▾
All authors Rui Chen, Daniele Leonardis, Domenico Chiaradia, Antonio Frisoli
Robotics
Curriculum Reinforcement Learning for Quadrotor Racing with Random Obstacles
Fangyu Sun, Fanxing Li, Yu Hu et al. (+4 more)
Autonomous drone racing has attracted increasing interest as a research topic for exploring the limits of agile flight. However, existing studies primarily focus on obstacle-free racetracks, while the perception and dynamic challenges introduced by obstacles remain underexplored, often resulting in low success rates and limited robustness in real-world flight. To this end, we propose a novel
vision-based curriculum reinforcement learning framework for training a robust controller capable of addressing unseen obstacles in drone racing. We combine multi-stage cu rriculum learning, domain randomization, and a multi-scene updating strategy to address the conflicting challenges of obstacle avoidance and gate traversal. Our end-to-end control policy is implemented as a single network, allowing high-speed flight of quadrotors in environments with variable obstacles. Both hardware-in-the-loop and real-world experiments demonstrate that our method achieves faster lap times and higher success rates than existing approaches, effectively advancing drone racing in obstacle-rich environments. The
video and code are available at: https://github.com/SJTU-ViSYS-team/CRL-Drone-Racing.
Read the full abstract ▾
All authors Fangyu Sun, Fanxing Li, Yu Hu, Linzuo Zhang, Yueqian Liu, Wenxian Yu, Danping Zou
Robotics
Autonomous Inspection of Power Line Insulators with UAV on an Unmapped Transmission Tower
Václav Riss, Vít Krátký, Robert Pěnička et al. (+1 more)
This paper introduces an online inspection
algorithm that enables an autonomous
UAV to fly around a transmission tower and obtain detailed inspection images without a prior map of the tower. Our
algorithm relies on camera-
LiDAR sensor fusion for online detection and localization of
insulators. In particular, the
algorithm is based on
insulator detection using a convolutional neural network, projection of
LiDAR points onto the image, and filtering them using the bounding boxes. The detection pipeline is coupled with several proposed
insulator localization methods based on DBSCAN, RANSAC, and PCA
algorithms. The performance of the proposed online inspection
algorithm and camera-
LiDAR sensor fusion pipeline is demonstrated through simulation and real-world flights. In simulation, we showed that our single-flight inspection strategy can save up to 24 % of total inspection time, compared to the two-flight strategy of scanning the tower and afterwards visiting the inspection waypoints in the optimal way. In a real-world experiment, the best performing proposed method achieves a mean horizontal and vertical localization error for the
insulator of 0.16 +- 0.08 m and 0.16 +- 0.11 m, respectively. Compared to the most relevant approach, the proposed method achieves more than an order of magnitude lower variance in horizontal
insulator localization error.
Read the full abstract ▾
All authors Václav Riss, Vít Krátký, Robert Pěnička, Martin Saska
Robotics
Learning Robust Control Policies for Inverted Pose on Miniature Blimp Robots
Yuanlin Yang, Lin Hong and Fumin Zhang
The ability to achieve and maintain inverted poses is essential for unlocking the full agility of miniature blimp
robots (MBRs). However, developing reliable inverted control strategies for MBRs remains challenging due to their complex and underactuated dynamics. To address this challenge, we propose a novel framework that enables robust control policy learning for inverted pose on MBRs. The proposed framework consists of three core stages. First, a high-fidelity three-dimensional (3D) simulation environment is constructed and calibrated using real-world MBR motion data. Second, a robust inverted control policy is trained in simulation using a modified Twin Delayed Deep Deterministic Policy Gradient (TD3)
algorithm combined with a domain randomization strategy. Third, a mapping layer is designed to bridge the sim-to-real gap and facilitate real-world deployment of the learned policy. Comprehensive evaluations in the simulation environment demonstrate that the learned policy achieves a higher success rate compared to the energy-shaping controller. Furthermore, experimental results confirm that the learned policy with a mapping layer enables an MBR to achieve and maintain a fully inverted pose in real-world settings.
Read the full abstract ▾
Robotics
Haoxuan Xu, Tianfu Li, Wenbo Chen et al. (+4 more)
Vision-
Language Navigation (VLN)
agents often struggle with long-horizon
reasoning in unseen environments, particularly when facing ambiguous, coarse-grained instructions. While recent advances use knowledge graph to enhance
reasoning, the potential of
multimodal event knowledge inspired by human episodic
memory remains underexplored. In this work, we propose an event-centric knowledge enhancement strategy for automated process knowledge mining and feature fusion to solve coarse-grained instruction and long-horizon
reasoning in VLN task. First, we construct YE-KG, the first large-scale
multimodal spatiotemporal knowledge graph, with over 86k nodes and 83k edges, derived from real-world indoor
videos. By leveraging
multimodal large
language models (i.e., LLaVa, GPT4), we extract unstructured
video streams into structured semantic-action-effect events to serve as explicit episodic
memory. Second, we introduce STE-VLN, which integrates the above graph into VLN models via a Coarse-to-Fine Hierarchical
Retrieval mechanism. This allows
agents to retrieve causal event sequences and dynamically fuse them with
egocentric visual observations. Experiments on REVERIE, R2R, and R2R-CE
benchmarks demonstrate the efficiency of our event-centric strategy, outperforming state-of-the-art approaches across diverse action spaces. Our data and code are available on the project website https://sites.google.com/view/y-event-kg/.
Read the full abstract ▾
All authors Haoxuan Xu, Tianfu Li, Wenbo Chen, Yi Liu, Xingxing Zuo, Yaoxian Song, Haoang Li
Robotics
Learning to Build: Autonomous Robotic Assembly of Stable Structures Without Predefined Plans
Jingwen Wang, Johannes Kirschner, Paul Rolland et al. (+2 more)
This paper presents a novel autonomous robotic assembly framework for constructing stable structures without relying on predefined architectural blueprints. Instead of following fixed plans, construction tasks are defined through targets and obstacles, allowing the system to adapt more flexibly to environmental uncertainty and variations during the building process. A reinforcement learning (RL) policy, trained using deep Q-learning with successor features, serves as the decision-making component. As a proof of concept, we evaluate the approach on a
benchmark of 15 2D robotic assembly tasks of discrete block construction. Experiments using a real-world closed-loop robotic setup demonstrate the feasibility of the method and its ability to handle construction noise. The results suggest that our framework offers a promising direction for more adaptable and robust robotic construction in real-world environments.
Read the full abstract ▾
All authors Jingwen Wang, Johannes Kirschner, Paul Rolland, Luis Salamanca, Stefana Parascho
Robotics
Teleoperated Omni-directional Dual Arm Mobile Manipulation Robotic System with Shared Control for Retail Store
Rolif Lima, Somdeb Saha, Nijil George et al. (+5 more)
The swiftly expanding retail sector is increasingly adopting autonomous mobile
robots empowered by artificial intelligence and machine learning
algorithms to gain an edge in the competitive
market. However, these autonomous
robots encounter challenges in adapting to the dynamic nature of retail products, often struggling to operate autonomously in novel situations. In this study, we introduce an omni-directional dual-arm mobile
robot specifically tailored for use in retail environments. Additionally, we propose a tele-operation method that enables shared control between the
robot and a human operator. This approach utilizes a Virtual Reality (VR) motion capture system to capture the operator's commands, which are then transmitted to the
robot located remotely in a retail setting. Furthermore, the
robot is equipped with heterogeneous grippers on both manipulators, facilitating the handling of a wide range of items. We validate the efficacy of the proposed system through testing in a mockup of retail environment, demonstrating its ability to manipulate various commonly encountered retail items using both single and dual-arm coordinated manipulation techniques.
Read the full abstract ▾
All authors Rolif Lima, Somdeb Saha, Nijil George, Vismay Vakharia, Shubham Parab, Sahil Gaonkar, Vighnesh Vatsal, Kaushik Das
Robotics
ABPolicy: Asynchronous B-Spline Flow Policy for Real-Time and Smooth Robotic Manipulation
Fan Yang, Peiguang Jing, Kaihua Qu et al. (+2 more)
Robotic manipulation requires policies that are smooth and responsive to evolving observations. However, synchronous inference in the raw action space introduces several challenges, including intra-chunk jitter, inter-chunk discontinuities, and stop-and-go execution. These issues undermine a policy's smoothness and its responsiveness to environmental changes. We propose ABPolicy, an asynchronous flow-matching policy that operates in a B-spline control-point action space. First, the B-spline representation ensures intra-chunk smoothness. Second, we introduce bidirectional action prediction coupled with refitting optimization to enforce inter-chunk continuity. Finally, by leveraging asynchronous inference, ABPolicy delivers real-time, continuous updates. We evaluate ABPolicy across seven tasks encompassing both static settings and dynamic settings with moving objects. Empirical results indicate that ABPolicy reduces trajectory jerk, leading to smoother motion and improved performance. Project website: https://teee000.github.io/ABPolicy/.
Read the full abstract ▾
All authors Fan Yang, Peiguang Jing, Kaihua Qu, Ningyuan Zhao, Yuting Su
Robotics
TSC: Topology-Conditioned Stackelberg Coordination for Multi-Agent Reinforcement Learning in Interactive Driving
Xiaotong Zhang, Gang Xiong, Yuanjing Wang et al. (+3 more)
Safe and efficient autonomous driving in dense traffic is fundamentally a decentralized multi-
agent coordination problem, where interactions at conflict points such as merging and weaving must be resolved reliably under partial observability. With only local and incomplete cues, interaction patterns can change rapidly, often causing unstable behaviors such as oscillatory yielding or unsafe commitments. Existing multi-
agent reinforcement learning (MARL) approaches either adopt synchronous decision-making, which exacerbate non-stationarity, or depend on centralized sequencing mechanisms that scale poorly as traffic density increases. To address these limitations, we propose
Topology-conditioned Stackelberg Coordination (TSC), a learning framework for decentralized interactive driving under communication-free execution, which extracts a time-varying directed priority graph from braid-inspired weaving relations between trajectories, thereby defining local leader-follower dependencies without constructing a global order of play. Conditioned on this graph, TSC endogenously factorizes dense interactions into graph-local Stackelberg subgames and, under centralized training and decentralized execution (CTDE), learns a sequential coordination policy that anticipates leaders via action prediction and trains followers through action-conditioned value learning to approximate local best responses, improving training stability and safety in dense traffic. Experiments across four dense traffic scenarios show that TSC achieves superior performance over representative MARL baselines across key metrics, most notably reducing collisions while maintaining competitive traffic efficiency and control smoothness.
Read the full abstract ▾
All authors Xiaotong Zhang, Gang Xiong, Yuanjing Wang, Siyu Teng, Alois Knoll, Long Chen
Robotics · Cross-listed from cs.CV
Bowen Yang, Zishuo Li, Yang Sun et al. (+15 more)
Embodied foundation models require large-scale, high-quality real-world interaction data for pre-training and scaling. However, existing data collection methods suffer from high infrastructure costs, complex hardware dependencies, and limited interaction scope, making scalable expansion challenging. In fact, humans themselves are ideal physically
embodied agents. Therefore, obtaining
egocentric real-world interaction data from globally distributed "human
agents" offers advantages of low cost and sustainability. To this end, we propose the Always-on
Egocentric (AoE) data collection system, which aims to simplify hardware dependencies by leveraging humans themselves and their smartphones, enabling low-cost, highly efficient, and scene-agnostic real-world interaction data collection to address the challenge of data scarcity. Specifically, we first employ an ergonomic neck-mounted smartphone holder to enable low-barrier, large-scale
egocentric data collection through a cloud-edge collaborative architecture. Second, we develop a cross-platform mobile APP that leverages on-device compute for real-time processing, while the cloud hosts automated labeling and filtering pipelines that transform raw
videos into high-quality training data. Finally, the AoE system supports distributed Ego
video data collection by anyone, anytime, and anywhere. We evaluate AoE on data preprocessing quality and downstream tasks, demonstrating that high-quality
egocentric data significantly boosts real-world generalization.
Read the full abstract ▾
All authors Bowen Yang, Zishuo Li, Yang Sun, Changtao Miao, Yifan Yang, Man Luo, Xiaotong Yan, Feng Jiang, Jinchuan Shi, Yankai Fu, Ning Chen, Junkai Zhao, Pengwei Wang, Guocai Yao, Shanghang Zhang, Hao Chen, Zhe Li, Kai Zhu
Robotics · Cross-listed from cs.CV
Altitude-Adaptive Vision-Only Geo-Localization for UAVs in GPS-Denied Environments
Xingyu Shao, Mengfan He, Liangzheng Sun et al. (+2 more)
Matching downward-looking unmanned aerial vehicle (
UAV) images to georeferenced satellite or aerial map tiles supports local earth observation, map-based interpretation, and coarse geo-initialization when direct positioning signals are degraded or unavailable. A major difficulty in this cross-platform matching task is the scale mismatch caused by large variations in
UAV altitude. To address this problem, we propose an altitude-adaptive aerial visual place recognition framework. The method first estimates relative altitude from a single downward-looking image by transforming the input into the frequency domain and formulating altitude estimation as a regression-as-classification problem. The estimated altitude is then used to crop the query image to a canonical scale, after which a classification-then-
retrieval visual place recognition module performs coarse map localization. To improve
retrieval robustness under varying image quality, we further introduce a quality-adaptive margin classifier and refine the final location by weighted coordinate estimation over the top retrieved candidates. Experiments on two synthetic
datasets and two real-flight
datasets show that the relative altitude estimation module improves downstream
retrieval performance under substantial altitude changes. With our visual place recognition module, altitude adaptation improves average R@1 and R@5 by 41.50 and 56.83 percentage points, respectively, compared with using the same
retrieval pipeline without altitude normalization, and the main
retrieval pipeline runs at 13.3 frames/s on the workstation used for evaluation. These results indicate that relative altitude estimation provides a useful scale prior for cross-altitude
UAV-to-map image matching using georeferenced remote sensing imagery.
Read the full abstract ▾
All authors Xingyu Shao, Mengfan He, Liangzheng Sun, Chunyu Li, Ziyang Meng
Robotics
Hybrid Offline-Online Reinforcement Learning for Sensorless, High-Precision Force Regulation in Surgical Robotic Grasping
Edoardo Fazzari, Omar Mohamed, Khalfan Hableel et al. (+2 more)
Precise grasp force regulation in tendon-driven
surgical instruments is fundamentally limited by nonlinear coupling between motor dynamics, transmission compliance, friction, and distal mechanics. Existing solutions typically rely on distal force sensing or analytical compensation, increasing hardware complexity or degrading performance under dynamic motion. We present a sensorless control framework that combines physics-consistent modeling and hybrid reinforcement learning to achieve high-precision distal force regulation in a proximally actuated
surgical end-effector. We develop a first-principles digital twin of the da Vinci Xi grasping mechanism that captures coupled electrical, transmission, and jaw dynamics within a unified differential-algebraic formulation. To safely learn control policies in this stiff and highly nonlinear system, we introduce a three-stage pipeline:(i)a receding-horizon CMA-ES oracle that generates dynamically feasible expert trajectories,(ii)fully offline policy learning via Implicit Q-Learning to ensure stable initialization without unsafe exploration, and (iii)online refinement using TD3 for adaptation to on-policy dynamics. The resulting policy directly maps proximal measurements to motor voltages and requires no distal sensing. In simulation, the controller maintains grasp force within 1% of the desired reference during multi-harmonic jaw motion. Hardware experiments demonstrate average force errors below 4% across diverse trajectories, validating sim-to-real transfer. The learned policy contains approximately 71k param and executes at kH rates, enabling real-time deployment. These results demonstrate that high-fidelity modeling combined with structured offline-online RL can recover precise distal force behavior without additional sensing, offering a scalable and mechanically compatible solution for
surgical robotic manipulation.
Read the full abstract ▾
All authors Edoardo Fazzari, Omar Mohamed, Khalfan Hableel, Hamdan Alhadhrami, Cesare Stefanini
Robotics
OmniXtreme: Breaking the Generality Barrier in High-Dynamic Humanoid Control
Yunshen Wang, Shaohang Zhu, Peiyuan Zhi et al. (+7 more)
High-fidelity motion tracking serves as the ultimate litmus test for generalizable, human-level motor skills. However, current policies often hit a "generality barrier": as motion libraries scale in diversity, tracking fidelity inevitably collapses - especially for real-world deployment of high-dynamic motions. We identify this failure as the result of two compounding factors: the learning bottleneck in scaling multi-motion optimization and the physical executability constraints that arise in real-world actuation. To overcome these challenges, we introduce OmniXtreme, a scalable framework that decouples general motor skill learning from sim-to-real physical skill refinement. Our approach uses a flow-matching policy with high-capacity architectures to scale representation capacity without interference-intensive multi-motion RL optimization, followed by an actuation-aware refinement phase that ensures robust performance on physical hardware. Extensive experiments demonstrate that OmniXtreme maintains high-fidelity tracking across diverse, high-difficulty
datasets. On real
robots, the unified policy successfully executes multiple extreme motions, effectively breaking the long-standing fidelity-scalability trade-off in high-dynamic
humanoid control.
Read the full abstract ▾
All authors Yunshen Wang, Shaohang Zhu, Peiyuan Zhi, Yuhan Li, Jiaxin Li, Yong-Lu Li, Yuchen Xiao, Xingxing Wang, Baoxiong Jia, Siyuan Huang
Robotics
OmniTrack: General Motion Tracking via Physics-Consistent Reference
Yuhan Li, Peiyuan Zhi, Yunshen Wang et al. (+6 more)
Learning motion tracking from rich human motion data is a foundational task for achieving general control in
humanoid robots, enabling them to perform diverse behaviors. However, discrepancies in morphology and dynamics between humans and
robots, combined with data noise, introduce physically infeasible artifacts in reference motions, such as floating and penetration. During both training and execution, these artifacts create a conflict between following inaccurate reference motions and maintaining the
robot's stability, hindering the development of a generalizable motion tracking policy. To address these challenges, we introduce OmniTrack, a general tracking framework that explicitly decouples physical feasibility from general motion tracking. In the first stage, a privileged generalist policy generates physically plausible motions that strictly adhere to the
robot's dynamics via trajectory rollout in simulation. In the second stage, the general control policy is trained to track these physically feasible motions, ensuring stable and coherent control transfer to the real
robot. Experiments show that OmniTrack improves tracking accuracy and demonstrates strong generalization to unseen motions. In real-world tests, OmniTrack achieves hour-long, consistent, and stable tracking, including complex acrobatic motions such as flips and cartwheels. Additionally, we show that OmniTrack supports human-style stable and dynamic online teleoperation, highlighting its robustness and adaptability to varying user inputs.
Read the full abstract ▾
All authors Yuhan Li, Peiyuan Zhi, Yunshen Wang, Tengyu Liu, Sixu Yan, Wenyu Liu, Xinggang Wang, Baoxiong Jia, Siyuan Huang
Robotics
Acceleration-Based Control of Fixed-Wing UAVs for Guidance Applications
Jixiang Wang, Siyuan Yang, Ziyi Wu et al. (+5 more)
Acceleration-commanded guidance laws (e.g., proportional
navigation) are attractive for high-level decision making, but their direct deployment on fixed-wing
UAVs is challenging because accelerations are not directly actuated and must be realized through attitude and thrust under flight-envelope constraints. This paper presents an acceleration-level outer-loop control framework that converts commanded tangential and normal accelerations into executable body-rate and normalized thrust commands compatible with mainstream autopilots (e.g., PX4/APM). For the normal channel, we derive an engineering mapping from the desired normal acceleration to roll- and pitch-rate commands that regulate the direction and magnitude of the lift vector under small-angle assumptions. For the tangential channel, we introduce an energy-based formulation inspired by total energy control and identify an empirical thrust-energy acceleration relationship directly from flight data, avoiding explicit propulsion modeling or thrust bench calibration. We further discuss priority handling between normal and tangential accelerations under saturation and non-level maneuvers. Extensive real-flight experiments on a VTOL fixed-wing platform demonstrate accurate acceleration tracking and enable practical implementation of proportional
navigation using only body-rate and normalized thrust interfaces.
Read the full abstract ▾
All authors Jixiang Wang, Siyuan Yang, Ziyi Wu, Siqi Wei, Ashay Wakode, Agata Barcis, Hung Nguyen, Shaoming He
Robotics
StemVLA:An Open-Source Vision-Language-Action Model with Future 3D Spatial Geometry Knowledge and 4D Historical Representation
Jiasong Xiao, Yutao She, Kai Li et al. (+2 more)
Vision-
language-action (VLA) models integrate visual observations and
language instructions to predict
robot actions, demonstrating promising generalization in manipulation tasks. However, most existing approaches primarily rely on direct mappings from 2D visual inputs to action sequences, without explicitly modeling the underlying 3D spatial structure or temporal world dynamics. Such representations may limit spatial
reasoning and long-horizon decision-making in dynamic environments. To address this limitation, we propose StemVLA, a novel framework that explicitly incorporates both future-oriented 3D spatial knowledge and historical 4D spatiotemporal representations into action prediction. First, instead of relying solely on observed images, StemVLA forecasts structured 3D future spatial-
geometric world knowledge, enabling the model to anticipate upcoming scene geometry and object configurations. Second, to capture temporal consistency and motion dynamics, we feed historical image frames into a pretrained
video-geometry
transformer backbone to extract implicit 3D world representations, and further aggregate them across time using a temporal
attention module, termed VideoFormer [20], forming a unified 4D historical spatiotemporal representation. By jointly modeling 2D observations, predicted 3D future structure, and aggregated 4D temporal dynamics, StemVLA enables more comprehensive world understanding for
robot manipulation. Extensive experiments in simulation demonstrate that Stem-VLA achieves an average accuracy of 92.0% across the LIBERO subsets, and 86.0% on the long-horizon LIBERO-Long subset.
Read the full abstract ▾
All authors Jiasong Xiao, Yutao She, Kai Li, Yuyang Sha, Ziang Cheng
Robotics
SAGE-LLM: Towards Safe and Generalizable LLM Controller with Fuzzy-CBF Verification and Graph-Structured Knowledge Retrieval for UAV Decision
Wenzhe Zhao, Yang Zhao, Ganchao Liu et al. (+4 more)
In
UAV dynamic decision, complex and variable hazardous factors pose severe challenges to the generalization capability of
algorithms. Despite offering semantic understanding and scene generalization, Large
Language Models (LLM) lack domain-specific
UAV control knowledge and formal safety assurances, restricting their direct applicability. To bridge this gap, this paper proposes a train-free two-layer decision architecture based on LLMs, integrating high-level safety planning with low-level precise control. The framework introduces three key contributions: 1) A fuzzy Control Barrier Function verification mechanism for semantically-augmented actions, providing provable safety certification for LLM outputs. 2) A
star-hierarchical graph-based
retrieval-augmented generation system, enabling efficient, elastic, and interpretable scene adaptation. 3) Systematic experimental validation in pursuit-evasion scenarios with unknown obstacles and emergent threats, demonstrating that our SAGE-LLM maintains performance while significantly enhancing safety and generalization without online training. The proposed framework demonstrates strong extensibility, suggesting its potential for generalization to broader
embodied intelligence systems and safety-critical control domains.
Read the full abstract ▾
All authors Wenzhe Zhao, Yang Zhao, Ganchao Liu, Zhiyu Jiang, Dandan Ma, Zihao Li, Xuelong Li
Robotics
A Reliable Indoor Navigation System for Humans Using AR-based Technique
Vijay U.Rathod, Manav S.Sharma, Shambhavi Verma et al. (+3 more)
Reliable
navigation systems are not available indoors, such as in campuses and small areas. Users must depend on confusing, time-consuming static signage or floor maps. In this paper, an AR-based technique has been applied to campus and small-site
navigation, where Vuforia Area Target is used for environment modeling. AI
navigation's NavMesh component is used for
navigation purposes, and the A*
algorithm is used within this component for shortest path calculation. Compared to Dijkstra's
algorithm, it can reach a solution about two to three times faster for smaller search spaces. In many cases, Dijkstra's
algorithm has difficulty performing well in high-complexity environments where
memory usage grows and processing times increase. Compared to older approaches such as GPS, real-time processing and AR overlays can be combined to provide intuitive directions for users while dynamically updating the path in response to environmental changes. Experimental results indicate significantly improved
navigation accuracy, better user experience, and greater efficiency compared to traditional methods. These results show that AR technology integrated with existing pathfinding
algorithms is feasible and scalable, making it a user-friendly solution for indoor
navigation. Although highly effective in limited and defined indoor spaces, further optimization of NavMesh is required for large or highly dynamic environments.
Read the full abstract ▾
All authors Vijay U.Rathod, Manav S.Sharma, Shambhavi Verma, Aadi Joshi, Sachin Aage, Sujal Shahane
Robotics
Interpretable Multimodal Gesture Recognition for Drone and Mobile Robot Teleoperation via Log-Likelihood Ratio Fusion
Seungyeol Baek, Jaspreet Singh, Lala Shakti Swarup Ray et al. (+3 more)
Human operators are still frequently exposed to hazardous environments such as disaster zones and industrial facilities, where intuitive and reliable teleoperation of mobile
robots and Unmanned Aerial Vehicles (
UAVs) is essential. In this context, hands-free teleoperation enhances operator mobility and situational awareness, thereby improving safety in hazardous environments. While
vision-based gesture recognition has been explored as one method for hands-free teleoperation, its performance often deteriorates under occlusions, lighting variations, and cluttered backgrounds, limiting its applicability in real-world operations. To overcome these limitations, we propose a
multimodal gesture recognition framework that integrates inertial data (accelerometer, gyroscope, and orientation) from Apple Watches on both wrists with capacitive sensing signals from custom gloves. We design a late fusion strategy based on the log-likelihood ratio (LLR), which not only enhances recognition performance but also provides interpretability by quantifying modality-specific contributions. To support this research, we introduce a new
dataset of 20 distinct gestures inspired by aircraft marshalling signals, comprising synchronized RGB
video, IMU, and capacitive sensor data. Experimental results demonstrate that our framework achieves performance comparable to a state-of-the-art
vision-based baseline while significantly reducing computational cost, model size, and training time, making it well suited for real-time
robot control. We therefore underscore the potential of sensor-based
multimodal fusion as a robust and interpretable solution for gesture-driven mobile
robot and drone teleoperation.
Read the full abstract ▾
All authors Seungyeol Baek, Jaspreet Singh, Lala Shakti Swarup Ray, Hymalai Bello, Paul Lukowicz, Sungho Suh
Robotics
Physics-Embedded Neural ODEs for Learning Antagonistic Pneumatic Artificial Muscle Dynamics
Xinyao Wang and Jonathan Realmuto
Pneumatic artificial muscles (PAMs) enable compliant actuation for soft wearable, assistive, and interactive
robots. When arranged antagonistically, PAMs can provide variable impedance through co-contraction but exhibit coupled, nonlinear, and hysteretic dynamics that challenge modeling and control. This paper presents a hybrid neural ordinary differential equation (Neural ODE) framework that embeds physical structure into a learned model of antagonistic PAM dynamics. The formulation combines parametric joint mechanics and pneumatic state dynamics with a neural network force component that captures antagonistic coupling and rate-dependent hysteresis. \rev{The forward model was trained on 29 selected co-contraction conditions and predicted joint motion and chamber pressures over 196 held-out conditions with a mean R$^2$ of 0.88.} An inverse formulation, derived from the learned dynamics, computes pressure commands offline for desired motion and stiffness profiles, tracked in closed loop during execution. Experimental validation demonstrates reliable stiffness control across 126-176 N/mm and consistent impedance behavior across operating velocities, in contrast to a static model, which shows degraded stiffness consistency at higher velocities.
Read the full abstract ▾
Robotics
SpikingTac: A Miniaturized Neuromorphic Visuotactile Sensor for High-Precision Dynamic Tactile Imprint Tracking
Tianyu Jiang, Chaofan Zhang, Shaolin Zhang et al. (+2 more)
High-speed event-driven tactile sensors are essential for achieving human-like dynamic manipulation, yet their integration is often limited by the bulkiness of standard event cameras. This paper presents SpikingTac, a miniaturized, highly integrated neuromorphic tactile sensor featuring a custom standalone event camera module, achieved with a total material cost of less than \$150. We construct a global dynamic state map coupled with an unsupervised denoising network to enable precise tracking at a 1000~Hz perception rate and 350~Hz tracking frequency. Addressing the viscoelastic hysteresis of silicone elastomers, we propose a hysteresis-aware incremental update law with a spatial gain damping mechanism. Experimental results demonstrate exceptional zero-point stability, achieving a 100% return-to-origin success rate with a minimal mean bias of 0.8039 pixels, even under extreme torsional deformations. In dynamic tasks, SpikingTac limits the obstacle-avoidance overshoot to 6.2~mm, representing a 5-fold performance improvement over conventional frame-based sensors. Furthermore, the sensor achieves sub-millimeter
geometric accuracy, with Root Mean Square Error (RMSE) of 0.0952~mm in localization and 0.0452~mm in radius measurement.
Read the full abstract ▾
All authors Tianyu Jiang, Chaofan Zhang, Shaolin Zhang, Shaowei Cui, Shuo Wang
Robotics
FAVLA: A Force-Adaptive Fast-Slow VLA model for Contact-Rich Robotic Manipulation
Yao Li, Peiyuan Tang, Wuyang Zhang et al. (+7 more)
Force/torque feedback can substantially improve
Vision-
Language-Action (VLA) models on contact-rich manipulation, but most existing approaches fuse all modalities at a single operating frequency. This design ignores the mismatched sampling rates of real
robot sensors, forcing downsampling of the high-frequency contact cues needed for reactive correction. Combined with common VLM-action-expert (AE) pipelines that execute action chunks largely open loop between expensive VLM updates, unified-frequency fusion often yields delayed responses to impacts, stick-slip, and force spikes. We propose FAVLA, a force-adaptive fast-slow VLA that decouples slow perception planning from fast contact-aware control. FAVLA runs a slow VLM at a fixed low frequency to encode modalities to produce latent representations and to predict near-future force variation. A fast AE then executes at a variable high frequency, conditioning on the latest force sequence data to generate reactive actions. We further introduce a force adapter that injects high-frequency force features into multiple AE layers, and adaptively schedules the AE's execution frequency based on the VLM's predicted force variation. Extensive experiments on contact-rich tasks demonstrate that FAVLA significantly outperforms baselines, achieving superior reactivity and success rates, especially with a smaller contact force during manipulation.
Read the full abstract ▾
All authors Yao Li, Peiyuan Tang, Wuyang Zhang, Chengyang Zhu, Yifan Duan, Weikai Shi, Xiaodong Zhang, Zijiang Yang, Jianmin Ji, Yanyong Zhang
Robotics
MicroPush: A Lightweight Simulator for Sim-to-Real Microrobotic Pushing and Assembly
Yanda Yang, Zion Lee and Sambeeta Das
Developing learning-based autonomy for microrobotic manipulation is difficult because physical trials are slow, variable, and poorly suited to large-scale training. We present MicroPush, a lightweight simulation and benchmarking environment for pushing and assembly with
magnetic rolling microrobots. The simulator combines overdamped planar dynamics with calibrated rolling actuation, reduced-order contact handling, short-range interaction corrections, and optional background flow, while providing common interfaces for planning, model-based control, reinforcement learning (RL), interactive visualization, and headless evaluation. In no-render tests, MicroPush remains substantially faster than real time, reaching 37.0x real time with 100 passive particles. We further evaluate the simulator through zero-shot sim-to-real policy transfer. A Residual RL-MPC policy is trained entirely in MicroPush, frozen after simulation training, and deployed on the physical system without real-world policy fine-tuning. A 20 {\mu}m
magnetic microrobot pushes a similarly sized silica particle along a figure-eight trajectory with 1.91 {\mu}m median and 4.65 {\mu}m 95th-percentile path error, and the same autonomy stack performs sequential line, triangle, and six-particle hexagonal assembly. These results show that a deliberately reduced-order simulator can provide an efficient training and development substrate for sim-to-real microrobotic autonomy.
Read the full abstract ▾
Robotics
KEEP: A KV-Cache-Centric Memory Management System for Efficient Embodied Planning
Zebin Yang, Tong Xie, Baotong Lu et al. (+3 more)
Memory-augmented Large
Language Models (LLMs) have demonstrated remarkable capability for complex and long-horizon
embodied planning. By keeping track of past experiences and environmental states,
memory enables LLMs to maintain a global view, thereby avoiding repetitive exploration. However, existing approaches often store the
memory as raw text, leading to excessively long
prompts and high prefill latency. While it is possible to store and reuse the KV caches, the efficiency benefits are greatly undermined due to frequent KV cache updates. In this paper, we propose KEEP, a KV-cache-centric
memory management system for efficient
embodied planning. KEEP features 3 key innovations: (1) a Static-Dynamic
Memory Construction
algorithm that reduces KV cache recomputation by mixed-granularity
memory group; (2) a Multi-hop
Memory Re-computation
algorithm that dynamically identifies important cross-
attention among different
memory groups and reconstructs
memory interactions iteratively; (3) a Layer-balanced
Memory Loading that eliminates unbalanced KV cache loading and cross-
attention computation across different layers. Extensive experimental results have demonstrated that KEEP achieves 2.68x speedup with negligible accuracy loss compared with text-based
memory methods on ALFRED
dataset. Compared with the KV re-computation method CacheBlend (EuroSys'25), KEEP shows 4.13% success rate improvement and 1.90x time-to-first-
token (TTFT) reduction. Our code is available on https://github.com/PKU-SEC-Lab/KEEP_
Embodied_
Memory.
Read the full abstract ▾
All authors Zebin Yang, Tong Xie, Baotong Lu, Shaoshan Liu, Bo Yu, Meng Li
Robotics
VCA: Vision-Click-Action Framework for Precise Manipulation of Segmented Objects in Target Ambiguous Environments
Donggeon Kim, Seungwon Jan, Hyeonjun Park et al. (+1 more)
The reliance on
language in
Vision-
Language-Action (VLA) models introduces ambiguity, cognitive overhead, and difficulties in precise object identification and sequential task execution, particularly in environments with multiple visually similar objects. To address these limitations, we propose
Vision-Click-Action (VCA), a framework that replaces verbose textual commands with direct, click-based visual interaction using pretrained
segmentation models. By allowing operators to specify target objects clearly through visual selection in the
robot's 2D camera view, VCA reduces interpretation errors, lowers cognitive load, and provides a practical and scalable alternative to
language-driven interfaces for real-world robotic manipulation. Experimental results validate that the proposed VCA framework achieves effective instance-level manipulation of specified target objects. Experiment
videos are available at https://robrosinc.github.io/vca/.
Read the full abstract ▾
All authors Donggeon Kim, Seungwon Jan, Hyeonjun Park, Daegyu Lim
Robotics
Tilt-X: Enabling Compliant Aerial Manipulation through a Tiltable-Extensible Continuum Manipulator
Anuraj Uthayasooriyan, Krishna Manaswi Digumarti, Jack Breward et al. (+3 more)
Aerial manipulators extend the reach and manipulation capabilities of uncrewed multirotor aerial vehicles for inspection, agriculture, sampling, and delivery. Continuum arm aerial manipulation systems offer lightweight, dexterous, and compliant interaction opportunities. Existing designs allow manipulation only below the
UAV which restricts their deployability in multiple directions and through clutter. They are also sensitive to propeller downwash. Addressing these limitations, we present Tilt-X, a continuum arm aerial manipulator that integrates a tilting mechanism, a telescopic stage, and a cable-driven continuum section. We present its design and kinematic model and validate it through flight demonstrations. Tilt-X enables a volumetric workspace with up to 75 mm extension and planar orientations between 0$^\circ$ to 90$^\circ$. Experiments comparing end effector pose with and without downwash quantitatively measure its accuracy, providing critical evidence to guide the design and control of reliable aerial manipulators. Results show stabilisation of end effector pose as the manipulator extends out of the propeller influence zone.
Read the full abstract ▾
All authors Anuraj Uthayasooriyan, Krishna Manaswi Digumarti, Jack Breward, Fernando Vanegas, Julian Galvez-Serna, Felipe Gonzalez