Robotics
GNIO: Gated Neural Inertial Odometry
Dapeng Feng, Yizhen Yin, Zhiqiang Chen et al. (+2 more)
Inertial
navigation using low-cost MEMS sensors is plagued by rapid drift due to sensor noise and bias instability. While recent data-driven approaches have made significant strides, they often struggle with micro-drifts during stationarity and mode fusion during complex motion transitions due to their reliance on fixed-window regression. In this work, we introduce Gated Neural Inertial Odometry (GNIO), a novel learning-based framework that explicitly models motion validity and context. We propose two key architectural innovations: \ding{182} a learnable Motion Bank that queries a global dictionary of motion patterns to provide semantic context beyond the local receptive field, and \ding{183} a Gated Prediction Head that decomposes displacement into magnitude and direction. This gating mechanism acts as a soft, differentiable Zero-Velocity Update (ZUPT), dynamically suppressing sensor noise during stationary periods while scaling predictions during dynamic motion. Extensive experiments across four public
benchmarks demonstrate that GNIO significantly reduces position drift compared to state-of-the-art CNN and
Transformer-based baselines. Notably, GNIO achieves a $60.21\%$ reduction in trajectory error on the OxIOD
dataset and exhibits superior generalization in challenging scenarios involving frequent stops and irregular motion speeds.
Read the full abstract ▾
All authors Dapeng Feng, Yizhen Yin, Zhiqiang Chen, Yuhua Qi, Hongbo Chen
Robotics · Cross-listed from eess.SY
Encirclement Guaranteed Finite-Time Capture against Unknown Evader Strategies
Dinesh Patra, Prajakta Surve, Ashish R. Hota et al. (+1 more)
We consider a pursuit-evasion scenario involving a group of pursuers and a single evader in a two-dimensional unbounded environment. The pursuers aim to capture the evader in finite time while ensuring the evader remains enclosed within the convex hull of their positions until capture, without knowledge of the evader's heading angle. Prior works have addressed the problem of encirclement and capture separately in different contexts. In this paper, we present a class of strategies for the pursuers that guarantee capture in finite time while maintaining encirclement, irrespective of the evader's strategy. Furthermore, we derive an upper bound on the time to capture. Numerical results highlight the effectiveness of the proposed framework against a range of evader strategies.
Read the full abstract ▾
All authors Dinesh Patra, Prajakta Surve, Ashish R. Hota, Shaunak D. Bopardikar
Robotics
MoE-ACT: Scaling Multi-Task Bimanual Manipulation with Sparse Task-Conditioned Mixture-of-Experts Transformers
Kangjun Guo, Haichao Liu, Yanji Sun et al. (+3 more)
Developing a unified policy for multi-task robotic manipulation remains challenging due to policy degradation from task interference and negative transfer. In this work, we propose Mixture-of-Experts-Enhanced Action Chunking
Transformer (MoE-ACT), a parameter-efficient multi-task visuomotor framework tailored for bimanual manipulation. MoE-ACT incorporates sparse MoE layers into the ACT encoder, dynamically routing image
tokens to selected experts based on task context, visual observations, and proprioceptive states. Furthermore, the framework incorporates task-conditioned Feature-wise Linear Modulation (FiLM) in the action decoder alongside multi-scale cross-
attention, ensuring precise task grounding and capturing fine-grained spatial cues. Extensive evaluations on the RoboTwin 2.0
benchmark across 16 challenging bimanual tasks demonstrate that MoE-ACT achieves an average success rate of 62.0%, outperforming standard ACT by 17.4 percentage points. Crucially, with only 195M activated parameters, MoE-ACT surpasses the 16-fold larger foundation model $\pi$0 (3.24B) by 5.1 percentage points, exhibiting substantial gains in parameter efficiency and deployment feasibility. Real-
robot dual-arm experiments consistently confirm its superior multi-task execution capabilities. Our open-source project page can be found at https://j3k7.github.io/MoE-ACT/.
Read the full abstract ▾
All authors Kangjun Guo, Haichao Liu, Yanji Sun, Ruhan Zhao, Jinni Zhou, Jun Ma
Robotics
HapticVLA: Contact-Rich Manipulation via Vision-Language-Action Model without Inference-Time Tactile Sensing
Konstantin Gubernatorov, Mikhail Sannikov, Ilya Mikhalchuk et al. (+7 more)
Tactile sensing is a crucial capability for
Vision-
Language-Action (VLA) architectures, as it enables dexterous and safe manipulation in contact-rich tasks. However, reliance on dedicated tactile hardware increases cost and reduces reproducibility across robotic platforms. We argue that tactile-aware manipulation can be learned offline and deployed without direct haptic feedback at inference. To this end, we present HapticVLA, which proceeds in two tightly coupled stages: Safety-Aware Reward-Weighted Flow Matching (SA-RWFM) and Tactile
Distillation (TD). SA-RWFM trains a flow-matching action expert that incorporates precomputed, safety-aware tactile rewards penalizing excessive grasping force and suboptimal grasping trajectories. TD further transfers this tactile-aware capability into a conventional VLA: we distill a compact tactile
token from the SA-RWFM teacher and train a student VLA to predict that
token from
vision and state modalities, enabling tactile-aware action generation at inference without requiring on-board tactile sensors. This design preserves contact-rich tactile-aware
reasoning within VLA while removing the need for on-board tactile sensors during deployment. On real-world experiments, HapticVLA achieves a mean success rate of 86.7%, consistently outperforming baseline VLAs - including versions provided with direct tactile feedback during inference.
Read the full abstract ▾
All authors Konstantin Gubernatorov, Mikhail Sannikov, Ilya Mikhalchuk, Egor Kuznetsov, Makar Artemov, Ogunwoye Faith Oluwatobi, Marcelino Fernando, Artem Asanov, Ziang Guo, Dzmitry Tsetserukou
Robotics
A Methodology for Dynamic Parameters Identification of 3-DOF Parallel Robots in Terms of Relevant Parameters
Miguel Díaz-Rodríguez, Vicente Mata, Angel Valera et al. (+1 more)
The identification of dynamic parameters in mechanical systems is important for improving model-based control as well as for performing realistic dynamic simulations. Generally, when identification techniques are applied only a subset of so-called base parameters can be identified. More even, some of these parameters cannot be identified properly given that they have a small contribution to the
robot dynamics and hence in the presence of noise in measurements and discrepancy in modeling, their quality of being identifiable decreases. For this reason, a strategy for dynamic parameter identification of fully parallel
robots in terms of a subset called relevant parameters is put forward. The objective of the proposed methodology is to start from a full dynamic model, then simplification concerning the geometry of each link and, the
symmetry due to legs of fully parallel
robots, are carried out. After that, the identification is done by Weighted Least Squares. Then, with statistical considerations the model is reduced until the physical feasibility conditions are met. The application of the propose strategy has been experimentally tested on two difierent configurations of actual 3-DOF parallel
robots. The response of the inverse and forward dynamics of the identified models agrees with experiments. In order to evaluate the forward dynamics response, an approach for obtaining the forward dynamics in terms of the relevant parameters is also proposed.
Read the full abstract ▾
All authors Miguel Díaz-Rodríguez, Vicente Mata, Angel Valera, Alvaro Page
Robotics
Coupled Particle Filters for Robust Affordance Estimation
Patrick Lowin, Vito Mengers and Oliver Brock
Robotic affordance estimation is challenging due to visual,
geometric, and semantic ambiguities in sensory input. We propose a method that disambiguates these signals using two coupled recursive
estimators for sub-aspects of affordances: graspable and movable regions. Each
estimator encodes property-specific regularities to reduce uncertainty, while their coupling enables bidirectional information exchange that focuses
attention on regions where both agree, i.e., affordances. Evaluated on a real-world
dataset, our method outperforms three recent affordance
estimators (Where2Act, Hands-as-Probes, and HRP) by 308%, 245%, and 257% in precision, and remains robust under challenging conditions such as low light or cluttered environments. Furthermore, our method achieves a 70% success rate in our real-world evaluation. These results demonstrate that coupling complementary
estimators yields precise, robust, and embodiment-appropriate affordance predictions.
Read the full abstract ▾
Robotics
NavGSim: High-Fidelity Gaussian Splatting Simulator for Large-Scale Navigation
Jiahang Liu, Yuanxing Duan, Jiazhao Zhang et al. (+4 more)
Simulating realistic environments for
robots is widely recognized as a critical challenge in
robot learning, particularly in terms of rendering and physical simulation. This challenge becomes even more pronounced in
navigation tasks, where trajectories often extend across multiple rooms or entire floors. In this work, we present NavGSim, a Gaussian Splatting-based simulator designed to generate high-fidelity, large-scale
navigation environments. Built upon a hierarchical 3D Gaussian Splatting framework, NavGSim enables photorealistic rendering in expansive scenes spanning hundreds of square meters. To simulate
navigation collisions, we introduce a Gaussian Splatting-based slice technique that directly extracts navigable areas from reconstructed Gaussians. Additionally, for ease of use, we provide comprehensive NavGSim APIs supporting multi-GPU development, including tools for custom scene reconstruction,
robot configuration, policy training, and evaluation. To evaluate NavGSim's effectiveness, we train a
Vision-
Language-Action (VLA) model using trajectories collected from NavGSim and assess its performance in both simulated and real-world environments. Our results demonstrate that NavGSim significantly enhances the VLA model's scene understanding, enabling the policy to handle diverse
navigation queries effectively.
Read the full abstract ▾
All authors Jiahang Liu, Yuanxing Duan, Jiazhao Zhang, Minghan Li, Shaoan Wang, Zhizheng Zhang, He Wang
Robotics
What Matters for Scalable and Robust Learning in End-to-End Driving Planners?
David Holtz, Niklas Hanselmann, Simon Doll et al. (+2 more)
End-to-end autonomous driving has gained significant
attention for its potential to learn robust behavior in interactive scenarios and scale with data. Popular architectures often build on separate modules for perception and planning connected through latent representations, such as bird's eye view feature grids, to maintain end-to-end differentiability. This paradigm emerged mostly on open-loop
datasets, with evaluation focusing not only on driving performance, but also intermediate perception tasks. Unfortunately, architectural advances that excel in open-loop often fail to translate to scalable learning of robust closed-loop driving. In this paper, we systematically re-examine the impact of common architectural patterns on closed-loop performance: (1) high-resolution perceptual representations, (2) disentangled trajectory representations, and (3) generative planning. Crucially, our analysis evaluates the combined impact of these patterns, revealing both unexpected limitations as well as underexplored synergies. Building on these insights, we introduce BevAD, a novel lightweight and highly scalable end-to-end driving architecture. BevAD achieves 72.7% success rate on the Bench2Drive
benchmark and demonstrates strong data-scaling behavior using pure imitation learning. Our code and models are publicly available here: https://dmholtz.github.io/bevad/
Read the full abstract ▾
All authors David Holtz, Niklas Hanselmann, Simon Doll, Marius Cordts, Bernt Schiele
Robotics
KiRAS: Keyframe Guided Self-Imitation for Robust and Adaptive Skill Learning in Quadruped Robots
Xiaoyi Wei, Peng Zhai, Jiaxin Tu et al. (+5 more)
With advances in reinforcement learning and imitation learning, quadruped
robots can acquire diverse skills within a single policy by imitating multiple skill-specific
datasets. However, the lack of
datasets on complex terrains limits the ability of such multi-skill policies to generalize effectively in unstructured environments. Inspired by animation, we adopt keyframes as minimal and universal skill representations, relaxing
dataset constraints and enabling the integration of terrain adaptability with skill diversity. We propose Keyframe Guided Self-Imitation for Robust and Adaptive Skill Learning (KiRAS), an end-to-end framework for acquiring and transitioning between diverse skill primitives on complex terrains. KiRAS first learns diverse skills on flat terrain through keyframe-guided self-imitation, eliminating the need for expert
datasets; then continues training the same policy network on rough terrains to enhance robustness. To eliminate catastrophic forgetting, a proficiency-based Skill Initialization Technique is introduced. Experiments on Solo-8 and Unitree Go1
robots show that KiRAS enables robust skill acquisition and smooth transitions across challenging terrains. This framework demonstrates its potential as a lightweight platform for multi-skill generation and
dataset collection. It further enables flexible skill transitions that enhance locomotion on challenging terrains.
Read the full abstract ▾
All authors Xiaoyi Wei, Peng Zhai, Jiaxin Tu, Yueqi Zhang, Yuqi Li, Zonghao Zhang, Hu Zhou, Lihua Zhang
Robotics
ForceVLA2: Unleashing Hybrid Force-Position Control with Force Awareness for Contact-Rich Manipulation
Yang Li, Zhaxizhuoma, Hongru Jiang et al. (+11 more)
Embodied intelligence for contact-rich manipulation has predominantly relied on position control, while explicit awareness and regulation of interaction forces remain under-explored, limiting stability, precision, and robustness in real-world tasks. We propose ForceVLA2, an end-to-end
vision-
language-action framework that equips
robots with hybrid force-position control and explicit force awareness. ForceVLA2 introduces force-based
prompts into the VLM expert to construct force-aware task concepts across stages, and employs a Cross-Scale Mixture-of-Experts (MoE) in the action expert to adaptively fuse these concepts with real-time interaction forces for closed-loop hybrid force-position regulation. To support learning and evaluation, we construct ForceVLA2-
Dataset, containing 1,000 trajectories over 5 contact-rich tasks, including wiping, pressing, and assembling, with multi-view images, task
prompts, proprioceptive state, and force signals. Extensive experiments show that ForceVLA2 substantially improves success rates and reliability in contact-rich manipulation, outperforming pi0 and pi0.5 by 48.0% and 35.0%, respectively, across the 5 tasks, and mitigating common failure modes such as arm overload and unstable contact, thereby actively advancing force-aware interactive physical intelligence in VLAs. The project page is available at https://sites.google.com/view/force-vla2/home.
Read the full abstract ▾
All authors Yang Li, Zhaxizhuoma, Hongru Jiang, Junjie Xia, Hongquan Zhang, Jinda Du, Yunsong Zhou, Jia Zeng, Ce Hao, Jieji Ren, Qiaojun Yu, Cewu Lu, Yu Qiao, Jiangmiao Pang
Robotics
Master Micro Residual Correction with Adaptive Tactile Fusion and Force-Mixed Control for Contact-Rich Manipulation
Xingting Li, Yifan Xie, Han Liu et al. (+4 more)
Robotic contact-rich and fine-grained manipulation remains a significant challenge due to complex interaction dynamics and the competing requirements of multi-timescale control. While current visual imitation learning methods excel at long-horizon planning, they often fail to perceive critical interaction cues like friction variations or incipient slip, and struggle to balance global task coherence with local reactive feedback. To address these challenges, we propose M2-ResiPolicy, a novel Master-Micro residual control architecture that synergizes high-level action guidance with low-level correction. The framework consists of a Master-Guidance Policy (MGP) operating at 10 Hz, which generates temporally consistent action chunks via a
diffusion-based backbone and employs a tactile-intensity-driven adaptive fusion mechanism to dynamically modulate perceptual weights between
vision and touch. Simultaneously, a high-frequency (60 Hz) Micro-Residual Corrector (MRC) utilizes a lightweight GRU to provide real-time action compensation based on TCP wrench feedback. This policy is further integrated with a force-mixed PBIC execution layer, effectively regulating contact forces to ensure interaction safety. Experiments across several demanding tasks including fragile object grasping and precision insertion, demonstrate that M2-ResiPolicy significantly outperforms standard
Diffusion Policy (DP) and state-of-the-art Reactive
Diffusion Policy (RDP), achieving a 93% damage-free success rate in chip grasping and superior force regulation stability.
Read the full abstract ▾
All authors Xingting Li, Yifan Xie, Han Liu, Wei Hou, Guangyu Chen, Shoujie Li, Wenbo Ding
Robotics
Confusion-Aware In-Context-Learning for Vision-Language Models in Robotic Manipulation
Yayun He, Zuheng Kang, Botao Zhao et al. (+3 more)
Vision-
language models (VLMs) have significantly improved the generalization capabilities of robotic manipulation. However, VLM-based systems often suffer from a lack of robustness, leading to unpredictable errors, particularly in scenarios involving confusable objects. Our preliminary analysis reveals that these failures are mainly caused by shortcut learning problem inherently in VLMs, limiting their ability to accurately distinguish between confusable features. To this end, we propose Confusion-Aware In-Context Learning (CAICL), a method that enhances VLM performance in confusable scenarios for robotic manipulation. The approach begins with confusion localization and analysis, identifying potential sources of confusion. This information is then used as a
prompt for the VLM to focus on features most likely to cause misidentification. Extensive experiments on the VIMA-Bench show that CAICL effectively addresses the shortcut learning issue, achieving a 85.5% success rate and showing good stability across tasks with different degrees of generalization.
Read the full abstract ▾
All authors Yayun He, Zuheng Kang, Botao Zhao, Zhouyin Wu, Junqing Peng, Jianzong Wang
Robotics
A Novel Camera-to-Robot Calibration Method for Vision-Based Floor Measurements
Jan Andre Rudolph, Dennis Haitz and Markus Ulrich
A novel hand-eye calibration method for ground-observing mobile
robots is proposed. While cameras on mobile
robots are common, they are rarely used for ground-observing measurement tasks.
Laser trackers are increasingly used in robotics for precise localization. A referencing plate is designed to combine the two measurement modalities of
laser-tracker 3D metrology and camera-based 2D imaging. It incorporates reflector nests for pose acquisition using a
laser tracker and a camera calibration target that is observed by the
robot-mounted camera. The procedure comprises estimating the plate pose, the plate-camera pose, and the
robot pose, followed by computing the
robot-camera transformation. Experiments indicate sub-millimeter repeatability.
Read the full abstract ▾
Robotics
BodyGuards: Escorting by Multiple Robots in Unknown Environment under Limited Communication
Zhuoli Tian, Yanze Bao and Meng Guo
Multi-
robot systems are increasingly deployed in high-risk missions such as reconnaissance, disaster response, and subterranean operations. Protecting a human operator while navigating unknown and
adversarial environments remains a critical challenge, especially when the communication among the operator and
robots is restricted. Unlike existing collaborative exploration methods that aim for complete coverage, this work focuses on task-oriented exploration to minimize the
navigation time of the operator to reach its goal while ensuring safety under
adversarial threats. A novel escorting framework BodyGuards, is proposed to explicitly integrate seamlessly collaborative exploration, inter-
robot-operator communication and escorting. The framework consists of three core components: (I) a dynamic movement strategy for the operator that maintains a local map with risk zones for proactive path planning; (II) a dual-mode robotic strategy combining frontier based exploration with optimized return events to balance exploration, threat detection, and intermittent communication; and (III) multi-
robot coordination protocols that jointly plan exploration and information sharing for efficient escorting. Extensive human-in-the-loop simulations and hardware experiments demonstrate that the method significantly reduces operator risk and mission time, outperforming baselines in
adversarial and constrained environments.
Read the full abstract ▾
Robotics
AeroGrab: A Unified Framework for Aerial Grasping in Cluttered Environments
Shivansh Pratap Singh, Naveen Sudheer Nair, Samaksh Ujjawal et al. (+4 more)
Reliable aerial grasping in cluttered environments remains challenging due to occlusions and collision risks. Existing aerial manipulation pipelines largely rely on centroid-based grasping and lack integration between the grasp pose generation models, active exploration, and
language-level task specification, resulting in the absence of a complete end-to-end system. In this work, we present an integrated pipeline for reliable aerial grasping in cluttered environments. Given a scene and a
language instruction, the system identifies the target object and actively explores it to gain better views of the object. During exploration, a grasp generation network predicts multiple 6-DoF grasp candidates for each view. Each candidate is evaluated using a collision-aware feasibility framework, and the overall best grasp is selected and executed using standard trajectory generation and control methods. Experiments in cluttered real-world scenarios demonstrate robust and reliable grasp execution, highlighting the effectiveness of combining active perception with feasibility-aware grasp selection for aerial manipulation.
Read the full abstract ▾
All authors Shivansh Pratap Singh, Naveen Sudheer Nair, Samaksh Ujjawal, Sarthak Mishra, Soham Patil, Rishabh Dev Yadav, Spandan Roy
Robotics
HALO:Closing Sim-to-Real Gap for Heavy-loaded Humanoid Agile Motion Skills via Differentiable Simulation
Xingyi Wang, Chenyun Zhang, Weiji Xie et al. (+4 more)
Humanoid robots deployed in real-world scenarios often need to carry unknown payloads, which introduce significant mismatch and degrade the effectiveness of simulation-to-reality reinforcement learning methods. To address this challenge, we propose a two-stage gradient-based system identification framework built on the differentiable simulator MuJoCo XLA. The first stage calibrates the nominal
robot model using real-world data to reduce intrinsic sim-to-real discrepancies, while the second stage further identifies the mass distribution of the unknown payload. By explicitly reducing structured model bias prior to policy training, our approach enables zero-shot transfer of reinforcement learning policies to hardware under heavy-load conditions. Extensive simulation and real-world experiments demonstrate more precise parameter identification, improved motion tracking accuracy, and substantially enhanced agility and robustness compared to existing baselines. Project Page: https://mwondering.github.io/
halo-
humanoid/
Read the full abstract ▾
All authors Xingyi Wang, Chenyun Zhang, Weiji Xie, Chao Yu, Wei Song, Chenjia Bai, Shiqiang Zhu
Robotics
Multi-Mode Pneumatic Artificial Muscles Driven by Hybrid Positive-Negative Pressure
Siyuan Feng, Ruoyu Feng and Shuguang Li
Artificial muscles embody human aspirations for engineering lifelike robotic movements. This paper introduces an architecture for Inflatable Fluid-Driven Origami-Inspired Artificial Muscles (IN-FOAMs). A typical IN-FOAM consists of an inflatable skeleton enclosed within an outer skin, which can be driven using a combination of positive and negative pressures (e.g., compressed air and
vacuum). IN-FOAMs are manufactured using low-cost heat-sealable sheet materials through heat-pressing and heat-sealing processes. Thus, they can be ultra-thin when not actuated, making them flexible, lightweight, and portable. The skeleton patterns are programmable, enabling a variety of motions, including contracting, bending, twisting, and rotating, based on specific skeleton designs. We conducted comprehensive experimental, theoretical, and numerical studies to investigate IN-FOAM's basic mechanical behavior and properties. The results show that IN-FOAM's output force and contraction can be tuned through multiple operation modes with the applied hybrid positive-negative pressure. Additionally, we propose multilayer skeleton structures to enhance the contraction ratio further, and we demonstrate a multi-channel skeleton approach that allows the integration of multiple motion modes into a single IN-FOAM. These findings indicate that IN-FOAMs hold great potential for future applications in flexible wearable devices and compact soft robotic systems.
Read the full abstract ▾
Robotics
AnolisVLA: Lightweight Vision-Language-Action Model with Deep State Space Models for Mobile Manipulation
Yusuke Takagi, Motonari Kambara, Daichi Yashima et al. (+3 more)
In this study, we address the problem of
language-guided robotic manipulation, where a
robot is required to manipulate a wide range of objects based on visual observations and natural
language instructions. This task is essential for service
robots that operate in human environments, and requires safety, efficiency, and task-level generality. Although
Vision-
Language-Action models (VLAs) have demonstrated strong performance for this task, their deployment in resource-constrained environments remains challenging because of the computational cost of standard
transformer backbones. To overcome this limitation, we propose AnolisVLA, a lightweight VLA that uses a deep state space model to process
multimodal sequences efficiently. The model uses lightweight, fast sequential state modeling to process visual and textual inputs. This design enables efficient trajectory generation. We evaluated the proposed method in simulation and on two real
robot platforms. Compared with SmolVLA at a comparable parameter scale, AnolisVLA achieved higher task success rates, faster inference, and comparable GPU
memory usage. In mobile manipulation, it outperformed the larger, 3-billion-parameter $\pi_{0.5}$ model by 21 percentage points in task success rate while achieving approximately three times faster inference.
Read the full abstract ▾
All authors Yusuke Takagi, Motonari Kambara, Daichi Yashima, Koki Seno, Kento Tokura, Komei Sugiura
Robotics
CycleRL: Sim-to-Real Deep Reinforcement Learning for Robust Autonomous Bicycle Control
Gelu Liu, Teng Wang, Zhijie Wu et al. (+3 more)
Autonomous bicycles offer a promising agile solution for urban mobility and last-mile logistics. However, conventional control strategies often struggle with underactuated nonlinear dynamics, suffering from sensitivity to model mismatches and limited adaptability to real-world uncertainties. To address this, we develop CycleRL, a comprehensive sim-to-real framework for robust autonomous bicycle control. Our approach establishes a direct perception-to-action mapping within the high-fidelity NVIDIA Isaac Sim environment, leveraging Proximal Policy Optimization (PPO) to optimize the control policy. The framework features a composite reward function tailored for concurrent balance maintenance, velocity tracking, and steering control. Crucially, systematic domain randomization is employed to reduce the reliance on precise system modeling, bridge the simulation-to-reality gap and facilitate direct transfer. In simulation, CycleRL achieves promising performance, including a 99.90% balance success rate, a heading tracking error of 1.15{\deg}, and a velocity tracking error of 0.18 m/s. These quantitative results, coupled with successful hardware deployment, validate DRL as an effective paradigm for autonomous bicycle control, offering superior adaptability over traditional methods.
Video demonstrations are available at https://cpnt-lab.github.io/CycleRL/.
Read the full abstract ▾
All authors Gelu Liu, Teng Wang, Zhijie Wu, Junliang Wu, Songyuan Li, Xiangwei Zhu
Robotics · Cross-listed from cs.CV
Thermal Image Refinement with Depth Estimation using Recurrent Networks for Monocular ORB-SLAM3
Hürkan Şahin, Huy Xuan Pham, Van Huyen Dang et al. (+2 more)
Autonomous
navigation in GPS-denied and visually degraded environments remains challenging for unmanned aerial vehicles (
UAVs). To this end, we investigate the use of a
monocular thermal camera as a standalone sensor on a
UAV platform for real-time depth estimation and simultaneous localization and mapping (SLAM). To extract depth information from thermal images, we propose a novel pipeline employing a lightweight supervised network with recurrent blocks (RBs) integrated to capture temporal dependencies, enabling more robust predictions. The network combines lightweight convolutional backbones with a thermal refinement network (T-RefNet) to refine raw thermal inputs and enhance feature visibility. The refined thermal images and predicted depth maps are integrated into ORB-SLAM3, enabling thermal-only localization. Unlike previous methods, the network is trained on a custom non-radiometric
dataset, obviating the need for high-cost radiometric thermal cameras. Experimental results on
datasets and
UAV flights demonstrate competitive depth accuracy and robust SLAM performance under low-light conditions. On the radiometric VIVID++ (indoor-dark)
dataset, our method achieves an absolute relative error of approximately 0.06, compared to baselines exceeding 0.11. In our non-radiometric indoor set, baseline errors remain above 0.24, whereas our approach remains below 0.10. Thermal-only ORB-SLAM3 maintains a mean trajectory error under 0.4 m.
Read the full abstract ▾
All authors Hürkan Şahin, Huy Xuan Pham, Van Huyen Dang, Alper Yegenoglu, Erdal Kayacan
Robotics
ReMAP-DP: Reprojected Multi-view Aligned PointMaps for Diffusion Policy
Xinzhang Yang, Renjun Wu, Jinyan Liu et al. (+1 more)
Generalist
robot policies built upon 2D visual representations excel at semantic
reasoning but inherently lack the explicit 3D spatial awareness required for high-precision tasks. Existing 3D integration methods struggle to bridge this gap due to the structural irregularity of sparse point clouds and the
geometric distortion introduced by multi-view orthographic rendering. To overcome these barriers, we present ReMAP-DP, a novel framework synergizing standardized perspective reprojection with a structure-aware dual-stream
diffusion policy. By coupling the re-projected views with pixel-aligned PointMaps, our dual-stream architecture leverages learnable modality
embeddings to fuse frozen semantic features and explicit
geometric descriptors, ensuring precise implicit patch-level
alignment. Extensive experiments across simulation and real-world environments demonstrate ReMAP-DP's superior performance in diverse manipulation tasks. On RoboTwin 2.0, it attains a 59.3% average success rate, outperforming the DP3 baseline by +6.6%. On ManiSkill 3, our method yields a 28% improvement over DP3 on the geometrically challenging Stack Cube task. Furthermore, ReMAP-DP exhibits remarkable real-world robustness, executing high-precision and dynamic manipulations with superior data efficiency from only a handful of demonstrations. Project page is available at: https://icr-lab.github.io/ReMAP-DP/
Read the full abstract ▾
All authors Xinzhang Yang, Renjun Wu, Jinyan Liu, Xuesong Li
Robotics · Cross-listed from cs.CV
Voronoi-based Second-order Descriptor with Whitened Metric in LiDAR Place Recognition
Jaein Kim, Hee Bin Yoo, Dong-Sig Han et al. (+1 more)
The pooling layer plays a vital role in aggregating local descriptors into the metrizable global descriptor in the
LiDAR Place Recognition (LPR). In particular, the second-order pooling is capable of capturing higher-order interactions among local descriptors. However, its existing methods in the LPR adhere to conventional implementations and post-normalization, and incur the descriptor unsuitable for Euclidean distancing. Based on the recent interpretation that associates NetVLAD with the second-order statistics, we propose to integrate second-order pooling with the inductive bias from Voronoi
cells. Our novel pooling method aggregates local descriptors to form the second-order
matrix and whitens the global descriptor to implicitly measure the Mahalanobis distance while conserving the cluster property from Voronoi
cells, addressing its numerical instability during learning with diverse techniques. We demonstrate its performance gains through the experiments conducted on the Oxford Robotcar and Wild-Places
benchmarks and analyze the numerical effect of the proposed whitening
algorithm.
Read the full abstract ▾
All authors Jaein Kim, Hee Bin Yoo, Dong-Sig Han, Byoung-Tak Zhang
Robotics
Learning from Mistakes: Post-Training for Driving VLA with Takeover Data
Yinfeng Gao, Deqing Liu, Qichao Zhang et al. (+7 more)
Current
Vision-
Language-Action (VLA) paradigms in end-to-end autonomous driving rely on offline training from static
datasets, leaving them vulnerable to distribution shift. Recent post-training methods use takeover data to mitigate this by augmenting the
dataset with high-quality expert takeover samples, yet they suffer from two key limitations: supervision restricted to the period after the takeover moments leads to policies with limited safety margins, and passive preference optimization lacks active exploration for optimal performance. In this paper, we propose TakeVLA, a novel VLA post-training framework that overcomes these shortcomings through two complementary innovations. First, we introduce pre-takeover
language supervision, which allows the VLA to learn from mistakes proactively. By explicitly teaching the model about what to do in error-prone situations, we cultivate a precautionary mindset that anticipates hazards early and substantially enlarges safety margins. Second, we propose Scenario Dreaming, a reinforcement fine-tuning paradigm that operates in reconstruceted takeover scenarios, encouraging active exploration beyond mere preference fitting. Experiments on the Bench2Drive
benchmark demonstrate that TakeVLA achieves state-of-the-art closed-loop performance, surpassing the strong VLA baseline SimLingo by 4.93 in driving score, with an enhanced safety margin as evidenced by an 11.76% increase in average TTC.
Read the full abstract ▾
All authors Yinfeng Gao, Deqing Liu, Qichao Zhang, Yupeng Zheng, Haochen Tian, Guang Li, Hangjun Ye, Long Chen, Da-Wei Ding, Dongbin Zhao
Robotics · Cross-listed from eess.SY
Intelligent Control of Differential Drive Robots Subject to Unmodeled Dynamics with EKF-based State Estimation
Amos Alwala, Yuchen Hu, Gabriel da Silva Lima et al. (+1 more)
Reliable control and state estimation of differential drive
robots (DDR) operating in dynamic and uncertain environments remains a challenge, particularly when system dynamics are partially unknown and sensor measurements are prone to degradation. This work introduces a unified control and state estimation framework that combines a Lyapunov-based nonlinear controller and Adaptive Neural Networks (ANN) with Extended Kalman Filter (EKF)-based multi-sensor fusion. The proposed controller leverages the universal approximation property of neural networks to model unknown nonlinearities in real time. An online adaptation scheme updates the weights of the radial basis function (RBF), the architecture chosen for the ANN. The learned dynamics are integrated into a feedback linearization (FBL) control law, for which theoretical guarantees of closed-loop stability and asymptotic convergence in a trajectory-tracking task are established through a Lyapunov-like stability analysis. To ensure robust state estimation, the EKF fuses inertial measurement unit (IMU) and odometry from
monocular, 2D-
LiDAR and wheel encoders. The fused state estimate drives the intelligent controller, ensuring consistent performance even under drift, wheel slip, sensor noise and failure. Gazebo simulations and real-world experiments are done using DDR, demonstrating the effectiveness of the approach in terms of improved velocity tracking performance with reduction in linear and angular velocity errors up to $53.91\%$ and $29.0\%$ in comparison to the baseline FBL.
Read the full abstract ▾
All authors Amos Alwala, Yuchen Hu, Gabriel da Silva Lima, Wallace Moreira Bessa
Robotics · Cross-listed from eess.SY
Generalizable Optimal Control with Transformers: One Policy Across Diverse Systems
Turki Bin Mohaya, Maitham F. AL-Sunni, John M. Dolan et al. (+1 more)
Classical optimal control designs a separate controller for each plant. Even for the Linear Quadratic Regulator (LQR), every new model must be identified and its Riccati equation re-solved. We ask whether a single learned policy can instead serve an entire family of systems, and we show that one
transformer can. We train the policy to imitate optimal LQR state feedback across a collection of heterogeneous Multiple-Input, Multiple-Output (
MIMO) Linear Time-Invariant (LTI) systems that differ in their state and input dimensions and in their cost objectives. A shared representation lets the same parameters control every member of the family. It combines system-wise standardization, zero-padding and masking across dimensions, and an explicit encoding of the cost
matrices. At run time, the policy maps a short window of recent states and the specified cost to a control action. It uses no plant
matrices and identifies the dynamics implicitly from the state history. We evaluate on $28$ simulated systems over $9{,}675$ closed-loop rollouts, and no unstable rollout was observed in any of them. On the systems seen during training, it attains a median relative sub-optimality of $0.022\%$, even under parameter perturbations of up to $\pm10\%$. It transfers to unseen systems with lightweight fine-tuning, reaching a median sub-optimality of $0.19\%$. These results support
transformers as generalizable near-optimal controllers for structured families of linear systems.
Read the full abstract ▾
All authors Turki Bin Mohaya, Maitham F. AL-Sunni, John M. Dolan, Peter Seiler
Robotics
PerlAD: Towards Enhanced Closed-loop End-to-end Autonomous Driving with Pseudo-simulation-based Reinforcement Learning
Yinfeng Gao, Qichao Zhang, Deqing Liu et al. (+8 more)
End-to-end autonomous driving policies based on Imitation Learning (IL) often struggle in closed-loop execution due to the misalignment between inadequate open-loop training objectives and real driving requirements. While Reinforcement Learning (RL) offers a solution by directly optimizing driving goals via reward signals, the rendering-based training environments introduce the rendering gap and are inefficient due to high computational costs. To overcome these challenges, we present a novel Pseudo-simulation-based RL method for closed-loop end-to-end autonomous driving, PerlAD. Based on offline
datasets, PerlAD constructs a pseudo-simulation that operates in vector space, enabling efficient, rendering-free trial-and-error training. To bridge the gap between static
datasets and dynamic closed-loop environments, PerlAD introduces a prediction world model that generates reactive
agent trajectories conditioned on the ego vehicle's plan. Furthermore, to facilitate efficient planning, PerlAD utilizes a hierarchical decoupled planner that combines IL for lateral path generation and RL for longitudinal speed optimization. Comprehensive experimental results demonstrate that PerlAD achieves state-of-the-art performance on the Bench2Drive
benchmark, surpassing the previous E2E RL method by 10.29% in Driving Score without requiring expensive online interactions. Additional evaluations on the DOS
benchmark further confirm its reliability in handling safety-critical occlusion scenarios.
Read the full abstract ▾
All authors Yinfeng Gao, Qichao Zhang, Deqing Liu, Zhongpu Xia, Guang Li, Kun Ma, Guang Chen, Hangjun Ye, Long Chen, Da-Wei Ding, Dongbin Zhao
Robotics
From Folding Mechanics to Robotic Function: A Unified Modeling Framework for Compliant Origami
Bohan Zhang, Bo Wang, Huajiang Ouyang et al. (+5 more)
Origami inspired architectures offer a powerful route toward lightweight, reconfigurable, and programmable robotic systems. Yet, a unified mechanics framework capable of seamlessly bridging rigid folding, elastic deformation, and stability driven transitions in compliant origami remains lacking. Here, we introduce a geometry consistent modeling framework based on discrete differential geometry (DDG) that unifies panel elasticity and crease rotation within a single variational formulation. By
embedding crease panel coupling directly into a mid edge
geometric discretization, the framework naturally captures rigid folding limits, distributed bending, multistability, and nonlinear dynamic snap through within one mechanically consistent structure. This unified description enables programmable control of stability and deformation across rigid and compliant regimes, allowing origami structures to transition from static folding mechanisms to active robotic modules. An implicit dynamic formulation incorporating
gravity, contact, friction, and
magnetic actuation further supports strongly coupled multiphysics simulations. Through representative examples spanning single fold bifurcation, deployable Miura membranes, bistable Waterbomb modules, and Kresling based crawling
robots, we demonstrate how geometry driven mechanics directly informs robotic functionality. This work establishes discrete differential geometry as a foundational design
language for intelligent origami robotics, enabling predictive modeling, stability programming, and mechanics guided robotic actuation within a unified computational platform.
Read the full abstract ▾
All authors Bohan Zhang, Bo Wang, Huajiang Ouyang, Zhigang Wu, Haohao Bi, Jiawei Xu, Mingchao Liu, Weicheng Huang
Robotics
ViSA: Visited-State Augmentation for Generalized Goal-Space Contrastive Reinforcement Learning
Issa Nakamura, Tomoya Yamanokuchi, Yuki Kadokawa et al. (+5 more)
Goal-Conditioned Reinforcement Learning (GCRL) is a framework for learning a policy that can reach arbitrarily given goals. In particular, Contrastive Reinforcement Learning (CRL) provides a framework for policy updates using an approximation of the value function estimated via contrastive learning, achieving higher sample efficiency compared to conventional methods. However, since CRL treats the visited state as a pseudo-goal during learning, it can accurately estimate the value function only for limited goals. To address this issue, we propose a novel data augmentation approach for CRL called ViSA (Visited-State Augmentation). ViSA consists of two components: 1) generating augmented state samples, with the aim of augmenting hard-to-visit state samples during on-policy exploration, and 2) learning consistent
embedding space, which uses an augmented state as auxiliary information to regularize the
embedding space by reformulating the objective function of the
embedding space based on mutual information. We evaluate ViSA in simulation and real-world robotic tasks and show improved goal-space generalization, which permits accurate value estimation for hard-to-visit goals. Further details can be found on the project page: https://issa-n.github.io/projectPage_ViSA/
Read the full abstract ▾
All authors Issa Nakamura, Tomoya Yamanokuchi, Yuki Kadokawa, Jia Qu, Shun Otsub, Ken Miyamoto, Shotaro Miwa, Takamitsu Matsubara
Robotics
Yoshiki Yamamoto, Maina Sogabe, Shunichi Hirahara et al. (+3 more)
Robotic surgery for minimally invasive surgery can reduce the surgeon's workload by autonomously guiding robotic forceps. Movement of the
robot is restricted around a fixed insertion port. The
robot often encounters angle limitations during operation. Also, the surface of the abdominal cavity is non-concave, making it computationally expensive to find the desired path.In this work, to solve these problems, we propose a method for path planning in joint space by transforming the position into a Riemannian
manifold. An edge cost function is defined to search for a desired path in the joint space and reduce the range of motion of the joints. We found that the organ is mostly non-concave, making it easy to find the optimal path using gradient descent method. Experimental results demonstrated that the proposed method reduces the range of joint angle movement compared to calculations in position space.
Read the full abstract ▾
All authors Yoshiki Yamamoto, Maina Sogabe, Shunichi Hirahara, Toshiki Kaisaki, Tetsuro Miyazaki, Kenji Kawashima
Robotics · Cross-listed from cs.CV
AutoMoT: A Unified Vision-Language-Action Model with Asynchronous Mixture-of-Transformers for End-to-End Autonomous Driving
Wenhui Huang, Songyan Zhang, Qihang Huang et al. (+6 more)
Integrating
vision-
language models (VLMs) into end-to-end (E2E) autonomous driving (AD) systems has shown promise in improving scene understanding. However, existing integration strategies suffer from several limitations: they either struggle to resolve distribution misalignment between
reasoning and action spaces, underexploit the general
reasoning capabilities of pretrained VLMs, or incur substantial inference latency during action policy generation, which degrades driving performance. To address these challenges, we propose AutoMoT in this work, an end-to-end AD framework that unifies
reasoning and action generation within a single
vision-
language-action (VLA) model. Our approach leverages a mixture-of-
transformer (MoT) architecture with joint
attention sharing, which preserves the general
reasoning capabilities of pre-trained VLMs while enabling efficient fast-slow inference through asynchronous execution at different task frequencies. Extensive experiments on multiple
benchmarks, under both open- and closed-loop settings, demonstrate that AutoMoT achieves competitive performance compared to state-of-the-art methods. We further investigate the functional boundary of pre-trained VLMs in AD, examining when AD-tailored fine-tuning is necessary. Our results show that pre-trained VLMs can achieve competitive multi-task scene understanding performance through semantic prompting alone, while fine-tuning remains essential for action-level tasks such as decision-making and trajectory planning. We refer to https://automot-website.github.io/ for the demonstration
videos and qualitative results.
Read the full abstract ▾
All authors Wenhui Huang, Songyan Zhang, Qihang Huang, Zhidong Wang, Zhiqi Mao, Collister Chua, Zhan Chen, Long Chen, Chen Lv