Robotics
Data-driven Interpretable Hybrid Robot Dynamics
Christopher E. Mower, Rui Zong and Haitham Bou-Ammar
We study data-driven identification of interpretable hybrid
robot dynamics, where an analytical rigid-body dynamics model is complemented by a learned residual torque term. Using symbolic regression and sparse identification of nonlinear dynamics (SINDy), we recover compact closed-form expressions for this residual from joint-space data. In simulation on a 7-DoF Franka arm with known dynamics, these interpretable models accurately recover inertial, Coriolis,
gravity, and viscous effects with very small relative error and outperform neural-network baselines in both accuracy and generalization. On real data from a 7-DoF WAM arm, symbolic-regression residuals generalize substantially better than SINDy and neural networks, which tend to overfit, and suggest candidate new closed-form formulations that extend the nominal dynamics model for this
robot. Overall, the results indicate that interpretable residual dynamics models provide compact, accurate, and physically meaningful alternatives to black-box function approximators for torque prediction.
Read the full abstract ▾
Robotics
MI-SLAM: Magnetic Inertial SLAM Systems
Chuan Huang, Gustaf Hendeby and Isaac Skog
Spatially inhomogeneous
magnetic fields offer a valuable, non-visual information source for positioning. Among systems leveraging this,
magnetic field-based simultaneous localization and mapping (SLAM) systems are particularly attractive. These systems execute positioning and
magnetic field mapping tasks simultaneously, and they have bounded positioning error within previously visited regions. However, state-of-the-art
magnetic-field SLAM methods typically require low-drift odometry data provided by visual odometry, a wheel encoder, or pedestrian dead-reckoning technology, which may not be available under certain situations. To address this limitation, this work proposes two
magnetic inertial SLAM (MI-SLAM) systems — a loosely coupled version and a tightly coupled version — that rely solely on low-cost sensors: an inertial measurement unit (IMU) and seven magnetometers. Both systems use a
magnetic field aided inertial
navigation system (MAINS) to obtain odometry. The key difference between the two systems is whether the
navigation state estimation is done in one or two steps. These systems are evaluated in real-world indoor environments with multi-floor structures. The proposed MI-SLAM systems outperform the state-of-the-art
magnetic-inertial odometry system, MAINS, achieving a positioning RMSE below 1 meter on a trajectory lasting approximately 200 seconds. A potential application of the proposed systems is for the positioning of emergency response officers in mine or fire rescue operations, where one cannot rely on a GNSS or visual-based localization system.
Read the full abstract ▾
Robotics · Cross-listed from cs.LG
CHyLL: Learning Continuous Neural Representations of Hybrid Systems
Sangli Teng, Hang Liu, Jingyu Song et al. (+1 more)
Learning the flows of hybrid systems that have both continuous and discrete time dynamics is challenging. The existing method learns the dynamics in each discrete mode, which suffers from the combination of mode switching and discontinuities in the flows. In this work, we propose CHyLL (Continuous Hybrid System Learning in Latent Space), which learns a continuous neural representation of a hybrid system without trajectory
segmentation, event functions, or mode switching. The key insight of CHyLL is that the reset map glues the state space at the guard surface, reformulating the state space as a piecewise smooth quotient
manifold where the flow becomes spatially continuous. Building upon these insights and the
embedding theorems grounded in differential
topology, CHyLL concurrently learns a singularity-free neural
embedding in a higher-dimensional space and the continuous flow in it. We showcase that CHyLL can accurately predict the flow of hybrid systems with superior accuracy and identify the topological invariants of the hybrid systems. Finally, we apply CHyLL to the
stochastic optimal control problem.
Read the full abstract ▾
All authors Sangli Teng, Hang Liu, Jingyu Song, Koushil Sreenath
Robotics
Fast Functionally Redundant Inverse Kinematics for Robotic Toolpath Optimisation in Manufacturing Tasks
Andrew Razjigaev, Hans Lohr, Alejandro Vargas-Uscategui et al. (+2 more)
Industrial automation with six-axis robotic arms is critical for many manufacturing tasks, including welding and additive manufacturing applications; however, many of these operations are functionally redundant due to the symmetrical tool axis, which effectively makes the operation a five-axis task. Exploiting this redundancy is crucial for achieving the desired workspace and dexterity required for the feasibility and optimisation of toolpath planning. Inverse kinematics
algorithms can solve this in a fast, reactive framework, but these techniques are underutilised over the more computationally expensive offline planning methods. We propose a novel
algorithm to solve functionally redundant inverse kinematics for robotic manipulation utilising a task space decomposition approach, the damped least-squares method and Halley's method to achieve fast and robust solutions with reduced joint motion. We evaluate our methodology in the case of toolpath optimisation in a cold spray coating application on a non-planar surface. The functionally redundant inverse kinematics
algorithm can quickly solve motion plans that minimise joint motion, expanding the feasible operating space of the complex toolpath. We validate our approach on an industrial ABB manipulator and cold-spray gun executing the computed toolpath.
Read the full abstract ▾
All authors Andrew Razjigaev, Hans Lohr, Alejandro Vargas-Uscategui, Peter King, Tirthankar Bandyopadhyay
Robotics
Push Smarter, Not Harder: Hierarchical RL-Diffusion Policy for Efficient Nonprehensile Manipulation
Steven Caro and Stephen L. Smith
Nonprehensile manipulation, such as pushing objects across cluttered environments, presents a challenging control problem due to complex contact dynamics and long-horizon planning requirements. In this work, we propose HeRD, a hierarchical reinforcement learning-
diffusion policy that decomposes pushing tasks into two levels: high-level goal selection and low-level trajectory generation. We employ a high-level reinforcement learning (RL)
agent to select intermediate spatial goals, and a low-level goal-conditioned
diffusion model to generate feasible, efficient trajectories to reach them. This architecture combines the long-term reward maximizing behaviour of RL with the generative capabilities of
diffusion models. We evaluate our method in a 2D simulation environment and show that it outperforms the state-of-the-art baseline in success rate, path efficiency, and generalization across multiple environment configurations. Our results suggest that hierarchical control with generative low-level planning is a promising direction for scalable, goal-directed nonprehensile manipulation. Code, documentation, and trained models are available: https://github.com/carosteven/HeRD.
Read the full abstract ▾
Robotics
Openpi Comet: Competition Solution For 2025 BEHAVIOR Challenge
Junjie Bai, Yu-Wei Chao, Qizhi Chen et al. (+13 more)
The 2025 BEHAVIOR Challenge is designed to rigorously track progress toward solving long-horizon tasks by physical
agents in simulated environments. BEHAVIOR-1K focuses on everyday household tasks that people most want
robots to assist with and these tasks introduce long-horizon mobile manipulation challenges in realistic settings, bridging the gap between current research and real-world, human-centric applications. This report presents our solution to the 2025 BEHAVIOR Challenge in a very close 2nd place and substantially outperforms the rest of the submissions. Building on $\pi_{0.5}$, we focus on systematically building our solution by studying the effects of training techniques and data. Through careful ablation studies, we reveal the scaling benefits in both the pre-training and post-training phases, leading to a validation Q-score of 0.345, significantly surpassing previous state-of-the-art performance. We summarize our practical lessons and design recommendations that we hope will provide actionable insights for the broader
embodied AI community when adapting powerful foundation models to complex
embodied scenarios. Project page: https://github.com/mli0603/openpi-comet
Read the full abstract ▾
All authors Junjie Bai, Yu-Wei Chao, Qizhi Chen, Jinwei Gu, Moo Jin Kim, Zhaoshuo Li, Xuan Li, Tsung-Yi Lin, Ming-Yu Liu, Nic Ma, Kaichun Mo, Delin Qu, Shangkun Sun, Hongchi Xia, Fangyin Wei, Xiaohui Zeng
Robotics · Cross-listed from cs.LG
Closing the Train-Test Gap in World Models for Gradient-Based Planning
Rohun Agrawal, Nimit Kalra, Arjun Parthasarathy et al. (+4 more)
World models paired with model predictive control (MPC) can be trained offline on large-scale
datasets of expert trajectories and enable generalization to a wide range of planning tasks at inference time. Compared to traditional MPC procedures, which rely on slow search
algorithms or on iteratively solving optimization problems exactly, gradient-based planning offers a computationally efficient alternative. However, the performance of gradient-based planning has thus far lagged behind that of other approaches. In this paper, we propose improved methods for training world models that enable efficient gradient-based planning. We begin with the observation that although a world model is trained on a next-state prediction objective, it is used at test-time to instead estimate a sequence of actions. The goal of our work is to close this train-test gap. To that end, we propose train-time data synthesis techniques that enable significantly improved gradient-based planning with existing world models. At test time, our approach outperforms or matches the classical gradient-free cross-
entropy method (CEM) across a variety of object manipulation and
navigation tasks in 10% of the time budget.
Read the full abstract ▾
All authors Rohun Agrawal, Nimit Kalra, Arjun Parthasarathy, Yann LeCun, Oumayma Bounou, Pavel Izmailov, Micah Goldblum
Robotics
HiF-VLA: Hindsight, Insight and Foresight through Motion Representation for Vision-Language-Action Models
Minghui Lin, Pengxiang Ding, Shu Wang et al. (+7 more)
Vision-
Language-Action (VLA) models have recently enabled robotic manipulation by grounding visual and linguistic cues into actions. However, most VLAs assume the Markov property, relying only on the current observation and thus suffering from temporal myopia that degrades long-horizon coherence. In this work, we view motion as a more compact and informative representation of temporal context and world dynamics, capturing inter-state changes while filtering static pixel-level noise. From this perspective, HiF-VLA equips a motion-centric world model for the VLA, enabling
agents to reason about temporal dynamics for future
evolution during action generation. Building on this idea, we propose HiF-VLA (Hindsight, Insight, and Foresight for VLAs), a unified framework that leverages motion for bidirectional temporal
reasoning. HiF-VLA encodes past dynamics through hindsight priors, anticipates future motion via foresight
reasoning, and integrates both through a hindsight-modulated joint expert to enable a ”think-while-acting” paradigm for long-horizon manipulation. As a result, HiF-VLA surpasses strong baselines on LIBERO-Long and CALVIN ABC-D
benchmarks, while incurring negligible additional inference latency. Furthermore, HiF-VLA achieves substantial improvements in real-world long-horizon manipulation tasks, demonstrating its broad effectiveness in practical robotic settings.
Read the full abstract ▾
All authors Minghui Lin, Pengxiang Ding, Shu Wang, Zifeng Zhuang, Yang Liu, Xinyang Tong, Wenxuan Song, Shangke Lyu, Siteng Huang, Donglin Wang
Robotics
Token Expand-Merge: Training-Free Token Compression for Vision-Language-Action Models
Yifan Ye, Jiaqi Ma, Jun Cen et al. (+1 more)
Vision-
Language-Action (VLA) models pretrained on large-scale
multimodal datasets have emerged as powerful foundations for robotic perception and control. However, their massive scale, often billions of parameters, poses significant challenges for real-time deployment, as inference becomes computationally expensive and latency-sensitive in dynamic environments. To address this, we propose
Token Expand-and-Merge-VLA (TEAM-VLA), a training-free
token compression framework that accelerates VLA inference while preserving task performance. TEAM-VLA introduces a dynamic
token expansion mechanism that identifies and samples additional informative
tokens in the spatial vicinity of
attention-highlighted regions, enhancing contextual completeness. These expanded
tokens are then selectively merged in deeper layers under action-aware guidance, effectively reducing redundancy while maintaining semantic coherence. By coupling expansion and merging within a single feed-forward pass, TEAM-VLA achieves a balanced trade-off between efficiency and effectiveness, without any retraining or parameter updates. Extensive experiments on LIBERO
benchmark demonstrate that TEAM-VLA consistently improves inference speed while maintaining or even surpassing the task success rate of full VLA models. The code is public available on \href{https://github.com/Jasper-aaa/TEAM-VLA}{https://github.com/Jasper-aaa/TEAM-VLA}
Read the full abstract ▾
All authors Yifan Ye, Jiaqi Ma, Jun Cen, Zhihe Lu
Robotics
LISN: Language-Instructed Social Navigation with VLM-based Controller Modulating
Junting Chen, Yunchuan Li, Panfeng Jiang et al. (+5 more)
Towards human-
robot coexistence, socially aware
navigation is significant for mobile
robots. Yet existing studies on this area focus mainly on path efficiency and pedestrian collision avoidance, which are essential but represent only a fraction of social
navigation. Beyond these basics,
robots must also comply with user instructions, aligning their actions to task goals and social norms expressed by humans. In this work, we present LISN-Bench, the first simulation-based
benchmark for
language-instructed social
navigation. Built on Rosnav-Arena 3.0, it is the first standardized social
navigation benchmark to incorporate instruction following and scene understanding across diverse contexts. To address this task, we further propose Social-Nav-Modulator, a fast-slow hierarchical system where a VLM
agent modulates costmaps and controller parameters. Decoupling low-level action generation from the slower VLM loop reduces reliance on high-frequency VLM inference while improving dynamic avoidance and perception adaptability. Our method achieves an average success rate of 91.3%, which is greater than 63% than the most competitive baseline, with most of the improvements observed in challenging tasks such as following a person in a crowd and navigating while strictly avoiding instruction-forbidden regions. The project website is at: https://social-nav.github.io/LISN-project/
Read the full abstract ▾
All authors Junting Chen, Yunchuan Li, Panfeng Jiang, Jiacheng Du, Zixuan Chen, Chenrui Tie, Jiajun Deng, Lin Shao
Robotics
Py-DiSMech: A Scalable and Efficient Framework for Discrete Differential Geometry-Based Modeling and Control of Soft Robots
Radha Lahoti, Ryan Chaiyakul and M. Khalid Jawed
High-fidelity simulation has become essential to the design and control of soft
robots, where large
geometric deformations and complex contact interactions challenge conventional modeling tools. Recent advances in the field demand simulation frameworks that combine physical accuracy, computational scalability, and seamless integration with modern control and optimization pipelines. In this work, we present Py-DiSMech, a Python-based, open-source simulation framework for modeling and control of soft robotic structures grounded in the principles of Discrete Differential Geometry (DDG). By discretizing
geometric quantities such as
curvature and strain directly on meshes, Py-DiSMech captures the nonlinear deformation of rods, shells, and hybrid structures with high fidelity and reduced computational cost. The framework introduces (i) a fully vectorized NumPy implementation achieving order-of-magnitude speed-ups over existing geometry-based simulators; (ii) a penalty-energy-based fully implicit contact model that supports rod-rod, rod-shell, and shell-shell interactions; (iii) a natural-strain-based feedback-control module featuring a proportional-integral (PI) controller for shape regulation and trajectory tracking; and (iv) a modular, object-oriented software design enabling user-defined elastic energies, actuation schemes, and integration with machine-learning libraries.
Benchmark comparisons demonstrate that Py-DiSMech substantially outperforms the state-of-the-art simulator Elastica in computational efficiency while maintaining physical accuracy. Together, these features establish Py-DiSMech as a scalable, extensible platform for simulation-driven design, control validation, and sim-to-real research in soft robotics.
Read the full abstract ▾
Robotics
YOPO-Nav: Visual Navigation using 3DGS Graphs from One-Pass Videos
Ryan Meegan, Adam D'Souza, Bryan Bo Cao et al. (+2 more)
Visual
navigation has emerged as a practical alternative to traditional robotic
navigation pipelines that rely on detailed mapping and path planning. However, constructing and maintaining 3D maps is often computationally expensive and
memory-intensive. We address the problem of visual
navigation when exploration
videos of a large environment are available. The
videos serve as a visual reference, allowing a
robot to retrace the explored trajectories without relying on metric maps. Our proposed method, YOPO-Nav (You Only Pass Once), encodes an environment into a compact spatial representation composed of interconnected local 3D Gaussian Splatting (3DGS) models. During
navigation, the framework aligns the
robot's current visual observation with this representation and predicts actions that guide it back toward the demonstrated trajectory. YOPO-Nav employs a hierarchical design: a visual place recognition (VPR) module provides coarse localization, while the local 3DGS models refine the goal and intermediate poses to generate control actions. To evaluate our approach, we introduce the YOPO-Campus
dataset, comprising 4 hours of
egocentric video and
robot controller inputs from over 6 km of human-teleoperated
robot trajectories. We
benchmark recent visual
navigation methods on trajectories from YOPO-Campus using a Clearpath Jackal
robot. Experimental results show YOPO-Nav provides excellent performance in image-goal
navigation for real-world scenes on a physical
robot. The
dataset and code will be made publicly available for visual
navigation and scene representation research.
Read the full abstract ▾
All authors Ryan Meegan, Adam D'Souza, Bryan Bo Cao, Shubham Jain, Kristin Dana
Robotics
Visual Heading Prediction for Autonomous Aerial Vehicles
Reza Ahmari, Ahmad Mohammadi, Vahid Hemmati et al. (+5 more)
The integration of Unmanned Aerial Vehicles (
UAVs) and Unmanned Ground Vehicles (UGVs) is increasingly central to the development of intelligent autonomous systems for applications such as search and rescue, environmental monitoring, and logistics. However, precise coordination between these platforms in real-time scenarios presents major challenges, particularly when external localization infrastructure such as GPS or GNSS is unavailable or degraded [1]. This paper proposes a
vision-based, data-driven framework for real-time
UAV-UGV integration, with a focus on robust UGV detection and heading angle prediction for
navigation and coordination. The system employs a fine-tuned YOLOv5 model to detect UGVs and extract bounding box features, which are then used by a lightweight artificial neural network (ANN) to estimate the
UAV's required heading angle. A VICON motion capture system was used to generate ground-truth data during training, resulting in a
dataset of over 13,000 annotated images collected in a controlled lab environment. The trained ANN achieves a mean absolute error of 0.1506{\deg} and a root mean squared error of 0.1957{\deg}, offering accurate heading angle predictions using only
monocular camera inputs. Experimental evaluations achieve 95% accuracy in UGV detection. This work contributes a
vision-based, infrastructure- independent solution that demonstrates strong potential for deployment in GPS/GNSS-denied environments, supporting reliable multi-
agent coordination under realistic dynamic conditions. A demonstration
video showcasing the system's real-time performance, including UGV detection, heading angle prediction, and
UAV alignment under dynamic conditions, is available at: https://github.com/Kooroshraf/
UAV-UGV-Integration
Read the full abstract ▾
All authors Reza Ahmari, Ahmad Mohammadi, Vahid Hemmati, Mohammed Mynuddin, Parham Kebria, Mahmoud Nabil Mahmoud, Xiaohong Yuan, Abdollah Homaifar
Robotics
Simultaneous Tactile-Visual Perception for Learning Multimodal Robot Manipulation
Yuyang Li, Yinghan Chen, Zihang Zhao et al. (+4 more)
Robotic manipulation requires both rich
multimodal perception and effective learning frameworks to handle complex real-world tasks. See-through-skin (STS) sensors, which combine tactile and visual perception, offer promising sensing capabilities, while modern imitation learning provides powerful tools for policy acquisition. However, existing STS designs lack simultaneous
multimodal perception and suffer from unreliable tactile tracking. Furthermore, integrating these rich
multimodal signals into learning-based manipulation pipelines remains an open challenge. We introduce TacThru, an STS sensor enabling simultaneous visual perception and robust tactile signal extraction, and TacThru-UMI, an imitation learning framework that leverages these
multimodal signals for manipulation. Our sensor features a fully transparent elastomer, persistent illumination, novel keyline markers, and efficient tracking, while our learning system integrates these signals through a
Transformer-based
Diffusion Policy. Experiments on five challenging real-world tasks show that TacThru-UMI achieves an average success rate of 85.5%, significantly outperforming the baselines of tactile policy(66.3%) and
vision-only policy (55.4%). The system excels in critical scenarios, including contact detection with thin and soft objects and precision manipulation requiring
multimodal coordination. This work demonstrates that combining simultaneous
multimodal perception with modern learning frameworks enables more precise, adaptable robotic manipulation.
Read the full abstract ▾
All authors Yuyang Li, Yinghan Chen, Zihang Zhao, Puhao Li, Tengyu Liu, Siyuan Huang, Yixin Zhu
Robotics
Bridging the Basilisk Astrodynamics Framework with ROS 2 for Modular Spacecraft Simulation and Hardware Integration
Elias Krantz, Ngai Nam Chan, Gunnar Tibert et al. (+2 more)
Integrating high-fidelity spacecraft simulators with modular robotics frameworks remains a challenge for autonomy development. This paper presents a lightweight, open-source communication bridge between the Basilisk astrodynamics simulator and the
Robot Operating System 2 (ROS 2), enabling real-time, bidirectional data exchange for spacecraft control. The bridge requires no changes to Basilisk's core and integrates seamlessly with ROS 2 nodes. We demonstrate its use in a leader-follower formation flying scenario using nonlinear model predictive control, deployed identically in both simulation and on the ATMOS planar microgravity testbed. This setup supports rapid development, hardware-in-the-loop testing, and seamless transition from simulation to hardware. The bridge offers a flexible and scalable platform for modular spacecraft autonomy and reproducible research workflows.
Read the full abstract ▾
All authors Elias Krantz, Ngai Nam Chan, Gunnar Tibert, Huina Mao, Christer Fuglesang
Robotics
High-Resolution Water Sampling via a Solar-Powered Autonomous Surface Vehicle
Misael Mamani, Mariel Fernandez, Grace Luna et al. (+5 more)
Accurate water quality assessment requires spatially resolved sampling, yet most unmanned surface vehicles (USVs) can collect only a limited number of samples or rely on single-point sensors with poor representativeness. This work presents a
solar-powered, fully autonomous USV featuring a novel syringe-based sampling architecture capable of acquiring 72 discrete, contamination-minimized water samples per mission. The vehicle incorporates a ROS 2 autonomy stack with GPS-RTK
navigation,
LiDAR and stereo-
vision obstacle detection, Nav2-based mission planning, and long-range LoRa supervision, enabling dependable execution of sampling routes in unstructured environments. The platform integrates a behavior-tree autonomy architecture adapted from Nav2, enabling mission-level
reasoning and perception-aware
navigation. A modular 6x12 sampling system, controlled by distributed micro-ROS nodes, provides deterministic actuation, fault isolation, and rapid module replacement, achieving spatial coverage beyond previously reported USV-based samplers. Field trials in Achocalla Lagoon (La Paz, Bolivia) demonstrated 87% waypoint accuracy, stable autonomous
navigation, and accurate physicochemical measurements (temperature, pH, conductivity, total dissolved solids) comparable to manually collected references. These results demonstrate that the platform enables reliable high-resolution sampling and autonomous mission execution, providing a scalable solution for aquatic monitoring in remote environments.
Read the full abstract ▾
All authors Misael Mamani, Mariel Fernandez, Grace Luna, Steffani Limachi, Leonel Apaza, Carolina Montes-Dávalos, Marcelo Herrera, Edwin Salcedo
Robotics
ReMoSPLAT: Reactive Mobile Manipulation Control on a Gaussian Splat
Nicolas Marticorena, Tobias Fischer and Niko Suenderhauf
Reactive control can gracefully coordinate the motion of the base and the arm of a mobile manipulator. However, incorporating an accurate representation of the environment to avoid obstacles without involving costly planning remains a challenge. In this work, we present ReMoSPLAT, a reactive controller based on a quadratic program formulation for mobile manipulation that leverages a Gaussian Splat representation for collision avoidance. By integrating additional constraints and costs into the optimisation formulation, a mobile manipulator platform can reach its intended end effector pose while avoiding obstacles, even in cluttered scenes. We investigate the trade-offs of two methods for efficiently calculating
robot-obstacle distances, comparing a purely
geometric approach with a rasterisation-based approach. Our simulation experiments on both synthetic and real-world scans demonstrate the feasibility of the proposed method, achieving performance comparable to controllers that rely on perfect ground-truth information. We further validate the approach on a real
robot platform more details: https://remosplat.github.io
Read the full abstract ▾
Robotics
Minghao Guo, Meng Cao, Jiachen Tao et al. (+5 more)
Most existing
Vision-
Language-Action (VLA) models rely primarily on RGB information, while ignoring
geometric cues crucial for spatial
reasoning and manipulation. In this work, we introduce GLaD, a geometry-aware VLA framework that incorporates 3D
geometric priors during pretraining through knowledge
distillation. Rather than distilling
geometric features solely into the
vision encoder, we align the LLM's hidden states corresponding to visual
tokens with features from a frozen geometry-aware
vision transformer (VGGT), ensuring that
geometric understanding is deeply integrated into the
multimodal representations that drive action prediction. Pretrained on the Bridge
dataset with this geometry
distillation mechanism, GLaD achieves 94.1% average success rate across four LIBERO task suites, outperforming UniVLA (92.5%) which uses identical pretraining data. These results validate that geometry-aware pretraining enhances spatial
reasoning and policy generalization without requiring explicit depth sensors or 3D annotations.
Read the full abstract ▾
All authors Minghao Guo, Meng Cao, Jiachen Tao, Rongtao Xu, Yan Yan, Xiaodan Liang, Ivan Laptev, Xiaojun Chang
Robotics
Super4DR: 4D Radar-centric Self-supervised Odometry and Gaussian-based Map Optimization
Zhiheng Li, Weihua Wang, Qiang Shen et al. (+2 more)
Conventional odometry and mapping methods using visual or
LiDAR data often struggle under poor illumination and adverse weather conditions. Although 4D radar is suited for such environments, its sparse and noisy point clouds hinder accurate odometry estimation, while the radar maps suffer from obscure and incomplete structures. Thus, we propose Super4DR, a 4D radar-centric framework for learning-based odometry estimation and gaussian-based map optimization. First, we design a cluster-aware odometry network that incorporates object-level cues from the clustered radar points for inter-frame matching, alongside a hierarchical self-supervision mechanism to overcome outliers through spatio-temporal consistency, knowledge transfer, and feature contrast. Second, we propose using 3D gaussians as an intermediate representation, coupled with a radar-specific growth strategy, selective separation, and multi-view regularization, to recover blurry map areas and those undetected based on image texture. Experiments show that Super4DR achieves a 67% performance gain over prior self-supervised methods, nearly matches supervised odometry, and narrows the map quality disparity with
LiDAR while enabling multi-modal image rendering.
Read the full abstract ▾
All authors Zhiheng Li, Weihua Wang, Qiang Shen, Yichen Zhao, Zheng Fang
Robotics
UrbanNav: Learning Language-Guided Urban Navigation from Web-Scale Human Trajectories
Yanghong Mei, Yirong Yang, Longteng Guo et al. (+5 more)
Navigating complex urban environments using natural
language instructions poses significant challenges for
embodied agents, including noisy
language instructions, ambiguous spatial references, diverse landmarks, and dynamic street scenes. Current visual
navigation methods are typically limited to simulated or off-street environments, and often rely on precise goal formats, such as specific coordinates or images. This limits their effectiveness for autonomous
agents like last-mile delivery
robots navigating unfamiliar cities. To address these limitations, we introduce UrbanNav, a scalable framework that trains
embodied agents to follow free-form
language instructions in diverse urban settings. Leveraging web-scale city walking
videos, we develop an scalable annotation pipeline that aligns human
navigation trajectories with
language instructions grounded in real-world landmarks. UrbanNav encompasses over 1,500 hours of
navigation data and 3 million instruction-trajectory-landmark triplets, capturing a wide range of urban scenarios. Our model learns robust
navigation policies to tackle complex urban scenarios, demonstrating superior spatial
reasoning, robustness to noisy instructions, and generalization to unseen urban settings. Experimental results show that UrbanNav significantly outperforms existing methods, highlighting the potential of large-scale web
video data to enable
language-guided, real-world urban
navigation for
embodied agents.
Read the full abstract ▾
All authors Yanghong Mei, Yirong Yang, Longteng Guo, Qunbo Wang, Ming-Ming Yu, Xingjian He, Wenjun Wu, Jing Liu
Robotics · Cross-listed from cs.CV
CS3D: An Efficient Facial Expression Recognition via Event Vision
Zhe Wang, Qijin Song, Yucen Peng et al. (+1 more)
Responsive and accurate facial expression recognition is crucial to human-
robot interaction for daily service
robots. Nowadays, event cameras are becoming more widely adopted as they surpass RGB cameras in capturing facial expression changes due to their high temporal resolution, low latency, computational efficiency, and robustness in low-light conditions. Despite these advantages, event-based approaches still encounter practical challenges, particularly in adopting mainstream deep learning models. Traditional deep learning methods for facial expression analysis are energy-intensive, making them difficult to deploy on edge computing devices and thereby increasing costs, especially for high-frequency, dynamic, event
vision-based approaches. To address this challenging issue, we proposed the CS3D framework by decomposing the Convolutional 3D method to reduce the computational complexity and energy consumption. Additionally, by utilizing soft spiking
neurons and a spatial-temporal
attention mechanism, the ability to retain information is enhanced, thus improving the accuracy of facial expression detection. Experimental results indicate that our proposed CS3D method attains higher accuracy on multiple
datasets compared to architectures such as the RNN,
Transformer, and C3D, while the energy consumption of the CS3D method is just 21.97% of the original C3D required on the same device.
Read the full abstract ▾
All authors Zhe Wang, Qijin Song, Yucen Peng, Weibang Bai
Robotics
Mastering Diverse, Unknown, and Cluttered Tracks for Robust Vision-Based Drone Racing
Feng Yu, Yu Hu, Yang Su et al. (+3 more)
Most reinforcement learning(RL)-based methods for drone racing target fixed, obstacle-free tracks, leaving the generalization to unknown, cluttered environments largely unaddressed. This challenge stems from the need to balance racing speed and collision avoidance, limited feasible space causing policy exploration trapped in local optima during training, and perceptual ambiguity between gates and obstacles in depth maps-especially when gate positions are only coarsely specified. To overcome these issues, we propose a two-phase learning framework: an initial soft-collision training phase that preserves policy exploration for high-speed flight, followed by a hard-collision refinement phase that enforces robust obstacle avoidance. An adaptive, noise-augmented curriculum with an asymmetric actor-critic architecture gradually shifts the policy's reliance from privileged gate-state information to depth-based visual input. We further impose Lipschitz constraints and integrate a track-primitive generator to enhance motion stability and cross-environment generalization. We evaluate our framework through extensive simulation and ablation studies, and validate it in real-world experiments on a computationally constrained quadrotor. The system achieves agile flight while remaining robust to gate-position errors, developing a generalizable drone racing framework with the capability to operate in diverse, partially unknown and cluttered environments. https://yufengsjtu.github.io/MasterRacing.github.io/
Read the full abstract ▾
All authors Feng Yu, Yu Hu, Yang Su, Yang Deng, Linzuo Zhang, Danping Zou
Robotics
REASAN: Learning Reactive Safe Navigation for Legged Robots
Qihao Yuan, Ziyu Cao, Ming Cao et al. (+1 more)
We present a novel modularized end-to-end framework for legged reactive
navigation in complex dynamic environments using a single light detection and ranging (
LiDAR) sensor. The system comprises four simulation-trained modules: three reinforcement-learning (RL) policies for locomotion, safety shielding, and
navigation, and a
transformer-based exteroceptive
estimator that processes raw point-cloud inputs. This modular decomposition of complex legged motor-control tasks enables lightweight neural networks with simple architectures, trained using standard RL practices with targeted reward shaping and curriculum design, without reliance on heuristics or sophisticated policy-switching mechanisms. We conduct comprehensive ablations to validate our design choices and demonstrate improved robustness compared to existing approaches in challenging
navigation tasks. The resulting reactive safe
navigation (REASAN) system achieves fully onboard and real-time reactive
navigation across both single- and multi-
robot settings in complex environments. We release our training and deployment code at https://github.com/ASIG-X/REASAN.
Read the full abstract ▾
All authors Qihao Yuan, Ziyu Cao, Ming Cao, Kailai Li
Robotics
Donato Caramia, Florian T. Pokorny, Giuseppe Triggiani et al. (+3 more)
Occlusions in robotic bin picking compromise accurate and reliable grasp planning. We present ViTA-Seg, a class-agnostic
Vision Transformer framework for real-time amodal
segmentation that leverages global
attention to recover complete object masks, including hidden regions. We proposte two architectures: a) Single-Head for amodal mask prediction; b) Dual-Head for amodal and occluded mask prediction. We also introduce ViTA-SimData, a photo-realistic synthetic
dataset tailored to industrial bin-picking scenario. Extensive experiments on two amodal
benchmarks, COOCA and KINS, demonstrate that ViTA-Seg Dual Head achieves strong amodal and occlusion
segmentation accuracy with computational efficiency, enabling robust, real-time robotic manipulation.
Read the full abstract ▾
All authors Donato Caramia, Florian T. Pokorny, Giuseppe Triggiani, Denis Ruffino, David Naso, Paolo Roberto Massenio
Robotics
On Mobile Ad Hoc Networks for Coverage of Partially Observable Worlds
Edwin Meriaux, Shuo Wen, Louis-Roy Langevin et al. (+3 more)
This paper addresses the movement and placement of mobile
agents to establish a communication network in initially unknown environments. We cast the problem in a computational-
geometric framework by relating the coverage problem and line-of-sight constraints to the Cooperative Guard Art Gallery Problem, and introduce its partially observable variant, the Partially Observable Cooperative Guard Art Gallery Problem (POCGAGP). We then present two
algorithms that solve POCGAGP: CADENCE, a centralized planner that incrementally selects 270 degree corners at which to deploy
agents, and DADENCE, a decentralized scheme that coordinates
agents using local information and lightweight messaging. Both approaches operate under partial observability and target simultaneous coverage and connectivity. We evaluate the methods in simulation across 1,500 test cases of varied size and structure, demonstrating consistent success in forming connected networks while covering and exploring unknown space. These results highlight the value of
geometric abstractions for communication-driven exploration and show that decentralized policies are competitive with centralized performance while retaining scalability.
Read the full abstract ▾
All authors Edwin Meriaux, Shuo Wen, Louis-Roy Langevin, Doina Precup, Antonio Loría, Gregory Dudek
Robotics
Development of a Compliant Gripper for Safe Robot-Assisted Trouser Dressing-Undressing
Jayant Unde, Takumi Inden, Yuki Wakayama et al. (+4 more)
In recent years, many countries, including Japan, have rapidly aging populations, making the preservation of seniors' quality of life a significant concern. For elderly people with impaired physical abilities, support for toileting is one of the most important issues. This paper details the design, development, experimental assessment, and potential application of the gripper system, with a focus on the unique requirements and obstacles involved in aiding elderly or hemiplegic individuals in dressing and undressing trousers. The gripper we propose seeks to find the right balance between compliance and grasping forces, ensuring precise manipulation while maintaining a safe and compliant interaction with the users. The gripper's integration into a custom–built robotic manipulator system provides a comprehensive solution for assisting hemiplegic individuals in their dressing and undressing tasks. Experimental evaluations and comparisons with existing studies demonstrate the gripper's ability to successfully assist in both dressing and dressing of trousers in confined spaces with a high success rate. This research contributes to the advancement of assistive robotics, empowering elderly, and physically impaired individuals to maintain their independence and improve their quality of life.
Read the full abstract ▾
All authors Jayant Unde, Takumi Inden, Yuki Wakayama, Jacinto Colan, Yaonan Zhu, Tadayoshi Aoyama, Yasuhisa Hasegawa
Robotics
Calibrated Keyframe-Segment Selection for Descriptor-Agnostic LiDAR Loop Closure in Repetitive Environments
Jaehyun Kim, Seungwon Choi, Wonseok Kang et al. (+1 more)
False
LiDAR loop closures can distort pose graphs in repetitive environments, where one query may match several distant places. Genuine revisits, unlike many perceptual aliases, tend to remain consistent across neighboring keyframes. We present Query-Calibrated Segmental Admission (QCSA), a loop-candidate selection method between descriptor
retrieval and point-cloud registration. QCSA maps descriptor scores to evidence that distinguishes true loops from hard negatives, accumulates it over short keyframe segments, and admits only the best segment above a threshold fitted to the strongest false segment for each calibration query. The admitted segment forms two local submaps for one Generalized Iterative Closest Point (GICP) registration. QCSA requires pairwise descriptor scores and separate calibration for each descriptor and setting. Across seven descriptors on the SNU Library
Dataset (SNULib), QCSA lowers average trajectory error with 53.8% fewer registration attempts than Top-1. With common local submaps and downstream processing on NCLT, it also reduces registration attempts, constraints exceeding the pose-error tolerance, and trajectory error relative to Top-1. HeLiPR Bridge further evaluates repeated outdoor geometry. Our implementation and
dataset will be released at: https://github.com/wanderingcar/snu_library_
dataset.
Read the full abstract ▾
All authors Jaehyun Kim, Seungwon Choi, Wonseok Kang, Tae-Wan Kim
Robotics
A Hierarchical, Model-Based System for High-Performance Humanoid Soccer
Quanyou Wang, Mingzhang Zhu, Ruochen Hou et al. (+16 more)
The development of athletic
humanoid robots has gained significant
attention as advances in actuation, sensing, and control enable increasingly dynamic, real-world capabilities. RoboCup, an international competition of fully autonomous
humanoid robots, provides a uniquely challenging
benchmark for such systems, culminating in the long-term goal of competing against human soccer players by 2050. This paper presents the hardware and software innovations underlying our team's victory in the RoboCup 2024 Adult-Sized
Humanoid Soccer Competition. On the hardware side, we introduce an adult-sized
humanoid platform built with lightweight structural components, high-torque quasi-direct-drive actuators, and a specialized foot design that enables powerful in-gait kicks while preserving locomotion robustness. On the software side, we develop an integrated perception and localization framework that combines stereo
vision, object detection, and landmark-based fusion to provide reliable estimates of the ball, goals, teammates, and opponents. A mid-level
navigation stack then generates collision-aware, dynamically feasible trajectories, while a centralized behavior manager coordinates high-level decision making, role selection, and kick execution based on the evolving game state. The seamless integration of these subsystems results in fast, precise, and tactically effective gameplay, enabling robust performance under the dynamic and
adversarial conditions of real matches. This paper presents the design principles, system architecture, and experimental results that contributed to ARTEMIS's success as the 2024 Adult-Sized
Humanoid Soccer champion.
Read the full abstract ▾
All authors Quanyou Wang, Mingzhang Zhu, Ruochen Hou, Kay Gillespie, Alvin Zhu, Shiqi Wang, Yicheng Wang, Gaberiel I. Fernandez, Yeting Liu, Colin Togashi, Hyunwoo Nam, Aditya Navghare, Alex Xu, Taoyuanmin Zhu, Min Sung Ahn, Arturo Flores Alvarez, Justin Quan, Ethan Hong, Dennis W. Hong
Robotics
D$^2$GSLAM: 4D Dynamic Gaussian Splatting SLAM
Siting Zhu, Yuxiang Huang, Wenhua Wu et al. (+4 more)
Recent advances in Dense Simultaneous Localization and Mapping (SLAM) have demonstrated remarkable performance in static environments. However, dense SLAM in dynamic environments remains challenging. Most methods directly remove dynamic objects and focus solely on static scene reconstruction, which ignores the motion information contained in these dynamic objects. In this paper, we present D$^2$GSLAM, a novel dynamic SLAM system utilizing Gaussian representation, which simultaneously performs accurate dynamic reconstruction and robust tracking within dynamic environments. Our system is composed of four key components: (i) We propose a
geometric-
prompt dynamic separation method to distinguish between static and dynamic elements of the scene. This approach leverages the
geometric consistency of Gaussian representation and scene geometry to obtain coarse dynamic regions. The regions then serve as
prompts to guide the refinement of the coarse mask for achieving accurate motion mask. (ii) To facilitate accurate and efficient mapping of the dynamic scene, we introduce dynamic-static composite representation that integrates static 3D Gaussians with dynamic 4D Gaussians. This representation allows for modeling the transitions between static and dynamic states of objects in the scene for composite mapping and optimization. (iii) We employ a progressive pose refinement strategy that leverages both the multi-view consistency of static scene geometry and motion information from dynamic objects to achieve accurate camera tracking. (iv) We introduce a motion consistency loss, which leverages the temporal continuity in object motions for accurate dynamic modeling. Our D$^2$GSLAM demonstrates superior performance on dynamic scenes in terms of mapping and tracking accuracy, while also showing capability in accurate dynamic modeling.
Read the full abstract ▾
All authors Siting Zhu, Yuxiang Huang, Wenhua Wu, Chaokang Jiang, Yongbo Chen, I-Ming Chen, Hesheng Wang
Robotics
Generalizable Collaborative Search-and-Capture in Cluttered Environments via Path-Guided MAPPO and Directional Frontier Allocation
Jialin Ying, Zhihao Li, Zicheng Dong et al. (+2 more)
Collaborative pursuit-evasion in cluttered environments presents significant challenges due to sparse rewards and constrained Fields of View (FOV). Standard Multi-
Agent Reinforcement Learning (MARL) often suffers from inefficient exploration and fails to scale to large scenarios. We propose PGF-MAPPO (Path-Guided Frontier MAPPO), a hierarchical framework bridging topological planning with reactive control. To resolve local minima and sparse rewards, we integrate an A*-based potential field for dense reward shaping. Furthermore, we introduce Directional Frontier Allocation, combining Farthest Point Sampling (FPS) with
geometric angle suppression to enforce spatial dispersion and accelerate coverage. The architecture employs a parameter-shared decentralized critic, maintaining O(1) model complexity suitable for robotic swarms. Experiments demonstrate that PGF-MAPPO achieves superior capture efficiency against faster evaders. Policies trained on 10x10 maps exhibit robust zero-shot generalization to unseen 20x20 environments, significantly outperforming rule-based and learning-based baselines.
Read the full abstract ▾
All authors Jialin Ying, Zhihao Li, Zicheng Dong, Guohua Wu, Yihuan Liao