Robotics
Online Learning of Robust Legged Odometry with Minimal Exteroceptive Supervision
Abhijeet M. Kulkarni, Yuze Du and Guoquan Huang
Robust locomotion and
navigation for legged
robots relies heavily on dependable odometry. Traditional multi-sensor fusion for such state estimation requires meticulous sensor calibration and platform-specific kinematic modeling, which complicates deployment. Industrially packaged exteroceptive sensors can provide accurate motion tracking but remain vulnerable to perceptually degraded conditions. We thus develop a plug-and-play, robust legged odometry system that eliminates the need for explicit exteroceptive-to-proprioceptive calibration or system kinematic modeling. Our approach leverages established exteroceptive motion pipelines as a continuous supervisory signal to train an online learned velocity neural network directly from proprioceptive data. An Invariant EKF (InEKF) is then used to fuse the learned proprioceptive or exteroceptive velocity (if any) and IMU data. When exteroception fails due to environmental degradation, the system seamlessly falls back to using the learned proprioceptive model, yielding a resilient legged odometry that readily adapts to new hardware. We demonstrate the platform-agnostic, easily deployable nature of our approach on different quadruped platforms, showcasing promising results in maintaining robust motion estimation across challenging scenarios.
Read the full abstract ▾
Robotics
Energy-based Compositional Diffusion Planning
Tao Sun, Utkarsh Aashu Mishra, Jiaxin Lu et al. (+2 more)
Compositional
diffusion planners aim to solve long-horizon robotic tasks using short training trajectories. Yet, current approaches often rely on the heuristic stitching of local predictions. We show that the resulting stitched update is generally a non-conservative field} that does not mathematically correspond to any valid global trajectory log-density function. We propose Energy-based Compositional Diffuser (ECD), a framework that formulates the global trajectory as the minimizer of the sum of local bridge potentials. This energy-based perspective defines a conservative correction field and contains a boundary reaction term that heuristic stitching omits. To enable efficient inference, we further introduce a Markov-based score approximation that computes the reaction term via a single block-tridiagonal solve, maintaining time complexity linear in the planning horizon. Empirically, ECD achieves state-of-the-art success rates on a range of OGBench stitching tasks, while nearly matching the inference speed of heuristic stitching methods. Code is available at https://github.com/GradientSpaces/ECD.
Read the full abstract ▾
All authors Tao Sun, Utkarsh Aashu Mishra, Jiaxin Lu, Danfei Xu, Iro Armeni
Robotics
VQActFlow: Vector-Quantized Action Mode Steering for Multi-Task Robot Manipulation
Zhigen Zhao, Mark Leggiero, Yipu Chen et al. (+5 more)
Multi-task
robot manipulation policies are challenging to learn from demonstration because traditionally a single network must select among qualitatively different action modes from a
multimodal demonstration distribution, conditioned on
language and visual context. A wrong mode selection means executing the wrong task or an action infeasible in the scene. Tokenizing continuous actions into a learned discrete codebook separates these modes at the representation level, offering structural advantages for multi-task learning. We propose VQActFlow, a multi-task manipulation policy that tokenizes action chunks and generates code sequences via Variational Flow Matching. VQActFlow maintains an explicit preference over action modes throughout generation. Inference-time guidance acts on this preference to steer mode commitment. We instantiate this with classifier-free guidance over
language conditioning, which steers the policy toward the instructed action mode, and a learned codebook critic that supplies a complementary feasibility signal. We evaluate VQActFlow on three platforms: the LIBERO simulation
benchmarks, a Unitree G1
humanoid performing whole-body pick-and-place, and an ALOHA-style bimanual platform performing contact-rich tasks. Across these
benchmarks, VQActFlow outperforms both continuous and discrete baselines.
Read the full abstract ▾
All authors Zhigen Zhao, Mark Leggiero, Yipu Chen, Haoran Liu, Yifan Wu, Huishu Xue, Sirui Zhan, Ye Zhao
Robotics · Cross-listed from cs.CV
Boundary-by-Mask: Few-Shot Instance Segmentation with Mask-Conditioned Boundary Learning for Texture-Poor Industrial Parts
Yutaka Yoshinaga, Naoya Chiba and Koichi Hashimoto
Recent advances in large pre-trained models have led to remarkable progress in instance
segmentation on general images. However, industrial scenarios remain challenging. Instance definitions are often application-specific and inconsistent, and the domain gap from general imagery is substantial due to weak textures and limited contextual cues. Consequently, a direct application of existing models is unreliable. We propose Boundary-by-Mask, a few-shot instance
segmentation framework that supervises boundaries instead of interior appearance. Given a few RGB images and corresponding instance masks, the method extracts rich visual features using a foundation-model encoder and trains a lightweight Signed Distance Function (SDF) head to predict boundary-aware distance maps.
Segmentation masks are obtained through an SDF-to-mask reconstruction process. By explicitly estimating contours, the framework achieves reliable instance separation even on low-texture and color-uniform surfaces. The instance definition is conditioned by the instance mask. Replacing the mask specifies the
segmentation target, such as the whole object or a sub-part. A pixel-wise shallow MLP head enables rapid training. Experiments on industrial parts and food items with ambiguous boundaries show strong few-shot generalization, robustness in feature-poor conditions, and precise control over mask-level targets.
Read the full abstract ▾
Robotics
Conflict-Aware Switching for CBF-CLF-Based Multi-Goal Navigation
Rohan Walia and Kevin Leahy
Quadratic programs (QPs) using Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs) are widely used for safe control in reach-and-avoid
navigation. However, the inherently conflicting nature of CBF and CLF constraints can lead to performance degradation, including slowdowns and deadlocks. This issue is exacerbated in multi-goal scenarios, where multiple nominal control objectives must be satisfied under shared safety constraints. Existing approaches for preemptive safety are often computationally expensive or overly conservative, while methods that relax or switch between nominal objectives are not well-suited for sequential goal-to-goal
navigation. To address these limitations, we propose a conflict-aware switching strategy that detects high-conflict conditions and switches between available nominal control objectives to reduce constraint conflict. We apply this approach to multi-
agent, multi-goal reach-and-avoid scenarios under CBF-CLF-QP control. Compared to a baseline sequential goal traversal strategy, our method reduces both completion time and timeout rates, demonstrating improved performance in satisfying all nominal control objectives while respecting safety constraints.
Read the full abstract ▾
Robotics
Robot Critics that Sweat the Small Stuff
Sruthi Sudhakar, Junbang Liang, Sreehari Rammohan et al. (+3 more)
Large
vision-
language models contain several priors about the world and object interactions, making them useful critics during inference to steer
robot policies towards success. However, closed-loop
robot manipulation requires judging small visual differences between success and failure, which remains a challenge for current VLMs. We introduce a method to fine-tune critics by constructing pairwise progress supervision using success and failure rollouts obtained from a policy. Our fine-tuned critic excels at fine-grained progress
reasoning and subtle failure detection, outperforming prior progress
reasoning baselines. Additionally, we use an action-conditioned
video model to predict the visual effect of several candidate actions sampled from a policy, and show that our critic can correctly identify successful candidates to execute, improving the average policy success rate by 11% across real-world tasks and 5.9% across simulation tasks.
Read the full abstract ▾
All authors Sruthi Sudhakar, Junbang Liang, Sreehari Rammohan, Pavel Tokmakov, Richard Zemel, Carl Vondrick
Robotics
LOGOS: LiDAR-Only Gaussian Elevation Splatting for Unified Tiny Obstacle Segmentation
Nan Ming, Yeqiang Qian, Chunxiang Wang et al. (+1 more)
Robust obstacle
segmentation is essential for the safety of intelligent
robots, where
LiDAR-based perception systems play a fundamental role in the
robot-environment interaction. While extensive
LiDAR-based approaches have demonstrated high performance on common obstacles in urban scenarios, their results on tiny obstacles such as curbs, gravel, and potholes remain unsatisfactory due to the significant similarity between tiny obstacles and inherent road undulations. Moreover, their
segmentation accuracy even deteriorates sharply when the
LiDAR scans suffer from degradation in challenging off-road scenes. To overcome these bottlenecks, we propose LOGOS, a
LiDAR-only unified tiny obstacle
segmentation system, which models the road surface as a continuous mixture of 2D Gaussian primitives and distinguishes tiny obstacles via high-presicion elevation estimation. Unlike existing Gaussian splatting methods that rely on iterative RGB training, LOGOS is a backpropagation-free
LiDAR-only approach. It directly estimates Gaussian parameters via a freespace-aware initialization by incrementally
pruning non-road primitives using smoothness constraints. Subsequently, pointwise signed distances are computed via a novel normal-aware elevation splatting function, ensuring robustness to both flat and sloped terrains. We evaluate LOGOS on a highly heterogeneous
benchmark of point cloud frames collected from urban mobility scenarios and mining haulage off-road environments. These data are practically acquired using different
LiDAR sensors and exhibit large variations in point density, terrain roughness, and obstacle types. Experiments on the road and off-road scenes demonstrate that LOGOS significantly outperforms other state-of-the-art methods, particularly in degraded point cloud regions and challenging off-road scenarios, while maintaining real-time efficiency.
Read the full abstract ▾
All authors Nan Ming, Yeqiang Qian, Chunxiang Wang, Ming Yang
Robotics
A Stitch in Time Saves Nine: Preserving Policy Compatibility Under Perception Updates in End-to-End Autonomous Driving
Yueyuan Li, Yifei Xiao, Mingyang Jiang et al. (+3 more)
End-to-end autonomous driving systems tightly couple perception and decision-making through latent representations. Consequently, updates to perception models can alter these representations and degrade the performance of downstream policies that remain fixed. Existing solutions typically rely on policy retraining or architectural decoupling, both of which incur substantial computation and validation costs. In this paper, we formulate the model stitching problem for end-to-end autonomous driving and test the hypothesis that policy compatibility can be preserved through lightweight latent-space
alignment. We study low-complexity model stitching methods, including linear and convolutional stitchers, for restoring compatibility between updated perception modules and frozen downstream policy modules. Experiments demonstrate that stitching effectively preserves downstream driving behavior under diverse perception updates, including changes in random initialization, sensor configuration, and training domain. In the most challenging cross-domain setting from nuScenes to CARLA, convolutional stitching retains over 91% of the no-shift driving score while reducing adaptation time from \SI{22.18}{h} to \SI{0.91}{h}. These results suggest that model stitching provides an effective and computationally efficient alternative to retraining or fine-tuning for maintaining end-to-end autonomous driving systems. The model will be open-sourced upon paper acceptance at https://github.com/SCP-CN-001/model-stitching to support further research and development in autonomous driving.
Read the full abstract ▾
All authors Yueyuan Li, Yifei Xiao, Mingyang Jiang, Xiang Zuo, Songan Zhang, Ming Yang
Robotics
UniviewVLA: A Unified Multiview Vision-Language-Action Model with World Modeling
Tao Xu, Runhao Zhang, Zhijian Huang et al. (+7 more)
Occluded tasks remain a bottleneck in
robot manipulation. Existing solutions either deploy additional physical cameras requiring training-inference camera parity, or rely on explicit 3D reconstruction with high computational cost. Moreover, both approaches rely on standard
agent-view and wrist-view observations, while failing to capture occlusion information and future scene
evolution. To this end, we propose UniviewVLA, a unified multiview
Vision-
Language-Action model with world modeling, which infers multiview scene
evolution for action prediction from only standard two-camera observations. We demonstrate that by leveraging generated multiview future views from the world model, UniviewVLA reveals occluded cues and models future scene
evolution, improving action prediction and removing the need for extra hardware or explicit reconstruction. Besides, to accelerate inference while preserving prediction accuracy, UniviewVLA develops Motion-Informative
Token Compression, which compresses each generated view from 625 to 16
tokens and reduces per-view latency from 6-7s to 0.2-0.3s. UniviewVLA also proposes training-free Action-
Entropy View Selection, which dynamically identifies the most action-informative view at different inference stages. Extensive experiments show that UniviewVLA achieves 95.8% on LIBERO and 4.60 on CALVIN ABCD to D, both standard occlusion-free
benchmarks. On customized occlusion-focused tasks, it improves success rate from 40.0% to 73.3%, and average real-
robot success rate by 33.4 points, demonstrating stronger occlusion-focused performance without sacrificing standard occlusion-free
benchmarks.
Read the full abstract ▾
All authors Tao Xu, Runhao Zhang, Zhijian Huang, Jiayi Guan, Jiaxin Wang, Yifan Ding, Yong-Lu Li, Long Chen, Guang Chen, Jinghui Lu
Robotics
Decoupling the Declarative from the Procedural in Vision-Language-Action Models
Nikolaos Tsagkas, Andreas Sochopoulos, Chris Xiaoxuan Lu et al. (+2 more)
Deploying generalist robotic
agents in the real world requires transferable skills. Specifically, a policy trained to clone a behavior from object-specific demonstrations must generalize beyond that object, otherwise data collection requirements become intractable. Recently, fine-tuning of pre-trained billion-parameter
Vision-
Language Models (VLMs), initially on large-scale
robot datasets and then on fewer scenario-specific demonstrations, has emerged as the predominant paradigm for designing
Vision-
Language-Action (VLA) models. While these policies achieve state-of-the-art manipulation performance in-distribution, they remain brittle to minor spatial, semantic, and task variations. In this work, we address the inability of current models to decouple the declarative (i.e., concepts and entity semantics) from the procedural knowledge (i.e., how to do something) encoded in their parameters, which is a fundamental bottleneck for zero-shot skill transfer to novel objects. To address this, we propose w$^{2}$VLA, a new VLA model with restructured information flow. Rather than feeding all
multimodal tokens from the VLM encoder into a large, opaque
transformer-based action expert, our approach modulates the
robot state sequence with visual, spatial, and skill information in a compositional and interpretable manner. Unlike popular, state-of-the-art VLAs, we show that our modular approach successfully decouples knowledge representations, enabling robust behavior cloning and unprecedented zero-shot skill transfer capabilities across dissimilar, unseen objects.
Read the full abstract ▾
All authors Nikolaos Tsagkas, Andreas Sochopoulos, Chris Xiaoxuan Lu, Oisin Mac Aodha, Alexandros Kouris
Robotics
ASCII Art Turns LLMs into VLA Controllers
Yitao Jiang, Roy Xing, Luyang Zhao et al. (+3 more)
Vision–
Language–Action (VLA) controllers are often built by extending
vision–
language models (VLMs) with action supervision, relying on
multimodal backbones with large data and compute requirements. We demonstrate that a text-only large
language model (LLM) can be adapted into a VLA-style controller when visual observations are rendered into a text input using an ASCII representation. This ASCII-as-
vision interface enables existing training and deployment stacks for LLMs to efficiently condition on visual state, follow natural-
language instructions, and produce constrained, executable actions. We fine-tune and compare multiple LLMs and VLMs across model families and scales, using both expert demonstrations from a planning-based teacher, as well as DAgger for iterative improvement. In a 2D manipulation
benchmark, in both simulation and on a physical manipulator, the resulting controllers can identify task-relevant entities and plan feasible action sequences. Our results suggest that ASCII rendering can serve as a lightweight, interpretable modality bridge from images to text, complementing conventional VLA pipelines, and opening directions for VLA research with text-only backbones.
Read the full abstract ▾
All authors Yitao Jiang, Roy Xing, Luyang Zhao, Brian Plancher, Muhao Chen, Devin Balkcom
Robotics
Manipulider: A Multi-Engine Buoyancy-Controlled Robot for Thrusterless Underwater Gliding and Manipulation
Yitao Jiang, Yewei Huang, Weizhi Cao et al. (+5 more)
The Manipulider is a buoyancy-actuated
underwater robot that enables thrusterless, glide-like locomotion and attitude-based manipulation, while providing a
magnetic modular interface for rapid payload swapping (e.g., a gripper or sensors). Four syringe-based buoyancy engines distributed around the body jointly regulate net buoyancy and the center of buoyancy, allowing the vehicle to maintain large tilt angles through static force balance without continuous thrust and to avoid propeller
entanglement risks. We present the mechanical and electrical design, calibration procedure, and control architecture. Experiments with a gripper attached (no external payload) show a controllable buoyancy-displacement range of 40 mL per engine ({\approx}160 g total buoyancy authority), maximum statically stable tilts of 64.6{\deg} (single-engine) and 61.8{\deg} (dual-engine), and representative vertical and tilt-transition dynamics. We further demonstrate tilt regulation, controlled ascent/descent primitives, and a proof-of-concept gripper-based payload-transport sequence without thrusters.
Read the full abstract ▾
All authors Yitao Jiang, Yewei Huang, Weizhi Cao, Mingi Jeong, Alberto Quattrini Li, Luyang Zhao, Muhao Chen, Devin Balkcom
Robotics · Cross-listed from cs.CV
Technical Report for ICRA 2026 GOOSE 2D Fine-Grained Semantic Segmentation Challenge: Exploring Query-Based Segmentation and Increased Spatial Context for Outdoor Scene Understanding
David Pascual-Hernández, Roberto Calvo-Palomino, Inmaculada Mora-Jiménez et al. (+1 more)
In this report, we present our submission to the GOOSE 2D Fine-Grained Semantic
Segmentation Challenge, organized as part of the Workshop on Field Robotics at ICRA 2026. The challenge combines data from the GOOSE and GOOSE-Ex
datasets, which comprise more than 13k images captured from 4 distinct camera setups, annotated using a hierarchical taxonomy of 56 fine-grained classes and 11 broader categories. Starting from SegFormer as a baseline, we progressively improve
segmentation performance through increased training crop sizes, a transition to the query-based Mask2Former architecture, and test-time augmentation. Our experiments show that query-based
segmentation significantly outperforms the baseline model. Furthermore, increasing the crop size used during training yields substantial gains, highlighting the relevance of preserving scene context for fine-grained semantic disambiguation. Our final submission, using test-time augmentation, achieves an mIoU of 69.6% on the challenge test set, providing a strong baseline for fine-grained semantic
segmentation in outdoor environments. To facilitate reproducibility and future research, code and weights will be made publicly available at https://github.com/RoboticsLabURJC/outdoor-fine-grained-
segmentation .
Read the full abstract ▾
All authors David Pascual-Hernández, Roberto Calvo-Palomino, Inmaculada Mora-Jiménez, Jose María Cañas-Plaza
Robotics
Temporal logics and formal synthesis for robot planning and control
Jana Tumova, Joris Verhagen and Matti Vahs
As
robots move from controlled environments into real-world settings, it becomes increasingly crucial to ensure that they perform as expected. A key step toward that goal is a rigorous specification of the desired
robot behavior, capturing intricate temporal, spatial, and logical requirements. Complementing this, plan and control synthesis methods are needed to fulfill these specifications with provable guarantees. This manuscript presents temporal logics - particularly linear and signal temporal logic - as expressive specification
languages for
robot behavior over time. We then discuss principles of formal synthesis, from discrete graph- and game-based approaches to sampling-based motion planning, trajectory optimization, and control-certificate-based synthesis. Finally, we outline challenges in deploying formal synthesis in real-world robotics, emphasizing the interplay between modeling fidelity, computational tractability, and the types of rigorous guarantees that can be achieved.
Read the full abstract ▾
Robotics
Robot Self-Improvement via Human-Video Dynamics Models
Hanzhi Chen, Anran Zhang, Simon Schaefer et al. (+5 more)
A central question in
robot learning is how to acquire skills from the kinds of data that humans learn from: passive observation,
embodied practice, and the experience of failure. Human
videos provide the first of these in abundance, and prior work has shown they can initialize useful policies. Far less clear is whether they can support the second and third: whether priors extracted from human
videos can ground a
robot's own attempts well enough to evaluate them, correct them, and improve from them. In this work, we show that human
videos can be used to learn embodiment-agnostic action, dynamics, and value representations that transfer across
robot embodiments, providing the predictive foundation required for
robots to autonomously improve from their own rollouts and failures. We introduce Dynamics-Guided Action Correction (DGAC), a training-free approach that leverages these adapted models to repair failed states: each failure becomes a query for which the learned models propose and rank corrective actions, turning failures into supervision for the next policy update. Across seven real-world manipulation tasks spanning both a mobile manipulator and a static manipulator arm, our approach improves success rates from 40% to 81% across multiple policy backbones, demonstrating cross-embodiment
robot self-improvement from human-
video priors. These results show that human priors and
robot failures can be combined to enable scalable autonomous policy improvement. Project page: https://ethz-mrl.github.io/
robot-self-improvement-website/.
Read the full abstract ▾
All authors Hanzhi Chen, Anran Zhang, Simon Schaefer, Kejia Chen, Shi Chen, Daniel Cremers, Oier Mees, Stefan Leutenegger
Robotics
Rithvik Jonna, Aakash Gurram, Man Namgung et al. (+2 more)
Vision-
Language Models (VLMs) for
embodied navigation rely on selecting a fixed number of frames from a growing trajectory history. As episodes extend, this selection grows increasingly sparse, yet prior work shows no accuracy gain when scaling from 8 to 64 frames, suggesting the bottleneck is not frame quantity but the representation itself. Sparse frame selection cannot capture the structured behavioral signal that long-horizon
reasoning requires: turning patterns, cumulative displacement, and path
topology. We introduce BIT-Nav (
Brain-Inspired Trajectory
Memory for
Navigation), a framework that augments frozen VLM
navigation pipelines with a compact learned trajectory
memory. Motivated by hippocampal path integration, where spatial experience is compressed into structured episodic traces rather than stored as raw sensory replay, BIT-Nav trains a Bi-GRU encoder over action and relative pose sequences via a multi-positive InfoNCE contrastive objective on trajectory prefixes sharing the same behavioral intent. The resulting
embedding is projected into the VLM
token space via a lightweight MLP and injected as a single
memory token at each decision step, conditioning the model on structured motion history at constant
token cost regardless of episode length
Read the full abstract ▾
All authors Rithvik Jonna, Aakash Gurram, Man Namgung, Wyatt Mackey, Tinoosh Mohsenin
Robotics
Overcoming Imperfect Kinematics in Surgical Robotics Through Sim-to-Real Visuomotor Learning
Zhaoxuan Yan, Kaizhong Deng, Zhaoyang Jacopo Hu et al. (+2 more)
Robot-Assisted Surgery is integral to modern minimally invasive procedures, with automation emerging as the next frontier to enhance precision and reduce surgeon fatigue. This
evolution is largely impeded by the inherent kinematic inaccuracies of
surgical robots, where unreliable internal sensors lead to significant control errors. While previous methods attempted to mitigate these issues through complex model-based calibration, they often suffer from high cost and limited effectiveness. This work utilises a learning-policy to actively compensate for hardware inaccuracies using closed-loop visual feedback that was trained from a teacher-student learning framework. The policy can fuse unreliable internal readings with precise external visual data, allowing it to correct for kinematic errors in real time without needing a perfect physical model. The learned policy was successfully deployed on the da Vinci Research Kit, where experiments validated the fundamental feasibility of using external
vision to overcome internal sensor deficits. This research provides a foundational and reliable control methodology, paving the way for more advanced and robust
surgical automation.
Read the full abstract ▾
All authors Zhaoxuan Yan, Kaizhong Deng, Zhaoyang Jacopo Hu, George P. Mylonas, Daniel S. Elson
Robotics
Long-Distance Real-World Navigation of the Legged-Wheeled Robot Go2-W Using Deep Reinforcement Learning
Takaaki Matsuzawa, Kiyoshi Irie, Tomoaki Yoshida et al. (+3 more)
Legged-wheeled
robots have long been studied for their potential to combine the efficient flat-ground mobility of wheels with the rough-terrain capability of legs. However, examples of their application to long-range autonomous
navigation in real environments remain limited. This paper reports our effort to build a deep reinforcement learning (DRL) based locomotion controller and an autonomous
navigation system for the commercially available legged-wheeled
robot Go2-W, and to apply them to long-range autonomous
navigation in a real environment. For locomotion control, we extended a proprioception-only policy, which we had previously developed for quadruped
robots, to the 16-DoF legged-wheeled
robot. We also found that wheeled locomotion concentrates the load on the hip joints and causes heat concentration that hinders sustained travel, and obtained a policy that suppresses it by distributing the load. We evaluated the system at the Tsukuba Challenge 2025, demonstrating that it can autonomously traverse an approximately 2.8 km route including sidewalks, a park, and stairs without stopping due to overheating.
Read the full abstract ▾
All authors Takaaki Matsuzawa, Kiyoshi Irie, Tomoaki Yoshida, Taro Suzuki, Yoshitaka Hara, Masahiro Tomono
Robotics · Cross-listed from cs.CV
FLM-Occ: Feed-forward Likelihood Maximization for Efficient Indoor Occupancy Prediction
Guangcheng Chen, Lihuang Fang, Huaqi Tao et al. (+3 more)
Recent indoor occupancy prediction methods adopt Gaussian primitives as a sparse 3D representation for computational efficiency. However, their training relies on voxel classification, which imposes only local constraints and lacks global supervision on the distribution of the primitives. Therefore, they inevitably predict spurious primitives in empty regions, undermining both representational and computational efficiency. To address this, we propose Feed-forward Likelihood Maximization (FLM), a novel framework that reformulates occupancy prediction as voxel distribution estimation. In FLM, a network is trained to predict a mixture model that maximizes the likelihood over ground-truth occupied voxels in a feed-forward manner. To enable end-to-end training of networks and voxelization of a standard mixture model, we define mixture weights as normalized primitive volumes to implicitly enforce simplex constraints and derive novel voxelization formulas. Based on FLM, our FLM-Occ, a novel method that is capable of relocating randomly initialized primitives over long distances to model a scene. On Occ-ScanNet, FLM-Occ achieves superior accuracy using only 32 superquadrics, 2.7% of the prior SoTA, while running 3.7 times faster.
Read the full abstract ▾
All authors Guangcheng Chen, Lihuang Fang, Huaqi Tao, Yicheng He, Li He, Hong Zhang
Robotics
NAC: Neural Action Codec for Vision-Language-Action Models
Ahad Jawaid and Yu Xiang
Vision-
language-action (VLA) models rely on discrete action tokenizers to bridge continuous
robot control and autoregressive sequence modeling, yet existing tokenizers often trade off between compression, latency, and downstream performance. We revisit this design through the lens of neural audio codecs - convolutional encoder-decoder architectures with residual vector quantization that serve as the standard front end for audio foundation models. Motivated by their success, we introduce the Neural Action Codec (NAC), which treats short
robot action trajectories as multi-channel 1D signals and compresses them using a multi-scale RVQGAN architecture. With adaptations to the action representation, compression rate, and reconstruction objective, audio-codec-style models can autoencode actions with high fidelity without substantial architectural changes. NAC provides a compact, ordered
token space via offset codebooks, enabling standard autoregressive policies to operate over short, structured sequences. Meanwhile, a Vocos-style decoder with an ISTFT head and
adversarial discriminators recovers action trajectories. Across LIBERO-10, RoboMimic, and a suite of real-world manipulation tasks, NAC achieves high reconstruction fidelity and higher average success rates than binning, FAST, and prior VQ-based tokenizers at comparable or better compression rates. These results demonstrate that repurposed neural audio codecs offer a strong, practical backbone for learned action tokenization in modern VLAs.
Read the full abstract ▾
Robotics · Cross-listed from cs.AI
Mind the Noise: Sensitivity of Transformer-based Interaction-Aware Trajectory Prediction Models to Noisy Data
Shahab Salehi, Luca Lusvarghi, Miguel Sepulcre et al. (+1 more)
Trajectory prediction allows autonomous vehicles to anticipate the future behavior of surrounding objects (or
agents) and, accordingly, maximize the safety and efficiency of their driving. State-of-the-art Transformed-based interaction-aware trajectory prediction models, which rely on
attention mechanisms to capture multi-
agent interactions and maximize prediction accuracy, are commonly trained and evaluated on long-range high-quality
datasets. These
datasets are typically obtained by aggregating data from multiple vehicles or drones and removing any object detection or tracking noise offline. Yet, information about a surrounding object's state (its position, speed, heading) is far from being noiseless in real-world deployments. Object state estimation is affected by perception uncertainties and localization errors that can be particularly large for objects received via Vehicle-to-Everything (V2X) communications. In this paper, we analyze the impact of noisy object state information on the trajectory prediction accuracy of a state-of-the-art
Transformer-based interaction-aware trajectory prediction model. Our study demonstrates that trajectory prediction accuracy can rapidly deteriorate as the noise intensity increases. Numerical results show that the prediction accuracy can reduce by a 1.3x factor under small noise levels and by as much as a 3.9x factor under the highest (yet realistic) noise conditions. These findings reveal the strong sensitivity of trajectory prediction models to noisy data, underscoring the need for more realistic training and evaluation
datasets as well as noise mitigation strategies.
Read the full abstract ▾
All authors Shahab Salehi, Luca Lusvarghi, Miguel Sepulcre, Javier Gozalvez
Robotics
RogueRover: Autonomous Rogue Device Localization for Incident Response
Priyanka Prakash Surve, Asaf Shabtai and Yuval Elovici
Physically localizing unauthorized wireless devices remains a critical bottleneck in cyber-physical security operations, where rogue access points can provide entry points for lateral movement and persistent compromise. While such devices can often be detected through network-side mechanisms, determining their physical location typically requires dense sensing infrastructure, site-specific RF fingerprinting, or manual inspection, limiting timely incident response. We investigate whether a single commodity
robot can autonomously detect and localize rogue wireless devices under zero-configuration constraints, without RF fingerprinting, pre-installed sensors, or site calibration. We present RogueRover, an end-to-end system in which a quadruped
robot autonomously patrols, collects spatially labeled RSSI measurements via a standard 802.11 interface, and estimates device locations offline. We evaluate the system across 11 patrol runs in a real indoor environment, with 6 rogue devices deployed under heterogeneous propagation conditions. Across 62 AP-patrol sessions, RogueRover achieves a median single-patrol localization error of 1.62 m without prior RF knowledge. Under multi-run aggregation, five of six devices are localized within 1 m. A blind trial validates the full pipeline, correctly identifying rogue devices among 73 observed BSSIDs and localizing them with errors of 0.34 m and 1.84 m. Across environments, simple weighted-centroid
estimators perform comparably to, or better than, parametric path-loss models, indicating that measurement coverage from autonomous patrols is the primary determinant of localization accuracy under zero-prior constraints. Our results demonstrate that infrastructure-free, autonomous localization is feasible in practice, enabling rapid physical incident response in cyber-physical environments without additional sensing infrastructure.
Read the full abstract ▾
Robotics
Spectral GS-SLAM: Observability-Aware, Degeneracy-Robust Tracking for Real-Time 3D Gaussian Splatting SLAM
Edward Beng Wai Tan, Siew-Kei Lam and Dongshuo Zhang
Recent 3DGS-SLAM systems enable real-time operation by leveraging conventional feature matching or ICP-based tracking, thereby avoiding the heavy dense photometric optimization used in earlier approaches. However, feature matching remains prone to failure in textureless environments, while ICP-based tracking struggles in structureless or geometrically degenerate scenes due to ill-conditioned optimization. To address this issue, we propose Spectral GS-SLAM, an efficient yet robust tracking framework that integrates ICP with complementary feature-based constraints. Our method mitigates numerical instability by adaptively compensating under-constrained directions in degenerate scenarios, without interfering with the shared Gaussian representation used for mapping. We further introduce a Gaussian-aware planarity weighting mechanism that exploits the intrinsic covariance structure of 3D Gaussians to characterize scene geometry and guide information fusion. Extensive evaluations on challenging TUM RGB-D sequences demonstrate that Spectral GS-SLAM achieves real-time performance (40.14 FPS) while maintaining consistent tracking in both structureless and featureless environments. The proposed method preserves trajectory integrity in degenerate scenes while maintaining competitive performance in non-adverse conditions.
Read the full abstract ▾
Robotics
A Human-Inspired Thumb-Index Robotic Hand with Strain Gauges Embedded in Soft Joints
Jonas Papenbrock, Shubhan Patni, Tomaso Lisini Baldi et al. (+4 more)
Human hand grasp adaptation depends mainly on the synergy between physical structure and biological feedback. Inspired by this biomechanical principle, the Safe Thumb-Index Robotic (STIR) Hand was developed as a minimal, lightweight, and low-cost two-digit prototype featuring an asymmetric thumb-index configuration. By pairing an underactuated, tendon-driven mechanical design with flexible strain gauges embedded into silicone-encapsulated soft joints, the system achieves passive grasp adaptation while establishing both internal proprioception and external perception. Unsupervised analysis was carried out on a
dataset of the STIR hand grasping 20 different objects, along with an object classification task and an ablation study to highlight the contribution of the soft joint sensors. The object classification task discriminated object size, shape, and material stiffness with a high classification accuracy. In contrast to traditional industrial grippers and robotic hands, the STIR Hand demonstrates that sensorized compliant joints significantly improve overall sensitivity and ensure safe grasping, while remaining independent of additional fingertip tactile elements or external
vision systems. Finally, a comparison to similar devices grasping identical objects validates the utility of the STIR Hand.
Read the full abstract ▾
All authors Jonas Papenbrock, Shubhan Patni, Tomaso Lisini Baldi, Michele Guerri, Elia Landi, Ada Fort, Matej Hoffmann
Robotics
Ultra-Fusion: A Resilient Tightly-Coupled Multi-Sensor Fusion SLAM Framework under Sensor Degradation and Spatiotemporal Perturbation for Intelligent Transportation Systems
Yihong Tian, Junjie Zhang, Liuyang Li et al. (+3 more)
Reliable localization is essential for intelligent transportation systems (ITS), including autonomous vehicles, quadruped last-mile carriers, and infrastructure-inspection unmanned aerial vehicles (
UAVs). Although tightly-coupled multi-sensor fusion improves accuracy in favorable conditions, deployed systems remain vulnerable to sensor degradation – poor illumination,
LiDAR degeneracy, wheel slippage, and GNSS outage – and to spatiotemporal calibration errors. These failures are common in urban canyons, tunnels, and high-speed corridors, where localization drift can degrade route tracking, tunnel passage continuity, and local map
alignment. This paper presents Ultra-Fusion, a tightly-coupled multi-sensor localization framework based on a unified sliding-window
estimator. Asynchronous measurements are timestamp-ordered and converted into optional factors within one optimization window, supporting WIO, VIO, LIO, and LVIO with optional wheel and GNSS augmentation. Observability-aware initialization selects the bootstrap mode, factor-wise reliability scheduling gates degraded measurements, and online
LiDAR–IMU spatiotemporal calibration refines temporal offsets and rotational extrinsics during operation. We extend the M3DGR
benchmark with simulation trajectories and evaluate more than 60 open-source SLAM systems on M3DGR, M2DGR-Plus, KAIST, GrandTour, and MARS-LVIG. The results show competitive accuracy across wheeled, legged, and aerial platforms under long-duration and high-speed operation, degradation, and calibration perturbation, improving localization availability for road-level autonomy, campus and warehouse mobility, and low-altitude aerial inspection. To benefit the industrial and academic community, we will release source code and
datasets upon paper acceptance.
Read the full abstract ▾
All authors Yihong Tian, Junjie Zhang, Liuyang Li, Deteng Zhang, Yunfei Zuo, Jie Yin
Robotics
FleetAgent: Teleoperation Assistant for Autonomous Fleets via Vectorized V2N Messages
Juntong Peng, Qi Chen, Deyuan Qu et al. (+3 more)
Large-scale autonomous fleets rely on teleoperation to resolve rare failures, yet streaming raw sensor data from many vehicles is costly, and remote operators can only monitor a limited number of vehicles at a time. We introduce FleetAgent, a cloud-hosted
multimodal large
language model (MLLM) assistant that consumes compact vectorized vehicle-to-network (V2N) messages, such as map elements, detected objects, and the ego planned path. It provides a structured natural-
language response (including narration, explanation, and evaluation of the plan and scene), along with an intervention urgency score for operator prioritization. To make structured messages compatible with
token-based MLLMs, we propose VecFormer, a vector-to-
embedding interface with differentiable top-K context selection that bounds context length and GPU KV-cache growth, enabling more efficient batch processing, which is important under the context of cloud-hosted large-scale fleet management. We also construct VecEval, a nuScenes-derived
dataset with paired human and synthetic imperfect plans and human-verified
language labels, to facilitate the training and evaluation of our proposed system. Our proposed system can reduce uplink payload by up to 625 times compared with raw images and reduce KV-cache
memory by 16.54 times compared with original text descriptions. On VecEval, FleetAgent improves Lingo-Judge score by 16.8% and reduces intervention failure rate by 19.9%, compared with Qwen2.5-VL-7B using
language descriptions. These results demonstrate that FleetAgent can utilize compact structured V2N messaging to enable efficient, explainable teleoperation monitoring for autonomous fleets.
Read the full abstract ▾
All authors Juntong Peng, Qi Chen, Deyuan Qu, Takayuki Shimizu, Yaobin Chen, Ziran Wang
Robotics
A UAV-Mounted Sensor Network for Close-Range Inspection of Wind Turbine Rotor Blades
Jost Wittmann, Ahmed Abdullahi Hassan, Nils Kömpe et al. (+1 more)
Inspection of offshore wind turbine rotor blades is critical for predictive maintenance to maximise efficiency and extend operational lifetime. However, it remains a challenging task due to remote locations, large structural dimensions, and the limitations of current
UAV-compatible sensor systems. While existing approaches can detect certain types of surface anomalies, reliable classification of defect types often remains a manual and error-prone process. This paper presents the design of a
UAV-mounted
multimodal sensor network combining an industrial RGB camera, a passive thermal infrared camera, and an in-house developed 3D scanner. All sensors are co-calibrated into a common coordinate frame, enabling spatial superimposition of
geometric, colour, and thermal data. The system is designed to operate at close range, addressing three fundamental sensing challenges: platform motion, large field of view, and millimetre-level measurement accuracy. Preliminary laboratory results demonstrate synchronised multi-sensor acquisition and initial point cloud reconstructions, forming the basis for future airborne inspection trials.
Read the full abstract ▾
All authors Jost Wittmann, Ahmed Abdullahi Hassan, Nils Kömpe, Andreas Nüchter
Robotics
A scalar per patch from pre-trained ViTs enables fast moving navigation in the real world
Steeven Janny, Leonid Antsfeld and Christian Wolf
Trained policies for real-world robotics rely on computer
vision components, typically in the form of pre-trained visual encoders. These encoders are an essential component and it has been shown that their power does not emerge from training on robotics downstream losses alone. Pre-training with auxiliary losses in the form of computer-
vision pre-text tasks is a defining factor and heavily conditions
agent performance in robotics tasks. In this unprecedented large-scale study, we ran 966
navigation episodes of static point goal
navigation in a real-world building for 24km and asked which components really matter for the computer
vision aspects of robotics: we evaluate state-of-the art visual encoders in realistic conditions. We explore the usefulness of heterogeneous multi-teacher
distillation leading to encoders with multiple different and complementary skills. We investigate how much information from these encoders is necessary for robotics by bottlenecking them in a principled and spatially useful way and we show that this leads to the emergence of interpretable features linked to affordances. We also argue that training policies on RGB data alone does not lead to an optimal usage of visual features and show this by finetuning policies pre-trained on privileged information. All in all, we paint a more complete picture of what aspects of computer
vision are relevant for real-world
navigation.
Read the full abstract ▾
Robotics · Cross-listed from eess.SY
Discrete Geometric Modeling and Extended State Estimation of Continuum Robots
Maximilian Herrmann, Leander Pfeiffer and Paul Kotyczka
In this paper, we present a fully discrete approach for the accurate and numerically efficient dynamical modeling and state estimation of continuum
robots. The model is based on geometrically exact beams in a minimal, strain-based formulation and derived in the framework of Lie group variational integrators, allowing to preserve important
geometric properties that we exploit to achieve high accuracy and numerical efficiency. We then propose a disturbance observer based on an extended Kalman filter formulation that reliably estimates system states as well as model uncertainties and external disturbances. Experiments on a real system validate the accuracy and efficiency of the proposed model and observer.
Read the full abstract ▾
Robotics
Remember what you did?: Learning Behavioral Memories for Partially Observable Object Manipulation
Kuancheng Wang, Seungho Yeom, Jinglin Cao et al. (+3 more)
Long horizon, contact-rich manipulation is inherently partially observable. This is as a single visual observation rarely captures a
robot's full action context, including prior attempts, interactions, or progress. Consequently, standard visuomotor policies or
vision-
language-action models are prone to struggle in such tasks due to a lack of
memory. To address this, we introduce Compressed Action
Memory Policy (CAMP) based on the insight that a
robot's own action history serves as a highly informative, self-supervised signal, enabling the policy to learn a robust, compact history representation. In our approach, we train a
memory module to maintain a compressed representation of past actions, forcing it to encode a latent behavioral
memory of all the
robot's past interactions that can then be used to better contextualize future actions. This allows our approach to implicitly track generalized task progress and learn from failed attempts without any additional supervision, or external oversight. We evaluate CAMP across four real-
robot setups and two novel simulation
benchmarks:
Memory-T-Bench and
Memory-Manip-Bench. By demonstrating substantial gains over state-of-the-art baselines, CAMP is, to our knowledge, the first policy to demonstrate substantial success on contact-rich partially observable manipulation tasks purely through learned
memory.
Read the full abstract ▾
All authors Kuancheng Wang, Seungho Yeom, Jinglin Cao, Yuheng Zhi, Nikhil Shinde, Michael Yip