Robotics · Cross-listed from math.DS
Spectral Kernel Dynamics for Planetary Surface Graphs: Distinction Dynamics and Topological Conservation
Jnaneshwar Das
The spectral kernel field equation R[k] = T[k] lacks a conservation-law analog. We prove (i) the fixed-point flow is strictly volume-expanding (tr DF > 0), precluding automatic conservation, and (ii) the conservation deficit per mode equals the Hessian stability margin exactly: D_m = -Delta'. Closing the deficit requires a scene-side compensating contribution, which we formalise as the distinction dynamics equation dc/dt = G[c, h_t], with MaxCal-optimal realisation G_opt. On fixed-
topology 3D surface graphs we derive a conditional
topology-preserving compression
theorem: retaining k >= beta_0 + beta_1 modes (under a spectral-ordering assumption) preserves all Betti-number charges; we include a worked short-cycle counterexample (figure-eight) calibrating when the assumption fails. A triple necessary spectral diagnostic – Fiedler-mode concentration, elevated curl energy, anomalous beta_1 – is derived for planetary drainage networks at O(N) cost. Two internal real-data sequences serve as preliminary consistency checks; full
benchmarks and adaptive-
topology extensions are deferred.
Read the full abstract ▾
Robotics
LiDAR-based Crowd Navigation with Visible Edge Group Representation
Allan Wang and Aaron Steinfeld
Robot navigation in crowded pedestrian environments is a well-known challenge and we explore the practical deployment of group-based representations in this setting. Pedestrian groups have been empirically shown to enable a mobile
robot's
navigation behavior to be safer and more social. However, existing approaches either explored groups only in limited scenarios with no high-density crowds or depended on external detection modules to track individuals, which are prone to noise and errors due to occlusions in crowds. We show that group prediction accuracy affects
navigation performance only marginally in crowded environments. Based on this observation, we propose the visible edge-based group representation. We additionally demonstrate via simulation experiments that our
navigation framework, integrated with the simplified group representation, performs comparatively in terms of safety and socialness in dense crowds, while achieving faster computation speed. Finally, we deploy our
navigation framework on a real
robot to explore the benefits of practically deploying group-based representations in the real world.
Read the full abstract ▾
Robotics
Autonomous Vehicle Collision Avoidance With Racing Parameterized Deep Reinforcement Learning
Shathushan Sivashangaran, Vihaan Dutta, Apoorva Khairnar et al. (+2 more)
Road traffic accidents are a leading cause of fatalities worldwide. In the US, human error causes 94% of crashes, resulting in excess of 7,000 pedestrian fatalities and $500 billion in costs annually. Autonomous Vehicles (AVs) with emergency collision avoidance systems that operate at the limits of vehicle dynamics at a high frequency, a dual constraint of nonlinear kinodynamic accuracy and computational efficiency, further enhance safety benefits during adverse weather and cybersecurity breaches, and to evade dangerous human driving when AVs and human drivers share roads. This paper parameterizes a Deep Reinforcement Learning (DRL) collision avoidance policy Out-Of-Distribution (OOD) utilizing race car overtaking, without explicit
geometric mimicry reference trajectory guidance, in simulation, with a physics-informed, simulator exploit-aware reward to encode nonlinear vehicle kinodynamics. Two policies are evaluated, a default uni-direction and a reversed heading variant that navigates in the opposite direction to other cars, which both consistently outperform a Model Predictive Control and Artificial Potential Function (MPC-APF) baseline, with zero-shot transfer to proportionally scaled hardware, across three intersection collision scenarios, at 31x fewer Floating Point Operations (FLOPS) and 64x lower inference latency. The reversed heading policy outperforms the default racing overtaking policy in head-to-head collisions by 30% and the baseline by 50%, and matches the former in side collisions, where both DRL policies evade 10% greater than numerical optimal control.
Read the full abstract ▾
All authors Shathushan Sivashangaran, Vihaan Dutta, Apoorva Khairnar, Sepideh Gohari, Azim Eskandarian
Robotics
Rewind-IL: Online Failure Detection and State Respawning for Imitation Learning
Gehan Zheng, Sanjay Seenivasan, Matthew Johnson-Roberson et al. (+1 more)
Imitation learning has enabled
robots to acquire complex visuomotor manipulation skills from demonstrations, but deployment failures remain a major obstacle, especially for long-horizon action-chunked policies. Once execution drifts off the demonstration
manifold, these policies often continue producing locally plausible actions without recovering from the failure. Existing runtime monitors either require failure data, over-trigger under benign feature drift, or stop at failure detection without providing a recovery mechanism. We present Rewind-IL, a training-free online safeguard framework for generative action-chunked imitation policies. Rewind-IL combines a zero-shot failure detector based on Temporal Inter-chunk Discrepancy Estimate (TIDE), calibrated with split conformal prediction, with a state-respawning mechanism that returns the
robot to a semantically verified safe intermediate state. Offline, a
vision-
language model identifies recovery checkpoints in demonstrations, and the frozen policy encoder is used to construct a compact checkpoint feature database. Online, Rewind-IL monitors self-consistency in overlapping action chunks, tracks similarity to the checkpoint library, and, upon failure, rewinds execution to the latest verified safe state before restarting inference from a clean policy state. Experiments on real-world and simulated long-horizon manipulation tasks, including transfer to flow-matching action-chunked policies, demonstrate that policy-internal consistency coupled with semantically grounded respawning offers a practical route to improved reliability in imitation learning. Supplemental materials are available at https://sjay05.github.io/rewind-il
Read the full abstract ▾
All authors Gehan Zheng, Sanjay Seenivasan, Matthew Johnson-Roberson, Weiming Zhi
Robotics
SentryVLA: Uncertainty-Guided Action Selection and State Monitoring for Vision-Language-Action Robotic Control
Zongyao Lyu, Lingling Chen and William J. Beksi
Vision-
language-action (VLA) models have emerged as generalist robotic controllers that map visual observations and
language instructions to continuous actions. However, pretrained VLAs typically provide limited information about the reliability of their generated actions, making deployment challenging when execution failures must be anticipated. We address this problem by introducing SentryVLA, a lightweight runtime wrapper for frozen VLA policies. At the action level, SentryVLA uses conformalized
quantile regression to estimate a calibrated upper bound on the deviation between sampled VLA actions and expert-like behavior under the calibration distribution. The predicted deviation is then used to rank multiple
stochastic candidates for action selection. At the execution-monitoring level, SentryVLA applies a Mahalanobis-distance-based anomaly score to the contextualized latent execution state extracted before final action prediction, providing a complementary signal for detecting abnormal execution states during closed-loop control. The framework does not require retraining or architectural modification of the underlying VLA policy. Experiments in simulation and on a physical
robot show that SentryVLA improves action selection and provides useful runtime warning signals for abnormal execution states. These findings demonstrate the practical value of post-hoc reliability scoring for improving the runtime behavior of pretrained VLA policies. Project webpage: https://robotic-
vision-lab.github.io/sentryvla/.
Read the full abstract ▾
Robotics
Diffusion-Based Optimization for Accelerated Convergence of Redundant Dual-Arm Minimum Time Problems
Jushan Chen, Jonathan Fried and Santiago Paternain
We present a framework leveraging a novel variant of the model-based
diffusion algorithm to minimize the time required for a redundant dual-arm
robot configuration to follow a desired relative Cartesian path. Our prior work proposed a bi-level optimization approach for the dual-arm problem, where we derived the analytical solution to the lower-level convex sub-problem and solved the high-level nonconvex problem using a primal-dual approach. However, the gradient-based nature leads to a large computation overhead, and it prohibits directly imposing an $L_{\infty}$ Cartesian error constraint along the joint trajectory due to the sparsity of the gradient. In this work, we propose a
diffusion-based framework that relies on probabilistic sampling to tackle the aforementioned challenges in the nonconvex high-level problem, leading to a 35x reduction in the runtime and 34% less Cartesian error compared to our prior work.
Read the full abstract ▾
Robotics
Emergency Stopping for Liquid-manipulating Robots
Samuli Hynninen and Ville Kyrki
Manipulating open
liquid containers is challenging because
liquids are highly sensitive to vessel accelerations and jerks. Although spill-free
liquid manipulation has been widely studied, emergency stopping under unexpected hazards has received little
attention, despite the fact that abrupt braking may cause hazardous spills. This letter presents an emergency stop system for
robots manipulating
liquids in open containers. We formulate emergency stopping as an optimal control problem and solve it in a model predictive control framework to generate time-optimal, spill-free stopping trajectories. The method operates as a plug-and-play safety layer on top of existing slosh-free motion planning methods, enabling immediate reaction to detected hazards while accounting for nonlinear
liquid dynamics. We demonstrate, through simulation and on a 7-DoF Franka Emika Panda
robot, that the proposed approach achieves fast emergency stopping without spilling.
Read the full abstract ▾
Robotics
Human Cognition in Machines: A Unified Perspective of World Models
Timothy Rupprecht, Pu Zhao, Amir Taherin et al. (+20 more)
This report of world models distinguishes prior works by the cognitive functions they innovate. Many works claim an almost human-like cognitive capability in their world models. To evaluate these claims requires a proper grounding in first principles from human and machine cognition theory. In moving towards human-like world models we present a conceptual unified framework for world models that fully incorporates all the cognitive functions (i.e.,
memory, perception,
language,
reasoning, imagining, motivation, and metacognition) and identify gaps in existing research as a guide for future states of the art. In particular, we find that motivation (especially intrinsic motivation) and metacognition remain drastically under-researched, and we propose concrete directions to address these gaps informed by active inference and global workspace theory. We also introduce epistemic world models, a new category encompassing
agent frameworks for scientific discovery that operate over structured knowledge. Our taxonomy, applied to
video,
embodied, and epistemic world models, suggests research directions where prior taxonomies have not.
Read the full abstract ▾
All authors Timothy Rupprecht, Pu Zhao, Amir Taherin, Arash Akbari, Arman Akbari, Yumei He, Tooba Imtiaz, Sean Duffy, Juyi Lin, Yixiao Chen, Rahul Chowdhury, Enfu Nan, Yixin Shen, Yifan Cao, Haochen Zeng, Weiwei Chen, Geng Yuan, Jennifer Dy, Sarah Ostadabbas, Xuan Zhang, David Kaeli, Edmund Yeh, Yanzhi Wang
Robotics · Cross-listed from cs.CV
FineCog-Nav: Integrating Fine-grained Cognitive Modules for Zero-shot Multimodal UAV Navigation
Dian Shao, Zhengzheng Xu, Peiyang Wang et al. (+4 more)
UAV vision-
language navigation (VLN) requires an
agent to navigate complex 3D environments from an
egocentric perspective while following ambiguous multi-step instructions over long horizons. Existing zero-shot methods remain limited, as they often rely on large base models, generic
prompts, and loosely coordinated modules. In this work, we propose FineCog-Nav, a top-down framework inspired by human cognition that organizes
navigation into fine-grained modules for
language processing, perception,
attention,
memory, imagination,
reasoning, and decision-making. Each module is driven by a moderate-sized foundation model with role-specific
prompts and structured input-output protocols, enabling effective collaboration and improved interpretability. To support fine-grained evaluation, we construct AerialVLN-Fine, a curated
benchmark of 300 trajectories derived from AerialVLN, with sentence-level instruction-trajectory
alignment and refined instructions containing explicit visual endpoints and landmark references. Experiments show that FineCog-Nav consistently outperforms zero-shot baselines in instruction adherence, long-horizon planning, and generalization to unseen environments. These results suggest the effectiveness of fine-grained cognitive modularization for zero-shot aerial
navigation. Project page: https://smartdianlab.github.io/projects-FineCogNav.
Read the full abstract ▾
All authors Dian Shao, Zhengzheng Xu, Peiyang Wang, Like Liu, Yule Wang, Jieqi Shi, Jing Huo
Robotics
Semantic Area Graph Reasoning for Multi-Robot Language-Guided Search
Ruiyang Wang, Hao-Lun Hsu, Jiwoo Kim et al. (+1 more)
Coordinating multi-
robot systems (MRS) to search in unknown environments is particularly challenging for tasks that require semantic
reasoning beyond
geometric exploration. Classical coordination strategies rely on frontier coverage or information gain and cannot incorporate high-level task intent, such as searching for objects associated with specific room types. We propose Semantic Area Graph
Reasoning (SAGR), a hierarchical framework that enables Large
Language Models (LLMs) to coordinate multi-
robot exploration and semantic search through a structured semantic-topological abstraction of the environment. SAGR incrementally constructs a semantic area graph from a semantic occupancy map, encoding room instances, connectivity, frontier availability, and
robot states into a compact task-relevant representation for LLM
reasoning. The LLM performs high-level semantic room assignment based on spatial structure and task context, while deterministic frontier planning and local
navigation handle
geometric execution within assigned rooms. Experiments on the Habitat-Matterport3D
dataset across 100 scenarios show that SAGR remains competitive with state-of-the-art exploration methods while consistently improving semantic target search efficiency, with up to 18.8% in large environments. These results highlight the value of structured semantic abstractions as an effective interface between LLM-based
reasoning and multi-
robot coordination in complex indoor environments.
Read the full abstract ▾
All authors Ruiyang Wang, Hao-Lun Hsu, Jiwoo Kim, Miroslav Pajic
Robotics
DENALI: A Dataset Enabling Non-Line-of-Sight Spatial Reasoning with Low-Cost LiDARs
Nikhil Behari, Diego Rivero, Luke Apostolides et al. (+3 more)
Consumer
LiDARs in mobile devices and
robots typically output a single depth value per pixel. Yet internally, they record full time-resolved histograms containing direct and multi-bounce light returns; these multi-bounce returns encode rich non-line-of-sight (NLOS) cues that can enable perception of hidden objects in a scene. However, severe hardware limitations of consumer
LiDARs make NLOS reconstruction with conventional methods difficult. In this work, we motivate a complementary direction: enabling NLOS perception with low-cost
LiDARs through data-driven inference. We present DENALI, the first large-scale real-world
dataset of space-time histograms from low-cost
LiDARs capturing hidden objects. We capture time-resolved
LiDAR histograms for 72,000 hidden-object scenes across diverse object shapes, positions, lighting conditions, and spatial resolutions. Using our
dataset, we show that consumer
LiDARs can enable accurate, data-driven NLOS perception. We further identify key scene and modeling factors that limit performance, as well as simulation-fidelity gaps that hinder current sim-to-real transfer, motivating future work toward scalable NLOS
vision with consumer
LiDARs.
Read the full abstract ▾
All authors Nikhil Behari, Diego Rivero, Luke Apostolides, Suman Ghosh, Paul Pu Liang, Ramesh Raskar
Robotics · Cross-listed from cs.CV
SENSE: Stereo OpEN Vocabulary SEmantic Segmentation
Thomas Campagnolo, Ezio Malis, Philippe Martinet et al. (+1 more)
Open-vocabulary semantic
segmentation enables models to segment objects or image regions beyond fixed class sets, offering flexibility in dynamic environments. However, existing methods often rely on single-view images and struggle with spatial precision, especially under occlusions and near object boundaries. We propose SENSE, the first work on Stereo OpEN Vocabulary SEmantic
Segmentation, which leverages stereo
vision and
vision-
language models to enhance open-vocabulary semantic
segmentation. By incorporating stereo image pairs, we introduce
geometric cues that improve spatial
reasoning and
segmentation accuracy. Trained on the PhraseStereo
dataset, our approach achieves strong performance in phrase-grounded tasks and demonstrates generalization in zero-shot settings. On PhraseStereo, we show a +2.9% improvement in Average Precision over the baseline method and +0.76% over the best competing method. SENSE also provides a relative improvement of +3.5% mIoU on Cityscapes and +18% on KITTI compared to the baseline work. By jointly
reasoning over semantics and geometry, SENSE supports accurate scene understanding from natural
language, essential for autonomous
robots and Intelligent Transportation Systems.
Read the full abstract ▾
All authors Thomas Campagnolo, Ezio Malis, Philippe Martinet, Gaétan Bahl
Robotics
VADF: Vision-Adaptive Diffusion Policy Framework for Efficient Robotic Manipulation
Xinglei Yu, Zhenyang Liu, Shufeng Nan et al. (+2 more)
Diffusion policies are becoming mainstream in robotic manipulation but suffer from hard negative class imbalance due to uniform sampling and lack of sample difficulty awareness, leading to slow training convergence and frequent inference timeout failures. We propose VADF (
Vision-Adaptive
Diffusion Policy Framework), a
vision-driven dual-adaptive framework that significantly reduces convergence steps and achieves early success in inference, with model-agnostic design enabling seamless integration into any
diffusion policy architecture. During training, we introduce Adaptive Loss Network (ALN), a lightweight MLP-based loss predictor that quantifies per-step sample difficulty in real time. Guided by hard negative mining, it performs weighted sampling to prioritize high-loss regions, enabling adaptive weight updates and faster convergence. In inference, we design the Hierarchical
Vision Task Segmenter (HVTS), which decomposes high-level task instructions into multi-stage low-level sub-instructions based on visual input. It adaptively segments action sequences into simple and complex subtasks by assigning shorter noise schedules with longer direct execution sequences to simple actions, and longer noise steps with shorter execution sequences to complex ones, thereby dramatically reducing computational overhead and significantly improving the early success rate.
Read the full abstract ▾
All authors Xinglei Yu, Zhenyang Liu, Shufeng Nan, Simo Wu, Yanwei Fu
Robotics
A Reconfigurable Pneumatic Joint Enabling Localized Selective Stiffening and Shape Locking in Vine-Inspired Robots
Ayodele James Oyejide, Ustaz A. Yaqub, Samir Erturk et al. (+2 more)
Vine-inspired
robots achieve large workspace coverage through tip eversion, enabling safe
navigation in confined and cluttered environments. However, their deployment in free space is fundamentally limited by low axial stiffness, poor load-bearing capacity, and the inability to retain shape during and after steering. In this work, we propose a reconfigurable pneumatic joint (RPJ) architecture that introduces discrete, pressure-tunable stiffness along the
robot body without compromising continuous growth. Each RPJ module comprises symmetrically distributed pneumatic chambers that locally increase bending stiffness when pressurized, enabling decoupling between global compliance and localized
rigidity. We integrate the RPJs into a soft growing
robot with tendon-driven steering and develop a compact base station for mid-air eversion. System characterization and experimental validation demonstrate moderate pressure requirements for eversion, as well as comparable localized stiffening and steering performance to layer-jamming mechanisms. Demonstrations further show that the proposed
robot achieves improved shape retention during bending, reduced gravitational deflection under load, cascading retraction, and reliable payload transport up to 202 g in free space. The RPJ mechanism establishes a practical pathway toward structurally adaptive vine
robots for manipulation-oriented tasks such as object sorting and adaptive exploration in unconstrained environments.
Read the full abstract ▾
All authors Ayodele James Oyejide, Ustaz A. Yaqub, Samir Erturk, Eray A. Baran, Fabio Stroppa
Robotics
Robust Fleet Sizing for Multi-UAV Inspection Missions under Synchronized Replacement Demand
Vishal Ramesh and Antony Thomas
Multi-
UAV inspection missions require spare drones to replace active drones during recharging cycles. Existing fleet-sizing approaches often assume steady-state operating conditions that do not apply to finite-horizon missions, or they treat replacement requests as statistically independent events. The latter provides per-request blocking guarantees that fail to translate to mission-level reliability when demands cluster. This paper identifies a structural failure mode where efficient routing assigns similar workloads to each
UAV, leading to synchronized battery depletion and replacement bursts that exhaust the spare pool even when average capacity is sufficient. We derive a closed-form sufficient fleet-sizing rule, k = m(ceil(R) + 1), where m is the number of active
UAVs and R is the recovery-to-active time ratio. This additive buffer of m spares absorbs worst-case synchronized demand at recovery-cycle boundaries and ensures mission-level reliability even when all
UAVs deplete simultaneously. Monte Carlo validation across five scenarios (m in [2, 10], R in [0.87, 3.39], 1000 trials each) shows that Erlang-B sizing with a per-request blocking target epsilon = 0.01 drops to 69.9% mission success at R = 3.39, with 95% of spare exhaustion events concentrated in the top-decile 5-minute demand windows. In contrast, the proposed rule maintains 99.8% success (Wilson 95% lower bound 99.3%) across all tested conditions, including wind variability up to CV = 0.30, while requiring only four additional drones in the most demanding scenario.
Read the full abstract ▾
Robotics
DTEA: A Dual-Topology Elastic Actuator Enabling Real-Time Switching Between Series and Parallel Compliance
Vishal Ramesh, Aman Singh and Shishir Kolathaya
Series and parallel elastic actuators offer complementary but mutually exclusive advantages, yet no existing actuator enables real-time transition between these
topologies during operation. This paper presents a novel actuator design called the Dual-
Topology Elastic Actuator (DTEA), which enables dynamic switching between SEA and PEA
topologies during operation. A proof-of-concept prototype of the DTEA is developed to demonstrate the feasibility of the
topology-switching mechanism. Experiments are conducted to evaluate the robustness and timing of the switching mechanism under operational conditions. The actuator successfully performed 324
topology-switching cycles under load without damage, demonstrating the robustness of the mechanism. The measured switching time between SEA and PEA modes is under 33.33 ms. Additional experiments are conducted to characterize the static stiffness and disturbance rejection performance in both SEA and PEA modes. Static stiffness tests show that the PEA mode is 1.53x stiffer than the SEA mode, with KSEA = 5.57 +/- 0.02 Nm/rad and KPEA = 8.54 +/- 0.02 Nm/rad. Disturbance rejection experiments show that the mean peak deflection in SEA mode is 2.26x larger than in PEA mode (5.2 deg vs. 2.3 deg), while the mean settling time is 3.45x longer (1380 ms vs. 400 ms). The observed behaviors are consistent with the known characteristics of conventional SEA and PEA actuators, validating the functionality of both modes in the DTEA actuator.
Read the full abstract ▾
Robotics
Environment-Adaptive Solid-State LiDAR-Inertial Odometry
Zhi Zhang, Chalermchon Satirapod, Bingtao Ma et al. (+1 more)
Solid-state
LiDAR-inertial SLAM has attracted significant
attention due to its advantages in speed and robustness. However, achieving accurate mapping in extreme environments remains challenging due to severe
geometric degeneracy and unreliable observations, which often lead to ill-conditioned optimization and map inconsistencies. To address these challenges, we propose an environment-adaptive solid-state
LiDAR-inertial odometry that integrates local normal-vector constraints with degeneracy-aware map maintenance to enhance localization accuracy. Specifically, we introduce local normal-vector constraints to improve the stability of state estimation, effectively suppressing localization drift in degenerate scenarios. Furthermore, we design a degeneration-guided map update strategy to improve map precision. Benefiting from the refined map representation, localization accuracy is further enhanced in subsequent estimation. Experimental results demonstrate that the proposed method achieves superior mapping accuracy and robustness in extreme and perceptually degraded environments, with an average RMSE reduction of up to 12.8% compared to the baseline method.
Read the full abstract ▾
All authors Zhi Zhang, Chalermchon Satirapod, Bingtao Ma, Changjun Gu
Robotics
Limits of Lamarckian Evolution Under Pressure of Morphological Novelty
Jed R Muff, Karine Miras and A.E. Eiben
Lamarckian inheritance has been shown to be a powerful accelerator in systems where the joint
evolution of
robot morphologies and controllers is enhanced with individual learning. Its defining advantage lies in the offspring inheriting controllers learned by their parents. The efficacy of this option, however, relies on morphological similarity between parent and offspring. In this study, we examine how Lamarckian inheritance performs when the search process is driven toward high morphological variance, potentially straining the requirement for parent-offspring similarity. Using a system of modular
robots that can evolve and learn to solve a locomotion task, we compare Darwinian and Lamarckian
evolution to determine how they respond to shifting from pure task-based selection to a multi-objective pressure that also rewards morphological novelty. Our results confirm that Lamarckian
evolution outperforms Darwinian
evolution when optimizing task-performance alone. However, introducing selection pressure for morphological diversity causes a substantial performance drop, which is much greater in the Lamarckian system. Further analyses show that promoting diversity reduces parent-offspring similarity, which in turn reduces the benefits of inheriting controllers learned by parents. These results reveal the limits of Lamarckian
evolution by exposing a fundamental trade-off between inheritance-based exploitation and diversity-driven exploration.
Read the full abstract ▾
Robotics · Cross-listed from cs.CV
Continual Hand-Eye Calibration for Open-world Robotic Manipulation
Fazeng Li, Gan Sun, Chenxi Liu et al. (+3 more)
Hand-eye calibration through visual localization is a critical capability for robotic manipulation in open-world environments. However, most deep learning-based calibration models suffer from catastrophic forgetting when adapting into unseen data amongst open-world scene changes, while simple rehearsal-based continual learning strategy cannot well mitigate this issue. To overcome this challenge, we propose a continual hand-eye calibration framework, enabling
robots to adapt to sequentially encountered open-world manipulation scenes through spatially replay strategy and structure-preserving
distillation. Specifically, a Spatial-Aware Replay Strategy (SARS) constructs a geometrically uniform replay buffer that ensures comprehensive coverage of each scene pose space, replacing redundant adjacent frames with maximally informative viewpoints. Meanwhile, a Structure-Preserving Dual
Distillation (SPDD) is proposed to decompose localization knowledge into coarse scene layout and fine pose precision, and distills them separately to alleviate both types of forgetting during continual adaptation. As a new manipulation scene arrives, SARS provides geometrically representative replay samples from all prior scenes, and SPDD applies structured
distillation on these samples to retain previously learned knowledge. After training on the new scene, SARS incorporates selected samples from the new scene into the replay buffer for future rehearsal, allowing the model to continuously accumulate multi-scene calibration capability. Experiments on multiple public
datasets show significant anti scene forgetting performance, maintaining accuracy on past scenes while preserving adaptation to new scenes, confirming the effectiveness of the framework.
Read the full abstract ▾
All authors Fazeng Li, Gan Sun, Chenxi Liu, Yao He, Wei Cong, Yang Cong
Robotics
From Seeing to Simulating: Generative High-Fidelity Simulation with Digital Cousins for Generalizable Robot Learning and Evaluation
Jasper Lu, Zhenhao Shen, Yuanfei Wang et al. (+8 more)
Learning robust
robot policies in real-world environments requires diverse data augmentation, yet scaling real-world data collection is costly due to the need for acquiring physical assets and reconfiguring environments. Therefore, augmenting real-world scenes into simulation has become a practical augmentation for efficient learning and evaluation. We present a generative framework that establishes a generative real-to-sim mapping from real-world panoramas to high-fidelity simulation scenes, and further synthesize diverse cousin scenes via semantic and
geometric editing. Combined with high-quality physics engines and realistic assets, the generated scenes support interactive manipulation tasks. Additionally, we incorporate multi-room stitching to construct consistent large-scale environments for long-horizon
navigation across complex layouts. Experiments demonstrate a strong sim-to-real correlation validating our platform's fidelity, and show that extensively scaling up data generation leads to significantly better generalization to unseen scene and object variations, demonstrating the effectiveness of Digital Cousins for generalizable
robot learning and evaluation.
Read the full abstract ▾
All authors Jasper Lu, Zhenhao Shen, Yuanfei Wang, Shugao Liu, Shengqiang Xu, Shawn Xie, Jingkai Xu, Feng Jiang, Jade Yang, Chen Xie, Ruihai Wu
Robotics
Fuzzy Logic Theory-based Adaptive Reward Shaping for Robust Reinforcement Learning (FARS)
Hürkan Şahin, Van Huyen Dang, Erdi Sayar et al. (+2 more)
Reinforcement learning (RL) often struggles in real-world tasks with high-dimensional state spaces and long horizons, where sparse or fixed rewards severely slow down exploration and cause
agents to get trapped in local optima. This paper presents a fuzzy logic based reward shaping method that integrates human intuition into RL reward design. By encoding expert knowledge into adaptive and interpreable terms, fuzzy rules promote stable learning and reduce sensitivity to hyperparameters. The proposed method leverages these properties to adapt reward contributions based on the
agent state, enabling smoother transitions between fast motion and precise control in challenging
navigation tasks. Extensive simulation results on autonomous drone racing
benchmarks show stable learning behavior and consistent task performance across scenarios of increasing difficulty. The proposed method achieves faster convergence and reduced performance variability across training seeds in more challenging environments, with success rates improving by up to approximately 5 percent compared to non fuzzy reward formulations.
Read the full abstract ▾
All authors Hürkan Şahin, Van Huyen Dang, Erdi Sayar, Alper Yegenoglu, Erdal Kayacan
Robotics · Cross-listed from cs.CV
PLAF: Pixel-wise Language-Aligned Feature Extraction for Efficient 3D Scene Understanding
Junjie Wen, Junlin He, Fei Ma et al. (+1 more)
Accurate open-vocabulary 3D scene understanding requires semantic representations that are both
language-aligned and spatially precise at the pixel level, while remaining scalable when lifted to 3D space. However, existing representations struggle to jointly satisfy these requirements, and densely propagating pixel-wise semantics to 3D often results in substantial redundancy, leading to inefficient storage and querying in large-scale scenes. To address these challenges, we present PLAF, a Pixel-wise
Language-Aligned Feature extraction framework that enables dense and accurate semantic
alignment in 2D without sacrificing open-vocabulary expressiveness. Building upon this representation, we further design an efficient semantic storage and querying scheme that significantly reduces redundancy across both 2D and 3D domains. Experimental results show that PLAF provides a strong semantic foundation for accurate and efficient open-vocabulary 3D scene understanding. The codes are publicly available at https://github.com/RockWenJJ/PLAF.
Read the full abstract ▾
All authors Junjie Wen, Junlin He, Fei Ma, Jinqiang Cui
Robotics
Long-Term Memory for VLA-based Agents in Open-World Task Execution
Xu Huang, Weixin Mao, Yinhao Li et al. (+2 more)
Vision-
Language-Action (VLA) models have demonstrated significant potential for
embodied decision-making; however, their application in complex chemical laboratory automation remains restricted by limited long-horizon
reasoning and the absence of persistent experience accumulation. Existing frameworks typically treat planning and execution as decoupled processes, often failing to consolidate successful strategies, which results in inefficient trial-and-error in multi-stage protocols. In this paper, we propose ChemBot, a dual-layer, closed-loop framework that integrates an autonomous AI
agent with a progress-aware VLA model (Skill-VLA) for hierarchical task decomposition and execution. ChemBot utilizes a dual-layer
memory architecture to consolidate successful trajectories into retrievable assets, while a Model Context Protocol (MCP) server facilitates efficient sub-
agent and tool orchestration. To address the inherent limitations of VLA models, we further implement a future-state-based asynchronous inference mechanism to mitigate trajectory discontinuities. Extensive experiments on collaborative
robots demonstrate that ChemBot achieves superior operational safety, precision, and task success rates compared to existing VLA baselines in complex, long-horizon chemical experimentation.
Read the full abstract ▾
All authors Xu Huang, Weixin Mao, Yinhao Li, Hua Chen, Jiabao Zhao
Robotics
Contact-Aware Planning and Control of Continuum Robots in Highly Constrained Environments
Aedan Mangan, Kehan Long, Ki Myung Brian Lee et al. (+3 more)
Continuum
robots are well suited for navigating confined and fragile environments, such as vascular or endoluminal anatomy, where contact with surrounding structures is often unavoidable. While controlled contact can assist motion, unfavorable contact can degrade controllability, induce kinematic singularities, or introduce safety risks. We present a contact-aware planning approach that evaluates contact quality, penalizing hazardous interactions, while permitting benign contact. The planner produces kinematically feasible trajectories and contact-aware Jacobians which can be used for closed-loop control in hardware experiments. We validate the approach by testing the integrated system (planning, control, and mechanical design) on anatomical models from patient scans. The planner generates effective plans for three common anatomical environments, and, in all hardware trials, the continuum
robot was able to reach the target while avoiding dangerous tip contact (100% success). Mean tracking errors were 1.9 +/- 0.5 mm, 1.2 +/- 0.1 mm, and 1.7 +/- 0.2 mm across the three different environments. Ablation studies showed that penalizing end-of-continuum-segment (ECS) contact improved manipulability and prevented hardware failures. Overall, this work enables reliable, contact-aware
navigation in highly constrained environments.
Read the full abstract ▾
All authors Aedan Mangan, Kehan Long, Ki Myung Brian Lee, Miheer Potdar, Nikolay Atanasov, Tania K. Morimoto
Robotics
Factor Graph-Based Continuum Robot State Estimation with Magnus Expansion Priors
Lorenzo Ticozzi, Sabyasachi Dash, Yangkun Liu et al. (+2 more)
Reconstructing the shape of continuum manipulators from sparse, noisy sensor data is a challenging task, owing to the infinite-dimensional nature of such systems. Existing approaches broadly trade off between parametric methods that yield compact state representations but lack probabilistic structure, and Cosserat rod inference on factor graphs, which provides principled uncertainty quantification at the cost of a state dimension that grows with the spatial discretization. This letter combines the strengths of both these approaches by estimating the coefficients of a compact
Geometric Variable Strain (GVS) parameterization within a factor graph framework. A novel kinematic factor, derived from the Magnus expansion of the strain field, encodes the closed-form rod geometry as a prior constraint linking the GVS strain coefficients to the backbone pose variables. The resulting formulation yields a compact state vector, is directly amenable to model-based control, and it retains the modularity, probabilistic treatment, and computational efficiency of factor graph inference. The proposed method is evaluated on both simulated and experimental data and compared against a Gaussian Process (GP) regression baseline, demonstrating increased accuracy in partial and sparse measurement scenarios while preserving computational efficiency.
Read the full abstract ▾
All authors Lorenzo Ticozzi, Sabyasachi Dash, Yangkun Liu, Patricio A. Vela, Panagiotis Tsiotras
Robotics
GaussianFlow SLAM: Monocular Gaussian Splatting SLAM Guided by GaussianFlow
Dong-Uk Seo, Jinwoo Jeon, Eungchang Mason Lee et al. (+1 more)
Gaussian splatting has recently gained traction as a compelling map representation for SLAM systems, enabling dense and photo-realistic scene modeling. However, its application to
monocular SLAM remains challenging due to the lack of reliable
geometric cues from
monocular input. Without
geometric supervision, mapping or tracking could fall in local-minima, resulting in structural degeneracies and inaccuracies. To address this challenge, we propose GaussianFlow SLAM, a
monocular 3DGS-SLAM that leverages optical flow as a geometry-aware cue to guide the optimization of both the scene structure and camera poses. By encouraging the projected motion of Gaussians, termed GaussianFlow, to align with the optical flow, our method introduces consistent structural cues to regularize both map reconstruction and pose estimation. Furthermore, we introduce normalized error-based densification and
pruning modules to refine inactive and unstable Gaussians, thereby contributing to improved map quality and pose accuracy. Experiments conducted on public
datasets demonstrate that our method achieves superior rendering quality and tracking accuracy compared with state-of-the-art
algorithms. The source code is available at: https://github.com/url-kaist/gaussianflow-slam.
Read the full abstract ▾
All authors Dong-Uk Seo, Jinwoo Jeon, Eungchang Mason Lee, Hyun Myung