Robotics
RAF-VLA: Representation Alignment with the Future for End-to-End Autonomous Driving
Dogun Kim, Yongjae Lee, Joonhee Lim et al. (+4 more)
Recent
Vision-
Language-Action (VLA) models for autonomous driving have incorporated world modeling by predicting future driving scenes alongside driving actions, demonstrating strong planning performance. Future driving scenes are utilized as dense supervision, encouraging the policy to learn rich internal representations useful for planning. However, these World-Modeling VLAs rely on explicit future generation to learn such representations, thereby introducing two key limitations: additional training burden and inference latency. To address these limitations, we propose RAF-VLA (Representation
Alignment with the Future), a VLA-based autonomous driving framework that shapes planning-relevant internal representations through direct guidance from future-frame representations. RAF-VLA employs Future-Aligned Supervised Fine-Tuning, in which a straightforward regularization aligns the policy's hidden states with future-frame representations obtained from a pretrained world encoder while learning driving actions. This simple
alignment allows RAF-VLA to avoid the training burden and inference latency associated with future generation. Extensive experiments on the NAVSIM
benchmark show that RAF-VLA achieves competitive planning performance against state-of-the-art VLA planners with substantially fewer training samples seen. Moreover, RAF-VLA incurs only 3.8% training overhead and a negligible 1 ms inference overhead.
Read the full abstract ▾
All authors Dogun Kim, Yongjae Lee, Joonhee Lim, Yeina Lee, Junhyeok Park, Moogeun Park, Dongsuk Kum
Robotics
Vision-Language Grounded Task-Context-Aware Imitation Learning for Robotic Disassembly
Jeon Ho Kang, Igal Tamarkin, Ethan Niu et al. (+2 more)
Real-world robotic disassembly requires long-horizon execution, where
robots must perform ordered sequences of manipulation tasks across multiple parts within a single scene. Multiple valid task goals and diverse assembly configurations make it difficult for imitation policies to infer the intended skill from raw observations alone, particularly when training data cannot cover the combinatorial diversity of real-world configurations and part geometries. We show that incorporating task context through
language alleviates these challenges by providing explicit structure for skill selection and associating
language-specified tasks with their corresponding manipulation targets in the visual scene. The proposed framework combines hierarchical task selection with task-context-aware imitation learning to ground
language instructions in spatial visual representations for robotic disassembly. The resulting framework generalizes across diverse connector geometries and assembly configurations without requiring explicit object annotations. Our method improves end-to-end task success by 35 percentage points over the baseline
diffusion policy and by 75 percentage points over the previous task-context-aware baseline.
Read the full abstract ▾
All authors Jeon Ho Kang, Igal Tamarkin, Ethan Niu, Ian Novales, Satyandra K. Gupta
Robotics
LEAP: Learning Emergent Active Perception for Quadruped Navigation
Ü. Bora Gökbakan, Stéphane Caron and Philippe Souères
Active perception allows autonomous
agents to select their viewpoints rather than passively process the viewpoints given to them, enabling them to target where to reduce uncertainty about their environment. Learned systems typically encourage this behavior with hand-designed proxy objectives, such as coverage or curiosity bonuses, that may conflict with the task. In this work, we propose a method to learn emergent active perception (LEAP) without augmentation of the task objective. We formulate the problem of goal-oriented navigation over hazardous terrains with goals that must be discovered visually. We then propose an architecture for navigation policies with active perception, and train them on a terrain curriculum where task pressure alone leads to the emergence of gaze control. Key to this emergence, LEAP works on a gaze-invariant representation that integrates depth images into egocentric belief maps. We validate its performance in held-out evaluation scenarios, where it achieves a 92.7% success rate, compared to 74.2% for scripted or 34.5% for passive perception, and comes within 4.6 points of a privileged oracle. We validate that LEAP navigation policies, unchanged, can be directly applied to steering quadrupedal locomotion policies in physics simulation.
Read the full abstract ▾
Robotics
Flexible-body Modeling, Kinematic Identification, and Assembly Accuracy of Overconstrained Spatial Linkages
Daniel Huczala, Michael Pieber, Johannes Gerstmayr et al. (+6 more)
Overconstrained rational single-loop linkages are efficient, compact, and low-cost custom mechanisms, yet their deployment in industrial settings is limited. In simulations, rigid body formulations fail due to redundant constraints. This study presents a flexible multibody modeling framework based on the floating frame of reference formulation, and delivers an overall accuracy analysis of assembled linkages prototypes. The approach is validated against 3D-printed PLA prototypes of a Bennett four-bar mechanism, including variants with intentional joint-axis misalignment, which theoretically, from the rigid body point of view, cannot be assembled. A supplementary contribution is delivered in the form of a kinematic parameter identification methodology suited for this type of mechanism with ill-conditioned Jacobian. The experimental and simulation results are compared and reveal that these overconstrained mechanisms exhibit a self-assembling tendency – structural compliance drives the assembly toward the ideal
geometric configuration, distributing constraint stress throughout the structure. Additional qualitative demonstrations using cardboard tubes and bamboo sticks as link building blocks confirm that functional mechanisms can be realized from low-cost, unconventional materials with limited manufacturing accuracy. The proposed modeling pipeline is fully algorithmic and enables design optimization in the future.
Read the full abstract ▾
All authors Daniel Huczala, Michael Pieber, Johannes Gerstmayr, Andreas Mair, Frederik Schulte, Silvia Glas, Tomas Postulka, Ales Vysocky, Martin Pfurner
Robotics
Predictive Varanus: Combining CSP Conformance Monitoring with Predictive LTL Runtime Verification
Angelo Ferrando, Matt Luckcuck and Pedro Ribeiro
Runtime Verification is well suited to autonomous and robotic systems because it checks the behaviour that is actually observed during execution. Its main limitation, however, is that it is usually reactive: the monitor detects a violation only after the system has already performed a bad event. This can be too late in domains where failures are costly or unsafe. In this paper we present PREDICTIVE VARANUS, a two-stage verification pipeline that combines VARANUS, a runtime verifier that uses models written in the process
algebra Communicating Sequential Processes (CSP), with predictive runtime verification for LTL. A CSP model is first used as a conformance gate over the observed event trace; the same model is then translated into a Buchi automaton that constrains the futures explored by a predictive LTL monitor. In this way, out-of-model behaviour is rejected immediately, while model-consistent prefixes can be classified as already guaranteeing satisfaction, already forcing violation, or still being inconclusive for the monitored temporal property. We formalise the combined monitor, explain its implementation, and illustrate the approach on a robotic rover for nuclear-store inspection. The case study shows how the combination of CSP validation and predictive LTL can provide earlier verdicts than standard runtime monitoring while reusing an existing design-time CSP model.
Read the full abstract ▾
Robotics
Modality-Autoregressive World-Action Models
Adam Hung, Bardienus P. Duisterhof, Deva Ramanan et al. (+1 more)
World-action models (WAMs) jointly model future observations and actions, typically predicting the future as RGB images. Other visual modalities such as depth, pretrained visual features, and point tracks can more efficiently capture
geometric, semantic, and motion features. However, how best to combine these modalities within WAMs remains an open question. We introduce ModAR, the first WAM to autoregressively denoise multiple future modalities before predicting actions. This allows each prediction to condition on previously generated modalities. We train from scratch to systematically study how training-data mixtures, predicted modalities, and WAM formulations affect performance. In our evaluations, WAMs benefit from predicting point tracks, DINO features, and depth maps, while additionally predicting future RGB does not provide a consistent benefit. We also find that ModAR's sequential generation outperforms existing WAM formulations, with the highest average success rate at all evaluated data scales. We also fine-tune the
video-model-initialized WAM Flex-$\pi$ on the same data; ModAR achieves a slightly higher observed average success rate (75% vs. 72%) while using approximately $20\times$ fewer training FLOPs and no pretraining. On three real-world bimanual tasks, ModAR outperforms baselines and improves with human
videos.
Read the full abstract ▾
All authors Adam Hung, Bardienus P. Duisterhof, Deva Ramanan, Jeffrey Ichnowski
Robotics · Cross-listed from cs.LG
ENCP: Episode-Normalized Conformal Prediction for Vision-and-Language Navigation
Vicky Feliren, A. Taufiq Asyhari and Muhamad Risqi U. Saputra
Uncertainty estimation for
Vision-
Language-Navigation (VLN) models is a critical task since it can help identify ambiguous and unreliable predictions, enabling
agents to make safer navigation decisions. As one of the most advanced uncertainty estimation frameworks, conformal prediction (CP) offers a promising approach for uncertainty estimation in VLN. However, given that VLN
agent requires a sequence of steps, standard calibration in conformal prediction fails to provide coverage guarantee it promises over a dependent, variable-length VLN episode. To this end, we propose Episode-Normalized Conformal Prediction (ENCP), which rescales a nonconformity score by the policy's residual confidence and calibrates one maximum score per episode. Under exchangeable calibration and test episodes, this construction covers the ground truth at every step with probability at least $1 - \alpha$, while allowing dependence among steps within an episode. Across four VLN policies and three nonconformity scores on R2R and REVERIE
dataset, ENCP meets all reported empirical step-coverage targets on the seen-to-unseen evaluation. These results demonstrate that ENCP can provide model-agnostic uncertainty estimates, which might be useful for determining when a VLN
agent should defer to a more capable predictor, including human assistance.
Read the full abstract ▾
Robotics
Dissecting Motion-Prior Regularization for Data-Scarce Robotic Insertion
Ning Hu, Shuai Li and Jindong Tan
This study asks whether training-time motion-prior regularization can improve insertion success when a
diffusion policy is learned from only 15 demonstrations. Minimum jerk discourages abrupt changes in predicted translational acceleration; speed-
curvature regularization instead couples movement speed to path geometry. These are candidate mechanisms for task completion, not safety guarantees. We compare the priors individually and jointly, neither prior, and generic smoothness, with 80 real-
robot trials per setting pooled over four recorded condition classes. Joint and minimum-jerk-only settings each achieved 70/80 successes (87.5%), versus 69/80 (86.3%) for speed-
curvature only, 66/80 (82.5%) for neither prior, and 67/80 (83.8%) for generic smoothness. Success rates and Wilson 95% confidence intervals are visualized for direct comparison. Joint regularization exceeded neither by 5.0 percentage points but provided no observed gain over minimum jerk alone. The results motivate minimum jerk as the simpler candidate for replication, without establishing synergy, biomechanical specificity, improved safety, or distribution-shift robustness.
Read the full abstract ▾
Robotics
Gaussian Processes for Modelling Spatial Fields with Robot Swarms
Guillermo Legarda Herranz, Gianpiero Francesca and Mauro Birattari
Robot swarms, by virtue of their decentralised architecture, are a natural tool for scalable, robust modelling of spatial fields, such as water temperature, wind velocity, or terrain elevation. However, existing methods rely on external positioning systems that allow each
robot to determine its own position in space. Here, we introduce location-unaware Gaussian process regression (LU-GPR) as a solution to the modelling of spatial fields in the absence of such positioning systems. LU-GPR allows each
robot to infer the posterior mean and variance of the field in space, while simultaneously agreeing on a common frame of reference with its peers, using only local sensing and communication. We propose an online
algorithm that allows each
robot to consistently infer local estimates as its local frame of reference converges to the common one. By means of a product of experts model, each
robot also combines the estimates of its peers with its own to obtain a global model. Our results show that LU-GPR scales well with the number of
robots and is robust to limited communication ranges. We also demonstrate how it can be used in real-world monitoring scenarios to estimate the flow of an evacuating crowd.
Read the full abstract ▾
Robotics
Hamilton-Jacobi Reachability for Hybrid Systems: Unified Goal-Driven Control with Safety Guarantees
Javier Borquez, Shuang Peng and Somil Bansal
Hybrid dynamical systems provide a powerful modeling framework for robotic systems, particularly in contact-rich environments. However, ensuring safety and performance in such systems remains challenging due to the intricate coupling between continuous dynamics and discrete mode transitions. In this work, we extend classical Hamilton-Jacobi (HJ) reachability analysis, a formal verification method for continuous-time nonlinear systems, to hybrid dynamical systems. Our framework characterizes safe sets for hybrid systems through a generalized value function defined over both discrete and continuous states while accounting for control constraints and model uncertainty. We additionally provide a numerical
algorithm to compute this value function. Building on these safe sets, we propose two different mechanisms to integrate performance objectives. First, we introduce a hybrid least-restrictive safety filter that intervenes on both the discrete and continuous components of a nominal controller only when necessary to avoid unsafe states, thereby preserving nominal behavior whenever possible. Second, we formulate and compute hybrid backward reach-avoid tubes, enabling the simultaneous enforcement of safety and goal-reaching behavior, an extension not previously addressed within hybrid HJ reachability. This enables the synthesis of continuous and discrete control policies that guarantee both safety and task completion. We validate our framework through simulation studies and real-world experiments on a quadrupedal
robot, demonstrating its effectiveness in hybrid mode planning and safety-critical applications.
Read the full abstract ▾
Robotics · Cross-listed from cs.CV
SlotDiT: Object-Centric Representations for Diffusion Transformers
Gjergj Plepi and Sven Behnke
Text-conditioned latent
diffusion models perform strongly in
video generation and are promising backbones for robotic applications. However, existing approaches rely on pixel-level or VAE-based latent representations that lack explicit semantic structure, leaving the impact of the representation space largely unexplored. Slot-based object-centric representations offer a structured alternative by decomposing scenes into object-level latents, or slots. While they have shown success in dynamics modeling and planning, they have not yet been explored for
diffusion-based generative modeling. We introduce SlotDiT, a text-guided
Diffusion Transformer (DiT) that operates in a slot-based latent space. Given a reference image and a
language instruction, SlotDiT decomposes the scene into object-centric slots representing individual entities. Conditioned on the instruction and observed scene context, the model autoregressively denoises future slot trajectories to predict scene dynamics. To systematically investigate latent-space design for
diffusion transformers, we compare slot-based representations against VAE-based and semantics-aligned alternatives within a unified DiT framework. Our experiments show that using slots as DiT latents yields competitive
video generation quality while consistently improving task-completion rates across four robotic
datasets. Furthermore, their compact representation provides a computationally efficient alternative to VAE-based and semantics-aligned latent spaces. Overall, our results demonstrate that object-centric structure is a powerful inductive bias for
diffusion-based generative modeling in robotic environments. The project page is available at https://slot-dit.github.io/.
Read the full abstract ▾
Robotics
Optimized Wrench Polytope Analysis for Real-Time Stability Control of Legged Robots in Complex Multi-Contact Configurations
Friedrich Graaf, Elias Birkefeld, Christian Eichmann et al. (+5 more)
Legged
robots offer a variety of automation applications in real-world scenarios. But areas that are difficult to traverse, like slopes, caves, or scaffolding, still pose a great challenge for traversal. To tackle this problem, we propose an optimized
algorithm for evaluating the full actuatable wrench polytope for arbitrary contact scenarios. With our improved analysis
algorithm, the torques for each joint of the
robot can be calculated within a control frequency of 49 Hz. The achieved speedup allows for deployment within a regular control loop for actuating
robot poses for different contact scenarios. We evaluated our stability controller extensively in simulation scenarios and validated its applicability by deploying it on actual walking
robot hardware. The proposed controller achieved stability in very complex scenarios that are currently not achievable by any other controller.
Read the full abstract ▾
All authors Friedrich Graaf, Elias Birkefeld, Christian Eichmann, Elias Hofele, Tristan Schnell, Georg Heppner, Arne Roennau, Rüdiger Dillmann
Robotics
Residual Fault Adaptation for Dexterous In-Hand Manipulation Under Runtime Joint Faults
Linan Deng, Xing Liu, Lin Hong et al. (+4 more)
Dexterous in-hand manipulation requires coordinated control of multiple actuated joints, and a runtime joint fault can abruptly disrupt the contact configuration required for successful manipulation. In this work, we propose residual fault adaptation (RFA), a teacher-anchored framework for compensating for hidden command-channel faults. RFA retains a frozen healthy teacher to provide nominal behavior and trains a recurrent residual policy to infer corrective actions from proprioceptive and command-response history. During training, fault-injection domain randomization (FIDR) varies the fault mode, affected joint, severity, and onset time, while adaptive sampling increases the frequency of fault modes associated with lower recent performance. A frozen Direct FIDR policy provides a distributional reference only on fault-active training samples and is absent from deployment. The deployed controller receives neither fault labels nor controller-switching signals. Simulation experiments on the dexterous hand indicate that RFA can improve manipulation performance relative to the healthy policy under a fixed mixed-fault protocol. Real-
robot experiments with software-injected faults further demonstrate zero-shot deployment of the learned adaptation policy.
Read the full abstract ▾
All authors Linan Deng, Xing Liu, Lin Hong, Feng Hua, Guijun Ma, Zuogong Yue, Fumin Zhang
Robotics
Exact Fusion and Coordinated Exploration in Multi-Robot Active Inference
Peng Wu, Mohsen Imani, Amidu Kamara et al. (+3 more)
Robot teams that learn a common environment model exchange belief summaries and plan by the expected information gain of their actions. Under conjugate exponential-family beliefs the shared belief is counted once per
robot at two points: at fusion, the product of local posteriors counts the common prior $n$ times, and at planning, every
robot scores its plan under the same belief and the team converges on the same unknown. Both errors are removed by adding evidence increments to the shared natural parameter, realized increments at fusion and expected increments at planning. The expected increment of a committed teammate gives the next
robot its conditional gain; corrected gains sum to the joint gain, the redundancy removed equals the total correlation of the planned observation streams, and sequential commitment keeps the $1/2$ greedy guarantee. The expected increment is exact for Gaussian beliefs with fixed sampling paths and for Dirichlet beliefs under the novelty approximation of discrete active inference, whose team objective has a closed concave form within an explicit bound of the exact mutual information, and fails for finite hypothesis classes, where a short exact enumeration replaces it. Experiments on cooperative RockSample, foraging, and field monitoring show that fusion correction leaves exploration redundancy unchanged, anticipated evidence removes it, and sequential commitment recovers most of the value of centralized joint planning at cost linear in the team size.
Read the full abstract ▾
All authors Peng Wu, Mohsen Imani, Amidu Kamara, Md Tamzeed Islam, Seyede Fatemeh Ghoreishi, Mahdi Imani
Robotics
XPACE: Joint World and Action Modeling from Heterogeneous Experience
Jiacheng Wei, Jerry Bai, Xiaoyu Yue et al. (+13 more)
A general-purpose
robot needs to draw on diverse experience, choose actions, and anticipate how those actions will change the world. We introduce XPACE, a unified embodied world model that serves as both a world action model, jointly predicting executable
robot actions and future
video, and a world simulator, predicting the visual consequences of prescribed actions. Our key insight is that
video prediction can both connect heterogeneous experience to action learning and generate new experience for policy improvement. With a shared
video backbone between the policy and simulator, we use action-unlabeled
video to learn visual dynamics and action-labeled human and
robot demonstrations to jointly learn
video and action prediction. Building on this architecture, a coarse-to-fine training curriculum progressively emphasizes
robot control while retaining human experience, allowing the policy to learn behaviors beyond those covered by
robot demonstrations. Beyond learning from recorded experience, XPACE uses its simulator to create additional recovery supervision for the policy. Specifically, we adapt the simulator to its own generated context, synthesize deviation-recovery trajectories around expert demonstrations, and fine-tune the policy on filtered recovery examples. Experiments on XPENG's IRON humanoid
robot show that heterogeneous training improves robustness and enables transfer of human-observed skills to tasks absent from
robot demonstrations, while recovery data generated by the model's own simulator further improves real-world task completion. Together, these results demonstrate how joint world and action modeling connects learning from heterogeneous experience with simulation-driven policy self-improvement.
Read the full abstract ▾
All authors Jiacheng Wei, Jerry Bai, Xiaoyu Yue, Zidong Wang, Xiaoyang Guo, Cheng Chen, Fanqi Pu, Fan Wu, Zhixu Yue, Yizhuo Li, Feng Qiu, Bo Liu, Yuying Ge, Hui Zhou, Chenyi Chen, Yixiao Ge
Robotics · Cross-listed from cs.CR
RobResilience: Implementing and Evaluating a Resilience Framework for Cyber-Physical Embodied Systems
Gysella Imrell, Emanuele Miotto, Mahya Mohammadi Kashani et al. (+2 more)
In embodied cyber-physical systems, active cyberattacks pose an immediate threat not just to data, but to physical integrity and human safety. While existing security approaches excel at detection, they lack the runtime mechanisms to determine whether a disruption is tolerable or if performance degradation remains within safe operational bounds. This gap leaves autonomous systems vulnerable to graceful failure paralysis, where they cannot distinguish between a safe, degraded state and a catastrophic hazard during an ongoing attack. This paper presents RobResilience, an implementation of a formal resilience framework for embodied cyber-physical systems in a Webots simulation environment, using a PR2
robot and ROS2. The framework evaluates three predicates at runtime: tolerable disruption ($\delta$), tolerable degradation ($\gamma$), and mitigation feasibility ($\mu$), over a compromised device set derived from IDS confidence scores. When resilience is lost, the framework triggers available mitigation strategies. We evaluate our implementation through eight attack scenarios that systematically cover all possible combinations of the predicate state space, varying attack targets, degradation rates, and mitigation availability. Results confirm that the runtime behaviour of the implementation is consistent with the theoretical definitions.
Read the full abstract ▾
All authors Gysella Imrell, Emanuele Miotto, Mahya Mohammadi Kashani, Mauro Conti, Alberto Giaretta
Robotics
Online Geometric Change Detection via Scene Decomposition
David Thorne, Samuel Jia Cong Chua, Nakul Joshi et al. (+4 more)
Autonomous
robots are increasingly deployed on long duration single- and multi-session missions in dynamic environments, where the ability to identify environmental changes such as fallen trees or opened doors provides important contextual information for online planning. We propose a framework called Change Detection via Scene Decomposition (CDSD) for accurate online
geometric change detection using
LiDAR or RGB-D sensors. Recent advances in
geometric SLAM have made it possible to generate dense, tightly aligned maps without post processing, but comparing global maps across entire sessions is computationally expensive and does not allow for single-session online change detection. CDSD instead spatially decomposes mapped environments into unique scenes where changes can be found efficiently by comparing dense, local subsets of the global map called submaps. As the first submap-based approach for
geometric change detection, we identify and address the following core challenges: 1) identifying appropriate scenes for change detection that require minimal redundant information; 2) generating dense and representative submaps for each scene; 3) detecting changes between submaps with differing fields of view; and 4) processing detected changes for real-time map reconstruction. Results demonstrate our
algorithm on custom
datasets collected at the Army Research Laboratory facility in Graces Quarters, Maryland, and on open-source multi-session change detection
datasets.
Read the full abstract ▾
All authors David Thorne, Samuel Jia Cong Chua, Nakul Joshi, Aiden Wong, Christa S. Robison, Philip Osteen, Brett T. Lopez
Robotics · Cross-listed from cs.NE
Machine Zygote: Causal Biparental Heredity Before Learning in a Germline–Soma Artificial Agent
Lyes Saad Saoud
Artificial ontogeny, developmental encodings,
robot reproduction, and inherited controllers are established research directions, yet a narrower question remains: can a newborn artificial
agent exhibit measurable biparental heredity before learning, and can that dependence be isolated causally rather than inferred only from parent-offspring resemblance? We introduce Machine Zygote, a computational germline-soma architecture designed to test this question. Two parental germlines are independently mutated and recombined into a zygote that parameterizes development of an initially generic eight-module soma, which is then frozen and evaluated without learning. A preregistered 4 x 4 diallel of 640 offspring shows significant dam and sire dependence for five of six behavioral traits after Holm correction, with parental and interaction components accounting for 36-53 percent of modeled variance across five principal traits. In matched-background interventions (n=60), substituting one parental germline while holding recombination and
stochastic background fixed causes phenotype shifts exceeding a same-parent re-mutation control for five of six traits for both parental channels. Recombination also yields excess transgressive offspring for speed and gait frequency. A preregistered developmental-dependence hypothesis is not supported: a quasistatic no-dynamics ablation preserves the mean phenotype distribution while altering parental variance structure. Thus the study supports causal biparental pre-learning heredity in this simulation, but not the stronger claim that recurrent developmental dynamics are necessary. It does not establish physical heredity, biological genetics, or autonomous
evolution. The contribution is an intervention-centered framework and reproducible
benchmark for separating heredity, development,
stochastic variation, and post-birth learning.
Read the full abstract ▾
Robotics
Escape-Aware Control Barrier Functions for Quadrotor Safety under Body-Rate Limits
Lei Shi, Haosong Wen and Qichao Liu
Control barrier functions for input-constrained systems place the admissible input set inside the definition of the safe set, yet the resulting barrier is almost always a function of the state alone; On a quadrotor this is not cosmetic: because the thrust vector must be reoriented before it can decelerate an approach, and reorientation is limited by the attainable body rate, a state-only barrier certifies states from which no escape is reachable in time; We characterize the certification gap in closed form and show its width is proportional to closing speed and inversely proportional to the body-rate limit; We then define an escape barrier on the augmented pair of state and previously applied input, with escape authority measured over the one-step reachable thrust cap; It admits a closed form and an analytic inverse for the maximum certifiable closing speed, and embeds in a predictive controller at no additional state cost; Across 550 paired closed-loop episodes on a 13-state quadrotor, the proposed controller completes every tested scenario, whereas the stopping-distance barrier enforced over the same horizon fails 15% and 25% of episodes in exactly the two scenarios that enter the predicted gap; Against an online backup-CBF baseline enforcing the same escape condition at the reached state, it holds a 29-74 degree larger directional margin and 3-18 times the clearance, and an independent conservative rollout referee finds no certified state from which escape fails.
Read the full abstract ▾
Robotics · Cross-listed from cs.MA
Calibrate Once, Fly Any Team: Residual-Grounded Low-Fidelity Training for Cooperative Drone Swarms
Maxim Mednikov and Oren Gal
Training multi-
agent drone-swarm policies directly in high-fidelity (HF) rigid-body physics is accurate but computationally expensive. This cost scales poorly with team size, as each additional
agent multiplies contact-resolution complexity and sharply raises the in-simulation crash rate. To address this, we propose a mixed-fidelity training scheme that eliminates HF reinforcement learning entirely. A single shared, decentralized policy is optimized inside a fully-differentiable, JAX-native low-fidelity (LF) point-mass simulator. The simulator is corrected by a small, per-
agent bagged residual ensemble fit once, offline, using short calibration flights in the HF simulator. Because calibration requires only one isolated drone, the data collection budget does not compound with team size. Reference trajectories are generated by rolling out an existing LF-only policy and tracked in the HF simulator by a zero-training PD controller. Evaluated across four cooperative drone tasks and team sizes from 3 to 18, the residual-corrected policy outperforms an uncorrected LF baseline in all combinations, and a from-scratch HF policy in 22 of 24 combinations tested. It trails an HF-finetuned policy by a margin that narrows steadily with team size. Ultimately, the proposed method achieves near-equivalent performance at the largest team sizes at a fraction of the computational cost, completely avoiding the high crash rates typical of HF training.
Read the full abstract ▾
Robotics
CAD-Based Relation Learning and Geometric-Symbolic Planning for Robotic Assembly
Fabian Harlacher and Christian Friedrich
Assembly Sequence Planning (ASP) remains a challenging problem due to its combinatorial nature, making exhaustive planning approaches impractical for complex industrial assemblies. Furthermore, many CAD models lack reliable semantic contact information or require extensive manual preprocessing, limiting the applicability of existing methods. This paper presents a hybrid ASP framework combining learning-based relation extraction with
geometric-symbolic
reasoning to generate feasible robotic disassembly sequences from imperfect CAD data. A neural network predicts semantic
geometric relations from point clouds, while human-in-the-loop verification enables correction of uncertain predictions and planning failures. Extracted relations are transformed into a symbolic assembly graph, enabling a
geometric-symbolic planner to efficiently compute locally valid sets of robotic manipulation primitives. A visibility-based ray-casting strategy guides the search for feasible disassembly directions without requiring an exhaustive combinatorial search, while the local solution space enables efficient sequence optimization. The framework is evaluated on an introduced assembly
dataset and on the ASAP test
dataset. On the ASAP test
dataset, the proposed planner achieves an 85.83% planning success rate while reducing the median planning time by more than one order of magnitude across all assembly sizes and by more than a factor of 50 for assemblies with more than 30 components compared to the baseline. The results demonstrate that the proposed hybrid framework enables efficient robotic assembly sequence planning from imperfect CAD data while substantially reducing planning time. By combining learning-based feature segmentation, human-in-the-loop verification, and
geometric-symbolic
reasoning, the framework provides a practical foundation for scalable and adaptable robotic assembly and disassembly planning.
Read the full abstract ▾
Robotics
Port-Hamiltonian Koopman Operator Synthesis for Mechanical Systems
Rajpal Singh, Aditya Singh and Jishnu Keshavan
Finite-dimensional Koopman models enable efficient linear prediction and control of nonlinear robotic systems. However, models learned purely from trajectory data may violate the energetic structure of the underlying mechanics, producing predictions that exhibit artificial energy growth and diverge under recursive propagation. This work presents a structure-preserving Koopman framework for Euler-Lagrange systems built on generalized-momentum coordinates. The momentum transformation exposes the mechanical actuation as a known, state-independent port, which is preserved explicitly in the lifted dynamics. A structure-constrained neural architecture is developed to jointly learn the lifting functions and a port-
Hamiltonian Koopman generator, rendering the learned dynamics passive by construction rather than through penalty terms or post-hoc projection. A Cayley-midpoint discretization further preserves the corresponding storage-dissipation balance exactly in discrete time. These properties are established analytically by deriving the discrete storage balance and associated stability guarantees of the learned predictor. Simulation and experimental studies demonstrate improved prediction accuracy, data efficiency, and closed-loop tracking over Koopman baselines, with increasing gains for higher-dimensional systems.
Read the full abstract ▾
Robotics
DriveMCP: An Agentic AI framework for Advanced Driver Assistance System
Farzad Nadiri, Mehdi Cina and Ahmad B.Rad
An agentic AI driver-assistance framework that integrates perception, compliance
reasoning, vehicle-state interpretation, and safety arbitration into a modular and auditable pipeline. The architecture, referred to as DriveMCP, incorporates a sensor-like perception stack alongside DriveLM as the
vision-
language front end to generate a graph-structured scene understanding (Graph Visual Question Answering) and
language-grounded driving information. Key compliance elements in world_state, including posted speed limits and jurisdiction cues, are derived from DriveLM outputs through a structured parsing layer rather than being injected as simulator ground truth. A stateful orchestration layer coordinates specialized experts exposed as Model Context Protocol (MCP) servers: (i) a Rules server that performs
retrieval-augmented compliance
reasoning over jurisdiction-specific traffic codes and sign conventions, (ii) a Weather server that estimates traction risk and contextual speed advisories, and (iii) an MCP-CAN server that surfaces Controller Area Network (CAN)/On-Board Diagnostics (OBD) telemetry and diagnostic context for health-aware risk shaping. These outputs are fused to generate a structured decision that
prompts a recommended course of action. The outcome is then further filtered by a Responsibility-Sensitive Safety (RSS)-inspired guardrail that arbitrates speak versus act decisions under bounded online adaptation. In CARLA simulation across multilingual, cross-border, and dynamic speed-limit scenarios, DriveMCP reduces traffic infractions and overspeed relative to the VLM-Direct, VLM-Direct+RAG, and VLM-Tools-NoArbiter baselines, while improving hazard response time and maintaining sub-second advisory latency.
Read the full abstract ▾
Robotics
Swim-and-Breach at Palm Scale: A Rudder-Steered Two-Propeller Underwater Robot Platform with Differential-Thrust Pitch Control
Daehyun Choi, Ian Bergerson, Hengjia Zhu et al. (+2 more)
We present a palm-scale (65 mm, 34 g) swim-and-breach
robot platform. Two vertically stacked propellers provide both propulsion and differential-thrust pitch control under a proportional-integral-derivative (PID) loop, and a tail rudder adds yaw control. The hull, evaluated by flow simulation, reduces the drag five-fold relative to an equivalent cuboid, and the propellers are optimized using B-series modeling validated by dynamometer measurements. The current
robot swims at 13.9 body lengths per second and turns at 209 deg per second, corresponding to the upper limits reported for underwater
robots. In free swimming, the pitch loop turns the body to any commanded nose-up pitch angle, and, with the rudder stabilizing the exit, the current
robot leaps 1.6 body lengths high and 3.7 long in a seamless cruise-leap-cruise sequence. The platform can be used to build small-scale
robots that cross barriers and dry gaps between pools for inspection in streams, flooded structures, and industrial systems.
Read the full abstract ▾
All authors Daehyun Choi, Ian Bergerson, Hengjia Zhu, Tianjun Lan, Saad Bhamla
Robotics
FluxVLA Engine: A One-Stop VLA Engineering Platform for Embodied Intelligence
Yinhao Li, Weixin Mao, Zihan Lan et al. (+21 more)
Vision-
language-action (VLA) models, world-action models (WAMs), and offline reinforcement learning methods are rapidly expanding the design space of embodied policies, yet turning these
algorithms into reliable
robot systems remains constrained by fragmented data formats, training stacks, evaluation protocols, inference runtimes, and embodiment-specific interfaces. We present $\mathrm{FluxVLA}$ Engine, an open, configuration-driven platform that turns heterogeneous embodied-policy components into a reproducible data-to-deployment workflow. Rather than introducing another policy model, $\mathrm{FluxVLA}$ standardizes interfaces for
datasets, visual-
language and world models, action heads, reward- or advantage-weighted learning, distributed training, simulation evaluation, optimized inference, and
robot operators. The engine further integrates compositional dual-arm simulation, scalable automatic data generation, and model-decoupled human-in-the-loop rollout, takeover, correction collection, and reward annotation. For responsive physical execution, it combines Real-Time Chunking (RTC) with accelerated inference backends, lightweight remote GPU serving, and configurable trajectory post-processing. Together, these capabilities connect offline learning, simulation validation, online correction, and real-
robot execution through shared and auditable contracts. $\mathrm{FluxVLA}$ therefore targets the engineering bottlenecks separating promising embodied-learning
algorithms from reproducible evaluation and dependable deployment. Code is available at https://github.com/FluxVLA/FluxVLA
Read the full abstract ▾
All authors Yinhao Li, Weixin Mao, Zihan Lan, Jikun Rong, Qirui Hu, Yiming Zhang, Weipeng Deng, Bowen Shen, Minzhao Zhu, Yiming Mao, Yan Yang, Chenguang Cui, Hongyuan Chen, Xu Huang, Zheyi Zhao, Pinxi Shen, Bozhen He, Zhen Fu, Yifan Wang, Zexin Zhang, Ang Gao, Haoyu Chen, Chengqi Shi, Hua Chen
Robotics
TIO-Former: Ultra-Lightweight 6-Directional ToF-Inertial Odometry for Nano-UAVs via a Streaming Causal Transformer
Yang Liu, Yifan He, Wenhao Zhao et al. (+9 more)
Autonomous nano-
UAV navigation requires accurate ego-motion estimation under stringent size, weight, power, and computing (SWaP-C) constraints, where visual sensors and
LiDARs exceed payload limits, optical flow degrades in low-texture scenes, and inertial-only state estimation is susceptible to accumulated drift. While multi-zone time-of-flight (ToF) arrays provide a lightweight metric complement, 6-DoF estimation from merely 384 ranges per frame is challenged by invalid returns, anisotropic observability, and temporal computational scaling. We propose TIO-FORMER, a camera-free, optical-flow-free, and mapless range-inertial odometry framework driven by an IMU and an ultra-lightweight (15 g) payload of six orthogonal 8 x 8 ToF arrays. Our frontend pairs consecutive range grids with a bilateral gated difference, while IMU-guided cross-
attention dynamically routes directional features conditioned on platform kinematics. A Streaming Causal
Transformer couples an uncompressed Local KV cache with compressed Chunk-FIFO
memory, maintaining bounded inference cost and
memory footprint independent of flight duration. In real-flight evaluations, TIO-FORMER reduces open-loop position error by 54.4% compared to nano-
UAV optical flow and by 66.4%-89.1% over learned inertial baselines. We also evaluate performance across multiple environments and robustness under severe sensing degradation. Deployed on an edge RISC-V companion computer, TIO-FORMER achieves a P95 latency of 10.466 ms and peak resident
memory of 6.324 MiB (less than 5 percent system RAM), demonstrating that sparse range sensing provides practical
geometric anchoring for resource-constrained micro-aerial
robots. Code is available at https://github.com/Ly041021/TIO-Former.
Read the full abstract ▾
All authors Yang Liu, Yifan He, Wenhao Zhao, Xiangyu Mo, Yang Xu, Hao Wei, Mingze Ma, Huan Li, Yifan Wu, Fei Gao, Zipeng Dai, Xin Zhou
Robotics
Fleet-To-Lab: A Transfer Learning Framework For Lunar Rover Slippage Estimation Via Model Fusion
Riccardo Viviano, Saki Omi, Andrej Orsula et al. (+1 more)
Accurate wheel slip estimation is essential for autonomous lunar rover mobility and navigation. Machine Learning models trained on terrestrial data generalize poorly to lunar terrain, and real lunar
datasets are scarce due to the limited number of missions and costly data acquisition. We present Fleet-to-Lab, a transfer learning framework that leverages proprioceptive data collected by previously deployed heterogeneous lunar rovers to mitigate the Earth-Moon domain gap in slip estimation for a future deployable unit. We fuse several heterogeneous expert models into a single architecture, using a modest
dataset collected after the rover deployment. We propose AcoMerge, a new hybrid swarm-intelligence
algorithm that performs model fusion by searching for an optimal combi- nation of expert parameters. Experiments conducted in a high- fidelity physics simulation show balanced accuracy and macro- F1 improvements compared to deep model fusion baselines. AcoMerge exhibits competitive performance with joint training on deep architectures, while achieving higher macro-F1 and balanced accuracy on a smaller model. Overall, our framework shows model fusion as a possible transfer learning alternative for slippage estimation in space robotic missions with limited data.
Read the full abstract ▾
All authors Riccardo Viviano, Saki Omi, Andrej Orsula, Miguel Olivares-Mendez
Robotics
Fingers as Legs: Learning Self-Supported Locomotion and Manipulation with an Anthropomorphic Hand
Amirhossein Kazemipour, Hehui Zheng and Robert Katzschmann
A walking robotic hand must use the same fingers to move its body, support its weight, and interact with the environment. We show how an anthropomorphic hand can learn these skills while retaining its finger design and position controller. Onboard power and computation make the platform self-contained. Our reinforcement learning approach accounts for the hand's unequal fingers, with training in a simulator calibrated from hardware measurements. In simulation, the hand moves faster with our reward formulation than with tuned rewards originally designed for quadrupeds. On hardware, task-specific policies enable untethered crawling, steering, and fall recovery. While supporting its own weight, the hand also executes successive keyboard commands without
vision and pushes an object to targets using overhead visual feedback. These results demonstrate a compact mobile manipulator that reuses its fingers for locomotion and interaction, without a separate locomotion mechanism.
Read the full abstract ▾
Robotics
HuMemSLAM: Efficient Human-Inspired Semantic Place Recognition for Robust Visual SLAM
Mayowa Adebambo, Sebastian Donnelly, Armand Amaritei et al. (+2 more)
Autonomous systems require reliable place recognition for efficient and effective simultaneous localisation and mapping (SLAM). Traditional
geometric visual SLAM approaches rely on low-level features and
geometric consistency, but remain vulnerable to perceptual aliasing, where different places appear similar, and perceptual variation, where the same place appears different. Although semantic SLAM and modern learned visual place recognition (VPR) methods improve robustness under challenging perceptual conditions, real-time deployment requires both high
retrieval accuracy and low latency. Inspired by human
memory and perception, we propose HuMem-VPR, which exploits the bidirectional relationship between bottom-up perceptual evidence and top-down contextual
reasoning to achieve high-level place understanding. We further introduce HuMemSLAM, the integration of HuMem-VPR with ORB-SLAM3. HuMem VPR achieved the highest aggregate
retrieval accuracy on the real-image
benchmark, competitive accuracy on the CARLA
benchmark, and approximately two to three times lower latency than the evaluated state-of-the-art VPR methods. Across the evaluated
dataset families and online experiments, HuMemSLAM substantially improved integrated Recall @1 over ORB-SLAM3's native
retrieval while reducing the proposals submitted to its
geometric backend.
Read the full abstract ▾
All authors Mayowa Adebambo, Sebastian Donnelly, Armand Amaritei, Andrew Bradley, Alexander Rast
Robotics
Kernel-Based Metrics Learning for Uncertain Opponent Vehicle Trajectory Prediction in Autonomous Racing
Hojin Lee, Youngim Nam, Sanghun Lee et al. (+1 more)
Autonomous racing confronts significant challenges in safely overtaking Opponent Vehicles (OVs) that exhibit uncertain trajectories, stemming from unknown driving policies. To address these challenges, this study proposes heterogeneous kernel metrics for Deep Kernel Learning (DKL), designed to robustly capture the diverse driving policies of OVs, and carry out precise trajectory predictions along with the associated uncertainties. A key virtue of the proposed kernel metrics lies in their ability to align similar driving policies and disjoin dissimilar ones in an unsupervised manner, given the observed interactions between the Ego Vehicle (EV) and OVs. The efficacy of the proposed method is substantiated through experimental studies on a 1/10th scale racecar platform, demonstrating improved prediction accuracy and thereby safely overtaking against OVs. Furthermore, our method is computationally efficient for onboard computing units, affirming its viability in fast-paced racing environments. The
video and source code can be found at https://github.com/HMCL-UNIST/OpponentPredictionWithKMDKL.git.
Read the full abstract ▾
All authors Hojin Lee, Youngim Nam, Sanghun Lee, Cheolhyeon Kwon