Robotics
Geometric Coherence via Weighted Matching for 3D Heterogeneous Multi-Agent Reach-Avoid Games
Prajwal Vijay
We study assignment quality in 3D heterogeneous multi-
agent reach-avoid games and identify a recurring failure mode of cardinality-only matching in geometrically structured scenarios, which we term
Geometric Sprawl. In these cases, multiple maximum-cardinality assignments are available, but some induce spatially incoherent pairings and inefficient pursuit trajectories. Building on the evasion-space framework of Yan et al.~\cite{yan2022}, we introduce a cardinality-first weighted sequential matching method in which the Hamilton–Jacobi–Isaacs interception value $z_I(s,j)$ is used as a secondary assignment weight. Each sequential stage is solved with a min-cost max-flow backend, while the unweighted baseline uses the same solver with the weight term removed. We evaluate both methods on a deterministic 35-scenario
benchmark (7 families & 5 initialization variants) under two regimes: a diagnostic stationary-unmatched setting and a hybrid saddle-point setting with goal-directed unmatched evaders. On this
benchmark, the weighted method resolves 13 of 15
geometric stress-test instances that cause repeated timeouts for the unweighted baseline in the diagnostic regime, and under the hybrid regime improves mean captures from $3.20$ to $3.91$ (22% increase) and mean interception height from $3.87$ to $5.36$ (40% increase). We also report lower path tortuosity and lower angular-effort proxy values, suggesting smoother pursuit trajectories in this first-order simulation model. We release the simulator and
benchmark suite at \href{https://github.com/Prajwal-Vijay/
geometric-coherence-weighted-matching}{github.com/Prajwal-Vijay/
geometric-coherence-weighted-matching} to support reproducible evaluation of assignment strategies for 3D reach-avoid games.
Read the full abstract ▾
Robotics · Cross-listed from eess.SY
A Geometric Decision Procedure for STL Feasibility and Repair
Avinash Malik
Feasibility of a Signal Temporal Logic (STL) specification asks whether, from a given initial state, an admissible control input can satisfy its continuous-time requirements under actuator limits. Establishing feasibility is essential for detecting unrealizable tasks and repairing specifications before execution. State-of-the-art Mixed-Integer Linear Programming (MILP) methods face growing model sizes as temporal horizons and formula complexity increase, and can accept control witnesses that violate continuous-time STL semantics between samples. We propose a
geometric approach to STL feasibility and temporal repair that translates temporal requirements into spatial constraints evaluated at the initial state. Its compiled feasibility check is independent of horizon length for fixed compiled dimensions. We prove nonnegative continuous robustness under a joint controller assumption. On 6D
UAV benchmarks, median
geometric repair times are 0.265–1.510 ms; we show up to 1.68x10^4 speedup over MILP feasibility checker. All
geometric runs pass continuous STL validation, whereas majority of completed MILP control witnesses violate continuous-time STL.
Read the full abstract ▾
Robotics
Robot World Models Are Not Invariant to How the Actions Are Written
Ahmed Karim and Leon Chlon
A
robot policy is trained with one of two action parameterizations: absolute joint targets, or deltas relative to the current state. The choice is a live engineering decision in
robot learning, and a world model conditioned on actions inherits it silently. We show the inheritance is catastrophic. A latent dynamics model trained on one parameterization and handed the identical commanded trajectory written in the other collapses:
retrieval degrades by 2.6-13.4x across three
robot datasets and two morphologies, goal-conditioned action selection falls from 53% to 15%, and on PushT the two beliefs about the same future are near-orthogonal (cos = 0.067, worst case -0.377), so the predictor does not degrade gracefully, it answers a different question. This is not a distribution-shift artifact in the usual sense: the two encodings are mutually reconstructible at R^2 = 0.996 given the joint input, so no information is lost, and we give the test that separates a valid re-parameterization from a lossy summary or a sensor swap. The test rejected three of the four axes we proposed. The defect lives in the action channel, which the invariance literature for visual models does not examine: work there concerns crops, jitter and camera pose, while the parameterization of the commands goes unaudited. The repair is averaging over the two encodings, and where it goes matters. Averaging the objective restores task performance by itself; averaging the outputs, safe for probabilities by concavity, is not available for direction-valued prediction, where the normalized mean can score below every member of the orbit. What objective-averaging leaves behind is the tail: worst-case agreement stays at 0.78, a disagreement penalty closes it to 0.995, and over a latent rollout it is the difference between a worst case that erodes and one that holds. On PushT, averaging alone does not repair the axis.
Read the full abstract ▾
Robotics · Cross-listed from cs.NI
Orientation-Aware Control and Trajectory Design for Aerial RIS-Assisted Wireless Communications
Abdoul Karim A. H. Saliah, Hajar El Hammouti, Daniel Bonilla Licea et al. (+1 more)
Aerial Reconfigurable Intelligent Surfaces (ARISs) can enhance wireless connectivity in obstructed environments by combining reconfigurable metasurfaces with aerial mobility. However, in practical deployments, the Reconfigurable Intelligent Surface (RIS) is rigidly mounted on a multi-rotor Unmanned Aerial Vehicle (
UAV), which creates coupled position-orientation dynamics that directly affect communication quality. Since RIS performance is highly sensitive to orientation, point-mass or orientation-invariant models may lead to optimistic performance estimates. This paper proposes an orientation-aware ARIS framework that explicitly accounts for
UAV dynamics, actuation limits, and orientation-dependent channels. The proposed approach combines a communications-guided reference trajectory planner with a Nonlinear Model Predictive Control for dynamically feasible trajectory tracking and favorable RIS orientation, while employing lightweight beamforming and RIS phase-shift updates. Simulations show that accounting for orientation improves user throughput by 13.77%, while the proposed joint communication updates improve network throughput by 28% over phase-shift-only optimization and by at least a factor of 2.5 over beamforming-only optimization. These results highlight the importance of orientation-aware modeling and joint communication-control design in ARIS-assisted wireless systems.
Read the full abstract ▾
All authors Abdoul Karim A. H. Saliah, Hajar El Hammouti, Daniel Bonilla Licea, Giuseppe Silano
Robotics · Cross-listed from eess.SY
Task aware Dynamic Movement Primitives for failure detection and recovery in contact rich manipulation
Bhavnashri A, Sobia Shafi, Krishnapuram Himavarshini et al. (+1 more)
Assembly remains a challenging robotic manipulation task in presence of tight tolerances and complex contact interactions. While Learning from Demonstration (LfD) frameworks like Dynamic Movement Primitives (DMPs) can effectively encode trajectories from a single demonstration, they are highly sensitive to variations in initial grasp configurations and external contact forces. Such variations often lead to task failures during the contact rich phases. This paper presents a task aware failure detection and recovery framework that integrates DMP based trajectory generation with real time stage classification. Utilizing Quadratic Discriminant Analysis (QDA) trained on
multimodal sensor data, the framework segments execution into approach,
alignment, and insertion stages for a Peg in Hole (PiH) assembly operation. By using goal relative position data as features, this classification generalizes to unseen goal positions without requiring retraining, matching the inherent generalization capability of DMPs. Anomaly detection is performed online using a Mahalanobis distance metric computed over force features, isolating contact induced failures from nominal trajectory execution. Upon failure detection, a spiral search recovery policy is triggered to actively realign the peg under contact before resuming the learned DMP insertion. The proposed approach is evaluated on an experimental setup achieving 95% stage classification accuracy, and demonstrates reliable failure recovery under lateral misalignments of up to 3 mm using only a single demonstration.
Read the full abstract ▾
All authors Bhavnashri A, Sobia Shafi, Krishnapuram Himavarshini, Anuj Tiwari
Robotics
Probabilistic Scene Graphs: Hierarchical Representation and Real-time System
Waqas Ali, Michele Antonazzi, Timon Homberger et al. (+4 more)
3D scene graphs provide semantically rich and hierarchical representations for
robot perception. However, existing systems do not maintain uncertainty as an explicit belief or propagate it through the operations that construct and refine the graph. We introduce Probabilistic Scene Graph (PSG), a generalization of the conventional scene graph that represents a posterior over possible graphs, factorized into a discrete graph structure of entities, relations, and semantic attributes, and continuous states that ground them spatially, with uncertainty maintained over both components. Geometry is carried directly by the nodes rather than selected from a separately constructed metric map, so a metric map, where needed, follows from the graph rather than preceding it. We instantiate PSG's probabilistic spatial grounding with hierarchical graphs of Gaussians (HGG): each object primitive is represented by a full-covariance Gaussian under a Normal-Inverse-Wishart belief, and the same parametrization applied recursively within a node yields a geometry graph that resolves its surface at finer resolution. We then build a mapping pipeline that preserves these beliefs throughout graph construction and refinement: a purely graph-based coarse-to-fine
alignment registers observations by comparing node beliefs, while a nested Expectation-Maximization and factor-graph optimization jointly refines poses, object parameters, and internal geometry. Across six
datasets spanning indoor RGB-D, outdoor
LiDAR, and cross-modality deployment, HGG operates at sensor rate with near-constant
memory and achieves state-of-the-art object accuracy and zero-shot graph
alignment.
Read the full abstract ▾
All authors Waqas Ali, Michele Antonazzi, Timon Homberger, Thien-Minh Nguyen, Lukas Rosenberger Schmid, Patric Jensfelt, Yixi Cai
Robotics
AquaCap: A Training-Free Underwater Embodied Agent with Code-as-Policy
Xiaoshi Li, Yule Xu, Chunghiu Kong et al. (+5 more)
Recent advances in
vision-
language-action models have stimulated growing interest in underwater embodied intelligence. However, their reliance on large-scale interaction data limits their applicability underwater, where data collection is costly and scarce. To address this challenge, we present AquaCap, a training-free Code-as-Policy framework for autonomous underwater navigation and manipulation. AquaCap employs a dual-layer
agent that translates task instructions and environmental observations into condition-aware plans and executable control programs. Structured perception then provides the
agent with semantic,
geometric, and reliability-aware observations under degraded underwater conditions. A failure-aware
memory diagnoses unsuccessful actions and supports closed-loop replanning and code revision. This design enables online adaptation without task-specific training or parameter updates. AquaCap achieves a 66.43% success rate in simulation. Real-world experiments further demonstrate autonomous grasping and object transport with an ROV, including the manipulation of targets displaced by hydrodynamic disturbances.
Read the full abstract ▾
All authors Xiaoshi Li, Yule Xu, Chunghiu Kong, Yizhou Zhou, Yang Liu, Hao Yang, Zihao Huang, Yunxiao Shan
Robotics
Tip Manipulation in Soft Everting Robots via Wall Retraction and Deployable Fingers
Nelson Badillo Perez, Niccolo Pagliarani, Matteo Cianchetti et al. (+1 more)
Soft everting
robots can traverse long, confined paths by continuously growing, yet active interaction remains limited to a single tool fixed at or near the tip, unable to be repositioned on demand and difficult to reconcile with the
robot's soft body. We introduce a tip-manipulation and multi-tool deployment strategy for soft everting
robots based on wall retraction, implemented with a base roller assembly that independently meters membrane flow in the outer wall while a tail spool regulates growth in the internal tail section. Coordinated wall and tail actuation decouples
robot length from membrane-material position, enabling membrane-mounted devices to be transported, exposed, and repositioned at selected locations near the distal tip. We pair this capability with ultralight pleated inflatable fingers integrated into the membrane, fabricated from TPU-coated nylon with an internal airtight bladder. The fingers achieve large bending at low pressures (approximately 100 degrees at 50 kPa in high-pleat designs) and generate blocking forces up to 1.9 N, while remaining limp during transport. The system demonstrates adaptive grasping across diverse household objects (21 to 550 g; 16 to 200 mm), three-dimensional object manipulation and stacking, environmentally braced extension, distal camera panning for confined-space inspection, and controlled sequential payload delivery. These results enable embodied and reversible tip manipulation for soft growing
robots in cluttered and tortuous environments.
Read the full abstract ▾
All authors Nelson Badillo Perez, Niccolo Pagliarani, Matteo Cianchetti, Robert D. Howe
Robotics
Selective Commitment for Language-Guided Object Retrieval under Partial Observability
Wonhee Koh, Sushil Samuel Dinesh, Hansol Ko et al. (+2 more)
Language-guided object
retrieval under partial observability requires deciding whether to gather more evidence, interact with the scene, grasp a candidate, or abstain. We present a closed-loop framework that coordinates these decisions for retrieving a target specified in relation to a reference container. The framework maintains a persistent joint belief over target identity, container relation, and presence through tracked-object, unobserved-target, and target-absent hypotheses. View-conditioned categorical VLM observations update this belief; conformal grasp eligibility and
robot feasibility govern commitment, while finite-horizon belief-space planning selects information-gathering actions. Across five different scenarios, our proposed method succeeds in 19/25 simulation episodes versus 12/25 for the best-performing task-adapted baseline and is the only evaluated policy to achieve at least one success in each scenario. Ablations show that cross-view
memory improves success under partial occlusion, while the full system does not consistently outperform simplified variants. Real-
robot trials demonstrate closed-loop re-observation and autonomous recovery from injected grasp failures, while injected viewpoint failures end in false defer. Experimental results demonstrate the feasibility of coordinating evidence gathering and selective grasp commitment within a unified framework for
retrieval under partial observability.
Read the full abstract ▾
All authors Wonhee Koh, Sushil Samuel Dinesh, Hansol Ko, Shinkyu Park, Eungjoo Lee
Robotics
Verti-WM: A Physics-Aided Exteroceptive World Model for Off-Road Reinforcement Learning
Chenhui Pan, Tong Xu and Xuesu Xiao
Reinforcement learning for off-road navigation requires extensive vehicle-terrain interaction data, which are costly to collect in high-fidelity simulation. World models offer a promising alternative by replacing simulator roll-outs during policy optimization. However, an off-road world model must condition state transitions on exteroceptive terrain information, which proprioception alone does not provide. This challenge is further amplified by the need to model both rigid and deformable terrain, where data-driven and physics-based approaches offer complementary strengths. We propose Verti-WM, a physics-aided exteroceptive world model that recurrently fuses a frozen
Transformer for rigid terrain and a neuro-symbolic terramechanics model for deformable terrain. Elevation and semantic observations queried from a supplied map at each predicted pose condition fusion, enabling six-degree-of-freedom rollouts for policy optimization without further simulator access. Verti-WM reduces prediction error by 34.6% and 21.7% over data-driven and physics-based baselines, respectively. Policies trained entirely within Verti-WM achieve comparable task success rates while reducing computation time by 23.6X relative to direct training in the high-fidelity simulator. We further validate Verti-WM using real-world data, enabling policy optimization within learned real-world kinodynamics and achieving a 80% success rate on the Verti-4-Wheeler platform, compared with 40% for direct sim-to-real transfer.
Read the full abstract ▾
Robotics
Search, Ground, Plan: Functional Sufficiency for Task and Motion Planning under Incomplete Scene Knowledge
Narendhiran Vijayakumar, Nav Singhal, Girish Varma et al. (+1 more)
Foundation models (FMs) have expanded task and motion planning (TAMP) to manipulation problems specified through
language and visual observations. However, incomplete scene knowledge leaves a critical gap between understanding what the task requires and knowing whether the physical scene can actually realize it. We introduce GRAB-TAMP, an FM-based TAMP framework that searches for scene entities required for task completion, grounds functional roles to valid physical objects, and plans only after a complete joint assignment establishes functional sufficiency. We represent the task through functional roles, relations, and assignment constraints, and incrementally inspect the scene while requirements remain unresolved, verifying candidate objects through semantic,
geometric, and relational checks. We evaluate GRAB-TAMP across 32 scene variants spanning Kitchen, Living Room, and Workshop domains. Across 200 feasible trials, our approach achieves 54.0% end-to-end success with 67.3% plan goal coverage. Compared with three FM-based TAMP frameworks under the same execution setting, GRAB-TAMP improves end-to-end success by 25.7 percentage points over the mean baseline. Implementation and evaluation code: https://github.com/Narendhiranv04/GRAB-TAMP
Read the full abstract ▾
All authors Narendhiran Vijayakumar, Nav Singhal, Girish Varma, Antony Thomas
Robotics
DiagGen: Agentic Generation of Deformable Assets with Sim-based Diagnostics for Robotic Simulation
Guanxiong Chen, Yiduo Qu, Qianjun Xia et al. (+12 more)
While simulation-ready deformable assets are essential for in-silico robotic manipulation tasks, existing generation frameworks typically assess physical plausibility after generation, leaving an object's simulated response unused as feedback for repairing upstream errors. We present DiagGen, an agentic framework that turns a single in-the-wild image into a simulation-ready deformable asset through a generate–simulate–diagnose–refine loop. DiagGen constructs part-aware geometry and material parameters, then uses a VLM (
vision-
language model)-based
agent to select semantically informative regions, probe them in a physics simulator, observe material responses, and route evidence-backed repair cues to the responsible generation stage. Experiments on 40 assets show that diagnostics provides useful repair cues and can moderately improve the quality of generated deformable assets. Finally, we show that unlike assets generated from visual foundation models which may not be simulatable, DiagGen-generated deformables can be directly dropped into a high-fidelity physical simulator for the planning and simulation of contact-rich pick-and-place tasks. The project's website is https://diaggen.github.io/.
Read the full abstract ▾
All authors Guanxiong Chen, Yiduo Qu, Qianjun Xia, Pengyu Jing, Yixian Cheng, Bole Ma, Pengzhi Yang, Bingyang Zhou, Ziming Li, Shashwat Suri, Gongbo Sun, Chao Liu, Peter Yichen Chen, Ziqiu Zeng, Fan Shi
Robotics
Splat-CBF: Safe Next-Best-View Control in 3D Gaussian-Splat Maps
Amirhossein Mollaei Khass, Athanasios Cosse and Nader Motee
Where to look and how to move? A
robot navigating an unmapped environment must do both at once, and the two goals pull against each other. The regions most worth observing are the ones the map knows least about, and those are exactly where the
robot cannot trust its collision margins. We resolve this tension by introducing Splat-CBF, an active perception control barrier function that steers the camera toward the next best view while collision avoidance is enforced as a hard constraint. Safety is enforced by a risk-aware control barrier function that turns the Average Value-at-Risk of the Gaussian field into a single smooth hard constraint. Perception is enforced by a second barrier that rewards camera orientations with high expected Fisher information gain near the
robot's planned path. The two meet in a quadratic program where safety is hard and perception is soft, with a slack penalty that adapts to how often perception has already been relaxed and how close the
robot is to an uncertain region. We verify the method in indoor simulations, a Isaac Kinova manipulator and in experiments on an Ackermann-drive
robot. Our results assert that
robot navigates faster, gathers more information, and runs faster online than safety-only and perception-only baselines, giving up informative motion only when safety requires it.
Read the full abstract ▾
Robotics
On the Control of Mobile Ad-Hoc Agent Deployments in Partially Observed Space
Edwin Meriaux, Louis-Roy Langevin, Shuo Wen et al. (+3 more)
We study the online deployment of mobile ad hoc networks in unknown orthogonal environments, formalized as the Partially Observable Cooperative Guard Art Gallery Problem. We give a full proof that CADENCE
algorithms achieve full coverage while maintaining a connected visibility graph using at most $n/2 + h - 2$
agents in orthogonal worlds with $n$ corners and $h$ holes. We further evaluate deployment-order heuristics that reduce
agent count and deployment time in practice.
Read the full abstract ▾
All authors Edwin Meriaux, Louis-Roy Langevin, Shuo Wen, Ndiamé Ndiaye, Gregory Dudek, Antonio Loría
Robotics
Anatomy of a Closed-Loop Collapse: A Causal Case Study of a Compressed VLA Policy
Fengze Jia
Compressed manipulation policies can pass offline evaluation while failing in closed-loop execution; this dissociation is established in prior work and is not our claim. We contribute a causal anatomy of one naturally occurring case. An 8-layer
distillation of Octo-Base retains 86% of parameters, passes every offline check we applied (0.996 and 1.000 teacher-ratios on the family's own validation metrics), and collapses in closed loop: 0/72 vs. the teacher's 40/72 on a simulated WidowX pick-and-place task. The collapse is structured, not diffuse: early task stages degrade gradually (the student moves the object at 90% of the teacher's rate and grasps at 55%), while transport-to-target fails categorically, at 0% in every training variant. Paired action-trace forensics isolate the signature: a negative, late-heavy $z$ residual, roughly 10x its post-repair magnitude, and persistent across the base
distillation and both continuation branches. Four standard therapies fail under matched controls: continued training and in-domain offline data leave success at zero, even though the latter measurably improves marginal action statistics; command-level compensation recovers nothing at any offset, although the same perturbations degrade healthy policies; clamping the symptom in the command channel preserves grasping, yet success stays at floor. A minimal-pair intervention that substitutes half of the training stream with deployment-distribution teacher rollouts, with every other setting held fixed, restores parity with the teacher (18/36 vs. 17/36 held-out), eliminates that signature, and recovers a teacher-like perturbation-response profile. We claim existence, not universality. Operationally, offline gates, including a family's own validation metrics, are insufficient acceptance tests for compressed policies; a few dozen closed-loop trials sufficed to find what they missed.
Read the full abstract ▾
Robotics
Fast and Robust Temporal Logic Planning via ADMM-based Trajectory Optimization
Lukas Pries, Joris Verhagen, Jon Arrizabalaga et al. (+3 more)
We present a fast numerical method for safe continuous-time motion planning under Temporal Logic (TL) specifications. The method generates smooth continuous trajectories that remain collision-free while robustly satisfying temporal and logical task requirements. A central component of our method is the formulation of nonconvex safety and logic constraints as unions of convex sets where associated discrete decisions are encoded in a joint feasibility graph. This graph representation allows Euclidean projection onto the feasible set and proximal robustness maximization to be reformulated as shortest- and widest-path problems, respectively. Building on this structure, we develop a nonconvex splitting method based on the Alternating Direction Method of Multipliers (ADMM), which decouples smooth spatio-temporal trajectory optimization from nonsmooth discrete constraint handling within the optimization. The resulting
algorithm exhibits reliable convergence across
benchmarks and scales to large-scale motion-planning problems, providing a 4.7x average speedup over the state of the art on discrete and continuous-time logic problems.
Read the full abstract ▾
All authors Lukas Pries, Joris Verhagen, Jon Arrizabalaga, Jana Tumova, Markus Ryll, Zachary Manchester
Robotics · Cross-listed from cs.CV
M3GA-Wild: A Large-Scale Dataset and Benchmark for Multi-Modal Multi-session Ground-to-Aerial Place Recognition in Forests
Ethan Griffiths, Maryam Haghighat, Simon Denman et al. (+2 more)
We present M3GA-Wild, the first
benchmark for multi-modal, multi-session ground-to-aerial place recognition in forests. M3GA-Wild unifies and extends existing forest localisation
datasets, providing a holistic
benchmark with synchronised RGB imagery and
LiDAR from ground traversals spanning 36 km, aligned high-resolution aerial imagery and multi-altitude
LiDAR covering 370 hectares, and accurate geo-referenced 6-DoF poses for precise evaluation. M3GA-Wild captures diverse forest scenes with varying viewpoints, occlusion, and environmental conditions, enabling systematic evaluation of visual,
LiDAR, cross-modal, and multi-modal methods. Baseline experiments show that
LiDAR-based approaches significantly outperform
vision-only methods under severe viewpoint differences, while current multi-modal fusion strategies yield limited gains due to poor cross-modal
alignment. By pairing aerial RGB imagery with geo-referenced aerial
LiDAR, M3GA-Wild also enables evaluation of foundation models for monocular depth estimation as a cheap source of 3D geometry from forest imagery, with initial experiments revealing shortfalls of current methods. These results highlight key challenges in cross-platform localisation, including modality misalignment and severe domain gaps. M3GA-Wild establishes a new
benchmark to support research in robust multi-modal localisation and long-term autonomy in unstructured natural environments. The
dataset and code will be available upon acceptance.
Read the full abstract ▾
All authors Ethan Griffiths, Maryam Haghighat, Simon Denman, Clinton Fookes, Milad Ramezani
Robotics
Connectivity-Aware Exploration of Robotic Grasp Spaces
Maksim A Kazanskii
Robotic grasping is typically formulated as the problem of identifying successful actions from a space of candidate grasp poses. However, the organization of successful actions within this space has received less
attention. We study the multiscale structure of viable robotic grasps in $SE(3)$ and investigate whether this structure can be exploited for more efficient exploration. Using a large-scale grasp
dataset, we show that successful grasp sets exhibit heterogeneous and reproducible connectivity structure across objects. We then introduce a connectivity-aware sampling strategy that incrementally explores the currently observed grasp space by prioritizing potential bridges between components, structural frontiers, boundary extensions, and
geometric novelty. In controlled reconstruction experiments, the method recovers the connectivity structure of successful grasp sets substantially more efficiently than random sampling and farthest-point sampling. We further evaluate whether connectivity acquired under hidden grasp viability can improve subsequent grasp discovery, and whether structural experience from previously explored objects can be retrieved and transferred to unseen objects. These results suggest that the spatial organization of viable actions provides information relevant to grasp-space exploration beyond the viability of individual candidate actions. More broadly, they motivate structure-aware exploration as a means of exploiting the geometry of viable action spaces in robotic manipulation.
Read the full abstract ▾
Robotics
A Horizon-slicing Approach to Minimum Obstacle Displacement Planning for Robot Navigation
Antony Thomas, Giulio Ferro, Fulvio Mastrogiovanni et al. (+2 more)
In this paper, we investigate the Minimum Obstacle Displacement Planning problem from a
robot motion planning perspective. The problem involves determining a feasible path to a goal location by displacing movable obstacles when no collision-free path initially exists. We show that this problem is computationally challenging and, in particular, NP-hard when obstacles are modeled as polygons in the plane. Besides an exact formulation of the minimum obstacle displacement problem generalizing other problems in the literature, and the associated optimal solution, this paper proposes an approximate solution that is less intensive from a computational standpoint, and differs from the optimal solution by a fraction of the optimal cost, being able to trade-off between path length and amount of obstacle displacements.
Read the full abstract ▾
All authors Antony Thomas, Giulio Ferro, Fulvio Mastrogiovanni, Michela Robba, Marco Baglietto
Robotics
An Empirical Study and Open Testbed for Federated Fine-Tuning of Vision-Language-Action Models
Zhekai Duan, Kevin Ziyang Xie, Xinyu Tan et al. (+5 more)
Adapting a pretrained
Vision-
Language-Action (VLA) model to a new
robot, environment, or task requires demonstrations that are collected locally and often discarded.
Federated learning is a promising approach to exploiting such distributed demonstrations by learning a shared policy. However, whether it can adapt large pretrained VLAs remains an open question, and a lack of reproducible
benchmarks for pretrained VLAs and reusable training frameworks makes existing results difficult to compare. In this paper, we conduct a systematic study of
federated fine-tuning of three modern pretrained VLA policies on the 40 simulated tasks of the LIBERO manipulation
benchmark, and on six real-world tasks in two real-
robot experiments, with demonstrations collected across two and three sites, respectively. Our study analyzes the key choices in this setting, spanning multiple
federated parameter scopes, three aggregation
algorithms, and evaluation under distribution shift. Based on the study, we derive a series of lessons, including the dominance of the
federated scope over the choice of aggregation
algorithm and the difficulty of matching centralized fine-tuning on physical
robots, where cross-site heterogeneity is stronger than simulation captures. We also highlight opportunities for
federated VLA learning, such as the ability to match centralized fine-tuning on heterogeneous data, to remain at least as robust as centralized fine-tuning under distribution shift, and to personalize, with each client federating part of the policy and keeping the rest local, which helps where the policy's pretraining is weak but leaves no usable global model. We open-source \decentvla{}, the model- and runtime-agnostic testbed behind the study, to facilitate future research and fair comparisons in
federated VLA learning.
Read the full abstract ▾
All authors Zhekai Duan, Kevin Ziyang Xie, Xinyu Tan, Shikai Geng, Chengxu Zhou, Ramana Kompella, Gaowen Liu, Chris Xiaoxuan Lu
Robotics · Cross-listed from cs.CV
General Collaborative Intelligence: Architecting Cognition for Resilient Multi-Agent Ecosystems
Lei Zhang, Chun Ye, Le Yang et al. (+4 more)
Multi-
agent unmanned systems are moving from isolated, ego-centric sensing toward collaborative intelligence, in which distributed
agents exchange compact features to overcome a local observation trap that no single
agent can escape: occlusions, finite sensor range, and environmental degradation. The field has matured across architectural, communication, embodied, resilience, and trust dimensions, yet existing surveys examine these dimensions in isolation and rarely expose their dependencies. This review offers a unified synthesis through two complementary lenses. The first is a five-dimensional taxonomy spanning collaboration stage, communication paradigm, fusion architecture, learning strategy, and application domain. The second is three cognitive synergy conditions, Semantic Disambiguation, Pragmatic Information Exchange, and Proactive Informational Foraging, that turn cognitive synergy into operational criteria. Across these lenses we survey collaboration architectures and
topologies, neural-communication co-design that treats the channel as a differentiable pipeline component, embodied action-perception loops via multi-
agent reinforcement learning, and resilience mechanisms for synchronization, uncertainty quantification, and label-efficient learning. We then map these advances onto four operational domains, V2X, unmanned aerial, industrial logistics, and smart cities, and onto the safety-
privacy-utility triad. To counter
benchmark saturation and evaluation fragmentation, we propose GCI-Bench, a five-pillar scoring protocol with a maturity model that makes the trade-offs of collaborative methods comparable across studies. A critical reflection on reproducibility, the sim-to-real gulf, and conditions under which collaboration degrades performance identifies open challenges and charts directions toward general collaborative intelligence under real-world uncertainty.
Read the full abstract ▾
All authors Lei Zhang, Chun Ye, Le Yang, Zhaozhong Wang, Deng-Ping Fan, Hang Dai, Binglu Wang
Robotics
Transferring the Intelligence of VLMs to Robotic Control
Meng-Hao Guo, Zhe-Han Mo, Jia-Jun Wang et al. (+6 more)
Humans can seamlessly adapt to both physical and digital worlds, suggesting that while a digital-to-real gap exists in embodiment, environment and task, human intelligence itself may transfer across this gap. This naturally raises a fundamental question: can the intelligence of
vision-
language models (VLMs) similarly generalize from the digital world to the physical world for robotic control? We investigate this question through RoboDawn, a human-intuitive interface that exposes robotic control to an agentic VLM through a compact set of discrete translation, rotation, and gripper commands. Using this interface, the VLM controls a
robot in a closed loop: it observes the current visual state, reasons about the next action, executes it, and adapts subsequent decisions to the resulting state. Furthermore, we introduce an in-context learning (ICL) scheme that uses a few demonstrations to ground the VLM in both interface usage and task-solving strategies. Experiments on RoboTwin 2.0 C2R and RoboDojo demonstrate that RoboDawn achieves strong performance without task-specific
robot training. In the zero-shot setting, RoboDawn outperforms several strong policies trained on benchmarkspecific
robot data, while a single in-context demonstration further yields substantial performance gains and establishes state-of-the-art (SOTA) results. On RoboTwin 2.0 C2R, the success rate increases from 53.2% zero-shot to 73.6% one-shot, exceeding the solid baseline {\pi}0.5 (46.0%). Similar gains are observed on RoboDojo, where success rate improves from 35.67% zero-shot to 47.17% one-shot. The same framework also transfers to real-world
robots, performing block-in-basket and block stacking on Franka.
Read the full abstract ▾
All authors Meng-Hao Guo, Zhe-Han Mo, Jia-Jun Wang, Yi Zhang, Kejin Wang, Yi-Xuan Deng, Jia-Peng Zhang, Yongming Rao, Shi-Min Hu
Robotics
Prescribed-Time Contracting-Boundary Control of a Tendon-Driven Flexible Arm
Yi Lu, Chao Tang, Zhiji Han et al. (+1 more)
This study develops a prescribed-time performance-shaping control method for
curvature tracking of a single-segment flexible arm actuated by three antagonistic tendon pairs. A Cartesian
curvature representation is introduced to avoid the undefined bending direction at the straight configuration and to establish an explicit six-tendon kinematic mapping. A cubic performance boundary contracts smoothly from an initially admissible error bound to a nonzero terminal accuracy bound within a prescribed time. Based on this boundary, a dual transformation combining static symmetric error scaling and time-varying behavior shaping maps the tracking error into a fixed unit box. The resulting controller guarantees boundary invariance, prescribed-time entry into the terminal accuracy region, and subsequent asymptotic convergence. Numerical evaluations with Python and OpenCR–MuJoCo, together with a supervised reduced-order experiment on a two-section, four-channel platform, provide complementary validation. Across six experimental trials, no violation of the prescribed boundary is observed, and the proposed controller reduces the mean terminal
curvature RMSE by 32.5% relative to a matched baseline, with comparable terminal-band entry times. These results support the feasibility of the proposed approach in the reduced-order experimental setting.
Read the full abstract ▾
All authors Yi Lu, Chao Tang, Zhiji Han, Hongdu Wang
Robotics
MIRA-PRM: Mission-Informed Reusable Roadmap Planning for Mobile Gas-Sensing Inspection
Gongsen Wang, Siyuan Wang, Xinyuan Wang et al. (+4 more)
Mobile gas-sensing inspection requires a mobile platform to reliably reach ordered sampling poses. We present MIRA-PRM, a mission-informed probabilistic roadmap that combines geometry-, mission-flow-, and inspection-conditioned sampling with locally adaptive connectivity and evidence-gated refinement. Eight development campaigns comprising 142,448 planner runs evaluated mission scaling, repeated-query adaptation, finite node budgets, target distributions, local map repair, ablation, cross-family comparison, and parameter sensitivity. MIRA-PRM maintained 100% seed-level success as ordered targets increased from two to eight, and achieved 97.67% seed-level success under a 40-node ceiling, compared with 67.17% for PRM and 86.50% for PRM*. Local repair reduced event time by 79.9-81.9% and node additions by 90.8-94.3% relative to rebuilding. On a separate frozen holdout of 12 maps, 48 ordered missions, and three seeds per map-mission unit, MIRA-PRM produced 96/144 validated successes, versus 82/144 for PRM, 89/144 for PRM*, and 70/144 for LE-HG-PRM. On paired common-success runs, MIRA-PRM reduced wall time by 16.89% relative to PRM* and 81.01% relative to LE-HG-PRM, but was 52.23% slower than PRM; paths were 0.54-1.86% longer. All validated paths passed independent full-segment checks and clearance non-inferiority. The results support a reliability-oriented trade-off in structured simulations and motivate external-map and physical mobile-sensing validation.
Read the full abstract ▾
All authors Gongsen Wang, Siyuan Wang, Xinyuan Wang, Yuhan Wen, Feng Yang, Zhen Tian, Hoi Leong Lee
Robotics
Embodied Snap: Octopus-Inspired Distributed Reach-and-Attach with a Speed-Limited Soft Arm
Linxin Hou, Zhihang Qin, Heyang Zou et al. (+5 more)
Reach-and-attach of soft robotic arms with passive suction requires accurate targeting and sufficient contact speed, yet geared actuators can impose a speed limit that improved trajectory tracking alone cannot overcome. This paper proposes an embodied snap controller that separates slow servo-driven preloading from rapid elastic release, enabling a compliant arm to move beyond its direct tendon-driven speed limit. Octopus biology motivates the controller's section-wise organizational prior, rather than reproduction of the octopus nervous system. A learned policy shared across three sections selects preloads, aim, tendon slack, and release timing, determining where, how, and when to load and release the body. The policy is optimized offline using a hardware-validated recurrent model within experimentally supported bounds. Across five optimization seeds and 400 unseen simulated targets, attachment success is $(73\pm4)\%$ at a $5\text{ cm}$ lateral tolerance, and the shared policy reaches the matched centralized controller's mean final reward after a median $17\%$ of the common evaluation budget. Hardware characterization achieves tip speeds of 1.56-1.64 m/s, at least $108\%$ above direct tendon-driven release. In 18 open-loop hardware trials across six placements, 17 exceed the 1 m/s snap threshold and nine retrieve the object, with successful
retrieval at five placements. These results demonstrate a practical division of responsibility in the control problem: learned control prepares the body, and passive body mechanics execute the rapid movement needed for dynamic reach-and-attach.
Read the full abstract ▾
All authors Linxin Hou, Zhihang Qin, Heyang Zou, Qirui Wu, Peiyi Wang, Muhammad Sunny Nazeer, Yongxin Guo, Cecilia Laschi
Robotics
"Dear LLaVA, Please Drive": A Depth-Aware Vision-Language Agent for Closed-Loop Robotic Control
Sebastian Berger, Katharina Winter and Fabian B. Flohr
Vision-
language models (VLMs) provide a compelling foundation for
reasoning-driven mobile navigation, offering rich contextual understanding and strong generalization from large-scale pretraining. Most existing navigation frameworks rely on imitation learning and therefore require substantial labeled trajectory data, limiting their scalability and robustness. In this work, we propose a parameter-efficient approach to fine-tune a pretrained VLM for autonomous navigation using an Imperative Learning paradigm. By optimizing against differentiable
geometric cost fields rather than labeled trajectories, our model learns to generate collision-free paths exclusively from stereoscopic depth observations. We introduce a unified end-to-end navigation pipeline for natural-
language-driven robotic control. This system leverages a shared VLM backbone with task-specific Low-Rank Adaptation (LoRA) modules, effectively bridging the gap from semantic target selection to low-level trajectory planning. Our approach achieves competitive Success weighted by Path Length (SPL) in unseen environments while updating less than 1% of the model's total parameters. Qualitative real-world experiments validate sim-to-real generalization and stable path planning without fine-tuning on real-world data. These results highlight a practical approach for deploying VLM-based
agents on mobile
robots, enabling high-level semantic navigation without the prohibitive requirement for large-scale, labeled trajectory data.
Read the full abstract ▾
Robotics · Cross-listed from cs.LG
Computationally efficient safe exploration in reinforcement learning
Shreeram Murali, Shankar A. Deka and Dominik Baumann
Reinforcement learning in real-life applications requires safety guarantees during exploration. Typical reinforcement learning
algorithms do not provide such guarantees, and many modifications that do rely on Gaussian processes (GPs), which have a large computational cost. We propose a computationally lightweight
algorithm based on the Nadaraya-Watson
estimator that safely explores and optimizes constrained Markov decision processes (MDPs). Our
algorithm, CoLSafe-MDP, uses an
estimator that scales in constant-time with bounds on the estimates, a significant improvement from its GP-based counterparts that scale cubically with the number of data points. We then evaluate its performance in a grid-based environment and on observational Martian terrain data.
Read the full abstract ▾
Robotics
H-VLA: Hierarchical Vision-Language-Action Model with Key-Action Reasoning and Motion Planning in a Unified Action Space
Xiongfeng Peng, Lu Xu, Yandong Wang et al. (+10 more)
Vision-
Language-Action (VLA) models have shown strong potential for robotic manipulation, but many existing methods still rely on direct mappings from
language and visual observations to dense actions. This formulation can weaken the semantic
reasoning capability inherited from pre-trained
Vision-
Language Models (VLMs), which are mainly optimized for visual-linguistic understanding rather than low-level control, and becomes fragile under spatial variations, including changes in object positions, scene layouts,
robot embodiments, and camera viewpoints. To address these limitations, we propose H-VLA, a hierarchical VLA framework that decouples high-level key-action
reasoning from low-level motion generation. H-VLA combines a Key-Action Model for predicting a key-action as the next manipulation subgoal, a Motion Planning Model for generating dense future actions conditioned on the predicted key-action, and a Unified Camera-Centric Action Space for consistent representation across
datasets, embodiments, and viewpoints. We further adopt a two-stage training strategy that emphasizes key-action
reasoning during pre-training and dense motion generation during fine-tuning. Experiments show that H-VLA achieves strong performance on SimplerEnv, reaching 91% on Google
Robot visual matching, 84% on Google
Robot variant aggregation, and 81% on WidowX visual matching. On Agilex real-
robot tasks, H-VLA improves over the strongest baseline by 10, 47, and 16 percentage points under in-distribution, out-of-distribution position, and out-of-distribution scene/object settings, respectively.
Read the full abstract ▾
All authors Xiongfeng Peng, Lu Xu, Yandong Wang, Jiaqian Yu, Zirui Zheng, Yamin Mao, Weiming Li, Inseop Chung, Hyun-woong Cho, Jaewook Yoo, Dongwook Lee, Daehyun Ji, Chao Zhang