Robotics
Nastaran Darabi and Amit Ranjan Trivedi
Vision language action (VLA) models enable generalist robotic
agents but often exhibit
language ignorance, relying on visual shortcuts and remaining insensitive to instruction changes. We present Prospective Grounding and
Alignment VLA (ProGAL-VLA), which constructs a 3D entity-centric graph (GSM), uses a slow planner to produce symbolic sub-goals, and aligns them with grounded entities via a Grounding
Alignment Contrastive (GAC) loss. All actions are conditioned on a verified goal
embedding $g_t$, whose
attention entropy provides an intrinsic ambiguity signal. On LIBERO-Plus, ProGAL-VLA increases robustness under
robot perturbations from 30.3 to 71.5 percent, reduces
language ignorance by 3x-4x, and improves entity
retrieval from 0.41 to 0.71 Recall@1. On the Custom Ambiguity
Benchmark, it reaches AUROC 0.81 (vs. 0.52), AUPR 0.79, and raises clarification on ambiguous inputs from 0.09 to 0.81 without harming unambiguous success. The verification bottleneck increases mutual information of
language-actions, the GAC loss imposes an entity-level InfoNCE bound, and
attention entropy yields calibrated selective prediction, indicating that explicit verified grounding is an effective path toward instruction-sensitive, ambiguity-aware
agents.
Read the full abstract ▾
Robotics
Kinematics of continuum planar grasping
Udit Halder, Nicolas Echeverria Zambrano and Xincheng Li
This paper presents an analytical framework to study the geometry arising when a soft continuum arm grasps a planar object. Both the arm centerline and the object boundary are modeled as smooth curves. The grasping problem is formulated as a kinematic boundary following problem, in which the object boundary acts as the arm's 'shadow curve'. This formulation leads to a set of reduced kinematic equations expressed in terms of relative
geometric shape variables, with the arm
curvature serving as the control input. An optimal control problem is formulated to determine feasible arm shapes that achieve optimal grasping configurations, and its solution is obtained using Pontryagin's Maximum Principle. Based on the resulting optimal grasp kinematics, a class of continuum grasp quality metrics is proposed using the algebraic properties of the associated continuum grasp map. Feedback control aspects in the dynamic setting are also discussed. The proposed methodology is illustrated through systematic numerical simulations.
Read the full abstract ▾
Robotics · Cross-listed from eess.SY
On Feedback Speed Control for a Planar Tracking
Xincheng Li, Tengyue Liu and Udit Halder
This paper investigates a planar tracking problem between a leader and follower
agent. We propose a novel feedback speed control law, paired with a constant bearing steering strategy, to maintain an abreast formation between the two
agents. We prove that the proposed control yields asymptotic stability of the closed-loop system when the steering of the leader is known. For the case when the leader's steering is unavailable to the follower, we show that the system is still input-to-state stable with respect to the leader's steering viewed as an input. Furthermore, we demonstrate that if the leader's steering is periodic, the follower will asymptotically converge to a periodic orbit with the same period. We validate these results through numerical simulations and experimental implementations on mobile
robots. Finally, we demonstrate the scalability of the proposed approach by extending the two-
agent control law to an N-
agent chain network, illustrating its implications for directional information propagation in biological and engineered flocks.
Read the full abstract ▾
Robotics
Spectral Kernel Dynamics via Maximum Caliber: Fixed Points, Geodesics, and Phase Transitions
Jnaneshwar Das
We derive a closed-form
geometric functional for kernel dynamics on finite graphs by applying the Maximum Caliber (MaxCal) variational principle to the spectral transfer function h(lambda) of the graph
Laplacian eigenbasis. The main result is that the MaxCal stationarity condition decouples into N one-dimensional problems with explicit solution: h*(lambda_l) = h_0(lambda_l) exp(-1 - T_l[h*]), yielding self-consistent (fixed-point) kernels via exponential tilting (Corollary 1), log-linear Fisher-Rao geodesics (Corollary 2), a diagonal Hessian stability criterion (Corollary 3), and an l^2_+ isometry for the spectral kernel space (Proposition 3). The spectral
entropy H[h_t] provides a computable O(N) early-warning signal for network-structural phase transitions (Remark 7). All claims are numerically verified on the path graph P_8 with a Gaussian mutual-information source, using the open-source kernelcal library. The framework is grounded in a structural analogy with Einstein's field equations, used as a guiding template rather than an established equivalence; explicit limits are stated in Section 6.
Read the full abstract ▾
Robotics
Physics-Informed Reinforcement Learning of Spatial Density Velocity Potentials for Map-Free Racing
Shathushan Sivashangaran, Apoorva Khairnar, Sepideh Gohari et al. (+2 more)
Autonomous racing without prebuilt maps is a grand challenge for embedded robotics that requires kinodynamic planning from instantaneous sensor data at the acceleration and tire friction limits. Out-Of-Distribution (OOD) generalization to various racetrack configurations utilizes Machine Learning (ML) to encode the mathematical relation between sensor data and vehicle actuation for end-to-end control, with implicit localization. These comprise Behavioral Cloning (BC) that is capped to human reaction times and Deep Reinforcement Learning (DRL) which requires large-scale collisions for comprehensive training that can be infeasible without simulation but is arduous to transfer to reality, thus exhibiting greater performance than BC in simulation, but actuation instability on hardware. This paper presents a DRL method that parameterizes nonlinear vehicle dynamics from the spectral distribution of depth measurements with a non-
geometric, physics-informed reward, to infer vehicle time-optimal and overtaking racing controls with an Artificial Neural Network (ANN) that utilizes less than 1% of the computation of BC and model-based DRL. Slaloming from simulation to reality transfer and variance-induced conservatism are eliminated with the combination of a physics engine exploit-aware reward and the replacement of an explicit collision penalty with an implicit truncation of the value horizon. The policy outperforms human demonstrations by 12% in OOD tracks on proportionally scaled hardware, by maximizing the friction circle with tire dynamics that resemble an empirical Pacejka tire model. System identification illuminates a functional bifurcation where the first layer compresses spatial observations to extract digitized track features with higher resolution in corner apexes, and the second encodes nonlinear dynamics.
Read the full abstract ▾
All authors Shathushan Sivashangaran, Apoorva Khairnar, Sepideh Gohari, Vihaan Dutta, Azim Eskandarian
Robotics
Sim-to-Real Transfer for Muscle-Actuated Robots via Generalized Actuator Networks
Jan Schneider, Mridul Mahajan, Le Chen et al. (+4 more)
Tendon drives paired with soft muscle actuation enable faster and safer
robots while potentially accelerating skill acquisition. Still, these systems are rarely used in practice due to inherent nonlinearities, friction, and hysteresis, which complicate modeling and control. So far, these challenges have hindered policy transfer from simulation to real systems. To bridge this gap, we propose a sim-to-real pipeline that learns a neural network model of this complex actuation and leverages established rigid body simulation for the arm dynamics and interactions with the environment. Our method, called Generalized Actuator Network (GenAN), enables actuation model identification across a wide range of
robots by learning directly from joint position trajectories rather than requiring torque sensors. Using GenAN on PAMY2, a tendon-driven
robot powered by pneumatic artificial muscles, we successfully deploy dynamic but precise goal-reaching, ball-in-a-cup, and table tennis policies, trained entirely in simulation. To the best of our knowledge, this result constitutes the first successful sim-to-real transfer for a four-degrees-of-freedom muscle-actuated
robot arm.
Read the full abstract ▾
All authors Jan Schneider, Mridul Mahajan, Le Chen, Simon Guist, Bernhard Schölkopf, Ingmar Posner, Dieter Büchler
Robotics · Cross-listed from cs.CV
Incremental Semantics-Aided Meshing from LiDAR-Inertial Odometry and RGB Direct Label Transfer
Muhammad Affan, Ville Lehtola and George Vosselman
Geometric high-fidelity mesh reconstruction from
LiDAR-inertial scans remains challenging in large, complex indoor environments – such as cultural buildings – where point cloud sparsity,
geometric drift, and fixed fusion parameters produce holes, over-smoothing, and spurious surfaces at structural boundaries. We propose a modular, incremental RGB+
LiDAR pipeline that generates incremental semantics-aided high-quality meshes from indoor scans through scan frame-based direct label transfer. A
vision foundation model labels each incoming RGB frame; labels are incrementally projected and fused onto a
LiDAR-inertial odometry map; and an incremental semantics-aware Truncated Signed Distance Function (TSDF) fusion step produces the final mesh via marching cubes. This frame-level fusion strategy preserves the
geometric fidelity of
LiDAR while leveraging rich visual semantics to resolve
geometric ambiguities at reconstruction boundaries caused by
LiDAR point-cloud sparsity and
geometric drift. We demonstrate that semantic guidance improves
geometric reconstruction quality; quantitative evaluation is therefore performed using
geometric metrics on the Oxford Spires
dataset, while results from the NTU VIRAL
dataset are analyzed qualitatively. The proposed method outperforms state-of-the-art
geometric baselines ImMesh and Voxblox, demonstrating the benefit of semantics-aided fusion for
geometric mesh quality. The resulting semantically labelled meshes are of value when reconstructing Universal Scene Description (USD) assets, offering a path from indoor
LiDAR scanning to XR and digital modeling.
Read the full abstract ▾
Robotics
SafeMind: A Risk-Aware Differentiable Control Framework for Adaptive and Safe Quadruped Locomotion
Zukun Zhang, Kai Shu and Mingqiao Mo
Learning-based quadruped controllers achieve impressive agility but typically lack formal safety guarantees under model uncertainty, perception noise, and unstructured contact conditions. We introduce SafeMind, a differentiable
stochastic safety-control framework that unifies probabilistic Control Barrier Functions with semantic context understanding and meta-adaptive risk calibration. SafeMind explicitly models epistemic and aleatoric uncertainty through a variance-aware barrier constraint embedded in a differentiable quadratic program, thereby preserving gradient flow for end-to-end training. A semantics-to-constraint encoder modulates safety margins using perceptual or
language cues, while a meta-adaptive learner continuously adjusts risk sensitivity across environments. We provide theoretical conditions for probabilistic forward invariance, feasibility, and stability under
stochastic dynamics. SafeMind is deployed on Unitree A1 and ANYmal C at 200~Hz and validated across 12 terrain types, dynamic obstacles, morphology perturbations, and semantically defined tasks. Experiments show that SafeMind reduces safety violations by 3–10x and energy consumption by 10–15% relative to state-of-the-art CBF, MPC, and hybrid RL baselines, while maintaining real-time control performance.
Read the full abstract ▾
Robotics
Adaptor: Advancing Assistive Teleoperation with Few-Shot Learning and Cross-Operator Generalization
Yu Liu, Yihang Yin, Tianlv Huang et al. (+8 more)
Assistive teleoperation enhances efficiency via shared control, yet inter-operator variability, stemming from diverse habits and expertise, induces highly heterogeneous trajectory distributions that undermine intent recognition stability. We present Adaptor, a few-shot framework for robust cross-operator intent recognition. The Adaptor bridges the domain gap through two stages: (i) preprocessing, which models intent uncertainty by synthesizing trajectory perturbations via noise injection and performs geometry-aware keyframe extraction; and (ii) policy learning, which encodes the processed trajectories with an Intention Expert and fuses them with the pre-trained
vision-
language model context to condition an Action Expert for action generation. Experiments on real-world and simulated
benchmarks demonstrate that Adaptor achieves state-of-the-art performance, improving success rates and efficiency over baselines. Moreover, the method exhibits low variance across operators with varying expertise, demonstrating robust cross-operator generalization.
Read the full abstract ▾
All authors Yu Liu, Yihang Yin, Tianlv Huang, Fei Yan, Yuan Xu, Weinan Hong, Wei Han, Yue Cao, Xiangyu Chen, Zipei Fan, Xuan Song
Robotics
Musculoskeletal Motion Imitation for Learning Personalized Exoskeleton Control Policy in Impaired Gait
Itak Choi, Ilseung Park, Eni Halilaj et al. (+1 more)
Designing generalizable control policies for lower-limb exoskeletons remains fundamentally constrained by exhaustive data collection or iterative optimization procedures, which limit accessibility to clinical populations. To address this challenge, we introduce a device-agnostic framework that combines physiologically plausible musculoskeletal simulation with reinforcement learning to enable scalable personalized exoskeleton assistance for both able-bodied and clinical populations. Our control policies not only generate physiologically plausible locomotion dynamics but also capture clinically observed compensatory strategies under targeted muscular deficits, providing a unified computational model of both healthy and pathological gait. Without task-specific tuning, the resulting exoskeleton control policies produce assistive torque profiles at the hip and ankle that align with state-of-the-art profiles validated in human experiments, while consistently reducing metabolic cost across walking speeds. For simulated impaired-gait models, the learned control policies yield asymmetric, deficit-specific exoskeleton assistance that improves both energetic efficiency and bilateral kinematic
symmetry without explicit prescription of the target gait pattern. These results demonstrate that physiologically plausible musculoskeletal simulation via reinforcement learning can serve as a scalable foundation for personalized exoskeleton control across both able-bodied and clinical populations, eliminating the need for extensive physical trials.
Read the full abstract ▾
All authors Itak Choi, Ilseung Park, Eni Halilaj, Inseung Kang
Robotics · Cross-listed from cs.CV
PhysInOne: Visual Physics Learning and Reasoning in One Suite
Siyuan Zhou, Hejun Wang, Hu Cheng et al. (+36 more)
We present PhysInOne, a large-scale synthetic
dataset addressing the critical scarcity of physically-grounded training data for AI systems. Unlike existing
datasets limited to merely hundreds or thousands of examples, PhysInOne provides 2 million
videos across 153,810 dynamic 3D scenes, covering 71 basic physical phenomena in mechanics, optics, fluid dynamics, and magnetism. Distinct from previous works, our scenes feature multiobject interactions against complex backgrounds, with comprehensive ground-truth annotations including 3D geometry, semantics, dynamic motion, physical properties, and text descriptions. We demonstrate PhysInOne's efficacy across four emerging applications: physics-aware
video generation, long-/short-term future frame prediction, physical property estimation, and motion transfer. Experiments show that fine-tuning foundation models on PhysInOne significantly enhances physical plausibility, while also exposing critical gaps in modeling complex physical dynamics and estimating intrinsic properties. As the largest
dataset of its kind, orders of magnitude beyond prior works, PhysInOne establishes a new
benchmark for advancing physics-grounded world models in generation, simulation, and
embodied AI.
Read the full abstract ▾
All authors Siyuan Zhou, Hejun Wang, Hu Cheng, Jinxi Li, Dongsheng Wang, Junwei Jiang, Yixiao Jin, Jiayue Huang, Shiwei Mao, Shangjia Liu, Yafei Yang, Hongkang Song, Shenxing Wei, Zihui Zhang, Peng Huang, Shijie Liu, Zhengli Hao, Hao Li, Yitian Li, Wenqi Zhou, Zhihan Zhao, Zongqi He, Hongtao Wen, Shouwang Huang, Peng Yun, Bowen Cheng, Pok Kazaf Fu, Wai Kit Lai, Jiahao Chen, Kaiyuan Wang, Zhixuan Sun, Ziqi Li, Haochen Hu, Di Zhang, Chun Ho Yuen, Bing Wang, Zhihua Wang, Chuhang Zou, Bo Yang
Robotics · Cross-listed from eess.SY
Decentralized Opinion-Integrated Decision making at Unsignalized Intersections via Signed Networks
Bhaskar Varma, Ying Shuai Quan, Karl D. von Ellenrieder et al. (+1 more)
In this letter, we consider the problem of decentralized decision making among connected autonomous vehicles at unsignalized intersections, where existing centralized approaches do not scale gracefully under mixed maneuver intentions and coordinator failure. We propose a closed-loop opinion-dynamic decision model for intersection coordination, where vehicles exchange intent through dual signed networks: a conflict
topology based communication network and a commitment-driven belief network that enable cooperation without a centralized coordinator. Continuous opinion states modulate velocity optimizer weights prior to commitment; a closed-form predictive feasibility gate then freezes each vehicle's decision into a GO or YIELD commitment, which propagates back through the belief network to pre-condition neighbor behavior ahead of physical conflicts. Crossing order emerges from
geometric feasibility and arrival priority without the use of joint optimization or a solver. The approach is validated across three scenarios spanning fully competitive, merge, and mixed conflict
topologies. The results demonstrate collision-free coordination and lower last-vehicle exit times compared to first come first served (FCFS) in all conflict non-trivial configurations.
Read the full abstract ▾
All authors Bhaskar Varma, Ying Shuai Quan, Karl D. von Ellenrieder, Paolo Falcone
Robotics
VAG: Dual-Stream Video-Action Generation for Embodied Data Synthesis
Xiaolei Lang, Yang Wang, Yukun Zhou et al. (+10 more)
Recent advances in
robot foundation models trained on large-scale human teleoperation data have enabled
robots to perform increasingly complex real-world tasks. However, scaling these systems remains difficult because collecting task-specific demonstrations is expensive and labor-intensive. Synthetic data, especially generated
videos, offer a promising direction, but existing World Models (WMs) are not directly suitable for policy learning since they do not provide paired action trajectories. World-Action (WA) models partially address this by predicting actions with visual outputs, yet often lack strong
video-action
alignment, while two-stage pipelines that generate
video first and then infer actions introduce inefficiency and error accumulation. To address these limitations, we propose VAG, a unified flow-matching-based dual-stream framework that jointly generates
video and action under visual and
language conditioning. By synchronizing denoising in both branches and using an adaptive 3D pooling mechanism to transfer compact global
video context to the action branch, VAG improves cross-modal consistency during generation. Across both simulated and real-world settings, VAG produces aligned
video-action pairs with competitive prediction quality, supports executable trajectory replay, and provides useful synthetic pretraining data that improves downstream policy generalization, indicating its potential as a practical world-action model for
embodied data synthesis.
Read the full abstract ▾
All authors Xiaolei Lang, Yang Wang, Yukun Zhou, Chaojun Ni, Kerui Li, Jiagang Zhu, Tianze Liu, Jiajun Lv, Xingxing Zuo, Yun Ye, Guan Huang, Xiaofeng Wang, Zheng Zhu
Robotics
Multimodal Anomaly Detection for Human-Robot Interaction
Guilherme Ribeiro, Iordanis Antypas, Leonardo Bizzaro et al. (+2 more)
Ensuring safety and reliability in human-
robot interaction (HRI) requires the timely detection of unexpected events that could lead to system failures or unsafe behaviours. Anomaly detection thus plays a critical role in enabling
robots to recognize and respond to deviations from normal operation during collaborative tasks. While reconstruction models have been actively explored in HRI, approaches that operate directly on feature vectors remain largely unexplored. In this work, we propose MADRI, a framework that first transforms
video streams into semantically meaningful feature vectors before performing reconstruction-based anomaly detection. Additionally, we augment these visual feature vectors with the
robot's internal sensors' readings and a Scene Graph, enabling the model to capture both external anomalies in the visual environment and internal failures within the
robot itself. To evaluate our approach, we collected a custom
dataset consisting of a simple pick-and-place robotic task under normal and anomalous conditions. Experimental results demonstrate that reconstruction on
vision-based feature vectors alone is effective for detecting anomalies, while incorporating other modalities further improves detection performance, highlighting the benefits of
multimodal feature reconstruction for robust anomaly detection in human-
robot collaboration.
Read the full abstract ▾
All authors Guilherme Ribeiro, Iordanis Antypas, Leonardo Bizzaro, João Bimbo, Nuno Cruz Garcia
Robotics
Robust Adaptive Backstepping Impedance Control of Robots in Unknown Environments
Reza Nazmara, Alap Kshirsagar, Jan Peters et al. (+1 more)
This paper presents a Robust Adaptive Backstepping Impedance Control (RABIC) strategy for
robots operating in contact-rich and uncertain environments. The proposed control strategy considers the complete coupled dynamics of the system and explicitly accounts for key sources of uncertainty, including external disturbances and unmodeled dynamics, while not requiring the
robot's dynamic parameters in implementation. We propose a backstepping-based adaptive impedance control scheme for the inner loop to track the reference impedance model. To handle uncertainties, we employ a Taylor series-based
estimator for system dynamics and an adaptive
estimator for determining the upper bound of external forces. Stability analysis demonstrates the semi-global practical finite-time stability of the overall system. To demonstrate the effectiveness of the proposed method, a simulated mobile manipulator scenario and experimental evaluations on a real Franka Emika Panda
robot were conducted. The proposed approach exhibits safer performance compared to PD control while ensuring trajectory tracking and force monitoring. Overall, the RABIC framework provides a solid basis for future research on adaptive and learning-based impedance control for coupled mobile and fixed serially linked manipulators.
Read the full abstract ▾
All authors Reza Nazmara, Alap Kshirsagar, Jan Peters, A. Pedro Aguiar
Robotics
Online Intention Prediction via Control-Informed Learning
Tianyu Zhou, Zihao Liang, Zehui Lu et al. (+1 more)
This paper presents an online intention prediction framework for estimating the goal state of autonomous systems in real time, even when intention is time-varying, and system dynamics or objectives include unknown parameters. The problem is formulated as an inverse optimal control / inverse reinforcement learning task, with the intention treated as a parameter in the objective. A shifting horizon strategy discounts outdated information, while online control-informed learning enables efficient gradient computation and online parameter updates. Simulations under varying noise levels and hardware experiments on a quadrotor drone demonstrate that the proposed approach achieves accurate, adaptive intention prediction in complex environments.
Read the full abstract ▾
All authors Tianyu Zhou, Zihao Liang, Zehui Lu, Shaoshuai Mou
Robotics
A Benchmark of Dexterity for Anthropomorphic Robotic Hands
Davide Liconti, Yuning Zhou, Yasunori Toshimitsu et al. (+2 more)
Dexterity is a central yet ambiguously defined concept in the design and evaluation of anthropomorphic robotic hands. In practice, the term is often used inconsistently, with different systems evaluated under disparate criteria, making meaningful comparisons across designs difficult. This highlights the need for a unified, performance-based definition of dexterity grounded in measurable outcomes rather than proxy metrics. In this work, we introduce POMDAR, a comprehensive dexterity
benchmark that formalizes dexterity as task performance across a structured set of manipulation and grasping motions. The
benchmark was systematically derived from established taxonomies in human motor control. It is implemented in both real-world and simulation and includes four manipulation configurations: vertical and horizontal configurations, continuous rotation, and pure grasping. The task designs contain mechanical scaffolding to constrain task motion, suppress compensatory strategies, and enable metrics to be measured unambiguously. We define a quantitative scoring metric combining task correctness and execution speed, effectively measuring dexterity as throughput. This enables objective, reproducible, and interpretable evaluation across different hand designs. POMDAR provides an open-source, standardized, and taxonomy-grounded
benchmark for consistent comparison and evaluation of anthropomorphic
robot hands to facilitate a systematic advancement of dexterous manipulation platforms. CAD, simulation files, and evaluation
videos are publicly available at https://srl-ethz.github.io/POMDAR/.
Read the full abstract ▾
All authors Davide Liconti, Yuning Zhou, Yasunori Toshimitsu, Ronan Hinchet, Robert K. Katzschmann
Robotics
Characterizing Lidar Range-Measurement Ambiguity due to Multiple Returns
Jason H. Rife and Yifan Li
Reliable position and attitude sensing is critical for highly automated vehicles that operate on conventional roadways.
Lidar sensors are increasingly incorporated into pose-estimation systems. Despite its great utility,
lidar is a complex sensor, and its performance in roadway environments is not yet well understood. For instance, it is often assumed in
lidar-localization
algorithms that a
lidar will always identify a unique surface along a given raypath. However, this assumption is not always true, as ample prior evidence exists to suggest that
lidar units may generate measurements probabilistically when more than one scattering surface appears within the
lidar's conical beam. In this paper, we analyze
lidar datasets to characterize cases with probabilistic returns along particular raypaths. Our contribution is to present representative cumulative distribution functions (CDFs) for raypaths observed by two different mechanically rotating
lidar units with stationary bases. In subsequent discussion, we outline a qualitative methodology to assess the effect of probabilistic multi-return cases on
lidar-based localization.
Read the full abstract ▾
Robotics
Soft Electroadhesive Feet for Micro Aerial Robots Perching on Smooth and Curved Surfaces
Chen Liu, Sonu Feroz and Ketao Zhang
Electroadhesion (EA) provides electrically switchable adhesion and is a promising mechanism for perching micro aerial
robots on smooth surfaces. However, practical implementations of soft and stretchable EA pads for aerial perching remain limited. This work presents (i) an efficient workflow for fabricating soft, stretchable electroadhesive pads with sinusoidal wave and concentric-circle electrodes in multiple sizes, (ii) a controlled experimental comparison of normal and shear adhesion under inactive (0 kV) and active (4.8 kV) conditions using an Instron-based setup, and (iii) a perching demonstration using a Crazyflie quadrotor equipped with electroadhesive feet on flat and curved substrates. Experimental results show that shear adhesion dominates, reaching forces on the order of 3 N with partial pad contact, while normal adhesion is comparatively small and strongly dependent on substrate properties. The Crazyflie prototype demonstrates repeatable attachment on smooth plastic surfaces, including curved geometries, as well as rapid detachment when the voltage is removed. These results highlight the potential of soft electroadhesive feet for lightweight and reliable perching in micro aerial vehicles (MAVs).
Read the full abstract ▾
Robotics · Cross-listed from cs.MM
2D or 3D: Who Governs Salience in VLA Models? – Tri-Stage Token Pruning Framework with Modality Salience Awareness
Zihao Zheng, Sicheng Tian, Zhihao Mao et al. (+8 more)
Vision-
Language-Action (VLA) models have emerged as the mainstream of
embodied intelligence. Recent VLA models have expanded their input modalities from 2D-only to 2D+3D paradigms, forming multi-visual-modal VLA (MVLA) models. Despite achieving improved spatial perception, MVLA faces a greater acceleration demand due to the increased number of input
tokens caused by modal expansion.
Token pruning is an effective optimization methods tailored to MVLA models. However, existing
token pruning schemes are designed for 2D-only VLA models, ignoring 2D/3D modality salience differences. In this paper, we follow the application process of multi-modal data in MVLA models and develop a tri-stage analysis to capture the discrepancy and dynamics of 2D/3D modality salience. Based on these, we propose a corresponding tri-stage
token pruning framework for MVLA models to achieve optimal 2D/3D
token selection and efficient
pruning. Experiments show that our framework achieves up to a 2.55x inference speedup with minimal accuracy loss, while only costing 5.8% overhead. Our Code is coming soon.
Read the full abstract ▾
All authors Zihao Zheng, Sicheng Tian, Zhihao Mao, Lingyue Zhang, Chenyue Li, Ziyun Zhang, Hong Gao, Yuchen Huang, Yutong Xu, Guojie Luo, Xiang Chen
Robotics
On the Terminology and Geometric Aspects of Redundant Parallel Manipulators
Andreas Mueller
Parallel kinematics machines (PKM) can exhibit kinematic as well as actuation redundancy. While the meaning of kinematic redundancy has been clarified already for serial manipulators, actuation redundancy, that is only possible in PKM, is differently classified in the literature. In this paper a consistent terminology for general redundant PKM is proposed. A kinematic model is introduced with the configuration space (c-space) as central part. The notion of kinematic redundancy is recalled for PKM. C-space, output, and input singularities are distinguished. The significance of the c-space geometry is emphasized, and it is pointed out geometrically that input singularities can be avoided by redundant actuation schemes. In order to distinguish different actuation schemes of PKM a non-linear control system is introduced whose dynamics evolves on the c-space. The degree of actuation (DOA) is introduced as the number of independent control vector fields, and PKM are classified as full-actuated and underactuated. Relating this DOA to the degree of freedom (DOF) allows to classify the actuation redundancy.
Read the full abstract ▾
Robotics · Cross-listed from cs.NI
"Take Me Home, Wi-Fi Drone": A Drone-based Wireless System for Wilderness Search and Rescue
Weiying Hou, Luca Jiang-Tao Yu and Chenshu Wu
Wilderness Search and Rescue (WiSAR) represents a longstanding and critical societal challenge, demanding innovative and automatic technological solutions. In this paper, we introduce Wi2SAR, a novel autonomous drone-based wireless system for long-range, through-occlusion WiSAR operations, without relying on existing infrastructure. Our basic insight is to leverage the automatic reconnection behavior of modern Wi-Fi devices to known networks. By mimicking these networks via on-drone Wi-Fi, Wi2SAR uniquely facilitates the discovery and localization of victims through their accompanying mobile devices. Translating this simple idea into a practical system poses substantial technical challenges. Wi2SAR overcomes these challenges via three distinct innovations: (1) a rapid and energy-efficient device discovery mechanism to discover and identify the target victim, (2) a novel RSS-only, long-range direction finding approach using a 3D-printed Luneburg Lens, amplifying the directional signal strength differences and significantly extending the operational range, and (3) an adaptive drone
navigation scheme that guides the drone toward the target efficiently. We implement an end-to-end prototype and evaluate Wi2SAR across various mobile devices and real-world wilderness scenarios. Experimental results demonstrate Wi2SAR's high performance, efficiency, and practicality, highlighting its potential to advance autonomous WiSAR solutions. Wi2SAR is open-sourced at https://aiot-lab.github.io/Wi2SAR to facilitate further research and real-world deployment.
Read the full abstract ▾
Robotics · Cross-listed from cs.CV
Physically Grounded 3D Generative Reconstruction under Hand Occlusion using Proprioception and Multi-Contact Touch
Gabriele Mario Caddeo, Pasquale Marra and Lorenzo Natale
We propose a
multimodal, physically grounded approach for metric-scale amodal object reconstruction and pose estimation under severe hand occlusion. Unlike prior occlusion-aware 3D generation methods that rely only on
vision, we leverage physical interaction signals: proprioception provides the posed hand geometry, and multi-contact touch constrains where the object surface must lie, reducing ambiguity in occluded regions. We represent object structure as a pose-aware, camera-aligned signed distance field (SDF) and learn a compact latent space with a Structure-VAE. In this latent space, we train a conditional flow-matching
diffusion model, pretraining on
vision-only images and finetuning on occluded manipulation scenes while conditioning on visible RGB evidence, occluder/visibility masks, the hand latent representation, and tactile information. Crucially, we incorporate physics-based objectives and differentiable decoder-guidance during finetuning and inference to reduce hand–object interpenetration and to align the reconstructed surface with contact observations. Because our method produces a metric, physically consistent structure estimate, it integrates naturally into existing two-stage reconstruction pipelines, where a downstream module refines geometry and predicts appearance. Experiments in simulation show that adding proprioception and touch substantially improves completion under occlusion and yields physically plausible reconstructions at correct real-world scale compared to
vision-only baselines; we further validate transfer by deploying the model on a real
humanoid robot with an end-effector different from those used during training.
Read the full abstract ▾
Robotics · Cross-listed from cs.CV
Fine-Grained Action Segmentation for Renorrhaphy in Robot-Assisted Partial Nephrectomy
Jiaheng Dai, Huanrong Liu, Tailai Zhou et al. (+7 more)
Fine-grained action
segmentation during renorrhaphy in
robot-assisted partial nephrectomy requires frame-level recognition of visually similar suturing gestures with variable duration and substantial class imbalance. The SIA-RAPN
benchmark defines this problem on 50 clinical
videos acquired with the da Vinci Xi system and annotated with 12 frame-level labels. The
benchmark compares four temporal models built on I3D features: MS-TCN++, AsFormer, TUT, and DiffAct. Evaluation uses balanced accuracy, edit score, segmental F1 at overlap thresholds of 10, 25, and 50, frame-wise accuracy, and frame-wise mean average precision. In addition to the primary evaluation across five released split configurations on SIA-RAPN, the
benchmark reports cross-domain results on a separate single-port RAPN
dataset. Across the strongest reported values over those five runs on the primary
dataset, DiffAct achieves the highest F1, frame-wise accuracy, edit score, and frame mAP, while MS-TCN++ attains the highest balanced accuracy.
Read the full abstract ▾
All authors Jiaheng Dai, Huanrong Liu, Tailai Zhou, Tongyu Jia, Qin Liu, Yutong Ban, Zeju Li, Yu Gao, Xin Ma, Qingbiao Li
Robotics
TriDeliver: Cooperative Air-Ground Instant Delivery with UAVs, Couriers, and Crowdsourced Ground Vehicles
Junhui Gao, Yan Pan, Qianru Wang et al. (+4 more)
Instant delivery, shipping items before critical deadlines, is essential in daily life. While multiple delivery
agents, such as couriers, Unmanned Aerial Vehicles (
UAVs), and crowdsourced
agents, have been widely employed, each of them faces inherent limitations (e.g., low efficiency/labor shortages, flight control, and dynamic capabilities, respectively), preventing them from meeting the surging demands alone. This paper proposes TriDeliver, the first hierarchical cooperative framework, integrating human couriers,
UAVs, and crowdsourced ground vehicles (GVs) for efficient instant delivery. To obtain the initial scheduling knowledge for GVs and
UAVs as well as improve the cooperative delivery performance, we design a Transfer Learning (TL)-based
algorithm to extract delivery knowledge from couriers' behavioral history and transfer their knowledge to
UAVs and GVs with fine-tunings, which is then used to dispatch parcels for efficient delivery. Evaluated on one-month real-world trajectory and delivery
datasets, it has been demonstrated that 1) by integrating couriers,
UAVs, and crowdsourced GVs, TriDeliver reduces the delivery cost by $65.8\%$ versus state-of-the-art cooperative delivery by
UAVs and couriers; 2) TriDeliver achieves further improvements in terms of delivery time ($-17.7\%$), delivery cost ($-9.8\%$), and impacts on original tasks of crowdsourced GVs ($-43.6\%$), even with the representation of the transferred knowledge by simple neural networks, respectively.
Read the full abstract ▾
All authors Junhui Gao, Yan Pan, Qianru Wang, Wenzhe Hou, Yiqin Deng, Liangliang Jiang, Yuguang Fang
Robotics
Towards Lifelong Aerial Autonomy: Geometric Memory Management for Continual Visual Place Recognition in Dynamic Environments
Xingyu Shao, Zhiqiang Yan, Liangzheng Sun et al. (+5 more)
Robust geo-localization under changing environmental and operational conditions is critical for long-term aerial autonomy. Aerial visual place recognition (VPR) commonly uses pre-acquired remote-sensing imagery of the intended operating area, so the geographic label space can remain fixed while successive airborne missions introduce substantial visual distribution shifts. Continual adaptation to these shifts can cause catastrophic forgetting. We therefore formulate aerial VPR as a mission-based domain-incremental learning (DIL) problem and develop a heterogeneous
memory framework. Before sequential adaptation, the satellite reference
dataset is used once to train the initial model and construct a static satellite exemplar
memory; a bounded replay buffer then retains selected airborne observations across missions. For replay management, we compare loss- and diversity-based selection criteria and introduce DBS-Hybrid, which combines prototype-based diversity trimming with representative-first feature-space coverage. Experiments on 21 visible and infrared
UAV missions evaluate generalization to held-out missions, immediate adaptation, and knowledge retention. Under the primary Forward mission order, DBS-Hybrid achieves the highest mean final average accuracy, generalization, and knowledge retention among the evaluated methods, improving over the Random baseline by $5.06$, $5.32$, and $6.33$ percentage points, respectively, and improving backward transfer from -6.41% to 1.07%. Across five additional random mission orders, DBS-Hybrid ranks second in mean final average accuracy, backward transfer, generalization, and knowledge retention. Overall, heterogeneous
memory and diversity-aware replay provide an effective basis for continual aerial VPR in mapped operating areas.
Read the full abstract ▾
All authors Xingyu Shao, Zhiqiang Yan, Liangzheng Sun, Mengfan He, Chao Chen, Jinhui Zhang, Chunyu Li, Ziyang Meng
Robotics
V-CAGE: Vision-Closed-Loop Agentic Generation Engine for Robotic Manipulation
Yaru Liu, Ao-bo Wang and Nanyang Ye
Scaling
Vision-
Language-Action (VLA) models requires massive
datasets that are both semantically coherent and physically feasible. However, existing scene generation methods often lack context-awareness, making it difficult to synthesize high-fidelity environments embedded with rich semantic information, frequently resulting in unreachable target positions that cause tasks to fail prematurely. We present V-CAGE (
Vision-Closed-loop Agentic Generation Engine), an agentic framework for autonomous robotic data synthesis. Unlike traditional scripted pipelines, V-CAGE operates as an
embodied agentic system, leveraging foundation models to bridge high-level semantic
reasoning with low-level physical interaction. Specifically, we introduce Inpainting-Guided Scene Construction to systematically arrange context-aware layouts, ensuring that the generated scenes are both semantically structured and kinematically reachable. To ensure trajectory correctness, we integrate functional metadata with a
Vision-
Language Model based closed-loop verification mechanism, acting as a visual critic to rigorously filter out silent failures and sever the error propagation chain. Finally, to overcome the storage bottleneck of massive
video datasets, we implement a perceptually-driven compression
algorithm that achieves over 90% filesize reduction without compromising downstream VLA training efficacy. By centralizing semantic layout planning and visual self-verification, V-CAGE automates the end-to-end pipeline, enabling the highly scalable synthesis of diverse, high-quality robotic manipulation
datasets.
Read the full abstract ▾
Robotics
Zhi Jing, Jinbin Qiao, Ouyang Lu et al. (+5 more)
Spatial
reasoning is a fundamental capability for
embodied intelligence, especially for fine-grained manipulation tasks such as robotic assembly. Recent methods based on
vision-
language models (VLMs) largely rely on coarse 2D perception and struggle to perform accurate
reasoning over complex 3D geometry. To address this limitation, we propose AssemLM, a spatial
multimodal large
language model for robotic assembly that integrates assembly manuals, point clouds, and textual instructions to predict task-critical 6D assembly poses with explicit
geometric understanding. To bridge raw 3D perception and high-level linguistic
reasoning, AssemLM employs a specialized point cloud encoder to extract fine-grained
geometric and rotational features for accurate 3D spatial
reasoning in assembly tasks. In addition, we introduce AssemBench, a large-scale
benchmark for assembly-oriented spatial
reasoning with over 900K
multimodal samples and precise 6D pose annotations, extending evaluation from 2D grounding to full 3D
geometric inference. Extensive experiments and real-
robot evaluations demonstrate that AssemLM achieves state-of-the-art 6D pose
reasoning performance and effectively supports fine-grained, multi-step assembly tasks in real-world settings. Code, models, and the AssemBench
dataset will be made publicly available.
Read the full abstract ▾
All authors Zhi Jing, Jinbin Qiao, Ouyang Lu, Jicong Ao, Shuang Qiu, Huazhe Xu, Yu-Gang Jiang, Chenjia Bai
Robotics · Cross-listed from cs.LG
WOMBET: World Model-Based Experience Transfer for Robust and Sample-efficient Reinforcement Learning
Mintae Kim and Koushil Sreenath
Reinforcement learning (RL) in robotics is often limited by the cost and risk of data collection, motivating experience transfer from a source task to a target task. Offline-to-online RL leverages prior data but typically assumes a given fixed
dataset and does not address how to generate reliable data for transfer. We propose World Model-Based Experience Transfer (WOMBET), a framework that jointly generates and utilizes prior data. WOMBET learns a world model in the source task and generates offline data via uncertainty-penalized planning, followed by filtering trajectories with high return and low epistemic uncertainty. It then performs online fine-tuning in the target task using adaptive sampling between offline and online data, enabling a stable transition from prior-driven initialization to task-specific adaptation. We show that the uncertainty-penalized objective provides a lower bound on the true return and derive a finite-sample error decomposition capturing distribution mismatch and approximation error. Empirically, WOMBET improves sample efficiency and final performance over strong baselines on continuous control
benchmarks, demonstrating the benefit of jointly optimizing data generation and transfer.
Read the full abstract ▾
Robotics · Cross-listed from cs.CV
TouchAnything: Diffusion-Guided 3D Reconstruction from Sparse Robot Touches
Langzhe Gu, Hung-Jui Huang, Mohamad Qadri et al. (+2 more)
Accurate object geometry estimation is essential for many downstream tasks, including robotic manipulation and physical interaction. Although
vision is the dominant modality for shape perception, it becomes unreliable under occlusions or challenging lighting conditions. In such scenarios, tactile sensing provides direct
geometric information through physical contact. However, reconstructing global 3D geometry from sparse local touches alone is fundamentally underconstrained. We present TouchAnything, a framework that leverages a pretrained large-scale 2D
vision diffusion model as a semantic and
geometric prior for 3D reconstruction from sparse tactile measurements. Unlike prior work that trains category-specific reconstruction networks or learns
diffusion models directly from tactile data, we transfer the
geometric knowledge encoded in pretrained visual
diffusion models to the tactile domain. Given sparse contact constraints and a coarse class-level description of the object, we formulate reconstruction as an optimization problem that enforces tactile consistency while guiding solutions toward shapes consistent with the
diffusion prior. Our method reconstructs accurate geometries from only a few touches, outperforms existing baselines, and enables open-world 3D reconstruction of previously unseen object instances. Our project page is https://grange007.github.io/touchanything .
Read the full abstract ▾
All authors Langzhe Gu, Hung-Jui Huang, Mohamad Qadri, Michael Kaess, Wenzhen Yuan