Robotics · Cross-listed from cs.CV
UniPart: Towards Zero-shot Language-Grounded 3D Part Segmentation for Embodied Interaction
Xinqiang Yu, Zekun qi, Jiawei He et al. (+6 more)
Fine-grained robotic manipulation depends on understanding parts, not only whole objects. Existing 3D foundation models tend to be either generalized but object-aware, or part-aware but limited to closed-set taxonomies, which weakens zero-shot transfer. We study text-conditioned 3D part segmentation, where a free-form phrase selects a functional part on point cloud. We introduce UniPart, a feed-forward cross-modal 3D
Transformer that conditions CLIP text
embedding. To scale supervision, we build LangPart-1M with 160K+ Objaverse assets and 8M text to part pairs using multi-view consistent part generation. We further manually label a high-quality subset, LangPart-4K, for fine-tuning and evaluation. UniPart achieves strong zero-shot results on open-vocabulary part
benchmarks and transfers to
language-conditioned part grasping in real world.
Read the full abstract ▾
All authors Xinqiang Yu, Zekun qi, Jiawei He, Wenyao Zhang, Xuchuan Chen, Guaocai Yao, Li Yi, Zhaoxiang Zhang, He Wang
Robotics
Before the Tipping Point: Force-Guided Active Perception for Shape-Agnostic Estimation of 3D Centers of Mass
Steven M. Hyland, Jing Xiao and Cagdas D. Onal
Estimating the 3D center of mass of unknown objects is challenging when grasping is infeasible, geometry is irregular, or mass distribution is uneven. We present a force-based method that estimates CoM height and mass from a single sub-critical tipping experiment by a
robot manipulator. The
robot applies a quasistatic elevated push and retract motion, using force-angle measurements recorded during tipping to identify parameters from the object trajectory. Our proposed push-retract cycle mitigates frictional bias, enabling generalized fitting. We experimentally validate our method using a
robot manipulator with a six-axis force torque sensor on varying types of objects without prior shape information and without specific models. We also propose a method to prevent toppling, keeping the object in a sub-critical tipping regime by leveraging a safety margin. In experimental studies, our method recovers mass, CoM height, and toppling angle with relative errors below 5.0 percent across all unknown objects. This work demonstrates reliable 3D inertial parameter estimation under proper safety thresholds in tipping. Our proposed method informs and enables reliable non-prehensile manipulation and robotic grasping of challenging objects that were previously infeasible.
Read the full abstract ▾
Robotics
From Transportation to Manipulation: Enabling Grasping in Magnetic Robotics
Lara Bergmann, Noah Greis, Cedric Grothues et al. (+2 more)
Magnetic levitation (MagLev) systems have great potential for application in high-mix, low-volume manufacturing due to their scalability and flexibility, enabling highly reconfigurable in-machine material flow. However, their manipulation capabilities remain largely unexploited, as current applications almost exclusively focus on transportation. To enable grasping and manipulation directly on MagLev systems without requiring additional costly handling equipment, such as industrial
robot arms, we present the Gripper MagBot, a low-cost parallel 6-DoF manipulator with an integrated 1-DoF gripper that mechanically couples three MagLev movers. The Gripper MagBot supports two operating configurations: a default mode and a single-track mode, selectable depending on the required stability and workspace footprint. To reconfigure a machine, the MagBot can be autonomously dropped off and picked up using a docking station. We showcase pick-and-place examples in simulation, as well as with the real Gripper MagBot using our inverse kinematics controller. CAD files, assembly instructions, a component list, and
videos are available at https://sites.google.com/view/gripper-magbot.
Read the full abstract ▾
All authors Lara Bergmann, Noah Greis, Cedric Grothues, Lisa-Marie Weigelt, Klaus Neumann
Robotics
A Multi-Vehicle Dataset with Camera, LiDAR, and Radar Sensors and Scanned 3D Models for Custom Auto-Annotation using RTK-GNSS
Philipp Berthold, Bianca Forkel and Mirko Maehlisch
Datasets are a crucial element in the development of perception
algorithms. They relate sensor measurement data to annotated reference information and allow for the deduction of sensor and object characteristics. In autonomous driving, the reference data commonly consist of semantic image segmentation, point-wise associations, or bounding box annotations. The
dataset proposed in this work, however, aims to dig deeper into the evaluation of measurement principles and provides scanned 3D models of all vehicles together with a pose and continuous kinematics reference obtained by RTK-GNSS. Combined, the state of the complete dynamic surrounding of the sensor vehicle is known for any point in time. Subsequent reference formats can be easily computed in user-defined granularity. This
dataset involves single-object and multi-object recordings with seven target vehicles. In particular, measurement effects such as occlusion, as well as reflections, can be evaluated, as the normals of the shape of the target vehicles are known. We describe the
dataset, discuss the technical background of its development, and briefly present exemplary evaluations.
Read the full abstract ▾
Robotics
Parameter Sensitivity Analysis for Aerial LiDAR-Inertial Odometries in low-altitude flights
Robert Milijas, Jose Ramiro Martinez-de Dios and Stjepan Bogdan
LiDAR-based SLAM (Simultaneous Localization and Mapping) and LIO (
LiDAR-inertial odometry)
algorithms are often used for precise navigation of unmanned aerial vehicles, especially during interactions with the aerial
robot's environment. However, the performance of these
algorithms is greatly dependent on the scenario,
LiDAR, and
robot motion characteristics, often requiring an intensive tuning process to achieve the desired performance. To aid these tuning efforts, this paper analyzes the influence on performance of the parameters of an EKF-based LIO
algorithm (FAST-LIO2) and the LIO module of a graph-based SLAM
algorithm (Cartographer) on aerial
LiDAR SLAM
datasets recorded using different
LiDARs in low-to-moderate-altitude flights in diverse environments. The analysis is conducted on the absolute trajectory error (ATE) resulting from processing the
datasets with the LIO
algorithms configured with each combination of parameters obtained in an exhaustive grid search. The relationship between individual parameters and the ATE results is assessed using Pearson's correlation, while the influence of each parameter is assessed using random forest permutation importance analyses with random forest models trained to predict the resulting ATE values based on the choice of parameters. The performed analysis obtains for Cartographer and FAST-LIO2: i) the identification of parameters with stronger influence in performance, ii) a simplified tuning procedure, and iii) tuning recommendations. Using the proposed tuning recommendations, both
algorithms obtain on the analyzed
datasets ATE values within 5 cm to the optimal performance found in the grid search procedure in 94% of the analyzed cases.
Read the full abstract ▾
Robotics
VertexCBF: Improving Neural Control Barrier Functions via Vertex-Restricted Control Search
Bojan Derajić, Sebastian Bernhard and Wolfgang Hönig
As the number of autonomous
robots continues to grow, safety becomes increasingly important. Control barrier functions (CBFs) provide a theoretically grounded framework for ensuring safety, but existing design methods often face limitations in effectiveness, scalability, or interpretability, and may result in overly conservative safe sets. In this paper, we propose VertexCBF, a framework for learning neural CBFs in a scalable, systematic, and explainable way. We approximate the stationary Hamilton–Jacobi value function using a neural network trained via a combination of physics-informed and sparsely supervised learning. By exploiting control-affine dynamics and a convex polytope control set, under which the
Hamiltonian is maximized at the control vertices, we efficiently generate supervision points via GPU-parallel vertex-restricted tree search, while a residual architecture guarantees that the learned CBF is never larger than the specified constraint function. We evaluate the method on 15 systems and compare it against relevant baselines, showing that it reliably recovers large safe sets where the baselines are conservative or fail completely. In addition, we perform a hardware experiment in which a mobile
robot safely avoids pedestrians using a neural CBF trained with our method.
Read the full abstract ▾
Robotics
High-Fidelity Multi-Body Simulator for Autonomous Racing
Nicola Musiu, Francesco Iacovacci, Fausto Lupo et al. (+11 more)
We present a custom high-fidelity vehicle dynamics simulation environment for testing and validation of Autonomous Racing software. The digital twin of the autonomous vehicle is developed in Dymola, using racecar dynamics modeling libraries to build a complete multi-body model. A 3D road surface, including elevation profiles and curbs, is implemented using the Curved Regular Grid (CRG) standard. The model is exported from Dymola as a Functional Mock-up Unit (FMU) and integrated into a custom software-in-the-loop simulator, where communication interfaces with the autonomous racing stack were developed in C++. A calibration procedure based on experimental data is also presented, along with a validation study to further support the quality of the proposed framework. The simulator runs in real time on a portable computer and provides reliable ground truth for
algorithms validation prior to real-world deployment.
Read the full abstract ▾
All authors Nicola Musiu, Francesco Iacovacci, Fausto Lupo, Matteo Pini, Giovanni Scapicchi, Francesco Moretti, Eugenio Mascaro, Pietro Musso, Ayoub Raji, Marko Bertogna, Vincenzo Maria Arricale, Angelo Lo Sapio, Alessandro Piccarelli, Garron Fish
Robotics
Control Architecture for Safe Grasping of Fragile Objects Using a Coarse Position-Controlled Gripper
Marko Pavlic, Moritz Geier, Timo Markert et al. (+1 more)
Robots are increasingly used in unstructured environments. The need for them to safely grasp unknown objects without damaging them becomes crucial. Humans achieve this by sensing and quickly responding by adjusting their grasping force. Similarly, effective grasp acquisition in
robots requires compliant interaction strategies that can adapt to uncertain object properties and adjust to any instabilities during manipulation. We present a geometry-aware force/torque-based contact estimation method for a coarse position-controlled gripper, combined with an adaptive admittance controller for safe grasp acquisition. The desired contact forces are estimated online to keep stable contact with objects of unknown properties. This enables compliant and stable grasps while avoiding excessive forces. Experiments with objects of different sizes, shapes, stiffnesses, and weights show that the proposed
algorithm not only prevents slippage but also applies minimal force to safely grasp an object without causing excessive deformation.
Read the full abstract ▾
All authors Marko Pavlic, Moritz Geier, Timo Markert, Darius Burschka
Robotics
Improving Imitation Learning Efficiency for Manipulation through Geometric Prior Pretraining
Shogo Iwakata, Tomohiro Motoda, Ryosuke Yamada et al. (+8 more)
Applying an imitation learning policy to a new manipulation task usually requires collecting new demonstrations and retraining the model, which makes sample efficiency a practical concern. Pretraining on large-scale
robot datasets is effective in this respect, but such
datasets are costly to collect and train on, while data augmentation techniques typically require a new round of data generation and retraining for each task. A complementary question is what useful prior can be provided to a policy at negligible cost before any task-specific data are collected. In this study, we construct a
geometric visual pretraining
dataset in which each scene contains only a plane, an object, and a hand, and trajectories are generated automatically. The scenes contain neither textures nor backgrounds; pretraining primarily exposes the policy to the
geometric relationship between the hand and the object. Furthermore, representing the hand as a cube avoids tailoring the
dataset to a specific
robot morphology. We evaluate this
geometric prior using ACT on three simulated
robots across five manipulation tasks each, as well as on three real-world
robot tasks. Across many of these
robot–task combinations, fine-tuning from the
geometric prior achieves higher success rates in the early stages of training than training from scratch while using only a small number of task demonstrations. These results suggest that even highly simplified
geometric scenes can provide a useful initialization that transfers across
robots and to real-world tasks when task data are limited.
Read the full abstract ▾
All authors Shogo Iwakata, Tomohiro Motoda, Ryosuke Yamada, Koshi Makihara, Ryoichi Nakajo, Keitaro Tanaka, Masaki Murooka, Roman Mykhailyshyn, Hirokatsu Kataoka, Shigeo Morishima, Yukiyasu Domae
Robotics
Size Doesn't Matter: Material-State Reinforcement Learning for Excavator Transferable Soil Manipulation
Lennart Werner, Pol Eyschen, Sean Costello et al. (+3 more)
Earthmoving tasks such as excavation, backfilling, or embankment construction require deliberate repositioning of deformable soil. For these tasks, human operators use all shovel faces, while autonomous systems so far are limited to excavation and dumping. Current methods often rely on heuristic models but do not incorporate soil mechanics. We address this shortcoming by using Reinforcement Learning in a GPU-parallelized Material Point Method particle simulation. Our controllers are conditioned on material state such as shape and compactness, enabling skills that use multiple contact faces of the tool and displace material both inside and outside of the shovel. To use the same learned weights across machines, our policies operate in a normalized end-effector space and are deployed through a calibrated machine interface. We evaluate this calibrated transfer on an 11.5t hydraulic excavator and a 500g tabletop
robot. We validate performance through autonomous construction of a 42m long, 2.1m high embankment in 45min, executing 201 individual policy strokes without failure, retry, or operator intervention. In a direct comparison, the autonomous controller matches an expert operator's progression speed and produces a higher, more consistent embankment. Additional qualitative backfilling and compaction experiments demonstrate the material-state awareness and calibrated transfer across machines.
Read the full abstract ▾
All authors Lennart Werner, Pol Eyschen, Sean Costello, Pierluigi Micarelli, Andrei Cramariuc, Marco Hutter
Robotics
Driving Context-guided Model Predictive Planning and Control for Autonomous Car Racing at the Limit and Beyond
Ayoub Raji, Federico Sacco, Nicola Musiu et al. (+1 more)
This paper presents a Model Predictive Control-based motion planning and control pipeline for autonomous car racing capable of adapting to different driving contexts, such as overtaking, nominal driving, and countersteering. A Cost Blending state machine manages the identification of different driving contexts and the selection of their predefined weights to be applied to the Model Predictive Planning (MPP) and Control (MPC) modules. The two optimization-based solutions share the same problem formulation and model, differing only in horizon length, rate, tuning, and in their open-loop versus closed-loop approach to maximize the effectiveness of their interaction. The work is validated on the fully autonomous open-wheel racecar Superformula EAV-25, with a lap time achieved that is within 2% of the best human driver reference. The results demonstrate the capability of the solution in driving at the limit of handling, smoothly executing overtaking maneuvers, and quickly reacting to high oversteering conditions to recover the vehicle stability.
Read the full abstract ▾
All authors Ayoub Raji, Federico Sacco, Nicola Musiu, Marko Bertogna
Robotics
Breaking the Vision-Action Shortcut: Latent Interface Training for Generalizable Robotics Foundation Models
Jianman Lin, Shailesh Shailesh, Zhongyi Luo et al. (+1 more)
Robot foundation models achieve strong in-distribution performance but often degrade under visual distribution shifts. When learning to generate actions from pretrained visual representations, models may exploit task-irrelevant visual cues that correlate with demonstrated actions within the training distribution. Such
vision-action shortcuts can undermine generalization when these correlations change under distribution shifts. Mitigating these shortcuts requires constraining how visual information is used for action generation while preserving task-relevant spatial information. We propose Latent Interface Training (LIT), a framework-agnostic two-stage strategy that first establishes a spatial-goal-conditioned action prior without images, then constrains visual conditioning through a pose-supervised latent interface. Stage 1 trains the action expert to generate action chunks conditioned on
language,
robot state, and each demonstrated chunk's terminal SE(3) end-effector pose, learning goal-directed action generation independently of visual cues. Stage 2 introduces a latent interface that aggregates visual and semantic representations and serves as the pretrained action expert's only visual conditioning pathway. The interface is supervised to reconstruct the terminal pose previously used to condition Stage 1, encouraging it to retain the goal-relevant spatial information needed for action generation. Across four
vision-
language-action and world-action architectures (Pi0.5, MolmoAct2, FAST-WAM, and ImageWAM), LIT improves overall LIBERO-Plus success by 3.87-10.70 percentage points while preserving or improving average LIBERO success. Real-world evaluations show 13.30-16.70 percentage-point gains in success aggregated across three tasks under unseen camera configurations, lighting variations, and distractors.
Read the full abstract ▾
All authors Jianman Lin, Shailesh Shailesh, Zhongyi Luo, Jiafei Duan
Robotics
Online Material Estimation for Conditioned Diffusion Policy in Shaping Deformable Linear Objects
Ryunosuke Yamada, Tomohiro Motoda, Yukiyasu Domae et al. (+1 more)
Shape control of deformable linear objects (DLOs) is challenging for imitation learning because deformation behavior varies with material properties such as stiffness and elasticity, so a single policy must generate different action sequences for different objects even when the goal shape is identical. We propose a
diffusion policy conditioned on material labels that are estimated online during manipulation. A recurrent estimation network predicts the material label of the grasped object from the time series of multi-view images and
robot joint states, and the predicted label conditions the
diffusion policy at every inference step. We collected 480 real-
robot demonstrations covering four DLO materials and three groove-placement tasks, and compared per-material specialist policies, a task-conditioned policy without material labels, a policy conditioned on ground-truth material labels, and the proposed policy. Conditioning on ground-truth material labels improved the average success rate from 45.8% to 60.0% over the task-only policy, and the proposed policy reached 60.8% without any prior material information, matching the policy given ground-truth labels. A post-hoc analysis shows that the
estimator extracts material-related information from the manipulation observations and that the
diffusion policy responds to the resulting conditioning signal, while the one pronounced failure case is associated with persistent confusion between two similar materials.
Read the full abstract ▾
All authors Ryunosuke Yamada, Tomohiro Motoda, Yukiyasu Domae, Tokuo Tsuji
Robotics
ProClosure: Hierarchical Room-Object Assignment using Progressive Boundary Closure from Monocular Video
Vinoth Kumar Muthuraj, Soumyadeep Banik, Kushal Sharma et al. (+1 more)
A 3D scene graph groups objects into rooms. When a
robot is asked to fetch an object from the kitchen, that grouping is what tells it where to look. An object recorded in the wrong room is not retrievable by a query naming the correct room. We introduce Progressive Boundary Closure, which recovers room layer from a monocular RGB
video. A SLAM front end and an open-vocabulary segmenter supply a structural point cloud, camera trajectory and object tracks. The cloud is rasterised into a top-down map, rooms are recovered from it, and each object takes the room holding most of its extent. The difficulty lies in the map itself. Walls are recorded only where the camera looked, so a gap in the boundary may be a doorway or a stretch of wall that was never observed; nothing distinguishes the two. Prior methods treat both as passages, merging rooms that should remain separate. We observe that both require the same treatment: a room should not extend across either, so both are closed and need not be distinguished. Such an opening closes under a small amount of boundary growth, and few sightlines cross it, so points in different rooms rarely see one another. We use the first to recover rooms and the second to assign objects to them. Rooms are obtained by Progressively thickening the boundary inward and freezing each free-space region once it becomes enclosed, so every opening seals at its own scale rather than at a radius fixed in advance. Camera poses are used as seeds, which removes the sampling heuristic and makes the segmentation deterministic. Over 10 floors of 6 HM3D-Semantics scenes, scored against HOV-SG on identical top-down maps, we recover 74 rooms for 72 annotated regions (HOV-SG: 44), raising room F_1 from 0.741 to 0.890 at IoU 0.25 at some cost in precision, and object-to-room ARI from 0.488 to 0.696 (p=0.002, ahead on every floor).
Read the full abstract ▾
All authors Vinoth Kumar Muthuraj, Soumyadeep Banik, Kushal Sharma, Hardik Jain
Robotics
Quantifying Spectral Differences in Vehicle Kinematics Between Production Autonomous and Human-Driven Vehicles Across Driving Scenarios
Peiyi Fang, Xiangyu Li, Yonglin Weng et al. (+1 more)
Differences in vehicle kinematic characteristics between production autonomous vehicles (PAVs) and human-driven vehicles (HVs) have been limitedly investigated by empirical studies. Most recent studies rely on simulation-based models, while some further investigate low-level adaptive cruise control (ACC) systems in controlled experiments. These methods commonly adapt some time-domain metrics to characterize PAV-HV differences across limited driving conditions. However, current PAVs equipped with high-level autonomous driving systems generate driving behaviors in a black box using data-driven models. These fundamentally different mechanisms for generating behaviors may produce distinct kinematic characteristics in traffic. More importantly, these time-domain metrics cannot reflect frequency-related traffic dynamics across different driving scenarios. Thus, this study adapted a real-world PAV
dataset with four PAV platforms and developed a frequency-domain framework to quantify kinematic differences between PAVs and HVs across diverse driving scenarios, including varying driving states, lighting, weather, and vehicle densities. The framework transforms kinematic signals into the frequency domain and extracts spectral features, and then compares these features between PAVs and HVs based on kernel density estimation and Wasserstein distance. The results reveal clear scenario-dependent PAV-HV spectral differences. Specifically, speed-related differences were consistently smaller during car-following than cruising, while rainy conditions consistently enlarged acceleration-related differences compared with clear conditions. These findings highlight the necessity of multi-scenario evaluations and demonstrate the value of frequency-domain analysis for characterizing PAV-HV kinematic differences under real-world conditions.
Read the full abstract ▾
All authors Peiyi Fang, Xiangyu Li, Yonglin Weng, Ke Ma
Robotics
A Hierarchical Coverage Path Planning Algorithm for Unknown Environments
Zongyuan Shen, Haodong Liu, Gao Wang et al. (+4 more)
This paper presents an online coverage path planning
algorithm for unknown environments. During navigation, the initially unknown search area is progressively decomposed into disconnected subareas as new obstacle information is acquired and coverage proceeds. These subareas are organized in an incrementally constructed decomposition tree that preserves their hierarchical parent-child relationships. Based on this tree, a global coverage tour is maintained and updated online by prioritizing newly generated child subareas according to their exploration states and distances from the
robot. A local planner then generates coverage motions within each selected subarea, allowing the
robot to adapt its trajectory as the environment is gradually revealed. Its performance is evaluated via high-fidelity simulations in complex scenarios. The results show improved coverage efficiency in terms of path length and overlap ratio in comparison to three baseline
algorithms.
Read the full abstract ▾
All authors Zongyuan Shen, Haodong Liu, Gao Wang, Hongbin Ma, Yaming Ou, Shancheng Zhao, Dehua Zhou
Robotics · Cross-listed from cs.CV
DRS-VPT: Directly Relocalizing in a Scan with Vision Point Transformers
Lanke Frank Tarimo Fu and Maurice Fallon
We present DRS-VPT, a feed-forward
transformer architecture for foundational image-to-scan registration. Given query images and a reference 3D point cloud, the model predicts the scan pose and point map alongside the poses and point maps of each camera, all expressed in the first camera's frame. It additionally predicts a coarse-to- fine pyramid of per-point and per-pixel features for direct reprojective
alignment of the scan to the first image. This formulation unifies downstream tasks such as camera-
LiDAR calibration in autonomous driving and indoor camera-to-map relocalization. A single DRS-VPT model achieves state-of-the-art performance for image-to-
LiDAR registration in autonomous driving, competitive indoor relocalization without training map-specific weights, and strong zero-shot transfer to unseen environments. We also show qualitatively that the model learns complex scan-to-image projection properties such as occlusion of back-facing points.
Read the full abstract ▾
Robotics
STAR: Sparse Tactile Representation Learning in Vision-Tactile-Language-Action Models for Dexterous Manipulation
Xiangcheng Liu, Tianhao Wu, Le Zheng et al. (+6 more)
Dexterous manipulation requires coordinated multi-finger control and effective tactile feedback, yet learning these capabilities remains challenging due to the lack of large-scale real-world data and the difficulty of extracting effective representations from sparse tactile signals. We build a
robot platform and teleoperation system to collect a 200-hour bimanual dexterous manipulation
dataset with synchronized visual, tactile, and
language annotations, comprising 10,576 trajectories across 65 tasks, 69.5% of which involve dexterous multi-finger manipulation. We further propose
STAR, an integrated training recipe for
vision-tactile-
language-action (VTLA) models that addresses the spatial, temporal, and informational sparsity of tactile signals through visual-tactile joint pre-training, sparse-global tactile
token representation, and sparse future tactile prediction. Trained on this
dataset,
STAR achieves a 61% average success rate across four real-world tasks with 100 post-training trajectories per task, demonstrating dexterous performance under task-specific post-training.
Read the full abstract ▾
All authors Xiangcheng Liu, Tianhao Wu, Le Zheng, Yidong Wang, Bowen Jiang, Mingjie Pan, Xinlin Ren, Yi Liu, Jianlan Luo
Robotics
Autonomous Precision Milling of Biological Structures via Generic Anatomical Priors and Active Boundary Perception
Enduo Zhao, Xiaofeng Lin, Yifan Wang et al. (+4 more)
Autonomous precision milling of biological structures is challenged by incomplete knowledge of target geometry, local material thickness, and critical internal boundaries. Subject-specific preoperative models can address
geometric and thickness variations, but static models cannot determine boundary status encountered during execution, while repeated target-specific imaging limits scalability. This article presents an uncertainty-aware autonomous milling framework that assigns complementary roles to generic anatomical priors and active boundary perception. A generic anatomical prior provides conservative global guidance and is transformed through semantic-guided registration and hybrid
vision-force calibration into
robot-executable guidance for individual targets. As milling approaches uncertain boundaries, the
robot actively probes the remaining structure and uses relative stiffness changes to estimate boundary status and structural detachability. A state-adaptive controller governs transitions between active perception and spatially selective incremental refinement, repeating this cycle until the termination criterion is satisfied. Hierarchical experiments on biological surrogates and in vivo mouse cranial window creation demonstrate accurate anatomical prior transfer, reliable boundary adaptation, and autonomous precision milling of biological structures.
Read the full abstract ▾
All authors Enduo Zhao, Xiaofeng Lin, Yifan Wang, Yuhan Song, Weihan Li, Saul Alexis Heredia Perez, Kanako Harada
Robotics
Communication-Constrained Multi-Robot Exploration With Adaptive Communication Windows
Ben Rossano, Jaein Lim and Jonathan P. How
Exploring unknown environments with multi-
robot teams can improve efficiency by allowing
robots to explore in parallel. However, realizing these gains requires effective information sharing. When communication is intermittent,
robots must balance the benefits of sharing information against the cost of diverting from exploration to establish communication. This paper introduces MACE, a decentralized exploration framework that actively evaluates whether establishing communication is worthwhile. At scheduled communication windows,
robots estimate the cost of reaching previously identified communication locations. By formulating this decision as a variant of the Vehicle Orienteering Problem,
robots evaluate routes based on the travel required to establish communication and the exploration that can be completed along the way. This approach enables
robots to communicate more frequently than under purely opportunistic strategies while reducing the unnecessary travel associated with fixed rendezvous strategies. Across a set of simulated environments with varying size and geometry, we demonstrate that MACE reduces the total exploration time by up to 23% compared to existing communication-constrained exploration strategies.
Read the full abstract ▾
Robotics
ArtManip: Category-Level Articulated In-Hand Manipulation
Yang Yang, Tengyu Liu, Puhao Li et al. (+6 more)
Category-level in-hand manipulation of articulated objects is a formidable yet underexplored challenge for dexterous robotic hands. This difficulty stems from two core bottlenecks: first, controlling an object's internal degrees of freedom is tightly coupled with maintaining grasp stability on a free-floating base; second, acquiring diverse object models and functional grasps at scale is highly labor-intensive, yet vital for generalization given the system's sensitivity to initial configurations. In this work, we present ArtManip, the first category-level articulated in-hand manipulation method that generalizes across object instances and diverse initial grasps. For initial configuration construction, we develop an automated pipeline that procedurally generates diverse articulated objects and synthesizes task-oriented functional grasps. For policy learning, we propose a robust two-stage training strategy that incorporates articulation physics randomization, reward curriculum, and latent representation
distillation to handle complex contact and joint dynamics during deployment. Extensive experiments across four object categories demonstrate that our policy generalizes to unseen instances and varied configurations in simulation, and achieves zero-shot transfer to 12 real-world objects featuring diverse shapes and joint mechanics.
Read the full abstract ▾
All authors Yang Yang, Tengyu Liu, Puhao Li, Zeyuan Chen, Yuyang Li, Xingwan Wang, Yingying Wu, Zhaopeng Cui, Siyuan Huang
Robotics
PATH: Continuous Target Sensing among Autonomous Cooperative Drones
Heegyeong Kim, Alice James, Avishkar Seth et al. (+4 more)
Continuous target sensing by uncrewed aerial vehicles (
UAVs) is constrained by limited flight endurance, motivating the transfer of tracking responsibility between cooperating
UAVs. Such a handoff requires the receiver to identify the same physical target currently tracked by the sender despite differences in viewpoint, scale, and target appearance. Existing approaches based on global target localization or appearance-based cross-view association are limited by positioning uncertainty or ambiguous visual features. This paper presents Perspective
Alignment & Tracking Handoff (PATH), a platform-agnostic, geometry-assisted sensing and verification framework for target handoff between two moving
UAVs. The sender reconstructs the tracked target as a metric 3D point using RGB-D sensing, while the receiver estimates its relative pose from a fiducial observation and projects the transmitted target point into its own image as a spatial prior for target acquisition. The receiver-generated candidate is then returned to the sender and verified through a cross-view Mutual Agreement Handshake before tracking responsibility is transferred. Real-world
UAV experiments show mean relative-position and target-position errors of 0.047~m and 0.030~m, respectively. Under visually ambiguous conditions, PATH achieves 96.0% frame-level receiver-side target acquisition accuracy, with 2.0% false-positive and 2.0% false-negative rates. A sensor-error sensitivity analysis shows that relative-pose uncertainty is the dominant contributor to receiver-view projection error. The implementation operates at
video rate with compact inter-
UAV communication below 16~kB/s at 60~Hz, demonstrating the feasibility of lightweight geometry-assisted target handoff on resource-constrained
UAV platforms.
Read the full abstract ▾
All authors Heegyeong Kim, Alice James, Avishkar Seth, Endrowednes Kuantama, Jane Williamson, Yimeng Feng, Richard Han
Robotics
IMPLY: Physically Anchored Consistency for World-Model Rollouts
Aman Mehta and Riya Baviskar
A world model asked what happens if an object is pushed at several speeds produces several futures. If the model has the object in mind, those futures agree about it: each implies the same mass and friction. The consistency checks now used to vet world-action models ask whether a model's futures agree with each other, and none of them knows any physics. We show that this is not enough, and what to do instead. IMPLY reads the physics each rollout implies by inverting a simulator and scores a set of rollouts by how well one object explains all of them, anchored to two calibration pushes the model has observed. In a controlled setting, self-consistency gives a perfect score to a model that ignores the object and always predicts a typical push; anchoring exposes it (AUROC 0.70 versus 1.00). On a real model, V-JEPA 2-AC adapted to the scene, the same thing happens. Given its own calibration pushes the model tracks the object (per-object correlation with the truth 0.91); given another object's, it does not (0.05). Self-consistency cannot tell these apart, preferring the right evidence on 52% of objects, chance level, while anchored disagreement prefers it on 73% and correlates 0.92-0.99 with the rollouts' error. Used to choose among candidate rollout sets, it comes within 0.003 of an oracle that sees the truth. A model that has internalised the wrong object is exactly as self-consistent as one that has internalised the right one; consistency has to be anchored to evidence.
Read the full abstract ▾
Robotics
FoldNet++: a Large-Scale Synthetic Dataset for Robotic T-Shirt Folding and Unfolding
Yuxing Chen, Zhiyuan Wei, Bowen Xiao et al. (+2 more)
Due to the highly deformable nature of garments, training a generalizable policy for robotic T-shirt folding and unfolding remains a significant challenge. In this work, we present a large-scale synthetic
dataset for robotic T-shirt folding and unfolding, covering 6 robotic embodiments, 1K T-shirts, 1K environmental assets, and 120K episodes with rich annotations, which can be used to train a wide range of manipulation policies. We first follow the FoldNet pipeline to generate a large-scale
dataset of physically simulatable T-shirts with diverse appearances and annotated semantic keypoints. Based on these semantic keypoints, we then generate manipulation demonstrations for different robotic embodiments through a unified rule-based framework. We use these demonstrations to train visuomotor policies, and experimental results demonstrate that models trained solely on our synthetic data can achieve over 90% end-to-end task success rates when directly deployed to unseen real-world environments and previously unseen T-shirts from arbitrary initial configurations. Project URL: https://pku-epic.github.io/FoldNetXX/.
Read the full abstract ▾
All authors Yuxing Chen, Zhiyuan Wei, Bowen Xiao, Zhizheng Zhang, He Wang
Robotics · Cross-listed from cs.AI
Decentralized Evolution of Hexapod Gaits with Independent Leg Controllers
Gary B. Parker, John Asaro and Jim O'Connor
This paper presents a novel approach to hexapod locomotion by evolving each leg's gait independently through a decentralized evolutionary
algorithm. Using the Webots simulator and the Mantis hexapod
robot, we optimize individual leg controllers without centralized coordination, allowing emergent behaviors to drive the development of efficient, coordinated locomotion. Our decentralized method is benchmarked against cooperative coevolution, demonstrating improved efficacy in generating stable and adaptive gaits while showing interesting emergent coordination. By enabling independent
evolution of leg controllers, this method reduces the complexity of gait optimization and highlights the potential of decentralized strategies for scalable and adaptive robotic systems.
Read the full abstract ▾
Robotics
Understanding Whole-Body Robot Teleoperation Strategies Under Diverse Task Objectives and Constraints
Tsung-Chi Lin, Juo-Tung Chen and Chien-Ming Huang
This work investigates the control strategies of complex whole-body
robot teleoperation that coordinate active perception, bimanual manipulation, and navigation. We developed a hybrid control framework, combining the free-form and constrained control, for the whole-body teleoperation of the TIAGo mobile manipulator. We conducted a user study to explore people's control strategies under different task constraints such as limited time and low tolerance of errors. Our results highlight the effective use of coordinated control in improving task efficiency and reducing the risk of reaching individual joint limits. We discuss our results and their implications for designing future whole-body
robot teleoperation systems.
Read the full abstract ▾
Robotics
READ: Learning Risk-Informed Fields for End-to-End Autonomous Driving
Zhiyuan Liu, Yuanxin Tian, Zehong Ke et al. (+5 more)
Autonomous driving requires more than recognizing what is present in a scene: a planner must determine how road structure, surrounding
agents, and their motion states should influence a future maneuver. Existing learning-based planners can capture these influences through latent scene features and trajectory decoders, but the relationship between environmental factors and candidate actions often remains implicit. This limits the ability to inspect, diagnose, or refine how scene context affects the safety of a predicted trajectory. Classical safety fields provide an explicit spatial representation of this relationship, but their risk shapes and relative weights are prescribed in advance and do not adapt to each scene. We introduce READ, a framework that learns an explicit, planning-aligned risk representation from complementary
geometric and behavioral constraints. READ instantiates this representation as a continuous spatiotemporal field, enabling differentiable queries along candidate trajectories. The learned field connects scene understanding with action selection by encouraging predicted trajectories to align with low-risk regions, while retaining a differentiable interface for trajectory evaluation and refinement. READ integrates with both end-to-end planners and
Vision-
Language-Action models. Experiments on NAVSIM show consistent gains across matched end-to-end backbones and strong performance in a VLA setting; READ also achieves competitive results on NAVSIM v2. These results establish learned spatial risk as an explicit, adaptable representation for safe planning.
Read the full abstract ▾
All authors Zhiyuan Liu, Yuanxin Tian, Zehong Ke, Jinhao Li, Hao Cheng, Zhenhua Xu, Wenhao Yu, Jianqiang Wang
Robotics
A Deployable Architecture for Robot-Mediated Tasks (DART): Evaluation in Socially Assistive Robot-Guided Cognitive Behavioral Therapy Exercises
Mina Kian, Lydia Ignatova, Jiong Wang et al. (+12 more)
Socially assistive
robots (SARs) can support structured health and well-being interventions, but hardware and cost constraints limit interaction complexity and longitudinal real-world deployments. We present DART: Deployable Architecture for
Robot-Mediated Tasks, an architecture that extends SARs through a web application and cloud infrastructure, enabling visual content, user input, remote computation, and persistent data storage synergistically with the
robot's physical embodiment,
speech, and movement. We evaluated DART by instantiating it in an interatively-developed full-stack HRI system for helping university students with elevated generalized anxiety to complete cognitive behavioral therapy (CBT) homework exercises. The resulting system, which used the low-cost open-source Blossom
robot platform, was refined and evaluated through a participatory design process and multiple user studies, and finally evaluated in an in-lab study with 103 participants, and then a six-week in-home deployment with four participants. In the in-lab evaluation, participants showed significant within-session reductions in stress, state anxiety, and negative affect, and gave the platform a mean System Usability Scale score of 78.89. In the home deployment, the mean System Usability Scale score was 87.5, with positive qualitative feedback on usability. Participants across both groups identified
speech input, visual presentation, and web-
robot synchronization as priorities for improvement. These findings validate DART as an effective architecture for extending the capabilities of a low-cost SAR in both in-lab single-session and in real-world longitudinal deployments.
Read the full abstract ▾
All authors Mina Kian, Lydia Ignatova, Jiong Wang, Ji Min Lee, Jiancheng Li, Qianwei Guo, Emily Weiss, Amy O'Connell, Kaitlin Zareno, Jiani Li, Reyna Patel, Leyaa George, Minyu Huang, Justin Yang, Maja J. Matarić
Robotics
DWMP: Leveraging Dual World Models for Humanoid Obstacle Traversal
Rongjun Jin, Jianming Ma and Yue Gao
Humanoid
robots must traverse cluttered obstacle fields using onboard proprioceptive and visual observations, yet existing methods usually process
multimodal observations without explicitly considering their different characteristics: proprioceptive observations are low-dimensional but governed by highly nonlinear
robot dynamics, while egocentric visual observations are high-dimensional, noisy, and redundant. We propose DWMP (Dual World Model Policy), a framework that provides the actor with separate but complementary world-model representations for humanoid obstacle traversal. A Koopman-based dynamics world model lifts proprioceptive observations into a latent space where their temporal
evolution is approximately linear, making the dynamics features easier for the actor to learn from. An RSSM-based visual world model compresses egocentric depth observations into compact
stochastic states while preserving obstacle-related geometry. The student policy receives the fused latent representation for action generation, combining linearized proprioceptive dynamics with compressed visual perception. Experiments in simulation and on a Unitree G1 humanoid
robot show that DWMP improves obstacle traversal performance over baselines and supports real-world deployment under randomized obstacle layouts.
Read the full abstract ▾
Robotics · Cross-listed from eess.SY
The Iterative Equivariant Filter
Pieter van Goor and James Richard Forbes
This paper presents the iterative equivariant filter (IterEqF). The iterative extended Kalman filter (IterEKF) replaces the standard EKF correction step with an iterative correction step that is the Gauss-Newton solution to a nonlinear weighted least squares problem. The standard equivariant filter (EqF) exploits and respects the underlying
symmetry of state estimation problems posed on homogeneous spaces and Lie groups. The motivation behind the IterEqF is to combine the features of both the IterEKF and the EqF, thus leading to a high-performance state estimation solution that is well suited to navigation problems. This paper derives the iteration procedure for the IterEqF update step and shows how the intrinsic nonlinearity of the approach naturally results in a `reset' of the filter's covariance into new coordinates. Monte-Carlo simulations of range-based localisation for a mobile
robot demonstrate the improvement in performance relative to a standard EqF, especially during the transient convergence.
Read the full abstract ▾