Robotics · Cross-listed from cs.LG
Industrial Kinematic Trajectory Model (IKTM): Coordinate-Free Autoregressive Generator
Max Amiri and David Eyers
Mobility simulation supports logistics, safety, and communications planning in industrial environments such as ports, mines, and airports. Existing trajectory models, however, rely on absolute coordinates, road-network
tokens, or semantic zones: representations that are site-specific and not well suited to unstructured industrial terrain. We introduce the Industrial Kinematic Trajectory Model (IKTM), a coordinate-free trajectory generator that represents industrial vehicle motion through kinematic sequences (speed and heading change) with no absolute spatial reference. IKTM uses an autoregressive causal
transformer with probabilistic mixture heads and extends our prior coordinate-free Markovian model with deep sequence modelling and an explicit duration-conditioning signal. Trained on one site and evaluated zero-shot on three unseen sites, it matches the small-turn shape of the empirical turn-rate distributions of held-out telematics; across all four sites, the per-site mean Jensen-Shannon divergence over 100 sampling seeds spans approximately 0.035-0.050 bits under an oracle-length duration-target protocol and approximately 0.032-0.042 bits under a fully zero-shot prior-length protocol, with similar ranges whether out-of-distribution (OOD) duration targets are drawn from each held-out site's empirical length distribution or from the Site A prior. Both protocols stay above the metric's sampling-noise floor (<=0.0049 bits). Paired by site, the oracle-length values are 5.3-6.0x lower than those of a re-implementation of our prior Markovian model under the same 1 Hz protocol. Termination is duration-conditioned rather than spatial: rollouts stop on 100% of trials with a length-tracking error of +0.0 +/- 0.0 s against the sampled target (100 of 100 exactly on target; N=100, T=0.2, untouched in-distribution test split).
Read the full abstract ▾
Robotics
Multi-Group Pipe Routing under Permanent Geometric Occupancy: Problem, Benchmark, and Classical Baselines
Deng Quan
Additive manufacturing (AM) enables compact hydraulic components whose internal fluid channels can follow free-form 3D paths rather than conventionally drilled holes. A representative case is multi-group channel layout in rotary direct-drive servo valves: once a channel is placed it permanently occupies volume, so later channels must clear earlier geometry–unlike classical multi-
agent pathfinding (MAPF), where
agents free space after moving. We study this setting as multi-group pipe routing under permanent
geometric occupancy. Our contributions are a problem formalization with
geometric dual-witness conflicts, a constructive 3D
benchmark (two corridor generators x two obstacle painters, controlled difficulty, feasibility witnesses), and baseline results for CBS, PBS, and priority planning adapted to this coupling. We evaluate by success rate under a fixed time budget. PlaneSlice hard instances clearly rank CBS above PBS and PP, while easier
cells validate the pipeline. The goal is a reproducible problem definition, suite, and classical baselines–not a new optimal MAPF
algorithm.
Read the full abstract ▾
Robotics
RoboGesture: Real-Time Semantic-aligned Co-Speech Gestures Generation for Humanoid Interaction
Zifan Wang, Ziang Ren, Pengyang Shi et al. (+7 more)
Enabling humanoid
robots to respond to human
speech with synchronized and semantically meaningful gestures is fundamental to natural human-
robot interaction. However, this task faces three critical barriers: the scarcity of semantically rich
datasets, the "modality eclipse" where models ignore audio cues in favor of kinematic inertia, and the sim-to-real gap regarding physical safety. We propose RoboGesture, a
robot-centric framework that co-designs data, modeling, and control to power a complete interactive human-humanoid system in which the
robot listens, responds, and gestures in real time. We first establish the RoboGesture
dataset featuring over 300 gesture categories and develop an automated pipeline to synthesize large-scale collision-free,
robot-specific audio-motion pairs. Our architecture features a Hierarchical Semantic-Acoustic Aligner that extracts multi-granular prosodic and semantic cues directly from raw audio
tokens. These cues drive a Streaming Conditional Motion Generator based on a
diffusion transformer with conditional flow matching. To ensure high responsiveness, we introduce Anti-Inertia CFG Masking, which prevents the model from collapsing into repetitive historical patterns by compelling it to proactively mine control signals from the audio modality. Finally, an MPC-based safety filter ensures real-time, collision-free execution on physical hardware. Experiments on a Unitree G1 humanoid demonstrate that RoboGesture generates safer, more rhythmic, and more semantically appropriate responses compared to state-of-the-art baselines.
Read the full abstract ▾
All authors Zifan Wang, Ziang Ren, Pengyang Shi, Zirui Wang, Chenghuai Lin, Tianze Wang, Zekun Qi, Liangliang Zhao, He Wang, Li Yi
Robotics
Coordinated Motion Planning for Multi-Arm Systems via Iterative LQ Games
Junyoung Kim, Hanwen Ren, Lei Zhang et al. (+1 more)
Multi-
agent motion planning for high-degree-of-freedom robotics manipulators in shared workspaces remains a fundamental yet challenging problem. Centralized planners often suffer from poor scalability, while decentralized approaches face robustness and safety concerns. Game-theoretic formulations offer a promising approach for modeling
agent interactions, potentially overcoming these limitations. However, their application to articulated multi-arm systems remains limited. This paper presents an iterative Linear Quadratic (LQ) game framework for multi-manipulator motion planning, where each manipulator is modeled as an independent
agent optimizing its own objective while interacting with other
agents based on shared global states and collision constraints. The method solves a series of local LQ games by linearizing the dynamics and approximating the cost around a nominal trajectory, with Riccati backward recursions yielding feedback Nash strategies. To address the challenges of articulated systems, we incorporate differentiable penalties for self-collision and inter-arm collision into the optimization pipeline, enabling coordinated, collision-aware trajectory generation. Experiments demonstrate that our framework produces smooth, safe, and efficient trajectories in high-dimensional settings, outperforming traditional methods. This highlights the effectiveness of differential game formulations for multi-
robot manipulation.
Read the full abstract ▾
All authors Junyoung Kim, Hanwen Ren, Lei Zhang, Ahmed H. Qureshi
Robotics · Cross-listed from eess.IV
CARDINAL Predicts Cardiovascular Risk From Non-contrast Cardiac CT
Roy Gabriel, Nattakorn Kittisut, Jamshid Hassanpour et al. (+6 more)
Cardiovascular risk prediction remains limited by incomplete clinical data and imaging biomarkers that reduce computed
tomography (CT) to a small number of handcrafted features. We developed CARDINAL (Cardiovascular Assessment via Representation learning from Deep Imaging with Nested Anatomical Latent
embeddings), a clinically grounded framework that learns compact representations from routine non-contrast cardiac CT for major adverse cardiovascular event (MACE) prediction. In 17,659 patients, CARDINAL was evaluated for 1-, 3-, 5-, and 10-year MACE prediction against American Heart Association (AHA) pooled cohort equations (PCE), AHA predicting risk of cardiovascular disease events (PREVENT), coronary artery calcium (CAC), segmentation-derived CT biomarkers, and 70-feature structural radiomics. Gains were largest at longer horizons. At 10 years, CARDINAL (joint) achieved an area under the receiver operating characteristic curve (AUROC) of 0.866 $\pm$ 0.020 and an area under the precision-recall curve (AUPRC) of 0.890 $\pm$ 0.015, compared with an AUROC of 0.826 $\pm$ 0.023 and an AUPRC of 0.826 $\pm$ 0.022 for structural radiomics, the strongest baseline. CARDINAL also achieved the highest survival concordance index (C-index), 0.753 $\pm$ 0.015, and high-versus-low risk-tertile hazard ratio, 10.78 $\pm$ 3.16, with favorable reclassification and exploratory calibration. These findings suggest that non-contrast cardiac CT contains prognostic information beyond conventional risk equations, CAC scoring, and engineered imaging biomarkers.
Read the full abstract ▾
All authors Roy Gabriel, Nattakorn Kittisut, Jamshid Hassanpour, Michael Galarnyk, Abanoub Abdelmalak, Marly van Assen, Carlo N. De Cecco, Arshed Quyyumi, Ali Adibi
Robotics
Distributed Model-Based Diffusion: Finite Horizon Contraction under Bounded Delay
Seth Golembeski, Keith L. Gibson, Alexander Gross et al. (+2 more)
Simultaneously optimizing the trajectories of multiple
agents is a challenging problem plagued by nonlinearity, nonconvexity, and the curse of dimensionality. A collection of interacting aerial vehicles or self-driving cars in an intersection are examples of complex multi-
agent systems that remain difficult to solve without many simplifying assumptions. The presence of communication latency between
agents further increases the difficulty. In this paper, we analyze Distributed Model-Based
Diffusion: a sampling-based Model-Predictive Control method suitable for highly nonlinear, nonconvex, nonsmooth, multi-
agent systems. We prove contraction and robustness to latency for multi-
agent, nonconvex problems, showing applicability to real-world constraints. We test the
algorithm on a circleswap task, a cooperative medium-fidelity driving task, and in an aerial combat scenario. Despite the addition of latency, our
algorithm improves circleswap makespan by 31% and increases aerial combat win rate by 25% compared to centralized Model-Based
Diffusion.
Read the full abstract ▾
All authors Seth Golembeski, Keith L. Gibson, Alexander Gross, Shreyas Kousik, Anirban Mazumdar
Robotics
One year in a forest: Analyzing the challenges of autonomous navigation in subarctic environments
Matěj Boxan, Nicolas Lauzon, Veronica Vannini et al. (+2 more)
Subarctic regions have the potential to see increased deployment of autonomous
robots in applications including forestry, mining, and environmental monitoring. In these conditions, an autonomous system's reliance on GNSS or cloud computing is precarious due to dense tree canopies and atmospheric attenuation, necessitating onboard sensing and data processing. However, established exteroceptive modalities, including cameras,
lidars, and radars, are typically evaluated in structured urban settings or in environments that lack significant seasonal variations. To address this, we present a field report on a year-long deployment of a mobile
robot in a subarctic boreal forest. We evaluate 64 km of data using nine odometry, localization, and mapping methods and assess their performance across seasonal changes. The performed experiments suggest that the environment changes significantly hinder the performance of state-of-the-art techniques, which show increased fragility when subject to conditions characterized by self-similar scenes or tall snowbanks. Additionally, complex Simultaneous Localization and Mapping (SLAM)
algorithms offer limited accuracy gains over a proprioceptive baseline while significantly increasing system fragility. Furthermore, by correlating the position drift with features and confidence weight distribution, we show that visual-based SLAM methods are particularly affected by the seasonal changes. Additionally, we investigate the task of cross-season localization in a prior map. While
lidar-based methods successfully completed localization runs between seasons, radar and visual methods are prone to failure due to a few matching features between runs, even within the same season. Finally, we detail the challenges and lessons learned from this year-long trial, including a multi-season Teach and Repeat (T&R) evaluation using both radar and
lidar-based pipelines.
Read the full abstract ▾
All authors Matěj Boxan, Nicolas Lauzon, Veronica Vannini, Mathis Turgeon-Roy, François Pomerleau
Robotics
Davood Soleymanzadeh, Kaidi Zhang, Zhiyuan Zhang et al. (+4 more)
Vision-
language-action models (VLAs) have shown strong promise for general-purpose robotic manipulation by mapping
language instructions and
vision observations directly to actions. However, most VLAs primarily condition action prediction on current observations and lack an explicit mechanism for
reasoning over future task dynamics, which is particularly important for fine-grained, contact-rich manipulation. We present PHR-VLA, a framework that enables planning-horizon
reasoning in VLAs through privileged latent representations of future dynamics. PHR-VLA introduces a lightweight auxiliary future head that, during training, aligns the VLA's internal representations with latent dynamics extracted from future observations. Evaluation results demonstrate that local, contact-centric, patch-level latent dynamics supervision from the wrist camera improves success rate on LIBERO from 84.1% to 88.4% and on real-world disassembly tasks from 63.3% to 82.5%. Patch-level supervision from a third-person camera also improves performance on Meta-World from 56.70% to 57.8%. These results demonstrate that privileged latent dynamics
alignment provides an effective training signal for improving anticipatory
reasoning in VLA policies. Project website: \href{https://davoodsz.github.io/PHR-VLA.github.io/}{https://davoodsz.github.io/PHR-VLA.github.io/}
Read the full abstract ▾
All authors Davood Soleymanzadeh, Kaidi Zhang, Zhiyuan Zhang, Bihao Zhang, Xiao Liang, Yu She, Minghui Zheng
Robotics
Beyond Data Scaling: Representation-Centric Continued Pre-training for Vision-Language-Action Models
Senqiao Yang, Chengyao Wang, Yuxin Chen et al. (+13 more)
Scaling
robot data is crucial for building generalist
Vision-
Language-Action (VLA) models, yet
robot trajectories are harder to scale than web-scale image-text data because embodied collection is costly and sparsely covers the physical world. This makes representation quality a central bottleneck: under a fixed
robot-data budget, continued pre-training must turn limited trajectories into transferable visual-action knowledge rather than merely fit actions. We propose VLAct, a VLA-oriented VLM backbone trained on broad, heterogeneous, multi-embodiment
robot data before task-specific fine-tuning. VLAct preserves the broad VLM prior and encourages shared action semantics across embodiments through VLM-prior preservation, multi-head continuous action co-supervision, and a partially unified cross-embodiment action layout, while allowing task-specific action heads during fine-tuning. Across simulation, real-world, and unseen-embodiment transfer, VLAct consistently improves downstream performance under fixed fine-tuning protocols. On LIBERO-Plus and RoboTwin 2.0, VLAct surpasses industrial VLA systems including ABot-M0 and LingBot-VLA, achieving success rates of 82.6% and 92.5%. On RoboDojo, VLAct ranks sixth among all policies by success rate and outperforms all explicitly designated world-action model (WAM) entries on both metrics. Most notably, on RoboCasa-GR1, an unseen humanoid embodiment, VLAct using only 20% of downstream trajectories outperforms the full-data GR00T-N1.6 baseline. These results are obtained using fully open-source data and only a 16-GPU training setup, showing that representation-centric continued pre-training can deliver highly competitive performance under a modest compute budget and is an important independent axis of VLA progress beyond data scaling.
Read the full abstract ▾
All authors Senqiao Yang, Chengyao Wang, Yuxin Chen, Zixuan Wang, Longxiang Tang, Haokun Gui, Jinhui Ye, Changsheng Lu, Xiaoyang Wu, Mingkang Zhu, Pengguang Chen, Shu Liu, Zhuotao Tian, Hengshuang Zhao, Bei Yu, Jiaya Jia
Robotics
Remote Human-Robot Interaction In Greenhouses via Virtual Reality: How Plant Canopy Structure Affects Leaf Disease and Soil Moisture Inspection
Daniel Udekwe and Hasan Seyyedhasani
This study evaluates the effectiveness of remote human-
robot interaction using virtual reality for leaf inspection and soil moisture assessment in a greenhouse environment. The robotic system comprised an unmanned ground vehicle and a robotic manipulator equipped with cameras, governed by kinematic models for navigation and manipulator control. Fourteen distinct plants were inspected across two experiments utilizing VR teleoperation, guided by a set of pre-specified research questions and hypotheses. In the leaf inspection experiments, cycle completion times varied from 3.3 to 8.0 s, and plant-based disease detection was achieved up to 88% accuracy; diseased-spot detection improved numerically in the second experiment, though this change was not statistically significant (p=0.378). For soil moisture assessment, the experiments achieved successful determination of watering needs in up to 64.3% of plants (9 of 14), with consistent success observed for plants 1, 2, 3, 8, 9, 10, and 13; however, this improvement was likewise not statistically significant (p=0.50). A post hoc analysis instead revealed that soil moisture assessment reliability was strongly and significantly predicted by plant canopy morphology (p<0.01): plants with broad, single-leaf canopies reached 100% success by the second experiment, versus only 16.7% for dense, compound canopies. A secondary analysis showed operators became measurably faster at attempting dense-canopy plants without a corresponding gain in success, indicating that camera occlusion, not operator skill or effort, is the dominant limiting factor. These findings show occlusion imposes a sensing limitation rather than a control or training deficiency, and that adapting camera viewpoint and sensing strategy to canopy density is needed to improve the system's accuracy and robustness.
Read the full abstract ▾
Robotics
Beyond Relative Geometry: Metric-Aware Geometry Perception for Robotics
Fengjun Zhong, Congjia Chen, Zhaoxu Liu et al. (+7 more)
Recent embodied models increasingly leverage
geometric representations to improve spatial
reasoning and robotic manipulation. However, existing reconstruction methods only reconstruct relative geometry with arbitrary scales, causing predicted object dimensions and spatial distances to vary across scenes, viewpoints, and input configurations. This inconsistency prevents
geometric perception from being directly aligned with robotic actions defined on the real-world scale. To address this limitation, we propose Metric-Aware Geometry Perception (MAGP), an end-to-end, plug-and-play framework for metric geometry reconstruction that can be seamlessly integrated into robotic policies. At its core, Metric Scale Equivariant Augmentation encourages the model to reconstruct metric geometry from camera parameters and depth observations, ensuring that the reconstructed geometry follows the metric scale specified by observations. Flexible Metric Conditioning further enables MAGP to support arbitrary view counts and combinations of camera and depth inputs, improving robustness to heterogeneous robotic sensing configurations. Together, these designs produce geometrically consistent reconstructions with stable object dimensions and spatial distances across scenes and sensing conditions. Experiments on ETH3D, MegaDepth, and ScanNet++ demonstrate that MAGP maintains strong relative geometry accuracy while reducing the absolute error by over an order of magnitude, from 2.01m to 0.07m. When integrated into multiple robotic policies, MAGP consistently improves performance on LIBERO, RoboTwin, and zero-shot LIBERO-Plus, with gains of up to 6.26% on RoboTwin. These results demonstrate the effectiveness and generalizability of metric geometry for robotic manipulation.
Read the full abstract ▾
All authors Fengjun Zhong, Congjia Chen, Zhaoxu Liu, Jinyang Du, Yuchen Gong, Enqi Mao, Ruihao Gong, ShuJie Wang, Xianglong Liu, Zhongliang Qiao
Robotics
CLAP: Cross-Embodiment Video World Models are Zero-Shot Physical Simulators
Kechen Liu and Ola Shorinwa
State-of-the-art action-conditioned
video models are typically restricted to a single
robot embodiment, preventing them from leveraging the vast corpus of heterogeneous
video data that contains rich signals for learning generalizable physics. To bridge this gap, we introduce CLAP, a framework for cross-embodiment action-conditioned
video generation capable of being trained on diverse, internet-scale
videos across human and robotic
agents. CLAP is grounded in the insight that universal physical laws govern spatiotemporal dynamics regardless of the actor. However, cross-embodiment learning is non-trivial because action representations vary sharply across
robot platforms and are typically absent in human
videos. CLAP addresses this fundamental challenge through the following core contributions. First, CLAP reconciles disparate action spaces using end-effector poses,
language instructions, and latent actions. Second, to resolve their individual limitations, CLAP introduces a curriculum-based cross-embodiment learning recipe that first learns foundational physical priors across unlabeled
video data using latent actions and subsequently grounds them in end-effector action spaces for zero-shot deployment to real-world tasks. Crucially, CLAP approaches or surpasses state-of-the-art single-embodiment
video models in challenging environments like DROID. These performance advantages compound via few-shot adaptation to establish a novel paradigm for training single-embodiment
video world models. Ultimately, CLAP delivers the most comprehensive suite of action-conditioned
video world models to date - spanning diverse action-conditioning spaces (end-effector,
language, and latent) and
robot morphologies (including cross-embodiment, DROID, Bridge, bimanual YAM
robots, and G1 humanoids). We open-source all code and models. Project Website at https://omni-clap.github.io .
Read the full abstract ▾
Robotics
FlashVLA: Streaming Action Decoding for Fast and Asynchronous VLA Inference
Zekai Li, Jiaming Tang and Zhijian Liu
Vision-
Language-Action (VLA) models are increasingly promising for robotic manipulation, yet their real-world deployment remains bottlenecked by high inference latency and unstable asynchronous execution. This challenge is particularly pronounced in flow-matching-based VLA models, where action decoding requires multiple iterative steps conditioned on the VLM context. While efficient inference methods improve control frequency and asynchronous methods reduce execution idle time, existing approaches often fail to jointly achieve low-latency inference and accurate, temporally consistent asynchronous execution. We introduce FlashVLA, a streaming action decoding framework that addresses both challenges in a unified formulation. FlashVLA maintains a streaming action buffer with multiple chunks at different noise levels and decodes them using chunk-wise causal
attention. This design allows FlashVLA to produce one executable action chunk per inference step. Moreover, its chunk-wise autoregressive formulation implicitly preserves action continuity, enabling smooth asynchronous execution without extra future-state conditioning. Across extensive simulated and real-world experiments, FlashVLA substantially improves inference speed while maintaining strong task performance. It can achieve $\geq$30\,Hz control frequency on a single GPU with smooth asynchronous inference in real-world deployment.
Read the full abstract ▾
Robotics
Embodied Scene Rearrangement Planning
Canzhi Chen, Zan Wang, Siqi Zhu et al. (+3 more)
This paper introduces Embodied Scene Rearrangement Planning (ESRP), a novel task requiring embodied
agents to rearrange furniture in 3D scenes to match a target configuration using only egocentric observations and a top-down target layout. Unlike prior rearrangement tasks, ESRP precludes global state access and introduces mutual object occlusions, reflecting the practical constraints of real-world robotic deployment. These factors make aligning partial egocentric observations with the global target layout particularly challenging for long-horizon planning. To facilitate research, we present ESRP-Bench, a comprehensive
benchmark built on OmniGibson featuring over 5,400 scene pairs and 8,200 objects. We define three multi-level metrics to evaluate rearrangement quality and provide four baselines: a hierarchical task-and-motion planning method, a
vision-
language-model-based method, and two learning-based approaches (IL and RL). Experimental results demonstrate that current methods struggle to complete the task efficiently, highlighting ESRP as a challenging frontier for embodied
agents in scene understanding and long-horizon task planning. This work serves as a stepping stone toward deploying intelligent
agents in real-world scenarios. Project page: https://pie-lab.cn/ESRP/.
Read the full abstract ▾
All authors Canzhi Chen, Zan Wang, Siqi Zhu, Qi Wu, Yixuan Li, Wei Liang
Robotics · Cross-listed from cs.CV
TADP: Task-Aware Deformable Prediction for Single-Stage 3D Object Detection
Su Wang, Yaochen Li, Min Yang et al. (+3 more)
Most single-stage 3D object detectors complete different tasks with the same extracted features. Nevertheless, it is impossible to project features into a common space that is adaptive for all the tasks. We present a novel task-aware deformable prediction (TADP) method for single-stage 3D object detection to solve this problem. Firstly, a triple feature refinement aggregation module is designed to extract three-level features adaptively. Additionally, we design the multi-scale feature aggregation block to fuse multi-scale features in a scale-aware manner. Finally, the prediction of each task is deformed with the designed plug-and-play task-aware deformation head. It can percept the emphasis and interaction of each task. We also designed three different deformation modules. The experimental results demonstrate that the proposed deformation head shows good results on other detection methods. The experimental results on the KITTI
dataset demonstrate that the car mAP is 80.91%, surpassing many state-of-the-art methods on the KITTI
benchmark.
Read the full abstract ▾
All authors Su Wang, Yaochen Li, Min Yang, Jiaohao Nie, Chang Liu, Yuehu Liu
Robotics · Cross-listed from math.OC
Marine Autonomous Vehicle Fleet Scheduling to Maximise Scientific Impact
Mehdi El Krari, Jonathan Smith and Maria Fox
The marine science community increasingly relies on Marine Autonomous Vehicles (MAVs) to collect the critical environmental data required to understand global ocean systems. However, as these operations scale, manually routing and planning large autonomous fleets becomes exponentially complex and time-consuming. To address this, we propose a mixed-integer linear programming (MILP) model designed to automate and optimise MAV deployment schedules. The model accounts for strict operational constraints, including battery capacities and time windows for data collection, while aiming to maximise total data collection and minimise both the number of deployed vehicles and their energy consumption. A key novelty of this framework is integrating conventional ship itineraries, allowing MAVs to support vessels with mid-mission battery swapping or accelerated transit between waypoints. Computational experiments demonstrate that the model is highly scalable, solving routing problems for fleets of dozens of MAVs in seconds, and scaling to hundreds of vehicles in only a few minutes. Beyond operational scheduling, the framework serves as a robust simulation tool for evaluating 'what-if' scenarios and analysing the impact of varying parameters on deployment strategies. Finally, the solution generates a suite of visualisations designed to enhance explainability and support strategic decision-making for stakeholders.
Read the full abstract ▾
Robotics
STEP: State-Aware Task Estimation and Planning with Multi-Modal LLMs for Human-Robot Collaboration
Maitrey Gramopadhye, Prakash Baskaran, Xiao Liu et al. (+2 more)
Effective human-
robot collaboration in industrial settings requires
robots to understand human intentions and assist with task planning, reducing workload. Recent works have explored the use of Multi-modal Large
Language Models (MM-LLMs) for task planning in such data-scarce scenarios, leveraging in-context learning to interpret user actions and generate long-horizon action plans in natural
language. However, MM-LLMs inherently lack an understanding of system states and do not track state transitions, often leading to hallucinated actions that deviate from the intended goal. Additionally, generating action plans in natural
language tends to limit the generated plans to a high level, introducing ambiguity in action execution. To address these limitations, we propose the State-aware Task
Estimator and Planner (STEP), which
prompts a MM-LLM to explicitly estimate the state of the system and predict the state transitions resulting from executed actions. By forecasting future states alongside actions, STEP ensures task-convergent planning while also providing additional assistance parameters necessary for executing the predicted actions. We evaluate STEP in a simulated environment using a
robot assembly task. Our approach outperforms the state-of-the-art by 32.8% in action executability and 14.8% in final-state error.
Read the full abstract ▾
All authors Maitrey Gramopadhye, Prakash Baskaran, Xiao Liu, Songpo Li, Soshi Iba
Robotics
Tensegrity Continuum Robots Enable Task-Adaptive Morphologies for Cooperative Behaviors
Mahmud Hasan Saikot, Sydney Spiegel, Sudheera Akalanka Kariyawasam et al. (+3 more)
Robots that can change their morphologies and behaviors for different tasks and environments hold great promise for adaptable, multifunctional systems. Modular reconfigurable
robots (MRRs) can achieve such functionalities by docking and rearranging individual units, but most rely on rigid modules that lack structural compliance, resulting in limited capabilities. Continuum
robots offer compliance through flexible backbones, yet they cannot self-reconfigure into task-adaptive multi-
robot configurations. Here, we introduce an MRR that unifies the advantages of both architectures by combining a tensegrity-based compliant body with claw-based connection mechanisms. Each
robot can manipulate and locomote independently, and multiple
robots can self-reconfigure into different morphologies (e.g., chains, loops, branches) for cooperative manipulation and locomotion. We demonstrate the
robots' capability across diverse tasks and environments, including coordinated object manipulation and transport,
multimodal locomotion, and loco-manipulation in real-world scenarios. These results lay a foundation for adaptable and multifunctional robotic collectives, with broad potential applications in manufacturing, space exploration, and search-and-rescue operations.
Read the full abstract ▾
All authors Mahmud Hasan Saikot, Sydney Spiegel, Sudheera Akalanka Kariyawasam, Andrew Stefka, Josh Chrisler, Jianguo Zhao
Robotics
Task-space model-based control of pneumatic soft actuators
Nithin S. Kumar, Joshua Gaston, D. Caleb Rucker et al. (+1 more)
Soft actuators enable dexterous and compliant interaction, but closed-loop task-space control remains challenging due to strong nonlinearities, distributed deformation, and uncertainty in their dynamics. This paper presents a real-time dynamic-model-based task-space feedback and estimation framework based on a non-minimal coordinate discrete elastic rod model formulated in absolute coordinates with holonomic constraints. The resulting structure preserves distributed mechanics while maintaining computational efficiency through sparse system
matrices, enabling real-time control with up to 10 discretized rods. A quasi-static feedforward inverse model is combined with a task-space PI controller and a dynamic observer that fuses measurement residuals as virtual forces, enabling full-state estimation from sparse sensing. The approach is experimentally validated on three planar pneumatic soft actuators with varying geometries. Across five tasks, including drawing the digits 0-9 across the workspace (3-18 mm/s tip speed), tracking periodic motion (up to 37 cm/s), cross-platform generalization, reduced sensing conditions, and real-time user-defined references, our method achieves 1.5-2.3 mm root mean square error (RMSE) for precision motions and 5.5-12.4 mm RMSE at 1-2 Hz. Results demonstrate that structured, non-minimal dynamic models can enable real-time, high-precision, moderate-bandwidth task-space control of planar soft pneumatic actuators in free space.
Read the full abstract ▾
All authors Nithin S. Kumar, Joshua Gaston, D. Caleb Rucker, Eric J. Barth
Robotics
Planning a Shared Modular Fixture Layout Across Robotic Disassembly Stages
Haohui Pan, Takuya Kiyokawa and Kensuke Harada
Stable support remains challenging in robotic disassembly of irregularly shaped products. As components are progressively removed, the available support surfaces, mass distribution, and task loads change throughout the process. A fixture layout designed for one workpiece state may therefore become infeasible at later stages, motivating unified support planning over the complete disassembly sequence. This paper presents a modular
vacuum-based fixturing system that plans one shared support configuration for the complete disassembly sequence of a screwdriver or shaver, allowing each sequence to proceed without fixture reconfiguration. To search the mixed continuous–discrete layout space under repeated cross-stage evaluation, a denoising
diffusion probabilistic model generates physics-informed initial configurations that are refined through
Bayesian optimization. Robotic screw and component-removal experiments verified the disassembly feasibility of the planned layouts, while 11 directional-load tests quantified their stability. Comparisons between the measured operational loads and directional responses yielded mean empirical stability margins of 66.9% for the screwdriver and 81.6% for the shaver. These results demonstrate that a product-specific shared layout can provide stable support throughout the tested robotic disassembly sequence.
Read the full abstract ▾
Robotics
Active sensing to characterize the heterogeneity of plant stress
Ayman Laaroussi, Peter Hanappe and David Colliaux
While most phenotyping platforms rely primarily on image-based measurements, advanced plant characterization requires the integration of active physiological sensing modali- ties such as chlorophyll fluorescence. We present an autonomous robotic platform designed to perform targeted fluorescence measurements on plant leaves. The system combines 3D plant reconstruction,
geometric analysis, and motion planning to localize suitable measurement points and generate collision-free trajectories for a robotic manipulator. A dense 3D model of the plant is reconstructed from multi-view data and used to extract candidate leaf surfaces based on orientation, accessibility, and sensing constraints. These targets are then integrated into a task-level planning framework that guides the end-effector to precise contact or near-contact configurations required for point-based fluorescence acquisition. The platform enables automated, repeatable, and spatially resolved physiological measurements that go beyond passive imaging. By tightly coupling perception,
geometric reasoning, and manipulation, the proposed system provides a robotics-driven approach to high-resolution plant phenotyping and opens new directions for autonomous agricultural inspection and plant-aware manipulation.
Read the full abstract ▾
Robotics
Pass the Bucket: Efficient, Robust, Local Load Balancing for Teams of Heterogeneous Robots
Tobias Wallner, Dominik Krupke, Arne Schmidt et al. (+1 more)
We study the problem of decentralized, self-organized task sharing for a swarm of heterogeneous
robots that collaborate in transportation or other objectives that require coordinated motion planning. To this end, we present theoretical and practical results for the simple but effective mechanism of bucket brigades for load balancing, in which a team of heterogenous
robots share a spatial task in a confined, one-dimensional space, while only being able to sense collisions with neighbors or walls. The goal is to optimize throughput of the overall system, without central control or information, aiming at an interval partition proportional to
robot velocities. We address possible chaotic system behavior by developing a stabilization mechanism based on simple local aid, a “
token”, that temporarily decelerates
robots after an encounter. This purely local change eliminates persistent oscillations, resulting in convergence towards a stable system state. We accelerate system convergence by comparing a single boundary
token to ubiquitous two-directional
tokens and optimizing the deceleration factor. Event-driven simulations report convergence times and robustness: For a large variety of perturbations (such as
robot deletion, position or velocity jittering), the system reliably re-converges. The results suggest a local, practical mechanism for robust load balancing for heterogeneous teams of
robots that promises an effective tool as basis for more complex scenarios.
Read the full abstract ▾
All authors Tobias Wallner, Dominik Krupke, Arne Schmidt, Sándor P. Fekete
Robotics
GRAFT: Grounded and Efficient Online Reinforcement Adaptation for Fine-Grained Robot Manipulation
Yibo Qiu, Haoliang Ye, Shu'ang Sun et al. (+3 more)
Pretrained
vision-
language-action (VLA) policies provide strong priors for
robot manipulation, yet adapting them online to fine-grained biomedical tasks remains challenging. Task success often hinges on subtle, view-dependent visual cues, while task-level rewards provide little guidance about which regions matter, making it difficult to learn task-relevant visual grounding from limited real-
robot interaction. Online adaptation is further constrained by the computational cost of VLA inference and replay-based updates. We introduce GRAFT (Grounded Reinforcement Adaptation for Fast Task Learning), a framework for efficient online VLA adaptation through grounded perception. GRAFT uses region-level supervision to learn view-specific visual anchors that focus perception on task-relevant local cues without requiring region proposals at deployment. It further combines single-step action generation with cached visual-
language prefix reuse to accelerate online learning. Across four biomedical manipulation tasks, GRAFT improves success rates by 32.5 percentage points under matched adaptation budgets, while reducing the computational overhead of online policy updates.
Read the full abstract ▾
All authors Yibo Qiu, Haoliang Ye, Shu'ang Sun, Zan Huang, Ronald X Xu, Mingzhai Sun
Robotics · Cross-listed from cs.CV
SpatialCrafter: Single Image World Modeling with Generative 3D Proxies
Chuan Fang, Lingteng Qiu, Yixun Liang et al. (+8 more)
Explorable image-to-scene generation is essential for applications in gaming, robotics, and virtual reality. Existing methods based on
video diffusion model (VDM) commonly rely on incomplete conditioning signals such as sparse point clouds or 2D panoramas, leading to
stochastic hallucinations, long-term drifts and suboptimal 3D consistency. We present SpatialCrafter, a novel two-stage framework that addresses these issues by introducing a global 3D proxy for high-fidelity image-to-scene generation. Specifically, we decompose the generation process into global proxy generation and appearance refinement. For proxy generation, we propose a Point-anchored Sparse Structure~(PaSS) Flow module that predicts a spatially aligned and geometrically consistent 3D proxy. For appearance refinement, we re-frame the VDM as a Generative Deferred Refiner which synthesizes high-frequency photorealistic details upon proxy-defined scene geometry. To better integrate the proxy with the pre-trained VDM, we introduce Parallel Geometry Injection and Proxy-Aware Corruption training strategies, which improve robustness to proxy artifacts without disrupting the pretrained generative
manifold. Furthermore, as no suitable
dataset exists for this explorable scene generation task, we construct a new large-scale
dataset of 115K scenes. To the best of our knowledge, it is the first hybrid
dataset for image-to-scene generation. Extensive experiments on both synthetic and real-world
datasets show that SpatialCrafter outperforms state-of-the-art methods, mitigates long-term drift, and remains robust and consistent under rapid camera motion and extreme viewpoint changes. Our project page: \href{https://fangchuan.github.io/SpatialCrafter/}{fangchuan.github.io/SpatialCrafter/}
Read the full abstract ▾
All authors Chuan Fang, Lingteng Qiu, Yixun Liang, Rui Chen, Kunming Luo, Zhaohua Zheng, Tongyuan Bai, Feipeng Tian, Zilong Dong, Zihan Zhou, Ping Tan
Robotics
Riemann-1.0: An Embodied World Action Model for Physical AI
Haofeng Sun, Jiangbo Pei, Fei Kang et al. (+13 more)
We introduce Riemann-1.0, a fully causal autoregressive World Action Model for embodied intelligence. Riemann-1.0 jointly models multi-view visual observations,
robot states, and embodiment-specific actions within a unified causal autoregressive sequence, representing
robot actions and world
evolution as causal state transitions. Unlike existing WAMs based on joint generation,
video-first prediction, or decoupled modeling paradigms, Riemann-1.0 unifies online
robot policy execution and action-conditioned world simulation within a single model, enabling it to function as both an executable
robot policy and a multi-embodiment visual world simulator. To scale embodied experience across heterogeneous data sources, we further develop a progressive embodied pretraining framework that unifies learning from egocentric human
videos, handheld-gripper demonstrations, and heterogeneous
robot trajectories under a shared World Action Modeling objective. Built upon 200K+ hours of interaction data, Riemann-1.0 progressively transfers large-scale embodied experience into executable
robot manipulation capabilities. Riemann-1.0 achieves state-of-the-art performance across both simulation
benchmarks and real-world manipulation tasks. It achieves success rates of 94.3% on RoboTwin2.0, 99.0% on LIBERO, and 62.6% on the long-horizon compositional
benchmark RoboCasa-365, outperforming the previous best method by 8.4% On long-horizon real-world manipulation tasks, Riemann-1.0 achieves a Success Rate (SR) of 85.0% and a Progress Success Rate (PSR) of 94.4%, exceeding the strongest open-source baseline by 15% in SR. These results demonstrate that unified World Action Modeling together with progressive embodied pretraining effectively transforms large-scale embodied experience into generalizable
robot manipulation capabilities.
Read the full abstract ▾
All authors Haofeng Sun, Jiangbo Pei, Fei Kang, Zexiang Liu, Yaokun Li, Boyi Jiang, Hua Xue, Cindy Zhou, Wei Li, Yichen Wei, Mengyin An, Fanliang Zhao, Biao Jiang, Zile Wang, Yang Liu, Yangguang Li
Robotics
Arbitrary-Order Hermite Interpolation of Rigid-Motion Jets via Hyper-Multidual Quaternions
Daniel Condurache
We study bilateral interpolation of finite-order rigid-motion jets represented by unit dual quaternions. An order-$n$ multidual (MD)
algebra is the truncated
polynomial algebra $\mathbb{R}[\varepsilon]/(\varepsilon^{n+1})$; hyper-multidual (HMD) quaternions are dual quaternions with coefficients in this
algebra. Temporal HMD transforms encode a pose and its derivatives, whereas a generic HMD curve need not be the temporal jet of its pose projection; we call this requirement holonomicity. We show that a temporal transform and its relative descriptor are
unitary and derive recursive coefficient constraints, together with a local realizability converse in an admissible logarithm chart. We then extend screw linear interpolation (ScLERP) algebraically to unit HMD quaternions. Although it matches complete endpoint transforms, direct HMD–ScLERP is generically non-holonomic for arbitrary endpoint jets. We give a coefficient criterion and explicit endpoint and first-order interior contact defects. A holonomic alternative is obtained by mapping endpoint transforms to logarithmic dual-quaternion coordinates, applying the degree-$(2n+1)$ Hermite
polynomial that matches derivatives through order $n$, and lifting by the exponential. HMD arithmetic also recovers higher-order rigid-motion acceleration fields without explicit differentiation of $\mathrm{dexp}$. Rotation and full $\mathrm{SE}(3)$ tests through second order, with an additional third-order
polynomial check, reproduce the stated defects and endpoint jets.
Read the full abstract ▾
Robotics
4DSynth: Controllable Procedural World Synthesis for Dynamic Embodied Simulation
Zehao Qi, Haochen Luo, Jia-Wang Bian et al. (+2 more)
Embodied
agents need environments that are visually diverse, physically interactive, and changing over time. Procedural simulators can generate large interactive scene collections, and recent 4D generators produce compelling visual dynamics. Combining these properties in one environment, however, still demands extensive manual effort, and the result is rarely editable or controllable enough to reuse at scale. We present 4DSynth, a controllable procedural system that turns a natural-
language description, a blueprint mask, or a single photograph into an editable 4D environment with explicit geometry, animated actors, collision-free trajectories, and physics-ready simulation state. Multiple scene routes share one geometry-grounded representation, so the same pipeline handles animation, camera planning, rendering, and task generation. To validate the full pipeline, we construct 4DSynth-Nav, an interactive navigation
benchmark generated entirely from 4DSynth's procedural scenes. Two
vision-
language models evaluated across three difficulty tiers both fail the majority of tasks and stall after early subtasks. The same procedural controllability that produces these environments also makes each failure reproducible and each difficulty axis independently tunable. This paper presents both a controllable generation pipeline and the scalable
benchmark it enables, offering a practical foundation for developing and evaluating embodied
agents.
Read the full abstract ▾
All authors Zehao Qi, Haochen Luo, Jia-Wang Bian, Zeyu Ma, Shuyang Sun
Robotics · Cross-listed from cs.MA
Dynamic Haven Selection for Multi-Agent Pickup and Delivery in Constrained Warehouses
Taisei Hirayama, Kohei Yoshida, Hiroki Sakaji et al. (+1 more)
Space-efficient warehouse layouts often contain single-
agent-width aisles and dead-end workstations where
robots have few places to wait without blocking others. In Multi-
Agent Pickup and Delivery (MAPD) on such constrained layouts,
robots must accept online pickup-delivery tasks while preserving protected waiting locations called Havens. The Safe HAven Retreat Planner (SHARP) introduced a mechanism that extends each committed task path with a validated retreat to the
agent's dedicated initial Haven, but fixed-Haven commitments can send
agents toward distant Havens after deliveries. We present A-sharp (Adaptive SHARP), which changes an
agent's retreat target at task assignment time. A naive switch can cause two
agents to rely on the same waiting location or let another committed path pass through a location that is still occupied or reserved. A-sharp prevents these failures with an availability test for candidate Havens and a pending-release rule that keeps the previous Haven protected until the
agent departs. Under explicit Haven-structure and Safe Interval Path Planning (SIPP) assumptions, we prove invariant preservation and finite-release completeness: every task in any finite release sequence is delivered in finite time. Across 72,000 runs on 14,400 paired map-
agent-count-rate-seed cases over four maps, both SHARP and A-sharp complete their respective 14,400 runs. For makespan (final delivery time), a prespecified paired comparison with Holm correction over all 138 configurations with more Havens than
agents finds A-sharp significantly better in 107 configurations and never significantly worse than SHARP; on the tested tree map, the median reduction is 16.7%.
Read the full abstract ▾
All authors Taisei Hirayama, Kohei Yoshida, Hiroki Sakaji, Itsuki Noda