Robotics
Fabric Pneumatic Artificial Muscle-Based Head-Neck Exosuit: Design, Modeling, and Evaluation
Katalin Schäffer, Ian Bales, Haohan Zhang et al. (+1 more)
Wearable exosuits assist human movement in tasks ranging from rehabilitation to daily activities; specifically, head-neck support is necessary for patients with certain neurological disorders. Rigid-link exoskeletons have shown to enable head-neck mobility compared to static braces, but their bulkiness and restrictive structure inspire designs using "soft" actuation methods. In this paper, we propose a fabric pneumatic artificial muscle-based exosuit design for head-neck support. We describe the design of our prototype and physics-based model, enabling us to derive actuator pressures required to compensate for gravitational load. Our modeled range of motion and workspace analysis indicate that the limited actuator lengths impose slight limitations (83% workspace coverage), and
gravity compensation imposes a more significant limitation (43% workspace coverage). We introduce compression force along the neck as a novel, potentially comfort-related metric. We further apply our model to compare the torque output of various actuator placement configurations, allowing us to select a design with stability in lateral deviation and high axial rotation torques. The model correctly predicts trends in measured data where wrapping the actuators around the neck is not a significant factor. Our test dummy and human user demonstration confirm that the exosuit can provide functional head support and trajectory tracking, underscoring the potential of artificial muscle-based soft actuation for head-neck mobility assistance.
Read the full abstract ▾
All authors Katalin Schäffer, Ian Bales, Haohan Zhang, Margaret McGuinness
Robotics
Learning Actionable Manipulation Recovery via Counterfactual Failure Synthesis
Dayou Li, Jiuzhou Lei, Hao Wang et al. (+6 more)
While recent foundation models have significantly advanced robotic manipulation, these systems still struggle to autonomously recover from execution errors. Current failure-learning paradigms rely on either costly and unsafe real-world data collection or simulator-based perturbations, which introduce a severe sim-to-real gap. Furthermore, existing visual analyzers predominantly output coarse, binary diagnoses rather than the executable, trajectory-level corrections required for actual recovery. To bridge the gap between failure diagnosis and actionable recovery, we introduce Dream2Fix, a framework that synthesizes photorealistic, counterfactual failure rollouts directly from successful real-world demonstrations. By perturbing actions within a generative world model, Dream2Fix creates paired failure-correction data without relying on simulators. To ensure the generated data is physically viable for
robot learning, we implement a structured verification mechanism that strictly filters rollouts for task validity, visual coherence, and kinematic safety. This engine produces a high-fidelity
dataset of over 120k paired samples. Using this
dataset, we fine-tune a
vision-
language model to jointly predict failure types and precise recovery trajectories, mapping visual anomalies directly to corrective actions. Extensive real-world robotic experiments show our approach achieves state-of-the-art correction accuracy, improving from 19.7% to 81.3% over prior baselines, and successfully enables zero-shot closed-loop failure recovery in physical deployments.
Read the full abstract ▾
All authors Dayou Li, Jiuzhou Lei, Hao Wang, Lulin Liu, Yunhao Yang, Zihan Wang, Bangya Liu, Minghui Zheng, Zhiwen Fan
Robotics · Cross-listed from eess.SY
Verification and Forward Invariance of Control Barrier Functions for Differential-Algebraic Systems
Hongchao Zhang, Mohamad H. Kazma, Meiyi Ma et al. (+2 more)
Differential-algebraic equations (DAEs) arise in power networks, chemical processes, and multibody systems, where algebraic constraints encode physical conservation laws. The safety of such systems is critical, yet safe control is challenging because algebraic constraints restrict allowable state trajectories. Control barrier functions (CBFs) provide computationally efficient safety filters for ordinary differential equation (ODE) systems. However, existing CBF methods are not directly applicable to DAEs due to potential conflicts between the CBF condition and the constraint
manifold. This paper introduces DAE-aware CBFs that incorporate the differential-algebraic structure through projected vector fields. We derive conditions that ensure forward invariance of safe sets while preserving algebraic constraints and extend the framework to higher-index DAEs. A systematic verification framework is developed, establishing necessary and sufficient conditions for
geometric correctness and feasibility of DAE-aware CBFs. For
polynomial systems, sum-of-squares certificates are provided, while for nonpolynomial and neural network candidates, satisfiability modulo theories are used for falsification. The approach is validated on wind turbine and flexible-link manipulator systems.
Read the full abstract ▾
All authors Hongchao Zhang, Mohamad H. Kazma, Meiyi Ma, Taylor T. Johnson, Ahmad F. Taha
Robotics
Safety-aware Goal-oriented Semantic Sensing, Communication, and Control for Robotics
Wenchao Wu, Shutong Chen, Wenjie Liu et al. (+3 more)
Wirelessly-connected robotic systems empower
robots with real-time intelligence by leveraging remote computing resources for decision-making. However, the data exchange between
robots and edge servers often overwhelms communication links, introducing latency that degrades task performance. To tackle this, goal-oriented semantic communication (GSC) has been introduced for wirelessly-connected robotic systems to extract and transmit only goal-relevant semantic representations. While this improves task effectiveness, it generally overlooks practical safety requirements. Meanwhile, existing robotics research often treats safety primarily as a control-level problem, without systematically considering safety across sensing, communication, and control in a closed-loop manner. To bridge this gap, we investigate how to enable safety-aware goal-oriented semantic (SA-GS) sensing, communication, and control co-design in wirelessly-connected robotic systems, aiming to maximize the robotic task effectiveness subject to practical safety requirements. We first introduce {an} architecture {for} wirelessly-connected robotic systems and representative use cases. We then summarize general safety requirements and effectiveness metrics across the use cases. Next, we systematically analyze the unique safety and effectiveness challenges in sensing, communication, and control. Based on these, we further present potential SA-GS research directions. Finally, an Unmanned Aerial Vehicle (
UAV) target tracking case study validates that one of the presented SA-GS research directions, i.e., semantic-based C&C packet execution, could significantly improve safety rate and tracking success rate by more than 2 times and 4.5 times, respectively.
Read the full abstract ▾
All authors Wenchao Wu, Shutong Chen, Wenjie Liu, Zhibo Pang, Yansha Deng, Robert Schober
Robotics
Spatially Grounded Long-Horizon Task Planning in the Wild
Sehun Jung, HyunJee Song, Dong-Hee Kim et al. (+4 more)
Recent advances in
robot manipulation increasingly leverage
Vision-
Language Models (VLMs) for high-level
reasoning, such as decomposing task instructions into sequential action plans expressed in natural
language that guide downstream low-level motor execution. However, current
benchmarks do not assess whether these plans are spatially executable, particularly in specifying the exact spatial locations where the
robot should interact to execute the plan, limiting evaluation of real-world manipulation capability. To bridge this gap, we define a novel task of grounded planning and introduce GroundedPlanBench, a newly curated
benchmark for spatially grounded long-horizon action planning in the wild. GroundedPlanBench jointly evaluates hierarchical sub-action planning and spatial action grounding (where to act), enabling systematic assessment of whether generated sub-actions are spatially executable for
robot manipulation. We further introduce
Video-to-Spatially Grounded Planning (V2GP), an automated data generation framework that leverages real-world
robot video demonstrations to improve spatially grounded long-horizon planning. Our evaluations reveal that spatially grounded long-horizon planning remains a major bottleneck for current VLMs. Our results demonstrate that V2GP provides a promising approach for improving both action planning and spatial grounding performance, validated on our
benchmark as well as through real-world
robot manipulation experiments, advancing progress toward spatially actionable planning.
Read the full abstract ▾
All authors Sehun Jung, HyunJee Song, Dong-Hee Kim, Reuben Tan, Jianfeng Gao, Yong Jae Lee, Donghyun Kim
Robotics · Cross-listed from cs.LG
PhysMoDPO: Physically-Plausible Humanoid Motion with Preference Optimization
Yangsong Zhang, Anujith Muraleedharan, Rikhat Akizhanov et al. (+5 more)
Recent progress in text-conditioned human motion generation has been largely driven by
diffusion models trained on large-scale human motion data. Building on this progress, recent methods attempt to transfer such models for character animation and real
robot control by applying a Whole-Body Controller (WBC) that converts
diffusion-generated motions into executable trajectories. While WBC trajectories become compliant with physics, they may expose substantial deviations from original motion. To address this issue, we here propose PhysMoDPO, a Direct Preference Optimization framework. Unlike prior work that relies on hand-crafted physics-aware heuristics such as foot-sliding penalties, we integrate WBC into our training pipeline and optimize
diffusion model such that the output of WBC becomes compliant both with physics and original text instructions. To train PhysMoDPO we deploy physics-based and task-specific rewards and use them to assign preference to synthesized trajectories. Our extensive experiments on text-to-motion and spatial control tasks demonstrate consistent improvements of PhysMoDPO in both physical realism and task-related metrics on simulated
robots. Moreover, we demonstrate that PhysMoDPO results in significant improvements when applied to zero-shot motion transfer in simulation and for real-world deployment on a G1
humanoid robot.
Read the full abstract ▾
All authors Yangsong Zhang, Anujith Muraleedharan, Rikhat Akizhanov, Abdul Ahad Butt, Gül Varol, Pascal Fua, Fabio Pizzati, Ivan Laptev
Robotics
Zihao Xin, Wentong Li, Yixuan Jiang et al. (+4 more)
Vision-and-
Language Navigation (VLN) requires
agents to follow long-horizon instructions and navigate complex 3D environments. However, existing approaches face two major challenges: constructing an effective long-term
memory bank and overcoming the compounding errors problem. To address these issues, we propose DecoVLN, an effective framework designed for robust streaming perception and closed-loop control in long-horizon
navigation. First, we formulate long-term
memory construction as an optimization problem and introduce adaptive refinement mechanism that selects frames from a historical candidate pool by iteratively optimizing a unified scoring function. This function jointly balances three key criteria: semantic relevance to the instruction, visual diversity from the selected
memory, and temporal coverage of the historical trajectory. Second, to alleviate compounding errors, we introduce a state-action pair-level corrective finetuning strategy. By leveraging geodesic distance between states to precisely quantify deviation from the expert trajectory, the
agent collects high-quality state-action pairs in the trusted region while filtering out the polluted data with low relevance. This improves both the efficiency and stability of error correction. Extensive experiments demonstrate the effectiveness of DecoVLN, and we have deployed it in real-world environments.
Read the full abstract ▾
All authors Zihao Xin, Wentong Li, Yixuan Jiang, Bin Wang, Runmin Cong, Jie Qin, Shengjun Huang
Robotics
Panoramic Multimodal Semantic Occupancy Prediction for Quadruped Robots
Guoqiang Zhao, Zhe Yang, Sheng Wu et al. (+5 more)
Panoramic imagery provides holistic 360{\deg} visual coverage for environmental perception in quadruped
robots. However, existing occupancy prediction methods are primarily designed for wheeled autonomous driving and rely heavily on RGB cues, which limits their robustness in complex, dynamically changing environments. To bridge this gap, we introduce PanoMMOcc, the first real-world panoramic
multimodal occupancy
dataset for quadruped
robots, comprising four sensing modalities collected across diverse scenes. We further propose VoxelHound, a panoramic
multimodal occupancy perception framework tailored to legged locomotion and spherical imaging. VoxelHound incorporates a Vertical Jitter Compensation (VJC) module to mitigate severe viewpoint perturbations caused by body pitch and roll during locomotion, enabling more consistent spatial
reasoning, and a
Multimodal Information
Prompt Fusion (MIPF) module to effectively integrate panoramic visual cues with auxiliary modalities for enhanced volumetric occupancy prediction. We also establish a comprehensive
benchmark on PanoMMOcc and provide detailed
dataset analyses to enable systematic evaluation in challenging
embodied perception scenarios. Extensive experiments demonstrate that VoxelHound achieves state-of-the-art performance on PanoMMOcc, with a +4.16 gain in mIoU. The
dataset and code will be publicly released to facilitate future research on panoramic
multimodal 3D perception for
embodied robotic systems at https://github.com/SXDR/PanoMMOcc.
Read the full abstract ▾
All authors Guoqiang Zhao, Zhe Yang, Sheng Wu, Fei Teng, Mengfei Duan, Yuanfan Zheng, Kai Luo, Kailun Yang
Robotics
A Feasibility-Enhanced Control Barrier Function Method for Multi-UAV Collision Avoidance
Qishen Zhong, Junlong Wu, Jian Yang et al. (+4 more)
This paper presents a feasibility-enhanced control barrier function (FECBF) framework for multi-
UAV collision avoidance. In dense multi-
UAV scenarios, the feasibility of the CBF quadratic program (CBF-QP) can be compromised due to internal incompatibility among multiple CBF constraints. To address this issue, we analyze the internal compatibility of CBF constraints and derive a sufficient condition for internal compatibility. Based on this condition, a sign-consistency constraint is introduced to mitigate internal incompatibility. The proposed constraint is incorporated into a decentralized CBF-QP formulation using worst-case estimates and slack variables. Simulation results demonstrate that the proposed method significantly reduces infeasibility and improves collision avoidance performance compared with existing baselines in dense scenarios. Additional simulations under varying time delays demonstrate the robustness of the proposed method. Real-world experiments validate the practical applicability of the proposed method.
Read the full abstract ▾
All authors Qishen Zhong, Junlong Wu, Jian Yang, Guanwei Xiao, Junqi Wu, Zimeng Jiang, Pingan Fang
Robotics
Evaluating VLMs' Spatial Reasoning Over Robot Motion: A Step Towards Robot Planning with Motion Preferences
Wenxi Wu, Jingjing Zhang and Martim Brandão
Understanding user instructions and object spatial relations in surrounding environments is crucial for intelligent
robot systems to assist humans in various tasks. The natural
language and spatial
reasoning capabilities of
Vision-
Language Models (VLMs) have the potential to enhance the generalization of
robot planners on new tasks, objects, and motion specifications. While foundation models have been applied to task planning, it is still unclear the degree to which they have the capability of spatial
reasoning required to enforce user preferences or constraints on motion, such as desired distances from objects, topological properties, or motion style preferences. In this paper, we evaluate the capability of four state-of-the-art VLMs at spatial
reasoning over
robot motion, using four different querying methods. Our results show that, with the highest-performing querying method, Qwen2.5-VL achieves 71.4% accuracy zero-shot and 75% on a smaller model after fine-tuning, and GPT-4o leads to lower performance. We evaluate two types of motion preferences (object-proximity and path-style), and we also analyze the trade-off between accuracy and computation cost in number of
tokens. This work shows some promise in the potential of VLM integration with
robot motion planning pipelines.
Read the full abstract ▾
Robotics
SldprtNet: A Large-Scale Multimodal Dataset for CAD Generation in Language-Driven 3D Design
Ruogu Li, Sikai Li, Yao Mu et al. (+1 more)
We introduce SldprtNet, a large-scale
dataset comprising over 242,000 industrial parts, designed for semantic-driven CAD modeling,
geometric deep learning, and the training and fine-tuning of
multimodal models for 3D design. The
dataset provides 3D models in both .step and .sldprt formats to support diverse training and testing. To enable parametric modeling and facilitate
dataset scalability, we developed supporting tools, an encoder and a decoder, which support 13 types of CAD commands and enable lossless transformation between 3D models and a structured text representation. Additionally, each sample is paired with a composite image created by merging seven rendered views from different viewpoints of the 3D model, effectively reducing input
token length and accelerating inference. By combining this image with the parameterized text output from the encoder, we employ the lightweight
multimodal language model Qwen2.5-VL-7B to generate a natural
language description of each part's appearance and functionality. To ensure accuracy, we manually verified and aligned the generated descriptions, rendered images, and 3D models. These descriptions, along with the parameterized modeling scripts, rendered images, and 3D model files, are fully aligned to construct SldprtNet. To assess its effectiveness, we fine-tuned baseline models on a
dataset subset, comparing image-plus-text inputs with text-only inputs. Results confirm the necessity and value of
multimodal datasets for CAD generation. It features carefully selected real-world industrial parts, supporting tools for scalable
dataset expansion, diverse modalities, and ensured diversity in model complexity and
geometric features, making it a comprehensive
multimodal dataset built for semantic-driven CAD modeling and cross-modal learning.
Read the full abstract ▾
All authors Ruogu Li, Sikai Li, Yao Mu, Mingyu Ding
Robotics · Cross-listed from cs.CV
InterEdit: Navigating Text-Guided 3D Dyadic Human Motion Editing
Yebin Yang, Di Wen, Lei Qi et al. (+10 more)
Text-guided 3D motion editing has seen success in single-person scenarios, but its extension to multi-person settings is less explored due to limited paired data and the complexity of inter-person interactions. We introduce the task of multi-person 3D motion editing, where a target motion is generated from a source and a text instruction. To support this, we propose InterEdit3D, a new
dataset with manual two-person motion change annotations, and a Text-guided Multi-human Motion Editing (TMME)
benchmark. We present InterEdit, a synchronized classifier-free conditional
diffusion model for TMME. It introduces Semantic-Aware Plan
Token Alignment with learnable
tokens to capture high-level interaction cues and an Interaction-Aware Frequency
Token Alignment strategy using DCT and energy pooling to model periodic motion dynamics. Experiments show that InterEdit improves text-to-motion consistency and edit fidelity, achieving state-of-the-art TMME performance. The
dataset and code will be released at https://github.com/YNG916/InterEdit.
Read the full abstract ▾
All authors Yebin Yang, Di Wen, Lei Qi, Weitong Kong, Junwei Zheng, Ruiping Liu, Yufan Chen, Chengzhi Wu, Kailun Yang, Yuqian Fu, Danda Pani Paudel, Luc Van Gool, Kunyu Peng
Robotics · Cross-listed from cs.CV
Yanpeng Zhao, Wentao Ding, Hongtao Li et al. (+2 more)
A recent trend in
vision-
language models (VLMs) has been to enhance their spatial cognition for
embodied domains. Despite progress, existing evaluations have been limited both in paradigm and in coverage, hindering rapid, iterative model development. To address these limitations, we propose ESPIRE, a diagnostic
benchmark for
embodied spatial
reasoning. ESPIRE offers a simulated world that physically grounds VLMs and evaluates them on spatial-
reasoning-centric robotic tasks, thus narrowing the gap between evaluation and real-world deployment. To adapt VLMs to robotic tasks, we decompose each task into localization and execution, and frame both as generative problems, in stark contrast to predominant discriminative evaluations (e.g., via visual-question answering) that rely on distractors and discard execution. This decomposition further enables a fine-grained analysis beyond passive spatial
reasoning toward
reasoning to act. We systematically design ESPIRE both at the instruction level and at the environment level, ensuring broad coverage of spatial
reasoning scenarios. We use ESPIRE to diagnose a range of frontier VLMs and provide in-depth analysis of their spatial
reasoning behaviors.
Read the full abstract ▾
All authors Yanpeng Zhao, Wentao Ding, Hongtao Li, Baoxiong Jia, Zilong Zheng
Robotics
From Passive Monitoring to Active Defence: Resilient Control of Manipulators Under Cyberattacks
Gabriele Gualandi and Alessandro V. Papadopoulos
Cyber-physical robotic systems are vulnerable to false data injection attacks (FDIAs), in which an adversary corrupts sensor signals while evading residual-based passive anomaly detectors such as the chi-squared test. Such stealthy attacks can induce substantial end-effector deviations without triggering alarms. This paper studies the resilience of redundant manipulators to stealthy FDIAs and advances the architecture from passive monitoring to active defence. We formulate a closed-loop model comprising a feedback-linearized manipulator, a steady-state Kalman filter, and a chi-squared-based anomaly detector. Building on this passive monitoring layer, we propose an active control-level defence that attenuates the control input through a monotone function of an anomaly score generated by a novel actuation-projected, measurement-free state predictor. The proposed design provides probabilistic guarantees on nominal actuation loss and preserves closed-loop stability. From the attacker perspective, we derive a convex QCQP for computing one-step optimal stealthy attacks. Simulations on a 6-DOF planar manipulator show that the proposed defence significantly reduces attack-induced end-effector deviation while preserving nominal task performance in the absence of attacks.
Read the full abstract ▾
Robotics
Route Fragmentation Based on Resource-centric Prioritisation for Efficient Multi-Robot Path Planning in Agricultural Environments
James R. Heselden and Gautham P. Das
Agricultural environments present high proportions of spatially dense
navigation bottlenecks for long-term
navigation and operational planning of agricultural mobile
robots. The existing
agent-centric multi-
robot path planning (MRPP) approaches resolve conflicts from the perspective of
agents, rather than from the resources under contention. Further, the density of such contentions limits the capabilities of spatial interleaving, a concept that many planners rely on to achieve high throughput. In this work, two variants of the priority-based Fragment Planner (FP) are presented as resource-centric MRPP
algorithms that leverage route fragmentation to enable partial route progression and limit the impact of binary-based waiting. These approaches are evaluated in lifelong simulation over a 3.6km topological map representing a commercial polytunnel environment. Their performances are contrasted against 5 baseline
algorithms with varying robotic fleet sizes. The Fragment Planners achieved significant gains in throughput compared with Prioritised Planning (PP) and Priority-Based Search (PBS)
algorithms. They further demonstrated a task throughput of 95% of the optimal task throughput over the same time period. This work shows that, for long-term deployment of agricultural
robots in corridor-dominant agricultural environments, resource-centric MRPP approaches are a necessity for high-efficacy operational planning.
Read the full abstract ▾
Robotics
Language-Grounded Decoupled Action Representation for Robotic Manipulation
Wuding Weng, Tongshu Wu, Liucheng Chen et al. (+5 more)
The heterogeneity between high-level
vision-
language understanding and low-level action control remains a fundamental challenge in robotic manipulation. Although recent methods have advanced task-specific action
alignment, they often struggle to generate robust and accurate actions for novel or semantically related tasks. To address this, we propose the
Language-Grounded Decoupled Action Representation (LaDA) framework, which leverages natural
language as a semantic bridge to connect perception and control. LaDA introduces a fine-grained intermediate layer of three interpretable action primitives–translation, rotation, and gripper control–providing explicit semantic structure for low-level actions. It further employs a semantic-guided soft-label contrastive learning objective to align similar action primitives across tasks, enhancing generalization and motion consistency. An adaptive weighting strategy, inspired by curriculum learning, dynamically balances contrastive and imitation objectives for stable and effective training. Extensive experiments on simulated
benchmarks (LIBERO and MimicGen) and real-world demonstrations validate that LaDA achieves strong performance and generalizes effectively to unseen or related tasks.
Read the full abstract ▾
All authors Wuding Weng, Tongshu Wu, Liucheng Chen, Siyu Xie, Zheng Wang, Xing Xu, Jingkuan Song, Heng Tao Shen
Robotics
Efficient Real-World Autonomous Racing via Attenuated Residual Policy Optimization
Raphael Trumpp, Denis Hoornaert, Mirco Theile et al. (+1 more)
Residual policy learning (RPL), in which a learned policy refines a static base policy using deep reinforcement learning (DRL), has shown strong performance across various robotic applications. Its effectiveness is particularly evident in autonomous racing, a domain that serves as a challenging
benchmark for real-world DRL. However, deploying RPL-based controllers introduces system complexity and increases inference latency. We address this by introducing an extension of RPL named attenuated residual policy optimization ($\alpha$-RPO). Unlike standard RPL, $\alpha$-RPO yields a standalone neural policy by progressively attenuating the base policy, which initially serves to bootstrap learning. Furthermore, this mechanism enables a form of privileged learning, where the base policy is permitted to use sensor modalities not required for final deployment. We design $\alpha$-RPO to integrate seamlessly with PPO, ensuring that the attenuated influence of the base controller is dynamically compensated during policy optimization. We evaluate $\alpha$-RPO by building a framework for 1:10-scaled autonomous racing around it. In both simulation and zero-shot real-world transfer to Roboracer cars, $\alpha$-RPO not only reduces system complexity but also improves driving performance compared to baselines - demonstrating its practicality for robotic deployment. Our code is available at: https://github.com/raphajaner/arpo_racing.
Read the full abstract ▾
All authors Raphael Trumpp, Denis Hoornaert, Mirco Theile, Marco Caccamo
Robotics
ReMem-VLA: Empowering Vision-Language-Action Model with Memory via Dual-Level Recurrent Queries
Hang Li, Fengyi Shen, Dong Chen et al. (+6 more)
Vision-
language-action (VLA) models for closed-loop
robot control are typically cast under the Markov assumption, making them prone to errors on tasks requiring historical context. To incorporate
memory, existing VLAs either retrieve from a
memory bank, which can be misled by distractors, or extend the frame window, whose fixed horizon still limits long-term retention. In this paper, we introduce ReMem-VLA, a Recurrent
Memory VLA model equipped with two sets of learnable queries: frame-level recurrent
memory queries for propagating information across consecutive frames to support short-term
memory, and chunk-level recurrent
memory queries for carrying context across temporal chunks for long-term
memory. These queries are trained end-to-end to aggregate and maintain relevant context over time, implicitly guiding the model's decisions without additional training or inference cost. Furthermore, to enhance visual
memory, we introduce Past Observation Prediction as an auxiliary training objective. Through extensive
memory-centric simulation and real-world
robot experiments, we demonstrate that ReMem-VLA exhibits strong
memory capabilities across multiple dimensions, including spatial, sequential, episodic, temporal, and visual
memory. ReMem-VLA significantly outperforms
memory-free VLA baselines $\pi$0.5 and OpenVLA-OFT and surpasses MemoryVLA on
memory-dependent tasks by a large margin.
Read the full abstract ▾
All authors Hang Li, Fengyi Shen, Dong Chen, Liudi Yang, Xudong Wang, Jinkui Shi, Zhenshan Bing, Ziyuan Liu, Alois Knoll
Robotics
Coordinated Manipulation of Hybrid Deformable-Rigid Objects in Constrained Environments
Anees Peringal, Anup Teejo Mathew, Panagiotis liatsis et al. (+1 more)
Coordinated robotic manipulation of deformable linear objects (DLOs), such as ropes and cables, has been widely studied; however, handling hybrid assemblies composed of both deformable and rigid elements in constrained environments remains challenging. This work presents a quasi-static optimization-based manipulation planner that employs a strain-based Cosserat rod model, extending rigid-body formulations to hybrid deformable linear objects (hDLO). The proposed planner exploits the compliance of deformable links to maneuver through constraints while achieving task-space objectives for the object that are unreachable with rigid tools. By leveraging a differentiable model with analytically derived gradients, the method achieves up to a 33x speedup over finite-difference baselines for inverse kinetostatic(IKS) problems. Furthermore, the subsequent trajectory optimization problem, warm-started using the IKS solution, is only practically realizable via analytical derivatives. The proposed
algorithm is validated in simulation on various hDLO systems and experimentally on a three-link hDLO manipulated in a constrained environment using a dual-arm robotic system. Experimental results confirm the planner's accuracy, yielding an average deformation error of approximately 3 cm (5% of the deformable link length) between the desired and measured marker positions. Finally, the proposed optimal planner is compared against a sampling-based feasibility planner adapted to the strain-based formulation. The results demonstrate the effectiveness and applicability of the proposed approach for robotic manipulation of hybrid assemblies in constrained environments.
Read the full abstract ▾
All authors Anees Peringal, Anup Teejo Mathew, Panagiotis liatsis, Federico Renda
Robotics
RoboStream: Weaving Spatio-Temporal Reasoning with Memory in Vision-Language Models for Robotics
Yuzhi Huang, Jie Wu, Weijue Bu et al. (+9 more)
Enabling reliable long-horizon robotic manipulation is a crucial step toward open-world
embodied intelligence. However, VLM-based planners treat each step as an isolated observation-to-action mapping, forcing them to reinfer scene geometry from raw pixels at every decision point while remaining unaware of how prior actions have reshaped the environment. Despite strong short-horizon performance, these systems lack the spatio-temporal
reasoning required for persistent
geometric anchoring and
memory of action-triggered state transitions. Without persistent state tracking, perceptual errors accumulate across the execution horizon, temporarily occluded objects are catastrophically forgotten, and these compounding failures lead to precondition violations that cascade through subsequent steps. In contrast, humans maintain a persistent mental model that continuously tracks spatial relations and action consequences across interactions rather than reconstructing them at each instant. Inspired by this human capacity for causal spatio-temporal
reasoning with persistent
memory, we propose RoboStream, a training-free framework that achieves
geometric anchoring through Spatio-Temporal Fusion
Tokens (STF-
Tokens), which bind visual evidence to 3D
geometric attributes for persistent object grounding, and maintains causal continuity via a Causal Spatio-Temporal Graph (CSTG) that records action-triggered state transitions across steps. This design enables the planner to trace causal chains and preserve object permanence under occlusion without additional training or fine-tuning. RoboStream achieves 90.5% on long-horizon RLBench and 44.4% on challenging real-world block-building tasks, where both SoFar and VoxPoser score 11.1%, demonstrating that spatio-temporal
reasoning and causal
memory are critical missing components for reliable long-horizon manipulation.
Read the full abstract ▾
All authors Yuzhi Huang, Jie Wu, Weijue Bu, Ziyi Xiong, Gaoyang Jiang, Ye Li, Kangye Ji, Shuzhao Xie, Yue Huang, Chenglei Wu, Jingyan Jiang, Zhi Wang
Robotics
MotionAnymesh: Physics-Grounded Articulation for Simulation-Ready Digital Twins
WenBo Xu, Liu Liu, Li Zhang et al. (+2 more)
Converting static 3D meshes into interactable articulated assets is crucial for
embodied AI and robotic simulation. However, existing zero-shot pipelines struggle with complex assets due to a critical lack of physical grounding. Specifically, ungrounded
Vision-
Language Models (VLMs) frequently suffer from kinematic
hallucinations, while unconstrained joint estimation inevitably leads to catastrophic mesh inter-penetration during physical simulation. To bridge this gap, we propose MotionAnymesh, an automated zero-shot framework that seamlessly transforms unstructured static meshes into simulation-ready digital twins. Our method features a kinematic-aware part
segmentation module that grounds VLM
reasoning with explicit SP4D physical priors, effectively eradicating kinematic
hallucinations. Furthermore, we introduce a geometry-physics joint estimation pipeline that combines robust type-aware initialization with physics-constrained trajectory optimization to rigorously guarantee collision-free articulation. Extensive experiments demonstrate that MotionAnymesh significantly outperforms state-of-the-art baselines in both
geometric precision and dynamic physical executability, providing highly reliable assets for downstream applications.
Read the full abstract ▾
All authors WenBo Xu, Liu Liu, Li Zhang, Dan Guo, RuoNan Liu
Robotics
GoalSwarm: Multi-UAV Semantic Coordination for Open-Vocabulary Object Navigation
MoniJesu Wonders James, Amir Atef Habel, Aleksey Fedoseev et al. (+1 more)
Cooperative visual semantic
navigation is a foundational capability for aerial
robot teams operating in unknown environments. However, achieving robust open-vocabulary object-goal
navigation remains challenging due to the computational constraints of deploying heavy perception models onboard and the complexity of decentralized multi-
agent coordination. We present GoalSwarm, a fully decentralized multi-
UAV framework for zero-shot semantic object-goal
navigation. Each
UAV collaboratively constructs a shared, lightweight 2D top-down semantic occupancy map by projecting depth observations from aerial vantage points, eliminating the computational burden of full 3D representations while preserving essential
geometric and semantic structure. The core contributions of GoalSwarm are threefold: (1) integration of zero-shot foundation model – SAM3 for open vocabulary detection and pixel-level
segmentation, enabling open-vocabulary target identification without task-specific training; (2) a
Bayesian Value Map that fuses multi-viewpoint detection confidences into a per-pixel goal-relevance distribution, enabling informed frontier scoring via Upper Confidence Bound (UCB) exploration; and (3) a decentralized coordination strategy combining semantic frontier extraction, cost-utility bidding with geodesic path costs, and spatial separation penalties to minimize redundant exploration across the swarm.
Read the full abstract ▾
All authors MoniJesu Wonders James, Amir Atef Habel, Aleksey Fedoseev, Dzmitry Tsetserokou
Robotics
Consistent and Efficient MSCKF-based LiDAR-Inertial Odometry with Inferred Cluster-to-Plane Constraints for UAVs
Jinwen Zhu, Xudong Zhao, Fangcheng Zhu et al. (+4 more)
Robust and accurate
navigation is critical for Unmanned Aerial Vehicles (
UAVs) especially for those with stringent Size, Weight, and Power (SWaP) constraints. However, most state-of-the-art (SOTA)
LiDAR-Inertial Odometry (LIO) systems still suffer from estimation inconsistency and computational bottlenecks when deployed on such platforms. To address these issues, this paper proposes a consistent and efficient tightly-coupled LIO framework tailored for
UAVs. Within the efficient Multi-State Constraint Kalman Filter (MSCKF) framework, we build coplanar constraints inferred from planar features observed across a sliding window. By applying null-space projection to sliding-window coplanar constraints, we eliminate the direct dependency on feature parameters in the state vector, thereby mitigating overconfidence and improving consistency. More importantly, to further boost the efficiency, we introduce a parallel voxel-based data association and a novel compact cluster-to-plane measurement model. This compact measurement model losslessly reduces observation dimensionality and significantly accelerating the update process. Extensive evaluations demonstrate that our method outperforms most state-of-the-art (SOTA) approaches by providing a superior balance of consistency and efficiency. It exhibits improved robustness in degenerate scenarios, achieves the lowest
memory usage via its map-free nature, and runs in real-time on resource-constrained embedded platforms (e.g., NVIDIA Jetson TX2).
Read the full abstract ▾
All authors Jinwen Zhu, Xudong Zhao, Fangcheng Zhu, Jun Hu, Shi Jin, Yinian Mao, Guoquan Huang
Robotics
Beyond Imitation: Reinforcement Learning Fine-Tuning for Adaptive Diffusion Navigation Policies
Junhe Sheng, Ruofei Bai, Kuan Xu et al. (+5 more)
Diffusion-based
robot navigation policies trained on large-scale imitation learning
datasets, can generate multi-modal trajectories directly from the
robot's visual observations, bypassing the traditional localization-mapping-planning pipeline and achieving strong zero-shot generalization. However, their performance remains constrained by the coverage of offline
datasets, and when deployed in unseen settings, distribution shift often leads to accumulated trajectory errors and safety-critical failures. Adapting
diffusion policies with reinforcement learning is challenging because their iterative denoising structure hinders effective gradient backpropagation, while also making the training of an additional value network computationally expensive and less stable. To address these issues, we propose a reinforcement learning fine-tuning framework tailored for
diffusion-based
navigation. The method leverages the inherent multi-trajectory sampling mechanism of
diffusion models and adopts Group Relative Policy Optimization (GRPO), which estimates relative advantages across sampled trajectories without requiring a separate value network. To preserve pretrained representations while enabling adaptation, we freeze the visual encoder and selectively update the higher decoder layers and action head, enhancing safety-aware behaviors through online environmental feedback. On the PointGoal task in Isaac Sim, our approach improves the Success Rate from 52.0% to 58.7% and SPL from 0.49 to 0.54 on unseen scenes, while reducing collision frequency. Additional experiments show that the fine-tuned policy transfers zero-shot to a real quadruped platform and maintains stable performance in geometrically out-of-distribution environments, suggesting improved adaptability and safe generalization to new domains.
Read the full abstract ▾
All authors Junhe Sheng, Ruofei Bai, Kuan Xu, Ruimeng Liu, Jie Chen, Shenghai Yuan, Wei-Yun Yau, Lihua Xie
Robotics · Cross-listed from eess.SP
AoI-FusionNet: Age-Aware Tightly Coupled Fusion of UWB-IMU under Sparse Ranging Conditions
Tehmina Bibi, Anselm Köhler, Jan-Thomas Fischer et al. (+1 more)
Accurate motion tracking of snow particles in avalanche events requires robust localization in global
navigation satellite system (GNSS)-denied outdoor environments. This paper introduces AoI-FusionNet, a tightly coupled deep learning-based fusion framework that directly combines raw ultra-wideband (UWB) time-of-flight (ToF) measurements with inertial measurement unit (IMU) data for 3D trajectory estimation. Unlike loose-coupled pipelines based on intermediate trilateration, the proposed approach operates directly on heterogeneous sensor inputs, enabling localization even under insufficient ranging availability. The framework integrates an Age-of-Information (AoI)-aware decay module to reduce the influence of stale UWB ranging measurements and a learned
attention gating mechanism that adaptively balances the contribution of UWB and IMU modalities based on measurement availability and temporal freshness. To evaluate robustness under limited data and measurement variability, we apply a
diffusion-based residual augmentation strategy during training, producing an augmented variant termed AoI-FusionNet-DGAN. We assess the performance of the proposed model using offline post-processing of real-world measurement data collected in an alpine environment and
benchmark it against UWB multilateration and loose-coupled fusion baselines. The results demonstrate that AoI-FusionNet substantially reduces mean and tail localization errors under intermittent and degraded sensing conditions.
Read the full abstract ▾
All authors Tehmina Bibi, Anselm Köhler, Jan-Thomas Fischer, Falko Dressler
Robotics
Prepare to Turn: Learning Anticipatory Maneuvering for Fast Sequential Quadrupedal Navigation
Zunzhi You, Yunke Wang, Haolan Guo et al. (+1 more)
Fast quadrupedal
navigation requires preparing for the next turn while approaching the current goal. Policies trained to reach and remain at a single target can instead produce stop-and-go motion when applied to sequential goals, compromising its agility potential. We introduce PrepTurn, a reinforcement learning framework for anticipatory maneuvering in sequential local
navigation. PrepTurn maps the current and next pose goals directly to joint actions, using the upcoming goal to inform how the
robot should approaches the current one. Its persistent-credit reward encourages timely completion of the whole sequence by continuing to reward reached goals while moving on and preparing subsequent maneuvers, and an automatic curriculum that gradually increases goal difficulty is critical for learning stable turning behaviors. Together, these components enable the policy to coordinate approach speed and reorientation across goal transitions without prescribed trajectories or arrival times. Simulation and onboard Unitree Go2 experiments demonstrate fast, efficient sequential traversal with high completion and reduced stop-and-go motion. We further show that when the goal-reaching tolerances are more relaxed, the policy learns to exploit the freedom allowed by the task with emergent behaviors, such as turning to the next goal in advance and cutting corners within the permitted regions behaviors, to traverse goal sequences more efficiently.
Read the full abstract ▾
All authors Zunzhi You, Yunke Wang, Haolan Guo, Chang Xu
Robotics
Reinforcement Learning for Elliptical Cylinder Motion Control Tasks
Pawel Marczewski, Paulina Superczynska, Jakub Bernat et al. (+1 more)
The control of devices with limited input always bring
attention to solve by research due to its difficulty and non-trival solution. For instance, the inverted pendulum is benchmarking problem in control theory and machine learning. In this work, we are focused on the elliptical cylinder and its motion under limited torque. The inspiration of the problem is from untethered
magnetic devices, which due to distance have to operate with limited input torque. In this work, the main goal is to define the control problem of
elliptic cylinder with limited input torque and solve it by Reinforcement Learning. As a classical baseline, we evaluate a two-stage controller composed of an energy-shaping swing-up law and a local Linear Quadratic Regulator (LQR) stabilizer around the target
equilibrium. The swing-up controller increases the system's mechanical energy to drive the state toward a neighborhood of the desired
equilibrium, a linearization of the nonlinear model yields an LQR that regulates the angle and angular-rate states to the target orientation with bounded input. This swing-up + LQR policy is a strong, interpretable reference for underactuated system and serves a point of comparison to the learned policy under identical limits and parameters. The solution shows that the learning is possible however, the different cases like stabilization in upward position or rotating of half turn are very difficult for increasing mass or ellipses with a strongly unequal perimeter ratio.
Read the full abstract ▾
All authors Pawel Marczewski, Paulina Superczynska, Jakub Bernat, Szymon Szczesny
Robotics
FLUX: Accelerating Cross-Embodiment Generative Navigation Policies via Rectified Flow and Static-to-Dynamic Learning
Zeying Gong, Yangyi Zhong, Yiyi Ding et al. (+6 more)
Autonomous
navigation requires a broad spectrum of skills, from static goal-reaching to dynamic social traversal, yet evaluation remains fragmented across disparate protocols. We introduce DynBench, a dynamic
navigation benchmark featuring physically valid crowd simulation. Combined with existing static protocols, it supports comprehensive evaluation across six fundamental
navigation tasks. Within this framework, we propose FLUX, the first flow-based unified
navigation policy. By linearizing probability flow, FLUX replaces iterative denoising with straight-line trajectories, improving per-step inference efficiency by 47% over prior flow-based methods and 29% over
diffusion-based ones. Following a static-to-dynamic curriculum, FLUX initially establishes
geometric priors and is subsequently refined through reinforcement learning in dynamic social environments. This regime not only strengthens socially-aware
navigation but also enhances static task robustness by capturing recovery behaviors through
stochastic action distributions. FLUX achieves state-of-the-art performance across all tasks and demonstrates zero-shot sim-to-real transfer on wheeled, quadrupedal, and
humanoid platforms without any fine-tuning.
Read the full abstract ▾
All authors Zeying Gong, Yangyi Zhong, Yiyi Ding, Tianshuai Hu, Guoyang Zhao, Lingdong Kong, Rong Li, Jiadi You, Junwei Liang
Robotics
Motion-Specific Battery Health Assessment for Quadrotors Using High-Fidelity Battery Models
Joonhee Kim, Sanghyun Park, Donghyeong Kim et al. (+2 more)
Quadrotor endurance is ultimately limited by battery behavior, yet most energy aware planning treats the battery as a simple energy reservoir and overlooks how flight motions induce dynamic current loads that accelerate battery degradation. This work presents an end to end framework for motion aware battery health assessment in quadrotors. We first design a wide range current sensing module to capture motion specific current profiles during real flights, preserving transient features. In parallel, a high fidelity battery model is calibrated using reference performance tests and a metaheuristic based on a degradation coupled electrochemical model.By simulating measured flight loads in the calibrated model, we systematically resolve how different flight motions translate into degradation modes loss of lithium inventory and loss of active material as well as internal side reactions. The results demonstrate that even when two flight profiles consume the same average energy, their transient load structures can drive different degradation pathways, emphasizing the need for motion-aware battery management that balances efficiency with battery degradation.
Read the full abstract ▾
All authors Joonhee Kim, Sanghyun Park, Donghyeong Kim, Eunseon Choi, Soohee Han
Robotics · Cross-listed from cs.CV
PVI: Plug-in Visual Injection for Vision-Language-Action Models
Zezhou Zhang, Songxin Zhang, Xiao Xiong et al. (+8 more)
VLA architectures that pair a pretrained VLM with a flow-matching action expert have emerged as a strong paradigm for
language-conditioned manipulation. Yet the VLM, optimized for semantic abstraction and typically conditioned on static visual observations, tends to attenuate fine-grained
geometric cues and often lacks explicit temporal evidence for the action expert. Prior work mitigates this by injecting auxiliary visual features, but existing approaches either focus on static spatial representations or require substantial architectural modifications to accommodate temporal inputs, leaving temporal information underexplored. We propose Plug-in Visual Injection (PVI), a lightweight, encoder-agnostic module that attaches to a pretrained action expert and injects auxiliary visual representations via zero-initialized residual pathways, preserving pretrained behavior with only single-stage fine-tuning. Using PVI, we obtain consistent gains over the base policy and a range of competitive alternative injection strategies, and our controlled study shows that temporal
video features (V-JEPA2) outperform strong static image features (DINOv2), with the largest gains on multi-phase tasks requiring state tracking and coordination. Real-
robot experiments on long-horizon bimanual cloth folding further demonstrate the practicality of PVI beyond simulation.
Read the full abstract ▾
All authors Zezhou Zhang, Songxin Zhang, Xiao Xiong, Junjie Zhang, Zejian Xie, Jingyi Xi, Zunyao Mao, Zan Mao, Zhixin Mai, Zhuoyang Song, Jiaxing Zhang