Robotics
RocketSmith: Agentic Additive Manufacturing of High-Powered Rockets
Peter Pak, Jesse Barkley, Rumi Loghmani et al. (+3 more)
RocketSmith is an agentic system which intelligently automates the DFAM process for the development of high powered rockets suitable for launch. The system utilizes a large
language model to orchestrate the execution of software tools to validate design characteristics such as flight stability and generate the parametric design components for the rocket assembly. A collection of subagents and skills enable optimization workflows of flight parameters via iteration in both zero-shot and human-in-the-loop workflows. With this system, four distinct high power rockets with various motor and assembly configurations were developed utilizing the unique design capabilities of additive manufacturing. These assembly components were fabricated using various FDM printers, manually evaluated for flight readiness, and flight tested at a launch event. From these tests, all rockets achieved a stable launch and two of the four rockets were successfully recovered in reflyable condition. The altimeter data validated that the rockets achieved an altitude 80% of the expected apogee predicted by the agentic system, establishing consistency between simulation and experimentation.
Read the full abstract ▾
All authors Peter Pak, Jesse Barkley, Rumi Loghmani, Derek Baich, Ananya Pamal, Amir Barati Farimani
Robotics · Cross-listed from cs.CV
Kailing Li, Tianwen Qian, Lijin Yang et al. (+4 more)
Vision-
Language Navigation (VLN) enables
embodied agents to reach target locations in unseen environments by following
language instructions. Despite recent progress with
vision-
language models (VLMs), a critical semantic-
geometric gap remains: while VLMs excel at
language and 2D visual understanding, they struggle with 3D spatial
reasoning and fail to capture the causal dynamics between actions and spatial transitions, resulting in unreliable
navigation, particularly in zero-shot settings. To bridge this gap, we propose a Hierarchical Semantic-
Geometric Map (HSGM) that transforms 3D
geometric information into a structured representation compatible with VLMs, effectively linking them to the physical world. Specifically, HSGM is represented as a multi-channel top-down map organized into three levels: (1)
geometric level that records navigable regions and obstacles, (2) semantic level that represents objects and their relations, and (3) decision level that supports high-level task
reasoning and goal selection. During
navigation, the VLM acts as a high-level semantic planner, interpreting the spatial layout encoded in the HSGM to select geometrically valid waypoints, while low-level, collision-free movements between waypoints are executed by a classical path-planning
algorithm, fully decoupling semantic
reasoning from action execution. Additionally, complex instructions are decomposed into subtasks to alleviate the problem of progress forgetting or hallucinating in long-horizon
navigation. Extensive experiments on R2R-CE and RxR-CE
benchmarks demonstrate that our zero-shot framework achieves state-of-the-art performance and even outperforms several supervised methods. Code is available at https://github.com/Teacher-Tom/HSGM_public.
Read the full abstract ▾
All authors Kailing Li, Tianwen Qian, Lijin Yang, Yuqian Fu, Jingyu Gong, Xiaoling Wang, Liang He
Robotics
A Factory-Floor Deployment Case Study of VLA Pipelines for Industrial Packaging Task: Workflow, Failures, and Lessons
Brian Zhu, Philipp Schmitt, Philine Meister et al. (+12 more)
Vision-
Language-Action (VLA) policies have shown promising manipulation capabilities, yet their practical impact is often limited by the reliability demands of real-world deployment. We present a deployment study of an industrial packaging task at Siemens Factory (GWE, Erlangen, Germany), where a
robot must pick a transparent accessory bag from a cluttered pile, insert it into the remaining cavity of a cardboard package, and ensure that the bag and its contents remain below the closing plane. Our goal is to understand the practical effort required to adapt a pretrained Pi0.5 policy to a single factory-floor task through iterative fine-tuning and deployment-driven refinement. The pipeline consists of repeated loops of data collection, curation, fine-tuning, evaluation, and targeted recovery data collection. We have accumulated 2535 episodes (10 hours) from the on-site factory settings. In this paper, we contribute an empirical account of a factory-floor VLA deployment, highlighting recurring failure modes and lessons that inform how to improve the deployment workflow.
Read the full abstract ▾
All authors Brian Zhu, Philipp Schmitt, Philine Meister, Lukas Gensler, Momen Khalil, Emmanuele Poggi, Johannes Hechtl, Carsten Braunroth, Kai Wurm, Gokul Narayanan, Eugen Solowjow, Georg von Wichert, Andre Scholz, Felix Albrecht, Maxmillian Metzner
Robotics
Closing the Loop in Teleoperation: Episode-Level Data Quality Assessment and Feedback for High-Quality Demonstration Collection
Gokul Narayanan, Yash Shahapurkar, Melih Erdogan et al. (+2 more)
Industrial automation is at a pivotal moment, as Physical AI is driving a transition from rigid, hand-engineered automation systems toward more flexible and adaptive systems. This shift has created a growing demand for large-scale, real-world
robot demonstration data, making teleoperation an increasingly important mechanism for data collection. However, high-quality teleoperated demonstrations remain difficult to obtain in practice, as novice operators often produce episodes that are task-successful but suboptimal for downstream use due to inefficient motion, repeated corrections, or operation near
robot joint limits. We present a Data Quality Assessment and Feedback (DQAF) framework that closes the loop in teleoperation by providing immediate post-episode feedback grounded in semantic task progress and
robot telemetry. The framework extracts quality relevant signals such as sub-task progress, motion smoothness, stalls, kinematic limits and converts them into structured quality assessments and actionable natural-
language feedback. Unlike binary success or failure feedback, the proposed system explains why an episode is suboptimal and highlights specific behaviors to correct in the next trial. We evaluate the framework through a diagnostic validation study and a pilot user study. In the validation study, the system is compared with a human reviewer during
dataset curation, producing rejection reasons and actionable feedback for improvement. In the pilot study with three novice operators across two manipulation tasks, the operator who received the systems immediate, automated post-episode feedback improved faster than those who did not, producing higher-quality demonstrations sooner.
Read the full abstract ▾
All authors Gokul Narayanan, Yash Shahapurkar, Melih Erdogan, Brian Zhu, Eugen Solowjow
Robotics
RCSP: Risk-Sensitive Conjectural Scenario Planning for Safe Dynamic Robot Navigation
Zhengye Han and Quanyan Zhu
Mobile
robots can fail before they collide: a velocity that is safe now may commit the
robot to a passage that moving obstacles will soon close. We study this predictive near-miss commitment problem and propose Risk-Sensitive Conjectural Scenario Planning (RCSP), a planning layer that evaluates candidate commands against plausible short-horizon obstacle futures. RCSP maintains a lightweight belief over local motion
conjectures, samples future interactions, penalizes high-risk tails, and executes through a local safety check. In controlled MuJoCo bottleneck tasks, the RCSP planner reaches the goal without collisions and yields higher secondary safety and path-quality point estimates than a non-adaptive predictor, with additional latency. In ROS2/Gazebo, adding the local safety layer to a standard Nav2 stack reduces dynamic near-miss failures. On official DynaBARN/Jackal transfer, tuned DWA and TEB remain stronger on strict
benchmark success, revealing the boundary of the approach. These simulation results position RCSP as a predictive-risk module that complements existing
navigation stacks in dynamic bottleneck regimes.
Read the full abstract ▾
Robotics
NightSight: Passive Computation for Navigation in Dark Using Events
Deepak Singh, Brijan Vaghasiya, Shreyas Khobragade et al. (+1 more)
Small aerial
robots are particularly well-suited for search and rescue in confined and hazardous environments due to their agility, low cost, and ability to traverse through cluttered spaces that are inaccessible to larger platforms. However, enabling autonomous
navigation in complete darkness remains a significant challenge, because small aerial
robots cannot easily accommodate perception systems that demand substantial payload, power, or computation. In this work, we present a lightweight perception approach that combines a
monocular event camera, a coded aperture lens, and an infrared dot projector to enable
navigation in such conditions. The projected pattern, when imaged through the coded aperture, produces depth dependent blur signatures that implicitly encode scene geometry. We train a convolutional neural network to decode these signatures into dense depth maps using only synthetic data generated from a simple planar wall setup. Despite this minimal training regime, the model generalizes zero-shot to complex real-world scenes. Our system operates in real time at 20 Hz on a NVIDIA Jetson Orin Nano, demonstrating suitability for resource-constrained platforms. We further analyze the impact of different coded aperture designs on depth estimation performance. Our approach gives high accuracy (l1 error 7.0cm) upto 2.5m range (2.80% error). These results highlight the potential of combining structured illumination, coded optics, and event-based sensing for enabling robust perception and
navigation in complete darkness.
Read the full abstract ▾
All authors Deepak Singh, Brijan Vaghasiya, Shreyas Khobragade, Nitin Sanket
Robotics
Collaborative Navigation and Exploration with $\beta$-Sparse Gaussian Processes
Evangelos Psomiadis, Dipankar Maity and Panagiotis Tsiotras
Collaborative
navigation of heterogeneous
robots in unknown environments poses significant challenges due to sensing, communication, and computational limitations. In this work, a lead
robot navigates toward a target while a mobile sensor
robot (e.g., a drone) assists by transmitting information about its locally observed map under bandwidth constraints. We propose a framework that enables the sensor to jointly select its transmitted map points and
navigation actions online, while also predicting unexplored regions of the environment. To this end, we present $\beta$-Sparse Gaussian Processes, a robust variational sparse Gaussian Process model for task-aware inducing point selection under cardinality constraints. Furthermore, we develop an action-selection strategy that balances task relevance with exploration. Simulations on Mars and Earth maps show that the framework can reduce path cost by 18% relative to no communication and decrease transmitted information by 76% compared to raw-data transmission baselines.
Read the full abstract ▾
Robotics · Cross-listed from cs.MA
Decoupled Delay Compensation: Enhancing Pre-trained MARL Policies via Learned Dynamics Filtering
Maxim Mednikov and Oren Gal
Real-world multi-
agent reinforcement learning (MARL) systems must often operate under stale observations,
stochastic communication delays, and intermittent packet loss. Policies trained under idealized synchronous conditions frequently exhibit significant performance degradation in these regimes because they act on outdated feedback. We propose a modular execution-stage state-estimation layer that replaces delayed communicated observations with current belief-state estimates. The framework integrates a learned Gated transition model with a recursive Kalman filtering layer to estimate instantaneous states from asynchronous measurements. A primary advantage of this approach is its modularity, The
estimator serves as a plug-in for pre-trained policies, requiring no modifications to the original MARL training
algorithm, architecture, or reward structure. Evaluation across diverse multi-
agent and continuous-control
benchmarks demonstrates that the proposed layer consistently enhances robustness to communication latency and message loss. The most significant performance gains are observed in coordination-intensive and dynamically unstable tasks where temporal consistency is critical for control.
Read the full abstract ▾
Robotics
PhyPush: One Push is All You Need for Sensorless Physical Property Estimation with Physics-Guided Transformers
Koyo Fujii, Luis Figueredo, Praminda Caleb-Solly et al. (+4 more)
Accurately estimating object mass and friction is fundamental to reliable robotic manipulation. While interactive perception is powerful, most approaches rely on specialized hardware like force/torque sensors, limiting scalability. This paper introduces PhyPush, a physics-guided
Transformer that estimates an object's mass and friction coefficient using only end-effector velocity from a single push, data readily available on standard robotic arms. By incorporating Newton's second law and the Coulomb friction model through a physics-guided loss, the model improves physical consistency and generalizes to unseen objects and surfaces. Across diverse setups, PhyPush consistently achieves highly accurate estimations in challenging out-of-domain conditions. In simulation, it reduces error by over 10% compared to a baseline with privileged force data, while in real-world experiments, it successfully zero-shot transfers from simulation to outperform a purely data-driven baseline.
Read the full abstract ▾
All authors Koyo Fujii, Luis Figueredo, Praminda Caleb-Solly, Ivan Boschi, Edoardo Ida', Marco Carricato, Aly Magassouba
Robotics
AnyScene: Towards Highly Controllable Driving Scene Generation at Anywhere and Beyond
Haiming Zhang, Junfei Zhou, Feng Jiang et al. (+6 more)
Generating high-fidelity and controllable synthetic data is critical for advancing end-to-end autonomous driving, particularly for addressing the long tail of rare safety-critical scenarios. Existing occupancy-guided methods typically rely on shallow conditioning mechanisms and reference-frame-dependent
video synthesis, which limits fine-grained controllability from arbitrary BEV layouts and restricts their applicability for scalable simulation. In this paper, we propose AnyScene, a unified occupancy-centric framework for driving scene generation. AnyScene generates semantic occupancy sequences from BEV layouts through a Spatial-Temporal Occupancy
Diffusion Transformer that jointly tokenizes BEV and occupancy features in an autoregressive manner. This design enables precise controllability from cross-
dataset and user-defined BEV inputs while naturally supporting long-horizon generation. Building upon the generated occupancy, a Geometry-Grounded View Expansion module treats occupancy as the canonical spatial representation and synthesizes temporally consistent multi-view driving
videos in a reference-free and autoregressive fashion, supporting flexible camera configurations at inference time. Extensive experiments demonstrate that AnyScene achieves state-of-the-art performance in both occupancy and
video generation. It exhibits strong generalization to unseen and customized layouts, and provides measurable benefits for downstream tasks such as sparse-view 3D reconstruction.
Read the full abstract ▾
All authors Haiming Zhang, Junfei Zhou, Feng Jiang, Jingzhong Li, Zhenglong Guo, Penglin Dai, Jifeng Dai, Yan Xie, Benjin Zhu
Robotics · Cross-listed from cs.CV
MIND: Multi-Scale Intent Diffusion for Text-Driven Physics-Based Humanoid Control
Bin Li, Ruichi Zhang, Han Liang et al. (+4 more)
Enabling physics-based
humanoids to execute diverse behaviors from high-level textual commands remains a significant challenge. Existing methods typically follow either a two-stage paradigm that combines kinematic motion generation with physics-based tracking, or an end-to-end imitation-learning paradigm that directly generates actions from text. However, the former suffers from the inherent domain shift between kinematic generation and physics-based tracking, while the latter struggles with the substantial modality gap between textual commands and low-level actions, limiting effective semantic
alignment. Notably,
humanoid states encode rich motion dynamics that are more semantically aligned with textual descriptions than low-level actions, making them a natural basis for deriving behavioral intent. Building upon this insight, we propose MIND, a novel end-to-end
diffusion framework for text-driven physics-based
humanoid control that leverages behavioral intent as a semantic bridge between textual commands and low-level actions. At its core, MIND introduces a multi-scale intent
diffusion mechanism, where a holistic intent predictor captures global behavioral dynamics to guide overall behavior synthesis, while an immediate intent predictor provides step-wise, fine-grained signals for local behavior refinement at each
diffusion step. This hierarchical intent formulation imposes a structured inductive bias for
humanoid control, improving semantic
alignment and behavioral naturalness. Furthermore, MIND encodes
humanoid states into a latent space to enable more effective semantic intent modeling. Extensive experiments demonstrate that MIND outperforms existing methods and synthesizes coherent, physically plausible, and semantically aligned
humanoid behaviors from text commands. Project page: https://binlee26.github.io/MIND_page.
Read the full abstract ▾
All authors Bin Li, Ruichi Zhang, Han Liang, Jingyan Zhang, Juze Zhang, Xin Chen, Jingya Wang
Robotics · Cross-listed from cs.CV
LRDDv3: High-Resolution Long-Range Drone Detection Dataset with Range Information and Thermal Data
Knut Peterson, Zaid Mayers, Azmain Yousuf et al. (+5 more)
Unmanned Aerial Vehicles (
UAVs) have quickly become common in various airspaces, representing a wide range of applications from recreation flying to commercial photography and package delivery. With the increasing prevalence of
UAVs, it becomes critical that both manned and unmanned aircraft can detect
UAVs and other flying objects from long range to effectively track movement and ensure safe operation in shared spaces. While several
datasets have been introduced for drone detection, the need for expanded high-quality data persists, especially in the area of high-resolution long-range drone data. To address this, we introduce a high-resolution
dataset of 102,532 long-range RGB images of drones, sampled at 5 FPS from 128 distinct
video clips taken mid flight during 17 different data collection days spread over 8 months to ensure a wide variety of lighting scenarios, flight locations, and background elements. The
dataset boasts comprehensive drone range information across the
dataset, as well as 29,630 IR images, all paired with RGB counterparts from the base
dataset. As one of the first drone detection
datasets to leverage 4K image resolution and paired 640x512 IR images, our work represents a significant advancement to enable the detection of drones at long range. For access to the complete
dataset, please visit https://research.coe.drexel.edu/ece/imaple/lrddv3/
Read the full abstract ▾
All authors Knut Peterson, Zaid Mayers, Azmain Yousuf, Priontu Chowdhury, Asher Zaczepinski, Solmaz Arezoomandan, Reihaneh Maarefdoust, David Han
Robotics · Cross-listed from cs.CV
AgentGrounder: Zero-Shot 3D Visual Pointcloud Grounding using Multimodal Language Models
Cuong Huynh, Maxim Popov, Denis Gridusov et al. (+1 more)
3D Visual Grounding (3DVG) is an essential capability for
embodied AI, requiring
agents to localize objects in 3D scenes based on natural
language descriptions. Recent zero-shot methods leverage 2D
vision-
language models (LVLMs). However, they often rely on existing sets of multi-view images and struggle with the limited semantic and spatial details provided by standard 3D
segmentation tools. We present $\textbf{AgentGrounder}$, a zero-shot 3D visual grounding framework that operates directly on colored point clouds without task-specific 3D training. Our approach follows a two-stage design: (1) an offline stage that applies 3D model to build an Object Lookup Table (OLT) with instance IDs, semantic labels, 3D bounding boxes; and (2) an online tool-driven
agent that decomposes each query, retrieves only relevant candidates from the OLT, performs
geometric scoring, and triggers image rendering on demand when additional visual evidence (e.g., color, material, or viewpoint-sensitive cues) is required. Compared with fixed anchor-target matching pipelines, this design reduces cascading matching errors and improves context-window efficiency by avoiding
prompts overloaded with irrelevant objects. We evaluate on ScanRefer and Nr3D under a zero-shot setting and observe consistent improvements over SeeGround in our setup, including +2.5% Acc@0.5 on ScanRefer and +6.3% on Nr3D, with a notable +6.3% gain on Nr3D view-independent queries. These results show that combining selective
retrieval,
geometric reasoning, and adaptive visual inspection yields a practical and robust foundation for open-vocabulary 3D grounding. Our code is available at https://github.com/be2rlab/AgentGrounder.
Read the full abstract ▾
All authors Cuong Huynh, Maxim Popov, Denis Gridusov, Sergey Kolyubin
Robotics
RePlan-Bot: Multi-Level Replanning for Embodied Instruction Following
Xicheng Gong, Guozheng Sun, Peiran Xu et al. (+1 more)
Embodied instruction following (EIF) requires
agents to understand and execute complex natural
language commands within interactive 3D environments. Despite recent advances, existing methods often fail in long-horizon planning and handling irreversible state changes, resulting in low task success rates. To address these challenges, we introduce RePlan-Bot, a novel EIF
agent that performs multi-level, continuous replanning throughout task execution. RePlan-Bot integrates a high-level LLM-based auditor for dynamic sub-goal adjustments guided by environmental feedback, a commonsense-guided search mechanism based on a multi-layered instance map for precise and structured object localization, and a lightweight ViT-based corrector to preemptively fix risky low-level actions. Evaluated on the ALFRED
benchmark, RePlan-Bot achieves state-of-the-art performance in both seen and unseen environments, demonstrating superior adaptability and reliability.
Read the full abstract ▾
All authors Xicheng Gong, Guozheng Sun, Peiran Xu, Yadong Mu
Robotics
When Search Becomes Memory: Accelerating Robot Design Discovery with Self-Evolving Skills
Yunfei Wang, Xiaohao Xu, Yang Li et al. (+1 more)
Large
language models (LLMs) are increasingly used as proposal generators for evolutionary
robot design, yet most loops remain memoryless: simulator results shape the next population but are not preserved as reusable design knowledge. We present Auto-Robotist, a self-evolving LLM
agent that distills morphology-search traces into an explicit natural-
language skill library. Each skill stores a structural archetype, evidence-grounded positive and negative rules, and the evaluated designs that support them, making design
memory inspectable rather than implicit in a population. During search, the
agent retrieves skills to condition LLM edits of elite bodies while retaining a Genetic
Algorithm (GA) mutation path for exploration; after evaluation, it updates the library through Add, Diagnose, and Merge. Across seven EvoGym tasks spanning locomotion, traversal, and object interaction, Auto-Robotist improves cold-start 5x5 search and transfers learned skills to 10x10 design spaces, where reference-conditioned transfer outperforms GA on every task. These results suggest that LLM
agents can convert expensive physical evaluations into reusable, auditable design principles. Our code is publicly available at https://github.com/wangyf9/Auto-Robotist .
Read the full abstract ▾
All authors Yunfei Wang, Xiaohao Xu, Yang Li, Xiaonan Huang
Robotics
OASIS: Observation-Action Space Alignment via SE(3) Trajectory Prediction for Robotic Manipulation
Xinzhe Chen, Sihua Ren, Liqi Huang et al. (+5 more)
Recent
vision-
language-action (VLA) models and world action models (WAMs) advance robotic manipulation by enriching intermediate representations with auxiliary spatial features or future visual-state prediction. However, these representations largely remain within the observation space and do not share the rigid-body geometry of the action space, forcing the action decoder to implicitly recover this geometry. We propose OASIS, a visuomotor policy that aligns the intermediate representation with the action space via $SE(3)$ end-effector trajectory prediction. OASIS couples a 3D-aware feature encoder that fuses
vision-
language and metric-depth features with an $SE(3)$ trajectory predictor that produces a camera-frame end-effector trajectory. Conditioned on the predictor's pose-supervised hidden states, the action decoder generates action chunks consistent with rigid-body motion. Across simulation and real-world experiments, OASIS outperforms VLA and WAM baselines in success rate and out-of-distribution generalization. Our project page is available at https://npuhandsome.github.io/OASIS_web.
Read the full abstract ▾
All authors Xinzhe Chen, Sihua Ren, Liqi Huang, Haowen Sun, Mingyang Li, Xingyu Chen, Zeyang Liu, Xuguang Lan
Robotics
Extending Embodied Question Answering from Perception to Decision
Xicheng Gong, Qiwei Li, Peiran Xu et al. (+1 more)
Embodied Question Answering (EQA) connects perception,
reasoning, and interaction within
embodied environments. However, existing
datasets and
benchmarks remain fragmented, each focusing on a limited subset of
reasoning skills such as spatial understanding or procedural
reasoning, without offering a unified large-scale framework for comprehensive evaluation. We present EQA-Decision, a large-scale
embodied QA
dataset that systematically covers four complementary dimensions of
embodied reasoning: static scene construction, spatial understanding, task dynamics
reasoning, and instant decision. The
dataset contains over four million question-answer pairs with hierarchical annotations across diverse
embodied scenarios. In addition, we develop RoboDecision, a strong baseline model aligned with the EQA-Decision
Benchmark, providing a unified framework that jointly evaluates perception,
reasoning, and action-level decision-making in
embodied environments. Results demonstrate that EQA-Decision effectively
benchmarks and enhances VLM capabilities in spatial and interaction
reasoning, providing a solid foundation for advancing
embodied intelligence research.
Read the full abstract ▾
All authors Xicheng Gong, Qiwei Li, Peiran Xu, Yadong Mu
Robotics
HoLoArm: Deformable Arms for Collision-Tolerant Quadrotor Flight
Quang Ngoc Pham, Jonas Eschmann, Yang Zhou et al. (+3 more)
The increasing use of drones in human-centric applications highlights the need for designs that can survive collisions and recover rapidly, minimizing risks to both humans and the environment. We present HoLoArm, a quadrotor with compliant arms inspired by the nodus structure of dragonfly wings. This design provides natural flexibility and resilience while preserving flight stability, which is further reinforced by the integration of a Reinforcement Learning (RL) control policy that enhances both recovery and hovering performance. Experimental results demonstrate that HoLoArm can passively deform in any direction, including axial one, and recover within 0.3-0.6 s depending on the direction and level of the impact. The drone can survive collisions at speeds up to 7.6 m/s and carry a 540 g payload while maintaining stable flight. This work contributes to the morphological design of soft aerial
robots with high agility and reliable safety, enabling operation in cluttered and human shared environments, and lays the groundwork for future fully soft drones that integrate compliant structures with intelligent control.
Read the full abstract ▾
All authors Quang Ngoc Pham, Jonas Eschmann, Yang Zhou, Alejandro Ojeda Olarte, Giuseppe Loianno, Van Anh Ho
Robotics
ParkourFormer: Integrating Predictive Supervision and Sequence Modeling into Parkour Locomotion
Yanheng Mai, Wenhao Xu, Zirui Huang et al. (+6 more)
Humanoid parkour requires locomotion policies to coordinate whole-body dynamics across rapidly changing terrains such as stairs, gaps, slopes, and obstacles. Existing learning-based
humanoid locomotion policies typically infer actions from current and historical observations, while the future
evolution of the
robot state is represented only implicitly. Explicitly anticipating short-horizon state
evolution is particularly beneficial for multiple terrains locomotion, where current actions must account for upcoming contact transitions and whole-body dynamics. We present ParkourForme, a
Transformer-based sequence modeling framework that reformulates
humanoid locomotion as a future-conditioned decision-making problem. The current
robot state queries historical sensorimotor trajectories through cross-
attention, while a lightweight prediction head forecasts short-horizon future proprioceptive states. The future states are predicted solely from historical observations and actions, with ground-truth future states used only as supervision targets, and are fused with temporal features to generate the current action without future-information leakage. We evaluate ParkourFormer on a diverse multi-terrain
humanoid parkour
benchmark including stairs, gaps, slopes, rough terrain, and obstacle traversal. ParkourFormer achieves a 93.96% mean traversal success rate, compared with 90.49% for a vanilla
Transformer and 87.16% for a MoE-MLP policy, while maintaining a single policy across all terrain types. These results demonstrate that explicit future-state modeling significantly improves robustness and generalization for agile whole-body locomotion.
Read the full abstract ▾
All authors Yanheng Mai, Wenhao Xu, Zirui Huang, Kailun Huang, Yifei Fu, Shengwei Dong, Xinjue Wang, Yanzhe Xie, Renjing Xu
Robotics
Implicit Null-space Manifold Generation for Redundant Robotic Systems
Taiki Ishigaki, Teresa Vidal-Calleja, Ko Ayusawa et al. (+1 more)
Robotic systems with redundant degrees of freedom can achieve the same task outcome using multiple configurations, resulting in solution sets that form
manifolds in the configuration space. Existing approaches typically exploit such redundancy locally through Jacobian-based techniques to compute individual solutions or trajectories. While effective for solution computation, these methods do not retain a representation of the geometry of the solution set itself. In this work, we adopt a representation-centric approach to estimate the
geometric structure of the solution space. We consider solution
manifolds induced by general task-defining maps and construct an implicit scalar field over the configuration space, whose zero-level set corresponds to the solution
manifold. To this end, we generate samples in the neighborhood of the solution
manifold using a Jacobian-guided exploration strategy, which efficiently captures its local and global structure. The resulting implicit representation is defined over the configuration space and naturally induces a continuous, distance field that encodes proximity to the solution
manifold. Experiments on a planar three-link
robot and a seven-degree-of-freedom Franka manipulator demonstrate the effectiveness of the proposed representation. Furthermore, the framework enables consistent modeling of solution spaces across families of tasks with continuous variation.
Read the full abstract ▾
All authors Taiki Ishigaki, Teresa Vidal-Calleja, Ko Ayusawa, Eiichi Yoshida
Robotics
HumanFlow – Diffusion-Driven MAV Navigation Among Humans via Tightly-Coupled Motion Tracking, Forecasting, and Control
Simon Schaefer, Joshua Näf and Stefan Leutenegger
Robust and accurate perception of humans in their 3D scene context is essential for integrating
robots into everyday environments. Existing approaches, however, often fail to predict plausible and accurate human motion estimates that are consistent with the surrounding scene, especially in the presence of heavy occlusions or partial visibility. This can limit both safety and efficiency for robotic operations. We introduce HumanFlow, a latent
diffusion model that unifies human motion tracking and
forecasting, conditioned on the 3D scene context. We show that our human motion model produces smooth and accurate predictions under challenging conditions, including heavy occlusions, and outperforms state-of-the-art methods in tracking accuracy while being significantly more efficient. Furthermore, we show how HumanFlow's latent space can be tightly coupled with control by conditioning a flow-matching-based, approximate MPC policy on these representations. We validate our policy in simulation with real human trajectories for MAV social
navigation, demonstrating superior
navigation performance and remaining collision-free, even under partial observability of the human.
Read the full abstract ▾
Robotics
Compliant Non-Prehensile Pushing Manipulation
Francesco Cufino, Mario Selvaggio, Fabio Amadio et al. (+1 more)
In this paper, we address the challenge of performing non-prehensile pushing operations with a compliant robotic manipulation system. To ensure safe operations in human-populated environments,
robots must comply with external physical interactions and exhibit a passive behavior. To achieve this, we extend a state-of-the-art pushing model to integrate it with impedance-controlled
robots. We develop a model predictive control framework built upon this model that enables performing compliant pushing through optimal modulation of the
robot's position/velocity set-point, jointly realizing the required pushing force and contact point adaptation to obtain a desired object motion. However, external interactions may induce tracking errors causing a consequent potentially indefinite increase of the pushing force. To prevent this, we integrate an energy tank passivity filter that further modulates the
robot velocity set-point to guarantee passivity and avoid uncontrolled energy buildup. The proposed method has been rigorously tested in simulation and validated through experiments on two different robotic systems, demonstrating passive compliance during human-
robot interactions, and assessing trajectory tracking performance and robustness to variations in the object's physical parameters.
Read the full abstract ▾
All authors Francesco Cufino, Mario Selvaggio, Fabio Amadio, Fabio Ruggiero
Robotics
G-DRAGON: Geospatial Reasoning and Dynamic Planning for Retrieval-Augmented Outdoor Navigation
Dongzhihan Wang, Yi Du, Jianan Sun et al. (+5 more)
Autonomous ground
robots operating in large-scale outdoor environments require both robust long-range
navigation and fine-grained ”last-mile” exploration. Current advances in visual-
language navigation (VLN) work well at short-range tasks, lacking geospatial grounding for long-distance missions. Some OpenStreetMap (OSM)-based methods relying on cloud-based Large
Language Models (LLMs) are prone to factual
hallucination and cannot conduct ”last-mile” exploration based on human instruction. To address these challenges, we present G-DRAGON, a
retrieval-augmented framework for outdoor, open-world
navigation. This framework maps natural-
language commands to versioned, local OSM entities via generative
retrieval based on lightweight LLM, yielding accurate coordinates for global route planning. A high-level planning module bridges global topological routes with the SLAM system, projecting geospatial waypoints into the
robot's navigable frame. For the ”last mile," the framework transitions to frontier-based exploration and open-set semantic voxel mapping to localize open-vocabulary targets. Experimental results in simulation demonstrate our framework outperforms state-of-the-art baselines. Furthermore, we validate the system in unseen real-world urban environments on an Unmanned Ground Vehicle (UGV), successfully completing person-search missions with trajectories of up to 500m.
Read the full abstract ▾
All authors Dongzhihan Wang, Yi Du, Jianan Sun, Yuan Xue, Yingchen Zhang, Bing Xiao, Chen Wang, Liang Xu
Robotics
Acting on the Unseen: Communication-Free Collaborative Filtering for Decentralized Multi-Robot Task Allocation
Alexander Apartsin, Yigal Meshulam and Yehudit Aperstein
Multi-
robot task allocation usually assumes some combination of communication, known task models, or a coordinator. We study the opposite extreme, a regime common in practice but overlooked in theory, which we name Zero-Knowledge MRTA (ZK-MRTA): a
robot team with no prior knowledge (no task models, not even the latent rank), no communication (no messages, no parameter sharing, no coordinator), and only a partial and privately-noisy view of a public stream of teammates' outcomes. A hidden low-rank structure governs which
robot suits which task, and there are far more tasks than rounds, so most (
robot, task) pairs are never attempted. Yet each
robot can act well on tasks it never attempted, and onboard new tasks, by running online low-rank collaborative filtering over the broadcast (SwarmCF). The advantage over any structure-free learner is categorical, not a constant factor: a structure-free learner is provably at the prior-mean error floor on unseen pairs. We prove a matching per-
robot sample complexity ({\Theta}(d) versus {\Theta}(n), in the rank d and the task count n), an anytime (cumulative-reward) separation under task scarcity, and a deterministic condition under which decentralized recovery from the masked broadcast is exact (validated empirically). Experiments quantify the value of the broadcast, a positive scaling law (per-
robot unseen-pair skill rises with team size), and the strongest masking-robustness and anytime profile among low-rank methods, recovering most (about 80% on earned skill) of a centralized full-communication ceiling, and holding under capacity-1 contention and in a robotics-grounded sensing instance.
Read the full abstract ▾
Robotics · Cross-listed from cs.CV
ComPose: A Unified Completion-Pose Framework for Robust Category-Level Object Pose Estimation
Huan Ren, Yihan Chen, Chuxin Wang et al. (+3 more)
Category-level object pose estimation aims to predict the pose and size of arbitrary objects in specific categories. Existing methods struggle with the inherent incompleteness of observed point clouds, which limits their ability to capture complete object shapes for robust pose
reasoning. While point cloud completion offers a promising solution, naively treating it as a separate preprocessing step for partial observations introduces compounding errors and additional computational overhead, ultimately hindering both accuracy and efficiency. To address these challenges, we propose ComPose, a novel unified framework that tightly integrates shape completion to provide complete
geometric cues for enhanced pose estimation. At the core of ComPose is a keypoint-based progressive completion module, which recovers full shape representations by progressively predicting a sparse set of keypoints and their surrounding dense point sets, empowering the keypoints to capture holistic object geometries. A
geometric relation encoding module further enriches keypoint features with both local and global
geometric context. In addition, we introduce a novel
geometric relation consistency loss to enforce structural
alignment between observed keypoints and their predicted NOCS coordinates, ensuring globally coherent coordinate transformations. Extensive experiments on standard
benchmarks demonstrate that our method outperforms state-of-the-art approaches without relying on category-level shape priors.
Read the full abstract ▾
All authors Huan Ren, Yihan Chen, Chuxin Wang, Nailong Liu, Wenfei Yang, Tianzhu Zhang