Robotics
Seeing What the Vehicle Sees: Video-Augmented Virtual Reality for Physical Autonomous Vehicles
Md Tanjemul Islam, Mohammad Shafin and Md Rafiul Kabir
Autonomous vehicles are expected to improve road safety and efficiency, but passengers often remain uncertain about what the vehicle perceives and why it acts as it does. Virtual reality (VR) offers a safe and repeatable medium for presenting this information, yet most passenger-facing VR studies rely on fully simulated vehicles or pre-scripted scenarios, so the motion and perception shown to the user do not originate from a physically operating autonomous system. This paper presents a
video-augmented VR framework that couples a physical ROS 2 autonomous
robot vehicle to a Unity 6 application deployed on a Meta Quest 3S headset. The vehicle state and live onboard camera stream are transmitted over two independent communication channels, allowing the virtual vehicle to mirror the physical
robot's motion while the passenger simultaneously views the vehicle's first-person camera feed and its navigation decisions through an in-vehicle dashboard interface. We evaluate the framework over 20 repeated closed-loop navigation trials. The system achieves a mean state-update latency of 29.63 ms, a mean relative route-progress error of 2.28% between the physical and virtual vehicles, and
video delivery at 10.006 frames per second with 0.25% frame loss. All monitored navigation decisions were correctly reflected in the VR interface with no missed or incorrect notifications. The results indicate that the framework can support temporally synchronized, semantically consistent, and accurate route-progress representation for immersive observation of physical autonomous-vehicle behavior.
Read the full abstract ▾
Robotics
Robots Influencing Humans to Reveal their Goals during Collaboration and Competition
Debasmita Ghose, Oz Gitelson, Michal Lewkowicz et al. (+3 more)
We propose a unified strategy for fast goal inference in human-
robot interaction. The core idea is to drive the human toward Critical Decision Points (CDPs)-states where competing human strategies prescribe different next actions and thus maximally reveal the goal. We formalise CDPs using a goal-conditioned policy divergence measure and incorporate them into a Receding-Horizon Planner that explores future action sequences while optimizing a cost function balancing task progress and information gain. We evaluate this approach in both a collaborative, fully observable cooking task and a competitive, partially observable hide-and-seek game, each in simulation and on real
robots. In both scenarios, our method infers human goals more accurately and earlier than baseline strategies.
Read the full abstract ▾
All authors Debasmita Ghose, Oz Gitelson, Michal Lewkowicz, Jake Brawer, Marynel Vazquez, Brian Scassellati
Robotics
Multi-robot Learning-based Informative Path Planning Using Spatio-Temporal Gaussian Process Kalman Filter
Muqing Cao, Yunwoo Lee, Junbin Yuan et al. (+2 more)
Multi-
robot informative path planning (IPP) for persistent target monitoring requires
robots to reason about spatial uncertainty, temporal
evolution, and practical sensing and communication constraints. Recent learning-based multi-
robot IPP methods use Gaussian Processes (GPs) for target uncertainty, but often rely on simplified sensing models and centralized belief updates. We propose a grid-based spatio-temporal GP-Kalman filtering framework for learning-based multi-
robot IPP. Instead of maintaining one GP per target, we represent anonymous target presence as a single latent field over a discrete workspace grid. The proposed recursive update considers all visible
cells inside a camera footprint and supports arbitrary fields of view and range-dependent noise. A GP-consistent temporal process update accounts for moving targets and stale information by inflating uncertainty over time. For decentralized deployment, each
robot maintains its own mapper and exchanges compact belief summaries rather than raw measurements. Received beliefs are fused using diagonal covariance intersection to remain conservative under unknown inter-
robot correlations. We integrate the mapper with a reinforcement-learning policy for graph-based neighbor selection. Simulation
benchmarks show about 20% lower average target uncertainty and improved target visitation compared with learning-based and classical auction/coverage baselines. Real-world two-
UAV experiments demonstrate transfer to outdoor multi-
robot search over a large field of more than 7000 square meters.
Read the full abstract ▾
All authors Muqing Cao, Yunwoo Lee, Junbin Yuan, Lorenzo Schenk, Sebastian Scherer
Robotics
Not All Agreement Counts as Corroboration: Provenance-Conserving Multi-View Fusion for Typed Action Admission in Human-Robot Collaboration
Zekai Jin, Hanrong Zhang, Yihong Tang et al. (+3 more)
Better probability scores do not establish that evidence has been counted correctly. Repeated inference over one observation can improve predictions without adding an evidential origin. Source-local numerical attributes alone cannot in general distinguish repeated derivations from separately countable acquisitions. PACT (Provenance-Aware evidence Conservation and Typed action admission) separates evidence magnitude from countability through a supplied provenance partition. Under singleton fidelity and insertion non-amplification, the coordinatewise meet is the unique pointwise greatest admissible within-component rule. Component budgets add under stated commensurability and separate-component additivity assumptions. Matched reassignments hold numerical outputs fixed while varying the counting relation. In four of 12 replicated-source tests on HandWritten, false refinement lowers macro-averaged negative log-likelihood and Brier score while increasing normalized common-support area under the risk-coverage curve (ncsAURC). In the controlled handover
benchmark, removing the constructed
adversarial-consensus condition leaves a 0.056 reduction in ncsAURC for provenance-partition aggregation relative to singleton aggregation under the same score functional. The corroboration contrast disappears, and method ranking remains selection-score dependent. In offline, reference-based human-
robot collaboration with four
prompts per camera and all other admission inputs fixed, duplicating each
prompt output within its camera from multiplicity one to eight leaves all 720 PACT typed responses per checkpoint unchanged. Probability quality and evidence countability require separate evaluation.
Read the full abstract ▾
All authors Zekai Jin, Hanrong Zhang, Yihong Tang, Fei Hu, Zhen Dong, Yi Shao
Robotics · Cross-listed from cs.CV
Beyond Textual Chain-of-Thought: A Survey on Action-Grounded Reasoning in Autonomous Driving
Zhengxu Tang, Xiaozhou Zhang, Guofeng Cui et al. (+10 more)
Chain-of-thought (CoT)
reasoning powers generative models by eliciting intermediate steps before producing an answer. In autonomous driving, the answer is a continuous action. Thus its
reasoning must share the same spatiotemporal structure as the physical world. This survey studies the resulting shift from textual CoT to action-grounded
reasoning. Surveying 171 papers, including 130 method papers and 41
benchmarks,
datasets, surveys, and analysis papers, we propose a representation-centered taxonomy that treats the form of the intermediate state as the organizing axis. We systematize the 130 methods into four categories:
language-based, visual-spatial, latent-dynamic, and externalized
reasoning, further divided into 13 subtypes tied to distinct regions of interests. Our synthesis shows that the open frontier of
reasoning in driving
agents lies in intermediate representations that can be grounded in the real world, coupled to real-time action, and verified under safety-critical systems. Project page: https://github.com/tangzhengxu/awesome-av-cot.
Read the full abstract ▾
All authors Zhengxu Tang, Xiaozhou Zhang, Guofeng Cui, Ziyu Gong, Zi Wang, Yunfei Shi, Ruifeng Deng, Chengzhi Qi, Ke Chen, Sachin Patil, Tianjun Xiao, Langechuan Liu, Pichao Wang
Robotics
Beyond Object Selection:Markerless Gaze-based Robot Placement at Arbitrary Positions
Yuzhi Lai, William Marx, Shenghai Yuan et al. (+3 more)
Gaze-based assistive manipulation typically supports object selection, while arbitrary-position placement requires accurate spatial
alignment between the headset and
robot. However, for gaze-based manipulation, pose accuracy does not necessarily translate into task accuracy: translational and rotational errors jointly affect the transformed gaze ray and may compensate for each other. To study cross-device
alignment from this task-oriented perspective, we present a markerless interaction framework and a dedicated cross-device
dataset. We propose Graph-based Reference Selection to address sparse
robot references. We further develop and
benchmark multiple task-specific
alignment pipelines under a unified protocol. Specifically, we introduce Gaze–Surface Intersection Error (GSIE), which directly measures the spatial error of the gaze-specified target. Experiments show that
alignment methods ranked highly by conventional pose metrics are not always optimal in GSIE, demonstrating the importance of evaluating gaze-based manipulation at the task level.
Read the full abstract ▾
All authors Yuzhi Lai, William Marx, Shenghai Yuan, Peizheng Li, Zhuoyu Ran, Andreas Zell
Robotics
Scene Graph-based Driving Scenario Extraction for Automotive Egocentric Datasets
Stefan Ramdhan, Kyanna Dagenais, Vera Pantelic et al. (+2 more)
Extracting scenarios from unlabelled real-world sensor data streams is a critical but challenging task in the development process of automated driving systems (ADS). Automatically sifting through large
datasets to spatially and temporally locate critical scenarios can enable scenario-based coverage analysis of ADS
datasets. In this paper, we present a method for extracting scenarios from egocentric
datasets using scene graphs and Linear Temporal Logic (LTL). We first process egocentric sensor data and HD maps to generate a sequence of scene graphs representing a driving scenario. Next, we use LTL to formally specify driving scenarios of interest, then extract all instances of the scenarios from the
dataset using an off-the-shelf model checker, which evaluates the LTL formula against the sequence of scene graphs. Our approach can be used on both simulated and real world
datasets. We evaluate the method on the training and validation
datasets from Argoverse 2 consisting of 850 15-second real-world driving logs, and several
videos of dashcam footage. We demonstrate the effectiveness of our approach for extracting and querying scenarios by evaluating against a rule-based
benchmark based on track annotations and HD maps.
Read the full abstract ▾
All authors Stefan Ramdhan, Kyanna Dagenais, Vera Pantelic, Victor Bandur, Mark Lawford
Robotics
Geometric analysis of generic 3R robots, and necessary and sufficient conditions for a class of orthogonal robots to have four IKS
Durgesh Haribhau Salunkhe and Abhilash Nayak
The kinematic analysis of a generic 3R
robot has been investigated with multiple approaches in the past. The algebraic approaches have established concrete results but are unfortunately limited to special classes or architectural simplifications.
Geometric approaches on the other hand have extended the analysis to generic
robots while also providing an intuitive understanding of their kinematic properties. We use the best of both approaches to present the kinematic analysis of a generic 3R
robot, using the inverse kinematic model inherited from a method based on conformal
geometric algebra. The paper discusses a generic framework to study the conditions for a 3R
robot to have four inverse kinematic solutions (IKS) and allows to study the distribution of IKS as seen in workspace. The necessary and sufficient conditions for a class of orthogonal
robots are presented using the proposed approach.
Read the full abstract ▾
Robotics
Inverse kinematic solution for generic 3R positional robots using Conformal Geometric Algebra
Abhilash Nayak and Durgesh Haribhau Salunkhe
The inverse kinematics of generic 3R
robots has been investigated through multiple approaches, mainly algebraic methods involving the solution of certain equation sets. Previous
geometric interpretations of the solution, characterized as the intersection of a pair of conics have been confined to the joint-space domain. In this article, we study the Inverse Kinematic Model (IKM) of 3R
robots, using the advantages of Conformal
Geometric Algebra (CGA) to provide further insights on its kinematic properties. Our approach directly yields a univariate
polynomial in terms of theta_2 without the need to eliminate theta_1 and theta_3 by reframing the problem as the intersection of two circles, which are fundamental elements within this algebraic framework.
Read the full abstract ▾
Robotics · Cross-listed from cs.CV
CoLT-Drive: Counterfactual Long-Tail Benchmarking and Knowledge-Preserving Adaptation for Driving Affordance Prediction
Zhengxu Tang, Guofeng Cui, Ziyu Gong et al. (+8 more)
Long-tail autonomous driving failures are often framed as rare-object recognition errors. We argue that this view is incomplete: the decision-critical question is not only whether a model recognizes an unusual object, but whether it infers how that object changes the ego vehicle's feasible high-level actions. We formalize this problem as decision-level driving affordance prediction, where a model maps a front-view image, ego-motion history, and navigation command to a structured longitudinal–lateral meta-action. To evaluate this capability, we introduce CoLT-Drive, a 3,536-sample counterfactual long-tail
benchmark that inserts rare objects into otherwise fixed driving scenes and measures whether models predict acceptable action pairs. To improve deployable small VLMs, we propose KPA, a knowledge-preserving adaptation framework that combines structured perception-to-decision prompting, SLERP-based expert merging, and RegMoE, a regime-aware LoRA mixture-of-experts module. KPA preserves the pretrained model's open-world knowledge while allocating lightweight adaptation capacity to different driving decision regimes. Experiments on an in-domain driving split and CoLT-Drive show that KPA achieves 60.8% pair accuracy on CoLT-Drive, outperforming the pretrained Qwen3-VL-2B baseline (50.3%) and LoRA SFT (32.4%) while maintaining competitive in-domain accuracy. Our
benchmark and code are available at https://huggingface.co/
datasets/tangzx2024/CoLT-Drive and https://github.com/tangzhengxu/CoLT-Drive.
Read the full abstract ▾
All authors Zhengxu Tang, Guofeng Cui, Ziyu Gong, Xiaozhou Zhang, Ruifeng Deng, Chengzhi Qi, Ke Chen, Sachin Patil, Tianjun Xiao, Langechuan Liu, Pichao Wang
Robotics · Cross-listed from cs.AI
IMPACT: Attention Is the Interaction Map for Scalable Interaction-Aware World Model Training
Rongze Tang, Jianjie Fang, Zhaolu Wang et al. (+8 more)
World models have made remarkable progress in action-conditioned future prediction for embodied
agents, yet still struggle to model physically plausible interactions. Existing approaches address this limitation by constraining the generation process with external representations encoding motion, geometry, or semantics. Obtaining these spatiotemporally dense representations typically requires auxiliary
estimators or manual annotations, limiting training scalability. We instead revisit the training objective and identify a supervision-allocation mismatch under the globally averaged mean squared error (MSE) denoising objective: prevalent static content dominates the optimization signal, leaving sparse dynamic-object regions critical to interaction generation disproportionately under-supervised. Motivated by this observation, we introduce IMPACT, a scalable Interaction-aware Model training framework with Prior-guided
Attention Calibration and Targeting. IMPACT uses cross-
attention associated with manipulated-object
tokens as an internal spatiotemporal prior for action-conditioned changes. It samples candidate regions from this prior, calibrates them with detached local prediction errors to construct an interaction map, and uses the map to reweight denoising supervision, requiring neither external representations nor inference-time modifications. Extensive experiments on
robot-arm and human-hand manipulation, spanning diverse control modalities and DiT backbones, show that IMPACT consistently outperforms the corresponding MSE-trained baselines, improving interaction fidelity, physical plausibility, and visual quality.
Read the full abstract ▾
All authors Rongze Tang, Jianjie Fang, Zhaolu Wang, Ziyou Wang, Xvyuan Liu, Haisheng Su, Xin Zhang, Wei Wu, Chen Gao, Yong Li, Zhibo Chen
Robotics · Cross-listed from cs.AI
Deploying and Evaluating a Smart-Agriculture Agentic Engine for Full-Season Soybean Farm Operations
Ao Qu, Panagiotis Michelakis, Linyuan Han et al. (+8 more)
This paper presents FAIRY, a full-stack smart-agriculture
agent system developed for and deployed to an operating soybean research farm at Harbin Institute of Technology's smart-agriculture site. We develop FAIRY to execute and evaluate agentic agronomic operations on full-season spatiotemporal workflows that span ridge preparation, planting, irrigation, fertilization, pest and disease treatment, harvest, grain handling, drying, and storage. FAIRY integrates APIs and infrastructure across production-grade machinery, fixed soil and canopy sensors, multispectral and thermal drones, satellite vegetation products, a weather station, calibrated crop-process models, agronomic records, and multi-season yield histories. The system is built around the novel "everything is an event" execution paradigm, which represents spatiotemporal world
evolution, remote sensing and
UAV observations, sensor readings, crop-growth transitions, machinery actions, and management interventions as state-changing events in a shared farm process engine. On top of this event-driven world model, FAIRY implements a complete agentic stack: a knowledge library of atomic agronomic skills; multi-
agent controller and orchestration backends; frontier- and edge-model execution; full-path trace logging; and deployment profiling on local nodes. We use FAIRY to evaluate nine state-of-the-art
agent controllers across one hundred full-season soybean scenarios that preserve the operational coupling between spatial observations in a 64-ridge field, temporal decision sequences, agronomic constraints, delayed effects, and final yield. We develop an evaluation suite that combines agentic success, full-path spatiotemporal correctness,
token cost, and edge-device runtime.
Read the full abstract ▾
All authors Ao Qu, Panagiotis Michelakis, Linyuan Han, Yiannis Hadjiyianni, Kun Ouyang, Konstantinos Siskos, Feng Li, Ran Meng, Jingchi Jiang, Dimitrios Stamoulis, Jie Liu
Robotics
SUN: Agentic Robot Policy Learning with Persistent Task Programs
Weiqi Wang, Zhi Li, Yudong Lei et al. (+7 more)
Model-based control can directly execute specified objectives, while learning can amortize such behaviors into reactive policies, making their combination a natural solution to multi-stage manipulation. We introduce Semantically UNified (SUN) Programs, typed executables that compile grounded relations into aligned optimal control objectives, satisfaction predicates, and learning rewards. Our harness, Kuafu, equips a foundation model as a task-level
agent to orchestrate scene preparation, verification, residual RL, and data production. The
agent uses program feedback to repair candidate programs and training diagnostics to calibrate relative reward weights, retaining accepted task semantics across tool calls. Across nine multi-stage manipulation tasks, Kuafu achieves 82.03% average success rate, significantly outperforming all learned baselines. Its learned controllers generate demonstrations at 10.57x the human-teleoperation rate, yielding data that improve visualpolicy success by 23.6 percentage points over the strongest baseline. The policies transfer zero-shot to physical Franka and Kinova
robots, demonstrating sim-to-real generalization.
Read the full abstract ▾
All authors Weiqi Wang, Zhi Li, Yudong Lei, David Martinez, Xiaofeng Gao, Yuxin Jiang, Chenfanfu Jiang, Yingnian Wu, Demetri Terzopoulos, Ran Gong
Robotics · Cross-listed from cs.CV
Christian Löwens, Thorben Funke and Alexandru Paul Condurache
End-to-end autonomous driving models plan future trajectories from raw sensor input. While earlier driving
benchmarks often measured deviation from the human trajectory, current
benchmarks such as NAVSIM and Bench2Drive evaluate models with richer simulation-based metrics intended to capture safe and compliant driving. A high
benchmark score should reflect that a model can understand the scene in front of it and act accordingly. But how much of that score specifically comes from reacting to the dynamic part of that scene? To probe this, we remove a model's camera input and replace it with
memories from prior drives at the same location. The retrieved
memories can provide persistent scene information, including road layout and location-conditioned regularities, but not the current traffic state. Surprisingly,
memory is nearly sufficient on NAVSIM, reaching or even exceeding the performance of leading end-to-end methods without actually observing the evaluated scene. Our results suggest that a high NAVSIM score does not require a planner to react to the current traffic scene and should be treated with caution. This effect is
benchmark-dependent: driving from
memory causes substantially larger performance drops on Bench2Drive and RealEngine. We provide our code at https://github.com/boschresearch/MemoryDrivoR .
Read the full abstract ▾
Robotics
DARP: A Calibrated Dual-Arm RGB-D-IR Dataset for Multi-View Robotic Perception
Manish Kansana, Mohammed Yusuf Mujawar, Sudip Mittal et al. (+2 more)
Robotic perception from a single viewpoint is often limited by self-occlusion and incomplete surface visibility. This paper presents DARP(Dual-Arm Robotic Perception) https://doi.org/10.21227/rmv3-be47, a calibrated dual-arm RGB-D-IR
dataset for object-centered robotic perception using two independently moving eye-in-hand manipulators positioned on opposite sides of a shared tabletop workspace. Each arm carries an Intel RealSense sensor that continuously records RGB, depth, and stereo infrared data while synchronized
robot joint states are logged for pose recovery. Objects are placed without fixed poses or marked locations, and the acquisition procedure performs automatic localization, cross-arm confirmation, adaptive viewpoint generation, and continuous
multimodal recording. DARP contains ten unique tabletop objects and preserves the original sensor recordings,
robot-state logs, object-level metadata, and calibration information required to reconstruct camera trajectories in a shared metric frame. To evaluate the
geometric consistency of the acquisition, we implement a deterministic multi-view fusion pipeline that converts calibrated RGB-D observations into complementary partial point clouds and measured surface meshes without using learned or generative completion methods. Evaluation on 224 held-out RGB-D keyframes comprising 1,563,466 three-dimensional query points yields a median point-to-mesh distance of 2.13~mm and an RMSE of 4.04~mm, with 96.56% of points within 10~mm of the measured-surface mesh. DARP is intended as a reusable resource for multi-view reconstruction, collaborative robotic perception,
multimodal fusion, active perception, and future learning-based
reasoning over partial object observations.
Read the full abstract ▾
All authors Manish Kansana, Mohammed Yusuf Mujawar, Sudip Mittal, Shahram Rahimi, Noorbakhsh Amiri Golilarz
Robotics
Autonomously Acquiring Robot Manipulation Skills with Language-Driven Quality-Diversity
Émiland Garrabé, Mahdi Khoramshahi and Stéphane Doncieux
Quality-diversity (QD)
algorithms have been gaining traction in
robot learning, where diverse motion primitive libraries allow
robots to adapt zero-shot to constraints at deployment time. However, such methods typically require expert designers to write the success condition, fitness and diversity metrics, and this strongly limits the
robot's autonomy. On the other hand, existing LLM-based reward-shaping techniques allow
robots to learn autonomously but only output single high-performing solutions, limiting the
robot's adaptability. In this paper, we propose an approach designed to output diverse motion primitive archives by autonomously leveraging quality-diversity
algorithms, only requiring a free-form description of the task in common
language. To address the difficulty of designing relevant fitness and diversity metrics, we propose an autonomous exploration mechanism able to reliably output sets of functionals covering the fitness and behavior descriptor (BD) space. First, we pose policy exploration as a functional design problem, where the functional spaces are lower-dimensional than the full BD and fitness spaces, and propose an LLM-based exploration scheme to sample from these low-dimensional spaces without any task-specific
prompts, fine-tuning or expert intervention. We adapt a multi-BD variant of the MAP-Elites success (MES)
algorithm, designed to leverage the heterogeneous BD samples. Finally, through experiments based on the genesis simulator, we show that our method effectively generates archives of diverse motion primitives, outperforming classical QD
algorithms with inferred and hand-written parametrizations on a set of $4$ robotic manipulation tasks.
Read the full abstract ▾
Robotics
LightNav-0: Eliciting VLM Spatial Intelligence for Generalist Embodied Navigation
Shaoan Wang, Aocheng Luo, Fei Huang et al. (+17 more)
Embodied navigation requires
agents to translate heterogeneous goals and visual observations into actions across tasks, environments, and
robot embodiments. Modern
vision-
language models (VLMs) already encode spatial priors for visual grounding, spatial
reasoning, and pointing, but these capabilities are rarely elicited directly for
robot control. Existing navigation systems instead rely on task- or embodiment-specific components, fragmenting perception,
reasoning, and action while offering limited generalization. Here we present LightNav-0, a compact generalist embodied navigation model that elicits the spatial intelligence of a pretrained VLM and aligns it with navigation, without task-specific prediction heads. LightNav-0 represents diverse navigation tasks through a unified
token interface: dual-channel pointing expresses task-, scene-, and embodiment-agnostic spatial intent, while a residual vector-quantized action tokenizer maps this intent to precise, embodiment-specific trajectories. Together with temporally aware visual history compression, ER mid-training, supervised fine-tuning, and reinforcement learning, this formulation supports instruction following, open-vocabulary object navigation, and visual tracking within a single model. The navigation training corpus spans 2K+ scenes and 4K+ hours of embodied navigation data. LightNav-ER, the embodied-
reasoning checkpoint used to initialize LightNav-0, attains the highest complete-set average across 8 embodied-
reasoning benchmarks, while LightNav-0 achieves state-of-the-art monocular success rates across all 10 public navigation simulation settings. Real-world evaluations further demonstrate zero-shot generalization across
robot embodiments, diverse scenes, and static and dynamic targets. These results establish compact VLMs as a unified and transferable backbone for generalist embodied navigation.
Read the full abstract ▾
All authors Shaoan Wang, Aocheng Luo, Fei Huang, Jingyi Xu, Xiaoyang Wang, Yueyu Wang, Qianli Ma, Fan Yang, Ran Mei, Jia Wei, Jiangpeng Hu, Xuhao Liu, Hongming Chen, Yuanbin Shao, Yiyang Lin, Ziliang Li, Liang Pan, Xinhang Liu, Yuntao Ma, Tingxiang Fan
Robotics · Cross-listed from physics.flu-dyn
The Structure of Merging Turbulent Jets Beneath a Small Quadrotor
Anoop Kiran, Nora Ayanian and Kenneth Breuer
The downwash wake of a hovering quadrotor governs both the vehicle's own performance and the safe spacing of multi-rotor formations. Prior measurements have largely characterized the mean flow, using single-point anemometry, volumetric tracking, or planar cuts through part of the rotor system. Higher-order turbulent statistics of the merged wake, and how they relate to canonical jet scaling, have remained unresolved, particularly for small quadrotors at the low-Reynolds-number end of the size range. Here, we present a detailed particle image velocimetry (PIV) study of the downwash of a hovering Crazyflie 2.1 quadrotor (arm length, $l = 46$ mm), sampled along a diagonal cut, passing through rotors along the
symmetry axis of the quadrotor, and a front-rotor cut, passing through adjacent rotors. The four rotor jets merge into a single column by $z/l \approx 5$, beyond which the mean velocity profiles progressively approach the canonical round-jet self-similar form, collapsing by $z/l \approx 13$ when scaled by the local centerline velocity and half-width. Centerline decay and half-width growth follow canonical scaling laws with an effective source diameter $D_\text{eff} = 2.29\,l$, effective Reynolds number $Re_{D_\text{eff}} = 3 \times 10^4$, at the low end of the range over which canonical jet scaling has been established, and spreading and decay constants nonetheless within the canonical round-jet range. Resolving both cuts shows that the turbulent normal stresses retain a bimodal, cut-dependent signature of the four-rotor source throughout the measurement domain.
Read the full abstract ▾
Robotics
SleepWalking: Privileged Representation Shaping for End-to-End Blind Locomotion in Legged Robots
Zheng Pan, Tenghui Wang, Peilin Li et al. (+5 more)
Partially observable locomotion requires a policy to act when task-relevant properties of the
robot–environment state are not fully specified by instantaneous observations. Existing approaches often address this challenge by explicitly estimating missing physical variables or processing extended observation histories through structured architectures. We take a different view: partial observability is fundamentally an information-retention problem. The decisive question is not how task-relevant information enters the network, but whether the policy's internal state retains it. Guided by this perspective, we propose SleepWalking for
Robot Locomotion (SWAQ), a one-stage end-to-end framework that uses next-step privileged physical reconstruction to shape what a recurrent history representation retains during policy learning, while the deployed actor uses only a direct history-to-action pathway. Under aligned training settings, SWAQ achieves a 15.0% higher peak mean terrain level than DWAQ, the strongest non-exteroceptive baseline, while using 44.4% fewer inference MACs per control step. Layerwise probes further show that information associated with the reconstructed physical variables remains linearly decodable through the policy head up to the layer preceding the action output. Complementary theoretical analysis relates privileged-variable recoverability to the achievable-return gap between history-based and privileged-information policy classes. These results suggest that semantic objectives can structure learning without requiring a corresponding architectural decomposition of the deployed controller.
Read the full abstract ▾
All authors Zheng Pan, Tenghui Wang, Peilin Li, Shiyu Zhou, Hao Sun, Yan Ma, Liang Yu, Liang He
Robotics
Zeva: In-Context Causal Learning for Generalizable Embodied Manipulation
Fu Chen, Xin Ding, Bingjia Huang et al. (+8 more)
Generalizable embodied manipulation remains difficult to achieve through pretraining alone, due to unseen physical conditions in the real world. We argue that
robots need to learn from their own physical interactions on the fly during real-world deployment and use this knowledge to inform subsequent actions. We present Zeva, the first framework that enables in-context learning from a
robot's own physical interaction experience while keeping the policy model frozen. Zeva employs a Causal Interaction Extractor to encode an executed action and its induced state change into a causal interaction signal, which is stored in a dual-timescale causal
memory. For subsequent actions, relevant causal interaction signals are retrieved from
memory and injected into the frozen policy model as context. Experiments in simulation and real-world manipulation demonstrate that Zeva achieves the best performance among the compared frontier VLAs and WAMs and, more importantly, enables self-
evolution during deployment without gradient updates. Its success rate continues to improve as the
robot accumulates interaction experience. Furthermore, the acquired interaction experience can generalize across tasks.
Read the full abstract ▾
All authors Fu Chen, Xin Ding, Bingjia Huang, Xiangyu Li, Mingju Wang, Jiawei He, Kun Li, Wei Sun, Yunxin Liu, Hao Wu, Ting Cao
Robotics · Cross-listed from math.OC
Provably Safe Decentralized Contingency MPC under State-Only Information and Limited Sensing for Nonlinear Multi-agent Systems
Max Studt and Georg Schildbach
This paper considers decentralized contingency MPC for multi-
agent control under a state-only information pattern, with particular focus on limited sensing and plug-and-play operation. The objective is to retain recursive feasibility, safety, and Lyapunov-type convergence while reducing conservatism in local interaction handling. The framework relies on
agent-wise fallback regions (safe sets) in which a feasible contingency maneuver to a safe
equilibrium is always available. A novel safe-set update mechanism is introduced that supports less conservative decentralized interaction while preserving the underlying guarantees. This, in turn, enables
memory-free local interaction and finite sensing ranges without requiring
agents to reconstruct the exact neighbor geometry. The resulting scheme remains fully decentralized and preserves the shared-first-input contingency MPC structure. Theoretical guarantees and simulation results illustrate the effectiveness of the approach in dense multi-
agent scenarios.
Read the full abstract ▾
Robotics
GAFT: Geo-Anchored Fine-Tuning for Hazard Identification from Rare Failures
Yanran Xu, Chuanhang Qiu, Yue Wang et al. (+2 more)
Off-road navigation can fail when physical structures induce irrecoverable states such as high-centering or entrapment, requiring human interventions. Identifying these structures is crucial, yet challenging. Such failure events are rare and costly to collect, resulting in limited training data. Moreover, the collected data associate frames with outcomes, but do not indicate the visual cues responsible for the failure. Learning directly from these data can therefore exploit scenario-specific visual cues, leading to poor generalization. We propose Geo-Anchored Fine-Tuning (GAFT), a parameter-efficient method that adapts a
vision foundation model with a geometry-derived prior. It guides LoRA adaptation by aligning a spatial
attention-rollout map with the geometry prior, while preserving pretrained representations. On an intervention-verified forest hazard
benchmark, across ten independently trained adaptations, GAFT consistently outperforms frozen DINOv2 and supervised PEFT baselines, improving the repeated leave-one-scenario-out mean $F_2$ from 0.0607 to 0.3757 with statistical significance under paired analysis. Within these independently trained models, the best-performing GAFT model achieves a repeated-LOSO $F_2$ of 0.570. Code and
benchmark: https://github.com/Xu-Yanran/geo_anchored_fine_tuning
Read the full abstract ▾
All authors Yanran Xu, Chuanhang Qiu, Yue Wang, Wenbo Wu, Zhaoxing Li
Robotics
A Dual-Cam Parallel Elastic Actuator with Shared Gas-Spring Compensation for Humanoid Ankles
Jingcheng Jiang, Yifang Zhang and Nikos G.Tsagarakis
To improve torque capacity and energy efficiency of humanoid ankles, this paper proposes a 2-DoF parallel elastic actuator (PEA). The main novelty of the proposed design lies in its dual-cam, single-gas-spring architecture, which enables torque compensation in both pitch and roll using a shared elastic element, thereby improving structural compactness compared with conventional multi-element compensation schemes. By leveraging parallel gas springs and customized cam modules, the proposed architecture provides dual-axis torque assistance tailored to specific task requirements. The second key contribution is the formulation of a coupled 2-DoF mathematical model that explicitly captures the interdependence between the two compensation units through the shared spring. Based on this model, an optimization-based design framework is developed to synthesize customized cam profiles from prescribed torque references, establishing a systematic link from task requirements to hardware realization. The complete lower-leg CAD integration is presented in detail. Static FEA and kinematic simulations confirm the design's feasibility and torque-relief effectiveness. The results highlight the proposed design as a compact, customizable solution for 2-DoF humanoid ankle torque compensation.
Read the full abstract ▾
Robotics
Learning to infer and manipulate through distributed whole-arm interaction in a soft robot
Chuhan Zhang, Ebrahim Shahabi, Kseniia Khomenko et al. (+2 more)
In animals such as elephants and octopuses, acquiring non-visual information about an object and physically engaging with it are inseparable processes mediated by rich, large-area interactions between compliant appendages and the environment. Soft
robots provide a natural platform for translating this principle into engineered systems. Yet current robotic intelligence makes limited use of physical interaction, treating it primarily as a disturbance to be rejected or, at best, as a means of compensating for object misalignment. Here, we introduce a physical intelligence framework in which distributed compliant interactions jointly reveal task-relevant information and organize manipulation behavior. This results in an intrinsically partially observable problem: key task-relevant information is never measured directly, but must instead be inferred from the history of physical interactions. We propose a reinforcement-learning architecture that addresses this challenge by learning a
memory-based control policy end-to-end. The key innovations making this possible are (i) a pretrained exploration policy that provides a reference for broad workspace exploration, (ii) joint optimization that integrates exploration and grasping objectives within a single recurrent policy, and (iii) a two-stage sim-to-real adaptation including observation mapping and policy fine-tuning. We demonstrate this principle through blind whole-arm grasping with a hybrid rigid-soft robotic arm that we equip with IMUs embedded directly within its compliant structure, providing its only source of proprioceptive sensing. The learned policy successfully identifies and grasps various objects by autonomously coordinating workspace exploration, object encounter and localization, inference of grasp-relevant properties, and stable whole-arm wrapping.
Read the full abstract ▾
All authors Chuhan Zhang, Ebrahim Shahabi, Kseniia Khomenko, Wei Pan, Cosimo Della Santina
Robotics · Cross-listed from cs.CV
Failure or Drift? Evaluating Monocular SLAM under Synthetic and Real-World Corruptions
Abhay Skaria Thomas, Shashank Agnihotri and Margret Keuper
Visual SLAM is commonly evaluated on clean trajectories, although deployment failures are often caused by adverse weather, illumination, blur, and sensor artifacts. Controlled corruptions are attractive because they isolate such factors, but a synthetic stress test is useful only when it leads to the same engineering conclusion as the condition it is intended to approximate. This work examines that question for monocular SLAM. We evaluate a classical feature-based system and two learned trackers under image-space, geometry-aware, and compound corruptions, and compare their behavior with adverse conditions from 4Seasons. Rather than reducing robustness to a single trajectory error, the evaluation separates explicit tracking failure from drift accumulated by methods that remain active. The results show that learned trackers largely replace catastrophic loss with sustained, and sometimes severe, drift. More importantly, the apparent ordering of the learned systems changes with the physical fidelity of the corruption: structured rain and fog proxies preserve the real-world ordering, whereas a simple illumination proxy does not. Code is available at: https://github.com/abhaythomas/master_thesis_vslamlab_robustness.
Read the full abstract ▾
Robotics
CIG-RL: Curiosity-Driven Information-Guided Reinforcement Learning for Source Term Estimation in Uncertain Environments
Junhee Lee, Seunghwan Kim, Hongro Jang et al. (+4 more)
Source term estimation (STE), which aims to estimate key properties of the gas source, is essential for identifying hazardous gas releases. Information-theoretic approaches have been adopted for autonomous STE using mobile sensors due to robustness in noisy environments, yet their online action selection incurs substantial computational cost. Deep reinforcement learning (DRL) provides a promising alternative with its fast decision-making capability. In DRL-based STE, the
agent selects actions based on belief states of the source term updated from noisy measurement sequences. However, existing methods rely on random exploration or solely on belief uncertainty reduction without an effective exploration strategy in DRL, which can limit policy robustness in noisy environments. To address this, we propose a curiosity-driven information-guided reinforcement learning for robust and efficient STE. The proposed method promotes active exploration of novel belief state transitions that have not been sufficiently explored during training. We further introduce an uncertainty-adaptive active perception reward to guide efficient source search under uncertainty. Simulations under high-noise conditions and real-world experiments demonstrate the robustness and feasibility of the proposed framework, highlighting its potential for practical STE problems.
Read the full abstract ▾
All authors Junhee Lee, Seunghwan Kim, Hongro Jang, Hyungjin Kim, Hyoungho Park, Changseung Kim, Hyondong Oh
Robotics
Temporal Forcing: 4D Representation Alignment for Vision-Language-Action Models
Xingyu Ding, Yuzhong Zhao, Chunhai Zhao et al. (+3 more)
Long-horizon robotic manipulation requires
vision-
language-action (VLA) models to track scene states and their
evolution beyond the current observation. However, simply conditioning policies on observation history does not guarantee that the history is effectively utilized: action supervision constrains what the policy should do, but only indirectly constrains what its history representations should retain. To resolve this, we present Temporal Forcing, a 4D representation
alignment framework that explicitly supervises latent temporal states and their transitions. Specifically, we first introduce a history pathway that compresses past observations into compact latent
tokens. We then align these
tokens and current-frame features with
geometric targets from a pretrained 4D foundation model, providing direct supervision at both the state and transition levels. The 4D foundation model and
alignment heads are used only for training-time supervision. Temporal Forcing improves average success from 96.6% to 98.8% on LIBERO, with the largest gain on LIBERO-Long (93.8% to 97.2%), and from 53.5% to 62.8% across twelve RoboTwin 2.0 tasks. Furthermore, Temporal Forcing increases full-task success from 20.0% to 43.3% on a physical multi-stage hidden-placement task. Controlled experiments show that 4D representation
alignment is crucial for making observation history beneficial to the model. Code will be publicly available.
Read the full abstract ▾
All authors Xingyu Ding, Yuzhong Zhao, Chunhai Zhao, Yinghuan Shi, Chaoyang Zhao, Yifan Zhang