Peer Revenue Equity in science

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Computer Science Robotics

cs.RO Watch Robotics

Friday 3 July

Robotics · Cross-listed from cs.LG

Anticipatory Reinforcement Learning for Trajectory Tracking

Georg Schäfer, Jakob Rehrl, Stefan Huber et al. (+1 more)

Deep reinforcement learning (DRL) in industrial control often suffers from lag and overshoot due to purely reactive control based on the current tracking error. To achieve anticipatory control without high computational overhead, we introduce a predictive formulation that augments the DRL state space with target velocities and future reference horizons. Evaluating eight configurations using proximal policy optimization (PPO) on a 1-degree-of-freedom (1-DoF) helicopter testbed, simulation results showed a 9-fold error reduction, lowering the mean absolute deviation from 2.73{\deg} to 0.31{\deg}. However, zero-shot transfer to physical hardware revealed a sim-to-real gap. Interestingly, a simpler configuration using a single, further look-ahead horizon matched the real-world top performance of the most complex model (1.11{\deg}). Overall, evaluating various combinations of prediction horizons and target velocities demonstrated that highly granular predictive data is not necessarily required for physical transfer.
Read the full abstract

All authors Georg Schäfer, Jakob Rehrl, Stefan Huber, Simon Hirlaender

Robotics · Cross-listed from cs.LG

Integrating Physics-Informed Neural Networks for Safe Reinforcement Learning in a 1-DoF Helicopter System

Georg Schäfer, Jakob Rehrl and Stefan Huber

Deep reinforcement learning (DRL) offers powerful control for industrial cyber-physical systems (ICPSs), but its "black-box" exploration risks violating strict hardware safety limits. Typically, these constraints are managed through complex reward shaping. In this work-in-progress paper, we embed a differentiable physics model directly into the proximal policy optimization (PPO) actor loss function. By simulating short-horizon future trajectories during training, the policy is penalized for anticipated safety violations independent of the task-reward signal. Evaluated on a simulated 1-degree-of-freedom helicopter testbed with strict pitch constraints, our physics-informed soft regularizations substantially reduce constraint violations while maintaining reliable target tracking.
Read the full abstract

Robotics · Cross-listed from cs.MA

LOTUSim: Multi-Domain Simulator for Marine Robotics

Cédric Buche, Juliette Grosset, Hélène Lechêne et al. (+4 more)

Simulation is essential for maritime robotics, supporting operator training, mission rehearsal, and human-vehicle interaction in environments where real-world testing is costly or hazardous. Existing simulators focus primarily on autonomy systems and often lack human-in-the-loop interaction and realistic environmental physics. This paper introduces LOTUSim, an open-source, real-time maritime simulator supporting multi-user interaction across aerial, surface, and underwater robotic systems for coordinated naval-style operations. The first contribution of this work is enabling real-time interactive performance for users while ensuring scalability to large fleets operating within a shared interactive simulation environment. Validation demonstrates robust human-in-the-loop performance, maintaining strict real-time execution and high visual fidelity while scaling to large heterogeneous maritime drone swarms. The second contribution is a computationally efficient, Ekman-inspired layered, underwater current model that captures wind-driven, depth-dependent flow dynamics with sufficient physical fidelity for large-scale simulations. Validation against ocean reanalysis data demonstrates substantially improved accuracy compared to commonly used stochastic Gauss-Markov current models. These results confirm LOTUSim's suitability as a simulation platform for operatorin-the-loop maritime robotics research.
Read the full abstract

All authors Cédric Buche, Juliette Grosset, Hélène Lechêne, Marie Dubromel, Pierig Havez-Bodivit, Malcom Neo, Julien Prodhon

Robotics

DRBA: Dynamic Robotic Balance Assistant – An assist-as-needed gait and balance rehabilitation robot for versatile training

Yifan Wang, Li Li, Youlong Wang et al. (+9 more)

The decline of human balance control due to aging and pathological conditions increases fall risk, a major concern in geriatric care and rehabilitation. Gait training is essential for balance recovery, enhancing walking ability and postural control. However, existing overground robotic gait trainers have limitations: body weight support systems are bulky and impractical for daily use, while end-effector-based systems often compromise transparency, altering natural gait dynamics. This paper presents the Dynamic Robotic Balance Assistant (DRBA), a novel gait trainer providing assist-as-needed body weight and balance support for various training scenarios. DRBA integrates a 3-degree-of-freedom (3-DoF) robotic arm for pelvic support with flexible motion, a compact sit-to-stand assistance module, and user-following and fall detection algorithms to ensure minimal interference and responsive support. Experimental results demonstrated high transparency, with minimal impact on natural gait dynamics. A patient trial with nine elderly patients with varying medical conditions and balance impairments (ranging from severe to mild) further validated DRBA's effectiveness. The results showed that DRBA-assisted training increased step length and walking speed compared to therapist-assisted gait training. Additionally, DRBA enabled users to perform tasks beyond their unaided ability, expanding rehabilitation possibilities. These findings highlight DRBA's potential to enhance rehabilitation outcomes by facilitating higher training intensity and enabling task-oriented exercises.
Read the full abstract

All authors Yifan Wang, Li Li, Youlong Wang, Chengyuan Yang, Sherwin Stephen Chan, Jiaye Chen, Xiaoyue Yan, Hao Wang, Xuesheng Gong, Jun Lin, Hongping Hu, Wei Tech Ang

Robotics

Beyond Heuristics: A Standardized Real2Sim Pipeline for Physical Human Robot Interaction in Human-in-the-Loop Simulation

Chengyuan Yang, Yifan Wang, Chun Kwang Tan et al. (+5 more)

The aging global population drives demand for assistive robots, yet the safety risks and costs of physical testing make Human-in-the-Loop (HITL) simulation an attractive alternative. Its fidelity for coupled systems, however, is limited by interaction models whose impedance parameters are tuned heuristically rather than identified from data. We present a Real2Sim pipeline that identifies the coupled Physical Human-Robot Interaction (pHRI) dynamics of a pelvis–strap interface on an overground mobile balance assistant. The interface is modeled as a 6-DoF viscoelastic mechanism whose 12 directional stiffness and damping parameters are identified per subject via Covariance Matrix Adaptation Evolution Strategy (CMA-ES), using the user's “Safe & Comfortable” feedback as a reproducible operating point that resolves harness-tightness ambiguity across anthropometrics. An intraclass-correlation analysis over a five-subject cohort separates shareable from subject-specific parameters, yielding a set of prior parameters derived from the existing data. Deploying this prior configures a previously unseen subject by refining only 5 of the 12 parameters. The calibrated model then reproduces the real interaction envelope and induces biomechanically accurate gait adaptations in the Human Digital Twin (HDT). Overly compliant and overly stiff settings, by contrast, fail as extreme settings, confirming a correct operating point that no heuristic tuning procedure can reliably select. The pipeline thus improves HITL simulation fidelity and supports the Human Digital Twin as a predictive tool for pre-clinical verification of personalized controllers.
Read the full abstract

All authors Chengyuan Yang, Yifan Wang, Chun Kwang Tan, Sherwin Stephen Chan, Youlong Wang, Xiaoyue Yan, Lei Li, Wei Tech Ang

Robotics

Function-Space Diffusion for Motion Planning

Zinuo Chang, Yipu Chen, Byoungwoo Park et al. (+2 more)

Diffusion-based motion planners have demonstrated strong performance in generating diverse and high-quality robot trajectories in cluttered environments with multiple feasible solutions. However, existing approaches typically operate on fixed-length waypoint sequences, making the learned model resolution-dependent, thereby preventing zero-shot generalization across resolutions. In this work, we propose Function-Space Diffusion for Motion Planning (FSD-MP), a diffusion-based motion planner that models trajectories as continuous functions and performs diffusion directly in function space, achieving discretization-invariant trajectory generation. We define a mode-wise forward process in the spectral domain, driven by Gaussian noise with a Matérn-type covariance, and parameterize the reverse process with a boundary-compatible Discrete Sine Transform-based Fourier Neural Operator (DST-FNO) that preserves start-goal constraints across resolutions. We evaluate FSD-MP on 2D point robot and 7-DoF Franka manipulator planning benchmarks. Our method achieves competitive planning performance at the training resolution and generalizes zero-shot across resolutions up to 16$\times$ higher, preserving consistent planning behavior without retraining. These results demonstrate that function-space diffusion provides an effective framework for discretization-invariant motion planning.
Read the full abstract

All authors Zinuo Chang, Yipu Chen, Byoungwoo Park, Hongzhe Yu, Yongxin Chen

Robotics

Longitudinal-Motion-Aware Lateral Control for Autonomous Vehicles: A Robust Nonlinear Control Framework

Sixu Li, Nitesh Kumar, Reyshwanth Ganeshan et al. (+3 more)

As autonomous vehicles (AVs) operate in increasingly dynamic traffic conditions, lateral control must be performed while longitudinal speed and acceleration vary. Yet many existing lateral controllers rely on constant-speed or operating-point-based assumptions, which can degrade performance during transient longitudinal maneuvers. Moreover, most methods assume precisely known vehicle parameters, despite real-world parametric uncertainties. To address these limitations, this paper presents a longitudinal-motion-aware robust nonlinear lateral control framework for AVs. It first derives a tracking error model that depends on varying longitudinal speed and acceleration. Using this model, feedback linearization is employed to obtain a linear input-output relation for lateral error tracking while embedding longitudinal motion into the control law. The resulting internal dynamics are then analyzed to ensure overall system stability. To address parameter uncertainty, two robust control designs with distinct implementation trade-offs are proposed: (i) a Lyapunov redesign (LR) approach inspired by sliding mode control, and (ii) an incremental nonlinear dynamic inversion (INDI) method. Both are rigorously analyzed and proven to ensure ultimate boundedness, with key robustness-tuning parameters explicitly identified. Simulations demonstrate enhanced tracking accuracy, consistent performance across varying speeds and accelerations, and robustness to model uncertainties, while also examining the effects of the robustness-related parameters. Real-vehicle tests further confirm real-time implementation and practical path-tracking performance on actual hardware.
Read the full abstract

All authors Sixu Li, Nitesh Kumar, Reyshwanth Ganeshan, Sivakumar Rathinam, Swaroop Darbha, Yang Zhou

Robotics

Continuous-Time Gaussian Belief Trees for Motion Planning

Rayan Mazouz, Qi Heng Ho, Zachary N. Sunberg et al. (+1 more)

We address sampling-based motion planning for continuous-time stochastic systems under process and measurement uncertainty, with probabilistic guarantees on safety and performance. The robot dynamics are modeled as a continuous-time linear stochastic differential equation, while sensor measurements arrive at discrete time instants. We derive an offline hybrid belief propagation model in which the belief evolves according to continuous-time ODEs between measurements and undergoes discrete Kalman filter update jumps at measurement times. To ensure safety, we introduce a belief-barrier-function-based safety checker for segment-level probabilistic verification. This enables the planner to certify safety over entire continuous trajectory segments and detect inter-sample chance-constraint violations that are missed by conventional node-based checks. Together, these components provide a principled framework for sampling-based belief planning that accounts for both continuous-time uncertainty propagation and continuous-time safety requirements. We integrate the method with RRT and SST planners and evaluate it across multiple benchmark environments. The results show that the proposed method achieves high success rates and robust enforcement of chance constraints, including in narrow-passage scenarios where discrete-time counterparts fail due to missed inter-sample unsafe behavior.
Read the full abstract

All authors Rayan Mazouz, Qi Heng Ho, Zachary N. Sunberg, Morteza Lahijanian

Robotics · Cross-listed from cs.CV

E-TraMamba: A New Paradigm for Efficient Long-Term 3D Feature Tracking with Event Cameras

Juwei Shen, Yujie Wu and Changwen Chen

Event-based 3D tracking enables low-latency and high-speed perception, while existing CNN- and Transformer-based trackers struggle to capture long-range spatiotemporal dependencies in sparse, noisy event streams, especially under real-time and efficiency constraints. To address these challenges, we present E-TraMamba, the first Mamba-based framework for 3D feature tracking on event data. This new framework adopts a linear state-space model for efficient long-range modeling and integrates a lightweight affine-transform predictor to maintain stable tracking under motion blur and occlusion. We also design an effective scheme to fuse multi-scale cues – local spatiotemporal patches, correlation maps, and positional embeddings – into a unified representation that enables stable and smooth 3D tracking. We construct a large-scale synthetic dataset, named EvD-PointOdyssey, which is generated with monocular rendering and provides synchronized event streams, depth maps, and accurate 3D trajectories for training and evaluating event-based 3D tracking models. Extensive experiments on event-based benchmarks demonstrate that E-TraMamba achieves state-of-the-art performance, delivering over $2\times$ longer feature lifetimes under strict accuracy thresholds (e.g., 0.1 m), with higher tracked-feature ratios and lower RMSE than all baselines. These results make E-TraMamba a strong candidate for low-latency visual odometry, real-time SLAM, and interactive robotics.
Read the full abstract

Robotics

DREAMSTEER: Latent World Models Can Steer VLA Policies During Deployment Without Any Finetuning

Hanchen Cui, Sergio Arnaud, Arjun Majumdar et al. (+5 more)

Pretrained vision-language-action (VLA) policies show promising zero-shot generalization, but often fail under deployment-time distribution shift, leading to decreased robustness and inconsistent instruction following. While prior work commonly tackles this by finetuning on in-distribution data, it assumes demonstrations collected on tasks in the target environment. In this work, we propose DREAMSTEER, a deployment-time steering framework for pretrained VLAs without any finetuning or parameter modifications. The key insight in DREAMSTEER is to leverage a latent world model and a value model to steer pretrained VLA policies. During deployment, DREAMSTEER samples candidate action chunks from a VLA policy and predefined motion primitives, imagines their outcomes using an action-conditioned latent world model, and ranks the imagined trajectories with a language-conditioned value model. Across four real-world manipulation benchmarks with unseen objects, DREAMSTEER improves task success rate from 23.75% to 66.25% and instruction-following accuracy from 38.75% to 56.25% over the base VLA policy.
Read the full abstract

All authors Hanchen Cui, Sergio Arnaud, Arjun Majumdar, Daniel Dugas, Elie Aljalbout, Karthik Desingh, Krishna Murthy Jatavallabhula, Franziska Meier

Robotics

Differential Amplifier-Inspired AmpAttention for Multi-View Robotic Manipulation

Jin Yang, Ping Wei and Nanning Zheng

Multi-view robotic manipulation methods with the attention mechanism have recently achieved significant progress in both training efficiency and task performance. However, the inherent redundancy, occlusion, and viewpoint dependency in robotic view images often lead to severe attention drift. To address this challenge, we propose AmpAttention, a novel attention mechanism inspired by differential amplifiers in analog circuits. It aims to suppress attention noise and capture high signal-to-noise ratio signals for more reliable perception. Based on this, we introduce the RVAF model, which integrates task-guided intra-view and inter-view AmpAttention. Compared to previous state-of-the-art methods, RVAF achieves the optimal average success rate across 18 RLBench tasks (249 variations) while reducing training time by 33.3%. RVAF also demonstrates strong potential in real-world high-precision tasks, exemplified by its ability to pick up a dart and accurately insert it into the red bullseye. Furthermore, we extend RVAF to RVAF++ by incorporating the SAM2 image encoder. RVAF++ achieves substantial gains on high-precision tasks, achieving a 91% success rate on the `insert peg' task. More qualitative results are provided at the anonymous project website https://anonymous.4open.science/w/RVAF-Anonymization.
Read the full abstract

Robotics

CLEAR: Closed-Loop Reinforcement Learning at Scale for End-to-End Autonomous Driving

Yunxiao Shi, Hong Cai, Mohammad Ghavamzadeh et al. (+1 more)

End-to-end autonomous driving (E2E-AD) aims to directly map raw sensor information to driving actions. Recently, with the rapid advancement of multi-modal large language models (MLLMs), researchers have proposed the paradigm of Vision-Language-Action (VLA) models for E2E-AD, where it seeks to integrate visual perception, language understanding and action prediction within a single policy. However, existing VLA-based policies largely adopts imitation learning, where it only learns to drive by optimizing distance-based metrics w.r.t. logged expert trajectories. Such distribution shift between open-loop training and closed-loop inference leads to suboptimal performance in closed-loop planning. To close this gap, we present CLEAR, a system that enables closed-loop training using Reinforcement Learning (RL) at scale for E2E-AD. We propose to learn a novel residual waypoint policy around the waypoint prior from pretrained VLA policies, effectively harnessing the knowledge within. On another front, one of the key challenges to scale up RL for vision-based policies is the number of parallel simulation environments since RL is data hungry. To that end, we design a heterogeneous pipeline that places the simulator and the VLA learner on distinct compute groups, which allows us to dramatically increase the number of simulation environments running in parallel while avoiding resource contention and maintaining training stability. We show that with a simple reward, CLEAR significantly outperforms previous methods and sets new state-of-the-art performance on the challenging benchmarks of CARLA longest6 v2 and Bench2Drive.
Read the full abstract

All authors Yunxiao Shi, Hong Cai, Mohammad Ghavamzadeh, Fatih Porikli

Robotics

TACO: TActile World Model as a Self-COrrector for Scalable Robot Policy Post-Training

Shengbang Liu, Yueru Jia, Yuyang Yan et al. (+7 more)

Vision-Language-Action models and World Action Models have shown promising generalization in robotic manipulation but remain fragile in contact-rich tasks, where contact perturbations can cause failures that are difficult to detect from vision alone. Corrective post-training with tactile feedback can improve recovery, but scaling such supervision through human intervention is costly. World models can synthesize additional training data, yet vision-only generation may produce visually plausible but contact-inconsistent trajectories. We therefore introduce TACO, a scalable robot policy post-training framework built on a compositional tactile world model. Given real rollouts, TACO follows a Recognize–Imagine–Label loop: an inverse dynamics and value model identifies failure-adjacent states using progress estimates, a visuo-tactile generation model imagines local corrections by jointly generating video and tactile sequences, and the inverse dynamics and value model labels them with corrective actions and progress scores. Candidates are filtered for kinematic feasibility and tactile plausibility, then selected by predicted progress gain. TACO aggregates demonstrations, real rollouts, and selected corrections for iterative post-training. It combines knowledge-insulated tactile adaptation with CFG-RL using binary advantage labels while keeping the pretrained VLM backbone fixed. Experiments on real-world tasks show that TACO improves the average task score from 0.375 to 0.825 after two post-training iterations.
Read the full abstract

All authors Shengbang Liu, Yueru Jia, Yuyang Yan, Jiaming Liu, Xinran Zhang, Qiuxuan Feng, Yandong Guo, Shiji Zhou, Boxin Shi, Shanghang Zhang

Thursday 2 July

Robotics

HERMES: Heterogeneous Edge-Relational Multi-Head Embedded SSM Attention for Traffic Conflict Prediction at Signalized Intersections

Md Monzurul Islam and Subasish Das

Surrogate safety measures (SSMs) enable proactive traffic safety assessment, but many existing methods evaluate pairwise interactions independently or flatten multi-agent scenes into fixed feature vectors, limiting their ability to represent heterogeneous interaction structure and evolving scene-level risk. This study formulates traffic conflict assessment as temporal heterogeneous scene-graph classification and proposes HERMES, a heterogeneous edge-relational graph neural network with SSM-informed multi-head attention. Vehicles and pedestrians are represented as heterogeneous nodes, while vehicle-vehicle, vehicle-pedestrian, and pedestrian-pedestrian interactions are encoded as relation-specific edges with continuous kinematic and surrogate-safety descriptors. Relation-specific attention, dynamic node-edge updates, safety-aware graph pooling, and temporal sequence learning are jointly used to estimate scene-level conflict probability. HERMES was evaluated using 109,028 trajectory-derived sequences from a signalized urban intersection and tested on an independently collected comparable intersection dataset. Enhanced HERMES achieved an AUC-ROC of 0.9898 +/- 0.0013, an AUC-PR of 0.9412 +/- 0.0067, and an F1 score of 0.8449 +/- 0.0103. At a 5% false-alarm rate, it detected 95.7% of conflict sequences, outperforming the strongest Transformer baseline and XGBoost. In zero-shot external evaluation, HERMES achieved an AUC-ROC of 0.9752 and an AUC-PR of 0.7829. Joint source-target training further improved target-site performance with limited target-site data. These findings show that preserving heterogeneous interaction topology, safety-informed edge semantics, and short-term temporal evolution improves scene-level conflict classification and supports transferable roadside safety monitoring at signalized intersections.
Read the full abstract

Robotics

Saturation-Aware Robust Trajectory Optimization for Reusable Launch Vehicles via Differentiable Physics

Liwei Chen and Tong Qin

The high-angle-of-attack flip maneuver of reusable launch vehicles presents significant challenges for robust trajectory optimization due to the combined effects of highly nonlinear dynamics, aerodynamic uncertainties, and actuator saturation. This paper presents a differentiable physics framework for saturation-aware robust trajectory optimization. At its core, a Differentiable Particle Tube Control (DPTC) scheme is developed to optimize uncertainty evolution through an ensemble-based distribution shaping strategy. State uncertainty is represented by a Lagrangian particle ensemble, while hard actuator projection operators are embedded directly into the computational graph, enabling the joint optimization of the nominal feedforward trajectory and a time-varying feedback policy via end-to-end backpropagation. The proposed framework is evaluated against an automatic differentiation-based Successive Convexification (AD-SCvx) baseline combined with a conventional covariance steering feedback strategy. Six-degree-of-freedom Monte Carlo simulations demonstrate that, although the baseline achieves nominal fuel-optimal solutions, its unconstrained feedback formulation becomes susceptible to actuator saturation under aerodynamic disturbances, leading to degraded closed-loop robustness. In contrast, the proposed DPTC framework proactively performs a constraint-aware performance trade-off by relaxing spatial tracking to preserve critical control authority. These results demonstrate that integrating differentiable physics with ensemble-based optimization provides an effective and practical framework for robust guidance in highly constrained aerospace flight systems.
Read the full abstract

Robotics · Cross-listed from cs.HC

The Individual-Targeting Assumption: A Systematic Review of Proactive Robots in Human Group Settings

Tauhid Tanjim, Tasmia Mayen, Malte F. Jung et al. (+1 more)

Proactive robots are increasingly deployed in public environments where people are encountered not as isolated individuals but as members of cohesive social groups. Yet whether the prevailing design paradigm in proactive human-robot interaction (HRI) accounts for the relational structure that defines a group as a social unit remains largely unexamined. Through a systematic review of 63 proactive HRI studies in group settings from 2000 to 2025, we identify a recurring tendency, the Individual-Targeting Assumption (ITA), in which robots treat co-present people as independent engagement targets. We find that ITA is present in 60.3% of the corpus, with group-aware approaches emerging almost entirely after the robot is already embedded in an ongoing interaction. Critically, how a robot should detect and negotiate entry into a pre-formed group before initiating contact remains unaddressed across the corpus. Three failure modes, engagement misdetection, social ratification blindness, and bystander neglect, emerge as recurring patterns when proactive robots interact with human groups. These findings reframe proactive HRI in group settings not as an extension of dyadic interaction but as a qualitatively distinct design problem, and they identify the robot's approach to the group during the entry phase as a critical and understudied open challenge.
Read the full abstract

All authors Tauhid Tanjim, Tasmia Mayen, Malte F. Jung, Susan R. Fussell

Robotics

Metallic Ultrasound Waveguides as a Distributed Tactile Sensing Platform

Alexandros Rosakis, Alessio Tamborini, Basile Fakhoury et al. (+2 more)

Tactile sensing is central to how robotic systems interact with the real world, yet current solutions face a tradeoff between sensing area and system complexity. This work investigates metallic ultrasound waveguides as distributed tactile sensors fully interrogated from a single proximal transducer. Using cylindrical indenters, we characterized the acoustic response to single and multi-point contacts with varying forces and contact materials. For single point indentation, the applied force was well captured by a linear relationship with the ratio of the reflection to transmission coefficients (F = a * R/T) across all nine tested materials (R2 >= 0.95). The calibration slope, a, correlated strongly with the material's effective contact modulus (log–log Pearson r=-0.98). The reflected energy partition was found to be a load-independent parameter related to the contacting material's properties, enabling material class differentiation independent of force. For the two-indenter experiment, both contact forces were recovered from the waveguide signal and were in close agreement with reference load cell measurements (contact 1, R2 = 0.97; contact 2, R2=0.95). The approach was extended to two-dimensional metallic sheets, confirming both contact localization and material-dependent effects. Overall, these results validate metallic waveguides as a robust platform for distributed tactile sensing, providing contact localization, force estimation, and material-class discrimination for the contacting body.
Read the full abstract

All authors Alexandros Rosakis, Alessio Tamborini, Basile Fakhoury, Cole Bailey, Morteza Gharib

Robotics

MorphQuad: Morphable Quadrotor for Superhuman Maneuverability, Manipulation, and Resiliency

Jose Diaz Peon Gonzalez Pacheco, Jiawei Xu, Andrew Zhao et al. (+7 more)

Infrastructure maintenance, contact-based inspection, and emergency response can benefit from aerial vehicles that act as a flying human hand with extreme maneuverability, manipulation, and resiliency (MMR): maneuverability to fly in arbitrary orientations to reach remote and tight locations; manipulation to point sensors, turn valves, and press tools at arbitrary orientations; resiliency to maintain accurate motion and force control despite disturbances from arbitrary directions, such as wind, ground effects, and friction. Realizing MMR on aerial vehicles requires not only omnidirectional flight; it also requires (I) vectoring of maximum thrust in any direction, to maximize capacity for contact-force application and disturbance rejection, (II) global stability, to enable control over any orientation/position, and (III) compact, standard designs that build upon platforms such as quadrotors to inherit technological know-how. No current aerial vehicle simultaneously enables I–III, due to structural and control limitations that constrain actuation. We present MorphQuad: a morphable quadrotor that enjoys MMR. Key to our approach is a hardware and control co-design: on hardware, we independently articulate each of the four rotor systems via two-axis gimbals; on control, we introduce globally-stable control, and energy-optimal thrust allocation that permits inter-rotor thrust cancellations only to avoid downwash interference and gimbal lock. With fully-onboard autonomy, MorphQuad demonstrates multi-revolution rotation while translating or hovering, for pipe inspection and target tracking (maneuverability); valve turning, perching, and object pressing and pushing with human-level strengths (manipulation); and wind rejection from any direction, even directed to a single rotor, and push-pull recovery (resiliency).
Read the full abstract

All authors Jose Diaz Peon Gonzalez Pacheco, Jiawei Xu, Andrew Zhao, Hongyu Zhou, Atharva Navsalkar, Andrew Scheffer, Amrith Malli Reddi, Sashreek Shankar, Yuqing Bao, Vasileios Tzoumas

Robotics · Cross-listed from cs.NE

A Spiking Sequence Generator for Polar Trajectories on Neuromorphic Hardware

William R.P. Nourse and Roger D. Quinn

Neuromorphic controllers for size, weight, and power-constrained systems require neural architectures that are both energy-efficient and interpretable at the level of system dynamics. However, existing approaches either rely on end-to-end trained spiking networks with limited interpretability, or on converted classical controllers that fail to fully exploit neuromorphic dynamics. We present a spiking neural network (SNN) architecture for generating polar trajectories, using a winner-take-all (WTA) architecture with accessory populations that induce controlled transitions in neural activity. We demonstrate tuning rules for these population dynamics, and utilize a form of shunting inhibition to enable independent control of direction, speed, and radius of the resulting polar trajectories. We implement the network on the SpiNNaker2 neuromorphic processor, and demonstrate a two to three orders of magnitude reduction in wall-clock step time and three to four orders of magnitude reduction in energy expenditure when compared to conventional computing platforms.
Read the full abstract

Robotics

EVA-Client: A Unified Data Collection, Inference, and Deployment Framework for Embodied Policies on Real Robots

Heqing Yang, Yang Yi, Liyao Wang et al. (+8 more)

We present EVA-Client, an open-source framework for deployment, data collection, and evaluation of trained manipulation policies on real robots. Sitting between a policy server and the physical hardware, EVA-Client unifies the real-robot stages of the policy iteration loop within a single codebase. It makes three contributions. First, a component-decoupled architecture in which robot backends, inference strategies, and transport middlewares form an orthogonal grid: adding a robot or a strategy touches only its own layer. Second, inspectable execution through Debug, Collect, and Eval workflows, with modes ranging from open-loop simulation to continuous real-time control. Third, every evaluation run doubles as a data collection, recording full rollouts in training-ready format alongside exhaustive logs and a side-by-side comparison viewer, so each evaluation feeds the next round of training rather than ending as an unrecorded impression. EVA-Client further consolidates major real-time inference strategies, synchronous and asynchronous execution, ACT-style temporal ensembling, Real-Time Chunking, and a naive-async ablation baseline, behind a single configuration surface.
Read the full abstract

All authors Heqing Yang, Yang Yi, Liyao Wang, Linqing Zhong, Donglin Yang, Ruipu Wu, Zitong Bai, Fengjiao Chen, Manyuan Zhang, Linjiang Huang, Si Liu

Robotics

GigaWorld-1: A Roadmap to Build World Models for Robot Policy Evaluation

GigaWorld Team, Angyuan Ma, Boyuan Wang et al. (+24 more)

Evaluating embodied robot foundation models remains a critical bottleneck; unlike large language models efficiently assessed via digital benchmarks, robotic policies require slow, costly real-world rollouts limited by hardware and human supervision, which has driven interest in world models as surrogate policy evaluators, yet the key properties that make a world model reliable for policy assessment remain poorly understood. This work presents a systematic study of world models for robotic policy evaluation and introduces WMBench, a benchmark constructed from real-robot teleoperation data and matched policy rollouts covering diverse manipulation tasks to enable controlled comparisons across model families, action encodings, rollout horizons, and evaluation metrics. Using WMBench, we analyze 7 video world models, 4 action representation schemes, and over 324,000 simulated policy rollouts paired with real robot executions, further enriching our analysis with large-scale community submissions from the CVPR 2026 GigaBrain Challenge, curated synthetic trajectories, and a training videos spanning more than 12,000 hours. Our experiments deliver three core insights: evaluator quality is dominated by long-horizon, action-faithful rollout consistency rather than short-term visual realism; pretraining gains stem not only from data scale but from balancing general world knowledge with robot-specific controllability; and architectural choices including action encoding, memory design, and evaluator-focused post-training strongly determine alignment with real-world robot behavior. Drawing on these results, we derive a practical design roadmap and realize it in GigaWorld-1, a world model specially optimized for policy evaluation, and we fully release our code, models, datasets, and toolkits to advance scalable evaluation research for embodied foundation models.
Read the full abstract

All authors GigaWorld Team, Angyuan Ma, Boyuan Wang, Bohan Li, Chaojun Ni, Guo Li, Guan Huang, Guosheng Zhao, Hao Li, Hengtao Li, Jingyu Liu, Jiwen Lu, Qiuping Deng, Tingdong Yu, Xuancheng Xu, Xinyu Zhou, Xiuwei Xu, Xinze Chen, Xiaofeng Wang, Xiaoyu Tian, Yang Wang, Yifan Chang, Yukun Zhou, Yun Ye, Zhenyu Wu, Zhanqian Wu, Zheng Zhu

Robotics

VT-WAM: Visual-Tactile World Action Model for Contact-Rich Manipulation

Shuai Tian, Yupeng Zheng, Yuhang Zheng et al. (+7 more)

Contact-rich manipulation requires policies to react to local deformation, pressure, slip, and friction, yet these cues are temporally sparse and often invisible in visual observations. Existing visual-tactile policies usually feed tactile observations directly into action prediction, but rarely model tactile deformation dynamics during action generation. In this paper, we introduce VT-WAM, a Visual-Tactile World Action Model that jointly learns future visual prediction, tactile deformation prediction, and action prediction within a unified flow matching framework. In particular, VT-WAM introduces (1) Asymmetric Mixture-of-Transformers (MoT) attention to bridge a first-frame visual anchor with temporal tactile dynamics, and (2) contact-gated Action-Visual-Tactile Attention Guidance (AVTAG) to encourage action queries to rely on tactile evidence during contact phases. Across six real-world contact-rich manipulation tasks, VT-WAM achieves a 71.67% average success rate, outperforming Fast-WAM by 26.67% and OmniVTLA by 35.84%. Ablations demonstrate that modeling tactile deformation dynamics and guiding contact-phase tactile attention are both important for contact-rich tasks. Project website: https://vt-wam.github.io/.
Read the full abstract

All authors Shuai Tian, Yupeng Zheng, Yuhang Zheng, Songen Gu, Yujie Zang, Yuxing Qin, Weize Li, Haoran Li, Wenchao Ding, Dongbin Zhao

Robotics

Embodied.cpp: A Portable Inference Runtime of Embodied AI Models on Heterogeneous Robots

Ling Xu, Borui Li, Hao Wu et al. (+8 more)

Embodied AI models now span vision-language-action (VLA) models and world-action models (WAMs), but practical deployment remains fragmented across model-specific Python stacks, backend assumptions, and robot-side glue code, especially on heterogeneous edge devices. Existing inference runtimes are designed mainly for request-response serving and therefore do not satisfy the runtime contract of embodied deployment: multi-rate execution inside closed-loop control, latency-first batch-1 inference on heterogeneous hardware, and extensible embodied interfaces beyond fixed token I/O. We present Embodied$.$cpp, a portable C++ inference runtime for embodied models. Based on an architectural analysis of representative VLA models and WAMs, Embodied$.$cpp captures a shared execution path and organizes it into five layers: input adapters, sequence builders, backbone execution, head plugins, and deployment adapters. The runtime provides modular multi-rate execution, latency-first fused inference, and extensible operator and I/O support, enabling deployment across heterogeneous devices, robots, and simulators through one backend abstraction. We evaluate Embodied$.$cpp on three VLA and two WAM models, using normalized comparisons across Python and C++ quantization configurations. Overall, Embodied$.$cpp achieves 1.05x-2.70x inference speedups and 7%-77% lower VRAM relative to Python baselines, while maintaining near-baseline success for most configurations. These results show that Embodied$.$cpp improves deployment efficiency while preserving high control quality across diverse embodied model architectures. Project Link: https://github.com/SEU-PAISys/Embodied.cpp
Read the full abstract

All authors Ling Xu, Borui Li, Hao Wu, Chuyu Han, Xiangyu Li, Mohan Hua, Shiqi Jiang, Ting Cao, Chuanyou Li, Sheng Zhong, Shuai Wang

Robotics

Controllable Sim Agents with Behavior Latents

Juanwu Lu, Junyu Zhu and Ziran Wang

Realistic traffic simulation requires agents that imitate logged behavior and can also be steered along interpretable axes. Such controllability enables engineers to isolate variables, reproduce specific edge cases, and test autonomous systems without real-world risk. We introduce Controllable Neural Variational Agents (CNeVA), a controllable simulated-agent framework that learns to infer a per-agent Gaussian behavior latent from per-channel discounted returns via a closed-form conjugate variational update, conditioning a rectified-flow trajectory generator trained on a mixed channel-mask curriculum for classifier-free guidance. To tackle scarcity in reward signals, we propose soft eligibility gates that replace hard binary thresholds with smooth exponential decay, preserving the gradient signal for near-threshold agents. On the Waymo Open Motion Dataset, CNeVA attains competitive realism on the benchmark while exposing per-channel controllability that the higher-ranked imitation models lack. Speed- and acceleration-based steering produces monotone responses without stall-induced reward hacking. Safety controllability is monotone and substantial with the introduction of soft eligibility. We manage to achieve steerable map compliance under a context-residual return measure. Furthermore, our experiment demonstrates that steering metrics must be read alongside physical-plausibility guardrails to avoid reward-hacking confounds.
Read the full abstract

Robotics

QuadRocket: An Aerial Robotic Testbed for Adaptive Thrust-Vector Control of Rocket-Like Vehicles

Pedro Santos, Joel Reis, Paulo Oliveira et al. (+1 more)

This paper presents QuadRocket, a quadrotor-based rocket prototype that provides a low-cost, low-risk platform for validating advanced thrust-vector control strategies for launch vehicle-type systems. The prototype consists of a cylindrical main body mounted on top of a quadrotor through a universal joint, forming a flying inverted pendulum with non-negligible inertia. For control design, the coupled system is modeled as a single axisymmetric rigid body actuated by a vectored force applied along its longitudinal axis. A reduced-attitude representation on the two sphere is adopted to explicitly exploit the vehicle's axial symmetry and to decouple yaw from the thrust-vector direction. On this model, we derive an adaptive backstepping controller that achieves almost global trajectory tracking in the presence of unknown constant disturbances, while a control-point transformation mitigates non minimum-phase behavior. The quadrotor is then treated as a thrust vector actuator, and a dynamic-surface-based attitude controller is designed to track the desired thrust-vector, accounting for actuation dynamics and avoiding explicit differentiation of virtual control signals. The complete architecture is evaluated in simulation and validated experimentally in an indoor motion-capture arena. Results demonstrate accurate trajectory tracking, effective disturbance compensation, and confirm the suitability of the QuadRocket as a versatile testbed for thrust-vector-controlled robotic vehicles.
Read the full abstract

All authors Pedro Santos, Joel Reis, Paulo Oliveira, Carlos Silvestre

Robotics

Learning Agile Intruder Interception using Differentiable Quadrotor Dynamics

Michael Anoruo, Xiaoyu Tian, Abhishek Rathod et al. (+5 more)

This paper presents a methodology for learning a control policy to intercept an intruder using the 3D direction unit vector to the intruder and the interceptor state. Prior deep reinforcement learning approaches assume either relative position or distance to the intruder is available, but this information is not readily accessible in real-world applications that employ passive, monocular camera sensors. Instead, we propose a solution that leverages an analytical policy gradient method using differentiable quadrotor dynamics to learn agile interception at speeds up to 10 m/s. The proposed approach outperforms baseline methods that utilize simplified point mass dynamics by an average of 30%.
Read the full abstract

All authors Michael Anoruo, Xiaoyu Tian, Abhishek Rathod, Timothy Naudet, Thomas Canchola, Eric Sturzinger, Kshitij Goel, Wennie Tabib

Robotics

Learning to Move Before Learning to Do: Task-Agnostic pretraining for VLAs

Junhao Shi, Siyin Wang, Xiaopeng Yu et al. (+3 more)

Vision-Language-Action (VLA) models are fundamentally bottlenecked by the scarcity of expert demonstrations – triplets of observations, instructions, and actions that are costly to collect at scale. We argue that this bottleneck stems from conflating two distinct learning objectives: acquiring physical competence (how to move) and acquiring semantic alignment (what to do). Crucially, only the latter requires language supervision. Building on this Decomposition Hypothesis, we propose Task-Agnostic Pretraining (TAP), a two-stage framework that first learns transferable motor priors from cheap, unlabeled interaction data – including discarded off-task trajectories and autonomous robot play – via a self-supervised Inverse Dynamics objective. A lightweight second stage then grounds these priors in language using minimal expert data. On the SIMPLER benchmark, TAP matches models trained on over 1M expert trajectories while using orders of magnitude less labeled data, yielding a 10% absolute gain over standard behavior cloning. On a real-world WidowX platform, TAP retains 25% success under camera perturbations where internet-scale baselines collapse to 0%, demonstrating that task-agnostic pretraining produces robust, transferable physical representations and offers a scalable path forward for Embodied AI.
Read the full abstract

All authors Junhao Shi, Siyin Wang, Xiaopeng Yu, Li Ji, Jingjing Gong, Xipeng Qiu

Robotics

WorldSample: Closed-loop Real-robot RL with World Modelling

Yuquan Xue, Le Xu, Zeyi Liu et al. (+5 more)

Reinforcement learning (RL) can overcome the demonstration-coverage limitation of imitation learning (IL) by allowing robots to improve through trial-and-error interaction beyond the states observed in demonstrations. However, deploying RL on real robots remains constrained by high interaction costs, since each physical rollout is costly and reflects only one realized action-outcome path. To address this challenge, we propose WorldSample, a physically grounded data augmentation framework for real-robot RL that closes a real-synthetic loop between physical rollouts, world-model generation, and policy improvement. Grounded on real rollouts, WorldSample generates high-fidelity synthetic transitions through a post-trained world model, which greatly lowers the visual hallucination. Specifically, rather than simply using these transitions as real-world experience, WorldSample introduces Policy-Paced Learning (PPL) to regulate the training process through sample selection and scheduling, balancing useful augmentation against value overestimation and mitigating the hallucination-induced noise. Experiments on robot manipulation tasks involving contact-rich and precise tasks show that WorldSample improves policy success rate by 28% while reducing training steps by 59% compared with baselines. Furthermore, WorldSample improves world model visual fidelity by 19.4dB in PSNR and 0.47 in SSIM over demonstration-only post-training, validating the effectiveness of the real-synthetic loop for both policy and world model performance.
Read the full abstract

All authors Yuquan Xue, Le Xu, Zeyi Liu, Zhenyu Wu, Zhengyi Gu, Xinyang Song, Bofang Jia, Ziwei Wang

Robotics

LIME: Learning Intent-aware Camera Motion from Egocentric Video

Boyang Sun, Jiajie Li, Yung-Hsu Yang et al. (+6 more)

Autonomous robots often need to move their camera before they can act: to inspect an object, reveal an occluded region, or obtain a view that responds to a user's intent. While vision-language navigation translates instructions to base motion and vision-language-action policies map instructions to manipulation actions, vision-language conditioned camera motion remains comparatively underexplored as a first-class action. We formulate vision-language conditioned camera motion generation: given a current RGB observation and a free-form natural-language intent, predict a relative target camera pose for the next observation. The challenge is that useful viewpoint changes depend on latent perceptual intent, ranging from coarse spatial moves to fine-grained inspection or occlusion revealing. To model this structure, we mine multi-intent camera-motion supervision from egocentric video via hindsight labeling, pairing each relative SE(3) target pose with plausible intents and observation-gain descriptions inferred from the reached view. We propose LIME, a vision-language camera-motion generator that combines an autoregressive observation-gain output with a continuous flow-matching pose head. This design lets the model jointly predict what the next view should reveal while representing multi-hypothesis target views. Across viewpoint-prediction experiments and downstream robotic tasks, we show that LIME learns intent-conditioned camera motion from passive egocentric video, turning ordinary human recordings into supervision for a reusable active-perception primitive that supports general-purpose viewpoint selection, manipulation, embodied question answering, and mid-distance navigation.
Read the full abstract

All authors Boyang Sun, Jiajie Li, Yung-Hsu Yang, Chenyangguang Zhang, Tim Engelbracht, Sunghwan Hong, Cesar Cadena, Marc Pollefeys, Hermann Blum

Robotics

ACID: Action Consistency via Inverse Dynamics for Planning with World Models

Gawon Seo, Dongwon Kim and Suha Kwak

Decision-time planning with action-conditioned world models has become a popular paradigm for embodied control. However, the standard planning cost judges a candidate solely by how close its predicted terminal state lies to the goal, leaving the realizability of the intermediate transitions unchecked–a predicted trajectory can look convincing while the environment rollout drifts away from it. In this paper, we propose ACID, a decision-time planning framework that introduces cycle action consistency: the action inferred backward from a predicted transition by an inverse dynamics model should recover the one that was conditioned on. We fold this per-step residual into the planning cost via a scale-invariant adaptive weight. Across four action-conditioned world models and eight tasks encompassing object manipulation and articulated control in simulation, visual navigation, and real-robot manipulation, ACID consistently improves planning and matches the baseline's accuracy with substantially less planning compute.
Read the full abstract