Papers

Motion planning for automated vehicles in interactive traffic has to simultaneously ensure safety, respect traffic rules, and maintain comfortable progress under multi-modal behavior uncertainty. Imposing all these requirements as hard constraints reduces feasibility, whereas treating them all as costs allows safety-critical requirements to be traded against progress. We propose a hybrid hard–soft Signal Temporal Logic (STL) MPC framework that optimizes a single shared ego trajectory against every retained behavior hypothesis: minimum-distance and speed predicates are imposed as explicit hard constraints, while soft STL robustness terms shape clearance, comfort, and interaction quality within the resulting feasible driving corridor. This retains robustness to discrete interaction modes without branch-specific contingent ego policies. We further identify substantial symbolic graph-construction overhead and introduce reachability-certified relevance screening and vectorized symbolic construction to reduce computational cost while preserving the feasible set and objective. Closed-loop Monte Carlo evaluation over six highway and intersection scenarios shows the highest strict success rate among the compared planners with no observed collisions or rule violations, at roughly a quarter of the planning latency of the unoptimized formulation. An ablation study separates the two layers: the hard constraints define the safety envelope, while the soft terms improve clearance and motion quality within it.

Collision-resilient airframes allow quadrotors to survive impacts, but do not ensure autonomous post-collision recovery. Innovation gating in stock onboard estimators often rejects valid external velocity measurements after high-speed impacts, compromising recovery. We present REFLEX, a collision-recovery flight mode that runs entirely on the flight controller, requiring neither contact sensors nor a companion computer. Its proprioceptive recurrent policy stabilizes the vehicle without position, velocity, heading, or altitude estimates. The translational estimate is reinitialized while REFLEX is active, and a trust gate checks estimator consistency before returning control to the stock autopilot. The policy is trained in simulation using proximal policy optimization, with episodes initialized from post-impact states measured in 47 real collisions. In hardware experiments, REFLEX recovered 19 of 22 wall impacts at 5.03 $\pm$ 0.09 m/s, compared with 1 of 14 for the matched stock autopilot. These included 15 consecutive recoveries in a single flight. A separate exploratory flight demonstrated two additional recoveries at 6.02 and 6.05 m/s within the same flight, exceeding the previously reported velocity for autonomous wall-collision recovery by 64%.

Behavioral foundation models (BFMs) provide humanoids with reusable repertoires of whole-body behaviors; in real-world humanoid control, the same behavioral intent may require fundamentally different motions under different terrain constraints. Covering such variation with terrain-matched demonstrations requires enumerating a combinatorial space of behaviors and terrain geometries. We introduce P-BFM, a perceptive behavioral foundation model that factorizes humanoid control into terrain-invariant intent and terrain-conditioned realization. A forward-backward representation maps motion references, target states, and reward functions into a shared intent space without terrain information, while a perceptive policy learns how to realize each intent under the surrounding terrain geometry. We further identify a complementary limitation of purely latent control: behavioral latents can preserve local motion patterns while losing precise global progress. We therefore introduce Explicit Root Instructions (ERI), which separates global transport (e.g., heading and planar velocity) from behavioral intent, while leaving local whole-body adaptation to the perceptive policy. This decomposition leads to a characteristic capability absent from conventional BFMs: one fixed intent induces different, terrain-appropriate behaviors, including changes in step height, stride, posture, foothold placement, and contacts, without terrain-specific demonstrations. Across challenging terrains, P-BFM improves motion fidelity, root-control precision, and perturbation recovery over strong baselines. A single frozen policy further supports motion tracking, goal reaching, reward inference, teleoperation, and terrain-aware recovery, and transfers zero-shot to a Unitree G1 using onboard depth sensing.

General-purpose multimodal foundation models are increasingly capable of directly controlling robots, yet improving their manipulation capabilities through autonomous interaction remains challenging. A key challenge to such self-improvement is learning from failures in complex manipulation tasks involving cross-stage dependencies, where an action may succeed locally but cause a later stage to fail. To address this challenge, we introduce ReTrace, a self-improvement framework for robotic manipulation through autonomous physical interaction and retrospective reasoning, without provided demonstrations or model-parameter updates. ReTrace retrospectively traces downstream failures to earlier decisions, proposes targeted corrections, and evaluates them through subsequent interactions. Across attempts, ReTrace progressively consolidates validated corrections into a reusable manipulation skill that guide subsequent execution. We evaluate ReTrace on four real-world manipulation tasks using a Franka Research 3 robot. On the two tasks with strong cross-stage dependencies, GPT-6 Astra-low improves from 0% to 80% success on plug insertion and from 0% to 100% on ball pouring, while GPT-6 Sol-xhigh improves from 0% to 10% and 0% to 90%, respectively. The learned skills also transfer across model and reasoning configurations and reduce inference-time decision cost in most evaluated settings.

Flow matching is a common generative recipe for robot foundation models. Vision-language-action models (VLAs) generate action chunks, while world-action models (WAMs) additionally generate future visual states. Both start generation from a fixed Gaussian prior, despite temporal continuity between past and future states and actions. This requires costly training and iterative sampling, especially for WAMs with an additional visual branch. We introduce $\Delta$-flow, which constructs the source from recent history with mild noise rather than from a fixed Gaussian prior. This moves the source closer to the target, reducing transport to a small residual $\Delta$ that can be learned efficiently and enables single-step inference. We integrate $\Delta$-flow into the VLA policy $\pi_{0.5}$ and the WAM policies ImageWAM and Fast-WAM. On LIBERO, ImageWAM reaches 90% success in one epoch rather than six, and Fast-WAM reaches 95% in two epochs rather than eight. With LoRA fine-tuning, $\pi_{0.5}$ exceeds its baseline's five-epoch performance after one epoch. Single-step inference gives $1.54\times$, $2.49\times$, and $4.45\times$ speedups on $\pi_{0.5}$, ImageWAM, and Fast-WAM, respectively. It preserves action smoothness, achieving a $4.5\times$ reduction in action jerk compared with one-step Gaussian sampling. $\Delta$-flow reaches state-of-the-art generalization performance on LIBERO-Plus. Action-history sources can be applied to pretrained WAM policies without retraining. Replacing the Gaussian action source at inference largely preserves task success with a single sampling step and can slightly improve it. We further study how action representation and visual information in action learning affect history-to-future prediction in $\Delta$-flow. These results show that $\Delta$-flow is simple yet effective, improving the efficiency and generalization of robot foundation models.

Human demonstrations can capture task-relevant manipulation without using the target robot, but the recorded hand motions cannot directly serve as robot action supervision. Hand reconstruction and motion retargeting provide a direct route to such supervision, yet recovering hand motion alone does not ensure physically consistent hand-object interactions. To address this issue, we introduce Hand2Robot, an object-centric 4D hand reconstruction framework for robot policy post-training. Given videos of humans performing the target task, Hand2Robot extracts interaction clips, reconstructs object geometry, and generates robot action supervision through hand-object interaction optimization and motion retargeting. For object tracking, we combine observation confidence with temporal motion constraints to select a globally consistent object pose trajectory from per-frame pose candidates. For hand refinement, we use object geometry and motion as a reference and combine staged 2D alignment with object-local contact constraints to refine hand trajectories and finger joint configurations. Experiments on DexYCB and HO3D-v3 show improved object pose accuracy and hand-object interaction quality. Experiments on three real-robot dexterous manipulation tasks further show that reconstructed and retargeted human demonstrations can effectively supplement limited robot data for target-task policy post-training.

Recent multimodal large language models (MLLMs) and vision–language–action models have provided robots with new capabilities for long-horizon task understanding and planning. In surgical robotics, however, end-to-end prediction of low-level actions remains limited by insufficient precision, opaque decisions, and scarce data. This paper presents SurgiCLAW, a hierarchical skill-based framework for precision surgical manipulation. SurgiCLAW confines an MLLM to intraoperative visual understanding, structured explanation, and atomic surgical-skill selection, while delegating physical execution to verifiable deterministic control programs. For brain–machine-interface implantation, a representative high-precision surgical task, we construct a state-to-skill invocation dataset and expand it through visual and state perturbations. The model first learns structured skill invocation through supervised fine-tuning and is then optimized using three rewards for action correctness, operation prerequisites, and format validity, improving action selection, operational safety, and output parseability. We evaluate subtask success, offline model comparison, real-robot performance, and ablations. On a fixed test set, SurgiCLAW achieves 94.6% skill-selection accuracy and 93.0% macro-averaged F1 (Macro-F1). Across 50 closed-loop real-robot implantation trials, it achieves an 88.0% task success rate.

Modern industrial automation demands autonomous collaborative systems capable of coordinated manipulation in workspaces with heterogeneous object arrangements. Existing systems rely on predetermined trajectories and rigid allocation strategies, causing suboptimal workspace utilization and collision risks. This paper presents a distance-based competitive task allocation framework for dual-arm systems, integrating real-time vision perception with collision-free motion planning. A temporal window mechanism enables each arm to evaluate object detections over five consecutive vision frames before committing to optimal distance-based selection, featuring dynamic reallocation and spatial conflict resolution at 0.01m threshold accuracy. An improved Artificial Potential Field (iAPF) framework provides coordinated dual-arm movements through exponential goal attraction, inter-arm repulsion, home-seeking, and velocity damping forces. Simulation results validate superiority over market-based, round-robin, and fixed allocation strategies in distance efficiency, collision risk, and workspace coverage. Hardware experiments using two 6-DOF Kinova Gen3 Lite manipulators demonstrate approximately 47% average velocity improvement over fixed allocation in homogeneous scenarios with zero collisions, while mixed object scenarios confirm universal handling with approximately 11% time overhead offset by superior safety margins. The framework integrates YOLOv8 OBB detection at 30 FPS with 10 Hz control loops, demonstrating practical deployment capability for industrial assembly and sorting operations.

Yoshikawa's Manipulability Index is generally used as a classical measure to define an optimal morphology of a manipulator. However, any such optimization is almost always studied as a purely kinematic quantity in simulation and is rarely validated on hardware. We present a reinforcement-learning framework with RSL-RL PPO that optimizes the morphology of a 2R SCARA-type manipulator arm with Yoshikawa's kinematic manipulability index as a core reward, and the analytically derived closed-form optimum for the circular task is used to verify that our method recovers it. Then the same method is applied to custom elliptical and rectangular paths having no analytical optimum. The RL-optimized morphologies are then transferred to a 2R hardware manipulator to validate its effect in reduction of joint load in the hardware. Compared to the worst morphology, the optimum morphology delivered by our method reduces second joint load by 38.4 +/- 0.6% for the ellipse path and 41.3 +/- 0.3% for the rectangle, and first joint load by 15%.

Trailer sway is a persistent contributor to severe crashes among light passenger vehicles towing trailers, yet it remains largely unaddressed by the autonomous vehicle systems now entering this new era of automation. Improper cargo placement rearward of the trailer axle reduces, and can invert, the tongue load transmitted through the hitch. The resulting loss of vertical load at the tow vehicle's rear axle diminishes the lateral tire force available there and degrades yaw stability of the coupled system, admitting a divergent lateral oscillation of the trailer known as trailer sway. Under adverse combinations of speed and loading, this oscillation can progress to jackknifing, in which the articulation angle diverges rapidly and directional control is lost. This work develops a linear state-space model of the yaw-plane dynamics of a light vehicle-trailer combination, parameterized to expose tongue load as an explicit stability-determining quantity. From this model a linear quadratic regulator is synthesized to suppress articulation-angle divergence. Because consumer trailers carry no sensors, the two trailer states the controller needs, trailer yaw rate and articulation angle, are estimated by a Kalman filter from measurements taken only on the tow vehicle. The controller is then implemented and evaluated in a MuJoCo multibody simulation across a range of longitudinal speeds and tongue-load conditions, including the negative-tongue-load regime. Results show a significant decrease in trailer oscillation when controlling a given vehicle's steering with an LQR using tow-vehicle IMU and LiDAR data. These findings indicate that LQR controlled anti-sway systems are a viable control method for future autonomous vehicles.

Launching a micro unmanned aerial vehicle (UAV) from a hovering carrier and recovering it in flight requires reliable confirmation that the child is securely seated before the carrier is allowed to descend. This is challenging because vision provides relative guidance but does not directly indicate load transfer, and a downward-facing camera can lose its fiducial near touchdown. This letter presents a mothership-child quadrotor system in which mission transitions are gated by complementary nonvisual sensing. An upward-facing single-beam LiDAR on the recovery deck provides a binary occupancy state using hysteresis and a debounce interval derived from the child's occluding footprint and crossing speed; real-time kinematic (RTK) positioning conditions the carrier hover; and carrier thrust provides a proprioceptive indication of load transfer. In ten outdoor missions using an F450-class mothership and a Q250-class child with camera-guided recovery, nine completed successfully while RTK-fixed positioning was maintained through release and capture. A complementary MuJoCo study of fully autonomous fiducial-based recovery achieved touchdown 0.077 m from the deck center, while marker detection was lost within the final approximately 0.2 m above the docked configuration. In simulation, carrier thrust changed by -17.0 N at release and +17.3 N at capture, agreeing within 6% with the child's 18.0 N hover thrust. The results support a sensing architecture in which vision guides the approach, while range and proprioceptive information corroborate the final docking decision.

A deterministic method for the problem of distributed area coverage path planning subject to the inclusion of a headland path is presented. Given a, in general, non-convex area to be covered that may, in general, include obstacle areas, multiple robots, at least one robot depot, and constraints on each machinery such as a working width, work capacity limits, travel capacity limits, nonholonomic dynamical constraints and, possibly, a robot start and end depot assignment, distributed paths for each machinery are calculated that collectively enable full area coverage. Variations are discussed, in particular, a scenario where unconstrained depots assignment can attain a global optimal solution to the problem. The importance of inclusion of a headland path for area coverage, as well as the benefits in comparison to an alternative Boustrophedon-pattern based approach are highlighted.

This paper presents a geometry-aware predictive control framework for bipedal obstacle crossing across walking and running regimes. The proposed approach integrates a phase-structured trajectory generator with a constrained model predictive control (MPC) tracking layer to generate coordinated pelvis and swing-foot motions during obstacle traversal. The crossing process is represented through five interpretable phases: pre-crossing, leading-leg clearance, straddle, trailing-leg clearance, and recovery. Within this structure, swing-foot elevation is adapted as a function of obstacle height, while lateral stance widening and pelvis repositioning are introduced as functions of obstacle depth to improve transverse clearance and body alignment. A continuous gait-parameter evolution enables smooth walk-to-run transition without discrete controller switching. The generated references are tracked using MPC to regulate motion smoothness, clearance feasibility, touchdown behavior, and lateral admissibility. The framework is evaluated in reduced-order simulation over five progressively difficult obstacles under walking and running conditions. The robot successfully crosses the first four obstacles, achieving minimum body-obstacle clearances of 0.036, 0.036, 0.059, and 0.017 m, respectively. The fifth and most demanding obstacle leads to collision, identifying the current operating boundary of the proposed reduced-order formulation. Overall, the method provides a compact, interpretable, and computationally efficient basis for intelligent obstacle-aware bipedal locomotion.

Quantization lowers deployment costs of vision-language-action (VLA) policies but can compromise task completion. Across five VLA models and 4,050 real-world trials, we systematically reveal that quantization-aware training (QAT) generally improves on post-training quantization (PTQ) but leaves substantial gaps to full precision (FP) on long-horizon tasks and under visual and semantic shifts, while its imitation-based objective does not explicitly use rollout success or partial completion. To exploit these outcome signals, we introduce Quantization-Conditioned Reinforcement Learning (QCRL), an offline RL method coupling progress-based advantage estimation with quantization-aware source conditioning. FP rollouts supply reference experience beyond action-level distillation, progress-scaled terminal penalty distinguishes partial completion, and quantization-aware conditioning selects advantageous reference behavior within the quantized policy. We evaluate weight-only INT4 W4A16 on $\pi_{0.5}$ and GR00T N1.5 and joint weight-activation NVFP4 W4A4 on $\pi_{0.5}$. Using only in-distribution data, QCRL reaches 99.3% of FP's mean score across three visual-shift conditions for $\pi_{0.5}$ with INT4 W4A16, alongside 42.9% model-memory savings. With NVFP4 W4A4, it reaches 100.7% of the same FP baseline, alongside a $1.73\times$ speedup using NVFP4/FP8 deployment. Hardware gains are relative to FP16 on Jetson Thor.

Long-horizon robotic manipulation is inherently composed of semantically meaningful sub-tasks. While Movement Primitives (MPs) can model these components, their composition typically relies on manual engineering or sample-inefficient reinforcement learning. Existing supervised methods are limited by the lack of MP labels in standard datasets and often fail to generalize across discrete and rhythmic movements. We propose Neural Dynamic Composition (NDC), a supervised framework to compose Dynamic Movement Primitives (DMPs) using only raw trajectories for supervision, given a single example of each primitive. Our approach exploits parallel combinations of DMPs as a differentiable relaxation of categorical primitive selection, and utilizes a warp-invariant loss formulation to identify primitives despite spatial or temporal variations. A staged training curriculum is employed to learn the selection policy and DMP parameters, unifying discrete and rhythmic movements in one framework. Evaluated on 2D handwriting and 3D robotic tasks, our method outperforms baselines on most tasks and metrics. An ablation study demonstrates how each design choice contributes to the system's performance.

World Action Models (WAMs), often built upon pretrained video-generation world models, have emerged as a promising framework for general-purpose and open-ended robot control. However, their real-world deployment often suffers from discontinuous and jerky execution. Existing methods typically address this issue by either improving inference efficiency or enhancing action continuity across prediction chunks, without jointly addressing both challenges. To bridge this gap, we propose StreamingWAM, a streaming-diffusion framework for WAM-based robot control. StreamingWAM combines a noise ladder for chunk-wise action prediction, a prefix rollout module for cross-chunk continuity, and dense video-history memory for temporally consistent conditioning. Experiments on LIBERO, LIBERO-Plus, RoboTwin 2.0, and real-world robotic tasks demonstrate that StreamingWAM improves action continuity across static and dynamic settings, achieving competitive performance while reducing steady-state replanning latency to 72.9 ms in simulation and 57.3 ms on the real robot. The method also produces smoother action transitions, while the streaming staircase reduces sensitivity to the execution horizon K.

Cooperative driving at mixed-traffic unsignalized intersections is challenged by imperfect vehicle-to-everything (V2X) communication, human-driven vehicle (HV) intentions that cannot be directly communicated, and the potential incompatibility of independently generated crossing decisions. This paper proposes State-Intent-Aware Graph Multi-Agent Reinforcement Learning for Cooperative Driving under Imperfect V2X (SIGN-MAPPO) to address these challenges through three coordinated mechanisms. Each decision-eligible connected and automated vehicle (CAV) first reconstructs a receiver-specific traffic graph from its ego state and the peer-state records available to it, from which a shared graph-attention policy generates a raw Stop/Go intention augmented with predicted HV-intent features. CAV intentions are then exchanged through timestamped vehicle-to-vehicle (V2V) messages, and a local conflict-aware arbitration layer converts each CAV's intention into a safe executable action despite potentially inconsistent peer information. In the principal Simulation of Urban MObility (SUMO) evaluation, SIGN-MAPPO achieves a 97.53% completion rate, a 0.10% collision rate, and a mean intersection traversal time of 35.28 s. Ablation studies further confirm the complementary contributions of HV-intent prediction and CAV-intent exchange to this performance. The trained policy also remains robust under the evaluated communication-delay and packet-loss ranges, and cross-scenario tests show complete traffic clearance across the tested settings, though safety margins narrow under compound unpredictable HV behavior.

Open-world mobile manipulation requires a robot to find an object, approach it, and move it to a target receptacle from a language instruction. Training vision-language model (VLM) agents for these tasks requires demonstrations of both action selection and active vision. Existing pipelines often convert planned trajectories into high-level action labels, but shortest-path progress provides limited supervision for deciding where to look. We introduce Agent-Interface Data Alignment (AIDA), a data collection pipeline in which a teacher directly selects and executes actions through the deployed agent's skill interface. For mobile manipulation, we design teachers that maintain target visibility, recover lost targets, and approach for picking or placing. We add template-based spatial reasoning supervision and use a shared harness to organize observations, history, and outputs during training and inference. The resulting agent, AIDA-OWMM, achieves 71.76% full-task success on 170 held-out OWMM episodes. Teacher sensitivity analysis shows that active-vision demonstrations improve student success. The simulation-trained agent achieves 38% end-to-end success on a real mobile robot without additional training. Applying AIDA to a navigation subset of EmbodiedBench further demonstrates the pipeline's utility for navigation tasks.

Soft wearable robots must commit to assistance around movement onset, when the available signals are ambiguous. Surface electromyography (sEMG) detects neuromuscular drive early but degrades under sweat and electrode shift, and kinematic sensors respond only once the limb moves. This letter presents a knitted textile interface that co-samples a three-channel mechanomyography (MMG) array and an anterior strain sensor on a single synchronous 500 Hz clock. Across five participants, MMG activation preceded strain-detected knee motion during sit-to-stand by per-channel medians of 322-344 ms. Sampled on the same clock, the sign of the strain slope resolved movement direction on 649 of 650 completed transitions with no training or per-participant calibration, raising four-class balanced accuracy from 90.3% to 95.6%. The fused signals drove a two-stage confirm-then-assist policy for a soft pneumatic knee. Across 240 trials in four participants, the onboard controller entered the assist state on 78 of 80 sit-to-stand trials and on none of the 80 isometric co-contractions. It also entered the assist state on 27 of 80 stand-to-sit trials and left it at the hold timeout, motivating an antagonistic flexor gauge to resolve this direction at onset. Co-sampling muscle acoustics with textile strain on one clock lets the inflation command be issued around movement onset.

Existing navigation systems commonly rely on dedicated depth sensors or explicit depth reconstruction, while learning to generate trajectories directly from RGB images remains challenging due to the sim-to-real domain gap. In this work, we propose Stereo-Planner, an end-to-end vision-to-trajectory framework that enables zero-shot sim-to-real planning from stereo RGB observations. The key idea is to formulate trajectory generation as state- and scale-conditioned querying of latent stereo geometry. A Scene Representor incorporates a pretrained vision grounding foundation model to extract geometric cues shared across simulated and real-world environments, supporting generalization across visual domains. A lightweight Trajectory Decoder then translates these representations into trajectories through interleaved attention, separating trajectory-state interaction from scene-feature retrieval and incorporating the stereo baseline as a metric-scale cue. With these designs, Stereo-Planner is trained entirely on simulated data by minimizing differentiable trajectory costs and deployed directly in real-world environments without fine-tuning. Benchmark results demonstrate strong cross-scene generalization, with a 95% success rate in simulation environments at a commanded speed of 8 m/s, and competitive performance against depth-based planners using only stereo RGB observations for scene perception. Furthermore, deployment on a compact quadrotor equipped with an NVIDIA Orin NX enables autonomous flight at speeds of up to 7 m/s, with a mean onboard vision-to-action inference time of 24.27 ms. The software, simulation environments, and hardware designs will be open-sourced at https://github.com/Stereo-Planner/Stereo-Planner.

Visual self-modelling lets robots learn body models from observations, but such models usually encode kinematics implicitly rather than exposing metric joint parameters for computation and control. Moreover, a good fit to images does not by itself establish accurate kinematics. Here we present an explicit analytic robot self-model that requires neither deformation networks nor URDF/CAD templates. Multi-view images and joint readings establish Gaussian appearance and provide motion evidence for rigid link assignments, whereas sparse end-effector poses independently identify metric screw axes. The resulting closed-form forward kinematics and analytic Jacobian can be evaluated beyond the fitting ranges. We validate the model on a simulated seven-axis arm, a simulated twelve-joint quadruped and a real six-axis arm. In simulation, it achieves millimetre-level reaching and pick-and-place, as well as closed-loop quadruped walking. On the real arm, it renders held-out configurations, predicts end-effector positions with a median error of 6.4 mm, compared with 11.7 mm for a calibrated nominal model, and supports reaching with external marker feedback. A controlled comparison shows that image fit alone is not enough: two Panda models differing only in their joint axes are 0.54 dB apart in PSNR, yet their end-effector prediction errors differ 13.7-fold.

Vision-Language-Action (VLA) models commonly generate robot actions token by token through Large Language Model (LLM) decoders, causing inference latency to grow with action sequence length and limiting closed-loop control frequency. We present Goal-Conditioned Vision-Predictor-Action (VPA), which replaces autoregressive decoders with a lightweight latent control loop. A shared vision encoder and a frozen text encoder embed observations, goal images, and instructions; a neuro-symbolic selector chooses a discrete skill; a Joint-Embedding Predictive Architecture (JEPA) predictor forecasts the next latent state; and a flow-matching solver generates an entire action chunk in a few integration steps, so that sequential network evaluations per decision step do not depend on the action horizon. We analyze inference complexity, regularization against representation collapse, and conditions under which latent sub-goal thresholding limits error accumulation across task phases. As an initial proof of concept, we evaluated VPA on the LIBERO-10 suite. The trainable model has under 20M parameters (checkpoint under 75 MB, excluding a frozen 63.4M-parameter CLIP text encoder); a decision step of a single-camera reference configuration takes 10-14 ms on an Apple M4; and trained only on LIBERO-10 demonstrations in under three hours on a single GPU, a two-camera policy succeeded in 40.0% of 200 evaluation episodes. These preliminary results from a single training run suggest the approach is workable. Code and weights are available at https://github.com/ylee-sling/Vision-Predictor-Action and https://huggingface.co/ylee-sling/vpa-libero10, and archived at https://doi.org/10.5281/zenodo.22960920.

Quadruped robots deployed for inspection and field missions must cross heterogeneous terrain, from stairs, slopes, and gaps to high platforms, stepping stones, and balance beams, where safe support is sparse or see-through. Existing methods interpose a terrain map or a depth image between sensing and control. Maps are costly to maintain, corrupted by odometry drift and outliers, and misled by see-through gratings, while depth cameras degrade with lighting and surface material. In this paper, we present PointWalker, a mapless framework whose single reinforcement learning policy directly consumes raw point clouds from four light detection and ranging (LiDAR) sensors. Voxelized points are encoded by linear self-attention and fused with proprioception through cross-attention, so terrain geometry reaches the policy without maps or foothold estimation. One proximal policy optimization (PPO) run, with terrain-aware rewards and dual-channel simulation of see-through gratings, yields one policy for all seven terrain classes with a 90.1% mean success rate in simulation. Deployed zero-shot on a Deep Robotics X30 at 50 Hz onboard, it crosses 0.8-m gaps, climbs 0.8-m platforms and 45{\deg} steel-grating stairs, carries a 50-kg payload, and completes a 2.5-km mountain route.

Camera-based occupancy prediction provides geometric and semantic scene understanding for robot perception and has been adopted in autonomous vehicle and UAV perception systems. Gaussian primitives offer a promising alternative by potentially representing only occupied regions, thereby concentrating computation and representational capacity where needed and reducing wasted computation and memory in empty space. However, existing Gaussian-based methods suffer from poor primitive utilization, as ineffective training objectives leave some Gaussians in empty space, while the limited expressiveness of Gaussian functions restricts their ability to model complex shapes. To address these limitations, we propose a novel 3D representation, Gaussian Neural Field (GNF), which can accurately map voxel positions to semantic occupancies. In GNF, Gaussians and a neural network jointly parameterize the field, where the Gaussians define its spatial support and provide features for neural decoding, while the neural network enables more expressive occupancy modeling than Gaussian functions alone. We further develop the Gaussian Neural Field Transformer (GNFormer), a streamlined and flexible method that predicts GNFs for camera-based occupancy prediction. Across Occ-ScanNet, SSCBench-KITTI-360, and nuScenes-SurroundOcc benchmarks, GNFormer achieves the highest IoU and mIoU and the fastest inference among the compared single-frame methods. In the experiments, GNFormer can represent indoor and outdoor scenes with only 256 and 1024 Gaussians, respectively, substantially fewer primitives than prior Gaussian-based methods. The code will be released.

Active debris removal requires a servicer spacecraft to approach a non-cooperative tumbling object, capture it, and arrest its rotation. This sequence is conventionally divided among phase-specific controllers, each depending on knowledge of the target's inertia, a dedicated grappling fixture, or a hand-designed switch at the contact-rich seam where such pipelines are most fragile. We show that a single constrained reinforcement learning policy can execute the entire mission, driving a dual-arm free-flying servicer from free-space closing through friction-based fixture-free capture to full detumbling of a target whose mass and inertia are unknown and whose state is observed only through noisy estimates. The enabling mechanism is a smooth proximity gate applied at three loci of a constrained Markov decision process. In the reward it induces the phase structure rather than imposing it; in the action space it forecloses a degenerate grab-and-pull exploit as a hard guarantee; and in the constraint set it resolves a normalization failure of constraint-based learning, in which a constraint spanning phases that load it unequally is rendered nearly inert in the phase where precision matters most. Both the failure mode and the remedy apply to any multi-phase problem of this kind. Over 1000 randomized trials the policy succeeds in 99.5% while holding its contact-force, actuator, base-reaction and collision limits throughout, transfers zero-shot to unseen geometries and centre-of-mass offsets, and extrapolates across a tenfold range of target mass.

Robotic systems for lunar surface operations must act autonomously and preserve their own integrity in an environment where failures are unrecoverable and continuous supervision is impossible. This paper presents a Constrained Reinforcement Learning framework for quadrupedal mobile manipulators that learns a single whole-body policy for tracking a 6D end-effector pose specified in task space, with operational limits enforced during learning through a constrained learning mechanism. We introduce a reward formulation in which the transition between locomotion and manipulation emerges from the reward geometry rather than from an explicit gating mechanism, a stability term derived from the Gravito-Inertial Inclination Margin and motivated by the reduced-gravity dynamics of the lunar surface, and a design rule for reward regularization that follows from the multiplicative structure of the termination-based objective. The framework is evaluated in a lunar-analog simulation spanning three terrain morphologies at five difficulty levels under lunar gravity, with regolith contact properties calibrated to the JSC-1a simulant and varied from loose to dense. The learned policy attains mean position and orientation errors of 3.22 cm and 6.83 deg. under nominal conditions, retains this precision across regolith densities and over the operationally representative range of terrain difficulty, and reduces soft-constraint violation time by up to an order of magnitude relative to an unconstrained baseline while incurring no hard-constraint violations.

Monocular depth estimation (MDE) has emerged as a cost-effective alternative to hardware-dependent 3D sensing modalities, yet predicting precise metric depth remains a persistent challenge due to intrinsic scale ambiguity, edge bleeding, and limited zero-shot generalization in unseen environments. Recent multi-modal approaches incorporate 1D/3D Single-Photon Avalanche Diode (SPAD) data to resolve these ambiguities, but they mainly treat SPAD noise as a harmful artifact required for denoising or leave the noise ignored. In this paper, we present an essentially different insight: native SPAD noise is an informative prior that implicitly encodes deterministic environmental characteristics. Driven by this paradigm shift, we introduce a novel, lightweight Test-Time Gated Attention framework by Harnessing SPAD Noise, therefore dubbed HSN. Tailored for indoor scenarios, HSN leverages the synergistic combination of 1D SPAD histograms and Gate Adapters (GAs). Through three complementary optimization phases, it generates channel-wise gated attention vectors to dynamically adjust encoder feature maps at inference time, effectively resolving ambiguous scale and boundary challenges without the heavy computational overhead or dense 3D modalities that other TTA methods suffer from. Extensive experiments on challenging real-world indoor benchmarks show that HSN achieves state-of-the-art metric depth accuracy in in-domain settings, while exhibiting even greater potential under zero-shot generalization to unfamiliar environments. Furthermore, interpretability analysis reveals that the proposed gated attention mechanism adaptively shifts attention toward deeper semantic layers under intensifying SPAD noise. Code will be made publicly available.

In embodied manipulation, Vision Language Action (VLA) models can execute atomic skills, but long-horizon manipulation requires recombining these skills without demonstrations for every skill combination. The full instruction route treats skill chaining as long-horizon imitation. In a controlled study, full instruction training on limited skill combination sequences reaches 40.0% ordered success on covered combinations but only 7.7% on recombination, suggesting memorization of sequence templates rather than robust skill recombination. We introduce Hierarchical Subtask Execution (HSE), a diagnostic framework that exposes skill recombination through a shared modular interface over the planner, trigger, and policy, decomposing long-horizon failures into separable modes. We introduce RoboCombine, a benchmark for skill recombination without combined skill training trajectories, with metrics for ordered success and transition conversion across subtask boundaries. RoboCombine reveals that atomic competence alone does not ensure ordered execution. Failures concentrate at transitions where downstream subtasks launch from handoff states from preceding subtasks rather than canonical atomic starts. We identify this mismatch as the start state gap. To address it, we introduce Pose Robust Task Initialization (PRTI), a lightweight atomic task post-training strategy that broadens start state coverage beyond canonical starts. Under oracle timed HSE on RoboCombine, PRTI reaches 18.16% ordered success, compared with 4.21% under random perturbation and 9.47% under a robot-reset oracle, without combined skill training trajectories. These results show that combinatorial generalization requires handoff robust atomic skills that convert first stage progress into reliable downstream execution, not merely decomposition into known subtasks. Code: https://github.com/AutoLab-SAI-SJTU/RoboCombine.

Haptic shared control in surgical robotics combines human-side and robot-side forces at a shared end effector while preserving operator involvement. However, the effects of geometric safety choices on task execution and force coordination remain unclear. We model the interaction as a two-agent cooperative Markov game under centralized training and decentralized execution, comparing learned human control with a task-tuned fixed-impedance baseline. This model-free multi-agent reinforcement learning framework supports joint policy adaptation without requiring a complete analytical interaction model. It compares MADDPG and rMAPPO with an epigraph-constrained approach under common geometric task definitions. In simulation, rMAPPO generally achieved the highest return and tracking performance, whereas the epigraph approach produced more conservative motions and lower geometric exposure, including no observed collision or dangerous-region exposure after geometry transfer. With real operator input, fixed-impedance-trained policies were faster and successful in all recorded trials. Policies trained with a learned partner exhibited greater command opposition, while the fixed epigraph condition produced the most aligned commands. This work provides a benchmark for studying human-control representation and constraint-handling trade-offs in haptic shared control.

VLMs provide promising solutions to embodied reward design. Most current methods that train VLMs as reward regressors not only require extensive annotations but are also primarily designed for finetuning pretrained policies, and perform less well in the more challenging from-scratch RL setting. This issue stems from a fundamental mismatch: regressing precise rewards is misaligned with what VLMs naturally excel at. We posit that it is more aligned with VLMs' inherent multi-modal VQA capabilities for the models to function as success detectors, and propose VLM success reward via Exploration-Augmented Key-Observation Gating (VLMeg) to well harness VLM for embodied reward design. As the core of VLMeg, the proposed exploration-augmented key-observation gating first determines whether the current RL episode is judgment-worthy based on its similarity to a generated success trajectory. Then, the key observation with the lowest confusion from the judgment-worthy episode will be selected for VLM evaluation as episodic reward. This gating mechanism not only avoids redundant calls but also enables higher detection accuracy of VLM. Considering the sparsity of judgment-worthy episodes, the advanced exploration incentives based on the generated trajectory are introduced, enabling more efficient exploration toward judgment-worthy trajectories for more VLM reward triggers and thus gaining higher RL efficiency. Extensive experiments in MetaWorld demonstrate VLMeg achieves state-of-the-art from-scratch RL, outperforming advanced VLM-based baselines on 12 challenging reward-free tasks. Furthermore, VLMeg also enables effective RL without any ground-truth reward in 3 challenging real-world experiments. Code will be open after acceptance at https://github.com/Raina915/VLMeg.