Robotics
A Minimal Optical-Flow Representation for Vision-Based Tactile Rotation Classification in Robotic Manipulation Across Gravity Domains
Oscar Martinez-Bernal, Mario Cavero-Vidal, Francesco Grella et al. (+1 more)
Vision-based tactile sensors provide rich contact information, but processing high-resolution images can be costly for resource-constrained platforms such as space
robots. This work investigates whether a compact representation of tactile motion can classify object rotation across different
gravity conditions. Dense optical flow from a simulated GelSight Mini is aggregated over a 7x9 grid into 126 features and used to classify the direction of load-induced rotation under Earth, Mars, Moon, and orbital
gravity.
Gravity causes a small but significant shift in these features, accounting for 1.6% of their variance (R2 = 0.016). Despite its small magnitude, this shift affects models trained only on Earth data: XGBoost accuracy decreases from 94.4% on Earth to 75.9% in orbit. In contrast, a single model trained across all four
gravity domains achieves 96.3% overall accuracy and 95.1%-97.0% across individual domains, without using
gravity as an input. The representation can also be reduced to 40 features while retaining 95.7% accuracy, with XGBoost requiring only 0.14 ms per inference. These findings show that Earth-
gravity performance alone is insufficient to establish the transferability of tactile perception for space robotic manipulation, highlighting the need to account for
gravity-induced domain shifts during training and validation.
Read the full abstract ▾
All authors Oscar Martinez-Bernal, Mario Cavero-Vidal, Francesco Grella, Carol Martinez
Robotics
SkillWeave: Weaving Heterogeneous Demonstrations into Long-Horizon Manipulation Skills
Ryosei Tamura, Xiaoxiang Dong, Uksang Yoo et al. (+4 more)
Dexterous manipulation requires both large-scale task progression and precise contact-rich interaction, making it challenging to collect demonstrations that effectively support both regimes. We present SkillWeave, a heterogeneous demonstration framework for long-horizon dexterous manipulation that combines teleoperation for coarse reaching and transport with kinesthetic teaching for precise, contact-rich skills. To address the visual mismatch introduced by the demonstrator's presence during kinesthetic data collection, we propose an object-mask-conditioned
diffusion policy that uses offline object segmentation for training supervision and a lightweight learned mask predictor at deployment, avoiding online segmentation and image inpainting. To mitigate distribution shift between independently trained sub-task policies, we introduce successor-aware terminal steering, which selects among actions sampled from the predecessor policy to guide the system toward states supported by the successor's demonstrated initial-state distribution. Across three real-world long-horizon tasks, SkillWeave achieves 27% average end-to-end success. Mask-conditioned kinesthetic policies improve dexterous sub-task success to an average of 65%, while successor-aware handoffs achieve an average composition efficiency of 87%. These results show that matching demonstration modality to interaction regime, explicitly addressing kinesthetic visual mismatch, and steering policy handoffs toward successor-supported states substantially improves long-horizon dexterous manipulation.
Videos and code are available at skillweave-authors.github.io .
Read the full abstract ▾
All authors Ryosei Tamura, Xiaoxiang Dong, Uksang Yoo, Yuemin Mao, Romina Mir, Jonathan Francis, Jeffrey Ichnowski
Robotics
Traceable World State: A Provenance-Aware State Representation and Deterministic Replay Framework for Robotic Systems
Zoe Li
Robotic systems operating over extended tasks must maintain a world state assembled from observations arriving at different times, with varying confidence and potential revisions. Conventional representations emphasize latest estimates, hindering fact provenance, decision reproduction, or execution auditing. We present Traceable World State (TWS), a middleware-neutral semantic representation and reference runtime for provenance-aware
robot world state. A TWS snapshot captures entities, relations, observations, confidence, and revision metadata. Validated update operations transform snapshots immutably, ordered updates support deterministic replay, and a canonical SHA-256 hash chain ensures tamper-evident logs. We evaluate TWS through schema conformance, complete state lifecycles, deterministic replay, and fault injection. Passing 38 tests across Python 3.10-3.14, the framework detects record corruptions, broken hash links, sequence discontinuities, and world mismatches. Across ten public BEHAVIOR-1K task definitions, TWS imported 153 entities and 146 relations with successful validation. On 103 NVIDIA Unitree G1 simulated trajectories containing 78,369 frames, TWS achieved exact terminal-state replay in all episodes and detected 412/412 injected corruptions with a 1.72% storage overhead over Plain JSONL.
Read the full abstract ▾
Robotics
HWAM: Humanoid World Action Model With Joint State–Action Generation
Yan Yang, Jikun Rong, Minzhao Zhu et al. (+7 more)
Humanoid
robots are a promising platform for general-purpose manipulation. Recent
Vision-
Language-Action (VLA) policies learn actions directly from
multimodal observations, while World Action Models (WAMs) further incorporate future visual prediction to improve action generation. However, in hierarchical humanoid systems, VLA and WAM policies output reference actions that are subsequently realized through whole-body control,
robot dynamics, balance, and contact. This hierarchy creates an action–execution gap: the reference produced by the policy can differ from the motion realized by the
robot. Without explicitly modeling the realized body state, future visual prediction must jointly explain scene
evolution and discrepancies between reference actions and executed motion, making it difficult to associate an action with its physical outcome. We propose HWAM, a Humanoid World Action Model with joint state–action generation, which makes the
robot's post-execution proprioceptive state an explicit prediction target. By jointly generating reference actions and their realized body states, HWAM directly incorporates supervision of executed motion into action learning. HWAM is trained through three complementary conditional paths. The Policy path jointly denoises state–action trajectories conditioned only on current observations, matching deployment conditions. Forward Dynamics Modeling (FDM) predicts future visual observations conditioned on actions and post-execution states, while Inverse Dynamics Modeling (IDM) reconstructs the joint trajectory from visual transitions. Together, these paths connect policy references, realized body motion, and visual outcomes. HWAM achieves the highest success rate among evaluated baselines on three real-
robot tasks on the LimX OLI humanoid. On Candy Picking, HWAM achieves a 70.6% success rate, compared with 43.3% for Fast-WAM.
Read the full abstract ▾
All authors Yan Yang, Jikun Rong, Minzhao Zhu, Zheyi Zhao, Qirui Hu, Zihan Lan, Weixin Mao, Yinhao Li, Zhen Fu, Hua Chen
Robotics
REACT: Rolling Denoising and Dual Decoupling for Reactive Robot Control with VLA Models
Houlong Xiong, Zhenqi Qiu, Zechen Wang et al. (+8 more)
Flow-based
vision-
language-action (VLA) models generate action chunks for temporally coherent
robot motion, but chunked control creates a fundamental closed-loop trade-off: long chunks provide smooth execution, whereas frequent replanning improves reactivity at the cost of action discontinuities. We introduce REACT, a rolling-denoising framework that makes flow-based VLAs more reactive while preserving long-horizon context. Instead of regenerating entire action chunks from scratch, REACT maintains a persistent action buffer with staggered flow timesteps. At each control step, the full horizon is denoised using the latest observation, the cleanest action block is executed, partially refined future blocks are shifted forward, and fresh noise is appended to the tail. As a result, each executed action block is refined across multiple recent observations before deployment. To support real-time control, we further introduce dual decoupling, which separates sensing, VLM encoding, DiT denoising, and action execution, enabling high-frequency observation updates and action streaming under practical compute constraints. Across the RoboTwin 2.0 simulation
benchmark and real-world tasks spanning bimanual manipulation and dynamic control on multiple
robot platforms, REACT improves task success and reduces reaction latency while producing smoother trajectories than frequent-replanning and asynchronous baselines.
Read the full abstract ▾
All authors Houlong Xiong, Zhenqi Qiu, Zechen Wang, Suohang Zhang, Yiyu Ren, Wanting Xu, Hongfei Niu, Chengyang He, Ge Sun, Ran Cheng, Qian Zhu
Robotics
Predicting Cable Dynamics with Physical Attention Bias
Avihai Giuili, Rotem Atari, Avishai Sintov et al. (+1 more)
Learned simulators for deformable linear objects (DLOs) such as cables have to predict the motion of cables they were not trained on and stay stable over long rollouts. Most of their error occurs where the cable touches itself or the floor.
Attention over all pairs of cable segments can represent contact between parts of the cable that are far apart along its length, but
attention has no notion of geometry. A cable has two pairwise distances, which agree only while it is straight: the arc-length distance along the cable, which governs elastic forces, and the Euclidean distance in space, which governs contact. We add a physical
attention bias, an additive term on the
attention logits with a learned rate, and ask which distance it should use. We compare no bias, each distance alone, and both distances on disjoint sets of heads, keeping the rest of the model and the training protocol fixed. A physical bias improves prediction on unseen cables. The gain is largest when
attention is the only mechanism that connects distant segments: there, the arc-length bias reduces prediction error by 15% and more than halves the drift in segment length. The Euclidean bias alone stays close to unbiased
attention, while assigning both distances across heads is best or near-best on every metric we report. Code and per-run records: https://github.com/avihaig/dlogps.
Read the full abstract ▾
All authors Avihai Giuili, Rotem Atari, Avishai Sintov, Maya Bechler-Speicher
Robotics
CAPABLE: Capability-Aware Policy Adaptation via Behavioral Latent Encoding
Mohammad Khoshnazar, Mohammad Dehghani Tezerjani, Deyuan Qu et al. (+6 more)
Vision-
language-action (VLA) policies assume the embodiment on which they were trained and can fail when a joint fault changes how commanded actions are physically executed. Existing fault-recovery methods often require task-specific retraining, fault labels, explicit diagnosis, or privileged embodiment information. We introduce CAPABLE, a unified capability-aware adaptation framework for frozen VLAs that integrates self-supervised capability inference with residual reinforcement learning. CAPABLE infers capability, how much of the commanded motion each joint actually realizes and how that motion contributes to end-effector behavior, online from command-response history and kinematics using a temporal encoder shared across joints, Jacobian grounding, cross-joint
attention, and self-supervised physical prediction. The resulting representation conditions a residual policy that adds bounded corrections to the VLA arm action without fault labels or faulty-joint identifiers. Across 28 LIBERO tasks, CAPABLE raises success on an actuator excluded from fault training from 24.8% to 59.3%, outperforming a parameter-matched global-history baseline by 17.4 points while preserving healthy performance. Leave-one-actuator-out experiments across six joints show that this transfer is not specific to one actuator, and additional evaluations characterize transfer to unseen fault families and demonstrate recovery on a physical Franka Panda. https://capable-vla.github.io/
Read the full abstract ▾
All authors Mohammad Khoshnazar, Mohammad Dehghani Tezerjani, Deyuan Qu, Zhiyuan Gao, Yanxiang Zhan, Jeroen Schafer, Andrew Melnik, Qing Yang, Michael Beetz
Robotics · Cross-listed from cs.CV
Learning Which Correspondences to Trust: Confidence-Weighted Event-Camera Localization in LiDAR Maps
Panagiotis Kiousis, Kuangyi Chen, Jun Zhang et al. (+1 more)
Localizing an event camera against a pre-built
LiDAR map can be cast as dense optical-flow estimation between a rendered depth view and an event image, followed by a Perspective-n-Point (PnP) solver over the induced 3D-2D correspondences. Existing pipelines rely on
geometric consensus during pose estimation, but do not explicitly model the reliability or pose informativeness, i.e., how strongly a correspondence constrains the camera pose, of individual correspondences. We show that the natural way to learn it – using the per-correspondence error to constrain the learning of confidence – suffers from a depth-dependent bias: small pixel errors reside predominantly at large depths and do not lead to high pose informativeness. Instead, in our method (
CELL), we learn a per-correspondence confidence end-to-end through the pose, using a differentiable probabilistic PnP whose log-partition term encourages weight configurations that yield a better-constrained pose distribution. The learned confidence is used in three ways: (i) it reweights the flow supervision in a decoupled training scheme that keeps pose gradients out of the flow/edge backbone; (ii) it drives a probabilistic correspondence selection at test time; and (iii) together with the network's edge-probability it weights a final edge-matching refinement. We further design a partial-completion depth representation that adds signal without hallucinating across large gaps. On M3ED and DSEC our full system improves over the LEAR baseline on the majority of the evaluated sequences: it reduces the median translation error by up to 26.9% and the median rotation error by up to 15.8%.
Read the full abstract ▾
All authors Panagiotis Kiousis, Kuangyi Chen, Jun Zhang, Friedrich Fraundorfer
Robotics
Reliability-Aware Future Conditioning for Temporally Robust Robot Manipulation
Mohammad Khoshnazar, Mohammad Dehghani Tezerjani, Zhiyuan Gao et al. (+8 more)
A generated
video of a task the
robot is about to perform is useful guidance only if it depicts the phase the
robot is actually in. We show that temporal misalignment can turn a task-consistent generated future into actively harmful guidance. On CALVIN, a five-frame early shift nearly erases the benefit of generated futures, reducing success from 81.3% to 54.8% against 54.0% without futures; imposed timing shifts reduce it even further to 34.2%, 19.8 points below the future-free policy. We introduce Reliability-Aware Future Conditioning (RAFC), which treats this as a control problem rather than a generation problem. At every step, RAFC estimates how far to trust the received clip and which nearby temporal hypothesis to prefer, falling back toward a static branch when neither fits, and it learns both from task reward alone without shift labels or
alignment supervision. RAFC sits on top of Future-Experience Conditioning (FEC), which builds the clip once from task grounding, a
robot-free digital-twin rollout, and mask-free
video diffusion. Under deliberately off-grid phase shifts and rate mismatch, RAFC substantially improves success under temporal mismatch. Candidate ensembling accounts for most of the recovery near
alignment, while learned reliability adds a further 7.0 percentage points over uniform averaging of the identical candidate bank under off-grid shifts. The gain holds on the evaluated task sets and survives on a Franka under natural timing mismatch nobody imposed, where aggregate success rises from 26.7% to 56.7%. All resources will be made publicly available. https://future-condition.github.io/.
Read the full abstract ▾
All authors Mohammad Khoshnazar, Mohammad Dehghani Tezerjani, Zhiyuan Gao, Deyuan Qu, Max Gandyra, Yanxiang Zhan, Mehreen Naeem, Andrew Melnik, Jeroen Schafer, Qing Yang, Michael Beetz
Robotics
Tell Robot What Not to Do: A Negation Understanding Perspective
Fazeng Li, Gan Sun, Hao Cheng et al. (+2 more)
Instruction following enables
robots to perform diverse tasks specified in natural
language, making it a fundamental capability for human-
robot interaction. Beyond communicating desired outcomes, users also need to specify constraints on what not to do. We investigate how to enable
vision-
language-action models (VLAs) to follow negated instructions, where
robots must accomplish task goals while respecting explicit exclusions. To this end, we propose NegaAlign, a parameter-efficient, plug-and-play framework that extends pretrained VLAs to follow negated instructions through image-
language supervision alone. Specifically, we introduce Negation Transformation Layers into selected layers of the
vision-
language backbone to reshape intermediate instruction representations. Meanwhile, a teacher-guided
alignment mechanism is designed to align instruction-relevant visual
tokens, transferring action-relevant grounding from instructions that satisfy the negated constraint. The training phase uses supervision constructed from existing demonstrations and updates only the inserted layers, keeping all pretrained parameters frozen, including the action generator. We further introduce NegaBench, a simulation
benchmark spanning 10 scenarios across five domains for systematically evaluating manipulation under negated constraints. Experiments across GR00T, $\pi_0$, and $\pi_{0.5}$ demonstrate consistent improvements in negated instruction following. With 11.6M trainable parameters, NegaAlign increases the negated-instruction success rate of $\pi_{0.5}$ from 2.60% to 88.45% on NegaBench and from 12.4% to 88.8% on real-world tasks, while retaining performance on affirmative instructions.
Read the full abstract ▾
All authors Fazeng Li, Gan Sun, Hao Cheng, Weihong Ren, Yang Cong
Robotics
TACROSS: An Efficient and Low-Cost Scalable Human Touch System Across Heterogeneous Tactile Sensors for Dexterous Robot Learning
Bo Chen, Huanzhang Hu, Junyang Ma et al. (+11 more)
Collecting tactile demonstrations on
robots is costly and slow, motivating the use of lower-cost human tactile gloves for scalable data collection. However, human capacitive/piezoresistive gloves and robotic tactile sensors differ fundamentally in transduction principle, sensor layout, spatial resolution, and dynamic response, making
alignment of raw sensor channels ill-posed. To address this problem, we present TACROSS, a scalable system for learning from human touch and transferring it to
robots that bridges this heterogeneity by aligning tactile streams at the level of contact events rather than raw sensor values. The hardware component of TACROSS integrates a piezoresistive glove with five layers and a cost of USD 10.86 with 285 sensing points. To align contact semantics, we design canonicalizers and residual adapters that map heterogeneous signals into a shared tactile latent with 256 dimensions via a temporal
Transformer with
attention across fingers. We further introduce a
robot-grounded policy learning scheme in which
robot demonstrations provide the sole source of ground-truth action supervision, while human demonstrations support tactile representation learning and provide confidence-weighted auxiliary supervision through valid retargeted hand targets. We evaluate our system on four contact-rich manipulation tasks. Compared to conventional teleoperation, our proposed system achieves a 3.5-fold efficiency improvement while reducing demonstration acquisition equipment cost by 95.7%. We will open-source the TACROSS hardware and software system and publicly release a tactile
dataset comprising over 150 hours of recordings. Project page: https://tacross-touch-project.github.io/.
Read the full abstract ▾
All authors Bo Chen, Huanzhang Hu, Junyang Ma, Bo Yue, Fangdi Yu, Haijier Chen, Xianxin Lai, Shuyu Pan, Zhen Yang, Xiaoquan Sun, Wenze Cui, Zhongliang Jiang, Shaopeng Liu, Jiayu Chen
Robotics
STAG: A Sparse Traversability-Aware Graph Representation from Grid-Based Costmaps for Robotic Navigation
Gabriel Manuel Garcia, Stéphanie Aravecchia and Miguel Angel Olivares-Mendez
Autonomous rovers navigating large unstructured environments need efficient global planning that accounts for terrain traversability. However, searching dense grid-based costmaps becomes computationally expensive as the mapped area grows. We introduce STAG, a Sparse Traversability-Aware Graph that converts costmaps into compact graphs. STAG combines a medial-axis topological backbone, representative nodes for homogeneous traversability regions, and transition nodes near strong traversability gradients. Edges encode geometry and traversability to account for path length and terrain difficulty. We compare A* on STAG and dense grids using synthetic cave maps, mine maps and the DARPA CERBERUS
dataset. Across five
benchmark categories comprising 203 map instances and 101,200 queries, STAG reduces median planning time by 3.4x to 9.9x and peak query
memory by 2.1x to 15.4x, with median relative path-length differences of -2.9% and +7.6%. STAG offers a compact representation for global planning, trading dense-grid traversability optimality for faster, less
memory-intensive search.
Read the full abstract ▾
Robotics · Cross-listed from cs.NI
Digital Twin for Pre-Deployment Validation of AI-Driven Safety-Critical Industrial Edge Control Loops
Sara Cavallero, Federico Tonini, Tony Chahoud et al. (+5 more)
Industrial environments are increasingly characterized by the tight interaction among physical processes, communication infrastructures, and intelligent applications. In this context, Digital Twins (DTs) have emerged as a key technology for system analysis and optimization. However, existing DT solutions typically focus either on industrial processes or communication networks, while lacking an integrated and application-aware perspective. To fill this gap, this paper proposes a modular DT framework for industrial environments that jointly models physical processes, wireless communications, and application logic within a unified architecture. The feasibility of the proposed framework is experimentally validated through a real-world Proof-of-Concept (PoC) implemented in the BI-REX pilot line, involving a 5G-connected Autonomous Mobile
Robot (AMR) transporting hazardous
liquids and remotely controlled by an AI-driven application. The proposed DT is used to reproduce the behaviour of the real deployment and to investigate the impact of different placements of the AI application, including on-premise, edge, and remote cloud execution scenarios. Experimental results demonstrate a close agreement between DT predictions and PoC measurements in terms of both network-level metrics, such as Reference Signal Received Power (RSRP) and latency, and end- to-end application metrics, including application-level Round- Trip-Time (RTT). Moreover, the analysis shows how inaccuracies of network modeling can critically affect the feasibility of latency-sensitive industrial control loops, highlighting the potential of integrated DTs as tools for the pre-deployment design and validation of next-generation industrial systems.
Read the full abstract ▾
All authors Sara Cavallero, Federico Tonini, Tony Chahoud, Giampaolo Cuozzo, Davide Borsatti, Walter Cerroni, Maurizio Fodrini, Riccardo Marini
Robotics
WAND: Learning Robust Navigation under Complex Wind Disturbances and Dense Obstacles for Quadrotors
Zhonghan Tang, Chenhui Li, Shuai Liang et al. (+5 more)
Robust navigation in cluttered environments remains a fundamental challenge for quadrotors, particularly when strong wind disturbances arise, which perturb vehicle dynamics, limit control authority, and substantially increase collision risk. Existing learning-based navigation policies typically rely on obstacle perception and proprioceptive observations, requiring the policy to infer time-varying disturbance effects implicitly and thereby limiting robustness under partial observability. This paper proposes WAND (Wind-Aware Navigation with Disturbance Estimation), a reinforcement learning framework for navigation under time-varying wind disturbances in dense obstacle fields. Specifically, WAND estimates wind-induced disturbance acceleration from historical proprioceptive states using a Temporal Convolutional Network (TCN). This estimation is integrated into the policy via a zero-initialized residual module, WindAdapter, while simultaneously providing feedforward compensation for low-level control. The dual use of the estimate couples disturbance-conditioned navigation with feedforward disturbance rejection. Across 12 wind-disturbed simulation settings, WAND improved the observed success rate by 8.3 percentage points on average relative to feedforward compensation alone. Controlled opposite-crosswind experiments further showed wind-direction-dependent trajectory adaptation. In indoor fan-induced flight tests, WAND succeeded in 18 of 20 trials, demonstrating the feasibility of real-time onboard navigation.
Read the full abstract ▾
All authors Zhonghan Tang, Chenhui Li, Shuai Liang, Zhongrui You, Jianan Li, Bin Zhao, Zhigang Wang, Xuelong Li
Robotics
PathTime-VLA: Path-Time Decoupling for Factorized Post-Training of Vision-Language-Action Policies
Qing Huang, Yifei Yang, Ziqing Zou et al. (+5 more)
Vision-
Language-Action (VLA) policies typically predict actions at fixed time intervals, coupling the route a
robot follows with its execution pace. This coupling complicates adaptation from teleoperation: useful
geometric guidance comes with timing shaped by interface delays and operator behavior. Our key insight is to bring the path-time parameterization of classical motion planning into the learned action representation of a VLA. We introduce PathTime-VLA, which represents motion as a progress-indexed interaction path $X(s)$ and a positive interval-time profile. The latter defines a monotone time law $t(s)$, yielding controller commands $X(s(t))$. For a given path, alternative executions are expressed through the time profile, allowing chunk-wise speed choices without changing the
geometric prediction target. This representation supports a staged post-training procedure: demonstrations and DAgger interventions establish a target-domain prior, Speed-DQN learns execution multipliers from
robot interaction, and Path-AWR uses rollout outcomes to refine the
diffusion path generator. A path-conditioned action expert realizes the resulting motions while maintaining distinct learning interfaces for path generation and execution timing. Across three tasks, the complete method achieves $58/60$ successes versus $57/60$ for PathTime-VLA under BC + DAgger at fixed $1\times$, with approximately $39$-$52\%$ shorter mean completion times over successful trials.
Read the full abstract ▾
All authors Qing Huang, Yifei Yang, Ziqing Zou, Anzhe Chen, Zhenjie Zhu, Yufei Wei, Rong Xiong, Yue Wang
Robotics
UltraLight Luma: A Novel Edge-Deployable Perception Network for Crop-Row Segmentation in Agricultural Robotics
Dhanushka Balasingham, Shamod Ginigaddarage, Hanojhan Rajahrajasingh et al. (+2 more)
Reliable crop-row perception is essential for autonomous agricultural
robots, but low-cost deployment is constrained by computation,
memory, power, and inference time. This paper addresses crop-row detection and navigation-line extraction using UltraLight Luma, a compact encoder-decoder segmentation network with a progressive learning strategy and only 13.1k trainable parameters. The predicted crop-row masks are processed using a post-processing
algorithm to estimate row
alignment for downstream visual servoing. UltraLight Luma improves parameter efficiency over U-Net, YOLOv8, and YOLOv26 while maintaining reliable crop-row detection performance. The model required only 25.23 mJ per inference, demonstrating its suitability for resource-constrained agricultural
robots.
Read the full abstract ▾
All authors Dhanushka Balasingham, Shamod Ginigaddarage, Hanojhan Rajahrajasingh, Nuwan Kodagoda, Rajitha de Silva
Robotics
2DGS-Planner: Rasterization-based Path Planning in 2D Gaussian Splatting Map
Jiwon Park, Dong-Uk Seo and Hyun Myung
Gaussian splatting provides an explicit and efficiently rasterizable scene representation for
robot navigation. However, individual Gaussian primitives may not reliably represent obstacles as they are jointly optimized through alpha-composited rendering from a finite set of reconstruction views. We propose 2DGS-Planner, a path planner for ground
robots that reads planning-relevant geometry from a 2D Gaussian splatting (2DGS) map through rasterization, rather than treating individual Gaussian primitives as obstacles. During offline roadmap construction, multi-view attribution converts rendered normal dispersion into structural scores for non-ground disks supported by the reconstruction views. These scores guide adaptive node sampling on the ground. Path-aligned orthographic queries validate candidate edges, while cylindrical queries estimate local clearance fields that are cached on the edges. During online planning, graph search initializes a route, and path refinement reuses the cached fields while accounting for the
robot's dimensions and ground constraints. Experiments demonstrate improved roadmap connectivity, more accurate clearance estimation, and higher planning success compared with the tested baselines. These results support rasterization as an effective
geometric query interface for planning directly on Gaussian maps. Code and data are available at https://2dgs-planner.github.io/
Read the full abstract ▾
Robotics
From Solo to Ensemble: A Hierarchical Framework for Composable Multi-Agent Human-Object Interaction
Zekai Deng, Kangyi Chen, Ye Shi et al. (+1 more)
Physics-based human-object interaction has achieved robust single-
agent manipulation skills, yet extending them to multi-
agent cooperative tasks remains challenging. Existing approaches typically adapt interaction policies through task-specific fine-tuning, which entangles low-level contact-rich execution with high-level coordination and limits reuse across object geometries, interaction types, and team sizes. We propose a hierarchical framework that converts a single-
agent HOI policy into a reusable Object-oriented Motion Skill. Specifically, we reinterpret teacher rollouts as object-oriented action supervision by extracting short-horizon object-proxy motions from executed trajectories, and distill task-specific teachers into a low-level skill operating in an Object-oriented Action Space. For downstream tasks, the distilled skill is frozen as a reusable executor, while a high-level policy coordinates multiple
agents by generating region-wise object-oriented actions conditioned on the shared object, task goal,
agent states, and local manipulation regions. This formulation shifts multi-
agent HOI learning from direct contact-rich full-body control to compact object-level proxy-motion coordination. Experiments on diverse HOI tasks show that the distilled Object-oriented Motion Skill supports robust proxy-motion execution and enables composable policy learning across different interaction types, object geometries, and team sizes.
Read the full abstract ▾
All authors Zekai Deng, Kangyi Chen, Ye Shi, Jingya Wang
Robotics · Cross-listed from cond-mat.soft
Autonomous thermodynamic cycles via robotic mobility and sensing
Sofia Kuperman, Ezra Ben-Abu, Yaron Veksler et al. (+3 more)
Thermodynamic cycles are the foundation of energy conversion across natural and engineered systems, transforming heat into useful work. However, these cycles traditionally operate between fixed thermal reservoirs, restricting them to specific locations and temperature differences. Here, we introduce autonomous thermodynamic cycles enabled by robotic mobility and sensing, allowing
robots to perform thermodynamic cycles by accessing spatially varying temperature fields. We experimentally realize this concept using multistable gas-filled capsules that circulate within the system across a thermal gradient. Our model reveals that rapid transitions in the capsules' energy states allow the system to operate as a mobile heat engine that harvests and stores energy. By linking the capsule-scale internal energy dynamics to the
robot's large-scale navigation strategy, we optimize locomotion paths that balance motion cost and energy harvesting. These findings demonstrate that thermodynamic cycles can emerge when autonomous systems navigate their environments, offering an artificial analog of organisms that forage for energy across spatial resources.
Read the full abstract ▾
All authors Sofia Kuperman, Ezra Ben-Abu, Yaron Veksler, Anna Zigelman, Sefi Givli, Amir D. Gat
Robotics
YOCO: You Only Calibrate Once! Fast Mocap Calibration for Dexterous Teleoperation
Yu Zhang, Yunqi Li, Yushi Du et al. (+2 more)
Dexterous teleoperation requires reliable human-hand state estimations. However, common low-cost motion-capture gloves and markerless trackers often exhibit biases that vary across users, glove fit, and recording sessions, degrading retargeting and demonstration quality. We present YOCO, a fast few-shot, fine-tuning-free calibration framework that corrects biased hand-pose streams from a small set of paired raw and target poses. Instead of optimizing a separate model for every operator or session, YOCO conditions a calibration HyperNet on the paired examples and predicts LoRA-style updates for a frozen MANO hand-estimation module, turning per-user calibration into a lightweight feed-forward adaptation step while preserving the
geometric prior of MANO and the efficiency of a compact
estimator. We train YOCO with synthetic drift augmentations on InterHand2.6M and evaluate on augmented InterHand sequences, offline real glove data, and dexterous teleoperation tasks. Across these settings, YOCO improves calibration efficiency, hand-state estimation quality and teleoperation performance compared with uncalibrated input and standard calibration baselines.
Read the full abstract ▾
All authors Yu Zhang, Yunqi Li, Yushi Du, Yi Ma, Yanchao Yang
Robotics
Scalable LEO Conjunction Screening using Adaptive Synthetic-Covariance Thresholds
Vedant Srinivas, Grace Ra Kim, Duncan Eddy et al. (+1 more)
As orbital populations grow, conjunction screening must search more object pairs for potential collision risks. Existing pipelines use early-stage filters, such as an apsis filter, to eliminate pairs whose orbital altitude ranges cannot produce a close approach. However, these filters apply the same fixed threshold to every object, making the screening unnecessarily conservative for well-constrained objects while potentially insufficient for objects with greater orbit-prediction uncertainty. In this work, we present an uncertainty-aware conjunction screening method that replaces a single fixed apsis threshold with object-specific thresholds derived from propagated radial uncertainty. The resulting thresholds vary across both objects and prediction time, becoming narrower when propagated uncertainty is small and wider when uncertainty grows. We pair this adaptive screening rule with a binary-search altitude sweep that makes resulting heterogeneous intervals efficient to evaluate at catalog scale with log-linear search complexity. We evaluate the method on 33,791 objects from the Space-Track General Perturbations (GP) catalog using the preceding eight public ephemeris snapshots and a seven-day prediction horizon. The adaptive sieve retains 162.8 million candidate pairs, reducing the candidate set by 28.78% relative to a fixed 50 km apsis threshold and by 71.48% relative to brute-force all-pairs screening. It retains all close approaches identified in two independently propagated reference sets. At the largest tested catalog size, the binary-search sweep is 27 times faster than exhaustive interval evaluation while returning the same candidate set. These results demonstrate that uncertainty-aware screening can substantially reduce the candidate burden passed to downstream conjunction analysis, providing a scalable path toward managing increasingly large and heterogeneous orbital populations.
Read the full abstract ▾
All authors Vedant Srinivas, Grace Ra Kim, Duncan Eddy, Mykel J. Kochenderfer
Robotics
Neural Networks for Temporal Pattern Recognition and Dynamic Arm Gesture Speed Estimation for Robot Control
Milán Zsolt Bagladi and László Gulyás
Deploying intelligent robotic systems that interact with humans through gestures requires neural networks capable of recognizing diverse temporal patterns. We present a systematic
benchmark of ten abstract sequential tasks–five permutation-invariant (set) and five order-dependent (sequence) problems–evaluated across eighteen neural network architectures spanning recurrent, convolutional,
attention-based, and set-function families. Beyond the core architecture-task grid, we explore numerous preprocessing and target-variable transformations, yielding more than 250 distinct experimental configurations. All variants are trained and tested under strictly identical conditions (fixed random seeds, shared hyperparameters, shared data splits) to ensure fair and reproducible comparison. Ranking across all ten tasks reveals four consistently top-performing architectures–BiGRU, TCN, Conv1D, and GRUReLU–all compact enough for real-time deployment (under 2,000 parameters in the
benchmark setting). Based on this ranking, we apply three architecturally diverse top models (BiGRU, TCN, and GRUReLU) to a practical robotics problem: estimating the execution speed of dynamic arm gestures from skeletal keypoint sequences. Three speed interpretations (peak count, period time, and mean spike spacing) are evaluated on a custom
dataset of eight traffic-related gesture classes comprising 256,710 frames recorded via OpenPose. The best configuration achieves a mean absolute error of 0.198 on the peak-count interpretation, corresponding to roughly 5% relative error, while the period-time interpretation reaches approximately 4% relative error, and the mean spike spacing interpretation approximately 8% relative error. These results demonstrate that neural networks can reliably estimate gesture speed from skeletal data, opening a path toward speed-aware gesture-controlled robotic systems.
Read the full abstract ▾
Robotics
Learning Language-Conditioned Traversability Representations for Adaptive Visual Navigation
Senda Chen, Changxu Cheng, Fangdi Li et al. (+2 more)
Traversability is essential for visual navigation but varies with
robot capabilities and user preferences. Conventional pipelines often rely on explicit costmaps or segmentation masks with predefined criteria, requiring hand-crafted rules and careful tuning. Moreover, viewpoint-dependent segmentation masks complicate asynchronous planning under perception latency. We present LaTraNav, a framework that learns
language-conditioned traversability representations for adaptive visual navigation. Its asynchronous architecture combines a slow
vision-
language model that produces latent representations of traversability and navigation goals, with a fast flow-matching planner conditioned on these representations. To train the system, we develop a simulation-based data generation pipeline with controllable trajectories, producing observations paired with
language instructions, traversability maps, goal locations, and diverse trajectories. Photorealistic image translation further enhances visual realism. Evaluations on
datasets from multiple sources demonstrate effective
language-guided traversability segmentation and goal localization by the slow VLM, alongside adaptive pixel-space path planning by the fast planner. Latent conditioning improves planning performance over explicit segmentation masks, while asynchronous scheduling increases the path-update rate by $6.05\times$ at the same semantic-update rate.
Read the full abstract ▾
All authors Senda Chen, Changxu Cheng, Fangdi Li, Tao Wang, Wuyue Zhao
Robotics
Acting from Belief, Looking When Needed: A Bayesian Spatial World Model for Navigation under Intermittent Perception
Feihong Yang, Xiang Long, Jincheng Yu et al. (+4 more)
Robot navigation commonly uses wide-coverage, high-frequency sensing to reduce partial observability; this reliance becomes restrictive when another task temporarily redirects a shared sensor from navigation, interrupting navigation-relevant observations. We study navigation under intermittent perception: acting from an internal spatial belief and looking again only when execution needs a new observation, potentially freeing the shared sensor for other tasks between navigation observations. ALONE, a
Bayesian spatial world model, propagates a structured spatial belief using executed actions and corrects it with selectively acquired observations; learned priors over common
geometric structures infer unobserved structure from available observation history. It decodes the belief into a spatial estimate for the motion-planning module and predicts a reliability map expressing confidence in the estimate's accuracy. ALONE requests an observation only if insufficient reliability hinders navigation and new evidence should make relevant-region spatial information more reliable; otherwise, it continues acting from the propagated belief. We instantiate ALONE for drone navigation with intermittent single-camera depth images. Across two simulated scene families, it achieves 98% and 97% closed-loop success at a 10 Hz decision rate. Among successful trials, median fractions of decision steps requiring a new depth observation are only 0.9% and 1.3%, respectively, demonstrating high navigation success with substantially reduced observation demand. Real-world indoor flight experiments further validate navigation under intermittent depth observations, with all 10 trials successful.
Read the full abstract ▾
All authors Feihong Yang, Xiang Long, Jincheng Yu, Jianfei Zhang, Guangjun Ge, Chao Wang, Yu Wang
Robotics
SDPAD: A Fully Spike-Driven Pipeline for End-to-End Autonomous Driving
Chengjun Zhang, Yuhao Zhang, Jie Yang et al. (+1 more)
End-to-end autonomous driving demands trajectory planners that are both highly accurate and cheap enough for edge deployment. State-of-the-art artificial neural network (ANN) planners meet the accuracy requirement at the cost of heavy dense computation, while spiking neural networks (SNNs)—though promising orders-of-magnitude energy savings through sparse, event-driven arithmetic—still lag far behind in planning accuracy. We present SDPAD, a fully spike-driven end-to-end planning pipeline that closes this gap. SDPAD converts a pre-trained ANN perception stack into integer-spike form via quantized ANN2SNN conversion, lifts multi-view images into the bird's-eye-view (BEV) space with a spike-driven-max (SDM) depth distribution (Spike-3D-Lift), and plans through the Spike-QFormer, a spiking query
transformer in which ego,
agent, and map queries distilled from the BEV scene are fused by learnable waypoint queries via cross-
attention, followed by deformable spike-cross-
attention refinement. Every operation is gated by integer spikes and inference is a single feed-forward pass without temporal simulation loops. On the nuScenes open-loop
benchmark, SDPAD achieves an average $L_2$ error of 0.40\,m and a collision rate of 0.12%, on par with strong ANN planners while consuming 69.9\,mJ—less than 2% of recent ANN baselines. In closed-loop evaluation on the NAVSIM navtest split, SDPAD reaches 86.3 PDMS, surpassing the previous SNN planner SAD by 4.3 points and matching mainstream ANN planners at a fraction of their energy. To our knowledge, SDPAD is the first fully spike-driven planner evaluated in end-to-end autonomous driving, demonstrating that SNNs can rival dense ANNs in complex driving tasks.
Read the full abstract ▾
All authors Chengjun Zhang, Yuhao Zhang, Jie Yang, Mohamad Sawan
Robotics
CoCam4D: Geometry-Aware Cooperative 4D Perception for Camera-Only Autonomous Driving
Soham Pahari, Sudip Das, Arindam Das et al. (+1 more)
Autonomous vehicles often suffer from limited perception due to occlusions, blind spots, limited sensor range, and the complex nature of surrounding environments. Multi-
agent collaborative perception (CP) addresses these challenges by allowing vehicles to share sensory information and reconstruct the scene cooperatively. However, camera-only perception remains fundamentally limited by the uncertainty of distance-dependent monocular depth estimation. We propose CoCam4D, a
Bayesian framework for collaborative perception that explicitly models
geometric uncertainty. It uses a VGGT-based feedforward network to generate 3D Gaussian scene representations with associated uncertainty estimates, enabling multiple vehicles or
agents to efficiently combine their observations. By sharing compact Gaussian primitives, reliable observations from one
agent can reduce the depth uncertainty of another without requiring
LiDAR sensors. To support real-world deployment, we introduce Dynamic Object Primitives (DOPs), a compact 35-byte representation designed for efficient C-V2X communication. Extensive experiments show that our proposed method consistently outperforms recent
vision-only methods, achieving improvements of 11.48% on OPV2V+ and 10.62% on DAIR-V2X-C, demonstrating the potential of geometrically grounded collaborative perception for
LiDAR-free autonomous driving.
Read the full abstract ▾
All authors Soham Pahari, Sudip Das, Arindam Das, Ujjwal Bhattacharya
Robotics
RAGNAROK: Radar-Aided Gravity-Normalized Alignment for Robust Open Keyframe-based Radar-Visual-Kinematic-Inertial SLAM
Hanjun Kim, Chiyun Noh, Sangwoo Jung et al. (+5 more)
Legged
robots offer superior mobility in unstructured environments, but reliable operation in such conditions requires robust state estimation. To address the vulnerability of proprioceptive
estimators in rough terrain, recent methods have incorporated radar to provide velocity measurements. However, their limited yaw observability still leads to drift, and failure-aware fusion for adverse environments remains underexplored. In this letter, we present RAGNAROK, the first radar-visual-kinematic-inertial SLAM designed for robust operation in challenging environments. It integrates slip- and rolling-contact-aware leg velocity estimation, a kinematics-aware radar factor, and degradation-aware image enhancement. We further incorporate a B-spline-based radar-aided proprioceptive backbone, adaptive weighting, and online extrinsic calibration. Extensive experiments on public and self-collected
datasets demonstrate that RAGNAROK achieves robust performance under challenging conditions and outperforms state-of-the-art baselines. The source code and
dataset are available at https://github.com/hanjun815/RAGNAROK.
Read the full abstract ▾
All authors Hanjun Kim, Chiyun Noh, Sangwoo Jung, Jaehyung Jung, Simon Boche, Cedric Le Gentil, Stefan Leutenegger, Ayoung Kim
Robotics
WARP-VLA: Wrist-Camera Adaptation for View-Robust Policy Execution in Vision-Language-Action Models
Junmyeong Lee, Dongmin Shin, Min-Gyu Park et al. (+3 more)
Despite recent advances in
Vision-
Language-Action models (VLAs) for robotic manipulation, their performance remains sensitive to changes in camera configuration. The problem becomes more evident in cross-setup deployment, as reproducing the exact camera pose used for training is nearly impossible. Unlike fixed external views, wrist views are more challenging because the camera moves with the
robot, causing even small mounting variations to alter fine-grained
geometric cues. To address this, we propose WARP-VLA, a camera-view robust VLA for diverse wrist camera configurations. WARP-VLA adopts a Mixture-of-Experts (MoE) architecture where individual experts learn view-specific feature transformations, and a router combines them based on implicit view information. This allows the policy to be deployed without requiring camera extrinsic parameters as additional input. Through experiments on the LIBERO
benchmark, WARP-VLA improves the average success rate of pi-0.5 from 39.2% to 78.3% under wrist-view perturbations. The real-
robot experiments further show that the feature-level adaptation learned in simulation successfully transfers to diverse deployment settings. To facilitate reproducibility and future research, we release our wrist viewpoint robustness
benchmark and a plug-and-play implementation.
Read the full abstract ▾
All authors Junmyeong Lee, Dongmin Shin, Min-Gyu Park, Wooseok Jeon, Inho Chang, Hae-Gon Jeon
Robotics
DAMP: Humanoid Locomotion via Denoised Belief Learning and Adversarial Motion Priors
Puying Shen, Wenhao Cui, Huaxing Huang et al. (+4 more)
Humanoid
robots possess the structural capability to traverse complex terrains. However, achieving stable t raversal without relying on perceived information remains challenging, particularly in complex environments. This paper introduces DAMP, a reinforcement learning framework aimed at achieving robust and naturalistic humanoid locomotion over challenging terrains, with the assumption that no perceived information is available. The framework leverages recurrent neural networks to capture temporal dependencies and implicitly infer privileged and other task-relevant latent information. By aligning the learned representations with the task objective, the method enables robust and goal-consistent policy learning. This end-to-end framework achieves transfer learning from simulation to real-world environments, demonstrating the proposed method's robustness and generalization capabilities. The
video of the real-world demonstration can be found at the following link: https://youtu.be/AkI7TZB2DDM.
Read the full abstract ▾
All authors Puying Shen, Wenhao Cui, Huaxing Huang, Bangyu Qin, Shengtao Li, Ziyang Dong, Guoteng Zhang
Robotics
RoboAware: Learning to Coordinate Embodied Skills from Counterfactual Outcomes
Bohan Zhou, Xingbei Chen, Emily Huang et al. (+17 more)
Embodied coding
agents can combine modular
robot skills with frozen end-to-end policies, yet effective composition requires anticipating which policy family will succeed in the current physical state. We present RoboAware, which builds on coding
agents' skill orchestration by learning only a state-conditioned responsibility coordinator from counterfactual outcomes. Inspired by the success of REPL, we propose the $P^5$ schema and formulate a hierarchical MDP based on it. $P^5$ organizes skills uniformly into five semantic stages, defining where responsibility can be compared. To address the lack of counterfactual branch outcomes in existing work, we introduce State-Locked Counterfactual Branching (SCB), which restores the same training state to generate and execute a code block from each admissible family, exposing outcomes that selected-branch experience leaves unobserved. Building on this, we propose Execution-Aware Learning (EAL), which combines Monte Carlo tree search with Q-learning to distill these outcomes into family-conditioned values. At deployment, the coordinator selects the policy family according to observable context, and the frozen coding
agent generates the next local code block. Comprehensive single-episode evaluations on 100 tasks show that RoboAware reaches a 77.0% overall success rate, with SOTA averages of 90.0% on RoboSuite, 73.8% on diverse LIBERO-Pro task clusters, and 90.0% on challenging RoboTwin bimanual tasks, outperforming existing code-as-policy and VLA-harness baselines.
Read the full abstract ▾
All authors Bohan Zhou, Xingbei Chen, Emily Huang, Weilin Ruan, Haojian Huang, Yehang Zhang, Zexi Li, Wenqian Li, Qize Yu, Zetian Song, Leyi Wu, Jinghao Li, Mingxuan Song, Xinrun Xu, Zongyang Qiu, Yangkai Wei, Tianyi Zhang, Kaiwen Zhou, Yinchuan Li, James Cheng