Robotics
Spotter: Let the Embodied Model Lead, and the VLM Reflect for It
Long Li, Qichao Zhao, Yue Yang et al. (+6 more)
Current embodied models do not respond to their own failures, although what just went wrong could inform a small adjustment on the next attempt, the kind of reflection behind the gains of thinking in
language models. We test whether they can repair a known error, which requires producing a correction and judging whether it is right. Stopped at a failure and allowed to retry, they seldom repair it through their own randomness or from a
language description of the error, and best-of-N selection cannot pick the successful candidate after a failure. We attribute this to training only on successful demonstrations and to inputs too narrow to show what went wrong, and conclude that reflection must come from a
vision-
language model (VLM), which takes in far more information, such as the episode history and text, and is more general. Prior VLM-led work has the VLM plan every step and invoke the embodied model as a tool, placing the VLM on the critical path. We propose Spotter, which reverses the roles: the embodied model leads and executes continuously, while the VLM runs in parallel, monitors through a lightweight local screener, intervenes only when an error is detected, reflects on and corrects it, and returns control. We run Spotter with Qwen and with GPT as the VLM, and both improve the embodied models; with GPT, Spotter improves Cosmos Policy and $\pi_{0.5}$ by 5.6 and 7.5 percentage points on RoboCasa, and raises $\pi_{0.5}$ from 47.2% to 57.0% on the Hard setting of RoboTwin 2.0 and from 53% to 83% on a real
robot. Because the VLM steps in only when an error is confirmed, a successful episode with Qwen takes only 13 to 16 s longer than with the embodied model alone and about 70% less time than with a VLM-led baseline using the same model. Our code is available at https://github.com/zqc3117/Spotter.
Read the full abstract ▾
All authors Long Li, Qichao Zhao, Yue Yang, Fan Xu, Zhe Wang, Alan Wee-Chung Liew, Chao Qu, Heng Tao Shen, Shirui Pan
Robotics
TaRL: Learning General and Physical Rewards from Tactile Demonstrations
Po-Yi Wu, Dao-Jan Chang, Shang-Ya Hsiao et al. (+3 more)
Contact-rich manipulation requires
robots to sequence precise contacts, maintain stable grasps, and apply directed forces. Reinforcement learning (RL) can acquire such behaviors automatically, but its performance hinges on reward design: sparse rewards reduce the learning efficiency, while dense rewards are hard to specify. Visual reward learning addresses this by inferring rewards from action-free demonstrations. Because it conditions only on visual observations, it fails to capture rewards beyond visual goals. We propose Tactile Reward Learning (TaRL), a framework that learns rewards from tactile demonstrations. TaRL takes a sequence of tactile deformation maps as input, and regresses task-completion progress from both successful and failed demonstrations. Because TaRL captures local
robot-object interaction, it provides informative feedback to learn firm grasps and correctly directed forces; meanwhile, it is robust to changes in scene layout such as object position. We evaluate TaRL on four manipulation tasks in simulation and two in the real world. Used as a shaping reward, it substantially improves both sample efficiency and final success rate, raising success on Nut threading from 34% to 56% in simulation and on cube pickup from 37% to 97% in the real world. Combining tactile with visual rewards improves performance further. TaRL also generalizes across object instances: trained on box placement and directly deployed to can placement, it significantly improves policy learning on the new task. Project page is available at https://embodiedai-ntu.github.io/tarl.
Read the full abstract ▾
All authors Po-Yi Wu, Dao-Jan Chang, Shang-Ya Hsiao, Hong-Ming Chen, Yu-Cheng Su, Tsung-Wei Ke
Robotics
Scale-Invariant Manipulability Shape Tracking Across Heterogeneous Manipulators
Geunwoo Kwon, Dong-gyu Lee, Kai Li et al. (+2 more)
When transferring manipulability across systems with different sizes and kinematic structures, matching absolute ellipsoid scale may be unnecessary when the goal is to reproduce orientation and semi-axis length ratios. Full-
matrix tracking, however, penalizes both shape and absolute-scale differences, even when only shape matching is required. We therefore propose a scale-invariant manipulability shape-tracking method that treats
matrices differing only by a positive scalar factor as equivalent and uses their unit-determinant representatives. We derive the differential of the unit-determinant shape representative and an orthonormal coordinate representation of the tangent tracking residual under the affine-invariant Riemannian metric (AIRM). The resulting scale-invariant objective is integrated with position and end-effector direction tasks in a constrained joint-velocity quadratic program. Simulations with four heterogeneous
robots evaluate
robot-to-
robot and human-to-
robot transfer. On three followers, the proposed method achieves endpoint shape distances of 9.30 x 10^-5 without scale tuning. With
robot-specific target scales tuned during motion, the Full method retains endpoint axis-ratio errors of 0.19-0.31 on KR500 and UR20. For human reaching with concurrent tasks, the proposed method yields dual force shapes elongated along X like the human target on all four
robots, with endpoint position errors of 2.4-5.6% of reference arm length versus up to 75% for the Full method tracking the original human ellipsoid.
Read the full abstract ▾
All authors Geunwoo Kwon, Dong-gyu Lee, Kai Li, Soonwoong Hwang, Wansoo Kim
Robotics
DRHeC: Differentiable Rendering for Hand-Eye Calibration with RGB-Based Gradients
Xiaotian Zhang, Yusheng Wang, Naoya Kagawa et al. (+4 more)
Accurate hand-eye calibration is crucial for precision manipulation. Traditional methods rely on markers, with their precision dependent on marker accuracy and observability. In contrast, markerless methods, such as learning-based approaches, use deep neural networks to directly extract keypoints or features from images, enabling the computation of hand-eye transformation with a single image and without the need for physical markers. Recently, differentiable rendering-based methods for hand-eye calibration have leveraged physical models to render binary masks and compare them with observations, enabling hand-eye calibration without fiducial markers in the calibration stage and providing interpretable optimization. While the state-of-the-art differentiable rendering methods achieve remarkable accuracy, the use of binary masks can result in the loss of internal profile details, reducing precision. Additionally, these methods can also suffer from unstable optimization and local minima. In this study, we propose a novel RGB-based differentiable rendering framework that provides richer
geometric and appearance cues by incorporating color and mask
geometric features, thereby improving calibration accuracy and optimization stability. Additionally, we propose a mask-guided image-to-image translation method to ensure explicit preservation of color and
geometric consistency throughout the translation. Our approach is validated through both simulation and real-world experiments, with results demonstrating strong accuracy and robustness and clear improvements over existing differentiable rendering methods. Our method achieves a grasping success rate of 88.9% and insertion success rate of 57.4% on the UR5e real-world experiment, outperforming the state-of-the-art differentiable rendering hand-eye calibration method EasyHeC by 46.3 and 48.1 percentage points, respectively.
Read the full abstract ▾
All authors Xiaotian Zhang, Yusheng Wang, Naoya Kagawa, Noritaka Takamura, Keiji Okuhara, Hiroyasu Baba, Jun Ota
Robotics
LexiconVLA: Learning Reusable Atomic Action Codebooks for Unseen Tasks
Zeming Wei, Jianheng Ye, Xinshuai Song et al. (+3 more)
Vision-
language-action (VLA) models struggle to reuse recurring interactions in unseen tasks. Our diagnostic study reveals that reliable task completion does not imply consistent execution of constituent atomic actions across task contexts. We present LexiconVLA, a retrievable atomic-action lexicon for cross-task reuse. Global and detail codebooks capture shared interaction structure and fine-grained execution variation, respectively, preserving both reusable patterns and execution details. Visual-Atomic Action
Alignment couples trajectory reconstruction from visual state changes with visual outcome prediction from action codes, grounding the lexicon in motion and its effects. We learn these codebooks with trajectory reconstruction and visual
alignment on our AtomAction
Dataset of 57,803 segments from 69 tasks. A planner and scene-aware adapter translate new goals into code-conditioned subtasks for a shared policy, without skill-specific experts or deployment-time parameter updates. Across five policy backbones on 26 RLBench tasks, LexiconVLA largely maintains performance on 18 seen tasks while improving success on 8 tasks held out from policy training. With BridgeVLA, unseen-task success rises from 16.67% to 34.17% (+17.50 percentage points), and overall success reaches 71.08%, the highest among methods with reported results. Real-
robot experiments demonstrate stepwise execution and failure recovery.
Read the full abstract ▾
All authors Zeming Wei, Jianheng Ye, Xinshuai Song, Sirui Chen, Yang Liu, Liang Lin
Robotics
Degeneracy-Orthogonal Geometric Constraints for LiDAR SLAM
Minseo Kim, Yina Kim, Jinhwa Hwang et al. (+1 more)
Autonomous
robot navigation relies on simultaneous localization and mapping (SLAM) to estimate motion and maintain an accurate pose within an environment. However, in axially uniform corridors such as long tunnels and pipelines,
LiDAR odometry is fundamentally limited by unconstrained drift along the feature-weak travel direction. This structural degeneracy cannot be resolved by local scan matching alone. To address this challenge, we propose the Degeneracy-orthogonal Contour Offset Descriptor (DeCOD), a structure-aligned
geometric descriptor for cross-sectional landmarks. Cross-sectional boundaries, such as pipe joints and structural rings, provide metric constraints along this degenerate axis, but distinguishing individual landmarks requires capturing subtle surface variations across nearly identical profiles. The descriptor parameterizes signed normal deviation from estimated boundary contours, and matching explicitly resolves heading ambiguity and decouples first-order contour errors by distortion estimation. Matched landmarks yield
geometric factors that enforce agreement in cross-section position and corridor axis
alignment during pose-graph optimization, correcting longitudinal drift while leaving rotation about the common axis unconstrained. On a public
benchmark and in field experiments, DeCOD achieves robust landmark
retrieval over standard 3D descriptors and successfully stabilizes trajectories across different odometry frontends, reliably constraining longitudinal drift under
geometric degeneracy.
Read the full abstract ▾
All authors Minseo Kim, Yina Kim, Jinhwa Hwang, Alex Junho Lee
Robotics
T$^2$Mem: Learning Test-Time Memory for Robotics
Yize Liu, Huang Huang, Yining Hong et al. (+4 more)
Memory-dependent robotic manipulation requires policies to use information that is no longer available in the current observation. Retaining history alone is insufficient:
memory must preserve information that supports future actions. One challenge is whether a
memory-free foundation model can learn to retain and use historical information from action demonstrations alone, without external
memory support. We introduce T$^2$Mem, a framework that develops this capability within a pretrained
vision-
language-action policy, without external
reasoning models or
memory-specific annotations. T$^2$Mem uses test-time training to encode observation history into compact fast weights through online self-supervised updates, avoiding repeated processing of the full history. An observation-grounded interface extracts
vision-
language information for
memory formation and supplies retrieved context to the action expert. Action supervision shapes what the
memory learns to retain and use, while alternating
memory-policy learning gives each component a fixed counterpart during optimization. Across 16 RoboMME tasks, T$^2$Mem improves average success from 17.93% to 56.83% over the
memory-free base policy and outperforms the recurrent-
memory methods reported in the
benchmark, while controlled profiling indicates at least 3x inference speedup over explicit methods. Project website: https://yzliu84.github.io/T2MEM-project/
Read the full abstract ▾
All authors Yize Liu, Huang Huang, Yining Hong, Zijian Du, Zhi Cao, Li Fei-Fei, Jiajun Wu
Robotics
Kinematic Nonlinear Spatio-Temporal Trajectory Warping for Contact-Rich Dexterous Manipulation Demonstrations
Hyojae Park, Arjun S. Lakshmipathy and Nancy S. Pollard
We present a straightforward but effective method for repurposing existing contact-rich dexterous manipulation demonstrations. Starting from inputs of hand and object trajectories, our method outputs high-quality nonlinear trajectory warps that account for intermediate waypoints, environmental barriers, temporal shifts, and varied start/end configurations. Foundational to our method is the utilization of contact distributions, which we show allows us to reliably compute complex and high-dimensional dexterous hand trajectories following a simple object-centric warp specification pipeline. We evaluate our method across 12 variations sourced from 4 demonstrations in a publicly available
dataset of human hand motion data, perform baseline comparisons, and demonstrate generalization of our approach to different manipulators. Results and code will be made available on publication.
Read the full abstract ▾
Robotics
Where Predictive Supervision Goes Shapes What VLA Policies Learn
Hanseul Kim, Jewon Yeom, Youngjoon Jeong et al. (+2 more)
Future prediction is increasingly used to improve
vision-
language-action (VLA) policies, based on the premise that anticipating scene
evolution encourages representations useful for control. However, forecast quality alone does not establish that a policy has learned a better representation for action. This distinction matters under distribution shift, where successful control depends on preserving spatial state and likely scene change beyond familiar configurations. We study what determines whether predictive supervision improves the visual representation used by a VLA policy. Through controlled comparisons with matched target constructions, prediction horizons, and training conditions, we find that different prediction interfaces produce markedly different forecasts and visual representations, including in the spatial, dynamics, and action information that transfers beyond familiar scenes. We trace these differences to how predictive errors shape the policy's visual stream. Consistent with this controlled finding, VLA policies trained with more direct, scene-matched future supervision show stronger robustness under simulated and physical distribution shifts. Together, our results frame future prediction as a representation-learning design problem whose value for control depends on whether its supervision reaches the representations through which the policy acts.
Read the full abstract ▾
All authors Hanseul Kim, Jewon Yeom, Youngjoon Jeong, Minsoo Jo, Taesup Kim
Robotics
Foundation-Model-Guided Topology-Aware Semantic Risk Fields for Manipulation
Giung Lee, Weihang Guo and Lydia E. Kavraki
Robot motion planning in everyday environments must satisfy hard
geometric constraints while accounting for context-dependent semantic risk. We present a foundation-model-guided,
topology-aware semantic risk field that extends manipulation safety beyond collision avoidance. For each manipulated-object/scene-object pair, a foundation model provides six directional risk weights and a pair-specific spatial decay scale. The method combines these priors with voxelized 3D scene geometry using
topology-aware shielding and geodesic spatial decay. A GPU-parallel backend batches object-level distance and risk computations to construct a dense 3D field that serves as a modular cost for downstream motion planning. We evaluate the field's shielding behavior under full and partial barriers and compare its 3D workspace representation with a pixel-wise semantic-prior baseline. Across three household simulation scenarios, trajectories optimized with the proposed field have lower semantic exposure than collision-only trajectories under the same
geometric constraints. We also evaluate the computational practicality and reliability of the supporting pipeline. Together, these results support the proposed field as a practical
topology-aware semantic cost representation for manipulation planning beyond collision avoidance.
Read the full abstract ▾
Robotics
RoboChrono: A Real Robot Benchmark for Streaming Task Understanding
Yuzhou Wu, Longteng Fan, Zimeng Li et al. (+22 more)
Understanding ongoing
robot manipulation requires models to interpret visual observations in relation to interaction history and task progress. We introduce RoboChrono, a
benchmark for streaming task understanding comprising 39 scenarios and 34,713 evaluation instances, constructed from real
robot executions and complementary bare-hand human recordings. The
benchmark evaluates seven tasks grouped into recognition,
alignment, and temporal grounding, covering action understanding and anticipation, visual correspondence, temporal ordering, and action localization. Zero-shot evaluation of 18
vision-
language models reveals substantial differences across tasks. GPT-6-Astra achieves 98.3% accuracy on Frame Matching but 68.3% on Frame Ordering, while RynnBrain1.1-122B-A10B exhibits a larger gap, reaching 95.4% and 32.9%, respectively. Input ablations on matched questions with five open-weight models further reveal distinct dependencies on visual evidence: removing visual observations reduces Current Action Recognition accuracy by 22.1 percentage points, whereas Next Action Prediction decreases by only 0.7 points. These findings show that strong visual matching does not consistently coincide with strong temporal ordering, and suggest that next-action prediction can be supported by task and action priors even when visual evidence is unavailable. RoboChrono provides a diagnostic setting for examining these differences, highlighting the need for capability-specific evaluation beyond aggregate scores when assessing task understanding in
robot manipulation.
Read the full abstract ▾
All authors Yuzhou Wu, Longteng Fan, Zimeng Li, Yu Wanchan, Ting Zhang, Yiyang Ma, Shihao Li, Wei Ying, Jianbin Qin, Jiajian Jing, Fangwen Chen, Yifan Wu, Zichen Zhang, Ruiqi Yang, Weibin Kong, Yihang Xu, Haoran Liu, Zonghang He, Xuyang Liu, YiFan Xiong, Siteng Huang, Tao Xu, Zhuo Xu, Long Chen, Ruoxiang Li
Robotics
OTRetarget: Joint Robot and Object Motion Retargeting via Optimal Transport
Guillaume Besset, Erwann Carn, Timothée Carecchio et al. (+5 more)
Transferring human motion to humanoid
robots requires adapting the demonstrated motion to the
robot morphology while preserving interactions with the environment. This is particularly challenging for loco-manipulation tasks, where contacts with the ground and manipulated objects must remain consistent despite differences in body proportions. Yet, skeletal motion alone does not fully describe these interactions, and fixing object trajectories limits the adaptation to a new embodiment. In this paper, we introduce OTR ETARGET, a unified approach to jointly retarget
robot and multi-object motion from human demonstrations. Our approach represents surface interactions through signed distances, closest surface points, and relative directions, and uses entropic optimal transport to transfer these quantities across human,
robot, and object geometries. We incorporate the resulting interaction targets into a constrained inverse kinematics formulation that balances contact preservation with motion style and jointly optimizes
robot and object poses at each frame. This formulation accommodates
robot-object and object-object interactions without rescaling the scene or the demonstration. We validate the proposed approach on OMOMO, where it achieves a
robot- object interaction Jaccard score of 87% and a depth error of 8.7 mm, compared with 28% and 29.3 mm for OmniRetarget. Finally, we demonstrate transfer to a physical G1 humanoid using whole-body policies trained with reinforcement learning on the retargeted references, across motions including two-handed box pick-and-place onto a table.
Read the full abstract ▾
All authors Guillaume Besset, Erwann Carn, Timothée Carecchio, Valentin Tordjman-Levavasseur, Fabian Schramm, Yann de Mont-Marin, Justin Carpentier, Ajay Suresha Sathya
Robotics
Executor-aware Candidate Selection via a Feasibility Certificate
Sooin Choi, Soonwoong Hwang and Wansoo Kim
Modular robotic systems often separate motion planning from a downstream executor that enforces state-dependent hard constraints. A candidate that is geometrically valid may therefore be incompatible with the executor's available command set. We present a certificate-based candidate-selection framework that constructs a command witness from the executor hard set at predicted rollout states and verifies it against the original constraints, without changing candidate generation, ranking, or the executor. Across 5,085 geometry-valid numerical evaluations on two
robot models, 795 admitted no executor-feasible command. The certificate is sufficient but conservative: none of the 795 was certified, while 7.09% of reference-feasible cases remained uncertified. In controlled FR3 and fixed-base RB-Y1 simulations, certificate admission frequently changed candidate selection, and a post-hoc exact linear-programming (LP) admission baseline revealed platform-dependent conservatism. Relative to geometry-based selection, certificate admission was associated with lower planner-command coverage and higher nominal tracking error, without a consistent advantage in reached-state interaction reserve. A planner-generated MoveIt/OMPL study further evaluates the same admission rule on externally generated candidate pools.
Read the full abstract ▾
Robotics
HACo: Learning Haptic Active Compliance for Force-Aware Dexterous Manipulation
Naisheng Ye, Yinzhe Zhou, Junkai Zhao et al. (+5 more)
Contact-rich dexterous manipulation requires policies that translate physical feedback into motion commands while regulating interaction loads across evolving multi-contact interactions. This requires haptic observations of contact state and action supervision showing how commands should adapt. Existing policies often overlook complementary fingertip tactile and joint-torque feedback, while common action targets either encode excessive loading or omit motion constrained by the object. We introduce HACo, a Haptic Active Compliance policy that learns force-regulating actions directly from haptic feedback. Compliance-regulated teleoperation converts operator inputs into controller-executable compliant actions that preserve motion intent while regulating loads. HACo learns these actions directly, using command-state discrepancy as auxiliary compliant-intent supervision. It combines local fingertip tactile responses with joint-torque feedback capturing load transmission through the articulated hand, including contacts beyond tactile coverage. A Compliance Grounding Module uses gated haptic cross-
attention to ground action generation in the evolving haptic state, enabling closed-loop force regulation without explicit online contact modeling. We evaluate HACo on a real-world
benchmark covering multi-contact friction, tangential interaction, fragile curved-surface contact, rotational torque, and deformable-object manipulation. Across 20 trials per task, HACo achieves an 83% mean success rate, compared with 35% for the strongest evaluated baseline. These results demonstrate active compliance across diverse force-sensitive dexterous manipulation tasks.
Read the full abstract ▾
All authors Naisheng Ye, Yinzhe Zhou, Junkai Zhao, Yuhang Lu, Checheng Yu, Zhenjie Yang, Pengwei Wang, Hongyang Li
Robotics
Simple Agentic Memory for Generalist Robot Policies
Yuyou Zhang, Yunbei Zhang, Miao Li et al. (+4 more)
Visual-
memory systems commonly retain or compress past observations.
Robot control additionally requires interaction-derived state that no individual frame may explicitly represent, such as persistent identity relations, accumulated progress, or ordered procedures. We introduce Simple Agentic
Robot Memory (SimpleARM), a training-free
memory layer for frozen generalist
robot policies. From the task instruction, SimpleARM specifies what to monitor; frozen perceptual tools maintain compact typed state online; structured access retrieves that state only when a proposed subgoal depends on history; and current-view grounding resolves recalled entities before execution. We evaluate SimpleARM on RoboMME, a
benchmark of
memory-dependent
robot manipulation tasks that require history information no longer available in the current observation. Across all 16 tasks and three policy seeds, SimpleARM achieves 67.17% mean success, compared with 44.51% for the strongest non-oracle baseline. Matched ablations show mechanism specificity: removing relation, reference, progress, or route state produces large losses where the affected state is retrieved for control, while largely sparing other tasks. These results support a state-based view of
robot memory: effective
memory for control is not simply retained visual history, but compact task-relevant state derived from the interaction history.
Read the full abstract ▾
All authors Yuyou Zhang, Yunbei Zhang, Miao Li, Janet Wang, Zijian Jin, Shilong Liu, Ding Zhao
Robotics
Cooperative Multi-Agent Vision-Language-Action Models via Reinforced Fine Tuning
Ruixiao Xu, Wong Lik Hang Kenny, Zhiqian Liu et al. (+12 more)
We study reinforcement learning (RL) methods for cooperative multi-
agent Vision-
Language-Action (VLA) models. This problem is challenging because VLAs are pretrained on large-scale single-
agent data and therefore lack the fine-grained coordination skills required for inter-
robot collaboration. Supervised fine-tuning (SFT) on multi-
robot demonstrations partially bridges this gap, but its performance is bounded by the demonstration data and cannot improve from its own experience. We present a three-stage reinforced fine-tuning (RFT) pipeline for multi-
agent VLAs. First, initialization-aware data collection sweeps over initial configurations and invokes human demonstrations only when the pretrained VLA repeatedly fails, yielding robustness to initialization shift with reduced human cost. Second, offline credit-filtered tuning assigns credit to individual
agents and fine-tunes on per-
agent trajectories with positive advantage rather than on entire joint rollouts. Third, we find existing online RL for VLAs are less effective for hard multi-
agent tasks, which we attribute to noisy co-exploration and unstable updates. We instead use online latent-space fine tuning, which freeze the VLA and perform RL in its latent noise space. We evaluate our multi-
agent VLA with both $\pi_0$ and $\pi_{0.5}$ backbones across 11 tasks in RoboTwin, RoboFactory and real-world manipulation with two Franka
robots. Our multi-
agent VLA improves the average success rate by $+23.1\%$, $+16.4\%$, and $+44\%$ on RoboTwin, RoboFactory, and real-world tasks, respectively. Code available at https://anonymous.4open.science/r/mavla_rft-2BC0/.
Read the full abstract ▾
All authors Ruixiao Xu, Wong Lik Hang Kenny, Zhiqian Liu, Jianing Guo, Hanxiao Li, Kejian Shi, Shuning Zhang, Pu Feng, Yongjia Ma, Yuqing Ma, Kai Chen, Qi Dou, Yaodong Yang, Xianglong Liu, Simin Li
Robotics
Inferring Soil Friction Angle from Robot Foot-Ground Force Histories: A Bayesian Inverse Approach to Proprioceptive Soil Sensing
Dawei Xu and Zhijie Wang
Foot-ground interaction signals recorded by quadruped
robots may enable spatially distributed, in situ characterization of soil strength. As a first step, we test whether the internal friction angle $\phi$ of cohesionless soil can be identified from the force history of a simplified rotating leg. A two-dimensional continuum model implemented with the material point method, benchmarked against measured rotating-leg force histories, generates the training data, and two Gaussian-process surrogates support
Bayesian inversion of the full histories. In matched-model experiments, the framework recovers 14 off-grid friction angles with a median absolute error of approximately $0.1^\circ$ (maximum $\sim 0.7^\circ$); the reported credible intervals contain the true value in every case. These results establish that $\phi$ is identifiable when the forward model is correctly specified, and support further development of proprioceptive soil sensing for spatially variable terrain, with applications from physics-grounded world models for
robot training to post-wildfire slope assessment.
Read the full abstract ▾
Robotics
EquivDP3: A SIM(3)-Invariant Point-Cloud Encoder for Data-Efficient Humanoid Loco-Manipulation
Abu Hanif Muhammad Syarubany and Chang D. Yoo
Visuomotor policies for humanoid loco-manipulation must generalize across object poses and lighting from only a handful of demonstrations. 3D
Diffusion Policy (DP3) conditions a
diffusion-based action generator on point-cloud features, but its PointNet-style encoder has no built-in equivariance to the rotations, translations, and scalings (SIM(3)) that manipulation tasks respect. EquiBot closed this gap for wheeled manipulators with a SIM(3)-equivariant Vector
Neuron Network (VNN) encoder. We extend this to a substantially more complex embodiment, the 43-joint Unitree G1 humanoid, and propose EquivDP3: a two-stage policy where a high-level
diffusion planner with a SIM(3)-equivariant VNN encoder emits 6 Hz whole-body command chunks, executed at 50 Hz by a frozen, pre-trained RL locomotion policy and a differential inverse-kinematics module for the arms, trained end-to-end by behavior cloning. Across two simulated IsaacLab
benchmarks and four non-equivariant baselines (5-100 demonstrations, in- and out-of-distribution), EquivDP3's advantage concentrates in the low-data regime: at 5-10 demonstrations it reaches 67.1% success versus 38.2-52.4% for the baselines, while by 50-100 all encoders converge (74.3-85.2%) and the ordering is no longer meaningful. A proprioception-only control confirms this gap is genuinely perceptual: with the point cloud removed, success drops to 31% vs. 60% (EquivDP3) at 5 demonstrations and 78% vs. 99% at 10, but vanishes by 50-100, showing the high-data plateau reflects a
benchmark ceiling, not five encoders learning the same invariance. The encoder costs only 0.8 ms of extra latency per action chunk over the PointNet encoder it replaces. Baking
geometric symmetry into a hierarchical
diffusion policy's perception backbone is a practical, nearly free way to improve data efficiency for humanoid loco-manipulation when demonstrations are scarce.
Read the full abstract ▾
Robotics
Riemannian Splat Regression Models for Learning Time Fields on Arbitrary Riemannian Manifolds
Anastasios Manganaris, Rohit Banerjee, Gokul Prabhakaran et al. (+1 more)
Motion planning on arbitrary Riemannian
manifolds is an important and difficult problem that frustrates typical planning methods for Euclidean spaces. In particular, motion planning methods that approximate optimal time-to-go functions with neural networks, e.g., Neural Time Fields (NTFields), cannot be directly applied without using ad-hoc coordinate projections into higher dimensions. Using these methods directly without such projections is desirable, as it promises to provide the lowest-possible-runtime method for obtaining optimal plans on high-dimensional
manifolds while using minimal model capacity. In this work, we develop a model that requires no coordinate projection and can learn arbitrary functions on Riemannian
manifolds by combining splat regression models with splats defined by wrapped Gaussian distributions. We successfully apply this model for learning arrival time fields on several Riemannian
manifolds, and we compare the accuracy and model size of this approach with multi-layer perceptrons adapted to work on each
manifold individually.
Read the full abstract ▾
All authors Anastasios Manganaris, Rohit Banerjee, Gokul Prabhakaran, Ahmed H. Qureshi
Robotics
A robust single-sensing-element tactile sensor for concurrent pressure and tackiness detection with real-time signal decoupling capability
Ying Yang, Mingwei Gu, Jia-Sen Xie et al. (+8 more)
Integrating tackiness sensation into the artificial skin of humanoid
robots significantly enhances their cognitive and operational capabilities. However existing tactile sensors face challenges in decoupling of the
multimodal signal and stability. Here we present a surface-soft tactile sensor that incorporates a Hall effect sensor and a soft
magnetic composite within a robust elastic framework. The sensor surface indents under pressure and bulges prominently when retracted from sticky surfaces dynamically altering the Hall sensor-magnet distance. This generates whole-process-traceable and baseline-separated signals enabling real-time differentiation between pressure and pull-off force. This single-sensing-element design facilitates bimodal sensing at the same contact spot while eliminate stress cross-talk enhancing both accuracy and sensitivity. The fusion of a robust framework and magneto-mechanical sensing mechanism equips the sensor with exceptional reliability and excellent signal baseline stability. This tactile sensor holds substantial potential for advancing robotic capabilities in evaluating adhesive properties monitoring rubber aging precisely handling lightweight objects and cognizing natural objects surface characteristics.
Read the full abstract ▾
All authors Ying Yang, Mingwei Gu, Jia-Sen Xie, Xingyu Ma, Yan-Na Lu, Lin Zheng, Jinhui Gu, Junshuai Chen, Yunjie Lu, Denys Makarov, Jin Ge
Robotics
Learning to Explore Hidden Kinematics for Articulated Object Manipulation
Ruiyao Liu, Boshu Lei, Zhuoyang Pan et al. (+1 more)
The kinematics of an articulated object is often ambiguous from
vision alone. Interaction resolves the ambiguity, and active perception methods exploit this by searching for the single action that most sharpens a belief over the kinematic parameters at each step. Such greedy search cannot be extended over a horizon without forward models of the contact and inertial dynamics, which are themselves unknown. We instead amortize action selection into training. We maintain a belief distribution over joint type and parameters, initialized from a generative prior and updated by
Bayesian filtering on the observed part motion. To condition the policy on this belief, we render it as a per-point articulation flow field, the motion that the current posterior predicts for every point on the object. Carrying the inductive bias of articulated motion, this representation generalizes better than a latent encoding of the belief or flow tracked from observation. We train the policy with reinforcement learning, rewarding the
entropy that each interaction removes from the posterior, so that informative exploration becomes learned behavior rather than a search at every step. Our method outperforms previous approaches across door and drawer manipulation on the PartManip
benchmark, and reaches 61.7% success on ArticuRiddle, a new
dataset of objects whose appearance implies the wrong articulation, against 44.4% for the best previous method. Project Website: https://hiddenkinematics.github.io/
Read the full abstract ▾
All authors Ruiyao Liu, Boshu Lei, Zhuoyang Pan, Kostas Daniilidis
Robotics
Planning Oriented 3D Scene Completion via Coupled TUDF Occupancy Representation Learning from Partial Observations
Tianyou Yu, Pengfei Zhao and Chao Xu
Partial observability remains a fundamental challenge in robotic navigation, where limited sensor coverage and occlusions leave large portions of the environment unobserved. Existing scene completion methods primarily focus on improving incomplete mapping or reconstructing partially observed 3D structures, but rarely investigate how scene completion can be designed to benefit downstream tasks such as path planning. In this work, we propose a path-planning-oriented 3D scene completion framework that moves beyond pure occupancy modeling toward a coupled
geometric formulation. Specifically, given partial
LiDAR observations as input, the proposed framework jointly predicts completed Truncated Unsigned Distance Field (TUDF)-based continuous
geometric representations and voxel-wise occupancy maps. This coupled representation allows the network to better reason about obstacle boundaries and free-space geometry. To fully exploit the synergy between the two representations, we introduce a bidirectionally coupled learning scheme, where TUDF features provide dense
geometric guidance to improve occupancy reconstruction, while occupancy features in turn offer complementary structural constraints that refine distance-field estimation. Consequently, the proposed network directly predicts complete occupancy and TUDF representations, allowing seamless integration of TUDF into trajectory planning without post-processing. Extensive experiments on unseen environments demonstrate that the proposed method consistently improves both
geometric reconstruction quality and downstream planning performance.
Read the full abstract ▾
Robotics
Reactive Real-Time Flow Policies via Asynchronous Distribution Alignment
Moritz Zoellner, Reece O'Mahoney, Ioannis Havoutis et al. (+1 more)
Generalist
robot policies such as
vision-
language-action models (VLAs) have achieved remarkable generalization, but their inference delays can conflict with the demands of real-time control. Asynchronous execution avoids pauses between action chunks by predicting the next sequence of actions while the
robot carries out the previous one. In this paper, we study whether asynchronous execution produces the same action distribution as the original VLA. We find that, for non-Markovian demonstrations, asynchronous execution can produce a fundamentally different action distribution, which can limit the policy's reactivity. In our method, we seek to restore this reactivity by aligning the asynchronously produced action distribution with that of the original VLA through two complementary mechanisms. First, Recursive Flow-Field
Distillation trains the asynchronous policy using the VLA's action-generation flow. We characterize the learned distribution theoretically and show experimentally that our asynchronous policy can generate nearly the full range of actions the original VLA would produce, while existing asynchronous methods recover only a fraction of that range. Second, Propose-Resolve prepares multiple action sequences asynchronously and uses the latest observation to select among them based on a lightweight approximation of their likelihood under the VLA's action distribution. Our resulting method matches the original VLA's success on LIBERO and retains about 80% of its success on RoboMimic, about 30 percentage points more than existing asynchronous methods.
Read the full abstract ▾
All authors Moritz Zoellner, Reece O'Mahoney, Ioannis Havoutis, Rohan Paleja
Robotics
Asymmetric Scout-Worker Reconnaissance for Route Validation in Unknown Environments
Kashif Khurshid Noori, Jaskrit Singh, Athanasios Ch. Kapoutsis et al. (+2 more)
This paper studies asymmetric scout-worker reconnaissance in unknown environments, where a small, agile autonomous scout explores routes for a larger worker
robot that must visit an ordered sequence of goal locations. Because the scout has a smaller footprint and greater mobility, a scout-traversable route may be infeasible for the worker; worker feasibility must therefore be inferred from scout observations. This setting is not explicitly addressed by existing exploration and replanning methods, which typically assume a single traversability model and seek optimal paths for the same
robot performing the exploration. We introduce a symbiotic scout-based framework that exploits the scout's superior mobility to explore only the portions of the unknown environment needed to identify worker-feasible path segments connecting the ordered goals. Evaluations in simulated and real-world settings demonstrate that the proposed approach validates feasible routes, repairs blocked segments with validated worker-feasible detours, and substantially reduces scout travel compared to baseline exploration and planning methods. A real-world indoor deployment further demonstrates the scout navigating narrow corridors to identify a worker-feasible route.
Read the full abstract ▾
All authors Kashif Khurshid Noori, Jaskrit Singh, Athanasios Ch. Kapoutsis, Jing Xiao, Constantinos Chamzas
Robotics
Trajectory-Level Mode Guidance for Controllable Diffusion-Based Multi-Robot Motion Planning
Tianyou Yu, Shengze Cai and Chao Xu
Motion planning often admits multiple feasible solutions, making
multimodal generation valuable, particularly for flexible multi-
robot coordination.
Diffusion models naturally learn such trajectory distributions, yet incorporating coarse and partial trajectory priors without restricting generation remains challenging. Such priors indicate a desirable region of the solution space rather than a single solution, motivating conditioned generation that preserves multimodality. In this paper, we guide trajectory generation in the clean trajectory space and progressively incorporate trajectory priors with a timestep-dependent guidance strength. At each reverse
diffusion step, the reconstructed clean trajectory provides a unified space for integrating planning costs and partial trajectory priors. Planning costs are incorporated through gradient-based refinement, while the partial prior is progressively injected at the corresponding noise levels with decreasing guidance strength. This guides generation toward the prior in early stages while gradually releasing the constraint to preserve the inherent multimodality of the
diffusion model. The framework naturally extends to multi-
robot planning by incorporating inter-
robot collision costs. Experiments on single- and multi-
robot planning tasks demonstrate controllable trajectory synthesis, diverse feasible solutions, and safe multi-
agent coordination.
Read the full abstract ▾
Robotics
Distilling Privileged Control Barrier Functions into RGB-Only Safety Filters for Dynamic Visual Navigation
Seungyeon Yoo, Gawon Lee, Seungwoo Jung et al. (+2 more)
RGB-only end-to-end visual navigation policies remain vulnerable to collisions in real-world dynamic environments, motivating a dedicated safety layer. Existing visual Control Barrier Function (CBF) approaches seek to provide safety from RGB observations, but often rely on real-time rendering or explicit scene reconstruction and are primarily designed for static scenes, limiting their practicality for onboard deployment. We propose a teacher-student visual
distillation framework that transfers the safety behavior of a privileged CBF teacher to an RGB-only student filter for dynamic environments. The student maps a short RGB history,
robot velocity, and a nominal control action directly to a safe action, while the teacher uses ground-truth
robot and obstacle states in a real-to-sim dynamic Gaussian Splatting environment. To reduce the teacher-student information gap, the teacher constructs safety constraints only from obstacles observable within the student's RGB history. It also accounts for obstacle-velocity uncertainty to improve robustness to motion variations, while action augmentation exposes the student to diverse safe and unsafe nominal actions to better capture the safety boundary. At deployment, the student requires only RGB observations and
robot velocity, without explicit 3D reconstruction or online rendering. Experiments show that the proposed method outperforms visual CBF baselines and improves the safety of RGB-based navigation policies under dynamic obstacle motion. Project page: https://syeon-yoo.github.io/distill-cbf-site/.
Read the full abstract ▾
All authors Seungyeon Yoo, Gawon Lee, Seungwoo Jung, Inkyu Jang, H. Jin Kim
Robotics
LIBERO-MAX: Do Robot Policies Adapt When the World Changes?
Yunbei Zhang, Zijian Jin, Yuanzhe Liu et al. (+14 more)
Robots must often continue a task after a target moves, the viewpoint shifts, or an obstacle appears, even though their earlier observations and committed actions reflect the previous scene. Many simulation robustness
benchmarks fix external conditions at reset, leaving this temporal challenge underexamined. We introduce LIBERO-MAX, a
benchmark of 8,000 paired cases spanning eight types of changes to geometry, observations, appearance, clutter, and paths. Each pair compares task execution with and without a mid-task event, holding the task, initial state, policy seed, and pre-event action sequence fixed. This controlled comparison distinguishes event-associated regressions from failures already present without the change. Across fourteen current VLA, hybrid, and world-action policies, events reduce success by 11.0-25.7 percentage points. Event profiles reveal shared vulnerabilities to geometry and observation changes, while policy-family rankings interleave. Camera controls show that robustness reflects both competence under the changed conditions and the trajectory from which they are encountered; varying query cadence does not eliminate the gap. Together, the paired protocol and temporal diagnostics establish LIBERO-MAX as a reproducible testbed for diagnosing failures under mid-execution changes and measuring progress toward
robot policies that remain effective as the world changes.
Read the full abstract ▾
All authors Yunbei Zhang, Zijian Jin, Yuanzhe Liu, Janet Wang, Xilun Zhang, Yuyou Zhang, Zhenyu Zhang, Daoan Zhang, Shuaicheng Niu, Gen Li, Jianfei Yang, Jihun Hamm, Ismini Lourentzou, Weirui Ye, Bo Liu, Peter Stone, Marco Pavone
Robotics
Closed-Form Cartesian Forward Kinetostatics for Spatial Multi-Segment Tendon-Driven Continuum Robots
Ke Wu, Fangju Yang, Xiaohui Zhang et al. (+4 more)
Forward kinetostatics of spatial tendon-driven continuum
robots typically requires a nonlinear
equilibrium solve for each actuation input. This paper develops a force-to-Cartesian-configuration model with a closed-form solution in quadratures for spatial multi-segment
robots under tendon actuation. The Cartesian backbone centerline and accumulated material twist serve as generalized coordinates, from which the strain measures and tendon geometry are derived. Variational
equilibrium yields explicit axial and bending relations and establishes zero
equilibrium material twist within the proposed model for admissible longitudinal non-helical routing. The solution is propagated segment by segment without an iterative
equilibrium solve, while retaining axial deformation, spatially varying axial and bending stiffnesses and tendon-routing diameter, and segment-dependent tendon participation. Numerical comparisons with a full-strain
geometric variable-strain model (GVS) yield maximum length-normalized tip-position discrepancies of 8.91 x 10^-6 and 1.01 x 10^-5 for the single- and three-segment
robots, respectively. Mean evaluation times of 1.52 {\mu}s and 2.94 {\mu}s, with corresponding speedups of approximately 1864x and 3348x over the baseline, demonstrate the computational advantage of the explicit force-to-configuration mapping in the reported
benchmark.
Read the full abstract ▾
All authors Ke Wu, Fangju Yang, Xiaohui Zhang, Junda He, Guanjun Bao, Jingang Yi, Jian S. Dai
Robotics
DQ-MPCC: Dual-Quaternion MPCC for Quadrotor Racing
Bryan S. Guevara, Luis F. Recalde, Guanrui Li et al. (+1 more)
Quadrotor racing demands aggressive attitude and progress control while passing through every gate, and conventional quadrotor MPCC formulations state the prediction model in inertial coordinates and the attitude error in the body frame. We present a Dual-Quaternion Model Predictive Contouring Control (DQ-MPCC) for quadrotor racing in which the pose is a unit dual quaternion and the contouring errors are projected onto the tangent space of the dual quaternion
manifold, expressed in the desired body frame: the same rigid-body dynamics as the conventional model, in unified pose-twist coordinates in the body frame. We compare DQ-MPCC against a baseline MPCC through Monte Carlo software-in-the-loop simulations and real-world racing on an eight-gate
circuit of 11x4.5x3.65 m. With the same gains in simulation and hardware, DQ-MPCC keeps every crossing of its completed flights within the prescribed
geometric margin, whereas the baseline exceeds it, its median worst-gate offset growing by 71.5% sim-to-real against a 10.1% decrease for DQ-MPCC. Among the configurations that keep every simulated gate crossing within the
geometric margin, DQ-MPCC also reduces the minimum lap time by 6.7%, and by 10.5% in the real-world flights, while running onboard at 100 Hz.
Read the full abstract ▾
All authors Bryan S. Guevara, Luis F. Recalde, Guanrui Li, Tiago Nascimento
Robotics
Staircase Policy: Streaming Inference for World-Action Models with Large Action Chunks
Guoheng Sun, Chen Chen, Jin Wang et al. (+2 more)
World-Action Models (WAMs) improve robotic manipulation by conditioning action generation on predicted future observations, but future prediction adds further inference overhead to already expensive iterative action generation. Action chunking can amortize this cost over multiple actions, yet performance degrades over long execution horizons because later actions remain conditioned on stale observations. We introduce STAIRCASE POLICY, a streaming inference and training framework that turns a flow-matching VLA into a JEPA-style WAM and partitions a large action chunk into sub-chunks at staggered denoising stages. Near-term actions are executed as soon as they become available, while later actions continue to be refined. At each sub-chunk boundary, the future latent is re-predicted from the latest observation and used to update all unexecuted actions, enabling long-horizon execution without repeated full policy inference. The resulting future-prediction error can further serve as a signal for adaptive chunking. S-WAM achieves 97.7% on LIBERO and 87.9% on LIBERO-Plus, and improves performance across multiple policy backbones and real-
robot tasks. It reaches 292.7 executed actions per second, $3.62\times$ the throughput of conventional execution at comparable accuracy, while reducing time-to-first-action from 123.6 to 73.3 ms. With additional inference optimizations, throughput further increases to 642.9 actions per second.
Read the full abstract ▾
All authors Guoheng Sun, Chen Chen, Jin Wang, Ang Li, Teresa Lv