Robotics
PRIMO: Prior-Informed Odometry from Human-Motion Tracking for Humanoid Robots
Xu Han, Angsong Li, Shaopeng Zhang et al. (+5 more)
Simulation-trained humanoid proprioceptive odometry faces two transfer challenges: training trajectories generated by specific
robot control policies intended for deployment cover only a limited range of motions, while sim-to-real mismatch can make unconstrained predictions unreliable. We address both with Prior-Informed Odometry from Human-Motion Tracking (PRIMO). On the data side, we generate odometry supervision by having the humanoid track diverse retargeted human motions in simulation, decoupling supervision from the deployment policies and broadening the training motion distribution. On the model side, a Prior-Informed
estimator uses physics- and
symmetry-informed priors to structure velocity and rotation prediction and a coarse raw-context pathway to preserve sensor context alongside encoded features, thereby strengthening sim-to-real generalization. Under a unified real-
robot protocol, PRIMO reduces mean error by 31.6%-61.7% relative to the strongest evaluated external baseline in each domain-metric comparison. Across two locomotion-policy revisions, policy specialists exhibit symmetric crossover, whereas Tracking-Locomotion training reduces mean opposite-policy simulation error by 86.8%-94.6%. On real dynamic motion, Tracking-Locomotion training reduces mean error by 69.2%-81.7% relative to training on the union of both deployment policies. Across the tested motion compositions, the Prior-Informed
estimator consistently lowers mean trajectory errors relative to its Unconstrained counterpart in both simulation and real-
robot evaluation. Code is available at https://github.com/Agibot-Spatial-Intelligence/PRIMO.
Read the full abstract ▾
All authors Xu Han, Angsong Li, Shaopeng Zhang, Enyu Li, Peiwen Lin, Chuang Wang, Yuan Zhuang, Haiyu Lan
Robotics
TaskAnchor: Grounding Task State in Reactive VLAs for Long-Horizon Manipulation
Hengyan Liu, Wenlve Zhou, Bo Yue et al. (+7 more)
Reactive
vision-
language-action (VLA) policies suffer from task-state aliasing in long-horizon manipulation, where identical
multimodal inputs call for distinct, context-dependent actions. Given that pretrained VLAs already possess rich control primitives to express diverse behaviors, we hypothesize that the execution bottleneck lies not in policy capacity, but in input ambiguity. In this paper, we propose TaskAnchor, a lightweight adapter that grounds task state by injecting execution context into the VLA's native input space. During post-training, TaskAnchor learns to represent the semantic execution stage as a milestone-supervised coordinate prepended to the
language instruction, while incorporating fine-grained historical evidence via a residual update to the current visual
tokens. This formulation avoids generating complex subtask instructions and leaves the backbone architecture unchanged. Across long-horizon
benchmarks, TaskAnchor delivers substantial gains, achieving approximately 6 times the average success rate of the pi0.5 and X-VLA baselines on RMBench and more than doubling the task success rate of pi0.5 on RoboMemArena. Real-
robot experiments further validate reliable multi-stage execution, with the same policy adapting its subsequent behaviors using earlier human interactions as in-context cues. Our project website is available at https://taskanchor.netlify.app/.
Read the full abstract ▾
All authors Hengyan Liu, Wenlve Zhou, Bo Yue, Yongyi Su, Ruixiang Wang, Zhanqi Zhang, Dekun Lu, Wei Gao, Xiaofen Xing, Kui Jia
Robotics
AR-WAM: A Visual-Conditioned Agent-Ready World Action Model for Robotic Manipulation
Yicheng Jiang, Zesen Gan, Xiaobo Wang et al. (+8 more)
As AI
agents become increasingly capable,
agent-driven robotic control is emerging as a compelling paradigm. However, prevailing
vision-
language-action (VLA) models and world action models (WAMs) still rely on natural-
language instructions to specify manipulation tasks, an ill-suited interface for
agent-driven control: referentially ambiguous, spatially imprecise, redundant with the
agent's inherent
language understanding, and entangling intent with execution. We present AR-WAM, a visual-conditioned,
agent-ready world action model that replaces
language with two complementary conditions: a visual grounding
prompt (a bounding box of the target) denoting the interaction object and location, and a learnable operation
token dictating the atomic skill to execute. Our compact 0.5B-parameter model, with a frozen pretrained visual encoder and no
language encoder, predicts scene
evolution within compact latent states while decoding actions, exposing the policy's intent through explicit, supervisable
reasoning signals. A model-agnostic compatibility layer provides three primitives (detect, execute, and query) so that local VLMs or online
agent APIs can drive the policy directly, with long-horizon
memory and closed-loop error recovery delegated to the
agent side. On RoboTwin 2.0, RMBench, and a real Astribot S1 dual-arm platform, AR-WAM attains the highest average success on standard manipulation (85.7% over the clean and randomized settings) and outperforms all baselines on
memory-dependent and real-
robot long-horizon tasks, improving success rates by 5.9% and 36.7%, respectively, while maintaining the lowest inference latency (14.1 ms). Project page is at https://ar-wam.github.io/.
Read the full abstract ▾
All authors Yicheng Jiang, Zesen Gan, Xiaobo Wang, Tianlun He, Chenxu Zhao, Minghui Wu, Xinyue Wang, Jiaxu Wang, Junhao He, Jianan Wang, Qiming Shao
Robotics · Cross-listed from cs.CV
G6D: Geometric Learning-Free RGB-D 6D Pose Solver for Robotic Manipulation
Yixuan Liang, William Chen, Yunan Wang et al. (+4 more)
6D object pose estimation is fundamental to robotic manipulation and automation. Recent zero-shot methods have significantly improved generalization to unseen objects, but most still rely on large-scale pretrained models with substantial GPU computation and
memory demands. These requirements complicate deployment on robotic platforms where perception, planning, and control share limited computational resources, while learned intermediate representations offer limited
geometric interpretability for task-specific adaptation. To address these limitations, we propose G6D, a learning-free, geometry-driven RGB-D 6D pose solver. Given an RGB-D observation, an object instance mask, camera intrinsics, and a CAD model, G6D generates pose hypotheses through template-based
geometric matching and refines them using silhouette and depth consistency, forming a purely geometry-driven pose estimation paradigm. This paradigm requires neither pretrained visual models nor target-specific training and preserves interpretable
geometric representations throughout pose estimation. Moreover, adjustable hypothesis counts provide flexible accuracy-computation trade-offs, while a CPU-only configuration supports deployment without GPU resources. Experiments on LineMOD and five BOP19
datasets demonstrate advanced performance. Real-world pick-and-place experiments further demonstrate G6D's applicability to robotic manipulation. The complete project is publicly available at https://ai4control.github.io/G6D-Project-Page .
Read the full abstract ▾
All authors Yixuan Liang, William Chen, Yunan Wang, Jizhou Yan, Zhao Jin, Changling Liu, Chuxiong Hu
Robotics · Cross-listed from cs.CV
MaskVLA: Visual Masking Against Trajectory Overfitting of Vision-Language-Action Model
Yuxuan Jiang, Jiaying Huang, Ge Wang et al. (+9 more)
Vision-
Language-Action (VLA) models integrate
vision-
language understanding with executable
robot actions, enabling end-to-end learning for
robot control. However, our empirical analysis reveals that existing models exhibit severe trajectory overfitting when finetuned on limited
datasets. To guide the model in effectively utilizing wrist camera information, we propose MaskVLA, a masking-based fine-tuning strategy. By randomly masking a small portion of the main camera's visual information, the model is guided to autonomously learn more fine-grained, task-relevant, and effective visual features. This process leads to the emergence of robust policies, thereby enhancing the model's capability to tackle complex manipulation tasks and improving its generalization performance. Our method has been comprehensively evaluated on RoboTwin 2.0, achieving an average success rate improvement of 23.2% and 16.8% compared to $\pi_0$ and OpenVLA-OFT, respectively. Furthermore, experiments on real-world ALOHA
robots also demonstrate the effectiveness of our approach.
Read the full abstract ▾
All authors Yuxuan Jiang, Jiaying Huang, Ge Wang, Shenhao Yan, Jiahao Yang, Chengsi Yao, Qi Liu, Qing Zhao, Shuguang Cui, Yiming Zhao, Yatong Han, Zhen Li
Robotics
PINGU: Extending Air-Bearing Spacecraft Emulators with Open-Source Actuators and Learned Control for Contact-Rich Proximity Operations
Ricard Marsal I Castan, Akiyoshi Uchida, Aman Arora et al. (+7 more)
Low-cost planar air-bearing testbeds have matured into a standard proxy for free-flying spacecraft GNC, but they remain largely thruster-only and are rarely equipped for contact-rich, inertia-coupled manipulation. Building on the open-source ATMOS testbed, we contribute a reaction wheel and two force/torque-sensed robotic arms (LEVION) with interchangeable end-effectors, integrated as first-class control actuators through a unified ROS 2 abstraction layer. On top of the software stack we build a reinforcement-learning training environment and digital twin, and a controller that exploits these added degrees of freedom, letting classical optimal controllers and learned policies be swapped on the same hardware without modification. We validate the integrated system, PINGU, across four
benchmark tasks: point-to-pose navigation (classical LQR vs. sim-to-real PPO), dynamic disturbance rejection under arm-induced center-of-mass shifts, reaction-wheel momentum stabilization, and force-controlled docking. The results show that these additions extend an ATMOS-class emulator into the contact-rich regime and bridge classical optimal control and reinforcement learning on one reproducible platform.
Read the full abstract ▾
All authors Ricard Marsal I Castan, Akiyoshi Uchida, Aman Arora, Pedro Lima, Matteo El-Hariry, Anrej Orsula, Francesco Grella, Antoine Richard, Cedric Pradalier, Miguel A. Olivarez-Mendez
Robotics · Cross-listed from cs.CV
Towards robust multimodal 3D object detection via visual foundation models
Ziying Song, Lin Liu, Hongyu Pan et al. (+4 more)
Multimodal 3D object detection is fundamental to robust perception in autonomous driving because it integrates complementary information from
LiDAR and camera sensors. However, existing methods often fail to maintain robustness under out-of-distribution (OOD) corruptions caused by sensor noise, adverse weather, and environmental changes. To address this problem, we propose RoboDistill, a robust and generalizable
multimodal 3D object detection framework that leverages visual foundation models (VFMs), such as the Segment Anything Model (SAM). First, we introduce SAM-AD, a domain-specific pretraining strategy that fine-tunes SAM on autonomous-driving imagery to extract feature representations with rich semantic information. Second, we design the AD Feature Pyramid Network (AD-FPN) to refine and upsample SAM features at multiple scales for seamless fusion with
LiDAR features. Third, we develop the Depth-Guided Wavelet
Attention (DGWA) module, which suppresses high-frequency sensor noise while preserving critical contextual information. Finally, we introduce KD Fusion, in which the pretrained SAM-AD serves as a teacher that distills high-quality visual knowledge into a lightweight point-cloud network, thereby improving robustness under noisy conditions. Extensive experiments across 27 challenging OOD corruption settings show that RoboDistill generally delivers stronger or competitive detection performance and robustness relative to representative state-of-the-art methods. This work bridges the gap between VFMs and 3D object detection and advances robust
multimodal perception for real-world autonomous-driving applications.
Read the full abstract ▾
All authors Ziying Song, Lin Liu, Hongyu Pan, Shaoqing Xu, Lei Yang, Mingzhe Guo, Caiyan Jia
Robotics
Imagine then Verify: Affordance-Targeted Active Perception for Task-Oriented Grasping in Cluttered Scenes
Jingzhi Cui, Xuefeng Liu, Feng Han et al. (+3 more)
Task-oriented grasping (TOG) requires
robots to grasp functional parts of objects (e.g., the handle of a mug for pouring), yet these affordance regions are frequently occluded in cluttered scenes. Active perception via next-best-view (NBV) planning can resolve such occlusions by moving the camera for more informative observations. However, existing NBV methods typically optimize viewpoints for grasping the target object as a whole without distinguishing which part is task-relevant. A naive adaptation, fully scanning the target object before predicting the affordance, wastes most of the viewpoint budget on task-irrelevant surfaces (e.g., the mug body for pouring). To address this, we propose ATAP, an Affordance-Targeted Active Perception framework that shifts viewpoint planning from exhaustive target scanning to targeted affordance verification. ATAP hypothesizes the occluded target geometry via a generative shape prior and predicts the affordance distribution over the imagined complete surface. In cluttered scenes, severe occlusion can make the location of the hidden affordance ambiguous, leaving multiple locations plausible given the partial observation. ATAP therefore introduces an uncertainty-aware viewpoint planner that jointly optimizes expected
entropy reduction over these competing hypotheses and expected affordance verification gain from real observations. This process iterates until the affordance is sufficiently verified for grasp execution. Experiments in simulation and real-world cluttered scenes show that ATAP substantially improves the functional grasp success rate over fixed-view TOG baselines, and outperforms reconstruction-based active perception with over 57% fewer NBV steps.
Read the full abstract ▾
All authors Jingzhi Cui, Xuefeng Liu, Feng Han, Xinyu Liu, Wei Chen, Jianwei Niu
Robotics
Elevator-VIGS: Separating Elevator Motion from Robot Motion in Visual-Inertial Gaussian Splatting SLAM
Rui Zhou, Zihan Zhu, Wei Zhang et al. (+3 more)
We present Elevator-VIGS, a visual-inertial 3D Gaussian Splatting SLAM system that keeps tracking and mapping through elevator rides. Inside a moving elevator, the two sensors are in conflict. The camera sees only the
robot's motion relative to the elevator, while the IMU senses that motion plus the elevator's motion relative to the world. This conflict is challenging for existing visual-inertial
estimators. If
vision dominates, the
estimator tracks only the
robot's motion within the elevator and misses the elevator's rise, and if the conflict remains, the
estimator diverges. We observe that the conflict comes from forcing both observations into a single coordinate frame. We instead estimate the
robot's pose in the elevator's coordinate frame, and the elevator's motion relative to the world as a per-keyframe transport state, the elevator's rise and vertical velocity, within dense visual-inertial bundle adjustment. Elevator-VIGS detects rides zero-shot with a
vision-
language model and a depth network, and constrains the transport state at the departure and the arrival. We record real-world and simulated elevator sequences. On these sequences, Elevator-VIGS achieves state-of-the-art tracking and rendering performance. On four elevator-free public
benchmarks it keeps the state-of-the-art performance of VIGS-SLAM. Project page: https://ruizhou-cn.github.io/elevator-vigs/.
Read the full abstract ▾
All authors Rui Zhou, Zihan Zhu, Wei Zhang, Zizhou Luo, Norbert Haala, Marc Pollefeys
Robotics
FeasibleFlow: One-Step Joint Transport of Configuration Feasibility and Trajectories for End-to-End Driving
Xiang Li, Bikun Wang, Qing Xu et al. (+1 more)
End-to-end autonomous driving maps current observations directly to future trajectories, yet those trajectories must remain valid as the scene evolves. Future state modeling aims to address this temporal mismatch, but general representations often contain information unrelated to ego planning and affect trajectory generation only through auxiliary supervision, static conditioning, or proposal evaluation. We propose FeasibleFlow, a one-step end-to-end generative framework that jointly transports a configuration-space feasibility field and
multimodal ego trajectories. Our Asymmetric Joint MeanFlow uses the pathwise Jacobian-vector product in the MeanFlow identity to incorporate field
evolution into trajectory transport. Because safety feedback is sparser than progress feedback, we further introduce the Anchor-relative ranker (ARR) and Pareto-ReinFlow to balance safety and progress in candidate selection and generation, respectively. Experiments on the NAVSIM
benchmark demonstrate the strong performance of FeasibleFlow and validate both the joint transport of feasibility and trajectories and the proposed safety-first mechanisms.
Read the full abstract ▾
All authors Xiang Li, Bikun Wang, Qing Xu, Jianjun Wang
Robotics
Design and Control of a Cable-Driven Switchable Actuator with Torque/Tension Dual Modes for Exoskeletons
YuanLong Ji, Xu Liu, Xinyuan Cai et al. (+8 more)
Existing wearable exoskeleton architectures are typically constrained by a single mechanical output modality, providing either joint torque around an anatomical joint or linear traction along a limb-training-oriented direction, which limits adaptability to diverse training scenarios. This letter presents a cable-driven switchable actuator (CDSA) that can rapidly switch between torque and tension modes while centralizing all sensing and actuation components at the proximal drive unit. A Coupled Movable Pulley Mechanism (CMPM) provides tension amplification at the distal end-effector, while a bidirectional Cable-Driven Ratchet Mechanism (CDRM) enables mode switching and preload regulation. To eliminate the need for distal instrumentation, multi-source proximal sensors are integrated with a data-driven fusion model to estimate distal output forces. An adaptive dual-mode force control strategy based on iterative learning control (ILC) is further developed. Platform experiments demonstrate transmission efficiencies of $(92.4 \pm 2.0)\%$ and $(96.5 \pm 3.3)\%$ in the torque and tension modes, respectively, along with a tension amplification ratio of $2.77 \pm 0.10$ under tension mode. Tracking tests on simulated knee-joint gait trajectories and short-stroke tension profiles yield stable control, with RMSEs of $(4.52 \pm 0.51)\%$ and $(3.15 \pm 0.19)\%$ of the uncontrolled peak value, respectively. Finally, seated human-coupled experiments validate the system's controllable force generation in both joint-torque and linear-traction application modes.
Read the full abstract ▾
All authors YuanLong Ji, Xu Liu, Xinyuan Cai, Qihan Ye, Xiangyu Xie, Ruizhe Jiang, Shuhan Xiang, Wenjing Liu, Qijun Wang, Yang Chen, Xingbang Yang
Robotics
Cognitive Action Reasoning for Proactive Robots from Human-Centered Multimodal Observations
Zhihao Gu, Kechao Zhu, Yuanfeng Wu et al. (+7 more)
Robots operating in human-centered environments are typically designed to execute explicit instructions, and most
robot-learning
datasets likewise pair observations with task instructions or low-level actions. Although recent work has begun to explore proactive embodied assistance, existing resources target different settings and action levels, leaving real-world human-centered
multimodal decision-making underexplored. We formulate this problem as Proactive
Robot Action
Reasoning (ProRobo), an upstream cognitive decision problem in which a
robot must determine which action to take based on
multimodal human and environmental cues without explicit action instructions. To support ProRobo, we introduce ProAction, a real-world
multimodal dataset containing 10K samples of visual observations, audio signals, and text inputs across 12 daily-life scenarios in five common scenes. To construct cognitively grounded high-level action supervision, we develop a two-stage human-in-the-loop pipeline that combines appraisal-guided candidate generation with Affective Theory-of-Mind-guided human refinement, explicitly incorporating contextual judgment about human states, urgency, feasibility, and potential risk into action annotation. Based on this supervision, we
benchmark representative
Multimodal Large
Language Models (MLLMs) and introduce MMC2Act, a reference model that implicitly learns the mapping from
multimodal observations to cognitively grounded high-level actions. Experiments across modality settings, subject-disjoint generalization, cross-
dataset transfer, and human evaluation show that general-purpose MLLMs struggle with proactively
reasoning high-level actions from
multimodal cues, whereas training on ProAction substantially improves performance.
Read the full abstract ▾
All authors Zhihao Gu, Kechao Zhu, Yuanfeng Wu, Mohan Liu, Ankit Kumar Shaw, ChenDong Hong, Xuanyu Chen, Dengchen Mei, Xu Tianyi, Lin Wang
Robotics
STRIDER: Stepping-Enabled Multi-Gait Hierarchical 3D Loco-Manipulation Framework for Humanoid Robots
Yuanzhuo Li, Wen Zhao, Zhe Yong et al. (+6 more)
Humanoid loco-manipulation faces two prominent limitations: controllers using continuous velocity commands cannot precisely regulate individual footholds, while specialized foothold-tracking modules are difficult to integrate with whole-body manipulation. Furthermore, standard action-based imitation
distillation primarily transfers expert actions, without explicitly encouraging a shared representation of heterogeneous skills. This paper introduces STRIDER, a hierarchical multi-gait framework to bridge these gaps. The framework integrates terrain-aware 3D stepping logic,
Adversarial Motion Priors (AMP)-based natural walking, and Cartesian upper-body control: its stepping expert selects feasible footholds in the stance-foot frame and generates clearance-aware swing trajectories. To fuse distinct walking and stepping experts into one executable student policy, we propose Latent
Distillation Proximal Policy Optimization (LD-PPO), a
distillation algorithm augmented with teacher-conditioned latent
alignment. By jointly optimizing on-policy reinforcement learning, DAgger-based action reconstruction, and latent
alignment, LD-PPO transfers expert actions while encouraging a shared skill representation across heterogeneous modes. Simulation and real-
robot evaluations on the TianGong Omni humanoid show that LD-PPO outperforms vanilla
distillation-PPO in foothold-tracking and posture-tracking accuracy. Deployed on hardware, STRIDER realizes multi-gait loco-manipulation with accurate foothold and end-effector tracking.
Read the full abstract ▾
All authors Yuanzhuo Li, Wen Zhao, Zhe Yong, Xiang Meng, Gang Han, Hengle Ren, Xiaoyang Zheng, Zhen Wang, Yijie Guo
Robotics · Cross-listed from cs.CV
Algebraic Consistency Alone Does Not Certify Temporal Structure in Latent Action Models
Di Wen, Ruodi Zhang, Kailun Yang et al. (+1 more)
Latent action models infer a code for the transition between two frames of action-free
video. Recent methods regularise this code to compose additively and reverse antisymmetrically, and report order-of-magnitude reductions in the resulting errors as a label-free certificate that the code has captured temporal structure. We show that this conclusion does not follow. Reconstruction drives the decoded transition toward a difference of state features, for which both identities hold for any pairing, a solution the metric cannot distinguish from one encoding nuisance state or a coordinate convention. Across five source domains, a trained but unconstrained counterpart already achieves 83-97% of the reduction relative to an untrained anchor. The residual fold is governed as much by the decoder family as by what is learned. A constrained model retrained after its temporal pairing is destroyed still reaches, in each domain, a lower error than the unconstrained model on real data. Downstream, preserving the temporal pairing yields no consistent advantage on LIBERO-GOAL or LIBERO-SPATIAL, and across the tested arms the code's mean linear action decodability falls as the algebraic error improves. We also test the most direct repair, a violation-contrastive objective that requires the
algebra to fail on destroyed pairings: in the tested configurations it yields only a marginal separation within the reconstruction budget, on training and test triples alike. We recommend a validation protocol that these methods currently lack: a baseline-corrected metric, retraining on destroyed pairings, and a seed-budget analysis.
Read the full abstract ▾
All authors Di Wen, Ruodi Zhang, Kailun Yang, Kunyu Peng
Robotics
Smartphone GNSS Booster: Centimeter-Level Pedestrian Positioning Using a Portable Signal Re-Radiator
Taro Suzuki
High-precision positioning using the global navigation satellite system (GNSS) embedded in smartphones is demanded for sidewalk-level pedestrian navigation and pinpoint location-based applications. However, current smartphone positioning accuracy for pedestrians remains at the meter level. This is mainly because limitations of the compact linearly polarized (LP)
antenna in smartphones increase GNSS observation noise and hinder stable carrier phase tracking. In this paper, we propose an external GNSS signal re-radiation system that boosts smartphone GNSS observation quality while still using the smartphone's built-in GNSS receiver and
antenna. The system directly connects a compact active helical
antenna and a thin passive patch
antenna for re-radiation, and is designed to be attached directly to the smartphone. The proposed GNSS booster achieves (1) reduced thermal noise and stable signal tracking via increased received signal strength, (2) improved multipath robustness compared with a LP
antenna, and (3) stabilization of
antenna phase center variation. Static experiments confirm a substantial improvement in the observation quality of GNSS carrier phase measurements compared with standard smartphone positioning. Furthermore, in pedestrian experiments, the proposed booster enabled stable carrier phase integer ambiguity resolutions, achieving centimeter-level positioning using a smartphone.
Read the full abstract ▾
Robotics · Cross-listed from cs.DC
Conflicting Pattern Formation by Teams of Anonymous, Fully Disoriented Robots
Animesh Maiti, Prakhar Shukla and Subhash Bhagat
Two groups of autonomous, anonymous, and oblivious mobile
robots are deployed in the two-dimensional Euclidean plane, each assigned a distinct task. We study a setting where the two groups must simultaneously solve two conflicting pattern formation problems: the gathering problem, where
robots gather at a point not known to them a priori, and the circle formation problem, where
robots occupy distinct positions on the boundary of a circle. Although each
robot knows its own task, it cannot identify other members of its group. A prior solution~\cite{Conflict-1} addressed this problem for asynchronous
robots having direction-only axis agreement and global weak multiplicity detection capability available to all
robots in both groups. In contrast, in this work, we consider fully disoriented
robots without any axis agreement or common chirality. We study the feasibility of a solution to this problem for disoriented
robots. We propose a distributed
algorithm that solves the problem for semi-synchronous disoriented
robots with non-rigid movements. Our proposed
algorithm assumes global weak multiplicity detection only for the gathering group, while for the circle formation group, it requires local weak multiplicity detection.
Read the full abstract ▾
Robotics
BiRoAD: Learning Shared and Role-Adaptive Representations for Bimanual Manipulation
Yan Shen, Yuchen Liu, Feng Jiang et al. (+5 more)
Bimanual manipulation requires policies that coordinate two arms while adapting their functional roles to scene geometry, object configuration, and task context. Learning such scene-conditioned role adaptation remains challenging, as demonstrations may contain uneven role distributions that limit generalization to underrepresented arm–role configurations. In addition, many bimanual policies predict actions in fixed left- and right-arm action spaces. While this provides a natural parameterization for
robot control, it does not explicitly specify how behaviors should transform when functional roles are exchanged across arms. Across different scene initializations, the two arms may follow a similar coordination pattern, but the role-specific behavior assigned to each arm should change with the scene. Therefore, we propose BiRoAD, a Bimanual Role-Adaptive Decomposition framework for learning shared and role-adaptive representations in bimanual policies. Given bimanual trajectory or action-
token features, BiRoAD decomposes these features into swap–symmetric and swap–antisymmetric components: the former captures coordination structure invariant to arm exchange, and the latter captures role-specific distinctions that vary consistently with functional role assignment. The two components are then recomposed as residual updates to the original paired arm representations, allowing BiRoAD to serve as a modular feature transformation without changing the policy inputs, imitation-learning objective, or requiring manually defined role labels. Across multiple bimanual manipulation tasks with balanced and imbalanced role distributions, BiRoAD improves robustness across role configurations over corresponding base policies, with notable gains on underrepresented role configurations.
Read the full abstract ▾
All authors Yan Shen, Yuchen Liu, Feng Jiang, Hangtian Hu, Xiaoqi Li, Shu Chen, Ruihai Wu, Hao Dong
Robotics
Receding-Horizon Pushing with Composable Object-Centric Policies
Zhiyi Yuan, Tianrun Hu, Anxing Xiao et al. (+3 more)
Non-prehensile manipulation is practical for relocating large, heavy, or geometrically ungraspable objects. Yet, long-horizon pushing of arbitrarily-shaped 3D objects couples three problems: 1) where to push the object so as to approach the target pose, 2) whether each push is stable and reachable, 3) whether subsequent actions remain feasible. We present an object-centric pushing policy within a feedback-guided hierarchical framework. At the low level, a learning-based policy predicts contact actions from a pose- and scale-normalized point cloud, conditioned on a near single-step subgoal. A stability score is applied to evaluate the predicted contacts by a quasi-static sliding-versus-tipping analysis. At the high level, BIT$^*$ first searches for an object path, and the next several subgoals are checked by contact prediction and
robot motion planning for future feasibility. Failed motion plans, as feedback, change the local path costs and trigger re-planning. During execution, only the first feasible action is executed. In simulation, we evaluate 22 objects in six different scenes, upon which we also conduct comprehensive ablation studies. Results demonstrate that our method outperforms baselines with a clear margin and can reliably achieve long-horizon object pushing tasks under different situations. We also report quantitative real-
robot experiments with a Franka arm and qualitative demonstrations with a mobile manipulator for large and heavy objects, with directly zero-shot sim-to-real transfer.
Read the full abstract ▾
All authors Zhiyi Yuan, Tianrun Hu, Anxing Xiao, Yuhong Deng, David Hsu, Hanbo Zhang
Robotics
RopeFormer: Cross-Trial Adaptation from Interaction History for Dynamic Rope Manipulation
Menglin Wu, Kaixiang Yao, Shangbo Luan et al. (+2 more)
Dynamic rope manipulation is highly sensitive to unknown object dynamics: the same
robot motion can produce substantially different responses across ropes, while explicitly identifying the relevant physical properties is difficult. We present RopeFormer, a history-conditioned framework that uses prior task interaction as context for subsequent control. The policy retains cross-trial action-response history while keeping its weights fixed and requires no explicit online rope-parameter estimation. In matched simulation evaluations across sustained single-arm rotation, bimanual rotation, and transient whipping, retaining context improves subsequent control relative to resetting the same checkpoint, with the benefit varying across rope dynamics and observation settings. We further deploy the frozen policies on a Unitree H1-2 with previously unseen physical ropes. From T1 to T3, target-acquisition time decreases by 30.9% for Rope Swing and 33.9% for Rope Twirl, while mean Rope Whip target hits increase from 0.2 to 2.3 out of three. These results show that prior interaction can provide effective control context for dynamic deformable-object manipulation.
Robot videos, code, and data are available at https://ropeformer.github.io/.
Read the full abstract ▾
All authors Menglin Wu, Kaixiang Yao, Shangbo Luan, Masayoshi Tomizuka, Yuxin Chen
Robotics
RiverVLN: Phase-Grounded Temporal Vision–Language Navigation for Unmanned Surface Vehicles
Jieling Wu, Yuehao Huang, Jiajun Lv et al. (+3 more)
Vision-
language navigation (VLN) has largely been developed for indoor and terrestrial
robots, where
language can often be treated as a static goal and motion is approximated by discrete or near-instantaneous actions. These assumptions break down for unmanned surface vehicles (USVs): river navigation requires continuous motion under inertia and limited maneuverability, while long-horizon instructions must be executed through sparse and visually ambiguous maritime landmarks. We introduce RiverVLN, to our knowledge the first
benchmark designed for long-horizon USV VLN under continuous riverine motion, and PGT-NAV, a phase-grounded temporal navigation framework for USVs. Rather than directly mapping an entire instruction to motion, PGT-NAV converts it into an ordered sequence of visually verifiable semantic phases and maintains the active phase online through grounded visual and motion evidence. This explicit semantic progress state is fused with visual-motion history and phase-specific grounding to predict six local SE(2) pose increments. The resulting trajectory is executed in a predict-execute-re-observe loop, where the vessel executes toward W3, updates phase and grounding, and replans through a map-based safety layer. Experiments show that PGT-NAV substantially reduces recursive position and heading drift relative to GNM-style and ViNT-style baselines and achieves an average success rate of 0.79 in Unity-ROS closed-loop navigation. Unseen bridge-opening trials and real-world USV experiments further demonstrate that the phase-grounded representation transfers from controlled evaluation to physical USV deployment.
Read the full abstract ▾
All authors Jieling Wu, Yuehao Huang, Jiajun Lv, Tao Huang, Yong Liu, Weiwei Liu
Robotics
HEARTH: An Object-Centric RGB-Thermal-3D Dataset for Temperature-Aware Robot Manipulation
Yuning Su, Borui Li, Yonghao Shi et al. (+2 more)
Language-guided manipulation can depend on physical properties that visible appearance does not reveal. Temperature is one such property, but object
datasets for
robot learning rarely associate measured temperatures with object appearance and geometry. We present HEARTH, an object-centric RGB-thermal-3D
dataset of 90 physical objects from 18 everyday categories, comprising 145 captured object states. Our pipeline maps apparent surface temperatures onto reconstructed meshes through camera calibration and pose transfer. The
dataset includes raw temperature measurements, camera parameters, RGB-textured meshes, and thermal textures for simulation. We use these assets to construct three LIBERO-derived tasks and collect 1,200 demonstrations for fine-tuning a pretrained
vision-
language-action (VLA) model, $\pi_{0.5}$. In an ablation study, adding thermal observations to the VLA increases success on temperature-dependent object-selection tasks from 35.0% for the RGB-only baseline to 75.0%. These results demonstrate the utility of HEARTH for training
robot policies to follow temperature-related instructions.
Read the full abstract ▾
All authors Yuning Su, Borui Li, Yonghao Shi, Bofei Liu, Xing-Dong Yang
Robotics
EmoPose: Vision-Language Model Guided Emotion-Aware Gesture Generation for Humanoid Robots
Daojie Peng, Bingtao Wang, Fulong Ma et al. (+3 more)
Socially competent humanoid
robots must communicate affect and intent through gesture as well as
speech, yet open-ended interaction must become motion that is both expressive and executable on a specific body. This demands semantic flexibility for contextual social intent while preserving deterministic, embodiment-aware
robot control. We present EmoPose, a
vision-
language model (VLM)-guided framework that bridges this gap through an executable semantic interface. Given
language, dialogue history, and optional visual context, the VLM selects an ordered gesture plan containing a communicative class, library variant, intensity, and
speech anchor. A scalable
robot-owned motion library defines the available expressive vocabulary and the source of 14-DoF joint targets. Pose Studio supports automatic trajectory generation, MuJoCo preview, and automatic synchronization of new library entries with the VLM guide; deterministic
robot-side modules validate plans, construct trajectories, schedule gestures, and manage queueing and interruption. This division lets the interaction repertoire grow for new social contexts without changing the control interface or delegating raw joint commands to the foundation model. On the EmoPose-Bench, structured GPT-5.5 planning reaches $98.25\pm0.52\%$ on the Easy tier and $76.50\pm0.54\%$ overall, exceeding same-model direct-label prompting. Further tests validate dialogue-context use and ordered multi-action composition. The system completes the nominal MuJoCo suite and realizes all 29 authored variants on the physical Unitree G1. A four-stop laboratory tour demonstrates expressive narration with interruption, camera-grounded dialogue, and navigation.
Read the full abstract ▾
All authors Daojie Peng, Bingtao Wang, Fulong Ma, Wenjun Yue, Liang Zhang, Jun Ma
Robotics
Towards Reliable Underwater Diver-Robot Interaction: Gesture Design, Interaction Logic, and Real-World Evaluation
Yingqi Liu, Kanzhong Yao, Zimeng Peng et al. (+4 more)
Underwater human–
robot interaction requires gesture commands that are both easy for divers to use and reliable for
robots to recognize. We investigate these aspects through a closed-loop diver–
robot interaction framework integrating a compact seven-gesture vocabulary, lightweight landmark-based recognition, and command-level interaction logic. We evaluate the framework through a user study and underwater
robot experiments in a laboratory tank and a swimming pool. The user study supported the reproducibility of the gestures after brief learning. Recognition analysis further showed that visual similarity was associated with gesture confusion, while intermediate poses during gesture formation introduced temporal ambiguity. Command-level processing mitigated the effects of transient recognition errors on
robot execution, reducing unintended triggers and premature task interruptions. These findings show that reliable underwater gesture interaction depends on human usability, gesture recognizability, and execution reliability in underwater interaction.
Read the full abstract ▾
All authors Yingqi Liu, Kanzhong Yao, Zimeng Peng, Yuanbo Bi, Anran Li, Zhe Sun, Xuelong Li
Robotics · Cross-listed from cs.HC
Identity Continuity in Long-Term Embodied AI Relationships: From Agent-Specific Identity Representation to Identity-Continuity Appraisal
Zijian Ru
Long-term embodied AI will undergo learning, model updates,
memory compression, hardware repair, and migration across embodiments. For users who have formed sustained relationships with such systems, these changes raise not only a problem of product consistency but also one of identity continuity: whether the changed system is still experienced as the same particular
agent. Existing research suggests that human-AI relationships may develop relational particularity, that robotics and artificial-identity research has identified identity and migration signals across embodiments, and that major updates or platform disruptions can be accompanied by relational loss and restoration desire. This article proposes a user-side framework in which long-term embodied AI is represented through an
agent-specific identity representation organized by at least three open identity-content domains: embodied-perceptual, psychological-behavioral, and relational-autobiographical. Information from these domains is not equally weighted; shared history, relational roles, and contingent responsiveness may make some information more identity-diagnostic than others. After system change, users may integrate continuity and discontinuity evidence in a weighted manner, yielding judgments along a continuum from relatively strong identity continuity through ambiguity or partial continuity to clear identity discontinuity. Causal-historical provenance and user participation are treated as contextual evidence rather than a fourth identity-content domain. The framework also proposes identity continuity as a psychological objective for lifecycle design, including
memory selection, model updating, and migration across embodiments, under constraints of
privacy and user control.
Read the full abstract ▾
Robotics · Cross-listed from cs.CV
BiView-Touch: Learning Bimanual Tactile Representations by Cross-Hand Completion
Chenxin Liang, Youchen Lai, Chuqiao Lyu et al. (+3 more)
Bimanual interaction produces complementary tactile views of the same physical process, yet existing tactile representation learning largely models the two hands independently or combines them only for downstream prediction, leaving their cross-hand relationship unexplored. To exploit this overlooked structure, we introduce BiView-Touch, a tactile-only framework that completes masked target-hand latents from the remaining visible target-hand regions and the synchronized full contralateral hand. A student encoder with a geometry-conditioned directional decoder predicts full-view EMA latent targets, while temporal and layout counterfactuals encourage sensitivity to synchronized and anatomically organized source information. Controlled ablations and source-context interventions show that BiView-Touch learns structured cross-hand dependence on temporally aligned and anatomically organized contralateral tactile context, rather than benefiting from bilateral input alone. On the public HumanTouch
dataset, its frozen representations consistently outperform representative self-supervised baselines across low-label settings. With only 5% downstream labels, BiView-Touch achieves relative balanced-accuracy gains of 7.1% on bilateral wrist-motion recognition and 14.1% on force-derived interaction-phase recognition. We further introduce BVT-20, a 20-task bilateral tactile
dataset, and demonstrate transfer across recording sessions and pretraining corpora, including transfer to a held-out bimanual task. Our code and
dataset details are available on the anonymous project page: https://anonymous.4open.science/w/biview-touch-review-site-050C/.
Read the full abstract ▾
All authors Chenxin Liang, Youchen Lai, Chuqiao Lyu, Tianxing Chen, Shoujie Li, Wenbo Ding
Robotics
Manipulation Feasible Navigation Among Movable Obstacles with Discrete Contact Pushing
Shaohu Wang, Aiguo Song, Yulong Yuan et al. (+3 more)
In environments with large movable obstacles, detour-only navigation can be inefficient or even infeasible, while obstacle interaction requires
reasoning about navigation benefit, feasible placement, and executable manipulation. We present a hierarchical navigation among movable obstacles (NAMO) framework for mobile manipulators. At the high level, the planner identifies key blocking obstacles from reference paths and searches for relocation plans that jointly satisfy
geometric, manipulation, and downstream navigation constraints. When direct relocation is hindered by other movable objects, a large
language model (LLM) is selectively invoked to infer auxiliary manipulation dependencies, which are then verified by deterministic
geometric planning. To execute the resulting relocation goals, we define discrete contact modes on the surfaces of box-shaped obstacles and select contact faces and regions online based on position and orientation errors, enabling straight, side, and corner pushing through contact switching. A recurrent reinforcement-learning policy coordinates the mobile base and manipulator to track tool center point (TCP) targets while preserving end-effector reachability during sustained pushing. Simulation and real-
robot experiments demonstrate feasible navigation-manipulation in detour, single- and multi-obstacle relocation, and dependency-constrained scenarios, validating the framework for interactive navigation with large non-graspable obstacles. The open-source project is available at https://cloudytosunny.github.io/NAMO_DCPushing/.
Read the full abstract ▾
All authors Shaohu Wang, Aiguo Song, Yulong Yuan, Zhongyu Sun, Tianyuan Miao, Qinjie Ji
Robotics
Shared Execution-Clock Drifting Policy for Dynamic Precision Manipulation
Zhenchen Dong, Qingran Wu, Jinna Fu et al. (+4 more)
Manipulation under time constraints requires both accurate actions and an execution rhythm that matches the evolving scene. This becomes critical when a
robot must intercept moving objects or complete a sequence of adjustments before a deadline. Although one-step policies reduce generation cost, their directly predicted action sequences leave temporal allocation implicit. We propose Shared Execution-Clock Drifting (SECD), which makes execution rhythm an explicit part of one-step action generation. Conditioned on an observation and a latent sample, the policy jointly predicts a progress-indexed action curve and a shared monotone clock that maps fixed control times to locations on the curve. Demonstration-derived
alignment anchors this decomposition, which is trained jointly through drifting on the decoded actions. The resulting policy retains a fixed-rate control interface and requires one network evaluation. We evaluate SECD across four real-
robot tasks with inference on NVIDIA Thor. Across 300 trials, it achieves 77.00% task-averaged success and outperforms the evaluated one-step baselines on every task, including 91% success in cup
retrieval from a 16 m/min conveyor and 54% in restoring and folding a crumpled shirt within 90 s. A fixed-clock variant reaches 79% on the same conveyor protocol. Complementary state-based RoboMimic experiments, including cross-seed ablations on Transport and Square, further support the joint design of the temporal representation and demonstration
alignment. Project page: https://secd-anonymous-ewn.pages.dev/
Read the full abstract ▾
All authors Zhenchen Dong, Qingran Wu, Jinna Fu, Jiaming Wu, Fulin Chen, Hongyu Yu, Yide Liu
Robotics
Safety-Critical Control under Uncertainty via Adaptive Conformal Quantile Prediction Intervals
Hao Zhou, Yanze Zhang, Yiwei Lyu et al. (+1 more)
Safety-critical control under uncertainty requires uncertainty representations that are both statistically valid (for certifiable performance) and compatible with enforceable safety constraints. However, existing methods often assume particular distributions of uncertainty for provable safety guarantees or establish symmetric and input-agnostic prediction intervals for robust safety, which can lead to misaligned or overly conservative safety constraints in control synthesis. In this paper, we introduce a novel safe control framework with adaptive uncertainty quantification that constructs calibrated and state-dependent prediction intervals to enable high-probability safety guarantees, while improving constrained control performance. The framework leverages adaptive conformal prediction (ACP) and extends it with conformal
quantile regression (CQR) to capture distribution-free, asymmetric uncertainty intervals with certifiable probabilistic coverage, and integrates the resulting uncertainty sets into a probabilistic control barrier function formulation to enforce robust safety with reduced conservativeness. This yields uncertainty-aware safe control constraints that can be incorporated within a model predictive control(MPC) framework to provide provably safe behaviors with high probability. Simulation and theoretical results are provided to demonstrate the effectiveness of our approach.
Read the full abstract ▾
All authors Hao Zhou, Yanze Zhang, Yiwei Lyu, Wenhao Luo
Robotics
SCULPT-VLA: Learning Structured Control through Staged Action Grounding
Wenbo Li, Yiteng Chen, Wei Zhang et al. (+3 more)
Vision-
language-action (VLA) policies increasingly incorporate structured intermediate supervision beyond action labels. Yet specifying what an intermediate representation should encode leaves open how action prediction learns to depend on it. We introduce SCULPT-VLA, a policy that learns structured control through staged action grounding. Its action-conditioning state comprises complementary factors for task progression, scene dynamics, and spatial grounding. Training first forms these factors with teacher scaffolds, then grounds coarse action prediction through their composition as scaffold inputs are withdrawn. Direct perceptual access is subsequently restored for continuous refinement, combining the learned state with perceptual detail. The curriculum separates learning to condition actions on structure from refining continuous control. Deployment requires neither teachers nor discrete-action autoregression. SCULPT-VLA achieves higher average success than shared-backbone baselines on LIBERO, SimplerEnv-WidowX, and RoboTwin 2.0 Full. On SimplerEnv-WidowX, final success is 83.5%, versus 71.3% when Stage-II action learning directly accesses
vision and
language. Across four physical
robot tasks, average success under the tested distribution shifts reaches 58.1%, compared with 45.6% for $\pi_{0.5}$. Training ablations and factor-wise interventions support the staged design and show that the learned state continues to contribute to control after direct perceptual access is restored.
Read the full abstract ▾
All authors Wenbo Li, Yiteng Chen, Wei Zhang, Wenhao Li, Jun Yang, Qingyao Wu
Robotics
Steering Through Contact: A Finite-Support Motion Model for Single-Track Center-Articulated Robots
Mohamed Dhia Ounally, Nicolas Samson, Mathis Turgeon-Roy et al. (+3 more)
Trajectory planning and control in field robotics rely on predicting how propulsion and steering affect vehicle motion when contact points undergo slip. For articulated vehicles, the point-contact kinematic model (PCK) accounts for the linkage geometry but neglects the rotational resistance distributed along the contacts. We propose a finite-support quadratic model (FSQ) for single-track, center-articulated vehicles that incorporates this resistance through a quasi-static balance of lateral slip. Our approach generalizes the standard PCK formulation by relaxing the contact-point assumption. An exact reduction of the quadratic slip cost to contact moments gives a compact closed-form solution for real-time prediction of lateral velocity and yaw rate. We evaluate the proposed method in real-world experiments across asphalt, grass, ice, and mixed routes, using more than 7 km of data. For five-second predictions, FSQ reduces the weighted median translation and yaw errors by 53.5% and 68.9%, respectively, relative to PCK.
Read the full abstract ▾
All authors Mohamed Dhia Ounally, Nicolas Samson, Mathis Turgeon-Roy, Veronica Vannini, Johann Laconte, François Pomerleau