Robotics
TEMPO: Semantic-Action Decoupled RL Post-Training for Vision-Language-Action Models
Ziheng Liu and Quantao Yang
Vision-
language-action (VLA) models are commonly adapted to downstream manipulation tasks via supervised fine-tuning (SFT) or online reinforcement learning (RL) post-training. SFT is prone to distribution mismatch, and existing RL approaches typically apply a single, uniform update strategy to all model components, ignoring their distinct functional roles. We propose TEMPO, a semantic-action decoupled, two-timescale RL post-training framework for VLA models. TEMPO freezes the pretrained
vision-
language backbone to preserve general semantic representations, and restricts adaptation to two components with dedicated RL optimization loops: the semantic projection layer and the low-level action expert. We update them at different rates–the semantic projection layer infrequently, to keep the latent action stable, and the action expert frequently, to rapidly incorporate control feedback from online interaction. This decoupling RL fine-tuning strategy prevents fast policy updates from destabilizing high-level semantic representations while still allowing the action expert to learn efficiently from online feedback. Experiments on the CALVIN
benchmark and real-world manipulation tasks demonstrate that TEMPO consistently outperforms both pretrained state-of-the-art VLA models and the RL post-training baseline, while reaching and maintaining higher evaluation rewards on two real-world tasks.
Read the full abstract ▾
Robotics · Cross-listed from cs.AI
WNM-3D: A World Navigation Model with 3D Scene Conditioning for Closed-Loop VLN
Yuehao Huang, Yunzi Wu, Xiaotao Zhang et al. (+7 more)
Recent
vision-
language navigation (VLN) systems increasingly adapt pretrained
vision-
language models (VLMs) into
vision-
language-action (VLA) policies that map egocentric observations and
language instructions directly to navigation actions. Although semantically capable, such action-centric training does not explicitly model how the
agent's visual observations should evolve under its predicted motion. Generative world-action models (WAMs) jointly predict future observations and actions, yet existing WAMs for continuous VLN do not condition joint future-view and action generation on geometry-aware representations inferred from the observed history. We present WNM-3D, a generative World Navigation Model with 3D scene conditioning for continuous VLN. To consolidate past observations into persistent scene context, a frozen feed-forward geometry encoder extracts geometry-aware representations from the monocular egocentric RGB history, and a trainable 3D Scene-to-
Token Adapter converts them into a fixed-length prefix in the
token space of the world-action
Diffusion Transformer. Through block-causal
attention, this prefix conditions every future
video-action block, providing a shared
geometric context for both future-view and action generation. We train WNM-3D through supervised world-action fine-tuning on A*-generated demonstrations, DAgger-style adaptation on policy-visited states, and Counterfactual DanceGRPO refinement for closed-loop execution. Experiments on GN-Bench show that WNM-3D outperforms strong VLM-based navigation policies and its 2D-conditioned counterpart in closed-loop navigation. Stage-wise ablations further show that DAgger-SFT provides the larger success-rate gain, while Counterfactual DanceGRPO subsequently improves both navigation success and path efficiency.
Read the full abstract ▾
All authors Yuehao Huang, Yunzi Wu, Xiaotao Zhang, Xinhai Li, Jiankun Dong, Jiajun Lv, Chi Zhang, Chenjia Bai, Yong Liu, Xuelong Li
Robotics
Representation Handoffs for OpenArm-Based Laboratory Mobile Manipulation
Yang Shen, Chonghao Cheng, Ziyi Zhao et al. (+6 more)
Open-source robotics and foundation models have lowered the barrier to embodied AI, yet
language-guided laboratory automation still requires reliable
alignment from instructions and observations to safe actions. This field report presents an OpenArm-based mobile manipulation prototype for laboratory-style tasks, built by integrating dual OpenArm manipulators with a mobile base, vertical slide, RGB-D sensing,
lidar-based mapping, ROS2/MoveIt execution, and profile-defined skill interfaces. The system is organized around representation handoffs: natural
language requests are constrained into registered skill calls, sensor observations are grounded into maps and object poses, object priors provide role and skill constraints, and runtime bindings compile validated skills into executable motion goals. We use dry-run traces and startup checks to evaluate this integration path, showing how the prototype exposes missing calibration, incomplete object assets, and unfinished real-scene visual grounding as explicit deployment blockers. These intermediate representations serve as practical debugging interfaces for integrating
language, perception, planning, and
robot safety in embodied systems.
Read the full abstract ▾
All authors Yang Shen, Chonghao Cheng, Ziyi Zhao, Jialuo Zhu, Zhenyi Yi, Qi Zhao, Jian Yang, Yuhui Shi, Chin-Teng Lin
Robotics
Identifying the Key Biomechanical Features of Movement Adaptation during Exoskeleton-Assisted Locomotion
Peter Seungjune Lee and Katja Mombaur
The understanding of natural human adaptation during exoskeleton-assisted locomotion - particularly individual differences in adaptation behaviors and temporal progression - remains limited. In this work, we investigate temporal
evolution of biomechanical variables to uncover participant-specific adaptation strategies across different exoskeleton-assisted locomotion scenarios. Nine healthy participants performed treadmill walking under three conditions: without an exoskeleton, with exoskeleton active ankle assistance, and with exoskeleton zero-torque. Lower limb kinematics, inter-joint coordination, and metabolic cost of transport (MCoT) were analyzed at both the group and individual levels. Results indicate that adaptation is gradual and highly individualized, with substantial variability in convergence timing and movement patterns across participants. Kinematic adaptation occurred asynchronously across lower limb, with larger fluctuations during the swing phase. Metabolic responses were heterogeneous and often non-convergent, highlighting the limitations of steady-state assumptions commonly adopted in the literature. These findings emphasize the importance of individual-level, temporal
evolution analyses for understanding adaptation dynamics in exoskeleton use.
Read the full abstract ▾
Robotics · Cross-listed from cs.LG
Synthetic LiDAR Data Generation and Deterministic Downsampling for Point Cloud Classification on the Edge
Niclas Meyer and Stefan Reitmann
Deploying three-dimensional deep learning frameworks to low-power embedded processors is bottlenecked by the unstructured nature of spatial data and the resource-intensive distance sorting
algorithms often used before neural network inference. To address this gap, this paper presents a hardware-constrained workflow optimized for native execution on the Raspberry Pi 5. To account for the reality gap between noiseless, clean computer-aided design (CAD)
datasets and real-world sensor data, we use physics-based simulation to construct a synthetic
LiDAR dataset. Cross-
dataset evaluations demonstrate a substantial drop in classification accuracy when networks trained on clean CAD data are evaluated on synthetic
LiDAR sensor data, highlighting the critical need for sensor-aware training. To address the latency bottleneck of traditional
geometric preprocessing on edge CPUs, we integrate an isolated, feature-driven Critical Points Layer (CPL) as a frontend filter. Our results show that the pretrained CPL deterministically compresses raw 1024-point clouds to a subset of 40 to 60 unique coordinates. When profiled on the ARM Cortex-A76 processor, the complete pipeline achieves an inference throughput of approximately 50 FPS while maintaining an instance classification accuracy of 88.36%, demonstrating the viability of deterministic real-time 3D perception at the edge.
Read the full abstract ▾
Robotics
LifelongCrossNav: Persistent 3D Semantic Memory for Cross-Floor Multi-Object Navigation
Zehui Li, Zihao Sun, Jiawei Xu et al. (+6 more)
Object-goal navigation has made substantial progress in semantic perception and exploration, yet persistent
memory for multi-object navigation and cross-floor navigation are still commonly addressed separately. We present LifelongCrossNav, a framework for sequential multi-object ObjectNav in unknown multi-floor indoor environments. Within each episode, the
agent receives an ordered sequence of object-goal queries while continuously maintaining a shared sparse 3D semantic voxel
memory. This
memory incrementally accumulates
geometric structure, traversability states, and
vision-
language features, allowing subsequent object-goal queries to retrieve previously acquired scene information without rebuilding the map. To support persistent search across floors, LifelongCrossNav combines support-aware 3D traversability mapping, stair-specific perception, and direction-aware stair traversal. A unified navigation policy coordinates same-floor frontier exploration, live and historical point-of-interest
retrieval, stair navigation, and target-object search and approach. We further introduce HM3D-MFMON, a
benchmark for sequential Multi-Floor Multi-Object Navigation built on HM3D scenes, including a dedicated subset in which completing the full sequence of object-goal subtasks requires at least one floor transition. Experimental results show that LifelongCrossNav consistently outperforms a representative planar persistent semantic-map baseline on HM3D-MFMON, demonstrating that persistent 3D semantic
memory and cross-floor traversability modeling effectively support sequential multi-object navigation in multi-floor environments.
Read the full abstract ▾
All authors Zehui Li, Zihao Sun, Jiawei Xu, Zheqi He, Xiaoqiang Zhang, Jing-Shu Zheng, Lu Liu, Dahui Gao, Xiuwan Chen
Robotics
Detection and Ranging of Transient Extrinsic Contacts Based on 6D Dynamic Tactile Sensing
Haowen Zheng, Yinghao Wu, Fuyuan Liu et al. (+2 more)
Delicate manipulation often involves transient and subtle collisions between a grasped object and the environment. While the human hand localizes these contacts effortlessly thanks to superior tactile sensitivity, robotic systems often lack the requisite resolution to acquire the information necessary for motion planning, resulting in clumsy manipulation or even task failure. Here, we propose transient extrinsic contact detection and ranging (TECDAR), a simple yet fast and efficient method for detecting and ranging extrinsic contact of grasped objects. Our design of gripper tips employs dynamic tactile sensing leveraging a single 2.5$\times$3 mm 6D inertial measurement unit. The sensor captures sub-millisecond tip deformations at a 7 kHz sampling rate, but operating on a data stream of only 84 KB/s. High bandwidth and compact data size enable the system to rapidly detect and localize contact between grasped objects and their surroundings. Specifically, fusing tactile data with
robot pose via an extended Kalman filter enables fast and precise localization of extrinsic contact, reaching millimeter-level accuracy within 180 ms. Experimental results demonstrate that the system achieves an average localization accuracy of approximately 7\,mm in both line-contact and point-contact localization tasks. Furthermore, this near-instantaneous localization enables the
robot to rectify its trajectory on a millisecond scale, facilitating precise tool manipulation and enhanced perception of complex environments purely through tactile exploration and mapping. We envision such techniques advancing the future of robotics across domains requiring delicate manipulation, including precision assembly, surgical assistance, and autonomous exploration in touch-dominant environments. Project page: humitlab.github.io/TECDAR/
Read the full abstract ▾
All authors Haowen Zheng, Yinghao Wu, Fuyuan Liu, Yichen Li, Yitian Shao
Robotics
M2-SMap: Memory-Efficient Semantic Mapping with Hierarchical Multi-Model Representation
QiYing Deng, ZhongLai Wang, Yuan Gao et al. (+1 more)
Dense point cloud maps, as a typically used mapping representation, are difficult to deploy on resource-constrained
robots because their
memory consumption grows rapidly with scene scale. Although compact single-model representations reduce
memory cost, their fixed
geometric expressiveness is insufficient for structurally diverse environments. Existing multi-model methods improve representational flexibility, yet their feature extraction and model selection are often dominated by local geometry, which can cause overfitting and adhesion between objects. To address these issues, this paper presents M2-SMap, a
memory-efficient semantic mapping framework based on hierarchical multi-model representation. First, a hierarchical
geometric decomposition partitions RGB-D point clouds into compact Gaussian components. Then, a projection-guided semantic annotation mechanism assigns instance identities to each component. Subsequently, these annotations are incorporated into an object-aware Gaussian fusion strategy. Furthermore, a multi-scale feature extraction strategy separates large planar regions, semantic objects, and complex residual structures, which are respectively represented by bounded planes, object-level superquadrics, and GMM primitives. Experiments on three RGB-D sequences show that M2-SMap runs in real time at no less than 29.37 Hz while achieving the lowest primitive count, with an average reduction of 18.7% over the best baseline. It also reduces the mean per-frame number of measured inter-object adhesion cases from 2.808 to 0, demonstrating efficient and semantically consistent scene representation.
Read the full abstract ▾
All authors QiYing Deng, ZhongLai Wang, Yuan Gao, Wei Dong
Robotics
AutoIntervene: Calibrated Intervention for Action-Chunking Imitation Learning Policies
Jinhe Tang and Weiming Zhi
Action-chunking visuomotor policies learn from demonstrations and improve temporal consistency by predicting short action sequences rather than single-step commands. Yet perception errors and execution drift can move the
robot outside the demonstration distribution, while the policy continues to produce smooth action chunks that are inconsistent with the observed state. We present AutoIntervene, an online framework that selectively transfers control between an action-chunking policy and an operator during deployment. AutoIntervene evaluates proposed chunks against a visual-action support
memory built from successful task executions, combining visual similarity with consistency between proposed and reference actions. Phase-local support governs policy-to-operator transfer within the current task phase, whereas global support governs the return to policy control after operator recovery. We calibrate separate switching thresholds for the two directions from empirical
quantiles of evaluation-level scores on held-out expert demonstrations, avoiding direct manual tuning of score cutoffs. Intervention segments retained from successful rollouts target learner-induced states and provide corrective supervision for subsequent policy updates. Experiments on real-world bimanual manipulation tasks show higher post-adaptation task success and lower operator-control time than manual intervention.
Videos and additional results are available at https://aus.bot/research/autointervene/.
Read the full abstract ▾
Robotics
C2Dex: Contact-Consistent Reconstruction and Retargeting for Dexterous Manipulation from Monocular Video
Jie Ren, Zhehao Jiang, Yinhong Yang et al. (+9 more)
High-quality demonstrations for dexterous
robot manipulation are costly and difficult to collect, whereas monocular human
videos provide a scalable source of diverse manipulation behaviors. However, transferring such demonstrations to dexterous
robots remains challenging: monocular hand-object interaction (HOI) reconstruction often produces temporally unstable contacts and physically implausible interactions, while conventional retargeting methods struggle to preserve task-relevant contacts and local interaction geometry across different hand embodiments. We present C2Dex, a
video-to-dexterous-manipulation framework built around a shared interaction representation: stable object-side contacts recovered by aggregating noisy frame-wise observations in the canonical object space. These stable contacts serve a dual role: as trajectory-level constraints that guide reconstruction toward temporally coherent and physically plausible human HOI trajectories, and as explicit transfer targets for the dexterous hand, where
Laplacian interaction optimization preserves the local hand-object geometry across embodiments and residual reinforcement learning refines the trajectory in simulation. Experiments on DexYCB and TACO show that C2Dex achieves end-to-end trajectory success rates of 57.78% and 26.67%, respectively, substantially outperforming the strongest baselines (17.78% and 10.00%) under identical evaluation criteria. Real-
robot replay experiments further demonstrate physical feasibility across diverse contact-rich manipulation tasks. Project page: https://k-jie.github.io/C2Dex/
Read the full abstract ▾
All authors Jie Ren, Zhehao Jiang, Yinhong Yang, Haorui Jia, Han Jiang, Ben Li, Yao Yao, Cheng Lin, Qiu Shen, Zhenshan Bing, Xiao-Xiao Long, Xun Cao
Robotics
Real-time Whole-Body Motion Planning for Mobile Manipulators Carrying Arbitrarily Shaped Payloads via Kinematically-Coupled SVSDF
Yisheng Li, Longji Yin, Tingrui Zhang et al. (+6 more)
Mobile manipulators are increasingly tasked with transporting large, non-convex payloads through cluttered environments, yet existing planners either oversimplify the payload geometry or fail to handle the kinematic coupling between manipulator links, leading to lost feasible space or stalled optimization. This letter presents a real-time whole-body motion planning framework for mobile manipulators carrying arbitrarily shaped payloads. The front-end employs a chain-decomposed kernel-based collision check that preserves the true geometry of the
robot and payload, with compact storage and fast bit-level queries. A mid-end preprocessing stage converts the front-end path into a continuous trajectory enforcing smoothness and feasibility, and executes it directly when collision-free to bypass the costly back-end. When refinement is required, the back-end performs trajectory optimization built on a Kinematically-Coupled SVSDF (KC-SVSDF), which propagates collision-avoidance gradients along the kinematic chain to produce coherent whole-body escape directions. Ablation studies, comparative
benchmarks against state-of-the-art baselines, and real-world experiments on a differential-drive mobile manipulator demonstrate that the proposed framework reliably transports large, non-convex payloads through tight passages and cluttered environments.
Read the full abstract ▾
All authors Yisheng Li, Longji Yin, Tingrui Zhang, Ruize Xue, Haoda Zhu, Nan Chen, Siqi Liang, Yuxi Liu, Fu Zhang
Robotics
Benchmarking and Reasoning Distillation of Large Language Models for Feedback Controller Design in Complex Dynamical Systems
Zhongchao Zhou, Yixuan Xie, Wenwei Yu et al. (+5 more)
Although remarkable capabilities have been demonstrated by Large
Language Models (LLMs) across scientific domains, feedback controller design remains underexplored. Existing
benchmarks focus mainly on linear single-Degree-of-Freedom (DoF) systems and large API-hosted models, leaving performance on complex controller-design tasks and feasibility for edge deployment unclear. To address these limitations, we introduce the Complex Dynamics-to-Control
Benchmark for Large
Language Models (CoDyControlBench), comprising 132 system configurations across five evaluation dimensions: number of DoF, system type, coupling level, damping regime, and controller type. Six state-of-the-art LLMs were evaluated over three independent runs, including three commercial models (GPT, Gemini, and Claude) and three open-source models (GLM, DeepSeek, and Qwen). GPT achieved the highest design success rate at 94.8%, whereas Qwen showed the lowest rate at 50.0%. Across the
benchmark dimensions, DoF and controller type exhibited the largest model-averaged variations in design success, with success-rate ranges of 36.3% and 17.6%, respectively, exceeding those associated with system type, coupling level, and damping regime. Comparison of GPT and Qwen showed that their performance gap arose mainly from the control-design knowledge, particularly gain selection and the use of transient-limiting mechanisms. For edge deployment, a specialized 1.5B-parameter model was developed through
reasoning distillation. The
reasoning-distilled model outperformed the answer-distilled and base model on CoDyControlBench, maintained stable performance across 1-6 DoFs, and achieved successful traget tracking in all three physical trials on a pneumatic-artificial-muscle-driven robotic arm. These results establish a
benchmark baseline and highlight the potential of lightweight, edge-deployable controller-design models.
Read the full abstract ▾
All authors Zhongchao Zhou, Yixuan Xie, Wenwei Yu, Yuxi Lu, Yaonan Zhu, Qian Niu, Yutaka Matsuo, Yusuke Iwasawa
Robotics
A Haptic Robot Finger Designed for Guqin Instrument Playing
Tianwei Zhang, Hanming Yan and Yang Yang. Ziya Wang
With the rapid advancement of humanoid robotics and embodied intelligence technologies, numerous musical instrument-playing
robots have emerged in recent years, such as pianos, chime bells, and taiko drums. These
robots primarily employ open-loop positional control, rendering them incapable of operating instruments requiring dexterous hands and precise tactile perception, such as a violin, guitar, and guqin. This paper describes the design and validation of a high-precision tactile-sensing finger. By mimicking the shape of the fingertip and fingernail found on a human finger, we develop a biomimetic
multimodal haptic fingertip and validate it on selected guqin string-contact tasks, including open-string and stopped-note comparisons, harmonic-tuning, and tactile-triggered bimanual coordination, using the guqin, a traditional Chinese musical instrument, as a challenging validation scenario rather than as a fully demonstrated robotic performance system. This research integrates tactile sensing with robotics technology, thereby contributing to applications in world heritage conservation and cultural dissemination.
Read the full abstract ▾
Robotics
Automated Terminal-to-Housing Assembly System for Flat Ribbon Cable Harness
Eunkyu Choi, Joonho Seo, Seungmin Lee et al. (+1 more)
This paper presents a sensor-minimal automated assembly system for bidirectional single-row flat ribbon cable harnesses (FRCHs). Unlike conventional peg-in-hole or single-terminal insertion tasks, FRCH assembly involves mechanically coupled multi-terminal insertion under flexible and dense
geometric constraints. To address this problem, the proposed system performs the assembly through a purely mechanical sequence consisting of Cable
Alignment, Lean & Slide, Weaving, and Clamping, without relying on active sensing or
vision. Each mechanism is designed to progressively reduce correlated terminal misalignment, insertion interference, and instability before final locking. Experiments on bidirectional single-row FRCHs achieved an 83.75% end-to-end process success rate over 80 trials, with success rates of 85.0% and 82.5% in the first and second halves, respectively. The cycle time was 33 s under half-speed operation. To the best of our knowledge, this work presents the first automated prototype for FRCH terminal-to-housing assembly for multi-pin housings.
Read the full abstract ▾
All authors Eunkyu Choi, Joonho Seo, Seungmin Lee, Seokhwan Jeong
Robotics
Decoupling Intention from Trajectory: A Representational Deduction Framework for World Action Models
Xiangkai Ma, Yue Ma, Junjie Wang et al. (+6 more)
World Action Models (WAMs) aim to construct a unified architecture capable of understanding world state
evolution and guiding to generative motion planning. However, existing visual branches focus on predicting static visual observation, rather than reflecting potential transition information that captures the
evolution of world states under motion interactions. This leads to representational
entanglement between high-level physical condition
evolution and low-level action trajectory generation within the Action Model, creating a structural bottleneck while weakening the predictive capability of world
evolution modeling for action generation. We propose PILOT (Physical Inference for Latent Optimized Trajectories), whose core Representational Deduction (RD) bridges this gap by integrating motion thought-of-chain (CoT) guidance as a native model capability. Specifically, RD aims to encourage the action branch to explicitly model potential state transition
tokens, which are retained as CoT in the
reasoning space to guide fine-grained motion trajectory. Experiments demonstrate that RD not only significantly improves the success rate and generalization ability of WAMs in complex robotic manipulation tasks but also enhances the model's physical interpretability by decoupling high-level motion semantics from low-level trajectory details. Furthermore, the abundant state transition supervision signals introduced by RD effectively alleviate the sparse supervision in action generation, enabling it to serve as an efficient few-shot real-
robot fine-tuning strategy and demonstrating superior scalability for migration to mainstream WAM architectures.
Read the full abstract ▾
All authors Xiangkai Ma, Yue Ma, Junjie Wang, Sheng Xu, Mingyang Li, Han Zhang, Yuzheng Zhuang, Wenzhong Li, Zhihao Yuan
Robotics
Exact Thrust-Reversal Limits of Bidirectional Propellers under Bounded Motor Inputs
Ahmed Ali, Chiara Gabellieri and Antonio Franchi
Bidirectional propellers are often treated as signed thrust sources, but their thrust is a signed-quadratic function of rotor speed.Thus, thrust reversal necessarily occurs through zero rotor speed, where the ability of a bounded motor torque to change thrust collapses.This work formalizes this obstruction by studying exact thrust-trajectory reproducibility under bounded motor inputs with prescribed smoothness.We derive a normalized thrust-coordinate model with vanishing input gain at zero thrust, and prove necessary and sufficient reproducibility conditions in terms of the zero-crossing order of the desired thrust.Generic reversals, in which thrust crosses zero with nonzero slope, require unbounded motor input; the resulting conditions provide direct design rules for shaping thrust reversals that avoid singular motor commands.We also derive the corresponding current and voltage regularity requirements for a DC motor driving a bidirectional propeller.Experiments on a motor-propeller setup validate the predicted reversal-order effects, showing localized current/voltage peaks and thrust-tracking degradation for linear reversals, but not for higher-order reversals.These results expose an intrinsic actuator-level limitation that must be considered in force, acceleration, and interaction-control references for aerial
robots.
Read the full abstract ▾
Robotics
Cross-View Action Consistency for Camera-Robust Vision-Language-Action Policies
Bingqi Huang, Bingchuan Wei, Xuan Wang et al. (+2 more)
Vision-
language-action (VLA) policies fine-tuned from a fixed scene camera can fail when the camera is moved, even when the task, objects,
language, and
robot state are unchanged. We study scene-camera viewpoint robustness using only a scene RGB image,
language, and proprioception, without camera labels, extrinsics, depth, or point-cloud inputs. The wrist stream is masked throughout to prevent an unperturbed visual shortcut from confounding attribution to scene-camera variation. For flow-based VLAs, we propose to regularize the action-flow velocity field, the quantity directly integrated to generate continuous action chunks. We construct action-equivalent view pairs by resetting original LIBERO demonstrations to the same MuJoCo state and rendering nominal and perturbed scene-camera views. Both views are supervised by flow matching, while a cross-view loss encourages their predicted action-flow velocities to agree at the same sampled flow coordinates. On the LIBERO-Plus camera-perturbation track, our method reaches 87.2$\pm$0.4% (4,797 rollouts per seed across 3 training seeds), +7.4pp over flow-matching-only training on the same paired data (79.8$\pm$0.8%, also 3 seeds) and +12.5pp over naive mixed-camera SFT, while maintaining nominal-camera ID performance (95.0$\pm$0.8%; same-data FM-only: 95.0$\pm$4.3%). A shuffled-pair control collapses to 25.8%, showing that the gain depends on action-equivalent pairing. On a real
robot, we evaluate three tabletop tasks with 10 rollouts per task and camera placement; held-out-camera success improves from 53.3% to 74.4% under the same single-scene-RGB inference interface.
Read the full abstract ▾
All authors Bingqi Huang, Bingchuan Wei, Xuan Wang, Yingkai Cai, Zhaokui Wang
Robotics · Cross-listed from cs.CV
Vernata: Self-Supervised Learning of LiDAR Point Representations
Oliver Lemke, Alexander Liniger, Abel Gawel et al. (+1 more)
LiDAR serves as a primary sensing modality for
robots operating in outdoor environments. However, the performance of deep learning models in this domain is severely limited by the scarcity of labeled data, a direct result of the high cost of 3D annotation. Self-supervised learning addresses this scarcity by learning general-purpose features from unlabeled data. In this work, we present a multi-modal, multi-teacher
distillation framework for self-supervised learning on outdoor
LiDAR point clouds. Building upon the Sonata architecture, we introduce Vernata, consisting of three extensions: sparse view augmentation to improve robustness against varying point densities, a
memory bank mechanism to stabilize resource-constrained training, and cross-modal
distillation utilizing dense, high-resolution 2D image features to enable fine-grained semantic guidance. We evaluate our method on the GrandTour, TartanGround, and Waymo
datasets, as well as data collected from our own robotic platforms. Our experiments demonstrate a significant performance improvement over Sonata baselines, yielding mIoU scores of 54.7 on TartanGround (+5.9 points, +12.1%) and 57.1 on Waymo (+7.3 points, +14.7%). Finally, we show that the self-supervised approach maintains strong performance even in reduced-modality settings (lacking color or normals), achieving competitive mIoU scores of 49.4 and 50.2 on the respective
datasets.
Read the full abstract ▾
All authors Oliver Lemke, Alexander Liniger, Abel Gawel, Marco Hutter
Robotics
Spatiotemporal Agility: Time-Constrained Reinforcement Learning for Vision-Guided Dynamic Quadrupedal Interception
Yidong Zhu, Zibo Dai, Tongning Zhang et al. (+2 more)
Legged
robots require robust agility to perceive and interact with complex and dynamic environments within a constrained time. However, most existing quadruped locomotion works rely on velocity-tracking policy, which struggle to reach precise targets within strict temporal constraints. Moreover, integrating real-time perception with agile locomotion for highly dynamic targets remains challenging due to sensor latency and processing delays. To concretely study and
benchmark such agility in dynamic settings, we introduce a challenging ball-catching task for legged
robots. This paper proposes an integrated framework that combines a
vision module for landing point and time prediction with a direct position and time conditioned RL locomotion policy, instead of intermediate velocity commands. Beyond the method design, this work presents a system-level contribution that completes real-time robotic interception system that integrates multi-camera perception, online trajectory prediction, low-latency target communication, and sim-to-real locomotion control into a closed-loop deployment pipeline. By explicitly predicting the future spatial-temporal target, our approach mitigates perception latency during dynamic interception. We conducted extensive ball-catching experiments for the legged
robot. Through comparative experiments against a velocity-tracking baseline, our direct target-conditioned approach achieves a higher success rate in catching balls with predicted landing spots within 2 meters and flight times between 0.8 and 1.2 seconds. This shows that the
robot has successfully completed the dynamic ball-catching task under our tested setup. Furthermore, our policy exhibits a smaller performance gap after deployment, suggesting improved sim-to-real behavior in these trials.
Read the full abstract ▾
All authors Yidong Zhu, Zibo Dai, Tongning Zhang, Leixin Chang, Hua Chen
Robotics
How Should I Pick a Foundation Model for My Robot? In Favor of a Community Evaluation Framework for Social Robots
Eric Nichols, Alva Markelius and Hatice Gunes
Researchers who seek to build social
robot applications on foundation models are faced with a difficult question: how should we pick a model? Public leaderboards offer little guidance: the demands of real-time, embodied social interaction lie largely outside their focus. And direct evaluation is impractical at scale: each embodied study requires scarce participant,
robot, and experimenter time. In this paper, we identify five evaluation dimensions for foundation models in social
robots: (i) conversational competence, (ii) user safety, (iii) embodied character, (iv) target scene effectiveness, and (v) audience appropriateness. To make model selection cheaper and better informed, we propose a three-tiered evaluation funnel paradigm that first filters with general metrics, then extends to simulated interactions, and terminates in more expensive,
robot-specific evaluation. We map all five dimensions across all three tiers, chart where applicable evaluation methods exist and are missing, and close with a call to action: let's build the evaluation framework together as a community.
Read the full abstract ▾
Robotics · Cross-listed from cs.CR
Rigid-Covert GNSS Spoofing of UAV Swarms: A Structural Blind Spot, Its Detection Limit, and Absolute-Anchor Defenses
Minseok Park and Joon Soo Yoo
Cooperative
UAV-swarm defenses commonly cross-check GNSS positions against measured inter-drone geometry. We show that this relative-geometry channel has a structural blind spot: a common, slowly varying translation (a rigid-covert shift, RigidShift) preserves all pairwise distances and is therefore unobservable to any relative-only detector (a gauge-freedom argument). We validate this blindness on distance-verification and semidefinite-feasibility baselines, while explicitly distinguishing it from onboard inertial/GNSS monitors that can raise a bare alarm but cannot recover the swarm's true position. To quantify when an external reference restores observability, we derive the drift-dependent detection floor $2\gamma/(1-t_s/T)$ for a calibrated anchor-residual detector and empirically identify an additional detector-specific noise floor (measured slope 2.66 vs. predicted 2.67). We then present a centralized anchor-rooted recovery pipeline that reconstructs swarm geometry from inter-drone ranges, aligns it to a trusted-anchor subset with Byzantine-robust fitting, and recovers the absolute positions of non-anchored drones. A segmented
estimator jointly estimates anchor drift, attack rate, and onset when no clean-epoch label is available. Across statistical simulations, ArduPilot software-in-the-loop experiments, and Gazebo experiments with rendered
vision anchors, the method recovers the positions of non-anchored drones to a median error of 0.39 m (20 seeds) under approximately 10.1 m of GNSS drift, and to 7.1 cm (5 seeds) in the rendered-
vision multi-SITL setting. We also characterize the explicit limits imposed by non-collinear anchor geometry, anchor coverage, $\tau\to0$ drift-attack aliasing, and majority anchor compromise. All evaluations are simulation-based and use no RF spoofing hardware or physical swarm.
Read the full abstract ▾
Robotics
Are Visual Place Recognition Models Recognizing Places or Conditions? Distractor-Augmented Evaluation and Condition Suppression
Beomsu Kim, Minwoo Jung and Giseop Kim
Long-term Visual Place Recognition (VPR) is typically evaluated by matching queries from one condition against a database from another. Crowdsourced map databases, however, may mix conditions and include images that resemble the query in condition but depict different places. In the presence of these distractors, a method may retrieve by condition similarity rather than place identity. We argue that this susceptibility arises because the discriminability of VPR methods allows them to encode information such as illumination, weather, and seasonal appearance in their descriptors. We therefore introduce Distractor-Augmented Recall (DAR) to isolate and quantify the effect of distractors, and propose condition suppression to remove condition information from VPR descriptors. Across eleven methods and six
datasets, method rankings under DAR@1 differ from those under Recall@1 (R@1), while applying INLP and LEACE as condition suppression methods generally improves DAR@1 without reducing R@1. Thus, distractor robustness is distinct from standard
retrieval performance and can be improved by suppressing condition information.
Read the full abstract ▾
Robotics
Unordered Landmark Visual Navigation
Hao Ren, Junzhe Zhu, Yihan Li et al. (+8 more)
Image-goal navigation is a fundamental capability for embodied AI, yet its practical deployment is strained by strong prior assumptions. Existing methods predominantly rely on temporally ordered
video streams or auxiliary sensors (e.g., depth,
LiDAR) to maintain spatial consistency. These sequential and
multimodal dependencies severely restrict scalability, especially when deploying
robots using crowd-sourced or pre-recorded unordered image collections. When temporal priors are removed, current methods struggle with severe perceptual aliasing, noisy associations, and catastrophic mapping failures. To address this underexplored challenge, we propose Unordered Landmark Visual Navigation (ULVN), a unified RGB-only framework free from temporal and odometric priors. ULVN systematically mitigates error accumulation by integrating mapping, localization, and planning. Specifically, it constructs a robust 2D topological map directly from unstructured images via calibrated
geometric verification and maximum spanning forest refinement. For closed-loop execution, ULVN abandons sequential heuristics, utilizing a graph-based belief propagation filter with
entropy-adaptive fusion for global localization and dynamic subgoal planning. Extensive experiments in simulation and real-world deployments demonstrate that ULVN significantly outperforms state-of-the-art methods.
Read the full abstract ▾
All authors Hao Ren, Junzhe Zhu, Yihan Li, Zetong Bi, Le Zheng, Zhi Li, Yiqing Yuan, Zhaoliang Wan, Dizhe Zhang, Lu Qi, Hui Cheng
Robotics
When Coordination Becomes a Threat: Communication Attacks in LLM-Controlled Multi-Robot Systems
Zhen Huang, Zhihuang Liu, Weijia Shi et al. (+3 more)
Large
Language Models (LLMs) are increasingly used as high-level planners in embodied multi-
robot systems, enabling
robots to interpret natural
language instructions and coordinate executable actions. Yet, this growing reliance on LLM planners also raises security concerns. Prior work has focused mainly on individual
robots, while communication risks in multi-
robot collaboration remain insufficiently understood. Existing multi-
robot studies are further limited to preliminary analysis under the Decentralized Multi-
agent System (DMAS) architecture, so it remains unclear whether these risks persist across other common communication architectures and how attacker access settings shape their propagation. To fill this gap, we formulate two communication attacks corresponding to distinct attacker access settings: the External Entry Point Attack and the Privileged In-System Attack. We evaluate both attacks across DMAS, HMAS-1, and HMAS-2 using three LLMs and five embodied multi-
robot tasks. Results show that unsafe information can turn into unsafe actions across all three architectures: DMAS reaches a 96.7% entry endorsement rate and a 100% post endorsement activation rate, HMAS-1 reaches a 97.8% unsafe action success rate, and HMAS-2 triggers 88.3% of task defined unsafe action slots. To mitigate risks from trusted information flow, we introduce the Claim Provenance and Verification (CPV) Gate, which verifies communicated claims before downstream reuse and reduces the violation rate from 70.0% to 36.6%.
Read the full abstract ▾
All authors Zhen Huang, Zhihuang Liu, Weijia Shi, Yifan Yang, Weishang Wu, Zhiping Cai
Robotics
R2S-EGO: Dual-Proxy Refinement for Sparse-Capture Real-to-Sim
Shuai Fang, Xin Deng, Yuchen Kang et al. (+2 more)
Real-to-sim (R2S) depends on scene representations that render observations along
robot ego trajectories, yet dense multi-view capture limits per-environment real-image capture-count efficiency, and sparse human capture can leave behavior-scoped
robot views under-supported. Camera-controlled synthesis can fill missing views, but its use in R2S requires behavior-admissible queries and capture-anchored structural conditioning. We present R2S-EGO, which couples a simulator-derived
robot proxy that represents the behavior-scoped executable query domain with a capture-anchored geometry proxy that supplies scene-specific structural conditions. Within this domain, fixed- budget selection targets current support deficits for which geometry support is available. The generated observations are assimilated as pseudo-observations to refine the visual asset, while real captures remain anchors. The fused geometry proxy also supplies the scene collision surface, which is refreshed between rounds. Together, these updates refine the existing simulation scene while its
robot dynamics and control stack stay fixed. Across 48 frozen Unitree G1 ego views in three Replica scenes, six-view R2S-EGO reaches 19.062 dB PSNR, compared with 14.226 dB for the strongest reported R2S baseline. Across five paired policy-training seeds, R2S-EGO achieves 82.5% +/- 6.8% real-G1 sitting success, compared with 10.0% +/- 10.5% for GaussGym.
Read the full abstract ▾
All authors Shuai Fang, Xin Deng, Yuchen Kang, Zhenjiang Li, Jie Chen
Robotics · Cross-listed from cs.ET
Ising Acceleration for Multi-Robot Multi-Target Planning
Ahmet Efe, Recep B. Uludag, Chris H. Kim et al. (+1 more)
Ising machines are emerging as promising hardware for combinatorial optimization. With recent advances in CMOS Ising technology, they are becoming attractive as low-power accelerator systems for robotics, where energy is limited and combinatorial optimization arises in multiple forms. However, a hardware-aware analysis of where such chips fit within a robotics planning stack is still missing. This paper studies the capabilities and limitations of CMOS Ising machines for low-power acceleration in multi-
robot multi-target planning. We analyze three planning layers—target sharing, tour construction, and pathfinding—using real 45-
spin all-to-all connected CMOS Ising chips as representative devices. We propose new Ising-based planning methods and a multi-mapping pipeline that uses
spin merging, coefficient quantization, and
spin-budget branching to adapt subproblems to
spin- and coefficient-limited hardware. Our results show that the proposed recursive target-sharing method naturally matches the Ising hardware, achieving up to 8,000x lower energy than a classical baseline. End to end, the Ising pipeline produces routes within 9% of a strong classical baseline at 130x lower energy, showing that compact CMOS Ising machines can be effective in selected parts of the planning stack.
Read the full abstract ▾
All authors Ahmet Efe, Recep B. Uludag, Chris H. Kim, Ulya R. Karpuzcu
Robotics
Is Forward Prediction Enough? Physical State Grounding for JEPA World Models
Haodong Yan, Jiaguan Zhu, Mingyuan Jia et al. (+12 more)
Learning structured and control-relevant latent representations remains a key challenge for world models. Recent JEPA-based world models learn action-conditioned predictive latent dynamics from observation sequences. However, their forward-prediction objectives do not explicitly enforce reliable identifiability of
robot-centric physical state from individual latents or state changes from latent pairs, which can limit downstream planning and policy performance. We propose PSG-JEPA, a physically grounded JEPA world model that shapes its latent space with two complementary grounding objectives beyond forward prediction: grounding individual latents in
robot proprioceptive state, and grounding latent pairs in multi-horizon joint-angle changes. Both objectives are applied only during training, leaving the inference architecture and computational cost unchanged. To comprehensively evaluate PSG-JEPA, we conduct experiments at three levels: (1) latent identifiability via probing, (2) goal-conditioned planning on frozen latents, and (3) policy learning in simulation and on a real
robot. Experiments demonstrate that our PSG-JEPA consistently outperforms state-of-the-art latent world-model baselines at all three levels.
Read the full abstract ▾
All authors Haodong Yan, Jiaguan Zhu, Mingyuan Jia, Ruiqing Yin, Junjie He, Zhide Zhong, Junfeng Li, Jinxuan Lu, Hengtao Li, Tianran Zhang, Jiayi Chen, Wenxuan Song, Wen Chen, Yuxiang Gao, Haoang Li
Robotics
AtlasVLA: Persistent World-Ego State Modeling for Vision-Language-Action Models
Guiyu Zhao, Longteng Guo, Yanghong Mei et al. (+7 more)
While
Vision-
Language-Action (VLA) models have advanced embodied AI, their fundamentally reactive paradigm severely limits performance in partially observable and long-horizon tasks. When restricted to a single wrist-mounted camera, they inevitably suffer from perception forgetting as objects exit the field of view, and temporal task-progress forgetting} during multi-step execution. To overcome these bottlenecks, we propose AtlasVLA, a novel framework that transitions from direct reactive manipulation to proactive
reasoning through a persistent world-ego state. AtlasVLA features a dual-
memory architecture: a 4D Persistent World State
Memory that lifts transient 2D observations into a globally updated, voxel-hashed spatial state to resolve visual blind spots, and an Ego-Working State
Memory that tracks historical ego state and task progress. By conditioning a
diffusion transformer (DiT) on this joint World-Ego state, AtlasVLA enables robust spatial
reasoning. Extensive evaluations across LIBERO, RLBench, and real-world
benchmarks demonstrate that AtlasVLA achieves state-of-the-art performance using solely a wrist camera. Remarkably, it decisively outperforms multi-view baselines, yielding absolute success rate improvements of 9.4% on LIBERO-Long and 17.5% in real-world long-horizon tasks.
Read the full abstract ▾
All authors Guiyu Zhao, Longteng Guo, Yanghong Mei, Zilin Zhu, Yu Zhang, Bin Cao, Mingming Yu, Xingjian He, Jie Jiang, Jing Liu
Robotics
Hoverflie: An empirical investigation of rotor shrouds to transform micro air vehicles into multi-modal hovercraft
Mrinmoy Modak and Daniel S. Drew
Small rotorcraft intended for use indoors or around the built environment have extremely limited flight duration. This paper presents the design and experimental characterization of a custom shroud system that transforms a Crazyflie 2.1 micro air vehicle into a multi-modal
robot capable of operating as a high-efficiency hovercraft or a free-flying drone. A custom experimental platform was developed for precise control of hover height and rotor duty cycle, and automated data logging of lift forces. Parametric testing of duct, intake, and nozzle geometries was performed to investigate the impact of shroud configuration on in-ground-effect and free-flight performance. An empirical model is developed which, unlike typical models for ground effect in rotorcraft, captures the suckdown effect that reduces force at intermediate height. It is shown that, through proper design of the shroud, beneficial ground effects can be increased while diminishing negative effects both close to the ground and in free flight. An optimized configuration exhibited nearly three times higher in-ground-effect force while maintaining comparable out-of-ground-effect aerodynamic thrust, although the added shroud mass reduces free-flight control authority. Lightweight shrouds are manufactured using thin-film thermoformed components, and total single-charge flight time is shown to increase by 60% in-ground-effect while decreasing by only 30% in free-flight as compared to the stock drone. Finally, controlled flight in the air, hovering close to the ground, and hover-to-flight transitions are demonstrated using a simple mode-switching controller, with tracking errors reported to quantify performance. This work provides an experimentally-validated and easily adoptable foundation for future research into lightweight ground-effect vehicles and hybrid drone-hovercraft systems.
Read the full abstract ▾
Robotics · Cross-listed from cs.MA
Scalable Long-Horizon Planning with Staggered Updates for Lifelong MAPF
Vaibhav Sanjay and Jiaoyang Li
Lifelong Multi-
Agent Path Finding (LMAPF) requires generating collision-free paths for large
agent fleets under strict real-time constraints. Reactive frameworks such as PIBT and Enhanced PIBT (EPIBT) scale effortlessly to thousands of
agents through rule-based, step-by-step coordination but suffer from severe temporal myopia, making them ineffective in scenarios where long-horizon
reasoning is essential. RHCR plans windowed paths over multi-step horizons but incurs substantial planning overheads that hinder scalability. TP tackles both challenges by planning only subsets of
agents at each timestep, yet its applicability is restricted to highly structured maps. To achieve long-horizon planning at scale across general maps, we propose Path Updates over Staggered Horizons (PUSH), a LMAPF planner capable of coordinating thousands of
agents in under a second while planning over multi-step horizons. PUSH combines the key advantages of PIBT, RHCR, and TP. Like TP, PUSH reduces computational complexity by planning only a subset of
agents at each timestep using staggered planning windows. Unlike TP, however, PUSH plans RHCR-style windowed paths in general maps without relying on restrictive map assumptions. To maintain high throughput in congested environments, PUSH further integrates EPIBT-inspired priority inheritance, backtracking, and anytime improvements into its windowed planning. Empirical evaluations across two realistic MAPF scenarios requiring long-horizon
reasoning show that PUSH scales to the same massive
agent loads as EPIBT (e.g., 10k
agents) while achieving significantly higher system throughput than all baselines.
Read the full abstract ▾