Robotics
A Bayesian framework for the uncanny valley in humanoid robot design
Shimon Honda, Rin Shibano and Hideyoshi Yanagisawa
The uncanny valley is a long-standing empirical rule in
humanoid robot design: making
robots more human-like can reduce, rather than increase, affinity. Yet existing guidelines, such as adopting
robot-like appearances, avoiding excessive realism, and reducing cross-modal mismatches, remain difficult to use for algorithmic design because they are not expressed as manipulable variables. Here, we propose a hierarchical
Bayesian generative model that operationalizes these guidelines as mathematical design variables. The model represents affinity toward
humanoid robots as posterior-weighted negative category-conditional surprise and explains category ambiguity and perceptual mismatch as increases in surprise. It maps uncanny-valley mechanisms onto four variables: deviation from the predicted
robot-category mean, inconsistency in human likeness across modalities, prediction uncertainty, and observational uncertainty. Simulations showed that category ambiguity and appearance–motion mismatch can produce affinity reductions, and that uncertainty reshapes the valley. In a human-subject experiment with
robot–human morphing images, we manipulated prediction uncertainty using blurred prior
robot stimuli and observational uncertainty using blurred evaluation stimuli. Increased observational uncertainty attenuated the decrease in familiarity ratings at intermediate human likeness, whereas low prediction uncertainty increased ratings for
robot-like appearances. This framework turns empirical uncanny-valley heuristics into a computational basis for algorithmically evaluating and optimizing
humanoid robot appearance and behavior.
Read the full abstract ▾
Robotics
A Risk-Field Enhanced Closed-Loop Digital Twin Framework for Autonomous Driving Safety Validation
Yongzhi Liu
Autonomous driving systems require reliable safety validation before real-world deployment. However, large-scale road testing is costly, difffcult to reproduce, and inefffcient for exposing rare safety-critical scenarios. Conventional simulation improves repeatability, but an offfine simulator alone cannot continuously connect physical trafffc states, virtual reconstruction,
algorithm evaluation, and scenario
evolution. This paper proposes a risk-ffeld enhanced closed-loop digital twin framework for autonomous driving safety validation. The framework integrates physical data acquisition, data synchronization, virtual twin reconstruction, risk-aware scenario generation, autonomous driving
algorithm evaluation, and safety analysis. A driving risk ffeld is introduced as a uniffed intermediate representation to describe obstacle, lane-departure, road-boundary, time-to-collision, and comfort-related risks around the ego vehicle. The risk ffeld ranks high-risk scenarios in the digital twin scenario library and provides dense safety guidance for reinforcement learning-based driving policies. A simulation-style evaluation protocol is designed to compare conventional reinforcement learning baselines, risk-penalty baselines, and the proposed risk-ffeld guided method. The study indicates that
embedding explicit risk structure into digital twins can make autonomous driving validation more targeted, interpretable, and reusable, while its practical effectiveness remains bounded by model ffdelity, risk calibration, and sim-to-real transfer.
Read the full abstract ▾
Robotics
OmniSCS: Omni Safety-Critical Scenario Synthesis for Autonomous Driving via a Fully Editable Driving World
Xiaoyun Dong, Qian Xu, Yang Lu et al. (+3 more)
The synthesis of safety-critical scenarios (SCS) and their evaluation through closed-loop simulations are crucial for developing robust autonomous driving systems. A key aspect of this process involves editing
agent states in both appearance and trajectory levels within existing scenes. However, current methods struggle to preserve data fidelity after scene editing and fail to efficiently generate high-quality SCS through such modifications. To overcome these limitations, we propose OmniSCS, an innovative system that generates photorealistic SCS with high physical fidelity while enabling closed-loop testing in synthetic environments. OmniSCS comprises two key modules: 1) A Fully Editable Driving World Construction module that maintains high-fidelity
agent appearance and background during scene editing via dual-strategy
agent reconstruction and depth-refinement background reconstruction methods. 2) A SCS Synthesis module that facilitates object insertion and
agent trajectory editing to synthesize diverse SCS while preserving data fidelity. Experiments on nuScenes, Waymo, and KITTI
datasets show that OmniSCS outperforms state-of-the-art methods in edited scene fidelity. We further validate its ability to enhance autonomous driving
algorithms and support real-time (13Hz) closed-loop testing. Overall, OmniSCS provides a safer, more effective, and cost-efficient solution for SCS optimization and testing in autonomous driving.
Read the full abstract ▾
All authors Xiaoyun Dong, Qian Xu, Yang Lu, Yang Lou, Yung-Hui Li, Jianping Wang
Robotics
SASGeo: Stability-Aware Semantic Map Localization for GNSS-Denied UAVs – A Framework and Synthetic Proof of Concept
Natalia Trukhina and Vadim Vashkelis
GNSS-denied unmanned aerial vehicles require occasional absolute position fixes to bound the drift of visual-inertial odometry. Cross-view image
retrieval can provide such fixes, but raw appearance is sensitive to season, illumination, viewpoint, map age, and sensor modality. We propose \sas, a semantic map-localization framework that represents the environment through persistent structures such as roads, buildings, waterways, railways, intersections, and field boundaries. The method combines semantic raster
alignment, relational graph evidence, feature stability and geographic distinctiveness, explicit positive/contradictory/unknown observations, and integrity-aware rejection of ambiguous fixes. Unlike a broad architecture-only proposal, this paper specifies concrete weighting and decision models and reports a reproducible synthetic proof of concept. In 220 randomized
retrieval trials with rotation, scale changes, partial crops, occlusion, simulated map changes, and hard semantic decoys, a global semantic descriptor achieved 58.6% Recall@1, while spatial semantic matching variants achieved 94.5-95.5%. Wilson 95% intervals separate the global descriptor from the spatial variants but overlap among the spatial variants, so the experiment supports semantic geometry rather than a definitive benefit from each proposed module. The preliminary experiment does not validate real-flight
navigation; rather, it demonstrates that structured semantic geometry can discriminate locations under controlled cross-view perturbations and identifies the harder aliasing, map-aging, and rejection tests required next.
Read the full abstract ▾
Robotics
Towards Reliable Aerial Ground Vehicle Collaboration: An Integrated Planning and Autonomy Framework for Field Deployment
Md Safwan Mondal, Luca Russo, James D. Humann et al. (+2 more)
Limited flight endurance significantly restricts the operational range of unmanned aerial vehicles (
UAVs) in long duration missions such as surveillance and inspection, where multiple spatially distributed Areas of Interest (AOIs) must be visited. These tasks require efficient routing determining the sequence of visits which directly impacts mission time, energy consumption, and overall feasibility. Pairing
UAVs with unmanned ground vehicles (UGVs) for mobile recharging offers a promising solution, but introduces a tightly coupled cooperative routing problem involving
UAV route planning, UGV road constrained movement, energy management, and rendezvous scheduling under uncertainty. In this work, we present an integrated planning and autonomy framework for reliable field deployment. We formulate the problem as an energy constrained cooperative routing task and solve it using a Deep Reinforcement Learning (DRL) based planner that jointly optimizes the
UAV visitation sequence and rendezvous locations with the UGV, outperforming baseline heuristics in minimizing total mission time. To bridge the gap between planning and execution, we introduce a standardized two layer YAML based mission API that captures environment states and structures lightweight, synchronized action sequences. This framework is supported by a complete autonomy stack using PX4/MAVSDK for
UAV control and ROS 2/Nav2 for UGV
navigation. Furthermore, we propose a lightweight Rendezvous Aware Replanner (RARP) that operates online to handle environmental uncertainties, reducing energy margin violations from 83.33% to 20.00%. The full system is validated through outdoor field experiments, demonstrating robust cooperative
navigation and adaptability in dynamic tasks, including a search and rescue scenario with
vision language model (VLM) based hazard detection
Read the full abstract ▾
All authors Md Safwan Mondal, Luca Russo, James D. Humann, James M. Dotterweich, Pranav Bhounsule
Robotics
CaLiSym: Learning Symplectic Dynamics of Real-World Systems through Structured Canonical Lifts
Aristotelis Papatheodorou, Pranav Vaidhyanathan, Natalia Ares et al. (+2 more)
Physics-informed learning promises data-efficient and stable dynamics prediction, yet its strongest
geometric guarantees have largely remained confined to closed conservative systems. This excludes robotic systems of interest, where actuation, dissipation, and constraints exchange energy and momentum with the environment. We introduce CaLiSym, a lightweight framework that extends
symplectic learning to such systems by changing where the
geometric prior is imposed. Rather than enforcing symplecticity on the measured state, CaLiSym embeds the state and its ports into a lifted phase space, where the dynamics evolve through a
symplectic map. The lift is explicit and algebraic, requiring neither recurrent latent states,
transformer decoders, implicit optimization, nor inference-time numerical integration. We instantiate the framework with SympNet predictors and introduce GRB-SympNet, a B-spline variant combining approximation with exact
symplectic structure. Experiments on a controlled dissipative double pendulum, a real-world quadrotor, and a contact-constrained real-world quadruped demonstrate the lowest out-of-distribution autoregressive rollout error across systems, improving by up to 69.5% while using fewer parameters and up to 85x fewer floating-point operations per step than sequence-model baselines. The lifted dynamics preserve the
symplectic form to numerical precision, extending
symplectic learning beyond conservative mechanics toward real-world robotics.
Read the full abstract ▾
All authors Aristotelis Papatheodorou, Pranav Vaidhyanathan, Natalia Ares, Ioannis Havoutis, Gerard J. Milburn
Robotics
PROBE-X: Learning-Free Cross-FOV Place Recognition
Jinseop Lee
Under field-of-view (FOV) mismatch, pooling
LiDAR features over unequal angular support can distort compact
retrieval keys and exclude correct matches before
geometric verification. We present PROBE-X, a learning-free method for single-scan cross-FOV place recognition. Building on PROBE's probabilistic occupancy representation, which models translation uncertainty, PROBE-X constructs ring-mean
retrieval keys using angular masks conditioned on up to four fixed hypotheses for the unknown relative heading. Candidate lists are merged by maximum cosine similarity. Each retained candidate is aligned under the hypotheses that retrieved it and assigned a
geometric score over the angular overlap at the refined heading. Across controlled-FOV and heterogeneous-
LiDAR experiments, PROBE-X achieves higher Recall@1 and PR-AUC than the evaluated single-scan baselines in most settings with limited-FOV queries and panoramic database scans. On the controlled asymmetric-FOV sweep, single-scan PROBE-X also outperforms the baselines evaluated with up to ten accumulated scans on both metrics. In two ablation settings, removing the angular masks from
retrieval-key construction reduces candidate coverage and Recall@1, even when
alignment and scoring still account for angular support. Project page: https://sites.google.com/view/probex-pr
Read the full abstract ▾
Robotics
A Continual Learning Framework for Adaptive Control of Modular Soft Robots
Nilay Kushawaha, Muhammad Sunny Nazeer, Baljinder Singh Bal et al. (+2 more)
Soft
robots have attracted significant
attention in applications such as medical intervention, rehabilitation, and robotic manipulation due to their inherent compliance, flexibility, and high degrees of freedom. Modular soft
robots (MSRs), composed of multiple interconnected segments, represent an emerging class of robotic systems with highly deformable and reconfigurable structures capable of performing complex tasks. However, designing controllers for MSRs remains challenging due to their nonlinear dynamics, modeling complexity, and hyper-redundant nature. Existing approaches typically require controllers to be retrained from scratch whenever the
robot morphology changes. In this work, we address these challenges through a continual learning inspired control framework capable of incrementally adapting to changes in
robot morphology while preserving previously acquired knowledge. Specifically, the proposed framework enables the controller to sequentially learn new MSR configurations without forgetting previously learned ones. In addition, for MSRs with fixed configurations, the same framework can be employed in a distributed manner to learn module-specific dynamics, enabling localized control and improved precision. The proposed approach is validated through closed-loop trajectory tracking experiments in simulation using a tendon-driven soft
robot, as well as on a real-world three-module pneumatic soft robotic arm. Furthermore, we demonstrate the adaptive capabilities of the framework through a reaching experiment in which the controller selectively activates only the necessary modules to reach a virtual target position, thereby reducing computational overhead.
Read the full abstract ▾
All authors Nilay Kushawaha, Muhammad Sunny Nazeer, Baljinder Singh Bal, Cecilia Laschi, Egidio Falotico
Robotics
EvoPlan: Evolutionary Neuro-Symbolic Robot Planning with Spatio-Temporal Guarantees
Bhavya Sai Nukapotula, Samin Moosavi, Haoze Wang et al. (+4 more)
LLM-based
robot planners are fluent but cannot guarantee that their plans are executable or safe. Classical PDDL planners can guarantee these properties, but only after the problem is fully specified, and they make poor use of an LLM's ability to read context and repair plans. This paper presents a neuro-symbolic framework with three parts. All LLM calls use a locally-hosted open-weight model, so the pipeline can be deployed on-
robot with no cloud dependency. First, an offline procedure that mines a single global Signal Temporal Logic (STL) constraint on mobility from demonstration data. The procedure recovers codified rules (e.g., stopping at red lights, mined from nuPlan driving logs) or population preferences (e.g., social-
navigation comfort, mined from SCAND teleoperation), depending on what the demonstrations encode. Because the demonstrations are a one-class signal, we generate the missing negatives with counterfactual perturbations and an LLM violation generator and then fit the constraint by evolutionary search. We use the mined constraint to shield a
vision-
language driving policy on Bench2Drive and two discrete-action
navigation policies on HA-VLN-CE. Second, an evolutionary PDDL planner: an LLM proposes and repairs plans, programmatic validators decide which ones survive, and the validated portion of the plan grows over iterations. We test the planner on the open-world ALFWorld Text
benchmark, where it beats strong baselines and stays robust when the goal vocabulary does not match the action-model vocabulary. Third, a constrained execution loop: the planner's plan is compiled into waypoints, the waypoints are checked against the mined constraint, and the planner re-plans on a violation. We illustrate the full pipeline via demonstrations using the Gazebo simulator.
Read the full abstract ▾
All authors Bhavya Sai Nukapotula, Samin Moosavi, Haoze Wang, Luke Duncan, Diya Shakkottai, Varun Murali, Srinivas Shakkottai
Robotics
Vision Language Action (VLA) Models for Unmanned Aerial Robotics and Bimanual Manipulation: A Review
Inkyu Sa, Chanoh Park, Hea-Min Lee et al. (+2 more)
Vision Language Action (VLA) models unify visual perception, natural-
language understanding, and action generation within a single foundation model, allowing a
robot to follow instructions such as fold the towel or fly to the red building directly from camera images. Because VLAs inherit world knowledge from internet-scale pre-training, they have become the dominant framework for learning-based manipulation, with bimanual coordination serving as the most demanding testbed: two arms with 7 degrees of freedom each must move in concert to fold, assemble, and reorient objects. Unmanned aerial robotics faces a structurally similar challenge: a drone must coordinate thrust, attitude, and increasingly gripper commands from visual observations under strict latency and payload constraints. This review covers 183 contributions spanning 2017-2026 and organized along seven dimensions: VLA architectures, training recipes, action representations, bimanual coordination (2022-2026), unmanned aerial vehicle (
UAV)
navigation and control (2017-2026),
language grounding, and cross-cutting concerns including
memory and world models. We show that the coordination strategies, training recipes, and action representations developed for bimanual VLAs transfer to unmanned aerial systems and identify fourteen research directions across both domains.
Read the full abstract ▾
All authors Inkyu Sa, Chanoh Park, Hea-Min Lee, Donghee Noh, Ho Seok Ahn
Robotics · Cross-listed from cs.CV
SPEAR: A Simulator for Photorealistic Embodied AI Research
Mike Roberts, Renhan Wang, Rushikesh Zawar et al. (+10 more)
Interactive simulators have become powerful tools for training
embodied agents and generating synthetic visual data, but existing photorealistic simulators suffer from limited generality, programmability, and rendering speed. We address these limitations by introducing SPEAR: A Simulator for Photorealistic
Embodied AI Research. At its core, SPEAR is a Python library that can connect to, and programmatically control, any Unreal Engine (UE) application via a modular plugin architecture. SPEAR exposes over 14K unique UE functions to Python, representing an order-of-magnitude increase in programmable functionality over existing UE-based simulators. Additionally, a single SPEAR instance can render 1920x1080 photorealistic beauty images directly into a user's NumPy array at 73 frames per second - an order of magnitude faster than existing UE plugins - while also providing ground truth image modalities that are not available in any existing UE-based simulator (e.g., a non-diffuse intrinsic image decomposition, material IDs, and physically based shading parameters). Finally, SPEAR introduces an expressive high-level programming model that enables users to specify complex graphs of UE work with arbitrary data dependencies among work items, and to execute these graphs deterministically within a single UE frame. We demonstrate the utility of SPEAR through a diverse collection of example applications: controlling multiple
embodied agents with distinct action spaces (e.g., humans, cars, and
robots) across several in-the-wild UE projects; rendering photorealistic city-scale environments; manipulating UE's procedural content generation systems; rendering synchronized multi-view images of detailed human faces; coordinating an interactive co-simulation with the MuJoCo physics simulator; and editing scenes with natural
language via an AI coding assistant.
Read the full abstract ▾
All authors Mike Roberts, Renhan Wang, Rushikesh Zawar, Rachith Dey-Prakash, Quentin Leboutet, Stephan R. Richter, Matthias Müller, German Ros, Rui Tang, Stefan Leutenegger, Yannick Hold-Geoffroy, Kalyan Sunkavalli, Vladlen Koltun
Robotics
CILC: Cryptographically-secure Inter-agent Loop Closure Candidate Detection for Multi-Agent Collaborative SLAM
Andrew Fishberg, Yixuan Jia and Jonathan P. How
Multi-
agent Simultaneous Localization and Mapping (SLAM) and collaborative SLAM (CSLAM) require
robots to continuously exchange global descriptors (GDs) to detect inter-
agent loop closures (ILCs). While encrypted
radios protect this traffic from external eavesdroppers, they offer no protection against a compromised swarm member. We show this threat is concrete by demonstrating how a corrupted
agent can reconstruct approximations of an honest
agent's imagery and trajectory from its public GD broadcasts. To address this, we propose CILC (Cryptographically-secure Inter-
agent Loop Closure candidate detection), a first-of-its-kind system leveraging Secure Multi-Party Computation (SMPC) to detect ILC candidates without exchanging GDs in the clear. Rather than securing the entire CSLAM pipeline, we apply SMPC only to ILC candidate detection (i.e., GD similarity comparison), a
privacy-sensitive yet computationally lightweight step, yielding an advantageous
privacy-to-overhead trade-off. We validate in both simulation and hardware experiments that CILC remains real-time and communication-feasible across
multimodal GDs (visual and
LiDAR), while mitigating information leakage to a compromised swarm
agent.
Read the full abstract ▾
Robotics
RoboSnap: One-Shot Real-to-Sim Scene Generation for Generalizable Robot Learning and Evaluation
Shujie Zhang, Jingkun Yi, Weipeng Zhong et al. (+6 more)
Recovering real-world scenes as interactive simulation environments can enable generalizable
robot learning and reproducible policy evaluation. However, constructing scenes that are both physically stable and visually faithful remains slow and expensive. In this work, we present RoboSnap, a real-to-sim framework that turns a single RGB image into a simulation-ready scene. The key idea is a layered design that separates the physics-critical interaction area from the surrounding visual context: collision-aware foreground assets are refined for stable
robot interaction, while a 3D Gaussian splatting visual layer preserves faithful background appearance under novel views. Experiments on DROID scenes and real-
robot tasks show that RoboSnap achieves reliable trajectory replay in the recovered scenes, supports task-specific synthetic data generation for policy training, and yields meaningful sim-real correlation for policy evaluation. To further support real-to-sim research, we introduce DROID-Sim, a real-to-sim companion
dataset constructed from 564 real-world scenes in DROID. Extensive experiments suggest that the value of real-to-sim methods lies not only in high-fidelity visual reconstruction, but in turning real environments into reusable infrastructure for
robot learning and evaluation.
Read the full abstract ▾
All authors Shujie Zhang, Jingkun Yi, Weipeng Zhong, Zirui Zhou, Yangkun Zhu, Hanqing Wang, Xudong Xu, Weinan Zhang, Chunhua Shen
Robotics
NativeMEM: Native Memory Compression for Long-Horizon Robotic Manipulation
Ziye Wang, Modi Shi, Chaojun Ni et al. (+5 more)
How can pretrained
Vision-
Language-Action (VLA) models retain long-horizon visual histories with high-frequency updates without sacrificing efficiency? Existing approaches rely on external
memory management, which restrains either the
memory horizon or the reactiveness of pretrained policies. To this end, we present NativeMEM, a VLA policy that features long-term and real-time updated
memory. At its core is an efficient
memory encoding scheme, Native
Memory Compression, which repurposes the VLA's own
vision encoder to compress each historical frame from each camera view into a single
token. Appended to the input sequence, these
memory tokens enable the pretrained VLA to attend over long-term history with negligible latency overhead, requiring neither an external planner nor a freshly initialized
memory module. To align the
memory tokens with the pretrained policy, we first develop a generic
memory tokenizer under the supervision of a frozen VLA on
memory-demanding data, and then unfreeze the VLA for task-specific fine-tuning. NativeMEM consistently outperforms prior methods, boosting success rates from 32.4% to 84.0% in simulation and up to 98.7% on real
robots, while maintaining low inference latency and GPU
memory usage. Notably, NativeMEM exhibits high data efficiency by achieving competitive results with prior arts using only 20% of the training data.
Read the full abstract ▾
All authors Ziye Wang, Modi Shi, Chaojun Ni, Jiazhi Yang, Mengdi Li, Zhizhong Su, Tianwei Lin, Hongyang Li
Robotics
Pelican-VLA 0.5: Attending Before Acting Benefits Generalization
Zeyuan Ding, Wenhai Liu, Yang Xu et al. (+6 more)
In this report, we present Pelican-VLA 0.5, a unified VLA model that integrates
vision-
language understanding, future-frame generation, and action prediction within a single architecture. Pelican-VLA 0.5 achieves
attention-level generalization: without object annotations,
segmentation masks,
attention supervision, or task-specific fine-tuning, its action pathway already focuses on the manipulation-relevant object and contact region. This behavior persists across unseen scenes and unseen
robot embodiments, and is substantially stronger than in other open-source VLA baselines. We verify that this ability originates from the learnable Bottleneck
Token inserted between perception and action: by routing task-relevant visual information through a compact bottleneck, the
tokens interface induces manipulation-centric
attention during pre-training and remains effective across different policy structures, including a MoT-style architecture.
Read the full abstract ▾
All authors Zeyuan Ding, Wenhai Liu, Yang Xu, Jiayu Hu, Yinda Chen, Yi Zhang, Yong Dai, Jian Tang, Xiaozhu Ju
Robotics · Cross-listed from cs.CV
MiLSD: A Micro Line-Segment Detector for Resource-Constrained Devices
Parsa Hassani Shariat Panahi, Amir Hossein Jalilvand and M. Hassan Najafi
Line segment detection is a key building block in visual SLAM, 3D reconstruction, and industrial inspection. Recent deep learning methods have greatly improved accuracy, yet even the smallest models require several megabytes of
memory, exceeding low-cost MCU capacity. This work investigates the maximum achievable accuracy under a sub-megabyte budget. We propose MiLSD, a detector tailored for MCU-level constraints, and systematically compare three output representations within a compact fully-convolutional backbone. Our study shows that the proposed F-Clip center-with-length-and-angle formulation learns most effectively at small model sizes. We find that 8-bit quantization preserves full-precision performance, while 4-bit quantization causes significant degradation, particularly in angle regression, with quantization-aware training recovering only part of the loss. With a one-megabyte activation budget and inference enhancements including sub-pixel decoding, test-time augmentation, and a lightweight verifier, MiLSD improves sAP10 on ShanghaiTech Wireframe from 10.6 (25k parameters, 0.25 MB) to 24.1 within 1 MB. Rather than competing with GPU-scale parsers, we map the accuracy
memory trade-off across representations, bit-widths, capacities, and post-processing strategies for embedded
vision systems.
Read the full abstract ▾
Robotics
Lift3D-VLA: Lifting VLA Models to 3D Geometry and Dynamics-Aware Manipulation
Jiaming Liu, Qingpo Wuwu, Nuowei Han et al. (+8 more)
Recently,
Vision-
Language-Action (VLA) models have demonstrated strong generalization across diverse tasks. However, effective robotic manipulation in physical environments fundamentally requires
geometric understanding and spatial
reasoning. While some VLA approaches attempt to incorporate 3D information, they are constrained by limited data availability and
geometric information loss in current 3D encoding pipelines, and fail to jointly capture 3D geometry and temporally structured actions in dynamic environments. To address these limitations, we introduce Lift3D-VLA, a unified VLA framework that equips models with explicit 3D point cloud
reasoning and enables temporally coherent action generation. First, building upon our previous work Lift3D, an enhanced 2D model-lifting strategy is proposed to geometrically align 3D points with pretrained 2D positional
embeddings. This design enables direct point-cloud encoding within the VLA
vision encoder while minimizing spatial information loss. Based on explicit 3D inputs, we propose Geometry-Centric Masked Autoencoding (GC-MAE), a dual-objective self-supervised framework that reconstructs the current point cloud while predicting its future
geometric evolution. This formulation allows the 2D
vision encoder to internalize both 3D structure and physical dynamics. To fully exploit 3D representations, we further design layer-wise temporal action modeling, which leverages multiple layers of the LLM to collaboratively predict action chunks, enabling temporally consistent predictions. Across 22 simulated tasks and 8 real-world manipulation tasks, Lift3D-VLA achieves 10.8% and 11.1% higher mean success rates on MetaWorld and RLBench than the best-performing prior VLA methods, and outperforms the strongest real-world baseline by 4 percentage points, while exhibiting stronger generalization to out-of-distribution perturbations.
Read the full abstract ▾
All authors Jiaming Liu, Qingpo Wuwu, Nuowei Han, Hao Chen, Zhuoyang Liu, Fan Fei, Yueru Jia, Chenyang Gu, Yandong Guo, Boxin Shi, Shanghang Zhang
Robotics
Embodied Human-Robot Interaction via Acoustics: A MARL Approach with AcoustoBots for Spatial Data Physicalization
Shiqi Liu, Narsimlu Kemsaram, Prateek Mittal et al. (+2 more)
Traditional data physicalization is often static and disconnected from real environments, limiting its ability to convey
embodied spatial dynamics and engage users. To address this limitation, we present AcoustoBots, a mobile acoustophoretic data-physicalization platform in which TurtleBot3
robots carry upward-facing 8 x 8 ultrasonic phased arrays. Each array levitates a particle whose height (1-10 cm) encodes a local urban scalar value, such as population density, noise, or traffic. A MARL (Multi-
Agent Reinforcement Learning) policy based on the Multi-
Agent Deep Deterministic Policy Gradient (MADDPG)
algorithm, with centralized training and decentralized execution, selects collision-aware
navigation actions, while a high-rate Gerchberg-Saxton-Phased Array of Transducers (GS-PAT) acoustic controller maintains trap stability and updates array phases to achieve the commanded height during motion. This creates a closed perception-display-action loop. We evaluate single-
robot city-to-city traversal and dual-
robot cooperative coverage on a 4 m x 3 m scaled UK map using PhaseSpace-based localization for repeatable multi-
robot trials. Results show stable in-motion levitation and consistent, location-dependent height rendering, with task success rates of 90% and 80% for the single and dual-
robot regimes, respectively, over 10 trials per regime, and low collision counts. These findings support acoustophoretic levitation as a simple, glanceable,
robot-mediated communication cue for
embodied human-
robot interaction in spatial analytics.
Read the full abstract ▾
All authors Shiqi Liu, Narsimlu Kemsaram, Prateek Mittal, Pengyuan Wei, Sriram Subramanian
Robotics
RynnWorld-4D: 4D Embodied World Models for Robotic Manipulation
Haoyu Zhao, Xingyue Zhao, Siteng Huang et al. (+3 more)
Robotic manipulation in the open world requires not only recognizing what a scene looks like, but also anticipating how its 3D structure moves under interaction. We argue that synchronized RGB, depth, and optical flow (RGB-DF) provide a physically grounded representation that captures the underlying 4D dynamics of a scene. Compared to 2D pixel
videos, this multi-modal synergy aligns visual appearance with
geometric structure and temporal motion, creating a representation space significantly closer to low-level end-effector actions demanded by robotic systems, narrowing the gap between world prediction and policy learning. Building on this insight, we introduce RynnWorld-4D, a generative model that co-produces future RGB frames, depth maps, and optical flow from a single RGB-D image and a
language instruction within one unified
diffusion process. This 4D world model features a tri-branch architecture that integrates cross-modal
attention with frame-wise 3D RoPE, ensuring that appearance, geometry, and motion evolve consistently. To supply training data at scale, we curate Rynn4DDataset 1.0, a massive
dataset of over 254.4 million frames across
egocentric human and robotic manipulation
videos with high-quality pseudo-labels for depth and optical flow. We further propose RynnWorld-4D-Policy, an inverse dynamics head that consumes the internal 4D representations of RynnWorld-4D in a single forward pass, bypassing expensive multi-step denoising, to output
robot actions in a closed-loop manner. Experiments show that RynnWorld-4D produces temporally and spatially coherent 4D predictions, and that RynnWorld-4D-Policy achieves state-of-the-art performance on real-world dexterous bimanual manipulation tasks, particularly excelling in tasks demanding spatial precision and temporal coordination.
Read the full abstract ▾
All authors Haoyu Zhao, Xingyue Zhao, Siteng Huang, Xin Li, Deli Zhao, Zhongyu Li
Robotics
RynnWorld-Teleop: An Action-Conditioned World Model for Digital Teleoperation
Haoyu Zhao, Xingyue Zhao, Hangyu Li et al. (+6 more)
Scaling
robot learning requires massive, diverse trajectory data, yet collection is currently bottlenecked by physical teleoperation, where every demonstration binds operator time to specific hardware and workspaces. We introduce digital teleoperation, a paradigm that decouples data collection from physical constraints by replacing the real
robot with a generative world model. In this framework, an operator's hand-pose stream drives a
robot-centric generative world model to synthesize high-fidelity
egocentric videos from a single reference image. The recorded pose stream serves as an embodiment-agnostic action label transferable to any target
robot via standard retargeting, yielding complete state-action trajectories for imitation learning independent of physical hardware. We instantiate this paradigm in RynnWorld-Teleop, a system that integrates depth-aware skeletal conditioning, progressive human-to-
robot training on a
video Diffusion Transformer (DiT), and streaming autoregressive
distillation. This pipeline compresses the generative process into a single-pass inference, enabling 40+ FPS, real-time interactive generation on a single H100 GPU. Policies trained exclusively on RynnWorld-Teleop-generated data achieve effective zero-shot Sim2Real transfer across dexterous and diverse bimanual tasks. Moreover, augmenting real-world
datasets with our digitally teleoperated data consistently improves success rates, demonstrating that RynnWorld-Teleop serves as a high-fidelity, scalable data engine for the next generation of robotic
agents.
Read the full abstract ▾
All authors Haoyu Zhao, Xingyue Zhao, Hangyu Li, Biao Gong, Kehan Li, Siteng Huang, Xin Li, Deli Zhao, Zhongyu Li
Robotics
UniLM-Nav: A Unified Framework for Zero-Shot Last-Mile Navigation
Zhuofan Zhang, Tianxu Wang, Guoxi Zhang et al. (+6 more)
Mobile manipulation requires a
robot to navigate to a target object or receptacle and then perform intended manipulation. However, reaching the vicinity of the target does not guarantee a manipulation-ready base pose, a problem known as last-mile
navigation. Prior methods for last-mile
navigation either rely on manual pose annotation or task-specific training, limiting their scalability to open-vocabulary settings with fine-grained spatial constraints. We propose UniLM-Nav, a unified framework for zero-shot open-vocabulary last-mile
navigation. UniLM-Nav decomposes last-mile
navigation into view selection, task-conditioned affordance grounding, and geometry-aware base-pose
reasoning, all resolved with a shared
multimodal large
language model (MLLM) backend. Specifically, UniLM-Nav first selects a reference view that best captures the target object or receptacle from recently collected observations. It then grounds task-relevant affordance point in the selected view and lifts the result into the
robot-centric coordinate frame. Finally, conditioned on the grounded affordance, task context, and
robot geometry, it infers a manipulation-ready base pose for the
robot. We evaluate UniLM-Nav on the OVMM
benchmark, where it outperforms the previous state-of-the-art method, MoTo, by 3.13 percentage points. Analyses show that the components of our method are crucial to final performance, and that the choice of MLLM also has a substantial effect. We further deploy UniLM-Nav on a Unitree B2 quadruped
robot with a 6-DoF Unitree Z1 manipulator, validating its applicability to real-world mobile manipulation tasks.
Read the full abstract ▾
All authors Zhuofan Zhang, Tianxu Wang, Guoxi Zhang, Yixiong Lin, Xilin Wang, Hongming Xu, Qing Li, Song-Chun Zhu, Lifeng Fan
Robotics
Neural-ESO: A Dual-Pathway Architecture for Provably Robust Learning-Based Control
Fan Zhang, Richie Suganda, Jinfeng Chen et al. (+4 more)
A learning-enabled disturbance-rejection framework based on a Neural Extended State Observer (Neural-ESO) is presented in this letter. Unlike existing learning-based control methods that largely rely on the learned model once deployed, Neural-ESO adopts a dual-pathway architecture: a predictive pathway uses a neural network to provide a feedforward disturbance estimate that accelerates convergence, while a corrective pathway employs a conventional ESO to compensate prediction errors and prevent over-reliance on the neural component. Using Lyapunov theory and a small-gain analysis, we show that enforcing a Lipschitz bound on the learning component guarantees uniform ultimate boundedness of the closed-loop error dynamics. The proposed framework is validated on a quadrotor landing task subject to strong ground-effect disturbances across normal and out-of-distribution scenarios, demonstrating accuracy-robustness trade-off and greater operational reliability during training, deployment, and transfer compared with state-of-the-art baselines.
Read the full abstract ▾
All authors Fan Zhang, Richie Suganda, Jinfeng Chen, Wenhua Liu, Hantao Fu, Bin Hu, Qin Lin
Robotics
Hypothesis-driven Model Expansion under Uncertainty for Open-World Robot Planning
Anxing Xiao, Hanbo Zhang, Tianrun Hu et al. (+1 more)
We consider an open-world planning setting in which service
robots must operate in unknown environments with incomplete knowledge of objects and actions. Traditional closed-world approaches with pre-programmed knowledge bases fail when
robots encounter unexpected situations and tasks, posing a fundamental challenge for autonomous knowledge expansion in human environments. In this work, we propose an open-world planning framework that enables
robots to automatically generate, verify, and update hypotheses about their abstract world models. Our key insight is to explicitly maintain uncertainty-aware knowledge expansion and integrate hypothesis verification into goal-reaching planning. The framework leverages foundation models to generate initial hypotheses over states and transitions, and applies automated planning to produce action sequences that jointly address hypothesis verification and task execution. Through iterative execution and refinement, the
robot expands its knowledge by incorporating verification feedback from the foundation models when hypotheses prove incorrect. Extensive experiments in simulated and real-world environments demonstrate that our framework enables autonomous knowledge expansion and effective operation in open-world settings. These results indicate that integrating uncertainty-aware model expansion from
robot foundation models with planning advances the practical deployment of household service
robots.
Read the full abstract ▾
All authors Anxing Xiao, Hanbo Zhang, Tianrun Hu, David Hsu
Robotics
Clustering-Embedded Model Predictive Path Integral Control: Avoiding Averaging-Induced Failure and Enabling Efficient Cluster Selection for Dynamic Obstacles
Zidong Liu, Kaixin Chang and Xu Chen
With the widespread availability of parallel computing hardware, sampling-based motion planning methods such as Model Predictive Path Integral (MPPI) control have become increasingly powerful for complex nonlinear systems in non-smooth task spaces. However, the sampling and forward-simulation pipeline in MPPI suffers from averaging-induced failure in cluttered environments, where the importance-weighted update averages incompatible rollouts and leads to hesitation or even collision when an obstacle lies directly ahead. This paper proposes Clustering-Embedded MPPI (CE-MPPI), a framework that architecturally resolves the averaging-induced failures inherent in standard MPPI within non-convex environments. Rather than simply mitigating interference, CE-MPPI redefines the control law by integrating a high-fidelity
pruning and clustering stage. By leveraging density-based spatial clustering of applications with noise (DBSCAN) alongside a novel
geometric direction feature that is extracted from collision-derived reference points, the system isolates feasible trajectory modes from the noise of infeasible rollouts. This is paired with an intelligent selection logic that optimizes for minimum cost in static scenes while actively steering opposite to obstacle flux in dynamic environments. Experiments in 2-D JAX-accelerated simulations show that CE-MPPI alleviates obstacle-front hesitation and avoids persistent coupling with moving obstacles in dynamic scenes. In particular, real-world tests on a 6-DoF UR5e manipulator with CUDA-parallel rollouts in Isaac Gym achieve a 48% reduction in time-to-goal and a 12% shorter end-effector path.
Read the full abstract ▾
Robotics
Hilti-Trimble-Oxford Dataset: 360 Visual-Inertial Benchmark with Floor Plan Priors for SLAM and Localization
Samuele Centanni, Yuhao Zhang, Yifu Tao et al. (+8 more)
Automated progress monitoring on construction sites is an active area of research and development.
Robot and human-carried mapping systems have been developed to build 3D maps of building and infrastructure projects. While
LiDAR-based mapping systems achieve high accuracy, the cost of
LiDAR can be prohibitive. Consumer-grade cameras with wide field of view ("360 cameras") combined with embedded inertial measurement units (IMUs) provide a cost-effective alternative. To support change detection and progress monitoring, highly accurate visual simultaneous localization and mapping (SLAM) and floor plan localization systems are required. In this paper, we present a high-quality
dataset collected at an active construction site, which captures realistic challenges such as variable lighting conditions, moving workers, fast motions, and repetitive structures. The
dataset offers thirty visual-inertial sequences recorded across ten floors over an eight-month period of the construction project. Ground truth trajectories are collected using a high quality
LiDAR-inertial SLAM system rigidly attached to the 360 camera. Additionally, we report the results of an open research challenge evaluating the best visual SLAM and localization systems from around the world. The Challenge attracted substantially higher participation in SLAM, with 62 teams compared to 22 in floor plan localization, which suggests that this task receives less
attention from researchers and motivates further exploration through proposed
dataset and challenge, available at: https://hilti-trimble-challenge.com/
dataset-2026.
Read the full abstract ▾
All authors Samuele Centanni, Yuhao Zhang, Yifu Tao, Julien Kindle, Frank Neuhaus, Tilman Koß, Aryaman Patel, Michael Helmberger, Emilia Szymańska, Torben Gräber, Maurice Fallon