Robotics · Cross-listed from cs.LG
Event-Centric World Modeling with Memory-Augmented Retrieval for Embodied Decision-Making
Zhaowen Fan, Rongchao Zhang and Yunxiang Han
Autonomous
agents operating in dynamic environments increasingly demand decision-making systems that are both efficient and interpretable. Hence we propose the Event-Retrieve-Action (ERA) framework, an alternative formulation for
embodied decision-making that bridges the gap between black-box imitation and interpretable
memory retrieval while enabling online refinement without retraining. The environment is represented as structured semantic events encoded into an interpretable latent representation, and decisions are generated by retrieving relevant prior experiences from a knowledge bank of event-action pairs. Final actions are produced through weighted aggregation of retrieved maneuvers, enabling transparent and physically consistent decision-making. Experiments in
UAV navigation demonstrate real-time performance and adaptive behavior in dynamic environments as a representative
embodied decision-making application scenario.
Read the full abstract ▾
Robotics
Evaluation as Evolution: Transforming Adversarial Diffusion into Closed-Loop Curricula for Autonomous Vehicles
Yicheng Guo, Jiaqi Liu, Chengkai Xu et al. (+2 more)
Autonomous vehicles in interactive traffic environments are often limited by the scarcity of safety-critical tail events in static
datasets, which biases learned policies toward average-case behaviors and reduces robustness. Existing evaluation methods attempt to address this through
adversarial stress testing, but are predominantly open-loop and post-hoc, making it difficult to incorporate discovered failures back into the training process. We introduce Evaluation as
Evolution ($E^2$), a closed-loop framework that transforms
adversarial generation from a static validation step into an adaptive evolutionary curriculum. Specifically, $E^2$ formulates
adversarial scenario synthesis as transport-regularized sparse control over a learned reverse-time SDE prior. To make this high-dimensional generation tractable, we utilize
topology-driven support selection to identify critical interacting
agents, and introduce Topological Anchoring to stabilize the process. This approach enables the targeted discovery of failure cases while strictly constraining deviations from realistic data distributions. Empirically, $E^2$ improves collision failure discovery by 9.01% on the nuScenes
dataset and up to 21.43% on the nuPlan
dataset over the strongest baselines, while maintaining low invalidity and high realism. It further yields substantial robustness gains when the resulting boundary cases are recycled for closed-loop policy fine-tuning.
Read the full abstract ▾
All authors Yicheng Guo, Jiaqi Liu, Chengkai Xu, Peng Hang, Jian Sun
Robotics · Cross-listed from cs.CV
MoRight: Motion Control Done Right
Shaowei Liu, Xuanchi Ren, Tianchang Shen et al. (+5 more)
Generating motion-controlled
videos–where user-specified actions drive physically plausible scene dynamics under freely chosen viewpoints–demands two capabilities: (1) disentangled motion control, allowing users to separately control the object motion and adjust camera viewpoint; and (2) motion causality, ensuring that user-driven actions trigger coherent reactions from other objects rather than merely displacing pixels. Existing methods fall short on both fronts: they entangle camera and object motion into a single tracking signal and treat motion as kinematic displacement without modeling causal relationships between object motion. We introduce MoRight, a unified framework that addresses both limitations through disentangled motion modeling. Object motion is specified in a canonical static-view and transferred to an arbitrary target camera viewpoint via temporal cross-view
attention, enabling disentangled camera and object control. We further decompose motion into active (user-driven) and passive (consequence) components, training the model to learn motion causality from data. At inference, users can either supply active motion and MoRight predicts consequences (forward
reasoning), or specify desired passive outcomes and MoRight recovers plausible driving actions (inverse
reasoning), all while freely adjusting the camera viewpoint. Experiments on three
benchmarks demonstrate state-of-the-art performance in generation quality, motion controllability, and interaction awareness.
Read the full abstract ▾
All authors Shaowei Liu, Xuanchi Ren, Tianchang Shen, Huan Ling, Saurabh Gupta, Shenlong Wang, Sanja Fidler, Jun Gao
Robotics
TAMEn: Tactile-Aware Manipulation Engine for Closed-Loop Data Collection in Contact-Rich Tasks
Longyan Wu, Jieji Ren, Chenghang Jiang et al. (+6 more)
Handheld paradigms offer an efficient and intuitive way for collecting large-scale demonstration of
robot manipulation. However, achieving contact-rich bimanual manipulation through these methods remains a pivotal challenge, which is substantially hindered by hardware adaptability and data efficacy. Prior hardware designs remain gripper-specific and often face a trade-off between tracking precision and portability. Furthermore, the lack of online feasibility checking during demonstration leads to poor replayability. More importantly, existing handheld setups struggle to collect interactive recovery data during
robot execution, lacking the authentic tactile information necessary for robust policy refinement. To bridge these gaps, we present TAMEn, a tactile-aware manipulation engine for closed-loop data collection in contact-rich tasks. Our system features a cross-morphology wearable interface that enables rapid adaptation across heterogeneous grippers. To balance data quality and environmental diversity, we implement a dual-modal acquisition pipeline: a precision mode leveraging motion capture for high-fidelity demonstrations, and a portable mode utilizing VR-based tracking for in-the-wild acquisition and tactile-visualized recovery teleoperation. Building on this hardware, we unify large-scale tactile pretraining, task-specific bimanual demonstrations, and human-in-the-loop recovery data into a pyramid-structured data regime, enabling closed-loop policy refinement. Experiments show that our feasibility-aware pipeline significantly improves demonstration replayability, and that the proposed visuo-tactile learning framework increases task success rates from 34% to 75% across diverse bimanual manipulation tasks. We further open-source the hardware and
dataset to facilitate reproducibility and support research in visuo-tactile manipulation.
Read the full abstract ▾
All authors Longyan Wu, Jieji Ren, Chenghang Jiang, Junxi Zhou, Shijia Peng, Ran Huang, Guoying Gu, Li Chen, Hongyang Li
Robotics
RoSHI: A Versatile Robot-oriented Suit for Human Data In-the-Wild
Wenjing Margaret Mao, Jefferson Ng, Luyang Hu et al. (+2 more)
Scaling up
robot learning will likely require human data containing rich and long-horizon interactions in the wild. Existing approaches for collecting such data trade off portability, robustness to occlusion, and global consistency. We introduce RoSHI, a hybrid wearable that fuses low-cost sparse IMUs with the Project Aria glasses to estimate the full 3D pose and body shape of the wearer in a metric global coordinate frame from
egocentric perception. This system is motivated by the complementarity of the two sensors: IMUs provide robustness to occlusions and high-speed motions, while
egocentric SLAM anchors long-horizon motion and stabilizes upper body pose. We collect a
dataset of agile activities to evaluate RoSHI. On this
dataset, we generally outperform other
egocentric baselines and perform comparably to a state-of-the-art exocentric baseline (SAM3D). Finally, we demonstrate that the motion data recorded from our system are suitable for real-world
humanoid policy learning. For
videos, data and more, visit the project webpage: https://roshi-mocap.github.io/
Read the full abstract ▾
All authors Wenjing Margaret Mao, Jefferson Ng, Luyang Hu, Daniel Gehrig, Antonio Loquercio
Robotics
Robots that learn to evaluate models of collective behavior
Mathis Hocke, Andreas Gerken, David Bierbach et al. (+2 more)
Understanding and modeling animal behavior is essential for studying collective motion, decision-making, and bio-inspired robotics. Yet, evaluating the accuracy of behavioral models still often relies on offline comparisons to static trajectory statistics. Here we introduce a reinforcement-learning-based framework that uses a biomimetic robotic fish (RoboFish) to evaluate computational models of live fish behavior through closed-loop interaction. We trained policies in simulation using four distinct fish models-a simple constant-follow baseline, two rule-based models, and a biologically grounded convolutional neural network model-and transferred these policies to the real RoboFish setup, where they interacted with live fish. Policies were trained to guide a simulated fish to goal locations, enabling us to quantify how the response of real fish differs from the simulated fish's response. We evaluate the fish models by quantifying the sim-to-real gaps, defined as the Wasserstein distance between simulated and real distributions of behavioral metrics such as goal-reaching performance, inter-individual distances, wall interactions, and
alignment. The neural network-based fish model exhibited the smallest gap across goal-reaching performance and most other metrics, indicating higher behavioral fidelity than conventional rule-based models under this
benchmark. More importantly, this separation shows that the proposed evaluation can quantitatively distinguish candidate models under matched closed-loop conditions. Our work demonstrates how learning-based robotic experiments can uncover deficiencies in behavioral models and provides a general framework for evaluating animal behavior models through
embodied interaction.
Read the full abstract ▾
All authors Mathis Hocke, Andreas Gerken, David Bierbach, Jens Krause, Tim Landgraf
Robotics
CADENCE: Context-Adaptive Depth Estimation for Navigation and Computational Efficiency
Timothy K Johnsen and Marco Levorato
Autonomous vehicles deployed in remote environments typically rely on embedded processors, compact batteries, and lightweight sensors. These hardware limitations conflict with the need to derive robust representations of the environment, which often requires executing computationally intensive deep neural networks for perception. To address this challenge, we present CADENCE, an adaptive system that dynamically scales the computational complexity of a slimmable
monocular depth estimation network in response to
navigation needs and environmental context. By closing the loop between perception fidelity and actuation requirements, CADENCE ensures high-precision computing is only used when mission-critical. We conduct evaluations on our released open-source testbed that integrates Microsoft AirSim with an NVIDIA Jetson Orin Nano. As compared to a state-of-the-art static approach, CADENCE decreases sensor acquisitions, power consumption, and inference latency by 9.67%, 16.1%, and 74.8%, respectively. The results demonstrate an overall reduction in energy expenditure by 75.0%, along with an increase in
navigation accuracy by 7.43%.
Read the full abstract ▾
Robotics
Robust Quadruped Locomotion via Evolutionary Reinforcement Learning
Brian McAteer and Karl Mason
Deep reinforcement learning has recently achieved strong results in quadrupedal locomotion, yet policies trained in simulation often fail to transfer when the environment changes. Evolutionary reinforcement learning aims to address this limitation by combining gradient-based policy optimisation with population-driven exploration. This work evaluates four methods on a simulated walking task: DDPG, TD3, and two Cross-
Entropy-based variants CEM-DDPG and CEM-TD3. All
agents are trained on flat terrain and later tested both on this domain and on a rough terrain not encountered during training. TD3 performs best among the standard deep RL baselines on flat ground with a mean reward of 5927.26, while CEM-TD3 achieves the highest rewards overall during training and evaluation 17611.41. Under the rough-terrain transfer test, performance of the deep RL methods drops sharply. DDPG achieves -1016.32 and TD3 achieves -99.73, whereas the evolutionary variants retain much of their capability. CEM-TD3 records the strongest transfer performance with a mean reward of 19574.33. These findings suggest that incorporating evolutionary search can reduce overfitting and improve policy robustness in locomotion tasks, particularly when deployment conditions differ from those seen during training.
Read the full abstract ▾
Robotics
An RTK-SLAM Dataset for Absolute Accuracy Evaluation in GNSS-Degraded Environments
Wei Zhang, Vincent Ress, David Skuddis et al. (+2 more)
RTK-SLAM systems integrate simultaneous localization and mapping (SLAM) with real-time kinematic (RTK) GNSS positioning, promising both relative consistency and globally referenced coordinates for efficient georeferenced surveying. A critical and underappreciated issue is that the standard evaluation metric, Absolute Trajectory Error (ATE), first fits an optimal rigid-body transformation between the estimated trajectory and reference before computing errors. This so-called SE(3)
alignment absorbs global drift and systematic errors, making trajectories appear more accurate than they are in practice, and is unsuitable for evaluating the global accuracy of RTK-SLAM. We present a geodetically referenced
dataset and evaluation methodology that expose this gap. A key design principle is that the RTK receiver is used solely as a system input, while ground truth is established independently via a geodetic total station. This separation is absent from all existing
datasets, where GNSS typically serves as (part of) the ground truth. The
dataset is collected with a handheld RTK-SLAM device, comprising two scenes. We evaluate
LiDAR-inertial, visual-inertial, and
LiDAR-visual-inertial RTK-SLAM systems alongside standalone RTK, reporting direct global accuracy and SE(3)-aligned relative accuracy to make the gap explicit. Results show that SE(3)
alignment can underestimate absolute positioning error by up to 76%. RTK-SLAM achieves centimeter-level absolute accuracy in open-sky conditions and maintains decimeter-level global accuracy indoors, where standalone RTK degrades to tens of meters. The
dataset, calibration files, and evaluation scripts are publicly available at https://rtk-slam-
dataset.github.io/.
Read the full abstract ▾
All authors Wei Zhang, Vincent Ress, David Skuddis, Uwe Soergel, Norbert Haala
Robotics
RankFormer: A Propose-then-Select Transformer for Multi-Agent Multimodal Trajectory Prediction
Diyi Liu, Zihan Niu, Tu Xu et al. (+2 more)
Predicting traffic
agent trajectories plays an important role in autonomous driving, traffic operations, transportation safety analysis, etc. Although many deep learning
algorithms are devised to predict future
agent trajectories, the trajectory prediction problem is still challenging due to the complexity of decision-making process, interactions with surrounding vehicles, and the existence of multiple possible intentions for the traveling
agents even under similar scenarios. Most existing methods are limited by the requirement of graph structures (e.g., Graph Neural Network) or the requirement of manually labeled intentions. In this study, we propose a pure
Transformer-based deep learning model for multi-modal trajectory prediction considering temporal dependencies and
agent-
agent spatial interactions. After encoding the historical trajectories, two parallel decoders are employed to generate trajectories and probabilities on separate decoder tracks. The model is evaluated on two real-world
datasets, one highway
dataset and the other pedestrian
dataset with solid performance. One important insight is that following a “propose-then-select” strategy, the
agent-
agent spatial interactions are only considered for probability estimation instead of trajectory generation. In summary, the proposed model provides a potential direction to design more robust and effective multi-modal trajectory prediction models.
Read the full abstract ▾
All authors Diyi Liu, Zihan Niu, Tu Xu, Xingchen Zhang, Lishan Sun
Robotics
Genie Sim PanoRecon: Fast Immersive Scene Generation from Single-View Panorama
Zhijun Li, Yongxin Su, Di Yang et al. (+4 more)
We present Genie Sim PanoRecon, a feed-forward Gaussian-splatting pipeline that delivers high-fidelity, low-cost 3D scenes for robotic manipulation simulation. The panorama input is decomposed into six non-overlapping cube-map faces, processed in parallel, and seamlessly reassembled. To guarantee
geometric consistency across views, we devise a depth-aware fusion strategy coupled with a training-free depth-injection module that steers the
monocular feed-forward network to generate coherent 3D Gaussians. The whole system reconstructs photo-realistic scenes in seconds and has been integrated into Genie Sim - a LLM-driven simulation platform for
embodied synthetic data generation and evaluation - to provide scalable backgrounds for manipulation tasks. For code details, please refer to: https://github.com/AgibotTech/genie_sim/tree/main/source/geniesim_world.
Read the full abstract ▾
All authors Zhijun Li, Yongxin Su, Di Yang, Jichao Wang, Zheyuan Xing, Qian Wang, Maoqing Yao
Robotics
Flow Motion Policy: Manipulator Motion Planning with Flow Matching Models
Davood Soleymanzadeh, Xiao Liang and Minghui Zheng
Open-loop end-to-end neural motion planners have recently been proposed to improve motion planning for robotic manipulators. These methods enable planning directly from sensor observations without relying on a privileged collision checker during motion planning. However, existing planners produce a single path for a given planning problem and cannot exploit their open-loop nature to propose multiple motion plans. To address this limitation, we introduce Flow Motion Policy, an open-loop neural motion planner that uses flow matching to generate a batch of motion plan proposals by learning a distribution over motion plans conditioned on the planning observation. At inference time, it samples multiple candidate motion plans to enable efficient best-of-$N$ inference while avoiding iterative collision checking during planning. We
benchmark the Flow Motion Policy against representative sampling-based, optimization-based and neural motion planning methods. Evaluation results demonstrate that Flow Motion Policy improves planning success and efficiency, highlighting the effectiveness of
stochastic generative policies for end-to-end motion planning and best-of-$N$ sampling. Project website: \href{https://davoodsz.github.io/FlowMotionPolicy.github.io/}{https://davoodsz.github.io/FlowMotionPolicy.github.io/}
Read the full abstract ▾
Robotics
AEROS: A Single-Agent Operating Architecture with Embodied Capability Modules
Xue Qin, Simin Luan, John See et al. (+2 more)
Robotic systems lack a principled abstraction for organizing intelligence, capabilities, and execution in a unified manner. Existing approaches either couple skills within monolithic architectures or decompose functionality into loosely coordinated modules or multiple
agents, often without a coherent model of identity and control authority. We argue that a
robot should be modeled as a single persistent intelligent subject whose capabilities are extended through installable packages. We formalize this view as AEROS (
Agent Execution Runtime Operating System), in which each
robot corresponds to one persistent
agent and capabilities are provided through
Embodied Capability Modules (ECMs). Each ECM encapsulates executable skills, models, and tools, while execution constraints and safety guarantees are enforced by a policy-separated runtime. This separation enables modular extensibility, composable capability execution, and consistent system-level safety. We evaluate a reference implementation in PyBullet simulation with a Franka Panda 7-DOF manipulator across eight experiments covering re-planning, failure recovery, policy enforcement, baseline comparison, cross-task generality, ECM hot-swapping, ablation, and failure boundary analysis. Over 100 randomized trials per condition, AEROS achieves 100% task success across three tasks versus baselines (BehaviorTree.CPP-style and ProgPrompt-style at 92–93%, flat pipeline at 67–73%), the policy layer blocks all invalid actions with zero false acceptances, runtime benefits generalize across tasks without task-specific tuning, and ECMs load at runtime with 100% post-swap success.
Read the full abstract ▾
All authors Xue Qin, Simin Luan, John See, Cong Yang, Zhijun Li
Robotics
Exploring the proprioceptive potential of joint receptors using a biomimetic robotic joint
Akihiro Miki, Shun Hasegawa, Sota Yuzaki et al. (+4 more)
In neuroscience, joint receptors have traditionally been viewed as limit detectors, providing positional information only at extreme joint angles, while muscle spindles are considered the primary sensors of joint angle position. However, joint receptors are widely distributed throughout the joint capsule, and their full role in proprioception remains unclear. In this study, we specifically focused on mimicking Type I joint receptors, which respond to slow and sustained movements, and quantified their proprioceptive potential using a biomimetic joint developed with robotics technology. Results showed that Type I-like joint receptors alone enabled proprioceptive sensing with an average error of less than 2 degrees in both bending and twisting motions. These findings suggest that joint receptors may play a greater role in proprioception than previously recognized and that the relative contributions of muscle spindles and joint receptors are differentially weighted within neural networks during development and
evolution. Furthermore, this work may
prompt new discussions on the differential proprioceptive deficits observed between the elbows and knees in patients with hereditary sensory and autonomic neuropathy type III. Together, these findings highlight the potential of biomimetics-based robotic approaches for advancing interdisciplinary research bridging neuroscience, medicine, and robotics.
Read the full abstract ▾
All authors Akihiro Miki, Shun Hasegawa, Sota Yuzaki, Yuta Sahara, Yoshimoto Ribayashi, Kento Kawaharazuka, Kei Okada
Robotics
KITE: Keyframe-Indexed Tokenized Evidence for VLM-Based Robot Failure Analysis
Mehdi Hosseinzadeh, King Hang Wong and Feras Dayoub
We present KITE, a training-free, keyframe-anchored, layout-grounded front-end that converts long
robot-execution
videos into compact, interpretable tokenized evidence for
vision-
language models (VLMs). KITE distills each trajectory into a small set of motion-salient keyframes with open-vocabulary detections and pairs each keyframe with a schematic bird's-eye-view (BEV) representation that encodes relative object layout, axes, timestamps, and detection confidence. These visual cues are serialized with
robot-profile and scene-context
tokens into a unified
prompt, allowing the same front-end to support failure detection, identification, localization, explanation, and correction with an off-the-shelf VLM. On the RoboFAC
benchmark, KITE with Qwen2.5-VL substantially improves over vanilla Qwen2.5-VL in the training-free setting, with especially large gains on simulation failure detection, identification, and localization, while remaining competitive with a RoboFAC-tuned baseline. A small QLoRA fine-tune further improves explanation and correction quality. We also report qualitative results on real dual-arm
robots, demonstrating the practical applicability of KITE as a structured and interpretable front-end for
robot failure analysis. Code and models are released on our project page: https://m80hz.github.io/kite/
Read the full abstract ▾
Robotics
Differentiable Environment-Trajectory Co-Optimization for Safe Multi-Agent Navigation
Zhan Gao, Gabriele Fadini, Stelian Coros et al. (+1 more)
The environment plays a critical role in multi-
agent navigation by imposing spatial constraints, rules, and limitations that
agents must navigate around. Traditional approaches treat the environment as fixed, without exploring its impact on
agents' performance. This work considers environment configurations as decision variables, alongside
agent actions, to jointly achieve safe
navigation. We formulate a bi-level problem, where the lower-level sub-problem optimizes
agent trajectories that minimize
navigation cost and the upper-level sub-problem optimizes environment configurations that maximize
navigation safety. We develop a differentiable optimization method that iteratively solves the lower-level sub-problem with interior point methods and the upper-level sub-problem with gradient ascent. A key challenge lies in analytically coupling these two levels. We address this by leveraging KKT conditions and the Implicit Function
Theorem to compute gradients of
agent trajectories w.r.t. environment parameters, enabling differentiation throughout the bi-level structure. Moreover, we propose a novel metric that quantifies
navigation safety as a criterion for the upper-level environment optimization, and prove its validity through measure theory. Our experiments validate the effectiveness of the proposed framework in a variety of safety-critical
navigation scenarios, inspired from warehouse logistics to urban transportation. The results demonstrate that optimized environments provide
navigation guidance, improving both
agents' safety and efficiency.
Read the full abstract ▾
All authors Zhan Gao, Gabriele Fadini, Stelian Coros, Amanda Prorok
Robotics
Learning-Based Strategy for Composite Robot Assembly Skill Adaptation
Khalil Abuibaid, Aleksandr Sidorenko, Achim Wagner et al. (+1 more)
Contact-rich robotic skills remain challenging for industrial
robots due to tight
geometric tolerances, frictional variability, and uncertain contact dynamics, particularly when using position-controlled manipulators. This paper presents a reusable and encapsulated skill-based strategy for peg-in-hole assembly, in which adaptation is achieved through Residual Reinforcement Learning (RRL). The assembly process is represented using composite skills with explicit pre-, post-, and invariant conditions, enabling modularity, reusability, and well-defined execution semantics across task variations. Safety and sample efficiency are promoted through RRL by restricting adaptation to residual refinements within each skill during contact-rich interactions, while the overall skill structure and execution flow remain invariant. The proposed approach is evaluated in MuJoCo simulation on a UR5e
robot equipped with a Robotiq gripper and trained using SAC and JAX. Results demonstrate that the proposed formulation enables robust execution of assembly skills, highlighting its suitability for industrial automation.
Read the full abstract ▾
All authors Khalil Abuibaid, Aleksandr Sidorenko, Achim Wagner, Martin Ruskowski
Robotics
Sustainable Transfer Learning for Adaptive Robot Skills
Khalil Abuibaid, Vinit Hegiste, Nigora Gafur et al. (+2 more)
Learning
robot skills from scratch is often time-consuming, while reusing data promotes sustainability and improves sample efficiency. This study investigates policy transfer across different robotic platforms, focusing on peg-in-hole task using reinforcement learning (RL). Policy training is carried out on two different
robots. Their policies are transferred and evaluated for zero-shot, fine-tuning, and training from scratch. Results indicate that zero-shot transfer leads to lower success rates and relatively longer task execution times, while fine-tuning significantly improves performance with fewer training time-steps. These findings highlight that policy transfer with adaptation techniques improves sample efficiency and generalization, reducing the need for extensive retraining and supporting sustainable robotic learning.
Read the full abstract ▾
All authors Khalil Abuibaid, Vinit Hegiste, Nigora Gafur, Achim Wagner, Martin Ruskowski
Robotics
Towards Multi-Object Nonprehensile Transportation via Shared Teleoperation: A Framework Based on Virtual Object Model Predictive Control
Xinyang Fan, Zhaoyang Chen, Shu Xin et al. (+4 more)
Multi-object nonprehensile transportation in teleoperation demands simultaneous trajectory tracking and tray orientation control. Existing methods often struggle with model dependency, uncertain parameters, and multi-object adaptability. We propose a shared teleoperation framework where humans and
robots share positioning control, while the
robot autonomously manages orientation to satisfy dynamic constraints. Key contributions include: 1) A theoretical dynamic constraint analysis utilizing a novel virtual object (VO)-based method to simplify constraints for trajectory planning. 2) An MPC-based trajectory smoothing
algorithm that enforces real-time constraints and coordinates user tracking with orientation control. 3) Validations demonstrating stable manipulation of nine objects at accelerations up to 2.4 m/s2. Compared to the baseline, our approach reduces sliding distance by 72.45% and eliminates tip-overs (0% vs. 13.9%), proving robust adaptability in complex scenarios.
Read the full abstract ▾
All authors Xinyang Fan, Zhaoyang Chen, Shu Xin, Yi Ren, Zainan Jiang, Fenglei Ni, Hong Liu
Robotics
Telecom World Models: Unifying Digital Twins, Foundation Models, and Predictive Planning for 6G
Hang Zou, Yuzhi Yang, Lina Bariah et al. (+15 more)
The integration of machine learning tools into telecom networks, has led to two prevailing paradigms, namely,
language-based systems, such as Large
Language Models (LLMs), and physics-based systems, such as Digital Twins (DTs). While LLM-based approaches enable flexible interaction and automation, they lack explicit representations of network dynamics. DTs, in contrast, offer a high-fidelity network simulation, but remain scenario-specific and are not designed for learning or decision-making under uncertainty. This gap becomes critical for 6G systems, where decisions must take into account the evolving network states, uncertainty, and the cascading effects of control actions across multiple layers. In this article, we introduce the {Telecom World Model}~(TWM) concept, an architecture for learned, action-conditioned, uncertainty-aware modeling of telecom system dynamics. We decompose the problem into two interacting worlds, a controllable system world consisting of operator-configurable settings and an external world that captures propagation, mobility, traffic, and failures. We propose a three-layer architecture, comprising a field world model for spatial environment prediction, a control/dynamics world model for action-conditioned Key Performance Indicator (KPI) trajectory prediction, and a telecom foundation model layer for intent translation and orchestration. We showcase a comparative analysis between existing paradigms, which demonstrates that TWM jointly provides telecom state grounding, fast action-conditioned roll-outs, calibrated uncertainty, multi-timescale dynamics, model-based planning, and LLM-integrated guardrails. Furthermore, we present a proof-of-concept on network slicing to validate the proposed architecture, showing that the full three-layer pipeline outperforms single-world baselines and accurately predicts KPI trajectories.
Read the full abstract ▾
All authors Hang Zou, Yuzhi Yang, Lina Bariah, Yu Tian, Yuhuan Lu, Bohao Wang, Anis Bara, Brahim Mefgouda, Hao Liu, Yiwei Tao, Sergy Petrov, Salma Cheour, Nassim Sehad, Sumudu Samarakoon, Chongwen Huang, Samson Lasaulce, Mehdi Bennis, Mérouane Debbah
Robotics · Cross-listed from cs.DC
Exploiting Aggregate Programming in a Multi-Robot Service Prototype
Giorgio Audrito, Andrea Basso, Daniele Bortoluzzi et al. (+3 more)
Multi-
robot systems are becoming increasingly relevant within diverse application domains, such as healthcare, exploration, and rescue missions. However, building such systems is still a significant challenge, since it adds the complexities of the physical nature of
robots and their environments to those inherent in coordinating any distributed (multi-
agent) system. Aggregate Programming (AP) has recently emerged as a promising approach to engineering resilient, distributed systems with proximity-based communication, and is notably supported by practical frameworks. In this paper we present a prototype of a multi-
robot service system, which adopts AP for the design and implementation of its coordination software. The prototype has been validated both with simulations, and with tests in a University library.
Read the full abstract ▾
All authors Giorgio Audrito, Andrea Basso, Daniele Bortoluzzi, Ferruccio Damiani, Giordano Scarso, Gianluca Torta
Robotics · Cross-listed from cs.CV
VGGT-SLAM++
Avilasha Mandal, Rajesh Kumar, Sudarshan Sunil Harithas et al. (+1 more)
We introduce VGGT-SLAM++, a complete visual SLAM system that leverages the geometry-rich outputs of the Visual Geometry Grounded
Transformer (VGGT). The system comprises a visual odometry (front-end) fusing the VGGT feed-forward
transformer and a Sim(3) solution, a Digital Elevation Map (DEM)-based graph construction module, and a back-end that jointly enable accurate large-scale mapping with bounded
memory. While prior
transformer-based SLAM pipelines such as VGGT-SLAM rely primarily on sparse loop closures or global Sim(3)
manifold constraints - allowing short-horizon pose drift - VGGT-SLAM++ restores high-cadence local bundle adjustment (LBA) through a spatially corrective back-end. For each VGGT submap, we construct a dense planar-canonical DEM, partition it into patches, and compute their DINOv2
embeddings to integrate the submap into a covisibility graph. Spatial neighbors are retrieved using a Visual Place Recognition (VPR) module within the covisibility window, triggering frequent local optimization that stabilizes trajectories. Across standard SLAM
benchmarks, VGGT-SLAM++ achieves state-of-the-art accuracy, substantially reducing short-term drift, accelerating graph convergence, and maintaining global consistency with compact DEM tiles and sublinear
retrieval.
Read the full abstract ▾
All authors Avilasha Mandal, Rajesh Kumar, Sudarshan Sunil Harithas, Chetan Arora
Robotics
RichMap: A Reachability Map Balancing Precision, Efficiency, and Flexibility for Rich Robot Manipulation Tasks
Yupu Lu, Yuxiang Ma and Jia Pan
This paper presents RichMap, a high-precision reachability map representation designed to balance efficiency and flexibility for versatile
robot manipulation tasks. By refining the classic grid-based structure, we propose a streamlined approach that achieves performance close to compact map forms (e.g., RM4D) while maintaining structural flexibility. Our method utilizes theoretical capacity bounds on $\mathbb{S}^2$ (or $SO(3)$) to ensure rigorous coverage and employs an asynchronous pipeline for efficient construction. We validate the map against comprehensive metrics, pursuing high prediction accuracy ($>98\%$), low false positive rates ($1\sim2\%$), and fast large-batch query ($\sim$15 $\mu$s/query). We extend the framework applications to quantify
robot workspace similarity via maximum mean discrepancy (MMD) metrics and demonstrate energy-based guidance for
diffusion policy transfer, achieving up to $26\%$ improvement for cross-embodiment scenarios in the block pushing experiment.
Read the full abstract ▾
Robotics · Cross-listed from cs.CY
Infrastructure First: Enabling Embodied AI for Science in the Global South
Shaoshan Liu, Jie Tang, Marwa S. Hassan et al. (+4 more)
Embodied AI for Science (EAI4S) brings intelligence into the laboratory by uniting perception,
reasoning, and robotic action to autonomously run experiments in the physical world. For the Global South, this shift is not about adopting advanced automation for its own sake, but about overcoming a fundamental capacity constraint: too few hands to run too many experiments. By enabling continuous, reliable experimentation under limits of manpower, power, and connectivity, EAI4S turns automation from a luxury into essential scientific infrastructure. The main obstacle, however, is not algorithmic capability. It is infrastructure. Open-source AI and foundation models have narrowed the knowledge gap, but EAI4S depends on dependable edge compute, energy-efficient hardware, modular robotic systems, localized data pipelines, and open standards. Without these foundations, even the most capable models remain trapped in well-resourced laboratories. This article argues for an infrastructure-first approach to EAI4S and outlines the practical requirements for deploying
embodied intelligence at scale, offering a concrete pathway for Global South institutions to translate AI advances into sustained scientific capacity and competitive research output.
Read the full abstract ▾
All authors Shaoshan Liu, Jie Tang, Marwa S. Hassan, Mohamed H. Sharkawy, Moustafa M. G. Fouda, Tiewei Shang, Zixin Wang
Robotics · Cross-listed from cs.MA
Logical Robots: Declarative Multi-Agent Programming in Logica
Evgeny Skvortsov, Yilin Xia, Ojaswa Garg et al. (+2 more)
We present Logical
Robots, an interactive multi-
agent simulation platform where autonomous
robot behavior is specified declaratively in the logic programming
language Logica.
Robot behavior is defined by logical predicates that map observations from simulated radar arrays and shared
memory to desired motor outputs. This approach allows low-level reactive control and high-level planning to coexist within a single programming environment, providing a coherent framework for exploring multi-
agent robot behavior.
Read the full abstract ▾
All authors Evgeny Skvortsov, Yilin Xia, Ojaswa Garg, Shawn Bowers, Bertram Ludäscher
Robotics
Train-Small Deploy-Large: Leveraging Diffusion-Based Multi-Robot Planning
Siddharth Singh, Soumee Guha, Qing Chang et al. (+1 more)
Learning based multi-
robot path planning methods struggle to scale or generalize to changes, particularly variations in the number of
robots during deployment. Most existing methods are trained on a fixed number of
robots and may tolerate a reduced number during testing, but typically fail when the number increases. Additionally, training such methods for a larger number of
agents can be both time consuming and computationally expensive. However, analytical methods can struggle to scale computationally or handle dynamic changes in the environment. In this work, we propose to leverage a
diffusion model based planner capable of handling dynamically varying number of
agents. Our approach is trained on a limited number of
agents and generalizes effectively to larger numbers of
agents during deployment. Results show that integrating a single shared
diffusion model based planner with dedicated inter-
agent attention computation and temporal convolution enables a train small deploy-large paradigm with good accuracy. We validate our method across multiple scenarios and compare the performance with existing multi-
agent reinforcement learning techniques and heuristic control based methods.
Read the full abstract ▾
All authors Siddharth Singh, Soumee Guha, Qing Chang, Scott Acton
Robotics
BiDexGrasp: Coordinated Bimanual Dexterous Grasps across Object Geometries and Sizes
Mu Lin, Yi-Lin Wei, Jiaxuan Chen et al. (+7 more)
Bimanual dexterous grasping is a fundamental and promising area in robotics, yet its progress is constrained by the lack of comprehensive
datasets and powerful generation models. In this work, we propose BiDexGrasp, consisting of a large-scale bimanual dexterous grasp
dataset and a novel learning-based framework. For
dataset construction, we propose a novel bimanual grasp synthesis pipeline to efficiently annotate physically feasible data. This pipeline addresses the challenges of high-dimensional bimanual grasping through a two-stage synthesis strategy of efficient region-based grasp initialization and decoupled force-closure grasp optimization. Powered by this pipeline, we construct a large-scale bimanual dexterous grasp
dataset, comprising 6351 diverse objects with sizes ranging from 30 to 80 cm, along with 9.53 million annotated grasp data. Based on this
dataset, we further introduce a novel learning-based dexterous grasping generation framework. The framework lies in two key designs: a bimanual coordination module and a geometry-size-adaptive grasp generation strategy to generate coordinated and high-quality grasps on unseen objects. Extensive experiments conducted in both simulation and real world demonstrate the superior performance of our proposed data synthesis pipeline and learned generative framework.
Read the full abstract ▾
All authors Mu Lin, Yi-Lin Wei, Jiaxuan Chen, Yuhao Lin, Shuoyu Chen, Jiangran Lyu, Jiayi Chen, Yansong Tang, He Wang, Wei-Shi Zheng