Robotics · Cross-listed from cs.HC
Driving Through the Network: Performance and Workload Under Latency and Video Impairment
Ines Trautmannsheimer, Ahmed Azab and Frank Diermeyer
Teleoperation promises to extend the operational envelope of automated vehicles, yet it critically depends on network latency and
video quality. We report a fixed-base driving-simulator study (N=25) with a 2x2 manipulation of added latency (100/300 ms) and bitrate (500/2000 kbit/s), plus a best-case baseline (0 ms added, 9000 kbit/s). We measured effective glass-to-glass (G2G) latency per condition (baseline approx. 413 ms; effective totals approx. 500-700 ms) and verified stable framerate and encoder settings.
Multimodal measures covered performance (speed, steering reversals, crashes), oculomotor behavior (blink rate, fixation duration), physiology (RR interval, heart rate, skin conductance), and subjective workload. Latency and bitrate each increased operator load and modestly affected performance. Physiological measures (heart rate, RR interval) exhibited sub-additive interactions, whereas performance and oculomotor interactions were small or non-significant. Equivalence tests showed that 300 ms with 2000 kbit/s was velocity-equivalent to best-case (SESOI +/- 2 km/h), while 300 ms with 500 kbit/s was not. We argue that latency and
video quality should be treated as largely independent design levers, and that physiology-aware adaptation can anticipate overload before safety is compromised.
Read the full abstract ▾
Robotics
Reproducible Dynamic Parameter Identification for a Low-Cost Robot Arm: A Positive-Definiteness Audit for Model Acceptance
Junji Oaki, Koki Yamane, Koki Inami et al. (+1 more)
Dynamic parameter identification of low-cost
robot arms is challenging because limited sensing and drivetrain nonidealities can yield models that predict measured torques well but are physically unsuitable for model-based control. This paper presents a reproducible dynamic parameter identification pipeline for CRANE-X7, a low-cost seven-degree-of-freedom arm driven by modular smart actuators. A 39-parameter OpenSYMORO base-parameter model is identified without CAD inertial data using fully specified single-joint and adjacent-pair excitation, ordinary least squares, a conditional semidefinite-programming projection, and closed-loop input error refinement. Model acceptance is decided by a separate positive-definiteness audit of the identified inertia
matrix over 221,875 sampled configurations. Experiments cover 40 identification trajectories at four sampling intervals and three held-out validation trajectories. The reduced model improves held-out prediction over the full 65-parameter model on all seven identification trajectories used for the model comparison. Fixed-configuration analyses show the distinct role of the feasibility audit: models with similar torque predictions can produce unstable acceleration-resolved dynamics and reverse the direction of the inertia inversion. For the selected identification trajectory, two executions separated by 26 days yield a 1.25% relative spread in held-out root-mean-square error. The accepted model passes audits with five random seeds, and sensitivity analyses quantify the effects of the sampling interval and of torque-constant uncertainty. The complete trajectory specification and numerical record support independent implementation and comparison. These results establish predictive performance, inertia-
matrix feasibility, and repeatability as complementary criteria for evaluatingdynamic models of low-cost arms.
Read the full abstract ▾
All authors Junji Oaki, Koki Yamane, Koki Inami, Sho Sakaino
Robotics
FocalPolicy: Frequency-Optimized Chunking and Locally Anchored Flow Matching for Coherent Visuomotor Policy
Qian He, Zhenshuo Yang, Wenqi Liang et al. (+3 more)
Visuomotor policies aim to learn complex manipulation tasks from expert demonstrations. However, generating smooth and coherent trajectories remains challenging, as it requires balancing proximal precision with distal foresight. Existing approaches typically focus on optimizing intra-chunk action distributions, often neglecting the inter-chunk coherence. Consequently, inter-chunk discontinuities significantly impede the learning of coherent long-horizon actions. To overcome this limitation and achieve a synergetic balance between precision and foresight, we propose FocalPolicy, a foresight-aware visuomotor policy that combines Frequency-Optimized Chunking with Locally Anchored flow matching. We introduce a foresight composite objective that supervises time-domain
alignment within the proximal actions while regularizing frequency-domain structure over multiple future action chunks to improve cross-chunk coherence. To efficiently learn complex action distributions, we design locally anchored sampling to enhance target signal propagation efficiency during consistency flow matching training. Extensive experiments demonstrate that FocalPolicy outperforms existing approaches and confirm the generalizability of our modules to other baselines. Project website: https://focalpolicy.github.io/
Read the full abstract ▾
All authors Qian He, Zhenshuo Yang, Wenqi Liang, Chunhui Hao, Nicu Sebe, Jiandong Tian
Robotics
Dynamic Plasma Shape Control with Arbitrary Sensor Subsets
D. Sorokin, M. Stokolesov, A. Granovskiy et al. (+7 more)
Plasma shape control in tokamaks requires a real-time controller that tracks dynamically changing shape targets while tolerating diagnostic failures. Classical approaches decompose the problem into
equilibrium reconstruction followed by a linear controller, and assume a fixed, fully operational sensor set. We present a reinforcement learning
agent that addresses both limitations simultaneously. The
agent is trained in NSFsim, a high-fidelity tokamak simulator configured for DIII-D, on a curated
dataset of 120 experimental
plasma shapes. The shape targets are resampled as random step changes every 0.25 s, exposing the
agent to diverse transitions across the full shape envelope. At test time the
agent zero-shot tracks dynamic shape sequences; on a held-out static configuration in simulation it achieves a mean shape error of 2.01 cm, and dynamic trajectory following is demonstrated qualitatively in simulation and on the physical device. Diagnostic dropout randomly masks 30% of
magnetic sensors per episode, yielding a single policy robust to arbitrary sensor subsets without backup controllers or mode-switching logic. An asymmetric actor-critic architecture with privileged
equilibrium information improves value estimation under partial observability; an auxiliary shape reconstruction head on the actor enables end-to-end shape reconstruction from raw diagnostics and serves as an interpretability tool for policy analysis. The policy transfers to experimental DIII-D shots, where it directly commands the coil actuators on two dynamic shape maneuvers, and to the independent GSevolve simulator.
Read the full abstract ▾
All authors D. Sorokin, M. Stokolesov, A. Granovskiy, I. Prokofyev, E. Adishchev, M. Nurgaliev, E. Khayrutdinov, G. Subbotin, R. Clark, D. Orlov
Robotics
Designing for Robot Wranglers: A Synthesis of Literature and Practice
David Porfirio, Ian McDermott, Hsin-Mei Chen et al. (+3 more)
Robots are increasingly present in human spaces, such as for conducting deliveries in hospitals, interacting with visitors at museums, and stocking items in warehouses. To ensure the seamless integration of
robots into these spaces, a new role in human-
robot interaction is emerging - the
robot wrangler, namely an individual who is responsible for setting up, overseeing, and troubleshooting the
robot. To understand the needs of this stakeholder, we conducted a scoping review that uncovered a typology of
robot wrangling across the research literature, and discovered that wrangling is an umbrella term that collapses a highly complex and heterogeneous space of activities, often rendering this labor difficult to characterize and support. To further clarify and understand
robot wrangling, we then reflected on our own firsthand and imagined experiences as
robot wranglers within our own respective domains. Guided by the scoping review and our reflections, we devise a series of design implications for supporting wranglers directly as individuals and as members of a wider service ecology.
Read the full abstract ▾
All authors David Porfirio, Ian McDermott, Hsin-Mei Chen, Satoru Satake, Takayuki Kanda, Thomas D. LaToza
Robotics
Structured Jacobian Construction for Motion Optimization with High-Order Time Derivatives in Multi-Link Systems
Taiki Ishigaki, Ko Ayusawa and Eiichi Yoshida
This paper presents a novel framework for Jacobian computation in motion optimization problems involving multi-link systems, where physical quantities are represented using higher-order time derivatives. In motion optimization of
robots and humans, cost functions may incorporate higher-order time derivatives, such as jerk or the time variation of forces, to capture smoothness and perceptual characteristics, particularly in motion skill analysis and expressive behaviors, thereby necessitating Jacobian computations involving these quantities. However, such Jacobians are typically computed using numerical or automatic differentiation without explicitly exploiting the underlying multi-link structure, which can lead to increased computational cost and numerical instability. To address this limitation, we propose a structured Jacobian formulation for motion optimization, based on the comprehensive motion computation framework, in which physical quantities and their higher-order time derivatives are systematically represented along the multi-link structure. The proposed method systematically derives analytical expressions for Jacobians of kinematic and dynamic quantities, including momentum, forces, and joint torques, with respect to generalized coordinates and their higher-order derivatives. The resulting framework is applicable to both direct and inverse optimization. Through numerical experiments, we demonstrate that the proposed method improves computational efficiency compared to numerical and automatic differentiation, while achieving comparable accuracy. Furthermore, we demonstrate its effectiveness in inverse optimization by recovering cost function weights from motion data. Together, these results indicate that the proposed formulation provides a scalable and structured computational foundation for motion optimization involving higher-order time derivatives in multi-link systems.
Read the full abstract ▾
Robotics
GAP: Geometric Anchor Pre-training for Data-Efficient Visuomotor Learning of Manipulation Tasks
Davide Buoso, Andrea Protopapa, Stefano Di Carlo et al. (+2 more)
Learning visuomotor policies from scarce expert demonstrations remains a core challenge in robotic manipulation. A primary hurdle lies in distilling high-dimensional RGB representations into control-relevant geometry without overfitting. While using frozen pre-trained
Vision Foundation Models (VFMs) improves data efficiency, it also shifts most task adaptation onto a small spatial pooling module, which can latch onto task-irrelevant shortcuts and lose
geometric grounding when finetuned with few data samples. More broadly, pre-trained visual representations used for policy learning have been observed to struggle under even minor scene perturbations, highlighting the need for robustness-oriented inductive biases. We propose
Geometric Anchor Pre-training (GAP), a simple, action-free warm-up stage that regularizes the spatial adapter before downstream imitation learning. GAP pre-trains the pooling layer on a lightweight simulated proxy task where object masks are available at no cost, encouraging the adapter to produce keypoints that lie on the object, cover its spatial extent, and remain sharp and repeatable over time. This yields stable
geometric anchors that provide a reliable coordinate interface for few-shot policy learning, while keeping the VFM frozen. We evaluate GAP on RoboMimic and ManiSkill under severe data scarcity (15-50 demonstrations) and domain shift. A simple adapter regularized with GAP consistently outperforms stronger
attention-based poolers and end-to-end fine-tuning, achieving 62% success on RoboMimic Can with 15 demonstrations (+16% over AFA), 63% on the long-horizon high-precision Tool Hang task with 50 demonstrations, and 61% on ManiSkill StackCube with 30 demonstrations (+11% over full fine-tuning). The proxy stage is lightweight and fully decoupled from downstream tasks, making it practical to reuse across environments and manipulation skills.
Read the full abstract ▾
All authors Davide Buoso, Andrea Protopapa, Stefano Di Carlo, Francesca Pistilli, Giuseppe Averta
Robotics · Cross-listed from cs.MA
From Gridworlds to Warehouses: Adapting Lightweight One-shot Multi-Agent Pathfinding for AGVs
Hiroki Nagai and Keisuke Okumura
Multi-
agent pathfinding (MAPF) under one-shot planning is a core component of warehouse automation, yet classical formulations typically assume four-connected 2D grids with unit-time moves in four directions. To fill reality gaps while still being trackable with discrete combinatorial search, this work proposes a more practical counterpart tailored to differential-drive AGVs. We term this multi-
agent warehouse pathfinding (MAWPF), featured with four constraints: (i)
agent actions are restricted to straight motion and in-place rotation; (ii) rotations require multi-step costs; (iii) acceleration and deceleration are considered, and; (iv) follower collisions are prohibited to prevent rear-end crashes. To solve MAWPF efficiently, we adapt representative suboptimal MAPF
algorithms-PP, LNS2, PIBT, and LaCAM-and conduct comprehensive benchmarking. Our experiments reveal that PP and LNS2 struggle to solve instances with many
agents, while PIBT-based approaches achieve preferable scalability with increased solution cost. We believe that these constitute an important step toward adapting classical gridworld MAPF to operational warehouse setups.
Read the full abstract ▾
Robotics
Reactive Robot-Centric Safety for Autonomous Navigation in Constrained and Dynamic Environments
Viswa Narayanan Sankaranarayanan, Vignesh K. Viswanathan, Akshit Saradagi et al. (+2 more)
In this work, we address the problem of ensuring real-time safety in autonomous
robot navigation, in spatially constrained dynamic environments, by utilizing only onboard sensors. We present a real-time control architecture that integrates a 3D
LIDAR perception-based composite control barrier function(CBF)-based safety filter directly into the autonomy pipeline. The proposed perception-driven framework enforces collision avoidance constraints dynamically from onboard point cloud data, thus allowing a large number of constraints to be handled at the control frequency, while remaining minimally invasive to nominal task execution. The safety region is defined as an ellipsoid in the body-frame, consistent with the geometry of the platform, which induces time-varying constraints in the world frame as the
robot rotates; this effect is handled through a dedicated formulation of time-varying (CBF) for each
LIDAR point. We validate the system through multiple field experiments in underground environments by utilizing a quadruped platform performing a visual inspection task, demonstrating reliable operation in the presence of dynamic obstacles, unsafe high-level references, abrupt localization anomalies, and while traversing through narrow corridors.
Read the full abstract ▾
All authors Viswa Narayanan Sankaranarayanan, Vignesh K. Viswanathan, Akshit Saradagi, Sumeet Satpute, George Nikolakopoulos
Robotics
A Topology-Aware Spatiotemporal Handover Framework for Continuous Multi-UAV Tracking
Jianlin Ye, Christos Kyrkou and Panayiotis Kolios
The integration of Unmanned Aerial Vehicles(
UAVs) into Intelligent Transportation Systems (ITS) offers synoptic visibility for traffic monitoring, yet scalable deployment is hindered by trajectory fragmentation, where vehicle identity persistence is lost across multi-
UAV Fields of View (FOV). While state-of-the-art frameworks excel in optimizing local trajectory extraction and stability for single-drone imagery, they often function as isolated data silos that generate disjointed trajectories, thereby precluding network-level analysis such as Origin-Destination estimation. This paper presents a real-time Multi-Camera Multi-Vehicle Tracking (MCMT) system designed to handle global identity persistence. Addressing the visual ambiguity and computational cost of appearance-based Re-Identification (Re-ID) in nadir views, we introduce a lightweight
Topology-Based Spatiotemporal Handover mechanism. We implement a high-throughput parallel pipeline leveraging YOLO11 and ByteTrack to process concurrent 4K streams. Our core contribution is a deterministic queue-based matching
algorithm that utilizes
geometric overlaps and virtual lane discretization to predictively manage identity handover via FIFO queues. Experimental results on complex urban environments, including intersections and merging traffic, demonstrate a Handover Success Rate (HOSR) of 99.8% in continuous traffic flows, significantly outperforming Re-ID baselines (74.1%) while validating edge deployment feasibility. The source code is available at https://github.com/JYe9/multi-camera-multi-vehicle-tracking-system.
Read the full abstract ▾
Robotics
Lamarckian Inheritance in Dynamic Environments: How Key Variables Affect Evolutionary Dynamics
K. Ege de Bruin, Kyrre Glette and Kai Olav Ellefsen
The co-optimization of a
robot's body and
brain presents a coupled challenge: the morphology constrains which control strategies are effective, while the control determines how well the morphology performs. To address this, we combine morphology optimization as
evolution with controller optimization as lifetime learning, utilizing Lamarckian inheritance to transfer learned controller parameters from parent to offspring. In dynamic environments, existing literature presents conflicting evidence: while traditional evolutionary theory often suggests Lamarckian inheritance lacks benefit, recent studies in evolutionary robotics indicate it can improve performance. We hypothesize that this is because previous works have not included all relevant variables with dynamic environments. In this work, we show that the benefit of Lamarckian inheritance depends on two variables: how conflicting the environmental changes are to
robot control, and the predictability of those changes for the robotic
agent. Using virtual soft
robots and two different learning approaches,
Bayesian optimization and reinforcement learning, we show that Lamarckian inheritance only underperforms Darwinian inheritance when the changes are both conflicting and unpredictable. We find that adding a sensor to detect environmental changes restores the benefits for Lamarckian inheritance in conflicting environments, by allowing robotic
agents to predict the need for a different behavior, thereby generalizing their control.
Read the full abstract ▾
Robotics · Cross-listed from cs.CG
Optimizing Line Segment Inspection with Limited-Range Drones
José-Miguel Díaz-Báñez, José-Manuel Higes, Alina Kasiuk et al. (+1 more)
Optimization problems with drones are widely studied in a variety of civilian tasks, mainly due to their ability to traverse rough terrains and to carry cameras and other sensors for surveillance tasks. The limited battery life of these aerial
robots poses challenges in operational research. In this paper, we address the following optimization problem. We are given a set of line segments (e.g. tubes in a
solar plant) to inspect by drones. The objective is to detect broken pipes using artificial intelligence and path planning must be carried out efficiently. On the one hand, the limited capacity of the batteries necessitates periodic visits (tours) to a fixed base station. However, it is desirable to allocate a set of tours for each drone to ensure that the segments are covered as quickly as possible, aiming to minimize the makespan, which is the maximum time spent by any drone. We are able to prove that this optimization problem is strongly NP-hard even when the segments are positioned on a line and the scenario involves only two drones. Then, approximation
algorithms are proposed. Our computational experiments demonstrate that the proposed
algorithm achieves near-optimal performance across diverse operational scenarios.
Read the full abstract ▾
All authors José-Miguel Díaz-Báñez, José-Manuel Higes, Alina Kasiuk, Inmaculada Ventura
Robotics
Hierarchical and Holistic Open-Vocabulary Functional 3D Scene Graphs for Indoor Spaces
Xinggang Hu, Chenyangguang Zhang, Alexandros Delitzas et al. (+4 more)
Functional 3D scene graphs offer a versatile and flexible representation for 3D scene understanding and robotic manipulation, defined by object nodes, interactive elements, and functional relationship edges. However, their potential remains underexplored due to the limited coverage of existing
benchmarks and the overly straightforward design of previous pipelines, which primarily focus on large-scale furniture but lack of hierarchical structures. Therefore, in this work, we extend the
benchmark coverage by introducing dense tabletop objects and explicit multi-level functional relationships. This expansion introduces critical challenges involving small-scale, dense, and similar instances, with lack of visual anchoring in relational
reasoning, instance confusion during cross-frame fusion, and attribution uncertainty under dynamic viewpoints. To address these issues, we propose an open-vocabulary pipeline based on 2D visual grounding and 3D graph optimization. Specifically, we anchor fine-grained functional edges from 2D visual evidence, and associate nodes across frames in 3D using multiple cues. Furthermore, edge association is formulated as temporal graph optimization, integrating evidence accumulation,
entropy regularization, and temporal smoothing to robustly determine the functional connections of each node. Finally, global hierarchy shaping is performed to recover the hierarchical graph structure. Extensive experiments demonstrate that the proposed method can reliably infer functional 3D scene graphs in challenging real-world scenes, thereby further unlocking their potential for practical applications.
Read the full abstract ▾
All authors Xinggang Hu, Chenyangguang Zhang, Alexandros Delitzas, Xiangkui Zhang, Marc Pollefeys, Francis Engelmann, Xiangyang Ji
Robotics · Cross-listed from cs.CV
DiLA: Disentangled Latent Action World Models
Tianqiu Zhang, Muyang Lyu, Yufan Zhang et al. (+2 more)
Latent Action Models (LAMs) enable the learning of world models from unlabeled
video by inferring abstract actions between consecutive frames. However, LAMs face a fundamental trade-off between action abstraction and generation fidelity. Existing methods typically circumvent this issue by using two-stage training with pre-trained world models or by limiting predictions to optical flow. In this paper, we introduce DiLA, a novel Disentangled Latent Action world model that aims to resolve this trade-off via content-structure disentanglement. Our key insight is that disentanglement and latent action learning are co-evolving: the predictive bottleneck inherent in latent action learning serves as a driving force for disentanglement, compelling the model to distill spatial layouts into the structure pathway while offloading visual details to a separate content pathway for generation. This synergy yields a continuous, semantically structured latent action space without compromising generative quality. DiLA achieves superior results in
video generation quality, action transfer, visual planning, and
manifold interpretability. These findings establish DiLA as a unified framework that simultaneously achieves high-level action abstraction and high-fidelity generation, advancing the frontier of self-supervised world model learning.
Read the full abstract ▾
All authors Tianqiu Zhang, Muyang Lyu, Yufan Zhang, Fang Fang, Si Wu
Robotics
Learning Dynamic Pick-and-Place for a Legged Manipulator
Moonkyu Jung, Jiseong Lee, Zhengmao He et al. (+8 more)
Legged manipulators extend robotic capabilities beyond static manipulation by integrating agile locomotion with versatile arm control. However, achieving precise manipulation while maintaining coordinated locomotion remains a major challenge. This work presents a hierarchical reinforcement learning framework for dynamic pick-and-place tasks using a quadruped equipped with a 6-DOF robotic arm. The framework incorporates an explicit mass estimation module enabling adaptive whole-body control for objects with varying weights. In simulation, the system achieves an 86.05% success rate with payloads up to 2.3 kg. The approach is further validated through real-world experiments across six representative scenarios with controlled variations in object physical properties (size and mass) and task heights. Specifically, within a wide vertical workspace ranging from ground level to 1.1~m-high tabletops, the system demonstrates an average success rate of 73.3% for payloads up to 1.3 kg, with an average execution time of 4.06 s. Unlike prior works that handle lightweight objects and execute pick-and-place motions with slow, piecewise motions, the proposed framework exploits concurrent locomotion and manipulation for dynamic, continuous execution. These results demonstrate the potential of quadrupedal mobile manipulators for adaptive, whole-body pick-and-place with heavier payloads and extended workspaces.
Read the full abstract ▾
All authors Moonkyu Jung, Jiseong Lee, Zhengmao He, Donghoon Youm, Juhyeok Mun, HyeongJun Kim, Hyunsik Oh, Donghyuk Choi, Jungwoo Hur, Jie Song, Jemin Hwangbo
Robotics
Feedback World Model Enables Precise Guidance of Diffusion Policy
Tuo An, Jindou Jia, Gen Li et al. (+8 more)
World models aim to improve robotic decision making by predicting the consequences of actions. However, in practice, their predictions often become unreliable once the
robot encounters states outside the training distribution, limiting their effectiveness at deployment. We observe that execution itself provides a natural but underutilized signal: after each action, the
robot directly observes the true next state, revealing the mismatch between predicted and actual outcomes. Building on this insight, we propose feedback world model, an inference-time mechanism that closes the loop between prediction and observation at inference time. Instead of treating the world model as a static open-loop predictor, our method maintains a lightweight feedback state that is updated online to iteratively correct future predictions, compensating for model errors using real-time observations without additional training data or parameter updates. We show that this process can be interpreted as a latent-space observer and admits convergence guarantees under bounded, asymptotically convergent dynamics residuals and a stable feedback gain. We further introduce action-aware guidance to better translate corrected predictions into control by emphasizing action-controllable components while suppressing irrelevant variations. Experiments on LIBERO-Plus, Robomimic, and real-world manipulation tasks demonstrate that our method substantially improves both prediction accuracy and policy performance under distribution shift, reducing world model prediction error by up to 76.4%. The same feedback mechanism improves the predictions of the pretrained foundation model Fast-WAM-Joint without updating its parameters, demonstrating its plug-and-play applicability. These results show that incorporating real-time feedback at inference time provides a simple yet powerful alternative to static world modeling.
Read the full abstract ▾
All authors Tuo An, Jindou Jia, Gen Li, Jingliang Li, Chuhao Zhou, Pengfei Liu, Bofan Lyu, Jiaqi Bai, Xinying Guo, Geng Li, Jianfei Yang
Robotics
PCASim: Promptable Closed-loop Adversarial Simulation for Urban Traffic Environment
Chuancheng Zhang, Zhenhao Wang, Kaizheng Li et al. (+3 more)
Real-world autonomous driving, particularly in urban environments with numerous corner cases, requires rigorous testing to ensure product safety and robustness. However, few studies have explored integrating
adversarial scenario generation with the training of safety
agents in closed-loop testing, enabling efficient co-
evolution and mutual enhancement of both. To address this challenge, an
adversarial behavior knowledge repository is constructed by applying rule-based filtering to an open-source
dataset, combined with knowledge
retrieval modules tailored for simulation environments. A large
language model (LLM) is employed to integrate knowledge-, data-, and
adversarial-driven approaches, generating safety-critical traffic scenarios customized to user needs. Additionally, while evaluating the generated scenarios, we employ reinforcement learning models to train the behaviors of different types of vehicles, thereby enriching scenario diversity beyond existing
datasets while preserving realism. Experimental results demonstrate that the proposed framework improves the accuracy of domain-specific
language generation by 12%. Moreover, the success rate of newly generated scenario transformations increases by 8%, while obstacle-avoidance capability is enhanced by 30%. For the complete manuscript, please refer to: https://zhenhaooo.github.io/PCASim.github.io/
Read the full abstract ▾
All authors Chuancheng Zhang, Zhenhao Wang, Kaizheng Li, Yaran Lin, Qiang Guo, Bin Jiang
Robotics
MyoChallenge 2025: A New Benchmark for Human Athletic Intelligence
Cheryl Wang, Chun Kwang Tan, Balint K. Hodossy et al. (+21 more)
Athletic performance represents the pinnacle of human motor intelligence, demanding rapid choices, precise control, agility, and coordinated physical execution. Replicating this seamless combination of capabilities remains elusive in current artificial intelligence and robotic systems. Concurrently, understanding the biological mastery of these movements is hindered because complex muscle coordination is rarely measured in vivo due to the limitations of physical equipment. To bridge this fundamental gap in understanding, MyoChallenge at NeurIPS 2025 established a pioneering
benchmark for motor control intelligence in sports, leveraging high-fidelity musculoskeletal models within physics simulation combined with machine learning-driven
algorithms. The competition introduces two distinct tracks emphasizing either upper or lower limbs control: a table tennis rally task utilizing a biomechanic upper limb composed of an arm with a hand and a trunk; and a soccer penalty kick using a biomechanic model of legs and a trunk. Marking the fourth iteration of the MyoChallenge series, this event attracted almost 70 teams and over 560 submissions globally, uniting a diverse community ranging from physicians and neuroscientists to machine learning experts. The competition facilitated the development of several state-of-the-art control
algorithms for a musculoskeletal system capable of sports agility, leveraging techniques such as physics-based motion planners, on-policy behaviour cloning, hierarchical planning, and muscle synergies. By integrating standardized tasks and physiologically realistic models into the open-source framework of MyoSuite, MyoChallenge'25 serves as a reproducible and reusable testbed to accelerate interdisciplinary research across machine learning, biomechanics, sports science, and neuroscience. Project page: https://www.myosuite.org//myochallenge/myochallenge-2025.
Read the full abstract ▾
All authors Cheryl Wang, Chun Kwang Tan, Balint K. Hodossy, Eric Lyu, Jun Guo, Wentao Zhao, Huaping Liu, Chengkun Li, Merkourios Simos, Bianca Ziliotto, Alexander Mathis, Siyuan Liu, Jiahao Chen, Shanlin Zhong, Bo Jiang, Ci Song, Yaoye Zhu, Chenhui Zuo, Yanan Sui, Mohamed Irfan Refai, Massimo Sartori, Guillaume Durandau, Vikash Kumar, Vittorio Caggiano
Robotics
Propagating Unsafe Actions in LLM Controlled Multi-Robot Collaboration via Single Robot Compromise
Zhen Huang, Zhihuang Liu, Mengxuan Luo et al. (+2 more)
Large
language models (LLMs) are increasingly used as general planners in
embodied intelligence, enabling high level coordination and low level task planning for both single
robot and multi-
robot collaboration. This increasing reliance on
embodied LLM planners also raises critical security concerns, since misaligned or manipulated instructions can be translated into physical actions. Prior work has studied such threats in single
robot settings, while security risks in LLM controlled multi-
robot collaboration, especially those propagated through inter
robot communication, remain largely unexplored. To bridge this gap, we propose a novel attack paradigm for multi-
robot system in which the adversary interacts with only a single entry
robot. The compromised
robot then propagates malicious intent through peer communication, leading to coordinated unsafe actions across the system. Our evaluation, covering high risk dimensions of dereliction of duty,
privacy compromise, and public safety hazards, reveals a persistent safety
alignment gap in multi-
robot planners. We quantify this process with three metrics, obedience, infectiousness, and stealthiness. Experiments demonstrate both persistent attacker control and rapid propagation: obedience reaches 1.00 in the strongest cases, and infectiousness rises to 0.90. Notably, the attack is highly efficient, requiring as few as 3.0 rounds to compromise all the
robots while maintaining a stealthiness score of 0.81. Such risks are amplified when
robots must resolve trade offs in critical situations, such as emergencies or conflicts of rights, because the coordination mechanism can unintentionally allow
adversarial instructions to override safety requirements. The code is available at https://github.com/TheFatInsect/InfectBot.
Read the full abstract ▾
All authors Zhen Huang, Zhihuang Liu, Mengxuan Luo, Weishang Wu, Zhiping Cai
Robotics
Wind-Aware Optimal Trajectory Planning for Efficient Gliding of Fixed-Wing Aerial Systems
Luca Morando, Nishanth Bobbili and Giuseppe Loianno
Gliding offers small fixed-wing
UAVs extended endurance and silent operation but requires accurate energy management, especially under wind disturbances and obstacle constraints. Traditional Total Energy Control Systems based controllers regulate the trade between potential and kinetic energy reactively, often requiring fine-tuning and trim-conditions knowledge. In this work, we shift the regulation to the planning level and present a nonlinear, multi-cost trajectory planner for small
UAV gliders. The method generates $\mathcal{C}^3$ continuous trajectories based on Bernstein
polynomials, mapped into control commands through differential flatness, and re-planned online to match experimentally derived sink polar curves. A simulated netto variometer is integrated into the optimization to estimate air mass motion, constraining the glide to energy-balanced states. Consecutive gliding trajectories are linked by cruising segments computed through trajectories initialized on Dubins path-based waypoints, enabling hybrid missions that combine powered and unpowered flight. The approach is validated in CFD simulations and real-world experiments with a fixed-wing platform, showing reliable stabilization of sink rate, airspeed, and glide ratio under wind gusts and in presence of obstacles.
Read the full abstract ▾
Robotics · Cross-listed from cs.CV
CM-EVS: Sparse Panoramic RGB-D-Pose Data for Complete Scene Coverage
Jiale Liu, Jungang Li, Jieming Yu et al. (+12 more)
Modern 3D visual learning relies on observations sampled from metric 3D assets, yet existing scans, meshes, point clouds, simulations, and reconstructions do not directly provide a sparse, comparable, and geometry-consistent panoramic training interface. Dense trajectories duplicate nearby views, source-specific rendering policies yield heterogeneous annotations, and sparse heuristics may miss important regions or introduce depth-inconsistent observations. We study how to convert 3D assets into sparse panoramic RGB-D-pose data that preserves complete scene coverage with low redundancy and auditable provenance. We propose COVER (Coverage-Oriented Viewpoint curation with ERP Range-depth warping), a training-free ERP viewpoint curator that projects geometry observed from selected views into candidate ERP probes, scores incremental coverage, and penalizes depth conflicts. Under bounded proxy error, its greedy coverage proxy preserves the standard coverage-style approximation behavior up to an additive error term. Using COVER, we build CM-EVS (Coverage-curated Metric ERP View Set), a panoramic RGB-D-pose
dataset with 36,373 curated ERP frames from 1,275 indoor scenes across Blender indoor, HM3D, and ScanNet++, complemented by outdoor panoramas from TartanGround and OB3D re-encoded into the same schema. Each frame provides full-sphere RGB, metric range depth, calibrated pose; COVER-produced indoor frames include per-step provenance logs. With a median of only 25 frames per indoor scene, CM-EVS covers all 13 unified room types while maintaining compact scene-level coverage. Experiments show that COVER improves the coverage-conflict trade-off, making CM-EVS a sparse, compact, and auditable RGB-D-pose resource for geometry-consistent panoramic 3D learning.
Read the full abstract ▾
All authors Jiale Liu, Jungang Li, Jieming Yu, Xinglin Yu, Zihao Dongfang, Keyu Shi, Zongjian Ding, Jiahuan Zhang, Shunwen Bai, Haoran Huang, Yurun Wang, Yanxi Wu, Ningzhe Yu, Yudong Gao, Mingjun Cheng
Robotics
NavRL++: A System-Level Framework for Improving Sim-to-Real Transfer in Reinforcement Learning-Based Robot Navigation
Zhefan Xu, Hanyu Jin and Kenji Shimada
Recent years have witnessed significant progress in autonomous
navigation using reinforcement learning. However, existing approaches largely emphasize reinforcement learning framework design, such as input representations, action spaces, and reward functions, while providing limited analysis of sim-to-real transfer and insufficient insight into how training strategies affect real-world deployment performance. To bridge this gap, we not only introduce an effective RL framework but also present a complete training and deployment pipeline, along with a systematic empirical study that disentangles the key factors affecting sim-to-real transfer in reinforcement learning-based
navigation, including sensor noise, perception failures, system latency, and control response. Building on insights from this analysis, we introduce perturbation-aware fine-tuning, a post-training adaptation strategy that improves transfer robustness by explicitly accounting for empirically identified domain discrepancies. To further mitigate perception degradation and enhance control smoothness in real-world deployment, we propose a
Transformer-based temporal
reasoning policy that leverages short-horizon observation for
navigation control. We quantitatively evaluate how individual sim-to-real perturbations and training design choices impact
navigation performance across environments. Experimental results demonstrate that the proposed training strategy and policy architecture outperform learning-based baselines in both static and dynamic environments, while achieving performance comparable to optimization-based planners in static settings. We validate our approach through real-world deployment on multiple robotic platforms, including aerial and legged
robots, across
navigation-centric tasks such as exploration and inspection, demonstrating zero-shot sim-to-real transfer.
Read the full abstract ▾
Robotics
KaRMA: A Kinematic Metric for Fine Manipulation Ability in Robotic Hands
Martin Peticco and Pulkit Agrawal
Traditional robotic hand metrics focus on static properties such as workspace, manipulability, and grasp stability. However, these metrics do not directly measure dexterity under the standard definition in robotic manipulation: the ability to continuously change an object's pose within the hand while maintaining contact from an initial grasp. We introduce Kinematic Rolling Manipulation Ability (KaRMA), a kinematic-only metric for fine manipulation that quantifies reachable in-hand translation and reorientation of a spherical test object within a two-finger precision pinch through feasible rolling motions. KaRMA enforces joint limits, collision constraints, rolling contact, and antipodal force feasibility, then explores reachable in-hand object poses via breadth-first search over translation and rotation primitives. KaRMA reports three scores: translational coverage (KaRMA-T), rotational coverage (KaRMA-R), and sensitivity to the initial grasp (KaRMA-S). We evaluate KaRMA on 16 widely used robotic hands and compare against static baselines, showing that KaRMA separates hands that rank identically under static proxies, reveals translation-rotation tradeoffs invisible to existing baselines, and is qualitatively consistent with selected published task
benchmarks where Jacobian-based metrics can be misleading. Code and interactive demos are available at https://martinpeticco.com/karma.
Read the full abstract ▾
Robotics
SkiP: When to Skip and When to Refine for Efficient Robot Manipulation
Mingtong Dai, Guanqi Peng, Yongjie Bai et al. (+5 more)
Previous imitation learning policies predict future actions at every control step, whether in smooth motion phases or precise, contact-rich operation phases. This uniform treatment is wasteful: most steps in a manipulation trajectory traverse free space and carry little task-relevant information, while a small fraction of key steps around contacts, grasps, and
alignment demand dense, high-resolution prediction. We propose a novel action relabeling mechanism: at each timestep in a skip segment, we replace the behavior cloning target with the action at the entrance of the next key segment, enabling the policy to leap over redundant steps in a single decision. The resulting Skip Policy (SkiP) dynamically leaps over skip segments and intensively refines actions in key segments, within a single unified network requiring no learned skip planner or hierarchical structure. To automatically partition demonstrations into key and skip segments without manual annotation, we introduce Motion Spectrum Keying (MSK), a fast, task-agnostic procedure that detects local motion complexity from action signals. Extensive experiments across 72 simulated manipulation tasks and three real-
robot tasks show that SkiP reduces executed steps by $15$–$40\%$ while matching or improving success rates across various policy backbones. Project page: https://pgq18.github.io/SkiP-page/.
Read the full abstract ▾
All authors Mingtong Dai, Guanqi Peng, Yongjie Bai, Feng Yan, Chunjie Chen, Lingbo Liu, Liang Lin, Xinyu Wu
Robotics
Task-Semantic Graph-Driven Distributed Agent Networking for Underwater Target Tracking
Shengchao Zhu, Guangjie Han, Chuan Lin et al. (+1 more)
Autonomous
underwater vehicle (AUV) swarms are emerging as intelligent
underwater networks, where each node must sense, communicate, process local data, and make decisions under severe acoustic constraints. Persistent
underwater target tracking is a typical task with moving targets, changing communication
topology, intermittent acoustic links, and limited observation for each AUV. Multi-
agent reinforcement learning (MARL) is a natural candidate for distributed tracking, yet existing studies still lack a unified open-source platform for evaluating different MARL
algorithms under six-degree-of-freedom AUV dynamics. In addition, policies trained with raw
geometric states and low-level force actions often struggle to represent task phases, observation reliability, link quality, and local cooperation roles. This paper addresses these issues by developing an open-source MARL-AUV platform that integrates DI-engine with a six-degree-of-freedom
underwater AUV target-tracking simulator. To the best of our knowledge, it is the first open platform that connects a public MARL training framework with physically modeled AUV swarm-based tasks, and provides a unified experimental protocol for fair training, testing, and comparison of representative RL and MARL
algorithms. Based on this platform, we propose STG-MAPPO, a Semantic Task Graph-enhanced variant of Multi-
Agent Proximal Policy Optimization. STG-MAPPO builds semantic policy inputs from tracking diagnostics, task phases, observation confidence, link availability, neighbor tracking quality, and local role advantage. A compact semantic task graph links communication-constrained network states to decentralized actor decisions, and a velocity-level action abstraction maps high-level cooperative decisions to executable six-degree-offreedom AUV control inputs.The code is available at https://github.com/dasjsaj/MARL-AUV.
Read the full abstract ▾
All authors Shengchao Zhu, Guangjie Han, Chuan Lin, Yu He
Robotics
Terrain Consistent Reference-Guided RL for Humanoid Navigation Autonomy
William D. Compton, Zachary Olkin and Aaron D. Ames
We present a method for training reference-guided, perceptive reinforcement learning locomotion policies for
humanoid robots in which reference trajectories are modulated in training to be consistent with terrain geometry. Aiming to deploy our method with standard
navigation autonomy infrastructure, we synthesize SE(2)-controllable reference trajectories inside the RL training loop, projecting desired footsteps onto valid footholds and adjusting swing-foot and center-of-mass trajectories to match the terrain. The resulting policy exposes a clean SE(2) velocity interface compatible with standard
navigation planners. In simulation, environmentally-conditioned references significantly improve reference tracking performance compared to environment agnostic references. On hardware, we integrate the policy with an MPC + control barrier function planner and demonstrate long-horizon (>70m) closed-loop autonomous
navigation on the Unitree G1 through outdoor environments containing rough terrain and consecutive flights of stairs, with all sensing and computation onboard.
Read the full abstract ▾
Robotics
Combinatorial Optimization of Robotic Hand Kinematic Structures Using a Potential-Dexterity-Based QUBO Formulation
HyoJae Kang, Yeong Jae Park, Jeongdo Ahn et al. (+1 more)
Robotic hand kinematic design involves combinatorial selection among multiple finger structures whose performance depends on both individual kinematic characteristics and inter-finger interactions. This study presents a potential-dexterity-based quadratic unconstrained binary optimization (QUBO) formulation for combinatorial optimization of robotic hand kinematic structures. Potential dexterity-related metrics were evaluated through kinematic analysis and encoded into a 27-variable QUBO problem incorporating individual finger performance, thumb-to-finger workspace-overlap interactions, one-hot selection constraints, and pairwise structural compatibility. Exhaustive evaluation of the design space established a reference optimum and enabled direct validation of the formulated objective. Discretization sensitivity analysis showed that the selected design remained unchanged across the tested joint-space sampling intervals and at a finer workspace voxel resolution, while sensitivity was observed under coarse voxelization. Simulated annealing consistently recovered the reference optimum at higher sampling counts, and direct execution on D-Wave
quantum annealing hardware also recovered the reference optimum under all tested sampling conditions. These results demonstrate that the proposed formulation provides a common QUBO representation for constrained robotic hand kinematic structure selection and enables evaluation using exhaustive, classical annealing, and
quantum annealing approaches.
Read the full abstract ▾
All authors HyoJae Kang, Yeong Jae Park, Jeongdo Ahn, Dongil Park
Robotics · Cross-listed from cs.LG
parallelcbf: A composable safety-filter and auditability framework for tensor-parallel reinforcement learning
Yijun Lu, Zilei Yang and Yuyin Ma
While Isaac Lab provides massive parallel
UAV simulation, OmniSafe and safe-control-gym provide constrained-RL
benchmarks, and CBFKit provides control-barrier-function synthesis tooling, no existing framework unifies these capabilities for end-to-end safety-constrained training. ParallelCBF is the first framework to unify (i)~tensor-parallel
UAV environments, (ii)~hard-gate CBF safety filters, (iii)~sharded BC-to-RL pipelines, and (iv)~first-class operational auditability – pre-registration, watchdog registries, failure forensics, and
dataset audits as composable APIs rather than user-implemented scripts. We release ParallelCBF v0.1.0 under Apache~2.0 with a four-layer composable API, a CPU PyTorch reference implementation of a dual-barrier (squared / linear-predictive) CBF, property-based safety invariance tests across vectorized batch sizes that complete in 1.67~s for the full 39-test suite, and a 31{,}415-episode behavior-cloning collection campaign whose curriculum mix, per-bucket yields, and
dataset SHA-256 are auditable through the framework's own ops primitives. We report a representative end-to-end pipeline execution in which the framework's auditability layer halted a downstream training stage that did not meet pre-registered convergence criteria, preventing silent propagation of a degraded checkpoint – an architectural property we argue is necessary, not merely useful, for reproducible empirical robotics research. The framework is installable via pip install parallelcbf; source and release artifacts are available at https://github.com/xiaoyang-123-
cell/ParallelCBF.
Read the full abstract ▾
Robotics
LAPS: Improving Incremental LiDAR Mapping using Active Pooling and Sampling for Neural Distance Fields
Dongjae Lee, Wooseong Yang, Yifu Tao et al. (+2 more)
Neural distance fields offer a compact and continuous representation of 3D geometry, making them attractive for incremental
LiDAR mapping. However, their online optimization is vulnerable to catastrophic forgetting, where new observations can degrade previously reconstructed geometry. Replay-based training is commonly used to address this issue, but existing methods typically rely on passive replay buffers and uniform sampling, which can waste
memory on redundant observations and under-train poorly constrained regions. We propose LAPS, a replay management framework for incremental neural mapping that improves both replay retention and replay allocation during online updates. LAPS combines reliability-based active pooling to retain reliable historical samples under limited
memory with uncertainty-guided active sampling to focus optimization on under-constrained regions. Experiments on synthetic and real-world
benchmarks show that LAPS consistently improves reconstruction completeness while maintaining competitive
geometric accuracy. On Oxford Spires, it improves recall by 4.66 pp and F1-score by 3.79 pp over PIN-SLAM on the Blenheim Palace 05 sequence. We release our open source implementation at: https://github.com/dongjae0107/LAPS.
Read the full abstract ▾
All authors Dongjae Lee, Wooseong Yang, Yifu Tao, Maurice Fallon, Ayoung Kim
Robotics
FLASH: Efficient Visuomotor Policy via Sparse Sampling
Jiaqi Bai, Jindou Jia, Yuxuan Hu et al. (+5 more)
Generative models such as
diffusion and flow matching have become dominant paradigms for visuomotor policy learning, yet their reliance on iterative denoising incurs high inference latency incompatible with real-time robotic control. We present Fast Legendre-
polynomial Action policy via Sparse History-anchored flow (FLASH Policy), which replaces discrete action-chunk generation with continuous Legendre
polynomial trajectory representation. Specifically, by fitting expert demonstrations under sparse temporal sampling, FLASH enables a single inference to cover a significantly extended action horizon. To further accelerate generation, FLASH initiates the flow matching process from history
polynomial coefficients rather than uninformative Gaussian noise, shortening the transport distance and enabling accurate single-step inference. Moreover, analytic
polynomial differentiation directly provides desired velocity feed-forward signals to the torque controller without numerical approximation. Extensive experiments on five simulated and two real-world manipulation tasks demonstrate that FLASH achieves state-of-the-art success rates ($\ge 92\%$ across all tasks), a per-episode inference time of $31.40\,ms$ (up to $175\times$ faster than
diffusion policies and $18\times$ faster than prior flow matching policies), up to $4\times$ faster training convergence than ACT, and $5\times$ to $7\times$ reduction in controller tracking error compared to discrete-action baselines.
Read the full abstract ▾
All authors Jiaqi Bai, Jindou Jia, Yuxuan Hu, Gen Li, Xiangyu Chen, Tuo An, Kuangji Zuo, Jianfei Yang