Robotics
Rationale Behind Human-Led Autonomous Truck Platooning
Yukun Lu, Chenzhao Li, Xintong Jiang et al. (+1 more)
Autonomous trucking has progressed rapidly in recent years, transitioning from early demonstrations to OEM-integrated commercial deployments. However, fully driverless freight operations across heterogeneous climates, infrastructure conditions, and regulatory environments remain technically and socially challenging. This paper presents a systematic rationale for human-led autonomous truck platooning as a pragmatic intermediate pathway. First, we analyze 53 major truck accidents across North America (2021-2026) and show that human-related factors remain the dominant contributors to severe crashes, highlighting both the need for advanced assistance/automated driving systems and the complexity of real-world driving environments. Second, we review recent industry developments and identify persistent limitations in long-tail edge cases, winter operations, remote-region logistics, and large-scale safety validation. Based on these findings, we argue that a human-in-the-loop (HiL) platooning architecture offers layered redundancy, adaptive judgment in uncertain conditions, and a scalable validation framework. Furthermore, the dual-use capability of follower vehicles enables an evolutionary transition from coordinated platooning to independent autonomous operation. Rather than representing a compromise, human-led platooning provides a technically grounded and societally aligned bridge toward large-scale autonomous freight deployment.
Read the full abstract ▾
All authors Yukun Lu, Chenzhao Li, Xintong Jiang, Qiaoxuan Zhang
Robotics
Wei Xue, Mingcheng Li, Xuecheng Wu et al. (+3 more)
Vision-and-
Language Navigation (VLN) requires
agents to accurately perceive complex visual environments and reason over
navigation instructions and histories. However, existing methods passively process redundant visual inputs and treat all historical contexts indiscriminately, resulting in inefficient perception and unfocused
reasoning. To address these challenges, we propose ProFocus, a training-free progressive framework that unifies \underline{Pro}active Perception and \underline{Focus}ed
Reasoning through collaboration between large
language models (LLMs) and
vision-
language models (VLMs). For proactive perception, ProFocus transforms panoramic observations into structured ego-centric semantic maps, enabling the orchestration
agent to identify missing visual information needed for reliable decision-making, and to generate targeted visual queries with corresponding focus regions that guide the perception
agent to acquire the required observations. For focused
reasoning, we propose Branch-Diverse Monte Carlo Tree Search (BD-MCTS) to identify top-$k$ high-value waypoints from extensive historical candidates. The decision
agent focuses
reasoning on the historical contexts associated with these waypoints, rather than considering all historical waypoints equally. Extensive experiments validate the effectiveness of ProFocus, achieving state-of-the-art performance among zero-shot methods on R2R and REVERIE
benchmarks.
Read the full abstract ▾
All authors Wei Xue, Mingcheng Li, Xuecheng Wu, Jingqun Tang, Dingkang Yang, Lihua Zhang
Robotics · Cross-listed from cs.PF
Characterizing VLA Models: Identifying the Action Generation Bottleneck for Edge AI Architectures
Manoj Vishwanathan, Suvinay Subramanian and Anand Raghunathan
Vision-
Language-Action (VLA) models are an emerging class of workloads critical for robotics and
embodied AI at the edge. As these models scale, they demonstrate significant capability gains, yet they must be deployed locally to meet the strict latency requirements of real-time applications. This paper characterizes VLA performance on two generations of edge hardware, viz. the Nvidia Jetson Orin and Thor platforms. Using MolmoAct-7B, a state-of-the-art VLA model, we identify a primary execution bottleneck: up to 75% of end-to-end latency is consumed by the
memory-bound action-generation phase. Through analytical modeling and simulations, we project the hardware requirements for scaling to 100B parameter models. We also explore the impact of high-bandwidth
memory technologies and processing-in-
memory (PIM) as promising future pathways in edge systems for
embodied AI.
Read the full abstract ▾
Robotics
Hybrid TD3: Overestimation Bias Analysis and Stable Policy Optimization for Hybrid Action Space
Thanh-Tuan Tran, Thanh Nguyen Canh, Nak Young Chong et al. (+1 more)
Reinforcement learning in discrete-continuous hybrid action spaces presents fundamental challenges for robotic manipulation, where high-level task decisions and low-level joint-space execution must be jointly optimized. Existing approaches either discretize continuous components or relax discrete choices into continuous approximations, which suffer from scalability limitations and training instability in high-dimensional action spaces and under domain randomization. In this paper, we propose Hybrid TD3, an extension of Twin Delayed Deep Deterministic Policy Gradient (TD3) that natively handles parameterized hybrid action spaces in a principled manner. We conduct a rigorous theoretical analysis of overestimation bias in hybrid action settings, deriving formal bounds under twin-critic architectures and establishing a complete bias ordering across five algorithmic variants under synchronized Gaussian error assumptions. Building on this analysis, we introduce a weighted clipped Q-learning target that marginalizes over the discrete action distribution, achieving equivalent bias reduction to standard clipped minimization while improving policy smoothness. Experimental results demonstrate that Hybrid TD3 achieves superior training stability and competitive performance against state-of-the-art hybrid action baselines.
Read the full abstract ▾
All authors Thanh-Tuan Tran, Thanh Nguyen Canh, Nak Young Chong, Xiem HoangVan
Robotics
Spherical Latent Motion Prior for Physics-Based Simulated Humanoid Control
Jing Tan, Weisheng Xu, Xiangrui Jiang et al. (+11 more)
Learning motion priors for physics-based
humanoid control is an active research topic. Existing approaches mainly include variational autoencoders (VAE) and
adversarial motion priors (AMP). VAE introduces information loss, and random latent sampling may sometimes produce invalid behaviors. AMP suffers from mode collapse and struggles to capture diverse motion skills. We present the Spherical Latent Motion Prior (SLMP), a two-stage method for learning motion priors. In the first stage, we train a high-quality motion tracking controller. In the second stage, we distill the tracking controller into a spherical latent space. A combination of
distillation, a discriminator, and a discriminator-guided local semantic consistency constraint shapes a structured latent action space, allowing stable random sampling without information loss. To evaluate SLMP, we collect a two-hour human combat motion capture
dataset and show that SLMP preserves fine motion detail without information loss, and random sampling yields semantically valid and stable behaviors. When applied to a two-
agent physics-based combat task, SLMP produces human-like and physically plausible combat behaviors only using simple rule-based rewards. Furthermore, SLMP generalizes across different
humanoid robot morphologies, demonstrating its transferability beyond a single simulated avatar.
Read the full abstract ▾
All authors Jing Tan, Weisheng Xu, Xiangrui Jiang, Jiaxi Zhang, Kun Yang, Kai Wu, Jiaqi Xiong, Shiting Chen, Yangfan Li, Yixiao Feng, Yuetong Fang, Yujia Zou, Yiqun Song, Renjing Xu
Robotics · Cross-listed from cs.LG
Integrating LTL Constraints into PPO for Safe Reinforcement Learning
Maifang Zhang, Hang Yu, Qian Zuo et al. (+3 more)
This paper proposes Proximal Policy Optimization with Linear Temporal Logic Constraints (PPO-LTL), a framework that integrates safety constraints written in LTL into PPO for safe reinforcement learning. LTL constraints offer rigorous representations of complex safety requirements, such as regulations that broadly exist in robotics, enabling systematic monitoring of safety requirements. Violations against LTL constraints are monitored by limit-deterministic Büchi automata, and then translated by a logic-to-cost mechanism into penalty signals. The signals are further employed for guiding the policy optimization via the Lagrangian scheme. Extensive experiments on the Zones and CARLA environments show that our PPO-LTL can consistently reduce safety violations, while maintaining competitive performance, against the state-of-the-art methods. The code is at https://github.com/EVIEHub/PPO-LTL.
Read the full abstract ▾
All authors Maifang Zhang, Hang Yu, Qian Zuo, Cheng Wang, Vaishak Belle, Fengxiang He
Robotics · Cross-listed from cs.AI
Information-Theoretic Framework for Self-Adapting Model Predictive Controllers
Wael Hafez and Amir Nazeri
Model Predictive Control (MPC) is a vital technique for autonomous systems, like Unmanned Aerial Vehicles (
UAVs), enabling optimized motion planning. However, traditional MPC struggles to adapt to real-time changes such as dynamic obstacles and shifting system dynamics, lacking inherent mechanisms for self-monitoring and adaptive optimization. Here, we introduce
Entanglement Learning (EL), an information-theoretic framework that enhances MPC adaptability through an Information Digital Twin (IDT). The IDT monitors and quantifies, in bits, the information flow between MPC inputs, control actions, and
UAV behavior. By introducing new information-theoretic metrics we call
entanglement metrics, it tracks variations in these dependencies. These metrics measure the mutual information between the optimizer's input, its control actions, and the resulting
UAV dynamics, enabling a deeper understanding of their interrelationships. This allows the IDT to detect performance deviations and generate real-time adaptive signals to recalibrate MPC parameters, preserving stability. Unlike traditional MPC, which relies on error-based feedback, this dual-feedback approach leverages information flow for proactive adaptation to evolving conditions. Scalable and leveraging existing infrastructure, this framework improves MPC reliability and robustness across diverse scenarios, extending beyond
UAV control to any MPC implementation requiring adaptive performance.
Read the full abstract ▾
Robotics
Certifiable Factor Graph Optimization
Zhexin Xu, Nikolas R. Sanderson, Hanna Jiamei Zhang et al. (+1 more)
We show that the factor graph and certifiable estimation paradigms, which have thus far been treated as essentially independent in the literature, can be naturally synthesized into a unified framework for certifiable factor graph optimization that combines the ease of use of the former with the strong performance guarantees of the latter. The key insight enabling our synthesis is that the core mathematical constructions used to develop certifiable
estimators (Shor's relaxation and Burer-Monteiro factorization) inherit a factor graph structure from the original problem: applying these transformations to a QCQP-representable estimation task with an associated factor graph model yields a lifted problem with identical factor graph connectivity whose constituent variables and factors are simple one-to-one algebraic transformations (lifts) of those appearing in the original QCQP's factor graph. This correspondence enables the Riemannian Staircase methodology for certifiable estimation to be easily instantiated and deployed using the same mature, highly-performant factor graph libraries and workflows already ubiquitously employed throughout robotics and computer
vision. Experimental evaluation on a variety of pose graph optimization, landmark SLAM, and range-aided SLAM
benchmarks demonstrates that our certifiable factor graph optimization methodology enables the implementation of certifiable
estimators that are functionally equivalent to current state-of-the-art hand-designed, problem-specific methods, while dramatically reducing the required implementation effort from the order of months to hours.
Read the full abstract ▾
All authors Zhexin Xu, Nikolas R. Sanderson, Hanna Jiamei Zhang, David M. Rosen
Robotics
RMBench: Memory-Dependent Robotic Manipulation Benchmark with Insights into Policy Design
Tianxing Chen, Yuran Wang, Mingleyang Li et al. (+16 more)
Robotic manipulation policies have made rapid progress in recent years, yet most existing approaches give limited consideration to
memory capabilities. Consequently, they struggle to solve tasks that require
reasoning over historical observations and maintaining task-relevant information over time, which are common requirements in real-world manipulation scenarios. Although several
memory-aware policies have been proposed, systematic evaluation of
memory-dependent manipulation remains underexplored, and the relationship between architectural design choices and
memory performance is still not well understood. To address this gap, we introduce RMBench, a simulation
benchmark comprising 9 manipulation tasks that span multiple levels of
memory complexity, enabling systematic evaluation of policy
memory capabilities. We further propose Mem-0, a modular manipulation policy with explicit
memory components designed to support controlled ablation studies. Through extensive simulation and real-world experiments, we identify
memory-related limitations in existing policies and provide empirical insights into how architectural design choices influence
memory performance. The website is available at https://rmbench.github.io/.
Read the full abstract ▾
All authors Tianxing Chen, Yuran Wang, Mingleyang Li, Yan Qin, Hao Shi, Zixuan Li, Yifan Hu, Yingsheng Zhang, Kaixuan Wang, Yue Chen, Hongcheng Wang, Junjie Wang, Tianhang Yang, Renjing Xu, Ruihai Wu, Yao Mu, Yaodong Yang, Hao Dong, Ping Luo
Robotics · Cross-listed from cs.CV
Monocular 3D Object Position Estimation with VLMs for Human-Robot Interaction
Ari Wahl, Dorian Gawlinski, David Przewozny et al. (+3 more)
Pre-trained general-purpose
Vision-
Language Models (VLM) hold the potential to enhance intuitive human-machine interactions due to their rich world knowledge and 2D object detection capabilities. However, VLMs for 3D coordinates detection tasks are rare. In this work, we investigate interactive abilities of VLMs by returning 3D object positions given a
monocular RGB image from a wrist-mounted camera, natural
language input, and
robot states. We collected and curated a heterogeneous
dataset of more than 100,000 images and finetuned a VLM using QLoRA with a custom regression head. By implementing conditional routing, our model maintains its ability to process general visual queries while adding specialized 3D position estimation capabilities. Our results demonstrate robust predictive performance with a median MAE of 13 mm on the test set and a five-fold improvement over a simpler baseline without finetuning. In about 25% of the cases, predictions are within a range considered acceptable for the
robot to interact with objects.
Read the full abstract ▾
All authors Ari Wahl, Dorian Gawlinski, David Przewozny, Paul Chojecki, Felix Bießmann, Sebastian Bosse
Robotics
Agent-Based Simulation of Trust Development in Human-Robot Teams: An Empirically-Validated Framework
Ravi Kalluri
This paper presents an empirically grounded
agent-based model capturing trust dynamics, workload distribution, and collaborative performance in human-
robot teams. The model, implemented in NetLogo 6.4.0, simulates teams of 2–10
agents performing tasks of varying complexity. We validate against Hancock et al.'s (2021) meta-analysis, achieving interval validity for 4 of 8 trust antecedent categories and strong ordinal validity (Spearman h̊o}=0.833\rho = 0.833 h̊o}=0.833). Sensitivity analysis using OFAT and full factorial designs (n=50n = 50 n=50 replications per condition) reveals
robot reliability exhibits the strongest effect on trust ({\eta}2=0.35\eta^2 = 0.35 {\eta}2=0.35) and dominates task success ({\eta}2=0.93\eta^2 = 0.93 {\eta}2=0.93) and productivity ({\eta}2=0.89\eta^2 = 0.89 {\eta}2=0.89), consistent with meta-analytic findings. Trust asymmetry ratios ranged from 0.07 to 0.55 – below the meta-analytic
benchmark of 1.50 – revealing that per-event asymmetry does not guarantee cumulative asymmetry when trust repair mechanisms remain active. Scenario analysis uncovered trust-performance decoupling: the Trust Recovery scenario achieved the highest productivity (4.29) despite the lowest trust (38.2), while the Unreliable
Robot scenario produced the highest trust (73.2) despite the lowest task success (33.4%), establishing calibration error as a critical diagnostic distinct from trust magnitude. Factorial ANOVA confirmed significant main effects for reliability, transparency, communication, and collaboration (p<.001p < .001 p<.001), explaining 45.4% of trust variance. The open-source implementation provides an evidence-based tool for identifying overtrust and undertrust conditions prior to deployment.
Read the full abstract ▾
Robotics
riMESA: Consensus ADMM for Real-World Collaborative SLAM
Daniel McGann and Michael Kaess
Collaborative Simultaneous Localization and Mapping (C-SLAM) is a fundamental capability for multi-
robot teams as it enables downstream tasks like planning and
navigation. However, existing C-SLAM back-end
algorithms that are required to solve this problem struggle to address the practical realities of real-world deployments (i.e. communication limitations, outlier measurements, and online operation). In this paper we propose Robust Incremental
Manifold Edge-based Separable ADMM (riMESA) – a robust, incremental, and distributed C-SLAM back-end that is resilient to outliers, reliable in the face of limited communication, and can compute accurate state estimates for a multi-
robot team in real-time. Through the development of riMESA, we, more broadly, make an argument for the use of Consensus Alternating Direction Method of Multipliers as a theoretical foundation for distributed optimization tasks in robotics like C-SLAM due to its flexibility, accuracy, and fast convergence. We conclude this work with an in-depth evaluation of riMESA on a variety of C-SLAM problem scenarios and communication network conditions using both synthetic and real-world C-SLAM data. These experiments demonstrate that riMESA is able to generalize across conditions, produce accurate state estimates, operate in real-time, and outperform the accuracy of prior works by a factor >7x on real-world
datasets.
Read the full abstract ▾
Robotics
Path Integral Particle Filtering for Hybrid Systems via Saltation Matrices
Karthik Shaji, Sreeranj Jayadevan, Bo Yuan et al. (+2 more)
State estimation for hybrid systems that undergo intermittent contact with their environments, such as extraplanetary
robots and satellites undergoing docking operations, is difficult due to the discrete uncertainty propagation during contact. To handle this propagation, this paper presents an optimal-control-based particle filtering method that leverages saltation
matrices to map out uncertainty propagation during contact events. By exploiting a path integral filtering framework that exploits the duality between smoothing and optimal control, the resulting state estimation
algorithm is robust to outlier effects, flexible to non-Gaussian noise distributions, and handles challenging contact dynamics in hybrid systems. To evaluate the validity and consistency of the proposed approach, this paper tests it against strong baselines on the
stochastic dynamics generated by a bouncing ball and spring loaded inverted pendulum.
Read the full abstract ▾
All authors Karthik Shaji, Sreeranj Jayadevan, Bo Yuan, Hongzhe Yu, Yongxin Chen
Robotics
RAG-RUSS: A Retrieval-Augmented Robotic Ultrasound for Autonomous Carotid Examination
Dianye Huang, Ziping Cong, Nassir Navab et al. (+1 more)
Robotic ultrasound (US) has recently attracted increasing
attention as a means to overcome the limitations of conventional US examinations, such as the strong operator dependence. However, the decision-making process of existing methods is often either rule-based or relies on end-to-end learning models that operate as black boxes. This has been seen as a main limit for clinical acceptance and raises safety concerns for widespread adoption in routine practice. To tackle this challenge, we introduce the RAG-RUSS, an interpretable framework capable of performing a full carotid examination in accordance with the clinical workflow while explicitly explaining both the current stage and the next planned action. Furthermore, given the scarcity of medical data, we incorporate
retrieval-augmented generation to enhance generalization and reduce dependence on large-scale training
datasets. The method was trained on data acquired from 28 volunteers, while an additional four volumetric scans recorded from previously unseen volunteers were reserved for testing. The results demonstrate that the method can explain the current scanning stage and autonomously plan probe motions to complete the carotid examination, encompassing both transverse and longitudinal planes.
Read the full abstract ▾
All authors Dianye Huang, Ziping Cong, Nassir Navab, Zhongliang Jiang
Robotics
D-REX: Differentiable Real-to-Sim-to-Real Engine for Learning Dexterous Grasping
Haozhe Lou, Mingtong Zhang, Haoran Geng et al. (+9 more)
Simulation provides a cost-effective and flexible platform for data generation and policy learning to develop robotic systems. However, bridging the gap between simulation and real-world dynamics remains a significant challenge, especially in physical parameter identification. In this work, we introduce a real-to-sim-to-real engine that leverages the Gaussian Splat representations to build a differentiable engine, enabling object mass identification from real-world visual observations and
robot control signals, while enabling grasping policy learning simultaneously. Through optimizing the mass of the manipulated object, our method automatically builds high-fidelity and physically plausible digital twins. Additionally, we propose a novel approach to train force-aware grasping policies from limited data by transferring feasible human demonstrations into simulated
robot demonstrations. Through comprehensive experiments, we demonstrate that our engine achieves accurate and robust performance in mass identification across various object geometries and mass values. Those optimized mass values facilitate force-aware policy learning, achieving superior and high performance in object grasping, effectively reducing the sim-to-real gap.
Read the full abstract ▾
All authors Haozhe Lou, Mingtong Zhang, Haoran Geng, Hanyang Zhou, Sicheng He, Zhiyuan Gao, Siheng Zhao, Jiageng Mao, Pieter Abbeel, Jitendra Malik, Daniel Seita, Yue Wang
Robotics
A Deployable Bio-inspired Compliant Leg Design for Enhanced Leaping in Quadruped Robots
Yiyang Chen, Yuxin Liu, Jinzheng Zhou et al. (+4 more)
Quadruped
robots are becoming increasingly essential for various applications, including industrial inspection and catastrophe search and rescue. These scenarios require
robots to possess enhanced agility and obstacle-
navigation skills. Nonetheless, the performance of current platforms is often constrained by insufficient peak motor power, limiting their ability to perform explosive jumps. To address this challenge, this paper proposes a bio-inspired method that emulates the energy-storage mechanism found in froghopper legs. We designed a Deployable Compliant Leg (DCL) utilizing a specialized 3D-printed elastic material, Polyether block amide (PEBA), featuring a lightweight internal
lattice structure. This structure functions analogously to biological tendons, storing elastic energy during the
robot's squatting phase and rapidly releasing it to augment motor output during the leap. The proposed mechanical design significantly enhances the
robot's vertical jumping capability. Through finite element analysis (FEA) and experimental validation, we demonstrate a relative performance improvement of 17.1% in vertical jumping height.
Read the full abstract ▾
All authors Yiyang Chen, Yuxin Liu, Jinzheng Zhou, Fanxin Wang, Qinglei Bu, Jie Sun, Yikun Cheng
Robotics
Pro-HOI: Perceptive Root-guided Humanoid-Object Interaction
Yuhang Lin, Jiyuan Shi, Dewei Wang et al. (+4 more)
Executing reliable
Humanoid-Object Interaction (HOI) tasks for
humanoid robots is hindered by the lack of generalized control interfaces and robust closed-loop perception mechanisms. In this work, we introduce Perceptive Root-guided
Humanoid-Object Interaction, Pro-HOI, a generalizable framework for robust
humanoid loco-manipulation. First, we collect box-carrying motions that are suitable for real-world deployment and optimize penetration artifacts through a Signed Distance Field loss. Second, we propose a novel training framework that conditions the policy on a desired root-trajectory while utilizing reference motion exclusively as a reward. This design not only eliminates the need for intricate reward tuning but also establishes root trajectory as a universal interface for high-level planners, enabling simultaneous
navigation and loco-manipulation. Furthermore, to ensure operational reliability, we incorporate a persistent object estimation module. By fusing real-time detection with Digital Twin, this module allows the
robot to autonomously detect slippage and trigger re-grasping maneuvers. Empirical validation on a Unitree G1
robot demonstrates that Pro-HOI significantly outperforms baselines in generalization and robustness, achieving reliable long-horizon execution in complex real-world scenarios.
Read the full abstract ▾
All authors Yuhang Lin, Jiyuan Shi, Dewei Wang, Jipeng Kong, Yong Liu, Chenjia Bai, Xuelong Li
Robotics
Fast Confidence-Aware Human Prediction via Hardware-accelerated Bayesian Inference for Safe Robot Navigation
Michael Lu, Minh Bui, Xubo Lyu et al. (+1 more)
As
robots increasingly integrate into everyday environments, ensuring their safe
navigation around humans becomes imperative. Efficient and safe motion planning requires
robots to account for human behavior, particularly in constrained spaces such as grocery stores or care homes, where interactions with multiple individuals are common. Prior research has employed
Bayesian frameworks to model human rationality based on navigational intent, enabling the prediction of probabilistic trajectories for planning purposes. In this work, we present a simple yet novel approach for confidence-aware prediction that treats future predictions as particles. This framework is highly parallelized and accelerated on an graphics processing unit (GPU). As a result, this enables longer-term predictions at a frequency of 125 Hz and can be easily extended for multi-human predictions. Compared to existing methods, our implementation supports finer prediction time steps, yielding more granular trajectory forecasts. This enhanced resolution allows motion planners to respond effectively to subtle changes in human behavior. We validate our approach through real-world experiments, demonstrating a
robot safely navigating among multiple humans with diverse navigational goals. Our results highlight the methods potential for robust and efficient human-
robot coexistence in dynamic environments.
Read the full abstract ▾
All authors Michael Lu, Minh Bui, Xubo Lyu, Mo Chen
Robotics
From Dialogue to Execution: Mixture-of-Agents Assisted Interactive Planning for Behavior Tree-Based Long-Horizon Robot Execution
Kanata Suzuki, Kazuki Hori, Haruka Miyoshi et al. (+2 more)
Interactive task planning with large
language models (LLMs) lets
robots generate high-level action plans from natural
language, but over long horizons it asks many questions, and tabular plan representations become hard to manage. We propose a framework that integrates Mixture-of-
Agents (MoA)-based proxy answering into interactive planning and generates Behavior Trees (BTs) for structured long-term execution. We formulate the MoA as an abstention-based delegation cascade: each expert
agent answers only the questions entailed by its own prerequisite description, forwards the rest unchanged, and the human user acts as the terminal fallback. The question set is thus partitioned disjointly, so no answer fusion or arbitration is required while every question is still resolved. The BT represents task logic hierarchically and enables retry and dynamic switching among
robot policies. Experiments on a cocktail-making task show that the method removes approximately 27% of the human responses while keeping the generated BTs within the baseline generator's own variance. Real-
robot experiments on a smoothie-making task further demonstrate successful long-horizon execution with adaptive policy switching and recovery from action failures. We further analyze the failure modes of the framework and show that its applicability boundary is set by the reliability of the weakest action node rather than by the planner. These results indicate that MoA-assisted interactive planning improves dialogue efficiency while preserving execution quality in real-world robotic tasks.
Read the full abstract ▾
All authors Kanata Suzuki, Kazuki Hori, Haruka Miyoshi, Shuhei Kurita, Tetsuya Ogata
Robotics
Compact Task-Aligned Imitation Learning for Laboratory Automation
Kanata Suzuki, Hanon Nakamura, Kana Miyamoto et al. (+1 more)
Robotic laboratory automation has traditionally relied on carefully engineered motion pipelines and task-specific hardware interfaces, resulting in high design cost and limited flexibility. While recent imitation learning techniques can generate general
robot behaviors, their large model sizes often require high-performance computational resources, limiting applicability in practical laboratory environments. In this study, we propose a compact imitation learning framework for laboratory automation using small foundation models. The proposed method, TVF-DiT, aligns a self-supervised
vision foundation model with a
vision-
language model through a compact adapter, and integrates them with a
Diffusion Transformer-based action expert. The entire model consists of fewer than 500M parameters, enabling inference on low-VRAM GPUs. Experiments on three real-world laboratory tasks - test tube cleaning, test tube arrangement, and powder transfer - demonstrate an average success rate of 86.6%, significantly outperforming alternative lightweight baselines. Furthermore, detailed task
prompts improve
vision-
language alignment and task performance. These results indicate that small foundation models, when properly aligned and integrated with
diffusion-based policy learning, can effectively support practical laboratory automation with limited computational resources.
Read the full abstract ▾
All authors Kanata Suzuki, Hanon Nakamura, Kana Miyamoto, Tetsuya Ogata
Robotics · Cross-listed from cs.CV
SMR-Net:Robot Snap Detection Based on Multi-Scale Features and Self-Attention Network
Kuanxu Hou
In
robot automated assembly, snap assembly precision and efficiency directly determine overall production quality. As a core prerequisite, snap detection and localization critically affect subsequent assembly success. Traditional visual methods suffer from poor robustness and large localization errors when handling complex scenarios (e.g., transparent or low-contrast snaps), failing to meet high-precision assembly demands. To address this, this paper designs a dedicated sensor and proposes SMR-Net, an self-
attention-based multi-scale object detection
algorithm, to synergistically enhance detection and localization performance. SMR-Net adopts an
attention-enhanced multi-scale feature fusion architecture: raw sensor data is encoded via an
attention-embedded feature extractor to strengthen key snap features and suppress noise; three multi-scale feature maps are processed in parallel with standard and dilated convolution for dimension unification while preserving resolution; an adaptive reweighting network dynamically assigns weights to fused features, generating fine representations integrating details and global semantics. Experimental results on Type A and Type B snap
datasets show SMR-Net outperforms traditional Faster R-CNN significantly: Intersection over Union (IoU) improves by 6.52% and 5.8%, and mean Average Precision (mAP) increases by 2.8% and 1.5% respectively. This fully demonstrates the method's superiority in complex snap detection and localization tasks.
Read the full abstract ▾
Robotics
An Open-Source Modular Benchmark for Diffusion-Based Motion Planning in Closed-Loop Autonomous Driving
Yun Li, Simon Thompson, Yidu Zhang et al. (+2 more)
Diffusion-based motion planners have achieved state-of-the-art results on
benchmarks such as nuPlan, yet their evaluation within closed-loop production autonomous driving stacks remains largely unexplored. Existing evaluations abstract away ROS 2 communication latency and real-time scheduling constraints, while monolithic ONNX deployment freezes all solver parameters at export time. We present an open-source modular
benchmark that addresses both gaps: using ONNX GraphSurgeon, we decompose a monolithic 18,398 node
diffusion planner into three independently executable modules and reimplement the DPM-Solver++ denoising loop in native C++. Integrated as a ROS 2 node within Autoware, the open-source AD stack deployed on real vehicles worldwide, the system enables runtime-configurable solver parameters without model recompilation and per-step observability of the denoising process, breaking the black box of monolithic deployment. Unlike evaluations in standalone simulators such as CARLA, our
benchmark operates within a production-grade stack and is validated through AWSIM closed-loop simulation. Through systematic comparison of DPM-Solver++ (first- and second-order) and DDIM across six step-count configurations (N in {3, 5, 7, 10, 15, 20}), we show that encoder caching yields a 3.2x latency reduction, and that second-order solving reduces FDE by 41% at N=3 compared to first-order. The complete codebase will be released as open-source, providing a direct path from simulation
benchmarks to real-vehicle deployment.
Read the full abstract ▾
All authors Yun Li, Simon Thompson, Yidu Zhang, Ehsan Javanmardi, Manabu Tsukada
Robotics
MiniUGV$_2$: A Compact UAV-Deployable Tracked Ground Vehicle with Manipulation Capabilities
Durgakant Pushp, Swapnil Kalhapure, Shaekh Mohammad Shithil et al. (+1 more)
Exploring and inspecting Hidden Spaces, defined as environments whose entrances are accessible only to aerial
robots but remain unexplored due to
geometric constraints, limited flight time, and communication loss, remains a major challenge. We present miniUGV$_2$, a compact
UAV-deployable tracked ground vehicle that extends
UAV capabilities into confined environments. The system introduces dual articulated arms, integrated
LiDAR and depth sensing, and modular electronics for enhanced autonomy. A novel tether module with an electro-permanent
magnetic head enables safe deployment,
retrieval, and optional detachment, thereby overcoming prior
entanglement issues. Experiments demonstrate robust terrain
navigation, self-righting, and manipulation of objects up to 3.5 kg, validating miniUGV$_2$ as a versatile platform for hybrid aerial-ground robotics.
Read the full abstract ▾
All authors Durgakant Pushp, Swapnil Kalhapure, Shaekh Mohammad Shithil, Lantao Liu
Robotics
HierKick: Hierarchical Reinforcement Learning for Vision-Guided Soccer Robot Control
Yizhi Chen, Zheng Zhang, Zhanxiang Cao et al. (+7 more)
Controlling soccer
robots involves multi-time-scale decision-making, which requires balancing long-term tactical planning and short-term motion execution. Traditional end-to-end reinforcement learning (RL) methods face challenges in complex dynamic environments. This paper proposes HierKick, a
vision-guided soccer
robot control framework based on dual-frequency hierarchical RL. The framework adopts a hierarchical control architecture featuring a 5 Hz high-level policy that integrates YOLOv8 for real-time detection and selects tasks via a coach model, and a pre-trained 50 Hz low-level controller for precise joint control. Through this architecture, the framework achieves the four steps of approaching, aligning, dribbling, and kicking. Experimental results show that the success rates of this framework are 95.2% in IsaacGym, 89.8% in Mujoco, and 80% in the real world. HierKick provides an effective hierarchical paradigm for
robot control in complex environments, extendable to multi-time-scale tasks, with its modular design and skill reuse offering a new path for intelligent
robot control.
Read the full abstract ▾
All authors Yizhi Chen, Zheng Zhang, Zhanxiang Cao, Yihe Chen, Shengcheng Fu, Liyun Yan, Yang Zhang, Jiali Liu, Haoyang Li, Yue Gao