Robotics
CompliantVLA-adaptor: VLM-Guided Variable Impedance Action for Safe Contact-Rich Manipulation
Heng Zhang, Wei-Hsing Huang, Qiyi Tong et al. (+7 more)
We propose a CompliantVLA-adaptor that augments the state-of-the-art
Vision-
Language-Action (VLA) models with
vision-
language model (VLM)-informed context-aware variable impedance control (VIC) to improve the safety and effectiveness of contact-rich robotic manipulation tasks. Existing VLA systems (e.g., RDT, Pi0.5, OpenVLA-oft) typically output position, but lack force-aware adaptation, leading to unsafe or failed interactions in physical tasks involving contact, compliance, or uncertainty. In the proposed CompliantVLA-adaptor, a VLM interprets task context from images and natural
language to adapt the stiffness and damping parameters of a VIC controller. These parameters are further regulated using real-time force/torque feedback to ensure interaction forces remain within safe thresholds. We demonstrate that our method outperforms the VLA baselines on a suite of complex contact-rich tasks, both in simulation and the real world, with improved success rates and reduced force violations. This work presents a promising path towards a safe foundation model for physical contact-rich manipulation. We release our code,
prompts, and force-torque-impedance-scenario context
datasets at https://sites.google.com/view/compliantvla.
Read the full abstract ▾
All authors Heng Zhang, Wei-Hsing Huang, Qiyi Tong, Gokhan Solak, Puze Liu, Kaidi Zhang, Sheng Liu, Jan Peters, Yu She, Arash Ajoudani
Robotics
An LLM-Agnostic, MAVLink-Based Drone Command and Control Interface and Agentic Harness Using the Model Context Protocol
Javier Noé Ramos Silva and Peter J. Burke
Artificial intelligence integrated with drone command and control (physical AI) offers a route to autonomy through large
language models (LLMs), yet a unified LLM-to-drone interface has been missing. We present an LLM-agnostic command-and-control interface and agentic harness joining the Model Context Protocol (MCP) to MAVLink, the near-universal drone command-and-telemetry standard: "DroneServer" gives any MCP-capable LLM command, telemetry, mission, and safety functions over ArduPilot and PX4. Its 98 tools cover 223 of 238 client-side methods of MavSDK, MAVLink's library; 61 of the 98 were exercised in software-in-the-loop. We treat the LLM as an untrusted commander: zero public ports (externally verified); every command validated server-side (tiers, confirmation handshakes, independent geofence, audit log), exercised by an
adversarial safety suite. Server-side mission state resolves the "fire-and-forget" mismatch: a scripted client's 37.8-minute mission survived a 4-minute disconnection. Over 1,000 simulated flights ran a ten-mission LLM-
UAV control
benchmark; eight of eleven models complete 90.0-100% of its six flying missions, and no aircraft left the permitted zone in 110 geofence-violation trials. On real hardware, the unmodified server commanded three quadcopters, GPS-guided and GPS-denied; five providers' models flew an unseen mission, ten of ten. The tests and
datasets include: repeated trials, standardized evaluation tasks, success/failure statistics, latency analysis, uncertainty treatment, comparative benchmarking across models/platforms,
prompt sets, mission complexity, tool-call traces, safety-intervention logs, and safety-performance assessment. The architecture is a scaffold for a higher-level drone-LLM operating system where plain-
language commands realize complex missions, and a path to
agent swarms commanding drone swarms under one security umbrella.
Read the full abstract ▾
Robotics
Neural Minimum-Distance Estimation for Collision-Aware Operation of Multi-Arm Laparoscopy Surgical Robots Through Learning-from-Simulation
Sarvin Ghiasi, Majid Roshanfar, Jake Barralet et al. (+2 more)
This study presents an integrated framework for enhancing the safety and operational efficiency of robotic arms in laparoscopic surgery by addressing minimum distance estimation between multi-arm manipulators and the associated collision-aware warning. By combining analytical modeling, real time simulation, and machine learning, the framework offers a robust solution for ensuring safe robotic operations. An analytical model was developed to estimate the minimum distances between robotic arms based on their joint configurations, offering theoretical calculations that serve as both a validation tool and a
benchmark. To complement this, a 3D simulation environment was created to model two 7 DOF Kinova robotic arms (Kinova inc., Boisbriand, QC, Canada), generating a diverse
dataset of configurations for distance estimation and collision warning. Using these insights, a deep residual neural network model was trained with joint configurations as inputs. On the held out validation set, the model achieves R2 = 0.940, RMSE = 42.0 mm, MAE = 28.7 mm, and a near zero mean bias, demonstrating strong predictive accuracy and consistent generalization across the workspace. The framework is intended as an early collision warning layer, where a warning is triggered when the predicted inter-arm distance falls below a 0.2 m threshold, which corresponds to a surface to surface clearance of approximately 50 mm given the Kinova Gen3 (Kinova inc., Boisbriand, QC, Canada) cross sectional radius. This work demonstrates the effectiveness of combining analytical modeling with machine learning to enhance the precision and reliability of multi-arm robotic systems.
Read the full abstract ▾
All authors Sarvin Ghiasi, Majid Roshanfar, Jake Barralet, Liane S. Feldman, Amir Hooshiar
Robotics
Learning a Unified Latent Space for Cross-Embodiment Robot Control
Yashuai Yan and Dongheui Lee
We present a scalable framework for cross-embodiment
humanoid robot control by learning a shared latent representation that unifies motion across humans and diverse
humanoid platforms, including single-arm, dual-arm, and legged
humanoid robots. Our method proceeds in two stages: first, we construct a decoupled latent space that captures localized motion patterns across different body parts using contrastive learning, enabling accurate and flexible motion retargeting even across
robots with diverse morphologies. To enhance
alignment between embodiments, we introduce tailored similarity metrics that combine joint rotation and end-effector positioning for critical segments, such as arms. Then, we train a goal-conditioned control policy directly within this latent space using only human data. Leveraging a conditional variational autoencoder, our policy learns to predict latent space displacements guided by intended goal directions. We show that the trained policy can be directly deployed on multiple
robots without any adaptation. Furthermore, our method supports the efficient addition of new
robots to the latent space by learning only a lightweight,
robot-specific
embedding layer. The learned latent policies can also be directly applied to the new
robots. Experimental results demonstrate that our approach enables robust, scalable, and embodiment-agnostic
robot control across a wide range of
humanoid platforms.
Read the full abstract ▾
Robotics
Preparation and Motion Study of Magnetically Driven Micro Soft Robot Mimicking the Cownose Ray
Jiaqing Chang, Song Gao, Chaowei Dong et al. (+2 more)
In narrow, unstructured
underwater environments such as environmental monitoring and minimally invasive medical procedures, micro soft
robots exhibit unique advantages due to their flexible movement capabilities and small size. At the same time, applying bionic technology to the structural design of micro soft
robots can significantly improve their swimming performance. However, limited by their miniaturization, these
robots are difficult to power internally and usually adopt a wireless power supply method. This study designs and fabricates a magnetically responsive, cownose ray-inspired micro soft
robot based on the swimming principle of the cownose ray. The
robot is made of a certain proportion of NdFeB and PDMS. Then, a three-dimensional Helmholtz coil is used to generate an oscillating harmonic
magnetic field to conduct swimming experiments on the
robot, exploring the influence of
magnetic field parameters on the
robot's swimming performance. The experimental results show that the swimming speed is the fastest at B = 5 mT and f = 11 Hz, reaching 5.25 mm/s, which is about 0.5 body lengths per second. In addition, by adjusting the current direction and frequency of the coil, the
robot can perform different swimming modes such as straight swimming, turning swimming, and directional swimming. By employing a stepwise adjustment method, the impact of response errors on the
robot's trajectory can be effectively reduced. This study demonstrates a method for magnetically driven micro soft
robots, laying a foundation for the application of wireless-driven
robots in
underwater narrow spaces.
Read the full abstract ▾
All authors Jiaqing Chang, Song Gao, Chaowei Dong, zhaobang Li, Yang Liu
Robotics · Cross-listed from cs.CV
Iterative Refinement Improves Compositional Image Generation
Shantanu Jaiswal, Mihir Prabhudesai, Nikash Bhardwaj et al. (+5 more)
Text-to-image (T2I) models have achieved remarkable progress, yet they continue to struggle with complex
prompts that require simultaneously handling multiple objects, relations, and attributes. Existing inference-time strategies, such as parallel sampling with verifiers or simply increasing denoising steps, can improve
prompt alignment but remain inadequate for richly compositional settings where many constraints must be satisfied. Inspired by the success of chain-of-thought
reasoning in large
language models, we propose an iterative test-time strategy in which a T2I model progressively refines its generations across multiple steps, guided by feedback from a
vision-
language model as the critic in the loop. Our approach is simple, requires no external tools or priors, and can be flexibly applied to a wide range of image generators and
vision-
language models. Empirically, we demonstrate consistent gains on image generation across
benchmarks: a 16.9% improvement in all-correct rate on ConceptMix (k=7), a 13.8% improvement on T2I-CompBench (3D-Spatial category) and a 12.5% improvement on Visual Jenga scene decomposition compared to compute-matched parallel sampling. Beyond quantitative gains, iterative refinement produces more faithful generations by decomposing complex
prompts into sequential corrections, with human evaluators preferring our method 58.7% of the time over 41.3% for the parallel baseline. Together, these findings highlight iterative self-correction as a broadly applicable principle for compositional image generation. Results and visualizations are available at https://iterative-img-gen.github.io/
Read the full abstract ▾
All authors Shantanu Jaiswal, Mihir Prabhudesai, Nikash Bhardwaj, Zheyang Qin, Amir Zadeh, Chuan Li, Katerina Fragkiadaki, Deepak Pathak
Robotics · Cross-listed from cs.CV
Rethinking Video Generation Model for the Embodied World
Yufan Deng, Zilin Pan, Hongyu Zhang et al. (+6 more)
Video generation models have significantly advanced
embodied intelligence, unlocking new possibilities for generating diverse
robot data that capture perception,
reasoning, and action in the physical world. However, synthesizing high-quality
videos that accurately reflect real-world robotic interactions remains challenging, and the lack of a standardized
benchmark limits fair comparisons and progress. To address this gap, we introduce a comprehensive robotics
benchmark, RBench, designed to evaluate
robot-oriented
video generation across five task domains and four distinct embodiments. It assesses both task-level correctness and visual fidelity through reproducible sub-metrics, including structural consistency, physical plausibility, and action completeness. Evaluation of 25 representative models highlights significant deficiencies in generating physically realistic
robot behaviors. Furthermore, the
benchmark achieves a Spearman correlation coefficient of 0.96 with human evaluations, validating its effectiveness. While RBench provides the necessary lens to identify these deficiencies, achieving physical realism requires moving beyond evaluation to address the critical shortage of high-quality training data. Driven by these insights, we introduce a refined four-stage data pipeline, resulting in RoVid-X, the largest open-source robotic
dataset for
video generation with 4 million annotated
video clips, covering thousands of tasks and enriched with comprehensive physical property annotations. Collectively, this synergistic ecosystem of evaluation and data establishes a robust foundation for rigorous assessment and scalable training of
video models, accelerating the
evolution of
embodied AI toward general intelligence.
Read the full abstract ▾
All authors Yufan Deng, Zilin Pan, Hongyu Zhang, Xiaojie Li, Ruoqing Hu, Yufei Ding, Yiming Zou, Yan Zeng, Daquan Zhou
Robotics · Cross-listed from cs.CV
FlowSSC: Universal Generative Monocular Semantic Scene Completion via One-Step Latent Diffusion
Zichen Xi, Hao-Xiang Chen, Nan Xue et al. (+5 more)
Semantic Scene Completion (SSC) from
monocular RGB images is a fundamental yet challenging task due to the inherent ambiguity of inferring occluded 3D geometry from a single view. While feed-forward methods have made progress, they often struggle to generate plausible details in occluded regions and preserve the fundamental spatial relationships of objects. Such accurate generative
reasoning capability for the entire 3D space is critical in real-world applications. In this paper, we present FlowSSC, the first generative framework applied directly to
monocular semantic scene completion. FlowSSC treats the SSC task as a conditional generation problem and can seamlessly integrate with existing feed-forward SSC methods to significantly boost their performance. To achieve real-time inference without compromising quality, we introduce Shortcut Flow-matching that operates in a compact triplane latent space. Unlike standard
diffusion models that require hundreds of steps, our method utilizes a shortcut mechanism to achieve high-fidelity generation in a single step, enabling practical deployment in autonomous systems. Extensive experiments on SemanticKITTI demonstrate that FlowSSC achieves state-of-the-art performance, significantly outperforming existing baselines.
Read the full abstract ▾
All authors Zichen Xi, Hao-Xiang Chen, Nan Xue, Hongyu Yan, Qi-Yuan Feng, Levent Burak Kara, Joaquim Jorge, Qun-Ce Xu
Robotics
MonoRace: Winning Champion-Level Drone Racing with Robust Monocular AI
Stavrow A. Bahnam, Robin Ferede, Till M. Blaha et al. (+6 more)
Autonomous drone racing represents a major frontier in robotics research. It requires an Artificial Intelligence (AI) that can run on board light-weight flying
robots under tight resource and time constraints, while pushing the physical system to its limits. The state of the art in this area consists of a system with a stereo camera and an inertial measurement unit (IMU) that beat human drone racing champions in a controlled indoor environment. Here, we present MonoRace: an onboard drone racing approach that uses a
monocular, rolling-shutter camera and IMU that generalizes to a competition environment without any external motion tracking system. The approach features robust state estimation that combines neural-network-based gate
segmentation with a drone model. Moreover, it includes an offline optimization procedure that leverages the known geometry of gates to refine any state estimation parameter. This offline optimization is based purely on onboard flight data and is important for fine-tuning the vital external camera calibration parameters. Furthermore, the guidance and control are performed by a neural network that foregoes inner loop controllers by directly sending motor commands. This small network runs on the flight controller at 500Hz. The proposed approach won the 2025 Abu Dhabi Autonomous Drone Racing Competition (A2RL), outperforming all competing AI teams and three human world champion pilots in a direct knockout tournament. It set a new milestone in autonomous drone racing research, reaching speeds up to 100 km/h on the competition track and successfully coping with problems such as camera interference and IMU saturation.
Read the full abstract ▾
All authors Stavrow A. Bahnam, Robin Ferede, Till M. Blaha, Anton E. Lang, Erin Lucassen, Quentin Missinne, Aderik E.C. Verraest, Christophe De Wagter, Guido C.H.E. de Croon
Robotics · Cross-listed from cs.AI
LangForce: Bayesian Decomposition of Vision Language Action Models via Latent Action Queries
Shijie Lian, Bin Yu, Xiaopeng Lin et al. (+6 more)
Vision-
Language-Action (VLA) models have shown promise in
robot manipulation but often struggle to generalize to new instructions or complex multi-task scenarios. We identify a critical pathology in current training paradigms where goal-driven data collection creates a
dataset bias. In such
datasets,
language instructions are highly predictable from visual observations alone, causing the conditional mutual information between instructions and actions to vanish, a phenomenon we term Information Collapse. Consequently, models degenerate into
vision-only policies that ignore
language constraints and fail in out-of-distribution (OOD) settings. To address this, we propose LangForce, a novel framework that enforces instruction following via
Bayesian decomposition. By introducing learnable Latent Action Queries, we construct a dual-branch architecture to estimate both a
vision-only prior $p(a \mid v)$ and a
language-conditioned posterior $\pi(a \mid v, \ell)$. We then optimize the policy to maximize the conditional Pointwise Mutual Information (PMI) between actions and instructions. This objective effectively penalizes the
vision shortcut and rewards actions that explicitly explain the
language command. Without requiring new data, LangForce significantly improves generalization. Extensive experiments across on SimplerEnv and RoboCasa demonstrate substantial gains, including an 11.3% improvement on the challenging OOD SimplerEnv
benchmark, validating the ability of our approach to robustly ground
language in action.
Read the full abstract ▾
All authors Shijie Lian, Bin Yu, Xiaopeng Lin, Laurence T. Yang, Zhaolong Shen, Changti Wu, Yuzhuo Miao, Cong Huang, Kai Chen
Robotics
V-CAGE: Context-Aware Generation and Verification for Scalable Long-Horizon Embodied Tasks
Yaru Liu, Ao-bo Wang and Nanyang Ye
Learning long-horizon
embodied behaviors from synthetic data remains challenging because generated scenes are often physically implausible,
language-driven programs frequently "succeed" without satisfying task semantics, and high-level instructions require grounding into executable action sequences. To address these limitations, we introduce V-CAGE, a closed-loop framework for generating robust, semantically aligned manipulation
datasets at scale. First, we propose a context-aware instantiation mechanism that enforces
geometric consistency during scene synthesis. By dynamically maintaining a map of prohibited spatial areas as objects are placed, our system prevents interpenetration and ensures reachable, conflict-free configurations in cluttered environments. Second, to bridge the gap between abstract intent and low-level control, we employ a hierarchical instruction decomposition module. This decomposes high-level goals (e.g., "get ready for work") into compositional action primitives, facilitating coherent long-horizon planning. Crucially, we enforce semantic correctness through a VLM-based verification loop. Acting as a visual critic, the VLM performs rigorous rejection sampling after each subtask, filtering out "silent failures" where code executes but fails to achieve the visual goal. Experiments demonstrate that V-CAGE yields
datasets with superior physical and semantic fidelity, significantly boosting the success rate and generalization of downstream policies compared to non-verified baselines.
Read the full abstract ▾
Robotics
Influence of Operator Expertise on Robot Supervision and Intervention
Yanran Jiang, Pavan Sikka, Leimin Tian et al. (+2 more)
With increasing levels of
robot autonomy,
robots are increasingly being supervised by users with varying levels of robotics expertise. As the diversity of the user population increases, it is important to understand how users with different expertise levels approach the supervision task and how this impacts performance of the human-
robot team. This exploratory study investigates how operators with varying expertise levels perceive information and make intervention decisions when supervising a remote
robot. We conducted a user study (N=27) where participants supervised a
robot autonomously exploring four unknown tunnel environments in a simulator, and provided waypoints to intervene when they believed the
robot had encountered difficulties. By analyzing the interaction data and questionnaire responses, we identify differing patterns in intervention timing and decision-making strategies across novice, intermediate, and expert users.
Read the full abstract ▾
All authors Yanran Jiang, Pavan Sikka, Leimin Tian, Dana Kuliic, Cecile Paris
Robotics
Systematic Evaluation of Hip Exoskeleton Assistance Parameters for Enhancing Gait Stability During Ground Slip Perturbations
Maria T. Tagliaferri and Inseung Kang
Falls are the leading cause of injury related hospitalization and mortality among older adults. Consequently, mitigating age-related declines in gait stability and reducing fall risk during walking is a critical goal for assistive devices. Lower-limb exoskeletons have the potential to support users in maintaining stability during walking. However, most exoskeleton controllers are optimized to reduce the energetic cost of walking rather than to improve stability. While some studies report stability benefits with assistance, the effects of specific parameters, such as assistance magnitude and duration, remain unexplored. To address this gap, we systematically modulated the magnitude and duration of torque provided by a bilateral hip exoskeleton during slip perturbations in eight healthy adults, quantifying stability using whole-body angular momentum (WBAM). WBAM responses were governed by a significant interaction between assistance magnitude and duration, with duration determining whether exoskeleton assistance was stabilizing or destabilizing relative to not wearing the exoskeleton device. Compared to an existing energy-optimized controller, experimentally identified stability-optimal parameters reduced WBAM range by 25.7% on average. Notably, substantial inter-subject variability was observed in the parameter combinations that minimized WBAM during perturbations. We found that optimizing exoskeleton assistance for energetic outcomes alone is insufficient for improving reactive stability during gait perturbations. Stability-focused exoskeleton control should prioritize temporal assistance parameters and include user-specific personalization. This study represents an important step toward personalized, stability-focused exoskeleton control, with direct implications for improving stability and reducing fall risk in older adults.
Read the full abstract ▾
Robotics
CADGrasp: Learning Contact and Collision Aware General Dexterous Grasping in Cluttered Scenes
Jiyao Zhang, Zhiyuan Ma, Tianhao Wu et al. (+2 more)
Dexterous grasping in cluttered environments presents substantial challenges due to the high degrees of freedom of dexterous hands, occlusion, and potential collisions arising from diverse object geometries and complex layouts. To address these challenges, we propose CADGrasp, a two-stage
algorithm for general dexterous grasping using single-view point cloud inputs. In the first stage, we predict sparse IBS, a scene-decoupled, contact- and collision-aware representation, as the optimization target. Sparse IBS compactly encodes the
geometric and contact relationships between the dexterous hand and the scene, enabling stable and collision-free dexterous grasp pose optimization. To enhance the prediction of this high-dimensional representation, we introduce an occupancy-
diffusion model with voxel-level conditional guidance and force closure score filtering. In the second stage, we develop several energy functions and ranking strategies for optimization based on sparse IBS to generate high-quality dexterous grasp poses. Extensive experiments in both simulated and real-world settings validate the effectiveness of our approach, demonstrating its capability to mitigate collisions while maintaining a high grasp success rate across diverse objects and complex scenes.
Read the full abstract ▾
All authors Jiyao Zhang, Zhiyuan Ma, Tianhao Wu, Zeyuan Chen, Hao Dong
Robotics
ExPrIS: Knowledge-Level Expectations as Priors for Object Interpretation from Sensor Data
Marian Renz, Martin Günther, Felix Igelbrink et al. (+2 more)
While deep learning has significantly advanced robotic object recognition, purely data-driven approaches often lack semantic consistency and fail to leverage valuable, pre-existing knowledge about the environment. This report presents the ExPrIS project, which addresses this challenge by investigating how knowledge-level expectations can serve as to improve object interpretation from sensor data. Our approach is based on the incremental construction of a 3D Semantic Scene Graph (3DSSG). We integrate expectations from two sources: contextual priors from past observations and semantic knowledge from external graphs like ConceptNet. These are embedded into a heterogeneous Graph Neural Network (GNN) to create an expectation-biased inference process. This method moves beyond static, frame-by-frame analysis to enhance the robustness and consistency of scene understanding over time. The report details this architecture, its evaluation, and outlines its planned integration on a mobile robotic platform.
Read the full abstract ▾
All authors Marian Renz, Martin Günther, Felix Igelbrink, Oscar Lima, Martin Atzmueller
Robotics
Risk Estimation for Automated Driving
Leon Tolksdorf, Arturo Tejada, Jonas Bauernfeind et al. (+2 more)
Safety is a central requirement for automated vehicles. As such, the assessment of risk in automated driving is key in supporting both motion planning technologies and safety evaluation. In automated driving, risk is characterized by two aspects. The first aspect is the uncertainty on the state estimates of other road participants by an automated vehicle. The second aspect is the severity of a collision event with said traffic participants. Here, the uncertainty aspect typically causes the risk to be non-zero for near-collision events. This makes risk particularly useful for automated vehicle motion planning. Namely, constraining or minimizing risk naturally navigates the automated vehicle around traffic participants while keeping a safety distance based on the level of uncertainty and the potential severity of the impending collision. Existing approaches to calculate the risk either resort to empirical modeling or severe approximations, and, hence, lack generalizability and accuracy. In this paper, we combine recent advances in collision probability estimation with the concept of collision severity to develop a general method for accurate risk estimation. The proposed method allows us to assign individual severity functions for different collision constellations, such as, e.g., frontal or side collisions. Furthermore, we show that the proposed approach is computationally efficient, which is beneficial, e.g., in real-time motion planning applications. The programming code for an exemplary implementation of Gaussian uncertainties is also provided.
Read the full abstract ▾
All authors Leon Tolksdorf, Arturo Tejada, Jonas Bauernfeind, Christian Birkner, Nathan van de Wouw
Robotics
DWPP: Dynamic Window Pure Pursuit Considering Velocity and Acceleration Constraints
Fumiya Ohnishi and Masaki Takahashi
Pure pursuit and its variants are widely used for mobile
robot path tracking owing to their simplicity and computational efficiency. However, many conventional approaches do not explicitly account for velocity and acceleration constraints, resulting in discrepancies between commanded and actual velocities that result in overshoot and degraded tracking performance. To address this problem, this paper proposes dynamic window pure pursuit (DWPP), which fundamentally reformulates the command velocity computation process to explicitly incorporate velocity and acceleration constraints. Specifically, DWPP formulates command velocity computation in the velocity space (the $v$-$\omega$ plane) and selects the command velocity as the point within the dynamic window that is closest to the line $\omega = \kappa v$. Experimental results demonstrate that DWPP avoids constraint-violating commands and achieves superior path-tracking accuracy compared with conventional pure pursuit methods. The proposed method has been integrated into the official Nav2 repository and is publicly available (https://github.com/ros-
navigation/
navigation2).
Read the full abstract ▾
Robotics
Graph-Based Adaptive Planning for Coordinated Dual-Arm Robotic Disassembly of Electronic Devices (eGRAP)
Adip Ranjan Das and Maria Koskinopoulou
E-waste is growing rapidly while recycling rates remain low. We propose an electronic-device Graph-based Adaptive Planning (eGRAP) that integrates
vision, dynamic planning, and dual-arm execution for autonomous disassembly. A camera-equipped arm identifies parts and estimates their poses, and a directed graph encodes which parts must be removed first. A scheduler uses topological ordering of this graph to select valid next steps and assign them to two
robot arms, allowing independent tasks to run in parallel. One arm carries a screwdriver (with an eye-in-hand depth camera) and the other holds or handles components. We demonstrate eGRAP on 3.5in hard drives: as parts are unscrewed and removed, the system updates its graph and plan online. Experiments show consistent full disassembly of each HDD, with high success rates and efficient cycle times, illustrating the method's ability to adaptively coordinate dual-arm tasks in real time.
Read the full abstract ▾
Robotics
HumanDiffusion: A Vision-Based Diffusion Trajectory Planner with Human-Conditioned Goals for Search and Rescue UAV
Faryal Batool, Iana Zhura, Valerii Serpiva et al. (+4 more)
Reliable human–
robot collaboration in emergency scenarios requires autonomous systems that can detect humans, infer
navigation goals, and operate safely in dynamic environments. This paper presents HumanDiffusion, a lightweight image-conditioned
diffusion planner that generates human-aware
navigation trajectories directly from RGB imagery. The system combines YOLO-11 based human detection with
diffusion-driven trajectory generation, enabling a quadrotor to approach a target person and deliver medical assistance without relying on prior maps or computationally intensive planning pipelines. Trajectories are predicted in pixel space, ensuring smooth motion and a consistent safety margin around humans. We evaluate HumanDiffusion in simulation and real-world indoor mock-disaster scenarios. On a 300-sample test set, the model achieves a mean squared error of 0.02 in pixel-space trajectory reconstruction. Real-world experiments demonstrate an overall mission success rate of 80% across accident-response and search-and-locate tasks with partial occlusions. These results indicate that human-conditioned
diffusion planning offers a practical and robust solution for human-aware
UAV navigation in time-critical assistance settings.
Read the full abstract ▾
All authors Faryal Batool, Iana Zhura, Valerii Serpiva, Roohan Ahmed Khan, Ivan Valuev, Issatay Tokmurziyev, Dzmitry Tsetserukou
Robotics
TIDAL: Temporally Interleaved Diffusion and Action Loop for High-Frequency VLA Control
Yuteng Sun, Haoran Wang, Ruofei Bai et al. (+4 more)
Large-scale
Vision-
Language-Action (VLA) models offer semantic generalization but suffer from high inference latency because they adopt a low-frequency batch-and-execute paradigm. This frequency mismatch creates an execution blind spot, causing failures in dynamic environments where targets move during the open-loop execution window. We propose
TIDAL (Temporally Interleaved
Diffusion and Action Loop), a hierarchical framework that decouples semantic
reasoning from high-frequency actuation.
TIDAL operates as a backbone-agnostic scheduler for
diffusion-based VLAs, using a dual-frequency architecture to redistribute the computational budget. Specifically, a low-frequency macro-intent loop caches semantic
embeddings, while a high-frequency micro-control loop interleaves single-step flow integration with execution, conditioning on the latest state fused with motion cues. To handle the resulting latency shift, we introduce a temporally misaligned training strategy where the policy learns stalenessaware compensation, conditioning on stale semantic intent alongside real-time proprioception.
TIDAL is architectural, making it orthogonal to system-level optimizations. Experiments show an average 2.5x performance gain over open-loop baselines in dynamic interception tasks. Despite a slight decrease in static success rates, our approach yields a 4x increase in feedback frequency and extends the effective horizon of semantic
embeddings beyond the native action chunk size. Under nonpaused physics and on a real
robot,
TIDAL remains robust to unmodeled dynamics, while standard open-loop baselines fail due to latency-induced error accumulation.
Read the full abstract ▾
All authors Yuteng Sun, Haoran Wang, Ruofei Bai, Zhengguo Li, Jun Li, Meng Yee Michael Chuah, Wei Yun Yau
Robotics
Vision-Language Models on the Edge for Real-Time Robotic Perception
Sarat Ahmad, Maryam Hafeez and Syed Ali Raza Zaidi
Vision-
Language Models (VLMs) enable
multimodal reasoning for robotic perception and interaction, but their deployment in real-world systems remains constrained by latency, limited onboard resources, and
privacy risks of cloud offloading. Edge intelligence within 6G, particularly Open RAN and Multi-access Edge Computing (MEC), offers a pathway to address these challenges by bringing computation closer to the data source. This work investigates the deployment of VLMs on ORAN/MEC infrastructure using the Unitree G1
humanoid robot as an
embodied testbed. We design a WebRTC-based pipeline that streams
multimodal data to an edge node and evaluate LLaMA-3.2-11B-
Vision-Instruct deployed at the edge versus in the cloud under real-time conditions. Our results show that edge deployment preserves near-cloud accuracy while reducing end-to-end latency by 5%. We further evaluate Qwen2-VL-2B-Instruct, a compact model optimized for resource-constrained environments, which achieves sub-second responsiveness, cutting latency by more than half but at the cost of accuracy.
Read the full abstract ▾
Robotics
HumanoidVLM: Vision-Language-Guided Impedance Control for Contact-Rich Humanoid Manipulation
Yara Mahmoud, Yasheerah Yaqoot, Miguel Altamirano Cabrera et al. (+1 more)
Humanoid robots must adapt their contact behavior to diverse objects and tasks, yet most controllers rely on fixed, hand-tuned impedance gains and gripper settings. This paper introduces HumanoidVLM, a
vision-
language driven
retrieval framework that enables the Unitree G1
humanoid to select task-appropriate Cartesian impedance parameters and gripper configurations directly from an
egocentric RGB image. The system couples a
vision-
language model for semantic task inference with a FAISS-based
Retrieval-Augmented Generation (RAG) module that retrieves experimentally validated stiffness-damping pairs and object-specific grasp angles from two custom databases, and executes them through a task-space impedance controller for compliant manipulation. We evaluate HumanoidVLM on 14 visual scenarios and achieve a
retrieval accuracy of 93%. Real-world experiments show stable interaction dynamics, with z-axis tracking errors typically within 1-3.5 cm and virtual forces consistent with task-dependent impedance settings. These results demonstrate the feasibility of linking semantic perception with
retrieval-based control as an interpretable path toward adaptive
humanoid manipulation.
Read the full abstract ▾
All authors Yara Mahmoud, Yasheerah Yaqoot, Miguel Altamirano Cabrera, Dzmitry Tsetserukou
Robotics
On-the-fly hand-eye calibration for the da Vinci surgical robot
Zejian Cui and Ferdinando Rodriguez y Baena
In
Robot-Assisted Minimally Invasive Surgery (RMIS), accurate tool localization is crucial to ensure patient safety and successful task execution. However, this remains challenging for cable-driven
robots, such as the da Vinci
robot, because erroneous encoder readings lead to pose estimation errors. In this study, we propose a calibration framework to produce accurate tool localization results through computing the hand-eye transformation
matrix on-the-fly. The framework consists of two interrelated
algorithms: the feature association block and the hand-eye calibration block, which provide robust correspondences for key points detected on
monocular images without pre-training, and offer the versatility to accommodate various
surgical scenarios by adopting an array of filter approaches, respectively. To validate its efficacy, we test the framework extensively on publicly available
video datasets that feature multiple
surgical instruments conducting tasks in both in vitro and ex vivo scenarios, under varying illumination conditions and with different levels of key point measurement accuracy. The results show a significant reduction in tool localization errors under the proposed calibration framework, with accuracies comparable to other state-of-the-art methods while being more time-efficient.
Read the full abstract ▾