Robotics
FactorDrive: Adaptive Multi-Step Reasoning Driven by Planning-Critical Factors for End-to-End Autonomous Driving
Guolei Huang, Tengfei She, Yuxuan Lu et al. (+3 more)
Vision-
language models (VLMs) have advanced scene understanding and enabled explicit
reasoning in end-to-end autonomous driving. However, existing methods insufficiently integrate spatial-physical evidence into planning
reasoning, while
reasoning adaptation remains coarse-grained and falls short of scene-specific planning demands. Furthermore,
reasoning-path optimization for higher planning quality remains largely unexplored in autonomous-driving post-training. To address these limitations, we propose FactorDrive, an end-to-end autonomous driving framework for adaptive multi-step
reasoning driven by planning-critical factors (PCFs). We first perform large-scale driving-domain instruction tuning to establish foundational driving knowledge. Building on this foundation, we construct PCF-CoT, a chain-of-thought (CoT)
dataset that grounds planning
reasoning in trajectory-relevant spatial-physical evidence and organizes
reasoning around scene-specific PCFs, enabling the composition and depth of
reasoning paths to adapt to different planning demands. We further introduce Quality Search-Guided Group Relative Policy Optimization (QS-GRPO), which guides Monte Carlo Tree Search (MCTS) with trajectory-level planning rewards to discover
reasoning paths with higher planning quality and uses the resulting responses to optimize the policy through GRPO, thereby improving trajectory planning performance. Extensive experiments on both open-loop (nuScenes) and closed-loop-oriented (NAVSIM)
benchmarks demonstrate that FactorDrive achieves state-of-the-art planning performance.
Read the full abstract ▾
All authors Guolei Huang, Tengfei She, Yuxuan Lu, Yao Huang, Yuqi Ye, Yongjun Shen
Robotics · Cross-listed from cs.CV
GenTrack3: Hybrid Stochastic-Deterministic Online Multi-Object Tracking with Cluster-Aware Association
Toan Van Nguyen, Rasmus G. K. Christiansen, Dirk Kraft et al. (+1 more)
Multi-object tracking (MOT) involves maintaining consistent target identities as objects dynamically enter and leave a scene. Deterministic approaches, such as tracking-by-detection with data association, produce reproducible results and are computationally efficient, but they rely heavily on motion models and are sensitive to noisy detections that can lead to association errors. In contrast,
stochastic methods explicitly model uncertainty and can better handle complex non-linear dynamics, albeit at the cost of increased computational complexity and variability arising from random sampling. This paper presents an online MOT framework that integrates deterministic and
stochastic principles to achieve robust tracking under uncertainty. Furthermore, a novel track-to-detection matching approach is introduced to enhance scalability with increasing target numbers while supporting group tracking. The tracking inference mechanism employs a tracklet that includes identifiers, states, velocities, track penalties and track ages of targets, supporting a systematic tracking pipeline. Each target is associated with a
stochastic particle set to compute the matching cost to detections. Reference implementations of the proposed approach and baseline trackers can be found on GitHub: https://github.com/SDU-VelKoTek/GenTrack3.
Read the full abstract ▾
All authors Toan Van Nguyen, Rasmus G. K. Christiansen, Dirk Kraft, Leon Bodenhagen
Robotics
TDMA Based Communications Control Co-Design for Cooperative Carrying: Delay Calibration and Sampling-Rate Optimization
Zahra Seifaei, Maximilian Luebke, Torsten Reissland et al. (+2 more)
Multi
robot teams performing cooperative transportation face a fundamental challenge: maintaining stable control while keeping communications efficient. This paper investigates how adaptive sampling time adjustment informed by measured network delay and strategic leader rotation can distribute wireless load fairly across the team. We use physics based simulation in MuJoCo with realistic wireless modeling, including time division multiple access, medium access control, jitter, queueing, and packet loss, to evaluate three control approaches: fixed sampling with static leadership, dynamic sampling with static leadership, and dynamic sampling with rotating leadership. Our results reveal an important trade off: dynamic sampling effectively reduces communications overhead without compromising control performance, while rotating the leader role meaningfully improves how fairly airtime is distributed all with negligible impact on the team carrying ability. to the best of our knowledge, being among the first to jointly examine dynamic sampling, rotating leadership, and wireless protocol interactions in physicsrealistic multi
robot cooperation, this work provides practical guidance for deploying coordinated robotic teams in real world settings where communications resources are limited.
Read the full abstract ▾
All authors Zahra Seifaei, Maximilian Luebke, Torsten Reissland, Danial Dehghani, Norman Franchi
Robotics
Model-Based Systems Engineering Framework for SysML-Driven Design of Autonomous UAVs
Deekshitha Angadi, Naveena Budda, Vikas Agarwal et al. (+3 more)
Autonomous Unmanned Aerial Vehicles (
UAVs) are complex cyber-physical systems that require the coordinated integration of flight control, navigation, perception, communication, power management, and mission-level decision-making under safety, timing, and reliability constraints. However, many autonomous
UAV development workflows still rely on document-centric requirements, separated architectural descriptions, and software implementation artifacts, which can lead to ambiguity, interface inconsistencies, and weak traceability during early design. This paper presents a Model-Based Systems Engineering (MBSE) design framework for the SysML-driven development of autonomous
UAVs. The proposed framework uses the Systems Modeling
Language (SysML) as a formal design backbone to structure
UAV development across four connected layers: stakeholder requirements, functional decomposition, logical architecture, and physical/software allocation. SysML requirement diagrams, activity diagrams, block definition diagrams, internal block diagrams, state machine diagrams, and parametric diagrams are used to capture the functional, structural, behavioral, interface, and performance aspects of the
UAV system. The logical architecture is then systematically mapped to a
Robot Operating System 2 (ROS 2) software architecture by relating SysML blocks to ROS 2 nodes, flow ports and connectors to topics, request-response interactions to services, and goal-oriented behaviors to actions. The framework is illustrated at the design level using representative autonomous
UAV mission scenarios, including autonomous take-off, waypoint navigation, hover stabilization, obstacle avoidance, return-to-home, and emergency handling. The resulting model supports requirement allocation, interface definition, subsystem responsibility assignment, and verification planning before simulation or physical deployment.
Read the full abstract ▾
All authors Deekshitha Angadi, Naveena Budda, Vikas Agarwal, Mohamed Samshad, Bharath Kumar Suryadevara, Narsimlu Kemsaram
Robotics
Robotic Fabric Alignment System for Sewing Using Global Local Weighted ICP
Wenbo Dong, Dipankar Bhattacharya Member, Kai Tang et al. (+5 more)
Accurate fabric
alignment is a critical step that must be performed before sewing. This paper presents a novel automated fabric
alignment system. The system estimates the poses of top and bottom fabric panels, lying flat and wrinkle-free in arbitrary positions, using a new Global Local Weighted Iterative Closest Point (GLW-ICP) method. The system then manipulates the top panel to achieve precise
alignment at both edges and sewing lines. Unlike conventional approaches, GLW-ICP robustly aligns both global edges and local sewing lines by globally aligning fabric edge points and locally aligning sewing line points to their corresponding CAD model points, while removing unmatched points in occluded regions. Real-world experiments with various fabric shapes show that the system consistently achieves millimeter-level
alignment accuracy under both occlusion and non-occlusion conditions, demonstrating its effectiveness and suitability for automated fabric
alignment in practical scenarios.
Read the full abstract ▾
All authors Wenbo Dong, Dipankar Bhattacharya Member, Kai Tang, Akinari Kobayashi, Fuyuki Tokuda, Akira Seino, Norman C. Tien, Kazuhiro Kosuge
Robotics · Cross-listed from cs.CV
A Height-Constrained 2-Point Minimal Solver for Pose Estimation from Active LED Markers with Event Cameras
Runze Yuan, Alexander Kappler, Jun Zhang et al. (+5 more)
In many autonomous applications requiring real-time localization, active marker-based systems are preferred due to their low latency and ease of deployment compared to computationally demanding feature-based methods. Event~\mbox{cameras} offer high temporal resolution and minimal delay and are commonly used with active LED markers for robust real-time localization. Existing methods typically rely on Perspective-n-Point (PnP) solvers for pose estimation. However, structured marker layouts can be challenging to deploy in space-constrained scenarios, while partial self-motion information (e.g.,
gravity direction and altitude) is readily available from onboard sensors. We derive a robust and accurate minimal solver that estimates camera pose from only two LED markers by incorporating known tilt angle and camera height measured by an onboard sensor, such as an IMU or an altimeter. The proposed formulation uniquely determines the camera pose through both a closed-form and a linear least-squares solution. We further analyze degenerate configurations and characterize the conditions under which height information does not contribute to rotation estimation. For evaluation, we developed an event-based active marker system to collect real-world data with ground truth from a motion capture system. Experiments on both synthetic and real data demonstrate improved accuracy over the state-of-the-art P2P solver and competitive performance relative to P3P.
Read the full abstract ▾
All authors Runze Yuan, Alexander Kappler, Jun Zhang, Kuangyi Chen, Fabio Morbidi, Pascal Vasseur, Cédric Demonceaux, Friedrich Fraundorfer
Robotics
HarnessWAM: Bridging Prediction and Deliberation in World Action Models
Zhaopeng Gu, Bingke Zhu, Tianxi Lin et al. (+9 more)
World Action Models~(WAMs) jointly learn environmental dynamics and
robot actions, introducing priors over physical
evolution into embodied control. However, finite-horizon prediction and action generation are insufficient for complex embodied tasks that require global planning, cross-stage state maintenance, execution verification, and failure recovery. We refer to this mismatch as the prediction-deliberation gap. To address this gap, we propose HarnessWAM, an agentic framework for WAMs. HarnessWAM employs a
vision-
language-model-based Task Manager to maintain an evidence-grounded scene belief and organize the global instruction into a structured task graph. Capability-conditioned executable-space projection then compiles the graph into WAM-supported skill sequences that satisfy task dependencies and embodiment-state constraints. During execution, HarnessWAM couples a lightweight progress
estimator with event-triggered Task Manager deliberation. The
estimator supplies frequent execution cues, while the Task Manager uses observations, task state, and interaction history to verify outcomes and decide whether to advance, acquire evidence, revise the plan, or recover. Local recovery preserves acquired scene knowledge so that execution can resume after failure. HarnessWAM achieves state-of-the-art average performance on RoboMemArena, with full-task and subtask success rates of 64.4% and 76.8%, respectively, and reaches 23.7% success on RoboCerebra Ideal. On a physical UR3 arm, full-task success reaches 95% for shape-conditioned tabletop tidying and 90% for
memory-dependent box exploration. These results demonstrate the effectiveness of model-external agentic orchestration in bridging the prediction-deliberation gap for complex embodied tasks.
Read the full abstract ▾
All authors Zhaopeng Gu, Bingke Zhu, Tianxi Lin, Guibo Zhu, Yingying Chen, Kai Wang, Tingyu Yuan, Fan Yang, Chaoyang Zhao, Zhaowen Li, Peng Su, Jinqiao Wang
Robotics
Tether-Inertial Localization for Planetary Drones
Dielof van Loon, Anton Bredenbeck, Lennart Puck et al. (+2 more)
Recent developments in planetary exploration have shown the potential of Unmanned Aerial Vehicles (
UAVs), such as the Ingenuity helicopter that provided valuable mapping data. However, limited payload capabilities constrain the flight times and compute available for localization, which restrict their applicability. By providing a tethered connection, issues such as battery and computational constraints are offloaded to the base rover. At the same time, the cable can be exploited for non-drifting localization. This work presents a novel Tether-Inertial Localization approach that uses tether length and angle measurements to estimate the
UAV position relative to its base. The method combines a computationally efficient analytical catenary model with a Gaussian Process (GP) residual error compensation. This accounts for systematic sensor inaccuracies and model limitations. Experimental validation across circular, triangular, and figure-eight trajectories with tether lengths up to 4.5 m and a total flight time of 37 minutes demonstrates the effectiveness of the proposed approach. Using only tether-based position estimates for feedback, the analytical catenary model achieves an average RMSE of 7.4 cm, which is further reduced to 5.2 cm through GP-based residual compensation, one order of magnitude better than the state-of-the-art. These results establish Tether-Inertial Localization as a practical alternative to
vision- and GNSS-based localization for Tethered Unmanned Aerial Vehicles (TUAVs).
Read the full abstract ▾
All authors Dielof van Loon, Anton Bredenbeck, Lennart Puck, Martin Azkarate, Salua Hamaza
Robotics
Rethink Before You Execute: Adaptive Execution for World Action Models
Feng Ye, Yiming Zhao, Yong Yu et al. (+5 more)
World Action Models (WAMs) jointly predict future actions and the
evolution of the environment. At each inference, a WAM generates a chunk of actions and the
robot executes a fixed prefix before replanning. We argue that this fixed execution horizon is poorly matched to execution dynamics: the chunk reliability varies across task stages, so when to replan depends on the result of accumulated execution, not on the step counts. We propose TempoWAM (Timing Execution by Monitoring Progress Online), a lightweight plug-and-play execution scheme for WAMs. A Recurrent Progress Monitor first estimates task progress from the current observation, task instruction, remaining actions, and execution history; and an Adaptive Execution Protocol then evaluates whether the chunk is advancing the task to decide if replanning is needed. To bridge the training-deployment gap, the protocol is calibrated by a task-dependent calibration factor with online adaptation. Experiments on LIBERO, RoboTwin, and real-world tasks show that TempoWAM consistently improves the efficiency-success trade-off of WAM execution. On real
robots, it reduces WAM inferences by 26.9% on easy tasks while maintaining success, and improves success by 13.3 points on difficult tasks.
Read the full abstract ▾
All authors Feng Ye, Yiming Zhao, Yong Yu, Hongxu Zhou, Yong Pan, Yuan Xue, Peng Jia, Chuanmin Jia
Robotics
Graph-Guided Safe Diffuser: Topological Graph Guidance for Safe Diffusion Planning
Nakgyu Yang, KwangBin Lee and SooJean Han
Many
diffusion-based planners enforce safety through inference-time guidance, but such interleaved trajectory deformations often degrade kinematic feasibility due to
manifold rupture. We propose Graph-Guided Safe Diffuser (G2SD), a hierarchical framework that leverages a high-level topological graph planner to guide a low-level
diffusion model. G2SD enforces safety at a structural level by abstracting the data
manifold into a learned latent graph, on which high-level planning is performed. Continuous trajectories are generated by
diffusion planners, which are conditioned on the graph node representations selected by the high-level planner. Theoretical analyses demonstrate conditions under which
manifold rupture occurs in
diffusion planners, and show that G2SD improves safety by reducing the constraint violation probability as the number of segments increases. Experiments demonstrate that G2SD substantially outperforms baselines, increasing goal-reaching rate without any collision from 40-50% to 98% in Maze2D navigation and also achieving superior task scores in locomotion.
Read the full abstract ▾
Robotics · Cross-listed from eess.SY
Trajectory-Induced Self-Calibration for Hidden-Target Localization Through an Unknown-Pose Range-Bearing Relay
Yash Bagla
This paper studies hidden-target localization from range-bearing packets reported by a relay beacon whose global position and yaw are unknown. The vehicle knows its own trajectory but never directly senses the target; the relay packet contains only local-frame range and bearing to the vehicle and to the hidden target. Unlike bearing-only network localization, relative-frame localization, and target-enclosing control, the target is neither directly observed in the vehicle frame nor treated as a node in a relative-sensing graph. The main result characterizes the minimal motion that removes the resulting calibration ambiguity: one vehicle pose leaves a continuous yaw/translation/target gauge, whereas two distinct vehicle-relative observations from one unknown-pose relay constructively determine relay yaw (modulo 2 pi), relay position, and the anchored target in the noiseless case. A local rank corollary, a shared-target multi-beacon extension, and a trajectory-spread conditioning lemma connect relay self-calibration to finite-window excitation and native range-bearing estimation. In Monte Carlo evaluation the
estimator recovers the hidden target with 5.5 mm RMSE, five times below the 30 mm per-packet range noise and thirteen times more accurate than a naive EKF baseline; it converges to the same accuracy from 2 m target offsets and 2.4 rad yaw errors, and Huber weighting preserves millimeter accuracy under 10% outlier corruption that drops the unprotected
estimator to a 0.10 success rate. Trajectory spread predicts
estimator quality: the two weakly excited trajectories carry condition numbers above 100 with success rates of 0.82 and 0.70, while every well-excited trajectory attains full success.
Read the full abstract ▾
Robotics
VANE: Reliable Test-Time Training for Vision-Language-Action Models via Future Visual Representation Prediction
Hongjin Ji, Guoyang Xia, Luoyang Sun et al. (+2 more)
Test-time training (TTT) offers a lightweight way to adapt
vision–
language–action (VLA) policies from unlabeled deployment streams, but it remains difficult to use reliably in closed-loop manipulation. A shared adaptation space can mix incompatible task corrections, while an online update can alter subsequent actions before its consequences are known. We introduce a reliable TTT framework for VLA policies (VANE). VANE conditions
prompt adaptation on the current
vision–
language context and learns from the future visual consequences of executed actions. Candidate updates are isolated from the live policy, evaluated on subsequent observations, and committed only when supported by future evidence, making adaptation selective and reversible. On SimplerEnv WidowX, VANE improves average success by $3.2$ percentage points over the corresponding TTT baseline. Results on Google
Robot further show that deployment-time gains remain task- and embodiment-dependent. Together, these results demonstrate a constrained, evidence-based approach to adapting VLA policies during interaction.
Read the full abstract ▾
All authors Hongjin Ji, Guoyang Xia, Luoyang Sun, Fangxiang Feng, Lei Ren
Robotics
Skills in Weights, Memory in Code: Hybrid Learning for Memory-Dependent Robot Manipulation
Yunhao Zhao, Zhenyang Ni, Haoyang Chen et al. (+2 more)
Modern
vision-
language-action (VLA) policies have acquired broad manipulation skills, but typically generate each action chunk from the current observation or a short fixed-length history. However, real-world manipulation is often non-Markovian, requiring
robots to retain and reason over task-relevant information from long-horizon interaction histories to determine the next action. To address this challenge, we propose HyMeS, a hybrid learning framework that leverages the
reasoning and
memory-management capabilities of coding
agents to steer a Markovian VLA for
memory-dependent manipulation. Specifically, HyMeS learns low-level motor skills through gradient-based imitation learning, while a coding
agent acquires high-level
memory-management strategies through heuristic learning by iteratively updating an executable heuristic system from rollout feedback. Furthermore, we close the loop between steering and execution through
multimodal stage-completion verification, which updates
memory using proprioceptive signals and multi-frame VLM judgments. Compared with end-to-end
memory-augmented VLAs, HyMeS requires demonstrations only for reusable motor skills rather than for every history-dependent task configuration, enabling data-efficient compositional generalization. On RoboMemArena, HyMeS improves mean cumulative success from 52.5% to 66.2% and mean task success from 41.3% to 60.1% over pi0.5, while outperforming PrediMem by 4.5 points in cumulative success and 14.5 points in task success.
Read the full abstract ▾
All authors Yunhao Zhao, Zhenyang Ni, Haoyang Chen, Ruohan Zhang, Qi Zhu
Robotics
Beyond the Plane: Coupling Planar Vehicle Dynamics with Three-Dimensional Road Geometry
Simon Sagmeister, Phillip Pitschi, Nico Haja et al. (+1 more)
Simulation is crucial for developing and testing autonomous driving systems. In particular, the development of localization and control
algorithms relies on an accurate vehicle dynamics simulation. However, most vehicle dynamics models are two-dimensional while real-world roads are three-dimensional. For example, effects from the three-dimensional road geometry on the Las Vegas Motor Speedway can increase the normal forces on the tires by more than 66% compared to the nominal load at standstill. As a result, even highly detailed planar vehicle dynamics models struggle to accurately reproduce the real vehicle's behavior. While solutions for three-dimensional vehicle dynamics exist, they are rarely adopted, computationally expensive, and complex. To address this issue, we present a novel method to couple planar vehicle dynamics models with real-world three-dimensional road geometry. We transform the planar vehicle state from the vehicle model's two-dimensional plane to its corresponding representation in three-dimensional space. Additionally, we calculate road-geometry-induced forces and moments and apply them to the planar vehicle model. We validate our approach using high-speed data recorded with a full-scale race car on the banked Las Vegas Motor Speedway. Furthermore, on synthetic tracks, we show that our method yields accurate results even in edge cases. Together, our results demonstrate that the gap between planar simulation and real-world three-dimensional roads can be closed without abandoning simpler planar models. To simplify adoption of our method, we provide the implementation as open-source software on github.com/TUMFTM/3d-road-geometry-coupling.
Read the full abstract ▾
All authors Simon Sagmeister, Phillip Pitschi, Nico Haja, Markus Lienkamp
Robotics
JEPA-WAM: Learning Vision-Language-Action Policies with Joint-Embedding World Modeling
Yihan Lin, Jiawei He, Shifeng Bao et al. (+6 more)
Robust
robot control benefits from explicitly modeling state transitions, but
video-generation world action models (WAMs) introduce substantial deployment cost. Existing latent WAMs avoid explicit future generation, but often compress predictive representations or separate predictive modeling from the representations used for action generation. We introduce JEPA-WAM, a latent WAM built in a pretrained V-JEPA space, which couples latent transition prediction with continuous action generation through a shared predictor. JEPA-WAM predicts a spatially structured joint current-future target that captures task-shared visual temporal structure between current and future observations, while preserving dense patch-level correspondence. Through the shared predictor, transition supervision directly shapes the backbone, from which dedicated representations are extracted for action prediction. The same design can also be instantiated in pretrained VLA policies while preserving their original perception and action pathways. On LIBERO-Plus, JEPA-WAM achieves 79.2%, the best result without large-scale
robot-policy pretraining, while its pretrained $\pi_{0.5}$ instantiation reaches 86.3%, achieving the best overall performance. Experiments on RoboTwin 2.0 and real-world bimanual manipulation further demonstrate strong generalization under visual and spatial shifts.
Read the full abstract ▾
All authors Yihan Lin, Jiawei He, Shifeng Bao, Chen Zhao, Yang Li, Xiaobo Wang, Yan Wang, Cheng Chi, Jing Zhang
Robotics
DH-VLM: Dual-Horizon Cooperative Latent Reasoning for Autonomous Driving
Ziyi Song, Chen Xia, Hang Yu et al. (+2 more)
Large-scale
language models for autonomous driving enable enhanced global understanding and long-horizon planning. However, when deployed in isolated vehicles, limited sensing range and occlusions restrict reliable decision-making, and the substantial computational and latency overhead makes on-board deployment impractical. Cooperative driving provides a potential solution by leveraging external
agents for information exchange, but existing methods remain limited in semantic
reasoning capability under practical constraints. To address these challenges, we propose DH-VLM, a dual-horizon cooperative latent
reasoning framework that enables asymmetric semantic cooperation between the infrastructure and ego vehicle. The infrastructure aggregates multi-layer hidden states to form a global-
reasoning horizon latent guidance, which is integrated into the ego model through an Infrastructure-Driven Latent
Evolution mechanism for conditional latent refinement. This enables the ego vehicle to leverage long-range contextual understanding while preserving autonomous decision-making within its local planning horizon. Furthermore, we construct a cooperation-oriented question-answer (QA)
dataset covering fundamental scene understanding and ego-personalized comprehension to support counterfactual and safety-aware
reasoning. Extensive experiments demonstrate that DH-VLM achieves state-of-the-art planning performance, outperforming the previous state of the art by 14.6% in L2 error and 26.9% in collision rate. Compared with query-based end-to-end cooperative driving methods, our approach reduces the communication cost by 57.3% and GPU
memory usage by 25.5%, while maintaining strong robustness against infrastructure guidance errors, providing a practical and robust paradigm for cooperative autonomous driving.
Read the full abstract ▾
All authors Ziyi Song, Chen Xia, Hang Yu, Sheng Zhou, Zhisheng Niu
Robotics
SAFE-CHEM: Uncertainty-Aware Policy Switching for Robust Robotic Chemistry
Laura Jones, Shazil Shahzad, Ayesha Sana et al. (+1 more)
The deployment of autonomous robotic systems in chemistry laboratories is accelerating experimental workflows and providing the foundational data for AI-driven scientific discovery. However, despite the success of data-driven methods in acquiring dexterous skills, safety remains a primary barrier to their deployment in high-risk domains, such as early-stage materials chemistry experiments. Specifically, learning-based policies frequently struggle to distinguish between safe and unsafe actions, leading to overconfident extrapolation and potentially catastrophic failures. To mitigate these safety risks, we propose SAFE-CHEM, an uncertainty-aware framework designed for robust, learning-based robotic chemists. Our approach leverages an ensemble of recurrent neural network-based imitation learning policies to quantify epistemic uncertainty online through the variance of action predictions. By characterising the success-conditioned density of this variance using kernel density estimation, we introduce a hybrid control architecture that autonomously switches from the learned policy to a deterministic, rule-based backup controller when uncertainty exceeds a calibrated safety threshold. We evaluate SAFE-CHEM across three fundamental laboratory manipulation tasks, where our empirical results demonstrate that this hybrid strategy improves overall task success rates and reduces critical safety violations compared to traditional single-policy baselines. Finally, we demonstrate the practical viability of the framework through zero-shot sim-to-real transfer onto a physical Franka Production 3
robot manipulator.
Read the full abstract ▾
All authors Laura Jones, Shazil Shahzad, Ayesha Sana, Gabriella Pizzuto
Robotics
WorldSimProbe: Diagnosing Simulator Faithfulness in Action-Conditioned World Models for Embodied Manipulation
Peterson Co, Sicheng Hu, Chunxuan Jiao et al. (+17 more)
Action-conditioned world models (ACWMs) promise to provide embodied AI with scalable predictive simulators for planning, policy evaluation, and data generation. Realizing this promise requires precise action-conditioned transitions rather than merely plausible outputs. Yet their applicability remains difficult to establish because prevailing evaluations emphasize visual quality, task outcomes, or coarse rollout-level responsiveness without directly testing simulator fidelity. To address this gap, we evaluate ACWMs through the observable capabilities expected of physical simulators. Accordingly, we formalize Observable Simulator Contract, a minimal contract that any action-conditioned physical simulator should satisfy: supplied actions must induce corresponding
agent motion, and environment responses must be grounded in that realized motion. To operationalize this contract, we introduce WorldSimProbe, comprising five controlled suites spanning local control sensitivity, global trajectory variation, source-diverse actions, interaction grounding, and dynamics. Suite-specific evaluators assess simulator-relative calibration, dense action-to-motion correspondence, false-interaction grounding, and primitive-level dynamics. We evaluate six open-source ACWMs on more than 18,000 instances across RoboTwin, ManiSkill, and LIBERO. World-SimProbe reveals systematic action-realization degradation across control variation, structured failures in interaction grounding and dynamics, and
benchmark signals consistent with human judgments and downstream outcomes. Together, this capability-based framework provides a transparent, and standardized paradigm for diagnosing ACWM simulator fidelity beyond coarse, task-directed evaluation.
Read the full abstract ▾
All authors Peterson Co, Sicheng Hu, Chunxuan Jiao, Hongyang Cheng, Yulin Luo, Yijie Xu, Sixiang Chen, Zhongxia Zhao, Zihao Wang, DaFeng Chi, Peidong Liu, YuTong Chen, Henghua Liu, Zhihao Yuan, Huizhu Jia, Yuzheng Zhuang, Tianle Zhang, Liang Lin, Huajie Tan, Shanghang Zhang
Robotics · Cross-listed from cs.AI
CADEngBench: It Looks Like CAD, but Does It Work? Evaluating Parametric Design, Assembly Reasoning, and Physics Simulation
Harmanjot Singh, Abhra Dubey and Jorge Alejandro Amador Herrera
A CAD model is not engineering-grade merely because it looks correct. It must satisfy design requirements, respond predictably to parameter changes, support controlled edits, match a reference structural response under a declared analysis, and connect to other parts through valid joints. We present CADEngBench, a two-track
benchmark for these capabilities. CADEngBench-P evaluates 300 parametric parts, each used for one zero-to-CAD task and one functional-editing task (600 tasks in total), through boundary-representation (B-Rep) validity, engineering and DFM checks, parameter-family perturbations, functional editing, and matched linear-static FEA in CalculiX. CADEngBench-A evaluates 150 body pairs through ranked joint
retrieval, exact face-and-edge grounding, joint-frame prediction, and kinematic verification. Across eight
multimodal, code-capable models, editing supplied CAD is substantially easier than generating it, while complex edits and matched FEA remain difficult. Assembly predictions often locate the relevant region but fail to recover the recorded joint or mating entities. These results show that CAD evaluation must test engineering behavior rather than appearance alone.
Read the full abstract ▾
Robotics · Cross-listed from eess.SY
Real-Time Nonlinear MPC via Sequential Quadratic Programming with Structure-Exploiting ADMM and Interior-Point Methods for Underactuated Double-Pendulum Swing-Up
Nick Karydakis and Konstantinos Chatzilygeroudis
The 4th "AI Olympics with RealAIGym" competition, to be held at IJCAI-ECAI 2026 in Bremen, challenges participants to develop a global control policy for swinging up and stabilizing an underactuated two-link system in its upright position. In contrast to previous editions, participants develop and evaluate their control strategies directly on remotely accessible CloudPendulum hardware, with limited interaction time and without prior knowledge of the system's model parameters. This paper presents an optimal-control-based approach employing real-time nonlinear model predictive control implemented using sequential quadratic programming. The results demonstrate that the proposed SQP-based MPC controller achieves reliable swing-up and stabilization performance, while maintaining robustness against disturbances.
Read the full abstract ▾
Robotics
Task-Oriented Formation Decision via Reinforcement Learning: Herding an Attacking Swarm
Zhaozong Wang, Guibin Sun, Jinyong Chen et al. (+1 more)
Multi-
robot systems can accomplish tasks that are difficult for a single
robot by organizing into task-specific formations. Different from existing studies on multi-
robot shape formation, we here study the task-oriented formation decision problem, with a focus on the herding task. This task is challenging due to the attackers' superior maneuverability and their unknown strategies. To address these challenges, we propose the following novel results. First, we encode the formation shape using a low-dimensional parameter vector. This parametric representation reformulates the formation decision as a parameter optimization problem, thereby resolving the limited flexibility of predefined shapes. By optimizing these formation parameters, the defenders' maneuverability disadvantage is mitigated through a formation shape that continuously adapts to task requirements. Second, we develop a reinforcement learning-based policy to regulate the formation parameters. Trained offline in simulations covering diverse attacking strategies, the learned policy can effectively handle
adversarial unpredictability during online deployment. Comparative simulations against three baselines demonstrate that our method can successfully accomplish challenging herding tasks. Additional scalability simulations further verify its applicability to simulated scenarios involving dozens of
robots. We also validate the practical feasibility of our method on a physical robotic platform with 3 attackers and 7 defenders.
Read the full abstract ▾
All authors Zhaozong Wang, Guibin Sun, Jinyong Chen, Rui Zhou
Robotics
Ultra-Low-Impedance Robotic Gripper for High-Bandwidth and Transparent Physical Interaction
Joon Lee, Ari Choi and Seokhwan Jeong
Conventional robotic grippers often use high-ratio transmissions to generate grasping torque and external force sensors to measure physical interaction. High-ratio transmissions increase friction, reflected inertia, and mechanical impedance, while external sensors add hardware complexity. To address these trade-offs, this study proposes a novel 9-DOF, three-fingered Differential Direct-Drive (DDD) gripper that combines DD motors with a low-ratio (1:2) differential transmission. The mechanism centralizes actuator mass at the base to minimize moving-link inertia, while the differential architecture couples two motors in parallel to amplify torque during flexion. Experiments show that the prototype delivers a nominal grasping force of approximately 18 N and a fingertip force of 4.7 N, while maintaining a low motor contribution to system inertia (0.236%) and low passive mechanical impedance, with a maximum measured value of 50.1 N/m when the motors are unpowered. The proposed hardware addresses the trade-offs among torque, physical transparency, and kinematic dexterity, providing a foundation for high-bandwidth interaction and sensorless proprioceptive force estimation.
Read the full abstract ▾
Robotics
SAIN: Structure-Aware Interactive Navigation with Active Dialogue Grounding for Mobile Robot
Yuhao Cao, Xiao Liu, Yang Xie et al. (+2 more)
Most existing
vision-
language navigation tasks assume that instructions are complete and unambiguous. However, real-world
robots often encounter natural human instructions that are ambiguous, underspecified, or incomplete, requiring them to resolve such uncertainties through active questioning. Interactive Instance Goal Navigation (IIGN) requires an embodied
agent to find the specific instance under an ambiguous category-level instruction through active dialogue. However, existing dialogue-enabled methods often consume oracle answers as transient textual context for immediate decisions, rather than persistent spatial or object-centric structured state. We present SAIN, a zero-shot framework that turns active dialogue into persistent navigation state. Instead of consuming oracle answers as one-step text hints, SAIN compiles them into target evidence, route-level corridor
memory, and object-candidate labels. These states are stored in structured value, room, graph, and object
memories, then consumed by a unified policy for frontier ranking and final target approach. On the VL-LN IIGN
benchmark, SAIN improves SR from 20.2 to 25.4 and SPL from 13.07 to 14.17 over the strongest reported dialogue-enabled baseline, while requiring no task-specific policy training. The results support dialogue-to-state conversion as an effective zero-shot mechanism for long-horizon interactive instance navigation. Project website: https://zorattc.github.io/SAIN/
Read the full abstract ▾
All authors Yuhao Cao, Xiao Liu, Yang Xie, Lu Liu, Haoyao Chen
Robotics · Cross-listed from cs.HC
Intuitive Directional Sense Presentation to the Torso Using McKibben-Based Surface Haptic Sensation in Immersive Space
Kenta Yokoe, Tadayoshi Aoyama, Yuki Funabora et al. (+2 more)
In recent years, systems that utilize immersive space have been developed in various fields. Immersive spaces often contain considerable amounts of visual information; therefore, users often fail to obtain their desired information. Therefore, various methods have been developed to guide users toward haptic sensations. However, many of these methods have limitations in terms of the intuitive perception of haptic sensation and require practice for familiarization with haptic sensation. Fabric actuators are wearable haptic devices that combine fabric and McKibben artificial muscles to provide wearers with surface haptic sensation. These sensations can be provided to a wide area of the body with intuitive perception, instead of only to a part of the body. This paper presents a novel air pressure adjustment method for whole-body motion guidance using surface haptic sensations provided by a wearable fabric actuator. The proposed system can provide users with a directional sense without visual information in an immersive space. The effectiveness of the proposed system was evaluated through subject experiments and statistical data analysis. Finally, a directional sense presentation was conducted for users performing micromanipulations in a mixed-reality space to demonstrate the applicability of the proposed system for teleoperation.
Read the full abstract ▾
All authors Kenta Yokoe, Tadayoshi Aoyama, Yuki Funabora, Masaru Takeuchi, Yasuhisa Hasegawa
Robotics · Cross-listed from cs.HC
Intuitive Hand Positional Guidance Using McKibben-Based Surface Tactile Sensations to Shoulder and Elbow
Kenta Yokoe, Yuki Funabora and Tadayoshi Aoyama
Hand positional guidance with intuitive perception is crucial for enhancing user interaction and task performance in immersive environments. However, conventional hand positional guidance methods, relying on tactile sensations, lack intuitiveness. Consequently, users require instruction on the relationship between the tactile sensation and target position of the guidance before using these methods. Additionally, the user needs training to become familiar with tactile sensations. This study presents a hand positional guidance system with intuitive perception that leverages McKibben-based surface tactile sensations directed to the shoulder and elbow. We developed a wearable fabric actuator that provides McKibben-based surface tactile sensations to induce six specific movements: elbow flexion, extension, shoulder abduction, adduction, horizontal abduction, and horizontal adduction. The effectiveness of the actuator was experimentally validated, demonstrating its high accuracy in intuitively inducing six movements. An
algorithm based on the
equilibrium point hypothesis and Weber-Fechner law was implemented to regulate the intensity of the tactile sensations for hand positional guidance. Furthermore, the accuracy and speed of the proposed system were compared with that of conventional guidance methods utilizing synthesized
speech and vibrotactile guidance.
Read the full abstract ▾
Robotics
Particle-Based Conformal Prediction for Contact-Aware Uncertainty Calibration in Stratified Configuration Spaces
Luís Marques, Kristian Popov and Dmitry Berenson
Reliable uncertainty representation is essential for deploying autonomous systems that interact with their environment, as
robots must reason about how uncertainty arising from both stochasticity and model mismatch is impacted by contacts with obstacles (e.g., when navigating through a cluttered environment or inserting a part into an assembly). We propose Calibrated Particle-sets for Trans-dimensional Uncertainty Representation (CaPTURe), a geometry-aware, conformal prediction-based
algorithm that generates probabilistically valid prediction regions of the unknown future system configuration using particle-based models of arbitrary fidelity. While calibrated uncertainty predictions are essential for safe and efficient planning, analytical or learned motion models are often inaccurate - due to limited data, simplifying assumptions, unmodeled effects, etc. - which can lead to unsafe executions or task failure. Additionally, when a
robot contacts an obstacle, the distribution of its future configurations can become
multimodal or disjoint, or lie along
manifolds of lower intrinsic dimension than the space of possible
robot configurations. Our method uses a calibration
dataset of system transitions to locally calibrate motion uncertainty estimates, constructing regions guaranteed to contain the future
robot configuration at a user-set probability. Our calibration procedure captures how motion uncertainty varies between contact-rich and contactless motions, leading to sufficient coverage in both cases. We evaluate our method on two simulated planning tasks: controlling a marble around a labyrinth and performing tight-tolerance peg-in-hole insertion with a manipulator. Compared to relevant baselines, CaPTURe achieves the user-specified coverage requirement both in and out of contact and achieves up to a 30% absolute improvement in task success rate over the best baseline.
Read the full abstract ▾
Robotics
SpeedTuning: Speeding Up Policy Execution with Lightweight Reinforcement Learning
David D. Yuan, Tony Z. Zhao, Kaylee Burns et al. (+1 more)
While learned robotic policies hold promise for advancing generalizable manipulation, their practical deployment is often hindered by suboptimal execution speeds. Imitation learning policies are inherently limited by hardware constraints and the speed of the operator during data collection. In addition, there are no established methods for accelerating policies learned via imitation, and the empirical relationship between execution speed and task success remains underexplored. To address these issues, we introduce SpeedTuning, a reinforcement learning framework specifically designed to enhance the speed of manipulation policies. SpeedTuning learns to predict the optimal execution speed for actions, thereby complementing a base policy without necessitating additional data collection. We provide empirical evidence that SpeedTuning achieves substantial improvements in execution speed, exceeding 2.4x speed-up, while preserving an adequate success rate compared to both the original task policy and straightforward speed-up methods such as linear interpolation at a fixed speed. We evaluate our approach across a diverse set of dynamic and precise tasks, including pouring, throwing, and picking, demonstrating its effectiveness and robustness in enhancing real-world robotic manipulation.
Videos and code are available at https://daivdyuan.github.io/speed-tuning/
Read the full abstract ▾
All authors David D. Yuan, Tony Z. Zhao, Kaylee Burns, Chelsea Finn
Robotics
High Fidelity Capture, Reconstruction, and Transfer of Human Demonstrations for Robot-Assisted Bathing
Arjun S. Lakshmipathy, Jonathan P. King, Ethan Zuo et al. (+6 more)
Despite the demand for
robots in high-value clinical tasks like bathing, contemporary systems still lack the safety and reliability required for complex, sustained physical interaction with humans. A key challenge hindering the development of such systems is that collecting, understanding, and effectively transferring highly dynamic, contact-rich human bathing demonstrations is difficult, even with modern motion and tactile sensing equipment. We present a straightforward, but effective framework for doing so with high fidelity by utilizing contact regions as a key processing primitive. We use our framework to build a
dataset of bathing demonstrations performed by trained clinicians on human subjects. We then use this
dataset to design and control an arm-mounted dexterous soft hand to perform bathing tasks on a mannequin using open- and closed-loop strategies. Our
dataset is the first to provide high quality synchronized motion, shape, contact, and force during sustained, contact-rich human-human interaction, and our transfer strategies demonstrate effective use of these data across multiple levels of the robotics stack. All relevant materials will be publicly released to enable further advancements in physical human-
robot interaction (pHRI) research.
Read the full abstract ▾
All authors Arjun S. Lakshmipathy, Jonathan P. King, Ethan Zuo, Rohit Satishkumar, Hongyi Chen, Jeffrey Ichnowski, Dan Ding, Zackory Erickson, Nancy S. Pollard
Robotics
Trajectory Divergence Horizon Decision for Reliable Dual-Arm Surgical Subtask Manipulation
Mingwu Su, Guankun Wang, Jinsong Lin et al. (+8 more)
Surgical robotic systems are increasingly being adopted as clinical workload rises, motivating autonomous solutions for repetitive manipulation subtasks. Learning-based controllers improve generalization compared with rule-based and analytic approaches, but most are trained for individual tasks and remain difficult to reuse across procedures.
Vision-
Language-Action (VLA) models provide a unified framework that integrates visual perception,
language grounding, and action generation, offering a promising path toward more composable surgical autonomy. However, existing VLA policies rely on fixed-length open-loop action sequences, where changing scene conditions can lead to accumulated errors and potential risks in surgical manipulation. To mitigate this issue, we formulate surgical VLA deployment as an adaptive execution-horizon decision problem and propose Trajectory Divergence Horizon Decision (TDHD), a test-time mechanism that estimates step-wise action reliability by measuring the divergence between two flow-matching-generated trajectories under small noise perturbations and truncates execution using a dual-threshold rule to trigger timely replanning. We further establish a real-world da Vinci-like dual-arm
benchmark with synchronized multi-view perception and
language instructions, and collect 600 teleoperated demonstrations across needle (reach, pick, regrasp) and tissue (reach, lift, resection) manipulation suites. On real hardware with 20 trials per task setting, TDHD consistently improves performance over the latest VLA baselines: success increases from 55% to 60% for needle manipulation and from 55% to 80% for tissue manipulation, with the largest gains observed in the final manipulation stages. These results highlight the importance of adaptive execution control for reliable deployment of VLA models in surgical robotic manipulation.
Read the full abstract ▾
All authors Mingwu Su, Guankun Wang, Jinsong Lin, Rulin Zhou, Ziyi Hao, Zhiwei Fang, Huxin Gao, Jiewen Lai, Jiazheng Wang, Fan Zhang, Hongliang Ren
Robotics
ROEVO: Robust Organized Edge Feature-based Visual Odometry Using RGB-D Cameras
Mingrui Liu, Xingxing Zuo, Renlang Huang et al. (+3 more)
This work presents a visual odometry (VO) system that leverages image edge features. Edges are spatially expressive cues commonly present across diverse environments, offering rich textural and structural information. However, existing edge-based VO methods often fail to fully exploit this potential. To this end, we introduce a novel feature representation termed organized edges, which transforms disjoint edge pixels into sequentialized clusters, enabling more effective retention and utilization of the underlying textural and structural information. Another nice property of this formulation is that organized edges can perform edge-level association across multiple frames, enabling the establishment of a co-visibility graph. To achieve precise and efficient pose estimation, we propose a range of particularly designed tracking and joint optimization methods based on the characteristics of organized edges. For tracking, we formulate edge-wise rather than pixel-wise residuals to achieve robust and accurate inter-frame registration. For joint optimization, we introduce a novel shape-preserving edge-fitting method and an organized edge-based Bundle Adjustment (BA) approach, which decomposes the traditional BA problem into fitting and registration to preserve the structural integrity. Based on these novel techniques, we develop a complete VO system that exclusively employs organized edge features, achieving efficient tracking and precise local mapping. Extensive experiments demonstrate its accuracy and robustness in indoor environments, outperforming or achieving comparable performance to state-of-the-art methods. The source code is publicly available at https://github.com/liumingrui814/ROEVO
Read the full abstract ▾
All authors Mingrui Liu, Xingxing Zuo, Renlang Huang, Minglei Zhao, Jiming Chen, Liang Li