Robotics
LV-Calib: LiDAR-Camera Extrinsic Calibration with Boundary-Response Modeling
Sheng Hong
We present LV-Calib, a calibration framework for
LiDAR-camera extrinsic estimation and
LiDAR boundary-response calibration using a printable planar target. The target serves as a shared observation carrier: visual fiducials provide indexed image measurements, while circular reflectivity boundaries provide
LiDAR-observable structural feature points. Instead of directly fitting boundary points as ideal
geometric contours, LV-Calib automatically crops background points, estimates the target plane, and iteratively refines accurate
LiDAR-side 3-D feature points from intensity and
geometric constraints. The refinement explicitly handles the broadened and distorted transition band induced by finite beam footprint and mixed-intensity returns around black-white reflectivity discontinuities. Given these refined
LiDAR features, we formulate a weighted reprojection-consistent extrinsic optimization with
LiDAR feature
alignment, where image observations are kept in the reprojection domain and
LiDAR feature residuals are weighted by refinement confidence. Finally, using the estimated extrinsic and the extracted transition band, LV-Calib calibrates the
LiDAR boundary response by estimating pitch-yaw-range residual statistics of boundary-overlap samples. Experiments on printed-board calibration data demonstrate sub-pixel reprojection accuracy, millimeter-level
LiDAR feature consistency, and improved odometry performance. Code and calibration data will be released for reproducible evaluation.
Read the full abstract ▾
Robotics
Inference-time Policy Steering via Vision and Touch
Yilin Wu, Zilin Si, Zeynep Temel et al. (+2 more)
Inference-time steering adapts pre-trained generative
robot policies during deployment by verifying candidate actions before execution. While prior methods typically perform this verification only with visual observations,
vision alone is often insufficient for contact-rich manipulation, where success depends on both global task progress and subtle local interactions such as contact force. We introduce ViTaL, a visuo-tactile inference-time steering framework that formulates
multimodal guidance as a bi-level optimization problem. At the high level, visual sampling-and-verification performs long-horizon mode selection, deciding what behavior the
robot should execute. At the low level, tactile-guided
diffusion editing refines the selected action sequence over a shorter horizon to satisfy local contact requirements. To support outcome-based steering, ViTaL learns a visuo-tactile latent world model and employs semantically aligned visual and tactile verifiers, including a novel text-conditioned tactile reward that scores predicted tactile futures directly in latent space. Across three real-world contact-rich manipulation tasks, ViTaL improves overall success by 51% over the base policy, outperforms unimodal steering by at least 33%, and exceeds naive
multimodal fusion by at least 20%. Website: https://yilin-wu98.github.io/vital_website.
Read the full abstract ▾
All authors Yilin Wu, Zilin Si, Zeynep Temel, Oliver Kroemer, Andrea Bajcsy
Robotics
Multimodal Physiological Assessment of Contact-Rich Physical Human-Robot Interaction Under Varying Environmental Conditions
Yanyi Chen, Xi Wang and Min Deng
Physical human-
robot interaction (pHRI) in real-world settings exposes operators to fluctuating environmental conditions during contact-rich tasks. Traditional task-centric evaluations overlook the physiological burdens imposed by these stressors. Therefore, we conducted a
multimodal empirical study involving contact-rich tracing tasks under 18 distinct combinations of temperature, acoustic noise, and illuminance. Synchronously, we recorded electrodermal activity (EDA), surface electromyography (sEMG), eye-tracking data, and subjective environmental comfort ratings. Evaluating these physiological signals alongside execution data revealed hidden physiological costs not captured by objective performance. The results revealed that task performance remained stable across all environmental conditions. Autonomic workload, indexed by tonic skin conductance level (SCL), increased with temperature, while physical and cognitive workload were unaffected. Perceived environmental comfort showed no significant association with tracing error or completion time. These findings reveal a compensatory mechanism where operators maintain consistent performance by increasing their physiological effort to suppress thermal discomfort. Such insight motivates the development of physiology-aware control architectures that leverage real-time physiological metrics to reduce operator workload in unstructured environments.
Read the full abstract ▾
Robotics
DynaHMRC: Decentralized Heterogeneous Multi-Robot Collaboration for Dynamic Tasks with Large Language Models
Wenhao Yu, Yu'ang Xie, Yifan Duan et al. (+5 more)
Large
language models (LLMs) provide
robots with richer task understanding and adaptability, making them promising for coordinating heterogeneous multi-
robot systems in long-horizon tasks. Despite this potential, several challenges remain underexplored: (1) Centralized LLM schedulers scale poorly as team size and environmental complexity increase. A single model must process excessive contextual information, and long-context approximation may degrade
reasoning quality; (2) Existing task formulations insufficiently consider dynamic settings, while robust adaptation to evolving task conditions is essential for real-world deployment; (3) Domain-specific data scarcity limits specialized robotic
reasoning, making proprietary general-purpose models inefficient for expert tasks. To address these limitations, we propose DynaHMRC, a decentralized framework in which each
robot acts as a role-aware LLM
agent. This design mitigates the single-model context bottleneck and supports flexible collaboration across heterogeneous team configurations. DynaHMRC organizes collaboration as a four-stage closed-loop process: self-description, task allocation with leadership bidding, leader election, and reflective execution, supported by executable
robot interfaces. We further develop a
benchmark covering three task families, four dynamic variations, and six team configurations to systematically study dynamic task modeling. In addition, we conduct an empirical analysis to guide the construction of domain-specific expert
datasets and fine-tune pretrained LLMs to improve specialized competence. Experiments show that DynaHMRC achieves higher success rates than strong baselines with fewer action and communication steps, while demonstrating promising scalability trends as team size grows within the evaluated settings.
Read the full abstract ▾
All authors Wenhao Yu, Yu'ang Xie, Yifan Duan, Jie Peng, Guanting Ye, Ka-Veng Yuen, Yanyong Zhang, Jianmin Ji
Robotics
VANDERER: Map-Free Exploration using Future-Aware and Visual-Curiosity-Guided Diffusion Policy
Venkata Naren Devarakonda, Raktim Gautam Goswami, Prashanth Krishnamurthy et al. (+1 more)
Mobile
agents require efficient exploration strategies to map unseen environments and autonomously plan tasks. Traditional methods rely on generating occupancy maps and optimizing the sequence in which unexplored regions are visited. However, in sensor-constrained settings, such as those limited to
monocular cameras, generating accurate occupancy maps is challenging. To address this, we propose VANDERER, an exploration framework that leverages a Visual Curiosity Module (VCM) to guide pre-trained
diffusion policies using only
monocular image data. This curiosity module predicts the outcomes of proposed actions via a
navigation world model and evaluates them through a curiosity cost. The cost then guides the
diffusion process toward generating actions that maximize exploration. Evaluated across diverse simulated environments, VANDERER consistently outperforms established baselines, exploring an average of 13.4% more area than NoMaD. Our results reveal a direct correlation between visual and
geometric curiosity in outdoor environments, demonstrating that VANDERER can effectively leverage this relationship for efficient exploration using sensor-constrained
agents.
Read the full abstract ▾
All authors Venkata Naren Devarakonda, Raktim Gautam Goswami, Prashanth Krishnamurthy, Farshad Khorrami
Robotics
TacStyle: Personalizing Tactile Robot Policies using Structured Behavior Representations
Kevin Robledo, Matías I. Torres Galaz, Kumar Dixhant Rai et al. (+3 more)
Robotic systems that assist humans should be capable of adapting their behaviors to individual user preferences. For instance, users may want a
robot arm to adjust the amount of force it applies while folding their laundry or cleaning furniture. Natural
language provides an intuitive way for humans to communicate such preferences. Recent progress in
language-conditioned
robot policies has shown that
robots can successfully use
language prompts to determine what task to perform. However, extending the same approach to realize how the task should be performed requires detailed labels describing the preferences or styles of trajectories in the task data. Not only is collecting such annotations challenging, but conditioning directly on these labels may also fail to provide fine-grained control over a continuous range of behaviors. For example, it can be difficult to convey the exact force that a
robot must apply through abstract instructions like "apply a bit more pressure than before". Therefore, in this work, we propose using
language to reason over preferred behaviors instead of directly generating them. We first learn a structured latent representation that organizes user preferences according to differences in the corresponding trajectories. Then, given a preference
prompt, we use a foundation model to interpret this latent space and choose a value that produces the desired behavior. Through both simulation and real-world experiments, we show that selecting
robot behaviors from an intuitively structured latent space enables more precise adaptation to user preferences while requiring significantly fewer preference labels than
language-conditioned policies.
Read the full abstract ▾
All authors Kevin Robledo, Matías I. Torres Galaz, Kumar Dixhant Rai, Shelly Sara Ulman, Tasmia Tasrin, Heramb Nemlekar
Robotics · Cross-listed from cs.CV
Instruct-Particulate: Scaling Feed-Forward 3D Object Articulation with Kinematic Control
Ruining Li, Yuxin Yao, Matt Zhou et al. (+5 more)
Reconstructing articulated 3D objects is important for animation, gaming, and robotic simulations. Recent neural networks can estimate the articulated structure of 3D objects, but their generalization remains limited by the scarcity of annotated data for this task. To address this gap, we introduce Instruct-Particulate, a model that takes a 3D mesh together with a target kinematic specification, including part descriptions, connectivity, joint types, and optional point
prompts, and predicts the corresponding kinematic part
segmentation and joint motion parameters. The kinematic specification disambiguates the task and allows the model to target annotations of different granularity, thereby making it possible to use more abundant heterogeneous training data. At test time, the kinematic specification can be obtained automatically from large-scale
vision-
language models, so the model can be applied to any input mesh. To train our model at scale, we construct a heterogeneous
dataset of more than 150,000 articulated 3D objects, extending existing publicly available collections with data obtained by partially labelling other 3D models (monolithic or already decomposed into parts) with kinematic labels by means of
vision-
language models. Experiments show that our model generalizes better across categories and to AI-generated meshes, enabling articulated asset reconstruction from real-world images via image-to-3D models.
Read the full abstract ▾
All authors Ruining Li, Yuxin Yao, Matt Zhou, Chuanxia Zheng, Christian Rupprecht, Joan Lasenby, Shangzhe Wu, Andrea Vedaldi
Robotics
EgoGuide: Egocentric Guidance for Efficient Robot-Free Demonstration Collection and Learning
Yue Xu, Mingtao Nie, Tianle Li et al. (+4 more)
Robot learning from real-world demonstrations is currently constrained by data scaling. Universal Manipulation Interface (UMI) provides an efficient
robot-free data collection interface, yet current UMI-style pipelines often collect redundant demonstrations and lack global scene context. To improve data efficiency, we present EgoGuide, a collection interface that records synchronized wrist and head/
egocentric observations and couples them with online visual-
geometric data quality guidance. We also introduce a Gated
Egocentric Residual Policy for robust learning from a viewpoint-varying
egocentric camera, allowing head/
egocentric context to correct ambiguous local observations while preserving stable wrist-view control. Real-world experiments show that EgoGuide reduces the required number of data episodes and improves data efficiency. The residual policy further improves robustness under visual occlusion. Project Page: https://silicx.github.io/EgoGuide
Read the full abstract ▾
All authors Yue Xu, Mingtao Nie, Tianle Li, Hong Li, Yibo Luo, Siyuan Huang, Yong-Lu Li
Robotics
Contact-Consistent Interaction Dynamics Normalization for Predictive Physical Human–Robot Interaction
Yongyan Cao
Safe physical human–
robot interaction on floating-base
robots requires interaction regulation under changing contact constraints. We develop a contact-consistent normalization in which the residual end-effector channel is represented as a linear double integrator in acceleration coordinates. Both discrete prediction
matrices are independent of configuration and support mode; posture and contact enter only through task-inertia force recovery and constraints. The controller combines a constant-Hessian receding-horizon QP, an acceleration-disturbance observer, and a priority-consistent realization. Classical operational-space impedance is shown to be the unconstrained infinite-horizon limit. MuJoCo experiments on a 17-DOF biped and a Menagerie-derived Unitree G1 model evaluate sustained forces, transmitted shocks, and scheduled contact-model changes. The observer sustains near-offset-free tracking across a genuine contact-set switch and a scheduled support-mode transition, while disturbance estimation—not contact consistency alone—is the dominant source of fixed-stance accuracy; covariance inflation gives only scenario-dependent transient benefit.
Read the full abstract ▾
Robotics
Safe Reinforcement Learning of Autonomous Highway Driving: A Unified Framework for Safety and Efficiency
Chufei Yan, Zhihao Cui, Yiyan Lv et al. (+3 more)
Deep reinforcement learning (DRL) offers a compelling route to decision-making for advanced autonomous vehicles (AVs), yet its trial-and-error nature makes it difficult to guarantee safety during training and to achieve both safety and efficiency at deployment. We propose a unified safe reinforcement learning (SRL) framework that integrates safe distance (SD), reward machines (RM), and mixture-of-experts (MoE), termed MoE-RM-SRL. For deployment, SD and RM jointly shape a rule-aware reward that encodes highway traffic regulations and stage-wise objectives, enabling safe and reliable behavior without sacrificing efficiency. For training, we introduce a sparsely gated MoE layer comprising up to 11 deep Q-networks (DQNs); an SD-based gating rule activates a minimal set of experts for lane-keeping and lane-changing, mitigating the instability, discontinuities, and impulsive transients commonly induced by switching between heterogeneous controllers (e.g., MPC/rule-based modules and learned policies). We implement the proposed architecture in CARLA and integrate it with a 6-DoF driver-in-the-loop virtual-reality (DiL-VR) platform. Experiments in
stochastic two-lane traffic show that MoE-RM-SRL substantially improves safety and efficiency over state-of-the-art baselines, and the framework naturally extends to multi-lane driving as well as on-ramp merging and exiting scenarios.
Read the full abstract ▾
All authors Chufei Yan, Zhihao Cui, Yiyan Lv, Taojie Chen, Ning Bian, Yulei Wang
Robotics
Interaction Dynamics for Dexterous Manipulation
Yongyan Cao
Dexterous manipulation is fundamentally a problem of interaction dynamics: the hand must track precise finger trajectories, regulate the contact force exchanged with grasped objects, respect actuation and safety limits, and remain predictable when contact persists – objectives in tension for any fixed-gain controller. A sustained contact torque $\tau_{\text{ext}}$ through a joint stiffness $K_d$ produces the structural bias $e_\infty=\tau_{\text{ext}}/K_d$, so stiffening for accuracy sacrifices contact safety while softening yields by design. We make these interaction dynamics explicit and actuator-agnostic through a constant-$A_d$ double-integrator backbone, instantiating the offset-free architecture established for physical human-
robot interaction (pHRI) and preserving its modeling assumptions on the reduced residual dynamics. An algebraic feedforward reduces the tendon transmission – hydraulic, cable, pneumatic, twisted-string, or series-elastic – to a constant-coefficient double integrator, so the QP cost inverse is precomputed offline and a 10-step receding-horizon QP runs at 500\,Hz under contact-force (ISO/TS 15066), actuation, and jerk constraints. An encoder-only augmented-Kalman disturbance state drives steady-state error to zero under constant contact loads in the nominal detectable case. In simulation, a hydraulically actuated finger – the worked example, adding pressure and cavitation constraints – attains 0.6\,mrad RMS, 0.1\,mrad steady-state, and 7.3\,mrad peak deflection under 1.5\,Nm contact: 153$\times$, 1500$\times$, and 21$\times$ better than classical impedance. The realized first-move stiffness (18$\to$323\,Nm/rad with update rate) is independently verified, and the architecture scales to a 16-DOF LEAP Hand MuJoCo model, recovering from 2.5\,N grasp disturbances within 0.7\,s.
Read the full abstract ▾
Robotics
What Robots Do Matters More Than What They Look Like: Task Context Shapes Trust in Educational HRI
Anna-Maria Velentza, Konstantina Nikou, Anne-Gwenn Bosser et al. (+1 more)
Socially assistive
robots (SARs) are increasingly deployed in educational and information-sharing contexts, supported by advances in large
language models that enable fluent real-time interaction. Despite the growing diversity of
robot embodiments, it remains unclear whether a single
robot appearance is appropriate across different interaction tasks or whether trust depends primarily on contextual factors. In this study, we examine how
robot appearance and task type jointly influence trust in
robots. Using a within-subjects
video-based experiment (N = 81), participants evaluated three
robots with distinct appearances while performing three educationally relevant tasks: teaching, procedural instruction, and personal-information discussion. Results from repeated-measures analyses show a strong main effect of task on trust, with participants reporting the highest trust during instructional guidance, moderate trust during teaching activities, and significantly lower trust when
robots requested personal information. In contrast,
robot appearance showed no significant main effect, and the interaction between appearance and task was marginal. These findings suggest that trust in human-
robot interaction is shaped more strongly by task context than by physical embodiment alone. By focusing on future educators as end users, this work contributes empirical evidence toward task-aware
robot deployment in educational environments and highlights the importance of aligning
robot roles and behaviors with interaction goals rather than relying solely on anthropomorphic design.
Read the full abstract ▾
All authors Anna-Maria Velentza, Konstantina Nikou, Anne-Gwenn Bosser, Nikolaos Fachantidis
Robotics
Sensitivity Shaping for Latent Modeling
Hongzhan Yu, Chenghao Li, Ruipeng Zhang et al. (+2 more)
Generative dynamics models enable planning in challenging systems, but safe deployment requires detecting policy-induced out-of-distribution (OOD) transitions. Existing methods typically treat learned dynamics as fixed and rely on post hoc support surrogates for OOD detection. This overlooks a critical failure mode: learned dynamics that are insensitive to control changes can map unsupported controls to latent predictions resembling demonstrated transitions, suppressing OOD signals despite large prediction errors. We introduce support-conditioned control-sensitivity regularization to preserve control-induced variation by promoting local responsiveness in well-supported training regions. Experiments in
vision-based obstacle avoidance, manipulation, and real-
robot navigation demonstrate improved OOD detection and safer closed-loop planning.
Read the full abstract ▾
All authors Hongzhan Yu, Chenghao Li, Ruipeng Zhang, Henrik Christensen, Sicun Gao
Robotics
ORCA: A Platform for Open-Source Dexterity Research
Francesco Capuano, Maximilian Eberlein, Fabrice Bourquin et al. (+1 more)
Robotics manipulation research increasingly focuses on two-finger parallel grippers for their effectiveness, affordability, and ease of teleoperation. Grippers are nonetheless limited by their form factor, often requiring bimanual setups even for simple reorientation tasks. Anthropomorphic hands are a more natural platform for dexterous
robot learning – closer to the human hand, and capable of learning from human
video – yet they remain hard to use in learning research: even where open and accessible hand hardware exists, the software for control, simulation, teleoperation, and retargeting is scattered in one-off code bases, and largely disconnected from the
robot-learning ecosystem. In this work, we introduce the \orca~learning stack, an open-source research stack for dexterity as a first-class
robot learning domain. Our \orca~stack unifies low-level control, simulation, teleoperation from a range of consumer platforms, and hand retargeting, behind a single interface, and integrates natively with popular
robot-learning frameworks such as \lerobot, so dexterous hand researchers can leverage the same data, training, and evaluation pipelines used for non-dexterous
robot learning. We demonstrate a complete end-to-end workflow, collecting expert demonstrations of an in-hand reorientation task by teleoperation with a consumer-grade VR headset, training an autonomous policy with \lerobot, and evaluating the learned policy in a fully reproducible and observable setup. We open-source the entire stack as a shared, reproducible foundation for dexterous-manipulation research.
Read the full abstract ▾
All authors Francesco Capuano, Maximilian Eberlein, Fabrice Bourquin, Clemens Claudio Christoph
Robotics
TRACE: Trajectory-Routed Causal Memory for Delayed-Evidence Visuomotor Imitation
Zihao Li, Ranpeng Qiu, Yincong Chen et al. (+2 more)
Robots under autonomous operation may require decisions based on evidence that is no longer visible. We study delayed-evidence tasks, where an early cue disappears before a later decision point, so visually similar observations can require different actions. In these settings, the current observation is not a sufficient state for control. We introduce TRAjectory-routed Causal Evidence (TRACE), a
memory framework for visuomotor imitation policies. TRACE stores task-relevant visual and
robot-state evidence, such as object identity, target choice, or route-dependent state, in a fixed-size latent
memory that remains bounded over long episodes. Instead of indexing
memory by raw time or manually provided task labels, TRACE uses path signatures: compact, order-sensitive features of the executed
robot-state trajectory. These signatures do not store the visual cue itself; rather, they provide trajectory-conditioned keys for writing and retrieving the evidence stored when the cue was visible. When the
robot later reaches an ambiguous observation, the policy conditions on TRACE
memory to recover the missing context and choose the correct branch. TRACE attaches through lightweight adapters to policies, without changing the policy backbone, action head, or imitation objective. Across real-world long-horizon manipulation tasks with visually ambiguous branch points, TRACE improves branch selection and task success over alternative baselines, including short-history and recurrent
memory. Project page: https://jeong-zju.github.io/trace
Read the full abstract ▾
All authors Zihao Li, Ranpeng Qiu, Yincong Chen, Guoqiang Ren, Weiming Zhi
Robotics · Cross-listed from cs.LG
Provably Safe, Yet Scalable Reinforcement Learning
Kai S. Yun, Zeyang Li and Navid Azizan
Safe reinforcement learning (RL) aims to learn policies that optimize rewards while satisfying constraints. Predominant approaches rely on soft-constrained policy optimization, which has achieved empirical success but does not provide formal safety guarantees for the learned policy. In contrast, methods with strict guarantees typically rely on explicit certificate functions, whose construction requires the direct synthesis and verification of control-invariant sets, a process that scales poorly with state dimension and often yields overly conservative behavior. In this paper, we present the Provably Safe, yet Scalable RL (PS2-RL) framework, a novel two-phase architecture for learning provably safe policies in a scalable manner, designed to overcome the key bottlenecks of prior methods. Rather than explicitly computing invariant sets, PS2-RL leverages a learned backup policy to forward-integrate the system dynamics, generating an implicit control-invariant set online. In the first phase, the backup policy is trained with our proposed safe-arrival value function, which characterizes the optimal backup policy for invariant-set construction. In the second phase, an RL policy is trained end-to-end through a differentiable projection layer that strictly enforces the safety guarantees induced by the learned backup policy. By maximizing the volume of the implicit control-invariant set in the first phase, the resulting PS2 policy from the second phase is performant and scalable, while maintaining provable safety. Crucially, PS2-RL imposes no restrictions on the underlying RL
algorithm and can be plugged into any existing training pipeline. We establish theoretical guarantees for the proposed framework and evaluate it on robotic control tasks with state dimensions up to 10, a regime in which prior provably safe RL methods struggle or become impractical.
Read the full abstract ▾
Robotics
Spatially Conditioned Diffusion Policy: Learning Precise and Robust Manipulation with a Single RGB Camera
Seoyoon Kim, Kanghyun Kim, Dongwoo Ko et al. (+2 more)
Recent visual imitation learning systems have widely adopted multi-camera setups with wrist-mounted cameras as the de facto standard. However, manipulation from a single global view remains challenging, as the policy should capture fine-grained interaction details and identify task-relevant regions without local wrist views. To address this challenge, we present Spatially Conditioned
Diffusion Policy (SCDP), a
diffusion-based visuomotor policy that achieves precise and robust manipulation in a single-camera setting. Our key idea is that end-effector trajectories can serve as visual
attention anchors that reflect task-relevant regions. Building on this idea, SCDP consists of two key components: (i) a visual encoder that produces multi-scale feature maps to capture both broader context and fine-grained visual features, and (ii) a spatial conditioning module that samples point-wise features along intermediate end-effector trajectories in the
diffusion loop. Extensive simulation experiments show that SCDP consistently outperforms strong single-view baselines and achieves performance comparable to multi-camera baselines. Real-world experiments further demonstrate precise manipulation and robustness to visual distractors, highlighting the potential of single-camera imitation learning.
Read the full abstract ▾
All authors Seoyoon Kim, Kanghyun Kim, Dongwoo Ko, Yeong Jin Heo, Min Jun Kim
Robotics
AERMANI-PLACE: Language Guided Object Placement with Aerial Manipulators
Sarthak Mishra, Ritama Sanyal, Rishabh Dev Yadav et al. (+2 more)
Object placement is a fundamental component of aerial manipulation tasks, yet existing systems typically require the desired placement position to be specified explicitly in metric coordinates. Such interfaces are not intuitive and require users to reason about coordinate frames and scene geometry, making them difficult to use in practical deployments. In contrast, humans often communicate spatial goals through a combination of
language and pointing gestures. Inspired by this observation, we present AERMANI-PLACE, a framework for
language-guided object placement with aerial manipulators. Given a scene image and a natural
language instruction, an image editing model generates a modified version of the scene containing a visual marker that indicates where the object should be placed. This marker is then grounded into the physical environment using depth observations to recover a metric place point, after which a placement trajectory is generated and executed by the aerial manipulator. We evaluate the proposed approach on a test set of 100
language-guided placement tasks and demonstrate successful execution on a real aerial manipulation platform. Experimental results show that the proposed method reliably infers placement locations from
language instructions with an average success rate of 87% on the test-set and transfers effectively to real-world aerial manipulation with an average success rate of 72%.
Video: https://youtu.be/SgwwgLBsv0g
Read the full abstract ▾
All authors Sarthak Mishra, Ritama Sanyal, Rishabh Dev Yadav, Wei Pan, Spandan Roy
Robotics
Physics-Grounded Causal Auditing of End-to-End Driving Planners
Zikun Guo, Minglan Chen, Jinyou Zhai et al. (+1 more)
End-to-end (E2E) autonomous-driving planners trained by imitation are prone to statistical shortcuts: they associate scene elements that merely co-occur with expert actions (a roadside object, a building facade) with driving decisions, rather than the variables that causally determine them. Such causal confusion silently compromises reliability in long-tail scenarios, and it is difficult to detect, because prevailing open-loop metrics (L2 displacement and collision rate) are dominated by ego status and do not indicate whether a planner depends on spurious cues. Existing remedies based on causal-intervention training require retraining large models and cannot audit a planner that is already deployed. We present CADET, a training-free framework that audits,
benchmarks, and repairs spurious reliance in pretrained E2E planners without any parameter update.
Read the full abstract ▾
All authors Zikun Guo, Minglan Chen, Jinyou Zhai, Rongjin Zou
Robotics
Kine2Go: Kinematic dataset for the Unitree Go2 robot with diverse gaits and motions
Władysław Pałucki, Paweł Siwak, Krzysztof Ciebiera et al. (+1 more)
The recent popularity of robotics, combined with the steadily decreasing cost of robotic hardware, has lowered the entry barrier to robotics research and enabled rapid advancements in the field. One of the primary examples is the Unitree Go2 quadruped
robot, which is often used by researchers in the areas of locomotion,
navigation, control, and others. Many researchers use the Go2
robot in combination with techniques like imitation learning, reinforcement learning, and behavioral cloning to allow machine learning systems to take full control of the
robot. At the same time, many of those techniques require demonstration data consisting of the
robot's kinematics information and actions applied to the motors. Obtaining such data is difficult, requires building complex pipelines, and can take significant time. To aid in those kinds of efforts, we present Kine2Go - a
dataset with 800 diverse gait kinematics trajectory motion data for the Unitree Go2
robot, derived from 40 distinct policies. Our pipeline accepts data from various quadruped morphologies and translates them to a Go2-compatible format. Then we use Reinforcement Learning to train policies following a given motion, and finally we gather data from those policies, which grants robust, perturbed kinematic data with corresponding motor-level actions.
Read the full abstract ▾
All authors Władysław Pałucki, Paweł Siwak, Krzysztof Ciebiera, Marek Cygan
Robotics
ForestBack: Breadcrumb-Based Pedestrian Dead Reckoning for Infrastructure-Free Return Navigation
Aueaphum Aueawatthanaphisut and Chanakan Chaipan
Reliable return
navigation remains an important challenge in GPS-denied environments where external positioning infrastructure may be unavailable or unreliable. This paper presents ForestBack, an infrastructure-free pedestrian return
navigation framework based on breadcrumb-based pedestrian dead reckoning (PDR). The system records a user's walking route as a sequence of reversible breadcrumb nodes and generates reverse-path guidance without requiring GPS, Wi-Fi, Bluetooth beacons, or pre-installed infrastructure. ForestBack integrates acceleration-based step detection, adaptive step-length estimation, magnetometer-assisted heading estimation, barometric-altitude correction, and bidirectional breadcrumb path reconstruction. The system was evaluated using an indoor obstacle-avoidance route with five checkpoints, where the user navigated around a central obstacle. A
dataset of 36 walking trials and 42,474 time-series samples was used for evaluation, including IMU signals, magnetometer readings, barometric variables, turn-event labels, ground-truth trajectories, baseline PDR outputs, proposed ForestBack outputs, and power-related measurements. Experimental results show that ForestBack reduced the mean RMSE from 1.129 m to 0.965 m compared with traditional PDR, corresponding to a 15.76% improvement. The mean final-position error was reduced from 1.781 m to 1.388 m, while turn-event detection consistency reached approximately 99.90%. These results indicate that ForestBack improves trajectory reconstruction and route-preserving return guidance in obstacle-avoidance scenarios. The released
dataset and analysis notebook support reproducibility and future benchmarking of infrastructure-free PDR-based return
navigation systems.
Read the full abstract ▾
Robotics · Cross-listed from cs.AI
Causal Object-Centric Models for Planning with Monte Carlo Tree Search
Rodion Vakhitov, Leonid Ugadiarov, Alexey Skrynnik et al. (+1 more)
We introduce COMET (Causal Object-centric Model for Efficient Tree search), a model-based reinforcement learning
algorithm that performs Monte Carlo Tree Search in a slot-structured latent space. COMET pairs a frozen unsupervised object-centric encoder with a
transformer-based world model, in which actions are bound to objects through a novel action-slot fusion mechanism that is used in slot transition prediction. Policy and value heads use object-causal
attention, modulating
token interactions by learned per-slot relevance scores so that decision-making concentrates on task-relevant entities. COMET adds an explicit object-level inductive bias to MuZero-style latent planning. Across eight visually and dynamically diverse tasks from the Object-Centric Visual RL
benchmark, ManiSkill, Robosuite, and VizDoom, COMET achieves a higher mean normalized score during the early stages of training compared to object-centric and monolithic baselines.
Read the full abstract ▾
All authors Rodion Vakhitov, Leonid Ugadiarov, Alexey Skrynnik, Aleksandr Panov
Robotics
Hy-Embodied-0.5-VLA: From Vision-Language-Action Models to a Real-World Robot Learning Stack
He Zhang, Lingzhu Xiang, Haitao Lin et al. (+23 more)
In this report, we present Hy-
Embodied-0.5-VLA, abbreviated as HyVLA-0.5, an end-to-end system that spans the full
robot learning stack: data collection, model design, continued pre-training and supervised fine-tuning, RL post-training, and real-world deployment. Each component serves a distinct role in this stack.
Read the full abstract ▾
All authors He Zhang, Lingzhu Xiang, Haitao Lin, Zeyu Huang, Minghui Wang, Dingyan Zhong, Yubo Dong, Yihao Wu, Yongming Rao, Dongsheng Zhang, Wanjia He, Ling Chen, Kai Huang, Jiahao Chen, Sichang Su, Xumin Yu, Ziyi Wang, Chengwei Zhu, Xiao Teng, Yuchun Guo, Yufeng Zhang, Yuandong Liu, Rui Wang, Zisheng Lu, Han Hu, Zhengyou Zhang
Robotics
Elastic Queries Reinforcement Learning: Self-Aware Policy Execution for VLA Models
Ge Wang, Xinyu Tan, Xiang Li et al. (+11 more)
Vision-
language-action (VLA) models are powerful action generators for
robot manipulation, but they are typically executed with fixed inference and replanning schedules. This
rigidity ignores the uneven difficulty of
robot control: contact-rich or uncertain states may need more computation and fresher feedback, while easier states can often be handled with fewer inference steps and longer open-loop execution. We propose Elastic Queries Reinforcement Learning (EQRL), a framework that makes each VLA policy query elastic. A lightweight latent-schedule adaptor jointly selects the latent input, denoising budget, and action chunk length, without fine-tuning the underlying VLA model. To make scheduling difficulty-aware, EQRL trains a critic over the joint latent-schedule action and derives a state difficulty signal from critic ensemble disagreement. This signal guides compute toward difficult states, while a learned residual allows task-driven correction. We formulate variable chunk execution as query-level macro-action RL with chunk-dependent discounting and an amortized number-of-function-evaluations (NFE) budget. Across simulation and real-
robot manipulation, EQRL reduces amortized inference cost while preserving or improving task success.
Read the full abstract ▾
All authors Ge Wang, Xinyu Tan, Xiang Li, Man Luo, Chengsi Yao, Shenhao Yan, Jiahao Yang, Fan Feng, Honghao Cai, Xiangyuan Wang, Zhixin Mai, Yiming Zhao, Yatong Han, Zhen Li
Robotics
Robust Fall Recovery for Armless Bipedal-Wheeled Robots Via Force-Guided Learning
Haidong Hou, Zhangguo Yu, Tao Han et al. (+6 more)
Fall recovery is critical for autonomous legged locomotion. Existing methods have demonstrated that some legged
robots, such as
humanoids and quadrupeds, are capable of fall recovery from diverse postures by utilizing arms or coordinating multi-legs to generate support forces. Without arms or other legs to provide supportive assistance, a bipedal-wheeled
robot must rely solely on the actuation of its legs, making recovery particularly difficult. To address this, we introduce FTSR (Force-guided Teacher-student framework with Stage-wise Rewards). The force-guided method constructs an external auxiliary force during simulation training that correlates directly with the
robot's real-time height, explicitly formulating this force as an optimizable constraint. Through constrained reinforcement learning, the policy is guided toward reducing force dependency gradually and increasing the body height, developing internal recovery strategies despite having no arms for support. Height-progressive stage-Wise rewards progressively structure posture stabilization during recovery and transition to sustained locomotion, integrated with teacher-student architecture distilling privileged knowledge of force effects and recovery dynamics. After simulation training, the policy is deployed on a physical armless bipedal-wheeled
robot and extensively evaluated. Experiments confirm robust and reliable fall recovery under diverse challenging conditions, demonstrating strong environmental adaptability and motion robustness, while maintaining full post-recovery motion capability. The framework also generalizes effectively to a high-DOF
humanoid, confirming its practical generalizability. The project page is available at https://2350575870.github.io/force-guided.github.io/
Read the full abstract ▾
All authors Haidong Hou, Zhangguo Yu, Tao Han, Hengbo Qi, Khaleel Ghazal, Yu Zhang, Yidong Du, Xuechao Chen, Fei Meng
Robotics
FloVerse: Floor Plan-Guided Multi-Modal Navigation
Weiqi Huang, Shuangyi Dong, Jiaxin Li et al. (+3 more)
Floor plans encapsulate compact spatial priors, enabling
agents to navigate unseen scenes more efficiently. While prior work has explored floor plan-guided
navigation, it has focused mainly on PointNav and a limited set of environments. To bridge this gap, we introduce FloVerse, a new task for floor plan-guided
embodied navigation that unifies PointNav, ObjectNav, and ImageNav. To support FloVerse, we assemble FloVerse-1.6K, a large-scale
dataset of 1.6K scenes from HM3D and Gibson 4+, paired with corresponding floor plans, comprising 240K expert trajectories and 12M RGBD frames. We further propose ThreeDiff, a two-stage imitation learning policy comprising a planner, a
diffusion-based
multimodal goal-
reasoning module trained via masked-modality modeling, and a refiner, a depth-based trajectory-refinement module for safe execution. Extensive experiments demonstrate that (1) floor-plan priors improve
navigation performance across all goal modalities, and (2) ThreeDiff implicitly captures spatial information from floor plans. These results underscore the effectiveness of spatial priors and validate our proposed unified approach for floor plan-guided
embodied navigation.
Read the full abstract ▾
All authors Weiqi Huang, Shuangyi Dong, Jiaxin Li, Yifei Guo, Zan Wang, Wei Liang
Robotics
ReactVLA: Fast and Lightweight Reactive Robot Manipulation via Improved Mean Flow Action Generation
Yanzhao Guo, Wenkai Chen and Jianwei Zhang
Diffusion-based
Vision-
Language-Action (VLA) policies have demonstrated strong capability in modeling expressive and
multimodal action distributions. However, their reliance on iterative sampling introduces substantial inference latency, which limits their applicability to reactive closed-loop
robot manipulation. To address this limitation, we propose ReactVLA, a lightweight and low-latency VLA framework for real-time robotic manipulation. ReactVLA combines two complementary designs: (1) an improved Mean Flow (iMF) action generator that reduces expensive multi-step
diffusion sampling to one-to-few-step action generation, and (2)
Attention Residuals (AttnRes), a dynamic depth-wise feature routing mechanism that replaces uniform residual accumulation to better preserve task-relevant
multimodal representations. We evaluate ReactVLA on large-scale simulation
benchmarks, including LIBERO and RoboIMI, as well as real-world robotic manipulation tasks. Experimental results show that ReactVLA consistently outperforms similarly sized VLA baselines, including SmolVLA and $\pi_0$. On challenging precision manipulation tasks, ReactVLA achieves up to a 1.65$\times$ improvement in task performance while providing more than a 4$\times$ increase in inference speed compared with leading VLA models. Finally, it reduces real-world policy latency to below 38.6 ms, enabling fast reactive control on physical
robot platforms. Please check out our project website at: https://game-loader.github.io/ReactVLA/.
Read the full abstract ▾
Robotics
Optimality-Preserving Decomposition for Scalable QAOA in Natural-Language-Guided Multi-Drone Assignment
Junyeop Bang, Byongho Lee, Dohyun An et al. (+1 more)
As multi-drone fleets scale, zone assignment rapidly evolves into an intractable NP-hard combinatorial problem that overwhelms classical exhaustive search. While
quantum optimization promises to shatter these classical bottlenecks, mapping complex spatial tasks from human intent to restricted
quantum hardware remains a severe challenge. To bridge this gap, we present an end-to-end framework integrating a fine-tuned Large
Language Model (LLM) front-end with a highly scalable, domain-specific
quantum-classical backend. The front-end utilizes Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO) to translate free-form natural
language instructions into structurally robust Quadratic Unconstrained Binary Optimization (QUBO) constraints without false negatives. To overcome the strict
qubit limits of near-term
quantum devices, our framework features a novel constraint-preserving graph partitioner and a compressed separator-based dynamic programming (DP) merge. By structurally encoding constraints via W-state initialization and XY-mixers in Conditional Value-at-Risk
Quantum Approximate Optimization (CVaR-QAOA), the pipeline stays highly compact. Empirical results demonstrate that this architecture circumvents classical scaling walls, recovering the global optimum on 100% of idealized oracle cases and 96.3% under real QAOA sampling, enabling natural-
language-guided task allocation at previously intractable scales.
Read the full abstract ▾
All authors Junyeop Bang, Byongho Lee, Dohyun An, Hwangnam Kim
Robotics
SyLink Hand: A Synergy-Inspired Linkage-Driven Anthropomorphic Hand for Human-Like Dexterity
Hao Wu, Yanzhe Wang, Yu Feng et al. (+4 more)
Designing anthropomorphic robotic hands that balance functional dexterity with mechanical simplicity remains a significant challenge. Inspired by human hand synergies, this paper presents the SyLink Hand, an anthropomorphic dexterous hand that integrates biomechanical synergy principles with linkage-driven transmission mechanisms to achieve a high degree of anthropomorphism in appearance, kinematics, and functionality within a compact and cost-effective architecture. Biomechanical analysis of natural hand motions using motion capture gloves reveals strong kinematic correlations among hand joints, providing the basis for a simplified yet functional degree-of-freedom (DOF) configuration. Guided by these synergistic characteristics, optimized linkage mechanisms are employed to coordinate multiple joint motions and reproduce natural finger trajectories. A novel spherical four-bar linkage is further proposed to achieve decoupled flexion/extension (Flex/Ext) and abduction/adduction (Abd/Add) at the metacarpophalangeal joint within a compact form factor. The resulting prototype integrates 19 joints driven by 11 actuators, with a total mass of 520g and a manufacturing cost of approximately USD 400. Experimental evaluations demonstrate its human-like kinematic performance, high load-bearing capability, and versatile grasping and manipulation skills. These results validate that the synergy-inspired, linkage-based design effectively balances anthropomorphism, mechanical simplicity, and functional versatility, highlighting its potential for practical deployment in dexterity-demanding robotic applications.
Read the full abstract ▾
All authors Hao Wu, Yanzhe Wang, Yu Feng, Yitong Li, Jingxiang Guo, Jian Liu, Jianshu Zhou
Robotics
When and How Severely: Scenario-Specific Safety Envelopes for Driving VLAs
Abhinaw Priyadershi and Jelena Frtunikj
Safety certification of
Vision-
Language-Action (VLA) driving planners under ISO 21448 (SOTIF) rests on an Operational Design Domain (ODD) specification that answers two complementary questions: when does the planner start to fail, and how severely does it fail once it does? We evaluate Alpamayo R1, a 10B-parameter open-weight driving VLA, on 15,968 (clip, attack) pairs. We find a conservative-aggregate gap: an aggregate safe threshold of $\sigma \leq 50$ under a 15% average displacement error (ADE) budget masks well-sampled scenarios that tolerate the top of the tested grid ($\sigma = 70$). A Gaussian Mixture Model (GMM) on the changed-explanation subset identifies six discrete severity bands (BIC-optimal $k{=}6$), so two perturbation conditions with the same mean error can differ materially in their share of high-severity (C4/C5) failures. Joining the two analyses on the same corpus surfaces a finding neither yields in isolation: the scenarios with the loosest noise thresholds are not those with the lowest high-severity rate: STOP_SIGNAL concentrates roughly $4\times$ the C4/C5 share of LANE_KEEPING despite tolerating a larger $\sigma$. A deployable SOTIF ODD specification for driving VLAs therefore requires a two-dimensional safety envelope, not a single aggregate value per hazard.
Read the full abstract ▾