Robotics
POIROT: Investigating Direct Tangible vs. Digitally Mediated Interaction and Attitude Moderation in Multi-party Murder Mystery Games
Wen Chen, Rongxi Chen, Shankai Chen et al. (+5 more)
As social
robots take on increasingly complex roles like game masters (GMs) in multi-party games, the expectation that physicality universally enhances user experience remains debated. This study challenges the "one-size-fits-all" view of tangible interaction by identifying a critical boundary condition: users' Negative Attitudes towards
Robots (NARS). In a between-subjects experiment (N = 67), a custom-built
robot GM facilitated a multi-party murder mystery game (MMG) by delivering clues either through direct tangible interaction or a digitally mediated interface. Baseline multivariate analysis (MANOVA) showed no significant main effect of delivery modality, confirming that tangibility alone does not guarantee superior engagement. However, primary analysis using multilevel linear models (MLM) revealed a reliable moderation: participants high in NARS experienced markedly lower narrative immersion under tangible delivery, whereas those with low NARS scores showed no such decrement. Qualitative findings further illuminate this divergence: tangibility provides novelty and engagement for some but imposes excessive proxemic friction for anxious users, for whom the digital interface acts as a protective social buffer. These results advance a conditional model of HRI and emphasize the necessity for adaptive systems that can tailor interaction modalities to user predispositions.
Read the full abstract ▾
All authors Wen Chen, Rongxi Chen, Shankai Chen, Huiyang Gong, Minghui Guo, Yingri Xu, Xintong Wu, Xinyi Fu
Robotics
UniGround: Universal 3D Visual Grounding via Training-Free Scene Parsing
Jiaxi Zhang, Yunheng Wang, Wei Lu et al. (+8 more)
3D Visual Grounding (3DVG) localizes objects from natural-
language descriptions in 3D scenes and is fundamental to
embodied AI applications. Although foundation models enable open-vocabulary
reasoning, they typically rely on pre-generated candidates, creating two sequential bottlenecks. The candidate bottleneck occurs when
dataset-specific 3D proposal models miss, fragment, or incorrectly group targets under distribution shifts, excluding them from VLM
reasoning. The evidence bottleneck stems from incomplete visual evidence: global renderings preserve spatial context but obscure object details, whereas candidate-centric views capture local appearance but lack global context. To address these bottlenecks, we propose UniGround, a zero-shot 3DVG framework that addresses both bottlenecks through Global Candidate Filtering and Contextual Precision Grounding. Global Candidate Filtering constructs
topology-consistent, class-agnostic candidates from 3D
topology and multi-view semantic cues, without
dataset-trained 3D detectors, task-specific proposal supervision, or predefined box and category priors. Contextual Precision Grounding jointly reasons over global spatial context and candidate-centric visual evidence, followed by closed-loop consistency verification for reliable target identification. UniGround achieves 46.1%/34.1% Acc@0.25/0.5 on ScanRefer and 28.7% Acc@0.25 on the evaluated ARKitScenes subset of EmbodiedScan. Further experiments demonstrate competitive grounding without
dataset-specific 3D priors, cross-
dataset generalization to unseen indoor scenes, and robustness to real-world reconstruction noise and practical domain shifts.
Read the full abstract ▾
All authors Jiaxi Zhang, Yunheng Wang, Wei Lu, Taowen Wang, Shuning Zhang, Yixiao Feng, Junzhe Xu, Shunwang Sun, Weisheng Xu, Yuetong Fang, Renjing Xu
Robotics
TRIAGE: Type-Routed Interventions via Aleatoric-Epistemic Gated Estimation in Robotic Manipulation and Adaptive Perception – Don't Treat All Uncertainty the Same
Divake Kumar, Sina Tayebati, Devashri Naik et al. (+4 more)
Most uncertainty-aware robotic systems collapse prediction uncertainty into a single scalar score and use it to trigger uniform corrective responses. This aggregation obscures whether uncertainty arises from corrupted observations or from mismatch between the learned model and the true system dynamics. As a result, corrective actions may be applied to the wrong component of the closed loop, degrading performance relative to leaving the policy unchanged. We introduce a lightweight post hoc framework that decomposes uncertainty into aleatoric and epistemic components and uses these signals to regulate system responses at inference time. Aleatoric uncertainty is estimated from deviations in the observation distribution using a Mahalanobis density model, while epistemic uncertainty is detected using a noise robust forward dynamics ensemble that isolates model mismatch from measurement corruption. The two signals remain empirically near orthogonal during closed loop execution and enable type specific responses. High aleatoric uncertainty triggers observation recovery, while high epistemic uncertainty moderates control actions. The same signals also regulate adaptive perception by guiding model capacity selection during tracking inference. Experiments demonstrate consistent improvements across both control and perception tasks. In robotic manipulation, the decomposed controller improves task success from 59.4% to 80.4% under compound perturbations and outperforms a combined uncertainty baseline by up to 21.0%. In adaptive tracking inference on MOT17, uncertainty-guided model selection reduces average compute by 58.2% relative to a fixed high capacity detector while preserving detection quality within 0.4%. Code and demo
videos are available at https://divake.github.io/uncertainty-decomposition/.
Read the full abstract ▾
All authors Divake Kumar, Sina Tayebati, Devashri Naik, Patrick Poggi, Amanda Sofie Rios, Nilesh Ahuja, Amit Ranjan Trivedi
Robotics
SaiVLA-0: Cerebrum–Pons–Cerebellum Tripartite Architecture for Compute-Aware Vision-Language-Action
Xiang Shi, Wenlong Huang, Menglin Zou et al. (+1 more)
We revisit
Vision-
Language-Action through a neuroscience-inspired triad. Biologically, the Cerebrum provides stable high-level
multimodal priors and remains frozen; the Pons Adapter integrates these cortical features with real-time proprioceptive inputs and compiles intent into execution-ready
tokens; and the Cerebellum (ParaCAT) performs fast, parallel categorical decoding for online control, with hysteresis/EMA/temperature/
entropy for stability. A fixed-ratio schedule and two-stage feature caching make the system compute-aware and reproducible. Inspired by active, foveated
vision, our wrist ROIs are geometrically tied to the end-effector via calibrated projection, providing a movement-stabilized, high-resolution view that is sensitive to fine-grained pose changes and complements the global context of the main view. The design is modular: upgrading the Cerebrum only retrains the Pons; changing
robots only trains the Cerebellum; cerebellum-only RL can further refine control without touching high-level semantics. As a concept-and-protocol paper with preliminary evidence, we outline a timing protocol under matched conditions (GPU, resolution, batch) to verify anticipated efficiency gains. We also report preliminary LIBERO evidence showing that split feature caching reduces training time (7.5h to 4.5h) and improves average success (86.5% to 92.5%) under official N1.5 head-only training, and that SaiVLA0 reaches 99.0% mean success.
Read the full abstract ▾
All authors Xiang Shi, Wenlong Huang, Menglin Zou, Xinhai Sun
Robotics
Towards Human-Like Manipulation through RL-Augmented Teleoperation and Mixture-of-Dexterous-Experts VLA
Tutian Tang, Xingyu Ji, Wanli Xing et al. (+7 more)
While
Vision-
Language-Action (VLA) models have demonstrated remarkable success in robotic manipulation, their application has largely been confined to low-degree-of-freedom end-effectors performing simple,
vision-guided pick-and-place tasks. Extending these models to human-like, bimanual dexterous manipulation-specifically contact-rich in-hand operations-introduces critical challenges in high-fidelity data acquisition, multi-skill learning, and
multimodal sensory fusion. In this paper, we propose an integrated framework to address these bottlenecks, built upon two components. First, we introduce IMCopilot (In-hand Manipulation Copilot), a suite of reinforcement learning-trained atomic skills that plays a dual role: it acts as a shared-autonomy assistant to simplify teleoperation data collection, and it serves as a callable low-level execution primitive for the VLA. Second, we present MoDE-VLA (Mixture-of-Dexterous-Experts VLA), an architecture that seamlessly integrates heterogeneous force and tactile modalities into a pretrained VLA backbone. By utilizing a residual injection mechanism, MoDE-VLA enables contact-aware refinement without degrading the model's pretrained knowledge. We validate our approach on four tasks of escalating complexity, demonstrating doubled success rate improvement over the baseline in dexterous contact-rich tasks.
Read the full abstract ▾
All authors Tutian Tang, Xingyu Ji, Wanli Xing, Ce Hao, Wenqiang Xu, Lin Shao, Cewu Lu, Qiaojun Yu, Jiangmiao Pang, Kaifeng Zhang
Robotics
DeReCo: Decoupling Representation and Coordination Learning for Object-Adaptive Decentralized Multi-Robot Cooperative Grasping and Lifting
Kazuki Shibata, Ryosuke Sota, Shandil Dhiresh Bosch et al. (+3 more)
Generalizing decentralized multi-
robot cooperative grasping and lifting across objects with diverse shapes and physical properties remains a fundamental challenge. Under decentralized execution, two key challenges arise: object-dependent representation learning under partial observability and coordination learning in multi-
agent reinforcement learning (MARL) under non-stationarity. A typical approach jointly optimizes object-dependent representations and coordinated policies in an end-to-end manner while randomizing object shapes and physical properties during training. However, this joint optimization tightly couples representation and coordination learning, which can introduce bidirectional interference: inaccurate representations under partial observability can degrade coordination learning, while non-stationarity in MARL can further degrade representation learning, leading to degraded task performance. Inspired by prior work on decoupling representation and control learning in single-
robot settings, we propose DeReCo, a MARL framework that decouples representation and coordination learning for object-adaptive decentralized multi-
robot cooperative grasping and lifting. DeReCo adopts a two-phase training strategy: (1) coordination learning with privileged object information, and (2) transition to decentralized execution through reconstruction of object-dependent representations from local observations and retraining of coordination policies using the reconstructed representations. Experimental results show that DeReCo outperforms baselines in simulation on three training objects, generalizes to six unseen objects with varying masses and friction coefficients under similar grasping configurations, and achieves superior performance on two unseen objects in real-
robot experiments.
Read the full abstract ▾
All authors Kazuki Shibata, Ryosuke Sota, Shandil Dhiresh Bosch, Yuki Kadokawa, Yoshihisa Tsurumine, Takamitsu Matsubara
Robotics
Adaptive Vision-Based Control of Redundant Robots with Null-Space Interaction for Human-Robot Collaboration
Xiangjie Yan, Chen Chen and Xiang Li
Human-
robot collaboration aims to extend human ability through cooperation with
robots. This technology is currently helping people with physical disabilities, has transformed the manufacturing process of companies, improved
surgical performance, and will likely revolutionize the daily lives of everyone in the future. Being able to enhance the performance of both sides, such that human-
robot collaboration outperforms a single
robot/human, remains an open issue. For safer and more effective collaboration, a new control scheme has been proposed for redundant
robots in this paper, consisting of an adaptive
vision-based control term in task space and an interactive control term in null space. Such a formulation allows the
robot to autonomously carry out tasks in an unknown environment without prior calibration while also interacting with humans to deal with unforeseen changes (e.g., potential collision, temporary needs) under the redundant configuration. The decoupling between task space and null space helps to explore the collaboration safely and effectively without affecting the main task of the
robot end-effector. The stability of the closed-loop system has been rigorously proved with Lyapunov methods, and both the convergence of the position error in task space and that of the damping model in null space are guaranteed. The experimental results of a
robot manipulator guided with the technology of augmented reality (AR) are presented to illustrate the performance of the control scheme.
Read the full abstract ▾
Robotics · Cross-listed from cs.HC
MRDrive: An Open Source Mixed Reality Driving Simulator for Automotive User Research
Patrick Ebel, Michał Patryk Miazga, Martin Lorenz et al. (+3 more)
Designing and evaluating in-vehicle interfaces requires experimental platforms that combine ecological validity with experimental control. Driving simulators are widely used for this purpose. However, they face a fundamental trade-off: high-fidelity physical simulators are costly and difficult to adapt, while virtual reality simulators provide flexibility at the expense of physical interaction with the vehicle. In this work, we present MRDrive, an open mixed-reality driving simulator designed to support HCI research on in-vehicle interaction,
attention, and explainability in manual and automated driving contexts. MRDrive enables drivers and passengers to interact with a real vehicle cabin while being fully immersed in a virtual driving environment. We demonstrate the capabilities of MRDrive through a small pilot study that illustrates how the simulator can be used to collect and analyze eye-tracking and touch interaction data in an automated driving scenario. MRDRive is available at: https://github.com/ciao-group/mrdrive
Read the full abstract ▾
All authors Patrick Ebel, Michał Patryk Miazga, Martin Lorenz, Timur Getselev, Pavlo Bazilinskyy, Celine Conzen
Robotics
See and Switch: Vision-Based Branching for Interactive Robot-Skill Programming
Petr Vanc, Jan Kristof Behrens, Václav Hlaváč et al. (+1 more)
Programming by demonstration (PbD) makes
robot programming accessible to non-experts, but scaling it to real-world variability remains a challenge for current teaching frameworks, especially when a
robot must select suitable task variants online from visual input. We present See & Switch, an interactive teaching-and-execution framework that represents tasks as graphs of skill parts connected by decision states, enabling conditional branching during replay. Its
vision-based Switcher uses eye-in-hand images to select the appropriate successor skill part and detect novel situations that require new demonstrations. The framework supports recovery demonstrations during execution through kinesthetic teaching, joystick control, and hand gestures. We evaluate See & Switch on three dexterous manipulation tasks with 8 novice users, collecting approx. 900 real-
robot execution rollouts. To isolate visual decision performance from timing errors during decision states, we evaluate the Switcher offline using user-gated decision state windows. In the evaluation within the decision state windows, the method achieves up to 90.6% branch-selection accuracy and detects anomalies with >90% accuracy in 47 of 79 decision states, demonstrating reliable switching based on visual input for conditional
robot-skill programming. We provide all code and experiment data at http://imitrob.ciirc.cvut.cz/publications/seeandswitch.
Read the full abstract ▾
All authors Petr Vanc, Jan Kristof Behrens, Václav Hlaváč, Karla Stepanova
Robotics · Cross-listed from eess.SY
Trajectory Tracking Control Design for Autonomous Helicopters with Guaranteed Error Bounds
Philipp Schitz, Johann C. Dauer and Paolo Mercorelli
This paper presents a systematic framework for computing formally guaranteed trajectory tracking error bounds for autonomous helicopters based on Robust Positive Invariant (RPI) sets. The approach establishes a closed-loop translational error dynamics which is cast into polytopic linear parameter-varying form with bounded additive and state-dependent disturbances. Ellipsoidal RPI sets are computed, yielding explicit position error bounds suitable as certified buffer zones in upper-level trajectory planning. Three controller architectures are compared with respect to the conservatism of their error bounds and tracking performance. Simulation results on a nonlinear helicopter model demonstrate that all architectures satisfy the derived bounds, while highlighting trade-offs between performance and the conservatism of the computed invariant set.
Read the full abstract ▾
Robotics
AffordGrasp: Cross-Modal Diffusion for Affordance-Aware Grasp Synthesis
Xiaofei Wu, Yi Zhang, Yumeng Liu et al. (+3 more)
Generating human grasping poses that accurately reflect both object geometry and user-specified interaction semantics is essential for natural hand-object interactions in AR/VR and
embodied AI. However, existing semantic grasping approaches struggle with the large modality gap between 3D object representations and textual instructions, and often lack explicit spatial or semantic constraints, leading to physically invalid or semantically inconsistent grasps. In this work, we present AffordGrasp, a
diffusion-based framework that produces physically stable and semantically faithful human grasps with high precision. We first introduce a scalable annotation pipeline that automatically enriches hand-object interaction
datasets with fine-grained structured
language labels capturing interaction intent. Building upon these annotations, AffordGrasp integrates an affordance-aware latent representation of hand poses with a dual-conditioning
diffusion process, enabling the model to jointly reason over object geometry, spatial affordances, and instruction semantics. A distribution adjustment module further enforces physical contact consistency and semantic
alignment. We evaluate AffordGrasp across four instruction-augmented
benchmarks derived from HO-3D, OakInk, GRAB, and AffordPose, and observe substantial improvements over state-of-the-art methods in grasp quality, semantic accuracy, and diversity.
Read the full abstract ▾
All authors Xiaofei Wu, Yi Zhang, Yumeng Liu, Yuexin Ma, Yujiao Shi, Xuming He
Robotics
Vector Field Augmented Differentiable Policy Learning for Vision-Based Drone Racing
Yang Su, Feng Yu, Yu Hu et al. (+4 more)
Autonomous drone racing in complex environments requires agile, high-speed flight while maintaining reliable obstacle avoidance. Differentiable-physics-based policy learning has recently demonstrated high sample efficiency and remarkable performance across various tasks, including agile drone flight and quadruped locomotion. However, applying such methods to drone racing remains difficult, as key objective like gate traversal are inherently hard to express as smooth, differentiable losses. To address these challenges, we propose DiffRacing, a novel vector field-augmented differentiable policy learning framework. DiffRacing integrates differentiable losses and vector fields into the training process to provide continuous and stable gradient signals, balancing obstacle avoidance and high-speed gate traversal. In addition, a differentiable Delta Action Model compensates for dynamics mismatch, enabling efficient sim-to-real transfer without explicit system identification. Extensive simulation and real-world experiments demonstrate that DiffRacing achieves superior sample efficiency, faster convergence, and robust flight performance, thereby demonstrating that vector fields can augment traditional gradient-based policy learning with a task-specific
geometric prior.
Read the full abstract ▾
All authors Yang Su, Feng Yu, Yu Hu, Xinze Niu, Linzuo Zhang, Fangyu Sun, Danping Zou
Robotics
Dual-Horizon Hybrid Internal Model for Low-Gravity Quadrupedal Jumping with Hardware-in-the-Loop Validation
Haozhe Xu, Yifei Zhao, Wenhao Feng et al. (+6 more)
Locomotion under reduced
gravity is commonly realized through jumping, yet continuous pronking in lunar
gravity remains challenging due to prolonged flight phases and sparse ground contact. The extended aerial duration increases landing impact sensitivity and makes stable attitude regulation over rough planetary terrain difficult. Existing approaches primarily address single jumps on flat surfaces and lack both continuous-terrain solutions and realistic hardware validation. This work presents a Dual-Horizon Hybrid Internal Model for continuous quadrupedal jumping under lunar
gravity using proprioceptive sensing only. Two temporal encoders capture complementary time scales: a short-horizon branch models rapid vertical dynamics with explicit vertical velocity estimation, while a long-horizon branch models horizontal motion trends and center-of-mass height
evolution across the jump cycle. The fused representation enables stable and continuous jumping under extended aerial phases characteristic of lunar
gravity. To provide hardware-in-the-loop validation, we develop the
MATRIX (Mixed-reality Adaptive Testbed for Robotic Integrated eXploration) platform, a digital-twin-driven system that offloads
gravity through a pulley-counterweight mechanism and maps Unreal Engine lunar terrain to a motion platform and treadmill in real time. Using
MATRIX, we demonstrate continuous jumping of a quadruped
robot under lunar-
gravity emulation across cratered lunar-like terrain.
Read the full abstract ▾
All authors Haozhe Xu, Yifei Zhao, Wenhao Feng, Zhipeng Wang, Hongrui Sang, Cheng Cheng, Xiuxian Li, Zhen Yin, Bin He
Robotics
Drag-Aware Aerodynamic Manipulability for Torque-Limited Redundant Multirotors: Aerodynamic Promptness based on the Symmetric Acceleration Capacity
Antonio Franchi
Aerodynamic promptness quantifies how rotor-speed variations generate multirotor wrench variations, but its Euclidean formulation assigns the same local cost to a given rotor acceleration at every operating speed. This work develops a capacity-aware extension for redundant multirotors with arbitrary numbers of heterogeneous rotors and wrench components. Under bounded motor torque and aerodynamic drag, each generally asymmetric instantaneous rotor-acceleration interval contains a largest zero-centered subset whose radius defines the symmetric acceleration capacity (SAC). The SAC induces a Riemannian metric on the positive-capacity rotor-speed region. Propagating its co-metric through the nonlinear rotor-speed-to-wrench differential yields a state-attached task-rate capability
matrix and ellipsoid. The corresponding inverse quadratic form equals the minimum normalized rotor-acceleration effort required to realize a prescribed wrench rate, while the ellipsoid volume defines the drag-aware aerodynamic manipulability (DAAM) index. Fiberwise DAAM maximization provides a task-coordinate-invariant criterion for selecting task-equivalent actuator states; its maximizing set exists on compact regular domains and can be nonconvex and set valued. Low-dimensional two- and three-rotor studies make the resulting fiberwise geometry and parameter dependence directly visible. A complementary two-rotor use case shows how DAAM can inform a continuous allocation section subject to directional motor-torque feasibility. For two heterogeneous propulsion systems, the resulting sections reduce saturation-induced force-tracking degradation relative to the pseudoinverse in the faster command bands
Read the full abstract ▾
Robotics · Cross-listed from cs.CV
TeamHOI: Learning a Unified Policy for Cooperative Human-Object Interactions with Any Team Size
Stefan Lionar and Gim Hee Lee
Physics-based
humanoid control has achieved remarkable progress in enabling realistic and high-performing single-
agent behaviors, yet extending these capabilities to cooperative human-object interaction (HOI) remains challenging. We present TeamHOI, a framework that enables a single decentralized policy to handle cooperative HOIs across any number of cooperating
agents. Each
agent operates using local observations while attending to other teammates through a
Transformer-based policy network with teammate
tokens, allowing scalable coordination across variable team sizes. To enforce motion realism while addressing the scarcity of cooperative HOI data, we further introduce a masked
Adversarial Motion Prior (AMP) strategy that uses single-human reference motions while masking object-interacting body parts during training. The masked regions are then guided through task rewards to produce diverse and physically plausible cooperative behaviors. We evaluate TeamHOI on a challenging cooperative carrying task involving two to eight
humanoid agents and varied object geometries. Finally, to promote stable carrying, we design a team-size- and shape-agnostic formation reward. TeamHOI achieves high success rates and demonstrates coherent cooperation across diverse configurations with a single policy.
Read the full abstract ▾
Robotics
VORL-EXPLORE: A Hybrid Learning Planning Approach to Multi-Robot Exploration in Dynamic Environments
Ning Liu, Sen Shen, Zheng Li et al. (+4 more)
Hierarchical multi-
robot exploration decouples frontier allocation from local
navigation, making systems brittle in dynamic environments. Without awareness of execution difficulty, allocators may send
robots toward frontiers whose routes are congested or unsafe, causing bottlenecks, oscillatory replanning, and redundant coverage. We propose VORL-EXPLORE, a hybrid learning and planning framework that makes exploration execution-aware through execution fidelity, an estimate of local navigability and execution reliability. This fidelity score is used to penalize risky or crowded frontiers in a fidelity-coupled Voronoi allocation objective with inter-
robot repulsion, and to arbitrate between global $A^*$ guidance and a reactive reinforcement learning policy during
navigation. As a result,
robots follow efficient global plans in reliable regions and rely more on safer local reactions in crowded or changing areas. The fidelity model is recalibrated using self-supervised pseudo-labels derived from recent progress and safety outcomes. Quantitative experiments on randomized dynamic grids and an ablation under severe traffic show higher success rates, shorter exploration lengths, and lower inter-
robot coverage overlap. A Gazebo factory scenario is included as a deployment illustration. The source code is available at https://github.com/21ning/VORL-EXPLORE.
Read the full abstract ▾
All authors Ning Liu, Sen Shen, Zheng Li, Sheng Liu, Dongkun Han, Shangke Lyu, Thomas Bräunl
Robotics · Cross-listed from cs.DC
ActECC: Action Chunk Temporal Harnessing for Edge-Cloud Collaborative VLA Inference System
Zihao Zheng, Zhihao Mao, Hui Huang et al. (+9 more)
VLA models have emerged as the mainstream for
embodied intelligence, which decouples
reasoning and execution via action chunks. To enable real-time deployment of VLA models, existing studies investigate edge-cloud collaborative inference, which pose two challenges to the action chunk mechanism. (1) Edge-cloud collaboration complicates system latency components, hindering the determination of valid ranges for action chunks. (2) Latency fluctuations in cross-edge update links under edge-cloud scenarios make it difficult to schedule replanning requests. To address these, we analyze the key factors affecting action chunk usable ranges and replanning request timing, revealing phase dependence and edge-cloud latency constraints. Based on these insights, we propose ActECC, which enables Action Chunk Temporal Harnessing through kinematics-guided usable range assessment and delay-aware replanning scheduling. Experiments show that ActECC reduces per-action latency by up to 27.4% and cloud calls by up to 45.1%, while lowering average execution error compared with existing edge-cloud collaborative.
Read the full abstract ▾
All authors Zihao Zheng, Zhihao Mao, Hui Huang, Sicheng Tian, Hangyu Cao, Ziyun Zhang, Chenyue Li, Jiayu Chen, Maoliang Li, Hailong Zou, Guojie Luo, Xiang Chen
Robotics
Unified Structural-Hydrodynamic Modeling of Underwater Underactuated Mechanisms and Soft Robots
Chenrui Zhang, Yiyuan Zhang, Yunfei Ye et al. (+3 more)
Underwater robots are widely deployed for ocean exploration and manipulation. Underactuated mechanisms are advantageous in aquatic environments because reducing actuator count lowers motor-leakage risk while introducing inherent mechanical compliance. However, accurate modeling of
underwater underactuated and soft robotic systems remains challenging, as it requires identifying high-dimensional structural and hydrodynamic parameters. In this work, we propose a trajectory-driven global optimization framework for unified structural-hydrodynamic modeling of
underwater multibody systems. Inspired by the Covariance
Matrix Adaptation
Evolution Strategy (CMA-ES), the proposed approach simultaneously identifies coupled elastic, damping, and distributed hydrodynamic parameters through trajectory-level matching between simulated and experimental motion. This enables high-fidelity reproduction of underactuated mechanisms and compliant soft robotic systems in
underwater environments, using as little as a single
video recording. We first validate the framework on a link-by-link underactuated multibody mechanism, demonstrating accurate identification of distributed hydrodynamic coefficients, with normalized end-effector position error below 5% across multiple trajectories, initial conditions, and both active-passive and fully passive configurations. The modeling strategy is further validated on an asymmetric octopus-inspired soft arm, confirming its effectiveness for compliant soft robotic systems. Finally, eight identified arms are assembled into a swimming octopus
robot, where the unified parameter set enables realistic whole-body behavior without additional retuning. These results demonstrate the scalability and transferability of the proposed structural-hydrodynamic modeling framework across
underwater underactuated and soft robotic systems.
Read the full abstract ▾
All authors Chenrui Zhang, Yiyuan Zhang, Yunfei Ye, Junkai Chen, Haozhe Wang, Cecilia Laschi
Robotics
Omnidirectional Humanoid Locomotion on Stairs via Unsafe Stepping Penalty and Sparse LiDAR Elevation Mapping
Yuzhi Jiang, Yujun Liang, Junhao Li et al. (+2 more)
Humanoid robots, characterized by numerous degrees of freedom and a high center of
gravity, are inherently unstable. Safe omnidirectional locomotion on stairs requires both omnidirectional terrain perception and reliable foothold selection. Existing methods often rely on forward-facing depth cameras, which create blind zones that restrict omnidirectional mobility. Furthermore, sparse post-contact unsafe stepping penalties lead to low learning efficiency and suboptimal strategies. To realize safe stair-traversal gaits, this paper introduces a single-stage training framework incorporating a dense unsafe stepping penalty that provides continuous feedback as the foot approaches a hazardous placement. To obtain stable and reliable elevation maps, we build a rolling point-cloud mapping system with spatiotemporal confidence decay and a self-protection zone mechanism, producing temporally consistent local maps. These maps are further refined by an Edge-Guided Asymmetric U-Net (EGAU), which mitigates reconstruction distortion caused by sparse
LiDAR returns on stair risers. Simulation and real-
robot experiments show that the proposed method achieves a near-100% safe stepping rate on stair terrains in simulation, while maintaining a remarkably high safe stepping rate in real-world deployments. Furthermore, it completes a continuous long-distance walking test on complex outdoor terrains, demonstrating reliable sim-to-real transfer and long-term stability.
Read the full abstract ▾
All authors Yuzhi Jiang, Yujun Liang, Junhao Li, Han Ding, Lijun Zhu
Robotics
Long-Short Term Agents for Pure-Vision Bronchoscopy Robotic Autonomy
Junyang Wu, Mingyi Luo, Fangfang Xie et al. (+7 more)
Accurate intraoperative
navigation is essential for
robot-assisted endoluminal intervention, but remains difficult because of limited endoscopic field of view and dynamic artifacts. Existing
navigation platforms often rely on external localization technologies, such as electromagnetic tracking or shape sensing, which increase hardware complexity and remain vulnerable to intraoperative anatomical mismatch. We present a
vision-only autonomy framework that performs long-horizon bronchoscopic
navigation using preoperative CT-derived virtual targets and live endoscopic
video, without external tracking during
navigation. The framework uses hierarchical long-short
agents: a short-term reactive
agent for continuous low-latency motion control, and a long-term strategic
agent for decision support at anatomically ambiguous points. When their recommendations conflict, a world-model critic predicts future visual states for candidate actions and selects the action whose predicted state best matches the target view. We evaluated the system in a high-fidelity airway phantom, three ex vivo porcine lungs, and a live porcine model. The system reached all planned segmental targets in the phantom, maintained 80% success to the eighth generation ex vivo, and achieved in vivo
navigation performance comparable to the expert bronchoscopist. These results support the preclinical feasibility of sensor-free autonomous bronchoscopic
navigation.
Read the full abstract ▾
All authors Junyang Wu, Mingyi Luo, Fangfang Xie, Minghui Zhang, Hanxiao Zhang, Chunxi Zhang, Junhao Wang, Jiayuan Sun, Yun Gu, Guang-Zhong Yang
Robotics · Cross-listed from cs.LG
DyQ-VLA: Temporal-Dynamic-Aware Quantization for Embodied Vision-Language-Action Models
Zihao Zheng, Hangyu Cao, Chibang Tao et al. (+9 more)
Vision-
language-action (VLA) models achieve favorable task performance, yet runtime errors in closed-loop execution evolve with alternating updates of actions and observations. Existing VLA quantization methods mainly trade off inference speed and model performance while rarely investigating how quantization alters runtime errors. By comparing error distributions between full-precision and quantized models, we identify distinct evolutionary patterns for the two types of errors: quantization enlarges the variance of translation error distributions and increases their dispersion, whereas the distribution center of rotation errors gradually shifts across execution steps, demonstrating cumulative drift. Motivated by this observation, we rethink the optimal quantization strategy for VLA models and propose DyQ-VLA, a runtime-error-aware quantization framework, which dynamically selects activation precision according to execution steps and tracks as well as compensates rotation errors via accumulated quantization residuals. Corresponding operators and runtime adaptation strategies are devised within the framework to enable dynamic-precision execution. Experiments show that DyQ-VLA achieves a 1.88 to 1.93 inference speedups while maintaining comparable or higher average task success rates. Moreover, it reduces mean execution steps by 17.7% to 19.8%. Our code is here: https://anonymous.4open.science/r/DyQ-VLA-7F51/.
Read the full abstract ▾
All authors Zihao Zheng, Hangyu Cao, Chibang Tao, Zhihao Mao, Lingyue Zhang, Maoliang Li, Jiayu Chen, Hailong Zou, Xuanzhe Liu, Donggang Cao, Hong Mei, Xiang Chen
Robotics
NaviDriveVLM: Decoupling High-Level Reasoning and Motion Planning for Autonomous Driving
Ximeng Tao, Pardis Taghavi, Dimitar Filev et al. (+2 more)
Vision-
language models (VLMs) have emerged as a promising direction for end-to-end autonomous driving (AD) by jointly modeling visual observations, driving context, and
language-based
reasoning. However, existing VLM-based systems face a trade-off between high-level
reasoning and motion planning: large models offer strong semantic understanding but are costly to adapt for precise control, whereas small VLM models can be fine-tuned efficiently but often exhibit weaker
reasoning. We propose NaviDriveVLM, a decoupled framework that separates
reasoning from action generation using a large-scale Navigator and a lightweight trainable Driver. This design preserves
reasoning ability, reduces training cost, and provides an explicit interpretable intermediate representation for downstream planning. Experiments on the nuScenes
benchmark show that NaviDriveVLM outperforms large VLM baselines in end-to-end motion planning.
Read the full abstract ▾
All authors Ximeng Tao, Pardis Taghavi, Dimitar Filev, Reza Langari, Gaurav Pandey
Robotics
RoboRouter: Training-Free Policy Routing for Robotic Manipulation
Yiteng Chen, Zhe Cao, Hongjia Ren et al. (+9 more)
Research on robotic manipulation has developed a diverse set of policy paradigms, including
vision-
language-action (VLA) models,
vision-action (VA) policies, and code-based compositional approaches. Concrete policies typically attain high success rates on specific task distributions, but limited generalization beyond it. Rather than proposing another monolithic policy, we propose to leverage the complementary strengths of existing approaches through intelligent policy routing. We introduce RoboRouter, a training-free framework that maintains a pool of heterogeneous policies and learns to select the best-performing policy for each task through accumulated execution experience. Given a new task, RoboRouter constructs a semantic task representation, retrieves historical records of similar tasks, predicts the optimal policy choice without requiring trial-and-error, and incorporates structured feedback to refine subsequent routing decisions. Integrating a new policy into the system requires only a lightweight evaluation and does not incur training overhead. Across simulation
benchmark and real-world evaluations, RoboRouter consistently outperforms individual policies, improving the average success rate by more than 3% in simulation and 13% in real-world settings, while preserving execution efficiency. Our results demonstrate that intelligent routing across heterogeneous, off-the-shelf policies provides a practical and scalable pathway toward building more capable robotic systems.
Read the full abstract ▾
All authors Yiteng Chen, Zhe Cao, Hongjia Ren, Chenjie Yang, Wenbo Li, Shiyi Wang, Yemin Wang, Li Zhang, Yanming Shao, Zhenjun Zhao, Huiping Zhuang, Qingyao Wu
Robotics
Identifying Influential Actions in Human-Robot Interactions
Haoyang Jiang, Chenfei Xu, Yuya Okadome et al. (+1 more)
Human-
robot interaction combines robotics, cognitive science, and human factors to study collaborative systems. This paper introduces a method for identifying influential
robot actions using transfer
entropy, a statistic that measures directed information transfer between time series. TE is effective for capturing complex, nonlinear interactions. We apply this method to analyze how
robot actions affect human behavior during a conversation with a remotely controlled
robot avatar. By focusing on the impact of proximity, our approach demonstrates TE's capability to identify key actions influencing human responses, highlighting its potential to improve the design and adaptability of robotic systems.
Read the full abstract ▾
All authors Haoyang Jiang, Chenfei Xu, Yuya Okadome, Yukata Nakamura
Robotics
Foundation and Small Models Coordination for Visuomotor Policy Learning
Haoran Ding, Liang Ma, Yaxun Yang et al. (+7 more)
Visuomotor policy learning enables
robots to perform a wide range of tasks, but small policy models often remain sensitive to changes in object and background appearance. In this work, we investigate the coordination of pretrained
vision foundation models with small policy models to improve appearance generalization. We propose a framework in which a small policy model operates on task-relevant visual observations constructed through semantic repainting. A
segmentation foundation model identifies the
robot and target object, which are rendered with fixed role colors on a constant background. An alternative representation replaces the target's role color with normalized
monocular depth predicted by a depth foundation model, providing additional
geometric cues. The perception models are adapted using in-distribution data where needed and held fixed during policy training. This design combines the perceptual capabilities of foundation models with a small policy model trained on the resulting observations for action prediction. Evaluations with flow matching policies on simulation
benchmarks, together with experiments on two real-world robotic tasks, demonstrate substantial improvements in task success under the evaluated appearance shifts.
Read the full abstract ▾
All authors Haoran Ding, Liang Ma, Yaxun Yang, Wen Yang, Tianyu Liu, Xiaodan Liang, Dezhen Song, Ivan Laptev, Yoshihiko Nakamura, Anqing Duan
Robotics
Language-Guided Grasping under Partial Observation for Mobile Manipulation in Field Inspection and Maintenance
Dilermando Almeida, Juliano Negri, Guilherme Lazzarini et al. (+5 more)
Offshore inspection and maintenance have increasingly been using legged
robots for routine sensing, yet many useful interventions still require physical interaction with tools, containers, and task-relevant objects. Employing
robots for these tasks can reduce operators' exposure in confined, elevated, or potentially explosive areas. This paper presents a
language-guided grasping pipeline for a legged mobile manipulator operating under partial observation. An operator defines the target, the system grounds it in RGB with open-vocabulary detection and promptable
segmentation, extracts an object-centric RGB-D point cloud, improves sparse geometry through depth compensation and point-cloud completion, and selects a 6-DoF grasp using collision, clearance, reachability, and approach constraints. The system is implemented on a quadruped
robot with an arm and evaluated in two cluttered tabletop scenes motivated by small-object
retrieval during inspection and maintenance. Across paired trials, the proposed pipeline achieved 9/10 successful grasps, compared with 3/10 for a view-dependent deployment baseline. In this controlled setting, object-centric completion and execution-aware selection reduced approach collisions and improved the reliability of
language-guided grasping for supervised field manipulation.
Read the full abstract ▾
All authors Dilermando Almeida, Juliano Negri, Guilherme Lazzarini, Thiago H. Segreto, Ranulfo Bezerra, Gustavo J. G. Lahr, Ricardo V. Godoy, Marcelo Becker