Robotics
Context-Aware Force Estimation for Deformable Tool Manipulation in Robotic Environmental Swabbing via Few-Shot Continual Adaptation
Siavash Mahmoudi, Chaitainya Kuppar Reddy, Yang Tian et al. (+1 more)
Robotic surface swabbing requires sustained interaction between a compliant tool and heterogeneous environments, where accurate estimation of tip-level contact force is critical for consistent sampling performance. However, deformable tool dynamics introduce nonlinear viscoelastic hysteresis that decouples wrist-mounted force measurements from true contact forces, while tool-integrated sensors are impractical for deployment due to sterility and disposability constraints. This paper presents a data-driven framework for contact force estimation in Deformable Tool Manipulation (DTM) that leverages proprioceptive sensing without requiring explicit physical models or permanent embedded sensing hardware at the tool tip. A recurrent architecture is first identified through a comparative evaluation of temporal models, where a compact LSTM achieves the lowest estimation error and sub-millisecond inference latency. To address generalization across unseen surfaces and tool compliance conditions, we introduce a parameter-isolated few-shot adaptation strategy that augments a frozen recurrent backbone with low-dimensional context
embeddings using feature-wise linear modulation (FiLM). Experiments on a UR5e platform across nine tool-surface interaction regimes demonstrate that the proposed approach significantly improves robustness under domain shift, reducing zero-shot estimation error by up to 63% while preserving baseline performance without catastrophic forgetting. These results show that separating shared deformation-history dynamics from domain-specific conditioning enables reliable force estimation for DTM in non-stationary environments.
Read the full abstract ▾
All authors Siavash Mahmoudi, Chaitainya Kuppar Reddy, Yang Tian, Dongyi Wang
Robotics
SonoRank: Towards Calibration-Free Real-Time Finger Flexion Detection from Forearm Ultrasound Sequences
Dean Zadok, Alon Wolf, Alex M. Bronstein et al. (+1 more)
Powered prosthetic hands are frequently abandoned, largely due to the limited functionality of current devices that rely on surface electromyography (sEMG). Sonomyography (ultrasound) has emerged as a promising alternative, owing to its ability to observe muscle activity in real time and control a greater number of degrees of freedom. Yet, existing ultrasound-based methods require per-user fine-tuning, limiting their commercialization. We propose SonoRank, an important step towards calibration-free finger flexion detection from forearm ultrasound
video. SonoRank first learns to rank pairs of ultrasound sequences by their relative motion magnitude for each of the five fingers. The learned representations are then fine-tuned to classify whether each finger is actively flexing, using a rest reference that is captured at the beginning of the operation. Under 12-fold leave-one-subject-out cross-validation on a
dataset of twelve subjects with synchronized kinematics, SonoRank achieves a 28% improvement in F1 score over direct classification baselines that skip the ranking stage. These results establish pairwise ranking as an effective pretraining signal for subject-independent control, bringing ultrasound-based prosthetics closer to practical, calibration-free deployment.
Read the full abstract ▾
All authors Dean Zadok, Alon Wolf, Alex M. Bronstein, Oren Salzman
Robotics
Generating Personalized Lower-Limb Kinematics Across Walking Speeds Using Subject-Conditioned Diffusion
Diya Dinesh, Adrian Krieger, Changseob Song et al. (+3 more)
Personalizing exoskeleton assistance requires user-specific gait data across many locomotor tasks, yet collecting this data demands repeated motion capture sessions that are costly, time-intensive, and especially burdensome for clinical populations. This challenge is most acute across walking speeds, where gait changes substantially and deviates further in clinical gait. This work introduces a subject-conditioned residual
diffusion framework that generates personalized lower-limb kinematics at unseen walking speeds from a subject's gait sequence at a single seen speed. Given sagittal-plane hip, knee, and ankle trajectories at a seen speed and a desired unseen speed, the model generates a residual that transforms the seen trajectory into the unseen one, using a
transformer denoiser conditioned on the subject's gait and the two speeds through feature-wise linear modulation. Trained only on able-bodied data, the model achieved a mean absolute error (MAE) of 3.4{\deg} on held-out able-bodied subjects. Without any stroke-specific fine-tuning, it achieved a 6.0{\deg} MAE on out-of-training-distribution stroke subjects, retaining subject identity for clinical gait. The framework reduced the MAE by over 70% relative to supervised feed-forward baselines, and a single seen speed matched the accuracy of four speeds within 0.4{\deg}. These results demonstrate that subject-conditioned residual
diffusion can synthesize personalized gait across speeds from minimal data, reducing the collection burden for downstream exoskeleton personalization.
Read the full abstract ▾
All authors Diya Dinesh, Adrian Krieger, Changseob Song, Dongho Park, Aaron J. Young, Inseung Kang
Robotics
Smooth Operator: A Real-Time Sampling-Based Algorithm for Kinematic Hand Retargeting
Robert Jomar Malate, Erik Bauer, Norica Bacuieti et al. (+4 more)
Advances in learning-based robotic manipulation, such as
Vision-
Language-Action (VLA) models and
Video Action Models (VAMs), heavily rely on high-quality teleoperation data. Their capabilities are strictly upper-bounded by the quality of the underlying human demonstrations. Current gradient-based retargeting
algorithms often converge to different local minima, resulting in jitter that affects data quality and teleoperation experience. To address this, we introduce the Sampling-Based Retargeter (SBR), a novel gradient-free retargeting method drawn from the rich literature of sampling-based control and explicitly designed for low-jitter, real-time kinematic retargeting. We evaluate SBR both in simulation and through a rigorous real-world user study involving 18 participants performing 3 complex manipulation tasks. Compared to gradient-based baselines, SBR achieved the highest overall task success rate (54.1%) while significantly reducing operator cognitive fatigue, recording the lowest NASA-TLX workload score (36.4 out of 100). Ultimately, we establish SBR as a highly effective, intuitive retargeter for dexterous manipulation, providing the community with a rigorous benchmarking methodology to guide future retargeting research.
Read the full abstract ▾
All authors Robert Jomar Malate, Erik Bauer, Norica Bacuieti, Stefanos Charalambous, Elvis Nava, Robert K. Katzschmann, Benedek Forrai
Robotics
Agent-Exploitation Affordances: From Basic to Complex Representation Patterns
Bastien Dussard, Aurélie Clodic and Guillaume Sarthou
In robotics, the capability of an artificial
agent to represent the range of its action possibilities, i.e. affordances, is crucial to understand how it can act on its environment. While functional affordances, which refer to the use of tools and objects, have been broadly studied in knowledge representation, the implications of a social context and the presence of other
agents have remained unexplored in this field. Consequently, in the field of social robotics, a multi-
agent context enables the
agents to engage in new actions that are potentially complementary to their individual capabilities, leading to the perspective of agentexploitation. This work focuses on the concept of cooperative affordance within the realm of social affordances. Cooperative affordances refer to situations where
agents interact with each other to extend their action possibilities range. From this definition, this paper proposes a tractable ontological representation of this concept with the aim of making it usable by an artificial
agent. Expanding on those elementary patterns, we illustrate the effectiveness of these representations by combining them to depict a diverse range of scenarios.
Read the full abstract ▾
Robotics
EmbodiedGen V2: An Agentic, Simulation-Ready 3D World Engine for Embodied AI
Xinjie Wang, Liu Liu, Taojun Ding et al. (+9 more)
We present EmbodiedGen V2, a generative 3D world engine for building executable policy-ready environments for
embodied intelligence. Sim-ready 3D asset generation has advanced rapidly, yet assembling such assets into policy-ready task environments remains largely manual, limiting scalable closed-loop learning. EmbodiedGen V2 addresses this gap through a unified sim-ready representation that connects cross-simulator assets, interaction affordances, task-driven worlds, large-scale multi-room scenes, and stateful Vibe Coding into a generative, editable, and reusable simulation pipeline. The generated environments support manipulation,
navigation, mobile manipulation, cross-simulator deployment, and
embodied policy training. In evaluation, the asset pipeline achieves 96.5% human acceptance and 98.6% collision success, and 83.3% of task-driven worlds are directly usable for downstream simulation without manual modification. Online reinforcement learning with generated environments further improves simulation success from 9.7% to 79.8%, and transfers to real
robots with task success increasing from 21.7% to 75.0%. These results establish EmbodiedGen V2 as scalable simulation infrastructure for training, evaluating, and deploying
embodied policies.
Read the full abstract ▾
All authors Xinjie Wang, Liu Liu, Taojun Ding, Andrew Choi, Chaodong Huang, Mengao Zhao, Ziang Li, Jackson Jiang, Chunlei Yu, Shengxiang Liu, Wei Xu, Zhizhong Su
Robotics
GeoGS-SLAM: Geometry-Only Gaussian Splatting for Dense Monocular SLAM
Lipu Zhou, Yaoyun Kang, Junxiang Pang et al. (+3 more)
Dense visual SLAM is a fundamental problem in robotics. Recent advances in 3DGS have demonstrated its potential for dense SLAM. Existing 3DGS frameworks focus on both appearance and geometry modeling. However, scene geometry is typically more critical for SLAM than novel view synthesis because downstream robotic tasks, such as
navigation and obstacle avoidance, rely primarily on accurate spatial geometry rather than photorealistic rendering. This observation raises a natural question: Is it feasible for 3DGS to perform 3D reconstruction without scene appearance modeling? Motivated by this, we propose Geometry-only Gaussian Splatting (GeoGS), which directly reconstructs scene geometry, and further present GeoGS-SLAM, a dense visual SLAM system built upon this representation. Specifically, GeoGS retains only spatial parameters to reduce the number of per-primitive parameters by over 80%. In contrast to existing 3DGS methods, GeoGS focuses solely on
geometric reconstruction, which significantly reduces the number of Gaussian primitives, accelerates
geometric convergence, and enhances robustness to illumination variations. In addition, we present an effective training framework that optimizes the Gaussian primitives via single-view and multi-view
geometric and photometric supervision, and speeds up geometry convergence with a local-plane driven initialization that better aligns primitives with local structures. Furthermore, we introduce a map update strategy for loop closure that globally transforms the Gaussian map to align it with the corrected pose estimates, thereby preventing map tearing caused by inconsistent per-viewpoint pose corrections in existing methods. Extensive experiments on synthetic and real-world
benchmarks demonstrate that our method outperforms SOTA methods in terms of online mapping efficiency and
geometric reconstruction quality.
Read the full abstract ▾
All authors Lipu Zhou, Yaoyun Kang, Junxiang Pang, Shengkai Sun, Tingting Bao, Kehan Wang
Robotics
Immersive Social Interaction with VR and LLM-Assisted Humanoids
Niraj Pudasaini, Geeta Chandra Raju Bethala, Pranav Doma et al. (+2 more)
Humanoid robots can extend human presence to remote, constrained, or hazardous environments, but existing teleoperation interfaces often require physically demanding motion tracking or cognitively demanding low-level control. This paper presents an immersive teleoperation framework that integrates voice-controlled locomotion, VR-based manipulation, and bidirectional social interaction for whole-body
humanoid control. Using Apple
Vision Pro, the operator receives
egocentric visual feedback, issues natural-
language locomotion commands, and teleoperates the
robot's arms and dexterous hands through wrist and finger tracking. An LLM-assisted voice-control module converts spoken instructions into high-level locomotion commands, while the manipulation module retargets human hand motions to the
robot through inverse kinematics and PD control. The system also records
multimodal data, including
egocentric RGB observations, voice/text commands, joint states, hand motions, and eye-gaze signals, supporting future imitation learning and autonomy. We evaluate the framework on a Unitree H1
humanoid equipped with dexterous hands in manipulation and social interaction tasks. Results show that novice users can successfully operate the system after brief familiarization, achieving 80% success in object manipulation and 70% success in a social cube-passing task. These results demonstrate the potential of immersive,
language-assisted teleoperation as an accessible interface for
humanoid interaction, remote assistance, and
multimodal data collection.
Read the full abstract ▾
All authors Niraj Pudasaini, Geeta Chandra Raju Bethala, Pranav Doma, Anthony Tzes, Yi Fang
Robotics
Initiation Safety: A Missing Dimension in Generalist-Robot Safety
Zhijin Meng and Francisco Cruz
Safety for generalist
robots is usually discussed in terms of motion or dialogue. We argue a third question is missing: should the
robot take its first hard-to-undo social action at all, such as a greeting, an uninvited grasp, or stepping into someone's space? We call this initiation authorization. Current frameworks rarely treat it as a separate safety layer. Today's stacks often skip this step: a high engagement score or a confident VLA rollout is treated as permission to act. But seeing a person is not the same as having their consent to be addressed. We frame initiation authorization within generalist-
robot safety and contrast it with post-plan VLA guardrails, implementing PAS (probe-authorize-speak) on a doorway
humanoid, comparing it with direct-init on logged traces, and proposing a three-condition user study, with open questions on metrics, governance, and where initiation ends and foundation-model generation begins.
Read the full abstract ▾
Robotics · Cross-listed from cs.MA
Multi-Agent Robotic Control with Onboard Vision-Language Models
Kajetan Rachwał, Maciej Majek, Bartłomiej Boczek et al. (+6 more)
Vision Language Models (VLMs) and
Vision Language Action (VLA) models have shown promise in robotic control. Yet, they face significant challenges regarding explainability, generalization, and compute requirements. This paper presents a Multi-
Agent System (MAS) architecture that addresses these limitations by deploying specialized
agents on onboard hardware - eliminating dependence on external compute. The system controls a multi-purpose autonomous mobile manipulator in a simulated industrial warehouse, fulfilling five task categories: safety inspection, warehouse maintenance, warehouse search, package quality verification, and responding to human requests. Compact VLMs (3-20B parameters) are used throughout, with fine-tuning applied to improve package inspection accuracy. A novel "Megamind" orchestration
agent mitigates context retention issues inherent to long-horizon planning with smaller models. The system was validated in a hardware-in-the-loop simulation using an AMD Ryzen(TM) AI mini PC. Results demonstrate that a fully onboard MAS architecture is a viable, cost-efficient alternative to cloud-dependent deployments, with strong potential for real-world transfer. The simulation environment has been released as open source under the Apache 2.0 licence.
Read the full abstract ▾
All authors Kajetan Rachwał, Maciej Majek, Bartłomiej Boczek, Jakub Matejczyk, Dominik Matejkowski, Adam Dąbrowski, Tim Seyde, Alexander Amini, Maria Ganzha
Robotics
Communicative Efficiency of Single vs. Multi-Axis Robot Neck Motion
Chapa Sirithunge, Haewon Jeong, Qinghua Guan et al. (+2 more)
Nonverbal communication through head and neck movement is fundamental to human social signalling, yet how robotic neck morphology translates motion into communicative information remains poorly understood. We present an information-theoretic framework characterising
robot neck movement as a communication channel, quantifying information transmitted and energy expended across varied configurations. Using a robotic neck platform, we recorded 84
video stimuli spanning three rotational degrees of freedom (DoF), varying amplitude, acceleration, and frequency, measuring Shannon
entropy of pixel-change signals alongside energy consumption. A perceptual study validated communicative interpretations of each motion. While humans typically engage one axis per gesture,
robots are unconstrained by biological architecture, motivating tests up to 3 DoF. Yet communicative information peaks at two DoF and decreases at three despite rising energy cost, a phenomenon we term the morphological information bottleneck. Motion parameter effects were parameter-dependent, some additive, others non-linear. We introduce the Motor Information Space, a framework mapping
entropy against energy to expose communicative efficiency across morphologies, in which the optimal configuration achieves 5.26 bits at competitive energy cost. Perception data further confirm multi-axis movements reduce clarity. These findings challenge the assumption that anatomical completeness improves robotic expressiveness, establishing a quantitative basis for morphological design in
robots, especially
humanoids.
Read the full abstract ▾
All authors Chapa Sirithunge, Haewon Jeong, Qinghua Guan, Fumiya Iida, Josie Hughes
Robotics
PLED-VINS: A Point-Line Event-Based Visual Inertial SLAM for Dynamic Environments
Seunghun Lee, Jihun Nam, Dong-Uk Seo et al. (+1 more)
Dynamic environments remain a fundamental challenge for visual SLAM, where unreliable observations from moving objects and rapid motion degrade state estimation accuracy. Although event cameras preserve fine-grained spatio-temporal information, most existing event-based SLAM frameworks still assume static scenes and lack approaches to estimate the reliability of features. To this end, we propose PLED-VINS, a
monocular event camera-based visual-inertial SLAM framework that enables robust state estimation in dynamic environments. We propose an
entropy-recency score map to characterize the temporal reliability of both point and line features based on event temporal statistics. Concurrently,
geometric reliability is estimated via a unified point-line robust bundle adjustment. Building upon these, we design an adaptive weighting strategy that fuses temporal and
geometric reliability, including motion-conditioned reliability modeling for line features, to suppress unreliable observations. Experimental results demonstrate that PLED-VINS improves state estimation on the evaluated dynamic sequences with moving objects.
Read the full abstract ▾
All authors Seunghun Lee, Jihun Nam, Dong-Uk Seo, Hyun Myung
Robotics
Behavior Foundations for Quadruped Robots: ABot-C0 Technical Report
Xufeng Zhao, Fuzhi Yang, Jianhui Chen et al. (+17 more)
The motion controller is one of the most fundamental modules in
embodied intelligence systems. Driven by large-scale human motion-capture data and the motion-tracking paradigm,
humanoid control has achieved remarkable progress in recent years. However, migrating this recipe to the quadrupedal setting is far less straightforward: animal motion data is scarcer and harder to capture at scale than human data, and cross-embodiment retargeting remains fragile. We present ABot-C0, a generalist motion-control system for quadruped
robots that establishes three complementary behavior foundations: a scalable multi-source motion-data pipeline, robust policy learning across motion tracking, locomotion, and scene interaction, and a unified deployment stack for reliable real-world operation. Fundamentally, we construct a data pyramid through conditional
video-generation synthesis, annotated motion capture, teleoperation, and human design, producing 16,074 physically feasible motion clips as the data foundation for diverse motion-learning demands. With large-scale motion data, a Flow-Matching generalist policy demonstrates, for the first time, a scaling law for quadruped motion tracking: performance improves consistently as training scales up, with zero-shot capability to track unseen motions. We then go a step further toward robust all-terrain locomotion by adopting a three-stage privileged-to-perceptive framework with temporal
LiDAR memory and terrain-predictive supervision. Collectively, these components form a motion generalist that coordinates multi-policy execution, smooth behavior transitions, energy-efficient control, and safety mechanisms for real-world deployment. Extensive experiments on urban-terrain autonomous
navigation and companion-style
multimodal interaction demonstrate that quadruped
robots can move beyond functional demos toward product-level behavioral intelligence.
Read the full abstract ▾
All authors Xufeng Zhao, Fuzhi Yang, Jianhui Chen, Li Gao, Zhang Meng, Jie Gao, Yao Zheng, Congyang Zhao, Tianxiong Lv, Menglin Yang, Minqi Gu, Yaru Zhao, Wenyu Liu, Honglin Han, Shihui Su, Zixiao Tang, Liu Liu, Mu Xu, Yang Cai, Wenbin Tang
Robotics
HumAIN: Human-Aware Implicit Social Robot Navigation
Daeun Song, Nhat Le, Jeffrey Chen et al. (+7 more)
Effective social
robot navigation requires sensitivity to human behavior, often revealed through subtle skeletal cues like gait and orientation. We present Human-Aware Implicit Social
Robot Navigation (HumAIN), a novel framework that fuses implicit social cues directly into the planning loop via knowledge
distillation. We first employ a
transformer-based teacher model that fuses rich multi-modal inputs, including historic images, skeletal keypoints,
robot state, and a
robot's target goal, to learn robust, human-aware representations for the
robot's future trajectory planning. To enable real-time deployment, we then distill this knowledge into a lightweight student model. By optimizing for both trajectory reconstruction and latent feature
alignment with the teacher, the student learns to infer complex social dynamics from minimal inputs. Bridging the prediction-planning gap with an efficient distilled architecture, our method enables
robots to reason about human behavior in a manner that is adaptive, robust, and socially compliant. We validate HumAIN through extensive experiments, where it improves trajectory prediction metrics by an average of 29.8% across all metrics compared to state-of-the-art baselines. These results highlight the benefit of using implicit, whole-body cues to achieve human-like
navigation awareness on resource-constrained platforms.
Read the full abstract ▾
All authors Daeun Song, Nhat Le, Jeffrey Chen, Mohammad Nazeri, Amirreza Payandeh, Rohan Chandra, Reuth Mirsky, Ross Mead, Ling Xiao, Xuesu Xiao
Robotics
Multimodal Voice Activity Projection for Turn-Taking in Social Robots with Voice-Activity-Related Pretrained Encoders
Antonio Cano, Guillermo Pérez, Luis Merino et al. (+1 more)
Turn-taking prediction is a key requirement for social
robots involved in human-human interaction, particularly in mediator settings, where the
robot must anticipate conversational dynamics rather than merely react to pauses. This work presents a
Multimodal Voice Activity Projection (MM-VAP) framework that extends the original audio-only VAP formulation to synchronized audio-visual inputs while preserving its self-supervised future-projection objective. The proposed approach builds on pretrained audio-visual backbones originally optimized for
speech-related tasks and adapts them through Low-Rank Adaptation to the
multimodal turn-taking problem. After independent speaker encoding, an inter-speaker
attention stage models the relational dynamics required to project future voice activity. In addition, a semantic consistency loss is introduced to regularize the 256-state output space according to higher-level dialogue activity patterns. Experiments on NoXi and NoXi+J showed improvements over the current baselines, particularly for some turn-taking events. Additional evaluation on the Haru EDR corpus further supported the suitability of this direction for mediation-oriented human-
robot interaction.
Read the full abstract ▾
All authors Antonio Cano, Guillermo Pérez, Luis Merino, Randy Gomez
Robotics
TouchWorld: A Predictive and Reactive Tactile Foundation Model for Dexterous Manipulation
Jianyi Zhou, Feiyang Hong, Yunhao Li et al. (+9 more)
Dexterous manipulation in everyday environments requires both anticipation and reaction: a
robot must predict how contact should evolve while rapidly correcting local errors caused by slip, misalignment, unstable grasping, or force mismatch.
Vision and
language provide semantic and
geometric guidance, but they cannot reliably reveal hidden contact states such as force, slip, and contact stability. Although tactile sensing exposes these physical cues, most existing policies treat touch as a low-frequency observation stream within a monolithic action model, coupling slow task
reasoning, action generation, and fast contact feedback in a single loop. We introduce TouchWorld, a predictive-and-reactive tactile foundation model for dexterous manipulation. TouchWorld uses a hierarchical policy that separates
vision-
language subtask planning, tactile world-model prediction, visuo-tactile goal-conditioned action generation, and high-frequency tactile residual refinement. A High-Level Planning Layer produces executable subtasks and predicts tactile subgoals; a Visuo-Tactile Goal-Conditioned Policy generates nominal action chunks; and a Tactile-Conditioned Refinement Policy performs online residual correction using recent tactile and proprioceptive feedback. By using touch as both a predictive contact reference and a fast feedback signal, TouchWorld preserves the semantic generalization of
vision-
language-action policies while improving local contact adaptation. Across six long-horizon and contact-rich dexterous manipulation tasks, TouchWorld achieves 65.0% success in the clean setting and 53.7% success under human perturbations, outperforming the strongest baseline by 15.7 and 18.5 percentage points, respectively.
Read the full abstract ▾
All authors Jianyi Zhou, Feiyang Hong, Yunhao Li, Yicheng Zhao, Yongjue Cen, Zirui Liu, Jiakang Huang, Zirui Chen, Ruiyang Zhang, Weizhuo Zhu, Xuhua Song, Shuo Yang
Robotics
Programmable Synchronization Graphs for Adaptive and Fault-Tolerant Modular Miniature Robots
Okan Kulekcioglu, Arqam Bin Ahmad, Ines Garcia et al. (+3 more)
Modular miniature
robots could provide scalable function in constrained environments, but coordinating many imperfect modules remains difficult when computation, communication and reliability are limited. A central robotics challenge is to coordinate many actuator-sensor modules without assigning a privileged leader, prescribing a fixed gait template, or relying on dense communication. Here we introduce a programmable synchronization-graph framework for modular miniature
robots in which each actuator-sensor pair is represented as a network node and locomotor coordination is encoded through graph coupling. Fixed intra-subgraph links synchronize heterogeneous actuator groups, whereas a small number of signed inter-subgraph links program phase relationships between groups. In physical
robot collectives with up to nine modules, graph coupling drives the emergence of synchronization, signed links tune the phase difference from in-phase to out-of-phase motion, and floor experiments produce gallop-like and trot-like contact patterns in a five-module
robot assembly. Replacing dense all-to-all coupling with sparse d-regular
topologies preserves synchronization while reducing the coupling burden. The same graph representation also captures fault tolerance: increasing graph degree increases the number of module deactivations tolerated before desynchronization. Finally, an upper-confidence-bound edge-selection
algorithm learns inter-subgraph links that drive the system toward target phase states. In a separate deactivation
benchmark, the graph-based controller avoids the leader-specific failure mode observed in centralized leader-follower control and reduces worst-case phase error by about threefold. These results establish programmable network
topology as a compact control layer for gait phase programming, online adaptation and robustness to unit loss in modular miniature
robots.
Read the full abstract ▾
All authors Okan Kulekcioglu, Arqam Bin Ahmad, Ines Garcia, Filipe Serra Alves, Onur Ozcan, M. Selim Hanay
Robotics
Manual, Joystick, or Haptic Control? An In Vitro Comparison of Navigation Strategies for Robotic Interventional Neuroradiology Procedures
Benjamin Jackson, Nikola Fischer, Harry Robershaw et al. (+16 more)
Objective: To evaluate robotic controller interfaces for interventional neuroradiology procedures in-vitro incorporating a force-sensing platform to assess safety. Methods: A custom endovascular
robot, device-mimicking controller, and sensorized neurovascular phantom were developed. Ten interventional neuroradiologists (4 novices, 6 experts) performed simulated navigations using four control modalities: device-mimicking controllers with and without haptic feedback, joystick-based input, and manual
navigation.
Navigation time, peak vessel-wall forces, incorrect catheterisations, and prolapse events were assessed, alongside user analyses. Results: Manual
navigation was fastest (mean 47.7 s) compared to haptic-on (248.7 s), haptic-off (314.7 s), and joystick (392.6 s) modalities (p<0.001). Regardless of controller type, vessel-wall forces were below the 0.70 N puncture threshold; therefore all modalities were considered safe. Joystick produced significantly more prolapse events than manual control (1.56 vs 0.13; p=0.018). Operator experience was relevant to performance: experts made fewer incorrect catheterisations than novices (0.25 vs 0.62; p=0.035) and applied less vessel-wall force (p<0.0005); these effects were sustained across controllers but accentuated when haptics were on. Users perceived haptic on and haptic off as similarly intuitive, and more intuitive than joystick (p=0.033). Conclusion: Device-mimicking robotic controllers outperform joystick interfaces on most metrics; haptic feedback shows promising but non-significant performance benefits.
Read the full abstract ▾
All authors Benjamin Jackson, Nikola Fischer, Harry Robershaw, Xingyu Chen, S.H.Hadi Sadati, Yang Li, Jeremy Lynch, Nasr Abdelsalam, Jonathon Buwanabala, Matthew Benger, Sara Sciacca, Naga Kandasamy, Marco Mancuso-Marcello, Parthiban Balasundaram, Sahan Guruge, Neelan Das, Alejandro Granados, Kawal Rhode, Thomas C Booth
Robotics · Cross-listed from cs.LG
Safe Reinforcement Learning using Ideas from Model Predictive Control
Georg Schäfer, Jakob Rehrl, Stefan Huber et al. (+1 more)
Reinforcement learning (RL) enables the synthesis of control policies directly from data, making it highly appealing for complex cyber-physical systems (CPSs) and robotics. A persistent challenge, however, is ensuring strict, hard safety constraints during the active learning phase. In real-world physical systems, violating mechanical limits can cause irreversible damage, necessitating that exploration remains strictly within safe operational regions. We propose a generalized framework that combines the adaptive, high-performance nature of deep reinforcement learning (DRL) with the formal safety guarantees of model predictive control (MPC). Using a mathematical model of the system dynamics, offline MPC computations define a feasible state-action space, representing all safe combinations of system states and control inputs that guarantee constraint satisfaction. During training and deployment, the RL
agent's instantaneous actions are projected onto this globally verified feasible set via a safety filter. We systematically evaluate our generalized approach on a non-linear 1-DoF laboratory testbed, demonstrating successful exploration and stable policy convergence on physical hardware.
Read the full abstract ▾
All authors Georg Schäfer, Jakob Rehrl, Stefan Huber, Simon Hirlaender
Robotics
Validate the Dream Before You Trust Its Verdict: Admissibility for World-Model Simulators
Christian Oefinger, Finn Rasmus Schäfer, Korbinian Moller et al. (+2 more)
Across robotics, World Models (WMs) are increasingly used to evaluate action policies by simulating the consequences of actions in an imagined world, and returning a success or safety verdict. Yet a verdict is only as trustworthy as the WM that produced it, and the WM itself needs to be certified. In
video-generation WMs, fidelity metrics such as Fréchet
Video Distance (FVD) reward visual realism, but ignore whether the world responds correctly to the policy's actions, including those unseen in training. Classical simulation-based validation assumes a trusted simulator evaluating an untrusted policy, whereas generative WMs are themselves unverified learned artifacts. Hence, we argue that any WM used as a test oracle must first be accredited before its verdicts can serve as evidence. Building on credibility practices from safety-critical simulation, including Verification, Validation & Accreditation (VV&A), Safety of the Intended Functionality (SOTIF), and scenario-based testing standards, we define an admissibility ladder (L0-L4) that a WM must climb before its closed-loop verdicts are accepted as assurance evidence. Our framework is embodiment-agnostic, and is instantiated in autonomous driving (AD), where assurance methods for traditional simulation are most mature. Applied to two driving WMs, the lower rungs reveal a reversal: the model that ranks higher on visual generation quality (L0) ranks lower on action-following (L1-L2), so visual fidelity does not predict the action-robustness a closed-loop verdict depends on.
Read the full abstract ▾
All authors Christian Oefinger, Finn Rasmus Schäfer, Korbinian Moller, Mattia Piccinini, Johannes Betz
Robotics
Disturbance-aware Motion Planning for Over-actuated Underwater Vehicles Exploiting Actuation Redundancy for High-fidelity 3D Reconstruction
Yuer Gao, Tongqing Xu, Qingyang Liu et al. (+1 more)
Underwater robots often operate near delicate targets where high-power thrusters resuspend sediments and induce
turbulence, degrading image quality at the sensor input. Conventional controllers optimize vehicle-centric objectives, such as tracking and stability, without accounting for the impact of actuation on sensing. We address this actuation-to-perception coupling by exploiting redundancy in over-actuated platforms. For an eight-thruster ROV, multiple thrust allocations can yield the same motion; we search this null space to minimize predicted disturbance in a task-relevant target region while enforcing motion constraints. Our method uses a control-oriented thruster-wake proxy derived from actuator-disk theory with directional attenuation and validated by PIV ($R^2 = 0.99$ near the wake axis; $R^2 > 0.82$ in the primary wake region), together with a real-time redundancy-resolving allocator running at 10 Hz (45 ms/solve). Across 440 trials, the approach reduces target-region particle velocity by 67% ($p < 0.001$), improves 3D reconstruction RMSE by 55% versus a disturbance-unaware baseline ($1.9 \pm 0.4$ mm vs. $4.3 \pm 1.8$ mm), and achieves a 98.5% reconstruction success rate. The framework supports autonomous scanning, which is quantitatively evaluated, and operator-assisted inspection, which is demonstrated in the supplementary materials.
Read the full abstract ▾
All authors Yuer Gao, Tongqing Xu, Qingyang Liu, Yi Cai
Robotics
Learning Spatiotemporal Tubes for Full Class of Signal Temporal Logic Tasks for Control of Unknown Systems under Input Constraints
Ahan Basu, Ratnangshu Das, Soumyodipta Nath et al. (+2 more)
This paper presents a Spatiotemporal Tube (STT)-based control framework for general unknown nonlinear Euler-Lagrange (EL) systems subject to input constraints, with the objective of satisfying Signal Temporal Logic (STL) specifications, where confinement of the system trajectory within the STT guarantees the satisfaction of the corresponding STL task. For both single and multi-
agent scenarios, the STT corresponding to each
agent is modeled as a time-varying ball, whose center and radius are jointly parameterized using a physics-informed neural network (PINN). The robustness metric associated with the STL specification corresponding to the
agents is incorporated into the training process as a loss function, enabling the learned tube to encode task-level temporal requirements. For a multi-
agent scenario, we introduce an additional robustness metric corresponding to the global task, which, when satisfied, ensures the tubes do not collide with each other. To ensure that the system trajectory remains within the learned STT and thereby satisfies the local and global STL specifications, we propose a control strategy that explicitly accounts for input constraints. In particular, a closed-form control law is developed to keep the trajectory inside the tube while regulating the motion of the tube by enforcing bounds on its
evolution depending on the input constraints of the system. The proposed approach has been validated over several case studies.
Read the full abstract ▾
All authors Ahan Basu, Ratnangshu Das, Soumyodipta Nath, Siyuan Liu, Pushpak Jagtap
Robotics
Compositional Motion Generation from Demonstration with Object-Centric Neural Fields
Ahmet Ercan Tekden and Yasemin Bekiroglu
Compositionality, by organizing complex behavior as combinations of simpler elements, enables
robot learning that is scalable and data efficient. Leveraging this principle, we propose a generative learning-from-demonstration framework that enables compositional modeling of robotic behavior by connecting perception and motion through shared object-level representations. We render scenes from object-centric neural representations that integrate canonical neural fields with latent-conditioned deformations, capturing positional and
geometric variations in a smooth, consistent, and interpretable way. For motion generation, a temporal mixture-of-experts (MoE) employs a gating mechanism to combine object-conditioned movement primitives over time, producing complete trajectories. This spatial-temporal compositionality maintains the data efficiency of movement primitives while grounding motion in visual structure, enabling systematic generalization across diverse scene configurations. In simulation, long-horizon manipulation tasks are successfully completed using the proposed model, which requires significantly less training data than other image-based baselines. Real-world experiments further demonstrate the method's robustness to noise, its ability to generalize at the category level through
language-based
segmentation models, and its capacity to operate directly on 3D scene representations.
Read the full abstract ▾
Robotics · Cross-listed from cs.NI
How the Fusion of Onboard Sensors and V2X Data can Improve (or not) the Cooperative Perception of Connected Automated Vehicles
Amir Mohammadisarab, Miguel Sepulcre, Luca Lusvarghi et al. (+1 more)
Automated vehicles rely on onboard sensors to perceive their surroundings and navigate autonomously. However, sensor performance may degrade under adverse weather conditions or when line-of-sight is obstructed. Cooperative perception (or collective perception) is expected to mitigate these limitations by enabling Connected and Automated Vehicles (CAVs) to share sensor data and collaboratively enhance situational awareness. Several studies have analyzed the potential of cooperative perception, yet the fusion of V2X data with information from onboard sensors has received limited focus. V2X data may contain errors that affect the quality of the fused data, and hence the effectiveness of cooperative perception. This study analyzes the impact of sensing measurement errors, V2X packet losses, and GNSS inaccuracies on the effectiveness of cooperative perception. The results highlight the potential of cooperative perception to enhance perception levels and range compared to using onboard sensors alone. However, they also identify key challenges related to the generation of ghost vehicles during the fusion process, which must be addressed to prevent V2X data from introducing additional errors when fused with onboard sensor data.
Read the full abstract ▾
All authors Amir Mohammadisarab, Miguel Sepulcre, Luca Lusvarghi, Javier Gozalvez
Robotics
GeoProp: Grounding Robot State in Vision for Generalist Manipulation
Guoyang Zhao, Quanhao Qian, Gongjie Zhang et al. (+5 more)
Proprioception is fundamental to robotic manipulation, yet standard fusion methods often treat it as an isolated vector lacking explicit
alignment with visual
tokens. Without a direct correspondence between 3D kinematics and 2D feature maps, manipulation policies struggle to ground the
robot's state within the scene, frequently underperforming even
vision-only baselines. To address this, we introduce GeoProp, a lightweight, plug-and-play adapter that aligns proprioception with
vision through explicit
geometric grounding and spatial feature sampling. GeoProp projects the
robot state onto the image plane to sample localized visual features, constructing a grounded state
token. It then injects state-derived spatial priors into the corresponding visual features via FiLM modulation. To capture motion intent, GeoProp further samples features at a short-horizon predicted coordinate derived from recent kinematics, providing look-ahead visual context. Across 67 tasks, GeoProp improves
Diffusion Policy by 8.7% on 63 simulation tasks and pi_0 by 4.0% on the RoboTwin subset, and yields a 10.6% average gain across both policy families in the real world, while adding only 2-3% to the parameter count. These results demonstrate that GeoProp is a simple yet high-impact inductive bias for generalist
embodied policies. Project page: https://alibaba-damo-academy.github.io/GeoProp/.
Read the full abstract ▾
All authors Guoyang Zhao, Quanhao Qian, Gongjie Zhang, Wenhao Li, Jiuniu Wang, Xiaowei Lu, Deli Zhao, Ran Xu
Robotics
PriGo: Test-Time Primitive Guidance to Diffusion and Flow Policies for Adaptive Robotic Manipulation
Zezeng Li, Enda Xiang, Thuy Tran et al. (+3 more)
Imitation learning has enabled remarkable progress in robotic manipulation, especially with
diffusion and flow-based policies that generate complex visuomotor behaviors directly from demonstrations. Yet, despite their strong performance, these policies often fail to generalize across tasks and environments. A key reason is that existing policies tend to imitate superficial action correlations rather than the underlying intent. Inspired by the compositional structure of human behaviors, we propose PriGo, a primitive-guided test-time adaptive framework for robust robotic manipulation. PriGo introduces PANet, a lightweight primitive prediction module that infers primitive distributions directly from observations. We further propose a differentiable primitive guidance mechanism that refines generated actions during inference, steering trajectories toward semantically consistent behaviors. Unlike prior primitive-conditioned approaches, PriGo operates entirely at test time and can be seamlessly integrated into pretrained
diffusion and flow policies without retraining. Extensive experiments on LIBERO, CALVIN, SIMPLER, and real-world robotic tasks demonstrate that PriGo consistently improves robustness, long-horizon execution, and generalization ability across both
diffusion and flow-based policies.
Read the full abstract ▾
All authors Zezeng Li, Enda Xiang, Thuy Tran, Di Huang, Momath Thiam, Liming Chen
Robotics
A Closed-Loop Multi-Agent Framework for Robust Multi-Robot Manipulation
Yi-Xiang He, Lan Wei, Haoming Cen et al. (+6 more)
Multi-
robot systems provide the parallelism and redundancy necessary for long-horizon tasks, while Large
Language Models (LLMs) offer the
reasoning capabilities to decompose these objectives into actionable plans. However, effectively grounding this high-level
reasoning in physical multi-
robot execution remains an open challenge. Existing LLM-based approaches fall mainly into two categories: Single-
robot methods achieve robust contact-rich manipulation but lack the coordination mechanisms required for tasks spanning multiple workspaces. Current multi-
robot frameworks focus on high-level planning, often treating manipulation as an idealized primitive that fails to account for real-world execution uncertainties. To address this, we propose a hierarchical closed-loop agentic LLM-based framework to ensure robust multi-
robot manipulation. Our system consists of three specialized
agents: the Planning
Agent decomposes instructions into allocated sub-tasks, the Manipulation
Agent for each
robot executes actions via adaptive tool use, and the Verification
Agent closes the loop by monitoring physical outcomes and feeding back semantic corrections. Extensive real-world experiments demonstrate that our framework achieves superior success rates, ensures robust adaptability ranging from single to cross workspace manipulation, and offers a generalizable approach for diverse manipulation tasks.
Read the full abstract ▾
All authors Yi-Xiang He, Lan Wei, Haoming Cen, Jian-Jian Jiang, Zhuohao Li, Guanxing Lu, Yihan Yang, Dandan Zhang, Wei-Shi Zheng
Robotics
Ace! Motion Planning of Professional-Level Table Tennis Serves with a Robot Arm
Guillem Torrente, Guilherme Jorge Maeda, Divij Grover et al. (+3 more)
Table tennis, a dynamic, compact, and popular sport, has received significant
attention as a robotics
benchmark over the last decades. Most of the research has focused on the rally aspect - returning an incoming ball - requiring high-speed
vision, agile motion planning, and tight closed-loop control. However, the other component of table tennis gameplay - the serve - is comparatively a quite unexplored research problem, that in fact requires pushing physics modeling and control to the extremes. Achieving competitive serves with a
robot presents domain-specific challenges, such as high-
spin generation from a spinless ball, precise aiming, or multi-objective optimization. In this work, we present a novel approach for generating official rule-compliant serves by combining motion primitives, Model Predictive Control, and
Bayesian Optimization. Serves generated in this way offer a wide and controllable variation of
spins of up to 550 rad/s, and speeds of up to 6.7 m/s, matching and even surpassing those of elite table tennis players.
Read the full abstract ▾
All authors Guillem Torrente, Guilherme Jorge Maeda, Divij Grover, Megumu Tsukamoto, Hamdi Sahloul, Peter Dürr
Robotics
WAM-TTT: Steering World-Action Models by Watching Human Play at Test Time
Yusen Feng, Bingchen Han, Jiangran Lyu et al. (+13 more)
Steering
robot foundation models (RFMs) toward new task variants or user-preferred behaviors remains challenging, often requiring additional
robot demonstrations, task-specific fine-tuning, or long-context conditioning. We present WAM-TTT, a test-time training framework for steering world action models from raw human
videos. Rather than treating human
videos as trajectories to imitate, WAM-TTT absorbs them into a lightweight adaptive
memory inside a frozen WAM through self-supervised
video prediction. To make this
memory useful for control, we introduce a meta-training stage that aligns human demonstrations with
robot behaviors using paired human-
robot data and a key–value
memory reconstruction objective. At test time, only unlabeled human
videos are required to adapt the
memory, while the pretrained WAM remains frozen. This enables efficient and reusable steering without
robot actions, human-side annotations, or task-specific fine-tuning, while preserving the generalization ability of the foundation model. Extensive experiments show that WAM-TTT consistently outperforms in-context human-
video conditioning baselines across diverse manipulation tasks and generalization settings.
Read the full abstract ▾
All authors Yusen Feng, Bingchen Han, Jiangran Lyu, Kai Liu, Yixin Zheng, Yuxuan Wan, Weiheng Liu, Sun Han, Ruiqin Li, Yulong Zhang, Fangfu Liu, Xuesong Shi, Libin Liu, Yizhou Wang, Zhizheng Zhang, He Wang
Robotics
SPECTRA: Context-Conditioned Spectral Movement Primitives for Robot Skill Generalization
Boxuan Zhang, Sheng Liu, Chenlin Ming et al. (+1 more)
Robot imitation learning for manipulation should preserve demonstrated task geometry while producing dynamically admissible
robot motions. Existing pipelines often learn task-dependent trajectories and impose execution limits afterward through filtering, smoothing, clipping, or time scaling, which may distort task-critical end-effector paths. We propose the Spectral Movement Primitive (SMP), a frequency-domain imitation learning framework that couples task-space skill generation with joint-space execution regulation. Demonstrations are represented by truncated finite-horizon
Fourier coefficients. An empirically selected low-frequency task band captures the dominant motion geometry, while higher harmonics contribute disproportionately to derivative growth. A frame-aware context-conditioned GMM/GMR prior predicts the task-band coefficients in a canonical task frame, and the resulting Cartesian trajectory is mapped to joint space through sequential inverse kinematics. A phase-coupled regulator then limits the requested phase progression without modifying the spectral coefficients, thereby enforcing joint velocity and acceleration limits while preserving the represented path. Experiments evaluate task-band reconstruction, robustness to composite demonstration corruption, out-of-distribution cross-board generalization, joint-space dynamic admissibility, end-effector path preservation, and deployment on a Franka Panda
robot. Results show compact
geometric reconstruction, consistent transfer across unseen task frames, substantial reductions in dynamic violations and jerk, and preservation of the intended end-effector path during phase regulation.
Read the full abstract ▾
All authors Boxuan Zhang, Sheng Liu, Chenlin Ming, Ahmed Abdelrahman