Robotics
Toward AI Autonomous Navigation for Mechanical Thrombectomy using Hierarchical Modular Multi-agent Reinforcement Learning (HM-MARL)
Harry Robertshaw, Nikola Fischer, Lennart Karstensen et al. (+6 more)
Mechanical thrombectomy (MT) is typically the optimal treatment for acute ischemic stroke involving large vessel occlusions, but access is limited due to geographic and logistical barriers. Reinforcement learning (RL) shows promise in autonomous endovascular
navigation, but generalization across 'long'
navigation tasks remains challenging. We propose a Hierarchical Modular Multi-
Agent Reinforcement Learning (HM-MARL) framework for autonomous two-device
navigation in vitro, enabling efficient and generalizable
navigation. HM-MARL was developed to autonomously navigate a guide catheter and guidewire from the femoral artery to the internal carotid artery (ICA). A modular multi-
agent approach was used to decompose the complex
navigation task into specialized subtasks, each trained using Soft Actor-Critic RL. The framework was validated in both in silico and in vitro testbeds to assess generalization and real-world feasibility. In silico, a single-vasculature model achieved 92-100% success rates on individual anatomies, while a multi-vasculature model achieved 56-80% across multiple patient anatomies. In vitro, both HM-MARL models successfully navigated 100% of trials from the femoral artery to the right common carotid artery and 80% to the right ICA but failed on the left-side vessel superhuman challenge due to the anatomy and catheter type used in
navigation. This study presents the first demonstration of in vitro autonomous
navigation in MT vasculature. While HM-MARL enables generalization across anatomies, the simulation-to-real transition introduces challenges. Future work will refine RL strategies using world models and validate performance on unseen in vitro data, advancing autonomous MT towards clinical translation.
Read the full abstract ▾
All authors Harry Robertshaw, Nikola Fischer, Lennart Karstensen, Benjamin Jackson, Xingyu Chen, S.M.Hadi Sadati, Christos Bergeles, Alejandro Granados, Thomas C Booth
Robotics
Robotic Fruits with Tunable Stiffness and Sensing: Towards a Methodology for Developing Realistic Physical Twins of Fruits
Saitarun Nadipineni, Keshav Pandiyan, Kaspar Althoefer et al. (+2 more)
The global agri-food sector faces increasing challenges from labour shortages, high consumer demand, and supply-chain disruptions, resulting in substantial losses of unharvested produce. Robotic harvesting has emerged as a promising alternative; however, evaluating and training soft grippers for delicate fruits remains difficult due to the highly variable mechanical properties of natural produce. This makes it difficult to establish reliable
benchmarks or data-driven control strategies. Existing testing practices rely on large quantities of real fruit to capture this variability, leading to inefficiency, higher costs, and waste. The methodology presented in this work aims to address these limitations by developing tunable soft physical twins that emulate the stiffness characteristics of real fruits at different ripeness levels. A fiber-reinforced pneumatic physical twin of a kiwi fruit was designed and fabricated to replicate the stiffness at different ripeness levels. Experimental results show that the stiffness of the physical twin can be tuned accurately over multiple trials (97.35 - 99.43% accuracy). Gripping tasks with a commercial robotic gripper showed that sensor feedback from the physical twin can reflect the applied gripping forces. Finally, a stress test was performed over 50 cycles showed reliable maintenance of desired stiffness (0.56 - 1.10% error). This work shows promise that robotic physical twins could adjust their stiffness to resemble that of real fruits. This can provide a sustainable, controllable platform for benchmarking and training robotic grippers.
Read the full abstract ▾
All authors Saitarun Nadipineni, Keshav Pandiyan, Kaspar Althoefer, Shinichi Hirai, Thilina Dulantha Lalitharatne
Robotics
Infinite-Dimensional Closed-Loop Inverse Kinematics for Soft Robots via Neural Operators
Carina Veil, Moritz Flaschel, Ellen Kuhl et al. (+1 more)
For fully actuated rigid
robots, kinematic inversion is a purely
geometric problem, efficiently solved by closed-loop inverse kinematics (CLIK) schemes that compute joint velocities to position the
robot body in space. For underactuated soft
robots, however, not all configurations are attainable through control action, making kinematic inversion extremely ill-posed. Extensions of CLIK address this by introducing end-to-end mappings from actuation to task space for the controller to operate on, but typically assume finite dimensions of the underlying virtual configuration space. In this work, we formulate CLIK in the infinite-dimensional domain to reason about the entire soft
robot shape while solving tasks, thereby skipping intermediate approximation steps. We do this by composing an actuation-to-shape map with a shape-to-task map, deriving the differential end-to-end kinematics via an infinite-dimensional chain rule, and thereby obtaining a Jacobian-based CLIK
algorithm. Since complex soft-
robot models are rarely available in closed form, and neither the actuation-to-shape mapping nor its Jacobian can typically be obtained analytically, we propose to learn it using differentiable neural operator networks. We first present an analytical study on a constant-
curvature segment, and then apply the neural version of the
algorithm to a three-fiber soft robotic arm whose underlying model relies on morphoelasticity and active filament theory.
Read the full abstract ▾
All authors Carina Veil, Moritz Flaschel, Ellen Kuhl, Cosimo Della Santina
Robotics
Soft Surfaced Vision-Based Tactile Sensing for Bipedal Robot Applications
Jaeeun Kim, Junhee Lim and Yu She
Legged locomotion benefits from
embodied sensing, where perception emerges from the physical interaction between body and environment. We present a soft-surfaced,
vision-based tactile foot sensor that endows a bipedal
robot with a skin-like deformable layer that captures contact deformations optically, turning foot-ground interactions into rich haptic signals. From a contact image stream, our method estimates contact pose (position and orientation), visualizes shear, computes center of pressure (CoP), classifies terrain, and detects
geometric features of the contact patch. We validate these capabilities on a tilting platform and in visually obscured conditions, showing that foot-borne tactile feedback improves balance control and terrain awareness beyond proprioception alone. These findings suggest that integrating tactile perception into legged
robot feet improves stability, adaptability, and environmental awareness, offering a promising direction toward more compliant and intelligent locomotion systems. For the supplementary
video, please visit: https://youtu.be/ceJiy9q_2Aw
Read the full abstract ▾
Robotics
FORMICA: Decision-Focused Learning for Communication-Free Multi-Robot Task Allocation
Antonio Lopez, Jack Muirhead and Carlo Pinciroli
Most multi-
robot task allocation methods rely on communication to resolve conflicts and reach consistent assignments. In environments with limited bandwidth, degraded infrastructure, or
adversarial interference, existing approaches degrade sharply. We introduce a learning-based framework that achieves high-quality task allocation without any
robot-to-
robot communication. The key idea is that
robots coordinate implicitly by predicting teammates' bids: if each
robot can anticipate competition for a task, it can adjust its choices accordingly. Our method predicts bid distributions to correct systematic errors in analytical mean-field approximations. While analytical predictions assume idealized conditions (uniform distributions, known bid functions), our learned approach adapts to task clustering and spatial heterogeneity. Inspired by Smart Predict-then-Optimize (SPO), we train predictors end-to-end to minimize Task Allocation Regret rather than prediction error. To scale to large swarms, we develop a mean-field approximation where each
robot predicts the distribution of competing bids rather than individual bids, reducing complexity from $O(NT)$ to $O(T)$. We call our approach FORMICA: Field-Oriented Regret-Minimizing Implicit Coordination
Algorithm. Experiments show FORMICA substantially outperforms a natural analytical baseline. In scenarios with 16
robots and 64 tasks, our approach improves system reward by 17% and approaches the optimal MILP solution. When deployed on larger scenarios (256
robots, 4096 tasks), the same model improves performance by 7%, demonstrating strong generalization. Training requires only 21 seconds on a laptop, enabling rapid adaptation to new environments.
Read the full abstract ▾
Robotics
OVerSeeC: Open-Vocabulary Costmap Generation from Satellite Images and Natural Language
Rwik Rana, Jesse Quattrociocchi, Dongmyeong Lee et al. (+5 more)
Aerial imagery provides essential global context for autonomous
navigation, enabling route planning at scales inaccessible to onboard sensing. We address the problem of generating global costmaps for long-range planning directly from satellite imagery when entities and mission-specific traversal rules are expressed in natural
language at test time. This setting is challenging since mission requirements vary, terrain entities may be unknown at deployment, and user
prompts often encode compositional traversal logic. Existing approaches relying on fixed ontologies and static cost mappings cannot accommodate such flexibility. While foundation models excel at
language interpretation and open-vocabulary perception, no single model can simultaneously parse nuanced mission directives, locate arbitrary entities in large-scale imagery, and synthesize them into an executable cost function for planners. We therefore propose OVerSeeC, a zero-shot modular framework that decomposes the problem into Interpret-Locate-Synthesize: (i) an LLM extracts entities and ranked preferences, (ii) an open-vocabulary
segmentation pipeline identifies these entities from high-resolution imagery, and (iii) the LLM uses the user's natural
language preferences and masks to synthesize executable costmap code. Empirically, OVerSeeC handles novel entities, respects ranked and compositional preferences, and produces routes consistent with human-drawn trajectories across diverse regions, demonstrating robustness to distribution shifts. This shows that modular composition of foundation models enables open-vocabulary, preference-aligned costmap generation for scalable, mission-adaptive global planning.
Read the full abstract ▾
All authors Rwik Rana, Jesse Quattrociocchi, Dongmyeong Lee, Christian Ellis, Amanda Adkins, Adam Uccello, Garrett Warnell, Joydeep Biswas
Robotics
Enhancing Goal Inference via Correction Timing
Anjiabei Wang, Shuangge Wang and Tesca Fitzgerald
Corrections offer a natural modality for people to provide feedback to a
robot, by (i) intervening in the
robot's behavior when they believe the
robot is failing (or will fail) the task objectives and (ii) modifying the
robot's behavior to successfully fulfill the task. Each correction offers information on what the
robot should and should not do, where the corrected behavior is more aligned with task objectives than the original behavior. Most prior work on learning from corrections involves interpreting a correction as a new demonstration (consisting of the modified
robot behavior), or a preference (for the modified trajectory compared to the
robot's original behavior). However, this overlooks one essential element of the correction feedback, which is the human's decision to intervene in the
robot's behavior in the first place. This decision can be influenced by multiple factors including the
robot's task progress,
alignment with human expectations, dynamics, motion legibility, and optimality. In this work, we investigate whether the timing of this decision can offer a useful signal for inferring these task-relevant influences. In particular, we investigate three potential applications for this learning signal: (1) identifying features of a
robot's motion that may
prompt people to correct it, (2) quickly inferring the final goal of a human's correction based on the timing and initial direction of their correction motion, and (3) learning more precise constraints for task objectives. Our results indicate that correction timing results in improved learning for the first two of these applications. Overall, our work provides new insights on the value of correction timing as a signal for
robot learning.
Read the full abstract ▾
Robotics
Design and Biomechanical Evaluation of a Lightweight Low-Complexity Soft Bilateral Ankle Exoskeleton
Josée Mallah, Zakii Javed, Zafer Azak et al. (+2 more)
Many people could benefit from exoskeleton assistance during gait, for either medical or nonmedical purposes. But exoskeletons bring added mass and structure, which in turn require compensating for. In this work, we present a lightweight, low-complexity, soft bilateral ankle exoskeleton for plantarflexion assistance, with a shoe attachment design that can be mounted on top of any pair of shoes. Experimental tests show no significant difference in lower limb kinematics and kinetics when wearing the exoskeleton in zero-torque mode relative to not wearing an exoskeleton, showing that our device does not obstruct healthy gait, and proving it as a compliant and comfortable device, promising to provide effective assistance. Hence, a control system was developed, and additional tests are underway.
Read the full abstract ▾
All authors Josée Mallah, Zakii Javed, Zafer Azak, Thomas Stone, Luigi G. Occhipinti
Robotics · Cross-listed from cs.CV
VLANeXt Family: A Systematic Study of VLA Models from Core Recipes to Emerging Paradigms
Xiao-Ming Wu, Kang Liao, Yihang Luo et al. (+6 more)
Following the rise of large foundation models,
Vision-
Language-Action models (VLAs) emerged, leveraging strong visual and
language understanding from
Vision-
Language Models for general-purpose policy learning. Yet, the current VLA landscape remains fragmented and exploratory. Although many groups have proposed their own VLA models, inconsistencies in training protocols and evaluation settings make it difficult to identify which design choices truly matter. To bring structure to this evolving space, we reexamine the VLA design space under a unified framework and evaluation setup. Starting from a simple VLA baseline similar to RT-2, which is the origin of VLA, we systematically dissect design choices along three dimensions: foundational components, perception essentials, and action modeling perspectives. From this study, we distill 12 key findings that together form a practical recipe for building strong VLA models. The outcome of this exploration is a simple yet effective model, VLANeXt. It outperforms the state-of-the-art methods on the LIBERO and LIBERO-plus
benchmarks and demonstrates strong performance in real-world experiments. Beyond identifying the core recipe, we further ask how far these design principles extend to the emerging paradigms in VLAs. We thus expand VLANeXt along several emerging directions, including model scaling, latent-action pretraining, latent predictive representation learning, and world action modeling. These studies give rise to the VLANeXt family, spanning compact and scaled VLA variants, latent-action models, JEPA-style predictive models, and World Action Models. Our results show that the core recipe provides a strong foundation across different model scales and emerging paradigms.
Read the full abstract ▾
All authors Xiao-Ming Wu, Kang Liao, Yihang Luo, Bin Fan, Jian-Jian Jiang, Runze Yang, Zhonghua Wu, Wei-Shi Zheng, Chen Change Loy
Robotics · Cross-listed from cs.CV
CapNav: Benchmarking Vision Language Models on Capability-conditioned Indoor Navigation
Xia Su, Ruiqi Chen, Benlin Liu et al. (+4 more)
Vision-
Language Models (VLMs) have shown remarkable progress in
Vision-
Language Navigation (VLN), offering new possibilities for
navigation decision-making that could benefit both robotic platforms and human users. However, real-world
navigation is inherently conditioned by the
agent's mobility constraints. For example, a sweeping
robot cannot traverse stairs, while a quadruped can. We introduce Capability-Conditioned
Navigation (CapNav), a
benchmark designed to evaluate how well VLMs can navigate complex indoor spaces given an
agent's specific physical and operational capabilities. CapNav defines five representative human and
robot agents, each described with physical dimensions, mobility capabilities, and environmental interaction abilities. CapNav provides 45 real-world indoor scenes, 473
navigation tasks, and 2365 QA pairs to test if VLMs can traverse indoor environments based on
agent capabilities. We evaluate 13 modern VLMs and find that current VLM's
navigation performance drops sharply as mobility constraints tighten, and that even state-of-the-art models struggle with obstacle types that require
reasoning on spatial dimensions. We conclude by discussing the implications for capability-aware
navigation and the opportunities for advancing
embodied spatial
reasoning in future VLMs. The
benchmark is available at https://github.com/makeabilitylab/CapNav
Read the full abstract ▾
All authors Xia Su, Ruiqi Chen, Benlin Liu, Jingwei Ma, Zonglin Di, Ranjay Krishna, Jon Froehlich
Robotics
Snapping Actuators with Asymmetric and Sequenced Motion
Xin Li, Ye Jin, Mohsen Jafarpour et al. (+2 more)
Snapping instabilities in soft structures offer a powerful pathway to achieve rapid and energy-efficient actuation. In this study, an eccentric dome-shaped snapping actuator is developed to generate controllable asymmetric motion through geometry-induced instability. Finite element simulations and experiments reveal consistent asymmetric deformation and the corresponding pressure characteristics. By coupling four snapping actuators in a pneumatic network, a compact quadrupedal
robot achieves coordinated wavelike locomotion using only a single pressure input. The
robot exhibits frequency-dependent performance with a maximum speed of 72.78~mm/s at 7.5~Hz. These findings demonstrate the potential of asymmetric snapping mechanisms for physically controlled actuation and lay the groundwork for fully untethered and efficient soft robotic systems.
Read the full abstract ▾
All authors Xin Li, Ye Jin, Mohsen Jafarpour, Hugo de Souza Oliveira, Edoardo Milana
Robotics
How Fast Can I Run My VLA? Demystifying VLA Inference Performance with VLA-Perf
Wenqi Jiang, Jason Clemons, Karu Sankaralingam et al. (+1 more)
Vision-
Language-Action (VLA) models have recently demonstrated impressive capabilities across various
embodied AI tasks. While deploying VLA models on real-world
robots imposes strict real-time inference constraints, the inference performance landscape of VLA remains poorly understood due to the large combinatorial space of model architectures and inference systems. In this paper, we ask a fundamental research question: How should we design future VLA models and systems to support real-time inference? To address this question, we first introduce VLA-Perf, an analytical performance model that can analyze inference performance for arbitrary combinations of VLA models and inference systems. Using VLA-Perf, we conduct the first systematic study of the VLA inference performance landscape. From a model-design perspective, we examine how inference performance is affected by model scaling, model architectural choices, long-context
video inputs, asynchronous inference, and dual-system model pipelines. From the deployment perspective, we analyze where VLA inference should be executed – on-device, on edge servers, or in the cloud – and how hardware capability and network performance jointly determine end-to-end latency. By distilling 15 key takeaways from our comprehensive evaluation, we hope this work can provide practical guidance for the design of future VLA models and inference systems.
Read the full abstract ▾
All authors Wenqi Jiang, Jason Clemons, Karu Sankaralingam, Christos Kozyrakis
Robotics
Learning to Tune Pure Pursuit in Autonomous Racing: Joint Lookahead and Steering-Gain Control with PPO
Mohamed Elgouhary and Amr S. El-Wakeel
Pure Pursuit (PP) is widely used in autonomous racing for real-time path tracking due to its efficiency and
geometric clarity, yet performance is highly sensitive to how key parameters-lookahead distance and steering gain-are chosen. Standard velocity-based schedules adjust these only approximately and often fail to transfer across tracks and speed profiles. We propose a reinforcement-learning (RL) approach that jointly chooses the lookahead Ld and a steering gain g online using Proximal Policy Optimization (PPO). The policy observes compact state features (speed and
curvature taps) and outputs (Ld, g) at each control step. Trained in F1TENTH Gym and deployed in a ROS 2 stack, the policy drives PP directly (with light smoothing) and requires no per-map retuning. Across simulation and real-car tests, the proposed RL-PP controller that jointly selects (Ld, g) consistently outperforms fixed-lookahead PP, velocity-scheduled adaptive PP, and an RL lookahead-only variant, and it also exceeds a kinematic MPC raceline tracker under our evaluated settings in lap time, path-tracking accuracy, and steering smoothness, demonstrating that policy-guided parameter tuning can reliably improve classical geometry-based control.
Read the full abstract ▾
Robotics
Ori-Sense: origami capacitive sensing for soft robotic applications
Hugo de Souza Oliveira, Xin Li, Mohsen Jafarpour et al. (+1 more)
This work introduces Ori-Sense, a compliant capacitive sensor inspired by the inverted Kresling origami pattern. The device translates torsional deformation into measurable capacitance changes, enabling proprioceptive feedback for soft robotic systems. Using dissolvable-core molding, we fabricated a monolithic silicone structure with embedded conductive TPU electrodes, forming an integrated soft capacitor. Mechanical characterization revealed low stiffness and minimal impedance, with torque values below 0.01 N mm for axial displacements between -15 mm and 15 mm, and up to 0.03 N mm at 30 degrees twist under compression. Finite-element simulations confirmed localized stresses along fold lines and validated the measured torque-rotation response. Electrical tests showed consistent capacitance modulation up to 30%, directly correlated with the twist angle, and maximal sensitivity of S_theta ~ 0.0067 pF/deg at 5 mm of axial deformation.
Read the full abstract ▾
All authors Hugo de Souza Oliveira, Xin Li, Mohsen Jafarpour, Edoardo Milana
Robotics
Zero-shot Interactive Perception
Venkatesh Sripada, Frank Guerin and Amir Ghalamzan
Interactive perception (IP) enables
robots to extract hidden information in their workspace and execute manipulation plans by physically interacting with objects and altering the state of the environment – crucial for resolving occlusions and ambiguity in complex, partially observable scenarios. We present Zero-Shot IP (ZS-IP), a novel framework that couples multi-strategy manipulation (pushing and grasping) with a
memory-driven
Vision Language Model (VLM) to guide robotic interactions and resolve semantic queries. ZS-IP integrates three key components: (1) an Enhanced Observation (EO) module that augments the VLM's visual perception with both conventional keypoints and our proposed pushlines – a novel 2D visual augmentation tailored to pushing actions, (2) a
memory-guided action module that reinforces semantic
reasoning through context lookup, and (3) a robotic controller that executes pushing, pulling, or grasping based on VLM output. Unlike grid-based augmentations optimized for pick-and-place, pushlines capture affordances for contact-rich actions, substantially improving pushing performance. We evaluate ZS-IP on a 7-DOF Franka Panda arm across diverse scenes with varying occlusions and task complexities. Our experiments demonstrate that ZS-IP outperforms passive and viewpoint-based perception techniques such as Mark-Based Visual Prompting (MOKA), particularly in pushing tasks, while preserving the integrity of non-target elements.
Read the full abstract ▾
Robotics
Downwash-aware Configuration Optimization for Modular Aerial Systems
Mengguang Li and Heinz Koeppl
This work proposes a framework that generates and optimally selects task-specific assembly configurations for a large group of homogeneous modular aerial systems, explicitly enforcing bounds on inter-module downwash. Prior work largely focuses on planar layouts and often ignores aerodynamic interference. In contrast, firstly we enumerate non-isomorphic connection
topologies at scale; secondly, we solve a nonlinear program to check feasibility and select the configuration that minimizes control input subject to actuation limits and downwash constraints. We evaluate the framework in physics-based simulation and demonstrate it in real-world experiments.
Read the full abstract ▾
Robotics
Tendon-Driven Reciprocating and Non-Reciprocating Motion via Snapping Metabeams
Mohsen Jafarpour, Ayberk Yüksek, Shahab Eshghi et al. (+2 more)
Snapping beams enable rapid
geometric transitions through nonlinear instability, offering an efficient means of generating motion in soft robotic systems. In this study, a tendon-driven mechanism consisting of spiral-based metabeams was developed to exploit this principle for producing both reciprocating and non-reciprocating motion. The snapping structures were fabricated using fused deposition modeling with polylactic acid (PLA) and experimentally tested under different boundary conditions to analyze their nonlinear behavior. The results show that the mechanical characteristics, including critical forces and stability, can be tuned solely by adjusting the boundary constraints. The spiral geometry allows large reversible deformation even when made from a relatively stiff material such as PLA, providing a straightforward design concept for controllable snapping behavior. The developed mechanism was further integrated into a swimming
robot, where tendon-driven fins exhibited two distinct actuation modes: reciprocating and non-reciprocating motion. The latter enabled efficient propulsion, producing a forward displacement of about 32 mm per 0.4 s cycle ($\approx$ 81 mm/s, equivalent to 0.4 body lengths per second). This study highlights the potential of geometry-driven snapping structures for efficient and programmable actuation in soft robotic systems.
Read the full abstract ▾
All authors Mohsen Jafarpour, Ayberk Yüksek, Shahab Eshghi, Stanislav Gorb, Edoardo Milana
Robotics · Cross-listed from cs.CV
Diff2DGS: Reliable Reconstruction of Occluded Surgical Scenes via 2D Gaussian Splatting
Tianyi Song, Danail Stoyanov, Evangelos Mazomenos et al. (+1 more)
Real-time reconstruction of deformable
surgical scenes is vital for advancing robotic surgery, improving intraoperative guidance, and enabling automation. Recent methods achieve dense reconstructions from da Vinci robotic surgery
videos, with Gaussian Splatting (GS) offering real-time performance via graphics acceleration. However, reconstruction quality in occluded regions remains limited, and depth accuracy has not been fully assessed, as
benchmarks like EndoNeRF and StereoMIS lack 3D ground truth. We propose Diff2DGS, a two-stage framework for reliable 3D reconstruction of occluded
surgical scenes. First, a
diffusion-based
video module with temporal priors inpaints tissue occluded by instruments with high spatiotemporal consistency. Second, we adapt 2D Gaussian Splatting (2DGS) with a Learnable Deformation Model (LDM) to capture dynamic tissue deformation and anatomical geometry, and introduce adaptive depth weight to improve
geometric fidelity. We further extend evaluation beyond image-quality metrics by performing quantitative depth analysis on the SCARED
dataset. Diff2DGS outperforms state-of-the-art methods in both appearance and geometry, reaching 38.02 dB PSNR on EndoNeRF and 34.40 dB on StereoMIS. Our experiments also show that optimizing image quality alone does not necessarily ensure accurate 3D geometry.The code is available at https://diff2dgs.github.io/.
Read the full abstract ▾
All authors Tianyi Song, Danail Stoyanov, Evangelos Mazomenos, Francisco Vasconcelos
Robotics
Learning Smooth Time-Varying Linear Policies with an Action Jacobian Penalty
Zhaoming Xie, Kevin Karol and Jessica Hodgins
Reinforcement learning provides a framework for learning control policies that can reproduce diverse motions for simulated characters. However, such policies often exploit unnatural high-frequency signals that are unachievable by humans or physical
robots, making them poor representations of real-world behaviors. Existing work addresses this issue by adding a reward term that penalizes a large change in actions over time. This term often requires substantial tuning efforts. We propose to use the action Jacobian penalty, which penalizes changes in action with respect to the changes in simulated state directly through auto differentiation. This effectively eliminates unrealistic high-frequency control signals without task specific tuning. While effective, the action Jacobian penalty introduces significant computational overhead when used with traditional fully connected neural network architectures. To mitigate this, we introduce a new architecture called a Linear Policy Net (LPN) that significantly reduces the computational burden for calculating the action Jacobian penalty during training. In addition, a LPN requires no parameter tuning, exhibits faster learning convergence compared to baseline methods, and can be more efficiently queried during inference time compared to a fully connected neural network. We demonstrate that a Linear Policy Net, combined with the action Jacobian penalty, is able to learn policies that generate smooth signals while solving a number of motion imitation tasks with different characteristics, including dynamic motions such as a backflip and various challenging parkour skills. Finally, we apply this approach to create policies for dynamic motions on a physical quadrupedal
robot equipped with an arm.
Read the full abstract ▾
Robotics
Role-Adaptive Collaborative Formation Planning for Team of Quadruped Robots in Cluttered Environments
Magnus Norén, Marios-Nektarios Stamatopoulos, Avijit Banerjee et al. (+1 more)
This paper presents a role-adaptive Leader-Follower-based formation planning and control framework for teams of quadruped
robots operating in cluttered environments. Unlike conventional methods with fixed leaders or rigid formation roles, the proposed approach integrates dynamic role assignment and partial goal planning, enabling flexible, collision-free
navigation in complex scenarios. Formation stability and inter-
robot safety are ensured through a virtual spring-damper system coupled with a novel obstacle avoidance layer that adaptively adjusts each
agent's velocity. A dynamic look-ahead reference generator further enhances flexibility, allowing temporary formation deformation to maneuver around obstacles while maintaining goal-directed motion. The Fast Marching Square (FM2)
algorithm provides the global path for the leader and local paths for the followers as the planning backbone. The framework is validated through extensive simulations and real-world experiments with teams of quadruped
robots. Results demonstrate smooth coordination, adaptive role switching, and robust formation maintenance in complex, unstructured environments. A
video featuring the simulation and physical experiments along with their associated visualizations can be found at https://youtu.be/scq37Tua9W4.
Read the full abstract ▾
All authors Magnus Norén, Marios-Nektarios Stamatopoulos, Avijit Banerjee, George Nikolakopoulos
Robotics
RoEL: Robust Event-based 3D Line Reconstruction
Gwangtak Bae, Jaeho Shin, Seunggu Kang et al. (+3 more)
Event cameras in motion tend to detect object boundaries or texture edges, which produce lines of brightness changes, especially in man-made environments. While lines can constitute a robust intermediate representation that is consistently observed, the sparse nature of lines may lead to drastic deterioration with minor estimation errors. Only a few previous works, often accompanied by additional sensors, utilize lines to compensate for the severe domain discrepancies of event sensors along with unpredictable noise characteristics. We propose a method that can stably extract tracks of varying appearances of lines using a clever algorithmic process that observes multiple representations from various time slices of events, compensating for potential adversaries within the event data. We then propose
geometric cost functions that can refine the 3D line maps and camera poses, eliminating projective distortions and depth ambiguities. The 3D line maps are highly compact and can be equipped with our proposed cost function, which can be adapted for any observations that can detect and extract line structures or projections of them, including 3D point cloud maps or image observations. We demonstrate that our formulation is powerful enough to exhibit a significant performance boost in event-based mapping and pose refinement across diverse
datasets, and can be flexibly applied to
multimodal scenarios. Our results confirm that the proposed line-based formulation is a robust and effective approach for the practical deployment of event-based perceptual modules. Project page: https://gwangtak.github.io/roel/
Read the full abstract ▾
All authors Gwangtak Bae, Jaeho Shin, Seunggu Kang, Junho Kim, Ayoung Kim, Young Min Kim
Robotics
SimVLA: A Simple VLA Baseline for Robotic Manipulation
Yuankai Luo, Woping Chen, Tong Liang et al. (+2 more)
Vision-
Language-Action (VLA) models have emerged as a promising paradigm for general-purpose robotic manipulation, leveraging large-scale pre-training to achieve strong performance. The field has rapidly evolved with additional spatial priors and diverse architectural innovations. However, these advancements are often accompanied by varying training recipes and implementation details, which can make it challenging to disentangle the precise source of empirical gains. In this work, we introduce SimVLA, a streamlined baseline designed to establish a transparent reference point for VLA research. By strictly decoupling perception from control, using a standard
vision-
language backbone and a lightweight action head, and standardizing critical training dynamics, we demonstrate that a minimal design can achieve state-of-the-art performance. Despite having only 0.5B parameters, SimVLA outperforms multi-billion-parameter models on standard simulation
benchmarks without
robot pretraining. SimVLA also reaches on-par real-
robot performance compared to pi0.5. Our results establish SimVLA as a robust, reproducible baseline that enables clear attribution of empirical gains to future architectural innovations. Website: https://frontierrobo.github.io/SimVLA
Read the full abstract ▾
All authors Yuankai Luo, Woping Chen, Tong Liang, Baiqiao Wang, Zhenguo Li
Robotics
Design and Characterization of a Dual-DOF Soft Shoulder Exosuit with Volume-Optimized Pneumatic Actuator
Rui Chen, Domenico Chiaradia, Daniele Leonardis et al. (+1 more)
Portable pneumatic systems for 2 degree-of-freedom (DOF) soft shoulder exosuits remain underexplored, and face fundamental trade-offs between torque output and dynamic response that are further compounded by the need for multiple actuators to support complex shoulder movement. This work addresses these constraints through a volume-optimized spindle-shaped angled actuator (SSAA) geometry: by reducing actuator volume by 35.7% (357mL vs. 555mL), the SSAA maintains 94.2% of output torque while achieving 35.2% faster dynamic response compared to uniform cylindrical designs. Building on the SSAA, we develop a curved abduction actuator (CAA) based on the SSAA geometry and a horizontal adduction actuator (HAA) based on the pouch motor principle, integrating both into a dual-DOF textile-based shoulder exosuit (390 g). The exosuit delivers multi-modal assistance spanning shoulder abduction, flexion, and horizontal adduction, depending on the actuation. User studies with 10 healthy participants reveal that the exosuit substantially reduces electromyographic (EMG) activity across both shoulder abduction and flexion tasks. For abduction with HAA only, the exosuit achieved up to 59% muscle activity reduction across seven muscles. For flexion, both the single-actuator configuration (HAA only) and the dual-actuator configuration (HAA,+,CAA) reduced EMG activity by up to 63.7% compared to no assistance. However, the incremental benefit of adding the CAA to existing HAA support was limited in healthy users during flexion, with statistically significant additional reductions observed only in pectoralis major. These experimental findings characterize actuator contributions in healthy users and provide design guidance for multi-DOF exosuit systems.
Read the full abstract ▾
All authors Rui Chen, Domenico Chiaradia, Daniele Leonardis, Antonio Frisoli
Robotics
Have We Mastered Scale in Deep Monocular Visual SLAM? The ScaleMaster Dataset and Benchmark
Hyoseok Ju, Bokeon Suh and Giseop Kim
Recent advances in deep
monocular visual Simultaneous Localization and Mapping (SLAM) have achieved impressive accuracy and dense reconstruction capabilities, yet their robustness to scale inconsistency in large-scale indoor environments remains largely unexplored. Existing
benchmarks are limited to room-scale or structurally simple settings, leaving critical issues of intra-session scale drift and inter-session scale ambiguity insufficiently addressed. To fill this gap, we introduce the ScaleMaster
Dataset, the first
benchmark explicitly designed to evaluate scale consistency under challenging scenarios such as multi-floor structures, long trajectories, repetitive views, and low-texture regions. We systematically analyze the vulnerability of state-of-the-art deep
monocular visual SLAM systems to scale inconsistency, providing both quantitative and qualitative evaluations. Crucially, our analysis extends beyond traditional trajectory metrics to include a direct map-to-map quality assessment using metrics like Chamfer distance against high-fidelity 3D ground truth. Our results reveal that while recent deep
monocular visual SLAM systems demonstrate strong performance on existing
benchmarks, they suffer from severe scale-related failures in realistic, large-scale indoor environments. By releasing the ScaleMaster
dataset and baseline results, we aim to establish a foundation for future research toward developing scale-consistent and reliable visual SLAM systems.
Read the full abstract ▾
Robotics
GrandTour: A Legged Robotics Dataset in the Wild for Multi-Modal Perception and State Estimation
Turcan Tuna, Jonas Frey, Frank Fu et al. (+5 more)
Accurate state estimation and multi-modal perception are prerequisites for autonomous legged
robots in complex, large-scale environments. To date, no large-scale public legged-
robot dataset captures the real-world conditions needed to develop and
benchmark algorithms for legged-
robot state estimation, perception, and
navigation. To address this, we introduce the GrandTour
dataset, a multi-modal legged-robotics
dataset collected across challenging outdoor and indoor environments, featuring an ANYbotics ANYmal-D quadruped equipped with the Boxi multi-modal sensor payload. GrandTour spans a broad range of environments and operational scenarios across distinct test sites, ranging from alpine scenery and forests to demolished buildings and urban areas, and covers a wide variation in scale, complexity, illumination, and weather conditions. The
dataset provides time-synchronized sensor data from spinning
LiDARs, multiple RGB cameras with complementary characteristics, proprioceptive sensors, and stereo depth cameras. Moreover, it includes high-precision ground-truth trajectories from satellite-based RTK-GNSS and a Leica Geosystems total station. This
dataset supports research in SLAM, high-precision state estimation, and multi-modal learning, enabling rigorous evaluation and development of new approaches to sensor fusion in legged robotic systems. With its extensive scope, GrandTour represents the largest open-access legged-robotics
dataset to date. The
dataset is available at https://grand-tour.leggedrobotics.com on HuggingFace (ROS-independent), and in ROS formats, along with tools and demo resources.
Read the full abstract ▾
All authors Turcan Tuna, Jonas Frey, Frank Fu, Katharine Patterson, Tianao Xu, Maurice Fallon, Cesar Cadena, Marco Hutter
Robotics
Interacting safely with cyclists using Hamilton-Jacobi reachability and reinforcement learning
Aarati Andrea Noronha and Jean Oh
In this paper, we present a framework for enabling autonomous vehicles to interact with cyclists in a manner that balances safety and optimality. The approach integrates Hamilton-Jacobi reachability analysis with deep Q-learning to jointly address safety guarantees and time-efficient
navigation. A value function is computed as the solution to a time-dependent Hamilton-Jacobi-Bellman inequality, providing a quantitative measure of safety for each system state. This safety metric is incorporated as a structured reward signal within a reinforcement learning framework. The method further models the cyclist's latent response to the vehicle, allowing disturbance inputs to reflect human comfort and behavioral adaptation. The proposed framework is evaluated through simulation and comparison with human driving behavior and an existing state-of-the-art method.
Read the full abstract ▾
Robotics · Cross-listed from cs.CR
Dynamic Deception: When Pedestrians Team Up to Fool Autonomous Cars
Masoud Jamshidiyan Tehrani, Marco Gabriel, Jinhan Kim et al. (+1 more)
Many
adversarial attacks on autonomous-driving perception models fail to cause system-level failures once deployed in a full driving stack. The main reason for such ineffectiveness is that once deployed in a system (e.g., within a simulator), attacks tend to be spatially or temporally short-lived, due to the vehicle's dynamics, hence rarely influencing the vehicle behaviour. In this paper, we address both limitations by introducing a system-level attack in which multiple dynamic elements (e.g., two pedestrians) carry
adversarial patches (e.g., on cloths) and jointly amplify their effect through coordination and motion. We evaluate our attacks in the CARLA simulator using a state-of-the-art autonomous driving
agent. At the system level, single-pedestrian attacks fail in all runs (out of 10), while dynamic collusion by two pedestrians induces full vehicle stops in up to 50% of runs, with static collusion yielding no successful attack at all. These results show that system-level failures arise only when
adversarial signals persist over time and are amplified through coordinated actors, exposing a gap between model-level robustness and end-to-end safety.
Read the full abstract ▾
All authors Masoud Jamshidiyan Tehrani, Marco Gabriel, Jinhan Kim, Paolo Tonella