Robotics
Kei Suzuki, Jing Liu, Ye Wang et al. (+4 more)
Cross-
robot policy learning – training a single policy to perform well across multiple embodiments – remains a central challenge in
robot learning.
Transformer-based policies, such as
vision-
language-action (VLA) models, are typically embodiment-agnostic and must infer kinematic structure purely from observations, which can reduce robustness across embodiments and even limit performance within a single embodiment. We propose an embodiment-aware
transformer policy that injects morphology via three mechanisms: (1) kinematic
tokens that factorize actions across joints and compress time through per-joint temporal chunking; (2) a
topology-aware
attention bias that encodes kinematic
topology as an inductive bias in self-
attention, encouraging message passing along kinematic edges; and (3) joint-attribute conditioning that augments
topology with per-joint descriptors to capture semantics beyond connectivity. Across a range of embodiments, this structured integration consistently improves performance over a vanilla pi0.5 VLA baseline, indicating improved robustness both within an embodiment and across embodiments.
Read the full abstract ▾
All authors Kei Suzuki, Jing Liu, Ye Wang, Chiori Hori, Matthew Brand, Diego Romeres, Toshiaki Koike-Akino
Robotics · Cross-listed from eess.SY
Design Framework and Manufacturing of an Active Magnetic Bearing Spindle for Micro-Milling Applications
Kazi Sher Ahmed and Bekir Bediz
Micro-milling spindles require high rotational speeds where conventional rolling element bearings face limitations such as friction and thermal expansion. Active
magnetic bearings (AMBs) address these challenges by providing non-contact and lubrication-free operation at ultra-high speeds with the ability to actively regulate spindle dynamics. The existing literature on AMB spindles has mainly reported specific prototype realizations or control system implementations for specific spindle dynamics. Consequently, design knowledge remains fragmented across isolated successful studies. This paper addresses this gap by presenting a systematic and iterative framework to design and manufacture a micro-milling AMB spindle. The process involves a multidisciplinary design flow with a focus on critical practical aspects of manufacturing. The realized spindle is reported as a case study.
Read the full abstract ▾
Robotics
EgoMoD: Predicting Global Maps of Dynamics from Local Egocentric Observations
Iacopo Catalano, David Morilla-Cabello, Jorge Pena-Queralta et al. (+1 more)
Efficient
navigation in dynamic environments requires anticipating how motion patterns evolve beyond the
robot's immediate perceptual range, enabling preemptive rather than purely reactive planning in crowded scenes. Maps of Dynamics (MoDs) offer a structured representation of motion tendencies in space useful for long-term global planning, but constructing them traditionally requires global environment observations over extended periods of time. We introduce EgoMoD, the first approach that learns to predict future MoDs directly from short
egocentric video clips collected during
robot operation. Our method learns to infer environment-wide motion tendencies from local dynamic cues using a
video- and pose-conditioned architecture trained with MoDs computed from external observations as privileged supervision, allowing local observations to serve as predictive signals of global motion structure. Thanks to this, we offer the capacity to forecast future motion dynamics over the whole environment rather than merely extend past patterns in the
robot's field of view. As a site-specific dynamic prior, EgoMoD replaces the external global sensing infrastructure required by prior MoD methods at inference time with standard onboard sensors. Experiments in large simulated environments show that EgoMoD predicts future MoDs under limited observability, while evaluation with real images showcases its zero-shot transferability to real systems.
Read the full abstract ▾
All authors Iacopo Catalano, David Morilla-Cabello, Jorge Pena-Queralta, Eduardo Montijano
Robotics
V-MORALS: Visual Morse Graph-Aided Estimation of Regions of Attraction in a Learned Latent Space
Faiz Aladin, Ashwin Balasubramanian, Lars Lindemann et al. (+1 more)
Reachability analysis has become increasingly important in robotics to distinguish safe from unsafe states. Unfortunately, existing reachability and safety analysis methods often fall short, as they typically require known system dynamics or large
datasets to estimate accurate system models, are computationally expensive, and assume full state information. A recent method, called MORALS, aims to address these shortcomings by using topological tools to estimate Regions of Attraction (ROA) in a low-dimensional latent space. However, MORALS still relies on full state knowledge and has not been studied when only sensor measurements are available. This paper presents Visual Morse Graph-Aided Estimation of Regions of Attraction in a Learned Latent Space (V-MORALS). V-MORALS takes in a
dataset of image-based trajectories of a system under a given controller, and learns a latent space for reachability analysis. Using this learned latent space, our method is able to generate well-defined Morse Graphs, from which we can compute ROAs for various systems and controllers. V-MORALS provides capabilities similar to the original MORALS architecture without relying on state knowledge, and using only high-level sensor data. Our project website is at: https://v-morals.onrender.com.
Read the full abstract ▾
All authors Faiz Aladin, Ashwin Balasubramanian, Lars Lindemann, Daniel Seita
Robotics
TaCarla: A comprehensive benchmarking dataset for end-to-end autonomous driving
Tugrul Gorgulu, Atakan Dag, M. Esat Kalfaoglu et al. (+4 more)
Collecting a high-quality
dataset is a critical task that demands meticulous
attention to detail, as overlooking certain aspects can render the entire
dataset unusable. Autonomous driving challenges remain a prominent area of research, requiring further exploration to enhance the perception and planning performance of vehicles. However, existing
datasets are often incomplete. For instance,
datasets that include perception information generally lack planning data, while planning
datasets typically consist of extensive driving sequences where the ego vehicle predominantly drives forward, offering limited behavioral diversity. In addition, many real
datasets struggle to evaluate their models, especially for planning tasks, since they lack a proper closed-loop evaluation setup. The CARLA Leaderboard 2.0 challenge, which provides a diverse set of scenarios to address the long-tail problem in autonomous driving, has emerged as a valuable alternative platform for developing perception and planning models in both open-loop and closed-loop evaluation setups. Nevertheless, existing
datasets collected on this platform present certain limitations. Some
datasets appear to be tailored primarily for limited sensor configuration, with particular sensor configurations. To support end-to-end autonomous driving research, we have collected a new
dataset comprising over 2.85 million frames using the CARLA simulation environment for the diverse Leaderboard 2.0 challenge scenarios. Our
dataset is designed not only for planning tasks but also supports dynamic object detection, lane divider detection, centerline detection, traffic light recognition, prediction tasks and visual
language action models . Furthermore, we demonstrate its versatility by training various models using our
dataset. Moreover, we also provide numerical rarity scores to understand how rarely the current state occurs in the
dataset.
Read the full abstract ▾
All authors Tugrul Gorgulu, Atakan Dag, M. Esat Kalfaoglu, Halil Ibrahim Kuru, Baris Can Cam, Halil Ibrahim Ozturk, Ozsel Kilinc
Robotics
Refining Almost-Safe Value Functions on the Fly
Sander Tonkens, Sosuke Kojima, Chenhao Liu et al. (+2 more)
Control Barrier Functions (CBFs) are a powerful tool for ensuring robotic safety, but designing or learning valid CBFs for complex systems is a significant challenge. While Hamilton-Jacobi Reachability provides a formal method for synthesizing safe value functions, it scales poorly and is typically performed offline, limiting its applicability in dynamic environments. This paper bridges the gap between offline synthesis and online adaptation. We introduce refineCBF for refining an approximate CBF - whether analytically derived, learned, or even unsafe - via warm-started HJ reachability. We then present its computationally efficient successor, HJ-Patch, which accelerates this process through localized updates. Both methods guarantee the recovery of a safe value function and can ensure monotonic safety improvements during adaptation. Our experiments validate our framework's primary contribution: in-the-loop, real-time adaptation, in simulation (with detailed value function analysis) and on physical hardware. Our experiments on ground vehicles and quadcopters show that our framework can successfully adapt to sudden environmental changes, such as new obstacles and unmodeled wind disturbances, providing a practical path toward deploying formally guaranteed safety in real-world settings.
Read the full abstract ▾
All authors Sander Tonkens, Sosuke Kojima, Chenhao Liu, Judy Masri, Sylvia Herbert
Robotics · Cross-listed from cs.MA
Optimization of Edge Directions and Weights for Mixed Guidance Graphs in Lifelong Multi-Agent Path Finding
Yulun Zhang, Varun Bhatt, Matthew C. Fontaine et al. (+2 more)
Multi-
Agent Path Finding (MAPF) aims to move
agents from their start to goal vertices on a graph. Lifelong MAPF (LMAPF) continuously assigns new goals to
agents as they complete current ones. To guide
agents' movement in LMAPF, prior works have proposed Guidance Graph Optimization (GGO) methods to optimize a guidance graph, which is a bidirected weighted graph whose directed edges represent moving and waiting actions with edge weights being action costs. Higher edge weights represent higher action costs. However, edge weights only provide soft guidance. An edge with a high weight only discourages
agents from using it, instead of prohibiting
agents from traversing it. In this paper, we explore the need to incorporate edge directions optimization into GGO, providing strict guidance. We generalize GGO to Mixed Guidance Graph Optimization (MGGO), presenting two MGGO methods capable of optimizing both edge weights and directions. The first optimizes edge directions and edge weights in two phases separately. The second applies Quality Diversity
algorithms to optimize a neural network capable of generating edge directions and weights. We also incorporate traffic patterns relevant to edge directions into a GGO method, making it capable of generating edge-direction-aware guidance graphs.
Read the full abstract ▾
All authors Yulun Zhang, Varun Bhatt, Matthew C. Fontaine, Stefanos Nikolaidis, Jiaoyang Li
Robotics
Printed helicoids with embedded air channels make sensorized segments for soft continuum robots
Annan Zhang, Hanna Matusik, Miguel Flores-Acton et al. (+3 more)
Soft
robots enable safe, adaptive interaction with complex environments but remain difficult to sense and control due to their highly deformable structures. Architected soft materials such as helicoid
lattices offer tunable stiffness and strength but are challenging to instrument because of their sparse geometry. We introduce a fabrication method for
embedding air channels into helicoid-based soft continuum
robots. Multi-material segments fabricated via
vision-controlled jetting in a single print interface with PCBs housing miniature pressure sensors and IMUs for distributed deformation sensing. We characterize the mechanical properties of four helicoid designs and validate the sensor response to fundamental deformation modes. To demonstrate the platform's scalability, we construct and mechanically evaluate a meter-scale, 14-DoF cable-driven soft arm capable of open-loop trajectory tracking and object grasping, with tactile-based stiffness detection demonstrated using the gripper sensors. This approach establishes a scalable fabrication strategy for sensorized architected materials in large-scale soft robotic systems.
Read the full abstract ▾
All authors Annan Zhang, Hanna Matusik, Miguel Flores-Acton, Emily R. Sologuren, Joshua Jacob, Daniela Rus
Robotics
Demystifying Action Space Design for Robotic Manipulation Policies
Yuchun Feng, Jinliang Zheng, Zhihao Wang et al. (+5 more)
The specification of the action space plays a pivotal role in imitation-based robotic manipulation policy learning, fundamentally shaping the optimization landscape of policy learning. While recent advances have focused heavily on scaling training data and model capacity, the choice of action space remains guided by ad-hoc heuristics or legacy designs, leading to an ambiguous understanding of robotic policy design philosophies. To address this ambiguity, we conducted a large-scale and systematic empirical study, confirming that the action space does have significant and complex impacts on robotic policy learning. We dissect the action design space along temporal and spatial axes, facilitating a structured analysis of how these choices govern both policy learnability and control stability. Based on 13,000+ real-world rollouts on a bimanual
robot and evaluation on 500+ trained models over four scenarios, we examine the trade-offs between absolute vs. delta representations, and joint-space vs. task-space parameterizations. Our large-scale results suggest that properly designing the policy to predict delta actions consistently improves performance, while joint-space and task-space representations offer complementary strengths, favoring control stability and generalization, respectively.
Read the full abstract ▾
All authors Yuchun Feng, Jinliang Zheng, Zhihao Wang, Dongxiu Liu, Jianxiong Li, Jiangmiao Pang, Tai Wang, Xianyuan Zhan
Robotics
Cybersecurity of Teleoperated Quadruped Robots: A Systematic Survey of Vulnerabilities, Threats, and Open Defense Gaps
Mohammad Sabouri
Teleoperated quadruped
robots are increasingly deployed in safety-critical missions – industrial inspection, military reconnaissance, and emergency response – yet the security of their communication and control infrastructure remains insufficiently characterized. Quadrupeds present distinct security challenges arising from dynamic stability constraints, gait-dependent vulnerability windows, substantial kinetic energy, and elevated operator cognitive load. This survey synthesizes peer-reviewed literature and vulnerability disclosures (2019–2025) to provide comprehensive analysis of cybersecurity threats, consequences, and countermeasures for teleoperated quadruped systems. We contribute: (i) a six-layer attack taxonomy spanning perception manipulation, VR/AR operator targeting, communication disruption, control signal attacks, localization spoofing, and network intrusion; (ii) systematic attack-to-consequence mapping with timing characterization; (iii) Technology Readiness Level classification exposing critical maturity gaps between field-deployed communication protections (TRL 7–9) and experimental perception/operator-layer defenses (TRL 3–5); (iv) comparative security analysis of six commercial platforms; (v) pragmatic deployment guidance stratified by implementation timeline; and (vi) eight prioritized research gaps with implementation roadmaps. Limitations: Platform assessments rely on publicly available information. Attack success rates derive from cited studies under controlled conditions and require domain-specific validation.
Read the full abstract ▾
Robotics · Cross-listed from cs.HC
Evaluating Zero-Shot and One-Shot Adaptation of Small Language Models in Leader-Follower Interaction
Rafael R. Baptista, André de Lima Salgado, Ricardo V. Godoy et al. (+3 more)
Leader-follower interaction is an important paradigm in human-
robot interaction (HRI). Yet, assigning roles in real time remains challenging for resource-constrained mobile and assistive
robots. While large
language models (LLMs) have shown promise for natural communication, their size and latency limit on-device deployment. Small
language models (SLMs) offer a potential alternative, but their effectiveness for role classification in HRI has not been systematically evaluated. In this paper, we present a
benchmark of SLMs for leader-follower communication, introducing a novel
dataset derived from a published database and augmented with synthetic samples to capture interaction-specific dynamics. We investigate two adaptation strategies:
prompt engineering and fine-tuning, studied under zero-shot and one-shot interaction modes, compared with an untrained baseline. Experiments with Qwen2.5-0.5B reveal that zero-shot fine-tuning achieves robust classification performance (86.66% accuracy) while maintaining low latency (22.2 ms per sample), significantly outperforming baseline and
prompt-engineered approaches. However, results also indicate a performance degradation in one-shot modes, where increased context length challenges the model's architectural capacity. These findings demonstrate that fine-tuned SLMs provide an effective solution for direct role assignment, while highlighting critical trade-offs between dialogue complexity and classification reliability on the edge.
Read the full abstract ▾
All authors Rafael R. Baptista, André de Lima Salgado, Ricardo V. Godoy, Marcelo Becker, Thiago Boaventura, Gustavo J. G. Lahr
Robotics
Interface-Aware Trajectory Reconstruction of Limited Demonstrations for Robot Learning
Demiana R. Barsoum, Mahdieh Nejati Javaremi, Larisa Y.C. Loke et al. (+1 more)
Assistive
robots offer agency to humans with severe motor impairments. Often, these users control high-DoF
robots through low-dimensional interfaces, such as using a 1-D sip-and-puff interface to operate a 6-DoF robotic arm. This mismatch results in having access to only a subset of control dimensions at a given time, imposing unintended and artificial constraints on
robot motion. As a result, interface-limited demonstrations embed suboptimal motions that reflect interface restrictions rather than user intent. To address this, we present a trajectory reconstruction
algorithm that reasons about task, environment, and interface constraints to lift demonstrations into the
robot's full control space. We evaluate our approach using real-world demonstrations of ADL-inspired tasks performed via a 2-D joystick and 1-D sip-and-puff control interface, teleoperating two distinct 7-DoF robotic arms. Analyses of the reconstructed demonstrations and derived control policies show that lifted trajectories are faster and more efficient than their interface-constrained counterparts while respecting user preferences.
Read the full abstract ▾
All authors Demiana R. Barsoum, Mahdieh Nejati Javaremi, Larisa Y.C. Loke, Brenna D. Argall
Robotics
Simple Models, Real Swimming: Digital Twins for Tendon-Driven Underwater Robots
Mike Y. Michelis, Nana Obayashi, Josie Hughes et al. (+1 more)
Mimicking the graceful motion of swimming animals remains a core challenge in soft robotics due to the complexity of fluid-structure interaction and the difficulty of controlling soft, biomimetic bodies. Existing modeling approaches are often computationally expensive and impractical for complex control or reinforcement learning needed for realistic motions to emerge in robotic systems. In this work, we present a tendon-driven fish
robot modeled in an efficient
underwater swimmer environment using a simplified, stateless hydrodynamics formulation implemented in the widespread robotics framework MuJoCo. With just two real-world swimming trajectories, we identify five fluid parameters that allow a matching to experimental behavior and generalize across a range of actuation frequencies. We show that this stateless fluid model can generalize to unseen actuation and outperform classical analytical models such as the elongated body theory. This simulation environment runs faster than real-time and can easily enable downstream learning
algorithms such as reinforcement learning for target tracking, reaching a 93% success rate. Due to the simplicity and ease of use of the model and our open-source simulation environment, our results show that even simple, stateless models – when carefully matched to physical data – can serve as effective digital twins for soft
underwater robots, opening up new directions for scalable learning and control in aquatic environments.
Read the full abstract ▾
All authors Mike Y. Michelis, Nana Obayashi, Josie Hughes, Robert K. Katzschmann
Robotics · Cross-listed from cs.LG
Mollified Value Learning
Hrishikesh Viswanath, Juanwu Lu, S. Talha Bukhari et al. (+4 more)
Offline goal-conditioned reinforcement learning (GCRL) learns goal-reaching behaviors from static
datasets, but accurate value estimation remains challenging under limited state-action coverage. Existing physics-informed approaches address this by imposing pointwise distance-like
geometric constraints derived from Hamilton–Jacobi–Bellman (HJB) optimality principles, often through first-order partial differential equations such as the Eikonal equation. However, enforcing local consistency through explicit differential structure can become unstable in complex, high-dimensional environments. Our key insight is to instead reinterpret distance-like constraints as an expectation over a local spatial measure. By aggregating constraints over this measure rather than evaluating them pointwise, the objective acts as a spatial mollifier, inducing distance-like value geometry without requiring expensive differential operators. We refer to this as Mollified Value Learning (MVL). Experiments across
navigation and high-dimensional robotic manipulation tasks show that MVL learns structured, value representations, improving goal-reaching performance, when used with implicit value representation learning methods. Open-source codes are available at https://github.com/HrishikeshVish/MVL.
Read the full abstract ▾
All authors Hrishikesh Viswanath, Juanwu Lu, S. Talha Bukhari, Mihir Chauhan, Damon Conover, Ziran Wang, Aniket Bera
Robotics · Cross-listed from cs.CV
Risk-Aware World Model Predictive Control for Generalizable End-to-End Autonomous Driving
Jiangxin Sun, Feng Xue, Teng Long et al. (+4 more)
With advances in imitation learning (IL) and large-scale driving
datasets, end-to-end autonomous driving (E2E-AD) has made great progress recently. Currently, IL-based methods have become a mainstream paradigm: models rely on standard driving behaviors given by experts, and learn to minimize the discrepancy between their actions and expert actions. However, this objective of "only driving like the expert" suffers from limited generalization: when encountering rare or unseen long-tail scenarios outside the distribution of expert demonstrations, models tend to produce unsafe decisions in the absence of prior experience. This raises a fundamental question: Can an E2E-AD system make reliable decisions without any expert action supervision? Motivated by this, we propose a unified framework named Risk-aware World Model Predictive Control (RaWMPC) to address this generalization dilemma through robust control, without reliance on expert demonstrations. Practically, RaWMPC leverages a world model to predict the consequences of multiple candidate actions and selects low-risk actions through explicit risk evaluation. To endow the world model with the ability to predict the outcomes of risky driving behaviors, we design a risk-aware interaction strategy that systematically exposes the world model to hazardous behaviors, making catastrophic outcomes predictable and thus avoidable. Furthermore, to generate low-risk candidate actions at test time, we introduce a self-evaluation
distillation method to distill riskavoidance capabilities from the well-trained world model into a generative action proposal network without any expert demonstration. Extensive experiments show that RaWMPC outperforms state-of-the-art methods in both in-distribution and out-of-distribution scenarios, while providing superior decision interpretability.
Read the full abstract ▾
All authors Jiangxin Sun, Feng Xue, Teng Long, Chang Liu, Jian-Fang Hu, Wei-Shi Zheng, Nicu Sebe
Robotics
SPARR: Simulation-based Policies with Asymmetric Real-world Residuals for Assembly
Yijie Guo, Iretiayo Akinola, Lars Johannsmeier et al. (+3 more)
Robotic assembly presents a long-standing challenge due to its requirement for precise, contact-rich manipulation. While simulation-based learning has enabled the development of robust assembly policies, their performance often degrades when deployed in real-world settings due to the sim-to-real gap. Conversely, real-world reinforcement learning (RL) methods avoid the sim-to-real gap, but rely heavily on human supervision and lack generalization ability to environmental changes. In this work, we propose a hybrid approach that combines a simulation-trained base policy with a real-world residual policy to efficiently adapt to real-world variations. The base policy, trained in simulation using low-level state observations and dense rewards, provides strong priors for initial behavior. The residual policy, learned in the real world using visual observations and sparse rewards, compensates for discrepancies in dynamics and sensor noise. Extensive real-world experiments demonstrate that our method, SPARR, achieves near-perfect success rates across diverse two-part assembly tasks. Compared to the state-of-the-art zero-shot sim-to-real methods, SPARR improves success rates by 38.4% while reducing cycle time by 29.7%. Moreover, SPARR requires no human expertise, in contrast to the state-of-the-art real-world RL approaches that depend heavily on human supervision.
Read the full abstract ▾
All authors Yijie Guo, Iretiayo Akinola, Lars Johannsmeier, Hugo Hadfield, Abhishek Gupta, Yashraj Narang
Robotics · Cross-listed from cs.CV
UniScale: Unified Scale-Aware 3D Reconstruction for Multi-View Understanding via Prior Injection for Robotic Perception
Mohammad Mahdavian, Gordon Tan, Binbin Xu et al. (+3 more)
We present UniScale, a unified, scale-aware multi-view 3D reconstruction framework for robotic applications that flexibly integrates
geometric priors through a modular, semantically informed design. In
vision-based robotic
navigation, the accurate extraction of environmental structure from raw image sequences is critical for downstream tasks. UniScale addresses this challenge with a single feed-forward network that jointly estimates camera intrinsics and extrinsics, scale-invariant depth and point maps, and the metric scale of a scene from multi-view images, while optionally incorporating auxiliary
geometric priors when available. By combining global contextual
reasoning with camera-aware feature representations, UniScale is able to recover the metric-scale of the scene. In robotic settings where camera intrinsics are known, they can be easily incorporated to improve performance, with additional gains obtained when camera poses are also available. This co-design enables robust, metric-aware 3D reconstruction within a single unified model. Importantly, UniScale does not require training from scratch, and leverages world priors exhibited in pre-existing models without
geometric encoding strategies, making it particularly suitable for resource-constrained robotic teams. We evaluate UniScale on multiple
benchmarks, demonstrating strong generalization and consistent performance across diverse environments. We will release our implementation upon acceptance.
Read the full abstract ▾
All authors Mohammad Mahdavian, Gordon Tan, Binbin Xu, Yuan Ren, Dongfeng Bai, Bingbing Liu
Robotics
Grasp, Slide, Roll: Comparative Analysis of Contact Modes for Tactile-Based Shape Reconstruction
Chung Hee Kim, Shivani Kamtikar, Tye Brady et al. (+2 more)
Tactile sensing allows
robots to gather detailed
geometric information about objects through physical interaction, complementing
vision-based approaches. However, efficiently acquiring useful tactile data remains challenging due to the time-consuming nature of physical contact and the need to strategically choose contact locations that maximize information gain while minimizing physical interactions. This paper studies how different contact modes affect object shape reconstruction using a tactile-enabled dexterous gripper. We compare three contact interaction modes: grasp-releasing, sliding induced by finger-grazing, and palm-rolling. These contact modes are combined with an information-theoretic exploration framework that guides subsequent sampling locations using a shape completion model. Our results show that the improved tactile sensing efficiency of finger-grazing and palm-rolling translates into faster convergence in shape reconstruction, requiring 34% fewer physical interactions while improving reconstruction accuracy by 55%. We validate our approach using a UR5e
robot arm equipped with an Inspire-
Robots Dexterous Hand, showing robust performance across primitive object geometries.
Read the full abstract ▾
All authors Chung Hee Kim, Shivani Kamtikar, Tye Brady, Taskin Padir, Joshua Migdal
Robotics · Cross-listed from cs.CV
Motion-aware Event Suppression for Event Cameras
Roberto Pellerito, Nico Messikommer, Giovanni Cioffi et al. (+2 more)
Event cameras report asynchronously per-pixel brightness changes with microsecond latency, encoding dynamic visual information as a sparse stream of events. However, their extreme temporal resolution floods perception systems with entangled events from ego-motion and independently moving objects (IMOs), which existing solutions fail to efficiently decouple, relying instead on prohibitive dense 3D reconstructions or limited hand-tuned filters. In this work, we introduce the first framework for Motion-aware Event Suppression, which learns to filter events triggered by IMOs and ego-motion in real time. Our model jointly segments IMOs in the current event stream while predicting their future motion, enabling anticipatory suppression of dynamic events before they occur. Our lightweight architecture achieves 173 Hz inference on consumer-grade GPUs with less than 1 GB of
memory usage, outperforming previous state-of-the-art methods on the challenging EVIMO
benchmark by 67% in
segmentation accuracy while operating at a 53% higher inference rate. Moreover, we demonstrate significant benefits for downstream applications: our method accelerates
Vision Transformer inference by 83% via
token pruning and improves event-based visual odometry accuracy, reducing Absolute Trajectory Error (ATE) by 13%.
Read the full abstract ▾
All authors Roberto Pellerito, Nico Messikommer, Giovanni Cioffi, Marco Cannici, Davide Scaramuzza
Robotics · Cross-listed from cs.CV
Latent Gaussian Splatting for 4D Panoptic Occupancy Tracking
Maximilian Luz, Rohit Mohan, Thomas Nürnberg et al. (+3 more)
Capturing 4D spatiotemporal scene structure is crucial for the safe and reliable operation of
robots in dynamic environments. However, existing approaches typically address only part of the problem: they either provide coarse
geometric tracking via bounding boxes or detailed 3D occupancy estimates that lack explicit temporal association and instance-level
reasoning. In this work, we present Latent Gaussian Splatting (LaGS) for 4D Panoptic Occupancy Tracking (4D-POT). We revisit the underlying representation and model 3D features as a sparse set of feature-bearing Gaussians. These act as dynamic, volume-oriented keypoints that enable spatially continuous, distance-weighted aggregation of multi-view features before being splatted into a voxel grid for decoding. This point-centric formulation enables flexible, data-dependent receptive fields and long-range spatial interactions that are difficult to capture with local and dense voxel-based operators. A hierarchical Gaussian representation further enables multi-scale
reasoning by combining global context from coarse super-points with fine-grained detail from higher-resolution streams. Extensive experiments on Occ3D nuScenes and Waymo demonstrate state-of-the-art performance for 4D-POT. We provide code and models at https://lags.cs.uni-freiburg.de/.
Read the full abstract ▾
All authors Maximilian Luz, Rohit Mohan, Thomas Nürnberg, Yakov Miron, Daniele Cattaneo, Abhinav Valada
Robotics · Cross-listed from cs.CV
FLIGHT: Fibonacci Lattice-based Inference for Geometric Heading in real-Time
David Dirnfeld, Fabien Delattre, Pedro Miraldo et al. (+1 more)
Estimating camera motion from
monocular video is a fundamental problem in computer
vision, central to tasks such as SLAM, visual odometry, and structure-from-motion. Existing methods that recover the camera's heading under known rotation, whether from an IMU or an optimization
algorithm, tend to perform well in low-noise, low-outlier conditions, but often decrease in accuracy or become computationally expensive as noise and outlier levels increase. To address these limitations, we propose a novel generalization of the Hough transform on the unit sphere (S(2)) to estimate the camera's heading. First, the method extracts correspondences between two frames and generates a great circle of directions compatible with each pair of correspondences. Then, by discretizing the unit sphere using a Fibonacci
lattice as bin centers, each great circle casts votes for a range of directions, ensuring that features unaffected by noise or dynamic objects vote consistently for the correct motion direction. Experimental results on three
datasets demonstrate that the proposed method is on the Pareto frontier of accuracy versus efficiency. Additionally, experiments on SLAM show that the proposed method reduces RMSE by correcting the heading during camera pose initialization.
Read the full abstract ▾
All authors David Dirnfeld, Fabien Delattre, Pedro Miraldo, Erik Learned-Miller
Robotics
Towards Intelligible Human-Robot Interaction: An Active Inference Approach to Occluded Pedestrian Scenarios
Kai Chen, Yuyao Huang and Guang Chen
The sudden appearance of occluded pedestrians presents a critical safety challenge in autonomous driving. Conventional rule-based or purely data-driven approaches struggle with the inherent high uncertainty of these long-tail scenarios. To tackle this challenge, we propose a novel framework grounded in Active Inference, which endows the
agent with a human-like, belief-driven mechanism. Our framework leverages a Rao-Blackwellized Particle Filter (RBPF) to efficiently estimate the pedestrian's hybrid state. To emulate human-like cognitive processes under uncertainty, we introduce a Conditional Belief Reset mechanism and a Hypothesis Injection technique to explicitly model beliefs about the pedestrian's multiple latent intentions. Planning is achieved via a Cross-
Entropy Method (CEM) enhanced Model Predictive Path Integral (MPPI) controller, which synergizes the efficient, iterative search of CEM with the inherent robustness of MPPI. Simulation experiments demonstrate that our approach significantly reduces the collision rate compared to reactive, rule-based, and reinforcement learning (RL) baselines, while also exhibiting explainable and human-like driving behavior that reflects the
agent's internal belief state.
Read the full abstract ▾
Robotics · Cross-listed from cs.CV
GeoWorld: Geometric World Models
Zeyu Zhang, Danning Li, Ian Reid et al. (+1 more)
Energy-based predictive world models provide a powerful approach for multi-step visual planning by
reasoning over latent energy landscapes rather than generating pixels. However, existing approaches face two major challenges: (i) their latent representations are typically learned in Euclidean space, neglecting the underlying
geometric and hierarchical structure among states, and (ii) they struggle with long-horizon prediction, which leads to rapid degradation across extended rollouts. To address these challenges, we introduce GeoWorld, a
geometric world model that preserves
geometric structure and hierarchical relations through a
Hyperbolic JEPA, which maps latent representations from Euclidean space onto
hyperbolic manifolds. We further introduce
Geometric Reinforcement Learning for energy-based optimization, enabling stable multi-step planning in
hyperbolic latent space. Extensive experiments on CrossTask and COIN demonstrate around 3% SR improvement in 3-step planning and 2% SR improvement in 4-step planning compared to the state-of-the-art V-JEPA 2. Project website: https://steve-zeyu-zhang.github.io/GeoWorld.
Read the full abstract ▾
All authors Zeyu Zhang, Danning Li, Ian Reid, Richard Hartley
Robotics
Marinarium: A Modular Experimental Facility for Reproducible Maritime and Space-Analog Field Robotics
Ignacio Torroba, David Dorner, Victor Nan Fernandez-Ayala et al. (+14 more)
Field robotics research in maritime and space domains is constrained by a persistent gap between low-cost, low-fidelity simulation and costly offshore experimentation. Instrumented water tanks partially bridge this gap but often provide limited sensing, restricted experimental capabilities, and weak integration with simulation tools. To address these limitations, we present Marinarium, a modular, standalone experimental facility that provides a cost-effective intermediate testbed between simulation and field deployment. Marinarium combines a fully instrumented
underwater and aerial operational volume with motion capture (MoCap), a retractable roof enabling both sheltered and open-air operation, a digital twin implemented in SMaRCSim, and direct integration with a planar space robotics laboratory, enabling both maritime and
underwater space-analog experimentation. We present the design rationale of the facility and validate its capabilities through four representative studies in field robotics: (i) data-driven system identification of
underwater vehicle dynamics; (ii) heterogeneous multi-domain robotic rendezvous; (iii) sim-to-real transfer for
underwater robotics using learned dynamics residuals; and (iv) cross-domain validation of spacecraft autonomy using
underwater surrogates. Together, these studies demonstrate that Marinarium enables reproducible, instrumented experimentation across multiple field robotics challenges that would otherwise require costly offshore deployments or be impractical to investigate using simulation alone.
Read the full abstract ▾
All authors Ignacio Torroba, David Dorner, Victor Nan Fernandez-Ayala, Mart Kartasev, Joris Verhagen, Elias Krantz, Gregorio Marchesini, Carl Ljung, Pedro Roque, Chelsea Sidrane, Linda Van der Spaa, Nicola De Carli, Petter Ogren, Christer Fuglesang, Jana Tumova, Dimos V. Dimarogonas, Ivan Stenius
Robotics
An Empirical Analysis of Cooperative Perception for Occlusion Risk Mitigation
Aihong Wang, Tenghui Xie, Fuxi Wen et al. (+1 more)
Occlusions present a significant challenge for connected and automated vehicles, as they can obscure critical road users from perception systems. Traditional risk metrics often fail to capture the cumulative nature of these threats over time adequately. In this paper, we propose a novel and universal risk assessment metric, the Risk of Tracking Loss (RTL), which aggregates instantaneous risk intensity throughout occluded periods. This provides a holistic risk profile that encompasses both high-intensity, short-term threats and prolonged exposure. Utilizing diverse and high-fidelity real-world
datasets, a large-scale statistical analysis is conducted to characterize occlusion risk and validate the effectiveness of the proposed metric. The metric is applied to evaluate different vehicle-to-everything (V2X) deployment strategies. Our study shows that full V2X penetration theoretically eliminates this risk, the reduction is highly nonlinear; a substantial statistical benefit requires a high penetration threshold of 75-90%. To overcome this limitation, we propose a novel asymmetric communication framework that allows even non-connected vehicles to receive warnings. Experimental results demonstrate that this paradigm achieves better risk mitigation performance. We found that our approach at 25% penetration outperforms the traditional symmetric model at 75%, and benefits saturate at only 50% penetration. This work provides a crucial risk assessment metric and a cost-effective, strategic roadmap for accelerating the safety benefits of V2X deployment.
Read the full abstract ▾
All authors Aihong Wang, Tenghui Xie, Fuxi Wen, Jun Li
Robotics
InCoM: Intent-Driven Perception and Structured Coordination for Mobile Manipulation
Jiahao Liu, Cui Wenbo, Zhongpu Xia et al. (+3 more)
Mobile manipulation is a fundamental capability for general-purpose robotic
agents, requiring both coordinated control of the mobile base and manipulator and robust perception under dynamically changing viewpoints. However, existing approaches face two key challenges: strong coupling between base and arm actions complicates control optimization, and perceptual
attention is often poorly allocated as viewpoints shift during mobile manipulation. We propose InCoM, an intent-driven perception and structured coordination framework for mobile manipulation. InCoM infers latent motion intent to dynamically reweight multi-scale perceptual features, enabling stage-adaptive allocation of perceptual
attention. To support robust cross-modal perception, InCoM further incorporates a
geometric-semantic affinity refinement mechanism that enhances
multimodal correspondence. On the control side, we design a decoupled coordinated flow matching action decoder that explicitly models bidirectional base-arm interactions, alleviating optimization difficulties caused by control coupling. Under matched non-privileged observations and task-specific training data, InCoM improves mean success over DSPv2 by 33.0, 26.1, and 23.6 percentage points on SetTable, TidyHouse, and PrepareGroceries, respectively. Furthermore, its effectiveness is consistently validated in real-world mobile manipulation tasks, where InCoM maintains a superior success rate over existing baselines.
Read the full abstract ▾
All authors Jiahao Liu, Cui Wenbo, Zhongpu Xia, Yongliang Wang, Haoran Li, Dongbin Zhao
Robotics
DigiArm: An Anthropomorphic 3D-Printed Prosthetic Hand with Enhanced Dexterity for Typing Tasks
Dean Zadok, Tom Naamani, Yuval Bar-Ratson et al. (+5 more)
Despite recent advancements, existing prosthetic limbs are unable to replicate the dexterity and intuitive control of the human hand. Current control systems for prosthetic hands are often limited to grasping, and commercial prosthetic hands lack the precision needed for dexterous manipulation or applications that require fine finger motions. Thus, there is a critical need for accessible and replicable prosthetic designs that enable individuals to interact with electronic devices and perform precise finger pressing, such as keyboard typing or piano playing, while preserving current prosthetic capabilities. This paper presents a low-cost, lightweight, 3D-printed robotic prosthetic hand, specifically engineered for enhanced dexterity with electronic devices such as a computer keyboard or piano, as well as general object manipulation. The robotic hand features a mechanism to adjust finger abduction/adduction spacing, a 2-D wrist with the inclusion of controlled ulnar/radial deviation optimized for typing, and control of independent finger pressing. We conducted a study to demonstrate how participants can use the robotic hand to perform keyboard typing and piano playing in real time, with different levels of finger and wrist motion. This supports the notion that our proposed design can allow for the execution of key typing motions more effectively than before, aiming to enhance the functionality of prosthetic hands.
Read the full abstract ▾
All authors Dean Zadok, Tom Naamani, Yuval Bar-Ratson, Elisha Barash, Oren Salzman, Alon Wolf, Alex M. Bronstein, Nili Krausz
Robotics
A Perspective on Open Challenges in Deformable Object Manipulation
Ryan Paul McKennaa and John Oyekan
Deformable object manipulation (DOM) represents a critical challenge in robotics, with applications spanning healthcare, manufacturing, food processing, and beyond. Unlike rigid objects, deformable objects exhibit infinite dimensionality, dynamic shape changes, and complex interactions with their environment, posing significant hurdles for perception, modeling, and control. This paper reviews the state of the art in DOM, focusing on key challenges such as occlusion handling, task generalization, and scalable, real-time solutions. It highlights advancements in
multimodal perception systems, including the integration of multi-camera setups, active
vision, and tactile sensing, which collectively address occlusion and improve adaptability in unstructured environments. Cutting-edge developments in physically informed reinforcement learning (RL) and differentiable simulations are explored, showcasing their impact on efficiency, precision, and scalability. The review also emphasizes the potential of simulated expert demonstrations and generative neural networks to standardize task specifications and bridge the simulation-to-reality gap. Finally, future directions are proposed, including the adoption of graph neural networks for high-level decision-making and the creation of comprehensive
datasets to enhance DOM's real-world applicability. By addressing these challenges, DOM research can pave the way for versatile robotic systems capable of handling diverse and dynamic tasks with deformable objects.
Read the full abstract ▾
Robotics
Automated Robotic Needle Puncture for Percutaneous Dilatational Tracheostomy
Yuan Tang, Bruno V. Adorno, Brendan A. McGrath et al. (+1 more)
Percutaneous dilatational tracheostomy (PDT) is frequently performed on patients in intensive care units for prolonged mechanical ventilation. The needle puncture, as the most critical step of PDT, could lead to adverse consequences such as major bleeding and posterior tracheal wall perforation if performed inaccurately. Current practices of PDT puncture are all performed manually with no
navigation assistance, which leads to large position and angular errors (5 mm and 30 degree). To improve the accuracy and reduce the difficulty of the PDT procedure, we propose a system that automates the needle insertion using a velocity-controlled robotic manipulator. Guided using pose data from two electromagnetic sensors, one at the needle tip and the other inside the trachea, the robotic system uses an adaptive constrained controller to adapt the uncertain kinematic parameters online and avoid collisions with the patient's body and tissues near the target. Simulations were performed to validate the controller's implementation, and then four hundred PDT punctures were performed on a mannequin to evaluate the position and angular accuracy. The absolute median puncture position error was 1.7 mm (IQR: 1.9 mm) and midline deviation was 4.13 degree (IQR: 4.55 degree), measured by the sensor inside the trachea. The small deviations from the nominal puncture in a simulated experimental setup and formal guarantees of collision-free insertions suggest the feasibility of the robotic PDT puncture.
Read the full abstract ▾
All authors Yuan Tang, Bruno V. Adorno, Brendan A. McGrath, Andrew Weightman
Robotics
Considering Perspectives for Automated Driving Ethics: Collective Risk in Vehicular Motion Planning
Leon Tolksdorf, Arturo Tejada, Christian Birkner et al. (+1 more)
Recent automated vehicle (AV) motion planning strategies evolve around minimizing risk in road traffic. However, they exclusively consider risk from the AV's perspective and, as such, do not address the ethicality of its decisions for other road users. We argue that this does not reduce the risk of each road user, as risk may be different from the perspective of each road user. Indeed, minimizing the risk from the AV's perspective may not imply that the risk from the perspective of other road users is also being minimized; in fact, it may even increase. To test this hypothesis, we propose an AV motion planning strategy that supports switching risk minimization strategies between all road user perspectives. We find that the risk from the perspective of other road users can generally be considered different to the risk from the AV's perspective. Taking a collective risk perspective, i.e., balancing the risks of all road users, we observe an AV that minimizes overall traffic risk the best, while putting itself at slightly higher risk for the benefit of others, which is consistent with human driving behavior. In addition, adopting a collective risk minimization strategy can also be beneficial to the AV's travel efficiency by acting assertively when other road users maintain a low risk estimate of the AV. Yet, the AV drives conservatively when its planned actions are less predictable to other road users, i.e., associated with high risk. We argue that such behavior is a form of self-reflection and a natural prerequisite for socially acceptable AV behavior. We conclude that to facilitate ethicality in road traffic that includes AVs, the risk-perspective of each road user must be considered in the decision-making of AVs.
Read the full abstract ▾
All authors Leon Tolksdorf, Arturo Tejada, Christian Birkner, Nathan van de Wouw