Robotics
SODA-CitrON: Static Object Data Association by Clustering Multi-Modal Sensor Detections Online
Jan Nausner, Kilian Wohlleben and Michael Hubner
The online fusion and tracking of static objects from heterogeneous sensor detections is a fundamental problem in robotics, autonomous systems, and environmental mapping. Although classical data association approaches such as JPDA are well suited for dynamic targets, they are less effective for static objects observed intermittently and with heterogeneous uncertainties, where motion models provide minimal discriminative power with respect to clutter. In this paper, we propose a novel method for static object data association by clustering multi-modal sensor detections online (SODA-CitrON), while simultaneously estimating positions and maintaining persistent tracks for an unknown number of objects. The proposed unsupervised machine learning approach operates in a fully online manner and handles temporally uncorrelated and multi-sensor measurements. Additionally, it has a worst-case loglinear complexity in the number of sensor detections while providing full output explainability. We evaluate the proposed approach in different Monte Carlo simulation scenarios and compare it against state-of-the-art methods, including POM-based filtering, DBSTREAM clustering, and JPDA. The results demonstrate that SODA-CitrON consistently outperforms the compared methods in terms of F1 score, position RMSE, MOTP, and MOTA in the static object mapping scenarios studied.
Read the full abstract ▾
Robotics
VLA Knows Its Limits: Adaptive Execution Horizons for Robot Policies
Haoxuan Wang, Gengyu Zhang, Yan Yan et al. (+2 more)
Action chunking has recently emerged as a standard practice in flow-based
Vision-
Language-Action (VLA) models. However, the effect and choice of the execution horizon - the number of actions to be executed from each predicted chunk - remains underexplored. In this work, we first show that varying the execution horizon leads to substantial performance deviations, with performance initially improving and then declining as the horizon increases. To uncover the reasons, we analyze the cross- and self-
attention weights in flow-based VLAs and reveal two key phenomena: (i) intra-chunk actions attend invariantly to
vision-
language tokens, limiting adaptability to environmental changes; and (ii) the initial and terminal action
tokens serve as stable anchors, forming latent centers around which intermediate actions are organized. Motivated by these insights, we interpret action self-
attention weights as a proxy for the model's predictive limit and propose AutoHorizon, the first test-time method that dynamically estimates the execution horizon for each predicted action chunk to adapt to changing perceptual conditions. Across simulated and real-world robotic manipulation tasks, AutoHorizon is performant, incurs negligible computational overhead, and generalizes across diverse tasks and flow-based models.
Read the full abstract ▾
All authors Haoxuan Wang, Gengyu Zhang, Yan Yan, Ramana Rao Kompella, Gaowen Liu
Robotics
Event-Driven On-Sensor Locomotion Mode Recognition Using a Shank-Mounted IMU with Embedded Machine Learning for Exoskeleton Control
Mohammadsaleh Razmi and Iman Shojaei
This work presents a wearable human activity recognition (HAR) system that performs real-time inference directly inside a shank-mounted inertial measurement unit (IMU) to support low-latency control of a lower-limb exoskeleton. Unlike conventional approaches that continuously stream raw inertial data to a microcontroller for classification, the proposed system executes activity recognition at the sensor level using the embedded Machine Learning Core (MLC) of the STMicroelectronics LSM6DSV16X IMU, allowing the host microcontroller to remain in a low-power state and read only the recognized activity label from IMU registers. While the system generalizes to multiple human activities, this paper focuses on three representative locomotion modes - stance, level walking, and stair ascent - using data collected from adult participants. A lightweight decision-tree model was configured and deployed for on-sensor execution using ST MEMS Studio, enabling continuous operation without custom machine learning code on the microcontroller. During operation, the IMU asserts an interrupt when motion or a new classification is detected; the microcontroller wakes, reads the MLC output registers, and forwards the inferred mode to the exoskeleton controller. This interrupt-driven, on-sensor inference architecture reduces computation and communication overhead while preserving battery energy and improving robustness in distinguishing level walking from stair ascent for torque-assist control.
Read the full abstract ▾
Robotics
Autonomous Sea Turtle Robot for Marine Fieldwork
Zach J. Patterson, Emily Sologuren, Levi Cai et al. (+4 more)
Autonomous
robots can transform how we observe marine ecosystems, but close-range operation in reefs and other cluttered habitats remains difficult. Vehicles must maneuver safely near animals and fragile structures while coping with currents, variable illumination and limited sensing. Previous approaches simplify these problems by leveraging soft materials and bioinspired swimming designs, but such platforms remain limited in terms of deployable autonomy. Here we present a sea turtle-inspired autonomous
underwater robot that closed the gap between bioinspired locomotion and field-ready autonomy through a tightly integrated,
vision-driven control stack. The
robot combines robust depth-heading stabilization with obstacle avoidance and target-centric control, enabling it to track and interact with moving objects in complex terrain. We validate the
robot in controlled pool experiments and in a live coral reef exhibit at the New England Aquarium, demonstrating stable operation and reliable tracking of fast-moving marine animals and human divers. To the best of our knowledge, this is the first integrated biomimetic robotic system, combining novel hardware, control, and field experiments, deployed to track and monitor real marine animals in their natural environment. During off-tether experiments, we demonstrate safe
navigation around obstacles (91% success rate in the aquarium exhibit) and introduce a low-compute onboard tracking mode. Together, these results establish a practical route toward soft-rigid hybrid, bioinspired
underwater robots capable of minimally disruptive exploration and close-range monitoring in sensitive ecosystems.
Read the full abstract ▾
All authors Zach J. Patterson, Emily Sologuren, Levi Cai, Daniel Kim, Alaa Maalouf, Pascal Spino, Daniela Rus
Robotics
Environment-Aware Learning of Smooth GNSS Covariance Dynamics for Autonomous Racing
Y. Deemo Chen, Arion Zimmermann, Thomas A. Berrueta et al. (+1 more)
Ensuring accurate and stable state estimation is a challenging task crucial to safety-critical domains such as high-speed autonomous racing, where measurement uncertainty must be both adaptive to the environment and temporally smooth for control. In this work, we develop a learning-based framework, LACE, capable of directly modeling the temporal dynamics of GNSS measurement covariance. We model the covariance
evolution as an exponentially stable dynamical system where a deep neural network (DNN) learns to predict the system's process noise from environmental features through an
attention mechanism. By using contraction-based stability and systematically imposing spectral constraints, we formally provide guarantees of exponential stability and smoothness for the resulting covariance dynamics. We validate our approach on an AV-24 autonomous racecar, demonstrating improved localization performance and smoother covariance estimates in challenging, GNSS-degraded environments. Our results highlight the promise of dynamically modeling the perceived uncertainty in state estimation problems that are tightly coupled with control sensitivity.
Read the full abstract ▾
All authors Y. Deemo Chen, Arion Zimmermann, Thomas A. Berrueta, Soon-Jo Chung
Robotics
CableRobotGraphSim: A Graph Neural Network for Modeling Partially Observable Cable-Driven Robot Dynamics
Nelson Chen, William R. Johnson III, Rebecca Kramer-Bottiglio et al. (+2 more)
General-purpose simulators have accelerated the development of
robots. Traditional simulators based on first-principles, however, typically require full-state observability or depend on parameter search for system identification. This work presents CableRobotGraphSim, a novel Graph Neural Network (GNN) model for cable-driven
robots that aims to address shortcomings of prior simulation solutions. By representing cable-driven
robots as graphs, with the rigid-bodies as nodes and the cables and contacts as edges, this model can quickly and accurately match the properties of other simulation models and real
robots, while ingesting only partially observable inputs. Accompanying the GNN model is a sim-and-real co-training procedure that promotes generalization and robustness to noisy real data. This model is further integrated with a Model Predictive Path Integral (MPPI) controller for closed-loop
navigation, which showcases the model's speed and accuracy.
Read the full abstract ▾
All authors Nelson Chen, William R. Johnson III, Rebecca Kramer-Bottiglio, Kostas Bekris, Mridul Aanjaneya
Robotics · Cross-listed from cs.LG
Uncertainty-Aware Diffusion Model for Multimodal Highway Trajectory Prediction via DDIM Sampling
Marion Neumeier, Niklas Roßberg, Michael Botsch et al. (+1 more)
Accurate and uncertainty-aware trajectory prediction remains a core challenge for autonomous driving, driven by complex multi-
agent interactions, diverse scene contexts and the inherently
stochastic nature of future motion.
Diffusion-based generative models have recently shown strong potential for capturing
multimodal futures, yet existing approaches such as cVMD suffer from slow sampling, limited exploitation of generative diversity and brittle scenario encodings. This work introduces cVMDx, an enhanced
diffusion-based trajectory prediction framework that improves efficiency, robustness and
multimodal predictive capability. Through DDIM sampling, cVMDx achieves up to a 100x reduction in inference time, enabling practical multi-sample generation for uncertainty estimation. A fitted Gaussian Mixture Model further provides tractable
multimodal predictions from the generated trajectories. In addition, a CVQ-VAE variant is evaluated for scenario encoding. Experiments on the publicly available highD
dataset show that cVMDx achieves higher accuracy and significantly improved efficiency over cVMD, enabling fully
stochastic,
multimodal trajectory prediction.
Read the full abstract ▾
All authors Marion Neumeier, Niklas Roßberg, Michael Botsch, Wolfgang Utschick
Robotics
Unified Complementarity-Based Contact Modeling and Planning for Soft Robots
Milad Azizkhani and Yue Chen
Soft
robots were introduced in large part to enable safe, adaptive interaction with the environment, and this interaction relies fundamentally on contact. However, modeling and planning contact-rich interactions for soft
robots remain challenging: dense contact candidates along the body create redundant constraints and rank-deficient LCPs, while the disparity between high stiffness and low friction introduces severe ill-conditioning. Existing approaches rely on problem-specific approximations or penalty-based treatments. This letter presents a unified complementarity-based framework for soft-
robot contact modeling and planning that brings contact modeling, manipulation, and planning into a unified, physically consistent formulation. We develop a robust Linear Complementarity Problem (LCP) model tailored to discretized soft
robots and address these challenges with a three-stage conditioning pipeline: inertial rank selection to remove redundant contacts, Ruiz equilibration to correct scale disparity and ill-conditioning, and lightweight Tikhonov regularization on normal blocks. Building on the same formulation, we introduce a kinematically guided warm-start strategy that enables dynamic trajectory optimization through contact using Mathematical Programs with Complementarity Constraints (MPCC) and demonstrate its effectiveness on contact-rich ball manipulation tasks. In conclusion, CUSP provides a new foundation for unifying contact modeling, simulation, and planning in soft robotics.
Read the full abstract ▾
Robotics
Learning Dynamic Rope Manipulation Using Task-Level Iterative Learning Control
Krishna Suresh and Chris Atkeson
We introduce a Task-Level Iterative Learning Control method for dynamic manipulation of ropes. We demonstrate this method on a non-planar rope manipulation task called the flying
knot. Using a single human demonstration and a simplified rope model, the method learns directly on hardware without reliance on large amounts of demonstration data or massive amounts of simulation. At each iteration, the
algorithm inverts a model of the
robot and rope by solving a quadratic program to propagate task-space errors into action updates. We evaluate performance across 7 different kinds of ropes, including chain, latex
surgical tubing, and braided and twisted ropes, ranging in thicknesses of 7–25\,mm and densities of 0.013–0.5\,kg/m. Learning achieves a 100% success rate within 10 trials on all ropes. Furthermore, the method can successfully transfer between most rope types in 2–5 trials. https://flying-
knots.github.io
Read the full abstract ▾
Robotics
Dual-Branch INS/GNSS Fusion with Inequality and Equality Constraints
Mor Levenhar and Itzik Klein
Reliable vehicle
navigation in urban environments remains a challenging problem due to frequent satellite signal blockages caused by tall buildings and complex infrastructure. While fusing inertial reading with satellite positioning in an extended Kalman filter provides short-term
navigation continuity, low-cost inertial sensors suffer from rapid error accumulation during prolonged outages. Existing information aiding approaches, such as the non-holonomic constraint, impose rigid equality assumptions on vehicle motion that may be violated under dynamic urban driving conditions, limiting their robustness precisely when aiding is most needed. In this paper, we propose a dual-branch information aiding framework that fuses equality and inequality motion constraints through a variance-weighted scheme, requiring only a software modification to an existing
navigation filter with no additional sensors or hardware. The proposed method is evaluated on four publicly available urban
datasets featuring various inertial sensors, road conditions, and dynamics, covering a total duration of 4.3 hours of recorded data. Under Full GNSS availability, the method reduces vertical position error by 16.7% and improves altitude accuracy by 50.1% over the standard non-holonomic constraint. Under GNSS-denied conditions, vertical drift is reduced by 24.2% and altitude accuracy improves by 20.2%. These results demonstrate that replacing hard motion equality assumptions with physically motivated inequality bounds is a practical and cost-free strategy for improving
navigation resilience, continuity, and drift robustness without relying on additional sensors, map data, or learned models.
Read the full abstract ▾
Robotics
Squint: Fast Visual Reinforcement Learning for Sim-to-Real Robotics
Abdulaziz Almuzairee and Henrik I. Christensen
Visual reinforcement learning is appealing for robotics but expensive. Off-policy methods are sample-efficient yet slow while on-policy methods parallelize well but waste samples. Recent work has shown that off-policy methods can train faster than on-policy methods in wall-clock time for state-based control. Extending this to
vision remains challenging, where high-dimensional input images complicate training dynamics and introduce substantial storage and encoding overhead. To address these challenges, we introduce Squint, a visual Soft Actor Critic method that achieves faster wall-clock training than prior visual off-policy and on-policy methods. Squint achieves this via parallel simulation, a distributional critic, resolution squinting, layer normalization, a tuned update-to-data ratio, and an optimized implementation. We evaluate on the SO-101 Task Set, a new suite of eight manipulation tasks in ManiSkill3 with heavy domain randomization, and demonstrate sim-to-real transfer to a real SO-101
robot. We train policies for 15 minutes on a single RTX 3090 GPU, with most tasks converging in under 6 minutes.
Read the full abstract ▾
Robotics · Cross-listed from cs.LG
Learning from Trials and Errors: Reflective Test-Time Planning for Embodied LLMs
Yining Hong, Huang Huang, Manling Li et al. (+4 more)
Embodied LLMs endow
robots with high-level task
reasoning, but they cannot reflect on what went wrong or why, turning deployment into a sequence of independent trials where mistakes repeat rather than accumulate into experience. Drawing upon human reflective practitioners, we introduce Reflective Test-Time Planning, which integrates two modes of reflection: reflection-in-action, where the
agent uses test-time scaling to generate and score multiple candidate actions using internal reflections before execution; and reflection-on-action, which uses test-time training to update both its internal reflection model and its action policy based on external reflections after execution. We also include retrospective reflection, allowing the
agent to re-evaluate earlier decisions and perform model updates with hindsight for proper long-horizon credit assignment. Experiments on our newly-designed Long-Horizon Household
benchmark and MuJoCo Cupboard Fitting
benchmark show significant gains over baseline models, with zero-shot generalization to photorealistic HM3D environments and real-
robot experiments on a Franka Panda arm. Ablations confirm that reflection-in-action and reflection-on-action are mutually dependent, and that retrospective reflection achieves better credit assignment than step-wise external feedback at lower computational overhead. Qualitative analyses further highlight behavioral correction through reflection.
Read the full abstract ▾
All authors Yining Hong, Huang Huang, Manling Li, Li Fei-Fei, Leonidas Guibas, Jiajun Wu, Yejin Choi
Robotics
Efficient Hierarchical Any-Angle Path Planning on Multi-Resolution 3D Grids
Victor Reijgwart, Cesar Cadena, Roland Siegwart et al. (+1 more)
Hierarchical, multi-resolution volumetric mapping approaches are widely used to represent large and complex environments as they can efficiently capture their occupancy and connectivity information. Yet widely used path planning methods such as sampling and trajectory optimization do not exploit this explicit connectivity information, and search-based methods such as A* suffer from scalability issues in large-scale high-resolution maps. In many applications, Euclidean shortest paths form the underpinning of the
navigation system. For such applications, any-angle planning methods, which find optimal paths by connecting corners of obstacles with straight-line segments, provide a simple and efficient solution. In this paper, we present a method that has the optimality and completeness properties of any-angle planners while overcoming computational tractability issues common to search-based methods by exploiting multi-resolution representations. Extensive experiments on real and synthetic environments demonstrate the proposed approach's solution quality and speed, outperforming even sampling-based methods. The framework is open-sourced to allow the robotics and planning community to build on our research.
Read the full abstract ▾
All authors Victor Reijgwart, Cesar Cadena, Roland Siegwart, Lionel Ott
Robotics
ActionReasoning: Robot Action Reasoning in 3D Space with LLM for Robotic Brick Stacking
Guangming Wang, Qizhen Ying, Yixiong Jing et al. (+2 more)
Classical robotic systems typically rely on custom planners designed for constrained environments. While effective in restricted settings, these systems lack generalization capabilities, limiting the scalability of
embodied AI and general-purpose
robots. Recent data-driven
Vision-
Language-Action (VLA) approaches aim to learn policies from large-scale simulation and real-world data. However, the continuous action space of the physical world significantly exceeds the representational capacity of linguistic
tokens, making it unclear if scaling data alone can yield general robotic intelligence. To address this gap, we propose ActionReasoning, an LLM-driven framework that performs explicit action
reasoning to produce physics-consistent, prior-guided decisions for robotic manipulation. ActionReasoning leverages the physical priors and real-world knowledge already encoded in Large
Language Models (LLMs) and structures them within a multi-
agent architecture. We instantiate this framework on a tractable case study of brick stacking, where the environment states are assumed to be already accurately measured. The environmental states are then serialized and passed to a multi-
agent LLM framework that generates physics-aware action plans. The experiments demonstrate that the proposed multi-
agent LLM framework enables stable brick placement while shifting effort from low-level domain-specific coding to high-level tool invocation and prompting, highlighting its potential for broader generalization. This work introduces a promising approach to bridging perception and execution in robotic manipulation by integrating physical
reasoning with LLMs.
Read the full abstract ▾
All authors Guangming Wang, Qizhen Ying, Yixiong Jing, Olaf Wysocki, Brian Sheil
Robotics
Quanxin Shou, Fangqi Zhu, Shawn Chen et al. (+9 more)
Vision-
Language-Action (VLA) models have shown strong performance in robotic manipulation, but often struggle in long-horizon or out-of-distribution scenarios due to the lack of explicit mechanisms for
multimodal reasoning and anticipating how the world will evolve under action. Recent works introduce textual chain-of-thought or visual subgoal prediction within VLA models to reason, but still fail to offer a unified human-like
reasoning framework for joint textual
reasoning, visual foresight, and action prediction. To this end, we propose
HALO, a unified VLA model that enables
embodied multimodal chain-of-thought (EM-CoT)
reasoning through a sequential process of textual task
reasoning, visual subgoal prediction for fine-grained guidance, and EM-CoT-augmented action prediction. We instantiate
HALO with a Mixture-of-
Transformers (MoT) architecture that decouples semantic
reasoning, visual foresight, and action prediction into specialized experts while allowing seamless cross-expert collaboration. To enable
HALO learning at scale, we introduce an automated pipeline to synthesize EM-CoT training data along with a carefully crafted training recipe. Extensive experiments demonstrate that: (1)
HALO achieves superior performance in both simulated and real-world environments, surpassing baseline policy pi_0 by 34.1% on RoboTwin
benchmark; (2) all proposed components of the training recipe and EM-CoT design help improve task success rate; and (3)
HALO exhibits strong generalization capabilities under aggressive unseen environmental randomization with our proposed EM-CoT
reasoning.
Read the full abstract ▾
All authors Quanxin Shou, Fangqi Zhu, Shawn Chen, Puxin Yan, Zhengyang Yan, Yikun Miao, Xiaoyi Pang, Zicong Hong, Ruikai Shi, Hao Huang, Jie Zhang, Song Guo
Robotics
A Micro-Macro Model of Encounter-Driven Information Diffusion in Robot Swarms
Davis S. Catherman and Carlo Pinciroli
In this paper, we propose the problem of Encounter-Driven Information
Diffusion (EDID). In EDID,
robots are allowed to exchange information only upon meeting. Crucially, EDID assumes that the
robots are not allowed to schedule their meetings. As such, the
robots have no means to anticipate when, where, and who they will meet. As a step towards the design of storage and routing
algorithms for EDID, in this paper we propose a model of information
diffusion that captures the essential dynamics of EDID. The model is derived from first principles and is composed of two levels: a micro model, based on a generalization of the concept of `mean free path'; and a macro model, which captures the global dynamics of information
diffusion. We validate the model through extensive
robot simulations, in which we consider swarm size, communication range, environment size, and different random motion regimes. We conclude the paper with a discussion of the implications of this model on the
algorithms that best support information
diffusion according to the parameters of interest.
Read the full abstract ▾
Robotics
Cooperative-Competitive Team Play of Real-World Craft Robots
Rui Zhao, Xihui Li, Yizheng Zhang et al. (+6 more)
Multi-
agent deep Reinforcement Learning (RL) has made significant progress in developing intelligent game-playing
agents in recent years. However, the efficient training of collective
robots using multi-
agent RL and the transfer of learned policies to real-world applications remain open research questions. In this work, we first develop a comprehensive robotic system, including simulation, distributed learning framework, and physical
robot components. We then propose and evaluate reinforcement learning techniques designed for efficient training of cooperative and competitive policies on this platform. To address the challenges of multi-
agent sim-to-real transfer, we introduce Out of Distribution State Initialization (OODSI) to mitigate the impact of the sim-to-real gap. In the experiments, OODSI improves the Sim2Real performance by 20%. We demonstrate the effectiveness of our approach through experiments with a multi-
robot car competitive game and a cooperative task in real-world settings.
Read the full abstract ▾
All authors Rui Zhao, Xihui Li, Yizheng Zhang, Yuzhen Liu, Zhong Zhang, Yufeng Zhang, Cheng Zhou, Zhengyou Zhang, Lei Han
Robotics · Cross-listed from cs.CV
Event-Aided Sharp Radiance Field Reconstruction for Fast-Flying Drones
Rong Zou, Marco Cannici and Davide Scaramuzza
Fast-flying aerial
robots promise rapid inspection under limited battery constraints, with direct applications in infrastructure inspection, terrain exploration, and search and rescue. However, high speeds lead to severe motion blur in images and induce significant drift and noise in pose estimates, making dense 3D reconstruction with Neural Radiance Fields (NeRFs) particularly challenging due to their high sensitivity to such degradations. In this work, we present a unified framework that leverages asynchronous event streams alongside motion-blurred frames to reconstruct high-fidelity radiance fields from agile drone flights. By
embedding event-image fusion into NeRF optimization and jointly refining event-based visual-inertial odometry priors using both event and frame modalities, our method recovers sharp radiance fields and accurate camera trajectories without ground-truth supervision. We validate our approach on both synthetic data and real-world sequences captured by a fast-flying drone. Despite highly dynamic drone flights, where RGB frames are severely degraded by motion blur and pose priors become unreliable, our method reconstructs high-fidelity radiance fields and preserves fine scene details, delivering a performance gain of over 50% on real-world data compared to state-of-the-art methods.
Read the full abstract ▾
Robotics · Cross-listed from cs.LG
Localized Dynamics-Aware Domain Adaption for Off-Dynamics Offline Reinforcement Learning
Zhangjie Xia, Yu Yang and Pan Xu
Off-dynamics offline reinforcement learning (RL) aims to learn a policy for a target domain using limited target data and abundant source data collected under different transition dynamics. Existing methods typically address dynamics mismatch either globally over the state space or via pointwise data filtering; these approaches can miss localized cross-domain similarities or incur high computational cost. We propose Localized Dynamics-Aware Domain Adaptation (LoDADA), which exploits localized dynamics mismatch to better reuse source data. LoDADA clusters transitions from source and target
datasets and estimates cluster-level dynamics discrepancy via domain discrimination. Source transitions from clusters with small discrepancy are retained, while those from clusters with large discrepancy are filtered out. This yields a fine-grained and scalable data selection strategy that avoids overly coarse global assumptions and expensive per-sample filtering. We provide theoretical insights and extensive experiments across environments with diverse global and local dynamics shifts. Results show that LoDADA consistently outperforms state-of-the-art off-dynamics offline RL methods by better leveraging localized distribution mismatch.
Read the full abstract ▾
Robotics
Surface-based Manipulation Using Tunable Compliant Porous-Elastic Soft Sensing
Gayatri Indukumar, Muhammad Awais, Diana Cafiso et al. (+2 more)
There is a growing need for soft robotic platforms that perform gentle, precise handling of a wide variety of objects. Existing surface-based manipulation systems, however, lack the compliance and tactile feedback needed for delicate handling. This work introduces the COmpliant Porous-Elastic Soft Sensing (COPESS) integrated with inductive sensors for adaptive object manipulation and localised sensing. The design features a tunable
lattice layer that simultaneously modulates mechanical compliance and sensing performance. By adjusting
lattice geometry, both stiffness and sensor response can be tailored to handle objects with varying mechanical properties. Experiments demonstrate that by easily adjusting one parameter, the
lattice density, from 7 % to 20 %, it is possible to significantly alter the sensitivity and operational force range (about -23x and 9x, respectively). This approach establishes a blueprint for creating adaptive, sensorized surfaces where mechanical and sensory properties are co-optimized, enabling passive, yet programmable, delicate manipulation.
Read the full abstract ▾
All authors Gayatri Indukumar, Muhammad Awais, Diana Cafiso, Matteo Lo Preti, Lucia Beccai
Robotics
Notes-to-Self: Scratchpad Augmented VLAs for Memory Dependent Manipulation Tasks
Sanjay Haresh, Daniel Dijkman, Apratim Bhattacharyya et al. (+1 more)
Many dexterous manipulation tasks are non-markovian in nature, yet little
attention has been paid to this fact in the recent upsurge of the
vision-
language-action (VLA) paradigm. Although they are successful in bringing internet-scale semantic understanding to robotics, existing VLAs are primarily "stateless" and struggle with
memory-dependent long horizon tasks. In this work, we explore a way to impart both spatial and temporal
memory to a VLA by incorporating a
language scratchpad. The scratchpad makes it possible to memorize task-specific information, such as object positions, and it allows the model to keep track of a plan and progress towards subgoals within that plan. We evaluate this approach on a split of
memory-dependent tasks from the ClevrSkills environment, on MemoryBench, as well as on a challenging real-world pick-and-place task. We show that incorporating a
language scratchpad significantly improves generalization on these tasks for both non-recurrent and recurrent models.
Read the full abstract ▾
All authors Sanjay Haresh, Daniel Dijkman, Apratim Bhattacharyya, Roland Memisevic
Robotics
A Robotic Testing Platform for Pipelined Discovery of Resilient Soft Actuators
Ang Li, Alexander Yin, Alexander White et al. (+6 more)
Short lifetime under high electrical fields hinders the widespread robotic application of linear dielectric elastomer actuators (DEAs). Systematic scanning is difficult due to time-consuming per-sample testing and the high-dimensional parameter space affecting performance. To address this, we propose an optimization pipeline enabled by a novel testing
robot capable of scanning DEA lifetime. The
robot integrates electro-mechanical property measurement, programmable voltage input, and multi-channel testing capacity. Using it, we scanned the lifetime of Elastosil-based linear actuators across parameters including input voltage magnitude, frequency, electrode material concentration, and electrical connection filler. The optimal parameter combinations improved operational lifetime under boundary operating conditions by up to 100% and were subsequently scaled up to achieve higher force and displacement output. The final product demonstrated resilience on a modular, scalable quadruped walking
robot with payload carrying capacity (>100% of its untethered body weight, and >700% of combined actuator weight). This work is the first to introduce a self-driving lab approach into robotic actuator design.
Read the full abstract ▾
All authors Ang Li, Alexander Yin, Alexander White, Sahib Sandhu, Matthew Francoeur, Victor Jimenez-Santiago, Van Remenar, Codrin Tugui, Mihai Duduta
Robotics
EKF-Based Depth Camera and Deep Learning Fusion for UAV-Person Distance Estimation and Following in SAR Operations
Luka Šiktar, Branimir Ćaran, Bojan Šekoranja et al. (+1 more)
Vision-based Unmanned Aerial Vehicles (
UAVs) frameworks aid human search tasks by detecting and recognizing specific individuals, then tracking and following them while maintaining a safe distance. A key safety requirement for
UAV following is the accurate estimation of the distance between camera and target object under real-world conditions, achieved by fusing multiple image modalities. As part of the system for automatic people detection and face recognition using deep learning, in this paper we present the fusion of depth camera measurements and
monocular camera-to-body distance estimation for robust tracking and following. Deep learning based filtering of depth camera data and estimation of camera-to-body distance from a
monocular camera are achieved with YOLO-pose, enabling real-time fusion of depth information using the Extended Kalman Filter (EKF)
algorithm. The proposed subsystem, designed for use in drones, estimates and measures the distance between the depth camera and the human body keypoints, to maintain the safe distance between the drone and the human target. Our system provides an accurate estimated distance, which has been validated against motion capture ground truth data. The system has been tested in real time indoors, where it reduces the average errors, RMSE and standard deviations of distance estimation up to 15,3% in three tested scenarios. Based on the test results, the EKF fusion-based approach increases the depth detection range by reducing the errors outside the optimal depth camera working range. It also shows improved robustness and precision in challenging conditions, such as reflections and poor visibility, making it suitable for SAR.
Read the full abstract ▾
All authors Luka Šiktar, Branimir Ćaran, Bojan Šekoranja, Marko Švaco
Robotics
LST-SLAM: A Stereo Thermal SLAM System for Kilometer-Scale Dynamic Environments
Zeyu Jiang, Kuan Xu and Changhao Chen
Thermal cameras offer strong potential for
robot perception under challenging illumination and weather conditions. However, thermal Simultaneous Localization and Mapping (SLAM) remains difficult due to unreliable feature extraction, unstable motion tracking, and inconsistent global pose and map construction, particularly in dynamic large-scale outdoor environments. To address these challenges, we propose LST-SLAM, a novel large-scale stereo thermal SLAM system that achieves robust performance in complex, dynamic scenes. Our approach combines self-supervised thermal feature learning, stereo dual-level motion tracking, and
geometric pose optimization. We also introduce a semantic-
geometric hybrid constraint that suppresses potentially dynamic features lacking strong inter-frame
geometric consistency. Furthermore, we develop an online incremental bag-of-words model for loop closure detection, coupled with global pose optimization to mitigate accumulated drift. Extensive experiments on kilometer-scale dynamic thermal
datasets show that LST-SLAM significantly outperforms recent representative SLAM systems, including AirSLAM and DROID-SLAM, in both robustness and accuracy.
Read the full abstract ▾
Robotics
ParkDiffusion++: Ego Intention Conditioned Joint Multi-Agent Trajectory Prediction for Automated Parking using Diffusion Models
Jiarong Wei, Anna Rehr, Christian Feist et al. (+1 more)
Automated parking is a challenging operational domain for advanced driver assistance systems, requiring robust scene understanding and interaction
reasoning. The key challenge is twofold: (i) predict multiple plausible ego intentions according to context and (ii) for each intention, predict the joint responses of surrounding
agents, enabling effective what-if decision-making. However, existing methods often fall short, typically treating these interdependent problems in isolation. We propose ParkDiffusion++, which jointly learns a multi-modal ego intention predictor and an ego-conditioned multi-
agent joint trajectory predictor for automated parking. Our approach makes several key contributions. First, we introduce an ego intention tokenizer that predicts a small set of discrete endpoint intentions from
agent histories and vectorized map polylines. Second, we perform ego-intention-conditioned joint prediction, yielding socially consistent predictions of the surrounding
agents for each possible ego intention. Third, we employ a lightweight safety-guided denoiser with different constraints to refine joint scenes during training, thus improving accuracy and safety. Fourth, we propose counterfactual knowledge
distillation, where an EMA teacher refined by a frozen safety-guided denoiser provides pseudo-targets that capture how
agents react to alternative ego intentions. Extensive evaluations demonstrate that ParkDiffusion++ achieves state-of-the-art performance on the Dragon Lake Parking (DLP)
dataset and the Intersections Drone (inD)
dataset. Importantly, qualitative what-if visualizations show that other
agents react appropriately to different ego intentions.
Read the full abstract ▾
All authors Jiarong Wei, Anna Rehr, Christian Feist, Abhinav Valada
Robotics
Computer-Aided Design of Rational Motions for 4R and 6R Spatial Mechanism Synthesis
Daniel Huczala, Severinas Zube, Martin Pfurner et al. (+2 more)
This paper focuses on
geometric methods for generating rational motions used in the design of single-loop rational linkages, 1-degree-of-freedom mechanisms that can execute prescribed spatial tasks. Building on established rational motion synthesis methods, we introduce a new interpolation scheme for seven 3D points based on cubic quaternionic Bezier curves. The resulting motion admits factorization, i.e. the synthesis of a spatial six-bar mechanism whose tool frame passes the specified seven points. To support engineering practice, we provide open-source CAD tools that implement also the other methods and provide fast visual evaluation of motion generation and mechanism synthesis.
Read the full abstract ▾
All authors Daniel Huczala, Severinas Zube, Martin Pfurner, Johannes Siegele, Frank C. Park
Robotics
Task-oriented grasping for dexterous robots using postural synergies and reinforcement learning
Dimitrios Dimou, José Santos-Victor and Plinio Moreno
In this paper, we address the problem of task-oriented grasping for
humanoid robots, emphasizing the need to align with human social norms and task-specific objectives. Existing methods, employ a variety of open-loop and closed-loop approaches but lack an end-to-end solution that can grasp several objects while taking into account the downstream task's constraints. Our proposed approach employs reinforcement learning to enhance task-oriented grasping, prioritizing the post-grasp intention of the
agent. We extract human grasp preferences from the ContactPose
dataset, and train a hand synergy model based on the Variational Autoencoder (VAE) to imitate the participant's grasping actions. Based on this data, we train an
agent able to grasp multiple objects while taking into account distinct post-grasp intentions that are task-specific. By combining data-driven insights from human grasping behavior with learning by exploration provided by reinforcement learning, we can develop
humanoid robots capable of context-aware manipulation actions, facilitating collaboration in human-centered environments.
Read the full abstract ▾
Robotics
GeCo-SRT: Geometry-aware Continual Adaptation for Robotic Cross-Task Sim-to-Real Transfer
Wenbo Yu, Wenke Xia, Weitao Zhang et al. (+1 more)
Bridging the sim-to-real gap is important for applying low-cost simulation data to real-world robotic systems. However, previous methods are severely limited by treating each transfer as an isolated endeavor, demanding repeated, costly tuning and wasting prior transfer experience. To move beyond isolated sim-to-real, we build a continual cross-task sim-to-real transfer paradigm centered on knowledge accumulation across iterative transfers, thereby enabling effective and efficient adaptation to novel tasks. Thus, we propose GeCo-SRT, a geometry-aware continual adaptation method. It utilizes domain-invariant and task-invariant knowledge from local
geometric features as a transferable foundation to accelerate adaptation during subsequent sim-to-real transfers. This method starts with a geometry-aware mixture-of-experts module, which dynamically activates experts to specialize in distinct
geometric knowledge to bridge observation sim-to-real gap. Further, the geometry-expert-guided prioritized experience replay module preferentially samples from underutilized experts, refreshing specialized knowledge to combat forgetting and maintain robust cross-task performance. Leveraging knowledge accumulated during iterative transfer, GeCo-SRT method not only achieves 52% average performance improvement over the baseline, but also demonstrates significant data efficiency for new task adaptation with only 1/6 data. We hope this work inspires approaches for efficient, low-cost cross-task sim-to-real transfer.
Read the full abstract ▾
All authors Wenbo Yu, Wenke Xia, Weitao Zhang, Di Hu
Robotics
KCFRC: Kinematic Collision-Aware Foothold Reachability Criteria for Legged Locomotion
Lei Ye, Haibo Gao, Huaiguang Yang et al. (+6 more)
Legged
robots face significant challenges in navigating complex environments, as they require precise real-time decisions for foothold selection and contact planning. While existing research has explored methods to select footholds based on terrain geometry or kinematics, a critical gap remains: few existing methods efficiently validate the existence of a non-collision swing trajectory. This paper addresses this gap by introducing KCFRC, a novel approach for efficient foothold reachability analysis. We first formally define the foothold reachability problem and establish a sufficient condition for foothold reachability. Based on this condition, we develop the KCFRC
algorithm, which enables
robots to validate foothold reachability in real time. Our experimental results demonstrate that KCFRC achieves remarkable time efficiency, completing foothold reachability checks for a single leg across 900 potential footholds in an average of 2 ms. Furthermore, we show that KCFRC can accelerate trajectory optimization and is particularly beneficial for contact planning in confined spaces, enhancing the adaptability and robustness of legged
robots in challenging environments.
Read the full abstract ▾
All authors Lei Ye, Haibo Gao, Huaiguang Yang, Peng Xu, Haoyu Wang, Tie Liu, Junqi Shan, Zongquan Deng, Liang Ding
Robotics · Cross-listed from cs.CV
RU4D-SLAM: Reweighting Uncertainty in Gaussian Splatting SLAM for 4D Scene Reconstruction
Yangfan Zhao, Hanwei Zhang, Ke Huang et al. (+3 more)
Combining 3D Gaussian splatting with Simultaneous Localization and Mapping (SLAM) has gained popularity as it enables continuous 3D environment reconstruction during motion. However, existing methods struggle in dynamic environments, particularly moving objects complicate 3D reconstruction and, in turn, hinder reliable tracking. The emergence of 4D reconstruction, especially 4D Gaussian splatting, offers a promising direction for addressing these challenges, yet its potential for 4D-aware SLAM remains largely underexplored. Along this direction, we propose a robust and efficient framework, namely Reweighting Uncertainty in Gaussian Splatting SLAM (RU4D-SLAM) for 4D scene reconstruction, that introduces temporal factors into spatial 3D representation while incorporating uncertainty-aware perception of scene changes, blurred image synthesis, and dynamic scene reconstruction. We enhance dynamic scene representation by integrating motion blur rendering, and improve uncertainty-aware tracking by extending per-pixel uncertainty modeling, which is originally designed for static scenarios, to handle blurred images. Furthermore, we propose a semantic-guided reweighting mechanism for per-pixel uncertainty estimation in dynamic scenes, and introduce a learnable opacity weight to support adaptive 4D mapping. Extensive experiments on standard
benchmarks demonstrate that our method substantially outperforms state-of-the-art approaches in both trajectory accuracy and 4D scene reconstruction, particularly in dynamic environments with moving objects and low-quality inputs. Code available: https://ru4d-slam.github.io
Read the full abstract ▾
All authors Yangfan Zhao, Hanwei Zhang, Ke Huang, Qiufeng Wang, Zhenzhou Shao, Dengyu Wu