Robotics
High-Speed, All-Terrain Autonomy: Ensuring Safety at the Limits of Mobility
James R. Baxter, Bogdan I. Epureanu, Paramsothy Jayakumar et al. (+1 more)
A novel local trajectory planner, capable of controlling an autonomous off-road vehicle on rugged terrain at high-speed is presented. Autonomous vehicles are currently unable to safely operate off-road at high-speed, as current approaches either fail to predict and mitigate rollovers induced by rough terrain or are not real-time feasible. To address this challenge, a novel model predictive control (MPC) formulation is developed for local trajectory planning. A new dynamics model for off-road vehicles on rough, non-planar terrain is derived and used for prediction. Extreme mobility, including tire liftoff without rollover, is safely enabled through a new energy-based constraint. The formulation is analytically shown to mitigate rollover types ignored by many state-of-the-art methods, and real-time feasibility is achieved through parallelized GPGPU computation. The planner's ability to provide safe, extreme trajectories is studied through both simulated trials and full-scale physical experiments. The results demonstrate fewer rollovers and more successes compared to a state-of-the-art baseline across several challenging scenarios that push the vehicle to its mobility limits.
Read the full abstract ▾
All authors James R. Baxter, Bogdan I. Epureanu, Paramsothy Jayakumar, Tulga Ersal
Robotics
An Open Source Computer Vision and Machine Learning Framework for Affordable Life Science Robotic Automation
Zachary Logan, Andrew Dudash and Daniel Negrón
We present an open-source robotic framework that integrates computer
vision and machine learning based inverse kinematics to enable low-cost laboratory automation tasks such as colony picking and
liquid handling. The system uses a custom trained U-net model for semantic
segmentation of microbial cultures, combined with Mixture Density Network for predicating joint angles of a simple 5-DOF
robot arm. We evaluated the framework using a modified
robot arm, upgraded with a custom
liquid handling end-effector. Experimental results demonstrate the framework's feasibility for precise, repeatable operations, with mean positional error below 1 mm and joint angle prediction errors below 4 degrees and colony detection capabilities with IoU score of 0.537 and Dice coefficient of 0.596.
Read the full abstract ▾
Robotics
TRGS-SLAM: IMU-Aided Gaussian Splatting SLAM for Blurry, Rolling Shutter, and Noisy Thermal Images
Spencer Carmichael and Katherine A. Skinner
Thermal cameras offer several advantages for simultaneous localization and mapping (SLAM) with mobile
robots: they provide a passive, low-power solution to operating in darkness, are invariant to rapidly changing or high dynamic range illumination, and can see through fog,
dust, and smoke. However, uncooled microbolometer thermal cameras, the only practical option in most robotics applications, suffer from significant motion blur, rolling shutter distortions, and fixed pattern noise. In this paper, we present TRGS-SLAM, a 3D Gaussian Splatting (3DGS) based thermal inertial SLAM system uniquely capable of handling these degradations. To overcome the challenges of thermal data, we introduce a model-aware 3DGS rendering method and several general innovations to 3DGS SLAM, including B-spline trajectory optimization with a two-stage IMU loss, view-diversity-based opacity resetting, and pose drift correction schemes. Our system demonstrates accurate tracking on real-world, fast motion, and high-noise thermal data that causes all other tested SLAM methods to fail. Moreover, through offline refinement of our SLAM results, we demonstrate thermal image restoration competitive with prior work that required ground truth poses.
Read the full abstract ▾
Robotics · Cross-listed from cs.CV
Scene Representation using 360{\deg} Saliency Graph and its Application in Vision-based Indoor Navigation
Preeti Meena, Himanshu Kumar and Sandeep Yadav
A Scene, represented visually using different formats such as RGB-D,
LiDAR scan, keypoints, rectangular, spherical, multi-views, etc., contains information implicitly embedded relevant to applications such as scene indexing,
vision-based
navigation. Thus, these representations may not be efficient for such applications. This paper proposes a novel 360{\deg} saliency graph representation of the scenes. This rich representation explicitly encodes the relevant visual, contextual, semantic, and
geometric information of the scene as nodes, edges, edge weights, and angular position in the 360{\deg} graph. Also, this representation is robust against scene view change and addresses challenges of indoor environments such as varied illumination, occlusions, and shadows as in the case of existing traditional methods. We have utilized this rich and efficient representation for
vision-based
navigation and compared it with existing
navigation methods using 360{\deg} scenes. However, these existing methods suffer from limitations of poor scene representation, lacking scene-specific information. This work utilizes the proposed representation first to localize the query scene in the given topological map, and then facilitate 2D
navigation by estimating the next required movement directions towards the target destination in the topological map by using the embedded
geometric information in the 360{\deg} saliency graph. Experimental results demonstrate the efficacy of the proposed 360{\deg} saliency graph representation in enhancing both scene localization and
vision-based indoor
navigation.
Read the full abstract ▾
Robotics · Cross-listed from cs.LG
Learning Sampled-data Control for Swarms via MeanFlow
Anqi Dong, Yongxin Chen, Karl H. Johansson et al. (+1 more)
Steering large-scale swarms with only limited control updates is often needed due to communication or computational constraints, yet most learning-based approaches do not account for this and instead model instantaneous velocity fields. As a result, the natural object for decision making is a finite-window control quantity rather than an infinitesimal one. To address this gap, we consider the recent machine learning framework MeanFlow and generalize it to the setting with general linear dynamic systems. This results in a new sampled-data learning framework that operates directly in control space and that can be applied for swarm steering. To this end, we learn the finite-horizon coefficient that parameterizes the minimum-energy control applied over each interval, and derive a differential identity that connects this quantity to a local bridge-induced supervision signal. This identity leads to a simple stop-gradient regression objective, allowing the interval coefficient field to be learned efficiently from bridge samples. The learned policy is deployed through sampled-data updates, guaranteeing that the resulting controller exactly respects the prescribed linear time-invariant dynamics and actuation channel. The resulting method enables few-step swarm steering at scale, while remaining consistent with the finite-window actuation structure of the underlying control system.
Read the full abstract ▾
All authors Anqi Dong, Yongxin Chen, Karl H. Johansson, Johan Karlsson
Robotics
IndoorR2X: Indoor Robot-to-Everything Coordination with LLM-Driven Planning
Fan Yang, Soumya Teotia, Shaunak A. Mehta et al. (+8 more)
Although
robot-to-
robot (R2R) communication improves indoor scene understanding beyond what a single
robot can achieve, R2R alone cannot overcome partial observability without substantial exploration overhead or scaling team size. In contrast, many indoor environments already include low-cost Internet of Things (IoT) sensors (e.g., cameras) that provide persistent, building-wide context beyond onboard perception. We therefore introduce IndoorR2X, a
benchmark and simulation framework for Large
Language Model (LLM)-driven multi-
robot task planning with
Robot-to-Everything (R2X) perception and communication in indoor environments. IndoorR2X integrates observations from mobile
robots and static IoT devices to construct a global semantic state that supports scalable scene understanding, reduces redundant exploration, and enables high-level coordination through LLM-based planning. IndoorR2X provides configurable simulation environments, sensor layouts,
robot teams, and task suites to systematically evaluate semantic-level coordination strategies. Extensive experiments across diverse settings demonstrate that IoT-augmented world modeling improves multi-
robot efficiency and reliability, and we highlight key insights and failure modes for advancing LLM-based collaboration between
robot teams and indoor IoT sensors. Project page: https://fandulu.github.io/IndoorR2X_project_page/.
Read the full abstract ▾
All authors Fan Yang, Soumya Teotia, Shaunak A. Mehta, Prajit KrisshnaKumar, Quanting Xie, Jun Liu, Yueqi Song, Wenkai Li, Atsunori Moteki, Kanji Uchino, Yonatan Bisk
Robotics
The Robot's Inner Critic: Self-Refinement of Social Behaviors through VLM-based Replanning
Jiyu Lim, Youngwoo Yoon and Kwanghyun Park
Conventional
robot social behavior generation has been limited in flexibility and autonomy, relying on predefined motions or human feedback. This study proposes CRISP (Critique-and-Replan for Interactive Social Presence), an autonomous framework where a
robot critiques and replans its own actions by leveraging a
Vision-
Language Model (VLM) as a `human-like social critic.' CRISP integrates (1) extraction of movable joints and constraints by analyzing the
robot's description file (e.g., MJCF), (2) generation of step-by-step behavior plans based on situational context, (3) generation of low-level joint control code by referencing visual information (joint range-of-motion visualizations), (4) VLM-based evaluation of social appropriateness and naturalness, including pinpointing erroneous steps, and (5) iterative refinement of behaviors through reward-based search. This approach is not tied to a specific
robot API; it can generate subtly different, human-like motions on various platforms using only the
robot's structure file. In a user study involving five different
robot types and 20 scenarios, including mobile manipulators and
humanoids, our proposed method achieved significantly higher preference and situational appropriateness ratings compared to previous methods. This research presents a general framework that minimizes human intervention while expanding the
robot's autonomous interaction capabilities and cross-platform applicability. Detailed result
videos and supplementary information regarding this work are available at: https://limjiyu99.github.io/inner-critic/
Read the full abstract ▾
Robotics
HortiMulti: A Multi-Sensor Dataset for Localisation and Mapping in Horticultural Polytunnels
Shuoyuan Xu, Zhipeng Zhong, Tiago Barros et al. (+4 more)
Agricultural robotics is gaining increasing relevance in both research and real-world deployment. As these systems are expected to operate autonomously in more complex tasks, the availability of representative real-world
datasets becomes essential. While domains such as urban and forestry robotics benefit from large and established
benchmarks, horticultural environments remain comparatively under-explored despite the economic significance of this sector. To address this gap, we present HortiMulti, a
multimodal, cross-season
dataset collected in commercial strawberry and raspberry polytunnels across an entire growing season, capturing substantial appearance variation, dynamic foliage, specular reflections from plastic covers, severe perceptual aliasing, and GNSS-unreliable conditions, all of which directly degrade existing localisation and perception
algorithms. The sensor suite includes two 3D
LiDARs, four RGB cameras, an IMU, GNSS, and wheel odometry. Ground truth trajectories are derived from a combination of Total Station surveying, AprilTag fiducial markers, and
LiDAR-inertial odometry, spanning dense, sparse, and marker-free coverage to support evaluation under both controlled and realistic conditions. We release time-synchronised raw measurements, calibration files, reference trajectories, and baseline
benchmarks for visual,
LiDAR, and multi-sensor SLAM, with results confirming that current state-of-the-art methods remain inadequate for reliable polytunnel deployment, establishing HortiMulti as a one-stop resource for developing and testing robotic perception systems in horticulture environments.
Read the full abstract ▾
All authors Shuoyuan Xu, Zhipeng Zhong, Tiago Barros, Matthew Coombes, Cristiano Premebida, Hao Wu, Cunjia Liu
Robotics
AGILE: A Comprehensive Workflow for Humanoid Loco-Manipulation Learning
Huihua Zhao, Rafael Cathomen, Lionel Gulich et al. (+6 more)
Recent advances in reinforcement learning (RL) have enabled impressive
humanoid behaviors in simulation, yet transferring these results to new
robots remains challenging. In many real deployments, the primary bottleneck is no longer simulation throughput or
algorithm design, but the absence of systematic infrastructure that links environment verification, training, evaluation, and deployment in a coherent loop. To address this gap, we present AGILE, an end-to-end workflow for
humanoid RL that standardizes the policy-development lifecycle to mitigate common sim-to-real failure modes. AGILE comprises four stages: (1) interactive environment verification, (2) reproducible training, (3) unified evaluation, and (4) descriptor-driven deployment via
robot/task configuration descriptors. For evaluation stage, AGILE supports both scenario-based tests and randomized rollouts under a shared suite of motion-quality diagnostics, enabling automated regression testing and principled robustness assessment. AGILE also incorporates a set of training stabilizations and algorithmic enhancements in training stage to improve optimization stability and sim-to-real transfer. With this pipeline in place, we validate AGILE across five representative
humanoid skills spanning locomotion, recovery, motion imitation, and loco-manipulation on two hardware platforms (Unitree G1 and Booster T1), achieving consistent sim-to-real transfer. Overall, AGILE shows that a standardized, end-to-end workflow can substantially improve the reliability and reproducibility of
humanoid RL development.
Read the full abstract ▾
All authors Huihua Zhao, Rafael Cathomen, Lionel Gulich, Wei Liu, Efe Arda Ongan, Michael Lin, Shalin Jain, Soha Pouya, Yan Chang
Robotics
KUKAloha: A General, Low-Cost, and Shared-Control based Teleoperation Framework for Construction Robot Arm
Yifan Xu, Qizhang Shen, Vineet Kamat et al. (+1 more)
This paper presents KUKAloha, a general, low-cost, and shared-control teleoperation framework designed for construction
robot arms. The proposed system employs a leader-follower paradigm in which a lightweight leading arm enables intuitive human guidance for coarse
robot motion, while an autonomous perception module based on AprilTag detection performs precise
alignment and grasp execution. By explicitly decoupling human control from fine manipulation, KUKAloha improves safety and repeatability when operating large-scale manipulators. We implement the framework on a KUKA
robot arm and conduct a usability study with representative construction manipulation tasks. Experimental results demonstrate that KUKAloha reduces operator workload, improves task completion efficiency, and provides a practical solution for scalable demonstration collection and shared human-
robot control in construction environments.
Read the full abstract ▾
All authors Yifan Xu, Qizhang Shen, Vineet Kamat, Carol Menassa
Robotics
Not an Obstacle for Dog, but a Hazard for Human: A Co-Ego Navigation System for Guide Dog Robots
Ruiping Liu, Jingqi Zhang, Junwei Zheng et al. (+8 more)
Guide dogs offer independence to Blind and Low-
Vision (BLV) individuals, yet their limited availability leaves the vast majority of BLV users without access. Quadruped robotic guide dogs present a promising alternative, but existing systems rely solely on the
robot's ground-level sensors for
navigation, overlooking a critical class of hazards: obstacles that are transparent to the
robot yet dangerous at human body height, such as bent branches. We term this the viewpoint asymmetry problem and present the first system to explicitly address it. Our Co-Ego system adopts a dual-branch obstacle avoidance framework that integrates the
robot-centric ground sensing with the user's elevated
egocentric perspective to ensure comprehensive
navigation safety. Deployed on a quadruped
robot, the system is evaluated in a controlled user study with sighted participants under blindfold across three conditions: unassisted, single-view, and cross-view fusion. Results demonstrate that cross-view fusion significantly reduces collision times and cognitive load, verifying the necessity of viewpoint complementarity for safe robotic guide dog
navigation.
Read the full abstract ▾
All authors Ruiping Liu, Jingqi Zhang, Junwei Zheng, Yufan Chen, Peter Seungjune Lee, Di Wen, Kunyu Peng, Jiaming Zhang, Kailun Yang, Katja Mombaur, Rainer Stiefelhagen
Robotics · Cross-listed from cs.LG
Spectral Alignment in Forward-Backward Representations via Temporal Abstraction
Seyed Mahdi B. Azad, Jasper Hoffmann, Iman Nematollahi et al. (+3 more)
Forward-backward (FB) representations provide a powerful framework for learning the successor representation (SR) in continuous spaces by enforcing a low-rank factorization. However, a fundamental spectral mismatch often exists between the high-rank transition dynamics of continuous environments and the low-rank bottleneck of the FB architecture, making accurate low-rank representation learning difficult. In this work, we analyze temporal abstraction as a mechanism to mitigate this mismatch. By characterizing the spectral properties of the transition operator, we show that temporal abstraction acts analogously to a low-pass filter that suppresses high-frequency spectral components. This suppression reduces the effective rank of the induced SR while preserving a formal bound on the resulting value function error. Empirically, we show that this
alignment is a key factor for stable FB learning, particularly at high discount factors where bootstrapping becomes error-prone. Our results identify temporal abstraction as a principled mechanism for shaping the spectral structure of the underlying MDP and enabling effective long-horizon representations in continuous control.
Read the full abstract ▾
All authors Seyed Mahdi B. Azad, Jasper Hoffmann, Iman Nematollahi, Hao Zhu, Abhinav Valada, Joschka Boedecker
Robotics · Cross-listed from cs.CV
A Unified Platform and Quality Assurance Framework for 3D Ultrasound Reconstruction with Robotic, Optical, and Electromagnetic Tracking
Lewis Howell, Manisha Waterston, Tze Min Wah et al. (+2 more)
Three-dimensional (3D) Ultrasound (US) can facilitate diagnosis, treatment planning, and image-guided therapy. However, current studies rarely provide a comprehensive evaluation of volumetric accuracy and reproducibility, highlighting the need for robust Quality Assurance (QA) frameworks, particularly for tracked 3D US reconstruction using freehand or robotic acquisition. This study presents a QA framework for 3D US reconstruction and a flexible open source platform for tracked US research. A custom phantom containing
geometric inclusions with varying
symmetry properties enables straightforward evaluation of optical, electromagnetic, and robotic kinematic tracking for 3D US at different scanning speeds and insonation angles. A standardised pipeline performs real-time
segmentation and 3D reconstruction of
geometric targets (DSC = 0.97, FPS = 46) without GPU acceleration, followed by automated registration and comparison with ground-truth geometries. Applying this framework showed that our robotic 3D US achieves state-of-the-art reconstruction performance (DSC-3D = 0.94 +- 0.01, HD95 = 1.17 +- 0.12), approaching the spatial resolution limit imposed by the transducer. This work establishes a flexible experimental platform and a reproducible validation methodology for 3D US reconstruction. The proposed framework enables robust cross-platform comparisons and improved reporting practices, supporting the safe and effective clinical translation of 3D ultrasound in diagnostic and image-guided therapy applications.
Read the full abstract ▾
All authors Lewis Howell, Manisha Waterston, Tze Min Wah, James H. Chandler, James R. McLaughlan
Robotics
Uncertainty Matters: Structured Probabilistic Online Mapping for Motion Prediction in Autonomous Driving
Pritom Gogoi, Faris Janjoš, Bin Yang et al. (+1 more)
Online map generation and trajectory prediction are critical components of the autonomous driving perception-prediction-planning pipeline. While modern vectorized mapping models achieve high
geometric accuracy, they typically treat map estimation as a deterministic task, discarding structural uncertainty. Existing probabilistic approaches often rely on diagonal covariance
matrices, which assume independence between points and fail to capture the strong spatial correlations inherent in road geometry. To address this, we propose a structured probabilistic formulation for online map generation. Our method explicitly models intra-element dependencies by predicting a dense covariance
matrix, parameterized via a Low-Rank plus Diagonal (LRPD) covariance decomposition. This formulation represents uncertainty as a combination of a low-rank component, which captures global spatial structure, and a diagonal component representing independent local noise, thereby capturing
geometric correlations without the prohibitive computational cost of full covariance
matrices. Evaluations on the nuScenes
dataset demonstrate that our uncertainty-aware framework yields consistent improvements in online map generation quality compared to deterministic baselines. Furthermore, our approach establishes new state-of-the-art performance for map-based motion prediction, highlighting the critical role of uncertainty in planning tasks. Code is published under link-available-soon.
Read the full abstract ▾
All authors Pritom Gogoi, Faris Janjoš, Bin Yang, Andreas Look
Robotics
GustPilot: A Hierarchical DRL-INDI Framework for Wind-Resilient Quadrotor Navigation
Amir Atef Habel, Roohan Ahmed Khan, Fawad Mehboob et al. (+2 more)
Wind disturbances remain a key barrier to reliable autonomous
navigation for lightweight quadrotors, where the rapidly varying airflow can destabilize both planning and tracking. This paper introduces GustPilot, a hierarchical wind-resilient
navigation stack in which a deep reinforcement learning (DRL) policy generates inertial-frame velocity reference for gate traversal. At the same time, a
geometric Incremental Nonlinear Dynamic Inversion (INDI) controller provides low-level tracking with fast residual disturbance rejection. The INDI layer achieves this by providing incremental feedback on both specific linear acceleration and angular acceleration rate, using onboard sensor measurements to reject wind disturbances rapidly. Robustness is obtained through a two-level strategy, wind-aware planning learned via fan-jet domain randomization during training, and rapid execution-time disturbance rejection by the INDI tracking controller. We evaluate GustPilot in real flights on a 50g quad-copter platform against a DRL-PID baseline across four scenarios ranging from no-wind to fully dynamic conditions with a moving gate and a moving disturbance source. Despite being trained only in a minimal single-gate and single-fan setup, the policy generalizes to significantly more complex environments (up to six gates and four fans) without retraining. Across 80 experiments, DRL-INDI achieves a 94.7% versus 55.0% for DRL-PID as average Overall Success Rate (OSR), reduces tracking RMSE up to 50%, and sustains speeds up to 1.34 m/s under wind disturbances up to 3.5 m/s. These results demonstrate that combining DRL-based velocity planning with structured INDI disturbance rejection provides a practical and generalizable approach to wind-resilient autonomous flight
navigation.
Read the full abstract ▾
All authors Amir Atef Habel, Roohan Ahmed Khan, Fawad Mehboob, Clement Fortin, Dzmitry Tsetserukou
Robotics
Radar-Inertial Odometry with Online Spatio-Temporal Calibration via Continuous-Time IMU Modeling
Vlaho-Josip Štironja, Luka Petrović, Juraj Peršić et al. (+2 more)
Radar-Inertial Odometry (RIO) has emerged as a robust alternative to
vision- and
LiDAR-based odometry in challenging conditions such as low light, fog, featureless environments, or in adverse weather. However, many existing RIO approaches assume known radar-IMU extrinsic calibration or rely on sufficient motion excitation for online extrinsic estimation, while temporal misalignment between sensors is often neglected or treated independently. In this work, we present a RIO framework that performs joint online spatial and temporal calibration within a factor-graph optimization formulation, based on continuous-time modeling of inertial measurements using uniform cubic B-splines. The proposed continuous-time representation of acceleration and angular velocity accurately captures the asynchronous nature of radar-IMU measurements, enabling reliable convergence of both the temporal offset and extrinsic calibration parameters, without relying on scan matching, target tracking, or environment-specific assumptions.
Read the full abstract ▾
All authors Vlaho-Josip Štironja, Luka Petrović, Juraj Peršić, Ivan Marković, Ivan Petrović
Robotics · Cross-listed from cs.CV
LIORNet: Self-Supervised LiDAR Snow Removal Framework for Autonomous Driving under Adverse Weather Conditions
Ji-il Park and Inwook Shim
LiDAR sensors provide high-resolution 3D perception and long-range detection, making them indispensable for autonomous driving and robotics. However, their performance significantly degrades under adverse weather conditions such as snow, rain, and fog, where spurious noise points dominate the point cloud and lead to false perception. To address this problem, various approaches have been proposed: distance-based filters exploiting spatial sparsity, intensity-based filters leveraging reflectance distributions, and learning-based methods that adapt to complex environments. Nevertheless, distance-based methods struggle to distinguish valid object points from noise, intensity-based methods often rely on fixed thresholds that lack adaptability to changing conditions, and learning-based methods suffer from the high cost of annotation, limited generalization, and computational overhead. In this study, we propose LIORNet, which eliminates these drawbacks and integrates the strengths of all three paradigms. LIORNet is built upon a U-Net++ backbone and employs a self-supervised learning strategy guided by pseudo-labels generated from multiple physical and statistical cues, including range-dependent intensity thresholds, snow reflectivity, point sparsity, and sensing range constraints. This design enables LIORNet to distinguish noise points from environmental structures without requiring manual annotations, thereby overcoming the difficulty of snow labeling and the limitations of single-principle approaches. Extensive experiments on the WADS and CADC
datasets demonstrate that LIORNet outperforms state-of-the-art filtering
algorithms in both accuracy and runtime while preserving critical environmental features. These results highlight LIORNet as a practical and robust solution for
LiDAR perception in extreme weather, with strong potential for real-time deployment in autonomous driving systems.
Read the full abstract ▾
Robotics · Cross-listed from cs.HC
Sense4HRI: A ROS 2 HRI Framework for Physiological Sensor Integration and Synchronized Logging
Manuel Scheibl, Julian Leichert, Sinem Görmez et al. (+1 more)
Physiological signals are increasingly relevant to estimate the mental states of users in human-
robot interaction (HRI), yet ROS 2-based HRI frameworks still lack reusable support to integrate such data streams in a standardized way. Therefore, we propose Sense4HRI, an adapted framework for human-
robot interaction in ROS 2 that integrates physiological measurements and derived user-state indicators. The framework is designed to be extensible, allowing the integration of additional physiological sensors, their interpretation, and
multimodal fusion to provide a robust assessment of the mental states of users. In addition, it introduces reusable interfaces for timestamped physiological time-series data and supports synchronized logging of physiological signals together with experiment context, enabling interoperable and traceable
multimodal analysis within ROS 2-based HRI systems.
Read the full abstract ▾
All authors Manuel Scheibl, Julian Leichert, Sinem Görmez, Britta Wrede
Robotics
Beyond detection: cooperative multi-agent reasoning for rapid onboard EO crisis response
Alejandro D. Mousist, Pedro Delgado de Robles Martín, Raquel Lladró Climent et al. (+1 more)
Rapid identification of hazardous events is essential for next-generation Earth Observation (EO) missions supporting disaster response. However, current monitoring pipelines remain largely ground-centric, introducing latency due to downlink limitations, multi-source data fusion constraints, and the computational cost of exhaustive scene analysis. This work proposes a hierarchical multi-
agent architecture for onboard EO processing under strict resource and bandwidth constraints. The system enables the exploitation of complementary
multimodal observations by coordinating specialized AI
agents within an event-driven decision pipeline. AI
agents can be deployed across multiple nodes in a distributed setting, such as satellite platforms. An Early Warning
agent generates fast hypotheses from onboard observations and selectively activates domain-specific analysis
agents, while a Decision
agent consolidates the evidence to issue a final alert. The architecture combines
vision-
language models, traditional remote sensing analysis tools, and role-specialized
agents to enable structured
reasoning over
multimodal observations while minimizing unnecessary computation. A proof-of-concept implementation was executed on the engineering model of an edge-computing platform currently deployed in orbit, using representative satellite data. Experiments on wildfire and flood monitoring scenarios show that the proposed routing-based pipeline significantly reduces computational overhead while maintaining coherent decision outputs, demonstrating the feasibility of distributed
agent-based
reasoning for future autonomous EO constellations.
Read the full abstract ▾
All authors Alejandro D. Mousist, Pedro Delgado de Robles Martín, Raquel Lladró Climent, Julian Cobos Aparicio
Robotics
Multi-Agent Motion Planning on Industrial Magnetic Levitation Platforms: A Hybrid ADMM-HOCBF approach
Bavo Tistaert, Stan Servaes, Alejandro Gonzalez-Garcia et al. (+4 more)
This paper presents a novel hybrid motion planning method for holonomic multi-
agent systems. The proposed decentralised model predictive control (MPC) framework tackles the intractability of classical centralised MPC for a growing number of
agents while providing safety guarantees. This is achieved by combining a decentralised version of the alternating direction method of multipliers (ADMM) with a centralised high-order control barrier function (HOCBF) architecture. Simulation results show significant improvement in scalability over classical centralised MPC. We validate the efficacy and real-time capability of the proposed method by developing a highly efficient C++ implementation and deploying the resulting trajectories on a real industrial
magnetic levitation platform.
Read the full abstract ▾
All authors Bavo Tistaert, Stan Servaes, Alejandro Gonzalez-Garcia, Ibrahim Ibrahim, Louis Callens, Jan Swevers, Wilm Decré
Robotics
Real-Time Structural Detection for Indoor Navigation from 3D LiDAR Using Bird's-Eye-View Images
Guanliang Li, Pedro Espinosa-Angulo, David Perez-Saura et al. (+1 more)
Efficient structural perception is essential for mapping and autonomous
navigation on resource-constrained
robots. Existing 3D methods are computationally prohibitive, while traditional 2D
geometric approaches lack robustness. This paper presents a lightweight, real-time framework that projects 3D
LiDAR data into 2D Bird's-Eye-View (BEV) images to enable efficient detection of structural elements relevant to mapping and
navigation. Within this representation, we systematically evaluate several feature extraction strategies, including classical
geometric techniques (Hough Transform, RANSAC, and LSD) and a deep learning detector based on YOLO-OBB. The resulting detections are integrated through a spatiotemporal fusion module that improves stability and robustness across consecutive frames. Experiments conducted on a standard mobile robotic platform highlight clear performance trade-offs. Classical methods such as Hough and LSD provide fast responses but exhibit strong sensitivity to noise, with LSD producing excessive segment fragmentation that leads to system congestion. RANSAC offers improved robustness but fails to meet real-time constraints. In contrast, the YOLO-OBB-based approach achieves the best balance between robustness and computational efficiency, maintaining an end-to-end latency (satisfying 10 Hz operation) while effectively filtering cluttered observations in a low-power single-board computer (SBC) without using GPU acceleration. The main contribution of this work is a computationally efficient BEV-based perception pipeline enabling reliable real-time structural detection from 3D
LiDAR on resource-constrained robotic platforms that cannot rely on GPU-intensive processing. The source code and pre-trained models are publicly available.
Read the full abstract ▾
All authors Guanliang Li, Pedro Espinosa-Angulo, David Perez-Saura, Santiago Tapia-Fernandez
Robotics · Cross-listed from eess.SY
Mixed-Integer vs. Continuous Model Predictive Control for Binary Thrusters: A Comparative Study
Franek Stark, Jakob Middelberg and Shubham Vyas
Binary on/off thrusters are commonly used for spacecraft attitude and position control during proximity operations. However, their discrete nature poses challenges for conventional continuous control methods. The control of these discrete actuators is either explicitly formulated as a mixed-integer optimization problem or handled in a two-layer approach, where a continuous controller's output is converted to binary commands using analog-to digital modulation techniques such as Delta-Sigma-modulation. This paper provides the first systematic comparison between these two paradigms for binary thruster control, contrasting continuous Model Predictive Control (MPC) with Delta-Sigma modulation against direct Mixed-Integer MPC (MIMPC) approaches. Furthermore, we propose a new variant of MPC for binary actuated systems, which is informed using the state of the Delta-Sigma Modulator. The two variations for the continuous MPC along with the MIMPC are evaluated through extensive simulations using ESA's REACSA platform. Results demonstrate that while all approaches perform similarly in high-thrust regimes, MIMPC achieves superior fuel efficiency in low-thrust conditions. Continuous MPC with modulation shows instabilities at higher thrust levels, while binary informed MPC, which incorporates modulator dynamics, improves robustness and reduces the efficiency gap to the MIMPC. It can be seen from the simulated and real-system experiments that MIMPC offers complete stability and fuel efficiency benefits, particularly for resource-constrained missions, while continuous control methods remain attractive for computationally limited applications.
Read the full abstract ▾
Robotics
Generalized Task-Driven Design of Soft Robots via Reduced-Order FEM-based Surrogate Modeling
Yao Yao, David Howard and Perla Maiolino
Task-driven design of soft
robots requires models that are physically accurate and computationally efficient, while remaining transferable across actuator designs and task scenarios. However, existing modeling approaches typically face a fundamental trade-off between physical fidelity and computational efficiency, which limits model reuse across design and task variations and constrains scalable task-driven optimization. This paper presents a unified reduced-order finite element method (FEM)-based surrogate modeling pipeline for generalized task-driven soft
robot design. High-fidelity FEM simulations characterize actuator behavior at the modular level, from which compact surrogate joint models are constructed for evaluation within a pseudo-rigid body model (PRBM). A meta-model maps actuator design parameters to surrogate representations, enabling rapid instantiation across a parameterized actuator family. The resulting models are embedded into a PRBM-based simulation environment, supporting task-level simulation and optimization under realistic physical constraints. The proposed pipeline is validated through sim-to-real transfer across multiple actuator types, including bellow-type pneumatic actuators and a tendon-driven soft finger, as well as two task-driven design studies: soft gripper co-design via Reinforcement Learning (RL) and 3D actuator shape matching via evolutionary optimization. The results demonstrate high accuracy, efficiency, and reliable reuse, providing a scalable foundation for autonomous task-driven soft
robot design.
Read the full abstract ▾
Robotics
Morphology-Consistent Humanoid Interaction through Robot-Centric Video Synthesis
Weisheng Xu, Jian Li, Yi Gu et al. (+12 more)
Equipping
humanoid robots with versatile interaction skills typically requires either extensive policy training or explicit human-to-
robot motion retargeting. However, learning-based policies face prohibitive data collection costs. Meanwhile, retargeting relies on human-centric pose estimation (e.g., SMPL), introducing a morphology gap. Skeletal scale mismatches result in severe spatial misalignments when mapped to
robots, compromising interaction success. In this work, we propose Dream2Act, a
robot-centric framework enabling zero-shot interaction through generative
video synthesis. Given a third-person image of the
robot and target object, our framework leverages
video generation models to envision the
robot completing the task with morphology-consistent motion. We employ a high-fidelity pose extraction system to recover physically feasible,
robot-native joint trajectories from these synthesized dreams, subsequently executed via a general-purpose whole-body controller. Operating strictly within the
robot-native coordinate space, Dream2Act avoids retargeting errors and eliminates task-specific policy training. We evaluate Dream2Act on the Unitree G1 across four whole-body mobile interaction tasks: ball kicking, sofa sitting, bag punching, and box hugging. Dream2Act achieves a 37.5% overall success rate, compared to 0% for conventional retargeting. While retargeting fails to establish correct physical contacts due to the morphology gap (with errors compounded during locomotion), Dream2Act maintains
robot-consistent spatial
alignment, enabling reliable contact formation and substantially higher task completion.
Read the full abstract ▾
All authors Weisheng Xu, Jian Li, Yi Gu, Bin Yang, Haodong Chen, Shuyi Lin, Mingqian Zhou, Jing Tan, Qiwei Wu, Xiangrui Jiang, Taowen Wang, Jiawen Wen, Qiwei Liang, Jiaxi Zhang, Renjing Xu
Robotics · Cross-listed from cs.CV
DynFlowDrive: Flow-Based Dynamic World Modeling for Autonomous Driving
Xiaolu Liu, Yicong Li, Song Wang et al. (+3 more)
Recently, world models have been incorporated into the autonomous driving systems to improve the planning reliability. Existing approaches typically predict future states through appearance generation or deterministic regression, which limits their ability to capture trajectory-conditioned scene
evolution and leads to unreliable action planning. To address this, we propose DynFlowDrive, a latent world model that leverages flow-based dynamics to model the transition of world states under different driving actions. By adopting the rectifiedflow formulation, the model learns a velocity field that describes how the scene state changes under different driving actions, enabling progressive prediction of future latent states. Building upon this, we further introduce a stability-aware multi-mode trajectory selection strategy that evaluates candidate trajectories according to the stability of the induced scene transitions. Extensive experiments on the nuScenes and NavSim
benchmarks demonstrate consistent improvements across diverse driving frameworks without introducing additional inference overhead. Source code will be abaliable at https://github.com/xiaolul2/DynFlowDrive.
Read the full abstract ▾
All authors Xiaolu Liu, Yicong Li, Song Wang, Junbo Chen, Angela Yao, Jianke Zhu
Robotics
Legged Autonomous Surface Science In Analogue Environments (LASSIE): Making Every Robotic Step Count in Planetary Exploration
Cristina G. Wilson, Marion Nachon, Shipeng Liu et al. (+20 more)
The ability to efficiently and effectively explore planetary surfaces is currently limited by the capability of wheeled rovers to traverse challenging terrains, and by pre-programmed data acquisition plans with limited in-situ flexibility. In this paper, we present two novel approaches to address these limitations: (i) high-mobility legged
robots that use direct surface interactions to collect rich information about the terrain's mechanics to guide exploration; (ii) human-inspired data acquisition
algorithms that enable
robots to reason about scientific hypotheses and adapt exploration priorities based on incoming ground-sensing measurements. We successfully verify our approach through lab work and field deployments in two planetary analog environments. The new capability for legged
robots to measure soil mechanical properties is shown to enable effective traversal of challenging terrains. When coupled with other geologic properties (e.g., composition, thermal properties, and grain size data etc), soil mechanical measurements reveal key factors governing the formation and development of geologic environments. We then demonstrate how human-inspired
algorithms turn terrain-sensing
robots into teammates, by supporting more flexible and adaptive data collection decisions with human scientists. Our approach therefore enables exploration of a wider range of planetary environments and new substrate investigation opportunities through integrated human-
robot systems that support maximum scientific return.
Read the full abstract ▾
All authors Cristina G. Wilson, Marion Nachon, Shipeng Liu, John G. Ruck, J. Diego Caporale, Benjamin E. McKeeby, Yifeng Zhang, Jordan M. Bretzfelder, John Bush, Alivia M. Eng, Ethan Fulcher, Emmy B. Hughes, Ian C. Rankin, Jelis J. Sostre Cortés, Sophie Silver, Michael R. Zanetti, Ryan C. Ewing, Kenton R. Fisher, Douglas J. Jerolmack, Daniel E. Koditschek, Frances Rivera-Hernández, Thomas F. Shipley, Feifei Qian
Robotics
Accurate Open-Loop Control of a Soft Continuum Robot Using Visually Learned Latent Dynamics
Henrik Krauss, Johann Licher, Naoya Takeishi et al. (+2 more)
This work addresses open-loop control of a soft continuum
robot (SCR) from
video-learned latent dynamics. Visual Oscillator Networks (VONs) from previous work are used, which provide mechanically interpretable 2D oscillator latents through an
attention broadcast decoder (ABCD). Open-loop, single-shooting optimal control is performed in latent space to track image-specified waypoints without camera feedback. An interactive SCR live simulator enables design of static, dynamic, and extrapolated targets and maps them to model-specific latent waypoints. On a two-segment pneumatic SCR, Koopman, MLP, and oscillator dynamics, each with and without ABCD, are evaluated on setpoint and dynamic trajectories. ABCD-based models consistently reduce image-space tracking error. The VON and ABCD-based Koopman models attain the lowest mean squared errors (MSEs). Ablation and simulation stress tests confirm that training and architecture choices support open-loop performance, including static holding, stable extrapolated
equilibria, and relaxation to rest. To the best of our knowledge, this is the first demonstration of reliable long-horizon open-loop control of a physical soft
robot using interpretable latent dynamics models learned solely from
video.
Read the full abstract ▾
All authors Henrik Krauss, Johann Licher, Naoya Takeishi, Annika Raatz, Takehisa Yairi
Robotics
ContractionPPO: Certified Reinforcement Learning via Differentiable Contraction Layers
Vrushabh Zinage, Narek Harutyunyan, Eric Verheyden et al. (+2 more)
Legged locomotion in unstructured environments demands not only high-performance control policies but also formal guarantees to ensure robustness under perturbations. Control methods often require carefully designed reference trajectories, which are challenging to construct in high-dimensional, contact-rich systems such as quadruped
robots. In contrast, Reinforcement Learning (RL) directly learns policies that implicitly generate motion, and uniquely benefits from access to privileged information, such as full state and dynamics during training, that is not available at deployment. We present ContractionPPO, a framework for certified robust planning and control of legged
robots by augmenting Proximal Policy Optimization (PPO) RL with a state-dependent contraction metric layer. This approach enables the policy to maximize performance while simultaneously producing a contraction metric that certifies incremental exponential stability of the simulated closed-loop system. The metric is parameterized as a Lipschitz neural network and trained jointly with the policy, either in parallel or as an auxiliary head of the PPO backbone. While the contraction metric is not deployed during real-world execution, we derive upper bounds on the worst-case contraction rate and show that these bounds ensure the learned contraction metric generalizes from simulation to real-world deployment. Our hardware experiments on quadruped locomotion demonstrate that ContractionPPO enables robust, certifiably stable control even under strong external perturbations.
Read the full abstract ▾
All authors Vrushabh Zinage, Narek Harutyunyan, Eric Verheyden, Fred Y. Hadaegh, Soon-Jo Chung
Robotics · Cross-listed from cs.CV
LoD-Loc v3: Generalized Aerial Localization in Dense Cities using Instance Silhouette Alignment
Shuaibang Peng, Juelin Zhu, Xia Li et al. (+4 more)
We present LoD-Loc v3, a novel method for generalized aerial visual localization in dense urban environments. While prior work LoD-Loc v2 achieves localization through semantic building silhouette
alignment with low-detail city models, it suffers from two key limitations: poor cross-scene generalization and frequent failure in dense building scenes. Our method addresses these challenges through two key innovations. First, we develop a new synthetic data generation pipeline that produces InsLoD-Loc - the largest instance
segmentation dataset for aerial imagery to date, comprising 100k images with precise instance building annotations. This enables trained models to exhibit remarkable zero-shot generalization capability. Second, we reformulate the localization paradigm by shifting from semantic to instance silhouette
alignment, which significantly reduces pose estimation ambiguity in dense scenes. Extensive experiments demonstrate that LoD-Loc v3 outperforms existing state-of-the-art (SOTA) baselines, achieving superior performance in both cross-scene and dense urban scenarios with a large margin. The project is available at https://nudt-sawlab.github.io/LoD-Locv3/.
Read the full abstract ▾
All authors Shuaibang Peng, Juelin Zhu, Xia Li, Kun Yang, Maojun Zhang, Yu Liu, Shen Yan
Robotics
CeRLP: A Cross-embodiment Robot Local Planning Framework for Visual Navigation
Haoyu Xi, Mingao Tan, Xinming Zhang et al. (+5 more)
Visual
navigation for cross-embodiment
robots is challenging due to variations in
robot and camera configurations, which can lead to the failure of
navigation tasks. Previous approaches typically rely on collecting massive
datasets across different
robots, which is highly data-intensive, or fine-tuning models, which is time-consuming. Furthermore, both methods often lack explicit consideration of
robot geometry. In this paper, we propose a Cross-embodiment
Robot Local Planning (CeRLP) framework for general visual
navigation, which abstracts visual information into a unified
geometric formulation and applies to heterogeneous
robots with varying physical dimensions, camera parameters, and camera types. CeRLP introduces a depth estimation scale correction method that utilizes offline pre-calibration to resolve the scale ambiguity of
monocular depth estimation, thereby recovering precise metric depth images. Furthermore, CeRLP designs a visual-to-scan abstraction module that projects varying visual inputs into height-adaptive
laser scans, making the policy robust to heterogeneous
robots. Experiments in simulation environments demonstrate that CeRLP outperforms comparative methods, validating its robust obstacle avoidance capabilities as a local planner. Additionally, extensive real-world experiments verify the effectiveness of CeRLP in tasks such as point-to-point
navigation and
vision-
language navigation, demonstrating its generalization across varying
robot and camera configurations.
Read the full abstract ▾
All authors Haoyu Xi, Mingao Tan, Xinming Zhang, Siwei Cheng, Shanze Wang, Yin Gu, Xiaoyu Shen, Wei Zhang