Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Computer Science Robotics

cs.RO Watch Robotics

Friday 20 February

Robotics

EgoPush: Egocentric Multi-Object Rearrangement for Mobile Robots via Constrained Teacher Observability

Boyuan An, Zhexiong Wang, Yipeng Wang et al. (+4 more)

Humans rearrange objects in cluttered environments using egocentric perception, actively moving to keep task-relevant spatial cues in view. Mobile robots have not matched this: rearrangement is usually built on a global pose estimate or a map, which is exactly what a robot carrying one camera lacks, while pushing keeps changing the scene it would have to be built from. We present EgoPush, which pushes objects into anchor-relative formations from onboard RGB-D alone, with no global localization, external tracking, or map at deployment, and transfers zero-shot to a TurtleBot in controlled and visually cluttered scenes. What makes this learnable turns out to be a property of the teacher rather than of the student: three privileged teachers trained with identical rewards, architecture, and hyperparameters all exceed $98\%$ success, yet their distilled egocentric students reach $0\%$, $54.8\%$, and $87.3\%$, the only variable being the teacher's observation function. EgoPush therefore trains the teacher under egocentric observability constraints, restricting it to visibility-limited cues and revealing target references only when the anchor is centrally visible, so that its supervision is recoverable by a depth-based student distilled online. Making the teacher trainable in the first place needs two further pieces: a role-grouped object-centric interface shared by teacher and student, and stage-wise temporally decayed rewards for long-horizon credit assignment. Videos, the playable task, and code are available at https://ai4ce.github.io/EgoPush/.
Read the full abstract

All authors Boyuan An, Zhexiong Wang, Yipeng Wang, Jiaqi Li, Sihang Li, Jing Zhang, Chen Feng

Robotics · Cross-listed from cs.AI

Cross-Embodiment Offline Reinforcement Learning for Heterogeneous Robot Datasets

Haruki Abe, Takayuki Osa, Yusuke Mukuta et al. (+1 more)

Scalable robot policy pre-training has been hindered by the high cost of collecting high-quality demonstrations for each platform. In this study, we address this issue by uniting offline reinforcement learning (offline RL) with cross-embodiment learning. Offline RL leverages both expert and abundant suboptimal data, and cross-embodiment learning aggregates heterogeneous robot trajectories across diverse morphologies to acquire universal control priors. We perform a systematic analysis of this offline RL and cross-embodiment paradigm, providing a principled understanding of its strengths and limitations. To evaluate this offline RL and cross-embodiment paradigm, we construct a suite of locomotion datasets spanning 16 distinct robot platforms. Our experiments confirm that this combined approach excels at pre-training with datasets rich in suboptimal trajectories, outperforming pure behavior cloning. However, as the proportion of suboptimal data and the number of robot types increase, we observe that conflicting gradients across morphologies begin to impede learning. To mitigate this, we introduce an embodiment-based grouping strategy in which robots are clustered by morphological similarity and the model is updated with a group gradient. This simple, static grouping substantially reduces inter-robot conflicts and outperforms existing conflict-resolution methods.
Read the full abstract

All authors Haruki Abe, Takayuki Osa, Yusuke Mukuta, Tatsuya Harada

Robotics · Cross-listed from cs.CV

AWARE: Action-Entropy-Weighted and Attention-Guided Retrieval of Observation Evidence for Vision-Language-Action Models

Jiabing Yang, Yixiang Chen, Yuan Xu et al. (+13 more)

Vision-Language-Action (VLA) models combine multimodal perception with robotic control, demonstrating remarkable potential for generalizable manipulation. Yet, enhancing their execution typically relies on auxiliary sensory inputs (e.g., depth, 3D point clouds) or costly fine-tuning. Inspired by the insight that Feed-Forward Networks (FFNs) act as key-value memory, we propose Action-Entropy-Weighted and Attention-Guided Retrieval of Observation Evidence (AWARE), an effective, training-free, and plug-and-play module for VLAs. Specifically, AWARE tracks layer-wise uncertainty via an element-wise Action Entropy metric, and retrieves observation evidence into the element's FFN through associative memory lookup. Crucially, the retrieved features are weighted by instruction-guided cross-modal attention to isolate task-relevant observation cues, while injection strength is adaptively scaled by the element's entropy and relative network depth. This selectively channels observation evidence to high-entropy action dimensions and deep layers where grounding decays, while leaving converged representations intact. Across LIBERO, SIMPLER, CALVIN, and real-world robot manipulation, AWARE consistently improves diverse VLA architectures with minimal computational overhead, achieving the largest gains on long-horizon tasks without architectural changes or additional training. Project page: https://jiabingyang01.github.io/AWARE/.
Read the full abstract

All authors Jiabing Yang, Yixiang Chen, Yuan Xu, Peiyan Li, Zichen Wen, Bowen Fang, Chenhang Cui, Tao Yu, Qisen Ma, Kai Wang, Ziheng He, Wen Ye, Xiangnan Wu, Yingda Li, Yan Huang, Liang Wang

Robotics

Quasi-Periodic Gaussian Process Predictive Iterative Learning Control

Unnati Nigam, Radhendushka Srivastava, Faezeh Marzbanrad et al. (+1 more)

Repetitive motion tasks are common in robotics, but performance can degrade over time due to environmental changes and robot wear and tear. Iterative learning control (ILC) improves performance by using information from previous iterations to compensate for expected errors in future iterations. This work incorporates the use of Quasi-Periodic Gaussian Processes (QPGPs) into a predictive ILC framework to model and forecast disturbances and drift across iterations. Using a recent structural equation formulation of QPGPs, the proposed approach enables efficient inference with complexity $\mathcal{O}(p^3)$ instead of $\mathcal{O}(i^2p^3)$, where $p$ denotes the number of points within an iteration and $i$ represents the total number of iterations, specially for larger $i$. This formulation also enables parameter estimation without loss of information, making continual GP learning computationally feasible within the control loop. By predicting next-iteration error profiles rather than relying only on past errors, the controller achieves faster convergence and maintains this under time-varying disturbances. We benchmark the method against both standard ILC and conventional Gaussian Process (GP)-based predictive ILC on three tasks, autonomous vehicle trajectory tracking, a three-link robotic manipulator, and a real-world Stretch robot experiment. Across all cases, the proposed approach converges faster and remains robust under injected and natural disturbances while reducing computational cost. This highlights its practicality across a range of repetitive dynamical systems.
Read the full abstract

All authors Unnati Nigam, Radhendushka Srivastava, Faezeh Marzbanrad, Michael Burke

Robotics · Cross-listed from cs.LG

Whole-Brain Connectomic Graph Model Enables Whole-Body Locomotion Control in Fruit Fly

Zehao Jin, Yaoye Zhu, Chen Zhang et al. (+1 more)

Animals perform coordinated whole-body movements under the control of neural systems shaped by brain-wide connectivity. The mapping of the whole-brain neural connections, or the connectomes, provides a natural graph for modeling sensorimotor information flow, yet its potential as a neural controller for embodied agents remains largely unexplored. Here, we introduce the Fly-connectomic Graph Model, which directly instantiates the whole-brain connectome of an adult Drosophila as a graph-structured neural controller for movements of a simulated biomechanical fruit fly via deep reinforcement learning. We achieve stable performance across diverse locomotion tasks, as well as better sample efficiency compared to both graph and non-graph baselines. Our results demonstrate a biologically informed way towards effective control policy design by translating whole-brain wiring principles into actionable architectural priors, while also improving the interpretability through dynamic information flow. This work also highlights the potential to bridge neuromechanics with embodied intelligence by providing a computational platform for investigating the sensorimotor transformation underlying animal behavior and a paradigm to advance the development of more nature-aligned intelligent systems.
Read the full abstract

All authors Zehao Jin, Yaoye Zhu, Chen Zhang, Yanan Sui

Robotics

Homotopic information gain for sparse active target tracking

Jennifer Wakulicz, Ki Myung Brian Lee, Teresa Vidal-Calleja et al. (+1 more)

The problem of planning sensing trajectories for a mobile robot to collect observations of a target and predict its future trajectory is known as active target tracking. Enabled by probabilistic motion models, one may solve this problem by exploring the belief space of all trajectory predictions given future sensing actions to maximise information gain. However, for multi-modal motion models the notion of information gain is often ill-defined. This paper proposes a planning approach designed around maximising information regarding the target's homotopy class, or high-level motion. We introduce homotopic information gain, a measure of the expected high-level trajectory information given by a measurement. We show that homotopic information gain is a lower bound for metric or low-level information gain, and is as sparsely distributed in the environment as obstacles are. Planning sensing trajectories to maximise homotopic information results in highly accurate trajectory estimates with fewer measurements than a metric information approach, as supported by our empirical evaluation on real and simulated pedestrian data.
Read the full abstract

All authors Jennifer Wakulicz, Ki Myung Brian Lee, Teresa Vidal-Calleja, Robert Fitch

Robotics

Latent Diffeomorphic Co-Design of End-Effectors for Deformable and Fragile Object Manipulation

Kei Ikemura, Yifei Dong and Florian T. Pokorny

Manipulating deformable and fragile objects remains a fundamental challenge in robotics due to complex contact dynamics and strict requirements on object integrity. Existing approaches typically optimize either end-effector design or control strategies in isolation, limiting achievable performance. In this work, we present the first co-design framework that jointly optimizes end-effector morphology and manipulation control for deformable and fragile object manipulation. We introduce (1) a latent diffeomorphic shape parameterization enabling expressive yet tractable end-effector geometry optimization, (2) a stress-aware bi-level co-design pipeline coupling morphology and control optimization, and (3) a privileged-to-pointcloud policy distillation scheme for zero-shot real-world deployment. We evaluate our approach on challenging food manipulation tasks, including grasping and pushing jelly and scooping fillets. Simulation and real-world experiments demonstrate the effectiveness of the proposed method.
Read the full abstract

Robotics

WHED: A Wearable Hand Exoskeleton for Natural, High-Quality Demonstration Collection

Mingzhang Zhu, Alvin Zhu, Jose Victor S. H. Ramos et al. (+9 more)

Scalable learning of dexterous manipulation remains bottlenecked by the difficulty of collecting natural, high-fidelity human demonstrations of multi-finger hands due to occlusion, complex hand kinematics, and contact-rich interactions. We present WHED, a wearable hand-exoskeleton system designed for in-the-wild demonstration capture, guided by two principles: wearability-first operation for extended use and a pose-tolerant, free-to-move thumb coupling that preserves natural thumb behaviors while maintaining a consistent mapping to the target robot thumb degrees of freedom. WHED integrates a linkage-driven finger interface with passive fit accommodation, a modified passive hand with robust proprioceptive sensing, and an onboard sensing/power module. We also provide an end-to-end data pipeline that synchronizes joint encoders, AR-based end-effector pose, and wrist-mounted visual observations, and supports post-processing for time alignment and replay. We demonstrate feasibility on representative grasping and manipulation sequences spanning precision pinch and full-hand enclosure grasps, and show qualitative consistency between collected demonstrations and replayed executions.
Read the full abstract

All authors Mingzhang Zhu, Alvin Zhu, Jose Victor S. H. Ramos, Beom Jun Kim, Yike Shi, Yufeng Wu, Ruochen Hou, Quanyou Wang, Eric Song, Tony Fan, Yuchen Cui, Dennis W. Hong

Thursday 19 February

Robotics · Cross-listed from cs.LG

MePoly: Max Entropy Polynomial Policy Optimization

Hang Liu, Sangli Teng and Maani Ghaffari

Stochastic Optimal Control provides a unified mathematical framework for solving complex decision-making problems, encompassing paradigms such as maximum entropy reinforcement learning(RL) and imitation learning(IL). However, conventional parametric policies often struggle to represent the multi-modality of the solutions. Though diffusion-based policies are aimed at recovering the multi-modality, they lack an explicit probability density, which complicates policy-gradient optimization. To bridge this gap, we propose MePoly, a novel policy parameterization based on polynomial energy-based models. MePoly provides an explicit, tractable probability density, enabling exact entropy maximization. Theoretically, we ground our method in the classical moment problem, leveraging the universal approximation capabilities for arbitrary distributions. Empirically, we demonstrate that MePoly effectively captures complex non-convex manifolds and outperforms baselines in performance across diverse benchmarks.
Read the full abstract

Robotics

Evolution of Safety Requirements in Industrial Robotics: Comparative Analysis of ISO 10218-1/2 (2011 vs. 2025) and Integration of ISO/TS 15066

Daniel Hartmann, Kristýna Hamříková, Aleš Vysocký et al. (+2 more)

Industrial robotics has established itself as an integral component of large-scale manufacturing enterprises. Simultaneously, collaborative robotics is gaining prominence, introducing novel paradigms of human-machine interaction. These advancements have necessitated a comprehensive revision of safety standards, specifically incorporating requirements for cybersecurity and protection against unauthorized access in networked robotic systems. This article presents a comparative analysis of the ISO 10218:2011 and ISO 10218:2025 standards, examining the evolution of their structure, terminology, technical requirements, and annexes. The analysis reveals significant expansions in functional safety and cybersecurity, the introduction of new classifications for robots and collaborative applications, and the normative integration of the technical specification ISO/TS 15066. Consequently, the new edition synthesizes mechanical, functional, and digital safety requirements, establishing a comprehensive framework for the design and operation of modern robotic systems.
Read the full abstract

All authors Daniel Hartmann, Kristýna Hamříková, Aleš Vysocký, Vendula Laciok, Aleš Bernatík

Robotics

Lend me an Ear: Speech Enhancement Using a Robotic Arm with a Microphone Array

Zachary Turcotte and François Grondin

Speech enhancement performance degrades significantly in noisy environments, limiting the deployment of speech-controlled technologies in industrial settings, such as manufacturing plants. Existing speech enhancement solutions primarily rely on advanced digital signal processing techniques, deep learning methods, or complex software optimization approaches. This paper introduces a novel speech enhancement robotic platform that can reconfigure the geometry of a microphone array and adapt to changing acoustic conditions. A sixteen-microphone array is mounted on a robotic arm manipulator with seven degrees of freedom. The microphones are divided into four groups of four, including one group positioned near the end-effector. The system reconfigures the array by adjusting the manipulator joint angles to place the end-effector microphones closer to the target speaker, thereby improving the reference signal quality. This proposed system is a multimodal sensing, reconfigurable audio capture device that integrates sound source localization techniques, computer vision, inverse kinematics, minimum variance distortionless response beamformer, and time-frequency masking using a deep neural network. Experimental results suggest that this approach outperforms other traditional recording configurations, achieving a higher average scale-invariant signal-to-distortion ratio and lower average word error rate across multiple input signal-to-noise ratio conditions.
Read the full abstract

Robotics

Reinforcement-Learning-Based Assistance Reduces Squat Effort with a Modular Hip–Knee Exoskeleton

Neethan Ratnakumar, Mariya Huzaifa Tohfafarosh, Saanya Jauhri et al. (+1 more)

Squatting is one of the most demanding lower-limb movements, requiring substantial muscular effort and coordination. Reducing the physical demands of this task through intelligent and personalized assistance has significant implications, particularly in industries involving repetitive low-level assembly activities. In this study, we evaluated the effectiveness of a neural network controller for a modular Hip-Knee exoskeleton designed to assist squatting tasks. The neural network controller was trained via reinforcement learning (RL) in a physics-based, human-exoskeleton interaction simulation environment. The controller generated real-time hip and knee assistance torques based on recent joint-angle and velocity histories. Five healthy adults performed three-minute metronome-guided squats under three conditions: (1) no exoskeleton (No-Exo), (2) exoskeleton with Zero-Torque, and (3) exoskeleton with active assistance (Assistance). Physiological effort was assessed using indirect calorimetry and heart rate monitoring, alongside concurrent kinematic data collection. Results show that the RL-based controller adapts to individuals by producing torque profiles tailored to each subject's kinematics and timing. Compared with the Zero-Torque and No-Exo condition, active assistance reduced the net metabolic rate by approximately 10%, with minor reductions observed in heart rate. However, assisted trials also exhibited reduced squat depth, reflected by smaller hip and knee flexion. These preliminary findings suggest that the proposed controller can effectively lower physiological effort during repetitive squatting, motivating further improvements in both hardware design and control strategies.
Read the full abstract

All authors Neethan Ratnakumar, Mariya Huzaifa Tohfafarosh, Saanya Jauhri, Xianlian Zhou

Robotics · Cross-listed from eess.SP

Driving-Over Detection in the Railway Environment

Tobias Herrmann, Nikolay Chenkov, Florian Stark et al. (+2 more)

To enable fully automated driving of trains, numerous new technological components must be introduced into the railway system. Tasks that are nowadays carried out by the operating stuff, need to be taken over by automatic systems. Therefore, equipment for automatic train operation and observing the environment is needed. Here, an important task is the detection of collisions, including both (1) collisions with the front of the train as well as (2) collisions with the wheel, corresponding to an driving-over event. Technologies for detecting the driving-over events are barely investigated nowadays. Therefore, detailed driving-over experiments were performed to gather knowledge for fully automated rail operations, using a variety of objects made from steel, wood, stone and bones. Based on the captured test data, three methods were developed to detect driving-over events automatically. The first method is based on convolutional neural networks and the other two methods are classical threshold-based approaches. The neural network based approach provides an mean accuracy of 99.6% while the classical approaches show 85% and 88.6%, respectively.
Read the full abstract

All authors Tobias Herrmann, Nikolay Chenkov, Florian Stark, Matthias Härter, Martin Köppel

Robotics · Cross-listed from cs.CV

When Vision Overrides Language: Evaluating and Mitigating Counterfactual Failures in VLAs

Yu Fang, Yuchun Feng, Dong Jing et al. (+5 more)

Vision-Language-Action models (VLAs) promise to ground language instructions in robot control, yet in practice often fail to faithfully follow language. When presented with instructions that lack strong scene-specific supervision, VLAs suffer from counterfactual failures: they act based on vision shortcuts induced by dataset biases, repeatedly executing well-learned behaviors and selecting objects frequently seen during training regardless of language intent. To systematically study it, we introduce LIBERO-CF, the first counterfactual benchmark for VLAs that evaluates language following capability by assigning alternative instructions under visually plausible LIBERO layouts. Our evaluation reveals that counterfactual failures are prevalent yet underexplored across state-of-the-art VLAs. We propose Counterfactual Action Guidance (CAG), a simple yet effective dual-branch inference scheme that explicitly regularizes language conditioning in VLAs. CAG combines a standard VLA policy with a language-unconditioned Vision-Action (VA) module, enabling counterfactual comparison during action selection. This design reduces reliance on visual shortcuts, improves robustness on under-observed tasks, and requires neither additional demonstrations nor modifications to existing architectures or pretrained models. Extensive experiments demonstrate its plug-and-play integration across diverse VLAs and consistent improvements. For example, on LIBERO-CF, CAG improves $\pi_{0.5}$ by 9.7% in language following accuracy and 3.6% in task success on under-observed tasks using a training-free strategy, with further gains of 15.5% and 8.5%, respectively, when paired with a VA model. In real-world evaluations, CAG reduces counterfactual failures of 9.4% and improves task success by 17.2% on average.
Read the full abstract

All authors Yu Fang, Yuchun Feng, Dong Jing, Jiaqi Liu, Yue Yang, Zhenyu Wei, Daniel Szafir, Mingyu Ding

Robotics

Graph Neural Model Predictive Control for High-Dimensional Systems

Patrick Benito Eberhard, Luis Pabon, Daniele Gammelli et al. (+5 more)

The control of high-dimensional systems, such as soft robots, requires models that faithfully capture complex dynamics while remaining computationally tractable. This work presents a framework that integrates Graph Neural Network (GNN)-based dynamics models with structure-exploiting Model Predictive Control to enable real-time control of high-dimensional systems. By representing the system as a graph with localized interactions, the GNN preserves sparsity, while a tailored condensing algorithm eliminates state variables from the control problem, ensuring efficient computation. The complexity of our condensing algorithm scales linearly with the number of system nodes, and leverages Graphics Processing Unit (GPU) parallelization to achieve real-time performance. The proposed approach is validated in simulation and experimentally on a physical soft robotic trunk. Results show that our method scales to systems with up to 1,000 nodes at 100 Hz in closed-loop, and demonstrates real-time reference tracking on hardware with sub-centimeter accuracy, outperforming baselines by 63.6%. Finally, we show the capability of our method to achieve effective full-body obstacle avoidance.
Read the full abstract

All authors Patrick Benito Eberhard, Luis Pabon, Daniele Gammelli, Hugo Buurmeijer, Amon Lahr, Mark Leone, Andrea Carron, Marco Pavone

Robotics

Conditional Flow Matching for Continuous Anomaly Detection in Autonomous Driving on a Manifold-Aware Spectral Space

Antonio Guillen-Perez

Safety validation for Level 4 autonomous vehicles (AVs) is currently bottlenecked by the inability to scale the detection of rare, high-risk long-tail scenarios using traditional rule-based heuristics. We present Deep-Flow, an unsupervised framework for safety-critical anomaly detection that utilizes Optimal Transport Conditional Flow Matching (OT-CFM) to characterize the continuous probability density of expert human driving behavior. Unlike standard generative approaches that operate in unstable, high-dimensional coordinate spaces, Deep-Flow constrains the generative process to a low-rank spectral manifold via a Principal Component Analysis (PCA) bottleneck. This ensures kinematic smoothness by design and enables the computation of the exact Jacobian trace for numerically stable, deterministic log-likelihood estimation. To resolve multi-modal ambiguity at complex junctions, we utilize an Early Fusion Transformer encoder with lane-aware goal conditioning, featuring a direct skip-connection to the flow head to maintain intent-integrity throughout the network. We introduce a kinematic complexity weighting scheme that prioritizes high-energy maneuvers (quantified via path tortuosity and jerk) during the simulation-free training process. Evaluated on the Waymo Open Motion Dataset (WOMD), our framework achieves an AUC-ROC of 0.766 against a heuristic golden set of safety-critical events. More significantly, our analysis reveals a fundamental distinction between kinematic danger and semantic non-compliance. Deep-Flow identifies a critical predictability gap by surfacing out-of-distribution behaviors, such as lane-boundary violations and non-normative junction maneuvers, that traditional safety filters overlook. This work provides a mathematically rigorous foundation for defining statistical safety gates, enabling objective, data-driven validation for the safe deployment of autonomous fleets.
Read the full abstract

Robotics

Hybrid System Planning using a Mixed-Integer ADMM Heuristic and Hybrid Zonotopes

Joshua A. Robbins, Andrew F. Thompson, Jonah J. Glunt et al. (+1 more)

Embedded optimization-based planning for hybrid systems is challenging due to the use of mixed-integer programming, which is computationally intensive and often sensitive to the specific numerical formulation. To address that challenge, this article proposes a framework for motion planning of hybrid systems that pairs hybrid zonotopes - an advanced set representation - with a new alternating direction method of multipliers (ADMM) mixed-integer programming heuristic. A general treatment of piecewise affine (PWA) system reachability analysis using hybrid zonotopes is presented and extended to formulate optimal planning problems. Sets produced using the proposed identities have lower memory complexity and tighter convex relaxations than equivalent sets produced from preexisting techniques. The proposed ADMM heuristic makes efficient use of the hybrid zonotope structure. For planning problems formulated as hybrid zonotopes, the proposed heuristic achieves improved convergence rates as compared to state-of-the-art mixed-integer programming heuristics. The proposed methods for hybrid system planning on embedded hardware are experimentally applied in a combined behavior and motion planning scenario for autonomous driving.
Read the full abstract

All authors Joshua A. Robbins, Andrew F. Thompson, Jonah J. Glunt, Herschel C. Pangborn

Robotics

FR-GESTURE: An RGBD Dataset For Gesture-based Human-Robot Interaction In First Responder Operations

Konstantinos Foteinos, Georgios Angelidis, Aggelos Psiris et al. (+3 more)

The ever increasing intensity and number of disasters make even more difficult the work of First Responders (FRs). Artificial intelligence and robotics solutions could facilitate their operations, compensating these difficulties. To this end, we propose a dataset for gesture-based UGV control by FRs, introducing a set of 12 commands, drawing inspiration from existing gestures used by FRs and tactical hand signals and refined after incorporating feedback from experienced FRs. Then we proceed with the data collection itself, resulting in 3312 RGBD pairs captured from 2 viewpoints and 7 distances. To the best of our knowledge, this is the first dataset especially intended for gesture-based UGV guidance by FRs. Finally we define evaluation protocols for our RGBD dataset, termed FR-GESTURE, and we perform baseline experiments, which are put forward for improvement. We have made data publicly available to promote future research on the domain: https://doi.org/10.5281/zenodo.18131333.
Read the full abstract

All authors Konstantinos Foteinos, Georgios Angelidis, Aggelos Psiris, Vasileios Argyriou, Panagiotis Sarigiannidis, Georgios Th. Papadopoulos

Robotics

IRIS: Learning-Driven Task-Specific Cinema Robot Arm for Visuomotor Motion Control

Qilong Cheng, Matthew Mackay and Ali Bereyhi

Robotic camera systems enable dynamic, repeatable motion beyond human capabilities, yet their adoption remains limited by the high cost and operational complexity of industrial-grade platforms. We present the Intelligent Robotic Imaging System (IRIS), a task-specific 6-DOF manipulator designed for autonomous, learning-driven cinematic motion control. IRIS integrates a lightweight, fully 3D-printed hardware design with a goal-conditioned visuomotor imitation learning framework based on Action Chunking with Transformers (ACT). The system learns object-aware and perceptually smooth camera trajectories directly from human demonstrations, eliminating the need for explicit geometric programming. The complete platform costs under $1,000 USD, supports a 1.5 kg payload, and achieves approximately 1 mm repeatability. Real-world experiments demonstrate accurate trajectory tracking, reliable autonomous execution, and generalization across diverse cinematic motions.
Read the full abstract

Robotics

RA-Nav: A Risk-Aware Navigation System Based on Semantic Segmentation for Aerial Robots in Unpredictable Environments

Ziyi Zong, Xin Dong, Jinwu Xiang et al. (+2 more)

Existing aerial robot navigation systems typically plan paths around static and dynamic obstacles, but fail to adapt when a static obstacle suddenly moves. Integrating environmental semantic awareness enables estimation of potential risks posed by suddenly moving obstacles. In this paper, we propose RA- Nav, a risk-aware navigation framework based on semantic segmentation. A lightweight multi-scale semantic segmentation network identifies obstacle categories in real time. These obstacles are further classified into three types: stationary, temporarily static, and dynamic. For each type, corresponding risk estimation functions are designed to enable real-time risk prediction, based on which a complete local risk map is constructed. Based on this map, the risk-informed path search algorithm is designed to guarantee planning that balances path efficiency and safety. Trajectory optimization is then applied to generate trajectories that are safe, smooth, and dynamically feasible. Comparative simulations demonstrate that RA-Nav achieves higher success rates than baselines in sudden obstacle state transition scenarios. Its effectiveness is further validated in simulations using real- world data.
Read the full abstract

All authors Ziyi Zong, Xin Dong, Jinwu Xiang, Daochun Li, Zhan Tu

Robotics · Cross-listed from eess.SY

Dodging the Moose: Experimental Insights in Real-Life Automated Collision Avoidance

Leila Gharavi, Simone Baldi, Yuki Hosomi et al. (+4 more)

The sudden appearance of a static obstacle on the road, i.e. the moose test, is a well-known emergency scenario in collision avoidance for automated driving. Model Predictive Control (MPC) has long been employed for planning and control of automated vehicles in the state of the art. However, real-time implementation of automated collision avoidance in emergency scenarios such as the moose test remains unaddressed due to the high computational demand of MPC for evasive action in such hazardous scenarios. This paper offers new insights into real-time collision avoidance via the experimental imple- mentation of MPC for motion planning after a sudden and unexpected appearance of a static obstacle. As the state-of-the-art nonlinear MPC shows limited capability to provide an acceptable solution in real-time, we propose a human-like feed-forward planner to assist when the MPC optimization problem is either infeasible or unable to find a suitable solution due to the poor quality of its initial guess. We introduce the concept of maximum steering maneuver to design the feed-forward planner and mimic a human-like reaction after detecting the static obstacle on the road. Real-life experiments are conducted across various speeds and level of emergency using FPEV2-Kanon electric vehicle. Moreover, we demonstrate the effectiveness of our planning strategy via comparison with the state-of- the-art MPC motion planner.
Read the full abstract

All authors Leila Gharavi, Simone Baldi, Yuki Hosomi, Tona Sato, Bart De Schutter, Binh-Minh Nguyen, Hiroshi Fujimoto

Robotics

Proximal powered knee placement: a case study

Kyle R. Embry, Lorenzo Vianello, Jim Lipsey et al. (+8 more)

Lower limb amputation affects millions worldwide, leading to impaired mobility, reduced walking speed, and limited participation in daily and social activities. Powered prosthetic knees can partially restore mobility by actively assisting knee joint torque, improving gait symmetry, sit-to-stand transitions, and walking speed. However, added mass from powered components may diminish these benefits, negatively affecting gait mechanics and increasing metabolic cost. Consequently, optimizing mass distribution, rather than simply minimizing total mass, may provide a more effective and practical solution. In this exploratory study, we evaluated the feasibility of above-knee powertrain placement for a powered prosthetic knee in a small cohort. Compared to below-knee placement, the above-knee configuration demonstrated improved walking speed (+9.2% for one participant) and cadence (+3.6%), with mixed effects on gait symmetry. Kinematic measures indicated similar knee range of motion and peak velocity across configurations. Additional testing on ramps and stairs confirmed the robustness of the control strategy across multiple locomotion tasks. These preliminary findings suggest that above-knee placement is functionally feasible and that careful mass distribution can preserve the benefits of powered assistance while mitigating adverse effects of added weight. Further studies are needed to confirm these trends and guide design and clinical recommendations.
Read the full abstract

All authors Kyle R. Embry, Lorenzo Vianello, Jim Lipsey, Frank Ursetta, Michael Stephens, Zhi Wang, Ann M. Simon, Andrea J. Ikeda, Suzanne B. Finucane, Shawana Anarwala, Levi J. Hargrove

Robotics

Optically Sensorized Electro-Ribbon Actuator (OS-ERA)

Carolina Gay, Petr Trunin, Diana Cafiso et al. (+3 more)

Electro-Ribbon Actuators (ERAs) are lightweight flexural actuators that exhibit ultrahigh displacement and fast movement. However, their embedded sensing relies on capacitive sensors with limited precision, which hinders accurate control. We introduce OS-ERA, an optically sensorized ERA that yields reliable proprioceptive information, and we focus on the design and integration of a sensing solution without affecting actuation. To analyse the complex curvature of an ERA in motion, we design and embed two soft optical waveguide sensors. A classifier is trained to map the sensing signals in order to distinguish eight bending states. We validate our model on six held-out trials and compare it against signals' trajectories learned from training runs. Across all tests, the sensing output signals follow the training manifold, and the predicted sequence mirrors real performance and confirms repeatability. Despite deliberate train-test mismatches in actuation speed, the signal trajectories preserve their shape, and classification remains consistently accurate, demonstrating practical voltage- and speed-invariance. As a result, OS-ERA classifies bending states with high fidelity; it is fast and repeatable, solving a longstanding bottleneck of the ERA, enabling steps toward closed-loop control.
Read the full abstract

All authors Carolina Gay, Petr Trunin, Diana Cafiso, Yuejun Xu, Majid Taghavi, Lucia Beccai

Robotics

A Cost-Effective and Climate-Resilient Air Pressure System for Rain Effect Reduction on Automated Vehicle Cameras

Mohamed Sabry, Joseba Gorospe and Cristina Olaverri-Monreal

Recent advances in automated vehicles have focused on improving perception performance under adverse weather conditions; however, research on physical hardware solutions remains limited, despite their importance for perception critical applications such as vehicle platooning. Existing approaches, such as hydrophilic or hydrophobic lenses and sprays, provide only partial mitigation, while industrial protection systems imply high cost and they do not enable scalability for automotive deployment. To address these limitations, this paper presents a cost-effective hardware solution for rainy conditions, designed to be compatible with multiple cameras simultaneously. Beyond its technical contribution, the proposed solution supports sustainability goals in transportation systems. By enabling compatibility with existing camera-based sensing platforms, the system extends the operational reliability of automated vehicles without requiring additional high-cost sensors or hardware replacements. This approach reduces resource consumption, supports modular upgrades, and promotes more cost-efficient deployment of automated vehicle technologies, particularly in challenging weather conditions where system failures would otherwise lead to inefficiencies and increased emissions. The proposed system was able to increase pedestrian detection accuracy of a Deep Learning model from 8.3% to 41.6%.
Read the full abstract

Robotics

3D-printed Soft Optical sensor with a Lens (SOLen) for light guidance in mechanosensing

Diana Cafiso, Petr Trunin, Carolina Gay et al. (+1 more)

Additive manufacturing is enabling soft robots with increasingly complex geometries, creating a demand for sensing solutions that remain compatible with single-material, one-step fabrication. Optical soft sensors are attractive for monolithic printing, but their performance is often degraded by uncontrolled light propagation (ambient coupling, leakage, scattering), while common miti- gation strategies typically require multimaterial interfaces. Here, we present an approach for 3D printed soft optical sensing (SOLen), in which a printed lens is placed in front of an emitter within a Y-shaped waveguide. The sensing mechanism relies on deformation-induced lens rotation and focal-spot translation, redistributing optical power between the two branches to generate a differential output that encodes both motion direction and amplitude. An acrylate polyurethane resin was modified with lauryl acrylate to improve compliance and optical transmittance, and single-layer optical characterization was used to derive wavelength-dependent refractive index and transmittance while minimizing DLP layer-related artifacts. The measured refractive index was used in simulations to design a lens profile for a target focal distance, which was then printed with sub-millimeter fidelity. Rotational tests demonstrated reproducible branch-selective signal switching over multiple cycles. These results establish a transferable material-to-optics workflow for soft optical sensors with lens with new functionalities for next-generation soft robots
Read the full abstract

All authors Diana Cafiso, Petr Trunin, Carolina Gay, Lucia Beccai

Robotics

Distributed Virtual Model Control for Scalable Human-Robot Collaboration in Shared Workspace

Yi Zhang, Omar Faris, Chapa Sirithunge et al. (+3 more)

We present a decentralized, agent agnostic, and safety-aware control framework for human-robot collaboration based on Virtual Model Control (VMC). In our approach, both humans and robots are embedded in the same virtual-component-shaped workspace, where motion is the result of the interaction with virtual springs and dampers rather than explicit trajectory planning. A decentralized, force-based stall detector identifies deadlocks, which are resolved through negotiation. This reduces the probability of robots getting stuck in the block placement task from up to 61.2% to zero in our experiments. The framework scales without structural changes thanks to the distributed implementation: in experiments we demonstrate safe collaboration with up to two robots and two humans, and in simulation up to four robots, maintaining inter-agent separation at around 20 cm. Results show that the method shapes robot behavior intuitively by adjusting control parameters and achieves deadlock-free operation across team sizes in all tested scenarios.
Read the full abstract

All authors Yi Zhang, Omar Faris, Chapa Sirithunge, Kai-Fung Chu, Fumiya Iida, Fulvio Forni

Robotics · Cross-listed from eess.SY

Bluetooth Phased-array Aided Inertial Navigation Using Factor Graphs: Experimental Verification

Glen Hjelmerud Mørkbak Sørensen, Torleiv H. Bryne, Kristoffer Gryte et al. (+1 more)

Phased-array Bluetooth systems have emerged as a low-cost alternative for performing aided inertial navigation in GNSS-denied use cases such as warehouse logistics, drone landings, and autonomous docking. Basing a navigation system off of commercial-off-the-shelf components may reduce the barrier of entry for phased-array radio navigation systems, albeit at the cost of significantly noisier measurements and relatively short feasible range. In this paper, we compare robust estimation strategies for a factor graph optimisation-based estimator using experimental data collected from multirotor drone flight. We evaluate performance in loss-of-GNSS scenarios when aided by Bluetooth angular measurements, as well as range or barometric pressure.
Read the full abstract

All authors Glen Hjelmerud Mørkbak Sørensen, Torleiv H. Bryne, Kristoffer Gryte, Tor Arne Johansen

Robotics

Contact-Anchored Proprioceptive Odometry for Legged and Wheel-Legged Robots

Minxing Sun and Yao Mao

Reliable odometry for legged robots without cameras or LiDAR remains challenging due to IMU drift and noisy joint velocity sensing. This paper presents a purely proprioceptive state estimator that uses only IMU and motor measurements to estimate body pose and velocity, with a unified formulation applicable to quadruped and wheel-legged robots and extensible to other legged morphologies. The key idea is to treat each reliable contact as a kinematic anchor: joint-torque–based foot wrench estimation selects stance contacts, and the corresponding footfall records provide intermittent world-frame constraints that suppress long-term drift. To prevent elevation drift during extended traversal, we introduce a lightweight height clustering and time-decay correction that snaps newly recorded footfall heights to previously observed support planes. For wheel-legged platforms, the recorded contact is further propagated by effective wheel rolling displacement with shank-motion compensation and a slope-aware rolling direction. To improve foot velocity observations under encoder quantization, we retain an inverse-kinematics cubature Kalman filter as an optional velocity-enhancement module that filters foot-end velocities from joint angles and velocities. The implementation further mitigates yaw drift through multi-contact geometric consistency, which is injected as a soft heading prior rather than as a hard reset of the attitude state. The method is evaluated on four quadruped platforms.
Read the full abstract

Robotics

FRAPPE: Infusing World Modeling into Generalist Policies via Multiple Future Representation Alignment

Han Zhao, Jingbo Wang, Wenxuan Song et al. (+5 more)

Enabling VLA models to predict environmental dynamics, known as world modeling, has been recognized as essential for improving robotic reasoning and generalization. However, current approaches face two main issues: 1. The training objective forces models to over-emphasize pixel-level reconstruction, which constrains semantic learning and generalization 2. Reliance on predicted future observations during inference often leads to error accumulation. To address these challenges, we introduce Future Representation Alignment via Parallel Progressive Expansion (FRAPPE). Our method adopts a two-stage fine-tuning strategy: In the mid-training phase, the model learns to predict the latent representations of future observations; In the post-training phase, we expand the computational workload in parallel and align the representation simultaneously with multiple different visual foundation models. By significantly improving fine-tuning efficiency and reducing dependence on action-annotated data, FRAPPE provides a scalable and data-efficient pathway to enhance world-awareness in generalist robotic policies. Experiments on the RoboTwin benchmark and real-world tasks demonstrate that FRAPPE outperforms state-of-the-art approaches and shows strong generalization in long-horizon and unseen scenarios.
Read the full abstract

All authors Han Zhao, Jingbo Wang, Wenxuan Song, Shuai Chen, Yang Liu, Yan Wang, Haoang Li, Donglin Wang

Robotics

Topology-Aware Decision Making for Multi-Session Localization and Mapping

Lorenzo Montano-Olivan, Julio A. Placed, Luis Montano et al. (+1 more)

Operating in previously visited environments is becoming increasingly crucial for autonomous systems, with direct applications in autonomous driving, surveying, and warehouse or household robotics. This repeated exposure to observing the same areas poses significant challenges for mapping and localization across sessions, particularly in deciding when a prior model is sufficient for reliable localization and when new mapping is required. In this work, we propose a novel multi-session framework that builds on map-based localization, in contrast to the common practice of greedily running full SLAM sessions and trying to find correspondences between the resulting maps. The core contribution is a principled, topology-driven mechanism to detect multi-session mapping needs from the pose-graph structure. Specifically, our approach uses spectral connectivity metrics on the joint pose-graph to identify disconnections and weakly constrained regions, and selectively triggers mapping and loop closing only when the pose-graph topology indicates insufficient structural support. The resulting map and pose-graph are seamlessly integrated into the existing model, reducing accumulated error and enhancing global consistency while avoiding redundant remapping. We validate our method on overlapping sequences from datasets and demonstrate its effectiveness in a real-world mine-like environment.
Read the full abstract

All authors Lorenzo Montano-Olivan, Julio A. Placed, Luis Montano, Maria T. Lazaro