Systems and Control
Forward-Invariant Policy Classes for Safe Reinforcement Learning in Multicopter Control
Chieh Tsai, Muhammad Junayed Hasan Zahed, Jinzhi Shen et al. (+5 more)
This paper proposes a reinforcement learning (RL) framework for safe gain scheduling based on forwardinvariance-induced action-space design. Under stated nominalmodel and inversion-domain assumptions, rather than enforcing safety through runtime shielding or penalty-based constraints, safety is embedded directly into the policy class. Specifically, we construct a finite library of feedback controllers sharing a common Lyapunov certificate that establishes forward invariance of a prescribed admissible set under arbitrary switching. Consequently, any policy whose actions are restricted to this library, including policies encountered during RL exploration, inherits the same certificate. Policy optimization can therefore focus on closed-loop performance without runtime safety filtering or action projection. The framework is instantiated for quadcopter hover regulation, where a DQN schedules among certified feedback controllers. Nonlinear MuJoCo simulations demonstrate state-dependent gain scheduling and empirically evaluate robustness to wind, model mismatch, sensor noise, and sensing delay. The results illustrate how safety certification can be separated from policy optimization by learning over a forward-invariant policy class.
Read the full abstract ▾
All authors Chieh Tsai, Muhammad Junayed Hasan Zahed, Jinzhi Shen, Yi Xie, Ruoshan Lan, Majed Obaid, Salim Hariri, Hossein Rastgoftar
Systems and Control
VHF Reconfigurable Intelligent Surfaces for Meteor Burst Communication
Rajiv Thummala, Luke Flores and Gregory Falco
Meteor Burst Communication (MBC) utilizes the transient ionized trails left by meteors to reflect Very High Frequency (VHF) signals, enabling long-range, beyond-line-of-sight communication without reliance on terrestrial or satellite infrastructure. Despite its resilience in austere and contested environments, MBC is limited by brief communication windows, low signal-to-noise ratio (SNR), and inconsistent channel availability. Recent advancements in Reconfigurable Intelligent Surfaces (RIS) offer a transformative solution by enabling dynamic control over signal propagation through programmable reflective elements. This paper proposes a novel system architecture that integrates RIS into MBC networks, incorporating real-time adaptive control, optimized deployment strategies, and Monte Carlo simulations to refine RIS placement and operational parameters.
Read the full abstract ▾
Systems and Control
Probability-Based Collision Risk Evaluation of Trajectories for Optimal Control Problems with Moving Obstacles
Florian Steppich and Matthias Gerdts
This paper presents a method for approximating occupancy distributions, a common probabilistic representation used in trajectory planning for autonomous vehicles operating in uncertain environments. The proposed method employs B-spline surfaces in conjunction with regularization techniques to ensure smoothness and differentiability. Interpreting the resulting approximation as an intensity function of a Poisson random field enables seamless integration of multiple occupancy distributions into a coherent risk representation, while retaining compatibility with the semantics of probability distributions. The method is tailored to optimal control problems (OCPs), where solvers benefit from gradient and higher-order derivative information. It also preserves information about the spatial and temporal structure of occupancy distributions, which is critical for handling dynamic obstacles. We demonstrate the feasibility of the approach in a trajectory planning scenario involving autonomous vehicles and moving obstacles with time-varying uncertainty. By enabling controlled risk acceptance, our formulation extends beyond conservative "no-collision" strategies and allows for admissible trajectories that would otherwise be ruled out. In our application scenario, we report a reduction in solver computational load of up to 50% through proper tuning of the regularization.
Read the full abstract ▾
Systems and Control
Data-Driven Communication Topology and Distributed Controller Synthesis: Control-Aware and Co-Design
Michael C. A. Nestor, Jiaxin Wang and Fei Teng
In distributed control schemes, the communication
topology that defines information sharing between
agents must be designed prior to online control execution. Previous data-driven works typically decouple the
topology and controller design problems. This paper investigates data-driven approaches to
topology design that guarantee the existence of a stabilizing controller, whilst trading off the level of required communication against control performance. We formulate a data-driven co-design scheme that directly co-optimizes the
topology and controller over the link costs and closed-loop control cost. When the
topology designer does not have access to the
agents' control objectives, we propose a control-aware approach that couples the two design problems within a sequential scheme. In both cases, the
topology is optimized within a mixed-integer semidefinite program (MISDP). Simulations including the IEEE 14-bus power system test case demonstrate the effectiveness of both co-design and control-aware schemes. Numerical examples show lower computation time for the control-aware case than for co-design.
Read the full abstract ▾
Systems and Control
A Frequency Domain Approach to Bounding Riccati Equation Perturbations
Rachel Newton, Laura Balzano and Peter Seiler
The discrete algebraic Riccati equation (DARE) is used to solve for the optimal feedback gain in linear-quadratic regulator (LQR) control of discrete-time systems. In this letter, we consider the effect of perturbations to the LQR data on the DARE solutions. We present a new frequency domain approach to derive an explicit bound on the difference in DARE solutions under such perturbations. Our bounds are similar to other results that have recently appeared in the literature with two distinctions. First, we use a frequency domain constant that measures the degree of closed-loop stability, in contrast to other results that use a time domain measure. Second, we start from weaker detectability assumptions on the LQR state cost
matrix.
Read the full abstract ▾
Systems and Control · Cross-listed from cs.RO
Onboard Vision and MPC Navigation for Underwater Robots: An Open BlueROV2 Platform for Multi-Robot Experiments & Docking
Victor Nan Fernandez-Ayala, Wiktor Kowalczyk, Cezary Banaszek et al. (+1 more)
Autonomous underwater
robots require robust perception, estimation and control to operate in confined environments. This paper presents an open-source BlueROV2 platform combining onboard
vision with nonlinear Model Predictive Control (NMPC) for autonomous navigation and docking. The platform integrates an NVIDIA Jetson Orin NX and an Intel RealSense D435i stereo camera in a modular pressure housing. Underwater-calibrated stereo depth and realtime object detection provide relative position measurements of nearby BlueROV2 vehicles in the camera and body frames. A quaternion-based
estimator fuses external pose and inertial measurements, while an NMPC controller based on a nonlinear six-degree-of-freedom model tracks planned navigation and docking trajectories. To support reproducible development, we also provide open-source physics-based PX4 SITL and Gazebo environments, multi-
robot simulation tools and a lowcost physical docking station. Experiments evaluate underwater perception, onboard computational performance, state estimation, trajectory tracking and autonomous docking.
Read the full abstract ▾
All authors Victor Nan Fernandez-Ayala, Wiktor Kowalczyk, Cezary Banaszek, Dimos V. Dimarogonas
Systems and Control · Cross-listed from cs.SE
An Empirical Study of Architectural Shift from Traditional to AI-Enabled Simulink Controllers
Hadiza Umar Yusuf and Khouloud Gaaloul
Effective AI adoption in cyber-physical systems (CPS) depends on
embedding design knowledge into engineering practice. Yet as AI-enabled components increasingly replace analytically derived control laws, this occurs without a systematic understanding of how controller architectures differ or remain similar across paradigms. We address this gap with an empirical study of traditional and AI-enabled Simulink controllers, guided by a literature-derived taxonomy of ten structural categories and nine functional roles. The study analyzes 62 real-world models spanning 8 controller types and 10 application domains, and surveys 13 practitioners, identifying three architectural tensions. First, subsystem organization dominates all controller structures regardless of paradigm, occupying 68-72% of controller footprint, while core control logic occupies minimal space. Second, AI-enabled controllers rely heavily on discrete dynamics and user-defined abstraction, categories largely absent from AI literature, exposing a gap between described and implemented architectures. Third, constraint enforcement blocks largely disappear from AI-enabled models despite practitioner expectations. This reveals a misalignment where safety mechanisms shift from explicit structure to implicit training-time artifacts, breaking traceability.
Read the full abstract ▾
Systems and Control
Condition-Based Maintenance of Degrading Assets underIntermittent Accessibility
Muhammet Ceyhan Sahin, Murat Yildirim and Onur A. Kilic
Many maintenance models implicitly assume that maintenance can be performed whenever intervention is warranted. In practice, however, environmental uncertainty, such as weather and sea conditions, can make maintenance opportunities intermittent and dynamically evolving. We consider a degrading asset for which preventive maintenance may be performed before failure, but intervention is possible only when the asset is accessible. Accessibility evolves stochastically over time and therefore affects not only whether maintenance can be performed now, but also the value of waiting for future opportunities. We formulate this setting as a finite-state Markov decision process under a long-run average cost criterion and investigate the structure of the optimal maintenance policy. Under monotone degradation and cost conditions, we show that an optimal policy retains a threshold form in asset condition, but unlike a single threshold, the optimal threshold varies with the accessibility state. Thus, intervention depends jointly on the asset condition and the expected
evolution of future maintenance opportunities. Through a numerical study motivated by offshore wind turbine maintenance, we examine how accessibility dynamics, degradation characteristics, and economic parameters shape the optimal thresholds and compare the optimal policy with constant-threshold and age-based maintenance policies. In a representative setting, adapting the condition threshold to accessibility reduces long-run average cost by 4.31% relative to an optimized condition-based maintenance policy. Relative to an optimized age-based policy, the optimal policy reduces long-run average cost by 33.95%. The results show that maintenance thresholds are not universal: they should adapt to the degradation and accessibility characteristics of the system, and failing to account for these conditions can lead to substantial cost increases.
Read the full abstract ▾
Systems and Control
Low-Rank and Lifted Semidefinite Programming for Mixed-Integer Polynomial Power Grid Optimization
Samuel Chevalier
Can a local solver return the guaranteed globally optimal solution to a nonconvex mixed-integer
polynomial power grid optimization problem? By mixing low-rank semidefinite programming (SDP) and moment-based lifting in the Lasserre hierarchy, this paper provides anecdotal evidence in the affirmative. Specifically, we iteratively tighten a moment-based relaxation of the AC Optimal Transmission Switching (AC-OTS) problem, including higher order moment-based constraints, rooted in the Lasserre hierarchy, until it is tight (i.e., primal feasible). We then re-solve this problem using a local, low-rank SDP solver (Knitro), plugging the associated dual variables into a bounding Lagrange dual function. By demonstrating primal and dual objective agreement and primal feasibility, we confirm the solution as the exact, global solution. From these initial results, we hypothesize that mixing (i) low-rank (i.e., highly scalable) SDP methods with (ii) moment-based relaxation tightening could be a scalable alternative to global branch-and-bound-based methods.
Read the full abstract ▾
Systems and Control · Cross-listed from math.OC
Unified Optimality Conditions for Stochastic Optimal Control in the Rough Path and Itô Frameworks
Thomas Lew
Stochastic differential equations (SDEs) can be studied via Itô calculus and rough path theory. For
stochastic optimal control, these two frameworks give distinct Pontryagin Maximum Principle (PMP) optimality conditions with forward-backward SDEs (FBSDEs) or rough differential equations. We show that the adjoint equations of the Itô and rough PMPs are connected via the conditional expectation $p_t^{\text{It\^{o}}}=\mathbb{E}[p_t^{\text{rough}} \mid \mathcal{F}_t]$, where $\mathcal{F}_t$ represents information available at time $t$. First, we derive a rough
stochastic PMP for problems with adapted controls that does not use FBSDEs. Its proof extends the rough
stochastic PMP over deterministic controls by considering
stochastic needle variations. Second, we derive a unified PMP connecting the Itô and rough PMPs, using Itô-Stratonovich conversion formulas and duality identities between the forward tangent and backward adjoint SDEs. As a first application, we rederive the adjoint matching method for fine-tuning generative models. As a second application, we propose an indirect shooting method for a class of feedback problems. Overall, these results give a new conditional bridge connecting two popular frameworks for
stochastic optimal control.
Read the full abstract ▾
Systems and Control
Timing Sensitivity in Actuated Traffic Signal Control: A simulation study on one urban network
Nitai Aharoni
A comparison between adaptive and fixed-time traffic signals can change when only their timing settings change. We examine this sensitivity in 240 simulation runs on one OpenStreetMap-derived urban network, using eight configurations, three arrival rates and ten paired demand seeds. The experiment varies the maximum green of presence-based actuation, retimes the fixed baseline, and includes a discharge-based phase-termination rule. The endpoint is finite-window time spent per offered trip, including entry waiting and penalties for removed or abandoned trips. At the highest tested load, actuation with a maximum green of twice the planned split has a mean endpoint 57.4 per cent above the original fixed plan. Reducing that multiplier to 1.25 puts it 70.7 per cent below the same plan. Against the best of three tested fixed plans, discharge-based termination reduces the mean endpoint by 14.8, 15.8 and 18.3 per cent across the three loads. Both timing sweeps favour their shortest tested setting, so neither identifies an optimum. This exploratory study uses synthetic demand, negligible start-up lost time and physical removal of persistently stationary vehicles. The results support testing timing sensitivity before attributing performance to a controller family; they do not establish a setting for real-world deployment.
Read the full abstract ▾
Systems and Control
Computing Scaled Relative Graphs of Discrete-Time LTI Systems: A Frequency-Domain Approach
Guoming Shi and Fredy Ruiz
The scaled relative graph (SRG) represents the joint gain and phase properties of an input–output operator in the complex plane. This paper characterizes the SRG closure of causal, stable, square discrete-time linear time-invariant (LTI) systems on one-sided square-summable sequences. The Beltrami–Klein image of this closure equals the convex hull of the numerical ranges of the transformed frequency-response
matrices. For real-coefficient single-input single-output systems, this yields the
hyperbolic convex hull of the discrete-time Nyquist locus. The proof addresses the Hardy-space restriction imposed by one-sided inputs, which excludes persistent sinusoids, by replacing them with explicitly constructed finite-duration sinusoids. This construction shows how points in frequency-wise SRGs arise as limits of operator SRG points generated by admissible inputs. The characterization enables model-based computation of the closure from a state-space realization using frequency-response evaluations, numerical ranges, and planar convex hulls, without solving linear
matrix inequalities.
Read the full abstract ▾
Systems and Control · Cross-listed from cs.AI
Stochastic World Models for Verifying Vision-Based Neural Feedback Systems
I. Samuel Akinwande, Mykel J. Kochenderfer and Clark Barrett
Verifying a
vision-based neural feedback system requires a model of the observations its controller acts upon. Such a model must capture the variation the sensor produces, while remaining tractable for closed-loop analysis. Generative
adversarial networks (GANs) have served as perception surrogates, but they are large, reproduce complex scenes poorly, and are hard to verify. We explore
stochastic world models as a richer class of perception surrogates. We train a world model with physically grounded latents, built from operations that standard verifiers bound. It reproduces held-out frames more faithfully than GAN surrogates with up to 130 times as many parameters. To verify these surrogates, we develop a procedure that combines falsification, adaptive refinement, symbolic, and backward analyses. On an emergency braking
benchmark with a GAN surrogate, our procedure resolves the entire state space, 38% of which the state-of-the-art verifier left unresolved. On the RGB version of the
benchmark, where no verification results have previously been reported, our procedure resolves over 80% of the state space with a world model surrogate.
Read the full abstract ▾
Systems and Control · Cross-listed from math.OC
Stochastic Lie-Bracket Approximations for Zeroth-Order Optimization on Manifolds
Mahmoud Abdelgalil, Miroslav Krstic and Jorge I. Poveda
We propose a continuous-time
algorithm for zeroth-order global optimization of smooth functions on compact
manifolds that synthesizes Riemannian gradient dynamics using only objective measurements, without evaluating gradients. The
algorithm is formulated as an ordinary differential equation driven by piecewise-smooth McShane-type approximations of the Wiener process, with a
geometric compensation term that depends only on the actuation frame and not on the objective function. Our main result establishes infinite-horizon practical probabilistic guarantees for fixed approximation parameters. To this end, we show that the
algorithm's trajectories approximate, in the mean-square sense, a Riemannian Langevin
diffusion whose unique invariant Gibbs measure concentrates on the set of global minimizers. We exploit this approximation to establish finite-horizon probabilistic convergence guarantees and directly analyze the
algorithm's long-time behavior to obtain the infinite-horizon result.
Read the full abstract ▾
Systems and Control · Cross-listed from math.OC
Non-Holonomic Gradient Play: Leafwise Nash Equilibria, Stability, and Deception
Mahmoud Abdelgalil, Miroslav Krstic and Jorge I. Poveda
We study generalized learning dynamics in multi-
agent systems whose joint state evolves on a
manifold and whose
agents act through state-dependent, potentially nonholonomic vector fields. Under a bundle-splitting condition, we show that these dynamics admit an intrinsic representation as projected Riemannian gradients, giving rise to a class of nonholonomic gradient play dynamics. We characterize the local stability of its
equilibria through an intrinsic linearization that explicitly captures the effects of the Riemannian connection and the nonholonomy of the actuation frame. The framework recovers classical gradient play on Euclidean spaces and
manifolds as special cases while accommodating nonholonomic actuation. We then develop a
geometric theory of deception under asymmetric information, whereby an
agent exploits privileged knowledge of other
agents' learning rules to manipulate the emerging
equilibrium. We show that deception effectively tilts the Riemannian geometry perceived by the oblivious
agents, distorting their projected gradient directions. We establish persistence of exponentially stable
equilibria over an open set of deception parameters and derive an explicit first-order characterization of the resulting
equilibrium displacement. Analytical and numerical examples illustrate the framework and its implications for
equilibrium manipulation in games.
Read the full abstract ▾