Papers

This paper studies $N$-player stochastic linear-quadratic (LQ) differential games from the perspective of $\alpha$-potential games. We first consider a closed-loop LQ game with multiplicative noise, where both the drift and the diffusion coefficients depend linearly on the state and the full control vector. For this model, we derive probabilistic and partial differential equation (PDE) representations for the first- and second-order linear derivatives of the players' cost function and prove the equivalence between them. We then develop an open-loop stochastic LQ \(\alpha\)-potential game framework. Using the linear derivative construction, we build an \(\alpha\)-potential function and derive an explicit upper bound for the approximation parameter \(\alpha\) in terms of the model coefficients and the admissible control radius. Moreover, the minimization of the \(\alpha\)-potential function is reduced to a finite-dimensional stochastic control problem by augmenting the state with the variational process, which yields an open-loop \(\alpha\)-Nash equilibrium. As an application, we revisit a network LQ game considered in \cite{GuoLiZhang2025} and show that the feedback representation obtained from our approach coincides with the feedback in the existing conditional McKean–Vlasov approach, while our characterization follows directly from a standard finite-dimensional LQ control problem.

We study internal controllability of Schrödinger equations on tori with controls acting only through their real parts. This real-part constraint leads to a real-linear control problem for which full null controllability may fail. We characterize the maximal null-controllable subspace and show that the uncontrollable directions are precisely given by the purely imaginary stationary modes. These results provide an explicit description of controllable and uncontrollable directions for Schrödinger equations with real-part controls. The characterization is obtained through observability inequalities with sharp stationary correction terms. For cylindrical open control regions, we prove such an estimate by a real-part compactness–uniqueness argument. For admissible measurable product control regions in the shifted free case, we establish the corresponding estimate for rough spacetime control sets. The proof combines complex-valued observability inputs with a double-spectrum analysis arising from the coupling of the forward and backward Schrödinger evolutions in the real-part observation.

We consider minimizing a smooth, strongly convex function over a convex set. Projected gradient descent is known to converge linearly in this setting, but each iteration requires a projection onto the feasible set, which may be computationally expensive. We show that when the feasible set is smooth, projection can be replaced by one gradient computation and a single supporting-tangent computation per iteration, while preserving linear convergence. Moreover, the required tangent can be approximated to sufficient accuracy using $\widetilde O(d)$ membership-oracle queries, where $d$ is the ambient dimension. Previously, projection-free linear convergence was known only for polyhedral sets or for sets that are both smooth and strongly convex.

Recent proliferation of data-optimization integration has led to a range of methods that aim to improve the statistical performance of data-driven optimization decisions. However, while many of these methods are motivated intuitively from a robustness or regularization perspective, their resulting statistical benefits are often unclear and, even if available, are established on a case-by-case basis. We provide a systematic dissection of data-driven optimization formulations using the view of "directionally perturbed" empirical optimization (EO). Specifically, this umbrella of formulations, which we call "EO+", covers many existing data-driven optimization methods, including regularization, distributionally robust optimization, transfer learning, and analogous methods for contextual optimization. On the one hand, we argue that without additional, correctly specified, side information, any EO+ method can result in at most second-order improvements. This provides a negative conclusion, namely “no free lunch is possible", on the statistical power of EO+. On the other hand, we show that when leveraging side information that is geometrically effective, achieving first-order improvements is possible by choosing hyperparameters that are significantly larger than what is typically suggested in the literature. Moreover, we construct a principled methodology based on excess risk estimation, via either system knowledge or bootstrap resampling, to maximize the first-order gain. We demonstrate how this gain connects to the control-variate principle, a variance reduction technique in the Monte Carlo simulation literature, which helps explain why geometrically effective side information is necessary.

We develop a concise error analysis for a linear-quadratic elliptic distributed optimal control problem with point tracking and pointwise control constraints, where a standard finite element discretization and a variational discretization are treated in a unified framework. The analysis is self-contained modulo standard techniques in finite element analysis and classical results for elliptic partial differential equations.

Hospitals and diagnostic laboratories rely on couriers to collect blood samples from geographically dispersed collection centres and deliver them for analysis before their short viability windows expire; late deliveries force costly re-collection and can delay diagnosis. We study the real-time dispatching of such a courier fleet, in which sample requests arrive stochastically at the centres throughout the day and a central dispatcher must repeatedly decide which vehicles to send, which centres each should visit, and whether to collect urgent samples immediately or consolidate them into later trips, subject to hard delivery deadlines and vehicle capacity limits. Unlike static planning models, which fix routes before demand is known, and reactive heuristics, which respond only to the current backlog, our approach anticipates future arrivals when weighing immediate collection against consolidation. We formulate the problem as a Markov decision process and develop a neural approximate dynamic programming framework for centralized dispatch. The method introduces a dual value function decomposition that separately represents vehicle states and collection-centre states through neural networks trained on post-decision states. These learned estimates are integrated through a matching formulation that selects dispatch actions while balancing supply availability, demand urgency, and downstream opportunity cost. Computational experiments on a realistic Greater Toronto Area network compare the proposed policy with myopic baselines and ablation variants. Results show that the proposed dual value function policy raises the share of sample volume delivered on time by 1 to 9 percentage points over myopic baselines, with the largest gains under tight fleet, capacity, deadline, and routing constraints; these on-time gains, in turn, reduce reliance on costly external couriers.

Stochastic gradient descent (SGD) optimization schemes are the methods of choice for the optimization of deep neural networks (DNNs) in artificial intelligence (AI) systems. Often not the standard SGD method is used but instead suitable accelerated, adaptive, and/or normalized variants of standard SGD such as Adam, AdamW, and MUON are employed to train large scale AI systems in practically relevant settings. The acceleration (higher order convergence speed) in all these popular optimizers relies on the momentum SGD optimizer. In this work we provide a rigorous error analysis for the momentum SGD optimizer. In particular, we establish convergence rates for the momentum optimizer in terms of the size of the learning rate (step size), the size of the mini-batch, and the size of the one-point convexity constant.

Evacuation planning for disaster preparedness requires making critical decisions under uncertainty before the number and spatial distribution of evacuees are known, including shelter location, evacuation route assignment, and relief supply prepositioning. Because these decisions are highly interdependent, planners must balance the competing objectives of maximizing relief demand coverage and minimizing evacuation time. We propose, to our knowledge, the first adaptive robust evacuation planning model to jointly optimize shelter locations, evacuation route assignments, relief supply prepositioning, and post-disaster relief item distribution. The model minimizes the worst-case weighted sum of unmet demand for relief items across shelters and the congestion-dependent evacuation time. We characterize theoretical complexity drivers of the resulting problem with mixed-integer recourse and develop a partition-and-bound algorithm that maintains tractability by selectively partitioning only the most critical subpartition of the uncertainty set while producing strong upper and lower bounds. To quantify the value of centralized route planning, we also formulate a user route choice alternative in which evacuees choose among acceptable routes. Computational experiments quantify the value of centralized route planning, which reduces worst-case unmet demand and evacuation time by up to 90.6% and 79.3%, respectively, relative to decentralized user route choice. Adaptive post-disaster supply redistribution further improves relief demand coverage. Coordination between evacuation routing and relief distribution creates substantial operational value under uncertainty. Centralized route planning primarily mitigates congestion by coordinating evacuee flows across shelters, whereas adaptive redistribution primarily improves relief demand coverage when relief supplies are scarce or inflexibly prepositioned.

As a popular optimization scheme, distributionally robust optimization (DRO) protects decisions against ambiguity in probability distributions. For (single-stage) DRO, prevailing dual reformulations can become difficult when model or ambiguity-set structures are complex. We study DRO from a primal perspective, working directly with distributions in ambiguity sets on closed, potentially unbounded sample spaces. This perspective leads to an algorithmic framework, referred to as BiCS, that constructs and leverages distribution cuts to achieve strong performance. We show that BiCS is applicable to standard DRO, almost-sure DRO, DRO with various chance constraints, and DRO with ambiguity sets strengthened by local information. Numerical experiments with moment and Wasserstein ambiguity sets show that this framework demonstrates superior performance, including solving cases where the examined compact reformulations are unavailable or computationally difficult. The local-information study also makes changes in worst-case distributions directly visible.

Many economic decision problems, lifecycle consumption-saving and dynamic portfolio choice, are finite-horizon stochastic control problems with continuous states and actions. When the state is low-dimensional these problems are solved by dynamic programming on a grid. The grid cost grows exponentially in the state dimension, known as the curse of dimensionality, which motivates replacing the value-function grid with a neural policy optimized directly through simulation. Such policies are usually studied in the high-dimensional settings that motivate them, precisely where no reference solution exists. So the contribution of any single architectural or training choice cannot be isolated and diagnosed. We therefore take a step back and treat both the architecture and the solution method as the objects of study. To this end, we consider a lifecycle problem with a sufficiently low-dimensional normalized state space to admit an accurate dynamic programming solution, which is used for evaluation. We compare four architectures. The simplest consists of a single time-conditioned network. We then consider two networks concatenated across the regime switch, followed by one network per date trained backward against frozen downstream policies. Finally, we evaluate a constrained variant of the per-date architecture. Decoupling the policy across time gives each date a short, well-posed objective, which we pair with direction-dominant optimization that normalizes away gradient magnitude. Architectures that lead to similar realized utility objective can nevertheless differ in whether they respect the underlying problem's economics. We therefore evaluate each design jointly based on welfare, a solution-free Bellman residual, shape restrictions, and the resulting policy functions.

We study elective surgery scheduling under uncertain procedure durations. Schedules based on mean durations or fixed buffering rules may appear efficient ex ante but become fragile in execution, as early overruns propagate through the day and expose later surgeries to accumulated delay. We propose a robust chance-constrained framework that separates uncertainty quantification from schedule optimization. A buffer engine converts distributional information into reliability-dependent buffered durations, while the scheduling model jointly selects assignments, sequences, start times, and surgery-level reliability levels from a discrete menu. Reliability therefore becomes an endogenous scheduling decision rather than a fixed service-level parameter. The framework accommodates average-reliability, worst-day, and hard-target risk postures. Comparisons with common-reliability and uniform proportional-buffer benchmarks show that the menu derives its value from exploiting surgery-level heterogeneity, allocating protection where it has the greatest operational value while avoiding unnecessary conservatism. In a rolling-origin case study at HLA Moncloa Hospital in Madrid, covering 10 instances with 45 to 227 surgeries, the approach reduces delays exceeding 90 minutes by 97%, lowers the 95th-percentile delay by approximately 620 minutes, and reduces total overtime by 42% relative to a deterministic mean-based baseline. By allocating buffers according to uncertainty and operational exposure, the framework translates heterogeneous duration data and risk preferences into schedules that are more reliable in execution and less conservatively buffered than one-size-fits-all rules.

Trajectory inference is a fundamental problem in many scientific domains: given a collection of unpaired snapshots of observations at discrete time points, the goal is to generate smooth trajectories that best resemble and interpolate the data. Existing algorithms exhibit computational challenges: they either rely on preprocessing subroutines to enforce smoothness or on simulation-based training objectives, both of which can be expensive. In order to overcome these limitations, we propose a new algorithm called Acceleration Matching (AM). Our approach consists of lifting the original interpolation problem to phase space and then regressing onto an explicit conditional acceleration field that induces random, smooth trajectories that agree with the prescribed marginals. Importantly, our resulting training algorithm only requires positional data, avoids trajectory simulation during training, and is devoid of expensive preprocessing. We provide ample numerical evidence suggesting that AM is competitive with or superior to existing algorithms on several benchmark problems from the existing literature.

We study convex-concave minimax optimization problems with bilinear coupling of the form $\min_{x\in \mathcal X}\max_{y\in \mathcal Y} \; f(x)+\langle y,\mathbf{B}x\rangle-g(y),$ where the functions $f$ and $g$ have Hölder continuous (sub)gradients. This setting covers a broad range of regimes, from nonsmooth problems with bounded subgradient variation to smooth problems with Lipschitz continuous gradients; for a smooth component used in the coupling-induced regularizer, its Lipschitz-gradient constant is assumed to hold in the ambient space. We propose a sliding method that exploits the composite structure of the problem by querying the oracles associated with $f$, $g$, and the bilinear coupling operator at prescribed frequencies determined by their individual properties. The method is based on a recursive sliding scheme for monotone variational inequalities. We establish convergence guarantees under Hölder continuity and show how the resulting complexity bounds depend explicitly on the Hölder exponents, Hölder constants, strong convexity parameters, and spectral properties of the coupling matrix. Our analysis covers nonstrongly convex and partially strongly convex regimes. For stochastic function oracles, we prove componentwise Hölder guarantees with separate variance costs: a uniform expected-gap bound on bounded domains and, when both functions are strongly convex on their feasible sets, a restarted Lyapunov guarantee without an anchor noise floor. Numerical experiments reproduce the predicted Hölder exponents and confirm that the number of gradient evaluations required for each function separates according to its own smoothness level rather than the worse of the two. A tomographic benchmark shows runtime gains when gradient evaluations are more expensive than the additional matrix-vector products.

In the standard Downsian model, two office-seeking parties converge to the median voter. However alienated voters may abstain and turn out only for a party within their tolerance radius. For single-peaked voter distributions, convergence survives but relocates to a central voter, the median of the electorate that participates at the convergent platform. However single-peakedness of the voter distribution is an empirically contested assumption. So we first characterize pure-strategy equilibrium for any continuous voter distribution. For general distributions, pure-strategy equilibrium can fail to exist or be non-unique, and existence of equilibrium need not persist as the tolerance radius of the voters increases. In order to resolve these issues, we propose a fundamental object: centripetal structure for which there is a single anchor platform toward which competition always pulls. We show this structure produces convergence at equilibrium under alienation based abstention. Our main result concerns the emergence and persistence of this new structure as the tolerance radius increases. Even though equilibria for office-seeking parties themselves can vanish and reappear as the radius grows, once centripetal structure emerges, it persists as long as the midpoint voter is not alienated. Moreover, the centripetal structure always emerges, and this structure classifies equilibrium completely when parties are policy motivated.

We study the iteratively regularized gradient method applied to the ill-posed problem of minimizing a general smooth functional on a Hilbert space. Under a source condition on the solution, we establish error estimates for this method when the objective functional is known exactly or only approximately. For these estimates we provide two proofs based on substantially different ideas and compare the assumptions and conclusions of the corresponding theorems. In addition, we present an improved proof of an error estimate for the Tikhonov method applied to ill-posed optimization problems with an exactly known functional. We also prove a theorem on the existence of a global minimizer of the Tikhonov functional without assuming weak lower semicontinuity of the original functional.

In this work, we propose an adaptive stepsize rule with guaranteed convergence for Distributed Gradient Tracking applied to scalar quadratic problems with heterogeneous curvatures. Most distributed gradient-based algorithms require a suitable stepsize selection. Available theoretical bounds are often overly conservative, while practical implementations typically rely on empirically tuned heuristics. Online adaptive strategies have only recently emerged for general distributed convex optimization, but their properties and performance remain only partially understood. To gain analytical insight, we focus on the informative setting of scalar quadratic costs, which allows us to explicitly capture the interplay between network topology and curvature heterogeneity. We derive a convergence bound parameterized only by the essential spectral radius of the consensus matrix and the heterogeneity of the local cost curvatures, both computable online without any prior knowledge of the optimization problem. Optimizing this bound yields a computationally tractable surrogate for the convergence rate and the optimal constant stepsize. The resulting stepsize admits an analytical interpretation, guarantees convergence for arbitrary network topologies and curvature heterogeneity, and is provably tight for complete graphs and homogeneous curvatures. Finally, extensive numerical simulations demonstrate that the proposed distributed adaptive strategy significantly outperforms existing offline and online stepsize selection rules in the considered setting.

In this paper, we address the problem of achieving fast convergence in gradient descent for quadratic functions without relying on a priori knowledge of global function parameters. Inspired by adaptive stepsize algorithms for smooth convex functions, we propose a computationally lightweight strategy based on running estimates of minimal and maximal local curvatures. We prove that our proposed algorithm converges to the optimal constant stepsize which achieves the fastest convergence. Simulations show that the convergence rate achieved by our proposed algorithm is comparable or superior to recent adaptive approaches both in the quadratic case under consideration and in a preliminary test on logistic classification.

The paper proposes a novel proximal difference-of-convex (DC) algorithmic framework to solve general non-convex, non-smooth optimization problems. By combining Barzilai-Borwein (BB) step sizes with nonmonotone line search strategies, our approach effectively overcomes the conservative step sizes and stability issues inherent in standard proximal DC algorithms. Furthermore, we develop extrapolation mechanisms to accelerate convergence while ensuring global stability. The global convergence of the proposed algorithms is rigorously established under the Kurdyka-Łojasiewicz property. Numerical experiments on the SCAD-regularized least squares problem and graphic Ginzburg-Landau image segmentation models demonstrate that the proposed methods achieve highly competitive efficiency and accuracy compared to existing DC algorithms.

We introduce a McKean–Vlasov semigroup in the presence of common noise and study its first-order differential structure. Given a terminal functional $G$, the semigroup is defined by propagating the initial law through a reference conditional McKean–Vlasov flow. Using a finite-particle approximation and a diagonal/off-diagonal decomposition of the tangent processes, we identify the Lions derivative of the propagated functional and derive a backward representation along arbitrary admissible conditional law flows. This formula involves only first-order derivatives in the measure variable and yields both a Feynman–Kac representation for linear equations on the Wasserstein space and, in the driftless case, a martingale representation problem whose unique solution is given by the McKean–Vlasov heat semigroup and its Lions derivative. We also discuss mixed finite-dimensional/Wasserstein representations and discrete path-dependent extensions.

We propose a framework for constructing approximate Nash equilibria in mean-field games (MFG) with common noise based on a two-time-scale structure. In our model, the common noise is carried by a fast variable evolving under ergodic dynamics, while the slow variable is either optimally controlled or stopped. The fast variable enters the dynamics of the slow one through the drift coefficient. The key idea is to construct an approximation to the solution of the full MFG with common noise using an “effective” MFG without common noise, where the coefficients are averaged with respect to the stationary measure of the fast-scale process. We construct an explicit $\varepsilon$-MFG equilibrium for the full MFG from the equilibrium for the effective MFG with randomized control and stopping. To this end, we obtain new results on existence of MFG equilibria with randomized stopping. We rely on strong convergence results for two-scale diffusions under structural assumptions on the MFG, and show that the time-scale separation parameter controls the error in the Nash equilibrium condition.

In this work, we present a novel dynamic proximal point algorithm for unconstrained optimization. The method generates a sequence of proximal subproblems, where the quadratic regularization term is weighted by a diagonal matrix that is updated adaptively at each iteration. Each subproblem is solved using an inner Newton's method combined with a line search, which provides a global convergence mechanism for the nonlinear solver. At the outer level, the algorithm updates the reference point and adjusts the regularization parameter based on the performance of the inner Newton solver. We derive the reduced linear system used to compute the Newton step, define the corresponding merit function, and discuss practical approaches for constructing the diagonal scaling matrix from derivative information. The paper also provides implementation-oriented pseudocode and stopping criteria that are consistent with the proposed method.

We study cutting planes for nonconvex quadratically constrained quadratic programs (QCQPs) through a project-then-lift approach inspired by mixed-integer rounding (MIR) inequalities. Given two base valid inequalities for the extended QCQP formulation, we project the associated two-row relaxation into a two-dimensional set and analyze the joint range of quadratic functions in two base inequalities. For the nonconvex joint range, we give a closed-form convex hull description of the projected set; for the convex joint range, we give its semidefinite representation. This yields a new family of joint-range inequalities, which can be lifted back to the extended QCQP formulation. MIR inequalities can handle “mixed” terms: continuous variables or fractional linear combinations of integer variables. Similarly, we propose more flexible secant mixed-joint-range inequalities, which better expose and exploit the nonconvex joint range. The proposed approach preserves sparsity, since the support of each lifted inequality is controlled by that of the base inequalities. In preliminary geometric experiments, the joint-range inequalities yield substantial area reduction of the projected relaxation constructed via reformulation-linearization-technique.

We study the oracle complexity of finding $\epsilon$-stationary points of $\rho$-weakly convex and $G$-Lipschitz functions, where stationarity is measured by the gradient of the Moreau envelope. We consider a first-order oracle that returns both the function value and the full subdifferential at every query point. We prove that every deterministic first-order algorithm requires $ \Omega({\rho G^2\Delta}/{\epsilon^4})$ oracle queries whenever $\Delta \leq {G^2}/{\rho}$, where $f(\bz)-\inf f \leq \Delta$. This lower bound matches the best known deterministic and stochastic first-order upper bounds, up to universal constants, and establishes the optimal deterministic oracle complexity. The result reveals a fundamental complexity separation between smooth nonconvex and nonsmooth weakly convex optimization. While smooth nonconvex minimization admits a $\Theta(\epsilon^{-2})$ oracle complexity, nonsmooth weakly convex optimization incurs an intrinsic additional $\epsilon^{-2}$ factor arising from nonsmooth geometry rather than stochasticity.

Augmented Lagrangian (AL) methods are a classical framework for constrained optimization, but for directly verifiable approximate KKT points, known first-order complexity bounds for standard inexact AL methods are suboptimal, while the best known proximal augmented Lagrangian (PAL) bounds retain an additional logarithmic factor. We consider linearly constrained convex composite problems with a smooth convex term and a possibly nonsmooth closed proper convex term with compact domain. We develop three inexact AL schemes that preserve the standard AL subproblem structure and attain the optimal primal-dual complexity $\mathcal O(\epsilon^{-1})$ in the convex setting, improving prior AL bounds of $\mathcal O(\epsilon^{-4/3})$, $\mathcal O(\epsilon^{-7/4})$, and $\mathcal O(\epsilon^{-2})$, and removing the logarithmic factor from PAL guarantees. Two variants are parameter-free, and all three admit nonergodic guarantees, including a stronger last-iterate guarantee for one variant. These results show that proximal regularization, ergodic averaging, and prior knowledge of problem-dependent constants are not intrinsic requirements for attaining optimal verifiable primal-dual complexity within the standard AL framework. A key ingredient is a parameter-free accelerated method that computes verifiable stationarity certificates for the standard, unregularized AL subproblems with optimal complexity. In the strongly convex setting, our methods attain near-optimal complexity $\mathcal O(\epsilon^{-1/2}\log(\epsilon^{-1}))$, with two parameter-free variants. Numerical experiments on six problem classes, including elastic-net least-squares regression, group-sparse Huberized support vector machines, and a quantum semidefinite program (SDP), demonstrate substantial computational advantages over a representative PAL method, with speedups frequently ranging from $5$ to $50$ times.

Distributionally robust optimization (DRO) with optimal transport ambiguity sets is traditionally solved by reformulating the minimax problem into a single-level convex program. While theoretically tractable, these reformulations introduce numerous auxiliary variables and demanding conic constraints that scale poorly in practice. In this paper, we address this challenge by reducing the inner worst-case expectation problem exactly to a scalar budget allocation task. This structural insight yields an efficient algorithm that bypasses large lifted reformulations, alongside a fast post-processing scheme to recover an optimal worst-case distribution supported on at most $N+1$ points, where $N$ denotes the sample size. We embed this procedure within an oracle-based distributional best-response framework to directly compute an approximate primal-dual solution to the overall DRO problem. Furthermore, we extend our analysis to the dual DRO formulation, proving the existence of a least-favorable distribution supported on at most $\min\{N+n+1, KN\}$ atoms, where $n$ and $K$ denote the decision dimension and number of loss components, respectively, and provide an efficient convex programming reduction to extract it from the solution of the primal DRO. Numerical experiments demonstrate that the proposed approach significantly outperforms state-of-the-art reformulation-based solvers.

This paper presents purely convex programs for passively safe impulsive rendezvous and proximity operations in cislunar orbits. Approach, arrival, and abort maneuvers are all designed and validated in the context of maneuver execution error and navigation uncertainty, and formulated for efficient onboard execution in the autonomous scenario. The outlined methods form the baseline onboard guidance routines for NASA's CAPSTONE 02 mission planned to demonstrate autonomous rendezvous and proximity operations capabilities in the southern 9:2 synodic near rectilinear halo orbit. High fidelity closed loop Monte Carlo simulations using the planned relative navigation sensor suite and measurement cadence verify the intended maneuver design performance.

Unit excitation (UE) is a common assumption in stochastic saddle avoidance: the stochastic error must have a uniformly positive component along every direction, in expectation. This condition gives a direct way to rule out convergence to strict saddles, but it also oversimplifies the actual noise structure, and does not match many stochastic optimization regimes. In overparameterized or interpolation models, the noise may vanish near stationarity. In finite-sum problems, the stochastic gradient noise may lie in a low-dimensional, data-dependent subspace. In these (common) scenarios, UE is naturally not satisfied. In this paper, we prove an abstract almost sure avoidance theorem for stochastic recursions without UE. The theorem replaces UE-type requirements by verifiable pathwise conditions. In applications, these conditions follow, e.g., from local smoothness and finite-moment assumptions under standard i.i.d. sampling, or from the finite-sum structure under without-replacement sampling. Since the stochastically sampled maps generally do not share a fixed point, the celebrated center-stable manifold argument used in deterministic analyses is not directly applicable. Instead, we use a path-dependent change of variables together with a pathwise Lyapunov–Perron-based proof strategy. As applications, we obtain strict saddle avoidance for stochastic mirror descent (including SGD) and for random reshuffling. For nonsmooth composite objectives, we prove avoidance results for a proximal-type stochastic gradient method. Combining these insights with suitable iterate convergence guarantees, this allows establishing convergence to local minimizers of the original objective function.

Forsythe conjectured in 1968 that the normalized residual directions of restarted conjugate gradients approach a two-cycle. We prove the conjecture for restart length two: unless the method terminates, its even and odd normalized residuals converge separately.

This paper considers boundary stabilization problems for a first-order hyperbolic partial integro-differential equation (PIDE) subject to concurrent input and state delays. The coexistence of these two types of delays complicates control design, especially under the case of large input delay that requires to predict more state information. A backstepping-based boundary controller is developed to achieve stabilization and delay compensation. The design relies on two affine Volterra transformations involving both Fredholm- and Volterra-type integral terms, which results in a four-PIDE kernel equations. To establish their well-posedness, the kernel domain is partitioned along characteristic lines into a finite number of triangular subregions, and the kernel solution is constructed successively in the preceding subregion to the next one. The finite-time stability of the resulting closed-loop system is established. Numerical simulations are provided to demonstrate the effectiveness of the proposed controller.

We study policy optimization for gain-scheduled linear quadratic regulation, where one schedule of gains, interpolated through fixed weighting functions, is optimized against a family of plants. The resulting cost can develop spurious local minima, and existing convergence certificates are either local or severely conservative. We establish an exact identity: when the gradient of the cost is evaluated with the minimizer's closed-loop covariances, the scheduled cost is star-convex about the minimizer. The identity holds on the entire feasible set, for any parametrization of the schedule. Convergence is governed by a single dimensionless ratio. Wherever the ratio satisfies a threshold condition, gradient descent converges linearly to the optimum on entire sublevel regions at an explicit rate; at every spurious stationary point the condition necessarily fails. Experiments that maximize the ratio directly show the threshold to be an active boundary of the landscape. This extended version contains the complete proofs and additional numerical studies omitted from the letter for space.