Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Mathematics Optimization and Control

math.OC Watch Optimization and Control (sign in to use)

Monday 12 October

Optimization and Control

Vanishing Events and Regularity-Conditioned Reachability in Linear Stieltjes Systems

We study linear Stieltjes control systems in which actuation is concentrated at finitely many temporal events, and we ask what happens when one of those events progressively loses its atomic weight in the associated Stieltjes measure. For constant coefficients, ordinary finite-atomic reachability is governed by a truncated Krylov space and is therefore insensitive to the positive values of the event weights. A different picture emerges once trajectory regularity is imposed. In the supercritical super-$g$-Hölder regime, we minimize the largest event-normalized state jump and characterize exactly the terminal directions for which this cost remains uniformly bounded as a prescribed event weight tends to zero. The characterization combines what can be reached before the vulnerable event, a drift-cancellability condition at that event, and the directions that remain available afterwards. Ordinary reachability of the deleted-event system can therefore be strictly larger than bounded-cost reachability in the singular limit. We also separate two mechanisms of divergence: loss of ordinary reachability after deletion and failure of the intermediate compatibility condition. For compatible targets, the minimum cost converges to a constrained deleted-event problem. A dimensionless position–velocity benchmark motivated by pulsed proximity maneuvers shows the distinction explicitly.
Read the full abstract

Optimization and Control

Mitigating Performance Variability in Mixed-Integer Programming Solvers via LP State Steering

Modern mixed-integer programming (MIP) solvers repeatedly solve linear programming (LP) relaxations, but the LP state transferred from the simplex algorithm to the branch-and-bound tree is typically treated as an artifact of perturbation, presolve history, and numerical trajectory. This is harmless in pure linear programming, where alternative optimal bases certify the same bound. In MIP, however, those bases can expose distinct fractional structures, active rows, reduced costs, branching candidates, and reoptimization states. The inherited LP state is therefore an initial condition for the integer search. We exploit this initial condition to mitigate computational performance variability. The proposed framework generates alternative LP states on or near degenerate optimal faces, evaluates them within an auxiliary solver environment, and either selects a single state or executes a portfolio of candidate states in parallel. A handcrafted synthetic instance isolates the relationship between the initial LP state, the fractional profile, the branching trajectory, and the resulting node-count distribution: root-state steering reduces the weighted variability score~(WVS) by 67% over 1,000 paired seeds and produces no node-count regressions relative to default SCIP. On a separate 130-instance MIPLIB 2017 validation set with up to 50 paired seeds per instance, the top-five policy improves solve completion from 3,792 to 3,894 runs, reduces the shifted geometric mean of the recorded node count from 2,574.7 to 1,198.0, and lowers the median WVS by 9% on instances reliably solved by the baseline. These results demonstrate that the branch-and-bound phase inherits a specific LP state rather than merely an LP bound, and that the active management of this state systematically mitigates computational performance variability.
Read the full abstract

Optimization and Control

Failure Modes of Interior-Point SDP Solvers on Moment–SOS Hierarchies

Standard semidefinite programming (SDP) solvers working in double precision can become unreliable on difficult problems, even when their output appears plausible. We study this issue for SDP problems arising from high-order moment-SOS hierarchies. Three polynomial optimization problems are analysed by comparing standard double-precision computations with reference solutions obtained using the extended-precision solver \mbox{Loraine\,64x4}. We observe two distinct patterns of numerical failure. In false convergence, the computed bounds form a smooth sequence but approach the limit at the wrong rate. In stagnation, the sequence develops a plateau or oscillations and no longer reflects the convergence of the underlying hierarchy. We introduce an empirical effective condition number based on the positive spectral components of the primal and dual solution matrices. It serves as a warning indicator rather than an error bound, and its growth accompanies numerical deterioration in our examples. We also examine the imbalance between primal and dual spectral conditioning and a Cholesky inverse residuum as potential diagnostic indicators. The results show that solver termination messages and apparently regular convergence curves may be insufficient to establish the reliability of high-order moment-SOS bounds.
Read the full abstract

Optimization and Control

Last-Iterate Convergence of Frank–Wolfe for Nonlinear Variational Inequalities on Polytopes

We consider the Frank–Wolfe algorithm for solving variational inequalities over compact polytopes under a monotone $C^1$ operator $\Phi$. The linear minimization oracle returns vertices, and we assume a coercivity condition on the Jacobian of $\Phi$ at a solution. We prove last-iterate convergence to the unique solution at rate $O(k^{-\min\{\alpha,1\}})$ for step sizes $\gamma_k=\alpha/(k+\beta)$ with $\beta\geq\alpha>0$. The bound holds uniformly over starting points and vertex tie-breaking. In particular, both $1/(k+1)$ and $2/(k+2)$ give $O(k^{-1})$ distance. A matching lower bound shows that this distance guarantee is worst-case order-optimal among prescribed admissible step-size schedules. We also obtain rates $O(k^{-\min\{\alpha,1\}})$ for the Frank–Wolfe gap and $O(k^{-\min\{\alpha,2\}})$ for the dual gap. These yield $O(\varepsilon^{-1})$ and $O(\varepsilon^{-1/2})$ calls to each oracle for sustained gap accuracy $\varepsilon$ when $\alpha\geq1$ and $\alpha\geq2$, respectively, with instance-dependent constants. We also treat more general prescribed step-size schedules.
Read the full abstract

Optimization and Control

A Composite Representation Theorem for Solution Mappings in Regularized Composite Optimization with Applications to Hausdorff Lipschitz Stability

We study the geometry and stability of solution mappings in regularized composite optimization, allowing multiple minimizers. We establish a composite representation that separates optimal linear images from their corresponding primal solutions through an image mapping and a generalized basis pursuit mapping. Under a fiber-attainment condition, the image mapping admits a reduced optimization formulation; neither representation requires convexity. For convex regularizers, suitable qualification conditions yield a Fenchel-conjugate representation of the reduced value function and a geometric characterization of the generalized basis pursuit mapping through affine fibers and a fixed lifting set. Using these representations, we establish sufficient conditions for local Hausdorff Lipschitz continuity of possibly set-valued solution mappings. We further show that the primal-space sufficient conditions imply the corresponding image-space conditions, with an additional injectivity requirement arising in the former. The results cover $\ell_1$-type and structured piecewise linear-quadratic regularizers under suitable assumptions. For classical $\ell_1$ regularization, our approach recovers an existing stability result without a finite sign-pattern decomposition. Numerical examples illustrate the recovery of complete solution sets, including a case with a nonpolyhedral graph of the generalized basis pursuit mapping.
Read the full abstract

Optimization and Control

A Feasible Direction Method with Active-Set Identification for Derivative-Free Optimization

We propose a model-based trust region method for derivative-free optimization with simulation-based inequality constraints. We are motivated by settings of simulation optimization in which both function evaluations at infeasible points may be unreliable and constraint evaluations are expensive. Our method maintains feasible incumbents and computes steps from a subproblem suitable for a feasible-direction method in order to encourage generation of feasible trial points. To reduce the computational cost of modeling the constraints, we describe an active-set estimation scheme that permits modeling with only a subset of constraints in each subproblem, as opposed to maintaining models of all constraints. We show that every limit point of the incumbents generated by our method is a Fritz-John point, and is additionally a KKT point under the Mangasarian-Fromovitz constraint qualification. Under standard nondegeneracy conditions, the active-set estimate identifies, within finite time, all constraints active at the limit point of the incumbents. Experiments on CUTEst problems show that our method generates feasible trial points far more often than a comparable SQP-based approach and solves more problems within a given budget with fewer function evaluations.
Read the full abstract

Optimization and Control

Matching Randomized First-Order Bounds for Higher-Order Smooth Nonconvex Optimization

Does randomization improve the worst-case first-order complexity of nonconvex optimization when the objective is very smooth? We prove that, for every fixed finite integer $p\ge3$, finding an $\eps$-stationary point requires $\Omega_p(\eps^{-5/3})$ exact value-and-gradient queries, even for $C^\infty$ objectives with initial gap at most one and simultaneous global Lipschitz budgets of one for derivatives of orders $1,\ldots,p$. Together with the known deterministic upper bound, this determines the accuracy exponent up to a single logarithmic factor. The lower bound holds for unrestricted adaptive randomized algorithms, unbounded query locations, and outputs that need not have been queried. One algorithm-independent hard distribution suffices in dimension $O_p(\eps^{-47/3})$. The construction combines a block-chain potential with an exact quartic core, repeated quadratic spectra, and two static Gaussian tensor perturbations. A scheduled Haar argument controls the entire adaptive query span. An innovation-only Gaussian comparison then bounds exact transcript information without assuming finite precision or a minimum query separation. This establishes the $5/3$ randomized accuracy barrier under every prescribed finite smoothness order, extending the previously matched third-order case.
Read the full abstract

All authors Haihan Zhang, Wendao Wu, Fangqing Zhang, Chenheng Zhang, Chunyuan Zheng, Cong Fang, Haoxuan Li, Zhouchen Lin

Optimization and Control · Cross-listed from math.NA

Edge-preserving Bayesian neural networks via hierarchical sparsity-promoting priors

Recovering piecewise-smooth functions with jump discontinuities from noisy, indirect, and incomplete data is a central task in computational inverse problems. Bayesian neural networks (BNNs) with heavy-tailed weight priors have emerged as effective edge-preserving priors for such problems. However, posterior inference for these models remains computationally challenging, and existing approaches settle for point estimates obtained by directly optimizing heavy-tailed log-densities. We propose GSBL-BNNs, which model the neural network parameters hierarchically using sparsity-promoting priors from generalized sparse Bayesian learning (GSBL). The hierarchical formulation recovers several popular marginal priors, including Laplace and Student's $t$ (including Cauchy). A key modeling ingredient, novel among edge-preserving BNNs and native to the hierarchical formulation, couples each neuron's bias to its weights through a shared hyperparameter, so that the induced jump discontinuities are free to occur anywhere in the domain while the conditionally Gaussian structure underlying efficient inference is preserved. Furthermore, we develop a block-coordinate descent (BCD) algorithm for maximum a posteriori (MAP) estimation. This BCD approach alternates between standard network training with iteratively reweighted $\ell^2$-regularization and exact, closed-form hyperparameter updates for gamma and inverse gamma hyperpriors. Building on the BCD algorithm, we perform approximate Bayesian inference and uncertainty quantification (UQ) using deep ensembles. Computational experiments—including signal denoising, deblurring, and inverting a two-dimensional hyperbolic conservation law— show that GSBL-BNNs deliver accurate, edge-preserving reconstructions with built-in UQ. Compared with Gaussian- and Cauchy-BNNs, they yield more accurate predictive means and avoid overconfident uncertainty estimates.
Read the full abstract

Optimization and Control

Matching Randomized Bounds for Finite-Sum Nonconvex-Concave Optimization

We characterize the randomized component-oracle complexity of finite-sum nonconvex-concave optimization with a bounded dual domain. For $\Phi(x)=\max_{y\in Y}n^{-1}\sum_{i=1}^n f_i(x,y)$, the goal is to find an $\varepsilon$-stationary point of the Moreau envelope $\Phi_{1/(2L)}$ with probability at least $2/3$ and a fixed query budget. Let $A=L\Delta/\varepsilon^2$, $B=LD_Y/\varepsilon$, and $\bar B=\max\{1,B\}$, where $\Delta$ bounds the initial primal gap and $D_Y$ bounds the dual diameter. For $A$ above a universal constant, the worst-case complexity is \[ \widetilde\Theta\!\left(n+A\min\left\{ \sqrt n\,\bar B+\bar B^4,\ n^{3/4}\bar B\right\}\right). \] The bounds agree within constants for $B\le n^{1/4}$ and within one logarithm for larger $B$. Upper bounds require only mean-squared averaged smoothness and concavity of the average in the dual variable; lower bounds already hold under individual smoothness. The lower constructions cover adaptive component selection, arbitrary query norms, private randomization, and outputs that were never queried. A cone construction gives the small-diameter bound, while a serial branch construction and balanced replication give the remaining two regimes. On the upper side, a persistent estimator with primal averaging complements accelerated proximal dual steps equipped with an explicit primal certificate. The characterization takes the worst case over admissible known dual domains and finite dimensions.
Read the full abstract

All authors Wendao Wu, Haihan Zhang, Qian Chen, Chenheng Zhang, Chunyuan Zheng, Cong Fang, Haoxuan Li, Zhouchen Lin

Optimization and Control

Decision-Making in Digital Twins: Optimization, Control, and Learning

A digital twin is a virtual representation of a physical system, continuously updated with data from the physical system, enabling dynamic information exchange and decision-making. This bidirectional interaction, which distinguishes digital twins from conventional simulation models, introduces new challenges to simulation-based optimization, control, and learning. This paper reviews recent developments in decision-making methodologies motivated by digital twins. We focus on three interconnected areas: simulation optimization, sequential decision making, and risk-aware decision making. For simulation optimization, we discuss data-driven simulation optimization under input uncertainty, streaming input data, decision-dependent observations, and limited computational and data-collection resources. For sequential decision making, we review the connections among digital twins, stochastic optimal control, and reinforcement learning, with emphasis on Bayesian approaches that update models and policies as new information becomes available. For risk-aware decision making, we discuss distributionally robust and Bayesian risk formulations as alternative approaches for addressing model uncertainty. Throughout the paper, we emphasize the common problem of making decisions while the underlying model is being updated. We conclude by identifying research challenges and opportunities.
Read the full abstract

Optimization and Control

Solution Set Dimensions in Convex Optimization and Primal-Polar Facial Signatures

We study the possible dimensions of solution sets in linearly perturbed convex optimization problems and their behavior under convex duality. The dimensions of solution sets and inverse solution sets capture two complementary forms of nonuniqueness: that of optimal solutions for a given perturbation, and that of perturbations for which a given point is optimal. Although these two families are linked by Fenchel conjugacy, we show that their collections of dimensions can be prescribed independently for finite supercoercive convex functions. As a geometric consequence, we prove that the facial dimension signatures of a convex body and its polar can be prescribed independently. The connection between these two realization results is obtained by homogenizing the epigraphs of a function and its conjugate. In contrast, under simultaneous facial exposedness, we establish a dimensional obstruction relating the smallest positive proper face dimensions of the body and its polar. These results resolve two open questions posed by Roshchina and Tunçel.
Read the full abstract

Optimization and Control

Extragradient and Past Extragradient Methods for $(L_0,L_1)$-Lipschitz Variational Inequalities

Variational inequalities (VIs) capture a broad class of optimization problems central to modern machine learning, including min-max optimization and equilibrium computation in multiplayer games. Traditionally, algorithms for solving VIs, such as Extragradient (EG) and Past Extragradient (PEG), have been analyzed under the standard $L$-Lipschitz assumption. However, this condition is often overly restrictive, and recent works have instead turned to the more relaxed $(L_0,L_1)$-Lipschitz setting. Existing guarantees for $(L_0,L_1)$-Lipschitz VIs require a quasi-sharpness condition [Vankov et al., 2024], while monotone guarantees for EG cover only unconstrained root finding [Choudhury and Loizou, 2025]. In this work, we provide the first convergence guarantees for EG and PEG for monotone $(L_0,L_1)$-Lipschitz VIs. We also improve upon prior convergence guarantees for strongly monotone VIs by supporting larger stepsizes in the constrained setting and eliminating exponential dependence on the initial distance from the solution for convergence rates in the unconstrained setting. Lastly, we propose line-search variants for when Lipschitz parameters are unknown. Numerical experiments complement our theoretical results and demonstrate the practical efficacy of our methods.
Read the full abstract

Optimization and Control

A Composite Bayesian Risk–Distributionally Robust Optimization Model for Stochastic Programming with Elliptical Distribution Ambiguity

We study a stochastic optimization problem in which the random vector follows a semiparametric elliptical distribution with unknown location and shape parameters and an unspecified radial distribution. Assuming that the decision maker only has access to samples of the random vector, we develop a framework to jointly address parameter uncertainty and distributional ambiguity arising from this composite structure. To update the posterior distribution of the parameters, a key challenge is that the density of an elliptical distribution depends on the unknown radial distribution, and hence conventional likelihood-based Bayesian inference is infeasible. To overcome this challenge, we propose a novel estimating-equation-based Gibbs inference framework that updates posterior distributions of model parameters solely through estimating equations. For elliptical distributions, we construct a pair of estimating equations for the location and shape parameters that are invariant to the radial component, which enables posterior learning without specifying the density. For the distributional uncertainty, we construct an ambiguity set for the radial distribution by transforming observed samples into pseudo-samples of the radial variable. Combining the above components, we formulate a composite Bayesian risk–distributionally robust optimization model, in which the inner problem captures worst-case radial distributions within the ambiguity set, and the outer problem evaluates the decision risk induced by the parameter uncertainty through a user-specified risk measure. We further develop a cutting-plane algorithm to solve the resulting optimization problem. Numerical experiments demonstrate that the proposed approach provides substantial performance improvements over benchmark methods, including sample average approximation and distributionally robust optimization.
Read the full abstract

Optimization and Control · Cross-listed from cs.LG

GpFlash: Maximizing Speed and Minimizing Memory for Gaussian Process Training and Prediction

This paper presents GpFlash, a Gaussian process library that is focused on fast training and posterior evaluations, minimal memory usage, and scalability to high-dimensional problems. GpFlash was compared to the popular GP libraries scikit-learn, GPy, GPflow, GPyTorch, and SMT on a MacBook Pro and the supercomputer Trillium. GpFlash was found to use less memory and to be faster for training and posterior evaluations relative to the other GP libraries for all cases considered. For example, when using $\nx = 10^4$ training points in a 10-dimensional space GpFlash required 9.6 to 35 times less memory and was 2.6 to 54 times faster than the other GP libraries for the evaluation of the marginal log likelihood and its gradient. The memory reduction was achieved by only using one matrix of size $\nx \times \nx$ to store different symmetric matrices in its lower and upper triangular entries. The computational speedup was the result of designing algorithms to access memory efficiently, using BLAS and LAPACK methods extensively, and factoring kernel equations so that expensive exponential calculations could be reused. GpFlash was more than 440 times faster than all of the other GP libraries considered when evaluating the posterior mean together with its gradient for the case with $10^4$ training points and 10 dimensions. GpFlash is available at https://github.com/marchildon/gpflash.
Read the full abstract

Optimization and Control · Cross-listed from stat.ML

Feature Normalization under Finite Training Budgets

Standardizing a rare feature can help stochastic gradient descent learn its effect within a finite budget, but it also changes the noise cost of learning. We characterize the lowest rare-subgroup risks attainable by standardized and unscaled training when each chooses a nonnegative constant learning rate and a deterministic stopping time under the same observation cap. For two independent binary features, we derive both optimized risks as prevalence vanishes while the expected number of rare observations remains finite. Uniform control over rates and stopping times reduces each limiting oracle to a scalar minimization. With positive noise, the preferred scaling changes exactly once as we move signal to the rare feature while the limiting risk at initialization stays fixed and positive. The formulas also explain how noise and budget affect the comparison. Separate finite-prevalence bounds favor standardization on a rare-dominant signal family and unscaled training on a common-dominant family. The results concern final iterates with population preprocessing and fresh observations. Exact-moment searches assess the approximation at finite prevalence. In a controlled study, both orderings persist under observation reuse, estimated preprocessing, and intercept fitting when tuning choices are fixed before evaluation. Selecting a predictor after observing training can reverse an ordering, even when selection uses exact population risk.
Read the full abstract

Optimization and Control

Matching Complexity for Second-Order Stationary Point with Sample Reuse

We consider stochastic nonconvex optimization with sample-wise Lipschitz gradients and Hessians, and reusable opaque sample handles. Gradient and complete-Hessian evaluations are charged separately, including both endpoints of a same-sample difference. For fixed positive problem parameters and in the high-dimensional worst case, we prove that the minimax total derivative-query complexity of finding an $(\varepsilon,\gamma)$-second-order stationary point is \begin{equation*} \widetilde\Theta(\varepsilon^{-3}+\gamma^{-3}). \end{equation*} The upper bound combines curvature-first randomly signed negative-curvature steps with binary-ancestor derivative estimators; a truncated increment estimate charges predictable drift linearly and martingale motion quadratically. The curvature lower bound uses a deterministic smooth bounded-gradient chain, a radial barrier, and a random rotation. The new first-order lower bound is specifically robust to persistent sample reuse and to complete-Hessian access. It uses a slowly parameterized conservative sign chain, a third-order reverse stick-breaking localization inequality, and stateful relative-entropy accounting for reusable handles. Thus the total rate is near-optimally matched for every relation between $\varepsilon$ and $\gamma$.
Read the full abstract

All authors Qian Chen, Wendao Wu, Haihan Zhang, Chenheng Zhang, Chunyuan Zheng, Cong Fang, Haoxuan Li, Zhouchen Lin

Optimization and Control

Calibrated $\alpha$-Barycenters for Multiobjective $f$-Divergence Optimization

Matching one distribution to many targets can require repeated evaluations of every target. CAL-$\alpha$-MODM aggregates a nonlinear predictor into one barycenter, calibrates its residual derivatives, and checks steps against the original $f$-divergence objective. Under smoothness assumptions, calibration controls parameter-step model error without requiring targets to be close to the model. Exact-oracle stationarity and local convergence guarantees are established. At common matching, canonical geometry cancels the leading quadratic iteration error, giving cubic convergence with second-order calibration and sufficiently accurate inner solves. High-precision experiments support this prediction. Sensitivity-based initialization also accelerates preference continuation along regular stationary branches. On 300-target Jensen–Shannon problems, CAL-p2 achieves median paired speedups of $3.15\times$ over L-BFGS and $1.10\times$ over second-order Direct-TR, across 23/25 and 24/25 objective-comparable stationary pairs. The savings arise when cheaper predictor queries offset calibration overhead.
Read the full abstract

Optimization and Control · Cross-listed from eess.SY

Venusian Radio Occultation Constellation Design and Reconfiguration Problems: A Benders Decomposition Approach

Driven by the scientific value of Venusian atmospheric studies, numerous missions, such as Magellan and Venus Express, have relied on single-spacecraft radio occultation (RO) experiments. Despite their proven success, data throughput is inherently limited by the use of a single transmitter and its relative geometry with respect to Earth-based receivers. Therefore, the literature proposes leveraging a constellation of spacecraft to increase RO measurement opportunities and improve spatial and temporal coverage; however, current efforts impose strict simplifying assumptions that limit their applicability and responsiveness to complex mission narratives. Beyond initial constellation design, evolving mission objectives may require event-based constellation reconfiguration during operations. To address both design and operational needs, we propose two problems formulated as integer linear programs: the RO Constellation Configuration Design Problem and the RO Constellation Reconfiguration Problem. Further, we address the programs' solution-space explosion with a Benders decomposition approach. Numerical experiments show the applicability of our problems for constellation configuration design, tackle the full mission pipeline, transitioning from global atmospheric profiling to on-demand regional profiling above areas of volcanic interest, and show that the Benders decomposition approach efficiently obtains high-quality solutions to large-scale instances.
Read the full abstract

Optimization and Control · Cross-listed from cs.DS

A Deterministic $0.401$-Approximation for Non-monotone Submodular Maximization under Matroid Constraints

Submodular maximization arises in a wide range of combinatorial optimization problems and has been extensively studied from the perspective of approximation algorithms. For maximizing nonnegative, non-monotone submodular functions subject to a matroid constraint, randomized algorithms achieve a $0.401$-approximation, whereas obtaining comparable deterministic guarantees remains challenging. In this paper, we present a deterministic $0.401$-approximation algorithm in the exact value and independence oracle model, improving upon the previous deterministic guarantee of $(0.385-\varepsilon)$ \cite{ChenGLSZ2026}. Our algorithm builds on the extended multilinear extension and provides a deterministic implementation of the recursive guided framework underlying the randomized $0.401$-approximation algorithm. The main technical ingredient is a direction oracle supported on at most two partitioned matroid bases. This oracle simultaneously satisfies the growth condition required by the extended-multilinear update and an approximate stationarity condition needed to construct the recursive guides. The resulting finite-support representation enables exact evaluation of the relevant expectations and marginal values without random sampling. The algorithm uses polynomially many value- and independence-oracle queries.
Read the full abstract

All authors Shengminjie Chen, Kaifeng Lin, Xiaoming Sun, Jialin Zhang

Optimization and Control

Matching Full-Hessian Oracle Complexity for Nonconvex–Strongly-Concave Minimax Optimization

We characterize the oracle complexity of finding first-order stationary points in smooth nonconvex–strongly-concave (NC–SC) minimax optimization when each query provides the full Hessian in addition to the function value and gradient. For $\Phi(x)=\max_y f(x,y)$, assume that $f$ has an $L$-Lipschitz gradient, a $\rho$-Lipschitz Hessian, and is $\mu$-strongly concave in $y$. Let $\kappa=L/\mu$, let $\Delta$ denote the initial value gap, and define $r=\rho\kappa^{3/2}\varepsilon/L^2$. We establish matching deterministic upper bounds and randomized lower bounds, up to logarithmic factors and additive initialization costs. When the leading term dominates initialization, the resulting minimax complexity is \[ \widetilde{\Theta}\!\left( \frac{\Delta L}{\varepsilon^2}\Psi_\kappa(r) \right), \qquad \Psi_\kappa(r)= \begin{cases} r^{1/2}, & 0<r\le 1,\\ r^{2/7}, & 1<r<\kappa^{7/4},\\ \sqrt{\kappa}, & r\ge\kappa^{7/4}. \end{cases} \] Despite the stronger oracle, the optimal complexity exhibits a nontrivial dependence on accuracy, conditioning, and Hessian smoothness. In particular, these regimes exhibit accuracy dependencies $\varepsilon^{-3/2}$, $\varepsilon^{-12/7}$, and $\varepsilon^{-2}$, respectively. The upper bounds combine second-order regularity estimates for a saddle Moreau envelope with certified accelerated reduced-dual solves and amortized warm starts. The lower bounds combine an all-derivative lift with a serial composition of private-handle constructions, using hard instances chosen independently of the algorithm's realized random seed. The resulting characterization quantifies improvements over the classical first-order benchmark under gradient smoothness alone and identifies the regime in which that benchmark is recovered.
Read the full abstract

All authors Fangqing Zhang, Haihan Zhang, Wendao Wu, Chenheng Zhang, Chunyuan Zheng, Cong Fang, Haoxuan Li, Zhouchen Lin

Optimization and Control

On the Stability of Power Systems with AI Data Centers: A Multi-Equilibrium Switched DAE Framework

The rapid growth of Artificial Intelligence (AI) data centers is introducing dynamic loads into power transmission systems as GPU clusters switch between computational phases with distinct power demands. These dynamics are naturally modeled by switched affine differential-algebraic equations (DAEs), where each mode induces a different equilibrium and consistency space. We study their stability using control and hybrid systems tools. Differential projectors derived from the quasi-Weierstrass form yield an equivalent hybrid representation, from which we characterize the $\omega$-limit set and establish Lyapunov conditions for practical stability with an explicit $\mathcal{KL}$ bound accounting for switching-induced resets. For stochastic inference workloads, we use Foster–Lyapunov techniques to establish probabilistic uniform global recurrence under mode-dependent dwell times and transition probabilities. The framework is validated on the IEEE 39-bus transmission network with co-located AI data centers using high-fidelity electromagnetic transient simulations of deterministic training and stochastic inference workloads.
Read the full abstract

All authors Vishal Shenoy, Damola Ajeyemi, Mahmoud Abdelgalil, Guido Cavraro, Emiliano Dall'Anese, Jorge I. Poveda

Optimization and Control

Globally Convergent Block Coordinate Descent Algorithms for Semi-Nonnegative Independent Component Analysis

We propose a subgradient-based block coordinate descent (BCD-SubG) framework for nonconvex optimization over products of probability simplices and a closed feasible set, motivated by semi-nonnegative independent component analysis (Semi-NNICA). Elementary column transformations are used to preserve simplex feasibility, while first-order information guides block and pair selection. We establish the global convergence of BCD-SubG using a constrained Lojasiewicz inequality; additional eventual-interiority conditions yield stationary limits and convergence rates. We then specialize BCD-SubG to the direct-fit and indirect-fit Semi-NNICA models and derive explicit updates that ensure global convergence without further assumptions. For both models, eventual interiority also yields stationary limits and convergence-rate estimates. Experiments on synthetic mixtures and simulated magnetic resonance spectroscopy data show generally lower recovery errors and shorter wall-clock times than the tested Semi-NNICA baselines. Joint diagonalization experiments further demonstrate lower final objective values.
Read the full abstract

All authors Zhenyu Cai, Wentao Ding, Jianze Li, Wen Huang

Optimization and Control · Cross-listed from physics.flu-dyn

Versatile two-layer model for thermohydraulic prediction of microchannel heat sinks

Three-dimensional conjugate heat-transfer simulations provide detailed predictions for microchannel heat sinks but can be computationally demanding when many geometries and operating conditions are evaluated. This study develops a versatile two-layer model (VTLM) that reduces the three-dimensional problem to two coupled two-dimensional fields while retaining the in-plane geometry of the flow passages and fins. Effects associated with the unresolved channel-height direction are incorporated through hydraulic and thermal closures that account for wall friction and the fluid- and fin-side contributions to interlayer heat transfer. The closures are evaluated from the representative channel geometry, flow-development condition, and fin configuration. The VTLM was assessed against three-dimensional reference models across broad ranges of channel aspect ratios and operating conditions, including uniform and nonuniform fin arrangements. Compared with the existing two-layer models, the VTLM showed improved overall agreement with the three-dimensional reference results. The results showed that prediction accuracy was governed mainly by the flow-development condition and channel aspect ratio, from which a conservative screening guideline was proposed for the examined straight-channel configurations. The assessment of finned expansion–contraction channels further supported the applicability of the VTLM to the investigated passages with abrupt width changes. Overall, the VTLM provides a reduced-dimensional framework for thermohydraulic analysis across the range of microchannel heat-sink configurations examined in this study without explicitly resolving the channel-height direction.
Read the full abstract

Optimization and Control

Preconditioned Latent Variable Proximal Point Method for Optimal Control Problems with Control Constraints

The latent variable proximal point (LVPP) method solves optimization problems in function spaces with pointwise constraints by a sequence of smooth subproblems whose iterates are strictly feasible. We apply it to linear-quadratic elliptic optimal control problems. Control and observation are distributed but may be restricted to subdomains. We prove that the LVPP subproblems are well posed in appropriate function spaces, that the iteration converges for every sequence of proximity parameters with divergent sum, and that it delivers the Lagrange multiplier of the constraint as a by-product. We interpret the Newton step for the LVPP subproblem as a linear-quadratic optimal control problem with a spatially varying control cost, and we show that Newton's method converges locally quadratically. The Newton systems are solved by the minimal residual method with a block-diagonal preconditioner. We prove that the spectrum of the Newton operator with respect to the inner product defined by the preconditioner is bounded independently of the proximity parameter, the iterate, the mesh and the observation and control domains. The results are combined into a complete algorithm, and tested in numerical experiments.
Read the full abstract

Optimization and Control · Cross-listed from math.AP

Minimal time for heat observability on almost-Riemannian surfaces

We determine the minimal observation time for the heat equation on a closed almost-Riemannian surface, for a class of observation sets leaving sufficiently thin collars around simple Baouendi-Grushin circles. The structure and density are analytic near these circles, and the coefficients may depend on both variables. The formula is expressed through a complex phase describing concentration near the singular set and extends the thresholds known for separated models. Regular unobserved regions away from the circles do not change this time, provided all other singularities are observed. For singular circles of constant bracket step greater than two, local analyticity implies failure of observability at every time for observation sets at positive distance. We also construct quasimodes concentrated near simple singular circles, with exponentially small errors. Their superpositions give a lower bound for the Schrödinger observation time.
Read the full abstract

Optimization and Control · Cross-listed from eess.SY

Scalable Identification of Physics-Based and Neural Dynamics from Partial Measurements

We present a scalable method for identifying nonlinear dynamics from partial measurements, supporting physics-based, neural-network, and gray-box models. We use the Prediction Error Method (PEM) coupled with state estimation performed by a Moving Horizon Estimation (MHE) scheme, which can account directly for general nonlinear dynamics. This results in a bi-level optimization problem that scales poorly with increasing model parameters. To improve computational efficiency, we derive a reformulation that avoids explicit parameter-wide sensitivity matrices and exploits the temporal structure of the underlying optimization problem. Numerical comparisons show up to a $215\times$ speedup in hypergradient evaluation. Racing-vehicle simulations demonstrate accurate prediction and physical-parameter recovery in tire-force identification. Real-world data show that the gray-box model improves prediction accuracy with a neural residual while retaining physical interpretability.
Read the full abstract

Optimization and Control

Towards learning-enhanced solvers for multi-parametric optimization

Neural networks (NNs) have been employed in various ways to accelerate the solution of multi-parametric optimization problems (OPs), which prominently arise in model predictive control (MPC). Early approaches focused on approximating the parametric solution (e.g., the optimal control law) directly by NNs. More recently, learning-based techniques have been used to emulate or enhance solver iterations. For the direct approximation of the parametric solution, it has been demonstrated that exploiting structural properties of the underlying OP facilitates the design of suitable NN architectures. In this paper, we show that structural analysis can also be leveraged to systematically integrate learning into classical optimization algorithms, thereby enhancing their performance. We formalize our concept for the case of multi-parametric quadratic programs (mpQPs), which frequently arise in linear-quadratic MPC, as a representative application. More precisely, we demonstrate that tailored NNs enable a parameter-dependent fine-tuning of the step sizes in a standard dual projected gradient scheme. The design of the NNs is guided by the observation that optimal step sizes, which permit obtaining the exact optimal solution in finitely many iterations, are piecewise constant functions over the parameter space (e.g., the state space in MPC). We establish key theoretical properties of the proposed approach and illustrate its effectiveness in a numerical case study.
Read the full abstract

Optimization and Control · Cross-listed from cs.LG

Asymptotic Risk of Learning-Rate Schedules for Least-Squares Regression with Discretely Distributed Features

We study the last iterate of stochastic gradient descent (SGD) on streaming least squares with discretely distributed features (Slock's model), which decouples the dynamics across eigendirections. Under power-law source and capacity conditions, we derive the exact finite-time risk and asymptotic equivalents of constant and Warmup-Stable-Decay (WSD) schedules. With a tuned base learning-rate, the resulting scaling law decays as $\Theta(T^{-q})$, where $T$ is the horizon, and $q$ is independent of the cooldown length $c \in (0,1]$ but is strictly smaller for the constant schedule: any cooldown of fixed relative length $c > 0$, however short, improves the risk by a polynomial factor, while its length only affects the constant, which is minimized by linear decay ($c = 1$). Finally, we show that the optimal risk depends on the batch size $b$ and horizon $T$ only through $bT$, up to a critical batch size.
Read the full abstract

Optimization and Control

Taming Nonconvexity with Epigraph Lifting

We consider inverse problems regularized by a trained neural network with convex activations and unrestricted weights. The resulting optimization problem is generally nonconvex, and gradient-based methods may converge to poor local minima. We introduce a penalized epigraph lifting of the network and show that the lifted problem admits a difference-of-convex (DC) decomposition. For Lipschitz continuous activations, we establish finite exactness: the original objective is recovered exactly at a finite and explicitly computable penalty threshold $\bar\rho$. Building on this structure, we propose H-DC (Homotopic DC), a homotopic scheme that solves a sequence of DC subproblems with increasing $\rho$, warm-starting each level from the previous solution. Numerical experiments on synthetic problems, a manifold denoiser regularizer, and image inpainting with a learned Mumford–Shah regularizer show that H-DC consistently escapes poor local minima and reaches solutions with lower objective values than gradient-based methods from the same initialization.
Read the full abstract

Optimization and Control · Cross-listed from eess.SY

Equilibria and consensus in discrete-time opinion dynamics with mixed weighted-averaging and weighted-median mechanisms

Opinion dynamics models relate interpersonal influence to consensus or persistent disagreement. We study a discrete-time model combining weighted-average and weighted-median updates on a fixed directed network. Individuals update synchronously with possibly different adjustment coefficients. We characterize equilibria through local balance conditions and opinion level sets, and establish convergence to a Lyapunov-stable equilibrium using nonexpansiveness and Krasnosel'skii–Mann theory. We define type-dependent cohesive sets: each weighted-average member assigns all its weight to the set, whereas each weighted-median member assigns at least one half. Consensus is reached from every initial condition if and only if no two such sets are disjoint. Otherwise, an open set of initial conditions leads to persistent disagreement. Replacing weighted-median individuals by weighted-average individuals preserves consensus whenever it is already guaranteed for all initial conditions. Four numerical studies examine the number and locations of weighted-average individuals. The frequency of reaching a prescribed consensus tolerance is nondecreasing with their fraction in the sampled ensembles, but individual realizations need not follow this trend.
Read the full abstract