Optimization and Control · Cross-listed from math.NA
Matrix nearness problems: Do real inputs admit real solutions?
Vanni Noferini and Matvei Zhukov
Given a real square
matrix $A$ and a nonempty closed target set of complex
matrices $\mathcal E$, invariant by complex conjugation and containing real
matrices, does the subset of $\mathcal{E}$ consisting of the
matrices nearest to $A$ in the Frobenius distance always contain a real
matrix? We give negative answers when $\mathcal{E}$ is either the set of normal
matrices (solving a question posed by N. Higham) or the set of
matrices whose
eigenvalues lie in the closed left half-plane (solving a question posed by the first author and F. Poloni). For the latter problem, we also argue that the ratio between the real and complex distances is unbounded in every dimension $n\geq 3$, and this holds for every distance induced by a unitarily invariant norm. For both problems and $n \geq 3$, we show that a real input $A$ uniformly drawn from the unit sphere $\|A\|_F=1$ has no real minimizer over $\mathcal{E}$ with probability strictly between $0$ and $1$. The paper is complemented by some further results that are valid for more general target sets $\mathcal{E}$.
Read the full abstract ▾
Optimization and Control · Cross-listed from math.NA
Computer-assisted local maximality of regular polygons for torsional rigidity
Beniamin Bogosel
We study torsional rigidity as a function of the labeled vertices of a convex polygon. Starting from the distributed second shape derivative, we derive the Hessian with respect to vertex coordinates. At a regular polygon, dihedral
symmetry makes this
matrix block circulant in radial-tangential coordinates, reducing its spectrum to the
eigenvalues of Hermitian
matrices of order two. We also derive an exact second-variation Galerkin identity and guaranteed functional residual majorants. Finite elements approximate the PDE solutions entering the Hessian, and FLINT/Arb provides the interval arithmetic needed for certification. In the scale-invariant setting, we certify exactly four zero
eigenvalues generated by similarities and $2n-4$ strictly negative
eigenvalues for $5\leq n\leq25$. The regular polygons in this range are therefore strict local maximizers, modulo similarities, of torsional rigidity divided by area squared. We prove broken regularity of the first and second material derivatives at each fixed regular polygon, and deduce an $O(h^2)$ error bound for the exact Galerkin Hessian and its
eigenvalues on meshes fitted to the coarse fan.
Read the full abstract ▾
Optimization and Control · Cross-listed from cs.LG
Privacy Preserving Gossip Learning
Erkan Bayram, Mohamed-Ali Belabbas and Tamer Başar
We propose a decentralized
privacy-preserving learning
algorithm in which each
agent holds a single private sample and a shared model. Samples are learned sequentially, and each update must preserve the endpoint mappings at previously learned samples while protecting private data. This gives each
agent three roles: (i) a learner that updates the model parameters, (ii) a teacher whose sample is learned at the current iteration, and (iii) a protected
agent whose sample has already been learned. We build on Tuning without Forgetting (TwF) method to preserve previously learned mappings and show that TwF provides an indistinguishability guarantee for the learner whenever the set of protected
agents contains another sample with the same label. For the teacher, we formulate a minimax optimal control problem that models the differential
privacy noise as a worst-case disturbance to prevent performance loss while maintaining the same level of
privacy for the gradient. For the protected
agents, we compute the projections locally and aggregate them using a private push-sum gossip protocol. We prove
geometric convergence of the decentralized gossip
algorithm and of the distributed projection for TwF.
Read the full abstract ▾
Optimization and Control
Rank-Dependent Error Bounds and Near-Optimality in Quantum Control via Hierarchical Tucker Surrogates
Nahid Binandeh Dehaghani, Rafal Wisniewski, Julian Berberich et al. (+1 more)
We develop a certified finite-horizon
quantum optimal-control framework based on fixed-rank Hierarchical Tucker (HT) surrogates. Under a uniform rank-dependent HT truncation-accuracy condition, we establish exponentially decaying trajectory, cost, and value-function errors and near-optimality of surrogate-generated controls. We further derive a logarithmic rank–performance relation, an a posteriori cost certificate from realized truncation perturbations, and guarantees for inexact surrogate optimization and control convergence under additional regularity conditions. Numerical experiments on controlled XXZ
spin chains provide finite-sample evidence consistent with the truncation condition and illustrate the trade-off between HT rank, representation size, surrogate accuracy, and control performance.
Read the full abstract ▾
All authors Nahid Binandeh Dehaghani, Rafal Wisniewski, Julian Berberich, A. Pedro Aguiar
Optimization and Control
Convergence of the Sinkhorn Riemannian metric for finitely supported measures
Gilles Mordant and Liane Xu
The Sinkhorn Riemannian metric characterizes the local behavior of the Sinkhorn divergence, a debiased version of
entropy-regularized optimal transport. Although the Sinkhorn divergence is known to approximate the squared Wasserstein-2 distance as the regularization parameter $\varepsilon$ goes to $0$, the behavior of the Sinkhorn Riemannian metric as $\varepsilon\to 0$ remains largely open. To the best of our knowledge, the only existing argument is for measures with a density and is still a formal computation. In this work, we prove the convergence of the Sinkhorn Riemannian metric as $\varepsilon\to 0$ in the case of finitely supported measures undergoing horizontal perturbations. Under our assumptions, our techniques additionally allow us to prove the convergence of higher order derivatives of the Sinkhorn Riemannian metric, thereby ensuring the convergence of the associated Riemann
curvature tensor and Christoffel symbols as $\varepsilon\to 0$. We also compare with the absolutely continuous setting and show where our approach fails for measures with a density.
Read the full abstract ▾
Optimization and Control
On the Homogenization of Norm and Time Optimal Controls for Parabolic Equations with Oscillating Coefficients
Jon Asier Bárcena-Petisco and Marius Tucsnak
We study homogenization of norm-optimal and time-optimal controls for a one-dimensional heat equation with rapidly oscillating periodic
diffusion coefficients. The controls act on a fixed subinterval and are subject to pointwise constraints in space and time. We first derive final-state observability and null-controllability estimates that are uniform with respect to the oscillation scale. Combining these estimates with periodic homogenization and an abstract theory of optimal controls for parabolic systems, we prove convergence of the norm-optimal values and controls to their homogenized counterparts. We then deduce convergence of the optimal times and time-optimal controls. The controls converge weakly-
star in the space of essentially bounded functions and strongly in every Lebesgue space with finite exponent, after extension to a common time interval when necessary.
Read the full abstract ▾
Optimization and Control
Exponential Turnpike in Optimal Boundary Control of Multi-Dimensional First-Order Hyperbolic Systems
Haitian Yang
We establish an exponential turnpike property for optimal boundary control of multi-dimensional first-order linear
hyperbolic systems in which both the control and observation operators are unbounded and the cost functions may be nonquadratic. The main assumptions are internal and boundary dissipation conditions. These conditions ensure that the associated input-to-state and input-to-output maps are bounded, with operator norms independent of the time horizon T. This uniform boundedness is essential to the proof, which combines weighted energy estimates for the forward and adjoint equations, input-output duality, and strong convexity. Finally, we apply the result to the linearized two-dimensional Saint-Venant equations with sluice-gate control and illustrate the turnpike behavior numerically.
Read the full abstract ▾
Optimization and Control · Cross-listed from stat.ML
Evaluation of optimisation and Bayesian inference methods for reaction rates in atmospheric chemical mechanisms
Valery Ashu, Wenqing Peng, Zhi-Song Liu et al. (+7 more)
Constraining reaction rate coefficients is a central challenge in the development of explicit atmospheric chemical mechanisms, particularly for autoxidation systems where many reaction pathways are only indirectly observed through high-resolution mass spectrometry. In this study, we evaluate rate-coefficient optimisation methods for a toy-case autoxidation mechanism using synthetic data with known ground truth. Two complementary approaches are compared: ODE-constrained neural-network optimisation, which provides efficient point estimates of uncertain rate coefficients, and the Markov Chain Monte Carlo (MCMC) approach, which samples the posterior distribution of rate coefficients and quantifies parameter uncertainty. The methods are tested using direct concentration observations and mass-spectral observations under different noise levels. For unperturbed and low-noise synthetic observations, both methods converged towards the known rate coefficients, with the neural-network optimiser providing faster point estimates. Under high-noise conditions (with the signal-to-noise ratio approximately S / N = 1), however, MCMC was substantially more robust in recovering the rate coefficients. The posterior analysis shows that mass-spectral aggregation broadens credible intervals even at low noise, and that high-noise mass spectra can leave many individual reaction rates weakly identifiable. Posterior predictive validation nevertheless shows how broad parameter uncertainty constrained by MCMC remains consistent with accurate reproduction of the observable mass spectrum. These results demonstrate that point-estimation and
Bayesian sampling methods provide complementary information: neural-network optimisation is effective for informative data, whereas MCMC is essential for diagnosing uncertainty, non-uniqueness, and identifiability in noisy or aggregated inverse problems.
Read the full abstract ▾
All authors Valery Ashu, Wenqing Peng, Zhi-Song Liu, Heikki Haario, Andreas Rupp, Taiwo Ashu, Petri Clusius, Lukas Pichelstorfer, Zihao Fu, Michael Boy
Optimization and Control
A proximal-linearized NEP method for composite optimization with conic and manifold constraints
Hao He, Ruyu Liu and Shaohua Pan
This paper studies difference-of-convex (DC) composite optimization problems with conic and
manifold constraints. By penalizing the conic constraint with a distance-based penalty, we propose an inexact proximal-linearized nonsmooth exact penalty (iPLNEP)
algorithm. The proposed method successively finds approximate minimizers of proximal-linearized subproblems over the tangent spaces of the
manifold based on computable inexactness criteria, while adaptively updating the proximal and penalty parameters. Under a boundedness assumption on the iterate and penalty parameter sequences, iPLNEP is shown to achieve an $O(\epsilon^{-2})$ worst-case iteration complexity bound for finding an $\epsilon$-stationary point. Using accelerated semi-proximal ADMM to solve the subproblems, we establish an overall oracle complexity bound of $O(\epsilon^{-4})$. Under an additional uniform error-bound condition, using semi-proximal ADMM as the subproblem solver yields an improved overall oracle complexity bound of $O(\epsilon^{-2}\log\epsilon^{-1})$. To the best of our knowledge, this provides the first proximal-linearized NEP framework with provable oracle complexity guarantees for DC composite optimization with conic and
manifold constraints. If, in addition, the associated potential function satisfies the KL property, the whole sequence of iterates converges to a stationary point. Extensive numerical experiments on composite optimization problems with orthogonal-
manifold, nonnegative cone, and second-order cone constraints demonstrate the effectiveness of the proposed method.
Read the full abstract ▾
Optimization and Control
SCMO: Stochastic Control for Optimization over Probability Measures on Infinite-Dimensional Spaces
Hang Cheung and Jinniao Qiu
We study objective-only optimization of possibly nonconvex and nonsmooth functionals over probability measures on a separable
Hilbert space, allowing the optimizer to be intrinsically non-Dirac. We introduce SCMO (
Stochastic Control Measure Optimizer), a gradient-free particle method derived from
entropy regularized
stochastic control. After finite-particle and Galerkin approximations, a Cole–Hopf transform represents the optimal feedback as a Gibbs-weighted terminal displacement. SCMO approximates this feedback by sampling context clouds, replacing one particle at a time with candidate draws, scoring the resulting empirical measures, and applying exponential reweighting. SCMO uses separate covariances for candidate proposals and particle updates, termed matched when equal and nonmatched otherwise; our analysis covers both settings. In the matched case, we establish PDE-free qualitative convergence and a projection-first quantitative bound separating particle, Galerkin-projection, and
entropy-regularization errors. For the practical multi-context implementation with nonmatched covariance, we prove finite-candidate consistency and show that the signed,
curvature-dependent effect of nonmatched covariance can reduce the resulting upper bound on the approximation error. Finally, experiments on function-space, trajectory-law, and contact-rich manipulation problems show that SCMO handles nonsmooth and nonconvex objectives, escapes suboptimal local basins, and recovers prescribed
multimodal, non-Dirac law structure. Code for reproducing our experimental results is available at https://github.com/HenryCHEUNG7373/SCMO.
Read the full abstract ▾
Optimization and Control · Cross-listed from quant-ph
Robustness of Hidden-Variable Theories and Matrix Scaling
Giulio Malavolta, Harold Nieuwboer, Akshay Ramachandran et al. (+1 more)
Motivated by
quantum foundations and complexity theory, Aaronson formalized a hidden-variable theory inspired by a proposal by Schrödinger. To any
quantum state and
unitary, this Schrödinger theory assigns a joint probability distribution via the Sinkhorn
algorithm: rescale the columns and rows of the entrywise modulus of the
unitary so that the marginals match the Born rule for the initial and final
quantum states, respectively. He conjectured that this map is robust, i.e., inverse polynomially small perturbations in the inputs lead to inverse polynomially small perturbations of the joint distribution, which is important for complexity theoretic applications. We present a counterexample to this
conjecture: We construct a pure state and two
unitaries that are exponentially close, yet the joint probability distributions assigned by the Schrödinger theory differ by at least an inverse-linear term in at least one entry, and by a constant in total variation distance. We also propose a modified version of Schrödinger's theory that satisfies robustness, while retaining all of its other desirable properties. This yields a complete picture of which axioms of Aaronson can be simultaneously satisfied by hidden-variable theories.
Read the full abstract ▾
All authors Giulio Malavolta, Harold Nieuwboer, Akshay Ramachandran, Michael Walter
Optimization and Control
Three-Phase Unbalance Mitigation via DSO-FRA Coordination: A GNB-Based Chance-Constrained Model Considering PV Uncertainty
Qun Zhou, Jiale Guo, Jing Huang et al. (+3 more)
Although flexible resource aggregators (FRAs), represented by electric vehicle aggregators (EVAs) and load aggregators (LAs), possess inherent flexibility to mitigate three-phase unbalance, the key challenge lies in how to effectively incentivize them to actively participate in unbalance mitigation. To address this, this paper proposes a generalized Nash bargaining (GNB)-based chance-constrained coordinated operation model for the distribution system operator (DSO) and FRAs. In this framework, the DSO and FRAs cooperate to mitigate three-phase unbalance, where FRAs provide flexibility to reduce the DSO's unbalance mitigation costs, and the DSO offers economic incentives to FRAs in return. Bargaining power is tailored according to each participant's contribution to unbalance mitigation to ensure fair profit allocation. Moreover, a scenario-based chance-constrained formulation is integrated to handle photovoltaic (PV) output uncertainties. To solve the model, a distributed proximal decomposition
algorithm (PDA) is introduced for the independent model, while the coordinated model is decomposed into a social welfare maximization subproblem and a payment bargaining subproblem, with an improved bilinear Benders decomposition
algorithm developed to solve the former. Numerical results validate the effectiveness of the proposed method in mitigating unbalance, ensuring fair profit distribution, and enhancing robustness against uncertainties.
Read the full abstract ▾
All authors Qun Zhou, Jiale Guo, Jing Huang, Minrui Leng, Xueshan Liu, Chenjia Gu
Optimization and Control
A Filippov theorem for Volterra sweeping processes with one-sided Lipschitz perturbation
Abderrahim Jourani, Diana Narváez and Emilio Vilches
We prove a Filippov-type stability
theorem for integro-differential sweeping processes of Volterra type with an outer multivalued perturbation, in a separable
Hilbert space, under the assumptions that the moving sets are uniformly prox-regular and Lipschitz continuous with respect to the Hausdorff distance and that the perturbation is one-sided Lipschitz. Given an absolutely continuous solution of the system perturbed both in the state argument of the multivalued term and by an outer integrable term, we show the existence of a solution of the original Volterra sweeping process and estimate the distance between the two trajectories explicitly in terms of the data of the problem. The estimate becomes sharper when only the outer perturbation is present, and it recovers the Lipschitz dependence on the initial condition of Filippov's classical
theorem. The proof combines a reduction of the constrained dynamics to an unconstrained differential inclusion, measurable selection arguments, and an enhanced version of Grönwall's inequality established in this work. As an application, we obtain the Lipschitz dependence of the attainable set on the initial set with respect to the Hausdorff distance, together with a one-sided estimate quantifying the effect of the perturbations. We also work out a spatially distributed fishery model with ecological
memory whose harvesting rule, triggered by the aggregate biomass, is one-sided Lipschitz but is not Lipschitz continuous with respect to the Hausdorff distance, so that the classical framework does not apply. For this model, our estimates produce a stability constant governed only by the biological data.
Read the full abstract ▾
Optimization and Control · Cross-listed from eess.SY
Computing Robustness to Sparse Diagonal Uncertainty
Christoffer Kjellson and Emma Tegling
A new robustness metric $\nu$ was recently proposed as a substitute for the structured singular value $\mu$ to better capture robustness to sparse diagonal uncertainty, but its computation has remained an open problem. In this paper, we show that computing $\nu$ is equivalent to maximizing the spectral radius of a nonnegative
matrix product. This equivalence allows us to transfer existing results on spectral-radius maximization to $\nu$, including a refined upper bound and conditions under which the bounds coincide. We then provide reformulations and structural results that enable an
algorithm to solve the nonconvex optimization problem for nontrivial problems using global solvers. This also enables identification of the most fragile parts of the system. Our results do not yet provide a scalable solution for computing $\nu$, but they are an important step toward computability and interpretability.
Read the full abstract ▾
Optimization and Control
Spectral-Residual Continuous Greedy for Tensor Sampling
Hao Li, Jie Xu and Zheng Xie
Sampling a multidomain tensor from limited measurements is fundamental in structured linear inverse problems. Kronecker-structured sampling avoids the full high-dimensional sensing
matrix, but design remains difficult: sequential discrete methods can commit the cross-mode budget too early, whereas standard continuous greedy avoids such early commitment at the cost of repeated state-dependent gradient evaluations. We propose spectral-residual continuous greedy (\alg) for frame-potential (FP) tensor sampling. \alg maintains a state-dependent fractional allocation before rounding. Mode-wise Gram
matrices provide safe gradient intervals for direction certification, while Shapley values prioritize unresolved gradient queries. SR-CG then applies deterministic rounding followed by path-guided exchange (PGX), which reuses the final fractional state to restrict candidate swaps and accepts only exact FP-decreasing exchanges. We establish a finite-step approximation guarantee that approaches the classical $1-1/e$ factor as direction certification becomes exact and finite-step residual error vanishes. Experiments show fewer exact gradient evaluations and better FP designs, with clearer gains on instances where the cross-mode budget allocation is difficult to determine. \alg also achieves lower average normalized mean-squared error (NMSE) than Greedy-FP at all tested noise levels, although the reconstruction gain is smaller than the FP gain.
Read the full abstract ▾
Optimization and Control
Convex Pursuit-Evasion Games for Spacecraft Proximity Operations on Circular and Elliptical Orbits
Omer Burak Iskender
Inspecting or servicing a non-cooperative spacecraft is a two-player game: the target can thrust to defeat the inspector's plan. Hamilton-Jacobi-Isaacs reachability answers such games exactly but its cost grows exponentially with state dimension, while learning-based controllers scale yet certify nothing. Convex formulations are the usual escape, typically cast as convex-concave saddle-point problems. That framing fails in the terminal-distance orbital game: with effort regularization, the payoff is convex in both controls, so no pure open-loop saddle point need exist. In its place we derive two exact one-sided guarantees read from the players' terminal reachable sets via support functions: an escape certificate proving the target can hold a guaranteed standoff, and a security strategy bounding the miss distance the inspector can force. Together they bracket the engagement without assuming player rationality, and over two hundred perturbed trials the escape certificate separated capture from escape without error. A projected extragradient method supplies a strategy pair in about 25 ms, certified in place by best-response gaps. The construction carries unchanged to elliptical reference orbits via Yamanaka-Ankersen dynamics, where the orbital phase moves miss distance by nearly a factor of two, reproduced by nonlinear Keplerian propagation. Under receding-horizon play the inspector captures in half of ten representative engagements. Angles-only navigation error, keep-out zones, multiple pursuers, and closed-loop play are treated as extensions.
Read the full abstract ▾
Optimization and Control
Periodic fixed-points and their algebraic characteristics in discrete-time Lur'e feedback systems
Kang Tong, Christian Grussler and Michelle S. Chong
We study the problem of identifying nontrivial, i.e., nonzero, periodic fixed-points in discrete-time Lur'e feedback systems. Using the circulant
matrix constructed from the transfer function of the linear subsystem, whether stable or unstable, we introduce an algebraic framework that allows us to determine when such fixed-points exist. This framework yields a sector bound defined by two vectors, whose slopes correspond to the maximum and minimum positive singular values of the circulant
matrix. Assuming that the nonlinear feedback function is memoryless, we show that a necessary condition for the existence of nontrivial $P$-periodic fixed-points is that the intersection of the continuous completion of the nonlinear feedback function with that sector bound contains at least one point other than the origin. Our characterization provides a unified condition valid for all periods $P$, and further enables us to derive upper bounds on the amplitudes of admissible periodic fixed-points with bounded feedback functions. In particular, for relay feedback systems with passive feedback functions, we derive both upper and lower bounds for the amplitudes of such periodic fixed-points.
Read the full abstract ▾
Optimization and Control · Cross-listed from cs.CV
Robust low-rank tensor completion via factorized weighted tensor schatten-p norm minimization
Binghao Wang, Feng Zhang, Wendong Wang et al. (+1 more)
Low-rank tensor factorization provides a flexible framework for completing multidimensional data from incomplete and corrupted observations. However, unweighted spectral regularizers impose a common shrinkage profile across singular components, which may excessively attenuate dominant low-rank components, and factorized variants either lack component-specific weighting or require costly singular value decompositions (SVDs). This paper proposes two weighted Schatten-$p$ tensor factorization models, termed \WSpTFI{} and \WSpTFII{}, under the tensor-tensor product (t-product) framework to address these limitations. \WSpTFI{} is motivated by a factorized weighted tensor Schatten-$p$ norm identity and permits flexible, possibly asymmetric factor exponents. \WSpTFII{} constructs a regularizer from transform-domain column-pair energies, yielding SVD-free main factor updates and a column-
pruning mechanism for reducing redundant rank components. This paper further develops an iteratively reweighted alternating direction method of multipliers (ADMM)-type scheme for \WSpTFI{} and an iteratively reweighted least squares (IRLS)–block successive upper-bound minimization (BSUM) scheme for \WSpTFII{}. Theoretical analysis establishes the weighted factorization relation and provides a conditional limiting Karush–Kuhn–Tucker (KKT) characterization for \WSpTFI{}. For \WSpTFII{}, the actual damped quadratic block updates yield a quantitative sufficient-decrease mechanism for the fixed-$\delta$ smoothed factor objective. This implies asymptotic regularity, and every accumulation point of the fixed-dimensional tail is stationary. Experiments on synthetic tensor completion, color-image restoration, hyperspectral inpainting, and printed-
circuit-board defect detection demonstrate competitive reconstruction quality and robustness under various degradation conditions.
Read the full abstract ▾
All authors Binghao Wang, Feng Zhang, Wendong Wang, Jianjun Wang
Optimization and Control
A hybrid crossover kangaroo escape optimization framework for engineering optimization and UAV path planning
Han-Bin Liu and Huayi Wei
Complex engineering optimization problems are often characterized by multimodality, high dimensionality, and nonlinear constraints, posing significant challenges for efficient and reliable computation. To address these challenges, this paper develops a hybrid crossover-based optimization framework that enhances population interaction and improves search efficiency. The proposed framework integrates two complementary mechanisms, namely a Levy long jump crossover strategy for global exploration and a horizontal-vertical crossover strategy for effective information exchange and local refinement, thereby improving convergence behavior and robustness. To support scalable computation, the method is implemented within a unified multi-backend computational framework based on FEALPy, enabling consistent and efficient execution across heterogeneous platforms, including NumPy and PyTorch on both CPU and GPU. This design enhances portability, reproducibility, and computational efficiency in large-scale optimization tasks. Extensive experiments on the IEEE CEC2022
benchmark suite demonstrate that the proposed framework achieves competitive or superior performance compared with several representative metaheuristic
algorithms, as validated by Wilcoxon rank-sum and Friedman statistical tests. In addition, the method shows strong performance on constrained engineering design problems. Finally, the proposed framework is applied to
UAV path planning, formulated as a constrained optimization problem, demonstrating its effectiveness and scalability in complex engineering scenarios.
Read the full abstract ▾
Optimization and Control
Optimal Deterministic First-Order Oracle Complexity for Nonconvex-Concave Minimax Optimization
Siyu Pan, Taoli Zheng and Jiajin Li
We study the deterministic first-order oracle complexity of smooth nonconvex-concave minimax optimization over a bounded convex dual domain. Let $\ell$ denote the joint smoothness constant, $D_{\mathcal{Y}}$ the diameter of the dual domain, and $\Delta$ the initial gap. We prove that every deterministic first-order
algorithm requires $\Omega(\ell^2D_{\mathcal{Y}}\Delta/\epsilon^3)$ oracle queries in the worst case to find an $\epsilon$-optimization-stationary point whenever $\epsilon\lesssim\min\{\ell D_{\mathcal{Y}},\sqrt{\ell\Delta}\}$. We then develop Tracked-FOAM, a first-order method that attains a matching upper bound, removing the logarithmic factor from previous upper bounds. Together, these results establish the optimal dependence on all problem parameters in the stated regime.
Read the full abstract ▾
Optimization and Control
Lower Bounds for Nonconvex-Concave Minimax Optimization
Qilong Wu, Zhihao Gu and Junchi Yang
We study lower bounds on the first-order oracle complexity of smooth nonconvex-concave minimax optimization. We consider objectives $f$ that are jointly $L$-smooth in the primal and dual variables $(x,y)$, concave in $y$, and whose primal value function $\Phi(x) := \max_{y\in\mathcal Y} f(x,y)$ satisfies the initial-gap condition $\Phi(0)-\inf_{x\in\mathcal X}\Phi(x)\le \Delta_\Phi$, with a bounded dual domain satisfying $\operatorname{diam}(\mathcal Y)\le D_{\mathcal Y}$. We measure stationarity by the norm of the gradient of the Moreau envelope of $\Phi+\iota_{\mathcal X}$ with parameter $1/(2L)$. We prove that any deterministic zero-respecting first-order
algorithm requires $\Omega\left(L^2D_{\mathcal Y}\Delta_\Phi\epsilon^{-3}\right)$ oracle evaluations to find an $\epsilon$-stationary point. Under an unbiased
stochastic first-order oracle with bounded variance, any
stochastic zero-respecting
algorithm requires $\Omega\left(L^3D_{\mathcal Y}^2\Delta_\Phi\epsilon^{-6}\right)$ oracle evaluations. The same lower bounds hold when $\Delta_\Phi$ is replaced by the initial primal-dual gap $\mathcal G_0$. These deterministic and
stochastic lower bounds match the corresponding upper bounds of [14] and [29], respectively, up to a logarithmic factor in the deterministic setting.
Read the full abstract ▾