Papers

This paper studies joint submodular welfare maximization and routing over graphs, where agents select items under diminishing returns and transport them through a network with congestion-dependent costs. Although welfare maximization admits matroid-based approximations and routing reduces to shortest paths under modular costs, their coupling creates supermodular interactions that break separability. We show that, for fixed routing, the objective remains submodular in the allocation variable, enabling a principled decomposition. Building on this property, we propose the Welfare-based Hierarchical Routing Algorithm (WHIRL), which alternates between tractable routing and allocation updates. Routing is initialized through its modular counterpart, while supermodular effects are modeled as bounded perturbations. The method has finite convergence guarantees and approximation bounds that depend explicitly on the deviation from modular routing. Numerical results illustrate the impact of routing-induced coupling and show that WHIRL achieves a favorable tradeoff between solution quality and computational cost.

Understanding generalization remains a central challenge in machine learning because it requires jointly considering data, architecture, and training dynamics. In this paper, we develop a theoretical framework that characterizes how these factors jointly shape generalization performance throughout training. More precisely, we study a broad class of neural networks trained under the $\ell^2$ loss by gradient descent (GD) with weight decay, and prove the convergence of GD to a neighbourhood of the global minimizers of the empirical loss. By partitioning the space based on the input data, we then decompose the population error into data error, optimization error, and prediction variation error, and bound them separately. In particular, for the prediction variation error, which measures the oscillations of the learned function, we propose (local) approximate homogeneity and derive explicit cellwise and layerwise bounds for its evolution along the training trajectory. These bounds yield two important implications: a necessary condition of improved generalization explains differences in layerwise generalization behavior; a sufficient condition describes delayed generalization and provides a theoretical characterization of grokking.

We consider simultaneous internal stabilization of square proper real-rational plants of arbitrary but fixed finite input-output dimension $n$, arranged as an affine right-coprime factor segment, under explicit nondegeneracy hypotheses. After a classical normalization, the remaining global spectral-cut constraint and the reconstruction constraints at finite poles and at infinity are encoded, without loss, as a finite-dimensional semialgebraic seed, using a function-level double-Cayley factorization. For input coefficients in an effectively presented real closed field, that seed yields an exact existence decision that does not enumerate controller McMillan degree; the decision engine is classical quantifier elimination. The same encoding, now ranging over rational radii, admissible seeds, and residual unimodular and rational Schur parameters, yields a surjective atlas of all proper real-rational common stabilizers of the segment. The results concern this structured class and are compatible with the rational undecidability of simultaneous stabilization of three general plants.

Euclidean sphere-constrained optimization problems represent perhaps the most basic class of nonconvex-constrained problems for which there exist efficient optimization algorithms for specific problem instances, such as the eigenvalue problems and learning problems arising in the recent literature. However, when and why we can efficiently optimize on the sphere using simple first-order methods is poorly understood. We introduce Riemannian Gradient Alignment (RGA) as a unifying structural condition enabling fast convergence. RGA is a parameterized directional error bound requiring a tangent vector to sufficiently align with a target solution vector. We prove that Riemannian Gradient Descent with diminishing step sizes converges linearly when RGA holds with order $r=1$, and at rate $O(k^{-1/(2r-2)})$ for $r>1$. The result applies even to tangent vector fields that are not gradients of any objective. Our main technical contribution is a family of derivative-based sufficient conditions for RGA. We prove these conditions for a range of learning problems, including multi-instance learning/max pooling, Gaussian halfspaces, single-index and generalized linear models, phase retrieval, complete orthogonal dictionary learning, and (sparse) PCA. The provided examples span distributional families that include standard Gaussian, conditionally Gaussian, and more general structured distributions. The provided guarantees cover objectives that are generally not (geodesically) convex, may be nonsmooth, and may arise from discrete distributions. These results establish tractability of problem classes where no prior error bound conditions were known, as well as unify, strengthen, and generalize several prior results.

This paper considers the design of optimal fixed-step first-order methods for high-dimensional minimization of $L$-smooth convex functions. For optimizing worst-case performance measured via suboptimality of the final function value (relative to the initial squared distance to a minimizer), we provide an algebraic proof of the optimality of the optimized gradient method (OGM) and establish its uniqueness among all fixed-step first-order methods. For the alternative measure of final squared gradient norm (relative to initial suboptimality), we prove the OGM-G method is optimal and uniquely so among fixed-step first-order methods. Finally, for the setting measuring the final squared gradient norm (relative to the initial squared distance to a minimizer), we show the recently proposed Lemniscate method is optimal and uniquely so. Our proofs rely on algebraic reductions for lower bound arguments rather than traditional information-theoretic bounds, which were previously only able to establish OGM's optimality but not uniqueness.

Full-parameter fine-tuning of large language models has substantial memory costs because backpropagation stores activations and gradients. Zeroth-order optimization avoids this by estimating update directions from loss evaluations, but existing methods require tuning a sensitive learning rate for each model and task. We propose the momentum $p$-dimensional subspace trust-region method (MpSub). At each iteration, MpSub searches within a $p$-dimensional subspace: one direction preserves historical momentum from the most recent accepted step, while the remaining directions explore via fresh random sampling. The subspace gradient is estimated by central differences, a trial step is computed from a linear trust-region model, and the trust-region radius adapts according to the agreement between predicted and observed loss reduction, eliminating the learning rate. For LLM fine-tuning, evaluations within an iteration share a minibatch, and directions are regenerated in place from seeds, using forward passes alone. For smooth deterministic objectives under unorthogonalized Gaussian directions, we bound the finite-difference error, quantify gradient energy captured by the subspace, and prove that $\lim_{k\to\infty} \|\nabla f(x_k)\|_2 = 0$ almost surely under a safeguarded radius update. Under a matched budget of 8,400 training-objective forward passes, we fine-tune OPT-125M and OPT-350M on CommitmentBank. With the same preset parameters at both model sizes, MpSub attains mean test accuracies of 0.673 and 0.690 over three seeds, matching tuned MeZO (0.685) without any learning-rate search.

We study the time-optimal control of a controllable linear system on a time horizon [0,T], focusing on the asymptotic switching density for large T. When the system matrix has only real eigenvalues, it is well-known that the number of switches is upper bounded uniformly in T; when it has complex eigenvalues, no such uniform bound exists, and the switching count instead typically grows with T. We characterize this growth for a system matrix with an arbitrary spectrum, allowing simultaneously for real eigenvalues, complex eigenvalues, and a non-trivial Jordan structure. If the dominant mode is complex, the number of switches grows at least linearly in T, with an explicit lower bound expressed via the mean motion problem and the Bohl-Weyl-Wintner formula. We illustrate the theory on a linearized aircraft pitch/altitude model, showing close agreement between the predicted asymptotic switching rate and numerically computed time-optimal controls.

This paper presents a new algorithm addressing the problem of stochastic optimization where the cost function depends on a vector of uncertain parameters with known statistics. The algorithm is parameterized so as to address various stochastic formulations spanning from Expectation-focused to Value-at-Risk (VaR) as well as Conditional-Value-at-Risk (CVaR)-focused formulations. The algorithm leverages a recently proposed gradient-based Search & Accelerate algorithm which is originally dedicated to deterministic optimization problems. The approach is based on a sequence of warm-started solutions of instances of the problem. These solutions together with a samples of other solutions belonging to the convex hull of the first ones constitute the set of admissible candidates. Among this discrete set of candidates, the optimal solution is selected with regards to a sample-based approximation of the targeted criterion. The relevance of the algorithm and its efficiency are discussed and shown using a tailored illustrative example.

City logistics addresses the design and management of efficient freight transportation in urban areas. Two-tier networks, where goods are moved by different vehicle fleets from distribution centers to customers using intermediate consolidation facilities, are particularly effective. Although studied from routing, location, and network design perspectives, the research progress is slowed by the lack of standardized, state-of-the-art benchmark data. Many studies rely on outdated and artificial or case-specific instances, limiting reproducibility, comparability, and generality of results. To address this gap, we identify general problem characteristics for two-tier city logistics, provide an instance generation framework CLInstGen that is applicable to a wide range of problem settings, and propose the first benchmark library CLLIB. The library and the CLInstGen code are open-source, allowing researchers to easily use, adapt, and generate instances for their specific applications. CLLIB is based on real-world geographies of urban regions and provides a curated collection of standardized instances for direct benchmarking of models and methods as well as statistical evaluation. A computational study with two different problem settings, namely tactical service network design on two tiers and vehicle routing with simultaneous pickups and deliveries on a single tier, confirms the versatility of both the generator and the library, and provides first numerical and managerial insights on the CLLIB benchmark instances.

Mathematical programming provides a common language for many decisions embedded in modern machine-learning (ML) and artificial-intelligence (AI) systems: selecting retrieval context, routing tokens, allocating inference compute, fitting structured predictors, protecting against distribution shift, and balancing competing objectives. However, the relevant literature is fragmented across optimization, information retrieval, recommendation, natural-language processing, computer vision, and learning theory. This paper organizes various applications under common mathematical programming paradigms: linear, quadratic, binary and mixed-integer, conic, bilevel, multi-objective, inverse, distributionally robust, submodular, and min–max optimization. We normalize the models with a mostly unified notation and, for every application, identify inputs, decision variables, a principal formulation, structural properties, solution strategies, and limitations. Across paradigms, we compare tractability, relaxation quality, decomposition, approximation guarantees, and scalability bottlenecks. The paper shows that mathematical programming is most useful not as a claim that all learning is LP or MIP, but as a disciplined interface between predictions and constrained decisions.

Controller design for physical systems directly from data is of tremendous interest in both the industry and academia. However, the research on direct data-driven control has thus far mainly focused on discrete-time linear time-invariant systems. To make the highly non-trivial step to continuous-time nonlinear systems, this paper explores an intermediate route through the framework of linear parameter-varying (LPV) systems. LPV systems can be used as a convenient surrogate for nonlinear systems to achieve systematic analysis and controller design. In this work, we generalize the LPV Fundamental Lemma for discrete-time systems towards a class of continuous-time LPV systems, achieving data-driven representations that can be used to design continuous-time LPV controllers.

The restricted augmented Zarankiewicz number \(z_L(m,n)\) yields core combinatorial lower bounds for the maximal SOS rank of biquadratic forms. All previously known infinite admissible graph families rely on \(K_{4t}\) incidence graphs, attaining an asymptotic relative gap limit of \(1/4\). This work develops a new infinite family built from \(K_{5t}\) incidence bipartite graphs with \(\mathbb{Z}_5\) cyclic labeling for block partitions. We construct valid nondegenerate intra-block and inter-block 2-edges, derive a quadratic closed-form lower bound of \(z_L\), and prove its relative gap converges asymptotically to \(2/5\). Full enumeration for \(t=1\) verifies the exact value \(z_L(10,5)=23\). Under nondegenerate and generalized \(C_4\)-free constraints, the ratio \(2/5\) is shown to be the maximal asymptotic ratio attainable under this block framework. Our results expand the library of extremal bipartite graphs and sharpen lower bounds for biquadratic SOS rank, with further open problems for general \(K_{kt}\) constructions outlined in closing.

In this paper, we consider the computation of $M$-eigenpairs of fourth order partially symmetric tensors arising from elasticity theory. First, a lifted fourth order tensor is constructed, and the original $M$-eigenvalue problem is reformulated as a parameterized generalized tensor eigenvalue problem under the $\mathbf B_{\alpha,\beta}$-normalization. Then, an exact correspondence between the two eigenvalue problems is established, which provides a procedure for computing all real $M$-eigenpairs through the proposed reformulation. Furthermore, perturbation bounds for the largest $M$-eigenvalue are derived, and the lifted reformulation is shown to preserve these bounds without introducing any additional relaxation. Finally, numerical experiments are reported to show the effectiveness of the proposed method.

We present a data-driven approach to identifying linear index-1 differential-algebraic pH systems (pH-DAEs) based on input-output measurements. In comparison to the identification of port-Hamiltonian (pH) systems, the algebraic constraint and the index condition pose additional challenges. First, we establish a structure-preserving formulation of the considered pH-DAE class and derive an implicit midpoint discretization that preserves the algebraic constraints and discrete dissipation inequality. We formulate the identification problem as a regularized least-squares minimization problem subject to the pH-DAE dynamics. Exploiting the index-1 structure, we reduce the constrained problem to an unconstrained optimization problem over the system parameters while preserving the port-Hamiltonian structure. Next, we derive an adjoint-based formulation to efficiently evaluate the gradient of the resulting reduced cost functional. This enables us to use gradient-based optimization methods for parameter estimation. Under suitable assumptions on the admissible parameter set, the existence of a minimizer is established. Numerical experiments demonstrate that the proposed approach can identify surrogate pH-DAE systems that accurately reproduce the input-output behavior of reference systems. Further investigations show the approach's potential for identifying reduced-order surrogate models. Cross-validation with independent input signals confirms the predictive capability of the identified models.

This paper studies the nondominated set for topology optimization of compliance and volume. In the multiobjective topology optimization literature, scalarization is a popular method to approximate this set, which is better known as the Pareto frontier. However, previous work indicates that the pointwise approximation obtained by scalarization obfuscates its underlying mathematical structure: under an imposed length scale, it consists of locally nondominated curve segments. To investigate this structure, the paper uses a simple methodology to (i) generate local optima with a tightly controlled topology and (ii) use continuation to extend these points to locally nondominated curves. The paper then performs two rounds of numerical experiments. The first round applies the methodology to several cantilever examples. This leads to novel insights regarding topological complexity and symmetry and confirms prior conclusions regarding continuity, smoothness and convexity. The second round investigates two numerical instabilities: (i) a lack of uniqueness and (ii) discontinuities of the locally nondominated curves. The former is linked to an observed flatness of the optimization landscape, whereas the latter is attributed to branch splitting and linked to bifurcation theory. Namely, we classify these discontinuities as higher-dimensional analogues of the subcritical, symmetry-breaking pitchfork bifurcation. The paper concludes with suggestions for future research to address these features of the multiobjective optimization landscape.

In this study, we show that a continuous-time risk-sensitive control problem is equivalent to a Rényi divergence minimization problem over trajectory path measures. Reformulating stochastic optimal control as probabilistic inference via Kullback-Leibler (KL) divergence minimization avoids the computational intractability of the Hamilton-Jacobi-Bellman equation. However, standard KL control is inherently risk-neutral, and recent minimax extensions remain restricted to risk-averse settings. Our equivalence result resolves this limitation by offering a unified probabilistic framework for arbitrary risk attitudes in continuous-time nonlinear systems. Based on Girsanov theorem, we explicitly map the risk sensitivity to the Rényi divergence order, deriving a noise-dependent control penalty scaled by risk preference. This formulation seamlessly modulates tail-weighting behaviors, interpolating between zero-forcing for risk-averse policies and mass-covering for risk-seeking policies. These findings bridge stochastic control and information-theoretic inference, providing a foundation for sampling-based control algorithms.

Pooling latent subpopulations can obscure relationships and yield misleading regression conclusions, including Simpson's paradox (SP). We propose a detector based on the steady-state dynamics of constant-step stochastic gradient descent (SGD). Unlike likelihood-based mixture tests and confounder-search methods, it requires neither a normal-mixture specification nor observed candidate confounders. Two linear pieces compete under a winner-take-all squared-error loss, and their normalized terminal separation forms the test statistic. Using diffusion approximations, we derive its asymptotic null distribution for general centered scalar covariates and for Gaussian multivariate covariates under symmetric noise. The distribution-dependent null center reduces to the dimension-free constant $4/\pi$ when both covariates and noise are Gaussian. We establish asymptotic size control and consistency against fixed mixture alternatives under regularity conditions. We extend the method to intercept and partial-mixture heterogeneity and study endogeneity, heteroskedasticity, and nonlinear misspecification. Simulations examine calibration, power, and robustness. Finally, a three-stage Detect–Screen–Verify toolkit separates evidence of heterogeneity from its substantive explanation. Across eight public datasets, it recovers four established SP benchmarks and identifies four cases that, to our knowledge, have not been documented previously. The detector requires neither latent-group labels nor a prespecified number of mixture components.

Generalized Nash Equilibrium Problems (GNEPs) often arise in multi-agent engineering applications that require distributed algorithms. Unlike traditional approaches that enforce consensus on multipliers, our method removes the need to share multipliers, reducing communication and improving privacy. As a result, different initializations can lead to different GNEs, including non-variational ones. We establish convergence under sufficient conditions using an input-to-state stability (ISS) framework.

Strongly convex minimization and its natural indefinite extension to strongly convex–strongly concave minimax problems combine quantitative control of curvature with a prescribed curvature orientation. We disentangle these two roles by retaining uniform nondegeneracy alone: curvature remains uniformly separated from zero but may have either sign, with no prescribed positive–negative splitting. Surprisingly, a large part of the familiar theory nevertheless re-emerges. We first derive an intrinsic formulation through first-order secant inequalities, making the gradient on $\mathbb{R}^d$ a global bi-Lipschitz homeomorphism and yielding a unique stationary point. We then pair signed Moreau envelopes to construct a smooth scalar merit that recovers the missing descent geometry at both zeroth and first order, and the resulting paired proximal descent method achieves global linear convergence with dimension-free first-order oracle complexity. Meanwhile, we show that this tractability can break down on restricted domains: merely assuming the existence of a stationary point in the domain may lead to the curse of dimensionality, even with access to an infinite-order oracle. To overcome this information barrier, we introduce certified feasibility, an observable localization condition that enables feasible continuation. Together, these results establish a first-order optimization theory under uniform nondegeneracy that spans geometry, computation, and information.

We study constrained online convex optimization with adversarial constraints and conditionally unbiased, finite-variance observations of constraint values and gradients. Under common feasibility, our \LEDGER\ algorithm attains $O(\sqrt T)$ expected regret and $O(\sqrt{T\log(eT)})$ expected budget violation, the largest cumulative overspend over any window. It uses a reflected exponential potential, clipped signed observations, and predictable adaptive regularization, with one feedback triple and one projection per round. Neither a Slater condition, independence between feedback channels, nor an absolute constraint-value bound is needed. A Gaussian testing lower bound proves that the budget rate has optimal horizon dependence under square-root regret at fixed positive noise, including the logarithm. The same obstruction holds for terminal violation, so the logarithm is not a cost of maximizing over windows; an $O(\sqrt T)$ budget bound instead forces linear regret. In contrast, fixed positive Gaussian value noise yields a joint regret–hard-violation lower bound of $\Omega(\min\{\sigma,1\}T/\log^2 T)$, even with exact gradients in one dimension. The hard-violation construction matches arbitrarily many moments while preserving a feasible-endpoint gap and constant endpoint probabilities. Together, the bounds separate uncertainty about hard feasibility from learnable signed budgets. Deterministic restarts remove the horizon input without changing either upper rate.

We study the first-order effects of nodal inputs on edge flows of network systems through controllability Gramians. We consider both discrete- and continuous-time dynamics subject to impulse and step input disturbances. To characterize their impact on network behavior, we introduce the notion of a vulnerability matrix (VM) and provide explicit Gramian-based expressions that reveal the critical role played by the interplay between network topology and dynamics. For the class of directed line networks, we particularize these closed-form expressions in terms of edge weights, input duration, and graph distance to the nodal input. Numerical simulations on directed line and random Erdős-Rényi networks show a tight correspondence between influential nodes identified with the VM and conventional time-domain performance metrics, such as $\mathscr{H}_2$ and $\mathscr{H}_{\infty}$.

This paper develops a spectral Koopman-Hopf framework for adversarial reachability analysis of nonlinear systems. By lifting the nonlinear drift dynamics into Koopman eigenfunction coordinates, the proposed approach transforms the original state-dependent Hamilton-Jacobi-Isaacs (HJI) equation into an approximate state-independent optimization problem in spectral coordinates. The transformed control and disturbance directions are approximated using least-squares spectral projections, and upper and lower bounds on the transformed Hamiltonian are used to derive corresponding bounds on the value function and backward reachable sets. Koopman eigenfunctions are computed using a path-integral-based Galerkin framework that avoids spatial discretization of the associated eigenfunction PDEs. Numerical examples demonstrate tractable approximation of backward reachable sets for nonlinear adversarial systems.

Without a constraint qualification, second-order cone programs can have a duality gap or an unattained dual value. For every such program with arbitrary real coefficients, we construct an exact dual using only affine equations and memberships in products of Lorentz cones. Every feasible dual point gives a valid bound. Whenever the primal is feasible with finite value, the dual attains that value. This answers a question raised by Pólik and Terlaky and reiterated in subsequent work: whether an exact dual for second-order cone programming can use only Lorentz-cone constraints. The same construction produces an affine second-order cone system that is feasible exactly when a given affine second-order cone system is infeasible, including in the weakly infeasible case. Both formulations have polynomial size and are obtained uniformly from the input coefficients using only fixed rational constants. These are formulation-size results over exact real data; they do not imply polynomial-time solvability or polynomial bounds on the bit length of rational certificates.

Convex optimization on polytopes arises in many areas of science. When the polytope is given implicitly or has exponentially many vertices and facets, standard methods may not apply or be ineffective. This is the case for moment polytopes, such as the entanglement polytopes, which play a foundational role in quantum information and algebraic complexity. They give rise to important entanglement measures and tensor parameters such as the quantum functionals, yet general effective methods for computing these quantities have been elusive. In this paper we address this challenge. We develop a first-order framework called Hadamard mirror descent to optimize suitable convex functions over moment polytopes and, more generally, the gradient sets of geodesically convex functions. It operates locally and does not rely on any explicit description of the polytope. Our framework extends mirror descent, an effective and widely used framework for convex optimization, from the Euclidean setting to Hadamard manifolds, and is motivated by a recent work by Hirai, which we interpret as a Hadamard version of mirror flow. Applying the framework to entanglement polytopes yields the first efficient first-order algorithms to compute the quantum functionals, the symmetric quantum functional, and the G-stable ranks, as well as a new direct algorithm for the non-commutative rank.

This paper proposes a novel composite boundary feedback control law that ensures input-to-state stability (ISS) for a coupled ODE-parabolic PDE system with time-varying coefficients in both subsystems. In controller design, we circumvent the need to directly solve coupled time-varying parabolic-hyperbolic kernel equations by employing an analytic pre-defined gain function and a time-varying Volterra kernel function to design the control law explicitly. In stability analysis, to address the simultaneous challenges of Dirichlet boundary disturbances and time-varying coefficients, we employ the square root of a time-varying positive definite matrix and a superlinear function to construct a nonquadratic Lyapunov function for the ODE and a generalized Lyapunov functional for the PDE, respectively, in the target system, thereby establishing the ISS in the $L^2$-norm of the closed-loop system. Numerical simulations are presented to illustrate the effectiveness of the proposed control scheme.

We investigate stochastic simple bilevel optimization with smooth and possibly nonconvex upper- and lower-level objectives. Existing stochastic extensions of dynamic barrier gradient descent (DBGD) either obtain fast convergence under an unverifiable trajectory-dependent “rare-visit” assumption, or remove this assumption at a substantially higher oracle cost. We show that a simple denominator-only regularization of the DBGD multiplier eliminates the need for such an assumption while preserving fast convergence rates. Specifically, our method achieves $(\varepsilon, \varepsilon)$-stationarity in $O(\varepsilon^{-2})$ iterations using $O(\varepsilon^{-4})$ upper-level and $O(\varepsilon^{-7})$ lower-level stochastic gradients, which improves upon the best assumption-free complexities. We additionally derive anytime parameter schedules.

The convergence rate of a free energy Wasserstein gradient flow is quantified by its so-called Polyak-Lojasiewicz (PL) constant $\lambda$, which relates the objective function to its dissipation along the flow. Such a flow is the mean-field limit as $N$ goes to infinity of a system of $N$ interacting particles, whose convergence rate in relative entropy towards its Gibbs measure is quantified by its log-Sobolev constant $\lambda_N$. Different behaviours of $\lambda_N$ as $N$ goes to infinity thus describe drastically different phenomena, such as fast relaxation or metastability, with many models undergoing phase transitions between these regimes, depending typically on temperature. Under fairly general conditions, a uniform-in-$N$ log-Sobolev constant (i.e. fast exponential convergence for the particle system) is known to induce a positive PL constant (i.e. exponential convergence for the mean-field flow). A conjecture was stated by Delgadino, Gvalani, Pavliotis and Smith according to which the converse implication was true ($\lambda>0$ implies $\liminf \lambda_N >0$), even with $\lim \lambda_N =\lambda$. First, we will prove this converse implication, although without the equality $\lim \lambda_N = \lambda$, for a general class of mean-field models. Second, we also notice that this implication fails if the free energy minimiser is not unique, and provide an explicit counter-example. Third, we also consider the same question of relating $N$-particle and mean-field inequalities in the context of more general Lojasiewicz inequalities, which correspond to polynomial (instead of exponential) convergence rates, and can describe the situation exactly at a phase transition.

We study stochastic convex optimization under asymmetric \((L_0,L_1)\)-generalized smoothness, a model motivated by machine-learning objectives whose local curvature may grow with the gradient norm. We assume an unbiased first-order oracle with additive norm-sub-Gaussian noise. Acceleration is difficult in this setting because momentum may enter regions of much larger curvature, while stochastic gradients cannot reliably certify an unrestricted trajectory. We propose ARC-SG, a two-phase accelerated method: Phase~I reduces excessively large gradients using a generalized-smoothness-aware stochastic step, then Phase~II solves strongly convex proximal subproblems by a restarted accelerated solver confined to certified smoothness balls. Exact proximal points do not increase the gradient norm, allowing these certificates to propagate through the outer loop. The contribution is a query-by-query certified-localization construction with explicit generalized-smoothness factors and a strongly convex restart extension. ARC-SG achieves, with high probability, an accelerated optimization contribution and smooth-subclass-optimal statistical dependence on accuracy, up to logarithmic and generalized-smoothness factors. Its convex accuracy exponents agree with a contemporaneous public stochastic-acceleration result under a broader smoothness and affine-variance model; our distinction is the certified geometry, explicit parameter accounting, and strongly convex guarantee. The results recover classical accelerated stochastic rates when \(L_1=0\). Experiments on objectives with unbounded gradients illustrate the two-phase mechanism and its finite-budget advantage.

Physical and synthetic models may describe complementary aspects of the same PDE-governed system while receiving different, possibly fragmented, observations. We propose Bi-Objective HYCO (Bi-HYCO), a cooperative framework that retains both representations and their local observational objectives while coupling their predicted states at unlabeled interaction points. These points contain no measurements and do not augment the data; they provide a communication mechanism in the common state space. The two criteria form a vector-valued objective, and weighted scalarizations provide computational realizations. For the deterministic shared-observation algorithm with fixed interaction points, we prove sufficient decrease and finite length of the whole alternating sequence, which converges to a mixed critical point under the stated Kurdyka-Lojasiewicz-type assumptions. Elliptic transmission and two-dimensional Navier-Stokes experiments assess parameter and state reconstruction, noise and scalarization effects, and PINN/XPINN references. Ablations show that removing state interaction while retaining aggregation deteriorates parameter recovery in the tested configurations, particularly for Navier-Stokes.

This paper presents novel extremum seeking algorithms inspired by RMSProp dynamics and based on Lie bracket approximation techniques. To inherit the behavior of the classical continuous-time RMSProp algorithm, we introduce an extremum seeking scheme that exploits second-order Lie brackets, allowing the excitation of squared gradient terms in the resulting Lie bracket system. Moreover, we propose two modified RMSProp-type algorithms that can be implemented using only first-order Lie bracket approximations, thereby reducing the complexity of the extremum seeking control design. The three resulting extremum seeking algorithms are compared through numerical simulations.