Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Mathematics Optimization and Control

math.OC Watch Optimization and Control (sign in to use)

Saturday 10 October

Optimization and Control

Duality Theory for Filter Stability in Hidden Markov Models

This article is concerned with the problem of nonlinear (stochastic) filter stability for a discrete time hidden Markov model (HMM) with finite state and observation spaces. For this purpose, a comprehensive duality theory is developed whereby the dual control system is a backward stochastic difference equation (BSDE). The duality is expressed in two inter-related forms. In its variational form, the minimum mean squared error of optimal filtering equals the value of a dual optimal control problem. In its structural form, controllability (resp. stabilizability) of the BSDE and observability (resp. detectability) of the HMM are dual properties. The dual optimal control problem supplies the Lyapunov function and the dissipation: the former is the value function (a variance), the latter the optimal running cost accumulated over the horizon. The main result is an invariance principle: under any one of three model properties—ergodicity of the Markov matrix, or observability or detectability of the HMM—zero dissipation over a finite horizon forces the Lyapunov function to vanish. A converse is also obtained: detectability is necessary. Two applications are given. The first is filter stability in chi-squared divergence: the invariance principle is applied in the manner of LaSalle, to limits of trajectories along which the dissipation vanishes asymptotically, the role of the model properties being to render the zero-dissipation set trivial. The second is a quantitative form of the invariance principle, through a Poincare constant, from which convergence rates follow. Both the model properties that govern stability and the Lyapunov function that certifies it are thus obtained from a dual optimal control problem, as in the classical theory for the linear Gaussian model.
Read the full abstract

Optimization and Control

Multilevel Monte Carlo for Zeroth-Order Stochastic Bilevel Optimization with Inexact Lower Level Solves

Variational regularization techniques are dominant in the field of mathematical imaging. The expressiveness of these methods heavily depends on the adopted regularization formulation which usually involves a number of parameters which have to be fine-tuned by the user. To circumvent the heavy computational costs associated with this parameter tuning, data-driven learning strategies have turned out to be a useful strategy. Selecting parameters by learning from data gives rise to a hierarchical (bilevel) optimization problem, which is in general computationally hard to solve. Our aim in this paper is to develop a data-efficient solution strategy for the bilevel learning problem. To that end, we propose a multilevel Monte Carlo derivative-free optimization method that couples Gaussian smoothing of the upper-level objective (parameter selection) with a hierarchy of lower-level (variational regularization) solution accuracies. Being agnostic towards the specific way the lower-level problem is solved, we derive explicit oracle complexity and computational cost statements for approximately solving the entire bilevel learning problem. Our statements are parameterized by the employed lower-level solution method, thus allowing for a direct cost accounting from the variational reconstruction method to the parameter selection problem. This enables us to give a universal end-to-end complexity analysis of bilevel learning, valid for the most important optimization routines in inverse problems, rather than for a specific choice of methods.
Read the full abstract

All authors Neil K. Chada, Mohammad S. Salehi, Mathias Staudigl, Simon Weissmann

Optimization and Control · Cross-listed from cs.RO

Training-Free Multi-Agent Cooperative Embodied Search from Uncertain Vision-Language Priors

Real-world Vision-Language Navigation (VLN) requires agents to act under partial observability, where map structure, semantic cues, and target locations may be incomplete or incorrectly inferred by foundation models. This paper presents a lightweight, training-free, and uncertainty-aware framework for cooperative multi-agent multi-target search in mapless environments. Instead of learning an end-to-end navigation policy, the framework separates high-level semantic reasoning from low-level execution through a scalable graph-optimization backbone. Multimodal Large Language Models (MLLMs) serve only as hypothesis generators, proposing semantic zones, target-location cues, and possible navigable connections. These outputs are treated as uncertain priors and further refined using online observations and graph-context evidence. Cooperative navigation is then formulated as a rolling-horizon multi-agent mixed-integer linear program with a lexicographic objective that first maximizes target-directed reward and then reduces travel distance, uncertain-edge use, and repeated visits. Experiments derived from 100 Matterport3D test cases show that the proposed framework improves target coverage and search efficiency over representative baseline methods, while shared information and joint planning enable more balanced workloads across agents. By operating on an abstract graph without relying on embodiment-specific policy parameters, the framework scales across varying numbers of agents and targets, generalizes to unseen environments, and accommodates different perception modules and low-level controllers without task-specific retraining.
Read the full abstract

All authors Fan Pu, Keshu Wu, Jiwan Jiang, Shiyu Shen, Zhiwen Fan, Zhengzhong Tu, Jiachen Li, Yang Zhou

Optimization and Control

Mean-Variance Portfolio Selection: Statistical Properties and Distributionally Robust Optimization

We study Markowitz's mean-variance portfolio optimization problem. When practically using this model, the mean vector and the covariance matrix of the assets returns often need to be estimated from the sample data. The sample errors will be propagated to the optimization output. In this paper, we consider three commonly used mean-variance models and build the asymptotic properties for the conventional sample approximations that are widely adopted and studied, by leveraging the stochastic optimization theory. We show that for all three models, under certain conditions the sample approximations have the desired consistency and achieve a convergence rate of square root of sample size, and the asymptotic variance depends on the first four moments of the returns. To address the variability and the uncertainty in the data driven portfolio optimization, we further consider the distributionally robust optimization (DRO) for the mean-variance model. For the ambiguity sets constructed by the $\phi$-divergence and the Wasserstein distance, we build the tractable reformulations for the DRO models. We conduct numerical experiments to test the asymptotic properties for the estimation. We also conduct experiments to illustrate that the asymptotic normality might not hold when the fourth moments of the returns do not exist.
Read the full abstract

Optimization and Control

An Exact Branch-Price-and-Cut Algorithm for a Truck-as-Hub Vehicle Routing Problem with Drones

Rural last-mile logistics remains challenging because of sparse demand, limited road infrastructure, and long travel distances. We study a Truck-as-Hub Vehicle Routing Problem with Drones in Rural Areas, in which multiple trucks each carry a dedicated fleet of drones. Drone launch and retrieval are restricted to predesignated passive synchronization pads, where a visiting truck temporarily acts as a mobile hub, may dispatch multiple drones simultaneously, and must wait until all launched drones return before continuing its route. We formulate a mixed-integer quadratic programming model that jointly accounts for customer delivery delays, truck return times, and operating costs. To solve the problem exactly, we transform the original truck–drone network so that each feasible route can be represented as a single path and develop a branch-price-and-cut algorithm with tailored pricing, cutting, branching, and parallel-computing procedures. Computational experiments show that the proposed algorithm proves optimality for most large-scale test cases and achieves optimality gaps of no more than 4% for the remaining cases, whereas Gurobi reaches the 1,800-second time limit on all large-scale instances when applied directly to the compact formulation. Sensitivity analyses show that drone operating costs and managerial priorities substantially affect drone utilization and customer delays, providing practical guidance for balancing service quality and operating cost in rural delivery systems.
Read the full abstract

All authors Fan Pu, Sivakumar Rathinam, Yunlong Zhang, Xiaopeng Li, Xiubin Wang, Yang Zhou

Optimization and Control · Cross-listed from cs.LG

Mixing Regularization for Average-Reward $Q$-Learning and $Q$-Planning

The average-reward setting for reinforcement learning is challenging because its Bellman operator is known to contract in span only when the underlying Markov chain is mixing. Whereas existing work typically assumes a contraction property, e.g., through environment mixing, we consider the setting where we are given a reward-agnostic $m$-step mixing policy, and we control the contraction and exploration properties of the learned policy by regularizing toward the mixing policy. We provide finite-time analysis of $Q$-learning algorithms that can be instantiated with multiple choices of regularization, with guarantees measured with respect to the regularized fixed point. Our framework exposes novel regularizer-dependent objects that control contraction, exploration, and operator bias. In the one-step setting, our rates match the best known rates for $Q$-iteration and synchronous $Q$-learning, and for $m$-step mixing we derive new rates. We also study $Q$-planning variants, and show that planning acts as time-localized learning, where the planning horizon $H$ trades off the bias from the boundary value against the sample complexity. Finally, we present numerical studies validating the analysis: (i) knowledge of a strong mixing policy improves convergence and exploration, and its advantage increases with bottleneck severity and, (ii) our theory predicts the convergence and bias rankings of regularizers from the $\alpha$-divergence family.
Read the full abstract

All authors Julien Roger, Sam Bodmer, Jian Qian, Benjamin Riviere

Optimization and Control

A Unified Kalman-Type Classification of Controllability for Linear Stochastic Systems

We provide a unified algebraic classification of approximate controllability, approximate null controllability, exact null controllability, and exact controllability for finite-dimensional linear stochastic control systems with constant coefficients and multidimensional Brownian noise. The controls may enter both the drift and the diffusion terms, and the joint diffusion-control matrix may have arbitrary rank. A finite decreasing sequence of subspaces defines a common geometric obstruction. Its vanishing characterizes the two approximate properties and exact null controllability. Exact controllability to arbitrary random targets additionally requires the joint diffusion-control matrix to be surjective. The codimension of the obstruction is given by a rank difference of explicit polynomial block matrices, yielding finite tests in the original coefficients. When the joint diffusion-control matrix is surjective, we obtain a finite Kalman-type full-rank condition for exact controllability. If the obstruction vanishes, a linear state feedback with a deterministic time-dependent gain drives every initial state to zero at any prescribed positive time with finite energy and polynomial small-time cost. An equivalent orthogonal elimination algorithm computes the rank differences with polynomial storage and cost, avoiding exponentially large block matrices. A two-noise example relates the input classification to terminal errors and control energy in penalized stochastic linear-quadratic control problems.
Read the full abstract

Optimization and Control

Risk-Constrained MPPI Tuning with Update-Aware MA-ES Racing

We formulate offline tuning of Model Predictive Path Integral (MPPI) controllers as a chance-constrained black-box optimization problem and develop an update-aware framework based on Matrix Adaptation Evolution Strategy (MA-ES). We use statistical information from noisy controller evaluations to determine when the evolutionary update is sufficiently reliable. When the recombination direction is certified, a derivative-free nonmonotone line search (NMLS) is used to refine the candidate solution, and a separate validation stage assesses the final frozen controller. We establish a worst-case arithmetic-complexity bound for both online MPPI control and offline tuning. Numerical experiments on TurtleBot 4, Jackal-like, and Ackermann models show that the proposed NMLS refinement is competitive with standard MA-ES, transfers across the three vehicle models without model-specific retuning, and improves several cost and validation measures. Both MA-ES variants also compare favorably with the tested randomized derivative-free line search (VRBBO) configuration. The results support the practical applicability and transferability of the proposed framework, while not claiming universal superiority or hardware-level safety.
Read the full abstract

Optimization and Control

Non-convex integer programming: a deterministic solution method of the large block sale problem

We study a non-convex, nonlinear integer mathematical program with a linear constraint. This problem originates in finance and consists of selling a large predetermined quantity of a financial asset over a fixed time horizon, while maximizing the total proceeds. The selling price is modeled using a deterministic penalty function capturing the market's adverse reaction to the selling flow. From a complexity perspective, the problem is in NP. We first solve the problem exactly using dynamic programming and propose several heuristics that yield tight numerical lower bounds. We then reformulate the problem into an equivalent factorable form and derive a tight convex relaxation using McCormick inequalities. This relaxation is embedded within an adapted Branch & Bound algorithm to find the global optimum in finite time. Numerical experiments show that our methods outperform several widely used generic solvers (LocalSolver, Scip, Baron, Couenne, and Bonmin) for selected instances, but do not yield tight upper bounds, beyond small instances.
Read the full abstract

Optimization and Control

Parameter-Free Adaptive Extragradient Methods for Generalized-Smooth Variational Inequalities

Variational inequalities (VIs) provide a unified framework for optimization, saddle-point, equilibrium, and learning problems, where operators need not be globally Lipschitz. The $\alpha$-symmetric generalized-smoothness condition allows local operator variation to grow with the operator norm, but existing extragradient methods often require unknown problem-specific constants. To address this issue, we propose PF-AEG, an explicit, line-search-free extragradient method whose stepsizes use quantities generated during the iterations, without prior knowledge of these constants. Under $p$-quasi-sharpness, we prove full-sequence convergence and last-iterate squared-distance rates: linear convergence for $0<p\le2$ and $\mathcal{O}(k^{-2/(p-2)})$ for $p>2$. The latter matches existing parameter-dependent rates under comparable assumptions, while the linear guarantee extends the previous $p=2$ result to $0<p\le2$. Building on the idea of PF-AEG, we develop AEG-VR, a variance-reduced adaptive extragradient method for finite-sum VIs. We prove almost-sure convergence and, under mild conditions, obtain analogous expected last-iterate rates, while retaining independence from the unknown problem-specific constants in stepsize selection. Experiments on synthetic generalized-smooth VIs and several learning problems show that the proposed methods are competitive with, and in several tested regimes outperform, existing baselines.
Read the full abstract

Optimization and Control · Cross-listed from q-fin.PM

Ex Ante and Scenario-Conditional Value of Information in Dynamic Portfolio Choice

Portfolio decisions are often made under uncertainty about expected returns, and investors revise their beliefs as new information arrives. Although richer information generally cannot reduce optimized value ex ante, the trading strategy it induces need not improve on the strategy induced by a less informative information structure in every fixed-drift scenario. We introduce scenario-conditional expected utility and value of information to evaluate strategies selected under the investor's current belief. In a finite-horizon Gaussian portfolio choice model, we compare a fixed-belief investor, a price observer, and an attentive investor who also processes an auxiliary signal under an attention capacity constraint. Ex ante, price learning and auxiliary attention are valuable, and the total value of attention increases with capacity. Scenario-conditionally, learning creates a correction benefit by reducing the initial trading-position gap relative to the full-information strategy, but also a fluctuation cost through belief revisions, so the ex ante ranking can reverse. We derive the current-belief distribution of scenario-conditional value and the associated underperformance probability, defined as the probability of negative scenario-conditional value relative to fixed-belief choice. Over sufficiently short horizons, the scenario-conditional value of learning is negative when belief error is small relative to posterior uncertainty. For a given horizon, sufficiently high attention capacity yields a higher scenario-conditional value than any fixed finite-capacity benchmark across all drift scenarios. We also derive sharp convergence rates to full-information performance under Shannon and Tsallis technologies.
Read the full abstract

Optimization and Control

A Unified Framework for Randomized Row-Action Methods for Structured Data Fitting and Constraint Geometry

We study randomized row-action methods for problems with linear observations, convex constraints, and non-Euclidean geometry. Our starting point is a unified formulation in which the data-fitting term acts in observation space, while the feasible set and the regularizer encode structure in variable space. It covers constrained least squares, nonnegative and simplex-constrained models, regularized data fitting, and equality-constrained Bregman selection problems. The structure of these problems leads naturally to two algorithmic branches. The first uses primal Bregman-proximal row steps for soft data-fitting problems. The second uses dual Bregman–Kaczmarz steps for problems with hard equality constraints. We first derive a common one-step inequality. We then give deterministic and stochastic convergence results for the soft branch under relative smoothness. For constrained least squares, the normalized row oracle has exact first- and second-moment identities. They yield an expected ergodic bound in the inconsistent case, an $O(1/K)$ bound in the interpolating case, and linear convergence of the actual iterates under a residual error bound. For equality-constrained selectors, we prove linear convergence in expected Bregman distance under a residual-to-Bregman error bound. We recall global sufficient conditions for this bound and derive local constants for the nonnegative quadratic geometry and the entropy geometry on the simplex. Numerical experiments illustrate the noisy stepsize trade-off and the local entropy selector theory, and show that entropic row-action updates can recover interpretable factors in a simplex-structured nonnegative matrix factorization problem.
Read the full abstract

Optimization and Control

Development and operational deployment of a staff rostering system using mathematical optimisation in a municipal water utility

We developed a staff rostering system for complex employment and staffing requirements at a municipal water utility and deployed it in routine operations at the Water Purification Office of Suita City Waterworks Bureau, Japan, in May 2026. We estimated that preparing a monthly roster took approximately 2-3 h after deployment, compared with 12-18 h previously. The system represents roster states, effective staffing and office assignments using binary and auxiliary integer variables and optimises the entire planning horizon. When the normal model yields no feasible solution, a second stage attempts to generate a reference roster by penalising weekly day-off shortages while retaining the other hard constraints. Outputs identify shortages and meeting-date changes applied before solving. Computational experiments used operational inputs for 14 staff over 245 days. The baseline was solved to optimality in approximately 177 s; a modified meeting-day scenario produced a reference roster with five staff-days of shortage. Independent monthly solving without history produced 16 staff-week cases of excess days off across two month-boundary weeks. This case demonstrates the practical value of optimisation-based decision support for staff rostering in municipal water utilities by showing how mathematical optimisation can be integrated into roster preparation and checking workflows with reduced preparation time.
Read the full abstract

Optimization and Control

Euler Cutting Planes for Fractional CVaR Programming

We maximise expected reward per unit of marginal tail risk, a reward-to-CVaR ratio against a baseline. The standard Dinkelbach route re-solves a Rockafellar–Uryasev LP or QP with one auxiliary variable per tail scenario at each multiplier update. Our method eliminates the multiplier iteration and keeps scenario auxiliaries out of the master, while still evaluating tail risk to generate cuts. A Charnes–Cooper/Schaible homogenisation removes the reward/risk multiplier, and the remaining convex constraint is separated on demand: each cut coefficient is an Euler allocation vector, read off a ranking and an average. The result is a single convex program, finitely terminating at the global optimum, with size independent of S — an LP in $M{+}1$ variables, or a second-order cone program in $M{+}2$ for a concave quadratic numerator. The method runs 20–100 times faster than the exact Dinkelbach formulation and only needs single-digit cut counts.
Read the full abstract

All authors Pranay Seshadri, Hubert Dzieciol, Zhe Feng, Alan Tua

Optimization and Control · Cross-listed from quant-ph

Entanglement-Capability-Aware Finite-Bond Circuit Cutting for Distributed Quantum Computing

Distributed quantum computing distributes workloads that exceed a single processor's capacity across resource-constrained components. Circuit cutting enables this decomposition, but its effectiveness depends on partition choice. We present SACE, a Simulation-Aware Circuit Partitioning framework with Entanglement-Capability-Aware Screening. EP-Cost screens partitions using a parameter-dependent entanglement-capability proxy and native implementation cost. Under reconstruction constraints, finite-bond refinement selects the candidate that minimizes the worst-subcircuit discrepancy between normalized MPS simulations at bond dimensions $\chi$ and $2\chi$. Across 14 exhaustively enumerated small-scale candidate spaces, EP-Cost-guided genetic algorithm selections attain co-optimal native quasiprobability decomposition (QPD) overhead, with a median within-circuit Spearman correlation of 0.9965 between EP-Cost and QPD overhead. At matched cut counts, EP-Cost reduces QPD overhead by $25.20\times$ and $16.89\times$ for 8- and 9-qubit QFT circuits, respectively, and improves sampled reconstruction. On 12-qubit approximate QFT, deterministic reconstruction reduces Pauli-observable mean absolute error in all six low-bond conditions and total-variation distance in five. With five cuts and 7,776 QPD terms, SACE is best on all four reconstruction metrics in four of six conditions. At 24 qubits, SACE improves exact-local QFT accuracy in all 30 instance-bond-dimension conditions against each EP-Cost-only, Kernighan-Lin, and spectral baseline. The 50-qubit QFT comparisons remain favorable as reference convergence improves. Weighted QAOA circuits provide complementary, more bond-sensitive evidence. Refinement benefits persist across three search methods; larger candidate panels improve the optimized proxy without guaranteeing lower exact-local error.
Read the full abstract

All authors Weidong Liu, Qingqing Jiang, Yufu Liu, Ruiqing He, Jiandong Shang, Hengliang Guo, Qiang Chen

Optimization and Control

A Safeguarded Accelerated Proximal ADMM Algorithm for Solving Structured Cardinality Penalized Optimization Problems

This paper considers a class of structured cardinality penalized optimization problems with nonsmooth convex loss functions, for which the proximal mapping associated with the structured $\ell_0$ penalty is generally unavailable. To overcome this difficulty, we develop a chain of exact reformulations based on variable splitting, exact penalization, and continuous relaxation, and establish rigorous equivalence relationships among these resulting models. In particular, we define a class of strong local minimizers for the original problem and show that these minimizers are equivalent to weak $d$-stationary points of the relaxation problem. Further, we propose a safeguarded accelerated proximal alternating direction method of multipliers (SAP-ADMM). We prove that the sequence generated by SAP-ADMM converges to a weak $d$-stationary point of the relaxation problem. Depending on the parameter settings and algorithmic regime, we establish $O(1/k)$ or $o(1/k)$ convergence rates for both the objective function value gap and the KKT residual. Numerical experiments on signal and image denoising demonstrate the effectiveness of both the proposed models and SAP-ADMM in terms of reconstruction quality and computational efficiency.
Read the full abstract

Optimization and Control

Consumption adjustment with mean-reverting risk premia: A singular stochastic control approach

We study a two-sided singular stochastic control problem in which changes in consumption carry utility costs and the risk premium follows the mean-reverting Kim-Omberg model. Optimal consumption is reflected between two moving barriers given by thresholds that depend on the current risk premium. These thresholds are the stopping boundaries of a Dynkin game for a two-dimensional diffusion driven by a single Brownian motion. A sharp occupation estimate shows that they are Lipschitz in logarithmic scale, uniformly in the cost of reducing consumption, and that smooth fit holds along the diffusion direction at every contact point. Outside a bounded interval of risk premia and at most two further points, the thresholds are infinitely differentiable. As the reduction cost grows, optimal consumption converges to the ratcheting policy.
Read the full abstract

Optimization and Control

Error Bounds and a Smoothing Penalty Method for Binary Orthogonal Optimization

Binary orthogonal optimization arises in semantic hashing, orthogonal binary projection, and Hadamard matrix design. Existing methods often handle binarity and orthogonality separately, so their outputs need not satisfy both constraints simultaneously and generally come without a residual-based guarantee of exact feasibility. Using an exact box-constrained reformulation, we establish local and global Lipschitz error bounds and prove that the Lipschitz exponent is optimal. In particular, when the binary orthogonal feasible set is nonempty, any box-constrained matrix whose orthogonality residual is below $1$ admits a feasible sign rounding, and the rounding distance is bounded by this residual. These bounds also yield a globally exact nonsmooth orthogonality-residual penalty over the box and a uniformly sharp local penalty threshold. We further develop a smoothing penalty method for binary orthogonal optimization (SPBO). We prove that every accumulation point is stationary for the limiting nonsmooth penalty problem. Numerical experiments on orthogonal binary projection, supervised hashing, and Hadamard completion show that SPBO substantially reduces orthogonality violations while retaining competitive objective values or retrieval accuracy.
Read the full abstract

Friday 9 October

Optimization and Control · Cross-listed from eess.SY

Deep learning from a control vantage point

This article offers an overview of recent developments in deep learning from a control vantage point. It reviews several classes of models including feedforward and residual neural networks, transformers, large-language models, and selective state space models while explaining how control theory plays an important role in understanding their training, properties, or architecture design. The article explains how residual neural networks can be interpreted as the discretization of a continuous-time control system thereby enabling the use of control theoretic techniques for its analysis. To illustrate the power of this bridge between control and deep learning, two technical results are discussed: 1) the ability of a residual neural network to approximate arbitrary continuous functions is a non-standard controllability problem that can be solved using geometric nonlinear control and 2) transformers, the key architectural component of most large language-models, can be seen the discretization of a consensus algorithm and thus large language-models exhibit consensus. Another point of contact between control theory and deep learning is the connection between backpropagation, the algorithm used to train neural networks, and optimal control. The article addresses how the older connection between these problems, established in the 80's using discrete-time models can be reconciled with the more recent approach using continuous-time models. Several open problems and directions for future research are discussed, including selective state-space models, a recent class of models inspired by the models used in control.
Read the full abstract

All authors Álvaro Rodríguez Abella, João Pedro Silvestre, Bahman Gharesifard, Paulo Tabuada

Optimization and Control

A splitting proximal method in portfolio optimization with entropic value-at-risk

Motivated by a recent paper where a convex portfolio optimization problem formulated by means of the entropic value-at-risk is solved via a primal-dual interior-point method, we propose an alternative way of dealing with such problems by means of splitting proximal point methods and making use of Lagrange duality. Numerical experiments where the new method is implemented for solving some portfolio optimization problems are presented as well. As byproducts, we also provide formulae for the projection on the epigraph of the weighted Shannon entropy, and the conjugate function and subdifferential of entropic value-at-risk.
Read the full abstract

Optimization and Control · Cross-listed from cs.LG

On the Depth Transfer of Neural Optimizers via L-Smoothness

Scaling deep learning models efficiently requires predictable performance transfer across compute regimes. Establishing reliable hyperparameter transfer allows optimal training dynamics and structural choices to be discovered on low-cost proxy models, avoiding catastrophic failures during massive-scale training. Existing depth-transfer analyses are architecture-specific and heuristic: hyperparameter transfer is only justified through toy blocks, and both rely on infinite-depth dynamics, a specific initialization, and feature-learning criteria. Building on $L$-smoothness framework of Xu et al. (2026), we understand hyperparameter transfer as how do the Lipschitz and $L$-smoothness constants of the loss scale with the depth $K$? We show that inverse-depth residual branch scaling $\alpha=\Theta(1/K)$ is sufficient to make both constants depth-independent at fixed width, uniformly over the bounded parameter domain and for arbitrary $p\to q$ geometries. Because the bounds depend on each residual branch only through its Lipschitz and smoothness constants, the argument applies to general residual networks (including branches of block depth $\geq 2$ such as transformer and ResNet blocks) without assumptions on initialization or correlations between layer weights. Since the smoothness constant controls the curvature penalty in the descent inequality, our bounds motivate a learning-rate scale without an additional depth-dependent curvature correction for updates bounded in the prescribed norm. This provides a geometric explanation of inverse-depth residual scaling and its application to the operator-norm geometries associated with AdamW, Muon, and MOGA. Learning-rate sweeps on GPT-2 (depths 2–128) and CIFAR-100 ResNets (8–32 residual blocks) confirm that Muon and MOGA exhibit stable learning-rate transfer under these prescriptions, with a common optimal rate across a 64-fold increase in depth.
Read the full abstract

All authors Maximilian Wong, Chenyu Xu, Jiajin Li, Yiping Lu

Optimization and Control

A Bilevel Market Equilibrium Model for Natural Gas Pipeline Network Operations: Convex Relaxations and Single-level Reformulations

We propose an integrated market equilibrium model for determining the spot market prices and flow rates in a natural gas pipeline network. The traditional approach solves the market equilibrium model separately from the natural gas network flow allocation problem in a sequential manner. Our proposed framework synthesizes marketing and operations by using a bilevel optimization formulation where the market equilibrium model in the upper level is connected and simultaneously solved with the natural gas network flow model in the lower level. We introduce convex relaxations for the nonconvex Weymouth pipe pressure drop equation and a surrogate for the compressor operating cost. Furthermore, we compare multiple single level reformulations of the bilevel problem based on Karush-Kuhn-Tucker (KKT) optimality conditions, Wolfe duality, and Lagrange duality conditions. Numerical studies on a small 6-node network demonstrates the efficacy and accuracy of our model and the comparison between different duality and optimality based reformulations.
Read the full abstract

All authors Novel Kumar Dey, Afrooz Jalilzadeh, Erfan Yazdandoost Hamedani, Saif R. Kazi, Anatoly Zlotnik

Optimization and Control

STREGO: A Scalable Implementation of a Trust-Region Framework for Noisy Bayesian Optimization

TREGO, a trust-region Bayesian optimization (BO) method for expensive blackbox problems, has convergence guarantees, but only for exact function values. In high dimension, its performance tends to deteriorate. This paper asks whether a trust-region BO method can be extended to handle noisy evaluations while being competitive in high dimension. We first generalize TREGO into a trust-region framework for noisy BO and extend its convergence analysis to noisy function estimates that are sufficiently accurate with sufficiently high probability. We then propose STREGO, a scalable implementation of this framework. Its global phase selects a batch of candidates by solving a bi-objective problem on the posterior mean of a Gaussian process and the integrated variance reduction. Unlike the predictive variance, this second objective does not favor boundary points merely because they are far from the data. Its local phase minimizes the posterior mean of a second Gaussian process, trained only on points close to the incumbent. STREGO is compared with TREGO, TURBO, BAXUS and NSMA-BO on synthetic and real-world problems of dimension $4$ to $124$, with and without noise. It gives the best average results in every synthetic setting tested and the best or tied-best solution on four of the six real-world problems. An implementation of STREGO is available at https://github.com/Mayousk/STREGO.
Read the full abstract

All authors Youssef Diouane, Issmail El Hallaoui, Meisseme Kadri, Amina Lamghari

Optimization and Control · Cross-listed from cs.LG

Decentralized Bandit Learning of Markov Perfect Equilibria

We study decentralized bandit learning in discounted infinite-horizon, general-sum Markov games with continuous actions. Agents observe the public state, but only their own realized rewards, and use simultaneous perturbation stochastic approximation to update their actions. Under strong monotonicity of the negative running-reward gradients and sufficiently small time steps, we prove that the fully decentralized procedure converges almost surely to the unique stationary Markov perfect equilibrium. We obtain an $O(n^{-1/2})$ convergence rate and present a natural extension to episodic finite-horizon learning.
Read the full abstract

Optimization and Control

Adaptive Utility Optimization for Stochastic Renewal Systems

This paper treats a system that performs back-to-back tasks. Each new task has a random matrix of parameters and can be processed in various processing modes. The processing mode determines the task duration and a vector of rewards (penalties are viewed as negative rewards). The reward vector can include quality, profit, or $-1$ times a penalty such as cost or energy. In wireless applications, tasks correspond to data transmissions and rewards/penalties correspond to throughput and distortion. Prior work treats the problem of maximizing the time average of a reward process subject to constraints. This paper seeks to maximize a concave function of the time average reward vector. This is difficult because it involves a multi-variable function of ratios of averages. We develop a single algorithm that does not know the probability distribution of the random task matrices. The algorithm is adaptive, meaning it responds efficiently (with optimal adaptation time) when probabilities change.
Read the full abstract

Optimization and Control

Cost-Aware Iteration Allocation for Sequential Multi-Fidelity First-Order Optimization

Optimization can be expensive when evaluating the objective and its gradient requires a large dataset or a costly simulation. A cheaper approximation (i.e., a surrogate) can provide a useful initial point, but the work spent on that surrogate must be justified by the saving on the original problem. We study how many iterations to perform on each surrogate before switching to a more accurate objective. For a given sequence of smooth, strongly convex objectives, we use convergence bounds and distances between minimizers to construct a model of the total iteration cost. We derive closed-form two-level budgets for Nesterov's accelerated gradient method and a convex reformulation for multiple levels. We also develop an online switching rule that updates the cost comparison using the gradient at the current point. We test the techniques on a logistic-regression problem using HIGGS data. The online rule reduces weighted iteration cost in all 27 tested configurations, with a median cost of 52.8% of direct optimization. The results illustrate why updating the switching decision can be more effective than fixing the budget from an initial convergence bound.
Read the full abstract

Optimization and Control · Cross-listed from eess.SY

Zero-Crossing Time Lower Bounds for the Homogeneous Super-Twisting Algorithm

High-frequency switching and chattering are well-known problems in sliding-mode control. Although the super-twisting algorithm produces a continuous control input, the sliding variable may still cross zero repeatedly near the sliding manifold. This paper studies how fast these crossings can occur and whether they can accumulate in finite time for the scalar homogeneous super-twisting algorithm. Two lower bounds on the time required to return to zero are derived, together with a bound on the time needed to complete a full excursion. These results give minimum separation times and upper bounds on the number of crossings associated with excursions above a prescribed amplitude. Although zero crossings may accumulate in finite time, the amplitudes of the complete excursions considered here must vanish and satisfy a summability condition. Thus, crossings associated with excursions above any prescribed nonzero amplitude cannot accumulate in finite time. When the perturbation-rate bound is zero, the estimates recover the scaling imposed by homogeneity. For the classical super-twisting algorithm, the square-root scaling also holds for nonzero perturbation-rate bounds. Numerical simulations illustrate the results.
Read the full abstract

Optimization and Control

Intrinsic Contraction Theory and Stabilization of Probability Measures on the Wasserstein Space

Deterministic flows can contract Wasserstein distances between probability laws without being contractive in point-state space. We develop a differential contraction theory for pushforward flows on forward-invariant subsets of the 2-Wasserstein space that quotients out the gauge freedom of the continuity equation. Weighted Helmholtz-Hodge decomposition and the Wasserstein contingent cone define the weighted horizontal tangent cone (whtc). Requiring the contraction quadratic form to be nonpositive only on the whtc yields the Intrinsic Wasserstein Contraction Condition (IWCC), which generalizes the classical contraction LMI and recovers it on Dirac measures. An almost-everywhere Euler approximation based on the superposition principle and Banach-valued Lebesgue differentiation theorem proves that IWCC is sufficient for intrinsic Wasserstein contraction. Law contraction without point-state contraction is illustrated by a rotationally symmetric shear example. Finally, a new comparison lemma yields an incremental Wasserstein ISS theorem; for Brockett's integrator, we prove exponential stabilization of a class of axisymmetric laws to the uniform law on a circle via a smooth feedback.
Read the full abstract

Optimization and Control

Null-controllable data for cascade heat equations: almost sharp spectral spaces and modal critical times

We study one-dimensional cascades of heat equations controlled through their last component. For a fixed time $T>0$, we consider the Hilbert space $\mathcal{C}_T$ of all initial data that can be steered exactly to zero at time $T$, endowed with the norm of the minimal $L^2$ control. This space can be strongly anisotropic: in explicit examples, it is incomparable with every isotropic spectral Sobolev or sub-Gaussian Gevrey scale associated with an adapted basis. For two equations, under the block Fattorini–Hautus condition, we give an almost sharp spectral description of $\mathcal{C}_T$. More precisely, in an adapted Riesz basis of generalized eigenvectors we define an explicit increasing scale $(\mathcal{V}_\tau)_{\tau\in\R}$ of Hilbert spaces such that $\mathcal{V}_{T-\varepsilon}\hookrightarrow\mathcal{C}_T\hookrightarrow\mathcal{V}_T$ for every $\varepsilon>0$. The weights of $\mathcal{V}_T$ combine heat dissipation with the loss of observation inside each double generalized eigenspace. When the coupling and control regions are disjoint, this recovers the known minimal null-control time $T^\star$ and identifies it as a transition time for controllable data: for $TT^\star$, the natural state space is properly contained in $\mathcal{C}_T$. We extend the analysis to arbitrary finite upper triangular heat cascades. The observation losses of the different directions in each spectral block may have distinct exponential orders, leading to a hierarchy of critical times rather than a single threshold. This hierarchy describes successive gains of null controllability on adapted modal subspaces and accounts for the interactions and cancellations between different coupling paths.
Read the full abstract

Optimization and Control

A Network Simplex Framework For Discrete Unbalanced Optimal Transport

The network simplex method is a classical algorithm for minimum cost flow problems formulated as linear programs, that has attracted recent interest from the optimal transport community. Indeed, discrete optimal transport is a subclass of minimum cost flow for which network simplex techniques compute exact solutions while leveraging the inherent problem sparsity. However, extending these approaches to other, nonlinear optimal transport formulations has yet to be thoroughly investigated. In this paper, we adapt the network simplex algorithm to unbalanced optimal transport. Our proposed framework is applicable to various unbalanced formulations, and can be implemented explicitly in the most common cases, including semi-unbalanced optimal transport. The resulting algorithms are memory-efficient, and perform competitively with existing approaches, as shown by our experiments on a range of transport and barycenter calculations.
Read the full abstract