Optimization and Control
Reducing the Lipschitz Constant in Tseng Algorithm and the Convergence Analysis
Ba Khiet Le, Minh Thuan Tran, Thi Thanh Tien Pham et al. (+1 more)
In this paper, we prove the weak and linear convergence of Tseng
algorithm applied to the zero of sum of two operators problem where each operator is not necessarily monotone while existing researches in the literature require the monotonicity of both operators. Consequently it allows us to decompose more efficiently and can reduce the Lipschitz constant significantly, which is meaningful especially in big data. Numerical examples supporting the theoretical results are also provided.
Read the full abstract ▾
All authors Ba Khiet Le, Minh Thuan Tran, Thi Thanh Tien Pham, Kim Thuong Vo
Optimization and Control · Cross-listed from quant-ph
Set-Packing and Sequence-Pair QUBOs for the 2D Cutting Stock Problem on Quantum Annealing Hardware
Miguel Sánchez-Beato, Raul Martinez, Matilde Osa et al. (+2 more)
The two-dimensional Cutting Stock Problem (2D-CSP) is an NP-hard problem with direct economic and environmental impacts on manufacturing and logistics. We encode its fixed-plate variant, with free piece repetition and full non-overlap and containment constraints, as a Quadratic Unconstrained Binary Optimization (QUBO) problem for
quantum annealing and compare two formulations from opposite encoding paradigms. The first was a coordinate-based set-packing model with one binary variable per candidate placement. Its variable count grows linearly with plate area and resolution, but its ground state is, by construction, a geometrically feasible maximum-area packing. The second is a coordinate-free sequence-pair model whose variable count is independent of plate resolution and size. We prove that this compactness has a structural limit: no coordinate-free QUBO of bounded interaction degree whose penalties vanish on every geometrically feasible layout can have a geometrically feasible ground state for 2D containment, because containment is a longest-path constraint that bounded-degree penalties cannot enforce on chains longer than their interaction order. We evaluate both formulations under multi-seed simulated annealing, simulated
quantum annealing, and an exact integer-programming baseline. Hardware experiments include D-Wave minor
embedding, a calibrated direct-QPU sweep, and Leap hybrid solvers in both penalty and constraint-native form, across a six-instance campaign with per-instance calibration. The hybrid solver returns our certificate configuration, tying its energy to thirteen decimal places while overflowing the plate. We claim no
quantum speedup. Our contribution is an impossibility result characterizing the limits of compact packing QUBOs, and a practical rule for choosing between the two formulations.
Read the full abstract ▾
All authors Miguel Sánchez-Beato, Raul Martinez, Matilde Osa, Jorge Parra, Mario Calonge
Optimization and Control
Stability Analysis and Data-driven Verification via Recurrent Lyapunov Functions
Roy Siegelmann, Fernando Paganini and Enrique Mallada
Lyapunov's direct method is a cornerstone of stability and control, but it hinges on finding a Lyapunov function, a task demanding ingenuity or computation. A key difficulty is that every sub-level set of the function must be forward invariant, coupling its geometry to the system's trajectories. We relax this by replacing invariance with recurrence: a set is recurrent if every trajectory starting in it returns within a finite time. This yields the notion of a Recurrent Lyapunov Function (RLF), whose sub-level sets need only be recurrent. We show that, under mild conditions, RLFs guarantee stability, and we introduce stronger notions yielding asymptotic and exponential stability. We also give norm-based converse
theorems: under the corresponding stability conditions, any norm is an RLF for their practical versions. We then develop GPU-based
algorithms that certify (practical) stability from trajectory data alone, without a Lyapunov function. Certifying stability up to an $\varepsilon$-neighborhood needs only $O(\log(1/\varepsilon))$ trajectory evaluations, with constants growing as the certified decay rate nears the true one, exposing an intrinsic performance-cost trade-off.
Read the full abstract ▾
Optimization and Control
Integrated Multi-Modal Transit Network Design via Dynamic Line Generation
Ning Duan, Oktay Günlük and Samitha Samaranayake
The integration of fixed-route public transit and on-demand mobility services presents both a modeling challenge and a computational opportunity for large-scale network design. We propose a flow-based mixed-integer programming formulation that jointly optimizes transit line planning and service frequencies while explicitly capturing first- and last-mile connectivity via on-demand services, under a fixed operating budget. To achieve tractability at urban scale, we develop a novel column generation heuristic scheme with tailored pricing subproblems. Applied to networks and demand in Boston and Chicago, the framework yields operationally feasible designs that substantially increase demand served. Relative to transit-only and on-demand-only baselines, ridership increases by up to 20.99% and 93.58% in Boston, and by up to 10.63% and 149.84% in Chicago. Compared to a multi-modal
benchmark, our approach improves ridership by 5.42% and 5.80% in Boston and Chicago, respectively. These results demonstrate that (i) joint co-design of transit routes, frequencies, and on-demand legs within a unified optimization framework yields substantially greater ridership than single-mode or decoupled approaches under equivalent budget constraints; and (ii) the proposed formulation and column generation pricing scheme admit tractable, operationally feasible, high-performing solutions relative to tested baselines.
Read the full abstract ▾
Optimization and Control
Isoperimetric problems related to extremal diameter graphs in 3D: theoretical and numerical aspects
Beniamin Bogosel
We study optimization problems for separable functionals of the Euclidean or spherical lengths of dual edge pairs in finite extremal unit-diameter configurations in three dimensions. For a fixed diameter graph, these problems lead to nonconvex constrained optimization of the vertex coordinates. We first prove that a convergent sequence of extremal configurations retains an extremal
geometric core after coincident points are merged and vertices incident to at most one diameter are removed. A spherical Crofton argument then gives a sharp lower bound for additive concave functionals, attained by the regular tetrahedron. We also analyze the effect of inserting or deleting dangling vertices. For the sum of products of spherical dual-edge lengths, we obtain an exact supremal reformulation of the three-dimensional Blaschke–Lebesgue area problem. The numerical study uses all 10,644 available extremal configurations with at most 16 vertices. We combine direct evaluation with gradient-based local optimization on each fixed graph and reconstruct the intrinsic diameter graph and its dual pairs after vertex collisions. For every supplied graph, the selected verified endpoint contains a regular tetrahedron. These computations do not certify the global maximum for any fixed graph, but they motivate structural
conjectures connecting tetrahedral containment with the Blaschke–Lebesgue problem.
Read the full abstract ▾
Optimization and Control · Cross-listed from cond-mat.stat-mech
Information-theoretic formulation of the Traveling Salesman Problem
Enrico Maria Fenoaltea, Riccardo Piombo and Aurelio Patelli
The Traveling Salesman Problem (TSP) asks for the shortest route to visit a set of cities exactly once. It combines a simple local rule - each city must be visited once - with a hard, global constraint- all cities must be traversed within a single cycle. We cast the problem within a probabilistic, information-theoretic framework. The coexistence of local and global constraints is precisely what makes the problem difficult to address in this framework: the local rule can be enforced through vertex-level constraints, whereas the global constraint cannot be captured by independent edge probabilities. We show that this obstacle can be overcome by defining a maximum-
entropy probability distribution over graphs, in which edge costs and degree constraints generate an assignment-like ensemble, and a global term, describing the hard constraint, tilts this ensemble toward
Hamiltonian cycles. To make the construction tractable, we derive a mean-field approximation in terms of edge occupancies and implement a differentiable cycle penalty that suppresses sub-tours. This leads to a self-consistent numerical procedure whose output is not only a candidate tour but also a probability
matrix encoding competing edges and degenerate solutions. We test the method on synthetic ensembles and on TSPLIB instances. The
algorithm converges to connected tours in
polynomial time, matching the best-known solution in the majority of instances and remaining within a small relative gap otherwise. Beyond its competitive performance, the proposed framework offers a general approach for handling hard constraints while reducing hard combinatorial optimization problems to simpler ones.
Read the full abstract ▾
Optimization and Control
Directional Optimality Conditions for Optimization Problems in Asplund Spaces
Weihao Mao and Jane J. Ye
This paper develops directional necessary optimality conditions for constrained optimization problems in Asplund spaces. Under directional metric subregularity, we first derive a directional optimality condition in terms of limiting subdifferentials. We then introduce sufficient conditions for directional metric subregularity and establish their relationships with directional pseudo-normality and quasi-normality. For systems with joint constraints, we further propose a joint criterion for directional metric subregularity and use it to obtain necessary optimality conditions. The results not only extend several finite-dimensional directional constructions to an Asplund-space setting, but also yield conclusions that are new even in finite dimensions.
Read the full abstract ▾
Optimization and Control · Cross-listed from cs.RO
Dispersive Forward Tree Search for Optimal Control: Coverage, Complexity, and Computation
Shashank A. Deshpande and Jonathan P. How
Steering-based planners require solutions to state-to-state boundary value problems, which can be inaccessible for nonlinear platforms. Forward propagation evades the steering requirement, but the finite-sample behavior of the associated planners remains uncharacterized and their implementations underperform in practice. This paper develops a propagation-based kinodynamic planner with deterministic finite-sample near-optimality guarantees. We work within the large class of differentially flat nonlinear systems and show that a forward tree of locally dispersive control commands contains a near-optimal trajectory at a certified tree size. We provide a general mechanism to construct dispersive command sets for control-affine systems, which are necessary to implement the search
algorithm prescribed by the theory. We show that covering the certified trajectory class irrespective of cost provably demands a tree exponentially sized in the problem horizon, and present a cost-conditioned dominance
pruning procedure that retains near-optimality at a tree size
polynomial in the horizon. We implement the resulting search
algorithm, Dispersive Forward Tree search (DFT*), as breadth-first expansion of the forward tree, which maps naturally onto parallel hardware. We design efficient dispersive samplers for the unicycle, the trailer car, and the quadrotor and evaluate challenging planning tasks for these platforms. DFT* delivers consistently competitive and often substantially better solution quality than state-of-the-art kinodynamic planners at comparable solution times on embedded-tier processors, accelerating further as parallel compute is scaled. We also implement DFT* in a receding-horizon loop to demonstrate real-time planning in dynamic environments at embedded-tier compute budgets.
Read the full abstract ▾
Optimization and Control
Multi-Objective Planning for Healthcare Facility Resilience: Mitigate Now or Respond Later?
Gizem Toplu-Tutay, John J. Hasenbein, Matt Kammer-Kerwick et al. (+1 more)
Flooding can damage healthcare facilities, interrupt local care capacity, and trigger costly patient evacuations. Addressing these risks requires long-term resilience investments under uncertainty. We develop a bi-objective two-stage
stochastic optimization model that jointly determines permanent facility hardening and scenario-dependent evacuation decisions. The first objective minimizes expected evacuation, physical damage, and business-interruption costs. The second minimizes a service-disruption impact index that combines the duration and scale of service loss with a place-based social-vulnerability weight. We develop an exact Benders decomposition and a scalable Lagrangian-dual method with tailored primal recovery,
embedding both within an adaptive procedure for constructing informative Pareto frontiers. A case study of 3,752 hospitals and nursing homes in Texas evaluates the framework under
climate-informed tropical-cyclone flood scenarios. The results show that minimizing economic losses alone can systematically allocate less protection to facilities located in socially vulnerable areas, while moderate movement along the Pareto frontier can substantially reduce disruption impacts at limited additional expected cost. The exact decomposition solves all tested instances, including larger stress tests for which the extensive formulation becomes
memory-limited, while the Lagrangian method provides substantially faster high-quality solutions.
Read the full abstract ▾
All authors Gizem Toplu-Tutay, John J. Hasenbein, Matt Kammer-Kerwick, Erhan Kutanoglu
Optimization and Control · Cross-listed from math.NA
Physics-Informed Neural Networks for Biot's Model via Fixed-Stress Splitting and Energy Natural Gradient Descent
Kexin Sun, Qiang Liu, Minfu Feng et al. (+1 more)
Physics-Informed Neural Networks (PINNs) have recently gained considerable
attention as a mesh-free framework for solving partial differential equations. Nevertheless, their performance deteriorates when applied to strongly coupled multiphysics systems, such as Biot's consolidation model, due to severely ill-conditioned optimization landscapes. In this work, we propose a robust PINN-based solver, termed FS-ENGD-PINN, which synergistically integrates physics-based decoupling with geometry-aware optimization. Specifically, the Fixed-Stress (FS) splitting scheme is employed to decompose the coupled poroelastic system into contractive mechanics and flow subproblems, thereby significantly improving training stability and convergence. To further accelerate optimization, we adopt Energy Natural Gradient Descent (ENGD), which approximates the Newton direction in function space effectively mitigates stiffness-induced slow convergence. Moreover, to address volumetric locking arising in the nearly incompressible regime, we incorporate a three-field mixed formulation with an additional total pressure variable into the PINN framework. Extensive numerical experiments demonstrate that the proposed FS-ENGD-PINN consistently outperforms standard PINN formulations in terms of accuracy and robustness, providing a unified and reliable learning-based solver for poroelasticity across a wide range of material parameters.
Read the full abstract ▾
All authors Kexin Sun, Qiang Liu, Minfu Feng, Mingchao Cai
Optimization and Control · Cross-listed from cs.LG
Muon with Finite Newton-Schulz: The Smoothing Benefit in Nonsmooth Nonconvex Optimization
Mingyi Li and Taira Tsuchiya
Muon has emerged as a strong optimizer for the
matrix-valued parameters in large
language model pretraining, approximately orthogonalizing its momentum with a few Newton-Schulz iterations. Existing theory either replaces this iteration with the exact polar factor it approximates, or treats its finite depth as an approximation error, and thus the iteration
Muon actually runs can only hurt the guarantees. We show that finite Newton-Schulz can instead be beneficial for nonsmooth nonconvex optimization. To this end, we analyze
Muon through the online-to-nonconvex conversion, which views the update rule as an online learner and converts its regret bound into a stationarity guarantee. The finite Newton-Schulz iteration smooths the discontinuous polar map into a Lipschitz map of the singular values, and
Muon with finite Newton-Schulz can be regarded as an online learner with a smoothed spectral potential. This smoothing is exactly what the conversion needs: we prove that a Newton-Schulz depth growing only logarithmically in the target accuracy suffices for convergence to stationary points in nonsmooth nonconvex optimization, whereas
Muon with the exact-polar update may fail to converge. The resulting sample complexity bounds match the best-known guarantees for nonsmooth nonconvex optimization and are optimal for smooth nonconvex optimization up to problem-dependent factors. The argument extends beyond Newton-Schulz to general spectral maps with the same smoothing property.
Read the full abstract ▾
Optimization and Control
The Value of Human Expertise
Bradley Sturt
We consider optimization applications with unknown parameters where the decision maker believes that the optimal value of the nominal problem-the optimization problem they would have solved if the true parameters were known-is unlikely to be large. This belief derives from information that humans have that is not captured in
datasets, obtained from domain knowledge and interacting with the physical world. We propose an approach to evaluating policies that provides tighter performance guarantees if the decision maker's belief happens to be correct. Our main result shows that if computing a policy's worst-case performance is a convex program, then the value of human expertise-the maximum improvement in performance guarantees that can be obtained from the belief about the nominal problem-is equal to the minimax gap of a max-min problem. We illustrate our developments in assortment optimization and shortest path problems.
Read the full abstract ▾
Optimization and Control
A unified continuous-discrete framework for Nesterov acceleration: transitions between convex and strongly convex regimes
Xin He and Ya-Ping Fang
Classical Nesterov acceleration employs different choices of damping and inertial parameters in the convex and strongly convex settings, both for continuous-time dynamics and for discrete
algorithms. When the strong convexity parameter is small, directly using the strongly convex damping or inertial coefficient may lead to slower early-stage convergence than the corresponding convex choice, despite its favorable asymptotic exponential or linear rate. We develop a unified continuous-discrete framework that encompasses both classical regimes and provides systematic transitions between them. The resulting coefficient families retain the accelerated convex behavior at early stages while attaining the strongly convex asymptotic rate. The continuous-time dynamics arise from a two-state coupling and are analyzed within a unified Lyapunov framework that yields simultaneous $\mathcal{O}(1/t^2)$ and exponential convergence estimates, thereby recovering the classical convex and strongly convex rates. We further derive two classes of accelerated forward-backward
algorithms by discretizing the proposed dynamics and establish convergence estimates covering the convex, strongly convex, and intermediate regimes. The framework recovers the classical Nesterov inertial coefficients and generates
hyperbolic, exponential, algebraic, and
polynomial transition families. Numerical experiments demonstrate the effectiveness of the proposed methods when the strong convexity parameter is small.
Read the full abstract ▾
Optimization and Control · Cross-listed from cs.CE
Efficient tensor bases for pairwise comparisons
Konrad Kułakowski and Ryszard Smarzewski
In this study, we construct the first orthogonal basis for additively consistent subspace in pairwise comparisons theory. This construction is based on our representation of additively consistent best approximations of skew-symmetric
matrices with respect to a tensor basis having minimal support. The orthogonal basis establishes the logarithmic consistent projection for the orthogonal windowing of pairwise comparisons
matrices. It is compared with the windowing of the Saaty and SVD types. These comparisons resulted in new composite formulae for logarithmic, Saaty, and SVD projections. The theoretical considerations presented in the paper are accompanied by numerous examples.
Read the full abstract ▾
Optimization and Control · Cross-listed from quant-ph
Computing key rates for one-sided device-independent quantum key distribution
Andreas Bluhm, Gereon Koßmann, Martin Sandfuchs et al. (+3 more)
The defining feature of one-sided device-independent
quantum key distribution is its asymmetric trust model in which only one party is characterized. This scenario sets an interesting middle ground between high key rates achievable by characterizing devices and the security of full device-independence. Here, we provide new tools, methods, and
benchmarks for calculating key rates in this setting. To achieve this, we develop and compare two extensions of the NPA hierarchy and derive finite-size security bounds against general attacks with arbitrary device
memory. The latter is based on the Generalized
Entropy Accumulation
Theorem. We then investigate the performance of various protocols: the BB84 protocol both with and without losses, a qutrit mutually unbiased bases protocol, and protocols based on Bell inequalties such as CHSH and $I_{3322}$. We find that the choice of which party is characterized can strongly affects the key rate, surprisingly without a universal ordering. Our work thus offers a general toolbox for calculating key rates for one-sided device-independent QKD protocols.
Read the full abstract ▾
All authors Andreas Bluhm, Gereon Koßmann, Martin Sandfuchs, René Schwonnek, Giuseppe Viola, Ramona Wolf
Optimization and Control · Cross-listed from cs.DC
Scalable Multi-GPU Simulation of 3D Multicellular Growth with RNN-Based Workload Balancing
Matvey Moisseyev, Huijing Du, Dandan Zheng et al. (+2 more)
Detailed multicellular growth simulations based on subcellular element models (SEMs) can capture complex tissue development, but their element-level interactions impose substantial computational cost. This work presents a scalable multi-GPU framework for 3D multicellular growth simulation that combines GPU acceleration, spatial binning, domain decomposition, and workload-aware partitioning.
Cell movement, growth, and division continuously reshape the spatial workload distribution, causing initially balanced partitions to become inefficient over time. To address this, we introduce an RNN-based load-balancing controller that observes recent per-rank execution times and partition states and learns residual corrections to a reactive boundary-adjustment rule. The controller is trained offline in a differentiable surrogate of the load-balancing loop with randomized workload dynamics, requiring no measured execution traces for training. We evaluate the framework in terms of single-GPU acceleration, multi-GPU computation scaling, controller-level load-balancing behavior, and end-to-end simulation performance, with comparisons against static partitioning, reactive load balancing, and conventional time-series prediction baselines. A representative embryonic epidermal development use case further demonstrates the type of spatially and temporally evolving workload targeted by the framework. In our evaluation, GPU acceleration with spatial binning accelerates the interaction computation by roughly three orders of magnitude over a serial CPU baseline. RNN-guided load balancing reduces the mean global imbalance from 11.3% under static partitioning to 3.5%, lowers end-to-end runtime by 9.0% relative to static partitioning, and reduces slice migration by 7.7x compared with the reactive baseline, showing that history-aware control can improve workload balance while avoiding unnecessary repartitioning.
Read the full abstract ▾
All authors Matvey Moisseyev, Huijing Du, Dandan Zheng, Chi Zhang, Hongfeng Yu
Optimization and Control
Sequential Stability of the Value Function and the Solution Mapping in Berge's Maximum Theorem via Variational Convergence
John Cotrina, Raúl Fierro and Rubén López
Berge's maximum
theorem ensures the continuity of the value function and the upper semicontinuity of the solution mapping in parametric optimization problems. This
theorem plays a central role in optimization theory, game theory, and dynamic programming. Motivated by the inherent inaccuracies in optimization data, this paper investigates the stability of such problems under sequential perturbations of both the objective function and the feasible mapping. The analysis focuses on the convergence of sequences of value functions and solution mappings via variational approximations of the data. To this end, we employ lower and upper continuous, epi- and hypo-convergence notions for functions, together with lower and upper continuous and graphical convergence notions for multifunctions. In addition, we study some relationships among these types of convergence and provide examples and counterexamples associated with the corresponding notions. Our results extend and complement existing stability results in the literature. We provide applications to generalized Nash
equilibrium problems, where stability is obtained via a direct approach, as well as to finite-horizon dynamic programming models under novel perturbation assumptions.
Read the full abstract ▾
Optimization and Control
A Decomposed Bilevel Search for Variable-Metric Proximal Gradient Methods
Xinpeng Li and Ya-xiang Yuan
Variable-metric proximal methods accelerate composite convex optimization, but the scaled proximal map induced by a quasi-Newton metric rarely has a closed form. We develop Decomposed Bilevel Search (DBS), based on a diagonal-plus-rank-one factor \(X=D+uv^\top\) whose diagonal scaling satisfies a weak secant equation. The induced inverse metric \(B^{-1}=XX^\top\) recovers zero-
memory DFP/BFGS-type Broyden members, while the factor form reduces each scaled proximal step to a two-dimensional monotone residual system. Each residual evaluation requires one diagonal-metric proximal map, and a certified bilevel solve reaches target accuracy \(\epsilon\) in \(\mathcal O((d+T_p)\log^2(1/\epsilon))\) work, where \(T_p\) is the cost of that proximal map. Under strong convexity, the outer method converges linearly with exact and inexact inner solves. In the scalar specialization \(D=\alpha I\), the oracle uses only ordinary proximal evaluations of the regularizer and no generalized Jacobian or active-set information. Experiments on ordered-weighted \(\ell_1\) (SLOPE/OWL) and group-lasso logistic regression evaluate both the inner oracle and the full outer method. The oracle solves high-dimensional SLOPE scaled proximal subproblems of condition number up to \(4\times10^6\) using a few hundred ordinary proximal evaluations. In SLOPE outer
benchmarks, the warm-started oracle keeps the scaled-proximal overhead controlled and DBS reaches stringent targets with far fewer gradient evaluations than Lipschitz-normalized FISTA; on real-sim this becomes a clear target-time advantage. On group-lasso logistic regression, DBS is reliable on synthetic correlated instances and fastest on a real-data grouped-copy instance.
Read the full abstract ▾
Optimization and Control · Cross-listed from cs.LG
Beyond Optimal Rates in Stochastic Optimization: Trajectory-Adaptive Stopping Rules
Liviu Aolaritei, Lucas Lévy, Francis Bach et al. (+1 more)
Stochastic gradient descent (SGD) is typically analyzed at a deterministic horizon chosen before the
algorithm is run, even though practical stopping decisions are made adaptively by inspecting the evolving trajectory. This mismatch creates a fundamental certification problem: fixed-time guarantees do not generally remain valid at data-dependent stopping times, while deterministic horizons derived from worst-case bounds can be highly conservative. We address this problem for strongly convex
stochastic optimization by constructing fully observable, trajectory-adaptive upper confidence sequences for the squared distance of the last iterate to the optimizer and the suboptimality of a weighted average. These bounds hold simultaneously over time, attain the optimal $1/t$ decay rate up to iterated-logarithmic factors in the worst case, and adapt to the realized
stochastic gradients, allowing SGD to stop as soon as a prescribed accuracy is certified without sacrificing statistical validity. Our approach treats the evolving SGD trajectory as a sequential experiment whose observations provide evidence about the unknown optimization error. To formalize this perspective, we develop new recursive confidence-sequence techniques and a general time-uniform empirical Bernstein inequality for adapted processes with time-varying conditional means and predictable ranges that may grow without bound. We further extend these confidence-sequence constructions to minibatch SGD, with the empirical Bernstein bounds exploiting the realized second-moment structure within each minibatch. Numerical experiments show that the resulting stopping rules can require several orders of magnitude fewer iterations than natural deterministic horizons.
Read the full abstract ▾
All authors Liviu Aolaritei, Lucas Lévy, Francis Bach, Michael I. Jordan
Optimization and Control · Cross-listed from math.NA
Adaptive Hybrid Subspace Levenberg Marquardt Algorithm with Adequacy Monitor for Large Scale Least Squares Problems
M. Duc Hoang and Timothy J. Lewis
The Levenberg-Marquardt (LM)
algorithm is the most widely used method for solving nonlinear least-squares problems, as it combines the robustness of steepest descent with the fast local convergence of the Gauss-Newton method. However, its computational cost can become prohibitive for large-scale problems because each iteration requires solving a large damped linear system, and conventional step acceptance strategies may require repeated solves as the damping parameter is adjusted. Despite this computational challenge, many large-scale least-squares problems exhibit effective low-dimensional structure, with only a small number of parameter-space directions strongly informed by the data. We propose an adaptive hybrid subspace Levenberg-Marquardt (HSLM)
algorithm that constructs a low-dimensional subspace from complementary sources of gradient,
memory, Krylov-subspace, and randomized
curvature information and computes a spectrally damped LM step within this subspace. A distinguishing feature of the method is a deterministic adequacy monitor that quantifies how much descent information is captured by the reduced space and adaptively enriches the subspace when necessary. Step acceptance is decoupled from damping adjustment: Armijo backtracking determines the accepted step length, while the ratio of actual to predicted reduction is used solely to update the damping parameter, thereby avoiding repeated damped-system solves during step acceptance. For the HSLM
algorithm, we establish global convergence to stationarity and prove local linear and superlinear convergence. Numerical experiments on neural-network training problems show that HSLM achieves convergence behavior comparable to classical and Krylov subspace LM (KSLM) while substantially reducing per-iteration computational cost, with increasing advantages observed as the parameter dimension grows.
Read the full abstract ▾
Optimization and Control
Inertial Primal-Dual Dynamics Methods Featuring Implicit Hessian-Driven Damping for Convex Optimization Problems in Continuous and Discrete Time
Xiangkai Sun, Zeying Gao, Liang He et al. (+1 more)
This paper investigates inertial primal-dual dynamics with implicit Hessian-driven damping for strongly convex optimization problems with linear equality constraints. We first establish fast convergence rates for the objective function value error, the feasibility measure, and both the trajectory and its corresponding velocity vector. By appropriately adjusting parameters, we show that the proposed system achieves exponential convergence rates. Through implicit time discretization of the dynamical system, we derive an inertial accelerated primal-dual
algorithm for solving the strongly convex optimization problems. Using Lyapunov-based method, we show that the proposed
algorithm achieves convergence rates consistent with those of its continuous-time counterpart. We also extend the obtained results to non-smooth convex optimization case. Furthermore, we conduct numerical experiments to illustrate theoretical results.
Read the full abstract ▾
All authors Xiangkai Sun, Zeying Gao, Liang He, Kok Lay Teo
Optimization and Control
A Multi-View Coupled Tensor Decomposition for Lightweight Online Adaptive Traffic Prediction
Quan Yu, Jie Ni, Yu-Hong Dai et al. (+1 more)
Accurate online traffic prediction is essential for intelligent transportation systems, where forecasting must be performed continuously under imperfect sensing conditions. Missing observations and anomalous disturbances make this task challenging, particularly when prediction relies on a single traffic view. This paper proposes a Multi-View Coupled Tensor Decomposition (MVCTD) model for online traffic prediction from imperfect multi-view observations, such as speed, flow, and occupancy. The proposed model uses coupled tensor decomposition to build a structured latent forecasting space, in which shared spatial structures across traffic views and view-specific temporal dynamics are jointly modeled. A group sparse regularization is further introduced to capture correlated abnormal responses induced by real traffic anomalies and thus reduce their influence on forecasts. For streaming deployment, MVCTD performs iterative refinement only on the current latent tensor, while the remaining model variables are updated by lightweight closed-form steps based on summarized historical information, thereby avoiding repeated optimization over the full historical sequence. Experiments on real-world traffic
datasets demonstrate that MVCTD achieves accurate forecasts with favorable runtime under severe missingness, confirming its suitability for online traffic prediction.
Read the full abstract ▾
All authors Quan Yu, Jie Ni, Yu-Hong Dai, Xiongjun Zhang
Optimization and Control
Accelerated Gradient Flow with Endogenous Gradient-Memory Anchor: Selection and Restoring Damping
Chinedu Izuchukwu, Ernest Obini, Jen-Chih Yao et al. (+1 more)
We introduce an accelerated gradient flow with an endogenous gradient-
memory anchor and a nonlinear restoring–damping feedback. The anchor evolves from a weighted history of the gradients, while the nonlinear feedback is modulated by the squared coordinatewise displacement from the anchor. The resulting dynamics uses only first-order information from the objective function and does not involve explicit Hessian information. Under suitable assumptions, we establish global existence and uniqueness of strong solutions. A Lyapunov analysis yields the accelerated objective-value estimate $F(x(t))-F^\star=\mathcal{O}(t^{-2})$, with the nonlinear restoring–damping term contributing additional dissipation. Under similar assumptions, the primal trajectory converges strongly to a minimizer. When, in addition, the solution set $S$ is affine, the limit is identified explicitly as the Euclidean projection $P_S(z_0)$ of the initial anchor $z_0$ onto $S$. In particular, for rank-deficient least-squares problems, the dynamics selects the least-squares solution closest to $z_0$. We also establish a finite weighted dissipation estimate for the phase variable. Moreover, in coordinates where the nonlinear feedback is active, if the trajectory remains separated from the anchor over an interval, then the corresponding homogeneous phase dynamics acquires an additional
polynomial decay factor. Numerical experiments illustrate the minimizer-selection, accelerated objective-value decay, and restoring–damping mechanisms.
Read the full abstract ▾
All authors Chinedu Izuchukwu, Ernest Obini, Jen-Chih Yao, Shengda Zeng
Optimization and Control · Cross-listed from cs.CG
Sequential Euclidean connections with exponential memory: distributional performance and adversarial robustness
Pedro M. M. de Castro
Points in the unit ball of $\mathbb R^d$ are processed sequentially. Each new point $p_i$ is connected to a state $x_{i-1}$ that summarizes earlier observations, after which $x_i=\gamma x_{i-1}+(1-\gamma)p_i$, with $0\leq\gamma\leq1$. The cost is the sum of the $\alpha$-powers of the connection lengths. This constant-gain rule interpolates between the input-order path and the
star centered at the initial point. For independent uniform points, we establish the stationary insertion-length distribution and prove that it decreases in
stochastic order as $\gamma$ increases. If $d+\alpha>2$, or if $(d,\alpha)=(1,1)$, the optimal constant parameter satisfies $1-\gamma_N^*=\Theta(N^{-1/2})$, with an explicit asymptotic constant and closed bounds. For $\alpha=1$, its leading expected tree length equals that of the center
star and is eventually smaller than the expected lengths of both endpoint constructions. For $\alpha=2$, the optimizer is unique and characterized exactly. For the same $N$, choosing $1-\gamma_N$ as a fixed positive multiple of $N^{-1/2}$ gives a sharp two-term expansion of the expected uniform-input cost and a maximal
adversarial mean cost of $1+O(N^{-1/2})$. For every fixed $0\leq\gamma<1$ and $0<\alpha\leq3$, the exact asymptotic
adversarial value is $(2/(1+\gamma))^\alpha$. When $d\geq2$, exponential weighting is within a factor smaller than $1.161^\alpha$ of the best fixed nonnegative weighted rule with the same average look-back, for $0<\alpha\leq3$. Comparison with the running mean highlights its time-homogeneous update, stationary coefficient profile, and fixed effective
memory.
Read the full abstract ▾
Optimization and Control
Schrödinger Bridges over Kinetic Swarming Models
Asmaa Eldesoukey, Md Zulfiqur Haider, Italo Napolitano et al. (+2 more)
Paradigmatic interaction models explain how collective behaviors can emerge in complex systems from interactions among the constituent
agents. In bio-inspired swarms, however, interactions alone may not suffice to bring the population to a desired aggregate configuration within a prescribed time horizon, as needed in applications ranging from targeted therapy to collective transport and emergency evacuation. In the present work, we consider finite-horizon minimum-energy collective steering for inertial swarms that are subject to
stochastic disturbances. We focus on the mean-field representations of these multi-
agent systems driven by Cucker–Smale
alignment or Morse attraction–repulsion interactions. Our objective is to steer the swarm between prescribed endpoint distributions using a state-feedback control, where the endpoint specifications can be full phase-space distributions (positions and velocities) or position marginals alone. Our formalism is rooted in the theory of Schrödinger bridges, which has inspired contemporary developments spanning statistical inference, biological modeling,
stochastic control, and generative learning. Within the bridges framework, the uncontrolled interacting
stochastic dynamics are viewed as a prior model, and the optimal control as the minimum-energy corrective drift needed to realize the prescribed distributions. We derive nonlinear, coupled necessary optimality systems with a time-symmetric structure reminiscent of classical Schrödinger bridges, and propose nested fixed-point schemes to numerically solve them. Numerical examples show that the obtained optimal control (corrective drift) can dynamically exploit or counteract the interaction forces, depending on whether the latter are favorable or
adversarial to the steering task.
Read the full abstract ▾
All authors Asmaa Eldesoukey, Md Zulfiqur Haider, Italo Napolitano, Yongxin Chen, Abhishek Halder
Optimization and Control
Optimal Control in Hilbert Complex Spaces with Finite Element Exterior Calculus
Farid Bozorgnia, Michael Holst, Anshu Kumar et al. (+3 more)
We develop a framework for PDE-constrained optimal control on
Hilbert complexes and its structure-preserving discretization by finite element exterior calculus (FEEC), with emphasis on problems where nontrivial
topology creates physically meaningful global modes. We separate the gauge-fixed local state from harmonic components representing global circulation or flux, and introduce finite-dimensional topological actuators whose reachable modes are determined jointly by domain
topology and actuator rank. We establish well-posedness, weak optimality conditions, reachability results, and FEEC error estimates preserving the harmonic structure. Numerical experiments on contractible and multiply connected domains demonstrate circulation and cavity- flux control, rank-dependent reachability, mesh-independent optimization, and
topology-induced singular convergence. We further extend the framework to time-dependent Maxwell control, where harmonic electric circulation and
magnetic flux become dynamical states exhibiting explicit conservation and reachability properties.
Read the full abstract ▾
All authors Farid Bozorgnia, Michael Holst, Anshu Kumar, Vyacheslav Kungurtsev, Martin Licht, Gantumur Tsogtgerel
Optimization and Control
Approximating Energy-Regulation Feasible Region of Virtual Power Plants: A Data-driven Inverse Optimization Approach
Ruike Lyu, Hongye Guo and Qixin Chen
System operators will probably allow virtual power plants (VPPs) to submit their feasible region (FR) for
market clearing and dispatch. A VPP needs to determine its FR to submit as a whole based on the individual operation model of its internal distributed energy resources (DERs), which is an FR aggregation problem. Existing FR aggregation approaches rely on analytical methods, which have issues with generality and adaptability. In this paper, we propose a data-driven approach to approximate the energy-regulation FR of VPPs. It adopts the virtual battery model to approximate the aggregate FR of a VPP and determines the model parameters through inverse optimization based on generated multi-scenario operation data using the original operation model. Numerical tests verified the accuracy of the proposed method. We believe that our work helps to better leverage the flexibility of DERs.
Read the full abstract ▾