Numerical Analysis · Cross-listed from stat.ME
A general-purpose sensitivity method for multiple simultaneous parameter perturbations in stochastic reaction networks
David F. Anderson and Jingyi Ma
Stochastic reaction networks are continuous-time Markov chain models for interacting populations, with applications in biochemistry, epidemiology, ecology, and related areas. We study finite-difference sensitivity estimation when a single
estimator requires several nearby parameterized paths. Existing variance-reducing couplings are typically pairwise, so that repeated use is either inefficient or requires application-specific choices in multi-path settings. We introduce the multi-path stacked coupling (MSC), a space-time Poisson construction that jointly generates any finite collection of parameterized paths. Each pairwise marginal of MSC has the same law as the corresponding split coupling pair, allowing existing variance bounds to transfer directly; in finite-state settings, we also obtain first-order expansions for the mean and second moment of finite-difference numerators. We apply MSC in three settings of practical importance: estimating many first derivatives simultaneously, estimating a single first derivative using a wider finite-difference stencil, and estimating higher-order derivatives. Numerical experiments on a processive phosphorylation network demonstrate strong performance in each of the three application areas considered, consistent with the theoretical advantages of MSC: across all three applications, MSC achieves the smallest root mean square error (RMSE) among the methods considered over the tested computational budgets.
Read the full abstract ▾
Numerical Analysis
Error attribution in gradient random walk methods for parabolic equations
Stephen Abkin and Prabir Daripa
Gradient random walk (GRW) methods represent a solution's spatial derivative by weighted particles and recover the solution by cumulative summation. We examine their accuracy for the one-dimensional heat, scalar cubic reaction-
diffusion, and Burgers' equations. Thirty-seed ensembles, paired reconstructions, and deterministic controls identify the sources of two apparent accuracy limits. For the heat equation, comparing cumulative bin sums at bin centers introduces a first-order
alignment error, which the exact reflected-particle distribution predicts and bin-edge comparison removes. An identity for the expected squared error predicts the remaining floor and the particle count for a target root-mean-square accuracy. With unequal weights, a count chosen in advance gives an error within 1% of its prediction. Reaction-front position, speed, and shape errors decrease under refinement. For Burgers' equation, exact transformed boundary data reduce the deterministic Cole-Hopf error about 490-fold, while fixed-endpoint particle runs approach their deterministic reference.
Read the full abstract ▾
Numerical Analysis
Two-level domain-decomposition AdaGrad method for scalable training of graph neural networks
Laurynas Varnas, Julien Herrmann, Alexander Heinlein et al. (+2 more)
Graph neural networks (GNNs) have emerged as a powerful framework for learning from graph-structured data. However, their efficient training remains challenging, particularly in distributed computing environments. This challenge arises from the use of message passing, which couples all graph nodes, leading to expensive optimization steps, high
memory requirements, and substantial communication overhead. To alleviate these limitations, we propose a novel domain-decomposition (DD) variant of AG2m, an AdaGrad method enhanced with second-order
curvature information and momentum, denoted by DD-AG2m. The proposed DD-AG2m alternates between AG2m optimization on the original (global) graph and AG2m optimization on the partitioned graphs. To incorporate global information at reduced cost, we further introduce a two-level variant (2DD-AG2m) that performs global optimization steps on a coarse graph obtained by randomly subsampling nodes within each subdomain. Numerical experiments spanning graph classification, node-level regression, and spatiotemporal forecasting tasks demonstrate that the proposed DD methods reduce the computational cost required to achieve the same predictive performance by a factor of 4-8. Moreover, for the fixed computational cost, they improve the predictive performance of GNNs by up to 22% compared with the baseline AG2m.
Read the full abstract ▾
All authors Laurynas Varnas, Julien Herrmann, Alexander Heinlein, Serge Gratton, Alena Kopaničáková
Numerical Analysis
Optimal compression of kernel matrices by interpolets with application to high-dimensional approximation
Helmut Harbrecht and Lucio Antonio Rosi
We consider the compression of kernel
matrices on the unit interval $[0,1]$ by interpolets that have sufficiently many vanishing moments. We define a compression rule which discards most
matrix coefficients without compromising the accuracy offered by the underlying discretization. Since interpolets can be scaled such that the compressed kernel
matrices are well conditioned, we derive a fully discrete scheme that solves a kernel interpolation problem under consideration in linear overall complexity. We finally generalize this approach to the unit $n$-cube $[0,1]^n$ by means of the sparse grid combination technique. Numerical experiments are carried out to validate the theoretical findings.
Read the full abstract ▾
Numerical Analysis · Cross-listed from math.ST
Non-asymptotic Analysis of Matérn Regression: The Roles of Target and Kernel Lengthscales
Daniel Sanz-Alonso
Theoretical guarantees for kernel regression are typically formulated in terms of smoothness, but practical accuracy depends critically on how the design resolution compares with the target and kernel lengthscales. We develop a finite-sample theory for Matérn regression on periodic domains with quasi-uniform designs, covering noiseless interpolation and noisy kernel ridge regression. Our minimax result shows that accurate recovery requires a design dense enough to resolve the target lengthscale and sufficient information at that scale to overcome noise. We prove that Matérn interpolation additionally requires the design to resolve the kernel lengthscale: if the kernel is too short relative to the point spacing, an additional error remains even when the target is well resolved. For noisy Matérn regression, we derive a three-term fixed-ridge risk characterization consisting of target-scale bias, kernel-scale bias, and variance, and show that optimizing over the ridge parameter yields four distinct contributions. When the target is no more than twice as smooth as the kernel, choosing a kernel lengthscale longer than the target does not worsen the oracle risk, whereas choosing one too short can. Thus these resolution conditions determine when accurate recovery becomes possible, while smoothness determines how rapidly the error decreases thereafter.
Read the full abstract ▾
Numerical Analysis
Quasi-optimality of adaptive FEM for optimal control problems involving Dirac measures governed by biharmonic equation
Asha K. Dond, Neela Nataraj and Subham Nayak
This article establishes the quasi-optimality of adaptive nonconforming finite element methods for a class of optimal control problems involving Dirac measures governed by the biharmonic equation. The nonconforming Morley finite elements are employed for discretising both the state and adjoint variables. A modified right-hand side through a companion operator that maps Morley finite elements toaconformingspacehelpstoovercomethechallengeinhandlingpointsourcesontheright-hand side. A priori and a posteriori error estimates for the optimal control problems are derived. Further,optimal convergence rates for adaptive finite element methods are established using an axiomatic framework: by proving key properties such as stability, reduction, discrete reliability, and quasi-orthogonality. Numerical experiments for three types of optimal control problems are discussed extensively and they validate the theoretical results.
Read the full abstract ▾
Numerical Analysis
A Jin–Xin Relaxation Gradual Convergence Method for Conservation-Law PINNs
Emmanuel Lorin, Yuchong Tang, Xu Yang et al. (+1 more)
The Jin–Xin relaxation of a nonlinear
hyperbolic conservation law introduces a relaxation parameter that controls the width of the internal layer resolving a shock; the discontinuity of the limiting conservation law emerges only in the singular limit as this width vanishes. Physics-informed neural networks (PINNs) use smooth network approximations and are therefore not well suited to this limit, while relaxation PINNs with a fixed parameter resolve only a single scale and cannot follow the multiscale transition toward the limiting solution. We propose the Jin–Xin relaxation gradual convergence method (JXRGCM), which treats the relaxation parameter as a continuation variable, annealing it to zero along a schedule and warm-starting each stage from the previous one, so that the approximation follows the relaxation profile through progressively sharper scales. Under the sub-characteristic condition we establish a stability estimate whose constant is independent of the relaxation parameter; combined with the relaxation limit, it yields for scalar conservation laws an $L^2$ convergence rate of $\mathcal{O}(\varepsilon^{1/4})$ toward the
entropy solution. Numerical experiments on the Burgers equation, the shallow-water dam-break problem, and the Sod shock tube show that JXRGCM improves shock and rarefaction resolution compared with fixed-parameter relaxation PINNs and other physics-informed approaches.
Read the full abstract ▾
All authors Emmanuel Lorin, Yuchong Tang, Xu Yang, Yi Zhu
Numerical Analysis
Stability analysis of consistent splitting implicit-explicit multistep methods up to ninth-order accuracy for incompressible flows
Yuanyuan Kang, Hong-lin Liao and Guidong Liu
This work presents a concise, unified stability theory of high-order decoupled \lan{implicit-explicit linear multistep (IELM)} methods based on the well-known consistent splitting technique for the incompressible Navier-Stokes equation. With the help of the recent semi-generating function approach and the global discrete energy analysis, one can establish the unconditional stability of a consistent splitting IELM method with respect to the $\ell^{\infty}(H^1)\cap \ell^{2}(H^2)$ norm if the associated implicit-explicit controllability intensity is larger than $\sqrt{2}/2$, a constant determined by the Stokes pressure estimate. It is shown that the $\beta$-parameterized GBDF-$\rmk$ ($2\le \rmk\le5$) schemes and $\gamma$-parameterized SIELM-$\rmk$ ($2\le \rmk\le9$) schemes can fulfill this requirement of implicit-explicit controllability intensity by choosing proper parameters so that they can theoretically maintain the unconditional stability of the associated consistent splitting IELM methods. Numerical experiments are also included to support our theory.
Read the full abstract ▾
Numerical Analysis
An Efficient IMEX-SDIRK2 mr-ccSAV Scheme for the Forced Navier–Stokes Equations with Uniform-in-Time Enstrophy Bounds
Honglin Liao, Haifeng Wang and Xiaoming Wang
We propose and analyze an IMEX-SDIRK2 mean-reverting concurrent-correction scalar auxiliary variable (mr-ccSAV) method for the forced two-dimensional periodic Navier–Stokes equations in vorticity form. The viscous term is treated by Alexander's SDIRK2 method and advection explicitly. Each stage requires two elliptic solves with the same shifted
Laplacian and the solution of either a cubic or a linear scalar algebraic equation. For initial vorticity in $\dot L^s(\Omega)$, $s>2$, a stage solution exists for every positive time step; uniqueness is established separately under an explicit small-step condition. The principal result is a direct, unconditional uniform-in-time enstrophy bound for arbitrary positive time steps. For persistently bounded forcing, this estimate is absorbing: the influence of the initial data decays, and the forcing contribution does not accumulate in time. Under additional regularity, uniformly bounded step sizes, and bounded neighboring step ratios, we also establish uniform-in-time $H^1$ and $H^2$ vorticity bounds without a small-step condition. For smooth solutions, the method converges optimally at second order. Numerical experiments confirm its accuracy, long-time robustness, and effectiveness of a companion embedded time-step selector.
Read the full abstract ▾
Numerical Analysis · Cross-listed from math.OC
Upper Hölderian with Explicit Exponent of Solution Mapping with Applications to Ball Constrained Least Squares Problems
Yu Wang and Shenglong Hu
In this paper, we propose an extension of the well-known Robinson implicit function
theorem for generalized equations from the upper Lipschitzian case to the upper Hölderian case. Explicit exponents dependence between the generalized equation and its linearization is determined. Applications to ball constrained least squares problems, including linear least squares and separable nonlinear least squares, are studied. In particular, we establish that the solution mapping of ball constrained linear least squares under linear perturbation is locally upper Hölder continuous with exponent $1/3$, which is of independent interest. Ultilizing the upper Hölderian version of the implicit function
theorem, we show the local upper Hölderian of the solution mapping of parametric ball constrained linear least squares.
Read the full abstract ▾
Numerical Analysis
On the extreme eigenvalues of the Gram Matrix in Physics-Informed Neural Networks for the Poisson Equation
Bangti Jin and Longjun Wu
The smallest and largest
eigenvalues of the Gram
matrix induced by the differential neural tangent kernel (DNTK) play a pivotal role in the analysis of over-parameterized PINNs trained by gradient type
algorithms. However, a theoretical analysis of the extreme
eigenvalues remains completely absent due to the challenge posed by the presence of multiple differential operators. In this work, we provide explicit lower and upper bounds for the extreme
eigenvalues of the infinite DNTK
matrix for the Poisson equation with the Dirichlet boundary condition for two-layer RePU neural networks without the bias term. The setting is fairly general with respect to the sampling points and input dimension \(d\): \(\delta\)-separated and additionally \(d\geq 3\) when deriving the lower bound of the smallest
eigenvalue. These results extend that for the neural tangent kernel, and to the best of our knowledge, represent the first results on the spectrum of the DNTK.
Read the full abstract ▾
Numerical Analysis · Cross-listed from cs.IT
Optimal Condition Numbers in Low-Rank Positive Semidefinite Matrix Sensing
Mingxuan Sun and Zhiqiang Xu
In this paper we focus on the stability of positive semidefinite
matrix sensing maps $\Phi_{\mathcal{A}}(X)=(\langle A_i,X\rangle)_{i=1}^m$ where $A_i\succeq 0$, $X\succeq0$ and $\operatorname{rank}(X)\le r$. We introduce the bi-Lipschitz constants of $\Phi_{\mathcal{A}}(X)$ and define the global condition numbers as the ratio of upper and lower Lipschitz constants. We give deterministic universal lower bounds for these condition numbers, that depend only on the rank $r$ and on the underlying field. We then investigate the random rank-one Gaussian measurements and show that our lower bounds on condition numbers are asymptotically sharp, and therefore the random rank-one Gaussian measurements are asymptotically optimal. As an application, we derive the stability guarantees for an $\ell_1$-residual PhaseLift-type
estimator at the optimal sampling scale.
Read the full abstract ▾
Numerical Analysis
A New Generalized Low-Rank Cholesky Factor ADI Algorithm for Large-Scale Stein Equations
Umair Zulfiqar
The low-rank alternating direction implicit (ADI) method is an efficient solver for large-scale Stein equations with low-rank solutions. This paper shows that, as in the continuous-time Lyapunov equation case, the low-rank Cholesky factor ADI (LRCF-ADI) method for Stein equations implicitly performs $\mathcal{H}_2$-pseudo-optimal model order reduction for discrete-time systems. This observation leads to an automatic shift-generation strategy, allowing LRCF-ADI to select subsequent shifts without user intervention. The standard LRCF-ADI method requires shifts outside the unit circle. We generalize the method to allow shifts anywhere in the complex plane, including on the unit circle. This extension enables numerical integration for frequency-limited Stein equations by interpolating the integrand at points on the unit circle. It also enables non-intrusive, data-driven balanced truncation and frequency-limited balanced truncation using experimentally measurable transfer function samples on the unit circle, without requiring access to a state-space realization. Numerical results for large-scale models demonstrate the effectiveness of the proposed methods as low-rank Stein equation solvers and data-driven model order reduction methods.
Read the full abstract ▾
Numerical Analysis · Cross-listed from math.DS
A Hybrid Physics-Informed Neural Network Framework for Computing Dispersion Relations of SH Waves in Generalized Hetrogeneous Layered Media with Applications
Subhajyoti Sarkar and Santimoy Kundu
This work presents a mathematical and computational framework for computing the dispersion relations of shear horizontal (SH) waves in continuously varying heterogeneous layered structures. The approach isolates the contribution of the heterogeneous layer from the complete dispersion relation, learns this contribution using a physics-informed neural network (PINN) and subsequently incorporates the trained model to determine the complete dispersion relation. The mathematical properties of the discretized problem are investigated, including the singularity of the finite-difference system and the oscillatory behavior of the layer solution, while a generalization-error estimate is established for the PINN approximation. The framework is first tested on a seismological configuration consisting of a heterogeneous sandstone layer over a granite half-space, where exponential heterogeneity is considered with independent variation rates in shear modulus and density. The proposed approach is validated against analytical solutions in special cases, while for general configurations the Haskell
matrix method demonstrates convergence toward the continuously varying dispersion relation predicted by the PINN as the number of homogeneous sublayers increases. Parametric studies further confirm consistency with the underlying physics. An important feature of the method is that the heterogeneous-layer equation can be trained independently and reused in multiple settings. To demonstrate this, the same trained network is coupled with piezoelectric and piezomagnetic substrates governed by fundamentally different physical laws, highlighting the potential of the PINN framework as a reusable computational module for dispersion analysis in heterogeneous layered media.
Read the full abstract ▾
Numerical Analysis · Cross-listed from cs.LG
Gaussian process learning with flow map refinement for parameter estimation in dynamical systems
Yue Hao and Dongwei Ye
Parameter estimation is a central task in data-driven learning of dynamical systems. It aims to recover the underlying physical parameters from observed time-series data, thereby providing interpretable insights into the physical mechanisms governing the system. Gradient/derivative matching methods based on Gaussian process provide an efficient way to perform parameter estimation. Those methods avoid repeated numerical integration and enforce local derivative consistency. However, such local matching may result in global inconsistency with the governing flow map, particularly under scarce and noisy observations. To address this limitation, we propose a framework based on Gaussian process learning with flow map refinement (GPL-FMR), a two-stage parameter estimation framework. The first stage is based on Gaussian process learning
algorithm and the posterior obtained from which is transferred as an informative prior to the second stage based on flow-map refinement. The second stage further improves the parameter estimation via optimisation based on global dynamical constraints. We demonstrate and analyse its performance on multiple numerical examples, including the Van der Pol oscillator, the Lotka-Volterra model, and the Lorenz-63 system. The results show that the proposed framework consistently improves parameter estimation accuracy, particularly under scarce and noisy observations.
Read the full abstract ▾
Numerical Analysis
Sharp CFL stability and temporal-dispersion optimization of symmetric splitting schemes for time-domain Maxwell equations
Hui Duan, Hongliang Li and Lunzhong Guo
We analyze coefficient design in a one-parameter family of explicit palindromic electric–
magnetic splittings for the time-domain Maxwell equations. After fourth-order staggered spatial discretization, the
Fourier amplification
matrix depends on the single scalar $g_2=a(1-2a)/2$. We prove that $a=1/4$ is the unique real coefficient maximizing the spectral CFL interval, with threshold $s_*=12/(7\sqrt d)$. We then identify a real-coefficient obstruction to higher phase accuracy: cancellation of the leading temporal phase defect requires $g_2=1/12$, whereas every real member satisfies $g_2\le 1/16$. The resulting complex-conjugate coefficients give fourth-order temporal phase accuracy for each fixed semidiscrete
Fourier mode and have threshold $6\sqrt3/(7\sqrt d)$, while the complete field update remains globally second order in time. For real Maxwell data, the physical output is the real projection of the complex trajectory; this projection is branch independent and preserves the second-order error bound. We further give an exactly equivalent doubled real-arithmetic realization, which clarifies the role of the auxiliary imaginary component without changing the numerical method. A semidiscrete convergence result and numerical experiments confirm the distinction between stability optimization and phase optimization.
Read the full abstract ▾
Numerical Analysis
Natural superconvergence points and asymptotic expansions for spline finite elements in one dimension
Peng Yang and Zhimin Zhang
We study the natural superconvergence points and asymptotic expansions of one-dimensional spline finite element approximations. For a spline space of degree $k$ and any smoothness $0\le\mu\le k-1$, we prove that the $s$-th derivative of the error exhibits enhanced convergence of order $O(h^{k+2-s})$ at points where $k-s$ is even, provided the mesh is symmetric within a region of size $Ch|\ln h|$ around the point. This condition is known to be optimal for the cases of low derivative order $s=0,1$; the present analysis shows that the same local condition is sufficient for all admissible $s$. Moreover, by expanding the error in Legendre
polynomials, a closure
theorem determines the leading-order Legendre coefficients (the asymptotic expansion of the error) by combining the Galerkin orthogonality with the superconvergence conditions. For $\mu=k-1$ (B-splines) and $\mu=k-2$, the Galerkin orthogonality conditions vanish and the coefficients are determined solely by the superconvergence conditions. The asymptotic expansion can be expressed through a simple antiderivative recurrence on Legendre
polynomials. The resulting
polynomial's zeros encode the complete set of superconvergence points for all derivative orders. Numerical experiments for selected $(k,\mu)$ pairs confirm the theoretical predictions.
Read the full abstract ▾