Numerical Analysis · Cross-listed from cs.CE
Latent Twin Operator
Deepanshu Verma, Riley Chen and Matthias Chung
Surrogate models deployed on real physical systems rarely see data at fixed resolutions: sensor configurations vary across deployments and may evolve over time as sensing infrastructure changes. We introduce the Latent Twin Operator (LTO), a latent-space surrogate for time-evolving PDEs with a Convolutional Conditional Neural Process (ConvCNP)-style encoder and decoder that accepts a context set of $N$ sensor observations with arbitrary placement and can be queried at any resolution. A learned latent
evolution map advances the encoded state directly between arbitrary time points, decoupling temporal
evolution from the observation and query discretizations. We derive an explicit $\mathcal{O}(N^{-2/(3D)})$ rate for the context discretization error in spatial dimension $D$ under quasi-uniform refinement. We verify the predicted decay empirically on a 2D heat-equation
benchmark. Across time-dependent PDE
benchmarks, LTO achieves strong accuracy under one-step comparisons and transfers from native to coarser spatial resolutions with fixed parameters. On Navier–Stokes, its direct latent
evolution further reduces error over longer prediction horizons relative to recursive evaluation.
Read the full abstract ▾
Numerical Analysis · Cross-listed from cs.LG
Universal Approximation of Measure-to-Measure Operators by Pushforwards
Takashi Furuya, Nicholas H. Nelsen and Frank Cole
Many learning tasks map an input distribution to an output distribution. A natural way to model such an operator is to transform each input sample using a continuous function that may depend on the entire input distribution, and then take the distribution of the transformed samples. This defines a measure-dependent pushforward model and includes measure-theoretic formulations of
transformers. We ask when such models can approximate arbitrary continuous operators between spaces of probability measures. We first show that universal approximation fails when atomic inputs are allowed: some continuous measure-to-measure operators that split or redistribute atomic mass cannot be approximated arbitrarily well by deterministic pushforward models. We then introduce the uniform level set condition, which requires a continuous measure-dependent scalarization whose shrinking level set neighborhoods carry uniformly vanishing mass over the input family. This condition is satisfied, in particular, by compact families of absolutely continuous measures. On every compact family satisfying this condition, we prove that any continuous measure-to-measure operator with outputs of finite $p$-th moment can be uniformly approximated, in the $p$-Wasserstein distance, by continuous measure-dependent pushforwards. Combining our
theorem with existing approximation results for measure-dependent in-context maps yields universal approximation by measure-theoretic
transformers. We also extend the framework to continuously-varying source measures, yielding a corresponding universality result for a class of pushforward models that are closely aligned with cross-
attention architectures.
Read the full abstract ▾
Numerical Analysis · Cross-listed from math.AP
Convex counterexamples to the Schiffer and Pompeiu conjectures in dimensions two to eighteen
Jizhou Guo
We construct the first convex planar counterexample to the Schiffer and Pompeiu
conjectures: a bounded strictly convex non-disc domain with real-analytic boundary carrying a nonconstant solution of $\Delta u+u=0$ with $u=1$ and $\partial_\nu u=0$ on the boundary. Together with the higher-dimensional constructions, this gives convex non-ball counterexamples in every dimension from two to eighteen, and in dimensions twenty and twenty-one, with real-analytic boundaries diffeomorphic to spheres and indicator
Fourier transforms vanishing on the unit sphere. In dimensions $4$, $6$, $8$, $10$ and $14$, planar reductions through compact Lie
algebras of rank two use Harish-Chandra's radial part formula and Kostant's convexity
theorem. In dimensions $3$, $5$ and $7$, an axisymmetric formulation on the unit ball gives a quartic equation in weighted coefficient spaces. The exact two-sided inverse of the residual-corrected linearisation combines a Dirichlet Helmholtz inverse, a holomorphic boundary real-part problem and the material-derivative identity; it gives the planar Schiffer domain and, by one dimension-parametrised argument, the domains in dimensions $5$, $9$, $11$ to $13$, $15$ to $18$, $20$ and $21$. We also construct the first convex planar counterexample to Berenstein's
conjecture: a strictly convex non-disc domain with real-analytic boundary carrying a sign-changing solution of $\Delta u+u=0$ with $u=0$ and $\partial_\nu u=1$ on the boundary. Planar existence for this problem is due to Colbrook, Sadeghi and Stepaniants. All existence proofs in this paper are computer-assisted.
Read the full abstract ▾
Numerical Analysis · Cross-listed from cs.LG
Quasi Linear Kernel Attention with Infinite Capacity
Nicolaj Rux, Johannes Hertrich and Sebastian Neumayer
The evaluation cost of
transformers with softmax
attention scales quadratically with sequence length. Kernel
attention addresses this by replacing softmax with a more general kernel function. In this paper, we aim to identify kernels that retain the expressivity of
attention while enabling quasi linear computation. To quantify expressivity, we introduce a capacity for each kernel, measuring the maximum sequence length for which the
attention matrix can approximate the identity. A higher capacity thus indicates greater expressivity. We show that expressive kernels like softmax, Gauss, and Laplace have infinite capacity. In contrast, common quasi linear kernels, such as those derived from finite dimensional feature maps, exhibit finite capacity. As a solution, we propose additive kernels constructed from univariate spline and
polynomial exponential kernels. We prove that these maintain infinite capacity while allowing quasi linear computation via sorting. Finally, we implement additive sorting kernels efficiently and
benchmark them against modern softmax backends, demonstrating advantages for long sequences.
Read the full abstract ▾
Numerical Analysis
GAC-PINN: Geometry-Adaptive and Constraint-Enhanced Physics-Informed Neural Networks
Yanxin Zhang, Yong Zhang and Houbiao Li
For systems with steep gradients, sharp interfaces, or severe spatio-temporal coupling, Physics-informed neural networks (PINNs) suffer from spectral bias,
geometric inflexibility, and boundary constraint conflicts, which undermine accuracy and convergence. To overcome these issues, we propose a geometry-adaptive and constraint-enhanced PINN (GAC-PINN). The framework comprises four components: a gradient-driven adaptive grid mapping (AGM) for diffeomorphic point concentration with Jacobian regularization, an adaptive bandwidth hard-constraint ansatz with spatially-varying boundary transition widths, a Gaussian
Fourier feature mapping as a spectral preconditioner to further enhance high-wavenumber representation, and an operator-aware router that automatically selects the appropriate hard-constraint construction based on whether the governing PDE contains temporal derivatives. An AGM callback mechanism and a three-stage training strategy ensure stable coordination.
Benchmarks including the viscous Burgers equation, a sharp-peaked 2D Poisson problem, and the Allen-Cahn phase-transition equation show that GAC-PINN attains relative (L^2) errors of ((1.747\pm 0.450)\times 10^{-4}), ((2.868\pm 0.947)\times 10^{-5}), and ((1.756 \pm 0.712)\times 10^{-3}), respectively, consistently outperforming the baselines. Ablation studies further reveal that AGM alone yields a substantially lower error than residual-based adaptive refinement (RAR), while RAR becomes beneficial only when combined with FFM, demonstrating a context-dependent module interaction. Convergence analysis verifies rapid error reduction and saturation with increasing resolution, establishing a practical adaptive framework for high-fidelity simulation of problems with localized sharp features in applied mechanics and computational physics.
Read the full abstract ▾
Numerical Analysis · Cross-listed from cs.MS
A performance enhancement of the Payne-Hanek range reduction algorithm
Tue Ly
Range reduction plays a crucial role in the accuracy and performance of evaluating trigonometric functions, and is often the primary bottleneck for large floating-point inputs. While fast
algorithms such as Cody–Waite work efficiently over narrow intervals, the Payne–Hanek
algorithm remains the standard technique for accurate reduction across large floating-point inputs. However, many existing implementations of Payne–Hanek suffer from high latency due to heavy branching, conversion overheads, and the use of multi-word integer arithmetic, which hinders SIMD vectorization. In this paper, we analyze and present a branch-free variation of the Payne–Hanek
algorithm using only floating-point arithmetic. Our method operates directly over large double-precision inputs ($|x| \ge 2^{16}$) and is well suited to hardware with FMA instructions. We formulate the precision constraints in terms of a truncation error budget, construct a compact lookup table indexed by the input exponent, and prove that the scaled reduced argument has absolute error below $2^{-110}$ and relative error below $2^{-60}$ for every input, including worst cases. The same routine can serve both as the complete range reduction of a single-stage implementation and as the fast path of a correctly rounded one, achieving higher throughput than existing implementations and lower latency than those returning a double-double reduced argument. The
algorithm is currently implemented in the LLVM libc project.
Read the full abstract ▾
Numerical Analysis
Refined Gauss–Lobatto bounds for odd-order convexity
Tomasz Kania and Szymon Wąsowicz
For functions that are convex of higher order, classical extremalities identify Gaussian quadrature as a lower bound and Gauss–Lobatto quadrature as an upper bound for the integral. We sharpen this bracket in every odd order: if $f$ is $(2n-1)$-convex on $[-1,1]$, then the exact integral $I[f]:=\int_{-1}^{1}f(x)\,dx$ lies not merely between the $n$-point Gauss–Legendre rule $G_n[f]$ and the $(n+1)$-point Gauss–Lobatto rule $L_{n+1}[f]$, but already between $G_n[f]$ and their midpoint. This yields the certified
estimator $Q_n[f]:=\frac{3}{4}G_n[f]+\frac{1}{4}L_{n+1}[f]$, with $|I[f]-Q_n[f]|\leqslant\frac{1}{4}|L_{n+1}[f]-G_n[f]|$, and the constant $\frac{1}{4}$ is sharp. We also record a complementary phenomenon in even order: for $2n$-convex functions the Gauss–Radau endpoint rules still bracket the integral, but the midpoint of the two Radau rules cannot yield a one-sided refinement. The obstruction is explained by an odd, sign-changing Radau quadrature kernel. Finally, we distinguish numerical certification from ordinary high-order approximation: we include concrete examples from spline, moment and statistical models and report experiments for truncated powers and near-pole Stieltjes kernels, where the shape certificate remains meaningful when classical smoothness-based error constants are unavailable or severely pessimistic.
Read the full abstract ▾
Numerical Analysis
A priori error analysis for one-parameter Lagrange and POD reduced order methods
Antti Hannukainen and Vigdis Toresen
This work deals with the numerical solution of parametric linear elliptic partial differential equations using the finite element method for spatial discretization and a proper orthogonal decomposition or Lagrange subspace method based ROM for treating the parameter dependency. We study a simple one-parameter model problem and prove an error estimate for these ROMs as a function of the snapshot point set. Sub-exponential convergence with respect to the number of snapshot points is established under very mild assumptions on the point set. Exponential convergence is proven for structured point sets. In contrast to existing error analysis of Lagrange subspace methods, our estimates are not based on interpolation or perturbation arguments. Rather, we define a new approximation operator from the snapshot subspace and prove error and stability estimates for it.
Read the full abstract ▾
Numerical Analysis
Addressing Data Engineering Challenges in the Cold-Chain Sector to Reduce its Environmental Impact Through Analytics
Mathilde Marcy, Camille Fertel, Thomas Suquet et al. (+1 more)
Data analytics offer a great opportunity for organizations within the cold-chain industry to become more sustainable and reduce their environmental footprint. Yet, despite recent progress in data sciences, they still face challenges to ensure data quality and streamline data processes through adapted full-scale ecosystems, limiting their ability to perform sustainable analytics. Most data techniques and tools stem from the data-science community and are not always suitably adapted to domain specificities. For instance, nearly all data-quality frameworks are designed to individually consider entities, i.e. representations of real-world elements. However, achieving environmental impact reduction in industrial settings, particularly within the cold-chain sector, often requires modelling and studying entire systems comprised of multiple interdependent entities. This publication challenges the application of conventional data science and engineering concepts in data ecosystems within the cold-chain sector and proposes an alternative approach to sustainably address prevalent challenges and thus optimize analytical processes.
Read the full abstract ▾
All authors Mathilde Marcy, Camille Fertel, Thomas Suquet, Gérald Cavalier
Numerical Analysis
Fully Discrete Active Flux Methods for the Euler Equations: Exploring Different Reconstructions and New Multi-Dimensional Evolution Operators
Christiane Helzel, Mária Lukáčová-Medvid'ová, Amelie Porfetye et al. (+1 more)
We present new fully discrete Cartesian grid Active Flux methods for the two-dimensional Euler equations of gas dynamics. A key component of these methods is the
evolution operator used to evolve point-value degrees of freedom. Therefore, we derive several new, third-order accurate, multidimensional
evolution operators for point-value updates of the linearized Euler equations using the method of bicharacteristics. We also investigate how the reconstruction and choice of approximate
evolution operator affect the accuracy and stability of the resulting methods.
Read the full abstract ▾
All authors Christiane Helzel, Mária Lukáčová-Medvid'ová, Amelie Porfetye, Zhuyan Tang
Numerical Analysis
The Numerical Onsager-Machlup action functional for Euler discretized SDEs driven by fractional Brownian motion
Shanqi Liu, Qiqi Lian, Jinqiao Duan et al. (+1 more)
In this work, different from previous results, the explicit expression of numerical Onsager-Machlup action functional for Euler discretized SDEs driven by fractional Brownian motion is derived provided the drift coefficient and numerical reference path satisfy some suitable conditions. Then numerical fractional Euler-Lagrange equations for numerical Onsager-Machlup action functional are also obtained. Finally, numerical experiments to illustrate and support the theoretical findings.
Read the full abstract ▾
All authors Shanqi Liu, Qiqi Lian, Jinqiao Duan, Hongjun Gao
Numerical Analysis
A weighted polygonal discontinuous Galerkin method for hierarchically coupled reaction-diffusion systems with cubic interactions
Mattia Corti
We develop and analyse a symmetric weighted interior penalty polytopal discontinuous Galerkin (SWIP-PolyDG) method for multi-species reaction–
diffusion systems with nonlinear reaction terms of up to cubic order. Such terms arise naturally when higher-order interactions are incorporated into population models, allowing non-additive effects among multiple species to influence local growth and conversion mechanisms. The proposed framework accommodates heterogeneous and possibly anisotropic
diffusion tensors on general polygonal meshes. To control the nonlinear coupling, we consider a hierarchical block structure in the reaction operator, whereby each population block depends only on its own variables and on those associated with preceding blocks. In addition, the cubic self-interactions within each block are assumed to have a dissipative diagonal structure. Under these hypotheses and proceeding recursively over the hierarchy of population blocks, we derive local-in-time stability estimates in two spatial dimensions for the semi-discrete formulation in both the $L^2(\Omega)$-and dG-norms. We further establish an a priori error estimate in a combined $L^2(\Omega)$-dG energy norm for sufficiently regular solutions for the semi-discrete formulation. Finally, numerical experiments confirm the predicted convergence behaviour and illustrate the robustness of the method under heterogeneous
diffusion.
Read the full abstract ▾
Numerical Analysis
Spectral analysis of a class of IRK stage preconditioners for linear wave equations
Michal Outrata and Aman Rani
We lay out the analysis of the “Kronecker preconditioners” adopted in [24] for the linear wave equations. Importantly, we deliver the analysis justifying the claims in~[24, Section 4.2] about these preconditioners and we unpack all of the necessary numerical background for a full understanding of the observed phenomena.
Read the full abstract ▾
Numerical Analysis
The paxotopy method for finding isolated roots of a system of nonlinear equations
Sándor Bozóki
We focus on systems of nonlinear equations, where some equations are non-
polynomial. Building on the remarkable developments in solving
polynomial systems during the recent decades, and on a few extensions to specific examples of the transcendental case, a general framework for finding isolated roots is proposed. The name paxotopy is proposed after (1)
Polynomial ApproXimation of the transcendental functions, then (2) solving the approximating
polynomial system by the homOTOPY continuation method, and finally (3) the roots of the
polynomial systems are used as starting points of a Newton's iteration to arrive at the solutions of the original system. These steps combine the advantages of local and global methods.
Read the full abstract ▾
Numerical Analysis · Cross-listed from cs.LG
Broken Symmetry in BF16 Attention: Why FlashAttention Gradients Blow Up Late in Training
Junlin Chen, Daize Dong, Huanwei Di et al. (+9 more)
BF16 is now standard in large-scale pretraining, including in fused
attention kernels such as FlashAttention, and these kernels are widely trusted. When we used FlashAttention-3 to pretrain a 450M-parameter
transformer on 50B
tokens, however, we ran into a problem: training was healthy for 25B
tokens, then the gradient norm grew a thousandfold and the loss ended 0.2 nats above FP32
attention, without a single NaN. Recomputing the
attention backward of just two layers in FP32 removes almost all of the excess gradient. Part of the cause is known: a fused multiply-add in the forward softmax, so far treated as an extreme-input NaN case and never fixed in FlashAttention-3. Repairing it stops the blow-up, but the query gradient is still wrong by more than its own size, and training still drives
attention logits to thousands of times their size under accurate gradients. The remaining error comes from a broken conservation law. The softmax score gradient sums to zero along every row, which makes the query gradient blind to where the keys sit as a group; rounding it to BF16 leaves a small nonzero sum that leaks the mean key into the gradient, and the leak grows exactly as late training makes keys large and
attention sharp. We introduce GProj (gauge projection), which restores the zero sum after the cast with two rank-one corrections per row. It cuts the remaining median query/key gradient errors from 219%/13% to 0.34%/0.37%, on par with FP32
attention, for 4.7% more time per training step. In matched from-scratch runs it trains to the same loss as FP32
attention, while FlashAttention-3 and key smoothing both destabilize.
Read the full abstract ▾
All authors Junlin Chen, Daize Dong, Huanwei Di, Haolong Jia, Jiawei Wu, Haotian Xie, Mingkai Zheng, Yang Li, Leshang Chen, Huishu Wang, Eric P. Xing, Hongyi Wang
Numerical Analysis
Immersogeometric fluid-structure interaction modeling of the human mitral valve and transcatheter edge-to-edge repair
Keon Ho Kim, Ashton M. Corpuz, Monu Jaiswal et al. (+2 more)
Transcatheter edge-to-edge repair (TEER) treats mitral regurgitation with a clip that grasps and coapts the mitral valve (MV) leaflets. Predicting its outcome requires resolving the interaction between the blood flow and the complete MV apparatus, in which highly deformable leaflets and chordae tendineae undergo large deformation and complex contact. In this work, we introduce an immersogeometric fluid-structure interaction (FSI) formulation in which the coupling among the valvular components, the surrounding flow, and the repair is enforced weakly, without requiring conforming discretizations. A fully parameterized reconstruction pipeline is developed to convert segmented images into analysis-ready isogeometric meshes. The leaflets and chordae are modeled as Kirchhoff-Love shells and cables, and the shell formulation is modified to include an in-vivo prestrain tensor that accounts for the loading already present in the imaged geometry. We propose a subcell approach that improves the quadrature for the interfacial constraint and eliminates the over-refinement otherwise needed to prevent unphysical leakage through the leaflets. The MV repair is modeled by a penalty formulation that enforces leaflet coaptation over a prescribed region, allowing clip number and placement to be varied without remeshing or modeling the device. A lumped-parameter model at the outflow tract of the left heart imposes a flow-dependent pressure rather than a prescribed waveform, so that the split between forward and regurgitant flow matches physiological behavior. We validate the reconstruction against the segmented imaging data, demonstrate mesh convergence of the proposed formulations, and apply the framework to a prolapsed MV and its single- and double-clip repairs to quantify pre- and post-operative changes in valve dynamics and left-heart hemodynamics.
Read the full abstract ▾
All authors Keon Ho Kim, Ashton M. Corpuz, Monu Jaiswal, Michael S. Sacks, Ming-Chen Hsu
Numerical Analysis · Cross-listed from math.AP
Long-time continuum limits for twisted Schrödinger lattices
Brian Choi
We prove
polynomial-in-time Sobolev bounds on the approximation error between the twisted discrete nonlinear Schrödinger equation and its
magnetic continuum limit. The analysis exploits complete integrability of the continuum flow through its Birkhoff representation to control approximation by a non-integrable
Hamiltonian finite-difference
lattice. We establish a focusing-defocusing dichotomy in uniform
polynomial control: the bounds hold for arbitrary defocusing data and focusing data of sufficiently small mass, while modulational instability and aliasing yield an obstruction for large focusing data. The accompanying algebraic spatial rates are generically optimal in the Baire-category sense. The proof combines Birkhoff coordinates with modified energies for discrete Sobolev growth, while
Fourier filtering recovers uniform spacetime estimates below the energy space despite degenerate
lattice dispersion.
Read the full abstract ▾