Statistical Methodology
Globally aligned Principal Component Analysis for multi-group data
Hedayat Fathi, Marzia A. Cremona and Federico Severino
We propose a novel principal component analysis (PCA) for multi-group
datasets, where the same numerical variables are measured across different groups of observations. Existing approaches either ignore group structure entirely by working with global (pooled) data, focus exclusively on local structure (group-wise PCA), or impose restrictive assumptions of common principal components. Our approach respects the multi-group nature of data while improving global comparability of components. We combine group-specific principal components with global ones through an explicit
alignment mechanism based on regularized optimization. We introduce the notion of globally aligned covariance
matrix, incorporating weighted contributions from global principal directions in the group-wise covariance
matrix. The
alignment strength is controlled by regularization parameters that can be tuned to achieve the desired trade-off. Through a comprehensive simulation study, we demonstrate that the proposed aligned PCA achieves a favorable compromise between capturing local variation within groups and maintaining interpretability and stability across groups. Furthermore, in an application to the 2021 Canadian Census socioeconomic data, the proposed aligned PCA yields more comparable and stable region-specific components than pooled or region-wise PCA.
Read the full abstract ▾
Machine Learning (Statistics)
Priors learned from legacy reconstructions inherit undetectable overconfidence
Ali Siahkoohi and Sina Alemohammad
Where truths are scarce (e.g., seismic and medical imaging), learned priors in ill-posed inverse problems are trained on archives of legacy reconstructions—i.e., an older method's outputs—and their reported uncertainty is taken as data-driven. We show that this prior is, in the population limit, exactly the regularizer that produced its archive of posterior samples, advanced one expectation–maximization step toward the truth. While the step improves the regularizer on the directions the measurements resolve, it leaves the regularizer's assumption on the operator's blind subspace unchanged. An archive of single-best reconstructions collapses the blind interval to zero width. Neither error is detectable in practice, as truths differing only on the blind subspace share the data law, and simulation-based calibration is neutral by construction. We identify from the operator alone which directions the measurements do not inform, and, given a handful of ground-truth models, build intervals there that contain the truth as often as they claim to. We validate these findings on a two-dimensional example with closed-form predictions and in controlled experiments on seismic-imaging and groundwater-flow operators, against priors trained on the truth.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
A Defense of the Quadratic Model
Alexandru Meterez, Pranav Ajit Nair, Depen Morwani et al. (+3 more)
Due to the complexity of neural network loss landscapes, optimization theory is forced to rely on idealized models, and there is generally a tradeoff between how theoretically tractable the model is, and how accurately it describes the true optimization dynamics. In this work, we stress test the simplest possible model of optimization – the quadratic model – and show that it can be surprisingly predictive in an LLM setting with 150M parameters and 3B training
tokens. Specifically, we show that Taylor expanding the model and the loss function at intermediate checkpoints through training can accurately predict the optimization dynamics over windows that can last up to 10% of training. Having established this agreement, we then turn to analyzing the structure of these local quadratic optimization problems through two lenses: the Hessian spectrum and local stability. Using Lanczos quadrature with extremely deep probes, we are able to estimate the Hessian spectrum deep into the tail, and we find a surprising amount of structure in both the
eigenvalues and eigenvectors, which depends on the batch size, preconditioner, and training time. We also empirically test local linear stability at intermediate checkpoints and compare it to theoretical predictions to demonstrate that optimization in LLMs typically occurs at a
stochastic edge of stability, whose nature is also determined by batch size. Our results indicate the quadratic model may be a theoretically tractable proxy for pretraining optimization dynamics.
Read the full abstract ▾
All authors Alexandru Meterez, Pranav Ajit Nair, Depen Morwani, Cengiz Pehlevan, Sham Kakade, Alex Damian
Machine Learning (Statistics) · Cross-listed from cs.LG
An Introduction to Bayesian and Frequentist Simulation-Based Inference with Machine Learning
Maximilian Dax, Theo Heimel and Gilles Louppe
Simulation-based inference (SBI) with machine learning is an increasingly important tool for solving inverse problems in science and engineering, including parameter inference and the inversion of detector effects. We provide an overview of the
Bayesian and frequentist statistical frameworks, describe how machine-learning-based SBI methods, such as neural posterior estimation and neural likelihood estimation, can be used for parameter estimation within these frameworks, and show that the same methods can also be applied to Empirical Bayes or unfolding tasks. We also discuss how to validate inference results and the limitations of SBI with machine learning.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Self-Poisoning in Adaptive Out-of-Distribution Detection: A Sharp-Threshold Theory and Certified Label-Free Calibration
Vishnu Bindu Balachandran
Test-time adaptive out-of-distribution (OOD) detectors update a
memory bank from the unlabelled stream. We show this adaptation obeys a provable dynamical law. Modelling bank
impurity as a generalized Pólya urn, we prove almost-sure convergence to a mean-field
equilibrium whose slope acts as a reproduction number. Below one,
impurity stays benign. Above one, the bank is fully poisoned and the detector collapses. The measured admission kernel is affine ($R^2 \ge 0.996$) with slope just below one in every encoder family (a protocol signature), so this detector class is near-critical by design, and across 96 settings the predicted threshold matches the empirical collapse, where ungated dictionaries lose up to $0.163$ AUROC. We then prove that a certified admission gate, reading only a frozen reserve, severs the feedback loop and removes the transition at every contamination rate, even adversarially, while controlling false positives label-free. For the complementary static-calibration failure under drift we give CDC, which restores nominal FPR label-free on all tested drift-affected
cells. Finally we prove a two-world impossibility
theorem. Drift and contamination are indistinguishable without labels, forcing a closed-form power ceiling our procedure approaches. Together these give a complete possibility/impossibility characterization of label-free adaptive OOD detection.
Read the full abstract ▾
Statistical Methodology
Equilibrium Causal Digital Twins: Validation, Transport, and Identification Limits
Faraz Dadgostari and Neda Nazemi
Digital twins are often used to predict how a system would respond to an intervention. In systems with feedback, a twin must reproduce an
equilibrium counterfactual, and a twin developed in one domain may fail after mechanisms change. We study when these predictions can be validated and transported. For
equilibrium causal games, we give conditions on the mechanisms,
equilibrium selection, and intervention design under which agreement with experimental distributions identifies the counterfactual of interest. We show why agreement of means and covariances is insufficient for distributional queries. We then introduce cyclic selection diagrams and derive criteria for direct reuse and for hybrid models that combine invariant source mechanisms with target information. An impossibility result constructs systems that agree under every experiment in a finite design but disagree on the target counterfactual, showing that validation requires structural assumptions. For linear models, we derive intervention requirements that depend on the mechanisms that changed, the observation model, and graph support. When point identification fails, we characterize the remaining range of query values. We also provide statistical tests for reconstructed means and covariances and illustrate the theory in synthetic feedback systems.
Read the full abstract ▾
Statistics Theory · Cross-listed from physics.soc-ph
Reconstructing the Probability Measure of a Multi-group Curie-Weiss Model with Interacting Groups
Miguel Ballesteros, Gerardo Franco Cordova, Fedro Guillen et al. (+1 more)
We study the problem of reconstructing the probability measure of a multi-group version of the Curie-Weiss or mean-field model of ferromagnetism from a sample of the voting behaviour the population. While originally used to study phase transitions in statistical mechanics, the Curie-Weiss or mean-field model has been applied to study phenomena where many
agents interact with each other, in particular in case of a heterogeneous population with identifiable subpopulations. The degree of social cohesion within social groups manifests in the way the members of the group influence each others' decisions as well as how they behave under outside influence from voters belonging to another group. Contrary to single-group Curie-Weiss models, here we have a larger number of coupling parameters which have to be estimated. While the maximum likelihood
estimator of the coupling parameters has desirable statistical properties in theory, computational challenges make applications to larger populations impractical. Therefore, we analyse an
estimator based on asymptotic approximations to the behaviour of the Curie-Weiss model valid for large populations. Due to the wide applicability of models such as Curie-Weiss, the
estimator is potentially useful in disciplines such as political science, sociology, automated voting, and preference aggregation.
Read the full abstract ▾
All authors Miguel Ballesteros, Gerardo Franco Cordova, Fedro Guillen, Gabor Toth
Statistics Applications · Cross-listed from cs.AI
Unsupervised Consensus-Based Anomaly Detection for Spatiotemporal Malaria Incidence in Ghana
T. Ansah-Narh and Y. Asare Afrane
A consensus anomaly detection framework was applied to monthly malaria surveillance data from Ghana (2014-2023) to identify atypical transmission patterns. Anomalies were highly structured in space and time. Ashanti and Northern Regions accounted for most recurrent anomalies, with persistent hotspots at Tamale, Kumasi, and Accra. A key finding was the spatial distinction between anomaly burden (cumulative cases during anomalous periods) and anomaly frequency (persistence of unusual behaviour). Tamale had the highest burden during anomalies, whereas the highest anomaly rates clustered in Ashanti districts, showing that high-burden areas are not necessarily those with the most frequent anomalous transmission. Anomalous months formed a statistically distinct group, with much higher case counts (Cohen's $d = 3.252$) and large seasonal deviations ($d > 1.2$) compared with normal months. Malaria burden alone provides an incomplete picture of transmission dynamics. By distinguishing where malaria is most prevalent from where transmission behaves most unusually, this framework can strengthen surveillance, prioritise investigations, and support targeted control strategies.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Zero-Flow Two-Sample Tests
Yakun Wang, Leyang Wang, Song Liu et al. (+1 more)
Motivated by the success of modern flow-based generative models in modeling complex data, we study two-sample testing through the lens of flow-based methods. We propose the Zero-Flow Two-Sample Test (ZF2ST), built on the zero-flow criterion, which characterizes distributional equality through a time-reversal antisymmetry of a learnable velocity field. We extend this criterion and further develop the Zero-Flow Discrepancy, an identifying discrepancy that controls the Wasserstein distance, and derive a variational representation in terms of a witness function. This representation naturally leads to a witness-based test whose power is governed by the signal-to-noise ratio (SNR), allowing direct power maximization for witness learning. ZF2ST learns the witness on one data split and performs testing on held-out samples, thereby maintaining Type-I error control and admitting a simple asymptotic null distribution. Experimentally, ZF2ST performs competitively across various synthetic and real-world
benchmarks, while showing particularly strong performance in distinguishing image distributions from different sources.
Read the full abstract ▾
All authors Yakun Wang, Leyang Wang, Song Liu, Taiji Suzuki
Statistics Theory
Optimal use of a black-box learner in semiparametric estimation
Yihong Gu
Consider the partial linear model $Y = \mu_0(X) + \beta_0 \cdot T + \varepsilon$ and $T = \pi_0(X) + u$ in the structure-agnostic setting, where we are blind to the structure $\mu_0$ and $\pi_0$ and estimate the nuisances by a black-box hypothesis class. The learnability of the class is characterized by the estimation error $\delta_s$ in the absence of model misspecification and its $L_2$ mis-specification error $\delta_{a, \mu}$ and $\delta_{a, \pi}$ for $\mu_0$ and $\pi_0$, respectively. We propose a novel
estimator of the target linear coefficient $\theta_0 = \beta_0$ with error rate \[ \frac{1}{\sqrt{n}} + \delta_{a, \mu} \cdot \delta_{a, \pi} + [\delta_s]^2. \] A matching lower bound is also established, implying that this rate is unimprovable. Compared with the product rate yielded by double machine learning (DML), our
estimator removes the suboptimal term $\max(\delta_{a, \mu}, \delta_{a, \pi})\cdot \delta_s$ at no extra cost or assumption. Building on the underlying insights, which are neither tailored to the one-learner setting nor the partial linear model, we propose Transductive
Adversarial Moment-calibrated Editing (TAME), which locally edits debiasing weights induced by black-box regression estimates on the inference sample through
adversarial conditional moment calibration. TAME can be combined with any initial black-box estimates and can strictly improve on DML guarantees when the nuisance difficulties are imbalanced. We discuss how to fully exploit the advantages introduced by TAME, including the gains from using two learners, the resulting under-smoothing principle for model selection, and extensions to other linear functional estimation problems.
Read the full abstract ▾
Statistics Theory · Cross-listed from quant-ph
Unified theory of classical and quantum semiparametric efficiency
Mankei Tsang
In classical and
quantum statistics, high-dimensional unknown parameters are abundant and it is often prudent to make minimal assumptions about them using so-called semiparametric models. To attack a wide range of semiparametric problems in one broad stroke, we present a unified treatment of statistical efficiency for classical and
quantum semiparametric models, generalizing the Cramér-Rao and Helstrom bounds beyond finite-dimensional parameters. We introduce the fundamental concepts in abstract and
geometric terms before applying them to many examples, covering general classical and
quantum models as well as the paradigmatic special cases of Gaussian and Poisson fields. We give an in-depth treatment of channels in the semiparametric efficiency theory and advocate the use of the singular value decomposition to elucidate the statistical effects of channels. To demonstrate the utility of the formalism, we apply it to coherent and incoherent optical imaging problems, assuming an arbitrary field or intensity on the object plane without parametric assumptions. Our formalism enables us to compute classical and
quantum limits to coherent and incoherent imaging resolution in statistical terms. For subdiffraction incoherent imaging, we demonstrate that spatial-mode demultiplexing can be far superior to direct imaging in estimating generalized
Fourier coefficients and come closer to the
quantum limits. We envision our theory becoming an essential tool for both classical and
quantum statistics with useful applications to sensing and imaging, whenever minimal assumptions about a high-dimensional parameter should be made.
Read the full abstract ▾
Statistical Methodology
Identification and Robust Inference for Multiple Treatments with Possibly Invalid Instruments
Ziwei Mei, Qingliang Fan and Zijian Guo
The instrumental variable (IV) method is widely used to infer causal effects in observational studies with unmeasured confounding, but invalid instruments can compromise both population identification and finite-sample inference. This paper studies linear IV models with multiple endogenous treatments and possibly invalid instruments. Identification of multiple effects is more delicate than in the single-treatment setting because a single instrument no longer identifies a single candidate effect; instead, each relevant instrument defines a hyperplane in the multidimensional effect space. For identification of multiple treatment effects, we introduce generalized plurality and majority rules which require a sufficiently large number of IVs to be valid. For inference, data-dependent instrument selection may fail to separate certain invalid IVs from valid ones, leading to undercoverage of confidence intervals when these invalid instruments are mistakenly selected as valid. We propose a sampling confidence interval for each treatment effect, which is robust to IV selection errors. We establish asymptotic coverage and parametric-rate length of our sampling confidence interval under regularity conditions and illustrate this method in Monte Carlo simulations and a Mendelian randomization application.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Finite-Sample Coverage Audits for High-Recall Candidate Generation: Certification and Learning-Theoretic Design
Martin Anthony and Kaveh Salehzadeh Nobari
An initial high-recall stage in an empirical pipeline decides which items pass to later review, labelling, or modelling, and relevant items it misses are lost to every subsequent stage. We study how many audit labels are needed to certify, with finite-sample validity, that this missed relevant mass is small, and our main results characterise the label complexity of this problem. We first show that no procedure using only labels from inside the candidate set can certify any non-trivial bound on the missed mass: the audit must sample the excluded pool, the only region where unrecovered relevant items can lie. We then prove a matching finite-corpus lower bound. Any valid audit that certifies fewer than $m$ missed relevant items with high probability when none are present, even if adaptive and permitted to label the entire included pool, must inspect on the order of $N_0/m$ excluded-pool labels. Excluded-pool
auditing is therefore minimax rate-optimal, not merely convenient, for missed-mass certification in the zero-miss regime. Building on this characterisation, we develop an exact finite-sample toolkit, using binomial and hypergeometric inversion rather than asymptotic approximation, that certifies missed mass, converts it to recall through a two-pool design, certifies pre-specified families of nested candidate generators simultaneously, and produces stress-test certificates against declared perturbation mechanisms. These certificates can be paired with observable review burden to select the least burdensome pre-specified candidate generator meeting a missed-mass target. Every guarantee holds under one discipline: the candidate generator, or the pre-specified family from which it is selected, and the audit rule are fixed before the certification labels are examined.
Read the full abstract ▾
Statistical Methodology · Cross-listed from cs.AI
Detecting LLM-Generated Tokens in Human–LLM Coauthored Text
Yangjun Lu, Hongyi Zhou, Fabian Spill et al. (+3 more)
The rise of human-AI collaborative writing has created a growing need for fine-grained detection methods that support localizing likely LLM-generated content in mixed-authorship documents. Existing methods for detecting LLM-generated text mainly focus on document-level classification and cannot identify which parts of the text are generated by LLMs. This paper introduces a new method to address this urgent need. Our method operates at the
token level, the natural unit of modern
language models, and builds on existing
token-level detection scores. The key idea is to smooth adjacent
token scores to reduce their variability, while using an adaptive Lepski-type rule to select the bandwidth according to the local authorship structure. Our method is simple to implement and does not require
token-level labeled data for training. Theoretically, we characterize this trade-off and show that the proposed method achieves favorable mean square error performance in estimating the underlying signal. Empirically, we demonstrate strong performance of our method against a wide range of baselines in both synthetic
datasets and a realistic
dataset. We deploy a publicly accessible website that implements the methods as well.
Read the full abstract ▾
All authors Yangjun Lu, Hongyi Zhou, Fabian Spill, Kai Ye, Chengchun Shi, Jin Zhu
Statistics Applications · Cross-listed from astro-ph.IM
Modeling Dependence Structures in Astronomical Multi-Band Time Series Data via Multi-Output Gaussian Processes
Samata Das, Lishan Shi, Yasaman Hamayouni et al. (+2 more)
Modern astronomical time-domain surveys routinely collect multi-band light curves that provide complementary information about the physical processes governing source variability. Gaussian processes (GPs) provide a flexible probabilistic framework for modeling irregularly sampled and noisy time-series data. While considerable
attention has been devoted to developing covariance kernels for individual time series, comparatively less
attention has been paid to the statistical representation of dependence among multiple photometric bands. In this work, we present a unified statistical framework for modeling such dependence structures using multi-output GPs. Within this framework, we consider two complementary formulations. The covariance-based formulation specifies dependence directly through
matrix-valued covariance functions and emphasizes the
stochastic properties of the observed light curves, including covariance functions and power spectral densities. In contrast, the latent-process formulation represents the observed light curves as transformations of latent GPs and emphasizes the physical mechanisms generating the observed dependence. To illustrate these formulations, we develop covariance-based and latent-process multi-output damped random walk models and derive their corresponding spectral representations. We further demonstrate the practical implications of dependence-structure modeling through applications to multi-band active
galactic nucleus variability and continuum reverberation mapping. Rather than advocating a universally preferred formulation, this work provides a principled basis for selecting dependence structures according to the scientific objectives and clarifies how this choice influences the statistical characterization and scientific interpretation of
stochastic variability in astronomical sources.
Read the full abstract ▾
All authors Samata Das, Lishan Shi, Yasaman Hamayouni, Hyungsuk Tak, Jong-Hak Woo
Machine Learning (Statistics) · Cross-listed from cs.LG
Semantic-Aware Task Clustering for Constructive and Cooperative Multi-Tasking
Ahmad Halimi Razlighi, Maximilian H. V. Tillmann, Edgar Beck et al. (+2 more)
Cooperative multi-task semantic communication (CMT-SemCom) improves task execution performance by leveraging shared representations. However, as we demonstrated in [1], cooperative multi-tasking can be either constructive or destructive, depending on the semantic relationships among tasks. To ensure constructive cooperation, we propose a semantic-aware task clustering method for CMT-SemCom. We have formulated a sequential multi-stage optimization problem in which semantically aligned tasks are clustered once after a short initial training phase, and then end-to-end (E2E) joint training is conducted exclusively within the discovered groups. Specifically, the problem decomposes into two stages: (i) a semantic clustering problem leveraging hierarchical density-based spatial clustering, and (ii) an intra-cluster E2E CMT-SemCom learning problem. Simulation results demonstrate that the proposed framework effectively mitigates destructive cooperation and negative transfer, yielding accuracy gains compared to unclustered multi-tasking and individual training baselines.
Read the full abstract ▾
All authors Ahmad Halimi Razlighi, Maximilian H. V. Tillmann, Edgar Beck, Bho Matthiesen, Armin Dekorsy
Machine Learning (Statistics) · Cross-listed from quant-ph
Cautious optimism for deep parameterized quantum circuits
Marie Kempkes, Elies Gil-Fuster, Carlos Bravo-Prieto et al. (+5 more)
A central challenge in
quantum machine learning is understanding the scaling behavior of parameterized
quantum circuits (PQCs). In particular, it remains unclear how their performance on unseen data changes as the number of trainable parameters increases. Prior works have derived formal generalization guarantees for
quantum models, but it is well-known that many such results do not fully characterize generalization behavior in practice. In this work, we show that gradient-based PQCs can exhibit improved performance on unseen data as model size increases, displaying the phenomenon of double descent. This contrasts with the traditional view that larger models lead to degraded generalization. We provide analytical results rigorously underpinning this behavior by leveraging add-one-in perturbation techniques and spectral properties of random
matrices. We support these results with numerical experiments on re-uploading PQCs across several data sets and training set sizes, consistently observing the predicted double descent behavior. While other obstacles on the path toward practical
quantum machine learning remain, our finding that deeper parameterized
quantum circuits do not necessarily exhibit degraded performance provides reasons for cautious optimism.
Read the full abstract ▾
All authors Marie Kempkes, Elies Gil-Fuster, Carlos Bravo-Prieto, Aroosa Ijaz, Alissa Wilms, Jens Eisert, Evert van Nieuwenburg, Vedran Dunjko
Statistical Methodology · Cross-listed from cs.LG
A Diffusion-Model Subpopulation Digital Twin for Mobile Health Deployment: A Case Study on the HeartSteps Intervention
Ziping Xu, Yuyi Chang, Chenshun Ni et al. (+5 more)
Mobile-health interventions increasingly use online learning and decision making
algorithms to personalize when to nudge users toward healthier behavior, but a poorly designed
algorithm can burden and disengage participants. New
algorithm design decisions should therefore be vetted against realistic simulated users before each real-life deployment. We propose a method to develop “JITAI-Twins”: digital twins of a target subpopulation for comparing candidate online
algorithms before a just-in-time adaptive intervention (JITAI) deployment. The method builds on a conditional time-series
diffusion model that is temporally consistent (future actions do not affect the generated past), and it supports repeated updating from three sources of information, in three steps: pre-training on a large observational
dataset, fine-tuning on small prior intervention deployments in related populations, and inference-time calibration to the next target population from domain-scientist expertise. We validate the twin at each pre-deployment stage of the long-running HeartSteps series (v2 through v4) of physical-activity suggestion intervention deployments, treating each successive deployment as an upcoming study. The proposed method reproduces the target subpopulation's temporal and between-participant structure better than simpler simulators. These results suggest that our twin can be used to simulate a target deployment before it runs, the prerequisite for testing and informing online
algorithm design decisions.
Read the full abstract ▾
All authors Ziping Xu, Yuyi Chang, Chenshun Ni, Nithin Sugavanam, Asim H. Gazi, Pedja Klasnja, Emre Ertin, Susan A. Murphy
Statistics Theory · Cross-listed from math.NA
3D Uncertainty Quantification for Photoacoustic Tomography
Babak Maboudi Afkham, Amal Mohammed A Alghami, Hassan Yazdanian et al. (+1 more)
Photoacoustic
tomography (PAT) is a promising modality for high-resolution biomedical imaging, motivating the need for reliable uncertainty quantification (UQ) of reconstructed images.
Bayesian approaches provide a rigorous framework for UQ but remain computationally challenging for realistic three-dimensional PAT and are sensitive to numerical approximations in the governing wave equation. We develop a finite-element
Bayesian UQ framework for PAT that accommodates complex computational domains and detector geometries while enabling large-scale three-dimensional inference. The proposed methodology reformulates the randomize-then-optimize (RTO) sampling strategy as a
matrix-free
algorithm that generates independent posterior samples using only forward and adjoint wave propagations. Particular
attention is given to constructing an adjoint discretization that forms an exact transpose pair with the discrete forward operator while remaining consistent with the continuous PAT adjoint, enabling efficient least-squares solvers within the sampling procedure. We investigate the influence of temporal discretization, artificial boundary conditions, and adjoint consistency on posterior uncertainty and identify discretization strategies that avoid numerical artifacts. The framework is validated against exact posterior statistics, existing
Bayesian PAT methods, and
Hamiltonian Monte Carlo using the No-U-Turn Sampler (NUTS), and is demonstrated on a three-dimensional problem with approximately $2\times 10^5$ unknowns on a general finite-element domain. To the best of our knowledge, this is the first large-scale
Bayesian PAT study on general three-dimensional finite-element geometries, and the methodology extends naturally to a broad class of linear PDE-constrained inverse problems.
Read the full abstract ▾
All authors Babak Maboudi Afkham, Amal Mohammed A Alghami, Hassan Yazdanian, Tanja Tarvainen
Machine Learning (Statistics) · Cross-listed from cs.LG
Hilbert Operator for Progressive Encoding (HOPE): A Mathematical Framework for Deconstructing Learned Representations in Deep Networks
Hossein Mobahi and Peter L. Bartlett
Deep neural networks encode complex representations, but deconstructing this internal knowledge remains a challenge. Given the link between learning and compression, network compression offers a promising lens to analyze this knowledge. However, standard compression heuristics often suffer from scale
symmetries and architectural biases. To resolve these, we introduce
Hilbert Operator for Progressive Encoding (HOPE), a mathematical framework to gradually deconstruct the representations in trained network weights. HOPE shifts network compression from the discrete domain into a
Hilbert space of continuous functions. By modeling individual
neurons as rank-1
Hilbert-Schmidt operators, HOPE unifies
pruning and
neuron merging as low-rank subspace projection. Extending this formulation, HOPE introduces macro block eviction to encompass multi-layer structures like entire residual pathways under the same unified metric. This unified approach enables unbiased architectural decisions across layers with different types and sizes. HOPE is a data-free and hyperparameter-free framework. We present proof-of-concept experiments in model compression and fine-tuning to highlight the practical potential of our theory.
Read the full abstract ▾
Statistical Methodology · Cross-listed from econ.EM
Uniformly Consistent Semi-nonparametric Demand Estimation with Micro-Data
Richard Grigorian
This paper develops a profiled sieve minimum-distance
estimator for a semi-nonparametric differentiated-products demand model with micro-level choice data. Building on Berry and Haile (2024), the
estimator uses within-
market variation in consumer
covariates to recover a flexible consumer-heterogeneity function and
market-specific composite intercepts. Excluded price instruments then separate these intercepts into a flexible price-side function and structural demand shocks. The main statistical challenge is that the number of profiled
market intercepts grows with the number of
markets. I show that, when both the number of
markets and the minimum within-
market sample size grow, this profiling step is asymptotically negligible and the common structural functions are uniformly consistently estimated. Monte Carlo evidence supports the consistency result and illustrates the value of flexible price-side estimation for counterfactual demand analysis.
Read the full abstract ▾
Statistical Computation
The acceptance-complement method revisited
Luc Devroye
We revisit the acceptance-complement method in random variate generation and show how it can replace the rejection method in many examples. While the rejection method has geometrically distributed execution times, the acceptance-complement method has a constant (deterministic) run time and qualifies as a “one-liner”. We show how this method can be used to efficiently generate random variates from several distributions, such as the gamma and beta. In addition, we show that there is an acceptance-complement method that is valid for all log-concave densities with known location of the mode and black-box type access to the density.
Read the full abstract ▾
Statistics Applications · Cross-listed from q-bio.MN
From Berg-Purcell precision bounds to clock-limited information capacity
Michał Komorowski
Physical limits to chemical sensing are traditionally expressed as Berg-Purcell bounds on estimation accuracy. Whether these bounds also limit the total amount of information a molecular receptor can transmit has remained unclear, despite the fact that cellular signaling performance is naturally quantified in bits rather than precision alone. Here we derive an explicit link between Berg-Purcell-type sensing limits and the information capacity of a single two-state receptor, yielding a compact expression that separates contributions from concentration range, receptor copy number, and averaging time. We show that, in the ideal fixed-time occupancy model,
diffusion-limited sampling alone does not define a finite global information bound: with a perfectly specified integration window and unbounded input range, information capacity grows without bound with dynamic range, albeit slowly. A finite saturation arises when time integration is treated as an explicit physical resource. Finite timing precision yields a clock-limited bound on information capacity, and in the high-occupancy regime information transmission crosses over from
diffusion-limited to clock-limited behavior. Together, these results establish a receptor-level information bound in bits that is finite once constraints on timing precision are taken into account.
Read the full abstract ▾
Machine Learning (Statistics)
Transformer-based Diffusion models for Hydrological Time Series Probabilistic Imputation and Forecasting
Ferdinand Bhavsar, Lionel Benoit, Maxime Savatier et al. (+1 more)
The modeling of hydrometeorological time series with limited observations is a key challenge in the monitoring of hydro-systems and water resources, as well as for flood or drought risk assessment. Due to the high variability of the underlying processes and the sparsity of available measurements, traditional statistical approaches often struggle to accurately represent their dynamics. In this context, recent advances in deep learning offer a promising direction for improving the representation and generation of complex temporal processes sampled at several observation sites. This study investigates the application of
transformer-based
diffusion models to the simulation and reconstruction of hydrological time series. The proposed framework is applied to the joint modeling of water quantity and quality at six sites spread across three adjacent headwater catchments located in North-East France on a limestone plateau covered by forests and field crops. The model is calibrated and validated using available observational data, which has been quality controlled and corrected for sensor drift and malfunction through collaborative efforts by LNE metrology expertise and Andra monthly quality control over more than 15 years. Its performance is compared with several established baseline approaches commonly used for time series modeling. Quantitative evaluation metrics are employed to assess the ability of the proposed method to reproduce key temporal characteristics of the observed signals in two settings: the imputation of incomplete time series and the
forecasting of upcoming hydrological conditions. Results support the effectiveness of the
transformer-based approach and highlight its capacity to capture and simulate the complex patterns present in hydrological data. In particular, the results indicate that
diffusion models can efficiently sample realistic time series distributions under observation settings with variable missing data for both
forecasting and imputation.
Read the full abstract ▾
All authors Ferdinand Bhavsar, Lionel Benoit, Maxime Savatier, Edith Gabriel
Statistics Applications
Enhancing Gender Equality Assessment through Object-Oriented Bayesian networks: the European Gender Equality Index Case
Lorenzo Giammei, Fulvia Mecatti, Flaminia Musella et al. (+1 more)
A novel data-driven framework is introduced to assess gender equality by complementing and empowering a widely used European gender composite indicator, the Gender Equality Index (GEI). The GEI synthetizes the latent construct of gender equality into a single score and is extensively employed for cross-country comparison and monitoring. While effective for communication and benchmarking, this practice is affected by conceptual and methodological limitations, including marginal analysis that leaves interactions and conditional (in)dependencies unmeasured, and a lack of predictive capability. To address these limitations, this paper proposes the use of Object-Oriented
Bayesian Networks (OOBNs) to model the GEI. By preserving the hierarchical structure of the index, OOBNs extend
Bayesian Networks and enable a multivariate and probabilistic representation of interdependencies among the components of gender equality. This approach advances intersectional gender statistics by shifting the focus from computing a single composite score to modelling the underlying mechanisms that shape gender inequalities. The proposed methodology enhances the assessment and monitoring of gender equality and adds a predictive dimension through scenario-based evaluation, thereby supporting Gender Impact Assessment and policy decision-making. An application to Italian official statistics illustrates the practical relevance of the framework and its applicability to other national contexts and policy needs.
Read the full abstract ▾
All authors Lorenzo Giammei, Fulvia Mecatti, Flaminia Musella, Paola Vicard
Machine Learning (Statistics) · Cross-listed from cs.LG
Smooth Neural Point Processes via B-Splines
Michele Bellomo, Riccardo Ramaschi, Alberto Dolara et al. (+1 more)
Temporal point processes (TPPs) provide a general and flexible framework for modeling sequences of events in continuous time. Neural networks have been successfully employed to model TPPs in a highly expressive and data-driven way. Neural TPPs are typically trained via Maximum Likelihood Estimation (MLE) by minimizing the negative log-likelihood (NLL), which depends on both the conditional intensity function (CIF) and its integral over time, the compensator. Recent neural TPP approaches enable exact evaluation of the NLL without numerical integration. However, these methods typically model the compensator rather than the CIF directly, impose constraints on the neural network architecture, and are computationally expensive during training, as event contributions to the NLL are evaluated sequentially rather than in parallel. In this work, we propose a novel neural TPP model that directly parametrizes the CIF as a non-negative combination of B-spline basis functions, whose coefficients are predicted by a neural network. This formulation enables exact evaluation of the NLL, preserves full flexibility in the neural architecture, allows efficient parallelization during training, and naturally supports CIF smoothness regularization through the integrated squared second derivative. Experiments on both synthetic and real-world
datasets show improved computational efficiency and predictive accuracy compared to the reference neural TPP baseline.
Read the full abstract ▾
All authors Michele Bellomo, Riccardo Ramaschi, Alberto Dolara, Tomaso Aste
Machine Learning (Statistics)
Automatic knot selection in smooth additive models
Nicolás Carrizosa, Vanesa Guerrero and María Durbán
B-spline regression constitutes a widely used framework for nonparametric modeling. The performance of this methodology depends on specifying the number and placement of changepoints, known as
knots, prior to the estimation process. Such
knot sequence determines the dimension of the B-spline basis used to represent the regression function and the number of coefficients to be estimated. Therefore, the
knots' choice affects the model's flexibility, influencing its smoothness and goodness-of-fit. Traditionally, this problem has been addressed either by explicitly selecting
knots, via
knot-selection
algorithms, or by regularization methods, such as P-splines, which automatically tune the regressor's smoothness. The latter have become the standard in generalized additive models (GAMs). In contrast,
knot-selection techniques, frequently neglected because of computational or modeling limitations, provide certain advantages which can be valuable in some contexts. In this work, we introduce a novel explicit
knot-selection technique for GAMs based on an extension of the adaptive splines (A-splines)
knot selection methodology, combined with a customized Fellner-Schall scheme for tuning the associated parameters. Our approach is evaluated on various synthetic and real
datasets and compared with P-splines and state-of-the-art
knot-selection techniques. The results indicate comparable performance, while producing models built on a substantially smaller number of basis elements.
Read the full abstract ▾
Statistics Applications
Risk-Limiting Audits for Parliamentary Majorities
Jack Freestone, Dennis Leung and Damjan Vukcevic
Existing methods for risk-limiting audits typically focus on certifying individual contests. In parliamentary elections, however, the politically relevant outcome is often whether a party has won enough seats to form government, not whether every reported seat outcome is correct. Extending on the work of Mohanty et al. (2019), we formulate the certification of a parliamentary majority as a partial conjunction testing problem: it is enough to verify that the reported winning party truly won at least a majority of its reported seats. Building on the SHANGRLA
auditing framework, we construct a sequential audit statistic for the majority outcome by combining seat-level statistics. We then propose adaptive sampling strategies that allocate
auditing effort across seats, including variants that learn to avoid spending excessive effort on seats that appear unlikely to have been truly won. Using simulations based on synthetic and real data, from the 2014 Indian Lok Sabha election, we show that
auditing the parliamentary majority can substantially reduce the number of ballots inspected (by almost a thousand-fold) compared to certifying every reported winning seat.
Read the full abstract ▾
Statistical Methodology
Information criteria exploiting latent structure for model selection in Structural Equation Models
Marion Naveau, Magalie Houée-Bigot, Matthieu Marbac et al. (+2 more)
Structural equation models (SEM) are widely used to describe dependency structures between latent variables, making model selection a key issue in many applications. Existing information criteria are generally based on the integrated observed-data likelihood and therefore do not explicitly account for the latent structure of the model. In this paper, we propose two new information criteria derived from the integrated complete-data likelihood. The first adapts the Integrated Completed Likelihood criterion to Gaussian SEM, while the second proposes an alternative approach to approximating the integrated observed-data log-likelihood by incorporating latent structural information and using an importance sampling strategy. Their performance is assessed through an extensive simulation study covering null, direct, indirect and complete latent structures under different sample sizes and signal strengths. The results show that the proposed importance sampling strategy provides robust and competitive model selection across a wide range of scenarios, whereas the proposed ICL criterion is particularly effective for recovering latent dependency structures when the latent variables are accurately estimated. These findings demonstrate the potential benefits of explicitly exploiting the latent structure when developing information criteria for structural equation models.
Read the full abstract ▾
All authors Marion Naveau, Magalie Houée-Bigot, Matthieu Marbac, Anouk Zancarini, Mathieu Emily
Statistics Applications
Trust by Context, Not by Design? A Quantitative Study of Data Donation Willingness for Open-Source Civic AI in Switzerland
Sabine Wildemann and Daniel Ambach
Civic AI systems increasingly support democratic participation, yet interactions with them may reveal sensitive political views, creating tension between improving AI models and residents' expectations of
privacy and consent. This study examines the conditions of transparency and user control under which Swiss residents are willing to donate their anonymized chatbot conversations to train an open-source AI model. A 2x2 between-subjects factorial design evaluated how a Data Nutrition Label and a granular consent dashboard influence donation decisions. The experiment was delivered via a
multilingual online survey featuring a custom chatbot powered by the Apertus-70B model. Analysis of the 205 participants revealed that neither transparency nor control significantly affected donation behavior. Rates were uniformly high (91.7% overall), producing a ceiling effect, and
Bayesian checks confirmed the absence of treatment effects. The dashboard raised perceived control but not donation, and high-control participants actively restricted their data-use settings. A qualitative analysis of 120 open-ended responses indicates that residents framed donation as a contribution to the public good, motivated by democratic participation, an open-source model, and research, while many regarded their anonymized queries as non-personal and therefore low in risk. Interpreted through the
privacy calculus, a high perceived benefit coincided with a low perceived risk under high institutional trust, so both sides of the trade-off aligned and interface design had little leverage. Offering control served less to raise donation than to let residents define the terms of their contribution.
Read the full abstract ▾