Statistical Methodology · Cross-listed from q-fin.PM
On Reference-Regulated Multiperiod Mean-Variance Portfolio Optimization in High Dimensions
Yutao Deng, Jianjun Gao and Weichen Wang
The multiperiod mean-variance (MV) portfolio optimization serves as a vital expansion of Markowitz's static MV portfolio selection framework. Just like its static counterpart, the multiperiod MV portfolio remains susceptible to estimation errors. We propose a reference-regulated multiperiod mean-variance (RRMV) framework that penalizes deviations from a reference policy. Therefore, this new optimization successfully combines the advantages of dynamic strategies and reference portfolios. A key contribution of this paper is the characterization of the out-of-sample Sharpe ratio under high-dimensional asymptotics with estimation errors in both the mean vector and the covariance
matrix. We show how the reference penalty and the investment horizon jointly affect the optimized portfolio performance, and how regularization operates differently from the single-period portfolio optimization. Extensive simulation and real data studies demonstrate that the proposed framework improves the stability and out-of-sample Sharpe ratios of multiperiod policies significantly.
Read the full abstract ▾
Statistics Applications · Cross-listed from cs.LG
Conformal Risk Prediction for Non-Alcoholic Fatty Liver Disease Using Gradient Boosting with Distribution-Free Coverages
Xinze Zhang
Non-alcoholic fatty liver disease (NAFLD) affects roughly 25% of global adults, posing substantial hepatic and cardiovascular risks. Yet, population-level screening tools remain inadequate. We present Method, a machine-learning framework for NAFLD risk prediction coupling gradient-boosted decision trees with conformal prediction to yield calibrated, distribution-free coverage guarantees on individual risk estimates. It integrates a mutual-information-based stability selection procedure to identify a compact, clinically interpretable feature subset via bootstrap resampling, constructing prediction sets whose marginal coverage provably exceeds a user-specified confidence level. We evaluated Method on a multicenter cohort from Guangzhou, China (primary n=2,187; external validation n=412) using 78 candidate features across demographics, metabolic biomarkers, and lifestyle factors. Method achieves an AUROC of 0.912 internally and 0.891 externally, outperforming deep neural networks, TabNet, support vector machines, and logistic regression. Conformal prediction sets achieve 91.3% empirical coverage at the 90% nominal level. A three-tier risk stratification derived from these scores separates the population into distinct groups, with the high-risk subgroup showing a 12-month progression rate 4.7 times that of the low-risk tier. The selected features – notably waist circumference, ALT, GGT, triglycerides, fasting glucose, and BMI – align with established metabolic risk factors, providing biological plausibility.
Read the full abstract ▾
Machine Learning (Statistics)
Target Updates May Stabilize Linear Q-Learning: Periodic and Soft Dynamics
Donghwan Lee
Periodic target updates in Q-learning and soft target updates in actor-critic methods are empirically well established stabilization mechanisms, but their precise theoretical explanation is still incomplete. This paper gives a rigorous and exact analysis of these mechanisms for Q-learning with linear function approximation (linear Q-learning) using the exact switched linear system (SLS) dynamics induced by the Bellman maximum and the joint spectral radius (JSR) of the resulting switching
matrix families. Although linear Q-learning can fail to converge in general, we prove that, under explicit spectral and step-size conditions, periodic hard target updates and soft target updates can guarantee convergence to the exact projected Q-Bellman solution. The main analysis is carried out for deterministic linear Q-learning, where the target-update mechanism is most transparent. Once the corresponding JSR certificate is established for the mean recursion, the
stochastic reinforcement-learning setting can be treated by replacing deterministic modes with sampled
stochastic modes and adding the corresponding
stochastic-noise analysis.
Read the full abstract ▾
Statistics Applications · Cross-listed from astro-ph.GA
The Information Content of Quasar Variability Light Curves: How Well Can we Infer Stochastic Model Parameters?
Brendon J. Brewer, Geraint F. Lewis, Xiang Yu et al. (+1 more)
Quasar variability, driven by multi-scale physical processing within a relativistic
accretion disk, is commonly modelled with
stochastic time series models. The simplest of these is the Damped Random Walk (DRW), also known as the Ornstein-Uhlenbeck (OU) process. Here, we demonstrate that, when fitting such a model to
quasar light curve data, the mean of the light curve, $\mu$, should not be fixed (which is the typical approach), as this leads to overconfident inferences about the variability timescale $\tau$, with substantially underestimated uncertainties. However, the short term
volatility parameter $\eta$ is typically very well constrained from short light curves. Through simulations, we compute information theoretic quantities such as the conditional
entropy and the mutual information, confirming that light curves provide much more information about $\eta$ than about $\tau$. As a result, we recommend that future
quasar variability studies focus on $\eta$ rather than $\tau$. To demonstrate this approach, we fit a hierarchical
Bayesian regression model for $\eta$ as a function of bolometric luminosity and rest wavelength to a
dataset of 570 light curves measured over decades. We perform the fit using a likelihood function that uses the light curves directly, rather than using intermediate $\eta$ values from individual light curve fits. We find that
volatility decreases as a function of both bolometric luminosity and rest wavelength. The
volatility also decreases more steeply with
redshift than time dilation alone would suggest, pointing to an increase in intrinsic
volatility as
quasars evolve over
cosmic time.
Read the full abstract ▾
All authors Brendon J. Brewer, Geraint F. Lewis, Xiang Yu, Yuan Li
Statistics Applications
Model complexity in econometrics - a combinatorial analysis
Vahidin Jeleskovic
Regression models and Vector Autoregressive Models (VARs) play crucial roles in econometrics by allowing the analysis of multiple variables simultaneously. Despite their utility, these models face challenges like underfitting and overfitting, especially when determining the optimal model specification, which can lead to significant computational costs. To address these challenges, econometricians often rely on widely adopted model selection criteria such as the Akaike Information Criterion (AIC) and
Bayesian Information Criterion (BIC). These criteria help balance model complexity and goodness of fit, aiding in the selection of the most suitable model specification for the given data. Nonetheless, there is a notable gap in existing research concerning the correct specification of these models, particularly in determining the optimal number of states a system can assume. Addressing this gap, we introduce a combinatorial framework designed to calculate the potential number of states in such econometric models. Our approach involves delineating four distinct stages in model development, each offering a range of specifications. This method enables a comprehensive combinatorial calculation of all possible states. The aim of this paper is to highlight this overlooked aspect of model specification and to spark a constructive dialogue within the empirical research community. By doing so, we hope to inspire further research that enhances the precision and applicability of econometric models. A theoretical complexity criterion is necessary to elucidate fundamental limitations and propose new objectives to pursue.
Read the full abstract ▾
Statistical Computation
Voronoi-Elitism Genetic Algorithm: A Generic Derivative-Free Routine With Theory and Implementation for Statistical Optimization
Anthony Haitao Zou, Yizhou Jake Cai and Ting Fung Ma
In this paper, we propose a generic optimization approach for challenging objective functions that finds applications in various statistical problems. We focus on objective functions with two parameter blocks of one amenable to analytic optimization, and another that is irregular or computationally expensive. To address this setting, we propose the Voronoi-Elitism Genetic
Algorithm (VEGA), a derivative-free optimization method that embeds
geometric information into genetic search. The proposed
algorithm retains elite candidates and constructs Voronoi-based neighborhoods around them, whose crossover and self-adaptive mutation balance exploitation of promising solutions with exploration of under-covered regions. We study the high dimensional behavior of genetic search by analyzing distance concentration, and the effects of population size and shrinking mutation, which shows that the
algorithm improves spatial coverage and yields sharper distance bounds under limited computational budgets. Simulation studies are conducted to compare VEGA with two genetic-type
algorithms competitors in finite samples. A real data application on Stack Exchange activity data further illustrates its ability to identify stable structural changes, implying the
algorithm is computationally flexible for high-dimensional, derivative-free optimization and applicable for various statistical problems.
Read the full abstract ▾
Machine Learning (Statistics)
Computation-Aware Kalman Filtering with Model Selection for Neural Dynamics
JR Huml, Jonathan Wenger and John P. Cunningham
Due to their explicit priors and ability to model uncertainty,
Bayesian methods have played a major role in dynamical latent variable modeling of single-
cell neural recordings. However, modern-sized
datasets have made overparameterized deep networks the preferred methods of choice due to their predictive power and favorable computational scaling. While many posterior approximations exist, all incur approximation errors. Recent work accounts for this error in the form of computational uncertainty but comes at the cost of quadratic complexity and assumes fixed model hyperparameters. Here we extend this development to model selection, including a novel training loss and optimization scheme, which yields tractable inference in large state-spaces. We introduce a framework, the Computation-Aware State-Space Model (CASSM), specifically designed for the scale-imbalanced regime, where the number of trials is significantly lower than the number of recorded
neurons. In this regime, for both synthetic and real data, we show that our method is competitive with data-hungry deep networks, with significantly improved uncertainty calibration over previous attempts to scale
Bayesian methods. Our experiments provide a roadmap to neuroscience researchers in choosing from a host of potential dynamical latent variable models given key
dataset properties and constraints.
Read the full abstract ▾
Statistical Methodology
Comb Test: Histogram Uniformity Testing Based on Discrete Total Variation
Nikola Banić and Neven Elezović
Histogram uniformity testing is a common statistical task usually performed using Pearson's chi-square test. This paper proposes a complementary test based on the discrete total variation of the bin counts, which is trivial to compute and, unlike chi-square, exploits bin ordering. Its exact null distribution follows from dynamic programming, and a gamma approximation, verified up to $N=5\cdot10^4$, extends it to large samples. Matching alternatives in $\ell_1$ distance locates the test: it gains over chi-square only in the period-2 comb direction and is least sensitive to smooth, single-jump, and depleted-block departures, for which chi-square is preferable. Experiments on converter nonlinearity, rounding bias, and real images confirm the analysis. Code and data are available at https://github.com/DiscreteTotalVariation/CombTest.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.AI
Transferring Information Across Interventions in Causal Bayesian Optimization
Mohammad Ali Javidian
Bayesian optimization is a popular way to optimize expensive systems, where every experiment, simulation, or intervention costs time or money. In its standard form, it treats the variables we control as plain inputs to a black box and cannot tell apart mere correlation from a real cause and effect. Causal
Bayesian optimization closes part of this gap by using a known causal graph together with observational data to decide which variables are worth intervening on. Existing methods, however, learn the effect of each possible intervention almost in isolation, even though in a causal system these effects usually share the same underlying mechanisms. We propose graph-coupled causal
Bayesian optimization, which ties the different intervention effects together through the uncertainty we have about a small set of shared causal parameters. The result is a causal kernel that lets evidence collected from one intervention improve our estimate of related interventions. For identifiable linear Gaussian causal models, we show that this kernel has low rank, bounded by the number of shared parameters rather than by the size of the intervention menu. This in turn yields an information-gain bound that grows only logarithmically in the optimization horizon, and a regret bound that cleanly separates three sources of error: optimization, causal estimation, and the choice of which intervention sets to consider. We also describe nonlinear and adaptive extensions. Across theory-aligned Gaussian systems, shared-mechanism stress tests, and standard causal optimization
benchmarks, the method keeps the benefits of causal
Bayesian optimization while transferring information across related interventions, with the clearest gains when direct interventions on the target's parents are unavailable and sparse interventional data must be reused across a large family of candidate interventions.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Leaf Spectral Reflectance Prediction Using Multi-Head Attention Neural Networks
Parastoo Farajpoor, Alireza Pourreza, Mohammadreza Narimani et al. (+2 more)
Accurate modeling of leaf spectral reflectance from physiological and biochemical traits is essential for advancing remote sensing applications in plant science and precision agriculture. Widely used radiative transfer models, such as PROSPECT-PRO, rely on generalized trait-reflectance relationships developed from a wide range of species, which may not fully capture the spectral behavior of specific crops like grapevines. In this study, we developed a trait-to-spectra prediction model using a multi-head
attention neural network trained on a grapevine-specific
dataset that includes 16 leaf traits measured across multiple varieties, growth stages, and years. The model was evaluated using stratified 5-fold cross-validation and achieved an average coefficient of determination (R^2) of 0.84 and normalized root mean squared error (NRMSE) of 1.52 percent, demonstrating high accuracy and generalizability. When compared to PROSPECT-PRO in forward mode, the neural network exhibited lower mean absolute error (MAE), especially in the near-infrared (NIR) and shortwave-infrared (SWIR) regions. These results emphasize the importance of species-specific modeling approaches and show that integrating biochemical and structural traits into data-driven architectures can significantly improve spectral prediction. The proposed model provides a robust framework for generating accurate leaf-level reflectance data, with potential applications in canopy trait
retrieval, vineyard monitoring, and remote sensing-driven crop management.
Read the full abstract ▾
All authors Parastoo Farajpoor, Alireza Pourreza, Mohammadreza Narimani, Ashraf El-Kereamy, Matthew W. Fidelibus
Statistical Methodology
Quantifying Evidential Rigor in Meta-Analytic Corpora: A Simulation-Characterized, Bias-Robust Bayesian Workflow with a Nutrition Case Study
Matt Hester
Conventional meta-analysis summarizes evidence through pooled estimates, intervals, and p-values, but these outputs do not directly measure evidence for an effect, evidence for no effect, or the degree to which conclusions depend on publication selection or small-study effects. We introduce a corpus-scale
Bayesian evidential-audit workflow for meta-analytic corpora. The workflow reconstructs or accepts study-level effects and standard errors, harmonizes directions, fits a matched
Bayesian random-effects baseline and a bias-aware model-averaged ensemble, and reports paired estimates with component and joint model-family evidence. The central estimand is rigor: a joint Bayes-factor summary combining resolved effect/no-effect evidence with absence of an explicit bias component in the fitted ensemble. Rigor is not a positive-finding score; no-effect evidence can score highly, whereas inconclusive or bias-dependent evidence scores poorly. We characterize the workflow using an ADEMP-framed simulation/resampling design with known-
cell synthetic simulation, empirical registry resampling, and empirical fitted-profile-weighted synthetic sampling. A nutrition intervention corpus provides the worked case study, where bias-aware fitting often attenuates conventional estimates and many nominally meaningful effects lose clean evidential support. A public companion repository provides empirical inputs, generated artifacts, simulation source/design files, and documentation for reproducing and adapting the audit.
Read the full abstract ▾
Machine Learning (Statistics)
On the Uncertainty Quantification Ability of Tabular Foundation Models
Tyler R. Johnson, Kian Ben-Jacob, Nima Negarandeh et al. (+2 more)
Foundation models (FMs) have achieved substantial success in generalizing across tasks without problemspecific training or fine-tuning. However, many critical applications in mechanics and computational science require not only accurate predictions but also reliable uncertainty quantification (UQ). Herein we investigate the UQ capabilities of tabular FMs in regression tasks through a comprehensive empirical study comparing Tabular Prior-Data Fitted Networks (TabPFN) against Gaussian processes (GPs). We systematically evaluate these two methods across a host of regression problems with varying complexity,
dataset sizes, and input dimensionalities. We use a default setting to build all the GPs and for a fair comparison against TabPFN v2.5. Our findings highlight an important trade-off between explicit and learned priors: while TabPFN achieves highly competitive performance for complex, high-dimensional problems with sufficient data, GPs often provide superior predictive accuracy and UQ in data-scarce settings. Moreover, when the chosen kernel constitutes a good prior for the underlying function, GP performance can substantially exceed that of TabPFN. Our results can be reproduced from https://github.com/kianswarehouse/GPvsPFN.
Read the full abstract ▾
All authors Tyler R. Johnson, Kian Ben-Jacob, Nima Negarandeh, Oriol Vendrell-Gallart, Ramin Bostanabad
Statistical Methodology
FlowSDR: Sufficient Dimension Reduction via Conditional Normalizing Flows
Yuexiao Dong, Kenichiro Mcalinn, Edoardo Airoldi et al. (+1 more)
Sufficient dimension reduction (SDR) seeks a low-dimensional linear projection of predictors that preserves the conditional distribution of the response. Existing methods target this conditional distribution indirectly, via inverse moments, local forward regression, or neural ensemble regression. We propose FlowSDR, a likelihood-based framework that jointly learns the projection and the conditional density by maximizing a conditional log-likelihood, with the density parameterized by monotone rational-quadratic spline flows. The
estimator is Fisher consistent under the SDR model, and its sample objective admits a population interpretation in terms of mutual information. As a complementary model within the same likelihood framework, we introduce the neural Gaussian SDR, a heteroscedastic conditional Gaussian model whose mean and variance are parameterized by shared neural-network functions of the projected predictors. In simulations spanning Gaussian errors, heavy-tailed distributions, two-component mixtures, and settings with tail behavior not captured by mean-variance structure, FlowSDR recovers the central subspace more accurately than existing SDR methods and the neural Gaussian SDR baseline. We further validate these advantages on a face-age prediction task using the UTKFace
dataset.
Read the full abstract ▾
All authors Yuexiao Dong, Kenichiro Mcalinn, Edoardo Airoldi, Lei Li
Machine Learning (Statistics) · Cross-listed from cs.LG
Sample Complexity and Decision-Theoretic Guarantees for Bayesian Model Averaging over Decision Trees with Catalan-Exponential Priors
Livija Jakaite and Vitaly Schetinin
We ask: when do
Bayesian model averaging (BMA) weights over decision trees carry sufficient epistemic information to justify committed exploitation of the averaging distribution? We answer this question in closed form for
Bayesian decision trees (BDTs) with Dirichlet-Multinomial leaf models and a Catalan-exponential tree-size prior (Schetinin&Jakaite, 2025), establishing a complete non-asymptotic theory of rational commitment thresholds.
Read the full abstract ▾
Statistical Methodology
Scale-Free Priors and Survival Dynamics: A Bayesian Framework for Conflict Duration
Tomasz F. Stepinski
We have developed a fully
Bayesian survival-analysis framework that reformulates inference about system lifetimes in terms of hazard and survival functions, and extends this representation to interacting actors. Starting from J.~Richard Gott's Copernican principle, we express the scale-free prior as a baseline hazard $\lambda(t)=1/t$, thereby linking a static prior over lifetimes to the dynamic
language of survival analysis. In this formulation,
Bayesian updating corresponds to conditioning on survival, while the resulting posterior distribution admits a natural representation in terms of hazard and survival functions. The approach is intended for settings where data are sparse or unreliable, and where a scale-free, assumption-light baseline is preferable to heavily parameterized models. Building on this foundation, we derive general expressions for two-actor systems that characterize joint survival, conditional lifetimes, and comparative outcomes without requiring a specific parametric form of interaction. This yields a flexible and modular framework in which baseline dynamics are separated from interaction effects, allowing different mechanisms to be incorporated transparently. Thus, the primary contribution is a general hazard-based formulation of
Bayesian updating and its extension to interacting systems To illustrate the framework, we consider a multiplicative resource-depletion specification in which interaction modifies the baseline hazard through cumulative engagement intensity. This example demonstrates how interaction terms can be embedded while preserving analytical tractability, including closed-form expressions under simplifying assumptions. We further provide a stylized application to an asymmetric two-actor conflict, the 2026 US/Israel–Iran hostilities, to highlight the qualitative implications of the approach.
Read the full abstract ▾
Statistics Theory
Statistical Inference on Gradient Flows
Tongyu Li and Alexander Giessing
Gradient-based
algorithms are central to modern statistical estimation, yet their statistical analysis is often restricted to fixed-time behavior, such as convergence to a population target or fluctuations at a prescribed iteration. In many applications, however, uncertainty quantification is needed along the entire optimization path, especially when the stopping time is data-dependent or divergent. In this paper, we develop a theory for time-uniform statistical inference on gradient flows arising from empirical risk minimization. We prove a uniform central limit
theorem that characterizes the deviation between empirical and population gradient flows as a continuous-time Gaussian process over the entire nonnegative real line. Building on this result, we introduce an
algorithm-aware covariance
estimator that evolves jointly with the gradient flow and avoids
matrix inversion, resampling, or sample splitting. We show that the covariance
estimator is uniformly consistent over time and use it to construct confidence intervals for the target parameter with asymptotically valid coverage. Our results connect optimization dynamics with statistical inference and provide practical tools for uncertainty quantification in gradient-based methods.
Read the full abstract ▾
Machine Learning (Statistics)
Distribution-free changepoint localization after sequential change detection
Aytijhya Saha and Aaditya Ramdas
This paper introduces a distribution-free framework for constructing post-detection confidence sets for changepoints after stopping a sequential change detection procedure. It is well known that conformal test martingales can be used to sequentially detect changes in distribution, but by themselves provide no inference for the time at which a proclaimed change occurred. Past work on post-detection inference requires pre- and post-change classes of distributions to be known, but this paper accomplishes localization of the changepoint without any distributional assumptions. We establish finite-sample coverage guarantees (conditional on correct detection). We provide non-asymptotic bounds on the conditional expected size of the confidence sets. Under suitable asymptotic regimes, we prove that the conditional expected size of the confidence set remains uniformly bounded and demonstrate strong empirical performance on simulated and real data. To the best of our knowledge, this is the first general distribution-free framework for sequential changepoint localization with valid post-detection coverage.
Read the full abstract ▾
Machine Learning (Statistics)
Variation Spaces for Encoder–Decoder Neural Operators: Approximation and Generalization
Jia-Qi Yang and Lei Shi
Inspired by the function-space theory of neural networks, we formulate and analyze a variation space for nonlinear operators between
Hilbert spaces, defined through vector-valued Borel measures of bounded variation. We characterize its unit ball as the closed convex hull of a vector-valued single-
neuron dictionary in Bochner spaces. For the ReLU activation, the bounded linear operators in this space are precisely the Schatten-$1$ operators, with equivalent norms. For operators in this space, we establish encoder–decoder approximation bounds in the Bochner $L^q$-norm, where the error decomposes into input and output encoding errors and a finite-width term of order $N^{-1/2}$. Under sub-Gaussian assumptions on the input and noise, we further derive high-probability generalization bounds for empirical least squares over path-norm-constrained encoder–decoder networks; the finite-sample contribution to the squared prediction error is of order $K^{-1/2}$ up to logarithmic factors. The finite-width and finite-sample constants are independent of the encoding dimensions and bases, with the latter also independent of the network width. When the encoding errors decay algebraically, these bounds yield algebraic approximation and learning rates, in contrast to the complexity barriers for Lipschitz and Fréchet differentiable operator classes.
Read the full abstract ▾
Statistical Methodology
Functional Clustering of Survival Data via Smoothed Log-Hazard Trajectories: A Risk-Dynamics Perspective
Anna De Magistris, Elvira Romano and Fabrizio Maturo
This paper investigates clustering in survival data by shifting the analytical focus from cumulative survival probabilities to instantaneous risk, as characterized by the hazard function. We model smoothed log-hazard trajectories as functional objects that capture the temporal
evolution of risk and propose a clustering framework based on Functional Principal Component Analysis applied to B-spline smoothed log-hazard trajectories. The number of retained functional principal components is selected before clustering using a 95% cumulative explained-variance rule, and clustering is then performed on the unstandardized FPCA scores. The proposed methodology is evaluated through simulation studies covering progressively complex scenarios, including overlapping and crossing hazard functions, cohort imbalance, heterogeneous risk profiles, and outlier contamination. The framework is further illustrated on two real-world clinical
datasets, the German Breast Cancer Study and the Primary Biliary Cirrhosis
dataset. Results show that the proposed log-hazard-based functional clustering framework provides an interpretable representation of relative temporal risk dynamics, with competitive internal cohesion and explicit robustness diagnostics when compared with cumulative-survival-based
benchmarks.
Read the full abstract ▾
Statistics Applications · Cross-listed from cs.CV
Analysis of Ethnic Disparities in Autism Spectrum Disorder among Toddlers
Aadithya Prabha Ramaharsha, Deevna Reddy and Uma Ranjan
Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder characterized by challenges in communication and behavior. This study examines the relationship between ethnicity and ASD traits, along with behavioural scores, sex and neonatal jaundice across three ethnic groups: White Europeans, Asians, and Middle Eastern individuals. We perform a logistic regression and show that ethnicity has a significant effect on incidence of ASD. White Europeans are 81% increased risk of ASD and Middle Easterners are at 79% reduced risk of ASD compared to Asians. We also confirm earlier studied which show that neonatal jaundice is a significant predictor of ASD, while male children are at much higher risk of ASD compared to female children. These results suggest the need for diagnostic frameworks and interventions that account for ethnic in the presentation and assessment of ASD traits
Read the full abstract ▾
Statistics Applications
Conditioning-Depth Diagnostics for Hidden Memory in Temporal Causal Discovery
S. A. Adedayo
Reliable causal discovery in timeseries requires conditioning sets that capture the system state. When predictive history is omitted, residual dependence can appear as direct causal links. We test state adequacy by measuring how inferred graphs change as conditioning depth increases while the reported causal lag stays fixed. Under an adequate finite-order Markov representation, graphs should stabilize once enough observed history is conditioned on; latent common drive, omitted lags, nonstationarity, and measurement dynamics can instead produce depth sensitivity. We formalize this idea with graph instability statistics and evaluate c-GC and c-GC*, the two learners whose depth parameter implements a matched fixed-horizon history intervention. PCMCI+ and JPCMCI+ are excluded from the primary comparison because adaptive parent selection makes nominal depth edge-specific. In paired simulations, a clean order-1 process was stable in every repeat, whereas an AR(1) latent common driver produced positive instability in all c-GC repeats and eight of ten c-GC* repeats. In calcium imaging recordings, connectivity drops at the first transition beyond the one-lag baseline and then levels off, but B=200 bootstrap calibration does not reject the fitted order-1 null. The workflow therefore flags hidden
memory or observed-state inadequacy without identifying the generating mechanism or recovering a latent graph.
Read the full abstract ▾
Statistical Methodology
Topological Ignorability for Structural Causal Effects Beyond Means
Usef Faghihi
Many interventions alter the structure of an outcome distribution rather than its mean: they can split a population into disconnected regimes, create loops or holes, generate branches, or reorganize an outcome cloud while leaving the average response nearly unchanged. In such settings, mean-based causal estimands such as the average treatment effect may miss important structural effects. We introduce topological-geometrical causal metrics based on summaries of interventional outcome laws, including density-superlevel Betti summaries, Euler signatures, and persistent-homology summaries. These metrics quantify structural differences between treated and untreated outcome laws beyond averages. We also study the assumptions needed for causal interpretation. We introduce topological ignorability, a topological analogue of conditional ignorability that requires invariance of the chosen structural feature rather than the full counterfactual distribution. When the chosen summary is injective, this condition coincides with weak ignorability; for noninjective summaries, it can identify the structural feature of interest without identifying the full interventional law. We define a
covariate-standardized topological-geometrical causal effect and develop practical
estimators. We validate the framework in two hidden-confounding
benchmarks: a fully synthetic exact
benchmark and a real-
covariate semi-synthetic
benchmark using Wisconsin breast-cancer
covariates. In both, weak ignorability fails and balancing observed
covariates nearly eliminates standardized mean differences, yet the coordinate-mean average treatment effect remains biased. By contrast, selected finite density-superlevel Betti and Euler contrasts remain stable across oracle, observational, and weighted analyses.
Read the full abstract ▾
Statistical Methodology · Cross-listed from cs.LG
Revisiting Neural Processes via Fourier Transform and Volterra Series
Peiman Mohseni, Nick Duffield and Raymond K. W. Wong
Modeling unknown latent functions from finite, irregularly sampled measurements is a recurring challenge across science and engineering. Neural processes (NPs), a family of probabilistic functional models, are promising solutions – especially when endowed with domain-specific
symmetries like translation equivariance, which improve sample efficiency and generalization. Yet existing translation-equivariant NPs face two limitations: (i) they stack generic components with non-linearities, obscuring the induced function class and limiting interpretability; and (ii) convolutional designs are limited by local receptive fields and the need to embed inputs onto a dense uniform grid, while
attention-based alternatives lift these restrictions at quadratic cost in the number of observations. We address both with two contributions. First, using the Volterra expansion, we approximate continuous translation-equivariant operators by sums of higher-order convolutions, yielding analytical transparency while admitting efficient evaluation via first-order convolutions. Second, we introduce set
Fourier convolutions (SFConvs), a frequency-domain parameterization that operates directly on irregularly sampled points, achieves approximately global receptive fields, and scales linearly in the number of observations. Building on these ideas, we propose two conditional NPs (CNPs): SFConvCNPs, which stack SFConv blocks with non-linearities, and SFVConvCNPs, which integrate the Volterra formulation. Experiments on synthetic and real-world
datasets demonstrate our methods' efficacy against state-of-the-art baselines.
Read the full abstract ▾
Statistical Methodology
Measuring the Symmetry–Data Exchange Rate
Ahmed M. Adly
Equivariance theory predicts that an architectural
symmetry prior reduces sample complexity by a factor of |G|; this is widely cited but rarely measured as a scaling law with controls that separate the prior from its confounds. On a controlled C_n-symmetric task, we report three findings. First, a wrong-group control with identical orbit size and matched compute is worse than no constraint (joint pairwise CI [+0.79, +3.26] excludes zero, robust across
estimators); misaligned constraint is actively harmful, not merely unhelpful. Second, an augmentation baseline equipped with test-time orbit averaging matches the equivariant model exactly – bit-identical per-epoch validation curves across matched
cells – so the architecture-vs-augmentation gap is conditional on asymmetric test-time computation, not unconditional. Third, the relative exchange rate beta_diff = 1.28 is consistent in sign and order of magnitude with the theoretical 1.0 (single-level CI [+0.92, +2.05]); the more conservative two-level bootstrap (seeds x group sizes) widens this to [-0.63, +1.72], including zero, and a finer-N replication on a sqrt(2)-spaced grid is inconclusive (point estimate -0.82). The methodological contributions – the relative-rate
estimator that cancels the shared-difficulty confound, the wrong-group control, and a pre-specified failure taxonomy – transfer to any inductive bias whose strength can be parameterised. Honest scoping: the primary
estimator beta_diff was adopted post-hoc after the initial analysis revealed a positive-slope identifiability problem; the design was never externally pre-registered; and the headline number rests on an OLS slope over seven group sizes on a coarse N grid. This is an exploratory study, not a confirmatory measurement; the wrong-group result is the cleanest finding and the one we report with the most confidence. A registered replication on fresh seeds is future work.
Read the full abstract ▾