Statistical Methodology
Sparse-Group Factor Analysis for High-Dimensional Time Series
Xin Wang and Xialu Liu
Factor analysis is a widely used technique for dimension reduction in high-dimensional data. However, a key challenge in factor models lies in the interpretability of the latent factors. One intuitive way to interpret these factors is through their associated loadings. Liu and Wang proposed a novel framework that redefines factor models with sparse loadings to enhance interpretability. In many high-dimensional time series applications, variables exhibit natural group structures. Building on this idea, our paper incorporates domain knowledge and prior information by modeling both individual sparsity and group sparsity in the loading
matrix. This dual-sparsity framework further improves the interpretability of the estimated factors. We develop an
algorithm to estimate both the loading
matrix and the common component, and we establish the asymptotic properties of the resulting
estimators. Simulation studies demonstrate the strong performance of the proposed method, and a real-data application illustrates how incorporating prior knowledge leads to more interpretable results.
Read the full abstract ▾
Statistical Methodology
A new composite Mann-Whitney test for two-sample survival comparisons with right-censored data
Abid Hussain and Touqeer Ahmad
A fundamental challenge in comparing two survival distributions with right censored data is the selection of an appropriate nonparametric test, as the power of standard tests like the Log rank and Wilcoxon is highly dependent on the often unknown nature of the alternative hypothesis. This paper introduces a new, distribution free two sample test designed to overcome this limitation. The proposed method is based on a strategic decomposition of the data into uncensored and censored subsets, from which a composite test statistic is constructed as the sum of two independent Mann Whitney statistics. This design allows the test to automatically and inherently adapt to various patterns of difference including early, late, and crossing hazards without requiring pre specified parameters, pre testing, or complex weighting schemes. An extensive Monte Carlo simulation study demonstrates that the proposed test robustly maintains the nominal Type I error rate. Crucially, its power is highly competitive with the optimal traditional tests in standard scenarios and superior in complex settings with crossing survival curves, while also exhibiting remarkable robustness to high levels of censoring. The test power effectively approximates the maximum power achievable by either the Log rank or Wilcoxon tests across a wide range of alternatives, offering a powerful, versatile, and computationally simple tool for survival analysis.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.AI
Integrating Bayesian methods with neural network–based model predictive control: a review
Asli Karacelik
In this review, we assess the use of
Bayesian methods in model predictive control (MPC), focusing on neural-network-based modeling, control design, and uncertainty quantification. We systematically analyze individual studies and how they are implemented in practice. While
Bayesian approaches are increasingly adopted to capture and propagate uncertainty in MPC, reported gains in performance and robustness remain fragmented, with inconsistent baselines and limited reliability analyses. We therefore argue for standardized
benchmarks, ablation studies, and transparent reporting to rigorously determine the effectiveness of
Bayesian techniques for MPC.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Tensor-on-tensor Regression Neural Networks for Process Modeling with High-dimensional Data
Qian Wang, Mohammad N. Bisheh and Kamran Paynabar
Modern sensing and metrology systems now stream terabytes of heterogeneous, high-dimensional (HD) data profiles, images, and dense point clouds, whose natural representation is multi-way tensors. Understanding such data requires regression models that preserve tensor geometry, yet remain expressive enough to capture the pronounced nonlinear interactions that dominate many industrial and mechanical processes. Existing tensor-based regressors meet the first requirement but remain essentially linear. Conversely, conventional neural networks offer nonlinearity only after flattening, thereby discarding spatial structure and incurring prohibitive parameter counts. This paper introduces a Tensor-on-Tensor Regression Neural Network (TRNN) that unifies these two paradigms.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Computing frustration and near-monotonicity in deep neural networks
Joel Wendin, Erik G. Larsson and Claudio Altafini
For the signed graph associated to a deep neural network, one can compute the frustration level, i.e., test how close or distant the graph is to structural balance. For all the pretrained deep convolutional neural networks we consider, we find that the frustration is always less than expected from null models. From a statistical physics point of view, and in particular in reference to an Ising
spin glass model, the reduced frustration indicates that the amount of disorder encoded in the network is less than in the null models. From a functional point of view, low frustration (i.e., proximity to structural balance) means that the function representing the network behaves near-monotonically, i.e., more similarly to a monotone function than in the null models. Evidence of near-monotonic behavior along the partial order determined by frustration is observed for all networks we consider. This confirms that the class of deep convolutional neural networks tends to have a more ordered behavior than expected from null models, and suggests a novel form of implicit regularization.
Read the full abstract ▾
Statistical Methodology
An efficient hybrid approach of quantile and expectile regression
Abdellah Atanane, Abdallah Mkhadri and Karim Oualkacha
Quantiles and expectiles are determined by different loss functions: asymmetric least absolute deviation for
quantiles and asymmetric squared loss for expectiles. This distinction ensures that
quantile regression methods are robust to outliers but somewhat less effective than expectile regression, especially for normally distributed data. However, expectile regression is vulnerable to lack of robustness, especially for heavy-tailed distributions. To address this trade-off between robustness and effectiveness, we propose a novel approach. By introducing a parameter $\gamma$ that ranges between 0 and 1, we combine the aforementioned loss functions, resulting in a hybrid approach of
quantiles and expectiles. This fusion leads to the estimation of a new type of location parameter family within the linear regression framework, termed Hybrid of
Quantile and Expectile Regression (HQER). The asymptotic properties of the resulting estimaror are then established. Through simulation studies, we compare the asymptotic relative efficiency of the HQER
estimator with its competitors, namely the
quantile, expectile, and $k$th power expectile regression
estimators. Our results show that HQER outperforms its competitors in several simulation scenarios. In addition, we apply HQER to a real
dataset to illustrate its practical utility.
Read the full abstract ▾
Statistics Theory · Cross-listed from math.AG
Hadamard ranks of algebraic varieties
Dario Antolini, Guido Montúfar and Alessandro Oneto
Motivated by the study of decompositions of tensors as Hadamard products (i.e., coefficient-wise products) of low-rank tensors, we introduce the notion of Hadamard rank of a given point with respect to a projective variety: if it exists, it is the smallest number of points in the variety such that the given point is equal to their Hadamard product. We prove that if the variety $X$ is not contained in a coordinate hyperplane or a binomial hypersurface, then the generic point has a finite $X$-Hadamard-rank. Although the Hadamard rank might not be well defined for special points, we prove that the general Hadamard rank with respect to secant varieties of toric varieties is finite and the maximum Hadamard rank for points with no coordinates equal to zero is at most twice the generic rank. In particular, we focus on Hadamard ranks with respect to secant varieties of toric varieties since they provide a
geometric framework in which Hadamard decompositions of tensors can be interpreted. Finally, we give a lower bound to the dimension of Hadamard products of secant varieties of toric varieties: this allows us to deduce the general Hadamard rank with respect to secant varieties of several Segre-Veronese varieties.
Read the full abstract ▾
Statistics Applications · Cross-listed from cs.AI
Efficient Prediction of Pass@k Scaling in Large Language Models
Joshua Kazdan, Rylan Schaeffer, Youssef Allouah et al. (+4 more)
Assessing the capabilities and risks of frontier AI systems is a critical area of research, and recent work has shown that repeated sampling from models can dramatically increase both. For instance, repeated sampling has been shown to increase their capabilities, such as solving difficult math and coding problems, but it has also been shown to increase their potential for harm, such as being jailbroken. Such results raise a crucial question for both capability and safety
forecasting: how can one accurately predict a model's behavior when scaled to a massive number of attempts, given a vastly smaller sampling budget? This question is directly relevant to model providers, who serve hundreds of millions of users daily, and to governmental regulators, who seek to prevent harms. To answer this questions, we make three contributions. First, we find that standard methods for fitting these laws suffer from statistical shortcomings that hinder predictive accuracy, especially in data-limited scenarios. Second, we remedy these shortcomings by introducing a robust estimation framework, which uses a beta-binomial distribution to generate more accurate predictions from limited data. Third, we propose a dynamic sampling strategy that allocates a greater budget to harder problems. Combined, these innovations enable more reliable prediction of rare risks and capabilities at a fraction of the computational cost.
Read the full abstract ▾
All authors Joshua Kazdan, Rylan Schaeffer, Youssef Allouah, Colin Sullivan, Kyssen Yu, Noam Levi, Sanmi Koyejo
Machine Learning (Statistics) · Cross-listed from cs.LG
TopInG: Topologically Interpretable Graph Learning via Persistent Rationale Filtration
Cheng Xin, Fan Xu, Xin Ding et al. (+2 more)
Graph Neural Networks (GNNs) have shown remarkable success across various scientific fields, yet their adoption in critical decision-making is often hindered by a lack of interpretability. Recently, intrinsically interpretable GNNs have been studied to provide insights into model predictions by identifying rationale substructures in graphs. However, existing methods face challenges when the underlying rationale subgraphs are complex and varied. In this work, we propose TopInG: Topologically Interpretable Graph Learning, a novel topological framework that leverages persistent homology to identify persistent rationale subgraphs. TopInG employs a rationale filtration learning approach to model an autoregressive generation process of rationale subgraphs, and introduces a self-adjusted topological constraint, termed topological discrepancy, to enforce a persistent topological distinction between rationale subgraphs and irrelevant counterparts. We provide theoretical guarantees that our loss function is uniquely optimized by the ground truth under specific conditions. Extensive experiments demonstrate TopInG's effectiveness in tackling key challenges, such as handling variform rationale subgraphs, balancing predictive performance with interpretability, and mitigating spurious correlations. Results show that our approach improves upon state-of-the-art methods on both predictive accuracy and interpretation quality.
Read the full abstract ▾
All authors Cheng Xin, Fan Xu, Xin Ding, Jie Gao, Jiaxin Ding
Statistical Methodology
WOW: WAIC-Optimized Gating of Mixture Priors for External Data Borrowing
Shouhao Zhou, Qiuxin Gao, Chenqi Fu et al. (+1 more)
The integration of external data using
Bayesian mixture priors has become a powerful approach in clinical trials, offering significant potential to improve trial efficiency. Despite their strengths in analytical tractability and practical flexibility, existing methods such as the robust meta-analytic-predictive (rMAP) and self-adapting mixture (SAM) often presume borrowing without rigorously assessing whether external information is appropriate to incorporate. When external and concurrent data are discordant, excessive borrowing can bias estimation and lead to misleading conclusions. To address this, we introduce WOW, a Kullback-Leibler-based gating strategy guided by the widely applicable information criterion (WAIC). Within the mixture-prior framework, WAIC-Optimized Weighting (WOW) conducts a preliminary compatibility assessment between external and concurrent trial data to determine eligibility for borrowing. Only if this gating criterion is satisfied does borrowing proceed; a downstream mixture prior procedure, using user-specified fixed or adaptive weights, can then be applied to determine the amount of borrowing. Simulation studies demonstrate that incorporating the WOW strategy before
Bayesian mixture prior borrowing methods effectively mitigates excessive borrowing and improves estimation accuracy. A real-data illustration further highlights the feasibility and interpretability of the proposed gate-then-borrow strategy. By providing a practical safeguard against inappropriate borrowing, WOW strengthens the reliability of mixture-prior methods and supports better decision-making in clinical trials.
Read the full abstract ▾
All authors Shouhao Zhou, Qiuxin Gao, Chenqi Fu, Yanxun Xu
Statistics Theory · Cross-listed from cs.LG
ResCP: Reservoir Conformal Prediction for Time Series Forecasting
Roberto Neglia, Andrea Cini, Michael M. Bronstein et al. (+1 more)
Conformal prediction offers a powerful framework for building distribution-free prediction intervals for exchangeable data. Existing methods that extend conformal prediction to sequential data rely on fitting a relatively complex model to capture temporal dependencies. However, these methods can fail if the sample size is small and often require expensive retraining when the underlying data distribution changes. To overcome these limitations, we propose Reservoir Conformal Prediction (ResCP), a novel training-free conformal prediction method for time series. Our approach leverages the efficiency and representation learning capabilities of reservoir computing to dynamically reweight conformity scores. In particular, we compute similarity scores among reservoir states and use them to adaptively reweight the observed residuals at each step. With this approach, ResCP enables us to account for local temporal dynamics when modeling the error distribution without compromising computational scalability. We prove that, under reasonable assumptions, ResCP achieves asymptotic conditional coverage, and we empirically demonstrate its effectiveness across diverse
forecasting tasks.
Read the full abstract ▾
All authors Roberto Neglia, Andrea Cini, Michael M. Bronstein, Filippo Maria Bianchi
Machine Learning (Statistics)
Causal Abstractions, Categorically Unified
Markus Englberger and Devendra Singh Dhami
We present a categorical framework for relating causal models that represent the same system at different levels of abstraction. We define a causal abstraction as natural transformations between appropriate Markov functors, which concisely consolidate desirable properties a causal abstraction should exhibit. Our approach unifies and generalizes previously considered causal abstractions, and we obtain categorical proofs and generalizations of existing results on causal abstractions. Using string diagrammatical tools, we can explicitly describe the graphs that serve as consistent abstractions of a low-level graph under interventions. We discuss how methods from mechanistic interpretability, such as
circuit analysis and sparse autoencoders, fit within our categorical framework. We also show how applying do-calculus on a high-level graphical abstraction of an acyclic-directed mixed graph (ADMG), when unobserved confounders are present, gives valid results on the low-level graph, thus generalizing an earlier statement by Anand et al. (2023). We argue that our framework is more suitable for modeling causal abstractions compared to existing categorical frameworks. Finally, we discuss how notions such as $\tau$-consistency and constructive $\tau$-abstractions can be recovered with our framework.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Rethinking Langevin Thompson Sampling from A Stochastic Approximation Perspective
Weixin Wang, Haoyang Zheng, Guang Lin et al. (+2 more)
Langevin-based Thompson Sampling (TS) methods for multi-armed bandits use likelihood gradients to approximate posterior samples when exact sampling is intractable. As new rewards arrive, the posterior changes, and these methods run an inner Markov chain to approximate the updated target. To simplify this repeated sampling procedure, we introduce TS-SA, which incorporates
stochastic approximation (SA) within the TS framework. In each round, TS-SA takes one Langevin step using the newly observed reward and applies an SA update to combine the noisy proposal with the current parameter estimate. By retaining this estimate across rounds, TS-SA maintains one continuing Markov chain for each arm, preserving the influence of past observations without storing or sampling them again. Within this recursion, the proposal step size remains fixed across rounds, while the SA weights decrease as observations accumulate to control the effect of noise on the estimate. This continuing-chain formulation simplifies the theoretical analysis, allowing us to establish a near-optimal worst-case regret bound of $\widetilde{\mathcal O}(\sqrt{KT})$ under log-concavity and smoothness assumptions with one update per new reward. Our experiments further demonstrate that TS-SA, using a single-step Langevin update after an initial warm-up, achieves strong performance against standard baselines.
Read the full abstract ▾
All authors Weixin Wang, Haoyang Zheng, Guang Lin, Wei Deng, Pan Xu
Machine Learning (Statistics)
Curiosity-Driven Development of Action and Language in Robots Through Self-Exploration
Theodore Jerome Tinker, Kenji Doya and Jun Tani
Infants acquire
language with generalization from minimal experience, whereas large
language models require billions of training
tokens. What underlies efficient development in humans? We investigated this problem through experiments wherein robotic
agents learn to perform actions associated with imperative sentences (e.g., push red cube) via curiosity-driven self-exploration. Our approach amortizes active inference using Q-learning, enabling intrinsically motivated developmental learning. The simulations reveal key findings corresponding to observations in developmental psychology. i) Generalization improves drastically as the scale of compositional elements increases. ii) Curiosity-driven exploration enables faster learning. iii) Rote pairing of sentences and actions precedes compositional generalization. iv) Exception-handling induces U-shaped developmental performance, a pattern like representational redescription in child
language learning. These results suggest that curiosity-driven active inference accounts for how intrinsically motivated sensorimotor-linguistic learning supports scalable compositional generalization and exception handling in humans and artificial
agents.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Reinforce-Ada: An Adaptive Sampling Framework under Non-linear RL Objectives
Wei Xiong, Chenlu Ye, Baohao Liao et al. (+6 more)
Reinforcement learning (RL) for large
language model
reasoning is frequently hindered by signal loss, a phenomenon where standard uniform sampling with small group sizes fails to uncover informative learning signals for difficult
prompts. We demonstrate that this collapse is a statistical artifact of undersampling rather than an inherent model limitation. To address this systematically, we introduce a theoretical framework based on optimizing a non-linear RL objective (e.g., log-likelihood). We show that this objective naturally induces a weighted gradient
estimator that prioritizes difficult
prompts, which can be robustly realized through adaptive sampling. Guided by this framework, we propose Reinforce-Ada, a family of
algorithms that dynamically allocates inference budgets based on
prompt difficulty, effectively scaling up RL compute to where it is needed most. Unlike passive filtering methods that discard low-signal
prompts, Reinforce-Ada actively invests compute to recover them. We introduce two efficient realizations: an estimation-based approach and a model-free sequential sampling approach. Extensive experiments across multiple
benchmarks show that Reinforce-Ada significantly outperforms uniform baselines like GRPO, recovering lost signals and accelerating convergence by up to $2\times$ while maintaining the same total inference budget. Code is available at https://github.com/RLHFlow/Reinforce-Ada.
Read the full abstract ▾
All authors Wei Xiong, Chenlu Ye, Baohao Liao, Hanze Dong, Xinxing Xu, Christof Monz, Jiang Bian, Nan Jiang, Tong Zhang
Statistics Theory
Structural Identifiability of Graphical Continuous Lyapunov Models
Carlos Améndola, Tobias Boege, Benjamin Hollering et al. (+1 more)
We prove two characterizations of model equivalence of acyclic graphical continuous Lyapunov models (GCLMs) with uncorrelated noise. The first result shows that two graphs are model equivalent if and only if they have the same skeleton and equivalent induced 4-node subgraphs. We also give a transformational characterization via structured edge reversals. The two
theorems are Lyapunov analogues of celebrated results for
Bayesian networks by Verma and Pearl, and Chickering, respectively. Our results have broad consequences for the theory of causal inference of GCLMs. First, we find that model equivalence classes of acyclic GCLMs refine the corresponding classes of
Bayesian networks. Furthermore, we obtain
polynomial-time
algorithms to test model equivalence and structural identifiability of given directed acyclic graphs.
Read the full abstract ▾
All authors Carlos Améndola, Tobias Boege, Benjamin Hollering, Pratik Misra
Statistics Theory
Pivotal CLTs for Pseudolikelihood via Conditional Centering in Dependent Random Fields
Nabarun Deb
In this paper, we study fluctuations of conditionally centered statistics of the form $$N^{-1/2}\sum_{i=1}^N c_i(g(\sigma_i)-\mathbb{E}_N[g(\sigma_i)|\sigma_j,j\neq i])$$ where $(\sigma_1,\ldots ,\sigma_N)$ are sampled from a dependent random field, and $g$ is some bounded function. Our first main result shows that under weak smoothness assumptions on the conditional means (which cover both sparse and dense interactions), the above statistic converges to a Gaussian scale mixture with a random scale determined by a quadratic variance and an interaction component. We also show that under appropriate studentization, the limit becomes a pivotal Gaussian. We leverage this theory to develop a general asymptotic framework for maximum pseudolikelihood (MPLE) inference in dependent random fields. We apply our results to Ising models with pairwise as well as higher-order interactions and exponential random graph models (ERGMs). In particular, we obtain a joint central limit
theorem for the inverse temperature and magnetization parameters via the joint MPLE (to our knowledge, the first such result in dense, irregular regimes), and we derive conditionally centered edge CLTs and marginal MPLE CLTs for ERGMs without restricting to the “sub-critical" region. Our proof is based on a method of moments approach via combinatorial decision-tree
pruning, which may be of independent interest.
Read the full abstract ▾
Machine Learning (Statistics)
Embracing Discrete Search: A Reasonable Approach to Causal Structure Learning
Marcel Wienöbst, Leonard Henckel and Sebastian Weichwald
We present FLOP (Fast Learning of Order and Parents), a score-based causal discovery
algorithm for linear models. It pairs fast parent selection with iterative Cholesky-based score updates, cutting run-times over prior
algorithms. This makes it feasible to fully embrace discrete search, enabling iterated local search with principled order initialization to find graphs with scores at or close to the global optimum. The resulting structures are highly accurate across
benchmarks, with near-perfect recovery in standard settings. This performance calls for revisiting discrete search over graphs as a reasonable approach to causal discovery.
Read the full abstract ▾
Statistical Methodology · Cross-listed from cs.CL
Mind Your Tone: Investigating How Prompt Politeness Affects LLM Accuracy (short paper)
Om Dobariya and Akhil Kumar
The wording of natural
language prompts has been shown to influence the performance of large
language models (LLMs), yet the role of politeness and tone remains underexplored. In this study, we investigate how varying levels of
prompt politeness affect model accuracy on multiple-choice questions. We created a
dataset of 50 base questions spanning mathematics, science, and history, each rewritten into five tone variants: Very Polite, Polite, Neutral, Rude, and Very Rude, yielding 250 unique
prompts. Using ChatGPT 4o, we evaluated responses across these conditions and applied paired sample t-tests to assess statistical significance. Contrary to expectations, impolite
prompts consistently outperformed polite ones, with accuracy ranging from 80.8% for Very Polite
prompts to 84.8% for Very Rude
prompts. These findings differ from earlier studies that associated rudeness with poorer outcomes, suggesting that newer LLMs may respond differently to tonal variation. Our results highlight the importance of studying pragmatic aspects of prompting and raise broader questions about the social dimensions of human-AI interaction.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
On Structured State-Space Duality
Jerry Yao-Chieh Hu, Xiwen Zhang, Ali ElSheikh et al. (+2 more)
Structured State-Space Duality (SSD) [Dao & Gu, ICML 2024] is an equivalence between a simple Structured State-Space Model (SSM) and a masked
attention mechanism. In particular, a state-space model with a scalar-times-identity state
matrix is equivalent to a masked self-
attention with a $1$-semiseparable causal mask. Consequently, the same sequence transformation (model) has two algorithmic realizations: as a linear-time $O(T)$ recurrence or as a quadratic-time $O(T^2)$
attention. In this note, we formalize and generalize this duality: (i) we extend SSD from the scalar-identity case to general diagonal SSMs (diagonal state
matrices); (ii) we show that these diagonal SSMs match the scalar case's training complexity lower bounds while supporting richer dynamics; (iii) we establish a necessary and sufficient condition under which an SSM is equivalent to $1$-semiseparable masked
attention; and (iv) we show that such duality fails to extend to standard softmax
attention due to rank explosion. Together, these results tighten bridge between recurrent SSMs and
Transformers, and widen the design space for expressive yet efficient sequence models.
Read the full abstract ▾
All authors Jerry Yao-Chieh Hu, Xiwen Zhang, Ali ElSheikh, Weimin Wu, Han Liu
Machine Learning (Statistics) · Cross-listed from cs.DM
Abigail Hickok and Andrew J. Blumberg
Read the full abstract ▾
Statistical Methodology
MCMC for State Space models
Paul Fearnhead and Chris Sherlock
A state-space model is a time-series model that has an unobserved latent process from which we take noisy measurements over time. The observations are conditionally independent given the latent process and the latent process itself is Markovian. These properties lead to simplifications for the conditional distribution of the latent process given the parameters and the observations. This chapter looks at how we can leverage the properties of state-space models to construct efficient MCMC samplers. We consider a range of Gibbs-sampler schemes, including those which use the forward-backward
algorithm to simulate from the full conditional of the latent process given the parameters. For models where the forward-backward
algorithm is not applicable we look at particle MCMC
algorithms that, given the parameters, use particle filters to approximately simulate from the latent process or estimate the likelihood of the observations. Throughout, we provide intuition and informally discuss theory about the properties of the model that impact the efficiency of the different
algorithms and how approaches such as reparameterization can improve mixing.
Read the full abstract ▾
Machine Learning (Statistics)
Set to Be Fair: Demographic Parity Constraints for Set-Valued Classification
Eyal Cohen, Christophe Denis and Mohamed Hebiri
Set-valued classification is used in multiclass settings where confusion between classes can occur and lead to misleading predictions. However, its application may amplify discriminatory bias motivating the development of set-valued approaches under
fairness constraints. In this paper, we address the problem of set-valued classification under demographic parity and expected size constraints. We propose two complementary strategies: an oracle-based method that minimizes classification risk while satisfying both constraints, and a computationally efficient proxy that prioritizes constraint satisfaction. For both strategies, we derive closed-form expressions for the (optimal) fair set-valued classifiers and use these to build plug-in, data-driven procedures for empirical predictions. We establish distribution-free convergence rates for violations of the size and
fairness constraints for both methods, and under mild assumptions we also provide excess-risk bounds for the oracle-based approach. Empirical results demonstrate the effectiveness of both strategies and highlight the efficiency of our proxy method.
Read the full abstract ▾
Statistical Methodology
Selection Bias in Hybrid Randomized Controlled Trials using External Controls: A Simulation Study
Han Chang Chiam, Franz König and Martin Posch
Hybrid randomized controlled trials (hybrid RCTs) integrate external control data, such as historical or concurrent data, with data from randomized trials. While numerous frequentist and
Bayesian methods, such as the test-then-pool and Meta-Analytic-Predictive prior, have been developed to account for potential disagreement between the external control and randomized data, they cannot ensure strict type I error rate control. However, these methods can reduce biases stemming from systematic differences between external controls and trial data. A critical yet underexplored issue in hybrid RCTs is the prespecification of external data to be used in analysis. The validity of statistical conclusions in hybrid RCTs depends on the assumption that external control selection is independent of historical trials outcomes. In practice, historical data may be accessible during the planning stage, potentially influencing important decisions, such as which historical
datasets to include or the sample size of the prospective part of the hybrid trial, thus introducing bias. Such data-driven design choices can be an additional source of bias, which can occur even when historical and prospective controls are exchangeable. Through a simulation study, we quantify the biases introduced by outcome-dependent selection of historical controls in hybrid RCTs using both
Bayesian and frequentist approaches, and discuss potential strategies to mitigate this bias. Our scenarios consider variability and time trends in the historical studies, distributional shifts between historical and prospective control groups, sample sizes and allocation ratios, as well as the number of studies included. The impact of different rules for selecting external controls is demonstrated using a clinical trial example.
Read the full abstract ▾
Machine Learning (Statistics)
A Noise Resilient Approach for Robust Hurst Exponent Estimation
Malith Premarathna, Fabrizio Ruggeri and Dixon Vimalajeewa
Understanding signal behavior across scales is vital in areas such as natural phenomena analysis and financial modeling. A key property is self-similarity, quantified by the Hurst exponent (H), which reveals long-term dependencies. Wavelet-based methods are effective for estimating H due to their multi-scale analysis capability, but additive noise in real-world measurements often degrades accuracy. We propose Noise-Controlled ALPHEE (NC-ALPHEE), an enhancement of the Average Level-Pairwise Hurst Exponent
Estimator (ALPHEE), incorporating noise mitigation and generating multiple level-pairwise estimates from signal energy pairs. A neural network (NN) combines these estimates, replacing traditional averaging. This adaptive learning maintains ALPHEE's behavior in noise-free cases while improving performance in noisy conditions. Extensive simulations show that in noise-free data, NC-ALPHEE matches ALPHEE's accuracy using both averaging and NN-based methods. Under noise, however, traditional averaging deteriorates and requires impractical level restrictions, while NC-ALPHEE consistently outperforms existing techniques without such constraints. NC-ALPHEE offers a robust, adaptive approach for H estimation, significantly enhancing the reliability of wavelet-based methods in noisy environments.
Read the full abstract ▾
Statistics Applications · Cross-listed from q-bio.PE
Ecosystem Recovery to Historical Targets Becomes Unattainable Under Modelled Fishing and Climate in the Barents Sea
Matthew Hatton, Jack H Laverick, Neil Banas et al. (+1 more)
Climate change and fisheries jointly shape the resilience of the Barents Sea marine ecosystem, yet the recovery of key fish populations to
climate and anthropogenic disturbances requires further investigation. This study examines how fishing pressure and
climate change, driven by the NEMO-MEDUSA Earth system model, influence the recovery times of Demersal and Planktivorous fish in the Barents Sea. We used the StrathE2EPolar end-to-end ecosystem model to simulate transient dynamics under increasing fishing pressure scenarios, and quantified recovery times for Demersal, Planktivorous, and ecosystem-wide groups relative to a shifting unfished baseline. Recovery times increased with both fishing intensity and
climate change, by as much as 18 years for Demersal fish and 54 years for Planktivorous fish across all fishing scenarios. At the ecosystem level, recovery was constrained by the slow rebound of top predators, many of which experienced biomass collapse under
climate change, preventing recovery to a shifting baseline. Our results suggest that fishing pressure in tandem with
climate change substantially reduces ecosystem resilience, highlighting the importance of sustainable harvest strategies in a changing
climate.
Read the full abstract ▾
All authors Matthew Hatton, Jack H Laverick, Neil Banas, Michael Heath
Machine Learning (Statistics)
Kernel ridge regression under power-law data: spectrum and generalization
Arie Wortsman and Bruno Loureiro
In this work, we investigate high-dimensional kernel ridge regression (KRR) on i.i.d. Gaussian data with anisotropic power-law covariance. This setting differs fundamentally from the classical source & capacity conditions for KRR, where power-law assumptions are typically imposed on the kernel eigen-spectrum itself. Our contributions are twofold. First, we derive an explicit characterization of the kernel spectrum for
polynomial inner-product kernels, giving a precise description of how the kernel eigen-spectrum inherits the data decay. Second, we provide an asymptotic analysis of the excess risk in the high-dimensional regime for a particular kernel with this spectral behavior, showing that the sample complexity is governed by the effective dimension of the data rather than the ambient dimension. These results establish a fundamental advantage of learning with power-law anisotropic data over isotropic data. To our knowledge, this is the first rigorous treatment of non-linear KRR under power-law data.
Read the full abstract ▾
Statistics Theory · Cross-listed from cs.LG
When Do Credal Sets Stabilize? Fixed-Point Theorems for Credal Set Updates
Michele Caprio, Siu Lun Chau and Krikamol Muandet
Many machine learning
algorithms rely on iterative updates of uncertainty representations, ranging from variational inference and expectation-maximization, to reinforcement learning, continual learning, and multi-
agent learning. In the presence of imprecision and ambiguity, credal sets – closed, convex sets of probability distributions – have emerged as a popular framework for representing imprecise probabilistic beliefs. Under such imprecision, many learning problems in imprecise probabilistic machine learning (IPML) may be viewed as processes involving successive applications of update rules on credal sets. This naturally raises the question of whether this iterative process converges to stable fixed points – or, more generally, under what conditions on the updating mechanism such fixed points exist, and whether they can be attained. We provide the first analysis of this problem, and illustrate our findings using Credal
Bayesian Deep Learning as a concrete example. Our work demonstrates that incorporating imprecision into the learning process not only enriches the representation of uncertainty, but also reveals structural conditions under which stability emerges, thereby offering new insights into the dynamics of iterative learning under imprecision.
Read the full abstract ▾
Machine Learning (Statistics)
Fisher-Bingham-like normalizing flows on the sphere
Thorsten Glüsenkamp
A generic D-dimensional Gaussian can be conditioned or projected onto the D-1 unit sphere, thereby leading to the well-known Fisher-Bingham (FB) or Angular Gaussian (AG) distribution families, respectively. These are some of the most fundamental distributions on the sphere, yet cannot straightforwardly be written as a normalizing flow except in two special cases: the von-Mises Fisher in D=3 and the central angular Gaussian in any D. In this paper, we describe how to generalize these special cases to a family of normalizing flows that behave similarly to the full FB or AG family in any D. We call them "zoom-linear-project" (ZLP)-Fisher flows. Unlike a normal Fisher-Bingham distribution, their composition allows to gradually add complexity as needed. Furthermore, they can naturally handle conditional density estimation with target distributions that vary by orders of magnitude in scale - a setting that is important in astronomical applications but that existing flows often struggle with. A particularly useful member of the new family is the Kent analogue that can cheaply upgrade any flow in this situation to yield better performance.
Read the full abstract ▾
Statistics Theory
Deconvolution of arbitrary distribution functions and densities
Henrik Kaiser
In this article we propose a novel approach for the deconvolution of the distribution function associated with an arbitrary probability measure (and possibly existing density). We first show that the initial convolution equation always can be transformed to a convolution equation that involves a symmetric distribution function, whose characteristic function has its values in the unit interval. As a consequence, the characteristic function of the target measure turns out as the limit of a
geometric series. By truncation of this series, approximations for distribution function and density are established. The convergence properties of these approximations are examined in detail across diverse setups.
Read the full abstract ▾