Peer Revenue An exchange of ideas

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Statistics

stat Watch Statistics (sign in to use)

Wednesday 29 April

Statistics Theory · Cross-listed from math.PR

Sharp One-Dimensional Sub-Gaussian Comparison in Convex Order

We prove that any random variable $X$ whose moment generating function is point-wise upper bounded by that of $ G \sim \mathcal{N}(0,1) $ must be dominated by $ G/\mathbb{E}[|G|] $ in convex order, meaning $ \mathbb{E}[f(X)] \le \mathbb{E}[f(G/\mathbb{E}[|G|])] $ for all convex $f$. This is sharp as witnessed by $ X \sim \mathrm{Unif}(\{-1,1\}) $ and $ f(x) = |x| $.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Semi-supervised learning with max-margin graph cuts

This paper proposes a novel algorithm for semisupervised learning. This algorithm learns graph cuts that maximize the margin with respect to the labels induced by the harmonic function solution. We motivate the approach, compare it to existing work, and prove a bound on its generalization error. The quality of our solutions is evaluated on a synthetic problem and three UCI ML repository datasets. In most cases, we outperform manifold regularization of support vector machines, which is a state-of-the-art approach to semi-supervised max-margin learning.
Read the full abstract

All authors Branislav Kveton, Michal Valko, Ali Rahimi, Ling Huang

Statistical Methodology

Minimum Covariance Determinant Estimator and Outlier Detection for Interval-valued Data

Interval-valued data are one of the most common symbolic data types, which enables the preservation of the underlying variability of the data. The interval mean and covariance matrix can be estimated using the barycenter approach based on the Mallows distance. However, as for conventional data, classical estimates can be significantly affected by anomalous data points, frequently present in real-life datasets. To address this problem, we develop a robust alternative which estimates location and scale by extending the Minimum Covariance Determinant estimator to interval-valued data. The algorithm yields a robust Interval-Mahalanobis distance, which can be used to detect anomalous observations based on adaptive cutoff values. Through extensive simulation studies across various contamination levels, we demonstrate that the interval-valued robust estimator consistently outperforms classical methods in covariance matrix estimation and achieves superior outlier detection accuracy. Finally, the applicability and effectiveness of the proposed method are illustrated through real-world datasets.
Read the full abstract

All authors Catarina P. Loureiro, M. Rosário Oliveira, Paula Brito, Lina Oliveira

Statistical Methodology

CARhy: Comprehensive Analyses of Circadian Rhythms in Transcriptomic Experiments with Multiple Conditions

Circadian rhythms are endogenous oscillations that regulate various physiological processes and their disruption has been linked to many diseases, making it important to determine how gene-expression rhythms are altered across genotypes, treatments, or environmental exposures. Existing approaches for circadian transcriptomic analysis are often limited to pairwise comparisons or to a single aspect of rhythmic behavior, making them inadequate for comprehensive inference in multi-condition experimental designs. We propose CARhy (Comprehensive Analysis of Rhythmicity), a unified statistical framework for transcriptomic data collected under more than two conditions. Based on first-harmonic Fourier regression, CARhy provides formal tests for the presence of rhythmicity and for differences across conditions in rhythmicity, amplitude, phase, and baseline level. By allowing condition-specific variances and accommodating unbalanced designs, the framework remains reliable under heteroscedastic noise and realistic sampling constraints. Simulations show that CARhy controls type I error and false discovery rates well while achieving higher power than existing approaches in challenging settings. In mouse liver transcriptomic data, CARhy offers an interpretable and practical tool for characterizing how circadian rhythms differ across multiple experimental conditions. CARhy is implemented as an R package and is publicly available at: https://github.com/DrHuang123/Comprehensive-Analyses-of-Circadian-Rhythms-CARhy.git.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.IT

Source-Side Sufficiency for the Information Bottleneck: Exact Reduction and Finite-Block Equivalence

The input side of the Information Bottleneck may contain task-irrelevant variation that still costs rate. We identify this cost exactly. Let T be a source and C a relevance variable. Suppose the deterministic statistic Z = phi(T) satisfies C-Z-T. For any encoder p(X|T), its conditional average over the fibres of phi preserves I(X;C) and lowers the rate by I(X;T|Z). The reverse pullback preserves both coordinates. These maps establish equality of the relevance-rate curves and Lagrangian infima on standard Borel spaces for every tradeoff parameter. They also characterise all attained optima. Every full-source minimiser factors through Z, and every reduced minimiser pulls back to T. When C is finite and distortion is logarithmic loss, replacement of T^n by Z^n also preserves the optimal remote distortion at every blocklength and message budget. The operational rate-distortion functions are therefore equal.
Read the full abstract

Statistical Methodology

Flexible semiparametric modeling with application to Causal Inference

This paper proposes a flexible new framework for constructing Neyman-orthogonal scores in semiparametric models involving infinite-dimensional nuisance parameters. While locally estimation is vital for integrating machine learning into econometrics, deriving orthogonal scores for complex models remains a major challenge. We provide explicit construction strategies for broad classes of settings. The proposed framework ensures asymptotic normality of target parameter estimators in a way that does not depend on the method used to construct the nuisance parameter estimators, provided they are $o_p(n^{-\1/4})$-consistent. We apply the proposed methodology to causal inference with a binary instrumental variable, developing a novel, robust estimator for treatment effects. Numerical studies demonstrate that our approach significantly outperforms naive alternatives in finite samples. An empirical application to the Oregon Health Insurance Experiment illustrates the framework's utility in providing robust causal evidence.
Read the full abstract

All authors Kun Ren, Wen Su, Li Liu, Ian W. McKeague, Xingqiu Zhao

Statistics Theory

A Leakage Bound for Confidence Sets after Black-Box Selection

In many analyses the object reported at the end is not fixed in advance, but is chosen after a preliminary search over variables, subgroups, transformations, models or contrasts. Classical selective-inference methods are most effective when this search can be written as an explicit selection event. This note treats the less structured case in which the selection rule is a black box and inference is required for the target indexed by the selected object. We show that, for any fixed-target confidence procedure, selected-target noncoverage is bounded by the nominal fixed-target noncoverage plus the average total variation distance between the marginal law of the inferential data and its conditional law given the selected object. A mutual-information bound follows immediately. The result recovers sample splitting as the zero-leakage case and gives explicit guarantees for noisy screening through a Gaussian information bound. Thus the inferential cost of black-box selection is quantified by the information that the selected object carries about the inferential sample.
Read the full abstract

Machine Learning (Statistics)

Laplace Approximation for Bayesian Tensor Network Kernel Machines

Uncertainty estimation is essential for robust decision-making in the presence of ambiguous or out-of-distribution inputs. Gaussian Processes (GPs) are classical kernel-based models that offer principled uncertainty quantification and perform well on small- to medium-scale datasets. Alternatively, formulating the weight space learning problem under tensor network assumptions yields scalable tensor network kernel machines. However, these assumptions break Gaussianity, complicating standard probabilistic inference. This raises a fundamental question: how can tensor network kernel machines provide principled uncertainty estimates? We propose a novel Bayesian Tensor Network Kernel Machine (LA-TNKM) that employs a (linearized) Laplace approximation for Bayesian inference. A comprehensive set of numerical experiments shows that the proposed method consistently matches or surpasses Gaussian Processes and Bayesian Neural Networks (BNNs) across diverse UCI regression benchmarks, highlighting both its effectiveness and practical relevance.
Read the full abstract

Statistical Methodology

Nonlinear Probabilistic Forecast Reconciliation

Forecast reconciliation adjusts independently generated forecasts across multiple time series so that they satisfy some known constraints. While probabilistic forecast reconciliation is well established for linear constraints, some practical forecasting problems involve nonlinear relationships among time series. In this paper, we address probabilistic forecast reconciliation with nonlinear constraints for the first time. We extend both reconciliation via projection and conditioning to the case of nonlinear constraints. The former approach reconciles forecast samples by mapping them onto the nonlinear coherent manifold. The latter approach conditions the joint forecast distribution on the constraints; the reconciled distribution is approximated with an algorithm based on the Unscented Kalman Filter (UKF). We evaluate both methods on synthetic and real datasets. Empirically, both reconciliation approaches generally improve forecast accuracy. The UKF-based approach achieves the best overall performance, is consistently retained in the Model Confidence Set – either alone or alongside other reconciliation methods – and is substantially faster than the projection-based approach.
Read the full abstract

All authors Anubhab Biswas, Lorenzo Zambon, Lorenzo Nespoli, Giorgio Corani

Statistics Applications · Cross-listed from cs.LG

Electricity price forecasting across Norway's five bidding zones in the post-crisis era

Norway's electricity market is heavily dominated by hydropower, but the 2021-2022 energy crisis and stronger integration with Continental Europe have fundamentally altered price formation, reducing the reliability of forecasting models calibrated on historical data. Despite the critical need for updated models, a unified benchmark evaluating feature contributions across all structurally diverse Norwegian bidding zones remains lacking. Here we present a comprehensive evaluation of one-step-ahead forecasting of the Nord Pool market across all five Norwegian bidding zones. We constructed a multimodal hourly dataset spanning 2019-2025 and evaluated eight forecasting model families, including Light Gradient Boosting Machine (LightGBM), autoregressive models with exogenous variables, and advanced deep learning architectures, using a strictly causal test set. We implemented robust rolling-origin backtesting, leave-one-group-out feature ablation, and conditional regime analysis to dissect model performance and feature utility. Our results show that LightGBM achieves the best performance in every zone, with mean absolute error ranging from 1.60 to 5.58 euros per megawatt-hour, while a ridge-regularized autoregressive model with exogenous variables remains a highly competitive linear benchmark in northern zones. Feature ablation reveals that models relying solely on lagged prices and calendar variables achieve high accuracy and often match or closely approach the performance of the full multimodal model. However, conditional regime analysis demonstrates that external features like reservoir levels and gas prices remain crucial to stratify forecast errors, which consistently increase under stressed market regimes. This highlights the practical value of model interpretability and regime awareness for decision makers facing structural changes in market dynamics.
Read the full abstract

All authors My Thi Diem Phan, Trung Tuyen Truong, Hoai Phuong Ha, Dat Thanh Nguyen

Statistical Methodology

Principled Estimation and Prediction with Competing Risks: a Bayesian Nonparametric Approach

Competing risks occur in survival analysis when multiple causes of death are present. They play a prominent role in several domains extending beyond biostatistics to encompass epidemiology, actuarial sciences, and reliability theory. This paper adopts a multi-state modeling framework to competing risks. We introduce a class of flexible nonparametric priors, defined through hierarchical completely random measures, to model the transition probabilities, and identify the specific (conditionally) conjugate member of this general class. Furthermore, we determine the joint marginal distribution of the data and of a latent random partition, and characterize the posterior distribution of the model. Leveraging these distributional results, we evaluate the predictive probability that a future event is of a specific type (e.g. death from a particular cause), as a function of the time at which the event occurs. The resulting function, derived on sound principles, is termed the prediction curve, and represents a major innovation in the literature. In addition, we provide posterior estimates for the survival function, and for the cause-specific incidence and subdistribution functions. Suitable simulation algorithms for posterior inference are also devised. The model's performance, as well as the algorithms' effectiveness, is evaluated through simulation studies. Finally, we illustrate our approach on clinical datasets.
Read the full abstract

Machine Learning (Statistics)

Deep-testing: the case of dependence detection

Deep learning methods have proved highly effective for classification and image recognition problems. In this paper, we ask whether this success can be transferred to hypothesis testing: if a neural network can distinguish, for example, an image of a handwritten digit from another, can it also distinguish an "image of a sample" (such as a scatter plot) generated under a given statistical model from one generated outside that model? Motivated by this idea, we propose a novel procedure called deep-testing, which approaches the classical inferential problem of hypothesis testing through deep learning. More specifically, the test statistic is a classification map learned by a deep neural network from simulated data satisfying the null and alternative hypotheses, leveraging its strong discriminating power to construct a highly powerful test. As a proof of concept, we apply deep-testing to the problem of independence testing, arguably one of the most important problems in statistics. In a large-scale simulation study, deep-testing achieves the highest overall power against nineteen competing methods across a broad range of complex dependence structures, confirming the viability of the proposed approach.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Large-scale semi-supervised learning with online spectral graph sparsification

We introduce Sparse-HFS, a scalable algorithm that can compute solutions to SSL problems using only O(n polylog(n)) space and O(m polylog(n)) time.

Statistical Methodology

ARMA approximation of a Non-separable Spatio-Temporal Model with Fractional Smoothnesses in Space and Time

The Matérn covariance model is ubiquitous in spatial modelling, but there is no default choice for spatio-temporal modelling. In this paper, we consider the recently proposed “diffusion-based” extension of the spatial Matérn covariance model to a spatio-temporal non-separable covariance model that allows fractional smoothnesses in space and in time. The model is described in terms of a space-time fractional stochastic partial differential equation, but currently proposed computational approaches have strong restrictions on the possible smoothnesses in time. We propose a discretization method based on rational approximations in time to handle arbitrary smoothnesses, which leads to a vector autoregressive moving average process (VARMA). We prove that the covariance function of the approximation converges pointwise, determine explicit convergence rates as a function of spatial and temporal resolutions and the accuracy of the rational approximation, and conduct numerical verification to demonstrate small pointwise error for low orders of the VARMA process. Through a simulation study, we demonstrate that the parameters can be estimated back and that correctly specifying the temporal smoothness is especially important for forecasting. The approach is illustrated for three months of daily mean temperatures in mainland France.
Read the full abstract

Statistical Methodology

Recipes for Calibration Checks in Safety-Critical Applications

Safety-critical prediction systems, such as autonomous vehicles, weather forecasters, and medical monitors, commonly rely on probabilistic forecasters. These forecasters make predictions about possible future outcomes, and their quality and robustness needs to be validated and certified. Often, only accuracy – the mean of the predictions – is evaluated against true outcomes. However, for safety-critical scenarios and decision making under uncertainty, the full distributional properties of the forecasts should be checked: do the observed prediction errors actually follow the forecasted probability distributions? To this end, we introduce a framework for calibration checks: statistical tests that validate distributional properties of forecasts when measured over many samples. In order to support ease-of-use in real-world operations, these checks produce a single accept/reject decision for data collected from a forecaster. This contrasts typical calibration calculations which produce one or multiple continuous calibration scores and require expertise to implement in a validation workflow. We further support operationalization by introducing modifications to calibration testing that (a) reject only overconfident predictions, allowing for pessimistic or cautious predictions in safety-critical settings, and (b) tolerate small, operationally acceptable deviations even for large numbers of validation samples. We organize the calibration checking process into a modular pipeline comprising four steps: (i) the data model, (ii) the chosen metric, (iii) the hypothesis formulation, and (iv) the testing procedure. Each step consists of independently swappable components, thereby supporting a large variety of possible use-cases and trade-offs. We demonstrate the applicability of the framework on two complementary example problems, weather forecasting and robot pose estimation.
Read the full abstract

Statistical Methodology

Sample splitting for valid statistical inference in target trial emulations

\noindent Target trial emulation has improved comparative effectiveness research by making the causal question, assumptions, and analysis plan explicit. However, target trial protocols are usually developed iteratively. That is, after examining the data, investigators revise the protocol to reflect which target trials the observational data can realistically support. This iterative procedure is part of normal scientific practice, but it raises concerns about selective choices and invalid statistical inference. A simple procedure based on sample splitting addresses the concerns about valid inference. In the initial split, investigators can explore the data to define a target trial protocol. When these choices are made, the target trial protocol is implemented on the second split. Even though the investigators used the data to select the target trial protocol, the statistical inference has the usual coverage guarantees. The procedure intentionally mirrors how trialists move from pilot studies to a phase 3 trial. First, they use data from pilots and early-phase trials to decide on a final protocol. Then they implement this protocol and analyze a new set of data in a phase 3 trial.
Read the full abstract

Statistical Methodology

Longitudinal Outcomes Truncated by Death: Causal Estimands and Bayesian Estimators

In randomized controlled trials with longitudinal outcomes, death before the end of follow-up poses a fundamental challenge: after death, the outcome is no longer a real-valued measurement. This complicates the definition and interpretation of causal estimands, particularly when treatment may affect both survival and longitudinal outcomes. We review existing estimands for longitudinal outcomes truncated by death and clarify the assumptions required for their identification and estimation. We show that these estimands fall into two broad classes, distinguished by whether they require additional assumptions to compare longitudinal outcomes beyond death. Such assumptions may be inappropriate in chronic diseases, either because i) death and longitudinal outcomes are driven by the same underlying biological process or ii) the relative desirability of survival with poor function versus death may depend on individual preferences. We compare the behavior of the estimands in a simulation study using Bayesian estimators and illustrate their use with data from a randomized controlled trial in amyotrophic lateral sclerosis. We argue that, in the presence of death truncation, pairing the survivor average causal effect with the restricted mean survival time estimand provides an interpretable characterization of treatment effects on longitudinal and survival outcomes.
Read the full abstract

Statistics Theory · Cross-listed from math.PR

Limiting spectral distributions of large consistent rank correlation matrices

We study random matrices whose entries are obtained by applying consistent rank correlations, such as Hoeffding's $D$, pairwise to a high-dimensional random vector with mutually independent components. Prior work has shown that, in the proportional high-dimensional regime, the empirical spectral distributions of large Kendall's tau and Spearman's rho matrices converge weakly almost surely to the Marchenko–Pastur law. By contrast, we prove that for consistent rank correlations such as Hoeffding's $D$, the limiting spectral distribution is given by the semicircle law. Our result thus generalizes a recent work of Dong, Han, and Yao (2025), who considered the special case of Chatterjee's rank correlation and established the first semicircle law for a large correlation matrix in the proportional regime.
Read the full abstract

Machine Learning (Statistics)

Probabilistic data quality assessment for structural monitoring data via outlier-resistant conditional diffusion model

Data quality assessment is an essential step that ensures the reliability of the subsequent structural health monitoring (SHM) tasks. This study proposes a prediction deviation-based SHM data quality assessment method using a univariate implicit auto-regressive model, enabling outlier diagnosis and data cleaning. The proposed conditional diffusion model (CDM) augments the standard diffusion model with a conditional embedding module to incorporate temporal context, quartile normalization to mitigate distribution skew, and a Huber loss to enhance robustness against outliers. Within this univariate implicit autoregressive framework, each data point is assigned an outlier probability, quantifying its degree of "outlier-ness", and a global quality evaluation score is computed to characterize the overall dataset quality. Extensive case studies utilizing operational data from real-world structures demonstrate that the proposed framework significantly improves the accuracy of data quality assessment, outperforming other strong baselines representative of clustering, isolation-based, and deep reconstruction methods. The effectiveness and robustness of the proposed framework are further demonstrated by the findings of ablation experiments and hyperparameter analysis.
Read the full abstract

Statistics Applications

A spatio-temporal statistical framework for heatwave attribution under climate change

We develop a unified statistical framework for attributing heatwaves as spatio-temporal phenomena under climate change. We quantify the impact of anthropogenic forcing on the probability and persistence of heatwaves not captured by standard marginal extreme-value approaches. Our methodology constructs a generative model for daily temperature fields that separates marginal nonstationarity from spatio-temporal dependence. We combine three components: a Bayesian spatial quantile regression model for the bulk of the data; a nonstationary spatial generalized extreme value model for tail behavior; and a copula-based model capturing both asymptotic dependence and independence in the extremes. The framework is applied to the CMIP6 MRI-ESM2 climate model, contrasting factual and counterfactual scenarios for probabilistic attribution. Our results show that the approach captures key heatwave characteristics inaccessible to traditional methods, enabling direct estimation of event-level attribution metrics. Overall, it provides a flexible basis for analyzing and attributing complex climate extremes as space-time objects.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Addressing Performance Saturation for LLM RL via Precise Entropy Curve Control

Reinforcement learning (RL) has enabled complex reasoning abilities in large language models (LLMs). However, most RL algorithms suffer from performance saturation, preventing continued gains as RL training scales. This problem can be characterized by the collapse of entropy, a key diagnostic for exploration in RL. Existing attempts focus on preventing entropy collapse through regularization or clipping. However, their resulting entropy curves often exhibit instability in the long term, which hinders performance gains. In this paper, we introduce Entrocraft, a simple rejection-sampling approach that realizes user-customized entropy schedule by biasing the advantage distributions. Entrocraft requires no objective regularization and is advantage-estimator-agnostic. Theoretically, we relate per-step entropy change to the advantage distribution under minimal assumptions. This explains the behavior of existing RL and entropy-preserving methods. Entrocraft also enables a systematic study of entropy schedules, which reveals that linear annealing, which starts high and decays to a slightly lower target, performs best. Empirically, Entrocraft addresses performance saturation, significantly improving generalization, output diversity, and long-term training. It enables a 4B model to outperform an 8B baseline, sustains improvement for up to 4x longer before plateauing, and raises pass@K by 50% over the baseline.
Read the full abstract

All authors Bolian Li, Yifan Wang, Yi Ding, Anamika Lochab, Ananth Grama, Ruqi Zhang

Statistical Methodology

TWICEBEE: A Two-stage Intra-patient Curve-free Bayesian Decision-Theoretic Dose Escalation Design

We propose a novel Phase I intra-patient dose-escalation design tailored for multi-cycle immunotherapy settings, in which toxicity at a fixed dose level is clinically expected to decrease over successive treatment cycles. This design was motivated by a phase I trial of CAR T cell therapy, an emerging cellular immunotherapy with established applications in cancer and growing investigation in autoimmune disease. The design is intended for settings in which nonincreasing cycle-specific toxicity assumption is clinically justified. Specifically, we build on the extrapolation property of the modified curve-free Bayesian decision-theoretic (c-CFBD) design for two-agent trials (Xu, et al. 2025), treating treatment cycle as a second dimension. By redefining the partial order, the c-CFBD framework can accommodate the reduction in toxicity across cycles. The proposed design adopts a two-stage structure: an initial accelerated titration stage to rapidly explore dose levels, followed by a c-CFBD stage to improve safety and estimate the cycle-specific maximum tolerated dose sequence. Simulation studies across a range of scenarios demonstrate favorable operating characteristics.
Read the full abstract

All authors Dehua Bi, Katherine Ryan, Sabine Heitzeneder, Zina Good, John S. Tamaresis, Robert Lowsky, Michelle Monje, Crystal Mackall, Ying Lu

Statistics Applications

The Difference Between "Replicable" and "Not Replicable" Is Not Itself Scientifically Replicable

Replication studies estimate the replicability rate of scientific results by aggregating binary verdicts of experiments as “replicated” and “not replicated”. These estimates assume rarely attainable exact replications. Experiments often do not share a common data-generating process, leading to non-exact replications. We formalize two statistical interpretations of non-exactness. In a shared latent rate model (benchmark), experiments are exchangeable conditional on a common random replicability rate. In a conditionally independent rates model (operational), each experiment has its own random replicability rate, drawn independently from a population. Under the benchmark model, even small variability in assumed replicability rates induces a variance floor of the estimated mean replicability rate that is irreducible by more replications. Under the operational model, the degree of non-exactness is not identifiable from binary verdict data, which contain no information about between-experiment heterogeneity. Researchers cannot tell which precision regime they are in, and high- and low-replicability sequences cannot be distinguished in principle. Binary verdicts systematically understate uncertainty in reporting scientific results, making high- and low-replicability sequences appear discriminable, when they are not. Common sources of heterogeneity amplify these problems as shown in a reanalysis of Many Labs~4 data. The practice of aggregating replicability rates across heterogeneous literatures produces averages that conflate incommensurable experimental regimes and lack a stable scientific interpretation. Hence, replicability rates as currently used do not provide a reliable demarcation criterion for scientific results. The replication crisis, if there is one, cannot be established by methods used to declare it.
Read the full abstract

Statistical Methodology · Cross-listed from cs.CL

A New Semisupervised Technique for Polarity Analysis using Masked Language Models

I developed a new version of Latent Semantic Scaling (LSS) employing word2vec as a masked language model. Unlike original spatial models, it assigns polarity scores to words and documents as predicted probabilities of seed words to occur in given contexts. These probabilistic polarity scores are more accurate, interpretable and consistent than those spatial polarity models can produce in text analysis. I demonstrate these advantages by applying both probabilistic and spatial models to China Daily's coverage of China and other countries during the coronavirus disease (COVID) pandemic in terms of achievement in health issues. The result suggests that more advanced masked language models would further improve the semisupervised machine learning technique.
Read the full abstract

Statistics Applications

Pricing Global Macroeconomic Risk in Equity Markets: Evidence from Selected G20 Economies

This study investigates whether international equity markets systematically price global macroeconomic risks. The empirical analysis is conducted using monthly excess returns for ten G20 countries over the period 2000-2024. A Dynamic Factor Model (DFM) is employed to extract latent global factors from a set of macroeconomic variables capturing global inflation, real activity, monetary policy, term structure, exchange rates, volatility, and oil prices. The model selection criteria of the dynamic factor framework, which support a 3 factor specification that is parsimonious. The Fama MacBeth regressions demonstrate the low explanatory power of the 3-factor model. In contrast, a 4 factor specification results in economically large and statistically significant factor loadings, an obvious rise in explanatory power, and a significant improvement in model performance. The results indicate that a four-factor specification provides the best balance between explanatory power and model stability, significantly improving the ability to explain cross-sectional variation in excess returns , with all factors statistically significant. The Capital Asset Pricing Model, while offering a parsimonious and stable benchmark with consistently significant market betas, exhibits limited explanatory power due to its single factor structure. Overall, the findings suggest that macro driven latent factors extracted through the DFM provide a more comprehensive and empirically robust framework for international asset pricing than the CAPM, highlighting the importance of incorporating multiple sources of systematic risk in explaining cross-country equity returns.
Read the full abstract

Tuesday 28 April

Statistics Theory

Adaptive Confidence Intervals in Efron's Gaussian Two-Groups Model

Robust uncertainty quantification is increasingly important in modern data analysis and is often formalized under Huber's model, which allows an $\varepsilon$-fraction of arbitrary corruptions. In many experimental sciences, however, the measurement protocol is well controlled, and contamination is more plausibly introduced upstream. Motivated by this noise-oblivious nature of adversaries, we study confidence intervals for the null location parameter $\theta$ in Efron's Gaussian two-groups model, where an unknown fraction $\varepsilon$ of observations have arbitrarily shifted means, but all samples share the same law of additive Gaussian measurement noise with variance $\sigma^2$. We characterize the minimax-optimal length among confidence intervals with a prescribed coverage level uniformly over the unknown contamination proportion and all noise-oblivious adversaries. Although prior work has shown that the minimax point estimation rate of theta does not deteriorate when $\varepsilon$ becomes unknown, our results reveal that, with a given $\sigma^2$, the minimax-optimal length of confidence intervals that are adaptive to unknown $\varepsilon$ is of order $\sigma (n^{-1/4}+\varepsilon^{1/2}/\max\{1, \log(en \varepsilon^2)\}^{1/2})$, which is polynomially worse than the optimal length when $\varepsilon$ is known. When the variance $\sigma^2$ is also unknown, we show a further degradation: no adaptive confidence interval can be shorter than $\Omega(\sigma n^{-1/8})$. Algorithmically, we introduce a Fourier-based certification procedure built on Carathéodory's positive-semidefiniteness constraints. By scanning candidate points and accepting those whose residual characteristic function is certifiably consistent with a Gaussian location mixture, our algorithm attains the minimax lower bound in the known-variance setting and is computable in polynomial time.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.IR

Value-Aware Product Recommendation by Customer Segmentation using a suitable High-Dimensional Similarity Measure

This paper presents a novel value-aware approach to product recommendation that simultaneously addresses the high dimensionality and sparsity of user-item data while explicitly incorporating the contribution of each product and user to overall sales revenue. The proposed framework encodes revenue contributions in the user-item matrix and computes customer similarity directly on this basis using suitable distance measures. This enables the segmentation of users according to the revenue-based similarity of their purchase baskets and supports recommendations aligned with profitability objectives. We compare conventional similarity metrics with a novel alternative tailored to high-dimensional contexts and propose three recommendation strategies based on revenue share, product popularity, and expected profit generation. The effectiveness of the proposed method is validated through simulation experiments and a real-world application using the UCI Online Retail dataset.
Read the full abstract

All authors María Florencia Acosta, Rodrigo García Arancibia, Pamela Llop, Mariel Lovatto, Lucas Mansilla

Statistics Theory

Asymptotics of ultra-high-dimensional generalized spiked sample covariance matrix

This paper investigates the asymptotics of eigenstructure of sample covariance matrix under the spiked covariance matrix model in ultra-high-dimensional settings, where the dimensionality can grow much faster than the sample size with $ p \asymp n^{\alpha} $, $ \alpha > 1 $. We establish the first-order convergence limits of eigenvalue locations and eigenvector projections of properly scaled sample covariance matrix. Our results are extensions of \cite{bloemendal16,ding21}.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from eess.SY

Co-Learning Port-Hamiltonian Systems and Optimal Energy-Shaping Control

We develop a physics-informed learning framework for energy-shaping control of port-Hamiltonian (pH) systems from trajectory data. The proposed approach co-learns a pH system model and an optimal energy-balancing passivity-based controller (EB-PBC) through alternating optimization with policy-aware data collection. At each iteration, the system model is refined using trajectory data collected under the current control policy, and the controller is re-optimized on the updated model. Both components are parameterized by neural networks that embed the pH dynamics and EB-PBC structure, ensuring interpretability in terms of energy interactions. The learned controller renders the closed-loop system inherently passive and provably stable, and exploits passive plant dynamics without canceling the natural potential. A dissipation regularization enforces strict energy decay during training, thereby enhancing robustness to sim-to-real gaps. The proposed framework is validated on state-regulation and swing-up tasks for planar and torsional pendulum systems.
Read the full abstract

Machine Learning (Statistics) · Cross-listed from cs.LG

Budget-Constrained Causal Bandits: Bridging Uplift Modeling and Sequential Decision-Making

Treatment allocation under budget constraints is a central challenge in digital advertising. The standard approach trains an offline uplift model on historical data, then solves a constrained optimization to allocate budget. This fails in cold-start settings where little historical data exists. We propose Budget-Constrained Causal Bandits (BCCB), an online framework that learns which users respond to ads while simultaneously spending the budget. BCCB unifies three components: learning individual-level treatment effects, exploring users whose response is uncertain, and pacing the budget over time. We derive the per-arrival decision rule as the KKT condition of a Lagrangian relaxation of the budgeted causal-allocation objective, providing a principled foundation for the algorithm. We evaluate on the Criteo Uplift dataset using 20 random seeds with paired statistical tests. Our central finding is a data-efficiency crossover at n = 7,500 historical observations (paired one-sided t-test, p = 0.043): below this threshold, offline pipelines either fail or produce unreliable allocations, while BCCB operates from the first user. BCCB exhibits 2-4x lower run-to-run variance than offline methods and outperforms all four online baselines (Thompson Sampling, budgeted Thompson Sampling, HTE Greedy, and Uplifting Bandits) at every budget level tested (p < 0.001). These results give practitioners a concrete decision rule for choosing between offline and online paradigms.
Read the full abstract

2604.26169 · 28 Apr · 17 pages, 5 tables. v2: Expanded to 20 random seeds with paired statistical tests; formal crossover test at n=7,500 (p=0.043); added Uplifting Bandits (Hsieh et al., NeurIPS 2022) as fifth baseline; Lagrangian derivation of decision rule with Proposition 1; hyperparameter sensitivity analysis; corrected mathematical claims