Papers

Numerous studies have shown the harmful effects of airborne pollutants on human health. Vulnerable groups and communities often bear a disproportionately larger health burden due to exposure to airborne pollutants. Thus, there is a need to design policies that effectively reduce the public health burdens while ensuring cost-effective policy interventions. Designing policies that optimally benefit the population while ensuring equity between groups under cost constraints is a challenging statistical and causal inference problem. In the context of environmental policy this is further complicated by the fact that interventions target emission sources but health impacts occur in potentially distant communities due to atmospheric pollutant transport – a setting known as bipartite network interference (BNI). To address these issues, we propose a fair policy learning approach under BNI. Our approach allows to learn cost-effective policies under fairness constraints even accounting for complex BNI data structures. We derive asymptotic properties and demonstrate finite sample performance via Monte Carlo simulations. Finally, we apply the proposed method to a real-world dataset linking power plant scrubber installations to Medicare health records for more than 2 million individuals in the U.S. Our method determine fair scrubber allocations to reduce mortality under fairness and cost constraints.

Current navigation systems conflate time-to-drive with the true time-to-arrive by ignoring parking search duration and the final walking leg. Such underestimation can significantly affect user experience, mode choice, congestion, and emissions. To address this issue, this paper introduces the probability-aware parking selection problem, which aims to direct drivers to the best parking location rather than straight to their destination. An adaptable dynamic programming framework is proposed that leverages probabilistic, lot-level availability to minimize the expected time-to-arrive. Closed-form analysis determines when it is optimal to target a specific parking lot or explore alternatives, as well as the expected time cost. Sensitivity analysis and three illustrative cases are examined, demonstrating the model's ability to account for the dynamic nature of parking availability. Given the high cost of permanent sensing infrastructure, we assess the error rates of using stochastic observations to estimate availability. Experiments with real-world data from the US city of Seattle indicate this approach's viability, with mean absolute error decreasing from 7% to below 2% as observation frequency increases. In data-based simulations, probability-aware strategies demonstrate time savings up to 66% relative to probability-unaware baselines, yet still take up to 123% longer than time-to-drive estimates.

In this study, we introduce a sophisticated generative conditional strategy designed to impute missing values within datasets, an area of considerable importance in statistical analysis. Specifically, we initially elucidate the theoretical underpinnings of the Generative Conditional Missing Imputation Networks (GCMI), demonstrating its robust properties in the context of the Missing Completely at Random (MCAR) and the Missing at Random (MAR) mechanisms. Subsequently, we enhance the robustness and accuracy of GCMI by integrating a multiple imputation framework using a chained equations approach. This innovation serves to bolster model stability and improve imputation performance significantly. Finally, through a series of meticulous simulations and empirical assessments utilizing benchmark datasets, we establish the superior efficacy of our proposed methods when juxtaposed with other leading imputation techniques currently available. This comprehensive evaluation not only underscores the practicality of GCMI but also affirms its potential as a leading-edge tool in the field of statistical data analysis.

Kladia Liquidity Deflator (KLD) is an XRPL-based, debt-indexed token whose supply dynamics respond directly to a debt index derived from macroeconomic data sources. The model links indebtedness to deterministic adjustments in issuance, burns, and escrow release caps, creating a rule-based deflationary mechanism that strengthens as debt rises. With a fixed maximum supply of 10 billion KLD, the mechanism is implemented through XRPL oracles and governance. Escrow locking depends on the TokenEscrow amendment; until it is active network-wide, allocations will be secured in a multi-signature vault with published rules and public monitoring. KLD provides a transparent and mathematically grounded framework for a macro-responsive digital asset.

Conformal prediction provides distribution-free coverage guarantees, but these degrade under distribution shift - and practitioners lack tools to anticipate which deployed models will fail before observing test data. We propose SHapley Additive exPlanations (SHAP) concentration - the fraction of feature importance concentrated in the top feature - as a pre-deployment diagnostic for conformal prediction vulnerability in gradient-boosted classifiers. Using COVID-19 as a naturalistic case study, eight supply chain tasks experience identical temporal shift yet coverage drops ranging from negligible to catastrophic. Feature-importance concentration is strongly associated with failure severity across 16 multiclass tasks in 9 domains, while standard distributional shift detectors detect shift uniformly across tasks but cannot distinguish catastrophic from robust outcomes. External validation across 9 non-supply-chain datasets shows partial transfer. We prove a formal theorem showing that Adaptive Prediction Sets conformity-score bounds worsen monotonically with concentration under explicit assumptions, verified empirically. The diagnostic identifies concentrated-dependence failures characteristic of gradient-boosted models but does not detect global-sensitivity failures observed in neural networks. A decision framework operationalizes the diagnostic as an exploratory pre-deployment rule with an uncertainty band around a concentration threshold.

Topological Data Analysis Ball Mapper (TDABM) offers a model-free visualization of multivariate data which does not necessitate the information loss associated with dimensionality reduction. TDABM Dlotko (2019) produces a cover of a multidimensional point cloud using equal size balls, the radius of the ball is the only parameter. A TDABM visualization retains the full structure of the data. The graphs produced by TDABM can convey coloration according to further variables, model residuals, or variables within the multivariate data. An expanding literature makes use of the power of TDABM across Finance, Economics, Geography, Medicine and Chemistry amongst others. We provide an introduction to TDABM and the ballmapper package for Stata.

We mathematically axiomatise the stochastics of counterfactuals, by introducing two related frameworks, called counterfactual probability spaces and counterfactual causal spaces, which we collectively term counterfactual spaces. They are, respectively, probability and causal spaces whose underlying measurable spaces are products of world-specific measurable spaces. In contrast to more familiar accounts of counterfactuals founded on causal models, we do not view interventions as a necessary component of a theory of counterfactuals. As an alternative to Pearl's celebrated ladder of causation, we view counterfactuals and interventions are orthogonal concepts, respectively mathematised in counterfactual probability spaces and causal spaces. The two concepts are then combined to form counterfactual causal spaces. At the heart of our theory is the notion of shared information between the worlds, encoded completely within the probability measure and causal kernels, and whose extremes are characterised by independence and synchronisation of worlds. Compared to existing frameworks, counterfactual spaces enable the mathematical treatment of a strictly broader spectrum of counterfactuals.

Precision medicine has led to a paradigm shift allowing the development of targeted drugs that are agnostic to the tumor location. In this context, basket trials aim to identify which tumor types - or baskets - would benefit from the targeted therapy among patients with the same molecular marker or mutation. We propose the implementation of continuous monitoring for basket trials to increase the likelihood of early identification of non-promising baskets. Although the current Bayesian trial designs available in the literature can incorporate more than one interim analysis, most of them have high computational cost, and none of them handle delayed outcomes that are expected for targeted treatments such as immunotherapies. We leverage the Bayesian empirical approach proposed by Fujiwara et al., which has low computational cost. We also extend ideas of Cai et al to address the practical challenge of performing interim analysis with delayed outcomes using multiple imputation. Operating characteristics of four different strategies to handle delayed outcomes in basket trials are compared in an extensive simulation study with the benchmark strategy where trial accrual is put on hold until complete data is observed to make a decision. The optimal handling of missing data at interim analyses is trial-dependent. With slow accrual, missingness is minimal even with continuous monitoring, favoring simpler approaches over computationally intensive methods. Although individual sample-size savings are small, multiple imputation becomes more appealing when sample size savings scale with the number of baskets and agents tested.

We study multi-user contextual bandits where users are related by a graph and their reward functions exhibit both non-linear behavior and graph homophily. We introduce a principled joint penalty for the collection of user reward functions $\{f_u\}$, combining a graph smoothness term based on RKHS distances with an individual roughness penalty. Our central contribution is proving that this penalty is equivalent to the squared norm within a single, unified multi-user RKHS. We explicitly derive its reproducing kernel, which elegantly fuses the graph Laplacian with the base arm kernel. This unification allows us to reframe the problem as learning a single ”lifted” function, enabling the design of principled algorithms, LK-GP-UCB and LK-GP-TS, that leverage Gaussian Process posteriors over this new kernel for exploration. We provide high-probability regret bounds that scale with an effective dimension of the multi-user kernel, replacing dependencies on user count or ambient dimension. Empirically, our methods outperform strong linear and non-graph-aware baselines in non-linear settings and remain competitive even when the true rewards are linear. Our work delivers a unified, theoretically grounded, and practical framework that bridges Laplacian regularization with kernelized bandits for structured exploration.

The method of imsets, introduced by Studený, provides a geometric and combinatorial description of conditional independence statements. Elementary conditional independence statements over a finite set of discrete random variables correspond to column vectors of a matrix generating a polyhedral cone, and the associated toric ideals encode algebraic relations among these statements. In this paper, we study discrete probability distributions on sets of three and four random variables, including both binary variables and combinations of binary and ternary variables. We investigate the structure of conditional independence ideals arising from elementary and non-elementary CI relations and analyze the algebraic properties of imsetal models induced by faces of the elementary imset cone. Our results highlight connections between combinatorial CI relations, their associated ideals, and the geometry of imset cones.

Existing methods of vector autoregressive model for multivariate time series analysis make use of low-rank matrix approximation or Tucker decomposition to reduce the dimension of the over-parameterization issue. In this paper, we propose a sparse Tucker decomposition method with graph regularization for high-dimensional vector autoregressive time series. By stacking the time-series transition matrices into a third-order tensor, the sparse Tucker decomposition is employed to characterize important interactions within the transition third-order tensor and reduce the number of parameters. Moreover, the graph regularization is employed to measure the local consistency of the response, predictor and temporal factor matrices in the vector autoregressive model.The two proposed regularization techniques can be shown to more accurate parameters estimation. A non-asymptotic error bound of the estimator of the proposed method is established, which is lower than those of the existing matrix or tensor based methods. A proximal alternating linearized minimization algorithm is designed to solve the resulting model and its global convergence is established under very mild conditions. Extensive numerical experiments on synthetic data and real-world datasets are carried out to verify the superior performance of the proposed method over existing state-of-the-art methods.

This paper investigates the asymptotic distribution of a wavelet-based NKK periodogram constructed from least absolute deviations (LAD) harmonic regression at a fixed resolution level. Using a wavelet representation of the underlying time series, we analyze the probabilistic structure of the resulting periodogram under long-range dependence. It is shown that, under suitable regularity conditions, the NKK periodogram converges in distribution to a nonstandard limit characterized as a quadratic form in a Gaussian random vector, whose covariance structure depends on the memory properties of the process and on the chosen wavelet filters. This result establishes a rigorous theoretical foundation for the use of robust wavelet-based periodograms in the spectral analysis of long-memory time series with heavy-tailed inovations.

The Stable Unit Treatment Value Assumption (SUTVA) includes the condition that there are no multiple versions of treatment in causal inference. Though we could not control the implementation of treatment in observational studies, multiple versions may exist in the treatment. It has been pointed out that ignoring such multiple versions of treatment can lead to biased estimates of causal effects, but a causal inference framework that explicitly deals with the unbiased identification and estimation has not been fully developed yet. Thus, it is difficult to obtain a deeper understanding for mechanisms of the complex treatments. In this paper, we introduce the Mixture-of-Experts framework into causal inference to estimate causal contrasts between underlying versions of a treatment, even when the versions are not observed. Numerical experiments demonstrate the effectiveness of the proposed method.

We derive an estimator of the spectral density of a functional time series that is the output of a multilayer perceptron neural network. The estimator is motivated by difficulties with the computation of existing spectral density estimators for time series of functions defined on very large grids that arise, for example, in climate compute models and medical scans. Existing estimators use autocovariance kernels represented as large $G \times G$ matrices, where $G$ is the number of grid points on which the functions are evaluated. In many recent applications, functions are defined on 2D and 3D domains, and $G$ can be of the order $G \sim 10^5$, making the evaluation of the autocovariance kernels computationally intensive or even impossible. We use the theory of spectral functional principal components to derive our deep learning estimator and prove that it is a universal approximator to the spectral density under general assumptions. Our estimator can be trained without computing the autocovariance kernels and it can be parallelized to provide the estimates much faster than existing approaches. We validate its performance by simulations and an application to fMRI images.

We introduce the concept of geometric extremal graphical models, which are defined through the gauge function of the limit set obtained from suitably scaled random vectors in light-tailed margins. For block graphs, we prove results relating to the propagation of various extremal dependence coefficients along the graph. A particular focus is placed on coefficients that link to the framework of conditional extreme value theory, which are especially interesting when variables do not all attain their most extreme values simultaneously. We also consider results related to the case when variables do exhibit joint extreme behaviour. Through the recent translation of the geometric approach for multivariate extremes to a statistical modelling framework, geometric extremal graphical models, and results relating to them, pave the way for an approach to modelling of high dimensional extremes with complex extremal dependence structures.

Detecting unobserved confounders is crucial for reliable causal inference in observational studies. Existing methods require either linearity assumptions or multiple heterogeneous environments, limiting applicability to nonlinear single-environment settings. To bridge this gap, we propose Kernel Regression Confounder Detection (KRCD), a novel method for detecting unobserved confounding in nonlinear observational data under single-environment conditions. KRCD leverages reproducing kernel Hilbert spaces to model complex dependencies. By comparing standard and higherorder kernel regressions, we derive a test statistic whose significant deviation from zero indicates unobserved confounding. Theoretically, we prove two key results: First, in infinite samples, regression coefficients coincide if and only if no unobserved confounders exist. Second, finite-sample differences converge to zero-mean Gaussian distributions with tractable variance. Extensive experiments on synthetic benchmarks and the Twins dataset demonstrate that KRCD not only outperforms existing baselines but also achieves superior computational efficiency.

This paper introduces a unified profile quasi-likelihood framework for generalised linear models built on a topologically complete extension to Kendall's $\tau$ with finite sample power and Type-I error rates. By establishing equivalence between continuous $L^2$ profile estimators and discrete Generalized Gibbs exponential families, we prove local asymptotic minimax efficiency over the copula tangent space. The framework yields monotone-affine invariant inference, sub-Gaussian concentration bounds, and exact finite-sample $t$-pivot tail majorization under arbitrary tie configurations. We illustrate the methodology through three canonical applications: bivariate rank correlation, paired testing, and intraclass correlation.

End member analysis (EMA) unmixes grain size distribution (GSD) data into a mixture of end members (EMs), thus helping understand sediment provenance and depositional regimes and processes. In highly mixed data sets, however, many EMA algorithms find EMs which are still a mixture of true EMs. To overcome this, we propose maximum volume constrained EMA (MVC-EMA), which finds EMs as different as possible. We provide a uniqueness theorem and a quadratic programming algorithm for MVC-EMA. Experimental results show that MVC-EMA can effectively find true EMs in highly mixed data sets.

This study investigates the predictive capacity of environmental, temporal, and spatial factors on traffic accident severity in the United States. Using a dataset of 500,000 U.S. traffic accidents spanning 2016-2023, we trained an XGBoost classifier optimized through randomized search cross-validation and adjusted for class imbalance via class weighting. The final model achieves an overall accuracy of 78%, with strong performance on the majority class (Severity 2), attaining 87% precision and recall. Feature importance analysis reveals that time of day, geographic location, and weather-related variables, including visibility, temperature, and wind speed, rank among the strongest predictors of accident severity. However, contrary to initial hypotheses, precipitation and visibility demonstrate limited predictive power, potentially reflecting behavioral adaptation by drivers under overtly hazardous conditions. The dataset's predominance of mid-level severity accidents constrains the model's capacity to learn meaningful patterns for extreme cases, highlighting the need for alternative sampling strategies, enhanced feature engineering, and integration of external datasets. These findings contribute to evidence-based traffic management and suggest future directions for severity prediction research.

Generally, Lasso, Adaptive Lasso, and SCAD are standard approaches in variable selection in the presence of a large number of predictors. In recent years, during intensity function estimation for spatial point processes with a diverging number of predictors, many researchers have considered these penalized methods. But we have discussed a multi-resolution perspective for the variable selection method for spatial point process data. Its advantage is twofold: it not only efficiently selects the predictors but also provides the idea of which points are liable for selecting a predictor at a specific resolution. Actually, our research is motivated by the crime and accident occurrences in St. Louis and its neighborhoods. It is more relevant to select predictors at the local level, and thus we get the idea of which set of predictors is relevant for the occurrences of crime or accident in which parts of St. Louis. We describe the simulation results to justify the accuracy of local-level variable selection during intensity function estimation.

Resilient intermodal freight networks are vital for sustaining supply chains amid increasing threats from natural hazards and cyberattacks. While transportation resilience has been widely studied, understanding how random and targeted disruptions affect both structural connectivity and functional performance remains a key challenge. To address this, our study evaluates the robustness of the U.S. intermodal freight network, comprising rail and water modes, using a simulation-based framework that integrates graph-theoretic metrics with flow-weighted centrality measures. We examine disruption scenarios including random failures as well as targeted node and edge removals based on static and dynamically updated degree and betweenness centrality. To reflect more realistic conditions, we also consider flow-weighted degree centralities and partial node degradation. Two resilience indicators are used: the size of the giant connected component (GCC) to measure structural connectivity, and flow-weighted network efficiency (NE) to assess freight mobility under disruption. Results show that progressively degrading nodes ranked by Weighted Degree Centrality to 60% of their original functionality causes a sharper decline in normalized NE, for up to approximately 45 affected nodes, than complete failure (100% loss of functionality) applied to nodes targeted by weighted betweenness centrality or selected at random. This highlights how partial degradation of high-tonnage hubs can produce disproportionately large functional losses. The findings emphasize the need for resilience strategies that go beyond network topology to incorporate freight flow dynamics.

Blind source separation, particularly through independent component analysis (ICA), is widely utilized across various signal processing domains for disentangling underlying components from observed mixed signals, owing to its fully data-driven nature that minimizes reliance on prior assumptions. However, conventional ICA methods rely on an assumption of linear mixing, limiting their ability to capture complex nonlinear relationships and to maintain robustness in noisy environments. In this work, we present deep deterministic nonlinear independent component analysis (DDICA), a novel deep neural network-based framework designed to address these limitations. DDICA leverages a matrix-based entropy function to directly optimize the independence criterion via stochastic gradient descent, bypassing the need for variational approximations or adversarial schemes. This results in a streamlined training process and improved resilience to noise. We validated the effectiveness and generalizability of DDICA across a range of applications, including simulated signal mixtures, hyperspectral image unmixing, modeling of primary visual receptive fields, and resting-state functional magnetic resonance imaging (fMRI) data analysis. Experimental results demonstrate that DDICA effectively separates independent components with high accuracy across a range of applications. These findings suggest that DDICA offers a robust and versatile solution for blind source separation in diverse signal processing tasks.

Topological methods have the potential of exploring data clouds without making assumptions on their the structure. Here we propose a hierarchical topological clustering algorithm that can be implemented with any distance choice. The persistence of outliers and clusters of arbitrary shape is inferred from the resulting hierarchy. We demonstrate the potential of the algorithm on selected datasets in which outliers play relevant roles, consisting of images, medical and economic data. These methods can provide meaningful clusters in situations in which other techniques fail to do so.

Patients in clinical studies often exhibit heterogeneous treatment effect (HTE). Classical subgroup analyses provide inferential tools to test for effect modification, while modern machine learning methods estimate the Conditional Average Treatment Effect (CATE) to enable individual level prediction. Each paradigm has limitations: inference focused approaches may sacrifice predictive utility, and prediction focused approaches often lack statistical guarantees. We present a hybrid two-stage workflow that integrates these perspectives. Stage 1 applies statistical inference to test whether credible treatment effect heterogeneity exists with the protection against spurious findings. Stage 2 translates heterogeneity evidence into individualized treatment policies, evaluated by cross fitted doubly robust (DR) metrics with Neyman-Pearson (NP) constraints on harm. We illustrate the workflow with working examples based on simulated data and a real ACTG 175 HIV trial. This tutorial provides practical implementation checklists and discusses links to sponsor oriented HTE workflows, offering a transparent and auditable pathway from heterogeneity assessment to individualized treatment policies.

We investigate Bayes posterior distributions in high-dimensional generalized linear models (GLMs) under the proportional asymptotics regime, where the number of features and samples diverge at a comparable rate. Specifically, we characterize the limiting behavior of finite-dimensional marginals of the posterior. We establish that the posterior does not contract in this setting. Yet, the finite-dimensional posterior marginals converge to Gaussian tilts of the prior, where the mean of the Gaussian depends on the true signal coordinates of interest. Notably, the effect of the prior survives even in the limit of large samples and dimensions. We further characterize the behavior of the posterior mean and demonstrate that the posterior mean can strictly outperform the maximum likelihood estimate in mean-squared error in natural examples. Importantly, our results hold regardless of the sparsity level of the underlying signal. On the technical front, we introduce leave-one-out strategies for studying these marginals that may be of independent interest for analyzing low-dimensional functionals of high-dimensional signals in other Bayesian inference problems.

In fixed-confidence best arm identification (BAI), the objective is to quickly identify the optimal option while controlling the probability of error below a desired threshold. Despite the plethora of BAI algorithms, existing methods typically fall short in practical settings, as stringent exact error control requires using loose tail inequalities and/or parametric restrictions. To overcome these limitations, we introduce a relaxed formulation that requires valid error control asymptotically with respect to a minimum sample size. This aligns with many real-world settings that often involve weak signals, high desired significance, and post-experiment inference requirements, all of which necessitate long horizons. This allows us to achieve tighter optimality, while better handling flexible nonparametric outcome distributions and fully leveraging individual-level contexts. We develop a novel asymptotic anytime-valid confidence sequences over arm indices, and we use it to design a new BAI algorithm for our asymptotic framework. Our method flexibly incorporates covariates for variance reduction and ensures approximate error control in fully nonparametric settings. Under mild convergence assumptions, we provide asymptotic bounds on the sample complexity and show the worst-case sample complexity of our approach matches the best-case sample complexity of Gaussian BAI under exact error guarantees and known variances. Experiments suggest our approach reduces average sample complexities while maintaining error control.

Online joint estimation of a dynamical model's unknown parameters and states with uncertainty quantification is crucial in many applications. For example, digital twins dynamically update their knowledge of model parameters and states to support prediction and decision-making. Reliability and computational speed are vital for DTs. Online parameter-state estimation ensures computational efficiency, while uncertainty quantification is essential for making reliable predictions and decisions. In parameter-state estimation, the joint distribution of the state and model parameters conditioned on the data, termed the joint posterior, provides accurate uncertainty quantification. Because the joint posterior is generally intractable to compute, this paper presents an online variational inference framework to compute its approximation at each time step. The approximation is factorized into a marginal distribution over the model parameters and a state distribution conditioned on the parameters. This factorization enables recursive updates through a two-stage procedure: first, the parameter posterior is approximated via variational inference; second, the state distribution conditioned on the parameters is computed using Gaussian filtering based on the approximate parameter posterior. The algorithmic design is supported by a theorem establishing upper bounds on the joint posterior approximation error. Numerical experiments demonstrate that the proposed method (i) accurately infers both unobserved states and unknown parameters of dynamical and observation models; (ii) remains robust under noisy, partial observations and model discrepancies in a chaotic Lorenz'96 system; and (iii) scales effectively to a high-dimensional state-space system arising from the spatial discretization of a convection-diffusion equation. outperforming the joint ensemble Kalman filter in this setting.

Many regularization priors for Bayesian regression assume the regression coefficients are a priori independent. In particular this is the case for standard Bayesian treatments of the lasso and the elastic net. While independence may be reasonable in some data-analytic settings, incorporating dependence in these prior distributions provides greater modeling flexibility. This paper introduces the orthant normal distribution in its general form and shows how it can be used to structure prior dependence in the Bayesian elastic net regression model. An L1-regularized version of Zellner's g prior is introduced as a special case, creating a new link between the literature on penalized optimization and an important class of regression priors. Computation is challenging due to an intractable normalizing constant in the prior. We avoid this issue by modifying slightly a standard prior of convenience for the hyperparameters in such a way to enable simple and fast Gibbs sampling of the posterior distribution. The benefit of including structured prior dependence in the Bayesian elastic net regression model is demonstrated through simulation and a near-infrared spectroscopy data example.

Many applications involve estimating the mean of multiple binomial outcomes as a common problem – assessing intergenerational mobility of census tracts, estimating prevalence of infectious diseases across countries, and measuring click-through rates for different demographic groups. The most standard approach is to report the plain average of each outcome. Despite simplicity, the estimates are noisy when the sample sizes or mean parameters are small. In contrast, the Empirical Bayes (EB) methods are able to boost the average accuracy by borrowing information across tasks. Nevertheless, the EB methods require a Bayesian model where the parameters are sampled from a prior distribution which, unlike the commonly-studied Gaussian case, is unidentified due to discreteness of binomial measurements. Even if the prior distribution is known, the computation is difficult when the sample sizes are heterogeneous as there is no simple joint conjugate prior for the sample size and mean parameter. In this paper, we consider the compound decision framework which treats the sample size and mean parameters as fixed quantities. We develop an approximate Stein's Unbiased Risk Estimator (SURE) for the average mean squared error given any class of estimators. For a class of machine learning-assisted linear shrinkage estimators, we establish asymptotic optimality, regret bounds, and valid inference. Unlike existing work, we work with the binomials directly without resorting to Gaussian approximations. This allows us to work with small sample sizes and/or mean parameters in both one-sample and two-sample settings. We demonstrate our approach using three datasets on firm discrimination, education outcomes, and innovation rates.

We consider the extent to which we can learn from a completely randomized experiment whether all individuals have treatment effects that are weakly of the same sign, a condition we call monotonicity. From a classical sampling perspective, it is well-known that monotonicity is not falsifiable. We show that from the design-based perspective—in which the potential outcomes are fixed and only treatment assignment is stochastic—that the distribution of treatment effects in the finite population (and hence whether monotonicity holds) is formally identified. Nevertheless, we show that the scope for learning about violations of monotonicity is severely limited. Frequentist tests of monotonicity have generically poor power, and there exist (non-degenerate) Bayesian priors that never update about whether monotonicity holds. Estimators of the magnitude of the violation of monotonicity are likewise shown to have poor minimax mean-squared error.