Papers

The difference-in-differences (DID) research design is a key identification strategy which allows researchers to estimate causal effects under the parallel trends assumption. While the parallel trends assumption is counterfactual and cannot be tested directly, researchers often examine pre-treatment periods to check whether the time trends are parallel before treatment is administered. A recent literature has shown that existing preliminary tests have adverse effects on conventional statistical methods for estimation and inference, including low power, bias, and undercoverage. In this paper, we describe simple preliminary tests and corresponding confidence intervals for the causal effect which overcome these issues. Under mild separation conditions, the preliminary test is shown to be consistent and the confidence intervals for the causal effect have valid coverage conditional on passing the test. Our results hold under what we refer to as the conditional extrapolation assumption, which posits a relationship between the unidentified post-treatment violation of parallel trends and the identified pre-treatment violations. We view the conditional extrapolation assumption as one formalization of the assumption which is implicitly held when conducting a preliminary test for parallel trends. To illustrate the performance of the proposed methods, we use synthetic data as well as data on recentralization of public services in Vietnam and right-to-carry laws in Virginia.

Forecast combination and model averaging have become popular tools in forecasting and prediction, both of which combine a set of candidate estimates with certain weights and are often shown to outperform single estimates. A data-driven method to determine combination/averaging weights typically optimizes a criterion under certain weight constraints. While a large number of studies have been devoted to developing and comparing various weight choice criteria, the role of weight constraints on the properties of combination forecasts is relatively less understood, and the use of various constraints in practice is also rather arbitrary. In this study, we summarize prevalent weight constraints used in the literature, and theoretically and numerically compare how they influence the properties of the combined forecast. Our findings not only provide a comprehensive understanding on the role of various weight constraints but also practical guidance for empirical researchers how to choose relevant constraints based on prior information and targets.

This paper introduces a unified family of smoothed quantile estimators that continuously interpolate between classical empirical quantiles and the sample mean. The estimators q(z, h) are defined as minimizers of a regularized objective function depending on two parameters: a smoothing parameter h $\ge$ 0 and a location parameter z $\in$ R. When h = 0 and z $\in$ (-1, 1), the estimator reduces to the empirical quantile of order $\tau$ = (1z)/2; as h $\rightarrow$ $\infty$, it converges to the sample mean for any fixed z. We establish consistency, asymptotic normality, and an explicit variance expression characterizing the efficiency-robustness trade-off induced by h. A key geometric insight shows that for each fixed quantile level $\tau$ , the admissible parameter pairs (z, h) lie on a straight line in the parameter space, along which the population quantile remains constant while asymptotic efficiency varies. The analysis reveals two regimes: under light-tailed distributions (e.g., Gaussian), smoothing yields a monotonic but asymptotic variance reduction with no finite optimum; under heavy-tailed distributions (e.g., Laplace), a finite smoothing level h * ($\tau$ ) > 0 achieves strict efficiency improvement over the classical empirical quantile. Numerical illustrations confirm these theoretical predictions and highlight how smoothing balances robustness and efficiency across quantile levels.

Multi-output Gaussian process (MOGP) regression allows modelling dependencies among multiple correlated response variables. Similarly to standard Gaussian processes, MOGPs are sensitive to model misspecification and outliers, which can distort predictions within individual outputs. This situation can be further exacerbated by multiple anomalous response variables whose errors propagate due to correlations between outputs. To handle this situation, we extend and generalise the robust and conjugate Gaussian process (RCGP) framework introduced by Altamirano et al. (2024). This results in the multi-output RCGP (MO-RCGP): a provably robust MOGP that is conjugate, and jointly captures correlations across outputs. We thoroughly evaluate our approach through applications in finance and cancer research.

Given a noisy linear measurement $y = Ax + \xi$ of a distribution $p(x)$, and a good approximation to the prior $p(x)$, when can we sample from the posterior $p(x \mid y)$? Posterior sampling provides an accurate and fair framework for tasks such as inpainting, deblurring, and MRI reconstruction, and several heuristics attempt to approximate it. Unfortunately, approximate posterior sampling is computationally intractable in general. To sidestep this hardness, we focus on (local or global) log-concave distributions $p(x)$. In this regime, Langevin dynamics yields posterior samples when the exact scores of $p(x)$ are available, but it is brittle to score–estimation error, requiring an MGF bound (sub-exponential error). By contrast, in the unconditional setting, diffusion models succeed with only an $L^2$ bound on the score error. We prove that combining diffusion models with an annealed variant of Langevin dynamics achieves conditional sampling in polynomial time using merely an $L^4$ bound on the score error.

Adam [Kingma & Ba, 2015] is the de facto optimizer in deep learning, yet its theoretical understanding remains limited. Prior analyses show that Adam favors solutions aligned with $\ell_\infty$-geometry, but these results are restricted to the full-batch regime. In this work, we study the implicit bias of incremental Adam (using one sample per step) for logistic regression on linearly separable data, and show that its bias can deviate from the full-batch behavior. As an extreme example, we construct datasets on which incremental Adam provably converges to the $\ell_2$-max-margin classifier, in contrast to the $\ell_\infty$-max-margin bias of full-batch Adam. For general datasets, we characterize its bias using a proxy algorithm for the $\beta_2 \to 1$ limit. This proxy maximizes a data-adaptive Mahalanobis-norm margin, whose associated covariance matrix is determined by a data-dependent dual fixed-point formulation. We further present concrete datasets where this bias reduces to the standard $\ell_2$- and $\ell_\infty$-max-margin classifiers. As a counterpoint, we prove that Signum [Bernstein et al., 2018] converges to the $\ell_\infty$-max-margin classifier for any batch size. Overall, our results highlight that the implicit bias of Adam crucially depends on both the batching scheme and the dataset, while Signum remains invariant.

Characterizing the genetic basis of survival traits, such as age at disease onset, is critical for risk stratification, early intervention, and elucidating biological mechanisms that can inform therapeutic development. However, time-to-event outcomes in human cohorts are frequently right-censored, complicating both the estimation and partitioning of total heritability. Modern biobanks linked to electronic health records offer the unprecedented power to dissect the genetic basis of age-at-diagnosis traits at large scale. Yet, few methods exist for estimating and partitioning the total heritability of censored survival traits. Existing methods impose restrictive distributional assumptions on genetic and environmental effects and are not scalable to large biobanks with a million subjects. We introduce a censored multiple variance component model to robustly estimate the total heritability of survival traits under right-censoring. We demonstrate through extensive simulations that the method provides accurate total heritability estimates of right-censored traits at censoring rates up to 80% given sufficient sample size. The method is computationally efficient in estimating one hundred genetic variance components of a survival trait using large-scale biobank genotype data consisting of a million subjects and a million SNPs in nine hours, including uncertainty quantification. We apply our method to estimate the total heritability of four age-at-diagnosis traits from the UK Biobank study. Our results establish a scalable and robust framework for heritability analysis of right-censored survival traits in large-scale genetic studies.

In this work we extend the results developed in 2022 for a sequential change detection algorithm making use of Page's CUSUM statistic, the empirical distribution as an estimate of the pre-change distribution, and a universal code as a tool for estimating the post-change distribution, from the i.i.d. case to the Markov setup.

Spatial regression models have a variety of applications in several fields ranging from economics to public health. Typically, it is of interest to select important exogenous predictors of the spatially autocorrelated response variable. In this paper, we propose variable selection in linear spatial lag models by means of the focussed information criterion (FIC). The FIC-based variable selection involves the minimization of the asymptotic risk in the estimation of a certain parametric focus function of interest under potential model misspecification. We systematically investigate the key asymptotics of the maximum likelihood estimators under the sequence of locally perturbed mutually contiguous probability models. Using these results, we obtain the expressions for the bias and the variance of the estimated focus leading to the desired FIC formula. We provide practically useful focus functions that account for various spatial characteristics such as mean response, variability in the estimation and spatial spillover effects. Furthermore, we develop an averaged version of the FIC that incorporates varying covariate levels while evaluating the models. The empirical performance of the proposed methodology is demonstrated through simulations and real data analysis.

Physics-informed machine learning (PIML) integrates prior physical information, often in the form of differential equation constraints, into the process of fitting machine learning models to physical data. Popular PIML approaches, including neural operators, physics-informed neural networks, neural ordinary differential equations, and neural discrete equilibria, are typically fit to objectives that simultaneously include both data and physical constraints. However, the multi-objective nature of this approach creates ambiguity in the measurement of model quality. This is related to a poor understanding of epistemic uncertainty, and it can lead to surprising failure modes, even when existing statistical metrics suggest strong fits. Working within a Gaussian process regression framework, we introduce the Physics-Informed Log Evidence (PILE) score. Bypassing the ambiguities of test losses, the PILE score is a single, uncertainty-aware metric that provides a selection principle for hyperparameters of a PIML model. We show that PILE minimization yields excellent choices for a wide variety of model parameters, including kernel bandwidth, least squares regularization weights, and even kernel function selection. We also show that, even prior to data acquisition, a special 'data-free' case of the PILE score identifies a priori kernel choices that are 'well-adapted' to a given PDE. Beyond the kernel setting, we anticipate that the PILE score can be extended to PIML at large, and we outline approaches to do so.

Cancer cells acquire DNA mutations through many processes, such as environmental exposures and dysregulated repair mechanisms, and each process consistently produces distinct mutation types at characteristic frequencies, referred to as its signature. The usual approach for inferring these signatures is to decompose the matrix of mutation counts from a sample of tumors via non-negative matrix factorization (NMF). However, existing methods do not model the heterogeneity of mutation rates along the genome, which is driven in part by observed genomic features. In this paper, we introduce Poisson process factorization (PPF), which addresses this limitation by employing an inhomogeneous Poisson point process model to infer mutational signatures and their activities as they vary across the genome. PPF generalizes the baseline NMF model by representing a patient's exposure to each signature as a locus-specific function that depends on genomic covariates and patient-specific copy numbers via a log-linear model. We apply PPF to a sample of 113 breast tumors with $707{,}104$ mutations, using covariates representing histone modifications, cell replication timing, nucleosome positioning, and DNA methylation. Our analysis quantifies the joint effects of these features on the mutational processes in breast cancer, and estimates patient-specific activities of each signature as a function of genomic position.

This paper proposes a novel paradigm for machine learning that moves beyond traditional parameter optimization. Unlike conventional approaches that search for optimal parameters within a fixed geometric space, our core idea is to treat the model itself as a malleable geometric entity. Specifically, we optimize the metric tensor field on a manifold with a predefined topology, thereby dynamically shaping the geometric structure of the model space. To achieve this, we construct a variational framework whose loss function carefully balances data fidelity against the intrinsic geometric complexity of the manifold. The former ensures the model effectively explains observed data, while the latter acts as a regularizer, penalizing overly curved or irregular geometries to encourage simpler models and prevent overfitting. To address the computational challenges of this infinite-dimensional optimization problem, we introduce a practical method based on discrete differential geometry: the continuous manifold is discretized into a triangular mesh, and the metric tensor is parameterized by edge lengths, enabling efficient optimization using automatic differentiation tools. Theoretical analysis reveals a profound analogy between our framework and the Einstein-Hilbert action in general relativity, providing an elegant physical interpretation for the concept of "data-driven geometry". We further argue that even with fixed topology, metric optimization offers significantly greater expressive power than models with fixed geometry. This work lays a solid foundation for constructing fully dynamic "meta-learners" capable of autonomously evolving their geometry and topology, and it points to broad application prospects in areas such as scientific model discovery and robust representation learning.

We propose a data-driven framework for efficiently solving quadratic programming (QP) problems by reducing the number of variables in high-dimensional QPs using instance-specific projection. A graph neural network-based model is designed to generate projections tailored to each QP instance, enabling us to produce high-quality solutions even for previously unseen problems. The model is trained on heterogeneous QPs to minimize the expected objective value evaluated on the projected solutions. This is formulated as a bilevel optimization problem; the inner optimization solves the QP under a given projection using a QP solver, while the outer optimization updates the model parameters. We develop an efficient algorithm to solve this bilevel optimization problem, which computes parameter gradients without backpropagating through the solver. We provide a theoretical analysis of the generalization ability of solving QPs with projection matrices generated by neural networks. Experimental results demonstrate that our method produces high-quality feasible solutions with reduced computation time, outperforming existing methods.

This paper reinterprets the Synthetic Control (SC) framework through the lens of weighting philosophy, arguing that the contrast between traditional SC and Difference-in-Differences (DID) reflects two distinct modeling mindsets: sparse versus dense weighting schemes. Rather than viewing sparsity as inherently superior, we treat it as a modeling choice simple but potentially fragile. We propose an L-infinity-regularized SC method that combines the strengths of both approaches. Like DID, it employs a denser weighting scheme that distributes weights more evenly across control units, enhancing robustness and reducing overreliance on a few control units. Like traditional SC, it remains flexible and data-driven, increasing the likelihood of satisfying the parallel trends assumption while preserving interpretability. We develop an interior point algorithm for efficient computation, derive asymptotic theory under weak dependence, and demonstrate strong finite-sample performance through simulations and real-world applications.

We study nonparametric distance-based (isotropic) local polynomial methods for estimating the boundary average treatment effect curve, a causal functional that captures treatment effect heterogeneity in boundary discontinuity designs. We establish identification, estimation, and inference results both pointwise and uniformly along the treatment assignment boundary. We show that the geometric regularity of the boundary, a one-dimensional manifold, plays a central role in determining feasible convergence rates and valid inference procedures. Our theoretical contributions are threefold. First, we derive uniform lower and upper bounds on the convergence rate of the misspecification bias of isotropic local polynomial estimators. Second, we obtain uniform distributional approximations that justify boundary-robust inference. Third, we establish minimax lower bounds for a broad class of nonparametric isotropic regression estimators. These results yield practical guidance for empirical implementation, including new bandwidth selection rules that adapt to local irregularities of the treatment-assignment boundary. We illustrate the proposed methods using simulation evidence and an empirical application, and provide companion general-purpose software.

Class imbalance complicates probabilistic classification because standard training objectives emphasize majority-class performance. Synthetic oversampling can reduce imbalance, but discrepancies between the synthetic and target minority distributions may bias the fitted classifier, especially because synthetic samples depend on the observed data. We propose a bias-correction procedure that estimates the generator-induced loss discrepancy from a held-out subset of majority observations and transfers this correction to the minority class under a uniform bias-transfer condition. We establish finite-sample bounds for bias transfer and for the excess balanced risk of the resulting empirical risk minimizer, and characterize a regime in which SMOTE induces non-negligible loss bias. The framework can also be implemented in imbalanced multi-task learning and propensity-score estimation, with details provided in the Supplementary Material. Real data analyses show that the correction is most useful when synthetic distortion is appreciable.

We establish a rigorous asymptotic theory for the joint estimation of roughness and scale parameters in two-dimensional Gaussian random fields with power-law generalized covariances \cite{Matheron1973, Stein1999, Yaglom1987}. Our main results are bivariate central limit theorems for a class of method-of-moments estimators under increasing-domain and fixed-domain asymptotics. The fixed-domain result follows immediately from the increasing-domain result from the self-similarity of Gaussian random fields with power-law generalized covariances \cite{IstasLang1997, Coeurjolly2001, ZhuStein2002}. These results provide a unified distributional framework across these two classical regimes \cite{AvramLeonenkoSakhno2010-ESAIM, BiermeBonamiLeon2011-EJP} that makes the unusual behavior of the estimates under fixed-domain asymptotics intuitively obvious. Our increasing-domain asymptotic results use spatial averages of quadratic forms of (iterated) bilinear product difference filters that yield explicit expressions for the estimates of roughness and scale to which existing theorems on such averages \cite{BreuerMajor1983,Hannan1970} can be readily applied. We further show that the asymptotics remain valid under modestly irregular sampling due to jitter or missing observations. For the fixed-domain setting, the results extend to models that behave sufficiently like the power-law model at high frequencies such as the often used Matérn model \cite{ZhuStein2006, WangLoh2011EJS, KaufmanShaby2017EJS}.

Kernel logistic regression (KLR) is a powerful classification method widely applied across diverse domains. In many real-world scenarios, indefinite kernels capture more domain-specific structural information than positive definite kernels. This paper proposes a novel $L_1$-norm regularized indefinite kernel logistic regression (RIKLR) model, which extends the existing IKLR framework by introducing sparsity via an $L_1$-norm penalty. The introduction of this regularization enhances interpretability and generalization while introducing nonsmoothness and nonconvexity into the optimization landscape. To address these challenges, a theoretically grounded and computationally efficient proximal linearized algorithm is developed. Experimental results on multiple benchmark datasets demonstrate the superior performance of the proposed method in terms of both accuracy and sparsity.

Cycle rickshaw pullers are highly vulnerable to extreme heat, yet little is known about how their physiological biomarkers respond under such conditions. This study collected real-time weather and physiological data using wearable sensors from 100 rickshaw pullers in Dhaka, Bangladesh. In addition, interviews with 12 pullers explored their knowledge, perceptions, and experiences related to climate change. We developed a Linear Gaussian Bayesian Network (LGBN) regression model to predict key physiological biomarkers based on activity, weather, and demographic features. The model achieved normalized mean absolute error values of 0.82, 0.47, 0.65, and 0.67 for skin temperature, relative cardiac cost, skin conductance response, and skin conductance level, respectively. Using projections from 18 CMIP6 climate models, we layered the LGBN on future climate forecasts to analyze survivability for current (2023-2025) and future years (2026-2100). Based on thresholds of WBGT above 31.1{\deg}C and skin temperature above 35{\deg}C, 32% of rickshaw pullers already face high heat exposure risk. By 2026-2030, this percentage may rise to 37% with average exposure lasting nearly 12 minutes, or about two-thirds of the trip duration. A thematic analysis of interviews complements these findings, showing that rickshaw pullers recognize their increasing climate vulnerability and express concern about its effects on health and occupational survivability.

This project began by constructing an index of economic insecurity using multiple socioeconomic indicators. Although poverty alone predicted SNAP participation more accurately than the composite index, its explanatory power was weaker than anticipated, echoing past findings that enrollment cannot be explained by income alone. This led to a shift in focus: identifying ZIP codes with high poverty but unexpectedly low SNAP participation, areas defined here as having a SNAP Gap, where ZIPs fall in the top 30 percent of family poverty and the bottom 10 percent of SNAP enrollment. Using nationally available ZIP level data from 2014 to 2023, I trained logistic classification models on four interpretable structural indicators: lack of vehicle, lack of internet access, lack of computer access, and percentage of adults with only a high school diploma. The most effective model relies on just two predictors, vehicle access and education, and outperforms tree based classifiers in both precision and calibration. Results show that economic insecurity is consistently concentrated in rural ZIP codes, with transportation access emerging as the most stable barrier to program take up. This study provides a nationwide diagnostic framework that can inform the development of scalable screening tools for targeting outreach and improving benefit access in underserved communities.

flowengineR is an R package designed to provide a modular and extensible framework for building reproducible algorithmic workflows for general-purpose machine learning pipelines. It is motivated by the rapidly evolving field of algorithmic fairness, where new metrics, mitigation strategies, and methods continuously emerge. A central challenge in fairness, but also far beyond, is that existing toolkits either focus narrowly on single interventions or treat reproducibility and extensibility as secondary considerations rather than core design principles. flowengineR addresses this by introducing a unified architecture of standardized engines for data splitting, execution, preprocessing, training, inprocessing, postprocessing, evaluation, and reporting. Each engine encapsulates one methodological task yet communicates via a lightweight interface, ensuring workflows remain transparent, auditable, and easily extensible. Although implemented in R, flowengineR builds on ideas from workflow languages (CWL, YAWL), graph-oriented visual programming languages (KNIME), and R frameworks (BatchJobs, batchtools). Its emphasis, however, is less on orchestrating engines for resilient parallel execution than on the straightforward setup and management of distinct engines as data structures. This orthogonalization enables distributed responsibilities, independent development, and streamlined integration. In the context of fairness, by structuring fairness methods as interchangeable engines, flowengineR lets researchers integrate, compare, and evaluate interventions across the modeling pipeline. At the same time, the architecture generalizes to explainability, robustness, and compliance metrics without core modifications. While motivated by fairness, flowengineR ultimately provides a general infrastructure for any workflow context where reproducibility, transparency, and extensibility are essential.

Automated monitoring of marine mammals in the St. Lawrence Estuary faces extreme challenges: calls span low-frequency moans to ultrasonic clicks, often overlap, and are embedded in variable anthropogenic and environmental noise. We introduce a multi-step, attention-guided framework that first segments spectrograms to generate soft masks of biologically relevant energy and then fuses these masks with the raw inputs for multi-band, denoised classification. Image and mask embeddings are integrated via mid-level fusion, enabling the model to focus on salient spectrogram regions while preserving global context. Using real-world recordings from the Saguenay St. Lawrence Marine Park Research Station in Canada, we demonstrate that segmentation-driven attention and mid-level fusion improve signal discrimination, reduce false positive detections, and produce reliable representations for operational marine mammal monitoring across diverse environmental conditions and signal-to-noise ratios. Beyond in-distribution evaluation, we further assess the generalization of Mask-Guided Classification (MGC) under distributional shifts by testing on spectrograms generated with alternative acoustic transformations. While high-capacity baseline models lose accuracy in this Out-of-distribution (OOD) setting, MGC maintains stable performance, with even simple fusion mechanisms (gated, concat) achieving comparable results across distributions. This robustness highlights the capacity of MGC to learn transferable representations rather than overfitting to a specific transformation, thereby reinforcing its suitability for large-scale, real-world biodiversity monitoring. We show that in all experimental settings, the MGC framework consistently outperforms baseline architectures, yielding substantial gains in accuracy on both in-distribution and OOD data.

Reliable uncertainty quantification is crucial for reinforcement learning (RL) in high-stakes settings. We propose a unified conformal prediction framework for infinite-horizon policy evaluation that constructs distribution-free prediction intervals {for returns} in both on-policy and off-policy settings. Our method integrates distributional RL with conformal calibration, addressing challenges such as unobserved returns, temporal dependencies, and distributional shifts. We propose a modular pseudo-return construction based on truncated rollouts and a time-aware calibration strategy using experience replay and weighted subsampling. These innovations mitigate model bias and restore approximate exchangeability, enabling uncertainty quantification even under policy shifts. Our theoretical analysis provides coverage guarantees that account for model misspecification and importance weight estimation. Empirical results, including experiments in synthetic and benchmark environments like Mountain Car, show that our method significantly improves coverage and reliability over standard distributional RL baselines.

The InfoNCE objective, originally introduced for contrastive representation learning, has become a popular choice for mutual information (MI) estimation, despite its indirect connection to MI. In this paper, we demonstrate why InfoNCE should not be regarded as a valid MI estimator, and we introduce a simple modification, which we refer to as InfoNCE-anchor, for accurate MI estimation. Our modification introduces an auxiliary anchor class, enabling consistent density ratio estimation and yielding a plug-in MI estimator with significantly reduced bias. Beyond this, we generalize our framework using proper scoring rules, which recover InfoNCE-anchor as a special case when the log score is employed. This formulation unifies a broad spectrum of contrastive objectives, including NCE, InfoNCE, and $f$-divergence variants, under a single principled framework. Empirically, we find that InfoNCE-anchor with the log score achieves the most accurate MI estimates; however, in self-supervised representation learning experiments, we find that the anchor does not improve the downstream task performance. These findings corroborate that contrastive representation learning benefits not from accurate MI estimation per se, but from the learning of structured density ratios.

We present a tractable framework for detecting changes in performance metrics and apply these methods to Major League Baseball (MLB) batting and pitching data from the 2023 and 2024 seasons. We propose a changepoint detection algorithm that combines a likelihood-based approach with split-sample inference to better control false positives, using either nonparametric tests or tests appropriate to the underlying data distribution. These tests incorporate a shift parameter, allowing users to specify the minimum magnitude of change to detect. We demonstrate the utility of this approach across simulation studies and several baseball applications: detecting changes in batter plate discipline metrics (e.g., chase and whiff rate), identifying velocity changes in pitcher fastballs, and validating velocity changepoints against a curated quasi-ground-truth dataset of pitchers who transitioned from relief to starting roles. Our method flags meaningful changes in 91% of these "ground-truth" cases and reveals that, for some metrics, more than 60% of detected changes occur in-season. While developed for baseball, the proposed framework is broadly applicable to any setting involving monitoring of individual performance over time.

We propose a mathematically principled PDE gradient flow framework for distributionally robust optimization (DRO). Exploiting the recent advances in the intersection of Markov Chain Monte Carlo sampling and gradient flow theory, we show that our theoretical framework can be implemented as practical algorithms for sampling from worst-case distributions and, consequently, DRO. While numerous previous works have proposed various reformulation techniques and iterative algorithms, we contribute a sound gradient flow view of the distributional optimization that can be used to construct new algorithms. As an example of applications, we solve a class of Wasserstein and Sinkhorn DRO problems using the recently-discovered Wasserstein Fisher-Rao and Stein variational gradient flows. Notably, we also show some simple reductions of our framework recover exactly previously proposed popular DRO methods, and provide new insights into their theoretical limit and optimization dynamics. Numerical studies based on stochastic gradient descent provide empirical backing for our theoretical findings.

We consider the problem of estimating a causal effect in a multi-domain setting. The causal effect of interest is confounded by an unobserved confounder and can change between the different domains. We assume that we have access to a proxy of the hidden confounder and that all variables are discrete or categorical. We propose methodology to estimate the causal effect in the target domain, where we assume to observe only the proxy variable. Under these conditions, we prove identifiability (even when treatment and response variables are continuous). We introduce two estimation techniques, prove consistency, and derive confidence intervals. The theoretical results are supported by simulation studies and a real-world example studying the causal effect of website rankings on consumer choices.

We study the recovery of the distribution function $F_X$ of a random variable $X$ that is subject to an independent additive random error $\varepsilon$. To be precise, it is assumed that the target variable $X$ is available only in the form of a blurred surrogate $Y = X + \varepsilon$. The distribution function $F_Y$ then corresponds to the convolution of $F_X$ and $F_\varepsilon$, so that the reconstruction of $F_X$ is some kind of deconvolution problem. Those have a long history in mathematics and various approaches have been proposed in the past. Most of them use integral transforms or matrix algorithms. The present article avoids these tools and is entirely confined to the domain of distribution functions. Our main idea relies on a transformation of a first kind to a second kind integral equation. Thereof, starting with a right-lateral discrete target and error variable, a representation for $F_X$ in terms of available quantities is obtained, which facilitates the unbiased estimation through a $Y$-sample. It turns out that these results even extend to cases in which $X$ is not discrete. Finally, in a general setup, our approach gives rise to an approximation for $F_X$ as a certain Neumann sum. The properties of this sum are briefly examined theoretically and visually. The paper is concluded with a short discussion of operator theoretical aspects and an outlook on further research. Various plots underline our results and illustrate the capabilities of our functions with regard to estimation.

We derive a family of Markov Chain Monte Carlo (MCMC) sampling methods based on following ray paths in a medium where the refractive index $n(x)$ is a function of the desired likelihood $\mathcal{L}(x)$, extending past work on isokinetic sampling. The simplest ray tracing method propagates rays at constant speed through parameter space, leading to orders of magnitude higher resilience to heating for stochastic gradients as compared to both Hamiltonian Monte Carlo (HMC) and Stochastic Gradient HMC, as well as the ability to cross any likelihood barrier, including holes in parameter space. Using the simplest ray tracing method, we sample the posterior distributions of neural network outputs for a variety of different architectures, including a preliminary exploration of the 1.5 billion-parameter GPT-2 (Generative Pre-trained Transformer 2) architecture, all on a single consumer-level GPU. We also show that prior samplers including traditional HMC, the original microcanonical HMC, Metropolis, Gibbs, and even Monte Carlo integration are special cases within a generalized ray tracing framework, which can sample according to an arbitrary weighting function. Public code and documentation for C, JAX, and PyTorch are available at https://bitbucket.org/pbehroozi/ray-tracing-sampler/src

We consider a stochastic multi-armed bandit problem with i.i.d. rewards where the expected reward function is multimodal with at most m modes. We propose the first known computationally tractable algorithm for computing the solution to the Graves-Lai optimization problem, which in turn enables the implementation of asymptotically optimal algorithms for this bandit problem. The code for the proposed algorithms is publicly available at https://github.com/wilrev/MultimodalBandits