Papers

Modern deep learning techniques focus on extracting intricate information from data to achieve accurate predictions. However, the training datasets may be crowdsourced and include sensitive information, such as personal contact details, financial data, and medical records. As a result, there is a growing emphasis on developing privacy-preserving training algorithms for neural networks that maintain good performance while preserving privacy. In this paper, we investigate the generalization and privacy performances of the differentially private gradient descent (DP-GD) algorithm, which is a private variant of the gradient descent (GD) by incorporating additional noise into the gradients during each iteration. Moreover, we identify a concrete learning task where DP-GD can achieve superior generalization performance compared to GD in training two-layer Huberized ReLU convolutional neural networks (CNNs). Specifically, we demonstrate that, under mild conditions, a small signal-to-noise ratio can result in GD producing training models with poor test accuracy, whereas DP-GD can yield training models with good test accuracy and privacy guarantees if the signal-to-noise ratio is not too small. This indicates that DP-GD has the potential to enhance model performance while ensuring privacy protection in certain learning tasks. Numerical simulations are further conducted to support our theoretical results.

To analyze longitudinal zero-inflated count data, we extend existing models by introducing marginalized zero-inflated Poisson (MZIP) models with random effects, which explicitly capture the marginal effect of covariates and address limitations of previous methods. These models provide a clearer interpretation of the overall mean effect of covariates on zero-inflated count data. To further accommodate overdispersion, we develop marginalized zero-inflated negative binomial (MZINB) models. Both models incorporate subject-specific heterogeneity through a flexible random effects covariance structure. Simulation studies are conducted to evaluate the performance of the MZIP and MZINB models, comparing their inference under both homogeneous and heterogeneous random effects. Finally, we illustrate the applicability of the proposed models through an analysis of systemic lupus erythematosus data.

In 1957, Lindley published "A statistical paradox" in Biometrika, revealing a fundamental conflict between frequentist and Bayesian inference as sample size approaches infinity. We present a new paradox of a different kind: a conflict within Bayesian inference itself. In the normal model with known variance, we prove that for any two-sided statistically significant result at the 0.05 level there exist prior variances such that the Bayes factor indicates evidence for the alternative with one choice while indicating evidence for the null with another. Thus, the same data, testing the same hypothesis, can yield opposite conclusions depending solely on prior choice. This answers Robert's 2016 call to investigate the impact of the prior scale on Bayes factors and formalises his concern that this choice involves arbitrariness to a high degree. Unlike the Jeffreys-Lindley paradox, which requires sample size approaching infinity, the paradox we identify occurs with realistic sample sizes.

This work presents a probabilistic digital twin framework for response prediction in dynamical systems governed by misspecified physics. The approach integrates Gaussian Process Latent Force Models (GPLFM) and Bayesian Neural Networks (BNNs) to enable end-to-end uncertainty-aware inference and prediction. In the diagnosis phase, model-form errors (MFEs) are treated as latent input forces to a nominal linear dynamical system and jointly estimated with system states using GPLFM from sensor measurements. A BNN is then trained on posterior samples to learn a probabilistic nonlinear mapping from system states to MFEs, while capturing diagnostic uncertainty. For prognosis, this mapping is used to generate pseudo-measurements, enabling state prediction via Kalman filtering. The framework allows for systematic propagation of uncertainty from diagnosis to prediction, a key capability for trustworthy digital twins. The framework is demonstrated using four nonlinear examples: a single degree of freedom (DOF) oscillator, a multi-DOF system, and two established benchmarks – the Bouc-Wen hysteretic system and the Silverbox experimental dataset – highlighting its predictive accuracy and robustness to model misspecification.

We study the problem of selecting a small, representative action subset from an extremely large action space shared across a family of reinforcement learning (RL) environments – a fundamental challenge in applications like inventory management and recommendation systems, where direct learning over the entire space is intractable. Our goal is to identify a fixed subset of actions that, for every environment in the family, contains a near-optimal action, thereby enabling efficient learning without exhaustively evaluating all actions. This work extends our prior results for meta-bandits to the more general setting of Markov Decision Processes (MDPs). We prove that our existing algorithm achieves performance comparable to using the full action space. This theoretical guarantee is established under a relaxed, non-centered sub-Gaussian process model, which accommodates greater environmental heterogeneity. Consequently, our approach provides a computationally and sample-efficient solution for large-scale combinatorial decision-making under uncertainty.

Temporal point processes (TPPs) provide a natural mathematical framework for modeling heartbeats due to capturing underlying physiological inductive biases. In this work, we apply density-based neural TPPs to model heartbeat dynamics from 18 subjects. We adapt a goodness-of-fit framework from classical point process literature to Neural TPPs and use it to optimize hyperparameters, identify appropriate training sequence lengths to capture temporal dependencies, and demonstrate zero-shot predictive capability on heartbeat data.

Support vector machines are widely used in machine learning classification tasks, but traditional SVM models suffer from sensitivity to outliers and instability in resampling, which limits their performance in practical applications. To address these issues, this paper proposes a novel rescaled Huberized pinball loss function with asymmetric, non-convex, and smooth properties. Based on this loss function, we develop a corresponding SVM model called RHPSVM (Rescaled Huberized Pinball Loss Support Vector Machine). Theoretical analyses demonstrate that RHPSVM conforms to Bayesian rules, has a strict generalization error bound, a bounded influence function, and controllable optimality conditions, ensuring excellent classification accuracy, outlier insensitivity, and resampling stability. Additionally, RHPSVM can be extended to various advanced SVM variants by adjusting parameters, enhancing its flexibility. We transform the non-convex optimization problem of RHPSVM into a series of convex subproblems using the concave-convex procedure (CCCP) and solve it with the ClipDCD algorithm, which is proven to be convergent. Experimental results on simulated data, UCI datasets, and small-sample crop leaf image classification tasks show that RHPSVM outperforms existing SVM models in both noisy and noise-free scenarios, especially in handling high-dimensional small-sample data.

We present a scalable framework for computing polygenic risk scores (PRS) in high-dimensional genomic settings using the recently introduced Univariate-Guided Sparse Regression (uniLasso). UniLasso is a two-stage penalized regression procedure that leverages univariate coefficients and magnitudes to stabilize feature selection and enhance interpretability. Building on its theoretical and empirical advantages, we adapt uniLasso for application to the UK Biobank, a population-based repository comprising over one million genetic variants measured on hundreds of thousands of individuals from the United Kingdom. We further extend the framework to incorporate external summary statistics to increase predictive accuracy. Our results demonstrate that uniLasso attains predictive performance comparable to standard Lasso while selecting substantially fewer variants, yielding sparser and more interpretable models. Moreover, it exhibits superior performance in estimating PRS relative to its competitors, such as PRS-CS. Integrating external scores further improves prediction while maintaining sparsity.

Reliable and timely dengue predictions can provide an actionable lead time for targeted vector control and clinical preparedness, reducing preventable disease and health-system costs in at-risk communities. However, many forecasting approaches depend on site-specific covariates and entomological surveillance, which limits portability to data-sparse settings. In this work, we mathematically prove that a parabolic ICC structure, previously established for the basic SIR model, also holds in a substantially more complex model: a two-population (human-mosquito) four-serotype dengue transmission model with primary and secondary infections and mild/severe disease classes. To predict the number of new cases, we propose a data-parsimonious (DP) framework built on the incidence-cumulative cases (ICC) curve that requires only the human incidence time series of the target season and estimates only two parameters, thereby reducing estimation noise and computational burden. We further develop a Bayesian extension that accounts for case-reporting and fitting uncertainty, producing calibrated predictive intervals. The Bayesian model yields improved predictive performance compared to the parabolic ICC model. We evaluate the framework for dengue outbreaks in Florida in 2022-2023, where standardized clinical tests and reporting support accurate case determination.

Uncertainty quantification is vital for decision-making and risk assessment in machine learning. Mean-variance regression models, which predict both a mean and residual noise for each data point, provide a simple approach to uncertainty quantification. However, overparameterized mean-variance models struggle with signal-to-noise ambiguity, deciding whether prediction targets should be attributed to signal (mean) or noise (variance). At one extreme, models fit all training targets perfectly with zero residual noise, while at the other, they provide constant, uninformative predictions and explain the targets as noise. We observe a sharp phase transition between these extremes, driven by model regularization. Empirical studies with varying regularization levels illustrate this transition, revealing substantial variability across repeated runs. To explain this behavior, we develop a statistical field theory framework, which captures the observed phase transition in alignment with experimental results. This analysis reduces the regularization hyperparameter search space from two dimensions to one, significantly lowering computational costs. Experiments on UCI datasets and the large-scale ClimSim dataset demonstrate robust calibration performance, effectively quantifying predictive uncertainty.

Limited overlap between treated and control groups is a key challenge in observational analysis. Standard approaches like trimming importance weights can reduce variance but introduce a fundamental bias. We propose a sensitivity framework for contextualizing findings under limited overlap, where we assess how irregular the outcome function has to be in order for the main finding to be invalidated. Our approach is based on worst-case confidence bounds on the bias introduced by standard trimming practices, under explicit assumptions necessary to extrapolate counterfactual estimates from regions of overlap to those without. Empirically, we demonstrate how our sensitivity framework protects against spurious findings by quantifying uncertainty in regions with limited overlap.

Two binary instrumental variables (IVs) are nested if individuals who comply under one binary IV also comply under the other. This situation often arises when the two IVs represent different intensities of encouragement or discouragement to take the treatment, with one stronger than the other. In a nested IV structure, treatment effects can be identified for two latent subgroups: always-compliers and switchers. Always-compliers are individuals who comply even under the weaker IV, while switchers are those who do not comply under the weaker IV but do under the stronger IV. We introduce a novel pair-of-pairs nested IV design, where each matched stratum consists of four units organized in two pairs. We develop design-based inference for the always-complier sample average treatment effect and switcher sample average treatment effect. In a nested IV analysis, IV assignment is randomized within each IV pair; however, whether a study unit receives the weaker or stronger IV may not be randomized. To address this complication, we then propose a novel partly biased randomization scheme and study design-based inference under this new scheme. Using extensive simulation studies, we demonstrate the validity of the proposed method even in challenging scenarios with small sample sizes and a low proportion of switchers. Applying the nested IV framework, we estimated that 52.2% (95% CI: 50.4%-53.9%) of participants enrolled at the Henry Ford Health System in the Prostate, Lung, Colorectal, and Ovarian Cancer Screening Trial were always-compliers, while 26.7% (95% CI: 24.5%-28.9%) were switchers. Among always-compliers, flexible sigmoidoscopy was associated with a trend toward a decreased colorectal cancer rate. No effect was detected among switchers. This offers a richer interpretation of why no increase in the intention-to-treat effect was observed after 1997, even though the compliance rate rose.

The paper examines the construction and analysis of a new class of mixed exponential statistical structures that combine the properties of stochastic models and linear positive operators. The relevance of the topic is driven by the growing need to develop a unified theoretical framework capable of describing both continuous and discrete random structures that possess approximation properties. The aim of the study is to introduce and analyze a generalized family of mixed exponential statistical structures and their corresponding linear positive operators, which include known operators as particular cases. We define auxiliary statistical structures B and H through differential relations between their elements, and construct the main Phillips-type structure. Recurrent relations for the central moments are obtained, their properties are established, and the convergence and approximation accuracy of the constructed operators are investigated. The proposed approach allows mixed exponential structures to be viewed as a generalization of known statistical systems, providing a unified analytical and stochastic description. The results demonstrate that mixed exponential statistical structures can be used to develop new classes of positive operators with controllable preservation and approximation properties. The proposed methodology forms a basis for further research in constructing multidimensional statistical structures, analyzing operators in weighted spaces, and studying their asymptotic characteristics.

Bayesian optimization is widely used for optimizing expensive black box functions, but most existing approaches focus on scalar responses. In many scientific and engineering settings the response is functional, varying smoothly over an index such as time or wavelength, which makes classical formulations inadequate. Existing methods often minimize integrated error, which captures average performance but neglects worst case deviations. To address this limitation we propose min-max Functional Bayesian Optimization (MM-FBO), a framework that directly minimizes the maximum error across the functional domain. Functional responses are represented using functional principal component analysis, and Gaussian process surrogates are constructed for the principal component scores. Building on this representation, MM-FBO introduces an integrated uncertainty acquisition function that balances exploitation of worst case expected error with exploration across the functional domain. We provide two theoretical guarantees: a discretization bound for the worst case objective, and a consistency result showing that as the surrogate becomes accurate and uncertainty vanishes, the acquisition converges to the true min-max objective. We validate the method through experiments on synthetic benchmarks and physics inspired case studies involving electromagnetic scattering by metaphotonic devices and vapor phase infiltration. Results show that MM-FBO consistently outperforms existing baselines and highlights the importance of explicitly modeling functional uncertainty in Bayesian optimization.

Low-rank factorization of weight matrices is a promising approach to compress large language models (LLMs), but its practical effectiveness depends on allocating ranks across layers and efficiently representing the resulting factors. We introduce three complementary methods for improving SVD-based LLM compression. First, FermiGrad formulates global layer-wise rank allocation as a differentiable optimization problem by replacing discrete singular-value truncation with continuous Fermi gates. Because FermiGrad optimizes only rank parameters over precomputed low-rank factors, it avoids repeated activation decompositions and backpropagation through SVD operations. Second, the Targeted Compressor extends FermiGrad with a task-specific objective that directs limited model capacity toward a selected domain. At 60% parameter density for Llama-3.1-8B-Instruct, SciQ-targeted compression achieves 60% MMLU accuracy, exceeding the reported quantized Dobi-SVD result by 10 percentage points, while deliberately trading performance on unrelated tasks for domain specialization. Third, PivGa provides a lossless secondary compression of low-rank factors by exploiting the gauge freedom in their parametrization. The proposed framework is modular, computationally efficient, and applicable at scale, as demonstrated by compressing Llama-3.3-70B-Instruct to 50% parameter density.

Difference-in-differences (DID) is one of the most widely used causal inference frameworks in observational studies. However, most existing DID methods are designed for binary treatments and cannot be readily applied to non-binary treatment settings. Although recent work has begun to extend DID to non-binary (e.g., continuous) treatments, these approaches typically require strong additional assumptions, including parametric outcome models or the presence of idealized comparison units with (nearly) static treatment levels over time (commonly called “stayers” or “quasi-stayers”). In this technical note, we introduce a new non-bipartite matching framework for DID that naturally accommodates general treatment types (e.g., binary, ordinal, or continuous). Our framework makes three main contributions. First, we develop an optimal non-bipartite matching design for DID that jointly balances baseline covariates across comparable units (reducing bias) and maximizes contrasts in treatment trajectories over time (improving efficiency). Second, we establish a post-matching randomization condition, the design-based counterpart to the traditional parallel-trends assumption, which enables valid design-based inference. Third, we introduce the sample average DID ratio, a finite-population-valid and fully nonparametric causal estimand applicable to arbitrary treatment types. Our design-based approach that preserves the full treatment-dose information, avoids parametric assumptions, does not rely on the existence of stayers or quasi-stayers, and operates entirely within a finite-population framework, without appealing to hypothetical super-populations or outcome distributions.

Estimating the size of marginalized populations is a persistent challenge in survey statistics and public health, especially where stigma and legal restrictions exclude such groups from census and administrative data. Migrant domestic workers in Jordan represent one such population. We employ the Network Scale-up Method using the direct probe group method, estimating probe group sizes from survey respondents' own membership rather than relying on external counts. Using data from a nationally representative household survey in Jordan, we combine the direct probe group method with Bayesian logistic mixed-effects models to stabilize small-area estimates at the Governorate level. We validate the method against census data, demonstrating that direct probe group estimates yield reliable inference and provide a practical alternative where known probe group sizes are unavailable. Our results highlight regional variation in social network size and connectivity to migrant domestic workers. We argue that the direct probe group method is more likely to satisfy the conditions required for unbiased estimation than relying on official record sizes. This work provides the first systematic validation of the direct probe group method in a small-area setting and offers guidance for adapting the Network Scale-up Method to surveys with limited sample sizes.

Most scientific domains elicit the development of efficient algorithms and accessible scientific software. This thesis unifies our developments in three broad domains: Quasi-Monte Carlo (QMC) methods for efficient high-dimensional integration, Gaussian process (GP) regression for high-dimensional interpolation with built-in uncertainty quantification, and scientific machine learning (sciML) for modeling partial differential equations (PDEs) with mesh-free solvers. For QMC, we built new algorithms for vectorized error estimation and developed QMCPy (https://qmcsoftware.github.io/QMCSoftware/): an open-source Python interface to randomized low-discrepancy sequence generators, automatic variable transforms, adaptive error estimation procedures, and diverse use cases. For GPs, we derived new digitally-shift-invariant kernels of higher-order smoothness, developed novel fast multitask GP algorithms, and produced the scalable Python software FastGPs (https://alegresor.github.io/fastgps/). For sciML, we developed a new algorithm capable of machine precision recovery of PDEs with random coefficients. We have also studied a number of applications including GPs for probability of failure estimation, multilevel GPs for the Darcy flow equation, neural surrogates for modeling radiative transfer, and fast GPs for Bayesian multilevel QMC.

We study Sparse Multiple Kernel Learning (SMKL), which is the problem of selecting a sparse convex combination of prespecified kernels for support vector binary classification. Unlike prevailing l1 regularized approaches that approximate a sparsifying penalty, we formulate the problem by imposing an explicit cardinality constraint on the kernel weights and add an l2 penalty for robustness. We solve the resulting non-convex minimax problem via an alternating best response algorithm with two subproblems: the alpha subproblem is a standard kernel SVM dual solved via LIBSVM, while the beta subproblem admits an efficient solution via the Greedy Selector and Simplex Projector algorithm. We reformulate SMKL as a mixed integer semidefinite optimization problem and derive a hierarchy of semidefinite convex relaxations which can be used to certify near-optimality of the solutions returned by our best response algorithm and also to warm start it. On ten UCI benchmarks, our method with random initialization outperforms state-of-the-art MKL approaches in out-of-sample prediction accuracy on average by 3.34 percentage points (relative to the best performing benchmark) while selecting a small number of candidate kernels in comparable runtime. With warm starting, our method outperforms the best performing benchmark's out-of-sample prediction accuracy on average by 4.05 percentage points. Our convex relaxations provide a certificate that in several cases, the solution returned by our best response algorithm is the globally optimal solution.

Score-based generative models require guidance in order to generate plausible, on-manifold samples. The most popular guidance method, Classifier-Free Guidance (CFG), is only applicable in settings with labeled data and requires training an additional unconditional score-based model. More recently, Auto-Guidance adopts a smaller, less capable version of the original model to guide generation. While each method effectively promotes the fidelity of generated data, each requires labeled data or the training of additional models, making it challenging to guide score-based models when (labeled) training data are not available or training new models is not feasible. We make the surprising discovery that the positive curvature of log density estimates in saddle regions provides strong guidance for score-based models. Motivated by this, we develop saddle-free guidance (SFG) which maintains estimates of maximal positive curvature of the log density to guide individual score-based models. SFG has the same computational cost of classifier-free guidance, does not require additional training, and works with off-the-shelf diffusion and flow matching models. Our experiments indicate that SFG achieves state-of-the-art FID and FD-DINOv2 metrics in single-model unconditional ImageNet-512 generation. When SFG is combined with Auto-Guidance, its unconditional samples achieve general state-of-the-art in FD-DINOv2 score. Our experiments with FLUX.1-dev and Stable Diffusion v3.5 indicate that SFG boosts the diversity of output images compared to CFG while maintaining excellent prompt adherence and image fidelity.

Determining whether vaccine efficacy wanes is important for individual and public decision making. Yet, quantification of waning is a subtle task. The classical approaches cannot be interpreted as measures of declining efficacy unless we impose unreasonable assumptions. Recently, formal causal estimands designed to quantify vaccine waning have been proposed. These estimands can be bounded under weaker assumptions, but the bounds are often too wide to make claims about the presence of waning. We propose a different approach: a formal test to assess whether a treatment effect is constant over time at the individual level. This test provides a considerable power gain over existing approaches and is valid under interpretable assumptions in vaccine trials. We illustrate the increase in power through real and simulated examples, using three different approaches to compute the test statistics. Two of these approaches are based solely on summary data, accessible from existing clinical trials. Beyond our test, we also give new results that bound the waning effect. We use our methods to reanalyze data from a randomized controlled trial of the BNT162b2 COVID-19 vaccine. While prior analysis did not establish waning, our test rejects the null hypothesis of no waning.

South Korea faces the dual challenge of managing growing distributed solar energy surpluses and the high energy demand of industries like Bitcoin mining. Leveraging mining operations as a flexible load to monetize this `net-metering surplus' presents a viable synergy, but requires a robust site selection methodology. Traditional GIS-based Multi-Criteria Decision Analysis (MCDA) struggles with subjective weighting and integrating heterogeneous spatial data (areal-level and lattice-level). This thesis develops and implements a Two-Stage Hierarchical Optimization framework to overcome these limitations. Stage 1 (Areal-Level) employs a cost-benefit optimization to determine the optimal number ($K^*$) and combination of regions, maximizing a final adjusted net profit by balancing surplus power revenue against detailed land and non-linear infrastructure costs. Stage 2 (Point-Level) then uses a GIS-based sliding window search within these selected regions, applying topographic (slope $< 6.0^\circ$) and land-use constraints at a 30m resolution to identify physically constructible `unit sites'. The model identified an optimal configuration of $K^*=3$ regions (Yongin, Damyang, Miryang) yielding a maximum potential net profit of approximately \$307 million. Crucially, the Stage 2 screening revealed that Yongin, the most profitable region, was also the most physically constrained, 87% of sites filtered out. This research contributes a scalable, objective framework for energy infrastructure siting that effectively integrates multi-scale spatial data. It provides a data-driven strategy for policymakers and grid operators (like Korea Electric Power Corporation) to monetize curtailed renewables and enhance grid stability.

This article aims to propose and elucidate the analytical concepts of "individual scientist" and "integrated scientist" to depict the fundamental transformation in the modes of scientific research actors throughout the history of science. The "individual scientist" represents an early modern scientific research panel characterized by independence, egalitarian collaboration, and personal recognition, while the "integrated scientist" emerged in the context of "big science," marked by hierarchical teams, division of labor, collaboration, and the concentration of recognition on team leaders. Through historical review and case analysis, this article explores the underlying drivers of this transformation and focuses on its challenges and reconstructions concerning the name-based scientific reward system, aiming to provide a reflective perspective for contemporary scientific governance and research evaluation.

Causal effect estimation in networked systems is central to data-driven decision making. In such settings, interventions on one unit can spill over to others, and in complex physical or social systems, the interaction pathways driving these interference structures remain largely unobserved. We argue that for identifying population-level causal effects, it is not necessary to recover the exact network structure; instead, it suffices to characterize how those interactions contribute to the evolution of outcomes. Building on this principle, we study an evolution-based approach that investigates how outcomes change across observation rounds in response to interventions, hence compensating for missing network information. Using an exposure-mapping perspective, we give an axiomatic characterization of when the empirical distribution of outcomes follows a low-dimensional recursive equation, and identify minimal structural conditions under which such evolution mappings exist. We frame this as a distributional counterpart to difference-in-differences. Rather than assuming parallel paths for individual units, it exploits parallel evolution patterns across treatment scenarios to estimate counterfactual trajectories. A key insight is that treatment randomization plays a role beyond eliminating latent confounding; it induces an implicit sampling from hidden interference channels, enabling consistent learning about heterogeneous spillover effects. We highlight causal message passing as an instantiation of this method in dense networks while extending to more general interference structures, including influencer networks where a small set of units drives most spillovers. Finally, we discuss the limits of this approach, showing that strong temporal trends or endogenous interference can undermine identification.

AI/ML models have rapidly gained prominence as innovations for solving previously unsolved problems and their unintended consequences from amplifying human biases. Advocates for responsible AI/ML have sought ways to draw on the richer causal models of system dynamics to better inform the development of responsible AI/ML. However, a major barrier to advancing this work is the difficulty of bringing together methods rooted in different underlying assumptions (i.e., Dana Meadow's "the unavoidable a priori"). This paper brings system dynamics and structural equation modeling together into a common mathematical framework that can be used to generate systems from distributions, develop methods, and compare results to inform the underlying epistemology of system dynamics for data science and AI/ML applications.

Instrumental variable regression is a foundational tool for causal analysis across the social and biomedical sciences. Recent advances use kernel methods to estimate nonparametric causal relationships, with general data types, while retaining a simple closed-form expression. Empirical researchers ultimately need reliable inference on causal estimates; however, uniform confidence sets for the method remain unavailable. To fill this gap, we develop valid and sharp confidence sets for kernel instrumental variable regression, allowing general nonlinearities and data types. Computationally, our bootstrap procedure requires only a single run of the kernel instrumental variable regression estimator. Theoretically, it relies on the same key assumptions. Overall, we provide a practical procedure for inference that substantially increases the value of kernel methods for causal analysis.

Model selection in penalized regression critically depends on an accurate assessment of model complexity, commonly quantified through the effective degrees of freedom. While the Lasso admits a simple and unbiased characterization, given by the size of the active set, this property does not extend to adaptive penalization methods, despite the widespread use of this approximation in practice. To solve this issue, in this paper we derive a novel unbiased estimator of the effective degrees of freedom for the Adaptive Lasso within Stein's unbiased risk estimation framework. Our analysis reveals additional terms induced by data-dependent penalization, reflecting the role of adaptive weights and regularization in determining model complexity. We further revisit the Group Lasso, providing an alternative derivation of its degrees of freedom, and extend these results to the Adaptive Group Lasso. Importantly, we characterize the behavior of the degrees of freedom along the regularization path beyond the orthonormal design setting commonly assumed in the literature, providing a new theoretical description of this behavior under general design matrices. By correcting the common misuse of active set size as a proxy for degrees of freedom, our results enable more reliable risk estimation and inference, offering a rigorous foundation for understanding model complexity in adaptive penalized regression.

Markov Chain Monte Carlo (MCMC) is a flexible approach to approximate sampling from intractable probability distributions, with a rich theoretical foundation and comprising a wealth of exemplar algorithms. While the qualitative correctness of MCMC algorithms is often easy to ensure, their practical efficiency is contingent on the `target' distribution being reasonably well-behaved. In this work, we concern ourself with the scenario in which this good behaviour is called into question, reviewing an emerging line of work on `robust' MCMC algorithms which can perform acceptably even in the face of certain pathologies. We focus on two particular pathologies which, while simple, can already have dramatic effects on standard `local' algorithms. The first is roughness, whereby the target distribution varies so rapidly that the numerical stability of the algorithm is tenuous. The second is flatness, whereby the landscape of the target distribution is instead so barren and uninformative that one becomes lost in uninteresting parts of the state space. In each case, we formulate the pathology in concrete terms, review a range of proposed algorithmic remedies to the pathology, and outline promising directions for future research.

Quantifying how genomic features influence different parts of an outcome distribution requires statistical tools that go beyond mean regression, especially in ultrahigh-dimensional settings. Motivated by the study of LINE-1 activity in cancer, we propose StaRQR-K, a stabilized regional quantile regression framework with model-X knockoffs for false discovery rate control. StaRQR-K identifies CpG sites whose methylation levels are associated with specific quantile regions of an outcome, allowing detection of heterogeneous and tail-sensitive effects. The method combines an efficient regional quantile sure independence screening procedure with a winsorizing-based model-X knockoff filter, providing false discovery rate (FDR) control for regional quantile regression. Simulation studies show that StaRQR-K achieves valid FDR control and substantially higher power than existing approaches. In an application to The Cancer Genome Atlas head and neck cancer cohort, StaRQR-K reveals quantile-region-specific associations between CpG methylation and LINE-1 activity that improve out-of-sample prediction and highlight genomic regions with known functional relevance.

Spatially varying coefficients (SVC) models allow for marginal effects to be non-stationary over space and thus offer a higher degree of flexibility with respect to standard geostatistical models with external drift. At the same time, SVC models have the advantage that they are easily interpretable. They offer a flexible framework for understanding how the relationships between dependent and independent variables vary across space. The most common methods for modelling such data are the Geographically Weighted Regression (GWR) and Bayesian Gaussian Process (Bayes-GP). The Bayesian SVC model, which assumes that the coefficients follow Gaussian processes, provides a rigorous approach to account for spatial non-stationarity. However, the computational cost of Bayes-GP models can be prohibitively high when dealing with large datasets or/and when using a large number of covariates, due to the repeated inversion of dense covariance matrices required at each Markov chain Monte Carlo (MCMC) iteration. In this study, we propose an efficient Bayes-GP modeling framework leveraging the Vecchia approximation to reduce computational complexity while maintaining accuracy. The proposed method is applied to a challenging soil pollution data set in Toulouse, France, characterized by a high degree of censorship (two-thirds censored observations) and spatial clustering. Our results demonstrate the ability of the Vecchia-based Bayes-GP model to capture spatially varying effects and provide meaningful insights into spatial heterogeneity, even under the constraints of censored data.