Papers

We propose a simple two-stage order selector for finite-order ARFIMA models. First, a preliminary log-periodogram estimate of the memory parameter is used to fractionally filter the data. Second, a Hannan–Rissanen residual construction is applied to the filtered series, and the autoregressive and moving-average orders are selected by a generalized information criterion over a growing candidate rectangle. The search bounds are allowed to satisfy \(P_n,Q_n\to\infty\), whereas the true orders remain fixed and finite. The penalty is allowed to be larger than the ordinary BIC penalty so that it dominates the error introduced by preliminary long-memory estimation and by the Hannan–Rissanen residual approximation. We prove a uniform residual-variance approximation over the growing rectangle and combine it with a population separation argument between the true finite ARMA representation and underfitted alternatives. The resulting generalized information-criterion selector is consistent.

The inverse Gaussian (IG) process is a widely used model for univariate degradation data. For bivariate degradation data involving two performance characteristics (PCs), dependence is often introduced through an unobserved shared frailty factor combined with IG processes. Previous studies typically assume a specific frailty distribution, such as normal or gamma, although such choices are difficult to justify because the frailty is unobserved. This paper proposes a general IG GG framework for modeling bivariate degradation data with dependent PCs. Each degradation process is modeled using an IG process, while the shared frailty follows the generalized gamma (GG) family, which includes exponential, gamma, Weibull, and lognormal distributions as special cases. The proposed framework allows flexible selection of an appropriate frailty distribution within the GG family, leading to improved model fitting. Convenient parameter estimation procedures are developed and evaluated through simulation studies, demonstrating satisfactory performance. The proposed model is applied to fatigue crack data and compared with several existing frailty based and copula based models. Results show that the IG GG model provides a superior fit. System reliability estimation under the IG GG framework is also discussed.

In one extension of scalar-on-function regression modeling, the covariate is taken to be a density that is estimated from a finite number of measurements gathered for each observational unit. When this number of measurements is relatively small, the estimated coefficient function suffers from attenuation bias. This paper studies how the bias depends on the number of measurements per unit and proposes a bias-correction method based on simulation extrapolation (SIMEX). We establish that the bias decreases monotonically as the number of measurements per unit increases. The proposed SIMEX procedure applies bootstrap resampling to simulate smaller measurement counts and then extrapolates to infinitely many measurements, thereby correcting finite-measurement bias. A comprehensive simulation study, conducted over a range of sample sizes and noise levels, shows that the mean integrated squared error of the coefficient function decreases with more measurements per unit and that the SIMEX-extrapolated estimates achieve lower bias than the naive estimates based on the full set of measurements. The practical utility of the method is further illustrated through an application to the National Health and Nutrition Examination Survey, for which we relate 24-hour physical activity profiles to all-cause mortality. This example supports the validity of the method and demonstrates its ability to detect and correct for finite-measurement bias.

We revisit the problem of sequentially testing the mean of bounded distributions in a level-$\alpha$ power-one framework. We study a $\mathrm{KL_{inf}}$-based sequential test that is known to attain the information-theoretic lower bound on the expected stopping time with exact constants as $\alpha \to 0$. Going beyond first-order asymptotics, we establish a central limit theorem (CLT) for the stopping time of this test. Our analysis proceeds in two steps. First, we prove a novel CLT for the $\mathrm{KL_{inf}}$ statistic itself, characterizing its fluctuations around its deterministic limit. We then leverage this result to show that the stopping time, centered appropriately and scaled by $\sqrt{\log(1/\alpha)}$, converges in distribution to a Gaussian limit with an explicit variance. This yields a second-order characterization of an asymptotically optimal sequential test for bounded distributions. Finally, we present numerical experiments that corroborate our theoretical findings.

Spatial transcriptomics is a rapidly growing technique that captures gene expression together with spatial coordinates in intact tissue sections, enabling in situ mapping of transcriptional activity. This technology offers unprecedented opportunities to study tissue heterogeneity and spatial gene expression patterns. Uncovering the associations between spatially variable gene modules and spot types can advance our understanding of pathological mechanisms. However, rigorous statistical methods that exploit spatial information to achieve spatially coherent co-clustering of spots and genes are still lacking, and theoretical investigations in this direction remain limited. We propose a fused spatial latent block model (F-SpLBM). Our model uses the LBM to uncover co-expression patterns between spots and genes, penalized fusion to automatically determine the number of co-clusters, and the Potts model to incorporate spatial information. We establish that the fusion-based procedure recovers the true block structure with the misclassification rate converging at a super-polynomial rate. We also prove asymptotic normality of the parameter estimators and quantify the accuracy gain from spatial smoothing. Simulations and real-data analyses demonstrate that F-SpLBM yields spatially coherent and biologically interpretable clustering results.

Watermarking methods for language models have been studied extensively in the autoregressive setting, where tokens are generated sequentially. These works largely focus on local-context schemes that perturb the next token's distribution as a function of its preceding tokens. In diffusion language models, distributions over many unresolved positions are jointly sampled, allowing additive statistics of the entire sequence to be tractable during generation. We propose a watermark for masked diffusion language models that controls a global, vector-valued sketch representation of the text. Compared to context-dependent watermarking, the sketch formulation decouples detection from the local contexts seen during generation, resulting in an order-agnostic statistic and a watermarking rule which does not manifest as a simple token bias. We analyze the distortion, soundness, and robustness properties of the method.

In this paper, we study the gradient descent dynamics for jointly training both layers of a one-hidden-layer ReLU network to fit a linear target function. Concretely, we consider a realizable setting where inputs are drawn i.i.d. from a Gaussian distribution and labels follow a planted linear model. This stylized framework captures salient features of end-to-end training in inverse problems and certain auto-encoder models. Despite its apparent simplicity, the dynamics remain poorly understood, in part because the loss landscape contains multiple non-strict saddle points, making it unclear why gradient descent from random initialization reliably escapes bad stationary regions. We provide a detailed characterization of the optimization landscape and prove that gradient descent from a moderately small random initialization-simultaneously training both layers-converges to a global minimizer at a linear rate with order-wise optimal sample complexity. Our analysis tracks the trajectory through three phases: an alignment phase in which hidden weights progressively align with the planted direction while the output weights maintain the correct sign pattern; a growth phase in which the norms of both layers increase while preserving alignment; and a local refinement phase in which the aligned neurons rapidly converge to the planted direction, yielding fast local convergence. To rigorously show that GD avoids non-strict saddles, we develop trajectory-level control arguments for the end-to-end dynamics. In addition, we establish novel uniform concentration results that hold along the entire trajectory, and are essential for obtaining order-wise optimal sample complexity. We corroborate our theory with extensive experiments across a range of configurations.

Real estate valuation is a structured regression problem in which prices are governed by heterogeneous feature types, sparse regional effects, nonlinear interactions, and the practical logic of comparable properties. Standard multilayer perceptrons treat each row as an isolated vector and must learn locality, scale sensitivity, and categorical matching from supervision alone. Gradient-boosted decision trees provide strong tabular baselines, but their feature-centric splitting mechanism does not explicitly model the retrieval of similar historical observations. This paper presents RowNet, a retrieval-based neural architecture for real estate price-per-square-meter prediction. RowNet represents a query property through pairwise similarity features against a memory bank of labeled properties. A first retrieval layer estimates a coarse target from feature-only similarities. A second layer augments the memory comparison with target-consistency features and uses multiple learned attention heads to retrieve complementary comparable sets. A final mixture-of-experts module combines learned gating, residual correction, entropy regularization, and head-diversity regularization to produce the prediction.

This document provides in-depth details for the derivation of the univariate splitting algorithm developed in arXiv:2606.01530. The algorithm approximates the standard, 1-D Gaussian distribution with a mixture of uniformly spaced homoscedastic Gaussian components. The solution is found by minimising the squared $L^2$ norm of the mismatch between the approximation and the original Gaussian. This text presents asymptotic analyses of the proposed splitting in the limit of small step $h$ between the mixand means and in the limit of large number of mixands $M$.

A common heuristic used to explain the generalization of first-order gradient methods on non-convex neural networks is that "flat interpolators generalize well" (Hochreiter and Schmidhuber, 1994; Keskar et al., 2017), where flatness can be measured by the trace of the Hessian of the empirical loss. However, Dinh et al. 2017) showed that, using symmetry of the network that can change flatness while keeping the population and empirical losses unchanged, any interpolator can be made sharper or flatter. This result makes the earlier heuristic statement vacuous. In this paper, we show that for learning an unknown multi-index model with $2$-layer non-convex homogeneous neural networks, there is a connection between flatness and generalization, despite the existence of symmetries. This connection pertains to the "flattest" interpolators, i.e., the interpolators that have orderwise minimum flatness among all interpolators. First, we show that there exists a natural class of non-generalizing interpolators whose flatness cannot be made closer to the flattest possible, even using symmetries. Second, we show that for data generated by a sum of single-index models, if the approximation error and label noise are low, any flattest interpolator achieves small population loss, i.e., the flattest interpolators always generalize. This establishes a direct link between flatness and generalization which applies to a large class of activations and realistic data distributions.

We show every multi-group learner in the leave-one-out transductive setting may incur a surprisingly large multiplicative penalty in its error rate on some group relative to the error rate achievable in the single-group setting. Indeed, the penalty can increase linearly with the number of groups, up to roughly the square-root of the sample size. We also show that in the analogous (group-realizable) statistical setting, the penalty is always at most a poly-logarithmic factor in the sample size, independent of the number of groups.

This paper presents saCI, an R package that implements the stochastic approximation method for constructing nonparametric confidence intervals for Pearson's correlation coefficient. The package is based on the algorithm proposed by Garthwaite (1996) and further developed by Xiong & Xu (2016). The implementation provides both the stochastic approximation (SA) method and the bootstrap BCa method for comparison, along with an interactive Shiny application for exploratory analysis. The package has been successfully published on CRAN, demonstrating its compliance with R package standards and reproducibility.

When assessing the causal effect of an exposure on two or more outcomes in an observational study, a linear combination of outcomes may lessen the sensitivity of a test of the global null hypothesis to potential unmeasured biases. While all linear combinations of scored outcomes can be considered using Scheffe projections or constrained variants thereof, finding the combination that minimizes sensitivity to unmeasured biases requires corrections for multiple testing, which can erode power, especially when many outcomes are of interest. To mitigate this issue, we propose splitting the sample into a planning sample to identify an optimal linear combination and an analysis sample to conduct inference. We provide a novel characterization of the set of linear combinations for which this approach is guaranteed to achieve the same asymptotic power as full-sample alternatives and conduct extensive simulation studies that demonstrate enhanced power in finite samples. Finally, we apply our method to investigate the effects of poverty on the emergence of cardiovascular disease risk factors in children and adolescents. We discover adverse consequences on outcomes related to body composition, physical activity, and tobacco exposure. Although the impact of poverty on elevated tobacco exposure shows some robustness to unmeasured confounding, the other findings remain sensitive to potential biases.

The deep neural network is a widely used framework in machine learning that has been widely applied in various fields. However, deep neural networks often involve a large number of parameters and inputs, many of which may be irrelevant to the goal or true output. These parameters and input variables not only increase computational complexity, but also contribute to additional computational cost. One solution to this problem is knockoff methods, which have proven successful in controlling false discovery rates in high-dimensional regression. Building on the knockoff methods and using the regularised neural network, this paper proposes three variable screening methods under the condition of controlling false discovery rates: one layer filter, multiple layers filter, and variable weight aggregation filter. In comparison with existing algorithms, we find that our algorithms show satisfactory performance.

Machine learning's reliance on sensitive data necessitates privacy-preserving techniques like Differentially Private Stochastic Gradient Descent (DPSGD). However, DPSGD suffers from substantial utility degradation and slow convergence due to gradient clipping and noise injection. Prior works have attempted to improve DPSGD from various perspectives; notably, the Differentially Private Selective Update and Release (DPSUR) algorithm has achieved remarkable model utility. However, the privacy accounting in DPSUR overlooks the variation in sampling probability introduced by the selective release mechanism, which compromises the rigor of its privacy guarantees. To address these limitations, we re-evaluate the privacy analysis of the selective release mechanism and propose a novel algorithm: Differentially Private Selective Release based on Clipped Gradients (DPSR-CG). Through a rigorous, newly derived privacy analysis and extensive experiments on multiple datasets (MNIST, CIFAR-10, IMDB, and FMNIST), we demonstrate that our DPSR-CG mechanism maintains strict privacy guarantees while achieving exceptional model performance.

Forecast reconciliation usually starts from a fixed measurement system and asks how forecasts should be projected onto a coherent space. We ask a different question: which additional linear measurements should be forecast and included in the reconciliation system? We propose REGAIN, a reconciliation-gain framework that learns normalized auxiliary directions, forecasts the induced series with a frozen forecasting oracle, and selects directions by their target-weighted loss reduction after augmented generalized least-squares reconciliation. Unlike variance-based components or predictability-based auxiliary selection, REGAIN optimizes the downstream effect of an auxiliary measurement on the final reconciled forecasts. We provide a statistical characterization showing that useful auxiliary directions must provide complementary information about unresolved target uncertainty, rather than merely being easy to forecast. The analysis also clarifies the covariance-risk reduction mechanism, the role of bias changes in realized quadratic risk, and the stability of estimated gain signals. A stagewise learning algorithm with held-out gain screening is developed, together with an optional joint refinement step. Experiments on Beijing PM2.5 and Australian Tourism data show that gain-selected measurements can improve both ordinary multivariate and hierarchical forecasts, especially when they reveal residual uncertainty not captured by the original measurement system.

Differentially private stochastic gradient descent (DP-SGD) injects noise into every updated coordinate, making the injected noise energy scale with the ambient parameter dimension \(d\). We ask when private training can update fewer coordinates without losing the signal needed for optimization. We propose TP-TopK (Two-Phase TopK DP-SGD), a two-phase method for coordinate-sparse private training without public data, in which a private warm-up phase identifies a coordinate support used to guide the main training phase. We give a criterion characterizing when coordinate restriction can be beneficial, show via a nonconvex stationarity bound that under this condition the relevant noise term scales with the active dimension \(k\) rather than the full parameter dimension \(d\), and provide a lower bound on the reliability of warm-up-based coordinate ranking. Experiments on MNIST, FMNIST, and CIFAR-10 show that learned coordinate supports can retain more gradient energy than size-matched random supports, with the largest gains when the active dimension is small and warm-up scores are informative.

This paper introduces a novel perspective on the use of reverse diffusion processes for sampling from unnormalized densities. The central idea is to embed the target density as the marginal at the initial time of a suitably constructed diffusion process evolving over a finite horizon. In contrast to existing approaches, the proposed methodology involves neither time discretization error nor score function estimation, so that Monte Carlo variability is the only source of approximation. A key theoretical result characterizes the Radon-Nikodym derivative of the reverse diffusion transition distribution with respect to that of an Ornstein-Uhlenbeck (OU) process. This representation provides a tractable change-of-measure formulation and serves as the foundation for two distinct classes of Monte Carlo algorithms. The first class approximates the reverse transition distribution via a sequence of pseudo-marginal Metropolis-Hastings MCMC algorithms. The resulting scheme produces an approximate i.i.d. sample from the target distribution and is fully parallelizable, as trajectories can be generated independently. The second class consists of MCMC algorithms targeting the joint law of the whole diffusion path in $[0,T]$, for a suitably chosen horizon $T$. The proposed samplers combine three types of updates. One update simulates the diffusion forward in time according to an OU dynamics, conditional on its initial value. The remaining two update the backward component via Metropolis-type steps: one conditions on the terminal value at time $T$ and the other one does not. In both cases, acceptance probabilities are implemented using Barker-type Bernoulli factory constructions. The proposed methods perform well for targets with multimodality and complex dependence structures, providing a scalable and efficient alternative to the widely used random-walk Metropolis algorithm.

Gauss's method of orbit determination (OD) is one of the most popular, minimal assumption target tracking techniques in astrodynamics, especially for generating an initial state estimate. However, due to Gauss's method's assumption of Keplerian motion (part of the larger two-body problem), this method cannot be applied in a cislunar environment, where three body, non-planar effects dominate. In this work, we showcase a hybrid Particle Gaussian Mixture (H-PGM) filtering method, a purely recursive probabilistic OD framework that relies upon a sequential combination of the Markov Chain Monte Carlo (MCMC) based Particle Gaussian Mixture-II (PGM-II) and Kalman update based Particle Gaussian Mixture-I (PGM-I) filters. This method allows us to fuse probabilistic information with angles-only observations from terrestrial telescopes for short- and long-term cislunar target tracking. This method also allows us to fuse other target a priori information in an effort to reduce target uncertainty in the short term. This hybrid filtering technique is demonstrated for several popular and important cislunar orbit regimes and compared with several homogeneous and hybrid filtering frameworks.

One of the primary challenges in Bayesian inference on the parameters of a diffusion model from discrete observations is the unavailability of an analytical expression for the transition density function between consecutive observation times, which is needed to derive the likelihood function. Extending previous studies that solve Fokker-Planck (FP) type partial differential equations with Normalizing Flows, we propose a new Normalizing Flow architecture to learn the transition density function of the diffusion process between two observation times. We do so by solving in a Neural Galerkin framework the associated FP equation with a Dirac mass as initial condition, over a specified training distribution of the initial datum and the coefficients of the diffusion. We specifically focus on processes whose diffusion matrix vanishes in certain inaccessible boundary regions, such as Stochastic Volatility models that satisfy a Feller condition. The product of the obtained transition densities evaluated along the observed trajectory approximates the likelihood function, thereby enabling cheap posterior sampling via Markov chain Monte Carlo (MCMC). After the offline training phase, inference becomes significantly more efficient, as it avoids the need to solve the FP equation in real time for each parameter proposed by the MCMC sampler or to rely on other likelihood-free methods for Bayesian inference that involve repeated simulation of diffusion bridges.

Best Linear Unbiased Prediction (BLUP) has been a dominant approach in Generalized Linear Mixed Models, spatial models, and Gaussian Process Regression (GPR). In addition to their optimal properties, BLUP procedures quantify prediction uncertainty. However, the general implementation of BLUP goes as follows: (i) assume the probability distribution and covariance function are known and that only the covariance parameter values are unknown; (ii) plug in parameter estimates into BLUP equations to get the Estimated Best Linear Unbiased Prediction (EBLUP) and its variance. In applications, the reality is that the true covariance function for the process is unknown and choosing the wrong covariance model, particularly its smoothness, to estimate parameters yields a quasi-EBLUP whose prediction variance is biased downward. Focusing on a GPR context, in this paper we first demonstrate that the effect of misspecification on the mean squared prediction error (MSPE) of the quasi-EBLUP converges to a positive constant when the working and true measures are non-equivalent, and is smooth in the prediction location. We then propose a new way to estimate the MSPE of the quasi-EBLUP that accounts for covariance function uncertainty. Our new estimator is compared to four other prediction variance estimators. The new prediction variance estimator generally performs better than all other competitors, and the larger the misspecification of the covariance smoothness, the wider the difference among MSPE estimators.

In Bayesian mixture models and other exchangeable-component models, the posterior is invariant under permutation of component labels, creating m! equivalent modes-the label-switching problem. Standard MCMC methods either mix poorly across these modes or rely on post-hoc relabelling that cannot guarantee the sampler has converged. We propose Folded Transport MCMC (FolT-MCMC), which eliminates label switching before sampling by restricting the Markov chain to a fundamental domain-a sorted or reflected subspace containing exactly one representative from each symmetric mode. The proposal is a learned normalising flow whose density is symmetrised over the group orbits, ensuring correct targeting on the reduced space. We show that this construction preserves a computable convergence diagnostic based on the oscillation of the log-density ratio, and that the diagnostic becomes sharper on the fundamental domain whenever the original-space flow under-covers one or more symmetric modes. Experiments on Gaussian mixtures (d=2-20), label-switching targets (up to 24 equivalent modes), a standard Bayesian three-component mixture posterior, and real accelerometer data from a supertall building show improvement ratios of 2x to 145x, with the folded diagnostic stable across dimensions while the unfolded diagnostic collapses.

We study online learning with an additional offline dataset in the stochastic linear bandit setting. Although this problem arises frequently in practice, the offline-to-online tradeoff remains poorly understood in structured environments. We propose a linear bandit algorithm that balances this tradeoff: it relies on offline data during early rounds, and increasingly favors exploration as the horizon grows. We establish regret bounds showing that our method is simultaneously competitive with both purely online and purely offline solutions. In particular, it achieves sublinear regret relative to the optimal action in the number of online interactions, while its regret relative to an offline reference decreases as the number of offline samples grows. Empirical results further demonstrate its effectiveness across various problem parameters.

Modern protein representation models support tasks such as enzyme design and drug discovery, but their reliance on static data such as sequence and native structure limits their ability to capture the conformational dynamics that drive protein function. We investigate whether persistent homology (PH) can provide descriptors shared across diverse proteins that retain global structure, fine-grained conformational variability, and kinetically relevant information without large-scale pretraining. We introduce the masked Flood complex, i.e., an adaptation of a recently proposed simplicial complex construction, that incorporates domain knowledge to emphasize inter-residue structure at low computational cost. We then use it to compute PH on molecular dynamics (MD) sampled structures, vectorize the persistence diagrams into a shared coordinate system, and probe the capacity of these representations in terms of the aforementioned aspects. To assess the amount of kinetic information, we learn low-dimensional embeddings from time-lagged observations and evaluate Markov state models (MSMs) estimated from them. Using these MSMs to guide training of the recent marsfm generative framework improves several ensemble statistics relative to the original model. After finetuning on lower-temperature MD data and adapting the sampling procedure, the resulting model also shows promising transfer to fast folding proteins.

Large language models hallucinate confidently, making uncertainty quantification (UQ) essential for reliable deployment. Existing methods rely predominantly on token-level signals, leaving the geometric structure of intermediate hidden states underused. In this paper, we take the geometric complexity of hidden-state matrices as a measure of the global uncertainty of LLMs, while treating token-level uncertainty estimation as a local metric. We show that hidden-state geometric entropy (global uncertainty) and token-level entropy (local uncertainty) are statistically near-orthogonal, capturing distinct failure regimes for reliability prediction. In particular, global geometry recovers the confident-but-wrong failure mode that local signals systematically miss. Building on this, we propose Global-Local Uncertainty (GLU), an unsupervised, single-pass score that fuses the two signals via a multiplicative gate. Across three model families and six benchmarks, GLU matches or outperforms all unsupervised baselines while requiring only a single forward pass and remaining length-normalized and architecture-agnostic.

Reconstruction-based methods are widely used for time series anomaly detection, where models are trained to reconstruct subsequences, and anomalies are identified through reconstruction errors. However, reported results are often hard to compare due to heterogeneous evaluation practices and underspecified inference procedures. In this paper, we revisit reconstruction-based anomaly detection in the univariate offline setting and study the role of the inference stride, which controls whether subsequences are processed as disjoint windows or with overlap. We propose a unified training, tuning, and multi-seed evaluation protocol on the curated TSB-AD benchmark, and study how overlapping inference affects anomaly detection performance for a range of reconstruction models, including PCA-based baselines, DLinear, an AutoEncoder, TimesNet, and Transformer variants. The results show that across all models, overlapping windows yield consistent improvements, with average relative gain up to +28%, and can alter method rankings. We further analyze variability across datasets, random seeds, and hyperparameter configurations. Finally, we complement the benchmark study with an evaluation on the full UCR archive using localization criteria aligned with sliding-window reconstruction. Overall, our results highlight that reconstruction-based anomaly detection performance depends not only on model architecture and training, but also on inference choices, motivating a clear and reproducible protocol. Our results show that reconstructionbased baselines achieve strong performance on both TSB-AD and UCR benchmarks, supporting them as competitive and practical approaches for univariate time series anomaly detection.

Selecting a clustering algorithm and its hyperparameters without labels is a common difficulty in engineering machine learning pipelines that work with unsupervised analysis of sensor, image, or process data. Clustering validation indices (CVIs) provide internal scores for ranking candidate clusterings, but most popular CVIs are built from Euclidean compactness and separation terms and so tend to favour compact, convex partitions. Their performance is known to degrade on non convex, irregular, or variable density data, where kernel transformations or alternative distance measures are typically used at the cost of additional tuning and computation. This paper introduces the Central Description Length (CDL) clustering validation index. CDL uses the observed within cluster compactness, the estimated cluster centers, and the estimated cluster covariances to compute a probabilistic upper bound on the description length associated with the unobservable true cluster centers. The bound condenses intra cluster compactness and centroid displacement into a single computable quantity and is evaluated on the partition produced by any clustering algorithm. The implementation uses only observable quantities (the data, the partition, the estimated centers, and the estimated covariances) and does not use ground truth labels. On synthetic benchmarks with non convex and arbitrary shape clusters, CDL-CVI selected the reference number of clusters more often and reached higher Adjusted Rand Index (ARI) values than the conventional CVIs we tested, without an additional kernel preprocessing stage. On image benchmarks (MNIST, CIFAR-10, STL-10) clustered from frozen unsupervised embeddings, CDL-CVI returned cluster numbers close to the reference class counts across K-means, DBSCAN, and spectral clustering in the reported trials.

We quantify the value of structural knowledge, restrictions a modeler places on the world before seeing data. Our analytic workhorse is the local sensitivity of an estimand to distributional perturbations in the Otto-Wasserstein geometry: the largest first-order change in the estimand per unit displacement of probability mass (transport), equal to the dual norm of the spatial gradient of the efficient influence function. The modeler encodes structural knowledge by restricting the class of transports, and the resulting reduction in sensitivity is the value of their inductive bias. We illustrate by shedding light on a longstanding puzzle in causal inference where classical semiparametric efficiency bounds remain identical regardless of whether the propensity score is known, despite observed practical difficulties when it is unknown. Our approach characterizes how known propensities significantly reduce sensitivity to misspecification.

The estimation of concentration parameter in Fisher-von Mises-Langevin distribution is the directional statistics analogue of the estimation of the precision matrix for the Gaussian distribution. In this work we show that unbiased estimation of this parameter is impossible. With this realization in hand, we provide an alternative parameterization of the Fisher-von Mises-Langevin distribution in terms of the squared concentration, which we term the intensity. We fruther show that unbiased estimation of thereof is possible, and provide (almost) unbiased estimators thereof in terms of a partial sum U-statistic. We showcase our new estimator on synthetic data, New York taxi trip data, and on spherical word embeddings.

Multilevel regression and poststratification (MrP) has become a workhorse method for estimating population quantities from non-probability surveys, and is the primary model-based alternative to traditional survey calibration weighting methods, such as raking. For simple linear regression models, MrP methods admit “equivalent weights”, allowing for direct comparisons between MrP and traditional calibration weighting. Such weights, however, have been unavailable for the most widely used MrP models, such as logistic regression. In this paper, we develop a natural generalization, “MrP locally equivalent weights” (MrPlew), which represent MrP as a weighting-style estimator that is locally equivalent to calibration weights near the observed responses. This enables a suite of standard weighting diagnostics, including frequentist sampling variability, covariate balance, and subgroup contribution. We formally justify the use of MrPlew in these cases: we prove the MrPlew-based variance estimator is asymptotically equivalent to the infinitesimal jackknife for common exponential family models, and we introduce a novel class of model checks based on invariance to data perturbations that generalize covariate balance and subgroup contribution to nonlinear models. We further show that MrPlew can be computed easily using existing MCMC samples and provide open-source software to compute MrPlew using the output of standard software. We illustrate our approach for several canonical studies that use MrP, including via a logistic regression outcome model, showing that implied covariate balance can sometimes be worse for MrP than for raking. Given the ease of computing, we recommend making MrPlew a standard part of the MrP model interrogation workflow.