Papers

Population-level policies are introduced at a fixed time and evaluated from a single series of aggregate outcomes, and the interrupted time series design estimates the total shift in an outcome after the intervention. When the policy is expected to act through a measurable pathway, the total effect is less informative than its decomposition into direct and indirect effects. We formulate causal mediation for a single interrupted time series and study stabilized mediator weighting as the estimator of the natural direct and indirect effects. Because the intervention is a deterministic function, the exposure weight equals one and the exposure contrast is identified through the segmented-regression level shift, so only the mediator pathway is weighted. We add a cumulative mediator weight that carries the lagged confounder history, incorporate a concurrent event as a second interruption, and replace variance formulas that treat the estimated weights as fixed with a block-residual bootstrap that keeps the deterministic exposure timing intact and resamples the mediator and outcome residuals in moving blocks. In a simulation calibrated to daily data, the unweighted product-of-coefficients estimator is biased under mediator-outcome confounding, with an indirect-effect bias near $0.19$ and coverage of $0.003$, whereas stabilized weighting reduces the bias to about $0.03$ and improves indirect-effect coverage from near zero to about $0.83$. Applied to the 2019 termination of Ontario's Drive Clean vehicle emissions testing program, the method estimates a direct reduction in ground-level ozone of $2.113$ parts per billion (95% interval $-3.384$ to $-0.841$) that is robust across four Toronto regions, a positive but heterogeneous indirect effect through nitrogen dioxide, and a total effect near the boundary of significance; a pre-pandemic sensitivity analysis agrees with the primary results.

We develop a semiparametric kinematic state-space framework for estimating the latent dynamics of global warming from noisy monthly temperature observations. The framework estimates the warming level, warming rate, and warming acceleration jointly, while incorporating covariates, measurement error, serial dependence, and time-varying error variance directly in the model. Applying the framework to five major global temperature records, and comparing it with nonparametric kernel and parametric benchmark estimators, we show that the inferred recent warming dynamics depend crucially on the parametric restrictions imposed on the warming path. The current underlying warming level is robustly estimated across datasets and methods, with estimates close to \(1.40\,^{\circ}\mathrm{C}\) above pre-industrial levels. Estimates of the current warming rate differ materially across estimator classes: flexible local estimators imply substantially higher current rates than the parametric benchmarks. Our preferred state-space specification estimates the current warming rate at approximately \(0.47\,^{\circ}\mathrm{C} \, \textnormal{decade}^{-1}\). Endpoint acceleration estimates from the flexible estimators are positive across all five records, although their statistical significance varies across records and methods. Average acceleration since 2015 is significantly positive across all five records in the preferred state-space model, with a cross-dataset mean of approximately \(0.19\,^{\circ}\mathrm{C} \, \textnormal{decade}^{-2}\). Our results indicate that global warming is proceeding faster, and has accelerated more strongly in the recent period, than conventional estimates suggest.

An increasing number of scholars use AI to measure variables they subsequently include in downstream analyses. Although AI-measured variables are often analyzed as if observed without error, ignoring prediction errors in automated measurement leads to substantial bias and invalid confidence intervals in downstream analyses, even if AI measurement accuracy is high, e.g., above 90%. Existing solutions, such as design-based supervised learning and prediction-powered inference, combine error-prone AI-based measurements with gold-standard labels, which may be costly and difficult to obtain in some application areas. In this paper, we propose debiased inference with multiple imperfect measurements (DMM), a framework that combines multiple error-prone AI measurements to enable valid downstream inference without gold-standard labels. Building on the established results on CP decomposition, DMM assumes that these measurements are independent conditional on the latent true label and observed unit-level features, such as text features represented by embeddings. This framework allows for unknown misclassification rates to vary across annotation methods (e.g., large language models) and across units of annotation (e.g., texts). Under this assumption, we use semiparametric inference theory to prove that the DMM estimator is consistent and asymptotically normal, enabling valid inference for a wide range of downstream statistical analyses common in the social sciences. Our simulation results show that DMM yields valid inference and that adding accurate, though imperfect, measurements can improve efficiency. Focusing on common applications of large language model annotations, we also develop diagnostics to assess the conditional independence assumption.

For a centered, variance-one random variable $X$ with finite fourth moment, let $x$ be the vector of square-free degree-$d$ monomials in $n$ independent copies of $X$. At the critical scale $d^2/n \to \lambda \in (0,\infty)$, the normalized squared length of $x$ converges to the lognormal variable $R = \exp(\sqrt{\lambda v}\, Z - \lambda v/2)$, where $v = \mathbb{E} X^4 - 1$ and $Z$ is standard normal. If $\binom{n}{d}/N \to c \in (0,\infty)$, the sample covariance of $N$ independent copies of $x$ has an almost-sure limiting spectral law: the free compound-Poisson law with rate $1/c$ and jump distribution $\operatorname{Law}(cR)$. It reduces to Marchenko-Pastur when $\lambda v = 0$. The proof shows that subtracting the contribution of the sample length leaves vanishing quadratic-form fluctuations, even when $\mathbb{E} X^3 \neq 0$; length and direction need not be independent.

We consider the family ${\mathcal{C}}_d^{\Pi_{d-1}}$ of all $d$-dimensional copulas whose $(d-1)$-dimensional marginals are all equal to the $(d-1)$-dimensional product copula $\Pi_{d-1}$ and tackle the natural question, `how far away' from the $d$-dimensional product copula $\Pi_d$ elements in ${\mathcal{C}}_d^{\Pi_{d-1}}$ can be. We provide definitive answers for both, the uniform metric $d_\infty$ as well as the stronger, conditioning-based metric $D_1$. The established results clearly indicate that the family ${\mathcal{C}}_d^{\Pi_{d-1}}$ is larger than one might expect.

Traffic signal control is a challenging sequential decision-making problem requiring a balance between throughput, vehicle delay fairness, signal stability, predictable switching, and emergency vehicle prioritisation. Existing reinforcement learning (RL) approaches often rely on predefined objectives and offer limited adaptability to changing operational priorities. They may also generalise poorly across geometrically equivalent intersections. To address these limitations, SIGMA (Symmetry-aware, Intelligent, Geometric, Multi-faced Adaptive traffic control) is proposed as a multi-faced RL framework augmented with a large language model (LLM) for dynamic objective adaptation and orientation-invariant policy learning. Natural language emergency instructions are translated into priority vectors that guide a multi-faced actor-critic controller. Symmetry-aware rotational augmentation supports transfer across homogeneous four-way intersections, while an offline-to-online learning strategy enables stable initialisation and adaptation to evolving traffic conditions. Structural properties concerning emergency service levels, graceful degradation under LLM failure, and sensitivity to traffic demand perturbations are established, alongside bootstrap-based statistical validation. Experiments in SUMO across four urban intersections modelled on Kolkata traffic scenarios compare SIGMA with fixed-time, actuated, and Deep Q-Network controllers. Reductions in average waiting times, emergency waiting times, and queue lengths are observed, alongside improved traffic throughput. Empirical reliability and ablation analyses further assess robustness to component failures and geometric orientation. SIGMA integrates language-guided objective adaptation, multi-faced policy learning, and statistical reliability assessment into an adaptive framework for intelligent traffic control.

The spacing of data contains information about the underlying modality. Sections of consistent, similar spacing correspond to modes while it increases between them. We have defined parametric, runs-based non-parametric, and data driven tests to evaluate these features — flats and peaks — and determine the presence and location of multiple modes. This report will check the tests in two situations. By varying a bi-modal setup we can control the changes in spacing and determine the resolution and sensitivity of the tests. By applying them to the large number of test cases that exist in the literature from other modality studies we check the stability and consistency of the results. We will also evaluate the accuracy of the null-distribution models of the features. The results show that the spacing does reflect the data's modality, that the tests do screen marginal cases, and where the analysis begins to break down.

Estimating the difference of two Stein's score functions is a fundamental problem in generative modeling. In particular, score differences arise naturally in transfer learning, where the score difference provides the mechanism for adapting a pre-trained model to a new target distribution, and in diffusion model-based post-training methods such as discriminator guidance. Existing estimators for score differences in these settings either lack of statistical consistency or are difficult to scale up in high-dimensions. We propose a statistically consistent and scalable estimator for score differences based on Sobolev regularization, which plays a crucial role in ensuring consistency and stablizing the training in the small-sample regime. Mathematically, we establish a convergence rate of $O(n^{-\frac{s-1}{d+2s-2}})$ where $d$ is the dimension and $s$ denotes the smoothness of the underlying densities, and provide a minimax lower bound of $\tilde{\Omega}(n^{-\frac{2(s-1)}{d+2s}})$ (in mean-squared error). Empirically, our estimator exhibits significantly improved stability in small-sample regimes compared to existing methods. We demonstrate its effectiveness on real-world tasks, including transfer learning for ECG signal generation, where it substantially outperforms non-regularized score difference estimators in downstream classification performance.

The spacing of a unimodal variate resembles a `U' with a flat bottom and rapidly increasing values in the tails. When combined into multi-modal setups, the transition between variates into their tails forms local increases in the spacing. These peak features signal the data is multi-modal while the flats locate the modes. We develop tests of the features, including parametric models based on an assumed null univariate distribution, the number of runs or longest run in the signed difference of the interval spacing, and feature reconstruction by means of permutations of the runs or bootstrap samples from a pool based on the difference of the signal. We also try combining existing changepoint detectors to look where the behavior of the spacing changes. This paper describes the processing of the spacing and models and tests in full, gives examples of their use, and summarizes the performance in general.

Selecting a small, diverse subset from a large candidate pool often means balancing several incompatible notions of diversity. In trademark curation, for instance, a subset should cover both the language used to describe marks and the visual space of their logos. A single determinantal point process (\DPP) kernel can hide failure in one view, and averaging kernels replaces the multi-view relaxation by an ordinary single-kernel spectral problem. We formulate fair multi-view determinant selection: maximize the weakest per-view log determinant of a size-$k$ subset. We smooth this nonsmooth objective and relax it to the Stiefel manifold. The relaxation embeds every discrete subset exactly, but unlike its single-view counterpart it has no closed-form spectral solution in general. Its stationarity condition is a gauge-invariant nonlinear eigenvalue problem with eigenvector-dependent, view-adaptive weights. We derive an adaptive self-consistent-field (\SCF) solver with damping and level shifting, and round the resulting subspace by leverage-score screening followed by fair local refinement. The solver needs only feature-map products for each view. We report conflicting-view synthetic experiments and specify a multimodal USPTO protocol; the real-data multimodal results require aligned logo embeddings and are not claimed in this version.

We give a two-player zero-sum repeated game between a learner and nature whose value identity generates Bayesian updating and an exact accounting of exponential-weights regret at once, and supplies the comparator-class variational form that a wide class of concentration phenomena share. The terminal payoff is the most a comparator can gain at fixed relative entropy from the prior, and the one-step constraint is an information budget on nature's move under the learner's mixed action. With the learner's move otherwise unrestricted, Gibbs/Bayes weights emerge as its unique Bellman equalizer – the mixed action that makes the per-round loss independent of which direction nature moves – with log-partition functions playing the role of value functions. The regret decomposes exactly into three parts: a per-round information loss reflecting the variation in observed outcomes, an additive retempering drift that accounts exactly for any change of measurement scale between rounds, and the information the comparator carries relative to the prior. The variance and bounded-range proxies that drive standard regret bounds are looser relaxations of this decomposition, which holds generally and governs them all. Both players' strategies are read off from the decomposition term by term, and repeated play yields an information-theoretic ledger of self-play in place of the usual quadratic-variation surrogate. The same comparator-class geometry accounts for the classical large-deviation bounds, and methods across bandits, posterior sampling, aggregation, and boosting are specializations of the one regret decomposition.

We study how fast experimental designs can approach the semiparametric efficiency bound in finite samples, as measured by the excess variance of unadjusted treatment effect estimation. We prove an impossibility theorem: under weak conditions, no design can approach the variance bound uniformly over smooth outcome models unless covariate dimension $d \ll \log n$. Even in experiments with thousands of units, this permits only a handful of covariates. Motivated by this, we propose new designs based on discrepancy minimization that instead attempt to control imbalances over restricted-complexity nonparametric function classes. Such designs achieve fast rates to their corresponding restricted efficiency targets, permitting $d \ll n$ covariates in an additive nonparametric specification. They can also be combined with matching to protect against unmodeled outcome variation. In simulations calibrated to 12 published experiments, our designs reduce variance relative to matched pairs randomization in every empirical setting.

Categorising invoices into the correct General Ledger (GL) code underpins financial reporting and tax compliance. This is a skilled accounting judgement rather than a routine task: the correct category depends subtly on the nature of the purchasing business, the vendor and the invoice text. Whilst AI is increasingly being adopted across industries to automate tasks, including invoice categorisation, implementations built on in-house small language models (SLMs) can simultaneously reduce cost and improve data security, confidentiality, and interpretability. We investigate this approach by first analysing the pre-trained embedding geometry of a small sentence transformer (SBERT) and classic SLM (DeBERTa). The sentence-embedding space of this financial corpus is globally anisotropic but composed of locally isotropic clusters, extending prior token-level findings to sentence embeddings in a financial setting, and these clusters are strongly correlated with the vendor identity. SBERT fine-tuned on a single GPU reaches 0.96 accuracy on invoice classification, above both a zero-shot LLM and a vendor identity baseline, increasing performance for smaller, challenging categories and new clients. For this important generalisation problem, SBERT reaches 0.9 F1 with roughly 100 client-specific invoices, showing that an in-house SLM implementation is promising. Combining these results with geometric analysis shows that pre-trained embedding geometry is associated with classification performance and reveals a counterintuitive finding that a structured input that would help a human reader does not improve the SLM performance.

I develop a reproducible framework for evaluating individual batting and bowling performances in Twenty20 (T20) cricket on one interpretable scale of runs above expectation, built from two ball-level primitives. The first, Runs Above Expected (RAE), is the residual between the runs scored on a delivery and a contextual expectation of what an average performer would produce in the same situation. That expectation is a multiplicative log-linear model of the cohort scoring rate, whose per-cell estimator is shown to be a conditional Poisson maximum likelihood multiplier. It is fitted by iterated backfitting and stabilised by empirical Bayes shrinkage, so that thinly sampled contexts are pooled towards the population. A single opposition symmetry places run-scoring and run-prevention on the same footing. The second primitive, Dismissal Adjusted Runs (DAR), prices a dismissal in that currency as the runs it forgoes, read off a batting side value function solved by dynamic programming through a Bellman expectation recursion, under observed play. Because dismissal is the expected end of every innings, the realised wicket cost (realDAR) is centered against its expectation (xDAR) under a league dismissal hazard rate. The centered quantity is a run-weighted mean zero martingale residual of the dismissal process, so a player is charged only for departing from average behaviour. The two primitives sum to a symmetric Impact, which makes batting and bowling comparable in centre as well as in unit. Estimated on over 2.7 million legal deliveries of men's T20 cricket, the framework recovers known contextual structure, agrees with the conventional rates it refines while correcting their context-blindness, and yields face-valid player, innings and season leaderboards for the Indian Premier League.

We introduce Heteroskedastic Normalized Vecchia Gaussian Processes (HetNV), a scalable framework for Gaussian process regression with input-dependent observation noise. HetNV combines Vecchia likelihood approximations on normalized inputs with residual-based nonparametric variance estimation. The latent mean is estimated via a Vecchia GP with observation-specific nugget variances, while the log noise variance is obtained by smoothing stabilized log-squared residual pseudo-responses that account for kriging uncertainty and current nugget estimates, using LOESS in one dimension and thin plate spline generalized additive models in two dimensions. The method alternates between mean and variance updates, avoiding latent-variable inference for the variance process. For fixed neighborhood size ($m$) and number of observations ($n$), the dominant per-iteration cost is the Vecchia update, scaling as $O(nm^2)$. Simulation studies show improved recovery of input-dependent uncertainty relative to homoskedastic Vecchia models while maintaining competitive mean prediction accuracy, with additional gains in two-dimensional mean estimation. An application to spacecraft plasma measurements demonstrates how locally adaptive uncertainty estimates influence downstream signal-detection decisions in large, noisy, heteroskedastic settings.

Randomized trials provide internally valid treatment-effect evidence, but trial participants may not represent the target population. In contrast, observational studies are often closer to the target population, but their treatment assignment may be affected by possible hidden confounding. We develop a robust posterior-drift framework for estimating the average treatment effect in an observational target population when exact conditional-effect transportability may fail. The framework represents observational conditional potential-outcome regressions as their randomized-trial counterparts plus source-specific drifts. The randomized trial serves as an internally valid anchor, while the observational study supplies the target covariate distribution and partial information about the target causal contrast. To account for possible hidden confounding, we consider a Rosenbaum-type uncertainty set induced by a sensitivity parameter on the generalized propensity score and estimate the drift through a minimax worst-case risk criterion. We derive efficiency results in auxiliary regimes, establish uniform concentration and near-optimality guarantees for the minimax estimator, and handle general parametric and smooth nonparametric drift classes. Simulations and an ACTG 175–WIHS application show that the proposed analysis yields more cautious and interpretable target-population effect estimates than exact-transportability analyses.

Uncertainty visualisation is important for data transparency, especially for map visualisations where data is often aggregated. Despite the importance of this area, studies evaluating uncertainty visualisation lack consensus and produce conflicting results. This work introduces a new evaluation approach for uncertainty visualisation that attempts to assess uncertainty as noise, rather than signal. We evaluate five methods of visualising uncertainty: standard choropleth maps, value/variance bivariate maps, value-suppressing uncertainty palettes, overlaid sampling, and pixelated sampling maps. Built on principles of implicit testing, we put an 'uncertainty visualisation' spin on the classic Ishihara colourblind test to create a novel test that is able to evaluate uncertainty as noise. We compare signal visibility to conventional hypothesis tests at various levels of group separation. By building our experimental design on top of established graphics theory, we isolate the plot components that facilitate successful signal suppression and establish foundational theory for the perception of uncertainty visualisation.

We develop a general class of non-reversible Hamiltonian Monte Carlo dynamics on discrete state spaces. The method augments the discrete state with a continuous momentum variable and does not require a continuous embedding of the discrete state space. We establish conditions for invariance of the target distribution and study the resulting processes in terms of ergodicity, exponential contractivity, asymptotic variance and relaxation time. We then derive scaling limits on increasingly fine lattices and high-dimensional hypercubes. In both settings, suitable rescalings converge to Hamiltonian dynamics in continuous space, revealing a diffusive-to-ballistic speed-up over reversible samplers, even for heterogeneous target distributions where standard non-reversible methods become diffusive. The proposed dynamics can be simulated exactly in continuous time, given access to the target distribution at all neighbours of the current state. To reduce computational cost, we provide approximation schemes based on splitting, $\tau$-leaping and gradient approximations. We illustrate the proposed framework numerically on examples involving high-dimensional hypercubes, mixture models and permutations.

Let $W_N$ be an exact length-$N$ block from a strictly positive i.i.d. source $\mathbf p$ on a fixed finite alphabet. We prove that the canonical T-complexity $c_T$ satisfies \[ \frac{c_T(W_N)}{e^{-\gamma}h(\mathbf p) N/\log N}\longrightarrow1 \] in probability and in $L^r$ for every fixed $1\le r<\infty$, where $h(\mathbf p)$ is the source entropy in nats and $\gamma$ is the Euler-Mascheroni constant. The proof combines an exact length budget for canonical recovery, a critical-scale $E_1$ estimate for an ideal backward chain, and an exact finite-block boundary representation. An exact Doob-transform identity expresses the finite-boundary law relative to the ideal law conditioned at each step to avoid the current history-dependent successor codeword. A history-uniform renewal estimate then makes the telescoping endpoint density uniformly asymptotic to one, so no one-step approximation errors accumulate.

For an AR($1$) sequence with coefficient $0 < a < 1$, the probability of staying above zero decays exponentially in the number of steps; at the unit root $a=1$ it decays polynomially, with order $n^{-1/2}$ for symmetric increments. We study this transition for AR($1$) sequences driven by symmetric and asymmetric $\alpha$-stable innovations. We write $\Lambda(a,\alpha)$ for the symmetric exponential persistence rate and $\Lambda(a,\alpha,\rho)$ for the rate of a strictly stable law with positivity parameter $\rho$. The entire chain admits an exact representation through a single stable Lévy process observed on a geometrically expanding time grid. In the symmetric case, comparison with continuous half-line survival gives $\Lambda(a,\alpha) \leq \frac{\alpha}{2}\log{(1/a)}$. For $0 < \alpha < 2$, this bound disproves the stable specialization of a conjecture of Hinrichs, Kolb and Wachtel concerning regularly varying innovation tails. Combining stable closure under subsampling with a monotonicity coupling yields a lower bound of the same near-unit order. This proves $\Lambda(a,\alpha) \asymp \log{(1/a)}$ as $a \uparrow 1$ and shows that $\Lambda(a,\alpha)/\log{(1/a)}$ converges to a limit in $(0,\alpha/2]$ equal to the supremum of that ratio over $0 < a < 1$. A Lamperti transformation reduces identification of this constant to a dense-sampling problem for a stationary stable Ornstein-Uhlenbeck process. Existing Gaussian theory determines the sharp value at $\alpha=2$; for $0 < \alpha < 2$, rescued crossings between observations remain the obstacle. For asymmetric strictly stable innovations with $\alpha \neq 1$ and admissible $\rho\in(0,1)$, the corresponding bounds are $\Lambda(a,\alpha,\rho) \leq \alpha(1-\rho)\log{(1/a)}$ and $\Lambda(a,\alpha,\rho) \asymp \log{(1/a)}$. At $\alpha=2$, only the symmetric case $\rho=1/2$ occurs.

This short work describes an extension of the permanental process model which includes fixed effects. By starting with a prior on the fixed effects coefficients we show that, in the diffuse prior limit, the intensity function of the permanental process can be found using the representer theorem and naturally decomposed into a fixed effects term and a function which is an element of a Reproducing Kernel Hilbert Space (RKHS). We show that the limiting equivalent kernel defines an RKHS whose squared norm is exactly the limiting penalty. This allows for straightforward scientific interpretation of permanental process models and the easy incorporation of domain knowledge into the estimation process.

Urban rail transit reliability is a critical measure of system performance, yet its causal determinants remain poorly quantified due to high-dimensional and interdependent influencing factors. This study investigates reliability patterns across 46 international metro operators between 1994 and 2024 using the CoMET benchmarking database, incorporating more than 90 candidate variables spanning technical, operational, financial, environmental, and macroeconomic conditions. Based on domain knowledge, literature synthesis, and variable construction, four operational determinants are designed to capture three mechanisms: demand pressure, service supply, and demand-supply imbalance, while the remaining variables are screened and incorporated as confounders where theoretically appropriate. Double/Debiased Machine Learning (DML) adapted for panel data is introduced to urban rail reliability analysis to quantify the net causal effects of these determinants under complex and nonlinear relationships. The framework combines flexible machine learning with panel fixed or random effects within-operator temporal variation, reducing bias from high-dimensional confounding, model misspecification, and unobserved operator heterogeneity. The results identify three distinct operational mechanisms. Higher passenger demand intensity increases incident rates by 0.38% (p<0.001). On the supply side, greater fleet supply adequacy and car-based operational intensity reduce incident rates by 0.52% (p<0.05) and 0.80% (p<0.01), respectively. Capacity utilization, which reflects the imbalance between demand and available supply, increases incident rates by 0.49% (p<0.001). These findings show that metro reliability depends not only on the level of demand or supply alone, but also on whether service provision keeps pace with passenger demand.

Compositional predictors, such as microbiome abundances, pose unique challenges in variable selection due to their unit-sum constraint and inherent dependencies. Existing approaches often rely on fixed association graphs derived from phylogenetic or ecological distances, which may not reflect outcome-relevant relationships. We propose GRACE (GRaph-Adaptive horseshoe for Compositional rEgression), a fully Bayesian framework that enforces compositional constraints, performs variable selection, and adaptively learns an outcome-driven shrinkage graph. GRACE achieves compositionality through a novel linear reparameterization of regression coefficients, while a structured horseshoe prior induces sparsity and smooths coefficients along the learned graph. Graph learning is accomplished via scaled beta2 priors on edge weights, providing both outcome-specific adaptation and posterior uncertainty quantification. We develop an efficient Gibbs sampler incorporating elliptical slice sampling to ensure scalability in high dimensions. Through extensive simulations, GRACE demonstrates competitive predictive accuracy and improved graph recovery compared with existing methods, particularly under graph misspecification. Application to oral microbiome data from the ORIGINS study identifies taxa associated with insulin resistance and yields an outcome-driven graph summarizing how those taxa relate to the outcome, a structure that differs substantially from phylogenetic or co-occurrence networks. These findings highlight that fixed predictor graphs useful for regularization may not faithfully represent outcome-relevant feature relationships, underscoring the need for adaptive, outcome-informed approaches in compositional regression.

Multi-armed bandit algorithms are evaluated by regret, yet comparable regret can coexist with different allocations across independent runs. We study the trade-off between worst-case regret $\mathcal{R}_{K,T}$ and instability $\mathcal S_{K,T}$, defined as the largest standard deviation of a terminal pull count, for $K$ arms and $T$ rounds. We prove the finite-time lower bound $\mathcal R_{K,T}\mathcal S_{K,T}\ge C T^{3/2}$, where $C$ is independent of $K$ and $T$, under a finite-time regret condition and without the regularity assumptions imposed in the prior asymptotic analysis. We also introduce Stabilized Lower-Envelope UCB (\textup{SLE-UCB}), a new tunable algorithm combining a running lower-envelope index with a decreasing pull-count stabilizer. \textup{SLE-UCB} satisfies $\mathcal R_{K,T}\mathcal S_{K,T}=O(T^{3/2}\log K)$, with an implicit constant independent of $K$ and $T$, matching the lower bound exactly in $T$ and within a logarithmic factor in $K$. To prove the instability bound, we develop a new offline top-prefix representation that removes path dependence from online decisions. Together with single-reward perturbations and the Efron–Stein inequality, this representation controls pull-count variance. Thus, regret and instability depend reciprocally on $K$, while their product has no polynomial dependence on $K$. These results resolve the open question raised in the literature concerning the sharp arm-dependent regret–instability frontier.

Accurate building height information at the individual footprint scale is essential for material stock accounting and post-disaster damage assessments yet remains difficult to obtain at city scale in the Global South where airborne LiDAR coverage is rare and commercial very high-resolution imagery is cost-prohibitive or unavailable. While recent works have demonstrated building height estimation using freely available Sentinel imagery, the resolution ceiling of resulting products is still coarse for material stock analysis. This study incorporates products derived from data freely accessible under scientific research licenses, TerraSAR-X StripMap and PlanetScope, alongside Sentinel-1 to predict building heights in a large city in Brazil. To account for the spatial autocorrelation in the training set, features from all sources are integrated in a geographically weighted random forest model, returning an RMSE of 5.34 m and R2 of 0.756 against a LiDAR reference dataset. Local feature importance showed predictor dominance to vary consistently across intra-urban contexts, with footprint geometry dominating for low-rise buildings, shadow-derived height for taller and more isolated structures, and spectral reflectance for the tallest buildings in the set. Sentinel-1 backscatter and InSAR occupy complementary spatial niches, with no single sensor uniformly preferable across the set. Results provide optioneering guidance and insight over satellite-derived products predictive relevance in distinct contexts, which global machine learning or neural network models cannot offer.

We develop blinded sample size re-estimation strategies for McNemar's test based on paired binary primary and secondary short-term surrogate endpoints. The development is motivated by a prospective randomized clinical trial on childhood glaucoma. A conditional power expression for McNemar's test given the primary endpoint at an interim analysis is derived and complemented by a sample size re-estimation rule. We show that this procedure preserves the type I error rate while allowing the second-stage sample size to be chosen to attain a prespecified target power. In the case where for some patients only a short-term surrogate endpoint is available at interim, we introduce a surrogate-based re-estimation approach that conditions on all possible numbers of primary-endpoint discordant pairs using transition rates from the surrogate to the primary outcome. We show how these transition rates can be estimated from data on a subsample for which both surrogate and primary endpoint are available. We derive the resulting surrogate endpoint-based conditional power and sample size rule and illustrate their use with the example of the motivating trial.

Take three frontier mixture-of-experts models (Alibaba, OpenAI, NVIDIA; 3.6-4.0B active parameters each) and fine-tune them to reason in a low-resource language. On accuracy benchmarks almost nothing happens, and the benchmark itself is noise at this scale: changing only the random seed moves the score by 7.7 points, more than every data and recipe effect we measured. That null is our first result. The real changes live where accuracy cannot see. Base models never think in Greek: 0 of 1,000 reasoning traces, even when the question is Greek, so the model answers correctly while reasoning in a form its user cannot read, audit, or correct. After supervised fine-tuning (SFT), every released checkpoint reasons in the language of the question on ~98% of items, one family at 3x fewer tokens, with judged grammaticality improving on all four models and general ability within a few points of each base: nothing was forgotten, and fluency was gained. We propose six behavioural dimensions that make such changes measurable, each gated to reject any metric that correlates with output length, and we report how our own instruments lied: six failures, each caught by a control. What SFT cannot do is fix its own defects: a quarter of answers skip the requested format, answers leak into the reasoning channel, and an explicit "think in English" is obeyed under half the time. Reinforcement learning with verifiable rewards, pre-registered before training, fixes the first two outright (fallback 24% to 2.5%, leak 3.5% to 0.0%, both against a flat random-reward control) and moves the third (+9.1pp), while the Greek reasoning habit survives an accuracy-only gradient untouched. We release five checkpoints. The instruments, the controls and the pre-registration travel to any low-resource language; Greek is the case that let us measure them.

Modern machine learning is leading to substantial gains in precision, flexibility, and computational efficiency in fundamental physics. Statistical validation, uncertainty quantification, and robustness assessment are less systematically addressed. The VERaiPHY initiative (Validation & Evaluation for Robust AI in PHYsics) is a series of articles developed within the PHYSTAT programme, aimed at establishing statistical standards for the development, evaluation, and deployment of ML techniques. Each article focuses on a specific methodological domain from a statistics perspective and clarifies statistical questions, tests, and the interpretation of results. This opening article establishes the probabilistic, statistical, and machine learning foundations that the later contributions assume, together with the notation used throughout.

Spectral imaging provides chemically specific and spatially resolved analysis of microplastics, but its routine application is hindered by large data volumes, acquisition artefacts, spectral variability, and misidentification of polymers due to alike spectra. This study proposes a multi-level preprocessing and modelling framework for FT-IR spectral imaging of microplastics that integrates image-level, tile-level, and spectral-level corrections with scalable identification strategies. Image-level variation associated with changing acquisition conditions was done with latent variable selection, while a background-based tile correction reduced illumination-related artefacts. Spectral preprocessing combined baseline correction, smoothing, derivative calculation, normalization, and wavelength selection, and only particle spectra were retained for further analysis to improve computational efficiency. For scalable identification, clustering was applied to particle spectra and spectral library matching was performed on cluster centroids instead of individual pixels. Among twelve evaluated matching strategies, a sign-invariant derivative-based cosine similarity method achieved perfect classification accuracy for polystyrene (PS), polyethylene terephthalate (PET), polyethylene (PE), and polypropylene (PP). The clustering-based workflow also produced more spatially coherent particle maps than direct software-based matching while substantially reducing processing time. The framework was evaluated for supervised classification-based MP indentification. These results show that multi-level correction combined with cluster-centroid spectral matching improves the robustness, efficiency, and interpretability of spectral-imaging-based microplastic identification.

We characterize bivariate measures of concordance of degree at most two in the sense that the measures evaluated at copula-mixture segments are polynomials of degree at most two. Our main device is the canonical polarization, which converts quadraticity of a functional into separate affinity, thereby allowing an integral representation for affine functionals to be applied sectionwise. Using this technique, we prove that measures of degree at most two are in bijection with copula-indexed measure fields satisfying several properties including square-group equivariance. The field at independence determines the affine linearization, while its displacement determines the purely quadratic part. Moreover, exact degree two is equivalent to field non-constancy. Leveraging our characterization result, we provide a construction of degree-two measures of concordance based on affine copula operators that are square-group equivariant. As a more concrete example, we derive a necessary and sufficient condition for a weighted averaging operator of square-group transformed copulas to generate a measure of concordance of degree two.