Statistics Applications · Cross-listed from eess.SP
Optimizing GEDI Simulator Configuration for European Temperate Forests
Selim Behloul, Nikola Besic, Steven Hancock et al. (+5 more)
Accurate estimation of aboveground biomass density is essential for quantifying forest carbon stocks. NASA's GEDI mission provides valuable canopy structure data, but its sparse sampling necessitates the use of simulators to calibrate biomass models at field inventory locations. The widely used simulator of Hancock et al. (2019) emulates GEDI waveforms from airborne
LiDAR point clouds, yet it has never been validated over European temperate forests. Here, we compare approximately 9,500 pairs of observed and simulated GEDI relative height (RH) profiles across French forests using the national airborne
LiDAR program as input. We separate two sources of error: waveform modeling differences, assessed by referencing both simulated and real RH metrics to a common ALS-derived ground elevation, and ground detection bias, evaluated by comparing each GEDI L2A processing
algorithm against the ALS reference. Under the baseline configuration, the mean absolute bias across the full RH profile reaches 0.69 m in leaf-on and 1.28 m in leaf-off conditions. Switching to intensity-based return weighting and selecting the a3 L2A
algorithm reduces these biases to 0.44 m and 0.40 m respectively. The a3
algorithm also achieves near-unbiased ground detection (-0.03 m versus -0.88 m for the default), directly reducing a previously overlooked source of error. We also show that leaf-off acquisitions and low-sensitivity shots, both typically excluded from standard biomass products, are simulated as reliably as their counterparts, substantially expanding the potential calibration and inference
datasets.
Read the full abstract ▾
All authors Selim Behloul, Nikola Besic, Steven Hancock, Cedric Vega, Sylvie Durrieu, Jean-Pierre Renaud, Ibrahim Fayad, Philippe Ciais
Machine Learning (Statistics) · Cross-listed from gr-qc
Improving the Sensitivity of Gravitational Wave Detection with Weighted Conformal Prediction
Ann-Kristin Malz, Gregory Ashton and Nicolo Colombo
In the last decade, kilometre-scale interferometric gravitational-wave detectors have observed hundreds of compact binary mergers, the majority of which are binary black holes. However, the data are noise-dominated, and multiple independent search
algorithms (pipelines) are used to enhance sensitivity and improve robustness. Rather than the standard approach of selecting the most significant pipeline output, we combine the outputs from all pipelines using a conformal prediction-based framework to provide statistically rigorous confidence estimates for candidate events. While combining pipelines improves sensitivity and ranking robustness, it requires a principled statistical framework that remains valid as data properties evolve across observing runs. A key challenge is distribution shifts between simulated
datasets used for training and calibration and the real, unlabelled, observations used for testing, which can invalidate coverage guarantees and bias confidence estimates. In this work, we address this challenge by incorporating likelihood-ratio reweighting into our conformal prediction framework to account for
covariate shift. Using mock
datasets containing simulated signals, we demonstrate that weighted conformal prediction restores well-calibrated coverage under
covariate shift and increases the confidence of events near the detection threshold, recovering true signals that would otherwise be missed.
Read the full abstract ▾
Statistics Applications · Cross-listed from cs.CY
Buyer Artificial Intelligence-Enabled Environmental Governance and Supplier Environmental Controversies: An Organizational Information Processing and Signaling
Yongchao Martin Ma and Xinya Guan
Environmental controversies in global supply chains pose significant risks for global buyers. This study examines whether overseas suppliers' exposure to buyers' artificial intelligence (AI)-enabled environmental governance reduces supplier environmental controversies. Drawing on organizational information processing theory and signaling theory, we investigate how suppliers' exposure to AI-enabled governance influences their environmental controversies and the institutional contingencies under which this effect varies. Using text analysis to measure buyer AI-enabled environmental governance, we analyze panel data on 2,505 suppliers of U.S.-listed firms across 41 countries from 2020 to 2024 with multidimensional fixed-effects models. We find that suppliers' exposure to buyer AI-enabled environmental governance is negatively associated with supplier environmental controversies in the following year. This negative relationship is stronger in supplier countries with higher AI readiness and regulatory quality. The study contributes to research on AI-enabled sustainability governance and sustainable supply chain risk management.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.CV
Your Model Already Knows Don't Teach It, Learn to Ask It: Soft Prompting for Few-Shot Adaptation of Vision-Language Models
Gautam Rajendrakumar Gare, Siyi Li, Hewei Wang et al. (+5 more)
We address few-shot object detection with
vision-
language models (VLMs) in out-of-domain settings such as aerial, industrial, and medical imagery, using only ten annotated images for supervision. Existing adaptation methods are discrete
prompt optimization and LoRA fine-tuning. We revisit a third option: soft prompting, where a small number of continuous
prompt tokens are optimized while the pretrained backbone remains frozen. We identify two key design choices. First, placing
prompt tokens at the cross-modal boundary between visual and text
tokens outperforms other placements (10.0 vs. 8.4 mAP). Second, initializing
prompts from the empty space
token outperforms semantic and random initialization. With these choices, one to three learned
tokens (7,168 parameters on average) match the best LoRA configuration on Roboflow20-VL (14.2 mAP, 10-shot) while training over 20,000x fewer parameters. Soft prompting remains harder to optimize, exhibiting higher variance across random seeds. Unlike LoRA, however, it causes no forgetting: the LoRA rank matching our accuracy reduces NaturalBench VQA accuracy by 35% relative, rising to 56% at the largest rank, whereas soft prompting leaves pretrained performance unchanged. The learned
tokens behave like
prompts rather than weights. They transfer to a newer model without retraining (+0.8 mAP on Qwen3.5-9B) and can be verbalized into readable
prompts competitive with
prompt-search methods (matching DetPO and outperforming GEPA). The approach also extends beyond detection. On RoboCasa manipulation tasks, the frozen $\pi_{0.5}$
vision-
language-action policy benefits from soft prompting, matching the LoRA baseline on two of three tasks when
tokens are placed at the gradient bottleneck. These results suggest modern VLMs already encode much of what is needed for specialized domains; the challenge is learning how to ask.
Read the full abstract ▾
All authors Gautam Rajendrakumar Gare, Siyi Li, Hewei Wang, Cesar Daniel Hernandez, Wei Zhao, Wolfgang M. Pauli, John Galeotti, Deva Ramanan
Statistical Methodology
Byzantine-tolerant distributed learning of finite mixture models under partial corruptions
Yimei Zhang, Jiahua Chen, Xiaozhou Wang et al. (+2 more)
Finite mixture models characterize heterogeneous populations and are increasingly fitted to distributed data using split-and-conquer procedures that aggregate local mixture estimates at a central server. The aggregation step can be seriously compromised when transmitted local mixture estimates are partially or completely corrupted. To guard against Byzantine failures, existing robust aggregation methods have been developed for settings in which a local mixture estimate is either entirely authentic or entirely corrupted. Such methods can discard useful information when only some component estimates are corrupted. We consider component-wise Byzantine failure, in which some component estimates may be corrupted, but for each mixture component, a majority of the corresponding local estimates remain authentic. We propose component-wise filtered mixture reduction (CFMR), which selects a data-driven anchor, aligns transmitted components, filters each aligned cluster by a majority radius, and aggregates the retained estimates through mixture reduction. By filtering out only unreliable components, CFMR preserves authentic information from partially corrupted machines without requiring knowledge of the failure rates. We establish an adaptive convergence bound for CFMR and show that, under suitable conditions, it attains the oracle rate that would be achieved if the authentic component estimates were known in advance. Simulations and a real-data application show that CFMR remains close to the component-level oracle, whereas whole-machine filtering and unprotected aggregation can deteriorate substantially.
Read the full abstract ▾
All authors Yimei Zhang, Jiahua Chen, Xiaozhou Wang, Yan Shuo Tan, Qiong Zhang
Machine Learning (Statistics)
A Hilbert-Valued Functional Decomposition Framework for Explaining Time-Dependent Outputs
Sophie Hanna Langbein, Niklas Koenen, Marvin N. Wright et al. (+1 more)
Feature-based explanations quantify features' influence on model predictions, but are primarily designed for scalar outputs. In many applications, however, outputs are functional or multivariate, such as time-dependent trajectories in demand forecasting. Consequently, existing approaches typically explain each output location independently, ignoring dependencies across the output components. We address this limitation by developing a unified framework for feature-based explanations of time-dependent outputs. Specifically, we generalize functional decomposition to
Hilbert-valued prediction functions and extend an existing feature-based explanation framework to this setting. Our framework introduces kernel-based output representations that enable time-dependency-aware explanations at multiple levels of temporal granularity, including time-specific, time-resolved, and time-aggregated, while providing a unified view in which existing methods arise as special cases. We validate our framework on synthetic and real-world data, including intraday financial
market volatility prediction and energy demand forecasting.
Read the full abstract ▾
All authors Sophie Hanna Langbein, Niklas Koenen, Marvin N. Wright, Julia Herbinger
Statistical Methodology
A unified framework for spatially resolved cortical activation analysis
Lars Knieper, Nadia Müller-Voggel, Tobias Hepp et al. (+2 more)
Cluster-based permutation tests are widely used for analyzing MEG data, even though they are limited to cluster-level inference and do not provide spatially resolved effect estimates. We propose a regression-based framework for modeling
brain activity directly on the cortical surface. Spatial effects are represented using Wendland radial basis functions, and model-based gradient boosting is employed for data-driven selection and estimation of localized activation patterns. This yields interpretable, spatially resolved effect estimates while mitigating the need for extensive multiple testing correction. In a simulation study with heterogeneous signal structures, the proposed approach recovers localized effects that are difficult to detect using cluster-based methods. An application to experimental MEG data further illustrates its ability to reveal spatially specific activation patterns. Overall, this unified framework provides a flexible and interpretable approach for modeling cortical surface data within a unified statistical setting.
Read the full abstract ▾
All authors Lars Knieper, Nadia Müller-Voggel, Tobias Hepp, Anna von Plessen, Elisabeth Bergherr
Statistics Theory · Cross-listed from math.NA
Minimax-Optimal Early Stopping for Continuous-Time SGD via the Discrepancy Principle
Tim Jahn, Loucas Pillaud-Vivien and Adrien Schertzer
We study early stopping for a continuous-time model of
stochastic gradient descent (SGD) in ill-posed linear inverse problems. We consider an a posteriori stopping rule based on the discrepancy principle, which stops the dynamics once the residual reaches the noise level. Unlike deterministic gradient flow, the
stochastic dynamics exhibits persistent multiplicative fluctuations whose amplitude scales with the residual itself. Under a source condition on the initial error and a
polynomial bound on the spectrum of the empirical covariance operator, we prove that, with high probability, the error at this random stopping time achieves the minimax rate over the corresponding source class, up to a logarithmic factor. Our results therefore establish the discrepancy principle as an adaptive regularization strategy for continuous-time SGD that is minimax-optimal up to a logarithmic factor, despite the persistent fluctuations induced by
stochastic sampling. To our knowledge, this is the first convergence-rate result for a continuous-time model of
stochastic gradient descent in inverse problems under an a posteriori stopping rule: existing analyses, including those of variance-reduced variants, bound the stopping time but do not quantify the error attained there.
Read the full abstract ▾
Statistical Methodology
Global and local indicators of spatial connectivity for areal data
Álvaro Briz-Redón and Marco Ruiz-Valderrama
Methods for areal data commonly describe spatial structure through global autocorrelation measures or local indicators based on neighboring values. In many applications, however, it is also relevant to know whether high- or low-valued areas form connected spatial structures. Motivated by topological data analysis, we develop global and local indicators of spatial connectivity for observations defined on an areal adjacency graph. We construct thresholded spatial graphs and use the Betti-0 curve to record their number of connected components. We further derive a local decomposition in which each area's contribution to the Betti-0 curve is expressed as an activation term minus a merging term. Each component fusion contributes to the merging term through a symmetric allocation between the entering area and the neighboring areas through which the fusion occurs. Integrating the merging contributions yields area-level indicators of participation in the connectivity of high- or low-valued regions. Global and local inference is based on random relabeling, with a global test for departures from spatial exchangeability and a conditional test for the local indicators. We illustrate the framework using COVID-19 incidence in Italian provinces during two epidemic waves. Significant departures toward greater connectivity were found for both high- and low-incidence areas, with the largest discrepancy corresponding to high incidence during the first wave. The proposed indicators complement conventional measures of spatial association and are applicable whenever the spatial coherence of threshold exceedances or deficits is of interest.
Read the full abstract ▾
Statistical Methodology
Optimal Covariate Adjustment beyond the Average Treatment Effect: Treated-Population and Overlap-Weighted Estimands
Shoki Okubo
Graphical causal inference supplies a complete theory of efficient
covariate adjustment for the average treatment effect: one adjustment set, computable from the graph, is optimal under every compatible distribution. We show that this is a property of the average treatment effect's inverse-prevalence weights, not of causal estimands in general. For the average treatment effect on the treated we index the efficiency bound by the adjustment set and derive exact identities for its change under treatment-side and outcome-side extensions of a valid set.
Covariates that predict only the treated-arm outcome are exactly efficiency-neutral, and
covariates that predict the control-arm outcome can strictly increase the bound when the propensity is below one half – a reversal of the supplementation lemma whose source is an arithmetic-
geometric-mean inequality that holds for the average treatment effect and fails for the treated-population estimand. A construction with two faithful distributions on one graph proves that no graphical optimality criterion exists for the treated-population estimand; under no effect modification the ATE-optimal set is nonetheless optimal among the graphically valid sets, with an exact expression for its advantage. The results extend to weighted average treatment effects with propensity-dependent weights, yielding symmetric thresholds for overlap weights, an estimand-drift phenomenon under instrument adjustment, and a characterization of constant weights as the only smooth positive weights for which outcome-side supplementation never increases the bound. Simulations and the LaLonde data provide illustrations.
Read the full abstract ▾
Statistical Methodology · Cross-listed from cs.LG
Hierarchical Clustering Can Jointly Satisfy Richness, Consistency, and Scale Invariance
Daichi Kuroda, Maximilien Dreveton, Matthias Grossglauser et al. (+1 more)
Despite its ubiquity, clustering lacks a universally accepted definition of what is a cluster. Kleinberg's Impossibility
Theorem formalizes this difficulty by showing that no flat clustering method can simultaneously satisfy three natural axioms: scale invariance, richness, and consistency. In this paper, we ask whether this impossibility persists when the output is a hierarchy rather than a single partition. We show that, in contrast to the flat clustering setting, the hierarchical analog of these axioms are jointly satisfiable. In fact, there exist uncountably many hierarchical clustering methods satisfying these axioms, which we call admissible. We explicitly construct several admissible methods, including methods based on well-separated clusters and a non-binary version of single linkage. For certain pairs of admissible methods, the hierarchy produced by one always refines that produced by the other. This refinement relation defines a partial order on the class of admissible methods. This partially ordered set has no greatest element and contains uncountably many pairwise incompatible maximal elements, revealing substantial diversity among admissible methods. Nevertheless, this diversity is constrained: every admissible method contains a hierarchy of sufficiently well-separated clusters, and every finite collection of admissible methods shares such a nontrivial common backbone.
Read the full abstract ▾
All authors Daichi Kuroda, Maximilien Dreveton, Matthias Grossglauser, Patrick Thiran
Statistics Theory
Spike Estimation from Heteroscedastic Noise via Random Splitting
Zhigang Bao, Kha Man Cheong, Yuji Li et al. (+1 more)
In this paper, we consider a spiked Wigner type
matrix with a heteroscedastic and unknown variance profile. It is well known that in the supercritical regime of the BBP transition, strong spikes can create outliers in the spectrum. Unfortunately, in the heteroscedastic case, in general it is not possible to estimate the spike strength from these observed outlier consistently, as the latter is a solution to a Dyson equation with unknown parameters from the variance profile. In this paper, inspired by the work on sparse
matrix completion \citep{BordenaveCosteNadakuditi2023}, we introduce an asymmetrized model by randomly splitting the spiked
matrix into two parts, which transforms the noisy Wigner type
matrix into a non Hermitian random
matrix, while preserving the Hermitian spikes at the cost of a dilution. We establish a BBP type transition for the asymmetrized model, from which we can estimate the strength of the spikes precisely, even without knowing the variance profile of the noise part. We then further apply our approach to study the correlation between two correlated spiked models, where the spike/signal parts of the two models are correlated, and the noise parts are independent but may both be heteroscedastic. By applying our asymmetrization approach to the two models separately and also jointly, we are able to obtain a precise estimate of the correlation between the signal parts of the two models.
Read the full abstract ▾
All authors Zhigang Bao, Kha Man Cheong, Yuji Li, Jiaxin Qiu
Statistics Theory
Small noise asymptotics for linear parabolic SPDEs in two space dimensions with unknown damping factors
Yozo Tonaki, Yusuke Kaino and Masayuki Uchida
We study parametric estimation for second order linear parabolic
stochastic partial differential equations in two space dimensions with a small
volatility parameter driven by a $Q$-Wiener process with an unknown damping parameter using high frequency spatio-temporal data. We first provide an
estimator for the damping parameter of the $Q$-Wiener process utilizing realized quadratic variations based on spatial and temporal increments. We next propose minimum contrast
estimators of the diffusive and advective parameters in the SPDE using a contrast function with the proposed
estimator of the damping parameter. We then construct a quasi-maximum likelihood
estimator of the reaction parameter in the SPDE using the approximate coordinate process derived from the
estimators of the diffusive and advective parameters. We also provide simulation results of the proposed
estimators.
Read the full abstract ▾
Statistical Methodology
Beyond Tweedie's Formula: Conditional Score Modeling for Empirical Bayes Inference
Shonosuke Sugasawa and Zhigen Zhao
We propose conditional f-modeling (Cf-modeling), a framework for empirical Bayes inference with
covariates. A central identity shows that the conditional marginal score function determines not only the posterior mean through Tweedie's formula, but also the posterior moment-generating function, providing a basis for recovering posterior quantities without explicit prior modeling. Motivated by this observation, we treat the conditional marginal score as the primary object of inference and estimate it directly using an energy-based representation and Hyvärinen score matching, thereby avoiding potentially intractable
covariate-dependent normalizing constants. The resulting framework flexibly accommodates
covariate effects and heteroscedasticity and provides a practical approach to posterior moment estimation and uncertainty quantification. We demonstrate the effectiveness of the proposed method through simulations and an RNA-seq application.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
How Wrong Can a Good Predictor Be? Diverging Updates with Vanishing Predictive KL
Qifu Wen, Shuaijun Liu, Zihan Zhou et al. (+2 more)
Accurate posterior prediction need not require accurate approximation of
Bayesian updates. We prove that an unbounded gap between the update maps can coexist with vanishing predictive KL for every fixed finite $K\ge2$ in a stationary symmetric Gaussian HMM. Exact
Bayesian mixing and an explicit deterministic radial filter act on the same $K-1$ belief coordinates. As $q\to0^+$, their separation in centered logits in the worst case grows at least linearly in the natural confidence scale $L_K(q)$, while their categorical $D_{\mathrm{KL}}(\mathrm{exact}\|\mathrm{radial})$ vanishes at the same explicit witness. Along stationary HMM trajectories, the expected terminal KL between filtered posteriors also converges to zero at $H(q)=\lceil-\log(q)/c\rceil+1$. Typical blocks without switches drive both filters into a common confidence cone, where softmax
curvature suppresses their disagreement; a single Gaussian maximal event controls adaptive noise. A sweep with equally spaced Gaussians over $K\in\{2,4,8\}$ illustrates the opposing trends, and binary controls at long horizons compare saturating and nonsaturating recurrences. The result isolates two missing links between internal update gaps and predictive cost: the contribution of separating states to expected loss and decoder sensitivity. Thus even an unbounded internal update gap does not by itself certify predictive failure. The construction is fixed in $K$ and does not provide a universal criterion for when compression is harmless or characterize when internal gaps must incur task loss.
Read the full abstract ▾
All authors Qifu Wen, Shuaijun Liu, Zihan Zhou, Xi Zeng, Ningxin Su
Statistical Methodology
Is the Linear Threshold Good Enough? A Scale-Free Parameter and Adequacy Test for Curvature-Induced Threshold Displacement
Subir Hait
Applied work often locates a threshold by linearizing a smooth function about a reference point and solving for the crossing. When the function is curved, the linear crossing can be substantively displaced even when standard errors are valid. We introduce the
curvature-overstatement parameter $\Theta_{COT}=\log(|h_2^*|/|h_1^*|)$, the log ratio of second- to first-order threshold displacement. On the quadratic branch continuous with the linear solution, $|h_2^*/h_1^*|=2/(1+\sqrt{1-u})$, where $u=2qa/b^2$ is a dimensionless index formed from the local gap, slope, and
curvature. Thus $\Theta_{COT}$ is scale-free, depends on the local parameters through one scalar, and has regular-domain range $(-\infty,\log 2)$, with boundary limit $\log 2$ at tangency. We derive regular asymptotic inference, characterize local-to-tangency and weak-slope failures, and give a remainder bound linking the second-order crossing to the true threshold. The main practical contribution is an adequacy test that can affirm that the linear threshold is accurate within a prespecified proportional tolerance, rather than treating failure to detect
curvature as evidence of adequacy. Monte Carlo results confirm regular-case calibration, the predicted nonstandard behavior near tangency, and failure under weak slope, while bootstrap diagnostics identify regimes in which regular inference should not be used. COT therefore provides an effect-size scale, adequacy test, and diagnostics for deciding whether a first-order threshold is accurate enough to report.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from math.OC
Conformal-DRO: Distributionally Robust Optimization with Conformalized Ambiguity Set
Luhao Zhang and Shixiang Zhu
Data-driven distributionally robust optimization (DRO) typically treats the conditional outcome law as fixed and uses ambiguity sets to capture estimation error. This paper studies latent distributional heterogeneity, where each instance has an unobserved law but contributes only one observation, so uncertainty persists even if the mixture law is known. We propose Conformal-DRO, which uses nested conformal regions to construct an ambiguity set for the future latent law. Under exchangeability, the set covers this law with probability at least $1-\alpha$ in finite samples, without estimating underlying latent laws or their mixing mechanism. The conformal path induces a data-driven transport geometry, while $\alpha$ determines the radius. The worst-case problem reduces to a finite linear program over conformal shells and admits sparse
adversarial solutions. The resulting robust value provides a finite-sample certificate for the selected decision's expected cost.
Read the full abstract ▾
Statistical Methodology
Bayesian Variable Selection for High-Dimensional Predictors with Missing Psychometric Outcomes
Zongyue Teng, Shujie Ma, Timothy J. Hohman et al. (+2 more)
High-dimensional,
multimodal predictors and partially observed multivariate outcomes are common in psychometric research. However, existing regularization methods often do not accommodate hierarchical predictor structures and are primarily designed for univariate outcomes. We propose SHIM, a
Bayesian framework for structured variable selection that combines hierarchical horseshoe shrinkage with a
Bayesian treatment of missing outcomes. The framework jointly accommodates predictor hierarchies, dependence among outcomes, and incomplete multivariate responses. We establish theoretical properties of the proposed prior specification and evaluate SHIM through simulation studies. The results demonstrate that SHIM balances sensitivity with false-positive control while yielding accurate coefficient estimates and well-calibrated uncertainty quantification. We further apply SHIM to data from an Alzheimer's disease cohort to characterize associations between
multimodal neuroimaging measures and multivariate neuropsychological outcomes and to generate posterior-based multiple imputations for downstream analyses of the relationships between fluid biomarkers and cognition. An R package, shim, is publicly available to facilitate implementation.
Read the full abstract ▾
All authors Zongyue Teng, Shujie Ma, Timothy J. Hohman, Angela L. Jefferson, Panpan Zhang
Statistics Theory
A Scale Invariance Property of PCA
Jonathan Landy
The PCA
algorithm is sensitive to changes in measurement scale. Measuring one variable of a system in inches rather than centimeters, say, alters both its principal axes and principal
eigenvalues. Although this scale dependence is generally complicated, we show here that it nevertheless obeys a strict invariance property: under a continuous scale adjustment, the initial state's $k$-th largest principal component (ordered by
eigenvalue) continuously evolves into the final state's $k$-th largest principal component, for each $k$. In this sense, we can say that the modes of PCA are "order-stable" with respect to changes in measurement scale. A special case occurs when scaling along directions that are orthogonal to some modes. Here, apparent
eigenvalue crossings can occur. However, we show that we can interpret these apparent crossings as cases where the modes instantaneously swap their orientation, in this way maintaining the required order stability.
Read the full abstract ▾
Statistical Methodology
Discretization in covariate-adaptive randomization: gains and losses
Zixuan Zhao and Feifang Hu
Covariate-adaptive randomization(CAR) is widely implemented in clinical trials to balance prognostic
covariates across treatment arms. Continuous
covariates are often discretized into strata in practice, yet their consequences are not clearly understood. This paper provides a comprehensive study of the impact of discretization on both the CAR design process and the inferential results thereafter. We establish the asymptotic properties of both imbalance measures and treatment effect
estimators under discretized and non-discretized settings. Practical recommendations are given on when and how discretization should be employed. We show that discretization in design is generally recommended, as it enhances robustness against model misspecification. However, if the true model is known, the most efficient strategy is to balance
covariates according to that model in the design. The theoretical results are corroborated by extensive simulation studies and an empirical application to a diabetes trial
dataset. Together, the results clarify the gains and losses of discretization in CAR and pave the way for learning impact of discretization to other designs and beyond.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Importance Weighting for Unlabeled-unlabeled Learning under Distribution Shift
Atsutoshi Kumagai, Tomoharu Iwata, Hiroshi Takahashi et al. (+3 more)
Unlabeled-unlabeled (UU) learning allows us to learn a binary classifier from two sets of unlabeled data with different class-priors. It is a general framework because it includes a wide variety of supervised learning such as positive-unlabeled (PU) learning, noisy label learning, and similarity-based learning. Existing UU learning assumes that the test and training distributions have the same class-conditional densities. However, this assumption rarely holds in practice due to distribution shifts. This paper proposes a distribution shift adaptation method for UU learning that uses UU data in the training distribution and a few UU data in the test distribution. The proposed method is based on the importance weighting, which minimizes the test risk by using training data with estimated importance weights. Although existing importance weighting methods cannot handle UU data, we show that it can be done in a principled manner. Thanks to the generality of UU learning, our method can handle various learning problems such as PU and noisy label learning under distribution shift within a single framework while existing methods are usually tailored to a specific problem. Moreover, it does not require any assumption of the shift types such as
covariate shift. We experimentally demonstrate the effectiveness of the proposed method with real-world
datasets.
Read the full abstract ▾
All authors Atsutoshi Kumagai, Tomoharu Iwata, Hiroshi Takahashi, Taishi Nishiyama, Kazuki Adachi, Yasuhiro Fujiwara
Machine Learning (Statistics) · Cross-listed from cs.LG
Phases in a class of associative memories via hidden neurons
Toshihiro Ota and Masato Taki
Associative
memory in the Hopfield network is attractor dynamics in a disordered many-body system, and higher-order and exponential extensions turn its
retrieval update into softmax
attention. The
polynomial and exponential regimes have been analyzed by different methods, with no common architecture in which to ask what fixes the storage scale. In this paper we study the bipartite architecture of Krotov and Hopfield, which we call the class $H$, whose model is fixed by a Lagrangian for each layer, taking the hidden
neurons as the order parameter of
retrieval. At
polynomial load the replica method yields the replica-symmetric phase diagrams and closed-form capacities, and the crosstalk moment is common to Ising and spherical visible
neurons, so their differences come from the visible
entropy. With a softmax hidden layer the load is exponential, and a copy representation maps the thermodynamics onto random-energy-model counting, with paramagnetic, condensed, and frozen phases. Heating destabilizes
retrieval by quantized reassignments of
attention, and typical Gaussian patterns remain metastable at every load. The regimes differ in their crosstalk statistics, central-limit at
polynomial load and large-deviation at exponential load, and the class $H$ splits
retrieval into two roles, the visible Lagrangian fixing stability and the hidden one the storage scale, two axes that may also guide the design of new Lagrangians.
Read the full abstract ▾
Statistical Methodology
Generalized Ridge Refitting for the Lasso and Prediction Improvement Bounds
Guo Liu
We study a class of Lasso based
estimators obtained by applying a quadratic correction on the Lasso equicorrelation set. The penalty
matrix determines both the magnitude and geometry of the correction and contains, among other cases, the isotropic Lasso–Ridge correction, least squares refitting, Gram proportional interpolation between the Lasso and least squares, and coordinate specific penalties. We first derive a closed form representation and isolate the positive gain component of the resulting prediction improvement. We then control the remaining
stochastic linear term in expectation by localizing the random signed equicorrelation model around a deterministic reference support. This yields a finite sample expectation bound that explicitly accounts for the randomness induced by Lasso model selection. The resulting decomposition provides a unified framework for understanding when Lasso based quadratic corrections can improve prediction.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
AUC Maximization from Biased Positive-unlabeled Data with Confidence
Atsutoshi Kumagai, Tomoharu Iwata, Hiroshi Takahashi et al. (+3 more)
Maximizing the area under the receiver operating characteristic curve (AUC) is a standard approach to imbalanced binary classification. Although positive and negative data are required for maximizing the AUC, negative data are often difficult to collect in some real-world applications due to
privacy concerns or the need for specialized expertise to annotate them. Thus, AUC maximization from positive and unlabeled (PU) data has been attracting
attention. Existing methods assume that labeled positive data are unbiased samples from the true positive distribution. However, this ideal assumption is often violated in practice. In this paper, we propose a method to maximize the AUC from biased PU data. To address the bias, our key idea is to exploit confidence, i.e., the probability that an instance is positive, associated with the small number of labeled positive data. We derive an
estimator of the AUC risk using biased PU data with confidence, enabling AUC maximization under such bias. We further show that the rewritten AUC risk induces a Bayes-optimal AUC ranking even when the available confidence is any strictly increasing transformation of the true posterior probability. We experimentally show the effectiveness of our method on eight real-world
datasets.
Read the full abstract ▾
All authors Atsutoshi Kumagai, Tomoharu Iwata, Hiroshi Takahashi, Taishi Nishiyama, Kazuki Adachi, Yasuhiro Fujiwara
Statistical Methodology
Surprise Reduction and Nullification in Bayesian and Inverse Bayesian Inference under Ambiguous Prediction-Error Attribution
Shuji Shinohara, Daiki Morita, Yoshihiro Nakajima et al. (+4 more)
In non-stationary environments, prediction errors may signal environmental change or transient outliers, and adaptive systems must track such changes without overreacting to outliers. We distinguish surprise reduction, which updates beliefs to fit observations, from surprise nullification, which weakens constraints imposed by the predictive structure, and formalize both within
Bayesian and inverse
Bayesian (BIB) inference. Belief and likelihood updates are derived from variational objectives sharing a nullification strength, determined endogenously by minimizing surprise under the candidate post-update predictive distribution. In the Gaussian case, nullification expands belief and likelihood variances by a common factor relative to standard
Bayesian updating, leaving the ratio unchanged. BIB thus defers attribution of the prediction error, committing to neither latent-state change nor observation-process uncertainty. The nullification strength is carried over as a candidate and is maintained or released according to the predictive surprise of the next observation. In a mean estimation task with outliers and changepoints, no scanned parameter setting of a Sage-Husa-type adaptive Kalman filter, fixed-strength BIB variant, or belief-forgetting-only variant outperforms BIB in both changepoint tracking and post-outlier stability. An oracle-informed reduced
Bayesian model tracks changepoints better but is less stable after outliers. Although BIB maintains no explicit hypotheses about changepoints or outliers, it generates event-dependent dynamics. The learning rate increases after changepoints, whereas after outliers, nullification is released, and this increase is suppressed. Deferring attribution and letting subsequent observations differentiate the responses may constitute a principle of adaptive inference in non-stationary environments.
Read the full abstract ▾
All authors Shuji Shinohara, Daiki Morita, Yoshihiro Nakajima, Takeshi Takano, Masakazu Higuchi, Ung-il Chunge, Yukio-Pegio Gunji