Machine Learning (Statistics)
Uncovering Locally Low-dimensional Structure in Networks by Locally Optimal Spectral Embedding
Hannah Sansford, Nick Whiteley and Patrick Rubin-Delanchy
Standard Adjacency Spectral
Embedding (ASE) relies on a global low-rank assumption often incompatible with the sparse, transitive structure of real-world networks, causing local
geometric features to be 'smeared'. To address this, we introduce Local Adjacency Spectral
Embedding (LASE), which uncovers locally low-dimensional structure via weighted spectral decomposition. Under a latent position model with a kernel feature map, we treat the image of latent positions as a locally low-dimensional set in infinite-dimensional feature space. We establish finite-sample bounds quantifying the trade-off between the statistical cost of localisation and the reduced truncation error achieved by targeting a locally low-dimensional region of the
embedding. Furthermore, we prove that sufficient localisation induces rapid spectral decay and the emergence of a distinct spectral gap, theoretically justifying low-dimensional local
embeddings. Experiments on synthetic and real networks show that LASE improves local reconstruction and visualisation over global and subgraph baselines, and we introduce UMAP-LASE for assembling overlapping local
embeddings into high-fidelity global visualisations.
Read the full abstract ▾
Statistical Methodology
Bayesian Model Calibration with Integrated Discrepancy: Addressing Inexact Dislocation Dynamics Models
Liam Myhill, Enrique Martinez Saez and Sez Russcher
In this work, a novel approach to
Bayesian model calibration routines is developed which reinterprets the traditional definition of model discrepancy as defined by Kennedy and O'Hagan (KOH). The novelty lies in the integration of $\delta_\theta(x_i)$ GPs within the simulator, which is approximated as a GP surrogate model to ensure computational tractability. This approach assumes that the utilized simulator sufficiently predicts observed trends when calibrated with respect to the application domain, and that all model-form errors can be attributed to uncertainty in the input parameters. In contrast, the KOH method assumes discrepancy to be inherently decoupled from the simulator, acting as a 'catch-all' for various sources of model error. The new method is applied to Molecular Dynamics observations of the critical stress to drive dislocation dipoles, and equivalent predictions using a Discrete Dislocation Dynamics simulator whose coarse-grained physical interpretation of the underlying physical mechanisms requires calibration against MD observations. We present an overview of similar state-aware calibration routines; differentiate the provided approach through redefining the commonly used discrepancy Gaussian process and
benchmark against KOH. A philosophical argument as to when application of the proposed method is appropriate is provided, and future directions for expanding upon this methodology are proposed.
Read the full abstract ▾
Statistical Methodology
Distributionally balanced sampling designs
Anton Grafström and Wilmer Prentius
We propose Distributionally Balanced Designs (DBD), a new class of probability sampling designs that target representativeness at the level of the full auxiliary distribution rather than selected moments. In disciplines such as ecology, forestry, and environmental sciences, where field data collection is expensive, maximizing the information extracted from a limited sample is critical. More precisely, DBD can be viewed as minimum discrepancy designs that minimize the expected discrepancy between the sample and population auxiliary distributions. The key idea is to construct samples whose empirical auxiliary distribution closely matches that of the population. We present a first implementation of DBD based on an optimized circular ordering of the population, combined with random selection of a contiguous block of units. The ordering is chosen to minimize the design-expected energy distance, a discrepancy measure that captures differences between distributions beyond low-order moments. This criterion promotes strong spatial spread, and yields low variance for Horvitz-Thompson
estimators of totals of functions that vary smoothly with respect to auxiliaries. Simulation results show that approximate DBD achieves better distributional fit than state-of-the-art methods such as the local pivotal and local cube designs. Hence, DBD can improve the reliability of estimates from costly field data, making distributional balancing effective for constructing representative surveys in resource-constrained applications.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Deep Generative Transformers for Probabilistic Time Series and Spatiotemporal Forecasting
Rajdeep Pathak, Rahul Goswami, Madhurima Panja et al. (+2 more)
Reliable uncertainty quantification is paramount for
forecasting multivariate time series and spatiotemporal data. While
Transformer architectures excel at sequence modeling, current probabilistic approaches typically rely on restrictive parametric likelihoods or
quantile-based objectives, thereby limiting their ability to capture complex joint distributions in correlated time series. To overcome these limitations, we propose Enformer and its spatiotemporal extension, GEnformer. These models synthesize the expressive power of
Transformers with engression, a
stochastic learning paradigm for modeling conditional distributions. By injecting
stochastic noise and optimizing a strictly proper scoring objective, our frameworks directly learn conditional predictive distributions without imposing parametric assumptions. This design ensures the generation of coherent multivariate trajectories while maintaining the
Transformer's efficacy in modeling long-range dependencies and cross-series interactions. The probabilistic capability of Enformer is achieved with an asymptotic overhead of only a constant factor over a deterministic
Transformer with an identical configuration. We extensively evaluate our frameworks on prominent multivariate
benchmarks for temporal dynamics and real-world epidemic
datasets for spatiotemporal dynamics. Empirical results demonstrate that both frameworks yield calibrated probabilistic forecasts and consistently outperform state-of-the-art baselines.
Read the full abstract ▾
All authors Rajdeep Pathak, Rahul Goswami, Madhurima Panja, Palash Ghosh, Tanujit Chakraborty
Statistical Methodology · Cross-listed from cs.LG
Causal Representation Learning with Optimal Compression under Complex Treatments
Wanting Liang, Haoang Chi and Zhiheng Zhang
Estimating Individual Treatment Effects (ITE) in multi-treatment scenarios faces two critical challenges: the Hyperparameter Selection Dilemma for balancing weights and the Curse of Dimensionality in computational scalability. This paper derives a novel multi-treatment generalization bound and proposes a theoretical
estimator for the optimal balancing weight $\alpha$, eliminating expensive heuristic tuning. We investigate three balancing strategies: Pairwise, One-vs-All (OVA), and Treatment Aggregation. While OVA achieves superior precision in low-dimensional settings, our proposed Treatment Aggregation ensures both accuracy and O(1) scalability as the treatment space expands. Furthermore, we extend our framework to a generative architecture, Multi-Treatment CausalEGM, which preserves the Wasserstein geodesic structure of the treatment
manifold. Experiments on semi-synthetic and image
datasets demonstrate that our approach significantly outperforms traditional models in estimation accuracy and efficiency, particularly in large-scale intervention scenarios.
Read the full abstract ▾
Statistics Applications · Cross-listed from q-fin.RM
Deriving the term-structure of loan write-off risk under IFRS 9 by using survival analysis: A benchmark study
Arno Botha, Mohammed Gabru, Marcel Muller et al. (+1 more)
The estimation of marginal loan write-off probabilities is a non-trivial task when modelling the loss given default (LGD) risk parameter in credit risk. We explore two types of survival models in estimating the overall write-off probability over default spell time, where these probabilities form the term-structure of write-off risk in aggregate. These survival models include a discrete-time hazard (DtH) model and a conditional inference survival tree. Both models are compared to a cross-sectional logistic regression model for write-off risk. All of these (first-stage) models are then ensconced in a broader two-stage LGD-modelling approach, wherein a loss severity model is estimated in the second stage. In expanding the model suite, a novel dichotomisation step is introduced for collapsing the write-off probability into a 0/1-value, prior to LGD-calculation. A
benchmark study is subsequently conducted amongst the resulting LGD-models. We find that the DtH-model outperforms other two-stage LGD-models admirably across most diagnostics. However, a single-stage LGD-model still had the best results, likely due to the peculiar `L-shaped' LGD-distribution in our data. Ultimately, we believe that our tutorial-style work can enhance LGD-modelling practices when estimating the expected credit loss under IFRS 9.
Read the full abstract ▾
All authors Arno Botha, Mohammed Gabru, Marcel Muller, Janette Larney
Statistical Methodology
Data Fusion with Distributional Equivalence Test-then-pool
Linying Yang, Xing Liu and Robin J. Evans
Randomized controlled trials (RCTs) are the gold standard for causal inference, yet practical constraints often limit the size of the concurrent control arm. Borrowing control data from previous trials offers a potential efficiency gain, but naive borrowing can induce bias when historical and current populations differ. Existing test-then-pool (TTP) procedures address this concern by testing for equality of control outcomes between historical and concurrent trials before borrowing; however, standard implementations may suffer from reduced power or inadequate control of the Type-I error rate. We develop a new TTP framework that fuses control arms while rigorously controlling the Type-I error rate of the final treatment effect test. Our method employs kernel two-sample testing via maximum mean discrepancy (MMD) to capture distributional differences, and equivalence testing to avoid introducing uncontrolled bias, providing a more flexible and informative criterion for pooling. To ensure valid inference, we introduce partial bootstrap and partial permutation procedures for approximating null distributions in the presence of heterogeneous controls. We further establish the overall validity and consistency. We provide empirical studies demonstrating that the proposed approach achieves higher power than standard TTP methods while maintaining nominal error control, highlighting its value as a principled tool for leveraging historical controls in modern clinical trials.
Read the full abstract ▾
Machine Learning (Statistics)
Hypercomplex Widely Linear Processing: Fundamentals for Quaternion Machine Learning
Sayed Pouria Talebi and Clive Cheong Took
Numerous attempts have been made to replicate the success of complex-valued
algebra in engineering and science to other hypercomplex domains such as quaternions, tessarines, biquaternions, and octonions. Perhaps, none have matched the success of quaternions. The most useful feature of quaternions lies in their ability to model three-dimensional rotations which, in turn, have found various industrial applications such as in aeronautics and computergraphics. Recently, we have witnessed a renaissance of quaternions due to the rise of machine learning. To equip the reader to contribute to this emerging research area, this chapter lays down the foundation for: - augmented statistics for modelling quaternion-valued random processes, - widely linear models to exploit such advanced statistics, - quaternion calculus and
algebra for algorithmic derivations, - mean square estimation for practical considerations. For ease of exposure, several examples are offered to facilitate the learning, understanding, and(hopefully) the adoption of this multidimensional domain.
Read the full abstract ▾
Statistical Methodology
Robust Sequential Hypothesis Testing with Generalized Estimating Equations for Incomplete Clustered and Longitudinal Data
Nathan T. Provost and Abdus S. Wahed
Existing sequential generalized estimating equation methodology for longitudinal and group-correlated data focuses on narrow hypotheses concerning treatment efficacy and often makes modeling assumptions that impede the desirable robustness of the involved test statistics. Drawing upon the well-established theory of incremental information gain for well-posed sequential analyses, we develop an approach that does not rely on modeling assumptions that infringe upon the robustness of the resulting
estimators while simultaneously testing a much wider range of hypotheses. Our methodology provides general submatrix-level asymptotic theory for the evaluation of joint covariance
matrices of sequential test statistics. Moreover, this framework allows us to construct a novel approach to computing efficacy boundaries, the likes of which can be estimated with greater precision at later interim times. These constructions also accommodate accessible multiple imputation procedures, thereby allowing for our approach to be applied to incomplete
datasets. Type I error and power are assessed through a series of comprehensive simulations mirroring the simulations of recent work to facilitate a proper comparison. We conclude by applying our methods to a
dataset from a longitudinal study concerning the impact of race on the efficacy a treatment for hepatitis C.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Language Generation with Replay: A Learning-Theoretic View of Model Collapse
Giorgio Racca, Michal Valko and Amartya Sanyal
As scaling laws push the training of frontier large
language models (LLMs) toward ever-growing data requirements, training pipelines are approaching a regime where much of the publicly available online text may be consumed. At the same time, widespread LLM usage increases the volume of machine-generated content on the web; together, these trends raise the likelihood of generated text re-entering future training corpora, increasing the associated risk of performance degradation often called model collapse. In practice, model developers address this concern through data cleaning, watermarking, synthetic-data policies, or, in some cases, blissful ignorance. However, the problem of model collapse in generative models has not been examined from a learning-theoretic perspective: we study it through the theoretical lens of the
language generation in the limit framework, introducing a replay adversary that augments the example stream with the generator's own past outputs. Our main contribution is a fine-grained learning-theoretic characterization of when replay fundamentally limits generation: while replay is benign for the strongest notion of uniform generation, it provably creates separations for the weaker notions of non-uniform generation and generation in the limit. Interestingly, our positive results mirror heuristics widely used in practice, such as data cleaning, watermarking, and output filtering, while our separations show when these ideas can fail.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
A Further Efficient Algorithm with Best-of-Both-Worlds Guarantees for $m$-Set Semi-Bandit Problem
Botao Chen, Jongyeong Lee, Chansoo Kim et al. (+1 more)
This paper studies the optimality and complexity of Follow-the-Perturbed-Leader (FTPL) policy in $m$-set semi-bandit problems. FTPL has been studied extensively as a promising candidate of an efficient
algorithm with favorable regret for
adversarial combinatorial semi-bandits. Nevertheless, the optimality of FTPL has still been unknown unlike Follow-the-Regularized-Leader (FTRL) whose optimality has been proved for various tasks of online learning. In this paper, we extend the analysis of FTPL with
geometric resampling (GR) to $m$-set semi-bandits, which is a special case of combinatorial semi-bandits, showing that FTPL with Fréchet and Pareto distributions with certain parameters achieves the best possible regret of $O(\sqrt{mdT})$ in
adversarial setting. We also show that FTPL with Fréchet and Pareto distributions with a certain parameter achieves a logarithmic regret for
stochastic setting, meaning the Best-of-Both-Worlds optimality of FTPL for $m$-set semi-bandit problems. Furthermore, we extend the conditional
geometric resampling to $m$-set semi-bandits for efficient loss estimation in FTPL, reducing the computational complexity from $O(d^2)$ of the original
geometric resampling to $O(md(\log(d/m)+1))$ without sacrificing the regret performance.
Read the full abstract ▾
All authors Botao Chen, Jongyeong Lee, Chansoo Kim, Junya Honda
Statistical Methodology
Causal Influence Maximization with Steady-State Guarantees
Renjie Cao, Zhuoxin Yan, Zichen Wang et al. (+2 more)
Classical influence maximization optimizes expected reach, which can be misaligned with welfare when exposures have heterogeneous, saturating, or adverse effects. We study seed selection for steady-state causal welfare under network interference. Under our exposure-separable outcome model and weak independent live-edge propagation, a deterministic surrogate built from expected activations and exposure counts approximates welfare with uniform $O(\varepsilon^2)$ error for a fixed graph and seed budget, where $\varepsilon$ bounds edge activation probabilities. Building on this reduction, our two-stage method, CIM, learns shape-constrained exposure-response functions from logged
diffusion-outcome data and estimates expected activations and exposures by Monte Carlo simulation. For unregularized fits on a design that covers the evaluated exposures, and under noise and simulation conditions, we bound the welfare-estimation error uniformly over seed sets, separating structural, statistical, and simulation errors; with an approximation guarantee for the optimizer, the bound transfers to the selected seeds. Experiments on real network
topologies with simulated outcomes and a synthetic
benchmark show that CIM attains the highest mean welfare among the compared methods.
Read the full abstract ▾
All authors Renjie Cao, Zhuoxin Yan, Zichen Wang, Xinyan Su, Zhiheng Zhang
Machine Learning (Statistics) · Cross-listed from cs.LG
Exploiting Expertise of Non-Expert and Diverse Agents in Social Bandit Learning: A Free Energy Approach
Erfan Mirzaei, Seyed Pooya Shariatpanahi, Alireza Tavakoli et al. (+2 more)
Personalized AI-based services involve a population of individual reinforcement learning
agents. However, most reinforcement learning
algorithms focus on harnessing individual learning and fail to leverage the social learning capabilities commonly exhibited by humans and animals. Social learning integrates individual experience with observing others' behavior, presenting opportunities for improved learning outcomes. In this study, we focus on a social bandit learning scenario where a social
agent observes other
agents' actions without knowledge of their rewards. The
agents independently pursue their own policy without explicit motivation to teach each other. We propose a free energy-based social bandit learning
algorithm over the policy space, where the social
agent evaluates others' expertise levels without resorting to any oracle or social norms. Accordingly, the social
agent integrates its direct experiences in the environment and others' estimated policies. The theoretical convergence of our
algorithm to the optimal policy is proven. Empirical evaluations validate the superiority of our social learning method over alternative approaches in various scenarios. Our
algorithm strategically identifies the relevant
agents, even in the presence of random or suboptimal
agents, and skillfully exploits their behavioral information. In addition to societies including expert
agents, in the presence of relevant but non-expert
agents, our
algorithm significantly enhances individual learning performance, where most related methods fail. Importantly, it also maintains logarithmic regret.
Read the full abstract ▾
All authors Erfan Mirzaei, Seyed Pooya Shariatpanahi, Alireza Tavakoli, Reshad Hosseini, Majid Nili Ahmadabadi
Statistical Methodology
Including historical control data in simultaneous inference for pre-clinical multi-arm studies
Max Menssen, Carsten Kneuer, Gyamfi Akyianu et al. (+3 more)
In pre- and non-clinical toxicology, the reduction of animal use is highly desireable. Although approaches for possible sample size reduction in the concurrent control group were suggested previously under the virtual control groups framework for continuous endpoints, methodology that is applicable to binary outcomes that occur in long-term carcinogenicity studies is currently missing. In order to augment animals in the current control group with historical control data, we propose approaches that rely on dynamic
Bayesian borrowing and simultaneous credible intervals for risk ratios. Several operation characteristics such as familywise error rate (FWER) and power are assessed via Monte-Carlo simulations and compared to the ones of approaches that rely on pooling of historical and current observations. It turned out that under optimal conditions,
Bayesian approaches based on robustified prior distributions enable a substantial reduction of the control groups sample size, while still controlling the FWER up to a satisfactory level. Furthermore, at least to some extend, these approaches were able to protect against possible drift. This hightlights the potential of
Bayesian study designs to reduce animal use in toxicology through re-use of the large pool of existing control data.
Read the full abstract ▾
All authors Max Menssen, Carsten Kneuer, Gyamfi Akyianu, Christian Röver, Tim Friede, Frank Schaarschmidt
Statistical Methodology
A Semiparametric Nonlinear Mixed Effects Model with Penalized Splines Using Automatic Differentiation
Matteo D'Alessandro, Magne Thoresen and Øystein Sørensen
We present an estimation procedure for nonlinear mixed-effects models in which the population trajectory is represented by penalized splines and adapted to individuals via subject-specific transformation parameters. By exploiting the mixed model representation of penalized splines, the level of smoothness can be estimated jointly with other variance components. The integration over random effects needed to obtain the marginal likelihood is carried out using the Laplace approximation. Exact derivatives for evaluation and maximization of the resulting likelihood are obtained via automatic differentiation implemented through Template Model Builder. In simulation studies, the method produces improved inferential performance and reduced computational burden when compared to the existing procedures. The approach is further illustrated through a case study on infant height growth in the first two years of life.
Read the full abstract ▾
Statistical Methodology
Cross-Validation in Bipartite Networks
Bokai Yang, Yuanxing Chen and Yuhong Yang
Bipartite networks, which encode interactions between two distinct types of entities, arise widely in applications and exhibit inherent asymmetry across node sets. Despite a growing literature on bipartite community detection, estimating community numbers $(K_1, K_2)$, a critical issue for bipartite network analysis, remains theoretically underdeveloped without any model selection consistency established, to our knowledge. Indeed, the inherent asymmetry and the two-dimensional parameter space with possibly drastically different $K_1$ and $K_2$ pose unique challenges that differ from unipartite cases. In particular, the candidate models may simultaneously overfit one node set while underfitting the other. To address these challenges, we propose Bipartite Cross-Validation (BCV), a penalized cross-validation framework that jointly selects $(K_1,K_2)$ in a fully data-driven manner. We establish the first model selection consistency for bipartite networks, notably accommodating the regime where the numbers of communities scale with the network size, revealing the intricate interplay between sparsity and model complexity. Simulations and real-data applications demonstrate strong finite-sample performance of BCV.
Read the full abstract ▾
Statistical Methodology
Effective Degrees of Freedom for Balanced Repeated Replication and Paired Jackknife Variance Estimates: A Unified Approach via Stratum Contrasts
Matthias von Davier
Balanced repeated replication (BRR) and the jackknife are two widely used methods for estimating variances in stratified samples with two primary sampling units per stratum. While both methods produce variance
estimators that can be expressed as sums of squared stratum-level contrasts, they differ fundamentally in their construction and in the dependence structure of their replicate estimates. This article examines the independence properties of the components contributing to these variance
estimators. For BRR, we show that although the replicate estimates themselves are correlated, the balancing property of Hadamard
matrices collapses the variance
estimator into a sum of independent stratum-specific components. For the jackknife, the independence of components follows directly from the construction. Using these independence results, we derive the variance of each variance
estimator and establish a direct connection to the Welch-Satterthwaite degrees of freedom approximation. This yields a practical formula for estimating degrees of freedom when constructing confidence intervals for population totals. The derivation highlights the unified treatment of both replication methods and provides insights into their relative efficiency and applicability.
Read the full abstract ▾
Machine Learning (Statistics)
Decomposing Observational Multiplicity in Decision Trees: Leaf and Structural Regret
Mustafa Cavus
Many machine learning tasks admit multiple models that perform almost equally well, a phenomenon known as predictive multiplicity. A fundamental source of this multiplicity is observational multiplicity, which arises from the
stochastic nature of label collection: observed training labels represent only a single realization of the underlying ground-truth probabilities. While theoretical frameworks for observational multiplicity have been established for logistic regression, their implications for non-smooth, partition-based models like decision trees remain underexplored. In this paper, we introduce two complementary notions of observational multiplicity for decision tree classifiers: leaf regret and structural regret. Leaf regret quantifies the intrinsic variability of predictions within a fixed leaf due to finite-sample noise, while structural regret captures variability induced by the instability of the learned tree structure itself. We provide a formal decomposition of observational multiplicity into these two components and establish statistical guarantees. Our experimental evaluation across diverse credit risk scoring
datasets confirms the near-perfect
alignment between our theoretical decomposition and the empirically observed variance. Notably, we find that structural regret is the primary driver of observational multiplicity, accounting for over 15 times the variability of leaf regret in some
datasets. Furthermore, we demonstrate that utilizing these regret measures as an abstention mechanism in selective prediction can effectively identify arbitrary regions and improve model safety, elevating recall from 92% to 100% on the most stable sub-populations. These results establish a rigorous framework for quantifying observational multiplicity, aligning with recent advances in algorithmic safety and interpretability.
Read the full abstract ▾
Statistics Applications
On the Unit Teissier Distribution: Properties, Estimation Procedures and Applications
Zuber Akhter, Mohamed A. Abdelaziz, M.Z. Anis et al. (+1 more)
The Teissier distribution, originally proposed by Teissier [31], was designed to model mortality due to aging in domestic animals. More recently, Krishna et al. [19] introduced the Unit Teissier (UT) distribution on the interval (0, 1) through the transformation $X=e^{-Y}$, where $Y$ follows the Teissier distribution. In their work, the authors derived several fundamental properties of the UT distribution and investigated parameter estimation using maximum likelihood, least squares, weighted least squares and
Bayesian methods. Building upon this work, the present paper develops additional theoretical and inferential results for the UT distribution. In particular, closed-form expressions for single moments of order statistics and L-moments are obtained, and characterization results based on truncated moments are established. Furthermore, several alternative parameter estimation methods are considered, including maximum product of spacings, Cramér-von Mises, Anderson-Darling, right-tail Anderson-Darling, percentile and L-moment estimation, while the estimation methods previously studied by Krishna et al. [19] are also included for comparison. Extensive simulation studies under various parameter settings and sample sizes are conducted to assess and compare the performance of the
estimators. Finally, the flexibility and practical utility of the UT distribution are demonstrated using a real
dataset.
Read the full abstract ▾
All authors Zuber Akhter, Mohamed A. Abdelaziz, M.Z. Anis, Ahmed Z. Afify
Statistics Applications
One-Shot Individual Claims Reserving
Ronald Richman and Mario V. Wüthrich
Individual claims reserving has not yet become established in actuarial practice. We attribute this to the absence of a satisfactory methodology: existing approaches tend to be either overly complex or insufficiently flexible and robust for practical use. Building on the classical chain-ladder (CL) method, we introduced a new perspective on individual claims reserving in Richman and Wüthrich [arXiv:2602.15385]. This manuscript has sparked considerable discussion within the actuarial community. The aim of the present paper is to continue and deepen that discussion, with the ultimate goal of advancing toward a new standard for micro-level reserving.
Read the full abstract ▾
Statistical Methodology · Cross-listed from math.OC
Simultaneous estimation of multiple discrete unimodal distributions under stochastic order constraints
Yasuhiro Yoshida, Noriyoshi Sukegawa and Jiro Iwanaga
We study the problem of estimating multiple discrete unimodal distributions, motivated by search behavior analysis on a real-world platform. To incorporate prior knowledge of precedence relations among distributions, we impose
stochastic order constraints and formulate the estimation task as a mixed-integer convex quadratic optimization problem. Experiments on both synthetic and real
datasets show that the proposed method reduces the Jensen-Shannon divergence by 2.2% on average (up to 6.3%) when the sample size is small, while performing comparably to existing methods when sufficient data are available.
Read the full abstract ▾
Statistical Methodology
Robust Joint Modeling for Data with Continuous and Binary Responses
Yu Wang, Ran Jin and Lulu Kang
In many supervised learning applications, the response consists of both continuous and binary outcomes. Studies have shown that jointly modeling such mixed-type responses can substantially improve predictive performance compared to separate analyses. But outliers pose a new challenge to the existing likelihood-based modeling approaches. In this paper, we propose a new robust joint modeling framework for data with both continuous and binary responses. It is based on the density power divergence (DPD) loss function with the $\ell_1$ regularization. The proposed framework leads to a sparse
estimator that simultaneously predicts continuous and binary responses in high-dimensional input settings while down-weighting influential outliers and mislabeled samples. We also develop an efficient proximal gradient
algorithm with Barzilai-Borwein spectral step size and a robust information criterion (RIC) for data-driven selection of the penalty parameters. Extensive simulation studies under a variety of contamination schemes demonstrate that the proposed method achieves lower prediction error and more accurate parameter estimation than several competing approaches. A real case study on wafer lapping in
semiconductor manufacturing further illustrates the practical gains in predictive accuracy, robustness, and interpretability of the proposed framework.
Read the full abstract ▾
Statistical Methodology · Cross-listed from econ.EM
Variance Estimation with Dependence and Heterogeneous Means
Luther Yap
This paper develops a framework for variance estimation under dependence and heterogeneous means. This paper shows that consistent estimation of the variance target is impossible in general, and characterizes necessary and sufficient conditions for conservative variance estimation using dual cones. To choose among the valid
estimators, this paper formulates three criteria – minimal correction, pointwise level estimand, and pointwise MSE – and shows how an
eigenvalue truncation solution is optimal under all three criteria. This characterization and solution allow us to assess if existing variance
estimators are valid and optimal in their respective settings, and construct the first optimal variance
estimator that is simultaneously robust to heterogeneous means and cross-cluster serial correlation.
Read the full abstract ▾
Statistical Methodology
Graph Generation Methods under Partial Information
Tong Sun, Jianshu Hao, Michael C. Fu et al. (+1 more)
We study the problem of generating graphs with prescribed degree sequences for bipartite, directed, and undirected networks. We first propose a sequential method for bipartite graph generation and establish a necessary and sufficient interval condition that characterizes the admissible number of connections at each step, thereby guaranteeing global feasibility. Based on this result, we develop bipartite graph enumeration and sampling
algorithms suitable for different problem sizes. We then extend these bipartite graph
algorithms to the directed and undirected cases by incorporating additional connection constraints, as well as feasibility verification and symmetric connection steps, while preserving the same algorithmic principles. Finally, numerical experiments demonstrate the performance of the proposed
algorithms, particularly their scalability to large instances where existing methods become computationally prohibitive.
Read the full abstract ▾
All authors Tong Sun, Jianshu Hao, Michael C. Fu, Guangxin Jiang
Statistical Methodology
Genya Kobayashi, Shonosuke Sugasawa, Yuta Yamauchi et al. (+1 more)
This paper proposes dynamic
Bayesian regression
quantile synthesis (DRQS), a novel method for
quantile forecasting within the
Bayesian predictive synthesis (BPS) framework designed to combine
quantile-specific information from multiple
agent models. While existing BPS approaches primarily focus on mean
forecasting, our method directly targets the conditional
quantiles of the response variable by utilizing the asymmetric Laplace distribution for the synthesis function. The resulting framework can be interpreted as a dynamic
quantile linear model with latent predictors. We extend the univariate DRQS to a multivariate setting-factor DRQS (FDRQS)-by introducing a time-varying latent factor structure for the synthesis weights. This allows the model to leverage cross-sectional dependencies and shared information across multiple time series simultaneously. We develop an efficient Markov chain Monte Carlo (MCMC)
algorithm for posterior inference, utilizing data augmentation and forward-filtering backward-sampling. Empirical applications to US inflation and global GDP growth demonstrate the improved performance of the proposed methods for
quantile forecasting. In particular, FDRQS exhibits superior resilience during periods of extreme economic stress, such as the COVID-19 pandemic, by adaptively rebalancing
agent contributions and capturing emergent global dependencies.
Read the full abstract ▾
All authors Genya Kobayashi, Shonosuke Sugasawa, Yuta Yamauchi, Dongu Han
Statistical Methodology
Prediction-Oriented Transfer Learning for Survival Analysis
Yu Gu, Donglin Zeng and D. Y. Lin
Transfer learning is beneficial for survival analysis, especially when the target study has a limited number of events. However, existing transfer learning methods rely on the restrictive assumption that the target and source studies share similar parameters under Cox models, and most require access to individual-level source data. In this article, we propose a novel transfer learning framework that enhances model-based survival prediction by transferring predictive rather than distributional knowledge from source studies. Our approach employs flexible semiparametric transformation models for the target data while eliminating the need to model or share the source data. The ingeniously designed penalty enables simple and stable computation via an EM
algorithm. We rigorously establish the asymptotic properties of the proposed
estimator and show that it achieves a faster convergence rate than the target-only
estimator when source knowledge is sufficiently accurate. We demonstrate the advantages of our methods through extensive simulation studies and an application to two major breast cancer studies, with the objective of improving survival prediction in a target cohort using predictive information from an external study.
Read the full abstract ▾
Statistical Methodology
Bayesian Modular Inference for Copula Models with Potentially Misspecified Marginals
Lucas Kock, David T. Frazier, Michael Stanley Smith et al. (+1 more)
Copula models of multivariate data are popular because they allow separate specification of marginal distributions and the copula function. These components can be treated as inter-related modules in a modified
Bayesian inference approach called ”cutting feedback” that is robust to their misspecification. Recent work uses a two module approach, where all $d$ marginals form a single module, to robustify inference for the marginals against copula function misspecification, or vice versa. However, marginals can exhibit differing levels of misspecification, making it attractive to assign each its own module with an individual influence parameter controlling its contribution to a joint semi-modular inference (SMI) posterior. This generalizes existing two module SMI methods, which interpolate between cut and conventional posteriors using a single influence parameter. We develop a novel copula SMI method and select the influence parameters using
Bayesian optimization. It provides an efficient continuous relaxation of the discrete optimization problem over $2^d$ cut/uncut configurations. We establish theoretical properties of the resulting semi-modular posterior and demonstrate the approach on simulated and real data. The real data application uses a skew-normal copula model of asymmetric dependence between equity
volatility and bond yields, where robustifying copula estimation against marginal misspecification is strongly motivated.
Read the full abstract ▾
All authors Lucas Kock, David T. Frazier, Michael Stanley Smith, David J. Nott