Machine Learning (Statistics) · Cross-listed from cs.LG
Time series forecasting with Hahn Kolmogorov-Arnold networks
Md Zahidul Hasan, A. Ben Hamza and Nizar Bouguila
Recent
Transformer- and MLP-based models have demonstrated strong performance in long-term time series
forecasting, yet
Transformers remain limited by their quadratic complexity and permutation-equivariant
attention, while MLPs exhibit spectral bias. We propose HaKAN, a versatile model based on Kolmogorov-Arnold Networks (KANs), leveraging Hahn
polynomial-based learnable activation functions and providing a lightweight and interpretable alternative for multivariate time series
forecasting. Our model integrates channel independence, patching, a stack of Hahn-KAN blocks with residual connections, and a bottleneck structure comprised of two fully connected layers. The Hahn-KAN block consists of inter- and intra-patch KAN layers to effectively capture both global and local temporal patterns. Extensive experiments on various
forecasting benchmarks demonstrate that our model consistently outperforms recent state-of-the-art methods, with ablation studies validating the effectiveness of its core components.
Read the full abstract ▾
Statistical Methodology
Examining the Efficacy of Coarsened Exact Matching as an Alternative to Propensity Score Matching
Fei Wan
Coarsened exact matching (CEM) is often promoted as a superior alternative to propensity score matching (PSM) for addressing imbalance, model dependence, bias, and efficiency. However, this recommendation remains uncertain. First, CEM is commonly mischaracterized as exact matching, despite relying on coarsened rather than original variables. This inexactness in matching introduces residual confounding, which necessitates accurate modeling of the outcome-confounder relationship post-matching to mitigate bias, thereby increasing vulnerability to model misspecification. Second, prior studies overlook that any imbalance between treated and untreated subjects matched on the same propensity score is attributable to random variation. Thus, claims that CEM outperforms PSM in reducing imbalance are unfounded, particularly when using metrics like Mahalanobis distance, which do not account for chance imbalance in PSM. Our simulations show that PSM reduces imbalance more effectively than CEM when evaluated with multivariate standardized mean differences (SMD), and unadjusted analyses indicate greater bias with CEM. While adjusted analyses in both CEM with autocoarsening and PSM may perform similarly when matching on few variables, CEM suffers from the curse of dimensionality as the number of factors increases, resulting in substantial data loss and unstable estimates. Increasing the level of coarsening may mitigate data loss but exacerbates residual confounding and model dependence. In contrast, both analytical results and simulations demonstrate that PSM is more robust to model misspecification and thus less model-dependent. Therefore, CEM is not a viable alternative to PSM when matching on a large number of
covariates.
Read the full abstract ▾
Machine Learning (Statistics)
A Cherry-Picking Approach to Large Load Shaping for More Effective Carbon Reduction
Bokan Chen, Raiden Hasegawa, Adriaan Hilbers et al. (+7 more)
Shaping multi-megawatt loads, such as data centers, impacts generator dispatch on the electric grid, which in turn affects system CO2 emissions and energy cost. Substantiating the effectiveness of prevalent load shaping strategies, such as those based on grid-level average carbon intensity, locational marginal price, or marginal emissions, is challenging due to the lack of detailed counterfactual data required for accurate attribution. This study uses a series of calibrated granular ERCOT day-ahead direct current optimal power flow (DC-OPF) simulations for counterfactual analysis of a broad set of load shaping strategies on grid CO2 emissions and cost of electricity. In terms of annual grid level CO2 emissions reductions, LMP-based shaping outperforms other common strategies, but can be significantly improved upon. Examining the performance of practicable strategies under different grid conditions motivates a more effective load shaping approach: one that "cherry-picks" a daily strategy based on observable grid signals and historical data. The cherry-picking approach to power load shaping is applicable to any large flexible consumer on the electricity grid, such as data centers, distributed energy resources and Virtual Power Plants (VPPs).
Read the full abstract ▾
All authors Bokan Chen, Raiden Hasegawa, Adriaan Hilbers, Ross Koningstein, Ana Radovanović, Utkarsh Shah, Gabriela Volpato, Mohamed Ahmed, Tim Cary, Rod Frowd
Statistics Applications
Bayesian Multiple Testing for Suicide Risk in Pharmacoepidemiology: Leveraging Co-Prescription Patterns
Soumya Sahu, Kwan Hur, Dulal K. Bhaumik et al. (+1 more)
Suicide is the tenth leading cause of death in the United States, yet evidence on medication-related risk or protection remains limited. Most post-marketing studies examine one drug class at a time or rely on empirical-Bayes shrinkage with conservative multiplicity corrections, sacrificing power to detect clinically meaningful signals. We introduce a unified
Bayesian spike-and-slab framework that advances both applied suicide research and statistical methodology. Substantively, we screen 922 prescription drugs across 150 million patients in U.S. commercial claims (2003 to 2014), leveraging real-world co-prescription patterns to inform a covariance prior that adaptively borrows strength across pharmacologically related
agents. Statistically, the model couples this structured prior with
Bayesian false-discovery-rate control, illustrating how network-guided variable selection can improve rare-event surveillance in high dimensions. Relative to the seminal empirical-Bayes analysis of Gibbons et al. (2019), our approach reconfirms the key harmful (e.g., alprazolam, hydrocodone) and protective (e.g., mirtazapine, folic acid) signals while revealing additional associations, such as a high-risk opioid combination and several folate-linked
agents with potential preventive benefit that had been overlooked. A focused re-analysis of 18 antidepressants shows how alternative co-prescription metrics modulate effect estimates, shedding light on competitive versus complementary prescribing. These findings generate actionable hypotheses for clinicians and regulators and showcase the value of structured
Bayesian modeling in pharmacovigilance.
Read the full abstract ▾
All authors Soumya Sahu, Kwan Hur, Dulal K. Bhaumik, Robert Gibbons
Machine Learning (Statistics)
Boosting methods for interval-censored data with regression and classification
Yuan Bian, Grace Y. Yi and Wenqing He
Boosting has garnered significant interest across both machine learning and statistical communities. Traditional boosting
algorithms, designed for fully observed random samples, often struggle with real-world problems, particularly with interval-censored data. This type of data is common in survival analysis and time-to-event studies where exact event times are unobserved but fall within known intervals. Effective handling of such data is crucial in fields like medical research, reliability engineering, and social sciences. In this work, we introduce novel nonparametric boosting methods for regression and classification tasks with interval-censored data. Our approaches leverage censoring unbiased transformations to adjust loss functions and impute transformed responses while maintaining model accuracy. Implemented via functional gradient descent, these methods ensure scalability and adaptability. We rigorously establish their theoretical properties, including optimality and mean squared error trade-offs. Our proposed methods not only offer a robust framework for enhancing predictive accuracy in domains where interval-censored data are common but also complement existing work, expanding the applicability of existing boosting techniques. Empirical studies demonstrate robust performance across various finite-sample scenarios, highlighting the practical utility of our approaches.
Read the full abstract ▾
Statistical Methodology
Transportability of Regression Calibration with External Validation Studies for Measurement Error Correction
Zexiang Li, Donna Spiegelman, Molin Wang et al. (+2 more)
In nutritional and environmental epidemiology, exposures are impractical to measure accurately, while practical measures for these exposures are often subject to substantial measurement error. Regression calibration is among the most used measurement error correction methods with external validation studies. The use of external studies to assess the measurement error process always carries the risk of introducing estimation bias into the main study analysis. Although the transportability of regression calibration is usually assumed for practical epidemiology studies, it has not been well studied. In this work, under the measurement error process with a mixture of Berkson-like and classical-like errors, we investigate conditions under which the effect estimate from regression calibration with an external validation study is unbiased for the association between exposure and health outcome. We further examine departures from the transportability assumption, under which the regression calibration
estimator is itself biased. However, we theoretically prove that, in most cases, it yields lower bias than the naive method. The derived conditions are confirmed through simulation studies and further verified in an example investigating the association between the risk of cardiovascular disease and moderate physical activity in the health professional follow-up study.
Read the full abstract ▾
All authors Zexiang Li, Donna Spiegelman, Molin Wang, Zuoheng Wang, Xin Zhou
Machine Learning (Statistics) · Cross-listed from cs.CV
Feature-Space Generative Models for One-Shot Class-Incremental Learning
Jack Foster, Kirill Paramonov, Mete Ozay et al. (+1 more)
Few-shot class-incremental learning (FSCIL) is a paradigm where a model, initially trained on a
dataset of base classes, must adapt to an expanding problem space by recognizing novel classes with limited data. We focus on the challenging FSCIL setup where a model receives only a single sample (1-shot) for each novel class and no further training or model alterations are allowed after the base training phase. This makes generalization to novel classes particularly difficult. We propose a novel approach predicated on the hypothesis that base and novel class
embeddings have structural similarity. We map the original
embedding space into a residual space by subtracting the class prototype (i.e., the average class
embedding) of input samples. Then, we leverage generative modeling with VAE or
diffusion models to learn the multi-modal distribution of residuals over the base classes, and we use this as a valuable structural prior to improve recognition of novel classes. Our approach, Gen1S, consistently improves novel class recognition over the state of the art across multiple
benchmarks and backbone architectures.
Read the full abstract ▾
All authors Jack Foster, Kirill Paramonov, Mete Ozay, Umberto Michieli
Statistics Theory
The Hellinger Bounds on the Kullback-Leibler Divergence and the Bernstein Norm
Tetsuya Kaji
The Kullback-Leibler divergence, the Kullback-Leibler variation, and the Bernstein "norm" are used to quantify discrepancies among probability distributions in likelihood models such as nonparametric maximum likelihood and nonparametric Bayes. They are closely related to the Hellinger distance, which is often easier to work with. Consequently, it is of interest to characterize conditions under which the Hellinger distance serves as an upper bound for these measures. This article characterizes a necessary and sufficient condition for each of the discrepancy measures to be bounded by the Hellinger distance. It accommodates unbounded likelihood ratios and generalizes all previously known results. We then apply it to relax the regularity condition for the sieve maximum likelihood
estimator.
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.DC
Load Balancing with Evolving Workloads in Large-Scale Language Model Serving
Zixi Chen, Tianci Bu, Chendong Song et al. (+3 more)
Serving large
language models requires assigning requests whose workloads grow during processing and whose completion times are unknown. With a synchronization barrier, lightly loaded workers wait for the slowest worker at every decode iteration. We model this loss through aggregate workload imbalance and formulate a joint admission and assignment problem with sticky requests and capacity limits. Short lookahead incorporates predicted completions of ongoing requests. For exact optimization without lookahead, we prove a comparison under independent
geometric decode lengths, bounded independent nondegenerate
prompt sizes, and batched releases satisfying a cumulative supply condition. The long-run expected imbalance improvement over first come, first served admission with request-count join-shortest-queue grows at least as the square root of the product of worker capacity and the logarithm of worker count, in the stated scaling regime. The comparison extends to capacity-aware round robin, random routing, and power of two choices. BalanceRoute approximations connect the formulation to practical routing. Experiments using up to 144 NPUs with proprietary and public traces show lower workload spread and higher throughput than the evaluated baselines. These empirical results complement the guarantee for exact optimization.
Read the full abstract ▾
All authors Zixi Chen, Tianci Bu, Chendong Song, Xin Lu, Yinyu Ye, Zijie Zhou
Statistical Methodology · Cross-listed from econ.EM
Best Feasible Conditional Critical Values for a More Powerful Subvector Anderson-Rubin Test
Jesse Hoekstra and Frank Windmeijer
For subvector inference in the linear instrumental variables model under homoskedasticity but allowing for weak instruments, Guggenberger, Kleibergen and Mavroeidis (2019) (GKM) propose a conditional subvector Anderson and Rubin (1949) (AR) test that uses data-dependent critical values that adapt to the strength of the parameters not under test. This test has correct size and strictly higher power than the test that uses standard asymptotic chi-square critical values. The subvector AR test is the minimum
eigenvalue of a data-dependent
matrix. The GKM critical value function conditions on the largest
eigenvalue of this
matrix. Instead, this paper considers a data-dependent critical value function that conditions on the second-smallest
eigenvalue, as this
eigenvalue is the appropriate indicator of weak identification. We find that the data-dependent critical value function of GKM also applies to this conditioning and show that this test has correct size and power strictly higher than the GKM test when the number of parameters not under test is larger than one. This is the best feasible test, where we consider feasible to be within the class of conditioning on a single
eigenvalue. Our proposed procedure further applies to the subvector AR test statistic that is robust to an approximate Kronecker product structure of conditional heteroskedasticity as proposed by Guggenberger, Kleibergen and Mavroeidis (2024), carrying over its power advantage to this setting as well.
Read the full abstract ▾
Statistics Theory
On the consistency of the posterior distribution for nonlinear PDE parameter identification
Yuxin Fan and Bangti Jin
In this work, we investigate the estimation of a parameter in PDEs using
Bayesian procedures, and focus on posterior distributions constructed using Gaussian process priors, and its variational approximation. We establish contraction rates for the posterior distribution and the variational approximation in the regime of low-regularity parameters. Specifically, the ground truth is only assumed to belong to the support space of the prior rather than to its associated reproducing kernel
Hilbert space. Also we derive contraction rates for the posterior distribution constructed using the randomly truncated Gaussian process prior. The analysis relies on a delicate approximation argument that approximates low-regularity ground truths by suitable elements in the reproducing kernel
Hilbert space and balances various error sources. We illustrate the general theory on three nonlinear inverse problems for PDEs.
Read the full abstract ▾
Statistical Methodology
Sensitivity analysis for incremental effects, with application to a study of victimization & offending
Shuying Shen, Valerio Bacak and Edward H. Kennedy
Sensitivity analysis for unmeasured confounding under incremental propensity score interventions remains relatively underdeveloped. Incremental interventions define
stochastic treatment regimes by multiplying the odds of treatment, offering a flexible framework for causal effect estimation. To study incremental effects when there are unobserved confounders, we adopt Rosenbaum's sensitivity model in single time point settings, and propose a doubly robust
estimator for the resulting effect bounds. The bound
estimators are asymptotically normal under mild conditions on nuisance function estimation. We show that incremental effect bounds can be narrower or wider than those for mean potential outcomes, and that the bounds must lie between the expected minimum and maximum of the conditional bounds on E(Y^0|X) and E(Y^1|X). For time-varying treatments, we consider the marginal sensitivity model. Although sharp bounds for incremental effects are identifiable from longitudinal data under this model, practical
estimators have not yet been established; we discuss this challenge and provide partial results toward implementation. Finally, we apply our methods to study the effect of victimization on subsequent offending using data from the National Longitudinal Study of Adolescent to Adult Health (Add Health), illustrating the robustness of our findings in an empirical setting.
Read the full abstract ▾
Statistical Methodology
Group Permutation Testing in Linear Model: Sharp Validity, Power Improvement, and Extension Beyond Exchangeability
Zonghan Li, Hongyi Zhou and Zhiheng Zhang
We consider finite-sample inference for a single regression coefficient in the fixed-design linear model $Y = Z\beta + bX + \varepsilon$, where $\varepsilon\in\mathbb{R}^n$ may exhibit complex dependence or heterogeneity. We develop a group permutation framework, yielding a unified and analyzable randomization structure for linear-model testing. Under exchangeable errors, we place permutation-augmented regression tests within this group-theoretic setting and show that a grouped version of PALMRT controls Type I error at level at most $2\alpha$ for any permutation group; moreover, we provide an worst-case construction demonstrating that the factor $2$ is sharp and cannot be improved without additional assumptions. Second, we relate the Type II error to a design-dependent
geometric separation. We formulate it as a combinatorial optimization problem over permutation groups and bound it under additional mild sub-Gaussian assumptions. For the Type II error upper bound control, we propose a constructive
algorithm for the permutation strategy that is better (at least no worse) than the i.i.d. permutation, with simulations empirically indicating substantial power gains, especially under heavy-tailed designs. Finally, we extend group-based CPT and PALMRT beyond exchangeability by connecting rank-based randomization arguments to conformal inference. The resulting weighted group tests satisfy finite-sample Type I error bounds that degrade gracefully with a weighted average of total variation distances between $\varepsilon$ and its group-permuted versions, recovering exact validity when these discrepancies vanish and yielding quantitative robustness otherwise. Taken together, the group-permutation viewpoint provides a principled bridge from exact randomization validity to design-adaptive power and quantitative robustness under approximate
symmetries.
Read the full abstract ▾
Statistical Methodology · Cross-listed from econ.EM
The Proximal Surrogate Index: Long-Term Treatment Effects under Unobserved Confounding
Ting-Chih Hung and Yu-Chang Chen
We study the identification and estimation of long-term treatment effects under unobserved confounding by combining an experimental sample, where the long-term outcome is missing, with an observational sample, where the treatment assignment is unobserved. While standard surrogate index methods fail when unobserved confounders exist, we establish novel identification results by leveraging proxy variables for the unobserved confounders. We further develop multiply robust estimation and inference procedures based on these results. Applying our method to the Job Corps program, we demonstrate its ability to recover experimental
benchmarks even when unobserved confounders bias standard surrogate index estimates.
Read the full abstract ▾
Statistical Methodology
Bidirectional causal inference for binary outcomes in the presence of unmeasured confounding
Yafang Deng, Kang Shuai and Shanshan Luo
Bidirectional causal relationships arising from mutual interactions between variables are commonly observed within biomedical, econometrical, and social science contexts. When such relationships are further complicated by unobserved factors, identifying causal effects in both directions becomes especially challenging. For continuous variables, methods that utilize two instrumental variables from both directions have been proposed to explore bidirectional causal effects within linear models. However, the existing techniques are not immediately applicable when the key variables of interest are binary. To address these issues, we propose a structural equation modeling approach that links observed binary variables to latent continuous variables through a constrained mapping. We firstly establish the identification results for bidirectional causal effects using a pair of instrumental variables. Then, we develop an estimation method for the corresponding causal parameters. We also conduct sensitivity analysis under scenarios where certain identification conditions are violated. Finally, we apply our approach to investigate the bidirectional causal relationship between heart disease and diabetes, demonstrating its practical utility in biomedical researches.
Read the full abstract ▾
Statistics Theory · Cross-listed from cs.LG
A Mosco sufficient condition for intrinsic stability of non-unique convex Empirical Risk Minimization
Karim Bounja, Lahcen Laayouni and Abdeljalil Sakat
Empirical risk minimization (ERM) stability is usually studied via single-valued outputs, while convex non-strict losses yield set-valued minimizers. We identify Painlevé-Kuratowski upper semicontinuity (PK-u.s.c.) as the intrinsic stability notion for the ERM solution correspondence (set-level Hadamard well-posedness) and a prerequisite to interpret stability of selections. We then characterize a minimal non-degenerate qualitative regime: Mosco-consistent perturbations and locally bounded minimizers imply PK-u.s.c., minimal-value continuity, and consistency of vanishing-gap near-minimizers. Quadratic growth yields explicit quantitative deviation bounds.
Read the full abstract ▾