Papers

While continuous diffusion has shown remarkable success in continuous domains such as image generation, its direct application to discrete data has underperformed pure discrete formulations. To understand this gap, we introduce token identifiability, an analytical framework characterizing how Gaussian noise corrupts discrete data through two mechanisms: discrete identity corruption and continuous rank degradation. We reveal that these mechanisms scale differently with vocabulary size, creating a temporal dissonance that forces a tradeoff between learning continuous geometry and discrete structure. To address this, we propose CANDI (Continuous ANd DIscrete diffusion), a hybrid framework that decouples discrete and continuous corruption, enabling simultaneous learning of both. This unlocks the benefits of continuous diffusion for discrete spaces: on controlled generation, CANDI enables classifier-based guidance with off-the-shelf classifiers through simple gradient addition; on text generation, CANDI outperforms masked diffusion at low NFE, demonstrating the value of learning continuous gradients for discrete spaces. We include the code on the project page: https://patrickpynadath1.github.io/candi-lander.

We study anomaly detection in images under a fixed-camera environment and propose a doubly smoothed (DS) density estimator that exploits spatial structure to improve estimation accuracy. The DS estimator applies kernel smoothing twice: first over the value domain to obtain location-wise classical nonparametric density (CD) estimates, and then over the spatial domain to borrow information from neighboring locations. Under appropriate regularity conditions, we show that the DS estimator achieves smaller asymptotic bias, variance, and mean squared error than the CD estimator. To address the increased computational cost of the DS estimator, we introduce a grid point approximation (GPA) technique that reduces the computation cost of inference without sacrificing the estimation accuracy. A rule-of-thumb bandwidth is derived for practical use. Extensive simulations show that GPA-DS achieves the lowest MSE with near real-time speed. In a large-scale case study on underground mine surveillance, GPA-DS enables remarkable sub-image extraction of anomalous regions after which a lightweight MobileNet classifier achieves $\approx$99% out-of-sample accuracy for unsafe act detection.

Quantum imaginary time evolution (QITE) algorithm is one of the most promising variational quantum algorithms (VQAs), bridging the current era of Noisy Intermediate-Scale Quantum devices and the future of fully fault-tolerant quantum computing. Although practical demonstrations of QITE and its potential advantages over the general VQA trained with vanilla gradient descent (GD) in certain tasks have been reported, a first-principle, theoretical understanding of QITE remains limited. Here, we aim to develop an analytic theory for the dynamics of QITE. First, we show that QITE can be interpreted as a form of a general VQA trained with Quantum Natural Gradient Descent (QNGD), where the inverse quantum Fisher information matrix serves as the learning-rate tensor. This equivalence is established not only at the level of gradient update rules, but also through the action principle: the variational principle can be directly connected to the geometric geodesic distance in the quantum Fisher information metric, up to an integration constant. Second, for wide quantum neural networks, we employ the quantum neural tangent kernel framework to construct an analytic model for QITE. We prove that QITE always converges faster than GD-based VQA, though this advantage is suppressed by the exponential growth of Hilbert space dimension. This helps explain certain experimental results in quantum computational chemistry. Our theory encompasses linear, quadratic, and more general loss functions. We validate the analytic results through numerical simulations. Our findings establish a theoretical foundation for QITE dynamics and provide analytic insights for the first-principle design of variational quantum algorithms.

Estimating effects of spatially structured exposures is complicated by unmeasured spatial confounders, which undermine identifiability in spatial linear regression models unless structural assumptions are imposed. We develop a general framework for effect estimation in spatial regression models that relaxes the commonly assumed requirement that exposures contain higher-frequency variation than confounders. We propose basis voting, a plurality-rule estimator - novel in the spatial literature - that consistently identifies causal effects only under the assumption that, in a spatial basis expansion of the exposure and confounder, there exist several basis functions in the support of the exposure but not the confounder. This assumption generalizes existing assumptions of differential basis support used for identification of the causal effect under spatial confounding, and does not require prior knowledge of which basis functions satisfy this support condition. We design this estimator as the mode of several candidate estimators each computed based on a single working basis function. We also show that the standard projection-based candidate estimator typically used in other plurality-rule based methods is inefficient, and provide a more efficient novel candidate. Extensive simulations and a real-world application demonstrate that our approach reliably recovers unbiased causal estimates whenever exposure and confounder signals are separable on a plurality of basis functions. By not relying on higher-frequency variation, our method remains applicable to settings where exposures are smooth spatial functions, such as distance to pollution sources or major roadways, common in environmental studies.

Maximum entropy has become a mainstream off-policy reinforcement learning (RL) framework for balancing exploitation and exploration. However, two bottlenecks still limit further performance improvement: (1) non-stationary Q-value estimation caused by jointly injecting entropy and updating its weighting parameter, i.e., temperature; and (2) short-sighted local entropy tuning that adjusts temperature only according to the current single-step entropy, without considering the effect of cumulative entropy over time. In this paper, we extends maximum entropy framework by proposing a trajectory entropy-constrained reinforcement learning (TECRL) framework to address these two challenges. Within this framework, we first separately learn two Q-functions, one associated with reward and the other with entropy, ensuring clean and stable value targets unaffected by temperature updates. Then, the dedicated entropy Q-function, explicitly quantifying the expected cumulative entropy, enables us to enforce a trajectory entropy constraint and consequently control the policy long-term stochasticity. Building on this TECRL framework, we develop a practical off-policy algorithm, DSAC-E, by extending the state-of-the-art distributional soft actor-critic with three refinements (DSAC-T). Empirical results on the OpenAI Gym benchmark demonstrate that our DSAC-E can achieve higher returns and better stability.

The third Bitcoin halving that took place in May 2020 cut down the mining reward from 12.5 to 6.25 BTC per block and thus slowed down the rate of issuance of new Bitcoins, making it more scarce. The fourth and most recent halving happened in April 2024, cutting the block reward further to 3.125 BTC. If the demand did not decrease simultaneously after these halvings, then the neoclassical economic theory posits that the price of Bitcoin should have increased due to the halving. But did it, in fact, increase for that reason, or is this a post hoc fallacy? This paper uses synthetic control to construct a weighted Bitcoin that is different from its counterpart in one aspect - it did not undergo halving. Comparing the price trajectory of the actual and the simulated Bitcoins, I find evidence of a positive effect of the 2024 Bitcoin halving on its price three months later. The magnitude of this effect is one fifth of the total percentage change in the price of Bitcoin during the study period - from April 2, 2023, to July 21, 2024 (17 months). The second part of the study fails to obtain a statistically significant and robust causal estimate of the effect of the 2020 Bitcoin halving on Bitcoin's price. This is the first paper analyzing the effect of halving causally, building on the existing body of correlational research.

This paper develops a dual-channel framework for analyzing technology diffusion that integrates spatial decay mechanisms from continuous functional analysis with network contagion dynamics from spectral graph theory. Building on our previous studies, which establish Navier-Stokes-based approaches to spatial treatment effects and financial network fragility, we demonstrate that technology adoption spreads simultaneously through both geographic proximity and supply chain connections. Using comprehensive data on six technologies adopted by 500 firms over 2010-2023, we document three key findings. First, technology adoption exhibits strong exponential geographic decay with spatial decay rate $\kappa \approx 0.043$ per kilometer, implying a spatial boundary of $d^* \approx 69$ kilometers beyond which spillovers are negligible (R-squared = 0.99). Second, supply chain connections create technology-specific networks whose algebraic connectivity ($\lambda_2$) increases 300-380 percent as adoption spreads, with correlation between $\lambda_2$ and adoption exceeding 0.95 across all technologies. Third, traditional difference-in-differences methods that ignore spatial and network structure exhibit 61 percent bias in estimated treatment effects. An event study around COVID-19 reveals that network fragility increased 24.5 percent post-shock, amplifying treatment effects through supply chain spillovers in a manner analogous to financial contagion documented in our recent study. Our framework provides micro-foundations for technology policy: interventions have spatial reach of 69 kilometers and network amplification factor of 10.8, requiring coordinated geographic and supply chain targeting for optimal effectiveness.

This paper develops a continuous framework for analyzing financial contagion that incorporates both geographic proximity and interbank network linkages. The framework characterizes stress propagation through a master equation whose solution admits a Feynman-Kac representation as expected cumulative stress along stochastic paths through spatial-network space. From this representation, I derive the General Equilibrium Amplification Factor – a structural measure of systemic importance that captures the ratio of total system-wide effects to direct effects following a localized shock. The amplification factor decomposes naturally into spatial, network, and interaction components, revealing which transmission channels contribute most to each institution's systemic importance. The framework nests discrete cascade models as a limiting case when jump intensity becomes infinite above default thresholds, clarifying that continuous and discrete approaches describe different regimes of the same phenomenon. Empirical validation using 38 global banks across the 2008 financial crisis and COVID-19 pandemic demonstrates that the amplification factor correctly identifies systemically important institutions (Pearson correlation $\rho = -0.450$, $p = 0.080$ between amplification factor and crisis drawdowns) and predicts crisis outcomes out-of-sample ($\rho = -0.352$ for COVID-19). Robustness analysis using cumulative abnormal returns – a measure more directly connected to the Feynman-Kac integral – strengthens these findings ($\rho = -0.512$, $p = 0.042$). Time-series analysis confirms that average pairwise bank correlations track macroeconomic stress indicators ($\rho = 0.265$ with VIX, $p < 0.001$). Comparing the two crises reveals that COVID-19 produced a sharper correlation spike (+93%) despite smaller equity losses, reflecting different contagion dynamics for exogenous versus endogenous shocks.

The Wasserstein distance, also known as the Earth mover distance or optimal transport distance, is a widely used measure of similarity between probability distributions. This paper presents an linear programming based implementation of the multi-dimensional Wasserstein distance function in Scipy, a powerful scientific computing package in Python. Building upon the existing one-dimensional scipy.stats.wasserstein_distance function, our work extends its capabilities to handle multi-dimensional distributions. To compute the multi-dimensional Wasserstein distance, we developed an implementation that transforms the problem into a linear programming problem. We utilized the scipy linear programming solver to effectively solve this transformed problem. The proposed implementation includes thorough documentation and comprehensive test cases to ensure accuracy and reliability. The resulting feature is set to be merged into the main Scipy development branch and will be included in the upcoming release, further enhancing the capabilities of Scipy in the field of multi-dimensional statistical analysis.

We consider solving nonlinear optimization problems with a stochastic objective and deterministic equality constraints, assuming that only zero-order information is available for both the objective and constraints, and that the objective is also subject to random sampling noise. Under this setting, we propose a Derivative-Free Stochastic Sequential Quadratic Programming (DF-SSQP) method. Due to the lack of derivative information, we adopt a simultaneous perturbation stochastic approximation (SPSA) technique to randomly estimate the gradients and Hessians of both the objective and constraints. This approach requires only a dimension-independent number of zero-order evaluations – as few as eight – at each iteration step. A key distinction between our derivative-free and existing derivative-based SSQP methods lies in the intricate random bias introduced into the gradient and Hessian estimates of the objective and constraints, brought by stochastic zero-order approximations. To address this issue, we introduce an online debiasing technique based on momentum-style estimators that properly aggregate past gradient and Hessian estimates to reduce stochastic noise, while avoiding excessive memory costs via a moving averaging scheme. Under standard assumptions, we establish the global almost-sure convergence of the proposed DF-SSQP method. Notably, we further complement the global analysis with local convergence guarantees by demonstrating that the rescaled iterates exhibit asymptotic normality, with a limiting covariance matrix resembling the minimax optimal covariance achieved by derivative-based methods, albeit larger due to the absence of derivative information. Our local analysis enables online statistical inference of model parameters leveraging DF-SSQP. Numerical experiments on benchmark nonlinear problems demonstrate both the global and local behavior of DF-SSQP.

We develop a formal statistical framework for classical multidimensional scaling (CMDS) applied to noisy dissimilarity data. We establish distributional convergence results for the embeddings produced by CMDS for various noise models, which enable the construction of bona~fide uniform confidence sets for the latent configuration, up to rigid transformations. We further propose bootstrap procedures for constructing these confidence sets and provide theoretical guarantees for their validity. We find that the multiplier bootstrap adapts automatically to heteroscedastic noise such as multiplicative noise, while the empirical bootstrap seems to require homoscedasticity. Either form of bootstrap, when valid, is shown to substantially improve finite-sample accuracy. The empirical performance of the proposed methods is demonstrated through numerical experiments.

We study how large an $\ell^2$ ellipsoid is by introducing type-$\tau$ integrals that capture the average decay of its semi-axes. These integrals turn out to be closely related to standard complexity measures: we show that the metric entropy of the ellipsoid is asymptotically equivalent to the type-1 integral, and that the minimax risk in non-parametric estimation is asymptotically determined by the type-2 and type-3 integrals. This allows us to retrieve and sharpen classical results about metric entropy and minimax risk of ellipsoids through a systematic analysis of the type-$\tau$ integrals, and yields an explicit formula linking the two. As an application, we improve on the best-known characterization of the metric entropy of the Sobolev ellipsoid, and extend Pinsker's Sobolev theorem in two ways: (i) to any bounded open domain in arbitrary finite dimension, and (ii) by providing the second-order term in the asymptotic expansion of the minimax risk.

Low-precision streaming PCA estimates the top principal component in a streaming setting under limited precision. We establish an information-theoretic lower bound on the quantization resolution required to achieve a target accuracy for the leading eigenvector. We study Oja's algorithm for streaming PCA under linear and nonlinear stochastic quantization. The quantized variants use unbiased stochastic quantization of the weight vector and the updates. Under mild moment and spectral-gap assumptions on the data distribution, we show that a batched version achieves the lower bound up to logarithmic factors under both schemes. This leads to a nearly dimension-free quantization error in the nonlinear quantization setting. Empirical evaluations on synthetic streams validate our theoretical findings and demonstrate that our low-precision methods closely track the performance of standard Oja's algorithm.

Quantile regression is fundamental to distributional modeling, yet independent estimation of multiple quantiles frequently produces crossing – where estimated quantile functions violate monotonicity, implying impossible negative probability densities. While Constrained Joint Quantile Regression (CJQR) elegantly enforces non-crossing by construction, existing formulations via Linear Programming exhibit $O((qn)^3)$ complexity, rendering them impractical for large-scale applications. We present the first scalable solution using PyTorch automatic differentiation: CJQR-ALM, combining the Augmented Lagrangian Method with differentiable pinball loss and L-BFGS optimization. Our approach reduces computational complexity to $O(n)$, achieving near-zero crossing rates on datasets exceeding 70,000 observations within minutes. The differentiable formulation naturally extends to neural network architectures for non-linear conditional quantile estimation. Application to Student Growth Percentile calculations demonstrates practical utility for educational assessment, while simulation studies show negligible accuracy cost (RMSE increase $\approx 2.4$ points) relative to unconstrained estimation – a favorable trade-off for applications requiring valid probability statements across finance, healthcare, and engineering.

The research introduces a reproducible framework for transforming raw, heterogeneous sensor streams into aligned, semantically meaningful representations for multimodal human activity recognition. Grounded in the Carnegie Mellon University Multi-Modal Activity Database (CMU-MMAC) database and focused on the naturalistic Subject 07 Brownie session, the study traces the full pipeline from data ingestion to modeling and interpretation. Unlike black box preprocessing, a unified preprocessing workflow is proposed that temporally aligns video, audio, and RFID through resampling, grayscale conversion, sliding-window segmentation, and modality-specific normalization, producing standardized fused tensors suitable for downstream learning. Building on this foundation, the work systematically compares early, late, and hybrid fusion strategies using LSTM-based models implemented with PyTorch and TensorFlow, showing that late fusion consistently achieves the highest validation accuracy, with hybrid fusion outperforming early fusion. To evaluate interpretability and modality contribution, PCA and t-SNE visualizations reveal coherent temporal structure and confirm that the video carries stronger discriminative power than audio, while their combination yields substantial performance gains. Incorporating sparse, asynchronous RFID signals further improves accuracy by over 50% and boosts macro-averaged ROC-AUC, demonstrating the added value of object-interaction cues. Overall, the framework contributes a modular, empirically validated approach to multimodal fusion that links preprocessing design, fusion architecture, and interpretability, offering a transferable template for intelligent systems operating in complex, real-world activity settings.

Accurately estimating heterogeneous treatment effects (HTE) in longitudinal settings is essential for personalized decision-making across healthcare, public policy, education, and digital marketing. However, time-varying interventions introduce many unique challenges, such as carryover effects, time-varying heterogeneity, and post-treatment bias, which are not addressed by standard HTE methods. To address these challenges, we introduce TERRA (Transformer-Enabled Recursive R-learner), which facilitates longitudinal HTE estimation with flexible temporal modeling and learning. TERRA has two components. First, we use a Transformer architecture to encode full treatment-feature histories, enabling the representation of long-range temporal dependencies and carryover effects, hence capturing individual- and time-specific treatment effect variation more comprehensively. Second, we develop a recursive residual-learning formulation that generalizes the classical structural nested mean models (SNMMs) beyond parametric specifications, addressing post-treatment bias while reducing reliance on functional assumptions. In simulations and data applications, TERRA consistently outperforms strong baselines in HTE estimation in both accuracy and stability, highlighting the value of combining principled causal structure with high-capacity sequence models for longitudinal HTE.

The classical Davis-Kahan theorem provides an efficient bound on the perturbation of eigenspaces of a matrix under a large (eigenvalue) gap condition. In this paper, we consider the case when the gap is moderate. Using a bootstrapping argument, we obtain a new bound which is efficient when the perturbation matrix is uncorrelated to the ground matrix. We believe that this bound is sharp up to a logarithmic term.

This work introduces a method to equip data-driven polynomial chaos expansion surrogate models with intervals that quantify the predictive uncertainty of the surrogate. To that end, jackknife-based conformal prediction is integrated into regression-based polynomial chaos expansions. The jackknife algorithm uses leave-one-out residuals to generate predictive intervals around the predictions of the polynomial chaos surrogate. The jackknife+ extension additionally requires leave-one-out model predictions. Both methods allow to use the entire dataset for model training and do not require a hold-out dataset for prediction interval calibration. The key to efficient implementation is to leverage the linearity of the polynomial chaos regression model, so that leave-one-out residuals and, if necessary, leave-one-out model predictions can be computed with analytical, closed-form expressions. This eliminates the need for repeated model re-training. The conformalized polynomial chaos expansion method is first validated on four benchmark models and then applied to two electrical engineering design use-cases. The method produces predictive intervals that provide the target coverages, even for low-accuracy models trained with small datasets. At the same time, training data availability plays a crucial role in improving the empirical coverage and narrowing the predictive interval, as well as in reducing their variability over different training datasets.

We investigate the online detection of changepoints in the distribution of a sequence of observations using a class of degenerate U-statistic-type processes. We consider an ordinary (Chu–Stinchcombe–White-type) detector and a Page-type detector under open- and closed-ended monitoring, and introduce an expanding-baseline Page-type procedure that incorporates sufficiently old monitoring observations into the baseline sample. Under the null, we derive weak limits for all three procedures and justify a Monte Carlo approximation to their critical values. For the ordinary and Page-type detectors, we also establish consistency and limiting distributions for detection delays under both early and late changes. The theory requires only square summability of the eigenvalues associated with the degenerate kernel operator, rather than the stronger absolute-summability condition often imposed in related work. Simulations show competitive performance relative to recent mean-, covariance-, and empirical-CDF-based monitors, and an application to multivariate compressor-sensor data from a metro train illustrates the methodology.

As machine learning (ML) systems are increasingly adopted in high-stakes decision-making domains, ensuring fairness in their outputs has become a central challenge. At the core of fair ML research are the datasets used to investigate bias and develop mitigation strategies. Yet, much of the existing work relies on a narrow selection of datasets–often arbitrarily chosen, inconsistently processed, and lacking in diversity–undermining the generalizability and reproducibility of results. To address these limitations, we present FairGround: a unified framework, data corpus, and Python package aimed at advancing reproducible research and critical data studies in fair ML classification. FairGround currently comprises 44 tabular datasets, each annotated with rich fairness-relevant metadata. Our accompanying Python package standardizes dataset loading, preprocessing, transformation, and splitting, streamlining experimental workflows. By providing a diverse and well-documented dataset corpus along with robust tooling, FairGround enables the development of fairer, more reliable, and more reproducible ML models. All resources are publicly available to support open and collaborative research.

We study the problem of estimating the average treatment effect (ATE) under sequentially adaptive treatment assignment mechanisms. In contrast to classical completely randomized designs, we consider a setting in which the probability of assigning treatment to each experimental unit may depend on prior assignments and observed outcomes. Within the potential outcomes framework, we propose and analyze two natural estimators for the ATE: the inverse propensity weighted (IPW) estimator and an augmented IPW (AIPW) estimator. The cornerstone of our analysis is the concept of design stability, which requires that as the number of units grows, either the assignment probabilities converge, or sample averages of the inverse propensity scores and of the inverse complement propensity scores converge in probability to fixed, non-random limits. Our main results establish central limit theorems for both the IPW and AIPW estimators under design stability and provide explicit expressions for their asymptotic variances. We further propose estimators for these variances, enabling the construction of asymptotically valid confidence intervals. Finally, we illustrate our theoretical results in the context of Wei's adaptive coin design and Efron's biased coin design, highlighting the applicability of the proposed methods to sequential experimentation with adaptive randomization.

Constitutive model discovery refers to the task of identifying an appropriate model structure, usually from a predefined model library, while simultaneously inferring its material parameters. The data used for model discovery are measured in mechanical tests and are thus inevitably affected by noise which, in turn, induces uncertainties. Previously proposed methods for uncertainty quantification in model discovery either require the selection of a prior for the material parameters, are restricted to linear coefficients of the model library or are limited in the flexibility of the inferred parameter probability distribution. We therefore propose a partially Bayesian framework for uncertainty quantification in model discovery that does not require prior selection for the material parameters and also allows for the discovery of constitutive models with inner-non-linear parameters: First, we augment the available stress-deformation data with a Gaussian process. Second, we approximate the parameter distribution by a normalizing flow, which allows for modeling complex joint distributions. Third, we distill the parameter distribution by matching the distribution of stress-deformation functions induced by the parameters with the Gaussian process posterior. Fourth, we perform a Sobol' sensitivity analysis to obtain a sparse and interpretable model. We demonstrate the capability of our framework for both isotropic and experimental anisotropic data.

Cannabis consumption impairs key driving skills and increases crash risk, yet few objective, validated tools exists to identify acute cannabis use or impairment in traffic safety settings. Pupil response to light has emerged as a promising biomarker of recent cannabis use, but its predictive utility remains underexplored. We propose two functional accelerated failure time (AFT) models for predicting time since cannabis use from pupil light response curves. The linear functional AFT (lfAFT) model provides a simple and interpretable framework that summarizes the overall contribution of a functional covariate to time-since-smoking, while the additive functional AFT (afAFT) model generalizes this structure by allowing effects to vary flexibly with both magnitude and location of the functional covariate. Estimation is computationally efficient and straightforward to implement. Simulation studies show that the proposed methods achieve strong estimation accuracy and predictive performance across various scenarios and remain robust to moderate model misspecification. Application to pupillometry data from the Colorado Cannabis & Driving Study demonstrates that pupil light response curves contain meaningful predictive signal, underscoring the potential of these models for traffic safety and broader biomedical applications.

The European Union Emissions Trading System (EU ETS), the world's first and largest cap-and-trade carbon market, is a cornerstone of EU climate policy. This study provides a comprehensive empirical analysis of the EU carbon market's efficiency, price dynamics, and structural network from 2010 to 2020. First, we identify significant price clustering and short-term return predictability using an AR-GARCH model, achieving around 60 percent directional accuracy and a 80 percent hit rate within forecasted confidence intervals. These observed patterns motivate a deeper exploration of market structure. Second, leveraging this insight, a weighted network analysis of inter-country transactions uncovers a concentrated market where a few registries dominate high-value flows and exert disproportionate influence. Finally, building upon the network findings, country-specific log-log regressions of price on traded quantity reveal heterogeneous and sometimes counter-intuitive elasticities; in several cases, positive elasticities exceed unity, indicating that trading volumes rise with prices, a deviation from conventional demand behavior that highlights potential inefficiencies driven by speculation, strategic behavior, or policy distortions. Collectively, these results point to persistent inefficiencies within the EU ETS, including partial predictability, asymmetric market power, and anomalous price-volume relationships, implying that while the system has driven decarbonization, its trading and pricing mechanisms remain imperfect.

Recent interpretability work on large language models (LLMs) has been increasingly dominated by a feature-discovery approach with the help of proxy modules. Then, the quality of features learned by, e.g., sparse auto-encoders (SAEs), is evaluated. This paradigm naturally raises a critical question: do such learned features have better properties than those already represented within the original model parameters, and unfortunately, only a few studies have made such comparisons systematically so far. In this work, we revisit the interpretability of feature vectors stored in feed-forward (FF) layers, given the perspective of FF as key-value memories, with modern interpretability benchmarks. Our extensive evaluation revealed that SAE and FFs exhibits a similar range of interpretability, although SAEs displayed an observable but minimal improvement in some aspects. Furthermore, in certain aspects, surprisingly, even vanilla FFs yielded better interpretability than the SAEs, and features discovered in SAEs and FFs diverged. These bring questions about the advantage of SAEs from both perspectives of feature quality and faithfulness, compared to directly interpreting FF feature vectors, and FF key-value parameters serve as a strong baseline in modern interpretability research.

There has been a growing interest in anomaly detection problems recently, whilst their focuses are mostly on anomalies taking place on the time index. In this work, we investigate a new anomaly-in-mean problem in multidimensional spatial lattice, that is, to detect the number and locations of anomaly “spatial regions” from the baseline. In addition to the classic minimization over the cost function with a $L_0$ penalization, we introduce an innovative penalty on the area of the minimum convex hull that covers the anomaly regions. We show that the proposed method yields a consistent estimation of the number and locations of spatial anomalies. Under the minimax framework, we characterize the optimal detection error for multidimensional spatial anomaly detection problem and reveal the trade-off between detection performance and the geometric flexibility of anomaly region shapes. Large-scale Monte Carlo simulations are carried out to examine the numeric performance of the method. The method has a wide range of applications in real-world problems. As an example, we apply it to detect the marine heatwaves using the sea surface temperature data from the European Space Agency.

We consider a node-monitor pair, where the node's state varies with time. The monitor needs to track the node's state at all times; however, there is a fixed cost for each state query. So the monitor may instead predict the state using time-series forecasting methods, including time-series foundation models (TSFMs), and query only when prediction uncertainty is high. Since query decisions influence prediction accuracy, determining when to query is nontrivial. A natural approach is a greedy policy that predicts when the expected prediction loss is below the query cost and queries otherwise. We analyze this policy in a Markovian setting, where the optimal (OPT) strategy is a state-dependent threshold policy minimizing the time-averaged sum of query cost and prediction losses. We show that, in general, the greedy policy is suboptimal and can have an unbounded competitive ratio, but under common conditions such as identically distributed transition probabilities, it performs close to OPT. For the case of unknown transition probabilities, we further propose a projected stochastic gradient descent (PSGD)-based learning variant of the greedy policy, which achieves a favorable predict-query tradeoff with improved computational efficiency compared to OPT.

Uncovering the causal mechanisms of complex real-world systems remains a significant challenge, as these systems often entail high data collection costs and involve unknown interventions. We introduce MetaCaDI, the first framework to cast the identification of unknown interventions as a meta-learning problem, explicitly leveraging a jointly learned causal graph. MetaCaDI is a Bayesian framework that learns a shared causal structure across multiple environments and is optimized to rapidly adapt to new, few-shot intervention target identification tasks. A key innovation is our model's analytical adaptation, which uses a closed-form solution to bypass expensive and potentially unstable gradient-based bilevel optimization. Extensive experiments on synthetic and complex gene expression data demonstrate that MetaCaDI significantly outperforms state-of-the-art methods. It excels at identifying intervention targets from as few as 3 samples - where existing methods collapse to random chance - while robustly recovering the shared causal graph, proving its effectiveness in data-scarce scenarios.

Bayesian formulation of modern day signal processing problems has called for improved Markov chain Monte Carlo (MCMC) sampling algorithms for inference. The need for efficient sampling techniques has become indispensable for high dimensional distributions that often characterize many core signal processing problems, e.g., image denoising, sparse signal recovery, etc. A major issue in building effective sampling strategies, however, is the non-differentiability of the underlying posterior density. Such posteriors are popular in models designed to recover sparse signals. As a result, the use of efficient gradient-based MCMC sampling techniques becomes difficult. We circumvent this problem by proposing a Proximal Hamiltonian Monte Carlo (p-HMC) algorithm, which leverages elements from convex optimization like proximal mappings and Moreau-Yosida (MY) envelopes within Hamiltonian dynamics. Our method improves upon the current state of the art non-smooth Hamiltonian Monte Carlo as it achieves a relatively sharper approximation of the gradient of log posterior density and a computational burden of at most the current state-of-the-art. A chief contribution of this work is the theoretical analysis of p-HMC. We provide conditions for geometric ergodicity of the underlying HMC chain. On the practical front, we propose guidance on choosing the key p-HMC hyperparameter – the regularization parameter in the MY-envelope. We demonstrate p-HMC's efficiency over other MCMC algorithms on benchmark problems of logistic regression and low-rank matrix estimation.

We evaluate the performance of targeted maximum likelihood estimation (TMLE) for estimating the average treatment effect in missing data scenarios under varying levels of positivity violations. We employ model- and design-based simulations, with the latter using undersmoothed highly adaptive lasso on the 'WASH Benefits Bangladesh' dataset to mimic real-world complexities. Five missingness-directed acyclic graphs are considered, capturing common missing data mechanisms in epidemiological research, particularly in one-point exposure studies. These mechanisms include also not-at-random missingness in the exposure, outcome, and confounders. We compare eight missing data methods in conjunction with TMLE as the analysis method, distinguishing between non-multiple imputation (non-MI) and multiple imputation (MI) approaches. The MI approaches use both parametric and machine-learning models. Results show that non-MI methods, particularly complete cases with TMLE incorporating an outcome-missingness model, exhibit lower bias compared to all other evaluated missing data methods and greater robustness against positivity violations across. In Comparison MI with classification and regression trees (CART) achieve lower root mean squared error, while often maintaining nominal coverage rates. Our findings highlight the trade-offs between bias and coverage, and we recommend using complete cases with TMLE incorporating an outcome-missingness model for bias reduction and MI CART when accurate confidence intervals are the priority.