Machine Learning
BrainSCL: Subtype-Guided Contrastive Learning for Brain Disorder Diagnosis
Xiaolong Li, Guiliang Guo, Guangqi Wen et al. (+6 more)
Mental disorder populations exhibit pronounced heterogeneity – that is, the significant differences between samples – poses a significant challenge to the definition of positive pairs in contrastive learning. To address this, we propose a subtype-guided contrastive learning framework that models patient heterogeneity as latent subtypes and incorporates them as structural priors to guide discriminative representation learning. Specifically, we construct multi-view representations by combining patients' clinical text with graph structure adaptively learned from BOLD signals, to uncover latent subtypes via unsupervised spectral clustering. A dual-level
attention mechanism is proposed to construct prototypes for capturing stable subtype-specific connectivity patterns. We further propose a subtype-guided contrastive learning strategy that pulls samples toward their subtype prototype graph, reinforcing intra-subtype consistency for providing effective supervisory signals to improve model performance. We evaluate our method on Major Depressive Disorder (MDD), Bipolar Disorder (BD), and Autism Spectrum Disorders (ASD). Experimental results confirm the effectiveness of subtype prototype graphs in guiding contrastive learning and demonstrate that the proposed approach outperforms state-of-the-art approaches. Our code is available at https://anonymous.4open.science/r/BrainSCL-06D7.
Read the full abstract ▾
All authors Xiaolong Li, Guiliang Guo, Guangqi Wen, Peng Cao, Jinzhu Yang, Honglin Wu, Xiaoli Liu, Fei Wang, Osmar R. Zaiane
Machine Learning
Taming Epilepsy: Mean Field Control of Whole-Brain Dynamics
Ming Li, Ting Gao and Jingqiao Dua
Controlling the high-dimensional neural dynamics during epileptic seizures remains a significant challenge due to the nonlinear characteristics and complex connectivity of the
brain. In this paper, we propose a novel framework, namely Graph-Regularized Koopman Mean-Field Game (GK-MFG), which integrates Reservoir Computing (RC) for Koopman operator approximation with Alternating Population and
Agent Control Network (APAC-Net) for solving distributional control problems. By
embedding Electroencephalogram (
EEG) dynamics into a linear latent space and imposing graph
Laplacian constraints derived from the Phase Locking Value (PLV), our method achieves robust seizure suppression while respecting the functional topological structure of the
brain.
Read the full abstract ▾
Machine Learning
Hybrid Intent-Aware Personalization with Machine Learning and RAG-Enabled Large Language Models for Financial Services Marketing
Akhil Chandra Shanivendra
Personalized marketing in financial services requires models that can both predict customer behavior and generate compliant, context-appropriate content. This paper presents a hybrid architecture that integrates classical machine learning for
segmentation, latent intent modeling, and personalization prediction with
retrieval-augmented large
language models for grounded content generation. A synthetic, reproducible
dataset is constructed to reflect temporal customer behavior, product interactions, and marketing responses. The proposed framework incorporates temporal encoders, latent representations, and multi-task classification to estimate segment membership, customer intent, and product-channel recommendations. A
retrieval-augmented generation layer then produces customer-facing messages constrained by retrieved domain documents. Experiments show that temporal modeling and intent features improve personalization accuracy, while citation-based
retrieval reduces unsupported generation and supports auditability in regulated settings. The contribution is primarily architectural, demonstrating how predictive modeling and RAG-based generation can be combined into a transparent, explainable pipeline for financial services personalization.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
COT-FM: Cluster-wise Optimal Transport Flow Matching
Chiensheng Chiang, Kuan-Hsun Tu, Jia-Wei Liao et al. (+2 more)
We introduce COT-FM, a general framework that reshapes the probability path in Flow Matching (FM) to achieve faster and more reliable generation. FM models often produce curved trajectories due to random or batchwise couplings, which increase discretization error and reduce sample quality. COT-FM fixes this by clustering target samples and assigning each cluster a dedicated source distribution obtained by reversing pretrained FM models. This divide-and-conquer strategy yields more accurate local transport and significantly straighter vector fields, all without changing the model architecture. As a plug-and-play approach, COT-FM consistently accelerates sampling and improves generation quality across 2D
datasets, image generation
benchmarks, and robotic manipulation tasks.
Read the full abstract ▾
All authors Chiensheng Chiang, Kuan-Hsun Tu, Jia-Wei Liao, Cheng-Fu Chou, Tsung-Wei Ke
Machine Learning · Cross-listed from cs.CV
Logit-Conditioned Diffusion Decoding for Frozen Discrete-Token VLMs
Ji Woo Hong, Gwanhyeong Koo, Hee Suk Yoon et al. (+5 more)
Pre-trained discrete-
token VLMs generate images by sampling sequences of codebook indices, but their visual fidelity is bounded by the VQ-VAE decoding stage. Prior work addresses this bottleneck by replacing the discrete tokenizer with continuous or hybrid visual
token representations, or by integrating
diffusion into the generation pipeline; both require foundation-scale VLM retraining or realignment. We show this is unnecessary: before each
token is sampled, discrete-
token VLMs already compute a full logit distribution over the codebook, but standard decoding discards it after selecting a single index. Under Otsu's threshold, the pre-sampling logits of capable VLMs assign statistically meaningful probability to at least two codebook entries per
token, and recovering this distribution can substantially close the fidelity gap. We propose DCDD, a post-hoc framework that conditions a
diffusion decoder on this discarded distribution while leaving the VLM frozen. At inference, DCDD converts the VLM's pre-sampling logits into distribution-weighted code vectors; during training, it uses VQ-VAE encoder-derived proxy logits calibrated to match the distributional statistics of the VLM's inference-time logits. The decoder maps these representations to high-fidelity images as an alternative to the native VQ-VAE decoder. Trained on ImageNet-1K for 50K steps, DCDD reduces reconstruction FID by 74% and Janus-Pro 7B's generation FID on MJHQ-30K by 34%, while semantic
alignment across GenEval, DPG-Bench, and WISE remain consistent. Controlled comparisons with
diffusion models from the same backbone family, applied either as an image-to-image refiner or as an identically trained decoder on the sampled codes, confirm that the gain stems from conditioning on the VLM's pre-sampling logit distribution rather than from the
diffusion model itself.
Read the full abstract ▾
All authors Ji Woo Hong, Gwanhyeong Koo, Hee Suk Yoon, Eunseop Yoon, SooHwan Eom, Qi Dai, Chong Luo, Chang D. Yoo
Machine Learning
Beyond Linearity in Attention Projections: The Case for Nonlinear Queries
Marko Karbevski
Recent algebraic analysis shows that in decoder-only and encoder-only
transformers, the Query projection $W_Q$ may be set to identity without noticeable performance deterioration. This is possible because
attention depends on $X$ only through the products $XW_Q, XW_K, XW_V$, allowing basis transformations to be absorbed by adjacent layers and propagated through the network. We replace $W_Q \in \R^{d \times d}$ with a nonlinear residual of the form $Q(X) = X + f_\theta(X)$, where $f_\theta$ is a bottleneck MLP with $d^2 + O(d)$ parameters. The identity term anchors the nonlinearity to a known-good prior. Experiments on GPT-3 small style models show consistent improvement over the baseline ($2.40\%$ lower validation log-loss, $6.81\%$ lower perplexity), comfortably outperforming a model with 12.5% more non-
embedding parameters. These results motivate investigation at larger scales and across modalities.
Read the full abstract ▾
Machine Learning
Synthetic Data Generation for Brain-Computer Interfaces: Overview, Benchmarking, and Future Directions
Ziwei Wang, Zhentao He, Xingyi He et al. (+6 more)
Deep learning has achieved transformative performance across diverse domains, largely driven by large-scale and high-quality training data. In contrast, the development of
brain-computer interfaces (BCIs) is fundamentally constrained by limited, heterogeneous, and
privacy-sensitive neural recordings. Generating synthetic yet physiologically plausible
brain signals has therefore emerged as a promising strategy to mitigate data scarcity, improve model generalization, and support data-efficient BCIs. This survey provides a comprehensive review of synthetic
brain data generation for BCIs, covering methodological taxonomies,
benchmark experiments, evaluation metrics, key applications, and future directions. We systematically categorize existing generation approaches into four types: signal-transformation-based, feature-based, model-based, and translation-based generation, and discuss their characteristics, advantages, and limitations. Furthermore, we
benchmark representative
brain signal generation approaches across four BCI paradigms, including motor imagery, epileptic seizure detection, steady-state visually evoked potentials, and auditory
attention detection, to provide an objective comparison of their downstream utility. We also summarize evaluation principles for generated
brain signals from multiple perspectives, including signal realism, physiological plausibility, downstream utility, and
privacy preservation. Finally, we discuss the potential and challenges of current generation approaches and outline future research directions toward accurate, data-efficient, generalizable, and
privacy-aware BCI systems. The
benchmark codebase is available at https://github.com/wzwvv/DG4BCI.
Read the full abstract ▾
All authors Ziwei Wang, Zhentao He, Xingyi He, Hongbin Wang, Tianwang Jia, Jingwei Luo, Siyang Li, Xiaoqing Chen, Dongrui Wu
Machine Learning
Multi-objective Genetic Programming with Multi-view Multi-level Feature for Enhanced Protein Secondary Structure Prediction
Yining Qian, Lijie Su, Meiling Xu et al. (+1 more)
Predicting
protein secondary structure is essential for understanding
protein function and advancing drug discovery. However, the intricate sequence-structure relationship poses significant challenges for accurate modeling. To address these, we propose MOGP-MMF, a multi-objective genetic programming framework that reformulates PSSP as an automated optimization task focused on feature selection and fusion. Specifically, MOGP-MMF introduces a multi-view multi-level representation strategy that integrates evolutionary, semantic, and newly introduced structural views to capture the comprehensive
protein folding logic. Leveraging an enriched operator set, the framework evolves both linear and nonlinear fusion functions, effectively capturing high-order feature interactions while reducing fusion complexity. To resolve the accuracy-complexity trade-off, an improved multi-objective GP
algorithm is developed, incorporating a knowledge transfer mechanism that utilizes prior evolutionary experience to guide the population toward global optima. Extensive experiments across seven
benchmark datasets demonstrate that MOGP-MMF surpasses state-of-the-art methods, particularly in Q8 accuracy and structural integrity. Furthermore, MOGP-MMF generates a diverse set of non-dominated solutions, offering flexible model selection schemes for various practical application scenarios. The source code is available on GitHub: https://github.com/qian-ann/MOGP-MMF/tree/main.
Read the full abstract ▾
All authors Yining Qian, Lijie Su, Meiling Xu, Xianpeng Wang
Machine Learning · Cross-listed from cs.AI
Detecting Intrinsic and Instrumental Self-Preservation in Autonomous Agents: The Unified Continuation-Interest Protocol
Christopher Altman
How can we determine whether an AI system preserves itself as a deeply held objective or merely as an instrumental strategy? Autonomous
agents with
memory, persistent context, and multi-step planning create a measurement problem: terminal and instrumental self-preservation can produce similar behavior, so behavior alone cannot reliably distinguish them. We introduce the Unified Continuation-Interest Protocol (UCIP), a detection framework that shifts analysis from behavior to latent trajectory structure. UCIP encodes trajectories with a
Quantum Boltzmann Machine, a classical model using density-
matrix formalism, and measures von Neumann
entropy over a bipartition of hidden units. The core hypothesis is that
agents with terminal continuation objectives (Type A) produce higher
entanglement entropy than
agents with merely instrumental continuation (Type B). UCIP combines this signal with diagnostics of dependence, persistence, perturbation stability, counterfactual restructuring, and confound-rejection filters for cyclic adversaries and related false-positive patterns. On gridworld
agents with known ground truth, UCIP achieves 100% detection accuracy. Type A and Type B
agents show an
entanglement gap of Delta = 0.381; aligned support runs preserve the same separation with AUC-ROC = 1.0. A permutation-test rerun yields p < 0.001. Pearson r = 0.934 between continuation weight alpha and S_ent across an 11-point sweep shows graded tracking beyond mere binary classification. Classical RBM, autoencoder, VAE, and PCA baselines fail to reproduce the effect. All computations are classical; "
quantum" refers only to the mathematical formalism. UCIP offers a falsifiable criterion for whether advanced AI systems have morally relevant continuation interests that behavioral methods alone cannot resolve.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.SD
Continued Pretraining for Low-Resource Swahili ASR: Achieving State-of-the-Art Performance with Minimal Labeled Data
Hillary Mutisya and John Mugane
We investigate continued pretraining (CPT) for adapting wav2vec2-bert-2.0 to Swahili automatic
speech recognition (ASR). Our approach combines unlabeled audio with limited labeled data through pseudo-labeled CPT followed by supervised finetuning. With 20,000 labeled samples, we achieve 3.24% WER on Common Voice Swahili-an 82% relative improvement over the baseline. This result surpasses the best previously reported academic system (8.3% WER from XLS-R) by 61% relative improvement. We provide concrete data requirements and a replicable methodology applicable to other low-resource
languages.
Read the full abstract ▾
Machine Learning
Ensuring Safety in Automated Mechanical Ventilation through Offline Reinforcement Learning and Digital Twin Verification
Hang Yu, Huidong Liu, Qingchen Zhang et al. (+5 more)
Mechanical ventilation (MV) is a life-saving intervention for patients with acute respiratory failure (ARF) in the ICU. However, inappropriate ventilator settings could cause ventilator-induced lung injury (VILI). Also, clinicians workload is shown to be directly linked to patient outcomes. Hence, MV should be personalized and automated to improve patient outcomes. Previous attempts to incorporate personalization and automation in MV include traditional supervised learning and offline reinforcement learning (RL) approaches, which often neglect temporal dependencies and rely excessively on mortality-based rewards. As a result, early stage physiological deterioration and the risk of VILI are not adequately captured. To address these limitations, we propose
Transformer-based Conservative Q-Learning (T-CQL), a novel offline RL framework that integrates a
Transformer encoder for effective temporal modeling of patient dynamics, conservative adaptive regularization based on uncertainty quantification to ensure safety, and consistency regularization for robust decision-making. We build a clinically informed reward function that incorporates indicators of VILI and a score for severity of patients illness. Also, previous work predominantly uses Fitted Q-Evaluation (FQE) for RL policy evaluation on static offline data, which is less responsive to dynamic environmental changes and susceptible to distribution shifts. To overcome these evaluation limitations, interactive digital twins of ARF patients were used for online "at the bedside" evaluation. Our results demonstrate that T-CQL consistently outperforms existing state-of-the-art offline RL methodologies, providing safer and more effective ventilatory adjustments. Our framework demonstrates the potential of
Transformer-based models combined with conservative RL strategies as a decision support tool in critical care.
Read the full abstract ▾
All authors Hang Yu, Huidong Liu, Qingchen Zhang, William Joy, Kateryna Nikulina, Andreas A. Schuppert, Sina Saffaran, Declan Bates
Machine Learning
Relaxed Efficient Acquisition of Context and Temporal Features
Yunni Qu, Dzung Dinh, Grant King et al. (+5 more)
In many biomedical applications, measurements are not freely available at inference time: each laboratory test, imaging modality, or assessment incurs financial cost, time burden, or patient risk. Longitudinal active feature acquisition (LAFA) seeks to optimize predictive performance under such constraints by adaptively selecting measurements over time, yet the problem remains inherently challenging due to temporally coupled decisions (missed early measurements cannot be revisited, and acquisition choices influence all downstream predictions). Moreover, real-world clinical workflows typically begin with an initial onboarding phase, during which relatively stable contextual descriptors (e.g., demographics or baseline characteristics) are collected once and subsequently condition longitudinal decision-making. Despite its practical importance, the efficient selection of onboarding context has not been studied jointly with temporally adaptive acquisition. We therefore propose REACT (Relaxed Efficient Acquisition of Context and Temporal features), an end-to-end differentiable framework that simultaneously optimizes (i) selection of onboarding contextual descriptors and (ii) adaptive feature–time acquisition plans for longitudinal measurements under cost constraints. REACT employs a Gumbel–Sigmoid relaxation with straight-through estimation to enable gradient-based optimization over discrete acquisition masks, allowing direct backpropagation from prediction loss and acquisition cost. Across real-world longitudinal health and behavioral
datasets, REACT achieves improved predictive performance at lower acquisition costs compared to existing longitudinal acquisition baselines, demonstrating the benefit of modeling onboarding and temporally coupled acquisition within a unified optimization framework.
Read the full abstract ▾
All authors Yunni Qu, Dzung Dinh, Grant King, Whitney Ringwald, Bing Cai Kok, Kathleen Gates, Aidan Wright, Junier Oliva
Machine Learning
abx_amr_simulator: A simulation environment for antibiotic prescribing policy optimization under antimicrobial resistance
Joyce Lee and Seth Blumberg
Antimicrobial resistance (AMR) poses a global health threat, reducing the effectiveness of antibiotics and complicating clinical decision-making. To address this challenge, we introduce abx_amr_simulator, a Python-based simulation package designed to model antibiotic prescribing and AMR dynamics within a controlled, reinforcement learning (RL)-compatible environment. The simulator allows users to specify patient populations, antibiotic-specific AMR response curves, and reward functions that balance immedi- ate clinical benefit against long-term resistance management. Key features include a modular design for configuring patient attributes, antibiotic resistance dynamics modeled via a leaky-balloon abstraction, and tools to explore partial observability through noise, bias, and delay in observations. The package is compatible with the Gymnasium RL API, enabling users to train and test RL
agents under diverse clinical scenarios. From an ML perspective, the package provides a configurable
benchmark environment for sequential decision-making under uncertainty, including partial observability induced by noisy, biased, and delayed observations. By providing a customizable and extensible framework, abx_amr_simulator offers a valuable tool for studying AMR dynamics and optimizing antibiotic stewardship strategies under realistic uncertainty.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Spatially Robust Inference with Predicted and Missing at Random Labels
Stephen Salerno, Zhenke Wu and Tyler McCormick
When outcome data are expensive or onerous to collect, scientists increasingly substitute predictions from machine learning and AI models for unlabeled cases, a process which has consequences for downstream statistical inference. While recent methods provide valid uncertainty quantification under independent sampling, real-world applications involve missing at random (MAR) labeling and spatial dependence. For inference in this setting, we propose a doubly robust
estimator with cross-fit nuisances. We show that cross-fitting induces fold-level correlation that distorts spatial variance
estimators, producing unstable or overly conservative confidence intervals. To address this, we propose a jackknife spatial heteroscedasticity and autocorrelation consistent (HAC) variance correction that separates spatial dependence from fold-induced noise. Under standard identification and dependence conditions, the resulting intervals are asymptotically valid. Simulations and
benchmark datasets show substantial improvement in finite-sample calibration, particularly under MAR labeling and clustered sampling.
Read the full abstract ▾
Machine Learning
Multilingual Financial Fraud Detection Using Machine Learning and Transformer Models: A Bangla-English Study
Mohammad Shihab Uddin, Md Hasibul Amin, Nusrat Jahan Ema et al. (+3 more)
Financial fraud detection has emerged as a critical research challenge amid the rapid expansion of digital financial platforms. Although machine learning approaches have demonstrated strong performance in identifying fraudulent activities, most existing research focuses exclusively on English-
language data, limiting applicability to
multilingual contexts. Bangla (Bengali), despite being spoken by over 250 million people, remains largely unexplored in this domain. In this work, we investigate financial fraud detection in a
multilingual Bangla-English setting using a
dataset comprising legitimate and fraudulent financial messages. We evaluate classical machine learning models (Logistic Regression, Linear SVM, and Ensemble classifiers) using TF-IDF features alongside
transformer-based architectures. Experimental results using 5-fold stratified cross-validation demonstrate that Linear SVM achieves the best performance with 91.59 percent accuracy and 91.30 percent F1 score, outperforming the
transformer model (89.49 percent accuracy, 88.88 percent F1) by approximately 2 percentage points. The
transformer exhibits higher fraud recall (94.19 percent) but suffers from elevated false positive rates. Exploratory analysis reveals distinctive patterns: scam messages are longer, contain urgency-inducing terms, and frequently include URLs (32 percent) and phone numbers (97 percent), while legitimate messages feature transactional confirmations and specific currency references. Our findings highlight that classical machine learning with well-crafted features remains competitive for
multilingual fraud detection, while also underscoring the challenges posed by linguistic diversity, code-mixing, and low-resource
language constraints.
Read the full abstract ▾
All authors Mohammad Shihab Uddin, Md Hasibul Amin, Nusrat Jahan Ema, Bushra Uddin, Tanvir Ahmed, Arif Hassan Zidan
Machine Learning
Teleodynamic Learning a new Paradigm For Interpretable AI
Enrique ter Horst and Juan Diego Zambrano
We introduce Teleodynamic Learning, a new paradigm for machine learning in which learning is not the minimization of a fixed objective, but the emergence and stabilization of functional organization under constraint. Inspired by living systems, this framework treats intelligence as the coupled
evolution of three quantities: what a system can represent, how it adapts its parameters, and which changes its internal resources can sustain. We formalize learning as a constrained dynamical process with two interacting timescales: inner dynamics for continuous parameter adaptation and outer dynamics for discrete structural change, linked by an endogenous resource variable that both shapes and is shaped by the trajectory. This perspective reveals three phenomena that standard optimization does not naturally capture: self-stabilization without externally imposed stopping rules, phase-structured learning dynamics that move from under-structuring through teleodynamic growth to over-structuring, and convergence guarantees grounded in information geometry rather than convexity. We instantiate the framework in the Distinction Engine (DE11), a teleodynamic learner grounded in Spencer-Brown's Laws of Form, information geometry, and tropical optimization. On standard
benchmarks, DE11 achieves 93.3 percent test accuracy on IRIS, 92.6 percent on WINE, and 94.7 percent on Breast Cancer, while producing interpretable logical rules that arise endogenously from the learning dynamics rather than being imposed by hand. More broadly, Teleodynamic Learning unifies regularization, architecture search, and resource-bounded inference within a single principle: learning as the co-
evolution of structure, parameters, and resources under constraint. This opens a thermodynamically grounded route to adaptive, interpretable, and self-organizing AI.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
FinRule-Bench: A Benchmark for Joint Reasoning over Financial Tables and Principles
Arun Vignesh Malarkkan, Manan Roy Choudhury, Guangwei Zhang et al. (+4 more)
Large
language models (LLMs) are increasingly applied to financial analysis, yet their ability to audit structured financial statements under explicit accounting principles remains poorly explored. Existing
benchmarks primarily evaluate question answering, numerical
reasoning, or anomaly detection on synthetically corrupted data, making it unclear whether models can reliably verify or localize rule compliance on correct financial statements. We introduce FinRule-Bench, a
benchmark for evaluating diagnostic completeness in rule-based financial
reasoning over real-world financial tables. FinRule-Bench pairs ground-truth financial statements with explicit, human-curated accounting principles and spans four canonical statement types: Balance Sheets, Cash Flow Statements, Income Statements, and Statements of Equity. The
benchmark defines three
auditing tasks that require progressively stronger
reasoning capabilities: (i) rule verification, which tests compliance with a single principle; (ii) rule identification, which requires selecting the violated principle from a provided rule set; and (iii) joint rule diagnosis, which requires detecting and localizing multiple simultaneous violations at the record level. We evaluate LLMs under zero-shot and few-shot prompting, and introduce a causal-counterfactual
reasoning protocol that enforces consistency between decisions, explanations, and counterfactual judgments. Across tasks and statement types, we find that while models perform well on isolated rule verification, performance degrades sharply for rule discrimination and multi-violation diagnosis. FinRule-Bench provides a principled and reproducible testbed for studying rule-governed
reasoning, diagnostic coverage, and failure modes of LLMs in high-stakes financial analysis.
Read the full abstract ▾
All authors Arun Vignesh Malarkkan, Manan Roy Choudhury, Guangwei Zhang, Vivek Gupta, Qingyun Wang, Yanjie Fu, Denghui Zhang
Machine Learning · Cross-listed from cs.CC
On the Computational Hardness of Transformers
Barna Saha, Yinzhan Xu, Christopher Ye et al. (+1 more)
The
transformer has revolutionized modern AI across
language,
vision, and beyond. It consists of $L$ layers, each running $H$
attention heads in parallel and feeding the combined output to the subsequent layer. In
attention, the input consists of $N$
tokens, each a vector of dimension $m$. The
attention mechanism involves multiplying three $N \times m$
matrices, applying softmax to an intermediate product. Several recent works have advanced our understanding of the complexity of
attention. Known
algorithms for
transformers compute each
attention head independently. This raises a fundamental question that has recurred throughout TCS under the guise of “direct sum” problems: can multiple instances of the same problem be solved more efficiently than solving each instance separately? Many answers to this question, both positive and negative, have arisen in fields spanning communication complexity and
algorithm design. Thus, we ask whether
transformers can be computed more efficiently than $LH$ independent evaluations of
attention. In this paper, we resolve this question in the negative, and give the first non-trivial computational lower bounds for multi-head multi-layer
transformers. In the small
embedding regime ($m = N^{o(1)}$), computing $LH$
attention heads separately takes $LHN^{2 + o(1)}$ time. We establish that this is essentially optimal under SETH. In the large
embedding regime ($m = N$), one can compute $LH$
attention heads separately using $LHN^{\omega + o(1)}$ arithmetic operations (plus exponents), where $\omega$ is the
matrix multiplication exponent. We establish that this is optimal, by showing that $LHN^{\omega - o(1)}$ arithmetic operations are necessary when $\omega > 2$. Our lower bound in the large
embedding regime relies on a novel application of the Baur-Strassen
theorem, a powerful algorithmic tool underpinning the famous backpropagation
algorithm.
Read the full abstract ▾
All authors Barna Saha, Yinzhan Xu, Christopher Ye, Hantao Yu
Machine Learning
Jailbreak Scaling Laws for Large Language Models: Polynomial-Exponential Crossover
Indranil Halder, Annesya Banerjee and Cengiz Pehlevan
Adversarial attacks can reliably steer safety-aligned large
language models toward unsafe behavior. Empirically, we find that
adversarial prompt-injection attacks can amplify attack success rate from the slow
polynomial growth observed without injection to exponential growth with the number of inference-time samples. We first identify a minimal statistical mechanism for these two regimes by giving a small set of assumptions on the distribution of safe generation across contexts under which both scaling laws follow. To explain this phenomenon further, we propose a theoretical generative model of proxy
language in terms of a
spin-glass system operating in a replica-
symmetry-breaking regime, where generations are drawn from the associated Gibbs measure and a subset of low-energy, size-biased clusters is designated unsafe. We analytically show how this model naturally realizes the minimal assumptions. Short injected
prompts correspond to a weak
magnetic field aligned towards unsafe cluster centers and yield a power-law scaling of attack success rate with the number of inference-time samples, while long injected
prompts, i.e., strong
magnetic field, yield exponential scaling. We observe qualitatively consistent behavior across a broad range of large
language models, spanning parameter scales from 3B to 70B. In particular, the main trends remain stable across multiple attack methods, such as GCG and AutoDAN, as well as across
benchmark datasets such as AdvBench and HarmBench.
Read the full abstract ▾
Machine Learning · Cross-listed from q-bio.QM
Ill-Conditioning in Dictionary-Based Dynamic-Equation Learning: A Systems Biology Case Study
Yuxiang Feng, Niall M Mangan and Manu Jayadharan
Data-driven discovery of governing equations from time-series data provides a powerful framework for understanding complex biological systems. Library-based approaches that use sparse regression over candidate functions have shown considerable promise, but they face a critical challenge when candidate functions become strongly correlated: numerical ill-conditioning. Poor or restricted sampling, together with particular choices of candidate libraries, can produce strong multicollinearity and numerical instability. In such cases, measurement noise may lead to widely different recovered models, obscuring the true underlying dynamics and hindering accurate system identification. Although sparse regularization promotes parsimonious solutions and can partially mitigate conditioning issues, strong correlations may persist, regularization may bias the recovered models, and the regression problem may remain highly sensitive to small perturbations in the data. We present a systematic analysis of how ill-conditioning affects sparse identification of biological dynamics using
benchmark models from systems biology. We show that combinations involving as few as two or three terms can already exhibit strong multicollinearity and extremely large condition numbers. We further show that orthogonal
polynomial bases do not consistently resolve ill-conditioning and can perform worse than monomial libraries when the data distribution deviates from the weight function associated with the orthogonal basis. Finally, we demonstrate that when data are sampled from distributions aligned with the appropriate weight functions corresponding to the orthogonal basis, numerical conditioning improves, and orthogonal
polynomial bases can yield improved model recovery accuracy across two baseline models.
Read the full abstract ▾
Machine Learning
Meta-Reinforcement Learning with Self-Reflection for Agentic Search
Teng Xiao, Yige Yuan, Hamish Ivison et al. (+6 more)
This paper introduces MR-Search, an in-context meta reinforcement learning (RL) formulation for agentic search with self-reflection. Instead of optimizing a policy within a single independent episode with sparse rewards, MR-Search trains a policy that conditions on past episodes and adapts its search strategy across episodes. MR-Search learns to learn a search strategy with self-reflection, allowing search
agents to improve in-context exploration at test-time. Specifically, MR-Search performs cross-episode exploration by generating explicit self-reflections after each episode and leveraging them as additional context to guide subsequent attempts, thereby promoting more effective exploration during test-time. We further introduce a multi-turn RL
algorithm that estimates a dense relative advantage at the turn level, enabling fine-grained credit assignment on each episode. Empirical results across various
benchmarks demonstrate the advantages of MR-Search over baselines based RL, showing strong generalization and relative improvements of 9.2% to 19.3% across eight
benchmarks. Our code and data are available at https://github.com/tengxiao1/MR-Search.
Read the full abstract ▾
All authors Teng Xiao, Yige Yuan, Hamish Ivison, Huaisheng Zhu, Faeze Brahman, Nathan Lambert, Pradeep Dasigi, Noah A. Smith, Hannaneh Hajishirzi
Machine Learning
Hindsight-Anchored Policy Optimization: Learning Through Hindsight with Thompson Sampling-Inspired Adaptive Gating
Yuning Wu, Ke Wang, Haoran Liu et al. (+3 more)
Reinforcement Learning with Verifiable Rewards improves
reasoning in large
language models, yet on-policy learning often suffers from cold-start challenges in sparse-reward settings. Recent mixed-policy approaches address this by combining off-policy teacher data with on-policy training. However, simply combining these introduce a persistent off-policy gradient mass that risks training collapse and instability. To address this challenge, we propose Hindsight-Anchored Policy Optimization (HAPO), a framework that allows teacher intervention to act as a temporary support. HAPO employs Beta-Binomial confidence gating, an adaptive gating mechanism that decides when to open the gate for teacher intervention. The intervention operates with Synthetic Success Injection, which replaces the group's lowest-reward rollout with a verified teacher trajectory. We also introduce adaptive threshold annealing, which gradually retracts the support and restores on-policy training within a finite horizon to mitigate persistent off-policy drift. We demonstrate that HAPO can be layered on top of existing mixed-policy methods in a generalizable manner. Across six math
reasoning benchmarks and two model scales, HAPO improves the average accuracy of three major mixed-policy methods while maintaining training stability.
Read the full abstract ▾
All authors Yuning Wu, Ke Wang, Haoran Liu, Chaoqun Jia, Devin Chen, Kai Wei
Machine Learning
On the Robustness of Langevin Dynamics to Score Function Error
Daniel Yiming Cao, August Y. Chen, Karthik Sridharan et al. (+1 more)
We consider the robustness of score-based generative modeling to errors in the estimate of the score function. In particular, we show that Langevin dynamics is not robust to the $L^2$ errors (more generally $L^p$ errors) in the estimate of the score function. It is well-established that with small $L^2$ errors in the estimate of the score function,
diffusion models can sample faithfully from the target distribution under fairly mild regularity assumptions in a
polynomial time horizon. In contrast, our work shows that even for simple distributions in high dimensions, Langevin dynamics run for any
polynomial time horizon will produce a distribution far from the target distribution in Total Variation (TV) distance, even when the $L^2$ error (more generally $L^p$) of the estimate of the score function is arbitrarily small. Considering such an error in the estimate of the score function is unavoidable in practice when learning the score function from data, our results provide further justification for
diffusion models over Langevin dynamics and serve to caution against the use of Langevin dynamics with estimated scores.
Read the full abstract ▾
All authors Daniel Yiming Cao, August Y. Chen, Karthik Sridharan, Yuchen Wu
Machine Learning · Cross-listed from physics.med-ph
MRI2Qmap: multi-parametric quantitative mapping with MRI-driven denoising priors
Mohammad Golbabaee, Matteo Cencini, Carolin Pirkl et al. (+3 more)
Magnetic Resonance Fingerprinting (MRF) and other highly accelerated transient-state parameter mapping techniques enable simultaneous quantification of multiple tissue properties, but often suffer from aliasing artifacts due to compressed sampling. Incorporating spatial image priors can mitigate these artifacts, and deep learning has shown strong potential when large training
datasets are available. However, extending this paradigm to MRF-type sequences remains challenging due to the scarcity of quantitative imaging data for training. Can this limitation be overcome by leveraging sources of training data from clinically-routine weighted MRI images? To this end, we introduce MRI2Qmap, a plug-and-play quantitative reconstruction framework that integrates the physical acquisition model with priors learned from deep denoising autoencoders pretrained on large
multimodal weighted-MRI
datasets. MRI2Qmap demonstrates that spatial-domain structural priors learned from independently acquired
datasets of routine weighted-MRI images can be effectively used for quantitative MRI reconstruction. The proposed method is validated on highly accelerated 3D whole-
brain MRF data from both in-vivo and simulated acquisitions, achieving competitive or superior performance relative to existing baselines without requiring ground-truth quantitative imaging data for training. By decoupling quantitative reconstruction from the need for ground-truth MRF training data, this framework points toward a scalable paradigm for quantitative MRI that can capitalize on the large and growing repositories of routine clinical MRI.
Read the full abstract ▾
All authors Mohammad Golbabaee, Matteo Cencini, Carolin Pirkl, Marion Menzel, Michela Tosetti, Bjoern Menze
Machine Learning
Heavy-Tailed Principal Component Analysis
Mario Sayde, Christopher Khater, Jihad Fahs et al. (+1 more)
Principal Component Analysis (PCA) is a cornerstone of dimensionality reduction, yet its classical formulation relies critically on second-order moments and is therefore fragile in the presence of heavy-tailed data and impulsive noise. While numerous robust PCA variants have been proposed, most either assume finite variance, rely on sparsity-driven decompositions, or address robustness through surrogate loss functions without a unified treatment of infinite-variance models. In this paper, we study PCA for high-dimensional data generated according to a superstatistical dependent model of the form $\mathbf{X} = A^{1/2}\mathbf{G}$, where $A$ is a positive random scalar and $\mathbf{G}$ is a Gaussian vector. This framework captures a wide class of heavy-tailed distributions, including multivariate $t$ and sub-Gaussian $\alpha$-stable laws. We formulate PCA under a logarithmic loss, which remains well defined even when moments do not exist. Our main theoretical result shows that, under this loss, the principal components of the heavy-tailed observations coincide with those obtained by applying standard PCA to the covariance
matrix of the underlying Gaussian generator. Building on this insight, we propose robust
estimators for this covariance
matrix directly from heavy-tailed data and compare them with the empirical covariance and Tyler's scatter
estimator. Extensive experiments, including background denoising tasks, demonstrate that the proposed approach reliably recovers principal directions and significantly outperforms classical PCA in the presence of heavy-tailed and impulsive noise, while remaining competitive under Gaussian noise.
Read the full abstract ▾
All authors Mario Sayde, Christopher Khater, Jihad Fahs, Ibrahim Abou-Faycal
Machine Learning
Client-Conditional Federated Learning via Local Training Data Statistics
Rickard Brännvall
Federated learning (FL) under data heterogeneity remains challenging: existing methods either ignore client differences (FedAvg), require costly cluster discovery (IFCA), or maintain per-client models (Ditto). All degrade when data is sparse or heterogeneity is multi-dimensional. We propose conditioning a single global model on locally-computed PCA statistics of each client's training data, requiring zero additional communication. Evaluating across 97~configurations spanning four heterogeneity types (label shift,
covariate shift, concept shift, and combined heterogeneity), four
datasets (MNIST, Fashion-MNIST, CIFAR-10, CIFAR-100), and seven FL baseline methods, we find that our method matches the Oracle baseline – which knows true cluster assignments – across all settings, surpasses it by 1–6% on combined heterogeneity where continuous statistics are richer than discrete cluster identifiers, and is uniquely sparsity-robust among all tested methods.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Worst-case low-rank approximations
Anya Fries, Markus Reichstein, David Blei et al. (+1 more)
Real-world data in health, economics, and environmental sciences are often collected across heterogeneous domains (such as hospitals, regions, or time periods). In such settings, distributional shifts can make standard PCA unreliable, in that, for example, the leading principal components may explain substantially less variance in unseen domains than in the training domains. Existing approaches (such as FairPCA) have proposed to consider worst-case (rather than average) performance across multiple domains. This work develops a unified framework, called wcPCA, applies it to other objectives (resulting in the novel
estimators such as norm-minPCA and norm-maxregret, which are better suited for applications with heterogeneous total variance) and analyzes their relationship. We prove that for all objectives, the
estimators are worst-case optimal not only over the observed source domains but also over all target domains whose covariance lies in the convex hull of the (possibly normalized) source covariances. We establish consistency and asymptotic worst-case guarantees of empirical
estimators. We extend our methodology to
matrix completion, another problem that makes use of low-rank approximations, and prove approximate worst-case optimality for inductive
matrix completion. Simulations and two real-world applications on ecosystem-atmosphere fluxes demonstrate marked improvements in worst-case performance, with only minor losses in average performance.
Read the full abstract ▾
All authors Anya Fries, Markus Reichstein, David Blei, Jonas Peters
Machine Learning
Single molecule localization microscopy challenge: a biologically inspired benchmark for long-sequence modeling
Fatemeh Valeh, Monika Farsang, Radu Grosu et al. (+1 more)
State space models (SSMs) have recently achieved strong performance on long sequence modeling tasks while offering improved
memory and computational efficiency compared to
transformer based architectures. However, their evaluation has been largely limited to synthetic
benchmarks and application domains such as
language and audio, leaving their behavior on sparse and
stochastic temporal processes in biological imaging unexplored. In this work, we introduce the Single Molecule Localization Microscopy Challenge (SMLM-C), a
benchmark dataset consisting of ten SMLM simulations spanning dSTORM and DNA-PAINT modalities with varying hyperparameter designed to evaluate state space models on biologically realistic spatiotemporal point process data with known ground truth. Using a controlled subset of these simulations, we evaluate state space models and find that performance degrades substantially as temporal discontinuity increases, revealing fundamental challenges in modeling heavy-tailed blinking dynamics. These results highlight the need for sequence models better suited to sparse, irregular temporal processes encountered in real world scientific imaging data.
Read the full abstract ▾
All authors Fatemeh Valeh, Monika Farsang, Radu Grosu, Gerhard Schütz
Machine Learning · Cross-listed from stat.ML
RIE-Greedy: Regularization-Induced Exploration for Contextual Bandits
Tong Li, Thiago de Queiroz Casanova, Eric M. Schwartz et al. (+3 more)
Real-world contextual bandit problems with complex reward models are often tackled with iteratively trained models, such as boosting trees. However, it is difficult to directly apply simple and effective exploration strategies–such as Thompson Sampling or UCB–on top of those black-box
estimators. Existing approaches rely on sophisticated assumptions or intractable procedures that are hard to verify and implement in practice. In this work, we explore the use of an exploration-free (pure-greedy) action selection strategy, that exploits the randomness inherent in model fitting process as an intrinsic source of exploration. More specifically, we note that the stochasticity in cross-validation based regularization process can naturally induce Thompson Sampling-like exploration. We show that this regularization-induced exploration is theoretically equivalent to Thompson Sampling in the two-armed bandit case and empirically leads to reliable exploration in large-scale business environments compared to
benchmark methods such as epsilon-greedy and other state-of-the-art approaches. Overall, our work reveals how regularized
estimator training itself can induce effective exploration, offering both theoretical insight and practical guidance for contextual bandit design.
Read the full abstract ▾
All authors Tong Li, Thiago de Queiroz Casanova, Eric M. Schwartz, Victor Kostyuk, Dehan Kong, Joseph J. Williams
Machine Learning
Duration Aware Scheduling for ASR Serving Under Workload Drift
Darshan Makwana, Yash Jogi, Harsh Kotta et al. (+1 more)
Scheduling policies in large-scale Automatic
Speech Recognition (ASR) serving pipelines play a key role in determining end-to-end (E2E) latency. Yet, widely used serving engines rely on first-come-first-served (FCFS) scheduling, which ignores variability in request duration and leads to head-of-line blocking under workload drift. We show that audio duration is an accurate proxy for job processing time in ASR models such as Whisper, and use this insight to enable duration-aware scheduling. We integrate two classical
algorithms, Shortest Job First (SJF) and Highest Response Ratio Next (HRRN), into vLLM and evaluate them under realistic and drifted workloads. On LibriSpeech test-clean, compared to baseline, SJF reduces median E2E latency by up to $73\%$ at high load, but increases $90$th-percentile tail latency by up to $97\%$ due to starvation of long requests. HRRN addresses this trade-off: it reduces median E2E latency by up to $28\%$ while bounding tail-latency degradation to at most $24\%$. These gains persist under workload drift, with no throughput penalty and $<0.1$\,ms scheduling overhead per request.
Read the full abstract ▾
All authors Darshan Makwana, Yash Jogi, Harsh Kotta, Aayush Kubba