Machine Learning
Dimensional Peeking for Low-Variance Gradients in Zeroth-Order Discrete Optimization via Simulation
Philipp Andelfinger and Wentong Cai
Gradient-based optimization methods are commonly used to identify local optima in high-dimensional spaces. When derivatives cannot be evaluated directly,
stochastic estimators can provide approximate gradients. However, these
estimators' perturbation-based sampling of the objective function introduces variance that can lead to slow convergence. In this paper, we present dimensional peeking, a variance reduction method for gradient estimation in discrete optimization via simulation. By lifting the sampling granularity from scalar values to classes of values that follow the same control flow path, we increase the information gathered per simulation evaluation. Our derivation from an established smoothed gradient
estimator shows that the method does not introduce any bias. We present an implementation via a custom numerical data type to transparently carry out dimensional peeking over C++ programs. Variance reductions by factors of up to 7.9 are observed for three simulation-based optimization problems with high-dimensional input. The optimization progress compared to three meta-heuristics shows that dimensional peeking increases the competitiveness of zeroth-order optimization for discrete and non-convex simulations.
Read the full abstract ▾
Machine Learning · Cross-listed from math.NA
Large Language Models: A Mathematical Formulation
Ricardo Baptista, Andrew Stuart and Son Tran
Large
language models (LLMs) process and predict sequences containing text to answer questions, and address tasks including document summarization, providing recommendations, writing software and solving quantitative problems. We provide a mathematical framework for LLMs by describing the encoding of text sequences into sequences of
tokens, defining the architecture for next-
token prediction models, explaining how these models are learned from data, and demonstrating how they are deployed to address a variety of tasks. The mathematical sophistication required to understand this material is not high, and relies on straightforward ideas from information theory, probability and optimization. Nonetheless, the combination of ideas resting on these different components from the mathematical sciences yields a complex algorithmic structure; and this algorithmic structure has demonstrated remarkable empirical successes. The mathematical framework established here provides a platform from which it is possible to formulate and address questions concerning the accuracy, efficiency and robustness of the
algorithms that constitute LLMs. The framework also suggests directions for development of modified and new methodologies.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Ego4OOD: Rethinking Egocentric Video Domain Generalization via Covariate Shift Scoring
Zahra Vaseqi and James Clark
Egocentric video action recognition under domain shifts remains challenging due to large intra-class spatio-temporal variability, long-tailed feature distributions, and strong correlations between actions and environments. Existing
benchmarks for
egocentric domain generalization often conflate
covariate shifts with concept shifts, making it difficult to reliably evaluate a model's ability to generalize across input distributions. To address this limitation, we introduce Ego4OOD, a domain generalization
benchmark derived from Ego4D that emphasizes measurable
covariate diversity while reducing concept shift through semantically coherent, moment-level action categories. Ego4OOD spans eight geographically distinct domains and is accompanied by a clustering-based
covariate shift metric that provides a quantitative proxy for domain difficulty. We further leverage a one-vs-all binary training objective that decomposes multi-class action recognition into independent binary classification tasks. This formulation is particularly well-suited for
covariate shift by reducing interference between visually similar classes under feature distribution shift. Using this formulation, we show that a lightweight two-layer fully connected network achieves performance competitive with state-of-the-art
egocentric domain generalization methods on both Argo1M and Ego4OOD, despite using fewer parameters and no additional modalities. Our empirical analysis demonstrates a clear relationship between measured
covariate shift and recognition performance, highlighting the importance of controlled
benchmarks and quantitative domain characterization for studying out-of-distribution generalization in
egocentric video.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
A Contrastive Pre-trained Foundation Model for Deciphering Imaging Noisomics across Modalities
Yuanjie Gu, Yiqun Wang, Chaohui Yu et al. (+4 more)
Characterizing imaging noise is notoriously data-intensive and device-dependent, as modern sensors entangle physical signals with complex algorithmic artifacts. Current paradigms struggle to disentangle these factors without massive supervised
datasets, often reducing noise to mere interference rather than an information resource. Here, we introduce "Noisomics", a framework shifting the focus from suppression to systematic noise decoding via the Contrastive Pre-trained (CoP) Foundation Model. By leveraging the
manifold hypothesis and synthetic noise genome, CoP employs contrastive learning to disentangle semantic signals from
stochastic perturbations. Crucially, CoP breaks traditional deep learning scaling laws, achieving superior performance with only 100 training samples, outperforming supervised baselines trained on 100,000 samples, thereby reducing data and computational dependency by three orders of magnitude. Extensive benchmarking across 12 diverse out-of-domain
datasets confirms its robust zero-shot generalization, demonstrating a 63.8% reduction in estimation error and an 85.1% improvement in the coefficient of determination compared to the conventional training strategy. We demonstrate CoP's utility across scales: from deciphering non-linear hardware-noise interplay in consumer photography to optimizing
photon-efficient protocols for deep-tissue microscopy. By decoding noise as a multi-parametric footprint, our work redefines
stochastic degradation as a vital information resource, empowering precise imaging diagnostics without prior device calibration.
Read the full abstract ▾
All authors Yuanjie Gu, Yiqun Wang, Chaohui Yu, Ang Xuan, Fan Wang, Zhi Lu, Biqin Dong
Machine Learning · Cross-listed from eess.AS
Test-Time Adaptation for Speech Emotion Recognition
Jiaheng Dong, Hong Jia and Ting Dang
The practical utility of
Speech Emotion Recognition (SER) systems is undermined by their fragility to domain shifts, such as speaker variability, the distinction between acted and naturalistic
emotions, and cross-corpus variations. While domain adaptation and fine-tuning are widely studied, they require either source data or labelled target data, which are often unavailable or raise
privacy concerns in SER. Test-time adaptation (TTA) bridges this gap by adapting models at inference using only unlabeled target data. Yet, having been predominantly designed for image classification and
speech recognition, the efficacy of TTA for mitigating the unique domain shifts in SER has not been investigated. In this paper, we present the first systematic evaluation and comparison covering 11 TTA methods across three representative SER tasks. The results indicate that backpropagation-free TTA methods are the most promising. Conversely,
entropy minimization and pseudo-labeling generally fail, as their core assumption of a single, confident ground-truth label is incompatible with the inherent ambiguity of emotional expression. Further, no single method universally excels, and its effectiveness is highly dependent on the distributional shifts and tasks.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.SE
VibeTensor: System Software for Deep Learning, Fully Generated by AI Agents
Bing Xu, Terry Chen, Fengzhe Zhou et al. (+12 more)
VIBETENSOR is an open-source research system software stack for deep learning, generated by LLM-powered coding
agents under high-level human guidance. In this paper, "fully generated" refers to code provenance: implementation changes were produced and applied as
agent-proposed diffs; validation relied on
agent-run builds, tests, and differential checks, without per-change manual diff review. It implements a PyTorch-style eager tensor library with a C++20 core (CPU+CUDA), a torch-like Python overlay via nanobind, and an experimental Node.js/TypeScript interface. Unlike thin bindings, VIBETENSOR includes its own tensor/storage system, schema-lite dispatcher, reverse-mode autograd, CUDA runtime (streams/events/graphs), a stream-ordered caching allocator with diagnostics, and a stable C ABI for dynamically loaded operator plugins. We view this release as a milestone for AI-assisted software engineering: it shows coding
agents can generate a coherent deep learning runtime spanning
language bindings down to CUDA
memory management, validated primarily by builds and tests. We describe the architecture, summarize the workflow used to produce and validate the system, and evaluate the artifact. We report repository scale and test-suite composition, and summarize reproducible microbenchmarks from an accompanying AI-generated kernel suite, including fused
attention versus PyTorch SDPA/FlashAttention. We also report end-to-end training sanity checks on 3 small workloads (sequence reversal, ViT, miniGPT) on NVIDIA H100 (Hopper, SM90) and Blackwell-class GPUs; multi-GPU results are Blackwell-only and use an optional CUTLASS-based ring-allreduce plugin gated on CUDA 13+ and sm103a toolchain support. Finally, we discuss failure modes in generated system software, including a "Frankenstein" composition effect where locally correct subsystems interact to yield globally suboptimal performance.
Read the full abstract ▾
All authors Bing Xu, Terry Chen, Fengzhe Zhou, Tianqi Chen, Yangqing Jia, Vinod Grover, Haicheng Wu, Wei Liu, Craig Wittenbrink, Wen-mei Hwu, Roger Bringmann, Ming-Yu Liu, Luis Ceze, Michael Lightstone, Humphrey Shi
Machine Learning
PRISM: Deriving a White-Box Transformer as a Signal-Noise Decomposition Operator via Maximum Coding Rate Reduction
Dongchen Huang
Deep learning models, particularly
Transformers, are often criticized as "black boxes" and lack interpretability. We propose Prism, a white-box
attention-based architecture derived from the principles of Maximizing Coding Rate Reduction ($\text{MCR}^2$). By modeling the
attention mechanism as a gradient ascent process on a distinct signal-noise
manifold, we introduce a specific irrational frequency separation ($\pi$-RoPE) to enforce incoherence between signal (semantic) and noise (syntactic) subspaces. We show empirical evidence that these
geometric inductive biases can induce unsupervised functional disentanglement alone. Prism spontaneously specializes its
attention heads into spectrally distinct regimes: low-frequency heads capturing long-range causal dependencies (signal) and high-frequency heads handling local syntactic constraints and structural artifacts. To provide a theoretical grounding for these spectral phenomena, we draw an analogy between
attention mechanism and a
Hamiltonian dynamical system and identify that the standard
geometric progression of Rotary Positional
Embeddings (RoPE) induces dense resonance networks (Arnold Tongues), leading to feature rank collapse. Empirical validation on 124M-parameter models trained on OpenWebText demonstrates that Prism spontaneously isolates the
Attention Sink pathology and maintains isentropic information flow across layers. Further, we suggest a physics-informed plug-and-play intervention KAM-RoPE for large
language models (LLMs). Our results suggest that interpretability and performance can be unified through principled
geometric construction, offering a theoretically grounded alternative to heuristic architectural modifications
Read the full abstract ▾
Machine Learning · Cross-listed from eess.IV
A Machine Vision Approach to Preliminary Skin Lesion Assessments
Ali Khreis, Ro'Yah Radaideh and Quinn McGill
Early detection of malignant skin lesions is critical for improving patient outcomes in aggressive, metastatic skin cancers. This study evaluates a comprehensive system for preliminary skin lesion assessment that combines the clinically established ABCD rule of dermoscopy (analyzing Asymmetry, Borders, Color, and Dermoscopic Structures) with machine learning classification. Using a 1,000-image subset of the HAM10000
dataset, the system implements an automated, rule-based pipeline to compute a Total Dermoscopy Score (TDS) for each lesion. This handcrafted approach is compared against various machine learning solutions, including traditional classifiers (Logistic Regression, Random Forest, and SVM) and deep learning models. While the rule-based system provides high clinical interpretability, results indicate a performance bottleneck when reducing complex morphology to five numerical features. Experimental findings show that transfer learning with EfficientNet-B0 failed significantly due to domain shift between natural and medical images. In contrast, a custom three-layer Convolutional Neural Network (CNN) trained from scratch achieved 78.5% accuracy and 86.5% recall on median-filtered images, representing a 19-point accuracy improvement over traditional methods. The results demonstrate that direct pixel-level learning captures diagnostic patterns beyond handcrafted features and that purpose-built lightweight architectures can outperform large pretrained models for small, domain-specific medical
datasets.
Read the full abstract ▾
Machine Learning
QUAIL: Quantization Aware Unlearning for Mitigating Misinformation in LLMs
Himanshu Mishra and Kanwal Mehreen
Machine unlearning aims to remove specific knowledge (e.g., copyrighted or private data) from a trained model without full retraining. In practice, models are often quantized (e.g., 4-bit) for deployment, but we find that quantization can catastrophically restore forgotten information [1]. In this paper, we (1) analyze why low-bit quantization undermines unlearning, and (2) propose a quantization-aware unlearning method to mitigate this. We first compute weight-change statistics and bucket overlaps in quantization to show that typical unlearning updates are too small to cross quantization thresholds. Building on this insight, we introduce a logits space hinge loss: for each forget example, we force the output logits of the unlearned model to differ from the original model by at least a margin (half the quantization step). This ensures forgotten examples remain distinguishable even after quantization. We evaluate on
language and classification tasks (including a Twitter misinformation
dataset) and show our method preserves forgetting under 4-bit quantization, whereas existing methods almost entirely recover the forgotten knowledge.
Read the full abstract ▾
Machine Learning
Machine learning-enhanced non-amnestic Alzheimer's disease diagnosis from MRI and clinical features
Megan A. Witherow, Michael L. Evans, Ahmed Temtam et al. (+2 more)
Alzheimer's disease (AD), defined as an abnormal buildup of amyloid plaques and tau tangles in the
brain can be diagnosed with high accuracy based on
protein biomarkers via PET or CSF analysis. However, due to the invasive nature of biomarker collection, most AD diagnoses are made in
memory clinics using cognitive tests and evaluation of hippocampal atrophy based on MRI. While clinical assessment and hippocampal volume show high diagnostic accuracy for amnestic or typical AD (tAD), a substantial subgroup of AD patients with atypical presentation (atAD) are routinely misdiagnosed. To improve diagnosis of atAD patients, we propose a machine learning approach to distinguish between atAD and non-AD cognitive impairment using clinical testing battery and MRI data collected as standard-of-care. We develop and evaluate our approach using 1410 subjects across four groups (273 tAD, 184 atAD, 235 non-AD, and 685 cognitively normal) collected from one private data set and two public data sets from the National Alzheimer's Coordinating Center (NACC) and the Alzheimer's Disease Neuroimaging Initiative (ADNI). We perform multiple atAD vs. non-AD classification experiments using clinical features and hippocampal volume as well as a comprehensive set of MRI features from across the
brain. The best performance is achieved by incorporating additional important MRI features, which outperforms using hippocampal volume alone. Furthermore, we use the Boruta statistical approach to identify and visualize significant
brain regions distinguishing between diagnostic groups. Our ML approach improves the percentage of correctly diagnosed atAD cases (the recall) from 52% to 69% for NACC and from 34% to 77% for ADNI, while achieving high precision. The proposed approach has important implications for improving diagnostic accuracy for non-amnestic atAD in clinical settings using only clinical testing battery and MRI.
Read the full abstract ▾
All authors Megan A. Witherow, Michael L. Evans, Ahmed Temtam, Hamid R. Okhravi, Khan M. Iftekharuddin
Machine Learning · Cross-listed from cs.IR
DS@GT at TREC TOT 2025: Bridging Vague Recollection with Fusion Retrieval and Learned Reranking
Wenxin Zhou, Ritesh Mehta and Anthony Miyaguchi
We develop a two-stage
retrieval system that combines multiple complementary
retrieval methods with a learned reranker and LLM-based reranking, to address the TREC Tip-of-the-Tongue (ToT) task. In the first stage, we employ hybrid
retrieval that merges LLM-based
retrieval, sparse (BM25), and dense (BGE-M3)
retrieval methods. We also introduce topic-aware multi-index dense
retrieval that partitions the Wikipedia corpus into 24 topical domains. In the second stage, we evaluate both a trained LambdaMART reranker and LLM-based reranking. To support model training, we generate 5000 synthetic ToT queries using LLMs. Our best system achieves recall of 0.66 and NDCG@1000 of 0.41 on the test set by combining hybrid
retrieval with Gemini-2.5-flash reranking, demonstrating the effectiveness of fusion
retrieval.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
DeltaDorsal: Enhancing Hand Pose Estimation with Dorsal Features in Egocentric Views
William Huang, Siyou Pei, Leyi Zou et al. (+3 more)
The proliferation of XR devices has made
egocentric hand pose estimation a vital task, yet this perspective is inherently challenged by frequent finger occlusions. To address this, we propose a novel approach that leverages the rich information in dorsal hand skin deformation, unlocked by recent advances in dense visual featurizers. We introduce a dual-stream delta encoder that learns pose by contrasting features from a dynamic hand with a baseline relaxed position. Our evaluation demonstrates that, using only cropped dorsal images, our method reduces the Mean Per Joint Angle Error (MPJAE) by 18% in self-occluded scenarios (fingers >= 50% occluded) compared to state-of-the-art techniques that depend on the whole hand's geometry and large model backbones. Consequently, our method not only enhances the reliability of downstream tasks like index finger pinch and tap estimation in occluded scenarios but also unlocks new interaction paradigms, such as detecting isometric force for a surface "click" without visible movement while minimizing model size.
Read the full abstract ▾
All authors William Huang, Siyou Pei, Leyi Zou, Eric J. Gonzalez, Ishan Chatterjee, Yang Zhang
Machine Learning · Cross-listed from cs.CL
ViT Registers and Fractal ViT
Jason Chuan-Chih Chou, Abhinav Kumar and Shivank Garg
Drawing inspiration from recent findings including surprisingly decent performance of
transformers without positional encoding (NoPE) in the domain of
language models and how registers (additional throwaway
tokens not tied to input) may improve the performance of large
vision transformers (ViTs), we invent and test a variant of ViT called fractal ViT that breaks permutation invariance among the
tokens by applying an
attention mask between the regular
tokens and “summary
tokens” similar to registers, in isolation or in combination with various positional encodings. These models do not improve upon ViT with registers, highlighting the fact that these findings may be scale, domain, or application-specific.
Read the full abstract ▾
Machine Learning
SAGE-FM: A lightweight and interpretable spatial transcriptomics foundation model
Xianghao Zhan, Jingyu Xu, Yuanning Zheng et al. (+2 more)
Spatial transcriptomics enables spatial gene expression profiling, motivating computational models that capture spatially conditioned regulatory relationships. We introduce SAGE-FM, a lightweight spatial transcriptomics foundation model based on graph convolutional networks (GCNs) trained with a masked central spot prediction objective. Trained on 416 human Visium samples spanning 15 organs, SAGE-FM learns spatially coherent
embeddings that robustly recover masked genes, with 91% of masked genes showing significant correlations (p < 0.05). The
embeddings generated by SAGE-FM outperform MOFA and existing spatial transcriptomics methods in unsupervised clustering and preservation of biological heterogeneity. SAGE-FM generalizes to downstream tasks, enabling 81% accuracy in pathologist-defined spot annotation in oropharyngeal squamous
cell carcinoma and improving glioblastoma subtype prediction relative to MOFA. In silico perturbation experiments further demonstrate that the model captures directional ligand-receptor and upstream-downstream regulatory effects consistent with ground truth. These results demonstrate that simple, parameter-efficient GCNs can serve as biologically interpretable and spatially aware foundation models for large-scale spatial transcriptomics.
Read the full abstract ▾
All authors Xianghao Zhan, Jingyu Xu, Yuanning Zheng, Zinaida Good, Olivier Gevaert
Machine Learning
Data-driven Lake Water Quality Forecasting for Time Series with Missing Data using Machine Learning
Rishit Chatterjee and Tahiya Chowdhury
Volunteer-led lake monitoring yields irregular, seasonal time series with many gaps arising from ice cover, weather-related access constraints, and occasional human errors, complicating
forecasting and early warning of harmful algal blooms. We study Secchi Disk Depth (SDD)
forecasting on a 30-lake, data-rich subset drawn from three decades of in-situ records collected across Maine lakes. Missingness is handled via Multiple Imputation by Chained Equations (MICE), and we evaluate performance with a normalized Mean Absolute Error (nMAE) metric for cross-lake comparability. Among six candidates, ridge regression provides the best mean test performance. Using ridge regression, we then quantify the minimal sample size, showing that under a backward, recent-history protocol, the model reaches within 5% of full-history accuracy with approximately 176 training samples per lake on average. We also identify a minimal feature set, where a compact four-feature subset matches the thirteen-feature baseline within the same 5% tolerance. Bringing these results together, we introduce a joint feasibility function that identifies the minimal training history and fewest predictors sufficient to achieve the target of staying within 5% of the complete-history, full-feature baseline. In our study, meeting the 5% accuracy target required about 64 recent samples and just one predictor per lake, highlighting the practicality of targeted monitoring. Hence, our joint feasibility strategy unifies recent-history length and feature choice under a fixed accuracy target, yielding a simple, efficient rule for setting sampling effort and measurement priorities for lake researchers.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Low-Dimensional Adaptation of Rectified Flow: A Diffusion and Stochastic Localization Perspective
Saptarshi Roy, Alessandro Rinaldo and Purnamrita Sarkar
In recent years, Rectified flow (RF) has gained considerable popularity largely due to its generation efficiency and state-of-the-art performance. In this paper, we investigate how well RF automatically adapts to the intrinsic low dimensionality of the support of the target distribution to accelerate sampling. We show that, using a carefully designed \blue{end-point adapted} time-discretization scheme and with sufficiently accurate drift estimates, the RF sampler enjoys an iteration complexity of order $O(k/\varepsilon)$ (up to log factors), where $\varepsilon$ is the precision in total variation distance and $k$ is the intrinsic dimension of the target distribution. In addition, we show that the denoising
diffusion probabilistic model (DDPM) procedure is equivalent to a
stochastic version of RF by establishing a novel connection between these processes and
stochastic localization. Building on this connection, we further design a
stochastic RF sampler that also adapts to the low-dimensionality of the target distribution under mild requirements on the accuracy of the drift estimates, and also with a specific time schedule. We illustrate the efficacy of newly designed time-discretization schedules with simulations on the synthetic data and text-to-image (T2I) data experiments.
Read the full abstract ▾
Machine Learning
MARS: Unleashing the Power of Speculative Decoding via Margin-Aware Verification
Jingwei Song, Xinyu Wang, Hanbin Wang et al. (+6 more)
Speculative Decoding (SD) accelerates autoregressive large
language model (LLM) inference by decoupling generation and verification. While recent methods improve draft quality by tightly coupling the drafter with the target model, the verification mechanism itself remains largely unchanged, relying on strict
token-level rejection sampling. In practice, modern LLMs frequently operate in low-margin regimes where the target model exhibits weak preference among top candidates. In such cases, rejecting plausible runner-up
tokens yields negligible information gain while incurring substantial rollback cost, leading to a fundamental inefficiency in verification. We propose Margin-Aware Speculative Verification, a training-free and domain-agnostic verification strategy that adapts to the target model's local decisiveness. Our method conditions verification on decision stability measured directly from the target logits and relaxes rejection only when strict verification provides minimal benefit. Importantly, the approach modifies only the verification rule and is fully compatible with existing target-coupled speculative decoding frameworks. Extensive experiments across model scales ranging from 8B to 235B demonstrate that our method delivers consistent and significant inference speedups over state-of-the-art baselines while preserving generation quality across diverse
benchmarks. The code is available at https://github.com/5SSjw/MARS.
Read the full abstract ▾
All authors Jingwei Song, Xinyu Wang, Hanbin Wang, Xiaoxuan Lei, Bill Shi, Shixin Han, Eric Yang, Xiao-Wen Chang, Lynn Ai
Machine Learning
Martingale Foresight Sampling: A Principled Approach to Inference-Time LLM Decoding
Huayu Li, ZhengXiao He, Siyuan Tian et al. (+2 more)
Standard autoregressive decoding in large
language models (LLMs) is inherently short-sighted, often failing to find globally optimal
reasoning paths due to its
token-by-
token generation process. While inference-time strategies like foresight sampling attempt to mitigate this by simulating future steps, they typically rely on ad-hoc heuristics for valuing paths and
pruning the search space. This paper introduces Martingale Foresight Sampling (MFS), a principled framework that reformulates LLM decoding as a problem of identifying an optimal
stochastic process. By modeling the quality of a
reasoning path as a
stochastic process, we leverage Martingale theory to design a theoretically-grounded
algorithm. Our approach replaces heuristic mechanisms with principles from probability theory: step valuation is derived from the Doob Decomposition
Theorem to measure a path's predictable advantage, path selection uses Optional Stopping Theory for principled
pruning of suboptimal candidates, and an adaptive stopping rule based on the Martingale Convergence
Theorem terminates exploration once a path's quality has provably converged. Experiments on six
reasoning benchmarks demonstrate that MFS surpasses state-of-the-art methods in accuracy while significantly improving computational efficiency. Code will be released at https://github.com/miraclehetech/EACL2026-Martingale-Foresight-Sampling.
Read the full abstract ▾
All authors Huayu Li, ZhengXiao He, Siyuan Tian, Jinghao Wen, Ao Li
Machine Learning
Early predicting of hospital admission using machine learning algorithms: Priority queues approach
Jakub Antczak, James Montgomery, Małgorzata O'Reilly et al. (+2 more)
Emergency Department overcrowding is a critical issue that compromises patient safety and operational efficiency, necessitating accurate demand
forecasting for effective resource allocation. This study evaluates and compares three distinct predictive models: Seasonal AutoRegressive Integrated Moving Average with eXogenous regressors (SARIMAX), EXtreme Gradient Boosting (XGBoost) and Long Short-Term
Memory (LSTM) networks for
forecasting daily ED arrivals over a seven-day horizon. Utilizing data from an Australian tertiary referral hospital spanning January 2017 to December 2021, this research distinguishes itself by decomposing demand into eight specific ward categories and stratifying patients by clinical complexity. To address data distortions caused by the COVID-19 pandemic, the study employs the Prophet model to generate synthetic counterfactual values for the anomalous period. Experimental results demonstrate that all three proposed models consistently outperform a seasonal naive baseline. XGBoost demonstrated the highest accuracy for predicting total daily admissions with a Mean Absolute Error of 6.63, while the statistical SARIMAX model proved marginally superior for
forecasting major complexity cases with an MAE of 3.77. The study concludes that while these techniques successfully reproduce regular day-to-day patterns, they share a common limitation in underestimating sudden, infrequent surges in patient volume.
Read the full abstract ▾
All authors Jakub Antczak, James Montgomery, Małgorzata O'Reilly, Zbigniew Palmowski, Richard Turner
Machine Learning
BadImplant: Injection-based Multi-Targeted Graph Backdoor Attack
Md Nabi Newaz Khan, Abdullah Arafat Miah and Yu Bi
Graph neural network (GNN) have demonstrated exceptional performance in solving critical problems across diverse domains yet remain susceptible to
backdoor attacks. Existing studies on
backdoor attack for graph classification are limited to single target attack using subgraph replacement based mechanism where the attacker implants only one trigger into the GNN model. In this paper, we introduce the first multi-targeted
backdoor attack for graph classification task, where multiple triggers simultaneously redirect predictions to different target labels. Instead of subgraph replacement, we propose subgraph injection which preserves the structure of the original graphs while poisoning the clean graphs. Extensive experiments demonstrate the efficacy of our approach, where our attack achieves high attack success rates for all target labels with minimal impact on the clean accuracy. Experimental results on five
dataset demonstrate the superior performance of our attack framework compared to the conventional subgraph replacement-based attack. Our analysis on four GNN models confirms the generalization capability of our attack which is effective regardless of the GNN model architectures and training parameters settings. We further investigate the impact of the attack design parameters including injection methods, number of connections, trigger sizes, trigger edge density and poisoning ratios. Additionally, our evaluation against state-of-the-art defenses (randomized smoothing and fine-
pruning) demonstrates the robustness of our proposed multi-target attacks. This work highlights the GNN vulnerability against multi-targeted
backdoor attack in graph classification task. Our source codes will be available at https://github.com/SiSL-URI/Multi-Targeted-Graph-
Backdoor-Attack.
Read the full abstract ▾
Machine Learning
Panther: Faster and Cheaper Computations with Randomized Numerical Linear Algebra
Fahd Seddik, Abdulrahman Elbedewy, Gaser Sami et al. (+2 more)
Training modern deep learning models is increasingly constrained by GPU
memory and compute limits. While Randomized Numerical Linear
Algebra (RandNLA) offers proven techniques to compress these models, the lack of a unified, production-grade library prevents widely adopting these methods. We present Panther, a PyTorch-compatible library that consolidates established RandNLA
algorithms into a single high-performance framework. Panther engineers efficient, drop-in replacements for standard components including sketched linear layers, 2D convolution, multi-head
attention, and randomized
matrix decompositions (such as pivoted CholeskyQR). By implementing a custom C++/CUDA backend (pawX), Panther provides an optimized implementation that can run on both CPUs and GPUs. We demonstrate the effectiveness of RandNLA techniques and Panther's ease of adoption. By replacing standard PyTorch linear layers with Panther layers (requiring only a few lines of code) we achieve significant
memory savings (up to 75%) on BERT while maintaining comparable loss. Source code is available (MIT License) at https://github.com/FahdSeddik/panther, along with demonstration
video at https://youtu.be/7M3RQb4KWxs.
Read the full abstract ▾
All authors Fahd Seddik, Abdulrahman Elbedewy, Gaser Sami, Mohamed Abdelmoniem, Yahia Zakaria
Machine Learning
Learning from Synthetic Data: Limitations of ERM
Kareem Amin, Alex Bie, Weiwei Kong et al. (+2 more)
The prevalence and low cost of LLMs have led to a rise of synthetic content. From review sites to court documents, "natural" content has been contaminated by data points that appear similar to natural data, but are in fact LLM-generated. In this work we revisit fundamental learning theory questions in this, now ubiquitous, setting. We model this scenario as a sequence of learning tasks where the input is a mix of natural and synthetic data, and the learning
algorithms are oblivious to the origin of any individual example. We study the possibilities and limitations of ERM in this setting. For the problem of estimating the mean of an arbitrary $d$-dimensional distribution, we find that while ERM converges to the true mean, it is outperformed by an
algorithm that assigns non-uniform weights to examples from different generations of data. For the PAC learning setting, the disparity is even more stark. We find that ERM does not always converge to the true concept, echoing the model collapse literature. However, we show there are
algorithms capable of learning the correct hypothesis for arbitrary VC classes and arbitrary amounts of contamination.
Read the full abstract ▾
All authors Kareem Amin, Alex Bie, Weiwei Kong, Umar Syed, Sergei Vassilvitskii
Machine Learning · Cross-listed from cs.AI
A tensor network formalism for neuro-symbolic AI
Alex Goessmann, Janina Schütte, Maximilian Fröhlich et al. (+1 more)
The unification of neural and symbolic approaches to artificial intelligence remains a central open challenge. In this work, we introduce a tensor network formalism, which captures sparsity principles originating in the different approaches in tensor decompositions. In particular, we describe a basis encoding scheme for functions and model neural decompositions as tensor decompositions. The proposed formalism can be applied to represent logical formulas and probability distributions as structured tensor decompositions. This unified treatment identifies tensor network contractions as a fundamental inference class and formulates efficiently scaling
reasoning algorithms, originating from probability theory and propositional logic, as contraction message passing schemes. The framework enables the definition and training of hybrid logical and probabilistic models, which we call Hybrid Logic Network. The theoretical concepts are accompanied by the python library tnreason, which enables the implementation and practical use of the proposed architectures.
Read the full abstract ▾
All authors Alex Goessmann, Janina Schütte, Maximilian Fröhlich, Martin Eigel
Machine Learning
CASL: Concept-Aligned Sparse Latents for Interpreting Diffusion Models
Zhenghao He, Guangzhi Xiong, Boyang Wang et al. (+2 more)
Internal activations of
diffusion models encode rich semantic information, but interpreting such representations remains challenging. While Sparse Autoencoders (SAEs) have shown promise in disentangling latent representations, existing SAE-based methods for
diffusion model understanding rely on unsupervised approaches that fail to align sparse features with human-understandable concepts. This limits their ability to provide reliable semantic control over generated images. We introduce CASL (Concept-Aligned Sparse Latents), a supervised framework that aligns sparse latent dimensions of
diffusion models with semantic concepts. CASL first trains an SAE on frozen U-Net activations to obtain disentangled latent representations, and then learns a lightweight linear mapping that associates each concept with a small set of relevant latent dimensions. To validate the semantic meaning of these aligned directions, we propose CASL-Steer, a controlled latent intervention that shifts activations along the learned concept axis. Unlike editing methods, CASL-Steer is used solely as a causal probe to reveal how concept-aligned latents influence generated content. We further introduce the Editing Precision Ratio (EPR), a metric that jointly measures concept specificity and the preservation of unrelated attributes. Experiments show that our method achieves superior editing precision and interpretability compared to existing approaches. To the best of our knowledge, this is the first work to achieve supervised
alignment between latent representations and semantic concepts in
diffusion models.
Read the full abstract ▾
All authors Zhenghao He, Guangzhi Xiong, Boyang Wang, Sanchit Sinha, Aidong Zhang
Machine Learning
Lattice: A Confidence-Gated Hybrid System for Uncertainty-Aware Sequential Prediction with Behavioral Archetypes
Lorian Bannis
We introduce
Lattice, a hybrid sequential prediction system that conditionally activates learned behavioral structure using binary confidence gating. The system summarizes behavior windows as behavioral archetypes and activates archetype-based scoring only when an in-support confidence signal exceeds a validation-calibrated threshold, falling back to backbone predictions when uncertain. Our primary estimand is the controlled effect of adding
Lattice to a fixed backbone on identical test rows. On MovieLens (30 paired seeds, full-catalog ranking), LSTM+
Lattice improves HR@10 by +31.7% (gated) versus the LSTM backbone alone (p much less than 10^-20); ungated fusion reaches +58.7% on the same protocol. We do not claim gating maximizes pooled accuracy. With backbone-native archetypes (fit in each backbone's
embedding space), gated lifts of +13.3% (
transformer) and +17.0% (SASRec) hold under the same evaluation design. A prior approximately 0%
transformer row in version 1 reflected an invalid cross-backbone transfer, not evidence that composition cannot help stronger encoders. Amazon Electronics provides supporting cross-domain evidence (+124.0% gated, 15 seeds, high variance). Controlled shift checks (appendix) illustrate gate refusal under distribution shift. Standalone SASRec and BERT4Rec scores are contextual references, not the target estimand. We report what composition achieves and when it activates; production calibration and implementation details remain proprietary pending patent prosecution.
Read the full abstract ▾
Machine Learning
Ambient Dataloops: Generative Models for Dataset Refinement
Adrián Rodríguez-Muñoz, William Daspit, Adam Klivans et al. (+3 more)
We propose Ambient Dataloops, an iterative framework for refining
datasets that makes it easier for
diffusion models to learn the underlying data distribution. Modern
datasets contain samples of highly varying quality, and training directly on such heterogeneous data often yields suboptimal models. We propose a
dataset-model co-
evolution process; at each iteration of our method, the
dataset becomes progressively higher quality, and the model improves accordingly. To avoid destructive self-consuming loops, at each generation, we treat the synthetically improved samples as noisy, but at a slightly lower noisy level than the previous iteration, and we use Ambient
Diffusion techniques for learning under corruption. Empirically, Ambient Dataloops achieve state-of-the-art performance in unconditional and text-conditional image generation and de novo
protein design. We further provide a theoretical justification for the proposed framework that captures the benefits of the data looping procedure.
Read the full abstract ▾
All authors Adrián Rodríguez-Muñoz, William Daspit, Adam Klivans, Antonio Torralba, Constantinos Daskalakis, Giannis Daras
Machine Learning · Cross-listed from cs.CV
CURE: Curriculum-guided Multi-task Training for Reliable Anatomy Grounded Report Generation
Pablo Messina, Andrés Villa, Juan León Alcázar et al. (+5 more)
Medical
vision-
language models can automate the generation of radiology reports but struggle with accurate visual grounding and factual consistency. Existing models often misalign textual findings with visual evidence, leading to unreliable or weakly grounded predictions. We present CURE, an error-aware curriculum learning framework that improves grounding and report quality without any additional data. CURE fine-tunes a
multimodal instructional model on phrase grounding, grounded report generation, and anatomy-grounded report generation using public
datasets. The method dynamically adjusts sampling based on model performance, emphasizing harder samples to improve spatial and textual
alignment. CURE improves grounding accuracy by +0.35 IoU, boosts report quality by +0.192 CXRFEScore, and reduces
hallucinations by 18.6%. CURE is a data-efficient framework that enhances both grounding accuracy and report reliability. Code is available at https://github.com/PabloMessina/CURE and model weights at https://huggingface.co/pamessina/medgemma-4b-it-cure
Read the full abstract ▾
All authors Pablo Messina, Andrés Villa, Juan León Alcázar, Karen Sánchez, Carlos Hinojosa, Denis Parra, Álvaro Soto, Bernard Ghanem
Machine Learning
Attention-Informed Surrogates for Navigating Power-Performance Trade-offs in HPC
Ashna Nawar Ahmed, Banooqa Banday, Terry Jones et al. (+1 more)
High-Performance Computing (HPC) schedulers must balance user performance with facility-wide resource constraints. The task boils down to selecting the optimal number of nodes for a given job. We present a surrogate-assisted multi-objective
Bayesian optimization (MOBO) framework to automate this complex decision. Our core hypothesis is that surrogate models informed by
attention-based
embeddings of job telemetry can capture performance dynamics more effectively than standard regression techniques. We pair this with an intelligent sample acquisition strategy to ensure the approach is data-efficient. On two production HPC
datasets, our
embedding-informed method consistently identified higher-quality Pareto fronts of runtime-power trade-offs compared to baselines. Furthermore, our intelligent data sampling strategy drastically reduced training costs while improving the stability of the results. To our knowledge, this is the first work to successfully apply
embedding-informed surrogates in a MOBO framework to the HPC scheduling problem, jointly optimizing for performance and power on production workloads.
Read the full abstract ▾
All authors Ashna Nawar Ahmed, Banooqa Banday, Terry Jones, Tanzima Z. Islam
Machine Learning · Cross-listed from cs.AI
GeMM-GAN: A Multimodal Generative Model Conditioned on Histopathology Images and Clinical Descriptions for Gene Expression Profile Generation
Francesca Pia Panaccione, Carlo Sgaravatti and Pietro Pinoli
Biomedical research increasingly relies on integrating diverse data modalities, including gene expression profiles, medical images, and clinical metadata. While medical images and clinical metadata are routinely collected in clinical practice, gene expression data presents unique challenges for widespread research use, mainly due to stringent
privacy regulations and costly laboratory experiments. To address these limitations, we present GeMM-GAN, a novel Generative
Adversarial Network conditioned on histopathology tissue slides and clinical metadata, designed to synthesize realistic gene expression profiles. GeMM-GAN combines a
Transformer Encoder for image patches with a final Cross
Attention mechanism between patches and text
tokens, producing a conditioning vector to guide a generative model in generating biologically coherent gene expression profiles. We evaluate our approach on the TCGA
dataset and demonstrate that our framework outperforms standard generative models and generates more realistic and functionally meaningful gene expression profiles, improving by more than 11% the accuracy on downstream disease type prediction compared to current state-of-the-art generative models. Code will be available at: https://github.com/francescapia/GeMM-GAN
Read the full abstract ▾
Machine Learning
FedUMM: A General Framework for Federated Learning with Unified Multimodal Models
Zhaolong Su, Leheng Zhao, Xiaoying Wu et al. (+2 more)
Unified
multimodal models (UMMs) are emerging as strong foundation models that can do both generation and understanding tasks in a single architecture. However, they are typically trained in centralized settings where all training and downstream
datasets are gathered in a central server, limiting the deployment in
privacy-sensitive and geographically distributed scenarios. In this paper, we present FedUMM, a general
federated learning framework for UMMs under non-IID
multimodal data with low communication cost. Built on NVIDIA
FLARE, FedUMM instantiates federation for a BLIP3o backbone via parameter-efficient fine-tuning: clients train lightweight LoRA adapters while freezing the foundation models, and the server aggregates only adapter updates. We evaluate on VQA v2 and the GenEval compositional generation
benchmarks under Dirichlet-controlled heterogeneity with up to 16 clients. Results show slight degradation as client count and heterogeneity increase, while remaining competitive with centralized training. We further analyze computation–communication trade-offs and demonstrate that adapter-only federation reduces per-round communication by over an order of magnitude compared to full fine-tuning, enabling practical
federated UMM training. This work provides empirical experience for future research on
privacy-preserving
federated unified
multimodal models.
Read the full abstract ▾
All authors Zhaolong Su, Leheng Zhao, Xiaoying Wu, Ziyue Xu, Jindong Wang