Machine Learning · Cross-listed from cs.CL
GATech at AbjadGenEval Shared Task: Multilingual Embeddings for Arabic Machine-Generated Text Classification
Ahmed Khaled Khamis
We present our approach to the AbjadGenEval shared task on detecting AI-generated Arabic text. We fine-tuned the
multilingual E5-large encoder for binary classification, and we explored several pooling strategies to pool
token representations, including weighted layer pooling, multi-head
attention pooling, and gated fusion. Interestingly, none of these outperformed simple mean pooling, which achieved an F1 of 0.75 on the test set. We believe this is because complex pooling methods introduce additional parameters that need more data to train properly, whereas mean pooling offers a stable baseline that generalizes well even with limited examples. We also observe a clear pattern in the data: human-written texts tend to be significantly longer than machine-generated ones.
Read the full abstract ▾
Machine Learning
Khatri-Rao Clustering for Data Summarization
Martino Ciaperoni, Collin Leiber, Aristides Gionis et al. (+1 more)
As
datasets continue to grow in size and complexity, finding succinct yet accurate data summaries poses a key challenge. Centroid-based clustering, a widely adopted approach to address this challenge, finds informative summaries of
datasets in terms of few prototypes, each representing a cluster in the data. Despite their wide adoption, the resulting data summaries often contain redundancies, limiting their effectiveness particularly in
datasets characterized by a large number of underlying clusters. To overcome this limitation, we introduce the Khatri-Rao clustering paradigm that extends traditional centroid-based clustering to produce more succinct but equally accurate data summaries by postulating that centroids arise from the interaction of two or more succinct sets of protocentroids. We study two central approaches to centroid-based clustering, namely the well-established k-Means
algorithm and the increasingly popular topic of deep clustering, under the lens of the Khatri-Rao paradigm. To this end, we introduce the Khatri-Rao k-Means
algorithm and the Khatri-Rao deep clustering framework. Extensive experiments show that Khatri-Rao k-Means can strike a more favorable trade-off between succinctness and accuracy in data summarization than standard k-Means. Leveraging representation learning, the Khatri-Rao deep clustering framework offers even greater benefits, reducing even more the size of data summaries given by deep clustering while preserving their accuracy.
Read the full abstract ▾
All authors Martino Ciaperoni, Collin Leiber, Aristides Gionis, Heikki Mannila
Machine Learning
Switchable Activation Networks
Laha Ale, Ning Zhang, Scott A. King et al. (+1 more)
Deep neural networks, and more recently large-scale generative models such as large
language models (LLMs) and large
vision-action models (LVAs), achieve remarkable performance across diverse domains, yet their prohibitive computational cost hinders deployment in resource-constrained environments. Existing efficiency techniques offer only partial remedies: dropout improves regularization during training but leaves inference unchanged, while
pruning and low-rank factorization compress models post hoc into static forms with limited adaptability. Here we introduce SWAN (Switchable Activation Networks), a framework that equips each neural unit with a deterministic, input-dependent binary gate, enabling the network to learn when a unit should be active or inactive. This dynamic control mechanism allocates computation adaptively, reducing redundancy while preserving accuracy. Unlike traditional
pruning, SWAN does not simply shrink networks after training; instead, it learns structured, context-dependent activation patterns that support both efficient dynamic inference and conversion into compact dense models for deployment. By reframing efficiency as a problem of learned activation control, SWAN unifies the strengths of sparsity,
pruning, and adaptive inference within a single paradigm. Beyond computational gains, this perspective suggests a more general principle of neural computation, where activation is not fixed but context-dependent, pointing toward sustainable AI, edge intelligence, and future architectures inspired by the adaptability of biological
brains.
Read the full abstract ▾
All authors Laha Ale, Ning Zhang, Scott A. King, Pingzhi Fan
Machine Learning
FuzzingRL: Reinforcement Fuzz-Testing for Revealing VLM Failures
Jiajun Xu, Jiageng Mao, Ang Qi et al. (+5 more)
Vision Language Models (VLMs) are prone to errors, and identifying where these errors occur is critical for ensuring the reliability and safety of AI systems. In this paper, we propose an approach that automatically generates questions designed to deliberately induce incorrect responses from VLMs, thereby revealing their vulnerabilities. The core of this approach lies in fuzz testing and reinforcement finetuning: we transform a single input query into a large set of diverse variants through
vision and
language fuzzing. Based on the fuzzing outcomes, the question generator is further instructed by
adversarial reinforcement fine-tuning to produce increasingly challenging queries that trigger model failures. With this approach, we can consistently drive down a target VLM's answer accuracy – for example, the accuracy of Qwen2.5-VL-32B on our generated questions drops from 86.58% to 65.53% in four RL iterations. Moreover, a fuzzing policy trained against a single target VLM transfers to multiple other VLMs, producing challenging queries that degrade their performance as well.
Read the full abstract ▾
All authors Jiajun Xu, Jiageng Mao, Ang Qi, Weiduo Yuan, Alexander Romanus, Helen Xia, Vitor Campagnolo Guizilini, Yue Wang
Machine Learning
Uncertainty-Calibrated Spatiotemporal Field Diffusion with Sparse Supervision
Kevin Valencia, Xihaier Luo, Shinjae Yoo et al. (+1 more)
Physical fields are typically observed only at sparse, time-varying sensor locations, making
forecasting and reconstruction ill-posed and uncertainty-critical. We present SOLID, a mask-conditioned
diffusion framework that learns spatiotemporal dynamics from sparse observations alone: training and evaluation use only observed target locations, requiring no dense fields and no pre-imputation. Unlike prior work that trains on dense reanalysis or simulations and only tests under sparsity, SOLID is trained end-to-end with sparse supervision only. SOLID conditions each denoising step on the measured values and their locations, and introduces a dual-masking objective that (i) emphasizes learning in unobserved void regions while (ii) upweights overlap pixels where inputs and targets provide the most reliable anchors. This strict sparse-conditioning pathway enables posterior sampling of full fields consistent with the measurements, achieving up to an order-of-magnitude improvement in probabilistic error and yielding calibrated uncertainty maps (h̊o} > 0.7) under severe sparsity.
Read the full abstract ▾
All authors Kevin Valencia, Xihaier Luo, Shinjae Yoo, David Keetae Park
Machine Learning
Flowers: A Warp Drive for Neural PDE Solvers
Till Muser, Alexandra Spitzer, Matti Lassas et al. (+2 more)
We introduce Flowers, a neural architecture for learning PDE solution operators built entirely from multihead warps. Aside from pointwise channel mixing and a multiscale scaffold, Flowers use no
Fourier multipliers, no dot-product
attention, and no convolutional mixing. Each head predicts a displacement field and warps the mixed input features. Motivated by physics and computational efficiency, displacements are predicted pointwise, without any spatial aggregation, and nonlocality enters only through sparse sampling at source coordinates, one per head. Stacking warps in multiscale residual blocks yields Flowers, which implement adaptive, global interactions at linear cost. We theoretically motivate this design through three complementary lenses: flow maps for conservation laws, waves in inhomogeneous media, and a kinetic-theoretic continuum limit. Flowers achieve excellent performance on a broad suite of 2D and 3D time-dependent PDE
benchmarks, particularly flows and waves. A compact 17M-parameter model consistently outperforms
Fourier, convolution, and
attention-based baselines of similar size, while a 150M-parameter variant improves over recent
transformer-based foundation models with much more parameters, data, and training compute.
Read the full abstract ▾
All authors Till Muser, Alexandra Spitzer, Matti Lassas, Maarten V. de Hoop, Ivan Dokmanić
Machine Learning
Agent Memory Below the Prompt: Persistent Q4 KV Cache for Multi-Agent LLM Inference on Edge Devices
Yakov Pyotr Shkolnikov
Multi-
agent LLM systems on edge devices face a
memory management problem: device RAM is too small to hold every
agent's KV cache simultaneously. On Apple M4 Pro with 10.2 GB of cache budget, only 3
agents fit at 8K context in FP16. A 10-
agent workflow must constantly evict and reload caches. Without persistence, every eviction forces a full re-prefill through the model – 15.7 seconds per
agent at 4K context. We address this by persisting each
agent's KV cache to disk in 4-bit quantized format and reloading it directly into the
attention layer, eliminating redundant O(n) prefill computation via direct cache restoration. The system comprises three components: a block pool providing per-
agent isolated Q4 KV caches in safetensors format, a BatchQuantizedKVCache for concurrent inference over multiple
agents' quantized caches, and cross-phase context injection that accumulates
attention state across conversation phases without re-computation. Evaluated on three architectures (Gemma 3 12B, dense GQA, 48 layers; DeepSeek-Coder-V2-Lite 16B, MoE MLA, 27 layers; Llama 3.1 8B, dense GQA, 32 layers), cache restoration reduces time-to-first-
token by up to 136x (Gemma: 22–136x at 4K–32K; DeepSeek: 11–76x at 4K–32K; Llama: 24–111x at 4K–16K; 3–10x at 1K). Q4 quantization fits 4x more
agent contexts into fixed device
memory than FP16. Perplexity measured with actual Q4 KV caches shows -0.7% for Gemma, +2.8% for Llama, and +3.0% for DeepSeek. Open-source at https://github.com/yshk-mxim/
agent-
memory
Read the full abstract ▾
Machine Learning
Wideband Power Amplifier Behavioral Modeling Using an Amplitude Conditioned LSTM
Abdelrahman Abdelsalam and You Fei
Wideband power amplifiers exhibit complex nonlinear and
memory effects that challenge traditional behavioral modeling approaches. This paper proposes a novel amplitude conditioned long short-term
memory (AC-LSTM) network that introduces explicit amplitude-dependent gating to enhance the modeling of wideband PA dynamics. The architecture incorporates a Feature-wise Linear Modulation (FiLM) layer that conditions the LSTM's forget gate on the instantaneous input amplitude, providing a physics-aware inductive bias for capturing amplitude-dependent
memory effects. Experimental validation using a 100 MHz 5G NR signal and a GaN PA demonstrates that the proposed AC-LSTM achieves a normalized mean square error (NMSE) of -41.25 dB, representing a 1.15 dB improvement over standard LSTM and 7.45 dB improvement over augmented real-valued time-delay neural network (ARVTDNN) baselines. The model also closely matches the measured PA's spectral characteristics with an adjacent channel power ratio (ACPR) of -28.58 dB. These results shows the effectiveness of amplitude conditioning for improving both time-domain accuracy and spectral fidelity in wide-band PA behavioral modeling.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.AP
Using Artificial Neural Networks to Predict Claim Duration in a Work Injury Compensation Environment
Anthony Almudevar
Currently, work injury compensation boards in Canada track injury information using a standard system of codes (under the National Work Injury Statistics Program (NWISP)). These codes capture the medical nature and original cause of the injury in some detail, hence they potentially contain information which may be used to predict the severity of an injury and the resulting time loss from work. Claim duration easurements and forecasts are central to the operation of a work injury compensation program. However, due to the complexity of the codes traditional statistical modelling techniques are of limited value. We will describe an artificial neural network implementation of Cox proportional hazards regression due to Ripley (1998 thesis) which is used as the basis for a model for the prediction of claim duration within a work injury compensation environment. The model accepts as input the injury codes, as well as basic demographic and workplace information. The output consists of a claim duration prediction in the form of a distribution. The input represents information available when a claim is first filed, and may therefore be used in a claims management setting. We will describe the model selection procedure, as well as a procedure for accepting inputs with missing
covariates.
Read the full abstract ▾
Machine Learning
SEval-NAS: A Search-Agnostic Evaluation for Neural Architecture Search
Atah Nuh Mih, Jianzhou Wang, Truong Thanh Hung Nguyen et al. (+1 more)
Neural architecture search (NAS) automates the discovery of neural networks that meet specified criteria, yet its evaluation procedures are often hardcoded, limiting the ability to introduce new metrics. This issue is especially pronounced in hardware-aware NAS, where objectives depend on target devices such as edge hardware. To address this limitation, we propose SEval-NAS, a metric-evaluation mechanism that converts architectures to strings, embeds them as vectors, and predicts performance metrics. Using NATS-Bench and HW-NAS-Bench, we evaluated accuracy, latency, and
memory. Kendall's $\tau$ correlations showed stronger latency and
memory predictions than accuracy, indicating the suitability of SEval-NAS as a hardware cost predictor. We further integrated SEval-NAS into FreeREA to evaluate metrics not originally included. The method successfully ranked FreeREA-generated architectures, maintained search time, and required minimal algorithmic changes. Our implementation is available at: https://github.com/Analytics-Everywhere-Lab/neural-architecture-search
Read the full abstract ▾
All authors Atah Nuh Mih, Jianzhou Wang, Truong Thanh Hung Nguyen, Hung Cao
Machine Learning · Cross-listed from stat.OT
Profiling vs. Case-specific Evidence: A Probabilistic Analysis
Marcello Di Bello, Nicolò Cangiotti and Michele Loi
The use of profiling evidence in criminal trials is a longstanding controversy in legal epistemology and evidence law theory. Many scholars, even when they oppose its use at trial, still assume that profiling evidence can be probative of guilt. We reject that assumption. Profiling evidence may support a generic hypothesis, but is not evidence that the defendant is guilty of the specific crime of which they are accused. We contrast profiling evidence with case-specific evidence, which speaks more directly to the facts of the case. Our critique departs from others by grounding the argument in a probabilistic analysis of evidentiary value. We also explore the implications of our account for debates about stereotyping.
Read the full abstract ▾
Machine Learning · Cross-listed from q-bio.BM
Exploring Drug Safety Through Knowledge Graphs: Protein Kinase Inhibitors as a Case Study
David Jackson, Michael Gertz and Jürgen Hesser
Adverse Drug Reactions (ADRs) are a leading cause of morbidity and mortality. Existing prediction methods rely mainly on chemical similarity, machine learning on structured databases, or isolated target profiles, but often fail to integrate heterogeneous, partly unstructured evidence effectively. We present a knowledge graph-based framework that unifies diverse sources, drug-target data (ChEMBL), clinical trial literature (PubMed), trial metadata (ClinicalTrials.gov), and post-marketing safety reports (FAERS) into a single evidence-weighted bipartite network of drugs and medical conditions. Applied to 400
protein kinase inhibitors, the resulting network enables contextual comparison of efficacy (HR, PFS, OS), phenotypic and target similarity, and ADR prediction via target-to-adverse-event correlations. A non-small
cell lung cancer case study correctly highlights established and candidate drugs, target communities (ERbB, ALK, VEGF), and tolerability differences. Designed as an orthogonal, extensible analysis and search tool rather than a replacement for current models, the framework excels at revealing complex patterns, supporting hypothesis generation, and enhancing pharmacovigilance. Code and data are publicly available at https://github.com/davidjackson99/PKI_KG.
Read the full abstract ▾
Machine Learning · Cross-listed from physics.comp-ph
Adaptive Uncertainty-Guided Surrogates for Efficient phase field Modeling of Dendritic Solidification
Eider Garate-Perez, Kerman López de Calle-Etxabe, Oihana Garcia et al. (+3 more)
The high computational cost of phase field simulations remains a major limitation for predicting dendritic solidification in metals, particularly in additive manufacturing, where microstructural control is critical. This work presents a surrogate model for dendritic solidification that employs uncertainty-driven adaptive sampling with XGBoost and CNNs, including a self-supervised strategy, to efficiently approximate the spatio-temporal
evolution while reducing costly phase field simulations. The proposed adaptive strategy leverages model uncertainty, approximated via Monte Carlo dropout for CNNs and bagging for XGBoost, to identify high-uncertainty regions where new samples are generated locally within hyperspheres, progressively refining the spatio-temporal design space and achieving accurate predictions with significantly fewer phase field simulations than an Optimal Latin Hypercube Sampling optimized via discrete Particle Swarm Optimization (OLHS-PSO). The framework systematically investigates how temporal instance selection, adaptive sampling, and the choice between domain-informed and data-driven surrogates affect spatio-temporal model performance. Evaluation considers not only computational cost but also the number of expensive phase field simulations, surrogate accuracy, and associated $CO_2$ emissions, providing a comprehensive assessment of model performance as well as their related environmental impact.
Read the full abstract ▾
All authors Eider Garate-Perez, Kerman López de Calle-Etxabe, Oihana Garcia, Borja Calvo, Meritxell Gómez-Omella, Jon Lambarri
Machine Learning · Cross-listed from physics.comp-ph
A comparative study of transformer models and recurrent neural networks for path-dependent composite materials
Petter Uvdal and Mohsen Mirkhalaf
Accurate modeling of Short Fiber Reinforced Composites (SFRCs) remains computationally expensive for full-field simulations. Data-driven surrogate models using Artificial Neural Networks (ANNs) have been proposed as an efficient alternative to numerical modeling, where Recurrent Neural Networks (RNNs) are increasingly being used for path-dependent multiscale modeling by predicting the homogenized response of a Representative Volume Element (RVE). However, recently,
transformer models have been developed and they offer scalability and efficient parallelization, yet have not been systematically compared with RNNs in this field. In this study, we perform a systematic comparison between RNNs and
transformer models trained on sequences of homogenized response of SFRC RVEs. We study the effect on two types of hyperparameters, namely architectural hyperparameters (such as the number of GRU layers, hidden size, number of
attention heads, and encoder blocks) and training hyperparameters (such as learning rate and batch size). Both sets of hyperparameters are tuned using
Bayesian optimization. We then analyze scaling laws with respect to
dataset size and inference accuracy in interpolation and extrapolation regimes. The results show that while
transformer models remain competitive in terms of accuracy on large
datasets, the RNNs demonstrate better accuracy on small
datasets and show better extrapolation performance. Furthermore, under extrapolation, there is a clear difference, where the RNN remains accurate, while the
transformer model performs poorly. On the other hand, the
transformer model is 7 times faster at inference, requiring 0.5 ms per prediction compared to the 3.5 ms per prediction for the RNN model.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Mitigating Shortcut Learning via Feature Disentanglement in Medical Imaging: A Benchmark Study
Sarah Müller and Philipp Berens
Although deep learning models in medical imaging often achieve excellent classification performance, they can rely on shortcut learning, exploiting spurious correlations or confounding factors that are not causally related to the target task. This poses risks in clinical settings, where models must generalize across institutions, populations, and acquisition conditions. Feature disentanglement is a promising approach to mitigate shortcut learning by separating task-relevant information from confounder-related features in latent representations. In this study, we systematically evaluated feature disentanglement methods for mitigating shortcuts in medical imaging, including
adversarial learning and latent space splitting based on dependence minimization. We assessed classification performance and disentanglement quality using latent space analyses across one artificial and two medical
datasets with natural and synthetic confounders. We also examined robustness under varying levels of confounding and compared computational efficiency across methods. We found that shortcut mitigation methods improved classification performance under strong spurious correlations during training. Latent space analyses revealed differences in representation quality not captured by classification metrics, highlighting the strengths and limitations of each method. Model reliance on shortcuts depended on the degree of confounding in the training data. The best-performing models combine data-centric rebalancing with model-centric disentanglement, achieving stronger and more robust shortcut mitigation than rebalancing alone while maintaining similar computational efficiency. The project code is publicly available at https://github.com/berenslab/medical-shortcut-mitigation.
Read the full abstract ▾
Machine Learning
Quantifying LLM Attention-Head Stability: Implications for Circuit Universality
Karan Bali, Jack Stanley, Praneet Suresh et al. (+1 more)
In mechanistic interpretability, recent work scrutinizes
transformer "
circuits" - sparse, mono or multi layer sub computations, that may reflect human understandable functions. Yet, these network
circuits are rarely acid-tested for their stability across different instances of the same deep learning architecture. Without this, it remains unclear whether reported
circuits emerge universally across labs or turn out to be idiosyncratic to a particular estimation instance, potentially limiting confidence in safety-critical settings. Here, we systematically study stability across-refits in increasingly complex
transformer language models of various sizes. We quantify, layer by layer, how similarly
attention heads learn representations across independently initialized training runs. Our rigorous experiments show that (1) middle-layer heads are the least stable yet the most representationally distinct; (2) deeper models exhibit stronger mid-depth divergence; (3) unstable heads in deeper layers become more functionally important than their peers from the same layer; (4) applying weight decay optimization substantially improves
attention-head stability across random model initializations; and (5) the residual stream is comparatively stable. Our findings establish the cross-instance robustness of
circuits as an essential yet underappreciated prerequisite for scalable oversight, drawing contours around possible white-box monitorability of AI systems.
Read the full abstract ▾
All authors Karan Bali, Jack Stanley, Praneet Suresh, Danilo Bzdok
Machine Learning
Real-time Secondary Crash Likelihood Prediction Excluding Post Primary Crash Features
Lei Han, Mohamed Abdel-Aty, Zubayer Islam et al. (+1 more)
Secondary crash likelihood prediction is a critical component of an active traffic management system to mitigate congestion and adverse impacts caused by secondary crashes. However, existing approaches mainly rely on post-crash features (e.g., crash type and severity) that are rarely available in real time, limiting their practical applicability. To address this limitation, we propose a hybrid secondary crash likelihood prediction framework that does not depend on post-crash features. A dynamic spatiotemporal window is designed to extract real-time traffic flow and environmental features from primary crash locations and their upstream segments. The framework includes three models: a primary crash model to estimate the likelihood of secondary crash occurrence, and two secondary crash models to evaluate traffic conditions at crash and upstream segments under different comparative scenarios. An ensemble learning strategy integrating six machine learning
algorithms is developed to enhance predictive performance, and a voting-based mechanism combines the outputs of the three models. Experiments on Florida freeways demonstrate that the proposed hybrid framework correctly identifies 91% of secondary crashes with a low false alarm rate of 0.20. The Area Under the ROC Curve improves from 0.654, 0.744, and 0.902 for the individual models to 0.952 for the hybrid model, outperforming previous studies.
Read the full abstract ▾
All authors Lei Han, Mohamed Abdel-Aty, Zubayer Islam, Chenzhu Wang
Machine Learning · Cross-listed from cs.MA
Self-Evolving Multi-Agent Network for Industrial IoT Predictive Maintenance
Rebin Saleh, Khanh Pham Dinh, Balázs Villányi et al. (+1 more)
Industrial IoT predictive maintenance requires systems capable of real-time anomaly detection without sacrificing interpretability or demanding excessive computational resources. Traditional approaches rely on static, offline-trained models that cannot adapt to evolving operational conditions, while LLM-based monolithic systems demand prohibitive
memory and latency, rendering them impractical for on-site edge deployment. We introduce SEMAS, a self-evolving hierarchical multi-
agent system that distributes specialized
agents across Edge, Fog, and Cloud computational tiers. Edge
agents perform lightweight feature extraction and pre-filtering; Fog
agents execute diversified ensemble detection with dynamic consensus voting; and Cloud
agents continuously optimize system policies via Proximal Policy Optimization (PPO) while maintaining asynchronous, non-blocking inference. The framework incorporates LLM-based response generation for explainability and
federated knowledge aggregation for adaptive policy distribution. This architecture enables resource-aware specialization without sacrificing real-time performance or model interpretability. Empirical evaluation on two industrial
benchmarks (Boiler Emulator and Wind Turbine) demonstrates that SEMAS achieves superior anomaly detection performance with exceptional stability under adaptation, sustains prediction accuracy across evolving operational contexts, and delivers substantial latency improvements enabling genuine real-time deployment. Ablation studies confirm that PPO-driven policy
evolution, consensus voting, and
federated aggregation each contribute materially to system effectiveness. These findings indicate that resource-aware, self-evolving 1multi-
agent coordination is essential for production-ready industrial IoT predictive maintenance under strict latency and explainability constraints.
Read the full abstract ▾
All authors Rebin Saleh, Khanh Pham Dinh, Balázs Villányi, Truong-Son Hy
Machine Learning · Cross-listed from q-bio.QM
Exploring the Utility of MALDI-TOF Mass Spectrometry and Antimicrobial Resistance in Hospital Outbreak Detection
Chang Liu, Jieshi Chen, Alexander J. Sundermann et al. (+5 more)
Accurate and timely identification of hospital outbreak clusters is crucial for preventing the spread of infections that have epidemic potential. While assessing pathogen similarity through whole genome sequencing (WGS) is considered the gold standard for outbreak detection, its high cost and lengthy turnaround time preclude routine implementation in clinical laboratories. We explore the utility of two rapid and cost-effective alternatives to WGS,
matrix-assisted
laser desorption ionization-time of flight (MALDI-TOF) mass spectrometry and antimicrobial resistance (AR) patterns. We develop a machine learning framework that extracts informative representations from MALDI-TOF spectra and AR patterns for outbreak detection and explore their fusion. Through multi-species analyses, we demonstrate that in some cases MALDI-TOF and AR have the potential to reduce reliance on WGS, enabling more accessible and rapid outbreak surveillance.
Read the full abstract ▾
All authors Chang Liu, Jieshi Chen, Alexander J. Sundermann, Kathleen Shutt, Marissa P. Griffith, Lora Lee Pless, Lee H. Harrison, Artur W. Dubrawski
Machine Learning
A Few-Shot LLM Framework for Extreme Day Classification in Electricity Markets
Saud Alghumayjan, Ming Yi and Bolun Xu
This paper proposes a few-shot classification framework based on Large
Language Models (LLMs) to predict whether the next day will have spikes in real-time electricity prices. The approach aggregates system state information, including electricity demand, renewable generation, weather forecasts, and recent electricity prices, into a set of statistical features that are formatted as natural-
language prompts and fed to an LLM along with general instructions. The model then determines the likelihood that the next day would be a spike day and reports a confidence score. Using historical data from the Texas electricity
market, we demonstrate that this few-shot approach achieves performance comparable to supervised machine learning models, such as Support Vector Machines and XGBoost, and outperforms the latter two when limited historical data are available. These findings highlight the potential of LLMs as a data-efficient tool for classifying electricity price spikes in settings with scarce data.
Read the full abstract ▾
Machine Learning
MMCAformer: Macro-Micro Cross-Attention Transformer for Traffic Speed Prediction with Microscopic Connected Vehicle Driving Behavior
Lei Han, Mohamed Abdel-Aty, Younggun Kim et al. (+2 more)
Accurate speed prediction is crucial for proactive traffic management to enhance traffic efficiency and safety. Existing studies have primarily relied on aggregated, macroscopic traffic flow data to predict future traffic trends, whereas road traffic dynamics are also influenced by individual, microscopic human driving behaviors. Recent Connected Vehicle (CV) data provide rich driving behavior features, offering new opportunities to incorporate these behavioral insights into speed prediction. To this end, we propose the Macro-Micro Cross-
Attention Transformer (MMCAformer) to integrate CV data-based micro driving behavior features with macro traffic features for speed prediction. Specifically, MMCAformer employs self-
attention to learn intrinsic dependencies in macro traffic flow and cross-
attention to capture spatiotemporal interplays between macro traffic status and micro driving behavior. MMCAformer is optimized with a Student-t negative log-likelihood loss to provide point-wise speed prediction and estimate uncertainty. Experiments on four Florida freeways demonstrate the superior performance of the proposed MMCAformer compared to baselines. Compared with only using macro features, introducing micro driving behavior features not only enhances prediction accuracy (e.g., overall RMSE, MAE, and MAPE reduced by 9.0%, 6.9%, and 10.2%, respectively) but also shrinks model prediction uncertainty (e.g., mean predictive intervals decreased by 10.1-24.0% across the four freeways). Results reveal that hard braking and acceleration frequencies emerge as the most influential features. Such improvements are more pronounced under congested, low-speed traffic conditions.
Read the full abstract ▾
All authors Lei Han, Mohamed Abdel-Aty, Younggun Kim, Yang-Jun Joo, Zubayer Islam
Machine Learning · Cross-listed from cs.CR
Intent Laundering: AI Safety Datasets Are Not What They Seem
Shahriar Golchin and Marc Wetter
We systematically evaluate the quality of widely used
adversarial safety
datasets from two perspectives: in isolation and in practice. In isolation, we examine how well these
datasets reflect real-world
adversarial attacks based on three defining properties: being driven by ulterior intent, well-crafted, and out-of-distribution. We find that these
datasets overrely on "triggering cues": words or phrases with overt negative/sensitive connotations that are intended to trigger safety mechanisms explicitly, which is unrealistic compared to real-world attacks. In practice, we evaluate whether these
datasets genuinely measure safety risks or merely provoke refusals through triggering cues. To explore this, we introduce "intent laundering": a procedure that abstracts away triggering cues from
adversarial attacks (data points) while strictly preserving their malicious intent and all relevant details. Our results show that current
adversarial safety
datasets fail to faithfully represent real-world
adversarial behavior due to their overreliance on triggering cues. Once these cues are removed, all previously evaluated "reasonably safe" models become unsafe, including Gemini 3 Pro and Claude Sonnet 3.7/4. Moreover, when intent laundering is adapted as a jailbreaking technique, it consistently achieves high attack success rates, ranging from 90.00% to 100.00%, under fully black-box access. Overall, our findings expose a significant disconnect between how existing
datasets evaluate model safety and how real-world adversaries behave.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
Mobility-Aware Cache Framework for Scalable LLM-Based Human Mobility Simulation
Hua Yan, Heng Tan, Yingxue Zhang et al. (+1 more)
Simulating large-scale human mobility is fundamental to understanding population movement patterns and supporting real-world geospatial applications such as urban planning, epidemic response, and transportation analysis. Recent works treat large
language models (LLMs) as human
agents to simulate realistic mobility behaviors using structured
reasoning, but their high computational cost limits scalability. To address this, we design a mobility-aware cache framework named MobCache that leverages reconstructible caches to enable efficient large-scale human mobility simulations. It consists of: (1) a
reasoning component that encodes each
reasoning step as a latent-space
embedding and uses a latent-space evaluator to enable the reuse and recombination of
reasoning steps; and (2) a decoding component that employs a lightweight decoder trained with mobility law-constrained
distillation to translate latent-space
reasoning chains into natural
language, thereby improving simulation efficiency while maintaining fidelity. Experiments show that MobCache significantly improves efficiency across multiple dimensions while maintaining performance comparable to state-of-the-art LLM-based methods.
Read the full abstract ▾
All authors Hua Yan, Heng Tan, Yingxue Zhang, Yu Yang
Machine Learning
Why Any-Order Autoregressive Models Need Two-Stream Attention: A Structural-Semantic Tradeoff
Patrick Pynadath and Ruqi Zhang
Any-order autoregressive models (AO-ARMs) offer a promising path toward efficient masked
diffusion by enabling native key-value caching, but competitive performance has so far required two-stream
attention, typically motivated as a means of decoupling
token content from position. In this work, we argue that two-stream
attention may be serving a more subtle role. We identify a structural-semantic tradeoff in any-order generation: the hidden representation at each step must simultaneously attend to semantically informative
tokens for prediction and structurally recent
tokens for summarization, objectives that compete for
attention capacity in a single stream but can specialize across two streams. To isolate this tradeoff from position-content separation, we propose Decoupled RoPE, a modification to rotary position
embeddings that provides target position information without revealing target content. Decoupled RoPE performs competitively at short sequence lengths–where semantic and structural proximity coincide–but degrades as sequence length increases and the two orderings diverge. These results suggest that the success of two-stream
attention stems not merely from separating position from content, but from circumventing the deeper structural-semantic tradeoff inherent to any-order generation.
Read the full abstract ▾
Machine Learning · Cross-listed from physics.ao-ph
Examining Fast Radiatively Driven Responses Using Machine-Learning Weather Emulators
Ankur Mahesh, William D. Collins, Travis A. O'Brien et al. (+10 more)
The response of the
climate system to increased greenhouse gases and other radiative perturbations is governed by a combination of fast and slow feedbacks. Slow feedbacks are typically activated in response to changes in ocean temperatures on decadal timescales and manifest as changes in climatic state with no recent historical analogue. However, fast feedbacks are activated in response to rapid atmospheric physical processes on weekly timescales, and they are already operative in the present-day
climate. This distinction implies that the physics of fast radiative feedbacks is present in the historical meteorological reanalyses used to train many recent successful machine-learning-based (ML) emulators of weather and
climate. In addition, these feedbacks are functional under the historical boundary conditions pertaining to the top-of-atmosphere radiative balance and sea-surface temperatures. Together, these factors imply that we can use historically trained ML weather emulators to study the response of radiative-convective
equilibrium (RCE), and hence the global hydrological cycle, to perturbations in carbon dioxide and other well-mixed greenhouse gases. Without retraining on prospective Earth system conditions, we use ML weather emulators to quantify the fast precipitation response to reduced and elevated carbon dioxed concentrations with no recent historical precedent. We show that the responses from historically trained emulators agree with those produced by full-physics Earth System Models (ESMs). In conclusion, we discuss the prospects for and advantages from using ESMs and ML emulators to study fast processes in global
climate.
Read the full abstract ▾
All authors Ankur Mahesh, William D. Collins, Travis A. O'Brien, Paul B. Goddard, Sinclaire Zebaze, Shashank Subramanian, James P.C. Duncan, Oliver Watt-Meyer, Boris Bonev, Thorsten Kurth, Karthik Kashinath, Michael S. Pritchard, Da Yang
Machine Learning · Cross-listed from cs.CV
LGQ: Learnable Geometric Quantization for Image Tokenization
Idil Bilge Altun, Mert Onur Cakiroglu, Elham Buxton et al. (+2 more)
Recent collapse-free quantizers such as FSQ achieve stable training by replacing the learnable codebook with an engineered geometry: a fixed scalar grid whose structure is dictated by the codebook size K. We show this trade-off is unnecessary. We introduce Learnable
Geometric Quantization (LGQ), which retains a learnable codebook of codes and performs soft-to-hard assignment via temperature annealing, regularized by two cheap terms: a diversity term scaled by codebook size that penalizes concentrated batch-average usage is the primary driver of collapse resistance, complemented by a peakedness term that sharpens each
token's soft-assignment toward one-hot; together they prevent codebook collapse without EMA, reset heuristics, or codebook reparameterization. Under a fixed VQ-GAN backbone, we
benchmark LGQ against RotVQ, FSQ, LFQ, SimVQ, and IBQ on ImageNet 256x256 at K = 16,384, and sweep LGQ over K in {4096, ..., 65,536} without any per-K hyperparameter tuning. LGQ attains the best reconstruction FID at K = 16,384 while maintaining 100% codebook utilization, and continues to improve as the codebook grows to K = 65,536, holding 100% utilization at every K. Training MaskGIT on the frozen tokenizers, LGQ further attains the best class-conditional generation among the compared quantizers, leading on reconstruction and generation alike. Code is available at https://anonymous.4open.science/r/lgq-anon-E12C/.
Read the full abstract ▾
All authors Idil Bilge Altun, Mert Onur Cakiroglu, Elham Buxton, Mehmet Dalkilic, Hasan Kurban
Machine Learning · Cross-listed from cs.CL
Activation Steering via Generative Causal Mediation
Aruna Sankaranarayanan, Amir Zur, Atticus Geiger et al. (+1 more)
Where should we intervene in a
language model (LM) to localize and control behaviors that are diffused across many
tokens of a long-form response? We introduce Generative Causal Mediation (GCM), a procedure for selecting model components (e.g.,
attention heads) from contrastive long-form responses, to steer such diffuse concepts (e.g., talk in verse vs. talk in prose). In GCM, we first construct a
dataset of contrasting behavioral inputs and long-form responses. Then, we quantify how model components mediate the concept and select the strongest mediators for steering. We evaluate GCM on three behaviors–refusal, sycophancy, and style transfer–across three
language models. GCM successfully localizes concepts expressed in long-form responses and outperforms correlational probe-based baselines when steering with a sparse set of
attention heads. Together, these results demonstrate that GCM provides an effective approach for localizing from and controlling the long-form responses of LMs.
Read the full abstract ▾
All authors Aruna Sankaranarayanan, Amir Zur, Atticus Geiger, Dylan Hadfield-Menell
Machine Learning · Cross-listed from cs.AR
DARTH-PUM: A Hybrid Processing-Using-Memory Architecture
Ryan Wong, Ben Feinberg and Saugata Ghose
Analog processing-using-
memory (PUM; a.k.a. in-
memory computing) makes use of electrical interactions inside
memory arrays to perform bulk
matrix-vector multiplication (MVM) operations. However, many popular
matrix-based kernels need to execute non-MVM operations, which analog PUM cannot directly perform. To retain its energy efficiency, analog PUM architectures augment
memory arrays with CMOS-based domain-specific fixed-function hardware to provide complete kernel functionality, but the difficulty of integrating such specialized CMOS logic with
memory arrays has largely limited analog PUM to being an accelerator for machine learning inference, or for closely related kernels. An opportunity exists to harness analog PUM for general-purpose computation: recent works have shown that
memory arrays can also perform Boolean PUM operations, albeit with very different supporting hardware and electrical signals than analog PUM. We propose DARTH-PUM, a general-purpose hybrid PUM architecture that tackles key hardware and software challenges to integrating analog PUM and digital PUM. We propose optimized peripheral circuitry, coordinating hardware to manage and interface between both types of PUM, an easy-to-use programming interface, and low-cost support for flexible data widths. These design elements allow us to build a practical PUM architecture that can execute kernels fully in
memory, and can scale easily to cater to domains ranging from embedded applications to large-scale data-driven computing. We show how three popular applications (AES encryption, convolutional neural networks, large
language models) can map to and benefit from DARTH-PUM, with speedups of 59.4x, 14.8x, and 40.8x over an analog+CPU baseline.
Read the full abstract ▾
Machine Learning
Omni-iEEG: A Large-Scale, Comprehensive iEEG Dataset and Benchmark for Epilepsy Research
Chenda Duan, Yipeng Zhang, Sotaro Kanai et al. (+9 more)
Epilepsy affects over 50 million people worldwide, and one-third of patients suffer drug-resistant seizures where surgery offers the best chance of seizure freedom. Accurate localization of the epileptogenic zone (EZ) relies on intracranial
EEG (iEEG). Clinical workflows, however, remain constrained by labor-intensive manual review. At the same time, existing data-driven approaches are typically developed on single-center
datasets that are inconsistent in format and metadata, lack standardized
benchmarks, and rarely release pathological event annotations, creating barriers to reproducibility, cross-center validation, and clinical relevance. With extensive efforts to reconcile heterogeneous iEEG formats, metadata, and recordings across publicly available sources, we present $\textbf{Omni-iEEG}$, a large-scale, pre-
surgical iEEG resource comprising $\textbf{302 patients}$ and $\textbf{178 hours}$ of high-resolution recordings. The
dataset includes harmonized clinical metadata such as seizure onset zones, resections, and
surgical outcomes, all validated by board-certified epileptologists. In addition, Omni-iEEG provides over 36K expert-validated annotations of pathological events, enabling robust biomarker studies. Omni-iEEG serves as a bridge between machine learning and epilepsy research. It defines clinically meaningful tasks with unified evaluation metrics grounded in clinical priors, enabling systematic evaluation of models in clinically relevant settings. Beyond benchmarking, we demonstrate the potential of end-to-end modeling on long iEEG segments and highlight the transferability of representations pretrained on non-neurophysiological domains. Together, these contributions establish Omni-iEEG as a foundation for reproducible, generalizable, and clinically translatable epilepsy research. The project page with
dataset and code links is available at omni-ieeg.github.io/omni-ieeg.
Read the full abstract ▾
All authors Chenda Duan, Yipeng Zhang, Sotaro Kanai, Yuanyi Ding, Atsuro Daida, Pengyue Yu, Tiancheng Zheng, Naoto Kuroda, Shaun A. Hussain, Eishi Asano, Hiroki Nariai, Vwani Roychowdhury
Machine Learning · Cross-listed from cs.SE
The Limits of Long-Context Reasoning in Automated Bug Fixing
Ravi Raju, Mengmeng Ji, Shubhangi Upasani et al. (+2 more)
Rapidly increasing context lengths have led to the assumption that large
language models (LLMs) can directly reason over entire codebases. Concurrently, recent advances in LLMs have enabled strong performance on software engineering
benchmarks, particularly when paired with agentic workflows. In this work, we systematically evaluate whether current LLMs can reliably perform long-context code debugging and patch generation. Using SWE-bench Verified as a controlled experimental setting, we first evaluate state-of-the-art models within an agentic harness (mini-SWE-
agent), where performance improves substantially: GPT-5-nano achieves up to a 31% resolve rate on 100 samples, and open-source models such as Deepseek-R1-0528 obtain competitive results. However,
token-level analysis shows that successful agentic trajectories typically remain under 20k-30k
tokens, and that longer accumulated contexts correlate with lower success rates, indicating that agentic success primarily arises from task decomposition into short-context steps rather than effective long-context
reasoning. To directly test long-context capability, we construct a data pipeline where we artificially inflate the context length of the input by placing the relevant files into the context (ensuring perfect
retrieval recall); we then study single-shot patch generation under genuinely long contexts (64k
tokens). Despite this setup, performance degrades sharply: Qwen3-Coder-30B-A3B achieves only a 7% resolve rate at 64k context, while GPT-5-nano solves none of the tasks. Qualitative analysis reveals systematic failure modes, including hallucinated diffs, incorrect file targets, and malformed patch headers. Overall, our findings highlight a significant gap between nominal context length and usable context capacity in current LLMs, and suggest that existing agentic coding
benchmarks do not meaningfully evaluate long-context
reasoning.
Read the full abstract ▾
All authors Ravi Raju, Mengmeng Ji, Shubhangi Upasani, Bo Li, Urmish Thakker