Machine Learning
Prototype-based Explainable Neural Networks with Channel-specific Reasoning for Geospatial Learning Tasks
Anushka Narayanan and Karianne J. Bergen
Explainable AI (XAI) is essential for understanding machine learning (ML) decision-making and ensuring model trustworthiness in scientific applications. Prototype-based XAI methods offer an intrinsically interpretable alternative to post-hoc approaches which often yield inconsistent explanations. Prototype-based XAI methods make predictions based on the similarity between inputs and learned prototypes that represent typical characteristics of target classes. However, existing prototype-based models are primarily designed for standard RGB image data and are not optimized for the distinct, variable-specific channels commonly found in geoscientific image and raster
datasets. In this study, we develop a prototype-based XAI approach tailored for multi-channel geospatial data, where each channel represents a distinct physical environmental variable or spectral channel. Our approach enables the model to identify separate, channel-specific prototypical characteristics sourced from multiple distinct training examples that inform how these features individually and in combination influence model prediction while achieving comparable performance to standard neural networks. We demonstrate this method through two geoscientific case studies: (1) classification of Madden Julian Oscillation phases using multi-variable
climate data and (2) land-use classification from multispectral satellite imagery. This approach produces both local (instance-level) and global (model-level) explanations for providing insights into feature-relevance across channels. By explicitly incorporating channel-prototypes into the prediction process, we discuss how this approach enhances the transparency and trustworthiness of ML models for geoscientific learning tasks.
Read the full abstract ▾
Machine Learning
In-Run Data Shapley for Adam Optimizer
Meng Ding, Zeqing Zhang, Di Wang et al. (+1 more)
Reliable data attribution is essential for mitigating bias and reducing computational waste in modern machine learning, with the Shapley value serving as the theoretical gold standard. While recent "In-Run" methods bypass the prohibitive cost of retraining by estimating contributions dynamically, they heavily rely on the linear structure of
Stochastic Gradient Descent (SGD) and fail to capture the complex dynamics of adaptive optimizers like Adam. In this work, we demonstrate that data attribution is inherently optimizer-dependent: we show that SGD-based proxies diverge significantly from true contributions under Adam (Pearson $R \approx 0.11$), rendering them ineffective for modern training pipelines. To bridge this gap, we propose Adam-Aware In-Run Data Shapley. We derive a closed-form approximation that restores additivity by redefining utility under a fixed-state assumption and enable scalable computation via a novel Linearized Ghost Approximation. This technique linearizes the variance-dependent scaling term, allowing us to compute pairwise gradient dot-products without materializing per-sample gradients. Extensive experiments show that our method achieves near-perfect fidelity to ground-truth marginal contributions ($R > 0.99$) while retaining $\sim$95% of standard training throughput. Furthermore, our Adam-aware attribution significantly outperforms SGD-based baselines in data attribution downstream tasks.
Read the full abstract ▾
All authors Meng Ding, Zeqing Zhang, Di Wang, Lijie Hu
Machine Learning
Harvest: Opportunistic Peer-to-Peer GPU Caching for LLM Inference
Nikhil Gopal and Kostis Kaffes
Large
Language Model (LLM) inference is increasingly constrained by GPU
memory capacity rather than compute throughput, driven by growing model sizes and the linear growth of the key-value (KV) cache during autoregressive decoding. Existing approaches mitigate
memory pressure by offloading model state and KV tensors to host
memory, but incur substantial latency due to limited PCIe bandwidth. We present Harvest, an opportunistic GPU cache management framework that exploits high-bandwidth peer-to-peer GPU interconnects to dynamically place model weights and KV cache in unused GPU
memory. Harvest treats peer GPU
memory as a transient cache tier, preserving correctness while reducing data movement overhead under dynamic
memory availability. We demonstrate significant throughput speedup of more than 2 times by using Harvest to accelerate the
retrieval of two widely-used inference components: expert layer weights and KV cache entries.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Detecting AI-Generated Content in Academic Peer Reviews
Siyuan Shen and Kai Wang
The growing availability of large
language models (LLMs) has raised questions about their role in academic peer review. This study examines the temporal emergence of AI-generated content in peer reviews by applying a detection model trained on historical reviews to later review cycles at International Conference on Learning Representations (ICLR) and Nature Communications (NC). We observe minimal detection of AI-generated content before 2022, followed by a substantial increase through 2025, with approximately 20% of ICLR reviews and 12% of Nature Communications reviews classified as AI-generated in 2025. The most pronounced growth of AI-generated reviews in NC occurs between the third and fourth quarter of 2024. Together, these findings provide suggestive evidence of a rapidly increasing presence of AI-assisted content in peer review and highlight the need for further study of its implications for scholarly evaluation.
Read the full abstract ▾
Machine Learning
Optimal Transport-Guided Adversarial Attacks on Graph Neural Network-Based Bot Detection
Kunal Mukherjee, Zulfikar Alom, Tran Gia Bao Ngo et al. (+2 more)
The rise of bot accounts on social media poses significant risks to public discourse. To address this threat, modern bot detectors increasingly rely on Graph Neural Networks (GNNs). However, the effectiveness of these GNN-based detectors in real-world settings remains poorly understood. In practice, attackers continuously adapt their strategies as well as must operate under domain-specific and temporal constraints, which can fundamentally limit the applicability of existing attack methods. As a result, there is a critical need for robust GNN-based bot detection methods under realistic, constraint-aware attack scenarios. To address this gap, we introduce BOCLOAK to systematically evaluate the robustness of GNN-based social bot detection via both edge editing and node injection
adversarial attacks under realistic constraints. BOCLOAK constructs a probability measure over spatio-temporal neighbor features and learns an optimal transport geometry that separates human and bot behaviors. It then decodes transport plans into sparse, plausible edge edits that evade detection while obeying real-world constraints. We evaluate BOCLOAK across three social bot
datasets, five state-of-the-art bot detectors, three
adversarial defenses, and compare it against four leading graph
adversarial attack baselines. BOCLOAK achieves up to 80.13% higher attack success rates while using 99.80% less GPU
memory under realistic real-world constraints. Most importantly, BOCLOAK shows that optimal transport provides a lightweight, principled framework for bridging the gap between
adversarial attacks and real-world bot detection.
Read the full abstract ▾
All authors Kunal Mukherjee, Zulfikar Alom, Tran Gia Bao Ngo, Cuneyt Gurcan Akcora, Murat Kantarcioglu
Machine Learning
Beyond the Loss Curve: Scaling Laws, Active Learning, and the Limits of Learning from Exact Posteriors
Arian Khorasani, Nathaniel Chen, Yug D Oswal et al. (+3 more)
How close are neural networks to the best they could possibly do? Standard
benchmarks cannot answer this because they lack access to the true posterior p(y|x). We use class-conditional normalizing flows as oracles that make exact posteriors tractable on realistic images (AFHQ, ImageNet). This enables five lines of investigation. Scaling laws: Prediction error decomposes into irreducible aleatoric uncertainty and reducible epistemic error; the epistemic component follows a power law in
dataset size, continuing to shrink even when total loss plateaus. Limits of learning: The aleatoric floor is exactly measurable, and architectures differ markedly in how they approach it: ResNets exhibit clean power-law scaling while
Vision Transformers stall in low-data regimes. Soft labels: Oracle posteriors contain learnable structure beyond class labels: training with exact posteriors outperforms hard labels and yields near-perfect calibration. Distribution shift: The oracle computes exact KL divergence of controlled perturbations, revealing that shift type matters more than shift magnitude: class imbalance barely affects accuracy at divergence values where input noise causes catastrophic degradation. Active learning: Exact epistemic uncertainty distinguishes genuinely informative samples from inherently ambiguous ones, improving sample efficiency. Our framework reveals that standard metrics hide ongoing learning, mask architectural differences, and cannot diagnose the nature of distribution shift.
Read the full abstract ▾
All authors Arian Khorasani, Nathaniel Chen, Yug D Oswal, Akshat Santhana Gopalan, Egemen Kolemen, Ravid Shwartz-Ziv
Machine Learning · Cross-listed from cs.CV
Opportunistic Promptable Segmentation: Leveraging Routine Radiological Annotations to Guide 3D CT Lesion Segmentation
Samuel Church, Joshua D. Warner, Danyal Maqbool et al. (+4 more)
The development of machine learning models for CT imaging depends on the availability of large, high-quality, and diverse annotated
datasets. Although large volumes of CT images and reports are readily available in clinical picture archiving and communication systems (PACS), 3D
segmentations of critical findings are costly to obtain, typically requiring extensive manual annotation by radiologists. On the other hand, it is common for radiologists to provide limited annotations of findings during routine reads, such as line measurements and arrows, that are often stored in PACS as GSPS objects. We posit that these sparse annotations can be extracted along with CT volumes and converted into 3D
segmentations using promptable
segmentation models, a paradigm we term Opportunistic Promptable
Segmentation. To enable this paradigm, we propose SAM2CT, the first promptable
segmentation model designed to convert radiologist annotations into 3D
segmentations in CT volumes. SAM2CT builds upon SAM2 by extending the
prompt encoder to support arrow and line inputs and by introducing
Memory-Conditioned
Memories (MCM), a
memory encoding strategy tailored to 3D medical volumes. On public lesion
segmentation benchmarks, SAM2CT outperforms existing promptable
segmentation models and similarly trained baselines, achieving Dice similarity coefficients of 0.649 for arrow
prompts and 0.757 for line
prompts. Applying the model to pre-existing GSPS annotations from a clinical PACS (N = 60), SAM2CT generates 3D
segmentations that are clinically acceptable or require only minor adjustments in 87% of cases, as scored by radiologists. Additionally, SAM2CT demonstrates strong zero-shot performance on select Emergency Department findings. These results suggest that large-scale mining of historical GSPS annotations represents a promising and scalable approach for generating 3D CT
segmentation datasets.
Read the full abstract ▾
All authors Samuel Church, Joshua D. Warner, Danyal Maqbool, Xin Tie, Junjie Hu, Meghan G. Lubner, Tyler J. Bradshaw
Machine Learning · Cross-listed from cs.CR
Syntax- and Compilation-Preserving Evasion of LLM Vulnerability Detectors
Luze Sun, Alina Oprea and Eric Wong
LLM-based vulnerability detectors are increasingly deployed in CI/CD security gating, yet their resilience to evasion under syntax- and compilation-preserving edits remains poorly understood. We evaluate five attack variants spanning four carrier families of behavior-preserving code transformations on a unified C/C++
benchmark ($N=5000$) and introduce Complete Resistance (CR), measuring the fraction of correctly detected vulnerabilities that withstand all attack variants. Our findings reveal a significant robustness gap: models achieving 70%+ clean recall exhibit CR as low as 0.12%, meaning over 87% of detected vulnerabilities can be evaded by at least one syntax-preserving edit. Universal
adversarial strings optimized on a 14B surrogate transfer effectively to black-box APIs including GPT-4o, while on-target optimization further amplifies evasion (up to 92.5% ASR). These results indicate that clean
benchmark accuracy alone is insufficient as a security guarantee for deployed vulnerability detectors.
Read the full abstract ▾
Machine Learning
Neural Ising Machines via Unrolling and Zeroth-Order Training
Sam Reifenstein and Timothee Leleu
We propose a data-driven heuristic for NP-hard Ising and Max-Cut optimization that learns the update rule of an iterative dynamical system. The method learns a shared, node-wise update rule that maps local interaction fields to
spin updates, parameterized by a compact multilayer perceptron with a small number of parameters. Training is performed using a zeroth-order optimizer, since backpropagation through long, recurrent Ising-machine dynamics leads to unstable and poorly informative gradients. We call this approach a neural network parameterized Ising machine (NPIM). Despite its low parameter count, the learned dynamics recover effective algorithmic structure, including momentum-like behavior and time-varying schedules, enabling efficient search in highly non-convex energy landscapes. Across standard Ising and neural combinatorial optimization
benchmarks, NPIM achieves competitive solution quality and time-to-solution relative to recent learning-based methods and strong classical Ising-machine heuristics.
Read the full abstract ▾
Machine Learning
Agentic Framework for Epidemiological Modeling
Rituparna Datta, Zihan Guan, Baltazar Espinoza et al. (+5 more)
Epidemic modeling is essential for public health planning, yet traditional approaches rely on fixed model classes that require manual redesign as pathogens, policies, and scenario assumptions evolve. We introduce EPIAGENT, an agentic framework that automatically synthesizes, calibrates, verifies, and refines epidemiological simulators by modeling disease progression as an iterative program synthesis problem. A central design choice is an explicit epidemiological flow graph intermediate representation that links scenario specifications to model structure and enables strong, modular correctness checks before code is generated. Verified flow graphs are then compiled into mechanistic models supporting interpretable parameter learning under physical and epidemiological constraints. Evaluation on epidemiological scenario case studies demonstrates that EPIAGENT captures complex growth dynamics and produces epidemiologically consistent counterfactual projections across varying vaccination and immune escape assumptions. Our results show that the agentic feedback loop prevents degeneration and significantly accelerates convergence toward valid models by mimicking professional expert workflows.
Read the full abstract ▾
All authors Rituparna Datta, Zihan Guan, Baltazar Espinoza, Yiqi Su, Priya Pitre, Srini Venkatramanan, Naren Ramakrishnan, Anil Vullikanti
Machine Learning
From Observations to States: Latent Time Series Forecasting
Jie Yang, Yifan Hu, Yuante Li et al. (+3 more)
Deep learning has achieved strong performance in Time Series
Forecasting (TSF). However, we identify a critical representation paradox, termed Latent Chaos: models with accurate predictions often learn latent representations that are temporally disordered and lack continuity. We attribute this to the dominant observation-space
forecasting paradigm, where minimizing point-wise errors on noisy and partially observed data encourages shortcut solutions instead of the recovery of underlying system dynamics. To address this, we propose Latent Time Series
Forecasting (LatentTSF), a paradigm that shifts TSF from observation regression to latent state prediction. LatentTSF employs an AutoEncoder to project each observation into a learned latent state space and performs
forecasting entirely in this space, allowing the model to focus on learning structured temporal dynamics. We provide an information-theoretic analysis showing that the latent objectives can be motivated as surrogates for maximizing mutual information between predicted and ground-truth latent states and future observations. Extensive experiments on widely-used
benchmarks confirm that LatentTSF effectively mitigates latent chaos, yielding consistent improvements in both
forecasting accuracy and representation quality. Our code is available at https://github.com/Muyiiiii/LatentTSF.
Read the full abstract ▾
All authors Jie Yang, Yifan Hu, Yuante Li, Kexin Zhang, Kaize Ding, Philip S. Yu
Machine Learning
Self-Attention at Constant Cost per Token via Symmetry-Aware Taylor Approximation
Franz A. Heinsen and Leo Kozachkov
The most widely used artificial intelligence (AI) models today are
Transformers employing self-
attention. In its standard form, self-
attention incurs costs that increase with context length, driving demand for storage, compute, and energy that is now outstripping society's ability to provide them. To help address this issue, we show that self-
attention is efficiently computable to arbitrary precision with constant cost per
token, achieving orders-of-magnitude reductions in
memory use and computation. We derive our formulation by decomposing the conventional formulation's Taylor expansion into expressions over symmetric chains of tensor products. We exploit their
symmetry to obtain feed-forward transformations that efficiently map queries and keys to coordinates in a minimal
polynomial-kernel feature basis. Notably, cost is fixed inversely in proportion to head size, enabling application over a greater number of heads per
token than otherwise feasible. We implement our formulation and empirically validate its correctness. Our work enables unbounded
token generation at modest fixed cost, substantially reducing the infrastructure and energy demands of large-scale
Transformer models. The mathematical techniques we introduce are of independent interest.
Read the full abstract ▾
Machine Learning
Generation Order and Parallel Decoding in Masked Diffusion Models: An Information-Theoretic Perspective
Shaorong Zhang, Longxuan Yu, Rob Brekelmans et al. (+3 more)
Masked
Diffusion Models (MDMs) significantly accelerate inference by trading off sequential determinism. However, the theoretical mechanisms governing generation order and the risks inherent in parallelization remain under-explored. In this work, we provide a unified information-theoretic framework to decouple and analyze two fundamental sources of failure: order sensitivity and parallelization bias. Our analysis yields three key insights: (1) The benefits of Easy-First decoding (prioritizing low-
entropy tokens) are magnified as model error increases; (2) factorized parallel decoding introduces intrinsic sampling errors that can lead to arbitrary large Reverse KL divergence, capturing "incoherence" failures that standard Forward KL metrics overlook; and (3) while verification can eliminate sampling error, it incurs an exponential cost governed by the total correlation within a block. Conversely, heuristics like remasking, though computationally efficient, cannot guarantee distributional correctness. Experiments on a controlled Block-HMM and large-scale MDMs (LLaDA) for arithmetic
reasoning validate our theoretical framework.
Read the full abstract ▾
All authors Shaorong Zhang, Longxuan Yu, Rob Brekelmans, Luhan Tang, Salman Asif, Greg Ver Steeg
Machine Learning
Constrained Bilevel Reinforcement Learning via Approximate Lower-Level Responses
Naman Saxena and Vaneet Aggarwal
We study constrained bilevel reinforcement learning in which the lower-level problem is a constrained Markov decision process with a generally parameterized policy. The lower-level solution map can be set-valued and need not be Lipschitz continuous in the upper-level variable, so standard hypergradient analyses do not apply. We therefore define the upper-level objective as the expected upper-level loss of the policy returned by a fixed
stochastic lower-level
algorithm, which we call the approximate response. This objective is well defined without uniqueness of the lower-level solution. To compute its gradient, we randomize the reward and constraint values used during lower-level training without changing their means, so that the upper-level variable affects training only through known probabilities. This yields an unbiased
estimator of the upper-level objective's gradient that does not differentiate through the lower-level updates. We establish lower-level reward and feasibility guarantees, upper-level stationarity, and a total sample complexity of $\widetilde{\mathcal{O}}(\delta_\ell^{-6}\epsilon_o^{-2})$ over both levels, where $\delta_\ell$ is the lower-level accuracy and $\epsilon_o$ is the upper-level stationarity error. Setting $\delta_\ell=\epsilon_o=\epsilon$ gives $\widetilde{\mathcal{O}}(\epsilon^{-8})$.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Semantic Calibration Prevails Where Token Confidence Fails: Benchmarking Long-Form Scientific QA
Philip Müller, Nicholas Popovič, Michael Färber et al. (+1 more)
Reliable uncertainty quantification (UQ) is essential for safe deployment of large
language models (LLMs) in scientific question answering, where long-form outputs exceed practical human verification at scale. We introduce the first large-scale
benchmark for UQ calibration in long-form,
reasoning-demanding scientific QA, evaluating four UQ methods on 685,000 responses across up to 20 LLMs and seven
datasets, supported by an extensible open-source framework whose shared-generation design enables reproducible cross-method comparisons. Instruction tuning is shown to associate with systematic
token probability polarization, collapsing confidence distributions and undermining the reliability of
token-level uncertainty signals.
Reasoning model families diverge: some reproduce this polarization while others actively mitigate it, a pattern that clusters by provider and suggests training pipeline design as a key differentiating factor. Verbalized and
token-aggregation sequence-level methods fail systematically. Only semantic consistency, as measured by consistency of the final answer, yields well-calibrated outputs, providing the first large-scale evidence that semantic calibration persists in multi-step, dependency-rich
reasoning settings.
Read the full abstract ▾
All authors Philip Müller, Nicholas Popovič, Michael Färber, Peter Steinbach
Machine Learning
VoxServe: Streaming-Centric Serving System for Speech Language Models
Keisuke Kamahori, Rohan Kadekodi, Garv Ghai et al. (+5 more)
Speech Language Models (SpeechLMs) are increasingly deployed for streaming
speech generation, where audio playback must start quickly and never stall. Serving them requires running two very different model components on the same system. An autoregressive LLM backbone emits audio
tokens, and a codec converts chunks of those
tokens into waveforms. Existing systems run the two as separately scheduled stages, and therefore either serialize them, let them interfere, or strand capacity by placing them on separate devices. We observe that overlapping the backbone and the codec on separate parts of the GPU raises throughput. Although this increases codec latency, most chunks can tolerate it, as long as it neither lies on the critical path of time-to-first-audio (TTFA) nor causes a playback stall. Based on this insight, we present VoxServe, a SpeechLM serving system that allocates GPU resources according to the urgency of chunks. VoxServe executes work on three asynchronous lanes. The LM lane runs the backbone on a large partition of the GPU's streaming multiprocessors, the background codec lane runs on a small partition beside it, and the urgent lane uses the whole GPU. The scheduler queues each request to the LM lane and routes every chunk the backbone completes by its urgency. A request's first chunk and chunks close to their playback deadline go to the urgent lane, and the rest are batched on the background lane. On Qwen3-TTS, VoxServe sustains 1.5x the request rate of the best existing system at a 200ms TTFA target without playback stalls. The code of VoxServe is available at https://github.com/vox-serve/vox-serve.
Read the full abstract ▾
All authors Keisuke Kamahori, Rohan Kadekodi, Garv Ghai, Wei-Tzu Lee, Atindra Jha, Stephanie Wang, Arvind Krishnamurthy, Baris Kasikci
Machine Learning
TABES: Trajectory-Aware Backward-on-Entropy Steering for Masked Diffusion Models
Shreshth Saini, Avinab Saha, Balu Adsumilli et al. (+3 more)
Masked
Diffusion Models (MDMs) have emerged as a promising non-autoregressive paradigm for generative tasks, offering parallel decoding and bidirectional context utilization. However, current sampling methods rely on simple confidence-based heuristics that ignore the long-term impact of local decisions, leading to trajectory lock-in where early
hallucinations cascade into global incoherence. While search-based methods mitigate this, they incur prohibitive computational costs ($O(K)$ forward passes per step). In this work, we propose Backward-on-
Entropy (BoE) Steering, a gradient-guided inference framework that approximates infinite-horizon lookahead via a single backward pass. We formally derive the
Token Influence Score (TIS) from a first-order expansion of the trajectory cost functional, proving that the gradient of future
entropy with respect to input
embeddings serves as an optimal control signal for minimizing uncertainty. To ensure scalability, we introduce ActiveQueryAttention, a sparse adjoint primitive that exploits the structure of the masking objective to reduce backward pass complexity. BoE achieves a superior Pareto frontier for inference-time scaling compared to existing unmasking methods, demonstrating that gradient-guided steering offers a mathematically principled and efficient path to robust non-autoregressive generation. We will release the code.
Read the full abstract ▾
All authors Shreshth Saini, Avinab Saha, Balu Adsumilli, Neil Birkbeck, Yilin Wang, Alan C. Bovik
Machine Learning · Cross-listed from cs.CV
CAPA: Contribution-Aware Pruning and FFN Approximation for Efficient Large Vision-Language Models
Samyak Jha and Junho Kim
Efficient inference in Large
Vision-
Language Models is constrained by the high cost of processing thousands of visual
tokens, yet it remains unclear which
tokens and computations can be safely removed. While
attention scores are commonly used to estimate visual
token importance, they are an imperfect proxy for actual contribution. We show that
Attention Contribution, which weights
attention probabilities by value vector magnitude, provides a more accurate criterion for visual
token selection. Our empirical analysis reveals that visual
attention sinks are functionally heterogeneous, comprising Probability Dumps with low contribution that can be safely pruned, and Structural Anchors with high contribution essential for maintaining model performance. Further, we identify substantial redundancy in Feed-Forward Networks (FFNs) associated with visual
tokens, particularly in intermediate layers where image
tokens exhibit linear behavior. Based on our findings, we introduce CAPA (Contribution-Aware
Pruning and FFN Approximation), a dual-strategy framework that prunes visual
tokens using
attention contribution at critical functional transitions and reduces FFN computation through efficient linear approximations. Experiments on various
benchmarks across baselines show that CAPA achieves competent efficiency–performance trade-offs with improved robustness.
Read the full abstract ▾
Machine Learning
Green-NAS: A Global-Scale Multi-Objective Neural Architecture Search for Robust and Efficient Edge-Native Weather Forecasting
Md Muhtasim Munif Fahim, Soyda Humyra Yesmin, Saiful Islam et al. (+7 more)
We introduce Green-NAS, a multi-objective NAS (neural architecture search) framework designed for low-resource environments using weather
forecasting as a case study. By adhering to 'Green AI' principles, the framework explicitly minimizes computational energy costs and carbon footprints, prioritizing sustainable deployment over raw computational scale. The Green-NAS architecture search method is optimized for both model accuracy and efficiency to find lightweight models with high accuracy and very few model parameters; this is accomplished through an optimization process that simultaneously optimizes multiple objectives. Our best-performing model, Green-NAS-A, achieved an RMSE of 0.0988 (i.e., within 1.4% of our manually tuned baseline) using only 153k model parameters, which is 239 times fewer than other globally applied weather
forecasting models, such as GraphCast. In addition, we also describe how the use of transfer learning will improve the weather
forecasting accuracy by approximately 5.2%, in comparison to a naive approach of training a new model for each city, when there is limited historical weather data available for that city.
Read the full abstract ▾
All authors Md Muhtasim Munif Fahim, Soyda Humyra Yesmin, Saiful Islam, Md. Palash Bin Faruque, Md. A. Salam, Md. Mahfuz Uddin, Samiul Islam, Tofayel Ahmed, Md. Binyamin, Md. Rezaul Karim
Machine Learning · Cross-listed from cs.CL
DIVERGE: Diversity-Enhanced RAG for Open-Ended Information Seeking
Tianyi Hu, Niket Tandon and Akhil Arora
Existing
retrieval-augmented generation (RAG) systems often assume that each query has a single correct answer. This assumption overlooks open-ended information-seeking scenarios where multiple plausible answers are valuable, and where diversity is important for creativity,
fairness, and inclusive access to information. We show that standard RAG systems fail to fully use diverse retrieved contexts: simply increasing
retrieval diversity does not necessarily lead to diverse generations. To address this limitation, we propose Diverge, a plug-and-play agentic RAG framework that improves the diversity–quality trade-off through iterative, reflection-guided exploration of diverse viewpoints and diversity-aware
retrieval support. We further introduce evaluation metrics for characterizing the diversity-quality trade-off in open-ended question answering. Experiments across multiple real-world
datasets and backbone LLMs show that Diverge achieves the best trade-off among competitive baselines, increasing diversity by $\sim2\times$ without noticeable quality degradation. These results reveal a systematic limitation of current RAGs and show the value of explicit diversity modeling.
Read the full abstract ▾
Machine Learning
GRIP2: A Robust and Powerful Deep Knockoff Method for Feature Selection
Bob Junyi Zou and Lu Tian
Identifying truly predictive
covariates while strictly controlling false discoveries remains a fundamental challenge in nonlinear, highly correlated, and low signal-to-noise regimes, where deep learning based feature selection methods are most attractive. We propose Group Regularization Importance Persistence in 2 Dimensions (GRIP2), a deep knockoff feature importance statistic that integrates first-layer feature activity over a two-dimensional regularization surface controlling both sparsity strength and sparsification geometry. To approximate this surface integral in a single training run, we introduce efficient block-
stochastic sampling, which aggregates feature activity magnitudes across diverse regularization regimes along the optimization trajectory. The resulting statistics are antisymmetric by construction, ensuring finite-sample FDR control. In extensive experiments on synthetic and semi-real data, GRIP2 demonstrates improved robustness to feature correlation and noise level: in high correlation and low signal-to-noise ratio regimes where standard deep learning based feature selectors may struggle, our method retains high power and stability. Finally, on real-world HIV drug resistance data, GRIP2 recovers known resistance-associated mutations with power better than established linear baselines, confirming its reliability in practice.
Read the full abstract ▾
Machine Learning
Dispersion Loss Counteracts Embedding Condensation and Improves Generalization in Small Language Models
Chen Liu, Xingzhi Sun, Xi Xiao et al. (+8 more)
Large
language models (LLMs) achieve remarkable performance through ever-increasing parameter counts, but scaling incurs steep computational costs. To better understand LLM scaling, we study representational differences between LLMs and their smaller counterparts, with the goal of replicating the representational qualities of larger models in smaller models. We observe a
geometric phenomenon which we term $\textbf{embedding condensation}$, where
token embeddings collapse into a narrow cone-like subspace in some
language models. Through systematic analyses across multiple
Transformer families, we show that small models such as $\texttt{GPT2}$ and $\texttt{Qwen3-0.6B}$ exhibit severe condensation, whereas larger models such as $\texttt{GPT2-xl}$ and $\texttt{Qwen3-32B}$ are more resistant to this phenomenon. Additional observations show that
embedding condensation is not reliably mitigated by knowledge
distillation from larger models. To fight against it, we formulate a dispersion loss that explicitly encourages
embedding dispersion during training. Experiments demonstrate that it mitigates condensation, recovers dispersion patterns seen in larger models, and yields performance gains across 10
benchmarks. We believe this work offers a principled path toward improving smaller
Transformers without additional parameters.
Read the full abstract ▾
All authors Chen Liu, Xingzhi Sun, Xi Xiao, Alexandre Van Tassel, Ke Xu, Kristof Reimann, Danqi Liao, Mark Gerstein, Tianyang Wang, Xiao Wang, Smita Krishnaswamy
Machine Learning · Cross-listed from cs.CV
Interpretable Unsupervised Deformable Image Registration via Confidence-bound Multi-Hop Visual Reasoning
Zafar Iqbal, Anwar Ul Haq and Srimannarayana Grandhi
Unsupervised deformable image registration requires aligning complex anatomical structures without reference labels, making interpretability and reliability critical. Existing deep learning methods achieve considerable accuracy but often lack transparency, leading to error drift and reduced clinical trust. We propose a novel Multi-Hop Visual Chain of
Reasoning (VCoR) framework that reformulates registration as a progressive
reasoning process. Inspired by the iterative nature of clinical decision-making, each visual
reasoning hop integrates a Localized Spatial Refinement (LSR) module to enrich feature representations and a Cross-Reference
Attention (CRA) mechanism that leads the iterative refinement process, preserving anatomical consistency. This multi-hop strategy enables robust handling of large deformations and produces a transparent sequence of intermediate predictions with a theoretical bound. Beyond accuracy, our framework offers built-in interpretability by estimating uncertainty via the stability and convergence of deformation fields across hops. Extensive evaluations on two challenging public
datasets, DIR-Lab 4D CT (lung) and IXI T1-weighted MRI (
brain), demonstrate that VCoR achieves competitive registration accuracy while offering rich intermediate visualizations and confidence measures. By
embedding an implicit visual
reasoning paradigm, we present an interpretable, reliable, and clinically viable unsupervised medical image registration.
Read the full abstract ▾
Machine Learning
Analyzing Shapley Additive Explanations to Understand Anomaly Detection Algorithm Behaviors and Their Complementarity
Jordan Levy, Paul Saves, Moncef Garouani et al. (+2 more)
Unsupervised anomaly detection is a challenging problem due to the diversity of data distributions and the lack of labels. Ensemble methods are often adopted to mitigate these challenges by combining multiple detectors, which can reduce individual biases and increase robustness. Yet building an ensemble that is genuinely complementary remains challenging, since many detectors rely on similar decision cues and end up producing redundant anomaly scores. As a result, the potential of ensemble learning is often limited by the difficulty of identifying models that truly capture different types of irregularities. To address this, we propose a methodology for characterizing anomaly detectors through their decision mechanisms. Using SHapley Additive exPlanations, we quantify how each model attributes importance to input features, and we use these attribution profiles to measure similarity between detectors. We show that detectors with similar explanations tend to produce correlated anomaly scores and identify largely overlapping anomalies. Conversely, explanation divergence reliably indicates complementary detection behavior. Our results demonstrate that explanation-driven metrics offer a different criterion than raw outputs for selecting models in an ensemble. However, we also demonstrate that diversity alone is insufficient; high individual model performance remains a prerequisite for effective ensembles. By explicitly targeting explanation diversity while maintaining model quality, we are able to construct ensembles that are more diverse, more complementary, and ultimately more effective for unsupervised anomaly detection.
Read the full abstract ▾
All authors Jordan Levy, Paul Saves, Moncef Garouani, Nicolas Verstaevel, Benoit Gaudou
Machine Learning
Reducing Class-Wise Performance Disparity via Margin Regularization
Beier Zhu, Kesen Zhao, Jiequan Cui et al. (+4 more)
Deep neural networks often exhibit substantial disparities in class-wise accuracy, even when trained on class-balanced data, posing concerns for reliable deployment. While prior efforts have explored empirical remedies, a theoretical understanding of such performance disparities in classification remains limited. In this work, we present Margin Regularization for Performance Disparity Reduction (MR$^2$), a theoretically principled regularization for classification by dynamically adjusting margins in both the logit and representation spaces. Our analysis establishes a margin-based, class-sensitive generalization bound that reveals how per-class feature variability contributes to error, motivating the use of larger margins for hard classes. Guided by this insight, MR$^2$ optimizes per-class logit margins proportional to feature spread and penalizes excessive representation margins to enhance intra-class compactness. Experiments on seven
datasets, including ImageNet, and diverse pre-trained backbones (MAE, MoCov2, CLIP) demonstrate that MR$^2$ not only improves overall accuracy but also significantly boosts hard class performance without trading off easy classes, thus reducing performance disparity. Code is available at: https://github.com/BeierZhu/MR2
Read the full abstract ▾
All authors Beier Zhu, Kesen Zhao, Jiequan Cui, Qianru Sun, Yuan Zhou, Xun Yang, Hanwang Zhang
Machine Learning
Reducing Memorisation in Generative Models via Riemannian Bayesian Inference
Johanna Marie Gegenfurtner, Albert Kjøller Jacobsen, Naima Elosegui Borras et al. (+2 more)
Modern generative models can produce realistic samples, however, balancing memorisation and generalisation remains an open problem. We approach this challenge from a
Bayesian perspective by focusing on the parameter space of flow matching and
diffusion models and constructing a predictive posterior that better captures the variability of the data distribution. In particular, we capture the geometry of the loss using a Riemannian metric and leverage a flexible approximate posterior that adapts to the local structure of the loss landscape. This approach allows us to sample generative models that resemble the original model, but exhibit reduced memorisation. Empirically, we demonstrate that the proposed approach reduces memorisation while preserving generalisation. Further, we provide a theoretical analysis of our method, which explains our findings. Overall, our work illustrates how considering the geometry of the loss enables effective use of the parameter space, even for complex high-dimensional generative models.
Read the full abstract ▾
All authors Johanna Marie Gegenfurtner, Albert Kjøller Jacobsen, Naima Elosegui Borras, Alejandro Valverde Mahou, Georgios Arvanitidis
Machine Learning
GEPC: Group-Equivariant Posterior Consistency for Out-of-Distribution Detection in Diffusion Models
Yadang Alexis Rouzoumka, Jean Pinsolle, Eugénie Terreaux et al. (+3 more)
Diffusion models learn a time-indexed score field $\mathbf{s}_\theta(\mathbf{x}_t,t)$ that often inherits approximate equivariances (flips, rotations, circular shifts) from in-distribution (ID) data and convolutional backbones. Most
diffusion-based out-of-distribution (OOD) detectors exploit score magnitude or local geometry (energies,
curvature, covariance spectra) and largely ignore equivariances. We introduce Group-Equivariant Posterior Consistency (GEPC), a training-free probe that measures how consistently the learned score transforms under a finite group $\mathcal{G}$, detecting equivariance breaking even when score magnitude remains unchanged. At the population level, we propose the ideal GEPC residual, which averages an equivariance-residual functional over $\mathcal{G}$, and we derive ID upper bounds and OOD lower bounds under mild assumptions. GEPC requires only score evaluations and produces interpretable equivariance-breaking maps. On OOD image
benchmark datasets, we show that GEPC achieves competitive or improved AUROC compared to recent
diffusion-based baselines while remaining computationally lightweight. On high-resolution synthetic aperture radar imagery where OOD corresponds to targets or anomalies in clutter, GEPC yields strong target-background separation and visually interpretable equivariance-breaking maps. Code is available at https://github.com/RouzAY/gepc-
diffusion/.
Read the full abstract ▾
All authors Yadang Alexis Rouzoumka, Jean Pinsolle, Eugénie Terreaux, Christèle Morisseau, Jean-Philippe Ovarlez, Chengfang Ren
Machine Learning · Cross-listed from cs.AI
From Gameplay Traces to Game Mechanics: Causal Induction with Large Language Models
Mohit Jiwatode, Alexander Dockhorn and Bodo Rosenhahn
Deep learning
agents can achieve high performance in complex game domains without often understanding the underlying causal game mechanics. To address this, we investigate Causal Induction: the ability to infer governing laws from observational data, by tasking Large
Language Models (LLMs) with reverse-engineering
Video Game Description
Language (VGDL) rules from gameplay traces. To reduce redundancy, we select nine representative games from the General
Video Game AI (GVGAI) framework using semantic
embeddings and clustering. We compare two approaches to VGDL generation: direct code generation from observations, and a two-stage method that first infers a structural causal model (SCM) and then translates it into VGDL. Both approaches are evaluated across multiple prompting strategies and controlled context regimes, varying the amount and form of information provided to the model, from just raw gameplay observations to partial VGDL specifications. Results show that the SCM-based approach more often produces VGDL descriptions closer to the ground truth than direct generation, achieving preference win rates of up to 81% in blind evaluations and yielding fewer logically inconsistent rules. These learned SCMs can be used for downstream use cases such as causal reinforcement learning, interpretable
agents, and procedurally generating novel but logically consistent games.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
Learning to Price: Interpretable Attribute-Level Models for Dynamic Markets
Srividhya Sethuraman and Chandrashekar Lakshminarayanan
Dynamic pricing in high-dimensional
markets poses fundamental challenges of scalability, uncertainty, and interpretability. Existing low-rank bandit formulations learn efficiently but rely on latent features that obscure how individual product attributes influence price. We address this by introducing an interpretable \emph{Additive Feature Decomposition-based Low-Dimensional Demand (AFDLD) model}, where product prices are expressed as the sum of attribute-level contributions and substitution effects are explicitly modeled. Building on this structure, we propose ADEPT (Additive DEcomposition for Pricing with cross-elasticity and Time-adaptive learning)-a projection-free, gradient-free online learning
algorithm that operates directly in attribute space and achieves a sublinear regret of $\tilde{\mathcal{O}}(\sqrt{d}T^{3/4})$. Through controlled synthetic studies and real-world
datasets, we show that ADEPT (i) learns near-optimal prices under dynamic
market conditions, (ii) adapts rapidly to shocks and drifts, and (iii) yields transparent, attribute-level price explanations. The results demonstrate that interpretability and efficiency in autonomous pricing
agents can be achieved jointly through structured, attribute-driven representations.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CR
RPP: A Certified Poisoned-Sample Detection Framework for Backdoor Attacks under Dataset Imbalance
Miao Lin, Feng Yu, Rui Ning et al. (+6 more)
Deep neural networks are highly susceptible to
backdoor attacks, yet most defense methods to date rely on balanced data, overlooking the pervasive class imbalance in real-world scenarios that can amplify
backdoor threats. This paper presents the first in-depth investigation of how the
dataset imbalance amplifies
backdoor vulnerability, showing that (i) the imbalance induces a majority-class bias that increases susceptibility and (ii) conventional defenses degrade significantly as the imbalance grows. To address this, we propose Randomized Probability Perturbation (RPP), a certified poisoned-sample detection framework that operates in a black-box setting using only model output probabilities. For any inspected sample, RPP determines whether the input has been
backdoor-manipulated, while offering provable within-domain detectability guarantees and a probabilistic upper bound on the false positive rate. Extensive experiments on five
benchmarks (MNIST, SVHN, CIFAR-10, TinyImageNet and ImageNet10) covering 10
backdoor attacks and 12 baseline defenses show that RPP achieves significantly higher detection accuracy than state-of-the-art defenses, particularly under
dataset imbalance. RPP establishes a theoretical and practical foundation for defending against
backdoor attacks in real-world environments with imbalanced data.
Read the full abstract ▾
All authors Miao Lin, Feng Yu, Rui Ning, Lusi Li, Jiawei Chen, Qian Lou, Mengxin Zheng, Chunsheng Xin, Hongyi Wu