Machine Learning · Cross-listed from cs.AI
A Rational Account of Categorization Based on Information Theory
Christopher J. MacLellan, Karthik Singaravadivelan, Xin Lian et al. (+2 more)
We present a new theory of categorization based on an information-theoretic rational analysis. To evaluate this theory, we investigate how well it can account for key findings from classic categorization experiments conducted by Hayes-Roth and Hayes-Roth (1977), Medin and Schaffer (1978), and Smith and Minda (1998). We find that it explains the human categorization behavior as well as (or better) than the independent cue and context models (Medin & Schaffer, 1978), the rational model of categorization (Anderson, 1991), and a hierarchical Dirichlet process model (Griffiths et al., 2007).
Read the full abstract ▾
All authors Christopher J. MacLellan, Karthik Singaravadivelan, Xin Lian, Zekun Wang, Pat Langley
Machine Learning · Cross-listed from cs.DC
A Holistic Framework for Automated Configuration Recommendation for Cloud Service Monitoring
Anson Bastos, Shreeya Venneti, Anjaly Parayil et al. (+3 more)
Reliability of large-scale cloud services is critical for user satisfaction and business continuity. Despite significant investments in reliability engineering, production incidents remain inevitable, often leading to customer impact and operational overhead. In large cloud companies, multiple services are deployed across regions necessitating robust health monitoring systems. However, the current monitor configuration process is manual, largely reactive and ad hoc, resulting in gaps in coverage and redundant alerts. In this paper, we present a comprehensive study of monitor creation in Microsoft, identifying key components in the existing process. We further design a modular recommendation framework that processes the graph structured service entities to suggest optimal monitor configurations. Through extensive experimentation on historical data and user study of recommendations for production services at Microsoft, we demonstrate the efficacy of our approach in providing relevant recommendations for monitor configurations.
Read the full abstract ▾
All authors Anson Bastos, Shreeya Venneti, Anjaly Parayil, Ayush Choure, Chetan Bansal, Rujia Wang
Machine Learning
Knowledge Graph and Hypergraph Transformers with Repository-Attention and Journey-Based Role Transport
Mahesh Godavarti
We present a concise architecture for joint training on sentences and structured data while keeping knowledge and
language representations separable. The model treats knowledge graphs and hypergraphs as structured instances with role slots and encodes them into a key-value repository that a
language transformer can attend over.
Attention is conditioned by journey-based role transport, which unifies edge-labeled KG traversal, hyperedge traversal, and sentence structure. We outline a dual-stream architecture, hierarchical layer groups with instance-local, neighborhood, and global mixing
attention,
retrieval over a separate repository, and multi-task objectives spanning masked
language modeling, link prediction, and role-consistency denoising. The result is an explicit, inspectable separation between linguistic context and structured knowledge, while still enabling tight
alignment through cross-
attention.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
From Exact Hits to Close Enough: Semantic Caching for LLM Embeddings
Dvir David Biton and Roy Friedman
The rapid adoption of large
language models (LLMs) has created demand for faster responses and lower costs. Semantic caching, reusing semantically similar requests via their
embeddings, addresses this need but breaks classic cache assumptions and raises new challenges. In this paper, we explore offline policies for semantic caching, proving that implementing an optimal offline policy is NP-hard, and propose several
polynomial-time heuristics. We also present online semantic aware cache policies that combine recency, frequency, and locality. Evaluations on diverse
datasets show that while frequency based policies are strong baselines, our novel variant improves semantic accuracy. Our findings reveal effective strategies for current systems and highlight substantial headroom for future innovation. All code is open source.
Read the full abstract ▾
Machine Learning
Patient-Centered, Graph-Augmented Artificial Intelligence-Enabled Passive Surveillance for Early Stroke Risk Detection in High-Risk Individuals
Jiyeong Kim, Stephen P. Ma, Nirali Vora et al. (+11 more)
Stroke affected millions annually, yet poor symptom recognition often delayed care-seeking. To address risk recognition gap, we developed a passive surveillance system for early stroke risk detection using patient-reported symptoms among individuals with diabetes. Constructing a symptom taxonomy grounded in patients own
language and a dual machine learning pipeline (heterogeneous GNN and EN/LASSO), we identified symptom patterns associated with subsequent stroke. We translated findings into a hybrid risk screening system integrating symptom relevance and temporal proximity, evaluated across 3-90 day windows through EHR-based simulations. Under conservative thresholds, intentionally designed to minimize false alerts, the screening system achieved high specificity (1.00) and prevalence-adjusted positive predictive value (1.00), with good sensitivity (0.72), an expected trade-off prioritizing precision, that was highest in 90-day window. Patient-reported
language alone supported high-precision, low-burden early stroke risk detection, that could offer a valuable time window for clinical evaluation and intervention for high-risk individuals.
Read the full abstract ▾
All authors Jiyeong Kim, Stephen P. Ma, Nirali Vora, Nicholas W. Larsen, Julia Adler-Milstein, Jonathan H. Chen, Selen Bozkurt, Abeed Sarker, Juhee Cho, Jindeok Joo, Natali Pageler, Fatima Rodriguez, Christopher Sharp, Eleni Linos
Machine Learning
AngelSlim: A more accessible, comprehensive, and efficient toolkit for large model compression
Rui Cen, QiangQiang Hu, Hong Huang et al. (+10 more)
This technical report introduces AngelSlim, a comprehensive and versatile toolkit for large model compression developed by the Tencent Hunyuan team. By consolidating cutting-edge
algorithms, including quantization, speculative decoding,
token pruning, and
distillation. AngelSlim provides a unified pipeline that streamlines the transition from model compression to industrial-scale deployment. To facilitate efficient acceleration, we integrate state-of-the-art FP8 and INT8 Post-Training Quantization (PTQ)
algorithms alongside pioneering research in ultra-low-bit regimes, featuring HY-1.8B-int2 as the first industrially viable 2-bit large model. Beyond quantization, we propose a training-aligned speculative decoding framework compatible with
multimodal architectures and modern inference engines, achieving 1.8x to 2.0x throughput gains without compromising output correctness. Furthermore, we develop a training-free sparse
attention framework that reduces Time-to-First-
Token (TTFT) in long-context scenarios by decoupling sparse kernels from model architectures through a hybrid of static patterns and dynamic
token selection. For
multimodal models, AngelSlim incorporates specialized
pruning strategies, namely IDPruner for optimizing
vision tokens via Maximal Marginal Relevance and Samp for adaptive audio
token merging and
pruning. By integrating these compression strategies from low-level implementations, AngelSlim enables
algorithm-focused research and tool-assisted deployment.
Read the full abstract ▾
All authors Rui Cen, QiangQiang Hu, Hong Huang, Hong Liu, Song Liu, Xin Luo, Lin Niu, Yifan Tan, Decheng Wu, Linchuan Xie, Rubing Yang, Guanghua Yu, Jianchen Zhu
Machine Learning
Urban Vibrancy Embedding and Application on Traffic Prediction
Sumin Han, Jisun An and Dongman Lee
Urban vibrancy reflects the dynamic human activity within urban spaces and is often measured using mobile data that captures floating population trends. This study proposes a novel approach to derive Urban Vibrancy
embeddings from real-time floating population data to enhance traffic prediction models. Specifically, we utilize variational autoencoders (VAE) to compress this data into actionable
embeddings, which are then integrated with long short-term
memory (LSTM) networks to predict future
embeddings. These are subsequently applied in a sequence-to-sequence framework for traffic
forecasting. Our contributions are threefold: (1) We use principal component analysis (PCA) to interpret the
embeddings, revealing temporal patterns such as weekday versus weekend distinctions and seasonal patterns; (2) We propose a method that combines VAE and LSTM, enabling
forecasting dynamic urban knowledge
embedding; and (3) Our approach improves accuracy and responsiveness in traffic prediction models, including RNN, DCRNN, GTS, and GMAN. This study demonstrates the potential of Urban Vibrancy
embeddings to advance traffic prediction and offer a more nuanced analysis of urban mobility.
Read the full abstract ▾
Machine Learning · Cross-listed from eess.SP
Deep Neural Network Architectures for Electrocardiogram Classification: A Comprehensive Evaluation
Yun Song, Wenjia Zheng, Tiedan Chen et al. (+3 more)
With the rising prevalence of cardiovascular diseases, electrocardiograms (ECG) remain essential for the non-invasive detection of cardiac abnormalities. This study presents a comprehensive evaluation of deep neural network architectures for automated arrhythmia classification, integrating temporal modeling,
attention mechanisms, and ensemble strategies. To address data scarcity in minority classes, the MIT-BIH Arrhythmia
dataset was augmented using a Generative
Adversarial Network (GAN). We developed and compared four distinct architectures, including Convolutional Neural Networks (CNN), CNN combined with Long Short-Term
Memory (CNN-LSTM), CNN-LSTM with
Attention, and 1D Residual Networks (ResNet-1D), to capture both local morphological features and long-term temporal dependencies. Performance was rigorously evaluated using accuracy, F1-score, and Area Under the Curve (AUC) with 95% confidence intervals to ensure statistical robustness, while Gradient-weighted Class Activation Mapping (Grad-CAM) was employed to validate model interpretability. Experimental results indicate that the CNN-LSTM model achieved the optimal stand-alone balance between sensitivity and specificity, yielding an F1-score of 0.951. Conversely, the CNN-LSTM-
Attention and ResNet-1D models exhibited higher sensitivity to class imbalance. To mitigate this, a dynamic ensemble fusion strategy was introduced; specifically, the Top2-Weighted ensemble achieved the highest overall performance with an F1-score of 0.958. These findings demonstrate that leveraging complementary deep architectures significantly enhances classification reliability, providing a robust and interpretable foundation for intelligent arrhythmia detection systems.
Read the full abstract ▾
All authors Yun Song, Wenjia Zheng, Tiedan Chen, Ziyu Wang, Jiazhao Shi, Yisong Chen
Machine Learning · Cross-listed from physics.comp-ph
Surrogate Modeling for Neutron Transport: A Neural Operator Approach
Md Hossain Sahadath, Qiyun Cheng, Shaowu Pan et al. (+1 more)
This work introduces a neural operator based surrogate modeling framework for
neutron transport computation. Two architectures, the Deep Operator Network (DeepONet) and the
Fourier Neural Operator (FNO), were trained for fixed source problems to learn the mapping from anisotropic
neutron sources, Q(x,{\mu}), to the corresponding angular fluxes, {\psi}(x,{\mu}), in a one-dimensional slab geometry. Three distinct models were trained for each neural operator, corresponding to different scattering ratios (c = 0.1, 0.5, & 1.0), providing insight into their performance across distinct transport regimes (absorption-dominated, moderate, and scattering-dominated). The models were subsequently evaluated on a wide range of previously unseen source configurations, demonstrating that FNO generally achieves higher predictive accuracy, while DeepONet offers greater computational efficiency. Both models offered significant speedups that become increasingly pronounced as the scattering ratio increases, requiring <0.3% of the runtime of a conventional S_N solver. The surrogate models were further incorporated into the S_N k-
eigenvalue solver, replacing the computationally intensive transport sweep loop with a single forward pass. Across varying fission cross sections and spatial-angular grids, both neural operator solvers reproduced reference
eigenvalues with deviations up to 135 pcm for DeepONet and 112 pcm for FNO, while reducing runtime to <0.1% of that of the S_N solver on relatively fine grids. These results demonstrate the strong potential of neural operator frameworks as accurate, efficient, and generalizable surrogates for
neutron transport, paving the way for real-time digital twin applications and repeated evaluations, such as in design optimization.
Read the full abstract ▾
All authors Md Hossain Sahadath, Qiyun Cheng, Shaowu Pan, Wei Ji
Machine Learning · Cross-listed from cs.NI
Benchmarking Anomaly Detection Across Heterogeneous Cloud Telemetry Datasets
Mohammad Saiful Islam and Andriy Miranskyy
Anomaly detection is important for keeping cloud systems reliable and stable. Deep learning has improved time-series anomaly detection, but most models are evaluated on one
dataset at a time. This raises questions about whether these models can handle different types of telemetry, especially in large-scale and high-dimensional environments. In this study, we evaluate four deep learning models, GRU, TCN,
Transformer, and TSMixer. We also include Isolation Forest as a classical baseline. The models are tested across four telemetry
datasets: the Numenta Anomaly
Benchmark, Microsoft Cloud Monitoring
dataset, Exathlon
dataset, and IBM Console
dataset. These
datasets differ in structure, dimensionality, and labelling strategy. They include univariate time series, synthetic multivariate workloads, and real-world production telemetry with over 100,000 features. We use a unified training and evaluation pipeline across all
datasets. The evaluation includes NAB-style metrics to capture early detection behaviour for
datasets where anomalies persist over contiguous time intervals. This enables window-based scoring in settings where anomalies occur over contiguous time intervals, even when labels are recorded at the point level. The unified setup enables consistent analysis of model behaviour under shared scoring and calibration assumptions. Our results demonstrate that anomaly detection performance in cloud systems is governed not only by model architecture, but critically by calibration stability and feature-space geometry. By releasing our preprocessing pipelines,
benchmark configuration, and evaluation artifacts, we aim to support reproducible and deployment-aware evaluation of anomaly detection systems for cloud environments.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Explanatory Interactive Machine Learning for Bias Mitigation in Visual Gender Classification
Nathanya Satriani, Djordje Slijepčević, Markus Schedl et al. (+1 more)
Explanatory interactive learning (XIL) enables users to guide model training in machine learning (ML) by providing feedback on the model's explanations, thereby helping it to focus on features that are relevant to the prediction from the user's perspective. In this study, we explore the capability of this learning paradigm to mitigate bias and spurious correlations in visual classifiers, specifically in scenarios prone to data bias, such as gender classification. We investigate two methodologically different state-of-the-art XIL strategies, i.e., CAIPI and Right for the Right Reasons (RRR), as well as a novel hybrid approach that combines both strategies. The results are evaluated quantitatively by comparing
segmentation masks with explanations generated using Gradient-weighted Class Activation Mapping (GradCAM) and Bounded Logit
Attention (BLA). Experimental results demonstrate the effectiveness of these methods in (i) guiding ML models to focus on relevant image features, particularly when CAIPI is used, and (ii) reducing model bias (i.e., balancing the misclassification rates between male and female predictions). Our analysis further supports the potential of XIL methods to improve
fairness in gender classifiers. Overall, the increased transparency and
fairness obtained by XIL leads to slight performance decreases with an exception being CAIPI, which shows potential to even improve classification accuracy.
Read the full abstract ▾
All authors Nathanya Satriani, Djordje Slijepčević, Markus Schedl, Matthias Zeppelzauer
Machine Learning · Cross-listed from math.NA
SVD-Preconditioned Gradient Descent Method for Solving Nonlinear Least Squares Problems
Zhipeng Chang, Wenrui Hao and Nian Liu
This paper introduces a novel optimization
algorithm designed for nonlinear least-squares problems. The method is derived by preconditioning the gradient descent direction using the Singular Value Decomposition (SVD) of the Jacobian. This SVD-based preconditioner is then integrated with the first- and second-moment adaptive learning rate mechanism of the Adam optimizer. We establish the local linear convergence of the proposed method under standard regularity assumptions and prove global convergence for a modified version of the
algorithm under suitable conditions. The effectiveness of the approach is demonstrated experimentally across a range of tasks, including function approximation, partial differential equation (PDE) solving, and image classification on the CIFAR-10
dataset. Results show that the proposed method consistently outperforms standard Adam, achieving faster convergence and lower error in both regression and classification settings.
Read the full abstract ▾
Machine Learning
Efficient Adaptive Data Analysis over Dense Distributions
Joon Suk Huh
Modern data workflows are inherently adaptive, repeatedly querying the same
dataset to refine and validate sequential decisions, but such adaptivity can lead to overfitting and invalid statistical inference. Adaptive Data Analysis (ADA) mechanisms address this challenge; however, there is a fundamental tension between computational efficiency and sample complexity. For $T$ rounds of adaptive analysis, computationally efficient
algorithms typically incur suboptimal $O(\sqrt{T})$ sample complexity, whereas for arbitrary distributions,
algorithms with $O(\log T)$ sample complexity are computationally intractable under standard cryptographic assumptions. In this work, we shed light on this trade-off by identifying a natural class of data distributions under which both computational efficiency and logarithmic sample complexity in $T$ are achievable. We propose a computationally efficient ADA mechanism that attains $O(\log T)$ sample complexity, optimal in its dependence on $T$, when the data distribution is dense with respect to a known prior. This setting includes, in particular, feature-label data distributions arising in distribution-specific learning. As a consequence, our mechanism also yields a sample-efficient (i.e., $O(\log T)$ samples) statistical query oracle in the distribution-specific setting. Moreover, although our
algorithm is not based on differential
privacy, it satisfies a relaxed
privacy notion known as Predicate Singling Out (PSO) security (Cohen and Nissim 2020) whenever the number of its possible update transcripts is
polynomial in the sample size. Our results thus reveal an inherent connection between adaptive data analysis and
privacy beyond differential
privacy.
Read the full abstract ▾
Machine Learning
The Laplacian Keyboard: Beyond the Linear Span
Siddarth Chandrasekar and Marlos C. Machado
Across scientific disciplines,
Laplacian eigenvectors serve as a fundamental basis for simplifying complex systems, from signal processing to
quantum mechanics. In reinforcement learning (RL), they similarly form a basis over the state space, enabling reward functions to be approximated by projection onto a small set of eigenvectors. This projection makes zero-shot control possible, but it also imposes a fundamental limitation: the induced policies are only as expressive as the linear span of the chosen eigenvectors. We introduce the
Laplacian Keyboard (LK), a hierarchical framework that goes beyond this linear span. LK constructs a task-agnostic library of behaviors from these eigenvectors, forming a behavior basis guaranteed to contain the optimal policy for any reward within the linear span. A meta-policy learns to stitch these behaviors dynamically, enabling efficient learning of policies outside the original linear constraints. We establish theoretical bounds on zero-shot approximation error and demonstrate empirically that LK improves over the zero-shot solution while achieving better sample efficiency compared to standard RL methods.
Read the full abstract ▾
Machine Learning
Do We Need Adam? Surprisingly Strong and Sparse Reinforcement Learning with SGD in LLMs
Sagnik Mukherjee, Lifan Yuan, Pavan Jayasinha et al. (+2 more)
Reinforcement learning (RL), particularly RL from verifiable reward (RLVR), has become a crucial phase of training large
language models (LLMs) and a key focus of current scaling efforts. However, optimization practices in RL largely follow those of next-
token prediction stages (e.g., pretraining and supervised fine-tuning), despite fundamental differences between RL and these stages highlighted by recent work. One such practice is the use of the AdamW optimizer, which is widely adopted for training large-scale
transformers despite its high
memory overhead. Our analysis shows that both momentum and adaptive learning rates in AdamW are less influential in RL than in SFT, leading us to hypothesize that RL benefits less from Adam-style per-parameter adaptive learning rates and momentum. Confirming this hypothesis, our experiments demonstrate that the substantially more
memory-efficient SGD, which is known to perform poorly in supervised learning of large-scale
transformers, matches or even outperforms AdamW in RL for LLMs. Remarkably, full fine-tuning with SGD updates fewer than 0.02% of model parameters without any sparsity-promoting regularization, more than 1000 times fewer than AdamW. Our analysis offers potential reasons for this update sparsity. These findings provide new insights into the optimization dynamics of RL in LLMs and show that RL can be substantially more parameter-efficient than previously recognized.
Read the full abstract ▾
All authors Sagnik Mukherjee, Lifan Yuan, Pavan Jayasinha, Dilek Hakkani-Tür, Hao Peng
Machine Learning
ParisKV: Fast and Drift-Robust KV-Cache Retrieval for Long-Context LLMs
Yanlin Qi, Xinhang Chen, Huiqiang Jiang et al. (+3 more)
KV-cache
retrieval is essential for long-context LLM inference, yet existing methods struggle with distribution drift and high latency at scale. We introduce ParisKV, a drift-robust, GPU-native KV-cache
retrieval framework based on collision-based candidate selection, followed by a quantized inner-product reranking
estimator. For million-
token contexts, ParisKV supports CPU-offloaded KV caches via Unified Virtual Addressing (UVA), enabling on-demand top-$k$ fetching with minimal overhead. ParisKV matches or outperforms full
attention quality on long-input and long-generation
benchmarks. It achieves state-of-the-art long-context decoding efficiency: it matches or exceeds full
attention speed even at batch size 1 for long contexts, delivers up to 2.8$\times$ higher throughput within full
attention's runnable range, and scales to million-
token contexts where full
attention runs out of
memory. At million-
token scale, ParisKV reduces decode latency by 17$\times$ and 44$\times$ compared to MagicPIG and PQCache, respectively, two state-of-the-art KV-cache Top-$k$
retrieval baselines, code is available at https://github.com/amy-77/ParisKV/tree/main.
Read the full abstract ▾
All authors Yanlin Qi, Xinhang Chen, Huiqiang Jiang, Qitong Wang, Botao Peng, Themis Palpanas
Machine Learning
Efficient Planning in Reinforcement Learning via Model Introspection
Gabriel Stella
Reinforcement learning and classical planning are typically seen as two distinct problems, with differing formulations necessitating different solutions. Yet, when humans are given a task, regardless of the way it is specified, they can often derive the additional information needed to solve the problem efficiently. The key to this ability is introspection: by
reasoning about their internal models of the problem, humans directly synthesize additional task-relevant information. In this paper, we propose that this introspection can be thought of as program analysis. We discuss examples of how this approach can be applied to various kinds of models used in reinforcement learning. We then describe an
algorithm that enables efficient goal-oriented planning over the class of models used in relational reinforcement learning, demonstrating a novel link between reinforcement learning and classical planning.
Read the full abstract ▾
Machine Learning
Analyzing and Guiding Zero-Shot Posterior Sampling in Diffusion Models
Roi Benita, Michael Elad and Joseph Keshet
Recovering a signal from its degraded measurements is a long standing challenge in science and engineering. Recently, zero-shot
diffusion based methods have been proposed for such inverse problems, offering a posterior sampling based solution that leverages prior knowledge. Such
algorithms incorporate the observations through inference, often leaning on manual tuning and heuristics. In this work we propose a rigorous analysis of these approximate posterior samplers, relying on a Gaussianity assumption of the prior. Under this regime, we show that both the ideal posterior sampler and
diffusion-based reconstruction
algorithms can be expressed in closed-form, enabling their thorough analysis and comparisons in the spectral domain. Building on these representations, we introduce a principled framework for parameter design, replacing heuristic selection strategies used to date. The proposed approach is method-agnostic and yields tailored parameter choices that jointly account for the characteristics of the prior, the degraded signal, and the
diffusion dynamics. We show that our spectral recommendations differ structurally from standard heuristics and vary with the
diffusion step size, resulting in a consistent balance between perceptual quality and signal fidelity.
Read the full abstract ▾
Machine Learning
Towards Robust Scaling Laws for Optimizers
Alexandra Volkova, Mher Safaryan, Christoph H. Lampert et al. (+1 more)
The quality of Large
Language Model (LLM) pretraining depends on multiple factors, including the compute budget and the choice of optimization
algorithm. Empirical scaling laws are widely used to predict loss as model size and training data grow, however, almost all existing studies fix the optimizer (typically AdamW). At the same time, a new generation of optimizers (e.g.,
Muon, Shampoo, SOAP) promises faster and more stable convergence, but their relationship with model and data scaling is not yet well understood. In this work, we study scaling laws across different optimizers. Empirically, we show that 1) separate Chinchilla-style scaling laws for each optimizer are ill-conditioned and have highly correlated parameters. Instead, 2) we propose a more robust law with shared power-law exponents and optimizer-specific rescaling factors, which enable direct comparison between optimizers. Finally, 3) we provide a theoretical analysis of gradient-based methods for the proxy task of a convex quadratic objective, demonstrating that Chinchilla-style scaling laws emerge naturally as a result of loss decomposition into irreducible, approximation, and optimization errors.
Read the full abstract ▾
All authors Alexandra Volkova, Mher Safaryan, Christoph H. Lampert, Dan Alistarh
Machine Learning · Cross-listed from stat.ML
On Generation in Metric Spaces
Jiaxun Li, Vinod Raman and Ambuj Tewari
We study generation in separable metric instance spaces. We extend the
language generation framework from Kleinberg and Mullainathan [2024] beyond countable domains by defining novelty through metric separation and allowing asymmetric novelty parameters for the adversary and the generator. We introduce the $(\varepsilon,\varepsilon')$-closure dimension, a scale-sensitive analogue of closure dimension, which yields characterizations of uniform and non-uniform generatability and a sufficient condition for generation in the limit. Along the way, we identify a sharp
geometric contrast. Namely, in doubling spaces, including all finite-dimensional normed spaces, generatability is stable across novelty scales and invariant under equivalent metrics. In general metric spaces, however, generatability can be highly scale-sensitive and metric-dependent; even in the natural infinite-dimensional
Hilbert space $\ell^2$, all notions of generation may fail abruptly as the novelty parameters vary.
Read the full abstract ▾
Machine Learning
Quantifying Explanation Quality in Graph Neural Networks using Out-of-Distribution Generalization
Ding Zhang, Siddharth Betala and Chirag Agarwal
Evaluating the quality of post-hoc explanations for Graph Neural Networks (GNNs) remains a significant challenge. While recent years have seen an increasing development of explainability methods, current evaluation metrics (e.g., fidelity, sparsity) often fail to assess whether an explanation identifies the true underlying causal variables. To address this, we propose the Explanation-Generalization Score (EGS), a metric that quantifies the causal relevance of GNN explanations. EGS is founded on the principle of feature invariance and posits that if an explanation captures true causal drivers, it should lead to stable predictions across distribution shifts. To quantify this, we introduce a framework that trains GNNs using explanatory subgraphs and evaluates their performance in Out-of-Distribution (OOD) settings (here, OOD generalization serves as a rigorous proxy for the explanation's causal validity). Through large-scale validation involving 11,200 model combinations across synthetic and real-world
datasets, our results demonstrate that EGS provides a principled
benchmark for ranking explainers based on their ability to capture causal substructures, offering a robust alternative to traditional fidelity-based metrics.
Read the full abstract ▾
Machine Learning
Dense Feature Learning via Linear Structure Preservation in Medical Data
Yuanyun Zhang, Mingxuan Zhang, Siyuan Li et al. (+4 more)
Deep learning models for medical data are typically trained using task specific objectives that encourage representations to collapse onto a small number of discriminative directions. While effective for individual prediction problems, this paradigm underutilizes the rich structure of clinical data and limits the transferability, stability, and interpretability of learned features. In this work, we propose dense feature learning, a representation centric framework that explicitly shapes the linear structure of medical
embeddings. Our approach operates directly on
embedding matrices, encouraging spectral balance, subspace consistency, and feature orthogonality through objectives defined entirely in terms of linear algebraic properties. Without relying on labels or generative reconstruction, dense feature learning produces representations with higher effective rank, improved conditioning, and greater stability across time. Empirical evaluations across longitudinal EHR data, clinical text, and
multimodal patient representations demonstrate consistent improvements in downstream linear performance, robustness, and subspace
alignment compared to supervised and self supervised baselines. These results suggest that learning to span clinical variation may be as important as learning to predict clinical outcomes, and position representation geometry as a first class objective in medical AI.
Read the full abstract ▾
All authors Yuanyun Zhang, Mingxuan Zhang, Siyuan Li, Zihan Wang, Haoran Chen, Wenbo Zhou, Shi Li
Machine Learning
On the Infinite Width and Depth Limits of Predictive Coding Networks
Francesco Innocenti, El Mehdi Achour and Rafal Bogacz
Predictive coding (PC) is a biologically plausible alternative to standard backpropagation (BP) that minimises an energy function with respect to network activities before updating weights. Recent work has improved the training stability of deep PC networks (PCNs) by leveraging some BP-inspired reparameterisations, but the scalability and theoretical basis of these methods remain unclear. To address this gap, we study the infinite width and depth limits of PCNs. For linear networks, we derive stable and "non-lazy" parameterisations when scaling both the model width and depth, revealing that the output of standard PCNs explodes with width during training. Moreover, under stable parameterisations, we show that the gradients computed by PC at activity
equilibrium converge to the BP gradients for networks that are much wider than deep ($depth/width\to0$). Experiments show high gradient
alignment between PC and BP at large width for different nonlinear models, including convolutional networks and
transformers. Overall, this work constrains the parameterisations that are scalable with PC, while suggesting how BP could be implemented using only local updates in much wider than deep networks like the
brain.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Vision and Language: Novel Representations and Artificial intelligence for Driving Scene Safety Assessment and Autonomous Vehicle Planning
Ross Greer, Maitrayee Keskar, Angel Martinez-Sanchez et al. (+3 more)
Vision-
language models (VLMs) have recently emerged as powerful representation learning systems that align visual observations with natural
language concepts, offering new opportunities for semantic
reasoning in safety-critical autonomous driving. This paper investigates how
vision-
language representations support driving scene safety assessment and decision-making when integrated into perception, prediction, and planning pipelines. We study three complementary system-level use cases. First, we introduce a lightweight, category-agnostic hazard screening approach leveraging CLIP-based image-text similarity to produce a low-latency semantic hazard signal. This enables robust detection of diverse and out-of-distribution road hazards without explicit object detection or visual question answering. Second, we examine the integration of scene-level
vision-
language embeddings into a
transformer-based trajectory planning framework using the Waymo Open
Dataset. Our results show that naively conditioning planners on global
embeddings does not improve trajectory accuracy, highlighting the importance of representation-task
alignment and motivating the development of task-informed extraction methods for safety-critical planning. Third, we investigate natural
language as an explicit behavioral constraint on motion planning using the doScenes
dataset. In this setting, passenger-style instructions grounded in visual scene elements suppress rare but severe planning failures and improve safety-aligned behavior in ambiguous scenarios. Taken together, these findings demonstrate that
vision-
language representations hold significant promise for autonomous driving safety when used to express semantic risk, intent, and behavioral constraints. Realizing this potential is fundamentally an engineering problem requiring careful system design and structured grounding rather than direct feature injection.
Read the full abstract ▾
All authors Ross Greer, Maitrayee Keskar, Angel Martinez-Sanchez, Parthib Roy, Shashank Shriram, Mohan Trivedi
Machine Learning
Spectral Gating Networks
Jusheng Zhang, Yijia Fan, Kaitong Cai et al. (+5 more)
Gating mechanisms are ubiquitous, yet a complementary question in feed-forward networks remains under-explored: how to introduce frequency-rich expressivity without sacrificing stability and scalability? This tension is exposed by spline-based Kolmogorov-Arnold Network (KAN) parameterizations, where grid refinement can induce parameter growth and brittle optimization in high dimensions. To propose a stability-preserving way to inject spectral capacity into existing MLP/FFN layers under fixed parameter and training budgets, we introduce Spectral Gating Networks (SGN), a drop-in spectral reparameterization. SGN augments a standard activation pathway with a compact spectral pathway and learnable gates that allow the model to start from a stable base behavior and progressively allocate capacity to spectral features during training. The spectral pathway is instantiated with trainable Random
Fourier Features (learned frequencies and phases), replacing grid-based splines and removing resolution dependence. A hybrid GELU-
Fourier formulation further improves optimization robustness while enhancing high-frequency fidelity. Across
vision, NLP, audio, and PDE
benchmarks, SGN consistently improves accuracy-efficiency trade-offs under comparable computational budgets, achieving 93.15% accuracy on CIFAR-10 and up to 11.7x faster inference than spline-based KAN variants. Code and trained models will be released.
Read the full abstract ▾
All authors Jusheng Zhang, Yijia Fan, Kaitong Cai, Jing Yang, Yongsen Zheng, Kwok-Yan Lam, Liang Lin, Keze Wang