Machine Learning · Cross-listed from stat.ML
Revisiting Incremental Stochastic Majorization-Minimization Algorithms with Applications to Mixture of Experts
TrungKhang Tran, TrungTin Nguyen, Gersende Fort et al. (+5 more)
Processing high-volume, streaming data is increasingly common in modern statistics and machine learning, where batch-mode
algorithms are often impractical because they require repeated passes over the full
dataset. This has motivated incremental
stochastic estimation methods, including the incremental
stochastic Expectation-Maximization (EM)
algorithm formulated via
stochastic approximation. In this work, we revisit and analyze an incremental
stochastic variant of the Majorization-Minimization (MM)
algorithm, which generalizes incremental
stochastic EM as a special case. Our approach relaxes key EM requirements, such as explicit latent-variable representations, enabling broader applicability and greater algorithmic flexibility. We establish theoretical guarantees for the incremental
stochastic MM
algorithm, proving consistency in the sense that the iterates converge to a stationary point characterized by a vanishing gradient of the objective. We demonstrate these advantages on a softmax-gated mixture of experts (MoE) regression problem, for which no
stochastic EM
algorithm is available. Empirically, our method consistently outperforms widely used
stochastic optimizers, including
stochastic gradient descent, root mean square propagation, adaptive moment estimation, and second-order clipped
stochastic optimization. These results support the development of new incremental
stochastic algorithms, given the central role of softmax-gated MoE architectures in contemporary deep neural networks for heterogeneous data modeling. Beyond synthetic experiments, we also validate practical effectiveness on two real-world
datasets, including a bioinformatics study of dent maize genotypes under drought stress that integrates high-dimensional proteomics with ecophysiological traits, where incremental
stochastic MM yields stable gains in predictive performance.
Read the full abstract ▾
All authors TrungKhang Tran, TrungTin Nguyen, Gersende Fort, Tung Doan, Hien Duy Nguyen, Binh T. Nguyen, Florence Forbes, Christopher Drovandi
Machine Learning
Unsupervised Learning of Efficient Exploration: Pre-training Adaptive Policies via Self-Imposed Goals
Octavio Pappalardo
Unsupervised pre-training can equip reinforcement learning
agents with prior knowledge and accelerate learning in downstream tasks. A promising direction, grounded in human development, investigates
agents that learn by setting and pursuing their own goals. The core challenge lies in how to effectively generate, select, and learn from such goals. Our focus is on broad distributions of downstream tasks where solving every task zero-shot is infeasible. Such settings naturally arise when the target tasks lie outside of the pre-training distribution or when their identities are unknown to the
agent. In this work, we (i) optimize for efficient multi-episode exploration and adaptation within a meta-learning framework, and (ii) guide the training curriculum with evolving estimates of the
agent's post-adaptation performance. We present ULEE, an unsupervised meta-learning method that combines an in-context learner with an
adversarial goal-generation strategy that maintains training at the frontier of the
agent's capabilities. On XLand-MiniGrid
benchmarks, ULEE pre-training yields improved exploration and adaptation abilities that generalize to novel objectives, environment dynamics, and map structures. The resulting policy attains improved zero-shot and few-shot performance, and provides a strong initialization for longer fine-tuning processes. It outperforms learning from scratch, DIAYN pre-training, and alternative curricula. Code is available at: https://github.com/Octavio-Pappalardo/ulee-jax
Read the full abstract ▾
Machine Learning
Component-Aware Pruning Framework for Neural Network Controllers via Gradient-Based Importance Estimation
Ganesh Sundaram, Jonas Ulmen and Daniel Görges
The transition from monolithic to multi-component neural architectures in advanced neural network controllers poses substantial challenges due to the high computational complexity of the latter. Conventional model compression techniques for complexity reduction, such as structured
pruning based on norm-based metrics to estimate the relative importance of distinct parameter groups, often fail to capture functional significance. This paper introduces a component-aware
pruning framework that utilizes gradient information to compute three distinct importance metrics during training: Gradient Accumulation, Fisher Information, and
Bayesian Uncertainty. Experimental results with an autoencoder and a TD-MPC
agent demonstrate that the proposed framework reveals critical structural dependencies and dynamic shifts in importance that static heuristics often miss, supporting more informed compression decisions.
Read the full abstract ▾
Machine Learning
To Grok Grokking: Provable Grokking in Ridge Regression
Mingyue Xu, Gal Vardi and Itay Safran
We study grokking, the onset of generalization long after overfitting, in a classical ridge regression setting. We prove end-to-end grokking results for learning over-parameterized linear regression models using gradient descent with weight decay. Specifically, we prove that the following stages occur: (i) the model overfits the training data early during training; (ii) poor generalization persists long after overfitting has manifested; and (iii) the generalization error eventually becomes arbitrarily small. Moreover, we show, both theoretically and empirically, that grokking can be amplified or eliminated in a principled manner through proper hyperparameter tuning. To the best of our knowledge, these are the first rigorous quantitative bounds on the generalization delay (which we refer to as the "grokking time") in terms of training hyperparameters. Lastly, going beyond the linear setting, we empirically demonstrate that our quantitative bounds also capture the behavior of grokking on non-linear neural networks. Our results suggest that grokking is not an inherent failure mode of deep learning, but rather a consequence of specific training conditions, and thus does not require fundamental changes to the model architecture or learning
algorithm to avoid.
Read the full abstract ▾
Machine Learning
Knowledge-Aware Evolution for Task-Free Streaming Federated Continual Learning with Arbitrary Class Overlap
Sixing Tan and Xianmin Liu
Federated Continual Learning (FCL) leverages inter-client collaboration to better balance new knowledge acquisition and old knowledge retention on non-stationary data. However, existing FCL methods struggle to adapt to streaming scenarios where sequential and ephemerally accessible data chunks lack task identifiers and exhibit arbitrary class overlap, leading to confusion between old and new knowledge and an inability to sustain local inference on all encountered classes. To address this, we propose FedKACE with three components: 1) an adaptive mechanism that determines when to switch the inference model from the local to the global one to improve client-side inference performance; 2) a responsive gradient-balanced replay scheme that utilizes the ratio of the squared L2 gradient norms to balance client-specific knowledge between new acquisition and old retention; 3) a holistic buffer maintenance strategy that preserves highly informative and boundary-significant samples to enhance knowledge retention under class overlap.Experiments across multiple scenarios and theoretical analysis demonstrate the effectiveness of FedKACE.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
GAVEL: Towards Rule-Based Safety Through Activation Monitoring
Shir Rozenfeld, Rahul Pankajakshan, Itay Zloczower et al. (+3 more)
Large
language models (LLMs) are increasingly paired with activation-based monitoring to detect and prevent harmful behaviors that may not be apparent at the surface-text level. However, existing activation safety approaches, trained on broad misuse
datasets, struggle with poor precision, limited flexibility, and lack of interpretability. This paper introduces a new paradigm: rule-based activation safety, inspired by rule-sharing practices in cybersecurity. We propose modeling activations as cognitive elements (CEs), fine-grained, interpretable factors such as 'making a threat' and 'payment processing', that can be composed to capture nuanced, domain-specific behaviors with higher precision. Building on this representation, we present a practical framework that defines predicate rules over CEs and detects violations in real time. This enables practitioners to configure and update safeguards without retraining models or detectors, while supporting transparency and auditability. Our results show that compositional rule-based activation safety improves precision, supports domain customization, and lays the groundwork for scalable, interpretable, and auditable AI governance. We open source GAVEL and introduce GAVEL Studio, an interactive rule authoring and management tool. Code and
datasets are available at github.com/Offensive-AI-Lab/gavel.
Read the full abstract ▾
All authors Shir Rozenfeld, Rahul Pankajakshan, Itay Zloczower, Eyal Lenga, Gilad Gressel, Yisroel Mirsky
Machine Learning
The Effect of Architecture During Continual Learning
Allyson Hahn and Krishnan Raghavan
Continual learning is a challenge for models with static architecture, as they fail to adapt to when data distributions evolve across tasks. We introduce a mathematical framework that jointly models architecture and weights in a Sobolev space, enabling a rigorous investigation into the role of neural network architecture in continual learning and its effect on the forgetting loss. We derive necessary conditions for the continual learning solution and prove that learning only model weights is insufficient to mitigate catastrophic forgetting under distribution shifts. Consequently, we prove that by learning the architecture and weights simultaneously at each task, we can reduce catastrophic forgetting. To learn weights and architecture simultaneously, we formulate continual learning as a bilevel optimization problem: the upper level selects an optimal architecture for a given task, while the lower level computes optimal weights via dynamic programming over all tasks. To solve the upper level problem, we introduce a derivative-free direct search
algorithm to determine the optimal architecture. Once found, we must transfer knowledge from the current architecture to the optimal one. However, the optimal architecture will result in a weights parameter space different from the current architecture (i.e., dimensions of weights
matrices will not match). To bridge the dimensionality gap, we develop a low-rank transfer mechanism to map knowledge across architectures of mismatched dimensions. Empirical studies across regression and classification problems, including feedforward, convolutional, and graph neural networks, demonstrate that learning the optimal architecture and weights simultaneously yields substantially improved performance (up to two orders of magnitude), reduced forgetting, and enhanced robustness to noise compared with static architecture approaches.
Read the full abstract ▾
Machine Learning
Provable Learning of Random Hierarchy Models and Hierarchical Shallow-to-Deep Chaining
Yunwei Ren, Yatin Dandi, Florent Krzakala et al. (+1 more)
The empirical success of deep learning is often attributed to deep networks' ability to exploit hierarchical structure in data, constructing increasingly complex features across layers. Yet despite substantial progress in deep learning theory, most optimization results sill focus on networks with only two or three layers, leaving the theoretical understanding of hierarchical learning in genuinely deep models limited. This leads to a natural question: can we prove that deep networks, trained with gradient-based methods and standard input-label pairs, can efficiently exploit hierarchical structure? In this work, we consider Random Hierarchy Models – a hierarchical context-free grammar introduced by arXiv:2307.02129 and conjectured to separate deep and shallow networks. We prove that, under mild conditions, a deep convolutional network can be efficiently trained to learn this function class. Our proof builds on a general observation: if intermediate layers can receive clean signal from the labels and the relevant features are weakly identifiable, then layerwise training each individual layer suffices to hierarchically learn the target function.
Read the full abstract ▾
All authors Yunwei Ren, Yatin Dandi, Florent Krzakala, Jason D. Lee
Machine Learning · Cross-listed from stat.ML
Regularized $f$-Divergence Kernel Tests
Mónica Ribero, Antonin Schrab and Arthur Gretton
We propose a framework to construct practical kernel-based two-sample tests from the family of $f$-divergences. The test statistic is computed from the witness function of a regularized variational representation of the divergence, which we estimate using kernel methods. The proposed test is adaptive over hyperparameters such as the kernel bandwidth and the regularization parameter. We provide theoretical guarantees for statistical test power across our family of $f$-divergence estimates. While our test covers a variety of $f$-divergences, we bring particular focus to the Hockey-Stick divergence, motivated by its applications to differential
privacy auditing and machine unlearning evaluation. For two-sample testing, experiments demonstrate that different $f$-divergences are sensitive to different localized differences, illustrating the importance of leveraging diverse statistics. For machine unlearning, we propose a relative test that distinguishes true unlearning failures from safe distributional variations.
Read the full abstract ▾
Machine Learning
GraphDLG: Exploring Deep Leakage from Gradients in Federated Graph Learning
Shuyue Wei, Wantong Chen, Tongyu Wei et al. (+3 more)
Federated graph learning (FGL) has recently emerged as a promising
privacy-preserving paradigm that enables distributed graph learning across multiple data owners. A critical
privacy concern in
federated learning is whether an adversary can recover raw data from shared gradients, a vulnerability known as deep leakage from gradients (DLG). However, most prior studies on the DLG problem focused on image or text data, and it remains an open question whether graphs can be effectively recovered, particularly when the graph structure and node features are uniquely entangled in GNNs. In this work, we first theoretically analyze the components in FGL and derive a crucial insight: once the graph structure is recovered, node features can be obtained through a closed-form recursive rule. Building on this analysis, we propose GraphDLG, a novel approach to recover raw training graphs from shared gradients in FGL, which can utilize randomly generated graphs or client-side training graphs as auxiliaries to enhance recovery. Extensive experiments demonstrate that GraphDLG outperforms existing solutions by successfully decoupling the graph structure and node features, achieving improvements of over 5.46% (by MSE) for node feature reconstruction and over 25.04% (by AUC) for graph structure reconstruction.
Read the full abstract ▾
All authors Shuyue Wei, Wantong Chen, Tongyu Wei, Chen Gong, Yongxin Tong, Lizhen Cui
Machine Learning · Cross-listed from eess.IV
Interpretable and backpropagation-free Green Learning for efficient multi-task echocardiographic segmentation and classification
Jyun-Ping Kao, Jiaxin Yang, C.-C. Jay Kuo et al. (+1 more)
Echocardiography is a cornerstone for managing heart failure (HF), with Left Ventricular Ejection Fraction (LVEF) being a critical metric for guiding therapy. However, manual LVEF assessment suffers from high inter-observer variability, while existing Deep Learning (DL) models are often computationally intensive and data-hungry "black boxes" that impede clinical trust and adoption. Here, we propose a backpropagation-free multi-task Green Learning (MTGL) framework that performs simultaneous Left Ventricle (LV)
segmentation and LVEF classification. Our framework integrates an unsupervised VoxelHop encoder for hierarchical spatio-temporal feature extraction with a multi-level regression decoder and an XG-Boost classifier. On the EchoNet-Dynamic
dataset, our MTGL model achieves state-of-the-art classification and
segmentation performance, attaining a classification accuracy of 94.3% and a Dice Similarity Coefficient (DSC) of 0.912, significantly outperforming several advanced 3D DL models. Crucially, our model achieves this with over an order of magnitude fewer parameters, demonstrating exceptional computational efficiency. This work demonstrates that the GL paradigm can deliver highly accurate, efficient, and interpretable solutions for complex medical image analysis, paving the way for more sustainable and trustworthy artificial intelligence in clinical practice.
Read the full abstract ▾
All authors Jyun-Ping Kao, Jiaxin Yang, C.-C. Jay Kuo, Jonghye Woo
Machine Learning · Cross-listed from quant-ph
Quantum Circuit Pre-Synthesis: Learning Local Edits to Reduce $T$-count
Daniele Lizzio Bosco, Lukasz Cincio, Giuseppe Serra et al. (+1 more)
Compiling
quantum circuits into Clifford+$T$ gates is a central task for fault-tolerant
quantum computing using stabilizer codes. In the near term, $T$ gates will dominate the cost of fault tolerant implementations, and any reduction in the number of such expensive gates could mean the difference between being able to run a
circuit or not. While exact synthesis is exponentially hard in the number of
qubits, local synthesis approaches are commonly used to compile large
circuits by decomposing them into substructures. However, composing local methods leads to suboptimal compilations in key metrics such as $T$-count or
circuit depth, and their performance strongly depends on
circuit representation. In this work, we address this challenge by proposing Q-PreSyn, a strategy that, given a set of local edits preserving
circuit equivalence, uses a RL
agent to identify effective sequences of such actions and thereby obtain
circuit representations that yield a reduced $T$-count upon synthesis. Experimental results of our proposed strategy, applied on top of well-known synthesis
algorithms, show up to a $20\%$ reduction in $T$-count on
circuits with up to 25
qubits, without introducing any additional approximation error prior to synthesis.
Read the full abstract ▾
All authors Daniele Lizzio Bosco, Lukasz Cincio, Giuseppe Serra, M. Cerezo
Machine Learning
Stability and Generalization of Stochastic Nonconvex Optimization under Heavy-Tailed Noise: From Gradient Norm to Goldstein Stationarity
Hongxu Chen, Ke Wei, Xiaoming Yuan et al. (+1 more)
The empirical evidence indicates that
stochastic optimization with heavy-tailed gradient noise is more appropriate to characterize the training of machine learning models than that with standard bounded gradient variance noise. Most existing works on this phenomenon focus on the convergence of optimization errors, while the analysis for generalization bounds under the heavy-tailed gradient noise remains limited. In this paper, we develop stability-based frameworks to establish dimension-independent generalization bounds of nonconvex optimization under heavy-tailed noise for both smooth and nonsmooth settings. Specifically, we introduce a truncation argument to achieve the generalization error bound based on the algorithmic stability under finite $p$th moment conditions with $p\in(1,2]$. For the smooth case, we apply our framework to derive population gradient norm bounds for several popular
stochastic optimization
algorithms, including clipped and normalized
stochastic gradient descent, as well as their mini-batch and momentum variants. For the nonsmooth case, we establish a population Goldstein stationarity guarantee for online-to-nonconvex conversion methods. It is worth noting that our theory for the nonsmooth setting is established by introducing an on-average gradient stability for the smooth surrogate function, which does not require the additional weak convexity assumption used in existing analysis based on the Moreau envelope.
Read the full abstract ▾
All authors Hongxu Chen, Ke Wei, Xiaoming Yuan, Luo Luo
Machine Learning
Improving Policy Exploitation in Online Reinforcement Learning with Instant Retrospect Action
Gong Gao, Weidong Zhao, Xianhui Liu et al. (+1 more)
Existing value-based online reinforcement learning (RL)
algorithms suffer from slow policy exploitation due to ineffective exploration and delayed policy updates. To address these challenges, we propose an
algorithm called Instant Retrospect Action (IRA). Specifically, we propose Q-Representation Discrepancy
Evolution (RDE) to facilitate Q-network representation learning, enabling discriminative representations for neighboring state-action pairs. In addition, we adopt an explicit method to policy constraints by enabling Greedy Action Guidance (GAG). This is achieved through backtracking historical actions, which effectively enhances the policy update process. Our proposed method relies on providing the learning
algorithm with accurate $k$-nearest-neighbor action value estimates and learning to design a fast-adaptable policy through policy constraints. We further propose the Instant Policy Update (IPU) mechanism, which enhances policy exploitation by systematically increasing the frequency of policy updates. We further discover that the early-stage training conservatism of the IRA method can alleviate the overestimation bias problem in value-based RL. Experimental results show that IRA can significantly improve the learning efficiency and final performance of online RL
algorithms on eight MuJoCo continuous control tasks.The code is available at https://github.com/2706853499/IRA.
Read the full abstract ▾
All authors Gong Gao, Weidong Zhao, Xianhui Liu, Ning Jia
Machine Learning
Rethinking Divisive Hierarchical Clustering from a Distributional Perspective
Kaifeng Zhang, Kai Ming Ting, Tianrun Liang et al. (+1 more)
We uncover that current objective-based Divisive Hierarchical Clustering (DHC) methods produce a dendrogram that does not have three desired properties i.e., no unwarranted splitting, group similar clusters into a same subset, ground-truth correspondence. This shortcoming has their root cause in using a set-oriented bisecting assessment criterion. We show that this shortcoming can be addressed by using a distributional kernel, instead of the set-oriented criterion; and the resultant clusters achieve a new distribution-oriented objective to maximize the total similarity of all clusters (TSC). Our theoretical analysis shows that the resultant dendrogram guarantees a lower bound of TSC. The empirical evaluation shows the effectiveness of our proposed method on artificial and Spatial Transcriptomics (bioinformatics)
datasets. Our proposed method successfully creates a dendrogram that is consistent with the biological regions in a Spatial Transcriptomics
dataset, whereas other contenders fail.
Read the full abstract ▾
All authors Kaifeng Zhang, Kai Ming Ting, Tianrun Liang, Qiuran Zhao
Machine Learning
Scalable Exploration for High-Dimensional Continuous Control via Value-Guided Flow
Yunyue Wei, Chenhui Zuo and Yanan Sui
Controlling high-dimensional systems in biological and robotic applications is challenging due to expansive state-action spaces, where effective exploration is critical. Commonly used exploration strategies in reinforcement learning are largely undirected with sharp degradation as action dimensionality grows. Many existing methods resort to dimensionality reduction, which constrains policy expressiveness and forfeits system flexibility. We introduce Q-guided Flow Exploration (Qflex), a scalable reinforcement learning method that conducts exploration directly in the native high-dimensional action space. During training, Qflex traverses actions from a learnable source distribution along a probability flow induced by the learned value function, aligning exploration with task-relevant gradients rather than isotropic noise. Our proposed method substantially outperforms representative online reinforcement learning baselines across diverse high-dimensional continuous-control
benchmarks. Qflex also successfully controls a full-body human musculoskeletal model to perform agile, complex movements, demonstrating superior scalability and sample efficiency in very high-dimensional settings. Our results indicate that value-guided flows offer a principled and practical route to exploration at scale.
Read the full abstract ▾
Machine Learning
Generalizable Multimodal Large Language Model Editing via Invariant Trajectory Learning
Jiajie Su, Haoyuan Wang, Xiaohua Feng et al. (+6 more)
Knowledge editing emerges as a crucial technique for efficiently correcting incorrect or outdated knowledge in large
language models (LLM). Existing editing methods rely on a rigid mapping from parameter or module modifications to output, which causes the generalization limitation in
Multimodal LLM (MLLM). In this paper, we reformulate MLLM editing as an out-of-distribution (OOD) generalization problem, where the goal is to discern semantic shift with factual shift and thus achieve robust editing among diverse cross-modal prompting. The key challenge of this OOD problem lies in identifying invariant causal trajectories that generalize accurately while suppressing spurious correlations. To address it, we propose ODEdit, a plug-and-play invariant learning based framework that optimizes the tripartite OOD risk objective to simultaneously enhance editing reliability, locality, and generality.We further introduce an edit trajectory invariant learning method, which integrates a total variation penalty into the risk minimization objective to stabilize edit trajectories against environmental variations. Theoretical analysis and extensive experiments demonstrate the effectiveness of ODEdit.
Read the full abstract ▾
All authors Jiajie Su, Haoyuan Wang, Xiaohua Feng, Yunshan Ma, Xiaobo Xia, Yuyuan Li, Xiaolin Zheng, Jianmao Xiao, Chaochao Chen
Machine Learning · Cross-listed from cs.CV
Self-Supervised Weight Templates for Scalable Vision Model Initialization
Yucheng Xie, Fu Feng, Ruixiao Shi et al. (+3 more)
The increasing scale and complexity of modern model parameters underscore the importance of pre-trained models. However, deployment often demands architectures of varying sizes, exposing limitations of conventional pre-training and fine-tuning. To address this, we propose SWEET, a self-supervised framework that performs constraint-based pre-training to enable scalable initialization in
vision tasks. Instead of pre-training a fixed-size model, we learn a shared weight template and size-specific weight scalers under Tucker-based factorization, which promotes modularity and supports flexible adaptation to architectures with varying depths and widths. Target models are subsequently initialized by composing and reweighting the template through lightweight weight scalers, whose parameters can be efficiently learned from minimal training data. To further enhance flexibility in width expansion, we introduce width-wise
stochastic scaling, which regularizes the template along width-related dimensions and encourages robust, width-invariant representations for improved cross-width generalization. Extensive experiments on classification, detection,
segmentation and generation tasks demonstrate the state-of-the-art performance of SWEET for initializing variable-sized
vision models.
Read the full abstract ▾
All authors Yucheng Xie, Fu Feng, Ruixiao Shi, Jing Wang, Yong Rui, Xin Geng
Machine Learning
LoPRo: Enhancing Low-Rank Quantization via Permuted Block-Wise Rotation
Hongyaoxing Gu, Lijuan Hu, Liye Yu et al. (+2 more)
Post-training quantization (PTQ) enables effective model compression while preserving relatively high accuracy. Current weight-only PTQ methods primarily focus on the challenging sub-3-bit regime, where approaches often suffer significant accuracy degradation, typically requiring fine-tuning to achieve competitive performance. In this work, we revisit the fundamental characteristics of weight quantization and analyze the challenges in quantizing the residual
matrix under low-rank approximation. We propose LoPRo, a novel fine-tuning-free PTQ
algorithm that enhances residual
matrix quantization by applying block-wise permutation and Walsh-Hadamard transformations to rotate columns of similar importance, while explicitly preserving the quantization accuracy of the most salient column blocks. Furthermore, we introduce a mixed-precision fast low-rank decomposition based on rank-1 sketch (R1SVD) to further minimize quantization costs. Experiments demonstrate that LoPRo outperforms existing fine-tuning-free PTQ methods at both 2-bit and 3-bit quantization, achieving accuracy comparable to fine-tuning baselines. Specifically, LoPRo achieves state-of-the-art quantization accuracy on LLaMA-2 and LLaMA-3 series models while delivering up to a 4$\times$ speedup. In the MoE model Mixtral-8x7B, LoPRo completes quantization within 2.5 hours, simultaneously reducing perplexity by 0.4$\downarrow$ and improving accuracy by 8%$\uparrow$. Moreover, compared to other low-rank quantization methods, LoPRo achieves superior accuracy with a significantly lower rank, while maintaining high inference efficiency and minimal additional latency.
Read the full abstract ▾
All authors Hongyaoxing Gu, Lijuan Hu, Liye Yu, Haowei Li, Fangfang Liu
Machine Learning
Cross-Domain Offshore Wind Power Forecasting: Transfer Learning Through Meteorological Clusters
Dominic Weisser, Chloé Hashimoto-Cullen and Benjamin Guedj
Ambitious decarbonisation targets are rapidly increasing the commission of new offshore wind farms. For these newly commissioned plants to run, accurate power forecasts are needed from the onset. These allow grid stability, good reserve management and efficient energy trading. Despite machine learning models having strong performances, they tend to require large volumes of site-specific data that new farms do not yet have. To overcome this data scarcity, we propose a novel transfer learning framework that clusters power output according to
covariate meteorological features. Rather than training a single, general-purpose model, we thus forecast with an ensemble of expert models, each trained on a cluster. As these pre-trained models each specialise in a distinct weather pattern, they adapt efficiently to new sites and capture transferable,
climate-dependent dynamics. Our contributions are two-fold - we propose this novel framework and comprehensively evaluate it on eight offshore wind farms, achieving accurate cross-domain
forecasting with under five months of site-specific data. Our experiments achieve a MAE of 3.52%, providing empirical verification that reliable forecasts do not require a full annual cycle. Beyond power
forecasting, this
climate-aware transfer learning method opens new opportunities for offshore wind applications such as early-stage wind resource assessment, where reducing data requirements can significantly accelerate project development whilst effectively mitigating its inherent risks.
Read the full abstract ▾
Machine Learning
ProToken: Token-Level Attribution for Federated Large Language Models
Waris Gill, Ahmad Humayun, Ali Anwar et al. (+1 more)
Federated Learning (FL) enables collaborative training of Large
Language Models (LLMs) across distributed data sources while preserving
privacy. However, when
federated LLMs are deployed in critical applications, it remains unclear which client(s) contributed to specific generated responses, hindering debugging, malicious client identification, fair reward allocation, and trust verification. We present ProToken, a novel Provenance methodology for
Token-level attribution in
federated LLMs that addresses client attribution during autoregressive text generation while maintaining FL
privacy constraints. ProToken leverages two key insights to enable provenance at each
token: (1)
transformer architectures concentrate task-specific signals in later blocks, enabling strategic layer selection for computational tractability, and (2) gradient-based relevance weighting filters out irrelevant neural activations, focusing attribution on
neurons that directly influence
token generation. We evaluate ProToken across 16 configurations spanning four LLM architectures (Gemma, Llama, Qwen, SmolLM) and four domains (medical, financial, mathematical, coding). ProToken achieves 98% average attribution accuracy in correctly localizing responsible client(s), and maintains high accuracy when the number of clients are scaled, validating its practical viability for real-world deployment settings.
Read the full abstract ▾
All authors Waris Gill, Ahmad Humayun, Ali Anwar, Muhammad Ali Gulzar
Machine Learning
Grasynda: Graph-based Synthetic Time Series Generation
Luis Amorim, Moises Santos, Paulo J. Azevedo et al. (+2 more)
Data augmentation is a crucial tool in time series
forecasting, especially for deep learning architectures that require a large training sample size to generalize effectively. However, extensive
datasets are not always available in real-world scenarios. Although many data augmentation methods exist, their limitations include the use of transformations that do not adequately preserve data properties. This paper introduces Grasynda, a novel graph-based approach for synthetic time series generation that: (1) converts univariate time series into a network structure using a graph representation, where each state is a node and each transition is represented as a directed edge; and (2) encodes their temporal dynamics in a transition probability
matrix. We performed an extensive evaluation of Grasynda as a data augmentation method for time series
forecasting. We use three neural network variations on six
benchmark datasets. The results indicate that Grasynda consistently outperforms other time series data augmentation methods, including ones used in state-of-the-art time series foundation models. The method and all experiments are publicly available.
Read the full abstract ▾
All authors Luis Amorim, Moises Santos, Paulo J. Azevedo, Carlos Soares, Vitor Cerqueira
Machine Learning · Cross-listed from cs.CL
SynCABEL: Synthetic Contextualized Augmentation for Biomedical Entity Linking
Adam Remaki, Christel Gérardin, Eulàlia Farré-Maduell et al. (+2 more)
We present SynCABEL (Synthetic Contextualized Augmentation for Biomedical Entity Linking), a framework that addresses a central bottleneck in supervised biomedical entity linking (BEL): the scarcity of expert-annotated training data. SynCABEL leverages large
language models to generate context-rich synthetic training examples for all candidate concepts in a target knowledge base, providing broad supervision without manual annotation. We demonstrate that SynCABEL, when combined with decoder-only models and guided inference, establishes new state-of-the-art results across three widely used
multilingual benchmarks: MedMentions for English, QUAERO for French, and SPACCC for Spanish. Evaluating data efficiency, we show that SynCABEL reaches the performance of full human supervision using up to 60% less annotated data, substantially reducing reliance on labor-intensive and costly expert labeling. Finally, acknowledging that standard evaluation based on exact code matching often underestimates clinically valid predictions due to ontology redundancy, we introduce an LLM-as-a-judge protocol. This analysis reveals that SynCABEL significantly improves the rate of clinically valid predictions. Our synthetic
datasets, models, and code are released to support reproducibility and future research.
Read the full abstract ▾
All authors Adam Remaki, Christel Gérardin, Eulàlia Farré-Maduell, Martin Krallinger, Xavier Tannier
Machine Learning · Cross-listed from cs.CV
KeepLoRA: Continual Learning with Residual Gradient Adaptation
Mao-Lin Luo, Zi-Hao Zhou, Yi-Lin Zhang et al. (+3 more)
Continual learning for pre-trained
vision-
language models requires balancing three competing objectives: retaining pre-trained knowledge, preserving knowledge from a sequence of learned tasks, and maintaining the plasticity to acquire new knowledge. This paper presents a simple but effective approach called KeepLoRA to effectively balance these objectives. We first analyze the knowledge retention mechanism within the model parameter space and find that general knowledge is mainly encoded in the principal subspace, while task-specific knowledge is encoded in the residual subspace. Motivated by this finding, KeepLoRA learns new tasks by restricting LoRA parameter updates in the residual subspace to prevent interfering with previously learned capabilities. Specifically, we infuse knowledge for a new task by projecting its gradient onto a subspace orthogonal to both the principal subspace of pre-trained model and the dominant directions of previous task features. Our theoretical and empirical analyses confirm that KeepLoRA balances the three objectives and achieves state-of-the-art performance. The implementation code is available at https://github.com/MaolinLuo/KeepLoRA.
Read the full abstract ▾
All authors Mao-Lin Luo, Zi-Hao Zhou, Yi-Lin Zhang, Yuanyu Wan, Tong Wei, Min-Ling Zhang
Machine Learning
Tracking Drift: Variation-Aware Entropy Scheduling for Non-Stationary Reinforcement Learning
Tongxi Wang, Zhuoyang Xia, Xinran Chen et al. (+1 more)
Real-world reinforcement learning often faces environment drift, but most existing methods rely on static
entropy coefficients/target
entropy, causing over-exploration during stable periods and under-exploration after drift, and leaving unanswered the principled question of how exploration intensity should scale with drift magnitude. We show that, under standard assumptions,
entropy scheduling in non-stationary maximum-
entropy RL can be cast as the dynamic-regret trade-off between tracking a drifting comparator and stabilizing updates, yielding a square-root scaling rule for the
entropy weight in terms of a online non-stationarity proxy. Building on this, we propose AES–Adaptive
Entropy Scheduling–which adaptively adjusts the
entropy coefficient/temperature online using observable drift proxies during training, requiring almost no structural changes and incurring minimal overhead. Across 4
algorithm variants, 12 tasks, and 4 drift modes, AES significantly reduces the fraction of performance degradation caused by drift and accelerates recovery after abrupt changes.
Read the full abstract ▾
All authors Tongxi Wang, Zhuoyang Xia, Xinran Chen, Shan Liu
Machine Learning
R^3: Replay, Reflection, and Ranking Rewards for LLM Reinforcement Learning
Zhizheng Jiang, Kang Zhao, Weikai Xu et al. (+5 more)
Large
reasoning models (LRMs) aim to solve diverse and complex problems through structured
reasoning. Recent advances in group-based policy optimization methods have shown promise in enabling stable advantage estimation without reliance on process-level annotations. However, these methods rely on advantage gaps induced by high-quality samples within the same batch, which makes the training process fragile and inefficient when intra-group advantages collapse under challenging tasks. To address these problems, we propose a reinforcement learning mechanism named \emph{R^3} that along three directions: (1) a \emph{cross-context \underline{R}eplay} strategy that maintains the intra-group advantage by recalling valuable examples from historical trajectories of the same query, (2) an \emph{in-context self-\underline{R}eflection} mechanism enabling models to refine outputs by leveraging past failures, and (3) a \emph{structural
entropy \underline{R}anking reward}, which assigns relative rewards to truncated or failed samples by ranking responses based on
token-level
entropy patterns, capturing both local exploration and global stability. We implement our method on Deepseek-R1-Distill-Qwen-1.5B and train it on the DeepscaleR-40k in the math domain. Experiments demonstrate our method achieves SoTA performance on several math
benchmarks, representing significant improvements and fewer
reasoning tokens over the base models. Code and model will be released.
Read the full abstract ▾
All authors Zhizheng Jiang, Kang Zhao, Weikai Xu, Xinkui Lin, Wei Liu, Jian Luan, Shuo Shang, Peng Han
Machine Learning · Cross-listed from cs.CV
The pretraining domain outweighs the training objective in setting the privacy-utility trade-off of differentially private medical image analysis
Soroosh Tayebi Arasteh, Mina Farajiamiri, Mahshad Lotfinia et al. (+7 more)
Differential
privacy limits how much each training image can influence a medical imaging model but lowers its diagnostic accuracy. Pretrained weights reduce this loss. It is unknown whether the pretraining objective or domain improves accuracy more. Here we train chest radiograph classifiers under differential
privacy from five starting points on 591,884 radiographs from five
datasets. We show that supervised pretraining on chest radiographs ranks first in 24 of 25 combinations of
dataset and
privacy setting. Measured by the area under the receiver operating characteristic curve, its lead over supervised pretraining on natural images is 2.6% without
privacy and 15.3% at the strictest budget. From that starting point, changing the domain improves this area 2.2 to 3.7 times more than changing the objective. Private pretraining on chest radiographs lowers this area by about 5% under
privacy but outperforms the three starting points without patient data at the strictest budget. Under
privacy, supervised chest radiograph pretraining leads in the worst-performing demographic subgroup.
Read the full abstract ▾
All authors Soroosh Tayebi Arasteh, Mina Farajiamiri, Mahshad Lotfinia, Behrus Hinrichs-Puladi, Jonas Bienzeisler, Mohamed Alhaskir, Mirabela Rusu, Christiane Kuhl, Sven Nebelung, Daniel Truhn
Machine Learning
Safe Exploration via Policy Priors
Manuel Wendl, Yarden As, Manish Prajapat et al. (+3 more)
Safe exploration is a key requirement for reinforcement learning (RL)
agents to learn and adapt online, beyond controlled (e.g. simulated) environments. In this work, we tackle this challenge by utilizing suboptimal yet conservative policies (e.g., obtained from offline data or simulators) as priors. Our approach, SOOPER, uses probabilistic dynamics models to optimistically explore, yet pessimistically fall back to the conservative policy prior if needed. We prove that SOOPER guarantees safety throughout learning, and establish convergence to an optimal policy by bounding its cumulative regret. Extensive experiments on key safe RL
benchmarks and real-world hardware demonstrate that SOOPER is scalable, outperforms the state-of-the-art and validate our theoretical guarantees in practice.
Read the full abstract ▾
All authors Manuel Wendl, Yarden As, Manish Prajapat, Anton Pollak, Stelian Coros, Andreas Krause
Machine Learning
Explicit Multi-head Attention for Inter-head Interaction in Large Language Models
Runyu Peng, Yunhua Zhou, Demin Song et al. (+4 more)
In large
language models built upon the
Transformer architecture, recent studies have shown that inter-head interaction can enhance
attention performance. Motivated by this, we propose Multi-head Explicit
Attention (MEA), a simple yet effective
attention variant that explicitly models cross-head interaction. MEA consists of two key components: a Head-level Linear Composition (HLC) module that separately applies learnable linear combinations to the key and value vectors across heads, thereby enabling rich inter-head communication; and a head-level Group Normalization layer that aligns the statistical properties of the recombined heads. MEA shows strong robustness in pretraining, which allows the use of larger learning rates that lead to faster convergence, ultimately resulting in lower validation loss and improved performance across a range of tasks. Furthermore, we explore the parameter efficiency of MEA by reducing the number of
attention heads and leveraging HLC to reconstruct them using low-rank "virtual heads". This enables a practical key-value cache compression strategy that reduces KV-cache
memory usage by 50% with negligible performance loss on knowledge-intensive and scientific
reasoning tasks, and only a 3.59% accuracy drop for Olympiad-level mathematical
benchmarks.
Read the full abstract ▾
All authors Runyu Peng, Yunhua Zhou, Demin Song, Kai Lv, Bo Wang, Qipeng Guo, Xipeng Qiu
Machine Learning · Cross-listed from cs.CV
GMS-CAVP: Improving Audio-Video Correspondence with Multi-Scale Contrastive and Generative Pretraining
Shentong Mo, Zehua Chen and Jun Zhu
Recent advances in
video-audio (V-A) understanding and generation have increasingly relied on joint V-A
embeddings, which serve as the foundation for tasks such as cross-modal
retrieval and generation. While prior methods like CAVP effectively model semantic and temporal correspondences between modalities using contrastive objectives, their performance remains suboptimal. A key limitation is the insufficient modeling of the dense, multi-scale nature of both
video and audio signals, correspondences often span fine- to coarse-grained spatial-temporal structures, which are underutilized in existing frameworks. To this end, we propose GMS-CAVP, a novel framework that combines Multi-Scale
Video-Audio
Alignment and Multi-Scale Spatial-Temporal
Diffusion-based pretraining objectives to enhance V-A correspondence modeling. First, GMS-CAVP introduces a multi-scale contrastive learning strategy that captures semantic and temporal relations across varying granularities. Second, we go beyond traditional contrastive learning by incorporating a
diffusion-based generative objective, enabling modality translation and synthesis between
video and audio. This unified discriminative-generative formulation facilitates deeper cross-modal understanding and paves the way for high-fidelity generation. Extensive experiments on VGGSound, AudioSet, and Panda70M demonstrate that GMS-CAVP outperforms previous methods in generation and
retrieval.
Read the full abstract ▾