Machine Learning
Bi-cLSTM: Residual-Corrected Bidirectional LSTM for Aero-Engine RUL Estimation
Rafi Hassan Chowdhury, Nabil Daiyan, Faria Ahmed et al. (+2 more)
Accurate Remaining Useful Life (RUL) prediction is a key requirement for effective Prognostics and Health Management (PHM) in safety-critical systems such as aero-engines. Existing deep learning approaches, particularly LSTM-based models, often struggle to generalize across varying operating conditions and are sensitive to noise in multivariate sensor data. To address these challenges, we propose a novel Bidirectional Residual Corrected LSTM (Bi-cLSTM) model for robust RUL estimation. The proposed architecture combines bidirectional temporal modeling with an adaptive residual correction mechanism to iteratively refine sequence representations. In addition, we introduce a condition-aware preprocessing pipeline incorporating regime-based normalization, feature selection, and exponential smoothing to improve robustness under complex operating environments. Extensive experiments on all four subsets of the NASA C-MAPSS
dataset demonstrate that the proposed Bi-cLSTM consistently outperforms LSTM-based baselines and achieves competitive state-of-the-art performance, particularly in challenging multi-condition scenarios. These results highlight the effectiveness of combining bidirectional temporal learning with residual correction for reliable RUL prediction.
Read the full abstract ▾
All authors Rafi Hassan Chowdhury, Nabil Daiyan, Faria Ahmed, Md Redwan Iqbal, Morsalin Sheikh
Machine Learning
ResGene-T: A Tensor-Based Residual Network Approach for Genomic Prediction
Kuldeep Pathak, Kapil Ahuja and Eric de Sturler
In this work, we propose a new deep learning model for Genomic Prediction (GP), which involves correlating genotypic data with phenotypic. The genotypes are typically fed as a sequence of characters to the 1D-Convolution Neural Network layer of the underlying deep learning model. Inspired by earlier work that represented genotype as a 2D-image for genotype-phenotype classification, we extend this idea to GP, which is a regression task. We use a ResNet-18 as the underlying architecture, and term this model as ResGene-2D. Although the 2D-image representation captures biological interactions well, it requires all the layers of the model to do so. This limits training efficiency. Thus, as seen in the earlier work that proposed a 2D-image representation, our ResGene-2D performs almost the same as other models (3% improvement). To overcome this, we propose a novel idea of converting the 2D-image into a 3D/ tensor and feed this to the ResNet-18 architecture, and term this model as ResGene-T. We evaluate our proposed models on three crop species having ten phenotypic traits and compare it with seven most popular models (two statistical, two machine learning, and three deep learning). ResGene-T performs the best among all these seven methods (gains from 14.51% to 41.51%).
Read the full abstract ▾
Machine Learning
To Use or not to Use Muon: How Simplicity Bias in Optimizers Matters
Sara Dragutinović, Yedi Zhang and Rajesh Ranganath
While Adam has long been the ubiquitous default optimizer for deep neural networks,
Muon has recently seen rapid adoption due to its superior training speed. Although much of the literature focuses on validating the benefits of
Muon, our work investigates the potential downsides of the mechanism driving this speedup. On the theoretical front, we analyze the learning dynamics of simplified
Muon on deep linear networks and linear
attention. Our analysis reveals that
Muon gains speed by avoiding saddle points, but does so at the expense of the simplicity bias characteristic of Gradient Descent (GD), where the complexity of the functional solution learned grows sequentially. Experiments demonstrate the consequences of losing the simplicity bias, showing that
Muon struggles to uncover common underlying structure across tasks and may be prone to fitting spurious features. More broadly, this paper serves as a reminder that faster optimization is rarely a free lunch; improvements in optimization can come at the cost of changes in the inductive biases that shape generalization.
Read the full abstract ▾
Machine Learning · Cross-listed from physics.med-ph
Data-driven Synthesis of Magnetic Resonance Spectroscopy Data using a Variational Autoencoder
Dennis M.J. van de Sande, Julian P. Merkofer, Sina Amirrajab et al. (+4 more)
The development of deep learning methods for
magnetic resonance spectroscopy (MRS) is often hindered by limited availability of large, high-quality training
datasets. While physics-based simulations are commonly used to mitigate this limitation, accurately modeling all in-vivo signal components remains challenging. In this work, we propose a data-driven framework for synthesizing in-vivo MRS data using a variational autoencoder (VAE) trained exclusively on measured single-voxel spectroscopy data. The model learns a low-dimensional latent representation of complex-valued spectra and enables generation of new samples through latent-space sampling and interpolation. The generative performance of the proposed approach is evaluated using a comprehensive set of complementary analyses, including reconstruction quality, feature-level similarity using low-dimensional
embeddings, application-based signal quality metrics, and metabolite quantification agreement. The results demonstrate that the VAE accurately reconstructs dominant spectral patterns and generates synthetic spectra that occupy the same feature space as in-vivo data. In an example application targeting GABA-edited spectroscopy, augmenting limited subsets of transients with synthetic spectra improves signal quality metrics such as signal-to-noise ratio, linewidth, and shape scores. However, the results also reveal limitations of the generative approach, including under-representation of
stochastic noise and reduced accuracy in absolute metabolite quantification, particularly for applications sensitive to concentration estimates. These findings highlight both potential and limitations of data-driven MRS synthesis. Beyond the proposed model, this study introduces a structured evaluation framework for generative MRS methods, emphasizing the importance of application-aware validation when synthetic data are used for downstream analysis.
Read the full abstract ▾
All authors Dennis M.J. van de Sande, Julian P. Merkofer, Sina Amirrajab, Mitko Veta, Gerhard S. Drenthen, Jacobus F.A. Jansen, Marcel Breeuwer
Machine Learning · Cross-listed from cs.AI
MO-MIX: Multi-Objective Multi-Agent Cooperative Decision-Making With Deep Reinforcement Learning
Tianmeng Hu, Biao Luo, Chunhua Yang et al. (+1 more)
Deep reinforcement learning (RL) has been applied extensively to solve complex decision-making problems. In many real-world scenarios, tasks often have several conflicting objectives and may require multiple
agents to cooperate, which are the multi-objective multi-
agent decision-making problems. However, only few works have been conducted on this intersection. Existing approaches are limited to separate fields and can only handle multi-
agent decision-making with a single objective, or multi-objective decision-making with a single
agent. In this paper, we propose MO-MIX to solve the multi-objective multi-
agent reinforcement learning (MOMARL) problem. Our approach is based on the centralized training with decentralized execution (CTDE) framework. A weight vector representing preference over the objectives is fed into the decentralized
agent network as a condition for local action-value function estimation, while a mixing network with parallel architecture is used to estimate the joint action-value function. In addition, an exploration guide approach is applied to improve the uniformity of the final non-dominated solutions. Experiments demonstrate that the proposed method can effectively solve the multi-objective multi-
agent cooperative decision-making problem and generate an approximation of the Pareto set. Our approach not only significantly outperforms the baseline method in all four kinds of evaluation metrics, but also requires less computational cost.
Read the full abstract ▾
All authors Tianmeng Hu, Biao Luo, Chunhua Yang, Tingwen Huang
Machine Learning
MARS: Harmonizing Multimodal Convergence via Adaptive Rank Search
Minkyoung Cho, Insu Jang, Shuowei Jin et al. (+5 more)
Fine-tuning
Multimodal Large
Language Models (MLLMs) with parameter-efficient methods like Low-Rank Adaptation (LoRA) is crucial for task adaptation. However, imbalanced training dynamics across modalities often lead to suboptimal accuracy due to negative interference, a challenge typically addressed with inefficient heuristic methods such as manually tuning separate learning rates. To overcome this, we introduce MARS (
Multimodal Adaptive Rank Search), an approach to discover optimal rank pairs that balance training dynamics while maximizing performance. Our key innovation, a proposed framework of dual scaling laws, enables this search: one law models module-specific convergence time to prune the search space to candidates with aligned dynamics, while the other predicts final task performance to select the optimal pair from the pruned set. By re-purposing the LoRA rank as a controller for modality-specific convergence speed, MARS outperforms baseline methods and provides a robust, automated strategy for optimizing MLLM fine-tuning.
Read the full abstract ▾
All authors Minkyoung Cho, Insu Jang, Shuowei Jin, Zesen Zhao, Adityan Jothi, Ethem F. Can, Min-Hung Chen, Z. Morley Mao
Machine Learning
Frozen Policy Iteration: Computationally Efficient RL under Linear $Q^{\pi}$ Realizability for Deterministic Dynamics
Yijing Ke, Zihan Zhang and Ruosong Wang
We study computationally and statistically efficient reinforcement learning under the linear $Q^{\pi}$ realizability assumption, where any policy's $Q$-function is linear in a given state-action feature representation. Prior methods in this setting are either computationally intractable, or require (local) access to a simulator. In this paper, we propose a computationally efficient online RL
algorithm, named Frozen Policy Iteration, under the linear $Q^{\pi}$ realizability setting that works for Markov Decision Processes (MDPs) with
stochastic initial states,
stochastic rewards and deterministic transitions. Our
algorithm achieves a regret bound of $\widetilde{O}(\sqrt{d^2H^6T})$, where $d$ is the dimensionality of the feature space, $H$ is the horizon length, and $T$ is the total number of episodes. Our regret bound is optimal for linear (contextual) bandits which is a special case of our setting with $H = 1$. Existing policy iteration
algorithms under the same setting heavily rely on repeatedly sampling the same state by access to the simulator, which is not implementable in the online setting with
stochastic initial states studied in this paper. In contrast, our new
algorithm circumvents this limitation by strategically using only high-confidence part of the trajectory data and freezing the policy for well-explored states, which ensures that all data used by our
algorithm remains effectively on-policy during the whole course of learning. We further demonstrate the versatility of our approach by extending it to the Uniform-PAC setting and to function classes with bounded eluder dimension.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CR
IU: Imperceptible Universal Backdoor Attack
Hsin Lin, Yan-Lun Chen, Ren-Hung Hwang et al. (+1 more)
Backdoor attacks pose a critical threat to the security of deep neural networks, yet existing efforts on universal
backdoors often rely on visually salient patterns, making them easier to detect and less practical at scale. In this work, we introduce a novel imperceptible universal
backdoor attack that simultaneously controls all target classes with minimal poisoning while preserving stealth. Our key idea is to leverage graph convolutional networks (GCNs) to model inter-class relationships and generate class-specific perturbations that are both effective and visually invisible. The proposed framework optimizes a dual-objective loss that balances stealthiness (measured by perceptual similarity metrics such as PSNR) and attack success rate (ASR), enabling scalable, multi-target
backdoor injection. Extensive experiments on ImageNet-1K with ResNet architectures demonstrate that our method achieves high ASR (up to 91.3%) under poisoning rates as low as 0.16%, while maintaining benign accuracy and evading state-of-the-art defenses. These results highlight the emerging risks of invisible universal
backdoors and call for more robust detection and mitigation strategies.
Read the full abstract ▾
All authors Hsin Lin, Yan-Lun Chen, Ren-Hung Hwang, Chia-Mu Yu
Machine Learning
Reward-Modulated Local Learning in Spiking Encoders: Controlled Benchmarks with STDP and Hybrid Rate Readouts
Debjyoti Chakraborty
This paper presents a controlled empirical study of biologically motivated local learning for handwritten digit recognition. We evaluate an STDP-inspired competitive proxy and a practical hybrid
benchmark built on the same spiking population encoder. The proxy is motivated by leaky integrate-and-fire E/I
circuit models with three-factor delayed reward modulation. The hybrid update is local in pre x post rates but uses supervised labels and no timing-based credit assignment. On sklearn digits, fixed-seed evaluation shows classical pixel baselines from 98.06 to 98.22% accuracy, while local spike-based models reach 86.39 +/- 4.75% (hybrid default) and 87.17 +/- 3.74% (STDP-style competitive proxy). Ablations identify normalization and reward-shaping settings as the strongest observed levers, with a best hybrid ablation of 95.52 +/- 1.11%. A network-free synthetic temporal
benchmark supports the same timing-versus-rate interpretation under matched local-update training. A descriptive 2x2 analysis further shows reward-shaping effects can reverse sign across stabilization regimes, so reward-shaping conclusions should be reported jointly with normalization settings.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Polynomial Mixing for Efficient Self-supervised Speech Encoders
Eva Feillet, Ryan Whetten, David Picard et al. (+1 more)
State-of-the-art
speech-to-text models typically employ
Transformer-based encoders that model
token dependencies via self-
attention mechanisms. However, the quadratic complexity of self-
attention in both
memory and computation imposes significant constraints on scalability. In this work, we propose a novel
token-mixing mechanism, the
Polynomial Mixer (PoM), as a drop-in replacement for multi-head self-
attention. PoM computes a
polynomial representation of the input with linear complexity with respect to the input sequence length. We integrate PoM into a self-supervised
speech representation learning framework based on BEST-RQ and evaluate its performance on downstream
speech recognition tasks. Experimental results demonstrate that PoM achieves a competitive word error rate compared to full self-
attention and other linear-complexity alternatives, offering an improved trade-off between performance and efficiency in time and
memory.
Read the full abstract ▾
All authors Eva Feillet, Ryan Whetten, David Picard, Alexandre Allauzen
Machine Learning · Cross-listed from cs.CV
Xiaohan Zhao, Xinyi Shang, Jiacheng Liu et al. (+1 more)
Dataset pruning has been widely studied for 2D images to remove redundancy and accelerate training, while particular
pruning methods for 3D data remain largely unexplored. In this work, we study
dataset pruning for 3D data, where its observed common long-tail class distribution nature make optimization under conventional evaluation metrics Overall Accuracy (OA) and Mean Accuracy (mAcc) inherently conflicting, and further make
pruning particularly challenging. To address this, we formulate
pruning as approximating the full-data expected risk with a weighted subset, which reveals two key errors: coverage error from insufficient representativeness and prior-mismatch bias from inconsistency between subset-induced class weights and target metrics. We propose representation-aware subset selection with per-class retention quotas for long-tail coverage, and prior-invariant teacher supervision using calibrated soft labels and
embedding-geometry
distillation. The retention quota also serves as a switch to control the OA-mAcc trade-off. Extensive experiments on 3D
datasets show that our method can improve both metrics across multiple settings while adapting to different downstream preferences. Our code is available at https://github.com/XiaohanZhao123/3D-
Dataset-
Pruning.
Read the full abstract ▾
All authors Xiaohan Zhao, Xinyi Shang, Jiacheng Liu, Zhiqiang Shen
Machine Learning · Cross-listed from cs.IR
RAIE: Region-Aware Incremental Preference Editing with LoRA for LLM-based Recommendation
Jin Zeng, Yupeng Qi, Hui Li et al. (+4 more)
Large
language models (LLMs) are increasingly adopted as the backbone of recommender systems. However, user-item interactions in real-world scenarios are non-stationary, making preference drift over time inevitable. Existing model update strategies mainly rely on global fine-tuning or pointwise editing, but they face two fundamental challenges: (i) imbalanced update granularity, where global updates perturb behaviors unrelated to the target while pointwise edits fail to capture broader preference shifts; (ii) unstable incremental updates, where repeated edits interfere with prior adaptations, leading to catastrophic forgetting and inconsistent recommendations. To address these issues, we propose Region-Aware Incremental Editing (RAIE), a plug-in framework that freezes the backbone model and performs region-level updates. RAIE first constructs semantically coherent preference regions via spherical k-means in the representation space. It then assigns incoming sequences to regions via confidence-aware gating and performs three localized edit operations - Update, Expand, and Add - to dynamically revise the affected region. Each region is equipped with a dedicated Low-Rank Adaptation (LoRA) module, which is trained only on the region's updated data. During inference, RAIE routes each user sequence to its corresponding region and activates the region-specific adapter for prediction. Experiments on two
benchmark datasets under a time-sliced protocol that segments data into Set-up (S), Finetune (F), and Test (T) show that RAIE significantly outperforms state-of-the-art baselines while effectively mitigating forgetting. These results demonstrate that region-aware editing offers an accurate and scalable mechanism for continual adaptation in dynamic recommendation scenarios. Our code is available at https://github.com/fengaogao/RAIE.
Read the full abstract ▾
All authors Jin Zeng, Yupeng Qi, Hui Li, Chengming Li, Ziyu Lyu, Lixin Cui, Lu Bai
Machine Learning
Retrodictive Forecasting: A Proof-of-Concept for Exploiting Temporal Asymmetry in Time Series Prediction
Cedric Damour
We propose a retrodictive
forecasting paradigm for time series: instead of predicting the future from the past, we identify the future that best explains the observed present via inverse MAP optimization over a Conditional Variational Autoencoder (CVAE). This conditioning is a statistical modeling choice for
Bayesian inversion; it does not assert that future events cause past observations. The approach is theoretically grounded in an information-theoretic arrow-of-time measure: the symmetrized Kullback-Leibler divergence between forward and time-reversed trajectory ensembles provides both the conceptual rationale and an operational GO/NO-GO diagnostic for applicability. We implement the paradigm as MAP inference over an inverse CVAE with a learned RealNVP normalizing-flow prior and evaluate it on six time series cases: four synthetic processes with controlled temporal asymmetry and two ERA5 reanalysis
datasets (wind speed and
solar irradiance). The work makes four contributions: (i) a formal retrodictive inference formulation; (ii) an inverse CVAE architecture; (iii) a model-free irreversibility diagnostic; and (iv) a falsifiable validation protocol with four pre-specified predictions. All pre-specified predictions are empirically supported: the diagnostic correctly classifies all six cases; the learned flow prior improves over an isotropic Gaussian baseline on GO cases; the inverse MAP yields no spurious advantage on time-reversible dynamics; and on irreversible GO cases, it achieves competitive or superior RMSE relative to forward baselines, with a statistically significant 17.7% reduction over a forward MLP on ERA5
solar irradiance. These results provide a structured proof-of-concept that retrodictive
forecasting can constitute a viable alternative to conventional forward prediction when statistical time-irreversibility is present and exploitable.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.IR
Stop Treating Collisions Equally: Qualification-Aware Semantic ID Learning for Recommendation at Industrial Scale
Zheng Hu, Yuxin Chen, Yongsen Pan et al. (+13 more)
Semantic IDs (SIDs) are compact discrete representations derived from
multimodal item features, serving as a unified abstraction for ID-based and generative recommendation. However, learning high-quality SIDs remains challenging due to two issues. (1) Collision problem: the quantized
token space is prone to collisions, in which semantically distinct items are assigned identical or overly similar SID compositions, resulting in semantic
entanglement. (2) Collision-signal heterogeneity: collisions are not uniformly harmful. Some reflect genuine conflicts between semantically unrelated items, while others stem from benign redundancy or systematic data effects. To address these challenges, we propose Qualification-Aware Semantic ID Learning (QuaSID), an end-to-end framework that learns collision-qualified SIDs by selectively repelling qualified conflict pairs and scaling the repulsion strength by collision severity. QuaSID consists of two mechanisms: Hamming-guided Margin Repulsion, which translates low-Hamming SID overlaps into explicit, severity-scaled
geometric constraints on the encoder space; and Conflict-Aware Valid Pair Masking, which masks protocol-induced benign overlaps to denoise repulsion supervision. In addition, QuaSID incorporates a dual-tower contrastive objective to inject collaborative signals into tokenization. Experiments on public
benchmarks and industrial data validate QuaSID. On public
datasets, QuaSID consistently outperforms strong baselines, improving top-K ranking quality by 5.9% over the best baseline while increasing SID composition diversity. In an online A/B test on Kuaishou e-commerce with a 5% traffic split, QuaSID increases ranking GMV-S2 by 2.38% and improves completed orders on cold-start
retrieval by up to 6.42%. Finally, we show that the proposed repulsion loss is plug-and-play and enhances a range of SID learning frameworks across
datasets.
Read the full abstract ▾
All authors Zheng Hu, Yuxin Chen, Yongsen Pan, Xu Yuan, Yuting Yin, Daoyuan Wang, Boyang Xia, Zefei Luo, Hongyang Wang, Songhao Ni, Dongxu Liang, Jun Wang, Shimin Cai, Tao Zhou, Fuji Ren, Wenwu Ou
Machine Learning
Adapt Data to Model: Adaptive Transformation Optimization for Domain-shared Time Series Foundation Models
Yunzhong Qiu, Zhiyao Cen, Zhongyi Pei et al. (+2 more)
Large time series models (LTMs) have emerged as powerful tools for universal
forecasting, yet they often struggle with the inherent diversity and nonstationarity of real-world time series data, leading to an unsatisfactory trade-off between
forecasting accuracy and generalization. Rather than continually finetuning new LTM instances for each domain, we propose a data-centric framework, time-series adaptive transformation optimization (TATO), that enables a single frozen pre-trained LTM to adapt to diverse downstream domains through an optimally configured transformation pipeline. Specifically, TATO constructs three representative types of transformations, including context slicing, scale normalization, and outlier correction, to help LTMs better align with target domain characteristics. To ensure robustness, we incorporate carefully selected time series augmentations and a two-stage ranking mechanism that filters out pipelines underperforming on specific metrics. Extensive experiments on state-of-the-art LTMs and widely used
datasets demonstrate that TATO consistently and significantly improves domain-adaptive
forecasting performance, achieving a maximum reduction in MSE of 65.4% and an average reduction of 13.6%. Moreover, TATO is highly efficient, typically completing optimization in under 2 minutes, making it practical for real-world deployment. The source code is available at https://github.com/thulab/TATO.
Read the full abstract ▾
All authors Yunzhong Qiu, Zhiyao Cen, Zhongyi Pei, Chen Wang, Jianmin Wang
Machine Learning
IDER: IDempotent Experience Replay for Reliable Continual Learning
Zhanwang Liu, Yuting Li, Haoyuan Gao et al. (+4 more)
Catastrophic forgetting, the tendency of neural networks to forget previously learned knowledge when learning new tasks, has been a major challenge in continual learning (CL). To tackle this challenge, CL methods have been proposed and shown to reduce forgetting. Furthermore, CL models deployed in mission-critical settings can benefit from uncertainty awareness by calibrating their predictions to reliably assess their confidences. However, existing uncertainty-aware continual learning methods suffer from high computational overhead and incompatibility with mainstream replay methods. To address this, we propose idempotent experience replay (IDER), a novel approach based on the idempotent property where repeated function applications yield the same output. Specifically, we first adapt the training loss to make model idempotent on current data streams. In addition, we introduce an idempotence
distillation loss. We feed the output of the current model back into the old checkpoint and then minimize the distance between this reprocessed output and the original output of the current model. This yields a simple and effective new baseline for building reliable continual learners, which can be seamlessly integrated with other CL approaches. Extensive experiments on different CL
benchmarks demonstrate that IDER consistently improves prediction reliability while simultaneously boosting accuracy and reducing forgetting. Our results suggest the potential of idempotence as a promising principle for deploying efficient and trustworthy continual learning systems in real-world applications.Our code is available at https://github.com/YutingLi0606/Idempotent-Continual-Learning.
Read the full abstract ▾
All authors Zhanwang Liu, Yuting Li, Haoyuan Gao, Yexin Li, Linghe Kong, Lichao Sun, Weiran Huang
Machine Learning
Multi-Domain Riemannian Graph Gluing for Building Graph Foundation Models
Li Sun, Zhenhao Huang, Silei Chen et al. (+4 more)
Multi-domain graph pre-training integrates knowledge from diverse domains to enhance performance in the target domains, which is crucial for building graph foundation models. Despite initial success, existing solutions often fall short of answering a fundamental question: how is knowledge integrated or transferred across domains? This theoretical limitation motivates us to rethink the consistency and transferability between model pre-training and domain adaptation. In this paper, we propose a fresh Riemannian geometry perspective, whose core idea is to merge any graph
dataset into a unified, smooth Riemannian
manifold, enabling a systematic understanding of knowledge integration and transfer. To achieve this, our key contribution is the theoretical establishment of neural
manifold gluing, which first characterizes local geometry using an adaptive orthogonal frame and then "glues" the local pieces together into a coherent whole. Building on this theory, we present the GraphGlue framework, which supports batched pre-training with EMA prototyping and provides a transferability measure based on
geometric consistence. Extensive experiments demonstrate its superior performance across diverse graph domains. Moreover, we empirically validated GraphGlue's
geometric scaling law, showing that larger quantities of
datasets improve model transferability by producing a smoother
manifold. Codes are available at https://github.com/RiemannGraph/GraphGlue.
Read the full abstract ▾
All authors Li Sun, Zhenhao Huang, Silei Chen, Lanxu Yang, Junda Ye, Sen Su, Philip S. Yu
Machine Learning · Cross-listed from cs.SD
CMI-RewardBench: Evaluating Music Reward Models with Compositional Multimodal Instruction
Yinghao Ma, Haiwen Xia, Hewei Gao et al. (+9 more)
While music generation models have evolved to handle complex
multimodal inputs mixing text, lyrics, and reference audio, evaluation mechanisms have lagged behind. In this paper, we bridge this critical gap by establishing a comprehensive ecosystem for music reward modeling under Compositional
Multimodal Instruction (CMI), where the generated music may be conditioned on text descriptions, lyrics, and audio
prompts. We first introduce CMI-Pref-Pseudo, a large-scale preference
dataset comprising 110k pseudo-labeled samples, and CMI-Pref, a high-quality, human-annotated corpus tailored for fine-grained
alignment tasks. To unify the evaluation landscape, we propose CMI-RewardBench, a unified
benchmark that evaluates music reward models on heterogeneous samples across musicality, text-music
alignment, and compositional instruction
alignment. Leveraging these resources, we develop CMI reward models (CMI-RMs), a parameter-efficient reward model family capable of processing heterogeneous inputs. We evaluate their correlation with human judgment scores on musicality and
alignment on CMI-Pref along with previous
datasets. Further experiments demonstrate that CMI-RM not only correlates strongly with human judgments, but also enables effective inference-time scaling via top-k filtering. Code is available at GitHub (https://github.com/Haiwen-Xia/CMI-RewardBench). Model weights: CMI-RM (https://huggingface.co/HaiwenXia/CMI-RM).
Datasets: CMI-Pref-Pseudo (https://huggingface.co/
datasets/HaiwenXia/cmi-pref-pseudo) and CMI-Pref (https://huggingface.co/
datasets/HaiwenXia/cmi-pref)
Read the full abstract ▾
All authors Yinghao Ma, Haiwen Xia, Hewei Gao, Weixiong Chen, Yuxin Ye, Yuchen Yang, Sungkyun Chang, Mingshuo Ding, Yizhi Li, Ruibin Yuan, Simon Dixon, Emmanouil Benetos
Machine Learning
Learning to Explore: Policy-Guided Outlier Synthesis for Graph Out-of-Distribution Detection
Li Sun, Lanxu Yang, Jiayu Tian et al. (+6 more)
Detecting out-of-distribution (OOD) graphs is crucial for ensuring the safety and reliability of Graph Neural Networks. In unsupervised graph-level OOD detection, models are typically trained using only in-distribution (ID) data, resulting in incomplete feature space characterization and weak decision boundaries. Although synthesizing outliers offers a promising solution, existing approaches rely on fixed, non-adaptive sampling heuristics (e.g., distance- or density-based), limiting their ability to explore informative OOD regions. We propose a Policy-Guided Outlier Synthesis (PGOS) framework that replaces static heuristics with a learned exploration strategy. Specifically, PGOS trains a reinforcement learning
agent to navigate low-density regions in a structured latent space and sample representations that most effectively refine the OOD decision boundary. These representations are then decoded into high-quality pseudo-OOD graphs to improve detector robustness. Extensive experiments demonstrate that PGOS achieves state-of-the-art performance on multiple graph OOD and anomaly detection
benchmarks.
Read the full abstract ▾
All authors Li Sun, Lanxu Yang, Jiayu Tian, Bowen Fang, Xiaoyan Yu, Junda Ye, Peng Tang, Hao Peng, Philip S. Yu
Machine Learning · Cross-listed from cs.AI
Heterophily-Agnostic Hypergraph Neural Networks with Riemannian Local Exchanger
Li Sun, Ming Zhang, Wenxin Jin et al. (+5 more)
Hypergraphs are the natural description of higher-order interactions among objects, widely applied in social network analysis, cross-modal
retrieval, etc. Hypergraph Neural Networks (HGNNs) have become the dominant solution for learning on hypergraphs. Traditional HGNNs are extended from message passing graph neural networks, following the homophily assumption, and thus struggle with the prevalent heterophilic hypergraphs that call for long-range dependence modeling. In this paper, we achieve heterophily-agnostic message passing through the lens of Riemannian geometry. The key insight lies in the connection between oversquashing and hypergraph bottleneck within the framework of Riemannian
manifold heat flow. Building on this, we propose the novel idea of locally adapting the bottlenecks of different subhypergraphs. The core innovation of the proposed mechanism is the design of an adaptive local (heat) exchanger. Specifically, it captures the rich long-range dependencies via the Robin condition, and preserves the representation distinguishability via source terms, thereby enabling heterophily-agnostic message passing with theoretical guarantees. Based on this theoretical foundation, we present a novel Heat-Exchanger with Adaptive Locality for Hypergraph Neural Network (HealHGNN), designed as a node-hyperedge bidirectional systems with linear complexity in the number of nodes and hyperedges. Extensive experiments on both homophilic and heterophilic cases show that HealHGNN achieves the state-of-the-art performance.
Read the full abstract ▾
All authors Li Sun, Ming Zhang, Wenxin Jin, Zhongtian Sun, Zhenhao Huang, Hao Peng, Sen Su, Philip Yu
Machine Learning · Cross-listed from cs.RO
LangGap: Diagnosing and Closing the Language Gap in Vision-Language-Action Models
Yuchen Hou and Lin Zhao
Vision-
Language-Action (VLA) models achieve over 95% success on standard
benchmarks. However, through systematic experiments, we find that current state-of-the-art VLA models largely ignore
language instructions. Prior work lacks: (1) systematic semantic perturbation diagnostics, (2) a
benchmark that forces
language understanding by design, and (3) linguistically diverse training data. This paper constructs the LangGap
benchmark, based on a four-dimensional semantic perturbation method – varying instruction semantics while keeping the tabletop layout fixed – revealing
language understanding deficits in {\pi}0.5. Existing
benchmarks like LIBERO assign only one task per layout, underutilizing available objects and target locations; LangGap fully diversifies pick-and-place tasks under identical layouts, forcing models to truly understand
language. Experiments show that targeted data augmentation can partially close the
language gap – success rate improves from 0% to 90% with single-task training, and 0% to 28% with multi-task training. However, as semantic diversity of extended tasks increases, model learning capacity proves severely insufficient; even trained tasks perform poorly. This reveals a fundamental challenge for VLA models in understanding diverse
language instructions – precisely the long-term value of LangGap.
Read the full abstract ▾
Machine Learning
Energy-Efficient Information Representation in MNIST Classification Using Biologically Inspired Learning
Patrick Stricker, Florian Röhrbein and Andreas Knoblauch
Efficient representation learning is essential for optimal information storage and classification. However, it is frequently overlooked in artificial neural networks (ANNs). This neglect results in networks that can become overparameterized by factors of up to 13, increasing redundancy and energy consumption. As the demand for large
language models (LLMs) and their scale increase, these issues are further highlighted, raising significant ethical and environmental concerns. We analyze our previously developed biologically inspired learning rule using information-theoretic concepts, evaluating its efficiency on the MNIST classification task. The proposed rule, which emulates the
brain's structural plasticity, naturally prevents overparameterization by optimizing synaptic usage and retaining only the essential number of synapses. Furthermore, it outperforms backpropagation (BP) in terms of efficiency and storage capacity. It also eliminates the need for pre-optimization of network architecture, enhances adaptability, and reflects the
brain's ability to reserve 'space' for new
memories. This approach advances scalable and energy-efficient AI and provides a promising framework for developing
brain-inspired models that optimize resource allocation and adaptability.
Read the full abstract ▾
Machine Learning
Unlearning Evaluation through Subset Statistical Independence
Chenhao Zhang, Muxing Li, Feng Liu et al. (+2 more)
Evaluating machine unlearning remains challenging, as existing methods typically require retraining reference models or performing membership inference attacks, both of which rely on prior access to training configuration or supervision labels, making them impractical in realistic scenarios. Motivated by the fact that most unlearning
algorithms remove a small, random subset of the training data, we propose a subset-level evaluation framework based on statistical independence. Specifically, we design a tailored use of the
Hilbert-Schmidt Independence Criterion to assess whether the model outputs on a given subset exhibit statistical dependence, without requiring model retraining or auxiliary classifiers. Our method provides a simple, standalone evaluation procedure that aligns with unlearning workflows. Extensive experiments demonstrate that our approach reliably distinguishes in-training from out-of-training subsets and clearly differentiates unlearning effectiveness, even when existing evaluations fall short.
Read the full abstract ▾
All authors Chenhao Zhang, Muxing Li, Feng Liu, Weitong Chen, Miao Xu
Machine Learning
DeepAFL: Deep Analytic Federated Learning
Jianheng Tang, Yajiang Huang, Kejia Fan et al. (+8 more)
Federated Learning (FL) is a popular distributed learning paradigm to break down data silo. Traditional FL approaches largely rely on gradient-based updates, facing significant issues about heterogeneity, scalability, convergence, and overhead, etc. Recently, some analytic-learning-based work has attempted to handle these issues by eliminating gradient-based updates via analytical (i.e., closed-form) solutions. Despite achieving superior invariance to data heterogeneity, these approaches are fundamentally limited by their single-layer linear model with a frozen pre-trained backbone. As a result, they can only achieve suboptimal performance due to their lack of representation learning capabilities. In this paper, to enable representable analytic models while preserving the ideal invariance to data heterogeneity for FL, we propose our Deep Analytic
Federated Learning approach, named DeepAFL. Drawing inspiration from the great success of ResNet in gradient-based learning, we design gradient-free residual blocks in our DeepAFL with analytical solutions. We introduce an efficient layer-wise protocol for training our deep analytic models layer by layer in FL through least squares. Both theoretical analyses and empirical evaluations validate our DeepAFL's superior performance with its dual advantages in heterogeneity invariance and representation learning, outperforming state-of-the-art baselines by up to 5.68%-8.42% across three
benchmark datasets.
Read the full abstract ▾
All authors Jianheng Tang, Yajiang Huang, Kejia Fan, Feijiang Han, Jiaxu Li, Jinfeng Xu, Run He, Anfeng Liu, Houbing Herbert Song, Huiping Zhuang, Yunhuai Liu
Machine Learning · Cross-listed from cs.SE
TopoEdge: Topology-Grounded Agentic Framework for Edge Networking Code Generation and Repair
Haomin Qi, Bohan Liu, Zihan Dai et al. (+1 more)
TopoEdge is a
topology-grounded, edge-deployable framework for end-to-end software-defined networking (SDN) configuration generation and repair, motivated by the brittleness of configuration artefacts under
topology variation and by strict operational constraints on latency,
privacy, and on-site execution. TopoEdge represents each target
topology as a router-level graph and embeds it using a contrastively trained graph neural network (GNN), enabling nearest-neighbour
retrieval of a verified reference configuration paired with an executable Python driver (a Topotest/pytest test script that orchestrates the emulated network and checks protocol assertions). The target
topology, retrieved reference
topology, and reference driver are assembled into a
topology-grounded
retrieval-augmented generation context (TopoRAG), which grounds a distributed, execution-centric generate–verify–repair loop coordinated by a central controller and realised by three role-specialised
agents: (i) a Planning
agent that produces a
topology-consistent configuration plan and a per-device skeleton; (ii) a Generation
agent that materialises executable configuration artefacts, including device configurations and the driver; and (iii) a Verification
agent that runs the FRRouting Topotest/pytest harness, compresses failures into a compact trace, and emits localised patch directives for iterative repair.
Read the full abstract ▾
All authors Haomin Qi, Bohan Liu, Zihan Dai, Yunkai Gao
Machine Learning
Enhancing Molecular Property Predictions by Learning from Bond Modelling and Interactions
Yunqing Liu, Yi Zhou and Wenqi Fan
Molecule representation learning is crucial for understanding and predicting molecular properties. However, conventional atom-centric models, which treat chemical bonds merely as pairwise interactions, often overlook complex bond-level phenomena like resonance and stereoselectivity. This oversight limits their predictive accuracy for nuanced chemical behaviors. To address this limitation, we introduce DeMol, a dual-graph framework whose architecture is motivated by a rigorous information-theoretic analysis demonstrating the information gain from a bond-centric perspective. DeMol explicitly models molecules through parallel atom-centric and bond-centric channels. These are synergistically fused by multi-scale Double-Helix Blocks designed to learn intricate atom-atom, atom-bond, and bond-bond interactions. The framework's
geometric consistency is further enhanced by a regularization term based on covalent radii to enforce chemically plausible structures. Comprehensive evaluations on diverse
benchmarks, including PCQM4Mv2, OC20 IS2RE, QM9, and MoleculeNet, show that DeMol establishes a new state-of-the-art, outperforming existing methods. These results confirm the superiority of explicitly modelling bond information and interactions, paving the way for more robust and accurate molecular machine learning.
Read the full abstract ▾
Machine Learning
Learning to Attack: A Bandit Approach to Adversarial Context Poisoning
Ray Telikani and Amir H. Gandomi
Neural contextual bandits are vulnerable to
adversarial attacks, where subtle perturbations to rewards, actions, or contexts induce suboptimal decisions. We introduce AdvBandit, a black-box adaptive attack that formulates context poisoning as a continuous-armed bandit problem, enabling the attacker to jointly learn and exploit the victim's evolving policy. The attacker requires no access to the victim's internal parameters, reward function, or gradient information; instead, it constructs a surrogate model using a maximum-
entropy inverse reinforcement learning module from observed context-action pairs and optimizes perturbations against this surrogate using projected gradient descent. An upper confidence bound-aware Gaussian process guides arm selection. An attack-budget control mechanism is also introduced to limit detection risk and overhead. We provide theoretical guarantees, including sublinear attacker regret and lower bounds on victim regret linear in the number of attacks. Experiments on three real-world
datasets (Yelp, MovieLens, and Disin) against various victim contextual bandits demonstrate that our attack model achieves higher cumulative victim regret than state-of-the-art baselines.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.PF
GCL-Sampler: Discovering Kernel Similarity for Sampled GPU Simulation via Graph Contrastive Learning
Jiaqi Wang, Jingwei Sun, Jiyu Luo et al. (+2 more)
GPU architectural simulation is orders of magnitude slower than native execution, necessitating workload sampling for practical speedups. Existing methods rely on hand-crafted features with limited expressiveness, yielding either aggressive sampling with high errors or conservative sampling with constrained speedups. To address these issues, we propose GCL-Sampler, a sampling framework that leverages Relational Graph Convolutional Networks with contrastive learning to automatically discover high-dimensional kernel similarities from trace graphs. By encoding instruction sequences and data dependencies into graph
embeddings, GCL-Sampler captures rich structural and semantic properties of program execution, enabling both high fidelity and substantial speedup. Evaluations on extensive
benchmarks show that GCL-Sampler achieves 258.94x average speedup against full workload with 0.37% error, outperforming state-of-the-art methods, PKA (129.23x, 20.90%), Sieve (94.90x, 4.10%) and STEM+ROOT (56.57x, 0.38%).
Read the full abstract ▾
All authors Jiaqi Wang, Jingwei Sun, Jiyu Luo, Han Li, Guangzhong Sun
Machine Learning
Spectral Condition for $\mu$P under Width-Depth Scaling
Chenyu Zheng, Rongzhen Wang, Xinyu Zhang et al. (+1 more)
Generative foundation models are increasingly scaled in both width and depth, posing significant challenges for stable feature learning and reliable hyperparameter (HP) transfer across model sizes. While maximal update parameterization~($\mu$P) has provided a principled solution to both problems for width scaling, existing extensions to the joint width-depth scaling regime remain fragmented, architecture- and optimizer-specific, and often rely on technically involved theories. In this work, we develop a simple and unified spectral framework for $\mu$P under joint width-depth scaling. For deep residual networks whose residual blocks contain $k$ transformations, the framework specifies how the norms of weights and their per-step updates should scale with width and depth. It reveals a fundamental transition from $k=1$ to $k\geq 2$, {unifying previously disparate $\mu$P formulations and identifying the CompleteP-style parameterization ($k\geq 2$)} as more appropriate for practical architectures with multi-transformation branches such as
Transformers. Building on this framework, we derive a general recipe for implementing $\mu$P across a broad class of optimizers by mapping spectral constraints to {concrete, optimizer-specific HP parameterizations}, recovering existing results and extending them to additional optimizers. {Finally, experiments on GPT-2 style
language models show that the CompleteP-style parameterization ($k\geq 2$) achieves stable feature learning and robust HP transfer under width-depth scaling, whereas standard parameterization and the Depth-$\mu$P-style parameterization ($k=1$) often fail to do so.} These results support the practical effectiveness of the proposed spectral framework.
Read the full abstract ▾
All authors Chenyu Zheng, Rongzhen Wang, Xinyu Zhang, Chongxuan Li
Machine Learning
Mathematical Foundations of Poisoning Attacks on Linear Regression over Cumulative Distribution Functions
Atsuki Sato, Martin Aumüller and Yusuke Matsui
Learned indexes are a class of index data structures that enable fast search by approximating the cumulative distribution function (CDF) using machine learning models (Kraska et al., SIGMOD'18). However, recent studies have shown that learned indexes are vulnerable to poisoning attacks, where injecting a small number of poison keys into the training data can significantly degrade model accuracy and reduce index performance (Kornaropoulos et al., SIGMOD'22). In this work, we provide a rigorous theoretical analysis of poisoning attacks targeting linear regression models over CDFs, one of the most basic regression models and a core component in many learned indexes. Our main contributions are as follows: (i) We present a theoretical proof characterizing the optimal single-point poisoning attack and show that the existing method yields the optimal attack. (ii) We show that in multi-point attacks, the existing greedy approach is not always optimal, and we rigorously derive the key properties that an optimal attack should satisfy. (iii) We propose a method to compute an upper bound of the multi-point poisoning attack's impact and empirically demonstrate that the loss under the greedy approach is often close to this bound. Our study deepens the theoretical understanding of attack strategies against linear regression models on CDFs and provides a foundation for the theoretical evaluation of attacks and defenses on learned indexes.
Read the full abstract ▾