Machine Learning · Cross-listed from stat.ML
Position: Don't be Afraid of Over-Smoothing And Over-Squashing
Niklas Kormann, Benjamin Doerr and Johannes F. Lutzeyer
Over-smoothing and over-squashing have been extensively studied in the literature on Graph Neural Networks (GNNs) over the past years. We challenge this prevailing focus in GNN research, arguing that these phenomena are less critical for practical applications than assumed. We suggest that performance decreases often stem from uninformative receptive fields rather than over-smoothing. We support this position with extensive experiments on several standard
benchmark datasets, demonstrating that accuracy and over-smoothing are mostly uncorrelated and that optimal model depths remain small even with mitigation techniques, thus highlighting the negligible role of over-smoothing. Similarly, we challenge that over-squashing is always detrimental in practical applications. Instead, we posit that the distribution of relevant information over the graph frequently factorises and is often localised within a small k-hop neighbourhood, questioning the necessity of jointly observing entire receptive fields or engaging in an extensive search for long-range interactions. The results of our experiments show that architectural interventions designed to mitigate over-squashing fail to yield significant performance gains. This position paper advocates for a paradigm shift in theoretical research, urging a diligent analysis of learning tasks and
datasets using statistics that measure the underlying distribution of label-relevant information to better understand their localisation and factorisation.
Read the full abstract ▾
Machine Learning
PLANET v2.0: A comprehensive Protein-Ligand Affinity Prediction Model Based on Mixture Density Network
Haotian Gao, Xiangying Zhang, Jingyuan Li et al. (+4 more)
Drug discovery represents a time-consuming and financially intensive process, and virtual screening can accelerate it. Scoring functions, as one of the tools guiding virtual screening, have their precision closely tied to screening efficiency. In our previous study, we developed a graph neural network model called
PLANET (
Protein-Ligand Affinity prediction NETwork), but it suffers from the defect in representing
protein-ligand contact maps. Incorrect binding modes inevitably lead to poor affinity predictions, so accurate prediction of the
protein-ligand contact map is desired to improve
PLANET. In this study, we have proposed
PLANET v2.0 as an upgraded version. The model is trained via multi-objective training strategy and incorporates the Mixture Density Network to predict binding modes. Except for the probability density distributions of non-covalent interactions, we innovatively employ another Gaussian mixture model to describe the relationship between distance and energy of each interaction pair and predict
protein-ligand affinity like calculating the mathematical expectation. As on the CASF-2016
benchmark,
PLANET v2.0 demonstrates excellent scoring power, ranking power, and docking power. The screening power of
PLANET v2.0 gets notably improved compared to
PLANET and Glide SP and it demonstrates robust validation on a commercial ultra-large-scale
dataset. Given its efficiency and accuracy,
PLANET v2.0 can hopefully become one of the practical tools for virtual screening workflows.
PLANET v2.0 is freely available at https://www.pdbbind-plus.org.cn/planetv2.
Read the full abstract ▾
All authors Haotian Gao, Xiangying Zhang, Jingyuan Li, Xinchong Chen, Haojie Wang, Yifei Qi, Renxiao Wang
Machine Learning
Adapting Neural Posterior Estimators for Simulation-Based Inference in Agent-Based Models
Junyao Zhang, Jinglai Li and Junqi Tang
Agent-based models (ABMs) are widely used in economics and social sciences. They represent heterogeneous personal behaviors and their interactions by using simulations, whereas their likelihoods are often intractable for parameter estimations. Therefore, most traditional statistical inference techniques are hindered. Driven by recent advances in probabilistic machine learning, recent works have applied neural networks to achieve likelihood-free inference in large-scale ABMs. Specifically, Neural Posterior Estimation using normalizing flows provide amortized
Bayesian inference from simulations, yet a pre-trained
estimator may become inaccurate when the target task changes. In this work, we investigate the practicality of adapting a pre-trained conditional normalizing flow through a unified sequential neural posterior estimation test-time training (SNPE-TTT) framework. The framework includes four adaptation strategies: full-parameter adaptation, low-rank adaptation, and two update spaces identified from target-task gradients. We evaluate all four methods across economic simulation models with both tractable and intractable likelihoods. These
benchmarks demonstrate that the sequential neural posterior estimation test-time training framework can adapt pre-trained posterior
estimators to related inference tasks using a limited budget of target simulations. Posterior recovery and computational benefits depend on the task and adaptation strategy, with lower target-stage cost in some settings but no consistently superior posterior accuracy.
Read the full abstract ▾
Machine Learning
SCALPEL: Selective Capability Ablation via Low-rank Parameter Editing for Large Language Model Interpretability Analysis
Zihao Fu, Xufeng Duan and Zhenguang G. Cai
Large
language models excel across diverse domains, yet their deployment in healthcare, legal systems, and autonomous decision-making remains limited by incomplete understanding of their internal mechanisms. As these models integrate into high-stakes systems, understanding how they encode capabilities has become fundamental to interpretability research. Traditional approaches identify important modules through gradient attribution or activation analysis, assuming specific capabilities map to specific components. However, this oversimplifies neural computation: modules may contribute to multiple capabilities simultaneously, while single capabilities may distribute across multiple modules. These coarse-grained analyses fail to capture fine-grained, distributed capability encoding. We present SCALPEL (Selective Capability Ablation via Low-rank Parameter Editing for Large
language models), a framework representing capabilities as low-rank parameter subspaces rather than discrete modules. Our key insight is that capabilities can be characterized by low-rank modifications distributed across layers and modules, enabling precise capability removal without affecting others. By training LoRA adapters to reduce distinguishing correct from incorrect answers while preserving general
language modeling quality, SCALPEL identifies low-rank representations responsible for particular capabilities while remaining disentangled from others. Experiments across diverse capability and linguistic tasks from BLiMP demonstrate that SCALPEL successfully removes target capabilities while preserving general capabilities, providing fine-grained insights into capability distribution across parameter space. Results reveal that capabilities exhibit low-rank structure and can be selectively ablated through targeted parameter-space interventions, offering nuanced understanding of capability encoding in LLMs.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Outcome-Grounded Advantage Reshaping for Fine-Grained Credit Assignment in Mathematical Reasoning
Ziheng Li, Liu Kang, Feng Xiao et al. (+7 more)
Group Relative Policy Optimization (GRPO) has emerged as a promising critic-free reinforcement learning paradigm for
reasoning tasks. However, standard GRPO employs a coarse-grained credit assignment mechanism that propagates group-level rewards uniformly to to every
token in a sequence, neglecting the varying contribution of individual
reasoning steps. We address this limitation by introducing Outcome-grounded Advantage Reshaping (OAR), a fine-grained credit assignment mechanism that redistributes advantages based on how much each
token influences the model's final answer. We instantiate OAR via two complementary strategies: (1) OAR-P, which estimates outcome sensitivity through counterfactual
token perturbations, serving as a high-fidelity attribution signal; (2) OAR-G, which uses an input-gradient sensitivity proxy to approximate the influence signal with a single backward pass. These importance signals are integrated with a conservative Bi-Level advantage reshaping scheme that suppresses low-impact
tokens and boosts pivotal ones while preserving the overall advantage mass. Empirical results on extensive mathematical
reasoning benchmarks demonstrate that while OAR-P sets the performance upper bound, OAR-G achieves comparable gains with negligible computational overhead, both significantly outperforming a strong GRPO baseline, pushing the boundaries of critic-free LLM
reasoning.
Read the full abstract ▾
All authors Ziheng Li, Liu Kang, Feng Xiao, Luxi Xing, Qingyi Si, Zhuoran Li, Weikang Gong, Deqing Yang, Yanghua Xiao, Hongcheng Guo
Machine Learning
OceanSAR-2: A Universal Feature Extractor for SAR Ocean Observation
Alexandre Tuel, Thomas Kerdreux, Quentin Febvre et al. (+6 more)
We present OceanSAR-2, the second generation of our foundation model for SAR-based ocean observation. Building on our earlier release, which pioneered self-supervised learning on Sentinel-1 Wave Mode data, OceanSAR-2 relies on improved SSL training and dynamic data curation strategies, which enhances performance while reducing training cost. OceanSAR-2 demonstrates strong transfer performance across downstream tasks, including geophysical pattern classification, ocean surface wind vector and significant wave height estimation, and iceberg detection. We release standardized
benchmark datasets, providing a foundation for systematic evaluation and advancement of SAR models for ocean applications.
Read the full abstract ▾
All authors Alexandre Tuel, Thomas Kerdreux, Quentin Febvre, Alexis Mouche, Antoine Grouazel, Jean-Renaud Miadana, Antoine Audras, Chen Wang, Bertrand Chapron
Machine Learning
The Post-Training Dilemma: Why We Should Rethink the Sequential SFT-RL Paradigm
Xueyan Niu, Bo Bai, Wei Han et al. (+1 more)
This position paper argues that the sequential supervised fine-tuning followed by reinforcement learning (SFT-then-RL) pipeline—the de facto paradigm for large
language model (LLM) post-training—has a fundamental property that cannot be fixed by hyperparameter tuning: SFT and RL optimize over nearly orthogonal parameter subspaces, so RL inevitably degrades the capabilities acquired during the preceding SFT phase. Under the Polyak-Lojasiewicz (PL) condition, which is empirically observed for over-parameterized networks, we show that RL increases the SFT loss by at least $\frac{\mu}{2}\mathrm{dist}(\theta_{\mathrm{RL}}, \Theta^*_{\mathrm{SFT}})^2$ for some $\mu>0$. Although the KL penalty can constrain distributional shift, it cannot correct the underlying gradient misalignment between the two objectives. Experiments on Qwen3-0.6B across four tasks confirm the theoretical predictions: on MATH-500, Alpaca, HH-RLHF, and SST-2, the SFT loss increases after the RL phase. We identify the underlying mechanism as persistent near-orthogonality between the gradients of the two objectives, paired with opposing landscape dynamics: SFT flattens the loss landscape while RL re-sharpens it. Based on our analysis, we propose three promising research directions: (1) joint optimization of the two objectives with gradient surgery to resolve conflicting updates; (2) landscape-aware scheduling that terminates RL training at the theoretically optimal point; and (3) characterizing the model scale threshold above which SFT becomes unnecessary.
Read the full abstract ▾
All authors Xueyan Niu, Bo Bai, Wei Han, Weixi Zhang
Machine Learning
Computing patient similarity based on unstructured clinical notes
Petr Zelina, Marko Řeháček, Jana Halámková et al. (+3 more)
Clinical notes hold rich yet unstructured details about diagnoses, treatments, and outcomes that are vital to precision medicine but hard to exploit at scale. We introduce a method that represents each patient as a
matrix built from aggregated
embeddings of all their notes, enabling robust patient similarity computation based on their latent low-rank representations. Using clinical notes of 4,267 Czech breast-cancer patients and expert similarity labels from Masaryk Memorial Cancer Institute, we evaluate several
matrix-based similarity measures and analyze their strengths and limitations across different similarity facets, such as clinical history, treatment, and adverse events. The results demonstrate the usefulness of the presented method for downstream tasks, such as personalized therapy recommendations or toxicity warnings.
Read the full abstract ▾
All authors Petr Zelina, Marko Řeháček, Jana Halámková, Lucia Bohovicová, Martin Rusinko, Vít Nováček
Machine Learning
CompNO: A Novel Foundation Model approach for solving Partial Differential Equations
Hamda Hmida, Hsiu-Wen Chang Joly and Youssef Mesri
Partial differential equations (PDEs) govern a wide range of physical phenomena, but their numerical solution remains computationally demanding, especially when repeated simulations are required across many parameter settings. Recent Scientific Foundation Models (SFMs) aim to alleviate this cost by learning universal surrogates from large collections of simulated systems, yet they typically rely on monolithic architectures with limited interpretability and high pretraining expense. In this work we introduce Compositional Neural Operators (CompNO), a compositional neural operator framework for parametric PDEs. Instead of pretraining a single large model on heterogeneous data, CompNO first learns a library of Foundation Blocks, where each block is a parametric
Fourier neural operator specialized to a fundamental differential operator (e.g. convection,
diffusion, nonlinear convection). These blocks are then assembled, via lightweight Adaptation Blocks, into task-specific solvers that approximate the temporal
evolution operator for target PDEs. A dedicated boundary-condition operator further enforces Dirichlet constraints exactly at inference time. We validate CompNO on one-dimensional convection,
diffusion, convection–
diffusion and Burgers' equations from the PDEBench suite. The proposed framework achieves lower relative L2 error than strong baselines (PFNO, PDEFormer and in-context learning based models) on linear parametric systems, while remaining competitive on nonlinear Burgers' flows. The model maintains exact boundary satisfaction with zero loss at domain boundaries, and exhibits robust generalization across a broad range of Peclet and Reynolds numbers. These results demonstrate that compositional neural operators provide a scalable and physically interpretable pathway towards foundation models for PDEs.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.SD
SEE: Signal Embedding Energy for Quantifying Noise Interference in Large Audio Language Models
Yuanhe Zhang, Jiayu Tian, Yibo Zhang et al. (+5 more)
Large Audio
Language Models (LALMs) have been widely applied in real-time scenarios, such as in-car assistants and online meeting comprehension. In practice, audio inputs are often corrupted by device and environmental noise, leading to performance degradation. However, existing LALM studies on noise lack quantitative analysis and rely mainly on intuition and empirical observation, thus failing to understand practical robustness. To address this issue, we introduce Signal
Embedding Energy (SEE), a method for quantifying the impact of noise intensity on LALM inputs, enabling the differentiation of LALM robustness in real-world deployments. SEE introduces a perspective based on structured activation subspaces derived from the model's internal representations, which more accurately captures its perception of noise than raw audio features. Across experiments, SEE exhibits a strong correlation with LALM performance, achieving a correlation of 0.98. Surprisingly, traditional audio denoising methods are only marginally effective for LALMs, and, in some cases, even increase SEE and impair performance. This suggests a mismatch between
speech-centric denoising objectives and the noise sensitivity of modern LALMs. Therefore, we propose a mitigation strategy derived from SEE to denoise LALM inputs, outperforming existing denoising methods. This paper introduces a novel metric for noise quantification in LALMs, providing guidance for robustness improvements in real-world deployments.
Read the full abstract ▾
All authors Yuanhe Zhang, Jiayu Tian, Yibo Zhang, Shilinlu Yan, Liang Lin, Zhenhong Zhou, Li Sun, Sen Su
Machine Learning · Cross-listed from math.OC
Convergence Rate Analysis of the AdamW-style Shampoo: Unifying One-Sided and Two-Sided Preconditioning
Huan Li, Yiming Dong and Zhouchen Lin
This paper studies AdamW-style Shampoo, an effective variant of the classical Shampoo that won the external tuning track of the AlgoPerf neural network training competition. Our analysis unifies one-sided and two-sided preconditioning. When the exponents of the two preconditioners sum to $1/2$, we establish the convergence rate $\frac{1}{K}\sum_{k=1}^KE\left[||\nabla f(X_k)||_*\right]\leq O(\frac{\sqrt{m+n}C}{K^{1/4}})$, where $K$ represents the number of iterations, $(m,n)$ denotes the dimensions of the
matrix-valued parameters, and $C$ matches the constant appearing in the optimal convergence rate of SGD. Theoretically, the
nuclear norm and Frobenius norm satisfy $||\nabla f(X)||_F\leq ||\nabla f(X)||_*\leq \sqrt{\min\{m,n\}}||\nabla f(X)||_F$, which suggests that our convergence rate is analogous to the optimal $\frac{1}{K}\sum_{k=1}^KE\left[||\nabla f(X_k)||_F\right]\leq O(\frac{C}{K^{1/4}})$ convergence rate of SGD in the ideal case where $||\nabla f(X)||_*= \Theta(\sqrt{\min\{m,n\}})||\nabla f(X)||_F$ and $m$ and $n$ are of comparable magnitude. Then, we extend our analysis to settings where the preconditioning exponents do not sum to 1/2, and establish convergence with an explicit but more involved rate.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Robust Bayesian Inference via Variational Approximations of Generalized Rho-Posteriors
EL Mahdi Khribch and Pierre Alquier
We introduce the $\widetilde{\rho}$-posterior, a modified version of the $\rho$-posterior, obtained by replacing the supremum over competitor parameters with a softmax aggregation. This modification allows a PAC-
Bayesian analysis of the $\widetilde{\rho}$-posterior. This yields finite-sample oracle inequalities with explicit convergence rates that inherit the key robustness properties of the original framework, in particular, graceful degradation under model misspecification and data contamination. Crucially, the PAC-
Bayesian oracle inequalities extend to variational approximations of the $\widetilde{\rho}$-posterior, providing theoretical guarantees for tractable inference. Numerical experiments on exponential families, regression, and real-world
datasets confirm that the resulting variational procedures achieve robustness competitive with theoretical predictions at computational cost comparable to standard variational Bayes.
Read the full abstract ▾
Machine Learning
Segmental Advantage Estimation: Enhancing PPO for Long-Context LLM Training
Xue Gong, Qi Yi, Ziyuan Nan et al. (+8 more)
Training Large
Language Models (LLMs) for
reasoning tasks is increasingly driven by Reinforcement Learning with Verifiable Rewards (RLVR), where Proximal Policy Optimization (PPO) provides a principled framework for stable policy updates. However, the practical application of PPO is hindered by unreliable advantage estimation in the sparse-reward RLVR regime. This issue arises because the sparse rewards in RLVR lead to inaccurate intermediate value predictions, which in turn introduce significant bias when aggregated at every
token by Generalized Advantage Estimation (GAE). To address this, we introduce Segmental Advantage Estimation (SAE), which mitigates the bias that GAE can incur in RLVR. Our key insight is that aggregating $n$-step advantages at every
token(as in GAE) is unnecessary and often introduces excessive bias, since individual
tokens carry minimal information. Instead, SAE first partitions the generated sequence into coherent sub-segments using low-probability
tokens as heuristic boundaries. It then selectively computes variance-reduced advantage estimates only from these information-rich segment transitions, effectively filtering out noise from intermediate
tokens. Our experiments demonstrate that SAE achieves superior performance, with marked improvements in final scores, training stability, and sample efficiency. These gains are shown to be consistent across multiple model sizes, and a correlation analysis confirms that our proposed advantage
estimator achieves a higher correlation with an approximate ground-truth advantage, justifying its superior performance.
Read the full abstract ▾
All authors Xue Gong, Qi Yi, Ziyuan Nan, Guanhua Huang, Kejiao Li, Yuhao Jiang, Ruibin Xiong, Zenan Xu, Jiaming Guo, Shaohui Peng, Bo Zhou
Machine Learning
BEAT-Net: Injecting Biomimetic Spatio-Temporal Priors for Interpretable ECG Diagnosis
Runze Ma, Haonan Lyu, Shunbo Jia et al. (+5 more)
Automated electrocardiogram diagnosis using deep learning remains limited by signal-agnostic representations that treat multi-lead recordings as undifferentiated time-series or images, forcing models to rediscover physiological structure implicitly. This leads to data inefficiency, poor generalization, and opaque decision boundaries misaligned with clinical
reasoning. We present BEAT-Net, a supervised biomimetic framework that integrates QRS-centered biological tokenization with a hierarchical architecture mirroring the cardiologist's workflow. A QRS tokenizer converts continuous signals into semantically complete heartbeat sequences, which are processed through four specialized stages: morphological feature extraction via a Word Encoder, lead-invariant normalization through a Spatial Operator, temporal context injection by a Temporal Operator, and global
reasoning using a
Transformer-based Sentence Encoder. Evaluated across three large-scale
benchmarks including PTB-XL, CPSC2018, and CSN, BEAT-Net achieves diagnostic accuracy of 0.924 AUC, comparable to dominant CNN baselines at 0.925 AUC, while reducing parameters by 95 percent from 2.06 million to 0.7 million. Critically, BEAT-Net surpasses the 39.5-million-parameter foundation model HeartLang on morphological Form classification, reaching 0.901 AUC compared to HeartLang's 0.832 AUC, while attaining full CNN-level performance using only 35 percent of training data and exhibiting superior cross-
dataset generalization. Learned
attention patterns spontaneously align with established clinical heuristics, demonstrating that explicit physiological structure provides a more efficient and interpretable alternative to massive pre-training for clinical deployment.
Read the full abstract ▾
All authors Runze Ma, Haonan Lyu, Shunbo Jia, Qiang Yang, Muzi Xu, Jiaqi Zhang, Zihe Luo, Caizhi Liao
Machine Learning
Explaining Machine Learning Predictive Models through Conditional Expectation Methods
Silvia Ruiz-España, Laura Arnal, François Signol et al. (+2 more)
The rapid adoption of complex Artificial Intelligence (AI) and Machine Learning (ML) models has led to their characterization as black boxes due to the difficulty of explaining their internal decision-making processes. This lack of transparency hinders users' ability to understand, validate and trust model behavior, particularly in high-risk applications. Although explainable AI (XAI) has made significant progress, there remains a need for versatile and effective techniques to address increasingly complex models. This work introduces Multivariate Conditional Expectation (MUCE), a model-agnostic method for local explainability designed to capture prediction changes from feature interactions. MUCE extends Individual Conditional Expectation (ICE) by exploring a multivariate grid of values in the neighborhood of a given observation at inference time, providing graphical explanations that illustrate the local
evolution of model predictions. In addition, two quantitative indices, stability and uncertainty, summarize local behavior and assess model reliability. Uncertainty is further decomposed into uncertainty+ and uncertainty- to capture asymmetric effects that global measures may overlook. The proposed method is validated using XGBoost models trained on three
datasets: two synthetic (2D and 3D) to evaluate behavior near decision boundaries, and one transformed real-world
dataset to test adaptability to heterogeneous feature types. Results show that MUCE effectively captures complex local model behavior, while the stability and uncertainty indices provide meaningful insight into prediction confidence. MUCE, together with the ICE modification and the proposed indices, offers a practical contribution to local explainability, enabling both graphical and quantitative insights that enhance the interpretability of predictive models and support more trustworthy and transparent decision-making.
Read the full abstract ▾
All authors Silvia Ruiz-España, Laura Arnal, François Signol, Juan-Carlos Perez-Cortes, Joaquim Arlandis
Machine Learning · Cross-listed from cs.AI
ARM: Role-Conditioned Neuron Transplantation for Training-Free Generalist LLM Agent Merging
Zhuoka Feng, Kang Chen, Sihan Zhao et al. (+7 more)
Interactive large
language model
agents have advanced rapidly, but most remain specialized to a single environment and fail to adapt robustly to other environments. Model merging offers a training-free alternative by integrating multiple experts into a single model. In this paper, we propose
Agent-Role Merging (ARM), an activation-guided, role-conditioned
neuron transplantation method for model merging in LLM
agents. ARM improves existing merging methods from static natural
language tasks to multi-turn
agent scenarios, and over the generalization ability across various interactive environments. This is achieved with a well designed 3-step framework: 1) constructing merged backbones, 2) selection based on its role-conditioned activation analysis, and 3)
neuron transplantation for fine-grained refinements. Without gradient-based optimization, ARM improves cross-
benchmark generalization while enjoying efficiency. Across diverse domains, the model obtained via ARM merging outperforms prior model merging methods and domain-specific expert models, while demonstrating strong out-of-domain generalization.
Read the full abstract ▾
All authors Zhuoka Feng, Kang Chen, Sihan Zhao, Kai Xiong, Yaoning Wang, Minshen Yu, Junjie Nian, Changyi Xiao, Yixin Cao, Yugang Jiang
Machine Learning · Cross-listed from astro-ph.GA
Photometric Redshift Estimation Using Scaled Ensemble Learning
Swagata Biswas, Shubhrangshu Ghosh, Avyarthana Ghosh et al. (+5 more)
The development of the state-of-the-art telescopic systems capable of performing expansive sky surveys such as the Sloan Digital Sky Survey, Euclid, and the Rubin Observatory's Legacy Survey of Space and Time (LSST) has significantly advanced efforts to refine cosmological models. These advances offer deeper insight into persistent challenges in astrophysics and our understanding of the Universe's
evolution. A critical component of this progress is the reliable estimation of photometric
redshifts (Pz). To improve the precision and efficiency of such estimations, the application of machine learning (ML) techniques to large-scale astronomical
datasets has become essential. This study presents a new ensemble-based ML framework aimed at predicting Pz for faint
galaxies and higher
redshift ranges, relying solely on optical (grizy) photometric data. The proposed architecture integrates several learning
algorithms, including gradient boosting machine, extreme gradient boosting, k-nearest neighbors, and artificial neural networks, within a scaled ensemble structure. By using bagged input data, the ensemble approach delivers improved predictive performance compared to stand-alone models. The framework demonstrates consistent accuracy in estimating
redshifts, maintaining strong performance up to z ~ 4. The model is validated using publicly available data from the Hyper Suprime-Cam Strategic Survey Program by the Subaru
Telescope. Our results show marked improvements in the precision and reliability of Pz estimation. Furthermore, this approach closely adheres to-and in certain instances exceeds-the
benchmarks specified in the LSST Science Requirements Document. Evaluation metrics include catastrophic outlier, bias, and rms.
Read the full abstract ▾
All authors Swagata Biswas, Shubhrangshu Ghosh, Avyarthana Ghosh, Yogesh Wadadekar, Abhishek Roy Choudhury, Arijit Mukherjee, Shailesh Deshpande, Arpan Pal
Machine Learning
Kernel Alignment-based Multi-view Unsupervised Feature Selection with Sample-level Adaptive Graph Learning
Yalan Tan, Yanyong Huang, Zongxin Shen et al. (+3 more)
Although multi-view unsupervised feature selection (MUFS) has demonstrated success in dimensionality reduction for unlabeled multi-view data, most existing methods reduce feature redundancy by focusing on linear correlations among features but often overlook complex nonlinear dependencies. This limits the effectiveness of feature selection. In addition, existing methods fuse similarity graphs from multiple views by employing sample-invariant weights to preserve local structure. However, this process fails to account for differences in local neighborhood clarity among samples within each view, thereby hindering accurate characterization of the intrinsic local structure of the data. In this paper, we propose a Kernel
Alignment-based multi-view unsupervised FeatUre selection with Sample-level adaptive graph lEarning method (KAFUSE) to address these issues. Specifically, we first employ kernel
alignment with an orthogonal constraint to reduce feature redundancy in both linear and nonlinear relationships. Then, a cross-view consistent similarity graph is learned by applying sample-level fusion to each slice of a tensor formed by stacking similarity graphs from different views, which automatically adjusts the view weights for each sample during fusion. These two steps are integrated into a unified model for feature selection, enabling mutual enhancement between them. Extensive experiments on real multi-view
datasets demonstrate the superiority of KAFUSE over state-of-the-art methods.
Read the full abstract ▾
All authors Yalan Tan, Yanyong Huang, Zongxin Shen, Dongjie Wang, Fengmao Lv, Tianrui Li
Machine Learning · Cross-listed from stat.ML
Covariance-Driven Regression Trees: Reducing Overfitting in CART
Likun Zhang and Wei Ma
Decision trees are powerful machine learning
algorithms, widely used in fields such as economics and medicine for their simplicity and interpretability. However, decision trees such as CART are prone to overfitting, especially when grown deep or the sample size is small. Conventional methods to reduce overfitting include pre-
pruning and post-
pruning, which constrain the growth of uninformative branches. In this paper, we propose a complementary approach by introducing a covariance-driven splitting criterion for regression trees (CovRT). This method is more robust to overfitting than the empirical risk minimization criterion used in CART, as it produces more balanced and stable splits and more effectively identifies
covariates with true signals. We establish an oracle inequality of CovRT and prove that its predictive accuracy is comparable to that of CART in high-dimensional settings. We find that CovRT achieves superior prediction accuracy compared to CART in both simulations and real-world tasks.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CR
A High-Recall Cost-Sensitive Machine Learning Framework for Real-Time Online Banking Transaction Fraud Detection
Karthikeyan V. R., Premnath S., Kavinraaj S. et al. (+1 more)
Fraudulent activities on digital banking services are becoming more intricate by the day, challenging existing defenses. While older rule driven methods struggle to keep pace, even precision focused
algorithms fall short when new scams are introduced. These tools typically overlook subtle shifts in criminal behavior, missing crucial signals. Because silent breaches cost institutions far more than flagged but legitimate actions, catching every possible case is crucial. High sensitivity to actual threats becomes essential when oversight leads to heavy losses. One key aim here involves reducing missed fraud cases without spiking incorrect alerts too much. This study builds a system using group learning methods adjusted through smart threshold choices. Using real world transaction records shared openly, where cheating acts rarely appear among normal activities, tests are run under practical skewed distributions. The outcomes reveal that approximately 98 percent of actual fraud is detected, outperforming standard setups that rely on unchanging rules when dealing with uneven examples across classes. When tested in live settings, the fraud detection system connects directly to an online banking transaction flow, stopping questionable activities before they are completed. Alongside this setup, a browser add on built for Chrome is designed to flag deceptive web links and reduce threats from harmful sites. These results show that adjusting decisions by cost impact and validating across entire systems makes deployment more stable and realistic for today's digital banking platforms.
Read the full abstract ▾
All authors Karthikeyan V. R., Premnath S., Kavinraaj S., J. Sangeetha
Machine Learning
Pseudodata-guided Invariant Representation Learning Boosts the Out-of-Distribution Generalization in Enzymatic Kinetic Parameter Prediction
Haomin Wu, Zhiwei Nie, Hongyu Zhang et al. (+1 more)
Accurate prediction of enzyme kinetic parameters is essential for understanding catalytic mechanisms and guiding enzyme engineering.However, existing deep learning-based enzyme-substrate interaction (ESI) predictors often exhibit performance degradation on sequence-divergent, out-of-distribution (OOD) cases, limiting robustness under biologically relevant perturbations.We propose O$^2$DENet, a lightweight, plug-and-play module that enhances OOD generalization via biologically and chemically informed perturbation augmentation and invariant representation learning.O$^2$DENet introduces enzyme-substrate perturbations and enforces consistency between original and augmented enzyme-substrate-pair representations to encourage invariance to distributional shifts.When integrated with representative ESI models, O$^2$DENet consistently improves predictive performance for both $k_{cat}$ and $K_m$ across stringent sequence-identity-based OOD
benchmarks, achieving state-of-the-art results among the evaluated methods in terms of accuracy and robustness metrics.Overall, O$^2$DENet provides a general and effective strategy to enhance the stability and deployability of data-driven enzyme kinetics predictors for real-world enzyme engineering applications.
Read the full abstract ▾
All authors Haomin Wu, Zhiwei Nie, Hongyu Zhang, Zhixiang Ren
Machine Learning
Simulated Annealing-based Candidate Optimization for Batch Acquisition Functions
Sk Md Ahnaf Akif Alvi, Raymundo Arróyave and Douglas Allaire
Bayesian Optimization with multi-objective acquisition functions such as q-Expected Hypervolume Improvement (qEHVI) requires efficient candidate optimization to maximize acquisition function values. Traditional approaches rely on continuous optimization methods like Sequential Least Squares Programming (SLSQP) for candidate selection. However, these gradient-based methods can become trapped in local optima, particularly in complex or high-dimensional objective landscapes. This paper presents a simulated annealing-based approach for candidate optimization in batch acquisition functions as an alternative to conventional continuous optimization methods. We evaluate our simulated annealing approach against SLSQP across four
benchmark multi-objective optimization problems: ZDT1 (30D, 2 objectives), DTLZ2 (7D, 3 objectives), Kursawe (3D, 2 objectives), and Latent-Aware (4D, 2 objectives). Our results demonstrate that simulated annealing consistently achieves superior hypervolume performance compared to SLSQP in most test functions. The improvement is particularly pronounced for DTLZ2 and Latent-Aware problems, where simulated annealing reaches significantly higher hypervolume values and maintains better convergence characteristics. The histogram analysis of objective space coverage further reveals that simulated annealing explores more diverse and optimal regions of the Pareto front. These findings suggest that metaheuristic optimization approaches like simulated annealing can provide more robust and effective candidate optimization for multi-objective
Bayesian optimization, offering a promising alternative to traditional gradient-based methods for batch acquisition function optimization.
Read the full abstract ▾
Machine Learning
Innovation Capacity of Dynamical Learning Systems
Anthony M. Polloreno
In noisy physical reservoirs, the classical information-processing capacity $C_{\mathrm{ip}}$ quantifies how well a linear readout can realize tasks measurable from the input history, yet $C_{\mathrm{ip}}$ can be far smaller than the observed rank of the readout covariance. We explain this “missing capacity” by introducing the innovation capacity $C_{\mathrm{i}}$, the total capacity allocated to readout components orthogonal to the input filtration (Doob innovations, including input-noise mixing). Using a basis-free
Hilbert-space formulation of the predictable/innovation decomposition, we prove the conservation law $C_{\mathrm{ip}}+C_{\mathrm{i}}=\mathrm{rank}(\Sigma_{XX})\le d$, so predictable and innovation capacities exactly partition the rank of the observable readout dimension covariance $\Sigma_{XX}\in \mathbb{R}^{\rm d\times d}$. In linear-Gaussian Johnson-Nyquist regimes, $\Sigma_{XX}(T)=S+T N_0$, the split becomes a generalized-
eigenvalue shrinkage rule and gives an explicit monotone tradeoff between temperature and predictable capacity. Geometrically, in whitened coordinates the predictable and innovation components correspond to complementary covariance ellipsoids, making $C_{\mathrm{i}}$ a trace-controlled innovation budget. A large $C_{\mathrm{i}}$ forces a high-dimensional innovation subspace with a variance floor and under mild mixing and anti-concentration assumptions this yields extensive innovation-block differential
entropy and exponentially many distinguishable histories. Finally, we give an information-theoretic lower bound showing that learning the induced innovation-block law in total variation requires a number of samples that scales with the effective innovation dimension, supporting the generative utility of noisy physical reservoirs.
Read the full abstract ▾
Machine Learning
DDT: A Dual-Masking Dual-Expert Transformer for Energy Time-Series Forecasting
Mingnan Zhu, Qixuan Zhang, Yixuan Cheng et al. (+2 more)
Accurate energy time-series
forecasting is crucial for ensuring grid stability and promoting the integration of renewable energy, yet it faces significant challenges from complex temporal dependencies and the heterogeneity of multi-source data. To address these issues, we propose DDT, a novel and robust deep learning framework for high-precision time-series
forecasting. At its core, DDT introduces two key innovations. First, we design a dual-masking mechanism that synergistically combines a strict causal mask with a data-driven dynamic mask. This novel design ensures theoretical causal consistency while adaptively focusing on the most salient historical information, overcoming the
rigidity of traditional masking techniques. Second, our architecture features a dual-expert system that decouples the modeling of temporal dynamics and cross-variable correlations into parallel, specialized pathways, which are then intelligently integrated through a dynamic gated fusion module. We conducted extensive experiments on 7 challenging energy
benchmark datasets, including ETTh, Electricity, and
Solar. The results demonstrate that DDT consistently outperforms strong state-of-the-art baselines across all prediction horizons, establishing a new
benchmark for the task.
Read the full abstract ▾
All authors Mingnan Zhu, Qixuan Zhang, Yixuan Cheng, Fangzhou Gu, Shiming Lin
Machine Learning · Cross-listed from stat.ML
Multi-environment Invariance Learning with Missing Data
Yiran Jia and Jelena Bradic
Learning models that can handle distribution shifts is a key challenge in domain generalization. Invariance learning, an approach that focuses on identifying features invariant across environments, improves model generalization by capturing stable relationships, which may represent causal effects when the data distribution is encoded within a structural equation model (SEM) and satisfies modularity conditions. This has led to a growing body of work that builds on invariance learning, leveraging the inherent heterogeneity across environments to develop methods that provide causal explanations while enhancing robust prediction. However, in many practical scenarios, obtaining complete outcome data from each environment is challenging due to the high cost or complexity of data collection. This limitation in available data hinders the development of models that fully leverage environmental heterogeneity, making it crucial to address missing outcomes to improve both causal insights and robust prediction. In this work, we derive an
estimator from the invariance objective under missing outcomes. We establish non-asymptotic guarantees on variable selection property and $\ell_2$ error convergence rates, which are influenced by the proportion of missing data and the quality of imputation models across environments. We evaluate the performance of the new
estimator through extensive simulations and demonstrate its application using the UCI Bike Sharing
dataset to predict the count of bike rentals. The results show that despite relying on a biased imputation model, the
estimator is efficient and achieves lower prediction error, provided the bias is within a reasonable range.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
Learning to Trust the Crowd: A Multi-Model Consensus Reasoning Engine for Large Language Models
Pranav Kallem
Large
language models (LLMs) achieve strong average performance yet remain unreliable at the instance level, with frequent
hallucinations, brittle failures, and poorly calibrated confidence. We study reliability through the lens of multi-model consensus: given responses from several heterogeneous LLMs, can we learn which answer is most likely correct for a given query? We introduce a Multi-Model Consensus
Reasoning Engine that treats the set of LLM outputs as input to a supervised meta-learner. The system maps natural
language responses into structured features using semantic
embeddings, pairwise similarity and clustering statistics, lexical and structural cues,
reasoning-quality scores, confidence estimates, and model-specific priors, and then applies gradient-boosted trees, listwise ranking, and graph neural networks over similarity graphs of answers. Using three open-weight LLMs evaluated on compact, resource-constrained subsets of GSM8K, ARC-Challenge, HellaSwag, and TruthfulQA, our best graph-
attention-based consensus model improves macro-average accuracy by 4.6 percentage points over the strongest single LLM and by 8.1 points over majority vote, while also yielding lower Brier scores and fewer TruthfulQA
hallucinations. Ablation and feature-importance analyses show that semantic agreement and clustering features are most influential, with
reasoning-quality and model-prior features providing complementary gains, suggesting supervised multi-model consensus is a practical route toward more reliable LLM behavior, even in a modest single-machine setup.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
Consolidation or Adaptation? PRISM: Disentangling SFT and RL Data via Gradient Concentration
Yang Zhao, Yangou Ouyang, Xiao Ding et al. (+8 more)
While Hybrid Supervised Fine-Tuning (SFT) followed by Reinforcement Learning (RL) has become the standard paradigm for training LLM
agents, effective mechanisms for data allocation between these stages remain largely underexplored. Current data arbitration strategies often rely on surface-level heuristics that fail to diagnose intrinsic learning needs. Since SFT targets pattern consolidation through imitation while RL drives structural adaptation via exploration, misaligning data with these functional roles causes severe optimization interference. We propose PRISM, a dynamics-aware framework grounded in Schema Theory that arbitrates data based on its degree of cognitive conflict with the model's existing knowledge. By analyzing the spatial
geometric structure of gradients, PRISM identifies data triggering high spatial concentration as high-conflict signals that require RL for structural restructuring. In contrast, data yielding diffuse updates is routed to SFT for efficient consolidation. Extensive experiments on WebShop and ALFWorld demonstrate that PRISM achieves a Pareto improvement, outperforming state-of-the-art hybrid methods while reducing computational costs by up to 3.22$\times$. Our findings suggest that disentangling data based on internal optimization regimes is crucial for scalable and robust
agent alignment.
Read the full abstract ▾
All authors Yang Zhao, Yangou Ouyang, Xiao Ding, Hepeng Wang, Bibo Cai, Kai Xiong, Jinglong Gao, Zhouhao Sun, Li Du, Bing Qin, Ting Liu
Machine Learning
MAESTRO: Meta-learning Adaptive Estimation of Scalarization Trade-offs for Reward Optimization
Yang Zhao, Hepeng Wang, Xiao Ding et al. (+8 more)
Group-Relative Policy Optimization (GRPO) has emerged as an efficient paradigm for aligning Large
Language Models (LLMs), yet its efficacy is primarily confined to domains with verifiable ground truths. Extending GRPO to open-domain settings remains a critical challenge, as unconstrained generation entails multi-faceted and often conflicting objectives - such as creativity versus factuality - where rigid, static reward scalarization is inherently suboptimal. To address this, we propose MAESTRO (Meta-learning Adaptive Estimation of Scalarization Trade-offs for Reward Optimization), which introduces a meta-cognitive orchestration layer that treats reward scalarization as a dynamic latent policy, leveraging the model's terminal hidden states as a semantic bottleneck to perceive task-specific priorities. We formulate this as a contextual bandit problem within a bi-level optimization framework, where a lightweight Conductor network co-evolves with the policy by utilizing group-relative advantages as a meta-reward signal. Across seven
benchmarks, MAESTRO consistently outperforms single-reward and static multi-objective baselines, while preserving the efficiency advantages of GRPO, and in some settings even reducing redundant generation.
Read the full abstract ▾
All authors Yang Zhao, Hepeng Wang, Xiao Ding, Yangou Ouyang, Bibo Cai, Kai Xiong, Jinglong Gao, Zhouhao Sun, Li Du, Bing Qin, Ting Liu
Machine Learning
CalPro: Prior-Aware Evidential–Conformal Prediction with Structure-Aware Guarantees for Protein Structures
Ibne Farabi Shihab, Sanjeda Akter and Anuj Sharma
Deep
protein structure predictors such as AlphaFold provide confidence estimates (e.g., pLDDT) that are often miscalibrated and degrade under distribution shifts across experimental modalities, temporal changes, and intrinsically disordered regions. We introduce CalPro, a prior-aware evidential-conformal framework for shift-robust uncertainty quantification. CalPro combines (i) a
geometric evidential head that outputs Normal-Inverse-Gamma predictive distributions via a graph-based architecture; (ii) a differentiable conformal layer that enables end-to-end training with finite-sample coverage guarantees; and (iii) domain priors (disorder, flexibility) encoded as soft constraints. We derive structure-aware coverage guarantees under distribution shift using PAC-
Bayesian bounds over ambiguity sets, and show that CalPro maintains near-nominal coverage while producing tighter intervals than standard conformal methods in regions where priors are informative. Empirically, CalPro exhibits at most 5% coverage degradation across modalities (vs. 15-25% for baselines), reduces calibration error by 30-50%, and improves downstream ligand-docking success by 25%. Beyond
proteins, CalPro applies to structured regression tasks in which priors encode local reliability, validated on non-biological
benchmarks.
Read the full abstract ▾
Machine Learning
Safeguarding LLM Fine-tuning via Push-Pull Distributional Alignment
Haozhong Wang, Zhuo Li, Yibo Yang et al. (+3 more)
The inherent safety
alignment of Large
Language Models (LLMs) is prone to erosion during fine-tuning, even when using seemingly innocuous
datasets. While existing defenses attempt to mitigate this via data selection, they typically rely on heuristic, instance-level assessments that neglect the global geometry of the data distribution and fail to explicitly repel harmful patterns. To address this, we introduce Safety Optimal Transport (SOT), a novel framework that reframes safe fine-tuning from an instance-level filtering challenge to a distribution-level
alignment task grounded in Optimal Transport (OT). At its core is a dual-reference “push-pull” weight-learning mechanism: SOT optimizes sample importance by actively pulling the downstream distribution towards a trusted safe anchor while simultaneously pushing it away from a general harmful reference. This establishes a robust
geometric safety boundary that effectively purifies the training data. Extensive experiments across diverse model families and domains demonstrate that SOT significantly enhances model safety while maintaining competitive downstream performance, achieving a superior safety-utility trade-off compared to baselines.
Read the full abstract ▾
All authors Haozhong Wang, Zhuo Li, Yibo Yang, He Zhao, Hongyuan Zha, Dandan Guo