Machine Learning
Beyond local reconstruction: Geometry-preserving output alignment for binary LLM quantization
Dung Anh Hoang, Cuong Pham, Cuong Nguyen et al. (+3 more)
Recent advances in post-training quantization for large
language models (LLMs) have demonstrated that low-bit quantization methods can maintain most of the original model performance. Despite such successes, 1-bit quantization remains particularly challenging. A common strategy in 1-bit quantization is to determine binary weights by matching full-precision parameters, following a weight-driven criterion. However, this objective is not directly aligned with the quantized model's objective, which is to preserve the model's output behavior under the impact of quantization. A natural alternative is to adopt output-driven criteria that minimize discrepancies in model outputs using calibration data. Surprisingly, naive output-driven approaches often perform even worse in the 1-bit regime. In this paper, we show that this failure arises from two fundamental issues: error accumulation across layers and, more critically, anisotropic distortion of the representation space. Based on this observation, we introduce a geometry-preserving output-
alignment framework for 1-bit LLM quantization. First, we formulate output
alignment as a constrained optimization problem that improves reconstruction while explicitly controlling changes in
token similarity, preventing reconstruction gains from being achieved at the expense of representation geometry. Second, we propose a geometry-aware salient-mask selection strategy that identifies quantization configurations according to their ability to preserve
token relations, rather than relying solely on reconstruction error. These components enable effective output-driven optimization while retaining the structural information lost by naive
alignment. Extensive experiments across multiple LLM families and downstream tasks demonstrate consistent improvements over existing 1-bit PTQ methods.
Read the full abstract ▾
All authors Dung Anh Hoang, Cuong Pham, Cuong Nguyen, Trung le, Jianfei Cai, Thanh-Toan Do
Machine Learning
Physic-HM: Restoring Physical Generative Logic in Multimodal Anomaly Detection via Hierarchical Modulation
Xiao Liu, Junchen Jin, Yanjie Zhao et al. (+1 more)
Multimodal Unsupervised Anomaly Detection (UAD) is critical for quality assurance in smart manufacturing, particularly in complex processes like robotic welding. However, existing methods often suffer from process-logic blindness, treating process modalities (e.g., real-time
video, audio, and sensors) and result modalities (e.g., post-weld images) as symmetric feature sources, thereby ignoring the inherent unidirectional physical generative logic. Furthermore, the heterogeneity gap between high-dimensional visual data and low-dimensional sensor signals frequently leads to critical process context being drowned out. In this paper, we propose Physic-HM, a
multimodal UAD framework that explicitly incorporates physical inductive bias to model the process-to-result dependency. Specifically, our framework incorporates two key innovations: a Sensor-Guided PHM Modulation mechanism that utilizes low-dimensional sensor signals as context to guide high-dimensional audio-visual feature extraction, and a Physic-Hierarchical architecture that enforces a unidirectional generative mapping to identify anomalies that violate physical consistency. Extensive experiments on Weld-4M
benchmark demonstrate that Physic-HM achieves a SOTA I-AUROC of 90.7%. The source code of Physic-HM will be released after the paper is accepted.
Read the full abstract ▾
All authors Xiao Liu, Junchen Jin, Yanjie Zhao, Zhixuan Xing
Machine Learning
Variance-Aware Prior-Based Tree Policies for Monte Carlo Tree Search
Maximilian Weichart
Monte Carlo Tree Search (MCTS) has profoundly influenced reinforcement learning (RL) by integrating planning and learning in tasks requiring long-horizon
reasoning, exemplified by the AlphaZero family of
algorithms. Central to MCTS is the search strategy, governed by a tree policy based on an upper confidence bound (UCB) applied to trees (UCT). A key factor in the success of AlphaZero is the introduction of a prior term in the UCB1-based tree policy PUCT, which improves exploration efficiency and thus accelerates training. While many alternative UCBs with stronger theoretical guarantees than UCB1 exist, extending them to prior-based UCTs has been challenging, since PUCT was derived empirically rather than from first principles. Recent work retrospectively justified PUCT by framing MCTS as a regularized policy optimization (RPO) problem. Building on this perspective, we introduce Inverse-RPO, a general methodology that systematically derives prior-based UCTs from a broad class of prior-free UCBs. Applying this method to the variance-aware UCB-V, we obtain two new prior-based tree policies that incorporate variance estimates into the search. Experiments indicate that these variance-aware prior-based UCTs outperform PUCT across multiple
benchmarks without incurring additional computational cost. We also provide an extension of the mctx library supporting variance-aware UCTs, showing that the required code changes are minimal and intended to facilitate further research on principled prior-based UCTs. Code: github.com/Max-We/inverse-rpo.
Read the full abstract ▾
Machine Learning
Mechanical Strength Prediction of Steel-Polypropylene Fiber-based High-Performance Concrete Using Hybrid Machine Learning Algorithms
Jagaran Chakma, Zhiguang Zhou and Badhan Chakma
This research develops and evaluates machine learning models to predict the mechanical properties of steel-polypropylene fiber-reinforced high-performance concrete (HPC). Three model families were investigated: Extra Trees with XGBoost (ET-XGB), Random Forest with LightGBM (RF-LGBM), and
Transformer with XGBoost (
Transformer-XGB). The target properties included compressive strength (CS), flexural strength (FS), and tensile strength (TS), based on an extensive
dataset compiled from published experimental studies. Model training involved k-fold cross-validation, hyperparameter optimization, Shapley additive explanations (SHAP), and uncertainty analysis to ensure both robustness and interpretability. Among the tested approaches, the ET-XGB model achieved the highest overall accuracy, with testing R^2 values of 0.994 for CS, 0.944 for FS, and 0.978 for TS and exhibited lowest uncertainty for CS and TS (approximately 13-16% and 30.4%, respectively). The RF-LGBM model provided the most stable and reliable predictions for FS (R^2 0.977), yielding the lowest uncertainty for FS (approximately 5-33%). The
Transformer-XGB model demonstrated strong predictive capability (R^2 0.978 for TS and 0.967 for FS) but consistently showed the highest uncertainty, indicating reduced generalization reliability. SHAP analysis further indicated that fiber aspect ratios (AR1 and AR2), silica fume (Sfu), and steel fiber content (SF) were the most influential predictors of strength, whereas water content (W) and the water-binder ratio (w/b) consistently had negative effects. The findings confirm that machine learning models can provide accurate, interpretable, and generalizable predictions of HPC mechanical properties. These models offer valuable tools for optimizing concrete mix design and enhancing structural performance evaluation in engineering applications.
Read the full abstract ▾
Machine Learning
MAD-NG: Meta-Auto-Decoder Neural Galerkin Method for Solving Parametric Partial Differential Equations
Qiuqi Li, Yiting Liu, Jin Zhao et al. (+1 more)
Parametric partial differential equations (PDEs) are fundamental for modeling a wide range of physical and engineering systems influenced by uncertain or varying parameters. Traditional neural network-based solvers, such as Physics-Informed Neural Networks (PINNs) and Deep Galerkin Methods, often face challenges in generalization and long-time prediction efficiency due to their dependence on full space-time approximations. To address these issues, we propose a novel and scalable framework that significantly enhances the Neural Galerkin Method (NGM) by incorporating the Meta-Auto-Decoder (MAD) paradigm. Our approach leverages space-time decoupling to enable more stable and efficient time integration, while meta-learning-driven adaptation allows rapid generalization to unseen parameter configurations with minimal retraining. Furthermore, randomized sparse updates effectively reduce computational costs without compromising accuracy. Together, these advancements enable our method to achieve physically consistent, long-horizon predictions for complex parameterized
evolution equations with significantly lower computational overhead. Numerical experiments on
benchmark problems demonstrate that our methods performs comparatively well in terms of accuracy, robustness, and adaptability.
Read the full abstract ▾
All authors Qiuqi Li, Yiting Liu, Jin Zhao, Wencan Zhu
Machine Learning
A Data-Driven Multi-Objective Approach for Predicting Mechanical Performance, Flowability, and Porosity in Ultra-High-Performance Concrete (UHPC)
Jagaran Chakma, Zhiguang Zhou, Jyoti Chakma et al. (+1 more)
This study presents a data-driven, multi-objective approach to predict the mechanical performance, flow ability, and porosity of Ultra-High-Performance Concrete (UHPC). Out of 21 machine learning
algorithms tested, five high-performing models are selected, with XGBoost showing the best accuracy after hyperparameter tuning using Random Search and K-Fold Cross-Validation. The framework follows a two-stage process: the initial XGBoost model is built using raw data, and once selected as the final model, the
dataset is cleaned by (1) removing multicollinear features, (2) identifying outliers with Isolation Forest, and (3) selecting important features using SHAP analysis. The refined
dataset as model 2 is then used to retrain XGBoost, which achieves high prediction accuracy across all outputs. A graphical user interface (GUI) is also developed to support material designers. Overall, the proposed framework significantly improves the prediction accuracy and minimizes the need for extensive experimental testing in UHPC mix design.
Read the full abstract ▾
All authors Jagaran Chakma, Zhiguang Zhou, Jyoti Chakma, Cao YuSen
Machine Learning
An Empirical Study of Machine Learning Robustness and Scalability for Imbalanced Tabular Clinical Data in Emergency and Critical Care
Yusuf Brima and Marcellin Atemkeng
Every year, millions of patients pass through emergency departments and intensive care units, where clinicians must make high-stakes decisions under time pressure and uncertainty. Machine learning could support prediction of deterioration, triage, and rare critical outcomes, but clinical data are often severely imbalanced, biasing models toward majority classes and reducing predictive performance. Developing robust and efficient models for imbalanced clinical tabular data therefore remains an important challenge. We evaluated six model families on imbalanced tabular data from the MIMIC-IV-ED and eICU databases: Decision Tree, Random Forest, XGBoost, TabNet, TabICL, and TabPFN v2.6. Trainable models were optimized using
Bayesian hyperparameter tuning, while foundation models were evaluated in their pretrained inference regime without task-specific reweighting. Models were assessed using Macro F1-score, robustness to increasing imbalance, and computational scalability across seven clinical prediction tasks. Results differed across
datasets. On MIMIC-IV-ED, TabPFN v2.6 and TabICL achieved the strongest average Macro F1 ranks, with XGBoost remaining competitive. On eICU, XGBoost consistently performed best, followed by other tree-based methods, while foundation models achieved intermediate performance. Across both
datasets, TabNet showed the largest degradation under increasing imbalance and the highest computational cost. Training-time analysis showed that tree-based methods scaled most favorably with
dataset size, while foundation models offered low per-task adaptation cost. These findings suggest that no single model family dominates across all clinical settings. However, tabular foundation models are narrowing the performance gap with strong classical baselines while offering a distinct efficiency-performance trade-off that may benefit resource-constrained clinical environments.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.PL
Quantitative Verification of Omega-regular Properties in Probabilistic Programming
Peixin Wang, Jianhao Bai, Min Zhang et al. (+1 more)
Probabilistic programming provides a high-level framework for specifying statistical models as executable programs with built-in randomness and conditioning. Existing inference techniques, however, typically compute posterior distributions over program states at fixed time points, most often at termination, thereby failing to capture the temporal
evolution of probabilistic behaviors. We introduce temporal posterior inference (TPI), a new framework that unifies probabilistic programming with temporal logic by computing posterior distributions over execution traces that satisfy omega-regular specifications, conditioned on possibly temporal observations. To obtain rigorous quantitative guarantees, we develop a new method for computing upper and lower bounds on the satisfaction probabilities of omega-regular properties. Our approach decomposes Rabin acceptance conditions into persistence and recurrence components and constructs
stochastic barrier certificates that soundly bound each component. We implement our approach in a prototype tool, TPInfer, and evaluate it on a suite of
benchmarks, demonstrating effective and efficient inference over rich temporal properties in probabilistic models.
Read the full abstract ▾
All authors Peixin Wang, Jianhao Bai, Min Zhang, C.-H. Luke Ong
Machine Learning · Cross-listed from stat.ML
Residual Prior Diffusion: A Probabilistic Framework Integrating Coarse Latent Priors with Diffusion Models
Takuro Kutsuna
Diffusion models have become a central tool in deep generative modeling, but standard formulations rely on a single network and a single
diffusion schedule to transform a simple prior, typically a standard normal distribution, into the target data distribution. As a result, the model must simultaneously represent the global structure of the distribution and its fine-scale local variations, which becomes difficult when these scales are strongly mismatched. This issue arises both in natural images, where coarse
manifold-level structure and fine textures coexist, and in low-dimensional distributions with highly concentrated local structure. To address this issue, we propose Residual Prior
Diffusion (RPD), a two-stage framework in which a coarse prior model first captures the large-scale structure of the data distribution, and a
diffusion model is then trained to represent the residual between the prior and the target data distribution. We formulate RPD as an explicit probabilistic model with a tractable evidence lower bound, whose optimization reduces to the familiar objectives of noise prediction or velocity prediction. We further introduce auxiliary variables that leverage information from the prior model and theoretically analyze how they reduce the difficulty of the prediction problem in RPD. Experiments on synthetic
datasets with fine-grained local structure show that standard
diffusion models fail to capture local details, whereas RPD accurately captures fine-scale detail while preserving the large-scale structure of the distribution. On natural image generation tasks, RPD achieved generation quality that matched or exceeded that of representative
diffusion-based baselines and it maintained strong performance even with a small number of inference steps.
Read the full abstract ▾
Machine Learning · Cross-listed from physics.optics
Incorporating rank-free coupling and external field via an incoherent modulated spatial photonic Ising machine
Ze Zheng, Yuegang Li, Hang Xu et al. (+3 more)
Spatial photonic Ising machines offer a novel optical platform for optimization and
spin-model simulation, but existing diffraction-based schemes rely on auxiliary
spins or multiplexing to encode high-rank couplings and external fields, reducing either speed or
spin count. We demonstrate an amplitude-only, rank-free spatial photonic Ising machine in which arbitrary Ising
Hamiltonians are encoded as Hadamard products on aligned amplitude and binary spatial modulators and read out by a single-pixel intensity measurement. The machine directly programs fully connected 797-
spin Ising models with external fields at nearly 9-bit precision and operates at a constant iteration rate of ~200 Hz. By removing zero-valued product terms, the same architecture scales to sparse problems and experimentally solves a Max-Cut instance on a 424,108-vertex Mobius ladder graph. We also observe the phase transition of the Sherrington-Kirkpatrick model, demonstrating programmable optical simulation beyond low-rank couplings. These results establish amplitude modulation as a scalable route to programmable photonic Ising machines.
Read the full abstract ▾
All authors Ze Zheng, Yuegang Li, Hang Xu, Jingzheng Huang, Tailong Xiao, Guihua Zeng
Machine Learning
Videos are Sample-Efficient Supervisions: Behavior Cloning from Videos via Latent Representations
Xin Liu, Haoran Li and Dongbin Zhao
Humans can efficiently extract knowledge and learn skills from the
videos within only a few trials and errors. However, it poses a big challenge to replicate this learning process for autonomous
agents, due to the complexity of visual input, the absence of action or reward signals, and the limitations of interaction steps. In this paper, we propose a novel, unsupervised, and sample-efficient framework to achieve imitation learning from
videos (ILV), named Behavior Cloning from
Videos via Latent Representations (BCV-LR). BCV-LR extracts action-related latent features from high-dimensional
video inputs through self-supervised tasks, and then leverages a dynamics-based unsupervised objective to predict latent actions between consecutive frames. The pre-trained latent actions are fine-tuned and efficiently aligned to the real action space online (with collected interactions) for policy behavior cloning. The cloned policy in turn enriches the
agent experience for further latent action finetuning, resulting in an iterative policy improvement that is highly sample-efficient. We conduct extensive experiments on a set of challenging visual tasks, including both discrete control and continuous control. BCV-LR enables effective (even expert-level on some tasks) policy performance with only a few interactions, surpassing state-of-the-art ILV baselines and reinforcement learning methods (provided with environmental rewards) in terms of sample efficiency across 24/28 tasks. To the best of our knowledge, this work for the first time demonstrates that
videos can support extremely sample-efficient visual policy learning, without the need to access any other expert supervision.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
A Unified Definition of Hallucination: It's The World Model, Stupid!
Emmy Liu, Varun Gangal, Chelsea Zou et al. (+7 more)
Despite numerous attempts at mitigation since the inception of
language models,
hallucinations remain a persistent problem even in today's frontier LLMs. Why is this? We review existing definitions of
hallucination and fold them into a single, unified definition wherein prior definitions are subsumed. We argue that
hallucination can be unified by defining it as simply inaccurate (internal) world modeling, in a form where it is observable to the user. For example, stating a fact which contradicts a knowledge base OR producing a summary which contradicts the source. By varying the reference world model and conflict policy, our framework unifies prior definitions. We argue that this unified view is useful because it forces evaluations to clarify their assumed reference "world", distinguishes true
hallucinations from planning or reward errors, and provides a common
language for comparison across
benchmarks and discussion of mitigation strategies. Building on this definition, we also connect our framework to HalluWorld, a complementary
benchmark that instantiates fully specified reference world models for stress-testing model
hallucinations.
Read the full abstract ▾
All authors Emmy Liu, Varun Gangal, Chelsea Zou, Michael Yu, Xiaoqi Huang, Alex Chang, Zhuofu Tao, Karan Singh, Sachin Kumar, Steven Y. Feng
Machine Learning · Cross-listed from cs.CV
Haoyi Zhou, Shuo Li, Tianyu Chen et al. (+3 more)
While large
vision-
language models (VLMs) demonstrate strong long-context understanding, their prevalent small branches fail on linguistics-photography
alignment for a limited window size. We discover that knowledge
distillation improves students' capability as a complement to Rotary Position
Embeddings (RoPE) on window sizes (anchored from large models). Building on this insight, we propose LAid, which directly aims at the transfer of long-range
attention mechanisms through two complementary components: (1) a progressive distance-weighted
attention matching that dynamically emphasizes longer position differences during training, and (2) a learnable RoPE response gain modulation that selectively amplifies position sensitivity where needed. Extensive experiments across multiple model families demonstrate that LAid-distilled models achieve up to 3.2 times longer effective context windows compared to baseline small models, while maintaining or improving performance on standard VL
benchmarks. Spectral analysis also suggests that LAid successfully preserves crucial low-frequency
attention components that conventional methods fail to transfer. Our work not only provides practical techniques for building more efficient long-context VLMs but also offers theoretical insights into how positional understanding emerges and transfers during
distillation.
Read the full abstract ▾
All authors Haoyi Zhou, Shuo Li, Tianyu Chen, Qi Song, Chonghan Gao, Jianxin Li
Machine Learning
RefineBridge: Generative Bridge Models Improve Financial Forecasting by Foundation Models
Anthony Bolton, Wuyang Zhou, Zehua Chen et al. (+2 more)
Financial time series
forecasting is particularly challenging for
transformer-based time series foundation models (TSFMs) due to non-stationarity, heavy-tailed distributions, and high-frequency noise present in data. Low-rank adaptation (LoRA) has become a popular parameter-efficient method for adapting pre-trained TSFMs to downstream data domains. However, it still underperforms in financial data, as it preserves the network architecture and training objective of TSFMs rather than complementing the foundation model. To further enhance TSFMs, we propose a novel refinement module, RefineBridge, built upon a tractable Schrödinger Bridge (SB) generative framework. Given the forecasts of TSFM as generative prior and the observed ground truths as targets, RefineBridge learns context-conditioned
stochastic transport maps to improve TSFM predictions, iteratively approaching the ground-truth target from even a low-quality prior. Simulations on multiple financial
benchmarks demonstrate that RefineBridge consistently improves the performance of state-of-the-art TSFMs across different prediction horizons.
Read the full abstract ▾
All authors Anthony Bolton, Wuyang Zhou, Zehua Chen, Giorgos Iacovides, Danilo Mandic
Machine Learning · Cross-listed from cs.DC
nncase: An End-to-End Compiler for Efficient LLM Deployment on Heterogeneous Storage Architectures
Hui Guo, Qihang Zheng, Chenghai Huo et al. (+3 more)
The efficient deployment of large
language models (LLMs) is hindered by
memory architecture heterogeneity, where traditional compilers suffer from fragmented workflows and high adaptation costs. We present nncase, an open-source, end-to-end compilation framework designed to unify optimization across diverse targets. Central to nncase is an e-graph-based term rewriting engine that mitigates the phase ordering problem, enabling global exploration of computation and data movement strategies. The framework integrates three key modules: Auto Vectorize for adapting to heterogeneous computing units, Auto Distribution for searching parallel strategies with cost-aware communication optimization, and Auto Schedule for maximizing on-chip cache locality. Furthermore, a buffer-aware Codegen phase ensures efficient kernel instantiation. Evaluations show that nncase outperforms mainstream frameworks like MLC LLM and Intel IPEX on Qwen3 series models and achieves performance comparable to the hand-optimized llama.cpp on CPUs, demonstrating the viability of automated compilation for high-performance LLM deployment. The source code is available at https://github.com/kendryte/nncase.
Read the full abstract ▾
All authors Hui Guo, Qihang Zheng, Chenghai Huo, Dongliang Guo, Haoqi Yang, Yang Zhang
Machine Learning
AnchorGK: Anchor-based Incremental and Stratified Graph Learning Framework for Inductive Spatio-Temporal Kriging
Xiaobin Ren, Kaiqi Zhao, Katerina Taškova et al. (+1 more)
Spatio-temporal kriging is a fundamental problem in sensor networks, driven by the sparsity of deployed sensors and the resulting missing observations. Although recent approaches model spatial and temporal correlations, they often under-exploit two practical characteristics of real deployments: the sparse spatial distribution of locations and the heterogeneous availability of auxiliary features across locations. To address these challenges, we propose AnchorGK, an Anchor-based Incremental and Stratified Graph Learning framework for inductive spatio-temporal kriging. AnchorGK introduces anchor locations to stratify the data in a principled manner. Anchors are constructed according to feature availability, and strata are then formed around these anchors. This stratification serves two complementary roles. First, it explicitly represents and continuously updates correlations between unobserved regions and surrounding observed locations within a graph learning framework. Second, it enables the systematic use of all available features across strata via an incremental representation mechanism, mitigating feature incompleteness without discarding informative signals. Building on the stratified structure, we design a dual-view graph learning layer that jointly aggregates feature-relevant and location-relevant information, learning stratum-specific representations that support accurate inference under inductive settings. Extensive experiments on multiple
benchmark datasets demonstrate that AnchorGK consistently outperforms state-of-the-art baselines for spatio-temporal kriging.
Read the full abstract ▾
All authors Xiaobin Ren, Kaiqi Zhao, Katerina Taškova, Patricia Riddle
Machine Learning
Discovering Sparse Recovery Algorithms Using Neural Architecture Search
Patrick Yubeaton, Sarthak Gupta, M. Salman Asif et al. (+1 more)
The design of novel
algorithms for solving inverse problems in signal processing is an incredibly difficult, heuristic-driven, and time-consuming task. In this short paper, we the idea of automated
algorithm discovery in the signal processing context through meta-learning tools such as Neural Architecture Search (NAS). Specifically, we examine the Iterative Shrinkage Thresholding
Algorithm (ISTA) and its accelerated Fast ISTA (FISTA) variant as candidates for
algorithm rediscovery. We develop a meta-learning framework which is capable of rediscovering (several key elements of) the two aforementioned
algorithms when given a search space of over 50,000 variables. We then show how our framework can apply to various data distributions and
algorithms besides ISTA/FISTA.
Read the full abstract ▾
All authors Patrick Yubeaton, Sarthak Gupta, M. Salman Asif, Chinmay Hegde
Machine Learning
AVP-Fusion: Adaptive Multi-Modal Fusion and Contrastive Learning for Two-Stage Antiviral Peptide Identification
Xinru Wen, Weizhong Lin and Xuan Xiao
Accurate identification of antiviral peptides (AVPs) is critical for accelerating novel drug development. However, current computational methods struggle to capture intricate sequence dependencies and effectively handle ambiguous, hard-to-classify samples. To address these challenges, we propose AVP-Fusion, a novel two-stage deep learning framework integrating adaptive feature fusion and contrastive learning. Unlike traditional static feature concatenation, we construct a panoramic feature space using 10 distinct descriptors and introduce an Adaptive Gating Mechanism.This mechanism dynamically regulates the weights of local motifs extracted by CNNs and global dependencies captured by BiLSTMs based on sequence context. Furthermore, to address data distribution challenges, we employ a contrastive learning strategy driven by Online Hard Example Mining (OHEM) and BLOSUM62-based data augmentation, which significantly sharpens the model's decision boundaries. Experimental results on the
benchmark Set 1
dataset demonstrate that AVP-Fusion achieves an accuracy of 0.9531 and an MCC of 0.9064, significantly outperforming state-of-the-art methods. In the second stage, leveraging transfer learning, the model enables precise subclass prediction for six viral families and eight specific viruses, even under limited sample sizes. In summary, AVP-Fusion serves as a robust and interpretable tool for high-throughput antiviral drug screening.
Read the full abstract ▾
Machine Learning
Generative Actor Critic
Aoyang Qin, Deqian Kong, Wei Wang et al. (+3 more)
Conventional Reinforcement Learning (RL)
algorithms, typically focused on estimating or maximizing expected returns, face challenges when refining offline pretrained models with online experiences. This paper introduces Generative Actor Critic (GAC), a novel framework that decouples sequential decision-making by reframing policy evaluation as learning a generative model of the joint distribution over trajectories and returns, $p(\tau, y)$, and policy improvement as performing versatile inference on this learned model. To operationalize GAC, we introduce a specific instantiation based on a latent variable model that features continuous latent plan vectors. We develop novel inference strategies for both exploitation, by optimizing latent plans to maximize expected returns, and exploration, by sampling latent plans conditioned on dynamically adjusted target returns. Experiments on Gym-MuJoCo and Maze2D
benchmarks demonstrate GAC's strong offline performance and significantly enhanced offline-to-online improvement compared to state-of-the-art methods, even in absence of step-wise rewards.
Read the full abstract ▾
All authors Aoyang Qin, Deqian Kong, Wei Wang, Ying Nian Wu, Song-Chun Zhu, Sirui Xie
Machine Learning
First Provable Guarantees for Practical Private FL: Beyond Restrictive Assumptions
Egor Shulgin, Grigory Malinovsky, Sarit Khirirat et al. (+1 more)
Federated Learning (FL) enables collaborative training on decentralized data. Differential
privacy (DP) is crucial for FL, but current private methods often rely on unrealistic assumptions (e.g., bounded gradients or heterogeneity), hindering practical application. Existing works that relax these assumptions typically neglect practical FL features, including multiple local updates and partial client participation. We introduce Fed-$\alpha$-NormEC, the first differentially private FL framework providing provable convergence and DP guarantees under standard assumptions while fully supporting these practical features. Fed-$\alpha$-NormE integrates local updates (full and incremental gradient steps), separate server and client stepsizes, and, crucially, partial client participation, which is essential for real-world deployment and vital for
privacy amplification. Our theoretical guarantees are corroborated by experiments on private deep learning tasks.
Read the full abstract ▾
All authors Egor Shulgin, Grigory Malinovsky, Sarit Khirirat, Peter Richtárik
Machine Learning
Global-Graph Guided and Local-Graph Weighted Contrastive Learning for Unified Clustering on Incomplete and Noise Multi-View Data
Hongqing He, Jie Xu, Wenyuan Yang et al. (+3 more)
Recently, contrastive learning (CL) plays an important role in exploring complementary information for multi-view clustering (MVC) and has attracted increasing
attention. Nevertheless, real-world multi-view data suffer from data incompleteness or noise, resulting in rare-paired samples or mis-paired samples which significantly challenges the effectiveness of CL-based MVC. That is, rare-paired issue prevents MVC from extracting sufficient multi-view complementary information, and mis-paired issue causes contrastive learning to optimize the model in the wrong direction. To address these issues, we propose a unified CL-based MVC framework for enhancing clustering effectiveness on incomplete and noise multi-view data. First, to overcome the rare-paired issue, we design a global-graph guided contrastive learning, where all view samples construct a global-view affinity graph to form new sample pairs for fully exploring complementary information. Second, to mitigate the mis-paired issue, we propose a local-graph weighted contrastive learning, which leverages local neighbors to generate pair-wise weights to adaptively strength or weaken the pair-wise contrastive learning. Our method is imputation-free and can be integrated into a unified global-local graph-guided contrastive learning framework. Extensive experiments on both incomplete and noise settings of multi-view data demonstrate that our method achieves superior performance compared with state-of-the-art approaches.
Read the full abstract ▾
All authors Hongqing He, Jie Xu, Wenyuan Yang, Yonghua Zhu, Guoqiu Wen, Xiaofeng Zhu
Machine Learning
Perplexity-Aware Data Scaling Law: Perplexity Landscapes Predict Performance for Continual Pre-training
Lei Liu, Hao Zhu, Yue Shen et al. (+4 more)
Continual Pre-training (CPT) serves as a fundamental approach for adapting foundation models to domain-specific applications. Scaling laws for pre-training define a power-law relationship between
dataset size and the test loss of an LLM. However, the marginal gains from simply increasing data for CPT diminish rapidly, yielding suboptimal data utilization and inefficient training. To address this challenge, we propose a novel perplexity-aware data scaling law to establish a predictive relationship between the perplexity landscape of domain-specific data and the test loss. Our approach leverages the perplexity derived from the pre-trained model on domain data as a proxy for estimating the knowledge gap, effectively quantifying the informational perplexity landscape of candidate training samples. By fitting this scaling law across diverse perplexity regimes, we enable adaptive selection of high-utility data subsets, prioritizing content that maximizes knowledge absorption while minimizing redundancy and noise. Extensive experiments demonstrate that our method consistently identifies near-optimal training subsets and achieves superior performance on both medical and general-domain
benchmarks.
Read the full abstract ▾
All authors Lei Liu, Hao Zhu, Yue Shen, Zhixuan Chu, Jian Wang, Jinjie Gu, Kui Ren
Machine Learning
Missing Pattern Tree based Decision Grouping and Ensemble for Enhancing Pair Utilization in Deep Incomplete Multi-View Clustering
Jie Xu, Wenyuan Yang, Yazhou Ren et al. (+3 more)
Real-world multi-view data often exhibit highly inconsistent missing patterns, posing significant challenges for incomplete multi-view clustering (IMVC). Although existing IMVC methods have made progress from both imputation-based and imputation-free routes, they largely overlook the issue of pair underutilization. Specifically, inconsistent missing patterns prevent incomplete but available multi-view pairs from being fully exploited, thereby limiting the model performance. To address this limitation, we propose a novel missing-pattern tree based IMVC framework. Specifically, to fully leverage available multi-view pairs, we first introduce a missing-pattern tree model to group data into multiple decision sets according to their missing patterns, and then perform multi-view clustering within each set. Furthermore, a multi-view decision ensemble module is proposed to aggregate clustering results across all decision sets. This module infers uncertainty-based weights to suppress unreliable clustering decisions and produce robust outputs. Finally, we develop an ensemble-to-individual knowledge
distillation module module, which transfers ensemble knowledge to view-specific clustering models. This design enables mutual enhancement between ensemble and individual modules by optimizing cross-view consistency and inter-cluster discrimination losses. Extensive theoretical analysis supports our key designs, and empirical experiments on multiple
benchmark datasets demonstrate that our method effectively mitigates the pair underutilization issue and achieve superior IMVC performance.
Read the full abstract ▾
All authors Jie Xu, Wenyuan Yang, Yazhou Ren, Lifang He, Philip S. Yu, Xiaofeng Zhu
Machine Learning
MotionTeller: Multi-modal Integration of Wearable Time-Series with LLMs for Health and Behavioral Understanding
Aiwei Zhang, Arvind Pillai, Andrew Campbell et al. (+1 more)
As wearable sensing becomes increasingly pervasive, a key challenge remains: how can we generate natural
language summaries from raw physiological signals such as actigraphy - minute-level movement data collected via accelerometers? In this work, we introduce MotionTeller, a generative framework that natively integrates minute-level wearable activity data with large
language models (LLMs). MotionTeller combines a pretrained actigraphy encoder with a lightweight projection module that maps behavioral
embeddings into the
token space of a frozen decoder-only LLM, enabling free-text, autoregressive generation of daily behavioral summaries. We construct a novel
dataset of 54383 (actigraphy, text) pairs derived from real-world NHANES recordings, and train the model using cross-
entropy loss with supervision only on the
language tokens. MotionTeller achieves high semantic fidelity (BERTScore-F1 = 0.924) and lexical accuracy (ROUGE-1 = 0.722), outperforming
prompt-based baselines by 7 percent in ROUGE-1. The average training loss converges to 0.38 by epoch 15, indicating stable optimization. Qualitative analysis confirms that MotionTeller captures circadian structure and behavioral transitions, while PCA plots reveal enhanced cluster
alignment in
embedding space post-training. Together, these results position MotionTeller as a scalable, interpretable system for transforming wearable sensor data into fluent, human-centered descriptions, introducing new pathways for behavioral monitoring, clinical review, and personalized health interventions.
Read the full abstract ▾
All authors Aiwei Zhang, Arvind Pillai, Andrew Campbell, Nicholas C. Jacobson
Machine Learning
When Bayesian Tensor Completion Meets Multioutput Gaussian Processes: Functional Universality and Rank Learning
Siyuan Li, Shikai Fang, Lei Cheng et al. (+4 more)
Functional tensor decomposition can analyze multi-dimensional data with real-valued indices, paving the path for applications in machine learning and signal processing. A limitation of existing approaches is the assumption that the tensor rank-a critical parameter governing model complexity-is known. However, determining the optimal rank is a non-deterministic
polynomial-time hard (NP-hard) task and there is a limited understanding regarding the expressive power of functional low-rank tensor models for continuous signals. We propose a rank-revealing functional
Bayesian tensor completion (RR-FBTC) method. Modeling the latent functions through carefully designed multioutput Gaussian processes, RR-FBTC handles tensors with real-valued indices while enabling automatic tensor rank determination during the inference process. We establish the universal approximation property of the model for continuous multi-dimensional signals, demonstrating its expressive power in a concise format. To learn this model, we employ the variational inference framework and derive an efficient
algorithm with closed-form updates. Experiments on both synthetic and real-world
datasets demonstrate the effectiveness and superiority of the RR-FBTC over state-of-the-art approaches. The code is available at https://github.com/OceanSTARLab/RR-FBTC.
Read the full abstract ▾
All authors Siyuan Li, Shikai Fang, Lei Cheng, Feng Yin, Yik-Chung Wu, Peter Gerstoft, Sergios Theodoridis
Machine Learning · Cross-listed from quant-ph
Quantum Nondecimated Wavelet Transform: Theory, Circuits, and Applications
Brani Vidakovic
The
quantum nondecimated wavelet transform uses a shift register to apply circular shifts to an amplitude encoded signal before a single orthogonal wavelet transform. We show that tracing out the shift register gives level probabilities equal to the normalized nondecimated wavelet energies, invariant under circular shifts. Its diagonal gives moving averages of squared coefficients. We derive a multinomial model for the level counts and show how to read the autocovariance at lag one for all but the coarsest detail level. For these global statistics, direct
Fourier basis measurement of the input is at least as precise and uses a simpler
circuit. For denoising, we insert a block encoded shrinkage operator between the transform and its inverse. Postselection on the shift register implements the cycle spinning average exactly as a linear combination of
unitaries. Weights shared within each level give a translation equivariant filter with zero phase. Local weights also preserve translation equivariance when indexed by the nondecimated translate. The
circuit provides access to individual decimated branches and combines them coherently. We illustrate the method with James Stein shrinkage, local empirical Wiener filtering, and a scaling analysis of turbulent wind velocity. Gate simulations verify the construction, and we report state preparation, weight estimation, and output costs separately.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
CCAD: Compressed Global Feature Conditioned Anomaly Detection
Xiao Jin, Liang Diao, Qixin Xiao et al. (+4 more)
Anomaly detection holds considerable industrial significance, especially in scenarios with limited anomalous data. Currently, reconstruction-based and unsupervised representation-based approaches are the primary focus. However, unsupervised representation-based methods struggle to extract robust features under domain shift, whereas reconstruction-based methods often suffer from low training efficiency and performance degradation due to insufficient constraints. To address these challenges, we propose a novel method named Compressed Global Feature Conditioned Anomaly Detection (CCAD). CCAD synergizes the strengths of both paradigms by adapting global features as a new modality condition for the reconstruction model. Furthermore, we design an adaptive compression mechanism to enhance both generalization and training efficiency. Extensive experiments demonstrate that CCAD consistently outperforms state-of-the-art methods in terms of AUC while achieving faster convergence. In addition, we contribute a reorganized and re-annotated version of the DAGM 2007
dataset with new annotations to further validate our method's effectiveness. The code for reproducing main results is available at https://github.com/chloeqxq/CCAD.
Read the full abstract ▾
All authors Xiao Jin, Liang Diao, Qixin Xiao, Yifan Hu, Ziqi Zhang, Yuchen Liu, Haisong Gu
Machine Learning
Statistical vs. Deep Learning Models for Estimating Substance Overdose Excess Mortality in the US
Sukanya Krishna, Marie-Laure Charpignon and Maimuna Majumder
Substance overdose mortality in the United States claimed over 80,000 lives in 2023, with the COVID-19 pandemic exacerbating existing trends through healthcare disruptions and behavioral changes. Estimating excess mortality, defined as deaths beyond expected levels based on pre-pandemic patterns, is essential for understanding pandemic impacts and informing intervention strategies. However, traditional statistical methods like SARIMA assume linearity, stationarity, and fixed seasonality, which may not hold under structural disruptions. We present a systematic comparison of SARIMA against three deep learning (DL) architectures (LSTM, Seq2Seq, and
Transformer) for counterfactual mortality estimation using national CDC data (2015-2019 for training/validation, 2020-2023 for projection). We contribute empirical evidence that LSTM achieves superior point estimation (17.08% MAPE vs. 23.88% for SARIMA) and better-calibrated uncertainty (68.8% vs. 47.9% prediction interval coverage) when projecting under regime change. We also demonstrate that
attention-based models (Seq2Seq,
Transformer) underperform due to overfitting to historical means rather than capturing emergent trends. Ourreproducible pipeline incorporates conformal prediction intervals and convergence analysis across 60+ trials per configuration, and we provide an open-source framework deployable with 15 state health departments. Our findings establish that carefully validated DL models can provide more reliable counterfactual estimates than traditional methods for public health planning, while highlighting the need for calibration techniques when deploying neural
forecasting in high-stakes domains.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
An approach to Fisher-Rao metric for infinite dimensional non-parametric information geometry
Bing Cheng and Howell Tong
Being infinite dimensional, non-parametric information geometry has long faced an "intractability barrier" due to the fact that the Fisher-Rao metric is now a functional incurring difficulties in defining its inverse. This paper introduces a novel framework to resolve the intractability with an Orthogonal Decomposition of the Tangent Space ($T_fM = S \oplus S^{\perp}$), where $S$ represents an observable
covariate subspace. Through the decomposition, we derive the
Covariate Fisher Information
Matrix (cFIM), denoted as ${\bf G}_f$, which is a finite-dimensional and computable representative of information extractable from the
manifold's geometry. Significantly, by proving the Trace
Theorem: $H_G(f) = \text{Tr}({\bf G}_f)$, we establish a rigorous foundation for the G-
entropy previously introduced by us, thereby identifying it as a fundamental
geometric invariant representing the total explainable statistical information captured by the probability distribution associated with a model. Furthermore, we establish a link between ${\bf G}_f$ and the second derivative (i.e. the
curvature) of the KL-divergence, leading to the notion of
Covariate Cramér-Rao Lower Bound(CRLB). We demonstrate that ${\bf G}_f$ is congruent to the Efficient Fisher Information
Matrix, thereby providing fundamental limits of variance for semi-parametric
estimators. Finally, we apply our
geometric framework to the
Manifold Hypothesis, lifting the latter from a heuristic assumption into a testable condition of rank-deficiency within the cFIM. By defining the Information Capture Ratio, we provide a rigorous method for estimating intrinsic dimensionality in high-dimensional data. In short, our work bridges the gap between abstract information geometry and the demand of explainable AI, by providing a tractable path for assessing the statistical coverage and the efficiency of non-parametric models.
Read the full abstract ▾
Machine Learning
RLLaVA: An RL-central Framework for Language and Vision Assistants
Lei Zhao, Zihao Ma, Boyu Lin et al. (+3 more)
We present an RL-central framework for
Language and
Vision Assistants (RLLaVA) with its formulation of Markov decision process (MDP). RLLaVA decouples RL algorithmic logic from model architecture and distributed execution, supporting researchers in implementing new RL
algorithms with minimal code, and to plug in a broad family of RL methods and
vision-
language models (VLMs) while remaining agnostic to specific training and inference engines. RLLaVA makes resource-efficient training of 1B–7B models feasible on common GPUs; notably, 4B-scale models can be trained end-to-end with full-parameter updates on a single 24GB GPU. Experiments on multi-modal and agentic tasks demonstrate that RLLaVA has task extensibility, and the models trained with it consistently improve performance over base models, competitive with other specially engineered RL frameworks. The code is available at https://github.com/TinyLoopX/RLLaVA.
Read the full abstract ▾
All authors Lei Zhao, Zihao Ma, Boyu Lin, Yuhe Liu, Wenjun Wu, Lei Huang