Machine Learning
GREPO: A Benchmark for Graph Neural Networks on Repository-Level Bug Localization
Juntong Wang, Libin Chen, Xiyuan Wang et al. (+4 more)
Repository-level bug localization-the task of identifying where code must be modified to fix a bug-is a critical software engineering challenge. Standard Large
Language Modles (LLMs) are often unsuitable for this task due to context window limitations that prevent them from processing entire code repositories. As a result, various
retrieval methods are commonly used, including keyword matching, text similarity, and simple graph-based heuristics such as Breadth-First Search. Graph Neural Networks (GNNs) offer a promising alternative due to their ability to model complex, repository-wide dependencies; however, their application has been hindered by the lack of a dedicated
benchmark. To address this gap, we introduce GREPO, the first GNN
benchmark for repository-scale bug localization tasks. GREPO comprises 86 Python repositories and 47294 bug-fixing tasks, providing graph-based data structures ready for direct GNN processing. Our evaluation of various GNN architectures shows outstanding performance compared to established information
retrieval baselines. This work highlights the potential of GNNs for bug localization and established GREPO as a foundation resource for future research, The code is available at https://github.com/qingpingmo/GREPO.
Read the full abstract ▾
All authors Juntong Wang, Libin Chen, Xiyuan Wang, Shijia Kang, Haotong Yang, Da Zheng, Muhan Zhang
Machine Learning
Sufficient Conditions for Stability of Minimum-Norm Interpolating Deep ReLU Networks
Ouns El Harzli, Yoonsoo Nam, Ilja Kuzborskij et al. (+2 more)
Algorithmic stability is a classical framework for analyzing the generalization error of learning
algorithms. It predicts that an
algorithm has small generalization error if it is insensitive to small perturbations in the training set such as the removal or replacement of a training point. While stability has been demonstrated for numerous well-known
algorithms, this framework has had limited success in analyses of deep neural networks. In this paper we study the algorithmic stability of deep ReLU homogeneous neural networks that achieve zero training error using parameters with the smallest $L_2$ norm, also known as the minimum-norm interpolation, a phenomenon that can be observed in overparameterized models trained by gradient-based
algorithms. We investigate sufficient conditions for such networks to be stable. We find that 1) such networks are stable when they contain a (possibly small) stable sub-network, followed by a layer with a low-rank weight
matrix, and 2) such networks are not guaranteed to be stable even when they contain a stable sub-network, if the following layer is not low-rank. The low-rank assumption is inspired by recent empirical and theoretical results which demonstrate that training deep neural networks is biased towards low-rank weight
matrices, for minimum-norm interpolation and weight-decay regularization.
Read the full abstract ▾
All authors Ouns El Harzli, Yoonsoo Nam, Ilja Kuzborskij, Bernardo Cuenca Grau, Ard A. Louis
Machine Learning · Cross-listed from stat.ML
How Accurately Can a Gaussian Approximate Stochastic Approximation Iterates?
Shaan Ul Haque, Zedong Wang, Zixuan Zhang et al. (+1 more)
Stochastic approximation (SA) is a method for finding the root of an operator perturbed by noise. The focus of this paper is studying the distribution of SA iterates in finite time. In general, it is not possible to characterize the exact distribution, and therefore our goal is to find an approximation which can yield useful tail bounds. Inspired by the rich literature on the asymptotic normality of rescaled SA iterates, we approximate the pre-limit distributions by a sequence of Gaussians whose covariance is recursively defined. In particular, we establish explicit bounds on the Wasserstein-1 distance between the rescaled iterate at time $k$ and the aforementioned Gaussian for various choices of step-sizes. Since these covariances converge to the classical asymptotic limit, our analysis also provides a convergence rate for asymptotic normality as a by-product. As an immediate consequence of our bounds, we obtain tail bounds on the error of SA iterates at any time. Finally, we establish the sharpness of our rates by providing matching lower bounds and validate our findings through simulations. We obtain the sharp rates by first studying the convergence rate of the discrete Ornstein-Uhlenbeck (O-U) process driven by general noise, whose stationary distribution is identical to the limiting Gaussian distribution of the rescaled SA iterates. We believe that this is of independent interest, given its connection to sampling literature. The analysis involves adapting Stein's method for Gaussian approximation to handle the
matrix weighted sum of i.i.d. random variables. The desired finite-time bounds for SA are obtained by characterizing the error dynamics between the rescaled SA iterate and the discrete time O-U process and combining it with the convergence rate of the latter process.
Read the full abstract ▾
All authors Shaan Ul Haque, Zedong Wang, Zixuan Zhang, Siva Theja Maguluri
Machine Learning · Cross-listed from cs.CV
RPGD: RANSAC-P3P Gradient Descent for Extrinsic Calibration in 3D Human Pose Estimation
Zhanyu Tuo
In this paper, we propose RPGD (RANSAC-P3P Gradient Descent), a human-pose-driven extrinsic calibration framework that robustly aligns MoCap-based 3D skeletal data with
monocular or multi-view RGB cameras using only natural human motion. RPGD formulates extrinsic calibration as a coarse-to-fine problem tailored to human poses, combining the global robustness of RANSAC-P3P with Gradient-Descent-based refinement. We evaluate RPGD on three large-scale public 3D HPE
datasets as well as on a self-collected in-the-wild
dataset. Experimental results demonstrate that RPGD consistently recovers extrinsic parameters with accuracy comparable to the provided ground truth, achieving sub-pixel MPJPE reprojection error even in challenging, noisy settings. These results indicate that RPGD provides a practical and automatic solution for reliable extrinsic calibration of large-scale 3D HPE
dataset collection.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Parameter-Efficient Fine-Tuning of DINOv2 for Large-Scale Font Classification
Daniel Chen, Zaria Zinn and Marcus Lowe
We introduce GoogleFontsBench, the first public
benchmark for classifying open-source web fonts, addressing a gap left by existing
benchmarks that cover only commercial typefaces. GoogleFontsBench comprises 394 font variants across 32 Google Fonts families, a reproducible synthetic data generation pipeline (~575 images per variant, ~226K total), and a typographically-grounded evaluation metric (SWER) that weights errors by visual severity. We establish baselines using six fine-tuning strategies on a DINOv2
Vision Transformer backbone. Parameter-efficient adaptation with LoRA achieves 99.0% top-1 accuracy while training only 1% of the model's 87.2M parameters, with errors 140x less severe than random guessing. We release the
benchmark, all trained models, and the full training pipeline as open-source resources.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
Ambient Physics: Training Neural PDE Solvers with Partial Observations
Harris Abdul Majid, Giannis Daras, Francesco Tudisco et al. (+1 more)
In many scientific settings, acquiring complete observations of PDE coefficients and solutions can be expensive, hazardous, or impossible. Recent
diffusion-based methods can reconstruct fields given partial observations, but require complete observations for training. We introduce Ambient Physics, a framework for learning the joint distribution of coefficient-solution pairs directly from partial observations, without requiring a single complete observation. The key idea is to randomly mask a subset of already-observed measurements and supervise on them, so the model cannot distinguish "truly unobserved" from "artificially unobserved", and must produce plausible predictions everywhere. Ambient Physics achieves state-of-the-art reconstruction performance. Compared with prior
diffusion-based methods, it achieves a 62.51$\%$ reduction in average overall error while using 125$\times$ fewer function evaluations. We also identify a "one-point transition": masking a single already-observed point enables learning from partial observations across architectures and measurement patterns. Ambient Physics thus enables scientific progress in settings where complete observations are unavailable.
Read the full abstract ▾
All authors Harris Abdul Majid, Giannis Daras, Francesco Tudisco, Steven McDonagh
Machine Learning · Cross-listed from math.ST
Ensemble-Conditional Gaussian Processes (Ens-CGP): Representation, Geometry, and Inference
Sai Ravela, Jae Deok Kim, Kenneth Gee et al. (+4 more)
We formulate Ensemble-Conditional Gaussian Processes (Ens-CGP), a finite-dimensional synthesis that centers ensemble-based inference on the conditional Gaussian law. Conditional Gaussian processes (CGP) arise directly from Gaussian processes under conditioning and, in linear-Gaussian settings, define the full posterior distribution for a Gaussian prior and linear observations. Classical Kalman filtering is a recursive
algorithm that computes this same conditional law under dynamical assumptions; the conditional Gaussian law itself is therefore the underlying representational object, while the filter is one computational realization. In this sense, CGP provides the probabilistic foundation for Kalman-type methods as well as equivalent formulations as a strictly convex quadratic program (MAP estimation), RKHS-regularized regression, and classical regularization. Ens-CGP is the ensemble instantiation of this object, obtained by treating empirical ensemble moments as a (possibly low-rank) Gaussian prior and performing exact conditioning. By separating representation (GP -> CGP -> Ens-CGP) from computation (Kalman filters, EnKF variants, and iterative ensemble schemes), the framework links an earlier-established representational foundation for inference to ensemble-derived priors and clarifies the relationships among probabilistic, variational, and ensemble perspectives.
Read the full abstract ▾
All authors Sai Ravela, Jae Deok Kim, Kenneth Gee, Xingjian Yan, Samson Mercier, Lubna Albarghouty, Anamitra Saha
Machine Learning · Cross-listed from cs.AI
Enabling Option Learning in Sparse Rewards with Hindsight Experience Replay
Gabriel Romio, Mateus Begnini Melchiades, Bruno Castro da Silva et al. (+1 more)
Hierarchical Reinforcement Learning (HRL) frameworks like Option-Critic (OC) and Multi-updates Option Critic (MOC) have introduced significant advancements in learning reusable options. However, these methods underperform in multi-goal environments with sparse rewards, where actions must be linked to temporally distant outcomes. To address this limitation, we first propose MOC-HER, which integrates the Hindsight Experience Replay (HER) mechanism into the MOC framework. By relabeling goals from achieved outcomes, MOC-HER can solve sparse reward environments that are intractable for the original MOC. However, this approach is insufficient for object manipulation tasks, where the reward depends on the object reaching the goal rather than on the
agent's direct interaction. This makes it extremely difficult for HRL
agents to discover how to interact with these objects. To overcome this issue, we introduce Dual Objectives Hindsight Experience Replay (2HER), a novel extension that creates two sets of virtual goals. In addition to relabeling goals based on the object's final state (standard HER), 2HER also generates goals from the
agent's effector positions, rewarding the
agent for both interacting with the object and completing the task. Experimental results in robotic manipulation environments show that MOC-2HER achieves success rates of up to 90%, compared to less than 11% for both MOC and MOC-HER. These results highlight the effectiveness of our dual objective relabeling strategy in sparse reward, multi-goal tasks.
Read the full abstract ▾
All authors Gabriel Romio, Mateus Begnini Melchiades, Bruno Castro da Silva, Gabriel de Oliveira Ramos
Machine Learning · Cross-listed from cs.NE
Evolving Multi-Channel Confidence-Aware Activation Functions for Missing Data with Channel Propagation
Naeem Shahabi Sani, Ferial Najiantabriz, Shayan Shafaei et al. (+1 more)
Learning in the presence of missing data can result in biased predictions and poor generalizability, among other difficulties, which data imputation methods only partially address. In neural networks, activation functions significantly affect performance yet typical options (e.g., ReLU, Swish) operate only on feature values and do not account for missingness indicators or confidence scores. We propose Three-Channel Evolved Activations (3C-EA), which we evolve using Genetic Programming to produce multivariate activation functions f(x, m, c) in the form of trees that take (i) the feature value x, (ii) a missingness indicator m, and (iii) an imputation confidence score c. To make these activations useful beyond the input layer, we introduce ChannelProp, an
algorithm that deterministically propagates missingness and confidence values via linear layers based on weight magnitudes, retaining reliability signals throughout the network. We evaluate 3C-EA and ChannelProp on
datasets with natural and injected (MCAR/MAR/MNAR) missingness at multiple rates under identical preprocessing and splits. Results indicate that integrating missingness and confidence inputs into the activation search improves classification performance under missingness.
Read the full abstract ▾
All authors Naeem Shahabi Sani, Ferial Najiantabriz, Shayan Shafaei, Dean F. Hougen
Machine Learning
sleep2vec: Unified Cross-Modal Alignment for Heterogeneous Nocturnal Biosignals
Weixuan Yuan, Zengrui Jin, Yichen Wang et al. (+4 more)
Tasks ranging from sleep staging to clinical diagnosis traditionally rely on standard polysomnography (PSG) devices, bedside monitors and wearable devices, which capture diverse nocturnal biosignals (e.g.,
EEG, EOG, ECG, SpO$_2$). However, heterogeneity across devices and frequent sensor dropout pose significant challenges for unified modelling of these
multimodal signals. We present sleep2vec, a foundation model for diverse and incomplete nocturnal biosignals that learns a shared representation via cross-modal
alignment. sleep2vec is contrastively pre-trained on 42,249 overnight recordings spanning nine modalities using a Demography, Age, Site & History-aware InfoNCE objective that incorporates physiological and acquisition metadata (e.g., age, gender, recording site) to dynamically weight negatives and mitigate cohort-specific shortcuts. On downstream sleep staging and clinical outcome assessment, sleep2vec consistently outperforms strong baselines and remains robust to any subset of available modalities and sensor dropout. We further characterize, to our knowledge for the first time, scaling laws for nocturnal biosignals with respect to modality diversity and model capacity. Together, these results show that unified cross-modal
alignment, coupled with principled scaling, enables label-efficient, general-purpose modelling of real-world nocturnal biosignals.
Read the full abstract ▾
All authors Weixuan Yuan, Zengrui Jin, Yichen Wang, Donglin Xie, Ziyi Ye, Chao Zhang, Xuesong Chen
Machine Learning
Testing For Distribution Shifts with Conditional Conformal Test Martingales
Shalev Shaer, Yarin Bar, Drew Prinster et al. (+1 more)
We propose a sequential test for detecting arbitrary distribution shifts that allows conformal test martingales (CTMs) to work under a fixed, reference-conditional setting. Existing CTM detectors construct test martingales by continually growing a reference set with each incoming sample, using it to assess how atypical the new sample is relative to past observations. While this design yields anytime-valid type-I error control, it suffers from test-time contamination: after a change, post-shift observations enter the reference set and dilute the evidence for distribution shift, increasing detection delay and reducing power. In contrast, our method avoids contamination by design by comparing each new sample to a fixed null reference
dataset. Our main technical contribution is a robust martingale construction that remains valid conditional on the null reference data, achieved by explicitly accounting for the estimation error in the reference distribution induced by the finite reference set. This yields anytime-valid type-I error control together with guarantees of asymptotic power one and bounded expected detection delay. Empirically, our method detects shifts faster than standard CTMs, providing a powerful and reliable distribution-shift detector.
Read the full abstract ▾
All authors Shalev Shaer, Yarin Bar, Drew Prinster, Yaniv Romano
Machine Learning · Cross-listed from nlin.CD
Physics constraints and response validation in discrete-time reduced-order modeling: from idealized turbulent systems to climate dynamics
Fabrizio Falasca and Laure Zanna
A central challenge across science and engineering is to build data-driven reduced-order models of turbulent dynamical systems that reproduce stationary statistics, predict responses to external perturbations, and remain practical for real-world applications. To this end, we introduce an abstract discrete-time formulation of turbulent dynamical systems with exact energy-conserving nonlinearities. Parameterizing this structure with neural networks yields stable, physics-constrained reduced-order models. We then use the fluctuation-dissipation
theorem (FDT) to validate the emulators' forced responses from unperturbed data alone, testing models beyond stationary statistics. The FDT also identifies candidate direct causal links, which are used as regularization terms only when they pass validation. We first test the framework on two idealized models of geophysical
turbulence: the proposed emulators reproduce stationary statistics and accurately predict responses to weak and strong forcings, despite being trained solely on unperturbed data. We then move beyond idealized systems to model tropical
climate dynamics from reanalysis data, where data scarcity, partial observability, and sensitivity to choices of
stochastic parameterizations become central challenges. The resulting physics-constrained model reproduces key statistics of the El Niño-Southern Oscillation (ENSO) and qualitatively captures its cumulative responses to perturbations; augmenting it with a non-Markovian
stochastic closure substantially improves quantitative agreement with the FDT
benchmark. This response-validated model is then used to characterize long-term causal drivers of ENSO variability. The proposed methodology establishes a modular framework for stable reduced-order models capable of probing causal mechanisms in realistic, partially observed turbulent systems.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
VAR-3D: View-aware Auto-Regressive Model for Text-to-3D Generation via a 3D Tokenizer
Zongcheng Han, Dongyan Cao, Haoran Sun et al. (+1 more)
Recent advances in auto-regressive
transformers have achieved remarkable success in generative modeling. However, text-to-3D generation remains challenging, primarily due to bottlenecks in learning discrete 3D representations. Specifically, existing approaches often suffer from information loss during encoding, causing representational distortion before the quantization process. This effect is further amplified by vector quantization, ultimately degrading the
geometric coherence of text-conditioned 3D shapes. Moreover, the conventional two-stage training paradigm induces an objective mismatch between reconstruction and text-conditioned auto-regressive generation. To address these issues, we propose View-aware Auto-Regressive 3D (VAR-3D), which intergrates a view-aware 3D Vector Quantized-Variational AutoEncoder (VQ-VAE) to convert the complex
geometric structure of 3D models into discrete
tokens. Additionally, we introduce a rendering-supervised training strategy that couples discrete
token prediction with visual reconstruction, encouraging the generative process to better preserve visual fidelity and structural consistency relative to the input text. Experiments demonstrate that VAR-3D significantly outperforms existing methods in both generation quality and text-3D
alignment.
Read the full abstract ▾
All authors Zongcheng Han, Dongyan Cao, Haoran Sun, Yu Hong
Machine Learning
Pawsterior: Variational Flow Matching for Structured Simulation-Based Inference
Jorge Carrasco-Pollo, Floor Eijkelboom and Jan-Willem van de Meent
We introduce Pawsterior, a variational flow-matching framework for improved and extended simulation-based inference (SBI). Many SBI problems involve posteriors constrained by structured domains, such as bounded physical parameters or hybrid discrete-continuous variables, yet standard flow-matching methods typically operate in unconstrained spaces. This mismatch leads to inefficient learning and difficulty respecting physical constraints. Our contributions are twofold. First, generalizing the
geometric inductive bias of CatFlow, we formalize endpoint-induced affine
geometric confinement, a principle that incorporates domain geometry directly into the inference process via a two-sided variational model. This formulation improves numerical stability during sampling and leads to consistently better posterior fidelity, as demonstrated by improved classifier two-sample test performance across standard SBI
benchmarks. Second, and more importantly, our variational parameterization enables SBI tasks involving discrete latent structure (e.g., switching systems) that are fundamentally incompatible with conventional flow-matching approaches. By addressing both
geometric constraints and discrete latent structure, Pawsterior provides a principled way to apply flow-matching in a broader range of structured SBI settings.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.NE
A Unified Physics-Informed Neural Network for Modeling Coupled Electro- and Elastodynamic Wave Propagation Using Three-Stage Loss Optimization
Suhas Suresh Bharadwaj and Reuben Thomas Thovelil
Physics-Informed Neural Networks present a novel approach in SciML that integrates physical laws in the form of partial differential equations directly into the NN through soft constraints in the loss function. This work studies the application of PINNs to solve a one dimensional coupled electro-elastodynamic system modeling linear piezoelectricity in stress-charge form, governed by elastodynamic and electrodynamic equations. Our simulation employs a feedforward architecture, mapping space-time coordinates to mechanical displacement and electric potential. Our PINN model achieved global relative L2 errors of 2.34 and 4.87 percent for displacement and electric potential respectively. The results validate PINNs as effective mesh free solvers for coupled time-dependent PDE systems, though challenges remain regarding error accumulation and stiffness in coupled
eigenvalue systems.
Read the full abstract ▾
Machine Learning
Mean Flow Policy with Instantaneous Velocity Constraint for One-step Action Generation
Guojian Zhan, Letian Tao, Pengcheng Wang et al. (+6 more)
Learning expressive and efficient policy functions is a promising direction in reinforcement learning (RL). While flow-based policies have recently proven effective in modeling complex action distributions with a fast deterministic sampling process, they still face a trade-off between expressiveness and computational burden, which is typically controlled by the number of flow steps. In this work, we propose mean velocity policy (MVP), a new generative policy function that models the mean velocity field to achieve the fastest one-step action generation. To ensure its high expressiveness, an instantaneous velocity constraint (IVC) is introduced on the mean velocity field during training. We theoretically prove that this design explicitly serves as a crucial boundary condition, thereby improving learning accuracy and enhancing policy expressiveness. Empirically, our MVP achieves state-of-the-art success rates across several challenging robotic manipulation tasks from Robomimic and OGBench. It also delivers substantial improvements in training and inference speed over existing flow-based policy baselines.
Read the full abstract ▾
All authors Guojian Zhan, Letian Tao, Pengcheng Wang, Yixiao Wang, Yiheng Li, Yuxin Chen, Hongyang Li, Masayoshi Tomizuka, Shengbo Eben Li
Machine Learning
AnomaMind: Coarse-to-Fine Agentic Diagnosis via Hybrid Inference for Time Series Anomaly Detection
Xiaoyu Tao, Yuchong Wu, Mingyue Cheng et al. (+2 more)
Time series anomaly detection (TSAD) supports the analysis of web service key performance indicators (KPIs) by identifying anomalous intervals. Recent large
language model (LLM)-based agentic approaches have achieved promising results through
reasoning and tool use. Despite their effectiveness, precise anomaly localization remains challenging, as diverse anomaly patterns demand targeted numerical evidence and iterative refinement to accurately determine anomaly boundaries. In this work, we propose AnomaMind, a coarse-to-fine agentic diagnosis framework for TSAD. AnomaMind first identifies suspicious regions from time-series visualizations, then adaptively acquires targeted numerical evidence through multi-turn tool interactions, and finally refines anomaly decisions through self-reflection. This process is supported by a tool engine that integrates knowledge
memory and numerical analysis. To coordinate this diagnostic process, we design a hybrid inference strategy that employs frozen general-purpose LLMs for initial localization, tool orchestration, and evaluation, alongside a specialized LLM detector for evidence-grounded interval prediction. We optimize only the specialized detector through supervised fine-tuning (SFT) on diagnostic trajectories and reinforcement learning (RL) with verifiable detection rewards. Experiments on four
benchmarks spanning web traffic, service KPIs, and system-monitoring signals, under both in-domain and multi-domain training settings, demonstrate competitive performance and support the effectiveness of coarse-to-fine adaptive diagnosis through targeted evidence acquisition and self-reflective refinement.The code is available at https://github.com/Xiaoyu-Tao/AnomaMind-TS.
Read the full abstract ▾
All authors Xiaoyu Tao, Yuchong Wu, Mingyue Cheng, Ze Guo, Min Wu
Machine Learning
Fast Physics-Driven Untrained Network for Highly Nonlinear Inverse Scattering Problems
Yutong Du, Zicheng Liu, Yi Huang et al. (+4 more)
Untrained neural networks (UNNs) offer high-fidelity electromagnetic inverse scattering reconstruction but are computationally limited by high-dimensional spatial-domain optimization. We propose a Real-Time Physics-Driven
Fourier-Spectral (PDF) solver that achieves sub-second reconstruction through spectral-domain dimensionality reduction. By expanding induced currents using a truncated
Fourier basis, the optimization is confined to a compact low-frequency parameter space supported by scattering measurements. The solver integrates a contraction integral equation (CIE) to mitigate high-contrast nonlinearity and a contrast-compensated operator (CCO) to correct spectral-induced attenuation. Furthermore, a bridge-suppressing loss is formulated to enhance boundary sharpness between adjacent scatterers. Numerical and experimental results demonstrate a 100-fold speedup over state-of-the-art UNNs with robust performance under noise and
antenna uncertainties, enabling real-time microwave imaging applications.
Read the full abstract ▾
All authors Yutong Du, Zicheng Liu, Yi Huang, Bazargul Matkerim, Bo Qi, Yali Zong, Peixian Han
Machine Learning · Cross-listed from cs.AI
Attention in Constant Time: Vashista Sparse Attention for Long-Context Decoding with Exponential Guarantees
Vashista Nobaub
Large
language models spend most of their inference cost on
attention over long contexts, yet empirical behavior suggests that only a small subset of
tokens meaningfully contributes to each query. We formalize this phenomenon by modeling
attention as a projection onto the convex hull of key vectors and analyzing its entropic (softmax-like) relaxation. Our main theoretical contribution is a face-stability
theorem showing that, under a strict complementarity margin (a support gap (\Delta) certified by KKT multipliers), entropic
attention concentrates on a constant-size active face: the total mass assigned to inactive
tokens decays exponentially as (\exp(-\Omega(\Delta/\varepsilon))), while the error on the active face scales linearly in the temperature/regularization parameter (\varepsilon). This yields a practical criterion for when sparse long-context decoding is safe and provides a principled knob to trade accuracy for compute. Building on these guarantees, we introduce Vashista Sparse
Attention, a drop-in mechanism that maintains a small candidate set per query through a paging-style context selection strategy compatible with modern inference stacks. Across long-context evaluations, we observe stable constant-size effective support, strong wall-clock speedups, and minimal quality degradation in the regimes predicted by the support-gap diagnostics. Finally, we discuss deployment implications for
privacy-sensitive and air-gapped settings, where interchangeable
attention modules enable predictable latency and cost without external
retrieval dependencies.
Read the full abstract ▾
Machine Learning
Learning Agentic Time Series Forecasting via Trajectory Distillation and Memory-Based Reinforcement Learning
Xiaoyu Tao, Mingyue Cheng, Yaguo Liu et al. (+4 more)
Recent agentic time series
forecasting (TSF) approaches use large
language models (LLMs) to autonomously orchestrate analytical and
forecasting tools and reason about temporal patterns using the resulting evidence. However, many such approaches rely on powerful LLMs in training-free,
prompt-driven workflows, leaving open whether smaller models can learn these interaction and
reasoning capabilities. In this paper, we introduce Cast-R1, an 8Bparameter agentic
forecasting model trained through trajectory
distillation and multi-turn reinforcement learning with structured
memory. First, we develop a
forecasting agent harness with specialized analysis and prediction tools to generate multi-turn interaction trajectories using a teacher LLM, which are subsequently evaluated and selected for supervised fine-tuning. Second, we apply group relative policy optimization (GRPO) across multi-turn
forecasting episodes, with structured
memory preserving diagnostic evidence and intermediate forecasts for state construction. Rewards based on prediction accuracy and temporal structure guide tool selection, model routing, and forecast refinement. A difficulty-aware curriculum combines sample filtering and progressive sample ordering during training. Extensive experiments on multiple
datasets demonstrate the effectiveness of Cast-R1. These findings highlight the potential of
forecasting-specific post-training to equip smaller LLMs with multi-turn, tool-augmented
forecasting capabilities within structured workflows.
Read the full abstract ▾
All authors Xiaoyu Tao, Mingyue Cheng, Yaguo Liu, Xingpeng Gao, Qingsong Wen, Shijin Wang, Enhong chen
Machine Learning
MechPert: Mechanistic Consensus as an Inductive Bias for Unseen Perturbation Prediction
Marc Boubnovski Martell, Josefa Lia Stoisser, Lawrence Phillips et al. (+6 more)
Predicting transcriptional responses to unseen genetic perturbations is essential for understanding gene regulation and prioritizing large-scale perturbation experiments. Existing approaches either rely on static, potentially incomplete knowledge graphs, or
prompt language models for functionally similar genes, retrieving associations shaped by symmetric co-occurrence in scientific text rather than directed regulatory logic. We introduce MechPert, a lightweight framework that encourages LLM
agents to generate directed regulatory hypotheses rather than relying solely on functional similarity. Multiple
agents independently propose candidate regulators with associated confidence scores; these are aggregated through a consensus mechanism that filters spurious associations, producing weighted neighborhoods for downstream prediction. We evaluate MechPert on Perturb-seq
benchmarks across four human
cell lines. For perturbation prediction in low-data regimes ($N=50$ observed perturbations), MechPert improves Pearson correlation by up to 10.5% over similarity-based baselines. For experimental design, MechPert-selected anchor genes outperform standard network centrality heuristics by up to 46% in well-characterized
cell lines.
Read the full abstract ▾
All authors Marc Boubnovski Martell, Josefa Lia Stoisser, Lawrence Phillips, Aditya Misra, Robert Kitchen, Jesper Ferkinghoff-Borg, Jialin Yu, Philip Torr, Kaspar Märten
Machine Learning
MEMTS: Internalizing Domain Knowledge via Parameterized Memory for Retrieval-Free Domain Adaptation of Time Series Foundation Models
Xiaoyun Yu, Li fan, Xiangfei Qiu et al. (+7 more)
While Time Series Foundation Models (TSFMs) have demonstrated exceptional performance in generalized
forecasting, their performance often degrades significantly when deployed in real-world vertical domains characterized by temporal distribution shifts and domain-specific periodic structures. Current solutions are primarily constrained by two paradigms: Domain-Adaptive Pretraining (DAPT), which improves short-term domain fitting but frequently disrupts previously learned global temporal patterns due to catastrophic forgetting; and
Retrieval-Augmented Generation (RAG), which incorporates external knowledge but introduces substantial
retrieval overhead. This creates a severe scalability bottleneck that fails to meet the high-efficiency requirements of real-time stream processing. To break this impasse, we propose
Memory for Time Series (MEMTS), a lightweight and plug-and-play method for
retrieval-free domain adaptation in time series
forecasting. The key component of MEMTS is a Knowledge Persistence Module (KPM), which internalizes domain-specific temporal dynamics, such as recurring seasonal patterns and trends into a compact set of learnable latent prototypes. In doing so, it transforms fragmented historical observations into continuous, parameterized knowledge representations. This paradigm shift enables MEMTS to achieve accurate domain adaptation with constant-time inference and near-zero latency, while effectively mitigating catastrophic forgetting of general temporal patterns, all without requiring any architectural modifications to the frozen TSFM backbone. Extensive experiments on multiple
datasets demonstrate the SOTA performance of MEMTS.
Read the full abstract ▾
All authors Xiaoyun Yu, Li fan, Xiangfei Qiu, Nanqing Dong, Yonggui Huang, Honggang Qi, Geguang Pu, Wanli Ouyang, Xi Chen, Jilin Hu
Machine Learning
On Representation Redundancy in Large-Scale Instruction Tuning Data Selection
Youwei Shu, Shaomian Zheng, Dingnan Jin et al. (+5 more)
Data quality is a crucial factor in large
language models training. While prior work has shown that models trained on smaller, high-quality
datasets can outperform those trained on much larger but noisy or low-quality corpora, systematic methods for industrial-scale data selection in instruction tuning remain underexplored. In this work, we study instruction-tuning data selection through the lens of semantic representation similarity and identify a key limitation of state-of-the-art LLM encoders: they produce highly redundant semantic
embeddings. To mitigate this redundancy, we propose Compressed Representation Data Selection (CRDS), a novel framework with two variants. CRDS-R applies Rademacher random projection followed by concatenation of
transformer hidden-layer representations, while CRDS-W employs whitening-based dimensionality reduction to improve representational quality. Experimental results demonstrate that both variants substantially enhance data quality and consistently outperform state-of-the-art representation-based selection methods. Notably, CRDS-W achieves strong performance using only 3.5% of the data, surpassing the full-data baseline by an average of 0.71% across four
datasets. Our code is available at https://github.com/tdano1/CRDS.
Read the full abstract ▾
All authors Youwei Shu, Shaomian Zheng, Dingnan Jin, Wenjie Qu, Ziyao Guo, Qing Cui, Jun Zhou, Jiaheng Zhang
Machine Learning · Cross-listed from eess.IV
NeuroMambaLLM: Dynamic Graph Learning of fMRI Functional Connectivity in Autistic Brains Using Mamba and Language Model Reasoning
Yasaman Torabi, Parsa Razmara, Hamed Ajorlou et al. (+1 more)
Large
Language Models (LLMs) have demonstrated strong semantic
reasoning across
multimodal domains. However, their integration with graph-based models of
brain connectivity remains limited. In addition, most existing fMRI analysis methods rely on static Functional Connectivity (FC) representations, which obscure transient neural dynamics critical for neurodevelopmental disorders such as autism. Recent state-space approaches, including Mamba, model temporal structure efficiently, but are typically used as standalone feature extractors without explicit high-level
reasoning. We propose NeuroMambaLLM, an end-to-end framework that integrates dynamic latent graph learning and selective state-space temporal modelling with LLMs. The proposed method learns the functional connectivity dynamically from raw Blood-Oxygen-Level-Dependent (BOLD) time series, replacing fixed correlation graphs with adaptive latent connectivity while suppressing motion-related artifacts and capturing long-range temporal dependencies. The resulting dynamic
brain representations are projected into the
embedding space of an LLM model, where the base
language model remains frozen and lightweight low-rank adaptation (LoRA) modules are trained for parameter-efficient
alignment. This design enables the LLM to perform both diagnostic classification and
language-based
reasoning, allowing it to analyze dynamic fMRI patterns and generate clinically meaningful textual reports.
Read the full abstract ▾
All authors Yasaman Torabi, Parsa Razmara, Hamed Ajorlou, Bardia Baraeinejad
Machine Learning
Discrete Double-Bracket Flows for Isotropic-Noise Invariant Eigendecomposition
ZhiMing Li and JiaHe Feng
We study eigendecomposition on $SO(n)$ under streaming observations $C_k = C_{\mathrm{sig}} + \sigma_k^2 I + E_k$, where the isotropic background $\sigma_k^2 I$ may be time-varying and arbitrarily large. Standard
algorithms couple their stability to $\lVert C_k \rVert_2 \approx \sigma^2$, forcing step sizes, contraction rates, and iteration counts to degrade with the noise floor. We observe that $\sigma^2 I$ lies in the center of the
matrix algebra and therefore *should never enter* the eigenspace dynamics. We construct a discrete double-bracket flow whose skew-symmetric generator $\Omega = [A, \operatorname{diag}(A)]$ operates in the tangent Lie
algebra $\mathfrak{so}(n)$, where scalar multiples of the identity vanish by antisymmetry. The resulting trajectory, Lyapunov function, and maximal stable step size $\eta_{\max} = 1/L_C$ depend exclusively on the trace-free signal $C_e$ – achieving pointwise, pathwise $\sigma^2$-invariance. We establish input-to-state stability with a noise ball governed solely by trace-free perturbations, prove global convergence via strict-saddle geometry and a discrete Łojasiewicz argument, and extend the framework to top-$k$ eigentracking on the Stiefel
manifold $\operatorname{St}(k,n)$ at cost $k$
matrix-vector products per step.
Read the full abstract ▾
Machine Learning
Data-driven Bi-level Optimization of Thermal Power Systems with embedded Artificial Neural Networks
Talha Ansar, Muhammad Mujtaba Abbas, Ramit Debnath et al. (+2 more)
Industrial thermal power systems have coupled performance variables with hierarchical order of importance, making their simultaneous optimization computationally challenging or infeasible. This barrier limits the integrated and computationally scaleable operation optimization of industrial thermal power systems. To address this issue for large-scale engineering systems, we present a fully machine learning-powered bi-level optimization framework for data-driven optimization of industrial thermal power systems. The objective functions of upper and lower levels are approximated by artificial neural network (ANN) models and the lower-level problem is analytically embedded through Karush-Kuhn-Tucker (KKT) optimality conditions. The reformulated single level optimization framework integrating ANN models and KKT constraints (ANN-KKT) is validated on
benchmark problems and on real-world power generation operation of 660 MW coal power plant and 395 MW gas turbine system. The results reveal a comparable solutions obtained from the proposed ANN-KKT framework to the bi-level solutions of the
benchmark problems. Marginal computational time requirement (0.22 to 0.88 s) to compute optimal solutions yields 583 MW (coal) and 402 MW (gas turbine) of power output at optimal turbine heat rate of 7337 kJ/kWh and 7542 kJ/kWh, respectively. In addition, the method expands to delineate a feasible and robust operating envelope that accounts for uncertainty in operating variables while maximizing thermal efficiency in various scenarios. These results demonstrate that ANN-KKT offers a scalable and computationally efficient route for hierarchical, data-driven optimization of industrial thermal power systems, achieving energy-efficient operations of large-scale engineering systems and contributing to industry 5.0.
Read the full abstract ▾
All authors Talha Ansar, Muhammad Mujtaba Abbas, Ramit Debnath, Vivek Dua, Waqar Muhammad Ashraf
Machine Learning · Cross-listed from cs.CV
Fine-tuned Vision Language Model for Localization of Parasitic Eggs in Microscopic Images
Chan Hao Sien, Hezerul Abdul Karim and Nouar AlDahoul
Soil-transmitted helminth (STH) infections continuously affect a large proportion of the global population, particularly in tropical and sub-tropical regions, where access to specialized diagnostic expertise is limited. Although manual microscopic diagnosis of parasitic eggs remains the diagnostic gold standard, the approach can be labour-intensive, time-consuming, and prone to human error. This paper aims to utilize a
vision language model (VLM) such as Microsoft Florence that was fine-tuned to localize all parasitic eggs within microscopic images. The preliminary results show that our localization VLM performs comparatively better than the other object detection methods, such as EfficientDet, with an mIOU of 0.94. This finding demonstrates the potential of the proposed VLM to serve as a core component of an automated framework, offering a scalable engineering solution for intelligent parasitological diagnosis.
Read the full abstract ▾
Machine Learning
HBVLA: Pushing 1-Bit Post-Training Quantization for Vision-Language-Action Models
Xin Yan, Zhenglin Wan, Feiyang Ye et al. (+4 more)
Vision-
Language-Action (VLA) models enable instruction-following
embodied control, but their large compute and
memory footprints hinder deployment on resource-constrained
robots and edge platforms. While reducing weights to 1-bit precision through binarization can greatly improve efficiency, existing methods fail to narrow the distribution gap between binarized and full-precision weights, causing quantization errors to accumulate under long-horizon closed-loop execution and severely degrade actions. To fill this gap, we propose HBVLA, a VLA-tailored binarization framework. First, we use a policy-aware enhanced Hessian to identify weights that are truly critical for action generation. Then, we employ a sparse orthogonal transform for non-salient weights to induce a low-
entropy intermediate state. Finally, we quantize both salient and non-salient weights in the Harr domain with group-wise 1-bit quantization. We have evaluated our approach on different VLAs: on LIBERO, quantized OpenVLA-OFT retains 92.2% of full-precision performance; on SimplerEnv, quantized CogAct retains 93.6%, significantly outperforming state-of-the-art binarization methods. We further validate our method on real-world evaluation suite and the results show that HBVLA incurs only marginal success-rate degradation compared to the full-precision model, demonstrating robust deployability under tight hardware constraints. Our work provides a practical foundation for ultra-low-bit quantization of VLAs, enabling more reliable deployment on hardware-limited robotic platforms.
Read the full abstract ▾
All authors Xin Yan, Zhenglin Wan, Feiyang Ye, Xingrui Yu, Hangyu Du, Yang You, Ivor Tsang
Machine Learning
Near-Optimal Regret for Policy Optimization in Contextual MDPs with General Offline Function Approximation
Orin Levy, Aviv Rosenberg, Alon Cohen et al. (+1 more)
We introduce OPO-CMDP, the first policy optimization
algorithm for
stochastic Contextual Markov Decision Process (CMDPs) under general offline function approximation. Our approach achieves a high probability regret bound of $\widetilde{O}(H^4\sqrt{T|S||A|\log(|\mathcal{F}||\mathcal{P}|)}),$ where $S$ and $A$ denote the state and action spaces, $H$ the horizon length, $T$ the number of episodes, and $\mathcal{F}, \mathcal{P}$ the finite function classes used to approximate the losses and dynamics, respectively. This is the first regret bound with optimal dependence on $|S|$ and $|A|$, directly improving the current state-of-the-art (Qian, Hu, and Simchi-Levi, 2024). These results demonstrate that optimistic policy optimization provides a natural, computationally superior and theoretically near-optimal path for solving CMDPs.
Read the full abstract ▾
All authors Orin Levy, Aviv Rosenberg, Alon Cohen, Yishay Mansour
Machine Learning
Optimal Regret for Policy Optimization in Contextual Bandits
Orin Levy and Yishay Mansour
We present the first high-probability optimal regret bound for a policy optimization technique applied to the problem of
stochastic contextual multi-armed bandit (CMAB) with general offline function approximation. Our
algorithm is both efficient and achieves an optimal regret bound of $\widetilde{O}(\sqrt{ K|\mathcal{A}|\log|\mathcal{F}|})$, where $K$ is the number of rounds, $\mathcal{A}$ is the set of arms, and $\mathcal{F}$ is the function class used to approximate the losses. Our results bridge the gap between theory and practice, demonstrating that the widely used policy optimization methods for the contextual bandit problem can achieve a rigorously-proved optimal regret bound. We support our theoretical results with an empirical evaluation of our
algorithm.
Read the full abstract ▾