Machine Learning · Cross-listed from stat.ML
VSCOUT: A Hybrid Variational Autoencoder Approach to Outlier Detection in High-Dimensional Retrospective Monitoring
Waldyn G. Martinez
Modern industrial and service processes generate high-dimensional, non-Gaussian, and contamination-prone data that challenge the foundational assumptions of classical Statistical Process Control (SPC). Heavy tails, multimodality, nonlinear dependencies, and sparse special-cause observations can distort baseline estimation, mask true anomalies, and prevent reliable identification of an in-control (IC) reference set. To address these challenges, we introduce VSCOUT, a distribution-free framework designed specifically for retrospective (Phase I) monitoring in high-dimensional settings. VSCOUT combines an Automatic Relevance Determination Variational Autoencoder (ARD-VAE) architecture with ensemble-based latent outlier filtering and changepoint detection. The ARD prior isolates the most informative latent dimensions, while the ensemble and changepoint filters identify pointwise and structural contamination within the determined latent space. A second-stage retraining step removes flagged observations and re-estimates the latent structure using only the retained inliers, mitigating masking and stabilizing the IC latent
manifold. This two-stage refinement produces a clean and reliable IC baseline suitable for subsequent Phase II deployment. Extensive experiments across
benchmark datasets demonstrate that VSCOUT achieves superior sensitivity to special-cause structure while maintaining controlled false alarms, outperforming classical SPC procedures, robust
estimators, and modern machine-learning baselines. Its scalability, distributional flexibility, and resilience to complex contamination patterns position VSCOUT as a practical and effective method for retrospective modeling and anomaly detection in AI-enabled environments.
Read the full abstract ▾
Machine Learning
Training Reasoning Models on Saturated Problems via Failure-Prefix Conditioning
Minwu Kim, Safal Shrestha, Anubhav Shrestha et al. (+1 more)
As Reinforcement Learning with Verifiable Rewards (RLVR) substantially improves the
reasoning abilities of large
language models (LLMs), a new bottleneck emerges: more training problems become saturated, that is, the LLM answers the questions correctly for nearly every rollout. On such problems, rewards provide little useful learning signal. While collecting harder problems is a natural response, it is costly and increasingly difficult. We propose failure-prefix conditioning, a simple method that unlocks the remaining signal in saturated problems by shifting exploration toward failure-prone
reasoning states. By conditioning on prefixes of rare incorrect trajectories, the method improves the model's ability to recover from misleading early
reasoning. We observe that failure-prefix conditioning consistently improves performance where standard RLVR stalls, and achieves gains comparable to training on newly collected medium-difficulty problems. We further analyze the model's robustness, finding that our method reduces performance degradation under misleading failure prefixes, albeit with a mild trade-off in adherence to correct early
reasoning. Finally, we demonstrate that an iterative approach, which refreshes failure prefixes during training, unlocks additional gains after performance plateaus. Overall, our results show that saturated problems still contain valuable learning signal, and that failure-prefix conditioning provides an effective way to unlock it.
Read the full abstract ▾
All authors Minwu Kim, Safal Shrestha, Anubhav Shrestha, Keith Ross
Machine Learning · Cross-listed from stat.ML
Demystifying Prediction Powered Inference
Yilin Song, Dan M. Kluger, Harsh Parikh et al. (+1 more)
Machine learning predictions are increasingly used to supplement incomplete or costly-to-measure outcomes in fields such as biomedical research, environmental science, and social science. However, treating predictions as ground truth introduces bias while ignoring them wastes valuable information. Prediction-Powered Inference (PPI) offers a principled framework that leverages predictions from large unlabeled
datasets to improve statistical efficiency while maintaining valid inference through explicit bias correction using a smaller labeled subset. Despite its potential, the growing PPI variants and the subtle distinctions between them have made it challenging for practitioners to determine when and how to apply these methods responsibly. This paper demystifies PPI by synthesizing its theoretical foundations, methodological extensions, connections to existing statistics literature, and diagnostic tools into a unified practical workflow. Using the MOSAIKS housing price data, we show that PPI variants produce tighter confidence intervals than complete-case analysis, but that double-dipping, i.e. reusing training data for inference, leads to anti-conservative confidence intervals and below-nominal coverage. Under missing-not-at-random mechanisms, all methods, including classical inference using only labeled data, yield biased estimates. We provide a decision flowchart linking assumption violations to appropriate PPI variants, a summary table of representative methods, and practical diagnostic strategies for evaluating core assumptions. By framing PPI as a general recipe rather than a single
estimator, this work bridges methodological innovation and applied practice, helping researchers responsibly integrate predictions into valid inference.
Read the full abstract ▾
All authors Yilin Song, Dan M. Kluger, Harsh Parikh, Tian Gu
Machine Learning
GNN Explanations that do not Explain and How to find Them
Steve Azzolin, Stefano Teso, Bruno Lepri et al. (+2 more)
Explanations provided by Self-explainable Graph Neural Networks (SE-GNNs) are fundamental for understanding the model's inner workings and for identifying potential misuse of sensitive attributes. Although recent works have highlighted that these explanations can be suboptimal and potentially misleading, a characterization of their failure cases is unavailable. In this work, we identify a critical failure of SE-GNN explanations: explanations can be unambiguously unrelated to how the SE-GNNs infer labels. We show that, on the one hand, many SE-GNNs can achieve optimal true risk while producing these degenerate explanations, and on the other, most faithfulness metrics can fail to identify these failure modes. Our empirical analysis reveals that degenerate explanations can be maliciously planted (allowing an attacker to hide the use of sensitive attributes) and can also emerge naturally, highlighting the need for reliable
auditing. To address this, we introduce a novel faithfulness metric that reliably marks degenerate explanations as unfaithful, in both malicious and natural settings. Our code is available at https://github.com/steveazzolin/gnn_deg_expl.
Read the full abstract ▾
All authors Steve Azzolin, Stefano Teso, Bruno Lepri, Andrea Passerini, Sagar Malhotra
Machine Learning · Cross-listed from cs.SE
Context-Augmented Code Generation Using Programming Knowledge Graphs
Shahd Seddik, Fahd Seddik, Iman Saberi et al. (+3 more)
Large
Language Models (LLMs) excel at code generation but struggle with complex problems.
Retrieval-Augmented Generation (RAG) mitigates this issue by integrating external knowledge, yet
retrieval models often miss relevant context, and generation models hallucinate with irrelevant data. We propose Programming Knowledge Graph (PKG) for semantic representation and fine-grained
retrieval of code and text. Our approach enhances
retrieval precision through tree
pruning and mitigates
hallucinations via a re-ranking mechanism that integrates non-RAG solutions. Structuring external data into finer-grained nodes improves
retrieval granularity. Evaluations on HumanEval and MBPP show up to 20% pass@1 accuracy gains and a 34% improvement over baselines on MBPP. Our findings demonstrate that our proposed PKG approach along with re-ranker effectively address complex problems while maintaining minimal negative impact on solutions that are already correct without RAG. The replication package is published at https://github.com/iamshahd/ProgrammingKnowledgeGraph
Read the full abstract ▾
All authors Shahd Seddik, Fahd Seddik, Iman Saberi, Fatemeh Fard, Minh Hieu Huynh, Patanamon Thongtanunam
Machine Learning
Reinforcement Learning via Self-Distillation
Jonas Hübotter, Frederike Lübeck, Lejs Behric et al. (+8 more)
Large
language models are increasingly post-trained with reinforcement learning in verifiable domains such as code and math. Yet, current methods for reinforcement learning with verifiable rewards (RLVR) learn only from a scalar outcome reward per attempt, creating a severe credit-assignment bottleneck. Many verifiable environments actually provide rich textual feedback, such as runtime errors or judge evaluations, that explain why an attempt failed. We formalize this setting as reinforcement learning with rich feedback and introduce Self-
Distillation Policy Optimization (SDPO), which converts tokenized feedback into a dense learning signal without any external teacher or explicit reward model. SDPO treats the current model conditioned on feedback as a self-teacher and distills its feedback-informed next-
token predictions back into the policy. In this way, SDPO leverages the model's ability to retrospectively identify its own mistakes in-context. Across scientific
reasoning, tool use, and competitive programming on LiveCodeBench v6, SDPO improves sample efficiency and final accuracy over strong RLVR baselines. Notably, SDPO also outperforms baselines in standard RLVR environments that only return scalar feedback by using successful rollouts as implicit feedback for failed attempts. Finally, applying SDPO to individual questions at test time accelerates discovery on difficult binary-reward tasks, achieving the same discovery probability as best-of-k sampling or multi-turn conversations with 3x fewer attempts.
Read the full abstract ▾
All authors Jonas Hübotter, Frederike Lübeck, Lejs Behric, Anton Baumann, Marco Bagatella, Daniel Marta, Ido Hakimi, Idan Shenfeld, Thomas Kleine Buening, Carlos Guestrin, Andreas Krause
Machine Learning
Conditional PED-ANOVA: Hyperparameter Importance in Hierarchical & Dynamic Search Spaces
Kaito Baba, Yoshihiko Ozaki and Shuhei Watanabe
We propose conditional PED-ANOVA (condPED-ANOVA), a principled framework for estimating hyperparameter importance (HPI) in conditional search spaces, where the presence or domain of a hyperparameter can depend on other hyperparameters. Although the original PED-ANOVA provides a fast and efficient way to estimate HPI within the top-performing regions of the search space, it assumes a fixed, unconditional search space and therefore cannot properly handle conditional hyperparameters. To address this, we introduce a conditional HPI for top-performing regions and derive a closed-form
estimator that accurately reflects conditional activation and domain changes. Experiments show that naive adaptations of existing HPI
estimators yield misleading or uninterpretable importances in conditional settings, whereas condPED-ANOVA consistently provides meaningful importances that reflect the underlying conditional structure. Our code is publicly available at https://github.com/kAIto47802/condPED-ANOVA.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Dissecting Multimodal In-Context Learning: Modality Asymmetries and Circuit Dynamics in modern Transformers
Yiran Huang, Karsten Roth, Quentin Bouniot et al. (+2 more)
Transformer-based
multimodal large
language models often exhibit in-context learning (ICL) abilities. Motivated by this phenomenon, we ask: how do
transformers learn to associate information across modalities from in-context examples? We investigate this question through controlled experiments on small
transformers trained on synthetic classification tasks, enabling precise manipulation of data statistics and model architecture. We begin by revisiting core principles of unimodal ICL in modern
transformers. While several prior findings replicate, we find that Rotary Position
Embeddings (RoPE) increases the data complexity threshold for ICL. Extending to the
multimodal setting reveals a fundamental learning asymmetry: when pretrained on high-diversity data from a primary modality, surprisingly low data complexity in the secondary modality suffices for
multimodal ICL to emerge. Mechanistic analysis shows that both settings rely on an induction-style mechanism that copies labels from matching in-context exemplars;
multimodal training refines and extends these
circuits across modalities. Our findings provide a mechanistic foundation for understanding
multimodal ICL in modern
transformers and introduce a controlled testbed for future investigation. Code is available at: https://github.com/YiranHuangIrene/
multimodal-icl
Read the full abstract ▾
All authors Yiran Huang, Karsten Roth, Quentin Bouniot, Wenjia Xu, Zeynep Akata
Machine Learning · Cross-listed from cs.CL
SERA: Soft-Verified Efficient Repository Agents
Ethan Shen, Daniel Tormoen, Saurabh Shah et al. (+2 more)
Open-weight coding
agents should hold a fundamental advantage over closed-source systems because they can specialize to private codebases, encoding repository-specific information directly in their weights. Yet the cost and complexity of training has kept this advantage theoretical until now. We present Soft-Verified Efficient Repository
Agents (SERA), an efficient method for training coding
agents that enables the rapid and cheap creation of
agents specialized to private codebases. Using Soft Verified Generation (SVG), we generate thousands of trajectories from any code repository, without requiring unit tests. Beyond repository specialization, we apply SVG to a larger corpus of codebases, generating 200,000+ synthetic trajectories. Using only supervised finetuning (SFT), SERA achieves leading results among fully open-source (open data, method, code) models while matching the performance of open-weight models like Devstral-Small-2. Creating SERA models is 26x cheaper than reinforcement learning and 57x cheaper than previous synthetic data methods to reach equivalent performance. We use our
dataset to provide detailed analysis of scaling laws, ablations, and confounding factors for training coding
agents. Overall, we believe our work will greatly accelerate research on open coding
agents and showcase the advantage of open-source models that can adapt to private codebases. We release SERA as the first model in Ai2's Open Coding
Agents series, along with all our code, data, and Claude Code integration to support the research community.
Read the full abstract ▾
All authors Ethan Shen, Daniel Tormoen, Saurabh Shah, Ali Farhadi, Tim Dettmers
Machine Learning · Cross-listed from quant-ph
Neural Quantum States in Mixed Precision
Massimo Solinas, Agnes Valenti, Nawaf Bou-Rabee et al. (+1 more)
Scientific computing has long relied on double precision (64-bit floating point) arithmetic to guarantee accuracy in simulations of real-world phenomena. However, the growing availability of hardware accelerators such as Graphics Processing Units (GPUs) has made low-precision formats attractive due to their superior performance, reduced
memory footprint, and improved energy efficiency. In this work, we investigate the role of mixed-precision arithmetic in neural-network based Variational Monte Carlo (VMC), a widely used method for solving computationally otherwise intractable
quantum many-body systems. We first derive general analytical bounds on the error introduced by reduced precision on Metropolis-Hastings MCMC, and then empirically validate these bounds on the use-case of VMC. We demonstrate that significant portions of the
algorithm, in particular, sampling the
quantum state, can be executed in half precision without loss of accuracy. More broadly, this work provides a theoretical framework to assess the applicability of mixed-precision arithmetic in machine-learning approaches that rely on MCMC sampling. In the context of VMC, we additionally demonstrate the practical effectiveness of mixed-precision strategies, enabling more scalable and energy-efficient simulations of
quantum many-body systems.
Read the full abstract ▾
All authors Massimo Solinas, Agnes Valenti, Nawaf Bou-Rabee, Roeland Wiersema
Machine Learning
Active Learning for Decision Trees with Provable Guarantees
Arshia Soltani Moakhar, Tanapoom Laoaron, Faraz Ghahremani et al. (+2 more)
This paper advances the theoretical understanding of active learning label complexity for decision trees as binary classifiers. We make two main contributions. First, we provide the first analysis of the disagreement coefficient for decision trees-a key parameter governing active learning label complexity. Our analysis holds under two natural assumptions required for achieving polylogarithmic label complexity, (i) each root-to-leaf path queries distinct feature dimensions, and (ii) the input data has a regular, grid-like structure. We show these assumptions are essential, as relaxing them leads to
polynomial label complexity. Second, we present the first general active learning
algorithm for binary classification that achieves a multiplicative error guarantee, producing a $(1+\epsilon)$-approximate classifier. By combining these results, we design an active learning
algorithm for decision trees that uses only a polylogarithmic number of label queries in the
dataset size, under the stated assumptions. Finally, we establish a label complexity lower bound, showing our
algorithm's dependence on the error tolerance $\epsilon$ is close to optimal.
Read the full abstract ▾
All authors Arshia Soltani Moakhar, Tanapoom Laoaron, Faraz Ghahremani, Kiarash Banihashem, MohammadTaghi Hajiaghayi
Machine Learning
When More Data Doesn't Help: Limits of Adaptation in Multitask Learning
Steve Hanneke and Mingyue Xu
Multitask learning and related frameworks have achieved tremendous success in modern applications. In multitask learning problem, we are given a set of heterogeneous
datasets collected from related source tasks and hope to enhance the performance above what we could hope to achieve by solving each of them individually. The recent work of arXiv:2006.15785 has showed that, without access to distributional information, no
algorithm based on aggregating samples alone can guarantee optimal risk as long as the sample size per task is bounded. In this paper, we focus on understanding the statistical limits of multitask learning. We go beyond the no-free-lunch
theorem in arXiv:2006.15785 by establishing a stronger impossibility result of adaptation that holds for arbitrarily large sample size per task. This improvement conveys an important message that the hardness of multitask learning cannot be overcame by having abundant data per task. We also discuss the notion of optimal adaptivity that may be of future interests.
Read the full abstract ▾
Machine Learning
Smoothing the Black-Box: Signed-Distance Supervision for Black-Box Model Copying
Rubén Jiménez and Oriol Pujol
Deployed machine learning systems must continuously evolve as data, architectures, and regulations change, often without access to original training data or model internals. In such settings, black-box copying provides a practical refactoring mechanism, i.e. upgrading legacy models by learning replicas from input-output queries alone. When restricted to hard-label outputs, copying turns into a discontinuous surface reconstruction problem from pointwise queries, severely limiting the ability to recover boundary geometry efficiently. We propose a distance-based copying (
distillation) framework that replaces hard-label supervision with signed distances to the teacher's decision boundary, converting copying into a smooth regression problem that exploits local geometry. We develop an $\alpha$-governed smoothing and regularization scheme with Hölder/Lipschitz control over the induced target surface, and introduce two model-agnostic
algorithms to estimate signed distances under label-only access. Experiments on synthetic problems and UCI
benchmarks show consistent improvements in fidelity and generalization accuracy over hard-label baselines, while enabling distance outputs as uncertainty-related signals for black-box replicas.
Read the full abstract ▾
Machine Learning
COMET-SG1: Lightweight Autoregressive Regressor for Edge and Embedded AI
Shakhyar Gogoi
COMET-SG1 is a lightweight, stability-oriented autoregressive regression model designed for time-series prediction on edge and embedded AI systems. Unlike recurrent neural networks or
transformer-based sequence models, COMET-SG1 operates through linear behavior-space encoding,
memory-anchored transition estimation, and deterministic state updates. This structure prioritizes bounded long-horizon behavior under fully autoregressive inference, a critical requirement for edge deployment where prediction errors accumulate over time. Experiments on non-stationary synthetic time-series data demonstrate that COMET-SG1 achieves competitive short-horizon accuracy while exhibiting significantly reduced long-horizon drift compared to MLP, LSTM, and k-nearest neighbor baselines. With a compact parameter footprint and operations compatible with fixed-point arithmetic, COMET-SG1 provides a practical and interpretable approach for stable autoregressive prediction in edge and embedded AI applications.
Read the full abstract ▾
Machine Learning · Cross-listed from q-bio.PE
Cross-Country Learning for National Infectious Disease Forecasting Using European Data
Zacharias Komodromos, Kleanthis Malialis, Artemis Kontou et al. (+1 more)
Accurate
forecasting of infectious disease incidence is critical for public health planning and timely intervention. While most data-driven
forecasting approaches rely primarily on historical data from a single country, such data are often limited in length and variability, restricting the performance of machine learning (ML) models. In this work, we investigate a cross-country learning approach for infectious disease
forecasting, in which a single model is trained on time series data from multiple countries and evaluated on a country of interest. This setting enables the model to exploit shared epidemic dynamics across countries and to benefit from an enlarged training set. We examine this approach through a case study on COVID-19 case
forecasting in Cyprus, using surveillance data of European countries. We evaluate multiple models and analyse the impact of the lookback window length and cross-country 'data augmentation' on multi-step
forecasting performance. The results show that combining data from other countries can lead to consistent improvements over models trained solely on national data. Although the focus is on Cyprus and COVID-19, the framework and findings provide promising insights for infectious disease
forecasting in settings with limited national data.
Read the full abstract ▾
All authors Zacharias Komodromos, Kleanthis Malialis, Artemis Kontou, Panayiotis Kolios
Machine Learning · Cross-listed from eess.IV
Leveraging Second-Order Curvature for Efficient Learned Image Compression: Theory and Empirical Evidence
Yichi Zhang and Fengqing Zhu
Training learned image compression (LIC) models entails navigating a challenging optimization landscape defined by the fundamental trade-off between rate and distortion. Standard first-order optimizers, such as SGD and Adam, struggle with gradient conflicts arising from competing objectives, leading to slow convergence and suboptimal rate-distortion performance. In this work, we demonstrate that a simple utilization of a second-order quasi-Newton optimizer, SOAP, dramatically improves both training efficiency and final performance across diverse LICs. Our theoretical and empirical analyses reveal that Newton preconditioning inherently resolves the intra-step and inter-step update conflicts intrinsic to the R-D objective, facilitating faster, more stable convergence. Beyond acceleration, we uncover a critical deployability benefit: second-order trained models exhibit significantly fewer activation and latent outliers. This substantially enhances robustness to post-training quantization. Together, these results establish second-order optimization, achievable as a seamless drop-in replacement of the imported optimizer, as a powerful, practical tool for advancing the efficiency and real-world readiness of LICs.
Read the full abstract ▾
Machine Learning
Less is More: Clustered Cross-Covariance Control for Offline RL
Nan Qiao, Sheng Yue, Shuning Wang et al. (+2 more)
A fundamental challenge in offline reinforcement learning is distributional shift. Scarce data or
datasets dominated by out-of-distribution (OOD) areas exacerbate this issue. Our theoretical analysis and experiments show that the standard squared error objective induces a harmful TD cross covariance. This effect amplifies in OOD areas, biasing optimization and degrading policy learning. To counteract this mechanism, we develop two complementary strategies: partitioned buffer sampling that restricts updates to localized replay partitions, attenuates irregular covariance effects, and aligns update directions, yielding a scheme that is easy to integrate with existing implementations, namely Clustered Cross-Covariance Control for TD (C^4). We also introduce an explicit gradient-based corrective penalty that cancels the covariance induced bias within each update. We prove that buffer partitioning preserves the lower bound property of the maximization objective, and that these constraints mitigate excessive conservatism in extreme OOD areas without altering the core behavior of policy constrained offline reinforcement learning. Empirically, our method showcases higher stability and up to 30% improvement in returns over prior methods, especially with small
datasets and splits that emphasize OOD areas.
Read the full abstract ▾
All authors Nan Qiao, Sheng Yue, Shuning Wang, Yongheng Deng, Ju Ren
Machine Learning
Supervised Guidance Training for Infinite-Dimensional Diffusion Models
Elizabeth L. Baker, Alexander Denker and Jes Frellsen
Score-based
diffusion models have recently been extended to infinite-dimensional function spaces, with uses such as inverse problems arising from partial differential equations. In the
Bayesian formulation of inverse problems, the aim is to sample from a posterior distribution over functions obtained by conditioning a prior on noisy observations. While
diffusion models provide expressive priors in function space, the theory of conditioning them to sample from the posterior remains open. We address this, assuming that either the prior lies in the Cameron-Martin space, or is absolutely continuous with respect to a Gaussian measure. We prove that the models can be conditioned using an infinite-dimensional extension of Doob's $h$-transform, and that the conditional score decomposes into an unconditional score and a guidance term. As the guidance term is intractable, we propose a simulation-free score matching objective (called Supervised Guidance Training) enabling efficient and stable posterior sampling. We illustrate the theory with numerical examples on
Bayesian inverse problems in function spaces. In summary, our work offers the first function-space method for fine-tuning trained
diffusion models to accurately sample from a posterior.
Read the full abstract ▾
Machine Learning
GraphAllocBench: A Flexible Benchmark for Preference-Conditioned Multi-Objective Policy Learning
Zhiheng Jiang, Yunzhe Wang, Ryan Marr et al. (+3 more)
Preference-Conditioned Policy Learning (PCPL) in Multi-Objective Reinforcement Learning (MORL) approximates diverse Pareto-optimal solutions by conditioning a single policy on user-specified preferences, enabling run-time adaptation to arbitrary trade-offs without retraining. However, existing PCPL
benchmarks are largely restricted to toy tasks and fixed environments, limiting their realism and scalability. To address this gap, we introduce GraphAllocBench, a flexible
benchmark built on CityPlannerEnv, a novel graph-based resource allocation sandbox inspired by city management. GraphAllocBench provides a rich suite of problems with customizable objective functions, varying preference conditions, complex Pareto Fronts, and high-dimensional scalability. We further propose two supplementary metrics – Proportion of Non-Dominated Solutions (PNDS) and Ordering Score (OS) – that capture prediction reliability and preference consistency while complementing the widely used hypervolume metric. Through experiments with several state-of-the-art PCPL
algorithms and our own MLP and graph-aware PCPL-PPO baseline, we show that GraphAllocBench exposes distinct failure modes that hypervolume alone does not capture but our supplementary metrics reveal, while motivating graph-based approaches such as Graph Neural Networks (GNNs) for scaling to complex, high-dimensional allocation tasks. By letting users freely vary objectives, preferences, and allocation rules, GraphAllocBench serves as a versatile and extensible testbed for advancing PCPL.
Read the full abstract ▾
All authors Zhiheng Jiang, Yunzhe Wang, Ryan Marr, Ellen Novoseller, Benjamin T. Files, Volkan Ustun
Machine Learning
HESTIA: A Hessian-Guided Differentiable Quantization-Aware Training Framework for Extremely Low-Bit LLMs
Guoan Wang, Feiyu Wang, Zongwei Lv et al. (+2 more)
As large
language models (LLMs) continue to scale, deployment is increasingly bottlenecked by the
memory wall, motivating a shift toward extremely low-bit quantization. However, most quantization-aware training (QAT) methods apply hard rounding and the straight-through
estimator (STE) from the beginning of the training, which prematurely discretizes the optimization landscape and induces persistent gradient mismatch between latent weights and quantized weights, hindering effective optimization of quantized models. To address this, we propose Hestia, a Hessian-guided differentiable QAT framework for extremely low-bit LLMs, which replaces the rigid step function with a temperature-controlled softmax relaxation to maintain gradient flow early in training while progressively hardening quantization. Furthermore, Hestia leverages a tensor-wise Hessian trace metric as a lightweight
curvature signal to drive fine-grained temperature annealing, enabling sensitivity-aware discretization across the model. Evaluations on Llama-3.2 show that Hestia consistently outperforms existing ternary QAT baselines, yielding average zero-shot improvements of 5.39% and 4.34% for the 1B and 3B models. These results indicate that Hessian-guided relaxation effectively recovers representational capacity, establishing a more robust training path for 1.58-bit LLMs. The code is available at https://github.com/hestia2026/Hestia.
Read the full abstract ▾
All authors Guoan Wang, Feiyu Wang, Zongwei Lv, Yikun Zong, Tong Yang
Machine Learning
SA-PEF: Step-Ahead Partial Error Feedback for Efficient Federated Learning
Dawit Kiros Redie, Reza Arablouei and Stefan Werner
Biased gradient compression with error feedback (EF) reduces communication in
federated learning (FL), but under non-IID data, the residual error can decay slowly, causing gradient mismatch and stalled progress in the early rounds. We propose step-ahead partial error feedback (SA-PEF), which integrates step-ahead (SA) correction with partial error feedback (PEF). SA-PEF recovers EF when the step-ahead coefficient $\alpha=0$ and step-ahead EF (SAEF) when $\alpha=1$. For non-convex objectives and $\delta$-contractive compressors, we establish a second-moment bound and a residual recursion that guarantee convergence to stationarity under heterogeneous data and partial client participation. The resulting rates match standard non-convex Fed-SGD guarantees up to constant factors, achieving $O((\eta,\eta_0TR)^{-1})$ convergence to a variance/heterogeneity floor with a fixed inner step size. Our analysis reveals a step-ahead-controlled residual contraction $\rho_r$ that explains the observed acceleration in the early training phase. To balance SAEF's rapid warm-up with EF's long-term stability, we select $\alpha$ near its theory-predicted optimum. Experiments across diverse architectures and
datasets show that SA-PEF consistently reaches target accuracy faster than EF.
Read the full abstract ▾
Machine Learning
Continual GUI Agents
Ziwei Liu, Borui Kang, Hangjie Yuan et al. (+4 more)
As digital environments (data distribution) are in flux, with new GUI data arriving over time-introducing new domains or resolutions-
agents trained on static environments deteriorate in performance. In this work, we introduce Continual GUI
Agents, a new task that requires GUI
agents to perform continual learning under shifted domains and resolutions. We find existing methods fail to maintain stable grounding as GUI distributions shift over time, due to the diversity of UI interaction points and regions in fluxing scenarios. To address this, we introduce GUI-Anchoring in Flux (GUI-AiF), a new reinforcement fine-tuning framework that stabilizes continual learning through two novel rewards: Anchoring Point Reward in Flux (APR-iF) and Anchoring Region Reward in Flux (ARR-iF). These rewards guide the
agents to align with shifting interaction points and regions, mitigating the tendency of existing reward strategies to over-adapt to static grounding cues (e.g., fixed coordinates or element scales). Extensive experiments show GUI-AiF surpasses state-of-the-art baselines. Our work establishes the first continual learning framework for GUI
agents, revealing the untapped potential of reinforcement fine-tuning for continual GUI
Agents.
Read the full abstract ▾
All authors Ziwei Liu, Borui Kang, Hangjie Yuan, Zixiang Zhao, Wei Li, Yifan Zhu, Tao Feng
Machine Learning
Deep Semi-Supervised Survival Analysis for Predicting Cancer Prognosis
Anchen Sun, Zhibin Chen and Xiaodong Cai
The Cox Proportional Hazards (PH) model is widely used in survival analysis. Recently, artificial neural network (ANN)-based Cox-PH models have been developed. However, training these Cox models with high-dimensional features typically requires a substantial number of labeled samples containing information about time-to-event. The limited availability of labeled data for training often constrains the performance of ANN-based Cox models. To address this issue, we employed a deep semi-supervised learning (DSSL) approach to develop single- and multi-modal ANN-based Cox models based on the Mean Teacher (MT) framework, which utilizes both labeled and unlabeled data for training. We applied our model, named Cox-MT, to predict the prognosis of several types of cancer using data from The Cancer Genome Atlas (TCGA). Our single-modal Cox-MT models, utilizing TCGA RNA-seq data or whole slide images, significantly outperformed the existing ANN-based Cox model, Cox-nnet, using the same data set across four types of cancer considered. As the number of unlabeled samples increased, the performance of Cox-MT significantly improved with a given set of labeled data. Furthermore, our multi-modal Cox-MT model demonstrated considerably better performance than the single-modal model. In summary, the Cox-MT model effectively leverages both labeled and unlabeled data to significantly enhance prediction accuracy compared to existing ANN-based Cox models trained solely on labeled data.
Read the full abstract ▾
Machine Learning
Adapting the Behavior of Reinforcement Learning Agents to Changing Action Spaces and Reward Functions
Raul de la Rosa, Ivana Dusparic and Nicolas Cardozo
Reinforcement Learning (RL)
agents often struggle in real-world applications where environmental conditions are non-stationary, particularly when reward functions shift or the available action space expands. This paper introduces MORPHIN, a self-adaptive Q-learning framework that enables on-the-fly adaptation without full retraining. By integrating concept drift detection with dynamic adjustments to learning and exploration hyperparameters, MORPHIN adapts
agents to changes in both the reward function and on-the-fly expansions of the
agent's action space, while preserving prior policy knowledge to prevent catastrophic forgetting. We validate our approach using a Gridworld
benchmark and a traffic signal control simulation. The results demonstrate that MORPHIN achieves superior convergence speed and continuous adaptation compared to a standard Q-learning baseline, improving learning efficiency by up to 1.7x.
Read the full abstract ▾
Machine Learning · Cross-listed from hep-lat
A scalable flow-based approach to mitigate topological freezing
Claudio Bonanno, Andrea Bulgarelli, Elia Cellini et al. (+4 more)
As
lattice gauge theories with non-trivial topological features are driven towards the continuum limit, standard Markov Chain Monte Carlo simulations suffer for topological freezing, i.e., a dramatic growth of autocorrelations in topological observables. A widely used strategy is the adoption of Open Boundary Conditions (OBC), which restores ergodic sampling of
topology but at the price of breaking translation invariance and introducing unphysical boundary artifacts. In this contribution we summarize a scalable, exact flow-based strategy to remove them by transporting configurations from a prior with a OBC defect to a fully periodic ensemble, and apply it to 4d SU(3) Yang–Mills theory. The method is based on a
Stochastic Normalizing Flow (SNF) that alternates non-
equilibrium Monte Carlo updates with localized, gauge-equivariant defect coupling layers implemented via masked parametric stout smearing. Training is performed by minimizing the average dissipated work, equivalent to a Kullback–Leibler divergence between forward and reverse non-
equilibrium path measures, to achieve more reversible trajectories and improved efficiency. We discuss the scaling with the number of degrees of freedom affected by the defect and show that defect SNFs achieve better performances than purely
stochastic non-
equilibrium methods at comparable cost. Finally, we validate the approach by reproducing reference results for the topological susceptibility.
Read the full abstract ▾
All authors Claudio Bonanno, Andrea Bulgarelli, Elia Cellini, Alessandro Nada, Dario Panfalone, Davide Vadacchino, Lorenzo Verzichelli
Machine Learning
Structurally Human, Semantically Biased: Detecting LLM-Generated References with Embeddings and GNNs
Melika Mobini, Vincent Holst, Floriano Tori et al. (+2 more)
Large
language models are increasingly used to curate bibliographies, raising the question: are their reference lists distinguishable from human ones? We build paired citation graphs, ground truth and GPT-4o-generated (from parametric knowledge), for 10,000 focal papers ($\approx$ 275k references) from SciSciNet, and added a field-matched random baseline that preserves out-degree and field distributions while breaking latent structure. We compare (i) structure-only node features (degree/closeness/eigenvector centrality, clustering, edge count) with (ii) 3072-D title/abstract
embeddings, using an RF on graph-level aggregates and Graph Neural Networks with node features. Structure alone barely separates GPT from ground truth (RF accuracy $\approx$ 0.60) despite cleanly rejecting the random baseline ($\approx$ 0.89–0.92). By contrast,
embeddings sharply increase separability: RF on aggregated
embeddings reaches $\approx$ 0.83, and GNNs with
embedding node features achieve 93% test accuracy on GPT vs.\ ground truth. We show the robustness of our findings by replicating the pipeline with Claude Sonnet 4.5 and with multiple
embedding models (OpenAI and SPECTER), with RF separability for ground truth vs.\ Claude $\approx 0.77$ and clean rejection of the random baseline. Thus, LLM bibliographies, generated purely from parametric knowledge, closely mimic human citation
topology, but leave detectable semantic fingerprints; detection and debiasing should target content signals rather than global graph structure.
Read the full abstract ▾
All authors Melika Mobini, Vincent Holst, Floriano Tori, Andres Algaba, Vincent Ginis
Machine Learning · Cross-listed from cs.AI
Enterprise Resource Planning Using Multi-type Transformers in Ferro-Titanium Industry
Samira Yazdanpourmoghadam, Mahan Balal Pour and Vahid Partovi Nia
Combinatorial optimization problems such as the Job-Shop Scheduling Problem (JSP) and Knapsack Problem (KP) are fundamental challenges in operations research, logistics, and eterprise resource planning (ERP). These problems often require sophisticated
algorithms to achieve near-optimal solutions within practical time constraints. Recent advances in deep learning have introduced
transformer-based architectures as promising alternatives to traditional heuristics and metaheuristics. We leverage the Multi-Type
Transformer (MTT) architecture to address these
benchmarks in a unified framework. We present an extensive experimental evaluation across standard
benchmark datasets for JSP and KP, demonstrating that MTT achieves competitive performance on different size of these
benchmark problems. We showcase the potential of multi-type
attention on a real application in Ferro-Titanium industry. To the best of our knowledge, we are the first to apply multi-type
transformers in real manufacturing.
Read the full abstract ▾
Machine Learning
Is Pure Exploitation Sufficient in Exogenous MDPs with Linear Function Approximation?
Hao Liang, Jiayu Cheng, Sean R. Sinclair et al. (+1 more)
Exogenous MDPs (Exo-MDPs) capture sequential decision-making where uncertainty comes solely from exogenous inputs that evolve independently of the learner's actions. This structure is especially common in operations research applications such as inventory control, energy storage, and resource allocation, where exogenous randomness (e.g., demand, arrivals, or prices) drives system behavior. Despite decades of empirical evidence that greedy, exploitation-only methods work remarkably well in these settings, theory has lagged behind: all existing regret guarantees for Exo-MDPs rely on explicit exploration or tabular assumptions. We show that exploration is unnecessary. We propose Pure Exploitation Learning (PEL) and prove the first general finite-sample regret bounds for exploitation-only
algorithms in Exo-MDPs. In the tabular case, PEL achieves $\widetilde{O}(H^2|\Xi|\sqrt{K})$. For large, continuous endogenous state spaces, we introduce LSVI-PE, a simple linear-approximation method whose regret is
polynomial in the feature dimension, exogenous state space, and horizon, independent of the endogenous state and action spaces. Our analysis introduces two new tools: counterfactual trajectories and Bellman-closed feature transport, which together allow greedy policies to have accurate value estimates without optimism. Experiments on synthetic and resource-management tasks show that PEL consistently outperforming baselines. Overall, our results overturn the conventional wisdom that exploration is required, demonstrating that in Exo-MDPs, pure exploitation is enough.
Read the full abstract ▾
All authors Hao Liang, Jiayu Cheng, Sean R. Sinclair, Yali Du
Machine Learning
Optimal Transport Group Counterfactual Explanations
Enrique Valero-Leal, Bernd Bischl, Pedro Larrañaga et al. (+2 more)
Group counterfactual explanations find a set of counterfactual instances to explain a group of input instances contrastively. However, existing methods either (i) optimize counterfactuals only for a fixed group and do not generalize to new group members, (ii) strictly rely on strong model assumptions (e.g., linearity) for tractability or/and (iii) poorly control the counterfactual group geometry distortion. We instead learn an explicit optimal transport map that sends any group instance to its counterfactual without re-optimization, minimizing the group's total transport cost. This enables generalization with fewer parameters, making it easier to interpret the common actionable recourse. For linear classifiers, we prove that functions representing group counterfactuals are derived via mathematical optimization, identifying the underlying convex optimization type (QP, QCQP, ...). Experiments show that they accurately generalize, preserve group geometry and incur only negligible additional transport cost compared to baseline methods. If model linearity cannot be exploited, our approach also significantly outperforms the baselines.
Read the full abstract ▾
All authors Enrique Valero-Leal, Bernd Bischl, Pedro Larrañaga, Concha Bielza, Giuseppe Casalicchio
Machine Learning
Positive-Unlabeled Reinforcement Learning Distillation for On-Premise Small Models
Zhiqiang Kou, Junyang Chen, Xin-Qiang Cai et al. (+8 more)
Due to constraints on
privacy, cost, and latency, on-premise deployment of small models is increasingly common. However, most practical pipelines stop at supervised fine-tuning (SFT) and fail to reach the reinforcement learning (RL)
alignment stage. The main reason is that RL
alignment typically requires either expensive human preference annotation or heavy reliance on high-quality reward models with large-scale API usage and ongoing engineering maintenance, both of which are ill-suited to on-premise settings. To bridge this gap, we propose a positive-unlabeled (PU) RL
distillation method for on-premise small-model deployment. Without human-labeled preferences or a reward model, our method distills the teacher's preference-optimization capability from black-box generations into a locally trainable student. For each
prompt, we query the teacher once to obtain an anchor response, locally sample multiple student candidates, and perform anchor-conditioned self-ranking to induce pairwise or listwise preferences, enabling a fully local training loop via direct preference optimization or group relative policy optimization. Theoretical analysis justifies that the induced preference signal by our method is order-consistent and concentrates on near-optimal candidates, supporting its stability for preference optimization. Experiments demonstrate that our method achieves consistently strong performance under a low-cost setting.
Read the full abstract ▾
All authors Zhiqiang Kou, Junyang Chen, Xin-Qiang Cai, Xiaobo Xia, Ming-Kun Xie, Dong-Dong Wu, Biao Liu, Yuheng Jia, Xin Geng, Masashi Sugiyama, Tat-Seng Chua