Machine Learning
From Subtle to Significant: Prompt-Driven Self-Improving Optimization in Test-Time Graph OOD Detection
Luzhi Wang, Xuanshuo Fu, He Zhang et al. (+3 more)
Graph Out-of-Distribution (OOD) detection aims to identify whether a test graph deviates from the distribution of graphs observed during training, which is critical for ensuring the reliability of Graph Neural Networks (GNNs) when deployed in open-world scenarios. Recent advances in graph OOD detection have focused on test-time training techniques that facilitate OOD detection without accessing potential supervisory information (e.g., training data). However, most of these methods employ a one-pass inference paradigm, which prevents them from progressively correcting erroneous predictions to amplify OOD signals. To this end, we propose a Self-Improving Graph Out-of-Distribution detector (SIGOOD), which is an unsupervised framework that integrates continuous self-learning with test-time training for effective graph OOD detection. Specifically, SIGOOD generates a
prompt to construct a
prompt-enhanced graph that amplifies potential OOD signals. To optimize
prompts, SIGOOD introduces an Energy Preference Optimization (EPO) loss, which leverages energy variations between the original test graph and the
prompt-enhanced graph. By iteratively optimizing the
prompt by involving it into the detection model in a self-improving loop, the resulting optimal
prompt-enhanced graph is ultimately used for OOD detection. Comprehensive evaluations on 21 real-world
datasets confirm the effectiveness and outperformance of our SIGOOD method. The code is at https://github.com/Ee1s/SIGOOD.
Read the full abstract ▾
All authors Luzhi Wang, Xuanshuo Fu, He Zhang, Chuang Liu, Xiaobao Wang, Hongbo Liu
Machine Learning
SubQuad: Near-Quadratic-Free Structure Inference with Distribution-Balanced Objectives in Adaptive Receptor framework
Rong Fu, Zijian Zhang, Kun Liu et al. (+3 more)
Comparative analysis of adaptive immune repertoires at population scale is hampered by two practical bottlenecks: the near-quadratic cost of pairwise affinity evaluations and
dataset imbalances that obscure clinically important minority clonotypes. We introduce SubQuad, an end-to-end pipeline that addresses these challenges by combining antigen-aware, near-subquadratic
retrieval with GPU-accelerated affinity kernels, learned
multimodal fusion, and
fairness-constrained clustering. The system employs compact MinHash prefiltering to sharply reduce candidate comparisons, a differentiable gating module that adaptively weights complementary
alignment and
embedding channels on a per-pair basis, and an automated calibration routine that enforces proportional representation of rare antigen-specific subgroups. On large viral and tumor repertoires SubQuad achieves measured gains in throughput and peak
memory usage while preserving or improving recall@k, cluster purity, and subgroup equity. By co-designing indexing, similarity fusion, and equity-aware objectives, SubQuad offers a scalable, bias-aware platform for repertoire mining and downstream translational tasks such as vaccine target prioritization and biomarker discovery.
Read the full abstract ▾
All authors Rong Fu, Zijian Zhang, Kun Liu, Jiekai Wu, Xianda Li, Simon Fong
Machine Learning
The Sound of Death: Deep Learning Reveals Vascular Damage from Carotid Ultrasound
Christoph Balada, Aida Romano-Martinez, Payal Varshney et al. (+10 more)
Cardiovascular diseases (CVDs) remain the leading cause of mortality worldwide, yet early risk detection is often limited by available diagnostics. Carotid ultrasound, a non-invasive and widely accessible modality, encodes rich structural and hemodynamic information that is largely untapped. Here, we present a machine learning (ML) framework that extracts clinically meaningful representations of vascular damage (VD) from carotid ultrasound
videos, using hypertension as a weak proxy label. The model learns robust features that are biologically plausible, interpretable, and strongly associated with established cardiovascular risk factors, comorbidities, and laboratory measures. High VD stratifies individuals for myocardial infarction, cardiac death, and all-cause mortality, matching or outperforming conventional risk models such as SCORE2. Explainable AI analyses reveal that the model relies on vessel morphology and perivascular tissue characteristics, uncovering novel functional and anatomical signatures of vascular damage. This work demonstrates that routine carotid ultrasound contains far more prognostic information than previously recognized. Our approach provides a scalable, non-invasive, and cost-effective tool for population-wide cardiovascular risk assessment, enabling earlier and more personalized prevention strategies without reliance on laboratory tests or complex clinical inputs.
Read the full abstract ▾
All authors Christoph Balada, Aida Romano-Martinez, Payal Varshney, Vincent ten Cate, Katharina Geschke, Jonas Tesarz, Paul Claßen, Alexander K. Schuster, Dativa Tibyampansha, Karl-Patrik Kresoja, Philipp S. Wild, Sheraz Ahmed, Andreas Dengel
Machine Learning
Flickering Multi-Armed Bandits
Sourav Chakraborty, Amit Kiran Rege, Claire Monteleoni et al. (+1 more)
We introduce Flickering Multi-Armed Bandits (FMAB) to model sequential decision-making in environments with changing action availability, where accessibility of the next action is restricted to a subset dependent on the
agent's current choice. We formalize these constraints through stochastically evolving graphs where actions are limited to local neighborhoods. This mobility-constrained structure imposes a dual challenge: the statistical requirement of information acquisition and the physical overhead of
navigation. We analyze FMAB under i.i.d. Erdős–R'enyi and Edge-Markovian process, proposing a two-phase lazy random walk
algorithm for robust exploration. We establish high-probability sublinear regret bounds and prove near-optimality via a matching information-theoretic lower bound. Our results characterize the intrinsic cost of learning under local-move constraints, complemented by a robotic disaster-response simulation.
Read the full abstract ▾
All authors Sourav Chakraborty, Amit Kiran Rege, Claire Monteleoni, Lijun Chen
Machine Learning · Cross-listed from cs.CY
Open Datasets in Learning Analytics: Trends, Challenges, and Best PRACTICE
Valdemar Švábenský, Brendan Flanagan, Erwin Daniel López Zapata et al. (+1 more)
Open
datasets play a crucial role in three research domains that intersect data science and education: learning analytics, educational data mining, and artificial intelligence in education. Researchers in these domains apply computational methods to analyze data from educational contexts, aiming to better understand and improve teaching and learning. Providing open
datasets alongside research papers supports reproducibility, collaboration, and trust in research findings. It also provides individual benefits for authors, such as greater visibility, credibility, and citation potential. Despite these advantages, the availability of open
datasets and the associated practices within the learning analytics research communities, especially at their flagship conference venues, remain unclear. We surveyed available
datasets published alongside research papers in learning analytics. We manually examined 1,125 papers from three flagship conferences (LAK, EDM, and AIED) over the past five years. We discovered, categorized, and analyzed 172
datasets used in 204 publications. Our study presents the most comprehensive collection and analysis of open educational
datasets to date, along with the most detailed categorization. Of the 172
datasets identified, 143 were not captured in any prior survey of open data in learning analytics. We provide insights into the
datasets' context, analytical methods, use, and other properties. Based on this survey, we summarize the current gaps in the field. Furthermore, we list practical recommendations, advice, and 8-item guidelines under the acronym PRACTICE with a checklist to help researchers publish their data. Lastly, we share our original
dataset: an annotated inventory detailing the discovered
datasets and the corresponding publications. We hope these findings will support further adoption of open data practices in learning analytics communities and beyond.
Read the full abstract ▾
All authors Valdemar Švábenský, Brendan Flanagan, Erwin Daniel López Zapata, Atsushi Shimada
Machine Learning
LexiSafe: Offline Safe Reinforcement Learning with Lexicographic Safety-Reward Hierarchy
Hsin-Jung Yang, Zhanhong Jiang, Prajwal Koirala et al. (+3 more)
Offline safe reinforcement learning (RL) is increasingly important for cyber-physical systems (CPS), where safety violations during training are unacceptable and only pre-collected data are available. Existing offline safe RL methods typically balance reward-safety tradeoffs through constraint relaxation or joint optimization, but they often lack structural mechanisms to prevent safety drift. We propose LexiSafe, a lexicographic offline RL framework designed to preserve safety-aligned behavior. We first develop LexiSafe-SC, a single-cost formulation for standard offline safe RL, and derive safety-violation and performance-suboptimality bounds that together yield sample-complexity guarantees. We then extend the framework to hierarchical safety requirements with LexiSafe-MC, which supports multiple safety costs and admits its own sample-complexity analysis. Empirically, LexiSafe demonstrates reduced safety violations and improved task performance compared to constrained offline baselines. By unifying lexicographic prioritization with structural bias, LexiSafe offers a practical and theoretically grounded approach for safety-critical CPS decision-making.
Read the full abstract ▾
All authors Hsin-Jung Yang, Zhanhong Jiang, Prajwal Koirala, Qisai Liu, Cody Fleming, Soumik Sarkar
Machine Learning · Cross-listed from cs.AI
MedClarify: An information-seeking AI agent for medical diagnosis with case-specific follow-up questions
Hui Min Wong, Philip Heesen, Pascal Janetzky et al. (+2 more)
Large
language models (LLMs) are increasingly used for diagnostic tasks in medicine. In clinical practice, the correct diagnosis can rarely be immediately inferred from the initial patient presentation alone. Rather, reaching a diagnosis often involves systematic history taking, during which clinicians reason over multiple potential conditions through iterative questioning to resolve uncertainty. This process requires considering differential diagnoses and actively excluding emergencies that demand immediate intervention. Yet, the ability of medical LLMs to generate informative follow-up questions and thus reason over differential diagnoses remains underexplored. Here, we introduce MedClarify, an AI
agent for information-seeking that can generate follow-up questions for iterative
reasoning to support diagnostic decision-making. Specifically, MedClarify computes a list of candidate diagnoses analogous to a differential diagnosis, and then proactively generates follow-up questions aimed at reducing diagnostic uncertainty. By selecting the question with the highest expected information gain, MedClarify enables targeted, uncertainty-aware
reasoning to improve diagnostic performance. In our experiments, we first demonstrate the limitations of current LLMs in medical
reasoning, which often yield multiple, similarly likely diagnoses, especially when patient cases are incomplete or relevant information for diagnosis is missing. We then show that our information-theoretic
reasoning approach can generate effective follow-up questioning and thereby reduces diagnostic errors by ~27 percentage points (p.p.) compared to a standard single-shot LLM baseline. Altogether, MedClarify offers a path to improve medical LLMs through agentic information-seeking and to thus promote effective dialogues with medical LLMs that reflect the iterative and uncertain nature of real-world clinical
reasoning.
Read the full abstract ▾
All authors Hui Min Wong, Philip Heesen, Pascal Janetzky, Martin Bendszus, Stefan Feuerriegel
Machine Learning · Cross-listed from cs.CL
Representation Collapse in Machine Translation Through the Lens of Angular Dispersion
Evgeniia Tokarchuk, Maya K. Nachesa, Sergey Troshin et al. (+1 more)
Modern neural translation models based on the
Transformer architecture are known for their high performance, particularly when trained on high-resource
datasets. A standard next-
token prediction training strategy, while widely adopted in practice, may lead to overlooked artifacts such as representation collapse. Previous works have shown that this problem is especially pronounced in the representation of the deeper
Transformer layers, where it often fails to efficiently utilize the
geometric space. Representation collapse is even more evident in end-to-end training of continuous-output neural machine translation, where the trivial solution would be to set all vectors to the same value. In this work, we analyze the dynamics of representation collapse at different levels of discrete and continuous NMT
transformers throughout training. We incorporate an existing regularization method based on angular dispersion and demonstrate empirically that it not only mitigates collapse but also improves translation quality. Furthermore, we show that quantized models exhibit similar collapse behavior and that the benefits of regularization are preserved even after quantization.
Read the full abstract ▾
All authors Evgeniia Tokarchuk, Maya K. Nachesa, Sergey Troshin, Vlad Niculae
Machine Learning
Efficient privacy loss accounting for subsampling and random allocation
Vitaly Feldman and Moshe Shenfeld
We consider the
privacy amplification properties of a sampling scheme in which a user's data isused in $k$ steps chosen randomly and uniformly from a sequence (or set) of $t$ steps. This sampling scheme has been recently applied in the context of differentially private optimization (Chua et al., 2024a; Choquette-Choo et al., 2025) and communication-efficient high-dimensional private aggregation (Asi et al., 2026), where it was shown to have utility advantages over the standard Poisson sampling. Theoretical analyses of this sampling scheme (Feldman & Shenfeld, 2025; Dong et al., 2025) lead to bounds that are close to those of Poisson sampling, yet still have two significant shortcomings. First, in many practical settings, the resulting
privacy parameters are not tight due to the approximation steps in the analysis. Second, the computed parameters are either the hockey stick or Renyi divergence, both of which introduce overheads when used in
privacy loss accounting. In this work, we demonstrate that the
privacy loss distribution (PLD) of random allocation applied to any differentially private
algorithm can be computed efficiently. When applied to the Gaussian mechanism, our results demonstrate that the
privacy-utility trade-off for random allocation is at least as good as that of Poisson subsampling. In particular, random allocation is better suited for training via DP-SGD. To support these computations, our work develops new tools for general
privacy loss accounting based on a notion of PLD realization. This notion allows us to extend accurate
privacy loss accounting to subsampling which previously required manual noise-mechanism-specific analysis.
Read the full abstract ▾
Machine Learning · Cross-listed from quant-ph
Quantum Scrambling Born Machine
Marcin Płodzień
Quantum generative modeling, where the Born rule naturally defines probability distributions through measurement of parameterized
quantum states, is a promising near-term application of
quantum computing. We propose a
Quantum Scrambling Born Machine in which a fixed entangling
unitary – acting as a scrambling reservoir – provides multi-
qubit entanglement, while only single-
qubit rotations are optimized. We consider three entangling
unitaries – a Haar random
unitary and two physically realizable approximations, a finite-depth brickwork random
circuit and analog time
evolution under nearest-neighbor
spin-chain
Hamiltonians – and show that, for the
benchmark distributions and system sizes considered, once the entangler produces near-Haar-typical
entanglement the model learns the target distribution with weak sensitivity to the scrambler's microscopic origin. Finally, promoting the
Hamiltonian couplings to trainable parameters casts the generative task as a variational
Hamiltonian problem, with performance competitive with representative classical generative models at matched parameter count.
Read the full abstract ▾
Machine Learning
RLGT: A reinforcement learning framework for extremal graph theory
Ivan Damnjanović, Uroš Milivojević, Irena {\DJ}or{\dj}ević et al. (+1 more)
Reinforcement learning (RL) is a subfield of machine learning that focuses on developing models that can autonomously learn optimal decision-making strategies over time. In a recent pioneering paper, Wagner demonstrated how the Deep Cross-
Entropy RL method can be applied to tackle various problems from extremal graph theory by reformulating them as combinatorial optimization problems. Subsequently, many researchers became interested in refining and extending the framework introduced by Wagner, thereby creating various RL environments specialized for graph theory. Moreover, a number of problems from extremal graph theory were solved through the use of RL. In particular, several inequalities concerning the
Laplacian spectral radius of graphs were refuted, new lower bounds were obtained for certain Ramsey numbers, and contributions were made to the Turán-type extremal problem in which the forbidden structures are cycles of length three and four. Here, we present Reinforcement Learning for Graph Theory (RLGT), a novel RL framework that systematizes the previous work and provides support for both undirected and directed graphs, with or without loops, and with an arbitrary number of edge colors. The framework efficiently represents graphs and aims to facilitate future RL-based research in extremal graph theory through optimized computational performance and a clean and modular design.
Read the full abstract ▾
All authors Ivan Damnjanović, Uroš Milivojević, Irena {\DJ}or{\dj}ević, Dragan Stevanović
Machine Learning
Unified Latents (UL): How to train your latents
Jonathan Heek, Emiel Hoogeboom, Thomas Mensink et al. (+1 more)
We present Unified Latents (UL), a framework for learning latent representations that are jointly regularized by a
diffusion prior and decoded by a
diffusion model. By linking the encoder's output noise to the prior's minimum noise level, we obtain a simple training objective that provides a tight upper bound on the latent bitrate. On ImageNet-512, our approach achieves competitive FID of 1.4, with high reconstruction quality (PSNR) while requiring fewer training FLOPs than models trained on Stable
Diffusion latents. On Kinetics-600, we set a new state-of-the-art FVD of 1.3.
Read the full abstract ▾
All authors Jonathan Heek, Emiel Hoogeboom, Thomas Mensink, Tim Salimans
Machine Learning
Learning a Latent Pulse Shape Interface for Photoinjector Laser Systems
Alexander Klemps, Denis Ilia, Pradeep Kr. Banerjee et al. (+3 more)
Controlling the longitudinal
laser pulse shape in photoinjectors of Free-Electron
Lasers is a powerful lever for optimizing electron beam quality, but systematic exploration of the vast design space is limited by the cost of brute-force pulse propagation simulations. We present a generative modeling framework based on Wasserstein Autoencoders to learn a differentiable latent interface between pulse shaping and downstream beam dynamics. Our empirical findings show that the learned latent space is continuous and interpretable while maintaining high-fidelity reconstructions. Pulse families such as higher-order Gaussians trace coherent trajectories, while standardizing the temporal pulse lengths shows a latent organization correlated with pulse energy. Analysis via principal components and Gaussian Mixture Models reveals a well behaved latent geometry, enabling smooth transitions between distinct pulse types via linear interpolation. The model generalizes from simulated data to real experimental pulse measurements, accurately reconstructing pulses and
embedding them consistently into the learned
manifold. Overall, the approach reduces reliance on expensive pulse-propagation simulations and facilitates downstream beam dynamics simulation and analysis.
Read the full abstract ▾
All authors Alexander Klemps, Denis Ilia, Pradeep Kr. Banerjee, Ye Chen, Henrik Tünnermann, Nihat Ay
Machine Learning
SCOPE: Structured Prototype-Guided Adaptation for EEG Foundation Models with Limited Labels
Jingying Ma, Feng Wu, Yucheng Xing et al. (+5 more)
Electroencephalography (
EEG) foundation models (EFMs) have shown strong potential for transferable representation learning, yet their adaptation in realistic settings remains challenging when only a few labeled subjects are available. We show that this challenge stems from a structural mismatch between noisy, limited supervision and the highly plastic parameter space of EFMs, reflected in three key failure modes: overconfident miscalibration, prediction collapse, and representation drift caused by unconstrained parameter updates. To address these challenges, we propose SCOPE, a Structured COnfidence-aware Prototype-guided framework for label-limited EFM adaptation. SCOPE first constructs cohort-level external supervision to provide persistent guidance and further derives confidence-aware pseudo-labels to select reliable unlabeled samples for adaptation. Building on the constructed external supervision, SCOPE introduces ProAdapter, a lightweight prototype-conditioned adapter that modulates frozen EFMs to preserve pretrained representations. Experiments across 50 label-limited adaptation settings, covering 6
EEG tasks, 5 EFM backbones, and 5%-50% training labeled-subject ratios, show that SCOPE consistently achieves strong performance and efficiency.
Read the full abstract ▾
All authors Jingying Ma, Feng Wu, Yucheng Xing, Qika Lin, Tianyu Liu, Chenyu Liu, Ziyu Jia, Mengling Feng
Machine Learning
CounterFlowNet: From Minimal Changes to Meaningful Counterfactual Explanations
Oleksii Furman, Patryk Marszałek, Jan Masłowski et al. (+3 more)
Counterfactual explanations (CFs) provide human-interpretable insights into model's predictions by identifying minimal changes to input features that would alter the model's output. However, existing methods struggle to generate multiple high-quality explanations that (1) affect only a small portion of the features, (2) can be applied to tabular data with heterogeneous features, and (3) are consistent with the user-defined constraints. We propose CounterFlowNet, a generative approach that formulates CF generation as sequential feature modification using conditional Generative Flow Networks (GFlowNet). CounterFlowNet is trained to sample CFs proportionally to a user-specified reward function that can encode key CF desiderata: validity, sparsity, proximity and plausibility, encouraging high-quality explanations. The sequential formulation yields highly sparse edits, while a unified action space seamlessly supports continuous and categorical features. Moreover, actionability constraints, such as immutability and monotonicity of features, can be enforced at inference time via action masking, without retraining. Experiments on eight
datasets under two evaluation protocols demonstrate that CounterFlowNet achieves superior trade-offs between validity, sparsity, plausibility, and diversity with full satisfaction of the given constraints.
Read the full abstract ▾
All authors Oleksii Furman, Patryk Marszałek, Jan Masłowski, Piotr Gaiński, Maciej Zięba, Marek Śmieja
Machine Learning · Cross-listed from cs.AI
All Leaks Count, Some Count More: Interpretable Temporal Contamination Detection and Mitigation in LLM Backtesting
Zeyu Zhang, Ryan Chen and Bradly C. Stadie
Backtesting LLMs on resolved events assumes models reason only from pre-cutoff knowledge, yet pretrained models inevitably leak post-cutoff knowledge. We introduce a claim-level evaluation framework that decomposes prediction rationales into atomic claims and applies Shapley values to quantify each claim's decision impact, yielding Shapley-DCLR (Shapley-weighted Decision-Critical Leakage Rate) – an interpretable metric measuring what fraction of decision-driving
reasoning is contaminated. We further propose TimeSPEC (Time-Supervised Prediction with Extracted Claims), an inference-time architecture that interleaves temporally-filtered
retrieval with claim-level supervision, producing predictions grounded entirely in pre-cutoff evidence. Across three LLMs, the ablation experiments confirm
retrieval and supervision are jointly necessary; and a three-task probe further illstrates that the performance cost of temporal enforcement scales with each task's reliance on post-cutoff information.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CR
Privacy-Preserving Mechanisms Enable Cheap Verifiable Inference of LLMs
Arka Pal, Louai Zahran, William Gvozdjak et al. (+2 more)
As large
language models (LLMs) continue to grow in size, fewer users are able to host and run models locally. This has led to increased use of third-party hosting services. However, in this setting, there is a lack of guarantees on the computation performed by the inference provider. For example, a dishonest provider may replace an expensive large model with a cheaper-to-run weaker model and return the results from the weaker model to the user. Existing tools to verify inference typically rely on methods from cryptography such as zero-knowledge proofs (ZKPs), but these add significant computational overhead, and remain infeasible for use for large models. In this work, we develop a new insight – that given a method for performing private LLM inference, one can obtain forms of verified inference at marginal extra cost. Specifically, we propose two new protocols which leverage
privacy-preserving LLM inference in order to provide guarantees over the inference that was carried out. Our approaches are cheap, requiring the addition of a few extra
tokens of computation, and have little to no downstream impact. As the fastest
privacy-preserving inference methods are typically faster than ZK methods, the proposed protocols also improve verification runtime. Our work provides novel insights into the connections between
privacy and verifiability in LLM inference.
Read the full abstract ▾
All authors Arka Pal, Louai Zahran, William Gvozdjak, Akilesh Potti, Micah Goldblum
Machine Learning · Cross-listed from stat.ML
MGD: Moment Guided Diffusion for Maximum Entropy Generation
Etienne Lempereur, Nathanaël Cuvelle–Magar, Florentin Coeurdoux et al. (+2 more)
Generating samples from limited information is a fundamental problem across scientific domains. Classical maximum
entropy methods provide principled uncertainty quantification from moment constraints but require sampling via MCMC or Langevin dynamics, which typically exhibit exponential slowdown in high dimensions. In contrast, generative models based on
diffusion and flow matching efficiently transport noise to data but offer limited theoretical guarantees and can overfit when data is scarce. We introduce Moment Guided
Diffusion (MGD), which combines elements of both approaches. Building on the
stochastic interpolant framework, MGD samples maximum
entropy distributions by solving a
stochastic differential equation that guides moments toward prescribed values in finite time, thereby avoiding slow mixing in
equilibrium-based methods. We formally obtain, in the large-
volatility limit, convergence of MGD to the maximum
entropy distribution and derive a tractable
estimator of the resulting
entropy computed directly from the dynamics. Applications to financial time series, turbulent flows, and cosmological fields using wavelet scattering moments yield estimates of negentropy for high-dimensional multiscale processes.
Read the full abstract ▾
All authors Etienne Lempereur, Nathanaël Cuvelle–Magar, Florentin Coeurdoux, Stéphane Mallat, Eric Vanden-Eijnden
Machine Learning
SoftDTW-CUDA-Torch: Memory-Efficient GPU-Accelerated Soft Dynamic Time Warping for PyTorch
Ron Shapira Weber and Oren Freifeld
We present softdtw-cuda-torch, an open-source PyTorch library for computing Soft Dynamic Time Warping (SoftDTW) on GPUs. Our implementation addresses three key limitations of existing GPU implementations of SoftDTW: a hard sequence-length cap of 1024, numerical instability in the backward pass for small smoothing parameters, and excessive GPU
memory consumption from materializing pairwise distance tensors. We introduce (1) tiled anti-diagonal kernel execution that removes the sequence-length constraint, (2) a log-space back-ward pass that prevents floating-point overflow, and (3) a fused distance-computation mode that eliminates the O(BN M ) intermediate distance tensor, achieving up to 98%
memory reduction compared to prior work. The library supports arbitrary sequence lengths, full PyTorch autograd integration, and Soft-DTW Barycenter computation. Code is available at https://github.com/BGU-CS-VIL/sdtw-cuda-torch.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Anti-causal domain generalization: Leveraging unlabeled data
Sorawit Saengkyongam, Juan L. Gamella, Andrew C. Miller et al. (+3 more)
The problem of domain generalization concerns learning predictive models that are robust to distribution shifts when deployed in new, previously unseen environments. Existing methods typically require labeled data from multiple training environments, limiting their applicability when labeled data are scarce. In this work, we study domain generalization in an anti-causal setting, where the outcome causes the observed
covariates. Under this structure, environment perturbations that affect the
covariates do not propagate to the outcome, which motivates regularizing the model's sensitivity to these perturbations. Crucially, estimating these perturbation directions does not require labels, enabling us to leverage unlabeled data from multiple environments. We propose two methods that penalize the model's sensitivity to variations in the mean and covariance of the
covariates across environments, respectively, and prove that these methods have worst-case optimality guarantees under certain classes of environments. Finally, we demonstrate the empirical performance of our approach on a controlled physical system and a physiological signal
dataset.
Read the full abstract ▾
All authors Sorawit Saengkyongam, Juan L. Gamella, Andrew C. Miller, Jonas Peters, Nicolai Meinshausen, Christina Heinze-Deml
Machine Learning
Continual Uncertainty Learning for Robust Control of Nonlinear Systems with Multiple Heterogeneous Uncertainties
Heisei Yonezawa, Ansei Yonezawa and Itsuro Kajiwara
Robust control of mechanical systems with multiple uncertainties remains a fundamental challenge, particularly when nonlinear dynamics and operating-condition variations are intricately intertwined. Although deep reinforcement learning combined with domain randomization has shown promise in mitigating the sim-to-real gap, simultaneously handling all the sources of uncertainty often leads to sub-optimal policies and poor learning efficiency. This study proposes continual uncertainty learning (CUL), a curriculum-based continual learning framework for robust control of nonlinear systems on which multiple heterogeneous uncertainties are simultaneously superimposed. The core idea is to decompose the original control problem into a sequence of continual learning tasks by extending the system into a set of plants whose uncertainties are progressively expanded and diversified, so that the strategy for handling each uncertainty is acquired sequentially. Within this curriculum, the policy is updated across the plant sets under a
memory-efficient anti-forgetting regularization, which preserves the strategies acquired for earlier uncertainties. In parallel, a model-based controller that guarantees a shared baseline performance across all the plant sets is embedded in the learning process, so that the
agent learns only the residual compensation for each uncertainty, thereby substantially enhancing sample efficiency. The proposed framework is applied to the design of an active vibration controller for automotive powertrains as a practical industrial application. Comparative validation demonstrates that the resulting controller remains robust against structural nonlinearities and dynamic variations over a wide range of plant conditions while improving the control performance.
Read the full abstract ▾
Machine Learning
In-Context Learning in Linear vs. Quadratic Attention Models: An Empirical Study on Regression Tasks
Ayush Goel, Arjun Kohli and Sarvagya Somvanshi
Recent work has demonstrated that
transformers and linear
attention models can perform in-context learning (ICL) on simple function classes, such as linear regression. In this paper, we empirically study how these two
attention mechanisms differ in their ICL behavior on the canonical linear-regression task of Garg et al. We evaluate learning quality (MSE), convergence, and generalization behavior of each architecture. We also analyze how increasing model depth affects ICL performance. Our results illustrate both the similarities and limitations of linear
attention relative to quadratic
attention in this setting.
Read the full abstract ▾
Machine Learning
Powering Up Zeroth-Order Training via Subspace Gradient Orthogonalization
Yicheng Lang, Changsheng Wang, Yihua Zhang et al. (+4 more)
Zeroth-order (ZO) optimization provides a gradient-free alternative to first-order (FO) methods by estimating gradients via finite differences of function evaluations, and has recently emerged as a
memory-efficient paradigm for fine-tuning large-scale models by avoiding backpropagation. However, ZO optimization has a fundamental tension between accuracy and query efficiency. In this work, we show that ZO optimization can be substantially improved by unifying two complementary principles: (i) a projection-based subspace view that reduces gradient estimation variance by exploiting the intrinsic low-rank structure of model updates, and (ii)
Muon-style spectral optimization that applies gradient orthogonalization to extract informative spectral structure from noisy ZO gradients. These findings form a unified framework of subspace gradient orthogonalization, which we instantiate in a new method, ZO-
Muon, admitting a natural interpretation as a low-rank
Muon optimizer in the ZO setting. Extensive experiments on large
language models (LLMs) and
vision transformers (ViTs) demonstrate that ZO-
Muon significantly accelerates convergence and achieves a win-win improvement in accuracy and query/runtime efficiency. Notably, compared to the popular MeZO baseline, ZO-
Muon requires only 24.7% of the queries to reach the same SST-2 performance for LLM fine-tuning, and improves accuracy by 25.1% on ViT-L fine-tuning on CIFAR-100.
Read the full abstract ▾
All authors Yicheng Lang, Changsheng Wang, Yihua Zhang, Mingyi Hong, Zheng Zhang, Wotao Yin, Sijia Liu
Machine Learning
TimeOmni-VL: Unified Models for Time Series Understanding and Generation
Tong Guan, Sheng Pan, Johan Barthelemy et al. (+5 more)
Recent time series modeling faces a sharp divide between numerical generation and semantic understanding, with research showing that generation models often rely on superficial pattern matching, while understanding-oriented models struggle with high-fidelity numerical output. Although unified
multimodal models (UMMs) have bridged this gap in
vision, their potential for time series remains untapped. We propose TimeOmni-VL, the first
vision-centric framework that unifies time series understanding and generation through two key innovations: (1) Fidelity-preserving bidirectional mapping between time series and images (Bi-TSI), which advances Time Series-to-Image (TS2I) and Image-to-Time Series (I2TS) conversions to ensure near-lossless transformations. (2) Understanding-guided generation. We introduce TSUMM-Suite, a novel
dataset consisting of six understanding tasks rooted in time series analytics and coupled with two generation tasks. With a calibrated Chain-of-Thought, TimeOmni-VL is the first to leverage time series understanding as an explicit control signal for high-fidelity generation. Experiments confirm that this unified approach significantly improves semantic understanding and numerical precision, establishing a new frontier for
multimodal time series modeling.
Read the full abstract ▾
All authors Tong Guan, Sheng Pan, Johan Barthelemy, Zhao Li, Yujun Cai, Cesare Alippi, Ming Jin, Shirui Pan
Machine Learning
When More Experts Hurt: Underfitting in Multi-Expert Learning to Defer
Shuqi Liu, Yuzhou Cao, Lei Feng et al. (+2 more)
Learning to Defer (L2D) enables a classifier to abstain from predictions and defer to an expert, and has recently been extended to multi-expert settings. In this work, we show that multi-expert L2D is fundamentally more challenging than the single-expert case. With multiple experts, the classifier's underfitting becomes inherent, which seriously degrades prediction performance, whereas in the single-expert setting it arises only under specific conditions. We theoretically reveal that this stems from an intrinsic expert identifiability issue: learning which expert to trust from a diverse pool, a problem absent in the single-expert case and renders existing underfitting remedies failed. To tackle this issue, we propose PiCCE (Pick the Confident and Correct Expert), a surrogate-based method that adaptively identifies a reliable expert based on empirical evidence. PiCCE effectively reduces multi-expert L2D to a single-expert-like learning problem, thereby resolving multi expert underfitting. We further prove its statistical consistency and ability to recover class probabilities and expert accuracies. Extensive experiments across diverse settings, including real-world expert scenarios, validate our theoretical results and demonstrate improved performance.
Read the full abstract ▾
All authors Shuqi Liu, Yuzhou Cao, Lei Feng, Bo An, Luke Ong
Machine Learning
VP-VAE: Rethinking Vector Quantization via Adaptive Vector Perturbation
Linwei Zhai, Han Ding, Mingzhi Lin et al. (+5 more)
Vector Quantized Variational Autoencoders (VQ-VAEs) are fundamental to modern generative modeling, yet they often suffer from training instability and "codebook collapse" due to the inherent coupling of representation learning and discrete codebook optimization. In this paper, we propose VP-VAE (Vector Perturbation VAE), a novel paradigm that decouples representation learning from discretization by eliminating the need for an explicit codebook during training. Our key insight is that, from the neural network's viewpoint, performing quantization primarily manifests as injecting a structured perturbation in latent space. Accordingly, VP-VAE replaces the non-differentiable quantizer with distribution-consistent and scale-adaptive latent perturbations generated via Metropolis–Hastings sampling. This design enables stable training without a codebook while making the model robust to inference-time quantization error. Moreover, under the assumption of approximately uniform latent variables, we derive FSP (Finite Scalar Perturbation), a lightweight variant of VP-VAE that provides a unified theoretical explanation and a practical improvement for FSQ-style fixed quantizers. Extensive experiments on image and audio
benchmarks demonstrate that VP-VAE and FSP improve reconstruction fidelity and achieve substantially more balanced
token usage, while avoiding the instability inherent to coupled codebook training.
Read the full abstract ▾
All authors Linwei Zhai, Han Ding, Mingzhi Lin, Cui Zhao, Fei Wang, Ge Wang, Wang Zhi, Wei Xi
Machine Learning
TIFO: Time-Invariant Frequency Operator for Stationarity-Aware Representation Learning in Time Series
Xihao Piao, Zheng Chen, Lingwei Zhu et al. (+3 more)
Nonstationary time series
forecasting suffers from the distribution shift issue due to the different distributions that produce the training and test data. Existing methods attempt to alleviate the dependence by, e.g., removing low-order moments from each individual sample. These solutions fail to capture the underlying time-evolving structure across samples and do not model the complex time structure. In this paper, we aim to address the distribution shift in the frequency space by considering all possible time structures. To this end, we propose a Time-Invariant Frequency Operator (TIFO), which learns stationarity-aware weights over the frequency spectrum across the entire
dataset. The weight representation highlights stationary frequency components while suppressing non-stationary ones, thereby mitigating the distribution shift issue in time series. To justify our method, we show that the
Fourier transform of time series data implicitly induces eigen-decomposition in the frequency space. TIFO is a plug-and-play approach that can be seamlessly integrated into various
forecasting models. Experiments demonstrate our method achieves 18 top-1 and 6 top-2 results out of 28
forecasting settings. Notably, it yields 33.3% and 55.3% improvements in average MSE on the ETTm2
dataset. In addition, TIFO reduces computational costs by 60% -70% compared to baseline methods, demonstrating strong scalability across diverse
forecasting models.
Read the full abstract ▾
All authors Xihao Piao, Zheng Chen, Lingwei Zhu, Yushun Dong, Yasuko Matsubara, Yasushi Sakurai
Machine Learning
i-PhysGaussian: Implicit Physical Simulation for 3D Gaussian Splatting
Yicheng Cao, Zhuo Huang, Yu Yao et al. (+3 more)
Physical simulation predicts future states of objects based on material properties and external loads, enabling blueprints for both Industry and Engineering to conduct risk management. Current 3D reconstruction-based simulators typically rely on explicit, step-wise updates, which are sensitive to step time and suffer from rapid accuracy degradation under complicated scenarios, such as high-stiffness materials or quasi-static movement. To address this, we introduce i-PhysGaussian, a framework that couples 3D Gaussian Splatting (3DGS) with an implicit Material Point Method (MPM) integrator. Unlike explicit methods, our solution obtains an end-of-step state by minimizing a momentum-balance residual through implicit Newton-type optimization with a GMRES solver. This formulation significantly reduces time-step sensitivity and ensures physical consistency. Our results demonstrate that i-PhysGaussian maintains stability at up to 20x larger time steps than explicit baselines, preserving structural coherence and smooth motion even in complex dynamic transitions.
Read the full abstract ▾
All authors Yicheng Cao, Zhuo Huang, Yu Yao, Yiming Ying, Daoyi Dong, Tongliang Liu
Machine Learning · Cross-listed from stat.ML
Semi-Supervised Learning on Graphs using Graph Neural Networks
Juntong Chen, Claire Donnat, Olga Klopp et al. (+1 more)
Graph neural networks (GNNs) work remarkably well in semi-supervised node regression, yet a rigorous theory explaining when and why they succeed remains lacking. To address this gap, we study an aggregate-and-readout model that encompasses several common message passing architectures: node features are first propagated over the graph then mapped to responses via a nonlinear function. For least-squares estimation over GNNs with linear graph convolutions and a deep ReLU readout, we prove a sharp non-asymptotic risk bound that separates approximation,
stochastic, and optimization errors. The bound makes explicit how performance scales with the fraction of labeled nodes and graph-induced dependence. Approximation guarantees are further derived for graph-smoothing followed by smooth nonlinear readouts, yielding convergence rates that recover classical nonparametric behavior under full supervision while characterizing performance when labels are scarce. Numerical experiments validate our theory, providing a systematic framework for understanding GNN performance and limitations.
Read the full abstract ▾
All authors Juntong Chen, Claire Donnat, Olga Klopp, Johannes Schmidt-Hieber
Machine Learning · Cross-listed from cs.SI
Simplify to Amplify: Achieving Information-Theoretic Bounds with Fewer Steps in Spectral Community Detection
Sie Hendrata Dharmawan and Peter Chin
We propose a streamlined spectral
algorithm for community detection in the two-community
stochastic block model (SBM) under constant edge density assumptions. By reducing algorithmic complexity through the elimination of non-essential preprocessing steps, our method directly leverages the spectral properties of the adjacency
matrix. We demonstrate that our
algorithm exploits specific characteristics of the second eigenvector to achieve improved error bounds that approach information-theoretic limits, representing a significant improvement over existing methods. Theoretical analysis establishes that our error rates are tighter than previously reported bounds in the literature. Comprehensive experimental validation confirms our theoretical findings and demonstrates the practical effectiveness of the simplified approach. Our results suggest that algorithmic simplification, rather than increasing complexity, can lead to both computational efficiency and enhanced performance in spectral community detection.
Read the full abstract ▾