Machine Learning
Minimax Quantile Lower Bounds for Interactive Statistical Decision Making with Privacy
Raghav Bongole, Amirreza Zamani, Tobias J. Oechtering et al. (+1 more)
Minimax risk and regret are expectation-based criteria and do not capture rare but consequential failures. To address this concern, we develop a $\delta$-explicit minimax-
quantile theory for interactive statistical decision making (ISDM). We first provide structural relations between minimax
quantiles, lower minimax
quantiles, and minimax risk. This includes a
quantile-to-expectation conversion and an equivalence between strict and lower minimax
quantiles outside a countable set of confidence levels. We then derive two converse tools for ISDM: a high-probability interactive Fano's method and a high-probability interactive Le Cam's method. Then, we show that mutual-information (MI)
privacy can be handled in the same framework by restricting the admissible decision class. For coordinatewise Gaussian privatization, we derive a two-point template that isolates the
privacy-induced variance inflation. We instantiate this template for Gaussian mean estimation, and use the same two-point strategy directly for two-armed Gaussian bandits. We then derive a minimax
quantile lower bound for the $K$-armed Gaussian bandit problem, showing that the interactive Fano method captures the exploration cost over multiple possible best arms. The resulting lower bounds are explicit in the confidence level $\delta$ and in the
privacy budget for the private problems. They yield $\log(1/\delta)/n$ scaling for squared-error Gaussian mean estimation, $\sqrt{T\log(1/\delta)}$ scaling for two-armed bounded-mean Gaussian bandits, and $\sqrt{KT\log(1/\delta)}$-type scaling for the $K$-armed bandits, with
privacy appearing through a Gaussian variance-inflation factor for the private problems.
Read the full abstract ▾
All authors Raghav Bongole, Amirreza Zamani, Tobias J. Oechtering, Mikael Skoglund
Machine Learning
FLFL: Federated Latent Factor Learning for Private Recovery of Spatio-Temporal Signals
Chengjun Yu, Di Wu, Yi He et al. (+1 more)
Wireless sensor network (WSNs) stands out as a burgeoning and promising domain in intelligent sensing. Owing to various factors such as sudden sensor malfunctions or deliberate shutdown of partial nodes to save energy, the collected sensing signals from WSNs commonly have massive missing data, leading to adverse effects on subsequent analysis or decision-making. Latent factor learning (LFL) has proven to be highly effective in recovering the missing data for WSNs. However, the existing LFL models require the collected sensing signals to be maintained in one central place like a central server, which is becoming unacceptable for data owners who are getting increasingly
privacy-sensitive. To address this issue, this paper innovatively proposes a
federated latent factor learning (FLFL) model for
privacy-preserving spatio-temporal signal recovery. Its main idea is two-fold: 1) it designs a sensor-level
federated learning framework based on LFL, where each sensor only needs to upload gradient information rather than raw data for training a
privacy-preserving recovery model, and 2) it incorporates the spatio-temporal correlation into the designed
federated learning framework as the regularization constraint to improve its recovery accuracy. With such designs, FLFL can not only accurately recover the missing data of WSNs but also ensure data owners'
privacy-preserving of raw data. To evaluate the proposed FLFL model, extensive experiments have been conducted on four real-world WSN
datasets. The results demonstrate that FLFL significantly outperforms eight state-of-the-art
federated and non-
federated signal recovery models in terms of recovery accuracy with
privacy-preserving.
Read the full abstract ▾
All authors Chengjun Yu, Di Wu, Yi He, Jia Chen
Machine Learning
FlowTrain: Flow-Based Decoupled Training for Industrial-Grade Vision-Language Models
Zhida Jiang, Zhaolong Xing, Yang Pei et al. (+14 more)
Industrial-grade distributed training of
vision-
language models (VLMs) remains far less efficient than that of unimodal LLMs. Existing solutions either follow a monolithic design that assigns uniform parallelism to heterogeneous modules or adopt a disaggregated deployment that separates modules while executing them as a batch-synchronized pipeline. In this paper, we highlight that the above solutions are still not sufficient, and VLM training can be further decoupled. To this end, we present FlowTrain, a flow-based decoupled training framework that reformulates VLM training as a producer-consumer dataflow coordinated through a unified
memory pool. The encoder and backbone can progress independently over a global virtual address space. Since this execution decoupling fundamentally changes the optimization objective of allocation and scheduling, FlowTrain further introduces a heterogeneous parallel allocator that assigns module-specific parallelism strategies by solving a throughput matching problem. The dynamic packing scheduler is used to construct balanced microbatches at runtime according to the actual LLM-side computation cost. Extensive experiments on real-world workloads show that FlowTrain achieves over 50% MFU and up to 1.7x throughput improvement, narrowing the efficiency gap to LLM-only training.
Read the full abstract ▾
All authors Zhida Jiang, Zhaolong Xing, Yang Pei, Xiaolong Chen, Yuanhang Xiao, Chengzhi Huang, Xiyu Liu, Haopeng Liu, Qingyuan Sang, Lingfeng Zhou, Jiaxing Wang, Zicheng Zhang, Wenzhe Wang, Xinyu Liu, Yan Li, Zhen Chen, Ke Zhang
Machine Learning
PeLAP-A: Adaptive Latent Pruning for Lightweight Latent Diffusion Models
Kissa Zahra and Zaib Un Nisa
Latent
diffusion models achieve strong generative performance by operating in a compressed latent space produced by a variational autoencoder (VAE). However, it remains unclear whether all latent channels contribute equally to the
diffusion process, or whether significant redundancy exists. We introduce PeLAP-A (Adaptive Latent
Pruning for
Diffusion), a lightweight framework that augments a standard latent
diffusion pipeline with a learnable channel-wise importance predictor. A two-layer MLP operating on globally pooled latent features produces a soft mask that suppresses unimportant latent channels before they enter the denoising UNet. The entire system is trained jointly on CIFAR-10 under a combined
diffusion, reconstruction, and sparsity loss. Experiments reveal a striking result: under aggressive sparsity regularization (lambda = 0.01), the importance predictor drives all latent channels to near-zero yet the denoising UNet achieves lower
diffusion loss (0.0236 vs. 0.0240) and lower VAE reconstruction MSE (22.59 vs. 24.67) compared to the unpruned baseline. We term this the sparsity collapse phenomenon and provide an analysis of why it occurs and what it reveals about the information requirements of latent
diffusion models. These findings constitute an exploratory study of sparsity dynamics in latent
diffusion training, and demonstrate that denoising UNets can remain remarkably robust to latent channel suppression even under aggressive regularization. Code is available at: https://github.com/kissasium/PeLAP-A.git.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.IR
Who Owns the AI Recommendation? A Multi-Industry Empirical Map of Brand Category Ownership Across Large Language Models
Dmitrij Żatuchin
This exploratory study measures brand inclusion across five industries, 50 brands and 250 queries, each put five times to GPT-5.2, Gemini 3 Flash and Perplexity sonar-pro in February and September 2026 (3,614 and 3,750 scored answers). Category Inclusion Rate, Recommendation Share, Competitive
Vacuum Index and Co-Mention Asymmetry have stated denominators. February inclusion rates sit close together across an industry's sampled brands (mean Gini 0.30), while at least one brand is named in 80% or more of answers to 204 of 250 queries. Vacuums occur in 7.6% of queries; provisional open-vocabulary model readings suggest most reflect the sampled brand list. The partially pre-specified September replication shows strong cross-date Recommendation Share correlation (Spearman 0.994), unchanged
vacuum prevalence and agreement of 60.8% against February's 57.2%. The descriptive size association persists. Fixed margins do not account for all co-mention structure: 31 ordered pairs depart from the September null. Agreement exceeds the query-independent null, and both parametric-pair advantage intervals are positive. All ten clustering seeds yield zero emergent clusters. Matcher validation remains provisional pending author checks. The statistics describe system output and identify no causal mechanism.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Domain Adaptation Under Wireless Network Constraints: When Does It Become Green?
Illyyne Saffar, Aurélie Boisbunon and Shruti Bothe
The deployment of data-driven models in 6G wireless networks is increasingly challenged by frequent distribution shifts that degrade performance over time. Unsupervised Domain Adaptation (UDA) offers an alternative approach by adapting the trained model to a shifted domain without requiring labels. However, UDA pipelines are often more complex than single-task training due to additional modules and optimization procedures, raising a practical question: do the benefits of adaptation come at a higher energy cost, and how does this trade-off compare to retraining when labeling effort is also considered? In this work, we investigate the energy consumption of UDA and compare it to single task. We further propose a way to determine the minimum number of target domains for which UDA becomes more energy-efficient than retraining, taking into account the labeling cost. Our results aim to clarify when UDA should be preferred over classical train-from-scratch approaches from an energy and labeling-aware perspective.
Read the full abstract ▾
Machine Learning
Prime Fourier Embeddings: A Principled Basis for Modular Arithmetic
Hyunsang Hwang, Suhyun Bae and Donghun Lee
Numbers have algebraic structure that standard neural
embeddings often fail to expose. We introduce
Prime Fourier Embeddings (PFE), which encode integers as
prime-indexed (cos, sin) pairs derived from the harmonic analysis of Q, providing a pre-structured representation in which modular arithmetic reduces to selecting the relevant
prime channel rather than discovering algebraic structure from scratch. We prove that any linear map equivariant with respect to the product group action on PFE must be block-diagonal with one independent block per
prime – a consequence of Schur's lemma applied to the resulting character decomposition. For square-free composite
moduli, the Chinese Remainder
Theorem predicts which
prime channels are task-relevant. Both predictions are confirmed empirically: ablation studies show specialization ratios exceeding 500x between task-relevant and task-irrelevant channels, with perfect in-distribution test accuracy across all square-free composite
moduli tested.
Read the full abstract ▾
Machine Learning
EvoRubrics: Dynamic Rubrics as Rewards via Adversarial Co-Evolution for LLM Reinforcement Learning
Hongxin Ding, Baixiang Huang, Yue Fang et al. (+6 more)
Rubric-based rewards offer interpretable and fine-grained optimization signals for reinforcement learning in open-ended tasks where verifiable answers are unavailable. However, pre-constructed rubrics remain static throughout training, creating a fundamental mismatch with the evolving policy: fixed criteria gradually lose discriminative power as the model improves, leading to reward saturation and potential hacking. Recent dynamic rubric methods partially address this but rely on external frontier models or ground-truth answers, and update rubrics only at coarse granularity. We propose EvoRubrics, a co-evolutionary RL framework where a Policy LLM and a Rubric Generator jointly improve through
adversarial interaction within each training step. As the policy improves under the rubric generator's guidance, the rubric generator adapts its criteria to remain discriminative and informative, enabling evaluation to track the policy in real time and naturally inducing an automatic curriculum. Experiments show that EvoRubrics consistently outperforms static and dynamic rubric baselines across
benchmarks. The learned Rubric Generator further generalizes as a transferable reward model. Notably, even a fully self-supervised variant without any external supervision achieves meaningful gains, suggesting that co-
evolution between generation and evaluation alone can provide sufficiently rich learning signals. Our code is publicly available at https://anonymous.4open.science/r/EvoRubrics-2155/.
Read the full abstract ▾
All authors Hongxin Ding, Baixiang Huang, Yue Fang, Weibin Liao, Zheng Li, Jinyang Zhang, Zhijing Wu, Junfeng Zhao, Yasha Wang
Machine Learning
Counterfactual learning of new adaptive instructional policies using logged data
Samuel Girard, Sein Minn, Amel Bouzeghoub et al. (+1 more)
Optimizing instructional policies in Intelligent Tutoring Systems (ITS) typically requires costly online experimentation or student simulators that may fail to capture real-world dynamics. This paper introduces an offline contextual bandit framework that learns new adaptive policies directly from logged interaction data. By mapping student-item interactions onto a continuous latent proficiency-difficulty scale using a Rasch model, we cast the tutoring process as a continuous
stochastic bandit problem. We propose a novel reward function designed to optimize ”flow” by balancing task challenge with student success. Our approach includes a round-specific behavior policy estimation that serves as both a propensity model for off-policy evaluation and a diagnostic tool for ITS adaptivity. We demonstrate the efficacy of this framework across four large-scale real-world
datasets, achieving consistent policy improvements over the logged behavior policy. The results show that effective instructional policies can be learned and visualized within seconds of computation, providing a scalable path for improving adaptive learning systems without further data collection.
Read the full abstract ▾
All authors Samuel Girard, Sein Minn, Amel Bouzeghoub, Jill-Jênn Vie
Machine Learning
A Novel Approach to Temporal QoS Estimation via Extended Kalman Filter-Incorporated Latent Feature Analysis
Ye Yuan, Song Wang, Hongxun Zhou et al. (+2 more)
Predicting temporal Quality of Service (QoS) data is critical for optimizing network services and rationalizing resource allocation in cloud computing and service-oriented systems. Existing mainstream methods have achieved promising predictive performance. However, their purely data-driven manner limits their ability to capture non-stationary temporal patterns, thereby leading to accuracy degradation when temporal QoS data exhibits fluctuations. To tackle this limitation, we propose a novel Extended Kalman Filter-Enhanced Latent Feature Analysis (EKL) model to perform efficient and accurate temporal QoS prediction from the perspective of bidirectional model-data-driven learning. Its main idea is three-fold: a) designing a model-driven feature producer to obtain the temporal latent features to capture the intricate temporal pattern following the principle of an Extended Kalman Filter; b) building a data-driven feature producer based on the alternating least squares
algorithm to identify time-invariant latent features describing intrinsic user-service characteristics; c) exploiting a density-oriented parallel strategy that achieves workload balancing by sorting users in accordance with their service invocation density, which effectively elevates computational efficiency. In addition, we provide a rigorous theoretical analysis to formally prove the convergence of the proposed EKL. Experimental evaluations conducted on real-world temporal QoS
datasets reveal that our proposed EKL surpasses existing state-of-the-art models with respect to both computational efficiency and prediction accuracy for missing temporal QoS data.
Read the full abstract ▾
All authors Ye Yuan, Song Wang, Hongxun Zhou, Ling Wang, Xin Luo
Machine Learning · Cross-listed from cs.CV
From Point Estimates to Distributions: GMM Pooling for MIL in Preterm Birth Prediction
Hussain Alasmawi, Numan Saeed, Soha Said et al. (+1 more)
Preterm birth (PTB) prediction can enable targeted surveillance and timely intervention, yet most ultrasound-based models use a single selected transvaginal ultrasound (TVUS) frame per patient despite routine exams acquiring multiple cervical images. We formulate PTB prediction as a multiple instance learning (MIL) problem, representing each patient as a variable-sized bag of TVUS images with a single outcome label. To move beyond standard MIL aggregators that collapse a bag into a point estimate, we propose a Gaussian Mixture Model (GMM) pooling, which summarizes all images in a bag into a fixed-length representation by modeling their feature distribution. This design captures intra-patient variability. We evaluate the method on a private clinical cohort and on a public lymph node metastasis
benchmark. For PTB prediction, GMM pooling improves over the instance-based model PR-AUC from 0.44 to 0.56. On the lymph node
benchmark, it achieves state-of-the-art performance with 0.91 F1-score and 0.89 ROC-AUC for classification and 0.18 MAE for regression. The code is publicly available at https://github.com/HussainAlasmawi/GMM_Pooling.
Read the full abstract ▾
All authors Hussain Alasmawi, Numan Saeed, Soha Said, Mohammad Yaqub
Machine Learning · Cross-listed from cs.SE
EnerInfer: Energy-Aware On-Device LLM Inference
Bohua Zou, Nian Liu, Binqi Sun et al. (+6 more)
On-device LLM inference is increasingly attractive for
privacy-preserving, reliable, and cost-effective deployment, yet its energy and thermal costs remain a critical bottleneck. Existing systems primarily optimize for decoding speed, implicitly assuming that faster execution is always preferable. We show instead that on-device LLM inference often has exploitable configuration slack: modestly lowering NPU and
memory frequencies preserves quality of experience (QoE) while substantially improving energy efficiency and reducing heat. Realizing this opportunity in production is challenging. The most energy-efficient NPU/DDR setting varies with the model, inference engine, platform, and runtime conditions, with no stable ranking across configurations. Commercial devices further lack component-level power sensing, and shell temperature evolves with request arrivals, response lengths, and thermal history. To address these challenges, we propose EnerInfer, the first on-device LLM inference framework that jointly manages energy efficiency, throughput, and thermal comfort for LLM workloads. EnerInfer replaces per-model profiling and sensor-heavy control with disaggregated, model-structure-aware prediction and ranking-driven online feedback. It predicts throughput and power for unseen LLMs across NPU/DDR frequency settings, selects QoE-satisfying efficient configurations under runtime interference, and uses lightweight limited-horizon thermal prediction to dynamically switch between energy-optimized and thermally constrained inference. Evaluations on real-world LLMs show that EnerInfer improves energy efficiency by up to 65%, 12%, and 24% on phones, a laptop, and a development board, respectively, without QoE violation.
Read the full abstract ▾
All authors Bohua Zou, Nian Liu, Binqi Sun, Matteo Mascherin, Debayan Roy, Yutao Liu, Yu Peng, Ning Jia, Haibo Chen
Machine Learning
Group-Graph Policy Optimization for Long-Horizon Agentic Reinforcement Learning
Yunan Wang, Minghui Song, Zihan Zhang et al. (+6 more)
Group-based Reinforcement Learning (RL) has significantly enhanced Large
Language Models (LLMs) in agentic scenarios. To achieve finer-grained policy updates, recent agentic RL frameworks have shifted from trajectory-level to step-level training. However, long-horizon agentic RL suffers from severe reward sparsity and delay, as feedback is often deferred for dozens of interaction steps. While existing step-level frameworks refine training granularity, their credit assignment remains coarse-grained and still treats
agent exploration as isolated, linear trajectories. This oversimplified perspective ignores the inherent graph structure of state transitions, leading to high-variance state-value estimation and myopic, localized credit assignment. To overcome these critical bottlenecks, we propose Group-Graph Policy Optimization (G2PO), a novel group-based RL
algorithm tailored for multi-turn agentic tasks. G2PO explicitly transforms linear interaction trajectories into a global state-transition graph. By aggregating identical observations across different trajectories, we introduce group-aggregation state-value estimation that reduces sampling variance and trajectory-dependent bias. Furthermore, we redefine
agent actions as transitions between state nodes and propose an edge-centric advantage estimation strategy. By globally standardizing Temporal Difference (TD) errors across the entire graph, G2PO explicitly identifies and prioritizes critical transitions that drive absolute task progress. Extensive experiments on representative long-horizon
benchmarks-WebShop, ALFWorld, and AppWorld-demonstrate that G2PO substantially outperforms state-of-the-art
prompt-based and RL baselines, achieving remarkable success rate improvements of up to 22.2% over GRPO.
Read the full abstract ▾
All authors Yunan Wang, Minghui Song, Zihan Zhang, Shaohan Huang, Haizhen Huang, Furu Wei, Weiwei Deng, Feng Sun, Qi Zhang
Machine Learning
Do Sparse Autoencoders Learn Meaningful Concept Hierarchies?
Nils Grandien, David Steinmann, Felix Friedrich et al. (+1 more)
Sparse autoencoders (SAEs) have become an important tool for unsupervised concept discovery in large models. To make the resulting feature spaces more interpretable and manageable, recent approaches have begun imposing hierarchical structure, either explicitly or as an implicit effect of training constraints, yet rigorous comparison remains difficult. There are no agreed-upon requirements for what a meaningful feature hierarchy should satisfy, and evaluation has largely relied on qualitative illustrations with fragmented quantitative protocols. To address this, we derive a set of key requirements for generalization/specialization hierarchies in unsupervised concept discovery, drawing on semantic net and taxonomy research alongside recent SAE work, and use them to derive a concrete evaluation protocol. Applying this protocol to current SAE approaches trained on visual data, we find that while feature spaces generally provide a basis for sensible hierarchies, establishing good hierarchical structure remains challenging. In particular, feature absorption, both in its well-known hard form and in a continuous, soft form, systematically compromises hierarchy quality, pointing to a fundamental tension that future approaches will need to navigate.
Read the full abstract ▾
All authors Nils Grandien, David Steinmann, Felix Friedrich, Kristian Kersting
Machine Learning · Cross-listed from math.ST
Generalized nonparametric regression in reproducing kernel Hilbert spaces: Consistency and rates of convergence
Ioannis Kalogridis
We develop a comprehensive theory for regularized M-estimation in reproducing kernel
Hilbert spaces. Under mild conditions on the loss we establish existence and measurability of the
estimator, covering a wide range of convex and non-convex losses, including bounded robust losses. We further prove sharp rates of convergence with an explicit bias-variance decomposition governed by a novel complexity measure. We show that the variance is independent of misspecification, while the bias depends on a source condition parameter known in the learning literature. For tensor product Sobolev spaces we obtain new rates that connect to spaces of functions with dominating mixed smoothness, substantially extending existing results and explaining why these
estimators circumvent the curse of dimensionality. Our methodology, combining elements from both functional analysis and empirical process theory, allows for an asymptotic linearisation of the objective function that avoids both closed-form solutions and global Lipschitz assumptions, and may be of independent interest. The
estimators are implemented in C++ and theory is supported by numerical experiments.
Read the full abstract ▾
Machine Learning
Neural Architecture Search of Sample Reweighting Networks for Complex Distribution Shift
Keisuke Sugawara, Kento Uchida and Shinichi Shirakawa
Sample reweighting is a major approach to addressing distribution shifts, such as label noise and class imbalance. Meta-Weight-Net (MW-Net) is a promising sample reweighting network that computes weights based on classification loss. Although MW-Net improves prediction performance under a single type of distribution shift using a simple neural network, its performance degrades when facing both label noise and class imbalance, where it is hard to determine appropriate weights solely from classification loss and using a simple network. In this study, we introduce neural architecture search to MW-Net to mitigate such performance degradation. Using the tree-structured Parzen
estimator, we explore the optimal number of hidden layers and nodes and select the most suitable intermediate layer in the classification model to serve as the input for MW-Net. Experimental results on the CIFAR-10 and CIFAR-100
datasets that were modified to include both label noise and class imbalance demonstrate the effectiveness of neural architecture search for MW-Net.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Subject-Level Unknown-Identity Identification from Leap Motion Controller 2 Hand Landmarks
Bahar Moharrer, Susanna Cifani, Marco Raoul Marini et al. (+2 more)
This work studies subject recognition from Leap Motion Controller 2 (LMC2) hand landmark data under a subject-level unknown-identity identification protocol on the Multi View Leap2 Hand Pose (ML2HP)
dataset. Using only the landmark modality, we retain the original
geometric representation and enrich it with fingertip-to-palm distances and palm-normalized inter-finger angular descriptors. Evaluation is performed under a Leave-One-Subject-Out (LOSO) protocol in which, for each outer fold, one subject is excluded from the enrolled set and treated as unknown at test time. To avoid tuning on the true outer unknown subject, the unknown-rejection threshold is selected in an inner validation step by temporarily withholding one enrolled subject from the inner gallery and using it only for threshold estimation. We compare a tree ensemble baseline with two neural alternatives: a learned
embedding baseline based on centroid matching and cosine-similarity-based rejection, and an MLP+OpenMax model, which represents a more established open-set recognition approach. Under this evaluation setup, Extra Trees remains the strongest overall method, indicating that the main challenge on this
benchmark is not enrolled-subject discrimination alone, but robust score separation between known and unknown probes. The results support the feasibility of compact, interpretable landmark-based descriptors for contactless hand-based unknown-subject rejection and identification on a small-cohort
dataset.
Read the full abstract ▾
All authors Bahar Moharrer, Susanna Cifani, Marco Raoul Marini, Luigi Cinque, Maria De Marsico
Machine Learning · Cross-listed from cond-mat.dis-nn
Scalable Physics-Inspired Transformers for Spin Glasses
Lu Zhong, Wenli Duan, Jing Liu et al. (+2 more)
Efficient sampling of the Boltzmann distribution in frustrated
spin glasses is central to statistical mechanics and combinatorial optimization. Despite advances in machine-learning-based approaches, two issues persist: limited understanding of why variational models fail to benefit from increased scale, unlike the monotonic scaling law of large
language models; and high computational cost on large systems that negates advantages over classical sampling methods. Here, we develop a physics-inspired
transformer with interpretable sparse
attention and
spin-tailored positional
embeddings to address these challenges. By further leveraging FlashAttention for parallel ancestral sampling, it achieves up to two orders of magnitude speedup over vanilla variational autoregressive networks, enabling neural-network simulations of
spin-glass systems to unprecedented sizes on a single GPU. It can resolve full probability distributions, free energies, and overlap statistics across temperatures, for Sherrington-Kirkpatrick and 2D or 3D Edwards-Anderson models, where existing machine-learning methods encounter limitations at certain temperatures. This framework thus establishes a scalable paradigm for frustrated
spin-glass systems.
Read the full abstract ▾
All authors Lu Zhong, Wenli Duan, Jing Liu, Pan Zhang, Ying Tang
Machine Learning
Walk fast but be careful: Understanding Parallel Sampling in Masked Diffusion
Vansh Bansal, Cholyeon Cho, Syamantak Kumar et al. (+2 more)
In this paper, we use random walks on graphs as a verifiable sandbox for studying parallel sampling strategies in masked
diffusion models (MDMs). We train an MDM on random walk samples from a fixed graph. The graph and transition kernel are never shown to the model and serve as latent structure that is both controllable and enables evaluation. The framework provides a validity check for generated walks and a measure of distributional fidelity through the estimated transition kernel. Using simple graphs, we theoretically prove that parallel unmasking via widely used scores such as lowest
entropy is not uniformly better than random parallel sampling; even with exact conditional probabilities, performance critically depends on the conditional dependence structure induced by the graph, a phenomenon difficult to isolate in
benchmarks like Sudoku. We also develop training-free bisection samplers for MDMs, which take logarithmically many steps in the sequence length and are provably exact for random walks if the learned marginals are exact. Experiments on graph-walk tasks confirm that different parallel samplers perform better on different graph structures. Experiments on pretrained MDMs show that bisection-style samplers provide strong speed-quality tradeoffs on OpenWebText generation and
reasoning benchmarks including GSM8K, MBPP, and HumanEval. Together, these results use graph walks to uncover conditional dependence as a key principle of parallel MDM sampling and translate this insight into efficient samplers that transfer to
language generation and
reasoning.
Read the full abstract ▾
All authors Vansh Bansal, Cholyeon Cho, Syamantak Kumar, Sujay Sanghavi, Purnamrita Sarkar
Machine Learning
TaLK: Text-attributed Graph Dataset Distillation via Coupling Language Model with Graph-Aware Kernel
Yeongho Kim, Yeonje Choi and Kijung Shin
Text-attributed graphs (TAGs) are widely used in many real-world domains, and learning on TAGs requires jointly modeling text semantics and graph structure. A standard approach for modeling TAGs is to combine a
language model (LM) and a graph neural network (GNN), but joint training is computationally expensive and difficult to scale.
Dataset distillation is a promising way to reduce training costs, but existing methods are not well suited to TAGs because they are typically designed for a single modality or still require repeatedly training expensive LM-GNN models on the full
dataset during
distillation. To address this, we propose TaLK, an effective
dataset distillation method for TAGs that couples an LM with a graph-aware neural tangent kernel. This design enables efficient
dataset distillation, avoiding repeated joint training on the full
dataset while reflecting both textual and structural information for effective TAG learning. Experiments on multiple TAG
benchmarks show that TaLK consistently outperforms existing baselines and achieves up to 97% of full-
dataset performance with only 1% synthetic data.
Read the full abstract ▾
Machine Learning
Topological Out-of-Domain Generalization in Dynamical Systems Reconstruction
Georg Trede, Charlotte Ricarda Doll, Elias Weber et al. (+1 more)
Predicting the behavior of dynamical systems (DS) beyond the dynamical and parameter regimes observed in training is a pivotal and essentially unresolved problem in scientific ML. It is central to any good scientific theory, which we expect to be able to make predictions about regimes not covered by currently available data. Recent hierarchical and hyper-network guided approaches for DS reconstruction (DSR) enable training on many DS simultaneously, and revealed that extracted latent features are often related to crucial control parameters of the underlying DS that varied across the training corpus. However, true out-of-domain
forecasting abilities of these models, e.g., across tipping points, remain limited, and fine-tuning, or even full model retraining, on time series from the new dynamical regime is usually required. Here, we mathematically analyze the root of these limitations in previous model formulations and identify three core shortcomings rooted in a mismatch between structural assumptions of the reconstruction model and typical properties of physical systems. We propose a combination of remedies for these shortcomings, most importantly feature splitting, and furthermore derive a closed-form bound on the reliable extrapolation range. We demonstrate empirically that our techniques allow for accurate zero-shot prediction into new dynamical regimes, outside the observed training regime, as, e.g., encountered across tipping points.
Read the full abstract ▾
All authors Georg Trede, Charlotte Ricarda Doll, Elias Weber, Daniel Durstewitz
Machine Learning
Attacking the Trusted Imagination: Oracle-Level Integrity Attacks on Imagine-then-Act World Models
Linghan Chen, Kaiyan Ji and Minyu Guo
Many recent
vision-
language-action (VLA) policies adopt an imagine-then-act design. A world-action model (WAM) first imagines a short future as a latent trajectory z~, on which the action is then conditioned. We identify this trusted imagination, rather than the reactive policy, as the exposed attack surface. A downstream oracle, such as a safety gate, a visual model-predictive-control (MPC) planner, or an imagine-then-check verifier, consumes z~ as a prediction of the future. The robustness of the policy therefore does not entail the robustness of systems that rely on the WAM. The underlying phenomenon is an asymmetry. Corrupting the imagination is easy, since it requires only displacing z~ from its natural-future
manifold. Steering it precisely is hard, since it must reach a specified on-
manifold target. We adopt a capability-based threat model with an L-infinity-bounded observation perturbation. The attacker applies projected gradient descent through the fully differentiable observation-to-imagination map. The same off-
manifold property motivates a parameter-free denoiser detector. We evaluate three targets: RynnVLA-002, LingBot-VA, and LaDi-WM. Untargeted corruption is roughly 60x stronger than random and is detected at AUC 1.0. Targeted control remains bounded. An adaptive attacker evades detection only by forgoing corruption. The reactive policy remains robust to corrupted imagination. A native imagination-driven MPC, however, exhibits the first adversary-specific task failure (at epsilon=0.01, success 0.70 versus 0.05; Fisher p < 10^-4).
Read the full abstract ▾
Machine Learning
PG-MAP: Joint MAP Optimization for Inference-Time Alignment of Diffusion and Flow-Matching Models
Ruolan Sun and Pawel Polak
Inference-time
alignment of pretrained text-to-image models is typically performed along a single control axis, such as classifier-free guidance,
attention editing, or reward-based latent perturbations. This limitation prevents modeling joint dependencies between conditioning and latent variables and hinders transfer across generative transports. We propose PG-MAP, a training-free framework that formulates inference-time
alignment as a trajectory-level Gibbs-MAP / proximal energy optimization over the conditioning $c$ and latent state $z_t$ via a forward-consistency coupling, optionally guided by a frozen preference reward. This joint formulation enables coordinated updates across modalities while remaining compatible with both
diffusion and flow-matching models through transport-specific adaptations. Across
diffusion backbones (SD~1.5, SDXL), PG-MAP consistently improves
alignment metrics such as PickScore and Aesthetic, and can be effectively combined with tuned classifier-free guidance to achieve the strongest overall performance. On flow-matching models (SD3.5-medium), the framework reduces to a latent-only variant, achieving $\mathbf{91.9\%}$ PickScore and $75.7\%$ HPS win rates against a static baseline, with controlled experiments ruling out noise-related artifacts. Human evaluations further confirm consistent preference over strong baselines, including tuned CFG and compute-matched universal guidance. Finally, an oracle-routing analysis shows that the relative importance of conditioning and latent optimization depends on
prompt types, surfacing further headroom that a per-
prompt selector could exploit.
Read the full abstract ▾
Machine Learning · Cross-listed from eess.AS
Domain-incremental audio classification using domain-specific experts and prototype classifier
Jongyeon Park, Do-Hyeon Lim, Sang-won Park et al. (+4 more)
This technical report presents submission systems for Task 7(domain-incremental audio classification) of the DCASE 2026 Challenge. The main obstacle is that, the system is unable to access to past or future domain's data at once. We approached domain-incremental learning (DIL) as a frozen-feature replay problem. At each incremental stage, one or two compact experts are trained and then kept fixed; at the final stage, the penultimate features from all frozen experts are concatenated and used to train a lightweight per-class prototype classifier solely on cached features. This design prevents catastrophic forgetting by preserving each expert models at inference. To retain earlier-domain knowledge without storing raw audio, some experts were trained with DeepInversion-based generative replay. A cross-stage regression imputer was trained to fill the expert feature slots that did not yet exist at an ealier stage. We submit four fully DIL-compliant systems: three systems based on diverse frozen five-expert backbones and their cross-stack ensemble achieving 78.15% micro / 77.03% macro on the development set, outperforming every individual backbone on both evaluations.
Read the full abstract ▾
All authors Jongyeon Park, Do-Hyeon Lim, Sang-won Park, Hong Kook Kim, Kyungdeuk Ko, Hyeongcheol Geum, Jeong Eun Lim
Machine Learning
DT-GOL: Dual-Track Geometric Online Learning in Nonstationary Environment with Label Delay
Yulin Wang, Yi He, Dianlong You et al. (+1 more)
Online learning is crucial for handling complex data streams in big data applications. Recent research has begun to focus on dynamic scenarios, i.e., non-stationary environments. However, a crucial yet often overlooked aspect is label latency, where new data may not receive labels in time due to the slow and expensive labeling process, thus hindering rapid adaptation to dynamic environments. To resolve this impasse, we propose Dual-Track Geometry Online Learning (DT-GOL), a novel framework that shifts from temporal compensation to spatial
reasoning to bridge the supervised latency gap. By modeling the delay challenge as a semi-supervised task, we leverage real-time topological
evolution of features as a reliable
geometric surrogate for unobservable conceptual changes to achieve proactive supervised adaptation within the delay window. Unlike rigid self-training, we introduce a dynamic evidence calibration mechanism that distills
geometric information into soft labels that perceive uncertainty, effectively mitigating the confirmation bias inherent in hard pseudo-labels. Furthermore, to resolve the stability-plasticity dilemma, we design a decoupled dual-track architecture in which a master learner serves as a stable anchor, updated strictly from delayed ground truth, while a transient branch leverages soft
geometric knowledge for low-risk forward adaptation. Extensive experiments on real and synthetic
datasets demonstrate that DT-GOL significantly outperforms existing state-of-the-art baseline methods, especially in scenarios with concept drift.
Read the full abstract ▾
All authors Yulin Wang, Yi He, Dianlong You, Di Wu
Machine Learning
Neural Operator Processes for Probabilistic Operator Learning under Partial Observations
Jose Miguel Lara-Rangel and Serge Guillas
Neural operators learn mappings between function spaces, but are typically developed with dense input-output training fields and fully observed inputs at inference. Many scientific problems require instead predicting solution fields from sparse, irregular, or partial observations under uncertainty. We introduce Neural Operator Processes (NOPs), a framework that unifies neural-process conditioning with neural-operator decoding to predict full output fields from limited context. NOPs condition on sparse joint input-output observations and support deterministic and probabilistic prediction within a shared encoder-decoder architecture. We study two conditioning strategies, convolutional pooled summaries and query-aligned
attention, and analyze how their interaction with latent
stochastic variables depends on PDE geometry. Across function regression and three PDE
benchmarks, we find that sparse conditional operator learning is viable and can match dense-grid behavior in several regimes, that preserving local context-query geometry is essential in non-periodic settings but less so in spectrally smooth periodic regimes, and that uncertainty-aware operator learning succeeds when latent conditioning complements rather than overwrites the local
geometric pathway. These results provide a basis for probabilistic operator learning under partial observations and help bridge operator learning and probabilistic meta-learning in function space.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CR
CITADEL: CSI-Based Jamming Detection and Open-Set Classification for IIoT Networks
Aymen Bouferroum, Ildi Alla, Valeria Loscri et al. (+2 more)
Radio frequency jamming poses a critical threat to the availability of wireless Industrial Internet of Things (IIoT) networks. Existing detection and classification techniques are poorly suited to this setting: coarse signal-strength and cross-layer features lack information richness, while raw I/Q baseband approaches require hardware and throughput that is impractical at the scale of hundred-node IIoT deployments. This paper presents CITADEL, a lightweight two-stage hierarchical pipeline that uses only Channel State Information (CSI) measurements, which are natively available on commodity IIoT devices, to detect and classify jamming attacks including previously unseen ones. While prior work has shown that jamming leaves observable CSI signatures, CITADEL is the first system to translate this insight into an end-to-end pipeline that jointly achieves closed-set classification of known attacks, open-set detection of zero-day attacks, and resistance to
adversarial evasion. Evaluated across 6 known attack types and 15 zero-day scenarios, CITADEL achieves 100% known-attack detection and 97.1% zero-day detection at a 0.4% end-to-end false positive rate. Under
adversarial evaluation spanning white-box and black-box threat models, gradient-based evasion remains below 2% across all tested perturbation budgets and the strongest published CSI attack generator achieves less than 5% average evasion. A systematic comparison against eight baselines confirms that no existing method achieves comparable performance on CSI data across all three axes: detection, generalization, and robustness. The full pipeline completes inference in 14.2 ms at 95.9 mJ on an edge GPU, establishing CITADEL as a practical solution for large-scale IIoT network security.
Read the full abstract ▾
All authors Aymen Bouferroum, Ildi Alla, Valeria Loscri, Abderrahim Benslimane, Vincent Lenders
Machine Learning
Provable Benefits of RLVR over SFT for Reasoning Models: Learning to Backtrack Efficiently
Stanley Wei and Juno Kim
Recent advances in large
language models (LLMs) have demonstrated that reinforcement fine-tuning of pretrained base models can lead to significant gains in
reasoning performance at inference time. In this work, we theoretically analyze why reinforcement fine-tuning induces better
reasoning ability than purely supervised fine-tuning (SFT) methods. We model chain-of-thought (CoT)
reasoning as a pathfinding problem on graphs and compare the popular method of reinforcement learning with verifiable rewards (RLVR) against traditional SFT. We prove that SFT, when trained on golden shortest paths without negative examples, fails to learn how to efficiently backtrack. In contrast, an RLVR-trained model can learn how to efficiently backtrack from dead ends using only outcome reward. This leads to an exponential separation in inference-time compute between the two methods, and demonstrates that RLVR leads the model to learn the location of difficult decisions in a
reasoning chain, ultimately allowing for better allocation of inference-time compute. Finally, we show that the
reasoning traces of an RLVR model can be distilled to train a base model to backtrack efficiently as well.
Read the full abstract ▾
Machine Learning
FORGE: Fused On-Register Gradient Elimination for Memory-Efficient LLM Training
Dikshant Kukreja, Kritarth Prasad, Avinash Anand et al. (+6 more)
Reverse-mode differentiation computes every weight gradient, writes it to
memory, and only then lets the optimizer read it back. This two-phase schedule sets the
memory ceiling of modern training: at the seam between the phases, every layer's gradient is live at once. We argue that this materialized gradient is an artifact of how differentiation is staged, not a quantity that learning requires – and we eliminate it. FORGE folds the optimizer step into the backward pass and applies it one tile at a time, entirely in registers, so each gradient tile is consumed the instant it is produced and never becomes a tensor. The fusion changes only when the update happens, not what it computes: in full precision the fused step is provably exact – the identical optimizer update, for every element-wise rule – and that exactness survives tensor- and sequence-parallel sharding; in the bf16 and 8-bit regimes used in practice it is faithful rather than bit-identical, its deviation bounded and, for the weight store, rendered unbiased by
stochastic rounding. Because each gradient tile is born and consumed in the same registers, it is never converted down to bf16 to be stored and read back; FORGE thus preserves the full-precision fidelity that both bf16 and 8-bit optimizers lose to that conversion. Nor is the method tied to one architecture or one optimizer: linear layers are ubiquitous, and FORGE reclaims the gradient
memory of any of them under any element-wise rule. Empirically FORGE more than halves the
memory of an optimizer step and, at the small batch sizes typical of fine-tuning and continued pretraining, runs about 1.5x faster; integrated into tensor-parallel Megatron-LM it fits 8B training at four times the micro-batch a standard optimizer allows on the same GPUs.
Read the full abstract ▾
All authors Dikshant Kukreja, Kritarth Prasad, Avinash Anand, Zhengkui Wang, Erik Cambria, Timothy Liu, Aik Beng Ng, Simon See, Bapi Chatterjee
Machine Learning
EEG Benchmarking Needs a Task Specification Layer: NeuroDoc for Rulebook-Guided, Executable Benchmark Construction
Chengxuan Qin, Zhige Chen, Shu Peng et al. (+9 more)
Electroencephalography (
EEG) foundation models increasingly rely on multi-
dataset training and evaluation, yet public
EEG datasets still lack a shared task specification layer that can turn heterogeneous recordings into reusable
benchmark units. Existing standards organize files, metadata, and provenance, but they do not specify
EEG tasks under a common
language and rulebook, leaving critical task semantics scattered across papers, code, and manual interpretation. We investigate whether heterogeneous public
EEG datasets can be standardized through a structured task specification
language paired with a shared rulebook. Our methodology represents each
benchmark entry as a task document synchronized with an executable task kernel, with the rulebook defining task fields, evidence requirements, document-kernel
alignment, review states, and machine-checkable constraints. Using this methodology, we release a community-reviewed
EEG benchmark corpus centered on 53 completed and reviewed entries with 245 task definitions spanning diverse paradigms, and we introduce NeuroDoc and NeuroAudit as the operational support layer for rulebook-guided drafting, upgrading, review, amendment, and release management. We further examine whether the resulting
benchmark units can be instantiated in a shared downstream setting across four
EEG foundation model backbones, providing execution-based evidence for reusable, auditable, and executable
EEG benchmarking infrastructure.
Read the full abstract ▾
All authors Chengxuan Qin, Zhige Chen, Shu Peng, Rui Yang, Jiping Cui, Yikai Dong, Jun Li, Liu Peng, Zhida Shang, Mingze Tang, Kay Chen Tan, Jibin Wu