Machine Learning
A Token-Level Analysis of Sampled-Token Reverse-KL On-Policy Distillation
Bing Shao, Jiazheng Zhang, Long Ma et al. (+13 more)
On-policy
distillation (OPD) supervises a student on its own trajectories with
token-level signals from a frozen teacher, yet how a sampled loss allocates updates across
tokens remains poorly understood. We analyze the gradient of the per-
token K2
estimator of reverse KL with respect to the student logits. The $\ell_1$ norm of this gradient factorizes into the absolute teacher–student log-probability gap and a student-side softmax factor that grows as the sampled
token becomes less likely under the student. In our math-
distillation runs, these per-
token norms are highly non-uniform: low-student-probability
tokens account for a disproportionate share of their sum and are also enriched in large teacher–student gaps. As a lightweight intervention suggested by this analysis, we study Surprise-aware Reweighting (SuRe), a detached, bounded weighting rule that further amplifies this existing allocation. Across two Qwen3 student scales, SuRe improves several math metrics over vanilla OPD and shows no clear degradation on the selected out-of-domain
benchmarks. Our primary contribution is therefore a gradient-level characterization of reverse-KL OPD trained with the K2
estimator, with SuRe as one empirical instantiation.
Read the full abstract ▾
All authors Bing Shao, Jiazheng Zhang, Long Ma, Yujiong Shen, Senjie Jin, Xin Guo, Yuming Yang, Mingxu Chai, Zhiheng Xi, Boyang Liu, Junlin Shang, Peixin Wang, Qi Zhang, Tao Gui, Qi Zhang, Xuanjing Huang
Machine Learning
DCEO: Direct Causal Effect Optimization for Long-Term User Value Modeling in E-commerce Search
Junzhao Zhang, Tao Zhang, Liren Yu et al. (+4 more)
Industrial e-commerce search systems ultimately aim to optimize the user-level long-term objective, such as n-day cumulative purchases or gross merchandise value (GMV) per user. However, such objectives are defined at the user level, whereas search ranking is based on item-level scores within each request. Existing methods typically bridge this granularity gap through manually designed multi-objective fusion, where predictions of multiple item-level objectives, such as clicks, carts, purchases, and transaction value, are combined into a ranking score that serves as a proxy for the ultimate objective. Such hand-crafted fusion schemes rely on a small set of manually tuned weights, limiting fine-grained personalization and leading to suboptimal
alignment with the ultimate objective. In this paper, we propose DCEO (Direct Causal Effect Optimization), a data-driven framework for learning item-level proxy scores that are better aligned with the ultimate objective. We first aggregate the item-level proxy scores into a user-level proxy metric and quantify its
alignment with the ultimate objective using a relative causal effect. We then develop an actor-critic framework, where the critic estimates the ultimate objective for a given user-level proxy metric, and the actor dynamically generates context-dependent fusion weights over multiple objectives to construct the item-level proxy scores and is trained to directly optimize the relative causal effect. Extensive offline experiments and analyses demonstrate the effectiveness and interpretability of DCEO. In addition, DCEO has been deployed in a large-scale industrial e-commerce search system, outperforming the conventional GMV proxy by 0.36% in GMV in a 41-day online A/B test.
Read the full abstract ▾
All authors Junzhao Zhang, Tao Zhang, Liren Yu, Feiyi Dong, Zhixuan Zhang, Dan Ou, Haihong Tang
Machine Learning · Cross-listed from q-bio.QM
Propensity Straight-Through Gradients for Discrete Stochastic Systems
Jose M. G. Vilar and Leonor Saiz
We introduce the propensity straight-through (PST)
estimator for gradient-based learning through exact simulations of continuous-time Markov chains (CTMCs). These models describe discrete
stochastic dynamics across physics, chemistry, and biology, but their categorical event choices prevent ordinary differentiation. PST preserves exact forward trajectories and differentiates, in the backward pass, the normalized reaction propensities the simulator already computes. The affine state update makes this rule exact for the one-step conditional-mean sensitivity. We derive the per-step discrepancy for one-step sensitivities composed along nonlinear trajectories in closed form, show that it vanishes for affine downstream dependence, and bound it quadratically in the stoichiometric jump size. Across reversible dimerization, a genetic oscillator, a 50-task repressilator suite, and experimental patch-clamp recordings, PST recovers parameters to within 0.06-1.7% and converges 2.4-3.0-fold faster than the Gumbel-Softmax straight-through
estimator at matched computational cost. At deep-learning scale, PST trains all 203,796 parameters of a
stochastic reaction network using exact discrete events from initialization. By differentiating an exact conditional mean, PST eliminates reaction-selection temperature tuning and provides a scalable route to learning through exact
stochastic dynamics.
Read the full abstract ▾
Machine Learning
Frequency-aware forecasting for short-term typhoon gust prediction
Xuefei Wang, Tingyi Liu, Heng Zhang et al. (+2 more)
Accurate gust
forecasting under typhoon conditions remains challenging due to the highly non-stationary and multi-scale characteristics of extreme wind fluctuations. Existing deep learning models often struggle to simultaneously capture long-term trends and rapid local variations, resulting in degraded performance during extreme events. We propose WDANet, a frequency-aware
forecasting framework that integrates stationary wavelet decomposition, a Feature-wise Linear Modulation (FiLM) strategy, and a dual-branch encoder-decoder architecture, enabling separate modeling of trend and fluctuation components. Taking the offshore regions of the Western Pacific in China as an example, we conduct fine-grid wind gust prediction research. The results demonstrate that WDANet shows advantages for short lead times under the experimental setting across a 24-h
forecasting horizon and achieves higher prediction accuracy than ECMWF-HRES within the first 6 h. During extreme wind events, WDANet more accurately captures gust peaks and attains the best RMSE and MAE performance. These results highlight its potential for offshore wind power operation, disaster warning, and risk mitigation.
Read the full abstract ▾
All authors Xuefei Wang, Tingyi Liu, Heng Zhang, Lei Xu, Shengjun Zhang
Machine Learning
M-Fibration Theory with Applications to Weighted Graphs
Paolo Boldi, Osvaldo M. Velarde and Hernan A. Makse
The purpose of this paper is to provide a general, comprehensive, theoretical framework that allows one to deal with fibrations on graphs labelled on a commutative monoid. This is a genuine extension of the theory of graph fibrations (as introduced in "Fibrations of Graphs" [Discrete Math., vol. 243, pp. 21-66, 2002]), that makes it possible to deal with weighted graphs, and also graphs labelled with other algebraic structures. The derived theory also lends itself naturally to consider approximate fibrations. As an example, we show how the derived theory can be applied to the reduction of weighted networks, providing a strong theoretical underpinning to recent empirical results.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
JIT-Agent: Scaling Harness Intelligence via Just-in-Time Harness Evolution
Guibin Zhang, Leo Lu, Fangzhou Xie et al. (+15 more)
Agent capability is not determined by the model alone. The
agent harness, encompassing
memory management, planning strategy, action protocol, and tool/skill orchestration, can dominate the contribution of the underlying foundation model. Yet harness design remains manual, task-specific, and fundamentally unscalable. We present JIT-
Agent, a harness intelligence model trained to synthesize task-adaptive
agent harnesses on the fly for arbitrary off-the-shelf agentic LLMs. We formalize the
agent harness as a composable, machine-generatable artifact governed by a fixed four-module protocol, and train JIT-
Agent to customize harnesses for a given task at hand, repair harnesses for stable and reliable execution, and self-evolve by distilling performance signals from an expanding archive of prior harness configurations. Equipped with JIT-
Agent as a harness helper, DeepSeek-V4-Flash surpasses GPT-5.6 on DeepSearchQA (+9.1) and OdysseyBench (+4.3), while the already strong GLM-5.2 gains up to +20.2 points. Across controlled evaluations, JIT-
Agent-generated harnesses are performance-competitive with mature
agent runtimes such as OpenCode and Claude Code and consistently improve multi-scale model families of DeepSeek V4,
Mimo-V2.5, and Qwen3.6. To our knowledge, JIT-
Agent is the first model purpose-built for just-in-time harness generation, establishing harness intelligence as a trainable, transferable, and compounding dimension of
agent capability orthogonal to model scaling.
Read the full abstract ▾
All authors Guibin Zhang, Leo Lu, Fangzhou Xie, Kang Zhu, Junhao Wang, Zhifei Xie, Zhaochen Yu, Zihang Liu, Zhongxiang Sun, Qiankun Li, Yue Liao, Heng Chang, Xiaobin Hu, Qibing Ren, Wangchunshu Zhou, Chuanrui Hu, Yafeng Deng, Shuicheng Yan
Machine Learning · Cross-listed from cond-mat.mtrl-sci
A Hierarchical Synergistic Deep Learning Framework Integrating Composition, Structure, and Ionic Transport for Solid-State Electrolyte Discovery
Hongwei Du, Dingyang Lv, Baole Wei et al. (+5 more)
Inorganic solid-state electrolytes must combine high room-temperature ionic conductivity, a wide electrochemical window, excellent electronic insulation, and favorable mechanical compliance. Single models struggle to support reliable multi-objective screening across vast chemical spaces because of training-data distribution mismatch, cross-property
dataset heterogeneity, and scarce kinetic transport data. To overcome these limitations, we develop a hierarchical synergistic deep-learning framework that sequentially coordinates efficiency, accuracy, and reliability through four complementary modules. The in-house-developed L-G-DCNN and a multi-fidelity implementation built on DenseGNN serve as compositional and structural experts for thermodynamic coarse screening and multi-property evaluation, respectively; MatterSim and system-specific DeePMD models provide transport pre-assessment and kinetic validation. Systematic
benchmarks show that each module outperforms mainstream counterparts in its task, while retrospective validation establishes dual closed-loop verification of module-level accuracy and end-to-end workflow reliability. Applied to 30,364,908 Alex/ICSD-derived candidates, the framework identifies 97 high-performance candidates with room-temperature ionic conductivities of 0.109–59.0 mS/cm, including 94 halides, one borohydride, and two
oxides. Consistency with independent experimental data confirms that 76 of the 94 halides fall within reported high-conductivity structural regions. Analysis reveals that Li$^{+}$ jump-network connectivity, rather than the number of
geometric Li sites, is the core determinant of room-temperature ionic conductivity. Li-defect engineering effectively enhances
oxide transport, whereas the inherent
rigidity of the O$^{2-}$ framework suggests a potential upper limit on
oxide electrolyte performance.
Read the full abstract ▾
All authors Hongwei Du, Dingyang Lv, Baole Wei, Yongheng Li, Feng Yu, Ziheng Lu, Siqi Shi, Hong Wang
Machine Learning
Individual Fairness in Hierarchical Clustering
Binita Maity and Shrutimoy Das
Hierarchical clustering produces ultrametric representations that impose strong global
geometric constraints and may distort local similarities in ways that disproportionately affect individual data points. We study hierarchical clustering under an individual
fairness requirement that bounds relative distortion within local $k$-nearest neighborhoods. We formulate this requirement as a feasibility problem over dominated ultrametrics and characterize the minimal multiplicative slack required for feasibility. We identify a sharp local threshold, prove stability under bounded perturbations, establish monotonicity in $k$, and show an intrinsic $\Theta(\log n)$ separation between local and global realizability. Experiments on synthetic and real world
datasets support our theoretical results.
Read the full abstract ▾
Machine Learning
Beyond Scaling: Self-Evolving LLM Agents for Hardware Kernel Optimization via an Experience-Driven Workflow and Experience Graph Memory
Siyuan Chen, Runlin Hou, Shenxiu Wu et al. (+7 more)
Hardware kernel optimization requires repeated compilation, correctness testing, profiling, and revision. LLM
agents can automate parts of this process, and stronger foundation models, longer context windows, and longer execution horizons have improved optimization within individual tasks. These advances alone do not enable an
agent to learn from completed optimization runs. Existing kernel-optimization
agents seldom preserve a decision, its observed execution feedback, and the later decisions that use that evidence. Retaining every prior trajectory is also impractical because an expanding history competes with the current task for context. We present KOPE, an experience-driven framework for hardware kernel optimization. KOPE records optimization trajectories with correctness and performance feedback in Experience Graph
Memory, then uses Active Context Management and Injection to retrieve relevant experience under a fixed
token budget. The graph retains decision order, observed outcomes, and alternative branches, allowing evidence collected on the target hardware to inform later optimization steps and tasks. Under the same GLM-5.2 setting, the
geometric mean of KOPE's per-operator speedups is $1.54\times$ that of CANNBot, the strongest competing baseline. In a complete 53-operator ablation, Active Context Management and Injection raises pass rate from 60.0% to 84.6%, increases the evaluator-reported positive-field
geometric mean from 0.0382 to 0.0661, and reduces optimization
token consumption from 15.9B to 1.113B
tokens relative to passive
agent-led context construction. Enabling Experience Graph
Memory raises full-suite pass rate from 55.2% to 84.6% and yields a $1.43\times$
geometric-mean speedup on valid timing comparisons. These results support continual optimization through external experience while the foundation model remains fixed.
Read the full abstract ▾
All authors Siyuan Chen, Runlin Hou, Shenxiu Wu, Yansong Sun, Junming Cao, Yiyu Zhang, Shudi Shao, Junhao Qiu, Zhichao Lu, Qingfu Zhang
Machine Learning
Physics-Informed Foresight Pruning for Sparse PINN Solvers of Nonlinear PDEs
Ahmad Ishaque Karimi, Uvini Balasuriya Mudiyanselage and Kookjin Lee
Physics-informed neural networks (PINNs) often rely on over-parameterized models to optimize coupled solution and differential-residual objectives, leaving unclear how much capacity is necessary and what
pruning should preserve. We study foresight
pruning at initialization for sparse PirateNet PDE solvers. Standard neural tangent kernel spectrum-aware
pruning (NTK-SAP) aims to preserve output-side training dynamics but may overlook parameters whose main influence arises through derivatives in the governing equations. We introduce physics-informed spectrum-aware
pruning (PI-SAP), which assigns saliency using sensitivity of the PDE residual. Experiments on the Gray-Scott equations, complex Ginzburg-Landau equation, Burgers' equation, and linear convection equation show that PI-SAP more consistently preserves Gray-Scott residual fidelity and is competitive under aggressive sparsity. However, no criterion is uniformly optimal across equations or sparsity levels. Small-batch PINN-NTK diagnostics further show that residual fidelity, solution accuracy, and kernel conditioning are distinct objectives, motivating
pruning methods that explicitly balance solution-side and residual-side training dynamics during optimization.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Virgil: Navigating Explainability for Transformer-based Language Models
Martino Ciaperoni, Sezer Kutluk, Benedetta Muscato et al. (+2 more)
Explainability for
transformer-based
language models is becoming crucial as these systems are deployed in high-stakes applications. As a result, the ecosystem of explainability tools is rapidly evolving, becoming richer, but also more fragmented and harder to navigate. To address this challenge, we present Virgil, an interactive system that lets practitioners and researchers, including non-experts, navigate explainability tools for
transformer language models. Supported by a curated knowledge base, the system enables users to discover and compare explainability tools within a unified interface.
Read the full abstract ▾
All authors Martino Ciaperoni, Sezer Kutluk, Benedetta Muscato, Marta Marchiori Manerba, Fosca Giannotti
Machine Learning
Beyond Optimal Rates in Stochastic Optimization: Trajectory-Adaptive Stopping Rules
Liviu Aolaritei, Lucas Lévy, Francis Bach et al. (+1 more)
Stochastic gradient descent (SGD) is typically analyzed at a deterministic horizon chosen before the
algorithm is run, even though practical stopping decisions are made adaptively by inspecting the evolving trajectory. This mismatch creates a fundamental certification problem: fixed-time guarantees do not generally remain valid at data-dependent stopping times, while deterministic horizons derived from worst-case bounds can be highly conservative. We address this problem for strongly convex
stochastic optimization by constructing fully observable, trajectory-adaptive upper confidence sequences for the squared distance of the last iterate to the optimizer and the suboptimality of a weighted average. These bounds hold simultaneously over time, attain the optimal $1/t$ decay rate up to iterated-logarithmic factors in the worst case, and adapt to the realized
stochastic gradients, allowing SGD to stop as soon as a prescribed accuracy is certified without sacrificing statistical validity. Our approach treats the evolving SGD trajectory as a sequential experiment whose observations provide evidence about the unknown optimization error. To formalize this perspective, we develop new recursive confidence-sequence techniques and a general time-uniform empirical Bernstein inequality for adapted processes with time-varying conditional means and predictable ranges that may grow without bound. We further extend these confidence-sequence constructions to minibatch SGD, with the empirical Bernstein bounds exploiting the realized second-moment structure within each minibatch. Numerical experiments show that the resulting stopping rules can require several orders of magnitude fewer iterations than natural deterministic horizons.
Read the full abstract ▾
All authors Liviu Aolaritei, Lucas Lévy, Francis Bach, Michael I. Jordan
Machine Learning
Interpreting Protein Language Model Embeddings via Orthogonal Projection for Protein Fitness Prediction
Paulo Yanez Sarmiento, Pia Francesca Rissom, Manuel Pfeuffer et al. (+6 more)
Recently, there has been a growing adoption of
protein language models (PLMs) in biomedical science. Their
embeddings provide a rich numerical representation of
protein sequences which achieve state-of-the-art performance on several downstream tasks including
protein fitness prediction. However, PLM
embeddings are not directly interpretable and, thereby, it remains unclear what features they encode. To gain insight into which biochemical properties of the
protein are driving the prediction, we leverage an orthogonal projection technique that removes linear effects of known tabular features from
embeddings and extend it to high-order and interaction effects. In this way, we remove the effects of interpretable biochemical features from PLM
embeddings. In an ablation study, we show that this leads to a decrease in performance for a downstream classifier trained only on the
embeddings to predict
protein fitness. In an additional evaluation, we find that these biochemical features explain a substantial part of the variance in the predictions of this classifier. Hence, we can show that PLM
embeddings encode patterns correlated with biochemical properties and quantify their contribution to predicting
protein fitness. This computationally efficient approach is not limited to the features or
embeddings considered here and is readily transferable to problem settings beyond
protein fitness prediction.
Read the full abstract ▾
All authors Paulo Yanez Sarmiento, Pia Francesca Rissom, Manuel Pfeuffer, Marco Simnacher, Jordan F. Safer, Sumaiya Iqbal, Henrike O. Heyne, Nadja Klein, Bernhard Y. Renard
Machine Learning
Reflection Steering: Disentangling Reflection from Reasoning in Activation Space for Token-Efficient Inference
Jiarui Hu, Zhiyuan Wen, Xiaoyun Liu et al. (+2 more)
Large
reasoning models often produce
reasoning traces with verification, revision, and backtracking. When reflection merely re-checks established results, it wastes
reasoning tokens and increases latency. Most existing reflection steering methods add a label-derived mean-difference direction across preset layers, but its
entanglement with
reasoning and length signals destabilizes the accuracy-efficiency trade-off. In this paper, we propose Reflection Steering, a training-free framework for controlling reflection-associated computation within LLMs by disentangling reflection-related activations from general
reasoning. Specifically, we contrast reflective and non-reflective hidden states at each LLM layer, denoise the resulting reflection directions with PCA, and orthogonalize them against general-
reasoning directions. To limit downstream amplification from early-layer interventions, we calibrate each layer across multiple intervention strengths on a small set, retain only stable layers, and apply bounded projection removal to their residual-stream activations. We conduct extensive experiments across two public
benchmarks and three open-weight LLMs against state-of-the-art activation-steering baselines. Results show that Reflection Steering reduces
reasoning tokens by 16.9% on average across six matched settings. Besides, our method further introduces a bounded reflection intervention-strength parameter $\alpha$, enabling deployment-time adjustment to balance
token savings, accuracy, and generation stability.
Read the full abstract ▾
All authors Jiarui Hu, Zhiyuan Wen, Xiaoyun Liu, Jiaxing Shen, Yu Yang
Machine Learning · Cross-listed from cs.CV
CropCop: An Auditable 120-Class Plant-Health Model from Benchmark Reconstruction to a Quantised Runtime Artifact
Rana Muhammad Ahmed and Sabahat Abbas
A plant-health score can appear precise while resting on duplicated image families, a long-tailed label space, or a runtime file that was never evaluated. We present CropCop, a closed-set recognition system spanning 120 operational plant-health classes and an evidence chain from corpus reconstruction to direct execution of the final quantised artifact. Starting from 117,546 audited images, we rejected the inherited partition after confirming 3,233 duplicate relationships across split boundaries and froze a 109,107-image
benchmark with zero crossings among the audited trusted leakage groups and a 151.7 largest-to-smallest class ratio. A fully fine-tuned DINOv3 ConvNeXt-Tiny reference achieved 98.51% accuracy and 96.87% macro-F1 on the locked internal test. A compact MobileNetV4 Conv-Medium derivative achieved 98.46% accuracy and 96.27% macro-F1 without being presented as evidence for a new
distillation method. Validation-only post-training quantisation selected dynamic activations with per-channel weights, and the final 22.60 MiB ExecuTorch/XNNPACK PTE achieved 98.46% accuracy and 96.23% macro-F1 when executed directly. Only six of 16,363 top-1 decisions changed between the converted INT8 graph and the PTE, while paired analysis showed a modest class-balanced loss; an exploratory post hoc fruit-label slice localized a larger recall decline than aggregate accuracy revealed. CropCop establishes strong leakage-controlled internal recognition and software-runtime fidelity; it does not establish performance on unseen farms, camera pipelines, or physical Android hardware.
Read the full abstract ▾
Machine Learning
Resilient Decentralized Wireless Federated Learning via Gradient Tracking with AdamW
Nguyen Van Thieu, Ti Ti Nguyen, Ons Aouedi et al. (+2 more)
Wireless Internet-of-Things (IoT) edge networks require decentralized learning (DecL) methods that can operate reliably under both heterogeneous local data and communication-constrained wireless links. However, existing decentralized optimization schemes often incur substantial communication overhead and degraded performance when transmissions are constrained by strict airtime budgets, fading channels, and packet losses. This paper proposes QEF-GT-AdamW, a communication-efficient and outage-resilient
algorithm for DecL over wireless communication (WCom) networks. The proposed method combines gradient tracking to mitigate the effect of non-IID data, AdamW-based adaptive optimization to improve training stability, and dual-stream biased quantization with error feedback to reduce communication payloads for both model and tracking exchanges. To address unreliable broadcast communication, the proposed framework further employs a local fallback strategy when scheduled packets are not successfully received. We explicitly model the effect of bandwidth, transmit power, airtime constraints, and fading channels on DecL performance, and establish convergence guarantees for the proposed
algorithm under compressed and unreliable wireless communication. Experimental results on heterogeneous MNIST and CIFAR-10 settings show that QEF-GT-AdamW consistently improves robustness and convergence performance over representative DecL baselines while achieving favorable accuracy-communication trade-offs under limited wireless resources.
Read the full abstract ▾
All authors Nguyen Van Thieu, Ti Ti Nguyen, Ons Aouedi, Vu Nguyen Ha, Symeon Chatzinotas
Machine Learning · Cross-listed from math.NA
Adaptive Hybrid Subspace Levenberg Marquardt Algorithm with Adequacy Monitor for Large Scale Least Squares Problems
M. Duc Hoang and Timothy J. Lewis
The Levenberg-Marquardt (LM)
algorithm is the most widely used method for solving nonlinear least-squares problems, as it combines the robustness of steepest descent with the fast local convergence of the Gauss-Newton method. However, its computational cost can become prohibitive for large-scale problems because each iteration requires solving a large damped linear system, and conventional step acceptance strategies may require repeated solves as the damping parameter is adjusted. Despite this computational challenge, many large-scale least-squares problems exhibit effective low-dimensional structure, with only a small number of parameter-space directions strongly informed by the data. We propose an adaptive hybrid subspace Levenberg-Marquardt (HSLM)
algorithm that constructs a low-dimensional subspace from complementary sources of gradient,
memory, Krylov-subspace, and randomized
curvature information and computes a spectrally damped LM step within this subspace. A distinguishing feature of the method is a deterministic adequacy monitor that quantifies how much descent information is captured by the reduced space and adaptively enriches the subspace when necessary. Step acceptance is decoupled from damping adjustment: Armijo backtracking determines the accepted step length, while the ratio of actual to predicted reduction is used solely to update the damping parameter, thereby avoiding repeated damped-system solves during step acceptance. For the HSLM
algorithm, we establish global convergence to stationarity and prove local linear and superlinear convergence. Numerical experiments on neural-network training problems show that HSLM achieves convergence behavior comparable to classical and Krylov subspace LM (KSLM) while substantially reducing per-iteration computational cost, with increasing advantages observed as the parameter dimension grows.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Adaptive Regularization for Random Features: A Neighboring Early-Stopping Rule with Oracle-Rate Guarantees
Caixing Wang, Zhibo Chen and Yue Wang
Random feature methods provide a scalable approximation to kernel ridge regression (KRR), but the regularization parameter that yields the oracle learning rate depends on unknown smoothness and capacity parameters. In this work, we propose a neighboring early-stopping rule for adaptive regularization in KRR with random features (KRR-RF). The method uses a grid that is uniform in inverse regularization and compares only adjacent
estimators, reducing the number of discrepancy comparisons relative to standard all-pairs Lepskii-type procedures. Both the neighboring discrepancy and its empirical complexity term can be computed directly in the random feature space, without constructing the exact kernel Gram
matrix. We establish a high-probability comparison bound for neighboring KRR-RF
estimators and show that, under standard source and capacity conditions together with suitable grid and random feature budget conditions, the selected
estimator attains the oracle
polynomial learning rate up to logarithmic factors. The result allows the regularization parameter to be selected without prior knowledge of the source and capacity exponents and covers both well-specified and partially misspecified regimes. Our analysis is based on an empirical random feature effective dimension that connects the observable stopping threshold with the population complexity of the random feature model. Simulation and real-data experiments illustrate the prediction performance and computational behavior of the proposed method in comparison with standard tuning procedures.
Read the full abstract ▾
Machine Learning · Cross-listed from math.OC
A Multi-View Coupled Tensor Decomposition for Lightweight Online Adaptive Traffic Prediction
Quan Yu, Jie Ni, Yu-Hong Dai et al. (+1 more)
Accurate online traffic prediction is essential for intelligent transportation systems, where
forecasting must be performed continuously under imperfect sensing conditions. Missing observations and anomalous disturbances make this task challenging, particularly when prediction relies on a single traffic view. This paper proposes a Multi-View Coupled Tensor Decomposition (MVCTD) model for online traffic prediction from imperfect multi-view observations, such as speed, flow, and occupancy. The proposed model uses coupled tensor decomposition to build a structured latent
forecasting space, in which shared spatial structures across traffic views and view-specific temporal dynamics are jointly modeled. A group sparse regularization is further introduced to capture correlated abnormal responses induced by real traffic anomalies and thus reduce their influence on forecasts. For streaming deployment, MVCTD performs iterative refinement only on the current latent tensor, while the remaining model variables are updated by lightweight closed-form steps based on summarized historical information, thereby avoiding repeated optimization over the full historical sequence. Experiments on real-world traffic
datasets demonstrate that MVCTD achieves accurate forecasts with favorable runtime under severe missingness, confirming its suitability for online traffic prediction.
Read the full abstract ▾
All authors Quan Yu, Jie Ni, Yu-Hong Dai, Xiongjun Zhang
Machine Learning
FedQoS: Federated QoS-Risk Learning for Heterogeneous Indoor-Outdoor Access Selection
Nguyen Van Thieu, Ti Ti Nguyen, Ons Aouedi et al. (+3 more)
Reliable access selection in dynamic and heterogeneous indoor-outdoor environments is challenging because instantaneous
radio measurements alone cannot capture future QoS degradation caused by mobility, blockage, traffic load, and resource competition. This paper proposes FedQoS, a
federated QoS-risk learning framework for predicting the future reliability of candidate access links and supporting access-node selection without centralizing user-level network data. In FedQoS, each access node locally learns from its observed network logs, including
radio, traffic, load, and service-context features, while a global QoS-risk predictor is trained through
federated aggregation. The learned model estimates the probability of QoS failure for each candidate link, and the controller uses these risk scores to select reliable access nodes under dynamic network conditions. To evaluate the framework, we construct physics-based synthetic indoor-outdoor wireless
datasets using the Sionna framework, covering normal traffic, mobility, event-driven congestion, and non-IID client observations. Simulation results show that learning-based access selection substantially reduces the QoS-failure rate compared with signal-based and historical-QoS heuristic methods. FedQoS achieves near-centralized predictive performance and provides clear reliability gains under mild non-IID data while remaining competitive under the more challenging severe non-IID condition. These results demonstrate the potential of
federated QoS-risk learning for reliable, data-local access selection in dynamic wireless environments.
Read the full abstract ▾
All authors Nguyen Van Thieu, Ti Ti Nguyen, Ons Aouedi, Zerihun Huruy, Vu Nguyen Ha, Symeon Chatzinotas
Machine Learning
A Storage-Retrieval Gap in Parametric Knowledge Graph Memory
Martino M. L. Pulici, Cuong Xuan Chu, Evgeny Kharlamov et al. (+1 more)
Graph
retrieval-augmented generation places retrieved subgraphs into the model's context window at query time, paying a recurring
token cost and exposing source data on every call. We study an alternative: compiling a knowledge graph offline into a bank of LoRA adapters, one per entity, that serve as a parametric knowledge layer queried by injecting weights rather than text, at zero query-time context cost. On the MetaQA
dataset, we find that subgraph-trained adapters encode context-free factual knowledge that generalizes to unseen questions: on single-valued relations the adapter gains $+0.243$ exact-match score over a base model that is nearly blind closed-book ($0.007$), and only the correct adapter recovers this knowledge (an oracle gap of $+0.283$ over the base model). However, the stored knowledge is not recoverable by similarity: given a query with no subgraph,
embedding-based and weight-space geometry
retrieval both perform at chance, because a semantically neighbouring entity's adapter does not contain the answer - knowledge is stored locally and does not transfer. Weight geometry correlates with subgraph semantics ($\rho = +0.329$) but not with functional retrievability. We quantify the byte and context-
token costs against graph
retrieval-augmented generation and discuss deployment implications. Our results establish that parametric knowledge graph
memory is feasible for storing knowledge, and identify selecting and composing the right adapters by a mechanism other than semantic similarity as the central open problem - motivating a learned, query-conditioned composition mechanism.
Read the full abstract ▾
All authors Martino M. L. Pulici, Cuong Xuan Chu, Evgeny Kharlamov, Volker Tresp
Machine Learning · Cross-listed from cs.NI
AERIS: Offline Policy Improvement for Multi-UAV Integrated Sensing and Communication
Ziyuan Wang, Yifan Sui, Wei Wei et al. (+4 more)
Unmanned aerial vehicle (
UAV)-enabled integrated sensing and communication (
ISAC) is a promising 6G paradigm, but dynamic multi-
UAV ISAC control must jointly balance communication quality, sensing reliability, and flight safety under
stochastic mobility. Existing optimization methods often require repeated global non-convex solving, while online reinforcement learning (RL) depends on risky trial-and-error flights that may cause sensing loss or collision-risk events. This paper proposes AERIS, an offline policy improvement framework for multi-
UAV ISAC. AERIS learns from fixed flight logs under centralized training and decentralized execution, so each
UAV acts from local histories while training uses logged global information to assess team-level effects. We further design
STAR-CRDT, an offline multi-
agent RL
algorithm that performs support-aware local action rectification and distills only trusted improvements into the decentralized actor. We prove an offline-support policy improvement guarantee. Experiments show that
STAR-CRDT improves the main
ISAC objective return by 29.3% over the strongest baseline. It further improves communication sum rate, sensing pass rate, and sensing margin by 3.4%, 4.8%, and 69.1%, while reducing collision-risk events by 54.2%. On unseen real-road maps built from OpenStreetMap data,
STAR-CRDT still obtains the best return.
Read the full abstract ▾
All authors Ziyuan Wang, Yifan Sui, Wei Wei, Wenjie Xin, Zekai Zhang, Xiangwang Hou, Xiao-Ping Zhang
Machine Learning · Cross-listed from stat.ME
Functional linear regression from sparse to dense designs: a pooling-ridge method and minimax optimality
Shunxing Yan and Fang Yao
Functional data analysis is an important statistical field that treats data as random functions. In practice, the random functions are often not fully observed but instead measured at discrete times. While simpler problems, such as mean and covariance estimation, have been widely studied for discretely observed data, optimal estimation of linear regression for this data type has remained unsolved for over two decades. To tackle this fundamental challenge, we propose a novel approach, referred to as pooling ridge estimation, which combines the advantages of pooling strategy and RKHS-based method by incorporating the unbiased estimation of operators based on discretely observed measurements from all subjects. This unified estimation framework enables us to achieve minimax optimality in prediction risk in arbitrary sampling schemes ranging from sparse to dense designs, for both scalar-on-function and function-on-function regression models. Such methodological and theoretical advances are obtained for the first time and accurately reveal the influence of discrete sampling. For scalar-on-function regression, the phase transition occurs once, separating the convergence behavior into two distinct regimes. Remarkably, for function-on-function regression, up to three phase transitions may occur, determined by the sampling frequencies of the predictor/response functions. Finally, simulation experiments and two real data examples provide empirical support for the proposed methods.
Read the full abstract ▾
Machine Learning
Resolving Multi-Modal Regression by Difference-Quotient-Based Clustering:Fast Coarse Conditional-Label Assignment
Huang Weiquan
Multimodal regression suffers from the mean-collapse pathology: under squared loss, an unconstrained regressor converges to the conditional mean, which for K > 1 lies away from all modes. We attribute this failure to pairwise contradictions–samples with nearly identical inputs but distant outputs–and propose Difference-Quotient Clustering (DQC), which partitions data to minimize intra-cluster output-vs-input discrepancy. Each sample is assigned to the cluster that minimizes its maximum contradiction ratio; a logits generator and a conditional network are then trained on the resulting labels. Since the generating modality is unknown at test time, we evaluate via minimum squared error (minMSE) against all K true outputs. On synthetic
benchmarks (K=5, 10), DQC achieves test minMSE 0.19 (K=5, nx=500), versus 0.09 for an oracle, 1.08 for random labels, and 1.33 for mean collapse. We observe two empirical regularities: larger intra-cluster contradictions require deeper networks, and oracle labels generalize from fewer samples than cluster-derived equivalents. The clustering is a hard, parallelizable O(n^2/2) front-end for coarse conditional assignment, reducing the burden of downstream generative refinement. A second-stage re-clustering on residual errors is outlined as future work.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.GR
GLOSS: Geometric Local Self-Similarity Learning for Faithful Reference-Guided Texture Fill
Chenyue Cai, Anita Hu, James Lucas et al. (+2 more)
Using conditional image generators, texture artists can explore many single-view looks for an existing 3D shape. Despite impressive progress, state-of-the-art generative methods still struggle to generate a full object texture while closely adhering to fine scale
geometric detail and single view references, leaving little room for artists guidance. Furthermore, current automatic models lack the flexibility for artist to explore multiple textures from varied sources in an interactive and controllable manner. Unlike methods trained on large 3D
datasets that generate full object textures from global guidance, our work explores a local and less data-hungry approach to texture with explicit artist control. We leverage the
geometric self-similarity and geometry-texture correlation existing in many natural and man-made shapes; and train a shape-specific local texture generation and completion model. This model learns from existing image model priors and a single 3D shape, and is guided by attending to a set of geometry-aware reference patches. The trained shape-specific network can transfer any novel reference to the full target object texture through patchwise inpainting. We show improved or comparable quality to strong image-conditioned texture generation baselines, suggesting local texturing as a promising research direction. Our model also enables local geometry-conditioned texture inpainting, guided by artist-selected references, and generalizes to PBR materials and unseen meshes for texture transfer. We piloted our novel texture fill capability as a Blender addon with several 3D texturing professionals who reported positive feedback on the model's controllability, practical usefulness, and creative affordances.
Read the full abstract ▾
All authors Chenyue Cai, Anita Hu, James Lucas, Szymon Rusinkiewicz, Masha Shugrina
Machine Learning · Cross-listed from cs.AI
Training Alignment Auditors via Reinforcement Learning
Paul Rosu and Rowan Wang
Alignment auditing of frontier models increasingly relies on LLM auditors to surface undesirable behaviors at scale, but current automated auditors can struggle with coherent investigation and audit realism. In this work, we improve LLM auditors with reinforcement learning. In our best training environment, the policy investigates target models that potentially possess hidden behaviors planted via their system
prompt. An LLM judge, which knows whether the target has a hidden behavior, holistically compares the policy's investigation to a reference investigation to determine the reward. With systematic ablations, we find that pairwise rewards yield more robust training compared to pointwise rewards, and that adding targets without planted behaviors helps maintain a low false positive rate. Training improves investigation quality against targets with planted behaviors, the rate of concerning behaviors surfaced in unmodified production models, and audit realism, while false-positive rates stay below 1%. Furthermore,
auditing capabilities generalize across scaffolds: performance on AuditBench's adversarially fine-tuned targets substantially improves [Sheshadri et al., 2026].
Read the full abstract ▾
Machine Learning · Cross-listed from physics.comp-ph
Energy Yield and Lifetime Climate Classification via Machine Learning for Optimizing Photovoltaic Module Design and Materials
Youri Blom, Sofia Dutto, Alexandru Costache et al. (+7 more)
To resiliently and sustainably meet our future energy demand, photovoltaic (PV) modules must be deployed across a broad and diverse range of geographical regions with varying operating conditions. As these conditions strongly affect both performance and optimal system design, a dedicated PV-specific
climate classification can be of great use. In this work, we develop a
climate classification framework tailored to PV applications using a variety of machine learning (ML) techniques. Building on previous studies, our approach incorporates both energy yield, and for the first time, also the module lifetime with
climate dependent degradation. We generate an interpolated
dataset containing twelve input features and two target variables (i.e. energy yield and module lifetime). Feature importance analysis shows that annual global horizontal irradiation and ambient temperature are the most influential predictors. The most accurate regression model achieves root mean square errors (RMSE) of 0.007 MWh for energy yield and 1.5 years for lifetime prediction. The calculated feature importance scores are then integrated into a hierarchical clustering framework, resulting in 6 primary
climate clusters (Tropical, Desert, Continental, Temperate, Boreal, and Polar) and 15 corresponding subclusters. Our analysis shows that the low temperature continental
climate offers the highest discounted lifetime energy yield. These results can support a wide range of applications, including PV module optimization, system siting decisions, and comparative performance studies.
Read the full abstract ▾
All authors Youri Blom, Sofia Dutto, Alexandru Costache, Rowan Richie, Ruben Pelsser, Wesley Berger, Jing Sun, Rudi Santbergen, Olindo Isabella, Malte Ruben Vogt
Machine Learning
Joint Initialization of Flux Networks and Effective Multiplication Factor for Physics-Informed Neural Networks Solving Neutron Diffusion Problems
Qin Hang, Yangdi Yi, Jiayi Li et al. (+2 more)
Efficient determination of the effective multiplication factor (keff) is an important computational task in reactor core neutronics analysis. Physics-informed neural networks (PINNs) incorporate
neutron diffusion equations and boundary conditions into network training to efficiently determine the
neutron flux distribution and keff. To further improve the efficiency of keff calculations using PINNs, a Joint Initialization Physics-Informed Neural Network (JI-PINN) is proposed in this work. In this method, a low-resolution approximate solution to the K-
eigenvalue problem is used to construct a joint initial state for the flux network parameters and keff, and both are then jointly optimized under physical constraints. The proposed method was validated on a two-dimensional two-group two-material case, the IAEA 2D
benchmark, a two-dimensional two-group four-material case, and a three-dimensional single-group case. For these test cases, the total computational time was reduced by 25.4%, 38.2%, 49.4%, and 28.9%, respectively, while comparable solution accuracy was maintained. The occurrence of anomalous results associated with marked deviations of keff from the reference value was also reduced. The proposed method provides a more efficient and robust initialization strategy for solving
neutron diffusion K-
eigenvalue problem with PINNs.
Read the full abstract ▾
All authors Qin Hang, Yangdi Yi, Jiayi Li, Xu Wang, Heng Zhang
Machine Learning · Cross-listed from cs.AI
Distance Is Not Enough: Forget-Retain Alignment Gap Predicts LLM Relearning Robustness
Yi Chen, Hanna Hsieh, Shuhong Liu et al. (+4 more)
Machine unlearning aims to make a model forget specific data, yet unlearned LLMs often fail to stay unlearned: brief fine-tuning can revive removed knowledge. Existing robustness predictors rely on global weight-space displacement, but distance alone can be misleading when random or destructive updates collapse performance. We argue that relearning robustness depends on update structure: robust unlearning should affect forget-critical weights while sparing retain-critical ones. We introduce the Forget-Retain
Alignment Gap (FRAG), a training-free predictor that scores an update's forget-retain
alignment without running a relearning attack, and separates selective from dense updates more reliably than global distance. Building on the forget-critical, retain-sparing principle, Forget-Retain
Pruning (FRP) improves relearning robustness. Our results suggest that weight selectivity better explains robustness than distance alone. Code is available at https://github.com/Yi1-Chen/FRAG.
Read the full abstract ▾
All authors Yi Chen, Hanna Hsieh, Shuhong Liu, Chuanbo Hua, Zihan Ma, Kun Wang, Joo-Young Kim
Machine Learning · Cross-listed from math.NA
Data-driven Effective Modeling of Stochastic Chemical Reaction Networks
Yuan Chen, Weize Mao and Dongbin Xiu
The
Stochastic Simulation
Algorithm (SSA), widely considered an exact
algorithm for
stochastic chemical reaction networks, suffers from high computational cost. In this work, we propose a data-driven effective model that operates on a user-defined coarse time step independent of the underlying microscopic reaction-event scale. This is accomplished by directly approximating the finite-time transition kernel of the continuous-time Markov chain induced by SSA, using a generative machine learning model trained on short bursts of SSA simulation data. The trained model constructs a
stochastic propagator that recursively generates statistically consistent trajectories at the constant coarse time step, with significantly reduced computational cost. In this paper, we employ conditional normalizing flow as the
stochastic propagator. A comprehensive set of numerical examples is presented to demonstrate the accuracy and efficiency of the proposed method.
Read the full abstract ▾