Machine Learning
PIP$^2$ Net: Physics-informed Partition Penalty Deep Operator Network
Hongjin Mi, Huiqiang Lun, Changhong Mou et al. (+1 more)
Operator learning has become a powerful tool for accelerating the solution of parameterized partial differential equations (PDEs), enabling rapid prediction of full spatiotemporal fields for new initial conditions or forcing functions. Existing architectures such as DeepONet and the
Fourier Neural Operator (FNO) show strong empirical performance but often require large training
datasets, lack explicit physical structure, and may suffer from instability in their trunk-network features, where mode imbalance or collapse can hinder accurate operator approximation. Motivated by the stability and locality of classical partition-of-unity (PoU) methods, we investigate PoU-based regularization techniques for operator learning and develop a revised formulation of the existing POU–PI–DeepONet framework. The resulting Physics-informed Partition Penalty Deep Operator Network (PIP$^{2}$ Net) introduces a simplified and more principled partition penalty that improved the coordinated trunk outputs that leads to more expressiveness without sacrificing the flexibility of DeepONet. We evaluate PIP$^{2}$ Net on three nonlinear PDEs: the viscous Burgers equation, the Allen–Cahn equation, and a
diffusion–reaction system. The results show that it consistently outperforms DeepONet, PI-DeepONet, and POU-DeepONet in prediction accuracy and robustness.
Read the full abstract ▾
All authors Hongjin Mi, Huiqiang Lun, Changhong Mou, Yeyu Zhang
Machine Learning
Neural Modular Physics for Elastic Simulation
Yifei Li, Haixu Wu, Zeyi Xu et al. (+2 more)
Learning-based methods have made significant progress in physics simulation, typically approximating dynamics with a monolithic end-to-end optimized neural network. Although these models offer an effective way to simulation, they may lose essential features compared to traditional numerical simulators, such as physical interpretability and reliability. Drawing inspiration from classical simulators that operate in a modular fashion, this paper presents Neural Modular Physics (NMP) for elastic simulation, which combines the approximation capacity of neural networks with the physical reliability of traditional simulators. Beyond the previous monolithic learning paradigm, NMP enables direct supervision of intermediate quantities and physical constraints by decomposing elastic dynamics into physically meaningful neural modules connected through intermediate physical quantities. With a specialized architecture and training strategy, our method transforms the numerical computation flow into a modular neural simulator, achieving improved physical consistency and generalizability. Experimentally, NMP demonstrates superior generalization to unseen initial conditions and resolutions, stable long-horizon simulation, better preservation of physical properties compared to other neural simulators, and greater feasibility in scenarios with unknown underlying dynamics than traditional simulators.
Read the full abstract ▾
All authors Yifei Li, Haixu Wu, Zeyi Xu, Tuur Stuyck, Wojciech Matusik
Machine Learning
The Semantic Architect: How FEAML Bridges Structured Data and LLMs for Multi-Label Tasks
Wanfu Gao, Zebin He and Jun Gao
Existing feature engineering methods based on large
language models (LLMs) have not yet been applied to multi-label learning tasks. They lack the ability to model complex label dependencies and are not specifically adapted to the characteristics of multi-label tasks. To address the above issues, we propose Feature Engineering Automation for Multi-Label Learning (FEAML), an automated feature engineering method for multi-label classification which leverages the code generation capabilities of LLMs. By utilizing metadata and label co-occurrence
matrices, LLMs are guided to understand the relationships between data features and task objectives, based on which high-quality features are generated. The newly generated features are evaluated in terms of model accuracy to assess their effectiveness, while Pearson correlation coefficients are used to detect redundancy. FEAML further incorporates the evaluation results as feedback to drive LLMs to continuously optimize code generation in subsequent iterations. By integrating LLMs with a feedback mechanism, FEAML realizes an efficient, interpretable and self-improving feature engineering paradigm. Empirical results on various multi-label
datasets demonstrate that our FEAML outperforms other feature engineering methods.
Read the full abstract ▾
Machine Learning
The Semantic Illusion: Certified Limits of Embedding-Based Hallucination Detection in RAG Systems
Debu Sinha
Retrieval-Augmented Generation (RAG) systems remain susceptible to
hallucinations despite grounding in retrieved evidence. While current detection methods leverage
embedding similarity and natural
language inference (NLI), their reliability in safety-critical settings remains unproven. We apply conformal prediction to RAG
hallucination detection, transforming heuristic scores into decision sets with finite-sample coverage guarantees (1-alpha). Using calibration sets of n=600, we demonstrate a fundamental dichotomy: on synthetic
hallucinations (Natural Questions),
embedding methods achieve 95% coverage with 0% False Positive Rate (FPR). However, on real
hallucinations from RLHF-aligned models (HaluEval), the same methods fail catastrophically, yielding 100% FPR at target coverage. We analyze this failure through the lens of distributional tails, showing that while NLI models achieve acceptable AUC (0.81), the "hardest"
hallucinations are semantically indistinguishable from faithful responses, forcing conformal thresholds to reject nearly all valid outputs. Crucially, GPT-4 as a judge achieves 7% FPR (95% CI:[3.4%, 13.7%]) on the same data, proving the task is solvable via
reasoning but opaque to surface-level semantics–a phenomenon we term the "Semantic Illusion."
Read the full abstract ▾
Machine Learning
EMFusion: Uncertainty-Aware Conditional Diffusion Model for Multivariate Narrow-band Exposure Forecasting
Zijiang Yan, Yixiang Huang, Jianhua Pei et al. (+2 more)
The rapid growth in wireless infrastructure has increased the need to accurately estimate and forecast electromagnetic field (EMF) levels to ensure ongoing compliance, assess potential health impacts, and support efficient network planning. While existing studies rely on univariate
forecasting of wideband aggregate EMF data, multivariate narrow-band EMF
forecasting is needed to capture the inter-operator and inter-frequency variations essential for proactive network planning. To this end, this paper introduces EMFusion, a conditional
diffusion-based EMF
forecasting framework that integrates diverse contextual factors, such as time of day, season, and holidays, while providing uncertainty-aware probabilistic forecasts. The proposed architecture features a residual U-Net backbone enhanced by a cross-
attention mechanism that dynamically integrates external conditions to guide the generation process. Furthermore, EMFusion integrates an imputation-based sampling strategy that treats
forecasting as a structural inpainting task, ensuring temporal coherence even with irregular measurements. Unlike standard point forecasters, EMFusion generates empirical probabilistic prediction intervals from the learned conditional distribution, providing uncertainty-aware probabilistic
forecasting rather than simple point estimation. Numerical experiments conducted on the multivariate narrow-band EMF
datasets demonstrate that EMFusion with the contextual information of working hours outperforms the baseline models with or without conditions. The proposed EMFusion outperforms the best baseline by 23.85% in continuous ranked probability score (CRPS) and 13.93% in normalized root mean square error.
Read the full abstract ▾
All authors Zijiang Yan, Yixiang Huang, Jianhua Pei, Hina Tabassum, Luca Chiaraviglio
Machine Learning · Cross-listed from cs.CL
The Meta-Prompting Protocol: Orchestrating LLMs via Adversarial Feedback Loops
Fanzhe Fu
The transition of Large
Language Models (LLMs) from
stochastic chat interfaces to reliable software components necessitates a fundamental re-engineering of interaction paradigms. Current methodologies, predominantly heuristic-based "
prompt engineering," fail to provide the deterministic guarantees required for mission-critical applications. We introduce the Meta-Prompting Protocol, a rigorous theoretical framework that formalizes the orchestration of LLMs as a programmable, self-optimizing system. Central to this protocol is the
Adversarial Trinity, a tripartite
topology comprising a Generator (P), an Auditor (A), and an Optimizer (O). By treating natural
language instructions as differentiable variables within a semantic computation graph and utilizing textual critiques as gradients, this architecture mitigates
hallucination and prevents model collapse. We demonstrate the theoretical viability of this approach using declarative programming paradigms (DSPy) and automatic textual differentiation (TextGrad), establishing a foundation for "Observable Software Engineering" in the era of probabilistic computing.
Read the full abstract ▾
Machine Learning
Spectral Representation-based Reinforcement Learning
Chenxiao Gao, Haotian Sun, Na Li et al. (+2 more)
In real-world applications with large state and action spaces, reinforcement learning (RL) typically employs function approximations to represent core components like the policies, value functions, and dynamics models. Although powerful approximations such as neural networks offer great expressiveness, they often present theoretical ambiguities, suffer from optimization instability and exploration difficulty, and incur substantial computational costs in practice. In this paper, we introduce the perspective of spectral representations as a solution to address these difficulties in RL. Stemming from the spectral decomposition of the transition operator, this framework yields an effective abstraction of the system dynamics for subsequent policy optimization while also providing a clear theoretical characterization. We reveal how to construct spectral representations for transition operators that possess latent variable structures or energy-based structures, which implies different learning methods to extract spectral representations from data. Notably, each of these learning methods realizes an effective RL
algorithm under this framework. We also provably extend this spectral view to partially observable MDPs. Finally, we validate these
algorithms on over 20 challenging tasks from the DeepMind Control Suite, where they achieve performances comparable or superior to current state-of-the-art model-free and model-based baselines.
Read the full abstract ▾
All authors Chenxiao Gao, Haotian Sun, Na Li, Dale Schuurmans, Bo Dai
Machine Learning
Epistemic diversity across language models mitigates knowledge collapse
Damian Hodel and Jevin D. West
Artificial intelligence (AI) increasingly generates the very content used to train future AI systems. This feedback loop can degrade model quality, reduce informational diversity, and ultimately drive knowledge collapse, i.e. a degradation to a narrow and inaccurate set of ideas. We ask: to mitigate collapse, is it better to concentrate the internet's knowledge into a handful of dominant models (referred to as an AI monoculture), or to distribute it across a diverse ecosystem of models? To study the effect of diversity on model performance, we randomly segment the fixed training data across an increasing number of
language models and evaluate the resulting ecosystems of models over ten self-training iterations. Our results show that diversity improves long-term performance of models, while monoculture accelerates collapse. Specifically, we observe that the optimal diversity level (i.e., the level that maximizes performance) increases monotonically with the number of self-training iterations. The observed effect is robust across various experimental settings, including different model families, parameter sizes, mixing human- and model-generated data, and temperature sampling methods, demonstrating the significance of ecosystem diversity for mitigating collapse. Moreover, our experiments with increased model and
dataset sizes indicate that scaling up the system can amplify collapse in homogeneous ecosystems, thereby increasing the diversity benefits. In the presence of AI monoculture, our results suggest considering information environments with specialized AI models that maintain and enhance diversity in knowledge production, akin to the benefits of ecological diversity in biology and social systems.
Read the full abstract ▾
Machine Learning
Stock Pattern Assistant (SPA): A Deterministic and Explainable Framework for Structural Price Run Extraction and Event Correlation in Equity Markets
Sandeep Neela
Understanding how prices evolve over time often requires peeling back the layers of
market noise to identify clear, structural behavior. Many of the tools commonly used for this purpose technical indicators, chart heuristics, or even sophisticated predictive models leave important questions unanswered. Technical indicators depend on platform-specific rules, and predictive systems typically offer little in terms of explanation. In settings that demand transparency or auditability, this poses a significant challenge. We introduce the Stock Pattern Assistant (SPA), a deterministic framework designed to extract monotonic price runs, attach relevant public events through a symmetric correlation window, and generate explanations that are factual, historical, and guardrailed. SPA relies only on daily OHLCV data and a normalized event stream, making the pipeline straight-forward to audit and easy to reproduce. To illustrate SPA's behavior in practice, we evaluate it across four equities-AAPL, NVDA, SCHW, and PGR-chosen to span a range of
volatility regimes and sector characteristics. Although the evaluation period is modest, the results demonstrate how SPA consistently produces stable structural decompositions and contextual narratives. Ablation experiments further show how deterministic
segmentation, event
alignment, and constrained explanation each contribute to interpretability. SPA is not a
forecasting system, nor is it intended to produce trading signals. Its value lies in offering a transparent, reproducible view of historical price structure that can complement analyst workflows, risk reviews, and broader explainable-AI pipelines.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CR
SeBERTis: A Framework for Producing Classifiers of Security-Related Issue Reports
Sogol Masoumzadeh, Yufei Li, Shane McIntosh et al. (+2 more)
Monitoring issue tracker submissions is a crucial software maintenance activity. A key goal is the prioritization of high risk, security-related bugs. If such bugs can be recognized early, the risk of propagation to dependent products and endangerment of stakeholder benefits can be mitigated. To assist triage engineers with this task, several automatic detection techniques, from Machine Learning (ML) models to prompting Large
Language Models (LLMs), have been proposed. Although promising to some extent, prior techniques often memorize lexical cues as decision shortcuts, yielding low detection rate specifically for more complex submissions. As such, these classifiers do not yet reach the practical expectations of a real-time detector of security-related issues. To address these limitations, we propose SEBERTIS, a framework to train Deep Neural Networks (DNNs) as classifiers independent of lexical cues, so that they can confidently detect fully unseen security-related issues. SEBERTIS capitalizes on fine-tuning bidirectional
transformer architectures as Masked
Language Models (MLMs) on a series of semantically equivalent vocabulary to prediction labels (which we call Semantic Surrogates) when they have been replaced with a mask. Our SEBERTIS-trained classifier achieves a 0.9880 F1-score in detecting security-related issues of a curated corpus of 10,000 GitHub issue reports, substantially outperforming state-of-the-art issue classifiers, with 14.44%-96.98%, 15.40%-93.07%, and 14.90%-94.72% higher detection precision, recall, and F1-score over ML-based baselines. Our classifier also substantially surpasses LLM baselines, with an improvement of 23.20%-63.71%, 36.68%-85.63%, and 39.49%-74.53% for precision, recall, and F1-score.
Read the full abstract ▾
All authors Sogol Masoumzadeh, Yufei Li, Shane McIntosh, Dániel Varró, Lili Wei
Machine Learning
DreamPRM-Code: Function-as-Step Process Reward Model with Label Correction for LLM Coding
Ruiyi Zhang, Peijia Qin, Qi Cao et al. (+1 more)
Process Reward Models (PRMs) have become essential for improving Large
Language Models (LLMs) via test-time scaling, yet their effectiveness in coding remains limited due to the lack of meaningful step decompositions in code and the noise of Monte-Carlo-generated partial labels. We propose DreamPRM-Code, a coding-focused PRM that treats functions as
reasoning steps using a Chain-of-Function prompting strategy to induce modular code generation, enabling PRM training and application analogous to mathematical
reasoning tasks. To address label noise, DreamPRM-Code introduces a meta-learning-based correction mechanism that leverages clean final-solution unit-test labels and performs bi-level optimization to refine intermediate labels. Applying on test-time scaling, DreamPRM-Code achieved state-of-the-art performance on LiveCodeBench with 80.9 pass@1 rate, surpassing OpenAI o4-mini.
Read the full abstract ▾
All authors Ruiyi Zhang, Peijia Qin, Qi Cao, Pengtao Xie
Machine Learning · Cross-listed from cond-mat.mtrl-sci
Efficient Nudged Elastic Band Method using Neural Network Bayesian Algorithm Execution
Pranav Kakhandiki, Sathya Chitturi, Daniel Ratner et al. (+1 more)
The discovery of a minimum energy pathway (MEP) between metastable states is crucial for scientific tasks including catalyst and biomolecular design. However, the standard nudged elastic band (NEB)
algorithm requires hundreds to tens of thousands of compute-intensive simulations, making applications to complex systems prohibitively expensive. We introduce Neural Network
Bayesian Algorithm Execution (NN-BAX), a framework that jointly learns the energy landscape and the MEP. NN-BAX sequentially fine-tunes a foundation model by actively selecting samples targeted at improving the MEP. Tested on Lennard-Jones and Embedded Atom Method systems, our approach achieves a one to two order of magnitude reduction in energy and force evaluations with negligible loss in MEP accuracy and demonstrates scalability to >100-dimensional systems. This work is therefore a promising step towards removing the computational barrier for MEP discovery in scientifically relevant systems, suggesting that weeks-long calculations may be achieved in hours or days with minimal loss in accuracy.
Read the full abstract ▾
All authors Pranav Kakhandiki, Sathya Chitturi, Daniel Ratner, Sean Gasiorowski
Machine Learning
Adaptive Partitioning and Learning for Stochastic Control of Diffusion Processes
Hanqing Jin, Renyuan Xu and Yanzhao Yang
We study reinforcement learning for controlled
diffusion processes with unbounded continuous state spaces, bounded continuous actions, and polynomially growing rewards: settings that arise naturally in finance, economics, and operations research. To overcome the challenges of continuous and high-dimensional domains, we introduce a model-based
algorithm that adaptively partitions the joint state-action space. The
algorithm maintains
estimators of drift,
volatility, and rewards within each partition, refining the discretization whenever estimation bias exceeds statistical confidence. This adaptive scheme balances exploration and approximation, enabling efficient learning in unbounded domains. Our analysis establishes regret bounds that depend on the problem horizon, state dimension, reward growth order, and a newly defined notion of zooming dimension tailored to unbounded
diffusion processes. The bounds recover existing results for bounded settings as a special case, while extending theoretical guarantees to a broader class of
diffusion-type problems. Finally, we validate the effectiveness of our approach through numerical experiments, including applications to high-dimensional problems such as multi-asset mean-variance portfolio selection.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.SE
Imitation Game: Reproducing Deep Learning Bugs Leveraging an Intelligent Agent
Mehil B Shah, Mohammad Masudur Rahman and Foutse Khomh
Despite their wide adoption in various domains (e.g., healthcare, finance, software engineering), Deep Learning (DL)-based applications suffer from many bugs, failures, and vulnerabilities. Reproducing these bugs is essential for their resolution, but it is extremely challenging due to the inherent nondeterminism of DL models and their tight coupling with hardware and software environments. According to recent studies, only about 3% of DL bugs can be reliably reproduced using manual approaches. To address these challenges, we present RepGen, a novel, automated, and intelligent approach for reproducing deep learning bugs. RepGen constructs a learning-enhanced context from a project, develops a comprehensive plan for bug reproduction, employs an iterative generate-validate-refine mechanism, and thus generates such code using an LLM that reproduces the bug at hand. We evaluate RepGen on 106 real-world deep learning bugs and achieve a reproduction rate of 80.19%, a 19.81% improvement over the state-of-the-art measure. A developer study involving 27 participants shows that RepGen improves the success rate of DL bug reproduction by 23.35%, reduces the time to reproduce by 56.8%, and lowers participants' cognitive load.
Read the full abstract ▾
Machine Learning
Yaniv Leviathan, Matan Kalman and Yossi Matias
When not using
reasoning, repeating the input
prompt improves performance for popular models (Gemini, GPT, Claude, and Deepseek) without increasing the number of generated
tokens or latency.
Machine Learning
Softly Constrained Denoisers for Diffusion Models Applied to Partial Differential Equations
Victor M. Yeom-Song, Severi Rissanen, Arno Solin et al. (+2 more)
Diffusion models have become a powerful generative prior for solutions of partial differential equations (PDEs). Existing approaches enforce physical constraints either by adding the PDE residuals as loss regularizers or through inference-time adjustments. These methods bias the model away from the true data distribution, which is especially problematic when the governing PDE is misspecified. To circumvent these issues while making the most out of the PDE constraint, we introduce soft inductive biases into the denoiser architecture derived from the PDEs. We show that these softly constrained denoisers exploit constraint knowledge to improve compliance over standard denoisers, while maintaining enough flexibility to deviate from it in case of misspecification with respect to observed data.
Read the full abstract ▾
All authors Victor M. Yeom-Song, Severi Rissanen, Arno Solin, Samuel Kaski, Mingfei Sun