Machine Learning
SafeSeek: Universal Attribution of Safety Circuits in Language Models
Miao Yu, Siyuan Fu, Moayad Aloqaily et al. (+6 more)
Mechanistic interpretability reveals that safety-critical behaviors (e.g.,
alignment, jailbreak,
backdoor) in Large
Language Models (LLMs) are grounded in specialized functional components. However, existing safety attribution methods struggle with generalization and reliability due to their reliance on heuristic, domain-specific metrics and search
algorithms. To address this, we propose \ourmethod, a unified safety interpretability framework that identifies functionally complete safety
circuits in LLMs via optimization. Unlike methods focusing on isolated heads or
neurons, \ourmethod introduces differentiable binary masks to extract multi-granular
circuits through gradient descent on safety
datasets, while integrates Safety
Circuit Tuning to utilize these sparse
circuits for efficient safety fine-tuning. We validate \ourmethod in two key scenarios in LLM safety: (1)
backdoor attacks, identifying a
backdoor circuit with 0.42% sparsity, whose ablation eradicates the Attack Success Rate (ASR) from 100% $\to$ 0.4% while retaining over 99% general utility; (2) safety
alignment, localizing an
alignment circuit with 3.03% heads and 0.79%
neurons, whose removal spikes ASR from 0.8% $\to$ 96.9%, whereas excluding this
circuit during helpfulness fine-tuning maintains 96.5% safety retention.
Read the full abstract ▾
All authors Miao Yu, Siyuan Fu, Moayad Aloqaily, Zhenhong Zhou, Safa Otoum, Xing fan, Kun Wang, Yufei Guo, Qingsong Wen
Machine Learning
SynForceNet: A Force-Driven Global-Local Latent Representation Framework for Lithium-Ion Battery Fault Diagnosis
Rongxiu Chen and Yuting Su
Online safety fault diagnosis is essential for lithium-ion batteries in electric vehicles(EVs), particularly under complex and rare safety-critical conditions in real-world operation. In this work, we develop an online battery fault diagnosis network based on a deep anomaly detection framework combining kernel one-class classification and minimum-volume estimation. Mechanical constraints and spike-timing-dependent plasticity(STDP)-based dynamic representations are introduced to improve complex fault characterization and enable a more compact normal-state boundary. The proposed method is validated using 8.6 million valid data points collected from 20 EVs. Compared with several advanced baseline methods, it achieves average improvements of 7.59% in TPR, 27.92% in PPV, 18.28% in F1 score, and 23.68% in AUC. In addition, we analyze the spatial separation of fault representations before and after modeling, and further enhance framework robustness by learning the
manifold structure in the latent space. The results also suggest the possible presence of shared causal structures across different fault types, highlighting the promise of integrating deep learning with physical constraints and neural dynamics for battery safety diagnosis.
Read the full abstract ▾
Machine Learning
Permutation-Symmetrized Diffusion for Unconditional Molecular Generation
Gyeonghoon Ko and Juho Lee
Permutation invariance is fundamental in molecular point-cloud generation, yet most
diffusion models enforce it indirectly via permutation-equivariant networks on an ordered space. We propose to model
diffusion directly on the quotient
manifold $\tilde{\calX}=\sR^{d\times N}/S_N$, where all atom permutations are identified. We show that the heat kernel on $\tilde{\calX}$ admits an explicit expression as a sum of Euclidean heat kernels over permutations, which clarifies how
diffusion on the quotient differs from ordered-particle
diffusion. Training requires a permutation-symmetrized score involving an intractable sum over $S_N$; we derive an expectation form over a posterior on permutations and approximate it using MCMC in permutation space. We evaluate on unconditional 3D molecule generation on QM9 under the EQGAT-Diff protocol, using SemlaFlow-style backbone and treating all variables continuously. The results demonstrate that quotient-based permutation symmetrization is practical and yields competitive generation quality with improved efficiency.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Is AI Catching Up to Human Expression? Exploring Emotion, Personality, Authorship, and Linguistic Style in English and Arabic with Six Large Language Models
Nasser A Alsadhan
The advancing fluency of LLMs raises important questions about their ability to emulate complex human traits, including emotional expression and personality, across diverse linguistic and cultural contexts. This study investigates whether LLMs can convincingly mimic emotional nuance in English and personality markers in Arabic, a critical under-resourced
language with unique linguistic and cultural characteristics. We conduct two tasks across six models:Jais, Mistral, LLaMA, GPT-4o, Gemini, and DeepSeek. First, we evaluate whether machine classifiers can reliably distinguish between human-authored and AI-generated texts. Second, we assess the extent to which LLM-generated texts exhibit emotional or personality traits comparable to those of humans. Our results demonstrate that AI-generated texts are distinguishable from human-authored ones (F1>0.95), though classification performance deteriorates on paraphrased samples, indicating a reliance on superficial stylistic cues.
Emotion and personality classification experiments reveal significant generalization gaps: classifiers trained on human data perform poorly on AI-generated texts and vice versa, suggesting LLMs encode affective signals differently from humans. Importantly, augmenting training with AI-generated data enhances performance in the Arabic personality classification task, highlighting the potential of synthetic data to address challenges in under-resourced
languages. Model-specific analyses show that GPT-4o and Gemini exhibit superior affective coherence. Linguistic and psycholinguistic analyses reveal measurable divergences in tone, authenticity, and textual complexity between human and AI texts. These findings have implications for affective computing, authorship attribution, and responsible AI deployment, particularly within underresourced
language contexts where generative AI detection and
alignment pose unique challenges.
Read the full abstract ▾
Machine Learning
A Learning Method with Gap-Aware Generation for Heterogeneous DAG Scheduling
Ruisong Zhou, Haijun Zou, Li Zhou et al. (+2 more)
Efficient scheduling of directed acyclic graphs (DAGs) is a core problem in large-scale data-intensive computing systems, where query plans, data-processing workloads, and computation graphs consist of dependent tasks competing for limited heterogeneous resource pools. In practice, achieving high-performance execution requires schedulers to adapt across environments with varying resource pools and task types, while generating schedules under tight runtime budgets. We propose WeCAN, an end-to-end reinforcement learning framework for heterogeneous DAG scheduling that addresses task-pool compatibility coefficients and generation-induced optimality gaps. It adopts a two-stage single-pass design: a single forward pass produces task-pool scores and global parameters, followed by a generation map that constructs schedules without repeated network calls. Its weighted cross-
attention encoder models task-pool interactions gated by compatibility coefficients, and is size-agnostic to environment fluctuations. Moreover, widely used list-scheduling maps can incur generation-induced optimality gaps from restricted reachability. We introduce an order-space analysis that characterizes the reachable set of generation maps via feasible schedule orders, explains the mechanism behind generation-induced gaps, and yields sufficient conditions for gap elimination. Guided by these conditions, we design a skip-extended realization with an analytically parameterized decreasing skip rule, which enlarges the reachable order set while preserving single-pass efficiency. Experiments on real-world TPC-H query DAGs, resource-intensive workload
datasets, and ML-compiler computation graphs demonstrate improved makespan over strong baselines, with inference time comparable to classical heuristics and faster than multi-round neural schedulers.
Read the full abstract ▾
All authors Ruisong Zhou, Haijun Zou, Li Zhou, Chumin Sun, Zaiwen Wen
Machine Learning
Neural ODE and SDE Models for Adaptation and Planning in Model-Based Reinforcement Learning
Chao Han, Stefanos Ioannou, Luca Manneschi et al. (+4 more)
We investigate neural ordinary and
stochastic differential equations (neural ODEs and SDEs) to model
stochastic dynamics in fully and partially observed environments within a model-based reinforcement learning (RL) framework. Through a sequence of simulations, we show that neural SDEs more effectively capture the inherent stochasticity of transition dynamics, enabling high-performing policies with improved sample efficiency in challenging scenarios. We leverage neural ODEs and SDEs for efficient policy adaptation to changes in environment dynamics via inverse models, requiring only limited interactions with the new environment. To address partial observability, we introduce a latent SDE model that combines an ODE with a GAN-trained
stochastic component in latent space. Policies derived from this model provide a strong baseline, outperforming or matching general model-based and model-free approaches across
stochastic continuous-control
benchmarks. This work demonstrates the applicability of action-conditional latent SDEs for RL planning in environments with
stochastic transitions. Our code is available at: https://github.com/ChaoHan-UoS/NeuralRL
Read the full abstract ▾
All authors Chao Han, Stefanos Ioannou, Luca Manneschi, T.J. Hayward, Michael Mangan, Aditya Gilra, Eleni Vasilaki
Machine Learning · Cross-listed from cs.AI
MemCollab: Cross-Model Memory Collaboration via Contrastive Trajectory Distillation
Yurui Chang, Yiran Wu, Qingyun Wu et al. (+1 more)
LLM
agents increasingly rely on
memory mechanisms to reuse knowledge from past problem-solving experiences. However, existing methods typically construct
memory for a single
agent and reuse it with the same underlying model, tightly coupling stored knowledge to model-specific
reasoning styles. In heterogeneous deployments, where
agents may be instantiated with backbone models of different sizes, architectures, or specializations, this raises a key question: can a single
memory system be shared across
agents with different backbone models? We find that naive cross-model
memory transfer can degrade performance, because stored
memories often entangle task-relevant knowledge with model-specific biases. To address this challenge, we propose MemCollab, a collaborative
memory framework that builds shared cross-model
memory by contrasting
reasoning trajectories generated by different model-based
agents on the same task. Through this contrastive process, MemCollab distills abstract
reasoning constraints that capture shared task-level invariants while suppressing model-specific artifacts. We further introduce a task-aware
retrieval mechanism that conditions
memory access on task category, ensuring that only relevant constraints are retrieved at inference time. Experiments on mathematical
reasoning and code generation
benchmarks show that MemCollab consistently improves both accuracy and inference-time efficiency across diverse
agents, including settings with different model families. These results demonstrate that collaboratively constructed cross-model
memory can serve as a shared
reasoning resource for heterogeneous LLM-based
agents.
Read the full abstract ▾
All authors Yurui Chang, Yiran Wu, Qingyun Wu, Lu Lin
Machine Learning
GEM: Guided Expectation-Maximization for Behavior-Normalized Candidate Action Selection in Offline RL
Haoyu Wang, Jingcheng Wang, Shunyu Wu et al. (+1 more)
Offline reinforcement learning (RL) can fit strong value functions from fixed
datasets, yet reliable deployment still hinges on the action selection interface used to query them. When the
dataset induces a branched or
multimodal action landscape, unimodal policy extraction can blur competing hypotheses and yield "in-between" actions that are weakly supported by data, making decisions brittle even with a strong critic. We introduce GEM (Guided Expectation-Maximization), an analytical framework that makes action selection both
multimodal and explicitly controllable. GEM trains a Gaussian Mixture Model (GMM) actor via critic-guided, advantage-weighted EM-style updates that preserve distinct components while shifting probability mass toward high-value regions, and learns a tractable GMM behavior model to quantify support. During inference, GEM performs candidate-based selection: it generates a parallel candidate set and reranks actions using a conservative ensemble lower-confidence bound together with behavior-normalized support, where the behavior log-likelihood is standardized within each state's candidate set to yield stable, comparable control across states and candidate budgets. Empirically, GEM is competitive across D4RL
benchmarks, and offers a simple inference-time budget knob (candidate count) that trades compute for decision quality without retraining.
Read the full abstract ▾
All authors Haoyu Wang, Jingcheng Wang, Shunyu Wu, Xinwei Xiao
Machine Learning
General Machine Learning: Theory for Learning Under Variable Regimes
Aomar Osmani
We study learning under regime variation, where the learner, its
memory state, and the evaluative conditions may evolve over time. This paper is a foundational and structural contribution: its goal is to define the core learning-theoretic objects required for such settings and to establish their first
theorem-supporting consequences. The paper develops a regime-varying framework centered on admissible transport, protected-core preservation, and evaluator-aware learning
evolution. It records the immediate closure consequences of admissibility, develops a structural obstruction argument for faithful fixed-ontology reduction in genuinely multi-regime settings, and introduces a protected-stability template together with explicit numerical and symbolic witnesses on controlled subclasses, including convex and deductive settings. It also establishes
theorem-layer results on evaluator factorization, morphisms, composition, and partial kernel-level
alignment across semantically commensurable layers. A worked two-regime example makes the admissibility certificate, protected evaluative core, and regime-variation cost explicit on a controlled subclass. The symbolic component is deliberately restricted in scope: the paper establishes a first kernel-level compatibility result together with a controlled monotonic deductive witness. The manuscript should therefore be read as introducing a structured learning-theoretic framework for regime-varying learning together with its first
theorem-supporting layer, not as a complete quantitative theory of all learning systems.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Decoding AI Authorship: Can LLMs Truly Mimic Human Style Across Literature and Politics?
Nasser A Alsadhan
Amidst the rising capabilities of generative AI to mimic specific human styles, this study investigates the ability of state-of-the-art large
language models (LLMs), including GPT-4o, Gemini 1.5 Pro, and Claude Sonnet 3.5, to emulate the authorial signatures of prominent literary and political figures: Walt Whitman, William Wordsworth, Donald Trump, and Barack Obama. Utilizing a zero-shot prompting framework with strict thematic
alignment, we generated synthetic corpora evaluated through a complementary framework combining
transformer-based classification (BERT) and interpretable machine learning (XGBoost). Our methodology integrates Linguistic Inquiry and Word Count (LIWC) markers, perplexity, and readability indices to assess the divergence between AI-generated and human-authored text. Results demonstrate that AI-generated mimicry remains highly detectable, with XGBoost models trained on a restricted set of eight stylometric features achieving accuracy comparable to high-dimensional neural classifiers. Feature importance analyses identify perplexity as the primary discriminative metric, revealing a significant divergence in the
stochastic regularity of AI outputs compared to the higher variability of human writing. While LLMs exhibit distributional convergence with human authors on low-dimensional heuristic features, such as syntactic complexity and readability, they do not yet fully replicate the nuanced affective density and stylistic variance inherent in the human-authored corpus. By isolating the specific statistical gaps in current generative mimicry, this study provides a comprehensive
benchmark for LLM stylistic behavior and offers critical insights for authorship attribution in the digital humanities and social media.
Read the full abstract ▾
Machine Learning · Cross-listed from cond-mat.dis-nn
Generative Inversion of Spectroscopic Data for Amorphous Structure Elucidation
Jiawei Guo and Daniel Schwalbe-Koda
Determining atomistic structures from characterization data is one of the most common yet intricate problems in materials science. Particularly in amorphous materials, proposing structures that balance realism and agreement with experiments requires expert guidance, good interatomic potentials, or both. Here, we introduce GLASS, a generative framework that inverts multi-modal spectroscopic measurements into realistic atomistic structures without knowledge of the potential energy surface. A score-based model learns a structural prior from low-fidelity data and samples out-of-distribution structures conditioned on differentiable spectral targets. Reconstructions using pair distribution functions (PDFs), X-ray absorption spectroscopy, and diffraction measurements quantify the complementarity between spectral modalities and demonstrate that PDFs is the most informative probe for our framework. We use GLASS to rationalize three contested experimental problems: paracrystallinity in amorphous silicon, a
liquid-
liquid phase transition in sulfur, and ball-milled amorphous ice. In each case, generated structures reproduce experimental measurements and reveal mechanisms inaccessible to diffraction analysis alone.
Read the full abstract ▾
Machine Learning
A One-Inclusion Graph Approach to Multi-Group Learning
Noah Bergam, Samuel Deng and Daniel Hsu
We prove the tightest-known upper bounds on the sample complexity of multi-group learning. Our
algorithm extends the one-inclusion graph prediction strategy using a generalization of bipartite $b$-matching. In the group-realizable setting, we provide a lower bound confirming that our
algorithm's $\log n / n$ convergence rate is optimal in general. If one relaxes the learning objective such that the group on which we are evaluated is chosen obliviously of the sample, then our
algorithm achieves the optimal $1/n$ convergence rate under group-realizability.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Between Resolution Collapse and Variance Inflation: Weighted Conformal Anomaly Detection in Low-Data Regimes
Oliver Hennhöfer and Christine Preisach
Standard conformal anomaly detection provides marginal finite-sample guarantees under the assumption of exchangeability . However, real-world data often exhibit distribution shifts, necessitating a weighted conformal approach to adapt to local non-stationarity. We show that this adaptation induces a critical trade-off between the minimum attainable p-value and its stability. As importance weights localize to relevant calibration instances, the effective sample size decreases. This can render standard conformal p-values overly conservative for effective error control, while the smoothing technique used to mitigate this issue introduces conditional variance, potentially masking anomalies. We propose a continuous inference relaxation that resolves this dilemma by decoupling local adaptation from tail resolution via continuous weighted kernel density estimation. While relaxing finite-sample exactness to asymptotic validity, our method eliminates Monte Carlo variability and recovers the statistical power lost to discretization. Empirical evaluations confirm that our approach not only restores detection capabilities where discrete baselines yield zero discoveries, but outperforms standard methods in statistical power while maintaining valid marginal error control in practice.
Read the full abstract ▾
Machine Learning
Edoardo Cetin, Stefano Peluchetti, Emilio Castillo et al. (+3 more)
Scaling autoregressive large
language models (LLMs) has driven unprecedented progress but comes with vast computational costs. In this work, we tackle these costs by leveraging unstructured sparsity within an LLM's feedforward layers, the components accounting for most of the model parameters and execution FLOPs. To achieve this, we introduce a new sparse packing format and a set of CUDA kernels designed to seamlessly integrate with the optimized execution pipelines of modern GPUs, enabling efficient sparse computation during LLM inference and training. To substantiate our gains, we provide a quantitative study of LLM sparsity, demonstrating that simple L1 regularization can induce over 99% sparsity with negligible impact on downstream performance. When paired with our kernels, we show that these sparsity levels translate into substantial throughput, energy efficiency, and
memory usage benefits that increase with model scale. We will release all code and kernels under an open-source license to promote adoption and accelerate research toward establishing sparsity as a practical axis for improving the efficiency and scalability of modern foundation models.
Read the full abstract ▾
All authors Edoardo Cetin, Stefano Peluchetti, Emilio Castillo, Akira Naruse, Mana Murakami, Llion Jones
Machine Learning · Cross-listed from cs.GR
PhysSkin: Real-Time and Generalizable Physics-Based Animation via Self-Supervised Neural Skinning
Yuanhang Lei, Tao Cheng, Xingxuan Li et al. (+6 more)
Achieving real-time physics-based animation that generalizes across diverse 3D shapes and discretizations remains a fundamental challenge. We introduce PhysSkin, a physics-informed framework that addresses this challenge. In the spirit of Linear Blend Skinning, we learn continuous skinning fields as basis functions lifting motion subspace coordinates to full-space deformation, with subspace defined by handle transformations. To generate mesh-free, discretization-agnostic, and physically consistent skinning fields that generalize well across diverse 3D shapes, PhysSkin employs a new neural skinning fields autoencoder which consists of a
transformer-based encoder and a cross-
attention decoder. Furthermore, we also develop a novel physics-informed self-supervised learning strategy that incorporates on-the-fly skinning-field normalization and conflict-aware gradient correction, enabling effective balancing of energy minimization, spatial smoothness, and orthogonality constraints. PhysSkin shows outstanding performance on generalizable neural skinning and enables real-time physics-based animation.
Read the full abstract ▾
All authors Yuanhang Lei, Tao Cheng, Xingxuan Li, Boming Zhao, Siyuan Huang, Ruizhen Hu, Peter Yichen Chen, Hujun Bao, Zhaopeng Cui
Machine Learning
A Schrödinger Eigenfunction Method for Long-Horizon Stochastic Optimal Control
Louis Claeys, Artur Goldman, Zebang Shen et al. (+1 more)
High-dimensional
stochastic optimal control (SOC) becomes harder with longer planning horizons: existing methods scale linearly in the horizon $T$, with performance often deteriorating exponentially. We overcome these limitations for a subclass of linearly-solvable SOC problems-those whose uncontrolled drift is the gradient of a potential. In this setting, the Hamilton-Jacobi-Bellman equation reduces to a linear PDE governed by an operator $\mathcal{L}$. We prove that, under the gradient drift assumption, $\mathcal{L}$ is unitarily equivalent to a Schrödinger operator $\mathcal{S} = -\Delta + \mathcal{V}$ with purely discrete spectrum, allowing the long-horizon control to be efficiently described via the eigensystem of $\mathcal{L}$. This connection provides two key results: first, for a symmetric linear-quadratic regulator (LQR), $\mathcal{S}$ matches the
Hamiltonian of a
quantum harmonic oscillator, whose closed-form eigensystem yields an analytic solution to the symmetric LQR with arbitrary terminal cost. Second, in a more general setting, we learn the eigensystem of $\mathcal{L}$ using neural networks. We identify implicit reweighting issues with existing eigenfunction learning losses that degrade performance in control tasks, and propose a novel loss function to mitigate this. We evaluate our method on several long-horizon
benchmarks, achieving an order-of-magnitude improvement in control accuracy compared to state-of-the-art methods, while reducing
memory usage and runtime complexity from $\mathcal{O}(Td)$ to $\mathcal{O}(d)$.
Read the full abstract ▾
All authors Louis Claeys, Artur Goldman, Zebang Shen, Niao He
Machine Learning · Cross-listed from cs.CR
Adaptively Robust LLM Monitoring via Activation Watermarking
Toluwani Aremu, Daniil Ognev, Samuele Poppi et al. (+1 more)
Providers monitor deployed large
language models (LLMs) to detect misuse that they cannot prevent. LLM monitoring is deterministic and often openly available, so $\emph{adaptive}$ attackers with a local copy can search offline for
prompts that elicit harmful behavior and evade detection. These attacks are especially concerning because providers never observe the misuse and cannot patch their defenses post-hoc. The core challenge is resisting adaptive attackers while preserving detection rates against non-adaptive ones. We propose $\emph{Activation Watermarking}$ (AWM), which randomizes monitoring through limited fine-tuning that aligns the LLM's hidden states with a secret key-derived direction whenever a response violates a policy. Detection is a similarity test on activations the provider already computes, and attackers who know everything but the key must optimize against differently keyed surrogate detectors. At a matched $1\%$ false-positive rate, such surrogate-based attacks evade every evaluated baseline at least $79\%$ of the time, but AWM more than halves this evasion rate. AWM also achieves the lowest evasion rate on three of four non-adaptive jailbreak families at a small drop in utility measured across seven
benchmarks. Because random high-dimensional directions are near-orthogonal, assigning one watermark per policy enables attribution. Across 20 monitored policies, AWM identifies the violated policy with $80\%$ accuracy versus a $5\%$ chance baseline. AWM is (i) efficient, (ii) substantially more robust against adaptive attackers than related work and (iii) can attribute which policies were violated.
Read the full abstract ▾
All authors Toluwani Aremu, Daniil Ognev, Samuele Poppi, Nils Lukas
Machine Learning · Cross-listed from cs.CV
Conformal Cross-Modal Active Learning
Huy Hoang Nguyen, Cédric Jung, Shirin Salehi et al. (+3 more)
Foundation models for
vision have transformed visual recognition with powerful pretrained representations and strong zero-shot capabilities, yet their potential for data-efficient learning remains largely untapped. Active Learning (AL) aims to minimize annotation costs by strategically selecting the most informative samples for labeling, but existing methods largely overlook the rich
multimodal knowledge embedded in modern
vision-
language models (VLMs). We introduce Conformal Cross-Modal Acquisition (CCMA), a novel AL framework that bridges
vision and
language modalities through a teacher-student architecture. CCMA employs a pretrained VLM as a teacher to provide semantically grounded uncertainty estimates, conformally calibrated to guide sample selection for a
vision-only student model. By integrating
multimodal conformal scoring with diversity-aware selection strategies, CCMA achieves superior data efficiency across multiple
benchmarks. Our approach consistently outperforms state-of-the-art AL baselines, demonstrating clear advantages over methods relying solely on uncertainty or diversity metrics.
Read the full abstract ▾
All authors Huy Hoang Nguyen, Cédric Jung, Shirin Salehi, Tobias Glück, Anke Schmeink, Andreas Kugi
Machine Learning
DAK-UCB: Diversity-Aware Prompt Routing for LLMs and Generative Models
Donya Jafari and Farzan Farnia
The expansion of generative AI and LLM services underscores the growing need for adaptive mechanisms to select an appropriate available model to respond to a user's
prompts. Recent works have proposed offline and online learning formulations to identify the optimal generative AI model for an input
prompt, based solely on maximizing
prompt-based fidelity evaluation scores, e.g., CLIP-Score in text-to-image generation. However, such fidelity-based selection methods overlook the diversity of generated outputs, and hence, they can fail to address potential diversity shortcomings in the generated responses. In this paper, we introduce the Diversity-Aware Kernelized Upper Confidence Bound (DAK-UCB) method as a contextual bandit
algorithm for the online selection of generative models with diversity considerations. The proposed DAK-UCB method incorporates both fidelity and diversity-related metrics into the selection process. We design this framework based on
prompt-aware diversity score functions that decompose to a two-sample-based expectation over
prompt-output pairs in the previous generation rounds. Specifically, we illustrate the application of our framework using joint kernel distance and kernel
entropy measures. Our experimental results demonstrate the effectiveness of DAK-UCB in promoting diversity-aware model selection while maintaining fidelity in the generations for a sequence of
prompts. The code is available at https://github.com/Donya-Jafari/DAK-UCB.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
HGNet: Scalable Foundation Model for Automated Knowledge Graph Generation from Scientific Literature
Devvrat Joshi and Islem Rekik
Automated knowledge graph (KG) construction is essential for navigating the rapidly expanding body of scientific literature. However, existing approaches struggle to recognize long multi-word entities, often fail to generalize across domains, and typically overlook the hierarchical nature of scientific knowledge. While general-purpose large
language models (LLMs) offer adaptability, they are computationally expensive and yield inconsistent accuracy on specialized tasks. As a result, current KGs are shallow and inconsistent, limiting their utility for exploration and synthesis. We propose a two-stage framework for scalable, zero-shot scientific KG construction. The first stage, Z-NERD, introduces (i) Orthogonal Semantic Decomposition (OSD), which promotes domain-agnostic entity recognition by isolating semantic "turns" in text, and (ii) a Multi-Scale TCQK
attention mechanism that captures coherent multi-word entities through n-gram-aware
attention heads. The second stage, HGNet, performs relation extraction with hierarchy-aware message passing, explicitly modeling parent, child, and peer relations. To enforce global consistency, we introduce two complementary objectives: a Differentiable Hierarchy Loss to discourage cycles and shortcut edges, and a Continuum Abstraction Field (CAF) Loss that embeds abstraction levels along a learnable axis in Euclidean space. This is the first approach to formalize hierarchical abstraction as a continuous property within standard Euclidean
embeddings, offering a simpler alternative to
hyperbolic methods. We release SPHERE (https://github.com/basiralab/SPHERE), a multi-domain
benchmark for hierarchical relation extraction. Our framework establishes a new state of the art on SciERC, SciER, and SPHERE, improving NER by 8.08% and RE by 5.99% on out-of-distribution tests. In zero-shot settings, gains reach 10.76% for NER and 26.2% for RE.
Read the full abstract ▾
Machine Learning
A Bayesian Learning Approach for Drone Coverage Network: A Case Study on Cardiac Arrest in Scotland
Tathagata Basu, Edoardo Patelli, Gianluca Filippi et al. (+7 more)
Drones are becoming popular as a complementary system for Emergency Medical Services (EMS). Although several pilot studies and flight trials have shown the feasibility of drone-assisted Automated External Defibrillator (AED) delivery, running a full-scale operational network remains challenging due to high capital expenditure and environmental uncertainties. In this paper, we formulate a reliability-informed
Bayesian learning framework for designing drone-assisted AED delivery networks under environmental and operational uncertainty. We propose our objective function based on the survival probability of Out-off Hospital Cardiac Arrest (OHCA) patients to identify the ideal locations of drone stations. Moreover, we consider the coverage of existing EMS infrastructure to improve the response reliability in remote areas. We illustrate our proposed method using geographically referenced cardiac arrest data from Scotland. The result shows how environmental variability and spatial demand patterns influence optimal drone station placement across urban and rural regions. In addition, we assess the robustness of the network and evaluate its economic viability using a cost-effectiveness analysis based on expected Quality Adjusted Life Year (QALY). The findings suggest that drone-assisted AED delivery is expected to be cost-effective and has the potential to significantly improve the emergency response coverage in rural and urban areas with longer ambulance response times.
Read the full abstract ▾
All authors Tathagata Basu, Edoardo Patelli, Gianluca Filippi, Ben Parsonage, Christy Maddock, Massimiliano Vasile, Marco Fossati, Adam Loyd, Shaun Marshall, Paul Gowens
Machine Learning
Polaris: A Gödel Agent Framework for Small Language Models through Experience-Abstracted Policy Repair
Aditya Kakade, Vivek Srivastava and Shirish Karande
Gödel
agent realize recursive self-improvement: an
agent inspects its own policy and traces and then modifies that policy in a tested loop. We introduce Polaris, Gödel
agent for compact models that performs policy repair via experience abstraction, turning failures into policy updates through a structured cycle of analysis, strategy formation, abstraction, and minimal code patch repair with conservative checks. Unlike response level self correction or parameter tuning, Polaris makes policy level changes with small, auditable patches that persist in the policy and are reused on unseen instances within each
benchmark. As part of the loop, the
agent engages in meta
reasoning: it explains its errors, proposes concrete revisions to its own policy, and then updates the policy. To enable cumulative policy refinement, we introduce experience abstraction, which distills failures into compact, reusable strategies that transfer to unseen instances. On MGSM, DROP, GPQA, and LitBench (covering arithmetic
reasoning, compositional inference, graduate-level problem solving, and creative writing evaluation), a 7-billion-parameter model equipped with Polaris achieves consistent gains over the base policy and competitive baselines.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
High-Resolution Tensor-Network Fourier Methods for Exponentially Compressed Non-Gaussian Aggregate Distributions
Juan José Rodríguez-Aldavero and Juan José García-Ripoll
Characteristic functions of weighted sums of independent random variables exhibit low-rank structure in the quantized tensor train (QTT) representation, also known as
matrix product states (MPS), enabling up to exponential compression of their fully non-Gaussian probability distributions. Under variable independence, the global characteristic function factorizes into local terms. Its low-rank QTT structure arises from intrinsic spectral smoothness in continuous models, or from spectral energy concentration as the number of components $D$ grows in discrete models. We demonstrate this on weighted sums of Bernoulli and lognormal random variables. In the former, despite an
adversarial, incompressible small-$D$ regime, the characteristic function undergoes a sharp bond-dimension collapse for $D \gtrsim 300$ components, enabling polylogarithmic time and
memory scaling. In the latter, the approach reaches high-resolution discretizations of $N = 2^{30}$ frequency modes on standard hardware, far beyond the $N = 2^{24}$ ceiling of dense implementations. These compressed representations enable efficient computation of Value at Risk (VaR) and Expected Shortfall (ES), supporting applications in quantitative finance and beyond.
Read the full abstract ▾
Machine Learning
SpecXMaster Technical Report
Yutang Ge, Yaning Cui, Hanzheng Li et al. (+18 more)
Intelligent spectroscopy serves as a pivotal element in AI-driven closed-loop scientific discovery, functioning as the critical bridge between matter structure and artificial intelligence. However, conventional expert-dependent spectral interpretation encounters substantial hurdles, including susceptibility to human bias and error, dependence on limited specialized expertise, and variability across interpreters. To address these challenges, we propose SpecXMaster, an intelligent framework leveraging Agentic Reinforcement Learning (RL) for NMR molecular spectral interpretation. SpecXMaster enables automated extraction of multiplicity information from both 1H and 13C spectra directly from raw FID (free induction decay) data. This end-to-end pipeline enables fully automated interpretation of NMR spectra into chemical structures. It demonstrates superior performance across multiple public NMR interpretation
benchmarks and has been refined through iterative evaluations by professional chemical spectroscopists. We believe that SpecXMaster, as a novel methodological paradigm for spectral interpretation, will have a profound impact on the organic chemistry community.
Read the full abstract ▾
All authors Yutang Ge, Yaning Cui, Hanzheng Li, Jun-Jie Wang, Fanjie Xu, Jinhan Dong, Yongqi Jin, Dongxu Cui, Peng Jin, Guojiang Zhao, Hengxing Cai, Tianci Yangfeng, Xueqing Chen, Hongshuai Wang, Changhao Nai, Jiarun Liu, Zheng Zhang, Rong Zhu, Linfeng Zhang, Xiaohong Ji, Zhifeng Gao
Machine Learning
Policy-based Tuning of Autoregressive Image Models with Instance- and Distribution-Level Rewards
Orhun Bugra Baran, Melih Kandemir and Ramazan Gokberk Cinbis
Autoregressive (AR) models are highly effective for image generation, yet their standard maximum-likelihood estimation training lacks direct optimization for sample quality and diversity. While reinforcement learning (RL) has been used to align
diffusion models, these methods typically suffer from output diversity collapse. Similarly, concurrent RL methods for AR models rely strictly on instance-level rewards, often trading off distributional coverage for quality. To address these limitations, we propose a lightweight RL framework that casts
token-based AR synthesis as a Markov Decision Process, optimized via Group Relative Policy Optimization (GRPO). Our core contribution is the introduction of a novel distribution-level Leave-One-Out FID (LOO-FID) reward; by leveraging an exponential moving average of feature moments, it explicitly encourages sample diversity and prevents mode collapse during policy updates. We integrate this with composite instance-level rewards (CLIP and HPSv2) for strict semantic and perceptual fidelity, and stabilize the multi-objective learning with an adaptive
entropy regularization term. Extensive experiments on LlamaGen and VQGAN architectures demonstrate clear improvements across standard quality and diversity metrics within only a few hundred tuning iterations. The results also show that the model can be updated to produce competitive samples even without Classifier-Free Guidance, and bypass its 2x inference cost.
Read the full abstract ▾
Machine Learning
MsFormer: Enabling Robust Predictive Maintenance Services for Industrial Devices
Jiahui Zhou, Dan Li, Ruibing Jin et al. (+5 more)
Providing reliable predictive maintenance is a critical industrial AI service essential for ensuring the high availability of manufacturing devices. Existing deep-learning methods present competitive results on such tasks but lack a general service-oriented framework to capture complex dependencies in industrial IoT sensor data. While
Transformer-based models show strong sequence modeling capabilities, their direct deployment as robust AI services faces significant bottlenecks. Specifically, streaming sensor data collected in real-world service environments often exhibits multi-scale temporal correlations driven by machine working principles. Besides, the
datasets available for training time-to-failure predictive services are typically limited in size. These issues pose significant challenges for directly applying existing models as robust predictive services. To address these challenges, we propose MsFormer, a lightweight Multi-scale
Transformer designed as a unified AI service model for reliable industrial predictive maintenance. MsFormer incorporates a Multi-scale Sampling (MS) module and a tailored position encoding mechanism to capture sequential correlations across multi-streaming service data. Additionally, to accommodate data-scarce service environments, MsFormer adopts a lightweight
attention mechanism with straightforward pooling operations instead of self-
attention. Extensive experiments on real-world
datasets demonstrate that the proposed framework achieves significant performance improvements over state-of-the-art methods. Furthermore, MsFormer outperforms across industrial devices and operating conditions, demonstrating strong generalizability while maintaining a highly reliable Quality of Service (QoS).
Read the full abstract ▾
All authors Jiahui Zhou, Dan Li, Ruibing Jin, Jian Lou, Yanran Zhao, Zhenghua Chen, Zigui Jiang, See-Kiong Ng
Machine Learning
Generalization Bounds for Physics-Informed Neural Networks for the Incompressible Navier-Stokes Equations
Sebastien Andre-Sloan, Dibyakanti Kumar, Alejandro F Frangi et al. (+1 more)
This work establishes rigorous first-of-its-kind upper bounds on the generalization error for the method of approximating solutions to the (d+1)-dimensional incompressible Navier-Stokes equations by training depth-2 neural networks trained via the unsupervised Physics-Informed Neural Network (PINN) framework. This is achieved by bounding the Rademacher complexity of the PINN risk. For appropriately weight bounded net classes our derived generalization bounds do not explicitly depend on the network width and our framework characterizes the generalization gap in terms of the fluid's kinematic viscosity and loss regularization parameters. In particular, the resulting sample complexity bounds are dimension-independent. Our generalization bounds suggest using novel activation functions for solving fluid dynamics. We provide empirical validation of the suggested activation functions and the corresponding bounds on a PINN setup solving the Taylor-Green
vortex benchmark.
Read the full abstract ▾
All authors Sebastien Andre-Sloan, Dibyakanti Kumar, Alejandro F Frangi, Anirbit Mukherjee
Machine Learning · Cross-listed from cs.SE
Machine Learning Models for the Early Detection of Burnout in Software Engineering: a Systematic Literature Review
Tien Rahayu Tulili, Ayushi Rastogi and Andrea Capiluppi
Burnout is an occupational syndrome that, like many other professions, affects the majority of software engineers. Past research studies showed important trends, including an increasing use of machine learning techniques to allow for an early detection of burnout. This paper is a systematic literature review (SLR) of the research papers that proposed machine learning (ML) approaches, and focused on detecting burnout in software developers and IT professionals. Our objective is to review the accuracy and precision of the proposed ML techniques, and to formulate recommendations for future researchers interested to replicate or extend those studies. From our SLR we observed that a majority of primary studies focuses on detecting
emotions or utilise emotional dimensions to detect or predict the presence of burnout. We also performed a cross-sectional study to detect which ML approach shows a better performance at detecting
emotions; and which
dataset has more potential and expressivity to capture
emotions. We believe that, by identifying which ML tools and
datasets show a better performance at detecting
emotions, and indirectly at identifying burnout, our paper can be a valuable asset to progress in this important research direction.
Read the full abstract ▾
Machine Learning · Cross-listed from eess.AS
Prompt Amplification and Zero-Shot Late Fusion in Audio-Language Models for Speech Emotion Recognition
Saurabh Kataria and Xiao Hu
Audio-
Language Models (ALMs) are making strides in understanding
speech and non-
speech audio. However, domain-specialist Foundation Models (FMs) remain the best for closed-ended
speech processing tasks such as
Speech Emotion Recognition (SER). Using ALMs for Zero-shot SER is a popular choice, but their potential to work with specialists to achieve state-of-the-art (SOTA) performance remains unexplored. We propose ZS-Fuse, a late-fusion method that combines zero-shot
emotion estimates from a dual-encoder ALM with specialist FMs. To handle ambiguity in
emotions and sensitivity to
prompt choice, 1) we use a simple
prompt ensemble and 2) suggest a novel technique called
prompt amplification, which repeats audio and text queries to discover stronger zero-shot capabilities. We demonstrate the efficacy of our technique by evaluating ZS-Fuse with three dual-encoder ALMs and two FMs, and report improvements over SOTA baselines, such as WavLM-Large, on three
speech emotion recognition
datasets.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Post-Selection Distributional Model Evaluation
Amirmohammad Farzaneh and Osvaldo Simeone
Formal model evaluation methods typically certify that a model satisfies a prescribed target key performance indicator (KPI) level. However, in many applications, the relevant target KPI level may not be known a priori, and the user may instead wish to compare candidate models by analyzing the full trade-offs between performance and reliability achievable at test time by the models. This task, requiring the reliable estimate of the test-time KPI distributions, is made more complicated by the fact that the same data must often be used both to pre-select a subset of candidate models and to estimate their KPI distributions, causing a potential post-selection bias. In this work, we introduce post-selection distributional model evaluation (PS-DME), a general framework for statistically valid distributional model assessment after arbitrary data-dependent model pre-selection. Building on e-values, PS-DME controls post-selection false coverage rate (FCR) for the distributional KPI estimates and we establish explicit conditions under which it is provably more sample efficient than a baseline method based on sample splitting. Experiments on synthetic data, text-to-SQL decoding with large
language models, and telecom network performance evaluation demonstrate that PS-DME enables reliable comparison of candidate configurations across a range of reliability levels, supporting the statistically reliable exploration of performance–reliability trade-offs.
Read the full abstract ▾