Machine Learning · Cross-listed from cond-mat.dis-nn
From Classical to Quantum: Extending Prometheus for Unsupervised Discovery of Phase Transitions in Three Dimensions and Quantum Systems
Brandon Yee, Wilson Collins, Pairie Koh et al. (+1 more)
We extend the Prometheus framework for unsupervised phase transition discovery from two-dimensional classical systems to three-dimensional classical systems and
quantum many-body systems. Building upon preliminary observations from a 2D Ising model student abstract [Yee et al., 2026], we address two fundamental questions: (1) Does the framework scale to higher dimensions where exact solutions are unavailable? (2) Can it generalize to
quantum phase transitions driven by
quantum fluctuations rather than thermal fluctuations? For the 3D Ising model on
lattices up to $L{=}32$, we achieve critical temperature detection within 0.01% of literature values ($\Tc/J = 4.511 \pm 0.005$) and extract critical exponents with ${\geq}70\%$ accuracy, with statistical analysis correctly identifying the 3D Ising universality class ($p = 0.72$). For
quantum systems, we develop
quantum-aware VAE (Q-VAE) architectures operating on complex-valued wavefunctions with fidelity-based loss functions, achieving 2% accuracy in
quantum critical point detection for the transverse field Ising model. For the disordered TFIM, we perform a consistency check of activated dynamical scaling $\ln \xi \sim |h - \hc|^{-\psi}$, extracting tunneling exponent $\psi = 0.48 \pm 0.08$ consistent with theoretical predictions ($\psi = 0.5$, $\Delta\chi^2 = 12.3$, $p < 0.001$). This demonstrates that unsupervised learning can identify qualitatively different types of critical behavior, serving as a consistency check on known IRFP physics.
Read the full abstract ▾
All authors Brandon Yee, Wilson Collins, Pairie Koh, Maximilian Rutkowski
Machine Learning
BHyGNN+: Unsupervised Representation Learning for Heterophilic Hypergraphs
Tianyi Ma, Yiyue Qian, Zehong Wang et al. (+3 more)
Hypergraph Neural Networks (HyGNNs) have demonstrated remarkable success in modeling higher-order relationships among entities. However, their performance often degrades on heterophilic hypergraphs, where nodes connected by the same hyperedge tend to have dissimilar semantic representations or belong to different classes. While several HyGNNs, including our prior work BHyGNN, have been proposed to address heterophily, their reliance on labeled data significantly limits their applicability in real-world scenarios where annotations are scarce or costly. To overcome this limitation, we introduce BHyGNN+, a self-supervised learning framework that extends BHyGNN for representation learning on heterophilic hypergraphs without requiring ground-truth labels. The core idea of BHyGNN+ is hypergraph duality, a structural transformation where the roles of nodes and hyperedges are interchanged. By contrasting augmented views of a hypergraph against its dual using cosine similarity, our framework captures essential structural patterns in a fully unsupervised manner. Notably, this duality-based formulation eliminates the need for negative samples, a common requirement in existing hypergraph contrastive learning methods that is often difficult to satisfy in practice. Extensive experiments on eleven
benchmark datasets demonstrate that BHyGNN+ consistently outperforms state-of-the-art supervised and self-supervised baselines on both heterophilic and homophilic hypergraphs. Our results validate the effectiveness of leveraging hypergraph duality for self-supervised learning and establish a new paradigm for representation learning on challenging, unlabeled hypergraphs.
Read the full abstract ▾
All authors Tianyi Ma, Yiyue Qian, Zehong Wang, Zheyuan Zhang, Chuxu Zhang, Yanfang Ye
Machine Learning
Additive Control Variates Dominate Self-Normalisation in Off-Policy Evaluation
Olivier Jeunen and Shashank Gupta
Off-policy evaluation (OPE) is essential for assessing ranking and recommendation systems without costly online interventions. Self-Normalised Inverse Propensity Scoring (SNIPS) is a standard tool for variance reduction in OPE, leveraging a multiplicative control variate. Recent advances in off-policy learning suggest that additive control variates (baseline corrections) may offer superior performance, yet theoretical guarantees for evaluation are lacking. This paper provides a definitive answer: we prove that $\beta^\star$-IPS, an
estimator with an optimal additive baseline, asymptotically dominates SNIPS in Mean Squared Error. By analytically decomposing the variance gap, we show that SNIPS is asymptotically equivalent to using a specific – but generally sub-optimal – additive baseline. Our results theoretically justify shifting from self-normalisation to optimal baseline corrections for both ranking and recommendation.
Read the full abstract ▾
Machine Learning
Coverage Guarantees for Pseudo-Calibrated Conformal Prediction under Distribution Shift
Farbod Siahkali, Ashwin Verma and Vijay Gupta
Conformal prediction (CP) offers distribution-free marginal coverage guarantees under an exchangeability assumption, but these guarantees can fail if the data distribution shifts. We analyze the use of pseudo-calibration as a tool to counter this performance loss under a bounded label-conditional
covariate shift model. Using tools from domain adaptation, we derive a lower bound on target coverage in terms of the source-domain loss of the classifier and a Wasserstein measure of the shift. Using this result, we provide a method to design pseudo-calibrated sets that inflate the conformal threshold by a slack parameter to keep target coverage above a prescribed level. Finally, we propose a source-tuned pseudo-calibration
algorithm that interpolates between hard pseudo-labels and randomized labels as a function of classifier uncertainty. Numerical experiments show that our bounds qualitatively track pseudo-calibration behavior and that the source-tuned scheme mitigates coverage degradation under distribution shift while maintaining nontrivial prediction set sizes.
Read the full abstract ▾
Machine Learning · Cross-listed from physics.flu-dyn
Adjoint-based shape optimization of a ship hull using a Conditional Variational Autoencoder (CVAE) assisted propulsion surrogate model
Moloud Arian Maram, Georgios Bletsos, Thanh Tung Nguyen et al. (+3 more)
Adjoint-based shape optimization of ship hulls is a powerful tool for addressing high-dimensional design problems in naval architecture, particularly in minimizing the ship resistance. However, its application to vessels that employ complex propulsion systems introduces significant challenges. They arise from the need for transient simulations extending over long periods of time with small time steps and from the reverse temporal propagation of the primal and adjoint solutions. These challenges place considerable demands on the required storage and computing power, which significantly hamper the use of adjoint methods in the industry. To address this issue, we propose a machine learning-assisted optimization framework that employs a Conditional Variational Autoencoder-based surrogate model of the propulsion system. The surrogate model replicates the time-averaged flow field induced by a Voith Schneider Propeller and replaces the geometrically and time-resolved propeller with a data-driven approximation. Primal flow verification examples demonstrate that the surrogate model achieves significant computational savings while maintaining the necessary accuracy of the resolved propeller. Optimization studies demonstrate that neglecting the propulsion system can result in hull designs whose performance is inferior to that of the initial shape when subsequently validated using a numerically resolved propulsor. In contrast, the proposed method produces shapes that actually achieve more than an 8% reduction in resistance.
Read the full abstract ▾
All authors Moloud Arian Maram, Georgios Bletsos, Thanh Tung Nguyen, Ahmed Hassan, Michael Palm, Thomas Rung
Machine Learning
Picking the Right Specialist: Attentive Neural Process-based Selection of Task-Specialized Models as Tools for Agentic Healthcare Systems
Pramit Saha, Joshua Strong, Mohammad Alsharid et al. (+2 more)
Task-specialized models form the backbone of agentic healthcare systems, enabling the
agents to answer clinical queries across tasks such as disease diagnosis, localization, and report generation. Yet, for a given task, a single "best" model rarely exists. In practice, each task is better served by multiple competing specialist models where different models excel on different data samples. As a result, for any given query,
agents must reliably select the right specialist model from a heterogeneous pool of tool candidates. To this end, we introduce ToolSelect, which adaptively learns model selection for tools by minimizing a population risk over sampled specialist tool candidates using a consistent surrogate of the task-conditional selection loss. Concretely, we propose an Attentive Neural Process-based selector conditioned on the query and per-model behavioral summaries to choose among the specialist models. Motivated by the absence of any established testbed, we, for the first time, introduce an agentic Chest X-ray environment equipped with a diverse suite of task-specialized models (17 disease detection, 19 report generation, 6 visual grounding, and 13 VQA) and develop ToolSelectBench, a
benchmark of 1448 queries. Our results demonstrate that ToolSelect consistently outperforms 10 SOTA methods across four different task families.
Read the full abstract ▾
All authors Pramit Saha, Joshua Strong, Mohammad Alsharid, Divyanshu Mishra, J. Alison Noble
Machine Learning
Algorithmic Simplification of Neural Networks with Mosaic-of-Motifs
Pedram Bakhtiarifard, Tong Chen, Jonathan Wenshøj et al. (+2 more)
Large-scale deep learning models are well-suited for compression. Across a variety of tasks, methods like
pruning, quantization, and knowledge
distillation have been used to achieve massive reductions in model parameters with only marginal performance drops. This raises the central question: *Why are deep neural networks suited for compression?* In this work, we take up the perspective of algorithmic complexity to explain this behavior. We hypothesize that the parameters of trained models have more structure and, hence, exhibit lower algorithmic complexity compared to the weights at (random) initialization. Furthermore, model compression methods harness this reduced algorithmic complexity to compress models. Although an unconstrained parameterization of model weights, $\mathbf{w} \in \mathbb{R}^n$, can represent arbitrary weight assignments, the solutions found during training exhibit repeatability and structure, making them simpler to implement than a trivial program. To this end, we formalize the Kolmogorov complexity of $\mathbf{w}$ by $\mathcal{K}(\mathbf{w})$. We introduce a constrained parameterization $\widehat{\mathbf{w}}$ that partitions parameters into blocks of size $s$ and restricts each block to be selected from a set of $k$ reusable motifs, specified by a reuse pattern (or mosaic). The resulting method, $\mathit{Mosaic\text{-}of\text{-}Motifs}$ (MoMos), provides a theoretically justified parameterization that biases optimization toward algorithmically simpler solutions. Empirical evidence from multiple experiments shows that MoMos consistently lowers the algorithmic complexity of neural networks during training while preserving the performance of unconstrained models. These results suggest that parameter compressibility is not only observed after training, but can be induced from the optimization domain.
Read the full abstract ▾
All authors Pedram Bakhtiarifard, Tong Chen, Jonathan Wenshøj, Erik B Dam, Raghavendra Selvan
Machine Learning
Web-Scale Multimodal Summarization using CLIP-Based Semantic Alignment
Mounvik K and N Harshit
We introduce Web-Scale
Multimodal Summarization, a lightweight framework for generating summaries by combining retrieved text and image data from web sources. Given a user-defined topic, the system performs parallel web, news, and image searches. Retrieved images are ranked using a fine-tuned CLIP model to measure semantic
alignment with topic and text. Optional BLIP captioning enables image-only summaries for stronger
multimodal coherence.The pipeline supports features such as adjustable fetch limits, semantic filtering, summary styling, and downloading structured outputs. We expose the system via a Gradio-based API with controllable parameters and preconfigured presets.Evaluation on 500 image-caption pairs with 20:1 contrastive negatives yields a ROC-AUC of 0.9270, an F1-score of 0.6504, and an accuracy of 96.99%, demonstrating strong
multimodal alignment. This work provides a configurable, deployable tool for web-scale summarization that integrates
language,
retrieval, and
vision models in a user-extensible pipeline.
Read the full abstract ▾
Machine Learning · Cross-listed from cond-mat.dis-nn
Ramón Nartallo-Kaluarachchi, Renaud Lambiotte and Alain Goriely
Recurrent neural networks (RNNs) provide a theoretical framework for understanding computation in biological neural
circuits through their ability to support rich time-dependent behaviour. Whilst high-dimensional by construction, they can be used to encode low-dimensional dynamical systems, for example by restricting to low-rank connectivity, hence mirroring the emergence of latent neural
manifolds found in empirical neural data. Here, we exploit a relationship between RNNs and a class of universal approximators to train networks that can represent arbitrary, nonlinear
stochastic differential equations (SDEs), with given drift and
diffusion coefficients, within a low-dimensional latent subspace. We show that RNNs can faithfully embed the drift and
diffusion of a given SDE, including nonlinear and nonequilibrium dynamics such as chaotic attractors. We apply this approach by constructing RNN realisations of
stochastic systems that transiently explore various attractors through both input-driven switching and autonomous transitions driven by nonequilibrium currents, which we interpret as models of associative and sequential (episodic)
memory. To elucidate how these dynamics are encoded in the network, we introduce decompositions of the RNN based on its asymmetric connectivity and its time-irreversibility. Our results show that neural populations can implement a broad class of
stochastic dynamics within low-dimensional
manifolds, unifying ideas from the dynamics of neural computation, the theory of neural
manifolds, associative
memory, and nonequilibrium statistical mechanics.
Read the full abstract ▾
Machine Learning
On the Emergence of Implicit Curriculum in RLVR Learning Dynamics
Yu Huang, Zixin Wen, Yuejie Chi et al. (+4 more)
Reinforcement learning with verifiable rewards (RLVR) has been a main driver of recent breakthroughs in large
reasoning models. Yet it remains a mystery how rewards based solely on final outcomes can help overcome the long-horizon barrier to extended
reasoning. To understand this, we develop a theory of the training dynamics of RLVR for
transformers on compositional
reasoning tasks. Our theory shows that mixed-difficulty training naturally induces an implicit curriculum: without any explicit schedule, easier problems become learnable first and shape the frontier for harder ones, creating a learning progression from easy to hard during optimization. The effectiveness of this curriculum is governed by the smoothness of the difficulty spectrum. When the spectrum is smooth, training dynamics enter a well-behaved relay regime, in which persistent gradient signals on easier problems make slightly harder ones tractable and keep training at the edge of competence. When the spectrum contains abrupt discontinuities, training undergoes grokking-type phase transitions with prolonged plateaus before progress recurs. As a technical contribution, our analysis develops and adapts techniques from
Fourier analysis on finite groups to our setting. We validate the predicted mechanisms empirically via controlled synthetic experiments and real-model RLVR runs.
Read the full abstract ▾
All authors Yu Huang, Zixin Wen, Yuejie Chi, Yuting Wei, Aarti Singh, Yingbin Liang, Yuxin Chen
Machine Learning
Goldilocks RL: Tuning Task Difficulty to Escape Sparse Rewards for Reasoning
Ilia Mahrooghi, Aryo Lotfi and Emmanuel Abbe
Reinforcement learning has emerged as a powerful paradigm for unlocking
reasoning capabilities in
language models. However, relying on sparse rewards makes this process highly sample-inefficient, as models must navigate vast search spaces with minimal feedback. While classic curriculum learning aims to mitigate this by ordering data based on complexity, prior works have primarily targeted small
datasets and do not directly transfer to the large-scale settings typical of modern
language model training. Furthermore, the right ordering for a specific model is often unclear. To address this, we propose Goldilocks, an adaptive data-selection strategy that uses a Selector network to predict the standard deviation of rewards across the model's rollouts for each candidate question. The Selector prioritizes questions with high predicted reward variability, corresponding to questions that are neither too easy nor too hard for the model's current capabilities (Goldilocks principle), while training the model with GRPO. By leveraging the model's performance on seen samples, the Selector continuously adapts to the model's evolving abilities. Across the OpenMathReasoning and Polaris
datasets, Goldilocks consistently improves over standard GRPO, requiring up to 78% fewer optimization steps to reach the corresponding GRPO performance.
Read the full abstract ▾
Machine Learning · Cross-listed from cond-mat.mtrl-sci
Fast and accurate quasi-atom method for simultaneous atomistic and continuum simulation of solids
Artem Chuprov, Egor E. Nuzhin, Alexey A. Tsukanov et al. (+1 more)
We report a novel hybrid method of simultaneous atomistic simulation of solids in critical regions (contacts surfaces, cracks areas, etc.), along with continuum modeling of other parts. The continuum is treated in terms of quasi-atoms of different size, comprising composite medium. The parameters of interaction potential between the quasi-atoms are optimized to match elastic properties of the composite medium to those of the atomic one. The optimization method coincides conceptually with the online Machine Learning (ML) methods, making it computationally very efficient. Such an approach allows a straightforward application of standard software packages for molecular dynamics (MD), supplemented by the ML-based optimizer. The new method is applied to model systems with a simple, pairwise Lennard-Jones potential, as well with multi-body Tersoff potential, describing covalent bonds. Using LAMMPS software we simulate collision of particles of different size. Comparing simulation results, obtained by the novel method, with full-atomic simulations, we demonstrate its accuracy, validity and overwhelming superiority in computational speed. Furthermore, we compare our method with other hybrid methods, specifically, with the closest one – AtC (Atomic to Continuum) method. We demonstrate a significant superiority of our approach in computational speed and implementation convenience. Finally, we discuss a possible extension of the method for modeling other phenomena.
Read the full abstract ▾
All authors Artem Chuprov, Egor E. Nuzhin, Alexey A. Tsukanov, Nikolay V. Brilliantov
Machine Learning · Cross-listed from stat.ML
The Well-Tempered Classifier: Some Elementary Properties of Temperature Scaling
Pierre-Alexandre Mattei and Bruno Loureiro
Temperature scaling is a simple method that allows to control the uncertainty of probabilistic models. It is mostly used in two contexts: improving the calibration of classifiers and tuning the stochasticity of large
language models (LLMs). In both cases, temperature scaling is the most popular method for the job. Despite its popularity, a rigorous theoretical analysis of the properties of temperature scaling has remained elusive. We investigate here some of these properties. For classification, we show that increasing the temperature increases the uncertainty in the model in a very general sense (and in particular increases its
entropy). However, for LLMs, we challenge the common claim that increasing temperature increases diversity. Furthermore, we introduce two new characterisations of temperature scaling. The first one is
geometric: the tempered model is shown to be the information projection of the original model onto the set of models with a given
entropy. The second characterisation clarifies the role of temperature scaling as a submodel of more general linear scalers such as
matrix scaling and Dirichlet calibration: we show that temperature scaling is the only linear scaler that does not change the hard predictions of the model.
Read the full abstract ▾
Machine Learning
A Pragmatic Method for Comparing Clusterings with Overlaps and Outliers
Ryan DeWolfe, Paweł Prałat and François Théberge
Clustering
algorithms are an essential part of the unsupervised data science ecosystem, and extrinsic evaluation of clustering
algorithms requires a method for comparing the detected clustering to a ground truth clustering. In a general setting, the detected and ground truth clusterings may have outliers (objects belonging to no cluster), overlapping clusters (objects may belong to more than one cluster), or both, but methods for comparing these clusterings are currently undeveloped. In this note, we define a pragmatic similarity measure for comparing clusterings with overlaps and outliers, show that it has several desirable properties, and experimentally confirm that it is not subject to several common biases afflicting other clustering comparison measures.
Read the full abstract ▾
Machine Learning
BEACONS: Bounded-Error, Algebraically-Composable Neural Solvers for Partial Differential Equations
Jonathan Gorard, Ammar Hakim and James Juno
The traditional limitations of neural networks in reliably generalizing beyond the convex hulls of their training data present a significant problem for computational physics, in which one often wishes to solve PDEs in regimes far beyond anything which can be experimentally or analytically validated. In this paper, we show how it is possible to circumvent these limitations by constructing formally-verified neural network solvers for PDEs, with rigorous convergence, stability, and conservation properties, whose correctness can therefore be guaranteed even in extrapolatory regimes. By using the method of characteristics to predict the analytical properties of PDE solutions a priori (even in regions arbitrarily far from the training domain), we show how it is possible to construct rigorous extrapolatory bounds on the worst-case L^inf errors of shallow neural network approximations. Then, by decomposing PDE solutions into compositions of simpler functions, we show how it is possible to compose these shallow neural networks together to form deep architectures, based on ideas from compositional deep learning, in which the large L^inf errors in the approximations have been suppressed. The resulting framework, called BEACONS (Bounded-Error, Algebraically-COmposable Neural Solvers), comprises both an automatic code-generator for the neural solvers themselves, as well as a bespoke automated
theorem-proving system for producing machine-checkable certificates of correctness. We apply the framework to a variety of linear and non-linear PDEs, including the linear advection and inviscid Burgers' equations, as well as the full compressible Euler equations, in both 1D and 2D, and illustrate how BEACONS architectures are able to extrapolate solutions far beyond the training data in a reliable and bounded way. Various advantages of the approach over the classical PINN approach are discussed.
Read the full abstract ▾
Machine Learning
Atomix: Timely, Transactional Tool Use for Reliable Agentic Workflows
Bardia Mohammadi, Nearchos Potamitis, Lars Klein et al. (+2 more)
LLM
agents execute multi-step workflows that mutate external state through tools. Common orchestrators treat tool return as the settlement trigger, so faults, speculation, and concurrent
agents can leave partial effects, losing-branch residue, stale writes, or irreversible sends. Correct settlement needs two facts that retries, checkpoint replay, locks, and compensation each conflate: which effects must settle together, and when earlier conflicting work is exhausted. Atomix makes this split explicit with progress-aware transactions. The runtime records reads and effects during execution, seals a transaction when its footprint is complete, and commits only after per-resource frontiers show that no earlier conflicting work can still arrive. Commit is final settlement: Atomix releases bufferable effects, accepts reversible external effects as final, and lets irreversible effects leave the gate. Abort suppresses unreleased effects and compensates externalized reversible effects where possible. On representative
agent workloads, this composition improves clean recovery under injected faults, isolates contending and speculative work, and prevents correctly classified irreversible actions from leaking; microbenchmarks show microsecond-scale wrapper overhead relative to tool latency.
Read the full abstract ▾
All authors Bardia Mohammadi, Nearchos Potamitis, Lars Klein, Akhil Arora, Laurent Bindschaedler
Machine Learning · Cross-listed from cs.CV
Multi-dimensional Persistent Sheaf Laplacians for Image Analysis
Xiang Xiang Wang and Guo-Wei Wei
We propose a multi-dimensional persistent sheaf
Laplacian (MPSL) framework on simplicial complexes for image analysis. The proposed method is motivated by the strong sensitivity of commonly used dimensionality reduction techniques, such as principal component analysis (PCA), to the choice of reduced dimension. Rather than selecting a single reduced dimension or averaging results across dimensions, we exploit complementary advantages of multiple reduced dimensions. At a given dimension, image samples are regarded as simplicial complexes, and persistent sheaf
Laplacians are utilized to extract a multiscale localized topological spectral representation for individual image samples. Statistical summaries of the resulting spectra are then aggregated across scales and dimensions to form multiscale multi-dimensional image representations. We evaluate the proposed framework on the COIL20 and ETH80 image
datasets using standard classification protocols. Experimental results show that the proposed method provides more stable performance across a wide range of reduced dimensions and achieves consistent improvements to PCA-based baselines in moderate dimensional regimes.
Read the full abstract ▾
Machine Learning
Interactionless Inverse Reinforcement Learning: A Data-Centric Framework for Durable Alignment
Elias Malomgré and Pieter Simoens
AI
alignment is growing in importance, yet many current approaches learn safety behavior by directly modifying policy parameters, entangling normative constraints with the underlying policy. This often yields opaque, difficult-to-edit
alignment artifacts and reduces their reuse across models or deployments, a failure mode we term
Alignment Waste. We propose Interactionless Inverse Reinforcement Learning, a framework for learning inspectable, editable, and reusable reward artifacts separately from policy optimization. We further introduce the
Alignment Flywheel, a human-in-the-loop lifecycle for iteratively
auditing, patching, and hardening these artifacts through automated evaluation and refinement. Together, these ideas recast
alignment from a disposable training expense into a durable, verifiable engineering asset.
Read the full abstract ▾
Machine Learning · Cross-listed from eess.SP
RF-GPT: Teaching AI to See the Wireless World
Hang Zou, Yu Tian, Bohao Wang et al. (+4 more)
Large
language models (LLMs) and
multimodal models have become powerful general-purpose
reasoning systems. However,
radio-frequency (RF) signals, which underpin wireless systems, are still not natively supported by these models. Existing LLM-based approaches for telecom focus mainly on text and structured data, while conventional RF deep-learning models are built separately for specific signal-processing tasks, highlighting a clear gap between RF perception and high-level
reasoning. To bridge this gap, we introduce RF-GPT, a
radio-frequency
language model (RFLM) that utilizes the visual encoders of
multimodal LLMs to process and understand RF spectrograms. In this framework, complex in-phase/quadrature (IQ) waveforms are mapped to time-frequency spectrograms and then passed to pretrained visual encoders. The resulting representations are injected as RF
tokens into a decoder-only LLM, which generates RF-grounded answers, explanations, and structured outputs. To train RF-GPT, we perform supervised instruction fine-tuning of a pretrained
multimodal LLM using a fully synthetic RF corpus. Standards-compliant waveform generators produce wideband scenes for six wireless technologies, from which we derive time-frequency spectrograms, exact configuration metadata, and dense captions. A text-only LLM then converts these captions into RF-grounded instruction-answer pairs, yielding roughly 120,000 RF scenes and 0.625 million instruction examples without any manual labeling. Across
benchmarks for wideband modulation classification, overlap analysis, wireless-technology recognition, WLAN user counting, and 5G NR information extraction, RF-GPT achieves strong multi-task performance, whereas general-purpose VLMs with no RF grounding largely fail.
Read the full abstract ▾
All authors Hang Zou, Yu Tian, Bohao Wang, Lina Bariah, Samson Lasaulce, Chongwen Huang, Mérouane Debbah
Machine Learning · Cross-listed from q-bio.QM
Exploring the limits of pre-trained embeddings in machine-guided protein design: a case study on predicting AAV vector viability
Ana F. Rodrigues, Lucas Ferraz, Laura Balbi et al. (+2 more)
Effective representations of
protein sequences are widely recognized as a cornerstone of machine learning-based
protein design. Yet,
protein bioengineering poses unique challenges for sequence representation, as experimental
datasets typically feature few mutations, which are either sparsely distributed across the entire sequence or densely concentrated within localized regions. This limits the ability of sequence-level representations to extract functionally meaningful signals. In addition, comprehensive comparative studies remain scarce, despite their crucial role in clarifying which representations best encode relevant information and ultimately support superior predictive performance. In this study, we systematically evaluate multiple ProtBERT and ESM2
embedding variants as sequence representations, using the adeno-associated virus capsid as a case study and prototypical example of bioengineering, where functional optimization is targeted through highly localized sequence variation within an otherwise large
protein. Our results reveal that, prior to fine-tuning, amino acid-level
embeddings outperform sequence-level representations in supervised predictive tasks, whereas the latter tend to be more effective in unsupervised settings. However, optimal performance is only achieved when
embeddings are fine-tuned with task-specific labels, with sequence-level representations providing the best performance. Moreover, our findings indicate that the extent of sequence variation required to produce notable shifts in sequence representations exceeds what is typically explored in bioengineering studies, showing the need for fine-tuning in
datasets characterized by sparse or highly localized mutations.
Read the full abstract ▾
All authors Ana F. Rodrigues, Lucas Ferraz, Laura Balbi, Pedro Giesteira Cotovio, Catia Pesquita
Machine Learning
Learning State-Tracking from Code Using Linear RNNs
Julien Siems, Riccardo Grazzi, Korbinian Pöppel et al. (+3 more)
Over the last years, state-tracking tasks, particularly permutation composition, have become a testbed to understand the limits of sequence models architectures like
Transformers and RNNs (linear and non-linear). However, these are often sequence-to-sequence tasks: learning to map actions (permutations) to states, which is incompatible with the next-
token prediction setting commonly used to train
language models. We address this gap by converting permutation composition into code via REPL traces that interleave state-reveals through prints and variable transformations. We show that linear RNNs capable of state-tracking excel also in this setting, while
Transformers still fail. Motivated by this representation, we investigate why tracking states in code is generally difficult: actions are not always fully observable. We frame this as tracking the state of a probabilistic finite-state automaton with deterministic state reveals and show that linear RNNs can be worse than non-linear RNNs at tracking states in this setup.
Read the full abstract ▾
All authors Julien Siems, Riccardo Grazzi, Korbinian Pöppel, Kirill Kalinin, Hitesh Ballani, Babak Rahmani
Machine Learning · Cross-listed from cs.AI
JediKG: Return of the Schema
Ivan Diliso, Roberto Barile, Nicola Fanizzi et al. (+1 more)
Benchmark datasets for Knowledge Graph Refinement typically contain only ground facts, discarding the schema available in the source knowledge graph. This limits the evaluation of Neural-Symbolic methods that leverage expressive schemas. We present JEDIKG the first resource that systematically addresses these gaps through two components: JDEX a general configurable tool for extracting complete, portable, modular
dataset from any RDF/OWL knowledge graph; and JDSET, a curated suite of eleven
datasets with varying expressivity, volume, and domain. JDEX preserves full schema expressivity, employs standard OWL
reasoning to detect and correct inconsistencies and materialize implicit knowledge, applies schema modularization, and provides utilities for integration with mainstream machine learning libraries such as PyTorch and PyKEEN. We validated the completeness and portability of the
datasets across several dimensions and preliminarily proved the utility of JEDIKG in experimental workflows combining data-driven methods with schema-driven
reasoning.
Read the full abstract ▾
All authors Ivan Diliso, Roberto Barile, Nicola Fanizzi, Claudia d'Amato
Machine Learning
Extending Multi-Source Bayesian Optimization With Causality Principles
Luuk Jacobs and Mohammad Ali Javidian
Multi-Source
Bayesian Optimization (MSBO) serves as a variant of the traditional
Bayesian Optimization (BO) framework applicable to situations involving optimization of an objective black-box function over multiple information sources such as simulations, surrogate models, or real-world experiments. However, traditional MSBO assumes the input variables of the objective function to be independent and identically distributed, limiting its effectiveness in scenarios where causal information is available and interventions can be performed, such as clinical trials or policy-making. In the single-source domain, Causal
Bayesian Optimization (CBO) extends standard BO with the principles of causality, enabling better modeling of variable dependencies. This leads to more accurate optimization, improved decision-making, and more efficient use of low-cost information sources. In this article, we propose a principled integration of the MSBO and CBO methodologies in the multi-source domain, leveraging the strengths of both to enhance optimization efficiency and reduce computational complexity in higher-dimensional problems. We present the theoretical foundations of both Causal and Multi-Source
Bayesian Optimization, and demonstrate how their synergy informs our Multi-Source Causal
Bayesian Optimization (MSCBO)
algorithm. We compare the performance of MSCBO against its foundational counterparts for both synthetic and real-world
datasets with varying levels of noise, highlighting the robustness and applicability of MSCBO. Based on our findings, we conclude that integrating MSBO with the causality principles of CBO facilitates dimensionality reduction and lowers operational costs, ultimately improving convergence speed, performance, and scalability.
Read the full abstract ▾
Machine Learning
On the Stability of Nonlinear Dynamics in GD and SGD: Beyond Quadratic Potentials
Rotem Mulayoff and Sebastian U. Stich
The dynamical stability of the iterates during training plays a key role in determining the minima obtained by optimization
algorithms. For example, stable solutions of gradient descent (GD) correspond to flat minima, which have been associated with favorable features. While prior work often relies on linearization to determine stability, it remains unclear whether linearized dynamics faithfully capture the full nonlinear behavior. Recent work has shown that GD may stably oscillate near a linearly unstable minimum and still converge once the step size decays, indicating that linear analysis can be misleading. In this work, we explicitly study the effect of nonlinear terms. Specifically, we derive an exact criterion for stable oscillations of GD near minima in the multivariate setting. Our condition depends on high-order derivatives, generalizing existing results. Extending the analysis to
stochastic gradient descent (SGD), we show that nonlinear dynamics can diverge in expectation even if a single batch is unstable. This implies that stability can be dictated by a single batch that oscillates unstably, rather than an average effect, as linear analysis suggests. Finally, we prove that if all batches are linearly stable, the nonlinear dynamics of SGD are stable in expectation.
Read the full abstract ▾
Machine Learning · Cross-listed from eess.AS
SA-SSL-MOS: Self-supervised Learning MOS Prediction with Spectral Augmentation for Generalized Multi-Rate Speech Assessment
Fengyuan Cao, Xinyu Liang, Fredrik Cumlin et al. (+4 more)
Designing a
speech quality assessment (SQA) system for estimating mean-opinion-score (MOS) of multi-rate
speech with varying sampling frequency (16-48 kHz) is a challenging task. The challenge arises due to the limited availability of a MOS-labeled training
dataset comprising multi-rate
speech samples. While self-supervised learning (SSL) models have been widely adopted in SQA to boost performance, a key limitation is that they are pretrained on 16 kHz
speech and therefore discard high-frequency information present in higher sampling rates. To address this issue, we propose a spectrogram-augmented SSL method that incorporates high-frequency features (up to 48 kHz sampling rate) through a parallel-branch architecture. We further introduce a two-step training scheme: the model is first pre-trained on a large 48 kHz
dataset and then fine-tuned on a smaller multi-rate
dataset. Experimental results show that leveraging high-frequency information overlooked by SSL features is crucial for accurate multi-rate SQA, and that the proposed two-step training substantially improves generalization when multi-rate data is limited.
Read the full abstract ▾
All authors Fengyuan Cao, Xinyu Liang, Fredrik Cumlin, Victor Ungureanu, Chandan K. A. Reddy, Christian Schuldt, Saikat Chatterjee
Machine Learning · Cross-listed from cs.IR
Intent-Driven Dynamic Chunking: Segmenting Documents to Reflect Predicted Information Needs
Christos Koutsiaris
Breaking long documents into smaller segments is a fundamental challenge in information
retrieval. Whether for search engines, question-answering systems, or
retrieval-augmented generation (RAG), effective
segmentation determines how well systems can locate and return relevant information. However, traditional methods, such as fixed-length or coherence-based
segmentation, ignore user intent, leading to chunks that split answers or contain irrelevant noise. We introduce Intent-Driven Dynamic Chunking (IDC), a novel approach that uses predicted user queries to guide document
segmentation. IDC leverages a Large
Language Model to generate likely user intents for a document and then employs a dynamic programming
algorithm to find the globally optimal chunk boundaries. This represents a novel application of DP to intent-aware
segmentation that avoids greedy pitfalls. We evaluated IDC on six diverse question-answering
datasets, including news articles, Wikipedia, academic papers, and technical documentation. IDC outperformed traditional chunking strategies on five
datasets, improving top-1
retrieval accuracy by 5% to 67%, and matched the best baseline on the sixth. Additionally, IDC produced 40-60% fewer chunks than baseline methods while achieving 93-100% answer coverage. These results demonstrate that aligning document structure with anticipated information needs significantly boosts
retrieval performance, particularly for long and heterogeneous documents.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Emergently Misaligned Language Models Show Behavioral Self-Awareness That Shifts With Subsequent Realignment
Laurène Vaugrante, Anietta Weckauff and Thilo Hagendorff
Recent research has demonstrated that large
language models (LLMs) fine-tuned on incorrect trivia question-answer pairs exhibit toxicity - a phenomenon later termed "emergent misalignment". Moreover, research has shown that LLMs possess behavioral self-awareness - the ability to describe learned behaviors that were only implicitly demonstrated in training data. Here, we investigate the intersection of these phenomena. We fine-tune GPT-4.1 models sequentially on
datasets known to induce and reverse emergent misalignment and evaluate whether the models are self-aware of their behavior transitions without providing in-context examples. Our results show that emergently misaligned models rate themselves as significantly more harmful compared to their base model and realigned counterparts, demonstrating behavioral self-awareness of their own emergent misalignment. Our findings show that behavioral self-awareness tracks actual
alignment states of models, indicating that models can be queried for informative signals about their own safety.
Read the full abstract ▾
Machine Learning
Learning Structural Hardness for Combinatorial Auctions: Instance-Dependent Algorithm Selection via Graph Neural Networks
Sungwoo Kang
The Winner Determination Problem (WDP) in combinatorial auctions is NP-hard, and no existing method reliably predicts which instances will defeat fast greedy heuristics. The ML-for-combinatorial-optimization community has focused on learning to replace solvers, yet recent evidence shows that graph neural networks (GNNs) rarely outperform well-tuned classical methods on standard
benchmarks. We pursue a different objective: learning to predict when a given instance is hard for greedy allocation, enabling instance-dependent
algorithm selection. We design a 20-dimensional structural feature vector and train a lightweight MLP hardness classifier that predicts the greedy optimality gap with mean absolute error 0.033, Pearson correlation 0.937, and binary classification accuracy 94.7% across three random seeds. For instances identified as hard – those exhibiting “whale-fish” trap structure where greedy provably fails – we deploy a heterogeneous GNN specialist that achieves ${\approx}0\%$ optimality gap on all six
adversarial configurations tested (vs.\ 3.75–59.24% for greedy). A hybrid allocator combining the hardness classifier with GNN and greedy solvers achieves 0.51% overall gap on mixed distributions. Our honest evaluation on CATS
benchmarks confirms that GNNs do not outperform Gurobi (0.45–0.71 vs.\ 0.20 gap), motivating the
algorithm selection framing. Learning when to deploy expensive solvers is more tractable than learning to replace them.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
GOT-JEPA: Generic Object Tracking with Model Adaptation and Occlusion Handling using Joint-Embedding Predictive Architecture
Shih-Fang Chen, Jun-Cheng Chen, I-Hong Jhuo et al. (+1 more)
The human visual system tracks objects by integrating current observations with previously observed information, adapting to target and scene changes, and
reasoning about occlusion at fine granularity. In contrast, recent generic object trackers are often optimized for training targets, which limits robustness and generalization in unseen scenarios, and their occlusion
reasoning remains coarse, lacking detailed modeling of occlusion patterns. To address these limitations in generalization and occlusion perception, we propose GOT-JEPA, a model-predictive pretraining framework that extends JEPA from predicting image features to predicting tracking models. Given identical historical information, a teacher predictor generates pseudo-tracking models from a clean current frame, and a student predictor learns to predict the same pseudo-tracking models from a corrupted version of the current frame. This design provides stable pseudo supervision and explicitly trains the predictor to produce reliable tracking models under occlusions, distractors, and other adverse observations, improving generalization to dynamic environments. Building on GOT-JEPA, we further propose OccuSolver to enhance occlusion perception for object tracking. OccuSolver adapts a point-centric point tracker for object-aware visibility estimation and detailed occlusion-pattern capture. Conditioned on object priors iteratively generated by the tracker, OccuSolver incrementally refines visibility states, strengthens occlusion handling, and produces higher-quality reference labels that progressively improve subsequent model predictions. Extensive evaluations on seven
benchmarks show that our method effectively enhances tracker generalization and robustness.
Read the full abstract ▾
All authors Shih-Fang Chen, Jun-Cheng Chen, I-Hong Jhuo, Yen-Yu Lin
Machine Learning
Universal Algorithm-Implicit Learning
Stefano Woerner, Seong Joon Oh and Christian F. Baumgartner
Current meta-learning methods are constrained to narrow task distributions with fixed feature and label spaces, limiting applicability. Moreover, the current meta-learning literature uses key terms like "universal" and "general-purpose" inconsistently and lacks precise definitions, hindering comparability. We introduce a theoretical framework for meta-learning which formally defines practical universality and introduces a distinction between
algorithm-explicit and
algorithm-implicit learning, providing a principled vocabulary for
reasoning about universal meta-learning methods. Guided by this framework, we present TAIL, a
transformer-based
algorithm-implicit meta-learner that functions across tasks with varying domains, modalities, and label configurations. TAIL features three innovations over prior
transformer-based meta-learners: random projections for cross-modal feature encoding, random injection label
embeddings that extrapolate to larger label spaces, and efficient inline query processing. TAIL achieves state-of-the-art performance on standard few-shot
benchmarks while generalizing to unseen domains. Unlike other meta-learning methods, it also generalizes to unseen modalities, solving text classification tasks despite training exclusively on images, handles tasks with up to 20$\times$ more classes than seen during training, and provides orders-of-magnitude computational savings over prior
transformer-based approaches.
Read the full abstract ▾