Machine Learning · Cross-listed from cs.SD
FastWave: Optimized Diffusion Model for Audio Super-Resolution
Nikita Kuznetsov and Maksim Kaledin
Audio Super-Resolution is a set of techniques aimed at high-quality estimation of the given signal as if it would be sampled with higher sample rate. Among suggested methods there are
diffusion and flow models (which are considered slower), generative
adversarial networks (which are considered faster), however both approaches are currently presented by high-parametric networks, requiring high computational costs both for training and inference. We propose a solution to both these problems by re-considering the recent advances in the training of
diffusion models and applying them to super-resolution from any to 48 kHz sample rate. Our approach shows better results than NU-Wave 2 and is comparable to state-of-the-art models. Our model called FastWave has around 50 GFLOPs of computational complexity and 1.3 M parameters and can be trained with less resources and significantly faster than the majority of recently proposed
diffusion- and flow-based solutions. The code has been made publicly available.
Read the full abstract ▾
Machine Learning
When to restart? Exploring escalating restarts on convergence
Ayush K. Varshney, Šarūnas Girdzijauskas, Konstantinos Vandikas et al. (+1 more)
Learning rate scheduling plays a critical role in the optimization of deep neural networks, directly influencing convergence speed, stability, and generalization. While existing schedulers such as cosine annealing, cyclical learning rates, and warm restarts have shown promise, they often rely on fixed or periodic triggers that are agnostic to the training dynamics, such as stagnation or convergence behavior. In this work, we propose a simple yet effective strategy, which we call
Stochastic Gradient Descent with Escalating Restarts (SGD-ER). It adaptively increases the learning rate upon convergence. Our method monitors training progress and triggers restarts when stagnation is detected, linearly escalating the learning rate to escape sharp local minima and explore flatter regions of the loss landscape. We evaluate SGD-ER across CIFAR-10, CIFAR-100, and TinyImageNet on a range of architectures including ResNet-18/34/50, VGG-16, and DenseNet-101. Compared to standard schedulers, SGD-ER improves test accuracy by 0.5-4.5%, demonstrating the benefit of convergence-aware escalating restarts for better local optima.
Read the full abstract ▾
All authors Ayush K. Varshney, Šarūnas Girdzijauskas, Konstantinos Vandikas, Aneta Vulgarakis Feljan
Machine Learning
Reducing hyperparameter sensitivity in measurement-feedback based Ising machines
Toon Sevenants, Guy Van der Sande and Guy Verschaffelt
Analog Ising machines have been proposed as heuristic hardware solvers for combinatorial optimization problems, with the potential to outperform conventional approaches, provided that their hyperparameters are carefully tuned. Their temporal
evolution is often described using time-continuous dynamics. However, most experimental implementations rely on measurement-feedback architectures that operate in a time-discrete manner. We observe that in such setups, the range of effective hyperparameters is substantially smaller than in the envisioned time-continuous analog Ising machine. In this paper, we analyze this discrepancy and discuss its impact on the practical operation of Ising machines. Next, we propose and experimentally verify a method to reduce the sensitivity to hyperparameter selection of these measurement-feedback architectures.
Read the full abstract ▾
Machine Learning
Tuning Just Enough: Lightweight Backdoor Attacks on Multi-Encoder Diffusion Models
Ziyuan Chen, Yujin Jeong, Tobias Braun et al. (+1 more)
As text-to-image
diffusion models become increasingly deployed in real-world applications, concerns about
backdoor attacks have gained significant
attention. Prior work on text-based
backdoor attacks has largely focused on
diffusion models conditioned on a single lightweight text encoder. However, more recent
diffusion models that incorporate multiple large-scale text encoders remain underexplored in this context. Given the substantially increased number of trainable parameters introduced by multiple text encoders, an important question is whether
backdoor attacks can remain both efficient and effective in such settings. In this work, we study Stable
Diffusion 3, which uses three distinct text encoders and has not yet been systematically analyzed for text-encoder-based
backdoor vulnerabilities. To understand the role of text encoders in
backdoor attacks, we define four categories of attack targets and identify the minimal sets of encoders required to achieve effective performance for each attack objective. Based on this, we further propose Multi-Encoder Lightweight aTtacks (MELT), which trains only low-rank adapters while keeping the pretrained text encoder weight frozen. We demonstrate that tuning fewer than 0.2% of the total encoder parameters is sufficient for successful
backdoor attacks on Stable
Diffusion 3, revealing previously underexplored vulnerabilities in practical attack scenarios in multi-encoder settings.
Read the full abstract ▾
All authors Ziyuan Chen, Yujin Jeong, Tobias Braun, Anna Rohrbach
Machine Learning
FedCova: Robust Federated Covariance Learning Against Noisy Labels
Xiangyu Zhong, Xiaojun Yuan and Ying-Jun Angela Zhang
Noisy labels in distributed
datasets induce severe local overfitting and consequently compromise the global model in
federated learning (FL). Most existing solutions rely on selecting clean devices or aligning with public clean
datasets, rather than endowing the model itself with robustness. In this paper, we propose FedCova, a dependency-free
federated covariance learning framework that eliminates such external reliances by enhancing the model's intrinsic robustness via a new perspective on feature covariances. Specifically, FedCova encodes data into a discriminative but resilient feature space to tolerate label noise. Built on mutual information maximization, we design a novel objective for
federated lossy feature encoding that relies solely on class feature covariances with an error tolerance term. Leveraging feature subspaces characterized by covariances, we construct a subspace-augmented
federated classifier. FedCova unifies three key processes through the covariance: (1) training the network for feature encoding, (2) constructing a classifier directly from the learned features, and (3) correcting noisy labels based on feature subspaces. We implement FedCova across both symmetric and asymmetric noisy settings under heterogeneous data distribution. Experimental results on CIFAR-10/100 and real-world noisy
dataset Clothing1M demonstrate the superior robustness of FedCova compared with the state-of-the-art methods.
Read the full abstract ▾
Machine Learning · Cross-listed from quant-ph
Fermi-Dirac thermal measurements: A framework for quantum hypothesis testing and semidefinite optimization
Nana Liu and Mark M. Wilde
Quantum measurements are the means by which we recover messages encoded into
quantum states. They are at the forefront of
quantum hypothesis testing, wherein the goal is to perform an optimal measurement for arriving at a correct conclusion. Mathematically, a measurement operator is Hermitian with
eigenvalues in [0,1]. By noticing that this constraint on each
eigenvalue is the same as that imposed on
fermions by the Pauli exclusion principle, we interpret every eigenmode of a measurement operator as an independent effective fermionic mode. Under this perspective, various objective functions in
quantum hypothesis testing can be viewed as the total expected energy associated with these fermionic occupation numbers. By instead fixing a temperature and minimizing the total expected fermionic free energy, we find that optimal measurements for these modified objective functions are
Fermi-Dirac thermal measurements, wherein their
eigenvalues are specified by
Fermi-Dirac distributions. In the low-temperature limit, their performance closely approximates that of optimal measurements for
quantum hypothesis testing, and we show that their parameters can be learned by classical or hybrid
quantum-classical optimization
algorithms. This leads to a new
quantum machine-learning model, termed
Fermi-Dirac machines, consisting of parameterized
Fermi-Dirac thermal measurements-an alternative to
quantum Boltzmann machines based on thermal states. Beyond hypothesis testing, we show how general semidefinite optimization problems can be solved using this approach, leading to a novel paradigm for semidefinite optimization on
quantum computers, in which the goal is to implement thermal measurements rather than prepare thermal states. Finally, we propose
quantum algorithms for implementing
Fermi-Dirac thermal measurements, and we also propose second-order hybrid
quantum-classical optimization
algorithms.
Read the full abstract ▾
Machine Learning
Inference-Time Toxicity Mitigation in Protein Language Models
Manuel Fernández Burda, Santiago Aranguri, Iván Arcuschin Moreno et al. (+1 more)
Protein language models (PLMs) are becoming practical tools for de novo
protein design, yet their dual-use potential raises safety concerns. We show that domain adaptation to specific taxonomic groups can elicit toxic
protein generation, even when toxicity is not the training objective. To address this, we adapt Logit Diff Amplification (LDA) as an inference-time control mechanism for PLMs. LDA modifies
token probabilities by amplifying the logit difference between a baseline model and a toxicity-finetuned model, requiring no retraining. Across four taxonomic groups, LDA consistently reduces predicted toxicity rate (measured via ToxDL2) below the taxon-finetuned baseline while preserving biological plausibility. We evaluate quality using Fréchet ESM Distance and predicted foldability (pLDDT), finding that LDA maintains distributional similarity to natural
proteins and structural viability (unlike activation-based steering methods that tend to degrade sequence properties). Our results demonstrate that LDA provides a practical safety knob for
protein generators that mitigates elicited toxicity while retaining generative quality.
Read the full abstract ▾
All authors Manuel Fernández Burda, Santiago Aranguri, Iván Arcuschin Moreno, Enzo Ferrante
Machine Learning · Cross-listed from cs.CV
DQE-CIR: Distinctive Query Embeddings through Learnable Attribute Weights and Target Relative Negative Sampling in Composed Image Retrieval
Geon Park, Ji-Hoon Park and Seong-Whan Lee
Composed image
retrieval (CIR) addresses the task of retrieving a target image by jointly interpreting a reference image and a modification text that specifies the intended change. Most existing methods are still built upon contrastive learning frameworks that treat the ground truth image as the only positive instance and all remaining images as negatives. This strategy inevitably introduces relevance suppression, where semantically related yet valid images are incorrectly pushed away, and semantic confusion, where different modification intents collapse into overlapping regions of the
embedding space. As a result, the learned query representations often lack discriminativeness, particularly at fine-grained attribute modifications. To overcome these limitations, we propose distinctive query
embeddings through learnable attribute weights and target relative negative sampling (DQE-CIR), a method designed to learn distinctive query
embeddings by explicitly modeling target relative relevance during training. DQE-CIR incorporates learnable attribute weighting to emphasize distinctive visual features conditioned on the modification text, enabling more precise feature
alignment between
language and
vision. Furthermore, we introduce target relative negative sampling, which constructs a target relative similarity distribution and selects informative negatives from a mid-zone region that excludes both easy negatives and ambiguous false negatives. This strategy enables more reliable
retrieval for fine-grained attribute changes by improving query discriminativeness and reducing confusion caused by semantically similar but irrelevant candidates.
Read the full abstract ▾
Machine Learning
mlx-vis: GPU-Native Dimensionality Reduction on Apple Silicon
Han Xiao
Dimensionality reduction is a foundational tool for visualizing high-dimensional data, yet its reference implementations span a fragmented stack of CPU-bound Python packages that leaves the Metal GPU on Apple Silicon entirely unused. We present mlx-vis, a library that reimplements seven widely used dimensionality reduction methods and k-nearest neighbor graph construction in pure MLX, with every stage – from PCA preprocessing through
embedding optimization to a circle-splatting renderer – executing on GPU. On Fashion-MNIST 70K, all seven methods embed in 2.1–4.6 s on an M3 Ultra, achieving 3–13x speedups over CPU baselines while reducing the entire dependency stack to MLX and NumPy. The same pipeline scales to ten million points on a single workstation. Code at https://github.com/hanxiao/mlx-vis
Read the full abstract ▾
Machine Learning · Cross-listed from cs.SD
Multi-Stage Music Source Restoration with BandSplit-RoFormer Separation and HiFi++ GAN
Tobias Morocutti, Emmanouil Karystinaios, Jonathan Greif et al. (+1 more)
Music Source Restoration (MSR) targets recovery of original, unprocessed instrument stems from fully mixed and mastered audio, where production effects and distribution artifacts violate common linear-mixture assumptions. This technical report presents the CP-JKU team's system for the MSR ICASSP Challenge 2025. Our approach decomposes MSR into separation and restoration. First, a single BandSplit-RoFormer separator predicts eight stems plus an auxiliary other stem, and is trained with a three-stage curriculum that progresses from 4-stem warm-start fine-tuning (with LoRA) to 8-stem extension via head expansion. Second, we apply a HiFi++ GAN waveform restorer trained as a generalist and then specialized into eight instrument-specific experts.
Read the full abstract ▾
All authors Tobias Morocutti, Emmanouil Karystinaios, Jonathan Greif, Gerhard Widmer
Machine Learning
A Multi-Dimensional Quality Scoring Framework for Decentralized LLM Inference with Proof of Quality
Arther Tian, Alex Ding, Frank Chen et al. (+2 more)
Decentralized large
language model (LLM) inference networks can pool heterogeneous compute to scale serving, but they require lightweight and incentive-compatible mechanisms to assess output quality. Prior work introduced cost-aware Proof of Quality (PoQ) and adaptive robust PoQ to allocate rewards under evaluator heterogeneity and
adversarial behavior. In this paper, we focus on the quality signal itself and propose a multi-dimensional quality scoring framework that decomposes output quality into modular dimensions, including model and cost priors, structure quality, semantic quality, query-output
alignment, and agreement/uncertainty. Using logged outputs from QA and summarization tasks, we systematically audit dimension reliability and show that seemingly reasonable dimensions can be task-dependent and even negatively correlated with reference quality without calibration. While the default composite underperforms a strong single semantic evaluator, ablations reveal that removing unreliable dimensions and re-normalizing weights yields a calibrated composite that matches or exceeds the best single- evaluator and consensus baselines. Finally, we integrate the composite score as a drop-in quality signal in PoQ and demonstrate complementary benefits with robust aggregation and adaptive trust weighting under
adversarial evaluator attacks.
Read the full abstract ▾
All authors Arther Tian, Alex Ding, Frank Chen, Simon Wu, Aaron Chan
Machine Learning · Cross-listed from cs.LO
Continuous Modal Logical Neural Networks: Modal Reasoning via Stochastic Accessibility
Antonin Sulc
We propose Fluid Logic, a paradigm in which modal logical
reasoning, temporal, epistemic, doxastic, deontic, is lifted from discrete Kripke structures to continuous
manifolds via Neural
Stochastic Differential Equations (Neural SDEs). Each type of modal operator is backed by a dedicated Neural SDE, and nested formulas compose these SDEs in a single differentiable graph. A key instantiation is Logic-Informed Neural Networks (LINNs): analogous to Physics-Informed Neural Networks (PINNs), LINNs embed modal logical formulas such as ($\Box$ bounded) and ($\Diamond$ visits\_lobe) directly into the training loss, guiding neural networks to produce solutions that are structurally consistent with prescribed logical properties, without requiring knowledge of the governing equations. The resulting framework, Continuous Modal Logical Neural Networks (CMLNNs), yields several key properties: (i)
stochastic diffusion prevents quantifier collapse ($\Box$ and $\Diamond$ differ), unlike deterministic ODEs; (ii) modal operators are entropic risk measures, sound with respect to risk-based semantics with explicit Monte Carlo concentration guarantees; (iii)SDE-induced accessibility provides structural correspondence with classical modal axioms; (iv) parameterizing accessibility through dynamics reduces
memory from quadratic in world count to linear in parameters. Three case studies demonstrate that Fluid Logic and LINNs can guide neural networks to produce consistent solutions across diverse domains: epistemic/doxastic logic (multi-
robot hallucination detection), temporal logic (recovering the Lorenz attractor geometry from logical constraints alone), and deontic logic (learning safe confinement dynamics from a logical specification).
Read the full abstract ▾
Machine Learning
Fixed-Budget Constrained Best Arm Identification in Grouped Bandits
Raunak Mukherjee and Sharayu Moharir
We study fixed budget constrained best-arm identification in grouped bandits, where each arm consists of multiple independent attributes with
stochastic rewards. An arm is considered feasible only if all its attributes' means are above a given threshold. The aim is to find the feasible arm with the largest overall mean. We first derive a lower bound on the error probability for any
algorithm on this setting. We then propose Feasibility Constrained Successive Rejects (FCSR), a novel
algorithm that identifies the best arm while ensuring feasibility. We show it attains optimal dependence on problem parameters up to constant factors in the exponent. Empirically, FCSR outperforms natural baselines while preserving feasibility guarantees.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.IT
Training-Free Rate-Distortion-Perception Traversal With Diffusion
Yuhan Wang, Suzhi Bi and Ying-Jun Angela Zhang
The rate-distortion-perception (RDP) tradeoff characterizes the fundamental limits of lossy compression by jointly considering bitrate, reconstruction fidelity, and perceptual quality. While recent neural compression methods have improved perceptual performance, they typically operate at fixed points on the RDP surface, requiring retraining to target different tradeoffs. In this work, we propose a training-free framework that leverages pre-trained
diffusion models to traverse the entire RDP surface. Our approach integrates a reverse channel coding (RCC) module with a novel score-scaled probability flow ODE decoder. We theoretically prove that the proposed
diffusion decoder is optimal for the distortion-perception tradeoff under AWGN observations and that the overall framework with the RCC module achieves the optimal RDP function in the Gaussian case. Empirical results across multiple
datasets demonstrate the framework's flexibility and effectiveness in navigating the ternary RDP tradeoff using pre-trained
diffusion models. Our results establish a practical and theoretically grounded approach to adaptive, perception-aware compression.
Read the full abstract ▾
Machine Learning
On the Learnability of Offline Model-Based Optimization: A Ranking Perspective
Shen-Huan Lyu, Rong-Xi Tan, Ke Xue et al. (+4 more)
Offline model-based optimization (MBO) seeks to discover high-performing designs using only a fixed
dataset of past evaluations. Most existing methods rely on learning a surrogate model via regression and implicitly assume that good predictive accuracy leads to good optimization performance. In this work, we challenge this assumption and study offline MBO from a learnability perspective. We argue that offline optimization is fundamentally a problem of ranking high-quality designs rather than accurate value prediction. Specifically, we introduce an optimization-oriented risk based on ranking between near-optimal and suboptimal designs, and develop a unified theoretical framework that connects surrogate learning to final optimization. We prove the theoretical advantages of ranking over regression, and identify distributional mismatch between the training data and near-optimal designs as the dominant error. Inspired by this, we design a distribution-aware ranking method to reduce this mismatch. Empirical results across various tasks show that our approach outperforms twenty existing methods, validating our theoretical findings. Additionally, both theoretical and empirical results reveal intrinsic limitations in offline MBO, showing a regime in which no offline method can avoid over-optimistic extrapolation.
Read the full abstract ▾
All authors Shen-Huan Lyu, Rong-Xi Tan, Ke Xue, Yi-Xiao He, Yu Huang, Qingfu Zhang, Chao Qian
Machine Learning
Spectral Surgery: Training-Free Refinement of LoRA via Gradient-Guided Singular Value Reweighting
Zailong Tian, Yanzhe Chen, Zhuoheng Han et al. (+1 more)
Low-Rank Adaptation (LoRA) improves downstream performance by restricting task updates to a low-rank parameter subspace, yet how this limited capacity is allocated within a trained adapter remains unclear. Through a
geometric and empirical study across multiple tasks and backbones, we find that trained LoRA updates often exhibit an inefficient spectrum: task effects concentrate in a small subset of singular directions, while many remaining components are neutral or detrimental, motivating post-hoc refinement within the learned subspace. We propose Spectral Surgery, a training-free refinement that decomposes a LoRA update with SVD, estimates per-component sensitivity using gradients on a small calibration set, and reweights singular values under a magnitude constraint while keeping the learned directions fixed. Across Llama-3.1-8B and Qwen3-8B on four
benchmarks, Spectral Surgery yields consistent gains (up to +4.4 points on CommonsenseQA and +2.4 pass@1 on HumanEval) by adjusting only $\approx 1{,}000$ scalar coefficients. These results demonstrate that SVD-structured, low-cost parameter editing can serve as a practical route to improving trained LoRA adapters in a purely post-hoc manner.
Read the full abstract ▾
All authors Zailong Tian, Yanzhe Chen, Zhuoheng Han, Lizi Liao
Machine Learning
Specialization of softmax attention heads: insights from the high-dimensional single-location model
M. Sagitova, O. Duranthon and L. Zdeborová
Multi-head
attention enables
transformer models to represent multiple
attention patterns simultaneously. Empirically, head specialization emerges in distinct stages during training, while many heads remain redundant and learn similar representations. We propose a theoretical model capturing this phenomenon, based on the multi-index and single-location regression frameworks. In the first part, we analyze the training dynamics of multi-head softmax
attention under SGD, revealing an initial unspecialized phase followed by a multi-stage specialization phase in which different heads sequentially align with latent signal directions. In the second part, we study the impact of
attention activation functions on performance. We introduce the Bayes-softmax
attention, which achieves optimal prediction performance in this setting.
Read the full abstract ▾
Machine Learning
Dual-Solver: A Generalized ODE Solver for Diffusion Models with Dual Prediction
Soochul Park and Yeon Ju Lee
Diffusion models achieve state-of-the-art image quality. However, sampling is costly at inference time because it requires a large number of function evaluations (NFEs). To reduce NFEs, classical ODE numerical methods have been adopted. Yet, the choice of prediction type and integration domain leads to different sampling behaviors. To address these issues, we introduce Dual-Solver, which generalizes multistep samplers through learnable parameters that continuously (i) interpolate among prediction types, (ii) select the integration domain, and (iii) adjust the residual terms. It retains the standard predictor-corrector structure while preserving second-order local accuracy. These parameters are learned via a classification-based objective using a frozen pretrained classifier (e.g., MobileNet or CLIP). For ImageNet class-conditional generation (DiT, GM-DiT) and text-to-image generation (SANA, PixArt-$\alpha$), Dual-Solver improves FID and CLIP scores in the low-NFE regime ($3 \le$ NFE $\le 9$) across backbones.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CY
No Certificate, No Categorical Speech Act: A Brouwerian Assertibility Constraint for Public Reason
Michael Jülich
Generative AI can convert uncertainty into authoritative-seeming verdicts, displacing the justificatory work on which democratic epistemic agency depends. As a corrective, I propose a Brouwer-inspired assertibility constraint for responsible AI: in high-stakes domains, systems may assert or deny claims only if they can provide a publicly inspectable and contestable certificate of entitlement; otherwise they must return Undetermined. This constraint yields a three-status interface semantics (Asserted, Denied, Undetermined) whose statuses mark entitlement to categorical
speech rather than truth values of the underlying world-claim. The framework distinguishes internal entitlement from public standing while connecting them through the certificate as a boundary object. Philosophically, it develops a form of applied intuitionism: resistance to unwarranted completeness becomes a norm of public machine
speech, under which categorical commitment requires exhibited and contestable entitlement. I operationalize the constraint through structured certificates, decision-layer gates for threshold and argmax predicates, and a reason-coded output contract. A design lemma shows that, under a refutation-soundness condition on the contractually specified negative side, certificate-sound binary totality presupposes witnessed decidability on the declared scope. Under the proposed normative output contract, Undetermined is not a tunable reject option but a mandatory status whenever the available certificates do not uniquely license assertion or denial. By making outputs answerable to contestable warrants rather than confidence alone, the paper aims to preserve epistemic agency where automated
speech enters public justification.
Read the full abstract ▾
Machine Learning
TFWaveFormer: Temporal-Frequency Collaborative Multi-level Wavelet Transformer for Dynamic Link Prediction
Hantong Feng, Yonggang Wu, Duxin Chen et al. (+1 more)
Dynamic link prediction plays a crucial role in diverse applications including social network analysis, communication
forecasting, and financial modeling. While recent
Transformer-based approaches have demonstrated promising results in temporal graph learning, their performance remains limited when capturing complex multi-scale temporal dynamics. In this paper, we propose TFWaveFormer, a novel
Transformer architecture that integrates temporal-frequency analysis with multi-resolution wavelet decomposition to enhance dynamic link prediction. Our framework comprises three key components: (i) a temporal-frequency coordination mechanism that jointly models temporal and spectral representations, (ii) a learnable multi-resolution wavelet decomposition module that adaptively extracts multi-scale temporal patterns through parallel convolutions, replacing traditional iterative wavelet transforms, and (iii) a hybrid
Transformer module that effectively fuses local wavelet features with global temporal dependencies. Extensive experiments on
benchmark datasets demonstrate that TFWaveFormer achieves state-of-the-art performance, outperforming existing
Transformer-based and hybrid models by significant margins across multiple metrics. The superior performance of TFWaveFormer validates the effectiveness of combining temporal-frequency analysis with wavelet decomposition in capturing complex temporal dynamics for dynamic link prediction tasks.
Read the full abstract ▾
All authors Hantong Feng, Yonggang Wu, Duxin Chen, Wenwu Yu
Machine Learning · Cross-listed from cs.SE
LoRA-MME: Multi-Model Ensemble of LoRA-Tuned Encoders for Code Comment Classification
Md Akib Haider, Ahsan Bulbul, Nafis Fuad Shahid et al. (+2 more)
Code comment classification is a critical task for automated software documentation and analysis. In the context of the NLBSE'26 Tool Competition, we present LoRA-MME, a Multi-Model Ensemble architecture utilizing Parameter-Efficient Fine-Tuning (PEFT). Our approach addresses the multi-label classification challenge across Java, Python, and Pharo by combining the strengths of four distinct
transformer encoders: UniXcoder, CodeBERT, GraphCodeBERT, and CodeBERTa. By independently fine-tuning these models using Low-Rank Adaptation(LoRA) and aggregating their predictions via a learned weighted ensemble strategy, we maximize classification performance without the
memory overhead of full model fine-tuning. Our tool achieved an F1 Weighted score of 0.7906 and a Macro F1 of 0.6867 on the test set. However, the computational cost of the ensemble resulted in a final submission score of 41.20%, highlighting the trade-off between semantic accuracy and inference efficiency.
Read the full abstract ▾
All authors Md Akib Haider, Ahsan Bulbul, Nafis Fuad Shahid, Aimaan Ahmed, Mohammad Ishrak Abedin
Machine Learning
GIPO: Gaussian Importance Sampling Policy Optimization
Chengxuan Lu, Zhenquan Zhang, Shukuan Wang et al. (+3 more)
Post-training with reinforcement learning (RL) has recently shown strong promise for advancing
multimodal agents beyond supervised imitation. However, RL remains limited by poor data efficiency, particularly in settings where interaction data are scarce and quickly become outdated. To address this challenge, GIPO (Gaussian Importance sampling Policy Optimization) is proposed as a policy optimization objective based on truncated importance sampling, replacing hard clipping with a log-ratio-based Gaussian trust weight to softly damp extreme importance ratios while maintaining non-zero gradients. Theoretical analysis shows that GIPO introduces an implicit, tunable constraint on the update magnitude, while concentration bounds guarantee robustness and stability under finite-sample estimation. Experimental results show that GIPO achieves state-of-the-art performance among clipping-based baselines across a wide range of replay buffer sizes, from near on-policy to highly stale data, while exhibiting superior bias–variance trade-off, high training stability and improved sample efficiency. Code is available at https://github.com/distanceLu/GIPO.
Read the full abstract ▾
All authors Chengxuan Lu, Zhenquan Zhang, Shukuan Wang, Qunzhi Lin, Baigui Sun, Yang Liu
Machine Learning
Lang2Str: Two-Stage Crystal Structure Generation with LLMs and Continuous Flow Models
Cong Liu, Chengyue Gong, Zhenyu Liu et al. (+2 more)
Generative models hold great promise for accelerating material discovery but are often limited by their inflexible single-stage generative process in designing valid and diverse materials. To address this, we propose a two-stage generative framework, Lang2Str, that combines the strengths of large
language models (LLMs) and flow-based models for flexible and precise material generation. Our method frames the generative process as a conditional generative task, where an LLM provides high-level conditions by generating descriptions of material unit
cells'
geometric layouts and properties. These descriptions, informed by the LLM's extensive background knowledge, ensure reasonable structure designs. A conditioned flow model then decodes these textual conditions into precise continuous coordinates and unit
cell parameters. This staged approach combines the structured
reasoning of LLMs and the distribution modeling capabilities of flow models. Experimental results show that our method achieves competitive performance on ab initio material generation and
crystal structure prediction tasks, with generated structures exhibiting closer
alignment to ground truth in both geometry and energy levels, surpassing state-of-the-art models. The flexibility and modularity of our framework further enable fine-grained control over the generation process, potentially leading to more efficient and customizable material design.
Read the full abstract ▾
All authors Cong Liu, Chengyue Gong, Zhenyu Liu, Jiale Zhao, Yuxuan Zhang
Machine Learning · Cross-listed from cs.SI
How Predicted Links Influence Network Evolution: Disentangling Choice and Algorithmic Feedback in Dynamic Graphs
Mathilde Perez, Raphaël Romero, Jefrey Lijffijt et al. (+1 more)
Link prediction models are increasingly used to recommend interactions in evolving networks, yet their impact on network structure is typically assessed from static snapshots. In particular, observed homophily conflates intrinsic interaction tendencies with amplification effects induced by network dynamics and algorithmic feedback. We propose a temporal framework based on multivariate Hawkes processes that disentangles these two sources and introduce an instantaneous bias measure derived from interaction intensities, capturing current reinforcement dynamics beyond cumulative metrics. We provide a theoretical characterization of the stability and convergence of the induced dynamics, and experiments show that the proposed measure reliably reflects algorithmic feedback effects across different link prediction strategies.
Read the full abstract ▾
All authors Mathilde Perez, Raphaël Romero, Jefrey Lijffijt, Charlotte Laclau
Machine Learning · Cross-listed from cs.NI
Selecting Offline Reinforcement Learning Algorithms for Stochastic Network Control
Nicolas Helson, Pegah Alizadeh and Anastasios Giovanidis
Offline Reinforcement Learning (RL) is a promising approach for next-generation wireless networks, where online exploration is unsafe and large amounts of operational data can be reused across the model lifecycle. However, the behavior of offline RL
algorithms under genuinely
stochastic dynamics – inherent to wireless systems due to fading, noise, and traffic mobility – remains insufficiently understood. We address this gap by evaluating Bellman-based (Conservative Q-Learning), sequence-based (Decision
Transformers), and hybrid (Critic-Guided Decision
Transformers) offline RL methods in an open-access
stochastic telecom environment (mobile-env). Our results show that Conservative Q-Learning consistently produces more robust policies across different sources of stochasticity, making it a reliable default choice in lifecycle-driven AI management frameworks. Sequence-based methods remain competitive and can outperform Bellman-based approaches when sufficient high-return trajectories are available. These findings provide practical guidance for offline RL
algorithm selection in AI-driven network control pipelines, such as O-RAN and future 6G functions, where robustness and data availability are key operational constraints.
Read the full abstract ▾
Machine Learning
Hierarchical Inference and Closure Learning via Adaptive Surrogates for ODEs and PDEs
Pengyu Zhang, Arnaud Vadeboncoeur, Alex Glyn-Davies et al. (+1 more)
Inverse problems are the task of calibrating models to match data. They play a pivotal role in diverse engineering applications by allowing practitioners to align models with reality. In many applications, engineers and scientists do not have a complete picture of i) the detailed properties of a system (such as material properties, geometry, initial conditions, etc.); ii) the complete laws describing all dynamics at play (such as friction laws, complicated damping phenomena, and general nonlinear interactions). In this paper, we develop a principled methodology for leveraging data from collections of distinct yet related physical systems to jointly estimate the individual model parameters of each system, and learn the shared unknown dynamics in the form of an ML-based closure model. To robustly infer the unknown parameters for each system, we employ a hierarchical
Bayesian framework, which allows for the joint inference of multiple systems and their population-level statistics. To learn the closures, we use a maximum marginal likelihood estimate of a neural network embeded within the ODE/PDE formulation of the problem. To realize this framework we utilize the ensemble Metropolis-Adjusted Langevin
Algorithm (MALA) for stable and efficient sampling. To mitigate the computational bottleneck of repetitive forward evaluations in solving inverse problems, we introduce a bilevel optimization strategy to simultaneously train a surrogate forward model alongside the inference. Within this framework, we evaluate and compare distinct surrogate architectures, specifically
Fourier Neural Operators (FNO) and parametric Physics-Informed Neural Network (PINNs).
Read the full abstract ▾
All authors Pengyu Zhang, Arnaud Vadeboncoeur, Alex Glyn-Davies, Mark Girolami
Machine Learning
BD-Merging: Bias-Aware Dynamic Model Merging with Evidence-Guided Contrastive Learning
Yuhan Xie and Chen Lyu
Model Merging (MM) has emerged as a scalable paradigm for multi-task learning (MTL), enabling multiple task-specific models to be integrated without revisiting the original training data. Despite recent progress, the reliability of MM under test-time distribution shift remains insufficiently understood. Most existing MM methods typically assume that test data are clean and distributionally aligned with both the training and auxiliary sources. However, this assumption rarely holds in practice, often resulting in biased predictions with degraded generalization. To address this issue, we present BD-Merging, a bias-aware unsupervised model merging framework that explicitly models uncertainty to achieve adaptive reliability under distribution shift. First, BD-Merging introduces a joint evidential head that learns uncertainty over a unified label space, capturing cross-task semantic dependencies in MM. Second, building upon this evidential foundation, we propose an Adjacency Discrepancy Score (ADS) that quantifies evidential
alignment among neighboring samples. Third, guided by ADS, a discrepancy-aware contrastive learning mechanism refines the merged representation by aligning consistent samples and separating conflicting ones. Combined with general unsupervised learning, this process trains a debiased router that adaptively allocates task-specific or layer-specific weights on a per-sample basis, effectively mitigating the adverse effects of distribution shift. Extensive experiments across diverse tasks demonstrate that BD-Merging achieves superior effectiveness and robustness compared to state-of-the-art MM baselines.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
From Misclassifications to Outliers: Joint Reliability Assessment in Classification
Yang Li, Youyang Sha, Yinzhi Wang et al. (+4 more)
Building reliable classifiers is a fundamental challenge for deploying machine learning in real-world applications. A reliable system should not only detect out-of-distribution (OOD) inputs but also anticipate in-distribution (ID) errors by assigning low confidence to potentially misclassified samples. Yet, most prior work treats OOD detection and failure prediction as separated problems, overlooking their closed connection. We argue that reliability requires evaluating them jointly. To this end, we propose a unified evaluation framework that integrates OOD detection and failure prediction, quantified by our new metrics DS-F1 and DS-AURC, where DS denotes double scoring functions. Experiments on the OpenOOD
benchmark show that double scoring functions yield classifiers that are substantially more reliable than traditional single scoring approaches. Our analysis further reveals that OOD-based approaches provide notable gains under simple or far-OOD shifts, but only marginal benefits under more challenging near-OOD conditions. Beyond evaluation, we extend the reliable classifier SURE and introduce SURE+, a new approach that significantly improves reliability across diverse scenarios. Together, our framework, metrics, and method establish a new
benchmark for trustworthy classification and offer practical guidance for deploying robust models in real-world settings. The source code is publicly available at https://github.com/Intellindust-AI-Lab/SUREPlus.
Read the full abstract ▾
All authors Yang Li, Youyang Sha, Yinzhi Wang, Timothy Hospedales, Xi Shen, Shell Xu Hu, Xuanlong Yu
Machine Learning
PatchDecomp: Interpretable Patch-Based Time Series Forecasting
Hiroki Tomioka and Genta Yoshimura
Time series
forecasting, which predicts future values from past observations, plays a central role in many domains and has driven the development of highly accurate neural network models. However, the complexity of these models often limits human understanding of the rationale behind their predictions. We propose PatchDecomp, a neural network-based time series
forecasting method that achieves both high accuracy and interpretability. PatchDecomp divides input time series into subsequences (patches) and generates predictions by aggregating the contributions of each patch. This enables clear attribution of each patch, including those from exogenous variables, to the final prediction. Experiments on multiple
benchmark datasets demonstrate that PatchDecomp provides predictive performance comparable to recent
forecasting methods. Furthermore, we show that the model's explanations not only influence predicted values quantitatively but also offer qualitative interpretability through visualization of patch-wise contributions.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.RO
IROSA: Interactive Robot Skill Adaptation using Natural Language
Markus Knauer, Samuel Bustamante, Thomas Eiband et al. (+3 more)
Foundation models have demonstrated impressive capabilities across diverse domains, while imitation learning provides principled methods for
robot skill adaptation from limited data. Combining these approaches holds significant promise for direct application to robotics, yet this combination has received limited
attention, particularly for industrial deployment. We present a novel framework that enables open-vocabulary skill adaptation through a tool-based architecture, maintaining a protective abstraction layer between the
language model and
robot hardware. Our approach leverages pre-trained LLMs to select and parameterize specific tools for adapting
robot skills without requiring fine-tuning or direct model-to-
robot interaction. We demonstrate the framework on a 7-DoF torque-controlled
robot performing an industrial bearing ring insertion task, showing successful skill adaptation through natural
language commands for speed adjustment, trajectory correction, and obstacle avoidance while maintaining safety, transparency, and interpretability.
Read the full abstract ▾
All authors Markus Knauer, Samuel Bustamante, Thomas Eiband, Alin Albu-Schäffer, Freek Stulp, João Silvério