Machine Learning · Cross-listed from math.NA
Sampling via Stochastic Interpolants by Langevin-based Velocity and Initialization Estimation in Flow ODEs
Chenguang Duan, Yuling Jiao, Gabriele Steidl et al. (+3 more)
We propose a novel method for sampling from unnormalized Boltzmann densities based on a probability flow ordinary differential equation (ODE) derived from linear
stochastic interpolants. The key innovation of our approach is the use of a sequence of Langevin samplers to enable efficient simulation of the flow. Specifically, these Langevin samplers are employed (i) to generate samples from the interpolant distribution at intermediate times and (ii) to construct, starting from these intermediate times, a robust
estimator of the velocity field governing the probability flow ODE. Theoretically, we provide convergence guarantees for both Langevin components, and establish a non-asymptotic convergence rate for the probability flow ODE. Extensive numerical experiments demonstrate the efficiency of the proposed method on challenging
multimodal distributions across a range of dimensions, as well as its effectiveness in
Bayesian inference tasks.
Read the full abstract ▾
All authors Chenguang Duan, Yuling Jiao, Gabriele Steidl, Christian Wald, Jerry Zhijian Yang, Ruizhe Zhang
Machine Learning · Cross-listed from cs.NE
Building Supervision into Hebbian Plasticity through Spike Agreement
Gouri Lakshmi S, Athira Chandrasekharan, Harshit Kumar et al. (+3 more)
Supervised learning in spiking neural networks (SNNs) typically requires either gradient-based backpropagation, which sacrifices the Hebbian, spike-driven character of biological plasticity, or reward-modulated Spike-Timing-Dependent Plasticity (STDP), in which class supervision enters only as a scalar gate on an otherwise class-agnostic correlation signal. We propose Supervised Spike Agreement-Dependent Plasticity (Supervised SADP), a gradient-free supervised Hebbian learning
algorithm in which class information is embedded directly into the Hebbian plasticity computation rather than introduced through reward modulation. SADP trains the output layer via a supervised Hebbian rule that encodes class labels into output spike patterns, then trains the hidden layer by measuring each hidden
neuron's chance-corrected temporal agreement, Cohen's kappa, with the correct-class output spike train produced by the forward pass without gradient computation or external reward. A K-shift extension aggregates agreement over temporal offsets, providing robustness to spike-timing jitter at linear computational cost. We evaluate Supervised SADP against reward-modulated STDP across six
benchmark and medical imaging
datasets, four input encoding strategies, K_shift in {5,25}, and three reward modes (none, binary, margin). Supervised SADP outperforms STDP in a significant majority of comparisons. Under Poisson encoding, SADP achieves 86.46% on MNIST and 76.62% on Fashion-MNIST, outperforming the best STDP configurations by 23.66 and 23.29 percentage points, respectively. Across the encodings tested, including CNN-extracted features, SADP outperforms STDP in the large majority of
cells and trains 1.47x faster on average, with up to 2.86x speedup under Poisson inputs. These results position Supervised SADP as a stable, efficient, gradient-free alternative to reward-modulated STDP for supervised SNN learning.
Read the full abstract ▾
All authors Gouri Lakshmi S, Athira Chandrasekharan, Harshit Kumar, Muhammed Sahad E, Bikas C Das, Saptarshi Bej
Machine Learning
Your Group-Relative Advantage Is Biased
Fengkai Yang, Zherui Chen, Xiaohan Wang et al. (+10 more)
Reinforcement Learning from Verifier Rewards (RLVR) has emerged as a widely used approach for post-training large
language models on
reasoning tasks, with group-based methods such as GRPO and its variants gaining broad adoption. These methods rely on group-relative advantage estimation to avoid learned critics, yet its theoretical properties remain poorly understood. In this work, we characterize conditional estimation bias in group-relative advantages among non-degenerate groups. For mean-centered advantages under binary rewards, our analysis shows negative conditional mean estimation errors for hard
prompts and positive errors for easy
prompts. At the response level, correct answers to hard
prompts are underweighted, while incorrect answers to easy
prompts receive insufficient penalties. To address this issue, we propose History-Aware Adaptive Difficulty Weighting (HA-DW), an adaptive reweighting scheme that adjusts advantage estimates based on an evolving difficulty anchor and training dynamics. Both theoretical analysis and experiments on five mathematical
reasoning benchmarks demonstrate that HA-DW consistently improves performance when integrated into GRPO and its variants. Our results suggest that correcting biased advantage estimation is critical for robust and efficient RLVR training.
Read the full abstract ▾
All authors Fengkai Yang, Zherui Chen, Xiaohan Wang, Xiaodong Lu, Jiajun Chai, Guojun Yin, Wei Lin, Shuai Ma, Fuzhen Zhuang, Deqing Wang, Yaodong Yang, Jianxin Li, Yikun Ban
Machine Learning · Cross-listed from cs.CL
STAR: Detecting Inference-time Backdoors in LLM Reasoning via State-Transition Amplification Ratio
Seong-Gyu Park, Sohee Park, Jisu Lee et al. (+2 more)
Recent LLMs increasingly integrate
reasoning mechanisms like Chain-of-Thought (CoT). However, this explicit
reasoning exposes a new attack surface for inference-time
backdoors, which inject malicious
reasoning paths without altering model parameters. Because these attacks generate linguistically coherent paths, they effectively evade conventional detection. To address this, we propose
STAR (State-Transition Amplification Ratio), a framework that detects
backdoors by analyzing output probability shifts.
STAR exploits the statistical discrepancy where a malicious input-induced path exhibits high posterior probability despite a low prior probability in the model's general knowledge. We quantify this state-transition amplification and employ the CUSUM
algorithm to detect persistent anomalies. Experiments across diverse models (8B-70B) and five
benchmark datasets demonstrate that
STAR exhibits robust generalization capabilities, consistently achieving near-perfect performance (AUROC $\approx$ 1.0) with approximately $42\times$ greater efficiency than existing baselines. Furthermore, the framework proves robust against adaptive attacks attempting to bypass detection.
Read the full abstract ▾
All authors Seong-Gyu Park, Sohee Park, Jisu Lee, Hyunsik Na, Daeseon Choi
Machine Learning
Temporal Fusion Nexus: A task-agnostic multi-modal embedding model for clinical narratives and irregular time series in post-kidney transplant care
Aditya Kumar, Simon Rauch, Mario Cypko et al. (+10 more)
We introduce Temporal Fusion Nexus (TFN), a multi-modal and task-agnostic
embedding model to integrate irregular time series and unstructured clinical narratives. We analysed TFN in post-kidney transplant (KTx) care, with a retrospective cohort of 3382 patients, on three key outcomes: graft loss, graft rejection, and mortality. Compared to state-of-the-art model in post KTx care, TFN achieved higher performance for graft loss (AUC 0.96 vs. 0.94) and graft rejection (AUC 0.84 vs. 0.74). In mortality prediction, TFN yielded an AUC of 0.86. TFN outperformed unimodal baselines (approx 10% AUC improvement over time series only baseline, approx 5% AUC improvement over time series with static patient data). Integrating clinical text improved performance across all tasks. Disentanglement metrics confirmed robust and interpretable latent factors in the
embedding space, and SHAP-based attributions confirmed
alignment with clinical
reasoning. TFN has potential application in clinical tasks beyond KTx, where heterogeneous data sources, irregular longitudinal data, and rich narrative documentation are available.
Read the full abstract ▾
All authors Aditya Kumar, Simon Rauch, Mario Cypko, Marcel Naik, Matthieu-P Schapranow, Aadil Rashid, Fabian Halleck, Bilgin Osmanodja, Roland Roller, Lars Pape, Klemens Budde, Mario Schiffer, Oliver Amft
Machine Learning · Cross-listed from cs.IR
GraphFusionSBR: Denoising Multi-Channel Graphs for Session-Based Recommendation
Jia-Xin He and Hung-Hsuan Chen
Session-based recommendation systems must capture implicit user intents from sessions. However, existing models suffer from issues such as item interaction dominance and noisy sessions. We propose a multi-channel recommendation model, including a knowledge graph channel, a session hypergraph channel, and a session line graph channel, to capture information from multiple sources. Our model adaptively removes redundant edges in the knowledge graph channel to reduce noise. Knowledge graph representations cooperate with hypergraph representations for prediction to alleviate item dominance. We also generate in-session
attention for denoising. Finally, we maximize mutual information between the hypergraph and line graph channels as an auxiliary task. Experiments demonstrate that our method enhances the accuracy of various recommendations, including e-commerce and multimedia recommendations. We release the code on GitHub for reproducibility.\footnote{https://github.com/hohehohe0509/DSR-HK}
Read the full abstract ▾
Machine Learning · Cross-listed from cs.RO
AUV Trajectory Learning for Underwater Acoustic Energy Transfer and Age Minimization
Mohamed Afouene Melki, Mohammad Shehab and Mohamed-Slim Alouini
Internet of
underwater things (IoUT) is increasingly gathering
attention with the aim of monitoring sea life and deep ocean environment,
underwater surveillance as well as maintenance of
underwater installments. However, conventional IoUT devices, reliant on battery power, face limitations in lifespan and pose environmental hazards upon disposal. This paper introduces a sustainable approach for simultaneous information uplink from the IoUT devices and acoustic energy transfer (AET) to the devices via an autonomous
underwater vehicle (AUV), potentially enabling them to operate indefinitely. To tackle the time-sensitivity, we adopt age of information (AoI), and Jain's
fairness index. We develop two deep-reinforcement learning (DRL)
algorithms, offering a high-complexity, high-performance frequency division duplex (FDD) solution and a low-complexity, medium-performance time division duplex (TDD) approach. The results elucidate that the proposed FDD and TDD solutions significantly reduce the average AoI and boost the harvested energy as well as data collection
fairness compared to baseline approaches.
Read the full abstract ▾
Machine Learning
DiffMM: Efficient Method for Accurate Noisy and Sparse Trajectory Map Matching via One Step Diffusion
Chenxu Han, Sean Bin Yang and Jilin Hu
Map matching for sparse trajectories is a fundamental problem for many trajectory-based applications, e.g., traffic scheduling and traffic flow analysis. Existing methods for map matching are generally based on Hidden Markov Model (HMM) or encoder-decoder framework. However, these methods continue to face significant challenges when handling noisy or sparsely sampled GPS trajectories. To address these limitations, we propose DiffMM, an encoder-
diffusion-based map matching framework that produces effective yet efficient matching results through a one-step
diffusion process. We first introduce a road segment-aware trajectory encoder that jointly embeds the input trajectory and its surrounding candidate road segments into a shared latent space through an
attention mechanism. Next, we propose a one step
diffusion method to realize map matching through a shortcut model by leveraging the joint
embedding of the trajectory and candidate road segments as conditioning context. We conduct extensive experiments on large-scale trajectory
datasets, demonstrating that our approach consistently outperforms state-of-the-art map matching methods in terms of both accuracy and efficiency, particularly for sparse trajectories and complex road network
topologies.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
JudgeRLVR: Judge First, Generate Second for Efficient Reasoning
Jiangshan Duo, Hanyu Li, Hailin Zhang et al. (+3 more)
Reinforcement Learning with Verifiable Rewards (RLVR) has become a standard paradigm for
reasoning in Large
Language Models. However, optimizing solely for final-answer correctness often drives models into aimless, verbose exploration, where they rely on exhaustive trial-and-error tactics rather than structured planning to reach solutions. While heuristic constraints like length penalties can reduce verbosity, they often truncate essential
reasoning steps, creating a difficult trade-off between efficiency and verification. In this paper, we argue that discriminative capability is a prerequisite for efficient generation: by learning to distinguish valid solutions, a model can internalize a guidance signal that prunes the search space. We propose JudgeRLVR, a two-stage judge-then-generate paradigm. In the first stage, we train the model to judge solution responses with verifiable answers. In the second stage, we fine-tune the same model with vanilla generating RLVR initialized from the judge. Compared to Vanilla RLVR using the same math-domain training data, JudgeRLVR achieves a better quality–efficiency trade-off for Qwen3-30B-A3B: on in-domain math, it delivers about +3.7 points average accuracy gain with -42% average generation length; on out-of-domain
benchmarks, it delivers about +4.5 points average accuracy improvement, demonstrating enhanced generalization.
Read the full abstract ▾
All authors Jiangshan Duo, Hanyu Li, Hailin Zhang, Yudong Wang, Sujian Li, Liang Zhao
Machine Learning · Cross-listed from cs.CV
Zero-Shot Distracted Driver Detection via Vision Language Models with Double Decoupling
Takamichi Miyata, Sumiko Miyata and Andrew Morris
Distracted driving is a major cause of traffic collisions, calling for robust and scalable detection methods.
Vision-
language models (VLMs) enable strong zero-shot image classification, but existing VLM-based distracted driver detectors often underperform in real-world conditions. We identify subject-specific appearance variations (e.g., clothing, age, and gender) as a key bottleneck: VLMs entangle these factors with behavior cues, leading to decisions driven by who the driver is rather than what the driver is doing. To address this, we propose a subject decoupling framework that extracts a driver appearance
embedding and removes its influence from the image
embedding prior to zero-shot classification, thereby emphasizing distraction-relevant evidence. We further orthogonalize text
embeddings via metric projection onto Stiefel
manifold to improve separability while staying close to the original semantics. Experiments demonstrate consistent gains over prior baselines, indicating the promise of our approach for practical road-safety applications. Code is available at https://github.com/mtakamichi/ZVL-DDD
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Noise-Adaptive Regularization for Robust Multi-Label Remote Sensing Image Classification
Tom Burgert, Julia Henkel and Begüm Demir
The development of reliable methods for multi-label classification (MLC) has become a prominent research direction in remote sensing (RS). As the scale of RS data continues to expand, annotation procedures increasingly rely on thematic products or crowdsourced procedures to reduce the cost of manual annotation. While cost-effective, these strategies often introduce multi-label noise in the form of partially incorrect annotations. In MLC, label noise arises as additive noise, subtractive noise, or a combination of both in the form of mixed noise. Previous work has largely overlooked this distinction and commonly treats noisy annotations as supervised signals, lacking mechanisms that explicitly adapt learning behavior to different noise types. To address this limitation, we propose NAR, a noise-adaptive regularization method that explicitly distinguishes between additive and subtractive noise within a semi-supervised learning framework. NAR employs a confidence-based label handling mechanism that dynamically retains label entries with high confidence, temporarily deactivates entries with moderate confidence, and corrects low confidence entries via flipping. This selective attenuation of supervision is integrated with early-learning regularization (ELR) to stabilize training and mitigate overfitting to corrupted labels. Experiments across additive, subtractive, and mixed noise scenarios demonstrate that NAR consistently improves robustness compared with existing methods. Performance improvements are most pronounced under subtractive and mixed noise, indicating that adaptive suppression and selective correction of noisy supervision provide an effective strategy for noise robust learning in RS MLC.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Silence the Judge: Reinforcement Learning with Self-Verifier via Latent Geometric Clustering
Nonghai Zhang, Weitao Ma, Zhanyu Ma et al. (+5 more)
Group Relative Policy Optimization (GRPO) significantly enhances the
reasoning performance of Large
Language Models (LLMs). However, this success heavily relies on expensive external verifiers or human rules. Such dependency not only leads to significant computational costs and training latency, but also yields sparse rewards that hinder optimization efficiency. To address these challenges, we propose Latent-GRPO, a framework that derives intrinsic rewards directly from latent space geometry. Crucially, our empirical analysis reveals a compelling
geometric property: terminal
token representations of correct
reasoning trajectories form dense clusters with high intra-class similarity, whereas incorrect trajectories remain scattered as outliers. In light of this discovery, we introduce the Iterative Robust Centroid Estimation (IRCE)
algorithm, which generates dense, continuous rewards by mitigating magnitude fluctuations via spherical projection and estimating a robust “truth centroid” through iterative aggregation. Experimental results on multiple
datasets show that our method maintains model performance while achieving a training speedup of over 2x compared to baselines. Furthermore, extensive results demonstrate strong generalization ability and robustness. The code will be released soon.
Read the full abstract ▾
All authors Nonghai Zhang, Weitao Ma, Zhanyu Ma, Jun Xu, Jiuchong Gao, Jinghua Hao, Renqing He, Jingwen Xu
Machine Learning
Coverage Improvement and Fast Convergence of On-policy Preference Learning
Juno Kim, Jihun Yun, Jason D. Lee et al. (+1 more)
Online on-policy preference learning
algorithms for
language model
alignment such as online direct policy optimization (DPO) can significantly outperform their offline counterparts. We provide a theoretical explanation for this phenomenon by analyzing how the sampling policy's coverage evolves throughout on-policy training. We propose and rigorously justify the coverage improvement principle: with sufficient batch size, each update moves into a region around the target where coverage is uniformly better, making subsequent data increasingly informative and enabling rapid convergence. In the contextual bandit setting with Bradley-Terry preferences and linear softmax policy class, we show that on-policy DPO converges exponentially in the number of iterations for batch size exceeding a generalized coverage threshold. In contrast, any learner restricted to offline samples from the initial policy suffers a slower minimax rate, leading to a sharp separation in total sample complexity. Motivated by this analysis, we further propose a simple hybrid sampler based on a novel preferential G-optimal design, which removes dependence on coverage and guarantees convergence in just two rounds. Finally, we develop principled on-policy schemes for reward
distillation in the general function class setting, and show faster noiseless rates under an alternative deviation-based notion of coverage. Experimentally, we confirm that on-policy DPO and our proposed reward
distillation algorithms outperform their off-policy counterparts and enjoy stable, monotonic performance gains across iterations.
Read the full abstract ▾
All authors Juno Kim, Jihun Yun, Jason D. Lee, Kwang-Sung Jun
Machine Learning
Taxon: Hierarchical Tax Code Prediction with Semantically Aligned LLM Expert Guidance
Jihang Li, Qing Liu, Zulong Chen et al. (+4 more)
Tax code prediction is a crucial yet underexplored task in automating invoicing and compliance management for large-scale e-commerce platforms. Each product must be accurately mapped to a node within a multi-level taxonomic hierarchy defined by national standards, where errors lead to financial inconsistencies and regulatory risks. This paper presents Taxon, a semantically aligned and expert-guided framework for hierarchical tax code prediction. Taxon integrates (i) a feature-gating mixture-of-experts architecture that adaptively routes multi-modal features across taxonomy levels, and (ii) a semantic consistency model distilled from large
language models acting as domain experts to verify
alignment between product titles and official tax definitions. To address noisy supervision in real business records, we design a multi-source training pipeline that combines curated tax databases, invoice validation logs, and merchant registration data to provide both structural and semantic supervision. Extensive experiments on the proprietary TaxCode
dataset and public
benchmarks demonstrate that Taxon achieves state-of-the-art performance, outperforming strong baselines. Further, an additional full hierarchical paths reconstruction procedure significantly improves structural consistency, yielding the highest overall F1 scores. Taxon has been deployed in production within Alibaba's tax service system, handling an average of over 500,000 tax code queries per day and reaching peak volumes above five million requests during business event with improved accuracy, interpretability, and robustness.
Read the full abstract ▾
All authors Jihang Li, Qing Liu, Zulong Chen, Jing Wang, Wei Wang, Chuanfei Xu, Zeyi Wen
Machine Learning
Out-of-distribution generalization of deep-learning surrogates for 2D PDE-generated dynamics in the small-data regime
Binh Duong Nguyen and Stefan Sandfeld
Partial differential equations (PDEs) are a central tool for modeling the dynamics of physical, engineering, and materials systems, but high-fidelity simulations are often computationally expensive. At the same time, many scientific applications can be viewed as the
evolution of spatially distributed fields, making data-driven
forecasting of such fields a core task in scientific machine learning. In this work we study autoregressive deep-learning surrogates for two-dimensional PDE dynamics on periodic domains, focusing on generalization to out-of-distribution initial conditions within a fixed PDE and parameter regime and on strict small-data settings with at most $\mathcal{O}(10^2)$ simulated trajectories per system. We introduce a multi-channel U-Net [...], evaluate it on five qualitatively different PDE families and compare it to ViT, AFNO, PDE-
Transformer, and KAN-UNet under a common training setup. Across all
datasets, me-UNet matches or outperforms these more complex architectures in terms of field-space error, spectral similarity, and physics-based metrics for in-distribution rollouts, while requiring substantially less training time. It also generalizes qualitatively to unseen initial conditions with as few as $\approx 20$ training simulations. A data-efficiency study and Grad-CAM analysis further suggest that, in small-data periodic 2D PDE settings, convolutional architectures with inductive biases aligned to locality and periodic boundary conditions remain strong contenders for accurate and moderately out-of-distribution-robust surrogate modeling.
Read the full abstract ▾
Machine Learning
Controlled LLM Training on Spectral Sphere
Tian Xie, Haoming Luo, Haoyu Tang et al. (+9 more)
Scaling large models requires optimization strategies that ensure rapid convergence grounded in stability. Maximal Update Parametrization ($\boldsymbol{\mu}$P) provides a theoretical safeguard for width-invariant $\Theta(1)$ activation control, whereas emerging optimizers like
Muon are only “half-aligned” with these constraints: they control updates but allow weights to drift. To address this limitation, we introduce the Spectral Sphere Optimizer (SSO), which enforces strict module-wise spectral constraints on both weights and their updates. By deriving the steepest descent direction on the spectral sphere, SSO realizes a fully $\boldsymbol{\mu}$P-aligned optimization process. To enable large-scale training, we implement SSO as an efficient parallel
algorithm within Megatron. Through extensive pretraining on diverse architectures, including Dense 1.7B, MoE 8B-A1B, and 200-layer DeepNet models, SSO consistently outperforms AdamW and
Muon. Furthermore, we observe significant practical stability benefits, including improved MoE router load balancing, suppressed outliers, and strictly bounded activations.
Read the full abstract ▾
All authors Tian Xie, Haoming Luo, Haoyu Tang, Yiwen Hu, Jason Klein Liu, Qingnan Ren, Yang Wang, Wayne Xin Zhao, Rui Yan, Bing Su, Chong Luo, Baining Guo
Machine Learning · Cross-listed from cs.AI
Deconstructing Pre-training: Knowledge Attribution Analysis in MoE and Dense Models
Bo Wang, Junzhuo Li, Hong Chen et al. (+3 more)
Mixture-of-Experts (MoE) architectures decouple model capacity from per-
token computation, enabling scaling beyond the computational limits imposed by dense scaling laws. Yet how MoE architectures shape knowledge acquisition during pre-training, and how this process differs from dense architectures, remains unknown. To address this issue, we introduce Gated-LPI (Log-Probability Increase), a
neuron-level attribution metric that decomposes log-probability increase across
neurons. We present a time-resolved comparison of knowledge acquisition dynamics in MoE and dense architectures, tracking checkpoints over 1.2M training steps (~ 5.0T
tokens) and 600K training steps (~ 2.5T
tokens), respectively. Our experiments uncover three patterns: (1) Low-
entropy backbone. The top approximately 1% of MoE
neurons capture over 45% of positive updates, forming a high-utility core, which is absent in the dense baseline. (2) Early consolidation. The MoE model locks into a stable importance profile within < 100K steps, whereas the dense model remains volatile throughout training. (3) Functional robustness. Masking the ten most important MoE
attention heads reduces relational HIT@10 by < 10%, compared with > 50% for the dense model, showing that sparsity fosters distributed – rather than brittle – knowledge storage. These patterns collectively demonstrate that sparsity fosters an intrinsically stable and distributed computational backbone from early in training, helping bridge the gap between sparse architectures and training-time interpretability.
Read the full abstract ▾
All authors Bo Wang, Junzhuo Li, Hong Chen, Yuanlin Chu, Yuxuan Fan, Xuming Hu
Machine Learning
MMD Guidance: Training-Free Distribution Adaptation for Diffusion Models via Maximum Mean Discrepancy Guidance
Matina Mahdizadeh Sani, Nima Jamali, Mohammad Jalali et al. (+1 more)
Pre-trained
diffusion models have emerged as powerful generative priors for both unconditional and conditional sample generation, yet their outputs often deviate from the characteristics of user-specific target data. Such mismatches are especially problematic in domain adaptation tasks, where only a few reference examples are available and retraining the
diffusion model is infeasible. Existing inference-time guidance methods can adjust sampling trajectories, but they typically optimize surrogate objectives such as classifier likelihoods rather than directly aligning with the target distribution. We propose MMD Guidance, a training-free mechanism that augments the reverse
diffusion process with gradients of the Maximum Mean Discrepancy (MMD) between generated samples and a reference
dataset. MMD provides reliable distributional estimates from limited data, exhibits low variance in practice, and is efficiently differentiable, which makes it particularly well-suited for the guidance task. Our framework naturally extends to
prompt-aware adaptation in conditional generation models via product kernels. Also, it can be applied with computational efficiency in latent
diffusion models (LDMs), since guidance is applied in the latent space of the LDM. Experiments on synthetic and real-world
benchmarks demonstrate that MMD Guidance can achieve distributional
alignment while preserving sample fidelity. The project code is available at github.com/matinamehdizadeh/MMD-Guidance.
Read the full abstract ▾
All authors Matina Mahdizadeh Sani, Nima Jamali, Mohammad Jalali, Farzan Farnia
Machine Learning · Cross-listed from cs.CV
Geo-NVS-w: Geometry-Aware Novel View Synthesis In-the-Wild with an SDF Renderer
Anastasios Tsalakopoulos, Angelos Kanlis, Evangelos Chatzis et al. (+2 more)
We introduce Geo-NVS-w, a geometry-aware framework for high-fidelity novel view synthesis from unstructured, in-the-wild image collections. While existing in-the-wild methods already excel at novel view synthesis, they often lack
geometric grounding on complex surfaces, sometimes producing results that contain inconsistencies. Geo-NVS-w addresses this limitation by leveraging an underlying
geometric representation based on a Signed Distance Function (SDF) to guide the rendering process. This is complemented by a novel Geometry-Preservation Loss which ensures that fine structural details are preserved. Our framework achieves competitive rendering performance, while demonstrating a 4-5x reduction reduction in energy consumption compared to similar methods. We demonstrate that Geo-NVS-w is a robust method for in-the-wild NVS, yielding photorealistic results with sharp, geometrically coherent details.
Read the full abstract ▾
All authors Anastasios Tsalakopoulos, Angelos Kanlis, Evangelos Chatzis, Antonis Karakottas, Dimitrios Zarpalas
Machine Learning
Linear probing enables Ship-Radiated Noise recognition with pretrained audio embeddings
Hilde I. Hummel, Sandjai Bhulai, Rob D. van der Mei et al. (+1 more)
Even though the ocean covers the majority of the
planet's surface, it remains the least explored ecosystem. As light and
radio waves do not propagate through water,
underwater acoustics is the main choice for various ocean applications ranging from marine biology to pollution monitoring. Increasing levels of anthropogenic noise from ships contribute significantly to
underwater sound pollution, posing risks to marine ecosystems. This makes monitoring crucial to understand and quantify the impact of the ship radiated noise. Passive Acoustic Monitoring (PAM) systems are widely deployed for this purpose, generating years of
underwater recordings across diverse soundscapes. Manual analysis of such large-scale data is impractical, motivating the need for automated approaches based on machine learning. Recent advances in automatic
Underwater Acoustic Target Recognition (UATR) have largely relied on supervised learning, which is constrained by the scarcity of labeled data. Transfer Learning (TL) offers a promising alternative to mitigate this limitation. In this work, we conduct the first empirical comparative study of transfer learning for UATR, evaluating multiple pretrained audio models originating from diverse audio domains. The pretrained model weights are frozen, and the resulting
embeddings are analyzed through classification, clustering, and similarity-based evaluations. The analysis shows that the geometrical structure of the
embedding space is largely dominated by recording-specific characteristics. However, a simple linear probe can effectively suppress this recording-specific information and isolate ship-type features from these
embeddings. As a result, linear probing enables effective automatic UATR using pretrained audio models at low computational cost, significantly reducing the need for a large amounts of high-quality labeled ship recordings.
Read the full abstract ▾
All authors Hilde I. Hummel, Sandjai Bhulai, Rob D. van der Mei, Burooj Ghani
Machine Learning · Cross-listed from cs.IR
MLPlatt: Simple Calibration Framework for Ranking Models
Piotr Bajger, Roman Dusek, Krzysztof Galias et al. (+3 more)
Ranking models are extensively used in e-commerce for relevance estimation. These models often suffer from poor interpretability and no scale calibration, particularly when trained with typical ranking loss functions. This paper addresses the problem of post-hoc calibration of ranking models. We introduce MLPlatt: a simple yet effective ranking model calibration method that preserves the item ordering and converts ranker outputs to interpretable click-through rate (CTR) probabilities usable in downstream tasks. The method is context-aware by design and achieves good calibration metrics globally, and within strata corresponding to different values of a selected categorical field (such as user country or device), which is often important from a business perspective of an E-commerce platform. We demonstrate the superiority of MLPlatt over existing approaches on two
datasets, achieving an improvement of over 10% in F-ECE (Field Expected Calibration Error) compared to other methods. Most importantly, we show that high-quality calibration can be achieved without compromising the ranking quality.
Read the full abstract ▾
All authors Piotr Bajger, Roman Dusek, Krzysztof Galias, Paweł Młyniec, Aleksander Wawer, Paweł Zawistowski
Machine Learning
Automated Machine Learning in Radiomics: A Comparative Evaluation of Performance, Efficiency and Accessibility
Jose Lozano-Montoya, Emilio Soria-Olivas, Almudena Fuster-Matanzo et al. (+2 more)
Automated machine learning (AutoML) frameworks can lower technical barriers for predictive and prognostic model development in radiomics by enabling researchers without programming expertise to build models. However, their effectiveness in addressing radiomics-specific challenges remains unclear. This study evaluates the performance, efficiency, and accessibility of general-purpose and radiomics-specific AutoML frameworks on diverse radiomics classification tasks, thereby highlighting development needs for radiomics. Ten public/private radiomics
datasets with varied imaging modalities (CT/MRI), sizes, anatomies and endpoints were used. Six general-purpose and five radiomics-specific frameworks were tested with predefined parameters using standardized cross-validation. Evaluation metrics included AUC, runtime, together with qualitative aspects related to software status, accessibility, and interpretability. Simplatab, a radiomics-specific tool with a no-code interface, achieved the highest average test AUC (81.81%) with a moderate runtime (~1 hour). LightAutoML, a general-purpose framework, showed the fastest execution with competitive performance (78.74% mean AUC in six minutes). Most radiomics-specific frameworks were excluded from the performance analysis due to obsolescence, extensive programming requirements, or computational inefficiency. Conversely, general-purpose frameworks demonstrated higher accessibility and ease of implementation. Simplatab provides an effective balance of performance, efficiency, and accessibility for radiomics classification problems. However, significant gaps remain, including the lack of accessible survival analysis support and the limited integration of feature reproducibility and harmonization within current AutoML frameworks. Future research should focus on adapting AutoML solutions to better address these radiomics-specific challenges.
Read the full abstract ▾
All authors Jose Lozano-Montoya, Emilio Soria-Olivas, Almudena Fuster-Matanzo, Angel Alberich-Bayarri, Ana Jimenez-Pastor
Machine Learning · Cross-listed from q-bio.QM
Disentangling History and Propagation Dependencies in Cross-Subject Knee Contact Stress Prediction Using a Shared MeshGraphNet Backbone
Zhengye Pan, Jianwei Zuo and Jiajia Luo
Background:Subject-specific finite element analysis accurately characterizes knee joint mechanics but is computationally expensive. Deep surrogate models provide a rapid alternative, yet their generalization across subjects under limited pose and load inputs remains unclear. It remains unclear whether the dominant source of prediction uncertainty arises from temporal history dependence or spatial propagation dependence. Methods:To disentangle these factors, we employed a shared MGN backbone with a fixed mesh
topology. A
dataset of running trials from nine subjects was constructed using an OpenSim-FEBio workflow. We developed four model variants to isolate specific dependencies: (1) a baseline MGN; (2) CT-MGN, incorporating a Control
Transformer to encode short-horizon history; (3) MsgModMGN, applying state-conditioned modulation to message passing for adaptive propagation; (4) CT-MsgModMGN, combining both mechanisms. Models were evaluated using a rigorous grouped 3-fold cross-validation on unseen subjects.Results:The models incorporating history encoding significantly outperformed the baseline MGN and MsgModMGN in global accuracy and spatial consistency. Crucially, the CT module effectively mitigated the peak-shaving defect common in deep surrogates, significantly reducing peak stress prediction errors. In contrast, the spatial propagation modulation alone yielded no significant improvement over the baseline, and combining it with CT provided no additional benefit.Conclusion:Temporal history dependence, rather than spatial propagation modulation, is the primary driver of prediction uncertainty in cross-subject knee contact mechanics. Explicitly encoding short-horizon driver sequences enables the surrogate model to recover implicit phase information, thereby achieving superior fidelity in peak-stress capture and high-risk localization compared to purely state-based approaches.
Read the full abstract ▾
Machine Learning
Deep Exploration of Epoch-wise Double Descent in Noisy Data: Signal Separation, Large Activation, and Benign Overfitting
Tomoki Kubo, Ryuken Uda and Yusuke Iida
Deep double descent is one of the key phenomena underlying the generalization capability of deep learning models. In this study, epoch-wise double descent, which is delayed generalization following overfitting, was empirically investigated by focusing on the
evolution of internal structures. Fully connected neural networks of three different sizes were trained on the CIFAR-10
dataset with 30% label noise. By decomposing the loss curves into signal contributions from clean and noisy training data, the epoch-wise evolutions of internal signals were analyzed separately. Three main findings were obtained from this analysis. First, the model achieved strong re-generalization on test data even after perfectly fitting noisy training data during the double descent phase, corresponding to a "benign overfitting" state. Second, noisy data were learned after clean data, and as learning progressed, their corresponding internal activations became increasingly separated in outer layers; this enabled the model to overfit only noisy data. Third, a single, very large activation emerged in the shallow layer across all models; this phenomenon is referred as "outliers," "massive activa-tions," and "super activations" in recent large
language models and evolves with re-generalization. The magnitude of large activation correlated with input patterns but not with output patterns. These empirical findings directly link the recent key phenomena of "deep double descent," "benign overfitting," and "large activation", and support the proposal of a novel scenario for understanding deep double descent.
Read the full abstract ▾
Machine Learning
ORBIT: On-policy Exploration-Exploitation for Controllable Multi-Budget Reasoning
Kun Liang, Clive Bai, Xin Xu et al. (+5 more)
Recent Large
Reasoning Models (LRMs) achieve strong performance by leveraging long-form Chain-of-Thought (CoT)
reasoning, but uniformly applying overlong
reasoning at inference time incurs substantial and often unnecessary computational cost. To address this, prior work explores various strategies to infer an appropriate
reasoning budget from the input. However, such approaches are unreliable in the worst case, as estimating the minimal required
reasoning effort is fundamentally difficult, and they implicitly fix the trade-off between
reasoning cost and accuracy during training, limiting flexibility under varying deployment scenarios. Motivated by these limitations, we propose ORBIT, a controllable multi-budget
reasoning framework with well-separated
reasoning modes triggered by input. ORBIT employs multi-stage reinforcement learning to discover Pareto-optimal
reasoning behaviors at each effort, followed by on-policy
distillation to fuse these behaviors into a single unified model. Experiments show that ORBIT achieves (1) controllable
reasoning behavior over multiple modes, (2) competitive
reasoning density within each mode, and (3) integration of these frontier policies into a single unified student model while preserving clear mode separation and high per-mode performance.
Read the full abstract ▾
All authors Kun Liang, Clive Bai, Xin Xu, Chenming Tang, Sanwoo Lee, Weijie Liu, Saiyong Yang, Yunfang Wu
Machine Learning
Demystifying the Slash Pattern in Attention: The Role of RoPE
Yuan Cheng, Fengzhuo Zhang, Yunlong Hou et al. (+5 more)
Large
Language Models (LLMs) often exhibit slash
attention patterns, where
attention scores concentrate along the $\Delta$-th sub-diagonal for some offset $\Delta$. These patterns play a key role in passing information across
tokens. But why do they emerge? In this paper, we demystify the emergence of these Slash-Dominant Heads (SDHs) from both empirical and theoretical perspectives. First, by analyzing open-source LLMs, we find that SDHs are intrinsic to models and generalize to out-of-distribution
prompts. To explain the intrinsic emergence, we analyze the queries, keys, and Rotary Position
Embedding (RoPE), which jointly determine
attention scores. Our empirical analysis reveals two characteristic conditions of SDHs: (1) Queries and keys are almost rank-one, and (2) RoPE is dominated by medium- and high-frequency components. Under these conditions, queries and keys are nearly identical across
tokens, and interactions between medium- and high-frequency components of RoPE give rise to SDHs. Beyond empirical evidence, we theoretically show that these conditions are sufficient to ensure the emergence of SDHs by formalizing them as our modeling assumptions. Particularly, we analyze the training dynamics of a shallow
Transformer equipped with RoPE under these conditions, and prove that models trained via gradient descent exhibit SDHs. The SDHs generalize to out-of-distribution
prompts.
Read the full abstract ▾
All authors Yuan Cheng, Fengzhuo Zhang, Yunlong Hou, Cunxiao Du, Chao Du, Tianyu Pang, Aixin Sun, Zhuoran Yang
Machine Learning · Cross-listed from cs.IR
AgriLens: Semantic Retrieval in Agricultural Texts Using Topic Modeling and Language Models
Heba Shakeel, Tanvir Ahmad, Tanya Liyaqat et al. (+1 more)
As the volume of unstructured text continues to grow across domains, there is an urgent need for scalable methods that enable interpretable organization, summarization, and
retrieval of information. This work presents a unified framework for interpretable topic modeling, zero-shot topic labeling, and topic-guided semantic
retrieval over large agricultural text corpora. Leveraging BERTopic, we extract semantically coherent topics. Each topic is converted into a structured
prompt, enabling a
language model to generate meaningful topic labels and summaries in a zero-shot manner. Querying and document exploration are supported via dense
embeddings and vector search, while a dedicated evaluation module assesses topical coherence and bias. This framework supports scalable and interpretable information access in specialized domains where labeled data is limited.
Read the full abstract ▾
All authors Heba Shakeel, Tanvir Ahmad, Tanya Liyaqat, Chandni Saxena
Machine Learning · Cross-listed from cs.CV
One-Shot Identification with Different Neural Network Approaches
Janis Mohr and Jörg Frochte
Convolutional neural networks (CNNs) have been widely used in the computer
vision community, significantly improving the state-of-the-art. But learning good features often is computationally expensive in machine learning settings and is especially difficult when there is a lack of data. One-shot learning is one such area where only limited data is available. In one-shot learning, predictions have to be made after seeing only one example from one class, which requires special techniques. In this paper we explore different approaches to one-shot identification tasks in different domains including an industrial application and face recognition. We use a special technique with stacked images and use siamese capsule networks. It is encouraging to see that the approach using capsule architecture achieves strong results and exceeds other techniques on a wide range of
datasets from industrial application to face recognition
benchmarks while being easy to use and optimise.
Read the full abstract ▾
Machine Learning
A Usable GAN-Based Tool for Synthetic ECG Generation in Cardiac Amyloidosis Research
Francesco Speziale, Ugo Lomoio, Fabiola Boccuto et al. (+2 more)
Cardiac amyloidosis (CA) is a rare and underdiagnosed infiltrative cardiomyopathy, and available
datasets for machine-learning models are typically small, imbalanced and heterogeneous. This paper presents a Generative
Adversarial Network (GAN) and a graphical command-line interface for generating realistic synthetic electrocardiogram (ECG) beats to support early diagnosis and patient stratification in CA. The tool is designed for usability, allowing clinical researchers to train class-specific generators once and then interactively produce large volumes of labelled synthetic beats that preserve the distribution of minority classes.
Read the full abstract ▾
All authors Francesco Speziale, Ugo Lomoio, Fabiola Boccuto, Pierangelo Veltri, Pietro Hiram Guzzi
Machine Learning
On Evaluation of Unsupervised Feature Selection for Pattern Classification
Gyu-Il Kim, Dae-Won Kim and Jaesung Lee
Unsupervised feature selection aims to identify a compact subset of features that captures the intrinsic structure of data without supervised label. Most existing studies evaluate the performance of methods using the single-label
dataset that can be instantiated by selecting a label from multi-label data while maintaining the original features. Because the chosen label can vary arbitrarily depending on the experimental setting, the superiority among compared methods can be changed with regard to which label happens to be selected. Thus, evaluating unsupervised feature selection methods based solely on single-label accuracy is unreasonable for assessing their true discriminative ability. This study revisits this evaluation paradigm by adopting a multi-label classification framework. Experiments on 21 multi-label
datasets using several representative methods demonstrate that performance rankings differ markedly from those reported under single-label settings, suggesting the possibility of multi-label evaluation settings for fair and reliable comparison of unsupervised feature selection methods.
Read the full abstract ▾