Machine Learning · Cross-listed from cs.CV
Mohit Talreja, Joshua Diao, Jim Thannikary James et al. (+6 more)
Vision Language Models (VLMs) are good at recognizing the global location of a photograph – their geolocation prediction accuracy rivals the best human experts. But many VLMs are startlingly bad at explaining which image evidence led to their prediction, even when their location prediction is correct. In this paper, we introduce GeoRC, the first
benchmark for geolocation
reasoning chains sourced directly from Champion-tier GeoGuessr experts, including the reigning world champion. This
benchmark consists of 800 “ground truth”
reasoning chains across 500 query scenes from GeoGuessr maps, with expert chains addressing hundreds of different discriminative attributes, such as soil properties, architecture, and license plate shapes. We evaluate LLM-as-a-judge and VLM-as-a-judge strategies for scoring VLM-generated
reasoning chains against our expert
reasoning chains and find that Qwen 3 LLM-as-a-judge correlates best with human-expert scoring. Our
benchmark reveals that while large, closed-source VLMs such as Gemini and GPT 5 rival human experts at predicting locations, they still lag behind human experts when it comes to producing auditable
reasoning chains. Small open-weight VLMs such as Llama and Qwen catastrophically fail on our
benchmark – they perform only slightly better than a baseline in which an LLM hallucinates a
reasoning chain with oracle knowledge of the photo location but no visual information at all. We believe the gap between human experts and VLMs on this task points to VLM limitations at extracting fine-grained visual attributes from high resolution images. We open source our
benchmark for the community to use.
Read the full abstract ▾
All authors Mohit Talreja, Joshua Diao, Jim Thannikary James, Radu Casapu, Tejas Santanam, Ethan Mendes, Alan Ritter, Wei Xu, James Hays
Machine Learning
Model-Free Neural Filtering: A Comparison with Classical Filters in Nonlinear Systems
Zhuochen Liu, Hans Walker and Rahul Jain
Neural network models are increasingly used for state estimation in control and decision-making, yet it remains unclear to what extent they behave as principled filters in nonlinear dynamical systems. Unlike classical filters, which rely on explicit dynamics and noise models, neural
estimators can be trained purely from data. We present a systematic comparison between model-free neural
estimators and classical filtering methods across multiple nonlinear scenarios. On the neural side, we evaluate
Transformer-based models, recurrent neural networks, and state-space models; on the classical side, we compare against particle filters and nonlinear Kalman filters. Results show that structured state-space models (SSMs), in particular Mamba and Mamba-2, are consistently strong among neural
estimators. They approach strong classical filters in several nonlinear systems and outperform weaker classical baselines without access to system models, while the evaluated neural implementations achieve substantially higher inference throughput on the tested hardware. Accurate model-based filters can still dominate when their assumptions are well matched. We attribute the relative strength of SSMs to filtering-aligned inductive bias: recursive latent-state updates make them structurally closer to classical filters under fixed parameter budgets, finite data, and long-horizon evaluation.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.SD
Music Plagiarism Detection: Problem Formulation and a Segment-based Solution
Seonghyeon Go and Yumin Kim
Recently, the problem of music plagiarism has emerged as an even more pressing social issue. As music information
retrieval research advances, there is a growing effort to address issues related to music plagiarism. However, many studies, including our previous work, have conducted research without clearly defining what the music plagiarism detection task actually involves. This lack of a clear definition has slowed research progress and made it hard to apply results to real-world scenarios. To fix this situation, we defined how Music Plagiarism Detection is different from other MIR tasks and explained what problems need to be solved. We introduce the Similar Music Pair
dataset to support this newly defined task. In addition, we propose a method based on segment transcription as one way to solve the task. Our demo and
dataset are available at https://github.com/Mippia/ICASSP2026-MPD.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Hypersolid: Emergent Vision Representations via Short-Range Repulsion
Esteban Rodríguez-Betancourt and Edgar Casasola-Murillo
A central problem in self-supervised learning is preventing representation collapse. Most methods avoid it through global mechanisms, such as contrastive expansion, variance constraints, decorrelating dimensions, or enforcing certain output distributions. In this work, we study a different design: short-range repulsion. We introduce Hypersolid, a self-supervised objective that combines view
alignment with local collision avoidance. Our method induces a latent geometry of compact, semantically aligned neighborhoods with low anisotropy. This geometry is especially effective for unsupervised clustering and fine-grained separation, although it comes at the cost of weaker transferability.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
Position: Certifiable State Integrity Should Be Built from Local Validity, Not Global Scale
Enzo Nicolás Spotorno, Joao R. Campos and Antônio Augusto Medeiros Fröhlich
Breakthroughs in
language and
vision have motivated increasingly general foundation models for time series and physical dynamics, where evidence is promising but less mature. In safety-critical Cyber-Physical Systems (CPS), globally parameterized dynamics models must remain valid through regime shifts, degradation, and lifecycle updates while supporting traceable assurance evidence. We argue that shared global parameters couple adaptation, frequency-sensitive fault representation, and verification evidence in ways that obstruct retained regime knowledge and bounded analysis. For systems whose operating conditions can be covered by a finite set of locally valid models, assurance evidence should be organized around those bounded models and their explicit composition, not around a single globally adapted model; increasing scale cannot supply explicit validity boundaries. HYDRA is a reference architecture for this position: frozen local specialists, runtime arbitration, and explicit coverage-gap monitoring. It illustrates how modular auditability and formal analysis of bounded components support Certifiable State Integrity, while whole-system certification, switching guarantees, and empirical validation remain separate obligations.
Read the full abstract ▾
Machine Learning
Conditional Generative Framework with Peak-Aware Attention for Robust Chemical Detection under Interferences
Namkyung Yoon, Sanghong Kim and Hwangnam Kim
Gas chromatography-mass spectrometry (GC-MS) is a widely used analytical method for chemical substance detection, but measurement reliability tends to deteriorate in the presence of interfering substances. In particular, interfering substances cause nonspecific peaks, residence time shifts, and increased background noise, resulting in reduced sensitivity and false alarms. To overcome these challenges, in this paper, we propose an artificial intelligence discrimination framework based on a peak-aware conditional generative model to improve the reliability of GC-MS measurements under interference conditions. The framework is learned with a novel peak-aware mechanism that highlights the characteristic peaks of GC-MS data, allowing it to generate important spectral features more faithfully. In addition, chemical and solvent information is encoded in a latent vector embedded with it, allowing a conditional generative
adversarial neural network (CGAN) to generate a synthetic GC-MS signal consistent with the experimental conditions. This generates an experimental
dataset that assumes indirect substance situations in chemical substance data, where acquisition is limited without conducting real experiments. These data are used for the learning of AI-based GC-MS discrimination models to help in accurate chemical substance discrimination. We conduct various quantitative and qualitative evaluations of the generated simulated data to verify the validity of the proposed framework. We also verify how the generative model improves the performance of the AI discrimination framework. Representatively, the proposed method is shown to consistently achieve cosine similarity and Pearson correlation coefficient values above 0.9 while preserving peak number diversity and reducing false alarms in the discrimination model.
Read the full abstract ▾
Machine Learning
Less Noise, More Voice: Reinforcement Learning for Reasoning via Instruction Purification
Yiju Guo, Tianyi Hu, Zexu Sun et al. (+1 more)
Reinforcement Learning with Verifiable Rewards (RLVR) has advanced LLM
reasoning, but remains constrained by inefficient exploration under limited rollout budgets, leading to low sampling success and unstable training in complex tasks. We find that many exploration failures arise not from problem difficulty, but from a small number of
prompt tokens that introduce interference. Building on this insight, we propose the Less Noise Sampling Framework (LENS), which first
prompts by identifying and removing interference
tokens. then transfers successful rollouts from the purification process to supervise policy optimization on the original noisy
prompts, enabling the model to learn to ignore interference in the real-world, noisy prompting settings. Experimental results show that LENS significantly outperforms GRPO, delivering higher performance and faster convergence, with a 3.88% average gain and over 1.6$\times$ speedup on math
reasoning, and a 1.83% gain on scientific and general
reasoning. Our work highlights the critical role of
pruning interference
tokens in improving rollout efficiency, offering a new perspective for RLVR research.
Read the full abstract ▾
All authors Yiju Guo, Tianyi Hu, Zexu Sun, Yankai Lin
Machine Learning · Cross-listed from math.OC
Solving the Offline and Online Min-Max Problem of Non-smooth Submodular-Concave Functions: A Zeroth-Order Approach
Amir Ali Farzin, Yuen-Man Pun, Philipp Braun et al. (+2 more)
We consider max-min and min-max problems with objective functions that are possibly non-smooth, submodular with respect to the minimiser and concave with respect to the maximiser. We investigate the performance of a zeroth-order method applied to this problem. The method is based on the subgradient of the Lovász extension of the objective function with respect to the minimiser and based on Gaussian smoothing to estimate the smoothed function gradient with respect to the maximiser. In expectation sense, we prove the convergence of the
algorithm to an $\epsilon$-saddle point in the offline case. Moreover, we show that, in the expectation sense, in the online setting, the
algorithm achieves $O(\sqrt{N(1+\bar{P}_N)})$ online duality gap, where $N$ is the number of iterations and $\bar{P}_N$ is the path length of the sequence of optimal decisions. The complexity analysis and hyperparameter selection are presented for all the cases. The theoretical results are illustrated via numerical examples.
Read the full abstract ▾
All authors Amir Ali Farzin, Yuen-Man Pun, Philipp Braun, Tyler Summers, Iman Shames
Machine Learning
Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks
Luwei Sun, Dongrui Shen, Feng Chuanwen et al. (+3 more)
Motivated by challenges in conditional generative modeling, where the target conditional density takes the form of a ratio f1 over f2, this paper develops a theoretical framework for approximating such ratio-type functionals. Here, f1 and f2 are kernel-based marginal densities that capture structured interactions, a setting central to
diffusion-based generative models. We provide a concise proof for approximating these ratio-type functionals using deep neural networks with the SignReLU activation function, leveraging the activation's piecewise structure. Under standard regularity assumptions, we establish L^p(Omega) approximation bounds and convergence rates. Specializing to Denoising
Diffusion Probabilistic Models (DDPMs), we construct a SignReLU-based neural
estimator for the reverse process and derive bounds on the excess Kullback-Leibler (KL) risk between the generated and true data distributions. Our analysis decomposes this excess risk into approximation and estimation error components. These results provide generalization guarantees for finite-sample training of
diffusion-based generative models.
Read the full abstract ▾
All authors Luwei Sun, Dongrui Shen, Feng Chuanwen, Jianfe Li, Yulong Zhao, Han Feng
Machine Learning · Cross-listed from cs.DS
Characterizing the Effect of Noise in Language Generation in the Limit
Aaron Li and Ian Zhang
Kleinberg and Mullainathan recently proposed a formal framework for studying the phenomenon of
language generation, called
language generation in the limit. In this model, an adversary gives an enumeration of example strings from an unknown target
language, and the
algorithm is tasked with correctly generating unseen strings from the target
language within finite time. Refined notions of non-uniform and uniform generation were later introduced by Li, Raman, and Tewari (2025), and a noisy model was introduced by Raman and Raman (2025), which allows the adversary to insert extraneous strings. A natural question in the noisy model is to quantify the effect of noise, by studying the impact of each additional extraneous string. We show two complementary results in this setting. We first show that for both uniform and non-uniform generation, a single noisy string strictly reduces the set of collections that can be generated, thus answering an open question in Raman and Raman (2025). Then, we show for both uniform and non-uniform generation that generation with a single noisy string is equivalent to generation with any finite amount of noise, sharply contrasting with the strict hierarchy for noisy generation in the limit shown by Bai, Panigrahi, and Zhang (2026). Finally, we leverage our previous results to provide the first known characterization for non-uniform noise-dependent generatability.
Read the full abstract ▾
Machine Learning
PHDME: Physics-Informed Diffusion Models without Explicit Governing Equations
Kaiyuan Tan, Kendra Givens, Peilun Li et al. (+1 more)
Diffusion models provide expressive priors for
forecasting trajectories of dynamical systems, but are typically unreliable in the sparse data regime. Physics-informed machine learning (PIML) improves reliability in such settings; however, most methods require explicit governing equations during training, which are often only partially known due to complex and nonlinear dynamics. We introduce PHDME, a port-
Hamiltonian diffusion framework designed for sparse observations and incomplete physics. PHDME leverages port-
Hamiltonian structural prior but does not require full knowledge of the closed-form governing equations. Our approach first trains a Gaussian process distributed Port-
Hamiltonian system (GP-dPHS) on limited observations to capture an energy-based representation of the dynamics. The GP-dPHS is then used to generate a physically consistent artificial
dataset for
diffusion training, and to inform the
diffusion model with a structured physics residual loss. After training, the
diffusion model acts as an amortized sampler and forecaster for fast trajectory generation. Finally, we apply split conformal calibration to provide uncertainty statements for the generated predictions. Experiments on PDE
benchmarks and a real-world spring system show improved accuracy and physical consistency under data scarcity.
Read the full abstract ▾
All authors Kaiyuan Tan, Kendra Givens, Peilun Li, Thomas Beckers
Machine Learning · Cross-listed from astro-ph.HE
Data-Driven Generation of Neutron Star Equations of State Using Variational Autoencoders
Alex Ross, Tianqi Zhao and Sanjay Reddy
We develop a machine learning model based on a structured variational autoencoder (VAE) framework to reconstruct and generate
neutron star (NS) equations of state (EOS). The VAE consists of an encoder network that maps high-dimensional EOS data into a lower-dimensional latent space and a decoder network that reconstructs the full EOS from the latent representation. The latent space includes supervised NS observables derived from the training EOS data, as well as latent random variables corresponding to additional unspecified EOS features learned automatically. Sampling the latent space enables the generation of new, causal, and stable EOS models that satisfy astronomical constraints on the supervised NS observables, while allowing
Bayesian inference of the EOS incorporating additional multimessenger data, including gravitational waves from LIGO/Virgo and mass and radius measurements of
pulsars. Based on a VAE trained on a Skyrme EOS
dataset, we find that a latent space with two supervised NS observables, the maximum mass $(M_{\max})$ and the canonical radius $(R_{1.4})$, together with one latent random variable controlling the EOS near the crust–core transition, can already reconstruct Skyrme EOSs with high fidelity, achieving mean absolute percentage errors of approximately $(0.15\%)$ for $(M_{\max})$ and $(R_{1.4})$ derived from the decoder-reconstructed EOS.
Read the full abstract ▾
Machine Learning
Soft Quantization: Model Compression Via Weight Coupling
Daniel T. Bernstein, Luca Di Carlo and David Schwab
We show that introducing short-range attractive couplings between the weights of a neural network during training provides a novel avenue for model quantization. These couplings rapidly induce the discretization of a model's weight distribution, and they do so in a mixed-precision manner despite only relying on two additional hyperparameters. We demonstrate that, within an appropriate range of hyperparameters, our "soft quantization” scheme outperforms histogram-equalized post-training quantization on ResNet-20/CIFAR-10. Soft quantization provides both a new pipeline for the flexible compression of machine learning models and a new tool for investigating the trade-off between compression and generalization in high-dimensional loss landscapes.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
A Flexible Empirical Bayes Approach to Generalized Linear Models, with Applications to Sparse Logistic Regression
Dongyue Xie and Matthew Stephens
We introduce a flexible empirical Bayes approach for fitting
Bayesian generalized linear models. Specifically, we adopt a novel mean-field variational inference (VI) method and the prior is estimated within the VI
algorithm, making the method tuning-free. Unlike traditional VI methods that optimize the posterior density function, our approach directly optimizes the posterior mean and prior parameters. This formulation reduces the number of parameters to optimize and enables the use of scalable
algorithms such as L-BFGS and
stochastic gradient descent. Furthermore, our method automatically determines the optimal posterior based on the prior and likelihood, distinguishing it from existing VI methods that often assume a Gaussian variational. Our approach represents a unified framework applicable to a wide range of exponential family distributions, removing the need to develop unique VI methods for each combination of likelihood and prior distributions. We apply the framework to solve sparse logistic regression and demonstrate the superior predictive performance of our method in extensive numerical studies, by comparing it to prevalent sparse logistic regression approaches.
Read the full abstract ▾
Machine Learning
Temporal Context and Architecture: A Benchmark for Naturalistic EEG Decoding
Mehmet Ergezer
We study how model architecture and temporal context interact in naturalistic
EEG decoding. Using the HBN movie-watching
dataset, we
benchmark five architectures, CNN, LSTM, a stabilized
Transformer (EEGXF), S4, and S5, on a 4-class task across segment lengths from 8s to 128s. Accuracy improves with longer context: at 64s, S5 reaches 98.7%+/-0.6 and CNN 98.3%+/-0.3, while S5 uses ~20x fewer parameters than CNN. To probe real-world robustness, we evaluate zero-shot cross-frequency shifts, cross-task OOD inputs, and leave-one-subject-out generalization. S5 achieves stronger cross-subject accuracy but makes over-confident errors on OOD tasks; EEGXF is more conservative and stable under frequency shifts, though less calibrated in-distribution. These results reveal a practical efficiency-robustness trade-off: S5 for parameter-efficient peak accuracy; EEGXF when robustness and conservative uncertainty are critical.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Scaling Reasoning Hop Exposes Weaknesses: Demystifying and Improving Hop Generalization in Large Language Models
Zhaoyi Li, Jiatong Li, Gangwei Jiang et al. (+3 more)
Chain-of-thought (CoT)
reasoning has become the standard paradigm for enabling Large
Language Models (LLMs) to solve complex problems. However, recent studies reveal a sharp performance drop in
reasoning hop generalization scenarios, where the required number of
reasoning steps exceeds training distributions while the underlying
algorithm remains unchanged. The internal mechanisms driving this failure remain poorly understood. In this work, we conduct a systematic study on tasks from multiple domains, and find that errors concentrate at
token positions of a few critical error types, rather than being uniformly distributed. Closer inspection reveals that these
token-level erroneous predictions stem from internal competition mechanisms: certain
attention heads, termed erroneous processing heads (ep heads), tip the balance by amplifying incorrect
reasoning trajectories while suppressing correct ones. Notably, removing individual ep heads during inference can often restore the correct predictions. Motivated by these insights, we propose test-time correction of
reasoning, a lightweight intervention method that dynamically identifies and deactivates ep heads in the
reasoning process. Extensive experiments across different tasks and LLMs show that it consistently improves
reasoning hop generalization, highlighting both its effectiveness and potential.
Read the full abstract ▾
All authors Zhaoyi Li, Jiatong Li, Gangwei Jiang, Linqi Song, Defu Lian, Ying Wei
Machine Learning
A Sheaf-Theoretic and Topological Perspective on Complex Network Modeling and Attention Mechanisms in Graph Neural Models
Chuan-Shen Hu
Combinatorial and topological structures, such as graphs, simplicial complexes, and
cell complexes, form the foundation of
geometric and topological deep learning (GDL and TDL) architectures. These models aggregate signals over such domains, integrate local features, and generate representations for diverse real-world applications. However, the distribution and
diffusion behavior of GDL and TDL features during training remains an open and underexplored problem. Motivated by this gap, we introduce a cellular sheaf theoretic framework for modeling and analyzing the local consistency and harmonicity of node features and edge weights in graph-based architectures. By tracking local feature
alignments and agreements through sheaf structures, the framework offers a topological perspective on feature
diffusion and aggregation. Furthermore, a multiscale extension inspired by topological data analysis (TDA) is proposed to capture hierarchical feature interactions in graph models. This approach enables a joint characterization of GDL and TDL architectures based on their underlying
geometric and topological structures and the learned signals defined on them, providing insights for future studies on conventional tasks such as node classification, substructure detection, and community detection.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Scaling Embeddings Outperforms Scaling Experts in Language Models
Hong Liu, Jiaqi Zhang, Chao Wang et al. (+13 more)
While Mixture-of-Experts (MoE) architectures have become the standard for sparsity scaling in large
language models, they increasingly face diminishing returns and system-level bottlenecks. In this work, we explore
embedding scaling as a potent, orthogonal dimension for scaling sparsity. Through a comprehensive analysis and experiments, we identify specific regimes where
embedding scaling achieves a superior Pareto frontier compared to expert scaling. We systematically characterize the critical architectural factors governing this efficacy – ranging from parameter budgeting to the interplay with model width and depth. Moreover, by integrating tailored system optimizations and speculative decoding, we effectively convert this sparsity into tangible inference speedups. Guided by these insights, we introduce LongCat-Flash-Lite, a 68.5B parameter model with ~3B activated trained from scratch. Despite allocating over 30B parameters to
embeddings, LongCat-Flash-Lite not only surpasses parameter-equivalent MoE baselines but also exhibits exceptional competitiveness against existing models of comparable scale, particularly in agentic and coding domains.
Read the full abstract ▾
All authors Hong Liu, Jiaqi Zhang, Chao Wang, Xing Hu, Linkun Lyu, Jiaqi Sun, Xurui Yang, Bo Wang, Fengcun Li, Yulei Qian, Lingtong Si, Yerui Sun, Rumei Li, Peng Pei, Yuchen Xie, Xunliang Cai
Machine Learning
Rethinking Self-Training Based Cross-Subject Domain Adaptation for SSVEP Classification
Weiguang Wang, Yong Liu, Yingjie Gao et al. (+1 more)
Steady-state visually evoked potentials (SSVEP)-based
brain-computer interfaces (BCIs) are widely used due to their high signal-to-noise ratio and user-friendliness. Accurate decoding of SSVEP signals is crucial for interpreting user intentions in BCI applications. However, signal variability across subjects and the costly user-specific annotation limit recognition performance. Therefore, we propose a novel cross-subject domain adaptation method built upon the self-training paradigm. Specifically, a Filter-Bank Euclidean
Alignment (FBEA) strategy is designed to exploit frequency information from SSVEP filter banks. Then, we propose a Cross-Subject Self-Training (CSST) framework consisting of two stages: Pre-Training with
Adversarial Learning (PTAL), which aligns the source and target distributions, and Dual-Ensemble Self-Training (DEST), which refines pseudo-label quality. Moreover, we introduce a Time-Frequency Augmented Contrastive Learning (TFA-CL) module to enhance feature discriminability across multiple augmented views. Extensive experiments on the
Benchmark and BETA
datasets demonstrate that our approach achieves state-of-the-art performance across varying signal lengths, highlighting its superiority.
Read the full abstract ▾
All authors Weiguang Wang, Yong Liu, Yingjie Gao, Guangyuan Xu
Machine Learning · Cross-listed from stat.ML
Provably Reliable Classifier Guidance via Cross-Entropy Control
Sharan Sahu, Arisina Banerjee and Yuchen Wu
Classifier-guided
diffusion models generate conditional samples by augmenting the reverse-time score with the gradient of the log-probability predicted by a probabilistic classifier. In practice, this classifier is usually obtained by minimizing an empirical loss function. While existing statistical theory guarantees good generalization performance when the sample size is sufficiently large, it remains unclear whether such training yields an effective guidance mechanism. We study this question in the context of cross-
entropy loss, which is widely used for classifier training. Under mild smoothness assumptions on the classifier, we show that controlling the cross-
entropy at each
diffusion model step is sufficient to control the corresponding guidance error. In particular, probabilistic classifiers achieving conditional KL divergence $\varepsilon^2$ induce guidance vectors with mean squared error $\widetilde O(d \varepsilon )$, up to constant and logarithmic factors. Our result yields an upper bound on the sampling error of classifier-guided
diffusion models and bears resemblance to a reverse log-Sobolev–type inequality. To the best of our knowledge, this is the first result that quantitatively links classifier training to guidance
alignment in
diffusion models, providing both a theoretical explanation for the empirical success of classifier guidance, and principled guidelines for selecting classifiers that induce effective guidance.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.DC
ZipMoE: Efficient On-Device MoE Serving via Lossless Compression and Cache-Affinity Scheduling
Yuchen Yang, Yaru Zhao, Pu Yang et al. (+2 more)
While Mixture-of-Experts (MoE) architectures substantially bolster the expressive power of large-
language models, their prohibitive
memory footprint severely impedes the practical deployment on resource-constrained edge devices, especially when model behavior must be preserved without relying on lossy quantization. In this paper, we present ZipMoE, an efficient and semantically lossless on-device MoE serving system. ZipMoE exploits the synergy between the hardware properties of edge devices and the statistical redundancy inherent to MoE parameters via a caching-scheduling co-design with provable performance guarantee. Fundamentally, our design shifts the paradigm of on-device MoE inference from an I/O-bound bottleneck to a compute-centric workflow that enables efficient parallelization. We implement a prototype of ZipMoE and conduct extensive experiments on representative edge computing platforms using popular open-source MoE models and real-world workloads. Our evaluation reveals that ZipMoE achieves up to $72.77\%$ inference latency reduction and up to $6.76\times$ higher throughput than the state-of-the-art systems.Our code is available at: https://github.com/npnothard/ZipMoE-ICML26.
Read the full abstract ▾
All authors Yuchen Yang, Yaru Zhao, Pu Yang, Shaowei Wang, Zhi-Hua Zhou
Machine Learning · Cross-listed from cs.CV
FRISM: Fine-Grained Reasoning Injection via Subspace-Level Model Merging for Vision-Language Models
Chenyu Huang, Peng Ye, Xudong Tan et al. (+4 more)
Efficiently enhancing the
reasoning capabilities of
Vision-
Language Models (VLMs) by merging them with Large
Reasoning Models (LRMs) has emerged as a promising direction. However, existing methods typically operate at a coarse-grained layer level, which often leads to a trade-off between injecting
reasoning capabilities and preserving visual capabilities. To address this limitation, we propose FRISM (Fine-grained
Reasoning Injection via Subspace-level model Merging), a fine-grained
reasoning injection framework based on subspace-level model merging. Observing that different SVD subspaces contribute differently to
reasoning and perception, FRISM decomposes LRM task vectors via Singular Value Decomposition (SVD) and adaptively tunes the scaling coefficients of each subspace through learning to realize fine-grained
reasoning injection. Furthermore, we introduce a label-free self-
distillation learning strategy with dual-objective optimization using common
vision-
language perception
datasets. Extensive experiments demonstrate that FRISM effectively improves
reasoning capabilities while largely preserving the model's visual capabilities by consistently achieving strong performance across diverse visual-
language reasoning benchmarks.
Read the full abstract ▾
All authors Chenyu Huang, Peng Ye, Xudong Tan, Jinhan Mu, Shenghe Zheng, Li Shen, Tao Chen
Machine Learning · Cross-listed from cs.AI
Sycophantic Anchors: Localizing and Quantifying User Agreement in Reasoning Models
Jacek Duszenko
Reasoning models frequently agree with incorrect user suggestions – a behavior known as sycophancy. However, it is unclear where in the
reasoning trace this agreement originates and how strong the commitment is. We introduce sycophantic anchors – sentences identified via counterfactual analysis that commit models to user agreement. Across four
reasoning models spanning three architecture families (Llama, Qwen, Falcon-hybrid) and 1.5B–8B parameters, we analyze over 200,000 counterfactual rollouts and show that linear probes reliably detect sycophantic anchors (74–85% balanced accuracy), outperforming text-only baselines at high commitment levels – confirming they capture internal states beyond surface vocabulary. Regressors further predict commitment strength from activations ($R^2$ up to 0.74). We observe a consistent asymmetry: sycophancy leaves a stronger mechanistic footprint than correct
reasoning. We also find that sycophancy builds gradually during generation rather than being determined by the
prompt. These findings enable sentence-level detection and quantification of model misalignment mid-inference.
Read the full abstract ▾
Machine Learning
Rethinking Refinement: Correcting Generative Bias without Noise Injection
Xin Peng and Ang Gao
Generative models, including
diffusion and flow-based models, often exhibit systematic biases that degrade sample quality, particularly in high-dimensional settings. We revisit refinement methods and show that effective bias correction can be achieved as a post-hoc procedure, without noise injection or multi-step resampling of the sampling process. We propose a flow-matching-based Bi-stage Flow Refinement (BFR) framework with two refinement strategies operating at different stages: latent space
alignment for approximately invertible generators and data space refinement trained with lightweight augmentations. Unlike previous refiners that perturb sampling dynamics, BFR preserves the original ODE trajectory and applies deterministic corrections to generated samples. Experiments on MNIST, CIFAR-10, and FFHQ at 256x256 resolution demonstrate consistent improvements in fidelity and coverage; notably, starting from base samples with FID 3.95, latent space refinement achieves a state-of-the-art FID of 1.46 on MNIST using only a single additional function evaluation (1-NFE), while maintaining sample diversity.
Read the full abstract ▾
Machine Learning
Flow Perturbation++: Multi-Step Unbiased Jacobian Estimation for High-Dimensional Boltzmann Sampling
Xin Peng and Ang Gao
The scalability of continuous normalizing flows (CNFs) for unbiased Boltzmann sampling remains limited in high-dimensional systems due to the cost of Jacobian-determinant evaluation, which requires $D$ backpropagation passes through the flow layers. Existing
stochastic Jacobian
estimators such as the Hutchinson trace
estimator reduce computation but introduce bias, while the recently proposed Flow Perturbation method is unbiased yet suffers from high variance. We present Flow Perturbation++, a variance-reduced extension of Flow Perturbation that discretizes the probability-flow ODE and performs unbiased stepwise Jacobian estimation at each integration step. This multi-step construction retains the unbiasedness of Flow Perturbation while achieves substantially lower
estimator variance. Integrated into a Sequential Monte Carlo framework, Flow Perturbation++ achieves significantly improved
equilibrium sampling on a 1000D Gaussian Mixture Model and the all-atom Chignolin
protein compared with Hutchinson-based and single-step Flow Perturbation baselines.
Read the full abstract ▾
Machine Learning
Breaking the Reasoning Horizon in Entity Alignment Foundation Models
Yuanning Cui, Zequn Sun, Wei Hu et al. (+2 more)
Entity
alignment (EA) is critical for knowledge graph (KG) fusion. Existing EA models lack transferability and are incapable of aligning unseen KGs without retraining. While using graph foundation models (GFMs) offer a solution, we find that directly adapting GFMs to EA remains largely ineffective. This stems from a critical "
reasoning horizon gap": unlike link prediction in GFMs, EA necessitates capturing long-range dependencies across sparse and heterogeneous KG structuresTo address this challenge, we propose a EA foundation model driven by a parallel encoding strategy. We utilize seed EA pairs as local anchors to guide the information flow, initializing and encoding two parallel streams simultaneously. This facilitates anchor-conditioned message passing and significantly shortens the inference trajectory by leveraging local structural proximity instead of global search. Additionally, we incorporate a merged relation graph to model global dependencies and a learnable interaction module for precise matching. Extensive experiments verify the effectiveness of our framework, highlighting its strong generalizability to unseen KGs.
Read the full abstract ▾
All authors Yuanning Cui, Zequn Sun, Wei Hu, Kexuan Xin, Zhangjie Fu
Machine Learning
AC2L-GAD: Active Counterfactual Contrastive Learning for Graph Anomaly Detection
Kamal Berahmand, Saman Forouzandeh, Mehrnoush Mohammadi et al. (+2 more)
Graph anomaly detection aims to identify abnormal patterns in networks, but faces significant challenges from label scarcity and extreme class imbalance. While graph contrastive learning offers a promising unsupervised solution, existing methods suffer from two critical limitations: random augmentations break semantic consistency in positive pairs, while naive negative sampling produces trivial, uninformative contrasts. We propose AC2L-GAD, an Active Counterfactual Contrastive Learning framework that addresses both limitations through principled counterfactual
reasoning. By combining information-theoretic active selection with counterfactual generation, our approach identifies structurally complex nodes and generates anomaly-preserving positive augmentations alongside normal negative counterparts that provide hard contrasts, while restricting expensive counterfactual generation to a strategically selected subset. This design reduces computational overhead by approximately 65% compared to full-graph counterfactual generation while maintaining detection quality. Experiments on nine
benchmark datasets, including real-world financial transaction graphs from GADBench, show that AC2L-GAD achieves competitive or superior performance compared to state-of-the-art baselines, with notable gains in
datasets where anomalies exhibit complex attribute-structure interactions.
Read the full abstract ▾
All authors Kamal Berahmand, Saman Forouzandeh, Mehrnoush Mohammadi, Parham Moradi, Mahdi Jalili
Machine Learning
The Powers of Precision: Structure-Informed Detection in Complex Systems – From Customer Churn to Seizure Onset
Augusto Santos, Teresa Santos, Catarina Rodrigues et al. (+1 more)
Emergent phenomena – onset of epileptic seizures, sudden customer churn, or pandemic outbreaks – often arise from hidden causal interactions in complex systems. We propose a machine learning method for their early detection that addresses a core challenge: unveiling and harnessing a system's latent causal structure despite the data-generating process being unknown and partially observed. The method learns an optimal feature representation from a one-parameter family of
estimators – powers of the empirical covariance or precision
matrix – offering a principled way to tune in to the underlying structure driving the emergence of critical events. A supervised learning module then classifies the learned representation. We prove structural consistency of the family and demonstrate the empirical soundness of our approach on seizure detection and churn prediction, attaining competitive results in both. Beyond prediction, and toward explainability, we ascertain that the optimal covariance power exhibits evidence of good identifiability while capturing structural signatures, thus reconciling predictive performance with interpretable statistical structure.
Read the full abstract ▾
All authors Augusto Santos, Teresa Santos, Catarina Rodrigues, José M. F. Moura
Machine Learning
Learning What to Recommend: Minimax Optimal Simple Regret in Logistic Bandits
Shuai Liu, Alireza Bakhtiari, Alex Ayoub et al. (+2 more)
We study
stochastic logistic bandits with $d$-dimensional action features under the simple-regret objective, where a learner uses $T$ rounds of exploration to output a single final action. The logistic structure is essential here: because the informativeness of an action depends on the local
curvature of the sigmoid, actions that are best for immediate reward need not be the most useful for identifying the best final recommendation. We show that the first-order minimax difficulty is governed by $\kappa_*$, the inverse slope of the sigmoid at the optimal action. The lower bound is realized by a shifted saturated hard family in which saturation simultaneously limits the information available about the final decision and controls the value loss from a wrong recommendation. This reveals a hard mechanism distinct from cumulative-regret constructions, even though online-to-batch reductions recover the same leading order in expectation. We then develop two
curvature-aware
algorithms: \MULog, a pure-exploration method whose final recommendation satisfies a high-probability upper bound of order $\tilde O(d/\sqrt{\kappa_* T})$, matching the lower bound up to logarithmic factors, and \THATS, a Thompson-sampling-style method that provides a computationally lighter alternative. Experiments on both hard and easy geometries support the same picture: informative low-reward actions can make instances substantially easier, and the
curvature-aware methods exploit this structure especially effectively.
Read the full abstract ▾
All authors Shuai Liu, Alireza Bakhtiari, Alex Ayoub, Botao Hao, Csaba Szepesvári
Machine Learning · Cross-listed from cs.AI
FrontierScience: Evaluating AI's Ability to Perform Expert-Level Scientific Tasks
Miles Wang, Robi Lin, Kat Hu et al. (+4 more)
We introduce FrontierScience, a
benchmark evaluating expert-level scientific
reasoning in frontier
language models. Recent model progress has nearly saturated existing science
benchmarks, which often rely on multiple-choice knowledge questions or already published information. FrontierScience addresses this gap through two complementary tracks: (1) Olympiad, consisting of international olympiad problems at the level of IPhO, IChO, and IBO, and (2) Research, consisting of PhD-level, open-ended problems representative of sub-tasks in scientific research. FrontierScience contains several hundred questions (including 160 in the open-sourced gold set) covering subfields across physics, chemistry, and biology, from
quantum electrodynamics to synthetic organic chemistry. All Olympiad problems are originally produced by international Olympiad medalists and national team coaches to ensure standards of difficulty, originality, and factuality. All Research problems are research sub-tasks written and verified by PhD scientists (doctoral candidates, postdoctoral researchers, or professors). For Research, we introduce a granular rubric-based evaluation framework to assess model capabilities throughout the process of solving a research task, rather than judging only a standalone final answer.
Read the full abstract ▾
All authors Miles Wang, Robi Lin, Kat Hu, Joy Jiao, Neil Chowdhury, Ethan Chang, Tejal Patwardhan