Machine Learning · Cross-listed from stat.ML
Robust Automatic Differentiation of Square-Root Kalman Filters via Gramian Differentials
Adrien Corenflos
Square-root Kalman filters propagate state covariances in Cholesky-factor form for numerical stability, and are a natural target for gradient-based parameter learning in state-space models. Their core operation, triangularization of a
matrix $M \in \mathbb{R}^{n \times m}$, is computed via a QR decomposition in practice, but naively differentiating through it causes two problems: the semi-orthogonal factor is non-unique when $m > n$, yielding undefined gradients; and the standard Jacobian formula involves inverses, which diverges when $M$ is rank-deficient. Both are resolved by the observation that all filter outputs relevant to learning depend on the input
matrix only through the Gramian $MM^\top$, so the composite loss is smooth in $M$ even where the triangularization is not. We derive a closed-form chain-rule directly from the differential of this Gramian identity, prove it exact for the Kalman log-marginal likelihood and filtered moments, and extend it to rank-deficient inputs via a two-component decomposition: a column-space term based on the Moore–Penrose pseudoinverse, and a null-space correction for perturbations outside the column space of $M$.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Holographic Invariant Storage: Design-Time Safety Contracts via Vector Symbolic Architectures
Arsenios Scrivens
We introduce
Holographic Invariant Storage (HIS), a protocol that assembles known properties of bipolar Vector Symbolic Architectures into a design-time safety contract for LLM context-drift mitigation. The contract provides three closed-form guarantees evaluable before deployment: single-signal recovery fidelity converging to $1/\sqrt{2} \approx 0.707$ (regardless of noise depth or content), continuous-noise robustness $2\Phi(1/\sigma) - 1$, and multi-signal capacity degradation $\approx\sqrt{1/(K+1)}$. These bounds, validated by Monte Carlo simulation ($n = 1{,}000$), enable a systems engineer to budget recovery fidelity and codebook capacity at design time – a property no timer or
embedding-distance metric provides. A pilot behavioral experiment (four LLMs, 2B–7B, 720 trials) confirms that safety re-injection improves adherence at the 2B scale; full results are in an appendix.
Read the full abstract ▾
Machine Learning
Ghosts of Softmax: Complex Singularities That Limit Safe Step Sizes in Cross-Entropy
Piyush Sao
Optimization analyses for cross-
entropy training rely on local Taylor models of the loss to predict whether a proposed step will decrease the objective. These surrogates are reliable only inside the Taylor convergence radius of the true loss along the update direction. That radius is set not by real-line
curvature alone but by the nearest complex singularity. For cross-
entropy, the softmax partition function $F=\sum_j \exp(z_j)$ has complex zeros – “ghosts of softmax” – that induce logarithmic singularities in the loss and cap this radius. To make this geometry usable, we derive closed-form expressions under logit linearization along the proposed update direction. In the binary case, the exact radius is $\rho^*=\sqrt{\delta^2+ \pi^2}/\Delta_a$. In the multiclass case, we obtain the lower bound $\rho_a=\pi/\Delta_a$, where $\Delta_a=\max_k a_k-\min_k a_k$ is the spread of directional logit derivatives $a_k=\nabla z_k\cdot v$. This bound costs one Jacobian-vector product and reveals what makes a step fragile: samples that are both near a decision flip and highly sensitive to the proposed direction tighten the radius. The normalized step size $r=\tau/\rho_a$ separates safe from dangerous updates. Across six tested architectures and multiple step directions, no model fails for $r<1$, yet collapse appears once $r\ge 1$. Temperature scaling confirms the mechanism: normalizing by $\rho_a$ shrinks the onset-threshold spread from standard deviation $0.992$ to $0.164$. A controller that enforces $\tau\le\rho_a$ survives learning-rate spikes up to $10{,} 000\times$ in our tests, where gradient clipping still collapses. Together, these results identify a
geometric constraint on cross-
entropy optimization that operates through Taylor convergence rather than Hessian
curvature.
Read the full abstract ▾
Machine Learning
Probabilistic Gaussian Homotopy: A Probability-Space Continuation Framework for Nonconvex Optimization
Eshed Gal, Samy Wu Fung and Eldad Haber
We introduce Probabilistic Gaussian Homotopy (PGH), a probability-space continuation framework for nonconvex optimization. Unlike classical Gaussian homotopy, which smooths the objective and uniformly averages gradients, PGH deforms the associated Boltzmann distribution and induces Boltzmann-weighted aggregation of perturbed gradients, which exponentially biases descent directions toward low-energy regions. We show that PGH corresponds to a log-sum-exp (soft-min) homotopy that smooths a nonconvex objective at scale $\lambda>0$ and recovers the original objective as $\lambda\to 0$, yielding a posterior-mean generalization of the Moreau envelope, and we derive a dynamical system governing minimizer
evolution along an annealed homotopy path. This establishes a principled connection between Gaussian continuation,
Bayesian denoising, and
diffusion-style smoothing. We further propose Probabilistic Gaussian Homotopy Optimization (PGHO), a practical
stochastic algorithm based on Monte Carlo gradient estimation, and demonstrate strong performance on high-dimensional nonconvex
benchmarks and sparse recovery problems where classical gradient methods and objective-space smoothing frequently fail.
Read the full abstract ▾
Machine Learning
Exploring label correlations using decision templates for ensemble of classifier chains
Victor F. Rocha, Alexandre L. Rodrigues, Thiago Oliveira-Santos et al. (+1 more)
The use of ensemble-based multi-label methods has been shown to be effective in improving multi-label classification results. One of the most widely used ensemble-based multi-label classifiers is Ensemble of Classifier Chains. Decision templates for Ensemble of Classifier Chains (DTECC) is a fusion scheme based on Decision Templates that combines the predictions of Ensemble of Classifier Chains using information from the decision profile for each label, without considering information about other labels that might contribute to the classified result. Based on DTECC, this work proposes the Unconditionally Dependent Decision Templates for Ensemble of Classifier Chains (UDDTECC) method, a classifier fusion method that seeks to exploit correlations between labels in the fusion process. In this way, the classification of each label in the problem takes into account the label values that are considered conditionally dependent and that can lead to an improvement in the classification performance. The proposed method is experimentally compared with two traditional classifier fusion strategies and with a stacking-based strategy. Empirical evidence shows that using the proposed Decision Templates adaptation can improve the performance compared to the traditionally used fusion schemes on most of the evaluated metrics.
Read the full abstract ▾
All authors Victor F. Rocha, Alexandre L. Rodrigues, Thiago Oliveira-Santos, Flávio M. Varejão
Machine Learning · Cross-listed from cs.IR
AMES: Approximate Multi-modal Enterprise Search via Late Interaction Retrieval
Tony Joseph, Carlos Pareja, David Lopes Pegna et al. (+1 more)
We present AMES (Approximate
Multimodal Enterprise Search), a unified
multimodal late interaction
retrieval architecture which is backend agnostic. AMES demonstrates that fine-grained
multimodal late interaction
retrieval can be deployed within a production grade enterprise search engine without architectural redesign. Text
tokens, image patches, and
video frames are embedded into a shared representation space using multi-vector encoders, enabling cross-modal
retrieval without modality specific
retrieval logic. AMES employs a two-stage pipeline: parallel
token level ANN search with per document Top-M MaxSim approximation, followed by accelerator optimized Exact MaxSim re-ranking. Experiments on the ViDoRe V3
benchmark show that AMES achieves competitive ranking performance within a scalable, production ready Solr based system.
Read the full abstract ▾
All authors Tony Joseph, Carlos Pareja, David Lopes Pegna, Abhishek Singh
Machine Learning · Cross-listed from quant-ph
Active Sampling Sample-based Quantum Diagonalization from Finite-Shot Measurements
Rinka Miura
Near-term
quantum devices provide only finite-shot measurements and prepare imperfect, contaminated states. This motivates
algorithms that convert samples into reliable low-energy estimates without full
tomography or exhaustive measurements. We propose Active Sampling Sample-based
Quantum Diagonalization (AS-SQD), framing SQD as an active learning problem: given measured bitstrings, which additional basis states should be included to efficiently recover the ground-state energy? SQD restricts the
Hamiltonian to a selected set of basis states and classically diagonalizes the restricted
matrix. However, naive SQD using only sampled states suffers from bias under finite-shot sampling and excited-state contamination, while blind random expansion is inefficient as system size grows. We introduce a perturbation-theoretic acquisition function based on Epstein–Nesbet second-order energy corrections to rank candidate basis states connected to the current subspace. At each iteration, AS-SQD diagonalizes the restricted
Hamiltonian, generates connected candidates, and adds the most valuable ones according to this score. We evaluate AS-SQD on disordered Heisenberg and Transverse-Field Ising (TFIM)
spin chains up to 16
qubits under a preparation model mixing 80% ground state and 20% first excited state. Furthermore, we validate its robustness against real-world state preparation and measurement (SPAM) errors using physical samples from an IBM
Quantum processor. Across simulated and hardware evaluations, AS-SQD consistently achieves substantially lower absolute energy errors than standard SQD and random expansion. Detailed ablation studies demonstrate that physics-guided basis acquisition effectively concentrates computation on energetically relevant directions, bypassing exponential combinatorial bottlenecks.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.CO
From Proxy to Certificate: Two-Sided Ollivier-Ricci Bounds from Balanced Forman Curvature
Giorgio Micaletto and Tebe Nigrelli
Balanced Forman (BF)
curvature is used in graph learning as a cheap stand-in for Ollivier-
Ricci (OR)
curvature, yet it does not lower-bound OR
curvature. We show that the BF value of an edge certifies a two-sided interval for its lazy OR
curvature without solving any optimal transport problem, that BF coincides with a quadrangle-augmented Jost-Liu (JL) bound with its positive parts removed, and that when the 2-hop neighborhood is available, the interval can be sharpened at a cost of $\mathcal O(d_{\max}^{2.5})$ per edge. On 23
benchmark graphs, our quadrangle-augmented bound is tighter than the bounds of JL and Tian-Lubberts-Weber (TLW) on 13.8-100% of the edges and reduces their mean gap to the exact
curvature by a median of 31.2%; intersected with the bounds of TLW, the intervals settle the
curvature sign of more than half of the edges on 21 of the 23 graphs. On 15 real networks with up to 1,049,866 edges, a certify-then-solve procedure answers sign queries exactly while solving a median of 5.9 times fewer transport problems, for median speedups of 2$\times$ over exact evaluation, measured against a min-cut solver that is itself about two orders of magnitude faster than the widely used $\mathtt{GraphRicciCurvature}$. The main results are machine-checked in Lean 4.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Hide and Seek: Investigating Redundancy in Earth Observation Imagery
Tasos Papazafeiropoulos, Nikolaos Ioannis Bountos, Nikolas Papadopoulos et al. (+1 more)
The growing availability of Earth Observation (EO) data and recent advances in Computer
Vision have driven rapid progress in machine learning for EO, producing domain-specific models at ever-increasing scales. Yet this progress risks overlooking fundamental properties of EO data that distinguish it from other domains. We argue that EO data exhibit a multidimensional redundancy (spectral, temporal, spatial, and semantic) which has a more pronounced impact on the domain and its applications than what current literature reflects. To validate this hypothesis, we conduct a systematic domain-specific investigation examining the existence, consistency, and practical implications of this phenomenon across key dimensions of EO variability. Our findings confirm that redundancy in EO data is both substantial and pervasive: exploiting it yields comparable performance ($\approx98.5\%$ of baseline) at a fraction of the computational cost ($\approx4\times$ fewer GFLOPs), at both training and inference. Crucially, these gains are consistent across tasks, geospatial locations, sensors, ground sampling distances, and architectural designs; suggesting that multi-faceted redundancy is a structural property of EO data rather than an artifact of specific experimental choices. These results lay the groundwork for more efficient, scalable, and accessible large-scale EO models.
Read the full abstract ▾
All authors Tasos Papazafeiropoulos, Nikolaos Ioannis Bountos, Nikolas Papadopoulos, Ioannis Papoutsis
Machine Learning · Cross-listed from eess.AS
VoXtream2: Full-stream TTS with dynamic speaking rate control
Nikita Torgashov, Gustav Eje Henter and Gabriel Skantze
Full-stream text-to-
speech (TTS) for interactive systems must start speaking with minimal delay while remaining controllable as text arrives incrementally. We present VoXtream2, a zero-shot full-stream TTS model with dynamic speaking-rate control that can be updated mid-utterance on the fly. VoXtream2 combines a distribution matching mechanism over duration states with classifier-free guidance across conditioning signals to improve controllability and synthesis quality.
Prompt-text masking enables textless audio prompting, removing the need for
prompt transcription. Across standard zero-shot
benchmarks and a dedicated speaking-rate test set, VoXtream2 achieves competitive objective and subjective results against public baselines despite a smaller model and less training data. In full-stream mode, it runs 4 times faster than real time with 74 ms first-packet latency on a consumer GPU.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Standard Acquisition Is Sufficient for Asynchronous Bayesian Optimization
Ben Riegler, James Odgers and Vincent Fortuin
Asynchronous
Bayesian optimization is widely used for gradient-free optimization in domains with independent parallel experiments and varying evaluation times. Existing methods posit that standard acquisitions lead to redundant and repeated queries, proposing complex solutions to enforce diversity in queries. Challenging this fundamental premise, we show that methods, like the Upper Confidence Bound, can in fact achieve theoretical guarantees essentially equivalent to those of sequential Thompson sampling. A conceptual analysis of asynchronous
Bayesian optimization reveals that existing works neglect intermediate posterior updates, which we find to be generally sufficient to avoid redundant queries. Further investigation shows that by penalizing busy locations, diversity-enforcing methods can over-explore in asynchronous settings, reducing their performance. Our extensive experiments demonstrate that simple standard acquisition functions match or outperform purpose-built asynchronous methods across synthetic and real-world tasks.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Synthetic Melanoma Image Generation and Evaluation Using Generative Adversarial Networks
Pei-Yu Lin, Yidan Shen, Neville Mathew et al. (+6 more)
Melanoma is the most lethal form of skin cancer, and early detection is critical for improving patient outcomes. Although dermoscopy combined with deep learning has advanced automated skin-lesion analysis, progress is hindered by limited access to large, well-annotated
datasets and by severe class imbalance, where melanoma images are substantially underrepresented. To address these challenges, we present the first systematic benchmarking study comparing four GAN architectures-DCGAN, StyleGAN2, and two StyleGAN3 variants (T/R)-for high-resolution melanoma-specific synthesis. We train and optimize all models on two expert-annotated
benchmarks (ISIC 2018 and ISIC 2020) under unified preprocessing and hyperparameter exploration, with particular
attention to R1 regularization tuning. Image quality is assessed through a multi-faceted protocol combining distribution-level metrics (FID), sample-level representativeness (FMD), qualitative dermoscopic inspection, downstream classification with a frozen EfficientNet-based melanoma detector, and independent evaluation by two board-certified dermatologists. StyleGAN2 achieves the best balance of quantitative performance and perceptual quality, attaining FID scores of 24.8 (ISIC 2018) and 7.96 (ISIC 2020) at gamma=0.8. The frozen classifier recognizes 83% of StyleGAN2-generated images as melanoma, while dermatologists distinguish synthetic from real images at only 66.5% accuracy (chance = 50%), with low inter-rater agreement (kappa = 0.17). In a controlled augmentation experiment, adding synthetic melanoma images to address class imbalance improved melanoma detection AUC from 0.925 to 0.945 on a held-out real-image test set. These findings demonstrate that StyleGAN2-generated melanoma images preserve diagnostically relevant features and can provide a measurable benefit for mitigating class imbalance in melanoma-focused machine learning pipelines.
Read the full abstract ▾
All authors Pei-Yu Lin, Yidan Shen, Neville Mathew, Renjie Hu, Siyu Huang, Courtney M. Queen, Cameron E. West, Ana Ciurea, George Zouridakis
Machine Learning
Deep Invertible Autoencoders for Dimensionality Reduction of Dynamical Systems
Nicolò Botteghi, Silke Glas and Christoph Brune
Constructing reduced-order models (ROMs) capable of efficiently predicting the
evolution of parameter-dependent high-dimensional dynamical systems is crucial in many applications in engineering and applied sciences. A popular class of projection-based ROMs projects the high-dimensional full-order model (FOM) dynamics onto a low-dimensional
manifold. These projection-based ROMs approaches often rely on classical model reduction techniques such as proper orthogonal decomposition (POD) or, more recently, on neural network architectures such as autoencoders (AEs). In the case that the ROM is constructed by the POD, one has approximation guaranteed based based on the singular values of the problem at hand. However, POD-based techniques can suffer from slow decay of the singular values in transport- and advection-dominated problems. In contrast to that, AEs allow for better reduction capabilities than the POD, often with the first few modes, but at the price of theoretical considerations. In addition, it is often observed, that AEs exhibits a plateau of the projection error with the increment of the dimension of the trial
manifold. In this work, we propose an invertible AE architecture, named inv-AE, that computationally improves upon the stagnation of the reconstruction error typical of traditional AE architectures. Inv-AE is composed of several invertible neural network layers that allows for gradually recovering more information about the FOM solutions the more we increase the dimension of the reduced
manifold. Through the application of inv-AE to a 1-dimensional Burgers' equation, a 2-dimensional fluid flow around an obstacle with variable geometry, and a 3-dimensional Korteweg-de Vries, we show that (i) inv-AE mitigates the issue of the characteristic plateau of AEs and (ii) inv-AE can be combined with popular autoencoder-based ROM approaches, e.g., DL-ROM, to improve their accuracy.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.NE
Equivalence of approximation by networks of single- and multi-spike neurons
Dominik Dold and Philipp Christian Petersen
In a spiking neural network, is it enough for each
neuron to spike at most once? In recent work, approximation bounds for spiking neural networks have been derived, quantifying how well they can fit target functions. However, these results are only valid for
neurons that spike at most once, which is commonly thought to be a strong limitation. Here, we show that the opposite is true for a large class of spiking
neuron models, including the commonly used leaky integrate-and-fire model with subtractive reset: for every approximation bound that is valid for a set of multi-spike neural networks, there is an equivalent set of single-spike neural networks with only linearly more (or less)
neurons, in the maximum number of spikes, for which the bound holds. The same is true for the reverse direction too, showing that regarding their approximation capabilities in general machine learning tasks, single-spike and multi-spike neural networks are equivalent. Consequently, many approximation results in the literature for single-spike neural networks also hold for the multi-spike case.
Read the full abstract ▾
Machine Learning
Learning Not To Interfere: Adapting Experts For Model Merging
Pratik Ramesh, George Stoica, Arun Iyer et al. (+2 more)
Model merging has shown that multitask models can be created by directly combining the parameters of different models that are each specialized on tasks of interest. However, models trained independently on distinct tasks often exhibit interference that degrades the merged model's performance. To solve this problem, we formally define the notion of Cross-Task Interference as the drift in the representation of the merged model relative to its constituent models. Reducing cross-task interference is key to improving merging performance. To address this issue, we propose our method, Resolving Interference (RI), a light-weight adaptation framework which disentangles expert models to be functionally orthogonal to the space of other tasks, thereby reducing cross-task interference. RI does this whilst using only unlabeled auxiliary data as input (i.e., no task-data is needed), allowing it to be applied in data-scarce scenarios. RI consistently improves the performance of state-of-the-art merging methods by up to 3.8% and generalization to unseen domains by up to 2.3%. We also find RI to be robust to the source of auxiliary input while being significantly less sensitive to tuning of merging hyperparameters. Our codebase is available at: https://github.com/pramesh39/resolving_interference
Read the full abstract ▾
All authors Pratik Ramesh, George Stoica, Arun Iyer, Leshem Choshen, Judy Hoffman
Machine Learning · Cross-listed from cs.CR
Purifying Generative LLMs from Backdoors without Prior Knowledge or Clean Reference
Jianwei Li and Jung-Eun Kim
Backdoor attacks pose severe security threats to large
language models (LLMs), where a model behaves normally under benign inputs but produces malicious outputs when a hidden trigger appears. Existing
backdoor removal methods typically assume prior knowledge of triggers, access to a clean reference model, or rely on aggressive finetuning configurations, and are often limited to classification tasks. However, such assumptions fall apart in real-world instruction-tuned LLM settings. In this work, we propose a new framework for purifying instruction-tuned LLM without any prior trigger knowledge or clean references. Through systematic sanity checks, we find that
backdoor associations are redundantly encoded across MLP layers, while
attention modules primarily amplify trigger signals without establishing the behavior. Leveraging this insight, we shift the focus from isolating specific
backdoor triggers to cutting off the trigger-behavior associations, and design an immunization-inspired elimination approach: by constructing multiple synthetic backdoored variants of the given suspicious model, each trained with different malicious trigger-behavior pairs, and contrasting them with their clean counterparts. The recurring modifications across variants reveal a shared "
backdoor signature"-analogous to antigens in a virus. Guided by this signature, we neutralize highly suspicious components in LLM and apply lightweight finetuning to restore its fluency, producing purified models that withstand diverse
backdoor attacks and threat models while preserving generative capability.
Read the full abstract ▾
Machine Learning
Reconciling In-Context and In-Weight Learning via Dual Representation Space Encoding
Guanyu Chen, Ruichen Wang, Tianren Zhang et al. (+1 more)
In-context learning (ICL) is a valuable capability exhibited by
Transformers pretrained on diverse sequence tasks. However, previous studies have observed that ICL often conflicts with the model's inherent in-weight learning (IWL) ability. By examining the representation space learned by a toy model in synthetic experiments, we identify the shared encoding space for context and samples in
Transformers as a potential source of this conflict. To address this, we modify the model architecture to separately encode the context and samples into two distinct spaces: a task representation space and a sample representation space. We model these two spaces under a simple yet principled framework, assuming a linear representational structure and treating them as a pair of dual spaces. Both theoretical analysis and empirical results demonstrate the effectiveness of our proposed architecture, CoQE, in the single-value answer setting. It not only enhances ICL performance through improved representation learning, but also successfully reconciles ICL and IWL capabilities across synthetic few-shot classification and a newly designed pseudo-arithmetic task. Code: https://github.com/McGuinnessChen/dual-representation-space-encoding
Read the full abstract ▾
All authors Guanyu Chen, Ruichen Wang, Tianren Zhang, Feng Chen
Machine Learning
Scalable Machines with Intrinsic Higher Mental-State Dynamics
Ahsan Adeel and M. Bilal
Drawing on recent breakthroughs in cellular neurobiology and detailed biophysical modeling linking neocortical pyramidal
neurons to distinct mental-state regimes, this work introduces a mathematically grounded formulation showing how models (e.g.,
Transformers) can implement computational principles underlying awake imaginative thought to pre-select relevant information before
attention is applied via triadic modulation loops among queries ($Q$), keys ($K$), and values ($V$).~Scalability experiments on ImageNet-1K, benchmarked against a standard
Vision Transformer (ViT), demonstrate significantly faster learning with reduced computational demand (fewer heads, layers, and
tokens), consistent with our prior findings in reinforcement learning and
language modeling. The approach operates at approximately $\mathcal{O}(N)$ complexity with respect to the number of input
tokens $N$.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
MESD: A Risk-Sensitive Metric for Explanation Fairness Across Intersectional Subgroups
Gideon Popoola and John Sheppard
Fairness in machine learning is predominantly evaluated through outcome-oriented metrics, such as Demographic parity, which measure whether predictions are statistically consistent across protected groups. However, these metrics cannot detect whether a model uses systematically different
reasoning for different demographic groups, which violates procedural
fairness principles. This problem is compounded by intersectionality, where models may appear fair on individual attributes (e.g., race) while exhibiting significant disparities for intersectional subgroups (e.g., race $\times$ gender), a phenomenon known as
fairness gerrymandering. In this work, we introduce Multi-category Explanation Stability Disparity (MESD), a procedural
fairness metric that quantifies disparities in explanation quality across intersectional subgroups formed by the Cartesian product of multiple protected attributes. MESD integrates three components, which are label-aware aggregation aligned with outcome-conditional
fairness, empirical-Bayes shrinkage to stabilize estimates for small intersectional groups, and Conditional Value-at-Risk (CVaR) weighting to emphasize worst-case subgroup disparities. We integrate MESD within a multi-objective optimization framework (UEF) that jointly optimizes utility, outcome
fairness, and procedural
fairness using NSGA-II. We evaluated MESD and UEF on three
benchmark datasets along with four state-of-the-art methods in several experiments, and we demonstrate that MESD reveals procedural disparities invisible to outcome metrics alone. We position our contribution within procedural justice theory and discuss implications for regulatory compliance and intersectional equity.
Read the full abstract ▾
Machine Learning · Cross-listed from cond-mat.mtrl-sci
Diffusion-based Generative Machine Learning Model for Predicting Crack Propagation in Aluminum Nitride at the Atomic Scale
Jiali Lu and Shengfeng Yang
Predicting atomic-scale crack propagation in aluminum nitride (AlN) is critical for
semiconductor reliability but remains prohibitively expensive via molecular dynamics (MD). We develop a
diffusion-based generative machine learning model to predict atomic-scale crack propagation in AlN, a critical
semiconductor material, by conditioning solely on initial microstructure
embeddings. Trained on MD simulations of single-crack systems, the model achieves a significant speedup while accurately
forecasting dynamic fracture processes, including stress-driven crack initiation, crack branching, and atomic-scale bridging ligaments. Crucially, it demonstrates inherent physical fidelity by reproducing material-intrinsic mechanisms while disregarding periodic boundary artifacts, and generalizes to unseen multi-crack configurations. Validation against MD ground truth confirms the capability of the model to capture complex fracture physics without auxiliary stress or energy data, enabling rapid exploration of crack-mediated failure for
semiconductor reliability optimization.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Filtered Spectral Projection for Quantum Principal Component Analysis
Sk Mujaffar Hossain and Satadeep Bhattacharjee
Quantum principal component analysis (qPCA) is commonly formulated as the extraction of
eigenvalues and eigenvectors of a covariance-encoded density operator. Yet in many qPCA settings the practical goal is simpler: projection onto the dominant spectral subspace. Here we introduce a projection-first framework, the Filtered Spectral Projection
Algorithm (FSPA), which bypasses explicit
eigenvalue estimation while preserving the relevant spectral structure. FSPA amplifies any nonzero warm-start overlap with the leading subspace and remains robust in small-gap and near-degenerate regimes, without artificial
symmetry breaking in the absence of bias. We show that FSPA achieves an oracle complexity $\mathcal{O}((\log(1/\epsilon)+\log(1/|a_1|^2))/\log(\lambda_1/\lambda_2))$,which is tight by a matching lower bound, establishing it as anoptimal projection primitive. We derive a convergence rate for degenerate spectra, give a
circuit resource analysis with $n+\mathcal{O}(1)$
qubit overhead independent of system dimension, and extend the method to threshold spectral projection, Threshold-FSPA, which converges in $\mathcal{O}(\log(1/\epsilon))$ calls when the threshold lies between
eigenvalues. In the density
matrix exponentiation access model, FSPA gives an exponential copy-complexity advantage over classical methods. For classical
datasets, we show that for amplitude-encoded centered data the ensemble density
matrix $\rho=\sum_i p_i|\psi_i\rangle\langle\psi_i|$ equals the covariance
matrix. Numerical tests on chemistry density
matrices, noisy
circuit outputs, Breast Cancer Wisconsin, handwritten Digits, and 1–4-
qubit scalability confirm the theory. A minimal Qiskit implementation validates magnitude invariance, signal amplification, and no spurious
symmetry breaking. These results establish FSPA as an optimal and deployable
quantum spectral projection primitive.
Read the full abstract ▾
Machine Learning
Improving Channel Estimation via Multimodal Diffusion Models with Flow Matching
Xiaotian Fan, Xingyu Zhou, Le Liang et al. (+2 more)
Deep generative models offer a powerful alternative to conventional channel estimation by learning complex channel distributions. By integrating the rich environmental information available in modern sensing-aided networks, this paper proposes MultiCE-Flow, a
multimodal channel estimation framework based on flow matching and
diffusion transformer (DiT). We design a specialized
multimodal perception module that fuses
LiDAR, camera, and location data into a semantic condition, while treating sparse pilots as a structural condition. These conditions guide a DiT backbone to reconstruct high-fidelity channels. Unlike standard
diffusion models, we employ flow matching to learn a linear trajectory from noise to data, enabling efficient one-step sampling. By leveraging environmental semantics, our method mitigates the ill-posed nature of estimation with sparse pilots. Extensive experiments demonstrate that MultiCE-Flow consistently outperforms traditional baselines and existing generative models. Notably, it exhibits superior robustness to out-of-distribution scenarios and varying pilot densities, making it suitable for environment-aware communication systems.
Read the full abstract ▾
All authors Xiaotian Fan, Xingyu Zhou, Le Liang, Xiao Li, Shi Jin
Machine Learning
Modality-free Graph In-context Alignment
Wei Zhuo and Siqiang Luo
In-context learning (ICL) converts static encoders into task-conditioned reasoners, enabling adaptation to new data from just a few examples without updating pretrained parameters. This capability is essential for graph foundation models (GFMs) to approach LLM-level generality. Yet current GFMs struggle with cross-domain
alignment, typically relying on modality-specific encoders that fail when graphs are pre-vectorized or raw data is inaccessible. In this paper, we introduce Modality-Free Graph In-context
Alignment (MF-GIA), a framework that makes a pretrained graph encoder promptable for few-shot prediction across heterogeneous domains without modality assumptions. MF-GIA captures domain characteristics through gradient fingerprints, which parameterize lightweight transformations that align pre-encoded features and indexed labels into unified semantic spaces. During pretraining, a dual
prompt-aware
attention mechanism with episodic objective learns to match queries against aligned support examples to establish
prompt-based
reasoning capabilities. At inference, MF-GIA performs parameter-update-free adaptation using only a few-shot support set to trigger cross-domain
alignment and enable immediate prediction on unseen domains. Experiments demonstrate that MF-GIA achieves superior few-shot performance across diverse graph domains and strong generalization to unseen domains.
Read the full abstract ▾
Machine Learning
CHIMERA-Bench: A Benchmark Dataset for Epitope-Specific Antibody Design
Mansoor Ahmed, Nadeem Taj, Imdad Ullah Khan et al. (+2 more)
Computational antibody design has seen rapid methodological progress, with dozens of deep generative methods proposed in the past three years, yet the field lacks a standardized
benchmark for fair comparison and model development. These methods are evaluated on different SAbDab snapshots, non-overlapping test sets, and incompatible metrics, and the literature fragments the design problem into numerous sub-tasks with no common definition. We introduce CHIMERA-Bench: (CDR Modeling with Epitope-guided Redesign), a unified
benchmark built around a single canonical task: epitope-conditioned CDR sequence-structure co-design. CHIMERA-Bench provides three components. The first is a curated, deduplicated
dataset of 2,922 antibody-antigen complexes with epitope and paratope annotations. The second is a set of three biologically motivated splits that test generalization to unseen epitopes, unseen antigen folds, and prospective temporal targets. The third is a comprehensive evaluation protocol with five metric groups, including novel epitope-specificity measures. We
benchmark eleven methods spanning six generative paradigms and report results across all splits. CHIMERA-Bench is the largest
dataset of its kind for the antibody design problem, allowing the community to develop and test novel methods and evaluate their generalizability.
Read the full abstract ▾
All authors Mansoor Ahmed, Nadeem Taj, Imdad Ullah Khan, Hemanth Venkateswara, Murray Patterson
Machine Learning · Cross-listed from cs.AR
Dynamic Sparse Attention: Access Patterns and Architecture
Noam Levy
Dynamic sparse
attention (DSA) reduces the per-
token attention bandwidth by restricting computation to a top-k subset of cached key-value (KV) entries, but its
token-dependent selection pattern introduces a system-level challenge: the KV working set is fragmented, volatile, and difficult to prefetch, which can translate into poor cache locality and stalled decode throughput. We study these effects by implementing a lightweight indexer for DSA-style selection on multiple open-source backbones and logging per-layer KV indices during autoregressive decoding. Our analysis shows a gap in serving DSA backbones - a potential for a high volume of blocking LL (last level) cache miss events, causing inefficiency; we propose a novel LL cache reservation system to save KV
tokens in the LL cache between decode steps, combined with a
token-granularity LRU eviction policy, and show on the data we collected how this architecture can benefit serving with DSA implemented on different backbones. Finally, we propose directions for future architectural and algorithmic exploration to improve serving of DSA on modern inference platforms.
Read the full abstract ▾
Machine Learning
Outcome-Aware Tool Selection for Semantic Routers: Latency-Constrained Learning Without LLM Inference
Huamin Chen, Xunzhuo Liu, Junchen Jiang et al. (+2 more)
Semantic routers in LLM inference gateways select tools in the critical request path, where every millisecond of added latency compounds across millions of requests. We propose Outcome-Aware Tool Selection (OATS), which interpolates tool
embeddings toward the centroid of queries where they historically succeed – an offline process that adds no parameters, latency, or GPU cost at serving time. On MetaTool (199~tools, 4,287~queries), this improves NDCG@5 from 0.869 to 0.940; on ToolBench (2,413~APIs), from 0.834 to 0.848. We also evaluate two learned extensions: a 2,625-parameter MLP re-ranker and a 197K-parameter contrastive adapter. The MLP re-ranker hurts or matches baseline when outcome data is sparse relative to the tool set; the contrastive adapter provides comparable gains on MetaTool (NDCG@5: 0.931). All methods are evaluated on the same held-out 30% test split. The practical takeaway is to start with the zero-cost refinement and add learned components only when data density warrants it. All mechanisms run within single-digit millisecond CPU budgets.
Read the full abstract ▾
All authors Huamin Chen, Xunzhuo Liu, Junchen Jiang, Bowei He, Xue Liu
Machine Learning
Self-Flow-Matching assisted Full Waveform Inversion
Xinquan Huang and Paris Perdikaris
Full-waveform inversion (FWI) is a high-resolution seismic imaging method that estimates subsurface velocity by matching simulated and recorded waveforms. However, FWI is highly nonlinear, prone to cycle skipping, and sensitive to noise, particularly when low frequencies are missing or the initial model is poor, leading to failures under imperfect acquisition.
Diffusion-regularized FWI introduces generative priors to encourage geologically realistic models, but these priors typically require costly offline pretraining and can deteriorate under distribution shift. Moreover, they assume Gaussian initialization and a fixed noise schedule, in which it is unclear how to map a deterministic FWI iterate and its starting model to a well-defined
diffusion time or noise level. To address these limitations, we introduce Self-Flow-Matching assisted Full-Waveform Inversion (SFM-FWI), a physics-driven framework that eliminates the need for large-scale offline pretraining while avoiding the noise-level
alignment ambiguity. SFM-FWI leverages flow matching to learn a transport field without assuming Gaussian initialization or a predefined noise schedule, so the initial model can be used directly as the starting point of the dynamics. Our approach trains a single flow network online using the governing physics and observed data. At each outer iteration, we build an interpolated model and update the flow by backpropagating the FWI data misfit, providing self-supervision without external training pairs. Experiments on challenging synthetic
benchmarks show that SFM-FWI delivers more accurate reconstructions, greater noise robustness, and more stable convergence than standard FWI and pretraining-free regularization methods.
Read the full abstract ▾
Machine Learning
From Gradients to Riccati Geometry: Kalman World Models for Single-Pass Learning
Andrew Kiruluta
Backpropagation dominates modern machine learning, yet it is not the only principled method for optimizing dynamical systems. We propose Kalman World Models (KWM), a class of learned state-space models trained via recursive
Bayesian filtering rather than reverse-mode automatic differentiation. Instead of gradient descent updates, we replace parameter learning with Kalman-style gain adaptation. Training becomes online filtering; error signals become innovations. We further extend this framework to
transformer-based large
language models (LLMs), where internal activations are treated as latent dynamical states corrected via innovation terms. This yields a gradient-free training and adaptation paradigm grounded in control theory. We derive stability conditions, analyze computational complexity, and provide empirical results on sequence modeling tasks demonstrating competitive performance with improved robustness and continual adaptation properties.
Read the full abstract ▾
Machine Learning
PhysMoDPO: Physically-Plausible Humanoid Motion with Preference Optimization
Yangsong Zhang, Anujith Muraleedharan, Rikhat Akizhanov et al. (+5 more)
Recent progress in text-conditioned human motion generation has been largely driven by
diffusion models trained on large-scale human motion data. Building on this progress, recent methods attempt to transfer such models for character animation and real
robot control by applying a Whole-Body Controller (WBC) that converts
diffusion-generated motions into executable trajectories. While WBC trajectories become compliant with physics, they may expose substantial deviations from original motion. To address this issue, we here propose PhysMoDPO, a Direct Preference Optimization framework. Unlike prior work that relies on hand-crafted physics-aware heuristics such as foot-sliding penalties, we integrate WBC into our training pipeline and optimize
diffusion model such that the output of WBC becomes compliant both with physics and original text instructions. To train PhysMoDPO we deploy physics-based and task-specific rewards and use them to assign preference to synthesized trajectories. Our extensive experiments on text-to-motion and spatial control tasks demonstrate consistent improvements of PhysMoDPO in both physical realism and task-related metrics on simulated
robots. Moreover, we demonstrate that PhysMoDPO results in significant improvements when applied to zero-shot motion transfer in simulation and for real-world deployment on a G1
humanoid robot.
Read the full abstract ▾
All authors Yangsong Zhang, Anujith Muraleedharan, Rikhat Akizhanov, Abdul Ahad Butt, Gül Varol, Pascal Fua, Fabio Pizzati, Ivan Laptev
Machine Learning
Representation Learning for Spatiotemporal Physical Systems
Helen Qu, Rudy Morel, Michael McCabe et al. (+4 more)
Machine learning approaches to spatiotemporal physical systems have primarily focused on next-frame prediction, with the goal of learning an accurate emulator for the system's
evolution in time. However, these emulators are computationally expensive to train and are subject to performance pitfalls, such as compounding errors during autoregressive rollout. In this work, we take a different perspective and look at scientific tasks further downstream of predicting the next frame, such as estimation of a system's governing physical parameters. Accuracy on these tasks offers a uniquely quantifiable glimpse into the physical relevance of the representations of these models. We evaluate the effectiveness of general-purpose self-supervised methods in learning physics-grounded representations that are useful for downstream scientific tasks. Surprisingly, we find that not all methods designed for physical modeling outperform generic self-supervised learning methods on these tasks, and methods that learn in the latent space (e.g., joint
embedding predictive architectures, or JEPAs) outperform those optimizing pixel-level prediction objectives. Code is available at https://github.com/helenqu/physical-representation-learning.
Read the full abstract ▾
All authors Helen Qu, Rudy Morel, Michael McCabe, Alberto Bietti, François Lanusse, Shirley Ho, Yann LeCun