Machine Learning · Cross-listed from stat.ML
LoBoost: Model-Native Local Conformal Prediction for Gradient-Boosted Trees
Vagner Santos, Victor Coscrato, Luben Cabezas et al. (+2 more)
Gradient-boosted decision trees are among the strongest off-the-shelf predictors for tabular regression, but point predictions alone do not quantify uncertainty. Conformal prediction provides distribution-free marginal coverage, yet standard split conformal uses a single global residual
quantile and can adapt poorly to heteroscedasticity. We propose LoBoost, a model-native local conformal method that reuses the fitted ensemble's leaf structure to define a multiscale partition of the feature space. Each input is represented by the sequence of leaves it visits along the boosting path, and matching leaf prefixes define nested groups in which residual
quantiles are estimated locally. By reusing the predictive structure already learned by the model, LoBoost requires no auxiliary partition or nuisance model, no retraining, and no additional data split beyond standard conformal calibration. Our theory connects the stability of the fitted ensemble and the geometry of its induced
cells to local residual-score homogeneity, providing finite-sample coverage-error control and asymptotic pointwise validity as the local calibration size grows and the
cells become sufficiently homogeneous. Experiments show a transparent quality-cost trade-off: LoBoost improves over global split conformal on several heterogeneous
datasets, but it is not uniformly superior to the adaptive baselines and is not uniformly Pareto-optimal.
Read the full abstract ▾
All authors Vagner Santos, Victor Coscrato, Luben Cabezas, Rafael Izbicki, Thiago Ramos
Machine Learning · Cross-listed from cs.SD
mmWave Radar Aware Dual-Conditioned GAN for Speech Reconstruction of Signals With Low SNR
Jash Karani, Adithya Chittem, Deepan Roy et al. (+1 more)
Millimeter-wave (mmWave) radar captures are band-limited and noisy, making for difficult reconstruction of intelligible full-bandwidth
speech. In this work, we propose a two-stage
speech reconstruction pipeline for mmWave using a Radar-Aware Dual-conditioned Generative
Adversarial Network (RAD-GAN), which is capable of performing bandwidth extension on signals with low signal-to-noise ratios (-5 dB to -1 dB), captured through glass walls. We propose an mmWave-tailored Multi-Mel Discriminator (MMD) and a Residual Fusion Gate (RFG) to enhance the generator input to process multiple conditioning channels. The proposed two-stage pipeline involves pretraining the model on synthetically clipped clean
speech and finetuning on fused mel spectrograms generated by the RFG. We empirically show that the proposed method, trained on a limited
dataset, with no pre-trained modules, and no data augmentations, outperformed state-of-the-art approaches for this specific task. Audio examples of RAD-GAN are available online at https://rad-gan-demo-site.vercel.app/.
Read the full abstract ▾
All authors Jash Karani, Adithya Chittem, Deepan Roy, Sandeep Joshi
Machine Learning · Cross-listed from cs.GR
TopoEdit: Fast Post-Optimization Editing of Topology Optimized Structures
Hongrui Chen, Josephine V. Carstensen and Faez Ahmed
Despite
topology optimization producing high-performance structures, late-stage localized revisions remain brittle: direct density-space edits (e.g., warping pixels, inserting holes, swapping infill) can sever load paths and sharply degrade compliance, while re-running optimization is slow and may drift toward a qualitatively different design. We present TopoEdit, a fast post-optimization editor that demonstrates how structured latent
embeddings from a pre-trained
topology foundation model (OAT) can be repurposed as an interface for physics-aware engineering edits. Given an optimized
topology, TopoEdit encodes it into OAT's spatial latent, applies partial noising to preserve instance identity while increasing editability, and injects user intent through an edit-then-denoise
diffusion pipeline. We instantiate three edit operators: drag-based
topology warping with boundary-condition-consistent conditioning updates, shell-infill
lattice replacement using a
lattice-anchored reference latent with updated volume-fraction conditioning, and late-stage no-design region enforcement via masked latent overwrite followed by
diffusion-based recovery. A consistency-preserving guided DDIM procedure localizes changes while allowing global structural adaptation; multiple candidates can be sampled and selected using a compliance-aware criterion, with optional short SIMP refinement for warps. Across diverse case studies and large edit sweeps, TopoEdit produces intention-aligned modifications that better preserve mechanical performance and avoid catastrophic failure modes compared to direct density-space edits, while generating edited candidates in sub-second
diffusion time per sample.
Read the full abstract ▾
Machine Learning
Calibrated Test-Time Guidance for Bayesian Inference
Daniel Geyfman, Felix Draxler, Jan Groeneveld et al. (+3 more)
Test-time guidance is a widely used mechanism for steering pretrained
diffusion models toward outcomes specified by a reward function. Existing approaches, however, focus on maximizing reward rather than sampling from the true
Bayesian posterior, leading to miscalibrated inference. In this work, we show that common test-time guidance methods do not recover the correct posterior distribution and identify the structural approximations responsible for this failure. We then propose consistent alternative
estimators that enable calibrated sampling from the
Bayesian posterior. We significantly outperform previous methods on a set of
Bayesian inference tasks, and set a new state-of-the-art PSNR in black hole image reconstruction.
Read the full abstract ▾
All authors Daniel Geyfman, Felix Draxler, Jan Groeneveld, Hyunsoo Lee, Theofanis Karaletsos, Stephan Mandt
Machine Learning · Cross-listed from cs.CV
SimpleOCR: Rendering Visualized Questions to Teach MLLMs to Read
Yibo Peng, Peng Xia, Ding Zhong et al. (+6 more)
Despite the rapid advancements in
Multimodal Large
Language Models (MLLMs), a critical question regarding their visual grounding mechanism remains unanswered: do these models genuinely “read” text embedded in images, or do they merely rely on parametric shortcuts in the text
prompt? In this work, we diagnose this issue by introducing the Visualized-Question (VQ) setting, where text queries are rendered directly onto images to structurally mandate visual engagement. Our diagnostic experiments on Qwen2.5-VL reveal a startling capability-utilization gap: despite possessing strong OCR capabilities, models suffer a performance degradation of up to 12.7% in the VQ setting, exposing a deep-seated “modality laziness.” To bridge this gap, we propose SimpleOCR, a plug-and-play training strategy that imposes a structural constraint on the learning process. By transforming training samples into the VQ format with randomized styles, SimpleOCR effectively invalidates text-based shortcuts, compelling the model to activate and optimize its visual text extraction pathways. Empirically, SimpleOCR yields robust gains without architectural modifications. On four representative OOD
benchmarks, it surpasses the base model by 5.4% and GRPO based on original images by 2.7%, while exhibiting extreme data efficiency, achieving superior performance with 30x fewer samples (8.5K) than recent RL-based methods. Furthermore, its plug-and-play nature allows seamless integration with advanced RL strategies like NoisyRollout to yield complementary improvements. Code is available at https://github.com/aiming-lab/SimpleOCR.
Read the full abstract ▾
All authors Yibo Peng, Peng Xia, Ding Zhong, Kaide Zeng, Siwei Han, Yiyang Zhou, Jiaqi Liu, Ruiyi Zhang, Huaxiu Yao
Machine Learning · Cross-listed from cs.CL
Causality $\neq$ Invariance: Function and Concept Vectors in LLMs
Gustaw Opiełka, Hannes Rosenbusch and Claire E. Stevenson
Do large
language models (LLMs) represent concepts abstractly, i.e., independent of input format? We revisit Function Vectors (FVs), compact representations of in-context learning (ICL) tasks that causally drive task performance. Across multiple LLMs, we show that FVs are not fully invariant: FVs are nearly orthogonal when extracted from different input formats (e.g., open-ended vs. multiple-choice), even if both target the same concept. We identify Concept Vectors (CVs), which carry more stable concept representations. Like FVs, CVs are composed of
attention head outputs; however, unlike FVs, the constituent heads are selected using Representational Similarity Analysis (RSA) based on whether they encode concepts consistently across input formats. While these heads emerge in similar layers to FV-related heads, the two sets are largely distinct, suggesting different underlying mechanisms. Steering experiments reveal that FVs excel in-distribution, when extraction and application formats match (e.g., both open-ended in English), while CVs generalize better out-of-distribution across both question types (open-ended vs. multiple-choice) and
languages. Our results show that LLMs do contain abstract concept representations, but these differ from those that drive ICL performance.
Read the full abstract ▾
Machine Learning
Revisiting Chebyshev Polynomial and Anisotropic RBF Models for Tabular Regression
Luciano Gerber and Huw Lloyd
Smooth-basis models such as Chebyshev
polynomial regressors and radial basis function (RBF) networks are well established in numerical analysis. Their continuously differentiable prediction surfaces suit surrogate optimisation, sensitivity analysis, and other settings where the response varies gradually with inputs. Despite these properties, smooth models seldom appear in tabular regression, where tree ensembles dominate. We ask whether they can compete, benchmarking models across 55 regression
datasets organised by application domain. We develop an anisotropic RBF network with data-driven centre placement and gradient-based width optimisation, a ridge-regularised Chebyshev
polynomial regressor, and a smooth-tree hybrid (Chebyshev model tree), with all three released as scikit-learn-compatible packages. We
benchmark these against tree ensembles, a bin-based additive model, a pre-trained
transformer and standard baselines, evaluating accuracy alongside the train-test gap and prediction stability under retraining. The
transformer ranks first on accuracy across most
datasets, but its GPU dependence, prediction cost, and
dataset-size limits constrain deployment in the CPU-based settings common across applied science and industry. Among CPU-viable models, smooth models and tree ensembles are statistically tied on accuracy at the medium data scale that characterises most of our
datasets (up to roughly 50,000 samples and 50 features). In a full-scale check on eight
datasets, XGBoost achieved the highest mean accuracy, with the Chebyshev model tree close behind. At matched accuracy, smooth-basis models tend to exhibit tighter train-test gaps, an empirical proxy for prediction stability under retraining. We recommend including smooth-basis models among the candidate models for tabular regression, particularly when downstream use benefits from smooth prediction surfaces.
Read the full abstract ▾
Machine Learning
A Learning-Based Hybrid Decision Framework for Matching Systems with User Departure Detection
Ruiqi Zhou, Donghao Zhu and Houcai Shen
In matching
markets such as kidney exchanges and freight exchanges, delayed matching has been shown to improve overall
market efficiency. The benefits of delay are highly sensitive to participants' sojourn times and departure behavior, and delaying matches can impose significant costs, including longer waiting times and increased
market congestion. These competing effects make fixed matching policies inherently inflexible in dynamic environments. We propose a learning-based Hybrid framework that adaptively combines immediate and delayed matching. The framework continuously collects data on user departures over time, estimates the underlying departure distribution via regression, and determines whether to delay matching in the subsequent period based on a decision threshold that governs the system's tolerance for matching efficiency loss. The proposed framework can substantially reduce waiting times and congestion while sacrificing only a limited amount of matching efficiency. By dynamically adjusting its matching strategy, the Hybrid framework enables system performance to flexibly interpolate between purely greedy and purely patient policies, offering a robust and adaptive alternative to static matching mechanisms.
Read the full abstract ▾
Machine Learning
MolFM-Lite: Multi-Modal Molecular Property Prediction with Conformer Ensemble Attention and Cross-Modal Fusion
Syed Omer Shah, Mohammed Maqsood Ahmed, Danish Mohiuddin Mohammed et al. (+2 more)
Most machine learning models for molecular property prediction rely on a single molecular representation (either a sequence, a graph, or a 3D structure) and treat molecular geometry as static. We present MolFM-Lite, a multi-modal model that jointly encodes SELFIES sequences (1D), molecular graphs (2D), and conformer ensembles (3D) through cross-
attention fusion, while conditioning predictions on experimental context via Feature-wise Linear Modulation (FiLM). Our main methodological contributions are: (1) a conformer ensemble
attention mechanism that combines learnable
attention with Boltzmann-weighted priors over multiple RDKit-generated conformers, capturing the thermodynamic distribution of molecular shapes; and (2) a cross-modal fusion layer where each modality can attend to others, enabling complementary information sharing. We evaluate on four MoleculeNet scaffold-split
benchmarks using our model's own splits, and report all baselines re-evaluated under the same protocol. Comprehensive ablation studies across all four
datasets confirm that each architectural component contributes independently, with tri-modal fusion providing 7-11% AUC improvement over single-modality baselines and conformer ensembles adding approximately 2% over single-conformer variants. Pre-training on ZINC250K (~250K molecules) using cross-modal contrastive and masked-atom objectives enables effective weight initialization at modest compute cost. We release all code, trained models, and data splits to support reproducibility.
Read the full abstract ▾
All authors Syed Omer Shah, Mohammed Maqsood Ahmed, Danish Mohiuddin Mohammed, Shahnawaz Alam, Mohd Vahaj ur Rahman
Machine Learning
Predicting Multi-Drug Resistance in Bacterial Isolates Through Performance Comparison and LIME-based Interpretation of Classification Models
Santanam Wishal and Riad Sahara
The rise of Antimicrobial Resistance, particularly Multi-Drug Resistance (MDR), presents a critical challenge for clinical decision-making due to limited treatment options and delays in conventional susceptibility testing. This study proposes an interpretable machine learning framework to predict MDR in bacterial isolates using clinical features and antibiotic susceptibility patterns. Five classification models were evaluated, including Logistic Regression, Random Forest, AdaBoost, XGBoost, and LightGBM. The models were trained on a curated
dataset of 9,714 isolates, with resistance encoded at the antibiotic family level to capture cross-class resistance patterns consistent with MDR definitions. Performance assessment included accuracy, F1-score, AUC-ROC, and Matthews Correlation Coefficient. Ensemble models, particularly XGBoost and LightGBM, demonstrated superior predictive capability across all metrics. To address the clinical transparency gap, Local Interpretable Model-agnostic Explanations (LIME) was applied to generate instance-level explanations. LIME identified resistance to quinolones, Co-trimoxazole, Colistin, aminoglycosides, and Furanes as the strongest contributors to MDR predictions, aligning with known biological mechanisms. The results show that combining high-performing models with local interpretability provides both accuracy and actionable insights for antimicrobial stewardship. This framework supports earlier MDR identification and enhances trust in machine learning-assisted clinical decision support.
Read the full abstract ▾
Machine Learning
Disentangling Shared and Target-Enriched Topics via Background-Contrastive Non-negative Matrix Factorization
Yixuan Li, Archer Y. Yang and Yue Li
Biological signals of interest in high-dimensional data are often masked by dominant variation shared across conditions. This variation, arising from baseline biological structure or technical effects, can prevent standard dimensionality reduction methods from resolving condition-specific structure. The challenge is that these confounding topics are often unknown and mixed with biological signals. Existing background correction methods are either unscalable to high dimensions or not interpretable. We introduce background contrastive Non-negative
Matrix Factorization (\model), which extracts target-enriched latent topics by jointly factorizing a target
dataset and a matched background using shared non-negative bases under a contrastive objective that suppresses background-expressed structure. This approach yields non-negative components that are directly interpretable at the feature level, and explicitly isolates target-specific variation. \model is learned by an efficient multiplicative update
algorithm via
matrix multiplication such that it is highly efficient on GPU hardware and scalable to big data via minibatch training akin to deep learning approach. Across simulations and diverse biological
datasets, \model reveals signals obscured by conventional methods, including disease-associated programs in postmortem depressive
brain single-
cell RNA-seq, genotype-linked
protein expression patterns in mice, treatment-specific transcriptional changes in leukemia, and TP53-dependent drug responses in cancer
cell lines.
Read the full abstract ▾
Machine Learning
Learning geometry-dependent lead-field operators for forward ECG modeling
Arsenii Dokuchaev, Francesca Bonizzoni, Stefano Pagani et al. (+2 more)
Modern forward electrocardiogram (ECG) computational models rely on an accurate representation of the torso domain. The lead-field method enables fast ECG simulations while preserving full
geometric fidelity. Achieving high anatomical accuracy in torso representation is, however, challenging in clinical practice, as imaging protocols are typically focused on the heart and often do not include the entire torso. In addition, the computational cost of the lead-field method scales linearly with the number of electrodes, limiting its applicability in high-density recording settings. To date, no existing approach simultaneously achieves high anatomical fidelity, low data requirements and computational efficiency. In this work, we propose a shape-informed surrogate model of the lead-field operator that serves as a drop-in replacement for the full-order model in forward ECG simulations. The proposed framework consists of two components: a geometry-encoding module that maps anatomical shapes into a low-dimensional latent space, and a geometry-conditioned neural surrogate that predicts lead-field gradients from spatial coordinates, electrode positions and latent codes. The proposed method achieves high accuracy in approximating lead fields both within the torso (mean angular error 5{\deg}) and inside the heart, resulting in highly accurate ECG simulations (relative mean squared error <2.5%. The surrogate consistently outperforms the widely used pseudo lead-field approximation while preserving negligible inference cost. Owing to its compact latent representation, the method does not require a fully detailed torso
segmentation and can therefore be deployed in data-limited settings while preserving high-fidelity ECG simulations.
Read the full abstract ▾
All authors Arsenii Dokuchaev, Francesca Bonizzoni, Stefano Pagani, Francesco Regazzoni, Simone Pezzuto
Machine Learning
Structure and Redundancy in Large Language Models: A Spectral Study via Random Matrix Theory
Davide Ettori
This thesis addresses two persistent and closely related challenges in modern deep learning, reliability and efficiency, through a unified framework grounded in Spectral Geometry and Random
Matrix Theory (RMT). As deep networks and large
language models continue to scale, their internal behavior becomes increasingly opaque, leading to
hallucinations, fragile generalization under distribution shift, and growing computational and energy demands. By analyzing the
eigenvalue dynamics of hidden activations across layers and inputs, this work shows that spectral statistics provide a compact, stable, and interpretable lens on model behavior, capable of separating structured, causal representations from noise-dominated variability. Within this framework, the first contribution, EigenTrack, introduces a real-time method for detecting
hallucinations and out-of-distribution behavior in large
language and
vision-
language models. EigenTrack transforms streaming activations into spectral descriptors such as
entropy, variance, and deviations from the Marchenko-Pastur baseline, and models their temporal
evolution using lightweight recurrent classifiers, enabling early detection of reliability failures before they appear in model outputs while offering interpretable insight into representation dynamics. The second contribution, RMT-KD, presents a principled approach to compressing deep networks via random
matrix theoretic knowledge
distillation. By interpreting outlier
eigenvalues in activation spectra as carriers of task-relevant information, RMT-KD progressively projects networks onto lower-dimensional subspaces through iterative self-
distillation, yielding significantly more compact and energy-efficient models while preserving accuracy and dense, hardware-friendly structure.
Read the full abstract ▾
Machine Learning
A 1/R Law for Kurtosis Contrast in Balanced Mixtures
Yuda Bi, Wenjun Xiao, Linhao Bai et al. (+1 more)
Kurtosis-based Independent Component Analysis (ICA) weakens in wide, balanced mixtures. We prove a sharp redundancy law: for a standardized projection with effective width $R_{\mathrm{eff}}$ (participation ratio), the population excess kurtosis obeys $|\kappa(y)|=O(\kappa_{\max}/R_{\mathrm{eff}})$, yielding the order-tight $O(c_b\kappa_{\max}/R)$ under balance (typically $c_b=O(\log R)$). As an impossibility screen, under standard finite-moment conditions for sample kurtosis estimation, surpassing the $O(1/\sqrt{T})$ estimation scale requires $R\lesssim \kappa_{\max}\sqrt{T}$. We also show that purification – selecting $m\!\ll\!R$ sign-consistent sources – restores $R$-independent contrast $\Omega(1/m)$, with a simple data-driven heuristic. Synthetic experiments validate the predicted decay, the $\sqrt{T}$ crossover, and contrast recovery.
Read the full abstract ▾
All authors Yuda Bi, Wenjun Xiao, Linhao Bai, Vince D Calhoun
Machine Learning
Training Agents to Self-Report Misbehavior
Bruce W. Lee, Chen Yueh-Han and Tomek Korbak
Frontier AI
agents may pursue hidden goals while concealing their pursuit from oversight.
Alignment training aims to prevent such behavior by reinforcing the correct goals, but
alignment may not always succeed and can lead to unwanted side effects. We propose self-incrimination training, which instead trains
agents to produce a visible signal when they covertly misbehave. We train GPT-4.1 and Gemini-2.0
agents to call a report_scheming() tool when behaving deceptively and measure their ability to cause harm undetected in out-of-distribution environments. Self-incrimination significantly reduces the undetected successful attack rate, outperforming matched-capability monitors and
alignment baselines while preserving instruction hierarchy and incurring minimal safety tax on general capabilities. Unlike blackbox monitoring, self-incrimination performance is consistent across tasks regardless of how suspicious the misbehavior appears externally. The trained behavior persists under
adversarial prompt optimization and generalizes to settings where
agents pursue misaligned goals themselves rather than being instructed to misbehave. Our results suggest self-incrimination offers a viable path for reducing frontier misalignment risk, one that neither assumes misbehavior can be prevented nor that it can be reliably classified from the outside.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.DS
Testable Learning of General Halfspaces under Massart Noise
Ilias Diakonikolas, Giannis Iakovidis, Daniel M. Kane et al. (+1 more)
We study the algorithmic task of testably learning general Massart halfspaces under the Gaussian distribution. In the testable learning setting, the aim is the design of a tester-learner pair satisfying the following properties: (1) if the tester accepts, the learner outputs a hypothesis and a certificate that it achieves near-optimal error, and (2) it is highly unlikely that the tester rejects if the data satisfies the underlying assumptions. Our main result is the first testable learning
algorithm for general halfspaces with Massart noise and Gaussian marginals. The complexity of our
algorithm is $d^{\mathrm{polylog}(\min\{1/\gamma, 1/\epsilon \})}$, where $\epsilon$ is the excess error and $\gamma$ is the bias of the target halfspace, which qualitatively matches the known quasi-
polynomial Statistical Query lower bound for the non-testable setting. The analysis of our
algorithm hinges on a novel sandwiching
polynomial approximation to the sign function with multiplicative error that may be of broader interest.
Read the full abstract ▾
All authors Ilias Diakonikolas, Giannis Iakovidis, Daniel M. Kane, Sihan Liu
Machine Learning · Cross-listed from cs.MM
Decoding the Hook: A Multimodal LLM Framework for Analyzing the Hooking Period of Video Ads
Kunpeng Zhang, Poppy Zhang, Shawndra Hill et al. (+1 more)
Video-based ads are a vital medium for brands to engage consumers, with social media platforms leveraging user data to optimize ad delivery and boost engagement. A crucial but under-explored aspect is the 'hooking period', the first three seconds that capture viewer
attention and influence engagement metrics. Analyzing this brief window is challenging due to the
multimodal nature of
video content, which blends visual, auditory, and textual elements. Traditional methods often miss the nuanced interplay of these components, requiring advanced frameworks for thorough evaluation. This study presents a framework using
transformer-based
multimodal large
language models (MLLMs) to analyze the hooking period of
video ads. It tests two frame sampling strategies, uniform random sampling and key frame selection, to ensure balanced and representative acoustic feature extraction, capturing the full range of design elements. The hooking
video is processed by state-of-the-art MLLMs to generate descriptive analyses of the ad's initial impact, which are distilled into coherent topics using BERTopic for high-level abstraction. The framework also integrates features such as audio attributes and aggregated ad targeting information, enriching the feature set for further analysis. Empirical validation on large-scale real-world data from social media platforms demonstrates the efficacy of our framework, revealing correlations between hooking period features and key performance metrics like conversion per investment. The results highlight the practical applicability and predictive power of the approach, offering valuable insights for optimizing
video ad strategies. This study advances
video ad analysis by providing a scalable methodology for understanding and enhancing the initial moments of
video advertisements.
Read the full abstract ▾
All authors Kunpeng Zhang, Poppy Zhang, Shawndra Hill, Amel Awadelkarim
Machine Learning
AviaSafe: A Physics-Informed Data-Driven Model for Aviation Safety-Critical Cloud Forecasts
Zijian Zhu, Qiusheng Huang, Anboyu Guo et al. (+2 more)
Current AI weather
forecasting models predict conventional atmospheric variables but cannot distinguish between cloud microphysical species critical for aviation safety. We introduce AviaSafe, a hierarchical, physics-informed neural forecaster that produces global, six-hourly predictions of these four hydrometeor species for lead times up to 7 days. Our approach addresses the unique challenges of cloud prediction: extreme sparsity, discontinuous distributions, and complex microphysical interactions between species. We integrate the Icing Condition (IC) index from aviation meteorology as a physics-based constraint that identifies regions where supercooled water fuels explosive ice
crystal growth. The model employs a hierarchical architecture that first predicts cloud spatial distribution through masked
attention, then quantifies species concentrations within identified regions. Training on ERA5 reanalysis data, our model achieves lower RMSE for cloud species compared to baseline and outperforms operational numerical models on certain key variables at 7-day lead times. The ability to forecast individual cloud species enables new applications in aviation route optimization where distinguishing between ice and
liquid water determines engine icing risk.
Read the full abstract ▾
All authors Zijian Zhu, Qiusheng Huang, Anboyu Guo, Xiaohui Zhong, Hao Li
Machine Learning
Learning Rewards, Not Labels: Adversarial Inverse Reinforcement Learning for Machinery Fault Detection
Dhiraj Neupane, Richard Dazeley, Mohamed Reda Bouadjenek et al. (+1 more)
Reinforcement learning (RL) offers significant promise for machinery fault detection (MFD). However, most existing RL-based MFD approaches do not fully exploit RL's sequential decision-making strengths, often treating MFD as a simple guessing game (Contextual Bandits). To bridge this gap, we formulate MFD as an offline inverse reinforcement learning problem, where the
agent learns the reward dynamics directly from healthy operational sequences, thereby bypassing the need for manual reward engineering and fault labels. Our framework employs
Adversarial Inverse Reinforcement Learning to train a discriminator that distinguishes between normal (expert) and policy-generated transitions. The discriminator's learned reward serves as an anomaly score, indicating deviations from normal operating behaviour. When evaluated on three run-to-failure
benchmark datasets (HUMS2023, IMS, and XJTU-SY), the model consistently assigns low anomaly scores to normal samples and high scores to faulty ones, enabling early and robust fault detection. By aligning RL's sequential
reasoning with MFD's temporal structure, this work opens a path toward RL-based diagnostics in data-driven industrial settings.
Read the full abstract ▾
All authors Dhiraj Neupane, Richard Dazeley, Mohamed Reda Bouadjenek, Sunil Aryal
Machine Learning
UpSkill: Mutual Information Skill Learning for Structured Response Diversity in LLMs
Devan Shah, Owen Yang, Daniel Yang et al. (+2 more)
Reinforcement Learning with Verifiable Rewards (RLVR) has improved the
reasoning abilities of large
language models (LLMs) on mathematics and programming tasks, but standard approaches that optimize single-attempt accuracy can inadvertently suppress response diversity across repeated attempts, narrowing exploration and overlooking underrepresented strategies. We introduce UpSkill, a training time method that adapts Mutual Information Skill Learning (MISL) to LLMs for optimizing pass@k correctness. We propose a novel reward that we implement within Group Relative Policy Optimization (GRPO): a
token-level mutual information (MI) reward that encourages trajectory specificity to z. Experiments on GSM8K with three open-weight models, Llama 3.1-8B, Qwen 2.5-7B, and R1-Distilled-Qwen2.5-Math-1.5B, show that UpSkill improves multi-attempt metrics on the stronger base models, yielding mean gains of ~3% in pass@k for both Qwen and Llama without degrading pass@1. Additionally, we find both empirical and theoretical evidence that improvements in pass@k are closely tied to the mutual information objective.
Read the full abstract ▾
All authors Devan Shah, Owen Yang, Daniel Yang, Chongyi Zheng, Benjamin Eysenbach
Machine Learning
When Should a Model Change Its Mind? An Energy-Based Theory and Regularizer for Concept Drift in Electrocardiogram (ECG) Signals
Timothy Oladunni, Blessing Ojeme, Kyndal Maclin et al. (+1 more)
Models operating on dynamic physiologic signals must distinguish benign, label-preserving variability from true concept change. Existing concept-drift frameworks are largely distributional and provide no principled guidance on how much a model's internal representation may move when the underlying signal undergoes physiologically plausible fluctuations in energy. As a result, deep models often misinterpret harmless changes in amplitude, rate, or morphology as concept drift, yielding unstable predictions, particularly in
multimodal fusion settings. This study introduces Physiologic Energy Conservation Theory (PECT), an energy-based framework for concept stability in dynamic signals. PECT posits that under virtual drift, normalized latent displacement should scale proportionally with normalized signal energy change, while persistent violations of this proportionality indicate real concept drift. We operationalize this principle through Energy-Constrained Representation Learning (ECRL), a lightweight regularizer that penalizes energy-inconsistent latent movement without modifying encoder architectures or adding inference-time cost. Although PECT is formulated for dynamic signals in general, we instantiate and evaluate it on
multimodal ECG across seven unimodal and hybrid models. Experiments show that in the strongest trimodal hybrid (1D+2D+
Transformer), clean accuracy is largely preserved (96.0% to 94.1%), while perturbed accuracy improves substantially (72.6% to 85.5%) and fused representation drift decreases by over 45%. Similar trends are observed across all architectures, providing empirical evidence that PECT functions as an energy-drift law governing concept stability in continuous physiologic signals.
Read the full abstract ▾
All authors Timothy Oladunni, Blessing Ojeme, Kyndal Maclin, Clyde Baidoo
Machine Learning
Learning global river system hydrodynamics with a topology-informed foundation model
Hancheng Ren, Gang Zhao, Shuo Wang et al. (+12 more)
Accurate river prediction is essential for water, food, and energy security, yet extending it reliably from sparse in-situ gauges to the full river network remains an open challenge. Although machine learning has transformed Earth-system modelling, the same sparse and uneven observations have largely confined its advances for rivers to discrete basins rather than connected river systems. Here we introduce GraphRiverCast (GRC), a
topology-informed river foundation model for reach-level prediction across the global river network. Exploiting the connectivity and energetics of rivers, it is trained on sparse gauges and predicts daily multivariate hydrodynamics with no prior river conditions. GRC outperforms strong physics-based and machine-learning models. Attribution experiments show that, compared with standard autoregressive models that rely more on temporal recurrence, GRC reconstructs river states from
topology and forcing, learning the dynamics themselves and inheriting the self-correcting character of river systems. Its skill therefore barely decays with lead time, even far beyond its training horizon. It also generalises to ungauged reaches and finer-resolution networks, bringing reliable hydrodynamic prediction to poorly monitored regions. More broadly, harnessing the structure and energetics of a system offers a strategy for machine learning in other observation-limited Earth systems.
Read the full abstract ▾
All authors Hancheng Ren, Gang Zhao, Shuo Wang, Louise Slater, Dai Yamazaki, Shu Liu, Jingfang Fan, Xueying Li, Shibo Cui, Ziming Yu, Shengyu Kang, Depeng Zuo, Dingzhi Peng, Zongxue Xu, Bo Pang
Machine Learning
Manifold of Failure: Behavioral Attraction Basins in Language Models
Sarthak Munshi, Manish Bhatt, Vineeth Sai Narajala et al. (+4 more)
While prior work has focused on projecting
adversarial examples back onto the
manifold of natural data to restore safety, we argue that a comprehensive understanding of AI safety requires characterizing the unsafe regions themselves. This paper introduces a framework for systematically mapping the
Manifold of Failure in Large
Language Models (LLMs). We reframe the search for vulnerabilities as a quality diversity problem, using MAP-Elites to illuminate the continuous
topology of these failure regions, which we term behavioral attraction basins. Our quality metric,
Alignment Deviation, guides the search towards areas where the model's behavior diverges most from its intended
alignment. Across three LLMs: Llama-3-8B, GPT-OSS-20B, and GPT-5-Mini, we show that MAP-Elites achieves up to 63% behavioral coverage, discovers up to 370 distinct vulnerability niches, and reveals dramatically different model-specific topological signatures: Llama-3-8B exhibits a near-universal vulnerability plateau (mean
Alignment Deviation 0.93), GPT-OSS-20B shows a fragmented landscape with spatially concentrated basins (mean 0.73), and GPT-5-Mini demonstrates strong robustness with a ceiling at 0.50. Our approach produces interpretable, global maps of each model's safety landscape that no existing attack method (GCG, PAIR, or TAP) can provide, shifting the paradigm from finding discrete failures to understanding their underlying structure.
Read the full abstract ▾
All authors Sarthak Munshi, Manish Bhatt, Vineeth Sai Narajala, Idan Habler, Ammar Al-Kahfah, Ken Huang, Blake Gatto
Machine Learning · Cross-listed from q-bio.QM
What Topological and Geometric Structure Do Biological Foundation Models Learn? Evidence from 141 Hypotheses
Ihor Kendiukhov
When biological foundation models such as scGPT and Geneformer process single-
cell gene expression, what
geometric and topological structure forms in their internal representations? Is that structure biologically meaningful or a training artifact, and how confident should we be in such claims? We address these questions through autonomous large-scale hypothesis screening: an AI-driven executor-brainstormer loop that proposed, tested, and refined 141
geometric and topological hypotheses across 52 iterations, covering persistent homology,
manifold distances, cross-model
alignment, community structure, and directed
topology, all with explicit null controls and disjoint gene-pool splits. Three principal findings emerge. First, the models learn genuine
geometric structure. Gene
embedding neighborhoods exhibit non-trivial
topology, with persistent homology significant in 11 of 12
transformer layers at p < 0.05 in the weakest domain and 12 of 12 in the other two. A multi-level distance hierarchy shows that
manifold-aware metrics outperform Euclidean distance for identifying regulatory gene pairs, and graph community partitions track known transcription factor target relationships. Second, this structure is shared across independently trained models. CCA
alignment between scGPT and Geneformer yields canonical correlation of 0.80 and gene
retrieval accuracy of 72 percent, yet none of 19 tested methods reliably recover gene-level correspondences. The models agree on the global shape of gene space but not on precise gene placement. Third, the structure is more localized than it first appears. Under stringent null controls applied across all null families, robust signal concentrates in immune tissue, while lung and external lung signals weaken substantially.
Read the full abstract ▾
Machine Learning
Reliable XAI Explanations in Sudden Cardiac Death Prediction for Chagas Cardiomyopathy
Vinícius P. Chagas, Luiz H. T. Viana, Mac M. da S. Carlos et al. (+4 more)
Sudden cardiac death (SCD) is unpredictable, and its prediction in Chagas cardiomyopathy (CC) remains a significant challenge, especially in patients not classified as high risk. While AI and machine learning models improve risk stratification, their adoption is hindered by a lack of transparency, as they are often perceived as black boxes with unclear decision-making processes. Some approaches apply heuristic explanations without correctness guarantees, leading to mistakes in the decision-making process. To address this, we apply a logic-based explainability method with correctness guarantees to the problem of SCD prediction in CC. This explainability method, applied to an AI classifier with over 95% accuracy and recall, demonstrated strong predictive performance and 100% explanation fidelity. When compared to state-of-the-art heuristic methods, it showed superior consistency and robustness. This approach enhances clinical trust, facilitates the integration of AI-driven tools into practice, and promotes large-scale deployment, particularly in endemic regions where it is most needed.
Read the full abstract ▾
All authors Vinícius P. Chagas, Luiz H. T. Viana, Mac M. da S. Carlos, João P. V. Madeiro, Roberto C. Pedrosa, Thiago Alves Rocha, Carlos H. L. Cavalcante
Machine Learning · Cross-listed from cs.AI
Willem Schooltink and Fabio Massimo Zennaro
Abstractions of causal models allow for the coarsening of models such that relations of cause and effect are preserved. Whereas abstractions focus on the relation between two models, in this paper we study a framework for causal
embeddings which enable multiple detailed models to be mapped into sub-systems of a coarser causal model. We define causal
embeddings as a generalization of abstraction, and present a generalized notion of consistency. By defining a multi-resolution marginal problem, we showcase the relevance of causal
embeddings for both the statistical marginal problem and the causal marginal problem; furthermore, we illustrate its practical use in merging
datasets coming from models with different representations.
Read the full abstract ▾
Machine Learning
OmniZip: Learning a Unified and Lightweight Lossless Compressor for Multi-Modal Data
Yan Zhao, Zhengxue Cheng, Junxuan Zhang et al. (+4 more)
Lossless compression is essential for efficient data storage and transmission. Although learning-based lossless compressors achieve strong results, most of them are designed for a single modality, leading to redundant compressor deployments in multi-modal settings. Designing a unified multi-modal compressor is critical yet challenging, as different data types vary largely in format, dimension, and statistics. Multi-modal large
language models offer a promising resolution but remain too complex for practical use. Thus, we propose OmniZip, a unified and lightweight lossless compressor for multi-modal data (like image, text,
speech, tactile, database, and gene sequence). Built on a lightweight backbone, OmniZip incorporates three key components to enable efficient multi-modal lossless compression: a modality-unified tokenizer that reversibly transforms diverse data into
tokens, a modality-routing context learning mechanism that enables flexible multi-modal context modeling, and a modality-routing feedforward design that further enhances the model's nonlinear representation flexibility. A reparameterization training strategy is used to enhance model capacity. OmniZip outperforms or matches other state-of-the-art compressors on multiple modalities, achieving 42%, 57%, 62% and 42%, 53% higher compression efficiency than gzip on CLIC-M, TouchandGo, enwik9, LibriSpeech, and WikiSQL
datasets, respectively. It also supports near real-time inference on resource-constrained edge devices, reaching about 1MB/s on MacBook CPUs and iPhone NPUs. Our code is released at https://github.com/adminasmi/OmniZip-CVPR2026.
Read the full abstract ▾
All authors Yan Zhao, Zhengxue Cheng, Junxuan Zhang, Dajiang Zhou, Qunshan Gu, Qi Wang, Li Song
Machine Learning
Early Risk Stratification of Dosing Errors in Clinical Trials Using Machine Learning
Félicien Hêche, Sohrab Ferdowsi, Anthony Yazdani et al. (+2 more)
Objective: The objective of this study is to develop a machine learning (ML)-based framework for early risk stratification of clinical trials (CTs) according to their likelihood of exhibiting a high rate of dosing errors, using information available prior to trial initiation. Materials and Methods: We constructed a
dataset from ClinicalTrials.gov comprising 42,112 CTs. Structured, semi-structured trial data, and unstructured protocol-related free-text data were extracted. CTs were assigned binary labels indicating elevated dosing error rate, derived from adverse event reports, MedDRA terminology, and Wilson confidence intervals. We evaluated an XGBoost model trained on structured features, a ClinicalModernBERT model using textual data, and a simple late-fusion model combining both modalities. Post-hoc probability calibration was applied to enable interpretable, trial-level risk stratification. Results: The late-fusion model achieved the highest AUC-ROC (0.862). Beyond discrimination, calibrated outputs enabled robust stratification of CTs into predefined risk categories. The proportion of trials labeled as having an excessively high dosing error rate increased monotonically across higher predicted risk groups and aligned with the corresponding predicted probability ranges. Discussion: These findings indicate that dosing error risk can be anticipated at the trial level using pre-initiation information. Probability calibration was essential for translating model outputs into reliable and interpretable risk categories, while simple
multimodal integration yielded performance gains without requiring complex architectures. Conclusion: This study introduces a reproducible and scalable ML framework for early, trial-level risk stratification of CTs at risk of high dosing error rates, supporting proactive, risk-based quality management in clinical research.
Read the full abstract ▾
All authors Félicien Hêche, Sohrab Ferdowsi, Anthony Yazdani, Sara Sansaloni-Pastor, Douglas Teodoro
Machine Learning
BrepCoder: A Unified Multimodal Large Language Model for Multi-task B-rep Reasoning
Mingi Kim, Yongjun Kim, Jungwoo Kang et al. (+1 more)
Recent advancements in deep learning have actively addressed complex challenges within the Computer-Aided Design (CAD) domain.However, most existing approaches rely on task-specifi c models requiring structural modifi cations for new tasks, and they predominantly focus on point clouds or images rather than the industry-standard Boundary Representation (B-rep) format. To address these limitations, we propose BrepCoder, a unifi ed
Multimodal Large
Language Model (MLLM) that performs diverse CAD tasks from B-rep inputs. By leveraging the code generation capabilities of Large
Language Models (LLMs), we convert CAD modeling sequences into Python-like code and align them with B-rep. We then adopt a two-stage training strategy: First, pre-training on reverse engineering to learn
geometric features and design logic. Second, eff ectively extending the model to various downstream tasks such as completion, error correction, and CAD-QA. Consequently, by interpreting B-rep as structural code, BrepCoder achieves superior generalization across diverse tasks, demonstrating its potential as a general-purpose CAD
agent.
Read the full abstract ▾
All authors Mingi Kim, Yongjun Kim, Jungwoo Kang, Hyungki Kim
Machine Learning · Cross-listed from cs.CR
Differentially Private Truncation of Unbounded Data via Public Second Moments
Zilong Cao, Xuan Bi and Hai Zhang
Data
privacy is important in the AI era, and differential
privacy (DP) is one of the golden solutions. However, DP is typically applicable only if data have a bounded underlying distribution. We address this limitation by leveraging second-moment information from a small amount of public data. We propose Public-moment-guided Truncation (PMT), which transforms private data using the public second-moment
matrix and applies a principled truncation whose radius depends only on non-private quantities: data dimension and sample size. This transformation yields a well-conditioned second-moment
matrix, enabling its inversion with a significantly strengthened ability to resist the DP noise. Furthermore, we demonstrate the applicability of PMT by using penalized and generalized linear regressions. Specifically, we design new loss functions and
algorithms, ensuring that solutions in the transformed space can be mapped back to the original domain. We have established improvements in the models' DP estimation through theoretical error bounds, robustness guarantees, and convergence results, attributing the gains to the conditioning effect of PMT. Experiments on synthetic and real
datasets confirm that PMT substantially improves the accuracy and stability of DP models.
Read the full abstract ▾