Machine Learning · Cross-listed from physics.comp-ph
Multi-Fidelity Learning with Shallow Recurrent Decoders for Multi-Physics Applications
Stefano Riva, Carolina Introini, J. Nathan Kutz et al. (+1 more)
In reactor physics, neutronics and multi-physics phenomena can be modelled at different fidelity levels. High-fidelity models based on the Boltzmann transport equation, multi-group
diffusion, or computational fluid dynamics are computationally demanding, whereas simplified models, such as zero-dimensional lumped formulations, can be evaluated efficiently at the cost of neglecting spatial details. The computational intractability of detailed models translates into a scarcity of high-fidelity data and an abundance of low-fidelity data, motivating the development of multi-fidelity (MF) learning strategies able to map between the two. This work extends Shallow Recurrent Decoders (SHRED), a machine learning architecture that reconstructs high-dimensional fields from time-series measurements, to multi-fidelity reduced-order modelling, in which the input trajectories are provided by a low-fidelity model. The resulting MF-SHRED is assessed on three
benchmark problems: (i) a two-group point-kinetics-to-
diffusion; (ii) a non-linear reaction-advection-
diffusion system of six chemical species; and (iii) the coupled neutronics-thermal-hydraulics multi-physics model of the Molten Salt Fast Reactor (MSFR). Across all three cases, MF-SHRED reconstructs the high-fidelity fields with relative errors below a few percent, closely approaching the truncation error of the underlying proper orthogonal decomposition, while reducing the computational cost by three orders of magnitude relative to the corresponding high-fidelity solver. MF-SHRED performs comparably to the original sparse-sensor SHRED formulation. These results support the use of MF-SHRED as a non-intrusive reduced-order modelling strategy for reactor physics and multi-physics applications, particularly for design and safety analysis tasks that must be carried out before a facility is even built.
Read the full abstract ▾
All authors Stefano Riva, Carolina Introini, J. Nathan Kutz, Antonio Cammi
Machine Learning
State commitment learning: training language models to distinguish computation from memory
Fei Ding, Yongkang Zhang, Runhao Liu et al. (+3 more)
Reasoning language models do not distinguish
tokens used for computation from
tokens that constitute persistent state: once generated, all hidden thoughts remain in context and influence future predictions. As a result, downstream
reasoning may depend on failed attempts, dead ends, and private scratch work that should not be safely relied on later. We recast this phenomenon as a new training objective, state commitment learning: training models to explicitly distinguish information that should be committed as persistent state from temporary computation that can be discarded. We define a counterfactual criterion, persistent-state sufficiency, which makes it trainable and measurable whether an answer remains usable after hidden thoughts are erased. We then propose Counterfactual Erasure RL (CERL), which evaluates, under the same prefix, both a path that keeps hidden thoughts and a path that erases them, and gives reward only when the erasure path remains correct. We also introduce the Erasure Dependence Protocol and show across mathematics, long-chain logic, scientific QA, and multi-turn tool-use evaluation that CERL substantially reduces answer dependence on hidden thoughts without sacrificing accuracy, consistently outperforming correctness-only RL and long-answer SFT baselines.
Read the full abstract ▾
All authors Fei Ding, Yongkang Zhang, Runhao Liu, Yuhao Liao, Zijian Zeng, Huiming Yang
Machine Learning · Cross-listed from physics.comp-ph
A differentiable machine learning small-angle X-ray scattering analysis framework for structure elucidation of lipid nanoparticles
Maria Bånkestad, Sandra Barman, Magnus Röding et al. (+11 more)
Lipid nanoparticles (LNPs) are efficient delivery systems for negatively charged nucleic acids. Their multi-component architecture yields a core-shell structure. Small-angle X-ray scattering (SAXS) is an important characterization technique for LNPs, but recovering internal structure and size distribution from SAXS is an inverse problem with non-unique solutions. Realistic models are often too expensive for systematic exploration. We introduce a machine-learning-accelerated, differentiable framework for SAXS analysis of heterogeneous, polydisperse LNPs. The forward model combines a core-shell particle with a Gaussian random-field interior, a neural surrogate for the monodisperse SAXS map, and a differentiable layer integrating over particle-size distributions. The surrogate reduces prediction cost by four orders of magnitude, while differentiability enables large-scale multi-start fitting and ensemble identifiability analysis. Applied to synthetic and experimental MC3 LNP data, the framework shows that near-identical SAXS fits can arise from distinct parameter modes, with the experimental fits dominated by a trade-off between size-distribution and interior-structure parameters.
Read the full abstract ▾
All authors Maria Bånkestad, Sandra Barman, Magnus Röding, Erik Kaunisto, Viktoriia Meklesh, Audrey Gallud, Marco Mendez, Marianna Yanez Arteta, Stefan Norberg, Ann Terry, Smita Chakraborty, Shun Yu, Jerk Rönnols, Sepideh Pashami
Machine Learning · Cross-listed from q-bio.BM
An accurate nucleic acid-small molecule docking framework via geometric deep learning with large-scale pretraining
Shi Li, Xujun Zhang, Mingquan Liu et al. (+5 more)
Nucleic acids are increasingly recognized as therapeutic targets beyond conventional
protein-centered drug discovery, yet accurate and efficient docking of small molecules to nucleic acid structures remains challenging. Physics-based docking methods often show limited accuracy and efficiency, whereas deep learning approaches are constrained by the scarcity of experimentally resolved nucleic acid-ligand complexes. Here, we present NucleoDock, a deep learning framework for nucleic acid-small molecule docking. To address data scarcity, NucleoDock combines physics-guided large-scale pretraining on millions of docking-generated synthetic complexes with fine-tuning on curated experimental co-
crystal structures. It further integrates sequence- and structure-informed nucleotide representations with atomistic three-dimensional features to capture both biological context and binding-site geometry. A mixture density network-based
geometric scoring head is used to model conditional interaction-distance distributions for pose ranking. On an external
benchmark of 125 nucleic acid-ligand complexes, NucleoDock achieved a top-1 success rate of 56 percent at an RMSD cutoff of 2.0 Angstrom, outperforming rDock with 29 percent, while generating 100 poses in approximately 5 seconds per complex. Retrospective virtual screening on the ROBIN
benchmark further showed improved early enrichment. NucleoDock represents a step toward bridging the methodological gap between
protein- and nucleic acid-directed computational drug discovery.
Read the full abstract ▾
All authors Shi Li, Xujun Zhang, Mingquan Liu, Hui Zhang, Shuoying Jia, Yu Kang, Tingjun Hou, Peichen Pan
Machine Learning
Making Brain-Computer Interfaces More Secure
Md Fahimul Kabir Chowdhury and Gahangir Hossain
The development of
brain-computer interfaces (BCIs) based on electroencephalograms (EEGs) has advanced significantly mainly to machine learning. Although the majority of earlier research has been on increasing classification accuracy, relatively little focus has been placed on security and robustness. According to recent research,
EEG-based BCIs are susceptible to
adversarial attacks, which can cause misdiagnosis due to minute, well-crafted disturbances. Evaluating model robustness against such perturbations is therefore critical for ensuring reliable deployment. In this study, we propose a lightweight custom Convolutional Neural Network (CNN) architecture to investigate
adversarial robustness in
EEG-based BCIs. The suggested method is assessed using two
EEG datasets and contrasted with three novel CNN models tailored to
EEG, namely EEGNet, DeepConvNet, and SleepEEGNet, under gradient-based
adversarial attack scenarios. According to experimental findings, the suggested model continuously performs better in classification under
adversarial perturbations compared to baseline models, indicating improved robustness. These findings highlight the potential of lightweight architectures for enhancing the reliability of
EEG-based BCI systems under
adversarial conditions.
Read the full abstract ▾
Machine Learning
Spectral Asymptotics of Neural Network Loss Landscapes: An Exact Decomposition of the Curvature Exponent
Anherutowa Calvo
The
curvature exponent $\alpha$ in $h_k \propto \sigma_k^\alpha$ – governing how Hessian
eigenvalues scale with gradient singular values – varies systematically across layer types ($\alpha \approx 2$ for convolutions, $\approx 1$ for
transformer attention, $< 1$ for MLP up-projections). Why? We prove the Spectral
Alignment Decomposition: $\alpha = 2 + d\log\Phi_k / d\log\sigma_k$, where $\Phi_k$ measures
alignment between Kronecker factor eigenbases and gradient singular directions. This reduces "why does $\alpha$ vary?" to a
geometric question we answer for LayerNorm, residual connections, and softmax heads. The decomposition implies a spectral transfer identity $s = \alpha\gamma$ linking
curvature exponent, effective gradient rank-decay $\gamma$, and Hessian decay exponent $s$. The identity is algebraic; its empirical content is that $\alpha$ and $\gamma$, fit on independent data (HVPs vs. SVD), recover $s$ to ~2% median error across 93 layers, five architectures, and three
datasets – with no free parameters. A
zeta-function bound on participation ratio shows
curvature concentrates onto effectively one direction per layer. As a proof of concept, we derive the architecture-adaptive preconditioner $T(\sigma;\alpha)$ and show that Spectral Newton – implementing $T$ in the gradient singular basis – outperforms AdamW on
vision benchmarks where $\alpha \approx 2$.
Read the full abstract ▾
Machine Learning
Human-in-the-Loop Contextual Bandits for Short-Term Rental Dynamic Pricing: Structural Equivalence of Historical Warm-Up and Approval-Gated Live Learning
Oleg Miroshnichenko
Dynamic pricing in short-term rental (STR)
markets presents a distinctive challenge for online learning
algorithms: pricing decisions carry significant financial risk, operators require explainability, and
market feedback is sparse (one booking outcome per listed night). We introduce the Human-in-the-Loop Gated Bandit (HITL-GB) framework, in which a contextual bandit
algorithm generates price recommendations but a human
agent retains authority to accept, modify, or reject each recommendation before it is applied. We show that under this approval constraint, historical pricing data – collected under a prior deterministic policy – is structurally equivalent to on-policy warm-up data for initialising the bandit's posterior, bypassing the weeks-to-months cold-start period that renders pure online bandit learning impractical in sparse-feedback
markets. We formalise the approval-gated reward signal, derive a regularised ridge-regression warm-up procedure from historical episodes, and validate the approach on real STR production data (anonymised urban
market, 2 rooms, April 2022 – April 2026, 1,461 nightly pricing episodes). Our warm-up procedure compresses effective cold-start from ~150 episodes to ~30 episodes when initialising
agents from the Hierarchical Factored Thompson Sampling (HF-TS) family. We further argue that the structural equivalence result is domain-agnostic: any high-stakes domain where human approval is legally or operationally required – including clinical drug dosing, credit origination, content moderation, and radiological diagnosis – satisfies the same conditions and benefits from the same warm-up strategy. In regulated industries, mandatory human oversight is thus a statistical asset rather than a deployment constraint.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.IR
SentimentLens: Reconciling Sentiment and Ratings via Dual-Modality in the Hospitality Sector
Dineth Jayakody, Pasindu Thenahandi and Sampath Jayarathna
Online travel platforms generate vast volumes of user-generated hotel reviews, offering rich opportunities to understand traveler experiences at scale. However, transforming unstructured textual feedback into structured, actionable insights remains a challenging task. This paper presents SentimentLens, a scalable analysis system based on Aspect-Based Sentiment Analysis that performs knowledge extraction from unstructured hotel reviews and organizes them into interpretable service categories. SentimentLens integrates aspect term extraction, aspect sentiment classification, semantic category assignment, and multi-level analytical modules to support region-level, hotel-level, and category-level evaluation. The system is designed to operate across different geographic contexts and hospitality settings. To demonstrate its practical utility, we apply SentimentLens to a large real-world
dataset of over 10,000 publicly available hotel reviews. Through extensive analysis, the framework reveals how traveler sentiment varies across regions, service categories, and hotel archetypes. We further implement a cross-modal reconciliation of textual sentiment and numerical ratings to identify latent operational conflicts, structural inconsistencies in service quality, and high-impact improvement opportunities using importance–performance and
entropy-based analyses. The results show that SentimentLens effectively transforms large-scale unstructured reviews into actionable intelligence, supporting data-driven decision-making for hospitality management and tourism policy. While demonstrated using a national case study, the proposed system is generalizable to other destinations and review-driven service domains.
Read the full abstract ▾
Machine Learning
From Demonstrations to Rewards: Test-Time Prompt Optimization for VLM Reward Models
Christian Gumbsch, Leonardo Barcellona, Lennard Schünemann et al. (+7 more)
Reinforcement learning relies on accurate reward functions, which are often hand-crafted or even unavailable in real-world applications, such as robotics. Recent work has explored the zero-shot
reasoning capabilities of pre-trained
Vision-
Language Models (VLMs) as reward models. However, without careful
prompt engineering, these approaches tend to produce suboptimal rewards, where false positive predictions can severely degrade downstream policy learning. In robotics, limited
datasets comprising expert demonstrations are often collected to bootstrap policy learning. This scenario provides an opportunity to optimize a reward model prior policy training. We propose Demo2Reward a test-time adaptation technique to optimize the
language instruction of a reward model based on a few demonstrations (3-10 trajectories) to reduce false positives while preserving true positives. Crucially, this requires no additional model training or computation resources during policy learning. We show that Demo2Reward consistently outperforms existing zero- and few-shot VLM reward models across a range of simulated robotic tasks and policy backbones. Finally, we demonstrate that Demo2Reward effectively transfers to a real-world robotic learning scenario, enabling policy learning without manually engineering a reward function.
Read the full abstract ▾
All authors Christian Gumbsch, Leonardo Barcellona, Lennard Schünemann, Platon Karageorgis, Andrii Zadaianchuk, Zehao Wang, Sergey Zakharov, Fabien Despinoy, Rahaf Aljundi, Efstratios Gavves
Machine Learning
Hoeffding Concept Bottleneck Models with Applications to Overhead Images
Clément Bénard, Manon Arfib, Christophe Labreuche et al. (+1 more)
Explainability of deep learning
algorithms is critical for computer-
vision applications with high-stakes decisions. Concept bottleneck models (CBM) have recently shown promising performance to provide explainable and accurate predictions for classification problems, based on a bottleneck of high-level concepts. Existing CBM methods rely on a linear aggregation of the concept scores to compute predictions. However, a large number of concepts is often used in this linear approach, which undermines explainability and favors information leakage. In general, the underlying relation between concepts and output logits is not linear. Therefore, we introduce Hoeffding Concept Bottleneck Models (HCBM), which build on the Hoeffding functional decomposition of gradient-boosted trees to provide non-linear and sparse aggregations of concept scores, and generate compact predictions using
prime implicants. HCBM have strong theoretical properties of robustness to interconcept leakage, and outperform sparse linear CBM in practice, as shown in extensive experiments. Beyond classification, HCBM can be adapted to object detection, and we focus on a challenging case with overhead images to show the high performance of HCBM in these settings.
Read the full abstract ▾
All authors Clément Bénard, Manon Arfib, Christophe Labreuche, Victor Quétu
Machine Learning
DAStatFormer: A Hybrid Multibranch Transformer with Statistical Feature Integration for DAS-Based Pattern Recognitions
Michel Dione, Jerry Lonlac, Hélène Louis et al. (+2 more)
Distributed Acoustic Sensing (DAS) enables large-scale monitoring through optical fibers, but its high dimensionality and complex spatio-temporal patterns make event classification demanding. Existing deep learning approaches-CNNs, recurrent models, and
Transformer variants-either fail to capture long-range dependencies or require processing raw DAS
matrices at prohibitive cost. We propose DAStatFormer, a hybrid multibranch
Transformer that combines compact multidomain statistical features with Gated
Transformer Networks. Instead of raw signals, we extract 24 ANOVA-selected attributes per channel from the temporal, waveform, and spectral domains, reducing data size by orders of magnitude while preserving discriminative information. Each domain is processed via dedicated step-wise and channel-wise
attention branches, fused by an adaptive gating mechanism. Experiments on the open $\Phi$-OTDR
benchmark and a real-scenario DAS
dataset show that DAS-tatFormer achieves up to 99.4% accuracy and near-perfect real-world performance, while using significantly fewer parameters and lower inference cost than models such as DASFormer and DeepViT. These results demonstrate its suitability for scalable, real-time DAS-based monitoring. We release our code at https://github.com/MichelD-git/DAStatFormer
Read the full abstract ▾
All authors Michel Dione, Jerry Lonlac, Hélène Louis, Anthony Fleury, Stephane Lecoeuche
Machine Learning · Cross-listed from cs.CV
Planktonzilla: Multimodal dataset and models for understanding plankton ecosystems
Alan Gerson Contreras Montanares, Luis Valenzuela, Luis Martí et al. (+1 more)
Marine plankton underpin aquatic food webs and play a key role in global CO2 sequestration, making reliable species identification critical for understanding ocean health and
climate feedbacks. Existing classification models perform well on individual collections but fail to generalize across instruments and environments due to isolated training
datasets and inconsistent labels. To address this, we introduce Planktonzilla-17M, a unified
dataset consolidating publicly available plankton image collections spanning thirteen imaging systems. It comprises 17.4 million images with standardized taxonomy and geo-environmental metadata, including 3.74 million plankton images spanning over 602 taxonomic classes, of which 201 are identified at the species level, making it the largest and most comprehensive plankton image
dataset to date. Using this large-scale
dataset, we perform a controlled comparison between supervised and CLIP-style image–text training on a shared ViT backbone. We find that a supervised classifier matches or exceeds CLIP-style training when trained using taxonomic lineage as text. We further observe that BioCLIP and BioCLIP2 perform poorly on plankton in zero-shot and few-shot settings. Leveraging Planktonzilla-17M improves plankton classification performance, highlighting the limitations of current biological foundation models in marine imaging domains.
Read the full abstract ▾
All authors Alan Gerson Contreras Montanares, Luis Valenzuela, Luis Martí, Nayat Sanchez-Pi
Machine Learning
BitsMoE: Cost-Aware Bit Allocation in Spectral Space for MoE LLM Quantization
Jiayu Zhao, Zihan Teng, Minhao Fan et al. (+4 more)
Mixture-of-Experts (MoE) large
language models incur substantial
memory costs due to their large expert parameter counts. Mixed-precision quantization reduces these costs by allocating different bit-widths to experts or linear blocks according to their importance. However, assigning a single precision within each expert or linear block overlooks its internal structural heterogeneity. This limitation motivates two key questions: (1) how to define a fine-grained unit for quantization within a linear transformation; and (2) how to characterize the quantization cost of each unit under actual activation patterns and different bit-widths. To address these two questions, we propose BitsMoE, a cost-aware mixed-precision quantization framework built on two complementary techniques: (1) Shared-basis Spectral Decomposition (SSD) separates expert weights into a shared basis and expert-specific spectral components, defining structural quantization units while exploiting cross-expert redundancy. (2) Factorized Quantization Cost Modeling (FQCM) estimates component-wise costs from output reconstruction loss by combining intrinsic spectral importance, activation-dependent importance, and bit-width-dependent distortion. Using these component-wise costs, we formulate bit allocation as an integer linear program (ILP) that minimizes total modeled quantization cost under a fixed
memory budget. On Qwen3-30B-A3B at 2-bit, BitsMoE achieves 64.29% average accuracy over seven downstream tasks, outperforming the evaluated MoE-specific methods, including those using ILP-based bit allocation, and exceeding GEMQ by 2.80 percentage points. Under the same setting, it achieves a $16.47\times$ end-to-end offline quantization speedup over GEMQ. It also achieves up to $6.46\times$ the decode throughput of GPTQ.
Read the full abstract ▾
All authors Jiayu Zhao, Zihan Teng, Minhao Fan, Tianrui Ma, Wentao Ren, Song Chen, Weichen Liu
Machine Learning
Representation Alignment Rests on Linear Structure
Kiril Bangachev, Guy Bresler and Yury Polyanskiy
We investigate the Platonic Representation Hypothesis (PRH) through a tripartite statistical framework of representations: signal, bias, and noise. {1) Signal:} We propose that Platonic
alignment arises from the universal relationship between objects and attributes, which is encoded linearly in representations according to the Linear Representation Hypothesis (LRH). We provide evidence that LRH helps explain PRH by extracting linear object-attribute features with sparse autoencoders and showing that these sparse representations often exhibit stronger cross-modal
alignment than their dense counterparts. {2) Bias:} Models have different implicit biases due to the diverse architectures and training procedures used. We show that this difference can be partially mitigated. Centering and normalization consistently improve cross-model
alignment. {3) Noise:} Finite-sample training leads to noise in representations. We provide evidence that representational noise is driven by data scarcity by revealing a strong and consistent positive correlation between word frequency and
alignment in LLMs and text
embedding models. Synthesizing signal, bias, and noise, we propose a statistical model that refines the Linear Representation Hypothesis and explains further phenomena related to the
alignment of representations emerging from diverse modern AI architectures.
Read the full abstract ▾
Machine Learning
Balancing Multimodal Learning through Label Space Reshaping
Xiaoyu Ma, Weijie Zhang, Yuanhao Gao et al. (+3 more)
Multimodal learning often suffers from modality imbalance, where modalities that converge faster dominate optimization while others remain undertrained. Existing approaches typically mitigate this issue by strengthening the weak modality or adjusting optimization gradients. However, such strategies mainly compensate for optimization rate discrepancies, often at the expense of the strong modality's optimization capacity, without analyzing how these discrepancies arise at the modality level. Based on theoretical insights and empirical observations, we argue that the discrepancy of learning pace arises from differences in the mapping difficulty between modality-specific feature space and the shared label space. To address this issue, we propose Balanced
Multimodal Label Reshaping (BMLR), the first method that promotes
multimodal balance from the label-side design. BMLR reshapes the cross-modal label space to equalize mapping difficulty across modalities, thereby facilitating modality interaction and injecting richer inter-class information into each modality. Extensive experiments across multiple architectures demonstrate that BMLR consistently improves
multimodal performance and exhibits strong compatibility with diverse model designs. The source code will be released soon.
Read the full abstract ▾
All authors Xiaoyu Ma, Weijie Zhang, Yuanhao Gao, Han Miao, Yongjian Deng, Hao Chen
Machine Learning
TaxDistill: Improving Metagenomic Taxonomic Annotation via Distilled Genomic Foundation Models
Rongye Ye, Lun Li, Zheng Luo et al. (+3 more)
Metagenomic taxonomic annotation is essential for interpreting complex microbial communities, yet reliable annotation remains challenging under reference database incompleteness and ambiguous taxonomic boundaries. Existing similarity-based tools are efficient, but they often produce noisy pseudo-labels in complex environments; learning-based post-hoc correction methods can further inherit this noise when trained on hard pseudo-labels generated by upstream classifiers. We propose TaxDistill, a plug-and-play knowledge
distillation framework for reliable metagenomic taxonomic recalibration. TaxDistill uses the genomic foundation model GenomeOcean as a semantic teacher and performs one-way online
distillation, where the teacher is updated by hierarchical taxonomic supervision while its detached soft probability distributions guide a lightweight student network. This design mitigates the student's overfitting to noisy hard pseudo-labels and supports confidence based rejection of potentially unreliable predictions. Comprehensive experiments on seven diverse CAMI2
datasets with MMseqs2, Metabuli, and Kraken2 show that TaxDistill improves annotation reliability over existing baselines in most scenarios. For example, on the Gastrointestinal
dataset, TaxDistill improves the F1 score of MMseqs2 from 0.763 to 0.941, surpassing the Taxometer baseline. Our results suggest that large genomic foundation models can provide semantic supervision for noisy scientific annotations, enabling lightweight post-hoc recalibration of existing metagenomic annotation results.
Read the full abstract ▾
All authors Rongye Ye, Lun Li, Zheng Luo, Yiran Zhan, Zhang Zhang, Shuhui Song
Machine Learning
PrismFlow: Residual Dynamics for Flow Matching in Time-Series Generation
Junru Zhang, Lang Feng, Jinbo Wang et al. (+6 more)
Generating high-quality time-series data is challenging because real-world signals often exhibit
multimodal patterns and multiscale dynamics, including oscillations and high-frequency variations. Flow Matching (FM) offers an efficient alternative to
diffusion models, but practical implementations typically rely on a single finite-capacity global vector-field
estimator. In such heterogeneous temporal distributions, distinct regimes may pass through nearby flow states while requiring incompatible conditional velocities. A monolithic
estimator trained with the standard $\ell_2$ velocity-matching objective may therefore learn an overly smoothed approximation of the local transport field. This
estimator-level smoothing can attenuate branch-specific dynamics, leading to spectral distortion and poor mode coverage. To address this, we propose PrismFlow, a new FM method with Koopman-inspired dynamical experts. Each expert learns residual corrections in a latent space where local nonlinear temporal
evolution can be approximated by linear transitions. We further propose a confidence-aware Winner-Take-All (WTA) objective that updates only the expert best aligned with each sample while masking gradients to the others, encouraging mode-specific specialization. During sampling, the selected expert adds a residual dynamical correction to the global transport field, preserving FM stability while recovering fine-grained and high-frequency temporal structures. Across various
benchmarks, PrismFlow effectively mitigates the spectral contraction in standard FM and achieves state-of-the-art performance, with a 15.6% gain in Context-FID and a 38.6% improvement in Discriminative Score, while remaining robust in low-data settings and effective for
forecasting and imputation.
Read the full abstract ▾
All authors Junru Zhang, Lang Feng, Jinbo Wang, Xu Guo, Yucheng Wang, Han Yu, Min Wu, Yabo Dong, Duanqing Xu
Machine Learning
Continuity and Ordinality Matter: Constraining Time Series Tokens for Effective Time Series Analysis with Large Language Models
Musheng Li, Ziying Zhang, Cheng jin et al. (+1 more)
Token-based time series large
language models (TS-LLMs) have emerged as a promising direction for time series analysis and
reasoning. However, prior studies largely overlook the inherent continuity and ordinality of time series
tokens, which substantially limits model performance. In this paper, we argue that preserving these properties in time series
token embeddings is crucial for the effectiveness of
token-based TS-LLMs. To this end, we propose COM (Continuity and Ordinality Matter), a continuity- and ordinality-aware strategy that integrates
geometric constraints into both the initialization and training stages. Empirical results on multiple time series analysis
benchmarks demonstrate that COM consistently improves the performance of
token-based TS-LLMs, achieving competitive results and strong generalizability. Code is available at https://anonymous.4open.science/r/COM .
Read the full abstract ▾
All authors Musheng Li, Ziying Zhang, Cheng jin, Yuantao Gu
Machine Learning
Emergent Semantic Representations in World Models through Physical Interaction without Linguistic Supervision
Jiayi Fang
What does a world model learn from physical exploration, without any linguistic supervision? We argue the answer is organized by a single principle: the
geometric structure of the physical world. Training a VAE-based world model on random
embodied exploration, we find that its latent space develops spatial semantic structure that mirrors physical geometry – direction accuracy 0.677+-0.029 versus 0.547 for a randomly initialized encoder, and position RSA 0.192+-0.047 versus 0.029 for random encoders (6.6x improvement), showing that training induces genuine structural organization beyond CNN inductive bias. Across 20 temporal checkpoints, prediction performance and semantic
alignment co-improve (Spearman r=-0.61, p=0.004), consistent with the shared-driver account. We confirm this through a double knockout: standard KL regularization (beta=0.1) forces the encoder away from
geometric structure, and both prediction performance and semantic
alignment collapse simultaneously to near-chance by step 50,000 – exactly as the shared-driver account predicts. Reducing beta to 0.001 restores
geometric access and recovers both capabilities together. These findings establish physical world geometry as the organizing principle of world model representations, with direct implications for the design of semantically grounded
embodied agents.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.IR
A Unified Structured Query Understanding Framework for Industrial Semantic Search
Ping Liu, Qianqi Shen, Jianqiang Shen et al. (+15 more)
Query understanding in large-scale industrial search systems is typically implemented as a cascade of disparate, task-specific components. While individually optimizable, this fragmented architecture incurs high maintenance overhead and results in inconsistent behaviors, particularly for long-tail queries. In this work, we propose and deploy a unified structured query understanding system that consolidates these heterogeneous functions into a single Small
Language Model (SLM) that performs schema-constrained generation. To address the data bottlenecks inherent in unified modeling, we introduce Query Illuminator, a dual-purpose framework serving as: (i) a teacher model for high-quality auto-annotation and
distillation, and (ii) a surrogate judge for scalable evaluation where human labels are scarce. We validate this approach through extensive offline and online tests within LinkedIn's Job Search system. Furthermore, we demonstrate the framework's horizontal extensibility through a cross-domain case study on People Search. The results show improved user engagement and reduced operational costs, achieved while satisfying strict low-latency serving constraints on limited GPU resources.
Read the full abstract ▾
All authors Ping Liu, Qianqi Shen, Jianqiang Shen, Chunnan Yao, Kevin Kao, Rajat Arora, Dan Xu, Baofen Zheng, Yunxiang Ren, Benjamin Le, Ali Hooshmand, Igor Lapchuk, Juan Bottaro, Raghavan Muthuregunathan, Caleb Johnson, Liangjie Hong, Jingwei Wu, Wenjing Zhang
Machine Learning
Tackling Multimodal Learning Challenges with Mixture-of-Expert: A Survey
Liangwei Nathan Zheng, Wenhao Liang, Wei Emma Zhang et al. (+2 more)
Mixture-of-Experts (MoE) presents a naturally compatible and scalable framework for
multimodal learning, demonstrating strong adaptability across diverse modalities and tasks. Despite its growing success, a comprehensive and systematic review on the MoE metho addressing
multimodal challenges remains lacking. Existing surveys tend to evaluate either
multimodal learning or MoE independently from method taxonomy, overlooking the unique interplay between them. This survey fills that gap by answering a central question: How does MoE effectively resolve
multimodal challenges? We approach this from three key perspectives: (1) MoE as an Efficient
Multimodal Engine: enabling scalable
multimodal modeling by decoupling computational cost from parameter growth and mitigating modality redundancy through selective expert activation; (2) MoE as a
Multimodal Representation Learner: integrating complementary multi-opinion expert knowledge to enrich
alignment and interaction representations; and (3) MoE as a
Multimodal Adapter: providing a modular and flexible mechanism to model imperfect data scenarios such as modality imbalance and missing modality. Through our extensive literature review, we identify critical research gaps, including interpretable routing, expert communication, modality integration, and lifelong
multimodal learning. We position this survey as a foundation for future research toward interpretable and sustainable
multimodal Mixture-of-Experts system.
Read the full abstract ▾
All authors Liangwei Nathan Zheng, Wenhao Liang, Wei Emma Zhang, Lin Yue, Weitong Chen
Machine Learning
Neural Bayesian Sequential Routing
Yongchao Huang
Human decision-making is sequential and uncertainty-aware, yet standard neural networks often rely on static, dense forward computation with limited visibility into evidence acquisition, uncertainty
evolution, or when computation should stop. We introduce Neural
Bayesian Sequential Routing (NBSR), a framework that models neural inference as active evidence accumulation over a hierarchical Directed Acyclic Graph (DAG). Within a Dirichlet–Categorical conjugate framework, neural experts query a persistent global knowledge oracle to extract positive evidence vectors, which act as pseudo-counts and update a Dirichlet belief state by exact conjugate addition. Coupled with a Gumbel-Softmax Straight-Through
estimator, this update enables hard, path-dependent routing while preserving surrogate gradients for end-to-end training. The resulting Dirichlet precision and
entropy provide mechanisms for uncertainty quantification,
entropy-based early exiting, OOD abstention, and cost-aware evidence acquisition. We prove that, under strictly positive evidence extraction, total Dirichlet precision increases monotonically along any valid trajectory and marginal predictive variance is bounded, formalizing sequential “hypothesis sharpening”; under idealized capacity and optimization assumptions, the terminal Dirichlet expectation recovers the Bayes-optimal conditional distribution. Empirical evaluations across visual categorization, structured medical diagnosis,
language modeling, partially observable control, and cost-aware
Bayesian experimental design show that NBSR achieves competitive predictive performance while providing transparent routing traces, path-dependent evidence attribution, uncertainty-aware decision control, and resource-rational inference. Overall, NBSR offers a mathematically grounded framework for interpretable, modular, and resource-rational agentic AI.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CR
Enhancing Reliability in LLM-Based Secure Code Generation
Mohammed F. Kharma, Mohammad Alkhanafseh, Ahmed Sabbah et al. (+1 more)
Large
language models (LLMs) are widely used for code generation, but their security reliability remains inconsistent across
languages and prompting strategies. Existing
prompt engineering improves functional correctness but rarely ensures consistent security outcomes. We introduce the Mitigation-Aware Chain-of-Thought (MA-CoT) framework, which embeds task-specific CWE mitigation guidance and
language-aware safeguards to reduce recurring vulnerabilities in generated code. We evaluate MA-CoT across three LLMs (gpt-5, claude-4.5, gemini-2.5), three programming
languages (C, Java, Python), and four prompting strategies (Vanilla, Zero-shot, CoT, MA-CoT) on a 200-task primary
dataset, with external validation on LLMSecEval. Using static analysis with expert validation, MA-CoT reduces total security findings from 92 to 39 (57.6%) on the primary
dataset and from 73 to 4 (94.5%) on LLMSecEval. High-severity findings (Blocker + Critical) drop from 90 to 39 (56.7%) and from 45 to 2 (95.6%), respectively. Across both
datasets, MA-CoT is the only strategy that consistently improves security reliability; Zero-shot and CoT are less reliable and may increase vulnerability, especially in C. We further introduce a strict layered attribution of vulnerability drivers (
language-core vs. stack layers) and show that residual risk concentrates in hardening-oriented patterns (e.g., OS- and toolchain-dependent), motivating secure-by-construction primitives alongside prompting.
Read the full abstract ▾
All authors Mohammed F. Kharma, Mohammad Alkhanafseh, Ahmed Sabbah, David Mohaisen
Machine Learning
LLMs Show No Signs Of Individuated Metacognition
M. Moran and Mark Whiting
Confidence-weighted routing, selective abstention, and ensemble weighting all assume that a model's stated confidence is informative about its capability on the question being asked. They presume functional metacognition, the capacity to assess one's own capabilities, without exercising them. Aggregate calibration is well studied, with mixed results, but the underlying structure of elicited confidence is less well understood. We decompose binary confidence judgements from 20 frontier Large
Language Models (LLMs) across six
benchmarks using tetrachoric factor analysis paired with pairwise calibration, asking whether two models that differ in confidence also differ in performance. On factual recall and information
retrieval benchmarks the cross-model confidence
matrix is approximately rank-one and a single dominant factor captures most of the latent variance. Models retrieving facts share an item-level difficulty axis and differ mainly in their decision thresholds along it. Across all
benchmarks the relationship between confidence and performance collapses once items that all models agree on are removed. Inter-model pairwise calibration is small even where statistically significant, and what remains shrinks to nothing once base-rate differences along the shared factor are controlled for. Mathematical
reasoning is the apparent exception, but this turns out to be a confound where
reasoning models answer questions about their confidence by trying to solve them in their chain of thought, bypassing the sub-symbolic self-knowledge we seek to measure. We find no evidence for significant verbalised individuated metacognition in any tested domain.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CR
An Empirical Evaluation of LLM-Generated Code Security Across Prompting Methods
Mohammed Kharma, Ahmed Sabbah, Mohammad Alkhanafseh et al. (+2 more)
The growing use of Large
Language Models (LLMs) for automated code generation has enhanced software development efficiency, but often at the cost of security. Generated code frequently overlooks critical concerns, leaving it vulnerable to issues such as weak encryption and improper input validation. To investigate this problem, we present a comprehensive empirical evaluation of the security quality of LLM-generated code across five LLMs and four programming
languages (Java, C++, C, and Python), examining the impact of multiple
prompt engineering methods. We introduce a weaknesses-aware zero-shot chain-of-thought (WA-0CoT) prompting strategy that enriches
prompts with security context using CWE mappings to guide model
reasoning. Our empirical analysis, supported by chi-square tests, finds no statistically significant reductions in vulnerability frequency or density across
prompt methods. However, prompting strategies, including WA-0CoT, systematically influence the compositional distribution of CWE categories, with effects varying by programming
language. These findings suggest that while security-aware prompting alters the structure of generated weaknesses,
prompt engineering alone is insufficient to reliably reduce overall vulnerability levels. The results highlight the importance of
language-aware and model-aware
prompt design when evaluating the security properties of LLM-generated code.
Read the full abstract ▾
All authors Mohammed Kharma, Ahmed Sabbah, Mohammad Alkhanafseh, Mohammad Hammoudeh, David Mohaisen
Machine Learning
Private Adaptive Covariance Estimation via Gaussian Graphical Models
Cecilia Ferrando, Miguel Fuentes, Brett Mullins et al. (+2 more)
We propose PACE-GGM, a data-adaptive differentially private method for covariance estimation that concentrates its
privacy budget on the most informative entries of the empirical covariance
matrix, rather than perturbing all entries. This applies in the natural setting where the modeler supplies separate bounds for each variable, so that individual entries can be measured with less noise than the full
matrix. In each round, our method selects a poorly approximated entry, measures it using the Gaussian mechanism, and then reconstructs a full covariance
matrix using a maximum-
entropy reconstruction objective, leading to a Gaussian graphical model structure. Experiments on diverse real-world
datasets demonstrate consistent improvements in estimation error with respect to the Gaussian mechanism and other baselines, particularly in high-dimensional and low-to-moderate
privacy regimes.
Read the full abstract ▾
All authors Cecilia Ferrando, Miguel Fuentes, Brett Mullins, Cameron Musco, Daniel Sheldon
Machine Learning · Cross-listed from cs.CR
Concept Drift Adaptation Using Self-Supervised and Reinforcement Learning In Android Malware Detection
Ahmed Sabbah, Mohammad Kharma, Mohammad Alkhanafseh et al. (+3 more)
Android malware detectors often degrade after deployment because of concept drift, while full retraining at each maintenance step is costly. We propose a chronological adaptive maintenance framework that models deployment-time maintenance as a sequential decision problem. The framework learns a stable latent representation through self-supervised learning during initialization, freezes the encoder, measures latent drift in the fixed representation space, and performs lightweight downstream adaptation using a trainable adapter and classification head. A proximal policy optimization controller selects low-cost maintenance actions based on the detector state, including current utility, retention on a fixed
memory set, latent drift indicators, and update cost. We evaluate the framework under a causal deployment-style protocol on emulator and real Android malware
datasets with static and dynamic features. Results show that the RL controller provides a strong cost-aware adaptation strategy, consistently remaining among the top-performing policies while achieving a favorable balance between temporal performance,
memory retention, and maintenance cost under non-stationary deployment conditions.
Read the full abstract ▾
All authors Ahmed Sabbah, Mohammad Kharma, Mohammad Alkhanafseh, Radi Jarrar, Samer Zein, David Mohaisen
Machine Learning
TUBE: Tangent Upper Bound on Evidence for Discrete Diffusion Language Models
Arseny Ivanov, Sergei Kholkin, Vladislav Gromadskii et al. (+3 more)
Log-likelihood is a standard metric for evaluating generative models. Unfortunately, in contrast to autoregressive models (ARMs), discrete
diffusion models generally do not admit exact computation of this quantity. Existing evaluations, therefore, rely on the evidence lower bound (ELBO), leaving unclear how much higher the true value may be. We address this by introducing the Tangent Upper Bound on Evidence (TUBE), a variational upper bound on log-likelihood that admits an unbiased Monte Carlo
estimator. Our TUBE extends across latent-variable models, including masked
diffusion models (MDMs), any-order ARMs (AO-ARMs), and block variants of both. Applied to block MDMs and block AO-ARMs, TUBE reveals our key empirical finding that these models lie strictly below the exact ARM baseline, showing that ARMs still dominate in likelihood.
Read the full abstract ▾
All authors Arseny Ivanov, Sergei Kholkin, Vladislav Gromadskii, Grigoriy Ksenofontov, Ivan Oseledets, Alexander Korotin
Machine Learning
Faithfulness as Information Flow: Evaluating and Training Faithful Chain-of-Thought Reasoning
Jinghan Jia, Joe Benton and Eric Easley
Chain-of-thought (CoT)
reasoning is useful for monitoring
language models only when the
reasoning trace faithfully reflects the computation that produces the final answer. However, models can rely on
prompt-to-answer shortcuts that bypass the CoT, making the visible
reasoning trace misleading even when it appears plausible. We study CoT faithfulness through a structural information-flow perspective: faithful
reasoning should route answer-relevant information through the mediated path from
prompt to CoT to answer, rather than through a direct
prompt-to-answer shortcut. This perspective yields a task-agnostic framework based on three complementary properties, sufficiency, completeness, and necessity, which we instantiate with
entropy-based, masked-KL, and gradient-based diagnostics. We show that these metrics recover externally judged faithfulness differences in hinted
reasoning, and identify a low-
entropy failure mode of KL-based diagnostics where gradient-based measures remain more stable. Building on this analysis, we introduce update-time interventions for verifier-based on-policy RL, including
attention masking, backward-only gradient masking, CoT gradients, and
adversarial perturbations of
prompt representations. Across hinted arithmetic, reward-hackable code repair, and DAPO-Math models trained without hints but evaluated under wrong-hint injection, our interventions shift behavioral and structural indicators toward stronger CoT mediation. In particular, they make shortcut and reward-hacking behavior more transparent in the CoT and improve task-agnostic faithfulness metrics, while in some settings also reducing wrong-hint susceptibility. Our results suggest that controlling information flow during training is a practical route toward more faithful and monitorable CoT
reasoning. Code is available at https://github.com/safety-research/faithful-cot.
Read the full abstract ▾
Machine Learning
Fourier Feature Pyramids for Physics-Informed Neural Networks
Brandon Zhao, Yixuan Wang, Jonathan T. Barron et al. (+3 more)
We present an improved neural field architecture for solving partial differential equations (PDEs). Current physics-informed neural networks (PINNs) provide a flexible framework for solving PDEs, but they struggle to achieve highly accurate solutions and require computation that scales poorly with parameter count. Our model, which we call beignet (Bandlimited
Embedding with Interpolated Grid Network), replaces the random
Fourier feature
embedding used by existing PINN models with a trainable multi-resolution
Fourier feature pyramid. To query beignet at a continuous coordinate, we use
Fourier interpolation at each level of the pyramid to return features at the input coordinate, and then decode this vector with a fully-connected neural network trunk. Our model provides multiple benefits: 1) Spatial derivatives can be computed efficiently by using the chain rule to compose derivatives of the neural network computed with automatic differentiation with derivatives of the feature grid computed spectrally by the Fast
Fourier transform (FFT). 2) beignet can achieve higher accuracy in a compute-efficient manner by scaling the parameter count of this
Fourier feature pyramid, instead of the less-efficient strategy of scaling the neural network architecture. 3) beignet can directly control the representation bandlimit, resulting in more stable optimization for difficult PDEs. We demonstrate that beignet finds significantly more accurate solutions on PDE
benchmarks using fewer parameters than state-of-the-art PINN methods. We further evaluate beignet on the self-similar inviscid Burgers blowup problem and show that it can minimize residuals to near machine precision using Adam, an accuracy regime previously attained only by using computationally expensive higher-order optimizers.
Read the full abstract ▾
All authors Brandon Zhao, Yixuan Wang, Jonathan T. Barron, Katherine L. Bouman, Dor Verbin, Pratul P. Srinivasan