Machine Learning
What Cosine Similarity of Label Representations Can and Cannot Tell us
Beatrix M. G. Nielsen and Andreas Grivas
Cosine similarity is often used to measure the similarity of vector representations of neural network models. However, the cosine similarity of representations is not guaranteed to tell us anything about model probabilities. In this paper we show that for a softmax classifier, be it an image classifier or an autoregressive
language model, the cosine similarity between label representations (called unembeddings in the paper) does not give any information on the probabilities assigned by the model. Specifically, we prove that given two unembeddings, it is possible to create another model which assigns the same probabilities for all inputs, but where the cosine similarity between the representations is now either 1 or -1. We also show that for a sigmoid classifier (where each input can be assigned multiple labels), all pairwise cosine similarities between the unembeddings define the set of possible label combinations. However, for softmax classifiers (where each input is assigned a ranking of the labels from most to least likely), we need all pairwise cosine similarities between all differences of unembeddings to know which rankings the model can predict. We conclude that it is misleading to interpret the cosine similarity between unembeddings without reference to the classifier that produced them.
Read the full abstract ▾
Machine Learning
Survival In-Context: Amortized Bayesian Survival Analysis via Prior-Fitted Networks
Dmitrii Seletkov, Paul Hager, Georgios Kaissis et al. (+3 more)
Survival analysis is crucial for many medical applications, but remains challenging for modern machine learning due to limited data, censoring, and the heterogeneity of tabular
covariates. While the prior-fitted paradigm, which relies on pretraining models on large collections of synthetic
datasets, has recently facilitated tabular foundation models for classification and regression, its suitability for time-to-event modeling remains unclear. We propose a flexible survival data generation framework that defines a rich survival prior with explicit control over
covariates and time-event distributions. Building on this prior, we introduce Survival In-Context (SIC), a prior-fitted in-context learning model for survival analysis that is pretrained exclusively on synthetic data. SIC is trained to approximate
Bayesian posterior predictive inference under the synthetic survival prior, enabling individualized survival prediction in a single forward pass, requiring no task-specific training or hyperparameter tuning. Across a broad evaluation on real-world survival
datasets, SIC achieves competitive or superior performance compared to classical and deep survival models, particularly in small and medium-sized data regimes, highlighting the promise of a prior-fitted paradigm for survival analysis. The code and pretrained models are available at https://github.com/yayapa/sic
Read the full abstract ▾
All authors Dmitrii Seletkov, Paul Hager, Georgios Kaissis, Rickmer Braren, Daniel Rueckert, Raphael Rehms
Machine Learning · Cross-listed from eess.SY
From Big Data to Fast Data: Towards High-Quality Datasets for Machine Learning Applications from Closed-Loop Data Collection
Philipp Reis, Jacqueline Henle, Stefan Otten et al. (+1 more)
The increasing capabilities of machine learning models, such as
vision-
language and
multimodal language models, are placing growing demands on data in automotive systems engineering, making the quality and relevance of collected data enablers for the development and validation of such systems. Traditional Big Data approaches focus on large-scale data collection and offline processing, while Smart Data approaches improve data selection strategies but still rely on centralized and offline post-processing. This paper introduces the concept of Fast Data for automotive systems engineering. The approach shifts data selection and recording onto the vehicle as the data source. By enabling real-time, context-aware decisions on whether and which data should be recorded, data collection can be directly aligned with data quality objectives and collection strategies within a closed-loop. This results in
datasets with higher relevance, improved coverage of critical scenarios, and increased information density, while at the same time reducing irrelevant data and associated costs. The proposed approach provides a structured foundation for designing data collection strategies that are aligned with the needs of modern machine learning
algorithms. It supports efficient data acquisition and contributes to scalable and cost-effective ML development processes in automotive systems engineering.
Read the full abstract ▾
All authors Philipp Reis, Jacqueline Henle, Stefan Otten, Eric Sax
Machine Learning
An Isotropic Approach to Efficient Uncertainty Quantification with Gradient Norms
Nils Grünefeld, Jes Frellsen and Christian Hardmeier
Existing methods for quantifying predictive uncertainty in neural networks are either computationally intractable for large
language models or require access to training data that is typically unavailable. We derive a lightweight alternative through two approximations: a first-order Taylor expansion that expresses uncertainty in terms of the gradient of the prediction and the parameter covariance, and an isotropy assumption on the parameter covariance. Together, these yield epistemic uncertainty as the squared gradient norm and aleatoric uncertainty as the Bernoulli variance of the point prediction, from a single forward-backward pass through an unmodified pretrained model. We justify the isotropy assumption by showing that covariance estimates built from non-training data introduce structured distortions that isotropic covariance avoids, and that theoretical results on the spectral properties of large networks support the approximation at scale. Validation against reference Markov Chain Monte Carlo estimates on synthetic problems shows strong correspondence that improves with model size. We then use the estimates to investigate when each uncertainty type carries useful signal for predicting answer correctness in question answering with large
language models, revealing a
benchmark-dependent divergence: the combined estimate achieves the highest mean AUROC on TruthfulQA, where questions involve genuine conflict between plausible answers, but falls to near chance on TriviaQA's factual recall, suggesting that parameter-level uncertainty captures a fundamentally different signal than self-assessment methods.
Read the full abstract ▾
Machine Learning
mtslearn: Machine Learning in Python for Medical Time Series
Zhongheng Jiang, Yuechao Zhao, Donglin Xie et al. (+5 more)
Medical time-series data captures the dynamic progression of patient conditions, playing a vital role in modern clinical decision support systems. However, real-world clinical data is highly heterogeneous and inconsistently formatted. Furthermore, existing machine learning tools often have steep learning curves and fragmented workflows. Consequently, a significant gap remains between cutting-edge AI technologies and clinical application. To address this, we introduce mtslearn, an end-to-end integrated toolkit specifically designed for medical time-series data. First, the framework provides a unified data interface that automates the parsing and
alignment of wide, long, and flat data formats. This design significantly reduces data cleaning overhead. Building on this, mtslearn provides a complete pipeline from data reading and feature engineering to model training and result visualization. Furthermore, it offers flexible interfaces for custom
algorithms. Through a modular design, mtslearn simplifies complex data engineering tasks into a few lines of code. This significantly lowers the barrier to entry for clinicians with limited programming experience, empowering them to focus more on exploring medical hypotheses and accelerating the translation of advanced
algorithms into real-world clinical practice. mtslearn is publicly available at https://github.com/PKUDigitalHealth/mtslearn.
Read the full abstract ▾
All authors Zhongheng Jiang, Yuechao Zhao, Donglin Xie, Chenxi Sun, Rongchen Lu, Silu Luo, Zisheng Liang, Shenda Hong
Machine Learning · Cross-listed from cs.NI
Diffusion-Guided Cooperative Policy Learning for Target Tracking Based on Underwater Mobile Agent Networks
Jiaao Ma, Chuan Lin, Guangjie Han et al. (+3 more)
Multi-
agent reinforcement learning (MARL) provides a promising solution for cooperative target tracking in networks of autonomous
underwater vehicles (AUVs). However, existing methods still face three major challenges: 1) policy non-stationarity caused by concurrent updates among multiple
agents; 2) inefficient policy learning caused by the heterogeneous quality of experiences accumulated during exploration; and 3) policy drift between
stochastic exploration and deterministic execution under dynamic
underwater disturbances. To address these challenges, this paper develops a four-layer hierarchical MARL architecture comprising global training scheduling, multi-
agent coordination, local policy generation, and real-time action execution. Building on this architecture, we propose a Supervised
Diffusion-Aided MARL (SDA-MARL)
algorithm with three closely coupled mechanisms. First, a dual-decision policy integrates a
diffusion-based generative branch with a Deep Deterministic Policy Gradient (DDPG) branch, while segregated experience pools reduce training interference between the two branches. Second, a supervised sample-selection mechanism identifies high-quality tracking transitions and uses their actions to guide reverse
diffusion, enabling the generative policy to concentrate on effective regions of the action space. Third, a behavioral-cloning loss transfers
diffusion-generated actions to the deterministic DDPG Actor, thereby aligning exploration with execution and suppressing policy drift. Experiments conducted in six-degree-of-freedom
underwater environments across multiple AUV-target configurations show that SDA-MARL achieves faster convergence, higher tracking accuracy, more consistent inter-AUV velocities, and shorter tracking paths than the compared MARL methods.
Read the full abstract ▾
All authors Jiaao Ma, Chuan Lin, Guangjie Han, Shengchao Zhu, Zhenyu Wang, Chen An
Machine Learning · Cross-listed from cs.CV
Yubo Cui, Xianchao Guan, Zijun Xiong et al. (+1 more)
Pre-trained
vision-
language models (VLMs) exhibit strong zero-shot generalization but remain vulnerable to
adversarial perturbations. Existing classification-guided
adversarial fine-tuning methods often disrupt pre-trained cross-modal
alignment, weakening visual-textual correspondence and degrading zero-shot performance. In this paper, we propose an
Alignment-Guided Fine-Tuning (AGFT) framework that enhances zero-shot
adversarial robustness while preserving the cross-modal semantic structure. Unlike label-based methods that rely on hard labels and fail to maintain the relative relationships between image and text, AGFT leverages the probabilistic predictions of the original model for text-guided
adversarial training, which aligns
adversarial visual features with textual
embeddings via soft
alignment distributions, improving zero-shot
adversarial robustness. To address structural discrepancies introduced by fine-tuning, we introduce a distribution consistency calibration mechanism that adjusts the robust model output to match a temperature-scaled version of the pre-trained model predictions. Extensive experiments across multiple zero-shot
benchmarks demonstrate that AGFT outperforms state-of-the-art methods while significantly improving zero-shot
adversarial robustness.
Read the full abstract ▾
All authors Yubo Cui, Xianchao Guan, Zijun Xiong, Zheng Zhang
Machine Learning
Hybrid Quantum-Classical Spatiotemporal Forecasting for 3D Cloud Fields
Fu Wang, Qifeng Lu, Xinyu Long et al. (+4 more)
Accurate
forecasting of three-dimensional (3D) cloud fields is important for atmospheric analysis and short-range numerical weather prediction, yet it remains challenging because cloud
evolution involves cross-layer interactions, nonlocal dependencies, and multiscale spatiotemporal dynamics. Existing spatiotemporal prediction models based on convolutions, recurrence, or
attention often rely on locality-biased representations and therefore struggle to preserve fine cloud structures in volumetric
forecasting tasks. To address this issue, we propose QENO, a hybrid
quantum-inspired spatiotemporal
forecasting framework for 3D cloud fields. The proposed architecture consists of four components: a classical spatiotemporal encoder for compact latent representation, a
topology-aware
quantum enhancement block for modeling nonlocal couplings in latent space, a dynamic fusion temporal unit for integrating measurement-derived
quantum features with recurrent
memory, and a decoder for reconstructing future cloud volumes. Experiments on CMA-MESO 3D cloud fields show that QENO consistently outperforms representative baselines, including ConvLSTM, PredRNN++, Earthformer, TAU, and SimVP variants, in terms of MSE, MAE, RMSE, SSIM, and threshold-based detection metrics. In particular, QENO achieves an MSE of 0.2038, an RMSE of 0.4514, and an SSIM of 0.6291, while also maintaining a compact parameter budget. These results indicate that
topology-aware hybrid
quantum-classical feature modeling is a promising direction for 3D cloud structure
forecasting and atmospheric Earth observation data analysis.
Read the full abstract ▾
All authors Fu Wang, Qifeng Lu, Xinyu Long, Meng Zhang, Xiaofei Yang, Weijia Cao, Xiaowen Chu
Machine Learning
PRISM: PRIor from corpus Statistics for topic Modeling
Tal Ishon, Yoav Goldberg and Uri Shaham
Topic modeling seeks to uncover latent semantic structure in text, with LDA providing a foundational probabilistic framework. While recent methods often incorporate external knowledge (e.g., pre-trained
embeddings), such reliance limits applicability in emerging or underexplored domains. We introduce PRISM, a corpus-intrinsic method that derives a Dirichlet parameter from word co-occurrence statistics to initialize LDA without altering its generative process. Experiments on text and single
cell RNA-seq data show that PRISM improves topic coherence and interpretability, rivaling models that rely on external knowledge. These results underscore the value of corpus-driven initialization for topic modeling in resource-constrained settings. Code is available at: https://github.com/shaham-lab/PRISM.
Read the full abstract ▾
Machine Learning
Causality-inspired Federated Learning for Dynamic Spatio-Temporal Graphs
Yuxuan Liu, Wenchao Xu, Haozhao Wang et al. (+5 more)
Federated Graph Learning (FGL) has emerged as a powerful paradigm for decentralized training of graph neural networks while preserving data
privacy. However, existing FGL methods are predominantly designed for static graphs and rely on parameter averaging or distribution
alignment, which implicitly assume that all features are equally transferable across clients, overlooking both the spatial and temporal heterogeneity and the presence of client-specific knowledge in real-world graphs. In this work, we identify that such assumptions create a vicious cycle of spurious representation
entanglement, client-specific interference, and negative transfer, degrading generalization performance in
Federated Learning over Dynamic Spatio-Temporal Graphs (FSTG). To address this issue, we propose a novel causality-inspired framework named SC-FSGL, which explicitly decouples transferable causal knowledge from client-specific noise through representation-level interventions. Specifically, we introduce a Conditional Separation Module that simulates soft interventions through client conditioned masks, enabling the disentanglement of invariant spatio-temporal causal factors from spurious signals and mitigating representation
entanglement caused by client heterogeneity. In addition, we propose a Causal Codebook that clusters causal prototypes and aligns local representations via contrastive learning, promoting cross-client consistency and facilitating knowledge sharing across diverse spatio-temporal patterns. Experiments on five diverse heterogeneity Spatio-Temporal Graph (STG)
datasets show that SC-FSGL outperforms state-of-the-art methods.
Read the full abstract ▾
All authors Yuxuan Liu, Wenchao Xu, Haozhao Wang, Zhiming He, Zhaofeng Shi, Chongyang Xu, Peichao Wang, Boyuan Zhang
Machine Learning · Cross-listed from cs.CR
Deep Learning-Assisted Improved Differential Fault Attacks on Lightweight Stream Ciphers
Kok Ping Lim, Dongyang Jia and Iftekhar Salam
Lightweight cryptographic primitives are widely deployed in resource-constrained environments, particularly in Internet of Things (IoT) devices. Due to their public accessibility, these devices are vulnerable to physical attacks, especially fault attacks. Recently, deep learning-based cryptanalytic techniques have demonstrated promising results; however, their application to fault attacks remains limited, particularly for stream ciphers. In this work, we investigate the feasibility of deep learning assisted differential fault attacks on three lightweight stream ciphers, namely ACORNv3, MORUSv2, and ATOM, under a relaxed fault model in which a single-bit bit-flipping fault is injected at an unknown location. We develop and train multilayer perceptron (MLP) models to identify the fault locations. Experimental results show that the trained models achieve high identification accuracies of 0.999880, 0.999231, and 0.823568 for ACORNv3, MORUSv2 and ATOM, respectively, and outperform traditional signature-based methods. For the secret recovery process, we introduce a threshold-based method to optimize the number of fault injections required to recover the secret information. The results show that the initial state of ACORN can be recovered with 21 to 34 faults, while MORUS requires 213 to 248 faults, with at most 6 bits of guessing. Both attacks reduce the attack complexity compared to existing works. For ATOM, the results show that it possesses a higher security margin, as the majority of state bits in the Nonlinear Feedback Shift Register (NFSR) can only be recovered under a precise control model. To the best of our knowledge, this work provides the first experimental results of differential fault attacks on ATOM.
Read the full abstract ▾
Machine Learning
Finite-time analysis of Multi-timescale Stochastic Optimization Algorithms
Kaustubh Kartikey and Shalabh Bhatnagar
We present a finite-time analysis of two smoothed functional
stochastic approximation
algorithms for simulation-based optimization. The first is a two time-scale gradient-based method, while the second is a three time-scale Newton-based
algorithm that estimates both the gradient and the Hessian of the objective function $J$. Both
algorithms involve zeroth order estimates for the gradient/Hessian. Although the asymptotic convergence of these
algorithms has been established in prior work, finite-time guarantees of two-timescale
stochastic optimization
algorithms in zeroth order settings have not been provided previously. For our Newton
algorithm, we derive mean-squared error bounds for the Hessian
estimator and establish a finite-time bound on $\min\limits_{0 \le m \le T} \mathbb{E}\| \nabla J(\theta(m)) \|^2$, showing convergence to first-order stationary points. The analysis explicitly characterizes the interaction between multiple time-scales and the propagation of estimation errors. We further identify step-size choices that balance dominant error terms and achieve near-optimal convergence rates. We also provide corresponding finite-time guarantees for the gradient
algorithm under the same framework. The theoretical results are further validated through experiments on the Continuous Mountain Car environment.
Read the full abstract ▾
Machine Learning
Deep Learning-Based Anomaly Detection in Spacecraft Telemetry on Edge Devices
Christopher Goetze, Tim Schlippe and Daniel Lakey
Spacecraft anomaly detection is critical for mission safety, yet deploying sophisticated models on-board presents significant challenges due to hardware constraints. This paper investigates three approaches for spacecraft telemetry anomaly detection –
forecasting & threshold, direct classification, and image classification – and optimizes them for edge deployment using multi-objective neural architecture optimization on the European Space Agency Anomaly
Dataset. Our baseline experiments demonstrate that
forecasting & threshold achieves superior detection performance (92.7% Corrected Event-wise F0.5-score (CEF0.5)) [1] compared to alternatives. Through Pareto-optimal architecture optimization, we dramatically reduced computational requirements while maintaining capabilities – the optimized
forecasting & threshold model preserved 88.8% CEF0.5 while reducing RAM usage by 97.1% to just 59 KB and operations by 99.4%. Analysis of deployment viability shows our optimized models require just 0.36-6.25% of CubeSat RAM, making on-board anomaly detection practical even on highly constrained hardware. This research demonstrates that sophisticated anomaly detection capabilities can be successfully deployed within spacecraft edge computing constraints, providing near-instantaneous detection without exceeding hardware limitations or compromising mission safety.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AR
AP-DRL: A Synergistic Algorithm-Hardware Framework for Automatic Task Partitioning of Deep Reinforcement Learning on Versal ACAP
Enlai Li, Zhe Lin, Sharad Sinha et al. (+1 more)
Deep reinforcement learning has demonstrated remarkable success across various domains. However, the tight coupling between training and inference processes makes accelerating DRL training an essential challenge for DRL optimization. Two key issues hinder efficient DRL training: (1) the significant variation in computational intensity across different DRL
algorithms and even among operations within the same
algorithm complicates hardware platform selection, while (2) DRL's wide dynamic range could lead to substantial reward errors with conventional FP16+FP32 mixed-precision quantization. While existing work has primarily focused on accelerating DRL for specific computing units or optimizing inference-stage quantization, we propose AP-DRL to address the above challenges. AP-DRL is an automatic task partitioning framework that harnesses the heterogeneous architecture of AMD Versal ACAP (integrating CPUs, FPGAs, and AI Engines) to accelerate DRL training through intelligent hardware-aware optimization. Our approach begins with bottleneck analysis of CPU, FPGA, and AIE performance across diverse DRL workloads, informing the design principles for AP-DRL's inter-component task partitioning and quantization optimization. The framework then addresses the challenge of platform selection through design space exploration-based profiling and ILP-based partitioning models that match operations to optimal computing units based on their computational characteristics. For the quantization challenge, AP-DRL employs a hardware-aware
algorithm coordinating FP32 (CPU), FP16 (FPGA/DSP), and BF16 (AI Engine) operations by leveraging Versal ACAP's native support for these precision formats. Comprehensive experiments indicate that AP-DRL can achieve speedup of up to 4.17$\times$ over programmable logic and up to 3.82$\times$ over AI Engine baselines while maintaining training convergence.
Read the full abstract ▾
All authors Enlai Li, Zhe Lin, Sharad Sinha, Wei Zhang
Machine Learning · Cross-listed from cs.AI
Rigorous Explanations for Tree Ensembles
Yacine Izza, Alexey Ignatiev, Xuanxiang Huang et al. (+2 more)
Tree ensembles (TEs) find a multitude of practical applications. They represent one of the most general and accurate classes of machine learning methods. While they are typically quite concise in representation, their operation remains inscrutable to human decision makers. One solution to build trust in the operation of TEs is to automatically identify explanations for the predictions made. Evidently, we can only achieve trust using explanations, if those explanations are rigorous, that is truly reflect properties of the underlying predictor they explain This paper investigates the computation of rigorously-defined, logically-sound explanations for the concrete case of two well-known examples of tree ensembles, namely random forests and boosted trees.
Read the full abstract ▾
All authors Yacine Izza, Alexey Ignatiev, Xuanxiang Huang, Peter J. Stuckey, Joao Marques-Silva
Machine Learning · Cross-listed from cs.CR
Beyond Corner Patches: Semantics-Aware Backdoor Attack in Federated Learning
Kavindu Herath, Joshua Zhao and Saurabh Bagchi
Backdoor attacks on
federated learning (FL) are most often evaluated with synthetic corner patches or out-of-distribution (OOD) patterns that are unlikely to arise in practice. In this paper, we revisit the
backdoor threat to standard FL (a single global model) under a more realistic setting where triggers must be semantically meaningful, in-distribution, and visually plausible. We propose SABLE, a Semantics-Aware
Backdoor for LEarning in
federated settings, which constructs natural, content-consistent triggers (e.g., semantic attribute changes such as sunglasses) and optimizes an aggregation-aware malicious objective with feature separation and parameter regularization to keep attacker updates close to benign ones. We instantiate SABLE on CelebA hair-color classification and the German Traffic Sign Recognition
Benchmark (GTSRB), poisoning only a small, interpretable subset of each malicious client's local data while otherwise following the standard FL protocol. Across heterogeneous client partitions and multiple aggregation rules (FedAvg, Trimmed Mean, MultiKrum, and FLAME), our semantics-driven triggers achieve high targeted attack success rates while preserving benign test accuracy. These results show that semantics-aligned
backdoors remain a potent and practical threat in
federated learning, and that robustness claims based solely on synthetic patch triggers can be overly optimistic.
Read the full abstract ▾
Machine Learning
LGFNet: Local-Global Fusion Network with Fidelity Gap Delta Learning for Multi-Source Aerodynamics
Qinye Zhu, Yu Xiang, Jun Zhang et al. (+1 more)
The precise fusion of computational fluid dynamic (CFD) data, wind tunnel tests data, and flight tests data in aerodynamic area is essential for obtaining comprehensive knowledge of both localized flow structures and global aerodynamic trends across the entire flight envelope. However, existing methodologies often struggle to balance high-resolution local fidelity with wide-range global dependency, leading to either a loss of sharp discontinuities or an inability to capture long-range topological correlations. We propose Local-Global Fusion Network (LGFNet) for multi-scale feature decomposition to extract this dual-natured aerodynamic knowledge. To this end, LGFNet combines a spatial perception layer that integrates a sliding window mechanism with a relational
reasoning layer based on self-
attention, simultaneously reinforcing the continuity of fine-grained local features (e.g., shock waves) and capturing long-range flow information. Furthermore, the fidelity gap delta learning (FGDL) strategy is proposed to treat CFD data as a "low-frequency carrier" to explicitly approximate nonlinear discrepancies. This approach prevents unphysical smoothing while inheriting the foundational physical trends from the simulation baseline. Experiments demonstrate that LGFNet achieves state-of-the-art (SOTA) performance in both accuracy and uncertainty reduction across diverse aerodynamic scenarios.
Read the full abstract ▾
All authors Qinye Zhu, Yu Xiang, Jun Zhang, Wenyong Wang
Machine Learning
From Physics to Surrogate Intelligence: A Unified Electro-Thermo-Optimization Framework for TSV Networks
Mohamed Gharib, Leonid Popryho and Inna Partin-Vaisband
High-density through-substrate vias (TSVs) enable 2.5D/3D heterogeneous integration but introduce significant signal-integrity and thermal-reliability challenges due to electrical coupling, insertion loss, and self-heating. Conventional full-wave finite-element method (FEM) simulations provide high accuracy but become computationally prohibitive for large design-space exploration. This work presents a scalable electro–thermal modeling and optimization framework that combines physics-informed analytical modeling, graph neural network (GNN) surrogates, and full-wave sign-off validation. A multi-conductor analytical model computes broadband S-parameters and effective anisotropic thermal conductivities of TSV arrays, achieving $5\%$–$10\%$ relative Frobenius error (RFE) across array sizes up to $15\times15$. A physics-informed GNN surrogate (TSV-PhGNN), trained on analytical data and fine-tuned with HFSS simulations, generalizes to larger arrays with mean RFE below $5\%$ in-distribution. The surrogate is integrated into a multi-objective Pareto optimization framework targeting reflection coefficient, insertion loss, worst-case crosstalk (NEXT/FEXT), and effective thermal conductivity. Millions of TSV configurations can be explored within minutes, enabling exhaustive layout and
geometric optimization that would be infeasible using FEM alone. Final designs are validated with Ansys HFSS and Mechanical, showing strong agreement. The proposed framework enables rapid electro–thermal co-design of TSV arrays while reducing per-design evaluation time by more than six orders of magnitude.
Read the full abstract ▾
Machine Learning
Linear Generator Networks for Nonlinear Partial Differential Equations
Shafayeth Jamil and Rehan Kapadia
Most physical systems are governed by nonlinear partial differential equations, yet the analytical tools that make linear systems transparent, such as eigenspectra, dispersion relations and modal decomposition, remain unavailable in nonlinear settings. We introduce Linear Generator Network-Koopman (LGN-KM), a neural operator that learns a nonlinear lift and a continuous-time linear generator for the lifted dynamics through the decomposition $L_k = S - D_k$, where $S$ is skew-symmetric (conservative coupling) and $D_k$ is positive-definite diagonal (mode-selective dissipation). The decomposition guarantees stability by construction and exposes the eigenstructure directly, making linear-systems analysis available on a nonlinear PDE. On 2D Navier-Stokes
turbulence, the generator yields a complete multi-branch dispersion relation and models trained independently at viscosities $100\times$ apart learn the same coupling spectrum ($R^2 = 0.87$). The same architecture shows the corresponding diffusive damping on FitzHugh-Nagumo reaction-
diffusion, and for pulse propagation in optical fibers, the oscillation frequencies of the learned generator match the fiber's dispersion relation, giving the group-velocity dispersion to within $3.1\%$ with no fitted constant. The architecture additionally enables long-horizon stability, $O(1)$ continuous-time evaluation, and physics-informed cross-regime model transfer.
Read the full abstract ▾
Machine Learning
Monodense Deep Neural Model for Determining Item Price Elasticity
Lakshya Garg, Sai Yaswanth, Deep Narayan Mishra et al. (+3 more)
Item Price Elasticity is used to quantify the responsiveness of consumer demand to changes in item prices, enabling businesses to create pricing strategies and optimize revenue management. Sectors such as store retail, e-commerce, and consumer goods rely on elasticity information derived from historical sales and pricing data. This elasticity provides an understanding of purchasing behavior across different items, consumer discount sensitivity, and demand elastic departments. This information is particularly valuable for competitive
markets and resource-constrained businesses decision making which aims to maximize profitability and
market share. Price elasticity also uncovers historical shifts in consumer responsiveness over time. In this paper, we model item-level price elasticity using large-scale transactional
datasets, by proposing a novel elasticity estimation framework which has the capability to work in an absence of treatment control setting. We test this framework by using Machine learning based
algorithms listed below, including our newly proposed Monodense deep neural network. (1) Monodense-DL network – Hybrid neural network architecture combining
embedding, dense, and Monodense layers (2) DML – Double machine learning setting using regression models (3) LGBM – Light Gradient Boosting Model We evaluate our model on multi-category retail data spanning millions of transactions using a back testing framework. Experimental results demonstrate the superiority of our proposed neural network model within the framework compared to other prevalent ML based methods listed above.
Read the full abstract ▾
All authors Lakshya Garg, Sai Yaswanth, Deep Narayan Mishra, Karthik Kumaran, Anupriya Sharma, Mayank Uniyal
Machine Learning · Cross-listed from eess.SY
Real-Time Surrogate Modeling for Fast Transient Prediction in Inverter-Based Microgrids Using CNN and LightGBM
Osasumwen Cedric Ogiesoba-Eguakun, Kaveh Ashenayi and Suman Rath
Real-time monitoring of inverter-based microgrids is essential for stability, fault response, and operational decision-making. However, electromagnetic transient (EMT) simulations, required to capture fast inverter dynamics, are computationally intensive and unsuitable for real-time applications. This paper presents a data-driven surrogate modeling framework for fast prediction of microgrid behavior using convolutional neural networks (CNN) and Light Gradient Boosting Machine (LightGBM). The models are trained on a high-fidelity EMT digital twin
dataset of a microgrid with ten distributed generators under eleven operating and disturbance scenarios, including faults, noise, and communication delays. A sliding-window method is applied to predict important system variables, including voltage magnitude, frequency, total active power, and voltage dip. The results show that model performance changes depending on the type of variable being predicted. The CNN demonstrates high accuracy for time-dependent signals such as voltage, with an $R^2$ value of 0.84, whereas LightGBM shows better performance for structured and disturbance-related variables, achieving an $R^2$ of 0.999 for frequency and 0.75 for voltage dip. A combined CNN+LightGBM model delivers stable performance across all variables. Beyond accuracy, the surrogate models also provide major improvements in computational efficiency. LightGBM achieves more than $1000\times$ speedup and runs faster than real time, while the hybrid model achieves over $500\times$ speedup with near real-time performance. These findings show that data-driven surrogate models can effectively represent microgrid dynamics. They also support real-time and faster-than-real-time predictions. As a result, they are well-suited for applications such as monitoring, fault analysis, and control in inverter-based power systems.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
MemRerank: Preference Memory for Personalized Product Reranking
Zhiyuan Peng, Xuyang Wu, Huaixiao Tou et al. (+2 more)
LLM-based shopping
agents increasingly rely on long purchase histories and multi-turn interactions for personalization, yet naively appending raw history to
prompts is often ineffective due to noise, length, and relevance mismatch. We propose MemRerank, a preference
memory framework that distills user purchase history into concise, query-independent signals for personalized product reranking. To study this problem, we build an end-to-end
benchmark and evaluation framework centered on an LLM-based 1-in-5 selection task, which measures both
memory quality and downstream reranking utility. We further train the
memory extractor with reinforcement learning (RL), using downstream reranking performance as supervision. Experiments with two LLM-based rerankers show that MemRerank consistently outperforms no-
memory, raw-history, and off-the-shelf
memory baselines, yielding up to +10.61 absolute points in 1-in-5 accuracy. These results suggest that explicit preference
memory is a practical and effective building block for personalization in agentic e-commerce systems.
Read the full abstract ▾
All authors Zhiyuan Peng, Xuyang Wu, Huaixiao Tou, Yi Fang, Yu Gong
Machine Learning · Cross-listed from cs.CL
The Thiomi Dataset: A Large-Scale Multimodal Corpus for Low-Resource African Languages
Hillary Mutisya, John Mugane, Gavin Nyamboga et al. (+2 more)
We present the Thiomi
Dataset, a large-scale
multimodal corpus spanning ten African
languages across four
language families: Swahili, Kikuyu, Kamba, Kimeru, Luo, Maasai, Kipsigis, Somali (East Africa); Wolof (West Africa); and Fulani (West/Central Africa). The
dataset contains over 601,000 approved sentence-level text annotations and over 385,000 audio recordings, collected through a dedicated community data collection platform involving over 100 contributors. To validate the
dataset's utility, we train and evaluate ASR, MT, and TTS models, establishing baselines across all
languages. Our best ASR system achieves 3.24% WER on Swahili (Common Voice), reducing prior academic SOTA from 8.3% to 3.24% (5.1 percentage point absolute, 61% relative reduction), and 4.3% WER on Somali. The
dataset will be published on HuggingFace. We describe the collection platform, quality assurance workflows, and baseline experiments, and discuss implications for African
language technology infrastructure.
Read the full abstract ▾
All authors Hillary Mutisya, John Mugane, Gavin Nyamboga, Brian Chege, Maryruth Gathoni
Machine Learning
Stochastic Dimension Implicit Functional Projections for Global Integral Conservation in High-Dimensional PINNs
Zhangyong Liang, Huanhuan Gao and Manyu Xiao
Enforcing prescribed global integral constraints in mesh-free neural PDE solvers is challenging in high-dimensional domains. Existing projection methods for spatial integrals are often tied to fixed grids or uniform quadrature, which can conflict with randomly sampled physics-informed neural networks (PINNs) and scale poorly with dimension. High-order differential operators also increase reverse-mode automatic differentiation
memory costs. We propose
Stochastic Dimension Implicit Functional Projection (SDIFP), a quadrature-level framework for enforcing prescribed first and second spatial moments. SDIFP replaces tensor-product nodal projection by a global affine correction of the neural-network output, with two scalar coefficients determined from a weighted quadrature rule. Under positive target variance and nonzero empirical raw variance, this correction is the nearest-point projection, in the weighted quadrature norm, onto the empirical two-moment constraint set. Thus, the prescribed moments are exact for the selected quadrature rule, while continuum errors are quadrature errors of the corrected field. For decomposable high-dimensional linear operators, SDIFP combines affine moment correction with
stochastic operator-subset sampling. With independent residual and derivative sampling and conditionally unbiased coefficient-gradient estimation, the resulting
estimator is unbiased for the specified quadrature-based residual objective; the shared-subset fast mode is biased in general. SDIFP avoids tensor-product quadrature for moment enforcement, separates forward quadrature evaluation from the reverse-mode graph, and retains pointwise inference efficiency once the affine coefficients are fixed or precomputed.
Read the full abstract ▾
Machine Learning
Robust and Consistent Ski Rental with Distributional Advice
Jihwan Kim and Chenglin Fan
The ski rental problem is a canonical model for online decision-making under uncertainty, capturing the fundamental trade-off between repeated rental costs and a one-time purchase. While classical
algorithms focus on worst-case competitive ratios and recent "learning-augmented" methods leverage point-estimate predictions, neither approach fully exploits the richness of full distributional predictions while maintaining rigorous robustness guarantees. We address this gap by establishing a systematic framework that integrates distributional advice of unknown quality into both deterministic and randomized
algorithms. For the deterministic setting, we formalize the problem under perfect distributional prediction and derive an efficient
algorithm to compute the optimal threshold-buy day. We provide a rigorous performance analysis, identifying sufficient conditions on the predicted distribution under which the expected competitive ratio (ECR) matches the classic optimal randomized bound. To handle imperfect predictions, we propose the Clamp Policy, which restricts the buying threshold to a safe range controlled by a tunable parameter. We show that this policy is both robust, maintaining good performance even with large prediction errors, and consistent, approaching the optimal performance as predictions become accurate. For the randomized setting, we characterize the stopping distribution via a Water-Filling
Algorithm, which optimizes expected cost while strictly satisfying robustness constraints. Experimental results across diverse distributions (Gaussian,
geometric, and bi-modal) demonstrate that our framework improves consistency significantly over existing point-prediction baselines while maintaining comparable robustness.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Long-Document QA with Chain-of-Structured-Thought and Fine-Tuned SLMs
Zhuowen Liang, Xiaotian Lin, Zhengxuan Zhang et al. (+3 more)
Large
language models (LLMs) are widely applied to data analytics over documents, yet direct
reasoning over long, noisy documents remains brittle and error-prone. Hence, we study document question answering (QA) that consolidates dispersed evidence into a structured output (e.g., a table, graph, or chunks) to support reliable, verifiable QA. We propose a two-pillar framework, LiteCoST, to achieve both high accuracy and low latency with small
language models (SLMs). Pillar 1: Chain-of-Structured-Thought (CoST). We introduce a CoST template, a schema-aware instruction that guides a strong LLM to produce both a step-wise CoST trace and the corresponding structured output. The process induces a minimal structure, normalizes entities/units, aligns records, serializes the output, and verifies/refines it, yielding auditable supervision. Pillar 2: SLM fine-tuning. The compact models are trained on LLM-generated CoST data in two stages: Supervised Fine-Tuning for structural
alignment, followed by Group Relative Policy Optimization (GRPO) incorporating triple rewards for answer/format quality and process consistency. By distilling structure-first behavior into SLMs, this approach achieves LLM-comparable quality on multi-domain long-document QA using 3B/7B SLMs, while delivering 2-4x lower latency than GPT-4o and DeepSeek-R1 (671B). The code is available at https://github.com/HKUSTDial/LiteCoST.
Read the full abstract ▾
All authors Zhuowen Liang, Xiaotian Lin, Zhengxuan Zhang, Yuyu Luo, Haixun Wang, Nan Tang
Machine Learning
Derived Fields Increase the Recoverable Information in Budgeted Neural PDE Surrogate
Wenshuo Wang and Fan Zhang
For neural surrogates of time-dependent partial differential equations (PDEs), storage and
memory budgets require training data to be obtained by coarsening and quantizing very-high-resolution numerical simulations. Fine-scale information can thus be lost before it reaches the model. We analyze this effect in periodic incompressible Navier-Stokes: quantization spreads error evenly over the retained modes, so the finest retained band, which holds the least energy, is the most corrupted, and velocity and vorticity are distorted differently under the same operator. Building on this analysis, we propose Derived-Field Optimization (DerivOpt), which chooses which primitive and derived fields to carry from a supplied library, and how many bits each receives, by exactly minimizing a closed-form posterior risk under a calibrated channel model. This optimization problem always has an optimum and requires no knowledge of the governing equation. On PDEBench, DerivOpt not only lowers the pooled rollout nRMSE but also substantially reduces the fine-band error. These gains are already present in the predictor-free input state and add to those of a multiscale architecture. Which fields are carried, and at what precision, is thus an optimizable design variable upstream of the predictor that determines how much information a fixed budget leaves recoverable.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.SE
Software Vulnerability Detection Using a Lightweight Graph Neural Network
Miles Farmer, Ekincan Ufuktepe, Anne Watson et al. (+4 more)
Large
Language Models (LLMs) have emerged as a popular choice in vulnerability detection studies given their foundational capabilities, open source availability, and variety of models, but have limited scalability due to extensive compute requirements. Using the natural graph relational structure of code, we show that our proposed graph neural network (GNN) based deep learning model VulGNN for vulnerability detection can achieve performance almost on par with LLMs, but is 100 times smaller in size and fast to retrain and customize. We describe the VulGNN architecture, ablation studies on components, learning rates, and generalizability to different code
datasets. As a lightweight model for vulnerability analysis, VulGNN is efficient and deployable at the edge as part of real-world software development pipelines.
Read the full abstract ▾
All authors Miles Farmer, Ekincan Ufuktepe, Anne Watson, Hialo Muniz Carvalho, Vadim Okun, Zineb Maasaoui, Kannappan Palaniappan
Machine Learning
Improving Ensemble Forecasts of Abnormally Deflecting Tropical Cyclones with Fused Atmosphere-Ocean-Terrain Data
Qixiang Li, Yuan Zhou, Shuwei Huo et al. (+2 more)
Deep learning-based tropical cyclone (TC)
forecasting methods have demonstrated significant potential and application advantages, as they feature much lower computational cost and faster operation speed than numerical weather prediction models. However, existing deep learning methods still have key limitations: they can only process a single type of sequential trajectory data or homogeneous meteorological variables, and fail to achieve accurate
forecasting of abnormal deflected TCs. To address these challenges, we present two groundbreaking contributions. First, we have constructed a
multimodal and multi-source
dataset named AOT-TCs for TC
forecasting in the Northwest Pacific basin. As the first
dataset of its kind, it innovatively integrates heterogeneous variables from the atmosphere, ocean, and land, thus obtaining a comprehensive and information-rich meteorological
dataset. Second, based on the AOT-TCs
dataset, we propose a
forecasting model that can handle both normal and abnormally deflected TCs. This is the first TC
forecasting model to adopt an explicit atmosphere-ocean-terrain coupling architecture, enabling it to effectively capture complex interactions across physical domains. Extensive experiments on all TC cases in the Northwest Pacific from 2017 to 2024 show that our model achieves state-of-the-art performance in TC
forecasting: it not only significantly improves the
forecasting accuracy of normal TCs but also breaks through the technical bottleneck in
forecasting abnormally deflected TCs.
Read the full abstract ▾
All authors Qixiang Li, Yuan Zhou, Shuwei Huo, Chong Wang, Xiaofeng Li
Machine Learning
Cell-induced densification and tether formation in fibrous extracellular matrices with biomimetic physics-informed neural networks
Anci Lin, Zhiwen Zhang and Wenju Zhao
Nonconvex multi-well energies in
cell-induced phase transitions give rise to fine-scale microstructures, low-regularity transition layers and sharp interfaces, all of which pose numerical challenges for physics-informed learning. Here we introduce biomimetic physics-informed neural networks (Bio-PINNs), which implement a near-to-far curriculum by progressively revealing the computational domain away from the
cell boundary and combining this schedule with a deformation-uncertainty proxy that concentrates collocation points near evolving transition layers and tether-forming regions. Across single-
cell and multicellular
benchmarks, Bio-PINNs recover the densified phase more reliably near
cell boundaries and in intercellular gaps, while capturing tether morphology more faithfully than representative ungated and residual-driven adaptive baselines.
Read the full abstract ▾