Machine Learning
Do Diffusion Models Dream of Electric Planes? Discrete and Continuous Simulation-Based Inference for Aircraft Design
Aurelien Ghiglino, Daniel Elenius, Anirban Roy et al. (+7 more)
In this paper, we generate conceptual engineering designs of electric vertical take-off and landing (eVTOL) aircraft. We follow the paradigm of simulation-based inference (SBI), whereby we look to learn a posterior distribution over the full eVTOL design space. To learn this distribution, we sample over discrete aircraft configurations (
topologies) and their corresponding set of continuous parameters. Therefore, we introduce a hierarchical probabilistic model consisting of two
diffusion models. The first model leverages recent work on Riemannian
Diffusion Language Modeling (RDLM) and Unified World Models (UWMs) to enable us to sample
topologies from a discrete and continuous space. For the second model we introduce a masked
diffusion approach to sample the corresponding parameters conditioned on the
topology. Our approach rediscovers known trends and governing physical laws in aircraft design, while significantly accelerating design generation.
Read the full abstract ▾
All authors Aurelien Ghiglino, Daniel Elenius, Anirban Roy, Ramneet Kaur, Manoj Acharya, Colin Samplawski, Brian Matejek, Susmit Jha, Juan Alonso, Adam Cobb
Machine Learning · Cross-listed from cs.NE
Bullet Trains: Parallelizing Training of Temporally Precise Spiking Neural Networks
Todd Morrill, Christian Pehle and Anthony Zador
Continuous-time, event-native spiking neural networks (SNNs) operate strictly on spike events, treating spike timing and ordering as the representation rather than an artifact of time discretization. This viewpoint aligns with biological computation and with the native resolution of event sensors and neuromorphic processors, while enabling compute and
memory that scale with the number of events. However, two challenges hinder practical, end-to-end trainable event-based SNN systems: 1) exact charge–fire–reset dynamics impose inherently sequential processing of input spikes, and 2) precise spike times must be solved without time bins. We address both. First, we use parallel associative scans to consume multiple input spikes at once, yielding up to 44x speedups over sequential simulation while retaining exact hard-reset dynamics. Second, we implement differentiable spike time solvers that compute spike times to machine-precision without discrete-time approximations or restrictive analytic assumptions. We demonstrate the viability of training SNNs using our solutions on four event-based
datasets on GPUs.
Read the full abstract ▾
Machine Learning
FedTreeLoRA: Reconciling Statistical and Functional Heterogeneity in Federated LoRA Fine-Tuning
Jieming Bian, Lei Wang, Letian Zhang et al. (+1 more)
Federated Learning (FL) with Low-Rank Adaptation (LoRA) has become a standard for
privacy-preserving LLM fine-tuning. However, existing personalized methods predominantly operated under a restrictive Flat-Model Assumption: they addressed client-side statistical heterogeneity but treated the model as a monolithic block, ignoring the functional heterogeneity across LLM layers. We argue that these two statistical (horizontal) and functional (vertical) dimensions, are orthogonal in source yet coupled in interaction, implying that the optimal depth of parameter sharing is functionally dependent on client similarity. To address this, we propose FedTreeLoRA, a framework employing tree-structured aggregation for fine-grained, layer-wise
alignment. By dynamically constructing an aggregation hierarchy, FedTreeLoRA allows clients to share broad consensus on shallow `trunks' while progressively specializing on deep `branches'. Experiments on NLU and NLG
benchmarks demonstrate that FedTreeLoRA significantly outperforms state-of-the-art methods by effectively reconciling generalization and personalization.
Read the full abstract ▾
All authors Jieming Bian, Lei Wang, Letian Zhang, Jie Xu
Machine Learning
ICaRus: Identical Cache Reuse for Efficient Multi Model Inference
Sunghyeon Woo, Jaeeun Kil, Hoseung Kim et al. (+9 more)
Multi model inference has recently emerged as a prominent paradigm, particularly in the development of agentic AI systems. However, in such scenarios, each model must maintain its own Key-Value (KV) cache for the identical
prompt, leading to substantial
memory consumption. This explosive growth of KV caches forces LLM serving systems to evict previously stored caches, which in turn introduces significant recomputation overhead whenever the evicted caches are required again. Moreover, prefix caching is inherently infeasible across different models, forcing each model to recompute KV cache for the identical
prompt, which leads to significant overhead. To alleviate these issues, we propose Identical Cache Reuse (ICaRus), a novel architecture that allows multiple models to share identical KV caches across all layers. ICaRus is based on the key observation that a decoder-only
Transformer can be conceptually decomposed into a logical encoder, which generates KV caches, and a logical decoder, which predicts output
tokens from the KV caches. ICaRus fine-tunes only the logical decoder while freezing the logical encoder, enabling multiple models to share an identical KV cache. This eliminates cache
memory explosion and unexpected evictions while also allowing cross-model reuse of KV caches for new input
tokens, thereby removing redundant recomputation in multi model inference achieving both efficiency and scalability. Moreover, by incorporating lightweight adapters such as LoRA, ICaRus parallelizes KV cache generation and next-
token prediction during decoding. ICaRus achieves comparable accuracy to task-specific fine-tuned model across a diverse set of tasks, while allowing multiple specialized models to fully share KV caches. ICaRus achieves up to 11.1x lower P95 latency and 3.8x higher throughput in multi
agent workflow with 8 different models, compared to conventional multi model system.
Read the full abstract ▾
All authors Sunghyeon Woo, Jaeeun Kil, Hoseung Kim, Minsub Kim, Joonghoon Kim, Ahreum Seo, Sungjae Lee, Minjung Jo, Jiwon Ryu, Baeseong Park, Se Jung Kwon, Dongsoo Lee
Machine Learning
A Stability-Aware Frozen Euler Autoencoder for Physics-Informed Tracking in Continuum Mechanics (SAFE-PIT-CM)
Emil Hovad
Material parameters such as thermal diffusivity govern how microstructural fields evolve during processing, but difficult to measure directly. The Stability-Aware Frozen Euler Physics-Informed Tracking for Continuum Mechanics (SAFE-PIT-CM), is an autoencoder that embeds a frozen convolutional layer as a differentiable PDE solver in its latent-space transition to jointly recover
diffusion coefficients and the underlying physical field from temporal observations. When temporal snapshots are saved at intervals coarser than the simulation time step, a single forward Euler step violates the von Neumann stability condition, forcing the learned coefficient to collapse to an unphysical value. Sub-stepping with SAFE restores stability at negligible cost each sub-step is a single frozen convolution, far cheaper than processing more frames with recovery error converging monotonically with substep count. Validated on thermal
diffusion in metals, the method recovers both the
diffusion coefficient and the physical field with near-perfect accuracy, both with and yet without pre-training. Backpropagation through the frozen operator supervises an
attention-based parameter
estimator without labelled data. The architecture generalises to any PDE with a convolutional finite-difference discretisation.
Read the full abstract ▾
Machine Learning
Demand Acceptance using Reinforcement Learning for Dynamic Vehicle Routing Problem with Emission Quota
Farid Najar, Dominique Barth and Yann Strozecki
This paper introduces and formalizes the Dynamic and
Stochastic Vehicle Routing Problem with Emission Quota (DS-QVRP-RR), a novel routing problems that integrates dynamic demand acceptance and routing with a global emission constraint. A key contribution is a two-layer optimization framework designed to facilitate anticipatory rejections of demands and generation of new routes. To solve this, we develop hybrid
algorithms that combine reinforcement learning with combinatorial optimization techniques. We present a comprehensive computational study that compares our approach against traditional methods. Our findings demonstrate the relevance of our approach for different types of inputs, even when the horizon of the problem is uncertain.
Read the full abstract ▾
Machine Learning
Learning Retrieval Models with Sparse Autoencoders
Thibault Formal, Maxime Louis, Hervé Dejean et al. (+1 more)
Sparse autoencoders (SAEs) provide a powerful mechanism for decomposing the dense representations produced by Large
Language Models (LLMs) into interpretable latent features. We posit that SAEs constitute a natural foundation for Learned Sparse
Retrieval (LSR), whose objective is to encode queries and documents into high-dimensional sparse representations optimized for efficient
retrieval. In contrast to existing LSR approaches that project input sequences into the vocabulary space, SAE-based representations offer the potential to produce more semantically structured, expressive, and
language-agnostic features. Building on this insight, we introduce SPLARE, a method to train SAE-based LSR models. Our experiments, relying on recently released open-source SAEs, demonstrate that this technique consistently outperforms vocabulary-based LSR in
multilingual and out-of-domain settings. SPLARE-7B, a
multilingual retrieval model capable of producing generalizable sparse latent
embeddings for a wide range of
languages and domains, achieves top results on MMTEB's
multilingual and English
retrieval tasks. We also developed a 2B-parameter variant with a significantly lighter footprint.
Read the full abstract ▾
All authors Thibault Formal, Maxime Louis, Hervé Dejean, Stéphane Clinchant
Machine Learning
FastODT: A tree-based framework for efficient continual learning
Daniel Bretsko, Piotr Walas, Devashish Khulbe et al. (+3 more)
Machine learning models deployed in real-world settings must operate under evolving data distributions and constrained computational resources. This challenge is particularly acute in non-stationary domains such as energy time series, weather monitoring, and environmental sensing. To remain effective, models must support adaptability, continuous learning, and long-term knowledge retention. This paper introduces a oblivious tree-based model with Hoeffding bound controlling its growth. It seamlessly integrates rapid learning and inference with efficient
memory management and robust knowledge preservation, thus allowing for online learning. Extensive experiments across energy and environmental sensing time-series
benchmarks demonstrate that the proposed framework achieves performance competitive with, and in several cases surpassing, existing online and batch learning methods, while maintaining superior computational efficiency. Collectively, these results demonstrate that the proposed approach fulfills the core objectives of adaptability, continual updating, and efficient retraining without full model retraining. The framework provides a scalable and resource-aware foundation for deployment in real-world non-stationary environments where resources are constrained and sustained adaptation is essential.
Read the full abstract ▾
All authors Daniel Bretsko, Piotr Walas, Devashish Khulbe, Sebastian Stros, Stanislav Sobolevsky, Tomas Satura
Machine Learning
PREBA: Surgical Duration Prediction via PCA-Weighted Retrieval-Augmented LLMs and Bayesian Averaging Aggregation
Wanyin Wu, Kanxue Li, Baosheng Yu et al. (+4 more)
Accurate prediction of
surgical duration is pivotal for hospital resource management. Although recent supervised learning approaches, from machine learning (ML) to fine-tuned large
language models (LLMs), have shown strong performance, they remain constrained by the need for high-quality labeled data and computationally intensive training. In contrast, zero-shot LLM inference offers a promising training-free alternative, but it lacks grounding in institution-specific clinical evidence, making its predictions clinically misaligned and prone to instability. To address these limitations, we present PREBA, a
retrieval-augmented framework that integrates PCA-weighted
retrieval and
Bayesian averaging to ground LLM
reasoning in institution-specific evidence without requiring task-specific fine-tuning. Specifically, PREBA first encodes heterogeneous clinical features into a unified representation. It then performs PCA-weighted
retrieval, weighting each dimension by its contribution to the variation across institutional cases, instead of treating them as equally informative, and conditions the LLM to generate multiple candidate predictions. Finally,
Bayesian averaging consolidates these predictions with population-level statistical priors, yielding a calibrated estimate that remains stable and clinically plausible. We evaluate PREBA on four real-world
datasets spanning different institutions and
languages, using three state-of-the-art LLMs, including Qwen3, DeepSeek-R1, and HuatuoGPT-o1. PREBA consistently outperforms zero-shot inference, reducing MAE by up to 64.9% and raising R2 to as high as 0.63. It further attains accuracy competitive with, and on several
datasets superior to, fully supervised ML methods, while offering greater interpretability through retrieved exemplars and explicit priors. The code is available at https://github.com/kanxueli/PREBA.
Read the full abstract ▾
All authors Wanyin Wu, Kanxue Li, Baosheng Yu, Haoyun Zhao, Yibing Zhan, Dapeng Tao, Hua Jin
Machine Learning
Learning from Partial Chain-of-Thought via Truncated-Reasoning Self-Distillation
Gianluigi Silvestri and Edoardo Cetin
Reasoning-oriented
language models achieve strong performance by generating long chain-of-thought traces at inference time. However, this capability comes with substantial and often excessive computational cost, which can materialize in redundant or inefficient
reasoning. We study this setting and introduce Truncated-
Reasoning Self-
Distillation (TRSD), a lightweight post-training procedure that encourages models to produce correct predictions from partial
reasoning traces. In TRSD, a frozen teacher model first generates a full
reasoning trace and evaluates the corresponding answer distribution conditioned on the
prompt and the complete
reasoning to construct a synthetic training target. A student model with the same architecture is then trained to match the teacher's answer distribution while being conditioned only on a truncated prefix of its
reasoning trace. Across multiple
reasoning benchmarks and
token budgets, we demonstrate that TRSD improves robustness to truncated inference, with far reduced accuracy tradeoffs when applied to a diverse set of
reasoning models. Moreover, although never explicitly regularized for shorter generation during training, we also find that TRSD-trained models inherently output shorter
reasoning traces without truncation, significantly reducing inference-time costs even without artificial interventions.
Read the full abstract ▾
Machine Learning
Spatially Aware Deep Learning for Microclimate Prediction from High-Resolution Geospatial Imagery
Idan Sulami, Alon Itzkovitch, Michael R. Kearney et al. (+2 more)
Microclimate models are essential for linking
climate to ecological processes, yet most physically based frameworks estimate temperature independently for each spatial unit and rely on simplified representations of lateral heat exchange. As a result, the spatial scales over which surrounding environmental conditions influence local microclimates remain poorly quantified. Here, we show how remote sensing can help quantify the contribution of spatial context to microclimate temperature predictions. Building on convolutional neural network principles, we designed a task-specific deep neural network and trained a series of models in which the spatial extent of input data was systematically varied. Drone-derived spatial layers and meteorological data were used to predict ground temperature at a focal location, allowing direct assessment of how prediction accuracy changes with increasing spatial context. Our results show that incorporating spatially adjacent information substantially improves prediction accuracy, with diminishing returns beyond spatial extents of approximately 5-7 m. This characteristic scale indicates that ground temperatures are influenced not only by local surface properties, but also by horizontal heat transfer and radiative interactions operating across neighboring microhabitats. The magnitude of spatial effects varied systematically with time of day, microhabitat type, and local environmental characteristics, highlighting context-dependent spatial coupling in microclimate formation. By treating deep learning as a diagnostic tool rather than solely a predictive one, our approach provides a general and transferable method for quantifying spatial dependencies in microclimate models and informing the development of hybrid mechanistic-data-driven approaches that explicitly account for spatial interactions while retaining physical interpretability.
Read the full abstract ▾
All authors Idan Sulami, Alon Itzkovitch, Michael R. Kearney, Moni Shahar, Ofir Levy
Machine Learning
CAMEL-CLIP: Channel-aware Multimodal Electroencephalography-text Alignment for Generalizable Brain Foundation Models
Hanseul Choi, Jinyeong Park, Seongwon Jin et al. (+2 more)
Electroencephalography (
EEG) foundation models have shown promise for learning generalizable representations, yet they remain sensitive to channel heterogeneity, such as changes in channel composition or ordering. We propose channel-aware
multimodal EEG-text
alignment contrastive
language image pretraining (CAMEL-CLIP), a contrastive
EEG-text
multimodal foundation model designed to be robust to heterogeneous channel configurations and widely applicable to diverse downstream tasks. CAMEL-CLIP combines three components: (1) channel attribute-based positional encoding, which identifies channels using their anatomical attributes and reference type; (2) dynamic channel projection, which preserves channel-specific information across varying numbers of
EEG channels and combines textual reports with channel information to support channel-wise
EEG-text matching; and (3) dual-level contrastive learning, which aligns
EEG signals with text at both the individual-channel and whole-sample levels. Experimental results show that CAMEL-CLIP achieves the highest mean balanced accuracy among the evaluated baselines across multiple
datasets under linear probing. CAMEL-CLIP also shows high classification performance when transferred between different channel configurations without classifier retraining.
Read the full abstract ▾
All authors Hanseul Choi, Jinyeong Park, Seongwon Jin, Sungho Park, Jibum Kim
Machine Learning · Cross-listed from q-bio.OT
Unsupervised Anomaly Detection in Wearable Foot Sensor Data: A Baseline Feasibility Study Towards Diabetic Foot Ulcer Prevention
Md Tanvir Hasan Turja
Diabetic foot ulcers (DFUs) are a severe complication of diabetes associated with significant morbidity, amputation risk, and healthcare burden. Developing effective continuous monitoring frameworks requires first establishing reliable baseline models of normal foot biomechanics. This paper presents a feasibility study of an anomaly detection framework applied to time-series data from wearable foot sensors, specifically NTC thin-film thermocouples for temperature and FlexiForce A401 pressure sensors for plantar load monitoring. Data were collected from healthy adult subjects across 312 capture sessions on an instrumented pathway, generating 93,790 valid multi-sensor readings spanning September 2023 to June 2024. Two unsupervised
algorithms, Isolation Forest and K-Nearest Neighbors using Local Outlier Factor (KNN/LOF), were applied to detect statistical deviations in foot temperature and pressure signals. Results show that Isolation Forest is more sensitive to subtle, distributed anomalies, while KNN/LOF identifies concentrated extreme deviations but flags a higher proportion of sessions not corroborated by Isolation Forest. Since no clinical ground truth is available, this difference is interpreted as lower specificity under the shared 5 percent contamination assumption rather than a confirmed false-positive rate. A mild positive correlation (0.41-0.48) between pressure and temperature features supports the case for combined multi-modal monitoring. These findings establish a validated baseline analytical pipeline and provide a methodological foundation for future clinical validation studies involving diabetic patients, where the relationship between detected anomalies and DFU-related pathophysiology can be directly assessed.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AR
HTM-EAR: Importance-Preserving Tiered Memory with Hybrid Routing under Saturation
Shubham Kumar Singh
Memory constraints in long-running
agents require structured management of accumulated facts while preserving essential information under bounded context limits. We introduce HTM-EAR, a hierarchical tiered
memory substrate that integrates HNSW-based working
memory (L1) with archival storage (L2), combining importance-aware eviction and hybrid routing. When L1 reaches capacity, items are evicted using a weighted score of importance and usage. Queries are first resolved in L1; if similarity or entity coverage is insufficient,
retrieval falls back to L2, and candidates are re-ranked using a cross-encoder. We evaluate the system under sustained saturation (15,000 facts; L1 capacity 500; L2 capacity 5000) using synthetic streams across five random seeds and real BGL system logs. Ablation studies compare the full system against variants without cross-encoder re-ranking, without routing gates, with LRU eviction, and an oracle with unbounded
memory. Under saturation, the full model preserves active-query precision (MRR = 1.000) while enabling controlled forgetting of stale history, approaching oracle active performance (0.997 +/- 0.003). In contrast, LRU minimizes latency (21.1 ms) but permanently evicts 2416 essential facts. On BGL logs, the full system achieves MRR 0.336, close to the oracle (0.370), while LRU drops to 0.069. Code is publicly available at: https://github.com/shubham-61291/HTM-EAR
Read the full abstract ▾
Machine Learning · Cross-listed from cs.NE
Robust Parameter and State Estimation in Multiscale Neuronal Systems Using Physics-Informed Neural Networks
Changliang Wei, Yangyang Wang and Xueyu Zhu
Inferring biophysical parameters and hidden state variables from partial and noisy observations is a fundamental challenge in computational neuroscience. This problem is particularly difficult for fast - slow spiking and bursting models, where strong nonlinearities, multiscale dynamics, and limited observational data often lead to severe sensitivity to initial parameter guesses and convergence failure in the methods replying on the traditional numerical forward solvers. In this work, we developed a physics-informed neural network (PINN) framework for the joint reconstruction of unobserved state variables and the estimation of unknown biophysical parameters in neuronal models. We demonstrate the effectiveness of the method on biophysical
neuron models, including the Morris-Lecar model across multiple spiking and bursting regimes and a respiratory model
neuron. The method requires only partial voltage observations over short observation windows and remains robust even when initialized with non-informative parameter guesses. These results suggest that PINN can deliver robust and accurate parameter inference and state reconstruction, providing a promising alternative for inverse problems in multiscale neuronal dynamics, where traditional techniques often struggle.
Read the full abstract ▾
Machine Learning
HURRI-GAN: A Novel Approach for Hurricane Bias-Correction Beyond Gauge Stations using Generative Adversarial Networks
Noujoud Nadera, Hadi Majed, Stefanos Giaremis et al. (+4 more)
The coastal regions of the eastern and southern United States are impacted by severe storm events, leading to significant loss of life and properties. Accurately
forecasting storm surge and wind impacts from hurricanes is essential for mitigating some of the impacts, e.g., timely preparation of evacuations and other countermeasures. Physical simulation models like the ADCIRC hydrodynamics model, which run on high-performance computing resources, are sophisticated tools that produce increasingly accurate forecasts as the resolution of the computational meshes improves. However, a major drawback of these models is the significant time required to generate results at very high resolutions, which may not meet the near real-time demands of emergency responders. The presented work introduces HURRI-GAN, a novel AI-driven approach that augments the results produced by physical simulation models using time series generative
adversarial networks (TimeGAN) to compensate for systemic errors of the physical model, thus reducing the necessary mesh size and runtime without loss in
forecasting accuracy. We present first results in extrapolating model bias corrections for the spatial regions beyond the positions of the water level gauge stations. The presented results show that our methodology can accurately generate bias corrections at target locations spatially beyond gauge stations locations. The model's performance, as indicated by low root mean squared error (RMSE) values, highlights its capability to generate accurate extrapolated data. Applying the corrections generated by HURRI-GAN on the ADCIRC modeled water levels resulted in improving the overall prediction on the majority of the testing gauge stations.
Read the full abstract ▾
All authors Noujoud Nadera, Hadi Majed, Stefanos Giaremis, Rola El Osta, Clint Dawson, Carola Kaiser, Hartmut Kaiser
Machine Learning
Trust Aware Federated Learning for Secure Bone Healing Stage Interpretation in e-Health
Paul Shepherd, Tasos Dagiuklas, Bugra Alkan et al. (+2 more)
This paper presents a trust aware
federated learning (FL) framework for interpreting bone healing stages using spectral features derived from frequency response data. The primary objective is to address the challenge posed by either unreliable or
adversarial participants in distributed medical sensing environments. The framework employs a multi-layer perceptron model trained across simulated clients using the Flower FL framework. The proposed approach integrates an Adaptive Trust Score Scaling and Filtering (ATSSSF) mechanism with exponential moving average (EMA) smoothing to assess, validate and filter client contributions.Two trust score smoothing strategies have been investigated, one with a fixed factor and another that adapts according to trust score variability. Clients with low trust are excluded from aggregation and readmitted once their reliability improves, ensuring model integrity while maintaining inclusivity. Standard classification metrics have been used to compare the performance of ATSSSF with the baseline
Federated Averaging strategy. Experimental results demonstrate that adaptive trust management can improve both training stability and predictive performance by mitigating the negative effects of compromised clients while retaining robust detection capabilities. The work establishes the feasibility for adaptive trust mechanisms in
federated medical sensing and identifies extension to clinical cross silo aggregation as a future research direction.
Read the full abstract ▾
All authors Paul Shepherd, Tasos Dagiuklas, Bugra Alkan, Joaquim Bastos, Jonathan Rodriguez
Machine Learning · Cross-listed from cs.CL
Induced Numerical Instability: Hidden Costs in Multimodal Large Language Models
Wai Tuck Wong, Jun Sun and Arunesh Sinha
The use of
multimodal large
language models has become widespread, and as such the study of these models and their failure points has become of utmost importance. We study a novel mode of failure that causes degradation in performance indirectly by optimizing a loss term that seeks to maximize numerical instability in the inference stage of these models. We apply this loss term as the optimization target to construct images that, when used on
multimodal large
language models, cause significant degradation in the output. We validate our hypothesis on state of the art models large
vision language models (LLaVa-v1.5-7B, Idefics3-8B, SmolVLM-2B-Instruct) against standard
datasets (Flickr30k, MMVet, TextVQA, VQAv2, POPE, COCO) and show that performance degrades significantly, even with a very small change to the input image, compared to baselines. Our results uncover a fundamentally different vector of performance degradation, highlighting a failure mode not captured by
adversarial perturbations.
Read the full abstract ▾
Machine Learning
On Emergences of Non-Classical Statistical Characteristics in Classical Neural Networks
Hanyu Zhao, Yang Wu and Yuexian Hou
Inspired by measurement incompatibility and Bell-family inequalities in
quantum mechanics, we propose the Non-Classical Network (NCnet), a simple classical neural architecture that stably exhibits non-classical statistical behaviors under typical and interpretable experimental setups. We find non-classicality, measured by the $S$ statistic of CHSH inequality, arises from gradient competitions of hidden-layer
neurons shared by multi-tasks. Remarkably, even without physical links supporting explicit communication, one task head can implicitly sense the training task of other task heads via local loss oscillations, leading to non-local correlations in their training outcomes. Specifically, in the low-resource regime, the value of $S$ increases gradually with increasing resources and approaches toward its classical upper-bound 2, which implies that underfitting is alleviated with resources increase. As the model nears the critical scale required for adequate performance, $S$ may temporarily exceed 2. As resources continue to grow, $S$ then asymptotically decays down to and fluctuates around 2. Empirically, when model capacity is insufficient, $S$ is positively correlated with generalization performance, and the regime where $S$ first approaches $2$ often corresponding to good generalization. Overall, our results suggest that non-classical statistics can provide a novel perspective for understanding internal interactions and training dynamics of deep networks.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Universal Conceptual Structure in Neural Translation: Probing NLLB-200's Multilingual Geometry
Kyle Elliott Mathewson
Do
multilingual translation models organize meaning around a shared conceptual space, or do they primarily encode
language identity and surface form? We probe the encoder geometry of Meta's 600M-parameter NLLB-200 model across 135
languages and 101 Swadesh concepts in contextualized carrier-frame and decontextualized bare-word regimes against null and confound controls. First, two-way variance partitioning reveals an additive
language-shift plus conceptual-
manifold geometry:
language shifts account for 45.7% of variance on bare words vs. 13.7% in context, and mean-centering improves the between-to-within-concept distance ratio (vs. a 0.99 permutation null) by 2.14x on bare words (1.15 to 2.46; 2.27x across disjoint scripts) and 1.19x in context (2.25 to 2.69), with conceptual separation peaking at Layer 10 (mutual k-NN overlap jumping 4.35x). Second, semantic category explains 25.4% of Swadesh convergence variance ($F(7,93)=4.53$, $p=2.21\times 10^{-4}$; partial $F=7.74$ controlling for fertility and length)—over 20x orthography or phonology ($R^2\le 0.012$)—while the advantage of Swadesh over non-loanword concrete nouns ($d=0.54$–$0.69$) is mediated by Zipfian subword compactness ($p=7.43\times 10^{-7}$). Third,
language distances track ASJP NLD and LDND phonetic distances within subfamilies under block-permutation and cross-script controls ($\rho_s=0.65$–$0.67$, block $p=10^{-4}$; global bare-word Mantel $\rho_s=0.24$, $p=0.001$), and CLICS$^3$ colexified concepts lie closer across string-distinct, within-category ($\rho_s=0.32$), and bare-word ($d=1.79$) controls ($d=1.43$, $p=1.55\times 10^{-33}$). Finally, basic color terms recover the Berlin and Kay hierarchy ($\rho_s=0.91$–$0.93$, $p\le 2.18\times 10^{-4}$), and semantic offsets exceed distance-matched controls ($p=3.55\times 10^{-5}$, $d=1.06$). We release InterpretCognates, an interactive toolkit and reproducible suite.
Read the full abstract ▾
Machine Learning · Cross-listed from eess.AS
Whisper-RIR-Mega: A Paired Clean-Reverberant Speech Benchmark for ASR Robustness to Room Acoustics
Mandip Goswami
We introduce Whisper-RIR-Mega, a
benchmark dataset of paired clean and reverberant
speech for evaluating automatic
speech recognition (ASR) robustness to room acoustics. Each sample pairs a clean LibriSpeech utterance with the same utterance convolved with a real room impulse response from the RIR-Mega corpus, with stratified splits by reverberation time (RT60) and direct-to-reverberant ratio (DRR). We evaluate five Whisper models (tiny through large-v3) on 1600 test samples and report word error rate (WER) and character error rate (CER) under clean and reverberant conditions. Reverberation consistently degrades performance across all model sizes; the reverb penalty in WER ranges from 2.31 to 15.50 percentage points depending on the model. Whisper-large-v3 shows the smallest penalty; Whisper-tiny shows the largest. We release the
dataset, evaluation code, and baseline results to support reproducible research on robust ASR.
Read the full abstract ▾
Machine Learning
Improving Full Waveform Inversion in Large Model Era
Yinan Feng, Peng Jin, Yuzhe Guo et al. (+2 more)
Full Waveform Inversion (FWI) is a highly nonlinear and ill-posed problem that aims to recover subsurface velocity maps from surface-recorded seismic waveforms data. Existing data-driven FWI typically uses small models, as available
datasets have limited volume, geological diversity, and spatial extent, leading to substantial concerns about overfitting. Although they perform well on synthetic
datasets, current methods fail to generalize to more realistic geological structures. In this work, we show that a model trained entirely on simulated and relatively simple data can generalize remarkably well to challenging and unseen geological
benchmarks. We provide a working recipe that tames a billion-parameter model for FWI through coordinated scaling across three axes: model capacity, data diversity, and training strategy. Our model achieves state-of-the-art performance on OpenFWI and significantly narrows the generalization gap in data-driven FWI. Across six challenging geophysical
benchmarks, including Marmousi, 2D SEG/EAGE Salt and Overthrust, 2004 BP, Sigsbee, and SEAM Phase I, it infers complex structures absent from the training set and delivers significant performance improvements (SSIM from 0.5844 to 0.7669). Overall, our results demonstrate that with an appropriate scaling strategy, large models trained on simple synthetic data can achieve substantial generalization to more complex and realistic geological structures.
Read the full abstract ▾
All authors Yinan Feng, Peng Jin, Yuzhe Guo, Yinpeng Chen, Youzuo Lin
Machine Learning
Deep Learning-Based Meat Freshness Detection with Segmentation and OOD-Aware Classification
Hutama Arif Bramantyo, Mukarram Ali Faridi, Rui Chen et al. (+2 more)
In this study, we present a meat freshness classification framework from Red-Green-Blue (RGB) images that supports both packaged and unpackaged meat
datasets. The system classifies four in-distribution (ID) meat classes and uses an out-of-distribution (OOD)-aware abstention mechanism that flags low-confidence samples as No Result. The pipeline combines U-Net-based
segmentation with deep feature classifiers.
Segmentation is used as a preprocessing step to isolate the meat region and reduce background, producing more consistent inputs for classification. The
segmentation module achieved an Intersection over Union (IoU) of 75% and a Dice coefficient of 82%, producing standardized inputs for the classification stage. For classification, we
benchmark five backbones: Residual Network-50 (ResNet-50),
Vision Transformer-Base/16 (ViT-B/16), Swin
Transformer-Tiny (Swin-T), EfficientNet-B0, and MobileNetV3-Small. We use nested 5x3 cross-validation (CV) for model selection and hyperparameter tuning. On the held-out ID test set, EfficientNet-B0 achieves the highest accuracy (98.10%), followed by ResNet-50 and MobileNetV3-Small (both 97.63%) and Swin-T (97.51%), while ViT-B/16 is lower (94.42%). We additionally evaluate OOD scoring and thresholding using standard OOD metrics and sensitivity analysis over the abstention threshold. Finally, we report on-device latency using TensorFlow Lite (TFLite) on a smartphone, highlighting practical accuracy-latency trade-offs for future deployment.
Read the full abstract ▾
All authors Hutama Arif Bramantyo, Mukarram Ali Faridi, Rui Chen, Clarissa Harris, Yin Sun
Machine Learning
Quantifying Catastrophic Forgetting in IoT Intrusion Detection Systems
Sourasekhar Banerjee, David Bergqvist, Salman Toor et al. (+2 more)
Distribution shifts in attack patterns within RPL-based IoT networks pose a critical threat to the reliability and security of large-scale connected systems. Intrusion Detection Systems (IDS) trained on static
datasets often fail to generalize to unseen threats and suffer from catastrophic forgetting when updated with new attacks. Ensuring continual adaptability of IDS is therefore essential for maintaining robust IoT network defence. In this focused study, we formulate intrusion detection as a domain continual learning problem and propose a method-agnostic IDS framework that can integrate diverse continual learning strategies. We systematically
benchmark five representative approaches across multiple domain-ordering sequences using a comprehensive multi-attack
dataset comprising 48 domains. Results show that continual learning mitigates catastrophic forgetting while maintaining a balance between plasticity, stability, and efficiency, a crucial aspect for resource-constrained IoT environments. Among the methods, Replay-based approaches achieve the best overall performance, while Synaptic Intelligence (SI) delivers near-zero forgetting with high training efficiency, demonstrating strong potential for stable and sustainable IDS deployment in dynamic IoT networks.
Read the full abstract ▾
All authors Sourasekhar Banerjee, David Bergqvist, Salman Toor, Christian Rohner, Andreas Johnsson
Machine Learning · Cross-listed from math.NA
KROM: Kernelized Reduced Order Modeling
Aras Bacho, Jonghyeon Lee and Houman Owhadi
We propose KROM, a kernel-based reduced-order framework for fast solution of nonlinear partial differential equations. KROM formulates PDE solution as a minimum-norm (Gaussian-process) recovery problem in an RKHS, and accelerates the resulting kernel solves by sparsifying the precision
matrix via sparse Cholesky factorization. A central ingredient is an empirical kernel constructed from a snapshot library of PDE solutions (generated under varying forcings, initial data, boundary data, or parameters). This snapshot-driven kernel adapts to problem-specific structure – boundary behavior, oscillations, nonsmooth features, linear constraints, conservation and dissipation laws – thereby reducing the dependence on hand-tuned stationary kernels. The resulting method yields an implicit reduced model: after sparsification, only a localized subset of effective degrees of freedom is used online. We report numerical results for semilinear
elliptic equations, discontinuous-coefficient Darcy flow, viscous Burgers, Allen–Cahn, and two-dimensional Navier–Stokes, showing that empirical kernels can match or outperform Matérn baselines, especially in nonsmooth regimes. We also provide error bounds that separate discretization effects, snapshot-space approximation error, and sparse-Cholesky approximation error.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
How Large Language Models Get Stuck: Early structure with persistent errors
Alokesh Manna, William Snyder and Whitney Tabor
Linguistic insights may help make Large
Language Model (LLM) training more efficient. We trained Meta's OPT model on the 100M word BabyLM
dataset, and evaluated it on the BLiMP
benchmark, which consists of 67 classes, each defined by sentence pairs that differ in a targeted syntactic or semantic rule violation. We tested the model's preference for grammatical over ungrammatical sentences across training iterations and grammatical types. In nearly one-third of the BLiMP classes, OPT fails to consistently assign a higher likelihood to grammatical sentences, even after extensive training. When it fails, it often establishes a clear (erroneous) separation of the likelihoods at an early stage of processing and sustains this to the end of our training phase. We hypothesize that this mis-categorization is costly because it creates entrenched biases that must, eventually, be reversed in order for the model to perform well. We probe this phenomenon using a mixture of qualitative (based on linguistic theory and the theory of Deep Learning) and quantitative (based on numerical testing) assessments. Our qualitative assessments indicate that only some BLiMP tests are meaningful guides. We conclude by articulating a hypothesis, the Bigram Hypothesis, which claims that the learning process will exhibit erroneous entrenchment if bigram statistics bias the model toward wrong distinctions early in training, and we describe a method of testing the hypothesis on appropriately selected BLiMP classes.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.DC
Token Management in Multi-Tenant AI Inference Platforms
William J. Cunningham
Multi-tenant AI inference platforms must balance resource utilization against service-level guarantees under variable demand. Conventional approaches fail to achieve this balance: dedicated endpoints strand capacity on idle models, while rate limits ignore the heterogeneous cost of inference requests. We introduce
token pools, a control-plane abstraction that represents inference capacity as explicit entitlements expressed in inference-native units (
token throughput, KV cache, concurrency). Unlike rate limits, which govern request admission without regard to execution cost,
token pools authorize both admission and autoscaling from the same capacity model, ensuring consistency between what is promised and what is provisioned. The abstraction captures burst modes across multiple dimensions invisible to conventional throttling. Dynamic per-entitlement limits on each burst dimension enable fine-grained control over resource consumption while permitting work-conserving backfill by low-priority traffic. The design supports priority-aware allocation, service tiers with differentiated guarantees, and debt-based
fairness mechanisms, all without modifying the underlying inference runtime or cluster scheduler. In experiments on a Kubernetes cluster with vLLM backends,
token pools maintain a bounded P99 latency for guaranteed workloads during overload by selectively throttling spot traffic, while a baseline without admission control experiences unbounded latency degradation across all workloads. A second experiment demonstrates debt-based fair-share convergence among elastic workloads with heterogeneous SLO requirements during capacity scarcity.
Read the full abstract ▾
Machine Learning
StethoLM: Audio Language Model for Cardiopulmonary Analysis Across Clinical Tasks
Yishan Wang, Tsai-Ning Wang, Mathias Funk et al. (+1 more)
Listening to heart and lung sounds - auscultation - is one of the first and most fundamental steps in a clinical examination. Despite being fast and non-invasive, it demands years of experience to interpret subtle audio cues. Recent deep learning methods have made progress in automating cardiopulmonary sound analysis, yet most are restricted to simple classification and offer little clinical interpretability or decision support. We present StethoLM, the first audio-
language model specialized for cardiopulmonary auscultation, capable of performing instruction-driven clinical tasks across the full spectrum of auscultation analysis. StethoLM integrates audio encoding with a medical
language model backbone and is trained on StethoBench, a comprehensive
benchmark comprising 77,027 instruction-response pairs synthesized from 16,125 labeled cardiopulmonary recordings spanning seven clinical task categories: binary classification, detection, reporting,
reasoning, differential diagnosis, comparison, and location-based analysis. Through multi-stage training that combines supervised fine-tuning and direct preference optimization, StethoLM achieves substantial gains in performance and robustness on out-of-distribution data. Our work establishes a foundation for instruction-following AI systems in clinical auscultation.
Read the full abstract ▾
All authors Yishan Wang, Tsai-Ning Wang, Mathias Funk, Aaqib Saeed
Machine Learning · Cross-listed from cs.RO
Acoustic Sensing for Universal Jamming Grippers
Lion Weber, Theodor Wienert, Martin Splettstößer et al. (+2 more)
Universal jamming grippers excel at grasping unknown objects due to their compliant bodies. Traditional tactile sensors can compromise this compliance, reducing grasping performance. We present acoustic sensing as a form of morphological sensing, where the gripper's soft body itself becomes the sensor. A speaker and microphone are placed inside the gripper cavity, away from the deformable membrane, fully preserving compliance. Sound propagates through the gripper and object, encoding object properties, which are then reconstructed via machine learning. Our sensor achieves high spatial resolution in sensing object size (2.6 mm error) and orientation (0.6 deg error), remains robust to external noise levels of 80 dBA, and discriminates object materials (up to 100% accuracy) and 16 everyday objects (85.6% accuracy). We validate the sensor in a realistic tactile object sorting task, achieving 53 minutes of uninterrupted grasping and sensing, confirming the preserved grasping performance. Finally, we demonstrate that disentangled acoustic representations can be learned, improving robustness to irrelevant acoustic variations.
Read the full abstract ▾
All authors Lion Weber, Theodor Wienert, Martin Splettstößer, Alexander Koenig, Oliver Brock
Machine Learning · Cross-listed from cs.CR
Challenges in Enabling Private Data Valuation
Yiwei Fu, Tianhao Wang and Varun Chandrasekaran
Data valuation methods quantify how individual training examples contribute to a model's behavior, and are increasingly used for
dataset curation,
auditing, and emerging data
markets. As these techniques become operational, they raise serious
privacy concerns: valuation scores can reveal whether a person's data was included in training, whether it was unusually influential, or what sensitive patterns exist in proprietary
datasets. This motivates the study of
privacy-preserving data valuation. However,
privacy is fundamentally in tension with valuation utility under differential
privacy (DP). DP requires outputs to be insensitive to any single record, while valuation methods are explicitly designed to measure per-record influence. As a result, naive privatization often destroys the fine-grained distinctions needed to rank or attribute value, particularly in heterogeneous
datasets where rare examples exert outsized effects. In this work, we analyze the feasibility of DP-compatible data valuation. We identify the core algorithmic primitives across common valuation frameworks that induce prohibitive sensitivity, explaining why straightforward DP mechanisms fail. We further derive design principles for more
privacy-amenable valuation procedures and empirically characterize how
privacy constraints degrade ranking fidelity across representative methods and
datasets. Our results clarify the limits of current approaches and provide a foundation for developing valuation methods that remain useful under rigorous
privacy guarantees.
Read the full abstract ▾