Machine Learning
Dissipative Learning: A Framework for Viable Adaptive Systems
Laurent Caraffa
We propose a perspective in which learning is an intrinsically dissipative process. Forgetting and regularization are not heuristic add-ons but structural requirements for adaptive systems. Drawing on information theory, thermodynamics, and information geometry, we introduce the BEDS (
Bayesian Emergent Dissipative Structures) framework, modeling learning as the
evolution of compressed belief states under dissipation constraints. A central contribution is the Conditional Optimality
Theorem, showing that Fisher-Rao regularization measuring change via information divergence rather than Euclidean distance is the unique thermodynamically optimal regularization strategy, achieving minimal dissipation. Euclidean regularization is shown to be structurally suboptimal. The framework unifies existing methods (Ridge, SIGReg, EMA, SAC) as special cases of a single governing equation. Within this view, overfitting corresponds to over-crystallization, while catastrophic forgetting reflects insufficient dissipation control. The framework distinguishes BEDS-crystallizable problems, where beliefs converge to stable
equilibria, from BEDS-maintainable problems, which require continual adaptation. It extends naturally to continual and multi-
agent systems, where viability, stability under adaptation and finite resources replaces asymptotic optimality as the primary criterion. Overall, this work reframes learning as maintaining viable belief states under dissipation constraints, providing a principled lens on forgetting, regularization, and stability.
Read the full abstract ▾
Machine Learning
Streaming-dLLM: Accelerating Diffusion LLMs via Suffix Pruning and Dynamic Decoding
Zhongyu Xiao, Zhiwei Hao, Jianyuan Guo et al. (+4 more)
Diffusion Large
Language Models (dLLMs) offer a compelling paradigm for natural
language generation, leveraging parallel decoding and bidirectional
attention to achieve superior global coherence compared to autoregressive models. While recent works have accelerated inference via KV cache reuse or heuristic decoding, they overlook the intrinsic inefficiencies within the block-wise
diffusion process. Specifically, they suffer from spatial redundancy by modeling informative-sparse suffix regions uniformly and temporal inefficiency by applying fixed denoising schedules across all the decoding process. To address this, we propose Streaming-dLLM, a training-free framework that streamlines inference across both spatial and temporal dimensions. Spatially, we introduce attenuation guided suffix modeling to approximate the full context by
pruning redundant mask
tokens. Temporally, we employ a dynamic confidence aware strategy with an early exit mechanism, allowing the model to skip unnecessary iterations for converged
tokens. Extensive experiments show that Streaming-dLLM achieves up to 68.2X speedup while maintaining generation quality, highlighting its effectiveness in
diffusion decoding. The code is available at https://github.com/xiaoshideta/Streaming-dLLM.
Read the full abstract ▾
All authors Zhongyu Xiao, Zhiwei Hao, Jianyuan Guo, Yong Luo, Jia Liu, Jie Xu, Han Hu
Machine Learning
UniPACT: A Multimodal Framework for Prognostic Question Answering on Raw ECG and Structured EHR
Jialu Tang, Tong Xia, Yuan Lu et al. (+1 more)
Accurate clinical prognosis requires synthesizing structured Electronic Health Records (EHRs) with real-time physiological signals like the Electrocardiogram (ECG). Large
Language Models (LLMs) offer a powerful
reasoning engine for this task but struggle to natively process these heterogeneous, non-textual data types. To address this, we propose UniPACT (Unified Prognostic Question Answering for Clinical Time-series), a unified framework for prognostic question answering that bridges this modality gap. UniPACT's core contribution is a structured prompting mechanism that converts numerical EHR data into semantically rich text. This textualized patient context is then fused with representations learned directly from raw ECG waveforms, enabling an LLM to reason over both modalities holistically. We evaluate UniPACT on the comprehensive MDS-ED
benchmark, it achieves a state-of-the-art mean AUROC of 89.37% across a diverse set of prognostic tasks including diagnosis, deterioration, ICU admission, and mortality, outperforming specialized baselines. Further analysis demonstrates that our
multimodal, multi-task approach is critical for performance and provides robustness in missing data scenarios.
Read the full abstract ▾
All authors Jialu Tang, Tong Xia, Yuan Lu, Aaqib Saeed
Machine Learning · Cross-listed from cs.AI
Think Locally, Explain Globally: Graph-Guided LLM Investigations via Local Reasoning and Belief Propagation
Saurabh Jha, Rohan Arora, Bhavya et al. (+7 more)
LLM
agents excel when environments are mostly static and the needed information fits in a model's context window, but they often fail in open-ended investigations where explanations must be constructed by iteratively mining evidence from massive, heterogeneous operational data. These investigations exhibit hidden dependency structure: entities interact, signals co-vary, and the importance of a fact may only become clear after other evidence is discovered. Because the context window is bounded,
agents must summarize intermediate findings before their significance is known, increasing the risk of discarding key evidence. ReAct-style
agents are especially brittle in this regime. Their retrieve-summarize-reason loop makes conclusions sensitive to exploration order and introduces run-to-run non-determinism, producing a reliability gap where Pass-at-k may be high but Majority-at-k remains low. Simply sampling more rollouts or generating longer
reasoning traces does not reliably stabilize results, since hypotheses cannot be autonomously checked as new evidence arrives and there is no explicit mechanism for belief bookkeeping and revision. In addition, ReAct entangles semantic
reasoning with controller duties such as tool orchestration and state tracking, so execution errors and plan drift degrade
reasoning while consuming scarce context. We address these issues by formulating investigation as abductive
reasoning over a dependency graph and proposing EoG (Explanations over Graphs), a disaggregated framework in which an LLM performs bounded local evidence mining and labeling (cause vs symptom) while a deterministic controller manages traversal, state, and belief propagation to compute a minimal explanatory frontier. On a representative ITBench diagnostics task, EoG improves both accuracy and run-to-run consistency over ReAct baselines, including a 7x average gain in Majority-at-k entity F1.
Read the full abstract ▾
All authors Saurabh Jha, Rohan Arora, Bhavya, Noah Zheutlin, Paulina Toro Isaza, Laura Shwartz, Yu Deng, Daby Sow, Ruchi Mahindru, Ruchir Puri
Machine Learning
Causal Pre-training Under the Fairness Lens: An Empirical Study of TabPFN
Qinyi Liu, Mohammad Khalil and Naman Goel
Foundation models for tabular data, such as the Tabular Prior-data Fitted Network (TabPFN), are pre-trained on a massive number of synthetic
datasets generated by structural causal models (SCM). They leverage in-context learning to offer high predictive accuracy in real-world tasks. However, the
fairness properties of these foundational models, which incorporate ideas from causal
reasoning during pre-training, remain underexplored. In this work, we conduct a comprehensive empirical evaluation of TabPFN and its fine-tuned variants, assessing predictive performance,
fairness, and robustness across varying
dataset sizes and distributional shifts. Our results reveal that while TabPFN achieves stronger predictive accuracy compared to baselines and exhibits robustness to spurious correlations, improvements in
fairness are moderate and inconsistent, particularly under missing-not-at-random (MNAR)
covariate shifts. These findings suggest that the causal pre-training in TabPFN is helpful but insufficient for algorithmic
fairness, highlighting implications for deploying TabPFN (and similar) models in practice and the need for further
fairness interventions.
Read the full abstract ▾
Machine Learning
Adaptive Weighting in Knowledge Distillation: An Axiomatic Framework for Multi-Scale Teacher Ensemble Optimization
Aaron R. Flouro and Shawn P. Chadwick
Knowledge
distillation with multiple teachers is increasingly used to improve robustness, efficiency, and safety, yet existing approaches rely largely on heuristic or implementation-specific weighting schemes. This paper develops an operator-agnostic axiomatic framework for adaptive weighting in multi-teacher knowledge
distillation across three complementary scales:
token, task, and context. We formalize structural conditions under which adaptive weighting operators are well-defined, admit multiple non-equivalent implementations, and can be hierarchically composed via product-structure normalization. Within this framework, we establish existence and non-uniqueness of conforming operators, characterize convergence of gradient-based optimization under standard assumptions, analyze stability and perturbation robustness, and provide an abstract formulation of safety-constrained
distillation. The results decouple theoretical guarantees from specific weighting formulas, enabling principled analysis of adaptive
distillation methods under heterogeneity, distribution shift, and safety constraints.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CR
FARM: Few-shot Adaptive Malware Family Classification under Concept Drift
Numan Halit Guldemir, Oluwafemi Olukoya and Jesús Martínez-del-Rincón
Malware classification models often suffer performance degradation under concept drift due to evolving threat landscapes and the emergence of novel malware families. This paper presents FARM (Few-shot Adaptive Recognition of Malware), a unified framework for detecting and adapting to both
covariate drift and label drift in Windows Portable Executable (PE) malware family classification. FARM uses a triplet autoencoder to project samples into a discriminative latent space, enabling unsupervised drift detection through DBSCAN clustering and dynamic thresholding. To enable rapid adaptation, the framework employs a few-shot strategy that can incorporate new classes from only a small number of labeled samples. FARM also supports full retraining when sufficient drifted samples accumulate, allowing longer-term model updating. Experiments on the BenchMFC
dataset show that FARM improves classification performance under
covariate drift by 5.6%, and achieves an average F1 score of 0.85 on unseen malware families using few-shot adaptation, increasing to 0.94 after retraining. These results indicate that FARM provides an effective approach for drift-aware malware family classification in dynamic environments with limited supervision.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CY
Artificial Intelligence and Intellectual Property Rights: Comparative Transnational Policy Analysis
Sahibpreet Singh and Manjit Singh
Artificial intelligence's rapid integration with intellectual property rights necessitates assessment of its impact on trade secrets, copyrights and patents. This study addresses lacunae in existing laws where India lacks AI-specific provisions, creating doctrinal inconsistencies and enforcement inefficacies. Global discourse on AI-IPR protections remains nascent. The research identifies gaps in Indian IP laws' adaptability to AI-generated outputs: trade secret protection is inadequate against AI threats; standardized inventorship criteria are absent. Employing doctrinal and comparative methodology, it scrutinizes legislative texts, judicial precedents and policy instruments across India, US, UK and EU. Preliminary findings reveal shortcomings: India's contract law creates fragmented trade secret regime; Section 3(k) of Indian Patents Act blocks AI invention patenting; copyright varies in authorship attribution. The study proposes harmonized legal taxonomy accommodating AI's role while preserving innovation incentives. India's National AI Strategy (2024) shows progress but legislative clarity is imperative. This contributes to global discourse with AI-specific IP protections ensuring resilience and equitable innovation. Promising results underscore recalibrating India's IP jurisprudence for global
alignment.
Read the full abstract ▾
Machine Learning
EEG-FM-Compass: Progress, Benchmarking, and Future Directions for EEG Foundation Models
Dingkun Liu, Yuheng Chen, Zhu Chen et al. (+5 more)
Electroencephalography (
EEG) foundation models (FMs) have recently emerged as a promising paradigm for
brain-computer interfaces, aiming to learn transferable neural representations from large-scale heterogeneous recordings. Despite rapid progress, a fair and comprehensive comparison of existing
EEG FMs is still lacking, owing to inconsistent pre-training objectives, preprocessing choices, and downstream evaluation protocols. To fill this gap, we present
EEG-FM-Compass. We first review 55 representative models and organize their design choices into a unified taxonomic framework including data standardization, model architectures, and self-supervised pre-training strategies. We then evaluate 12 open source FMs and competitive specialist baselines across 13
EEG datasets spanning nine
brain-computer interface paradigms. Emphasizing real-world deployments, we consider both cross-subject generalization under a leave-one-subject-out protocol and rapid calibration under a within-subject few-shot setting. We further compare full-parameter fine-tuning with linear probing to assess the transferability of pre-trained representations, and examine the relationship between model scale and downstream performance. Our results indicate that: 1) linear probing is frequently insufficient; 2) specialist models trained from scratch remain competitive across many tasks; and 3) larger FMs do not necessarily yield better generalization performance under current data regimes and training practices.
Read the full abstract ▾
All authors Dingkun Liu, Yuheng Chen, Zhu Chen, Zhenyao Cui, Yaozhi Wen, Jiayu An, Jingwei Luo, Dongrui Wu
Machine Learning · Cross-listed from cs.CY
Comparative Algorithmic Governance of Public Health Instruments across India, EU, US and LMICs
Sahibpreet Singh
The study investigates the juridico-technological architecture of international public health instruments, focusing on their implementation across India, the European Union, the United States and low- and middle-income countries (LMICs), particularly in Sub-Saharan Africa. It addresses a research lacuna: the insufficient harmonisation between normative health law and algorithmic public health infrastructures in resource-constrained jurisdictions. The principal objective is to assess how artificial intelligence augments implementation of instruments grounded in IHR 2005 and the WHO FCTC while identifying doctrinal and infrastructural bottlenecks. Using comparative doctrinal analysis and legal-normative mapping, the study triangulates legislative instruments, WHO monitoring frameworks, AI systems including BlueDot, Aarogya Setu and EIOS, and compliance metrics. Preliminary results show that AI has improved early detection, surveillance precision and responsiveness in high-capacity jurisdictions, whereas LMICs face infrastructural deficits, data
privacy gaps and fragmented legal scaffolding. The findings highlight the relevance of the EU Artificial Intelligence Act and GDPR as regulatory prototypes for health-oriented algorithmic governance and contrast them with embryonic AI integration and limited internet penetration in many LMICs. The study argues for
embedding AI within a rights-compliant, supranationally coordinated regulatory framework to secure equitable health outcomes and stronger compliance. It proposes a model for algorithmic treaty-making inspired by FCTC architecture and calls for WHO-led compliance mechanisms modelled on the WTO Dispute Settlement Body to enhance pandemic preparedness, surveillance equity and transnational governance resilience.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
On the Emergence and Test-Time Use of Structural Information in Large Language Models
Michelle Chao Chen, Moritz Miller, Bernhard Schölkopf et al. (+1 more)
Learning structural information from observational data is central to producing new knowledge outside the training corpus. This holds for mechanistic understanding in scientific discovery as well as flexible test-time compositional generation. We thus study how
language models learn abstract structures and utilize the learnt structural information at test-time. To ensure a controlled setup, we design a natural
language dataset based on linguistic structural transformations. We empirically show that the emergence of learning structural information correlates with complex
reasoning tasks, and that the ability to perform test-time compositional generation remains limited.
Read the full abstract ▾
All authors Michelle Chao Chen, Moritz Miller, Bernhard Schölkopf, Siyuan Guo
Machine Learning
MergeMix: Optimizing Mid-Training Data Mixtures via Learnable Model Merging
Jiapeng Wang, Changxin Tian, Kunlong Chen et al. (+5 more)
Optimizing data mixtures is essential for unlocking the full potential of large
language models (LLMs), yet identifying the optimal composition remains computationally prohibitive due to reliance on heuristic trials or expensive proxy training. To address this, we introduce MergeMix, a novel approach that efficiently determines optimal data mixing ratios by repurposing model merging weights as a high-fidelity, low-cost performance proxy. By training domain-specific experts on minimal
tokens and optimizing their merging weights against downstream
benchmarks, MergeMix effectively optimizes the performance of data mixtures without incurring the cost of full-scale training. Extensive experiments on models with 8B and 16B parameters validate that MergeMix achieves performance comparable to or surpassing exhaustive manual tuning while drastically reducing search costs. Furthermore, MergeMix exhibits high rank consistency (Spearman $\rho > 0.9$) and strong cross-scale transferability, offering a scalable, automated solution for data mixture optimization.
Read the full abstract ▾
All authors Jiapeng Wang, Changxin Tian, Kunlong Chen, Ziqi Liu, Jiaxin Mao, Wayne Xin Zhao, Zhiqiang Zhang, Jun Zhou
Machine Learning · Cross-listed from cs.HC
RAICL: Retrieval-Augmented In-Context Learning for Vision-Language-Model Based EEG Seizure Detection
Siyang Li, Zhuoya Wang, Xiyan Gui et al. (+4 more)
Electroencephalogram (
EEG) decoding is a critical component of medical diagnostics, rehabilitation engineering, and
brain-computer interfaces. However, contemporary decoding methodologies remain heavily dependent on task-specific
datasets to train specialized neural network architectures. Consequently, limited data availability impedes the development of generalizable large
brain decoding models. In this work, we propose a paradigm shift from conventional signal-based decoding by leveraging large-scale
vision-
language models (VLMs) to analyze
EEG waveform plots. By converting multivariate
EEG signals into stacked waveform images and integrating neuroscience domain expertise into textual
prompts, we demonstrate that foundational VLMs can effectively differentiate between different patterns in the human
brain. To address the inherent non-stationarity of
EEG signals, we introduce a
Retrieval-Augmented In-Context Learning (RAICL) approach, which dynamically selects the most representative and relevant few-shot examples to condition the autoregressive outputs of the VLM. Experiments on
EEG-based seizure detection indicate that state-of-the-art VLMs under RAICL achieved better or comparable performance with traditional time series based approaches. These findings suggest a new direction in physiological signal processing that effectively bridges the modalities of
vision,
language, and neural activities. Furthermore, the utilization of off-the-shelf VLMs, without the need for retraining or downstream architecture construction, offers a readily deployable solution for clinical applications.
Read the full abstract ▾
All authors Siyang Li, Zhuoya Wang, Xiyan Gui, Xiaoqing Chen, Ziwei Wang, Yaozhi Wen, Dongrui Wu
Machine Learning · Cross-listed from cond-mat.mtrl-sci
An autonomous living database for perovskite photovoltaics
Sherjeel Shabih, Hampus Näsström, Sharat Patil et al. (+16 more)
Scientific discovery is severely bottlenecked by the inability of manual curation to keep pace with exponential publication rates. This creates a widening knowledge gap. This is especially stark in photovoltaics, where the leading database for perovskite
solar cells has been stagnant since 2021 despite massive ongoing research output. Here, we resolve this challenge by establishing an autonomous, self-updating living database (PERLA). Our pipeline integrates large
language models with physics-aware validation to extract complex device data from the continuous literature stream, achieving human-level precision (>90%) and eliminating annotator variance. By employing this system on the previously inaccessible post-2021 literature, we uncover critical evolutionary trends hidden by data lag: the field has decisively shifted toward inverted architectures employing self-assembled monolayers and formamidinium-rich compositions, driving a clear trajectory of sustained voltage loss reduction. PERLA transforms static publications into dynamic knowledge resources that enable data-driven discovery to operate at the speed of publication.
Read the full abstract ▾
All authors Sherjeel Shabih, Hampus Näsström, Sharat Patil, Asmin Askin, Keely Dodd-Clements, Jessica Helisa Hautrive Rossato, Hugo Gajardoni de Lemos, Yuxin Liu, Florian Mathies, Natalia Maticiuc, Rico Meitzner, Edgar Nandayapa, Juan José Patiño López, Yaru Wang, Lauri Himanen, Eva Unger, T. Jesper Jacobsson, José A. Márquez, Kevin Maik Jablonka
Machine Learning
Robust Computational Extraction of Non-Enhancing Hypercellular Tumor Regions from Clinical Imaging Data
A. Brawanski, Th. Schaffer, F. Raab et al. (+5 more)
Accurate identification of non-enhancing hypercellular (NEH) tumor regions is an unmet need in neuro-oncological imaging, with significant implications for patient management and treatment planning. We present a robust computational framework that generates probability maps of NEH regions from routine MRI data, leveraging multiple network architectures to address the inherent variability and lack of clear imaging boundaries. Our approach was validated against independent clinical markers – relative cerebral blood volume (rCBV) and enhancing tumor recurrence location (ETRL) – demonstrating both methodological robustness and biological relevance. This framework enables reliable, non-invasive mapping of NEH tumor compartments, supporting their integration as imaging biomarkers in clinical workflows and advancing precision oncology for
brain tumor patients.
Read the full abstract ▾
All authors A. Brawanski, Th. Schaffer, F. Raab, K.-M. Schebesch, M. Schrey, Chr. Doenitz, A. M. Tomé, E. W. Lang
Machine Learning · Cross-listed from math.OC
Differentiable Integer Linear Programming is not Differentiable & it's not a mere technical problem
Thanawat Sornwanee
We show how the differentiability method employed in the paper “Differentiable Integer Linear Programming”, Geng, et al., 2025 as shown in its
theorem 5 is incorrect. Moreover, there already exists some downstream work that inherits the same error. The underlying reason comes from that, though being continuous in expectation, the surrogate loss is discontinuous in almost every realization of the randomness, for the
stochastic gradient descent.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Beyond a Single Perspective: Text Anomaly Detection with Multi-View Language Representations
Yixin Liu, Kehan Yan, Shiyuan Li et al. (+2 more)
Text anomaly detection (TAD) plays a critical role in various
language-driven real-world applications, including harmful content moderation, phishing detection, and spam review filtering. While two-step "
embedding-detector" TAD methods have shown state-of-the-art performance, their effectiveness is often limited by the use of a single
embedding model and the lack of adaptability across diverse
datasets and anomaly types. To address these limitations, we propose to exploit the
embeddings from multiple pretrained
language models and integrate them into $MCA^2$, a multi-view TAD framework. $MCA^2$ adopts a multi-view reconstruction model to effectively extract normal textual patterns from multiple
embedding perspectives. To exploit inter-view complementarity, a contrastive collaboration module is designed to leverage and strengthen the interactions across different views. Moreover, an adaptive allocation module is developed to automatically assign the contribution weight of each view, thereby improving the adaptability to diverse
datasets. Extensive experiments on 10
benchmark datasets verify the effectiveness of $MCA^2$ against strong baselines. The source code of $MCA^2$ is available at https://github.com/yankehan/MCA2.
Read the full abstract ▾
All authors Yixin Liu, Kehan Yan, Shiyuan Li, Qingfeng Chen, Shirui Pan
Machine Learning
Shortcut Learning in Binary Classifier Black Boxes: Applications to Voice Anti-Spoofing and Biometrics
Md Sahidullah, Hye-jin Shim, Rosa Gonzalez Hautamäki et al. (+1 more)
The widespread adoption of deep-learning models in data-driven applications has drawn
attention to the potential risks associated with biased
datasets and models. Neglected or hidden biases within
datasets and models can lead to unexpected results. This study addresses the challenges of
dataset bias and explores “shortcut learning” or “Clever Hans effect” in binary classifiers. We propose a novel framework for analyzing the black-box classifiers and for examining the impact of both training and test data on classifier scores. Our framework incorporates intervention and observational perspectives, employing a linear mixed-effects model for post-hoc analysis. By evaluating classifier performance beyond error rates, we aim to provide insights into biased
datasets and offer a comprehensive understanding of their influence on classifier behavior. The effectiveness of our approach is demonstrated through experiments on audio anti-spoofing and speaker verification tasks using both statistical models and deep neural networks. The insights gained from this study have broader implications for tackling biases in other domains and advancing the field of explainable artificial intelligence.
Read the full abstract ▾
All authors Md Sahidullah, Hye-jin Shim, Rosa Gonzalez Hautamäki, Tomi H. Kinnunen
Machine Learning · Cross-listed from q-fin.ST
MarketGANs: Multivariate financial time-series data augmentation using generative adversarial networks
Jeonggyu Huh, Seungwon Jeong, Hyun-Gyoon Kim et al. (+2 more)
This paper introduces MarketGAN, a factor-based generative framework for high-dimensional asset return generation under severe data scarcity. We embed an explicit asset-pricing factor structure as an economic inductive bias and generate returns as a single joint vector, thereby preserving cross-sectional dependence and tail co-movement alongside inter-temporal dynamics. MarketGAN employs generative
adversarial learning with a temporal convolutional network (TCN) backbone, which models
stochastic, time-varying factor loadings and
volatilities and captures long-range temporal dependence. Using daily returns of large U.S. equities, we find that MarketGAN more closely matches empirical stylized facts of asset returns, including heavy-tailed marginal distributions,
volatility clustering, leverage effects, and, most notably, high-dimensional cross-sectional correlation structures and tail co-movement across assets, than conventional factor-model-based bootstrap approaches. In portfolio applications, covariance estimates derived from MarketGAN-generated samples outperform those derived from other methods when factor information is at least weakly informative, demonstrating tangible economic value.
Read the full abstract ▾
All authors Jeonggyu Huh, Seungwon Jeong, Hyun-Gyoon Kim, Hyeng Keun Koo, Byung Hwa Lim
Machine Learning
LLM-42: Enabling Determinism in LLM Inference with Verified Speculation
Raja Gond, Aditya K Kamath, Ramachandran Ramjee et al. (+1 more)
In LLM inference, the same
prompt may yield different outputs across different runs. At the system level, this non-determinism arises from floating-point non-associativity combined with dynamic batching and GPU kernels whose reduction orders vary with batch size. A straightforward way to eliminate non-determinism is to disable dynamic batching during inference, but doing so severely degrades throughput. Another approach is to make kernels batch-invariant; however, this tightly couples determinism to kernel design, requiring new implementations. This coupling also imposes fixed runtime overheads, regardless of how much of the workload actually requires determinism. Inspired by ideas from speculative decoding, we present LLM-42, a scheduling-based approach to enable determinism in LLM inference. Our key observation is that if a sequence is in a consistent state, the next emitted
token is likely to be consistent even with dynamic batching. Moreover, most GPU kernels use shape-consistent reductions. Leveraging these insights, LLM-42 decodes
tokens using a non-deterministic fast path and enforces determinism via a lightweight verify-rollback loop. The verifier replays candidate
tokens under a fixed-shape reduction schedule, commits those that are guaranteed to be consistent across runs, and rolls back those violating determinism. LLM-42 mostly re-uses existing kernels unchanged and incurs overhead only in proportion to the traffic that requires determinism.
Read the full abstract ▾
All authors Raja Gond, Aditya K Kamath, Ramachandran Ramjee, Ashish Panwar
Machine Learning · Cross-listed from cs.AI
HyCARD-Net: A Synergistic Hybrid Intelligence Framework for Cardiovascular Disease Diagnosis
Rajan Das Gupta, Xiaobin Wu, Xun Liu et al. (+1 more)
Cardiovascular disease (CVD) remains the foremost cause of mortality worldwide, underscoring the urgent need for intelligent and data-driven diagnostic tools. Traditional predictive models often struggle to generalize across heterogeneous
datasets and complex physiological patterns. To address this, we propose a hybrid ensemble framework that integrates deep learning architectures, Convolutional Neural Networks (CNN) and Long Short-Term
Memory (LSTM), with classical machine learning
algorithms, including K-Nearest Neighbor (KNN) and Extreme Gradient Boosting (XGB), using an ensemble voting mechanism. This approach combines the representational power of deep networks with the interpretability and efficiency of traditional models. Experiments on two publicly available Kaggle
datasets demonstrate that the proposed model achieves superior performance, reaching 82.30 percent accuracy on
Dataset I and 97.10 percent on
Dataset II, with consistent gains in precision, recall, and F1-score. These findings underscore the robustness and clinical potential of hybrid AI frameworks for predicting cardiovascular disease and facilitating early intervention. Furthermore, this study directly supports the United Nations Sustainable Development Goal 3 (Good Health and Well-being) by promoting early diagnosis, prevention, and management of non-communicable diseases through innovative, data-driven healthcare solutions.
Read the full abstract ▾
All authors Rajan Das Gupta, Xiaobin Wu, Xun Liu, Jiaqi He
Machine Learning
AR-Omni: A Unified Autoregressive Model for Any-to-Any Generation
Dongjie Cheng, Ruifeng Yuan, Yongqi Li et al. (+5 more)
Real-world perception and interaction are inherently
multimodal, encompassing not only
language but also
vision and
speech, which motivates the development of "Omni" MLLMs that support both
multimodal inputs and
multimodal outputs. While a sequence of omni MLLMs has emerged, most existing systems still rely on additional expert components to achieve
multimodal generation, limiting the simplicity of unified training and inference. Autoregressive (AR) modeling, with a single
token stream, a single next-
token objective, and a single decoder, is an elegant and scalable foundation in the text domain. Motivated by this, we present AR-Omni, a unified any-to-any model in the autoregressive paradigm without any expert decoders. AR-Omni supports autoregressive text and image generation, as well as streaming
speech generation, all under a single
Transformer decoder. We further address three practical issues in unified AR modeling: modality imbalance via task-aware loss reweighting, visual fidelity via a lightweight
token-level perceptual
alignment loss for image
tokens, and stability-creativity trade-offs via a finite-state decoding mechanism. Empirically, AR-Omni achieves strong quality across three modalities while remaining real-time, achieving a 0.88 real-time factor for
speech generation.
Read the full abstract ▾
All authors Dongjie Cheng, Ruifeng Yuan, Yongqi Li, Runyang You, Wenjie Wang, Liqiang Nie, Lei Zhang, Wenjie Li
Machine Learning · Cross-listed from eess.SP
Over-The-Air Extreme Learning Machines with XL Reception via Nonlinear Cascaded Metasurfaces
Kyriakos Stylianopoulos, Mattia Fabiani, Giulia Torcolacci et al. (+2 more)
The recently envisioned goal-oriented communications paradigm calls for the application of inference on wirelessly transferred data via Machine Learning (ML) tools. An emerging research direction deals with the realization of inference ML models directly in the physical layer of Multiple-Input Multiple-Output (
MIMO) systems, which, however, entails certain significant challenges. In this paper, leveraging the technology of programmable MetaSurfaces (MSs), we present an eXtremely Large (XL)
MIMO system that acts as an Extreme Learning Machine (ELM) performing binary classification tasks completely Over-The-Air (OTA), which can be trained in closed form. The proposed system comprises a receiver architecture consisting of densely parallel placed diffractive layers of XL MSs, also known as Stacked Intelligent Metasurfaces (SIM), followed by a single reception
radio-frequency chain. The front layer facing the XL
MIMO channel consists of identical unit
cells of a fixed NonLinear (NL) response, whereas the remaining layers of elements of tunable linear responses are utilized to approximate OTA the trained ELM weights. Our numerical investigations showcase that, in the XL regime of MS elements, the proposed XL-
MIMO-ELM system achieves performance comparable to that of digital and idealized ML models across diverse
datasets and wireless scenarios, thereby demonstrating the feasibility of
embedding OTA learning capabilities into future wireless systems.
Read the full abstract ▾
All authors Kyriakos Stylianopoulos, Mattia Fabiani, Giulia Torcolacci, Davide Dardari, George C. Alexandropoulos
Machine Learning · Cross-listed from cs.AI
A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Data
Kaituo Zhang, Mingzhi Hu, Hoang Anh Duy Le et al. (+9 more)
Large
Language Models (LLMs) have emerged as powerful tools for generating data across various modalities. By transforming data from a scarce resource into a controllable asset, LLMs mitigate the bottlenecks imposed by the acquisition costs of real-world data for model training, evaluation, and system iteration. However, ensuring the high quality of LLM-generated synthetic data remains a critical challenge. Existing research primarily focuses on generation methodologies, with limited direct
attention to the quality of the resulting data. Furthermore, most studies are restricted to single modalities, lacking a unified perspective across different data types. To bridge this gap, we propose the LLM Data Auditor framework. In this framework, we first describe how LLMs are utilized to generate data across six distinct modalities. More importantly, we systematically categorize intrinsic metrics for evaluating synthetic data from two dimensions: quality and trustworthiness. This approach shifts the focus from extrinsic evaluation, which relies on downstream task performance, to the inherent properties of the data itself. Using this evaluation system, we analyze the experimental evaluations of representative generation methods for each modality and identify substantial deficiencies in current evaluation practices. Based on these findings, we offer concrete recommendations for the community to improve the evaluation of data generation. Finally, the framework outlines methodologies for the practical application of synthetic data across different modalities.
Read the full abstract ▾
All authors Kaituo Zhang, Mingzhi Hu, Hoang Anh Duy Le, Fariha Kabir Torsha, Zhimeng Jiang, Minh Khai Bui, Chia-Yuan Chang, Yu-Neng Chuang, Zhen Xiong, Ying Lin, Guanchu Wang, Na Zou
Machine Learning
Do Reasoning Models Ask Better Questions? A Formal Information-Theoretic Analysis on Multi-Turn LLM Games
Daniel M. Pedrozo, Telma W. de L. Soares and Bryan L. M. de Oliveira
Large
Language Models (LLMs) excel at many tasks but still struggle with a critical ability for LLM-based
agents: asking good questions for resolving ambiguity in user requests. While prior work has explored information-seeking behavior through word games, existing
benchmarks lack comprehensive evaluation frameworks that provide both final and intermediate signals based on Information Gain (IG). Moreover, they rarely provide systematic comparisons between models that use chain-of-thought
reasoning and those that do not. We propose a multi-turn dialogue framework that quantitatively measures how effectively LLMs gather information through yes/no questions in a hierarchical knowledge graph environment. Our framework employs a triad of interacting LLM
agents that ask questions, answer them, and update the hypothesis space. We adopt IG as the main metric, grounded in Shannon
entropy, to assess query effectiveness at each turn and cumulatively. We instantiate our framework in a geographical Guess My City game setting organized in a five-level taxonomy and evaluate multiple LLM variants under fully and partially observable conditions, with and without Chain-of-Thought
reasoning. Our experiments demonstrate that, among the evaluated models, the ones with explicit
reasoning capabilities achieve higher IG per turn and reach solutions in fewer steps, particularly in partially observable settings. Analysis of
reasoning traces reveals that smaller models compensate for limited capacity through more aggressive exploration of candidate questions, while larger models exhibit higher assertiveness in selecting optimal queries, generating candidates with greater potential IG.
Read the full abstract ▾
Machine Learning
FedCCA: Client-Centric Adaptation against Data Heterogeneity in Federated Learning on IoT Devices
Kaile Wang, Jiannong Cao, Yu Yang et al. (+2 more)
With the rapid development of the Internet of Things (IoT), AI model training on private data such as human sensing data is highly desired.
Federated learning (FL) has emerged as a
privacy-preserving distributed training framework for this purpuse. However, the data heterogeneity issue among IoT devices can significantly degrade the model performance and convergence speed in FL. Existing approaches limit in fixed client selection and aggregation on cloud server, making the
privacy-preserving extraction of client-specific information during local training challenging. To this end, we propose Client-Centric Adaptation
federated learning (FedCCA), an
algorithm that optimally utilizes client-specific knowledge to learn a unique model for each client through selective adaptation, aiming to alleviate the influence of data heterogeneity. Specifically, FedCCA employs dynamic client selection and adaptive aggregation based on the additional client-specific encoder. To enhance multi-source knowledge transfer, we adopt an
attention-based global aggregation strategy. We conducted extensive experiments on diverse
datasets to assess the efficacy of FedCCA. The experimental results demonstrate that our approach exhibits a substantial performance advantage over competing baselines in addressing this specific problem.
Read the full abstract ▾
All authors Kaile Wang, Jiannong Cao, Yu Yang, Xiaoyin Li, Yinfeng Cao
Machine Learning · Cross-listed from cs.CL
Qingsen Ma, Dianyun Wang, Yaoye Wang et al. (+7 more)
Large
language models are increasingly applied to multi-document and long-form inputs, yet long-context inference remains
memory- and noise-inefficient. Key-value (KV) caching scales linearly with context length, while external
retrieval methods often return lexically similar but causally irrelevant passages. We present S3-
Attention, a
memory-first inference-time framework that treats long-context processing as
attention-aligned endogenous
retrieval. S3-
Attention decodes transient key and query projections into top-k sparse feature identifiers using lightweight sparse autoencoders, and constructs a CPU-based inverted index mapping features to
token positions or spans during a single streaming scan. This design allows the KV cache to be discarded entirely and bounds GPU
memory usage by the scan chunk size. At generation time, feature co-activation is used to retrieve compact evidence spans, optionally fused with BM25 for exact lexical matching. Under a unified LongBench evaluation protocol with fixed prompting, decoding, and matched
token budgets, S3-Hybrid closely matches full-context inference across multiple model families and improves robustness in several information-dense settings. We also report an engineering limitation of the current prototype, which incurs higher wall-clock latency than optimized full-KV baselines, motivating future kernel-level optimization.
Read the full abstract ▾
All authors Qingsen Ma, Dianyun Wang, Yaoye Wang, Lechen Ning, Sujie Zhu, Xiaohang Zhang, Jiaming Lyu, Linhao Ren, Zhenbo Xu, Zhaofeng He
Machine Learning · Cross-listed from cs.SD
Segment Length Matters: A Study of Segment Lengths on Audio Fingerprinting Performance
Ziling Gong, Yunyan Ouyang, Iram Kamdar et al. (+5 more)
Audio fingerprinting provides an identifiable representation of acoustic signals, which can be later used for identification and
retrieval systems. To obtain a discriminative representation, the input audio is usually segmented into shorter time intervals, allowing local acoustic features to be extracted and analyzed. Modern neural approaches typically operate on short, fixed-duration audio segments, yet the choice of segment duration is often made heuristically and rarely examined in depth. In this paper, we study how segment length affects audio fingerprinting performance. We extend an existing neural fingerprinting architecture to adopt various segment lengths and evaluate
retrieval accuracy across different segment lengths and query durations. Our results show that short segment lengths (0.5-second) generally achieve better performance. Moreover, we evaluate LLM capacity in recommending the best segment length, which shows that GPT-5-mini consistently gives the best suggestions across five considerations among three studied LLMs. Our findings provide practical guidance for selecting segment duration in large-scale neural audio
retrieval systems.
Read the full abstract ▾
All authors Ziling Gong, Yunyan Ouyang, Iram Kamdar, Melody Ma, Hongjie Chen, Franck Dernoncourt, Ryan A. Rossi, Nesreen K. Ahmed
Machine Learning
REV-INR: Regularized Evidential Implicit Neural Representation for Uncertainty-Aware Volume Visualization
Shanu Saklani, Tushar M. Athawale, Nairita Pal et al. (+3 more)
Applications of Implicit Neural Representations (INRs) have emerged as a promising deep learning approach for compactly representing large volumetric
datasets. These models can act as surrogates for volume data, enabling efficient storage and on-demand reconstruction via model predictions. However, conventional deterministic INRs only provide value predictions without insights into the model's prediction uncertainty or the impact of inherent noisiness in the data. This limitation can lead to unreliable data interpretation and visualization due to prediction inaccuracies in the reconstructed volume. Identifying erroneous results extracted from model-predicted data may be infeasible, as raw data may be unavailable due to its large size. To address this challenge, we introduce REV-INR, Regularized Evidential Implicit Neural Representation, which learns to predict data values accurately along with the associated coordinate-level data uncertainty and model uncertainty using only a single forward pass of the trained REV-INR during inference. By comprehensively comparing and contrasting REV-INR with existing well-established deep uncertainty estimation methods, we show that REV-INR achieves the best volume reconstruction quality with robust data (aleatoric) and model (epistemic) uncertainty estimates using the fastest inference time. Consequently, we demonstrate that REV-INR facilitates assessment of the reliability and trustworthiness of the extracted isosurfaces and volume visualization results, enabling analyses to be solely driven by model-predicted data.
Read the full abstract ▾
All authors Shanu Saklani, Tushar M. Athawale, Nairita Pal, David Pugmire, Christopher R. Johnson, Soumya Dutta
Machine Learning
Agentic reinforcement learning empowers next-generation chemical language models for molecular design and synthesis
Hao Li, He Cao, Shenyao Peng et al. (+7 more)
Language models are revolutionizing the biochemistry domain, assisting scientists in drug design and chemical synthesis with high efficiency. Yet current approaches struggle between small
language models prone to
hallucination and limited knowledge retention, and large cloud-based
language models plagued by
privacy risks and high inference costs. To bridge this gap, we introduce ChemCRAFT, a novel framework leveraging agentic reinforcement learning to decouple chemical
reasoning from knowledge storage. Instead of forcing the model to memorize vast chemical data, our approach empowers the
language model to interact with a sandbox for precise information
retrieval. This externalization of knowledge allows a locally deployable small model to achieve superior performance with minimal inference costs. To enable small
language models for
agent-calling ability, we build an agentic trajectory construction pipeline and a comprehensive chemical-
agent sandbox. Based on sandbox interactions, we constructed ChemToolDataset, the first large-scale chemical tool trajectory
dataset. Simultaneously, we propose SMILES-GRPO to build a dense chemical reward function, promoting the model's ability to call chemical
agents. Evaluations across diverse aspects of drug design show that ChemCRAFT outperforms current cloud-based LLMs in molecular structure analysis, molecular optimization, and synthesis pathway prediction, demonstrating that scientific
reasoning is not solely an emergent ability of model scale, but a learnable policy of tool orchestration. This work establishes a cost-effective and
privacy-preserving paradigm for AI-aided chemistry, opening new avenues for accelerating molecular discovery with locally deployable
agents. Code available at https://github.com/HowardLi1984/ChemCraft.
Read the full abstract ▾
All authors Hao Li, He Cao, Shenyao Peng, Zijing Liu, Bin Feng, Yu Wang, Zhiyuan Yan, Yonghong Tian, Yu Li, Li Yuan