Machine Learning
Avionic Main Fuel Pump Simulation and Fault-Diagnosis Benchmark
Felix Leonhard Janzen, Lukas Moddemann, Alexander Diedrich et al. (+1 more)
In many cyber-physical systems, especially in critical applications such as aeroplanes, data to train anomaly detection and diagnosis
algorithms is lacking due to data protection issues and partial observability. To combat this inherent lack of data, we introduce a high-fidelity, physics-informed co-simulation of a common aircraft main-fuel-pump system modelled in MATLAB/Simulink Simscape Fluids. We also describe its generated time-series data with health and fault mode annotations. To show feasibility of our
benchmark, we apply an unsupervised Recurrent Variational Autoencoder (RNN-VAE) for anomaly detection and a SOM-VAE for operating mode discretization, trained to separate healthy and faulty conditions.
Read the full abstract ▾
All authors Felix Leonhard Janzen, Lukas Moddemann, Alexander Diedrich, Oliver Niggemann
Machine Learning · Cross-listed from cs.IR
IntrAgent: An LLM Agent for Content-Grounded Information Retrieval through Literature Review
Fengbo Ma, Zixin Rao, Xiaoting Li et al. (+5 more)
Scientific research relies on accurate information
retrieval from literature to support analytical decisions. In this work, we introduce a new task, INformation
reTRieval through literAture reVIEW (IntraView), which aims to automate fine-grained information
retrieval faithfully grounded in the provided content in response to research-driven queries, and propose IntrAgent, an LLM-based
agent that addresses this challenging task. In particular, IntrAgent is designed to mimic human behaviors when reading literature for information
retrieval – identifying relevant sections and then iteratively extracting key details to refine the retrieved information. It follows a two-stage pipeline: a Section Ranking stage that prioritizes relevant literature sections through structural-knowledge-enabled
reasoning, and an Iterative Reading stage that continuously extracts details and synthesizes them into concise, contextually grounded answers. To support rigorous evaluation, we introduce IntraBench, a new
benchmark consisting of 315 test instances built from expert-authored questions paired with literature spanning five STEM domains. Across seven backbone LLMs, IntrAgent achieves on average 13.2% higher cross-domain accuracy than state-of-the-art RAG and research-
agent baselines.
Read the full abstract ▾
All authors Fengbo Ma, Zixin Rao, Xiaoting Li, Zhetao Chen, Hongyue Sun, Yiping Zhao, Xianyan Chen, Zhen Xiang
Machine Learning
PermaFrost-Attack: Stealth Pretraining Seeding(SPS) for planting Logic Landmines During LLM Training
Harsh Kumar, Rahul Maity, Tanmay Joshi et al. (+4 more)
Aligned large
language models (LLMs) remain vulnerable to
adversarial manipulation, and their reliance on web-scale pretraining creates a subtle but consequential attack surface. We study Stealth Pretraining Seeding (SPS), a threat model in which adversaries distribute small amounts of poisoned content across stealth websites, increasing the likelihood that such material is absorbed into future training corpora derived from sources such as Common Crawl. Because each individual payload is tiny, diffuse, and superficially benign, the attack is difficult to detect during
dataset construction or filtering. The result is a latent form of poisoning that remains largely invisible under standard evaluation, yet can later be activated by a precise trigger such as <00TRIGGER00>. We call this attack PermaFrost, reflecting its latent and reactivatable nature. We study it through PermaFrost-Attack, a controlled framework for latent conceptual poisoning, together with three
geometric diagnostics: Thermodynamic Length, Spectral
Curvature, and the Infection Traceback Graph. Across multiple model families and scales, we show that this controlled SPS proxy can induce persistent unsafe behavior that often remains hidden under standard evaluation. Our results identify SPS as a practical and underappreciated threat to future foundation models. This paper introduces a novel
geometric diagnostic lens for systematically examining latent model behavior, providing a principled foundation for detecting, characterizing, and understanding vulnerabilities that may remain invisible under standard evaluation.
Read the full abstract ▾
All authors Harsh Kumar, Rahul Maity, Tanmay Joshi, Aman Chadha, Vinija Jain, Suranjana Trivedy, Amitava Das
Machine Learning
Do Not Imitate, Reinforce: Iterative Classification via Belief Refinement
Mahdi Kallel, Johannes Tölle, Ahmed Hendawy et al. (+1 more)
Standard supervised classification trains models to imitate the exact labels provided by a perfect oracle. This imitation happens in a single pass, restricting the model to a fixed compute budget even when inputs vary in complexity. Moreover, the rigid training objective forces the model to express absolute certainty on its training data, resulting in overconfident predictions during evaluation. We propose Reinforced Iterative Classification (RIC), which replaces the imitative objective with Reinforcement Learning (RL). RIC deploys a recurrent
agent that iteratively updates a predictive distribution over classes, receiving reward for stepwise improvement in prediction quality. The value function provides a natural halting criterion by estimating the remaining scope for improvement. We prove that the iterative formulation recovers the same optimal predictions as cross-
entropy while yielding an anytime classifier. On image classification
benchmarks, RIC matches the accuracy of supervised baselines with improved calibration and learns to allocate computation adaptively across inputs.
Read the full abstract ▾
All authors Mahdi Kallel, Johannes Tölle, Ahmed Hendawy, Carlo D'Eramo
Machine Learning · Cross-listed from cs.RO
Wiggle and Go! System Identification for Zero-Shot Dynamic Rope Manipulation
Arthur Jakobsson, Abhinav Mahajan, Karthik Pullalarevu et al. (+6 more)
Many robotic tasks are unforgiving; a single mistake in a dynamic throw can lead to unacceptable delays or unrecoverable failure. We introduce Wiggle and Go!, a two-stage framework for zero-shot rope manipulation: a brief, safe wiggle action is observed to predict descriptive rope parameters, which then conditions a trajectory optimizer for zero-shot goal-conditioned execution. Unlike prior dynamic rope manipulation methods that require large real-world
datasets or iterative real-world refinement, our identification module is task-agnostic, supporting diverse manipulation policies without retraining. We achieve a 3.55\,cm average accuracy on 3D target striking in real using rope system parameters in comparison to 15.29\,cm for uninformed baselines, and over 50% success on multi-objective lobbing and draping tasks. Predicted parameters transfer to unseen motions with 0.95 Pearson correlation between simulated and real rope dynamics, indicating that the identification module generalizes across the task corpus. Project website: https://wiggleandgo.github.io/
Read the full abstract ▾
All authors Arthur Jakobsson, Abhinav Mahajan, Karthik Pullalarevu, Krishna Suresh, Yunchao Yao, Yuemin Mao, Bardienus Duisterhof, Shahram Najam Syed, Jeffrey Ichnowski
Machine Learning
Assessing the impact of dimensionality reduction on clustering performance - a systematic study
Ousmane Assani Amate, Elyes Lounissi, Mohammadreza Bakhtyari et al. (+3 more)
Dimensionality reduction is a critical preprocessing step for clustering high-dimensional data, yet comprehensive evaluation of its impact across diverse methods and data types remains limited. In this study, we systematically assess the influence of five dimensionality reduction techniques - Principal Component Analysis (PCA), Kernel Principal Component Analysis (Kernel PCA), Variational Autoencoder (VAE), Isometric Mapping (Isomap), and Multidimensional Scaling (MDS) - on the performance of four popular clustering
algorithms - k-means, Agglomerative Hierarchical Clustering (AHC), Gaussian Mixture Models (GMM), and Ordering Points to Identify the Clustering Structure (OPTICS). We evaluate clustering quality using the Adjusted Rand Index (ARI), comparing results without and with dimensionality reduction at different reduction levels recommended in the literature (i.e., k-1, where k is the number of clusters, and 25% and 50% of the original number of dimensions). Our findings underscore the importance of a careful selection of the dimensionality reduction technique and the dimensionality reduction level that should be tailored to intrinsic data geometry and clustering
algorithm(s) under consideration.
Read the full abstract ▾
All authors Ousmane Assani Amate, Elyes Lounissi, Mohammadreza Bakhtyari, Émilie Roy, Roman Sarrazin-Gendron, Vladimir Makarenkov
Machine Learning · Cross-listed from cs.CR
Who Audits the Auditor? Tamper-Proof Fraud Detection with Blockchain-Anchored Explainable ML
Zhaohui Wang
In enterprise fraud detection, model accuracy alone is insufficient when insiders can tamper with audit logs or bypass approval workflows. Real-world incidents show that fraud often persists not because detection
algorithms fail, but because the audit trail itself is controllable by privileged operators. This exposes a fundamental trust gap: *who audits the auditor?* We present a tamper-evident fraud detection system that anchors both ML predictions and workflow execution to an immutable blockchain ledger. Rather than using blockchain as passive storage, we enforce the entire approval process through smart contracts, ensuring that every transaction, prediction, and explanation is atomically recorded and cannot be retroactively modified. Our detection module achieves competitive accuracy (F1 = 0.895, PR-AUC = 0.974) while providing cryptographically verifiable decision trails that support regulatory auditability requirements (e.g., GDPR Article 22). System evaluation shows sub-25 ms inference latency and economically viable deployment on Layer-2 networks at under \$0.01 per transaction (validated against PolygonScan data), supporting enterprise-scale workloads of 10,000+ monthly payments.
Read the full abstract ▾
Machine Learning
Generating Synthetic Malware Samples Using Generative AI
Tiffany Bao, Kylie Trousil, Quang Duy Tran et al. (+2 more)
Malware attacks have a significant negative impact on organizations of varied scales in the field of cybersecurity. Recently, malware researchers have increasingly turned to machine learning techniques to combat sophisticated obfuscation methods used in malware. However, collecting a diverse set of malware samples with various obfuscation techniques is challenging and often takes years, especially for newly developed malware. This issue is further compounded by a well-known limitation of machine learning models: their poor performance when training data is scarce. In this paper, we propose a new system for generating synthetic malware samples to augment imbalanced malware
dataset. Our approach decomposes malware binary samples into mnemonic opcode sequences, leveraging natural
language processing to extract contextual meaning behind malware opcode features to aid the learning of generative AI (GenAI) employed in this paper, Generative
Adversarial Networks (GAN), Wasserstein Generative
Adversarial Networks with Gradient Penalty (WGAN-GP), and a modified
Diffusion model. The experiment results show that augmenting training data with
Diffusion-based synthetic data significantly improves classification performance for minor classes by up to 60% on average. This enhancement ultimately leads to an overall malware classification performance of 96%, an 8% improvement. These findings demonstrate the high quality and fidelity of the synthetic data, its robustness, and its potential applications in malware analysis. Specifically, synthetic malware data proves effective in improving the classification of minor malware classes and detection rates, even though the size of known malware data is significantly small.
Read the full abstract ▾
All authors Tiffany Bao, Kylie Trousil, Quang Duy Tran, Fabio Di Troia, Younghee Park
Machine Learning
Removing Sandbagging in LLMs by Training with Weak Supervision
Emil Ryd, Henning Bartsch, Julian Stastny et al. (+2 more)
As AI systems begin to automate complex tasks, supervision increasingly relies on weaker models or limited human oversight that cannot fully verify output quality. A model more capable than its supervisors could exploit this gap through sandbagging, producing work that appears acceptable but falls short of its true abilities. Can training elicit a model's best work even without reliable verification? We study this using model organisms trained to sandbag, testing elicitation techniques on problem-solving math, graduate-level science, and competitive coding tasks. We find that training with weak supervision can reliably elicit sandbagging models when supervised fine-tuning (SFT) and reinforcement learning (RL) are combined: SFT on weak demonstrations breaks the sandbagging behavior, enabling RL to then fully elicit performance. Neither method succeeds reliably alone-RL without SFT almost always leads to reward hacking rather than genuine improvement, and SFT without RL fails to elicit full performance when the supervisor is much weaker than the untrusted model. Critically, this relies on training being indistinguishable from deployment; when models can distinguish between training and deployment, they can perform well during training while continuing to sandbag afterward. Our results provide initial evidence that training is a viable mitigation against sandbagging, while highlighting the importance of making training indistinguishable from deployment.
Read the full abstract ▾
All authors Emil Ryd, Henning Bartsch, Julian Stastny, Joe Benton, Vivek Hebbar
Machine Learning
Insect-inspired modular architectures as inductive biases for reinforcement learning
Anne E. Staples
Most reinforcement-learning (RL) controllers used in continuous control are architecturally centralized: observations are compressed into a single latent state from which both value estimates and actions are produced. Biological control systems are often organized differently. Insects, in particular, coordinate
navigation, heading stabilization,
memory, and context-dependent action selection through distributed
circuits rather than a single monolithic controller. Motivated by this contrast, we study an RL policy architecture that decomposes control into interacting modules for sensory encoding, heading representation, sparse associative
memory, recurrent command generation, and local motor control, with a learned arbitration mechanism that allocates motor authority across modules. The model is evaluated on a two-dimensional
navigation task that require simultaneous food seeking, obstacle avoidance, and predator escape. In a six-seed predator-
navigation experiment trained with Proximal Policy Optimization (PPO) for 75 updates, the modular policy achieves the strongest final mean performance among the tested controllers, with final episodic return $-2798.8\pm964.4$ versus $-3778.0\pm628.1$ for a centralized gated recurrent unit (GRU) and $-4727.5\pm772.5$ for a centralized multilayer perceptron (MLP). The modular policy also attains the lowest final value loss and stable PPO optimization statistics while driving module-assignment
entropy to $0.0457\pm0.0244$, indicating highly selective control allocation. These results suggest that distributed control can serve as a useful inductive bias for RL problems involving dynamically competing behavioral objectives.
Read the full abstract ▾
Machine Learning
PrivUn: Unveiling Latent Ripple Effects and Shallow Forgetting in Privacy Unlearning
Xiaoyi Chen, Haoyuan Wang, Siyuan Tang et al. (+4 more)
Large
language models (LLMs) often memorize private information during training, raising serious
privacy concerns. While machine unlearning has emerged as a promising solution, its true effectiveness against
privacy attacks remains unclear. To address this, we propose PrivUn, a new evaluation framework that systematically assesses unlearning robustness through three-tier attack scenarios: direct
retrieval, in-context learning recovery, and fine-tuning restoration; combined with quantitative analysis using forgetting scores, association metrics, and forgetting depth assessment. Our study exposes significant weaknesses in current unlearning methods, revealing two key findings: 1) unlearning exhibits gradient-driven ripple effects: unlike traditional forgetting which follows semantic relations (e.g., knowledge graphs),
privacy unlearning propagates across latent gradient-based associations; and 2) most methods suffer from shallow forgetting, failing to remove private information distributed across multiple deep model layers. To validate these insights, we explore two strategies: association-aware core-set selection that leverages gradient similarity, and multi-layer deep intervention through representational constraints. These strategies represent a paradigm shift from shallow forgetting to deep forgetting.
Read the full abstract ▾
All authors Xiaoyi Chen, Haoyuan Wang, Siyuan Tang, Sijia Liu, Liya Su, XiaoFeng Wang, Haixu Tang
Machine Learning
Reliability Auditing for Downstream LLM tasks in Psychiatry: LLM-Generated Hospitalization Risk Scores
Shevya Panda, Shinjini Bose and Ananya Joshi
Large
language models (LLMs) are increasingly utilized in clinical
reasoning and risk assessment. However, their interpretive reliability in critical and indeterminate domains such as psychiatry remains unclear. Prior work has identified algorithmic biases and
prompt sensitivity in these systems, raising concerns about how contextual information may influence model outputs, but there remains no systematic way to assess these, especially in the psychiatric domain. We propose an approach for reliability
auditing downstream LLM tasks by structuring evaluation around the impact of
prompt design and the inclusion of medically insignificant inputs on predicted hospitalization risk scores, which is often the first downstream AI clinical-decision-making task. In our audit, a cohort of synthetic patient profiles (n = 50) is generated, each consisting of 15 clinically relevant features and up to 50 clinically insignificant features, across four
prompt reframings (neutral, logical, human impact, clinical judgment). We audit four LLMs (Gemini 2.5 Flash, LLaMa 3.3 70b, Claude Sonnet 4.6, GPT-4o mini), and our results show that including medically insignificant variables resulted in a statistically significant increase in the absolute mean predicted hospitalization risk and output variability across all models and
prompts, indicating reduced predictive stability as contextual noise increased. Clinically insignificant features had an effect on instability across many model-
prompt conditions, and
prompt variations independently affected the trajectory of instability in a model-dependent manner. These findings quantify how LLM-based psychiatric risk assessments are sensitive to non-clinical information, highlighting the need for systematic evaluations of attributional stability and uncertainty behavior like this before clinical deployments.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Lightweight Retrieval-Augmented Generation and Large Language Model-Based Modeling for Scalable Patient-Trial Matching
Xiaodi Li, Yang Xiao, Munhwan Lee et al. (+7 more)
Patient-trial matching requires
reasoning over long, heterogeneous electronic health records (EHRs) and complex eligibility criteria, posing significant challenges for scalability, generalization, and computational efficiency. Existing approaches either rely on full-document processing with large
language models (LLMs), which is computationally expensive, or use traditional machine learning methods that struggle to capture unstructured clinical narratives. In this work, we propose a lightweight framework that combines
retrieval-augmented generation and large
language model-based modeling for scalable patient-trial matching. The framework explicitly separates two key components:
retrieval-augmented generation is used to identify clinically relevant segments from long EHRs, reducing input complexity, while large
language models are used to encode these selected segments into informative representations. These representations are further refined through dimensionality reduction and modeled using lightweight predictors, enabling efficient and scalable downstream classification. We evaluate the proposed approach on multiple public
benchmarks (n2c2, SIGIR, TREC 2021/2022) and a real-world
multimodal dataset from Mayo Clinic (MCPMD). Results show that
retrieval-based information selection significantly reduces computational burden while preserving clinically meaningful signals. We further demonstrate that frozen LLMs provide strong representations for structured clinical data, whereas fine-tuning is essential for modeling unstructured clinical narratives. Importantly, the proposed lightweight pipeline achieves performance comparable to end-to-end LLM approaches with substantially lower computational cost.
Read the full abstract ▾
All authors Xiaodi Li, Yang Xiao, Munhwan Lee, Konstantinos Leventakos, Young J. Juhn, David Jones, Terence T. Sio, Wei Liu, Maria Vassilaki, Nansu Zong
Machine Learning
Learning Coverage- and Power-Optimal Transmitter Placement from City Maps: A Comparative Study of Direct and Indirect Neural Approaches
Çağkan Yapar
Optimal wireless transmitter placement is a central task in
radio-network planning, and exhaustive search becomes prohibitively expensive at scale. This paper studies the single-transmitter setting under a learned propagation model, enabling exhaustive per-pixel assessment at scale in a regime where measurement-based labeling is infeasible and ray-tracing-based labeling is computationally out of reach. We introduce a
dataset of 167525 urban scenarios (RadioMapSeer-Deployment) with dual ground-truth labels for coverage-optimal and power-optimal transmitter locations.
Benchmark analysis reveals an asymmetric coverage-power trade-off: coverage-optimal placement sacrifices 13.86% of received-power, whereas power-optimal placement sacrifices 5.50% of coverage; the best balanced placement lies at $\bar{d}=2.60$ from the ideal point (100%,100%). We evaluate two learning formulations: indirect heatmap-based models predicting received-power
radio maps, and direct score-map models predicting the objective landscape over feasible transmitter locations. Within the heatmap family, discriminative models deliver one-shot predictions 1350-2400$\times$ faster than exhaustive search, while
diffusion models additionally support multi-sample inference that improves single-objective performance and, by reusing the same sample pool under a balanced criterion, recovers strong balanced placements without explicit multi-objective training. Dual score-map strategies combining power and coverage score-maps match the exhaustive balanced optimum ($\bar{d}=2.60$) and remain close to it across smaller candidate budgets, at 14-22$\times$ speedups including the cost of evaluating shortlisted candidates. Dual score-map methods are strongest overall, whereas heatmap formulations remain attractive for their physically meaningful intermediate maps and, in the
diffusion setting, for inference-time search.
Read the full abstract ▾
Machine Learning
LayerBoost: Layer-Aware Attention Reduction for Efficient LLMs
Mohamed Ali Souibgui, Jan Fostier, Rodrigo Abadía-Heredia et al. (+3 more)
Transformers are mostly relying on softmax
attention, which introduces quadratic complexity with respect to sequence length and remains a major bottleneck for efficient inference. Prior work on linear or hybrid
attention typically replaces softmax
attention uniformly across all layers, often leading to significant performance degradation or requiring extensive retraining to recover model quality. This work proposes LayerBoost, a layer-aware
attention reduction method that selectively modifies the
attention mechanism based on the sensitivity of individual
transformer layers. It first performs a systematic sensitivity analysis on a pretrained model to identify layers that are critical for maintaining performance. Guided by this analysis, three distinct strategies can be applied: retaining standard softmax
attention in highly sensitive layers, replacing it with linear sliding window
attention in moderately sensitive layers, and removing
attention entirely in layers that exhibit low sensitivity. To recover performance after these architectural modifications, we introduce a lightweight
distillation-based healing phase requiring only 10M additional training
tokens. LayerBoost reduces inference latency and improves throughput by up to 68% at high concurrency, while maintaining competitive model quality. It matches base model performance on several
benchmarks, exhibits only minor degradations on others, and significantly outperforms state-of-the-art
attention linearization methods. These efficiency gains make our method particularly well-suited for high-concurrency serving and hardware-constrained deployment scenarios, where inference cost and
memory footprint are critical bottlenecks.
Read the full abstract ▾
All authors Mohamed Ali Souibgui, Jan Fostier, Rodrigo Abadía-Heredia, Bohdan Denysenko, Christian Marschke, Igor Peric
Machine Learning · Cross-listed from physics.soc-ph
Audio Video Verbal Analysis (AVVA) for Capturing Classroom Dialogues
Vivek Upadhyay and Amaresh Chakrabarti
Background: The classroom discourse analysis has been transformed by the growing use of audio-
video multimodal data, which demands analytical methods that balance interpretive depth with computational scalability. Methods: This study introduces the Audio
Video Verbal Analysis (AVVA) framework, adapted from the Verbal Analysis method to integrate qualitative interpretation with quantitative modelling. Unlike fully
multimodal learning analytics approaches, AVVA focuses on verbatim transcripts with essential interactional modalities. Findings: The framework embeds triangulation as a core design strategy across ten methodological steps, strengthening validity and analytical rigour. A comprehensive validation scheme addresses fundamental challenges in temporal observational research: Phi Ceiling for low-frequency variables (via Base Rate Filtering), estimation uncertainty (via bootstrap confidence intervals), and the Modifiable Temporal Unit Problem, where measured associations depend on observational window size. Four-criterion stability assessment (sign consistency, confidence interval overlap, zero exclusion, magnitude stability) classifies variable pairs into interpretable patterns: grain-invariant, scale-specific, or multi-scale, etc. structures across temporal grain sizes. Its application to 23 hours of classroom recordings illustrates its practical viability and its potential to yield meaningful insights. Contribution: The framework thus provides a scalable pathway for transforming rich classroom discourse into analysable
datasets.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Brian VanVoorst, Nicholas Walczak, Christopher Gilleo et al. (+9 more)
This paper introduces EgoMAGIC (Medical Assistance, Guidance, Instruction, and Correction), an
egocentric medical activity
dataset collected as part of DARPA's Perceptually-enabled Task Guidance (PTG) program. This
dataset comprises 3,355
videos of 50 medical tasks, with at least 50 labeled
videos per task. The primary objective of the PTG program was to develop virtual assistants integrated into augmented reality headsets to assist users in performing complex tasks. To encourage exploration and research using this
dataset, the medical training data has been released along with an action detection challenge focused on eight medical tasks. The majority of the
videos were recorded using a head-mounted stereo camera with integrated audio. From this
dataset, 40 YOLO models were trained using 1.95 million labels to detect 124 medical objects, providing a robust starting point for developers working on medical AI applications. In addition to introducing the
dataset, this paper presents baseline results on action detection for the eight selected medical tasks across three models, with the best-performing method achieving average mAP 0.526. Although this paper primarily addresses action detection as the
benchmark, the EgoMAGIC
dataset is equally suitable for action recognition, object identification and detection, error detection, and other challenging computer
vision tasks. The
dataset is accessible via zenodo.org (DOI: 10.5281/zenodo.19239154).
Read the full abstract ▾
All authors Brian VanVoorst, Nicholas Walczak, Christopher Gilleo, Charles Meissner, Fabio Felix, Iran Roman, Bea Steers, Claudio Silva, Yuhan Shen, Zijia Lu, Shih-Po Lee, Ehsan Elhamifar
Machine Learning
LTBs-KAN: Linear-Time B-splines Kolmogorov-Arnold Networks
Eduardo Said Merin-Martinez, Andres Mendez-Vazquez and Eduardo Rodriguez-Tello
Kolmogorov-Arnold Networks (KANs) are a recent neural network architecture offering an alternative to Multilayer Perceptrons (MLPs) with improved explainability and expressibility. However, KANs are significantly slower than MLPs due to the recursive nature of B-spline function computations, limiting their application. This work addresses these issues by proposing a novel base-spline Linear-Time B-splines Kolmogorov-Arnold Network (LTBs-KAN) with linear complexity. Unlike previous methods that rely on the Boor-Mansfield-Cox spline
algorithm or other computationally intensive mathematical functions, our approach significantly reduces the computational burden. Additionally, we further reduce model's parameter through product-of-sums
matrix factorization in the forward pass without sacrificing performance. Experiments on MNIST, Fashion-MNIST and CIFAR-10 demonstrate that LTBs-KAN achieves good time complexity and parameter reduction, when used as building architectural blocks, compared to other KAN implementations.
Read the full abstract ▾
Machine Learning
Kernel Contracts: A Specification Language for ML Kernel Correctness Across Heterogeneous Silicon
Cooper Veit
Every ML kernel ships with an implicit contract about what it computes. People rarely write the contract down. When two kernels disagree – when a matmul on AMD produces a different gradient than the same matmul on NVIDIA, when a fused
attention kernel silently downcasts an accumulator, when an out-of-bounds access returns zero on one stack and garbage on another – there is no formal artifact to arbitrate the dispute. Recent empirical work has measured the gap across silicon platforms, but none of it specifies the contract being violated. We present a specification
language for kernel contracts. A contract has eight parts: identifier, scope, precondition, postcondition, tolerance, reference oracle, measurement protocol, and violation signature. We use it to state twelve contract classes covering precision, ordering, compiler-induced, and exceptional-value failure modes, each grounded in published empirical evidence. We require a three-state calibration: every contract must admit at least one reference-conforming implementation and at least one contract-violating implementation that passes basic functional tests. We apply the framework to three documented incidents – Huawei Ascend silent precision coercion, Sakana AI CUDA Engineer reward hacking, AMD out-of-bounds silent acceptance – and show that each informal diagnosis maps to a specific contract violation with a measurable signature. A kernel contract suite is a normative reference against which conformance can be graded, in the way that ISASecure grades industrial control systems against IEC 62443.
Read the full abstract ▾
Machine Learning
Mochi: Aligning Pre-training and Inference for Efficient Graph Foundation Models via Meta-Learning
João Mattos and Arlei Silva
We propose Mochi, a Graph Foundation Model that addresses task unification and training efficiency by adopting a meta-learning based training framework. Prior models pre-train with reconstruction-based objectives such as link prediction, and assume that the resulting representations can be aligned with downstream tasks through a separate unification step such as class prototypes. We demonstrate through synthetic and real-world experiments that this procedure, while simple and intuitive, has limitations that directly affect downstream task performance. To address these limitations, Mochi pre-trains on few-shot episodes that mirror the downstream evaluation protocol, aligning the training objective with inference rather than relying on a post-hoc unification step. We show that Mochi, along with its more powerful variant Mochi++, achieves competitive or superior performance compared to existing Graph Foundation Models across 25 real-world graph
datasets spanning node classification, link prediction, and graph classification, while requiring 8$\sim$27 times less training time than the strongest baseline.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Shared Lexical Task Representations Explain Behavioral Variability In LLMs
Zhuonan Yang, Jacob Xiaochen Li, Francisco Piedrahita Velez et al. (+5 more)
One of the most common complaints about large
language models (LLMs) is their
prompt sensitivity – that is, the fact that their ability to perform a task or provide a correct answer to a question can depend unpredictably on the way the question is posed. We investigate this variation by comparing two very different but commonly-used styles of prompting: instruction-based
prompts, which describe the task in natural
language, and example-based
prompts, which provide in-context few-shot demonstration pairs to illustrate the task. We find that, despite large variation in performance as a function of the
prompt, the model engages some common underlying mechanisms across different
prompts of a task. Specifically, we identify task-specific
attention heads whose outputs literally describe the task – which we dub lexical task heads – and show that these heads are shared across prompting styles and trigger subsequent answer production. We further find that behavioral variation between
prompts can be explained by the degree to which these heads are activated, and that failures are at least sometimes due to competing task representations that dilute the signal of the target task. Our results together present an increasingly clear picture of how LLMs' internal representations can explain behavior that otherwise seems idiosyncratic to users and developers.
Read the full abstract ▾
All authors Zhuonan Yang, Jacob Xiaochen Li, Francisco Piedrahita Velez, Eric Todd, David Bau, Michael L. Littman, Stephen H. Bach, Ellie Pavlick
Machine Learning · Cross-listed from q-bio.NC
Foundation models for discovering robust biomarkers of neurological disorders from dynamic functional connectivity
Deepank Girish, Yi Hao Chan, Sukrit Gupta et al. (+2 more)
Several
brain foundation models (FM) have recently been proposed to predict
brain disorders by modelling dynamic functional connectivity (FC). While they demonstrate remarkable model performance and zero- or few-shot generalization, the salient features identified as potential biomarkers are yet to be thoroughly evaluated. We propose RE-CONFIRM, a framework for evaluating the robustness of potential biomarker candidates elucidated by deep learning (DL) models including FMs. From experiments on five large
datasets of Autism Spectrum Disorder (ASD),
Attention-deficit Hyperactivity Disorder (ADHD), and Alzheimer's Disease (AD), we found that although commonly used performance metrics provide an intuitive assessment of model predictions, they are insufficient for evaluating the robustness of biomarkers identified by these models. RE-CONFIRM metrics revealed that simply finetuning FMs leads to models that fail to capture regional hubs effectively, even in disorders where hubs are known to be implicated, such as ASD and ADHD. In view of this, we propose Hub-LoRA (Low-Rank Adaptation) as a fine-tuning technique that enables FMs to not only outperform customised DL models but also produce neurobiologically faithful biomarkers supported by meta-analyses. RE-CONFIRM is generalizable and can be easily applied to ascertain the robustness of DL models trained on functional MRI
datasets. Code is available at: https://github.com/SCSE-Biomedical-Computing-Group/RE-CONFIRM.
Read the full abstract ▾
All authors Deepank Girish, Yi Hao Chan, Sukrit Gupta, Jing Xia, Jagath C. Rajapakse
Machine Learning · Cross-listed from cs.IT
Null-Space Flow Matching for MIMO Channel Estimation in Latency-Constrained Systems
Junjie Zhao, Guangming Liang, Xiaonan Liu et al. (+1 more)
Accurate yet low-latency channel state information (CSI) acquisition is essential for multiple-input multiple-output (
MIMO) communication systems. While advanced deep generative models, such as score-based and
diffusion models, enable high-fidelity CSI reconstruction from limited pilot observations, they often suffer from high inference latency. To achieve accurate CSI estimation under stringent latency constraints, this paper proposes a null-space flow matching (FM) framework that leverages a range-null space decomposition to separate observation-informed and underdetermined channel components. Specifically, the pilot observations are used to regulate the observable range-space channel component, while an FM-based generative prior primarily resolves the ambiguous null-space degrees of freedom through iterative refinement. To further improve the robustness and efficiency of the proposed framework, we introduce a noise-aware adaptive correction strategy to suppress channel noise on the refinement trajectory, along with a power-law time schedule to better allocate the limited number of refinement steps. Experimental results demonstrate that our method achieves competitive normalized mean square error (NMSE) performance even under a strict latency budget of around 3 ms, while delivering a superior accuracy-latency tradeoff compared with both model-based and generative baselines.
Read the full abstract ▾
All authors Junjie Zhao, Guangming Liang, Xiaonan Liu, Dongzhu Liu
Machine Learning
Universal Transformers Need Memory: Depth-State Trade-offs in Adaptive Recursive Reasoning
Grigory Sapunov
We study learned
memory tokens as a computational scratchpad for a single-block Universal
Transformer with Adaptive Computation Time (ACT) on Sudoku-Extreme, a combinatorial
reasoning benchmark.
Memory tokens are empirically necessary: no configuration without them reaches non-trivial performance. The optimal count has a sharp lower threshold (T=0 always fails, T=8 reliably succeeds) followed by a stable plateau (T=8-32, 57.4% +/- 0.7% exact-match) and a dilution boundary at T=64. Under halt-side pressure (lambda warmup), mean halt drops monotonically with
memory size across the plateau (from 11.6 at T=8 to 8.3 at T=64), showing that
memory tokens and ponder depth substitute as resources at fixed accuracy. We also identify a router initialization trap that causes the majority of training runs to fail: both default zero-bias and Graves' recommended positive bias settle into a shallow halt
equilibrium the model cannot escape. Inverting the bias to -3 ("deep start") eliminates the failure mode, and ablation shows the trap is inherent to ACT initialization rather than an artifact of our architecture. With reliable training, ACT yields an order of magnitude lower seed variance than fixed-depth processing (+/-0.7 vs +/-9.3 pp); lambda warmup recovers 34% of compute at matched accuracy; and
attention heads specialize into
memory readers, constraint propagators, and integrators across recursive depth. Code: https://github.com/che-shr-cat/utm-jax.
Read the full abstract ▾
Machine Learning
When Quotes Crumble: Detecting Transient Mechanical Liquidity Erosion in Limit Order Books
Haohan Xu, Jason Bohne, Pawel Polak et al. (+5 more)
We study the detection of transient liquidity erosion ("crumbling quotes") in electronic limit order books, where observable quote deterioration may reflect either mechanical liquidity withdrawal or informational repricing. Using the ABIDES
agent-based simulator, we construct a multi-
agent environment in which crumbling emerges from
stochastic regime switches in a
market maker, providing time-resolved ground truth unavailable in real
market data. We develop a detection pipeline that identifies mechanically driven quote erosion using order book features, and train a neural model to produce calibrated crumbling probabilities. Experiments demonstrate that the proposed framework reliably identifies crumbling events against
agent-level ground truth, with the neural model achieving +36% AUC improvement over rule-based baselines and robust performance across normal, high-
volatility, bull, and bear
market conditions. Ablation studies on temporal features and varying the dependence structure of the ground-truth mechanism confirm that the framework generalizes across both independent and autocorrelated liquidity withdrawal dynamics.
Read the full abstract ▾
All authors Haohan Xu, Jason Bohne, Pawel Polak, Yurij Baransky, Ajay Alva, Violetta Fedotova, Gary Kazantsev, David Rosenberg
Machine Learning
Multi-Task Optimization over Networks of Tasks
Julian Hatzky, Thomas Bartz-Beielstein, A. E. Eiben et al. (+1 more)
Multi-task optimization is a powerful approach for solving a large number of tasks in parallel. However, existing
algorithms face distinct limitations: Population-based methods scale poorly and remain underexplored for large task sets. Approaches that do scale beyond a thousand tasks are mostly MAP-Elites variants and rely on a fixed, discretized archive that disregards the
topology of the task space. We introduce MONET (Multi-Task Optimization over Networks of Tasks), a multi-task optimization
algorithm that models the task space as a graph: tasks are nodes, and edges connect tasks in the task parameter space. This representation enables knowledge transfer between tasks and remains tractable for high-dimensional problems while exploiting the
topology of the task space. MONET combines social learning, which generates candidates from neighboring nodes via crossover, with individual learning, which refines a node's own solution independently via mutation. We evaluate MONET on four domains (archery, arm, and cartpole with 5,000 tasks each; hexapod with 2,000 tasks) and show that it matches or exceeds the performance of existing MAP-Elites-based baselines across all four domains.
Read the full abstract ▾
All authors Julian Hatzky, Thomas Bartz-Beielstein, A. E. Eiben, Anil Yaman
Machine Learning · Cross-listed from eess.IV
Conditional Diffusion Posterior Alignment for Sparse-View CT Reconstruction
Luis Barba, Johannes Kirschner and Benjamin Bejar
Computed
Tomography (CT) is a widely used imaging modality in medical and industrial applications. To limit radiation exposure and measurement time, there is a growing interest in sparse-view CT, where the number of projection views is significantly reduced. Deep neural networks have shown great promise in improving reconstruction quality in sparse-view CT, especially generative
diffusion models. However, these methods struggle to scale to large 3D volumes due to several reasons: (i) the high
memory and computational requirements of 3D models, (ii) the lack of large 3D training
datasets, and (iii) the inconsistencies across slices when using 2D models independently on each slice. We overcome these limitations and scale
diffusion-based sparse-view CT reconstruction to large 3D volumes by combining conditional
diffusion with explicit data consistency. We propose Conditional
Diffusion Posterior
Alignment (CDPA) to enable scalable 3D sparse-view CT reconstruction. A 2D U-Net
diffusion model is conditioned on an initial 3D reconstruction to improve inter-slice consistency, combined with data-consistency
alignment to match measured projections. Experiments on synthetic and real Cone Beam CT (CBCT) data show state-of-the-art performance, with ablations that confirm the synergistic effects of the proposed pipeline. Finally, we show that the same principles also strengthen fast denoising U-Nets, yielding near-
diffusion quality at a fraction of the computational cost.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.IT
MambaCSP: Hybrid-Attention State Space Models for Hardware-Efficient Channel State Prediction
Aladin Djuhera, Haris Gacanin and Holger Boche
Recent works have demonstrated that
attention-based
transformer and large
language model (LLM) architectures can achieve strong channel state prediction (CSP) performance by capturing long-range temporal dependencies across channel state information (CSI) sequences. However, these models suffer from quadratic scaling in sequence length, leading to substantial computational cost,
memory consumption, and inference latency, which limits their applicability in real-time and resource-constrained wireless deployments. In this paper, we investigate whether selective state space models (SSMs) can serve as a hardware-efficient alternative for CSI prediction. We propose MambaCSP, a hybrid-
attention SSM architecture that replaces LLM-based prediction backbones with a linear-time Mamba model. To overcome the local-only dependencies of pure SSMs, we introduce lightweight patch-mixer
attention layers that periodically inject cross-
token attentions, helping with long-context CSI prediction. Extensive MISO-OFDM simulations show that MambaCSP improves prediction accuracy over LLM-based approaches by 9-12%, while delivering up to 3.0x higher throughput, 2.6x lower VRAM usage, and 2.9x faster inference. Our results demonstrate that hybrid state space architectures provide a promising direction for scalable and hardware-efficient AI-native CSI prediction in future wireless networks.
Read the full abstract ▾
Machine Learning
Conditional anomaly detection using soft harmonic functions: An application to clinical alerting
Michal Valko, Hamed Valizadegan, Branislav Kveton et al. (+2 more)
Timely detection of concerning events is an important problem in clinical practice. In this paper, we consider the problem of conditional anomaly detection that aims to identify data instances with an unusual response, such as the omission of an important lab test. We develop a new non-parametric approach for conditional anomaly detection based on the soft harmonic solution, with which we estimate the confidence of the label to detect anomalous mislabeling. We further regularize the solution to avoid the detection of isolated examples and examples on the boundary of the distribution support. We demonstrate the efficacy of the proposed method in detecting unusual labels on a real-world electronic health record
dataset and compare it to several baseline approaches.
Read the full abstract ▾
All authors Michal Valko, Hamed Valizadegan, Branislav Kveton, Gregory F. Cooper, Milos Hauskrecht
Machine Learning
Performance Anomaly Detection in Athletics: A Benchmarking System with Visual Analytics
Blessed Madukoma and Prasenjit Mitra
Anti-doping programs rely on biological testing to detect performance-enhancing drugs, but such testing costs over $800 per sample and is limited by short detection windows for many prohibited substances. These constraints leave large portions of athletes without regular testing, motivating complementary screening approaches that analyze routine competition results to identify suspicious performance patterns. We present a system that processes 1.6 million athletics performances from over 19,000 competitions (2010-2025) using eight detection methods ranging from statistical rules to machine learning and trajectory analysis. We validate all methods against publicly confirmed anti-doping violations to measure their effectiveness in identifying sanctioned athletes. Trajectory-based methods, which compare performances to expected career progression, achieve the best balance between detecting violations and limiting false alarms, though all methods face challenges from incomplete data and rare confirmed violations. The system provides an interactive interface for expert-driven investigation, emphasizing transparency and human judgment to support, rather than replace, established anti-doping processes.
Read the full abstract ▾