Machine Learning · Cross-listed from cs.CL
Can Embedding Similarity Predict Cross-Lingual Transfer? A Systematic Study on African Languages
Tewodros Kederalah Idris, Prasenjit Mitra and Roald Eiselen
Cross-lingual transfer is essential for building NLP systems for low-resource African
languages, but practitioners lack reliable methods for selecting source
languages. We systematically evaluate five
embedding similarity metrics across 816 transfer experiments spanning three NLP tasks, three African-centric
multilingual models, and 12
languages from four
language families. We find that cosine gap and
retrieval-based metrics (P@1, CSLS) reliably predict transfer success ($\rho = 0.4-0.6$), while CKA shows negligible predictive power ($\rho \approx 0.1$). Critically, correlation signs reverse when pooling across models (Simpson's Paradox), so practitioners must validate per-model.
Embedding metrics achieve comparable predictive power to URIEL linguistic typology. Our results provide concrete guidance for source
language selection and highlight the importance of model-specific analysis.
Read the full abstract ▾
Machine Learning
Dynamic Hyperparameter Importance for Efficient Multi-Objective Optimization
Daphne Theodorakopoulos, Marcel Wever and Marius Lindauer
Choosing a suitable ML model is a complex task that can depend on several objectives, e.g., accuracy,
fairness, or energy consumption. In practice, this requires trading off multiple, often competing, objectives through multi-objective optimization (MOO). However, existing MOO methods typically treat all hyperparameters as equally important, disregarding that hyperparameter importance (HPI) can vary significantly across objectives. We propose a novel dynamic optimization approach that prioritizes the most influential hyperparameters based on varying objective trade-offs during the search, thereby accelerating empirical convergence. We advance prior work on HPI for MOO from post-analysis to direct, dynamic integration within the optimization, using the recent HPI method HyperSHAP. For this, we leverage the objective weightings naturally produced by the MOO
algorithm ParEGO and reduce the configuration space by fixing the unimportant hyperparameters, allowing the search to focus on the important ones. Eventually, we evaluate our method on diverse tasks from PyMOO and YAHPO-Gym. For HPO, integrating HPI yields up to 24% improvement in final Pareto front quality, while on synthetic data, integrating HPI achieves 2x better final results.
Read the full abstract ▾
Machine Learning
On the Convergence Behavior of Preconditioned Gradient Descent Toward the Rich Learning Regime
Shuai Jiang, Alexey Voronin, Eric Cyr et al. (+1 more)
Spectral bias, the tendency of neural networks to learn low frequencies first, can be both a blessing and a curse. While it enhances the generalization capabilities by suppressing high-frequency noise, it can be a limitation in scientific tasks that require capturing fine-scale structures. The delayed generalization phenomenon known as grokking is another barrier to rapid training of neural networks. Grokking has been hypothesized to arise as learning transitions from the NTK to the feature-rich regime. This paper explores the impact of preconditioned gradient descent (PGD), such as Gauss-Newton, on spectral bias and grokking phenomena. We demonstrate through theoretical and empirical results how PGD can mitigate issues associated with spectral bias. Additionally, building on the rich learning regime grokking hypothesis, we study how PGD can be used to reduce delays associated with grokking. Our
conjecture is that PGD, without the impediment of spectral bias, enables uniform exploration of the parameter space in the NTK regime. Our experimental results confirm this prediction, providing strong evidence that grokking represents a transitional behavior between the lazy regime characterized by the NTK and the rich regime. These findings deepen our understanding of the interplay between optimization dynamics, spectral bias, and the phases of neural network learning.
Read the full abstract ▾
All authors Shuai Jiang, Alexey Voronin, Eric Cyr, Ben Southworth
Machine Learning
Rapid Augmentations for Time Series (RATS): A High-Performance Library for Time Series Augmentation
Wadie Skaf, Felix Kern, Aryamaan Basu Roy et al. (+3 more)
Time series augmentation is critical for training robust deep learning models, particularly in domains where labelled data is scarce and expensive to obtain. However, existing augmentation libraries for time series, mainly written in Python, suffer from performance bottlenecks, where running time grows exponentially as
dataset sizes increase – an aspect limiting their applicability in large-scale, production-grade systems. We introduce RATS (Rapid Augmentations for Time Series), a high-performance library for time series augmentation written in Rust with Python bindings (RATSpy). RATS implements multiple augmentation methods spanning basic transformations, frequency-domain operations and time warping techniques, all accessible through a unified pipeline interface with built-in parallelisation. Comprehensive benchmarking of RATSpy versus a commonly used library (tasug) on 143
datasets demonstrates that RATSpy achieves an average speedup of 74.5% over tsaug (up to 94.8% on large
datasets), with up to 47.9% less peak
memory usage.
Read the full abstract ▾
All authors Wadie Skaf, Felix Kern, Aryamaan Basu Roy, Tejas Pradhan, Roman Kalkreuth, Holger Hoos
Machine Learning
Prompt-Counterfactual Explanations for Generative AI System Behavior
Sofie Goethals, Foster Provost and João Sedoc
As generative AI systems become integrated into real-world applications, organizations increasingly need to be able to understand and interpret their behavior. In particular, decision-makers need to understand what causes generative AI systems to exhibit specific output characteristics. Within this general topic, this paper examines a key question: what is it about the input – the
prompt – that causes an LLM-based generative AI system to produce output that exhibits specific characteristics, such as toxicity, negative sentiment, or political bias. To examine this question, we adapt a common technique from the Explainable AI literature: counterfactual explanations. We explain why traditional counterfactual explanations cannot be applied directly to generative AI systems, due to several differences in how generative AI systems function. We then propose a flexible framework that adapts counterfactual explanations to non-deterministic, generative AI systems in scenarios where downstream classifiers can reveal key characteristics of their outputs. Based on this framework, we introduce an
algorithm for generating
prompt-counterfactual explanations (PCEs). Finally, we demonstrate the production of counterfactual explanations for generative AI systems with three case studies, examining different output characteristics (viz., political leaning, toxicity, and sentiment). The case studies further show that PCEs can streamline
prompt engineering to suppress undesirable output characteristics and can enhance red-teaming efforts to uncover additional
prompts that elicit undesirable outputs. Ultimately, this work lays a foundation for
prompt-focused interpretability in generative AI: a capability that will become indispensable as these models are entrusted with higher-stakes tasks and subject to emerging regulatory requirements for transparency and accountability.
Read the full abstract ▾
Machine Learning
PersonaLedger: Generating Realistic Financial Transactions with Persona Conditioned LLMs and Rule Grounded Feedback
Dehao Yuan, Tyler Farnan, Stefan Tesliuc et al. (+8 more)
Strict
privacy regulations limit access to real transaction data, slowing open research in financial AI. Synthetic data can bridge this gap, but existing generators do not jointly achieve behavioral diversity and logical groundedness. Rule-driven simulators rely on hand-crafted workflows and shallow stochasticity, which miss the richness of human behavior. Learning-based generators such as GANs capture correlations yet often violate hard financial constraints and still require training on private data. We introduce PersonaLedger, a generation engine that uses a large
language model conditioned on rich user personas to produce diverse transaction streams, coupled with an expert configurable programmatic engine that maintains correctness. The LLM and engine interact in a closed loop: after each event, the engine updates the user state, enforces financial rules, and returns a context aware "nextprompt" that guides the LLM toward feasible next actions. With this engine, we create a public
dataset of 30 million transactions from 23,000 users and a
benchmark suite with two tasks, illiquidity classification and identity theft
segmentation. PersonaLedger offers a realistic,
privacy preserving resource that supports rigorous evaluation of
forecasting and anomaly detection models. PersonaLedger offers the community a rich, realistic, and
privacy preserving resource – complete with code, rules, and generation logs – to accelerate innovation in financial AI and enable rigorous, reproducible evaluation.
Read the full abstract ▾
All authors Dehao Yuan, Tyler Farnan, Stefan Tesliuc, Doron L Bergman, Yulun Wu, Xiaoyu Liu, Minghui Liu, James Montgomery, Nam H Nguyen, C. Bayan Bruss, Furong Huang
Machine Learning · Cross-listed from eess.SY
Finite Memory Belief Approximation for Optimal Control in Partially Observable Markov Decision Processes
Mintae Kim
We study finite
memory belief approximation for partially observable (PO)
stochastic optimal control (SOC) problems. While belief states are sufficient for SOC in partially observable Markov decision processes (POMDPs), they are generally infinite-dimensional and impractical. We interpret truncated input-output (IO) histories as inducing a belief approximation and develop a metric-based theory that directly relates information loss to control performance. Using the Wasserstein metric, we derive policy-conditional performance bounds that quantify value degradation induced by finite
memory along typical closed-loop trajectories. Our analysis proceeds via a fixed-policy comparison: we evaluate two cost functionals under the same closed-loop execution and isolate the effect of replacing the true belief by its finite
memory approximation inside the belief-level cost. For linear quadratic Gaussian (LQG) systems, we provide closed-form belief mismatch evaluation and empirically validate the predicted mechanism, demonstrating that belief mismatch decays approximately exponentially with
memory length and that the induced performance mismatch scales accordingly. Together, these results provide a metric-aware characterization of what finite
memory belief approximation can and cannot achieve in PO settings.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
LeafLife: An Explainable Deep Learning Framework with Robustness for Grape Leaf Disease Recognition
B. M. Shahria Alam and Md. Nasim Ahmed
Plant disease diagnosis is essential to farmers' management choices because plant diseases frequently lower crop yield and product quality. For harvests to flourish and agricultural productivity to boost, grape leaf disease detection is important. The plant disease
dataset contains grape leaf diseases total of 9,032 images of four classes, among them three classes are leaf diseases, and the other one is healthy leaves. After rigorous pre-processing
dataset was split (70% training, 20% validation, 10% testing), and two pre-trained models were deployed: InceptionV3 and Xception. Xception shows a promising result of 96.23% accuracy, which is remarkable than InceptionV3.
Adversarial Training is used for robustness, along with more transparency. Grad-CAM is integrated to confirm the leaf disease. Finally deployed a web application using Streamlit with a heatmap visualization and prediction with confidence level for robust grape leaf disease classification.
Read the full abstract ▾
Machine Learning · Cross-listed from quant-ph
Gradient descent reliably finds depth- and gate-optimal circuits for generic unitaries
Janani Gomathi and Alex Meiburg
When the gate set has continuous parameters, synthesizing a
unitary operator as a
quantum circuit is, in principle, always possible using exact methods. However, efficiently finding depth- and gate-minimal
circuits remains a major challenge. The landscape is very different for compiled
unitaries, which arise from programming and typically have short
circuits, as compared with generic
unitaries, which use all parameters and typically require
circuits of maximal size. Previous approaches based on random combinatorial search indicate a low success rate even when the
circuit ansatz is nominally adequately parameterized, motivating the use of heavily overparameterized
circuits. In this work, we present a gradient-based optimization framework that enables the synthesis of depth- and gate-optimal
circuits for generic
unitaries without overparameterization, even under restricted hardware connectivity. We prescribe parameter-optimal
circuit skeletons and eliminate the need for random combinatorial search. We further show that the poor performance of earlier random-search approaches can be attributed to the inadvertent selection of parameter-deficient
circuit topologies. By systematically avoiding such skeletons, our approach achieves reliable convergence while maintaining parameter efficiency.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
ToxiGAN: Toxic Data Augmentation via LLM-Guided Directional Adversarial Generation
Peiran Li, Jan Fillies and Adrian Paschke
Augmenting toxic
language data in a controllable and class-specific manner is crucial for improving robustness in toxicity classification, yet remains challenging due to limited supervision and distributional skew. We propose ToxiGAN, a class-aware text augmentation framework that combines
adversarial generation with semantic guidance from large
language models (LLMs). To address common issues in GAN-based augmentation such as mode collapse and semantic drift, ToxiGAN introduces a two-step directional training strategy and leverages LLM-generated neutral texts as semantic ballast. Unlike prior work that treats LLMs as static generators, our approach dynamically selects neutral exemplars to provide balanced guidance. Toxic samples are explicitly optimized to diverge from these exemplars, reinforcing class-specific contrastive signals. Experiments on four hate
speech benchmarks show that ToxiGAN achieves the strongest average performance in both macro-F1 and hate-F1, consistently outperforming traditional and LLM-based augmentation methods. Ablation and sensitivity analyses further confirm the benefits of semantic ballast and directional training in enhancing classifier robustness.
Read the full abstract ▾
Machine Learning
One Sample to Rule Them All: Extreme Data Efficiency in Multidiscipline Reasoning with Reinforcement Learning
Yiyuan Li, Zhen Huang, Yanan Wu et al. (+6 more)
The
reasoning ability of large
language models (LLMs) can be unleashed with reinforcement learning (RL) (OpenAI, 2024; DeepSeek-AI et al., 2025a; Zeng et al., 2025). The success of existing RL attempts in LLMs usually rely on high-quality samples of large volumes. In this paper, we challenge conventional assumptions about data requirements in RL for LLMs by demonstrating the effectiveness of one-shot reinforcement learning. Specifically, we introduce polymath learning, a framework for designing one training sample that elicits multidisciplinary
reasoning improvement. We present three key findings: (1) A single, strategically selected math
reasoning sample can produce significant performance improvements across multiple domains, including physics, chemistry, and biology; (2) Analysis of salient mathematical skills provides insight into the characteristics associated with effective polymath samples; and (3) An engineered synthetic sample that integrates multidisciplinary elements and broader skill coverage achieves stronger performance than naturally occurring individual samples. Across various
reasoning benchmarks, polymath learning achieves stronger performance than larger
datasets, demonstrating that
reasoning structure and skills in samples, rather than quantity, may be the key to unlock enhanced
reasoning capabilities in
language models. Our results suggest a shift, dubbed as sample engineering, toward precision engineering of samples that complements simply increasing data volume.
Read the full abstract ▾
All authors Yiyuan Li, Zhen Huang, Yanan Wu, Weixun Wang, Xuefeng Li, Yijia Luo, Wenbo Su, Bo Zheng, Pengfei Liu
Machine Learning
Time-Aware Synthetic Control
Saeyoung Rho, Cyrus Illick, Samhitha Narasipura et al. (+3 more)
The synthetic control (SC) framework is widely used for observational causal inference with time-series panel data. SC has been successful in diverse applications, but existing methods typically treat the ordering of pre-intervention time indices interchangeable. This invariance means they may not fully take advantage of temporal structure when strong trends are present. We propose Time-Aware Synthetic Control (TASC), which employs a state-space model with a constant trend while preserving a low-rank structure of the signal. TASC uses the Kalman filter and Rauch-Tung-Striebel smoother: it first fits a generative time-series model with expectation-maximization and then performs counterfactual inference. We evaluate TASC on both simulated and real-world
datasets, including policy evaluation and sports prediction. Our results suggest that TASC offers advantages in settings with strong temporal trends and high levels of observation noise.
Read the full abstract ▾
All authors Saeyoung Rho, Cyrus Illick, Samhitha Narasipura, Alberto Abadie, Daniel Hsu, Vishal Misra
Machine Learning
From Muscle to Text with MyoText: sEMG to Text via Finger Classification and Transformer-Based Decoding
Meghna Roy Chowdhury, Shreyas Sen and Yi Ding
Surface electromyography (sEMG) provides a direct neural interface for decoding muscle activity and offers a promising foundation for keyboard-free text input in wearable and mixed-reality systems. Previous sEMG-to-text studies mainly focused on recognizing letters directly from sEMG signals, forming an important first step toward translating muscle activity into text. Building on this foundation, we present MyoText, a hierarchical framework that decodes sEMG signals to text through physiologically grounded intermediate stages. MyoText first classifies finger activations from multichannel sEMG using a CNN-BiLSTM-
Attention model, applies ergonomic typing priors to infer letters, and reconstructs full sentences with a fine-tuned T5
transformer. This modular design mirrors the natural hierarchy of typing, linking muscle intent to
language output and reducing the search space for decoding. Evaluated on 30 users from the emg2qwerty
dataset, MyoText outperforms baselines by achieving 85.4% finger-classification accuracy, 5.4% character error rate (CER), and 6.5% word error rate (WER). Beyond accuracy gains, this methodology establishes a principled pathway from neuromuscular signals to text, providing a blueprint for virtual and augmented-reality typing interfaces that operate entirely without physical keyboards. By integrating ergonomic structure with
transformer-based linguistic
reasoning, MyoText advances the feasibility of seamless, wearable neural input for future ubiquitous computing environments.
Read the full abstract ▾
Machine Learning
ATLAS: Verifier-Guided Adaptive Latent Activation Steering for Efficient LLM Reasoning
Tuc Nguyen and Thai Le
Recent work on activation and latent steering has demonstrated that modifying internal representations can effectively guide large
language models (LLMs) toward improved
reasoning and efficiency without updating model parameters. However, most existing approaches rely on fixed steering policies and static intervention strengths, which limit their robustness across problem instances and often result in over- or under-steering. We propose Adaptive Test-time Latent Steering (ATLAS), a lightweight framework that dynamically controls steering decisions at inference time using a trained, lightweight verifier over the latent states. Given intermediate hidden states, the verifier predicts the quality of ongoing
reasoning and adaptively selects which steering action to apply, enabling per-example and per-step adjustment with minimal overhead. ATLAS provides a unified framework for combining learned latent verification with test-time activation steering, enabling adaptive
reasoning control without additional LLM decoding or inference-time process reward model calls. Experiments on multiple mathematical and coding
reasoning benchmarks show that ATLAS consistently outperforms both vanilla decoding and fixed steering baselines, achieving higher accuracy while substantially reducing test-time
token usage. These results demonstrate that verifier-guided latent adaptation provides an effective and scalable mechanism for controlling
reasoning efficiency without sacrificing solution quality. All source code will be publicly available.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Faithfulness Evaluation for Decoder-only LLM Attributions with Controlled Retained Information
Xin Huang and Antoni B. Chan
Large
Language Models (LLMs) are increasingly evaluated with input attribution methods, yet comparing such explanations remains challenging. Existing soft-perturbation faithfulness metrics, such as Soft-NC and Soft-NS, can conflate attribution quality with the number of words retained during perturbation: attribution methods with larger average scores may keep more words and therefore obtain inflated scores. To address this issue, we propose $\pi$-Soft-NC and $\pi$-Soft-NS, an evaluation framework that compares attribution methods under the same expected retaining probability, thus controlling the number of retained words. We further introduce Grad-ELLM, a gradient-based attribution method tailored to autoregressive decoder-only LLMs, which combines gradient-derived channel importance with
attention-derived
token importance at each decoding step. Experiments on classification and open-generation tasks with Llama and Mistral show that Grad-ELLM achieves strong comprehensiveness-oriented faithfulness under $\pi$-Soft-NC, while there is no dominant method under $\pi$-Soft-NS. Our evaluation metric serves as a rigorous framework to compare XAI methods for LLMs, which will support progress in the field.
Read the full abstract ▾
Machine Learning
Audit Me If You Can: Query-Efficient Active Fairness Auditing of Black-Box LLMs
David Hartmann, Lena Pohlmann, Lelia Hanslik et al. (+3 more)
Large
Language Models (LLMs) exhibit systematic biases across demographic groups.
Auditing is proposed as an accountability tool for black-box LLM applications, but suffers from resource-intensive query access. We conceptualise
auditing as uncertainty estimation over a target
fairness metric and introduce BAFA, the Bounded Active
Fairness Auditor for query-efficient
auditing of black-box LLMs. BAFA maintains a version space of surrogate models consistent with queried scores and computes uncertainty intervals for
fairness metrics (e.g., $\Delta$ AUC) via constrained empirical risk minimisation. Active query selection narrows these intervals to reduce estimation error. We evaluate BAFA on two standard
fairness dataset case studies: CivilComments and Bias-in-Bios, comparing against stratified sampling, power sampling, and ablations. BAFA achieves target error thresholds with up to 40$\times$ fewer queries than stratified sampling (e.g., 144 vs 5,956 queries at $\varepsilon=0.02$ for CivilComments) for tight thresholds, demonstrates substantially better performance over time, and shows lower variance across runs. These results suggest that active sampling can reduce resources needed for independent
fairness auditing with LLMs, supporting continuous model evaluations.
Read the full abstract ▾
All authors David Hartmann, Lena Pohlmann, Lelia Hanslik, Noah Gießing, Bettina Berendt, Pieter Delobelle
Machine Learning
Real-Time Adaptive Anomaly Detection in Industrial IoT Environments
Mahsa Raeiszadeh, Amin Ebrahimzadeh, Roch H. Glitho et al. (+2 more)
To ensure reliability and service availability, next-generation networks are expected to rely on automated anomaly detection systems powered by advanced machine learning methods with the capability of handling multi-dimensional data. Such multi-dimensional, heterogeneous data occurs mostly in today's industrial Internet of Things (IIoT), where real-time detection of anomalies is critical to prevent impending failures and resolve them in a timely manner. However, existing anomaly detection methods often fall short of effectively coping with the complexity and dynamism of multi-dimensional data streams in IIoT. In this paper, we propose an adaptive method for detecting anomalies in IIoT streaming data utilizing a multi-source prediction model and concept drift adaptation. The proposed anomaly detection
algorithm merges a prediction model into a novel drift adaptation method resulting in accurate and efficient anomaly detection that exhibits improved scalability. Our trace-driven evaluations indicate that the proposed method outperforms the state-of-the-art anomaly detection methods by achieving up to an 89.71% accuracy (in terms of Area under the Curve (AUC)) while meeting the given efficiency and scalability requirements.
Read the full abstract ▾
All authors Mahsa Raeiszadeh, Amin Ebrahimzadeh, Roch H. Glitho, Johan Eker, Raquel A. F. Mini
Machine Learning
Joint Encoding of KV-Cache Blocks for Scalable LLM Serving
Joseph Kampeas and Emir Haleva
Modern large
language models (LLMs) drive interactive AI systems but are bottlenecked by the
memory-heavy growth of key-value (KV) caches, which limits real-time throughput under concurrent loads. Existing KV-cache compression methods rely on rigid heuristics, disrupt tensor layouts, or require specialized compute, hindering scalability and deployment. We propose joint encoding of KV-cache blocks, which fuses similar blocks across requests and input chunks into shared representations while preserving standard cache structure. This alleviates the KV-cache
memory bottleneck, supporting high-concurrency serving without specialized hardware. Theoretically, we analyze the rate-distortion tradeoff of fused cache blocks under a Poisson process model. Empirically, our method achieves up to 4.38 $\times$ KV-cache compression with negligible accuracy loss across diverse LLMs and
benchmarks, outperforming recent structured and adaptive compression baselines. In real LLM serving, joint encoding improves the
token throughput by $\sim$40% on a single-machine vLLM
benchmark, demonstrating substantial gains in inference throughput. Code is available at https://github.com/sef1/kv_fast_fusion kv_joint_encoding.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Do LLMs Encode Functional Importance of Reasoning Tokens?
Janvijay Singh and Dilek Hakkani-Tür
Large
language models solve complex tasks by generating long
reasoning chains, achieving higher accuracy at the cost of increased computational cost and reduced ability to isolate functionally relevant
reasoning. Prior work on compact
reasoning shortens such chains through probabilistic sampling, heuristics, or supervision from frontier models, but offers limited insight into whether models internally encode
token-level functional importance for answer generation. We address this gap diagnostically and propose greedy
pruning, a likelihood-preserving deletion procedure that iteratively removes
reasoning tokens whose removal minimally degrades model likelihood under a specified objective, yielding length-controlled
reasoning chains. We evaluate pruned
reasoning in a
distillation framework and show that students trained on pruned chains outperform a frontier-model-supervised compression baseline at matched
reasoning lengths. Finally, our analysis reveals systematic
pruning patterns and shows that
attention scores can predict greedy
pruning ranks, further suggesting that models encode a nontrivial functional importance structure over
reasoning tokens.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
Explainable Fuzzy GNNs for Leak Detection in Water Distribution Networks
Qusai Khaled, Pasquale De Marinis, Moez Louati et al. (+3 more)
Timely leak detection in water distribution networks is critical for conserving resources and maintaining operational efficiency. Although Graph Neural Networks (GNNs) excel at capturing spatial-temporal dependencies in sensor data, their black-box nature and the limited work on graph-based explainable models for water networks hinder practical adoption. We propose an explainable GNN framework that integrates mutual information to identify critical network regions and fuzzy logic to provide clear, rule-based explanations for node classification tasks. After benchmarking several GNN architectures, we selected the generalized graph convolution network (GENConv) for its superior performance and developed a fuzzy-enhanced variant that offers intuitive explanations for classified leak locations. Our fuzzy graph neural network (FGENConv) achieved Graph F1 scores of 0.889 for detection and 0.814 for localization, slightly below the crisp GENConv 0.938 and 0.858, respectively. Yet it compensates by providing spatially localized, fuzzy rule-based explanations. By striking the right balance between precision and explainability, the proposed fuzzy network could enable hydraulic engineers to validate predicted leak locations, conserve human resources, and optimize maintenance strategies. The code is available at github.com/pasqualedem/GNNLeakDetection.
Read the full abstract ▾
All authors Qusai Khaled, Pasquale De Marinis, Moez Louati, David Ferras, Laura Genga, Uzay Kaymak
Machine Learning · Cross-listed from cs.CL
Temporal Graph Network: Hallucination Detection in Multi-Turn Conversation
Vidhi Rathore, Sambu Aneesh and Himanshu Singh
Hallucinations can be produced by conversational AI systems, particularly in multi-turn conversations where context changes and contradictions may eventually surface. By representing the entire conversation as a temporal graph, we present a novel graph-based method for detecting dialogue-level
hallucinations. Our framework models each dialogue as a node, encoding it using a sentence
transformer. We explore two different ways of connectivity: i) shared-entity edges, which connect turns that refer to the same entities; ii) temporal edges, which connect contiguous turns in the conversation. Message-passing is used to update the node
embeddings, allowing flow of information between related nodes. The context-aware node
embeddings are then combined using
attention pooling into a single vector, which is then passed on to a classifier to determine the presence and type of
hallucinations. We demonstrate that our method offers slightly improved performance over existing methods. Further, we show the
attention mechanism can be used to justify the decision making process. The code and model weights are made available at: https://github.com/sambuaneesh/anlp-project.
Read the full abstract ▾
Machine Learning
When the Coffee Feature Activates on Coffins: An Analysis of Feature Extraction and Steering for Mechanistic Interpretability
Raphael Ronge, Markus Maier and Frederick Eberhardt
Recent work by Anthropic on Mechanistic interpretability claims to understand and control Large
Language Models by extracting human-interpretable features from their neural activation patterns using sparse autoencoders (SAEs). If successful, this approach offers one of the most promising routes for human oversight in AI safety. We conduct an initial stress-test of these claims by replicating their main results with open-source SAEs for Llama 3.1. While we successfully reproduce basic feature extraction and steering capabilities, our investigation suggests that major caution is warranted regarding the generalizability of these claims. We find that feature steering exhibits substantial fragility, with sensitivity to layer selection, steering magnitude, and context. We observe non-standard activation behavior and demonstrate the difficulty to distinguish thematically similar features from one another. While SAE-based interpretability produces compelling demonstrations in selected cases, current methods often fall short of the systematic reliability required for safety-critical applications. This suggests a necessary shift in focus from prioritizing interpretability of internal representations toward reliable prediction and control of model output. Our work contributes to a more nuanced understanding of what mechanistic interpretability has achieved and highlights fundamental challenges for AI safety that remain unresolved.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Lil: Less is Less When Applying Post-Training Sparse-Attention Algorithms in Long-Decode Stage
Junhao Hu, Fangze Li, Mingtao Xu et al. (+9 more)
Large
language models (LLMs) demonstrate strong capabilities across a wide range of complex tasks and are increasingly deployed at scale, placing significant demands on inference efficiency. Prior work typically decomposes inference into prefill and decode stages, with the decode stage dominating total latency. To reduce time and
memory complexity in the decode stage, a line of work introduces sparse-
attention algorithms. In this paper, we show, both empirically and theoretically, that sparse
attention can paradoxically increase end-to-end complexity: information loss often induces significantly longer sequences, a phenomenon we term “Less is Less” (Lil). To mitigate the Lil problem, we propose an early-stopping
algorithm that detects the threshold where information loss exceeds information gain during sparse decoding. Our early-stopping
algorithm reduces
token consumption by up to 90% with a marginal accuracy degradation of less than 2% across
reasoning-intensive
benchmarks.
Read the full abstract ▾
All authors Junhao Hu, Fangze Li, Mingtao Xu, Feifan Meng, Shiju Zhao, Tiancheng Hu, Ting Peng, Anmin Liu, Wenrui Huang, Chenxu Liu, Ziyue Hua, Tao Xie
Machine Learning · Cross-listed from cs.RO
PiDR: Physics-Informed Inertial Dead Reckoning for Autonomous Platforms
Arup Kumar Sahoo and Itzik Klein
A fundamental requirement for full autonomy is the ability to sustain accurate
navigation in the absence of external data, such as GNSS signals or visual information. In these challenging environments, the platform must rely exclusively on inertial sensors, leading to pure inertial
navigation. However, the inherent noise and other error terms of the inertial sensors in such real-world scenarios will cause the
navigation solution to drift over time. Although conventional deep-learning models have emerged as a possible approach to inertial
navigation, they are inherently black-box in nature. Furthermore, they struggle to learn effectively with limited supervised sensor data and often fail to preserve physical principles. To address these limitations, we propose PiDR, a physics-informed inertial dead-reckoning framework for autonomous platforms in situations of pure inertial
navigation. PiDR offers transparency by explicitly integrating inertial
navigation principles into the network training process through the physics-informed residual component. PiDR plays a crucial role in mitigating abrupt trajectory deviations even under limited or sparse supervision. We evaluated PiDR on real-world
datasets collected by a mobile
robot and an autonomous
underwater vehicle. We obtained more than 29% positioning improvement in both
datasets, demonstrating the ability of PiDR to generalize different platforms operating in various environments and dynamics. Thus, PiDR offers a robust, lightweight, yet effective architecture and can be deployed on resource-constrained platforms, enabling real-time pure inertial
navigation in adverse scenarios.
Read the full abstract ▾
Machine Learning
Causal Manifold Fairness: Enforcing Geometric Invariance in Representation Learning
Vidhi Rathore
Fairness in machine learning is increasingly critical, yet standard approaches often treat data as static points in a high-dimensional space, ignoring the underlying generative structure. We posit that sensitive attributes (e.g., race, gender) do not merely shift data distributions but causally warp the geometry of the data
manifold itself. To address this, we introduce Causal
Manifold Fairness (CMF), a novel framework that bridges causal inference and
geometric deep learning. CMF learns a latent representation where the local Riemannian geometry, defined by the metric tensor and
curvature, remains invariant under counterfactual interventions on sensitive attributes. By enforcing constraints on the Jacobian and Hessian of the decoder, CMF ensures that the rules of the latent space (distances and shapes) are preserved across demographic groups. We validate CMF on synthetic Structural Causal Models (SCMs), demonstrating that it effectively disentangles sensitive
geometric warping while preserving task utility, offering a rigorous quantification of the
fairness-utility trade-off via
geometric metrics.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Flow Matching and Diffusion Models via PointNet for Generating Fluid Fields on Irregular Geometries
Ali Kashefi
We present two novel generative
geometric deep learning frameworks, termed Flow Matching PointNet and
Diffusion PointNet, for predicting fluid flow variables on irregular geometries by incorporating PointNet into flow matching and
diffusion models, respectively. In these frameworks, a reverse generative process reconstructs physical fields from standard Gaussian noise conditioned on unseen geometries. The proposed approaches operate directly on point-cloud representations of computational domains (e.g., grid vertices of finite-volume meshes) and therefore avoid the limitations of pixelation used to project geometries onto uniform
lattices, as is common in U-Net-based flow matching and
diffusion models. In contrast to graph neural network-based
diffusion models, Flow Matching PointNet and
Diffusion PointNet do not exhibit high-frequency noise artifacts in the predicted fields. Moreover, unlike such approaches, which require auxiliary intermediate networks to condition geometry, the proposed frameworks rely solely on PointNet, resulting in a simple and unified architecture. The performance of the proposed frameworks is evaluated on steady incompressible flow past a cylinder, using a
geometric dataset constructed by varying the cylinder's cross-sectional shape and orientation across samples. The results demonstrate that Flow Matching PointNet and
Diffusion PointNet achieve more accurate predictions of velocity and pressure fields, as well as lift and drag forces, and exhibit greater robustness to incomplete geometries compared to a vanilla PointNet with the same number of trainable parameters.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Dementia-R1: Reinforced Pretraining and Reasoning from Unstructured Clinical Notes for Real-World Dementia Prognosis
Choonghan Kim, Hyunmin Hwang, Hangeol Chang et al. (+4 more)
While Large
Language Models (LLMs) have shown strong performance on clinical text understanding, they struggle with longitudinal prediction tasks such as dementia prognosis, which require
reasoning over complex, non-monotonic symptom trajectories across multiple visits. Standard supervised training lacks explicit annotations for symptom
evolution, while direct Reinforcement Learning (RL) is hindered by sparse binary rewards. To address this challenge, we introduce Dementia-R1, an RL-based framework for longitudinal dementia prognosis from unstructured clinical notes. Our approach adopts a Cold-Start RL strategy that pre-trains the model to predict verifiable clinical indices extracted from patient histories, enhancing the capability to reason about disease progression before determining the final clinical status. Extensive experiments show that Dementia-R1 achieves the best overall performance on the AMC real-world unstructured cohort, reaching an AUROC of 84.02% and outperforming models up to 10x larger. The framework also generalizes to Parkinson's disease dementia prediction in an independent hospital cohort, achieving an AUROC of 78.37%. On the ADNI
benchmark, our 7B model attains the highest AUROC among all LLM baselines at 83.17%, demonstrating strong longitudinal
reasoning over fluctuating cognitive trajectories. Code is available at https://anonymous.4open.science/r/dementiar1-CDB5.
Read the full abstract ▾
All authors Choonghan Kim, Hyunmin Hwang, Hangeol Chang, Jaemin Kim, Jinse Park, Jae-Sung Lim, Jong Chul Ye
Machine Learning
In-Context Reinforcement Learning through Bayesian Fusion of Context and Value Prior
Anaïs Berkes, Vincent Taboga, Donna Vakalis et al. (+2 more)
In-context reinforcement learning (ICRL) promises fast adaptation to unseen environments without parameter updates, but current methods either cannot improve beyond the training distribution or require near-optimal data, limiting practical adoption. We introduce SPICE, a
Bayesian ICRL method that learns a prior over Q-values via deep ensemble and updates this prior at test-time using in-context information through
Bayesian updates. To recover from poor priors resulting from training on sub-optimal data, our online inference follows an Upper-Confidence Bound rule that favours exploration and adaptation. We prove that SPICE achieves regret-optimal behaviour in both
stochastic bandits and finite-horizon MDPs, even when pretrained only on suboptimal trajectories. We validate these findings empirically across bandit and control
benchmarks. SPICE achieves near-optimal decisions on unseen tasks, substantially reduces regret compared to prior ICRL and meta-RL approaches while rapidly adapting to unseen tasks and remaining robust under distribution shift.
Read the full abstract ▾
All authors Anaïs Berkes, Vincent Taboga, Donna Vakalis, David Rolnick, Yoshua Bengio
Machine Learning
Multi-Distribution Robust Conformal Prediction
Yuqi Yang and Ying Jin
In many
fairness and distribution robustness problems, one has access to labeled data from multiple source distributions yet the test data may come from an arbitrary member or a mixture of them. We study the problem of constructing a conformal prediction set that is uniformly valid across multiple, heterogeneous distributions, in the sense that no matter which distribution the test point is from, the coverage of the prediction set is guaranteed to exceed a pre-specified level. We first propose a max-p aggregation scheme that delivers finite-sample, multi-distribution coverage given any conformity scores associated with each distribution. Upon studying several efficiency optimization programs subject to uniform coverage, we prove the optimality and tightness of our aggregation scheme, and propose a general
algorithm to learn conformity scores that lead to efficient prediction sets after the aggregation under standard conditions. We discuss how our framework relates to group-wise distributionally robust optimization, sub-population shift,
fairness, and multi-source learning. In synthetic and real-data experiments, our method delivers valid worst-case coverage across multiple distributions while greatly reducing the set size compared with naively applying max-p aggregation to single-source conformity scores, and can be comparable in size to single-source prediction sets with popular, standard conformity scores.
Read the full abstract ▾
Machine Learning
From Memorization to Creativity: LLM as a Designer of Novel Neural Architectures
Waleed Khalid, Dmitry Ignatov and Radu Timofte
Large
language models (LLMs) excel in program synthesis, yet their capacity for neural architecture design – balancing syntactic reliability, performance, and structural novelty – remains underexplored. We present a closed-loop architecture synthesis pipeline within the NNGPT framework, in which a code-oriented LLM evolves over 22 supervised fine-tuning cycles. At each cycle, the LLM synthesizes PyTorch convolutional networks, validated via low-fidelity performance signals and filtered via a MinHash–Jaccard criterion to prevent structural redundancy before being incorporated into the LEMUR
dataset. High-performing candidates with novel architectures are converted into
prompt–code pairs for parameter-efficient LoRA fine-tuning. This feedback loop drives a measurable distributional shift, progressively internalizing empirical architectural priors such that valid and high-performing outputs evolve from scarce to dominant across cycles. On CIFAR-10, the valid generation rate stabilizes at 50.6% (peaking at 74.5%), mean first-epoch accuracy rises from 28.1% to 51.0%, and candidates exceeding 40% accuracy grow from 2.0% to 96.8%. Cross-
dataset transfer to CIFAR-100 and SVHN confirms that improved validity, shifted accuracy distributions, and sustained novelty generalize across
benchmarks of varying difficulty and visual domain. Across 22 cycles, 455 unique architectures absent from the original corpus are admitted under the novelty filter. By grounding synthesis in execution feedback and novelty filtering, we demonstrate that iterative self-supervised fine-tuning reshapes an LLM into a task-specialized architectural prior – improving generation reliability, proxy performance, and structural diversity – offering a reproducible, annotation-free alternative to hand-crafted search spaces.
Read the full abstract ▾