Machine Learning · Cross-listed from cs.CV
Consensus Clustering of Free-Viewing Gaze Data: New Insights into Human-Information Interaction
Beryl Gnanaraj, Jaya Sreevalsan-Nair, Saqib Alam Ansari et al. (+1 more)
Free-viewing gaze data provides a rich, task-free window into human visual
attention. Conventional exploratory data analysis of the data provides user
attention patterns through fixations and areas of interest. However, despite the richness of this gaze data, its human-information interaction (HII) patterns are understudied. We address this gap using consensus clustering of gaze data with respect to users and stimulus characteristics. We present a novel end-to-end unsupervised ensemble learning system for consensus clustering of free-viewing gaze
datasets, EnsembleGaze. With a goal of characterizing the user behavior and stimulus type, we propose a feature engineering step based on statistical descriptors of fixation-based distributions. EnsembleGaze involves consensus voting of selected clustering methods implemented on the feature vector to compute the co-association
matrix. Using the separate consensus clustering of users and stimuli as a baseline, we further propose two high-dimensional clustering strategies for determining gaze clusters based on joint user and image characterization. They are consensus subspace clustering and spectral biclustering. Clustering performance is evaluated using selected standard metrics and is further interpreted through image-level properties. Our system provides a replicable method for the unsupervised analysis of fixation behavior in scene perception research. Our results show that image stimuli groupings are highly consistent across methods, reflecting a robust ambient-versus-focal viewing mode distinction, whereas user groupings are image-context-dependent, a structure that only biclustering and the two-step conditional approaches are architecturally capable of recovering. Testing on the publicly available
datasets revealed
dataset-specific patterns, with each offering complementary insights through distinct clustering strategies.
Read the full abstract ▾
All authors Beryl Gnanaraj, Jaya Sreevalsan-Nair, Saqib Alam Ansari, Maanasa Rajaraman
Machine Learning
T3R: Deeper Test-Time Adaptation for Graph Neural Networks via Gradient Rotation
Huy Truong, Alexander Lazovik and Victoria Degeler
Graph Neural Networks (GNNs) deployed in real-world systems typically have fixed weights, often leading to degraded performance under distribution shifts. This issue can be mitigated by conventional fine-tuning, but in many real-world cases, collecting labeled data is expensive or infeasible. A potential approach is Test-Time Training (TTT), which adapts models' weights using unlabeled test data, yet it is typically limited to shallow updates that affect only a subset of model parameters. We propose T3R, leveraging multiple Rotograd
matrices to improve task affinity between the target and auxiliary tasks, essential for effective test-time training. T3R further introduces a rotation technique that reorients self-supervised signals using these
matrices to create surrogate gradients for the target task, allowing deeper adaptation across nearly the entire architecture. Empirically, T3R reduces MAE by 0.172 points over standard inference in regression
datasets and achieves at least 9.37% relative improvement on cross-domain OGB classification
benchmarks compared to models without adaptation. These results highlight the potential to develop an adaptation pipeline for graph-based systems, particularly in settings where conventional fine-tuning or retraining is infeasible.
Read the full abstract ▾
Machine Learning
Stabilizing Extrapolation in Looped Transformers via Learned Stochastic Stopping
Hsun-Yu Kuo, El Mahdi Chayti, Patrik Reizinger et al. (+2 more)
Looped
Transformers, which repeatedly apply a shared
transformer block, are an architecturally natural fit for variable-length algorithmic tasks. Although they can exhibit strong length generalization beyond the length of training sequences, this behavior is brittle, yielding high out-of-distribution (OOD) variance, even across well-performing in-distribution solutions. We trace this variance to the spurious correlation in simple algorithmic tasks between sequence length and number of loops. Introducing stochasticity into the number of loops during training sharply reduces OOD variance and stabilizes predictions across inference-time loop counts. To improve upon heuristic randomization schemes, we further analyze RL-Halting as a learned
stochastic schedule and find that it generally improves the accuracy-stability trade-off. Across binary addition, Dyck-1, Unique Set, and Copy, learned
stochastic stopping often improves this trade-off but can also stabilize a suboptimal computation. Our work suggests that "when to stop" should be treated as a training-time design choice, not merely an inference-time computation-allocation rule.
Read the full abstract ▾
All authors Hsun-Yu Kuo, El Mahdi Chayti, Patrik Reizinger, Wieland Brendel, Martin Jaggi
Machine Learning · Cross-listed from cs.AI
Exploration and Online Transfer with Behavioral Foundation Models
Louis Bagot, Mathieu Lefort and Laëtitia Matignon
Zero-shot Transfer in Reinforcement Learning (RL) aims to train an
agent that can generate optimal policies for any reward function, without additional learning at transfer time, while training only on reward-free trajectories. For their generality over tasks, such models are sometimes called “Behavioral Foundation Models” (BFMs). While they have shown strong performances and improvements in recent years, the current framework and
algorithms still assume that, during the transfer phase, the
agent is informed offline about the reward (the task to solve) through a
dataset of state-reward pairs, which it uses to pick the best policy to deploy. However, in practice if the reward is a black-box (e.g. direct user feedback), it is not possible to generate such a
dataset: it is necessary to observe the reward through interactions with the environment. In other words, the current framework of offline transfer is not aligned with the traditional RL setting of online learning through trial-and-error, which requires exploration in order to find rewards. This paper proposes to tackle this new online transfer in zero-shot RL, with the key insight that the BFM itself can be used to generate exploration policies. We show that it is possible to frame this online learning problem in terms of a bandit-like exploration-exploitation problem. More precisely, at each step the bandit
algorithm recommends a policy, the BFM executes it in the environment, which yields a reward and a new state; we repeat the process until we converge to the optimal policy. In the popular context of linear reward approximation, we derive a formulation inspired by Upper Confidence Bound and show that exploration can be achieved through the minimization of the
eigenvalues of an uncertainty
matrix. We evaluate qualitatively and quantitatively our framework on a simple environment to validate the concept of our method.
Read the full abstract ▾
Machine Learning · Cross-listed from eess.IV
A multi-architecture study of specificity refinement and false-positive mechanism analysis in prostate MRI
Yongbo Shu, Kewen Chen, Yifeng Yuan et al. (+5 more)
Objectives: To characterize residual false positives in prostate MRI detection, and to evaluate a lightweight post-hoc refinement head for case-level specificity. Materials and Methods: This retrospective study used PI-CAI (5-fold cross-validation) and Prostate158 (n=158; external). A context-aware evidence head and an 89,216-parameter refinement head were trained on a frozen detection backbone; the evidence head was also trained on four further backbones (bare nnU-Net, bare U-Net, bare Mamba, MIGF-Mamba). For each false-positive region, T2-weighted, apparent-
diffusion-coefficient, and high-b-value contrast ratios versus peri-lesional rings were compared against ground-truth lesions and contralateral benign regions. Results: False positives were closer to true cancers than to benign tissue in evidence and raw T2-weighted and apparent-
diffusion-coefficient contrast, reproducing 35/35 across five architectures (Cohen's d 1.10; FP/benign evidence ratio 2.38x) and 105/105 across modality-perturbation scenarios. On PI-CAI fold-0, refinement raised case-level specificity from 0.469 to 0.549 (+17.2%) at preserved sensitivity (0.943); 5-fold cross-validation showed fold-conditional behavior (9/15 observations positive; range -22% to +28%). On Prostate158, both models saturated (McNemar pooled p=0.69), while the false-positive contrast-matching finding replicated. Conclusion: Residual false positives are contrast-matched to cancer (sharing raw imaging features rather than histologically confirmed mimicry), reproducing across five architectures – a data-level imaging property, not model-specific artifacts; post-hoc refinement adds practical specificity in-domain but is fold-conditional.
Read the full abstract ▾
All authors Yongbo Shu, Kewen Chen, Yifeng Yuan, Zirui Xin, Luo Lei, Yang Yang, Xi Chen, Aijing Luo
Machine Learning
Atompack: A Storage and Distribution Layer for Read-Heavy Atomistic ML Training Datasets
Ali Ramlaoui, Daniel T. Speckhard, Sagar Pal et al. (+3 more)
Atomistic machine learning
datasets are increasingly used for training: large immutable snapshots are read repeatedly, shuffled across epochs, staged across clusters' storage systems, and republished as reusable scientific artifacts. This workload differs from interactive scientific curation, where mutable records and ad hoc inspection are often more important than random indexed throughput. We present Atompack, an append-oriented storage format and distribution layer designed around a simple workload: training pipelines usually consume complete molecular records, while the order of records is randomized by the learning
algorithm. Atompack appends records efficiently during
dataset construction, then commits an immutable index and serves records through a
memory-mapped read path optimized for training. We compare Atompack with HDF5, LMDB, and ASE baselines representing array stores, key-value records, serialized records, and object-oriented databases. The
benchmarks measure sequential reads, shuffled reads, shared-filesystem behavior, write throughput, and artifact size. On a representative 64-atom workload, Atompack is 96x faster than ASE LMDB on shuffled training-style reads while producing artifacts about 79% smaller. The results indicate that serving complete molecule records, rather than field chunks or reconstructed objects, improves shuffled training throughput while keeping artifacts compact enough for public distribution.
Read the full abstract ▾
All authors Ali Ramlaoui, Daniel T. Speckhard, Sagar Pal, Fragkiskos D. Malliaros, Alexandre Duval, Victor Schmidt
Machine Learning · Cross-listed from cs.AI
First-Order Temporal Logic Tensor Networks
Luca Boscarato, Ivan Donadello, Alessandro Artale et al. (+2 more)
Most of the existing neuro-symbolic AI methods focus on the scenario of static knowledge where objects do not change according to a temporal dimension. Temporal neuro-symbolic works are still under explored and are mainly developed for time-interval logic or propositional linear temporal logic. There is a lack of models studying linear temporal logics with predicates that deal with objects whose properties and relations change through the time. We present First-Order Temporal Logic Tensor Networks (FOT-LTN) that is an extension of Logic Tensor Networks (LTN) that fills this gap by considering a linear-temporal dimension. In particular, FOT-LTN joins the syntax of First-Order Linear Temporal Logic with the fuzzy (and real-valued) semantics of LTN obtaining a framework that supports both temporal operators and quantifiers and is totally differentiable. A first evaluation regards a temporal knowledge graph completion task on two synthetic
datasets showing better performance of FOT-LTN with respect to dedicated (purely neural) methods.
Read the full abstract ▾
All authors Luca Boscarato, Ivan Donadello, Alessandro Artale, Marco Montali, Fabrizio Maria Maggi
Machine Learning
NeuReasoner: Theory-grounded Mapping of Reasoning Elicitation Boundaries
Aydin Javadov, Shyngys Aitkazinov, Tobias Hoesli et al. (+3 more)
A growing body of work suggests that the
reasoning capabilities of large
language models are largely latent in their base form, with post-training primarily amplifying rather than introducing them. However, this evidence comes mainly from mathematical and coding
benchmarks, leaving the boundary conditions of that claim largely unexplored, namely which cognitive tasks can be recovered through elicitation and where that recovery fails. To investigate this, we introduce NeuReasoner, a theory-grounded elicitation instrument. At each step, an orchestrator pairs a Neuro Lens, inspired by functional specificity, with a Cognitive Lens, drawn from the Erotetic Theory of
Reasoning, and integrates their outputs through internal modularization of a single model, without external tools. We evaluate NeuReasoner on CogBench, a suite of behavioral tasks from cognitive psychology, alongside standard mathematical and coding
benchmarks, measuring both its improvement over vanilla inference and its ability to match a model's post-trained thinking mode. At sufficient scale, NeuReasoner matches or exceeds thinking-mode baselines on arithmetic
reasoning, code generation,
Bayesian reasoning, and reward learning; these gains persist against self-consistency and iterative-refinement baselines matched to NeuReasoner's per-decision call budget. Using NeuReasoner allows us to find clear boundaries: risk-taking and decision making under uncertainty remains hard to recover through elicitation alone, and model scale interacts with elicitation in both directions: widening its advantage on some cognitive signatures while erasing it on others. Overall, through NeuReasoner as a modular, interpretable, theory-grounded elicitation instrument, we empirically map where
reasoning elicitation succeeds and fails, beyond the mathematical and coding
benchmarks where prior claims have rested.
Read the full abstract ▾
All authors Aydin Javadov, Shyngys Aitkazinov, Tobias Hoesli, Florian von Wangenheim, Bjoern Schuller, Joseph Ollier
Machine Learning
DuoMem: Towards Capable On-Device Memory Agents via Dual-Space Distillation
Peyman Hosseini, Ondrej Bohdal, Ahmed Alajrami et al. (+6 more)
Large
Language Model (LLM)-based
agents can solve complex procedural tasks by interacting with environments over multiple turns, but this ability typically depends on large models, long contexts, and repeated inference calls. This makes advanced
memory-augmented
agents difficult to deploy on resource-constrained devices. We introduce DuoMem, a dual-space
distillation framework that transfers procedural problem-solving ability from a large teacher model to compact student models. DuoMem distils in two complementary spaces: (1)context-space
distillation, which replaces student-generated
memories with higher-quality teacher-generated procedural
memories prepended to the student's input, and (2)parameter-space
distillation, which fine-tunes lightweight LoRA adapters on successful teacher trajectories. Evaluated on ALFWorld, a challenging
embodied decision-making
benchmark, DuoMem boosts a 4B-parameter model from 4.3% to 77.9% task success rate, closing most of the gap to a 72B teacher model (87.1%), while adding fewer than 10M trainable parameters and only a few megabytes of pre-computed teacher
memories. Moreover, the DuoMem-enhanced 4B model completes tasks over 3x faster than the 72B teacher in wall-clock time, making it viable for real-time edge deployment, which would be challenging for the teacher.Extensive ablations across eight models spanning 2B-72B parameters reveal that both
distillation axes contribute complementary
Read the full abstract ▾
All authors Peyman Hosseini, Ondrej Bohdal, Ahmed Alajrami, Andrea Maracani, Ignacio Castro, Matthew Purver, Mete Ozay, Savas Ozkan, Taha Ceritli
Machine Learning
Exploiting Local Flatness for Efficient Out-of-Distribution Detection
Seonghwan Park, Hyunji Jung, Dongyeop Lee et al. (+1 more)
Detecting out-of-distribution (OOD) data is crucial for reliable machine learning deployment. Among detection strategies, post-hoc methods are particularly attractive due to their efficiency, as they operate directly on pre-trained networks without requiring retraining. Within this paradigm, one promising direction exploits loss-landscape
curvature to estimate model uncertainty; however, such methods incur substantial computational cost and rely on implicit assumptions about how landscape flatness differs between in-distribution (ID) and OOD data. In this work, we provide the first systematic investigation of this
curvature discrepancy and show that OOD inputs exhibit larger Hessian
curvature than ID data, with the gap widening under stronger distributional shifts. Motivated by these observations, we propose Fold, a lightweight flatness-modulated OOD detector that leverages the feature Hessian and partial feature normalization to improve ID-OOD separability while avoiding costly parameter-space
curvature approximations. To optimally adapt this normalization across diverse
datasets, we further introduce AutoFold, a self-supervised tuning scheme that synthesizes pseudo-OOD samples via ID logit masking for automatic calibration without requiring external data. Experiments on OOD
benchmarks show that Fold outperforms prior methods, improving the average AUROC by 1.63% and reducing FPR95 by 2.30%, while maintaining computational efficiency comparable to a standard forward pass. Supported by theoretical analysis and extensive ablations, Fold provides a principled and practical solution for robust real-world deployment.
Read the full abstract ▾
All authors Seonghwan Park, Hyunji Jung, Dongyeop Lee, Namhoon Lee
Machine Learning
Improved Predictive Performance and Interpretability for Mesomorphic Neural Networks Using Local Fidelity Regularization
Hugo L. Hammer, Vajira Thambawita, Kristoffer Herland Hellton et al. (+1 more)
Interpretable Mesomorphic Neural Networks (IMNs) offer a promising framework that combines the predictive power of deep neural networks with the interpretability of linear models. However, the original formulation lacks safeguards to ensure that the learned interpretations are in fact reliable. In particular, the network is free to concentrate all explanatory variance into a single weight of the linear output layer, achieving strong predictive performance while producing interpretations that are largely meaningless. Paradoxically, the L1 penalty proposed to encourage sparse solutions exacerbates this problem by further incentivizing such degenerate configurations. To address this vulnerability, we introduce Local Fidelity Regularization (LFR), a novel penalty term that prevents degenerate weight collapse by aligning the linear output weights with local data variations. This structural constraint guarantees faithful explanations and substantially improves the reliability of model interpretations. Furthermore, empirical evaluations across the OpenML
benchmark suite demonstrate that LFR does not compromise accuracy for explainability; rather, it achieved improved AUROC over the unregularized IMN. By yielding results highly competitive with state-of-the-art black-box models, LFR provides the dual benefit of reliable interpretability and superior predictive performance. Source code and usage instructions are available at https://github.com/hugohammer/LFR-IMN.git.
Read the full abstract ▾
All authors Hugo L. Hammer, Vajira Thambawita, Kristoffer Herland Hellton, Pål Halvorsen
Machine Learning · Cross-listed from cs.IR
Diagnosing and Mitigating Retrieval Bottlenecks in LLM-Based Cold-Start Recommendation
Zhe Dong, Fang Qin, Manish Shah et al. (+1 more)
Large
language models (LLMs) are increasingly used as rerankers in recommender systems, with the expectation that semantic understanding will help in cold-start and long-tail regimes. We test this assumption with a five-domain
benchmark that explicitly separates reranking quality from
retrieval coverage. In a positive-controlled regime where the gold item is guaranteed present, calibrated LLM rerankers fail to consistently outperform strong collaborative and content baselines under natural traffic, and within-family scaling from Qwen3-8B to Qwen3-32B narrows but does not close the gap on most domains. In a
retrieval-realistic regime where the gold item is not injected, the bottleneck is more severe: standard single retrievers place the gold item in a 200-item pool only 4.6-22.9% of the time, largely because 32-91% of cold-start targets are brand-new items with no training interactions. We introduce LHF, a validation-trained learned hybrid fusion layer over a multi-retriever union pool, as a
retrieval-side realizability baseline. LHF is the only combiner we test that beats every single retriever on all five domains and recovers 17-61% of oracle coverage headroom on content-rich domains, but only 5-7% on collaboratively strong domains. End-to-end experiments reveal the remaining mismatch: learned non-LLM ranking exploits the LHF pool, while
prompt-level LLM reranking often degrades it. LLMs exhibit pockets of semantic cold-start advantage, especially in text-rich domains when the item is already present, but this advantage is largely unreachable in current retrieve-then-rerank pipelines. We release the
benchmark protocol, splits,
prompts, evaluation tooling, and archived reproducibility artifacts: data at https://doi.org/10.5281/zenodo.20991039 and code at https://doi.org/10.5281/zenodo.20993306.
Read the full abstract ▾
All authors Zhe Dong, Fang Qin, Manish Shah, Yicheng Wang
Machine Learning
Bandwidth Selection in Kernel Density Estimation for Model Calibration
Han Zhou, Teodora Popordanoska and Matthew Blaschko
As deep learning models are increasingly deployed in high-stakes applications, providing well-calibrated uncertainty estimates has become as critical as achieving high predictive accuracy. While Kernel Density Estimation (KDE) has emerged as a smooth and continuous alternative to traditional binning for quantifying miscalibration, its reliability is heavily dependent on the choice of the kernel bandwidth. Standard selection techniques, such as Maximum Likelihood Estimation (MLE), often fail to produce optimal bandwidths for calibration tasks. In this work, we introduce Risk
Alignment (RA), a novel optimization framework that determines the optimal bandwidth by aligning KDE-reconstructed risk with empirical risk. We theoretically demonstrate that this
alignment minimizes calibration estimation bias across the data distribution, establishing a principled bandwidth selection criterion applicable to various metrics, including the challenging case of canonical calibration error. Extensive experiments across multiple architectures and
datasets show that RA consistently outperforms standard bandwidth selection methods, yielding more reliable calibration assessments.
Read the full abstract ▾
Machine Learning
EVAF: A Test-Retest Protocol for Selective Parametric Consolidation
Haoliang Han
Long-running
language agents need mechanisms for deciding which experiences should persist after the working context is gone.
Retrieval systems can reinsert past text, but they do not by themselves show that an experience has been selectively consolidated into the model's own behavior. We introduce EVAF, an Echo-Valence Attractor Field mechanism for gated LoRA consolidation, and a test-retest protocol for measuring selective parametric consolidation under controlled interference. Across GPT-2 and TinyLlama, EVAF preferentially consolidates high-valence, high-surprise experiences while preserving
retrieval-accessible factual
memory through a complementary routed
memory path. Test-retest measurements show stronger post-interference behavioral persistence than frozen,
retrieval-only, and ungated continual-update baselines, while keeping parameter drift and cross-persona contamination low. The results support a separation between
memory access and
memory depth: retrieving a fact and internalizing an experience are distinct computational operations.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
MemDelta: Controlled Baselines and Hidden Confounds in Agent Memory Evaluation
Kuan Wang
Agent memory systems are increasingly evaluated against RAG and full-context baselines, but reported gains often mix changes in the
memory method with changes in the
language model,
embedding model, or
retrieval pipeline, making it unclear what is actually being measured. We present MemDelta, a controlled evaluation protocol that varies one component at a time on LongMemEval-S (500 questions, 50+ sessions, three model families). Four findings emerge: (1) verbatim RAG matches full-context GPT-4o-mini (47.2% vs. 49.8%, p = 0.34), but the ranking reverses across models: Gemini gains +14pp from full context, while Sonnet gains +31pp from RAG, partly because it refuses 63% of full-context queries; (2) swapping only the
embedding model in an identical pipeline shifts accuracy by +6.2pp at n = 500 (p = 0.004), and Mem0 beats MiniLM-RAG by +11pp but loses to cloud-RAG by 1.2pp, so one variable flips the conclusion; (3)
agent self-
memory (42%) underperforms basic
retrieval (47%); (4) on 2 of 6 question types (n = 88), Mem0 matches cloud RAG (72.7% vs. 73.9%, p = 1.0) at 50x the cost, suggesting narrow rather than general gains. We recommend
memory evaluations fix
embedding models across comparisons, stratify by model family, and report write-path cost before attributing gains to architecture.
Read the full abstract ▾
Machine Learning
CW-B: Class Weighted Boosting Framework for Imbalance Resilient Multi Class Cardiac Phenotyping
Sijia Li, Xiaoyu Tan, Chen Zhan et al. (+3 more)
Cardiac discharge phenotyping informs post-discharge treatment and follow-up, but real-world records are often incomplete and class-imbalanced, increasing the risk of missed high-risk phenotypes. We propose CW-B, a clinical risk-aligned class-weighted XGBoost pipeline for five-class cardiac discharge phenotyping under real-world class imbalance and missingness. CW-B combines fold-specific class-balanced instance weighting, missingness-indicator augmentation, and classwise error
auditing to improve recognition of clinically prioritized phenotypes while preserving interpretable and auditable decision logic. In five-fold stratified cross-validation, CW-B achieves the best Accuracy, Macro-F1, Balanced Accuracy, and Prioritized F1 among tree-based, ensemble, and neural baselines. Overall, CW-B provides a practical and deployment-oriented approach for more reliable cardiac discharge phenotyping in real-world clinical settings.
Read the full abstract ▾
All authors Sijia Li, Xiaoyu Tan, Chen Zhan, Yuanji Ma, Haoyu Wang, Xihe Qiu
Machine Learning · Cross-listed from cs.IR
SABER-Math: Automated Benchmark for Information Retrieval Evaluation in Mathematics
Nikolay Georgiev, Maria Drencheva, Kseniia Ibragimova et al. (+3 more)
As agentic AI systems tackle more complex mathematical tasks, they increasingly rely on information
retrieval (IR) to search problem databases,
theorem libraries, and educational resources. However, choosing the right retriever remains difficult, as it is infeasible to directly isolate its effect on downstream performance. On the other hand, existing
retrieval-specific
benchmarks often fail to capture fine-grained mathematical relevance, penalizing relevant documents. We address this gap by introducing SABER-Math, the first fully automated
benchmark for evaluating mathematical IR without expert annotation. Starting from 283K high-school-level math problems with solutions, SABER-Math builds challenging reranking tasks in three steps: (i) first, LLMs extract concise solution summaries and mathematical topics for each problem; (ii) then, per-query relevant documents are discovered using ontology topic-based and lexical solutions-summary-based similarities, and (iii) finally, a Swiss-style LLM preference tournament produces fine-grained relevance ratings for the documents. We evaluate lexical retrievers, specialized mathematical
retrieval systems, and recent
embedding models. We find that while modern
embedding models substantially outperform classical and math-specific baselines, even the strongest systems struggle in symbol-heavy domains like
Algebra and Calculus. Importantly, we show that general-purpose IR
benchmarks such as MTEB do not reliably predict mathematical performance, especially for recent
embedding models, highlighting the need for math-specific
retrieval benchmarks.
Read the full abstract ▾
All authors Nikolay Georgiev, Maria Drencheva, Kseniia Ibragimova, Ivo Petrov, Dimitar I. Dimitrov, Martin Vechev
Machine Learning
Golden Hour Divide: Trauma Care Accessibility and Resource Vulnerability in Sri Lanka
Sonath Kirindage, Vihanga Nimsara, Sakindu Rajapaksa et al. (+6 more)
Timely intensive care dictates survival, yet emergency infrastructure remains unevenly distributed across Sri Lanka. While pre-hospital services have expanded, the transition to definitive care remains a critical bottleneck. This study evaluates national emergency resilience by quantifying the gap between clinical demand and the availability of specialized resources across all 25 districts. Using the latest national epidemiological data and terrain-aware H3 hexagonal modeling, we analyzed accessibility for seven critical conditions based on spatial gaps, clinical need-gaps, lethality, coverage, and resource availability. Based on these metrics, unsupervised K-Means clustering was applied to categorize districts into four policy-actionable archetypes: Critical Structural Exclusion, Institutional Mirages, Operational Capacity Strain, and High-Resilience
Benchmarks. Our study suggests that severe service deficits exist in the Northern and Eastern provinces, where spatial gaps exceed 70%, rendering the Golden Hour operationally impossible. Notably, specialist scarcity drives systemic pressure more than bed capacity; underserved regions effectively function as institutional mirages. This study suggests that improving accessibility by 25% in high-priority clusters would reduce the national need-gap by 9.65%, providing a roadmap for the strategic redistribution of specialists to ensure healthcare equity.
Read the full abstract ▾
All authors Sonath Kirindage, Vihanga Nimsara, Sakindu Rajapaksa, Kavyanga Hathurusinghe, Lahiru Dilshan, Subavarshana Arumugam, Nathali Athukorala, Sandareka Wickramanayake, Nisansa de Silva
Machine Learning
Same Concept, Different Directions: Cross-Modal Feature Heterogeneity in Sparse Autoencoders
Chungpa Lee, Jihoon Kwon, Kyle Min et al. (+1 more)
Vision-
language models map images and text into a joint
embedding space. However, these
embeddings often entangle multiple semantic features, which limits their interpretability and controllability. While sparse autoencoders have emerged as a useful tool for decomposing these
embeddings into monosemantic features, their application to joint
embedding spaces has largely relied on an implicit, untested assumption that semantically corresponding features share the same directions across modalities. In this paper, we challenge this assumption by identifying discrepancies in feature directions for the same concept across image and text modalities, a phenomenon we term cross-modal feature heterogeneity. We demonstrate that this heterogeneity is a key driver of the modality split, where a shared concept activates different latents depending on the modality. This finding further reveals why aligning latent activations alone is insufficient to resolve the underlying feature mismatch. Motivated by this observation, we propose an approach that trains modality-specific sparse autoencoders to preserve each modality's feature geometry, and then aligns corresponding features post hoc. Our method improves reconstruction fidelity and enhances performance in cross-modal
retrieval and concept steering.
Read the full abstract ▾
All authors Chungpa Lee, Jihoon Kwon, Kyle Min, Jy-yong Sohn
Machine Learning
Decision-Value Attribution in Predict-then-Optimize Systems
Konstantinos Ziliaskopoulos, Alexander Vinel and Alice E. Smith
Predictive models are increasingly embedded in operational decision-making, yet standard explanation methods typically explain forecasts rather than the decisions those forecasts induce. This distinction is important in predict-then-optimize systems: large forecast changes may leave the optimizer's action unchanged, while small changes can alter the selected decision and its realized value. We propose Decision Value Attribution (DVA), a Shapley-based framework for attributing the value of a fixed prediction–optimization pipeline. The framework defines cooperative games whose payoff is the downstream decision value, allowing the players to be information sources, optimization or design parameters, or both. We present three variants: InfoDVA attributes value to features, DesignDVA attributes value to operational configurations, and Decision-Value Interactions (DVI) quantifies how information and design jointly create value. We further distinguish post-DVA, which evaluates decisions using realized outcomes, from pre-DVA, which evaluates decisions under the model's full prediction. This separation turns attribution into a decision-level diagnostic of whether the model's operational beliefs align with realized performance. The resulting attributions are expressed in the units of the operational objective and decompose the gain or loss relative to a baseline. Case studies in electricity storage arbitrage and emergency medical service coverage show that predictive explanations can be poor proxies for operational value, that DVA can guide targeted information-control interventions, and that optimization configurations determine when predictive information is decision-relevant.
Read the full abstract ▾
Machine Learning · Cross-listed from math.NA
Implementation of Hyperelastic Physics-Augmented Neural Networks in the Explicit Finite Element Codes Simcenter Radioss and OpenRadioss with Applications to Impact Events
Lukas Maurer, Sascha Eisenträger, Marian Bulla et al. (+1 more)
Data-driven material modeling techniques have gained significant
attention due to their ability to capture complex constitutive behaviors beyond the limitations of classical material models. Physics-augmented neural networks (PANNs), which embed physical constraints directly into their architecture, combine the flexibility of machine learning with the reliability required for engineering simulations. This work presents an approach to integrate such network architectures into the explicit finite element solvers Simcenter Radioss and OpenRadioss (Siemens). A framework for transferring pretrained network architectures and their parameters to a standalone user material routine is developed. Networks are trained using PyTorch, though the procedure can be adapted to other frameworks such as TensorFlow, enabling the use of PANNs within existing finite element technology without requiring specialized solvers. Particular emphasis is placed on computational efficiency. The influence of network architecture on simulation performance is investigated, and strategies for reducing evaluation costs while preserving accuracy are discussed. Specifically, replacing the SoftPlus activation function with SQuarePlus is shown to reduce computational cost. A publicly available GitHub repository automates the generation of Fortran user material routines, requiring only the specification of the network architecture and trained parameters. An example impact simulation demonstrates that the generated PANN user material reproduces the nonlinear behavior characteristic of hyperelastic materials under large strains, providing a practical route toward machine-learning-based constitutive models in explicit finite element simulations.
Read the full abstract ▾
All authors Lukas Maurer, Sascha Eisenträger, Marian Bulla, Daniel Juhre
Machine Learning
RoAd-RL: A Unified Library and Benchmark for Robust Adversarial Reinforcement Learning
Adithya Mohan, Daniel Kriegl and Torsten Schön
Deep Reinforcement Learning (DRL) has achieved significant success in robotics and autonomous systems, yet remains vulnerable to
adversarial perturbations that can severely degrade performance. Research in
adversarial reinforcement learning is often limited by fragmented implementations, inconsistent evaluation protocols, and poor reproducibility. To address these challenges, we present RoAd-RL, an open-source benchmarking framework that provides unified abstractions for policies, attacks, defenses, and robustness metrics, together with reproducible evaluation pipelines and seamless integration with Stable-Baselines3 and Gymnasium. We evaluate DQN, PPO, and SAC
agents in LunarLander and Highway-v0 under 192 attack-defense configurations. Results reveal substantial variations in robustness across environments and show that some commonly used defenses can be more detrimental than the attacks they aim to mitigate, while temporal smoothing consistently achieves strong performance. RoAd-RL establishes a standardized
benchmark for
adversarial reinforcement learning research and is publicly available at https://pypi.org/project/road-rl.
Read the full abstract ▾
Machine Learning
Comparing Chatbot Performance Enhanced with Persistent Homology
Nithisha Raghavaraju, Barbara Giunti and Bastian Rieck
Chatbots have become increasingly prevalent across various domains, offering automated assistance in many areas, especially mental health support. The training is done using extremely large
datasets, which are sometimes not available in very specific domains. Moreover, it would sometimes be ideal to train the chatbot with personal information about the patients, which, of course, cannot be done on shared servers since it would violate patient confidentiality. Hence, being able to improve the performance of a chatbot, possibly trained locally and on a restricted
dataset, without having to increase the
dataset itself, would be extremely beneficial. In this work, we will enhance the input
datasets using persistent homology (PH) vectorizations computed from the raw
datasets themselves. Then we will compare, across several metrics, the performance of multiple chatbot models with or without the PH enhancement. Our experiments suggest that, while at times the PH enhancement is not particularly beneficial, it sometimes brings remarkable advantages for virtually no cost.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Linrui Ma, Chun Hei Lo, Xinyu Wang et al. (+12 more)
The quadratic computational cost of traditional
attention mechanisms poses a major bottleneck to the scalability and practical deployment of large
language models (LLMs), particularly in long-context scenarios. To improve efficiency, existing approaches often enforce rigid structural constraints such as local
attention windows. However, these strategies typically lead to substantial performance degradation on tasks requiring precise long-range recall. In this work, we propose MATCH, a scalable and efficient framework that augments sparsified
attention mechanisms with dynamically integrated in-context information through an efficient
retrieval system. Empirical results show that MATCH significantly improves the performance of sparse-
attention models on both synthetic and real-world natural-
language tasks. These findings highlight the versatility of MATCH as a general approach for enhancing in-context
retrieval capabilities while maintaining the efficiency benefits of sparse
attention architectures.
Read the full abstract ▾
All authors Linrui Ma, Chun Hei Lo, Xinyu Wang, Peng Lu, Xihao Yuan, Hanting Chen, Kai Han, Xinghao Chen, Chengjun Zhan, Hanlin Xu, Yichun Yin, Lifeng Shang, Feng Wen, Boxing Chen, Yufei Cui
Machine Learning
Theory of Continual Learning Against Data Poisoning Attacks
Yiting Hu and Lingjie Duan
Continual learning (CL), where a model is trained on a sequence of data tasks, is increasingly being adopted across key fields such as large
language models and image recognition, yet it remains highly vulnerable to data poisoning that triggers learning divergence or severe excess risk. Despite these threats, a principled theoretical foundation in CL for understanding attack and defense remains lacking. In this paper, we develop a theoretical framework to analyze strategic attacks and defenses in regularization-based CL, a cornerstone of recent CL theory. By framing the adversary-defender interaction as an online zero-sum game, we first establish a fundamental performance limit: no defense succeeds when an adversary poisons a linear proportion of tasks by injecting unbounded noise or pattern shifts in regularization-based CL. We then analyze two possibly defensible scenarios: infrequent attacks and bounded noise per attack. For the former regime, we propose a task-to-task verification mechanism to detect data poisoning and reduce cumulative bias for learning convergence. For the latter regime, we derive a robust defense that minimizes the model's sensitivity to poisoned features, provably accelerating the convergence rate. Extensive experiments on realistic tasks further validate our theoretical results.
Read the full abstract ▾
Machine Learning
The Forgetting-Retention Dilemma: Certified Unlearning Theory in Continual Learning
Yiting Hu, Lingjie Duan and Qian Zhang
Machine unlearning aims to eliminate the influence of specific data from trained models to safeguard
privacy. However, this presents a significant challenge in the context of continual learning (CL), where models update sequentially on dynamic
datasets. A major limitation is that current certified unlearning
algorithms fail to account for the complex, cumulative model
evolution inherent to CL framework. In this work, we establish the first theoretical foundation bridging CL and machine unlearning. We formulate the CL's unlearning objective as the minimization of post-unlearning excess risk, which decomposes into CL excess risk and unlearning loss, characterizing the fundamental trade-off between preserving historical knowledge and targeted forgetting. Under mild assumptions, we first establish an upper bound for the CL excess risk in non-convex models. We then adapt two certified unlearning approaches, gradient-based and Hessian-based, to the CL framework. Our analysis reveals that while the gradient-based approach is less effective than the Hessian-based method in minimizing unlearning loss, it offers the distinct advantage of nearly zero storage overhead for enabling unlearning. This insight motivates a hybrid strategy that reduces storage costs while maintaining post-unlearning performance. Experimental results further validate our theoretical findings.
Read the full abstract ▾
Machine Learning
Dual-Flow Reinforcement Learning with State-Aware Exploration
Qijun Li, Zheng Fu, Qi Song et al. (+4 more)
In complex continuous-control reinforcement learning tasks,
multimodal optimal actions often coincide with uncertain,
multimodal return distributions, making reliable value estimation and
multimodal exploration challenging. Existing value estimation methods using unimodal Gaussians restrict expressiveness and yield biased estimates. Recent generative policies can represent
multimodal actions but often collapse to a few modes and under-explore high-value areas of the action space. Motivated by these challenges, we propose Dual-Flow RL, a unified actor-critic framework that jointly models a continuous return distribution and a
multimodal policy distribution using conditional flow matching (CFM). This design supports reliable value estimation and sustained
multimodal exploration. To further enhance exploration, we introduce an
Entropy-Covariance Exploration Regulator (ECER) that enables state-aware exploration regulation leveraging policy
entropy and action-uncertainty covariance. Experiments on DeepMind Control Suite and
Humanoid-Bench show that Dual-Flow RL achieves state-of-the-art performance on most tasks, significantly outperforming prior
diffusion-based and flow-based methods.
Read the full abstract ▾
All authors Qijun Li, Zheng Fu, Qi Song, Yifei He, Weitao Zhou, Kun Jiang, Diange Yang
Machine Learning
Accelerating Q-learning through Efficient Value-Sharing across Actions
Prabhat Nagarajan, Brett Daley, Martha White et al. (+1 more)
Action values are foundational to many control
algorithms such as Q-learning. Therefore, efficient action-value learning is central to reinforcement learning (RL). However, learning them can be slow, requiring many updates to move values from their initialization, typically near zero, to their true values, which may be far from zero. Moreover, action-value learning
algorithms typically update each state-action pair independently, without learning a value that is common to all actions within a state. In this paper, we address these inefficiencies by introducing the mean-expansion layer, which accelerates action-value learning by sharing values across actions within a state and by changing the problem from directly learning potentially large action-values to learning a lower-norm representation of them. In deep RL, this layer can be applied as a parameter-free addition to Q-network architectures without altering the underlying
algorithm. Applied to deep Q-networks and implicit
quantile networks, it improves aggregate performance across 57 Atari 2600 games while increasing action gaps and dramatically reducing value overestimation.
Read the full abstract ▾
All authors Prabhat Nagarajan, Brett Daley, Martha White, Marlos C. Machado
Machine Learning · Cross-listed from cs.CR
Multi-Level Distributional Entropy from Flow Summary Statistics for Explainable Network Intrusion Detection
Mohamed Aly Bouke, Md Shohel Sayeed, Swee-Huay Heng et al. (+2 more)
Machine learning network intrusion detection systems (IDS) operate on aggregate flow statistics that discard the distributional structure of traffic, and although information-theoretic measures capture that structure, established
entropy estimators require raw packet sequences that pre-aggregated flow
datasets do not contain. No prior method derives
entropy from the summary statistics those records already hold. We introduce Multi-Level Distributional
Entropy (MDE), which computes interpretable information-theoretic features analytically from flow-level summary statistics at three levels, within-flow Gaussian differential
entropy, cross-directional Jensen-Shannon divergence (JSD), and Transmission Control Protocol (TCP) flag-incidence Shannon
entropy, with closed-form properties and no raw packet access; only imputation medians and score bounds are fitted on the training split. We pair the features with a leakage-free, fold-local evaluation protocol that reports the full operational metric suite across cross-validation, temporal, pseudo-live, cross-
dataset, and unseen-attack-family settings, on four
benchmarks (NSL-KDD, CICIDS-2017, CICIDS-2018, UNSW-NB15) with tree-ensemble classifiers and SHAP. The protocol exposes failure modes that aggregate weighted F1 conceals: on CICIDS-2018 an F1 of 0.73 hides a detection rate (DR) of 0.44, on held-out attack families F1 exceeds 0.998 while DR falls to zero, and a 703K-flow pseudo-live replay reveals a threshold-ranking divergence in which score ranking is largely preserved (area under the ROC curve, AUC, 0.84 to 0.86) while fixed-threshold detection collapses (DR 0.08). The
entropy features match conventional features within 0.1 percentage points of F1 and receive reproducible SHAP attributions (Spearman 0.84 to 0.94), so their contribution is a grounded, interpretable representation and an evaluation methodology rather than an accuracy gain.
Read the full abstract ▾
All authors Mohamed Aly Bouke, Md Shohel Sayeed, Swee-Huay Heng, Azizol Abdullah, Mohamed Othman
Machine Learning
What Drives the Inlier-Memorization Effect? A Theory of Outlier Detection via Early Training Dynamics
Kunwoong Kim and Dongha Kim
Outlier detection (OD) aims to identify anomalous instances by learning the underlying structure of normal data (inliers), and is particularly challenging in fully unsupervised settings where no information about anomalies is available during training. Recent advances have leveraged the inlier-memorization (IM) effect, a phenomenon in which deep models memorize inlier patterns earlier than those of outliers, as a powerful signal for distinguishing outliers. However, despite its empirical success, the theoretical understanding of the IM effect remains limited. In this work, we present a theoretical study of the IM effect. Focusing on a simple autoencoder, we show that, under mild assumptions, the model can successfully memorize inliers while failing to memorize outliers during certain stages of early training. In particular, we characterize not only the emergence of the IM effect, but also its strength and persistence, and analyze how these properties depend on the data distribution and parameter initialization. In addition, building on these insights, we derive simple yet practical guidelines for enhancing the IM effect, including data preprocessing and parameter initialization schemes, achieving state-of-the-art performance on the ADBench
datasets. Our findings provide a theoretical foundation for the IM effect and offer actionable directions for improving IM-based outlier detection methods.
Read the full abstract ▾