Machine Learning
Safety-Utility Conflicts Are Not Global: Surgical Alignment via Head-Level Diagnosis
Wang Cai, Yilin Wen, Jinchang Hou et al. (+5 more)
Safety
alignment in Large
Language Models (LLMs) inherently presents a multi-objective optimization conflict, often accompanied by an unintended degradation of general capabilities. Existing mitigation strategies typically rely on global gradient geometry to resolve these conflicts, yet they overlook Modular Heterogeneity within
Transformers, specifically that the functional sensitivity and degree of conflict vary substantially across different
attention heads. Such global approaches impose uniform update rules across all parameters, often resulting in suboptimal trade-offs by indiscriminately updating utility sensitive heads that exhibit intense gradient conflicts. To address this limitation, we propose Conflict-Aware Sparse Tuning (CAST), a framework that integrates head-level diagnosis with sparse fine-tuning. CAST first constructs a pre-
alignment conflict map by synthesizing Optimization Conflict and Functional Sensitivity, which then guides the selective update of parameters. Experiments reveal that
alignment conflicts in LLMs are not uniformly distributed. We find that the drop in general capabilities mainly comes from updating a small group of “high-conflict” heads. By simply skipping these heads during training, we significantly reduce this loss without compromising safety, offering an interpretable and parameter-efficient approach to improving the safety-utility trade-off.
Read the full abstract ▾
All authors Wang Cai, Yilin Wen, Jinchang Hou, Du Su, Guoqiu Wang, Zhonghou Lv, Chenfu Bao, Yunfang Wu
Machine Learning · Cross-listed from cs.AI
Towards a Mechanistic Understanding of Propositional Logical Reasoning in Large Language Models
Danchun Chen, Qiyao Yan, Chenpeng Wang et al. (+1 more)
Understanding how Large
Language Models (LLMs) perform logical
reasoning internally remains a fundamental challenge. While prior mechanistic studies focus on identifying task specific
circuits, they leave open the question of what computational strategies LLMs employ for propositional
reasoning. We address this gap with a causal mechanistic analysis on PropLogic-MI, a controlled
benchmark of 11 propositional rules across one- and two-hop tasks, applied to three model families (Qwen3, Llama-3.1, Mistral). Rather than asking which components are necessary, we ask how the
reasoning process is organized, and identify four interlocking mechanisms: Staged Computation, where early, middle, and late layers take on distinct functional roles; Information Transmission, where semantic content aggregates at boundary
tokens; Fact Retrospection, where fact
tokens stay causally active as binding lookup in middle layers and as sustained access in late layers under hard
reasoning loads; and Specialized
Attention Heads that structurally implement these patterns. A
prompt-order control that places the query before the facts confirms that this organization is model-internal rather than input-layout-induced. These findings show that pretrained LLMs solve propositional
reasoning through a structured, layer-organized process that recurs across models, rule categories, and
reasoning hops.
Read the full abstract ▾
All authors Danchun Chen, Qiyao Yan, Chenpeng Wang, Liangming Pan
Machine Learning
Lightweight Test-Time Adaptation for EMG-Based Gesture Recognition
Nia Touko, Matthew O A Ellis, Cristiano Capone et al. (+3 more)
Reliable long-term decoding of gestures from surface electromyography (EMG) is hindered by signal drift caused by electrode displacement, muscle fatigue, and/or posture changes. Although modern models achieve high intra-session accuracy, their performance often degrades substantially across recording sessions. Existing approaches to mitigate this problem typically rely on large training
datasets or computationally intensive pipelines that are unsuitable for energy-efficient wearable devices. We propose a lightweight test-time adaptation framework for EMG decoding. The framework includes three complementary adaptation strategies: (i) causal adaptive batch normalization for online statistical
alignment, (ii) Gaussian Mixture Model
alignment with experience replay to mitigate forgetting, and (iii) meta-learning for rapid few-shot calibration. We evaluate these methods on the multi-session NinaPro DB6
dataset. All approaches substantially improve inter-session robustness relative to a non-adaptive baseline while maintaining low computational overhead. Replay-regularized statistical
alignment provides the most stable adaptation under limited data, while meta-learning achieves the highest accuracy when sparse calibration labels are available. Overall, our self-supervised test-time adaptation methods reach up to 82% inter-session accuracy, significantly improving upon prior approaches while maintaining resource-efficient operation. These results demonstrate that lightweight test-time adaptation can enable robust, long-term EMG decoding for wearable or prosthetic applications.
Read the full abstract ▾
All authors Nia Touko, Matthew O A Ellis, Cristiano Capone, Alessio Burrello, Elisa Donati, Luca Manneschi
Machine Learning
Robust Physics Discovery from Highly Corrupted Data: A PINN Framework Applied to the Nonlinear Schrödinger Equation
Pietro de Oliveira Esteves
We demonstrate a deep learning framework capable of recovering physical parameters from the Nonlinear Schrodinger Equation (NLSE) under severe noise conditions. By integrating Physics-Informed Neural Networks (PINNs) with automatic differentiation, we achieve reconstruction of the nonlinear coefficient beta with less than 0.2 percent relative error using only 500 sparse, randomly sampled data points corrupted by 20 percent additive Gaussian noise, a regime where traditional finite difference methods typically fail due to noise amplification in numerical derivatives. We validate the method's generalization capabilities across different physical regimes (beta between 0.5 and 2.0) and varying data availability (between 100 and 1000 training points), demonstrating consistent sub-1 percent accuracy. Statistical analysis over multiple independent runs confirms robustness (standard deviation less than 0.15 percent for beta equals 1.0). The complete pipeline executes in approximately 80 minutes on modest cloud GPU resources (NVIDIA Tesla T4), making the approach accessible for widespread adoption. Our results indicate that physics-based regularization acts as an effective filter against high measurement uncertainty, positioning PINNs as a viable alternative to traditional optimization methods for inverse problems in spatiotemporal dynamics where experimental data is scarce and noisy. All code is made publicly available to facilitate reproducibility.
Read the full abstract ▾
Machine Learning
Agentic Rubrics as Contextual Verifiers for SWE Agents
Mohit Raghavendra, Anisha Gunjal, Bing Liu et al. (+1 more)
Verification is critical for improving
agents: it provides the reward signal for Reinforcement Learning and enables inference-time gains through Test-Time Scaling (TTS). Despite its importance, verification in software engineering (SWE)
agent settings often relies on code execution, which can be difficult to scale due to environment setup overhead. Scalable alternatives such as patch classifiers and heuristic methods exist, but they are less grounded in codebase context and harder to interpret. To this end, we explore Agentic Rubrics: an expert
agent interacts with the repository to create a context-grounded rubric checklist, and candidate patches are then scored against it without requiring test execution. On SWE-Bench Verified under parallel TTS evaluation, Agentic Rubrics achieve a score of 54.2% on Qwen3-Coder-30B-A3B and 40.6% on Qwen3-32B, with at least a +3.5 percentage-point gain over the strongest baseline in our comparison set. We further analyze rubric behavior, showing that rubric scores are consistent with ground-truth tests while also flagging issues that tests do not capture. Our ablations show that agentic context gathering is essential for producing codebase-specific, unambiguous criteria. Together, these results suggest that Agentic Rubrics provide an efficient, scalable, and granular verification signal for SWE
agents.
Read the full abstract ▾
All authors Mohit Raghavendra, Anisha Gunjal, Bing Liu, Yunzhong He
Machine Learning
Clinical Data Goes MEDS? Let's OWL make sense of it
Alberto Marfoglia, Jong Ho Jhee and Adrien Coulet
The application of machine learning on healthcare data is often hindered by the lack of standardized and semantically explicit representation, leading to limited interoperability and reproducibility across
datasets and experiments. The Medical Event Data Standard (MEDS) addresses these issues by introducing a minimal, event-centric data model designed for reproducible machine-learning workflows from health data. However, MEDS is defined as a data-format specification and does not natively provide integration with the Semantic Web ecosystem. In this article, we introduce MEDS-OWL, a lightweight OWL ontology that provides formal concepts and relations to represent MEDS
datasets as RDF graphs. Additionally, we implemented meds2rdf, a Python conversion library that transforms MEDS events into RDF graphs, ensuring conformance with the ontology. We evaluate the proposed approach on two
datasets: a synthetic clinical cohort describing care pathways for ruptured intracranial aneurysms, and a real-world subset of MIMIC-IV. To assess semantic consistency, we performed a SHACL validation against the resulting knowledge graphs. The first release of MEDS-OWL comprises 13 classes, 10 object properties, 20 data properties, and 24 OWL axioms. Combined with meds2rdf, it enables data transformation into FAIR-aligned
datasets, provenance-aware publishing, and interoperability of event-based clinical data. By bridging MEDS with the Semantic Web, this work contributes a reusable semantic layer for event-based clinical data and establishes a robust foundation for subsequent graph-based analytics.
Read the full abstract ▾
Machine Learning · Cross-listed from eess.IV
Scanner-Induced Domain Shifts Undermine the Robustness of Pathology Foundation Models
Erik Thiringer, Fredrik K. Gustafsson, Kajsa Ledesma Eriksson et al. (+1 more)
Pathology foundation models (PFMs) have become central to computational pathology, aiming to offer general encoders for feature extraction from whole-slide images (WSIs). Despite strong
benchmark performance, PFM robustness to real-world technical domain shifts, such as variability from whole-slide scanner devices, remains poorly understood. We systematically evaluated the robustness of 14 PFMs to scanner-induced variability, including state-of-the-art models, earlier self-supervised models, and a baseline trained on natural images. Using a multiscanner
dataset of 384 breast cancer WSIs scanned on five devices, we isolated scanner effects independently from biological and laboratory confounders. Robustness is assessed via complementary unsupervised
embedding analyses and a set of clinicopathological supervised prediction tasks. Our results demonstrate that current PFMs are not invariant to scanner-induced domain shifts. Most models encode pronounced scanner-specific variability in their
embedding spaces. While AUC often remains stable, this masks a critical failure mode: scanner variability systematically alters the
embedding space and impacts calibration of downstream model predictions, resulting in scanner-dependent bias that can impact reliability in clinical use cases. We further show that robustness is not a simple function of training data scale, model size, or model recency. None of the models provided reliable robustness against scanner-induced variability. While the models trained on the most diverse data, here represented by
vision-
language models, appear to have an advantage with respect to robustness, they underperformed on downstream supervised tasks. We conclude that development and evaluation of PFMs requires moving beyond accuracy-centric
benchmarks toward explicit evaluation and optimisation of
embedding stability and calibration under realistic acquisition variability.
Read the full abstract ▾
All authors Erik Thiringer, Fredrik K. Gustafsson, Kajsa Ledesma Eriksson, Mattias Rantalainen
Machine Learning · Cross-listed from cs.CL
FLEx: Language Modeling with Few-shot Language Explanations
Adar Avsian, Christopher Richardson, Anirudh Sundar et al. (+1 more)
Language models have become effective at a wide range of tasks, from math problem solving to open-domain question answering. However, they still make mistakes, and these mistakes are often repeated across related queries. Natural
language explanations can help correct these errors, but collecting them at scale may be infeasible, particularly in domains where expert annotators are required. To address this issue, we introduce FLEx ($\textbf{F}$ew-shot $\textbf{L}$anguage $\textbf{Ex}$planations), a method for improving model behavior using a small number of explanatory examples. FLEx selects representative model errors using
embedding-based clustering, verifies that the associated explanations correct those errors, and summarizes them into a
prompt prefix that is prepended at inference-time. This summary guides the model to avoid similar errors on new inputs, without modifying model weights. We evaluate FLEx on CounterBench, GSM8K, and ReasonIF. We find that FLEx consistently outperforms chain-of-thought (CoT) prompting across all three
datasets and reduces up to 83% of CoT's remaining errors.
Read the full abstract ▾
All authors Adar Avsian, Christopher Richardson, Anirudh Sundar, Larry Heck
Machine Learning · Cross-listed from stat.ML
A Theoretical and Empirical Taxonomy of Imbalance in Binary Classification
Rose Yvette Bandolo Essomba and Ernest Fokoué
Class imbalance significantly degrades classification performance, yet its effects are rarely analyzed from a unified theoretical perspective. We propose a principled framework based on three fundamental scales: the imbalance coefficient $\eta$, the sample–dimension ratio $\kappa$, and the intrinsic separability $\Delta$. Starting from the Gaussian Bayes classifier, we derive closed-form Bayes errors and show how imbalance shifts the discriminant boundary, yielding a deterioration slope that predicts four regimes: Normal, Mild, Extreme, and Catastrophic. Using a balanced high-dimensional genomic
dataset, we vary only $\eta$ while keeping $\kappa$ and $\Delta$ fixed. Across parametric and non-parametric models, empirical degradation closely follows theoretical predictions: minority Recall collapses once $\log(\eta)$ exceeds $\Delta\sqrt{\kappa}$, Precision increases asymmetrically, and F1-score and PR-AUC decline in line with the predicted regimes. These results show that the triplet $(\eta,\kappa,\Delta)$ provides a model-agnostic, geometrically grounded explanation of imbalance-induced deterioration.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
ContextFocus: Activation Steering for Contextual Faithfulness in Large Language Models
Nikhil Anand, Shwetha Somasundaram, Anirudh Phukan et al. (+2 more)
Large
Language Models (LLMs) encode vast amounts of parametric knowledge during pre-training. As world knowledge evolves, effective deployment increasingly depends on their ability to faithfully follow externally retrieved context. When such evidence conflicts with the model's internal knowledge, LLMs often default to memorized facts, producing unfaithful outputs. In this work, we introduce ContextFocus, a lightweight activation steering approach that improves context faithfulness in such knowledge-conflict settings while preserving fluency and efficiency. Unlike prior approaches, our solution requires no model finetuning and incurs minimal inference-time overhead, making it highly efficient. We evaluate ContextFocus on the ConFiQA
benchmark, comparing it against strong baselines including ContextDPO, COIECD, and prompting-based methods. Furthermore, we show that our method is complementary to prompting strategies and remains effective on larger models. Extensive experiments show that ContextFocus significantly improves contextual-faithfulness. Our results highlight the effectiveness, robustness, and efficiency of ContextFocus in improving contextual-faithfulness of LLM outputs.
Read the full abstract ▾
All authors Nikhil Anand, Shwetha Somasundaram, Anirudh Phukan, Apoorv Saxena, Koyel Mukherjee
Machine Learning
MORPHFED: Federated Learning for Cross-institutional Blood Morphology Analysis
Gabriel Ansah, Eden Ruffell, Delmiro Fernandez-Reyes et al. (+1 more)
Automated blood morphology analysis can support hematological diagnostics in low- and middle-income countries (LMICs) but remains sensitive to
dataset shifts from staining variability, imaging differences, and rare morphologies. Building centralized
datasets to capture this diversity is often infeasible due to
privacy regulations and data-sharing restrictions. We introduce a
federated learning framework for white blood
cell morphology analysis that enables collaborative training across institutions without exchanging training data. Using blood films from multiple clinical sites, our
federated models learn robust, domain-invariant representations while preserving complete data
privacy. Evaluations across convolutional and
transformer-based architectures show that
federated training achieves strong cross-site performance and improved generalization to unseen institutions compared to centralized training. These findings highlight
federated learning as a practical and
privacy-preserving approach for developing equitable, scalable, and generalizable medical imaging AI in resource-limited healthcare environments.
Read the full abstract ▾
All authors Gabriel Ansah, Eden Ruffell, Delmiro Fernandez-Reyes, Petru Manescu
Machine Learning · Cross-listed from math.OC
A Single-Loop Bilevel Deep Learning Method for Optimal Control of Obstacle Problems
Yongcun Song, Shangzhi Zeng, Jin Zhang et al. (+1 more)
Optimal control of obstacle problems arises in a wide range of applications and is computationally challenging due to its nonsmoothness, nonlinearity, and bilevel structure. Classical numerical approaches rely on mesh-based discretization and typically require solving a sequence of costly subproblems. In this work, we propose a single-loop bilevel deep learning method, which is mesh-free, scalable to high-dimensional and complex domains, and avoids repeated solution of discretized subproblems. The method employs constraint-
embedding neural networks to approximate the state and control and preserves the bilevel structure. To train the neural networks efficiently, we propose a Single-Loop
Stochastic First-Order Bilevel
Algorithm (S2-FOBA), which eliminates nested optimization and does not rely on restrictive lower-level uniqueness assumptions. We analyze the convergence behavior of S2-FOBA under mild assumptions. Numerical experiments on
benchmark examples, including distributed and obstacle control problems with regular and irregular obstacles on complex domains, demonstrate that the proposed method achieves satisfactory accuracy while reducing computational cost compared to classical numerical methods.
Read the full abstract ▾
All authors Yongcun Song, Shangzhi Zeng, Jin Zhang, Lvgang Zhang
Machine Learning
Causal Data Augmentation for Robust Fine-Tuning of Tabular Foundation Models
Magnus Bühler, Lennart Purucker and Frank Hutter
Fine-tuning tabular foundation models (TFMs) under data scarcity is challenging, as early stopping on even scarcer validation data often fails to capture true generalization performance. We propose CausalMixFT, a method that enhances fine-tuning robustness and downstream performance by generating structurally consistent synthetic samples using Structural Causal Models (SCMs) fitted on the target
dataset. This approach augments limited real data with causally informed synthetic examples, preserving feature dependencies while expanding training diversity. Evaluated across 33 classification
datasets from TabArena and over 2300 fine-tuning runs, our CausalMixFT method consistently improves median normalized ROC-AUC from 0.10 (standard fine-tuning) to 0.12, outperforming purely statistical generators such as CTGAN (-0.01), TabEBM (-0.04), and TableAugment (-0.09). Moreover, it narrows the median validation-test performance correlation gap from 0.67 to 0.30, enabling more reliable validation-based early stopping, a key step toward improving fine-tuning stability under data scarcity. These results demonstrate that incorporating causal structure into data augmentation provides an effective and principled route to fine-tuning tabular foundation models in low-data regimes.
Read the full abstract ▾
Machine Learning · Cross-listed from cond-mat.str-el
Point-Group Equivariant Neural Networks for Local Dynamics in Lattice Systems
Yunhao Fan and Gia-Wei Chern
We develop point-group equivariant neural networks for learning local mappings used in dynamical simulations of
lattice systems. The same network acts on a finite neighborhood around each site, respecting
lattice translations through shared parameters. The local configuration is decomposed into features that transform according to irreducible representations of the point group, and hidden features retain this organization throughout the network. Equivariant linear maps and
symmetry-allowed tensor products then learn nonlinear spatial correlations while preserving the required transformation laws. This approach avoids constructing a fixed set of hand-crafted invariant descriptors, whose selection requires choices about which local correlations to retain. It also allows the complexity of the learned mapping to grow without enlarging its spatial cutoff. We demonstrate the construction by learning the forces governing adiabatic dynamics in the square-
lattice Holstein model. Comparisons with exact diagonalization test the accuracy of the predicted forces and the
evolution of spatial correlations. Force evaluation scales linearly with system size, allowing the trained model to simulate charge-density-wave ordering on large
lattices. These simulations reproduce anomalously slow coarsening of the charge-density-wave domains, illustrating the use of the learned local mapping for collective dynamics.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.NI
Cells on Autopilot: Adaptive Cell (Re)Selection via Reinforcement Learning
Marvin Illian, Ramin Khalili, Antonio A. de A. Rocha et al. (+1 more)
The widespread deployment of 5G networks, together with the coexistence of 4G/LTE networks, provides mobile devices a diverse set of candidate
cells to connect to. However, associating mobile devices to
cells to maximize overall network performance, a.k.a.
cell (re)selection, remains a key challenge for mobile operators. Today,
cell (re)selection parameters are typically configured manually based on operator experience and rarely adapted to dynamic network conditions. In this work, we ask: Can an
agent automatically learn and adapt
cell (re)selection parameters to consistently improve network performance? We present a reinforcement learning (RL)-based framework called CellPilot that adaptively tunes
cell (re)selection parameters by learning spatiotemporal patterns of mobile network dynamics. Our study with real-world data demonstrates that even a lightweight RL
agent can outperform conventional heuristic reconfigurations by up to 167%, while generalizing effectively across different network scenarios. These results indicate that data-driven approaches can significantly improve
cell (re)selection configurations and enhance mobile network performance.
Read the full abstract ▾
All authors Marvin Illian, Ramin Khalili, Antonio A. de A. Rocha, Lin Wang
Machine Learning · Cross-listed from cs.CV
Unsupervised Modular Adaptive Region Growing and RegionMix Classification for Wind Turbine Segmentation
Raül Pérez-Gonzalo, Riccardo Magro, Andreas Espersen et al. (+1 more)
Reliable operation of wind turbines requires frequent inspections, as even minor surface damages can degrade aerodynamic performance, reduce energy output, and accelerate blade wear. Central to automating these inspections is the accurate
segmentation of turbine blades from visual data. This task is traditionally addressed through dense, pixel-wise deep learning models. However, such methods demand extensive annotated
datasets, posing scalability challenges. In this work, we introduce an annotation-efficient
segmentation approach that reframes the pixel-level task into a binary region classification problem. Image regions are generated using a fully unsupervised, interpretable Modular Adaptive Region Growing technique, guided by image-specific Adaptive Thresholding and enhanced by a Region Merging process that consolidates fragmented areas into coherent segments. To improve generalization and classification robustness, we introduce RegionMix, an augmentation strategy that synthesizes new training samples by combining distinct regions. Our framework demonstrates state-of-the-art
segmentation accuracy and strong cross-site generalization by consistently segmenting turbine blades across distinct windfarms.
Read the full abstract ▾
All authors Raül Pérez-Gonzalo, Riccardo Magro, Andreas Espersen, Antonio Agudo
Machine Learning
Minimum distance classification for nonlinear dynamical systems
Dominique Martinez
We address the problem of classifying trajectories or sequences generated by nonlinear dynamical systems, where each class corresponds to a distinct dynamical system. We propose Dynafit, a kernel-based method that learns a distance metric between training data and the underlying dynamics. New observations are assigned to the class whose dynamics best fit the observations according to the learned metric.The learning
algorithm approximates the Koopman operator, which globally linearizes the dynamics in a (potentially infinite-dimensional) feature space associated with a kernel function. The distance metric is computed in the feature space independently of its dimensionality by exploiting the kernel trick commonly used in machine learning. The kernel function can be tailored to incorporate prior knowledge of the dynamics when available. We consider a classical test example, the logistic map as a discrete dynamical system, and derive analytically the kernel function from the
polynomial Koopman basis that exactly linearizes the dynamics. Dynafit is applicable to a wide range of classification tasks involving nonlinear dynamical systems and sensors. We illustrate its effectiveness through three examples: chaos detection in the logistic map, recognition of handwritten dynamical patterns, and classification of visual dynamic textures.
Read the full abstract ▾
Machine Learning
Using Legacy Polysomnography Data to Train a Radar System to Quantify Sleep in Older Adults and People living with Dementia
M. Yin, K. G. Ravindran, C. Hadjipanayi et al. (+6 more)
Objective: Ultra-wideband radar technology offers a promising solution for unobtrusive and cost-effective in-home sleep monitoring. However, the limited availability of radar sleep data poses challenges in building robust models that generalize across diverse cohorts and environments. This study proposes a novel deep transfer learning framework to enhance sleep stage classification using radar data. Methods: An end-to-end neural network was developed to classify sleep stages based on nocturnal respiratory and motion signals. The network was trained using a combination of large-scale polysomnography (PSG)
datasets and radar data. A domain adaptation approach employing
adversarial learning was utilized to bridge the knowledge gap between PSG and radar signals. Validation was performed on a radar
dataset of 47 older adults (mean age: 71.2), including 18 participants with prodromal or mild Alzheimer disease. Results: The proposed network structure achieves an accuracy of 79.5% with a Kappa value of 0.65 when classifying wakefulness, rapid eye movement, light sleep and deep sleep. Experimental results confirm that our deep transfer learning approach significantly enhances automatic sleep staging performance in the target domain. Conclusion: This method effectively addresses challenges associated with data variability and limited sample size, substantially improving the reliability of automatic sleep staging models, especially in contexts where radar data is limited. Significance: The findings underscore the viability of UWB radar as a nonintrusive, forward-looking sleep assessment tool that could significantly benefit care for older people and people with neurodegenerative disorders.
Read the full abstract ▾
All authors M. Yin, K. G. Ravindran, C. Hadjipanayi, A. Bannon, A. Rapeaux, C. Della Monica, T. S. Lande, Derk-Jan Dijk, T. G. Constandinou
Machine Learning
LinkD: AutoRegressive Diffusion Model for Mechanical Linkage Synthesis
Yayati Jadhav and Amir Barati Farimani
Designing mechanical linkages to achieve target end-effector trajectories presents a fundamental challenge due to the intricate coupling between continuous node placements, discrete topological configurations, and nonlinear kinematic constraints. The highly nonlinear motion-to-configuration relationship means small perturbations in joint positions drastically alter trajectories, while the combinatorially expanding design space renders conventional optimization and heuristic methods computationally intractable. We introduce an autoregressive
diffusion framework that exploits the dyadic nature of linkage assembly by representing mechanisms as sequentially constructed graphs, where nodes correspond to joints and edges to rigid links. Our approach combines a causal
transformer with a Denoising
Diffusion Probabilistic Model (DDPM), both conditioned on target trajectories encoded via a
transformer encoder. The causal
transformer autoregressively predicts discrete
topology node-by-node, while the DDPM refines each node's spatial coordinates and edge connectivity to previously generated nodes. This sequential generation enables adaptive trial-and-error synthesis where problematic nodes exhibiting kinematic locking or collisions can be selectively regenerated, allowing autonomous correction of degenerate configurations during design. Our graph-based, data-driven methodology surpasses traditional optimization approaches, enabling scalable inverse design that generalizes to mechanisms with arbitrary node counts. We demonstrate successful synthesis of linkage systems containing up to 20 nodes with extensibility to N-node architectures. This work advances autoregressive graph generation methodologies and computational kinematic synthesis, establishing new paradigms for scalable inverse design of complex mechanical systems.
Read the full abstract ▾
Machine Learning
Symbolic Regression for Shared Expressions: Introducing Partial Parameter Sharing
Viktor Martinek and Roland Herzog
Symbolic regression aims to find symbolic expressions that describe
datasets. Due to its inherent interpretability, symbolic regression (SR) is a powerful paradigm for scientific discovery. Recent advances have expanded SR to describe related phenomena using a single expression with varying sets of parameters, thereby introducing a single categorical variable. To illustrate, this enables the search for a single expression describing temperaturedependent viscosity across multiple fluids, while simultaneously identifying a distinct set of fluid-specific parameters. We expand upon prior efforts by considering multiple categorical variables and introducing intermediate levels of parameter sharing. Rather than parameters being either entirely universal or entirely unique, some parameters can also be shared across specific categories while remaining distinct for others. This allows for separating universal effects (shared parameters), category-specific trends (partially-shared parameters), and category interactions (non-shared parameters). We test the limits of this setup in terms of reducing data requirements and transfer learning using a synthetic, fitting-only example. Furthermore, we apply the method to an astrophysics
dataset also used in a previous single-category study. In comparison, we achieve similar fit quality with significantly fewer parameters while extracting additional information about the problem.
Read the full abstract ▾
Machine Learning
Modeling Behavioral Patterns in News Recommendations Using Fuzzy Neural Networks
Kevin Innerebner, Stephan Bartl, Markus Reiter-Haas et al. (+1 more)
News recommender systems are increasingly driven by black-box models, offering little transparency for editorial decision-making. In this work, we introduce a transparent recommender system that uses fuzzy neural networks to learn human-readable rules from behavioral data for predicting article clicks. By extracting the rules at configurable thresholds, we can control rule complexity and thus, the level of interpretability. We evaluate our approach on two publicly available news
datasets (i.e., MIND and EB-NeRD) and show that we can accurately predict click behavior compared to several established baselines, while learning human-readable rules. Furthermore, we show that the learned rules reveal news consumption patterns, enabling editors to align content curation goals with target audience behavior.
Read the full abstract ▾
All authors Kevin Innerebner, Stephan Bartl, Markus Reiter-Haas, Elisabeth Lex
Machine Learning · Cross-listed from cs.SE
Using Small Language Models to Reverse-Engineer Machine Learning Pipelines Structures
Nicolas Lacroix, Mireille Blay-Fornarino, Sébastien Mosser et al. (+1 more)
Background: Extracting the stages that structure Machine Learning (ML) pipelines from source code is key for gaining a deeper understanding of data science practices. However, the diversity caused by the constant
evolution of the ML ecosystem (e.g.,
algorithms, libraries,
datasets) makes this task challenging. Existing approaches either depend on non-scalable, manual labeling, or on ML classifiers that do not properly support the diversity of the domain. These limitations highlight the need for more flexible and reliable solutions. Objective: We evaluate whether Small
Language Models (SLMs) can leverage their code understanding and classification abilities to address these limitations, and subsequently how they can advance our understanding of data science practices. Method: We conduct a confirmatory study based on two reference works selected for their relevance regarding current state-of-the-art's limitations. First, we compare several SLMs using Cochran's Q test. The best-performing model is then evaluated against the reference studies using two distinct McNemar's tests. We further analyze how variations in taxonomy definitions affect performance through an additional Cochran's Q test. Finally, a goodness-of-fit analysis is conducted using Pearson's chi-squared tests to compare our insights on data science practices with those from prior studies.
Read the full abstract ▾
All authors Nicolas Lacroix, Mireille Blay-Fornarino, Sébastien Mosser, Frederic Precioso
Machine Learning
Stage-specific cancer survival prediction enriched by explainable machine learning
Parisa Poorhasani and Bogdan Iancu
Despite the fact that cancer survivability rates vary greatly between stages, traditional survival prediction models have frequently been trained and assessed using examples from all combined phases of the disease. This method may result in an overestimation of performance and ignore the stage-specific variations. Using the SEER
dataset, we created and verified explainable machine learning (ML) models to predict stage-specific cancer survivability in colorectal, stomach, and liver cancers. ML-based cancer survival analysis has been a long-standing topic in the literature; however, studies involving the explainability and transparency of ML survivability models are limited. Our use of explainability techniques, including SHapley Additive exPlanations (SHAP) and Local Interpretable Model-agnostic Explanations (LIME), enabled us to illustrate significant feature-cancer stage interactions that would have remained hidden in traditional black-box models. We identified how certain demographic and clinical variables influenced survival differently across cancer stages and types. These insights provide not only transparency but also clinical relevance, supporting personalized treatment planning. By focusing on stage-specific models, this study provides new insights into the most important factors at each stage of cancer, offering transparency and potential clinical relevance to support personalized treatment planning.
Read the full abstract ▾
Machine Learning · Cross-listed from math.OC
Provably Finding a Hidden Dense Submatrix among Many Planted Dense Submatrices via Convex Programming
Valentine Olanubi, Phineas Agar and Brendan Ames
We consider the densest submatrix problem, which seeks the submatrix of fixed size of a given binary
matrix that contains the most nonzero entries. This problem is a natural generalization of fundamental problems in combinatorial optimization, e.g., the densest subgraph, maximum clique, and maximum edge biclique problems, and has wide application the study of complex networks. Much recent research has focused on the development of sufficient conditions for exact solution of the densest submatrix problem via convex relaxation. The vast majority of these sufficient conditions establish identification of the densest submatrix within a graph containing exactly one large dense submatrix hidden by noise. The assumptions of these underlying models are not observed in real-world networks, where the data may correspond to a
matrix containing many dense submatrices of varying sizes. We extend and generalize these results to the more realistic setting where the input
matrix may contain many large dense subgraphs. Specifically, we establish sufficient conditions under which we can expect to solve the densest submatrix problem in
polynomial time for random input
matrices sampled from a generalization of the
stochastic block model. Moreover, we also provide sufficient conditions for perfect recovery under a deterministic
adversarial. Numerical experiments involving randomly generated problem instances and real-world collaboration and communication networks are used empirically to verify the theoretical phase-transitions to perfect recovery given by these sufficient conditions.
Read the full abstract ▾
Machine Learning
FOREVER: Forgetting Curve-Inspired Memory Replay for Language Model Continual Learning
Yujie Feng, Hao Wang, Jian Li et al. (+6 more)
Continual learning (CL) for large
language models (LLMs) aims to enable sequential knowledge acquisition without catastrophic forgetting.
Memory replay methods are widely used for their practicality and effectiveness, but most rely on fixed, step-based heuristics that often misalign with the model's actual learning progress, since identical training steps can result in varying degrees of parameter change. Motivated by recent findings that LLM forgetting mirrors the Ebbinghaus human forgetting curve, we propose FOREVER (FORgEtting curVe-inspired
mEmory Replay), a novel CL framework that aligns replay schedules with a model-centric notion of time. FOREVER defines model time using the magnitude of optimizer updates, allowing forgetting curve-inspired replay intervals to align with the model's internal
evolution rather than raw training steps. Building on this approach, FOREVER incorporates a forgetting curve-based replay scheduler to determine when to replay and an intensity-aware regularization mechanism to adaptively control how to replay. Extensive experiments on three CL
benchmarks and models ranging from 0.6B to 13B parameters demonstrate that FOREVER consistently mitigates catastrophic forgetting.
Read the full abstract ▾
All authors Yujie Feng, Hao Wang, Jian Li, Xu Chu, Zhaolu Kang, Yiran Liu, Yasha Wang, Philip S. Yu, Xiao-Ming Wu
Machine Learning · Cross-listed from q-bio.PE
Bayes-PD: Exploring a Sequence to Binding Bayesian Neural Network model trained on Phage Display data
Ilann Amiaud-Plachy, Michael Blank, Oliver Bent et al. (+1 more)
Phage display is a powerful laboratory technique used to study the interactions between
proteins and other molecules, whether other
proteins, peptides, DNA or RNA. The under-utilisation of this data in conjunction with deep learning models for
protein design may be attributed to; high experimental noise levels; the complex nature of data pre-processing; and difficulty interpreting these experimental results. In this work, we propose a novel approach utilising a
Bayesian Neural Network within a training loop, in order to simulate the phage display experiment and its associated noise. Our goal is to investigate how understanding the experimental noise and model uncertainty can enable the reliable application of such models to reliably interpret phage display experiments. We validate our approach using actual binding affinity measurements instead of relying solely on proxy values derived from 'held-out' phage display rounds.
Read the full abstract ▾
All authors Ilann Amiaud-Plachy, Michael Blank, Oliver Bent, Sebastien Boyer
Machine Learning
A Gap Between Decision Trees and Neural Networks
Akash Kumar
We study when
geometric simplicity of decision boundaries, used here as a notion of interpretability, can conflict with accurate approximation of axis-aligned decision trees by shallow neural networks. Decision trees induce rule-based, axis-aligned decision regions (finite unions of boxes), whereas shallow ReLU networks are typically trained as score models whose predictions are obtained by thresholding. We analyze the infinite-width, bounded-norm, single-hidden-layer ReLU class through the Radon total variation ($\mathrm{R}\mathrm{TV}$) seminorm, which controls the
geometric complexity of level sets. We first show that the hard tree indicator $1_A$ has infinite $\mathrm{R}\mathrm{TV}$. Moreover, two natural split-wise continuous surrogates–piecewise-linear ramp smoothing and sigmoidal (logistic) smoothing–also have infinite $\mathrm{R}\mathrm{TV}$ in dimensions $d>1$, while Gaussian convolution yields finite $\mathrm{R}\mathrm{TV}$ but with an explicit exponential dependence on $d$. We then separate two goals that are often conflated: classification after thresholding (recovering the decision set) versus score learning (learning a calibrated score close to $1_A$). For classification, we construct a smooth barrier score $S_A$ with finite $\mathrm{R}\mathrm{TV}$ whose fixed threshold $\tau=1$ exactly recovers the box. Under a mild tube-mass condition near $\partial A$, we prove an $L_1(P)$ calibration bound that decays polynomially in a sharpness parameter, along with an explicit $\mathrm{R}\mathrm{TV}$ upper bound in terms of face measures. Experiments on synthetic unions of rectangles illustrate the resulting accuracy–complexity tradeoff and how threshold selection shifts where training lands along it.
Read the full abstract ▾
Machine Learning · Cross-listed from math.RT
An Algebraic Representation Theorem for Linear GENEOs in Machine Learning
Francesco Conti, Patrizio Frosini and Nicola Quercioli
Geometric and Topological Deep Learning enhance machine learning by exploiting
geometric and topological structures in data. In this context, Group Equivariant Non-Expansive Operators (GENEOs) provide a principled framework for encoding
symmetries and designing efficient, interpretable neural architectures. Originally introduced in Topological Data Analysis and later adopted in Deep Learning, GENEOs unify
geometric and topological approaches and include, as a special case, the operator computing persistence diagrams. Their theoretical foundations rely on group actions, equivariance, and compactness properties of operator spaces. GENEOs are defined between data spaces endowed with transformation groups, referred to as perception pairs. While previous representation results characterized linear GENEOs acting on identical perception pairs, many applications require operators between heterogeneous ones. In this work, we introduce and prove a new algebraic representation
theorem for linear GENEOs acting between different perception pairs. Under mild assumptions on data domains and group actions, this result provides a complete characterization of such operators and implicitly yields a computational method to generate all linear GENEOs between two perception pairs. The proof combines tools from the theory of
stochastic matrices, group actions, and invariant measures. In our paper, we also show that the space of linear GENEOs is compact and convex, making easier their use for learning and optimization. Our new
theorem enlarges the applicability of GENEOs, and allows for their composition with a broader class of current architectures. We show this with an application in image reconstruction.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
Current Agents Fail to Leverage World Model as Tool for Foresight
Cheng Qian, Emre Can Acikgoz, Bingxuan Li et al. (+8 more)
Agents built on
vision-
language models increasingly face tasks that demand anticipating future states rather than relying on short-horizon
reasoning. Generative world models offer a promising remedy:
agents could use them as external simulators to foresee outcomes before acting. This paper empirically examines whether current
agents can leverage such world models as tools to enhance their cognition. Across diverse agentic and visual question answering tasks, we observe that some
agents rarely invoke simulation (fewer than 1%), frequently misuse predicted rollouts (approximately 15%), and often exhibit inconsistent or even degraded performance (up to 5%) when simulation is available or enforced. Attribution analysis further indicates that the primary bottleneck lies in the
agents' capacity to decide when to simulate, how to interpret predicted outcomes, and how to integrate foresight into downstream
reasoning. These findings underscore the need for mechanisms that foster calibrated, strategic interaction with world models, paving the way toward more reliable anticipatory cognition in future
agent systems.
Read the full abstract ▾
All authors Cheng Qian, Emre Can Acikgoz, Bingxuan Li, Xiusi Chen, Yuji Zhang, Bingxiang He, Qinyu Luo, Dilek Hakkani-Tür, Gokhan Tur, Yunzhu Li, Heng Ji
Machine Learning
All-Quadrant Bounded Clipping GRPO: Closing the Unbounded Blind Spot for Stable and Generalizable Training
Chi Liu and Xin Chen
Group Relative Policy Optimization (GRPO) has emerged as a popular
algorithm for reinforcement learning with large
language models (LLMs). However, GRPO inherits PPO's
token-level clipping while replacing
token-level advantages with a single sequence-level advantage. Through a four-quadrant analysis of the (likelihood-ratio, advantage) space, we show that this combination leaves one quadrant – negative advantage combined with an increased likelihood ratio (Q4) – structurally unbounded, so that a few high-ratio
tokens can receive very large suppressive updates that collapse
entropy and narrow the
reasoning boundary. To address this, we propose All-Quadrant Bounded Clipping GRPO (ABC-GRPO), which applies unconditional clipping in all four quadrants through sign-dependent boundaries. ABC-GRPO clips the likelihood ratio before multiplying by the advantage, adding a trust-region floor in Q2 and a cap in Q4 – its negative-advantage branch coinciding with dual-clip PPO – to yield bounded per-step policy displacement in every quadrant. On mathematical
reasoning with Qwen3 base models, ABC-GRPO attains the highest Avg@64 and Pass@64: it is statistically superior to GRPO, SAPO, and dual-clip PPO and competitive with the strongest baseline (DAPO), while maintaining substantially higher
entropy; the gains transfer to MATH-500 and to out-of-domain code (HumanEval). Ablations isolate Q4 as the dominant blind spot.
Read the full abstract ▾