Machine Learning · Cross-listed from cs.AI
Hierarchical Causal Abduction: A Foundation Framework for Explainable Model Predictive Control
Ramesh Arvind Naagarajan, Zühal Wagner and Stefan Streif
Model Predictive Control (MPC) is widely used to operate safety-critical infrastructure by predicting future trajectories and optimizing control actions. However, nonlinear dynamics, hard safety constraints, and numerical optimization often render individual control moves opaque to human operators, undermining trust and hindering deployment. This paper presents Hierarchical Causal Abduction (HCA), which combines (i) physics-informed
reasoning via domain knowledge graphs, (ii) optimization evidence from Karush–Kuhn–Tucker (KKT) multipliers, and (iii) temporal causal discovery via the PCMCI
algorithm to generate faithful, human-interpretable explanations for control actions computed by nonlinear MPC. Across three diverse control applications (greenhouse
climate, building HVAC, chemical process engineering) with expert validation, HCA improves explanation accuracy by 53% over LIME (0.478 vs. 0.311) using a single set of cross-domain parameters without per-domain tuning; domain-specific KKT-threshold calibration over 2–3 days further increases accuracy to 0.88. Ablation studies confirm that each evidence source is essential, with 32–37% accuracy degradation when any component is removed, and HCA's ranking-and-validation methodology generalizes beyond MPC to other prediction-based decision systems, including learning-based control and trajectory planning.
Read the full abstract ▾
Machine Learning
Hierarchical End-to-End Taylor Bounds for Complete Neural Network Verification
Taha Entesari and Mahyar Fazlyab
Reachability analysis of neural networks, which seeks to compute or bound the set of outputs attainable over a given input domain, is central to certifying safety and robustness in learning-enabled physical systems. Since exact reachable set computation is generally intractable, existing methods typically rely on tractable overapproximations. Examining the state of the art for smooth, twice-differentiable networks, we observe that existing approaches exploit at most second-order information and do not systematically leverage higher-order information. In this work, we introduce HiTaB, a novel verification framework that exploits second-order smoothness through both the Hessian, $\nabla^2 f$, and its Lipschitz constant, $L_{\nabla^2 f}$. We further develop a unified hierarchy of zeroth-, first-, and second-order bounds, together with precise conditions under which higher-order approximations yield provable improvements. Our main technical contribution is a compositional procedure for efficiently bounding $L_{\nabla^2 f}$ in deep neural networks via layerwise propagation of
curvature bounds. We extend the framework to both $\ell_2$- and $\ell_\infty$-constrained input sets and show how it can be integrated into branch-and-bound verification pipelines. To our knowledge, this is the first practical reachability analysis framework for smooth neural networks that systematically exploits Lipschitz continuity of
curvature, leading to tighter and more informative safety certificates.
Read the full abstract ▾
Machine Learning
MulTaBench: Benchmarking Multimodal Tabular Learning with Text and Image
Alan Arazi, Eilam Shapira, Shoham Grunblat et al. (+8 more)
Tabular Foundation Models have recently established the state of the art in supervised tabular learning, by leveraging pretraining to learn generalizable representations of numerical and categorical structured data. However, they lack native support for unstructured modalities such as text and image, and rely on frozen, pretrained
embeddings to process them. On established
Multimodal Tabular Learning
benchmarks, we show that tuning the
embeddings to the task improves performance. Existing
benchmarks, however, often focus on the mere co-occurrence of modalities; this leads to high variance across
datasets and masks the benefits of task-specific tuning. To address this gap, we introduce MulTaBench, a
benchmark of 40
datasets, split equally between image-tabular and text-tabular tasks. We focus on predictive tasks where the modalities provide complementary predictive signal, and where generic
embeddings lose critical information, necessitating Target-Aware Representations that are aligned with the task. Our experimental results demonstrate that the gains from target-aware representation tuning generalize across both text and image modalities, several tabular learners, encoder scales, and
embedding dimensions. MulTaBench constitutes the largest image-tabular benchmarking effort to date, spanning high-impact domains such as healthcare and e-commerce. It is designed to enable the research of novel architectures which incorporate joint modeling and target-aware representations, paving the way for the development of novel
Multimodal Tabular Foundation Models.
Read the full abstract ▾
All authors Alan Arazi, Eilam Shapira, Shoham Grunblat, Mor Ventura, Elad Hoffer, Gioia Blayer, David Holzmüller, Lennart Purucker, Gaël Varoquaux, Frank Hutter, Roi Reichart
Machine Learning · Cross-listed from cond-mat.dis-nn
Exact Fixed-Point Constraints in Neural-ODEs with Provable Universality
Feliciano Giuseppe Pacifico, Duccio Fanelli, Lorenzo Buffoni et al. (+3 more)
We introduce a technique that enables Neural-ODEs to approximate arbitrary velocity fields with a priori planted fixed-points. Specifically, a recipe is given to explicitly accommodate for a finite collection of points in the reference multi-dimensional space of the Neural-ODE where the velocity field is exactly equal to zero. In this way, the gradient-based training is rigorously constrained inside the prescribed hypothesis class while leaving the expressive power of the Neural-ODE unaltered. We rigorously prove the universality of the Neural-ODE under any local constraints in the velocity field and give a computationally convenient way of imposing the fixed points. Our method is then tested on two paradigmatic physical models.
Read the full abstract ▾
All authors Feliciano Giuseppe Pacifico, Duccio Fanelli, Lorenzo Buffoni, Lorenzo Chicchi, Diego Febbe, Raffaele Marino
Machine Learning · Cross-listed from cs.AR
Reconfigurable Computing Challenge: Real-Time Graph Neural Networks for Online Event Selection in Big Science
Marc Neu, Frank Baptist, Thomas Lobmaier et al. (+3 more)
Graph neural networks are increasingly adopted in trigger systems for
collider experiments, where strict latency and throughput constraints render deployment on embedded platforms challenging. As detectors move towards higher granularity, the number of inputs per inference increase and FPGA-only solutions face resource bottlenecks. This work presents an end-to-end demonstrator for the real-time deployment of a dynamic Graph Neural Network for the Belle II electromagnetic calorimeter hardware trigger on the AMD Versal VCK190, leveraging both FPGA fabric and AI Engine tiles. We develop a Python-based semi-automated design flow covering operator fusion, partitioning, mapping, spatial parallelization, and kernel-level optimization. Our design achieves a throughput of 2.94 million events per second at an end-to-end latency of 7.15 microseconds. Compared to the FPGA-only baseline, this represents a 53% throughput improvement while reducing DSP utilization from 99% to 19% at 29% AI Engine tile utilization. To validate the deployment, an interactive visualization pipeline enables real-time monitoring of inference results on the physical demonstrator.
Read the full abstract ▾
All authors Marc Neu, Frank Baptist, Thomas Lobmaier, Fabio Papagno, Torben Ferber, Jürgen Becker
Machine Learning
Fairness vs Performance: Characterizing the Pareto Frontier of Algorithmic Decision Systems
Mieke Wilms and Christoph Heitz
Designing fair algorithmic decision systems requires balancing model performance with
fairness toward affected individuals: More
fairness might require sacrificing some performance and vice versa, yet the space of possible trade-offs is still poorly understood. We investigate
fairness in binary prediction-based decision problems by conceptualizing decision making as a multi-objective optimization problem that simultaneously considers decision-maker utility and group
fairness. We investigate the set of Pareto-optimal decision rules for arbitrary utility functions for decision maker, arbitrary population distributions, and a wide range of group
fairness metrics. We find that the Pareto frontier consists of deterministic, group-specific threshold rules applied to individuals' success probability. This complements existing optimality
theorems from literature which, for specific
fairness constraints, posit lower-bound threshold rules only. However we also show that, depending on the used
fairness metric, the Pareto frontier may include upper-bound threshold rules, thus preferring individuals with lower success probabilities. We show that the location of the Pareto frontier depends only on population characteristics, utility functions and
fairness score, but not on the technical design of the
algorithm - our findings hold for pre-, in-, and post-processing approaches alike. Our results generalize existing optimality
theorems for
fairness-constrained classification and extend them to generalized
fairness metrics and
fairness principles, and to partial
fairness regimes. This paper connects formal
fairness research with legal and ethical requirements to search for less discriminatory alternatives, offering a principled foundation for evaluating and comparing algorithmic decision systems.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
A Resilient Solution for Sewer Overflow Monitoring across Cloud and Edge
Vipin Singh, Tianheng Ling, Peter Ghaly et al. (+3 more)
Aging combined sewer systems in many historical cities are increasingly stressed by extreme rainfall events, which can trigger combined sewer overflows (CSO) with significant environmental and public health impacts.
Forecasting the filling dynamics of overflow basins is critical for anticipating capacity exceedance and enabling timely preventive actions for CSO. We present a web-based demonstrator that integrates Deep Learning
forecasting methods in both cloud and edge settings into an interactive monitoring dashboard for overflow monitoring, resilient to network outages. A
video showcase is available online (https://cloud.bht-berlin.de/index.php/s/b9xt4T3SdiLBiFZ).
Read the full abstract ▾
All authors Vipin Singh, Tianheng Ling, Peter Ghaly, Felix Grimmeisen, Gregor Schiele, Felix Biessmann
Machine Learning · Cross-listed from stat.ML
Amortizing Causal Sensitivity Analysis via Prior Data-Fitted Networks
Emil Javurek, Dennis Frauen, Marie Brockschmidt et al. (+2 more)
Causal sensitivity analysis aims to provide bounds for causal effect estimates in the presence of unobserved confounding. However, existing methods for causal sensitivity analysis are per-instance procedures, meaning that changes to the
dataset, causal query, sensitivity level, or treatment require new computation. Here, we instead present an in-context learning approach. Specifically, we propose an amortized approach to causal sensitivity analysis based on prior-data fitted networks. A key challenge is that the sensitivity bounds are not directly available when sampling training data. To address this, we develop a general prior-data construction that is applicable across the class of generalized treatment sensitivity models. Our construction involves a Lagrangian scalarization of the objective to generate training labels for the bounds through a tradeoff between causal effect min/max-imization and sensitivity model violation, which avoids model-specific analytical derivations. We further show that, under standard convexity and linearity conditions, our objective recovers the full Pareto frontier of solutions. Empirically, we demonstrate our amortized approach across various
datasets, causal queries, and sensitivity levels, where our approach achieves a test-time computation that is orders of magnitude faster than per-instance methods. To the best of our knowledge, ours is the first foundation model for in-context learning for causal sensitivity analysis.
Read the full abstract ▾
All authors Emil Javurek, Dennis Frauen, Marie Brockschmidt, Jonas Schweisthal, Stefan Feuerriegel
Machine Learning
Controllability in preference-conditioned multi-objective reinforcement learning
Pau de las Heras Molins, Beyazit Yalcinkaya, Lasse Peters et al. (+2 more)
Multi-objective reinforcement learning (MORL) allows a user to express preference over outcomes in terms of the relative importance of the objectives, but standard metrics cannot capture whether changes in preference reliably change the
agent's behavior in the intended way, a property termed controllability. As a result, preference-conditioned
agents can score well on standard MORL metrics while being insensitive to the preference input. If the ability to control
agents cannot be reliably assessed, the symbolic interface that MORL provides between user intent and
agent behavior is broken. Mainstream MORL metrics alone fail to measure the controllability of preference-conditioned
agents, motivating a complementary metric specifically designed to that end. We hope the results spur discussion in the community on existing evaluation protocols to consolidate advances in preference adaptation in MORL to larger and more complex problems.
Read the full abstract ▾
All authors Pau de las Heras Molins, Beyazit Yalcinkaya, Lasse Peters, David Fridovich-Keil, Georgios Bakirtzis
Machine Learning · Cross-listed from cs.CR
Acceptance Cards:A Four-Diagnostic Standard for Safe Fine-Tuning Defense Claims
Phongsakon Mark Konrad, Toygar Tanyel and Serkan Ayvaz
Safe fine-tuning defenses are often endorsed on the basis of a held-out gap reduction, but the same reduction can come from sampling noise, subject artifacts, capability loss, or a mechanism that does not transfer. We introduce Acceptance Cards: an evaluation protocol, a documentation object, an executable audit package, and a claim-specific evidential standard for safe fine-tuning defense claims. The protocol checks statistical reliability, fresh semantic generalization, mechanism
alignment, and cross-task transfer before treating a gap reduction as a full-card pass. Re-scored under this installed-gap protocol, SafeLoRA fails the full-card pass on Gemma-2-2B-it: under strict mechanism-class coding it fails all four diagnostics, and under a permissive shrinkage relabel it still fails three of four. This is a narrow installed-gap audit on one model family, not a global judgment of SafeLoRA's effectiveness. In a 46-
cell audit, no
cell satisfies the strict conjunction. The closest family is a near miss that passes reliability and mechanism checks where the required data are available, but fails the fresh-subject threshold, lacks a strict transfer pass, and carries a measurable deployment-accuracy cost.
Read the full abstract ▾
Machine Learning
Online Sharp-Calibrated Bayesian Optimization
Marshal Arijona Sinaga, Julien Martinelli, Teemu Turpeinen et al. (+1 more)
Bayesian optimization (BO) is a widely used framework for optimizing expensive black-box functions, commonly based on Gaussian process (GP) surrogate models. Its effectiveness relies on uncertainty quantification that is both sharp (informative) and well-calibrated along the BO trajectory. In practice, GP kernel hyperparameters are unknown and are refit online from sequentially collected (non-i.i.d.) data, which can yield miscalibrated or overly conservative uncertainty and lies outside the fixed-kernel assumptions of standard BO regret theory. We propose Online Sharp-Calibrated
Bayesian Optimization (OSCBO), a BO
algorithm that adaptively balances GP sharpness and calibration by casting hyperparameter selection as a constrained online-learning problem. We also show that OSCBO preserves sublinear regret bounds by leveraging the theoretical guarantees of the underlying online learning
algorithm. Empirically, OSCBO performs competitively across synthetic and real-world
benchmarks, ranking among the strongest methods in final simple regret while maintaining robust cumulative-regret behavior.
Read the full abstract ▾
All authors Marshal Arijona Sinaga, Julien Martinelli, Teemu Turpeinen, Samuel Kaski
Machine Learning · Cross-listed from stat.ML
Affine Tracing: A New Paradigm for Probabilistic Linear Solvers
Disha Hegde, Marvin Pförtner and Jon Cockayne
Probabilistic linear solvers (PLSs) return probability distributions that quantify uncertainty due to limited computation in the solution of linear systems. The literature has traditionally distinguished between
Bayesian PLSs, which condition a prior on information obtained from projections of the linear system, and probabilistic iterative methods (PIMs), which lift classical iterative solvers to probability space. In this work we show this dichotomy to be false:
Bayesian PLSs are a special case of non-stationary affine PIMs. In addition, we prove that any realistic affine PIM is calibrated. These results motivate a focus on (non-stationary) affine PIMs, but their practical adoption has been limited by the significant manual effort required to implement them. To address this, we introduce affine tracing, an algorithmic framework that automatically constructs a PIM from a standard implementation of an affine iterative method by passing symbolic tracers through the computation to build an affine computational graph. We show how this graph can be transformed to compute posterior covariances, and how equality saturation can be used to perform algebraic simplifications required for computation under specific prior choices. We demonstrate the framework by automatically generating a probabilistic multigrid solver and evaluate its performance in the context of Gaussian process approximation.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
EnergyLens: Interpretable Closed-Form Energy Models for Multimodal LLM Inference Serving
Vittorio Palladino, Gianluca Palermo, Michael E. Papka et al. (+1 more)
As large
language models span dense, mixture-of-experts, and state-space architectures and are deployed on heterogeneous accelerators under increasingly diverse
multimodal workloads, optimising inference energy has become as critical as optimizing latency and throughput. Existing approaches either treat latency as an energy proxy or rely on data-hungry black-box surrogates. Both fail under varying parallelism strategies: latency and energy optima diverge in over 20% of configurations we tested, and black-box surrogates require hundreds of profiling samples to generalize across model families and hardware. We present EnergyLens, which uses symbolic regression as a structure-discovery tool over profiling data to derive a single twelve-parameter closed-form energy model expressed in terms of system properties such as degree of parallelism, batch size, and sequence length. Unlike black-box surrogates, EnergyLens decouples tensor and pipeline parallelism contributions and separates prefill from decode energy, making its predictions physically interpretable and actionable. Fitted from as few as 50 profiling measurements, EnergyLens achieves 88.2% Top-1 configuration selection accuracy across many evaluation scenarios compared to 60.9% for the closest prior analytical baseline, matches the predictive accuracy of ensemble ML methods with 10x fewer profiling samples, and extrapolates reliably to unseen batch sizes and hardware platforms without structural modification, making it a practical, interpretable tool for energy-optimal LLM deployment.
Read the full abstract ▾
All authors Vittorio Palladino, Gianluca Palermo, Michael E. Papka, Zhiling Lan
Machine Learning
It's All Connected: Topology-Aware Structural Graph Encoding Improves Performance on Polymer Prediction
H. Ibrahim Erdogan, Punith Raviswamy, Nikita Agrawal et al. (+5 more)
Graph Neural Networks (GNNs) have achieved strong results in molecular property prediction, but
polymers present distinct challenges: labeled
datasets are scarce and small (typically in the order of hundreds of
polymers) due to the need for expensive experimentation, and complex
polymer chain distributions influence
polymer properties. Established practice in
polymer prediction represents
polymers solely by graphs of their repeat units, discarding the chain-scale morphology that governs key properties such as the glass transition temperature ($T_g$). In this work, we propose a principled graph construction that addresses this gap. Given a
polymer's molecular mass distribution (MMD), we sample representative chains from the Schulz-Zimm distribution and construct representative sets of large graphs encoding chain-scale
topology directly, with atoms and bonds featurized using rich chemical descriptors. We further pretrain GNN encoders via masked graph modeling on 100,000 unlabeled PSMILES strings before fine-tuning on labeled data. On a
dataset of 381
polymers (180 homopolymers and 201 copolymers), we show that graph construction and self-supervised pretraining are jointly necessary: without pretraining, the large graph method matches the repeat-unit baseline (28.40 K vs. 28.36 K RMSE); with pretraining, it achieves 24.76 K +/- 3.30 K, a 5.1% reduction in mean error over the pretrained repeat-unit baseline (26.08 K +/- 4.20 K, p < 0.001, 30 runs). An ablation removing chemical features degrades performance to 36.65 K, confirming both components are essential. Results are architecture-agnostic, holding for both GINE and GATv2 encoders.
Read the full abstract ▾
All authors H. Ibrahim Erdogan, Punith Raviswamy, Nikita Agrawal, Yannik Köster, Stefan Zechel, Ulrich S. Schubert, Ruben Mayer, Christopher Kuenneth
Machine Learning
PhysEDA: Physics-Aware Learning Framework for Efficient EDA With Manhattan Distance Decay
Zetao Yang
Electronic design automation (EDA) addresses placement, routing, timing analysis, and power-integrity verification for integrated
circuits. Learning methods –
attention (
Transformer) and reinforcement learning (RL) – have recently emerged on EDA tasks, yet face two common bottlenecks: vanilla
attention's quadratic complexity limits scaling, and data-scarce models overfit statistical noise and amplify weak long-range correlations against the underlying physics. We observe that EDA tasks share a physical prior – pairwise electrical and routing interactions decay exponentially along Manhattan distance – and integrate it as a unified inductive bias into both architecture and training. We propose PhysEDA, comprising two components Physics-Structured Linear
Attention (PSLA) folds the separable Manhattan decay into the linear-
attention kernel as a multiplicative bias, reducing complexity from quadratic to linear; Potential-Based Reward Shaping (PBRS) constructs a physical potential from the same kernel, providing dense reward signal under sparse RL while preserving the optimal policy via the policy-invariance
theorem. Across three EDA scenarios – decoupling-capacitor placement, macro placement, and IR-drop prediction – PhysEDA improves zero-shot cross-scale transfer by 56.8% and achieves 14x inference speedup with 98.5%
memory savings on 100x100 grids; PBRS adds another 10.8% in sparse-reward DPP.
Read the full abstract ▾
Machine Learning
Higher Resolution, Better Generalization: Unlocking Visual Scaling in Deep Reinforcement Learning
Raphael Trumpp, Ömer Veysel Çağatan, Bar{\i}ş Akgün et al. (+1 more)
Pixel-based deep reinforcement learning
agents are typically trained on heavily downsampled visual observations, a convention inherited from early
benchmarks rather than grounded in principled design. In this work, we show that observation resolution is a critical yet overlooked variable for policy learning: higher-resolution inputs can substantially improve both performance and generalization, provided the network architecture can process them effectively. We find that the widely used Impala encoder, which flattens spatial features into a vector, suffers from quadratic parameter growth as resolution increases and fails to leverage the additional visual detail. We test different modifications to the Impala architecture and conclude that, in particular, introducing a global average pooling layer, as in the Impoola architecture, yields consistent improvements across resolutions and network widths while decoupling parameter count from resolution - at their respective best conditions, visual scaling unlocks a 28 % performance gain for Impoola over Impala. These gains are strongest in environments that require precise perception of small or distant objects, and gradient saliency analysis suggests that the underlying mechanism is a more spatially localized visual
attention of the policy at higher resolutions. Our results challenge the prevailing practice of aggressive input downsampling and position resolution-independent architectures as a simple, effective path toward scalable visual deep reinforcement learning.
Read the full abstract ▾
All authors Raphael Trumpp, Ömer Veysel Çağatan, Bar{\i}ş Akgün, Marco Caccamo
Machine Learning · Cross-listed from cs.AI
Bridging Sequence and Graph Structure for Epigenetic Age Prediction
Yao Li, Xikun Zhang, Xiaotao Shen et al. (+4 more)
Epigenetic clocks based on DNA methylation have emerged as powerful tools for estimating biological age, with broad applications in aging research, age-related disease studies, and longevity science. Despite advances across machine learning approaches to epigenetic age prediction, spanning penalised linear regression, deep feedforward networks, residual architectures, and graph neural networks, no existing method jointly models co-methylation graph structure and site-specific DNA sequence context within a unified framework. We propose a unified sequence–graph integration framework for epigenetic age prediction that addresses this gap, integrating eight-dimensional DNA sequence statistical features through a lightweight gated modulation mechanism that adaptively scales each site's methylation signal according to its sequence-determined biological relevance prior to graph convolution. Evaluated on 3,707 blood methylation samples against a comprehensive set of baselines, our method achieves a test MAE of 3.149 years, a 12.8% improvement over the strongest graph-based baseline. Biologically informed statistical features outperform CNN-based sequence encoding, demonstrating that handcrafted sequence features are more effective than end-to-end learned representations in this data regime. Post-hoc interpretability analysis identifies CpG density and local adenine frequency as features with age-dependent importance shifts, consistent with known mechanisms of age-related hypermethylation at CpG-dense promoter regions. Our code is at https://github.com/yaoli2022/graphage-seq.
Read the full abstract ▾
All authors Yao Li, Xikun Zhang, Xiaotao Shen, Sonika Tyagi, Xin Zheng, Jiaxing Huang, Feng Xia
Machine Learning
HH-SAE: Discovering and Steering Hierarchical Knowledge of Complex Manifolds
Honghan Wu, Tianyan Wang, Jiacong Mi et al. (+2 more)
Rare semantic innovations in high-dimensional, mission-critical domains are often obscured by dense background contexts, a challenge we define as feature density conflict. We introduce the Hybrid Hierarchical SAE (HH-SAE) to resolve this by factorizing
manifolds into a nested hierarchy of Contextual ($L_0$), Atomic ($f_1$), and Compository ($f_2$) tiers. Evaluating across disparate
manifolds, HH-SAE demonstrates superior resolution by “fracturing” administrative clinical labels into physiological modes and achieving a peak cross-domain zero-shot AUC of 0.9156 in fraud detection. Path ablation confirms the architecture's structural necessity, revealing a 13.46% utility collapse when contextual subtraction is removed. Finally, knowledge-steered synthesis achieves a +9.9% AUPRC lift over state-of-the-art generators, proving that HH-SAE effectively prioritizes high-order mechanistic innovation over environmental proxies to enable high-precision discovery in high-stakes environments.
Read the full abstract ▾
All authors Honghan Wu, Tianyan Wang, Jiacong Mi, Zhoyang Jiang, Yunsoo Kim
Machine Learning
ConfoundingSHAP: Quantifying confounding strength in causal inference
Marie Brockschmidt, Santo M.A.R. Thies, Maresa Schröder et al. (+5 more)
In causal inference, confounders are variables that influence both treatment decisions and outcomes. However, unlike as in randomized clinical trials, the treatment assignment mechanism in observational studies is not known, and it is thus unclear which
covariates act as confounders. Here, we aim to generate insight for causal inference and answer: which of the observed
covariates act as confounders? We introduce ConfoundingSHAP, a Shapley-based method for attributing confounding strength to individual
covariates. Our contributions are twofold. First, we propose a Shapley game targeted to infer the confounding strength of the
covariates. Our resulting Shapley values differ from the standard applications of SHAP explanations on causal targets, such as understanding treatment effect heterogeneity, which are ill-suited for our task. Second, as our task requires evaluating the value function over many adjustment sets, we provide a scalable TabPFN-based estimation that avoids exhaustive refitting. We demonstrate the practical value across various
datasets, where ConfoundingSHAP provides informative explanations of which observed
covariates drive confounding and thereby helps to provide more insight for causal inference in practice.
Read the full abstract ▾
All authors Marie Brockschmidt, Santo M.A.R. Thies, Maresa Schröder, Dennis Frauen, Valentyn Melnychuk, Maximilian Muschalik, Eyke Hüllermeier, Stefan Feuerriegel
Machine Learning · Cross-listed from cs.AI
PrimeKG-CL: A Continual Graph Learning Benchmark on Evolving Biomedical Knowledge Graphs
Yousef A. Radwan, Yao Li, Qing Qing et al. (+5 more)
Biomedical knowledge graphs underwrite drug repurposing and clinical decision support, yet the upstream ontologies they depend on update on independent cycles that add millions of edges and deprecate hundreds of thousands more between releases. Yet existing continual graph learning has been studied almost exclusively on synthetic random splits of static, generic KGs, a regime that cannot reproduce the asynchronous, structured
evolution real biomedical KGs undergo. To this end, we introduce PrimeKG-CL, a CGL
benchmark built from nine authoritative biomedical databases (129K+ nodes, 8.1M+ edges, 10 node types, 30 relation types) with two genuine temporal snapshots (June 2021, July 2023; 5.83M edges added, 889K removed, 7.21M persistent), 10 entity-type-grouped tasks,
multimodal node features, and a per-task persistent/added/removed test stratification. On three tasks (biomedical relationship prediction, entity classification, KGQA), we evaluate six CL strategies across four KGE decoders, plus LKGE, an LLM-RAG
agent, and CMKL. We find that decoder choice and continual learning strategy interact strongly: no single strategy performs best across all decoders, and mismatched combinations can significantly degrade performance. Moreover, only DistMult exhibits a clear separation between persistent and deprecated knowledge, indicating that standard metrics conflate retention of still-valid facts with failure to forget outdated ones; this effect is absent under RotatE. In addition,
multimodal features improve entity-level tasks by up to 60%, and a recent CKGE framework (IncDE) failed to scale to our 5.67M-triple base task across five attempts up to 350GB RAM. Data, pipeline, baselines, and the stratified split are released openly.
Dataset:huggingface.co/
datasets/yradwan147/PrimeKGCL|Code:github.com/yradwan147/primekg-cl-neurips2026
Read the full abstract ▾
All authors Yousef A. Radwan, Yao Li, Qing Qing, Ziqi Xu, Xingtong Yu, Jiaxing Huang, Renqiang Luo, Xikun Zhang
Machine Learning
CMKL: Modality-Aware Continual Learning for Evolving Biomedical Knowledge Graphs
Yousef A. Radwan, Yao Li, Qing Qing et al. (+5 more)
Biomedical knowledge graphs are increasingly large, dynamic, and
multimodal, driven by rapid advances in biotechnology such as high-throughput sequencing. Machine learning models can infer previously unobserved biomedical relationships and characterize biomedical entities in these graphs, but existing knowledge graph
embedding methods and their continual learning extensions either assume static graph structure or fail to exploit
multimodal information under evolving data distributions. They also apply uniform regularization across all model parameters, ignoring that different modalities may exhibit distinct forgetting dynamics as the graph evolves. We propose the Continual
Multimodal Knowledge Graph Learner (CMKL), a CL framework for biomedical KGs that natively encodes structure, text, and molecules, fuses them through a Mixture-of-Experts (MoE) router, and protects previously learned knowledge with standard EWC regularization and a K-means-diverse
multimodal replay buffer. We evaluate CMKL on a 129K-entity biomedical continual
benchmark with 10 tasks. On continual biomedical entity classification, CMKL reaches AP 0.591 versus 0.370 for the strongest structural baseline, a 60% gain that is driven by access to
multimodal features and preserved across the sequence with near-zero forgetting (AF 0.008). On continual relationship prediction, CMKL reaches AP $0.062$, matching Naive Sequential and EWC (0.058) within seed noise and outperforming Joint Training (0.047, p=0.045) and LKGE (0.039). A frozen-text ablation reaches AP 0.136, more than double any jointly trained model, yet that signal is unreachable by margin-ranking gradients: the greedy-modality asymmetry lives at the representation level, not the fusion level, and MoE routing manages it by suppressing the unreachable modality without forcing it through a learned bottleneck. Code: github.com/yradwan147/cmkl-neurips2026
Read the full abstract ▾
All authors Yousef A. Radwan, Yao Li, Qing Qing, Ziqi Xu, Qixin Zhang, Yongcheng Jing, Renqiang Luo, Xikun Zhang
Machine Learning · Cross-listed from eess.SY
Priority-Driven Control and Communication in Decentralized Multi-Agent Systems via Reinforcement Learning
Qingyun Guo, Junyi Shi, Tomasz Piotr Kucner et al. (+1 more)
Event-triggered control provides a mechanism for avoiding excessive use of constrained communication bandwidth in networked multi-
agent systems. However, most existing methods rely on accurate system models, which may be unavailable in practice. In this work, we propose a model-free, priority-driven reinforcement learning
algorithm that learns communication priorities and control policies jointly from data in decentralized multi-
agent systems. By learning communication priorities, we circumvent the hybrid action space typical in event-triggered control with binary communication decisions. We evaluate our
algorithm on
benchmark tasks and demonstrate that it outperforms the baseline method.
Read the full abstract ▾
All authors Qingyun Guo, Junyi Shi, Tomasz Piotr Kucner, Dominik Baumann
Machine Learning · Cross-listed from cs.GT
Regret Minimization in Bilateral Trade With Perturbed Markets
Anna Lunghi, Matteo Castiglioni and Alberto Marchesi
We address the problem of maximizing Gain from Trade (GFT) in repeated buyer-seller exchanges subject to global budget balance constraints. While this problem is well-understood in purely
adversarial and
stochastic settings, these environments exhibit a sharp dichotomy:
adversarial environments allow for no-regret learning against the best fixed-price mechanism, whereas
stochastic environments allow for no-regret learning against the best distribution over prices that is budget balanced in expectation. This gap is significant, as policies balanced in expectation can increase the GFT by a multiplicative factor of two. In this work, we bridge these extremes by studying perturbed
markets, where an underlying
stochastic distribution is subject to an
adversarial corruption $C$. We design an
algorithm that adaptively scales with the level of corruption, achieving an $\tilde{\mathcal{O}}(T^{3/4}) + \mathcal{O}(C\log(T))$ regret bound against the best budget-balanced distribution over prices. Simultaneously, our
algorithm maintains the worst-case $\tilde{\mathcal{O}}(T^{3/4})$ regret bound relative to a per-round budget-balanced baseline, ensuring optimality even in fully
adversarial environments.
Read the full abstract ▾
Machine Learning
Formally Verifying Analog Neural Networks Under Process Variations Using Polynomial Zonotopes
Yasmine Abu-Haeyeh, Tobias Ladner, Matthias Althoff et al. (+1 more)
Analog neural networks are gaining
attention due to their efficiency in terms of power consumption and processing speed. However, since analog neural networks are implemented as physical
circuits, they are highly sensitive to manufacturing process variations, which can cause large deviations from the nominal model. We present a
polynomial-based model that resembles the performance of the
neuron circuit under process variations. This model is formally verified via reachability analysis using
polynomial zonotopes, thus avoiding conventional, time-consuming Monte Carlo simulations. We evaluate our proposed verification approach on three different
datasets and on fully-connected and convolutional analog neural networks. Our experimental results confirm the effectiveness of our verification approach by reducing the verification time from up to a day to seconds while enclosing up to 99% of the variation samples.
Read the full abstract ▾
All authors Yasmine Abu-Haeyeh, Tobias Ladner, Matthias Althoff, Lars Hedrich
Machine Learning
Can Muon Fine-tune Adam-Pretrained Models?
Xingyu Qu, Peigeng Huang and Samuel Horvath
Muon has emerged as an efficient alternative to Adam for pretraining, yet remains underused for fine-tuning. A key obstacle is that most open models are pretrained with Adam, and naively switching to
Muon for fine-tuning leads to degraded performance due to an optimizer mismatch. We investigate this mismatch through controlled experiments and relate it to the distinct implicit biases of Adam and
Muon. We provide evidence that the mismatch disrupts pretrained knowledge, and that this disruption scales with update strength. This leads us to hypothesize that constraining updates should mitigate the mismatch. We validate this with LoRA: across
language and
vision tasks, LoRA reduces the performance gap between Adam and
Muon observed under full fine-tuning. Studies on LoRA rank, catastrophic forgetting, and LoRA variants further confirm that mismatch severity correlates with update strength. These results shed light on how optimizer mismatch affects fine-tuning and how it can be mitigated. Our code is available at https://github.com/XingyuQu/
muon-finetune.
Read the full abstract ▾
Machine Learning
Self-Attention as a Covariance Readout: A Unified View of In-Context Learning and Repetition
Haoren Xu and Guanhua Fang
Large
language models (LLMs) exhibit two striking and ostensibly unrelated behaviours: in-context learning (ICL) and repetitive generation. In both, the model behaves as though it had summarised the context into a population-level statistic and discarded
token-level detail. We ask whether this “summarisation and forgetting” can be derived from the
attention mechanism itself, and answer in the affirmative. Under stationary, ergodic and elliptical inputs, the softmax
attention output converges almost surely to $\Theta_V\Sigma\Theta_K^{\top}\Theta_Q x_t$, where $\Sigma$ is the input covariance; the long-context limit is therefore a linear readout of the input's second-order statistics. Two consequences follow. (i) For in-context linear regression, a single softmax head can implement one step of population gradient descent. Stacking such heads with residual connections iterates this update and implements multiple gradient descent steps. (ii) Propagated across an $L$-layer
transformer, this readout drives the terminal hidden state at the parametric $1/t$ rate to a deterministic function of the current
token alone, so that autoregressive generation collapses asymptotically to a first-order Markov chain whose attracting orbits furnish a structural account of repetition and mode collapse. The two phenomena thus emerge as facets of a single covariance-readout principle.
Read the full abstract ▾
Machine Learning
QT-Net: Rethinking Evaluation of AI Models in Atomic Chemical Space
Pablo Martínez Crespo, Stefano Ribes, Richard Beckmann et al. (+6 more)
Atomic properties such as partial charges or multipoles encode chemically meaningful information that can inform downstream molecular property prediction, but their evaluation as machine learning targets has been complicated by the absence of a principled out-of-distribution evaluation protocol at the atomic level. In this work, we propose a held-out evaluation protocol that clusters atomic environments by SOAP descriptors and computes metrics accounting only for cluster labels unseen during training. Following this procedure, we use 5$\times$5 cross-validation and Tukey's HSD to run a statistically rigorous comparison of E(3)-equivariant against non-equivariant, rotationally augmented models for predicting electron populations and multipoles of H, C, N, and O atoms. Building on our results, we introduce the
Quantum Topological Neural Network (QT-Net), a rotationally augmented, non-equivariant graph neural network. We show that QT-Net can be used to infer properties of atoms in molecules from QM9 outside our training set, and that the atomic environments seen during training strongly affect the performance of molecular property prediction tasks. To further validate the framework, molecular dipole moments computed from QT-Net's per-atom outputs recover the ground-truth values reported in QM9. We release all code and data, including a JAX implementation of QT-Net.
Read the full abstract ▾
All authors Pablo Martínez Crespo, Stefano Ribes, Richard Beckmann, Marisa Gliege, Santiago Miret, Robert S. Jordan, Vijay Kris Narasimhan, Martin Rahm, Rocío Mercado
Machine Learning
AxiomOcean: Forecasting the Three-Dimensional Structure of the Upper Ocean
Sensen Wu, Yifan Chen, Guantao Pu et al. (+9 more)
Short-term ocean forecast skill depends strongly on the three-dimensional ocean structure of the upper ocean, which governs stratification, subsurface heat storage, and the response of the ocean to atmospheric forcing. However, AI ocean
forecasting models often fail to preserve this vertical structure, resulting in over-smoothed subsurface features and weak physical consistency under strong forcing. Here, we present AxiomOcean, a global AI ocean
forecasting model that explicitly represents vertical hierarchy and cross-layer dependence within the water column. By combining a fully three-dimensional encoder-backbone-decoder architecture with surface atmospheric forcing, AxiomOcean jointly predicts upper-ocean temperature, salinity, and three-dimensional currents at global 1/12{\deg} resolution down to 643 m depth. In 10-day forecasts, AxiomOcean outperforms an advanced AI comparison model across variables and lead times, reducing day-1 RMSE by approximately 20 to 35% while maintaining higher anomaly correlation. The gain is not achieved through excessive smoothing: AxiomOcean better preserves eddy kinetic energy, temperature and salinity variance. Its advantage also extends through the water column and remains evident across the equatorial Pacific, Kuroshio Extension, and Southern Ocean, yielding a more realistic reconstruction of upper-ocean heat content. These results show that explicitly preserving upper-ocean three-dimensional structure can improve both forecast accuracy and physical fidelity in AI ocean prediction.
Read the full abstract ▾
All authors Sensen Wu, Yifan Chen, Guantao Pu, Xiaoyao Sun, Yijun Chen, Jin Qi, Ming Kong, Keyi Yang, Lichen Xu, Wenguan Wang, Xiaofeng Li, Zhenhong Du
Machine Learning
SlimSpec: Low-Rank Draft LM-Head for Accelerated Speculative Decoding
Anton Plaksin, Sergei Krutikov, Sergei Skvortsov et al. (+1 more)
Speculative decoding speeds up autoregressive generation in Large
Language Models (LLMs) through a two-step procedure, where a lightweight draft model proposes
tokens which the target model then verifies in a single forward pass. Although the drafter network is small in modern architectures, its LM-head still performs projection to a large vocabulary, becoming one of the major computational bottlenecks. In prior work this issue has been predominantly addressed via static or dynamic vocabulary truncation. Yet mitigating the bottleneck, these methods bring in extra complexity, such as special vocabulary curation, sophisticated inference-time logic or modifications of the training setup. In this paper, we propose SlimSpec, a low-rank parameterization of the drafter's LM-head that compresses the inner representation rather than the output, preserving full vocabulary support. We evaluate our method with EAGLE-3 drafter across three target models and diverse
benchmarks in both latency- and throughput-bound inference regimes. SlimSpec achieves $4\text{-}5\times$ acceleration over the standard LM-head architecture while maintaining a competitive acceptance length, surpassing existing methods by up to $8\text{-}9\%$ of the end-to-end speedup. Our method requires minimal adjustments of training and inference pipelines. Combined with the aforementioned speedup improvements, it makes SlimSpec a strong alternative across wide variety of draft LM-head architectures.
Read the full abstract ▾
All authors Anton Plaksin, Sergei Krutikov, Sergei Skvortsov, Alexander Samarin
Machine Learning
Don't Fix the Basis – Learn It: Spectral Representation with Adaptive Basis Learning for PDEs
Xuxiang Zhao and Angelica I. Aviles-Rivero
Spectral neural operators achieve strong performance for PDE learning, but rely on fixed global bases that limit their ability to represent spatially heterogeneous and multiscale dynamics. We propose Adaptive Basis Learning (ABLE), a framework that learns data-dependent spectral representations instead of relying on predefined bases. ABLE constructs a spatially adaptive Parseval frame via a learned ancillary density, enabling the operator to act in a lifted spectral space while preserving invertibility and maintaining $O(N\log N)$ complexity through FFT-based implementation. This shifts the source of expressivity from spectral coefficients to the representation itself, allowing the model to capture localized structures and non-translation-invariant interactions more efficiently. ABLE integrates seamlessly into existing neural operator architectures as a drop-in replacement for spectral layers. Across a range of
benchmarks ABLE improves accuracy over strong baselines, with the largest gains in regimes characterized by sharp gradients and multiscale behavior. Moreover, augmenting existing models (e.g., U-FNO, HPM) with ABLE further enhances their performance, demonstrating its role as a general and complementary spectral refinement. Our results highlight that the data-driven choice of representation, rather than operator complexity alone, is a key bottleneck in neural operator design. By learning the basis itself, ABLE provides a principled and efficient framework for improving spectral methods in PDE learning.
Read the full abstract ▾