Machine Learning
StormNet: Improving storm surge predictions with a GNN-based spatio-temporal offset forecasting model
Noujoud Nader, Stefanos Giaremis, Clint Dawson et al. (+3 more)
Storm surge
forecasting remains a critical challenge in mitigating the impacts of tropical cyclones on coastal regions, particularly given recent trends of rapid intensification and increasing nearshore storm activity. Traditional high fidelity numerical models such as ADCIRC, while robust, are often hindered by inevitable uncertainties arising from various sources. To address these challenges, this study introduces StormNet, a spatio-temporal graph neural network (GNN) designed for bias correction of storm surge forecasts. StormNet integrates graph convolutional (GCN) and graph
attention (GAT) mechanisms with long short-term
memory (LSTM) components to capture complex spatial and temporal dependencies among water-level gauge stations. The model was trained using historical hurricane data from the U.S. Gulf Coast and evaluated on Hurricane Idalia (2023). Results demonstrate that StormNet can effectively reduce the root mean square error (RMSE) in water-level predictions by more than 70% for 48-hour forecasts and above 50% for 72-hour forecasts, as well as outperform a sequential LSTM baseline, particularly for longer prediction horizons. The model also exhibits low training time, enhancing its applicability in real-time operational
forecasting systems. Overall, StormNet provides a computationally efficient and physically meaningful framework for improving storm surge prediction accuracy and reliability during extreme weather events.
Read the full abstract ▾
All authors Noujoud Nader, Stefanos Giaremis, Clint Dawson, Carola Kaiser, Karame Mohammadiporshokooh, Hartmut Kaiser
Machine Learning
MGDA-Decoupled: Geometry-Aware Multi-Objective Optimisation for DPO-based LLM Alignment
Andor Vári-Kakas, Ji Won Park and Natasa Tagasovska
Aligning large
language models (LLMs) to desirable human values requires balancing multiple, potentially conflicting objectives such as helpfulness, truthfulness, and harmlessness, which presents a multi-objective optimisation challenge. Most
alignment pipelines rely on a fixed scalarisation of these objectives, which can introduce procedural unfairness by systematically under-weighting harder-to-optimise or minority objectives. To promote more equitable trade-offs, we introduce MGDA-Decoupled, a geometry-based multi-objective optimisation
algorithm that finds a shared descent direction while explicitly accounting for each objective's convergence dynamics. In contrast to prior methods that depend on reinforcement learning (e.g., GAPO) or explicit reward models (e.g., MODPO), our approach operates entirely within the lightweight Direct Preference Optimisation (DPO) paradigm. Experiments on the UltraFeedback
dataset show that geometry-aware methods – and MGDA-Decoupled in particular – achieve the highest win rates against golden responses, both overall and per objective.
Read the full abstract ▾
Machine Learning
Variance Is Not Importance: Structural Analysis of Transformer Compressibility Across Model Scales
Samuel Salfati
We present a systematic empirical study of
transformer compression through over 40 experiments on GPT-2 (124M parameters) and Mistral 7B (7.24B parameters). Our analysis covers spectral compression, block-level function replacement, rotation-based quantization, activation geometry, and adaptive early exit. We identify five structural properties relevant to compression. (1) Variance is not importance: high-variance activation directions are approximately 96 percent uncorrelated with predictive directions (measured via CCA), and projecting onto these subspaces preserves over 90 percent of variance while degrading perplexity. (2) Block linearity is conditional:
transformer blocks are approximately linear (R^2 ~ 0.95 on GPT-2, 0.93 on Mistral block 31) only under the correct upstream distribution; modifying earlier blocks induces distribution shift that degrades downstream approximations. (3) The reconstruction wall: approaches that factor weights into quantized components amplify errors through cross-terms, making direct quantization strictly superior. (4) Linearity increases with depth: Mistral 7B exhibits a progression from R^2 = 0.17 (block 0) to R^2 = 0.93 (block 31), indicating a division between nonlinear feature construction and linear refinement. (5) Approximately 30 percent of
tokens are computationally easy, confirmed via exit heads and KL divergence sensitivity. We demonstrate that single-block linear replacement achieves 34x compression with a 1.71 perplexity increase on the final block of Mistral 7B, while multi-block replacement fails due to residual error accumulation and distribution shift. These findings suggest fundamental limits to static post-training compression and motivate adaptive, per-
token computation as a more effective direction.
Read the full abstract ▾
Machine Learning
Improving clinical interpretability of linear neuroimaging models through feature whitening
Sara Petiton, Antoine Grigis, Raphaël Vock et al. (+1 more)
Linear models are widely used in computational neuroimaging to identify biomarkers associated with
brain pathologies. However, interpreting the learned weights remains challenging, as they do not always yield clinically meaningful insights. This difficulty arises in part from the inherent correlation between
brain regions, which causes linear weights to reflect shared rather than region-specific contributions. In particular, some groups of regions, including homologous structures in the left and right hemispheres, are known to exhibit strong anatomical correlations. In this work, we leverage this prior neuroanatomical knowledge to introduce a whitening approach applied to groups of regions with known shared variance, designed to disentangle overlapping information across correlated
brain measures. We additionally propose a regularized variant that allows controlled tuning of the degree of decorrelation. We evaluate this method using region-of-interest features in two psychiatric classification tasks, distinguishing individuals with bipolar disorder or schizophrenia from healthy controls. Importantly, unlike PCA or ICA which use whitening as a dimensionality reduction step, our approach decorrelates anatomically informed pairs of neuroanatomical regions while retaining the full input signal, making it specifically suited for feature interpretation rather than feature selection. Our findings demonstrate that whitening improves the interpretability of model weights while preserving predictive performance, providing a robust framework for linking linear model outputs to neurobiological mechanisms.
Read the full abstract ▾
All authors Sara Petiton, Antoine Grigis, Raphaël Vock, Edouard Duchesnay
Machine Learning
GRPO-VPS: Enhancing Group Relative Policy Optimization with Verifiable Process Supervision for Effective Reasoning
Jingyi Wang, Lei Zhu, Tengjin Weng et al. (+8 more)
Reinforcement Learning with Verifiable Rewards (RLVR) has advanced the
reasoning capabilities of Large
Language Models (LLMs) by leveraging direct outcome verification instead of learned reward models. Building on this paradigm, Group Relative Policy Optimization (GRPO) eliminates the need for critic models but suffers from indiscriminate credit assignment for intermediate steps, which limits its ability to identify effective
reasoning strategies and incurs overthinking. In this work, we introduce a model-free and verifiable process supervision via probing the model's belief in the correct answer throughout its
reasoning trajectory. By segmenting the generation into discrete steps and tracking the conditional probability of the correct answer appended at each segment boundary, we efficiently compute interpretable segment-wise progress measurements to refine GRPO's trajectory-level feedback. This approach enables more targeted and sample-efficient policy updates, while avoiding the need for intermediate supervision derived from costly Monte Carlo rollouts or auxiliary models. Experiments on mathematical and general-domain
benchmarks show consistent gains over GRPO across diverse models: up to 2.6-point accuracy improvements and 13.7%
reasoning-length reductions on math tasks, and up to 2.4 points and 4% on general-domain tasks, demonstrating strong generalization.
Read the full abstract ▾
All authors Jingyi Wang, Lei Zhu, Tengjin Weng, Song-Li Wu, Haochen Tan, Jierun Chen, Chaofan Tao, Haoli Bai, Lu Hou, Lifeng Shang, Xiao-Ping Zhang
Machine Learning · Cross-listed from math.MG
A weighted angle distance on strings
Grant Molnar
We define a multi-scale metric $d_\rho$ on strings by aggregating angle distances between all $n$-gram count vectors with exponential weights $\rho^n$. We
benchmark $d_\rho$ in DBSCAN clustering against edit and $n$-gram baselines, give a linear-time suffix-tree
algorithm for evaluation, prove metric and stability properties (including robustness under tandem-repeat stutters), and characterize isometries.
Read the full abstract ▾
Machine Learning
Occupancy Reward Shaping: Improving Credit Assignment for Offline Goal-Conditioned Reinforcement Learning
Aravind Venugopal, Jiayu Chen, Xudong Wu et al. (+3 more)
The temporal lag between actions and their long-term consequences makes credit assignment a challenge when learning goal-directed behaviors from data. Generative world models capture the distribution of future states an
agent may visit, indicating that they have captured temporal information. How can that temporal information be extracted to perform credit assignment? In this paper, we formalize how the temporal information stored in world models encodes the underlying geometry of the world. Leveraging optimal transport, we extract this geometry from a learned model of the occupancy measure into a reward function that captures goal-reaching information. Our resulting method, Occupancy Reward Shaping, largely mitigates the problem of credit assignment in sparse reward settings. ORS provably does not alter the optimal policy, yet empirically improves performance by 2.2x across 13 diverse long-horizon locomotion and manipulation tasks. Moreover, we demonstrate the effectiveness of ORS in the real world for controlling
nuclear fusion on 3 Tokamak control tasks. Code: https://github.com/aravindvenu7/occupancy_reward_shaping; Website: https://aravindvenu7.github.io/website/ors/
Read the full abstract ▾
All authors Aravind Venugopal, Jiayu Chen, Xudong Wu, Chongyi Zheng, Benjamin Eysenbach, Jeff Schneider
Machine Learning · Cross-listed from cs.AI
pAI/MSc: ML Theory Research with Humans on the Loop
Mahmoud Abdelmoneum, Pierfrancesco Beneventano and Tomaso Poggio
We present pAI/MSc, an open-source, customizable, modular multi-
agent system for academic research workflows. Our goal is not autonomous scientific ideation, nor fully automated research. It is narrower and more practical: to reduce by orders of magnitude the human steering required to turn a specified hypothesis into a literature-grounded, mathematically established, experimentally supported, submission-oriented manuscript draft. pAI/MSc is built with a current emphasis on machine learning theory and adjacent quantitative fields.
Read the full abstract ▾
Machine Learning
Too Sharp, Too Sure: When Calibration Follows Curvature
Alessandro Morosini, Matea Gjika, Tomaso Poggio et al. (+1 more)
Modern neural networks can achieve high accuracy while remaining poorly calibrated, producing confidence estimates that do not match empirical correctness. Yet calibration is often treated as a post-hoc attribute. We take a different perspective: we study calibration as a training-time phenomenon on small
vision tasks, and ask whether calibrated solutions can be obtained reliably by intervening on the training procedure. We identify a tight coupling between calibration,
curvature, and margins during training of deep networks under multiple gradient-based methods. Empirically, Expected Calibration Error (ECE) closely tracks
curvature-based sharpness throughout optimization. Mathematically, we show that both ECE and Gauss–Newton
curvature are controlled, up to problem-specific constants, by the same margin-dependent exponential tail functional along the trajectory. Guided by this mechanism, we introduce a margin-aware training objective that explicitly targets robust-margin tails and local smoothness, yielding improved out-of-sample calibration across optimizers without sacrificing accuracy.
Read the full abstract ▾
All authors Alessandro Morosini, Matea Gjika, Tomaso Poggio, Pierfrancesco Beneventano
Machine Learning · Cross-listed from cs.IR
Self-Aware Vector Embeddings for Retrieval-Augmented Generation: A Neuroscience-Inspired Framework for Temporal, Confidence-Weighted, and Relational Knowledge
Naizhong Xu
Modern
retrieval-augmented generation (RAG) systems treat vector
embeddings as static, context-free artifacts: an
embedding has no notion of when it was created, how trustworthy its source is, or which other
embeddings depend on it. This flattening of knowledge has a measurable cost: recent work on VersionRAG reports that conventional RAG achieves only 58% accuracy on versioned technical queries, because
retrieval returns semantically similar but temporally invalid content. We propose SmartVector, a framework that augments dense
embeddings with three explicit properties – temporal awareness, confidence decay, and relational awareness – and a five-stage lifecycle modeled on hippocampal-neocortical
memory consolidation. A
retrieval pipeline replaces pure cosine similarity with a four-signal score that mixes semantic relevance, temporal validity, live confidence, and graph-relational importance. A background consolidation
agent detects contradictions, builds dependency edges, and propagates updates along those edges as graph-neural-network-style messages. Confidence is governed by a closed-form function combining an Ebbinghaus-style exponential decay, user-feedback reconsolidation, and logarithmic access reinforcement. We formalize the model, relate it to temporal knowledge graph
embedding, agentic
memory architectures, and uncertainty-aware RAG, and present a reference implementation. On a reproducible synthetic versioned-policy
benchmark of 258 vectors and 138 queries, SmartVector roughly doubles top-1 accuracy over plain cosine RAG (62.0% vs. 31.0% on a held-out split), drops stale-answer rate from 35.0% to 13.3%, cuts Expected Calibration Error by nearly 2x (0.244 vs. 0.470), reduces re-
embedding cost per single-word edit by 77%, and is robust across contradiction-injection rates from 0% to 75%.
Read the full abstract ▾
Machine Learning
Differentially Private Clustered Federated Learning with Privacy-Preserving Initialization and Normality-Driven Aggregation
Jie Xu, Haaris Mehmood, Rogier Van Dalen et al. (+2 more)
Federated learning (FL) enables training of a global model while keeping raw data on end-devices. Despite this, FL has shown to leak private user information and thus in practice, it is often coupled with methods such as differential
privacy (DP) and secure vector sum to provide formal
privacy guarantees to its participants. In realistic cross-device deployments, the data are highly heterogeneous, so vanilla
federated learning converges slowly and generalizes poorly. Clustered
federated learning (CFL) mitigates this by segregating users into clusters, leading to lower intra-cluster data heterogeneity. Nevertheless, coupling CFL with DP remains challenging: the injected DP noise makes individual client updates excessively noisy, and the server is unable to initialize cluster centroids with the less noisy aggregated updates. To address this challenge, we propose PINA, a two-stage framework that first lets each client fine-tune a lightweight low-rank adaptation (LoRA) adapter and privately share a compressed sketch of the update. The server leverages these sketches to construct robust cluster centroids. In the second stage, PINA introduces a normality-driven aggregation mechanism that improves convergence and robustness. Our method retains the benefits of clustered FL while providing formal
privacy guarantees against an untrusted server. Extensive evaluations show that our proposed method outperforms state-of-the-art DP-FL
algorithms by an average of 2.9% in accuracy for
privacy budgets (epsilon in {2, 8}).
Read the full abstract ▾
All authors Jie Xu, Haaris Mehmood, Rogier Van Dalen, Karthikeyan Saravanan, Mete Ozay
Machine Learning · Cross-listed from cs.NE
An explicit operator explains end-to-end computation in the modern neural networks used for sequence and language modeling
Anif N. Shikder, Ramit Dey, Sayantan Auddy et al. (+6 more)
We establish a mathematical correspondence between state space models, a state-of-the-art architecture for capturing long-range dependencies in data, and an exactly solvable nonlinear oscillator network. As a specific example of this general correspondence, we analyze the diagonal linear time-invariant implementation of the Structured State Space Sequence model (S4). The correspondence embeds S4D, a specific implementation of S4, into a ring network
topology, in which recent inputs are encoded, as waves of activity traveling over the one-dimensional spatial layout of the network. We then derive an exact operator expression for the full forward pass of S4D, yielding an analytical characterization of its complete input-output map. This expression reveals that the nonlinear decoder in the system induces interactions between these information-carrying waves that enable classifying real-world sequences. These results generalize across modern SSM architectures, and show that they admit an exact mathematical description with a clear physical interpretation. These insights enable a new level of interpretability for these systems in terms of nonlinear oscillator networks.
Read the full abstract ▾
All authors Anif N. Shikder, Ramit Dey, Sayantan Auddy, Luisa Liboni, Alexandra N. Busch, Arthur Powanwe, Ján Mináč, Roberto C. Budzinski, Lyle E. Muller
Machine Learning
A Hierarchical MARL-Based Approach for Coordinated Retail P2P Trading and Wholesale Market Participation of DERs
Patrick Wilk, Ethan Cantor, Yikui Liu et al. (+1 more)
The ongoing shift towards decentralization of the electric energy sector, driven by the growing electrification across end-use sectors, and widespread adoption of distributed energy resources (DERs), necessitates their active participation in the electricity
markets to support grid operations. Furthermore, with bi-directional energy and communication flows becoming standard, intelligent, easy-to-deploy, resource-conservative demand-side participation is expected to play a critical role in securing power grid operational flexibility and
market efficiency. This work proposes a
market engagement framework that leverages a hierarchical multi-
agent deep reinforcement learning (MARL) approach to enable individual prosumers to participate in peer-to-peer retail auctions and further aggregate these intelligent prosumers to facilitate effective DER participation in wholesale
markets. Ultimately, a Stackelberg game is proposed to coordinate this hierarchical MARL-based DER
market participation framework toward enhanced
market performance.
Read the full abstract ▾
All authors Patrick Wilk, Ethan Cantor, Yikui Liu, Jie Li
Machine Learning · Cross-listed from cs.SE
Evaluating Assurance Cases as Text-Attributed Graphs for Structure and Provenance Analysis
Fariz Ikhwantri and Dusica Marijan
An assurance case is a structured argument document that justifies claims about a system's requirements or properties, which are supported by evidence. In regulated domains, these are crucial for meeting compliance and safety requirements to industry standards. We propose a graph diagnostic framework for analysing the structure and provenance of assurance cases. We focus on two main tasks: (1) link prediction, to learn and identify connections between argument elements, and (2) graph classification, to differentiate between assurance cases created by a state-of-the-art large
language model and those created by humans, aiming to detect bias. We compiled a publicly available
dataset of assurance cases, represented as graphs with nodes and edges, supporting both link prediction and provenance analysis. Experiments show that graph neural networks (GNNs) achieve strong link prediction performance (ROC-AUC 0.760) on real assurance cases and generalise well across domains and semi-supervised settings. For provenance detection, GNNs effectively distinguish human-authored from LLM-generated cases (F1 0.94). We observed that LLM-generated assurance cases have different hierarchical linking patterns compared to human-authored cases. Furthermore, existing GNN explanation methods show only moderate faithfulness, revealing a gap between predicted
reasoning and the true argument structure.
Read the full abstract ▾
Machine Learning
Amortized Vine Copulas for High-Dimensional Density and Information Estimation
Houman Safaai
Modeling high-dimensional dependencies while keeping likelihoods tractable re- mains challenging. Classical vine copulas provide an explicit dependence structure but require repeated, costly bivariate fitting. We propose Vine Denoising Copula (VDC)1, an amortized
estimator for simplified vines on continuous data. VDC trains a single bivariate denoising network on simulated copulas and reuses it across all vine edges. For each edge, the network maps an empirical histogram to a density grid, and iterative proportional fitting approximately enforces uniform grid marginals. The predicted densities and cached conditional transforms support stan- dard vine recursion, likelihood evaluation, sampling, and information estimation. This retains the classical vine factorization while replacing repeated per-edge fitting with reusable neural inference. Across synthetic and real-data
benchmarks, VDC combines accurate bivariate density estimation, competitive information estimates, and faster high-dimensional vine fitting after pretraining. Performance varies across
datasets; finite grid resolution and conditional prediction remain limitations. The explicit fitted structure supports edge- and tree-level information decomposition, making repeated dependence analysis more practical.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
On Bayesian Softmax-Gated Mixture-of-Experts Models
Nicola Bariletto, Huy Nguyen, Nhat Ho et al. (+1 more)
Mixture-of-experts models provide a flexible framework for learning complex probabilistic input-output relationships by combining multiple expert models through an input-dependent gating mechanism. These models have become increasingly prominent in modern machine learning, yet their theoretical properties in the
Bayesian framework remain largely unexplored. In this paper, we study
Bayesian mixture-of-experts models, focusing on the ubiquitous softmax-based gating mechanism. Specifically, we investigate the asymptotic behavior of the posterior distribution for three fundamental statistical tasks: density estimation, parameter estimation, and model selection. First, we establish posterior contraction rates for density estimation, both in the regimes with a fixed, known number of experts and with a random learnable number of experts. We then analyze parameter estimation and derive convergence guarantees based on tailored Voronoi-type losses, which account for the complex identifiability structure of mixture-of-experts models. Finally, we propose and analyze two complementary strategies for selecting the number of experts. Taken together, these results provide one of the first systematic theoretical analyses of
Bayesian mixture-of-experts models with softmax gating, and yield several theory-grounded insights for practical model design.
Read the full abstract ▾
All authors Nicola Bariletto, Huy Nguyen, Nhat Ho, Alessandro Rinaldo
Machine Learning · Cross-listed from stat.ML
Efficient Symbolic Computations for Identifying Causal Effects
Benjamin Hollering, Pratik Misra and Nils Sturma
Determining identifiability of causal effects from observational data under latent confounding is a central challenge in causal inference. For linear structural causal models, identifiability of causal effects is decidable through symbolic computation. However, standard approaches based on Gröbner bases become computationally infeasible beyond small settings due to their doubly exponential complexity. In this work, we study how to practically use symbolic computation for deciding rational identifiability. In particular, we present an efficient
algorithm that provably finds the lowest degree identifying formulas. For a causal effect of interest, if there exists an identification formula of a prespecified maximal degree, our
algorithm returns such a formula in quasi-
polynomial time.
Read the full abstract ▾
Machine Learning
CHASM: Unveiling Covert Advertisements on Chinese Social Media
Jingyi Zheng, Tianyi Hu, Yule Liu et al. (+5 more)
Current
benchmarks for evaluating large
language models (LLMs) in social media moderation completely overlook a serious threat: covert advertisements, which disguise themselves as regular posts to deceive and mislead consumers into making purchases, leading to significant ethical and legal concerns. In this paper, we present the CHASM, a first-of-its-kind
dataset designed to evaluate the capability of
Multimodal Large
Language Models (MLLMs) in detecting covert advertisements on social media. CHASM is a high-quality, anonymized, manually curated
dataset consisting of 4,992 instances, based on real-world scenarios from the Chinese social media platform Rednote. The
dataset was collected and annotated under strict
privacy protection and quality control protocols. It includes many product experience sharing posts that closely resemble covert advertisements, making the
dataset particularly challenging.The results show that under both zero-shot and in-context learning settings, none of the current MLLMs are sufficiently reliable for detecting covert advertisements.Our further experiments revealed that fine-tuning open-source MLLMs on our
dataset yielded noticeable performance gains. However, significant challenges persist, such as detecting subtle cues in comments and differences in visual and textual structures.We provide in-depth error analysis and outline future research directions. We hope our study can serve as a call for the research community and platform moderators to develop more precise defenses against this emerging threat.
Read the full abstract ▾
All authors Jingyi Zheng, Tianyi Hu, Yule Liu, Zhen Sun, Zongmin Zhang, Zifan Peng, Wenhan Dong, Xinlei He
Machine Learning
Explicit Dropout: Deterministic Regularization for Transformer Architectures
Vidhi Agrawal, Illia Oleksiienko and Alexandros Iosifidis
Dropout is a widely used regularization technique in deep learning, but its effects are typically realized through
stochastic masking rather than explicit optimization objectives. We propose a deterministic formulation that expresses dropout as an additive regularizer directly incorporated into the training loss. The framework derives explicit regularization terms for
Transformer architectures, covering
attention query, key, value, and feed-forward components with independently controllable strengths. This formulation removes reliance on
stochastic perturbations while providing clearer and fine-grained control over regularization strength. Experiments across image classification, temporal action detection, and audio classification show that explicit dropout matches or outperforms conventional implicit methods, with consistent gains when applied to
attention and feed-forward network layers. Ablation studies demonstrate stable performance and controllable regularization through regularization coefficients and dropout rates. Overall, explicit dropout offers a practical and interpretable alternative to
stochastic regularization while maintaining architectural flexibility across diverse tasks.
Read the full abstract ▾
Machine Learning
Efficient Test-Time Inference via Deterministic Exploration of Truncated Decoding Trees
Xueyan Li, Johannes Zenn, Ekaterina Fadeeva et al. (+3 more)
Self-consistency boosts inference-time performance by sampling multiple
reasoning traces in parallel and voting. However, in constrained domains like math and code, this strategy is compute-inefficient because it samples with replacement, repeatedly revisiting the same high-probability prefixes and duplicate completions. We propose Distinct Leaf Enumeration (DLE), a deterministic decoding method that treats truncated sampling as traversal of a pruned decoding tree and systematically enumerates distinct leaves instead of sampling with replacement. This strategy improves inference efficiency in two ways. Algorithmically, it increases coverage of the truncated search space under a fixed budget by exploring previously unvisited high-probability branches. Systemically, it reuses shared prefixes and reduces redundant
token generation. Empirically, DLE explores higher-quality
reasoning traces than
stochastic self-consistency, yielding better performance on math, coding, and general
reasoning tasks.
Read the full abstract ▾
All authors Xueyan Li, Johannes Zenn, Ekaterina Fadeeva, Guinan Su, Mrinmaya Sachan, Jonas Geiping
Machine Learning · Cross-listed from cs.CR
Towards Certified Malware Detection: Provable Guarantees Against Evasion Attacks
Nandakrishna Giri, Asmitha K. A., Serena Nicolazzo et al. (+2 more)
Machine learning-based static malware detectors remain vulnerable to
adversarial evasion techniques, such as metamorphic engine mutations. To address this vulnerability, we propose a certifiably robust malware detection framework based on randomized smoothing through feature ablation and targeted noise injection. During evaluation, our system analyzes an executable by generating multiple ablated variants, classifies them by using a smoothed classifier, and identifies the final label based on the majority vote. By analyzing the top-class voting distribution and the Wilson score interval, we derive a formal certificate that guarantees robustness within a specific radius against feature-space perturbations. We evaluate our approach by comparing the performance of the base classifier and the smoothed classifier on both clean executables and ablated variants generated using PyMetaEngine. Our results demonstrate that the proposed smoothed classifier successfully provides certifiable robustness against metamorphic evasion attacks without requiring modifications to the underlying machine learning architecture.
Read the full abstract ▾
All authors Nandakrishna Giri, Asmitha K. A., Serena Nicolazzo, Antonino Nocera, Vinod P
Machine Learning · Cross-listed from stat.ML
Decentralized Machine Learning with Centralized Performance Guarantees via Gibbs Algorithms
Yaiza Bermudez, Samir M. Perlaza and Iñaki Esnaola
In this paper, it is shown, for the first time, that centralized performance is achievable in decentralized learning without sharing the local
datasets. Specifically, when clients adopt an empirical risk minimization with relative-
entropy regularization (ERM-RER) learning framework and a forward-backward communication between clients is established, it suffices to share the locally obtained Gibbs measures to achieve the same performance as that of a centralized ERM-RER with access to all the
datasets. The core idea is that the Gibbs measure produced by client~$k$ is used, as reference measure, by client~$k+1$. This effectively establishes a principled way to encode prior information through a reference measure. In particular, achieving centralized performance in the decentralized setting requires a specific scaling of the regularization factors with the local sample sizes. Overall, this result opens the door to novel decentralized learning paradigms that shift the collaboration strategy from sharing data to sharing the local inductive bias via the reference measures over the set of models.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.NI
Forecasting Individual NetFlows using a Predictive Masked Graph Autoencoder
Georgios Anyfantis and Pere Barlet-Ros
In this paper, we propose a proof-of-concept Graph Neural Network model that can successfully predict network flow-level traffic (NetFlow) by accurately modelling the graph structure and the connection features. We use sliding-windows to split the network traffic in equal-sized heterogeneous bidirectional graphs containing IP, Port, and Connection nodes. We then use the GNN to model the
evolution of the graph structure and the connection features. Our approach shows superior results when identifying the Port and IP to which connections attach, while feature reconstruction remains competitive with strong
forecasting baselines. Overall, our work showcases the use of GNNs for per-flow NetFlow prediction.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.RO
Temporal Difference Calibration in Sequential Tasks: Application to Vision-Language-Action Models
Shelly Francis-Meretzki, Mirco Mutti, Yaniv Romano et al. (+1 more)
Recent advances in
vision-
language-action (VLA) models for robotics have highlighted the importance of reliable uncertainty quantification in sequential tasks. However, assessing and improving calibration in such settings remains mostly unexplored, especially when only partial trajectories are observed. In this work, we formulate sequential calibration for episodic tasks, where task-success confidence is produced along an episode, while success is determined at the end of it. We introduce a sequential extension of the Brier score and show that, for binary outcomes, its risk minimizer coincides with the VLA policy's value function. This connection bridges uncertainty calibration and reinforcement learning, enabling the use of temporal-difference (TD) value estimation as a principled calibration mechanism over time. We empirically show that TD calibration improves performance relative to the state-of-the-art on simulated and real-
robot data. Interestingly, we show that when calibrated using TD, the VLA's single-step action probabilities can yield competitive uncertainty estimates, in contrast to recent findings that employed different calibration techniques.
Read the full abstract ▾
All authors Shelly Francis-Meretzki, Mirco Mutti, Yaniv Romano, Aviv Tamar
Machine Learning · Cross-listed from cs.RO
MOMO: A framework for seamless physical, verbal, and graphical robot skill learning and adaptation
Markus Knauer, Edoardo Fiorini, Maximilian Mühlbauer et al. (+10 more)
Industrial
robot applications require increasingly flexible systems that non-expert users can easily adapt for varying tasks and environments. However, different adaptations benefit from different interaction modalities. We present an interactive framework that enables
robot skill adaptation through three complementary modalities: kinesthetic touch for precise spatial corrections, natural
language for high-level semantic modifications, and a graphical web interface for visualizing
geometric relations and trajectories, inspecting and adjusting parameters, and editing via-points by drag-and-drop. The framework integrates five components: energy-based human-intention detection, a tool-based LLM architecture (where the LLM selects and parameterizes predefined functions rather than generating code) for safe natural
language adaptation, Kernelized Movement Primitives (KMPs) for motion encoding, probabilistic Virtual Fixtures for guided demonstration recording, and ergodic control for surface finishing. We demonstrate that this tool-based LLM architecture generalizes skill adaptation from KMPs to ergodic control, enabling voice-commanded surface finishing. Validation on a 7-DoF torque-controlled
robot at the Automatica 2025 trade fair demonstrates the practical applicability of our approach in industrial settings.
Read the full abstract ▾
All authors Markus Knauer, Edoardo Fiorini, Maximilian Mühlbauer, Stefan Schneyer, Promwat Angsuratanawech, Florian Samuel Lay, Timo Bachmann, Samuel Bustamante, Korbinian Nottensteiner, Freek Stulp, Alin Albu-Schäffer, João Silvério, Thomas Eiband
Machine Learning · Cross-listed from physics.ao-ph
Mechanistic Interpretability Tool for AI Weather Models
Kirsten I. Tempest, Matthias Beylich and George C. Craig
Artificial Intelligence (AI) weather models are improving rapidly, and their forecasts are already competitive with long-established traditional Numerical Weather Prediction (NWP). To build confidence in this new methodology, it is critical that we understand how these predictions are generated. This is a huge challenge as these AI weather models remain largely black boxes. In other areas of Machine Learning (ML), mechanistic interpretability has emerged as a framework for understanding ML predictions by analysing the building blocks responsible for them. Here we present an open-source, highly adaptable tool which incorporates concepts from mechanistic interpretability. The tool organises internal latent representations from the model processor and allows for initial analyses, including cosine similarity and Principal Component Analysis (PCA), enabling the user to identify directions in latent space potentially associated with meteorological features. Applying our tool to the graph neural network GraphCast, we present preliminary case studies for mid-latitude synoptic-scale waves and specific humidity. These demonstrate the tool's ability to identify linear combinations of latent channels that appear to correspond to interpretable features.
Read the full abstract ▾
Machine Learning
Surrogate Functionals for Machine-Learned Orbital-Free Density Functional Theory
Roman Remme and Fred A. Hamprecht
We introduce surrogate functionals: machine-learned energy functionals for orbital-free density functional theory (OF-DFT) which are defined not by universal fidelity to a physical reference, but merely by the requirement that density optimization with a fixed procedure yields the true ground-state density. Helpfully, training surrogate functionals requires only ground-state densities, no energies or gradients away from the ground state. We here propose a gradient-descent-improvement loss that guarantees exponential convergence of the density to the ground state, and combine it with an adaptive sampling scheme that concentrates learning around the optimization trajectories actually visited during inference. On the QM9 and QMugs
benchmarks, surrogate functionals achieve density errors competitive with or improving upon the state of the art for fully supervised machine-learned OF-DFT, while eliminating the need for the $O(N^3)$ orthononormalization step required by prior work, yielding improved runtime scaling for larger systems.
Read the full abstract ▾
Machine Learning
The Origin of Edge of Stability
Elon Litman
Full-batch gradient descent on neural networks drives the largest Hessian
eigenvalue to the threshold $2/\eta$, where $\eta$ is the learning rate. This phenomenon, the Edge of Stability, has resisted a unified explanation: existing accounts establish self-regulation near the edge but do not explain why the trajectory is forced toward $2/\eta$ from arbitrary initialization. We introduce the edge coupling, a functional on consecutive iterate pairs whose coefficient is uniquely fixed by the gradient-descent update. Differencing its criticality condition yields a step recurrence with stability boundary $2/\eta$, and a second-order expansion yields a loss-change formula whose telescoping sum forces
curvature toward $2/\eta$. The two formulas involve different Hessian averages, but the mean value
theorem localizes each to the true Hessian at an interior point of the step segment, yielding exact forcing of the Hessian
eigenvalue with no gap. Setting both gradients of the edge coupling to zero classifies fixed points and period-two orbits; near a fixed point, the problem reduces to a function of the half-amplitude alone, which determines which directions support period-two orbits and on which side of the critical learning rate they appear.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.RO
VTouch++: A Multimodal Dataset with Vision-Based Tactile Enhancement for Bimanual Manipulation
Qianxi Hua, Xinyue Li, Zheng Yan et al. (+4 more)
Embodied intelligence has advanced rapidly in recent years; however, bimanual manipulation-especially in contact-rich tasks remains challenging. This is largely due to the lack of
datasets with rich physical interaction signals, systematic task organization, and sufficient scale. To address these limitations, we introduce the VTOUCH
dataset. It leverages
vision based tactile sensing to provide high-fidelity physical interaction signals, adopts a
matrix-style task design to enable systematic learning, and employs automated data collection pipelines covering real-world, demand-driven scenarios to ensure scalability. To further validate the effectiveness of the
dataset, we conduct extensive quantitative experiments on cross-modal
retrieval as well as real-
robot evaluation. Finally, we demonstrate real-world performance through generalizable inference across multiple
robots, policies, and tasks.
Read the full abstract ▾
All authors Qianxi Hua, Xinyue Li, Zheng Yan, Yang Li, Chi Zhang, Yongyao Li, Yufei Liu
Machine Learning · Cross-listed from cs.CL
DialToM: A Theory of Mind Benchmark for Forecasting State-Driven Dialogue Trajectories
Neemesh Yadav, Palakorn Achananuparp, Jing Jiang et al. (+1 more)
We introduce DialToM, an annotated Theory of Mind (ToM)
benchmark built from naturalistic human-human dialogues using a multiple-choice evaluation framework. Concurrent with recent work showing a gap between explicit mental-state inference and applied ToM in synthetic settings~\cite{gu2024simpletom}, we establish a stricter State-Driven Diagnostic Probe in which models must forecast state-consistent dialogue trajectories solely from isolated mental-state profiles without dialogue context. Our evaluation reveals a systematic
reasoning asymmetry – LLMs excel at inferring mental states (Literal ToM) but struggle to leverage them for social
forecasting (Functional ToM). Crucially, a domain expert achieves 100% accuracy on this task, proving its validity and establishing a stark human-AI capability gap. Further, a teacher-student
reasoning injection probe shows that Gemini 3 Pro – which establishes the leading baseline – possesses robust Functional ToM capabilities for context-free
forecasting that are transferable to weaker models. DialToM, its evaluation code, and
dataset are publicly available at https://github.com/Stealth-py/DialToM.
Read the full abstract ▾
All authors Neemesh Yadav, Palakorn Achananuparp, Jing Jiang, Ee-Peng Lim