Machine Learning
Sparse Delta Memory: Scaling the State of Linear RNNs through Sparsity
Loïc Cabannes, Pierre-Emmanuel Mazaré, Gergely Szilvasy et al. (+6 more)
Linear
attention models allow a fixed state size and a fixed amount of compute per
token. However, due to their limited state size, linear
attention models fall behind in long-context recall compared to softmax-
attention-based
transformer architectures. Increasing the state size of linear
attention improves recall performance but at the cost of higher FLOPs. In this work, we introduce Sparse Delta
Memory (SDM), an architecture that scales the hidden state of gated linear RNNs to orders of magnitude higher capacity using a sparse addressing scheme. SDM extends the Gated DeltaNet architecture by replacing the dense key-value outer product with sparse reads and writes to a large explicit
memory. We show that, under an isoFLOP constraint and with an identical number of parameters, a higher state
memory capacity significantly improves performance on in-context learning and long-context
retrieval tasks. Moreover, by learning the initial state of the SDM
memory and therefore using it as a parametric
memory, we show that the model further improves on a wide range of common-knowledge and
reasoning tasks.
Read the full abstract ▾
All authors Loïc Cabannes, Pierre-Emmanuel Mazaré, Gergely Szilvasy, Matthijs Douze, Maria Lomeli, Ilze Amanda Auzina, Justin Carpentier, Gabriel Synnaeve, Hervé Jégou
Machine Learning
Generalist Vision-Language Models for Fast Radio Burst detection: a zero-shot benchmark against a specialized detector
Raiff H. Santos, Amilcar R. Queiroz, Tharcisyo S. S. Duarte et al. (+2 more)
Fast
Radio Burst (FRB) detection increasingly relies on specialized deep learning models that require large task-specific training sets and cannot be redefined without retraining. We evaluate whether small, open-weight, locally run generalist
Vision-
Language Models (VLMs) can detect FRBs in dynamic spectra under a zero-shot,
prompt-only regime. On a balanced binary
benchmark of 2000 simulated L-band spectra, Gemma 4 E2B reaches an accuracy of 94.05%, statistically indistinguishable from the specialized detector SwinYNet (92.85%), with a far lower false-positive rate on structured RFI (4.8% vs. 24.6%) and none on pure noise, though SwinYNet ranks perfectly (ROC-AUC 1.0000 vs. 0.9520). Rewriting the
prompt alone reconfigures the same models for three-class FRB/RFI/noise classification, reaching up to 86.0% accuracy without a single false FRB while classifying each 2 s spectrum in 1.0–1.5 s, faster than the observation itself. Applied unchanged to the 1600 real FAST observations of FAST-FREX, they reject real interference almost perfectly (2 and 5 false positives in 1000 negatives) but recover only 28.5% and 27.0% of the 600 catalogued bursts, against 95.7% reported for SwinYNet on the same files. Stratifying those bursts by the dispersed signal in the image shows the limit to be the input representation rather than the classifier, recall rising to 84–85% where the sweep is unambiguous and collapsing to 1% on the 13% of positives carrying no detectable signal in a 2 s undedispersed full-band view. The simulated bursts are nearly 30 times brighter in median, and at matched brightness the recalls agree to within a few points.
Read the full abstract ▾
All authors Raiff H. Santos, Amilcar R. Queiroz, Tharcisyo S. S. Duarte, K. E. L. de Farias, Rafael A. Batista
Machine Learning · Cross-listed from cs.AI
Physics-Audited Agentic Discovery in Scientific Machine Learning
Diab W. Abueidda, Bilal Ahmed, Panos Pantidis et al. (+1 more)
Agentic discovery can yield mechanics surrogates with low validation error that nonetheless violate physical requirements. We present Physics-Audited Agentic SciML (PA-SciML), a framework combining large
language model
agents with physics
auditing. Physical requirements, encoded as contracts such as superposition, boundary conditions, and causality, are fixed before candidate search and tested on predicted fields without reference solutions, including through an
adversarial search. Audit results guide later proposals and model selection but are reported separately from validation error. Single-edit tests document reusable design changes. Four solid-mechanics problems of increasing complexity, from parametric elasticity to heterogeneous stiffness fields and cyclic plasticity with combined hardening, reveal causality, zero-input, and stress-bound violations in surrogates selected by error alone. In elastodynamics, two surrogates reach nearly equal error, but only the audit-enabled one is causal by construction. Audit-enabled surrogates pass the required checks at sampled inputs while matching or exceeding audit-disabled accuracy in most comparisons.
Read the full abstract ▾
All authors Diab W. Abueidda, Bilal Ahmed, Panos Pantidis, Mostafa E. Mobasher
Machine Learning
On Adversarial Vulnerability of Vision-Language Models through the Lens of Intermediate Spectral Subspaces
Chethan Krishnamurthy Ramanaik, Tobias Callies, Michael Hecht et al. (+1 more)
Adversarial vulnerability in deep neural networks (DNNs) has been studied from the perspectives of decision-boundary geometry, feature robustness, input-output Jacobians, and the instability of inverse problems. Here, we focus on the spectral structure of intermediate linear transformations that propagate information through modern DNNs, an unexplored mechanism of
adversarial vulnerability. Specifically, we investigate
transformer-based
vision-
language models, whose linear layers admit interpretable spectral decompositions and whose widespread adoption makes understanding their robustness increasingly important. We propose a white-box spectral-subspace-guided attack (SSGRA) that aligns intermediate representations with the subspace spanned by the bottom right singular vectors. Our experiments show improved attack effectiveness over existing baselines. In addition, SSGRA offers a spectral interpretation of
adversarial vulnerability in VLMs, providing insights for improving their robustness.
Read the full abstract ▾
All authors Chethan Krishnamurthy Ramanaik, Tobias Callies, Michael Hecht, Eirini Ntoutsi
Machine Learning · Cross-listed from cs.RO
Behavior Foundations for Quadruped Robots: ABot-C0 Technical Report
Xufeng Zhao, Fuzhi Yang, Jianhui Chen et al. (+17 more)
The motion controller is one of the most fundamental modules in
embodied intelligence systems. Driven by large-scale human motion-capture data and the motion-tracking paradigm,
humanoid control has achieved remarkable progress in recent years. However, migrating this recipe to the quadrupedal setting is far less straightforward: animal motion data is scarcer and harder to capture at scale than human data, and cross-embodiment retargeting remains fragile. We present ABot-C0, a generalist motion-control system for quadruped
robots that establishes three complementary behavior foundations: a scalable multi-source motion-data pipeline, robust policy learning across motion tracking, locomotion, and scene interaction, and a unified deployment stack for reliable real-world operation. Fundamentally, we construct a data pyramid through conditional
video-generation synthesis, annotated motion capture, teleoperation, and human design, producing 16,074 physically feasible motion clips as the data foundation for diverse motion-learning demands. With large-scale motion data, a Flow-Matching generalist policy demonstrates, for the first time, a scaling law for quadruped motion tracking: performance improves consistently as training scales up, with zero-shot capability to track unseen motions. We then go a step further toward robust all-terrain locomotion by adopting a three-stage privileged-to-perceptive framework with temporal
LiDAR memory and terrain-predictive supervision. Collectively, these components form a motion generalist that coordinates multi-policy execution, smooth behavior transitions, energy-efficient control, and safety mechanisms for real-world deployment. Extensive experiments on urban-terrain autonomous
navigation and companion-style
multimodal interaction demonstrate that quadruped
robots can move beyond functional demos toward product-level behavioral intelligence.
Read the full abstract ▾
All authors Xufeng Zhao, Fuzhi Yang, Jianhui Chen, Li Gao, Zhang Meng, Jie Gao, Yao Zheng, Congyang Zhao, Tianxiong Lv, Menglin Yang, Minqi Gu, Yaru Zhao, Wenyu Liu, Honglin Han, Shihui Su, Zixiao Tang, Liu Liu, Mu Xu, Yang Cai, Wenbin Tang
Machine Learning
Multi-Agent AI Control: Distributed Attacks Hamper Per-Instance Monitors
Oliver Makins, Orazio Angelini, Zohreh Shams et al. (+1 more)
AI control is a family of techniques to prevent an AI with malicious goals from subverting its operator's intent. AI Control usually studies a single
agent in one trajectory, but real deployments run many
agents over shared infrastructure, and the most severe risks (model-weight exfiltration, training-run poisoning) plausibly need several
agents acting in concert. We initiate the empirical study of multi-
agent AI control, formalising distributed attacks in which several
agents jointly aim for a malicious goal. We develop FakeLab: a synthetic AI-lab codebase (9 services, 86 benign tasks, 4 attack objectives). We evaluate single
agent monitoring against distributed attacks, varying the number of
agents, their coordination, model capabilities and precise monitoring configuration. Our central finding is the fragmentation effect: as more
agents coordinate to attack, per-
agent monitoring becomes less likely to catch any of the attackers. Secondly, this is not driven by the ratio of benign to malicious code – we argue this is likely dependent on model capability. Third, an explicit planner amplifies the fragmentation effect, raises the rate of completing the attack up to sevenfold, and mildly sharpens executors' poor monitor-theory-of-mind. Finally: a weak "trusted" monitor misses most attacks, while a stronger one cuts undetected success by more than an order of magnitude but remains imperfect. We are happy to share FakeLab, our control setting, with safety researchers on request.
Read the full abstract ▾
All authors Oliver Makins, Orazio Angelini, Zohreh Shams, Mary Phuong
Machine Learning
Latency-Aware Bid Acceptance under Operational Feasibility: A Public Benchmark with Hindsight Ceilings
Aswin Chandrasekaran
Online truckload bid acceptance is a closed-loop
stochastic decision problem in which a carrier or broker must, in real time, accept or reject a tendered load subject to operational feasibility, fleet repositioning costs, and opportunity cost against future demand. Public, reproducible
benchmarks for this problem are scarce: existing routing
benchmarks are static, while dynamic-fleet studies typically rely on private operator data. We introduce FreightBidBench, a public-calibrated, dependency-free, closed-loop
benchmark in which feasibility (pickup reach, appointment windows, simplified hours-of-service,
stochastic yard delays) and economics (service-failure penalty, terminal fleet value, daily price-premium window) are explicit, versioned, and reproducible from public Freight Analysis Framework and U.S. Department of Agriculture truck rate data. We develop two full-horizon hindsight ceilings: a simple LP style relaxation and a tighter Lagrangian-per-truck information relaxation that retains per-truck hours-of-service and sequencing structure and is 20.7% tighter than the LP relaxation on a tight-capacity scenario and 39.3% tighter on a scarce-capacity scenario. We introduce a parametric surrogate-rollout cascade with boundary-band and scarcity-pressure escalation triggers. On ten-seed tight and scarce scenarios, the best simple policy retains 91.0% and 86.5% of rollout profit and the standard-library surrogate 94.2% and 89.3%; a cascade at a single escalation band recovers roughly 98% on both at 40-56% of rollout's mean decision latency, and on the tight scenario is statistically indistinguishable from the rollout teacher (paired-bootstrap 95% CI on the profit delta spans zero).
Read the full abstract ▾
Machine Learning
Hypergraph Neural Stochastic Diffusion: An SDE Framework for Uncertainty Estimation
Zhiheng Zhou, Mengyao Zhou, Dengyi Zhao et al. (+2 more)
Hypergraph neural networks have shown powerful capability in modeling higher-order relations, yet their predictive uncertainty remains underexplored. Unlike pairwise graphs, uncertainty in hypergraphs arises not only from noisy attributes and ambiguous labels, but also from variations in node-hyperedge incidence structures and complex higher-order dependencies. Existing approaches mainly estimate uncertainty from final predictions or rely on computationally expensive ensembles and
Bayesian inference, limiting their ability to capture uncertainty
evolution during representation learning. In this paper, we propose Hypergraph Neural
Stochastic Diffusion(HyperNSD), a
stochastic differential equation framework for uncertainty estimation on hypergraphs. HyperNSD models hypergraph representations as
stochastic processes evolving over node-hyperedge incidence structures. A learnable drift function captures deterministic higher-order
diffusion dynamics, while a learnable
stochastic forcing function characterizes structural ambiguity and representation noise. Predictive uncertainty is directly quantified through the variability of
stochastic representation trajectories, providing an intrinsic uncertainty measure beyond post-hoc confidence scores. We formulate HyperNSD with neural drift and
diffusion networks, enabling joint learning of prediction and uncertainty propagation. Theoretical analyses establish well posedness, perturbation stability,permutation equivariance, and numerical convergence of the proposed
stochastic dynamics. Experiments on multiple hypergraph
benchmarks demonstrate that HyperNSD achieves reliable uncertainty estimation for out-of-distribution and misclassification detection while preserving competitive prediction accuracy. These results provide a principled
stochastic-dynamical framework for trustworthy higher-order representation learning.
Read the full abstract ▾
All authors Zhiheng Zhou, Mengyao Zhou, Dengyi Zhao, Xingqin Qi, Guiying Yan
Machine Learning · Cross-listed from cs.CL
R^3: Advertisement Compliance Rectification via Group-Relative Experience Extractor and Curriculum Reinforcement
Yuan Chen, Zhenyu Hu, Mengge Xue et al. (+5 more)
Rigorous content moderation is crucial for online advertising but leads to millions of daily rejections. This scale renders manual rectification infeasible, particularly for
video advertisements. However, existing safety-driven methods often suffer from aggressive over-editing, which compromises the advertiser's original semantic intent merely to satisfy compliance. In this work, we target the rectification of textual violations in
video ads, covering both
speech transcripts and on-screen text. We propose R^3, a novel framework designed to harmonize compliance with original semantic intent preservation. Our approach integrates three key innovations: (1) an experience-driven data synthesis framework that bootstraps high-quality supervision via a group-Relative compliance experience extractor; (2) a curriculum Reinforcement learning strategy with hierarchical rewards designed to enforce compliance while maximizing semantic consistency; and (3) a comprehensive
video Rectification framework seamlessly integrating text recognition, rewriting, and re-rendering for industrial deployment. Extensive experiments on industrial
datasets and online A/B testing demonstrate that R^3 significantly outperforms state-of-the-art baselines, achieving an optimal trade-off between violation rectification and intent preservation.
Read the full abstract ▾
All authors Yuan Chen, Zhenyu Hu, Mengge Xue, Te Cao, Liqun Liu, Peng Shu, Huan Yu, Jie Jiang
Machine Learning
Mechanistic Interpretability for Neural Networks: Circuits, Sparse Features and Symbolic Reasoning
Pranav Sawant and Jakub Krejčí
This article offers a comprehensive overview of mechanistic interpretability, an emerging field that seeks to reverse-engineer the internal
algorithms of modern neural networks. While traditional explainable AI methods often stop at surface-level input-output correlations, this approach directly addresses the opaque "black box" nature of machine learning models, which is essential for ensuring safety and auditability in high-stakes deployments. The paper provides a detailed examination of
Transformer circuit analysis, exploring how internal components like the residual stream,
attention mechanisms, and induction heads drive complex tasks and in-context learning. It subsequently tackles the core challenge of superposition and polysemanticity, demonstrating how tools like Sparse Autoencoders (SAEs) and transcoders can decompose tangled network activations into distinct, human-interpretable features. Furthermore, the paper explores methods for actively controlling and modifying model behavior through steering vectors and causal interventions. Finally, it connects these mechanistic insights with neurosymbolic AI frameworks designed to translate neural representations into explicit, executable logical rules.
Read the full abstract ▾
Machine Learning
FedCVESA: Taking Away Training Data in Federated Learning via Correlation Value Encoding and Segmented Aggregation
Chongkai Li, Bang Zhang and Wenjian Luo
Federated learning (FL) avoids explicit data exposure by keeping raw data on local clients, yet
privacy risks remain in the training process and the learned model itself. Recently, centralized Taking Away Training Data (TATD) attacks have shown that malicious training could abuse the memorization capacity of deep models to store and later recover training data. However, this memorization-based threat has not been systematically studied under FL environments, where multi-client averaging could overwrite encoded training data. In this paper, we study a white-box TATD attack in which a malicious server selects n target clients from K participating clients and actively writes private training data into the global model during
federated training. We propose FedCVESA, a
federated variant of Correlation Value Encoding Attack (CVEA), by adding a Pearson-correlation regularizer to the loss function of target clients, so that private training data are gradually encoded into selected model parameters, referred to as carrier parameters. To reduce the overwriting of carrier parameters during server aggregation, we further propose segmented aggregation over dispersed carrier parameters, preserving selected carrier parameters while keeping standard averaging on the remaining parameters. Experiments on MNIST, Fashion-MNIST, and CIFAR-10 under Dirichlet non-IID partitions show that the proposed method can steal semantically meaningful private training images from the trained model while maintaining acceptable main-task utility in a controlled proof-of-concept setting. These results demonstrate that FL can become a parameter-level memorization channel for active TATD attack under the studied white-box malicious-server setting.
Read the full abstract ▾
Machine Learning
Nonlinear Bandit
Tianshuo Zheng, Ting Wu, Zhi-Hua Zhou et al. (+1 more)
In this paper we first study the problem of generalized linear bandit (GLB) under heavy-tailed noise. The characteristics of heavy-tailed distributions are widely observed in real-world applications such as personalized recommendation, financial
markets, and medical treatments. Based on the online mirror descent (OMD) method, we propose an
algorithm EHM that extends the adaptive Huber loss method (Wang et al., 2025) with one-pass update ($\mathcal{O}(1)$ computational complexity with respect to current round $t$ and the time horizon $T$), which simultaneously achieves an almost optimal regret of $\widetilde{\mathcal{O}}(T^{\frac{1}{1+\epsilon}})$ where $T$ is the time horizon. In addition, by utilizing a special property of some link function (Sawarni et al., 2025), our
algorithm eliminates the need to know a commonly used parameter. Next, we study the GLB problem under the case when contextual characteristic becomes piecewise constant, and we slightly revised former
algorithm to obtain the PGLB-EHM
algorithm. After theoretical analysis, we prove that the regret upper bound order stays the same. Furthermore, we look deeper into a special case of nonlinear bandit (NB) and present the NB-EHM
algorithm with bisection method and special restriction. Eventually we utilize the affine lifting approach and show that the general NB problem can be applied with NB-EHM to achieve a sublinear regret bound.
Read the full abstract ▾
All authors Tianshuo Zheng, Ting Wu, Zhi-Hua Zhou, Keqin Liu
Machine Learning · Cross-listed from cond-mat.mtrl-sci
Bayesian Optimization of Genetic Algorithm Hyperparameters in a Multi-Fidelity Framework for Efficient Lattice Material Design
Sergei Zorkaltsev, Maciej Haranczyk and Christina Schenk
This study presents a multi-fidelity framework for the systematic optimization of genetic
algorithm (GA) hyperparameters. The framework integrates three fidelity levels: high-fidelity Fast
Fourier Transform (FFT) homogenization for validation, a medium-fidelity 3D convolutional neural network surrogate for rapid property evaluation, and a low-fidelity Gaussian process (GP) surrogate within a
Bayesian optimization (BO) framework to guide the hyperparameter search. Various acquisition functions are evaluated, with logNEI achieving the best performance by effectively accounting for the noise inherent in GA evaluations. The proposed framework identifies hyperparameter configurations that enable a 25-generation GA run to achieve elastic modulus values comparable to those obtained in a full 75-generation optimization. Furthermore, introducing a penalized BO objective significantly reduces the number of required
lattices with only minor decreases in absolute achieved elastic modulus, revealing a practical trade-off between performance and the number of structures that must be evaluated. High-fidelity FFT validation verifies the effectiveness of the surrogate-driven optimization strategy. The optimized hyperparameters allow for rapid convergence, eliminate the need for
lattice mutation, and reduce the overall computational cost by 24% (from 225 to 171 hours) while preserving mechanical performance. These results demonstrate the potential of multi-fidelity optimization as an efficient and practical approach for GA hyperparameter tuning and future experimental
lattice design studies.
Read the full abstract ▾
Machine Learning
BubbleSH: A Dataset of Rising Bubbles with Deformable Interfaces
Rachna Ramesh, Kiet Bennema ten Brinke, Douwe Orij et al. (+2 more)
Bubbly flows exhibit complex multiscale dynamics, with deformable bubbles interacting through the surrounding
liquid and giving rise to strongly coupled kinematic and morphological behavior. We present BubbleSH, a bubbly flows
dataset consisting of transient, three-dimensional bubble-swarm dynamics obtained from high-fidelity direct numerical simulations of bubbles rising in a periodic domain. The
dataset provides time-resolved bubble trajectories, velocities, and shape
evolution, with bubble morphology compactly represented using spherical harmonics. Designed to be lightweight yet physically expressive, the
dataset enables data-driven modeling of bubbly flow simulators where shape deformation and bubble-bubble interactions play a central role. We characterize the
dataset with bubble kinematics, morphology, and interaction patterns, and introduce evaluation metrics for both trajectory and shape prediction. The sensitivity of bubble-swarm dynamics to local perturbations makes BubbleSH particularly well suited to generative models that learn distributions over possible future trajectories. We evaluate a permutationally and translationally equivariant probabilistic emulator on BubbleSH given the proposed metrics. Therefore, we establish a compact, high-fidelity
dataset and a
benchmark for developing and evaluating data-driven models of deformable, chaotic multiphase systems.
Read the full abstract ▾
All authors Rachna Ramesh, Kiet Bennema ten Brinke, Douwe Orij, Ivo Roghair, Vlado Menkovski
Machine Learning
FMMVCC: Fuzzy Mamba-based Multi-View Contrastive Clustering for Univariate Time Series
Donato Cerciello, Leonardo Schiavo, Angel Panizo-LLedot et al. (+2 more)
In many realistic scenarios, large volumes of time series data are generated with limited or expensive annotations. This limitation makes supervised learning methods difficult to apply and leads to the use of unsupervised approaches capable of discovering meaningful structures directly from raw data. Clustering therefore plays a crucial role in organizing time series into groups that share similar temporal patterns, enabling exploratory analysis and downstream tasks without requiring manual labeling. However, existing deep clustering methods often struggle to capture long-range temporal dependencies or rely on architectures with high computational cost. This paper introduces FMMVCC, a Mamba-based deep clustering framework for time series that leverages state space sequence modeling to efficiently learn temporal representations with linear complexity. Additionally, it utilizes multi-view self-supervised learning with temporal masking and augmentations. Experimental evaluation in 15
benchmark datasets proves that FMMVCC consistently outperforms state-of-the-art baselines, achieving the best overall performance in 29 of 60 total metric evaluations and the highest average rank in all tested scenarios.
Read the full abstract ▾
All authors Donato Cerciello, Leonardo Schiavo, Angel Panizo-LLedot, Javier Huertas Tato, David Camacho
Machine Learning
Safe Reinforcement Learning using Ideas from Model Predictive Control
Georg Schäfer, Jakob Rehrl, Stefan Huber et al. (+1 more)
Reinforcement learning (RL) enables the synthesis of control policies directly from data, making it highly appealing for complex cyber-physical systems (CPSs) and robotics. A persistent challenge, however, is ensuring strict, hard safety constraints during the active learning phase. In real-world physical systems, violating mechanical limits can cause irreversible damage, necessitating that exploration remains strictly within safe operational regions. We propose a generalized framework that combines the adaptive, high-performance nature of deep reinforcement learning (DRL) with the formal safety guarantees of model predictive control (MPC). Using a mathematical model of the system dynamics, offline MPC computations define a feasible state-action space, representing all safe combinations of system states and control inputs that guarantee constraint satisfaction. During training and deployment, the RL
agent's instantaneous actions are projected onto this globally verified feasible set via a safety filter. We systematically evaluate our generalized approach on a non-linear 1-DoF laboratory testbed, demonstrating successful exploration and stable policy convergence on physical hardware.
Read the full abstract ▾
All authors Georg Schäfer, Jakob Rehrl, Stefan Huber, Simon Hirlaender
Machine Learning
ORCAID: Oblique Rule-Based Continuous-Action Interpretation for Deep RL Policies
Ignacio D. Lopez-Miguel, Ezio Bartocci, Thomas Eiter et al. (+1 more)
Explainability remains a key issue in reinforcement learning (RL). Distilling an interpretable policy from an
agent trained in a complex environment is particularly challenging when the action space is continuous. We introduce ORCAID, a novel method for extracting interpretable rule-based policies from RL
agents operating in mixed continuous-discrete environments with continuous action spaces. Our main contribution is an efficient oblique decision tree training
algorithm that partitions the state space by hyperplanes and fits local linear models. The key idea lies in a three-stage split search: efficient random initialization, local refinement, and backward elimination. Finally, adjacent leaves are merged to yield a concise set of interpretable rules describing a given deep RL policy. We evaluate ORCAID across multiple RL environments, demonstrating that the extracted rule-based policies maintain strong performance with a low number of parameters and can even be used to improve the performance of the original deep RL policy.
Read the full abstract ▾
All authors Ignacio D. Lopez-Miguel, Ezio Bartocci, Thomas Eiter, Martin Tappler
Machine Learning
HPG-Diff: Hierarchical physics-guided diffusion with differentiable connectivity constraints for topology optimization
Jinbo Yang, Mingyue Yuan, Boyuan Zhang et al. (+2 more)
Deep generative models offer a promising paradigm for
topology optimization, enabling rapid design exploration. However, these approaches lack intrinsic physics guidance, often leading to poor generalizability across unseen boundary conditions and the formation of floating material artifacts. To address these limitations, we propose Hierarchical Physics-Guided
Diffusion (HPG-Diff), a novel
diffusion framework that enforces physics consistency through two synergistic mechanisms. First, we introduce a hierarchical physics-guided strategy that aligns different precomputed physics features with the denoising process, guiding material distribution toward optimal load paths to enhance generalizability. Second, we propose a floating material suppression loss as a differentiable connectivity constraint inspired by thermal conduction to improve topological connectivity. By simulating a virtual heat propagation process from load positions, this mechanism explicitly penalizes floating material during training. Quantitative evaluations demonstrate that HPG-Diff achieves average compliance errors of 0.87% (in-distribution) and 5.29% (out-of-distribution), while reducing floating material ratios to 2.90% and 2.44%, respectively. Furthermore, case studies on a 3:1 rectangular domain, including cantilever and bridge
benchmarks, provide preliminary evidence that lightweight LoRA fine-tuning with a small
dataset can support the adaptation of HPG-Diff to rectangular non-square domains.
Read the full abstract ▾
All authors Jinbo Yang, Mingyue Yuan, Boyuan Zhang, Yoshifumi Kitamura, Shikai Jing
Machine Learning · Cross-listed from stat.ML
DiPhon: Diffusion on Graphons for Scalable Graph Generation
Sergio Rozada, Yiming Qin, Manuel Madeira et al. (+2 more)
Diffusion models represent a leading paradigm for graph generation, with notable impact in domains such as molecular design. Yet, scaling these models to large graphs remains an open problem. We approach this question in the dense-graph setting through the lens of graphons, the size-agnostic limit objects of dense graph sequences, to study how structural graph statistics behave across node-size scales. This perspective leads to DiPhon, a
diffusion framework for size-scalable graph generation. Specifically, we formulate a continuous
diffusion process on the graphon space via a Jacobi
stochastic differential equation (SDE), and propose DiPhon, a discretized graph-level process that mimics these dynamics on finite graphs. We further derive the corresponding reverse-time process, which requires access to the marginal score. For the Jacobi process, this score interestingly admits a tractable form, which we estimate from data via graph denoising and plug into the reverse process to generate graph samples. We prove that DiPhon matches exactly the first moment of the marginal distributions induced by the continuous graphon process, and approximates the second moment up to a closed-form discrepancy. Thus, DiPhon inherits key size-agnostic statistical properties of the graphon dynamics, providing a principled route toward scalable graph generation. Empirically, we demonstrate this scalability by training on small graphs and generating progressively larger graphs at inference time, without retraining, while preserving their core topological properties.
Read the full abstract ▾
All authors Sergio Rozada, Yiming Qin, Manuel Madeira, Pascal Frossard, Alejandro Ribeiro
Machine Learning · Cross-listed from cs.CV
Vision Foundation Models in Radiology: A Scoping Review of Data, Methodology, Evaluation and Clinical Translation
Alejandro Vergara-Richart, Xavier Rafael-Palou, Almudena Fuster-Matanzo et al. (+3 more)
Vision foundation models (VFMs) are increasingly being developed for radiological imaging, yet their definition, development and evaluation remain heterogeneous. We conducted a PRISMAScR scoping review of peer-reviewed studies published between January 2017 and March 2026 describing foundation models trained exclusively on radiological imaging data. Sixty-seven studies were included and mapped across three pillars: data scale and heterogeneity, architectural and pretraining scalability, and downstream transferability and generalization.
Datasets primarily covered
brain MRI, thoracoabdominal CT, and chest X-ray, ranging from fewer than 100,000 samples to multi-million-image cohorts.
Transformer-based architectures and self-supervised pretraining predominated, particularly masked image modeling, contrastive learning and multi-stage approaches. Evaluation focused mainly on
segmentation and classification, whereas cross-center, cross-scanner, anatomical and modality-shift validation was inconsistently reported.
Alignment with FUTURE-AI principles was uneven. Overall, radiology-specific VFMs show promising transferability, but clinical translation remains constrained by limited data representativeness, heterogeneous
benchmarks, incomplete reporting and insufficient deployment-oriented evaluation.
Read the full abstract ▾
All authors Alejandro Vergara-Richart, Xavier Rafael-Palou, Almudena Fuster-Matanzo, Ignacio Iborra Roncales, Ángel Alberich-Bayarri, Ana Jiménez-Pastor
Machine Learning · Cross-listed from cs.CR
Continual Learning With Participation Privacy: An Auditable Buffering-Aggregation Recipe
T-H. Hubert Chan, Elaine Shi, Mengshi Zhao et al. (+1 more)
Modern
federated and streaming learning systems often release intermediate models, so
privacy must hold for the full trajectory under adaptive interaction. Motivated by participation
privacy, we study single-edit neighboring user streams, where one insertion/deletion shifts all subsequent updates and defeats standard Hamming-neighbor continual-release analyses. We give an auditable modular recipe. A randomized buffering wrapper emits bins of size $[U,2U]$, reducing single-edit streams to a Hamming-style per-bin update stream with explicit backlog/delay guarantees, where $U$ is calibrated by the
privacy parameters $(\varepsilon,\delta)$. We then prove a certification
theorem identifying when a non-adaptive Hamming-neighbor DP proof for a continual primitive lifts to adaptive inputs: the primitive must use fresh per-round randomness and have a stable one-round
privacy profile under common adaptive context. Together, these ingredients yield trajectory-level $(\varepsilon,\delta)$-DP for single-edit streams using standard primitives (e.g., tree prefix sums), with an explicit
privacy–latency link via $U$.
Read the full abstract ▾
All authors T-H. Hubert Chan, Elaine Shi, Mengshi Zhao, Mingxun Zhou
Machine Learning
Causal Optimizer Interaction Calculus: Hidden Geometric Relaxation and Identifiable Interventions
Zavier Li
Optimizer experiments observe responses to algorithmic configurations without uniquely revealing hidden mechanisms. We develop a causal optimizer interaction calculus that separates pathwise realization, Mobius decomposition, and experimental identification. Under a fixed innovation coupling, every finite-horizon innovation-driven optimizer admits a behaviorally minimal pathwise realization. For any finite effect support and intervention design, an incidence operator gives the complete observational gauge, exact identifiability, sharp quotient stability, held-out predictions, and exact noiseless configuration complexity. Smooth hidden relaxation generates interactions through inverse hidden-state stiffness. Building on this structural law, we prove an observable-readout transfer
theorem: arbitrary smooth update or trace readouts inherit an explicit five-term interaction through first and second hidden responses. Unlike the reduced optimal value, a general readout has no universal interaction sign. Its Boolean effects remain exact integrals of continuous interaction
curvature and can therefore be identified by factorial interventions. We also derive Gaussian quotient minimax risk, exact confidence sets and tests, misspecification decomposition, certified downstream decisions, and optimal replication. A controlled real-data experiment on a 65-dimensional strongly convex logistic model validates the complete reduced-value chain. Boolean effects and independently integrated
curvature agree within 4.21e-11, while nine held-out continuous intensities agree within 8.88e-13. Gaussian campaigns attain the predicted coverage and power, and 4,500 real-minibatch observations reject an order-two interaction model. Neural trace audits provide complementary evidence that the declared response classes remain informative in nonconvex training.
Read the full abstract ▾
Machine Learning · Cross-listed from math.OC
Restricted Dynamic Geometric Complexity: Path-Space Reduction and Möbius–Jacobi Response
Zavier Li
Structured preconditioners restrict optimization to a small family of positive metrics, but endpoint condition-number reachability does not measure the
geometric effort required to reach a useful metric. We formulate this effort as a path-space value problem. Restricted dynamic
geometric complexity is the least affine-invariant length of an admissible metric path whose endpoint reaches a Hessian-relative generalized-
eigenvalue condition target. Path elimination gives an exact min-plus
semigroup and Bellman principle, while fixed-horizon kinetic energy is exactly squared complexity divided by twice the horizon. The main response result is global on a Hadamard state space: geodesic convexity produces a smooth intervention-cube path branch and a uniformly coercive Jacobi form, while one Green inverse generates the value Hessian, two-sided force-to-
curvature bounds, exact Möbius effects, and arbitrary prescribed finite-order responses. For the hard condition target, a bordered Jacobi–KKT
theorem differentiates the moving projection endpoint and multiplier on every regular active spectral stratum; its indefinite inverse also explains why hard-target interactions need not share the unconstrained sign. The theory specializes to affine-invariant positive-definite geometry. A determinant-one two-dimensional diagonal model has an exact target interval, a closed-form forced path, and a strictly negative-definite interaction
matrix. A moving diagonal Hessian gives a closed-form hard-target projection, multiplier, and pair effects of either sign, while a coordinate-sequential three-dimensional protocol yields an exact path metric strictly larger than the ambient projection distance. Thus the global Green and bordered hard-target responses are explicit laws of restricted metric-path elimination built on Bellman composition.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.RO
Validate the Dream Before You Trust Its Verdict: Admissibility for World-Model Simulators
Christian Oefinger, Finn Rasmus Schäfer, Korbinian Moller et al. (+2 more)
Across robotics, World Models (WMs) are increasingly used to evaluate action policies by simulating the consequences of actions in an imagined world, and returning a success or safety verdict. Yet a verdict is only as trustworthy as the WM that produced it, and the WM itself needs to be certified. In
video-generation WMs, fidelity metrics such as Fréchet
Video Distance (FVD) reward visual realism, but ignore whether the world responds correctly to the policy's actions, including those unseen in training. Classical simulation-based validation assumes a trusted simulator evaluating an untrusted policy, whereas generative WMs are themselves unverified learned artifacts. Hence, we argue that any WM used as a test oracle must first be accredited before its verdicts can serve as evidence. Building on credibility practices from safety-critical simulation, including Verification, Validation & Accreditation (VV&A), Safety of the Intended Functionality (SOTIF), and scenario-based testing standards, we define an admissibility ladder (L0-L4) that a WM must climb before its closed-loop verdicts are accepted as assurance evidence. Our framework is embodiment-agnostic, and is instantiated in autonomous driving (AD), where assurance methods for traditional simulation are most mature. Applied to two driving WMs, the lower rungs reveal a reversal: the model that ranks higher on visual generation quality (L0) ranks lower on action-following (L1-L2), so visual fidelity does not predict the action-robustness a closed-loop verdict depends on.
Read the full abstract ▾
All authors Christian Oefinger, Finn Rasmus Schäfer, Korbinian Moller, Mattia Piccinini, Johannes Betz
Machine Learning
Predicting LLM Safety Before Release by Simulating Deployment
Marcus Williams, Hannah Sheahan, Cameron Raymond et al. (+8 more)
Pre-deployment safety evaluations aim to inform the downstream risks of releasing a new AI model. Yet most evaluations provide limited evidence about how often undesired model behavior will occur in deployment: they generally have insufficient coverage, are unrepresentative, and are generally recognizable as tests. To address these concerns, we study a simple way to simulate a model deployment: starting from de-identified conversations from a previous model deployment, we hold fixed the initial conversation prefix and regenerate the next response using a candidate model. The resulting responses can then both be audited for novel misalignments and used to estimate the prevalence of model misbehavior before deployment. We evaluate deployment simulation across four GPT-5-series deployments, using registered, outcome-blinded predictions for GPT-5.4 and retrospective analyses of three earlier releases. We find that deployment simulation produces informative estimates of post-deployment misbehavior rates and outperforms baselines based on adversarially selected production data; its evaluation-awareness point estimates were also much closer to production traffic than those from traditional evaluations. We also identify the realism of tool resampling as a central challenge for further improving predictions and share results suggesting that this challenge is surmountable even in complex tool-use settings. Finally, we show that deployment simulation can be seeded from public chat
datasets and remain informative about production misbehavior rates, suggesting a path for external researchers to run deployment-grounded evaluations without access to private production logs. Overall, deployment simulation helps evaluators forecast how
language models will behave in the real world and supports more quantitative assessment of deployment risk.
Read the full abstract ▾
All authors Marcus Williams, Hannah Sheahan, Cameron Raymond, Tomek Korbak, Deng Pan, Peilin Yang, Leon Maksin, Ningyi Xie, Phillip Guo, Ian Kivlichan, Micah Carroll
Machine Learning · Cross-listed from cs.CV
Comparative Study of Domain-adapted VLMs for General Document Visual Question Answering
Miguel Lopez-Duran, Elena Marrero, Julian Fierrez et al. (+8 more)
Document Visual Question Answering (DocVQA) presents a complex
multimodal challenge, requiring models to exploit visual, textual, and layout information from documents. Although
Vision-
Language Models (VLMs) have shown remarkable performance in text-
vision tasks, their robustness and transferability to different document domains remains underexplored. In this study, we present a comprehensive evaluation of 8 open-source pretrained VLMs on DocVQA in three different document domains: industrial documents of varying type, infographics, and presentation slides. We systematically assess model performance under zero-shot evaluations, fully supervised finetuning with inter- and intra-
dataset evaluations, and few-shot learning evaluations of knowledge transfer between domains. Our findings demonstrate that while large pretrained VLMs possess strong zero-shot baselines for structured layouts, their performance strongly decreases on visually complex layouts of infographics and slides. Although parameter scaling is a dominant factor on performance, supervised finetuning yields higher relative gains in smaller architectures. Furthermore, our cross-domain and few-shot experiments show that visual understanding is the main bottleneck for DocVQA, not a lack of knowledge from the VLMs. Using 50 target domain samples, the models finetuned in DocVQA with
datasets of different domains rapidly adapt to the target domain documents, even surpassing their fully supervised counterparts in some cases.
Read the full abstract ▾
All authors Miguel Lopez-Duran, Elena Marrero, Julian Fierrez, Marta Robledo-Moreno, Ruben Vera-Rodriguez, Daniel DeAlcala, Aythami Morales, Ruben Tolosana, Oscar Delgado, Alvaro Ortigosa, Javier Ortega-Garcia
Machine Learning
Entropy Pacing Policy Optimization for Multi-Task Agentic Reinforcement Learning
Zetian Hu, Shunyu Liu, Junjie Zhang et al. (+4 more)
Recent breakthroughs of Reinforcement Learning (RL) have highlighted its potential for complex agentic Large
Language Model (LLM) tasks. However, existing efforts largely focus on single-task settings, whereas real-world deployment necessitates a generalist
agent capable of solving multiple tasks simultaneously. In this work, we identify a critical yet underexplored phenomenon in multi-task agentic RL: different tasks can exhibit exploration-exploitation pace mismatch. Specifically, easier tasks may converge early to low-
entropy policies that hinder learning on harder tasks, while harder tasks can, in turn, push easier tasks back toward high-
entropy exploration. This back-and-forth interaction creates inter-task
entropy crossovers and frequent
entropy spikes. Inspired by this observation, we introduce
Entropy Pacing Policy Optimization (EPPO) for multi-task agentic LLMs, which coordinates
entropy across tasks to stabilize multi-task optimization. At the core of EPPO is a task-wise dynamic clipping mechanism that replaces the fixed clipping threshold in Group Relative Policy Optimization (GRPO) with a task
entropy-aware adaptive bound, tightening updates for over-confident tasks while relaxing them for under-explored ones. Experiments on the multi-task agentic
benchmarks demonstrate that the proposed EPPO yields results superior to its counterparts.
Read the full abstract ▾
All authors Zetian Hu, Shunyu Liu, Junjie Zhang, Yongcheng Jing, Ting-En Lin, Yongbin Li, Dacheng Tao
Machine Learning
The Anatomy of Implicit Bias: Information Allocation in Neural Network Training
Zhang Gongyue, Wang Zhiyong, Liu Donghan et al. (+3 more)
Implicit bias is usually explained as the preference of an optimization process for certain final solutions and their geometry. This view helps explain where a model finally stops. It gives less direct explanation of how this bias is formed during training. This paper proposes a training-time information allocation view. Under this view, optimization forms a writing pattern for error signals across parameter paths, coordinate channels, and sample regions. This paper builds a set of observable allocation diagnostics. These diagnostics include gradient demand, actual update injection, coordinate gain induced by exponential moving averages, channel-level update ratios, and sample-wise loss distributions. To separate training progress from internal allocation, this paper introduces a collapse–persistence analysis. Under matched training loss, if external loss statistics collapse but internal allocation ratios remain separated, then the factor changes the internal allocation of the training signal. Overall, this paper extends the analysis of implicit bias from final-solution geometry to training-time signal allocation. The main claim is that implicit bias is not only reflected by the final solution. It is also reflected by which parameter paths, coordinate channels, and sample regions receive the error signal first and more strongly during training. Based on this view, this paper places different training factors into a unified information-allocation diagnostic framework. The framework gives a mechanism-level explanation of training-time implicit bias. It also provides a basis for future optimization methods that control training progress and signal allocation separately.
Read the full abstract ▾
All authors Zhang Gongyue, Wang Zhiyong, Liu Donghan, Ren Weihong, Sheng Yixuan, Liu Honghai
Machine Learning
Prior-matched evaluation of operational Earth-observation classifiers: a three-number reporting method demonstrated on Sentinel-1 internal-wave detection
João Pinelo, João Gonçalves, Arun Shukla et al. (+1 more)
The Internal Waves Service screens the Sentinel-1 Wave-mode archive for internal solitary waves, routing detections to experts whose adjudication time is the resource the effort exists to conserve. Because
attention is the cost of error, precision leads. Its classifier was trained and reported at a one-to-one class balance, fixed before the operational rate could be known. That rate has since emerged at roughly one scene in twenty, and a balanced-test score badly overstates the precision a validator meets. A model that scores 0.794 balanced-test precision scores 0.192 in real operation: the gap is a systematic artefact of reporting at the wrong prior, invisible to the metric most work quotes. We show the mismatch to be an evaluation problem in the costume of a training one at a fixed recall, prior correction and calibration cannot move precision, and answer it with a prior-matched reporting method based on three numbers: balanced-test, operational-prior, and real post-deployment, whose contrast is the honest measure. A precision-first, leakage-controlled development cycle then improves the classifier lever by lever, each promoted only against a pre-registered margin; negative variety and the aggregation head lifting, capacity paying once then stopping, calibration inert, so the honest negatives are as much a result as the gains. Holding recall at a floor of 0.80 and certifying against a sealed, single-read lockbox, the promoted model reports 0.927 precision at the operational prior; an out-of-time check confirms discrimination transfers to unseen periods while a fixed operating point does not. Prior-matched reporting, begin balanced, then move to the prior as the stream reveals it, transfers to any operational Earth-observation service bootstrapping a rare-event detector under a prior it has yet to discover.
Read the full abstract ▾
All authors João Pinelo, João Gonçalves, Arun Shukla, Adriana Santos-Ferreira
Machine Learning
Fractal KV-Cache Archives: Lossless Symbolic Storage with In-Place Retrieval for Long-Context LLM Inference
Vladimir Gusev
The key-value (KV) cache dominates the
memory cost of long-context autoregressive inference, and a growing body of work compresses it through quantization, eviction, or offloading. We study a complementary question: once a position's KV state has been quantized to codebook indices, how should the resulting symbol stream be stored, and can the storage layer do more than store? A family of contractive iterated-map codes that serialize a symbol sequence into a sequence of low-dimensional real vectors is revisited, and it is shown that they form a natural archive format for a quantized KV cache with the following features. The method provides exactly the access pattern a growing cache requires. It is lossless, it runs in linear time, and supports O(1) random access and O(1) amortized append. A controlled study of the quantizer feeding this archive is conducted on GPT-2 with 1024-
token contexts. Keeping a small exact window (4
attention sinks plus 32 recent
tokens) and archiving the rest, per-head residual vector quantization reduces the archived cache by 36-54x relative to an fp16 cache at a perplexity cost of 11-15%, and we quantify a sharp key/value asymmetry - quantizing keys is roughly 4x more damaging than quantizing values, consistent with prior low-bit KV work - and use it to allocate bits in a hybrid scheme. Finally, we show the archive is simultaneously a search index: approximate substring queries execute directly on the stored vectors, and matched context is decoded from the matched vector without ever materializing the surrounding text. We further characterize the archive's operating range: truncating stored points yields a lossy regime whose distortion we localize, and probability-weighting the maps recovers arithmetic coding, exposing a trade-off between rate efficiency, random access, and
memory. We release all code; every number reproduces on a laptop CPU.
Read the full abstract ▾