Machine Learning · Cross-listed from cs.CR
SEDULity: A Proof-of-Learning Framework for Distributed and Secure Blockchains with Efficient Useful Work
Weihang Cao, Mustafa Doger and Sennur Ulukus
The security and decentralization of Proof-of-Work (PoW) have been well-tested in existing blockchain systems. However, its tremendous energy waste has raised concerns about sustainability. Proof-of-Useful-Work (PoUW) aims to redirect the meaningless computation to meaningful tasks such as solving machine learning (ML) problems, giving rise to the branch of Proof-of-Learning (PoL). While previous studies have proposed various PoLs, they all, to some degree, suffer from security, decentralization, or efficiency issues. In this paper, we propose a PoL framework that trains ML models efficiently while maintaining blockchain security in a fully distributed manner. We name the framework SEDULity, which stands for a Secure, Efficient, Distributed, and Useful Learning-based blockchain system. Specifically, we encode the template block into the training process and design a useful function that is difficult to solve but relatively easy to verify, as a substitute for the PoW puzzle. We show that our framework is distributed, secure, and efficiently trains ML models. We further demonstrate that the proposed PoL framework can be extended to other types of useful work and design an incentive mechanism to incentivize task verification. We show theoretically that a rational miner is incentivized to train fully honestly with well-designed system parameters. Finally, we present simulation results to demonstrate the performance of our framework and validate our analysis.
Read the full abstract ▾
Machine Learning
From Code to Field: Evaluating the Robustness of Convolutional Neural Networks for Disease Diagnosis in Mango Leaves
Gabriel Vitorino de Andrade, Saulo Roberto dos Santos, Itallo Patrick Castro Alves da Silva et al. (+2 more)
The validation and verification of artificial intelligence (AI) models through robustness assessment are essential to guarantee the reliable performance of intelligent systems facing real-world challenges, such as image corruptions including noise, blurring, and weather variations. Despite the global importance of mango (Mangifera indica L.), there is a lack of studies on the robustness of models for the diagnosis of disease in its leaves. This paper proposes a methodology to evaluate convolutional neural networks (CNNs) under adverse conditions. We adapted the MangoLeafDB
dataset, generating MangoLeafDB-C with 19 types of artificial corruptions at five severity levels. We conducted a
benchmark comparing five architectures: ResNet-50, ResNet-101, VGG-16, Xception, and LCNN (the latter being a lightweight architecture designed specifically for mango leaf diagnosis). The metrics include the F1 score, the corruption error (CE) and the relative mean corruption error (relative mCE). The results show that LCNN outperformed complex models in corruptions that can be present in real-world scenarios such as Defocus Blur, Motion Blur, while also achieving the lowest mCE. Modern architectures (e.g., ResNet-101) exhibited significant performance degradation in corrupted scenarios, despite their high accuracy under ideal conditions. These findings suggest that lightweight and specialized models may be more suitable for real-world applications in edge devices, where robustness and efficiency are critical. The study highlights the need to incorporate robustness assessments in the development of intelligent systems for agriculture, particularly in regions with technological limitations.
Read the full abstract ▾
All authors Gabriel Vitorino de Andrade, Saulo Roberto dos Santos, Itallo Patrick Castro Alves da Silva, Emanuel Adler Medeiros Pereira, Erick de Andrade Barboza
Machine Learning · Cross-listed from stat.ML
Universality of high-dimensional scaling limits of stochastic gradient descent
Reza Gheissari and Aukosh Jagannath
We consider statistical tasks in high dimensions whose loss depends on the data only through its projection into a fixed-dimensional subspace spanned by the parameter vectors and certain ground truth vectors. This includes classifying mixture distributions with cross-
entropy loss with one and two-layer networks, and learning single and multi-index models with one and two-layer networks. When the data is drawn from an isotropic Gaussian mixture distribution, it is known that the
evolution of a finite family of summary statistics under
stochastic gradient descent converges to an autonomous ordinary differential equation (ODE), as the dimension and sample size go to $\infty$ and the step size goes to $0$ commensurately. Our main result is that these ODE limits are universal in that this limit is the same whenever the data is drawn from mixtures of arbitrary product distributions whose first two moments match the corresponding Gaussian distribution, provided the initialization and ground truth vectors are coordinate-delocalized. We complement this by proving two corresponding non-universality results. We provide a simple example where the ODE limits are non-universal if the initialization is coordinate aligned. We also show that the
stochastic differential equation limits arising as fluctuations of the summary statistics around their ODE's fixed points are not universal.
Read the full abstract ▾
Machine Learning
StutterFuse: Mitigating Modality Collapse in Stuttering Detection with Jaccard-Weighted Metric Learning and Gated Fusion
Guransh Singh and Md Shah Fahad
Stuttering detection breaks down when disfluencies overlap. Existing parametric models struggle to distinguish complex, simultaneous disfluencies (e.g., a 'block' with a 'prolongation') due to the scarcity of these specific combinations in training data. While
Retrieval-Augmented Generation (RAG) has revolutionized NLP by grounding models in external knowledge, this paradigm remains unexplored in pathological
speech processing. To bridge this gap, we introduce StutterFuse, the first
Retrieval-Augmented Classifier (RAC) for multi-label stuttering detection. By conditioning a Conformer encoder on a non-parametric
memory bank of clinical examples, we allow the model to classify by reference rather than memorization. We further identify and solve "Modality Collapse", an "Echo Chamber" effect where naive
retrieval boosts recall but degrades precision. We mitigate this using: (1) SetCon, a Jaccard-Weighted Metric Learning objective that optimizes for multi-label set similarity, and (2) a Gated Mixture-of-Experts fusion strategy that dynamically arbitrates between acoustic evidence and retrieved context. On the SEP-28k
dataset, StutterFuse achieves a weighted F1-score of 0.65, outperforming strong baselines and demonstrating remarkable zero-shot cross-lingual generalization.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Temporal Tokenization Strategies for Event Sequence Modeling with Large Language Models
Zefang Liu, Nam H. Nguyen, Yinzhu Quan et al. (+1 more)
Representing continuous time is a critical and under-explored challenge in modeling temporal event sequences with large
language models (LLMs). Various strategies like byte-level representations or calendar
tokens have been proposed. However, the optimal approach remains unclear, especially given the diverse statistical distributions of real-world event data, which range from smooth log-normal to discrete, spiky patterns. This paper presents a systematic empirical study of temporal tokenization for modeling event sequences with LLMs, comparing distinct encoding strategies: naive numeric strings, high-precision byte-level representations, human-semantic calendar
tokens, classic uniform binning, and adaptive residual scalar quantization. We evaluate these strategies by fine-tuning LLMs on real-world
datasets that exemplify these diverse distributions. Our analysis reveals that no single strategy is universally superior; instead, prediction performance depends heavily on aligning the tokenizer with the data's statistical properties, highlighting temporal tokenization as a critical yet often overlooked design dimension in LLM-based event modeling.
Read the full abstract ▾
All authors Zefang Liu, Nam H. Nguyen, Yinzhu Quan, Shi-Xiong Zhang
Machine Learning
LightTopoGAT: Enhancing Graph Attention Networks with Topological Features for Efficient Graph Classification
Ankit Sharma and Sayan Roy Gupta
Graph Neural Networks have demonstrated significant success in graph classification tasks, yet they often require substantial computational resources and struggle to capture global graph properties effectively. We introduce LightTopoGAT, a lightweight graph
attention network that enhances node features through topological augmentation by incorporating node degree and local clustering coefficient to improve graph representation learning. The proposed approach maintains parameter efficiency through streamlined
attention mechanisms while integrating structural information that is typically overlooked by local message passing schemes. Through comprehensive experiments on three
benchmark datasets, MUTAG, ENZYMES, and
PROTEINS, we show that LightTopoGAT achieves superior performance compared to established baselines including GCN, GraphSAGE, and standard GAT, with a 6.6 percent improvement in accuracy on MUTAG and a 2.2 percent improvement on
PROTEINS. Ablation studies further confirm that these performance gains arise directly from the inclusion of topological features, demonstrating a simple yet effective strategy for enhancing graph neural network performance without increasing architectural complexity.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Do-Undo Bench: Reversibility for Action Understanding in Image Generation
Shweta Mahajan, Shreya Kadambi, Hoang Le et al. (+4 more)
We introduce the Do-Undo task and
benchmark to address a critical gap in
vision-
language models: understanding and generating plausible scene transformations driven by real-world actions. Unlike prior work that relies on
prompt-based image generation and editing to perform action-conditioned image manipulation, our training hypothesis requires models to simulate the outcome of a real-world action and then reverse it to the original state. This forward-reverse requirement tests genuine cause-and-effect understanding rather than stylistic or semantic edits. We curate a high-quality
benchmark of reversible actions from real-world scenarios to enable robust action grounding. Our experiments reveal that current models struggle with action reversibility, highlighting the need to evaluate action understanding. Do-Undo provides an intuitive testbed for evaluating and advancing action-aware generation in
multimodal systems that must reason about real-world dynamics.
Read the full abstract ▾
All authors Shweta Mahajan, Shreya Kadambi, Hoang Le, Rajeev Yasarla, Apratim Bhattacharyya, Munawar Hayat, Fatih Porikli
Machine Learning · Cross-listed from cs.CL
Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models
Boxin Wang, Chankyu Lee, Nayeon Lee et al. (+9 more)
Building general-purpose
reasoning models with reinforcement learning (RL) entails substantial cross-domain heterogeneity, including large variation in inference-time response lengths and verification latency. Such variability complicates the RL infrastructure, slows training, and makes training curriculum (e.g., response length extension) and hyperparameter selection challenging. In this work, we propose cascaded domain-wise reinforcement learning (Cascade RL) to develop Nemotron-Cascade, capable of operating in both instruct and deep thinking modes, without any performance gap relative to a thinking-only counterpart. Departing from conventional approaches that blend heterogeneous
prompts from different domains, Cascade RL orchestrates sequential, domain-wise RL, reducing engineering complexity and delivering state-of-the-art performance across a wide range of
benchmarks. Notably, RLHF for
alignment, when used as a pre-step, boosts the model's
reasoning ability far beyond mere preference optimization, and subsequent domain-wise RLVR stages rarely degrade the
benchmark performance attained in earlier domains and may even improve it (see an illustration in Figure 1). Our 14B model, after RL, outperforms its SFT teacher, DeepSeek-R1-0528, on LiveCodeBench v5/v6/Pro and achieves silver-medal performance in the 2025 International Olympiad in Informatics (IOI). We transparently share our training and data recipes.
Read the full abstract ▾
All authors Boxin Wang, Chankyu Lee, Nayeon Lee, Sheng-Chieh Lin, Wenliang Dai, Yang Chen, Yangyi Chen, Zhuolin Yang, Zihan Liu, Mohammad Shoeybi, Bryan Catanzaro, Wei Ping
Machine Learning · Cross-listed from cs.CL
Textual Gradients are a Flawed Metaphor for Automatic Prompt Optimization
Daniel Melcer, Qi Chen, Wen-Hao Chiang et al. (+3 more)
A well-engineered
prompt can increase the performance of large
language models; automatic
prompt optimization techniques aim to increase performance without requiring human effort to tune the
prompts. One leading class of
prompt optimization techniques introduces the analogy of textual gradients. We investigate the behavior of these textual gradient methods through a series of experiments and case studies. While such methods often result in a performance improvement, our experiments suggest that the gradient analogy does not accurately explain their behavior. Our insights may inform the selection of
prompt optimization strategies, and development of new approaches.
Read the full abstract ▾
All authors Daniel Melcer, Qi Chen, Wen-Hao Chiang, Shweta Garg, Pranav Garg, Christian Bock
Machine Learning
Verification of Unknown Dynamical Systems via Autoencoder Latent Space
Robert Reed, Luca Laurenti and Morteza Lahijanian
Formal verification provides a powerful framework for proving that dynamical systems satisfy their specifications. However, these techniques face scalability challenges in high-dimensional settings, as they often rely on state-space discretization which grows exponentially with dimension. Learning-based approaches to dimensionality reduction have shown great potential to alleviate this problem. However, ensuring correctness of latent space verification results remains an open question. In this work, we provide a formal approach to reduce the dimensionality of systems via convex autoencoders and learn the dynamics in the latent space through a kernel-based method. We then construct a finite abstraction from the learned model in the latent space and guarantee that the abstraction contains the true behaviors of the original system. We show that the verification results in the latent space can be mapped back to the original system. Finally, we demonstrate the approach on multiple systems, including a 26D system controlled by a neural network, showing significant scalability improvements.
Read the full abstract ▾
Machine Learning
Image Diffusion Preview with Consistency Solver
Fu-Yun Wang, Hao Zhou, Liangzhe Yuan et al. (+8 more)
The slow inference process of image
diffusion models significantly degrades interactive user experiences. To address this, we introduce
Diffusion Preview, a novel paradigm employing rapid, low-step sampling to generate preliminary outputs for user evaluation, deferring full-step refinement until the preview is deemed satisfactory. Existing acceleration methods, including training-free solvers and post-training
distillation, struggle to deliver high-quality previews or ensure consistency between previews and final outputs. We propose ConsistencySolver derived from general linear multistep methods, a lightweight, trainable high-order solver optimized via Reinforcement Learning, that enhances preview quality and consistency. Experimental results demonstrate that ConsistencySolver significantly improves generation quality and consistency in low-step scenarios, making it ideal for efficient preview-and-refine workflows. Notably, it achieves FID scores on-par with Multistep DPM-Solver using 47% fewer steps, while outperforming
distillation baselines. Furthermore, user studies indicate our approach reduces overall user interaction time by nearly 50% while maintaining generation quality. Code is available at https://github.com/G-U-N/consolver.
Read the full abstract ▾
All authors Fu-Yun Wang, Hao Zhou, Liangzhe Yuan, Sanghyun Woo, Boqing Gong, Bohyung Han, Ming-Hsuan Yang, Han Zhang, Yukun Zhu, Ting Liu, Long Zhao
Machine Learning · Cross-listed from cs.CL
ReFusion: A Diffusion Large Language Model with Parallel Autoregressive Decoding
Jia-Nan Li, Jian Guan, Wei Wu et al. (+1 more)
Autoregressive models (ARMs) are hindered by slow sequential inference. While masked
diffusion models (MDMs) offer a parallel alternative, they suffer from critical drawbacks: high computational overhead from precluding Key-Value (KV) caching, and incoherent generation arising from learning dependencies over an intractable space of
token combinations. To address these limitations, we introduce ReFusion, a novel masked
diffusion model that integrates sequence reorganization into the causal
attention framework. By elevating parallel decoding from the
token level to a higher slot level, ReFusion interleaves inter-slot
diffusion-based selection with intra-slot autoregressive infilling, while reordering newly generated slots ahead of the remaining masks after each iteration. Consequently, this design simultaneously unlocks full KV cache reuse and reduces learning complexity from an intractable
token combination space to a manageable slot-level permutation space. Extensive experiments on seven diverse
benchmarks show that ReFusion not only overwhelmingly surpasses prior MDMs with a 34% performance gain and an over 18$\times$ speedup on average, but also bridges the performance gap to strong ARMs while maintaining a 2.33$\times$ average speedup.
Read the full abstract ▾
All authors Jia-Nan Li, Jian Guan, Wei Wu, Chongxuan Li
Machine Learning
DP-CSGP: Differentially Private Stochastic Gradient Push with Compressed Communication
Zehan Zhu, Heng Zhao, Yan Huang et al. (+3 more)
In this paper, we propose a Differentially Private
Stochastic Gradient Push with Compressed communication (termed DP-CSGP) for decentralized learning over directed graphs. Different from existing works, the proposed
algorithm is designed to maintain high model utility while ensuring both rigorous differential
privacy (DP) guarantees and efficient communication. For general non-convex and smooth objective functions, we show that the proposed
algorithm achieves a tight utility bound of $\mathcal{O}\left( \sqrt{d\log \left( \frac{1}{\delta} \right)}/(\sqrt{n}J\epsilon) \right)$ ($J$ and $d$ are the number of local samples and the dimension of decision variables, respectively) with $\left(\epsilon, \delta\right)$-DP guarantee for each node, matching that of decentralized counterparts with exact communication. Extensive experiments on
benchmark tasks show that, under the same
privacy budget, DP-CSGP achieves comparable model accuracy with significantly lower communication cost than existing decentralized counterparts with exact communication.
Read the full abstract ▾
All authors Zehan Zhu, Heng Zhao, Yan Huang, Joey Tianyi Zhou, Shouling Ji, Jinming Xu
Machine Learning
Superposition as Lossy Compression: Measure with Sparse Autoencoders and Connect to Adversarial Vulnerability
Leonard Bereska, Zoe Tzifa-Kratira, Reza Samavi et al. (+1 more)
Neural networks achieve remarkable performance through superposition: encoding multiple features as overlapping directions in activation space rather than dedicating individual
neurons to each feature. This challenges interpretability, yet we lack principled methods to measure superposition. We present an information-theoretic framework measuring a neural representation's effective degrees of freedom. We apply Shannon
entropy to sparse autoencoder activations to compute the number of effective features as the minimum
neurons needed for interference-free encoding. Equivalently, this measures how many "virtual
neurons" the network simulates through superposition. When networks encode more effective features than actual
neurons, they must accept interference as the price of compression. Our metric strongly correlates with ground truth in toy models, detects minimal superposition in algorithmic tasks, and reveals systematic reduction under dropout. Layer-wise patterns mirror intrinsic dimensionality studies on Pythia-70M. The metric also captures developmental dynamics, detecting sharp feature consolidation during grokking. Surprisingly,
adversarial training can increase effective features while improving robustness, contradicting the hypothesis that superposition causes vulnerability. Instead, the effect depends on task complexity and network capacity: simple tasks with ample capacity allow feature expansion (abundance regime), while complex tasks or limited capacity force reduction (scarcity regime). By defining superposition as lossy compression, this work enables principled measurement of how neural networks organize information under computational constraints, connecting superposition to
adversarial robustness.
Read the full abstract ▾
All authors Leonard Bereska, Zoe Tzifa-Kratira, Reza Samavi, Efstratios Gavves
Machine Learning · Cross-listed from stat.ML
A Nonparametric Statistics Approach to Feature Selection in Deep Neural Networks with Theoretical Guarantees
Junye Du, Zhenghao Li, Zhutong Gu et al. (+1 more)
This paper tackles the problem of feature selection in a highly challenging setting: $\mathbb{E}(y | \boldsymbol{x}) = G(\boldsymbol{x}_{\mathcal{S}_0})$, where $\mathcal{S}_0$ is the set of relevant features and $G$ is an unknown, potentially nonlinear function subject to mild smoothness conditions. Our approach begins with feature selection in deep neural networks, then generalizes the results to Hölder smooth functions by exploiting the strong approximation capabilities of neural networks. Unlike conventional optimization-based deep learning methods, we reformulate neural networks as index models and estimate $\mathcal{S}_0$ using the second-order Stein's formula. This gradient-descent-free strategy guarantees feature selection consistency with a sample size requirement of $n = \Omega(p^2)$, where $p$ is the feature dimension. To handle high-dimensional scenarios, we further introduce a screening-and-selection mechanism that achieves nonlinear selection consistency when $n = \Omega(s \log p)$, with $s$ representing the sparsity level. Additionally, we refit a neural network on the selected features for prediction and establish performance guarantees under a relaxed sparsity assumption. Extensive simulations and real-data analyses demonstrate the strong performance of our method even in the presence of complex feature interactions.
Read the full abstract ▾
All authors Junye Du, Zhenghao Li, Zhutong Gu, Long Feng
Machine Learning · Cross-listed from cs.CV
Pancakes: Consistent Multi-Protocol Image Segmentation Across Biomedical Domains
Marianne Rakic, Siyu Gai, Etienne Chollet et al. (+2 more)
A single biomedical image can be meaningfully segmented in multiple ways, depending on the desired application. For instance, a
brain MRI can be segmented according to tissue types, vascular territories, broad anatomical regions, fine-grained anatomy, or pathology, etc. Existing automatic
segmentation models typically either (1) support only a single protocol, the one they were trained on, or (2) require labor-intensive manual prompting to specify the desired
segmentation. We introduce Pancakes, a framework that, given a new image from a previously unseen domain, automatically generates multi-label
segmentation maps for multiple plausible protocols, while maintaining semantic consistency across related images. Pancakes introduces a new problem formulation that is not currently attainable by existing foundation models. In a series of experiments on seven held-out
datasets, we demonstrate that our model can significantly outperform existing foundation models in producing several plausible whole-image
segmentations, that are semantically coherent across images.
Read the full abstract ▾
All authors Marianne Rakic, Siyu Gai, Etienne Chollet, John V. Guttag, Adrian V. Dalca
Machine Learning · Cross-listed from physics.flu-dyn
Adaptive Sampling for Hydrodynamic Stability
Anshima Singh and David J. Silvester
An adaptive sampling approach for efficient detection of bifurcation boundaries in parametrized fluid flow problems is presented herein. The study extends the machine-learning approach of Silvester~(J. Comput. Phys., 553 (2026), 114743), where a classifier network was trained on preselected simulation data to identify bifurcated and nonbifurcated flow regimes. In contrast, the proposed methodology introduces adaptivity through a flow-based deep generative model that automatically refines the sampling of the parameter space. The strategy has two components: a classifier network maps the flow parameters to a bifurcation probability, and a probability density estimation technique (KRnet) for the generation of new samples at each adaptive step. The classifier output provides a probabilistic measure of flow stability, and the Shannon
entropy of these predictions is employed as an uncertainty indicator. KRnet is trained to approximate a probability density function that concentrates sampling in regions of high
entropy, thereby directing computational effort towards the evolving bifurcation boundary. This coupling between classification and generative modeling establishes a feedback-driven adaptive learning process analogous to error-indicator based refinement in contemporary partial differential equation solution strategies. Starting from a uniform parameter distribution, the new approach achieves accurate bifurcation boundary identification with significantly fewer Navier–Stokes simulations, providing a scalable foundation for high-dimensional stability analysis.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ME
Actively Learning Joint Contours of Multiple Computer Experiments
Shih-Ni Prim, Kevin R. Quinlan, Paul Hawkins et al. (+2 more)
Contour location—the process of sequentially training a surrogate model to identify the design inputs that result in a pre-specified response value from a single computer experiment—is a well-studied active learning problem. Here, we tackle a related but distinct problem: identifying the input configuration that returns pre-specified values of multiple computer experiments simultaneously. Motivated by computer experiments of the rotational torques acting upon a vehicle in flight, we aim to identify stable flight conditions that result in zero torque forces. We propose a “joint contour location” (jCL) scheme that strikes a strategic balance between exploring the multiple response surfaces while exploiting learning of the intersecting contours. Rather than working exploration and exploitation into a single acquisition function, we devise two distinct acquisition schemes with a decision rule to choose between the two, which also provides a natural stopping criterion if no solution is present. We employ traditional Gaussian processes (GPs), multitask GPs, and deep GPs, but our jCL procedure is applicable to any surrogate that can provide posterior predictive distributions. Our jCL designs significantly outperform existing (single response) CL strategies, optimization-based alternatives, and previous strategies for targeting joint contours, enabling us to efficiently locate the optimal configurations for our motivating computer experiments.
Read the full abstract ▾
All authors Shih-Ni Prim, Kevin R. Quinlan, Paul Hawkins, Jagadeesh Movva, Annie S. Booth
Machine Learning · Cross-listed from physics.geo-ph
Enhancing lithological interpretation from petrophysical well log of IODP expedition 390/393 using machine learning
Raj Sahu and Saumen Maiti
Enhanced lithological interpretation from well logs plays a key role in geological resource exploration and mapping, as well as in geo-environmental modeling studies. Core and cutting information is useful for making sound interpretations of well logs; however, these are rarely collected at each depth due to high costs. Moreover, well log interpretation using traditional methods is constrained by poor borehole conditions. Traditional statistical methods are mostly linear, often failing to discriminate between lithology and rock facies, particularly when dealing with overlapping well log signals characterized by the structural and compositional variation of rock types. In this study, we develop multiple supervised and unsupervised machine learning
algorithms to jointly analyze multivariate well log data from Integrated Ocean Drilling Program (IODP) expeditions 390 and 393 for enhanced lithological interpretations. Among the
algorithms, Logistic Regression, Decision Trees, Gradient Boosting, Support Vector Machines (SVM), k-Nearest Neighbors (KNN), and Multi-Layer Perceptron (MLP) neural network models, the Decision Tree and Gradient Boosting models outperformed the others, achieving an accuracy of 0.9950 and an F1-score of 0.9951. While unsupervised machine learning (ML) provides the foundation for cluster information that inherently supports the classification
algorithm, supervised ML is applied to devise a data-driven lithology clustering mechanism for IODP
datasets. The joint ML-based method developed here has the potential to be further explored for analyzing other well log
datasets from the world's oceans.
Read the full abstract ▾
Machine Learning
Async Control: Stress-testing Asynchronous Control Measures for LLM Agents
Asa Cooper Stickland, Jan Michelfeit, Arathi Mani et al. (+6 more)
LLM-based software engineering
agents are increasingly used in real-world development tasks, often with access to sensitive data or security-critical codebases. Such
agents could intentionally sabotage these codebases if they were misaligned. We investigate asynchronous monitoring, in which a monitoring system reviews
agent actions after the fact. Unlike synchronous monitoring, this approach does not impose runtime latency, while still attempting to disrupt attacks before irreversible harm occurs. We treat monitor development as an
adversarial game between a blue team (who design monitors) and a red team (who create sabotaging
agents). We attempt to set the game rules such that they upper bound the sabotage potential of an
agent based on Claude 4.1 Opus. To ground this game in a realistic, high-stakes deployment scenario, we develop a suite of 5 diverse software engineering environments that simulate tasks that an
agent might perform within an AI developer's internal infrastructure. Over the course of the game, we develop an ensemble monitor that achieves a 6% false negative rate at 1% false positive rate on a held out test environment. Then, we estimate risk of sabotage at deployment time by extrapolating from our monitor's false negative rate. We describe one simple model for this extrapolation, present a sensitivity analysis, and describe situations in which the model would be invalid. Code is available at: https://github.com/UKGovernmentBEIS/async-control.
Read the full abstract ▾
All authors Asa Cooper Stickland, Jan Michelfeit, Arathi Mani, Charlie Griffin, Ollie Matthews, Tomek Korbak, Rogan Inglis, Oliver Makins, Alan Cooney
Machine Learning · Cross-listed from q-bio.NC
A Deep Learning Model of Mental Rotation Informed by Interactive VR Experiments
Raymond Khazoum, Daniela Fernandes, Aleksandr Krylov et al. (+2 more)
Mental rotation – the ability to compare objects seen from different viewpoints – is a fundamental example of mental simulation and spatial world modeling in humans. Here we propose a mechanistic model of human mental rotation, leveraging recent advances in deep, equivariant, and neuro-symbolic learning. Our model consists of three stacked components: (1) an equivariant neural encoder, producing 3D spatial representations of objects from images, (2) a neuro-symbolic object encoder, deriving symbolic objects descriptions from these spatial representations, and (3) a neural decision
agent, comparing these symbolic descriptions to prescribe rotation simulations in 3D latent space via a recurrent pathway. Our model design is guided by the existing experimental literature on mental rotation, which we complemented with experiments in VR where participants could at times manipulate the objects to compare. Our model captures well the performance, response times and behavior of participants in our and others' experiments, and through ablation studies we demonstrate the necessity of each component. Our work adds to a recent collection of deep neural models of human spatial
reasoning, further demonstrating the potency of integrating deep, equivariant, and symbolic representations to model the human mind.
Read the full abstract ▾
All authors Raymond Khazoum, Daniela Fernandes, Aleksandr Krylov, Qin Li, Stephane Deny
Machine Learning
Learning under Distributional Drift: Prequential Reproducibility as an Intrinsic Statistical Resource
Sofiya Zaichyk
Statistical learning under distributional drift remains poorly characterized, especially in closed-loop settings where learning alters the data-generating law. We introduce an intrinsic drift budget $C_T$ that quantifies cumulative information-
geometric motion of the data distribution along the realized learner-environment trajectory, measured in Fisher-Rao distance. The budget separates exogenous environmental change from policy-sensitive feedback induced by the learner's actions. This gives a rate-based characterization of prequential reproducibility: when performance on the realized stream is used to predict one-step-ahead performance under the next distribution, the drift contribution enters through the average motion rate $C_T/T$, not through cumulative drift alone. We prove a drift-feedback bound of order $T^{-1/2}+C_T/T$, up to controlled second-order remainder terms, and establish a matching sharpness lower bound for the same prequential reproducibility gap on a canonical regular subclass. Thus the dependence on the average Fisher-Rao motion rate is tight up to constants: $C_T/T$ is sufficient for upper control and unavoidable on regular hard subclasses. We further prove an information-theoretic indistinguishability result showing that order-$C/T$ effects on the one-step-ahead target need not be identifiable from the realized performance stream alone. Finally, we show that fixed monitoring channels induce contracted observable Fisher motion, and experiments, including a misspecified real-data feedback setting, indicate that appropriately chosen channels can retain risk-relevant drift signal when the intrinsic data-generating law is unavailable. The resulting theory treats exogenous drift, adaptive data analysis, and performative feedback as different sources of Fisher-Rao motion along the same learner-environment trajectory.
Read the full abstract ▾
Machine Learning
On-Device Continual Learning for Unsupervised Visual Anomaly Detection in Dynamic Manufacturing
Haoyu Ren, Kay Koehle, Kirill Dorofeev et al. (+1 more)
In modern manufacturing, Visual Anomaly Detection (VAD) is essential for automated inspection and consistent product quality. Yet, increasingly dynamic and flexible production environments introduce key challenges: First, frequent product changes in small-batch and on-demand manufacturing require rapid model updates. Second, legacy edge hardware lacks the resources to train and run large AI models. Finally, both anomalous and normal training data are often scarce, particularly for newly introduced product variations. We investigate on-device continual learning for unsupervised VAD with localization, extending the PatchCore to incorporate online learning for real-world industrial scenarios. The proposed method leverages a lightweight feature extractor and an incremental coreset update mechanism based on k-center selection, enabling rapid,
memory-efficient adaptation from limited data while eliminating costly cloud retraining. Evaluations on an industrial use case are conducted using a testbed designed to emulate flexible production with frequent variant changes in a controlled environment. Our method achieves a 12% AUROC improvement over the baseline, an 80% reduction in
memory usage, and faster training compared to batch retraining. These results confirm that our method delivers accurate, resource-efficient, and adaptive VAD suitable for dynamic and smart manufacturing.
Read the full abstract ▾
All authors Haoyu Ren, Kay Koehle, Kirill Dorofeev, Darko Anicic
Machine Learning · Cross-listed from cs.IT
From Zipf's Law to Neural Scaling through Heaps' Law and Hilberg's Hypothesis
Łukasz Dębowski
We inspect the deductive connection between the neural scaling law and Zipf's law – two statements discussed in machine learning and quantitative linguistics. The neural scaling law describes how the cross
entropy rate of a foundation model – such as a large
language model – changes with respect to the amount of training
tokens, parameters, and compute. By contrast, Zipf's law posits that the distribution of
tokens exhibits a power law tail. Whereas similar claims have been made in more specific settings, we show that the neural scaling law is a consequence of Zipf's law under certain broad assumptions that we reveal systematically. The derivation steps are as follows: We derive Heaps' law on the vocabulary growth from Zipf's law, Hilberg's hypothesis on the
entropy scaling from Heaps' law, and the neural scaling from Hilberg's hypothesis. We illustrate these inference steps by a toy example of the Santa Fe process that satisfies all four statistical laws.
Read the full abstract ▾
Machine Learning · Cross-listed from eess.SY
Real-Time AI-Driven Milling Digital Twin Towards Extreme Low-Latency
Wenyi Liu, R. Sharma, W. "Grace" Guo et al. (+2 more)
Digital twin (DT) enables smart manufacturing by leveraging real-time data, AI models, and intelligent control systems. This paper presents a state-of-the-art analysis on the emerging field of DTs in the context of milling. The critical aspects of DT are explored through the lens of virtual models of physical milling, data flow from physical milling to virtual model, and feedback from virtual model to physical milling. Live data streaming protocols and virtual modeling methods are highlighted. A case study showcases the transformative capability of a real-time machine learning-driven live DT of tool-work contact in a milling process. Future research directions are outlined to achieve the goals of Industry 4.0 and beyond.
Read the full abstract ▾
All authors Wenyi Liu, R. Sharma, W. "Grace" Guo, J. Yi, Y.B. Guo
Machine Learning
Element-wise Modulation of Random Matrices for Efficient Neural Layers
Maksymilian Szorc
Fully connected layers are a primary source of
memory and computational overhead in deep neural networks due to their dense, often redundant parameterization. While various compression techniques exist, they frequently introduce complex engineering trade-offs or degrade model performance. We propose the Parametrized Random Projection (PRP) layer, a novel approach that decouples feature mixing from adaptation by utilizing a fixed random
matrix modulated by lightweight, learnable element-wise parameters. This architecture drastically reduces the trainable parameter count to a linear scale while retaining reliable accuracy across various
benchmarks. The design serves as a stable, computationally efficient solution for architectural scaling and deployment in resource-limited settings.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
NRR-Core: Non-Resolution Reasoning as a Computational Framework for Contextual Identity and Ambiguity Preservation
Kei Saito
Language-processing systems that optimize for a single resolved output risk losing ambiguity. With incomplete context, competing interpretations may be compressed prematurely. We specify Non-Resolution
Reasoning (NRR) as an explicit retention-commitment interface for preserving context-indexed alternatives until evidence supports commitment. NRR organizes context-indexed alternatives, independently active weights, declared retention and commitment operations, and non-destructive output projection around three principles: Context-indexed Non-Identity, Approximate Identity, and Non-Resolution. It specifies a retained state and candidate operator vocabulary, and proposes Multi-Vector
Embeddings, Non-Collapsing
Attention, and Contextual Identity Tracking as implementable architectural realizations. In a reproducible synthetic two-turn task, one gated Multi-Vector-
Embedding instantiation maintains high output
entropy before disambiguating context arrives ($H = 0.91$ bits, near the $1.0$-bit maximum), while a controlled single-
embedding baseline has low
entropy ($H = 0.15$ bits); both tested systems resolve correctly after context arrives. Thus, high pre-context output uncertainty and accurate later resolution can coexist in the tested gated configuration. This result does not validate the full NRR architecture or matched-parameter superiority; the specification, proposed components, and demonstrated behavior remain distinct contribution layers. NRR targets premature commitment, not commitment itself: alternatives can remain available while evidence is incomplete, and commitment occurs at explicit output or action gates. The question is not whether AI should resolve ambiguity, but when, how, and under whose control. Implementation: https://github.com/kei-saito-research/nrr-core. Series hub: https://github.com/kei-saito-research/nrr-series-hub.
Read the full abstract ▾
Machine Learning
DP-EMAR: A Differentially Private Framework for Autonomous Model Weight Repair in Federated IoT Systems
Chethana Prasad Kabgere and Shylaja S S
Federated Learning (FL) enables decentralized model training without sharing raw data, but model weight distortion remains a major challenge in resource constrained IoT networks. In multi tier
Federated IoT (Fed-IoT) systems, unstable connectivity and
adversarial interference can silently alter transmitted parameters, degrading convergence. We propose DP-EMAR, a differentially private, error model based autonomous repair framework that detects and reconstructs transmission induced distortions during FL aggregation. DP-EMAR estimates corruption patterns and applies adaptive correction before
privacy noise is added, enabling reliable in network repair without violating confidentiality. By integrating Differential
Privacy (DP) with Secure Aggregation (SA), the framework distinguishes DP noise from genuine transmission errors. Experiments on heterogeneous IoT sensor and graph
datasets show that DP-EMAR preserves convergence stability and maintains near baseline performance under communication corruption while ensuring strict (epsilon, delta)-DP guarantees. The framework enhances robustness, communication efficiency, and trust in
privacy preserving
Federated IoT learning.
Read the full abstract ▾
Machine Learning
SSAS: Cross-subject EEG-based Emotion Recognition through Source Selection with Adversarial Strategy
Yici Liu, Qi Wei Oung and Hoi Leong Lee
Electroencephalographic (
EEG) signals have long been applied in the field of affective
brain-computer interfaces (aBCIs). Cross-subject
EEG-based
emotion recognition has demonstrated significant potential in practical applications due to its suitability across diverse people. However, most studies on cross-subject
EEG-based
emotion recognition neglect the presence of inter-individual variability and negative transfer phenomena during model training. To address this issue, a cross-subject
EEG-based
emotion recognition through source selection with
adversarial strategy is introduced in this paper. The proposed method comprises two modules: the source selection network (SS) and the
adversarial strategies network (AS). The SS uses domain labels to reverse-engineer the training process of domain adaptation. Its key idea is to disrupt class separability and magnify inter-domain differences, thereby raising the classification difficulty and forcing the model to learn domain-invariant yet
emotion-relevant representations. The AS gets the source domain selection results and the pretrained domain discriminators from SS. The pretrained domain discriminators compute a novel loss aimed at enhancing the performance of domain classification during
adversarial training, ensuring the balance of
adversarial strategies. This paper provides theoretical insights into the proposed method and achieves outstanding performance on two
EEG-based
emotion datasets, SEED and SEED-IV. The code can be found at https://github.com/liuyici/SSAS.
Read the full abstract ▾
Machine Learning
XNNTab – Interpretable Neural Networks for Tabular Data using Sparse Autoencoders
Khawla Elhadri, Jörg Schlötterer and Christin Seifert
In data-driven applications relying on tabular data, where interpretability is key, machine learning models such as decision trees and linear regression are applied. Although neural networks can provide higher predictive performance, they are not used because of their blackbox nature. In this work, we present XNNTab, a neural architecture that combines the expressiveness of neural networks and interpretability. XNNTab first learns highly non-linear feature representations, which are decomposed into monosemantic features using a sparse autoencoder (SAE). These features are then assigned human-interpretable concepts, making the overall model prediction intrinsically interpretable. XNNTab outperforms interpretable predictive models, and achieves comparable performance to its non-interpretable counterparts.
Read the full abstract ▾