Machine Learning
Communication-Efficient Neural Tangent Kernels for Heterogeneous Decentralized Federated Learning
Li Xia
Decentralized
federated learning (DFL) enables collaborative model training without a central server, but converges slowly under statistical heterogeneity. Recent work has shown that neural tangent kernel (NTK) methods achieve faster convergence than gradient-based updates in DFL, while momentum has proven effective for accelerating gradient-based FL. However, applying momentum to NTK updates can destabilize training under heterogeneous data. We propose SPARK, which addresses this instability with a stage-wise annealed soft-label regularizer evaluated on neighborhood-aggregated data, so that momentum can accelerate NTK updates stably. Under high heterogeneity, SPARK converges about 3$\times$ faster than baselines and lowers the total communication to a target accuracy by up to about 70%, and it attains higher accuracy across heterogeneity levels. We further study random projection as an optional Jacobian-compression strategy for bandwidth-constrained settings. We validate the approach across multiple
datasets, network
topologies, and heterogeneity levels.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Limits To (Machine) Learning
Zhimin Chen, Bryan Kelly and Semyon Malamud
Machine learning (ML) methods are highly flexible, but their ability to approximate the true data-generating process is fundamentally constrained by finite samples. We characterize a universal lower bound, the Limits-to-Learning Gap (LLG), quantifying the unavoidable discrepancy between a model's empirical fit and the population
benchmark. Recovering the true population $R^2$, therefore, requires correcting observed predictive performance by this bound. Using a broad set of variables, including excess returns, yields, credit spreads, and valuation ratios, we find that the implied LLGs are large. This indicates that standard ML approaches can substantially understate true predictability in financial data. We also derive LLG-based refinements to the classic Hansen and Jagannathan (1991) bounds, analyze implications for parameter learning in general-
equilibrium settings, and show that the LLG provides a natural mechanism for generating excess
volatility.
Read the full abstract ▾
Machine Learning
Solving a Machine Learning Regression Problem Based on the Theory of Random Functions
Yuriy N. Bakhvalov
This paper studies a machine learning regression problem as a multivariate approximation problem using the framework of the theory of random functions. An ab initio derivation of a regression method is proposed, starting from postulates of indifference. It is shown that if a probability measure on an infinite-dimensional function space possesses natural
symmetries (invariance under translation, rotation, scaling, and Gaussianity), then the entire solution scheme, including the kernel form, the type of regularization, and the noise parameterization, follows analytically from these postulates. The resulting kernel coincides with a generalized polyharmonic spline; however, unlike existing approaches, it is not chosen empirically but arises as a consequence of the indifference principle. This result provides a theoretical foundation for a broad class of smoothing and interpolation methods, demonstrating their optimality in the absence of a priori information.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.NE
Self-Motivated Growing Neural Network for Adaptive Architecture via Local Structural Plasticity
Yiyang Jia and Chengxu Zhou
Control policies are often implemented with fixed-capacity multilayer perceptrons trained by backpropagation, which require architecture selection in advance and cannot adapt their capacity during learning. This paper introduces the Self-Motivated Growing Neural Network (SMGrNN), a gradient-trained controller whose
topology evolves online through a local Structural Plasticity Module (SPM). The SPM monitors edge-wise weight update statistics over short temporal windows and uses these local signals to trigger
neuron insertion and
pruning, while synaptic weights are optimized by a standard gradient-based optimizer. This allows network capacity to be adjusted during learning without manual architectural tuning. SMGrNN is evaluated on control
benchmarks via policy
distillation. Compared with multilayer perceptron baselines, it achieves similar or higher returns, lower variance, and task-appropriate network sizes. Ablation studies with growth disabled and growth-only variants isolate the role of structural plasticity, showing that adaptive growth improves reward stability while
pruning prevents uncontrolled expansion and supports compact network formation. These results establish the independent value of local structural plasticity within gradient-trained networks and motivate future investigation of whether similar structural rules can be extended to more local or spike-based learning settings.
Read the full abstract ▾
Machine Learning · Cross-listed from quant-ph
Practical Hybrid Quantum Language Models with Observable Readout on Real Hardware
Stefan Balauca, Ada-Astrid Balauca and Adrian Iftene
Hybrid
quantum-classical models represent a crucial step toward leveraging near-term
quantum devices for sequential data processing. We present
Quantum Recurrent Neural Networks (QRNNs) and
Quantum Convolutional Neural Networks (QCNNs) as hybrid
quantum language models, reporting the first empirical demonstration of generative
language modeling trained and evaluated end-to-end on real
quantum hardware. Our architecture combines hardware-optimized parametric
quantum circuits with a lightweight classical projection layer, utilizing a multi-sample SPSA strategy to efficiently train
quantum parameters despite hardware noise. To characterize the capabilities of these models, we introduce a synthetic
dataset designed to isolate syntactic dependencies in a controlled, low-resource environment. Experiments on IBM
Quantum processors reveal the critical trade-offs between
circuit depth and trainability, demonstrating that while noise remains a significant factor, observable-based readout enables the successful learning of sequential patterns on NISQ devices. These results establish a rigorous engineering baseline for generative
quantum natural
language processing, validating the feasibility of training complex sequence models on current
quantum hardware.
Read the full abstract ▾
Machine Learning
Multi-Trajectory Physics-Informed Neural Networks for HJB Equations with Hard-Zero Terminal Inventory: Optimal Execution on Synthetic & SPY Data
Anthime Valin
We study optimal trade execution with a hard-zero terminal inventory constraint, modeled via Hamilton-Jacobi-Bellman (HJB) equations. Vanilla PINNs often under-enforce this constraint and produce unstable controls. We propose a Multi-Trajectory PINN (MT-PINN) that adds a rollout-based trajectory loss and propagates a terminal penalty on terminal inventory via backpropagation-through-time, directly enforcing zero terminal inventory. A lightweight lambda-curriculum is adopted to stabilize training as the state expands from a risk-neutral reduced HJB to a risk-averse HJB. On the Gatheral-Schied single-asset model, MT-PINN aligns closely with their derived closed-form solutions and concentrates terminal inventory tightly around zero while reducing errors along optimal paths. We apply MT-PINNs on SPY intraday data, matching TWAP when risk-neutral, and achieving lower exposure and competitive costs, especially in falling windows, for higher risk-aversion.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Xue Li, Xiaonan Song and Henry Hu
Real-world deployment of
Vision-
Language Models (VLMs) is hindered by high computational demands, as existing architectures inefficiently process all
tokens uniformly. We introduce Adaptive
Token Pruning (ATP), a dynamic inference mechanism that retains only the most informative
tokens based on contextual relevance. ATP operates at the
vision-
language interface, assigning a hybrid importance score combining ViT CLS
attention (intra-modal saliency) and CLIP text-image similarity (inter-modal relevance) to keep top-K
tokens for the LLM. Unlike static compression, ATP adapts to each input without modifying the backbone. Proposed as a lightweight gating module, ATP is compatible with popular backbones like BLIP-2, LLaVA, and Flamingo. Preliminary evaluations across VQAv2, GQA, and COCO indicate that ATP reduces inference FLOPs by around 40% and achieves roughly 1.5x speedups in end-to-end latency with negligible accuracy loss (less than 1%). Qualitative analyses suggest ATP preserves visual grounding and enhances interpretability. Beyond efficiency, we investigate robustness under corruptions; observations suggest adaptive
pruning suppresses spurious correlations, improving stability. These findings imply that resource-constrained inference and model reliability are not competing objectives. Finally, we discuss ATP's role in efficient
multimodal edge computing pipelines.
Read the full abstract ▾
Machine Learning
Co-Exploration and Co-Exploitation via Shared Structure in Multi-Task Bandits
Sumantrak Mukherjee, Serafima Lebedeva, Valentin Margraf et al. (+6 more)
We propose a novel
Bayesian framework for efficient exploration in contextual multi-task multi-armed bandit settings, where the context is only observed partially and dependencies between reward distributions are induced by latent context variables. In order to exploit these structural dependencies, our approach integrates observations across all tasks and learns a global joint distribution, while still allowing personalised inference for new tasks. In this regard, we identify two key sources of epistemic uncertainty, namely structural uncertainty in the latent reward dependencies across arms and tasks, and user-specific uncertainty due to incomplete context and limited interaction history. To put our method into practice, we represent the joint distribution over tasks and rewards using a particle-based approximation of a log-density Gaussian process. This representation enables flexible, data-driven discovery of both inter-arm and inter-task dependencies without prior assumptions on the latent variables. Empirically, we demonstrate that our method outperforms baselines such as hierarchical model bandits, especially in settings with model misspecification or complex latent heterogeneity.
Read the full abstract ▾
All authors Sumantrak Mukherjee, Serafima Lebedeva, Valentin Margraf, Jonas Hanselle, Kanta Yamaoka, Viktor Bengs, Stefan Konigorski, Eyke Hüllermeier, Sebastian Josef Vollmer
Machine Learning · Cross-listed from cs.AI
WebOperator: Action-Aware Tree Search for Autonomous Agents in Web Environment
Mahir Labib Dihan, Tanzima Hashem, Mohammed Eunus Ali et al. (+1 more)
LLM-based
agents often operate in a greedy, step-by-step manner, selecting actions solely based on the current observation without considering long-term consequences or alternative paths. This lack of foresight is particularly problematic in web environments, which are only partially observable-limited to browser-visible content (e.g., DOM and UI elements)-where a single misstep often requires complex and brittle
navigation to undo. Without an explicit backtracking mechanism,
agents struggle to correct errors or systematically explore alternative paths. Tree-search methods provide a principled framework for such structured exploration, but existing approaches lack mechanisms for safe backtracking, making them prone to unintended side effects. They also assume that all actions are reversible, ignoring the presence of irreversible actions-limitations that reduce their effectiveness in realistic web tasks. To address these challenges, we introduce WebOperator, a tree-search framework that enables reliable backtracking and strategic exploration. Our method incorporates a best-first search strategy that ranks actions by both reward estimates and safety considerations, along with a robust backtracking mechanism that verifies the feasibility of previously visited paths before replaying them, preventing unintended side effects. To further guide exploration, WebOperator generates action candidates from multiple, varied
reasoning contexts to ensure diverse and robust exploration, and subsequently curates a high-quality action set by filtering out invalid actions pre-execution and merging semantically equivalent ones. Experimental results on WebArena and WebVoyager demonstrate the effectiveness of WebOperator. On WebArena, WebOperator achieves a state-of-the-art 54.6% success rate with gpt-4o, underscoring the critical advantage of integrating strategic foresight with safe execution.
Read the full abstract ▾
All authors Mahir Labib Dihan, Tanzima Hashem, Mohammed Eunus Ali, Md Rizwan Parvez
Machine Learning
Reassessing the Role of Supervised Fine-Tuning: An Empirical Study in VLM Reasoning
Yongcan Yu, Lingxiao He, Shuo Lu et al. (+10 more)
Recent advances in
vision-
language models (VLMs)
reasoning have been largely attributed to the rise of reinforcement Learning (RL), which has shifted the community's focus away from the supervised fine-tuning (SFT) paradigm. Many studies suggest that introducing the SFT stage not only fails to improve
reasoning ability but may also negatively impact model training. In this study, we revisit this RL-centric belief through a systematic and controlled comparison of SFT and RL on VLM
Reasoning. Using identical data sources, we find that the relative effectiveness of SFT and RL is conditional and strongly influenced by model capacity, data scale, and data distribution. Contrary to common assumptions, our findings show that SFT plays a crucial role across several scenarios: (1) Effectiveness for weaker models. SFT more reliably elicits
reasoning capabilities in smaller or weaker VLMs. (2) Data efficiency. SFT with only 2K achieves comparable or better
reasoning performance to RL with 20K. (3) Cross-modal transferability. SFT demonstrates stronger generalization across modalities. Moreover, we identify a pervasive issue of deceptive rewards, where higher rewards fail to correlate with better
reasoning accuracy in RL. These results challenge the prevailing "RL over SFT" narrative. They highlight that the role of SFT may have been underestimated and support a more balanced post-training pipeline in which SFT and RL function as complementary components.
Read the full abstract ▾
All authors Yongcan Yu, Lingxiao He, Shuo Lu, Lijun Sheng, Yinuo Xu, Yanbo Wang, Kuangpu Guo, Jianjie Cheng, Meng Wang, Qianlong Xie, Xingxing Wang, Dapeng Hu, Jian Liang
Machine Learning
How Prompts Move Language Model Behavior: Frames, Salience, and Construal as Semantic Control
Dongseok Kim, Hyoungsun Choi, Mohamed Jismy Aashik Rasool et al. (+1 more)
Prompt engineering is widely used to shape large
language model behavior, yet it is often treated as a practical heuristic rather than as a form of natural-
language control. This paper develops a cognitive-semantic account in which
prompts function as semantic conditions on how a fixed model interprets inputs, foregrounds information, and structures tasks. We formalize this account through three notions – frame activation, salience control, and construal selection – and study them in natural
language inference, claim verification, and multi-hop question answering. Across these settings,
prompts produce measurable changes in label judgments, evidence use, and answer-support organization, showing that
prompt effects differ not only in magnitude but also in semantic direction. The paper therefore reframes prompting as the analysis of how instructions move model behavior, rather than only whether they improve performance.
Read the full abstract ▾
All authors Dongseok Kim, Hyoungsun Choi, Mohamed Jismy Aashik Rasool, Gisung Oh
Machine Learning · Cross-listed from stat.ME
Robust Variational Bayes by Min-Max Median Aggregation
Jiawei Yan, Ju Liu, Weidong Liu et al. (+1 more)
We propose a robust and scalable variational Bayes (VB) framework designed to effectively handle contamination and outliers in
dataset. Our approach partitions the data into $m$ disjoint subsets and formulates a joint optimization problem based on robust aggregation principles. A key insight is that the full posterior distribution is equivalent to the minimizer of the mean Kullback-Leibler (KL) divergence from the $m$-powered local posterior distributions. To enhance robustness, we replace the mean KL divergence with a min-max median formulation. The min-max formulation not only ensures consistency between the KL minimizer and the Evidence Lower Bound (ELBO) maximizer but also facilitates the establishment of improved statistical rates for the mean of variational posterior. We observe a notable discrepancy in the $m$-powered marginal log likelihood function contingent on the presence of local latent variables. To address this, we treat these two scenarios separately to guarantee the consistency of the aggregated variational posterior. Specifically, when local latent variables are present, we introduce an aggregate-and-rescale strategy. Theoretically, we provide a non-asymptotic analysis of our proposed posterior, incorporating a refined analysis of Bernstein-von Mises (BvM)
theorem to accommodate a diverging number of subsets $m$. Our findings indicate that the two-stage approach yields a smaller approximation error compared to directly aggregating the $m$-powered local posteriors. Furthermore, we establish a nearly optimal statistical rate for the mean of the proposed posterior, advancing existing theories related to min-max median
estimators. The efficacy of our method is demonstrated through extensive simulation studies.
Read the full abstract ▾
All authors Jiawei Yan, Ju Liu, Weidong Liu, Jiyuan Tu
Machine Learning
On Approaches to Building Surrogate ODE Models for Diffusion Bridges
Maria Khilchuk, Vladimir Latypov, Pavel Kleshchev et al. (+1 more)
Diffusion and Schrödinger Bridge models have established state-of-the-art performance in generative modeling but are often hampered by significant computational costs and complex training procedures. While continuous-time bridges promise faster sampling, overparameterized neural networks describe their optimal dynamics, and the underlying
stochastic differential equations can be difficult to integrate efficiently. This work introduces a novel paradigm that uses surrogate models to create simpler, faster, and more flexible approximations of these dynamics. We propose two specific
algorithms: SINDy Flow Matching (SINDy-FM), which leverages sparse regression to identify interpretable, symbolic differential equations from data, and a Neural-ODE reformulation of the Schrödinger Bridge (DSBM-NeuralODE) for flexible continuous-time parameterization. Our experiments on Gaussian transport tasks and MNIST latent translation demonstrate that these surrogates achieve competitive performance while offering dramatic improvements in efficiency and interpretability. The symbolic SINDy-FM models, in particular, reduce parameter counts by several orders of magnitude and enable near-instantaneous inference, paving the way for a new class of tractable and high-performing bridge models for practical deployment.
Read the full abstract ▾
All authors Maria Khilchuk, Vladimir Latypov, Pavel Kleshchev, Alexander Hvatov
Machine Learning
DynaGen: Unifying Temporal Knowledge Graph Reasoning with Dynamic Subgraphs and Generative Regularization
Jiawei Shen, Jia Zhu, Hanghui Guo et al. (+6 more)
Temporal Knowledge Graph
Reasoning (TKGR) aims to complete missing factual elements along the timeline. Depending on the temporal position of the query, the task is categorized into interpolation and extrapolation. Existing interpolation methods typically embed temporal information into individual facts to complete missing historical knowledge, while extrapolation techniques often leverage sequence models over graph snapshots to identify recurring patterns for future event prediction. These methods face two critical challenges: limited contextual modeling in interpolation and cognitive generalization bias in extrapolation. To address these, we propose a unified method for TKGR, dubbed DynaGen. For interpolation, DynaGen dynamically constructs entity-centric subgraphs and processes them with a synergistic dual-branch GNN encoder to capture evolving structural context. For extrapolation, it applies a conditional
diffusion process, which forces the model to learn underlying evolutionary principles rather than just superficial patterns, enhancing its ability to predict unseen future events. Extensive experiments on six
benchmark datasets show DynaGen achieves state-of-the-art performance. On average, compared to the second-best models, DynaGen improves the Mean Reciprocal Rank (MRR) score by 2.61 points for interpolation and 1.45 points for extrapolation.
Read the full abstract ▾
All authors Jiawei Shen, Jia Zhu, Hanghui Guo, Weijie Shi, Guoqing Ma, Yidan Liang, Jingjiang Liu, Hao Chen, Shimin Di
Machine Learning
PerNodeDrop: A Method Balancing Specialized Subnets and Regularization in Deep Neural Networks
Gelesh G. Omathil
Deep neural networks possess strong representational capacity yet remain vulnerable to overfitting, primarily because
neurons tend to co-adapt in ways that, while capturing complex and fine-grained feature interactions, also reinforce spurious and non-generalizable patterns that inflate training performance but reduce reliability on unseen data. Noise-based regularizers such as Dropout and DropConnect address this issue by injecting
stochastic perturbations during training, but the noise they apply is typically uniform across a layer or across a batch of samples, which can suppress both harmful and beneficial co-adaptation. This work introduces PerNodeDrop, a lightweight
stochastic regularization method. It applies per-sample, per-node perturbations to break the uniformity of the noise injected by existing techniques, thereby allowing each node to experience input-specific variability. Hence, PerNodeDrop preserves useful co-adaptation while applying regularization. This narrows the gap between training and validation performance and improves reliability on unseen data, as evident from the experiments. Although superficially similar to DropConnect, PerNodeDrop operates at the sample level. It drops weights at the sample level, not the batch level. An expected-loss analysis formalizes how its perturbations attenuate excessive co-adaptation while retaining predictive interactions. Empirical evaluations on
vision, text, and audio
benchmarks indicate improved generalization relative to the standard noise-based regularizer.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Modeling Authorial Style in Urdu Novels Using Character Interaction Graphs and Graph Neural Networks
Hassan Mujtaba, Hamza Naveed and Hanzlah Munir
Authorship analysis has traditionally focused on lexical and stylistic cues within text, while higher-level narrative structure remains underexplored, particularly for low-resource
languages such as Urdu. This work proposes a graph-based framework that models Urdu novels as character interaction networks to examine whether authorial style can be inferred from narrative structure alone. Each novel is represented as a graph where nodes correspond to characters and edges denote their co-occurrence within narrative proximity. We systematically compare multiple graph representations, including global structural features, node-level semantic summaries, unsupervised graph
embeddings, and supervised graph neural networks. Experiments on a
dataset of 52 Urdu novels written by seven authors show that learned graph representations substantially outperform hand-crafted and unsupervised baselines, achieving up to 0.857 accuracy under a strict author-aware evaluation protocol.
Read the full abstract ▾
Machine Learning
Torch Geometric Pool: the PyTorch library for pooling in Graph Neural Networks
Carlo Abate, Ivan Marisca and Filippo Maria Bianchi
Torch
Geometric Pool (tgp) is a pooling library built on top of PyTorch
Geometric. Graph pooling methods differ in how they assign nodes to supernodes, how they handle batches, what they return after pooling, and whether they expose auxiliary losses. These differences make it hard to compare methods or reuse the same model code across them. tgp addresses this problem with a common software interface based on the Select-Reduce-Connect-Lift (SRCL) decomposition. The library provides 20 hierarchical poolers, standardized output objects, standalone readout modules, support for dense poolers in batched and unbatched mode, and workflows for caching and pre-coarsening. It is released under the MIT license on GitHub and PyPI, with comprehensive documentation, tutorials, and examples.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.DB
CoLSE: A Lightweight and Robust Hybrid Learned Model for Single-Table Cardinality Estimation using Joint CDF
Lankadinee Rathuwadu, Guanli Liu, Christopher Leckie et al. (+1 more)
Cardinality estimation (CE), the task of predicting the result size of queries is a critical component of query optimization. Accurate estimates are essential for generating efficient query execution plans. Recently, machine learning techniques have been applied to CE, broadly categorized into query-driven and data-driven approaches. Data-driven methods learn the joint distribution of data, while query-driven methods construct regression models that map query features to cardinalities. Ideally, a CE technique should strike a balance among three key factors: accuracy, efficiency, and
memory footprint. However, existing state-of-the-art models often fail to achieve this balance. To address this, we propose CoLSE, a hybrid learned approach for single-table cardinality estimation. CoLSE directly models the joint probability over queried intervals using a novel
algorithm based on copula theory and integrates a lightweight neural network to correct residual estimation errors. Experimental results show that CoLSE achieves a favorable trade-off among accuracy, training time, inference latency, and model size, outperforming existing state-of-the-art methods.
Read the full abstract ▾
All authors Lankadinee Rathuwadu, Guanli Liu, Christopher Leckie, Renata Borovica-Gajic
Machine Learning
Spectral Sentinel: Scalable Byzantine-Robust Decentralized Federated Learning via Sketched Random Matrix Theory on Blockchain
Animesh Mishra
Decentralized
federated learning (DFL) enables collaborative model training without centralized trust, but it remains vulnerable to Byzantine clients that poison gradients under heterogeneous (Non-IID) data. Existing defenses face a scalability trilemma: distance-based filtering (e.g., Krum) can reject legitimate Non-IID updates,
geometric-median methods incur prohibitive $O(n^2 d)$ cost, and many certified defenses are evaluated only on models below 100M parameters. We propose Spectral Sentinel, a Byzantine detection and aggregation framework that leverages a random-
matrix-theoretic signature: honest Non-IID gradients produce covariance eigenspectra whose bulk follows the Marchenko-Pastur law, while Byzantine perturbations induce detectable tail anomalies. Our
algorithm combines Frequent Directions sketching with data-dependent MP tracking, enabling detection on models up to 1.5B parameters using $O(k^2)$
memory with $k \ll d$. Under a $(\sigma,f)$ threat model with coordinate-wise honest variance bounded by $\sigma^2$ and $f < 1/2$ adversaries, we prove $(\epsilon,\delta)$-Byzantine resilience with convergence rate $O(\sigma f / \sqrt{T} + f^2 / T)$, and we provide a matching information-theoretic lower bound $\Omega(\sigma f / \sqrt{T})$, establishing minimax optimality. We implement the full system with blockchain integration on Polygon networks and validate it across 144 attack-aggregator configurations, achieving 78.4 percent average accuracy versus 48-63 percent for baseline methods.
Read the full abstract ▾
Machine Learning
Causal inference and model explainability analysis of a real world sales dataset
Pranav Gupta and Nithin Surendran
Most major retailers today have multiple divisions focused on various aspects, such as marketing, supply chain, online customer experience, store customer experience, employee productivity, and vendor fulfillment. They also regularly collect data corresponding to all these aspects as dashboards and weekly/monthly/quarterly reports. Although several machine learning and statistical techniques have been in place to analyze and predict key metrics, such models typically lack interpretability. Moreover, such techniques also do not allow the validation or discovery of causal links. In this paper, we aim to provide a recipe for applying model interpretability and causal inference for deriving sales insights. In this paper, we review the existing literature on causal inference and interpretability in the context of problems in e-commerce and retail, and apply them to a real-world
dataset. We find that an inherently explainable model has a lower variance of SHAP values, and show that including multiple confounders through a double machine learning approach allows us to get the correct sign of causal effect.
Read the full abstract ▾
Machine Learning
Exact Flow Linear Attention: Exact Solution from Continuous-Time Dynamics
Jingdi Lei, Di Zhang and Soujanya Poria
In this paper, we introduce Exact Flow Linear
Attention~(EFLA), an exact-flow formulation of delta-rule linear
attention. We show that the delta-rule update can be interpreted as an explicit Euler discretization of an underlying continuous-time system. EFLA replaces this first-order update with the exact closed-form flow. By exploiting the rank-1 structure of the dynamics
matrix, both the
matrix exponential and the input integral collapse to a simple update that preserves delta-rule linear
attention's algebraic structure, parameter count, linear-time complexity, and chunkwise parallelism. This
attention mechanism removes the Euler discretization error of the delta-rule dynamics without introducing additional parameters. Experiments on robustness tests,
language modeling
benchmarks, and the MAD synthetic
benchmark show that EFLA improves stability under corrupted and high-energy inputs, reduces perplexity, and achieves stronger downstream performance compared to SSM and Euler-style baselines. These results establish exact-flow integration as a principled and scalable update mechanism for delta-rule linear
attention.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CR
ceLLMate: Sandboxing Browser AI Agents
Luoxi Meng, Henry Feng, Ilia Shumailov et al. (+1 more)
Browser-using
agents (BUAs) are an emerging class of AI
agents that interact with web browsers in human-like ways, including clicking, scrolling, filling forms, and navigating across pages. While these
agents help automate repetitive online tasks, they are vulnerable to
prompt injection attacks that trick an
agent into performing undesired actions, such as leaking private information or issuing unintended state-changing requests. We propose ceLLMate, a browser-level sandboxing framework that restricts the
agent's ambient authority and reduces the blast radius of
prompt injections. We address the semantic gap challenge that is fundamental to BUAs – writing and enforcing security policies for low-level UI tools like clicks and keystrokes is brittle and error-prone. Our core insight is to perform sandboxing at the HTTP layer because all side-effecting UI operations will result in network communication to the website's backend. We implement ceLLMate as an
agent-agnostic browser extension and demonstrate how it enables sandboxing policies that block
prompt injection attacks in the WASP
benchmark with 7.25–15% latency overhead.
Read the full abstract ▾
All authors Luoxi Meng, Henry Feng, Ilia Shumailov, Earlence Fernandes
Machine Learning
Differentiable Energy-Based Regularization in GANs: A Simulator-Based Exploration of VQE-Inspired Auxiliary Losses
David Strnadel
This paper presents an exploratory, simulator-based proof of concept investigating whether differentiable energy terms derived from parameterized
quantum circuits can serve as auxiliary regularization signals in Generative
Adversarial Networks (GANs). We augment the Auxiliary Classifier GAN (ACGAN) generator objective with a Variational
Quantum Eigensolver (VQE)-inspired energy term computed from class-specific Ising
Hamiltonians using Qiskit's EstimatorQNN and TorchConnector. All experiments are performed on a noiseless statevector simulator with only four
qubits, using a deliberately simple
Hamiltonian parameterization. On MNIST, the energy-regularized model initially achieves high external-classifier accuracy (99-100 percent) within five epochs compared to 87.8 percent for an earlier, unmatched ACGAN baseline. However, a rigorous, pre-registered ablation study demonstrates that these improvements are fully replicated by simple classical alternatives, including learned per-class biases, MLP-based surrogates, random noise, and even an unregularized baseline under matched training conditions. All classical variants reach approximately 99 percent accuracy. For sample quality as measured by FID, classical baselines are not merely equivalent but systematically superior to the VQE-based formulation. We therefore report a clear negative result. The VQE-inspired energy term provides no measurable causal benefit beyond trivial classical regularizers in this setting. The primary contribution of this work is methodological, demonstrating both the technical feasibility of differentiable VQE integration into GAN training and the necessity of rigorous ablation studies to avoid spurious claims of
quantum-enhanced performance.
Read the full abstract ▾
Machine Learning · Cross-listed from quant-ph
Scalable Quantum Error Mitigation with Neighbor-Informed Learning
Zhenyu Chen, Bin Cheng, Minbo Gao et al. (+4 more)
Noise in
quantum hardware is the primary obstacle to realizing the transformative potential of
quantum computing.
Quantum error mitigation (QEM) offers a promising pathway to enhance computational accuracy on near-term devices, yet existing methods face a difficult trade-off between performance, resource overhead, and theoretical guarantees. In this work, we introduce neighbor-informed learning (NIL), a versatile and scalable QEM framework that unifies and strengthens existing methods such as zero-noise extrapolation (ZNE) and probabilistic error cancellation (PEC), while offering improved flexibility, accuracy, efficiency, and robustness. NIL learns to predict the ideal output of a target
quantum circuit from the noisy outputs of its structurally related “neighbor”
circuits. A key innovation is our 2-design training method, which generates training data for our machine learning model. In contrast to conventional learning-based QEM protocols that create training
circuits by replacing non-Clifford gates with uniformly random Clifford gates, our approach achieves higher accuracy and efficiency, as demonstrated by both theoretical analysis and numerical simulation. Furthermore, we prove that the required size of the training set scales only logarithmically with the total number of neighbor
circuits, enabling NIL to be applied to problems involving large-scale
quantum circuits. Our work establishes a theoretically grounded and practically efficient framework for QEM, paving a viable path toward achieving
quantum advantage on noisy hardware.
Read the full abstract ▾
All authors Zhenyu Chen, Bin Cheng, Minbo Gao, Xiaodie Lin, Ruiqi Zhang, Zhaohui Wei, Zhengfeng Ji
Machine Learning · Cross-listed from stat.ML
Mind the Jumps: A Scalable Robust Local Gaussian Process for Multidimensional Response Surfaces with Discontinuities
Isaac Adjetey and Yiyuan She
Modeling response surfaces with abrupt jumps and discontinuities remains a major challenge across scientific and engineering domains. Although Gaussian process models excel at capturing smooth nonlinear relationships, their stationarity assumptions limit their ability to adapt to sudden input-output variations. Existing nonstationary extensions, particularly those based on domain partitioning, often struggle with boundary inconsistencies, sensitivity to outliers, and scalability issues in higher-dimensional settings, leading to reduced predictive accuracy and unreliable parameter estimation. To address these challenges, this paper proposes the Robust Local Gaussian Process (RLGP) model, a framework that integrates adaptive nearest-neighbor selection with a sparsity-driven robustification mechanism. Unlike existing methods, RLGP leverages an optimization-based mean-shift adjustment after a multivariate perspective transformation combined with local neighborhood modeling to mitigate the influence of outliers. This approach improves predictive accuracy near discontinuities while enhancing robustness to data heterogeneity. Comprehensive evaluations on real-world
datasets show that RLGP consistently delivers high predictive accuracy and maintains competitive computational efficiency, especially in scenarios with sharp transitions and complex response structures. Scalability tests further confirm RLGP's stability and reliability in higher-dimensional settings, where other methods struggle. These results establish RLGP as an effective and practical solution for modeling nonstationary and discontinuous response surfaces across a wide range of applications.
Read the full abstract ▾
Machine Learning
On the Accuracy of Newton Step and Influence Function Data Attributions
Ittai Rubinstein and Samuel B. Hopkins
Data attribution aims to explain model predictions by estimating how they would change if certain training points were removed, and is used in a wide range of applications, from interpretability and credit assignment to unlearning and
privacy. Even in the relatively simple case of logistic regressions, existing mathematical analyses of leading data attribution methods such as Influence Functions (IF) and single Newton Step (NS) remain limited in two key ways. First, they rely on global strong convexity assumptions which are often not satisfied in practice. Second, the resulting bounds scale very poorly with the number of parameters ($d$) and the number of samples removed ($k$). As a result, these analyses are not tight enough to answer fundamental questions such as "what is the asymptotic scaling of the errors of each method?" or "which of these methods is more accurate for a given
dataset?" In this paper, we introduce a new analysis of the NS and IF data attribution methods for convex learning problems. To the best of our knowledge, this is the first analysis of these questions that does not assume global strong convexity and also the first explanation of [KATL19] and [RH25a]'s observation that NS data attribution is often more accurate than IF. We prove that for sufficiently well-behaved logistic regressions, our bounds are asymptotically tight up to poly-logarithmic factors, yielding scaling laws for the errors in the average-case sample removals. \[ \mathbb{E}_{T \subseteq [n],\, |T| = k} \bigl[ \|\hat{\theta}_T - \hat{\theta}_T^{\mathrm{NS}}\|_2 \bigr] = \widetilde{\Theta}\!\left(\frac{k d}{n^2}\right), \qquad \mathbb{E}_{T \subseteq [n],\, |T| = k} \bigl[ \|\hat{\theta}_T^{\mathrm{NS}} - \hat{\theta}_T^{\mathrm{IF}}\|_2 \bigr] = \widetilde{\Theta}\!\left( \frac{(k + d)\sqrt{k d}}{n^2} \right). \]
Read the full abstract ▾
Machine Learning
Optimal Mistake Bounds for Transductive Online Learning
Zachary Chase, Steve Hanneke, Shay Moran et al. (+1 more)
We resolve a 30-year-old open problem concerning the power of unlabeled data in online learning by tightly quantifying the gap between transductive and standard online learning. In the standard setting, the optimal mistake bound is characterized by the Littlestone dimension $d$ of the concept class $H$ (Littlestone 1987). We prove that in the transductive setting, the mistake bound is at least $\Omega(\sqrt{d})$. This constitutes an exponential improvement over previous lower bounds of $\Omega(\log\log d)$, $\Omega(\sqrt{\log d})$, and $\Omega(\log d)$, due respectively to Ben-David, Kushilevitz, and Mansour (1995, 1997) and Hanneke, Moran, and Shafer (2023). We also show that this lower bound is tight: for every $d$, there exists a class of Littlestone dimension $d$ with transductive mistake bound $O(\sqrt{d})$. Our upper bound also improves upon the best known upper bound of $(2/3)d$ from Ben-David, Kushilevitz, and Mansour (1997). These results establish a quadratic gap between transductive and standard online learning, thereby highlighting the benefit of advance access to the unlabeled instance sequence. This contrasts with the PAC setting, where transductive and standard learning exhibit similar sample complexities.
Read the full abstract ▾
All authors Zachary Chase, Steve Hanneke, Shay Moran, Jonathan Shafer
Machine Learning · Cross-listed from stat.ML
Iterative Sampling Methods for Sinkhorn Distributionally Robust Optimization
Jie Wang
Distributionally robust optimization (DRO) has emerged as a powerful paradigm for reliable decision-making under uncertainty. This paper focuses on DRO with ambiguity sets defined via the Sinkhorn discrepancy: an
entropy-regularized Wasserstein distance, referred to as Sinkhorn DRO. Existing work primarily addresses Sinkhorn DRO from a dual perspective, leveraging its formulation as a conditional
stochastic optimization problem, for which many
stochastic gradient methods are applicable. However, the theoretical analyses of such methods often rely on the boundedness of the loss function, and it is indirect to obtain the worst-case distribution associated with Sinkhorn DRO. In contrast, we study Sinkhorn DRO from the primal perspective, by reformulating it as a bilevel program with several infinite-dimensional lower-level subproblems over probability space. This formulation enables us to simultaneously obtain the optimal robust decision and the worst-case distribution, which is valuable in practical settings, such as generating stress-test scenarios or designing robust learning
algorithms. We propose both double-loop and single-loop sampling-based
algorithms with theoretical guarantees to solve this bilevel program. Finally, we demonstrate the effectiveness of our approach through a numerical study on
adversarial classification.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Supervised Contrastive Frame Aggregation for Video Representation Learning
Shaif Chowdhury, Mushfika Rahman and Greg Hamerly
We propose a supervised contrastive learning framework for
video representation learning that leverages temporally global context. We introduce a
video to image aggregation strategy that spatially arranges multiple frames from each
video into a single input image. This design enables the use of pre trained convolutional neural network backbones such as ResNet50 and avoids the computational overhead of complex
video transformer models. We then design a contrastive learning objective that directly compares pairwise projections generated by the model. Positive pairs are defined as projections from
videos sharing the same label while all other projections are treated as negatives. Multiple natural views of the same
video are created using different temporal frame samplings from the same underlying
video. Rather than relying on data augmentation these frame level variations produce diverse positive samples with global context and reduce overfitting. Experiments on the Penn Action and HMDB51
datasets demonstrate that the proposed method outperforms existing approaches in classification accuracy while requiring fewer computational resources. The proposed Supervised Contrastive Frame Aggregation method learns effective
video representations in both supervised and self supervised settings and supports
video based tasks such as classification and captioning. The method achieves seventy six percent classification accuracy on Penn Action compared to forty three percent achieved by ViVIT and forty eight percent accuracy on HMDB51 compared to thirty seven percent achieved by ViVIT.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
World Models Unlock Optimal Foraging Strategies in Reinforcement Learning Agents
Yesid Fonseca, Manuel S. Ríos, Nicanor Quijano et al. (+1 more)
Patch foraging involves the deliberate and planned process of determining the optimal time to depart from a resource-rich region and investigate potentially more beneficial alternatives. The Marginal Value
Theorem (MVT) is frequently used to characterize this process, offering an optimality model for such foraging behaviors. Although this model has been widely used to make predictions in behavioral ecology, discovering the computational mechanisms that facilitate the emergence of optimal patch-foraging decisions in biological foragers remains under investigation. Here, we show that artificial foragers equipped with learned world models naturally converge to MVT-aligned strategies. Using a model-based reinforcement learning
agent that acquires a parsimonious predictive representation of its environment, we demonstrate that anticipatory capabilities, rather than reward maximization alone, drive efficient patch-leaving behavior. Compared with standard model-free RL
agents, these model-based
agents exhibit decision patterns similar to many of their biological counterparts, suggesting that predictive world models can serve as a foundation for more explainable and biologically grounded decision-making in AI systems. Overall, our findings highlight the value of ecological optimality principles for advancing interpretable and adaptive AI.
Read the full abstract ▾
All authors Yesid Fonseca, Manuel S. Ríos, Nicanor Quijano, Luis F. Giraldo