Machine Learning · Cross-listed from eess.SP
Retrieval-Based Cross-Domain Generalization in Optical Networks via Global Features
Ali Al Housseini, Carlos Natalino, Paolo Monti et al. (+1 more)
We propose a
retrieval-based framework for crossdomain quality-of-transmission (QoT) estimation that leverages transferable feature representations while avoiding reliance on source-domain-specific decision boundaries. The proposed approach supports both zero-shot and few-shot adaptation without requiring model retraining. Experimental results on cross-domain QoT
datasets demonstrate improved generalization performance compared with conventional machine learning baselines and recent contrastive learning approaches, highlighting the potential of
retrieval-based inference for robust optical network automation.
Read the full abstract ▾
All authors Ali Al Housseini, Carlos Natalino, Paolo Monti, Omran Ayoub
Machine Learning · Cross-listed from cs.NE
NeuroSynth: A Biologically Inspired Continual Reinforcement Learning Architecture for Mitigating Catastrophic Forgetting
Yash Kini
Artificial Intelligence (AI) systems often perform well on isolated tasks but struggle under continual learning conditions, where training on new tasks can overwrite previously acquired knowledge, a failure mode known as catastrophic forgetting. Biological learning systems reduce this interference through complementary
memory processes involving rapid hippocampal encoding and slower cortical consolidation. This study introduces NeuroSynth, a
brain-inspired continual reinforcement learning architecture designed to mitigate catastrophic forgetting through a dual-pathway consolidation mechanism. NeuroSynth separates rapid task acquisition from long-term retention using distinct "plan" and "habit" pathways combined with replay and knowledge
distillation. NeuroSynth was evaluated against Proximal Policy Optimization (PPO) and Elastic Weight Consolidation (EWC) across three sequential
navigation tasks with changing goal locations in a non-revisitation continual learning setting. Across six independent seeds, NeuroSynth preserved substantially more early-task knowledge than PPO after sequential training, achieving 18.00% Task A success rate compared to 0.33% for PPO (p = 0.014929, Cohen's d = 1.49) and 35.33% Task B success rate compared to 0.00% for PPO (p = 0.002376, Cohen's d = 2.31). NeuroSynth also demonstrated higher final Task C performance than EWC, achieving 9.00% compared to 2.00% (p = 0.226643, Cohen's d = 0.56), indicating a moderate but not statistically significant advantage. These findings suggest that biologically inspired consolidation mechanisms may improve the stability-plasticity balance in continual reinforcement learning systems.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Neuromorphic Diffusion Language Models: Addressing Compute and Memory Bottlenecks via Sparsity and Block Denoising
Dengyu Wu, Clement Ruah, Jiechen Chen et al. (+2 more)
Autoregressive (AR) large
language models (LLMs) are inherently inefficient at inference time because each generated
token requires accessing the full set of model parameters, leading to low operational intensity and high energy consumption. Masked
diffusion language models (MDLMs) partially address this limitation for
memory-bound settings by allowing multiple
tokens to be generated per parameter access. In order to further enhance inference efficiency on modern platforms with extensive in-chip
memory, this work proposes neuromorphic MDLMs (N-MDLMs), which integrate block
diffusion with spike-based neuromorphic computation to jointly improve throughput and energy efficiency. While block
diffusion increases
token throughput by producing multiple
tokens per parameter access, spike-induced sparsity reduces effective parameter traffic and computations by skipping inactive channels. To analyze the synergistic effect of sparsity and
diffusion, we develop a
token-level roofline-inspired model that captures the combined impact of block-parallel generation and spike sparsity on decoding efficiency. Experimental results on translation tasks show that, thanks to spike-induced sparsity, N-MDLMs achieve substantial improvements in energy efficiency and throughput even in compute-bound platforms for which MDLMs would fail to improve over AR-LLMs.
Read the full abstract ▾
All authors Dengyu Wu, Clement Ruah, Jiechen Chen, Bipin Rajendran, Osvaldo Simeone
Machine Learning · Cross-listed from cs.CV
Gradient-Based Latent Decomposition Reveals Mechanisms of Feature Degradation in Weakly Supervised Mammography
Vinceline Bertrand and Ionut Cardei
Weakly supervised hierarchical models exhibit a persistent asymmetry: coarse lesion-type features are preserved under reconstruction while fine-grained malignancy cues degrade—a pattern with direct consequences for the clinical reliability of breast cancer screening pipelines. We introduce gradient-based orthogonal latent decomposition for hierarchical Variational Autoencoders~(H-VAEs) to mechanistically explain this asymmetry. The latent space is partitioned into a task-aligned component~($z_1$), shaped by coarse supervisory gradients, and an orthogonal residual~($z_{\text{res}}$) capturing remaining representational capacity. On~3,550 mammographic Regions of Interest~(ROIs) from CBIS-DDSM, only~$\sim$4.4% of latent magnitude aligns with supervisory gradients, leaving~$\sim$95.6% in the orthogonal residual upon which fine-grained pathology prediction primarily depends. The model achieves Stage-1~AUC~0.866 and Stage 2~AUC~0.552, with a reconstruction stability gap of $\Delta_{\text{diag}}=5\%$ ($p=0.005$) and a classification gap of $\Delta_{\text{AUC}}=0.314$ ($p{<}0.001$). Latent ablation confirms that features for both tasks reside heavily in~$z_{\text{res}}$, structurally explaining why reconstruction degrades pathology stability disproportionately. Comparisons with Multi-Instance Learning~(MIL) and Multi-Task Learning~(MTL) confirm generalization across architectures and modalities. These findings reveal that in high-dimensional spaces, a single coarse supervisory signal isolates only a sparse 1D latent direction, forcing critical fine-grained features into the vulnerable residual subspace.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Modeling Memory-Dependent Reliability of LLMs: A Hidden Markov Model
Robab Aghazadeh Chakherlou, Siddartha Khastgir, Peter Popov et al. (+1 more)
Reliability assessment of large
language models (LLMs) seeks to estimate the probability that a model produces correct responses under a specified operational profile. Conventional
benchmark-based evaluation, often summarized by aggregate accuracy, provides a point estimate of performance but does not characterize the uncertainty associated with reliability claims. Currently, statistical inference methods for LLM reliability assessment are emerging. However, a key assumption underlying these models is that test outcomes can be treated as independent repeated trials. This assumption may be inappropriate in sequential settings, where later responses depend on earlier interactions through retained context, error propagation, or an evolving interaction state. We extend a hierarchical
Bayesian framework for LLM reliability assessment by relaxing the assumption of independent task outcomes and introducing a Hidden Markov Model to capture sequential dependence in
benchmark-constructed interaction sessions. In this formulation, outcomes are generated from a latent interaction state evolving according to a first-order Markov process, capturing changes in interaction context. Through experiments using Anthropic Claude and OpenAI on four
datasets, we demonstrate the potential impact of sequential dependence on reliability assessment. The results suggest that ignoring sequential dependence may lead to overconfident reliability estimates.
Read the full abstract ▾
All authors Robab Aghazadeh Chakherlou, Siddartha Khastgir, Peter Popov, Xingyu Zhao
Machine Learning · Cross-listed from cs.NI
Invariant Discovery for Networked Systems
Hongyu Hè, Alexander Krentsel, Sylvia Ratnasamy et al. (+1 more)
Invariants, the relations expected to hold among measured signals of a network, underpin applications from verification to traffic generation, telemetry imputation, and input validation, yet writing them by hand demands rare expertise in both formal logic and networking. Automatic miners can help but fall short on two fronts: they still require the hardest input (the grammar of admissible invariants) and they learn only exact, “hard” rules, struggling with real-world approximation caused by inherent noise in data. LLMs are tools that can provide semantic
reasoning over data, but are non-deterministic and opaque in their learning. Our key idea is to partition the invariant search problem into an AI-driven grammar “discovery” problem, followed by a statistics-driven “search” problem within the learned grammar. Taken together, this allows non-deterministic,
hallucination-prone AI to help produce auditable invariants with formal guarantees. We design and implement such a system, Autogram, and evaluate it on both public and production telemetry data, recovering expert-derived invariants with high coverage and low false positives. We close with discussion on open problems on the path toward fully open-ended discovery.
Read the full abstract ▾
All authors Hongyu Hè, Alexander Krentsel, Sylvia Ratnasamy, Maria Apostolaki
Machine Learning
Discrepancy-Rounded Fair Bandits with Static and Time-Varying Exposure Floors
Ibne Farabi Shihab, Joyanta Jyoti Mondal and Anuj Sharma
Minimum-exposure constraints arise in recommendation, content curation, and regulated allocation when each provider, arm, or group must receive guaranteed exposure inside a period rather than only in aggregate. We study
stochastic bandits with exact exposure floors and show that the right object is a rounding problem: a fractional fair schedule is realized as integral pulls, and the exposure error is exactly a discrepancy vector. The main contribution is a blockwise model with time-varying floors. BDQ-UCB satisfies every block floor deterministically and has fair regret governed by the nonmandatory budget $R$, not the horizon $T$, with high-probability regret $O(\sqrt{KR\log(KT)})$. A MOSS residual variant attains $O(\sqrt{KR})$, and a matching lower bound gives the minimax rate $\Theta(\sqrt{KR})$, even with positive mandatory exposure; a kl-UCB$^{++}$ residual rule adds instance-dependent optimality. The formulation becomes essential for overlapping group floors: per-arm rounding can violate a group constraint by $\Omega(s)$ in the group size, whereas Beck–Fiala null-space rounding meets every group floor within the block budget with violation below the arm degree $t$, and composes with UCB at the same $R$-parametrized regret. For learned group plans, we close disjoint systems at $\widetilde\Theta(\sqrt{KT})$, give a dual-ledger decomposition explaining why naive index rules fail under overlap, and prove a plan-sampling rule that is pathwise feasible under an initial cover-slack condition and attains a conditional $\widetilde O(\sqrt{KT})$ guarantee, leaving the condition-free overlap rate open. Experiments on synthetic floors, MovieLens-100k genre exposure, and deployment stress tests show exact feasibility without penalty tuning and regret competitive with tuned Lagrangian baselines.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Amortized Bayesian Causal Discovery of Extended Factor Graphs
Yichen Gu, Yuxuan Song, Weizhou Qian et al. (+2 more)
Learning causal graphs from interventional data is a challenging problem with broad applications. In molecular biology, for example, a central goal is to uncover gene regulatory networks from large-scale perturbation data. An ideal
algorithm for this task should scale to thousands of nodes, incorporate interventions even when their targets are unknown, quantify uncertainty, and provide identifiability guarantees. However, existing approaches—e.g. approaches using score-based optimization or approximate
Bayesian inference—often fail to meet all of these criteria. To address these limitations, we develop Amortized
Bayesian Causal Discovery of Extended Factor Graphs (ABCDEFG). Our method guarantees exact acyclicity, scales to graphs with thousands of nodes, and naturally handles interventions even when their targets are unknown. Additionally, ABCDEFG estimates a posterior distribution whose maximum a posteriori estimate provably identifies the true causal graph up to an equivalence class. On simulated
datasets, ABCDEFG achieves state-of-the-art accuracy, producing a well-calibrated posterior distribution while outperforming previous score-based and approximate
Bayesian methods. Applied to large-scale single-
cell perturbation data, ABCDEFG identifies both established and novel gene targets of growth factors.
Read the full abstract ▾
All authors Yichen Gu, Yuxuan Song, Weizhou Qian, Yixin Wang, Joshua Welch
Machine Learning
Spectral-Aware Analytic Class-Incremental Learning for Long-Tailed Distributions
Quyen Tran, Hai Nguyen, Quan Dao et al. (+4 more)
Analytic Continual Learning (ACL) offers a computationally efficient alternative to gradient-based approaches. Recent ACL methods are based on Recursive Least Squares (RLS) and have achieved the state-of-the-art results compared to other alternatives. However, they falter significantly in Class-Incremental Learning scenarios characterized by Long-Tailed distributions. While the ill-conditioning of the autocorrelation (Gram)
matrix is a known limitation of RLS, we demonstrate that class imbalance exacerbates this issue into a distinct spectral pathology: "tail" classes suffer from severe spectral collapse, rendering their subspaces numerically indistinguishable from noise. Standard Ridge Regression ($L_2$) fails to address this effectively as it applies isotropic regularization - a uniform penalty that is insufficient to stabilize the tail without over-shrinking the head. To address this, we propose Geometry-Spectral Rectification (GSR), a theoretically grounded framework that treats long-tailed learning as a spectral regularization problem. Unlike standard isotropic regularization (Ridge) which uniformly penalizes all
eigenvalues, GSR acts as an anisotropic spectral filter, selectively inflating the collapsed
eigenvalues of tail classes. We construct a structured, data-dependent spectral perturbation
matrix $\Delta$ that selectively inflates collapsed tail eigen-directions of the Gram
matrix. Theoretical analysis proves that GSR guarantees an improved stable rank for the Gram
matrix, ensuring numerical stability. Extensive experiments show that GSR establishes a new state-of-the-art for analytic CIL, offering a superior trade-off between computational efficiency and robust generalization in long-tailed settings.
Read the full abstract ▾
All authors Quyen Tran, Hai Nguyen, Quan Dao, Zhuowei Li, Nam Le, Trung Le, Dimitris Metaxas
Machine Learning
Distribution-Specific Curvature Control with Finite-Sample Guarantees for Open-Weight Safety
Domenic Rosati, Ali Dadsetan, Hong Huang et al. (+5 more)
A short fine-tuning run can undo the safety guards of an open-weight model—retraining a refusal-trained assistant to aid weapons development or produce hate
speech. Preventing such harmful fine-tuning while retaining benign adaptability remains difficult: the only prior method with an explicit
curvature certificate, spectral deformation, inflates
curvature globally and thereby obstructs benign adaptation along with harmful adaptation. We propose HarmAlign, which applies function-preserving spectral deformation along a estimated contrastive activation subspace. We derive finite-sample bounds for the estimated subspace energy and the resulting local harmful-distribution
curvature lower bound. A stability–progress dichotomy for constant-step gradient descent turns the certified
curvature into conditional convergence-rate control. Empirically, within a fixed-architecture, finite-budget first-order threat model, HarmAlign blocks direct fine-tuning and three data- or objective-adaptive attacks across a hazardous-knowledge relearning setting and a harmful-assistance fine-tuning setting, while the protected benign tasks remain trainable. The block persists across the tested first-order optimizer variants over every attack checkpoint, and under out-of-distribution harmful fine-tuning, and it extends to important cases in our threat model: accidental safety degradation and emergent misalignment.
Read the full abstract ▾
All authors Domenic Rosati, Ali Dadsetan, Hong Huang, Xijie Zeng, Hassan Chowdhry, Subhabrata Majumdar, Hassan Sajjad, Frank Rudzicz
Machine Learning
Hidden Boundary Motion in Transformer Optimization: Function-Space Orthogonalization of Affine Weight and Bias Updates
Zhang Gongyue, Sheng Yixuan, Liu donghan et al. (+3 more)
Weights and biases are normally optimized as separate parameter tensors, yet they do not represent separate functions when the input to an affine layer has nonzero mean. For an affine map $z=Wx+b$ with input mean $\mu$, a weight update contains a sample-independent displacement $\Delta W\mu$ that is functionally indistinguishable from a bias update. We call this hidden contribution boundary motion and decompose each update into a centered, sample-varying shape component and a shared boundary component. On a four-layer
Transformer trained from scratch on IMDb, the bias-like term $g_b\mu^\top$ has a median norm equal to 0.664 of the raw weight-gradient norm across affine layers and training checkpoints. More strikingly, the median ratio $\norm{\Delta W\mu}/\norm{\Delta b}$ is 134.7, while $\norm{\Delta W\mu}/\norm{\Delta b+\Delta W\mu}$ is 0.994. Thus, under AdamW, the observed boundary motion is almost entirely realized through the weight
matrix rather than the explicit bias. We implement a diagnostic optimizer, Shape–Boundary Orthogonal AdamW (SBO-AdamW), that optimizes $g_W-g_b\mu^\top$ and $g_b$ with independent Adam states and compensates the weight-induced boundary displacement. In a single-seed experiment, SBO-AdamW raises validation accuracy from 81.68% to 85.81% and validation-selected test accuracy from 78.73% to 82.73%, with the best validation checkpoint occurring at step 800 instead of step 3000. However, the moving-batch-center compensation produces severe bias-coordinate drift and strongly reduces boundary energy. The present evidence therefore supports hidden boundary motion as an important optimization mechanism, but it does not yet establish a final general-purpose optimizer. A stable centered-affine parameterization is identified as the required next step.
Read the full abstract ▾
All authors Zhang Gongyue, Sheng Yixuan, Liu donghan, Wang Zhiyong, Ren Weihong, Liu honghai
Machine Learning · Cross-listed from cs.CL
Not All LLM Reasoning is Visible in the Chain-of-Thought
Vatsal Baherwani, Tom Goldstein and Ashwinee Panda
A key question for AI safety is whether a
language model expresses all of its
reasoning in its output
tokens. We demonstrate a concrete failure mode where frontier models exhibit invisible
reasoning by leveraging semantically irrelevant filler
tokens to improve performance on synthetic
reasoning tasks. We evaluate 13 frontier
language models across three tasks and find that many models benefit significantly from filler
tokens, with accuracy improvements of up to 13 percentage points. The benefit depends on which
tokens are used and differs across models. We further show that filler
tokens enable Claude Opus 4.5 to satisfy a hidden modular arithmetic constraint without sacrificing accuracy on its primary task, demonstrating that invisible
reasoning can serve objectives entirely invisible to CoT monitoring. Reinforcement learning gives Qwen3-235B strong preferences over filler
token content, but neither RL nor supervised fine-tuning produces a filler
token benefit that persists at test time. Our results indicate that frontier models already perform consequential computation with no interpretable trace in their output
tokens.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
Agent Team Work Zone: An Automated, Persistent Workspace for Long-Lived Claude Code Agent Teams
Shouren Wang
Large
Language Model (LLM)
agents have significantly improved coding and programming workflows. Claude Code, in particular, is one of the most powerful LLM coding
agents and is capable of conducting complex coding tasks. However, several drawbacks can undermine long-term agentic workflows. (1) Irrecoverable
agent teams: The
Agent Teams feature is powerful, but the working state accumulated by each teammate is lost and cannot be resumed once the process stops, for example, when a terminal is closed. (2) Compaction erodes working detail: Compaction condenses the conversation into a summary, causing an
agent's working details to become vague. (3) Agentic "technical debt": Over time, a user's decisions and the
agents' operations become trapped in compacted old chats, making the project increasingly difficult to maintain and review. (4) Heavy
prompt writing: Assigning or handing off tasks requires users to repeatedly write long
prompts to achieve the expected agentic performance. We propose ATWZ (
Agent Team Work Zone), a filesystem-based operations layer built around Claude Code's native
Agent Teams that addresses these problems. Its central design principle is to treat each
agent and teammate as a human employee and preserve their important working state in files stored in a dedicated directory called a "workstation," together with the skills, hooks, and scripts that use and maintain these files. With ATWZ, an
agent team can periodically back up its working state, allowing an
agent's knowledge to be recovered after compaction. After a process ends, the team can be restored with a single command. These features also substantially mitigate the agentic "technical debt" described above. Moreover, within ATWZ,
agent "employees" can send documents to one another, greatly reducing the effort required to write
prompts.
Read the full abstract ▾
Machine Learning
Beyond Directed Acyclic Graphs: Causal Zeros and Causal Differential Equations
Sergei V. Kalinin
Pearl's structural causal model (SCM) framework, built on directed acyclic graphs (DAGs) and the do-calculus, is the dominant formal
language for causal
reasoning. Yet it carries two structural restrictions: every relationship must be pre-specified as a directed causal edge, and feedback cycles are forbidden. This paper examines two classes of phenomena that strain these restrictions. First, symmetric physical and economic constraints, the ideal gas law being the canonical case, carry no intrinsic causal direction. Direction emerges only under intervention, and which variable is solved for must be specified as part of the intervention. We formalize such constraints as causal zeros within an Extended Causal Model by adding an activation operator, subject to local solvability and graph-admissibility conditions. Second, for the class of finite-propagation state-space systems considered here, we treat apparent instantaneous cycles as artifacts of suppressed time and ground both causal zeros and feedback in Causal Differential Equations (CDEs). In these, the transient regime is a time-unrolled acyclic causal process, and causal zeros arise as the defining functions of attracting
equilibrium manifolds; periodic and chaotic attractors define further regimes of the same dynamics, treated through attractor-relative intervention. We give the extended do-calculus, identifiability conditions, counterfactual semantics, and open problems.
Read the full abstract ▾
Machine Learning
Learning from the Descent Direction: Adaptive Gradient Descent under One-Sided Hölder Regularity
Arzu Ahmadova and Ismail Huseynov
We study adaptive gradient descent for continuously differentiable, possibly nonconvex objectives under one-sided Hölder regularity. Unlike classical Hölder- or Lipschitz-gradient assumptions, which control the full gradient variation, our condition bounds only the directional term appearing in the descent inequality. This can allow less conservative step sizes when large gradient changes are orthogonal to, or favorable along, the update direction. We propose an adaptive scalar-step method based on an estimate of positive one-sided Hölder
curvature, combined with a simple sufficient-decrease safeguard. For nonconvex objectives on a convex region containing the accepted update segments, we prove an explicit best-iterate stationarity bound with a rate determined by the Hölder exponent. Unlike predetermined diminishing step-size schemes, the method adapts to the local descent geometry. We evaluate the approach on two full-batch
benchmarks designed to separate directional
curvature from full gradient variation. On a binary classification problem, the method achieves the lowest final cross-
entropy, objective value, and gradient norm, together with the largest classification margin among the compared scalar gradient methods. On a nonconvex Hölder regression problem, it attains the lowest final objective gap and gradient norm. These results indicate that one-sided Hölder
curvature is an effective adaptive step-size signal when full-gradient variation is inflated by directions that do not hinder descent.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.GR
Meshless Domain Randomization via Explicit Parameter Perturbation of 3D Gaussian Splatting
Felipe Nunes Carbone de Carvalho, Joyce de Morais Souza, Alan de Aguiar et al. (+2 more)
Domain Randomization (DR) is a standard technique for closing the Sim-to-Real gap, yet traditional DR pipelines rely on classical computer graphics rendering driven by polygon meshes. For complex organic subjects, such as insect specimens, extracting and rendering textured meshes is challenging. To address this issue, we propose a meshless DR framework that operates on the parameter space of 3D Gaussian Splatting (3DGS). Our method employs two independent perturbation pipelines to synthesize randomized training
datasets. First, a Photometric DR pipeline alters the baked illumination and color balance by modulating the Spherical Harmonics (SH) coefficients. Second, a Procedural DR pipeline isolates the subject's
geometric shape by replacing its original textures with 3D spatial noise. Finally, these perturbed radiance fields are composited over stochastically varied backgrounds using a rasterization engine. Our parameter manipulation provides a meshless alternative for generating robust
datasets for complex geometries.
Read the full abstract ▾
All authors Felipe Nunes Carbone de Carvalho, Joyce de Morais Souza, Alan de Aguiar, Charles Morphy D. Santos, João Paulo Gois
Machine Learning
Efficient Learning of Truncated Boolean Product Distributions: Influence to the Rescue
Rohan Chauhan and Ioannis Panageas
Learning the natural parameters $z \in \mathbb{R}^n$ of discrete distributions $\mu_z$ from independent samples constrained to a subset $S \subseteq \{0,1\}^n$ is a foundational challenge in high-dimensional statistics. Existing methods for efficiently estimating truncated Boolean product distributions, notably the work of [Fotakis et al' COLT'20, Algorithmica '22], require either strong local connectivity assumptions on $S$ – a property denoted fatness – or stringent anti-concentration assumptions and necessitate the total mass of the truncation set to be a constant with respect to $n$. Moreover, the results in [Fotakis et al' COLT'20, Algorithmica '22] suffer from sample complexities that scale as $\Omega(2^n)$ if the mass of $S$ is exponentially small in $n$. In this work, we circumvent these limitations by analyzing the geometry of $S$ under the measure $\mu_z$. We refine the existing parameter estimation guarantees under the fatness assumption, improving the prior sample complexity to $O( \log n / \epsilon^2)$ for $\ell_\infty$-recovery, matching the untruncated minimax rate. We further generalize fatness using the notion of influence utilized in the analysis of Boolean functions and provide sufficient conditions for efficient inference. Notably, unlike previous work, our method does not require sampling at arbitrary parameterizations of the model. Lastly, we establish a theoretical lower bound demonstrating the sample complexity exhibits an intrinsic exponential dependence on the width of the model and the minimum distance between elements in the set.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.SE
Evaluating and Mitigating the Misguidance Effect of Buggy Code in LLM-Generated Unit Tests
Junda Zhao, Shurui Zhou and Eldan Cohen
While Large
Language Models (LLMs) show great promise for automating unit test generation, recent studies suggest that the quality of generated tests can be negatively impacted when models are prompted with buggy code. This paper presents a new metric to quantitatively measure the "misguidance effect," a phenomenon where buggy code steers LLMs toward generating tests that validate its erroneous behavior rather than expose it. Our analysis reveals that prompting LLMs with buggy code has a severe, twofold impact: it significantly increases "misguided tests" that assert incorrect behavior while simultaneously suppressing the generation of effective, bug-finding tests. We further corroborate this effect from a model-internal perspective, showing that buggy code skews LLMs' preference toward tests that assert the same erroneous behavior. To counter this, we introduce and validate a specification-based unit test generation paradigm that replaces the code under test in the
prompt with an LLM-generated specification docstring. Our results show that this paradigm effectively reduces misguided tests while substantially increasing effective tests, improves multi-round, feedback-driven test generation pipelines, and remains applicable to both buggy and bug-free code. Overall, these results suggest that specification-based prompting is a promising strategy for mitigating misguidance from buggy code in LLM-generated unit tests.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.SE
Do Coverage and Mutation Scores of LLM-Generated Test Suites Correlate with Their Effectiveness? (Replicability Study)
Junda Zhao, Shurui Zhou and Eldan Cohen
Recent advances in large
language models (LLMs) have driven growing interest in using LLMs to automate test generation. Prior work commonly evaluates generated test suites using proxy metrics such as code coverage and mutation score. However, studies by Inozemtseva et al. and Papadakis et al. show that, for human-written tests, correlations among coverage, mutation, and real-bug detection can largely vanish once test suite size is controlled, raising concerns about the validity of evaluations based on proxy metrics. It also remains unclear whether these conclusions carry over to LLM-generated tests, given that prevailing LLM-based test-generation workflows differ substantially from traditional approaches. In this paper, we conduct a large-scale replication study of these two prior works using a wide range of test suites generated by a diverse set of LLMs, and re-examine the relationships among coverage, mutation, and real-bug detection effectiveness. Our findings diverge substantially from prior results. We show that the usefulness of coverage and mutation is highly context-dependent: in regression-style settings where the code provided to the LLM can be reasonably assumed bug-free, these metrics can provide meaningful signals when comparing across models; in another common scenario where the code-under-test may already be buggy and the goal is to expose the bug within the code-under-test, they no longer serve as reliable indicators. We also find little evidence that test suite size is a dominant confounder for correlations among coverage, mutation, and real-bug detection for LLM-generated tests. Based on these findings, we discuss how to interpret results from prior studies and provide actionable guidance for evaluating LLM-based test generation.
Read the full abstract ▾
Machine Learning
Spatial Prediction of Soil Microplastics and Organic Matter Using Graph Attention Networks
Anik Dev Nath, Md Al Amin and Bikash Kumar Paul
Accurate estimation of soil microplastics and organic matter is essential to assess ecosystem health and support sustainable land use. This study presents a graph-based deep learning approach using Graph
Attention Networks (GATs) to model spatial dependencies among 91 georeferenced soil samples. By incorporating spatial coordinates, soil properties, and land use data, a two-layer GAT architecture was developed to capture local interactions. The final model showed strong performance, achieving RMSEs of 625.06 ($R^2 = 0.87$) for microplastics and 0.43 ($R^2 = 0.91$) for organic matter. However, cross-validation results revealed limited generalization, probably due to the small sample size and sparse graph structure. These findings demonstrate the potential of GATs for spatial soil prediction and underscore the need for dense
datasets and improved graph connectivity.
Read the full abstract ▾
Machine Learning
Same Predictions, Different Reasons: The Effect of Quantization on Model Explanations
Kazi Kamruzzaman Rabbi, Md. Zami Al Zunaed Farabe and M. Sohel Rahman
Post-training quantization (PTQ) has become a practical solution for deploying deep learning models on resource-constrained edge devices by compressing high-precision floating-point weights into low-precision representations without requiring retraining. Past research has demonstrated that quantization largely preserves classification accuracy; however, whether it also preserves the model's internal
reasoning remains an open question. This study presents a systematic evaluation on how static PTQ affects the interpretability / explainability of five widely used CNN architectures: VGG19, ResNet18, EfficientNet-B0, DenseNet161, and MobileNetV2 at INT8 and INT4 precision. We employ a dual interpretability framework that combines Grad-CAM for spatial
attention analysis with LIME for input-level feature attribution, and systematically compare full-precision and quantized models on two binary classification
datasets. Interpretability is evaluated using three complementary metrics: the Pearson correlation coefficient, structural similarity index, and top-20% IoU to capture distributional and structural variations in model explanations, supplemented by deletion/insertion faithfulness analysis. The results show that classification accuracy is not a reliable indicator of interpretability stability under reduced precision. DenseNet161 maintains strong feature consistency across both precision levels, whereas EfficientNet-B0, despite achieving competitive spatial
attention and classification accuracy at INT8 precision, exhibits a substantial degradation in input-level feature attribution. These findings have direct implications for the trustworthy deployment of quantized models in applications with high interpretability requirements, demonstrating that architecture selection is as important as the quantization strategy.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
What Can Be Enforced? A Theory of Certified Runtime Safety for Tool-Using Agents
Shawn Ray
Runtime guardrails act before irreversible tool calls, but their guarantees depend on what policy state is representable, what a judge observes, and whether intervention changes future behavior. We separate three questions. First, relative to fixed oracle predicates, a deterministic gate enforces exactly the nonempty safety policies whose good prefixes its register model recognizes; policy nontriviality is undecidable with two decrementable counters but in PSPACE for a separable monotone fragment. Second, under a fixed exogenous law, Neyman-Pearson gives the exact false-block/miss frontier and conformal calibration gives a finite-sample marginal certificate, possibly via block-all. Third, once blocking changes future proposals, static scores and ungated trajectories need not identify the closed-loop frontier; a specified finite controlled model instead yields an occupancy program. Bounded representation attacks add a robustness margin, so benign calibration alone does not transfer. Experiments target these distinctions through static diagnostics, controlled-model enumeration, representation rewrites, and paired closed-loop reruns.
Read the full abstract ▾
Machine Learning · Cross-listed from physics.optics
AI-interpreted Optical Scattering for Robust and Focal Depth-Aware Imaging
Eunji Ko, Patrick Ross, Corey Hart et al. (+1 more)
Optical scattering has conventionally been regarded as an impediment in imaging research due to the degradation of image quality during reconstruction. Nevertheless, this study explores two cases in which optical scattering may serve a beneficial role in image reconstruction tasks. We compared the No Scattering MNIST
dataset with three Scattering MNIST
datasets, each generated under distinct scattering conditions. To assess the information content of the resulting speckle patterns, we employed a Variational Autoencoder (VAE) approach which achieves accuracy comparable to state-of-the-art deep learning approaches, but has an interpretable latent space. We find that scattering can enhance data robustness against spatial pixel loss by effectively distributing information. We also demonstrate that scattering can enable distinctions of focal depth information. We anticipate that these findings will contribute to more efficient imaging techniques, particularly in the presence of obstacles and three-dimensional signals.
Read the full abstract ▾
All authors Eunji Ko, Patrick Ross, Corey Hart, Wolfgang Losert
Machine Learning · Cross-listed from cs.DB
Queryable Self-Organizing Maps: A Database Abstraction for Topology-Driven Data Exploration
Denis Mayr Lima Martins and Gottfried Vossen
Self-Organizing Maps (SOMs) have long been used as exploratory tools for high-dimensional data: they organize objects into a two-dimensional
topology that reveals clusters, gradients, sparse regions, dense regions, and boundaries. Yet, in modern data systems, SOMs are typically trained and visualized outside the DBMS, disconnected from the relational data they summarize. We introduce the abstraction of a queryable data map: a learned topological artifact consisting of representatives, neighborhood relations, object assignments, and derived summaries. We instantiate this idea with MapDB, a lightweight prototype that makes SOM artifacts queryable so users can explore data
topology without leaving the database. Experimental study shows that SOM training is feasible at moderate analytical scale, that map queries are interactive after materialization, and that SOM regions provide meaningful targets for exploratory SQL.
Read the full abstract ▾
Machine Learning
Frustratingly Simple Black-Box Adaptation of Language Models via Logit Bias
Ofek I. Cohen, Lior Shani, Aviv Rosenberg et al. (+3 more)
Many organizations aim to adapt
language models for internal use, both to improve performance on domain-specific tasks and to address
privacy concerns around sensitive data. However, such adaptation remains non-trivial: it often requires operationally challenging fine-tuning of open-source models or ad hoc
prompt optimization. We study a minimal alternative based on a simple API-level control: allowing users to bias the model's logits with a user-defined vector. We develop a black-box method for learning a single context-independent logit-bias vector, added at every decoding step, without modifying model weights or requiring gradients. Starting from a KL-regularized reinforcement learning (RL) objective, we characterize when such a fixed logit-bias vector can approximate the optimal prefix-dependent correction and derive a closed-form inverse-propensity
estimator from rollouts, rewards, and
token probabilities. Empirically, this simple decoding-time intervention improves over base models on mathematical and
reasoning benchmarks while using far fewer trainable parameters than conventional fine-tuning. Our results suggest that learned logit bias is a lightweight mechanism for adapting
language models under minimal access requirements.
Read the full abstract ▾
All authors Ofek I. Cohen, Lior Shani, Aviv Rosenberg, Ankur Samanta, Tal Wagner, Yonathan Efroni
Machine Learning
MEMENTO: Memory-Guided Memetic Code-as-Policy Evolution
Alkis Sygkounas, Victor Aregbede, Amy Loutfi et al. (+1 more)
Long-horizon
embodied tasks require policies that execute many dependent actions before task success can be observed. Representing policies as executable control pro- grams (code-as-policy) enables their decision logic to be inspected and revised after rollout evaluation. Revised programs can then be executed and compared by rollout performance, framing policy improvement as execution-guided program search. Evo- lutionary methods driven by large
language models (LLMs) provide a natural mecha- nism for this search by generating variants and selecting high-performing candidates. However, existing approaches primarily select among independently generated vari- ants and lack a sequential local improvement phase. We introduce MEMENTO, a
memory-guided single-elite memetic framework for code-as-policy
evolution. ME- MENTO first evolves a rollout evaluator that maps policy rollouts to scalar fitness and structured feedback metrics. Fitness selects accepted candidates and the next elite, while feedback metrics condition policy proposals generated by
memory-guided hill-climbing, macro-mutation, and crossover. We evaluate MEMENTO on two long- horizon
embodied domains: Robosuite Franka Tower-of-Hanoi manipulation and AI2- THOR household interaction. MEMENTO outperforms Eureka and REvolve, adapted as code-as-policy evolutionary baselines, in task success and generalization to held- out Robosuite object configurations and unseen AI2-THOR scenes. Ablations show that zero-shot generation and unevolved evaluators fail to solve either domain, and that removing policy-search branches reduces performance. Finally, we deploy the best-evolved Robosuite policy on a physical Franka
robot, demonstrating the feasibil- ity of sim-to-real transfer of the evolved code-as-policy. Code,
prompts, and
videos are available at: https://github.com/sygkounas/MEMENTO.
Read the full abstract ▾
All authors Alkis Sygkounas, Victor Aregbede, Amy Loutfi, Andreas Persson
Machine Learning · Cross-listed from quant-ph
Practical advantage beyond the quadratic speedup limit with fully-quantum walks
Massimiliano Incudini and Guglielmo Mazzola
We introduce a new class of fully-
quantum Metropolis walks in which both the proposal and acceptance steps are intrinsically
quantum. Unlike standard
quantum walks obtained by quantizing classically efficient Markov chains, our
algorithm employs
Hamiltonian simulation as a
quantum-native proposal mechanism, enlarging the class of
quantum walks beyond classical counterparts. We target the problem of sampling from the low-temperature Gibbs distribution of classical dense Ising models, within a fixed error in total variation distance. This approach achieves about a cubic
polynomial asymptotic advantage over previous
quantum-walks, resulting in a total sixth-degree
polynomial queries speedup compared to the best classical walk. This shows that speedups beyond the widely assumed quadratic limit are possible within the
quantum walk formalism. We perform a complete fault-tolerant compilation of all algorithmic primitives and
benchmark against CPU, GPU, and FPGA implementations of the best classical Markov chain. Under identical hardware assumptions, the resulting advantage runtime crossover is reduced from approximately $10^3$ years for conventional
quantum walks to less than one day. These results identify fully-
quantum Markov chains as a promising route toward practical
quantum advantage.
Read the full abstract ▾
Machine Learning
From Hybrid Mechanistic–Data-Driven Modeling Toward Neuro-Symbolic AI: What, Why, and How
Moein E. Samadi and Andreas Schuppert
Hybrid mechanistic/data-driven models, which combine first-principles with learned components, are increasingly used in process engineering and scientific machine learning. Common hybrid modeling designs are specified primarily through their architectures and training losses, which offers a limited basis for a shared semantic interface to compare or verify them across domains, with comparatively little
attention paid to epistemic uncertainty in the mechanistic part. We bridge hybrid modeling and neuro-symbolic (NeSy) AI by reconstructing these designs as instances of NeSy interface. The resulting translation, Hybrid-to-NeSy (H2N), places mechanistic knowledge on the
language side, learned modules on the belief side, and validity domains together with constraints on the logic side. For each design, H2N then yields an explicit NeSy inference functional and a logic-belief decomposition. From this decomposition we derive two metrics: structural violation rate (SVR), measuring whether the learned belief respects the mechanistic structure; and belief dispersion (BD), measuring how concentrated the learned plausibility is, serving as a hybrid model's epistemic uncertainty in its mechanistic part. We instantiate H2N on a case study of a structured hybrid model for binary classification under label noise and show that models with higher SVR and BD exhibit greater variability in held-out accuracy. Under structural distribution shift, H2N further quantifies a model's uncertainty during extrapolations, whereas test accuracy reveals the same shift only post hoc.
Read the full abstract ▾
Machine Learning
SLA-Constrained Carbon-Aware Routing in Geo-Distributed Serverless Clouds
Anmol Chaudhary and Rahul Mishra
Modern cloud deployments distribute applications across multiple geographic regions, yet standard routing mechanisms prioritize latency while ignoring the fluctuating carbon intensity of local power grids. Latency-driven routing incurs avoidable carbon emissions, particularly when cleaner regions are within acceptable latency bounds. The proposed model formulates the carbon-aware serverless routing problem as a constrained optimization over geo-distributed cloud regions and introduces an SLA-constrained carbon-aware routing policy that achieves optimal carbon reduction within the SLA-feasible region, evaluated using real carbon intensity measurements across 5 primary AWS deployments. Experimental results show that the proposed policy achieves up to 46.8% carbon reduction while maintaining zero SLA violations across all evaluated thresholds. The system reduces carbon by an average of 27.4% under mixed workloads, and the routing overhead is very low (less than 0.02% of total request latency). A scalability study across 12 AWS regions spanning 6 continents demonstrates that average carbon savings increase from 27.4% to 47.5% as routing flexibility expands under mixed workloads. The proposed work contributes to SDG 13 (
Climate Action) and SDG 7 (Affordable and Clean Energy) by enabling low-carbon routing decisions. These results indicate that cloud systems can achieve significant carbon savings without compromising user experience.
Read the full abstract ▾
Machine Learning
OrchNAS: Orchestrated Neural Architecture Search Service for Personalised Federated Edge Intelligence
Keya Patel, Sajib Mistry, Sheik Mohammad Mostakim Fattah et al. (+1 more)
We propose OrchNAS, an energy-aware, personalised,
federated edge intelligence framework that leverages a Neural Architecture Search Service to automatically design service-adaptive models for heterogeneous edge environments. The framework orchestrates the architecture search process on a server-side NAS service, enabling edge services to derive personalised architectures under device-level energy, computation, and
memory constraints. We introduce an energy-aware global architecture search mechanism that learns a compact global representation across heterogeneous services. We develop an energy-efficient architecture selection mechanism that enables each service to derive a personalised subnet that satisfies its resource constraints via a progressive, greedy, energy-aware
pruning strategy. We propose an energy-efficient personalised model optimisation scheme that updates service-adaptive parameters while preserving global representations, where a primal-dual optimisation mechanism enforces strict energy budgets during architecture adaptation. Experiments on real-world and
benchmark datasets demonstrate the effectiveness of the proposed approach.
Read the full abstract ▾
All authors Keya Patel, Sajib Mistry, Sheik Mohammad Mostakim Fattah, Aneesh Krishna