Artificial Intelligence · Cross-listed from cs.LG
Sticky Routing: Training MoE Models for Memory-Efficient Inference
Ali Kayyam
Mixture-of-Experts (MoE) models activate only a sparse subset of experts per
token, yet consecutive
tokens frequently activate different experts – causing constant weight swapping between slow storage and fast
memory on edge devices. Existing remedies are either system-level (caching heuristics) or post-hoc (router fine-tuning), leaving the root cause unchanged during pretraining. We propose StickyMoE, a differentiable routing consistency loss that penalises abrupt expert switches between adjacent
tokens, encouraging the router to maintain the same expert assignment across semantically coherent spans. StickyMoE requires no architectural changes, adds a single hyperparameter lambda, and unlike post-hoc methods, allows expert representations and routing decisions to co-adapt from the first training step. Experiments on small-scale MoE
language models show that StickyMoE reduces the expert switch rate by up to 60% with less than 4% perplexity degradation, Pareto-dominating post-hoc fine-tuning on the quality-locality frontier. Routing temporal locality is most efficiently instilled at training time.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
Signed Symmetric Quantization for Few-Bit Integers
Ian Colbert, Eashan Dash, Pablo Monteagudo-Lago et al. (+5 more)
The signed integer alphabet contains one more negative representable value than positive. Yet, by convention, the standard symmetric integer quantizer fixes its scale to be strictly positive, which assigns this extra representable value to the negative tail and can force clipping of positive outliers. In this work, we show that, at few-bit precision, such clipping is a non-trivial source of quantization error. Asymmetric quantization addresses this problem with a zero point, shifting the grid toward the observed data range; however, this flexibility is well-known to carry a runtime penalty. For example, in llama.cpp on an AMD EPYC(TM) "Turin" CPU, a 4-bit symmetric format uses up to 9% less
memory with up to 2.45$\times$ higher throughput than its asymmetric counterpart. We highlight signed symmetric quantization as a third option that retains the runtime profile of symmetric quantization without the penalty of the asymmetric format: our signed absmax grid places the extra representable value on the dominant-outlier tail through a principled and lightweight sign selection rule while keeping the zero point at zero. Our theoretical analysis offers two main results. First, we establish the signed absmax grid as conditionally bound-optimal on $\ell_2$ quantization error, and show that the condition holds for 88-99% of weight groups across pre-trained large
language models (LLMs) at low bit widths. Second, we show that negating the scale of a standard symmetric quantizer is analytically equivalent to a unit zero point shift on the same signed integer alphabet. We empirically validate our proposal on models from the Qwen3, Qwen3.5, and Llama3 families, and observe improvement in perplexity and downstream few-shot accuracy over the standard unsigned symmetric quantizer at no extra inference cost
Read the full abstract ▾
All authors Ian Colbert, Eashan Dash, Pablo Monteagudo-Lago, Juan Amboage, Srinidhi N, Giuseppe Franco, Nicholas J. Fraser, Arun Ramachandran
Artificial Intelligence · Cross-listed from cs.LG
iLENS: Interpretable LLM-Guided Mixture-of-Experts for Neuroimaging Survival Analysis
Farica Zhuang, Seong Woo Han, Zixuan Wen et al. (+3 more)
Alzheimer's Disease (AD) is a complex neurodegenerative disorder that continues to impact millions of people worldwide. Predicting AD conversion during the prodromal stage remains critical for disease understanding and patient care. As such, survival models are widely used for AD risk prediction, yet they are typically static predictors with limited interpretability and no capacity for natural
language reasoning. In this work, we propose iLENS, an interpretable large
language model (LLM) guided framework based on mixture-of-experts (MoE) for survival prediction in AD conversion. Our approach uses LLM to synthesize structured neuroimaging measurements and unstructured information to guide expert routing. Our framework demonstrates competitive predictive performance and capability in patient subtyping. Furthermore, our framework provides transparent, biologically grounded rationales for its routing decisions, bridging the gap between high-performance survival analysis and interpretable clinical decision support.
Read the full abstract ▾
All authors Farica Zhuang, Seong Woo Han, Zixuan Wen, Shu Yang, Yize Zhao, Li Shen
Artificial Intelligence · Cross-listed from cs.DL
Failure Modes of Large Language Models on Research-Level Mathematics: A Taxonomy and an Empirical Characterisation
Arnesh Banerjee and Ayushi Bhattacharjee
The "First Proof"
benchmark [1] posed ten research-level mathematics questions to the strongest publicly available LLMs and found them consistently wrong-not silent, but confidently, fluently wrong. This paper asks why. Working from the per-question post-mortems in First Proof's Appendix A, I identify four failure modes: citation fabrication (F1), premise smuggling (F2), silent problem reformulation (F3), and local-to-global compatibility gaps (F4). I then audit eight one-shot proofs generated by Gemini 2.5 Flash on Questions 1, 2, and 5 of the
benchmark, using two instruments built specifically to surface F1 and F2. The central finding is uncomfortable for anyone who sees
retrieval-augmented generation (RAG) as the obvious fix: not one of the eight proofs contained a confirmed fabricated citation, yet every single one contained at least one load-bearing claim asserted as a "fundamental result" or "standard argument" with no justification attached. That failure mode-F2, premise smuggling-is invisible to citation verification by design. A premise-audit instrument I introduce flags it at 100% precision (5/5 judge-confirmed flags are true positives) and 50% proof-level recall in this corpus. The taxonomy and the audit together suggest that the right long-term objective is building inference-time pipelines that prevent these failure modes from occurring, not just detecting them after the fact. Index Terms–Large
language models, mathematical
reasoning,
hallucination, premise smuggling, failure-mode taxonomy.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
LLM Evolution as an Industry-Scale Ecosystem: A Lifecycle Perspective on Continual Learning
Hao Jiang, Enneng Yang, Guojie Zhu et al. (+7 more)
Continual learning capability is critical for Industrial LLMs, as deployed models must be continuously updated to meet evolving requirements and environments, rather than repeatedly retrained from scratch. However, most existing research focuses on improvements on static
benchmarks, failing to capture real industrial needs. In this survey, we reformulate Industrial Continual Learning (ICL) for LLMs as a closed-loop update-and-release problem in a versioned ecosystem, where updates propagate hierarchically to industrial, application-specific models and LLM-powered applications, with capability inheritance and transfer across versions and model families. From this ecosystem perspective, we identify three core challenges: repeated adaptation erodes model plasticity, foundation-model upgrades break capability inheritance, and long-term sustainability is constrained by deployment requirements. We then organize the technical landscape of ICL around five lifecycle design principles: preserving plasticity headroom, treating upgrades as capability transfer, enabling trustworthy continual reinforcement learning, making training recipes self-optimizing, and building accountability as a base layer for long-term iteration. For each principle, we synthesize representative technical directions. Finally, we evaluate the maturity of each principle and its technical components via an evidence-based lens, identify key gaps hindering real-world deployment, and outline a practical ICL deployment blueprint and a pathway for feeding industrial realities back into academic research.
Read the full abstract ▾
All authors Hao Jiang, Enneng Yang, Guojie Zhu, Yibin Chen, Yunkun Xu, Zifu Kou, Jiayi Li, Chong Chen, Zhao Cao, Li Shen
Artificial Intelligence · Cross-listed from cs.LG
On-Device Neural Architecture Search
Andrea Mattia Garavagno, Edoardo Ragusa, Paolo Gastaldo et al. (+2 more)
This paper proposes a new approach to near-sensor computing, in which a lightweight Neural Architecture Search (NAS) is performed directly on the deployment device to find the best tiny neural architecture for analyzing the real-time data acquired through sensors. This new adaptation capability can be particularly useful in the case of human-machine interfaces for which the neural network analyzing the biometrical data can be re-designed each time the user changes, after a guided data collection procedure, fighting the typical data variations between individuals on a new level. To implement the proposed approach a new NAS has been designed and then validated on the Italian Sign
Language dataset (ISL), a collection of surface electromyography (sEMG) signals of the signs of the Italian alphabet, using several embedded systems. Moreover, further validation on the Case Western Reserve University
dataset (CWRU), a
benchmark for intelligent fault diagnosis, is presented to suggest another possible application of the proposed approach. When run on a Raspberry Pi 4, the proposed NAS performs beyond the state of the art proposing a tiny neural architecture having 0.63 times less RAM occupancy and 5.96 percentage points of more accuracy in the case of the ISL
dataset; and 0.44 times less RAM occupancy and 0.2 percentage points of more accuracy in the case of the CWRU
dataset.
Read the full abstract ▾
All authors Andrea Mattia Garavagno, Edoardo Ragusa, Paolo Gastaldo, Antonio Frisoli, Claudio Loconsole
Artificial Intelligence · Cross-listed from cs.LG
From Meta Idea to Advanced Mathematical Discovery – Human-AI Co-Discovery of Sign-Embedding Quantum Algorithms
Yanqiao Wang, Jin-Peng Liu, Peng Li et al. (+1 more)
AI-assisted mathematics is often evaluated on solving predefined problems. In practice, however, many important advances begin earlier, when a vague research intuition is transformed into a concrete problem, a promising route, and a
theorem family worth proving. This report studies that stage through a case study that led to sign-
embedding quantum algorithms for
matrix equations and
matrix functions, foundational primitives in
quantum linear
algebra and operator-output
quantum algorithms. The project began with a human-originated intuition that rational approximation is especially effective for jump-type functions such as the sign function, and might therefore serve as a design principle for
quantum algorithms. Rather than merely assisting after the problem was fixed, AI-assisted exploration, including workflows later integrated into the agentic AI-mathematician system AIM, played a key role in expanding this intuition into a route map, comparing candidate formulations, and converging toward sign
embedding as the central framework. AIM then helped connect a known
matrix-sign identity to wider classes of
matrix equations and
matrix functions, and drafted proof and complexity calculations. The decisive scientific judgments remained human: selecting which human-AI-expanded routes were worth pursuing, rejecting a Cayley-trapezoidal approximation when its validity required a hidden condition, and refining the Sylvester implementation from a coarse quadratic-gap query route to the final factorized and scaled analysis. The report argues that human-AI co-discovery workflows, with systems such as AIM as important components, are most valuable not as standalone
theorem provers, but as research partners for problem formation, connection discovery, derivation, and skeptical review inside a human-gated research loop.
Read the full abstract ▾
All authors Yanqiao Wang, Jin-Peng Liu, Peng Li, Yang Liu
Artificial Intelligence · Cross-listed from cs.LG
Dense Supervision Is Not Enough: The Readout Blind Spot in Looped Language Models
Rituraj Sharma and Tu Vu
Looped
language models turn hidden states into runtime state: each state is decoded for prediction and fed back into future computation. This creates a basic supervision question: which state variables does cross-
entropy actually control? We show that dense per-loop cross-
entropy controls the variables exposed by the readout, not every variable active in the recurrent transition. Hidden-state scale gives a concrete failure mode. Scale-invariant readouts such as RMSNorm and LayerNorm hide radial scale from the immediate cross-
entropy loss, while pre-norm residual recurrence continues to carry and update that same scale. Thus per-loop loss can make early exits usable without controlling recurrent scale. In 44M and 129M looped
transformers without inter-loop normalization, per-loop cross-
entropy through RMSNorm readouts still drives final hidden-state norms into the thousands or tens of thousands. Scale-visible readouts and explicit norm penalties keep norms in the tens, and scale-removing recurrence is the complementary architectural fix. The resulting design rule is simple: dense supervision trains exits; recurrent scale control requires either making scale visible to a loss or removing it from the loop. Consistent with this rule, scale-controlled variants achieve lower perplexity at matched inference-depth operating points in our variable-depth
benchmarks.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from eess.SP
Coordinate-Queryable Neural Field Reconstruction for EEG Spatial Super-Resolution with Unseen-Electrode Generation
Hongjun Liu, Leyu Zhou, Zijianghao Yang et al. (+1 more)
EEG spatial super-resolution (EEGSR) in real deployments is challenged by random channel missingness, unstable electrode quality, and changing visible-channel patterns caused by bad contacts or device variability. Most existing EEGSR methods learn a fixed low-to-high channel mapping under pre-defined input-output layouts, which makes them brittle when missing channels vary at test time. In this paper, we reformulate EEGSR as learning a shared conditional scalp field from partially observed support channels. Specifically, a position-guided encoder summarizes the observed
EEG channels and their coordinates into a latent condition, and a conditional implicit neural representation decoder reconstructs target
EEG signals by querying this condition at desired electrode coordinates. During inference, the model directly reconstructs unseen electrode signals from the available
EEG support and the queried coordinates. To strengthen the constraint of the encoded latent representation on the decoder and thereby construct a more stable scalp field consistent with the observed channels, we further introduce a fidelity-preserving channel corruption training strategy under mixed electrode states. Extensive experiments across multiple
EEG datasets demonstrate the effectiveness of our framework for both random missing-channel reconstruction and strict unseen-electrode signal generation. Notably, under the strict held-out-electrode setting on AAD, our method reduces NMSE by 37.5% and improves SNR by 2.12 dB over the strongest baseline, showing its ability to synthesize signals at electrode locations never exposed during training.
Read the full abstract ▾
All authors Hongjun Liu, Leyu Zhou, Zijianghao Yang, Chao Yao
Artificial Intelligence
Democratizing and accelerating AI-driven pathology research through agentic intelligence
Jiabo Ma, Cheng Jin, Yihui Wang et al. (+19 more)
Computational pathology has advanced rapidly with the emergence of foundation models, yet widespread adoption remains limited by substantial technical complexity and programming requirements. Here we present PathLab, an autonomous agentic framework that translates natural-
language research objectives into executable and validated computational pathology workflows through the structured composition of domain-specific skills and tools. By organizing workflow generation around reusable methodological modules, including data preprocessing, model development, evaluation and interpretation, PathLab enables studies to be specified at the level of scientific intent rather than implementation details. We evaluated PathLab across 12 public
datasets spanning four representative task families: region-of-interest classification, whole-slide image classification,
segmentation and survival prediction. Across all task categories, PathLab achieved non-inferior performance relative to expert implementations, while consistently enforcing semantic validity of user
prompts and proactively rejecting incompatible workflow specifications prior to execution. In controlled user studies, PathLab substantially reduced the time required to generate executable analytical pipelines and enabled domain experts without programming experience to independently design, execute and evaluate computational pathology studies. Together, these results establish PathLab as a reliable interface between biomedical intent and computational execution, enabling computational pathology studies to be designed at the level of scientific questions rather than programming expertise. By lowering technical barriers to advanced AI methodologies, PathLab provides a foundation for the broader democratization of computational pathology.
Read the full abstract ▾
All authors Jiabo Ma, Cheng Jin, Yihui Wang, Hao Jiang, Ling Liang, Yingxue Xu, Junlin Hou, Zhengrui Guo, Zhengyu Zhang, Yifei Xia, Hongyi Wang, Fengtao Zhou, Zhe Xu, Huajun Zhou, Jiarui Ouyang, Qian Zeng, On Ki Tang, Eunhyang Park, Carolyn Glass, Ronald Cheong Kin Chan, Li Liang, Hao Chen
Artificial Intelligence · Cross-listed from cs.CV
Jury Duty: Calibration and Orientation Failures in MLLM-as-a-Judge Under Cultural Ambiguity
Daniel Lee, Harsh Sharma, Eunkyu Park et al. (+5 more)
MLLM-as-a-Judge is conventionally validated by agreement with human annotations, but this metric is undefined when the human pool is culturally heterogeneous. We introduce VOIR DIRE, a
multimodal benchmark of 626 culturally paired image–
prompt artifacts spanning U.S. and mainland Chinese contexts across food, fashion, and architecture, with annotator pools that are within-pool reliable (a = 0.86/0.74) but cross-pool divergent on evaluation (Q1 r = -0.12). Across six MLLMs, the bias decomposes into two failures: a positivity-floor calibration failure (compressed scale use) and an orientation failure (default to one cultural norm). On this corpus, where contested items are sampled to split the two pools, the floor mechanically validates the more-permissive Chinese reading; persona prompting partially recovers calibration, but the orientation residual survives, evidence the tilt is not reducible to scale compression. Reference-pool in-context demonstrations deepen the orientation residual and inflate the high end rather than restoring use of the low end. Model origin adds a small additive tilt (~0.10 MAE) that is approximately invariant under demonstration. We recommend reporting
alignment against each reference pool separately and treating cross-pool divergence as a judge property.
Read the full abstract ▾
All authors Daniel Lee, Harsh Sharma, Eunkyu Park, Pranav Narayanan Venkit, Jeonghwan Kim, Kah Mun Chia, Andreas Vlachos, Shafiq Joty
Artificial Intelligence · Cross-listed from cs.NE
VQ4SNN: Vector Quantization for Memory-Efficient FPGA Spiking Neural Networks
Dimitrios Sekertzis and Giorgos Dimitrakopoulos
Spiking Neural Networks (SNNs) offer an energy-efficient paradigm for edge AI, making them attractive for hardware acceleration. However, deploying dense SNNs on FPGAs is constrained by limited on-chip
memory for synaptic weight storage. To address this bottleneck, we propose VQ4SNN, a hardware-aware architecture that reduces
memory requirements through Vector Quantization (VQ)-based weight sharing. To the best of our knowledge, this is the first application of VQ to pipelined spatial-dataflow SNN accelerators on FPGAs. VQ4SNN replaces conventional weight storage with a two-level
memory organization consisting of compact pointers and a shared codebook of quantized weight vectors. The proposed design integrates FPGA-aware
memory mapping with analytical VQ parameter selection, enabling efficient deployment on such accelerators while preserving inference accuracy. The experimental results show a reduction of 52-61% in the total number of BRAMs compared to the state-of-the-art uncompressed FPGA SNNs without increasing overall logic utilization.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.NE
A Formal Tool for Verification of Probabilistic Spiking Neural Networks Based on Quotient Abstractions
Nikan Zandian Jazi, Elisabetta De Maria and Christopher Leturc
Spiking Neural Networks (SNNs) model biological neural dynamics more faithfully than classical artificial networks, but their
stochastic, event-driven computation – rooted in ion-channel noise and unreliable synaptic vesicle release – demands probabilistic models for which deterministic abstractions are mathematically inadequate. Formal verification of such models via probabilistic model checking faces a fundamental barrier: the state space explosion problem, where the Discrete-Time Markov Chain (DTMC) encoding grows exponentially with the number of
neurons. General-purpose quotient model abstractions [1] can in principle mitigate this growth by partitioning membrane potentials into equivalence classes, but a naïve application to SNNs discards synaptic weight information, limiting the properties that can be verified. This paper introduces a weight-discretized quotient model abstraction that maps continuous synaptic weights to a compact integer range while preserving the relative contribution of each synapse, and presents CogSpike, a unified workbench that integrates SNN design, simulation, and PRISM-based formal verification within a single isomorphic tool chain. The discretization is accompanied by formal correctness guarantees: a two-sided fidelity
theorem confines any firing disagreement to a bounded gray zone around threshold, and an Asymptotic Silence
theorem gives the exact limit guarantee that unforced
neurons fall permanently silent. A
topology-dependent scaling analysis shows that the state space reduction compounds exponentially – approximately $17\times$ per
neuron for discretization parameter $W = 3$ – enabling verification of networks that are otherwise intractable, as confirmed empirically across seven canonical
topologies.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
NeuroShield: A Device-Agnostic Foundation Model for EEG Authentication
Matin Fallahi, Patricia Arias-Cabarcos and Thorsten Strufe
A central challenge in
EEG authentication is that models are typically tied to the acquisition settings in which they are trained. In particular, variations in headset hardware, channel layout, and signal duration create heterogeneous recordings that existing models are not designed to handle, causing each new headset or
dataset to be treated as a separate model-development problem. This fragmentation limits multi-
dataset learning, hinders knowledge transfer, and reduces model reusability. To address this limitation, we present NeuroShield, a reusable foundation model for
EEG authentication that learns identity-discriminative
embeddings from variable-channel and variable-length
EEG recordings through a dual-stage
transformer architecture. We pretrain NeuroShield on three public
EEG datasets comprising 15{,}762 subjects and 28{,}116 sessions, and evaluate transfer on two unseen downstream
datasets. Our evaluations show that, after fine-tuning, NeuroShield reduces equal error rate by 0.44–8.06 percentage points relative to the state of the art. NeuroShield further generalizes to segments longer than those seen during training and operates across channel layouts not encountered during pretraining. These results establish NeuroShield as a reusable and adaptable
EEG identity encoder across heterogeneous recording settings. We release NeuroShield as open source to support reproducibility and community adoption.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.NE
Genetic Algorithm Based Coordination and Optimization Model for Generation Grid Load Storage in Active Distribution Networks
Jinlu Zhang, Fujian Chi, Tianhan Ling et al. (+4 more)
Create an optimization framework that combines fuzzy logic and genetic
algorithms for risk assessment and coordination of generation, grid connection, load, and energy storage facilities in active distribution networks. In order to capture the system uncertainties caused by weather factors and dynamic user demands, this project uses fuzzy set representation to model renewable energy generation, load curves, and
market prices. In the evolutionary process, genetic
algorithms introduce fuzzy elements into system management, adjusting thru penalty factors to improve stability and seeking better scheduling or power dispatch solutions. At the same time, by penalizing uncertainty and constraint violations, the fitness function of the hybrid model aims to optimize the expected operational costs; it can produce feasible and economical results under unknown parameter conditions. Under conditions with a large supply of renewable energy and installed energy storage systems, simulation results for the IEEE-69 power system indicate that the fuzzy genetic
algorithm strategy effectively reduces technical constraints compared to deterministic optimization schemes. The total investment remains at a similar level. Other data indicate that thru fuzzy
reasoning optimization, unreasonable choices or impossible situations are avoided during the network adaptation process. This framework-based technical approach helps to construct an effective computational scheme that is aware of uncertainty in its outputs. This will provide a scientific basis for the uncertainty assessment in distribution network planning.
Read the full abstract ▾
All authors Jinlu Zhang, Fujian Chi, Tianhan Ling, Yulong He, Kejia Zhang, Hongxing Lv, Sheng Wang
Artificial Intelligence · Cross-listed from cs.CV
A Projection-Based Surrogate Gradient Interpretation for Neural Codec Wrappers
Esteban Pesnel, Julien Le Tanou, Michael Ropert et al. (+2 more)
Neural wrappers are learned pre-and postprocessing networks designed to enhance the performance of conventional
video codecs. Although these approaches can significantly improve compression efficiency, training them remains challenging due to the non-differentiability of
video codecs, which arises from the multiple discrete decisions involved in the encoding process. Surrogate gradients have recently emerged as an effective solution for enabling end-to-end learning with conventional codecs. They offer two main advantages: they avoid training an additional network to mimic the codec, and they can improve compression performance. In particular, the recently proposed SCALED method, which leverages the true compression error, has shown strong results for training neural pre-processors such as downscalers. However, this SCALED gradient was originally introduced as a reparameterization trick, which limits its interpretability. In this paper, we show that this surrogate gradient can be interpreted as a first-order local approximation of the
video codec, providing insight into its effectiveness. We further demonstrate that it is effective not only for learning downscaling operations, but also for the more challenging task of full neural wrapping with pre-and post-processing networks. Finally, we show that the approach generalizes well across different
video codecs, quality factors, and tasks, including multiple downscaling ratios, yielding BD-Rate (PSNR) reductions of up to -23.59% on x264 and -20.07% on VVenC relative to standard resampling baselines.
Read the full abstract ▾
All authors Esteban Pesnel, Julien Le Tanou, Michael Ropert, Aline Roumy, Thomas Maugey
Artificial Intelligence · Cross-listed from cs.LG
Towards CSI-Native Foundation Models: A Channel-Adaptive Roadmap for 6G
Chenyu Zhang, Xinchen Lyu, Chenshan Ren et al. (+2 more)
Wireless foundation models offer a path toward reusable channel state information (CSI) intelligence for sixth-generation (6G) systems. However, existing generic-backbone adaptation and CSI pretraining methods often treat CSI as task tensors rather than propagation-conditioned channel responses, thereby failing to capture the intrinsic time-frequency-spatial geometry of wireless environments. This paper presents a channel-adaptive roadmap toward CSI-native foundation models, proposing a unified framework that aligns pretraining, positional modeling, and
attention control with three channel requirements: scale-aware heterogeneous exposure, physical time-frequency-
antenna coordinates, and correlation-bounded
token interaction. Extensive experiments demonstrate the superiority of the proposed framework across three dimensions: zero-shot generalization, reducing NMSE by more than 4 dB across spatial-temporal-frequency tasks; scale extrapolation, yielding up to a 5.4 dB gain under 8 times unseen
antenna scaling; and inference efficiency, accelerating mobility-aware processing by up to 18.8%. A system-level evaluation with Sionna SYS further shows that the proposed framework uses only 7.01% of dense-pilot overhead, reaches -18.64 dB average NMSE, and improves average net spectral efficiency by 36.6% over dense LMMSE and 15.5% over WiFo, indicating that CSI-native representation learning can support pilot-efficient
radio access.
Read the full abstract ▾
All authors Chenyu Zhang, Xinchen Lyu, Chenshan Ren, Shuhan Liu, Qimei Cui
Artificial Intelligence
Agent Behavior Mining: Generative AI Agent Governance in Business Processes
Hoang Vu, Maximilian Körner, Adrian Rebmann et al. (+4 more)
As organizations increasingly deploy generative AI
agents to automate business processes, they face a governance dilemma: although these
agents can increase operational flexibility, their non-deterministic nature challenges the control and standardization that Business Process Management seeks to enforce. This paper addresses this invisible autonomy risk by introducing
Agent Behavior Mining, a governance capability that enables the application of process mining techniques to render generative AI
agent decision-making observable and traceable. We (1) improve the understanding of generative AI
agent behavior through an event data model that translates granular
agent activities – including
reasoning traces, tool usage, and
token costs – into standardized process logs; (2) instantiate the data model in a multi-
agent order-to-cash implementation, demonstrating how process managers can leverage
agent logs to detect policy deviations and quantify operational variability; and (3) evaluate the perceived practical utility of the approach in an exploratory study with 18 industry practitioners. The results indicate that practitioners view behavioral transparency as a prerequisite for trust and consider the ability to examine
agent reasoning as an important governance requirement for the next generation of AI-driven business processes.
Read the full abstract ▾
All authors Hoang Vu, Maximilian Körner, Adrian Rebmann, Gabriel Kevorkian, Michael Perscheid, Gregor Berg, Timotheus Kampik
Artificial Intelligence · Cross-listed from cs.CR
BELLS-O: Evaluating the Operational Trade-offs of LLM Supervision Systems
Leonhard Waibl, Felix Michalak and Hadrien Mariaccia
LLM supervision systems, namely input/output moderation filters and jailbreak detectors, are the primary safeguard against misuse in deployed AI applications, yet existing
benchmarks are often vendor-biased, omit cost and latency, and rarely compare specialized guardrails against repurposed generalist LLMs. We present BELLS-O (
Benchmark for the Evaluation of LLM Supervision Systems, Operational), the first independent operational
benchmark of LLM supervision systems. BELLS-O evaluates 28 systems from 17 providers: every major specialized guardrail (e.g., LlamaGuard-4, ShieldGemma-2, Lakera Guard) and frontier generalists repurposed as supervisors (e.g., GPT-5.4, Claude Sonnet 4.6, Grok-4.1), jointly on detection rate, false-positive rate, latency, and monetary cost. We cover input/output moderation across 11 harm categories and jailbreak detection across 13 attack techniques, using in-house
datasets built from handcrafted
prompts, expert-curated samples, and quality-controlled synthetic generation. To suppress latent generator fingerprints in synthetic data, every generated sample is paraphrased. Mapping the Pareto frontier reveals use-case-dependent tradeoffs. On content moderation, specialized supervisors are operationally dominant: top systems match frontier LLMs on detection (~95% vs. 94%) at comparably low false-positive rates (<=2%), while running 5-10x faster and ~10x cheaper. On jailbreak detection, the tradeoff shifts: frontier LLMs achieve higher detection and lower false-positive rates but at 10-50x higher cost and 5-10x higher latency. We release the
benchmark, framework, leaderboard, and
datasets as the first vendor-neutral basis for selecting safeguards under real deployment constraints.
Read the full abstract ▾
Artificial Intelligence
A Quantum-Assisted Agentic Distributed Artificial Intelligence Framework for Deadline-Bounded Orchestration of Hybrid Renewable Microgrids
Iacovos I. Ioannou, Saher Javaid, Minella Bezha et al. (+3 more)
The real-time orchestration of microgrids that combine fluctuating renewable sources, dispatchable units, storage and curtailable consumers requires the repeated solution of combinatorial dispatch and coalition formation problems under hard control deadlines. In this paper, a
quantum-assisted agentic distributed artificial intelligence (DAI) framework is proposed in which the dispatch problem of each control slot is formulated as a quadratic unconstrained binary optimization (QUBO) problem by Belief-Desire-Intention extended (BDIx)
agents and is solved by a portfolio of
quantum,
quantum-inspired and classical solvers. Solver selection is elevated to a first-class agentic deliberation action of the coordinator
agent. Learned beliefs about solver latencies are maintained and the solver intention that is expected to satisfy the prevailing deliberation deadline is committed in each slot. In addition, a belief-shaped storage valuation mechanism is introduced through which the storage
agent prices its energy at a discounted future-peak value, injecting intertemporal information into the otherwise myopic per-slot optimization. The framework is evaluated on a 24-hour simulation of a grid-connected microgrid with photovoltaic, wind, battery, genset and demand-response assets, with the
Quantum Approximate Optimization
Algorithm (QAOA) executed by statevector simulation and benchmarked per slot against tabu search, simulated annealing, binary particle swarm optimization, greedy descent and exhaustive enumeration. Zero deliberation deadlines are missed, the committed dispatch attains the exact optimum on every slot and the realized daily cost of 146.24 EUR equals the exact lower bound, with 97.83 percent renewable utilization and zero unserved energy. When the storage valuation mechanism is deactivated, the daily cost is increased to 152.75 EUR, a 4.5 percent increase.
Read the full abstract ▾
All authors Iacovos I. Ioannou, Saher Javaid, Minella Bezha, Yasuo Tan, Naoto Nagaoka, Vasos Vassiliou
Artificial Intelligence · Cross-listed from cs.LG
How Linear Is a Transformer Feed-Forward Block? Per-Block Linear Recoverability Is Learned, Not Architectural
Stuart Whipp
Transformer feed-forward networks (FFNs) are often treated as nonlinear stores of computation, yet how nonlinear a trained FFN block actually is has rarely been measured. We treat each FFN as a position-wise input-to-output map and split it into the exact least-squares linear approximation plus a residual. The held-out variance the closed-form linear map explains defines a block's linear recoverability (R^2_lin), an optimiser-free measure of its linearity. Across all twelve blocks of GPT-2, Pythia-160m, and llama-160m, R^2_lin is highly heterogeneous and non-monotone with depth, ranging from near-linear (>0.99) to strongly nonlinear (<0.3) between adjacent blocks, and is not set by the activation function: same-width GELU models GPT-2 and Pythia-160m have sharply different profiles, so recoverability is a learned property of individual trained blocks, not an architectural one. A low-rank bilinear probe of the residual recovers only a few points of R^2, with gain uncorrelated with residual nonlinearity: the unrecovered computation is not a single position-wise product but higher-order or distributed structure. The measurement also serves as a targeted compression signal: recoverable blocks admit large single-layer replacements (GPT-2's early FFN at 8x fewer parameters for +0.77 perplexity), while low-recoverability blocks flag where this is unsafe. It further exposes a methodological pitfall: trained linear baselines can badly under-converge on ill-conditioned
transformer activations, so we report the exact closed-form least-squares ceiling throughout.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
Emyx: Fast and efficient all-atom protein generation
Nicholas J. Williams, Ward Haddadin, Matteo P. Ferla et al. (+7 more)
Computational enzyme design requires generating
proteins that scaffold catalytic residues and ligands, a task that demands both
geometric accuracy and structural diversity from the underlying generative model. Current all-atom generators inherit expensive architectures from structure prediction, leading to high training costs and limited sample diversity. We argue that much of this complexity is unnecessary for generators, which condition on sparse
geometric constraints rather than rich co-evolutionary signals. Emyx is a 140M-parameter conditional flow matching model that concentrates capacity within standard
transformer blocks, replacing heavy
embedding stacks with lightweight conditional representations and sparse connectivity. We additionally derive an exact reparametrisation of the flow matching interpolant into the EDM noise-level framework, bridging flow matching training efficiency with state-of-the-art sampling methods designed for
diffusion models without retraining. Despite being the smallest model, Emyx outperforms both Proteína-Complexa and RFdiffusion3 against the AME enzyme design
benchmark across success rate under strict evaluation requiring both global fold recovery and catalytic geometry accuracy, structural novelty, scaffold diversity, and
geometric validity, while training in just $682$ GPU-hours, roughly $4\times$ less than RFdiffusion3.
Read the full abstract ▾
All authors Nicholas J. Williams, Ward Haddadin, Matteo P. Ferla, Constantin Schneider, Nicholas B. Woodall, Ruby Sedgwick, Christian D. Madsen, Andrew L. Hopkins, Douglas E. V. Pires, Edward O. Pyzer-Knapp
Artificial Intelligence · Cross-listed from cs.LG
Cost-Optimal LLM Routing with Limited User Feedback under User Satisfaction Guarantees
Herbert Woisetschläger, Arastun Mammadli, Ryan Zhang et al. (+1 more)
Inference costs for large
language model (LLM) applications are rapidly growing, driven by surging demand and rising infrastructure cost. Users expect high-quality responses, and in commercial settings this is formally codified in Service Level Agreements (SLAs), creating a fundamental tension between cost and quality. Recent progress on cost-aware LLM request routing has shown potential to resolve this tension, but existing approaches rely on complete feedback signals, offline training, extensive per-workload tuning, and most lack SLA guarantees or inference-time adaptivity. We introduce SLARouter, an online routing
algorithm that learns a cost-optimal policy from the sparse, one-sided user feedback available in production systems. SLARouter provides theoretical guarantees for both cost optimality and strict SLA compliance. Experiments across a wide range of LLM
benchmarks show that SLARouter satisfies SLA constraints without the need for per-
benchmark tuning, reducing operating cost by up to 2.2x over existing baselines.
Read the full abstract ▾
All authors Herbert Woisetschläger, Arastun Mammadli, Ryan Zhang, Shiqiang Wang
Artificial Intelligence · Cross-listed from cs.LG
Protein Representation Learning with Secondary-Structure and Energy-Filtered Hydrogen-Bond Graphs
Mohamed Mouhajir, Limei Wang, El Houcine Bergou et al. (+3 more)
Graph-based representations are widely used in
protein modeling, yet many existing approaches rely primarily on sequence adjacency or
geometric proximity, which only partially reflect the principles governing
protein folding.
Proteins instead adopt complex three-dimensional conformations organized around secondary structure elements, such as $\alpha$-helices and $\beta$-sheets, which encode recurring local motifs and stabilizing hydrogen-bond interactions. In this work, we introduce a secondary-structure-aware graph neural network for
protein representation learning. Residue-level node representations are augmented with secondary structure assignments, and graph edges are constructed from hydrogen-bond interactions filtered by their energetic strength. This design enables the model to capture both local structural context and long-range couplings that are central to
protein stability and function. We evaluate the proposed approach on commonly used
protein benchmarks and observe consistent improvements over existing graph-based methods. In addition, the resulting graph representations offer enhanced biological interpretability, as the learned connectivity aligns with established structural motifs. These findings suggest that incorporating secondary structure and energy-filtered hydrogen-bond
topology provides an effective inductive bias for
protein representation learning. The code is released at https://github.com/mohamedmohamed2021/SSProNet
Read the full abstract ▾
All authors Mohamed Mouhajir, Limei Wang, El Houcine Bergou, Hajar El Hammouti, Lamiae Azizi, Dongqi Fu
Artificial Intelligence · Cross-listed from econ.GN
A Knowledge Theory of Capital:The Value of Natural and Artificial Intelligence, Volume 1
Jeffrey Gardiner
This volume develops a knowledge theory of capital for economies in which productive capacity increasingly resides in software, data, models, routines, expertise, platforms, organizations, commons, and public epistemic infrastructure. Beginning from Adam Smith's theory of labour, stock, specialization, and
market extent, it asks what changes when knowledge becomes stock-like, mobile across forms, scalable, governable, recombinable, and imperfectly visible in accounting. The book introduces knowledge-bearing stock as the central object and analyses how it is generated, converted into governable form, deployed, improved through feedback, enclosed or shared, measured, impaired, and used as input to future production. It distinguishes
embodied, disembodied, institutionalized, commons, and public knowledge forms and develops concepts such as first conversion, cognitive enclosure, feedback capture, dark capital, and expected knowledge loss. The argument is conditional and testable: modern wealth depends not only on capital accumulation, but on how productive knowledge is governed.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.PL
ANEForge: Python for direct computation on the Apple Neural Engine
Spencer H. Bryngelson
ANEForge is a Python package that programs the Apple Neural Engine (ANE), the fixed-function neural accelerator on every recent Apple device, directly and without CoreML. In production the engine is reachable only through CoreML, which treats it as a scheduling option: no configuration requires the ANE, and a model can silently run on the CPU or GPU instead. ANEForge compiles a lazy tensor graph, built from 58 fused operators and 19 native bridge operators, into a single ANE program. The program is dispatched through the same ANE daemon and kernel-driver stack as Apple's internal framework. Beyond inference, the package reaches the engine's native fused
attention, streams int8, int4, and sparse weights, keeps decoder and optimizer state resident across steps, and runs the forward pass, backward pass, and optimizer update of training on the engine. A small fused program completes a call in about 90us, near the engine's 70us per-program dispatch floor, and a pretrained ResNet-18 forward runs end-to-end in 0.33ms. ResNet-18, a sentence encoder, and a
Vision Transformer run end-to-end against framework references, and a Stable
Diffusion U-Net validates its forward pass. ANEForge targets Apple Silicon under macOS 14 and later. Each release is verified against a recorded macOS and ANE-compiler version.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.NE
ZIVARI-TLBO: A Zero-Cost Inter-Group Evaluated-Elite Relay Mechanism for Teaching-Learning-Based Optimization
Pezhman Zivari
ZIVARI-TLBO is a grouped Teaching-Learning-Based Optimization (TLBO) method that augments an existing population-state controller with a fixed inter-group evaluated-elite relay. At each scheduled event, every group offers its already evaluated elite to the next group in a fixed ring; the elite replaces the receiver's worst eligible learner only when its stored objective value is better. Because the exact relay copies an already evaluated solution and its stored fitness, it requires no additional objective-function calls. The frozen gts-v4-cm-fixed implementation is evaluated under equal 10,000-evaluation budgets on eight classical functions at dimensions 10, 30, 50, and 100, with 30 matched seeds, and on five constrained engineering problems. A direct ablation against the same grouped landscape-aware controller without relay records 728/11/221 wins/ties/losses and a rank-biserial effect size of 0.624 across dimensions. In an eight-method multidimensional comparison, WOA obtains the best average rank (2.914) and ZIVARI-TLBO ranks second (3.382); ZIVARI-TLBO significantly outperforms TLBO, MCTLBO, DE, PSO, and GWO, loses significantly to WOA, and is not significantly different from HHO after Holm adjustment. Feasibility-aware engineering results are mixed and sensitive to the current static-penalty formulation. The evidence supports a scoped relay contribution and budget-consistent information-sharing mechanism, but not universal state-of-the-art, global-convergence, engineering-dominance, or CEC superiority claims.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.RO
ParkingTransformer: LLM-Enhanced End-to-End Trajectory Planning for Autonomous Parking
Hauteng Wu, Xu Li, Dong Kong et al. (+4 more)
End-to-end autonomous parking has emerged as a critical task within the realm of autonomous driving. However, existing methods suffer from black-box characteristics, lacking high-level semantic understanding and interpretability, which impedes the realization of seamless long-distance autonomous parking from the road to the target spot. To address these limitations, we propose ParkingTransformer, a novel framework that leverages multi-view perception and the scene understanding capability of Large
Language Models (LLMs). By combining trajectory queries with LLMs implicit state features, our method interacts directly with historical information and raw sensor data to output planning trajectories, eliminating the need for dense Bird's-View (BEV) representations. To compensate for the inadequate spatial
reasoning ability of LLMs, we introduce 3D positional encoding to explicitly inject spatial
geometric awareness. Furthermore, a fixed-window streaming mechanism is designed for historical information processing, significantly improving long-term temporal processing efficiency and inference speed. Additionally, a coarse-to-fine decoding strategy is employed to progressively enhance trajectory precision. Extensive closed-loop experiments are conducted on the CARLA simulator and real-world vehicle platforms. The results demonstrate that our method achieves a driving score of 61.32 in CARLA simulator and an average success rate of 88.70% in real-world experiments, validating the feasibility and effectiveness of the proposed
algorithms.
Read the full abstract ▾
All authors Hauteng Wu, Xu Li, Dong Kong, Zihang Wang, Xieyuanli Chen, Benwu Wang, Wenkai Zhu
Artificial Intelligence
Metric Match: A Subset Selection Approach to Evaluating LLM Judge Reliability
Alyssa Unell, Natalie Dullerud, Naomi Boneh et al. (+4 more)
LLM judges are used to reduce the need for costly human labor in evaluating open-ended text generation. However, the reliability of these judges depends critically on their
alignment with human raters – a property that itself depends on costly human annotations. In this work, we develop a method (Metric Match) for estimating correlation-based reliability metrics of LLM judges from limited annotations. Metric Match selects a subset of samples for human annotation such that the subset matches the population reliability metric with respect to acquired synthetic labels. We empirically show that Metric Match achieves a win-rate of 0.838 against random subset selection across four different correlation metrics and 15
datasets, with an 18.7% decrease in average estimation error and reduces annotation needs by 32.5%. We provide a cost model and highlight a medical case study where our method saves $1,041.67 compared to random selection for expert annotation. Further, we shift our task from reliability estimation to reliability classification of whether a given judge is above a deployment threshold, outperforming random selection with Metric Match. All project code is publicly available, and we additionally provide an installable package for ease of use.
Read the full abstract ▾
All authors Alyssa Unell, Natalie Dullerud, Naomi Boneh, Meena Jagadeesan, Tatsu Hashimoto, Nigam Shah, Sanmi Koyejo
Artificial Intelligence · Cross-listed from cs.MA
Resilient Consensus in Agentic AI
Sribalaji C. Anand and George J. Pappas
Large
language model (LLM)
agents are increasingly deployed in multi-
agent systems where they must coordinate and agree on shared decisions. We ask whether classical resilient consensus theory, developed for deterministic
agents, transfers to LLM
agents that may behave adversarially. Framing LLM agreement as a Byzantine consensus game, we run controlled experiments on complete and general communication graphs. We find that prompted LLM
agents fail to reach agreement that is achievable in principle: consensus can fail even in settings where classical theory guarantees that a convergent
algorithm exists, and this failure persists across temperatures and horizons. At the same time, wrapping the
agents with classical resilient consensus filters improves agreement. The benefit of filtering depends on how much robustness the underlying
topology already provides. Our results suggest that classical resilient consensus theory is a useful lens for the safety of agentic AI.
Read the full abstract ▾