Machine Learning · Cross-listed from cs.DC
Enhancing Model Context Protocol (MCP) with Context-Aware Server Collaboration
Meenakshi Amulya Jayanti and X.Y. Han
The Model Context Protocol (MCP) (MCP Community, 2025) has emerged as a widely used framework for enabling LLM-based
agents to communicate with external tools and services. The original MCP implementation (Anthropic, 2024) relies on a Large
Language Model (LLM) to decompose tasks and issue instructions to servers. In particular, the
agents, models, and servers are stateless and do not have access to a global context. However, in tasks involving LLM-driven coordination, it is natural that a Shared Context Store (SCS) could improve the efficiency and coherence of multi-
agent workflows by reducing redundancy and enabling knowledge transfer between servers. Thus, in this work, we design and assess the performance of a Context-Aware MCP (CA-MCP) that offloads execution logic to specialized MCP servers that read from and write to a shared context
memory, allowing them to coordinate more autonomously in real time. In this design, context management serves as the central mechanism that maintains continuity across task executions by tracking intermediate states and shared variables, thereby enabling persistent collaboration among
agents without repeated prompting. We present experiments showing that the CA-MCP can outperform the traditional MCP by reducing the number of LLM calls required for complex tasks and decreasing the frequency of response failures when task conditions are not satisfied. In particular, we conducted experiments on the TravelPlanner (Yang et al., 2024) and REALM-Bench (Geng & Chang, 2025)
benchmark datasets and observed statistically significant results indicating the potential advantages of incorporating a shared context store via CA-MCP in LLM-driven multi-
agent systems.
Read the full abstract ▾
Machine Learning · Cross-listed from physics.comp-ph
Multi-Scale Negative Coupled Information Systems (MNCIS): A Unified Spectral Topology Framework for Stability in Turbulence, AI, and Biology
Pengyue Hou
Complex dynamical systems frequently encounter a recurrent structural instability: the collapse of the spectral gap, driving the system toward a low-dimensional "Zero-Mode Attractor" (e.g., spectral pile-up or over-smoothing). Building upon recent global well-posedness estimates [Hou, arXiv:2601.00638], this work generalizes the Multi-Scale Negative Coupled Information System (MNCIS) framework. We postulate that global stability requires an active topological operator - Adaptive Spectral Negative Coupling (ASNC) - functioning as a state-dependent high-pass filter that penalizes
entropy accumulation at spectral boundaries. We validate this unified framework via three implementations: (1) Hydrodynamics: In 3D Navier-Stokes
turbulence ($N=256^3$), ASNC acts as a global-enstrophy adaptive sub-grid scale (SGS) model, stabilizing the inviscid limit and preserving the Kolmogorov $-5/3$ inertial range without artificial hyper-viscosity. Crucially, we verify that the operator remains dormant ($\gamma \approx 0$) during the linear growth phase of physical instabilities, functioning strictly as a conditional topological clamp. (2) Artificial Intelligence: Addressing Over-smoothing in Graph Neural Networks (GNNs), we implement ASNC as a parameter-free topological constraint. Unlike baselines (e.g., DeepGCNs) relying on dense residual connections, our framework enables the training of ultra-deep 64-layer networks without residual connections, maintaining perfectly stationary feature variance ($\sigma^2 \equiv 1.0$) on the ogbn-arxiv
benchmark. (3) Biological Physics: In reaction-
diffusion morphogenesis, it stabilizes Turing patterns against diffusive washout in high-
entropy regimes. Our results suggest that the MNCIS framework provides a base-independent topological condition for distinguishing viable complex systems from those collapsing into thermal
equilibrium.
Read the full abstract ▾
Machine Learning
A Preliminary Agentic Framework for Matrix Deflation
Paimon Goulart and Evangelos E. Papalexakis
Can a small team of
agents peel a
matrix apart, one rank-1 slice at a time? We propose an agentic approach to
matrix deflation in which a solver Large
Language Model (LLM) generates rank-1 Singular Value Decomposition (SVD) updates and a
Vision Language Model (VLM) accepts or rejects each update and decides when to stop, eliminating fixed norm thresholds. Solver stability is improved through in-context learning (ICL) and types of row/column permutations that expose visually coherent structure. We evaluate on Digits ($8{\times}8$), CIFAR-10 ($32{\times}32$ grayscale), and synthetic ($16{\times}16$)
matrices with and without Gaussian noise. In the synthetic noisy case, where the true construction rank $k$ is known, numerical deflation provides the noise target and our best agentic configuration differs by only $1.75$ RMSE of the target. For Digits and CIFAR-10, targets are defined by deflating until the Frobenius norm reaches $10\%$ of the original. Across all settings, our
agent achieves competitive results, suggesting that fully agentic, threshold-free deflation is a viable alternative to classical numerical
algorithms.
Read the full abstract ▾
Machine Learning
Forget Many, Forget Right: Scalable and Precise Concept Unlearning in Diffusion Models
Kaiyuan Deng, Gen Li, Yang Xiao et al. (+2 more)
Text-to-image
diffusion models have achieved remarkable progress, yet their use raises copyright and misuse concerns, prompting research into machine unlearning. However, extending multi-concept unlearning to large-scale scenarios remains difficult due to three challenges: (i) conflicting weight updates that hinder unlearning or degrade generation; (ii) imprecise mechanisms that cause collateral damage to similar content; and (iii) reliance on additional data or modules, creating scalability bottlenecks. To address these, we propose Scalable-Precise Concept Unlearning (ScaPre), a unified framework tailored for large-scale unlearning. ScaPre introduces a conflict-aware stable design, integrating spectral trace regularization and geometry
alignment to stabilize optimization, suppress conflicts, and preserve global structure. Furthermore, an Informax Decoupler identifies concept-relevant parameters and adaptively reweights updates, strictly confining unlearning to the target subspace. ScaPre yields an efficient closed-form solution without requiring auxiliary data or sub-models. Comprehensive experiments on objects, styles, and explicit content demonstrate that ScaPre effectively removes target concepts while maintaining generation quality. It forgets up to $\times \mathbf{5}$ more concepts than the best baseline within acceptable quality limits, achieving state-of-the-art precision and efficiency for large-scale unlearning. Code is available at https://github.com/kaiyuan02415/scapre
Read the full abstract ▾
All authors Kaiyuan Deng, Gen Li, Yang Xiao, Bo Hui, Xiaolong Ma
Machine Learning
Can we Improve Prediction of Psychotherapy Outcomes Through Pretraining With Simulated Data?
Niklas Jacobs, Manuel C. Voelkle, Norbert Kathmann et al. (+1 more)
In the context of personalized medicine, machine learning
algorithms are growing in popularity. These
algorithms require substantial information, which can be acquired effectively through the usage of previously gathered data. Open data and the utilization of synthetization techniques have been proposed to address this. In this paper, we propose and evaluate alternative approach that uses additional simulated data based on summary statistics published in the literature. The simulated data are used to pretrain random forests, which are afterwards fine-tuned on a real
dataset. We compare the predictive performance of the new approach to random forests trained only on the real data. A Monte Carlo Cross Validation (MCCV) framework with 100 iterations was employed to investigate significance and stability of the results. Since a first study yielded inconclusive results, a second study with improved methodology (i.e., systematic information extraction and different prediction outcome) was conducted. In Study 1, some pretrained random forests descriptively outperformed the standard random forest. However, this improvement was not significant (t(99) = 0.89, p = 0.19). Contrary to expectations, in Study 2 the random forest trained only with the real data outperformed the pretrained random forests. We conclude with a discussion of challenges, such as the scarcity of informative publications, and recommendations for future research.
Read the full abstract ▾
All authors Niklas Jacobs, Manuel C. Voelkle, Norbert Kathmann, Kevin Hilbert
Machine Learning
ECLIPTICA – A Framework for Switchable LLM Alignment via CITA - Contrastive Instruction-Tuned Alignment
Kapil Wanaskar, Gaytri Jena, Vinija Jain et al. (+2 more)
Alignment in large
language models (LLMs) is still largely static: after training, the policy is frozen. DPO, GRPO methods typically imprint one behavior into the weights, leaving little runtime control beyond
prompt hacks or expensive re-
alignment. We introduce ECLIPTICA, which treats
alignment as instruction-driven and runtime-controllable: natural-
language alignment instructions act as an explicit behavioral contract (stance, refusal boundary, verbosity) that modulates behavior on the fly under evolving safety requirements, user roles, and governance constraints. We introduce CITA (Contrastive Instruction-Tuned
Alignment), combining SFT with contrastive preference optimization under an explicit
geometric anchor to a reference model. This yields a stable Riemannian chart and keeps instruction updates within a shared neighborhood, so regimes stay nearby and traversable for reliable switching. To isolate policy switching from ordinary instruction following, we release the ECLIPTICA
benchmark: 3000 controlled cases (300
prompts x 10 instruction types) where the user request is fixed and only the
alignment instruction changes. On Llama-3.1-8B across five suites (ECLIPTICA, TruthfulQA, Conditional Safety, Length Control, LITMUS), CITA reaches 86.7% instruction-
alignment efficiency, beating DPO (56.1%), GRPO (36.1%), and PPO (20.4%).
Read the full abstract ▾
All authors Kapil Wanaskar, Gaytri Jena, Vinija Jain, Aman Chadha, Amitava Das
Machine Learning
PromptPort: A Reliability Layer for Cross-Model Structured Extraction
Varun Kotte
Structured extraction with LLMs fails in production not because models lack understanding, but because output formatting is unreliable across models and
prompts. A
prompt that returns clean JSON on GPT-4 may produce fenced, prose-wrapped, or malformed output on Llama, causing strict parsers to reject otherwise correct extractions. We formalize this as format collapse and introduce a dual-metric evaluation framework: ROS (strict parsing, measuring operational reliability) and CSS (post-canonicalization, measuring semantic capability). On a 37,346-example camera metadata
benchmark across six model families, we find severe format collapse (for example, Gemma-2B: ROS 0.116 versus CSS 0.246) and large cross-model portability gaps (0.4 to 0.6 F1). We then present PromptPort, a reliability layer combining deterministic canonicalization with a lightweight verifier (DistilBERT) and a safe-override policy. PromptPort recovers format failures (plus 6 to 8 F1), adds verifier-driven semantic selection (plus 14 to 16 F1 beyond canonicalization), and approaches per-field oracle performance (0.890 versus 0.896 in zero-shot) without modifying base models. The method generalizes to held-out model families and provides explicit abstention when uncertain, enabling reliable structured extraction in production deployments.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
Scaling Trends for Multi-Hop Contextual Reasoning in Mid-Scale Language Models
Brady Steele and Micah Katz
We present a controlled study of multi-hop contextual
reasoning in large
language models, providing a clean demonstration of the task-method dissociation: rule-based pattern matching achieves 100% success on structured information
retrieval but only 6.7% on tasks requiring cross-document
reasoning, while LLM-based multi-
agent systems show the inverse pattern, achieving up to 80% on
reasoning tasks where rule-based methods fail. Using a synthetic evaluation framework with 120 trials across four models (LLaMA-3 8B, LLaMA-2 13B, Mixtral 8x7B, DeepSeek-V2 16B), we report three key findings: (1) Multi-
agent amplification depends on base capability: statistically significant gains occur only for models with sufficient
reasoning ability (p < 0.001 for LLaMA-3 8B, p = 0.014 for Mixtral), with improvements of up to 46.7 percentage points, while weaker models show no benefit, suggesting amplification rather than compensation; (2) Active parameters predict
reasoning performance: Mixtral's performance aligns with its ~12B active parameters rather than 47B total, consistent with the hypothesis that inference-time compute drives
reasoning capability in MoE architectures; (3) Architecture quality matters: LLaMA-3 8B outperforms LLaMA-2 13B despite fewer parameters, consistent with known training improvements. Our results provide controlled quantitative evidence for intuitions about multi-
agent coordination and MoE scaling, while highlighting the dependence of multi-
agent benefits on base model capability. We release our evaluation framework to support reproducible research on
reasoning in mid-scale models.
Read the full abstract ▾
Machine Learning
Green MLOps: Closed-Loop, Energy-Aware Inference with NVIDIA Triton, FastAPI, and Bio-Inspired Thresholding
Mustapha Hamdi and Mourad Jabou
Energy efficiency is a first-order concern in AI deployment, as long-running inference can exceed training in cumulative carbon impact. We propose a bio-inspired framework that maps
protein-folding energy basins to inference cost landscapes and controls execution via a decaying, closed-loop threshold. A request is admitted only when the expected utility-to-energy trade-off is favorable (high confidence/utility at low marginal energy and congestion), biasing operation toward the first acceptable local basin rather than pursuing costly global minima. We evaluate DistilBERT and ResNet-18 served through FastAPI with ONNX Runtime and NVIDIA Triton on an RTX 4000 Ada GPU. Our ablation study reveals that the bio-controller reduces processing time by 42% compared to standard open-loop execution (0.50s vs 0.29s on A100 test set), with a minimal accuracy degradation (<0.5%). Furthermore, we establish the efficiency boundaries between lightweight local serving (ORT) and managed batching (Triton). The results connect biophysical energy models to Green MLOps and offer a practical, auditable basis for closed-loop energy-aware inference in production.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
Toward Maturity-Based Certification of Embodied AI: Quantifying Trustworthiness Through Measurement Mechanisms
Michael C. Darling, Alan H. Hesu, Michael A. Mardikes et al. (+2 more)
We propose a maturity-based framework for certifying
embodied AI systems through explicit measurement mechanisms. We argue that certifiable
embodied AI requires structured assessment frameworks, quantitative scoring mechanisms, and methods for navigating multi-objective trade-offs inherent in trustworthiness evaluation. We demonstrate this approach using uncertainty quantification as an exemplar measurement mechanism and illustrate feasibility through an Uncrewed Aircraft System (UAS) detection case study.
Read the full abstract ▾
All authors Michael C. Darling, Alan H. Hesu, Michael A. Mardikes, Brian C. McGuigan, Reed M. Milewicz
Machine Learning · Cross-listed from cs.CV
Latent Geometry of Taste: Scalable Low-Rank Matrix Factorization for Recommender Systems
Joshua Salako
Scalability and data sparsity remain critical bottlenecks for collaborative filtering on massive interaction
datasets. This work investigates the latent geometry of user preferences using the MovieLens 32M
dataset, implementing a high-performance, parallelized Alternating Least Squares (ALS) framework. Through extensive hyperparameter optimization, we demonstrate that constrained low-rank models significantly outperform higher dimensional counterparts in generalization, achieving an optimal balance between Root Mean Square Error (RMSE) and ranking precision. We visualize the learned
embedding space to reveal the unsupervised emergence of semantic genre clusters, confirming that the model captures deep structural relationships solely from interaction data. Finally, we validate the system's practical utility in a cold-start scenario, introducing a tunable scoring parameter to manage the trade-off between popularity bias and personalized affinity effectively. The codebase for this research can be found here: https://github.com/joshsalako/recommender.git
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Experimental Comparison of Light-Weight and Deep CNN Models Across Diverse Datasets
Md. Hefzul Hossain Papon and Shadman Rabby
Our results reveal that a well-regularized shallow architecture can serve as a highly competitive baseline across heterogeneous domains - from smart-city surveillance to agricultural variety classification - without requiring large GPUs or specialized pre-trained models. This work establishes a unified, reproducible
benchmark for multiple Bangladeshi
vision datasets and highlights the practical value of lightweight CNNs for real-world deployment in low-resource settings.
Read the full abstract ▾
Machine Learning
An Expectation-Maximization Algorithm for Domain Adaptation in Gaussian Causal Models
Mohammad Ali Javidian
We study the problem of imputing a designated target variable that is systematically missing in a shifted deployment domain, when a Gaussian causal DAG is available from a fully observed source domain. We propose a unified EM-based framework that combines source and target data through the DAG structure to transfer information from observed variables to the missing target. On the methodological side, we formulate a population EM operator in the DAG parameter space and introduce a first-order (gradient) EM update that replaces the costly generalized least-squares M-step with a single projected gradient step. Under standard local strong-concavity and smoothness assumptions and a BWY-style \cite{Balakrishnan2017EM} gradient-stability (bounded missing-information) condition, we show that this first-order EM operator is locally contractive around the true target parameters, yielding
geometric convergence and finite-sample guarantees on parameter error and the induced target-imputation error in Gaussian SEMs under
covariate shift and local mechanism shifts. Algorithmically, we exploit the known causal DAG to freeze source-invariant mechanisms and re-estimate only those conditional distributions directly affected by the shift, making the procedure scalable to higher-dimensional models. In experiments on a synthetic seven-node SEM, the 64-node MAGIC-IRRI genetic network, and the Sachs
protein-signaling data, the proposed DAG-aware first-order EM
algorithm improves target imputation accuracy over a fit-on-source
Bayesian network and a Kiiveri-style EM baseline, with the largest gains under pronounced domain shift.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ME
Measures of classification bias derived from sample size analysis
Ioannis Ivrissimtzis, Shauna Concannon, Matthew Houliston et al. (+1 more)
We propose the use of a simple intuitive principle for measuring algorithmic classification bias: the significance of the differences in a classifier's error rates across the various demographics is inversely commensurate with the sample size required to statistically detect them. That is, if large sample sizes are required to statistically establish biased behavior, the
algorithm is less biased, and vice versa. In a simple setting, we assume two distinct demographics, and non-parametric estimates of the error rates on them, e1 and e2, respectively. We use a well-known approximate formula for the sample size of the chi-squared test, and verify some basic desirable properties of the proposed measure. Next, we compare the proposed measure with two other commonly used statistics, the difference e2-e1 and the ratio e2/e1 of the error rates. We establish that the proposed measure is essentially different in that it can rank
algorithms for bias differently, and we discuss some of its advantages over the other two measures. Finally, we briefly discuss how some of the desirable properties of the proposed measure emanate from fundamental characteristics of the method, rather than the approximate sample size formula we used, and thus, are expected to hold in more complex settings with more than two demographics.
Read the full abstract ▾
All authors Ioannis Ivrissimtzis, Shauna Concannon, Matthew Houliston, Graham Roberts
Machine Learning · Cross-listed from stat.ML
Microeconomic Foundations of Multi-Agent Learning
Nassim Helou
Modern AI systems increasingly operate inside
markets and institutions where data, behavior, and incentives are endogenous. This paper develops an economic foundation for multi-
agent learning by studying a principal-
agent interaction in a Markov decision process with strategic externalities, where both the principal and the
agent learn over time. We propose a two-phase incentive mechanism that first estimates implementable transfers and then uses them to steer long-run dynamics; under mild regret-based rationality and exploration conditions, the mechanism achieves sublinear social-welfare regret and thus asymptotically optimal welfare. Simulations illustrate how even coarse incentives can correct inefficient learning under stateful externalities, highlighting the necessity of incentive-aware design for safe and welfare-aligned AI in
markets and insurance.
Read the full abstract ▾
Machine Learning
Soft Contextualized Encoder For User Defined Text Classification
Charu Maheshwari and Vyas Raina
User-Defined Text Classification (UDTC) considers the challenge of classifying input text to user-specified, previously unseen classes, a setting that arises frequently in real-world applications such as enterprise analytics, content moderation, and domain-specific information
retrieval. We propose a soft-contextualized encoder architecture for UDTC which contextualizes each candidate label with the label set and a static soft
prompt representation of the input query. Training on diverse, multi-source
datasets enables the model to generalize effectively to zero-shot classification over entirely unseen topic sets drawn from arbitrary domains. We evaluate the proposed architecture both on held-out in-distribution test data and on multiple unseen UDTC
benchmarks. Across
datasets, the model achieves state-of-the-art performance, consistently outperforming or matching the baselines.
Read the full abstract ▾
Machine Learning · Cross-listed from eess.SY
Local Updates in Distributed Optimization: Provable Acceleration and Topology Effects
Zuang Wang and Yongqiang Wang
Inspired by the success of performing multiple local optimization steps between communication rounds in
federated learning, incorporating such local updates into distributed optimization has recently attracted growing interest. However, unlike
federated learning, where local updates can accelerate training by reducing gradient estimation error under minibatch settings, it remains unclear whether similar benefits persist when exact gradients are available. Moreover, existing theoretical results typically require reducing the step size when multiple local updates are employed, which can entirely offset any potential benefit of these additional local updates. In this paper, we focus on the classic DIGing
algorithm and leverage the tight performance bounds provided by Performance Estimation Problems (PEP) to show that incorporating local updates can indeed accelerate distributed optimization. To the best of our knowledge, this is the first rigorous demonstration of such acceleration for a broad class of objective functions. Our analysis further reveals that, under an appropriate step size, performing only two local updates is sufficient to achieve the maximal possible improvement, and that additional local updates provide no further gains. Because more updates increase computational cost, these findings offer practical guidance for efficient implementation. We also show that these speed gains depend critically on the network structure, with sparser or less connected graphs, characterized by the spectral properties of the mixing
matrix, yielding smaller improvements. Extensive experiments on both synthetic and real-world
datasets corroborate the theoretical findings.
Read the full abstract ▾
Machine Learning
VNU-Bench: A Benchmarking Dataset for Multi-Source Multimodal News Video Understanding
Zibo Liu, Muyang Li, Zhe Jiang et al. (+1 more)
News
videos are carefully edited
multimodal narratives that combine narration, visuals, and external quotations into coherent storylines. In recent years, there have been significant advances in evaluating
multimodal large
language models (MLLMs) for news
video understanding. However, existing
benchmarks largely focus on single-source, intra-
video reasoning, where each report is processed in isolation. In contrast, real-world news consumption is inherently multi-sourced: the same event is reported by different outlets with complementary details, distinct narrative choices, and sometimes conflicting claims that unfold over time. Robust news understanding, therefore, requires models to compare perspectives from different sources, align
multimodal evidence across sources, and synthesize multi-source information. To fill this gap, we introduce VNU-Bench, the first
benchmark for multi-source, cross-
video understanding in the news domain. We design a set of new question types that are unique in testing models' ability of understanding multi-source
multimodal news from a variety of different angles. We design a novel hybrid human-model QA generation process that addresses the issues of scalability and quality control in building a large
dataset for cross-source news understanding. The
dataset comprises 429 news groups, 1,405
videos, and 2,501 high-quality questions. Comprehensive evaluation of both closed- and open-source
multimodal models shows that VNU-Bench poses substantial challenges for current MLLMs.
Read the full abstract ▾
All authors Zibo Liu, Muyang Li, Zhe Jiang, Shigang Chen
Machine Learning · Cross-listed from cs.CR
DeepLeak: Privacy Enhancing Hardening of Model Explanations Against Membership Leakage
Firas Ben Hmida, Zain Sbeih, Philemon Hailemariam et al. (+1 more)
Machine learning (ML) explainability is central to algorithmic transparency in high-stakes settings such as predictive diagnostics and loan approval. However, these same domains require rigorous
privacy guaranties, creating tension between interpretability and
privacy. Although prior work has shown that explanation methods can leak membership information, practitioners still lack systematic guidance on selecting or deploying explanation techniques that balance transparency with
privacy. We present DeepLeak, a system to audit and mitigate
privacy risks in post-hoc explanation methods. DeepLeak advances the state-of-the-art in three ways: (1) comprehensive leakage profiling: we develop a stronger explanation-aware membership inference attack (MIA) to quantify how much representative explanation methods leak membership information under default configurations; (2) lightweight hardening strategies: we introduce practical, model-agnostic mitigations, including sensitivity-calibrated noise, attribution clipping, and masking, that substantially reduce membership leakage while preserving explanation utility; and (3) root-cause analysis: through controlled experiments, we pinpoint algorithmic properties (e.g., attribution sparsity and sensitivity) that drive leakage. Evaluating 15 explanation techniques across four families on image
benchmarks, DeepLeak shows that default settings can leak up to 74.9% more membership information than previously reported. Our mitigations cut leakage by up to 95% (minimum 46.5%) with only <=3.3% utility loss on average. DeepLeak offers a systematic, reproducible path to safer explainability in
privacy-sensitive ML.
Read the full abstract ▾
All authors Firas Ben Hmida, Zain Sbeih, Philemon Hailemariam, Birhanu Eshete
Machine Learning
The Illusion of Specialization: Unveiling the Domain-Invariant "Standing Committee" in Mixture-of-Experts Models
Yan Wang, Yitao Xu, Nanhan Shen et al. (+3 more)
Mixture of Experts models are widely assumed to achieve domain specialization through sparse routing. In this work, we question this assumption by introducing COMMITTEEAUDIT, a post hoc framework that analyzes routing behavior at the level of expert groups rather than individual experts. Across three representative models and the MMLU
benchmark, we uncover a domain-invariant Standing Committee. This is a compact coalition of routed experts that consistently captures the majority of routing mass across domains, layers, and routing budgets, even when architectures already include shared experts. Qualitative analysis further shows that Standing Committees anchor
reasoning structure and syntax, while peripheral experts handle domain-specific knowledge. These findings reveal a strong structural bias toward centralized computation, suggesting that specialization in Mixture of Experts models is far less pervasive than commonly believed. This inherent bias also indicates that current training objectives, such as load-balancing losses that enforce uniform expert utilization, may be working against the model's natural optimization path, thereby limiting training efficiency and performance.
Read the full abstract ▾
All authors Yan Wang, Yitao Xu, Nanhan Shen, Jinyan Su, Jimin Huang, Zining Zhu
Machine Learning
Spectral Archaeology: The Causal Topology of Model Evolution
Valentin Noël
Behavioral
benchmarks tell us what a model does, but not how. We introduce a training-free mechanistic probe using
attention-graph spectra. Treating each layer as a
token graph, we compute algebraic connectivity ($\lambda_2$), smoothness, and spectral
entropy. Across 12 models and 10
languages, these measures yield stable “spectral fingerprints” that expose discontinuities missed by standard evaluation. We report four results. (1) Models undergoing specific curriculum transitions (e.g., code-to-chat) show an English-only, syntax-triggered connectivity failure on non-canonical constructions, reaching $\Delta\lambda_2 \approx -0.76$. We term this scar Passive-Triggered Connectivity Collapse (PTCC). Analysis of the Phi lineage reveals that PTCC appears and resolves across developmental stages, implicating brittle curriculum shifts rather than synthetic data per se. (2) PTCC reflects a specialization trade-off: strengthened formal routing at the expense of stylistic flexibility. (3) We identify four recurrent processing strategies; simple frozen-threshold rules enable perfect forensic identification across lineages. (4) Mechanistically, PTCC localizes to a sparse Layer 2 “compensatory patch” of heads that fails under syntactic stress; activation steering can partially restore connectivity, recovering $\approx 38\%$ of lost information flow. Finally, dominant topological regimes track tokenization density more than
language identity, suggesting “healthy” geometry varies systematically across scripts. Overall,
attention-graph spectra provide a practical tool for
auditing and training-regime verification.
Read the full abstract ▾
Machine Learning
Jailbreaking LLMs Without Gradients or Priors: Effective and Transferable Attacks
Zhakshylyk Nurlanov, Frank R. Schmidt and Florian Bernard
As Large
Language Models (LLMs) are increasingly deployed in safety-critical domains, rigorously evaluating their robustness against
adversarial jailbreaks is essential. However, current safety evaluations often overestimate robustness because existing automated attacks are limited by restrictive assumptions. They typically rely on handcrafted priors or require white-box access for gradient propagation. We challenge these constraints by demonstrating that
token-level iterative optimization can succeed without gradients or priors. We introduce RAILS (RAndom Iterative Local Search), a framework that operates solely on model logits. RAILS matches the effectiveness of gradient-based methods through two key innovations: a novel auto-regressive loss that enforces exact prefix matching, and a history-based selection strategy that bridges the gap between the proxy optimization objective and the true attack success rate. Crucially, by eliminating gradient dependency, RAILS enables cross-tokenizer ensemble attacks. This allows for the discovery of shared
adversarial patterns that generalize across disjoint vocabularies, significantly enhancing transferability to closed-source systems. Empirically, RAILS achieves near 100% success rates on multiple open-source models and high black-box attack transferability to closed-source systems like GPT and Gemini.
Read the full abstract ▾
Machine Learning
Sensor to Pixels: Decentralized Swarm Gathering via Image-Based Reinforcement Learning
Yigal Koifman, Eran Iceland, Erez Koifman et al. (+2 more)
This study highlights the potential of image-based reinforcement learning methods for addressing swarm-related tasks. In multi-
agent reinforcement learning, effective policy learning depends on how
agents sense, interpret, and process inputs. Traditional approaches often rely on handcrafted feature extraction or raw vector-based representations, which limit the scalability and efficiency of learned policies concerning input order and size. In this work we propose an image-based reinforcement learning method for decentralized control of a multi-
agent system, where observations are encoded as structured visual inputs that can be processed by Neural Networks, extracting its spatial features and producing novel decentralized motion control rules. We evaluate our approach on a multi-
agent convergence task of
agents with limited-range and bearing-only sensing that aim to keep the swarm cohesive during the aggregation. The
algorithm's performance is evaluated against two
benchmarks: an analytical solution proposed by Bellaiche and Bruckstein, which ensures convergence but progresses slowly, and VariAntNet, a neural network-based framework that converges much faster but shows medium success rates in hard constellations. Our method achieves high convergence, with a pace nearly matching that of VariAntNet. In some scenarios, it serves as the only practical alternative.
Read the full abstract ▾
All authors Yigal Koifman, Eran Iceland, Erez Koifman, Ariel Barel, Alfred M. Bruckstein
Machine Learning
Inferring Clinically Relevant Molecular Subtypes of Pancreatic Cancer from Routine Histopathology Using Deep Learning
Abdul Rehman Akbar, Alejandro Levya, Ashwini Esnakula et al. (+12 more)
Molecular subtyping of PDAC into basal-like and classical has established prognostic and predictive value. However, its use in clinical practice is limited by cost, turnaround time, and tissue requirements, thereby restricting its application in the management of PDAC. We introduce PanSubNet, an interpretable deep learning framework that predicts therapy-relevant molecular subtypes directly from standard H&E-stained WSIs. PanSubNet was developed using data from 1,055 patients across two multi-institutional cohorts (PANCAN, n=846; TCGA, n=209) with paired histology and RNA-seq data. Ground-truth labels were derived using the validated Moffitt 50-gene signature refined by GATA6 expression. The model employs dual-scale architecture that fuses cellular-level morphology with tissue-level architecture, leveraging
attention mechanisms for multi-scale representation learning and transparent feature attribution. On internal validation within PANCAN using five-fold cross-validation, PanSubNet achieved mean AUC of 88.5% with balanced sensitivity and specificity. External validation on the independent TCGA cohort without fine-tuning demonstrated robust generalizability (AUC 84.0%). PanSubNet preserved and, in metastatic disease, strengthened prognostic stratification compared to RNA-seq based labels. Prediction uncertainty linked to intermediate transcriptional states, not classification noise. Model predictions are aligned with established transcriptomic programs, differentiation markers, and DNA damage repair signatures. By enabling rapid, cost-effective molecular stratification from routine H&E-stained slides, PanSubNet offers a clinically deployable and interpretable tool for genetic subtyping. We are gathering data from two institutions to validate and assess real-world performance, supporting integration into digital pathology workflows and advancing precision oncology for PDAC.
Read the full abstract ▾
All authors Abdul Rehman Akbar, Alejandro Levya, Ashwini Esnakula, Elshad Hasanov, Anne Noonan, Lingbin Meng, Susan Tsai, Vaibhav Sahai, Midhun Malla, Sarbajit Mukherjee, Upender Manne, Anil Parwani, Wei Chen, Ashish Manne, Muhammad Khalid Khan Niazi
Machine Learning · Cross-listed from cs.CE
PIVONet: A Physically-Informed Variational Neuro ODE Model for Efficient Advection-Diffusion Fluid Simulation
Hei Shing Cheung, Qicheng Long and Zhiyue Lin
We present PIVONet (Physically-Informed Variational ODE Neural Network), a unified framework that integrates Neural Ordinary Differential Equations (Neuro-ODEs) with Continuous Normalizing Flows (CNFs) for
stochastic fluid simulation and visualization. First, we demonstrate that a physically informed model, parameterized by CNF parameters {\theta}, can be trained offline to yield an efficient surrogate simulator for a specific fluid system, eliminating the need to simulate the full dynamics explicitly. Second, by introducing a variational model with parameters {\phi} that captures latent stochasticity in observed fluid trajectories, we model the network output as a variational distribution and optimize a pathwise Evidence Lower Bound (ELBO), enabling
stochastic ODE integration that captures
turbulence and random fluctuations in fluid motion (advection-
diffusion behaviors).
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
Exploration Through Introspection: A Self-Aware Reward Model
Michael Petrowski and Milica Gašić
Understanding how artificial
agents model internal mental states is central to advancing Theory of Mind in AI. Evidence points to a unified system for self- and other-awareness. We explore this self-awareness by having reinforcement learning
agents infer their own internal states in gridworld environments. Specifically, we introduce an introspective exploration component that is inspired by biological pain as a learning signal by utilizing a hidden Markov model to infer "pain-belief" from online observations. This signal is integrated into a subjective reward function to study how self-awareness affects the
agent's learning abilities. Further, we use this computational framework to investigate the difference in performance between normal and chronic pain perception models. Results show that introspective
agents in general significantly outperform standard baseline
agents and can replicate complex human-like behaviors.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Metaphors are a Source of Cross-Domain Misalignment of Large Reasoning Models
Zhibo Hu, Chen Wang, Yanfeng Shu et al. (+2 more)
Earlier research has shown that metaphors influence human decision-making, raising the question of whether metaphors also influence large
language models (LLMs)'
reasoning pathways, given that their training data contain a large number of metaphors. In this work, we investigate the problem in the scope of the emergent misalignment problem, where LLMs can generalize patterns learned from misaligned content in one domain to another domain. We find strong evidence that metaphors in training data contribute to cross-domain misalignment in LLMs'
reasoning outputs. With metaphor-based interventions during continued pre-training and fine-tuning for inducing misalignment, models exhibit significantly different degrees of emergent cross-domain misalignment. We also observe similar effects in re-
alignment settings. As we further investigate this phenomenon, we find that metaphors are linked to the activation of latent features in large
reasoning models. By monitoring these latent features, we design a detector that predicts misaligned content with high accuracy.
Read the full abstract ▾
All authors Zhibo Hu, Chen Wang, Yanfeng Shu, Hye-young Paik, Liming Zhu
Machine Learning
SIGMA: Scalable Spectral Insights for LLM Model Collapse
Yi Gu, Lingyou Pang, Xiangkun Ye et al. (+4 more)
The rapid adoption of synthetic data for training Large
Language Models (LLMs) has introduced the technical challenge of "model collapse"-a degenerative process where recursive training on model-generated content leads to a contraction of distributional variance and representational quality. While the phenomenology of collapse is increasingly evident, rigorous methods to quantify and predict its onset in high-dimensional spaces remain elusive. In this paper, we introduce SIGMA (Spectral Inequalities for Gram
Matrix Analysis), a unified framework that
benchmarks model collapse through the spectral lens of the
embedding Gram
matrix. By deriving and utilizing deterministic and
stochastic bounds on the
matrix's spectrum, SIGMA provides a mathematically grounded metric to track the contraction of the representation space. Crucially, our
stochastic formulation enables scalable estimation of these bounds, making the framework applicable to large-scale foundation models where full eigendecomposition is intractable. We demonstrate that SIGMA effectively captures the transition towards degenerate states, offering both theoretical insights into the mechanics of collapse and a practical, scalable tool for monitoring the health of recursive training pipelines.
Read the full abstract ▾
All authors Yi Gu, Lingyou Pang, Xiangkun Ye, Tianyu Wang, Jianyu Lin, Carey E. Priebe, Alexander Aue