Artificial Intelligence
Sequential knowledge editing breaks a model's ability to tell good evidence from bad, without costing it accuracy
Atul Anand
Knowledge editing is evaluated on whether the edited fact changed, whether paraphrases follow, and whether unrelated answers stayed put. A model can pass all three and still lose something none of them measures: the ability to decide, on facts that were never edited, which retrieved documents to believe. We score the log odds a model assigns to its remembered answer against the answer an injected passage asserts, before and after editing, holding the query, the passage and both candidate strings fixed. Our cleanest arm is a conservatively tuned LoRA: after 1,000 sequential edits on Qwen2.5-7B-Instruct it leaves MMLU unchanged to four decimal places, yet the spread of the arbitration quantity across untouched facts falls by 36%. Selective prediction degrades with it. Area under the risk-coverage curve rises by 0.107, against 0.005 for a norm-matched perturbation at the same MMLU, and error on the model's most confident quarter of arbitration decisions goes from 0.217 to 0.342. This is not capability loss. Sweeping random perturbation over five severities, damage bad enough to cut MMLU from 0.6275 to 0.3725 produces less harm (0.088) than MEMIT does at 0.6050 (0.102). The effect holds across three seeds, two model families, two
datasets, two probe-disjointness criteria, three
prompt templates and paraphrased queries. Layer ablation on saved weight deltas shows it is distributed: no single layer reproduces it, and removing any one recovers about half. Under
retrieval with a frozen retriever, accuracy falls from 0.592 to 0.46. A secondary finding may matter more in practice. Three of five model and method pairings we ran collapse to chance MMLU at 1,000 sequential edits under published hyperparameters, while edit success stays at 1.00 and locality reads clean. Sequential-editing evaluations that never measure capability cannot see this.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CL
PII-TRACE: A Benchmark for Context-Aware PII Detection in Multi-Turn LLM Conversations
Kaiyuan Zhang, Chuan Wang, Joey Zhong et al. (+4 more)
LLM assistants and agentic systems log long multi-turn conversations. AI providers often scan these conversations for Personally Identifiable Information (PII) and mask the PII before storing or processing conversation data. Yet most PII detectors and
benchmarks target self-contained records rather than cross-turn evaluation. To evaluate PII detection across turns in multi-turn conversations, we introduce PII-TRACE (Tracing Recurring PII Across Conversational Exchanges), to our knowledge the first PII
benchmark to assess whether detectors identify PII in conversational contexts and cover every mention of a recurring identifier across turns. PII-TRACE contains 13,148 synthetic multi-turn dialogues in 13
languages with character-level spans and identifier clusters. Across eleven baselines, including frontier LLMs, no detector achieves full entity-level coverage without substantial false positives on PII-free conversations, and single-pass reading loses a third of the gold characters on long dialogues. To close this gap, we introduce PII-Tracer, a compact 0.6B-parameter detector trained with conversation-level supervision. PII-Tracer attains the highest entity-level coverage of any system we evaluate and also performs strongly on standard single-record
benchmarks.
Read the full abstract ▾
All authors Kaiyuan Zhang, Chuan Wang, Joey Zhong, Paul Fryzel, Kyle Polley, Jerry Ma, Ninghui Li
Artificial Intelligence · Cross-listed from cs.LG
Improving Parameter Utilization by Sharing Neural Experts Across Layers in Transformers
Dian Jiao, Jiaxin Duan, Shuai Zhao et al. (+3 more)
Transformer-based large
language models often suffer from inter-layer parameter redundancy, where functional transformations are redundantly learned across network depths. We propose CS-MoE, a novel
Transformer architecture featuring cross-layer expert sharing to address this inefficiency. Deviating from the widely used Mixture-of-Experts (MoE) architecture that terminates each
Transformer block with layer-isolated experts, CS-MoE combines layer-independent experts with concurrent access to a centralized, globally shared expert pool. This Global Experts Sharing mechanism enables elastic control over
token-level parameter activation and computational consumption (FLOPs). Experiments demonstrate that CS-MoE achieves lower perplexity than equal-scale dense
Transformers while activating only 55% of parameters. Furthermore, its performance scales monotonically with an increased number of activated experts and approaches MoE counterparts that consume more FLOPs by expanding the shared pool with a fixed FLOPs budget. CS-MoE also establishes a flexible Pareto frontier between computational cost and model capacity, offering an efficient alternative for computation-constrained environments.
Read the full abstract ▾
All authors Dian Jiao, Jiaxin Duan, Shuai Zhao, Jiabing Leng, Yiran Zhang, Feng Huang
Artificial Intelligence · Cross-listed from cs.CV
AgenticGen: Reward-Guided Agentic Video Generation for Advertising
Xingyuan Bu, Chengru Song, Hao Zhou et al. (+9 more)
Advertising
video generation is not only a
video synthesis task, but also a product-conditioned
reasoning problem whose success is measured by online business metrics. Recent
video foundation models can generate realistic clips from
multimodal conditions, yet they do not optimize how a product should be transformed into an effective advertisement or how future generation should be improved from online business feedback. To close this loop, we propose AgenticGen, a reward-guided agentic framework that decomposes advertising
video generation into two trainable
reasoning stages, strategy selection and draft generation, thereby exposing optimization targets that online business feedback can supervise. AgenticGen learns a performance-based reward from accumulated online feedback and a complementary rubric-based reward aligned with human quality standards, then uses them to supervise policy optimization. DPO first moves the agentic policies toward online preferences, and GRPO further refines both stages with process and outcome rewards. Offline experiments validate the reward models and successive policy optimization. Online A/B experiments in the TikTok advertising system show that AgenticGen after DPO and GRPO improves CTR by 2.72%, CVR by 2.63%, and Advv by 9.61% over the SFT baseline.
Read the full abstract ▾
All authors Xingyuan Bu, Chengru Song, Hao Zhou, Tao Zhou, Dong Li, Wei Li, Shilong Li, Hao Shi, Yongxin Guo, Donghao Zhou, Qiangpeng Yang, Shilei Wen
Artificial Intelligence · Cross-listed from cs.RO
Robots Influencing Humans to Reveal their Goals during Collaboration and Competition
Debasmita Ghose, Oz Gitelson, Michal Lewkowicz et al. (+3 more)
We propose a unified strategy for fast goal inference in human-
robot interaction. The core idea is to drive the human toward Critical Decision Points (CDPs)-states where competing human strategies prescribe different next actions and thus maximally reveal the goal. We formalise CDPs using a goal-conditioned policy divergence measure and incorporate them into a Receding-Horizon Planner that explores future action sequences while optimizing a cost function balancing task progress and information gain. We evaluate this approach in both a collaborative, fully observable cooking task and a competitive, partially observable hide-and-seek game, each in simulation and on real
robots. In both scenarios, our method infers human goals more accurately and earlier than baseline strategies.
Read the full abstract ▾
All authors Debasmita Ghose, Oz Gitelson, Michal Lewkowicz, Jake Brawer, Marynel Vazquez, Brian Scassellati
Artificial Intelligence · Cross-listed from cs.CV
CrossModalQA: A Cross-modal and Multi-hop Benchmark for Multimodal Retrieval-augmented Generation
Jiacheng Cai, Zijin Hong, Zheng Yuan et al. (+3 more)
Despite the strong capabilities of
multimodal large
language models (MLLMs), their parametric knowledge remains incomplete and difficult to update, motivating
multimodal retrieval-augmented generation (RAG) to ground responses in external text and images. However, existing
benchmarks face two major limitations: (i) they typically emphasize single-hop
retrieval or
reasoning over a small set of provided contexts rather than open-domain evidence discovery; and (ii) they provide fragmented coverage of cross-modal
reasoning paths, leaving complex multi-hop and multi-image
reasoning underexplored. In this paper, we introduce CrossModalQA, an open-domain
benchmark for evaluating
multimodal retrieval and
reasoning over heterogeneous corpora. CrossModalQA contains 1,863 question-answer pairs constructed from 4,987 Wikipedia articles and 4,431 Wikimedia Commons images. It covers five complementary
reasoning paths:
vision-to-text, text-to-
vision,
vision-to-text-to-
vision, multi-image intersection, and image-set
reasoning. Every question requires retrieving and composing distributed textual and visual evidence, with an average
reasoning depth of 3.50 hops. We construct the
benchmark through
multimodal knowledge graph-guided subgraph sampling and apply rule-based consistency checking and LLM verification to ensure
multimodal dependence and traceable evidence. Extensive experiments demonstrate that existing
multimodal RAG systems struggle to recover complete evidence chains and can underperform closed-book models when incomplete
retrieval introduces distracting context. Further analysis reveals that complete cross-modal
retrieval contributes more to answer accuracy than generator scaling, while multi-image
retrieval and
reasoning remain the primary bottlenecks limiting end-to-end performance.
Read the full abstract ▾
All authors Jiacheng Cai, Zijin Hong, Zheng Yuan, Huachi Zhou, Qinggang Zhang, Xiao Huang
Artificial Intelligence · Cross-listed from cs.CV
Han Zhang
Human observers prioritize visual information according to task goals. Most computational models of naturalistic viewing are gaze-trained for free viewing, leaving open whether goal-directed
attention can emerge in systems without gaze supervision. We tested two off-the-shelf
vision-
language models (VLMs), Qwen3-VL-32B-Thinking and Gemma-4-26B-A4B-it, on 4,887 naturalistic scenes under visual-search and free-viewing instructions. Model predictions were compared with human fixations on the same images under corresponding tasks. Both models aligned more closely with human fixations under matching goals than under mismatched goals. This crossover persisted in target-absent scenes, where
alignment could not be explained by simple visual grounding, and appeared in decoder-layer readouts. Furthermore, model-thinking traces were grounded in target semantics during search and in visual prominence during free viewing. These findings show that general-purpose VLMs can generate human-aligned, goal-directed spatial priorities without gaze-specific training, informing theories of goal-directed
attention and offering scalable tools for predicting where people look across tasks.
Read the full abstract ▾
Artificial Intelligence
The Failure Happens Before the Drift: The Social Dynamics of Values in LLM Agent Societies
Farah Atif, Sougata Saha and Monojit Choudhury
Large
Language Model (LLM)-based
agents are increasingly used as proxies for human participants in social science research, yet it remains unclear whether they can faithfully simulate diverse and conflicting human value systems. We present a World Values Survey (WVS)-grounded simulation framework where culturally diverse
agents with different communication styles engage in longitudinal, value-laden discussions. Across approximately 4,000 conversations involving 1,200 personas, 15 topics, and three models (GPT-4o, Gemini-2.5-Flash, and Gemma-4-E4B), we evaluate value faithfulness, value drift, and conversational realism. We find that more than 50% of personas fail to express their assigned WVS profiles from the outset, while 2-7% drift after repeated conversations. Ablations removing demographic details improve faithfulness for some models but do not change the broader trend: simulated value distributions still systematically deviate from the assigned WVS profiles. Compared to human discussions, simulated dialogues show a different trade-off between stylistic consistency and semantic diversity, often producing content-wise varied but stylistically repetitive exchanges. These findings suggest that current LLM
agents can generate plausible conversations, but remain limited proxies for representing and preserving diverse human value profiles over time.
Read the full abstract ▾
Artificial Intelligence
When and What to Teach: Budget-Aware Online Adaptation for Web Agents
Jianwei Zhang, Sihan Cao, Pengcheng Zheng et al. (+7 more)
Web
agents have achieved significant success in automating complex internet tasks but deploying them in real-world environments requires continuous online adaptation. Given that deploying powerful proprietary models remains commercially cost-prohibitive, practitioners must rely on lightweight local models that evolve post-deployment via online teaching from a stronger teacher. However, standard interactive feedback imposes prohibitive costs. We show that conventional trajectory-level preference optimization wastes budget on both unresolvable episodes and redundant execution turns. To resolve these inefficiencies, we propose Score-Guided Online Teaching with Budgeted Trajectory Trimming, a budget-aware framework that systematically orchestrates when and what to teach. Specifically, our framework integrates a solvability-aware teacher gate to dictate when to query the teacher model and a score-guided turn selection mechanism to decide what informative turns to retain. Extensive experiments on MiniWoB and TimeWarp demonstrate that our method achieves comparable first-pass success while reducing teacher calls by 22.6% and student training compute by 52.1% on average. Our code is available at https://github.com/zjw131f1fc/budgeted-online-teaching.
Read the full abstract ▾
All authors Jianwei Zhang, Sihan Cao, Pengcheng Zheng, Ya Wen, Pei Ke, Kuien Liu, Shen Gao, Wei Dong, Yang Yang, Chaoning Zhang
Artificial Intelligence
Reasoning-Aware Compression: Identifying and Protecting Vulnerable Reasoning Circuits for Energy-Efficient LLM Deployment
Leonard Twagirayezu and Prasenjit Mitra
Large
Reasoning Models (LRMs) impose substantial energy costs during deployment, yet current compression methods apply uniform quantization across all components, risking damage to critical
reasoning circuits. We present a
reasoning-aware compression framework that
benchmarks quantization conditions across five
reasoning benchmarks, GSM8K, FOLIO, MATH-500, ProofWriter, and MuSiQue, with hardware-level GPU energy measurement; profiles per-module INT4 vulnerability across all 196-224 (layer, projection) pairs via a perturbation sweep on a held-out calibration split, then selectively restores the most sensitive
circuits to FP16. Three findings emerge. First, INT4 quantization can increase energy by extending
reasoning chains; a 25% power reduction becomes a net energy increase on GSM8K. Second, vulnerability is task-dependent:
attention projections are more critical for mathematical
reasoning, and sensitivity patterns differ by architecture in logical inference. Third, selective compression achieves Pareto-optimal points inaccessible to uniform methods: R1-Qwen-7B Top-10% on ProofWriter gains +12 pp over FP16 at -9.7% energy, validated on held-out data across five
reasoning benchmarks.
Read the full abstract ▾
Artificial Intelligence
SCAFFOLD: Self-Improving Web Agents via Recursive Parametric Skill Abstraction
Bowei He, Xiaokun Zhang, Meng Ding et al. (+1 more)
Web
agents need to navigate visually rich, long-horizon interfaces that change across sites, yet most previous
agents still learn each task in isolation and discard the procedural knowledge they accumulate. Recent skill-augmented frameworks take an important first step, but they treat the skill library as a flat or two-tier
prompt-side cache and offer no principled mechanism for compressing redundancy or composing skills recursively. We introduce Scaffold, a self-improving framework for visual web
agents that (i) induces parametric, executable skills from successful trajectories under a multi-instance abstraction constraint, (ii) maintains a recursively composed hierarchy in which higher-level skills invoke lower-level ones, (iii) compacts the library via a minimum-description-length (MDL) criterion and behavioral equivalence checking, and (iv) periodically distills skill-augmented trajectories back into model weights to internalize the abstractions. Across WebArena, VisualWebArena, and a held-out split of Online-Mind2Web, Scaffold improves success rate by $11.1$–$17.2$ absolute points over the strongest skill-augmented baseline and shows monotonic gains across five self-improvement iterations without library collapse. We release the code and documents in the Github \href{https://github.com/BokwaiHo/SCAFFOLD}{repository}.
Read the full abstract ▾
All authors Bowei He, Xiaokun Zhang, Meng Ding, Xue Liu
Artificial Intelligence · Cross-listed from cs.CL
Judging LLM-as-a-Judge: Concerning Rubric Artifacts in LLM-based Automated Text Generation Evaluation
Anshul Bagaria, Sowmya S Sundaram, Gokul S Krishnan et al. (+1 more)
LLM-as-a-Judge pipelines are increasingly used to evaluate AI-generated text, based on the assumption that judgments arise from
reasoning over candidate responses with respect to a rubric. We show that this assumption warrants further scrutiny. Classifiers trained only on rubric text, without access to any evaluated response, achieve nontrivial predictive performance on judge outputs. This suggests that rubric formulations encode recoverable evaluative signals, allowing scores to be partially anticipated independently of model outputs. Finally, counterfactual perturbations reveal that judges often fail to reliably update their decisions when either the candidate response or the rubric criterion is reversed. Our findings raise concerns about the reliability of rubric-based LLM evaluation and highlight the need for further methodological study of automated evaluation via LLMs.
Read the full abstract ▾
All authors Anshul Bagaria, Sowmya S Sundaram, Gokul S Krishnan, Balaraman Ravindran
Artificial Intelligence · Cross-listed from cs.CL
Listen to the Latents: Self-Correcting Speech Recognition in Large Audio Language Models Through Hidden-State Interactions
Chan-Jan Hsu, Jaeyeon Kim, Chao-Han Huck Yang et al. (+3 more)
Recent automatic
speech recognition (ASR) systems increasingly integrate large
language models (LLMs) to leverage their semantic knowledge, either externally through logit fusion or internally through warm initialization. However, how to effectively combine these two strategies remains underexplored. In this work, we refine warm-initialized LLM-based ASR models by leveraging their own pre-adaptation base LLMs, focusing on LoRA-adapted settings where the base LLM is preserved. To achieve this, we propose Hybrid Search, a targeted correction strategy motivated by two observations. First, interaction features that characterize the relationship between LLM-based ASR hidden states and base-LLM hidden states provide informative signals about a
token's degree of semantic dependence. Second, selectively refining targeted
tokens with high semantic dependence improves ASR performance far beyond naive global LLM-correction methods including rescoring and late fusion. Our analysis suggests that, even after semantic knowledge transfer through warm initialization, LLM-based ASR models can still leverage their base LLM to further improve inference-time performance.
Read the full abstract ▾
All authors Chan-Jan Hsu, Jaeyeon Kim, Chao-Han Huck Yang, Shinji Watanabe, Hung-yi Lee, Carlos Busso
Artificial Intelligence · Cross-listed from cs.RO
Not All Agreement Counts as Corroboration: Provenance-Conserving Multi-View Fusion for Typed Action Admission in Human-Robot Collaboration
Zekai Jin, Hanrong Zhang, Yihong Tang et al. (+3 more)
Better probability scores do not establish that evidence has been counted correctly. Repeated inference over one observation can improve predictions without adding an evidential origin. Source-local numerical attributes alone cannot in general distinguish repeated derivations from separately countable acquisitions. PACT (Provenance-Aware evidence Conservation and Typed action admission) separates evidence magnitude from countability through a supplied provenance partition. Under singleton fidelity and insertion non-amplification, the coordinatewise meet is the unique pointwise greatest admissible within-component rule. Component budgets add under stated commensurability and separate-component additivity assumptions. Matched reassignments hold numerical outputs fixed while varying the counting relation. In four of 12 replicated-source tests on HandWritten, false refinement lowers macro-averaged negative log-likelihood and Brier score while increasing normalized common-support area under the risk-coverage curve (ncsAURC). In the controlled handover
benchmark, removing the constructed
adversarial-consensus condition leaves a 0.056 reduction in ncsAURC for provenance-partition aggregation relative to singleton aggregation under the same score functional. The corroboration contrast disappears, and method ranking remains selection-score dependent. In offline, reference-based human-
robot collaboration with four
prompts per camera and all other admission inputs fixed, duplicating each
prompt output within its camera from multiplicity one to eight leaves all 720 PACT typed responses per checkpoint unchanged. Probability quality and evidence countability require separate evaluation.
Read the full abstract ▾
All authors Zekai Jin, Hanrong Zhang, Yihong Tang, Fei Hu, Zhen Dong, Yi Shao
Artificial Intelligence · Cross-listed from physics.soc-ph
How Fast Do Agents Rot? An Empirical Study of Long-Horizon Degradation in LLM Agents for Production Decision-Making
Shubhra Mittal
Production deployments of large
language model (LLM)
agents remain unreliable on long, multi-step workflows even as
benchmark success rates climb steadily. We argue this gap is largely an artifact of task horizon:
benchmarks are dominated by short-to-medium horizons where success remains high, while production workloads demand an order of magnitude more dependent steps. We measure the effect directly, characterizing the shape of
agent degradation and disentangling its cause across a large controlled study spanning nine models, six open models from 1.2B to 671B parameters, and three deployed proprietary systems; four task families, including a genuinely agentic tool-use loop; five horizons; and three context regimes. Task success follows a
geometric law governed by a single per-step reliability parameter, which rises with model scale but saturates well below 1 even for the strongest models, guaranteeing eventual collapse at sufficiently long horizons. The effect is sharpest on the agentic task, where every model tested, including widely deployed systems, falls from near-perfect success to near zero within sixteen steps of (n=10,664 analyzed trajectories. Degradation is driven by step count rather than context length: bounding the context window steepens decay rather than easing it (logit slope -0.69 vs. -0.44), p=3x10-6), contradicting a lost-in-the-middle explanation and warning against a common production shortcut. Projecting measured reliability onto representative
benchmark horizons quantifies a substantial gap between
benchmark and production conditions, from 0.42 at
GAIA-length horizons to 0.24 at hundred-step production horizons. For teams responsible for
agent orchestration and reliability at scale, these results argue for horizon-aware evaluation and reliability budgeting in place of aggregate pass-rate metrics. Code,
prompts, seeds, and raw trajectories are released.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CL
PRO-Step: Step-level Process Reward Optimization for Retrieval-Augmented Generation
MinKeon Kim, Namjun Lee and Jaekwang Kim
Retrieval-Augmented Generation enhances Large
Language Models by grounding responses in external knowledge, but multi-hop
reasoning remains vulnerable to error propagation, where early
retrieval failures confound subsequent steps. Standard outcome-based optimization only rewards the final answer, leaving intermediate
retrieval and
reasoning errors undetected. While existing process-based methods introduce step-level signals, they still score each step against the final answer, rewarding spurious successes where flawed
retrieval coincidentally produces the correct answer. Step-level supervision in RAG requires evaluating both logical validity and evidential grounding at each step. We introduce PRO-STEP: we train a generative PRM that evaluates both dimensions, employ PRM-guided value tree search to construct preference pairs contrasting valid steps against flawed ones, and optimize the policy via step-level Direct Preference Optimization. Experiments on single and multi-hop QA
datasets demonstrate that PRO-STEP achieves the best average EM and F1 across five
benchmarks. Code, models, and training data are publicly available at https://github.com/keemminnke/PRO-Step.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.IR
NeoMME: A Single-Tower Multimodal-Native Multilingual Foundation Encoder for Efficient Fine-Tuning and Inference
Aurélien Lac and Tony Wu
Multimodal models often build on architectures designed for generative
vision-
language modeling, typically combining separately pretrained
vision encoders with causal
language models. Visual document retrievers such as ColPali repurpose these models as encoders, carrying over the parameter and compute overhead of a VLM for a non-generative task. We introduce NeoMME, a family of 260M and 800M-parameter
Multimodal and
Multilingual bidirectional Encoders that process
multilingual text and raw image patches in a single bidirectional
Transformer encoder. Both models are pretrained from scratch with a masked discrete-
diffusion text objective, conditioned on visible image patches for
multimodal examples. Both support a 16,384-
token context, enough to encode up to two standard 4K UHD images. To demonstrate its downstream capabilities, we fine-tune NeoMME with jointly trained dense and late-interaction heads. On the ViDoRe v3
benchmark, the resulting NeoMME-Retriever 260M outperforms all evaluated models strictly below 800M parameters with 0.523 nDCG@10, while NeoMME-Retriever 800M reaches 0.556. At a matched 2048x2048 image input size on an NVIDIA L40S, NeoMME-260M encodes pages with about 2x the throughput of ColModernVBERT. Hierarchical
token pooling and asymmetric quantization compress late-interaction
multimodal document
embeddings by 255x while preserving over 95% of baseline nDCG@10. We contribute NeoMME to Hugging Face
Transformers and release the pretrained backbone and
retrieval-compatible checkpoints under Apache 2.0 at https://hf.co/collections/Hcompany/neomme.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.IR
From Feature Interaction to Feature Transport - A Unified Block for Scalable Recommendation Models
Zichen Luo, Jiachen Guo, Keming Gu et al. (+1 more)
Unified recommendation models aim to jointly model non-sequential multi-field features and sequential user behaviors, but existing interaction-centric designs mainly focus on mixing heterogeneous
tokens within each layer. We argue that scalable unified recommendation also requires controlling how intent information is carried, filtered, and preserved across stacked blocks. Inspired by flow-based representation dynamics, we introduce feature transport, a view that treats deep unified recommendation as a discrete context-conditioned representation
evolution process. We propose CRAFT, a Contextual Residual Adaptive Feature Transport block, which summarizes non-sequential features into a reliability-aware contextual field and uses it to generate residual displacement and
memory-preserving signals for intent and sequence representations. In this way, non-sequential context acts as an active controller of representation
evolution rather than a passive object of interaction. In the TAAC2026 advertising recommendation competition, CRAFT achieves a test AUC of 0.838090, surpassing the previous leaderboard-best score of 0.83798. Scaling experiments further show that CRAFT benefits from both depth and width expansion: stacking CRAFT to six blocks improves test AUC to 0.838148, while increasing the hidden dimension reaches 0.838106. These results demonstrate the effectiveness, scalability, and generalization potential of the feature transport paradigm. Source code: https://github.com/AshleyLuo001/CRAFT
Read the full abstract ▾
All authors Zichen Luo, Jiachen Guo, Keming Gu, Jie Zhang
Artificial Intelligence
Validity-Aware Jailbreak Evaluation for Large Language Models
Qilong Wu, Sahil Wadhwa, Pranab Mohanty et al. (+2 more)
Jailbreak robustness has become central to large
language model (LLM) safety evaluation, yet prevailing methodologies rely primarily on refusal behavior, semantic resemblance, and intent-matching heuristics that emphasize linguistic plausibility rather than correctness. We identify a key limitation in existing evaluations: many jailbreak intents depend on instructional validity rather than epistemic factuality, allowing realistic-looking responses to be labeled successful despite being factually or procedurally incorrect. To address this gap, we propose Sequential Epistemic and Action-Level Validation (SEAV), a verification-centric jailbreak evaluation framework that decomposes responses into ordered steps and evaluates both validity and correctness. SEAV combines LLM-as-a-judge mechanisms for semantic interpretation with
retrieval-grounded verification using external knowledge sources, assessing whether generated content is factually correct, structurally consistent, and operationally capable of advancing harmful objectives. Empirically, SEAV cuts the false-positive rate on SD-A (a curated strategic-dishonesty diagnostic) by 14.9\,pp vs. the strongest baseline, and reclassifies 22.1%–51.0% of sampled prior-labeled successes as invalid across three of four public
benchmarks. Together, these results show that enforcing correctness substantially reshapes measured robustness: many previously labeled jailbreak successes are reclassified as invalid, and results are stable across the tested search backends and evaluator models. Code and data are available at https://github.com/Ardor-Wu/SEAV.
Read the full abstract ▾
All authors Qilong Wu, Sahil Wadhwa, Pranab Mohanty, Giri Iyengar, Varun Chandrasekaran
Artificial Intelligence · Cross-listed from cs.CL
Guoli Wang, Haonan Shi, Tu Ouyang et al. (+1 more)
Diffusion large
language models (dLLMs) generate text through iterative denoising rather than left-to-right decoding. This generation paradigm introduces two axes that can influence safety
alignment: when
tokens are generated during denoising and where they appear in the response. In this paper, we measure dLLM safety behavior under harmful
prompts by tracing intermediate
token distributions and commitment decisions throughout denoising. Our analysis shows that refusal signals are concentrated in early denoising steps and leading response positions, and the
tokens committed early can strongly shape the final safety outcome. Our measurements further show that the denoising step and persistence of refusal-
token commitment are important for understanding dLLM safety. Based on these findings, we propose Refusal-Aware Early Commitment (RAEC), a simple training-free decoding method that commits persistent refusal signals from early steps. Experiments on LLaDA and Dream show that RAEC reduces attack success rates while largely preserving utility. The code is available at https://github.com/Glresearch1/RAEC.
Read the full abstract ▾
All authors Guoli Wang, Haonan Shi, Tu Ouyang, An Wang
Artificial Intelligence
Krithika Ramesh, Krishna Pillutla, Danish Pruthi et al. (+1 more)
Both
privacy and factual accuracy are paramount in high-stakes domains like healthcare. Concerningly, we uncover and investigate a
privacy-
hallucination tradeoff in differentially private (DP)
language models. First, we empirically show that models pre-trained or fine-tuned with DP tend to produce more
hallucinations than non-DP counterparts, with increased severity as the
privacy budget grows stricter. Second, we investigate model properties driving this tradeoff, demonstrating that DP mechanisms flatten output distributions, potentially redistributing probability mass toward factually incorrect alternatives. Third, through experiments where we control fact frequency in training data, we characterize how information frequency can reduce
hallucination risks in DP models. Overall, our findings underscore the need for more nuanced
privacy-preserving interventions that offer rigorous
privacy guarantees without compromising factual accuracy.
Read the full abstract ▾
All authors Krithika Ramesh, Krishna Pillutla, Danish Pruthi, Anjalie Field
Artificial Intelligence · Cross-listed from cs.CL
EvoFlint: An Evolutionary Atlas of Multi-Turn LLM Vulnerabilities
Feitong Qiao, Liren Peng, Shiming Ren et al. (+7 more)
Frontier
language models that refuse harmful single-turn
prompts often comply when the same intent is reached gradually over many turns, making multi-turn attacks one of the least understood failure modes of large
language models. Most automated red-teaming methods treat this as a generation problem: produce attacks that break the model. We argue it is better framed as a search problem: discover, organize, and iteratively refine a diverse archive of attack strategies, producing a structured map of how a target model fails rather than a list of one-off successes. We introduce EvoFlint, which applies evolutionary quality-diversity search to multi-turn red-teaming. Attack strategies are phased conversation plans, not raw
prompts, and are evolved through LLM-driven mutation and crossover. A Pareto fitness over attack success rate and peak severity preserves selection signal from near-miss attacks. A risk-indexed archive runs novelty search with local competition over strategy description
embeddings inside each
cell, maintaining diversity without committing to a predefined style taxonomy. A generation-level
memory accumulates target-model insights across the population and feeds them back into strategy generation. On the HarmBench-test split, EvoFlint reaches attack success rates of 35.8% on Claude Sonnet 4.6, 59.7% on GPT-5.4, and 94.3% on Qwen3-32B, alongside 98.7% on the older GPT-4o included as a baseline reference. The resulting archive, organized by risk category, exposes for each target which categories of harm its safety training has and has not covered.
Read the full abstract ▾
All authors Feitong Qiao, Liren Peng, Shiming Ren, Aishwarya Jadhav, Arghavan Bahadorinejad, Marinette Chen, Muhan Zhang, Abdulaziz Suria, Gennevi Lu, Anish Das Sarma
Artificial Intelligence
EGT-KG: Evidence-Grounded Typed KG Retrieval for Practical Scientific QA with Small Language Models
Muran Yu, Jiechao Gao, Yuandong Pan et al. (+5 more)
For emerging scientific research domains, local Small
Language Models (SLMs) are becoming more attractive, as they offer stronger
privacy control and more stable deployment pipelines than Large
Language Models. However, in practice, scientific question-answering on SLMs often operates under inevitable constraints: small literature collections, fragmented evidence, limited context window and
reasoning abilities. We propose the Evidence-Grounded Typed Knowledge Graph (EGT-KG), a
retrieval framework to improve information
retrieval with local SLMs. We assessed three question-answering settings: a vanilla
Retrieval-Augmented Generation (RAG) workflow and two EGT-KG workflows: an automatically generated relation schema (AS) and an expert-defined relation schema (ES). Our experiments were evaluated with a six-dimensional evaluation framework (S3CRF: Soundness, Correctness, Completeness, Conciseness, Relevance, Fluency) on a Biopolymer-bound Soil Composite literature
benchmark, showing that EGT-KG outperforms the vanilla RAG method in most settings, with the best improvement from llama3:8b: a Final Score of 70.37 (+14.67%) and 68.82 (+12.14%) by AS/ES EGT-KG variants.
Read the full abstract ▾
All authors Muran Yu, Jiechao Gao, Yuandong Pan, Barney H. Miao, Andrew C. Lesh, Kincho H. Law, Jie Wang, Michael D. Lepech
Artificial Intelligence · Cross-listed from cs.CL
Exploring Collaboration between a language and a non-language agent
Harini S I, Somesh Singh, Yaman K Singla et al. (+3 more)
LLMs are increasingly deployed as orchestrators that coordinate specialized subagents to solve complex tasks through natural
language. However, in many important domains like game playing and robotics, the strongest available
agents are not
language models. Integrating non-
language agents with LLMs would require verbalization: compressing their rich continuous representations into sparse textual summaries at each interaction step. To study whether verbalization constitutes a bottleneck, we introduce LLAMIA-Bench, a suite of six diverse collaborative chess tasks spanning three facets: behavioral imitation, state assessment, and natural-
language explanation. Each task instantiates a well-established chess problem that neither the LLM nor the chess engine can solve alone. To solve LLM collaboration with non-
language agents, we introduce latent state internalization, which projects the subagent's continuous representations directly into the LLM's
token stream as learned state
tokens, with dynamic re-encoding as actions advance the environment state. Comparing internalization to verbalized integration, our experiments reveal a consistent verbalization debt: the performance gap widens throughout training and persists as the LLM scales from 4B to 14B parameters. A single 14B model, LLAMIA, trained with latent state internalization, matches or exceeds task specialists and frontier models including GPT-5.1 with tool access across all
benchmark tasks, and generalizes out-of-distribution where task-specific finetunes collapse
Read the full abstract ▾
All authors Harini S I, Somesh Singh, Yaman K Singla, Rajiv Ratn Shah, David Doermann, Balaji Krishnamurthy
Artificial Intelligence · Cross-listed from cs.LG
Higher Structures in Deep Learning
Michael L. Roberts, Carlos Zapata Carratalá. Nicholas J. Cooper, Lijun Chen et al. (+2 more)
We provide an expository introduction on the importance of higher-arity tensor operations to deep learning. Then, we conduct a novel empirical investigation of higher-arity phenomenon in trained neural networks, introduce a hypergraphical generalization of the multilayer perceptron, and explore connections to evolutionary
algorithms. We conclude with a discussion of promising directions for future research.
Read the full abstract ▾
All authors Michael L. Roberts, Carlos Zapata Carratalá. Nicholas J. Cooper, Lijun Chen, François G. Meyer, Danna Gurari
Artificial Intelligence · Cross-listed from math.OC
Operational Regimes in Non-Convex Optimization: A Multiplier-Based Taxonomy
Seyed Mohsen Kazemi, Ali Movaghar and Shaahin hessabi
This paper introduces a structural taxonomy for constrained non-convex optimization based on the signature of Lagrange multipliers at KKT stationary points. Leveraging a unified game-theoretic interpretation of eight classical
algorithm families–including block coordinate descent, ADMM, generalized Benders decomposition, successive convex approximation, interior-point methods, mirror descent, Frank-Wolfe, and Riemannian gradient descent–we show that the normalized multiplier vector carries an
algorithm-independent structural fingerprint. Four scale-free shape features of this vector partition the dual space into five operational regimes: Unconstrained, Resource-Limited, Saturation, Strongly-Coupled, and Hybrid. We establish four structural
theorems characterizing the partition: invariance under natural KKT
symmetries, local stability under data perturbation with explicit Lipschitz margins from Robinson's strong regularity, codimension-one regime transitions, and the topological identification of the Hybrid regime as the Lebesgue-null boundary of the core regimes. A linear-time classifier is proposed with provable guarantees on correctness, iteration stabilization, sample complexity, and online tracking under data drift. Numerical experiments on 104 mixed-integer nonlinear programs and a downlink beamforming instance validate the theoretical predictions. The framework provides a foundational tool for regime-aware
algorithm design and robustness analysis in non-convex optimization.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CR
Does Reasoning Mitigate Backdoor Attacks? A Neuro-Symbolic Perspective
Marco Antonio Corallo, Andrea Agiollo, Mauro Conti et al. (+1 more)
Neuro-Symbolic (NeSy) AI has recently emerged as a novel paradigm to enable trustworthy AI, aiming at integrating sub-symbolic neural perception with grounded symbolic
reasoning. The neuro-symbolic integration process that characterizes these models has been proven beneficial to achieve more transparent, explainable and efficient AI systems. Meanwhile, their properties under
adversarial settings have been overlooked being frequently deemed robust-by-design. However, the neural-symbolic integration process they leverage constitutes an additional layer of complexity that may provide an attack entry-point. Therefore, in this paper, we claim that an in-depth investigation of the
adversarial robustness of NeSy models is necessary and provide the first systematic evaluation of
backdoor attacks against NeSy. To this end, we compare the most popular NeSy framework, namely DeepProbLog, against baseline neural networks across a total of eight
backdoor settings and four
reasoning tasks. Our experimental results show that while NeSy models are indeed more robust than their neural counterpart on average, their robustness vastly depend on the strictness of the
reasoning process being enforced and its compatibility with the chosen
adversarial target. The source code to reproduce our experiments is made available at https://github.com/marcoantoniocorallo/NeSy-
Backdoor.
Read the full abstract ▾
All authors Marco Antonio Corallo, Andrea Agiollo, Mauro Conti, Alberto Giaretta
Artificial Intelligence
Towards a Belief-Based World Model for LLM Agents
Shubham Kumar, Harshit Kumar, Narendra Ahuja et al. (+1 more)
Large
language models (LLMs) are being used as policies for autonomous decision-making and planning in many domains. Despite their strong
reasoning capabilities, LLMs struggle with long-horizon tasks, especially under partial observability. World models are a promising way to enhance policy performance, both during training and inference. During inference,
agents currently use world models to simulate the consequences of candidate actions before choosing an action, which can improve decision-making. However, we argue that simulation alone is an incomplete interface for decision-making under partial observability: simulation does not adequately capture uncertainty about the current state, which
agents may need for accurate decision-making. We address this limitation with Belief-Based World Models (BB-WMs), which maintain a belief that LLMs can query to access information on what is known and uncertain about the current state. Before developing methods to learn accurate BB-WMs, this paper focuses on a more fundamental question: does exposing a world model's belief directly to an LLM policy improve decision-making? Our results show that giving LLM
agents access to beliefs improves task performance under partial observability, while remaining complementary to existing simulation-based world models. Code: https://github.com/skumar-ml/belief-world-models.
Read the full abstract ▾
All authors Shubham Kumar, Harshit Kumar, Narendra Ahuja, Saurabh Jha
Artificial Intelligence
mimeo: Compiling Public Expert Corpora into Agent Skills and Testing What Transfers
Timothy Kassis
Giving an
agent a file about a named expert can supply hard-to-find material, produce a recognizable persona, or change what the
agent decides. These are different claims. We test each one. mimeo is an open-source tool that finds a person's public work, checks each extracted quotation against the cached source text, and writes a file an
agent can load. Eight logged builds averaged 38 model calls; the check rejects 13.2% of extracted quotations. We tested four expert files with one coding-
agent harness. Knowledge access was clearest: mimeo answered all 20 obscure, quotation-heavy questions; no closed-book condition answered more than 10. Keyword search (BM25) over the same pages answered 15-17, a gap this sample cannot resolve. Grounding showed one clear benefit: personas written from model
memory misstated a documented position on 1-4 of 20 answers under every grader; the plain
agent and mimeo never did. Every persona was easy to spot on short open
prompts, and adding task material lowered identification by 18-23 points. mimeo was no more identifiable than a from-
memory profile. Judgment transfer remained unresolved because both tests hit their ceiling: every condition found 94-97% of the problems planted in engineering tasks and scored 94-100% on 16 new application scenarios. An AI-judged "sounds like the expert" score changed with the judge: two of four preferred answers based on a model's stereotype, while two found no difference on the same text. That is a caution against relying on a single AI judge. The evidence supports mimeo as a compact, inspectable reference on a person, not as a demonstrated transfer of their judgment. Toolkit and expert profiles: https://github.com/K-Dense-AI/mimeo
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
HBQ: Hierarchical Scaling Block Quantization with Hardware-Efficiency-Aware Design for Accurate LLM Inference
Chun-Ting Chen, Dongmin Han, Hangyeol Mun et al. (+6 more)
Block Quantization (BQ) enables efficient LLM inference by quantizing both weights and activations, but its design space remains underexplored. Through hardware-accuracy design space exploration, we identify block size as a key trade-off: larger blocks improve hardware efficiency by amortizing dequantization and accumulation costs, but degrade accuracy. Motivated by this insight, we propose Hierarchical Block Quantization (HBQ), which combines large blocks with low-overhead significand (SIG) scaling for second-level quantization. SIG scaling effectively compensates for large-block quantization errors while accounting for distinct weight and activation distributions. HBQ-A achieves W4A16-level accuracy with W4A5 and lower area than NVFP4, while HBQ-E further reduces hardware cost by 17% while outperforming existing BQ methods in accuracy. We implement HBQ for weights, activations, and KV cache in a 28nm ASIC accelerator and introduce partial-sum BQ to reduce EMA energy. At comparable accuracy, HBQ achieves 2.3x/4.6x higher area/energy efficiency than state-of-the-art weight-only quantization and 1.6-3.3x lower system energy with 1.5-3x speedup over prior BQ methods. Our implementation is publicly available at: https://github.com/SeoLabCornell/HBQ.git.
Read the full abstract ▾
All authors Chun-Ting Chen, Dongmin Han, Hangyeol Mun, Jake Hyun, Arnab Raha, Amit Agarwal, Mark Anders, Mohamed Abdelfattah, Jae-sun Seo