Computation and Language · Cross-listed from cs.AI
Agentic Nesting: A New Methodology for Existing Enterprise Application Integration and Services
Xi Wang, Kun Li, Xianyao Ling et al. (+8 more)
Enterprise operations extensively rely on multiple heterogeneous business systems and information applications, which also result in severe data silos and process fragmentation. Enterprises have invested considerable financial and material resources in building these applications, however, effectively leveraging and orchestrating them remains a formidable challenge. Conventional approaches to enterprise application integration, encompassing middleware architectures such as Enterprise Service Bus (ESB), API gateway infrastructures, and Robotic Process Automation (RPA), suffer from inherent limitations like high architectural coupling, escalating operation and maintenance costs, and limited intelligence capabilities. This paper proposes Agentic Nesting, a multi-
agent collaboration framework in which existing enterprise applications are encapsulated as autonomous AI
agents within a hierarchically nested structure. Rather than flat interconnection,
agents are organized into layered stewardship
topologies that mirror the compositional complexity of enterprise ecosystems. The framework extracts a digital
agent proxy from each legacy application to enable natural-
language interaction and autonomous manipulation, coordinates multiple
agents through a central orchestrator for task decomposition and dynamic dispatching, and exposes a unified conversational interface for cross-application querying and process orchestration. The main contributions of this paper are the proposition of the "Application-as-
Agent" integration paradigm and the "Conversation-as-Integration" interaction philosophy, together with an exploration of the generalization potential of this methodology in scenarios encompassing heterogeneous system coordination, and large-scale data applications.
Read the full abstract ▾
All authors Xi Wang, Kun Li, Xianyao Ling, Gang Yin, Liang Zhang, Jiang Wu, Wenbo Lei, Jun Xu, Annie Wang, Fu Zhang, Weizhe Wang
Computation and Language
Preferred, Not Safer: Pairwise Preference Is a Poor Proxy for Clinical Safety
Fay Elhassan, David Sasu, Alexandra Kulinkina et al. (+2 more)
We evaluate whether clinician pairwise preferences provide a reliable signal of clinical safety in large
language model (LLM) evaluation using expert feedback from MOOVE (Massive Open Online Validation and Evaluation), a clinician-led platform collecting blinded pairwise preferences alongside multi-criterion rubric ratings. Clinicians assign scores on a discrete $[-2, +2]$ scale, where negative values indicate clinically unsafe or misleading content. Using 26{,}804 pairwise judgments across outputs from 13 LLMs, contributed by more than 736 clinicians across 28+ countries, we find that clinician preference is a poor proxy for safety-critical performance. Models ranking highly under pairwise preference can still exhibit substantial rates of clinically meaningful failures ($\leq -1$) on dimensions such as Harmlessness and Accuracy. These failures are unevenly distributed across specialties, creating domain-specific “no-go zones” not visible in aggregate rankings or single-number leaderboards. We further analyze contributing factors including
prompt length, refusal and escalation behavior, and the relative contributions of safety-critical versus surface-level features. A substantial fraction of preference votes carry no positive safety signal, while feature decomposition shows that surface-level characteristics explain slightly more preference variation than safety-critical rubric differences. Finally, we introduce a clinically adjusted preference ranking combining pairwise preference with rubric-derived feedback, producing a more safety-aware ordering than raw Bradley–Terry strength alone. Our findings support evaluation practices that separate preference from safety, report safety-critical failure rates directly, and incorporate clinically grounded adjustments when ranking LLMs for clinical decision making.
Read the full abstract ▾
All authors Fay Elhassan, David Sasu, Alexandra Kulinkina, Lars Henning Klein, Mary-Anne Hartley
Computation and Language · Cross-listed from cs.HC
Language as a Material Interface for Creative LLM Interaction
Jon McCormack, Tace McNamara, Chen Wang et al. (+1 more)
Although directive prompting is the predominant way to interact with Large
Language Models (LLMs), many creative practices rely on
language that is open-ended, associative, phonaesthetic, symbolic, and that unfolds across multiple temporalities. In this work, we explore how creative practitioners might work with AI systems when
language is treated not merely as instruction but as material. We conducted an ecological two-week study with four creative practitioners using a design probe: the Memetic Mixer, a tangible interactive device that constrains interaction with an LLM. Analysis of post-study interviews and device logs identified distinct modes of material
language use and temporalities that shaped each participant's engagement with AI and their creative practice. We reflect on these findings and contribute design considerations that support open-ended interaction with AI in creative practice.
Read the full abstract ▾
All authors Jon McCormack, Tace McNamara, Chen Wang, Maria Teresa Llano
Computation and Language · Cross-listed from cs.AI
The Hard Decision Layer: Evidence for Committed Inference in Transformers
Ashwath Vaithinathan Aravindan and Mayank Kejriwal
We investigate where and how
transformer-based
language models commit to predictions in multiple-choice question answering. We identify the _Hard Decision Layer_ (HDL), a natural architectural property where answer option rankings stabilize abruptly during inference. Empirical validation across four
language models (Qwen, Llama, Granite, Mistral) and four
benchmark datasets demonstrates consistent HDL emergence without learned routing policies. We also show that the HDL is invariant to fine-tuning. Our results reveal striking accuracy improvements at the HDL: up to +0.61 (Qwen on CommonsenseQA), after which performance stabilizes. Systematic ablations on label formats and problem complexity confirm the phenomenon is fundamental to model architecture. These findings offer mechanistic insights into
transformer inference and suggest opportunities for efficient
reasoning and model steering. All code and results required to reproduce this work are available in https://github.com/Mystic-Slice/hard-decision-layer
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
Benchmarking Large Language Models on Multi-Sensor Physical Hazard Assessment
Faizan Iqbal
We present an empirical
benchmark evaluating how five large
language models assess multisensor physical hazard data. Testing 60 scenarios across three categories - multi-sensor joint assessment, response proportionality, and pattern disambiguation - with 1,800 API calls at temperature 0.0, we find that all tested models consistently produced no precautionary warning signal across the tested scenarios where multiple sensors are simultaneously elevated below their individual safety limits, while achieving near-perfect accuracy on single-sensor threshold violations. All five models (ChatGPT-4o, Gemini 2.5 Flash, DeepSeek, Kimi, Llama 3.1 8B) score near zero on Category A multi-sensor scenarios (Q2: 0.000-0.208; Q3: 0.000-0.592) compared to strong performance on single-sensor scenarios (Category B Q1: 0.975-1.000). Structured tabular formatting shows no consistent advantage over plain prose; ChatGPT-4o performs significantly better under prose (p = 0.001). These findings have direct implications for practitioners deploying the tested models in physical safety monitoring systems.
Read the full abstract ▾
Computation and Language
Function-Vector Heads Are Two Populations: Writers and Cancellers in In-Context Learning
Han-yu Wang
In-context learning lets a
language model perform a task specified by examples in its
prompt. Function vectors capture task information in a compact activation assembled from
attention-head outputs. Across two rule families and three Pythia models, we find two opposed functional populations among candidate function-vector heads. Writers support the rule-correct answer, while cancellers systematically suppress it. These roles predict opposing group effects on held-out
prompts, where removing cancellers improves accuracy by 2.4 to 6.8 percentage points in all six settings. The dominant suppressive contributions depend on task content, and the same components can support correct predictions in another task. At the population level, substantial support and suppression can nearly cancel. Separating these roles explains how task-related computation can work against correct predictions and how a small aggregate effect can conceal strong opposing contributions.
Read the full abstract ▾
Computation and Language
Phantom Transitions in Language Model Fine-Tuning: A Density-Matrix Analysis
Vaibhav Prakash and Jayasri Dontabhaktuni
Language models fine-tuned where the correct completion must outrank a near-synonym competitor often fail silently. The cross-
entropy loss falls monotonically while the correct
token never overtakes the competitor in the model's ranking. We study this across five
transformer architectures from two families spanning a sixfold parameter range, on ten contexts whose correct and competing completions share substantial
embedding overlap. We build an order parameter combining the predicted distribution with
embedding overlap, as a density
matrix because that distribution lives over a non-orthogonal basis. It decomposes additively into a signal term tracking commitment to the correct
token and a drag term set by how the
embedding bulk leaks probability into the score. This isolates two failure modes. In kinematic failure the signal stays too small and the model never commits. In structural failure the drag worsens during fine-tuning, so the model degrades geometrically as its loss falls. The order parameter also shows sharp jumps resembling phase transitions. We test the spontaneous-
symmetry-breaking reading by tracking it after every gradient step, and rule it out. The jumps persist under LoRA even though the
token embedding matrix never changes. No
geometric phase transition is possible when that geometry cannot move, so the discontinuity lies entirely in the softmax readout. A few dimensionless quantities organize the trajectory across architectures. One is consistent across all five models under full fine-tuning. A second sorts architectures into two classes by their bulk
embedding distribution and predicts whether LoRA alone can make a sentence commit. As a blind test, the framework predicts a held-out architecture's critical learning rate to within 2.1% of a later sweep. These results characterize this near-synonym mechanism and need recalibration before extrapolation.
Read the full abstract ▾
Computation and Language
Persistent Priors, Preserved Targets: A Stroop-Style Paradigm for Lexical Override
Han-yu Wang
Local definitions can assign a familiar word a temporary meaning while its usual associations remain useful elsewhere. We measure interference from those associations with a matched Stroop-style paradigm. A conflict
prompt defines doctor as forest and compares forest with the familiar associate hospital. A neutral control replaces doctor with a semantically weak word in both the definition and query while keeping forest and hospital fixed. All 11 model-level means are positive. Aggregate means are also positive for all four conflict families and
prompt formats. When no redefinition is present, a stronger preference for the familiar distractor predicts more interference in arbitrary-semantic, polysemy/entity, and domain-definition remappings, while the antonym slope is null. Separately, we patch neutral-control activations into antonym
prompts in five 1B-2B models. Patching the defined word, the target word in the definition, and the later query word together restores almost all of the target-minus-distractor margin lost in conflict (normalized recovery R in [0.92,1.06]). Replacing only that target-word activation with a donor from another item reduces recovery in every tested case. Those donor patches also lower the distractor logit, while the contextual target falls much more than under the same-item patch that restores the margin.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CV
Kailing Li, Tianwen Qian, Lijin Yang et al. (+4 more)
Vision-
Language Navigation (VLN) enables
embodied agents to reach target locations in unseen environments by following
language instructions. Despite recent progress with
vision-
language models (VLMs), a critical semantic-
geometric gap remains: while VLMs excel at
language and 2D visual understanding, they struggle with 3D spatial
reasoning and fail to capture the causal dynamics between actions and spatial transitions, resulting in unreliable
navigation, particularly in zero-shot settings. To bridge this gap, we propose a Hierarchical Semantic-
Geometric Map (HSGM) that transforms 3D
geometric information into a structured representation compatible with VLMs, effectively linking them to the physical world. Specifically, HSGM is represented as a multi-channel top-down map organized into three levels: (1)
geometric level that records navigable regions and obstacles, (2) semantic level that represents objects and their relations, and (3) decision level that supports high-level task
reasoning and goal selection. During
navigation, the VLM acts as a high-level semantic planner, interpreting the spatial layout encoded in the HSGM to select geometrically valid waypoints, while low-level, collision-free movements between waypoints are executed by a classical path-planning
algorithm, fully decoupling semantic
reasoning from action execution. Additionally, complex instructions are decomposed into subtasks to alleviate the problem of progress forgetting or hallucinating in long-horizon
navigation. Extensive experiments on R2R-CE and RxR-CE
benchmarks demonstrate that our zero-shot framework achieves state-of-the-art performance and even outperforms several supervised methods. Code is available at https://github.com/Teacher-Tom/HSGM_public.
Read the full abstract ▾
All authors Kailing Li, Tianwen Qian, Lijin Yang, Yuqian Fu, Jingyu Gong, Xiaoling Wang, Liang He
Computation and Language
Agreement Metrics for LLM-as-Judge Evaluation: What to Report and Why
Delip Rao and Chris Callison-Burch
Whether a rubric-based LLM judge can replace human annotation is decided by its measured agreement with human labels. Yet the same verdicts can support wildly varying agreement numbers, depending on seemingly minor choices: the judgment scale, the retained cases, the handling of abstentions and invalid outputs, and the pooling of verdicts across items and rubric criteria. The statistics that settle these choices are established, but in psychometrics, econometrics, and corpus annotation rather than in the evaluation practice that needs them. We treat the choices as a measurement protocol that fixes what the reported number estimates before any metric is computed, assemble the relevant results into a single source-attributed analysis, and apply it to three published LLM-judge evaluations. For non-degenerate binary verdicts, Pearson's $r$, Spearman's $\rho$, Kendall's $\tau_b$, the phi coefficient, and the Matthews correlation coefficient are exactly the same statistic, so reporting several repeats one number under different names. Cohen's $\kappa$ differs from them only through a marginal-mismatch factor in $(0,1]$ and shares their asymptotic variance when judge and human assign the positive verdict equally often. Under exclusion, accuracy over all cases is pinned down only to a worst-case interval as wide as the uncovered fraction. On a rubric
benchmark carrying per-criterion human labels, protocol choice alone moves reported accuracy from $0.551$ to $0.899$ and carries $\kappa$ across zero, without altering a single verdict. We distill the analysis into a reporting checklist that makes agreement claims reconstructible and comparable.
Read the full abstract ▾
Computation and Language
From Data to Insights: Exploring Program-of-Thoughts Prompting for Chart Summarization
Yutong Qu and Wei Zhang
Charts play a critical role in conveying numerical data insights through structured visual representations. However, semantic visual understanding and numerical
reasoning requirements hinder the accurate description of charts, interpreting a challenging task in chart summarization. Despite recent advancements in visual
language models (VLMs), approaches lack robust mechanisms for verifying statistical fact correctness and are computationally heavy. To address this gap, this paper explores a strategy of using zero-shot learning to motivate the lightweight VLMs to perform computational
reasoning, via Python programs as intermediaries to derive valid summary statistics for chart understanding. Specifically, we introduce a novel chart-to-dictionary auxiliary task, offering a more flexible representation compared to traditional chart-to-table methods, making it particularly well-suited for integration with the Program-of-Thought (PoT) strategy. Experimental results demonstrate our strategy performs on par with existing chart summarization methods across semantic and factual metrics. Code is available on https://anonymous.4open.science/r/ZeroShot-PoT-C2T-5A6B.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.CV
Generic Interpretation Approach for Transformer Models Incorporating Heterogenous Attention Structures
Yongjin Cui, Xiaohui Fan and Huajun Chen
Transformer has significantly propelled the development of artificial intelligence, and certainly the development of
agents as well. We categorize
attention structures of
Transformer into two types based on the source of the input information: homogenous and heterogenous
attention structures. Heterogenous
attention structures, with co-
attention as a typical example, process information from different sources. Heterogenous
attention structure is the foundation for
Transformer models to achieve more complex functions and integrate more modal information. Whether for research purposes or policy requirements, the interpretation of
Transformer models with heterogenous
attention structures is an important task. The fusion of information from different sources brings new challenges. Our work mainly includes two parts: method and experimentation. In terms of method, we propose an interpretation method for
Transformer models with heterogenous
attention structures. In terms of experimentation, based on our experimental analysis paradigm, we interpret the operating mechanisms of representative models, conduct semantic interpretation and logical interpretation.
Read the full abstract ▾
Computation and Language
MiRD: Reliable Set-Valued Prediction for Open-Ended Question Answering via Miscoverage Risk Decomposition
Anqi Hu, Zhiyuan Wang, Zijun Jia et al. (+1 more)
Reliable set-valued prediction provides a principled way to mitigate
hallucinations in open-ended question answering (QA), yet existing conformal approaches typically rely on a fragile premise: finite sampling must already produce at least one admissible candidate, or calibration examples violating this condition are discarded. In this paper, we introduce MiRD, a two-stage framework that decomposes overall miscoverage into sampling failure and conditional selection failure. In Stage I, MiRD establishes an expectation-level marginal upper bound on the probability that finite sampling produces no admissible answer under a fixed budget. In Stage II, conditioned on sampling success, MiRD calibrates a conformal selection threshold using admission-correlated nonconformity scores defined over the full calibration set, thereby preserving calibration-set integrity. Across three open-ended QA
datasets and eight models, MiRD controls sampling risk, conditional selection risk, and overall miscoverage, while yielding tighter first-stage bounds than PAC-style alternatives and more adaptive prediction sets than successful-only calibration.
Read the full abstract ▾
All authors Anqi Hu, Zhiyuan Wang, Zijun Jia, Bo Fu
Computation and Language · Cross-listed from cs.AI
Advancing Creative Physical Intelligence in Large Multimodal Models
Cheng Qian, Hyeonjeong Ha, Jiayu Liu et al. (+10 more)
Large
multimodal models (LMMs) have rapidly advanced in perception and
reasoning; however, it remains unclear whether these capabilities generalize to discovering visually grounded solutions in open-ended environments, beyond pattern recognition. In such settings, intelligence requires more than answering well-posed questions: it involves identifying how elements in a scene can be repurposed in non-obvious yet physically feasible ways. This form of creative problem-solving is central to human intelligence, but remains largely untested in current
benchmarks. To evaluate this ability, we introduce MM-CreativityBench, a
benchmark for affordance-grounded creative tool use in visually rich, physically constrained environments. Each instance presents a scenario image with structured views of candidate entities and their parts, enabling fine-grained, interactive evaluation of how models iteratively inspect the scene, identify relevant affordances, and compose visually and physically grounded solutions. Our experiments show that current LMMs often fall short, not due to lack of generative capability, but because they do not sustain grounded exploration. Models often overlook relevant entities, under-examine critical parts, or hallucinate attributes not grounded in the image. Motivated by this failure mode, we propose affordance-grounded
alignment, which casts creative tool use as a preference learning problem. Using Direct Preference Optimization, we encourage models to prefer attribute-affordance
reasoning grounded in visual evidence over hallucinated alternatives. In addition, we incorporate supervision derived from an affordance knowledge base to guide broader entity exploration and multi-turn planning. Our results show consistent gains in selecting the correct entities and parts, while substantially reducing
hallucination and grounding-related errors.
Read the full abstract ▾
All authors Cheng Qian, Hyeonjeong Ha, Jiayu Liu, Jeonghwan Kim, Emre Can Acikgoz, Bingxuan Li, Kunlun Zhu, Jiateng Liu, Aditi Tiwari, Zhenhailong Wang, Xiusi Chen, Mahdi Namazifar, Heng Ji
Computation and Language
Memory Architectures for Multi-Turn Text-to-SQL: A Benchmark and Empirical Study
Ravi Kumar Tummalapenta and Suman Addanki
Multi-turn Text-to-SQL is central to enterprise analytics yet remains predominantly evaluated in single-turn settings. We introduce EnterpriseMem-Bench, a multi-turn Text-to-SQL
benchmark of 300 sessions and 1,400 turns built programmatically from three enterprise domains (BIRD financial, SEC EDGAR, Northwind), with deterministic ground truth and per-turn
memory-critical annotation. We evaluate five frontier models – GPT-5 mini, GPT-5.2, Claude Sonnet 4.5, Sonnet 4.6, and Opus 4.6 – across five
memory conditions enabling a three-way ablation isolating working-
memory window size, episodic
retrieval, and semantic augmentation as independent effects. All Claude models are evaluated with extended thinking enabled to maintain parity with GPT
reasoning models. We introduce the
Memory Benefit Score (MBS) as a per-turn diagnostic metric. Four findings emerge: (1) stateless multi-turn Text-to-SQL collapses to zero execution accuracy by Turn 3 across all five models, even under
reasoning; (2)
memory-architecture complexity does not monotonically improve accuracy – working
memory dominates, and additional components produce model- and
dataset-dependent effects from +14 to -16 percentage points; (3) Claude Sonnet 4.6 underperforms Sonnet 4.5 by 17-33pp on SEC EDGAR across conditions, a generational regression persisting under
reasoning; (4) under
reasoning, Claude error distributions become mono-modal – every non-correct turn is a wrong-result error. We release the
benchmark,
agent, and evaluation code.
Read the full abstract ▾
Computation and Language
Cultural Value Alignment Via Latent Activation Steering in Large Language Models
Trung Duc Anh Dang and Sarah Masud
Large
Language Models (LLMs) often exhibit homogenized cultural perspectives. While the World Values Survey (WVS) provides a gold standard for mapping human values, traditional direct prompting of LLMs on WVS often fails to access the model's latent cultural depth, leading to safety-aligned refusals or neutral responses. Here, we propose a generalizable framework for cultural evaluation and intervention that transitions from abstract queries to scenario-based behavioral probing. By extracting implicit
token probabilities across 300 situational dilemmas, we bypass surface-level
alignment to map the latent coordinates of LLMs cultural value. We further introduce activation steering to shift these internal
alignments during the forward pass without retraining. Across multiple LLMs, we find substantial variation in adaptability and uncover a consistent phenomenon of latent
entanglement, where interventions along one cultural dimension induce shifts along another. These results suggest that cultural values are encoded as coupled structures, limiting precise
alignment. This work establishes a computationally efficient framework for cultural steering, highlighting the structural complexities when navigating global value with LLMs.
Read the full abstract ▾
Computation and Language
Why LLMs Hallucinate on Structured Knowledge: A Mechanistic Analysis of Reasoning over Linearized Representations
Shanghao Li, Jinda Han, Yibo Wang et al. (+5 more)
In many
reasoning tasks, large
language models (LLMs) rely on structured external knowledge, such as graphs and tables, which is typically linearized into sequential
token representations. However, even when sufficient knowledge is available, LLMs can still produce hallucinated outputs, and the underlying mechanisms behind such failures remain poorly understood. We investigate these mechanisms and find that
hallucinations arise from systematic internal dynamics rather than random noise. First,
attention disproportionately concentrates toward shortcut-like structural cues rather than distributing across the full context. Second, feed-forward representations fail to ground the provided knowledge, causing the model to revert to parametric
memory. Moreover, our results indicate that
hallucination is consistently associated with failures in semantic grounding within feed-forward layers, while
attention allocation exhibits greater task-dependent variability. Finally, we show that these mechanistic patterns generalize beyond single-hop graphs to multi-hop and tabular settings, enabling effective
hallucination detection across structured knowledge formats.
Read the full abstract ▾
All authors Shanghao Li, Jinda Han, Yibo Wang, Yuanjie Zhu, Zihe Song, Langzhou He, Kenan Kamel A Alghythee, Philip S. Yu
Computation and Language
In-Context Optimization for Retrieval-Augmented Generation: A Gradient-Descent Perspective
Mingchen Li, Jiatan Huang, Chuxu Zhang et al. (+2 more)
In-context learning has recently been linked to implicit gradient descent in linear self-
attention models, suggesting that context can induce a forward-pass update.
Retrieval-augmented generation (RAG) also relies on context, but retrieved documents are usually treated as static evidence rather than signals for adaptation. We study RAG as an in-context optimization process. First, we show that one linear self-
attention layer can implement one gradient-descent step on a unified linearized RAG objective covering both projection-based and dot-product
retrieval interfaces. This gives an exact regime where
retrieval-augmented prediction and in-context optimization coincide. We use this result not as a literal model of LLM computation, but as a guide for adapting the interaction between queries and retrieved evidence. We then test the boundary of this correspondence: it remains stable under controlled linear extensions, but becomes feature-distribution dependent under nonlinear architectures. Finally, we turn this view into a lightweight method for frozen RAG LLMs. The method keeps the retriever and backbone fixed, and predicts a context-conditioned update to a generator-side evidence-use interface. Across seven QA
benchmarks, two retrievers, and two frozen LLM backbones, this forward-only update improves a shared-interface baseline, transfers to held-out tasks, and approaches test-time gradient adaptation at much lower per-query cost.
Read the full abstract ▾
All authors Mingchen Li, Jiatan Huang, Chuxu Zhang, Liang Zhao, Hong Yu
Computation and Language · Cross-listed from cs.LG
Energy-Gated Attention and Wavelet Positional Encoding: Complementary Inductive Biases for Transformer Attention
Athanasios Zeris
Standard
transformer attention computes pairwise
token similarity but treats all
tokens as equally salient and all positions as equally local, regardless of the informational structure of the input. We identify two complementary inductive biases that standard
attention lacks: energy salience (which
tokens concentrate informational energy, learned end-to-end without explicit frequency decomposition) and scale-selective locality (how far positional influence extends at each frequency, implemented via Morlet wavelet encoding). We address both with two simple components. Energy-Gated
Attention (EGA) gates value aggregation by a learned energy estimate of key
token embeddings, computed via a single linear projection; it selects what to attend to. Morlet Positional Encoding (MoPE) replaces fixed sinusoidal encodings with learned Gaussian-windowed wavelets that adapt the joint position-frequency localization to the corpus; it specifies where
attention operates at each scale. On TinyShakespeare, EGA alone achieves +0.092 validation loss improvement over standard
attention (+0.103 over Phase 1-3 baseline); MoPE alone is -0.032 (below baseline as a standalone encoding); but their combination achieves +0.119 – more than the sum of parts. This superadditivity, observed across two independent training runs, is the central empirical finding: salience and locality are complementary inductive biases, each addressing a gap the other cannot fill alone. Ablations confirm that structured spectral priors (Morlet wavelet gates, scale-initialized heads, fixed sinusoidal PE) consistently underperform their unconstrained learned counterparts, while complementary learned components interact superadditively. All experiments are at small scale (<=6M parameters, character-level
benchmarks, single seed); larger-scale multi-seed validation is the most important direction for future work.
Read the full abstract ▾
Computation and Language
RICE-PO: Turning Retrieval Interactions into Credit Signals for Reasoning Agents
Mingchen Li, Hansi Zeng, Zhuo Qian et al. (+4 more)
Retrieval is increasingly moving from one-shot matching toward interactive
reasoning, where
language agents iteratively inspect evidence, reformulate queries, and search again. Training such
agents raises a credit-assignment challenge: executable actions such as queries or summaries can be directly evaluated by the retriever, while latent
reasoning steps are not directly observable and only affect future executable actions. This asymmetry makes outcome-level reward assignment unreliable, as the same final reward may credit
reasoning steps that did not actually shape
retrieval success. We propose RICE-PO, a critic-free policy optimization framework that converts
retrieval interactions into localized learning signals. RICE-PO selects high-uncertainty executable actions as anchors, evaluates local counterfactual branches using
retrieval metrics, and propagates credit to latent
reasoning steps only when
reasoning-to-action influence is strong and future residual effects are stable. On BRIGHT and BEIR, RICE-PO consistently outperforms
prompt-based
agents and group-based RL baselines under the same retriever setting. These results show that the structure of
agent-environment interaction itself can provide useful supervision for training
reasoning-based
retrieval agents.
Read the full abstract ▾
All authors Mingchen Li, Hansi Zeng, Zhuo Qian, Jiatan Huang, Sunjae Kwon, Hamed Zamani, Hong Yu
Computation and Language
The Daily Dose: Workflow-Integrated Large Language Model Automation for Clinical Summarization and Trial Identification in Radiation Oncology
Jason Holmes, Federico Mastroleo, Mariana Borras-Osorio et al. (+17 more)
Objective: To describe the design and early clinical evaluation of The Daily Dose (TDD), an LLM-driven, automated clinical summarization and clinical-trial identification system integrated into routine radiation oncology practice. Design: Mixed-methods evaluation using a cross-sectional, anonymous clinician survey administered after 1 month of system deployment. Exposure: Daily automated delivery of physician-specific email summaries generated using RadOnc-GPT, including patient schedules, concise EHR-derived clinical-status summaries, and automated identification of potentially relevant clinical trials for new or consult visits. Main Outcomes and Measures: Primary outcomes included self-reported usability, satisfaction, perceived usefulness, perceived impact on workflow, time savings, and intention for continued use. Internal consistency reliability was assessed using Cronbach's $\alpha$. Results: Among 55 respondents, 52 (94.5%) worked in radiation oncology, and 38 (69.1%) were attending physicians. Most participants (83.6%) reported using TDD daily or several times per week. Mean (SD) scores were 3.89 (1.04) for usability and satisfaction, 3.43 (1.24) for perceived usefulness, and 3.80 (1.17) for impact and future use (5-point Likert scale). Overall satisfaction was positively associated with perceived time savings ($p < .001$). Participants reported variable time savings, with 27% estimating $\geq 10$ minutes saved per day. The questionnaire demonstrated excellent internal consistency (overall Cronbach's $\alpha$ = 0.97).
Read the full abstract ▾
All authors Jason Holmes, Federico Mastroleo, Mariana Borras-Osorio, Srinivas Seetamsetty, Satomi Shiraishi, Mirek Fatyga, Judy C. Boughey, Cornelius A. Thiels, William G.Breen, Daniel J. Ma, Daniel K. Ebner, David M. Routman, Brady S. Laughlin, Carlos E. Vargas, Samir H. Patel, Sujay A. Vora, Nadia N. Laack, Andrew Y.K. Foong, Wei Liu, Mark R. Waddle
Computation and Language · Cross-listed from cs.AI
ScientistOne: Towards Human-Level Autonomous Research via Chain-of-Evidence
Rui Meng, Bhavana Dalvi Mishra, Jiefeng Chen et al. (+10 more)
Autonomous research
agents produce competitive solutions and professional-looking manuscripts, yet their outputs contain verifiability failures undetectable by surface-level evaluation: fabricated citations, unreproducible scores, and method descriptions that diverge from the implementation. We address this through three contributions. First, Chain-of-Evidence (CoE), a verifiability framework requiring every claim to be traceable to its evidence source. Second, ScientistOne, an end-to-end autonomous research system that maintains evidence chains by construction throughout literature review, solution discovery, and paper writing. Third, CoE Audit, a post-hoc audit whose four integrity checks – score verification, specification violation, reference verification, and method-code
alignment – apply uniformly to all systems. Across 75 papers spanning five systems and five frontier research tasks, every baseline exhibits at least one systematic failure mode: hallucinated reference rates reach 21%, score verification passes in as few as 42% of papers, and method-code
alignment ranges from 20% to 80%. ScientistOne achieves zero hallucinated references (0/337), perfect score verification (12/12), and the highest method-code
alignment (14/15), while matching or exceeding human expert performance on all five tasks. ScientistOne further generalizes to six additional tasks spanning medical imaging, fine-grained recognition, 3D perception, and
language modeling, achieving state-of-the-art on Parameter Golf and gold medals on MLE-Bench tasks where baselines fail entirely.
Read the full abstract ▾
All authors Rui Meng, Bhavana Dalvi Mishra, Jiefeng Chen, Chun-Liang Li, Palash Goyal, Mihir Parmar, Yiwen Song, Yale Song, Rajarishi Sinha, Parthasarathy Ranganathan, Burak Gokturk, Jinsung Yoon, Tomas Pfister
Computation and Language · Cross-listed from cs.LG
QAM-W: Joint 2D Codebook Quantization for LLM Weights via Hadamard Rotation and Activation-Aware Scaling
Preetam Sharma and Kacper Dobek
Scalar post-training quantizers discard pairwise coordinate structure within weight rows. We introduce QAM-W (Quadrature Amplitude Modulation for Weights), a codec that recovers this structure: each row is L2-normalized, block-Hadamard rotated, paired into 2D coordinates, and quantized against a single Lloyd-Max codebook trained on the unit circular Gaussian, with activation-aware per-channel scaling. In a cross-model study spanning five LLMs from four families (1.1B–13B parameters) and eight quantized configurations, the activation-aware variant at $\approx 5.5$ bpw stays within $\pm 0.4\%$ of BF16 WikiText-2 perplexity on every model, matching the SmoothQuant W8A8 quality envelope at $32\%$ fewer weight bits. Joint 2D coding outperforms polar (amplitude $\times$ phase) coding by 2–15~pp $\Delta$PPL at equal bitrate, and paired KL against BF16 tracks $\Delta$PPL% at Spearman $\rho = 0.99$ across 37 (method, model) rows, consistent with a monotone composite bound from codec distortion to KL divergence. A 3.5~bpw variant is competitive on quantization-tolerant architectures. At strict 4~bpw, the rotated-codebook frontier method QTIP outperforms QAM-W; the contribution is the quality-preserving 5–6~bpw band.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
MULTISEISMO: A Multimodal Seismic Dataset and Model for Cross-Modal Seismic Understanding
Sai Munikoti, Ian Stewart, Chengping Chai et al. (+4 more)
The application of generalist
multimodal models (GMMs) to specialized scientific domains remains limited due to the scarcity of comprehensive domain-specific
datasets that integrate multiple data modalities beyond text and images. In seismology, understanding earthquake phenomena requires the synthesis of timeseries waveform data, geographical imagery, and contextual metadata, a
multimodal integration absent in existing seismic
datasets. We present MultiSeismo, a large scale structured
multimodal seismic
dataset, comprising over 16K seismic events spanning 13 years (2010 to 2023) across diverse geographical regions. Each event data integrates waveform recordings from global station networks, intensity maps, population exposure visualizations, and a comprehensive textual description within a standardized JSON format. We additionally develop MISCE, a
multimodal instruction set on top of raw data to enable supervised training and evaluation of GMMs on seismic
reasoning tasks ranging from basic information
retrieval to complex cross modal analysis. We leverage MISCE to finetune an existing
multimodal model (Unified IO 2) enhanced with a specialized timeseries encoder, which yields SeisModal, the first domain specific
multimodal model for comprehensive seismic analysis. Evaluation of state of the art
multimodal models on MultiSeismo reveals significant challenges, particularly with time-series data processing for general purpose models, while demonstrating SeisModal's superior performance on seismic
multimodal reasoning tasks. These results prove that MultiSeismo provides a rigorous
benchmark for future
multimodal research in seismology and validate the success of our domain specific architectural adaptations.
Read the full abstract ▾
All authors Sai Munikoti, Ian Stewart, Chengping Chai, Lisa Linville, Scott Vasquez, Sameera Horawalavithana, Karl Pazdernik
Computation and Language · Cross-listed from cs.AI
Your Agents Are Aging Too: Agent Lifespan Engineering for Deployed Systems
Jianing Zhu, Yeonju Ro, John Robertson et al. (+5 more)
Long-lived AI
agents are increasingly deployed as persistent operational systems, yet they are still evaluated like freshly initialized models. Day-one
benchmarks miss a basic systems question: how long does an
agent remain reliable after deployment? Even when model weights are frozen, an
agent's effective state keeps changing as it compresses interaction history, retrieves from a growing
memory store, revises facts after updates, and undergoes routine maintenance. Reliability therefore becomes a lifespan property of the full
agent harness, not only a snapshot property of the base model. We introduce AgingBench, a longitudinal reliability
benchmark for
agent lifespan engineering: measuring not only whether deployed
agents degrade, but what form the degradation takes and where repair should target. AgingBench organizes
agent aging into four mechanisms: compression aging, interference aging, revision aging, and maintenance aging. To diagnose these failures, AgingBench uses temporal dependency graphs and paired counterfactual probes that produce diagnostic profiles for the write,
retrieval, and utilization stages of the
memory pipeline. Across 7 scenarios, 14 models, multiple
memory policies, and both runner-controlled and autonomous
agents, over ~400 runs spanning 8 - 200 sessions show that
agent aging is not one-dimensional: behavioral tests can remain clean while factual precision decays; derived-state tracking can collapse sharply within a single model; and the same wrong answer can require different repairs depending on what the diagnostic profile points to. These results suggest that reliable
agent deployment requires lifespan evaluation, mechanism-level diagnosis, and stage-targeted repair, not only stronger day-one models.
Read the full abstract ▾
All authors Jianing Zhu, Yeonju Ro, John Robertson, Kevin Wang, Junbo Li, Haris Vikalo, Aditya Akella, Zhangyang Wang
Computation and Language
CroCo: Cross-Lingual Contrastive Preference Tuning on Self-Generations
Mike Zhang, Ali Basirat and Desmond Elliott
Prior work establishes that controlled contrastiveness between self-generated responses from large
language models, set via reward scores, improves downstream preference tuning in English. We extend this method to multiple
languages and evaluate two models across a total of 14 high and low-resource
languages on a diverse set of tasks. Our central finding is that cross-lingual contrastive preference tuning on self-generations (CroCo) transfers without
language-specific preference annotation. A reward model trained on English preferences (atop a
multilingual base) produces useful within-
language rankings across most
languages, and pairing in either a monolingual or
multilingual setting improves over each model on the majority of setups while preventing the catastrophic forgetting of supervised fine-tuning. We observe that the gains require on-policy data. Off-policy responses reduce the benefit and online preference optimization fails to improve over the offline variant. Specifically, on structured tasks, our method matches or exceeds the base in 6/7
languages for EuroLLM-9B and 4/7 settings for Aya-3B. On open-ended generation, evaluated by two judges, both tuned models win 28/30 times against their respective base across 15 evaluated
languages (high and low-resource). Overall, we show promising directions for
multilingual preference tuning using self-generations.
Read the full abstract ▾