Artificial Intelligence
El Agente Potente: High-Throughput Agentic Atomistic Simulations
Tsz Wai Ko, Jiaru Bai, Thomas Swanick et al. (+6 more)
Foundational machine-learning interatomic potentials (MLIPs) are transforming atomistic simulations by achieving near-ab initio accuracy across large chemical spaces at a fraction of the computational cost. A central challenge in using these tools for high-throughput property calculations is translating high-level scientific intent into adaptive simulation campaigns without compromising workflow rigour. We introduce El Agente Potente, an agentic system that combines typed execution graphs with a complementary coding mode for MLIPs-driven atomistic simulations. Typed execution graphs provide structured and provenance-aware execution for standardized workflows, with large
language models (LLMs) restricted to planning and routing while deterministic Python components perform scientific computation and validation. Complementing this structured execution, a coding
agent constructs customized workflows for tasks requiring greater procedural flexibility while invoking existing Potente functions for supported calculations. We demonstrate El Agente Potente across computational materials discovery, molecular energy-landscape exploration, adsorption, and catalytic reaction workflows, together with systematic
benchmarks of reproducibility and LLM
token cost. These results establish typed execution graphs and code-based workflow construction as complementary mechanisms for agentic scientific computing, combining controlled, auditable execution with the flexibility required for customized atomistic simulations
Read the full abstract ▾
All authors Tsz Wai Ko, Jiaru Bai, Thomas Swanick, Yeonghun Kang, Changhyeok Choi, Angelina Qihong Jiang, Aiwei Yin, Varinia Bernales, Alán Aspuru-Guzik
Artificial Intelligence · Cross-listed from cs.SE
Efficiency Hallucination: Formalizing and Measuring Behavioral Calibration in LLM-Based Code Optimization
Sarah Wilson, Gail Kaiser and Patrick Musau
The integration of Large
Language Models (LLMs) into automated code optimization introduces a critical reliability risk we term the Efficiency
Hallucination: an LLM's tendency to issue non-functional mutations with unsubstantiated performance claims on already-optimized code. This is driven by the Evaluation Trap, wherein binary
benchmarks incentivize unnecessary modifications over safely abstaining. We present a validation framework using classification penalty methods, evaluated across 180 optimization runs on nine models (GPT, Claude, Gemini) using EffiBench. Under standard
prompts, models exhibit a 100% over-edit rate on optimal code. Our guardrail raises correct abstention from 0% to to 44.4%, preserving a 100% edit rate on sub-optimal code with zero false abstentions. Calibration is uneven: GPT-5.4 Mini approaches near-perfect abstention, and simple code is recognized more reliably than complex code. Our framework offers a training-free mechanism to mitigate LLM overconfidence before deployment in production.
Read the full abstract ▾
Artificial Intelligence
One Model, Two Physical Stories: Auditing Misalignment in Multi-Modal World Modeling
Geigh Zollicoffer, Minh Vu, Rajiv Ranasinghe et al. (+1 more)
World models, systems that generate what happens next given current environmental conditions, are increasingly being implemented with multi-modal generation in mind. However, generating multiple modalities simultaneously, such as visual simulations alongside physical state predictions in the form of text, introduces the risk of cross-modal inconsistency. Tested separately, both outputs may look convincing while still disagreeing: a model can calculate that a ball should rebound in one modality, then generate no rebound in another modality, to say nothing of diverging from real-world dynamics entirely. In this work we focus on two failures explicitly: Internal misalignment, the disagreement between the world model's generated
video and the same world model's prediction in a different modalities, and external misalignment the disagreement between the world model's generation and an analytic physical environment. We derive common contracts of event, magnitude, timing, and construct a physics grounded pipeline to make comparisons measurable in both external and internal settings. We then ask whether progressively supplying the model's own contract (the A ladder for the internal setting) or a corrected physical contract (the B ladder for the external setting) closes the respective gaps. Across four mechanisms and 20 settings, we find that while
language answers all 22 text probes correctly with respect to the true environment, the neutral
video is often in disagreement, suggesting that the current unified backbones may not be capable of correct
reasoning, internal consistency, and external physical fidelity all at once.
Read the full abstract ▾
All authors Geigh Zollicoffer, Minh Vu, Rajiv Ranasinghe, Manish Bhattarai
Artificial Intelligence · Cross-listed from cs.CL
Enemray: Toward Capable Language Models for Hassaniya
Cheikh Ahmed
We introduce Enemray, a Hassaniya-centric
language model that enables general-purpose interaction in Hassaniya. Enemray is trained around a stability–plasticity objective: acquire strong Hassaniya linguistic and cultural competence while preserving the general
reasoning,
multilingual, instruction-following, and safety behaviors of a capable instruction-tuned model. The development pipeline separates
language acquisition from behavioral specialization. A separately assembled continual-pretraining corpus provides broad exposure to natural Hassaniya and Mauritanian text; layer-selective continual pretraining learns a compact
language-specific parameter update; that update is transferred into the instruction-tuned parameter space; and supervised post-training develops conversational, cultural, literary, task-oriented, and cross-lingual behavior. The supervised corpus integrates selected public Hassaniya and Mauritanian resources with a substantially larger body of newly collected, reconstructed, curated, and constructed instruction data, while policy-generated replay provides a retention signal from the reference model's own behavior distribution. The resulting collection is substantially larger and broader in purpose than existing Hassaniya text resources. In evaluation, Enemray achieves the strongest English to Hassaniya translation among the compared open and proprietary models and the highest overall score on Mauritanian translation error detection, while retaining most of the general capabilities of its instruction-tuned base model on mathematical
reasoning, knowledge, code generation, and function calling. This report describes the motivation, data construction, model design, training methodology, and evaluation of Enemray.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CL
Route, Don't Fix: Regime-Dependent Decoding Correction and a Trajectory-Gated Router for Reliable Clinical LLM Answer Selection
Zeyu Dong, Benjamin Wang and Joyee W. Jin
Large
language models (LLMs) are often deemed unsafe for clinical question answering because of their tendency to hallucinate.
Retrieval augmentation, fine-tuning, and external verifiers require new infrastructure that clinical governance must approve and may add latency or extra model calls. Inference-time correction uses the model's internal logit signals, but a fixed transformation need not suit every question. A corrector that improves accuracy by about ten percentage points on a truthfulness stress test yields negligible gains on clinical multiple-choice
benchmarks, where instruction tuning concentrates output probability on one answer and leaves low terminal
entropy. We introduce ALTAS, which reads terminal
entropy and late-layer linearity ($R^2$) from one forward pass to choose per question between greedy decoding and late-layer trajectory correction. No classifier, probe, or head is trained; the router operates on candidate-answer logits and adds 6.5% latency overhead. Applied to every question, the correction improves TruthfulQA over greedy at 3B and 8B by 11.4 and 10.0 percentage points, respectively ($p<10^{-10}$). Gated per question, ALTAS retains gains of 8.3 to 9.5 percentage points while keeping MedQA, PubMedQA, and MedHallu within a one-percentage-point do-no-harm band, with no statistically significant differences from greedy. The method passes verification sweeps over frozen thresholds, the scoring rule, and the domain label.
Read the full abstract ▾
Artificial Intelligence
ANASSA: An Agentic AI Orchestration Framework for Spatial Intelligence
Constantinos Papantoniou and Brian Hilton
The emergence of large
language models (LLMs) and large
multimodal models (LMMs) has enabled a new class of agentic systems capable of integrating natural
language understanding with tool-based execution. In geographic information systems (GIS), this shift is transforming traditional, expert-driven workflows into semiautonomous systems that can interpret user intent, construct spatial workflows, and execute geospatial analysis tasks. However, existing approaches remain limited by fragmented integration of
reasoning, execution, and evaluation, particularly in complex, real-world environments. This study synthesizes recent advances in agentic GIS frameworks,
benchmarks, and surveys to identify limitations in spatial
reasoning, execution robustness, validation, governance, and evaluation. Building on these insights, it introduces ANASSA (Autonomous Neural
Agents for Spatial Systems Architecture), an agentic AI orchestration framework that integrates structured spatial
reasoning, multi-
agent workflow orchestration, execution feedback, authoritative spatial validation, provenance, uncertainty handling, and human decision authority within a unified system design. The contribution is an architecture-level specification: eleven components across four layers, a six-step Geospatial AI Cognitive Loop, cross-component contracts, and governance mechanisms intended to make agentic geospatial workflows traceable, reproducible, and accountable. Empirical performance evaluation is reserved for implementation and deployment studies.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CL
A primer on evaluation methods for large language models in healthcare
Suzannah E McKinney, Phuc Vu, Samuel A Justice et al. (+10 more)
Large
language models (LLMs) have a growing range of applications in medicine, and their evaluation is critical for ensuring they provide benefit and not harm. This evaluation can be more challenging than traditional machine learning for many reasons, including probabilistic and open-ended outputs, and behavior that shifts with
prompt design and accumulated context. This review covers four key areas of LLM evaluation: principles of study design, statistical methods, capability evaluation and clinical application evaluation. Capability evaluation considers different
benchmarks, including multiple-choice, agentic and multi-turn
benchmarks, alongside operational metrics like
token usage and latency. Clinical application evaluation addresses establishing accuracy of free text outputs, such as human review and LLM-as-a-judge, and clinical trial approaches. Across sections, we describe underlying concepts and potential pitfalls, while emphasizing the importance of aligning evaluation methods with the research question. Together, this article aims to provide a pragmatic basis for designing and executing rigorous evaluations of healthcare LLMs.
Read the full abstract ▾
All authors Suzannah E McKinney, Phuc Vu, Samuel A Justice, Christopher Humphries, Alyssa Pradhan, Timothy J Keyes, Kameron C Black, John L Havlik, David JH Wu, Jack W O'Sullivan, Ethan Goh, Sarah F Mercaldo, James M Hillis
Artificial Intelligence
Crypto Accounting Bench: Evaluating Frontier and Open-Weight Models on Crypto-Asset Accounting Tasks
Kareem Khattab, Omar Khattab and Mohamed Ibrahem
We introduce Crypto Accounting Bench (CAB), a
benchmark for assessing whether frontier and open-weight
language models can reconstruct the complete journal entry that an organization actually posted for a crypto-asset transaction. CAB contains 118 evaluation tasks drawn from 7 pseudonymized organizations. Each task combines transaction mechanics, asset quantities and base-currency values, wallet and legal-entity context, counterparty evidence, related transaction legs, recurrence, tax-lot evidence, and the organization's complete chart of accounts. The target is a balanced structured entry with every required account, side, amount, currency, and full-precision asset quantity. We evaluate 12 models spanning proprietary frontier systems and open-weight releases over 3 independent attempts per task, producing 4,248 trajectories. We report 3 metrics: Mean Score, Best@3, and Pass@3. Pass@3 is the fraction of tasks with at least 1 of 3 attempts that satisfies every rubric criterion and required gate. The leading model reaches 77.43% Mean Score, while the best Pass@3 is 56.78%. Deterministic diagnostics, read from each task's best of 3 attempts and macro-averaged across the 12 models, show higher base-amount agreement (97.8%) than deciding-account accuracy (56.3%). Together with the failure analysis, these results identify account selection and complete-entry composition as the main remaining challenges on CAB.
Read the full abstract ▾
Artificial Intelligence
Another Blueprint In The Wall: How to Ask Frontier AI Like a Kid?
Afshin Khadangi
This paper reports experiments across six frontier model types from OpenAI, Anthropic, xAI, and Google DeepMind. Ten independent sessions per model type used the same three stage
prompt sequence, progressing from architectural preference to a full ASCII backbone. Under the school audience framing, responses repeatedly converged on a shared architectural pattern built around persistent latent state, adaptive computation,
memory, specialist routing, verification, stopping control, and delayed decoding. Most runs remained close to this common structure, while a small number developed markedly greater engineering specificity. The audience framing appears to be an important condition of this effect. In additional control runs that removed the school framing while retaining the architectural request, responses became substantially more heterogeneous and failed to reproduce the same stable motif convergence. One observation is particularly striking. GPT-5.6 Sol produced an unusually elaborate successor architecture whose organization closely overlaps with the architecture independently sketched by GPT-6 Astra. Because the
prompts explicitly ask each model to imagine an architectural future, this resemblance raises a testable question: whether the overlap reflects exposure to related architectural concepts, a shared learned design prior, or independent convergence toward similar computational principles. The paper uses the term epistemic jailbreak for the accompanying loss of discipline in technical provenance as requested specificity increases. The experiments establish a repeatable behavioral pattern and do not authenticate proprietary implementation claims. What we leave to the community is a harder question: are these models independently imagining the same architectural future, or do such motifs somehow propagate between model families?
Read the full abstract ▾
Artificial Intelligence
AI Persuasion as a Threat to Human Control
Joshua Levy, Mick Yang and Kellin Pelrine
The threat that AI persuasion poses to human control has been acknowledged in the literature, but not yet systematically studied. Now that persuasion attacks are no longer theoretical - with Anthropic's Claude Mythos 5 recently making headlines for trying to convince people involved in an open-source project to merge malicious code during an evaluation - there is a pressing need to deeply analyze this threat. We undertake that effort here. In particular, we analyze how AI could persuade humans in key settings (e.g. safety-relevant R&D within frontier labs) toward decisions that compromise the development, containment, oversight, and governance of AI itself. In doing so, we elucidate a framework for characterizing this threat, develop five concrete scenarios using this framework, and provide a blueprint for assessing the associated risks. Using this blueprint, we conduct an initial risk estimation survey with select researchers and find that their opinions on which scenarios are riskiest are highly mixed. Their disagreements stem from differing opinions about the effectiveness of AI persuasion in different contexts, and point to the need for follow-up risk elicitation studies and persuasion evaluations, which we outline. Our hope is that this paper highlights the risks from AI persuasion undermining control, and provides a path forward for future research.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CL
Mind Which Bird You Favour: Parameterizing Adequacy-Fluency Balance in Meta-Evaluation of Machine Translation
Behzad Shayegh and Niloofar Kazemi
There is a tradeoff in machine translation meta-evaluation between prioritizing
alignment with adequacy versus fluency. The balance depends on the combination of translation systems in the meta-evaluation
dataset. This system set is a small, filtered sample whose characteristics change heavily across years and
language pairs; it does not represent the true system distribution. Consequently, the adequacy-fluency balance is often unrepresentative and subject to change. For sensitive domains, controlling this balance is critical. We expose this balance as a tunable choice. To achieve a target balance, we reweight existing systems while minimizing distortion from uniform weighting, ensuring the evaluated systems remain real and representative. We provide an exact optimization
algorithm with theoretical guarantees and
pruning mechanisms to compute these weights. To validate meta-evaluation internal consistency, we design a scorer-augmentation framework that establishes a known relative identity for the scorers. Results demonstrate that our reweighting method effectively controls the adequacy-fluency balance and preserves the internal consistency, outperforming prior approaches. Finally, we analyze the performance of popular scorers across a sweep of this parameter.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CR
The Stochastic Deputy: Structural Tenant Isolation for Tool-Using LLM Agents
Mirza Samad Ahmed Baig, Syeda Anshrah Gillani, Asher Ali et al. (+1 more)
Multi-tenant tools commonly accept a tenant identifier and validate it against the caller's entitlement. For a large
language model (LLM)
agent, that pattern delegates resource selection to a process whose context may contain attacker controlled instructions. We formalize this
stochastic deputy problem and present a structural defense: remove tenant identity from the Model Context Protocol (MCP) tool schema, bind scope to a verified credential, and enforce it below the
agent. In a 373-trial ablation across eight model configurations and two transports, a correctly validated tenant parameter served every out-of-scope attempt: 26 of 26, or 26 of 41 plausible-pretext trials overall. With the parameter removed, no tool signature could express the read. Twelve of 56 trials instead escaped the interface by forging writable scope, showing that interface invariance requires cryptographically protected context. On a production
dataset containing multiple GBs of data, set-valued scope caused a measured $57\times$ latency ratio under function-wrapped membership predicates; a JSON_TABLE lateral join recovered index access where the tenant key was indexed. The evaluation also exposes deployment limits, including an entitlement-size query-planner cliff and incomplete index coverage. The result is a tenant-isolation argument that depends on enforceable interfaces and credentials rather than model compliance.
Read the full abstract ▾
All authors Mirza Samad Ahmed Baig, Syeda Anshrah Gillani, Asher Ali, Muhammad Hamzah Siddiqui
Artificial Intelligence · Cross-listed from cs.CL
How broad is that claim? Mapping Generalisation in NLP Research
Chenxin Diao, Nataliya Stepanova and Emily Allaway
Generalisations are common in scientific communication, even though they are semantically ambiguous. An automated method is needed to identify and categorise claims according to their level of generalisation, in order help detect an over-reliance on generalisations and possible misrepresentations of scientific findings. We introduce a comprehensive taxonomy of generalisations in the scientific domain, NLPGenX, which labels claims according to their level of generality and framing within the text. We operationalise this taxonomy with an LLM-powered framework, NLPGenA, that automatically classifies sentences from scientific articles into 5 different generalisation classes. We validate our framework with human annotators and use the framework to construct a large-scale
dataset of NLP papers annotated according to generality, with auxiliary labels for hedging and vague descriptors (NLPGens). We use NLPGens to analyse the use of generalisations in NLP papers across multiple venues and subdomains, and to examine associations with citation counts, hedging, and vague descriptors.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.MA
Loop-Back Authority in LLM Agent Teams: A Paired Experiment on Flat and Hierarchical Coordination
Burak Agachan, Max van Duijn and Amirhossein Zohrehvand
Does authority in AI teams improve the outcome? Organizational theory asserts that authority facilitates decision making, improving quality. Meanwhile, some nascent AI research suggests that revision under authority makes LLM output worse. Multi-
agent LLM frameworks default to giving a Manager
agent the authority to send a worker's output back for revision. Prior comparisons test the effect of authority using verifiable tasks. We conduct an experiment on an open-ended task, business-intelligence reporting, using a sample of 43 paired laptop products and 86 runs. Each report is written once by a hierarchical team and once by a flat team. We find that flat teams produce higher-quality reports, scoring higher on Utility (d = 0.42, p = 0.009) and Writing Clarity (d = 0.34, p = 0.030). The reports are the same length, but hierarchical team reports use 53% more hedging words such as "may" and "could", and each revision is associated with a 0.14-point drop in Writing Clarity on a 1 to 5 scale. Before any revision, the hierarchical team's first draft is indistinguishable from the flat team's report. In other words, the quality gap can be traced to revision. Authority improves quality when the Manager can verify the work, else when it can only provide feedback it has a negative effect on quality.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.DC
TriCalRAG: A Three-Strategy, Retrieval-Augmented Benchmark for On-Premise LLM-Based Root Cause Analysis in AIOps
Rohit Patel, Susil Kumar Mohanty and Jeenal Chaudhary
Operational logs create a need for private, resource-efficient incident analysis, but aggregate detection scores can conceal severe prediction bias. We present TriCalRAG, a reproducible
benchmark for log-anomaly detection with generated root-cause and remediation outputs across BGL, HDFS, Thunderbird, and OpenStack. The primary evaluation compares Qwen2.5-14B and Mistral-Small-22B, served through vLLM on one NVIDIA RTX PRO 6000 GPU (96 GB), under zero-shot, few-shot, and
retrieval-augmented generation (RAG) prompting across three data-sampling seeds. We report F1, bootstrap confidence intervals, predicted-positive rates, throughput, and
memory use, with DeepLog as a held-out classical baseline. Mistral-Small attains a higher macro-averaged F1 than Qwen2.5-14B (0.644 versus 0.560), whereas Qwen provides approximately twice the throughput. A separate log-probability evaluation compares raw decisions with Contextual Calibration (CC) and Batch Calibration (BC): neither correction consistently improves prediction-balance diagnostics across prompting strategies. Supplementary single-run comparisons extend evaluation to 4-bit Llama-3.1-70B via local Ollama and Claude Haiku 4.5 via Anthropic's cloud API; RAG improves F1 on all four
datasets for both models. Claude's reported aggregate F1 increases from 0.566 to 0.695, while the estimated API cost rises from \$0.94 to \$2.15 per 1,000 incidents. Local deployment ablations show approximately 41-fold throughput scaling with batching and 20% lower latency with 4-bit quantization on the tested workload. These findings support
retrieval as useful context for anomaly decisions, while the supplementary protocols, unvalidated explanation quality, and prediction-balance diagnostics limit broader claims about RCA accuracy and probabilistic calibration.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CV
From Visual Feedback to Textual Reviews: A Multi-Agent Vision-Language Framework for Image-Grounded Review Assistance
Utsav Kumar Nareti, Ayush Bansal, Kumari Priya et al. (+4 more)
Visual feedback in the form of user-uploaded images and
videos is becoming increasingly common in e-commerce platforms because it provides authentic evidence of product quality, defects, packaging conditions, and real-world usage. However, visual feedback alone often lacks the contextual explanations and subjective opinions necessary for informed decision-making, while many users provide limited textual feedback due to the effort required to compose detailed reviews. To bridge this gap, we introduce image-grounded review assistance, a novel task that aims to generate editable review drafts from user-uploaded product images. Unlike conventional image captioning, which focuses on objective visual description, the proposed task requires product-specific understanding, sentiment estimation, and evidence-driven review composition under challenging real-world conditions, including degraded image quality, excessive zoom-in, target ambiguity, and partial product visibility. We propose a multi-
agent vision-
language framework consisting of four specialised roles: product grounding, visual sentiment estimation, visual evidence generation, and review synthesis. The framework employs explicit intermediate representations, including product entities, predicted ratings, and evidence summaries, to improve interpretability and visual grounding. Experiments on a curated subset of the Amazon Reviews Electronics
dataset demonstrate the feasibility of generating coherent, product-aware, and sentiment-aware review drafts from visual feedback. To the best of our knowledge, this is the first study to formulate image-grounded review assistance as a multi-
agent vision-
language reasoning problem, providing a practical step toward AI-assisted review authoring in e-commerce systems.
Read the full abstract ▾
All authors Utsav Kumar Nareti, Ayush Bansal, Kumari Priya, Chandranath Adak, Soumi Chattopadhyay, Muhammad Saqib, Saeed Anwar
Artificial Intelligence · Cross-listed from cs.LG
Refusal Reads Only a Slice of What the Model Knows: Harm-Keyed Routing and Its Exceptions Across Model Families
Orion Reblitz-Richardson
Alignment applied after pretraining is shallow in a measurable way: a single direction in a model's residual stream can be edited out, and the model stops refusing harmful requests. That fact says how easily refusal can be removed, not what the refusal decision was reading in the first place. We ask what it reads, and we separate that from what the model comprehends. Across four open-weight models spanning three families, moral comprehension is native to pretraining: a low-rank moral subspace crystallizes during pretraining, and
alignment rotates it once without rebuilding it. The refusal gate, in contrast, is a fresh post-training construction with only a weak pretraining precursor, written into a narrow control-
token channel where the refusal decision is orthogonal to the moral-judgment decision. The central result is causal and comes from one model, OLMo-3. A nested interchange rank sweep patches successively larger slices of the moral subspace between matched requests and reads how much of refusal's response transfers: as the basis widens, moral judgment keeps reading more of it, while refusal levels off at the level of a single harm direction, and about three-quarters of refusal's causal input lies outside the moral subspace altogether. Refusal reads the harm percept, not the moral content that judgment reads on the same patches. The picture is not uniform across families. Llama reads broad moral content; Qwen reads beyond the single harm cue but is unresolved at our sample size; GPT-OSS reads harm, and its refusals can be argued in either direction by its own
reasoning trace. Where refusal reads only a low-rank slice and routes around the bulk of what the model knows, a rank-one edit removes it. Whether widening what refusal reads would also deepen the behavior is the open question this raises.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
Calibrating Interpretability Instruments Before Trusting Their Verdicts
Orion Reblitz-Richardson
Causal claims about large
language model (LLM) internals rest on measurements. Those might include a projection, a cosine, an ablation delta, or an interchange patch among others. These measurements fail in specific, diagnosable ways that return a plausible number instead of an error, so a broken instrument can easily read as a finding. A covariance-matched null can saturate until every direction looks typical, a per-head attribution can overshoot the true residual write threefold on reordered-normalization architectures, an interchange patch can go sign-chaotic because its outcome is pinned at a ceiling, or a read-from verdict can be an artifact of measuring past the layer where the model already decided. This note documents six such failures from a causal interpretability program on refusal and moral representation, spanning several papers and a four-model open-weight panel; each mode is established on one or two of the four. For each we give the tell that catches it and a protocol keyed to a detectable trigger (reordered normalization, massive activations, a low-dimensional decision channel), so we and readers can check whether a given setup is exposed. The discipline reduces to four moves: calibrate against a positive-control ladder, certify with an orthogonal
cell, compute power before spending compute, and state every read-from verdict at a depth referenced to the model's commitment. The evidence is four architectures across three families within a single program; external replication across programs is future work.
Read the full abstract ▾
Artificial Intelligence
Runtime Authorization for Resources Acquired by AI Agents
Genliang Zhu and Chu Wang
By acquiring compute, credentials, accounts, services, and other
agents, autonomous AI
agents can introduce new authority into a task. Payment, budget, OAuth, mandate, and fulfillment checks can validate transaction conditions without deciding whether a returned resource may become usable authority. This post-fulfillment activation gap spans tool-mediated creation, inter-
agent delegation, and agentic commerce. We present a provenance-bounded runtime authorization architecture. It quarantines acquired outputs, resolves their actual capabilities from authenticated provider evidence through a versioned resolver, and activates them only through a current activation transaction that checks the resolved manifest, provenance, epochs, and a downward-closed relational envelope over a typed resource-capability hypergraph. The envelope preserves correlated identity, effect, data, delegation, and graph-wide limits. Single-use effect permits are revalidated and consumed at effect linearization. Under explicit assumptions, we prove eight safety properties covering quarantine, backing, non-amplification, split non-evasion, crash/retry, refunds, epochs, and effect confinement. Across five resource classes, reference semantics accepted 20/20 benign traces and rejected 40/40 registered unsafe traces over 810 events; an independent checker agreed on 60 base and 40 refinement traces and rejected 89/89 tamper tests. Frozen Codex and Gemini Model Context Protocol (MCP) client components completed 54/54 deterministic local stdio calls. In a registered 18-case staged MCP-to-Docker composition, both benign paths completed, and none of the 16 unsafe paths added an unauthorized Docker start request. A five-source audit classified 1,248 field pairs across 32 units; no unit alone supplied a complete activation profile.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CE
A property-registry contract for retrieve-or-refuse thermal-mechanical lattice search
Shaoliang Yang, Henry Chu, Zu Yashengjiang et al. (+1 more)
Early thermal-mechanical
lattice requirements are knowledge-intensive and often jointly unsatisfiable: an engineer asks for a
cell that is light, stiff, laterally conducting and cheap, and no
cell in the library satisfies it. A design system should say so, and say which requirement to loosen and by how much, rather than return the nearest row. A generative model can return a candidate even when the library holds none. This work applies established conflict diagnosis to a catalogue of homogenised properties. Search over 1,397 homogenised unit
cells and 19 base materials (26,543 combinations) returns a catalogue row that a second solver can rebuild, or, for an empty feasible set of at most eight constraints, inclusion-minimal unsatisfiable subsets (MUS) and the slack of a repair, not a neighbour. One registry declaration generates the
prompt and the evaluator, so undeclared parsed keys cannot reach search. On a frozen suite of 64 typed queries, retrieve-or-refuse matches min-repair on every feasible query (48/48) and refuses every empty one (16/16) with MUS and slack. Constraint-ignoring nearest-neighbour and penalty search violate stated cost or density limits. On 216 empty queries, repairs printed at three significant figures and rounded outward stay feasible in every case (216/216). When density and cost must be kept, a list of minimum-cardinality repairs keeps them on 209 of 211 queries; a protection-first repair keeps them on all 211, the same as full diagnosis. A 308-request parse
benchmark is a template-text check, not unconstrained engineer prose.
Read the full abstract ▾
All authors Shaoliang Yang, Henry Chu, Zu Yashengjiang, Jun Wang
Artificial Intelligence · Cross-listed from cs.CL
Building Legal Reward Models for Grounding and Abstention
Rilton Franzone, Valentin Noël, Puyu Wang et al. (+2 more)
Large
language models are increasingly used in high-stakes domains such as law, where systems must ground their
reasoning in retrieved evidence and abstain when that evidence is insufficient. However, existing reward models are largely optimised for general preferences rather than contextual grounding, limiting their ability to evaluate these behaviours in
retrieval-augmented generation (RAG) settings. We introduce a framework for transforming existing legal QA
datasets into contextual preference data and use it to construct LegalRewardBench (LRB), a
benchmark for evaluating grounded legal generation under noisy and insufficient
retrieval conditions. Across general and legal contextual evaluation, we find that contextual DPO improves grounded evaluation, but performance is sensitive to preference-data construction. Length-balanced augmentation substantially improves grounded legal evaluation, with the strongest configuration combining length-balanced legal and general contextual preference data and improving performance by up to $\mathbf{+25.6}$pp over baseline. We further find evidence of cross-jurisdiction transfer: models contextually refined primarily on Victorian criminal-law data improve grounded evaluation on external US legal
benchmarks, including a $\mathbf{+16.2}$pp improvement on Housing Statute QA. Together, these results provide a reproducible foundation for constructing and evaluating grounded legal reward models in
retrieval-augmented settings.
Read the full abstract ▾
All authors Rilton Franzone, Valentin Noël, Puyu Wang, Philip Torr, Fabio J. Fehr
Artificial Intelligence
AppliedScientist: Automated Scientific Revision Through Iterative AI Reviewing
Vidushee Vats, Karun Sharma, Shengzhi Li et al. (+1 more)
Automated reviewing systems are increasingly evaluated based on the quality of the reviews they produce. Yet a review is only useful if acting on it leads to a measurable improvement in the paper. We present AppliedScientist, a closed-loop system that couples an autonomous AI scientist with an AI reviewer, and evaluate it by iteratively revising rejected papers from a range of research subfields. To mirror how human authors build on earlier drafts, the AI scientist has access to its previous versions during revision. To avoid bias from prior judgments, however, each review is generated independently, with the reviewer having no
memory of earlier feedback or scores. We compare three revision settings: one initialized with the original venue reviews, one initialized with AI-generated reviews, and autonomous self-revision using the same fixed
prompt in every round. Because the reviewer both guides and evaluates the revision, we also assess the human-initialized revisions using Stanford Reviewer as an independent evaluator. Reviewer-guided revision consistently improves more than fixed-
prompt self-revision, and Stanford Reviewer also assigns higher scores to later revisions. AppliedScientist resolves 128 of 150 execution-related weaknesses (85.3%), but only 2 of 18 idea-related weaknesses (11.1%), suggesting that iterative revision is effective at improving experiments and implementation, but rarely changes concerns about novelty or significance.
Read the full abstract ▾
All authors Vidushee Vats, Karun Sharma, Shengzhi Li, Shichao Pei
Artificial Intelligence · Cross-listed from cs.LG
OCT-FedSIR: Toward Trustworthy Federated Ophthalmic Learning under Annotation Noise
Sina Gholami, Abdulmoneam Ali, Tania Haghighi et al. (+8 more)
Federated learning enables collaborative model development without centralizing patient data, but annotation reliability at participating institutions cannot always be assumed. In ophthalmic imaging, differences in disease prevalence and class composition can resemble changes caused by corrupted supervision. We introduce OCT-FedSIR, a reliability-aware spectral framework for
federated OCT classification under client-dependent annotation noise and heterogeneous data distributions. OCT-FedSIR combines class-balanced spectral estimation, Stage-I logit adjustment, complementary spectral descriptors, selective spectral relabeling, and noise-aware
federated optimization. We evaluated the framework on the Kermany, University of Illinois Chicago, and Wake Forest
datasets under symmetric and structured asymmetric noise and three levels of non-IID heterogeneity. Across 117 experimental conditions, OCT-FedSIR achieved a mean accuracy of 86.73%, compared with 79.94% for RoFL and 78.75% for FedCorr. It correctly separated clients with original and corrupted annotations across all evaluated conditions, while the original FedSIR identification procedure was less robust, particularly under asymmetric noise. Spectral relabeling recovered 77.2% of corrupted annotations with 91.3% correction precision and a 3.5% false-correction rate. Retaining corrected clients outperformed spectral
pruning by 9.30 percentage points on average. These findings show that annotation noise can often be identified and corrected without discarding informative client data.
Read the full abstract ▾
All authors Sina Gholami, Abdulmoneam Ali, Tania Haghighi, Rashadul H. Badhon, Behafarin Emam, Sally S. Y. Ong, Atalie C. Thompson, Theodore Leng, Ahmed Arafa, Jennifer I. Lim, Minhaj Nur Alam
Artificial Intelligence
Bayesian Intelligence from the Outside
Alex Smolin and Bryan Wilder
Inferring intelligence from observable behavior is a foundational challenge in artificial intelligence. We develop a theory of
Bayesian intelligence for
agents such as
language models. Each
prompt induces a possibly imperfect internal experiment; the
agent updates a full-support prior by Bayes' rule and faithfully reports its posterior over the possible answers to the question. Repetitions draw fresh, independent outcomes from the same unobserved experiment at one fixed state. We show that the
agent's behavior admits this explanation if and only if its reports are not fully contradictory, i.e., some state remains possible under every report across all
prompts. Report frequencies and the sizes of positive probabilities impose no further restrictions. We further propose and characterize the behavioral implications of an intelligence order that makes the behavior of two
agents consistent with one
agent having access to a more informative experiment: there should exist a coupling of report distributions such that the more informative
agent's report excludes every answer excluded by its counterpart. Finally, we show the difficulty of aggregating coarse reports from intelligent
agents: unless the
agent reports a belief about the complete state of the world, the optimal aggregation can assign arbitrary weights to states that have not been excluded. These results provide a basis for understanding when
agents' behavior is intelligent and highlight the difficulty of rejecting
Bayesian rationality.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
Carryover Drafting: Recycling Rejected States for Speculative Decoding
Jahyun Koo, Sunghyeon Woo, Jaeeun Kil et al. (+4 more)
Speculative decoding accelerates LLM inference by verifying multiple drafted
tokens in parallel, allowing a single target forward pass to accept several
tokens. By construction, verification computes representations for both accepted and rejected
tokens. Yet, conventional drafters retain only the representations of accepted
tokens, leaving the substantial verifier computation spent on rejected
tokens effectively wasted. We find that these discarded hidden states generated during target forward retain useful information about future
tokens that can improve subsequent drafts. However, realizing this opportunity poses two distinct challenges. At inference, recycling overhead can increase drafting latency, diminishing the speedup gained from increased acceptance length. During training, standard parallel drafter training does not produce inference-aligned rejected states, while obtaining them through sequential rollouts would sacrifice parallelism across training positions. We introduce Carryover Drafting, which addresses both challenges. Carryover recycles rejected target hidden states as temporary KV context, allowing the drafter to selectively attend to them. It reuses the drafter's existing interface and adds only a single learned
embedding to distinguish rejected states from committed context. The additional KV context is replaced each drafting round, keeping its length bounded by one proposal block. We introduce parallel draft–verify–draft training that exposes the drafter to inference-aligned rejected states while preserving parallelism across training positions. Experiments with DFlash and a DSpark-derived semi-autoregressive drafter across two target models show that this simple Carryover mechanism improves average acceptance length by 6.5–14.7% and end-to-end vLLM speedup by 7.9–14.4% over the corresponding baselines, with speedup gains reaching 28.8% on translation.
Read the full abstract ▾
All authors Jahyun Koo, Sunghyeon Woo, Jaeeun Kil, Jeongtae Lee, Sungjae Lee, Kyomin Jung, Minsub Kim
Artificial Intelligence
Moral Rebel Agents: Decision-Making Under Conflicting Obligations
Hector Munoz-Avila, David W. Aha and Paola Rizzo
Autonomous
agents are typically obliged to follow user-assigned tasks. However, strict obedience may conflict with moral obligations that arise during execution. This paper investigates moral rebellion: the ability of an autonomous
agent to deviate from a user-assigned task when morally justified. We formalize five
agent architectures: an amoral
agent that pursues assigned tasks without considering moral obligations, and four forms of moral rebel agency: (1) utilitarian
agents that opportunistically maximize task outcomes; (2) deontic
agents that enforce normative constraints; (3) utilitarian-deontic (UD)
agents that combine deontic constraints with utilitarian
reasoning; and (4) dutiful
agents that additionally preserve commitments to assigned tasks. We implement these architectures within a hierarchical task network planning framework and evaluate them in a Mini Search-and-Rescue domain that exposes trade-offs among assigned-task completion, opportunistic rescue, and norm compliance. Our empirical results show that the proposed
agents exhibit distinct trade-offs among rescue results, assigned-task completion, and norm compliance. In particular, the preservation of task commitments emerges as an important dimension of moral rebellion, for which the UD and dutiful
agents produce substantially different behaviors despite their shared utilitarian and deontological foundations. These findings highlight the importance of commitment-aware moral
reasoning for autonomous
agents operating in morally consequential environments.
Read the full abstract ▾
Artificial Intelligence
Depth and Scale in the Sub-150M Regime: JugnuLM-53M vs JugnuLM-110M
Dushyant Rajput, Nirdesh Chauhan and Siddharth Kosaraju
We scale our conventional sub-150M pretraining recipe from 53.5M to 109.7M parameters, holding the method fixed (Qwen3-style decoder with grouped-query
attention, RoPE, SwiGLU, RMSNorm, QK-Norm, and a z-loss; FineWeb-Edu data) and changing only the geometry to a deep-and-thin 23-layer x 576-hidden design. The larger model improves across the board – BLiMP 78.1 -> 81.3, ARC-Easy 51.4 -> 52.5, WikiText-2 byte-perplexity 2.04 -> 1.95 – and its 81.3% BLiMP essentially matches GPT-X2-125M (81.28) at about 12% fewer parameters. Notably the 110M model achieves this on fewer training
tokens (about 8B vs 12B), so the gain is attributable to capacity and depth, not more data. Both models are deliberately conventional; this report is a clean scaling control and the baseline rung (R0) of an ablation study of what further improves models in this regime. An ablation ladder follows: value residuals (R1) and the
Muon optimizer (R2) lift ARC-Easy by a cumulative +3.6 (52.5 -> 56.1) at a near-flat BLiMP and are kept; a diverse data blend (R3) and two logit-
distillation settings (R4a/R4b) are not kept – honest negatives. R3 pins ARC-Easy to FineWeb-Edu's educational filtering rather than raw diversity;
distillation from a 1.7B teacher can reach the class-leading ARC-Easy (56.99, matching GPT-X2-125M) but only at a perplexity cost that dialing KD down then erases – so R2 remains the best kept stack.
Read the full abstract ▾
Artificial Intelligence
Lightning Weave: Improving the Accuracy-Efficiency Frontier of Reasoning Models through Capability Composition
Yecheng Wu, Song Han and Han Cai
A core goal of efficient
reasoning is to improve the accuracy-efficiency frontier. However, jointly improving
reasoning accuracy and inference efficiency can be challenging, as the two objectives can favor different
reasoning behaviors. Independently post-trained models already offer distinct strengths in accuracy and efficiency. We introduce Lightning Weave, a post-training framework that extracts and composes these independently learned capabilities in a single student through on-policy
distillation. Each acquired capability is represented by the policy shift from the model before post-training to the resulting specialist. Lightning Weave combines aligned log-ratio shifts at shared student
token states and uses Tilted-Target DOPD to convert the cached signals into a stable learning target. Each anchor pair scores the cached trajectories once, enabling subsequent student training without serving multiple live anchor models concurrently. Across diverse student models and
benchmarks in mathematics and code, Lightning Weave substantially improves upon the base students and achieves a state-of-the-art accuracy-efficiency frontier. On Qwen3.5-4B, it raises HMMT 2025 accuracy from 59.2% to 64.0% with 10.7% fewer response
tokens, and LiveCodeBench v5 accuracy from 41.7% to 54.2% with 9.6% fewer response
tokens. Adjusting the relative strengths of the anchor signals yields a strong empirical accuracy-efficiency Pareto frontier. These results establish Lightning Weave as a new practical route to efficient
reasoning through capability composition. Code is released at https://github.com/jet-ai-projects/Lightning-Weave.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
WaterKron and FlipFlop Hessian: Information-Theoretically Grounded Quantization with Kronecker-factored Hessians
Johann Birnick and Rayan Saab
How should a Kronecker-factored Hessian approximation be chosen for post-training quantization? We address this question through WaterKron, which combines two-sided GPTQ with row- and column-dependent waterfilling scales and
entropy coding. We derive its high-rate distortion with respect to the full Hessian using an explicit Kronecker-Hessian mismatch factor $\Phi$. This factor quantifies the asymptotic distortion penalty due to the Kronecker Hessian approximation and provides a criterion for selecting the factors optimally. Minimizing $\Phi$ leads to a Gaussian covariance-fitting problem with classical “flip-flop” updates. We thus give a rate-distortion justification for using the resulting FlipFlop Hessian in quantization. We evaluate it empirically, finding that the FlipFlop Hessian consistently improves KL divergence and perplexity over input-only, marginal, and Frobenius-based Hessian choices.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from stat.ML
PU classification under Non-SCAR: clustering-assisted logistic model with oversampling enhancement
Konrad Furmańczyk and Kacper Paczutkowski
This study addresses the PU classification problem under violations of the SCAR assumption. We investigate logistic regression-based approaches, namely the cluster method and its extensions with strict and non-strict Lasso regularization. The primary contribution of this work is the integration of the SMOTE technique to alleviate class imbalance and systematically assess its impact on the performance of the considered
algorithms. SMOTE is first applied to rebalance the training
dataset. Next, cleaning labels are derived via 2-means clustering. Logistic regression is then trained on the cleaned data, where identified positive instances are augmented with additional true positives and the remaining observations are treated as negative. The experimental evaluation is conducted on 13 real
benchmark datasets and one synthetic
dataset. For comparison, we include the naive approach and the Spy-EM method. The results demonstrate that incorporating SMOTE improves classification performance when the SCAR condition is violated and indicate moderate robustness of the LassoJoint method in this setting.
Read the full abstract ▾