Artificial Intelligence
Can LLMs Be CEOs? Benchmarking Strategic Resource Reallocation with Multi-Role Agent Simulation
Yuyang Dai, Xueqing Peng, Lingfei Qian et al. (+1 more)
Evaluating the decision-making capabilities of large
language models (LLMs) is a growing research priority, yet existing
benchmarks focus on isolated cognitive tasks such as
reasoning, knowledge
retrieval, and economic rationality in stylized settings. These evaluations overlook the defining challenge of real executive decision-making: integrating conflicting recommendations from specialized stakeholders under information asymmetry, organizational constraints, and temporal dependencies. We introduce CEO-Bench, a multi-
agent benchmark that evaluates LLMs on CEO-level strategic resource reallocation – the process of redirecting capital across business units in a multi-round, constraint-rich organizational environment. In CEO-Bench, LLM
agents receive conflicting advice from four role-conditioned C-suite advisors (CFO, CTO, COO, CMO), each with private signals and distinct priorities, and must synthesize these into a concrete allocation plan evaluated along four dimensions: role integration, conditional boldness, history-sensitive judgment, and plan validity. Experiments across five frontier models on 13 scenarios reveal that all models achieve high structural validity but diverge sharply on strategic calibration – the hardest capability layer. We identify systematic failure modes including single-advisor capture, conservative default under ambiguity, and historical amnesia, and uncover a structural integration-boldness tradeoff: models that engage more deeply with conflicting perspectives tend to produce less decisive action. These findings delineate the current capability boundary of LLMs as organizational decision-makers and inform the design of future AI-assisted executive systems.
Read the full abstract ▾
All authors Yuyang Dai, Xueqing Peng, Lingfei Qian, Zhuohan Xie
Artificial Intelligence
Dissecting model behavior through agent trajectories
Gaurav Gupta, Vatshank Chaturvedi, Sudipta Sengupta et al. (+2 more)
AI
agent performance is not just a modeling problem, it is fundamentally a systems problem. The advanced capabilities of models are realized through
agent harnesses. Therefore, a gap between model assumptions and harness behavior can easily prevent the model's full capabilities from translating into
agent performance. We formalize this as the `intent-execution' gap: the mismatch between what the model intends and what the harness executes, and vice versa. We argue that minimizing this intent-execution gap is as important as other aspects of harness design such as tools and execution loops. To illustrate the impact of this harness-model
alignment, we develop a simple and customizable harness called `Simple Strands
Agent' (SSA). SSA aims to find the bulk of common patterns which generalize across different model families (such as Claude, Gemini, GPT, Grok, Qwen), as well as a small number of model-specific preferences. We make two contributions: (i) we reproduce or improve on the pass@1 performance reported by diverse model-provider families on popular agentic
benchmarks (SWE-Pro, SWE-Verified, Terminal-Bench-2, and Deep-SWE), and (ii) building on an \textbf{analysis of 154k trajectories generated by SSA}, we look beyond the pass@1 numbers which tend to be relatively even across frontier models. By representing
agent trajectories in code state-spaces, we compute solution distance and backtracking to observe model-level differences in problem-solving behavior. Finer-grained metrics such as edit frequency, testing activity, and phase-transitions reveal how individual models allocate effort across different stages of problem solving.
Read the full abstract ▾
All authors Gaurav Gupta, Vatshank Chaturvedi, Sudipta Sengupta, Jun Huan, Anoop Deoras
Artificial Intelligence
MapSatisfyBench: Benchmarking Satisfaction-Aware Map Agents through Behavior-Grounded Implicit Decision Factors
Lubin Bai, Mengyu Cao, Sixue Wang et al. (+5 more)
Large
language model
agents are increasingly integrated into map services. Since map services are embedded in everyday-life scenarios rather than professional task settings, users often express their needs informally, resulting in underspecified queries with many unspoken needs, namely, implicit decision factors that are critical for user satisfaction. Although clarification is an effective way to mitigate this issue, it increases user burden in daily interaction, and a capable
agent should first proactively recover such factors from available information sources. However, evaluating this ability is challenging. The first challenge is to determine which implicit decision factors are suitable for evaluation. A factor is evaluable only if it affects user acceptance and can be recovered from information available to the
agent before it responds. Second, user satisfaction cannot be reliably represented by a single reference answer, requiring a
benchmark that converts satisfaction-relevant factors into objective and quantifiable evaluation targets. To address these challenges, we propose a restore-identify-filter framework that reconstructs complete user needs from behavior-chain evidence, identifies implicit decision factors, and retains only those supported by pre-query evidence. Building on this methodology, we construct MapSatisfyBench from large-scale, real-world anonymized user data and annotate ground truth from five dimensions and enables full-chain evaluation of satisfaction-aware map
agents. Experiments show that current
agents generally perform well on explicit task completion, but remain limited in satisfying implicit decision factors and proactively acquiring the evidence needed for satisfaction-aware decisions. These findings establish MapSatisfyBench as a
benchmark for shifting map-
agent evaluation from task completion toward satisfaction-aware spatial decision making.
Read the full abstract ▾
All authors Lubin Bai, Mengyu Cao, Sixue Wang, Zhongwei Wan, Yue Pan, Jiale Hou, Xiang Li, Xiuyuan Zhang
Artificial Intelligence
A Machine-Learned Comorbidity Index
Suleman Baloch, Kishlay Jha, Alberto M. Segre et al. (+2 more)
Traditional comorbidity scores (e.g., Charlson and Elixhauser) are widely used for risk adjustment and patient stratification, but they have two key limitations: (i) they are largely mortality-centric and do not align well with other clinical outcomes, and (ii) their linear, rule-based structure cannot capture nonlinear, outcome-specific risk relationships. We propose a Machine-Learned Comorbidity Index (MLCI) that maps diagnosis codes to a single scalar by maximizing the normalized
Hilbert-Schmidt Independence Criterion (nHSIC) between the learned score and multiple clinical outcomes. MLCI captures nonlinear risk-outcome dependence and is supported by a theory that characterizes when a unified, informative admission-level ordering can be achieved across outcomes. Empirical results on multiple
benchmark electronic health record (EHR)
datasets show that MLCI outperforms strong baselines across multiple evaluation metrics.
Read the full abstract ▾
All authors Suleman Baloch, Kishlay Jha, Alberto M. Segre, Philip M. Polgreen, Bijaya Adhikari
Artificial Intelligence · Cross-listed from cs.CL
MODE-RAG: Manifold Outlier Diagnosis and Energy-based Retrieval-Augmented Generation Evaluation
Zehang Wei, Jiaxin Dai, Jiamin Yan et al. (+1 more)
While
Multimodal Retrieval-Augmented Generation (M-RAG) enhances Large
Vision-
Language Models, it remains highly susceptible to cross-modal
hallucinations, causal fabrications, and sycophancy. Furthermore, existing mitigation pipelines often face an intervention paradox: static rules tend to unnecessarily disrupt accurate generations, whereas leaving the multi-modal
reasoning completely unguided allows existing mismatches to cascade into severe logical fabrications. To quantify and mitigate these
hallucinations, we propose a Multi-
Agent system, MODE-RAG, driven by Variational Free Energy (VFE) and internal
attention states to dynamically gate interventions. High-risk queries are routed to five stage-specific
agents, integrating Monte Carlo Tree Search (MCTS) for rigorous causal derivation and logit perturbations to penalize sycophancy. Dedicated Correction and Overseer
agents ensure formatting stability and perform post-hoc factual verification. To objectively evaluate our approach, we introduce ModeVent, a challenging subset derived from the MultiVent
dataset. Extensive experiments indicate that our system effectively reduces
hallucination rates and logical fabrication, significantly improving the robustness of M-RAG systems.
Read the full abstract ▾
All authors Zehang Wei, Jiaxin Dai, Jiamin Yan, Xiang Xiang
Artificial Intelligence
Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
Xi Chu and Yupeng Hou
Large
language models (LLMs) are becoming a major way for consumers to find products, but we do not yet understand how brands compete in this new channel. We study brand dynamics in LLM recommendations using skincare products – a category where consumers cannot easily judge quality before buying and must rely on brand reputation – across three commercial LLMs (GPT-4o-mini, Claude Sonnet, Gemini 3 Flash), with a robustness check on search goods. In three experiments, we find: (1) a Conditional Monopoly where well-known brands get recommended 100% of the time (IAI = 10.0) when all products have the same specifications, but this dominance disappears with less than a +0.1-
star rating advantage for a competitor; (2) authority-style marketing
language, including fabricated clinical-evidence claims, breaks this monopoly at a Bias Surplus Value equal to +0.17 rating points, with each model responding differently; and (3) a social dilemma in multi-brand GEO competition: when all brands adopt the same optimization strategy, individual payoff falls from +0.802 to +0.007 in our payoff proxy, and non-participating brands receive zero recommendations in our tests. Our results suggest that generative engine optimization (GEO) should be studied not only as a security risk, but also as an emerging marketing practice that shapes
market competition.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CV
EV9V: An Echocardiographic Video Benchmark with Uncertainty Guided View Classification
Bo Gou, Jicheng Zhang, Jianlong Xiong et al. (+10 more)
Automated classification of standard echocardiographic views is crucial for efficient clinical workflow but faces three main challenges. First, publicly available
datasets are scarce and limited in scale and view coverage. Second, the performance of some modern
video-level architectures for echocardiographic view classification remains underexplored. Third, some view categories exhibit highly similar spatial appearances, making single-frame features insufficient for discrimination, while heterogeneous frame quality complicates robust temporal information fusion. To address these challenges, we release the Echocardiographic
Videos of Nine Views (EV9V)
dataset, comprising 5,138
videos, 910,579 frames, and 9 standard views, which is, to the best of our knowledge, the largest openly released echocardiography
video dataset. Furthermore, we propose a Spatio-Temporal Fusion Model (STFM), an efficient dual-stream framework that jointly captures spatial anatomical structures and temporal cardiac dynamics. The STFM leverages uncertainty-aware learning to preferentially sample representative
video segments during training and evidence-based fusion during inference, improving robustness to variations in frame quality across echocardiographic
videos. Extensive experiments demonstrate that STFM achieves competitive performance and effectiveness compared with diverse
video classification architectures, including two-dimensional and three-dimensional convolutional neural networks, recurrent neural networks, and
Transformers. The code is released at https://huggingface.co/
datasets/bgx666/EV9V and the EV9V
dataset is available at https://github.com/bgx666/stfm.
Read the full abstract ▾
All authors Bo Gou, Jicheng Zhang, Jianlong Xiong, Tao He, Hai Wu, Bentian Liu, Yijiao Wang, Yu Zhang, Yujia Yang, Yun Dai, Wei Liu, Jian Liu, Jie Wang
Artificial Intelligence · Cross-listed from eess.IV
Feynman Kac Reweighted Schrödinger Bridge Matching for Surface-Based Tau PET Harmonization
Jianwei Zhang, Xinyu Nie, Jiaxin Yue et al. (+1 more)
Tau positron emission
tomography (PET) is widely used for the in vivo characterization of disease stage and progression in Alzheimer's disease (AD). With the adoption of multiple tau PET tracers including AV-1451, PI-2620, MK-6240 with different binding behaviors in various large-scale studies, there is a great need of effective harmonization methods to enable the cross-tracer integration of tau PET
datasets. While previous methods such as CenTauR were proposed to standardize scalar tau PET measures, they are limited in accounting for the heterogeneity of tau pathology. In this work, we propose Feynman-Kac Reweighted Schrödinger Bridge Matching (FKRSBM), a surface-based framework for cross-tracer tau PET harmonization. FKRSBM learns a direct
stochastic transport between tracer domains using Schrödinger Bridge matching, avoiding the Gaussian-prior routing used in
diffusion-based translation. To promote biologically consistent transport, FKRSBM introduces an endpoint penalty favoring bridge pairings with matched tau-pathology status and implements it through a Feynman-Kac reweighted endpoint proposal. To preserve cortical organization, FKRSBM uses a spherical convolutional network for vertex-level harmonization on cortical surface meshes. In our experiments, we demonstrate our method by harmonizing Tau PET images acquired with the AV-1451 (n=1480) and PI-2620 (n=2458) tracers from two large-scale
datasets. Compared to previous methods including ComBat, CycleGAN,
Diffusion Model(DF), and unregularized Schrödinger Bridge Model(DSBM), the proposed FKRSBM method outperforms these baselines in subgroup-level
alignment, tau-positivity consistency, and diagnostic classification while preserving subject-specific cortical topography of tau pathology. The code is available at: https://github.com/jianweizhang17/FKRSBM.
Read the full abstract ▾
All authors Jianwei Zhang, Xinyu Nie, Jiaxin Yue, Yonggang Shi
Artificial Intelligence · Cross-listed from cs.SD
L-Proto: Language-Aware Episodic Prototypical Training for Multilingual Speaker Verification
Hyung-Seok Oh, Deok-Hyeon Cho, Seung-Bin Kim et al. (+1 more)
Multilingual speaker verification remains challenging because
language-dependent acoustic variability causes speaker identity to become entangled with linguistic characteristics, degrading generalization across
languages. In
multilingual training,
embeddings often encode
language cues with speaker identity, causing speakers to form
language-specific clusters. We propose L-Proto, a
language-aware episodic prototypical training strategy that constructs
language-consistent episodes. By sampling speakers from a single
language per episode, L-Proto reduces
language-driven variation during training and encourages
embeddings to focus more directly on speaker identity. Experiments on the TidyVoice Challenge
benchmark demonstrate consistent performance improvements over conventional fine-tuning and random episodic sampling across multiple backbone architectures.
Read the full abstract ▾
All authors Hyung-Seok Oh, Deok-Hyeon Cho, Seung-Bin Kim, Seong-Whan Lee
Artificial Intelligence · Cross-listed from cs.CV
Enhancing Pathological VLMs with Cross-scale Reasoning
Chi Phan, Tianyi Zhang, Qiaochu Xue et al. (+5 more)
Pathological images are inherently multi-scale, requiring pathologists to integrate evidence from global tissue architecture at low magnification to cellular morphology at higher magnification for accurate diagnosis. While existing pathological
datasets for
vision-
language models (VLMs) include various scales, they often lack explicit cross-scale
reasoning objectives. This limitation prevents VLMs from capturing essential cross-scale representations and learning evidence-based
reasoning. To bridge this gap, we introduce the first cross-scale training and evaluation paradigm that formulates pathology interpretation as multi-magnification
reasoning. However, creating such a task reveals a critical challenge: multi-image visual question answering (VQA) is prone to text-only shortcuts, which allow models to guess answers using magnification-dependent artifacts rather than visual evidence. To address this, we propose a leakage-aware curation pipeline that combines
adversarial text-only screening with constraint-guided question design. Using this pipeline, we construct Scale-VQA, a high-quality
benchmark with 4,685 multiple-choice questions grounded in 2,537 pathology images across multiple magnification levels. Finally, we present ScaleReasoner-R1, a model trained via reinforcement learning to optimize performance on cross-scale VQA tasks. ScaleReasoner-R1 achieves state-of-the-art performance on our cross-scale
reasoning benchmark and generalizes to SOTA performance on established single-scale
benchmarks. Findings suggest that even the limited cross-scale supervision can significantly improve pathological understanding. Code is available at https://github.com/iMVR-PL/ScaleReasoner-R1.
Read the full abstract ▾
All authors Chi Phan, Tianyi Zhang, Qiaochu Xue, Yufeng Wu, Dan Hu, Zeyu Liu, Sudong Wang, Yueming Jin
Artificial Intelligence · Cross-listed from cs.LG
Discrete Autoregressive Transformer for Generative Mechanism Synthesis
Anar Nurizada and Anurag Purwar
Planar path synthesis requires mechanisms whose coupler curves match a prescribed trajectory; the mapping from curve to linkage is inherently one-to-many across four-, six-, and eight-bar
topologies. We address this design problem with simulation-grounded evaluation on a curated corpus of over one million mechanisms, reporting Chamfer distance and dynamic time warping after forward kinematics and
geometric alignment. We formulate synthesis as conditional autoregressive sequence modeling: joint coordinates are uniformly quantized to
tokens and generated by a decoder-only
transformer with a variational-autoencoder (VAE) latent of the target curve and an explicit mechanism-type
token. Training combines
token cross-
entropy with a Gaussian-smoothed bin auxiliary loss that respects ordinal structure among bins. At inference, a bounded latent-noise schedule decodes all mechanism types at each noise level; we retain the top five candidates by
geometric error, yielding diverse accurate families without
dataset lookup. On held-out tests, aggregate mean Chamfer distance is $0.0132$ and mean dynamic time warping is $0.153$; a latent $k$-nearest-neighbor baseline that conditions on training-set neighbor latents in VAE space achieves matched-
topology mean Chamfer distance $0.0071$ and mean dynamic time warping $0.117$ using the same decoder.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CV
Graph Neural Networks for Semi-Supervised Image Classification with Multi-Feature Aggregation
Marina Chagas Bulach Gapski, Vinicius Atsushi Sato Kawai, Gustavo Rosseto Leticio et al. (+3 more)
Feature extraction involves the identification and extraction of salient characteristics or patterns, including edges, textures, shapes, and color attributes. Contemporary feature extractors predominantly leverage deep learning architectures, such as Convolutional Neural Networks (CNNs) and
Vision Transformers (VITs). The availability of diverse feature extractors in the literature provides a wide range of feature representations. Features extracted from an image depend on the specific application, the chosen extractor, and its configuration. Therefore, integrating complementary information by combining distinct extractors offers a promising way to enhance performance. Graph Neural Networks (GNNs), particularly Graph Convolutional Networks (GCNs), have emerged as powerful and widely adopted approaches for semi-supervised image classification, as they effectively leverage both labeled and unlabeled data while exploiting the underlying graph structures that capture relationships among samples. This study proposes a novel approach for GNNs in scenarios where labeled data is scarce, by integrating diverse sets of feature and graph representations derived from various extractors in classification scenarios. Experimental investigations were conducted, encompassing combinations of distinct feature and graph extractors, as well as rank aggregation strategies. The primary contributions of this work are underscored by the experimental findings, which demonstrate that the strategic combination of feature and graph representations, coupled with the application of
manifold learning for graph processing, leads to significant improvements in classification accuracy across the majority of experimental conditions. Furthermore, the utilization of rank aggregation techniques to integrate features from different extractors was shown to enhance classification accuracy.
Read the full abstract ▾
All authors Marina Chagas Bulach Gapski, Vinicius Atsushi Sato Kawai, Gustavo Rosseto Leticio, Lucas Pascotti Valem, Daniel Carlos Guimarães Pedronette, Mohand Said Allili
Artificial Intelligence
Treatment Response Optimized Clinical Decision Support AI System via Digital Twin Simulation
Xinyu Qin, Anil K. Sood, Ruiheng Yu et al. (+3 more)
Clinical decision support AI systems (CDSASs) must adapt to evolving patient conditions in real-time while adhering to strict safety constraints. We present an online adaptive framework that integrates Treatment Effect (TE) estimation to quantify clinical benefits, a patient Digital Twin (DT) to simulate treatment trajectories, and Reinforcement Learning (RL) for sequential decision-making. The AI system is initially trained on historical medical records and operates in a continuous learning loop. To ensure safety, a rule-based module monitors vital signs and blocks contraindicated treatments. Cases with strong internal model disagreement are flagged for clinician review, simulated in our experiments via a pre-trained outcome model. We validate our framework using both a synthetic clinical simulator and a real-world ovarian cancer
dataset from The Cancer Genome Atlas (TCGA). In both simulated and clinical settings, our method demonstrated superior effectiveness and stability in recommending treatments compared to standard computational baselines. Furthermore, the AI system maintains low latency and requires expert consultation for only a minority of cases in our experimental validation, demonstrating its potential as a safe, clinician-supervised tool for personalized medicine that continuously improves through practical use.
Read the full abstract ▾
All authors Xinyu Qin, Anil K. Sood, Ruiheng Yu, Sara Corvigno, Elaine Stur, Lu Wang
Artificial Intelligence · Cross-listed from cs.CV
Bridging Spatial And Frequency Views For Disaster Assessment: Benefits And Limitations
Shikha V. Chandel, Yadav Raj Ghimire, Timothy Agboada et al. (+1 more)
Rapid assessment of building damage from satellite imagery is essential for effective disaster response and recovery. While most deep learning methods rely on spatial-domain features, frequency-domain representations can capture complementary structural cues such as debris patterns and collapse-induced textures. This study presents a controlled comparison of spatial-domain, frequency-domain, and dual-domain deep learning approaches for multi-class building damage classification using post-disaster imagery from the xView2 (xBD)
dataset. To ensure
fairness, all models are built on an EfficientNet-B0 backbone and trained under identical settings, differing only in their input representations and fusion strategies. Performance is evaluated using accuracy, macro F1-score, per-class metrics, and confusion
matrices. Results show that dual-domain models provide measurable improvements over single-domain approaches. The dual spatial configuration achieves the highest test accuracy (0.4688) and lowest loss, while the spatial-only model attains the best macro F1-score (0.4254), indicating more balanced class performance. In contrast, frequency-only models perform worst and exhibit overfitting, suggesting limited generalization. Despite these gains, all models struggle to detect subtle damage levels, particularly the Minor class, due to class imbalance and fine-grained visual ambiguity. While dual-domain approaches improve detection of severe damage, challenges remain. These findings highlight the benefits and limitations of hybrid representations and motivate future work on data balancing, advanced fusion, and regularization.
Read the full abstract ▾
All authors Shikha V. Chandel, Yadav Raj Ghimire, Timothy Agboada, Leila Hashemi-Beni
Artificial Intelligence · Cross-listed from cs.LG
The Discrete-Log Clock: How a Transformer Learns Modular Multiplication
Huu Danh Nguyen
When small
transformers grok modular multiplication, prior work reports that the learned
embedding has a "dense"
Fourier spectrum requiring all frequencies. This contrasts with modular addition, where only a sparse set of key frequencies suffices. We show this density is an artifact of analyzing in the wrong basis. The natural
Fourier transform for multiplication is not the standard additive DFT but the multiplicative character transform, which decomposes functions on the multiplicative group $(\mathbb{Z}/p\mathbb{Z})^*$ into its irreducible representations. Applying this transform to a grokked
transformer trained on $a \cdot b \bmod 113$, we find the
embedding spectrum becomes highly sparse (Gini coefficient 0.58 vs. 0.07 in the additive basis) with only 4 key frequencies carrying significant energy. Furthermore, 96.9% of MLP
neurons are cleanly tuned to a single multiplicative frequency, and
neuron activation heatmaps reveal 2D-periodic structure when reordered by the discrete logarithm. These results demonstrate the
transformer reduces multiplication to addition in discrete-log space, implementing a "Discrete-Log Clock"
algorithm analogous to Nanda et al.'s Clock
algorithm for addition. The methodology generalizes: matching the analysis basis to the algebraic structure of the task reveals interpretable structure where standard tools see noise.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CR
SoK: AI-Augmented Binary Reversing
Yujeong Kwon, Yiyue Zhang, Kexin Pei et al. (+2 more)
Binary reversing is fundamental to software understanding, vulnerability discovery, malware investigation, and firmware
auditing. However, it remains inherently challenging due to the lossy transformation of semantic information during compilation. Recent advances in machine learning, large
language models (LLMs), and agentic AI systems have accelerated the adoption of AI-augmented binary reversing. Yet, the resulting body of work has become increasingly fragmented across reversing domains, artifact representations, learning approaches, and evaluation practices. This paper presents the first comprehensive systematization of knowledge on AI-augmented binary reversing. We collect 246 research papers published since 2015, and organize them into 22 binary reversing domains according to the inference tasks. We further introduce a unified taxonomy spanning conventional and AI-augmented reversing pipelines. Our taxonomy connects traditional analysis techniques, binary-derived artifacts, representation strategies, learning paradigms, and downstream inference tasks, while clarifying the emerging roles of LLMs and agentic AI systems. By establishing a common vocabulary and structured framework, we offer a holistic view of the field's
evolution over the past decade. Our study reveals common structures underlying seemingly disparate approaches, highlights persistent technical challenges and evaluation gaps, and identifies promising opportunities for future research. Collectively, these insights clarify the current state of the field and provide a foundation for the next generation of evidence-grounded and practically deployable AI-augmented binary reversing systems.
Read the full abstract ▾
All authors Yujeong Kwon, Yiyue Zhang, Kexin Pei, Dokyung Song, Hyungjoon Koo
Artificial Intelligence · Cross-listed from cs.CL
NarrativeWorldBench: A Frontier-Saturated Benchmark and a Latent World Model for Long-Horizon Co-Creative Audio Drama
Logan Mann, Abdur Rahman, Mohammad Saifullah et al. (+2 more)
Long-form serialized audio drama, with arcs that run for 200 to 800 episodes, is a major creative medium and a setting where frontier large
language models (LLMs) fail. We
benchmark 21 models, spanning classical, fine-tuned, open-frontier, closed-frontier, and
reasoning tiers, on a uniform set of structural narrative metrics. All closed-frontier systems saturate at a plot-beat F1 in the band [0.78, 0.81] and collapse by about -0.20 F1 at horizon h=200. We introduce NarrativeWorldBench, an open
benchmark of nine narrative-structure metrics evaluated across horizons h in {10, 20, 50, 100, 200}, with cross-lingual evaluation across four Indic
languages (Hindi, Tamil, Telugu, Marathi). We introduce N-VSSM, a Narrative Variational State-Space Model that maintains a structured 256-dimensional latent world state over more than 200 episodes via a Mamba-2 backbone with an event-conditioned posterior and an 8B decoder. N-VSSM holds plot-beat F1 >= 0.84 across all horizons at 4x lower compute than the closed-frontier band. A learned Cultural Transfer Function lifts cross-
language fidelity by +0.20 to +0.23 Likert points. In a within-subjects writer study (n = 12 professional authors, 240 trials), N-VSSM is preferred over Claude Opus 4.5 on long-arc consistency 71% of the time and rated +1.3 Likert points higher on controllability.
Read the full abstract ▾
All authors Logan Mann, Abdur Rahman, Mohammad Saifullah, Taaha Kazi, Vasu Sharma
Artificial Intelligence · Cross-listed from cs.CV
Visuals Lie, Consistency Speaks: Disentangling Spatial Attention from Reliability in Vision-Language Models
Logan Mann, Yi Xia, Ajit Saravanan et al. (+6 more)
Multimodal Foundation Models are increasingly used as
reasoning agents, making reliability, knowing when a model may hallucinate, critical. A common intuition, which we call the
Attention-Confidence Assumption, holds that reliability follows from "structural" visual perception: tight
attention on relevant regions should signal a trustworthy answer, while scattered
attention signals confusion. We challenge this through the VLM Reliability Probe (VRP), a systematic cross-family study of reliability signals in contemporary
Vision-
Language Models (VLMs). We introduce structural-
attention metrics, cluster counts (C_k) and spatial
entropy (H_s), to quantify the visual encoder's gaze, and track its
evolution (Delta H_s) across layers. This reveals a "Symbolic Detachment": models often "Early Lock" visual features only to diffuse
attention later, severing early perception from final generation. Contrary to the grounding hypothesis, we find a "Cluster Failure": spatial
attention has near-zero correlation (R approx 0.001) with accuracy. Instead, reliability is a phenomenon of generation dynamics and internal-state distributions. Self-Consistency, the agreement rate across sampled
reasoning paths, is the dominant predictor of truth (R = 0.429). Scaling causal interventions exposes a sharp architectural divergence: LLaVA locks its prediction in a fragile late-stage bottleneck, whereas PaliGemma and Qwen2-VL distribute reliability globally, staying resilient even when ~50% or more of their most predictive layer is destroyed. For current VLMs, reliability signals are detached from visual grounding maps and are best inferred from generation-time dynamics and hidden-state probes.
Read the full abstract ▾
All authors Logan Mann, Yi Xia, Ajit Saravanan, Ishan Dave, Saadullah Ismail, Shikhar Shiromani, Emily Huang, Ruizhe Li, Kevin Zhu
Artificial Intelligence · Cross-listed from cs.CV
TerraTransfer: Learning End-to-End Driving Policies Without Expert Demonstrations
Zikang Xiong, Weixin Li, Zhouchonghao Wu et al. (+6 more)
End-to-end autonomous driving has achieved state-of-the-art performance on
benchmarks and real-world deployments. Its standard training recipe, however, is expensive across all stages: collecting and labeling millions of driving frames is costly, and closed-loop RL on images is bottlenecked by the per-step cost of photorealistic rendering plus a forward pass through a large
vision backbone. Self-play in vectorized simulators changes the economics: millions of rollout steps per second, and a state distribution naturally rich in collisions, near-misses, and recoveries that no driving log contains. Our approach exploits this asymmetry by decoupling learning to drive from learning to see. We pretrain a single policy by self-play, then align its latent space with a pretrained
vision backbone, through the action KL divergence and a batch-relational low-rank structural loss. The action target comes from the self-play policy, so
alignment never supervises against a logged trajectory: a paired
dataset of (image, scene-state) frames suffices, with no need for the curated expert demonstrations that imitation pretraining is built on. On photorealistic 3D Gaussian splatting closed-loop scenarios, the resulting end-to-end policy matches or exceeds prior end-to-end methods.
Read the full abstract ▾
All authors Zikang Xiong, Weixin Li, Zhouchonghao Wu, Akshay Rangesh, Saarth Bonde, Grantland Hall, Chen Tang, Yihan Hu, Wei Zhan
Artificial Intelligence · Cross-listed from q-fin.RM
Model Validation of Agentic AI Systems: A POMDP-Based Framework for Belief-State, Forecast, and Policy Validation
Matthew Francis Dixon
Agentic artificial intelligence systems introduce a new class of model risk. Unlike traditional predictive models, autonomous
agents continuously acquire information, form beliefs regarding latent states of the environment, generate forecasts, select actions, and adapt their behavior over time. Existing validation methodologies focus primarily on predictive accuracy and therefore provide limited insight into the quality of the underlying decision process. This paper proposes a model validation framework for agentic AI based on Partially Observable Markov Decision Processes (POMDPs). The framework decomposes autonomous decision making into information, beliefs, forecasts, actions, and utility, allowing each component to be validated independently. Large
language models (LLMs) are formalized as approximate
Bayesian filtering operators, and a model-risk taxonomy is developed encompassing state-space, filtering, forecast, policy, utility-specification, and parameter risks. The model risk validation methodology is demonstrated through a portfolio-management case study in which an
agent infers latent
market regimes from
market and macroeconomic information, generates belief-conditioned forecasts, and constructs portfolios using a Black–Litterman framework. Empirical validation combines performance analysis, belief calibration diagnostics, coverage tests, ablation studies, and parameter-sensitivity analysis. The results indicate that latent-state inference contributes independently to decision quality and that the principal conclusions remain robust across a broad range of parameter values. The principal contribution of the paper is a practical framework for extending established model risk management concepts to autonomous AI systems and providing a rigorous foundation for their validation, governance, and monitoring.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CV
MeiBRD: Meta-Learning Intraoperative Biomechanical Residual Deformation
Casey Meisenzahl, Jon Heiselman, Michael Holtz et al. (+3 more)
Accurate intraoperative liver registration is challenging due to substantial soft-tissue deformation yet sparse intraoperative measurements. Biomechanical models regularize this ill-posedness with prior knowledge but exhibit persistent prediction bias due to simplifying assumptions, while data-driven learning solutions struggle with data efficiency, generalization, and physical plausibility. We propose a hybrid registration framework that adapts a biomechanical prior using sparse intraoperative correspondences. Rather than learning a full deformation field, we learn a residual deformation function that corrects linear biomechanical predictions, modeled as a graph neural
diffusion function with geometry-aware
attention over the 3D liver mesh. To enable long-range information transfer of sparse observations, we take a novel perspective of sparse intraoperative measurements as context samples where input-output pairs of the residual deformation function are fully observed, casting the problem into learning-to-learn this residual function from intraoperative context samples with feedforward meta-learners. Experiments on a deformable liver phantom
dataset demonstrate improved registration accuracy and generalization compared to rigid, biomechanical, and data-driven baselines, particularly for out-of-distribution geometries and deformations.
Read the full abstract ▾
All authors Casey Meisenzahl, Jon Heiselman, Michael Holtz, Yubo Ye, Michael Miga, Linwei Wang
Artificial Intelligence · Cross-listed from cs.CL
Implicit vs. Explicit Prompting Strategies for LVLMs in Referential Communication
Peter Zeng, Amie J. Paige, Weiling Li et al. (+3 more)
Two recent studies \citep{jones2026llms, zeng2026lvlms} reach apparently contradictory conclusions about whether large
vision-
language models (LVLMs) can coordinate similarly to humans on efficient referring expressions. We control for task differences between the studies while directly comparing their prompting styles. We replicate the finding that models can coordinate efficient referring expressions when explicitly prompted to do so, suggesting that other task differences are not responsible for divergent results. However, we also find that the same models fail to infer the need for communicative efficiency from a more implicit
prompt, highlighting critical differences between how humans and AI systems communicate.
Read the full abstract ▾
All authors Peter Zeng, Amie J. Paige, Weiling Li, Susan E. Brennan, Owen Rambow, Cameron R. Jones