Computation and Language · Cross-listed from cs.AI
Natural Language Input, Semantic Track Representation, and LLM Inference: Making the Maritime Information Exchange Model Tractable
Frederick Roth
We describe a practical architecture for making the Maritime Information Exchange Model (MIEM) and the broader Rich Semantic Track model tractable using current large
language model (LLM) technology. The barrier to adoption of semantic track models in defense and law enforcement has been the requirement that operators learn formal ontology
languages and manually encode observations as typed logical assertions. We propose eliminating this barrier entirely: operators contribute observations in natural
language; an LLM translates these into typed Semantic Assertion Records (SARs), which are named case frames that capture n-ary relations in a single compact structure; a knowledge graph accumulates the SARs; and a second LLM pass performs inference, anomaly detection, and hypothesis ranking over the graph. We work through two detailed examples (a 9/11-era pre-attack indicator scenario and a maritime cargo inspection scenario) showing the full pipeline from natural
language input to SAR representation to inference output. We argue that this architecture makes the Track Model and MIEM immediately deployable with current technology, establishes prior art against proprietary enclosure of the approach, and grounds the method in a theoretical framework connecting semantic track representations to neural
manifold geometry.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
Diagnostic Foundation for Evaluating LLMs' Research Integrity as Co-Scientists
Yash Tripathi, Silu Sharma, Sai Sidhanth Manoharan Jayanthi et al. (+2 more)
Language models are increasingly deployed as co-scientists, yet their ability to uphold research integrity under institutional pressure remains unmeasured. We introduce IntegrityBench, a
benchmark evaluating misconduct classification, ethical action
reasoning and artifact-grounded decision making across 36 paired tasks under a 5-level implicit-explicit pressure protocol spanning 3 domains and 4 research stages. Evaluating 18 frontier model variants, we find that under peak pressure, models fail roughly 1 in 3 integrity-critical decisions, and neither scale nor
reasoning ability reliably mitigates this. Explicit pressures induce compliance with misconduct, while implicit contextual reframing more often causes over-refusal of legitimate research tasks. Interestingly, models failing to classify research requests accurately perform equally or better on artifact-grounded decision making (85.7 vs. 79.4), suggesting the three facets are structurally dissociated and correct ethical action does not require accurate classification. Frontier models can thus appear helpful while harbouring integrity failures that create two distinct deployment risks: facilitating research misconduct and eroding trust in AI-assisted research.
Read the full abstract ▾
All authors Yash Tripathi, Silu Sharma, Sai Sidhanth Manoharan Jayanthi, Shivank Garg, Lin Li
Computation and Language
Predicting consumer-technology ownership without a diffusion history
Irina Vartanova, Niels Selling, Jennifer Viberg Johansson et al. (+1 more)
We test whether the perceived attributes of a consumer technology predict how widely it is owned. In a 2022 Prolific survey of US adults (n = 678), respondents rated 65 consumer technologies on six attributes. We then elicited the same ratings from two frontier
language models, Anthropic Claude Opus 4.7 and OpenAI GPT-5.5. We regress ownership prevalence on four UTAUT2 acceptance attributes plus a log-age
covariate with a sign-constrained penalized regression and evaluate it by holding out one technology at a time. The attribute model improves on a baseline of years-since-launch: mean absolute error falls by 17% with the human ratings, and by more with either model, most with Opus 4.7. Over the short 2022-to-2025 window, where ownership moved little, the same attributes do not improve on a no-change baseline. We set out the limitations of the approach, including the possibility that
language-model ratings reflect prior knowledge of these technologies rather than independent attribute
reasoning. We include a deployment illustration: 2027 ownership predictions for eleven products launched in 2025 and 2026.
Read the full abstract ▾
All authors Irina Vartanova, Niels Selling, Jennifer Viberg Johansson, Pontus Strimling
Computation and Language
Lost in Historical Time? A Polish History Matura Benchmark for Large Language Models
Adrian Trzoss, Kacper Dudzic, Wiktor Werner et al. (+1 more)
Language models are widely used by students as knowledge sources, yet
benchmarks rarely assess their interpretative historical
reasoning. We evaluate eight leading LLMs on the Polish high school exit exam (Matura) in history - three official papers from 2023-2025, comprising short-answer questions and extended essays - and compare model performance against the human examinee population. Although models score near the ceiling, aggregate scores mask distinct competency profiles: rankings are unstable across task types, source modalities, and geographical scopes, with a consistent penalty for Polish versus Global history content. Qualitative error analysis reveals two recurring failure modes - source decontextualization, when models reason from source content rather than treating it as an object of analysis, and temporal disorientation, when responses are historically misplaced. This study introduces the first LLM history
benchmark grounded in the Polish national curriculum.
Read the full abstract ▾
All authors Adrian Trzoss, Kacper Dudzic, Wiktor Werner, Marcin Moskalewicz
Computation and Language
Are Large Language Models Reliable Reviewers? A Benchmark for Error Detection in Financial Documents
Ying He, Zhouhong Gu, Zhecheng Hu et al. (+8 more)
Ensuring the accuracy of financial documents is critical for economic analysis, regulatory compliance, and corporate decision-making. Several studies have shown that Large
Language Models (LLMs) perform well in many financial tasks, such as stock price movements and financial analytics. However, a critical task remains unexplored: the ability of LLMs to identify errors in financial documents. In this paper, we introduce FinED-Bench, the first publicly
Benchmark for Financial Error Detection across three levels of cognitive complexity. FinED-Bench covers nine real-world financial scenarios, and includes over 900 documents reported in 2025 that are unseen by existing
language models. We detail the
benchmark construction process and evaluate several advanced LLMs (e.g., GPT-4o, Qwen3-14B) on this tasks, which requires both financial domain knowledge and
reasoning capabilities. Experimental results show that current LLMs still struggle with this task, especially in high-complexity cases. Besides, supervised fine-tuning can significantly improve the performance of weaker LLMs on this task. Our data and code are available at https://github.com/hedyHe/FinED-Bench.
Read the full abstract ▾
All authors Ying He, Zhouhong Gu, Zhecheng Hu, Yubo Zhou, Hao Shen, Jiaqing Liang, Zhaoqian Dai, Shuguang Ma, Fei Yu, Yanghua Xiao, Zhixu Li
Computation and Language
The "Knowledge-Behavior Gap" in Cultural Taboo Safety of Large Language Models
Ying He, Sihang Jiang, Xingzhou Chen et al. (+6 more)
Cultural taboo safety is essential for deploying large
language models (LLMs), as culturally insensitive outputs may cause offense or even social harm. However, existing cultural
benchmarks primarily assess cultural knowledge or values biases, while overlooking whether LLMs can recognize and respect cultural taboos, especially when taboos are implicitly hidden in seemingly harmless questions. Besides, cultural taboos are implicit, and context-dependent, thus poss unique challenges for reliable evaluation. To address these gaps, we introduce CulShield, the first public
benchmark dedicated to evaluating and improving the cultural taboo safety of LLMs. CulShield spans 77 countries and territories, and includes over 2,020 taboos. It evaluates models along both explicit knowledge and implicit behaviors. Experiments on several advanced LLMs (e.g., GPT-4o-mini, Gemini-2.5-pro) reveal a clear “knowledge-behavior gap”: models often fail to apply known taboos during interaction. We further show that variations in linguistic context can significantly affect LLMs' cultural taboo safety. Code and data is accessible here: https://github.com/hedyHe/CulShield.
Read the full abstract ▾
All authors Ying He, Sihang Jiang, Xingzhou Chen, Zhouhong Gu, Yiwei Gu, Minggui He, Shimin Tao, Hongxia Ma, Yanghua Xiao
Computation and Language
Class-Structure Preservation Beats Diversity: A Comprehensive Benchmark of Text Augmentation Methods for Imbalanced Text Classification
Keito Inoshita
With the rapid advancement of large
language models (LLMs), generative data augmentation has attracted considerable
attention for imbalanced text classification in natural
language processing. However, no empirical
benchmark to date has compared LLM-based augmentation against the
embedding-space SMOTE-style
retrieval (EmbSMOTE), a strong classical reference for imbalanced classification. In this study, a controlled
benchmark of 11 augmentation methods, spanning classical perturbation,
embedding-space
retrieval, and LLM-based generation, is newly constructed on seven public text classification
datasets covering class counts $K=2$-$28$ and imbalance ratios of 1.1 to over 500, evaluated with five random seeds per
cell using macro F1, Welch's $t$-tests, five distributional metrics, and an LLM-family sensitivity analysis based on Qwen3-8B. The experimental results reveal that all LLM-based methods are statistically equivalent or inferior to EmbSMOTE, with the performance gap widening monotonically as imbalance increases and reaching $\Delta\text{F1}_\text{macro}\!\approx\!0.063$ on GoEmotions-28. Furthermore, it is observed that surface-level uniqueness has negligible correlation with downstream performance, whereas LLM-specific artifacts, such as text elongation and label-distribution uniformization, are negatively associated with classification accuracy. Compared with six LLM-based and four classical augmentation baselines, these results demonstrate that the effective variable is not surface-level diversity but class-conditional structural fidelity, namely the degree to which augmented samples preserve the class-conditioned geometry of the training distribution. Accordingly,
retrieval-based oversampling should be adopted as the default for imbalanced multi-class classification, and a higher empirical bar should be required before LLM-based augmentation is deployed in practice.
Read the full abstract ▾
Computation and Language
Mimicry without understanding: the origins of decision bias in large language models
Eldad Yechiam and Adi Tarabeih
Large
Language models (LLMs) were found to be susceptible to a host of social, affective, and cognitive biases. We examined two mechanisms through which such biases can be generated even when human preferences (in the training data) are not biased or when they are correctly categorized as being biased. The first is faulty mimicry of preferences based on human behavior: this involves LLMs inferring human preferences even when behaviors are logically unrelated to preferences. The second is mimicry of explicitly biased human behaviors. In four studies focusing on economic biases, we find that ChatGPT-4o and Qwen exhibited social proof biases even when prompted with reports of human behaviors that were clearly non-indicative of individuals' actual preferences. LLMs also displayed loss aversion when it was explicitly described as a bias. Indeed, when prompted with detailed scientific reports, the extent of the bias (i.e., loss aversion) in the scientific report predicted LLMs' own subsequent bias. Scientific papers of biases can thus become self-fulfilling prophecies, at least when it comes to LLMs' responses. The current study goes beyond fleshing out LLM biases and sheds light on the underlying component processes.
Read the full abstract ▾
Computation and Language
SDAM: Structure-Difference-Aware Memory Evolution for Complex Text-to-SQL
Keyan Xu, Dingzirui Wang, Xuanliang Zhang et al. (+2 more)
Text-to-SQL aims to convert natural
language questions into executable SQL queries. While
memory-based
agent system improves complex SQL generation, existing
memory design neglect historical experience and suffer from weak structure analysis, shallow semantic understanding, and poor schema
alignment. To address these challenges, we propose SDAM. Specifically, SDAM identifies potential errors via a structure-difference aware
reasoning tree, extracts deep semantic rules through contradiction-aware reflection, and enhances structural consistency using a schema-grounded
memory evolution mechanism to bind
memory with database schemas. We integrate SDAM into a Text-to-SQL framework named SDAM-SQL. Experiment shows that SDAM-SQL achieves 2.0 and 0.4 improvement on BIRD-dev and Spider-test compared with mainstream Text-to-SQL methods, showing the effectiveness of SDAM-SQL.
Read the full abstract ▾
All authors Keyan Xu, Dingzirui Wang, Xuanliang Zhang, Qingfu Zhu, Wanxiang Che
Computation and Language
From Refuse to Richness: Rubric Rewards for Long-Form Hallucination Reinforcement Learning
Yudong Wang, Zhe Yang, Wenhan Ma et al. (+6 more)
Rewards that penalize unsupported claims can improve grounding in long-form generation, but they can also teach models to answer less. We study this refusal-to-richness trade-off in long-form
hallucination RL. Instead of using global richness proxies such as length, claim count, detail, or pairwise relevance, we represent each question with a key-point rubric that specifies the required and optional information a useful answer should cover. These rubrics define coverage directly and are used both for evaluation and as reward signals. Across grounding-only, proxy-based, rubric-only, and combined rewards, we find a stable trade-off: strict grounding rewards improve support but suppress coverage, while unconstrained rubric rewards improve coverage but weaken grounding. A soft combination of grounding, rubric coverage, and relevance gives the best balance in our experiments, improving in-distribution support while transferring better to out-of-distribution checklist tasks than either grounding-only or rubric-only rewards.
Read the full abstract ▾
All authors Yudong Wang, Zhe Yang, Wenhan Ma, Rang Li, Qibin Yang, Weimin Xiong, Jiangshan Duo, Liang Zhao, Zhifang Sui
Computation and Language
StorySpark: Module-wise Evolutionary Search for Story Premise Generation
Yang Yang, Zining Zhong, Qian Cao et al. (+5 more)
A story premise is the creative spark from which a full narrative can grow. Yet LLM-based story generation has mostly emphasized later-stage planning, controllability, coherence, and prose expansion, while premise-level ideation remains comparatively underexplored. We introduce StorySpark, a module-wise evolutionary search framework for story premise generation. StorySpark operates over interpretable narrative modules such as background, persona, event, ending, and twist, treating each active module not as a static field to fill once, but as a local search space conditioned on the partial premise built so far. For each module, it generates alternatives, evaluates them in context, refines them through feedback-driven mutation and recombination, preserves complementary strengths with Pareto-guided selection, and reallocates frontier capacity to balance branch coverage with promising directions. Multi-view automatic and human evaluations show that StorySpark produces stronger final premises than competitive baselines, with especially consistent gains in originality; when expanded with the same story writer, its premises also lead to higher-quality downstream stories while maintaining completeness, fascination, and diverse usable narrative directions.
Read the full abstract ▾
All authors Yang Yang, Zining Zhong, Qian Cao, Jindong Li, Boyun Xu, Kaishen Yuan, Menglin Yang, Yutao Yue
Computation and Language
HC-RAG: Evidence-Centric Retrieval-Augmented Generation over Heterogeneous Financial Filings
Siyuan Chen, Huaye Tan, You Li et al. (+1 more)
Financial question answering over annual reports requires more than retrieving semantically similar passages. It often involves identifying relevant companies and fiscal years, locating standardized filing sections, collecting textual and tabular evidence, and checking answers against the original documents. Existing RAG systems, however, usually flatten long filings into unordered chunks, pay limited
attention to the typed structure of financial reports, and use fixed text-table fusion strategies without considering query intent. To address these limitations, we propose HC-RAG, a hierarchical cross-modal
retrieval-augmented generation framework for evidence-centric financial QA. HC-RAG organizes filings into a typed financial evidence graph with documents, sections, text units, table units, and metadata nodes. It retrieves evidence through document-section-unit paths, aligns textual and tabular evidence in a shared
retrieval space, and routes evidence according to four semantic intents: calculation, trend, fact, and comparison. We further introduce Multi-Doc-2025, a
benchmark containing 2,327 expert-verified QA pairs from 179 SEC 10-K filings of 87 S&P 500 companies across fiscal years 2022–2024, with labels for intent, difficulty, and structural evidence attributes. Experiments on public financial QA
benchmarks and Multi-Doc-2025 show that HC-RAG improves both answer quality and evidence localization, especially in long-document, table-related, and cross-document settings. HC-RAG outperforms RAPTOR by 6.6 F1 points on DocFinQA and GraphRAG by 10.9 F1 points on Multi-Doc-2025. Evidence-level analysis and ablation studies show that the improvements mainly come from more accurate section localization, table grounding, cross-document evidence aggregation, and intent-aware text-table routing.
Read the full abstract ▾
All authors Siyuan Chen, Huaye Tan, You Li, Jiajun Liang
Computation and Language
Stateful Guardrails for Multi-Turn LLM Systems: A Conversational Risk Accumulation Framework
Sanjay Mishra, Divya Chukkapalli and Ganesh R. Naik
Most safety guardrails for large
language models (LLMs) evaluate each
prompt-response pair in isolation, which misses failures that arise only over a dialogue as benign turns compose into harm. We term this Conversational Risk Accumulation (CRA): gradual intent drift, fragmented assembly of prohibited instructions, and sensitivity build-up from repeated disclosures. We propose a session-layer CRA Framework that tracks three trajectory signals: semantic drift from a session anchor, a sensitivity-weighted information accumulation graph over extracted entities, and a compliance-gradient signal capturing increasing willingness to comply. For scoring, we provide (i) an unsupervised convex fusion for attribution and ablations, and (ii) CRA-Net DA, a compact learned trajectory model trained with family-
adversarial objectives to reduce length and topic-coverage confounds. To
benchmark CRA, we release CRA-Bench v0.1 (1,200 eight-turn sessions across three threat families with topic-matched benign twins), CRA-Bench v0.2 (LLM-paraphrased variants to reduce template artifacts), and an extended 5-family set (2,000 sessions adding persona priming and context stuffing). We introduce a trajectory-native evaluation protocol with session-level splits, mixed-set threshold calibration, Trajectory AUROC, turns-to-detection, calibrated false-positive metrics, bootstrap confidence intervals, leave-one-family-out diagnostic stress tests, and synthetic-to-human transfer checks. Claims focus on within-distribution session scoring on CRA-Bench and human-transfer subsets.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
ProHiFlo: Hierarchical Flow Matching with Functional Guidance for De Novo Protein Generation
Chuanzhen Wang, Meade Cleti and Pete Jano
De novo
protein generation has transformative potential in therapeutic design, enzyme engineering, and synthetic biology. While
diffusion-based and flow matching approaches have achieved progress, they typically operate at single resolution and lack mechanisms for incorporating functional constraints. We introduce ProHiFlo, a hierarchical flow matching framework with three innovations: (1) coarse-to-fine generation that models backbone geometry before refining to all-atom coordinates, reducing computational cost while maintaining accuracy; (2) functional guidance leveraging pretrained predictors to steer generation toward desired properties without retraining; (3) adaptive SE(3)-equivariant architecture for efficient multi-scale processing. Experiments on unconditional generation, motif scaffolding, and functional design demonstrate state-ofthe-art performance while requiring 4 fewer sampling steps. On enzyme active site scaffolding, ProHiFlo achieves 58.9% success rate compared to 41.2% for RFDiffusion.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
PreAct-Bench: Benchmarking Predictive Monitoring in LLMs
Hainiu Xu, Italo Luis da Silva, Jiangnan Ye et al. (+7 more)
Large
language models (LLMs) are increasingly deployed as autonomous
agents capable of executing multi-step action trajectories toward a given objective. While existing safety research has focused on detecting unethical behavior from complete trajectories, this paradigm is fundamentally retrospective: it identifies harm only after it has already occurred. In this work, we study a critical yet overlooked safety task, which we term Predictive Monitoring: given only a partial action trajectory, can a model infer whether it will culminate in an unethical action before the overt action is executed? To support this task, we present PreActBench, a
benchmark of 1,000 paired ethical and unethical action trajectories spanning five domains. We evaluate a range of LLMs, safety guardrail models, and latent probing methods across varying fractions of the action trajectory using our Prefix Foresight F1 metric. Results show that while humans achieve promising performance, predictive monitoring remains challenging even for strong models, highlighting the need for future-oriented risk
reasoning in LLM safety.
Read the full abstract ▾
All authors Hainiu Xu, Italo Luis da Silva, Jiangnan Ye, Yuhao Wang, Wei Liu, Linyi Yang, Jonathan Richard Schwarz, Nicola Paoletti, Yulan He, Hanqi Yan
Computation and Language · Cross-listed from cs.LG
SocraticPO: Policy Optimization via Interactive Guidance
Zirui Liu, Tingyue Pan, Jie Ouyang et al. (+8 more)
Reinforcement learning (RL) for large
language models usually supervises
reasoning with scalar outcome rewards, such as binary correctness. Such rewards provide an optimization direction but rarely explain how a model should revise its mistaken
reasoning, which can encourage shortcut learning and brittle policies. We propose SocraticPO (Socratic Policy Optimization), a policy-optimization framework that augments RL rollouts with Socratic-style natural-
language guidance. During rollout, the student first answers independently; if the answer is incorrect, a teacher diagnoses the attempt and provides concise corrective guidance, after which the student continues under the expanded context. Crucially, this guidance is paired with reward decay: correct answers obtained after teacher intervention only receive decayed rewards, preventing the policy from treating teacher help as a free path to reward. Since SocraticPO only modifies the rollout process while leaving the standard expected-reward objective intact, it can be plugged into existing policy-gradient backends such as Reinforce++. Moreover, because the teacher provides only text-level guidance, SocraticPO can leverage stronger black-box teacher models without requiring access to logits or distribution matching. On undergraduate-level scientific
reasoning benchmarks from SciKnowEval, SocraticPO improves over strong RL and self-
distillation baselines. Ablations show that both targeted guidance and reward decay are necessary, with reward decay mitigating reliance on assisted correction.
Read the full abstract ▾
All authors Zirui Liu, Tingyue Pan, Jie Ouyang, Qi Liu, Xianquan Wang, Jiayu Liu, Qingchuan Li, Jing Sha, Zhenya Huang, Shijin Wang, Enhong Chen
Computation and Language · Cross-listed from cs.LG
Streaming Knowledge Compilation: Proactive Materiality-Scored Pinning for Time-Evolving LLM Wikis
Juan M. Huerta
LLM wiki systems compile knowledge into pre-filled KV caches for efficient inference, but assume a static corpus – an assumption that fails whenever the underlying information landscape evolves. We formalize Streaming Knowledge Compilation: given a document stream, a fixed
token budget, and future queries unknown at ingestion time, maintain a compiled wiki that minimizes cumulative regret against an offline oracle with perfect foresight. The enabling insight is a materiality signal $\phi_t(k,n)\in[0,1]$ that scores document importance for entity $k$ at time $t$, acting as a query-relevance surrogate for proactive pinning before queries arrive; we prove an $O(\sqrt{T\log K})$ regret bound where $\varepsilon=\mathbb{E}[|\phi_t-\hat\phi_t|]$ is the only domain-specific quantity. We instantiate in two domains: finance, where $\phi_t$ is abnormal stock
volatility predicted by frozen Llama 3.1 8B classification head (AUROC = 0.728 on 76K articles, strict temporal split; $1.49\times$ higher realized forward
volatility for predicted-material articles); and Wikipedia, where $\phi_t$ is the Abnormal Edit Ratio (AER), a cross-sectionally normalized edit velocity – showing the same
algorithm generalizes beyond the finance domain. End-to-end QA evaluation on 173 matched pairs (finance) and 119 (Wikipedia) reveals a pervasive LLM-as-judge confound on post-training knowledge, establishing that regret analysis – not absolute QA scores – is the reliable evaluation metric for compiled knowledge systems. Finance cumulative regret converges to -20.0 (-0.12/step); Wikipedia to +16.0 (+0.13/step), with the positive sign confirming that Wikipedia edit content is genuinely post-training – richer context consistently improves scores (No Wiki 3.80 vs. Oracle 4.74) – and eliminates this confound. The $O(\sqrt{T\log K})$ guarantee applies to any domain where knowledge gaps can be predicted from streaming signals.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
Position: Don't Just "Fix it in Post": A Science of AI Must Study Training Dynamics
Stella Biderman, Mohammad Aflah Khan, Niloofar Mireshghallah et al. (+3 more)
What would it mean to have a scientific understanding of AI? Models are not static objects: they are snapshots of time-evolving processes shaped by data, objectives, architectures, and optimization dynamics. Yet much of AI research treats models as fixed artifacts, analyzing behaviors after training rather than asking why they emerge. This position paper argues that a science of AI must move beyond post-hoc fixes and study the training dynamics that produce model behavior. Such a science should support progressively stronger forms of understanding: predicting outcomes from early training signals, intervening when trajectories go wrong, and ultimately designing training procedures that more reliably produce desired properties. Scaling laws have made prediction routine for loss; the challenge is extending this success to capabilities, biases, robustness, and safety-relevant behaviors. We articulate requirements for such theories grounded in the history and philosophy of science, examine progress in mechanistic interpretability,
fairness, memorization, and simplicity bias, and identify concrete open problems.
Read the full abstract ▾
All authors Stella Biderman, Mohammad Aflah Khan, Niloofar Mireshghallah, Catherine Arnett, Fazl Barez, Naomi Saphra
Computation and Language · Cross-listed from cs.AI
EpiEvolve: Self-Evolving Agents for Streaming Pandemic Forecasting under Regime Shifts
Yiming Lu, Sihang Zeng, Zhengxu Tang et al. (+3 more)
Epidemic LLM forecasters are usually trained and evaluated as static supervised models, whereas operational pandemic
forecasting is a streaming process in which labels arrive after predictions and disease regimes shift over time. We study this mismatch in weekly COVID-19 hospitalization trend
forecasting across five variant regimes. We introduce EpiEvolve, a self-evolving
agent that wraps an LLM forecaster trained on the warm-start period and keeps its weights fixed during streaming. EpiEvolve adapts by storing forecast outcomes in a hierarchical episodic
memory, reflecting on delayed labels, retrieving cases relevant to the current regime, and distilling recurring errors into strategic rules. The resulting context lets the forecaster reuse its own past predictions and outcomes in later weeks while following a chronological protocol that prevents future leakage. On the streaming
dataset, EpiEvolve reaches $0.629$ average accuracy, compared with $0.561$ for the static backbone and $0.325$ for the external CDC ensemble, and reduces recovery lag after regime shifts from $5$ to $2$ weeks. Ablations show that reflection, strategic
memory, and regime-aware
retrieval each contribute to the gains.
Read the full abstract ▾
All authors Yiming Lu, Sihang Zeng, Zhengxu Tang, Max Lau, Fei Liu, Wei Jin
Computation and Language · Cross-listed from cs.AI
Severity-Aware Curriculum Learning with Multi-Model Response Selection for Medical Text Generation
Ahmed Alansary, Molham Mohamed and Ali Hamdi
Telehealth systems have become increasingly important for delivering accessible and timely medical information. Existing large
language models often struggle to provide consistent and contextually appropriate medical responses across varying levels of case severity. This limitation highlights the need for models that can effectively adapt to the progressive complexity in medical queries. To address this challenge, we introduce a severity-aware multi-model framework that integrates curriculum training strategy with relevance-based response selection. The proposed framework employs a three-stage curriculum learning strategy, where each model is trained sequentially on mild, moderate, and critical cases to progressively acquire domain knowledge. The approach uses five large
language models, each trained independently under the same curriculum. During inference, all models generate candidate responses, and the response with highest BERTScore is selected as the final output. The framework is trained and evaluated on the MAQA
dataset, which provides annotated medical question-answer pairs. Experimental results evaluated using BERTScore demonstrate that the proposed method achieves superior performance compared to both baseline and fine-tuned models, attaining 86.71% in the baseline setting and 90.30% after fine-tuning. These results highlight the effectiveness of combining curriculum learning with multi-model response selection in improving response quality and relevance in medical text generation.
Read the full abstract ▾
Computation and Language
A Multi-Model Metric-based Selection Framework for Abstractive Text summarization
Ahmed Alansary and Ali Hamdi
Automatic text summarization has become increasingly important due to the rapid growth of digital textual information. This paper presents a Multi-Model Summarization Framework designed to improve the robustness and quality of abstractive text summarization. Relying on a single model often leads to inconsistent summarization quality across articles with varying structures and topics. To address this limitation, the proposed framework integrates multiple fine-tuned
transformer-based summarization models and introduces a metric-based selection mechanism. In this framework, each model independently generates a candidate summary for the same input article. The generated summaries are then evaluated using automatic evaluation metrics that capture both lexical similarity and semantic relevance. Based on these scores, the framework selects the highest-quality summary as the final output. The models are fine-tuned and evaluated on the widely used CNN/DailyMail news summarization
dataset. Experimental results demonstrate that the proposed framework achieves the highest BERTScore among all compared methods with a score of 88.63%. It also outperforms several LLMs such as GPT3-D2, Falcon-7b, and Mpt-7b, highlighting its effectiveness and robustness. These findings highlight the effectiveness of leveraging multiple
transformer-based models within a metric-based selection strategy to improve the quality and robustness of automatic text summarization systems.
Read the full abstract ▾
Computation and Language
Localizing Prompt Ambiguity in Large Language Models with Probe-Targeted Attribution
Govind Ramesh, Yao Dou and Wei Xu
Prompt ambiguity is a common source of failure in large
language models, but is difficult to localize because it is a latent property of the
prompt, while existing attribution methods are designed to explain observable outputs such as logits or generated
tokens. We introduce PRIG, a gradient attribution method that uses a probe logit to attribute latent ambiguity to
token positions. Specifically, PRIG trains a linear probe to distinguish clear
prompts from ambiguous
prompts and attributes the probe score to earlier
token representations in the residual stream. To enable
token-level evaluation, we construct synthetic ambiguity
datasets across coding, math, and writing by rewriting one task-critical sentence per
prompt, and complement them with a human-written gold
benchmark. In this setting, PRIG localizes ambiguous spans substantially better than gradient attribution baselines, achieving 0.840 AUROC on the combined synthetic
benchmark and 0.891 AUROC on the gold set. It also outperforms GPT-5.4 on sentence-level ambiguity identification and retains useful signal out-of-domain. These results establish PRIG as a practical tool for identifying which parts of a
prompt are ambiguous. More broadly, they suggest that latent
prompt properties can be localized through intermediate representations, rather than through output-level attribution.
Read the full abstract ▾
Computation and Language
Multilingual Coreference Resolution via Cycle-Consistent Machine Translation
Adriana-Valentina Costache, Eduard Poesina, Silviu-Florin Gheorghe et al. (+2 more)
Coreference resolution is a core NLP task, having a broad range of downstream applications, e.g.~machine translation, question answering, document summarization, etc. While the task is well-studied in English, comparatively less
attention is dedicated to coreference resolution in other
languages, especially low-resource ones. To mitigate this gap, we propose a novel coreference resolution pipeline that harnesses machine translation (MT) from English to a target low-resource
language, to generate or expand training data. To automatically validate the quality of the translated samples, we back-translate the samples and assess the similarity with the original English samples via cosine similarity in the latent space of a BERT model. The resulting similarity scores are integrated into the loss function to weight training samples according to their MT cycle consistency. Extensive experiments on four low-resource
languages show that our pipeline brings significant performance gains in coreference resolution. Moreover, our pipeline enables accurate coreference resolution in
languages where no previous corpora were available.
Read the full abstract ▾
All authors Adriana-Valentina Costache, Eduard Poesina, Silviu-Florin Gheorghe, Paul Irofti, Radu Tudor Ionescu