Artificial Intelligence
StraTune: Adaptive Selection of Revision Operators for Self-Evolving LLM Skills
Zeping Liu, Yan Li, Ni Lao et al. (+2 more)
Large
language models (LLMs) can learn reusable textual skills from execution feedback without updating their parameters, but effectively deciding how to revise these skills remains a key challenge. Existing methods typically rely on a fixed revision operator, a search strategy and the revision forms applied under it. However, we observe that no single revision operator consistently performs best across tasks, and repeatedly applying an unsuitable operator can limit further improvement. We propose StraTune (strategy-guided skill tuning), which lets a frozen optimizer LLM choose the revision operator at every round from the optimization state, which is defined as the current execution feedback together with the recorded outcomes of earlier strategies and forms. Candidate skills from every revision operator pass one candidate evaluation, which screens for gains and regressions on a small sample set and validates them on a larger one, and every outcome is written back to the optimization state for later choices. Across four
benchmarks and two LLM settings, StraTune outperforms all five baselines in most settings. Ablations attribute the gains to the adaptive choice of the revision operator, since fixed, random, scheduled, and bandit strategy choices all score lower, and skills learned with a small target LLM also improve a stronger one. Code and learned skills are available at https://github.com/seai-lab/StraTune.
Read the full abstract ▾
All authors Zeping Liu, Yan Li, Ni Lao, Gil Wolff, Gengchen Mai
Artificial Intelligence
Can LLMs Predict the Future? A Brier Score Analysis of Prediction Markets
Yuanbo Li, Zekun Li and Xiaoyan cong
We study whether model upgrades improve probability estimates for prediction-
market questions. Our Resolved
Market Forecasting (RMF)
benchmark contains 3,000 resolved binary questions across nine domains, on which we evaluate six Claude and Qwen model variants using a question-only, zero-shot protocol. We assess Brier scores relative to an empirical base-rate predictor, examine their Murphy decomposition, and compare models through paired differences, with results stratified by event category and timing relative to training cutoffs. In the reported post-cutoff stratum, the four Claude models achieve Brier scores of 0.183-0.192, improving on the base-rate reference by 0.024-0.033. Qwen 32B does not significantly outperform that reference, although its paired Brier is 0.024 lower than that of the 7B checkpoint. The evaluated Claude version and tier upgrades yield no significant improvement. Within-model differences across event categories exceed the observed differences among Claude variants. These results show why
forecasting scores should be interpreted alongside simple probability baselines and question composition: under this protocol, newer versions or higher model tiers do not consistently produce more accurate probabilities.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CV
Multimodal LLMs Outperform Pathology Foundation Models in Cross-Domain Histological Similarity
Yishu Zhang, Yun Li and Daiwei Zhang
State-of-the-art pathology foundation models, trained on millions of histology tiles, can fail to preserve tissue similarity when comparisons cross slide or institution boundaries. We show that general-purpose
multimodal LLMs, without being trained as pathology foundation models, consistently outperform these specialized models in cross-domain histological similarity judgments. Using a relative similarity framework that we release as the MOSAIC (Model Similarity Assessment across Institutions and Cohorts)
benchmark, we evaluate 17 models across 6
datasets and find that pathology encoders often rank same-institution, different-disease tiles as more similar than same-disease, different-institution tiles, a clinically dangerous failure mode invisible to standard within-domain evaluations. LLMs appear less susceptible to this failure, likely because they perform semantic visual comparison of morphology and tissue architecture rather than relying on shortcut features tied to acquisition context. Scaling training data does not resolve the problem for pathology encoders, implicating the learning objective rather than data coverage. Our results expose a fundamental robustness gap in current pathology foundation models and establish
multimodal LLMs as a viable alternative for cross-institutional
retrieval,
dataset harmonization, and multi-site quality control. Code and data will be released upon acceptance.
Read the full abstract ▾
Artificial Intelligence
Counterfactual Self-Evolving Agents for Evidence-Grounded Reasoning
Xing Han, Yuxin Wang, Chen Chen et al. (+8 more)
Self-play proposer–solver methods improve
reasoning by generating tasks and learning from verified solutions. However, for evidence-identifiable tasks, where case-specific evidence and domain knowledge determine a checkable answer, self-play requires generating plausible cases whose answers can be independently verified. We introduce counterfactual self-
evolution, which generates counterfactual context for reconsidering the original case. A trainable Proposer constructs targeted evidence edits and describes potential outcome changes with causal explanations. We handcraft an expert-verified counterfactual instruction-tuning
dataset to teach the Proposer to generate high-quality counterfactuals across a broad range of action–outcome scenarios. Each counterfactual instruction-tuning example specifies an edit within a defined category and explains its hypothesized causal effect on the decision, teaching the Proposer to reason systematically about what changes and why. We instruction-tune the Proposer on these examples, then formulate a fine-tuning reward that integrates feedback from the Solver and Verifier. Across diverse counterfactual scenarios, this reward favors high-quality counterfactuals and warranted revisions, while penalizing changes that overturn correct decisions. The counterfactual context aims to correct errors and strengthen confidence in correct decisions. Accepted counterfactuals accumulate in
memory that supplies in-context evidence to the frozen Solver; the Solver adapts through evolving context rather than weight updates. We apply the framework to clinical
reasoning, fact verification, and business
reasoning. Our evaluation tracks performance over successive rounds as counterfactual
memory grows, including transfer to harder cases. Our method achieves superior results across diverse frontier models.
Read the full abstract ▾
All authors Xing Han, Yuxin Wang, Chen Chen, Wei Dai, Gautham Krishna Gudur, Shijun Li, Hsing-Huan Chung, Gregory D. Hager, Joydeep Ghosh, Paul Pu Liang, Suchi Saria
Artificial Intelligence · Cross-listed from cs.DC
ASCEND: Personal AI Agents for Autonomous Scientific Computing Across HPC Clusters and GPU Workstations
J. Paul Liu, Uthpala Herath and Andrew Petersen
Traditional scientific computing requires researchers to translate intent into environment configuration, resource requests, and executable jobs, then diagnose failures from scheduler and application logs. We present ASCEND (Autonomous Scientific Computing Engine and Novel Discovery), an AI-powered
agent system, operated through a command-line or a local web chat interface, that runs the
agent on the researcher's own laptop, reaching Slurm clusters and a GPU workstation over multiplexed authenticated SSH, with site-specific execution policies checked by locally executed tools. The
language-model
agent (Claude Code or Codex, chosen at each launch) is hosted remotely and holds no credentials; an account on each resource suffices, and a public installer links additional clusters or workstations. We report four recorded cases: (1) the
agent closed a failure-recovery loop on a planted tensor-device fault, submitting, diagnosing, repairing, and resubmitting it; (2) it reproduced the published evaluation of a weather-
forecasting model, recovering an incompletely stated evaluation protocol and agreeing with the published curves to 2.1% (z500) and 2.4% (t850), while identifying two discrepancies in the paper's released materials; (3) it parallelized a released 12,693-line geophysical solver under a requirement of bit-for-bit identity of exported outputs, reducing runtime from about twelve hours to about two; (4) that requirement exposed two instances of undefined behavior in the published solver, both repaired and reported upstream. These cases used ordinary scheduler commands; a separate, pre-specified evaluation of the optional policy validator rejected 29 of 30 constructed violations, held one for approval, and denied 3 of 14 legitimate requests. Autonomy was exercised under author supervision, and
agent transcripts were not retained, so intervention rates are not independently verifiable.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.RO
RoboFoundry: System-as-Policy Evolution for Self-Learning Embodied Agents
Jingsong Liang, Shuhao Liao, Shizhe Zhang et al. (+15 more)
A foundation model should not act in isolation as an
embodied agent. Yet, existing methods often optimize individual components of the
agent stack, such as
memory, context, skills, or action interfaces, rather than treating the supporting system itself as a unified policy. Moreover, interaction alone does not yield self-improvement unless execution experience is converted into persistent, validated system changes. We therefore propose RoboFoundry, the first
embodied agentic framework that formulates this process as Self-Evolving System-as-Policy. RoboFoundry diagnoses capability gaps in decision-making and
memory management, converts execution traces into validated task-specific system updates, and promotes recurring improvements to the general system.
Evolution operates over two complementary surfaces: a context system that manages active internal context and persistent file-system
memory, and a hierarchical skill system that organizes atomic skills, reusable compositions, and failure-conditioned recovery. A shared semantic interface separates embodiment-invariant decisions from embodiment-specific execution, allowing evolved system capabilities to transfer across heterogeneous
robots. On EmbodiedBench, RoboFoundry achieves state-of-the-art performance, notably improving GPT-5.5 by 27.8%. It also brings Qwen3.7-Plus to near parity with GPT-5.5 (70.3% vs. 72.7%), showing consistent gains from system-as-policy
evolution across foundation models. For long-horizon
memory, RoboFoundry outperforms all baselines on RoboMemArena by at least 39.0%, even against methods assisted by external foundation models. On LIBERO-PRO, it further outperforms Cap-
Agent0 by 243.8%-679.7% across all perturbation types. In real-world deployments, RoboFoundry demonstrates zero-shot transfer and online
evolution across
robots and tasks, highlighting its potential for fully autonomous
embodied agents.
Read the full abstract ▾
All authors Jingsong Liang, Shuhao Liao, Shizhe Zhang, Diyuan Hou, Yuxin Cai, Xinjian Deng, Chengyang He, Wenhui Huang, Runjia Tan, Zhidong Wang, Lan Yu, Xuesong Tian, Guillaume Sartoretti, Jie Luo, Yao Mu, Wenjun Wu, Wanhua Li, Chen Lv
Artificial Intelligence · Cross-listed from cs.CV
PolyTopoBench: A Benchmark for Complex Vector Polygon Generation from Remote Sensing Imagery
Zeping Liu, Ni Lao, Weiwei Sun et al. (+5 more)
Vector polygon generation converts visual inputs, e.g., remote sensing (RS) images, into vectorized polygonal geometries, supporting applications such as autonomous driving, vector map construction, and remote sensing. Early pipelines predict raster masks and post-process them into polygons, which prevents end-to-end optimization and may miss small objects or introduce inaccurate vertices. Recent methods directly generate vector polygons, but most focus on simple exterior contours, while they either cannot represent complex polygons with holes or fail to preserve their
topology. In this paper, we propose PolyTopoBench, a unified evaluation framework for vector polygon generation from RS images with explicit emphasis on complex polygons. PolyTopoBench evaluates both exterior and interior rings, and
benchmarks 11 representative methods, including
segmentation-based polygonization pipelines,
vision foundation model baselines, and specialized vector polygon generators, on two RS-image
datasets covering buildings, roads, vegetation, and unvegetated regions. Experiments show that existing methods often recover simple exterior boundaries but degrade substantially on polygons with holes or multiple rings. These results reveal complex polygon generation as an unresolved challenge and motivate
topology-aware
benchmarks and model designs. Code and data are available at https://github.com/seai-lab/PolyTopoBench.
Read the full abstract ▾
All authors Zeping Liu, Ni Lao, Weiwei Sun, Gil Wolff, Yiqun Xie, Liang Zhao, Junfeng Jiao, Gengchen Mai
Artificial Intelligence · Cross-listed from cs.LG
Logic Gate Networks and Lookup Table Networks as Lightweight Hardware Classifiers for Inter-patient ECG Arrhythmia Classification
Wout Mommen, Lars Keuninckx, Siddharth Patil et al. (+3 more)
Deep Differentiable Logic Gate Networks (LGNs) and Lookup Table Networks (LUTNs) offer a promising approach for very low power inference due to their use of simple binary logic operations instead of arithmetic. In this work, we generalize the logic gates of LGNs to more than two input pins, naturally arriving at networks consisting of $N$-input LUTs. To obtain a differentiable expression for training the $N$-LUT entries, we adopt the Boolean equation of a $2^N$:1 multiplexer (MUX) and optimize its input parameters during training. We investigate the applicability of LGNs and LUTNs to inter-patient ECG arrhythmia classification using the MIT-BIH data set. The proposed models achieve up to 94.41% accuracy and a $j\kappa$ index of 0.683 on a four-class task, showing a competitive performance compared to existing CNN-, SVM- and SNN-based methods. Our LGNs and LUTNs only require an estimated 2.89k to 6.17k FLOPs, including preprocessing and readout, which is three to six orders of magnitude less than state-of-the-art methods. We verified our design, which consists of the preprocessing pipeline and a 6-LUTN classifier, by implementing it on a Xilinx Zynq-7000 ZedBoard. The complete system consumes a dynamic energy of 8.25 $\mu$J/inference, of which only 0.46 nJ is utilized by the LUTN classifier. These results show that both LGNs and LUTNs can be employed as lightweight hardware-based classifiers for inter-patient ECG arrhythmia classification.
Read the full abstract ▾
All authors Wout Mommen, Lars Keuninckx, Siddharth Patil, Paul Detterer, Achiel Colpaert, Piet Wambacq
Artificial Intelligence · Cross-listed from cs.LG
Transfer Learning for Edge Classification on Dynamic Text-Attributed Graphs
Tyler Bonnet and Marek Rei
Learning transferable representations for dynamic text-attributed graphs (DyTAGs) requires models to capture underlying interaction dynamics that persist across domains. However, existing methods tend to overfit to domain-specific structural, temporal, and semantic patterns, limiting edge classification performance under distribution shifts. To expose and address this, we formally establish a leave-one-domain-out (LODO) transfer learning protocol for edge classification on DyTAGs. Under this protocol, we demonstrate that state-of-the-art self-supervised methods for dynamic graph learning perform poorly when transferred to unseen domains. Strikingly, existing methods underperform a structurally and temporally unaware Bag of Events (BoE) model we introduce, which inputs only unordered sequences of node and edge text features. Proceeding from the BoE, we propose Spatio-Temporal Semantic
Alignment (STSA), which integrates a spatio-temporal encoder that fuses representations of time deltas and node occurrence frequencies into a unified
manifold. STSA is trained with a Contrastive Semantic
Forecasting objective, which anchors edge representations to a multi-domain textual latent space initialized by a pretrained
language model, providing a robust prior that outperforms BoE and all existing methods we evaluate.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.RO
Scanning While Imagining: A Scene-Graph World Model for Robotic Ultrasound Navigation
Xuesong Li, Shuai Chen, Feng Li et al. (+3 more)
Ultrasound (US) acquisition depends on the operator's ability to interpret anatomy and anticipate how the view will change with probe motion. Many robotic US
navigation methods select actions without explicitly predicting these anatomical changes. We propose SonoGraph-WM, an action- and goal-conditioned world model for anticipatory probe
navigation. The model represents anatomy as scene graphs (SGs), capturing visible structures, their geometry, and spatial relationships without synthesizing US images. Given a history of SGs and probe poses, a unified
Transformer jointly predicts future SGs and poses. A receding-horizon planner recursively imagines candidate trajectories, selects the shortest predicted path reaching a goal graph, and follows it over a short execution horizon before replanning from new observations. To reduce reliance on tracked and anatomically annotated US sequences, we generate aligned SG–pose training data from computed
tomography (CT) label maps along surface-constrained probe trajectories. On four held-out CT cases, spatial relation F1 remains above 93% over 20 prediction steps, and closed-loop
navigation achieves 77.50% and 75.00% success for the gallbladder and pancreas, respectively, using annotation-derived SGs. In
robot–phantom
navigation experiments with label-map-derived SGs, the planner reached the target view in 73.7% of trials. These findings support CT-supervised anatomical world modeling for probe planning and highlight the importance of frequent observation updates for reliable
navigation. Project Page: https://noseefood.github.io/us-sonograph-wm/
Read the full abstract ▾
All authors Xuesong Li, Shuai Chen, Feng Li, Zhongliang Jiang, Nassir Navab, Yuan Bi
Artificial Intelligence · Cross-listed from eess.SP
Radiomap Blind Prediction under Incomplete Observation: Error Characterization and Correctable Propagation-Prior Learning
Xiaojie Li, Yu Han, Han Fang et al. (+4 more)
Radiomap blind prediction aims to infer radiomaps from observable representations of the propagation environment and base station configuration without field measurements. In practice, the observable representations are inherently incomplete. Thus, the target radiomap is not fully determined by the inputs when generalizing to unseen configurations or environments. Under incomplete observation, we establish a population-level theory of deterministic radiomap blind prediction that identifies the conditional mean as its optimal target and separates prediction error into reducible predictor approximation and irreducible uncertainty caused by missing physical information. The framework further characterizes the train-test risk gap and the uncertainty reduction enabled by observation enrichment. Building on it, we reveal the dual role of propagation priors: they provide physically grounded guidance, yet their implementable forms may bias the attainable predictor. This motivates RadioDecomp, which treats a prior-guided predictor as a correctable base and learns its remaining predictable discrepancy through residual refinement. To evaluate RadioDecomp across distinct propagation-prior designs, we instantiate it with a feature-guided monolithic base and a LoS-Shadow structured base, yielding RadioFR and RadioLSR, respectively. Across random, cross-configuration, and cross-environment settings, experiments confirm the benefit of propagation-related representations and show that both instantiations improve upon their respective bases. Further controlled studies on base capacity, training-support coverage, and observation coarsening corroborate the proposed analysis.
Read the full abstract ▾
All authors Xiaojie Li, Yu Han, Han Fang, Shangqing Liu, Guangxu Zhu, Shi Jin, Chao-Kai Wen
Artificial Intelligence
FinancialAuditBench: Benchmark Construction under Differential Privacy Using Real-World Priors
Jerry Huang, Sarvesh Babu, Matt Van Buren et al. (+5 more)
As AI
agents are becoming widely adopted in the financial services industry, careful measurement is essential to understand where they can be reliably deployed and where oversight and professional review remain necessary. Such measurement, however, is constrained by limited access to proprietary or
privacy-sensitive data. Existing
benchmarks therefore often rely on publicly available data, human- and/or LLM-authored tasks, or simplified settings. We introduce FinancialAuditBench, a
benchmark for evaluating
agents on financial statement audit tasks, along with a framework for systematically generating synthetic engagements. Our task generation framework leverages differentially private aggregate statistics from historical audits along with audit expertise contributed through over 1,100 hours of
benchmark development and review. FinancialAuditBench consists of 90 tasks spanning workpaper completion and review across six synthetic audit engagements, each containing an average of 179 files. Evaluation on eleven frontier models shows that while
agents complete substantial portions of staff-level audit tasks well, they sometimes perform inappropriate procedures or produce incorrect documentation. Beyond financial
auditing, our framework offers an approach for systematically generating synthetic tasks for model evaluation and training in
privacy-sensitive domains.
Read the full abstract ▾
All authors Jerry Huang, Sarvesh Babu, Matt Van Buren, Alexander Wang, Pranav Pillai, Arush Jain, James P. Burton, Julia Hockenmaier
Artificial Intelligence
Improving LLM Collaboration via Multi-Agent Preference Learning
Shuo Liu, Xinzichen Li, Tianle Chen et al. (+1 more)
Several works have explored multi-
agent reinforcement learning (MARL) in LLM collaboration. However, constructing reliable rewards is difficult in practice, as complete and accurate metrics are often unavailable and hard to aggregate. Preference learning provides an alternative by learning from comparative human or AI feedback. Yet, its extension to multi-
agent systems remains underexplored. To address this gap, we formulate preference-based multi-
agent systems (MAS) from decentralized and centralized collaboration perspectives. We also introduce a general multi-
agent preference learning framework (MAPL) to solve these problems. MAPL allows iterative updates by comparing the current solution with decentralized or centralized solutions generated by various
agents. We instantiate MAPL using MARL from human feedback (MARLHF) with a learned reward model and multi-
agent direct preference optimization (MADPO). Experiments on collaborative writing, coding, tool use, and travel planning show that MAPL can improve collaboration quality and efficiency while approaching the performance of MARL with fixed, well-defined rewards. Within MAPL, MARLHF generally outperforms MADPO on most tasks but remains sensitive to data coverage,
agent and comparator models, and the underlying MARL
algorithms.
Read the full abstract ▾
All authors Shuo Liu, Xinzichen Li, Tianle Chen, Christopher Amato
Artificial Intelligence
The Decomposition Tax: LLM Pipelines Lose Up to 40 Accuracy Points at Their Own Interfaces
Tianqi Bu, YuXuan Peng, Junteng Tu et al. (+1 more)
A four-stage LLM pipeline gives up as much as 40.5 accuracy points at its own interfaces (gemma-3-12B on MATH-500, Holm-corrected p = 1.66e-19; the largest tax in the primary family). We hold model, problem, stages, stage
prompts and completion budget fixed, vary only whether each stage can still see the original problem, and call the accuracy difference the decomposition tax. Across 21 open-weight models from nine organisations, on GSM-Hard and MATH-500 at n = 200 paired items per
cell, 70 of 118 primary-family tests survive Benjamini-Hochberg correction and 54 survive Holm. On GSM-Hard, a placebo recovers nothing: it carries at least 60% of the extra
tokens and at most one word of the problem. Builders design a pipeline one stage at a time, and its bill arrives at the interfaces between stages. Rewriting one stage's instruction moves gemma-3-12B's tax from 4.5 to 36.5 points, and adding "every relationship stated between them" to a stage that lists the numerical quantities lowers the tax on 9 of 9 models on MATH-500. Re-grounding, which shows a stage the original problem again, belongs after the loss. With one lossy interface, re-grounding the stage after it beats re-grounding the stage before it on 7 of 7 models on both
benchmarks; on MATH-500 the earlier repair is worse than none on 7 of 7. Newer models still pay: gemma-4-12B gives up 37.0 points, and the repair holds on all three of the newest models we test. A sealed held-out test refuted a stronger rule we registered, which predicted the paying stage from the interface and receiver types, so we locate the tax by measuring one stage at a time. The prescription has two parts: re-ground the stage after the lossy interface, and if a stage must list the quantities, tell it to keep the relationships.
Read the full abstract ▾
All authors Tianqi Bu, YuXuan Peng, Junteng Tu, Henghui Xiao
Artificial Intelligence
Routing Drift Alone Does Not Diagnose Failure in Merged MoE LLMs
Yuanyi Wang, Yanggan Gu, Su Lu et al. (+7 more)
Model merging efficiently combines specialized large
language models (LLMs) without joint retraining, but can substantially alter expert routing in Mixture-of-Experts (MoE) models. Such routing drift is often interpreted as routing failure, raising a fundamental question that remains unclear: does routing drift after MoE merging actually indicate routing failure, and what evidence should justify repair? We investigate these questions across DeepSeekMoE, OLMoE, and Qwen3-MoE proposing a routing analysis toolkit for controlled counterfactual interventions and
token-level analysis. By crossing source and merged router inputs and parameters, we attribute most expert reassignments to input shifts rather than parameter changes at the same layer. However, source-relative routing differences poorly predict next-
token likelihood gains from source-route restoration, and different expert selections can produce directionally similar mixture outputs. We therefore operationalize routing failure as task loss recoverable under a specified routing intervention, with non-routing parameters fixed. These tests detect recoverable loss under deliberate router corruption, whereas source-route restoration does not establish reliable task benefits in the evaluated merged models. Motivated by these, we propose Selective Router Repair (SRR) as a case study, and find that source-specialist
token-likelihood advantages do not reliably identify beneficial local corrections. Together, these findings show that routing drift alone is insufficient evidence of routing failure: source-informed corrections must be judged by their task-level intervention effects. The analysis toolkit and SRR code are released.
Read the full abstract ▾
All authors Yuanyi Wang, Yanggan Gu, Su Lu, Guanghao Zhu, Pengkai Wang, Yifan Yang, Congkai Xie, Zhaoyi Yan, Jianmin Wu, Hongxia Yang
Artificial Intelligence
Rank Collapse Is Recoverable, Growing $|Q|$ Is Not: Out-of-Sample Early Warning for Value Divergence in High-UTD Soft Actor-Critic
Tianqi Bu, YuXuan Peng, Junteng Tu et al. (+1 more)
Raising the update-to-data (UTD) ratio breaks off-policy critics in two ways grouped as "plasticity loss": collapsing representations and growing value magnitude $|Q|$. We separate them in Soft Actor-Critic (SAC) with scaled critics (width 2048, no normalization). Collapse is survivable: at UTD ratio 16, HalfCheetah critics with most units dormant keep learning, and the training guard, which stops runs whose loss or $|Q|$ explodes, never flags them. Within one high-UTD SAC configuration, runs start close together, and how far a critic's $\log_{10}|Q|$ has climbed by step 15k, its early growth, ranks the runs by how soon the guard flags them. At 15k, a flagged run's $|Q|$ sits a median of over a hundredfold below its flag level, yet the climb's rate already orders the flags (Harrell's C and out-of-sample AUC 0.78 on Walker2d, 0.98 on Ant, at UTD ratio 4). Dormancy does not. Aborting on this rate saves about a tenth of held-out Walker2d compute and stays net-positive live. A LayerNorm critic lowers the rate, removes the flag on Walker2d at UTD ratio 4 and lowers the return.
Read the full abstract ▾
All authors Tianqi Bu, YuXuan Peng, Junteng Tu, Henghui Xiao
Artificial Intelligence
When Can First-Order Models of Fine-Tuning Bound Forgetting?
Jianchang Su and Wei Zhang
Fine-tuning a
language model on new data can make it forget facts that it should keep. We ask whether measurements taken at the start of a fine-tuning run can bound, for each protected fact, the probability that the run makes the model forget it. In LoRA fine-tuning with
stochastic gradient descent on models from 0.6B to 14B parameters, a first-order response model estimated by finite-difference probes predicts changes of per-fact margins with correlation 0.974-0.998. Predictions of forgetting built on this model nevertheless failed, because forgetting requires parameter changes far outside the region in which the model was validated. The probes can, however, bound the probability that a margin first falls below a boundary near zero: we derive Freedman and Azuma first-passage bounds for a linear surrogate of the margin and test on new runs whether they hold for the model. The bounds contain a term R that measures how much the response coefficients change during the run. The simplified Freedman bound, which sets R = 0, certified most facts but was violated in 14 of 112 conditions, and every fact on which it was violated had R >= a, where a is the distance of the fact's margin to the boundary. The complete Freedman bound certifies only facts with R < a, and it held in every condition. On the violated facts, the spread of the margin across test runs was a median of 14.6 times the prediction of the response model, so the failures are breakdowns of the model, and in our data they occurred only where R >= a. We found this pattern post hoc and tested it in two preregistered confirmatory studies with 43 new conditions: the complete bound held in all of them, and the simplified bound failed there on only 3 facts, each with R >= a. First-order models of fine-tuning can thus bound forgetting on the facts whose response coefficients change by less than their distance to the boundary.
Read the full abstract ▾
Artificial Intelligence
Right Answer, Wrong Reason: Accuracy, Consistency, and Consensus Are Misleading Indicators of LLM Faithfulness in Clinical Decision Support
Bharath Kumar Bolla, Chaitali Deb Purkayastha and Vishnu Surya Reddy Nandi
Clinical Large
Language Models (LLMs) achieve strong medical-exam accuracy; however, a correct answer does not guarantee that the explanation names the concepts that actually drove the decision. We introduce three lightweight, directly interpretable metrics for this faithfulness gap: the Explanation Stability Index (ESI), which measures
reasoning consistency across repeated queries; the Causal Faithfulness Score (CFS), which tests whether cited concepts drive predictions via concept ablation; and the Perturbation Stability Score (PSS), which measures robustness to semantic-preserving paraphrases. By evaluating six LLMs on 150 MedQA-USMLE questions (900 model-question observations), we found that only 23.3% of the cited clinical concepts were causally necessary. Correct answers had lower CFS than incorrect answers (0.212 vs. 0.398), answer consistency negatively predicted CFS (Spearman r = -0.466), and model pairs could agree on answers while sharing only 8.8% of cited
reasoning concepts. These results show that accuracy, consistency, and consensus are incomplete safety signals for clinical decision-making support. The evidence is behavioral rather than mechanistic: concept ablation tests counterfactual sensitivity of outputs, not internal
circuits.
Read the full abstract ▾
Artificial Intelligence
Overwhelmed by Choice: Studying LLM Decision Making at Scale
Yu-Chi Lin, Aryan Seth, Anshul Aravind et al. (+4 more)
Multiple-choice and candidate-selection evaluations are widely used to assess LLM
reasoning and decision-making, yet most
benchmarks contain relatively small candidate sets. It remains unclear whether conclusions drawn from these settings remain valid as the candidate space scales. We systematically evaluate LLMs as the number of competing candidates increases and find substantial accuracy degradation across tasks, prompting strategies, and model scales. Controlled analyses show that standard long-context
retrieval explanations cannot fully account for this degradation. Instead, we identify two systematic failure patterns. First, gold-margin collapse: the score gap between the correct answer and the strongest distractor progressively shrinks, driven primarily by weakening confidence in the correct answer. Second, earlier candidate preferences become increasingly difficult to overturn, with later candidates exerting progressively weaker influence on the final prediction. Motivated by these findings, we evaluate hierarchical partitioning and permutation-based inference, which improve accuracy by roughly 20 percentage points at $N=160$ on both HotpotQA and MIMIC. Overall, our results identify candidate-set scale as an important evaluation-protocol variable and show that strong small-option performance does not necessarily imply robust large-scale candidate comparison.
Read the full abstract ▾
All authors Yu-Chi Lin, Aryan Seth, Anshul Aravind, Eugene Lee, Tanmay Parekh, Nanyun Peng, Kai-Wei Chang
Artificial Intelligence · Cross-listed from cs.LG
Mind the Spike: Mechanisms and Brittleness of Visual Massive Activations in Large Vision-Language Models
Jonas Ngnawé, Yann Pequignot, Sabyasachi Sahoo et al. (+3 more)
Large
vision-
language models (LVLMs) inherit massive activations from their text-only bases: spikes where a few fixed hidden channels receive values thousands of times above the typical magnitude. The text spike systematically appears in early layers at a fixed initial position, independently of input content. Visual spikes vary across images, but whether their formation follows a consistent pattern across LVLMs and how they respond to image perturbations remain open questions. We find that some LVLMs do not form visual spikes, while others spike at different rates, typically in deeper layers. We identify the trigger direction from model weights and an interpretable location rule: before the
language model decoder runs, eventual spike
tokens are largely restricted to those sharing least with the rest of the image. Crucially, visual spikes are strikingly brittle. Common corruptions frequently create and relocate spikes, and less often remove them, raising overall incidence. Our trigger-guided spike attack deliberately creates or removes spikes under a small $\ell_\infty$ budget, with 1/255 enough in nine of the ten models that spike. Finally, our preventive intervention removes only the trigger component before spikes erupt, eliminating or substantially reducing spikes on clean and perturbed images while leaving the other image
tokens nearly unchanged. Our study spans 25 adapter-based LVLMs built on 18 released text-only bases from 10 families, ranging from 2B to 72B parameters.
Read the full abstract ▾
All authors Jonas Ngnawé, Yann Pequignot, Sabyasachi Sahoo, Christian Gagné, Frédéric Precioso, Sanmi Koyejo
Artificial Intelligence
Beyond Accuracy: Counterfactual Fragility and Demographic Bias in Clinical Evaluation of LLMs
Chaitali Deb Purkayastha, Bharath Kumar Bolla and Vishnu Surya Reddy Nandi
Clinical LLM evaluation often emphasizes answer accuracy; however, accuracy alone does not test counterfactual consistency or demographic robustness. We evaluated six LLMs on 150 MedQA USMLE questions using two automated perturbation tests to assess their performance. The counterfactual validity (CFV) test asked each model to make a minimal, plausible clinical change that would make a different answer correct. The demographic robustness test added six demographic prefixes to the same vignette and compared the answers and explanations with a no demographic baseline. Of the 900 CFV attempts, 228 (25.3 %) were valid and 672 were invalid. Across 5,400 demographic comparisons, 1,097 answers were changed (20.3%). Automated judging identified 3,128 stereotype evidence flags, including 1,932 in the broad Other category. MedGemma 27B achieved the highest accuracy (87.1%) and CFV (63.3%), lowest answer change rate (16.0%), and low mean Explanation Demographic Dissonance (EDD) score (0.169). However, its accuracy still exceeded its CFV, indicating that correct answers do not guarantee reliable performance on the counterfactual validity task. OpenBioLLM had the highest answer change rate and EDD, whereas GLM had the highest stereotype flag rate. These findings show that accuracy, CFV, answer stability, EDD, and stereotype evidence capture different evaluation aspects. Because all judgments were automated and no clinician validation was available, the results support safety screening but do not establish clinical deployability of the model.
Read the full abstract ▾
Artificial Intelligence
Decision-Sufficient State Representations: Measuring and Reducing Write-Time Regret
Bingyu Shen and Boyang Li
Long tasks produce more history than an LLM
agent can hold in its context, and more than it uses reliably even when the history fits. A growing line of work therefore has
agents carry a short written state instead: at every step a writer rewrites the state, and a reader acts from the state alone. Steps stay cheap, but anything the writer drops is lost before later decisions reveal that they need it. We quantify this loss and ask whether training can reduce it. Comparing the written state with the best state of the same size written in hindsight, we split the reader's loss into a budget loss, which any state of that size must incur, and a write-time regret, which comes from the writer's choices. In TextWorld cooking games where we control how long a fact must be carried before it is needed, a 128-
token state holding the facts wins nearly every game, while prompted
language-model writers win at most 17%. Almost all of the loss is write-time regret, and it grows with the delay. We then train the writer from the reader's own loss. DSSR (decision-sufficient state representations) scores candidate states by how well the reader acts after the writer carries them forward, and teaches the writer to prefer the better ones. This forward-rolled score predicts game outcomes ($\rho = 0.48$), whereas scoring a candidate as a fixed context, as hindsight methods usually do, does not ($\rho \leq 0.07$). On a pre-registered test split opened once, training adds +7.0 [+1.9, +12.2] points of success when facts are needed soon, bringing a plain summary writer to the level of belief- and slot-based
memory prompts. The gain shrinks as the delay grows and is significant only at the shortest delay. We trace this limit to credit assignment: keeping a fact now pays off only if every later rewrite keeps it too, which a per-step score cannot see.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.SD
Finding Emotions Where They Belong: Rethinking Audio Emotion Recognition through Masked Temporal Affective Grounding
Abdelrahman Mohamed, Lars Kai Hansen and Zheng-Hua Tan
Audio
emotion recognition (AER) typically assigns a single label to an entire recording, leaving the temporal scope of that label ambiguous when multiple speakers and affective events are present. We address this limitation by reformulating AER as a Temporal Affective Grounding (TAG) task that associates
emotions with temporally bounded
speech spans and vocal tone descriptions. To support this formulation, we curate temporally annotated versions of existing
emotion recognition
datasets and construct recordings containing two to four affective
speech spans, including overlapping
speech. Training in this longer format with a standard
language-modeling objective can degrade both
emotion recognition and temporal grounding performance, while tone descriptions can provide shortcuts for
emotion prediction. To address these challenges, we introduce Masked Temporal Affective Grounding (M-TAG), a supervised training objective that combines full-sequence
language modeling with
emotion and timestamp cross-
entropy losses under
attention masking. The masking varies the context visible to
emotion-label
tokens to reduce reliance on shortcuts and improve generalization, while the timestamp loss incorporates a distance-aware weight to penalize larger temporal errors. We evaluate EMO-TAG, a model fine-tuned using our
dataset and objective, on
emotion recognition and affective temporal-grounding against three AER and audio-
language baselines: Flamingo-Next, Audio-Reasoner, and AffectGPT. Our results show that existing models achieve limited affective temporal-grounding despite competitive
emotion recognition performance.
Read the full abstract ▾
Artificial Intelligence
Nutri-ATLAS: Embodied Agent for Tabulated Lookup and Assistance for Smarter nutrition
Uttej Kallakuri, Boxun Hu, Ankur A. Butala et al. (+2 more)
Generative and Agentic IoT systems offer a promising foundation for digital healthcare applications that combine sensing, personalized
reasoning, and autonomous interaction in real-world environments. Nutrition assistance is a natural use case, but existing Large
Language Model (LLM)-based systems are often limited to passive text interaction and static context, making them unreliable when food descriptions are ambiguous or nutritional evidence is missing. We propose Nutri-ATLAS, an
Embodied Agent for Tabulated Lookup and Assistance for smarter nutrition in the real world. It integrates graph-grounded nutrition
reasoning, hardware-aware LLM selection, and
robot-based evidence acquisition. Nutri-ATLAS builds a unified Food-Nutrient knowledge graph from USDA FoodData Central and FoodKG and learns 64-dimensional GATv2 food and recipe
embeddings. A shared hybrid graph-text scoring mechanism supports food nutrition extraction, nutritional gap filling, substitute
retrieval, and recipe-level meal composition, while an LLM-guided skill interface navigates landmarks, updates dietary-context and food-accessibility
memory, and grounds recommendations in observed food availability. We evaluate Nutri-ATLAS across nutrient estimation, substitution
retrieval, recipe recommendation, patient-profile adherence, edge deployment, and real-world
embodied execution. On HealthyFoodSubs, the hybrid retriever achieves 37.9% MAP, 80.7% RR@5, and 90.1% RR@10. On NutriBench v2, Dense+GAT
retrieval grounds nutrient estimation across nine quantized Qwen3.5-9B configurations. On PFoodReQ, Nutri-ATLAS reaches 78.8% MAP, 83.0% MAR, and 77.5% F1. A patient-profile study shows adherence to allergy and healthy-target constraints for all selected cases.
Read the full abstract ▾
All authors Uttej Kallakuri, Boxun Hu, Ankur A. Butala, Najim Dehak, Tinoosh Mohsenin
Artificial Intelligence
Re-derivability Decides What a Staged Agent Pipeline Recovers After an Upstream Fault
Tianqi Bu, YuXuan Peng, Junteng Tu et al. (+1 more)
One variable sets what an upstream fault costs a staged pipeline of
language-model
agents: re-derivability, how much of what a stage needs it can rebuild from the original problem. Grounding an inspector
agent in that problem is worth +0.608 [+0.517, +0.700] to +0.358 over a blind one on four open-weight backbones served with thinking disabled, and on the two Qwen backbones the blind inspector changes no item at all. That head-to-head is exploratory. One deterministic fault enters the first stage, and we re-expose the original problem to $k = 0,\dots,3$ of the downstream stages with
agents, items, fault and
topology held fixed, on 120 gsm_hard items per arm at temperature zero. Accuracy under fault rises on four of four backbones, from +0.233 to +0.392, the largest Holm-adjusted $p$ being $2.1\times10^{-6}$. A registered kill test rules out
tokens. Blanking every word holds the word slots fixed, and retention tracks the visible fraction on four of four, climbing from 0.221 to 0.692 on the primary. Those two families are confirmatory and everything else here is exploratory. The interaction excludes zero on two of four backbones under the registered pipeline, four of four under a three-stage pipeline, and three of four under full message history, the primary at +0.317. On Llama-3.1-8B the fault carries no detectable cost at any dose, so the other three carry every claim about what a fault costs. Re-derivability also sets what the architecture costs, and no decomposition we measured reliably beats one direct call. With no fault injected the registered pipeline loses to that call by -0.267, -0.125 and -0.317, and on Phi-4 reads +0.058 at $p = 0.118$, which the test fails to separate from zero. The repair that works is cheap and front-loaded: the first re-grounded stage buys +0.394 of matched retention for +59.8
tokens per item on Qwen3-14B, and the stages after it buy nothing.
Read the full abstract ▾
All authors Tianqi Bu, YuXuan Peng, Junteng Tu, Henghui Xiao
Artificial Intelligence
PlanGuard: A Guardrail for Multi-Step Plan Safety in Embodied Agents
Junchi Chen, Changtao Miao, Yuxiao Xiang et al. (+9 more)
Embodied task planners may produce multi-step plans whose subtask dependencies and interactions with the environment create physical risks during execution. Yet existing safeguards overlook such compositional risks, as general-purpose guardrails focus on semantic harm and
embodied safety detectors assess subtasks in isolation. To address this gap, we introduce PlanGuard, the first pre-execution detector that evaluates the physical safety of a complete multi-step plan in its current environment. For training and evaluation, we construct a Multi-Step Plan Safety (MSP-Safe)
dataset through paired task construction, plan generation using diverse planners, and safety annotation by three judges. Task-oriented SFT on MSP-Safe establishes fundamental plan-safety assessment capabilities, yet a substantial gap remains between compact models suitable for real-time deployment and stronger but costlier large models. Accordingly, we propose Strong-Teacher Adaptive Compensation for On-Policy
Distillation (STAC-OPD), which provides compact models with adaptive strong-teacher supervision along their on-policy trajectories. It combines
token-level distribution transfer from a fine-tuned strong teacher with probability-routed sequence-level compensation, retaining student-generated targets when the student favors the reference safety decision and using teacher-reconstructed targets otherwise. Across all test subsets, PlanGuard-2B achieves average 87.15% ACC and 87.21% F1, demonstrating effective whole-plan physical-risk detection at compact model scale. Code and
dataset will be publicly released.
Read the full abstract ▾
All authors Junchi Chen, Changtao Miao, Yuxiao Xiang, Zhenchao Jin, Haojie Yuan, Qi Chu, Tao Gong, He Liu, Bo Zhang, Jiansheng Cai, Zhe Li, Nenghai Yu
Artificial Intelligence
AgentHabit: Characterizing Distinct Behaviors of Agents on Everyday Tasks
Woojung Song, Hoyeol Yang, Jeonghoon Shim et al. (+4 more)
Large
language model (LLM)
agents assist users with everyday tasks that can be completed in many reasonable ways. Even when their answers are useful, how
agents carry out these tasks may not match users' preferences and needs. For example,
agents differ in whether they ask clarifying questions or search the web. We introduce HABIT, a taxonomy of 23 behavioral axes in five categories, which three authors and three LLMs derive bottom-up from 408
agent trajectories across 17 domains. On held-out tasks, HABIT distinguishes models more clearly than existing taxonomies of human values and
agent actions while supporting comparably consistent annotation. Building on HABIT, we construct AgentHABIT, a
benchmark that profiles each
agent's behavioral tendencies from its trajectories on 86 everyday tasks. Profiling 18 models with AgentHABIT reveals a range of distinctive tendencies. For example, most GPT and Claude models state their assumptions and offer alternatives when requirements conflict, whereas Qwen and Google's models more often leave assumptions or changes to requirements unstated. These profiles remain recognizable even when built from entirely different sets of tasks, indicating that they reflect general tendencies rather than task-specific behavior. Prompting
agents to adopt specific behaviors shifts some axes readily but barely changes others, while fine-tuning on another model's trajectories changes only part of a model's profile and leaves much of it intact. Overall, HABIT and AgentHABIT provide a systematic framework for characterizing how
agents carry out everyday tasks beyond task success, offering insights to guide the development of
agents whose behavior better fits users' needs.
Read the full abstract ▾
All authors Woojung Song, Hoyeol Yang, Jeonghoon Shim, Sungjib Lim, Jonggeun Lee, Yunho Choi, Yohan Jo
Artificial Intelligence
$T^5$: Twin-Critic Training for Token-Level Thoughts in Reinforcement Mid-Training
Nan Qiao, Yebin Yang, Weinong Wang et al. (+10 more)
Reinforcement mid-training lets
language models learn internal thoughts from unlabeled text, but efficient
token-level credit assignment remains challenging. Existing group-relative methods require costly repeated generation. Learned critics offer single-rollout feedback, but accurate return prediction alone does not ensure reliable policy updates. Our analysis shows how training–inference mismatch and PPO clipping prevent a common offset in advantage estimates from cancelling out, introducing additional update drift. We propose \tfour{}, a twin-critic method that calibrates
token-level advantages from a single generated trajectory. After warmup and held-out qualification, the critics provide two advantage estimates, combined using action-dependent weights learned through a conditional-moment saddle-point objective. This objective brings the average advantage at each prefix toward zero, while a signal-retention constraint prevents the correction from erasing the learning signal. Sharing information across text positions avoids repeated sampling of each prefix. Theoretically, we characterize optimal mixing under the signal-retention constraint and establish an upper bound on residual mean-induced drift. Experiments show that, compared with the state-of-the-art critic-free method, \tfour{} improves mean
benchmark performance by 7.8% and reduces mean training-step time by up to 63.4%.
Read the full abstract ▾
All authors Nan Qiao, Yebin Yang, Weinong Wang, Shuning Wang, Shangpin Peng, Fengyuan Lu, Xinming Wang, Zhehan Kan, Ruixu Zhang, Songyang Zhang, Sheng Yue, Yonglong Tian, Ju Ren
Artificial Intelligence · Cross-listed from cs.LG
Getting Motif-ated: Controllable AI Compositions from Injected Motif Prompts
Chao Peter Yang, Cynthia Rudin, Yue Jiang et al. (+2 more)
Deep learning has transformed symbolic music generation by borrowing the training paradigms of large
language models, with systems such as NotaGen now producing complete, stylistically convincing classical scores from a short
prompt. These systems could become powerful creative partners, helping musicians generate endless possibilities. However, current systems expose almost no control handles on the music itself. In principle, control handles could be built into a foundation model trained from scratch, but this is rarely practical without massive amounts of quality annotated data and compute resources. We therefore present MotiGen, a recipe for retrofitting pretrained symbolic music models to use new instruction
prompts. MotiGen injects a musical motif as a structured
prompt line, reinforces it with a scalar
attention bias toward the motif
tokens, and learns the association with a two-phase curriculum. First, it learns from focused excerpts cropped around motif occurrences in the training data, then full scores including the motifs. Our experiments show that our model composes with the prompted motif in over 92.3% of generated pieces. Generated pieces using a variety of motifs are included in our sample site: https://motigen-site.github.io/.
Read the full abstract ▾
All authors Chao Peter Yang, Cynthia Rudin, Yue Jiang, Simon Mak, Stephen Ni-Hahn
Artificial Intelligence
CLAIRE: A Schema-Grounded Hybrid Workflow for Healthcare Administrative Form Completion
Garapati Keerthana and Manik Gupta
Healthcare administrative staff transfer structured information from electronic health records, referrals, claims systems, provider rosters, and work queues into dynamic forms. We developed and evaluated CLAIRE (Clinical
Language and Agentic Intelligence for
Reasoning and Entry), a hybrid workflow that separates field-state discovery, source-to-field mapping, deterministic validation, bounded correction, escalation, and audit tracing. We tested five synthetic healthcare administrative schemas, 1,000 source records, four interface variants, two data-quality suites, and six comparators, yielding 24,000
benchmark episodes. A separate strict-output audit evaluated direct mappings from Qwen2.5-1.5B and Qwen2.5-7B, and a trace-derived operational simulation covered 6,000 episodes. Under the evaluated synthetic
benchmark conditions, full CLAIRE achieved 1.000 episode success, field accuracy, required-field completion, and dependency completion in both suites; removing validation reduced stress-suite success to 0.500. In the simulation, 100.0% of clean and validation-stress episodes reached a staff-reviewable draft, compared with 68.6% of escalation challenge episodes, unsupported cases were blocked. Scenario-based savings were 149.7-165.5 seconds per case, not observed staff times. The findings support schema-grounded, validation-first healthcare administrative automation in which
language-model components assist mapping but do not authorize unsupported or consequential actions.
Read the full abstract ▾