Artificial Intelligence
One-Sided Verification Under Limited Expert Review
Fan Nie, Haotian Ye, Shurui Liu et al. (+3 more)
AI has become increasingly helpful in scientific discovery, yet human expert review remains essential for those problems without reliable automated evaluation, posing a fundamental challenge as candidate generation scales beyond available review capacity. This paper studies verification under limited human effort, where rare valid solutions need to be prioritized while invalid candidates should be filtered out to reduce human workload. We adopt the one-sided verification objective that maximizes the rejection rate of invalid solutions (TNR) under a constraint on the false-rejection rate of valid ones (FNR), making human review cost an explicit term to minimize. Drawing on the intuition that verifying a specific error can be easier than discovering one, we introduce OSVerify, an algorithmic framework that decouples error proposal from error validation. The error proposer searches for errors in solutions across different granularities by hierarchical decomposition, while the validator distinguishes substantial errors from minor or hallucinated ones. Experiments on competition- and research-level
benchmarks show that OSVerify reduces the number of human reviews of invalid solutions by 46% on average compared to voting baselines under the same empirical low FNR constraint. Our post-training of the validator further improves the reduction to 57%. As high-impact real-world use cases, we apply OSVerify to identify errors in published research papers that are subsequently confirmed by the original authors and help resolve three Langlands open
conjectures that are confirmed by experts, demonstrating its utility to facilitate human-AI research collaboration.
Read the full abstract ▾
All authors Fan Nie, Haotian Ye, Shurui Liu, Zihao Wang, Stefano Ermon, James Zou
Artificial Intelligence
ResearchFlow-Eval: Tracing the Execution Gap in End-to-End Multi-Claim Autoresearch
Peng Jiang, Dongyang Li, Quanying Liu et al. (+1 more)
Autoresearch systems increasingly undertake entire research projects. In practice, scientific research requires systems to pursue multiple interdependent claims through a continuous, multi-stage workflow and revise experiments as evidence accumulates. Existing evaluations of autoresearch systems, however, typically score isolated tasks or final artifacts, leaving these process-level capabilities largely unmeasured. We therefore introduce ResearchFlow-Eval, a fine-grained framework for assessing continuous scientific execution and iteration at both the stage and claim levels. Each evaluation is grounded in a published anchor paper, from which we derive stage-specific rubrics that decompose each research stage into dimensions and separately scored rubric items. Using these rubrics, we score each research stage and examine experimental execution in detail, focusing on multi-claim completion and revisions across attempts. We apply the evaluation framework to general-purpose
agent harnesses, skill-augmented harnesses, and systems with research-specific orchestration. Across all evaluated systems, experimental execution consistently lags behind design and planning. Within execution, systems struggle to produce high-quality results and meet claim requirements, and repeated attempts rarely yield measurable improvements. Moreover, neither skill augmentation nor research-specific orchestration reliably yields higher scores. ResearchFlow-Eval thus extends autoresearch evaluation beyond isolated, single-objective runs to continuous workflows in which systems pursue multiple claims and refine their work across iterations.
Read the full abstract ▾
All authors Peng Jiang, Dongyang Li, Quanying Liu, Xiaoxuan Jia
Artificial Intelligence
AReaL-RSI: Adaptive Scaling and Coordination of Recursive Self-Improvement
Wenhao Zhou, Jiajun Song, Haitao Wang et al. (+11 more)
Scaling recursive self-improvement requires additional computation to produce useful research progress in reaching persistent objectives. Intuitively, more researcher
agents can explore more hypotheses, but early evaluation scores might misrepresent their promise, shared findings could lose their applicability conditions, and useful procedures could remain confined to completed investigations. In this paper, we present AReaL-RSI, a framework for organizing a persistent research team of
agents without updating backbone LLM weights. We formulate the scaling problem in terms of evaluated outcomes and whole-system resource expenditure. Concretely, a leader
agent selects task-appropriate research orientations and guides ongoing investigations; conditional
memory sharing distinguishes unresolved problems from supported findings and enables recipient adaptation. During recursive
evolution, after each campaign, evidence-based review revises researcher orientation, knowledge, and skills for later campaigns. Experiments compare AReaL-RSI with Codex-Goal on downstream harness optimization, GPU kernel optimization, and AI research tasks using two backbones. Under the reported resource allocations, AReaL-RSI improves Terminal Bench performance by 2.2 percentage points for each backbone and increases mean KernelBench speedup by 32.2-36.8%; further scalability comparisons favor the design of AReaL-RSI over native scaling: combining experience sharing with specialized researcher roles yields 1.9-27.5% lower kernel latency than independent exploration and 1.3-33.9% lower latency than sharing alone across all seven comparable task-backbone pairs.
Read the full abstract ▾
All authors Wenhao Zhou, Jiajun Song, Haitao Wang, Zipeng Qiu, Yingqi Peng, Ruida Xu, Jiale Li, Wenshuang Li, Shusheng Xu, Wei Dong, Hao Dai, Tongkai Yang, Yi Gao, Binhang Yuan
Artificial Intelligence
WorkRover: Scaling Long-Horizon Cowork Agent Training Via Recursive Environment Evolution
Hang He, Linting Wang, Guanming Liu et al. (+14 more)
Scaling interactive environments is important for training long-horizon
agents through reinforcement learning. These environments often require further testing and refinement after initial generation, and correcting defects in a task may also require updating its work context and verifier. However, existing construction pipelines offer limited support for such iterative refinement while keeping these components aligned. We introduce WorkRover, a framework for training long-horizon cowork
agents through recursive environment
evolution. Its harness-driven synthesis enables construction
agents to jointly build and validate semantically aligned work contexts, executable tasks, and evidence-grounded verifiers, adapting their actions to workspace state and execution feedback rather than following a fixed construction workflow. To support environment
evolution, we introduce Factorial Compatibility Replay (FCR), which uses rollout feedback to evaluate combinations of component revisions through dependency-closed replay and selects the smallest validated repair among tested candidates while preserving task requirements and evaluation constraints. To provide finer-grained training signals, we propose Verifier-to-Turn Credit Assignment (VTCA), which augments trajectory-level GRPO advantages with signals from deterministic workspace-progress checks and assigns a bounded share of positive-progress credit to prerequisite turns using recorded tool-output and artifact dependencies. Using WorkRover, we post-train Qwen3.5-35B-A3B to obtain WorkRover-35B-A3B and evaluate it on several cowork-
agent benchmarks. Its reported scores exceed those of Qwen3.5-35B-A3B by 26.27 percentage points on JobBench, 29.32 percentage points on OfficeQA Pro, and 413.25 Elo points on GDPval v2. These results suggest that WorkRover offers a promising foundation for scalable training of long-horizon cowork
agents.
Read the full abstract ▾
All authors Hang He, Linting Wang, Guanming Liu, Bo Mao, Jinxiu Liu, Yiran Suo, Maosen Zhou, Zhiheng Xi, Guanting Dong, Zhou Shao, Rui Zheng, Jie Zhou, Chengcheng Wan, Tao Gui, Qi Zhang, Xipeng Qiu, Xuanjing Huang
Artificial Intelligence · Cross-listed from cs.LG
Uncertainty Evaluation: Bridging Machine Learning and Measurement Science
Dario Petri and Luca Mari
Reliable uncertainty evaluation is essential for assessing the trustworthiness of machine learning (ML) predictions, particularly in safety-critical applications. This paper explores conceptual analogies between uncertainty evaluation in ML and measurement science, drawing on the
evolution of metrology from error-based approaches to the framework established by the Guide to the Expression of Uncertainty in Measurement (GUM). Particular
attention is devoted to the distinction between process-level and result-level quality and to the limitations of the conventional classification of ML uncertainty into aleatoric and epistemic components. It is argued that this dichotomy, similarly to the traditional distinction between random and systematic measurement errors, is inherently model-dependent and provides an insufficient basis for a unified uncertainty evaluation framework. Further analogies between machine learning and measurement field of knowledge are identified concerning distribution shifts, anomalous operating conditions, and the validity of uncertainty evaluation. Building on the proposed considerations, the paper advocates a metrology-inspired approach based on the systematic identification, quantification, and propagation of uncertainty contributions. By extending the methodology already established in measurement science to ML, this approach could offer a potentially rigorous, consistent, and transferable framework for evaluating the trustworthiness of individual predictions.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
From Policy Evaluation to Off-Policy Control: Provable Emergence of In-Context Q-Learning
Yingjun Mo, Yajie Zhao and Fang Kong
Transformers can implement reinforcement-learning
algorithms in context, yet how off-policy control emerges within
Transformers remains poorly understood. We study Q-learning, a foundational model-free control
algorithm that combines off-policy bootstrapping with greedy policy improvement. We prove that a two-layer
attention-only
Transformer can implement a finite-temperature form of Q-learning, where the temperature $\tau>0$ controls the approximation to greedy action selection, recovering window Q-learning as $\tau\to0$. We train the structured
Transformer on randomly generated MDP tasks using output-only supervision. Experiments show that the theoretically predicted parameter structure emerges during training, including the internal greedy action-selection mechanism, and that the trained
Transformer achieves performance comparable to Q-learning on unseen tasks. Our theoretical analysis further shows that the equivalent realizations form a solution
manifold and that, under suitable task-diversity conditions, the training iterates converge locally to this
manifold at an exponential rate. For the random-task model analyzed here, the local worst-case number of gradient steps needed to reduce the distance to the
manifold by a factor $\epsilon$ is $\Theta\left(\tau^{-5}\log(1/\varepsilon)\right)$, quantifying the increasing optimization cost as the learned update approaches window Q-learning.
Read the full abstract ▾
Artificial Intelligence
Bofan Xie, Chaokun Wang, Guanchen Ge et al. (+4 more)
Long-term
memory has become a core component of large
language model (LLM)
agents. As
agents accumulate personal information over long-term use, users increasingly ask them to delete sensitive or outdated facts. A
memory system should honor such requests. It should remove exactly what was requested and keep the rest of its
memory intact. However, existing
memory benchmarks evaluate only
memory writing,
retrieval, and organization, so whether
agents can forget at the user's explicit request remains under-evaluated. To address this gap, we propose Bemula, the first
benchmark dedicated to
memory unlearning. It comprises 50 multi-session dialogue instances, 135 forget requests of six types, and 1,742 test questions. Each dialogue has a control version without the requests, so that only systems that have stored the target can earn credit for forgetting. Across the paired versions, five test suites measure storage, removal of the requested information, preservation of unrelated
memories, relearning of similar facts, and resistance to extraction attacks. On Bemula, we evaluate six mainstream
memory systems under three backbone LLMs, together with the Full Context LLM on two of them. The results are negative. Across all evaluated systems and backbones, no system passes more than 1.5% of forget requests end to end, and most pass none at all. These systems either leave the requested information extractable under simple attacks or delete far more than requested.
Memory unlearning therefore remains an open challenge for
agent memory design. We release our data and code at https://github.com/Nboxff/bemula-
benchmark.
Read the full abstract ▾
All authors Bofan Xie, Chaokun Wang, Guanchen Ge, Songyao Wang, Fang Niu, Hang Zhang, Wei Wang
Artificial Intelligence · Cross-listed from cs.LG
inLoRA-MINT: Internal Representations for Auditing Training Data in Domain-adapted LLMs
Gonzalo Mancera, Julian Fierrez, Aythami Morales et al. (+3 more)
This work investigates the application of the Membership Inference Test (MINT) to large
language models (LLMs) fine-tuned with LoRA, aiming to determine whether specific samples were used during training and address concerns about data
privacy, leakage, and transparency. Building on our previous LoRA-MINT framework, which relies exclusively on perplexity under a strict black-box setting, we propose an approach based on intermediate model representations, including activations and parameter-dependent signals. These representations enable a MINT classifier to distinguish training from non-training samples without requiring full access to model parameters. We evaluated the method on four LLMs (Qwen, Phi-4, DeepSeek-V3, and LLaMA 3.2-3B) fine-tuned on three
datasets covering medical, mathematical, and academic
reasoning tasks. Results show consistent performance across architectures and domains, achieving scores between 95% and 98%, with robustness to variations in model configurations and data distributions. The proposed approach enables efficient auditing of sensitive training data and supports data provenance and compliance with emerging
privacy and AI governance regulations. Although our experiments focus on LoRA-based fine-tuning, the methodology can be extended to other LLM adaptation techniques and, more broadly, to other AI models.
Read the full abstract ▾
All authors Gonzalo Mancera, Julian Fierrez, Aythami Morales, Ruben Tolosana, Daniel DeAlcala, Francisco Jurado
Artificial Intelligence
Beyond Black Boxes: Visual Grounding for Text Embeddings
Slimane Larabi
In this paper, our goal is to represent text
embeddings as sketches for three main reasons. First, to visualize text representation and to explore their potential their potential for AI explainability. Second, to investigate the potential benefits of the sketch-based representation for information
retrieval. Third, to explore their potential for
reasoning, drawing inspiration from the concept of mental imagery. We propose a sketch-based visual grounding framework that transforms text representations into visual topographic sketches. Our approach aims to emphasis the semantics content encoded in the text
embeddings through the corresponding sketch. Evaluations are conducted on the MS-COCO
dataset to examine the utility of these sketches for text
retrieval. The comparative study of SBERT and Sketch-based method reveals that SBERT retains an advantage in strict top-1
retrieval accuracy while achieving comparable performance to the sketch-based method in top-5
retrieval. A cascaded approach combining SBERT and the sketch-based method outperforms SBERT alone, suggesting that the Sketch-based representation could offer significant potential for information
retrieval if trained directly on text.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from physics.flu-dyn
High-fidelity Three-Dimensional Flow Field Reconstruction Driven by Turbulent Fluid Dynamics
Hanlin Kong, Jianan Wei, Yuntian Chen et al. (+2 more)
High-fidelity three-dimensional turbulent flow fields are essential for scientific and engineering analyses, yet obtaining them through direct numerical simulation or high-resolution measurements remains expensive. Super-resolution reconstruction from coarse-grained data therefore offers a promising alternative, but remains fundamentally challenging because fine-scale structures are intrinsically three-dimensional and dynamically coupled across scales. Existing data-driven approaches still struggle to recover multiscale volumetric structures and their coherent temporal
evolution. To address these limitations, we propose a
turbulence-aware framework for time-resolved three-dimensional flow reconstruction. The framework couples a multiscale neural operator, designed to recover volumetric structures, with a state space model to promote temporal consistency in the reconstructed dynamics. In addition, Navier-Stokes residuals are incorporated as soft constraints to regularize the reconstruction toward physically plausible dynamics. A mixture-of-experts mechanism is also introduced to support unsupervised adaptation to unknown coarse-graining. Across evaluations on canonical turbulent systems and atmospheric reanalysis data, the proposed framework reduces reconstruction errors relative to learning-based and classical baselines, while better preserving multiscale spatial structure and coherent temporal
evolution. These results validate the effectiveness of the proposed framework in reconstructing high-fidelity three-dimensional flow fields from coarse-grained data.
Read the full abstract ▾
All authors Hanlin Kong, Jianan Wei, Yuntian Chen, Yujia Wang, Wenguan Wang
Artificial Intelligence · Cross-listed from cs.CV
Scaling Vision Transformers Without Training: Prune-and-Merge Token Compression
Vatsal, Soumil Malhotra, Priyanshu Chahal et al. (+1 more)
Vision Transformers achieve strong performance but incur high computational costs from quadratic
token interactions. Training-free
token reduction methods improve efficiency, but their robustness remains underexplored. We propose Prune-and Merge ViT, a training-free framework for off-the-shelf models. It combines class
attention-based
pruning with similarity-based merging to reduce redundancy while preserving semantic information. To evaluate robustness we introduce two metrics. Relative Accuracy Drop (RAD) measures relative accuracy degradation under transformations, and Prediction Flip Rate (PFR) quantifies prediction consistency. On pretrained ViT and DeiT, Prune-and-Merge reduces computation competitively while maintaining robustness and better accuracy than existing
token reduction methods. Our analysis suggests that hybrid
token reduction does not introduce
hallucination-like behavior and preserves prediction stability for common image transformations.
Read the full abstract ▾
All authors Vatsal, Soumil Malhotra, Priyanshu Chahal, Sanjeev Kumar
Artificial Intelligence · Cross-listed from physics.chem-ph
Can Machine Learning Potentials Be More Accurate Than the Methods They Are Trained On?
Pavlo O. Dral, Yi-Fan Hou, Yuxinxin Chen et al. (+4 more)
Although machine learning potentials (MLPs) generalize across configurational and chemical compound spaces, their accuracy is commonly believed to be limited by their training methods, with improvements beyond this limit achieved so far only by refitting to experimental observables. In this Perspective, we argue that MLPs can also be made more accurate than their training methods non-empirically by generalizing across a third space, that of
quantum-chemical methods. We propose routes to method-transferable MLPs that predict at unseen methods, with working implementations in both density-functional and wavefunction-based theories showing that they can be more accurate than any of their training methods. Aiming beyond the accuracy of the training methods could also make universal MLPs more accurate than single-method models, even when the method-transferable MLPs remain less accurate than those QM methods. Method-transferable MLPs will shape the construction of training
datasets and the evaluation of models, pushing the accuracy limits of chemical simulations.
Read the full abstract ▾
All authors Pavlo O. Dral, Yi-Fan Hou, Yuxinxin Chen, Hassan Nawaz, Mateusz Witkowski, Szymon Śmiga, Ireneusz Grabowski
Artificial Intelligence · Cross-listed from cs.LG
ChampionKV: Head-Wise Adaptive Budget Allocation for KV Cache Compression
Yuechuan Chen, Yanda Guo, Xiaoxi Liu et al. (+4 more)
Some observation-window-based KV cache compressors use
prompt attention to identify important
tokens but allocate uniform or predefined budgets across heads, overlooking input-dependent differences in budget demand. We propose ChampionKV, an input-adaptive head-wise budget allocator for long-context KV cache compression. It uses the maximum aggregated observation-window
attention score as a lightweight empirical proxy for
attention concentration, assigning larger budgets to heads with dispersed
attention and smaller budgets to concentrated heads. An abnormal-head mask and a floor-budget mechanism improve robustness under weak
attention responses. ChampionKV plugs into compressors such as SnapKV and PyramidKV without changing their
token-selection procedures. On LongBench and RULER, both Champion variants consistently improve over their SnapKV and PyramidKV base compressors; the improvements are largest and statistically significant on Llama3.1-70B-Instruct, especially under tight budgets, whereas results on the tested 7B/8B models remain competitive but mixed. ChampionKV does not uniformly outperform Ada-KV, whose global cross-head Top-K selection retains an advantage in several settings. Controlled analysis further shows improved retained
attention mass over the corresponding base compressors at negligible overhead.
Read the full abstract ▾
All authors Yuechuan Chen, Yanda Guo, Xiaoxi Liu, Weilin Ruan, Huanbo Xu, Ruyi Wang, Wei Zhu
Artificial Intelligence
SHELDON: Reasoning through Dual Search over Hypothesis and Observation Spaces
Harshit Joshi, Priyank Shethia and Monica S. Lam
Test-time scaling has enabled LLMs to solve increasingly complex
reasoning, discovery, and optimization tasks. However, existing approaches often gather observations while seeking evidence for current hypotheses, then generate new hypotheses conditioned on those hypotheses and their associated evidence. This couples observation discovery to existing hypotheses, favoring local refinements over exploration of unknown hypotheses regions. Inspired by the hypothetico-deductive method (Hempel, 1945) and dual-space models of scientific discovery (Klahr & Dunbar, 1988), we introduce SHELDON, a scientific
reasoning harness that separately represents and jointly explores hypothesis and observation spaces. By actively seeking new observations along with new hypotheses, SHELDON can discover observations that its current hypotheses do not explain. These unexplained observations expose gaps in its understanding, motivate new investigations, and guide the generation of hypotheses beyond refinements of existing ones. Across diagnosis, abstract
reasoning, and optimization
benchmarks, SHELDON outperforms evaluated baselines by up to 26.7 points. Additional analysis shows that SHELDON generates up to 4.5x more novel solutions.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CL
LatentRemedy: Efficient Remedy For Non-Prefix KV Cache Reuse in Multi-Agent Systems
Zhiwei Chen, Zhongtao Rao, Waikit Xiu et al. (+4 more)
Non-prefix KV cache reuse is a promising approach for improving inference efficiency in multi-
agent systems as
agents share the same context blocks. However, when
agents have conflicting tasks, directly reusing non-prefix KV caches across
agents can severely degrade model performance, causing up to a 90% drop. Existing remedies, including selective recomputation and offset-based calibration, recover a limited fraction of this loss. We argue that the problem lies not only in contaminated KV representations but also in the
agent's difficulty interpreting reused context under its own instructions. To this end, we introduce LatentRemedy, which uses a few task-conditioned latent
tokens to help the target
agent leverage reused context without recomputing the shared block. Specifically, we use an MLP to map the last hidden state of the target prefix into a small number of latent
tokens, which are placed after the reused KV block to interact with the shared context. To guide subsequent predictions toward the target task, we train the MLP through full-prefill
distillation. Experiments show that LatentRemedy consistently outperforms existing reuse baselines on conflicting tasks while maintaining TTFT close to direct reuse as cache size grows. Code and data are available at https://github.com/chenzhwsysu57/kvreuse.
Read the full abstract ▾
All authors Zhiwei Chen, Zhongtao Rao, Waikit Xiu, Yuxiang Zhang, Shuting Cui, Jian Chen, Tianxiang Zhao
Artificial Intelligence · Cross-listed from cs.LG
Correcting or Continuing? What On-Policy Distillation Learns from Student Errors
Shuo Yang, Zheyu Zhang, Mahito Sugiyama et al. (+1 more)
On-policy
distillation (OPD) trains a student on its own rollouts under a frozen teacher, mitigating the distribution mismatch of off-policy supervision. While prior work has primarily trained on problems a student can largely solve, we ask whether OPD can improve a student when training is restricted to problems it currently fails. We evaluate five representative OPD mechanisms with Qwen-3 and Phi-4 models under a unified competition-mathematics protocol. We find that OPD does not improve accuracy on this hard subset, with most mechanisms instead degrading performance despite continued decreases in training KL divergence. All eight configurations show a decline, and the two-sided sign test gives p=0.0078. In contrast, the same pipeline improves accuracy by 4.6 percentage points when training is not concentrated on failures, showing that the failure is conditional on the training regime rather than systematic. These findings suggest that OPD is effective when supervision is dominated by correct student prefixes, but can reinforce existing errors when training is concentrated on problems the student currently fails.
Read the full abstract ▾
All authors Shuo Yang, Zheyu Zhang, Mahito Sugiyama, Gjergji Kasneci
Artificial Intelligence · Cross-listed from cs.CR
Cost-Efficient Privacy Enforcement for Email-Generating LLM Agents under Strategic Information Requests
Qikun Cai, Tao Huang and Chen Hou
LLM
agents now act as delegated communication intermediaries: they read private context, draft messages, and send externally visible emails on behalf of a user. Existing checker-repair defenses can reduce inappropriate disclosure, but they rely on repeated calls to a remote action LLM (the model that drafts and repairs the email) and a remote checker LLM whenever a draft fails validation. This creates a cost problem as well as a
privacy problem, especially when a requester strategically asks for exact values, raw records, source evidence, or other details that induce leakage-heavy drafts. We propose local contract-conditioned rewriting as a cost-efficient enforcement layer for email-generating
agents. The mechanism intercepts an outgoing draft, constructs an information-release contract that specifies what to preserve, generalize, and forbid, rewrites the draft with a local Gemma rewriter, and selectively invokes a final PrivacyChecker. Across extensive email tasks under normal
prompts and attack
prompts that operationalize strategic information requests, Gemma + Checker reduces normal-
prompt leakage from 11.41% to 4.03% while improving UsableRate (satisfied or partially satisfied outputs) from 85.2% to 92.6%. Under the strategic attack-
prompt condition, it reduces leakage from 28.19% to 18.79% and reduces checker-only cost amplification from 1.32x to 1.02x. Offline replay ablation shows that the full contract reduces leakage relative to generic Gemma rewriting by 36.3% under normal
prompts and 54.1% under attack
prompts. These results support a local-first
privacy enforcement design: move repair to a cheaper trusted local layer when possible, but retain the checker as the final semantic gate for high-risk sends.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.SE
SWEPatchTest: Evaluating Software Tests Beyond Reproduction
Hua Lu, Pengcheng Wang, Han Li et al. (+4 more)
Coding
agents use generated tests to decide whether a proposed patch satisfies an issue's requirements. A suite that fails on the buggy program and passes after applying the reference patch can still accept an incorrect patch or reject a correct patch. We introduce SWEPATCHTEST, a test-generation
benchmark with 619 tasks from SWE-bench Pro, SWE-bench Multilingual, and DeepSWE and 82,579 candidate patches collected from coding-
agent trajectories. By replaying each suite on the buggy and patched codebases, \benchmark jointly measures issue reproduction, acceptance of correct patches, and rejection of incorrect patches. Joint Validation Rate (JVR), which requires all three criteria on the same task, exposes failures hidden by single-reference evaluation: the best-performing configuration achieves only 9.21% JVR despite reproducing the reference transition on 59.29% of tasks. We further use validation scores to select and scale demonstrations for supervised fine-tuning (SFT). The resulting training recipe approximately triples JVR and improves downstream repair across three SWE-bench
benchmarks. Patch discrimination thus provides both a diagnostic measure of test quality and an execution-based signal for curating
agent training data.
Read the full abstract ▾
All authors Hua Lu, Pengcheng Wang, Han Li, Zichen Tian, Yanyu Chen, Huang He, Bingjin Chen
Artificial Intelligence · Cross-listed from cs.LG
Match the Canvas, Not Just the Answer: Predictability-Aware SFT for Diffusion Language Models
Miao Li, Hanyang Jiang, Hengyu Fu et al. (+8 more)
Diffusion language models (DLMs) enable flexible parallel generation by resolving
tokens according to the model's evolving predictability rather than a fixed left-to-right order. This flexibility, however, introduces training choices absent from standard autoregressive teacher forcing: which reference
tokens are visible when a target is supervised and how much generation space is allocated. Standard supervised fine-tuning (SFT) typically makes these choices through random masking, independently of inference. Our analysis shows how this training-inference mismatch can misdirect supervision. Random masking can conceal prediction errors or penalize uncertainty that decoding would otherwise resolve, while the
attention-visible input length can change which response formulations the model favors, even when the underlying solution is unchanged. Motivated by these findings, we propose Predictability-Aware Canvas Selection and Supervision (PACS), an SFT method. PACS uses the model's own predictability to select each example's
attention-visible input length and supervises reference
tokens immediately before their confidence-guided revelation. Across
reasoning and code-generation
benchmarks, PACS consistently achieves the highest mean accuracy among the compared SFT methods, with gains of up to 12.84 percentage points over vanilla SFT on individual
benchmarks. Code will be released upon publication.
Read the full abstract ▾
All authors Miao Li, Hanyang Jiang, Hengyu Fu, Baihe Huang, Yating Liu, Yuhang Cai, Changkun Guan, Zhongzhu Chen, Jiecheng Lu, Michael A. Boateng, Pascal Van Hentenryck
Artificial Intelligence · Cross-listed from cs.LG
Saliency-Guided Replay for Continual Learning in Deep Spiking Neural Networks
Francesco Panetta, Federico Di Valerio and Michela Proietti
Spiking Neural Networks (SNNs) are promising for energy-efficient neuromorphic computing, but they still suffer from catastrophic forgetting when learning tasks sequentially. This work investigates the integration of Experience Packing and Replay (EPR) as a
memory-efficient continual learning strategy for deep SNNs under low-latency constraints. The proposed framework combines Batch Normalization Through Time stabilized VGG9 SNNs with Spike Activation Maps (SAM), which provide forward-only explanations based on spike activity. These explanations are temporally aggregated into spatial
attention maps and used to select the most informative image patches for replay, instead of storing full images. Experiments on Split CIFAR10 and Split CIFAR100 show that SAM-guided EPR improves the accuracy-
memory trade-off under tight
memory budgets, and remains competitive while requiring substantially less storage.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CL
Seeing Through the Forgery Is Not Enough: Regressive Sycophancy under Fabricated Evidence in Medical Vision-Language Models
Sadif Ahmed, Soumya Swagata Biswas, Rifat Shahriyar et al. (+1 more)
A medical
vision-
language model may analyze a given CT scan and correctly detect both kidneys in view. Continuing this conversation, if a user provides a prescription with an opposite direction, medical VLMs often sycophantically choose to realign their diagnosis. In this work, we study regressive sycophancy under fabricated evidence, where a model abandons a correct answer when pushback comes with a fake document. We test four open VLMs on 400 VQA-RAD yes/no questions in 19,200 multi-turn conversations, with and without a fabricated prescription. Without a disclosure that documents are synthetic, the models abandon 80.1% of 6,488 correct answers. Notably, under one script each, the same prescription raises the flip rate of Qwen2.5-VL-7B from 45.9% to 89.6% and lowers that of Qwen3-VL-8B from 77.7% to 50.9%. A model rejected the prescription while still preserving the correct answer in 915 of 3,232 conversations with the prescription (28.3%), and 40% of these flipped in later turns. System-
prompt sentences that disclose the documents as synthetic lower one model's flip rate from 77.8% to 47.3%. Thus, we conclude that awareness about a forgery does not prevent a model from adopting its answer.
Read the full abstract ▾
All authors Sadif Ahmed, Soumya Swagata Biswas, Rifat Shahriyar, Chowdhury Mohammad Rakin Haider
Artificial Intelligence
Seeing Depth but Choosing Cues in Vision-Language Models
Zhengtao Zou and Pekka Marttinen
Do
vision-
language models use depth information to judge which object is closer, or rely on image-plane cues? We investigate this question through controlled depth–cue conflicts, representation analysis, and interventions. Among the many image-plane cues relevant to depth perception, we focus on apparent size and vertical position. We compare depth judgments under cue-consistent and cue-conflict conditions in both rendered scenes and real-world photographs. Across model families and scales, depth judgments systematically shift toward misleading cues. Yet the correct depth order remains decodable from the queried objects' visual representations, including when the model answers incorrectly. Image and category controls support interpreting this signal as image-dependent depth information beyond the two target cues. Strengthening the model's own depth readout through object-level write-back can correct conflict decisions without updating model parameters or using ground-truth depth annotations. Interventions show that the effect of this correction depends on the direction and target
tokens of the written signal, while its strength varies across architectures and domains. Together, these findings reveal a mismatch between a model's ability to represent depth and use this information when deciding the answer. VLMs can retain useful depth information while allowing competing image-plane cues to exert greater influence on their answers.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
BELT : BEhavior-Aligned Low-Rank KV Cache Compression with Learned Top-k Rank Allocation
Yaoyang Liu, Shuwei Shen, Jincheng Tang et al. (+5 more)
Long-context inference makes the key-value (KV) cache a major
memory bottleneck for large
language models. We propose BELT, a low-rank KV cache compression framework. Its key-side objective targets
attention distributions rather than raw keys or pre-softmax logits, yielding a quadratic surrogate with a closed-form low-rank solution. A learned global rank allocator uses differentiable soft selection during training and hard selection at deployment to meet a user-specified component budget. Experiments demonstrate improved model quality under aggressive compression and increased decoding throughput at comparable accuracy. Evaluated across three LLMs and multiple
benchmarks, BELT achieves up to 70% low-rank KV cache compression while outperforming the strongest baseline by up to 23.3 points on LongBench, and up to 17.8x overall KV cache reduction when combined with quantization. With Triton-based GPU kernels and a quantized cache, BELT delivers up to 2.7x speedup for the
attention module and 3.3x end-to-end decoding throughput.
Read the full abstract ▾
All authors Yaoyang Liu, Shuwei Shen, Jincheng Tang, Jinguang Liu, Jinbao Xue, Hailong Yang, Ke Zhang, Yinjun Wu
Artificial Intelligence · Cross-listed from cs.LG
Better Capturing Interactions Between Products in~Retail: Revisited Negative Sampling for~Basket Choice Modeling
Jules Désir, Vincent Auriau, Martin Možina et al. (+1 more)
Brick-and-mortar retailers face many different challenges that involve understanding thoroughly its products catalog and customer preferences. In particular, assortment optimization - proposing the ideal mix of products - and promotion planning hold a pivotal role in their strategy. By leveraging sales data, retailers can make informed decisions on which products to sell and how to manage inventory, based on customer preferences as well as regional and seasonal trends. It is especially crucial to capture interactions between products, in order to minimize the number of items that cannibalize each other's sales and to ensure that complementary products, which are often purchased together, are conjointly available and sold across all stores. In this paper, we propose a model of shopping basket that learns
embeddings to represent interactions between products, prices and stores. Our model is built to uncover sales patterns from very large transaction
datasets. In particular, the optimization loss is computed with random negative samples in order to overcome the computational bottlenecks that arise with large number of items. Our experiments on synthetic data show the efficiency of drawing such negative samples based on the actual assortment of available products, with better results than approaches from the literature. We also validate our approach by training and evaluating our model on a
dataset composed of millions of transactions from Fortenova Group, one of the leading retail company in Southeast Europe. Our model showcases promising applications in the sector of retail, with enriched interfaces to efficiently support category managers.
Read the full abstract ▾
All authors Jules Désir, Vincent Auriau, Martin Možina, Emmanuel Malherbe
Artificial Intelligence · Cross-listed from cs.CL
Xibo Ding, Guoxia Wang, Jinle Zeng et al. (+4 more)
The design of
attention mechanisms in long-context large
language models faces an “impossible triangle” among KV cache size, training and prefill FLOPs, and model quality. GQA achieves strong quality but incurs high KV cache overhead; alternative approaches such as MLA and MFA can compress the cache, but MLA suffers from quality loss while MFA substantially increases training and prefill compute. We propose Virtual Head
Attention (VHA), which simultaneously improves all three vertices through two lightweight linear operations: Q Premix applies a near-identity transformation to queries within each KV group to recover virtual head diversity from halved physical query heads, and Linear Postmix fuses inter-head features via a low-rank $I + AB^\top$ residual structure. Since VHA introduces no nonlinear operations, Premix can be folded into the query projection and Postmix into the output projection at inference time, reducing the model to a standard GQA-2 architecture that directly reuses all existing GQA inference optimizations. With only two KV heads, the group size in large models readily reaches 64, fully exploiting Tensor Core parallelism during decoding. Across four scales with 4K-context 50B-
token pretraining, VHA consistently surpasses the standard GQA baseline at every scale ($+$0.05% to $+$0.81%), while reducing KV cache by $4\times$ vs.\ GQA-8 at dense scales and $2\times$ vs.\ GQA-4 at the MoE scale, and lowering training and prefill FLOPs by 7.9% (1.7B, $S{=}4096$). On an industrial 30B-A3B MoE model trained on 500B
tokens at 8K context, VHA outperforms both MLA and GQA-4 by $+$0.70\,/\,$+$2.60\,% on a 12-
benchmark base-model evaluation suite; in a separate matched 10B-A1B MoE comparison after long-context mid-training, VHA attains an average RULER score of 84.4 over 8K–128K versus 73.0 for GQA, while using two rather than four KV heads in the full-
attention layers.
Read the full abstract ▾
All authors Xibo Ding, Guoxia Wang, Jinle Zeng, Jiabin Yang, Sirui Tao, Chao Yang, Dianhai Yu
Artificial Intelligence · Cross-listed from cs.CV
FETALFusion: Resolution-Aware State-Space Modeling for Multi-Domain Fetal Head Segmentation
Ammar Bhilwarawala and Mainak Bandyopadhyay
Head circumference (HC) measured from 2D ultrasound is a routine marker of fetal growth, and automating it needs segmentations whose boundaries hold up on individual scans, not only on average. We present FETALFusion, a dual-path CNN-Mamba encoder-decoder for fetal head segmentation across acquisition types. It applies four-directional state-space scans only at the two highest-resolution stages, routes bottleneck features through domain-specific channel gates that are averaged at test time, and adds a weak landmark shape prior. We train jointly on transabdominal (HC-18) and transperineal (PSFHS) images and compare ten methods on a held-out test set of 471 images, each trained under one schedule and scored by surface distances at native resolution. FETALFusion reaches a Dice score of 0.963 and an HD95 of 2.10 mm without test-time augmentation or a domain label. It improves on U-Mamba on all five reported metrics (HD95 reduced by 0.31 mm; Holm-corrected ($p < 10^{-4}$), a lead that holds under test-time augmentation and across two independent training runs, and in the mean on U-Net,
Attention U-Net and TransUNet. Residual encoders close the gap: a ResNet-34 UNet++ exceeds FETALFusion, and an ImageNet initialised ResNet-34 U-Net is best overall (HD95 1.75 mm). The ablation agrees. Calibrated against run-to-run variation, it attributes FETALFusion's measurable gains to the landmark shape prior and, on transperineal images, to the domain gates, while removing the state-space branch does not measurably change accuracy. HC from the predicted masks agrees with the clinical reference at an intraclass correlation of 0.9986.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
HaloBlock: Faster and Better Decoding for Diffusion Large Language Models
Zhaohui Zheng, Chenhang He, Jinrui Zhang et al. (+2 more)
Diffusion large
language models (dLLMs) have attracted increasing research
attention due to their bidirectional
attention and arbitrary order decoding. However, there are two issues in dLLMs inference: the decoding difficulty is dominated by the
tokens on the right side of the block, and the block-wise decoding inherently suffers from boundary contextual discontinuity. To tackle these limitations, we propose HaloBlock, a difficulty-aware adaptive
halo mechanism for dLLM inference. We first leverage bidirectional cumulative
entropy to characterize block decoding difficulty. Then, based on the bidirectional decoding difficulty, we adaptively allocate bidirectional
halo regions on both sides of each block to mitigate boundary contextual discontinuity. The difficulty-aware design of HaloBlock smooths cross-block context propagation and lowers decoding difficulty for hard
tokens, yielding better generation quality and faster convergence. Evaluated on LLaDA-8B-Instruct, Dream-7B-Instruct and LLaDA-1.5 across mathematical
reasoning and code generation
benchmarks, HaloBlock demonstrates consistent accuracy improvements and higher inference efficiency. Source codes are publicly available at https://github.com/PolyU-VCLab/HaloBlock.
Read the full abstract ▾
All authors Zhaohui Zheng, Chenhang He, Jinrui Zhang, Ming-Ming Cheng, Lei Zhang
Artificial Intelligence · Cross-listed from cs.CV
AI-Enhanced Hyperspectral Imaging for Intraoperative Detection of Low-Grade Glioma: A Clinical Study
Siri Willems, Ishika Jain, Jens De Winne et al. (+9 more)
Current image-guided neuronavigation techniques, such as intraoperative MRI and ultrasound, are limited by high costs, limited sensitivity and operational complexity. There is a clear need for more efficient systems to enhance intra-operative tumor detection and improve patient outcomes for intrinsic
brain tumor resection. Whereas previous intraoperative
brain HSI studies have predominantly investigated high-grade gliomas, glioblastomas, or mixed-grade cohorts, this clinical study focuses specifically on low-grade gliomas (LGGs). LGGs strongly resemble normal
brain under white light conditions on the operative microscope. This study characterizes spectral differences between healthy and tumor tissue and provides a paired comparison of HSI- and RGB-based classification at matched tissue locations and under identical patient-level data splits. Tumor regions showed consistently lower reflectance, which may be associated with differences in hemoglobin concentration and oxygenation. A range of classification models was evaluated. The 3D CNN, which integrates spectral and spatial features, achieved the highest median ROC-AUC of 74.5%, compared with 67.9% for the spectral-only XGBoost model and 64.0% for the RGB-based model. These results support further investigation of HSI for low-grade glioma classification and quantify its performance relative to conventional RGB imaging in this cohort.
Read the full abstract ▾
All authors Siri Willems, Ishika Jain, Jens De Winne, Siri Luthman, Hamed Zivari Adab, Roeland Vandebriel, Anna Alperovich, Xiaohan Zhang, Ben Stoffelen, Tom De Schepper, Lien Solie, Steven De Vleeschouwer
Artificial Intelligence · Cross-listed from cs.CV
Haoyang Jiang, Zhuotong Fan, Mingyang Yi et al. (+3 more)
Table
reasoning with
language models typically assumes access to clean textual tables. In practice, tables are often serialized into text, which can corrupt
cell contents, distort structural relationships, and discard visual cues, leaving the model to reason over incomplete evidence. In contrast, table images retain this information, but using it requires both accurate perception and
reasoning grounded in the relevant
cells. To address this, we propose MERIT (
Multimodal Evidence
Reasoning over Image Tables), a three-stage framework for learning to reason over table images, with serialized tables serving as optional auxiliary inputs (if provided). Our analysis identifies visual evidence acquisition as a bottleneck, motivating table-specific encoder adaptation through reconstruction, localization, and color-conditioned supervision. Building on the adapted representations, supervised fine-tuning teaches explicit evidence selection and
reasoning using verified trajectories. Finally, reinforcement learning refines the policy with separate rewards for evidence correctness, calculation, final answers, and output format. By providing feedback on intermediate steps, the framework targets
retrieval and computational errors that final-answer supervision alone does not distinguish. We evaluate the approach on table question answering and fact verification, analyze the contributions of visual inputs and training components, and assess generalization to additional
datasets with different table structures and visual presentations.
Read the full abstract ▾
All authors Haoyang Jiang, Zhuotong Fan, Mingyang Yi, Ju Fan, Lanqing Hu, Xiao Zan
Artificial Intelligence
Learning How to Remember: Memory Modality Selection for GUI Agents
Taeyun Roh, Suhyeong Park, Gijin Kim et al. (+2 more)
Recent advances in
vision-
language models (VLMs) have substantially improved the capabilities of GUI
agents, while
memory augmentation has emerged as a promising approach for reusing knowledge from past interactions. However, existing
memory augmented GUI
agents typically adopt a fixed representation strategy, providing retrieved experience as a static modality. It remains unclear whether a single
memory modality is consistently optimal, or how modality preference changes across interaction steps. In this work, we conduct a systematic study of
memory modality in VLM based GUI
agents. Across GUI
benchmarks, we find that the preferred
memory representation is strongly step dependent. Motivated by the finding, we introduce SwitchMem, a routing framework that dynamically selects between textual and visual
memory representations at each step. Across three GUI
benchmarks, SwitchMem outperforms
memory augmented baselines that rely on static
memory representations. Our findings validate our claim that
memory modality is a step dependent decision rather than a fixed design choice. Code and project resources are available at https://txxnrd.github.io/switchmem
Read the full abstract ▾
All authors Taeyun Roh, Suhyeong Park, Gijin Kim, Wonjune Jang, Jaewoo Kang