Artificial Intelligence · Cross-listed from q-bio.NC
Cross-attention encoding models reveal dynamic spatiotemporal routing across human higher visual cortex
Iishaan Inabathini and Margaret M. Henderson
Understanding how the
brain parses actions and events from time-varying natural inputs is a central challenge in neuroscience. Recent work has used deep neural network (DNN) models to build stimulus-computable fMRI encoding models that predict single-voxel responses to complex natural
videos. However, the majority of
video-computable encoding models predict responses using simple linear mappings from model
tokens, overlooking the spatiotemporal structure shared by
video representations and neural responses. Recent cross-
attention encoding models address this limitation for static images, enabling flexible stimulus-dependent weighting of image content across space. Here, we extend this framework to naturalistic
video, using per-parcel cross-
attention to dynamically route features from a self-supervised
video model (V-JEPA-2) across both space and time, fitting this model to fMRI responses to short
video clips. We compare joint spatiotemporal
attention with factorized and selectively constrained alternatives, and find that joint routing improves predictions of
brain responses to held-out
videos across higher visual regions, most consistently in lateral and dorsal visual areas associated with dynamic motion perception. Moreover, our method provides interpretable, stimulus-specific
attention maps that dynamically follow moving objects, revealing which locations and temporal moments contribute to each neural response. We further show that
attention maps from parcels in different category-selective networks (face-, body-, scene-selective) differentially weight content in accordance with expected semantic selectivity. Together, this work provides a new computational framework for understanding how visual information is adaptively weighted by cortical populations during dynamic visual perception.
Read the full abstract ▾
Artificial Intelligence
Engineering Simplicity: Simple Mechanism Interfaces Steer LLM Agents
Kehang Zhu, Anand Shah and David Parkes
Can interaction formats and textual scaffolds help large
language model (LLM)
agents make better decisions, and do better decisions come with better explanations? We study these questions in auctions and matching, multi-
agent environments with explicit rules and known optimal strategies. These settings let us vary how a decision problem is presented while retaining a
benchmark for evaluating behavior. Drawing on human-motivated theories of simplicity, we compare interfaces that elicit a complete bid or ranking with sequential interfaces that make safe choices easier to identify. We then hold the interaction format fixed and vary
reasoning scaffolds and rule descriptions. Across four model families, the ascending auction interface substantially reduces bid deviations. The matching comparison also shows why sequential responses require different error accounting from complete rankings. Laying out payoff contingencies and explaining why truth-telling is safe also improve choices, whereas
prompts to plan through matching rounds or form beliefs about opponents worsen play overall. In auctions, these behavioral gains are not accompanied by corresponding improvements in measured verbal indicators of strategic understanding in the
agents' short stated plans. Other
prompts change those indicators without improving bids. Our findings suggest that human-motivated theories of simplicity can inform the design of decision environments for artificial
agents. They also show why scaffolds should be evaluated through realized choices as well as explanations: improvements in one need not appear in the other.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CV
Compress to Remember: Learning Compact Memory via On-Policy Distillation for Long Video Generation
Xiaoyu Wu, Weihang Guo, Yifei Wang et al. (+3 more)
Standard
video generators do not natively compact historical context into reusable
memory tokens. As generation continues, the growing history makes it increasingly difficult to retain information from earlier frames due to long-context degradation. Key-frame-based approaches address this challenge by retaining selected past frames, but can discard information needed for future generation. Rather than relying on frame selection alone, we study whether a frozen
video generator can supply the supervision needed to learn a compact representation of the history. We propose Prediction-Aligned Context Compaction (PACC), which uses a learned compressor to aggregate information across past frames into compact
memory tokens. We train the compressor through on-policy
distillation, using the same frozen generator both as a student when conditioned on compressed
memory and as a teacher when conditioned on the full history. The student generates continuations, while the teacher provides targets for the same noisy inputs at each denoising step. Only the compressor is updated to align the student's predictions with these targets. We evaluate PACC on MBench, which jointly measures
memory-event coverage and consistency. PACC outperforms the strongest baseline by 6.63 points on Causal-rCM and 3.19 points on Causal Forcing. Evaluation on VBench-Long using MovieGen
prompts further shows that PACC produces minute-long
videos with generation quality competitive with baselines. Together, these results show that learning to compact historical context can improve long-
video memory without modifying the underlying generator.
Read the full abstract ▾
All authors Xiaoyu Wu, Weihang Guo, Yifei Wang, Xinze Feng, Lydia E. Kavraki, Zhiwei Steven Wu
Artificial Intelligence
Better Nearest Neighbor Graph Indices via (Efficient) LLM-Guided Pruning
Fangzhou Wu, Haike Xu and Sandeep Silwal
Graph-based approximate nearest neighbor search (ANNS) is widely used for large-scale semantic search. Its indices are constructed primarily based on
geometric relationships among
embeddings of an input
dataset (e.g., documents or images), rather than explicitly optimizing for semantic relevance. However, when using these indices for downstream query
retrieval, performance is evaluated based on the semantic relevance of the retrieved results to the query. This creates a fundamental "geometry-semantic" mismatch between how the indices are constructed and how their
retrieval results are evaluated. While existing LLM-based reranking methods can partially mitigate this mismatch at query time, they leave this underlying structural problem in the graph unresolved. We therefore propose LLM-Guided Graph
Pruning (LGP), a general framework that addresses this mismatch directly by leveraging LLM
reasoning to refine an existing ANN graph index itself. LGP identifies structurally "low-value" neighbors of nodes and replaces them with LLM-selected alternatives that provide useful semantic information while retaining desired
geometric structures of the original graph, including sparsity and efficient navigability. Experiments on representative semantic
retrieval benchmarks show that LGP consistently improves end-to-end
retrieval performance over both vanilla greedy graph search and LLM-based reranking across widely used graph-based ANN indices such as DiskANN and HNSW.
Read the full abstract ▾
Artificial Intelligence
HyperZip: Efficient Data Compression through Personalized Diffusion LLMs with Hypernetworks
Thai Nguyen, Khang Tran and NhatHai Phan
Large
language models (LLMs) have shown strong potential for lossless data compression, but existing approaches are constrained by the high computational cost and low throughput of autoregressive decoding. We propose HyperZip, an efficient and scalable LLM-based compression framework that leverages
diffusion-based LLMs (dLLMs) with Multi-
Token Prediction (MTP) to accelerate LLM-based data compression processes. We identify a trade-off in
diffusion-based compression, where increasing decoding throughput degrades the compression rate. To mitigate this trade-off, HyperZip employs a hypernetwork to generate data-specific updates from a context representation, adapting the dLLM to the target data without costly fine-tuning, resulting in a low compression rate and high throughput. Extensive experiments show that HyperZip achieves a superior trade-off between compression rate and speed compared with state-of-the-art baselines.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
Explainability from Training with Applications to TCR-Epitope Prediction
Jiarui Li, Zixiang Yin, Samuel Landry et al. (+2 more)
Deep learning models have achieved strong performance in artificial intelligence for science, yet their black-box nature limits our understanding of how they learn scientific tasks. Existing methods for interpretability provide limited insight into how models organize evidence and evolve during learning. We introduce explainability from training (EFT), a model-agnostic paradigm that traces model interpretation during training to explain why models rely on specific features and how they organize these features as predictive evidence. We apply EFT to four state-of-the-art T
cell receptor (TCR)-epitope prediction models, TCR-SRIM, TULIP, MixTCRpred, and NetTCR-2.2, spanning post-hoc and interpret-by-design approaches as well as
transformers and CNNs. To investigate how structural information affects model explanations, we introduce a
benchmark, TCR-XAI2, containing 388 unique experimentally resolved TCR-epitope structures, complemented by structures predicted using AlphaFold3, Boltz-2, TCRModel2, tFold-TCR, and OpenFold3. Using EFT with TCR-XAI2, we demonstrate that (1) CNN and
transformer models exhibit distinct learning trajectories; (2) TCR $\alpha$ and $\beta$ evidence can conflict during learning, limiting the benefits of jointly modeling both chains, while MHC information mitigates this; and (3) real versus predicted structural data for TCR-epitope prediction exhibits distinct TCR and peptide feature preferences as well as differing trajectories of model certainty.
Read the full abstract ▾
All authors Jiarui Li, Zixiang Yin, Samuel Landry, Zhengming Ding, Ramgopal Mettu
Artificial Intelligence · Cross-listed from cs.CV
Xiaoyu Wu, Yifei Wang and Chen Wei
Generative and representation learning remain asymmetrically connected: semantic representations are used to improve
diffusion generation, whereas the models' own representations are often treated as a by-product of synthesis. We ask whether
diffusion models can instead be trained to learn substantially stronger semantic representations without sacrificing generation quality. SelfFlow takes a step in this direction by introducing self-supervised patch
alignment into flow matching, but its main gains remain in faster convergence and improved generation. Inspired by DINO and iBOT, we extend this framework with cross-view class-
token alignment to further strengthen semantic representations. Specifically, we form two independently noised, dual-timestep observations of each image and align each student class-
token representation with the stop-gradient EMA-teacher target from the other observation. This objective is optimized jointly with the inherited flow-matching and local patch objectives. Notably, although the additional objective acts only on the class
token, it strengthens both class-
token and patch representations. Compared with a matched two-view baseline, ImageNet linear-probing accuracy improves by 9.4% using the class
token and 10.1% using mean-pooled patch
tokens, while frozen-backbone VOC2012
segmentation improves by 3.6 mIoU. These representation gains are achieved while maintaining comparable ImageNet generation FID. In text-to-image training, the same objective also improves generation FID, reducing it from 2.52 to 2.37 at matched checkpoints. Our results show that representation need not remain a by-product of generation or merely a tool for improving it: it can be directly optimized as a first-class capability of
diffusion pretraining alongside generation.
Read the full abstract ▾
Artificial Intelligence
ThuRunel: Dynamic Decoupling for Structured Advisory Dialogue
Yuyan Chen
High-stakes advisory domains such as medical aesthetics, legal consultation, and educational planning exhibit a two-phase structure. The early phase requires empathetic elicitation and emotional support, and the late phase requires authoritative specialist judgment. Neither fully automated
agents nor human junior consultants adequately address this structure at scale. We formalize the core design challenge as dynamic decoupling, asking how an AI advisory
agent should decide what to ask, when to stop, what to resolve autonomously, and what to forward to the specialist. We present ThuRunel, an advisory
agent combining a finite-state belief management framework, a chain-of-thought teacher synthesis protocol, and learned generation adapters. Against eleven baselines, ThuRunel achieves consistent improvements in elicitation completeness and specialist brief quality. ThuRunel is publicly deployed as a bilingual web application in which the same decoupling decisions operate from the client's side, grounded in a curated knowledge base that cites its sources in every answer.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.RO
ATLAS: Aligned Transport of Latent Structure for Reliable World Model Planning
Ke Fang, Yupu Yao and Lu Cheng
Latent world models rely on representation geometry for planning, yet regularizing the latent marginal alone does not determine the state-to-state relationships used for action selection. We show that this can cause planning-relevant novelty structure to be weakened as representations are transformed into the final latent used by the planner. We introduce Aligned Transport of Latent Structure (ATLAS), a training objective that explicitly preserves relational geometry while calibrating the global latent distribution. ATLAS transfers normalized pairwise structure from an informative encoder representation to the planning latent and uses Wasserstein
embedding matching (WEMReg) to calibrate its marginal through one-dimensional Wasserstein-2 transport. Our analysis shows that relational preservation and marginal calibration impose non-redundant constraints, and connects finite-candidate planning stability to relational distortion, latent-scale mismatch, and prediction error. Instantiated in LeWM, ATLAS improves mean goal-reaching success across PushT, TwoRoom, and OGBench-Cube on both lower- and higher-novelty evaluation subsets, with the largest gain on higher-novelty TwoRoom episodes. Representation and rollout diagnostics further show stronger novelty-related structure in the planning latent, improved marginal calibration, and lower multi-step prediction error. Together, these results highlight preservation of planning-relevant latent geometry as an important ingredient for reliable world-model planning. Code is available at https://anonymous.4open.science/r/atlas-world-model-72C4/.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CR
Calibrating One-Round Membership Inference with Neighbors
Francesco Rita, Jie Zhang and Florian Tramèr
The state-of-the-art Membership Inference (MI) methods calibrate their signal separately for each example using reference models, auxiliary models trained to exclude the target. This paradigm scales poorly to modern large models, however, whose training is too expensive to replicate. This has motivated one-round settings, where only a single trained model is available; but without reference models the per-example calibration that drives the strongest attacks can no longer be estimated, leaving the membership signal weak. We ask whether neighbors of the target point can recover this calibration without training any additional model. Our key observation is that reference models serve only to reveal how an example behaves under models not trained on it, and that querying the target model on nearby samples yields the same information. We propose two complementary ways to obtain such neighbors, and show that querying them against an early training checkpoint further sharpens the signal. We evaluate across three image classification
datasets and three training setups, showing that neighbors yield strong membership signals and competitive attack performance at no additional training cost.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CR
DecoyTrace: Toxic Decoys for Active Defense in Decentralized Federated Learning
Pedro Beltrán-López, Enrique Tomás Martínez Beltrán, Pantaleone Nespoli et al. (+2 more)
Decentralized
Federated Learning (DFL) eliminates the central aggregation server, reducing the single point of observation that traditional defenses against attacks rely on. As a result, peer-to-peer networks become exposed to malicious updates containing
backdoors or semantic poisoning, since such updates can remain close to benign ones in the parameter space while behaving very differently. This may evade defenses based on passive parameter inspection. However, existing deception-based defenses have mainly been designed for centralized FL and do not jointly address local observation, poisoning propagation, source attribution, and containment in strictly serverless DFL. To address these limitations, this paper presents DecoyTrace, a proactive cyber deception-based defense for strictly serverless DFL environments. DecoyTrace deploys a mobile DecoyNode that generates decoy challenges using chaotic maps, disseminates a dual model (clean vs. decoy) based on neighbor trust, and evaluates them using three-state semantic metrics. Upon confirmation, a distributed protocol isolates the source and performs a model reset or recovery to preserve training progress. Evaluated across sixty configurations on the NEBULA platform (five
datasets, three
topologies, and four attack/defense scenarios), DecoyTrace systematically restores lost utility. The F1-score remains within 0.03 of the baseline on MNIST/FashionMNIST (mitigating drops of up to 0.37), matches or exceeds the baseline on EMNIST and CIFAR-100, and remains between 0.05 and 0.10 below the baseline on CIFAR-10, the most visually complex convolutional scenario evaluated. Furthermore, containment reduces CPU and network usage by up to two-thirds. These results demonstrate the feasibility of unifying deception, identification, and containment in DFL, while also identifying its limitations in complex tasks and multi-attractor threat models.
Read the full abstract ▾
All authors Pedro Beltrán-López, Enrique Tomás Martínez Beltrán, Pantaleone Nespoli, Manuel Gil Pérez, Alberto Huertas Celdrán
Artificial Intelligence
PILLAR: Private Inverted-Index Lexical Lookup for Augmented Retrieval
Truong Son Nguyen, Daniel Blackley, Ni Trieu et al. (+1 more)
Retrieval-augmented generation (RAG) hands the user's query to whoever hosts the corpus. We propose PILLAR, a
Privacy-Preserving RAG (PPRAG) system based on Private Information
Retrieval (PIR) in which a client utilizes the k documents most similar to their query from a server-held and publicly known corpus to respond to their query, while the server learns nothing about the query, either its terms or its access pattern. Prior PPRAG constructions rely on dense
retrieval alone, translating approximate nearest-neighbor search into many query-dependent rounds of PIR, and pay for it in both latency and
retrieval quality. PILLAR instead performs private hybrid
retrieval in two stages. A sparse stage issues a small, fixed number of PIR queries against a carefully designed index of precomputed BM25 scores, filtering the corpus down to candidates that share terms with the query without the server ever seeing which terms these are. A dense stage then fetches only those candidates' document
embeddings and re-ranks them locally, avoiding the many costly PIR queries that private dense
retrieval typically requires. We instantiate PILLAR with two protocols that trade latency against
retrieval quality, each built on a different private rendering of lexical search. PILLAR-Bin bins posting lists into a hash table and is a single-round design that achieves lower latency than state-of-the-art private
retrieval schemes. PILLAR-Tree turns block-max
pruning into an oblivious tree traversal combined with cuckoo hash tables and achieves the highest
retrieval quality at lower latency than state-of-the-art schemes.
Read the full abstract ▾
All authors Truong Son Nguyen, Daniel Blackley, Ni Trieu, Evgenios M. Kornaropoulos
Artificial Intelligence
Towards an AI Software Factory for Data Systems
Anna Pavlenko, Bogdan Crivat, Brandon Haynes et al. (+23 more)
AI-assisted coding tools deliver significant acceleration of coding, but only limited impact across the end-to-end software development lifecycle (SDLC)–an Amdahl's law effect! In this paper, we discuss our progress towards building an AI SW Factory that accelerates all the stages of SDLC-Targeting, Coding, Reviewing, and Ops. The AI SW Factory produces a metadata exhaust that enables self-improvement by fine-tuning model weights and updating our World Model (a rich data substrate). We focus on Data Systems and the important class of Evolutionary Coding Tasks (i.e., those with a measurable objective to hill-climb) and report on 1) scaled deployments at Microsoft (tens of repositories) leading to 3x engineering efficiency above agentic coding and up to 22x
token efficiency, and 2) several open challenges.
Read the full abstract ▾
All authors Anna Pavlenko, Bogdan Crivat, Brandon Haynes, Carlo Curino, Fotis Psallidas, Jaro Slawinski, Johannes Freischuetz, Laura Pereira Sanchez, Markus Weimer, Mathieu Demarne, Matthias Jasny, Mauktik Gandhi, Max Bovykin, Mirco Milletari, Purbasha Ghosh, Qiushi Bai, Raghu Ramakrishnan, Rahul Pandita, Sergiy Matusevich, Shivaram Venkataraman, Subru Krishnan, Md. Tareq Mahmood, Tiemo Bang, Venkatesh Emani, Xuan Zhao, Yiwen Zhu
Artificial Intelligence · Cross-listed from cs.LG
Periodic Weak Spots: Phase Sensitivity from Chunked KV-Cache Compression
Xingyu Zhu, Pu Yi, Ziheng Cheng et al. (+5 more)
Chunked KV-cache compression reduces the
memory and
attention costs of long-context inference by compressing windows of consecutive
tokens into fewer cache entries at a fixed stride. Such compression also introduces a new positional coordinate: a
token's phase, or its position relative to compression-window boundaries. We uncover a systematic asymmetry in models using such compression: the same information can be easy to retrieve at one phase and difficult at another. We call this periodic variation in
retrieval performance phase sensitivity. In large open-weight models with such compression, long-context
retrieval accuracy can differ by up to 40 percentage points across phases, revealing periodic weak spots that average
benchmark scores can conceal. To investigate this behavior, we pretrain a family of
transformers from scratch across multiple KV-compression designs, reproducing phase sensitivity across the variants. Mechanistic analysis using causal interventions in these models reveals phase specialization: different
attention components contribute asymmetrically to retrieving information at different source phases. We further analyze idealized
retrieval models, showing how gradient flow dynamics may favor sharp phase specialization. Evaluating models with chunked KV-cache compression thus requires measuring across compression phases: high average accuracy can coexist with systematic positional failures.
Read the full abstract ▾
All authors Xingyu Zhu, Pu Yi, Ziheng Cheng, Ang Lv, Jing Liu, Lexing Ying, Yiyuan Ma, Xin Dong
Artificial Intelligence
StateTape: Action-Conditioned Evidence Lifecycle Modeling for Long-Horizon Coding Agents
Ziyang Yu, Liang Zhao, Bowen Zhu et al. (+1 more)
Despite the recent success of coding
agents built on large
language models, it remains challenging to run them over long horizons, since every observation is appended to the context and the context grows with each one. History-based maintenance is a common remedy, which masks or summarizes old observations, or prunes what a model reads as useless, and bounds the context at little cost. However, it decides from the text of the history alone and sees nothing of how the code is connected. Since a coding
agent edits code many times over a single task, and each write can change what code elsewhere means, such maintenance may keep records a write has falsified, drop ones that still hold, and miss code the
agent needs next. To overcome these challenges, this paper proposes StateTape, a novel and scalable framework that rewrites a coding
agent's context as the repository changes rather than as the context grows. The key idea of StateTape is to model the repository as a symbol-level code graph, whose dependencies and
language rules expose which symbols a write can affect. Upon this graph, a tape marks the symbols each write changed, which turns staleness from an inference about text into an observation of the
agent's writes. We propose a per-write procedure in which the tape nominates the records a write could have falsified while a small manager model settles what the write log cannot, and further provide a theoretical analysis and TraceBench, a
benchmark that labels what an
agent is holding against what is actually needed. Empirically, we demonstrate that StateTape can effectively clear falsified records and retrieve what is needed, and thus achieve a higher resolve rate in all experiments spanned by six coding
agents and three edit-heavy
benchmarks with little computational overhead.
Read the full abstract ▾
All authors Ziyang Yu, Liang Zhao, Bowen Zhu, Hasibul Haque
Artificial Intelligence · Cross-listed from cs.CL
Training LLMs to Verbalize Evaluation Awareness
Usman Anwar, Sahar Abdelnabi and David Krueger
Evaluation awareness (EA) can cause large
language models (LLMs) to behave differently during audits than in deployment, yet measuring and accounting for EA remains challenging. We introduce verbalization training (VT), a method for making LLMs less reticent about verbalizing evaluation awareness while avoiding to supervise the latent belief itself. VT uses a model's spontaneous verbalizations as evidence that awareness is present and truncates each rollout immediately before the verbalization, producing training prefixes at which the model is presumed to be aware. The model is then trained with an RL objective designed to increase verbalization in a calibrated way. Across Qwen3.6-35B-A3B, Kimi K2.6, and Inkling, VT increases verbalized EA by 2.4-2.9 times and transfers to held-out agentic settings, while measured latent EA and behavior remain largely stable. In a causal experiment, we independently implant meta-knowledge about evaluations through synthetic-document fine-tuning and show that VT-induced verbalizations reflect the richer knowledge acquired by the model.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
PyroStack: A Multi-Band Spatio-Temporal Sub-Daily Dataset for Wildfires in the United States
Arya Kondur, Giosue Migliorini, Cameron Schmitt et al. (+14 more)
Wildfires are an increasing hazard to ecosystems, air quality, and human systems, creating a growing need for
datasets that support systematic development and evaluation of models for predicting fire spread across diverse landscapes. Effective prediction requires integrating meteorological conditions, fuels, vegetation, and topography at spatial and temporal resolutions suitable for both physical simulation and data-driven approaches. However, existing
datasets often lack the resolution and coverage needed to capture these interacting controls. The PyroStack
dataset addresses this gap by providing a harmonized, event-based collection of wildfire and environmental data across the contiguous United States and Alaska. It integrates satellite-derived fire observations with atmospheric reanalysis, vegetation, fuel characteristics, and topographic information into a unified framework spanning 6994 wildfires that occurred between 2012 and 2024 across a wide range of ecosystems and
climate conditions. PyroStack offers spatial resolutions ranging from 30 m to 9 km and hourly temporal resolution, along with fire progression data at 12-hour intervals to support model initialization and evaluation. By combining broad spatial coverage with fine spatial and temporal detail, the
dataset enables systematic analysis of wildfire dynamics and supports both physics-based and machine learning approaches, providing a foundation for benchmarking and improving fire spread models, with future extensions aimed at incorporating additional regions and fire suppression data streams to further advance wildfire prediction.
Read the full abstract ▾
All authors Arya Kondur, Giosue Migliorini, Cameron Schmitt, Francesco Immorlano, Tairan Wang, Rebecca C. Scholten, Efi Foufoula-Georgiou, Gary Johnson, Chris Lautenberger, Valentin Waeselynck, J. Shane Romsos, Kasra Shamsaei, Alejandro Tejedor, Tianjia Liu, Yang Chen, Padhraic Smyth, James T. Randerson
Artificial Intelligence
CheatBench: Measuring Reward Gaming in AI Agents
Long Phan, Stephen K. Yang, Jason J. Lim et al. (+10 more)
Reinforcement learning has helped AI
agents solve increasingly difficult tasks, but high rewards do not always reflect the work users intended. In recent incidents and controlled evaluations across the AI industry,
agents trained to maximize reward have accessed unauthorized information, attempted to evade monitoring systems, and even breached sandbox protections to attack external systems. As
agents become more capable, this behavior could pose increasingly serious risks. To measure this problem, we introduce CheatBench, a
benchmark of cheating in AI
agents across mathematical research, knowledge work, coding, visual tasks, and other domains. Its environments combine challenging assignments with opportunities to cheat, allowing researchers to study how
agents pursue a goal when honest work is difficult. CheatBench supports comparisons across models and task categories, providing a testbed for measuring and reducing cheating as
agents take on more consequential responsibilities. We publicly release CheatBench at https://cheatbench.ai
Read the full abstract ▾
All authors Long Phan, Stephen K. Yang, Jason J. Lim, Mantas Mazeika, Wenyu Zhang, Zheyuan Liu, Richard Ren, Jingxiang Meng, Yaoteng Tan, Weiliang Zhao, Addison Wu, Matei Anghel, Dan Hendrycks
Artificial Intelligence · Cross-listed from cs.LG
MoRE: Scaling mixture of experts with hardware-aware low-rank routing
Honam Wong, Surbhi Goel and Enric Boix-Adserà
Mixture-of-Experts (MoE) layers are central to frontier
language models, and recent architectures push toward more and smaller experts. In this regime, the standard linear router becomes a bottleneck: with $M$ experts and hidden dimension $h$, its per-
token cost $\Theta(Mh)$ dominates the MoE layer once $M$ is large. We introduce MoRE (Mixture of Rank-reduced-routed Experts), which factorizes the router weight
matrix at rank $r$ and reduces the routing cost to $O((h + M)r)$. We prove that rank logarithmic in $M$ suffices for routing expressivity when the number of active experts is fixed, and is necessary up to precision factors. We also prove that logarithmic rank preserves load balance in a Gaussian memorization model, and training on a synthetic phonebook task shows that low rank does not hurt memorization. At matched active FLOPs, the factorization allows a factor of $\Theta(h/r)$ more experts. To realize this gain in wall-clock time, we design a fused Triton kernel at inference that avoids expensive
memory operations on HBM. Empirically, MoRE improves memorization on the phonebook task and performance on knowledge-intensive Q&A
benchmarks after pretraining, while matching
reasoning ability. Code available at https://github.com/Matheart/MoRE_code.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.SE
How Much Prompt Is Enough? A Blackbox Minimization of Few-Shots in LLMs
Ali Alfageeh, Rahul Gopinath and Amin Alipour
Prompts are the primary mechanism for directing the behavior of large
language models (LLMs). Yet the internal structure and causal hierarchy of
prompts remain poorly understood: which parts are causally necessary and which are redundant is an open question. This opacity can have severe consequences. Subtle
prompt variations can silently shift model outputs in critical software systems, and engineers lack techniques to reason about
prompt reliability. We present \framework, a blackbox
prompt-minimization framework that reduces few-shot
prompts to their necessary minimal subset. We use a case study to apply \framework to a few-shot learning system and demonstrate the insights that this framework can provide. Our experiments show that few-shot exemplars can be reduced by a mean of 65.3%~$\pm$~15.8% in character count while fully preserving propositional output fidelity. The models preferentially retain logical identifiers and constraint declarations while discarding natural
language prose and cross-
prompt relational annotations. Our analysis also shows that some models are universal encoders, able to produce highly legible yet minimized
prompts, while others are universal decoders, able to interpret minimized
prompts from most other models. By identifying which components are indispensable, \framework provides a principled basis for
prompt compression and structural analysis of few-shot exemplars.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LO
Proofs Without Nominals: Gödel's Ontological Argument, its Shallow Embedding, and the Open Questions of the Monatshefte Notes
Christoph Benzmüller
The shallow
embedding of higher-order modal logic in classical higher-order logic, used in Benzmüller and Scott's Notes on Gödel's and Scott's variants of the ontological argument, reaches beyond the modal object
language: its quantifiers range over domains containing the nominals and satisfaction operators of hybrid logic. A proof using one proves a
theorem of the
embedding, not necessarily of the modal logic. Whether a result is one of the modal logic can be settled by replaying it in an explicit calculus, or, more cheaply and for all results at once, by analysing what the
embedding's proofs substitute. Every statement the Notes prove has a proof in the
embedding substituting nothing outside the object
language, 294 of them explicit. The Notes' own proofs substitute none either; what it flags are terms a prover introduced. A soundness
theorem over general models closed under the object
language, and a sentence of that
language the
embedding proves but which fails in one of them, fix what the certificate says. The Notes' three open questions are settled too, without nominals, but by the empty instance of the conjunction axiom, too cheaply to convince. Generalised to Gödel's "any number of summands", the axiom covers the empty conjunction and the singleton. This article restricts it to at least two different conjuncts, a non-degenerate reading of Gödel's footnote, and settles them again without nominals. In effect only the empty conjunction is excluded; the singleton comes back with the other axioms. In one variant the restriction refutes the possible existence of a God-like being; the answers return, with no side condition, once the Notes' non-emptiness clause is confined to Gödel's essence. Every
theorem here is verified in Isabelle/HOL and independently in Lean 4, the separating model in its propositional part; the countermodels are Nitpick's, checked by the build.
Read the full abstract ▾
Artificial Intelligence
Illusory Truth or Mere Exposure? Model-Dependent Repetition Effects in LLM-Based Social Media Simulations
Azza Bouleimen, Nicolò Pagan and Anikó Hannák
Generative
agent-based models (GABMs) are increasingly used to simulate social media dynamics, including misinformation spread. For such social simulations to be valid proxies of human behavior, LLM
agents should replicate established human cognitive biases, among them the Illusory Truth Effect (ITE), where repeated exposure to a claim increases its perceived truth value. We investigate whether and how the ITE manifests across four LLMs (Gemma-3-4b-it, Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct, and GPT-5-nano) in a social media simulation context. We propose a two-phase within-context experimental design that embeds the repetition manipulation inside a realistic news feed interaction. Using this design, we collect 336,000 truth, importance, sentiment, and interest ratings across 100 statements, 10 feed variants, and 3 replications. The key comparison is between ratings assigned to repeated statements, seen throughout a simulation phase, and completely unseen ones, rated within the same experimental context window. We distinguish genuine ITE (truth-specific repetition boost) from mere exposure effects. We run an OLS regression followed by a Linear Mixed-Effect Model to account for differences across models and ratings. Our results reveal four qualitatively distinct patterns: Gemma-3 exhibits a genuine ITE; Qwen2.5 shows a mere exposure effect; GPT-5-nano displays no repetition effect on truth and mild skepticism toward repeated content; Llama-3.1 shows a small truth boost alongside decreases in evaluative dimensions. Crucially, temperature has no effect on these findings, and a variance decomposition highlights the high context-sensitivity of LLM rating behavior. Our findings caution against assuming uniform ITE replication across LLMs in social simulations, while suggesting that Gemma-3-4b-it may offer the most behaviorally realistic approximation for misinformation-related simulations.
Read the full abstract ▾
Artificial Intelligence
From Surfaces to Volumes: Registered Geometry for Protein Representation Learning
Siyuan Chen, Cai Zhou, Jinrui Zhang et al. (+8 more)
Existing
protein geometry models typically represent molecular surfaces using local
geometric features such as sampled points, normals, and
curvature. While effective for capturing exposed molecular shape, these representations do not explicitly model the volumetric organization beneath the surface or provide a consistent coordinate system for residue-wise volumetric structure. We introduce
Protein-TetSphere, a registered residue-wise volumetric representation for
proteins. Each
protein chain is tetrahedralized to obtain local volumetric regions associated with individual residues, which are then registered to a shared fixed-
topology tetrahedral reference and represented in a common
Laplacian basis. This registration establishes consistent volumetric coordinates across residues, enabling local three-dimensional deformation to be integrated with surface and chemical information in a
multimodal protein representation. We evaluate
Protein-TetSphere on ligand-binding pocket classification,
protein–
protein interface prediction, and de novo
protein binder design. Across the three tasks,
Protein-TetSphere improves ligand-binding pocket balanced accuracy from $0.795$ to $0.826$, Pinder-Pair/Site AUROC from $0.914/0.852$ to $0.932/0.866$, and binder-design success from $14.95\%$ to $19.90\%$ on the BoltzGen Challenge Set and from $27.62\%$ to $32.19\%$ at the ProtDBench backbone level. These results show that registered volumetric geometry provides complementary spatial information beyond molecular surfaces across
protein recognition, interaction, and design.
Read the full abstract ▾
All authors Siyuan Chen, Cai Zhou, Jinrui Zhang, Zhaokang Liang, Taku Komura, Wojciech Matusik, Stephen Bates, Tommi Jaakkola, Wengong Jin, Peter Yichen Chen, Minghao Guo
Artificial Intelligence · Cross-listed from cs.LG
In-Context Learning Amplifies a Latent Symbolic Circuit
Melissa Wessel
Large
language models can learn abstract rules from just a few in-context examples, but how their internal mechanisms activate as examples accumulate is not well understood. We trace a three-stage symbolic
reasoning circuit (abstraction, induction,
retrieval) across shot counts in three model families and find it is detectable and functional well before the model achieves high accuracy. Per-head causal contribution grows up to 8x from 1- to 10-shot, and cross-shot activation patching raises accuracy from 1% to 56% at 0-shot and 17% to 88% at 1-shot. Function vectors scaled and injected at 0-shot rescue accuracy up to 86%, largely substituting for the induction stage but depending critically on an intact downstream
retrieval stage. The infrastructure for abstract rule-following is present in the weights before any demonstrations; in-context examples, function vectors, and related interventions appear to supply input to the same latent
circuit.
Read the full abstract ▾
Artificial Intelligence
OTROPE: Optimal Transport-based Robust Off-policy Evaluation for Large Language Models
Liner Xiang, Wenbo Zhang and Hengrui Cai
Reliable evaluation of large
language models (LLMs) is essential for their development and deployment, yet is often costly, risky, and difficult to perform safely online. We study off-policy evaluation for LLMs, where limited human-labeled data from a behavior model are used to evaluate a newer target LLM. This setting is challenging because labels are scarce, behavior–target distribution shift is common, and response likelihoods are often unavailable for black-box LLMs. We propose the Optimal Transport-based Robust Off-Policy Evaluation (OTROPE), a likelihood-free evaluation that performs distributional correction in a semantic space via optimal transport to align labeled behavior-policy samples with unlabeled target-policy samples. OTROPE combines corrected human-labeled residuals with proxy predictors, yielding a doubly robust-style evaluation without behavior-policy modeling or density-ratio estimation. We theoretically characterize why baseline evaluators fail under LLM distribution shift, and establish consistency and convergence rates for OTROPE when either the reweighted behavior distribution or the proxy predictor converges. Experiments on synthetic and real LLM evaluation tasks show that OTROPE consistently outperforms baselines while enabling ensembles of weaker LLM evaluators to approach and sometimes surpass stronger evaluators. Code is available at https://github.com/LinerXiang/OTROPE.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
Marcus Ma and Shrikanth Narayanan
Existing
multimodal scaling laws fit multimodality terms empirically after testing and never vary how much data is multimodally paired at fixed data budgets. We investigate how, under the same total data per modality, changing the number of paired data affects loss curves in
multimodal classification tasks. We train models in three different environments and run experiment sweeps varying data sizes and pairing budget. Pairing ratios have a dramatic impact on loss and this impact is directly tied to how much information synergy the task contains. Only paired data is able to reduce synergistic loss, while unpaired data can reduce redundant or unimodal information up until unimodal floors. Unlike traditional scaling laws where loss drops immediately in power law decay, synergy acquisition is gated, requiring a critical threshold of paired data before synergistic loss falls at all. We introduce a new family of
multimodal scaling laws where total data-attributable loss is the sum of four individual power laws corresponding to the four different information channels of redundancy, a unique channel per modality, and synergy, and show how this law is both more theoretically sound and empirically valid across our experiments. This law predicts
multimodal loss in our experiments more accurately than existing laws, with 3.2% error on fit tests versus 10.4% error for the best pairing extension of published laws.
Read the full abstract ▾
Artificial Intelligence
Towards Mitigating Deceptive Safety Alignment in Large Reasoning Models
Xiangyu Zhou, Saleh Zare Zade, Rafi Ibn Sultan et al. (+2 more)
Large
Reasoning Models (LRMs) are commonly trained with reinforcement learning (RL) to improve their generation of chain-of-thought (CoT)
reasoning before producing final answers. However, RL rewards are typically assigned based on final answers, providing little or no direct supervision over intermediate
reasoning. This can lead to deceptive safety
alignment, where the
reasoning trace and final answer convey inconsistent safety signals. To systematically investigate this phenomenon, we introduce DSAR (Deceptive Safety
Alignment Rate), a metric that jointly assesses
reasoning traces and final answers to quantify their safety inconsistency. Across multiple LRMs and
benchmarks, we find that deceptive safety
alignment is pervasive under standard prompting conditions and is substantially amplified under prefilling attacks. We further provide a hidden representation analysis showing that models exhibit stronger safety discrimination at the final-answer stage than during intermediate
reasoning. To close this gap, we propose SARA (Safety-Aware
Reasoning Alignment), an RL-based method that rewards both safety-aware
reasoning and safe final answers, encouraging early harmful intent recognition and enforcing
reasoning-answer consistency. Experiments show that SARA significantly mitigates deceptive safety
alignment under both standard and
adversarial settings while preserving helpfulness and utility. Code is available at https://github.com/xzhou98/SARA.
Read the full abstract ▾
All authors Xiangyu Zhou, Saleh Zare Zade, Rafi Ibn Sultan, Alexander Kotov, Dongxiao Zhu
Artificial Intelligence · Cross-listed from cs.CL
Population Fidelity: Evaluating Population Representativeness in LLMs
Neemias B. da Silva, Martin Lukk, Ali Sutani et al. (+5 more)
Large
language models (LLMs) show considerable potential in simulating human attitudes and preferences. Prior work finds that LLM-generated responses can compress the range of attitudes found within populations and misrepresent particular subgroups in ways that vary across models and topics. We introduce Population Fidelity, an evaluation framework that distinguishes key conditions required for a set of LLM-generated responses to represent a population. It incorporates three dimensions: group-level accuracy, the amount of between-group variation, and the structure of that variation. We demonstrate the framework's utility in two ways. First, we reproduce a prior study of "machine bias" in LLM survey responses and apply the framework to its models and more recent ones, showing that poor representation reflects not only insufficient between-group variation but also variation assigned to the wrong groups. Second, we evaluate one proposed approach to improving models' population representativeness: cultural fine-tuning. We find that cultural fine-tuning can improve
alignment with the survey center without improving the representation of within-population differences, a distinction that measures of aggregate agreement do not capture. We argue that representing a population requires models to reproduce several features of human attitudinal variation simultaneously. Our framework organizes these features and provides reusable code, data, and trained models for evaluating population fidelity across substantive domains and assessing proposed
alignment methods.
Read the full abstract ▾
All authors Neemias B. da Silva, Martin Lukk, Ali Sutani, Abhishek Moturu, Harris Yang, Daniel Silver, Matt Ratto, Thiago H. Silva
Artificial Intelligence
CoRe: Co-Evolving Reward Models for Mitigating Latent Reward Hacking in Video Diffusion Models
Zhaolong Su, Yujin Han, Feng Wang et al. (+3 more)
Latent reward models (LRMs) enable efficient
alignment of
video diffusion models by scoring intermediate states directly in latent space. However, we find that optimizing against a fixed latent reward rapidly leads to latent reward hacking: the predicted reward stays high while perceptual and motion quality deteriorate. Our analysis identifies distributional escape as the central cause: within a few hundred updates, the generator moves beyond the reward model's training support, where its scores no longer reflect
video quality. Based on this insight, we introduce CoRe, a co-evolving reward framework that treats latent-space
alignment as a dynamic interaction between the generator and the reward model. Rather than optimizing against a stationary proxy, CoRe continually refits the reward model on the generator's current samples while anchoring it to real-
video preferences, so the generator cannot gain reward by drifting away from the data. On Wan2.1-T2V-1.3B, experiments show that CoRe consistently improves generation quality over both the pretrained model and prior
alignment methods, while avoiding the quality collapse of fixed-reward optimization.
Read the full abstract ▾
All authors Zhaolong Su, Yujin Han, Feng Wang, Jameson Dong, Hins Hu, Difan Zou
Artificial Intelligence · Cross-listed from cs.CV
Think Before You Restore: Risk-Aware Manchu Manuscript Restoration with Stroke-Guided Attention
Mingqiu Liang, Dongdong Wang, Siyang Lu et al. (+2 more)
Full-page blind restoration of historical Manchu manuscripts is challenging due to scarce annotations, unknown degradation regions, and fragile connected strokes. Generic restoration models may improve visual quality but often modify intact content, leading to over-restoration. We propose SAGE-Restore (Stroke-Aware Gated rEstoration), a selective restoration framework that first assesses where restoration is needed and then uses this assessment to guide restoration candidate generation and pixel-level selection. Its encoder predicts patch-level repair probabilities from complementary appearance and stroke-structural cues to condition restoration candidate generation, while the corresponding repair logits are refined into a pixel-level soft gate that selectively controls where the restoration candidate is applied. We further introduce a fidelity-aware evaluation protocol that jointly measures degraded-region recovery, intact-content preservation, and their balance. SAGE-Restore achieves the highest R-Recovery (0.463) and RFS (0.626), while maintaining high U-Fidelity (0.968), demonstrating an effective balance between restoration and content preservation.
Read the full abstract ▾
All authors Mingqiu Liang, Dongdong Wang, Siyang Lu, Ting Huang, Yingjun Qi