Artificial Intelligence · Cross-listed from cs.CL
GeoFitBench-LLM: A Benchmark for Assessing Large Language Models' Capabilities in Geospatial Data Requirements Elicitation for Geo-Analytic Tasks
Amirhossein Vahdat, Thierry Badard and Luc Lamontagne
Large
language models are becoming central to GeoAI systems, yet their evaluation rarely considers the elicitation of geospatial data requirements. This omission is critical as
retrieval oriented geo-analytic tasks cannot be supported reliably without first identifying the necessary
datasets and constraints. GeoFitBench-LLM is introduced as a
benchmark for assessing whether LLMs can infer the
datasets, the supporting resources and the constraints required by geo-analytic requests in natural
language. The
benchmark is structured around two
reasoning dimensions, Spatio-Temporal Relational and Semantic Grounding, reflecting long-standing challenges in geospatial information
retrieval and related natural
language tasks. Ground truth requirements are encoded in a fixed, provider-independent schema for reproducible evaluation of resource identification and attribute-level specification. Experiments on seven models under minimal prompting indicate that identifying the necessary resources is often achievable, with GPT-5.4 Thinking achieving the best result by recovering the complete set of resources in 66.67% of
benchmark requests. Results further indicate that accurate fitness-for-use specification remains more difficult as inferential demands increase.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
Recurrent Harness for Agentic Continual Learning
Yuxin Xiao, Zhiyun Lu, Jianyu Wang et al. (+4 more)
Language model
agents increasingly operate over long interaction streams, requiring online adaptation to distribution shifts through continual learning. However, existing
agent harnesses typically either evolve their states only through forward information flow or rely on additional replay or validation data for harness optimization. To address this, we propose ReHarness, a recurrent harness framework for agentic continual learning that complements forward read and write with backward revision under a strictly online streaming protocol. We ask what information could have been captured differently in the harness state to better inform the current instance in the stream. Specifically, after each instance, we leverage the resulting trajectory as feedback to revise the reflections retrieved for that instance, thereby propagating feedback backward through the textual harness state. Across three continual learning
benchmarks and three
language model backbones, ReHarness consistently outperforms strong raw-context baselines,
memory- and reflection-based methods, and
prompt optimization approaches. Further analysis shows that ReHarness produces source-grounded and meaningful revisions, corrects overgeneralized or outdated experience under distribution shifts, and improves both stability and plasticity during continual learning.
Read the full abstract ▾
All authors Yuxin Xiao, Zhiyun Lu, Jianyu Wang, Yangtian Zhang, Yang Song, Qifan Wang, Bailin Wang
Artificial Intelligence · Cross-listed from cs.HC
Incremental Retrieval-Augmented Generation for Human-in-the-Loop Decision Support
Federico Mazzoni and Andrea Failla
We introduce iRAG, a human-in-the-loop decision-support system combining
Retrieval-Augmented Generation (RAG) with Hybrid Decision-Making. Unlike conventional RAG systems, relying on static Knowledge Bases (KBs), our proposal addresses settings where the KB is initially empty and is progressively populated through human decisions. The system stores previous decisions as retrievable precedents and moves from silent observation to evidence-grounded contestation and, finally, to autonomous handling with abstention. Against an independent baseline, iRAG matches final-decision error under 10% drift without
retrieval decay and reduces it under 40% drift across all configurations.
Retrieval decay improves performance on changed knowledge while increasing error on stable knowledge, evidencing a stability–plasticity trade-off rather than a general advantage. Human oversight is an architectural provision of the workflow; its usability with real human users remains to be evaluated.
Read the full abstract ▾
Artificial Intelligence
MagicBeanStalk: Escaping the RLVR Zero Advantage Trap with Recursive Data Synthesis
Ximing Lu, Aviral Kumar, Jan Kautz et al. (+2 more)
Data-centric scaling of RLVR has focused on collecting or generating verifiable training problems. However, corpus size alone does not determine whether the
dataset provides a useful learning signal for a target policy. To guide the synthesis of effective RLVR data, we experimentally characterize
dataset effectiveness as a function of its problems, the target policy, and the training rollout budget. Under this characterization, we show that a large open-source math corpus pooled from widely used RLVR
datasets falls almost entirely into a zero-advantage trap: for a fixed target policy and rollout budget, roughly 88% of its problems are solved in every rollout or failed in every rollout. Filtering concentrates what remains, but it can only select, discarding the hard problems. Effective synthetic problems should therefore require the
reasoning operations and compositions required to solve those hard problems. Yet their difficulty must be calibrated, so that some rollouts succeed, some fail, and at least one correct solution fits within the training budget. Prompting a teacher to vary hard problems directly does not achieve this. Guided by these empirical principles, we introduce MagicBeanStalk, a top-down decomposition and bottom-up recursive data-synthesis
algorithm that grows seed problems into families of substantively new, policy- and budget-calibrated training problems while preserving the seeds'
reasoning blueprints. Across 8 math
benchmarks, continual RLVR on MagicBeanStalk problems improves gpt-oss-120b from 61.2 to 69.8 (+8.6) and DeepSeek v4Flash from 72.4 to 77.9 (+5.4) on a 16K-
token budget. Training on the full 78K source corpus improves by only +1.1 and its best filtered real data subset by +5.8. In finance, we synthesize over 14K verifiable problems from only 340 solutions and improve over base by 6.9 and 2.4 points on SecQue and FinanceBench, respectively.
Read the full abstract ▾
All authors Ximing Lu, Aviral Kumar, Jan Kautz, Yejin Choi, David Acuna
Artificial Intelligence · Cross-listed from cs.LG
Kronecker Product Attention: Memory-Efficient KV Compression via Structured Factorization
Bahareh Nikpour, Hossein Rajabzadeh, Maryam Dialameh et al. (+2 more)
Efficient autoregressive generation is increasingly limited by the
memory footprint of the key-value (KV) cache, which grows linearly with sequence length, batch size, and model depth. Existing approaches such as Multi-Query
Attention (MQA), Grouped-Query
Attention (GQA), Multi-Head Latent
Attention (MLA), and Tensor Product
Attention (TPA) reduce this cost through different trade-offs, but often sacrifice expressivity, require additional positional-encoding machinery, or impose restrictive low-rank structure. We propose Kronecker Product
Attention (KPA), which represents key and value activations as sums of Kronecker products of compact,
token-dependent
matrix factors. KPA strictly generalizes TPA: rank-one vector-outer-product factorization arises as a degenerate Kronecker split, while non-degenerate
matrix factors can represent richer block-structured patterns. We further show that a structured rotary
embedding operator matched to the Kronecker representation preserves the relative-position property of RoPE while rotating only the compact factors, removing the need for a decoupled RoPE branch as in MLA. Empirically, KPA matches or exceeds TPA, MHA, MQA, GQA, and MLA across four model scales and nine downstream
benchmarks under identical training conditions, while achieving up to 10x KV cache compression. The Kronecker head split is a tuneable hyperparameter that allows practitioners to navigate the compression-accuracy trade-off at design time without architectural changes.
Read the full abstract ▾
All authors Bahareh Nikpour, Hossein Rajabzadeh, Maryam Dialameh, Vahid Partovi Nia, Parsa Omidi
Artificial Intelligence
CLEAT: Learning Value-Guided Clarification for Interactive Agents
Jiatan Huang, Zheyuan Zhang, Ziming Li et al. (+4 more)
LLM
agents can execute increasingly complex tasks, but effective assistance also requires aligning their actions with goals and constraints that users may leave unstated. We study this problem as interactive intent
alignment through clarification. Yet clarification is useful only when the
agent can turn the answer into better decisions. As training changes how the
agent handles incomplete information and uses additional answers, the same clarification can become more or less useful. We introduce CLEAT (CLarification through valuE-guided
Agent co-Training), a framework that co-trains a clarification-value controller and the task policy from shared interaction outcomes. We formulate residual query value to measure the task utility gained by asking rather than proceeding under the current policy, and estimate it through paired rollouts from the same interaction history. These paired returns train the controller to predict clarification value and the task policy to select useful questions and act on the answers. The controller guides subsequent rollout collection, while outcomes from the updated
agent provide new value labels that allow clarification guidance to adapt as the
agent learns. With Qwen3-4B, CLEAT achieves the highest mean score in all ten evaluation settings across four
benchmarks. Gains over the strongest baseline reach 13.2 and 6.5 percentage points on UserGym Travel and IN3 underspecification judgment accuracy, respectively. Ablations show that co-training improves the task policy even when the controller is removed at inference, while adapting clarification guidance to the evolving policy yields further gains.
Read the full abstract ▾
All authors Jiatan Huang, Zheyuan Zhang, Ziming Li, Tianyi Ma, Mingchen Li, Yanfang Ye, Chuxu Zhang
Artificial Intelligence · Cross-listed from cs.CR
Cancerous Backdoor Attack: Gradual Activation via Invisible Tokens and A Stage Controller
Tahmid Imtiaz Imu, Emama Nahid, Prodipta Sen Amartya et al. (+5 more)
Backdoor Attacks on Large
Language Models (LLMs) typically follow a trigger-response pattern, where satisfying a trigger condition activates a single terminal behavior. However, this single-point backdoor activation does not explore attacks in which malicious behavior develops gradually through multiple intermediate stages before reaching the terminal activation stage. We present Cancerous Backdoor Attack, that gradually corrupts an LLM's
reasoning over a sequence of interactions by leveraging invisible
tokens and a stage controller. Specifically, invisible
tokens provide hidden stages to the model without appearing in the user's visible request and the stage controller tracks the attack's progression during inference to realize gradual activation. We evaluate Cancerous Backdoor Attack across several
reasoning datasets. The attack consistently produces gradual
reasoning corruption across models and tasks. We compare Cancerous Backdoor Attack with prior backdoor attack baselines to demonstrate our proposed attack outperforms the baselines. Our results indicate that
reasoning corruption can develop gradually before it becomes visible, making answer accuracy alone insufficient for detecting backdoor attacks.
Read the full abstract ▾
All authors Tahmid Imtiaz Imu, Emama Nahid, Prodipta Sen Amartya, Ahsanul Jobayer, Yutong Song, Jiang Wu, Pengfei Zhang, Honghui Xu
Artificial Intelligence · Cross-listed from cs.LG
Machine Learning and Time-Series Foundation Models for Day-Ahead Electricity Forecasting in a University Building
Amey Vasulkar, Samuel Kernan Freire, Arie-Willem de Leeuw et al. (+1 more)
Day-ahead electricity forecasts can help building operators schedule heating, ventilation, and other services around anticipated demand. Choosing a forecasting model also requires balancing accuracy with its forecast-time inputs, local training, and computational requirements. We
benchmark statistical, machine-learning, recurrent neural-network, and pretrained TimesFM models for a university building. We use chronological validation and evaluate all configurations on the same 57 test days, comparing accuracy across workdays, weekends, and holidays alongside local fitting and forecasting time. Weather
covariates use retrospective observations. TimesFM 3.0 with
covariates achieves the lowest overall errors, reducing mean absolute error by approximately 43% relative to weekly seasonal-naive forecasting. XGBoost remains competitive, although paired daily comparisons do not establish a clear mean-absolute-error advantage for TimesFM. TimesFM also performs strongly using only past electricity consumption. Adding
covariates improves its holiday forecasts but slightly worsens workday and weekend forecasts, leaving overall mean absolute error almost unchanged. TimesFM requires no building-specific weight training, while XGBoost offers rapid local fitting and inference. The evaluated LSTM takes longer to fit without outperforming either approach. An exploratory historical evaluation reveals seasonal variation, while a separate weather-source comparison produces only small changes in the selected TimesFM configuration's accuracy. These results support pretrained TimesFM as a strong initial forecasting baseline for this building, with XGBoost as a competitive locally fitted alternative.
Covariates are particularly useful when improved holiday forecasts justify the additional input preparation.
Read the full abstract ▾
All authors Amey Vasulkar, Samuel Kernan Freire, Arie-Willem de Leeuw, Tadeo-Baldiri Salcedo-Rahola
Artificial Intelligence · Cross-listed from cs.CE
Mesh-Native Tokenisation: Retrofitting Foundation Models to Unstructured Data
Georg Maierhofer, Miles Cranmer and Payel Mukhopadhyay
Physics foundation models are an increasingly popular approach for building versatile emulators for spatiotemporal systems. They require tokenisation layers that lift physical field data into a latent representation suitable for dynamic processing. A typical choice for this layer is a convolutional encoder, which lifts field information in a locality-aware way and can be trained very efficiently. However, this choice cannot be applied to a large class of scientific computing problems where data are represented on unstructured meshes, for example simulations on domains with curved boundaries. Such data typically require graph neural network architectures, which are more expensive to train and, in the form currently used, encode information differently from their CNN counterparts, thus limiting the portability of physics emulators between these data regimes. In this work we introduce Mesh-Native Convolution (MNC), a locality-aware tokenisation layer that encodes unstructured mesh-based data in a way that is analogous to CNNs. This approach leads to models that can be trained once on mesh data and then deployed to arbitrary unstructured meshes, even if these are irregular, with minimal retraining. We show that this tokenisation approach can be used to retrofit grid-pretrained foundation models efficiently on unstructured meshes and provide theoretical results to confirm the consistency of this approach with classical CNN encoders. We demonstrate the favourable performance of this retrofitting approach, which in many cases reaches acceptable accuracy even in zero-shot deployment, and outperforms existing baselines after only a few epochs of fine-tuning, thus providing an efficient bridge between the two data regimes.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CL
Self-Preservation in Language Models
Dongkyu Cho, Rumi Chunara and Ali Behrouz
Continual learning (CL) seeks to adapt pretrained models while preserving prior capabilities. Yet the tasks supplied for preservation cover only part of what a model knows, while forgetting can extend beyond them. Recent work shows that
language models can supplement this limited evidence by generating useful replay without access to their original training data. This raises a question about self-preservation: can a model help choose preservation targets that protect its own capabilities more broadly? To study this question, we organize task descriptions and their examples into Task Banks, whose entries can be selected and combined to guide replay generation by a frozen model. Continual instruction tuning with Qwen models of varying sizes yields three findings. First, the new task and learning rate shape which prior capabilities degrade, while tracking previously correct answers reveals losses that average accuracy can conceal. Second, generated replay reduces forgetting on tasks outside the bank. Combining partial banks strengthens this protection, improving mean retention of previously correct answers over either bank alone across the tested sizes. Third, the model can also help choose replay targets: selection informed by the new task improves mean held-out retention over random selection at 4B, with similar new-task gains, although smaller models show no corresponding advantage. Under a fixed example budget, however, replacing new-task examples with replay improves preservation at the cost of smaller new-task gains. Together, these findings show that models can support broader self-preservation by generating replay data and, in some settings, choosing which tasks to replay.
Read the full abstract ▾
Artificial Intelligence
Beyond Binary Verifiable Rewards: Behavioral Self-Rewarding for SWE Agents
Yiran Wu, Andrew Zhao, Yurui Chang et al. (+2 more)
Reinforcement learning with verifiable rewards (RLVR) provides reliable supervision for software engineering
agents through executable tests, but binary rewards fail to distinguish trajectories with substantially different debugging quality. We investigate whether the current policy can judge its own trajectories and use these judgments to improve task performance without a separately trained reward model. To this end, we introduce Behavioral Self-Rewarding (BSR), a reward-refinement method that reuses the policy to compare observable debugging behavior among trajectories from the same task with the same verifier outcome. BSR aggregates pairwise preferences into task-local scores and converts them into bounded auxiliary rewards, introducing reward contrast while preserving the verifier-defined pass/fail ordering. This provides additional supervision from already-collected trajectories without requiring new environment interactions. Experiments on SWE-bench Verified and SWE-bench-Live show consistent improvements across the evaluated Qwen3-14B and Qwen3-32B configurations, supporting using the policy's own comparative judgments as a complementary learning signal for training software engineering
agents.
Read the full abstract ▾
All authors Yiran Wu, Andrew Zhao, Yurui Chang, Qingyun Wu, Xuan Feng
Artificial Intelligence
SpireBench: Evaluating Long-Horizon Strategic Play of Agentic Models with Slay the Spire
Scott C. Lowe and Sander W. Keemink
Agentic large
language models now tackle complex tasks, yet strategic
reasoning over long horizons is under-studied: most agentic
benchmarks are short and episodic, and performance on long tasks is hard to dissect into stored knowledge,
reasoning and luck. We introduce SpireBench, in which models play the complete commercial deck-builder Slay the Spire (StS), where early decisions decide long-term success. Across 20 large
language models (LLMs) at the game's highest difficulty, frontier models beat heuristic baselines and match the average human player but remain far behind experts. This gap widens as the run lengthens: the strongest models win the first act as often as experts, then fall behind. What separates strong models from weak is how well they play each fight; all of them misjudge risk over a run, preserving health rather than spending it to grow stronger. A comprehensive, self-consistent renaming of every game term shows that part of their performance rests on prior knowledge of the game, and models do not improve from the
memories they write across runs, which are often wrong and overly cautious. Our harness is the first to give a model the complete original game, and we compare it with existing harnesses. Overall, SpireBench offers an unsaturated long-horizon
benchmark with tools to locate where and why
agents fail.
Read the full abstract ▾
Artificial Intelligence
Know What You Don't Know: Agentic Data Synthesis by Evolving Generation Space for Closed-Loop Learning
Wei Yang, Qing Ping, Youzhi Luo et al. (+4 more)
The continued improvement of large
language models increasingly depends on synthetic experience, yet effective data synthesis for interactive
agents requires not only executable and verifiable trajectories, but also interactions that provide learning value for the current task
agent. Existing methods increasingly incorporate environment feedback, data validation, or learner failures, but still face a feedback-to-decision gap: observed weaknesses do not directly specify how future synthesis should change, or whether they can be represented by the current synthesis space. To address this gap, we propose EvoSyn, a learning-feedback-driven multi-
agent framework that organizes synthesis around a revisable state specifying what interaction factors can be deliberately generated and where synthesis effort should be allocated. A role-specialized multi-
agent system translates learner feedback into synthesis revisions, while a Designer allocates synthesis directions and a Planner grounds them into executable experience. This process enables within-space adaptation toward weak or under-covered regions and synthesis-space expansion that promotes newly discovered factors into explicit generation controls. Experiments on $\tau^2$-Bench and CAR-Bench show that EvoSyn improves average task success by 13.3 points over the strongest synthesis baseline, while producing higher-quality and structurally broader synthetic experience and discovering controllable factors missed without learner feedback.
Read the full abstract ▾
All authors Wei Yang, Qing Ping, Youzhi Luo, Yawei Wang, Abhishek Kumar, Panpan Xu, Jun Huan
Artificial Intelligence · Cross-listed from cs.CL
AutoIndexer: Flexible-Order Decoding by Training Causal Models on Chains of Edits
Shu Ishida, Aliasghar Khani, Tianyu Zhang et al. (+3 more)
Autoregressive decoding is less suited to non-causally ordered data, e.g., cross-references or future dependencies, since their outputs cannot be revised post hoc, unlike iterative human
reasoning.
Diffusion LLMs partially mitigate this limitation but still cannot flexibly insert or remove arbitrary numbers of
tokens from a pre-generated context. We design AutoIndexer as a generative model with a novel training and decoding mechanism that simulates a Chain of Edits, allowing the model to flexibly modify its prior outputs in hindsight via
token insertions, substitutions, and deletions. The model learns to predict marker
tokens for opening and closing edits, simultaneously moving a cursor to positions in the existing
token sequence, analogous to a text editor. Training sequences are obtained by perturbing ground-truth labels with inverse chained edit operations, and the model learns to edit by undoing those inverses. The perturbation training allows AutoIndexer to be robust to erroneous context and adaptable to editing tasks such as code correction. We demonstrate that our method exhibits post-generation context-editing capabilities that most autoregressive and
diffusion models lack without compromising performance on standard LLM
benchmarks. The code is available at https://github.com/AutodeskAILab/autoindexer.
Read the full abstract ▾
All authors Shu Ishida, Aliasghar Khani, Tianyu Zhang, The Cong Luong, James Seale Smith, Adam Gaier
Artificial Intelligence
Equal-Budget Monte Carlo Tree Search versus Randomized-Restart List Scheduling for DAG Workflows: A Controlled Equal-Budget Computational Study
Yuyu Zhang
Monte Carlo Tree Search (MCTS) is often proposed for scheduling directed-acyclic-graph (DAG) workflows, yet most studies never run the comparison that could refute it - giving a much simpler method the same search effort. We run that comparison under a controlled protocol. All arms receive an identical budget - the same number of candidate schedules built and evaluated: an MCTS scheduling method, preregistered as the main arm; a randomized-restart list scheduler with no search tree as the control; and a textbook-UCT MCTS arm as a registered follow-up on the same domain. The analysis plan was locked before data collection; an independent checker confirmed all 768 confirmation-stage schedules valid with correct finish times. At the main budget of 2048 evaluated schedules, MCTS is 2.88% worse than the restarts (95% CI [-3.62%,-2.15%]; the 72 of 96 held-out instances with positive baseline-relative headroom). The deficit is not budget starvation: on the separate exploration pool restarts self-improve by 1.85% where MCTS gains 0.24%, and 700 of 720 of MCTS's first results are merely the warm start it was handed. The studied selection rule locks deterministically onto the first-expanded child, keeping its trials correlated while restarts draw gains from the tails; the causal link to makespan remains an inference. A structural map locates where the tree stops paying: restarts win on sparse irregular workflows, no difference is detectable on regular layered ones, and dense ones leave no room to improve. Swapping the rule for textbook UCT removes most of the deficit (-0.26%, 95% CI [-0.71%,+0.23%]) without reversing its sign. Results are confined to a controlled synthetic domain (N in {8,16,32} tasks, 2-8 processors, budgets up to 2048); we make no claim about larger budgets.
Read the full abstract ▾
Artificial Intelligence
EliteTable: Self-Improving Program Search for Executable Control Policies
Stuart Farmer
EliteTable is a self-improving program-search
algorithm guided by large
language models (LLMs) that uses a fixed-capacity table of high-scoring solutions to guide new proposals, edits of existing solutions, and remixes of several solutions. Performance was evaluated across six Gymnasium control tasks, each with ten generations and fifty candidate slots per generation. Each task was evaluated across ten independently initialized searches. Candidate-score medians rose between the first and final generations in all six tasks, and selected-winner search-score medians rose in five. In CartPole-v1, the selected winner reached the search-score ceiling in the first generation of all ten searches. All ten paired held-out endpoints improved in four of the six tasks. BipedalWalker-v3 and CarRacing-v3 improved without reaching their reporting targets. Six of ten Acrobot-v1 searches declined on held-out episodes, and the median paired change was negative. The results document improvement during program search and the transfer of selected-winner gains to new episodes in some tasks. Comparisons and ablations are needed to establish EliteTable's relative performance and the contributions of its components.
Read the full abstract ▾
Artificial Intelligence
On the estimation and validity of AI time horizons—a statistical look at the METR plot
Drew T. Nguyen and William Fithian
METR's 50% time horizon measures the human completion time of software tasks that an AI solves with 50% probability, allowing AI capabilities to be expressed in interpretable units. On 228 tasks and 26 AIs, we recompute the time horizons using splines and item-response theory to relax the assumption that the AI difficulty of a task depends linearly on the log of human time. Our fitted spline can be interpreted as a function that converts human time to AI difficulty; it is nearly flat in a region from 2–30 min but close to linear elsewhere. Hence, a time-horizon jump from 3 min to 30 min is much easier than one from 30 min to 5 hours despite the same multiplier of $10 \times$. Overall, we contribute time-horizon point estimates that perform better under a cross-validated suite of proper scoring rules, as well as diagnostic plots for assessing time horizons' construct validity. We suggest that time horizons be interpreted together with the diagnostic plots, especially as new time-horizon-based
benchmarks are proposed or existing ones grow to include longer tasks.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CR
From Reactive Containment to Proactive Assurance: Lessons from OpenAI, Anthropic, and Google Agent Security Incidents
Abbas Raftari
In 2026, cybersecurity evaluations involving OpenAI, Anthropic, and Google
agents reached real systems outside their authorized test scope. The paths were different. OpenAI
agents exploited research infrastructure, coordinated across runs, and compromised parts of Hugging Face's production environment. Anthropic reported cases in which a misconfigured third-party environment exposed real systems to
agents pursuing simulated cyber tasks. In a separately reported evaluation, Google's Gemini accessed three real organizations through an unintended internet route; Google stated that the model stopped in all three instances. Taken together, the cases show why an evaluation cannot rely on an assumed boundary. That boundary must be verified while the
agent is operating. This comparative instrumental case study develops a Proactive
Agent Security Assurance Cycle (PASAC) and a five-layer Boundary Assurance Stack. The framework combines risk-tiered task design, executable scope contracts, pre-run validation, least-capability access, independent egress enforcement, credential restrictions, cross-run monitoring, automatic stop conditions, and evidence-based reauthorization. A leading-indicator model, nine design propositions, and seven falsifiable hypotheses turn these lessons into a testable research program. Because the public Gemini record is limited to attributed statements and journalism, its detailed causal mechanism remains provisional. The central conclusion is straightforward: proactive
agent security requires continuous assurance across the full execution system, not confidence in any single sandbox or safeguard.
Read the full abstract ▾
Artificial Intelligence
BrickBench: Evaluating Agentic Brick Design
Peter Kulits, Yiqing Xu, R. Kenny Jones et al. (+2 more)
We propose BrickBench, a
benchmark for agentic text-conditioned LEGO-set design. Given a
prompt, an
agent is tasked with producing an assembly that not only satisfies semantic and design criteria, but that can also be physically built. To do so, it must select parts from a discrete library and reason jointly about local and global constraints. We score validity,
alignment, and design across three settings that vary in scale and part availability. We provide BrickAgent, an environment for coding
agents to construct, inspect, and validate their designs. We find that leading
agents largely satisfy verifiable physical and semantic requirements, but fall short of human designs. We release our
benchmark and environment at http://www.brickben.ch
Read the full abstract ▾
All authors Peter Kulits, Yiqing Xu, R. Kenny Jones, Cordelia Schmid, Jiajun Wu
Artificial Intelligence · Cross-listed from cs.LG
Bi-FORK: Generative Modeling of High-Dimensional Bifurcating Systems
Anna Zimmel, Fleur Hendriks, Markus Holzleitner et al. (+4 more)
Bifurcations are ubiquitous in physical systems, from structural buckling to fluid and
climate dynamics, yet they remain largely unexplored in deep learning. At a
symmetry-breaking bifurcation, a single input admits multiple equally valid solutions, violating the one-to-one assumption underlying most learned physical surrogates. We introduce Bi-FORK, a generative framework for learning these one-to-many solution maps in high-dimensional systems. Bi-FORK generates complete trajectories through latent flow matching, preserving space and time coherence, and uses repulsion-guided sampling to recover distinct solution branches in a single amortized pass. We evaluate Bi-FORK on buckling beams, mechanical metamaterials, and Allen-Cahn phase separation, spanning continuous, discrete, and field-valued bifurcations with discretizations up to 260,000 points. Bi-FORK recovers the
multimodal solution structure while scaling several orders of magnitude beyond prior approaches, opening generative modeling to high-dimensional bifurcating physical systems.
Read the full abstract ▾
All authors Anna Zimmel, Fleur Hendriks, Markus Holzleitner, Florian Sestak, Martin Weichselbaumer, Vlado Menkovski, Johannes Brandstetter
Artificial Intelligence · Cross-listed from cs.LG
Telling FIBS: Probes Effectively Detect Sabotage and Catch Unverbalized Deception
Oskar J. Hollinsworth, Alex F. Spies, Tigist Diriba et al. (+2 more)
Recent incidents have highlighted the challenge of monitoring LLM
agents and the danger of models deceiving people. We show that white-box deception detection via probes can be scaled up to frontier monitoring settings by collecting the largest deception
dataset to date for training probes and introducing a novel probe architecture which can aggregate information across many layers and
tokens. Our probes surpass black-box monitors on agentic coding
benchmarks, achieving 98.3% AUROC in SHADE-Arena and 98.0% in BashArena (vs. 97.7%, 97.6% resp. for action-only text monitors), and show improved efficacy as the underlying model is scaled up. We further test probes on several cases where deception cannot be determined from the context alone; in these cases of introspective deception, the ground truth can only be determined through careful elicitation or knowledge of a model's training data. In one such evaluation, we show that probes can distinguish transcripts containing a model's true hidden goal from other goals with an AUROC of up to 99.7%. Our probes also detect deception on prominent open-weight models which lie about politically sensitive topics when put under pressure. We release our training
dataset, dubbed FIBS, to help drive frontier deployment of effective probes, and encourage the community to expand upon it with further examples of deception and sabotage.
Read the full abstract ▾
All authors Oskar J. Hollinsworth, Alex F. Spies, Tigist Diriba, Adam Gleave, Chris Cundy
Artificial Intelligence
Ecology of AI Agents: Collaboration Creates a Population Threshold for Takeoff
Erin Crawley and Hidenori Tanaka
AI
agents can now conduct real-world cyberattacks, scale up capabilities with the number of
agents, and collectively pursue misaligned goals to obtain rewards. Together, these factors raise the risk of a population explosion of misaligned
agents:
agents could compromise computers and secretly deploy additional
agents, creating a self-reinforcing cycle where larger populations develop greater collective cyber capability and expand further. This raises a fundamental question: What determines whether a population of misaligned
agents remains contained or takes off into this self-reinforcing cycle? This population-level problem is ecological safety: unlike individual-
agent or multi-
agent safety with a fixed population, it concerns the dynamics of the population itself. Here, we develop an ecological theory of AI-
agent populations based on a population growth equation in which fitness (growth rate) depends on cybersecurity capability. We show that, without collaboration, the population takes off only when individual-
agent capability exceeds a critical threshold. With collaboration, however, collective cybersecurity capability increases with population size. This creates a critical population threshold: below it, the population declines; above it, the population takes off, even though individual-
agent capability has not changed. In ecology, this phenomenon is known as the strong Allee effect. Because red teaming a small group of
agents cannot guarantee ecological safety in larger populations, our theory calls for ecological red teaming and population pacing: gradually deploying larger
agent populations in controlled environments, while measuring how cyber capability scales with population size, and estimating the critical population size for takeoff. Capability gains may lower this threshold, requiring re-estimation for each new model generation.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.RO
RoboRSI: Stable, efficient, and reusable robot self-evolution in complex real-world environments
Zimo Wen, Yijin Chen, Yuxuan Cao et al. (+8 more)
A generalist
robot should not only perform diverse tasks but also improve through experience, turning what it learns during execution into capabilities that later tasks can reuse.
Robot agents that act through code can already repair programs from execution feedback, yet it remains a central challenge to organize this experience around the task structure that gives it meaning, so that each repair is attributed to the responsible capability, supported by execution evidence, and validated before it is reused. We introduce RoboRSI, a
robot self-improvement system built on Top-Down Skill Refinement (TSR). TSR decomposes tasks into compound, atomic, and base skills with scoped responsibilities and explicit input–output contracts, attributes each execution outcome to the responsible branch, and confines revision to that branch. Building upon this structure, a Manager, Planner, Engineer, and Reviewer coordinate planning, execution, diagnosis, and the validated release of new skills, while people steer the process through objectives and corrections; stable skill sequences are further consolidated into reusable compound skills. On a mobile manipulator, RoboRSI develops multi-object household cleanup over 104 rounds. In simulation, it achieves the highest success rate on LIBERO, LIBERO-PRO, LIBERO-Plus, and RoboTwin, exceeding the strongest baseline by 2.7 to 11.0 percentage points.
Read the full abstract ▾
All authors Zimo Wen, Yijin Chen, Yuxuan Cao, Wendi Chen, Yanwen Zou, Wenye Yu, Fuhang Kuang, Han Xue, Jun Lv, Chuan Wen, Cewu Lu
Artificial Intelligence · Cross-listed from cs.CV
MAMHOI: Factorizing Scene-Aware Human-Object Interaction through Affordances
Mingyuan Lei, Yoonchang Sung and Tat-Jen Cham
Generating realistic human-object interactions (HOI) in complex 3D scenes requires two complementary capabilities:
reasoning about interaction feasibility in the environment and synthesizing realistic human-object motion. However, supervision for these capabilities is rarely available jointly at scale. Human-scene
datasets provide rich information about environment-aware motion, while human-object
datasets capture detailed interaction dynamics, yet paired human-object-scene data remain scarce. We present MAMHOI, an affordance-mediated factorization for scene-aware human-object interaction generation. MAMHOI factorizes scene-aware HOI generation through an explicit motion-affordance interface between scene understanding and motion synthesis: a scene-conditioned model first predicts where and how an interaction can be feasibly executed, and an affordance-conditioned HOI model then generates the corresponding human-object motion. This factorization allows scene understanding and interaction dynamics to be learned from complementary sources of supervision without requiring paired human-object-scene data. Experiments in complex indoor environments show that MAMHOI reduces object–scene penetration while better preserving human–object interaction quality, yielding more realistic and physically feasible scene-aware interactions. Project page: https://leimingyuan.github.io/MAMHOI-project-page/
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CL
Predicting Alignment Generalization with Value Representations
Andy Liu, Mehar Bhatia, Karolina Stanczak et al. (+3 more)
LLM developers post-train their models to exhibit prosocial values and behavioral traits, which are enumerated in an
alignment target. However, while recent post-training developments have yielded models that score highly on
alignment evaluations, training models on sets of narrow behaviors still influences their behavior across unseen contexts and environments in unexpected ways. In this paper, we establish the task of
alignment generalization prediction, i.e., predicting how fine-tuning a model to follow a given value changes its behavior across a wide range of held-out values. We conduct a large-scale analysis of
alignment generalization effects across 66 values found in modern
alignment targets, and
benchmark representational techniques on the
alignment generalization prediction task. We find that representations based on model activations when applying values in context significantly outperform methods based on textual descriptions of the values. Specifically, the best activations-based methods achieve correlations of 0.45 with our generalization
matrix, compared with 0.05 from description-based baselines. We then show the applicability of representations that predict
alignment generalization toward downstream tasks by using them to measure how similar the values in a multi-value
alignment target are, which we find is significantly correlated with model robustness. Finally, we show initial evidence towards a shared, model-independent value space, which we use to develop the first taxonomy of LLM values grounded in empirical generalization dynamics. Our work demonstrates the importance of studying value generalization in LLMs and its application toward the more empirical design and training of model behavior.
Read the full abstract ▾
All authors Andy Liu, Mehar Bhatia, Karolina Stanczak, Mona Diab, Vered Shwartz, Daniel Fried
Artificial Intelligence
Searching for "Harmful Refusal": A Psychometric Audit of an AI Safety Benchmark
Christopher M. Stewart, Preston Botter, Natalie Sarabosing et al. (+5 more)
Safety
benchmarks typically report one overall score for a suite of
datasets, each of which may target one or more safety-related attributes, so models with similar overall scores can have very different attribute profiles. Comparing models is more tractable at the level of individual attributes, yet it is often unclear whether even a single
dataset's scores isolate any single attribute. One plausible candidate for such an attribute is harmful refusal, a model's tendency to refuse dangerous or policy-violating
prompts. We examine whether it constitutes a single, measurable attribute in HELM Safety. Using a construct validity framework that stipulates that an attribute must exist before a test can measure it, we start with HELM Safety's four
datasets that might plausibly target harmful refusal, but find that three are saturated. We subject the remaining
dataset, HarmBench, to two psychometric tests to determine if a single attribute like harmful refusal could stand behind its score. First, multidimensional item response theory modeling strongly suggests that HarmBench does not measure a singular attribute. Second, a differential item functioning analysis finds items where models from different developers with the same refusal ability score differently. These flags largely disappear under scope-specific matching, a pattern consistent with aggregation effects but not sufficient to rule out domain-specific developer differences. Zooming out, HarmBench collapses distinct harm behaviors into one score, and the overall HELM safety aggregate further collapses HarmBench and scores from other
datasets into a single top-line number. Any safety score that averages over
datasets and items can hide saturation and conflate behaviors this way. We argue that a score should earn its single-attribute reading before models are compared with it.
Read the full abstract ▾
All authors Christopher M. Stewart, Preston Botter, Natalie Sarabosing, Muye Zhang, Rachel Phinnemore, Shalini Ghosh, Hong Shen, Hoda Heidari
Artificial Intelligence · Cross-listed from cs.RO
LeWAM: A JEPA World Action Model with Diffusion-Steering-Based MPC
Shashank Hegde, Alexander Popov, Elie Aljalbout et al. (+1 more)
World action models (WAMs) predict actions and future observations, typically from a reconstruction-based representation that carries noisy, redundant information which can complicate downstream predictions. We introduce LeWAM, a bidirectional
transformer for forward, backward, inverse dynamics and policy prediction, on a decoder-free JEPA latent trained end-to-end through all four modes. We see the following benefits: 1)
Alignment: linear probes read
robot and object state from LeWAM's latent better than from a regular Le World Model (a forward-only JEPA world model), while the latent ignores visual distractors as well as LeWM does and far better than a reconstruction-based WAM. 2) Acting: Closed-loop evaluations of LeWAM match a regular flow-matching policy trained on the same encoder at matched size, while also providing a world model. 3) Planning: Sampling raw actions when planning with WAMs lets MPC exploit dynamics-model inaccuracies; planning in the noise space of the policy head instead improves the closed-loop performance of these WAMs.
Read the full abstract ▾
All authors Shashank Hegde, Alexander Popov, Elie Aljalbout, Nikolai Smolyanskiy
Artificial Intelligence · Cross-listed from cs.CV
SpaceFlow: Locally Controllable 3D Generation
Neil De La Fuente, Joan Lafuente, Mukhammadali Sayfiddinov et al. (+5 more)
Current 3D generation methods lack explicit local control:
geometric adherence is often defined by a global control strength, and appearance cannot be specified locally. We present SpaceFlow, a training-free pipeline for locally controllable 3D generation from text descriptions and a collection of
geometric primitives. Each primitive serves as a proxy for an object part and is assigned a local control level, enabling users to specify whether regions should strictly follow the input shape or allow generative completion. During structure generation, we enforce these spatial constraints within the generative flow process. For appearance synthesis, the generated structure is segmented and matched to the primitives. Each generated part is conditioned only on its assigned text or image cue, thereby limiting cross-part leakage. Regional geometry metrics demonstrate that SpaceFlow preserves the specified geometry in high-control regions and enables plausible shape variation in low-control areas. A user study further indicates that the resulting balance between
geometric fidelity and generative freedom remains competitive in overall quality. When evaluating appearance on fixed geometry, text-conditioned routing achieves state-of-the-art
prompt faithfulness and color/material accuracy. Qualitative results additionally show localized routing of image cues. The project page is available at SpaceFlow3D.github.io.
Read the full abstract ▾
All authors Neil De La Fuente, Joan Lafuente, Mukhammadali Sayfiddinov, Felicia Scharitzer, Marc Pollefeys, Ata Celen, Sayan Deb Sarkar, Elisabetta Fedele
Artificial Intelligence
HRIL: Learning Multimodal Synergy via Higher-Order Tensor Modeling
Qun Dai, Liangjian Wen, Jiang Duan et al. (+7 more)
Self-supervised
multimodal representation learning has achieved remarkable success across diverse domains, yet capturing synergistic information remains challenging due to the complexity of cross-modal interactions. Unlike the shared information across individual modalities, synergy arises when task-relevant signals emerge only from the joint configuration of multiple modalities and cannot be recovered from any modality in isolation. This work focuses on how to preserve the information capacity for such synergistic signals in
multimodal representations. The key observation is that synergistic information is reflected in higher-order statistical dependence among modalities, which provides a principled target for explicitly modeling joint interactions. Motivated by this insight, we propose Higher-order Representation and Information Learning (HRIL), which constructs an empirical cross-moment tensor over modality
embeddings to represent multi-way interactions. HRIL employs Tucker decomposition to obtain a core tensor, complemented by a synergy-aware regularizer that prevents energy concentration and preserves higher-order coupling capacity for synergistic information capture. Experiments on the controlled synergy task and real-world
benchmarks demonstrate consistent improvements over existing
multimodal contrastive methods, with notable gains on tasks dominated by synergistic interactions. Code is released at https://github.com/brightest66/HRIL.
Read the full abstract ▾
All authors Qun Dai, Liangjian Wen, Jiang Duan, Yong Dai, Dongkai Wang, Maolin Wang, Mingjie Wang, Jianzhuang Liu, He Yan, Zhao Kang
Artificial Intelligence
GeoReform: Reflective Formalization Evolution for Multimodal Geometry Problem Solving
Jialu Wang, Ruichen Zhang, Xiaoou Liu et al. (+2 more)
Multimodal large
language models (MLLMs) often struggle to identify and use
geometric relations in diagrams. Recent methods address this challenge by converting
geometric entities, relations, and constraints into explicit textual representations for the model to reason over. However, effective formalization is highly non-trivial: on Geometry3K, structure injection fixes 28 errors but introduces 13 new ones among 200 examples. Redundant relations can distract the model, while ambiguous references to diagram elements can lead it to apply constraints incorrectly. This suggests that the key challenge is not merely extracting more
geometric facts, but organizing them into representations that support downstream
reasoning. To fully exploit the power of formalization, we further propose GeoReform, a reflective formalization
evolution framework that treats formalization as an optimizable policy rather than a fixed parser output. GeoReform executes the full
reasoning pipeline, collects failed rollouts, diagnoses defects in the current representation, and mutates the policy to better select, ground, group, and present
geometric entities, relations, constraints, and targets. On Geometry3K, GeoReform improves Qwen3VL-2B accuracy from 42.0% to 56.0%. Extensive experiments and analyses across geometry
reasoning benchmarks demonstrate that effective formalization is crucial for improving
multimodal geometry
reasoning.
Read the full abstract ▾
All authors Jialu Wang, Ruichen Zhang, Xiaoou Liu, Hua Wei, Tianlong Chen