Artificial Intelligence · Cross-listed from cs.LG
Overcoming Challenges of Interpretive Structural Modeling with Large Language Models
Everett Rush, David J. Icove, Ari Kim et al. (+2 more)
Interpretive Structural Modeling (ISM) is a well-known process for multi-criteria decision making. The success of ISM over other methodologies is its ability to model causal relationships, the binary scale of factors, and resulting hierarchical representation. Traditionally, the modeling process is performed by repeated interactions with subject matter experts until consensus is reached. This process is tedious, labor-intense, and most importantly limits the ability of ISM to scale to studies with hundreds of variables. Drawing on existing work of causal graph discovery with large
language models (LLM) as imperfect experts, this work explores an integrated LLM-ISM approach for ISM. Pairwise, k-wise, rowwise, and full graph discovery methodologies are compared and evaluated. It is shown that causal graph discovery methods for ISM perform best using rowwise (SHD=160, F1-score=0.77) and full graph methods (SHD=135, F1-score=0.73).
Read the full abstract ▾
All authors Everett Rush, David J. Icove, Ari Kim, Byung H. Park, Michael A. Langston
Artificial Intelligence · Cross-listed from cs.LG
Counterfactual Predictions in Scientific Emulators Without Controlled Experiments
Dingling Yao, Kahaan Gandhi, Valentin Duruisseaux et al. (+3 more)
Many scientific questions require
reasoning about what was never observed: What if the conditions, interventions, or history had been different? Models can predict accurately on observed data yet fail on such what-if queries when correlated inputs are varied independently. A common remedy is to add controlled simulation data in which these factors are explicitly disentangled, but this requires access to a simulator, can be computationally expensive, and inherits the simulator's modeling assumptions. We introduce ReRoute, a framework for targeted scientific what-if prediction that combines factual data with partial mechanistic knowledge, without requiring controlled intervention data for adaptation. ReRoute fixes the queried input of a pretrained backbone to a reference value, reintroduces its variation through a known mechanistic pathway, and fine-tunes on the original factual data, while leaving downstream effects to the learned dynamics. We provide a causal identification result for this construction under explicit structural assumptions, with the core argument machine-checked in Lean. After showing that ReRoute achieves highly accurate counterfactual predictions in a controlled advection-
diffusion system where exact responses are available, we turn to state-of-the-art
climate emulation. On held-out coupled-
climate interventions, ReRoute reduces aggregate
climate error by 18.2-31.8% under severe CO$_2$ distribution shifts while preserving skill under standard conditions, at a small fraction of the cost of retraining on additional controlled simulations, without even accounting for the substantial expense of generating such data. Finally, on an emulator trained from historical ERA5 reanalysis, where no counterfactual reference exists, ReRoute preserves substantially more of the surface warming implied by the observed boundary conditions under a fixed-CO$_2$ counterfactual.
Read the full abstract ▾
All authors Dingling Yao, Kahaan Gandhi, Valentin Duruisseaux, Boris Bonev, Francesco Locatello, Anima Anandkumar
Artificial Intelligence · Cross-listed from q-bio.QM
Toward Controlling Biology with Language:Offline Learning of Prompt-Conditioned Interventions for Cells, Organoids, and Biobots
Nam H. Le, Douglas Blackiston, Michael Levin et al. (+1 more)
Artificial intelligence increasingly serves as a natural-
language interface to complex technical systems, letting people accomplish sophisticated tasks by describing what they want rather than specifying how to do it. Extending this interface to living systems is harder: unlike code or images, a biological intervention has no closed-form linguistic meaning, and the paired
language-intervention-outcome data needed to learn such a mapping is expensive to collect, since each example requires its own wet-lab experiment. One way around this is to treat an existing archive of interventions and their already-observed outcomes as a fixed, offline
dataset, and use a
vision-
language model to judge, without any new experiments, whether an archived outcome matches a natural-
language description. But whether that judgment is reliable enough to train a
language-to-intervention mapping on – without new experiments and without human validation – has remained unclear. Here we show that a natural-
language interface for a living organism – a xenobot, a synthetic multicellular construct with no nervous system – can be learned entirely offline this way, using a
vision-
language model's own judgment as the sole training reward: an instruction is mapped to the intervention already on record as producing the described behavior. This mapping generalizes to entirely new instructions, evaluated against archive data withheld from training (80.0% held-out accuracy vs a $66.7% chance baseline, matching a network trained directly on ground-truth labels).
Read the full abstract ▾
All authors Nam H. Le, Douglas Blackiston, Michael Levin, Josh Bongard
Artificial Intelligence · Cross-listed from cs.CL
Contextual trajectory and incremental contextual displacement: Towards using LLMs to understand dynamic, utterance-specific meaning construction
Grayson Wycliffe Storer and Julia Witte Zimmerman
Transformer-based large
language models (LLMs) such as RoBERTa represent text using contextual word
embeddings (CWEs), which alter the
embeddings associated with each
token based on surrounding context. We construct
token-wise incremental trajectories by repeatedly recomputing a
token's CWE as successive words are added to a sentence, yielding a representation of how contextualized
embeddings evolve as the utterance unfolds. We evaluate this approach using garden-path sentences as a test case with characteristic features.
Token-wise trajectories reproduce known features of garden-path processing, including disruption around the critical region, and reliably distinguish garden-path sentences from matched disambiguated controls. We introduce several metrics for quantifying representational displacement across contextual increments and show that trajectory information can be highly predictive of sentence type. We find that ambiguity-related information is recoverable not only from the sentence-level CLS representation but also from ordinary vocabulary
tokens, suggesting that utterance-level information is distributed across multiple representational scales. In exploratory analyses, we find qualitatively similar trajectory structures in other ambiguity- and misdirection-related linguistic phenomena. Together, these results establish
token-wise incremental trajectories as a promising framework for studying utterance-specific meaning construction using LLMs.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
SHARPO: Segment-Level Credit Assignment for Agentic Reinforcement Learning
Xinchen Du, Zhengze Zhou, Wenhui Zhu et al. (+4 more)
Agentic reinforcement learning (RL) trains a large
language model (LLM) to act over long, multi-step interactions. However, a single localized error can cause task failure, while trajectory-level rewards provide limited guidance for assigning credit to individual decisions. To address this limitation, we introduce Segment-level Hindsight Advantage Reweighting for Policy Optimization (SHARPO), a credit-assignment mechanism that refines Group Relative Policy Optimization (GRPO) at the level of environment-facing segments. Inspired by the existing on-policy self-
distillation (OPSD) method, SHARPO computes teacher-student log-probability gaps within each segment and uses the resulting signal to compute a bounded multiplier on the GRPO advantage. This multiplier is shared by all
tokens within the segment, allowing credit to vary across different segments. With Qwen2.5-7B-Instruct, SHARPO outperforms existing baselines on the ALFWorld and WebShop
benchmarks, including GRPO, SDAR, RLSD, and StepOPSD.
Read the full abstract ▾
All authors Xinchen Du, Zhengze Zhou, Wenhui Zhu, Han Yu, Sen Na, Rohit Jain, Alborz Geramifard
Artificial Intelligence
Kepler: Auditable World Models for ARC-AGI-3
Wensen Wu
ARC-AGI-3 evaluates
agents in interactive environments whose rules and objectives must be inferred from observation. We present Kepler, an open-source harness that represents hypotheses as executable world models and validates them through retrospective transition checks and conditional prediction checks. Under one frozen Claude Opus 5 configuration, Kepler obtained a server-verified 100.00 RHAE on all 25 public games, with no per-game model selection or score-conditioned reruns. On 181 of 183 completed levels, the final Opus attempt used no more actions than the corresponding median-human baseline. The retained board runs used 8,256 environment actions, of which 7,292 occurred in scored levels. Retained local provider-session records yield 858.0 million
tokens, 97.37% cache reads, and a \$777.72 cost at September 1, 2026 API list-equivalent rates. We also report three evaluation failures: source-code leakage that produced an invalid perfect run,
agents reconstructing a removed harness in a control condition, and autonomous repair masking a broken planner. A single-game observation case study showed that animation frames contained task-relevant information absent from settled text grids. Across the final Claude Opus 5 and GPT-5.6 Sol boards, 48 of 50 game-model
cells reached 100. These results indicate that public-set score alone has limited discriminative value and motivate first-attempt, cost-conditioned, and verification-aware reporting.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CL
Verbalized and Internal Probabilities Are Coupled in Large Language Models
Sinead Williamson, Jiaxuan Li, Nick Foti et al. (+2 more)
Large
language models carry an internal notion of uncertainty in their sampling distribution, i.e., the probabilities they place on generating one answer rather than another. They can also be asked to state a confidence, in words or as a number: a verbalized uncertainty. Prior work suggests that internal probabilities track relative frequencies in the training data, and that verbalized probabilities track explicit probabilistic assertions in the training data. However, we do not know whether these two readouts are aligned, except when frequencies and probabilistic assertions in the training data happen to align. This limits our understanding of when we can use verbalized uncertainties as a proxy for either training data frequencies, or a model's internal distribution. We resolve this gap by systematically exploring how LLMs probability readouts are impacted by training and in-context data, via intervening on the underlying uncertainty sources in the data. We find that both internal and verbalized probability readouts are impacted by both distributional and asserted uncertainty in the training data. Further, we find that verbalized and internal probabilities are aligned beyond what would be expected by independently tracking the same uncertainty sources, suggesting that verbalized probabilities can be used to probe a model's internal distribution.
Read the full abstract ▾
All authors Sinead Williamson, Jiaxuan Li, Nick Foti, Russ Webb, Masha Fedzechkina
Artificial Intelligence · Cross-listed from cs.LG
On-the-fly Weight Generation: A Hypernetwork Proof of Concept on ARC-1D
Fabio J. Fehr and Philip Torr
General-purpose models can adapt to many tasks from context, while specialised models can execute individual functions with less capacity. Yet obtaining such specialists requires task-specific training or adaptation. We ask whether they can instead be generated directly from a few demonstrations. Using ARC-1D as a controlled testbed, we show that individual transformations can be represented by tiny specialist models, and that a hypernetwork can generate their parameters from context. The generated parameters form a structured weight space, while the resulting specialists show partial compositional generalisation and generalisation to transformations not seen during training. In both settings, removing explicit task identifiers improves generalisation beyond the training transformations. Together, these results provide a proof of concept that few-shot task context can be compiled on-the-fly into compact executable model parameters, and that the resulting weight space can support reuse and generalisation beyond known functions.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.DB
TabJoinBench: A Benchmark for Joinable Table Discovery
Sandipan De, Jin Wang and Vivek Gupta
Join discovery aims to identify tables from large data repositories that can augment a query table with complementary information, enabling downstream tasks such as data exploration, feature engineering, and business intelligence. Although numerous join discovery methods have been proposed, existing studies rely on method-specific
benchmark construction, making reproducible and fair comparison difficult. We present TabJoinBench, a
benchmark for evaluating join discovery methods across semantic, relational, and hybrid data lake scenarios. TabJoinBench constructs query-candidate pairs using source-specific validation strategies, systematically introduces structural, representation, and semantic changes through composable perturbations while preserving reliable ground truth. We evaluate representative join discovery methods spanning set-based, feature-based, and learned approaches, together with general-purpose
language-model
embedding baselines, and publicly release the processed
datasets, ground-truth annotations, and generation pipeline to facilitate reproducible evaluation and future research.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
Training-Aware Target Coverage for Synthetic Data Selection
Yang Ba, Michelle V. Mancenido and Rong Pan
Synthetic data are increasingly used to scale LLM training, yet more synthetic data do not necessarily produce better models. Useful synthetic data must add information relevant to the target task without introducing errors that offset their benefit, and the value of an example can change as the training set grows. We develop a linear theory that characterizes this tradeoff and determines where synthetic data are useful, how much should be added, and the marginal value of adding one example to an existing set. The analysis shows the conditions when input coverage alone is sufficient and when synthetic errors must also be considered. Guided by these results, we introduce Training-Aware Target Coverage (TATC), a synthetic data selection method for LLM fine-tuning. TATC identifies candidates whose training effects are beneficial to the target task and selects among them to expand coverage of target-relevant directions not already represented by the available data. Experiments on text and image data verify the linear theory. With mathematical
reasoning tasks, TATC selects synthetic solutions for fine-tuning Qwen2.5-Math-1.5B-Instruct and outperforms alternative synthetic-data selection methods on GSM8K across selection budgets. In summary, we provide a principled approach to synthetic data selection by quantifying and maximizing its value to the target task.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CV
Video Generation Models: A Survey of Post-Training and Alignment
Chaoyu Li, Xiaoyi Gu, Yogesh Kulkarni et al. (+10 more)
Video generation has rapidly progressed from short, low-quality clips to high-resolution, long-duration sequences with complex spatiotemporal dynamics. Despite strong generative priors learned through large-scale pretraining, pretrained
video models often fail to reliably follow human intent, maintain temporal coherence, or satisfy physical and safety constraints. Compared with image and text generation,
alignment in
video generation presents unique challenges, including error accumulation over time, motion-appearance coupling, multi-objective trade-offs, and limited supervision for temporal properties. These challenges motivate systematic post-training strategies that adapt pretrained models without retraining them from scratch. In this survey, we present the first comprehensive review of post-training and
alignment in
video generation models. We frame post-training as a unifying framework and distinguish between implicit
alignment and explicit
alignment based on how
alignment signals are enforced. From this perspective, we organize existing approaches into four broad categories: supervised fine-tuning methods, self-training and
distillation methods, preference- and reward-based methods, and inference-time methods. This taxonomy provides a coherent view of how
alignment signals shape model behavior across both training and deployment. Beyond methodological advances, we review commonly used
datasets,
benchmarks, and evaluation practices, and discuss open challenges such as scalable reward design, long-horizon temporal consistency, stability-expressiveness trade-offs, and safety-aware generation. This survey aims to provide a structured conceptual foundation and practical guidance for advancing controllable and reliable
video generation models.
Read the full abstract ▾
All authors Chaoyu Li, Xiaoyi Gu, Yogesh Kulkarni, Eun Woo Im, Mohammadmahdi Honarmand, Zeyu Wang, Juntong Song, Fei Du, Xilin Jiang, Kexin Zheng, Tianzhi Li, Fei Tao, Pooyan Fazli
Artificial Intelligence
Sapien: A Stateful Policy Engine for Autonomous AI Agents
Corinn Tiffany, Wen Zhang, Eugene Bagdasarian et al. (+1 more)
Contextual security defenses prevent AI
agents from taking rogue actions by synthesizing a task-specific policy and enforcing it on the
agent's tool calls. In multi-step tasks, however, which actions are valid often depends on what the
agent has already done and learned. We present Sapien, a policy engine for enforcing stateful contextual policies. A Sapien policy specifies permitted tool-call sequences using a regular expression extended with stateful predicates, deferred policy generation, and scoped semantic checks. We show that Sapien stays within a few percent of an unconstrained
agent's utility. Even if the
agent is fully hijacked, Sapien's policies rule out 93-95% of attacks on AgentDojo and 62-85% on Toolathlon (twice as many as tool allowlists on long-horizon tasks).
Read the full abstract ▾
All authors Corinn Tiffany, Wen Zhang, Eugene Bagdasarian, Lillian Tsai
Artificial Intelligence
Enterprise Representation Simplification (ERS): Reducing Representational Complexity for Enterprise AI
Terry Dorsey and Kevin Huggins
Enterprise information is represented through artifacts shaped by applications, projects, technologies, organizational boundaries, and local requirements. These structures accumulate over time, creating representational complexity that must be maintained by the enterprise and interpreted by information consumers and AI systems. This paper introduces Enterprise Representation Simplification (ERS) as reducing unnecessary representational complexity while preserving required information within a defined scope, and Enterprise Representation Complexity (ERC), a representation-neutral model for comparing complexity across representation states. ERC characterizes representational extent through four dimensions: Representation Objects, Interactions, Behaviors, and Supporting Sources. Objects, Interactions, and Behaviors form dependent categories, while Supporting Sources characterize representation exposure. ERC is defined at representation and task levels, enabling comparison and distinguishing architectural simplification from
retrieval optimization. The paper develops two consequences of ERS. First, representational structures create lifecycle obligations for maintenance, governance, dependencies, change, enhancement, and operation. An economic model distinguishes recurring global representation cost, recurring task-level cost, and one-time transformation cost, enabling evaluation over a defined time horizon. Second, reductions in task-level ERC reduce the representational extent an AI system must identify, relate, and interpret. Text-to-SQL research provides evidence that reduced schema and
reasoning complexity can improve
reasoning accuracy. ERC is not a universal complexity, performance, or cost metric. It provides measurable architectural variables for comparing representational alternatives, transformation effects, economic outcomes, and AI
reasoning performance.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
Pre-training interventions, ex post facto: Grafting model beliefs across checkpoints
Peter Nutter, Dani Roytburg, Clément Dumas et al. (+2 more)
Pre-training interventions are critical to
alignment research, since beliefs formed during pre-training shape how a model generalizes from later training. One recently popular technique for such interventions is synthetic document fine-tuning (SDF), which aims to alter what the model believes. Ideally, synthetic documents would be mixed into pre- or mid-training, but every change to a pre-training corpus must be followed by a full post-training run before its effect can be measured, making iteration slow and expensive. Common practice instead applies SDF to an already post-trained model. This is known to leave artifacts and degrade capabilities, and, as we show, it makes the model treat fabricated entities unrelated to the documents as real, a failure we call reality drift. We propose grafting: train the SDF adapter on the pre-trained checkpoint, then add the learned weight update to the post-trained model, which approximates the faithful approach while reusing the existing post-training. We demonstrate this by installing false facts, training misaligned model organisms and applying a constitutional mid-training intervention, across model families up to 284B parameters. Grafting installs the target belief as strongly as SDF on the post-trained model while reducing both reality drift and the loss of preference coherence by more than half on average, and it stays closer to a faithful mid-training run. Because grafting requires no post-training, the same adapter can be applied to any later checkpoint, enabling researchers to iterate quickly on pre-training interventions at the cost of a single fine-tuning run.
Read the full abstract ▾
All authors Peter Nutter, Dani Roytburg, Clément Dumas, Jinghua Ou, Shi Feng
Artificial Intelligence · Cross-listed from cs.LG
Increasing Width Allows Greedy Layer-wise Training to Rival End-to-End Backpropagation in Self-Supervised Learning
Syon Mansur and Joel Zylberberg
End-to-end backpropagation has been the dominant mode of training in deep learning, allowing for the coordination of parameter updates across layers of a neural network. Prior studies have explored alternative – and, in some cases, simpler – training mechanisms, showing that they can sometimes achieve performance similar to backpropagation. However, the architectural conditions under which locally optimized networks, which avoid end-to-end backpropagation of error, can learn representations comparable to those learned through end-to-end training remain unclear. We aim to answer this question in the context of self-supervised learning, an important framework for large-scale pretraining in artificial intelligence. Here, we investigate how network width and depth affect the efficacy of greedy layer-wise and end-to-end self-supervised training in convolutional networks. We find that in wider networks, the benefits of end-to-end backpropagation over greedy layer-wise training shrink: in relatively shallow and very wide networks, we even observed higher performance in models trained with greedy layer-wise training. Subsequent analysis of the representations formed by these networks shows that very wide greedy-trained networks exhibit more favorable representational geometry than do networks trained end-to-end with backpropagation. This work shows that width can compensate for restricted credit assignment and identifies differences in representational geometry as a potential mechanism for their improved performance.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CV
Personalized Image Generation with Reasoning and Reflection
Bo Ni, Ngoc N. Tran, Qinwen Ge et al. (+10 more)
Personalized image generation has remained narrowly focused on conditional synthesis from curated visual exemplars, rather than capturing who a user is. In practice, however, a user's personal context is much richer, comprising reviews, posts, images, captions, and metadata accumulated over time. A truly personalized generator should leverage this history to produce images aligned with the user's lifestyle and aesthetic preferences. To this end, we introduce the first unified
benchmark for personalized image generation from user histories. The
benchmark comprises two complementary tasks and a multi-axis evaluation protocol that assesses target fidelity, visual quality, user distinguishability, semantic
alignment with the user's history, and task-specific utility. Grounded in real-world e-commerce and social media settings, the
benchmark includes: (1) Personalized Scene Generation, which places a given object in a scene that reflects a user's preferences and lifestyle, motivated by personalized product presentation; and (2) Personalized Creative Generation, which generates a novel image on a specified topic that is faithful to a user's aesthetic and visual identity, motivated by social media content creation. We further propose PEARL, which couples a
multimodal reasoner with a frozen image generator in an interleaved reason-reflect loop optimized with differential data reward. Across both tasks, PEARL outperforms strong baselines, achieving an average improvement of 15% across personalization metrics.
Read the full abstract ▾
All authors Bo Ni, Ngoc N. Tran, Qinwen Ge, Franck Dernoncourt, Seunghyun Yoon, Samyadeep Basu, Sungchul Kim, Puneet Mathur, Nedim Lipka, Tong Yu, Yu Wang, Ryan A. Rossi, Tyler Derr
Artificial Intelligence · Cross-listed from cs.LG
Benchmarking Generative Models for Near-Surface Data Assimilation on Real Station Observations
Ruizhe Huang, Qidong Yang, Jonathan Giezendanner et al. (+1 more)
Weather reanalysis products rely on computationally intensive numerical weather predictions followed by data assimilation that corrects the forecast toward observations. Recent advances in deep generative models offer a cheaper alternative that shifts much of this cost from inference to offline training. However, existing generative approaches have been evaluated on synthetic observations or under different
datasets and evaluation schemes, making it unclear which design choices improve real-world data assimilation. We present the first controlled
benchmark of generative data assimilation for single-time near-surface analysis from real weather station observations. Using 11,849 NOAA MADIS stations across the contiguous United States and four near-surface variables, we hold the
dataset, observation operator, and deep learning architecture fixed, and measure spatial generalization at held-out stations. The
benchmark compares the major design choices proposed for generative data assimilation, including
diffusion versus flow matching, pixel versus latent-space formulations, and multiple inference-time conditioning strategies, against a classical 3D-Var baseline. The
benchmark reveals three conclusions. First, the best generative methods outperform 3D-Var (35.7% vs. 33.3% RMSE improvement over ERA5), although 3D-Var receives the ERA5 field at the analysis time as its background and the generative methods receive none. Second, full-gradient guidance consistently outperforms stop-gradient and initial-noise optimization. Third, other choices provide little measurable benefit:
diffusion and flow matching perform nearly identically under matched conditions, and latent-space variable mixing does not help. Both advantages widen when stations are sparse. Together, these results identify which components of generative data assimilation improve spatial generalization in near-surface analysis.
Read the full abstract ▾
All authors Ruizhe Huang, Qidong Yang, Jonathan Giezendanner, Sherrie Wang
Artificial Intelligence · Cross-listed from cs.GT
Outer Diversity of Condorcet Domains
Piotr Faliszewski, Jan Jabrocki, Mateusz Słuszniak et al. (+3 more)
A Condorcet domain is a set of rankings over a given candidate set, such that every election that consists only of (an odd number of) votes from the domain has a transitive majority relation. We study outer diversity of Condorcet domains, i.e., a measure that quantifies expected swap distance from a random vote to a closest one in the domain. We numerically analyze outer diversity for maximal Condorcet domains with few candidates, and then we establish its asymptotic behavior for several special domains, mostly obtaining theoretical results.
Read the full abstract ▾
All authors Piotr Faliszewski, Jan Jabrocki, Mateusz Słuszniak, Krzysztof Sornat, Stanisław Szufa, Tomasz Wąs
Artificial Intelligence · Cross-listed from cs.CL
Sequential Functional Structured Tucker Compression for Large Language Model Attentions
Jiangfeng Chen, Xinyu Wang, Tianshuo Yan et al. (+4 more)
Post-training compression of LLM
attention is often formulated as independent
matrix approximation, ignoring both the shared structure among
attention projections and the representation shift introduced by earlier compression. We propose FTC, a sequential structured compression framework that adapts the approximation to the current compressed model while jointly exploiting the native Q/K/V head structure under a fixed storage budget. The output projection is handled separately to account for the changed post-
attention representation. FTC requires neither fine-tuning nor gradient-based recovery. Across seven decoder-only LLMs from 6B to 32B parameters, FTC achieves the lowest WikiText-2 perplexity among the compared methods at every tested keep ratio on five modern GQA models, with the largest gains under aggressive compression. The improvements transfer to downstream tasks and remain substantial at the 32B scale.
Read the full abstract ▾
All authors Jiangfeng Chen, Xinyu Wang, Tianshuo Yan, Hanwei Wu, Xiao-Wen Chang, Yang Zhang, Lei Ding
Artificial Intelligence
Robust Nash Alignment under Preference Uncertainty
Shihab Ahmed, Debamita Ghosh, David Tang et al. (+3 more)
Preference-based
alignment methods typically optimize against a single preference model, and can therefore be brittle when pairwise preferences are uncertain: noisy, heterogeneous, or shift after deployment. To address these issues, we propose Robust Nash
Alignment, a game-theoretic framework for
alignment to uncertain pairwise preferences. Our formulation has a major learner seeking a policy with a large worst-case win rate against both an
adversarial competitor and any preference kernel lying in an ambiguity set around a nominal preference. When the ambiguity set captures the uncertainty in preferences, the resulting robust objective of the game directly yields a certified lower bound on worst-case performance. However, we note this problem is computationally challenging to optimize, and to address this, we introduce a four-player primal-dual proxy game involving the leader policy, follower policy,
adversarial kernel, and dual variable, and develop a single-loop optimistic mirror descent-ascent
algorithm for it. We show that the proxy always lower-bounds the truncated hard-constrained objective, quantify the proxy-to-hard gap, and characterize an exactness condition under which the proxy recovers the robust objective. We then prove an \(\mathcal{O}(1/\sqrt{T})\) average-iteration convergence for the proxy-game duality gap, which implies a near-optimal robust policy for the original robust objective. Experiments on controlled tabular games and LLM
alignment with uncertain preference further validate the convergence theory and show improved performance over nominal baselines.
Read the full abstract ▾
All authors Shihab Ahmed, Debamita Ghosh, David Tang, Yudan Wang, Alvaro Velasquez, Yue Wang
Artificial Intelligence
ReLiveGym: Evaluating Long-Lived Agents over Weeks of Replayed Reality
Xisen Jin, Jingheng Li, Zhenglun Chen et al. (+2 more)
As large
language model (LLM)
agents become widely adopted, they are increasingly deployed for tasks that require persistent monitoring or recurring actions (e.g.,
market analysis). These
agents are expected to operate unattended for days or weeks, act at the right timing, and adapt to the dynamic environment over time. These challenges are not fully captured in the existing long-horizon
agent work, as they often consider a static environment that is not temporally changing. We introduce ReLiveGym, a diagnostic evaluation environment of long-lived tasks in which
agents act sparsely over simulated weeks of chronologically replayed real-world news,
market, and social-media streams. The tasks span diverse levels of time sensitivity,
reasoning intensity, and recurrence. Across eight base
language models, we investigate how model choice and harness design affect
agent performance on such long-lived tasks. Our results show that how
agents determine when to act arises as an important harness-design axis for long-lived tasks; and that the optimal design varies across tasks and sometimes model choices as well. We also evaluate how continuous learning from hindsight feedback affects performance and addresses failure modes observed in these long-lived tasks. These findings indicate model choice, action timing mechanism, and use of feedback as important considerations in the design of long-lived
agents. Code: https://github.com/SaharaLabsAI/ReLiveGym
Read the full abstract ▾
All authors Xisen Jin, Jingheng Li, Zhenglun Chen, Junyi Du, Xiang Ren
Artificial Intelligence · Cross-listed from cs.LG
Initialization Improves LLM-Driven Discovery
Mansi Sakarvadia, Marco Ciccone and Colin Raffel
Large
Language Models (LLMs) have been used for novel discovery of
algorithms,
theorems, drugs, and other tasks through the use of harnesses that
prompt an LLM to iteratively optimize an objective. In this work, we study the relationship between the population of previous iterates and eventual discovery success. We generalize past work on harness design to develop a suite of 12 harnesses called 'Modular' and characterize their performance across 5 diverse discovery tasks, finding that discovery success is brittle and sensitive to harness design. We uncover mode collapse, characterized by a dramatic drop in the diversity of iterates, as a common failure mode. We find that popular state-of-the-art harnesses and diversity-inducing harness interventions, which aim to prolong this collapse, yield inconsistent gains. Our results instead uncover that the performance of early discoveries is predictive of eventual success. We therefore propose a universally applicable intervention that performs an initial stage of parallel exploration in order to initialize subsequent iterative optimization. Our method provides consistent gains across many harnesses and target applications, confirming the importance of initialization in LLM-driven discovery.
Read the full abstract ▾
Artificial Intelligence
Meta-Multi-Agent Reinforcement Learning for Fast Adaptation of Interactive Policies with Applications to Autonomous Driving
Huiwen Yan, Kyriakos G. Vamvoudakis and Mushuang Liu
This paper develops a meta-multi-
agent reinforcement learning (meta-MARL) framework to enable fast adaptation of interactive policies in a multi-
agent system (MAS). Meta-reinforcement learning (meta-RL) enables
agents to rapidly adapt to new tasks/environments using a bi-level optimization mechanism. However, existing meta-RL generally focuses on single-
agent systems. Extending these frameworks and
algorithms to multi-
agent systems poses additional challenges, as tasks are characterized by not only the environment but also
agents' strategic interactions. To address these challenges, we model multi-
agent reinforcement learning (MARL) problems as Markov games (MGs) and develop a meta-MARL framework for rapid interactive policy adaptation across a distribution of MGs. A new concept, called meta-NE, is defined to describe the desired solution concept in a meta-MARL problem. Sufficient conditions for the equivalence between a meta-NE and a stationary point of the gradient-play-based meta-MARL
algorithm are established. Our evaluation on autonomous-driving tasks demonstrates that the proposed meta-MARL method achieves faster adaptation than pretrained MARL baselines, validating the effectiveness of our framework.
Read the full abstract ▾
Artificial Intelligence
R-GroundBench: A Diagnostic Benchmark for R-Group Groundingin Markush Molecular Editing
Xin Wang, Zichuan Ying, Xinna Lin et al. (+8 more)
Recent advances in AI for scientific discovery enable molecular understandingand design, yet
reasoning over incomplete chemical representations remainsunclear.Markush structures, which encode molecular families through variable R-groupplaceholders (\textit{R\textsubscript{1}}, \textit{R\textsubscript{2}}, X, etc.), are ubiquitous in pharmaceutical patents and requiregrounding across molecular, textual, and chemical information.However, existing molecule-
language benchmarks focus on fully specifiedmolecules, leaving R-group grounding largely unevaluated.We introduce R-GroundBench:, a diagnostic
benchmark built from real patent Markushstructures, featuring a Multiple-Choice (VQA) track with controlled difficultyand modality splits, and an open-ended Generation track.Our results reveal a substantial gap between recognition andmolecular grounding.While models achieve over 90% accuracy on Easy VQA, performance drops to56–66% on Hard VQA when shortcuts are controlled.Chemical-domain VLMs also remain unreliable, achieving only 25.7–46.2% on HardVQA despite domain-specific pretraining.Moreover, Generation Exact Match remains below 20% for most models and below8% when visual input is required.These findings reveal that current AI systems lack reliable grounding andexecution for Markush editing, highlighting challenges for AI-drivenscientific discovery.
Read the full abstract ▾
All authors Xin Wang, Zichuan Ying, Xinna Lin, Junqi Zhang, Hanyi Xiong, Tianyu Gao, Hairong Zhang, Qixiang Hua, Botian Shi, Zhenhailong Wang, Kaicheng Yu
Artificial Intelligence · Cross-listed from cs.CL
How Divergence Becomes Decision Flips in Compressed Language Models
Beatriz Almeida Felicio
Compression reports summarize how far a compressed
language model moved from the dense one, usually by a KL divergence; a deployment that relies on the dense model's outputs needs to know how many of its decisions changed. We show that total variation, not KL, answers this directly. Across 802 compressed and perturbed copies of 19 open models on five corpora and nine mechanically unrelated perturbation families, the rate at which the arg-max
token changes (the flip rate) tracks total variation at a ratio with median $1.05$, with no fitted constant. KL converts into flips only through its square root and a factor that varies fourfold across models and corpora, because KL averages over
tokens before the root is taken; first-order statistics averaged per
token, such as Hellinger distance, avoid this, but reports rarely give them. As a result, of two compressors reported on different models and corpora whose flip rates differ by at least $10%$, KL assigns the smaller divergence to the one that changes more decisions in $11%$ of cases, total variation in $1%$. Two pre-registered tests mark the limits: on a held-out code corpus the ratio held for all eight models while three predictions about KL each failed for half of them or more, and on three new models with real kernels it stayed in its band for 37 of 38 checkpoints but fell below one on code for two models. In vLLM speculative decoding, total variation measured under teacher forcing predicts greedy draft acceptance with a mean relative error of $1.1$–$2.4%$, without the task-specific calibration that KL needs.
Read the full abstract ▾
Artificial Intelligence
Backdoor Purification for LoRA-Tuned LLMs via Null-Space Projection
Jianwei Li and Jung-Eun Kim
With the rapid adoption of large
language models (LLMs) and parameter-efficient fine-tuning (PEFT) methods, the risk of backdoor attacks has become more severe. Existing backdoor purification methods typically rely on at least one of the strong assumptions, such as prior knowledge of triggers, access to clean references, or aggressive retraining, and they often lack comprehensive evaluations. These constraints substantially limit their practical applicability. To overcome these challenges, our work proposes purifying LoRA-tuned LLMs without these assumptions and even without post-hoc retraining of the suspect parameters. Our objective is to significantly reduce the attack success rates (ASR) while preserving both (i) the base model's general capabilities and (ii) the new downstream skills learned through the adapter. Through a series of ablation studies, we progressively scale our approach from a single layer in a text classification setting to a full-parameter LLM in the generative task. Through careful data curation and feature approximation, we extract high-fidelity backdoor directions and, for each layer or head, construct orthogonal null spaces in both the input and output channels, onto which the LoRA updates are projected. Empirically, our null-space projection method reduces the ASR from nearly 100% to less than 10%, while preserving the base model's benign performance and the adapter's learned abilities during downstream task adaptation.
Read the full abstract ▾
Artificial Intelligence
Ontology-Grounded, Reasoner-Verified Benchmarks for Evaluating LLM Reasoning in Scientific AI
Nishtha N. Vaidya, Stephan Grimm, Thomas Hubauer et al. (+1 more)
Large
language models (LLMs) increasingly underpin scientific AI applications that reason over structured knowledge, from biomedical question answering to materials informatics. However, their logical
reasoning often falls short, producing factual inaccuracies unacceptable in these settings. Reliable evaluation remains challenging: manual
dataset construction scales poorly, and LLM-based generation risks
embedding the very flaws it aims to measure. High-quality
benchmarks must ground both correct and incorrect labelled examples in explicit background knowledge, formally verifiable by a standard reasoner. We propose a pipeline that automatically generates ontology-grounded multiple-choice question (MCQ)
benchmarks from any sufficiently axiomatised OWL 2 ontology, with correct answers grounded in the ontology by design. Distractors are generated by perturbing the right-hand-side class expressions of class definition axioms, and their incorrectness is formally verified by an OWL reasoner via entailment checks. We evaluate the pipeline on three ontologies: Pizza (small, academic), PMDco (complex, materials science), and DOID (large, biomedical), generating 112, 2,491, and 15,216 MCQs respectively. Distractors span four semantic categories from class unsatisfiability to weakened subsumptions, enabling diagnostic evaluation of specific
reasoning failures. Items meet natural
language quality standards: mean LLM judge scores of 4.02, 4.36, and 3.36 out of 5 confirm fluency, and correct-answer-to-distractor similarity above 0.8 shows that wrong options cannot be dismissed on surface form alone. Six LLMs evaluated zero-shot achieve 41.1-76.8% accuracy, well above the 25% random-guessing baseline, indicating the
benchmarks are challenging and discriminative. This work is a step towards more reliable
benchmarks for assessing logical
reasoning in scientific AI.
Read the full abstract ▾
All authors Nishtha N. Vaidya, Stephan Grimm, Thomas Hubauer, Thomas A. Runkler
Artificial Intelligence · Cross-listed from cs.LG
Learning Transferable Skills using Goal-Conditioned Bisimulation
Mohammad Amin Abbasfar, Farbod Azimmohseni and Mohammad Hossein Rohban
Unsupervised skill discovery has emerged as a promising approach for leveraging reward-free
datasets to pretrain general-purpose policies. However, current skill discovery methods either require access to expert data or exhibit limited generalization, failing to transfer effectively to previously unseen layouts. A key challenge is to learn representations that capture the temporal structure of the environment while remaining robust to variations across layouts. To address this issue, we present an objective for learning action-aware temporal representations that satisfy the functional equivariance property while preserving the local temporal structure of the environment. Building upon this
embedding, we further propose unsupervised skill discovery using bisimulation, which learns transferable skills by conditioning the behavior of skills exclusively on the subset of state features that directly affect their execution. This enforces invariant behavior across different layouts, enabling skills to transfer effectively to other configurations. Finally, through comprehensive empirical evaluations, we show that skills learned in a given environment can be effectively applied to solve downstream tasks in various environment layouts, demonstrating strong out-of-distribution generalization.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
LabBook: Harnessing Experimental History for Efficient LLM-Driven Discovery
Bo Yuan, Wenqian Ye, Zelin Zhao et al. (+4 more)
Evolutionary approaches to LLM-driven discovery often generate new programs from a small set of selected ancestors. This keeps contexts manageable but can omit useful evidence from other experiments, whereas including the full experimental history produces long, redundant contexts. We introduce a simple, single-
agent discovery harness built around LabBook, an
agent-maintained
memory that serves two complementary roles: guiding
retrieval of relevant evidence from a complete experimental log and informing the generation of new solutions. At each iteration, the same
agent combines its
memory with retrieved evidence and jointly produces the next program and an updated LabBook. This separates complete history retention from selective context construction, without requiring an explicit population or branching search structure. On 49 Frontier-CS problems, LabBook improves the observed quality-cost trade-off over the evaluated evolutionary baselines with two backbones, while remaining competitive across nine additional mathematical, systems, and heuristic-design tasks. Code will be released at https://github.com/BoYuanVisionary/LabBook.
Read the full abstract ▾
All authors Bo Yuan, Wenqian Ye, Zelin Zhao, Lama Moukheiber, Henry Kautz, Aidong Zhang, Yongxin Chen
Artificial Intelligence · Cross-listed from cs.CL
Closing the Loop: Practical Training Recipes for Looped Language Models
Andrei Marchenko, Viacheslav Bezrukov, Oleg Kashurin et al. (+5 more)
Looped
language models increase effective depth by repeatedly applying a shared block of layers, but existing large-scale recipes require multi-stage training over trillions of
tokens, while the benefits of recurrence remain difficult to separate from differences in data and training. In this work, we establish practical training recipes for looped
language models, with three main results. (1) We develop a compute-efficient from-scratch pipeline that reduces the training budget from 7.7T
tokens in Ouro to 310B
tokens while retaining strong
reasoning performance. Pretraining followed by high-quality mid-training, together with learning-rate warmup and stronger exit-gate regularization, enables stable recurrent training without prior multi-stage schedules. (2) Under controlled comparisons, our 1.4B LoopLM outperforms a parameter-matched dense model trained on the same data and
token budget on all 12 evaluated
benchmarks, including +14 points on GSM8K, +10 on MATH, and +22 on DROP. At matched inference compute, it approaches a 3.9B dense model on mathematical
reasoning and reading comprehension while using only 36% as many parameters. (3) We introduce a minimal recipe for converting pretrained dense models into looped ones: a single learned input-mixing scalar and a smoothed exit loss, with no step-specific parameters. Applied to Qwen3-1.7B-Base, Looped Qwen improves over an identically continued dense baseline on every evaluated
benchmark across two data regimes, with statistically clear gains on GSM8K, MATH, and MMLU-Pro on the curated mixture. Together, these results make looped
language models substantially cheaper to train from scratch and practical to introduce into existing pretrained checkpoints, while isolating the gains due to recurrence itself.
Read the full abstract ▾
All authors Andrei Marchenko, Viacheslav Bezrukov, Oleg Kashurin, Inessa Fedorova, Dmitry Bocharov, Yuliana Shakhvalieva, Maria Tikhonova, Valerii Ternovskii