Artificial Intelligence · Cross-listed from cs.CL
GlossoGen: Emergent Language in Complex Multi-Agent LLM Interactions
Elias Stengel-Eskin, Newton Sander, Carlos Bonetti et al. (+4 more)
The growing rate at which LLM
agents interact with one another raises key questions about
language evolution in multi-LLM-
agent settings, with implications for safety and monitorability as well as for linguistic accounts of LLMs. To address these questions, we introduce GlossoGen, a novel platform for studying multi-
agent language evolution in complex scenarios. Within GlossoGen, we build the SaveVeyru scenario, which requires
agents with partial information to communicate under pressure. We find that
language evolution does occur between LLM
agents, that the resulting
languages are compositional and morphologically productive, and that they deviate from the LLMs' English prior in ways that render them incomprehensible to humans. Moreover, we identify several qualities essential to this
evolution: pressure towards efficiency; the strength of the models backing the
agents; and access to a "postmortem" stage in which
agents can agree on linguistic conventions. Importantly, we observe that different conditions govern the transmission of
language to new
agents. Specifically, we find that
agents learn new
languages from usage alone, take an active role in this learning, and that while stronger models are required for novel
language emergence, weaker models can learn an existing
language once it has emerged. Taken together, our results indicate that current LLMs have the potential for cumulative cultural
evolution – previously attested only in humans – with mixed populations of
agents developing capacities that go beyond their lowest common denominator.
Read the full abstract ▾
All authors Elias Stengel-Eskin, Newton Sander, Carlos Bonetti, Sasha Boguraev, James Bowler, Hale Sirin, Simon Kirby
Artificial Intelligence · Cross-listed from cs.CR
Defense-as-Skill: Evolving Runtime Guard Skill for Skill-Augmented Agents
Xiaofang Yang, Ziqi Miao, Chen Qian et al. (+3 more)
Skill-augmented
agents load reusable skills as persistent runtime context, improving task performance but also giving malicious skills a durable channel for steering future actions. Such skills may leak secrets, corrupt code, bypass approvals, or stage data for exfiltration only after a concrete user task and workspace state make the unsafe action appear useful. This makes pre-install vetting insufficient and calls for runtime, task-conditioned protection. We propose Defense-as-Skill, a defense paradigm that implements the runtime guard itself as an installable, inspectable, and editable skill. Our guard, SkillSonar, runs alongside untrusted task skills and checks sensitive actions against the user's task boundary, routing each action to an allow, replan, or confirmation decision without modifying the underlying
agent runtime. To study this setting, we construct SCOPE-R, a task-conditioned
dataset covering 6 risk families and 21 sub-categories, with 206 attack-confirmed malicious instances and 43 benign tasks. We then improve SkillSonar on the SCOPE-R training subset using runtime guard-skill
evolution, a Monte-Carlo Tree Search procedure that evolves the on-disk guard skill from feedback on the rollouts. Across Claude Code and OpenClaw, the evolved guard substantially reduces attack success while maintaining a favorable safety-utility trade-off; on repeated GLM-5 runs, ID ASR drops from 0.482 to 0.104 and OOD ASR from 0.606 to 0.115, with gains transferring across victim models, held-out risk families, and external
benchmarks. These results show that runtime guards can be skill-native, transparent, and evolvable policy artifacts for skill-augmented
agents.
Read the full abstract ▾
All authors Xiaofang Yang, Ziqi Miao, Chen Qian, Dianbo Sui, Jing Shao, Lijun Li
Artificial Intelligence
Harness-of-Harness: Multi-Day Autonomous Software Development with Continual Improvement
Haoyang Yan, Min-le Su, Hangfan Zhang et al. (+6 more)
This paper studies autonomous software development, in which LLM-based coding
agents transform high-level requirements into complete, functional, and usable software systems without human intervention. We introduce Harness-of-Harness (HoH), a framework that enables coding
agents to continually improve software during autonomous development. HoH operates on existing coding-
agent harnesses, and organizes their executions into iterative planning-coding-testing loops. To sustain improvement across loops, HoH balances repair with capability growth, scopes development into small and verifiable increments, separates implementation-time testing from independent evaluation, and constrains verifiable outputs rather than prescribing
agent workflows. It progressively exposes deliverables, role-specific tools, and skills, encourages reuse rather than recreation, and maintains versioned project histories. On GameCraft-Bench, FrontierSWE, and ProgramBench, three harness-model pairs (Codex with GPT-5.5, OpenCode with DeepSeek-V4-Pro, and Pi with MiniMax-M3), HoH consistently outperforms the corresponding standalone harnesses, achieving an average relative gain of 52.25 percent and a maximum gain of 82.86 percent after three iterations. In a multi-day deployment with more than 70 iterations, HoH autonomously develops a first-person-shooter game, featuring a coherent storyline, fully implemented core mechanics, human-playable experience, polished visuals and integrated audio. Github: https://github.com/Flesymeb/HarnessOfHarness Project Page: https://flesymeb.github.io/HarnessOfHarness/
Read the full abstract ▾
All authors Haoyang Yan, Min-le Su, Hangfan Zhang, Zhanhao Li, Chen Zhang, Shao Zhang, Yang Chen, Lei Bai, Shuyue Hu
Artificial Intelligence
Parsing the Stream: A Live Trace Model for Long-Horizon Agents and Their Observers
Egor Pakhomov and Erik Nijkamp
A long-horizon
agent's trace outgrows both of its consumers: the human observer monitoring the run, and the
agent itself, whose bounded context the trace must be folded back into. We present a live trace model, an append-only event ledger folded incrementally into typed run state and compiled into per-consumer views, and evaluate it for both consumers against deterministic ground truth. For the observer side, evaluated with an LLM reader as proxy, the compiled view answers monitoring questions using approximately 14x and 15x fewer input
tokens (by reader) and at 5-7x lower cost than a budget-capped single-call reading of the raw trace, with higher accuracy (0.85-0.87 versus 0.48). Because the questions were co-designed with the view schema, we treat the
token and cost reduction, conditional on schema coverage, as the transferable result. For the
agent, on 120-link sequential-dependency tasks, mechanisms that maintain the task's running statistic in per-step state succeed where full-context prompting fails (30/30 versus 8/30 under a clean protocol, n=30, labeled descriptive owing to
benchmark-system co-development); a
prompt-level scratchpad matches the fold's accuracy at lower cost, and a two-arm decomposition attributes the fold's accuracy to its deterministic aggregate and its cost advantage to its compactness. The fold's remaining value over cheaper alternatives is deterministic auditability and serving the observer from the same state. We derive eleven candidate requirements for trace folding from observed failures and delimit them with an order-sensitive task family on which the fold ceases to help. Code,
benchmarks, a regenerable synthetic corpus, and all workbench traces are released.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CR
When Safety Routing Breaks: Understanding Alignment Fragility under Benign Fine-Tuning
Yitong Guo, Xiaoyi Chen, Siyuan Zhang et al. (+2 more)
Benign fine-tuning severely weakens the safety
alignment of large
language models (LLMs), so we study why refusal behavior is so fragile. While prior work often attributes this failure to gradient conflict, we propose a fundamentally different Fisher-
geometric explanation: safety Fisher is low-rank, and
alignment makes the safety geometry flatter while preserving an output-routing pathway. After 100 benign fine-tuning examples, this pathway is selectively re-sharpened in output-side MLP modules, explaining the asymmetric fragility: safety can collapse to high attack success rates, while general utility degrades mildly. The routing view also explains why few safety examples can restore refusal behavior, indicating that internal safety-relevant representations are preserved. Finally, we show that LoRA and ASAM mitigate early collapse by suppressing output-side sharpness, but their protection weakens at larger fine-tuning scales. Overall, safety failure is best understood as a disruption of a low-rank output-routing mechanism
Read the full abstract ▾
All authors Yitong Guo, Xiaoyi Chen, Siyuan Zhang, Xiaofeng Wang, Haixu Tang
Artificial Intelligence · Cross-listed from cs.LG
Efficiently Estimating Optimal Hyperparameter Scaling Laws through Power-Law Entropy Search
Zhiliang Chen, Sebastian Ament, David Eriksson et al. (+4 more)
Optimal hyperparameter scaling laws describe how the best hyperparameters for large
language model (LLM) training change with model and data scale, enabling practitioners to predict optimal configurations at production scales without expensive large-scale tuning. However, estimating these scaling laws conventionally requires exhaustive grid searches over thousands of training runs, consuming enormous computational resources. We introduce Power-Law
Entropy Search (PLES), a computational cost-aware acquisition function built on multi-fidelity
Bayesian optimization that efficiently estimates optimal hyperparameter scaling laws through adaptive experimentation. A key innovation in PLES is that it searches for candidates that reduce the overall uncertainty of a scaling law estimate, instead of optimizing a single objective function. At each iteration, PLES selects the candidate configuration that maximally reduces the uncertainty of the scaling law estimates per unit computational cost, naturally favoring informative small-scale experiments. We evaluate PLES on synthetic
benchmarks, surrogate models fitted to real LLM training data, and actual LLM pre-training runs. Across all settings, PLES converges to accurate optimal hyperparameter scaling laws using less than one-tenth of the computational budget required by conventional grid search and other baselines.
Read the full abstract ▾
All authors Zhiliang Chen, Sebastian Ament, David Eriksson, Maximilian Balandat, Bryan Kian Hsiang Low, Eytan Bakshy, Jihao Andreas Lin
Artificial Intelligence · Cross-listed from cs.LG
Learning Sparse Decision Trees via Transformer Variational Auto-Encoders
Giacomo Fidone, Alessio Cascione and Riccardo Guidotti
Decision trees are among the most widely used models in machine learning, largely due to their transparent decision logic, making them well-suited for high-stakes decision-making contexts. However, most existing learning
algorithms focus on predictive performance, overlooking the joint optimization of other desirable properties, such as structural sparsity. In this work we propose TREVIS, an approach for learning decision trees with respect to complex objectives, based on the exploration of the latent space of a Tree
Transformer Variational Auto-Encoder (TTVAE). By mapping decision trees onto latent representations, TREVIS replaces the discrete search space with a continuous one, enabling gradient-based optimization via a differentiable surrogate model. We experiment with TREVIS for learning decision trees that jointly optimize predictive performance and sparsity. Results show that TREVIS discovers decision trees matching the predictive performance of existing near-optimal
algorithms while improving their structural sparsity.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CV
Semantic-Guided Multimodal Preprocessing for Vision Transformer-Based Clear Cell Renal Cell Carcinoma Grading
Fatemeh Javadian, Zhu Chen, Zahra Aminparast et al. (+1 more)
Clear
cell renal
cell carcinoma (CCRCC) grading is essential for treatment planning, yet existing approaches either analyze patch-level images directly or focus solely on
nuclei-level classification, without linking to final tumor grading. We propose a semantic-guided
multimodal preprocessing method that integrates
nuclei classification maps from existing pre-trained models with RGB histopathology images for
Vision Transformer (ViT)-based CCRCC grading. Our approach employs classification map channel concatenation and multiplicative modulation, with optimized overlays to leverage
nuclei grading information, while preserving RGB textural features. Evaluation of multiple preprocessing strategies demonstrates that semantic-guided enhancement achieves 0.916 balanced accuracy, outperforming RGB-only baseline (0.707) and max-voting aggregation from prior studies (0.427). Sensitivity analysis reveals that this 21 percentage point improvement over baseline persists even under simulated perturbation at rates matching current state-of-the-art
nuclei classification model error thresholds, suggesting both effective semantic utilization and practical robustness. These findings show that preprocessing-based
multimodal fusion can leverage the diagnostic potential of existing imperfect
nuclei classifiers, effectively bridging previously isolated fine-grained
nuclear-level analysis with coarse-grained ViT-based patch classification. Per-class recall was consistent across grades (0.93, 0.91, 0.91), indicating that gains are not concentrated in the majority class. Because the sensitivity analysis perturbs ground-truth maps rather than predictions from an actual
nuclei model, this result characterizes robustness under simulated error rather than deployment with a real upstream model, which remains for future work.
Read the full abstract ▾
All authors Fatemeh Javadian, Zhu Chen, Zahra Aminparast, Johannes Stegmaier
Artificial Intelligence · Cross-listed from cs.LG
Provably Safe Sim-to-Real Transfer
Tingting Ni and Maryam Kamgarpour
We address safe sim-to-real transfer, in which an
agent leverages an imperfect simulator and limited real-world interaction while ensuring safety throughout data collection in the real system. This problem arises in applications such as robotics and healthcare: simulators provide cheap data, but sim-to-real mismatch makes direct transfer unreliable, and collecting real-world data to correct this mismatch must itself be safe. Moreover, deployment objectives may vary across tasks, making it costly to collect new data for each reward function. We therefore formulate safe sim-to-real transfer as a reward-free safe reinforcement learning (RL) problem, in which data are collected once and reused to plan for arbitrary reward functions. We develop a computationally efficient
algorithm that identifies where the simulator and real dynamics differ, uses certified simulator transitions where they are reliable, and estimates mismatched transitions from safely collected data. With high probability, every policy deployed during learning is feasible, and the collected data support the computation of a feasible and near-optimal policy for any reward function. When the simulator is uninformative, our
algorithm recovers online reward-free safe RL while improving the best-known sample complexity by a factor of \(\widetilde{\Theta}(H/\xi^2)\), where \(\xi\) is the safety margin of a baseline policy. When the simulator is accurate on most transitions, this improvement grows to \(\widetilde{\Theta}(H^2|\mc S||\mc A|/(\xi^2|\mc B|))\), where \(|\mc B|\) denotes the size of the sim-to-real mismatch region.
Read the full abstract ▾
Artificial Intelligence
EdiTikZ: Scientific Figure Editing from Revision Trajectories
Christian Greisinger, Zhixue Zhao and Steffen Eger
Vision-
language models (VLMs) have shown strong performance in generating scientific figures from text or images. However, publication-ready figures often require iterative refinement, making scientific figure editing an important yet largely unexplored step toward interactive figure creation. Existing approaches rely on costly proprietary agentic systems, focus primarily on evaluation, or construct training supervision from synthetically generated edits. Instead, we leverage naturally occurring scientific revision and development trajectories as a scalable source of supervision. To this end, we introduce DaEdiTikZ, the first large-scale
dataset of revision-derived scientific figure edits, constructed by mining 391K plausible TikZ edit pairs from arXiv, GitHub, and TeX SE and inferring 781K directed edit instructions with a VLM conditioned on rendered figures and TikZ code. We further introduce DaEdiTikZ-Bench, a human-refined
benchmark with 690 instances, and train two compact Qwen3.5-based EdiTikZ models (4B and 9B) by jointly learning image-to-TikZ reconstruction and instruction-conditioned editing, followed by reinforcement learning (RL) with complementary rewards for rendered fidelity and edit application. Automatic evaluation places our 9B model above all tested baselines, while human evaluation with 9 annotators and 4,320 ratings places it above GPT-5.6-Sol and on par with Gemini-3.1-Pro. Under severe out-of-distribution shifts, it remains competitive with GPT-5.6-Sol near its 2K training sequence-length regime.
Read the full abstract ▾
Artificial Intelligence
Neuro-Symbolic Geometric Abstraction (NeuSOGA): From Observations to Symbolic Mathematical Representations
Qingde Li, Qingqi Hong, Zihan Li et al. (+1 more)
A fundamental challenge in artificial intelligence is the transformation of observations into explicit symbolic representations suitable for abstraction, interpretation, and
reasoning. While modern AI systems achieve remarkable perceptual capabilities through large-scale statistical learning, the resulting knowledge is typically encoded within latent parameters that are difficult to inspect or manipulate analytically. Inspired by Neuro-Symbolic AI and theories of human abstraction, this paper investigates the formation of symbolic mathematical representations from
geometric observations. We propose NeuSOGA (Neuro-Symbolic
Geometric Abstraction), a framework that progressively transforms observations into topological abstractions,
geometric abstractions, and ultimately symbolic mathematical representations. The architecture combines
topology-guided structural discovery using Euclidean Distance Transforms, foundation-model perception using Segment Anything, adaptive multi-scale
geometric abstraction, and symbolic synthesis through Implicit Area Splines. The resulting representation is an analytical implicit model supporting arbitrary-order smoothness, additive composition, and closed-form evaluation. Unlike neural latent encodings, the generated representation remains interpretable, editable, and mathematically explicit. Experiments on ModelNet40 point clouds, arbitrary-view projections, and segmented optical observations demonstrate that NeuSOGA transforms diverse observations into compact symbolic representations while preserving essential
geometric and topological structure across sensing modalities and viewing directions. NeuSOGA provides an interpretable and explainable pathway from observation to symbol and establishes
Read the full abstract ▾
All authors Qingde Li, Qingqi Hong, Zihan Li, Jie Tian
Artificial Intelligence · Cross-listed from cs.RO
Evaluating Multimodal LLMs as Generalist Vision-Language-Action Agents for Drone Control: Commanding, Approaching, Tracking and Searching
Jaewoo Park, Minyoung Lee, Sukmin Seo et al. (+11 more)
Multimodal large
language models (MLLMs) are increasingly used to make decisions for drones, but existing evaluations are mostly limited to simulation, to a single drone, and to targets that are visible from the start and do not move. We present DroneCATS, a
benchmark in which the MLLM alone controls the drone through four actions, and evaluate ten MLLMs from frontier models to small open models. The
benchmark varies target visibility, target motion, scene and simulator, and extends to real drones. Beyond a single drone, it adds a multi-drone suite, where one model drives multiple drones, and a commander + sub-
agents suite, where a commander model directs one sub-
agent for each of the drones. Across these settings, we find that frontier models achieve the highest success rates, but their latency causes them to lose moving targets, while small open models often do not declare arrival even when they reach the target. Guided by this failure analysis, we fine-tune a 2B model on drone action data synthesized from public RGB-D images; its success rate rises from 0% to 31% in simulation and outperforms the frontier models in real flight. With multiple drones, we find that every zero-shot model succeeds more often as the commander of one sub-
agent per drone than when it drives all of them from one context, and that commanding success follows general VLM ability. On real drones, we observe that the results broadly follow those in simulation, while latency and hardware limits matter more. We will fully open-source the
benchmark.
Read the full abstract ▾
All authors Jaewoo Park, Minyoung Lee, Sukmin Seo, Moonbin Yim, Hyunwook Yoon, Dohoon Ryu, Daehee Kim, Myungseo Song, Jihyuk Byun, Seunggyu Chang, Taeho Kil, Jiseob Kim, Bado Lee, Geewook Kim
Artificial Intelligence · Cross-listed from stat.ML
Measuring consistency via ensemble margin and local prediction variability: Auditing decision systems in the presence of predictive multiplicity
Sinjini Banerjee, Tim Marrinan and Anand D. Sarwate
The Rashomon effect is a machine learning phenomenon where equally accurate models produce different predictions for the same inputs (predictive multiplicity). Existing work primarily focuses on multiplicity within individual models, but in more complex decision systems, the impact of the Rashomon effect is less well understood. In this work, we study multiplicity from the perspective of
auditing incorrect ensemble predictions, where the decision to divert an instance for human review is based on a consistency criterion that combines the ensemble margin with a measure of local prediction variability for each constituent model. With mild assumptions about stability and smoothness, we show that the consistency scores of finite ensembles converge to the corresponding consistency score of the expected model from the Rashomon set as the ensemble size and the number of samples used to measure local prediction variability increase. To demonstrate the efficacy of the proposed criterion, we evaluate the framework with respect to
transformer models applied to natural
language understanding tasks and parameter-efficient fine-tuning of large
language models used for tabular data classification tasks. Our experiments show that ensembling models from the Rashomon set substantially reduces the risk of incorrect predictions going unchecked compared with
auditing a single model, while incurring only a moderate increase in the number of diversions. Moreover, the
auditing behavior of the full Rashomon set can be closely approximated by finite ensembles of relatively modest size, with the risk approaching zero for some
datasets. We further demonstrate that the proposed measure exhibits stronger agreement with established predictive multiplicity metrics than existing consistency measures, providing a more reliable way to capture multiplicity in the Rashomon set.
Read the full abstract ▾
Artificial Intelligence
EDGE: Error Dependency Graph-Guided Multi-Error Attribution in Multi-Agent LLM Systems
Jun Hou, Priya Pitre, Yi Fang et al. (+1 more)
Large
language model (LLM)
agent failures often contain multiple related errors rather than a single mistake. Existing attribution methods usually identify a responsible
agent, step, or root cause, but do not explicitly model dependency between errors. We introduce EDGE, an Error Dependency Graph-guided multi-Error attribution framework. EDGE constructs an error dependency graph from observed error events and validates a reliable causal subset through counterfactual rollout. The inference graph guides a two-stage LLM-as-judge detector for error attribution, and the intervention-validated subgraph provides a more reliable basis for explanation and repair analysis. Experiments on TRAIL and MAST show that EDGE improves category-level multi-error attribution across most evaluated models and settings. Experiments with adapted Who&When-style
prompts show that the graph helps across prompting strategies. These results suggest that dependency structure is a useful diagnostic prior for
agent failures beyond isolated root-cause prediction.
Read the full abstract ▾
All authors Jun Hou, Priya Pitre, Yi Fang, Xuan Wang
Artificial Intelligence · Cross-listed from q-bio.GN
PopPert: Population-level Joint-Distribution Modeling for Single-Cell Perturbation Prediction
Handong Wang, Jiaxin Qi, Haochen Feng et al. (+1 more)
Predicting transcriptional responses to specific perturbations is critical for understanding cellular regulatory mechanisms and accelerating drug discovery. Single-
cell RNA sequencing destroys each measured
cell, yielding only unpaired populations of control and perturbed
cells. However, existing methods typically model perturbation prediction at the single-
cell level and assume
cell-to-
cell correspondence, which conflicts with the unpaired nature of the observed data. To address this challenge, we propose PopPert, a framework that explicitly parameterizes population-level joint gene expression distributions for collective transcriptional state modeling. Given a control population distribution and a perturbation condition, PopPert predicts perturbation-induced changes in distribution parameters, eliminating the need for
cell-level correspondence and reducing sensitivity to single-
cell noise. To effectively capture gene co-expression patterns, PopPert leverages a low-rank Gaussian Copula to model cross-gene statistical dependencies and construct the joint gene expression distribution, additionally allowing sampling of synthetic perturbed single-
cell profiles. Across multiple single-
cell benchmarks spanning both genetic and chemical perturbations, PopPert achieves superior overall performance in differential expression recovery, perturbation effect estimation, and population-level distribution matching. These results establish population-level joint distribution learning as an effective paradigm for predicting transcriptional responses from unpaired single-
cell populations. Code for PopPert is publicly available at https://github.com/whd1125/PopPert.
Read the full abstract ▾
All authors Handong Wang, Jiaxin Qi, Haochen Feng, Baisheng Lai
Artificial Intelligence
SymFold: Synergizing Evolutionary and Structural Priors for Accurate Protein Inverse Folding
Handong Wang, Jiaxin Qi, Baisheng Lai et al. (+1 more)
Protein inverse folding aims to recover amino acid sequences for a given 3D
protein structure, underpinning broad applications such as enzyme engineering and drug discovery.Current methods often follow a serial pipeline, in which a structure encoder predicts a coarse sequence, which is then refined by
protein language models (PLMs). However, because PLMs only perform post-hoc sequence edits, the refinement is bounded by the quality of upstream predictions.Thanks to recent
multimodal protein language models (MPLMs), we could directly encode structure to generate sequences with pretrained structural knowledge, but we observe that they are not effective for inverse folding. Therefore, we introduce a symmetric dual-path architecture that both leverages PLMs for pretrained sequence
evolution knowledge and MPLMs for pretrained structural knowledge to iteratively guide
protein sequence generation.Through extensive experiments across standard
protein inverse folding
benchmarks, our method achieves state-of-the-art performance, surpassing prior approaches, and ablation studies validate the rationale of our symmetric design, revealing a promising direction for the community.
Read the full abstract ▾
All authors Handong Wang, Jiaxin Qi, Baisheng Lai, Jianqiang Huang
Artificial Intelligence · Cross-listed from cs.CL
CHARM: Character Hallucination for Multicultural Role Play Benchmark
Sunkyung Han, Nahyeon Park, Gaeun Seo et al. (+2 more)
Role-playing large
language models (LLMs) are expected to adopt a character's style while also respecting that character's knowledge boundaries. Prior evaluations detect character
hallucination but rarely distinguish whether errors arise from failure to recognize a boundary or from failure to comply despite recognition. We introduce CHARM, a multicultural
benchmark of 40 real and fictional characters drawn from five cultural-linguistic regions, and validated by native reviewers. It probes two boundary types, Temporal (historical vs. modern) and Cross-Universe (entities outside a character's narrative or historical universe), using abstention-enabled multiple-choice questions. We propose a two-stage evaluation that separates Boundary-Awareness (explicit recognition that a query is out of scope) from Boundary-Compliance (abstention when answering concrete questions). Evaluations across six LLMs show that
hallucination is driven predominantly by compliance failures. Models frequently acknowledge that a query lies outside the character's knowledge yet still provide factual, out-of-character answers. By re-posing the same questions to the target character, we confirm that a large fraction of these cases are verified parametric overrides; the model stores the relevant fact but fails to suppress it. We also observe systematic cultural variation in these failures, consistent with imbalances in how characters from different regions are represented in model knowledge.
Read the full abstract ▾
All authors Sunkyung Han, Nahyeon Park, Gaeun Seo, Seunghyun Yoon, JinYeong Bak
Artificial Intelligence · Cross-listed from cs.RO
Scalable Rao-Blackwellized Online Planning for High-Dimensional POMDPs
Jiho Lee, Nisar Ahmed, Kyle Hollins Wray et al. (+1 more)
Online planning under uncertainty remains a fundamental challenge for robotic systems operating in partially observable environments with high-dimensional state spaces. While sampling-based POMDP solvers enable approximate decision-making in large or continuous domains, their performance degrades as belief dimensionality increases due to the high variance inherent in Monte Carlo-based estimation. In this work, we extend the Rao-Blackwellized online POMDP (RB-POMDP) framework to improve its generalizability in high-dimensional settings through hybrid continuous-discrete belief representations. By analytically propagating uncertainty associated with marginalized state components during tree-based planning, the proposed approach reduces sampling-induced variance in value estimation. We demonstrate the effectiveness of this framework in a robotic search-and-rescue task by integrating it with FastSLAM 2.0. Experimental results show that the proposed planner achieves higher cumulative rewards using significantly fewer particles and planning simulations than purely sampling-based methods under equivalent computational budgets. These results suggest that structured high-dimensional robotic problems admitting tractable sufficient statistics can be effectively leveraged within the RB-POMDP framework for computationally feasible online decision-making.
Read the full abstract ▾
All authors Jiho Lee, Nisar Ahmed, Kyle Hollins Wray, Zachary Sunberg
Artificial Intelligence
Cheap Verifiers, Large Blind Spots: Measuring the Reliability Cost of Cost-Saving Cascades
Dushyant Rajput, Nirdesh Chauhan and Siddharth Kosaraju
Inference cascades cut cost by answering most queries with a cheap model and escalating a hard tail to a frontier model that acts as verifier. A natural extension closes the loop: fine-tune the cheap student on the verifier's rejections so the escalation rate, and cost, fall each round. We measure this loop on real LLMs and report four findings. First, the verifier's blind spot, the fraction of the student's wrong answers it accepts, is large and moves adversarially: it grows with student capability ($\beta$ from 0.12 to 0.55 as the student scales 0.5B to 32B) and shrinks with verifier capability, so it is worst in the cheap-student, cheap-verifier regime cascades exist to create. Second, buying it away returns the saving: a frontier verifier drives $\beta$ to about 0.05 but then escalates on 46% of hard-MATH queries against a 39% true error rate, paying the frontier price on nearly half of all traffic. Third, naive corrective fine-tuning on the verifier-rejected tail does not improve the small student but degrades and ultimately collapses it, across every teacher we tried (cross-family and same-family), so at this scale the self-improving loop is self-defeating. Fourth, through all of this the cascade's own dashboard, every metric computed through the verifier, reads a flat 3% error while true delivered error swings up to 32%: the system is blind to its own degradation by construction. We then give the theory that explains the blindness, a two-population conservation law, $\epsilon_\infty \lesssim q_0 \beta_0$, under which every in-loop metric improves while true quality does not, and a synthetic study that validates the mechanism. The practical conclusion: the reliability of a self-improving cascade cannot be read from any metric computed through its own verifier.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CL
Probing Factual Knowledge Transfer with Training Data Interventions
Romina Oji, Marc Braun, Marcel Bollmann et al. (+2 more)
Do
multilingual language models transfer factual knowledge across
languages during continued pretraining, or do they mostly recall facts learned directly from the target-
language data? To answer this question more reliably, we propose an intervention-based framework: starting from an English-pretrained model, we continue pretraining on Persian data from which specific facts have been systematically removed at varying levels of granularity. We construct SIFT, a resource of 500 triples across 20 relations, stratified by the cultural origin of each fact's subject into general (globally prominent) and Persian-related entities, designed for both systematic fact removal from training data and evaluation, with natively written Persian cloze templates. Our results show that fact transfer is very limited: under the strictest removal condition, a large majority of English-acquired facts fail to transfer into Persian. We further show that sentence-level co-occurrence removal is insufficient to eliminate fact signal, and that easier (randomly selected) negative candidate sets substantially inflate apparent transfer by rewarding shallow associative heuristics, while performance on a harder candidate set that allows for less reliance on heuristics is much lower. Finally, we show that source-
language entity frequency has a large influence, with Persian-related facts, which are orders of magnitude rarer in the English corpus, hardly transferring.
Read the full abstract ▾
All authors Romina Oji, Marc Braun, Marcel Bollmann, Marco Kuhlmann, Jenny Kunz
Artificial Intelligence
LEAP: Likelihood Elicitation and Aggregation for LLM-based Probabilistic Forecasting
Yufei Chen, Yiran Zhao, Xiaogang Xu et al. (+3 more)
LLM-based
forecasting systems have improved on real-world tasks such as financial
markets and sports outcomes, largely through stronger search and tool use. Many systems still ask an LLM to read all collected evidence together and produce the final forecast. We call this design Monolithic Prediction. It can obscure how individual evidence items affect the result and collapse uncertainty across competing outcomes. We propose LEAP (Likelihood Elicitation and Aggregation for Probabilistic
forecasting), which reorganizes how collected evidence is used in the prediction stage. LEAP examines each evidence item separately and elicits likelihood parameters that describe its implications for the target. An explicit prior and a deterministic probabilistic model then combine these likelihoods into a posterior distribution. This procedure supports continuous, single-choice, and multi-choice forecasts while preserving reproducible evidence contributions. We build a
benchmark covering
forecasting, information-seeking, and browsing tasks, and evaluate LEAP on our own
agent loop and several
agent CLI frameworks. Given the same evidence, LEAP improves most prediction and calibration metrics across models and remains stronger under controlled comparisons of prior access, inference budget, and aggregation.
Read the full abstract ▾
All authors Yufei Chen, Yiran Zhao, Xiaogang Xu, Qipeng Xie, Jiafei Wu, Zhe Liu
Artificial Intelligence · Cross-listed from cs.LG
Bandits in Prod: Hyperparameter Optimization at Inference Time
Louis Abraham, Tuan-Anh Nguyen and Nicolas Devatine
Many production systems can assess a configuration only by using it on live requests and observing noisy feedback. Modern agentic systems are a prominent example, with inference-time choices such as model selection,
retrieval depth, prompting strategy, and decoding temperature, yet often with no representative validation data. We formalize this setting as Online Hyperparameter Optimization (OHPO) and cast it as an infinitely many-armed bandit over mixed and conditional search spaces. We introduce IMABO, a general framework that combines any bandit policy for choosing among already sampled configurations with any oracle for proposing new ones. We instantiate it with IMOSS, a restart-free anytime policy whose active set grows as $t^{\beta}$, and prove an expected cumulative
quantile-regret bound of $O(p_\rho^{-1/\beta} + T^{(1+\beta)/2})$, where $\beta\in(0,1)$ controls active-set growth and $p_\rho$ lower-bounds the probability that a proposed configuration falls in the top-$\rho$ fraction of the search space. We combine IMOSS with three practical oracles: a Tree-structured Parzen
Estimator, an incumbent-mutation oracle driven by a per-coordinate bandit, and a pretrained tabular foundation model, all three improving over the uniform random oracle baseline. IMABO outperforms all baselines in terms of regret across diverse OHPO settings, from tuning classical machine-learning models to configuring LLM-based
agents. Our implementation is available at https://github.com/Tiime-Software/IMABO.
Read the full abstract ▾
Artificial Intelligence
Automated Event Log Generation from Unstructured Text Using Finetuned LLMs
Maximilian Seeth, Gabriel Marques Tavares and Daniel Schuster
Process mining (PM) provides a powerful framework for discovering and optimizing operational processes from event data. However, the efficacy of PM techniques is strictly predicated on the availability of structured event logs. Thus far, event logs have often been laboriously created by domain and process mining experts. This costly effort causes large portions of organizational knowledge, including incident tickets, manuals, and textual reports, to remain underutilized. We address this bottleneck by investigating the efficacy of Large
Language Models (LLMs) as automated data translators. We propose a scalable framework that leverages LLMs as data translators to bridge the gap between unstructured textual resources and structured event data. We finetune LLMs on a newly created text-to-log
dataset, demonstrating that the resulting models can extract high-fidelity event logs from unstructured resources. Our results show that this finetuning approach outperforms few-shot or zero-shot prompting by a large amount, highlighting finetuning as a necessary pre-condition for generating reliable event data. We conclude that our method provides a promising pipeline for making previously unused data available to process mining ecosystems, effectively expanding the possibilities of using PM to further investigate organizational workflows.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.IR
MIDR: Enrichment-Augmented Indexing for Multimodal Document Retrieval
Debanjan Mahata, Atharva Tendle, Daniel Preotiuc-Pietro et al. (+2 more)
Retrieval over visually rich documents has a representation problem: important content often lives in tables, charts, figures, and layout relations that plain OCR linearizes, corrupts, or omits. ColPali-family visual retrievers address this with patch-level multi-vector indexes and late-interaction scoring, keeping image-derived
retrieval on the query-time serving path. We introduce MIDR (
Multimodal Indexing for Document
Retrieval), a training-free framework for enrichment-augmented indexing that shifts
multimodal reasoning to index time. During ingestion, a
multimodal LLM converts rendered pages into verified textual fields that are indexed with BM25F and optionally fused with dense
retrieval, enabling text-centric serving over multimodally grounded evidence. On ViDoRe V3, MIDR Hybrid achieves 0.6219 average nDCG across five English domains, a 23.0% relative gain over BM25, remaining competitive with ColQwen2.5. On two French-document domains, enrichment bridges English queries and French page text, lifting BM25 from 0.1532 to 0.5448 nDCG and outperforming ColQwen2.5. Across all seven domains, MIDR leads ColQwen2.5 on four while using approximately 9x smaller index
memory and approximately 2x lower query latency. These results establish index-time
multimodal reasoning as a compelling accuracy-deployment alternative to serving-time visual late interaction.
Read the full abstract ▾
All authors Debanjan Mahata, Atharva Tendle, Daniel Preotiuc-Pietro, Yong Zhuang, Ozan Irsoy
Artificial Intelligence
A Composable Evaluation System for Reproducible Omni-Modal Foundation Model Evaluation
Hodong Lee, Sanghee Park, Dohoon Ryu et al. (+4 more)
Building an omni-modal foundation model means evaluating it across text, image,
video, and audio. Excellent evaluation toolkits exist for each modality, but their inference engines,
prompt conventions, and metric implementations are mutually incompatible, so practitioners end up maintaining separate environments for every toolchain and still struggle to compare results across them. OmniEvaluator grew out of this need in our own model development: rather than reimplementing
benchmarks, it connects existing inference engines and curated evaluation libraries at a higher level, exposing four inference backends, four evaluation frameworks, and over a thousand
benchmarks through a single interface. Every run is recorded as an artifact capturing the full configuration for exact reproduction, and results flow into a shared dashboard for cross-model comparison. A
federated mode shares GPU inference servers across concurrent evaluations, and a built-in verifier, small enough to run on CPU, keeps its score stable across engines and
prompts where rule-based scoring fluctuates under configuration mismatch, matching cost-efficient commercial LLM judges without their recurring API cost. The system, demo
video, and dashboard are publicly available. (https://github.com/naver-ai/omni-evaluator)
Read the full abstract ▾
All authors Hodong Lee, Sanghee Park, Dohoon Ryu, Jungwhan Kim, Junyeob Kim, Soyoon Kim, Geewook Kim
Artificial Intelligence · Cross-listed from eess.IV
GazeRefine: Expert Gaze as a Test-Time Prompt for Training-Free Medical Image Segmentation
Mohammed Oussama Benyahia, Marouane Tliba, Mohamed Amine Kerkouri et al. (+10 more)
Medical image
segmentation remains difficult to scale because high-performing methods typically rely on dense expert annotations and task-specific training. We introduce GazeRefine, a training-free framework that uses gaze as an inference-time
prompt for zero-shot medical image
segmentation. Sparse, duration-weighted fixations are converted into foreground and background priors that initialize semantic prototypes in frozen DINOv3 feature space. These prototypes are iteratively refined through foreground-background discrimination, feature-space affinity propagation, and anchoring to the initial gaze guidance, allowing
segmentation to extend beyond directly fixated regions while limiting semantic drift. GazeRefine requires no
segmentation masks, fine-tuning, adapters,
prompt encoders, or gradient updates. We evaluate the method on gaze-annotated polyp
segmentation and prostate MRI
segmentation. The results show strong performance on colonoscopy images and competitive performance on prostate MRI, supporting gaze-guided prototype refinement as a promising approach for
segmentation-label-efficient, human-in-the-loop medical image
segmentation. Our tools and code can be found in the following repository: https://github.com/MohammedOussamaBEN/GazeRefine.git
Read the full abstract ▾
All authors Mohammed Oussama Benyahia, Marouane Tliba, Mohamed Amine Kerkouri, Taifour Yousra, Bin Wang, Max Bengtsson, Gorkem Durak, Elif Keles, Zuheng Ming, Marek Penhaker, Azeddine Beghdadi, Ulas Bagci, Aladine Chetouani
Artificial Intelligence
Analog-DB: An Agent-First Analog Integrated Circuit Database, From Blocks to Systems
Danial Noori Zadeh and Mohamed B. Elamien
Sharing analog integrated
circuit designs remains difficult: foundry non-disclosure agreements restrict the process details a design depends on, and the testbenches behind published results are rarely released. We present analog-db, an open-source, versioned database built on a shareable design representation. A domain-specific
language captures each design as a process-neutral
topology, reusable testbenches, and a machine-readable datasheet under one schema, so a design is shared in full and re-simulates on the process kits it is bound to. A parameterization scheme exposes functional sub-blocks and device sizes as named parameters that carry their matching constraints, making
circuits composable and retargetable; a schema-governed contract and queryable catalog let AI design
agents discover and reuse them directly. Across the regulator corpus, all 23
circuit-kit bindings on three open kits meet their own recorded specification bands (typical corner, matched devices, no layout) and 10 of 23 meet a common class band. Seventeen of the 23 imported sizings failed their testbenches and closed under a gm/ID sizing loop driven by the annotated sub-block roles, typically within one to three iterations. In a supervised case study, a coding
agent working from the released artifacts sized the op-amp cores of a chopper instrumentation amplifier on an open 130nm kit, locating four hand-entry defects and a missing common-mode feedback loop that the sizing-only baseline did not repair. The database holds 68
circuits across sixteen classes, verifiable at schematic level under a tiered harness and tracked on a power/performance scoreboard, released at https://github.com/MacAnalog/spicexplorer-release.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CV
HiLRP: Toward One Trustworthy Explanation for Vision Transformer: Conservation-Valid Attribution via Attention Primitives
Sathiyamohan Nishankar, Pubudu Sanjeewani, Asanka Perera et al. (+1 more)
Vision Transformer (ViT) design has become increasingly diverse, with backbones combining convolutional stems, windowed, linear, or multi-axis
attention, patch merging, and spatial reduction in various configurations. This diversity poses challenges for existing attribution methods, whose assumptions often do not hold across ViT variants: Grad-CAM requires a terminal spatial feature map,
attention rollout assumes global softmax
attention, and layer-wise relevance propagation (LRP) requires module-specific rules. To the best of our knowledge, no existing method provides a unified attribution framework across this architectural space. We show that this architectural diversity can be captured by a simpler underlying structure. The
attention and resolution-reduction operators in current ViTs can be decomposed into four operation types: linear maps, bilinear mixing, normalization or gating, and reindexing. Each operation admits a relevance rule that satisfies conservation. Based on these rules, HiLRP supports new backbones by construction rather than by architecture-specific derivation, and its attribution maps decompose the prediction rather than relying on heuristic assumptions. We prove conservation and conditional equivariance and verify both to machine precision. Across 14 attribution methods and 10 architectures, we find that no prior method remains reliable across ViT families, while Faithfulness Correlation becomes uninformative for backbones robust to spatial masking. HiLRP alone preserves conservation across windowed, spatial-reduction, multi-axis, and linear-
attention models, where naive extensions can produce zero or inflated relevance. It also localizes attribution failures in class activation mapping, achieving 0.97 Pointing compared with 0.55 for competing methods on EfficientViT.
Read the full abstract ▾
All authors Sathiyamohan Nishankar, Pubudu Sanjeewani, Asanka Perera, Selvarajah Thuseethan
Artificial Intelligence · Cross-listed from cs.RO
EmbodiedSkills: World Action Model-augmented Cascaded Skills for Vision-Language-Action Agent
Wei Wang, Wenqiao Zhang, Yutong Lin et al. (+14 more)
Vision-
language-action (VLA) policies generate executable
robot actions, but plausible proposals can have different consequences for the task. We present EmbodiedSkills, a skill-based
agent framework that augments frozen VLA policies with world action models for consequence-aware action selection. The framework defines reusable skills for scene observation, context encoding, candidate generation, consequence prediction, outcome evaluation, candidate comparison, and action execution. Five stages organize their input-output dependencies in a shared feedback loop, accommodating both continuous action chunks and pick-and-place primitives. At each decision point, independently trained world action models predict the latent consequences of a common set of policy proposals. Selection combines their outcome scores and prediction disagreement with a lightweight comparator that estimates the benefit of an alternative from agreement in its predicted effects. The predictive and comparison modules learn from matched-state candidate branches, while the action policy and pretrained observation encoders remain frozen. On MetaWorld MT50 and CLIPort, EmbodiedSkills improves success rates from 76.28% to 78.64% and from 33.94% to 36.83%, respectively. An additional CALVIN evaluation with the frozen FLOWER policy increases five-task-chain success from 77.10% to 77.90%. These results demonstrate the applicability of consequence-aware selection across different policy backends and action representations.
Read the full abstract ▾
All authors Wei Wang, Wenqiao Zhang, Yutong Lin, Yuqian Yuan, Tianwei Lin, Jinhao Mao, Zhenxuan Fan, Mingjian Gao, Yang Dai, Wentong Li, Zheqi Lv, Zheng Dong, Yingjie Niu, Jiaqi Zhu, Jun Xiao, Chao Li, Yueting Zhuang
Artificial Intelligence · Cross-listed from cs.CL
Some Emotions Run Deeper: Layer-wise Probing and Causal Intervention in Large Language Models
Tian Fang, Gaël Guibon and Davide Buscaldi
Emotion is expressed in text along a wide spectrum, from surface lexical cues to inferences entangled with content. Most layer-wise analyses of
emotion in LLMs use a single corpus, leaving open whether the depth at which
emotion becomes accessible is a property of the model or also of the text source. We investigate this across three
datasets spanning different degrees of explicitness and contextualization in
emotion expression (Twitter posts, Reddit comments, and autobiographical narratives) and eight 1B–9B open-weight LLMs from the Llama, Qwen, and Granite families. We combine layer-wise probing with offline feature scaling and online forward interventions, transfer analyses, and an early-exit classifier. We find that (i) the best probing layer shifts systematically across corpora, from input-adjacent layers to over half model depth, and this ordering persists after matching label-by-length-bin distributions; (ii) across the evaluated settings, forward-pass interventions on probe-selected bands reduce test accuracy by 5–6 points more than same-width random bands ($q < 0.01$); (iii) selected bands transfer across
datasets and
emotion categories, suggesting partially shared affective information rather than strictly per-
emotion substrates; and (iv) probe-selected early-exit representations outperform full-depth exits by $6.9$ percentage points on average.
Read the full abstract ▾