Artificial Intelligence
When Rubrics Fail: Hallucinations Reveal Blind Spots in Medical AI Evaluation
Griffin Farrow, Lily Sijia Li, Jack Johnson et al. (+4 more)
Hallucinations can undermine clinician trust in LLMs, making it important that evaluation methods capture clinically relevant errors. Rubric-based evaluation has become the leading approach for assessing LLMs in medicine, but it is unclear whether rubric scores reflect such errors. We first study this in a controlled setting using MedHallu, finding that more specific rubrics better distinguish correct from hallucinated responses. To test this systematically, we develop a taxonomy of medical
hallucination types and a clinician-validated error-injection pipeline that creates matched correct and error-injected responses. Across HealthBench, HealthBench Professional, and LiveMedBench, our clinically relevant
hallucinations are missed by rubrics, often leaving scores unchanged. We find that rubrics are most effective when explicitly checking facts, and are less effective for additional or unexpected errors they do not anticipate. A preliminary
retrieval-based factuality check recovers some of the rubric-blind errors, suggesting a complementary approach. These findings reveal systematic blind spots in current medical evaluation of LLMs and suggest that rubric scores alone are insufficient to establish clinical reliability, potentially undermining clinician trust and confidence in clinical deployment.
Read the full abstract ▾
All authors Griffin Farrow, Lily Sijia Li, Jack Johnson, Tingyan Wang, Philip Torr, William Bolton, Fabio J. Fehr
Artificial Intelligence · Cross-listed from cs.LO
Supermartingale Certificates for Parametric MDPs
Kaushik Mallik and {\DH}or{\dj}e Žikelić
We consider the problems of formal verification and synthesis in parametric Markov decision processes (MDPs) with general measurable state and action spaces. The heart of our approach is a parameter flattening transformation, which allows us to transform parametric MDPs into semantically equivalent non-parametric MDPs. Building on this transformation, we introduce the novel notion of parametric supermartingale certificates, which generalize the traditional supermartingale certificates—used for non-parametric MDPs—to the parametric setting. We use our parametric supermartingale certificates to design
algorithms for verification and approximate synthesis in
polynomial arithmetic parametric MDPs. This leads to the first verification and synthesis
algorithms for parametric MDPs with general state and action spaces. We implement our
algorithms and experimentally evaluate them on several continuous parametric random walk
benchmarks.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
InRTL: Effective Intra-Inter Interaction Learning for Relational Tables
Weichen Li, Ken Zhong, Zheng Wang et al. (+2 more)
Relational table learning has recently emerged as an important research direction for modeling multiple tables connected through primary key-foreign key (PK-FK) relationships. Despite recent advances, a principled modeling framework tailored to this task remains underexplored. In this paper, we propose Intra-Inter Relational Table Learning (InRTL), a unified framework that explicitly models dependencies both within and across relational tables. Specifically, InRTL formalizes two complementary interaction patterns: intra-table interactions, describing associations among rows within the same table, and inter-table interactions, describing dependencies between rows across PK-FK-linked tables. To model these dependencies, we develop a column-aware table encoder to generate initial row representations, followed by
Transformer-based self-
attention and cross-
attention modules for intra-table and inter-table learning, respectively. To further improve scalability, InRTL incorporates linearized
attention and heterogeneous graph neural networks to simplify the self-
attention and cross-
attention operations. Extensive experiments on ten
datasets covering 24 real-world tasks demonstrate the effectiveness of our approach. Code is available at https://github.com/W1nterFloW/InRTL.
Read the full abstract ▾
All authors Weichen Li, Ken Zhong, Zheng Wang, Li Pan, Jianhua Li
Artificial Intelligence · Cross-listed from cs.SE
What is the Difference Between Me and You? Benchmarking the Quality Gap Between Human-Written and AI-Generated Code
Cristina Improta, Pietro Liguori and Domenico Cotroneo
AI coding assistants are becoming co-authors of production software, yet their evaluation centers on functional correctness, leaving open whether their code differs from human code in the quality dimensions dominating lifecycle cost. We compare human-written and AI-generated code at scale: 787,562 function pairs across Python, Java, and C, each human function mined from open-source repositories paired with implementations generated from its docstring by three AI assistants (OpenAI GPT models, DeepSeek-Coder, Qwen2.5-Coder). We characterize structural complexity and statistical naturalness, and map static-analysis findings onto Orthogonal Defect Classification for defects and the Common Weakness Enumeration for vulnerabilities, making authors and
languages directly comparable. AI-generated code is structurally compressed and stylistically templated: roughly half the size and branching of human code, clustering apart at the style level. Defect profiles differ in kind: human code concentrates issues of mature codebases, AI code repetitive boilerplate; security is
language-dependent, with LLMs producing more, and more severe, findings in Python and Java but fewer high-severity
memory-safety findings than humans in C. Once size is controlled for, complexity metrics carry little signal, while naturalness separates authors. Finally, we release CQBench, a
benchmark of 27,346 issue-prone tasks with baselines and an evaluation pipeline for quality assurance and security testing.
Read the full abstract ▾
Artificial Intelligence
Implicit Personality Representations in Humans and LLMs
Yilin Geng, Omri Abend, Eduard Hovy et al. (+1 more)
A century of psychology has found that the trait words people use to describe one another vary, but the relational structure among those traits, which ones go together and which oppose, is strikingly consistent across raters and cultures. We test whether the LLM (Qwen 2.5-7B-Instruct) reproduces this structure in its internal trait representations. From millions of crowd-sourced personality ratings of fictional characters, we build a human implicit-personality
matrix over hundreds of traits; from contrastive model activations, we build a matching
matrix over the same traits. The two relational structures align strongly (Mantel r = 0.77), and the agreement holds trait by trait as well as in aggregate. Two dominant axes of the model's trait representations recover the social and intellectual dimensions long known to organize human personality impressions, social warmth and intellectual competence. On held-out dialogue, projecting model activations onto these directions yields personality profiles that agree with human ratings. This work establishes a framework that enables comprehensive, human-grounded comparison between internal model trait geometry and the shared structure of human personality impressions.
Read the full abstract ▾
All authors Yilin Geng, Omri Abend, Eduard Hovy, Lea Frermann
Artificial Intelligence
Enabling and Understanding Personalization in AI-Generated Advertising Imagery
Victor Kolominsky-Rabas, Leopold Müller, Claudius Budcke et al. (+2 more)
Personalized marketing traditionally matches static products to customers, while dynamic creative optimization focuses mainly on AI-driven text personalization or basic product image modifications. We address this gap by developing and implementing an AI-based framework that generates personalized advertising imagery directly from customer data. We evaluate this framework in a two-stage within-subject study with N=100 participants across four products and three levels of personalization, varied by the amount and specificity of customer data used. Participants rated each image on attitude toward the advertisement, attitude toward the product, and purchase intention. Results show that participants perceive differences across personalization levels and evaluate AI-generated advertising imagery most positively at a moderate level of personalization. High personalization increases perceived personalization, which is positively associated with all three outcome measures, but also increases perceived creepiness, which is negatively associated with the outcomes and dominates the total effect.
Read the full abstract ▾
All authors Victor Kolominsky-Rabas, Leopold Müller, Claudius Budcke, Claas Christian Germelmann, Niklas Kühl
Artificial Intelligence
I Am AdMan: A Pipeline for Automatic Generation of Personalized Advertising Imagery
Victor Kolominsky-Rabas, Leopold Müller, Claudius Budcke et al. (+1 more)
Personalized marketing can increase customer engagement, satisfaction, and conversion. While existing personalization approaches have become effective at matching the right product to the right customer, the visual representation of advertisements remains generic and only weakly tailored to the individual. Prior research shows that generative artificial intelligence can improve the creation of personalized advertisements, particularly for text, and that image generation models can support scalable advertisement production. However, little research has examined how detailed customer information can be systematically translated into fully AI-generated, personalized advertising imagery at scale on a technical level. To address this gap, we propose AdMan, a multi-
agent pipeline that transforms customer data into personas, generates personalized advertisement images conditioned on product reference images, and applies an LLM-based judge
agent for automated quality control. We implement the pipeline with two different model configurations and evaluate it across four products, using six celebrity personas for qualitative inspection, and 100 real customer profiles, producing 1745 advertisements. The evaluation combines a qualitative expert focus group and a quantitative artifact-rate assessment. The results show that the pipeline can generate photorealistic and personalized advertisements. At the same time, performance varies substantially by product complexity and model configuration. Our findings extend the literature on AI-based personalized advertising by demonstrating the feasibility and current limitations of fully automated image generation for advertising.
Read the full abstract ▾
All authors Victor Kolominsky-Rabas, Leopold Müller, Claudius Budcke, Niklas Kühl
Artificial Intelligence
Residual Vector-based Reconstruction as Long-Context Recall Regardless of Context Window Size
MyungHoon Ryu, XinYu Piao and Jong-Kook Kim
Large
language models (LLMs) process long contexts, including long documents and lengthy conversations, but face
token-level
memory usage that increases proportionally to input length. Although model optimization and lossy
prompt compression are widely used, these methods still fail to solve the long-context recall problem beyond pretrained and size-constrained context windows. This paper proposes a long-context recall method that maintains near-constant GPU
memory usage as context length increases, without additional training. The main idea is to reconstruct facts using parameter activations in the LLM's feed-forward layers, which store residual vectors representing facts from the source document. Utilizing residual vectors allows the LLM to deterministically reconstruct query relevant facts without referencing the original document, preserving high fidelity and reducing
memory usage without fine-tuning weights. Experimental results show that the proposed method enables answering single-fact questions in two-million-
token story contexts where previous methods fail.
Read the full abstract ▾
Artificial Intelligence
Generative AI Use Cases In Real Estate Marketing: Adoption and Constraints in Germany
Victor Kolominsky-Rabas, Leopold Müller, Felicia Perpina et al. (+1 more)
Generative artificial intelligence (GenAI) is changing how work is organized and performed. Real estate marketing is a
prime example of this, yet evidence of GenAI in real estate
agents' day-to-day practice remains scarce. In this work, we report on our insights from a German-based empirical study with eleven semi-structured interviews. GenAI is already utilized across different activities, with marketing communication being the most prominent. Concrete use cases are emergent and unevenly adopted, with writing exposé texts being the only widely established one. Interaction is predominantly human-in-the-loop: GenAI drafts, structures, and retrieves, while real estate
agents curate, verify, and decide. Constraints stem less from model capability than from integration with listings and documents, data availability, and compliance in sensitive tasks. The study contributes a grounded map of existing and potential use cases and identifies tentative practical implications for adoption.
Read the full abstract ▾
All authors Victor Kolominsky-Rabas, Leopold Müller, Felicia Perpina, Niklas Kühl
Artificial Intelligence · Cross-listed from cs.SE
Separating Engineering Reasoning from DEXPI Serialization in LLM-Based Greenfield Surface-Process Design: A Three-Case Study for Underground Gas Storage
Qingchuan Zhu, Shuyue Tong and Pengju Ren
Large
language models can produce engineering descriptions and structured process representations, but standards-level serialization can substantially increase the generation burden. This diagnostic study examines whether separating engineering
reasoning from Data Exchange in the Process Industry (DEXPI) serialization changes where representation and engineering failures occur in constrained greenfield surface-process design for underground gas storage. We compare Direct DEXPI generation with generation of a lightweight Engineering Intermediate Representation (IR) on three cases: single-pressure injection, withdrawal and export, and dual-pressure injection. All six conditions use one fixed model snapshot, qwen3.8-max-0902, with one completed hosted generation per condition. Direct outputs are XSD-valid in 2 of 3 cases, while all 3 Engineering IR outputs are structurally valid under a minimal validator. Direct
prompt inputs contain approximately 121.8k-121.9k
tokens, compared with 617-699
tokens for the Engineering IR
prompts. Engineering feasibility does not uniformly favor the IR: one IR output is rejected for an explicit cooling-state contradiction. The cases also reveal two distinct Direct DEXPI failure modes: engineering inconsistency and standards-level serialization failure. The observed comparison shows that, in these evaluated method bundles, deferring DEXPI serialization substantially reduces representation burden and helps isolate serialization failure, but reducing representation burden alone does not eliminate engineering inconsistencies.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
Explaining Time Series Forecasting with Horizon-Resolved Attribution
Seunghan Lee, Jun Seo, Jaehoon Lee et al. (+9 more)
Recent advances in explaining time series (TS) models have produced methods that identify which past values a prediction depends on. However, most existing methods return a single importance vector, assuming that every predicted step depends on the same past values. In this paper, we show that this assumption does not hold, as different forecast steps depend on different past values. Motivated by this observation, we propose Horizon-Resolved eXplanation (HRX), which adds a horizon axis to the explanation, so that every forecast step receives its own importance map. HRX is a simple yet effective plug-in framework with three components: 1) an
estimator that reads these maps out of any differentiable forecaster without modifying the TS backbone, 2) an evaluation protocol that validates the horizon axis by measuring how much a single forecast step changes when the inputs an importance map ranks highest are removed, and 3) a rank criterion that predicts in advance whether the axis is worth resolving on a given TS. We further show that this step-wise dependence is low-dimensional, as the explanations of all steps are built from a few shared maps whose number does not grow with the forecast length. Extensive experiments across various backbones and
datasets show that the improvement comes from the horizon axis and holds for
estimators of previous explanation methods. Code is available at https://github.com/seunghan96/HRX.
Read the full abstract ▾
All authors Seunghan Lee, Jun Seo, Jaehoon Lee, Junhyeok Kang, Sangjun Han, Sungdong Yoo, Minjae Kim, Tae Yoon Lim, Dongwan Kang, Hwanil Choi, Soonyoung Lee, Wonbin Ahn
Artificial Intelligence
SteerDuplex: Steerable Duplex Speech Dialogue Models
Utkarsh Tyagi, Ramaneswaran Selvakumar, Advait Gosai et al. (+13 more)
Full-duplex spoken dialogue models support low-latency turn taking, interruption handling, and backchanneling, yet a key capability remains underexplored: steerability, the ability to reliably shift conversational behavior along attributes such as tone, persona, speaking rate, and voice style in response to user instructions. We introduce a taxonomy of text- and audio-based steerability that identifies substantial gaps in current full-duplex models. To address this gap, we introduce SteerDuplex, a Moshi-based full-duplex
speech model fine-tuned on natural conversations and synthetic dialogues targeting instruction following, vocal delivery,
reasoning, and duplex interaction. We further apply two-stage reinforcement learning (RL) with hybrid rewards, combining verifiable interaction checks and judge-based semantic feedback to improve timing and response continuity. To evaluate full-duplex spoken steerability, we introduce SteerBench, a
benchmark with 390 spoken
prompts and 1,067 human-authored binary audio and text rubrics spanning tone, persona, style/accent, and speed/length. On SteerBench, supervised training improves audio-steering average pass rate by 44.5 percentage points over the strongest evaluated open baseline. On Audio MultiChallenge, task average pass rate improves by 7 points over its strongest evaluated open baseline. RL further raises source-clean interruption response from 72.5% to 82.5% and reduces synthetic pause barge-in from 26.5% to 9%. Steering and aggregate task scores remain comparable or higher, while reward probes reveal reward hacking through incomplete responses. Our model and
benchmark support systematic research on spoken steerability, with reward analysis showing why timing gains must be evaluated alongside response completeness.
Read the full abstract ▾
All authors Utkarsh Tyagi, Ramaneswaran Selvakumar, Advait Gosai, Sonal Kumar, Nikhil Barhate, Isabell Sagar, Steven Li, Miheer Bavare, Daniel Quigley, Fabiola Tapia Carrillo, Jose M Patron E, Diego Macías Gutiérrez, Paul Song, Ramani Duraiswami, Dinesh Manocha, Yunzhong He
Artificial Intelligence · Cross-listed from cs.LG
Correlation-Guided Fast Machine Unlearning via Hessian Analysis
Ayushi Thakur, Ruchir Gupta, Amit Kumar Jaiswal et al. (+1 more)
The increasing adoption of machine learning in network and distributed security systems has created an urgent need for mechanisms that can selectively and efficiently remove the influence of specific training data to eliminate compromised or
adversarial data points from production models.
Privacy regulations such as GDPR's right to be forgotten also pose similar requirements. However, existing approximate unlearning techniques remain computationally prohibitive for deployment in real-world security systems, as they require repeated expensive Hessian-inverse-vector computations for each data point removal, creating a bottleneck when processing multiple related requests in scenarios such as intrusion detection systems, spam filters, and threat intelligence platforms. Thus, we introduce a computationally efficient unlearning framework that identifies correlated data points in the training set and applies a theoretically derived closed-form parameter update rule, achieving an $82\times$ wall-clock speedup over standard influence function unlearning while preserving model utility with a $10^{-2}$ improvement in accuracy over state-of-the-art baselines. Our method establishes theoretical guarantees and ensures numerical stability through Hessian damping. Our evaluation across seven diverse
dataset architecture combinations, including large-scale CIFAR-100 with ResNet-50, demonstrates superior forgetting effectiveness, with membership inference attack success rates of 0.660 and tug-of-war scores of 0.950.
Read the full abstract ▾
All authors Ayushi Thakur, Ruchir Gupta, Amit Kumar Jaiswal, Prayag Tiwari
Artificial Intelligence · Cross-listed from hep-ex
ResoSeg: Resonance Tagger using Transformer and Segment Model
Chunkai Li, Junhao Yin, Ke Li et al. (+1 more)
Deep learning has been widely applied across many areas of experimental high-energy physics, yet existing models address only event-level classification or object tagging and therefore still require reconstruction
algorithms tailored to each decay channel. We present the first application of
segmentation to resonance tagging at BESIII and introduce ResoSeg, a deep learning model that jointly performs particle-level
segmentation and event-level classification, enabling a one-pass analysis of resonance to anything decays while precisely reconstructing the relevant resonance properties. We demonstrate the reconstruction of $\eta_c$ with $e^+e^-\to\pi^+\pi^-h_c$, $h_c\to\gamma\eta_c$, $\eta_c\to\text{anything}$. The model is trained on BESIII-$\eta_c$
dataset, which is constructed with per-track true labels obtained via a Truth-Matching
Algorithm. Experimental results show that the average combined efficiency of ResoSeg is more than double that of the conventional 16-channel approach across energy points from 4.19 to 4.60\,GeV. The model generalizes to unseen energy points, adapts to other $\eta_c$ production modes through transfer learning, and remains robust against variations in the $\eta_c$ mass, width, and branching fractions, providing a general, resonance-aware model applicable beyond $\eta_c$ and BESIII. The source code is available at https://github.com/oashen/ResoSeg.
Read the full abstract ▾
All authors Chunkai Li, Junhao Yin, Ke Li, Jingde Chen
Artificial Intelligence · Cross-listed from cs.SD
Direct Preference Density Alignment for Conversational Audio Equalization
Ioannis Stylianou, Sven Ewan Shepstone, Jon Francombe et al. (+2 more)
Large
Language Model
alignment typically relies on learned proxy reward models, which significantly increase the
memory footprint during training and are notoriously prone to instability and reward hacking. While offline methods like Direct Preference Optimization (DPO) bypass the reward model, they lose the ability to perform online exploration. If no optimization constraints are applied, this can lead to format collapse in bounded, continuous spaces. To resolve this, we propose Direct Preference Density
Alignment: An alternative framework that removes the need for a learned proxy reward model while strictly preserving the benefits of online reinforcement learning. We leverage large-scale user data (approximately 90,000 samples) to construct non-parametric preference density maps, establishing an empirical reward surface. In addition to removing the reward model, Direct Preference Density
Alignment enables the combination of the online structural grounding of Group Relative Policy Optimization (GRPO) with the targeted offline refinement of DPO. We show that this GRPO+DPO combination achieves the highest performance, and in a blind audio equalization listening test, enables a 1.5B-parameter model to achieve perceptual parity with a carefully
prompt-engineered GPT-4o mini baseline, using only a fraction of the inference compute.
Read the full abstract ▾
All authors Ioannis Stylianou, Sven Ewan Shepstone, Jon Francombe, Pablo Martinez Nuevo, Zheng-Hua Tan
Artificial Intelligence
Beyond Generation and Accuracy: Diagnosing and Enhancing Visual Chain-of-Thought for Geometry Problem Solving
Zhitong Dong, Jicai Pan, Yingguo Gao et al. (+3 more)
While
multimodal reasoning has advanced rapidly, solving complex geometry problems critically hinges on active visual assistance, such as constructing auxiliary lines, spurring the rise of Visual Chain-of-Thought (VCoT). However, existing evaluations typically assess visual generation quality and final answer accuracy in isolation, failing to examine whether intermediate visual aids are geometrically valid, effectively utilized in subsequent
reasoning, or causally responsible for task success. To bridge this gap, we introduce GeoVAD-Bench, a diagnostic
benchmark that pairs a fine-grained five-dimensional trajectory diagnosis covering perception, auxiliary quality, utilization, deductive
reasoning, and final correctness with controlled No-Aux, Auto-Aux, and GT-Aux intervention settings to systematically isolate intermediate error modes, the causal gains of visual aids, and the resulting autonomy gap. Our findings reveal that while high-quality auxiliary aids offer substantial theoretical gains for
geometric problem solving, autonomous generation is frequently hampered by compounding errors across
geometric perception, faithful visual manipulation, visual-state grounding, and deductive
reasoning. Guided by these diagnostic insights, we establish a specialized data construction pipeline encompassing
geometric perception, diagram editing, and interleaved visual-textual
reasoning trajectories, and develop a progressive SFT and
multimodal RL training framework. The resulting model, GeoWeave-8B, outperforms the base model by +25.3% in final
geometric accuracy and achieves a +30.4% gain in process average across the four intermediate diagnostic dimensions.
Read the full abstract ▾
All authors Zhitong Dong, Jicai Pan, Yingguo Gao, Jingting Ding, Hao Chen, Jinjie Gu
Artificial Intelligence · Cross-listed from cs.LG
SIMS: Scale-Invariant Merit-Function-Based Scalarization for Multi-Task Learning
Zebin Chen, Fei Xing, Yang Chen et al. (+4 more)
Multi-task learning (MTL) requires navigating unavoidable trade-offs among competing objectives. This paradigm is frequently formulated as multi-objective optimization (MOO), where the scalarization is favored to reduce an MOO problem to a single objective. We empirically find that existing merit-function-based scalarization approaches are sensitive to the relative scales of different objectives in practical MTL, where task losses commonly differ by orders of magnitude. The optimization process often favors objectives with larger scales even though the underlying Pareto optimal solutions remains invariant to rescaling (i.e., multiplying an objective by a positive constant). To address this issue, we propose Scale-Invariant Merit-function-based Scalarization (SIMS) for MTL. Specifically, SIMS adopts a transformation-induced merit function to convert the MOO problem of MTL to a single objective that renders optimization invariant to the magnitudes of losses. Theoretically, we prove that the requirement for scale invariance uniquely determines this transformation to be logarithmic. We further show that this general transformation-induced merit function preserves weak Pareto optimality and admits a smooth surrogate with controllable approximation error. Extensive experiments on representative multi-task
benchmarks demonstrate that SIMS consistently outperforms existing scalarization methods and achieves state-of-the-art performance.
Read the full abstract ▾
All authors Zebin Chen, Fei Xing, Yang Chen, Hua Liu, Andy HF Chow, Yuhua Qian, Yu Zhang
Artificial Intelligence
Reproducing and Evaluating the Generalizability of Subliminal Learning in Open-Weight Models
Daan van der Weijden, Nathan Brack and Selene Baez Santamaria
In this reproduction paper we investigate subliminal learning, a consequence of
distillation where teacher models transmit behavioral preference traits through semantically unrelated data. The original paper explores two types of traits (animal preferences and misalignment), three data modalities (number sequences, code, and chain of thought), and several model families. We reproduce their experiments and extend the setup along three axes: new preference categories (actors and politicians), a new task (chess move generation), and an additional open-weight model (Ministral8B). We also run a controlled ablation on the numbers task's answer-space size (1-, 2-, and 3-digit sequences). We focus on open-weight models with accessible checkpoints on HuggingFace, since the original paper's GPT-4.x fine-tuning is no longer available. Our reproduction supports the original paper's claims, but our extensions show they are not universal as transmission strength varies across traits and tasks, and one model shows almost no effect at all.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
Clustering-Based Balanced Sampling and Allocation with Data Parallelism for High-Performance Fine-Tuning
Hyunjin Kim, Youngeun Nam, Jaemin Han et al. (+2 more)
Instruction-tuning
datasets for large
language models (LLMs) are often large, redundant, and imbalanced, limiting efficient adaptation. Naive large-batch fine-tuning repeatedly includes overrepresented sample groups while weakly covering underrepresented but informative ones, especially under data parallelism (DP) across multiple GPUs. We propose CluSTER, a Cluster-aware balanced Sampling framework for Training Efficient data Reduction in DP instruction tuning. CluSTER curates a representative reduced
dataset through gradient-space clustering and DP-aware balanced allocation, ensuring dual-level coverage across clusters and workers, while preserving the original data distribution by weighted update. As a result, CluSTER reduces redundant computation and improves training stability without compromising model quality. Across multiple instruction-tuning
datasets, CluSTER reduces training time by up to 69.6% with almost no accuracy loss compared to prior sampling and data reduction methods. Code is available at https://github.com/kaist-dmlab/CluSTER.
Read the full abstract ▾
All authors Hyunjin Kim, Youngeun Nam, Jaemin Han, Wonhyeok Choi, Jae-Gil Lee
Artificial Intelligence · Cross-listed from cs.LG
SCOPE-OPSD: Fisher-Conditioned Privileged Subspaces for On-Policy Self-Distillation
Yunmeng Chen, Kunyu Wang, Peihan Li et al. (+8 more)
On-policy self-
distillation (OPSD) scores student-generated prefixes with a solution-conditioned self-teacher, yet transfers supervision only through next-
token probabilities. We ask whether the aligned final-layer discrepancy offers a useful second channel, and how to test that channel without confusing its geometry with auxiliary strength. SCOPE-OPSD projects the privileged teacher-student residual onto a frozen rank-64 factor estimated from residual covariance and
language-model-head Fisher sensitivity. It reuses the forwards already required by OPSD and adds neither rollouts nor inference-time modules. A matched Random control preserves the structured factor's rank and nonzero spectrum and uses per-arm gradient-RMS calibration, isolating the effect of the data-dependent orientation. Across the complete 25/50/75/100-step trajectories for Qwen3-1.7B, 4B, and 8B, Structured is never below Pure OPSD, with strict gains in 11 of the 12 model-checkpoint combinations and an exact tie at 4B step 25. Structured also exceeds matched Random in 10 of the 12 combinations. At step 75 on Qwen3-1.7B, Structured exceeds matched Random by 1.39 Macro Avg@12 points in each of two independent training reruns. A cross-fitted diagnostic also shows 4.40 times greater held-out privileged-gap capture than the matched random orientation. The results support a compact, Fisher-conditioned privileged subspace for short-budget OPSD.
Read the full abstract ▾
All authors Yunmeng Chen, Kunyu Wang, Peihan Li, Yi Wang, Shuyin Xia, Yi Liu, Xinyong Cheng, Dehui Wang, Xiangyong Zhai, Yanxing Liu, Song Liu
Artificial Intelligence
From Collaboration to Capability: Internalizing Routed LLM Experts into Compact Reasoners
Frank Nie, Shuyao Wang and Ethan B. Liu
A compact controller can coordinate stronger experts by selecting whom to consult, formulating requests, and integrating their responses. We study whether learning from both the controller's decisions and the experts'
reasoning and code improves its generation after expert removal. We introduce Rivet for collaboration internalization: expert-augmented reinforcement learning applies a shared outcome signal to controller decisions and returned expert spans, and verified trajectory internalization consolidates complete successful interactions through format-aware supervised training. The deployed controller generates
reasoning, code, and interaction structure with local Python execution and no external LLM. Across seven competition-mathematics
benchmarks, RIVET-1.7B and RIVET-4B achieve average accuracies of $28.25\%$ and $44.16\%$; Stage~II improves RIVET-4B's accuracy after expert removal by $6.49$ points, and GPQA-Diamond results provide evidence of generalization to scientific
reasoning. Ablations show gains from ordinary trajectory supervision and additional format weighting, supporting the effectiveness of training on the content and structure of verified collaborations.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CL
Calibrated Ambiguity in Multimodal Language Models: Humans reach for cultural references, while models describe the picture
Cody Kommers, Mingrui Ye, Evelyn Gius et al. (+4 more)
Ambiguity is often treated as a bug for AI systems to resolve—but in human communication and culture, ambiguity can also be a generative resource. From humour to politics to art, people express themselves in words and images that are open enough to invite different interpretations, yet constrained enough to be interpretable. We operationalise this notion of calibrated ambiguity with a task drawn from the parlour game Dixit. We compare differences in clues generated by human vs
multimodal language models, based on a novel coding rubric for calibrated ambiguity, and find that models consistently exhibit ambiguity collapse (i.e., their outputs are over-specified, leaving no room for multiple legitimate interpretations). Unlike human clues, AI-generated clues also exhibit cultural flattening; they almost never make reference to culturally-situated knowledge, even when prompted to use allusion and figurative
language.
Read the full abstract ▾
All authors Cody Kommers, Mingrui Ye, Evelyn Gius, Daniela Mihai, Hoyt Long, Zheng Yuan, Drew Hemment
Artificial Intelligence · Cross-listed from cs.LG
TokenMapper: A Step Toward Interoperable Speech Token Translation
Tal Kozakov, Tal Rosenwein and Eliya Nachmani
Neural audio codecs discretize
speech into
token sequences, but the resulting
token spaces differ in vocabulary and codebook structure, preventing direct communication across models. This limitation affects applications such as conversational voice
agents and
speech to
speech translation systems where multiple
speech models must interact. As a result, transferring information between
speech systems typically requires decoding to waveform audio and re-encoding with a second tokenizer, increasing latency and introducing potential information loss. To address these limitations, we present TokenMapper, a direction aware framework for direct
token to
token translation between heterogeneous
speech tokenizers in the discrete domain. TokenMapper supports structurally mismatched
token spaces, including mappings between single codebook and multi codebook representations, under a shared effective
token rate. Experiments on GLM-4-Voice, MiMi and DualCodec show consistent cross model performance. Specifically, translation WER approaches native reconstructions within 2.5-6.8% absolute WER, human MOS for TokenMapper outputs ranges from 2.29 to 4.39, following the same direction level trends as UTMOS and end to end latency is reduced by 4.8-94.5% relative to waveform bridging, reaching up to 972 ms per utterance. These results provide a practical step toward cross model
speech token interoperability without intermediate waveform reconstruction.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.DC
RoofLang: Enabling AI-Driven Architecting of LLM Inference Systems
Ziyue Yang, Yuting Jiang, Lei Qu et al. (+1 more)
AI is beginning to make substantive contributions to LLM inference optimization. Existing AI optimizations are predominantly profiling-based. Profiling-bound feedback confines the search to the capabilities and performance of an existing software stack, preventing a fundamentally better architecture of LLM inference systems from being identified. To enable the AI-driven LLM inference system architecting loop, we argue that a general workload representation, a verifiable mutation space, and an implementation-independent evaluator are required. We present the RoofLang domain-specific
language (DSL) that provides these features. In our evaluation, RoofLang reveals that DeepSeek V4-series models could achieve 3.5-39.5$\times$ higher peak decode throughput than other representative models. This gap is disproportionate to their total parameter counts and arises largely from compact KV-cache designs that support larger batches and reduce
memory traffic. A persistent optimizer
agent further discovered several new architectures that improved both throughput and interactivity of DeepSeek V4 Pro on NVIDIA B300 by 6.23-50.1%.
Read the full abstract ▾
All authors Ziyue Yang, Yuting Jiang, Lei Qu, Peng Cheng
Artificial Intelligence · Cross-listed from cs.CL
Meddies-PII: A Multilingual Framework for Personally Identifiable Information Extraction in Clinical De-identification
Linh Uyen Le, Ho Dinh Tri, Christian Hoang et al. (+1 more)
Clinical de-identification relies on accurately identifying personally identifiable information (PII). However, manually annotated
datasets are costly to construct, while existing synthetic alternatives often provide limited details about their generation process or rely on relatively simple synthesis strategies. We introduce Meddies-PII-
Dataset, a corpus of one million synthetic clinical documents spanning seventeen
languages and nine PII labels. The documents are generated using attribute-conditioned
prompts and validated through thirteen deterministic gates that enforce structural and annotation consistency. To evaluate the
dataset's utility, we train Meddies-PII-Model, a BIOES
token classifier, and compare it with existing PII extraction systems using exact-match entity-level F1. Meddies-PII-Model achieves the highest performance among the evaluated systems on all reported
benchmarks, with a mean F1 of 0.827 across fifteen external
benchmarks, compared with 0.658 for the strongest baseline. Upon acceptance, we will publicly release the
dataset,
benchmark suite, model, generation framework, and evaluation code to support research on
multilingual clinical de-identification.
Read the full abstract ▾
All authors Linh Uyen Le, Ho Dinh Tri, Christian Hoang, Huy Hoang Ha
Artificial Intelligence · Cross-listed from cs.CV
Earth-Agent-Pro: Towards Real-World Full-Chain Earth Observation with Agents
Zhutao Lv, Chenhao Dang, Yi Feng et al. (+5 more)
Real-world Earth observation (EO)
agents must translate high-level scientific questions into executable workflows to acquire observations, prepare data, perform domain computations, and derive conclusions from runtime evidence. Existing EO
agents typically start from supplied observations, while
benchmarks typically provide prepared inputs or candidate answers, leaving full-chain open-world EO execution largely untested. We present Earth-
Agent-Pro, an execution-adaptive Plan-and-Execute framework using expert-authored skills to constrain planning and runtime tool use. Workflow-centered structured
memory records planned steps, accepted evidence, and their dependencies, enabling repair of only the affected workflow suffix when runtime evidence invalidates a step. Separate large
language model adapters use sequence-level supervised fine-tuning for planner workflow composition and node-level group relative policy optimization with locally verifiable rewards for executor tool-argument grounding. Earth-Bench-Pro instantiates 248 expert-curated task cores as 744 questions under three matched regimes. Its 248 Open-World Execution questions span RGB imagery, spectral observations, and remote sensing products, pairing high-level requests with runtime data requirements, executable trajectories, and open-ended answers grounded in execution evidence. With a shared GPT-5 backbone, Earth-
Agent-Pro achieves 66.13% LLM-as-Judge accuracy, exceeding ReAct by 20.95 points in this metric and 24.44 points in Tools-In-Order. Joint adapter tuning raises Qwen3.5-9B LLM-as-Judge accuracy from 38.31% to 50.00%, an 11.69-point gain over the untuned configuration. Planning-only evaluation and execution with the reference workflow show that the adapters improve workflow composition and argument grounding, respectively. Code and
datasets will be released soon.
Read the full abstract ▾
All authors Zhutao Lv, Chenhao Dang, Yi Feng, Yanpei Gong, Xiaolei Wang, Junyan Ye, Conghui He, Weijia Li
Artificial Intelligence · Cross-listed from physics.ao-ph
Optimizing Geoengineering Interventions Using Differentiable Climate Models
Pulkit Dubey, Dorian S. Abbot and Ashesh Chattopadhyay
The deployment of a geoengineering program to cool Earth's
climate may be imminent. It is crucial that tools be developed to ensure that such a program would achieve its objectives while minimizing disruption. Here we exploit recently developed differentiable atmospheric models to demonstrate a novel geoengineering control strategy. In the differentiable primitive-equation atmospheric model JAX-GCM we impose a uniform $+4$\,K ocean warming and ask what pattern of sea-surface temperature cooling – in five ocean-masked zonal bands of prescribed SST forcings whose amplitudes are free – returns land near-surface air temperature closest to the model's own unwarmed climatology. This idealized set-up represents a cooling pattern that could be delivered physically either by marine cloud brightening or stratospheric aerosol injection. Gradients through chaotic dynamics decorrelate from the true sensitivity beyond the Lyapunov horizon, so we optimize greedily over segments of 8 to 14 days, following receding-horizon control. The learned strategy removes $92.3 \pm 0.4\%$ of the realized land warming across a ten-member ensemble of two-year rollouts, and a three-year run sustains it. If we use the spatial pattern of land temperature as the optimization objective, the distributions of precipitation, evaporation, and specific humidity over land are restored as well, even though they are not included in the objective function. The learned strategy from JAX-GCM replayed in the AI emulators LUCIE and NeuralGCM without re-optimization is successful, suggesting robustness. These promising results demonstrate a strategy for designing optimal
climate interventions that can be applied broadly for geoengineering scenarios under consideration.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from eess.SY
Computing at Sea: Floating and Offshore Data Centres as a Pathway to Sustainable AI Infrastructure
Cheng Siong Chin, Jianhua Zhang and M. Venkateshkumar
The rapid expansion of artificial intelligence is transforming data centres into one of the world's fastest-growing sources of electricity demand. As AI systems scale in size and capability, the physical infrastructure supporting computation is approaching critical limits in energy availability, cooling capacity, land use, freshwater consumption, and carbon management. Conventional land-based data centres are increasingly constrained by urban land competition, grid congestion, environmental pressures, and lengthy permitting processes, raising fundamental questions about where future computing infrastructure can sustainably exist. This article examines floating and offshore data centres as an emerging alternative model for digital infrastructure. By relocating computation to marine environments, offshore systems can exploit the ocean's natural cooling capacity, reduce freshwater dependence, and enable direct integration with offshore renewable energy resources such as wind, wave, and
tidal power. Early deployments have demonstrated the potential for significantly improved energy efficiency and operational reliability compared with conventional facilities, while also opening new possibilities for distributed and resilient computing architectures. The article explores how offshore computing may reshape the future relationship between electrification, renewable energy, and large-scale AI infrastructure. It analyses the opportunities and trade-offs associated with marine deployment, including environmental impacts, engineering design challenges, economic feasibility, and regulatory governance. Rather than treating offshore data centres as experimental novelties, the article presents them as part of a broader systems-level transition in how society may power, cool, and sustain the next generation of computational growth.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CV
LettuceVisSim: A Simulator That Generates Lettuce Image Time-series for Vision-Based Reinforcement Learning
Ziye Zhu, Bert van 't Ooster, Congcong Sun et al. (+2 more)
Vision-based reinforcement learning holds strong potential for decision-making in controlled environment agriculture (CEA). However, its development is hindered by the scarcity of labelled crop images. To address this gap, LettuceVisSim, a lettuce growth simulator that generates labelled time series of crop images, was developed and validated. The simulator contains a process-based model (PBM) for shoot dry weight dynamics, a canopy layout
algorithm for deriving canopy layout representations from shoot dry weight, and a Unity rendering engine for image generation. Five findings support the simulator. First, the PBM reproduced shoot dry weight under dynamic plant-density management with $\mathrm{R}^{2}=0.84$. Second, a piecewise cubic regression mapped shoot dry weight to potential projected area with $\mathrm{R}^{2}=0.94$. Third, the canopy layout representation was validated using 12 experimental
datasets each having different dynamic environmental and spacing conditions. It reproduced the ground coverage ratio dynamics observed in measured images, achieving $\mathrm{R}^{2}=0.84$ when driven by measured shoot dry weight and $\mathrm{R}^{2}=0.40$ (0.76 excluding one outlier) when driven by PBM-simulated values. Fourth, the Unity rendering engine converted canopy layout representations into RGB and
segmentation images at less than 10~ms. Fifth, a demonstration showed that a lighting-control policy can be learned and applied by observing only crop images that were generated with LettuceVisSim, providing a proof of concept of
vision-based reinforcement learning in CEA using LettuceVisSim.
Read the full abstract ▾
All authors Ziye Zhu, Bert van 't Ooster, Congcong Sun, Eldert van Henten, Sjoerd Boersma
Artificial Intelligence
Information Specialization and Constrained Synthesis in Multi-Agent LLM Forecasting: A Prospective Live-Study of the 2026 FIFA World Cup
Julian Varghese, Lucas Bickmann and Sarah Sandmann
Large
language models are being organized into multi-
agent systems with specialized roles, but whether such specialization produces distinct forecasts and whether subsequent synthesis improves utility remains unclear. In this study, we carried out a live, prospective evaluation over the final 56 matches of the information-dense 2026 FIFA World Cup, keeping a frontier foundation model constant while assigning two primary
forecasting agents contrasting specialist roles: a quantitative specialist focusing on structured performance statistics and a news specialist focusing on current injuries, tactics and information from press conferences. Their forecasts were then reviewed by a separate critic before being combined by a meta-
agent, resulting in a sequential four-
agent model. Forecasts from the betting
market served as an external
benchmark. The news specialist obtained the highest mean probability-weighted Top-3 utility and matched the betting
market in Top-3 exact-score hits. Nevertheless, the two specialist forecasters agreed on at least two of the three scorelines in 50 out of 56 matches, and the meta-
agent never generated more than one scoreline outside the specialists' forecast set. These findings show that rapidly changing, unstructured information can provide a valuable
forecasting signal alongside structured statistics, whereas adding critic and meta-
agent stages does not necessarily create complementary information or improve on the strongest specialist.
Read the full abstract ▾