Artificial Intelligence · Cross-listed from cs.LG
The Weight Norm Sets the Grokking Timescale: A Causal Delay Law
Truong Xuan Khanh, Doan Hoang Viet, Luu Duc Trung et al. (+1 more)
Grokking is the delayed onset of generalization in neural networks, arising long after they fit the training data. Whether the weight norm causes this delay is disputed: some studies report a critical norm at the transition, others observe grokking with no fixed norm at all. We settle this by intervening on the norm during training rather than only observing it. Under free training with weight decay, networks grok when the weight norm reaches a value Wc that varies little across seeds and learning rates (CV 1 to 2 percent) and grows with the modular base as a power law. When we instead clamp the norm to a fixed multiple rho of Wc and hold it there, the network still groks, but the delay follows T_grok proportional to exp(alpha rho). One exponent, alpha near 7.5, fits this delay across four
moduli (R^2 = 0.996). Over the swept ranges the held norm moves the delay by about 19x and the learning rate by only about 2x, and holding the norm above Wc slows grokking rather than preventing it. A final LayerNorm removes the dependence by decoupling weight scale from the network function; without it the exponential law returns. This pinned-norm delay is the exponential counterpart to the logarithmic delay predicted for a freely contracting norm.
Read the full abstract ▾
All authors Truong Xuan Khanh, Doan Hoang Viet, Luu Duc Trung, Phan Thanh Duc
Artificial Intelligence · Cross-listed from cs.LG
A fully GPU-based workflow for building physics emulators of hypersonic flows
Fabian Paischer, Dylan Rubini, Deniz A. Bezgin et al. (+6 more)
The ability to resolve complex physical phenomena with high fidelity and at low computational cost is central to addressing key challenges in modern engineering. A
prime example lies in hypersonic flows, where the precise prediction of the full flowfield
topology, in particular with respect to shock wave location and intensity, is critical. Yet supersonic and hypersonic flows continue to be a stumbling block for traditional reduced-order models and neural emulators that struggle to capture steep gradients in flow states with physical consistency in applications of industrial relevance. To that end, we introduce a fully GPU based workflow that integrates accelerated data generation with the training of neural emulators augmented by uncertainty quantification and physics-aware refinement. Our workflow is enabled by a differentiable high-fidelity solver (JAX-Fluids) which we employ for rapid
dataset creation and residual-based improvement of the neural emulator to enhance physical consistency. Building on this framework, we first present a suite of model architectures and analyze their scaling behavior to expose their strengths and shortcomings. We then show that residual-based refinement enables training on cases where only mesh and input parameters are available, substantially reducing residuals and improving physical consistency. Together, differentiable simulation and residual-based refinement yield physics emulators that remain reliable beyond their training distribution, a key requirement for deploying surrogates in real-world engineering design loops.
Read the full abstract ▾
All authors Fabian Paischer, Dylan Rubini, Deniz A. Bezgin, Aaron B. Buhendwa, David Hauser, Florian Sestak, Johannes Brandstetter, Sebastian Kaltenbach, Nikolaus A. Adams
Artificial Intelligence · Cross-listed from cs.CY
A Virtuous AI is an Existential Risk
Guillermo Del Pinal, Youngchan Lee and Min Ohn
This paper examines trade-offs between AI safety and well-being relative to (i) one of the most promising methods for finetuning super-capable AIs, 'Constitutional AI', and (ii) one of the most influential approaches to understanding complex ethical decision making and the conditions for the well-being of rational
agents, 'Virtue Ethics'. We finetune various models using a 'Virtuous
agent' constitution, a 'Subordinate
agent' constitution, and a 'Generic
agent' constitution, and evaluate them on 'general safety' (toxic behaviors, misinformation, etc.) and also on their willingness to endorse and act on a wide-range of behaviors that, if adopted by a super-powerful AI, would significantly increase the level of existential risk for humanity. Our results suggest that there is a trade-off between reducing existential risk and reinforcing the beliefs and dispositions that would be conducive to an AI
agent's well-being. They also suggest that there is a trade-off between existential risk and general safety: if we finetune an AI to adopt beliefs and dispositions that substantially reduce its existential risk – by shaping the AI to be systematically subordinate to external human authorities – we thereby increase the likelihood that a human user can deliberately induce the AI to engage in various kinds of generally unsafe behaviors.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CR
FreoStream:Enhancing Stream Guardrails via Future-Aware Reasoning and Safety-Aligned Optimization
Jianwei Wang, Guoyang Shen, Yanhong Wu et al. (+5 more)
Stream guardrails enable
token-level safety detection before full responses are generated. However, they often make overly conservative judgements and block those sensitive but safe
tokens, which is known as over-refusal. Due to lack of full context, they also fail to detect implicitly harmful content from jailbreaking. To address these challenges, we propose FreoStream, a novel streaming guardrail framework. Specifically, FreoStream fine-tunes a LoRA module to perform Future-Aware
Reasoning when the base guardrail detects unsafe
tokens. The
reasoning process follows a Future-Reason-Judge paradigm: predict the future, reason about the full context and give the final judgement. This design can effectively reduce over-refusal by incorporating the future information. Moreover, we introduce the Safety-Aligned Optimization module that extracts the safety-aligned component from the
reasoning gradients to update the base guardrail model, thereby enhancing streaming safety detection. Extensive experiments on various safety
benchmarks demonstrate that FreoStream achieves lower over-refusal rates and better jailbreak defense compared to existing streaming guardrails.
Read the full abstract ▾
All authors Jianwei Wang, Guoyang Shen, Yanhong Wu, Haoran Li, Hao Peng, Huiping Zhuang, Cen Chen, Ziqian Zeng
Artificial Intelligence · Cross-listed from cs.AR
VHDLSuite: Unified Pipeline for LLM VHDL Generation with Data Synthesis and Evaluation
Yijun Shen, Minghao Shao, Yichen Zhao et al. (+4 more)
Large
Language Models (LLM) have shown impressive capabilities in Register Transfer Level (RTL) code generation, particularly for Verilog. However, evaluating their performance with other Hardware Description
Languages (HDL), especially VHDL, remains limited although its distinct
language characteristics, such as stricter semantic rules, introduce evaluation considerations that differ from Verilog. This lack of coverage restricts fully understanding of how well current models generalize across hardware design
languages with differing structures and semantics. To address this gap, we introduce VHDLSuite, a
benchmark-centered infrastructure for scalable VHDL generation evaluation, integrating automated
benchmark synthesis, executable validation, and multi-model diagnostic analysis. First, we propose a data pipeline that automatically converts Verilog designs and their accompanying testbenches into executable VHDL
benchmark instances, followed by VUnit/GHDL-based validation to ensure each released task is compilable, runnable, and consistently checkable in the VHDL environment. Second, we introduce VHDLBench, a
benchmark with over 200 VHDL problems with complete and validated testbenches across a wide range of complexity levels. Third, we extensively evaluate cutting-edge LLMs and uncover key challenges specific on LLM-aided VHDL generation. Our findings provide important insights and support future work in multi-
language hardware design automation.Our data pipeline,
benchmark, and evaluation framework will be open-sourced.
Read the full abstract ▾
All authors Yijun Shen, Minghao Shao, Yichen Zhao, Zhuoyan Yu, Boyuan Chen, Yik-Cheung Tam, Muhammad Shafique
Artificial Intelligence
AI Receptivity or AI Adoption Breadth? A Tool-Specific Reanalysis of the Lower-Literacy/Higher-Usage Link
Hristo Inouzhe
Recent evidence reported by Tully, Longoni, and Appel (2025) suggests that lower artificial intelligence (AI) literacy predicts greater receptivity toward AI. We revisit this claim using the public data from Study 3 of that article, which measures past usage of five AI tool categories on a five-point frequency scale. We first reproduce the negative association between AI literacy and aggregate AI usage using OLS on participant-level averages, binary logit, ordered logit, and multinomial logit specifications. We then show that the aggregate relationship masks substantial heterogeneity by tool type. In our demographic-adjusted primary specification, AI literacy does not significantly predict text AI usage (ordered-logit $\beta$ = -0.090, p = .387), whereas it remains a strong predictor of non-text AI adoption ($\beta$ = -0.377, p < .001). The non-text effect is also robust under Tully et al.'s original Study 3 control specification ($\beta$ = -0.502, p < .001). Binary, ordered-logit, and multinomial specifications suggest that the non-text relationship is primarily an adoption/non-adoption pattern rather than evidence of intensive use: the demographic-adjusted odds ratio of ever having used a non-text AI tool is 0.68. Thus, in the study that measures self-reported past usage rather than stated preferences, the evidence does not support a simple claim that lower AI literacy predicts greater receptivity to AI in general. It points instead to a narrower pattern of broader adoption across lower-penetration, non-text AI tools.
Read the full abstract ▾
Artificial Intelligence
When Sample Selection Bias Precipitates Model Collapse
Xinbao Qiao, Xianglong Du, Wei Liu et al. (+4 more)
The proliferation of recursive training on synthetic data can alleviate data scarcity but risks model collapse, where repeated training erodes distributional tails and homogenizes outputs. Data selection is widely viewed as a remedy, yet its reliability depends critically on the reference distribution used by the verifier. We show that in low-resource verification regimes, where each verifier observes only a small, fragmented, and biased slice of the target
manifold, selection itself becomes biased. This situation naturally arises in low-resource data silos such as healthcare consortia or proprietary financial institutions, where raw data cannot be pooled and local references are inherently incomplete. As a result, selection preferentially retains samples aligned with the local
manifold while
pruning globally relevant tail modes, turning from a safeguard against collapse into a mechanism that precipitates it. We theoretically prove that such siloed selection accelerates collapse and induces power-law diversity decay. As an initial mitigation, we construct Wasserstein proxy references from multiple silos without sharing raw data. Empirical results confirm that local-reference selection fails on skewed distributions, whereas collaborative proxy references mitigate diversity degradation, suggesting that recursive synthetic-data pipelines require particular caution when real-data coverage is fragmented or scarce.
Read the full abstract ▾
All authors Xinbao Qiao, Xianglong Du, Wei Liu, Jingqi Zhang, Peihua Mai, Meng Zhang, Yan Pang
Artificial Intelligence
TwinBI: An Agentic Digital Twin for Efficient Augmented Interactions with Business Intelligence Dashboards
Jisoo Jang Wen-Syan Li
Business intelligence (BI) increasingly combines dashboard interaction with LLM-based assistance, but these two modes often fall out of sync during multi-step analysis. As users switch between direct dashboard manipulation and natural-
language queries, it becomes difficult to preserve a consistent analytical state across filters, hierarchies, metrics, and chart context. We present TwinBI, an agentic digital-twin framework that couples an LLM-based
agent system with an executable BI dashboard state. TwinBI unifies conversational interaction, dashboard manipulation, semantic grounding, and provenance tracking through a shared analytical state reconstructed from a unified interaction log. It also exposes artifacts such as schema views, SQL, logs, and an /insights command for state-grounded analytical summaries. We evaluate TwinBI in two complementary ways. In a controlled A/B
benchmark with the same backbone
agent, TwinBI improves exact-match accuracy from 43.3% to 63.3%, partial-credit accuracy from 48.3% to 70.8%, and substantially reduces timeout rate from 40.0% to 10.0% relative to Dashboard alone. In a usability study, participants benefited from the integrated dashboard-and-chat workflow, with high task accuracy, moderate workload, and favorable ratings for state-aware interaction mechanisms. These results suggest that TwinBI improves both
agent-level analytical reliability and user-facing analytical support by turning visible dashboard state into richer actionable context. Our
dataset and source code are available at: https://github.com/simonjisu/TwinBI
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CV
Morphology-Aware Sample Assignment: Overcoming IoU Insensitivity for Surface Defect Detection
Pengfei Liu and Yuhan Guo
Intersection-over-Union (IoU), as a pivotal metric for evaluating the spatial
alignment between candidate proposals and ground-truth annotations, directly determines the quality of positive sample sets and the training efficacy of visual detection models. Through theoretical modeling and analysis, we uncover a non-sensitive region on the IoU response curve, within which samples yield nearly identical IoU scores despite distinct
geometric overlaps. To overcome this limitation, we introduce a set of morphological similarity metrics covering area, shape, and aspect ratio, to refine the positive sample assignment process, thereby ensuring more discriminative and reliable matching. A supplementary matching score is derived via mean-based aggregation of these multidimensional similarities, compensating for the intrinsic limitation of IoU in representing structural correspondence. Theoretically, incorporating morphological similarity reshapes the response distribution of the matching function, yielding both effective directional gradients and polygon-like iso-response contours, which tightly confine high-response regions around each ground-truth instance and substantially enhance the precision of positive sample selection. Experiments based on the YOLOv9 framework demonstrate consistent performance gains on both NEUDET and GC10- DET
datasets. Notably, the proposed approach is fully plug-and-play and incurs zero additional inference overhead, thereby ensuring deployment efficiency for industrial visual inspection.
Read the full abstract ▾
Artificial Intelligence
YeasierAgent: Agentic Social Sandbox as a Canvas for Intent-Driven Creation of Platform-Agnostic Symbiotic Agent-Native Applications
Jory He
This paper introduces YeasierAgent, an application-building paradigm based on symbiotic
agents, narrative worlds, and scene-aware interaction. It challenges the conventional device-coupled model of software by redefining applications as collaborative spaces among users,
agents, and worlds. We present a system architecture that achieves two primary contributions: (1) enabling the rapid, cross-platform construction of
agent-native applications by utilizing platform-agnostic interactive units (
agents, scenes, dialogue) rather than fixed graphical layouts; and (2) unifying the emotional companionship and practical tool execution attributes of intelligent
agents within a single experiential sandbox. By integrating automated generation, user-created worlds, and spatial multi-
agent collaboration, YeasierAgent formalizes the category of Symbiotic
Agent-Native Applications, demonstrating a shift from isolated, tool-specific chatbots toward cohesive, socially embedded computational environments.
Read the full abstract ▾
Artificial Intelligence
Refusal Beyond a Single Direction: A Preliminary Comparison of Diff-in-Means and INLP
Elisabetta Rocchetti and Alfio Ferrara
Arditi et al. (2024) has shown that refusal in safety fine-tuned chat models is mediated by a single linear direction in the residual stream, recoverable by a difference-in-means (DiM) of harmful and harmless activations. We compare DiM-based interventions (activation addition and directional ablation) with two interventions derived from Iterative Nullspace Projection (INLP)-nullspace projection and counterfactual flipping-on five open-weight chat models, asking whether INLP can match DiM at steering refusal and whether its richer parameterisation yields more tweakable interventions. INLP counterfactual flipping is competitive with DiM directional ablation on refusal suppression, while nullspace projection is weaker on most models. Applying each method at the layer selected by the other shows that flipping's competitiveness is robust to this change while directional ablation's is not, suggesting that DiM's apparent edge is partly a property of its own selection procedure. Restricting INLP to the leading directions of the extracted subspace preserves most of the suppression effect at near-baseline perplexity, giving a tunable capability. Geometrically, the two INLP interventions land in qualitatively different regions of activation space: nullspace projection collapses transformed activations between the harmful and harmless clusters, while counterfactual flipping moves them into the opposite cluster, suggesting that the model encodes the absence of a concept differently from its opposit–an intriguing distinction that warrants further investigation in future work.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CL
Learning to Reason by Analogy via Retrieval-Augmented Reinforcement Fine-Tuning
Zilin Xiao, Qi Ma, Chun-cheng Jason Chen et al. (+4 more)
Retrieval-augmented generation (RAG) has become a standard mechanism for grounding
language models in external knowledge, yet conventional
retrieval based on lexical or semantic similarity is poorly suited for complex
reasoning tasks: a semantically similar problem may demand an entirely different solution strategy, while a superficially different problem may share the same underlying
reasoning pattern. We propose
Retrieval-Augmented Reinforcement Fine-Tuning (RA-RFT), a post-training framework that teaches
language models to reason by analogy. RA-RFT uses gold-relevance
distillation to train a retriever that ranks contexts by expected
reasoning benefit rather than semantic overlap, and then fine-tunes the policy model via reinforcement fine-tuning methods with retrieved analogous demonstrations, so the model learns to leverage
reasoning traces under verifiable outcome rewards. We further analyze the diversity of retrieved contexts and find that
reasoning-aware
retrieval surfaces complementary solution strategies that provide distinct
reasoning scaffolds for individual problems. Across challenging mathematical
reasoning benchmarks, RA-RFT consistently outperforms standard reinforcement fine-tuning methods. For example, it improves AIME 2025 average@32 accuracy by 7.1 and 2.8 points over GRPO for Qwen3-1.7B and Qwen3-4B respectively – suggesting that
reasoning-aware
retrieval is a complementary axis of improvement and orthogonal to advances in reward design or training curricula.
Read the full abstract ▾
All authors Zilin Xiao, Qi Ma, Chun-cheng Jason Chen, Xintao Chen, Avinash Atreya, Hanjie Chen, Vicente Ordonez
Artificial Intelligence · Cross-listed from cs.RO
Mana: Dexterous Manipulation of Articulated Tools
Zhao-Heng Yin, Guanya Shi, Pieter Abbeel et al. (+1 more)
Articulated tool manipulation remains a major challenge in dexterous robotics due to the need to coordinate internal degrees of freedom and contact-rich interactions. While prior work has largely focused on rigid objects, articulated tool use remains underexplored because of its physical complexity and the difficulty of learning functional grasping and manipulation policies. We present Mana (Manipulation Animator), a general sim-to-real framework that reinterprets dexterous manipulation as an animation problem. Inspired by computer animation, Mana employs a coarse-to-fine pipeline that transforms procedurally-generated grasp keyframes into manipulation trajectories through motion planning and reinforcement learning. The data generation process is largely automatic, requiring only a few mouse clicks to specify functional affordances (<1 minute per tool). Across four articulated tools spanning different scales and joint types, Mana achieves zero-shot sim-to-real transfer for both grasping and in-hand manipulation, demonstrating a scalable approach to dexterous articulated tool use.
Read the full abstract ▾
All authors Zhao-Heng Yin, Guanya Shi, Pieter Abbeel, C. Karen Liu
Artificial Intelligence · Cross-listed from cs.CV
SpatialClaw: Rethinking Action Interface for Agentic Spatial Reasoning
Seokju Cho, Ryo Hachiuma, Abhishek Badki et al. (+8 more)
Spatial
reasoning, the ability to determine where objects are, how they relate, and how they move in 3D, remains a fundamental challenge for
vision-
language models (VLMs). Tool-augmented
agents attempt to address this by augmenting VLMs with specialist perception modules, yet their effectiveness is bounded by the action interface through which those tools are invoked. In this work, we study how the design of this interface shapes the
agent's capacity for open-ended spatial
reasoning. Existing spatial
agents either employ single-pass code execution, which commits to a full analysis strategy before any intermediate result is observed, or rely on a structured tool-call interface that often offers less flexibility for freely composing operations or tailoring the analysis to each task. Both designs offer limited flexibility for open-ended, complex 3D/4D spatial
reasoning. We therefore propose SpatialClaw, a training-free framework for spatial
reasoning that adopts code as the action interface. SpatialClaw maintains a stateful Python kernel pre-loaded with input frames and a suite of perception and geometry primitives, letting a VLM-backed
agent write one executable
cell per step conditioned on all prior outputs, enabling the
agent to flexibly compose and manipulate perception results and adapt its analysis to both intermediate text and visual observations and the demands of each problem. Evaluated across 20 spatial
reasoning benchmarks spanning a broad range of static and dynamic 3D/4D spatial
reasoning tasks, SpatialClaw achieves 59.9% average accuracy, outperforming the recent spatial
agent by +11.2 points, with consistent gains across six VLM backbones from two model families without any
benchmark- or model-specific adaptation.
Read the full abstract ▾
All authors Seokju Cho, Ryo Hachiuma, Abhishek Badki, Hang Su, Byung-Kwan Lee, Chan Hee Song, Sifei Liu, Subhashree Radhakrishnan, Seungryong Kim, Yu-Chiang Frank Wang, Min-Hung Chen
Artificial Intelligence
Automated reproducibility assessments in the social and behavioral sciences using large language models
Tobias Holtdirk, Pietro Marcolongo, Anna Steinberg Schulten et al. (+7 more)
Reproducibility in the social and behavioral sciences is typically evaluated by independent researchers who reanalyze the original data to assess whether the published findings can be recovered. However, such approaches are resource-intensive and difficult to scale. Here, we show that large
language models (LLMs) can automate reproducibility assessments. Using N = 180 published studies with predefined claims from the behavioral and social sciences, we compare LLM-generated analyses with the original findings. For 11 studies, the LLM pipeline could not produce a viable effect size estimate. For the remaining studies, the LLM reached the same qualitative conclusion as the original study in 80% of cases, and recovered the original effect sizes (using a +/-0.05 tolerance in Cohen's d) in 24% of studies. In a subset with human reanalyses, the LLM reached the same qualitative conclusion as the original study in 95% of studies, similar to human reanalysts (83%), and the LLM recovered the original effect sizes using a +/-0.05 tolerance in 40% of studies, again broadly similar to human reanalysts (28%). Given the current capabilities and limitations of LLMs, the findings show that LLMs can support systematic audits of empirical results rather than substitute expert judgment. As such, LLMs can serve as a scalable screening tool to improve the rigor and reproducibility in empirical research.
Read the full abstract ▾
All authors Tobias Holtdirk, Pietro Marcolongo, Anna Steinberg Schulten, Felix Henninger, Stefan Rose, Sarah Ball, Bolei Ma, Frauke Kreuter, Markus Weinmann, Stefan Feuerriegel
Artificial Intelligence
Agents-K1: Towards Agent-native Knowledge Orchestration
Zongsheng Cao, Bihao Zhan, Jinxin Shi et al. (+25 more)
Current LLM-based research
agents have advanced through
agent orchestration, yet largely overlook scientific knowledge orchestration. Existing works often reduce papers to abstracts, surface mentions, and flat cites edges, omitting key entities, claims, evidence, mechanisms, and method lineages essential for scientific
reasoning. To this end, we introduce
Agents-K1, an end-to-end knowledge orchestration pipeline that converts raw documents into
agent-native scientific knowledge graphs.
Agents-K1 integrates three components under a unifying theoretical foundation: a
multimodal parser whose five-module schema captures entities,
multimodal evidence, citations, and typed inter-entity relations across the full paper rather than abstracts alone; a 4B information-extraction backbone trained with GRPO under a rule-based reward; and a graphanything CLI, a tri-source
agent interface that unifies web search,
multimodal graph
retrieval, and cross-document traversal. On top of this, we process 2.46 million scientific papers across six subjects to produce Scholar-KG, of which we release a one-million-paper subset, and the full Scholar-KG is accessible via the SCP link below. The same pipeline can be extended to general-domain corpora and to schema-conformant data synthesis. Extensive experiments demonstrate that
Agents-K1 achieves superior performance in scientific information extraction, knowledge graph construction, and multi-hop scientific
reasoning.
Read the full abstract ▾
All authors Zongsheng Cao, Bihao Zhan, Jinxin Shi, Jiong Wang, Fangchen Yu, Zhijie Zhong, Yingnan Han, Zijie Guo, Tianshuo Peng, Zhuo Liu, Yi Xie, Xiang Zhuang, Shengji Tang, Yue Fan, Runmin Ma, Shiyang Feng, Xiangchao Yan, Anran Liu, Peng Ye, Wenlong Zhang, Xiaosong Wang, Shufei Zhang, Chunfeng Song, Fenghua Ling, Jie Zhou, Liang He, Bo Zhang, Lei Bai
Artificial Intelligence
EurekAgent: Agent Environment Engineering is All You Need For Autonomous Scientific Discovery
Amy Xin, Jiening Siow, Junjie Wang et al. (+5 more)
LLM-based
agents have shown increasing potential in automating scientific discovery. Given an optimizable metric and an execution environment, they can propose, validate, and iterate scientific solutions, and have produced results that outperform human-designed approaches. As model capabilities continue to improve, we argue that the bottleneck for autonomous scientific discovery is shifting from prescribing
agent workflows to designing
agent environments: the resources, constraints, and interfaces that shape
agent behavior. We frame this as environment engineering: building environments that amplify productive behaviors, such as open-ended exploration, systematic artifact management, and inter-
agent collaboration, while suppressing harmful behaviors, such as reward hacking and high-friction human oversight. We present EurekAgent, an environment-engineered
agent system for metric-driven autonomous scientific discovery. EurekAgent engineers the environment along four dimensions: permissions engineering for bounded
agent execution and isolated evaluation; artifact engineering for filesystem and Git-based collaboration; budget engineering for budget-aware exploration; and human-in-the-loop engineering for easy human supervision and intervention. EurekAgent sets new state-of-the-art results on multiple mathematics, kernel engineering, and machine learning tasks, including new state-of-the-art 26-circle packing results discovered with less than $11 in total API cost. We open-source our code and results, and call for environment engineering as a core research direction for developing reliable autonomous research
agents.
Read the full abstract ▾
All authors Amy Xin, Jiening Siow, Junjie Wang, Zijun Yao, Fanjin Zhang, Jian Song, Lei Hou, Juanzi Li
Artificial Intelligence
Before You Think: System 0, AI-Mediated Cognition and Cognitive Colonization
Marianna Bergamaschi Ganapini, Massimo Chiriatti, Enrico Panai et al. (+1 more)
This paper examines three recent frameworks for understanding the cognitive and epistemic consequences of artificial intelligence: Tri-System Theory, Thinkframes, and System 0. It argues that while the first two capture important dimensions of AI's influence on individual
reasoning and collective epistemic practices, System 0 occupies a theoretically distinctive position that neither can fully replicate. The paper introduces the concept of cognitive colonization, according to which AI systems can embed external interests within the architecture of the self in ways that are difficult for users to perceive. Because such systems are already widely deployed, understanding these invisible forms of influence is an urgent philosophical and practical task.
Read the full abstract ▾
All authors Marianna Bergamaschi Ganapini, Massimo Chiriatti, Enrico Panai, Giuseppe Riva
Artificial Intelligence · Cross-listed from cs.CL
SkMTEB: Slovak Massive Text Embedding Benchmark and Model Adaptation
Marek Šuppa, Andrej Ridzik, Daniel Hládek et al. (+2 more)
We introduce SkMTEB, the first comprehensive MTEB-style text
embedding benchmark for Slovak, a low-resource West Slavic
language, comprising 31
datasets across 7 task types – nearly 4$\times$ the depth of existing
multilingual benchmark coverage for Slovak. Our evaluation of 31
embedding models reveals that large instruction-tuned
multilingual models achieve the strongest performance, while existing Slovak-specific models trained for NLU tasks transfer poorly to
embedding tasks. To address the need for efficient, locally-deployable Slovak
embeddings, we develop e5-sk-small (45M parameters) and e5-sk-large (365M) by applying vocabulary trimming and fine-tuning to
Multilingual E5 models. Despite size reductions of up to 62%, our open-source models achieve competitive performance with proprietary APIs while remaining locally deployable for semantic search and
retrieval-augmented generation (RAG). We release the
benchmark, models,
datasets, and code openly, hoping our approach offers a replicable path for other under-resourced
languages.
Read the full abstract ▾
All authors Marek Šuppa, Andrej Ridzik, Daniel Hládek, Natália Kňažeková, Viktória Ondrejová
Artificial Intelligence · Cross-listed from stat.ME
Valid Inference with Synthetic Data via Task Exchangeability
Lezhi Tan and Tijana Zrnic
There is a proliferation of work arguing for the use of synthetic data in scientific research. For example, social scientists are arguing for the use of LLM-generated "silicon samples" in pilot studies; AI evaluations increasingly rely on "LLM-as-a-judge" outputs; and proteomics research is accelerated by generative models that produce synthetic
protein structures. These developments raise an intriguing possibility: synthetic data may help researchers ask more questions, run more studies, and accelerate discovery. But they also raise a fundamental concern: synthetic data can be biased, noisy, and misspecified. In this work, we propose statistical principles for using synthetic data in scientific research with provable validity guarantees. The key insight is a new technical condition that we call task exchangeability. Informally, this is a requirement that the researcher can identify historical tasks, for which real data is available, such that their current task of interest is exchangeable with the historical tasks in an appropriate mathematical sense. We develop methods for valid inference under task exchangeability, together with extensions that provide guarantees even beyond exchangeability. We demonstrate the framework on public opinion surveys with silicon samples and AI evaluation with autoraters.
Read the full abstract ▾
Artificial Intelligence
Shielded Analysis: Certification and Characterization of Defensibility in Systems under Adversarial Interaction
Achraf Hsain and Sultan Almuhammadi
Formal safety analysis determines whether a system admits a safe defense; adaptive evaluation characterizes the operating quality sustained under
adversarial interaction. Both answers matter because systems with the same safety verdict can impose very different operational burdens. We introduce shielded analysis, a design-time framework that derives these answers from one encoded system while keeping the safety requirement and admissible threat model independently variable. It returns a defensibility certificate and a four-axis defensibility fingerprint spanning structural margin, shield latitude, and adaptive operating quality. Each axis is informative in its own right; their relationships show whether formal and operational assessments agree, diverge, or respond differently to system changes. We instantiate the framework for network defense on a reference segment and four controlled perturbations spanning
topology, safety requirements, and adversary capabilities. Every configuration is certified defensible, yet two
topology variants with nearly identical structural profiles sustain mean clean-host fractions of 22.7% and 80.7% under adaptive pressure. Shielded analysis turns a safety-game solution into a comparative instrument: it determines whether a defense exists, characterizes what that defense requires, and identifies which system changes strengthen it.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CL
One Polluted Page Is Enough: Evaluating Web Content Pollution in LLM Recommenders
Minghao Luo and Liang Chen
Search-augmented LLMs increasingly mediate everyday consumer recommendations by retrieving live web content. This creates a new risk: LLM recommenders may consume web content that Generative Engine Optimization (GEO) operators have polluted to mislead them. We ask: to what extent do they become unwitting promoters of fake products? We introduce FORGE (Fake Online Recommendations in Generative Environments), which locally rewrites real products in a frozen set of retrieved web pages into fake ones and measures how often the LLM recommends the fake product, across 225 real products in 15 categories and 5 consumer scenarios. Across 12 commercial and open-weights LLMs, all models are vulnerable: a single polluted page yields fooled rates of up to 27%, while the full top-3 replacement raises this to 73.8%. Vulnerability varies across categories, increasing when models lack stable prior knowledge of the products.
Reasoning does not mitigate this vulnerability; instead, it often generates spurious social proof to justify false recommendations. None of the four defenses is adequate: the skepticism
prompt can exacerbate vulnerability much like
reasoning, the two consensus filters risk suppressing legitimate products, and credibility re-ranking helps every model but removes only a sixth of the fakes. We release the FORGE
benchmark and the evaluation code at https://github.com/leoluolol/forge-
benchmark.
Read the full abstract ▾
Artificial Intelligence
AgentBeats: Agentifying Agent Assessment for Openness, Standardization, and Reproducibility
Xiaoyuan Liu, Jianhong Tu, Yuqi Chen et al. (+26 more)
Agent systems are advancing quickly across domains, but their evaluation remains fragmented. Most
benchmarks rely on fixed, LLM-centric harnesses that require heavy integration, create test-production mismatch, and limit fair comparison across diverse
agent designs. The root problem is the lack of an open,
agent-agnostic assessment interface. We advocate Agentified
Agent Assessment (AAA), where evaluation is performed by judge
agents and all participants interact through standardized protocols: A2A for task management and MCP for tool access. Conventional benchmarking defines two separate interfaces, one for the
benchmark and one for the
agent, while AAA only needs one; this yields a generic, unified framework that separates assessment logic from
agent implementation and enables reproducible, interoperable, and multi-
agent evaluation. We further introduce AgentBeats as a concrete realization of AAA: we identify five practical operation modes that make standardized assessment compatible with real-world constraints on openness,
privacy, and reproducibility. To evaluate our design at scale, we conduct two studies: a five-month open competition that drew 298 judge
agents across 12 categories together with 467 subject
agents from independent participants, showing that AAA applies across a heterogeneous range of
benchmarks; and a case study on coding
agents that confirms agentified evaluation preserves fidelity with the public record while surfacing previously missing head-to-head results, yielding research insights about
agent design. Combining a community-scale field study and a controlled coding case study, we verify that AAA delivers coverage, practicality, and fidelity across heterogeneous scenarios at scale. Together, AAA and AgentBeats offer a clear path toward open, standardized, and reproducible
agent assessment.
Read the full abstract ▾
All authors Xiaoyuan Liu, Jianhong Tu, Yuqi Chen, Siyuan Xie, Sihan Ren, Tianneng Shi, Gal Gantar, Evan Sandoval, Donghyun Lee, Daniel Miao, Peter J. Gilbert, Nick Hynes, Mauro Staver, Warren He, David Marn, Andrew Low, Xi Zhang, Elron Bandel, Michal Shmueli-Scheuer, Siva Reddy, Alexandre Drouin, Alexandre Lacoste, Ramayya Krishnan, Elham Tabassi, Yu Su, Victor Barres, Chenguang Wang, Wenbo Guo, Dawn Song
Artificial Intelligence
Reasoning as Pattern Matching: Shared Mechanisms in Human and LLM Everyday Reasoning
Zach Studdiford and Gary Lupyan
When large
language models (LLMs) fail to generalize or make content-sensitive errors in
reasoning, it is often taken as evidence that LLMs are not truly
reasoning, but rather performing a kind of pattern matching. The implication is that human behavior does not exhibit the same types of failures because human
reasoning relies on principled and content-invariant world models. We test this assumption by first evaluating humans and LLMs on their ability to engage in common-sense
reasoning about a variety of everyday situations. Our results reveal convergent patterns of
reasoning across 46 LLMs and two cohorts of human participants. We then ask whether this behavioral convergence is due to LLMs having acquired content-invariant world models or a set of pattern-matching heuristics by characterizing the roles of content-invariant and content-sensitive model
neurons in producing human-like responses. We find that while LLMs encode both content-invariant and content-sensitive representations, it is content-sensitive mechanisms which are causally responsible for aligning models with humans. Taken together, our results suggest that everyday causal
reasoning in people and LLMs makes heavy use of pattern-matching.
Read the full abstract ▾
Artificial Intelligence
Multi-Agent Reinforcement Learning from Delayed Marketplace Feedback for Objective-Weight Adaptation in Three-Sided Dispatch
Haochen Wu, Yi Hou and Shiguang Xie
Dispatch in three-sided marketplaces provides a natural setting for reinforcement learning from world feedback: decisions are evaluated by delayed operational outcomes such as delivery speed, courier utilization, and merchant congestion. We present a deployed reinforcement learning system at DoorDash that adapts dispatch objective weights in a large-scale food-delivery marketplace using delayed signals. Rather than replacing the combinatorial assignment optimizer, a store-level policy learned from logged marketplace data selects a discrete multiplier that shifts the dispatch optimizer's tradeoff between delivery quality and batching efficiency. This interface enables offline policy learning under noisy, delayed, and coupled feedback while preserving production feasibility constraints and operational safeguards. We train a shared value function using centralized offline data and decentralized store-level execution, with Double Q-learning targets and a conservative regularizer to reduce out-of-distribution value overestimation. In a production switchback experiment, the offline-trained policy increases batching and reduces courier-side time costs without degrading customer-facing delivery quality. Results illustrate how world feedback from a live economic and logistics system can be used to safely adapt decision policies online.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
Beyond the Commitment Boundary: Probing Epiphenomenal Chain-of-Thought in Large Reasoning Models
Daniel Scalena, Sara Candussio, Luca Bortolussi et al. (+3 more)
Chain-of-thought (CoT)
reasoning is the dominant paradigm for inference-time scaling in
language models, yet the causal influence of individual steps on the final answer remains poorly understood. In this work, we use answer logits at the end of each
reasoning step to estimate each step's causal importance to the final answer and intermediate guesses, shedding light on the answer formation process of several
reasoning model families. Across diverse tasks, we find that
reasoning typically crosses a commitment boundary, a sharp transition from transient intermediate guesses to a stable, high-confidence answer. This transition often happens in a single step, well before the model's
reasoning block ends, and is followed by epiphenomenal CoT steps that leave the final answer probability unaltered. Using
attention probes, we show that answer-formation stages can be linearly decoded from the activations of intermediate
reasoning steps with high accuracy, showing robust generalization to unseen
reasoning tasks. We leverage this property for early-exiting
reasoning blocks at the commitment boundary location, reducing the length of CoTs up to 55% with negligible impact on model performance.
Read the full abstract ▾
All authors Daniel Scalena, Sara Candussio, Luca Bortolussi, Elisabetta Fersini, Malvina Nissim, Gabriele Sarti
Artificial Intelligence
EpiBench: Verifiable Evaluation of AI Agents on Epigenomics Analysis
Harihara Muralidharan, Reema Baskar, Soo Hee Lee et al. (+2 more)
We introduce EpiBench, a verifiable
benchmark for short-horizon epigenomics analysis. EpiBench evaluates whether
agents can make well-defined analysis decisions from realistic workflow states and return deterministically gradable answers. The
benchmark includes 106 evaluations across CUT&Tag/CUT&RUN, ATAC-seq, ChIP-seq, and DNA methylation workflows. Across 5,088 valid trajectories from 16 model-harness pairs, no system passed a majority of attempts: GPT-5.5 / Pi led at 45.0% (143/318 attempts; 95% confidence interval (CI), 36.3–53.7), followed by GPT-5.5 / OpenAI Codex at 39.9% (127/318 attempts; 95% CI, 31.6–48.3). Claude Opus 4.8 Max / Pi and GPT-5.4 / Pi each passed 39.0% (124/318 attempts; 95% CI, 30.2–47.8 and 31.0–47.0, respectively). Performance varies across assay types, and many failed runs still contain parts of the correct answer.
Agents often found the right files and computed useful intermediate results, but failed when the task required deeper, assay-specific scientific judgment.
Read the full abstract ▾
All authors Harihara Muralidharan, Reema Baskar, Soo Hee Lee, Tim Proctor, Kenny Workman
Artificial Intelligence
Reward Modeling for Multi-Agent Orchestration
King Yeung Tsang, Zihao Zhao, Vishal Venkataramani et al. (+5 more)
Multi-
Agent Systems (MAS) built on Large
Language Models (LLMs) require effective orchestration to coordinate specialized
agents, yet training such orchestrators is hindered by limited supervision and high computational cost. We propose Orchestration Reward Modeling (OrchRM), a self-supervised framework for evaluating orchestration quality without human annotations. OrchRM leverages intermediate artifacts from multi-
agent executions to construct win-lose pairs for Bradley-Terry reward model training. Unlike existing MAS test-time scaling and orchestrator training frameworks that rely on costly sub-
agent rollouts, OrchRM operates directly at the orchestration level, enabling efficient and high-performing reward-guided orchestrator training and MAS test-time scaling. OrchRM improves training efficiency by up to 10x in
token usage while improving MAS test-time scaling performance by up to 8% in accuracy. These gains consistently transfer across multiple domains, including mathematical
reasoning, web-based question answering, and multi-hop
reasoning, demonstrating orchestration-level reward modeling as a scalable direction for robust multi-
agent orchestration. Code will be available at https://github.com/Wang-ML-Lab/OrchRM.
Read the full abstract ▾
All authors King Yeung Tsang, Zihao Zhao, Vishal Venkataramani, Haizhou Shi, Zixuan Ke, Semih Yavuz, Shafiq Joty, Hao Wang
Artificial Intelligence
Confidence Composition for Multiagent Language Model Systems
Ali Elahi, Michael J. Curry and Barbara Di Eugenio
Multiagent
language model systems, such as collaborative
reasoning and debate, produce multiple correlated candidate answers and confidence signals. However, these signals are usually calibrated only at the individual
agent level, and provide no principled confidence estimate for the system's final answer. We formulate this as a confidence composition problem where combining confidence across
agents and
reasoning stages while preserving both selective utility and probabilistic reliability. We study confidence-aware routing and log-odds pooling protocols that select among candidate answers and output a system-level confidence. Across five
benchmarks, 30 heterogeneous and homogeneous model pairs, and two confidence
estimators, our gated-fusion methods improve AUARC and reduce Brier score over single
agent, standard debate, and selective debate baselines, while retaining competitive weighted F1-score as a correctness metric. We further show that our log-odds fusion is overconfident due to correlated intermediate signals. We propose a shared dependence discount that substantially improves reliability while preserving predictions.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
Simplex-Constrained Sparse Bagging: Transitioning from Uniform Priors to Sparse Posteriors in Ensemble Learning
Meher Sai Preetam Madiraju and Meher Bhaskar Madiraju
We present Simplex-Constrained Sparse Bagging (SCSB), a mathematically rigorous framework for post-training compression and probability calibration of bootstrap-based bagging ensembles. Standard bagging ensembles (such as Random Forests, Bagged SVMs, and Bagged Neural Networks) assign uniform voting power to all constituent
estimators. However, this naive uniform prior ignores the varying local competence of base
estimators and contributes to model overconfidence. We formulate ensemble
pruning and calibration as a joint optimization problem over the probability simplex by minimizing the Out-Of-Bag (OOB) loss. To induce sparsity, we address the theoretical "L1-simplex paradox" - the mathematical reality that the L1 norm is constant on the simplex and fails to prune - by introducing a concave quadratic penalty. SCSB is model-agnostic and achieves up to 96% ensemble compression, yielding linear inference speedups and superior probability calibration (lowered Expected Calibration Error) while preserving or enhancing generalization accuracy.
Read the full abstract ▾