Artificial Intelligence
Improved Multiplayer Bandit Algorithm for Bernoulli Rewards
Khang Nguyen, Ricardo Parada and William Chang
We study the multiplayer multi-armed bandit problem with information asymmetry under Bernoulli rewards, for three information structures: asymmetry in actions, in rewards, and in both. Replacing the Hoeffding-style confidence intervals of prior work with Kullback–Leibler (KL) divergence-based bounds gives strictly tighter regret guarantees in each case. We propose mKL-UCB, mKL-UCB-Intervals and mKL-DSEE, and show that the improvement factor is at least two by Pinsker's inequality and far larger when reward means are near zero or one. For asymmetry in rewards we prove that two arms' KL intervals separate after a deterministic number of samples, and that $M$ independent players accelerate elimination further.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from stat.ME
When Unpaired Sets Support Shared-Corruption Calibration: Moment Geometry and Two-Sample Precision
Shuheng Cao, Zhenhao Zhang, Ruiqi Chen et al. (+5 more)
Collections of diverse observations often share one acquisition, processing,
geometric, or channel corruption, while only an unpaired clean reference set is available. For a prescribed low-dimensional correction shared across observations, the observed and clean reference sets support inference only through the response of fixed moments. We formulate this problem as two-sample moment calibration and report a rank-aware information state combining local rank, scaled moment sensitivity, source-separated covariance, and a moment compatibility residual. Full rank gives local moment identifiability, whereas kernel directions remain unresolved to first order. A unified linearization separates observed-set and reference-set uncertainty. Under covariance weighting, the weakest scaled singular value determines worst-direction asymptotic amplification. For an orientation-preserving planar-similarity correction shared across observations, ensemble centroids and a nonzero third-order complex moment yield closed-form global population identification of translation, rotation, and isotropic scale under matched-population and no-clipping assumptions. Controlled validation tests the predicted $N^{-1}$ and $\sigma_{\min}^{-2}$ laws, Gaussian efficiency, and interval coverage. Bounded applications report color corrected-output quality, channel magnitude-response calibration, and a separate paired
geometric de-beautification result. The framework therefore reports missing or weak information instead of treating every fitted correction as identified.
Read the full abstract ▾
All authors Shuheng Cao, Zhenhao Zhang, Ruiqi Chen, Renjie Cao, Siyu Zhang, Zhaoxiang Feng, Lingwei Dang, Haoyang Wu
Artificial Intelligence
RCShift: Certifying When Partial Linkage Suffices for Finite-Sample Decisions
Shuheng Cao, Ruiqi Chen, Zhenhao Zhang et al. (+5 more)
Systems with costly gold outcomes and cheaper auxiliary observations must decide how much record linkage to retain. Complete pairing retains every joint counter, while separate margins retain none. Neither endpoint is calibrated to a declared finite-sample decision. Universal reconstruction can retain cycle directions invisible to the likelihood-ratio family. Family-exact storage can exceed what the decision requires because certified residual loss may fit within finite-sample slack. We introduce RCShift, which certifies two routes to sufficiency under a declared observation contract. Its exact mode characterizes minimum-cost family-exact storage through LR-visible cycle directions. Its approximate mode bounds reverse Le Cam deficiency. Its integer mode certifies whether a chosen set preserves the full experiment's minimum integer record count at specified size and power. In a rank-two witness, one aligned counter preserves a four-record minimum. An equal-cost misaligned counter and the margins require eleven records, while universal reconstruction requires two counters. A local perturbation has positive reverse deficiency yet retains the four-record minimum. Proof-checked scheduling bounds instantiate the contract before gold computation and yield exact reconstruction on the admitted tree support. RCShift turns partial-linkage storage into decision-calibrated measurement design for the declared family, costs, target, and common strictly positive support.
Read the full abstract ▾
All authors Shuheng Cao, Ruiqi Chen, Zhenhao Zhang, Renjie Cao, Siyu Zhang, Lingwei Dang, Jiajun Zhang, Tingting Dan
Artificial Intelligence
TREND-10K: A Comprehensive Dataset for Next-Generation Video Quality Assessment Based on Preference-Driven Media
Ziheng Jia, Zicheng Zhang, Junqi Zhang et al. (+4 more)
The increasing prominence of short-
video platforms, coupled with the advanced commercialization of AI-generated content (AIGC)
videos, has led to a shift in the types of
video media trend consumed by users in their daily lives. Traditional user-generated content (UGC) is gradually being replaced by professional short dramas and AIGC entertainment. Consequently, VQA for contemporary media content has become increasingly important. This requires a unified evaluation framework that can handle diverse
video content and evolving media trends. In this context, we introduce TREND-10K, a next-generation comprehensive VQA
dataset consisting of the trend-driven part and the static part, containing $10,000$
videos across a wide spectrum of content types. The trend-driven part is based on the TREND-Search framework, which captures user preference profiles from trending lists on online platforms and formulates sampling strategies based on these profiles. The static part, on the other hand, is composed of supplementary samples selected from publicly available
datasets. To support unified evaluation for various
video types, we incorporate three evaluation dimensions: technical, aesthetic, and AIGC-trace. Experiments show that our
dataset ensures high annotation quality and exhibits remarkable generalization across multiple content categories. In conclusion, our work presents a robust framework for advancing VQA, addressing challenges caused by the temporal
evolution of user perceptual habits and preferences.
Read the full abstract ▾
All authors Ziheng Jia, Zicheng Zhang, Junqi Zhang, Jiaying Qian, Jiarui Wang, Yushuo Zheng, Xiongkuo Min
Artificial Intelligence · Cross-listed from cs.CL
Magnitude Profile Pruning: Calibration-Free Structured Attention Head Removal for Transformer Compression
Kasun Dewage, Marianna Pensky, Heranga K. Rathnasekara et al. (+1 more)
Structured
pruning of
attention heads provides a hardware-friendly way to compress
Transformer language models. However, existing methods for measuring head-level importance require calibration data, gradient computation, or Hessian estimation. These requirements add extra overhead and make the methods depend on the data. Our work presents Magnitude Profile (MP) scoring, a training-free criterion for head importance that identifies dispensable heads through statistical outlier detection on weight row norms. Heads whose projection weights fall within the population bulk are pruned, while heads exhibiting outlier norms, which carry disproportionate representational capacity, are preserved. Our work further gives MP-G, a variant that handles Grouped Query
Attention (GQA) by distributing shared key-value group scores across associated query heads. Across five models evaluated on WikiText-2 perplexity at 12.5%-50% head sparsity, MP-G achieves the best perplexity on OPT-6.7B at all sparsity levels (18.46 at 12.5%, 27.87 at 25%, 152.0 at 50%). MP-G also gives the best results on RoBERTa-large at 12.5% and 25% sparsity, with perplexity values of 7.27 and 10.28, outperforming calibration-dependent baselines including Wanda-Head, SparseGPT-Head, and Gradient-Head. It requires zero forward passes, calibration samples, or gradient computation. At 50% sparsity, head
pruning yields up to 16% parameter reduction with 50%
attention FLOP savings. Our results show that weight-only statistical scoring can match or outperform data-dependent methods for structured head
pruning, providing a practical, zero-cost criterion for
Transformer compression.
Read the full abstract ▾
All authors Kasun Dewage, Marianna Pensky, Heranga K. Rathnasekara, Suranadi De Silva
Artificial Intelligence
Toward User-Mediated Self-Repair in Ubiquitous Robots Through Goal-Oriented Agentic AI
Morten Roed Frederiksen
Ubiquitous robotic systems often lack traditional visual interfaces, necessitating resilient natural
language interaction for maintenance and repair tasks. This paper presents a goal oriented agentic AI architecture designed to enable non-expert users to perform technical repairs through situated dialogue. The framework utilizes a multi-layered approach that decouples high-level strategic planning from reactive conversational execution to transform unconstrained human instructions into a structured hierarchy of goals. We conducted a study involving twenty participants to evaluate the system's efficacy using a physical hardware testbed. The architecture achieved a 95% task completion rate, and participants reported positive self-efficacy following real-time guidance that adapted to conversational diversions and linguistic variations. A comparative analysis with an online baseline revealed that the transition to a physical environment significantly decreased perceived social presence (p=.0005), and trust and competence, (p=.037), while the agentic framework remained robust throughout the interaction. These findings indicate that goal oriented agentic AI can support the sustainability of body-worn technologies by empowering users to perform critical maintenance in ubiquitous contexts.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CL
Behaviorally Grounded User Profiles from the Wild for Personalized Alignment and Multi-Perspective Reasoning
Yuxuan Li, Victor Zhong and Ehsan Kamalloo
Persona-driven techniques increasingly adapt large
language models (LLMs) to diverse contexts. However, existing methods predominantly rely on rigid, synthetic personas that flatten individual variation, rely on stereotypes, and miss the nuanced signals driving actual human preferences. We introduce profile behavioral grounding, a framework for extracting open-ended, high-fidelity user profiles directly from authentic, anonymized social media posts. We evaluate these profiles across two paradigms: train-time personalization via supervised finetuning (SFT) and non-parametric test-time multi-perspective
reasoning. Across complex recommendation and open-ended query
benchmarks, behaviorally grounded profiles consistently improve base models and outperform synthetic profile baselines, driving stronger parametric
alignment and enabling richer, multifaceted
reasoning. Our findings establish open-ended, behavior-derived profiles as a highly diverse and effective foundation for the next generation of personalized
language systems. Our code base is available at https://github.com/ServiceNow/behavior-grounding.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CL
Terminal-Bench-LILT: Multilingual Agentic Coding Benchmark Grounded in Language, Region, and Culture
Yunsu Kim, Kaden Uhlig, Ashwin Purohit et al. (+10 more)
Most evaluations for coding
agents are conducted exclusively in English, which does not reflect real-world
multilingual deployment. We present Terminal-Bench-LILT, a suite of 300 authentic coding tasks in ten
languages: Arabic, Czech, German, Spanish, Hindi, Japanese, Korean, Serbian, Turkish, and Chinese. Each task targets issues specific to non-English software development that have no direct English equivalent, e.g., internationalization, encoding, text normalization, and cultural conventions. All tasks are authored by native-speaker programmers and validated through a multi-stage quality control pipeline. Evaluation of six frontier models reveals that even the strongest model reaches only 63.1% pass rate, with many tasks unsolved by any model. Performance varies substantially by
language and does not track general coding
benchmark rankings, highlighting that
multilingual coding competence is a distinct and underexplored capability axis. Sample tasks are available at https://github.com/lilt/terminal-bench-lilt
Read the full abstract ▾
All authors Yunsu Kim, Kaden Uhlig, Ashwin Purohit, Milind Agarwal, Patrick Simianer, Anil Arslan, Kiarash Mokhtari, Thomas Zenkel, Johannes Mosig, Gabriel Bretschner, Shamik Bose, Joern Wuebker, John DeNero
Artificial Intelligence · Cross-listed from cs.CL
PromptKWS: A Novel Prompt-Guided Open-Vocabulary Keyword Spotting Framework
Gaopeng Xu, Chengfei Li, Xianliang Wang et al. (+5 more)
In this paper, we present PromptKWS, a novel
Prompt-guided keyword spotting (KWS) framework to improve the accuracy of open vocabulary KWS systems. In specific terms, we introduce the
Prompt Phrases Prediction Network (PPN), an encoder-decoder architecture designed to effectively extract keyword
prompts embeddings. we employ the PPN encoder to encode the keyword
prompts and infuse the
prompt embedding into the
Prompt-guided KWS encoder by utilizing a
Prompt-acoustic Multi-head Cross-
attention (MHCA). Experiments show that PromptKWS improves the wakeup rate by over 10% compared to baseline system. Notably, another strength of PromptKWS is its ability to effectively leverage keyword
prompts for adapting to complex real-world environments involving noise and pronunciation variations. In comparison to purely acoustic models, which often struggle in such situations, PromptKWS demonstrates remarkable performance, with an average accuracy improvement of over 15% in test sets.
Read the full abstract ▾
All authors Gaopeng Xu, Chengfei Li, Xianliang Wang, Lin Zhu, Juan Wei, Wenpeng Li, Jianwei Niu, Jie Gao
Artificial Intelligence
Reward-Oracle MCTS for Formal Theorem Proving: Sample-Efficient Search and the Need for Kernel-Level Proof Auditing
Bodla Krishna Vamshi and Haizhao Yang
Formal
theorem proving with large
language models remains challenging due to the difficulty of navigating large proof search spaces efficiently. Existing tree search approaches either feed verbose compiler error messages directly into the generation context, increasing context usage during search, or employ non-standard evaluation protocols that prevent direct comparison with established baselines. We propose a three-role Monte Carlo Tree Search (MCTS) framework that treats the Lean 4 compiler purely as a reward oracle using compiler output as a scalar signal for UCB-guided tree updates without feeding error content into the generation context. Our framework decomposes proof search into three roles: a generator for proof attempts, a decomposer for subgoal decomposition, and a critic for subgoal quality evaluation. We evaluate across 4
benchmarks spanning competition mathematics and physics (MiniF2F, PutnamBench, LeanPhysBench, PhysLeandata) with three prover models at standard proof attempt budgets (PAB@16 to PAB@256). Our method achieves 87.1% on MiniF2F with Goedel-Prover-V2-8B at PAB@256 and solves 26/659 PutnamBench problems at PAB@32 surpassing base sampling 18/659 at same proof attempt budget. Through an exhaustive axiom-level audit of every compiled proof, we further identify reward hacking in search-based
theorem proving: DeepSeek-Prover-V2-7B produces proofs on PutnamBench that pass compilation and the standard sorry-
token scan while depending on sorryAx. The audit removes 4 and 8 such proofs from whole-proof sampling at PAB@32 and PAB@128, and 11 and 19 from MCTS. We do not attribute these counts to the search procedure; we report them to establish that kernel-level
auditing is necessary for compiler-verified evaluation.
Read the full abstract ▾
Artificial Intelligence
Semantic Graph Unification for Industrial Digital Threads: Bridging 11 Heterogeneous Manufacturing Systems Through Ontology-Driven Knowledge Graphs
Grama Chethan
Modern manufacturing enterprises operate heterogeneous systems – ERP, MES, PLM, SCADA, QMS, SCM – each with its own data model and API. The resulting silos prevent holistic analysis, delay root-cause investigation, and obstruct Industry 4.0 traceability. Point-to-point integration scales as O(n^2) and accumulates brittle dependencies. This paper presents an open framework for semantic graph unification of industrial digital threads. An ontology-driven RDF knowledge graph unifies data from 11 simulated sources across nine domains through a five-stage ETL pipeline with automated entity resolution spanning 97 owl:sameAs identity links. The ontology encompasses 78 RDFS classes, 108 object properties, and 243 data properties, drawing on ISA-95, OPC UA, eClass, the Asset Administration Shell, RAMI 4.0, and additional standards. An automated discovery engine applies nine strategy categories – cross-station correlation, alarm coverage, ECN impact, CUSUM/EWMA drift detection – to surface insights spanning system boundaries. The primary empirical result: blocking 24 cross-system tools reduces recall from 1.00 to 0.31 (F1 from 1.00 to 0.48), showing that 69% of discoverable signals require cross-system graph joins. Leave-one-out ablation confirms six of nine strategies contribute unique signals. Verification against a 65-signal manifest (16 positive, 49 null) yields F1 = 1.00 (95% Clopper-Pearson CI [0.79, 1.00]); as the manifest was author-constructed, this constitutes verification not independent validation. The graph is exposed to LLM
agents via 287 Model Context Protocol tools as a SPARQL-native semantic layer. Five industry templates (aerospace, CPG, pharma, medical devices, turbine blades) demonstrate schema stability across manufacturing verticals.
Read the full abstract ▾
Artificial Intelligence
The Abstention Protocol: RCA for Clos Fabrics
Madhava Gaikwad and Deepak Pandey
Root cause analysis (RCA) in large datacenter networks is challenging because telemetry is noisy, partial, and asynchronous. Score-based approaches degrade under these conditions, often yielding unstable or incorrect attributions. We present CoreSec, a production RCA system that replaces weighted fusion with a PAM-style abstention
algebra. Telemetry
agents are composed using control flags that yield deterministic decisions and explicit abstention when evidence is ambiguous. CoreSec combines this
algebra with
topology-aware configurations that capture failure surfaces across Clos fabrics and converge monotonically as evidence accumulates. Deployed at hyperscale, CoreSec provides stable and explainable RCA behavior across diverse environments without retuning. Our experience shows that structured composition with abstention forms a practical foundation for automated RCA in real-world cloud networks.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.AR
TokenPowerSandbox: Evidence-Gated CPU-First Screening for Energy-Aware LLM Serving
Chenxu Niu
Energy-aware LLM serving requires comparing configurations under realistic request shapes, yet exhaustive target-GPU profiling is costly and a cheap predictor can be dangerously confident outside its measured scope. We present TokenPowerSandbox, an evidence-gated workflow that combines an interpretable CPU-resident projector, short target-GPU probes, full-workload verification, and tamper-evident freeze-before-measurement provenance. On one NVIDIA H100 80GB serving Qwen2.5-7B-Instruct with vLLM, three anchor repeats and six development workloads calibrate workload transfer. The same frozen model is evaluated on a blind holdout and a separately predeclared no-refit confirmation totaling 51 post-freeze runs. Energy MAPE is 6.23% and 7.35%, with Spearman rank correlations of 0.976 and 0.933. However, a predeclared TTFT gate passes at concurrency four (9.27% MAPE) and triggers abstention below four (64.80%), showing why energy accuracy cannot certify latency.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CL
Which Question Is Your Attention Metric Answering? Attention Rows as Compositional Data
Marios Papamichalis and Regina Ruane
Each row of a
transformer's
attention matrix is a probability distribution over
tokens, and in trained models most of that probability lands on a single sink
token, usually the first. Standard tools for comparing
attention rows (cosine similarity, Jensen–Shannon divergence, Shannon
entropy) therefore hinge on a choice papers rarely report: keep the sink, or drop it and renormalize. This choice can reverse conclusions. On ten pretrained models from five families, 17–47% of verdicts about which of two heads is more similar flip with the convention, and the most prominent structure in a standard BERT head-clustering pipeline is an artifact of it. The reason is that one-number summaries mix two questions: how much
attention the sink takes, and how the rest is divided among the content
tokens. Treating rows as compositional data separates them exactly: the Aitchison distance splits orthogonally into a sink term and a content term,
entropy splits by an exact identity, and the content distance is characterized by invariances the
transformer itself possesses. The separation matters in practice: most measured
entropy collapse during training is the sink growing, not
attention sharpening (30% of the drop at 70M parameters, 95% at 1B, 79% at 1.4B), and
pruning heads with the wrong channel can inflate perplexity more than a hundredfold. We map where each convention is safe, test a frozen out-of-sample predictor (one confirmation, one abstention, one failure), and release code regenerating every number.
Read the full abstract ▾
Artificial Intelligence
Beyond Pass@k: Measuring Reliability and Security of Agentic Code Generation
Jiajun Jiang, Sharon Zheng, Natan Vidra et al. (+1 more)
AI coding
agent benchmarks rank
agents with the Chen et al. (2021) pass@k
estimator, but current implementations misapply it: they set n to the number of unit tests in a single submission rather than the number of independent rollout attempts, conflating test-suite size with attempt independence. We diagnose this operationalization error, prove it by counterexample, and propose reliability@k, the same
estimator applied correctly, with n = independent rollouts and c = fully-passing rollouts per (task,
agent) pair. In a synthetic multi-rollout
benchmark, the misapplied metric inflates reported scores by 0.85-0.97 in absolute terms (0.96-0.98 reported vs. 0.00-0.12 corrected), and a cheap single-rollout proxy fails to substitute for repeated runs (Spearman $\rho = 0.417$). Motivated by evidence that functional correctness does not imply security safety, we additionally propose security-adjusted reliability@k, which counts only rollouts that are both functionally correct and free of high-severity insecure patterns. In an initial live-API test with three
agents, the adjustment did not change any ranking under our current scanner and threshold, so we present it as a proposed complementary lens whose decisive evaluation requires better-powered future runs. Finally, a preliminary 5-task SWE-bench Verified pilot observes the same core concern in a real repository setting: macro-averaged hidden-test pass rate was 0.80 while strict task resolution was 0.20.
Read the full abstract ▾
All authors Jiajun Jiang, Sharon Zheng, Natan Vidra, Spurthi Setty
Artificial Intelligence · Cross-listed from cs.CV
Path2ST: Hierarchical Cell-Tissue Grounded Cross-Modal Translation for Spatial Transcriptomics
Ruochen Liu and Wei Lou
Predicting spatial gene expression from hematoxylin and eosin (H&E)-stained images offers a cost-effective alternative to spatial transcriptomics (ST). However, existing methods treat H&E images as generic visual inputs and ignore their intrinsic biological hierarchy, where spatially organized
cell types collectively form functional tissue microenvironments that govern local gene expression programs. To bridge this gap, we formulate H&E-to-ST prediction as a cross-modal semantic translation task and propose Path2ST, a hierarchically grounded autoregressive framework featuring three key components: (i) a Hierarchical
Cell-Tissue Conditioning mechanism that fuses explicit and implicit cellular features with tissue-level semantic representations to construct hierarchical conditioning signals; (ii) a Scale-Adaptive Autoregressive Generation process over a hierarchical semantic vocabulary, enabling coarse-to-fine, biologically consistent expression synthesis; and (iii) SpectraLoss, a full-spectrum objective that jointly enforces ordinal fidelity, models transcriptional bursts, and aligns semantic structures with
cell types. Extensive experiments on three
datasets demonstrate state-of-the-art performance, validating that Path2ST generates highly accurate and spatially coherent transcriptomic profiles. The related code is released at https://github.com/RuochenLiu23/Path2ST.
Read the full abstract ▾
Artificial Intelligence
Semantic Uncertainty-Guided Orchestration in Hierarchical Multi-Agent Systems
John Knowlton, Aritra Guha and Risto Miikkulainen
As large
language model (LLM)-based multi-
agent systems become increasingly capable, coordinating
agents under uncertainty becomes a fundamental challenge. Existing orchestration strategies typically rely on fixed interaction patterns and often lack mechanisms for assessing the reliability of intermediate
reasoning steps, allowing errors and
hallucinations to propagate through the system. This paper introduces a semantic-uncertainty-guided orchestration approach, HASSUM as a general framework for uncertainty-aware coordination in multi-
agent systems. The method estimates uncertainty using semantic
entropy and semantic density, which measure trust at the level of answer semantics rather than output probabilities. These signals enable adaptive orchestration decisions, including output verification, selective reprompting, additional deliberation, and confidence-aware response selection. Because the approach operates independently of any particular
agent architecture, it can be integrated into a broad range of hierarchical and collaborative multi-
agent systems. The evaluations demonstrate an implementation within a hierarchical
agent framework and evaluate it on StrategyQA, JailbreakBench, and TruthfulQA
benchmarks. Across tasks that require complex
reasoning and are prone to ambiguity or
hallucinations, uncertainty-guided orchestration yields more reliable outcomes than uncertainty-unaware coordination. Semantic
entropy and semantic density in tandem outperformed either metric alone. Ablations testing different thresholds and model sizes demonstrated that both influence the effectiveness of semantic metrics. The results suggest that semantic uncertainty is a practical and general-purpose signal for improving robustness and trustworthiness in agentic AI systems.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CV
Equilibrium Forcing: Adaptive Video Generation Without Noise Conditioning
Hansen Jin Lillemark, Alex Rojas, Zachary Novack et al. (+5 more)
Standard autoregressive
video generation
algorithms based on
Diffusion and Flow Matching rely on rigid training objectives and static sampling schedules, limiting inference procedures from adapting to the data. We introduce
Equilibrium Forcing (EqF), a simplified framework for
video denoising generative models without noise level conditioning. EqF pioneers modular training- and inference-time designs for noise-unconditional generation that decouple learning the denoising field from sampling. This flexibility allows for inference-time
algorithms that operate in a closed loop by adapting to feedback from the sample, improving
video quality and consistency on challenging autoregressive
video generation
benchmarks. Extensive analysis elucidates exactly how removing the noise level conditioning enables EqF's data-dependent inference properties to surpass the performance of standard noise level-conditional denoising
video methods.
Read the full abstract ▾
All authors Hansen Jin Lillemark, Alex Rojas, Zachary Novack, Runqian Wang, Yilun Du, Yian Ma, Taylor Berg-Kirkpatrick, Rose Yu
Artificial Intelligence
Evaluating Agentic Learning Harness Capabilities Without Labels via the Scaling Hypothesis
Aryan Luthra, Kshitij Jain, Siddharth Arya et al. (+2 more)
Agentic "Continual Learning Harnesses", systems that pair an LLM with
retrieval or
memory to improve from feedback without retraining, have shown growing value in cybersecurity. But their value is conventionally measured by gains against labeled
benchmarks, an approach that often fails in operational security settings.
Benchmark labels are scarce, stale, and unrepresentative, so a practitioner often cannot tell whether a given harness helps at all or which of two is better for their task. Traditional LLM-as-a-judge offers little signal because it is no stronger than the
agent it evaluates, and
distillation is unreliable on scarce, sporadic, and biased labels. We propose a framework for evaluating learning harnesses end-to-end without a labeled
benchmark, grounded in the scaling hypothesis. A stronger teacher model provides sparsely sampled corrections to a smaller student with a continual learning harness. We score a harness by how much its student converges toward the teacher over time. Across security tasks, model families, and harness designs, we show that improvement relative to the teacher correlates with improvement relative to a held-out gold standard, validating teacher-relative lift as a proxy for true harness uplift when labels are absent. We further show that LLM-as-a-judge between similarly powered models yields no usable signal. These results suggest that a teacher-sized model can be improved through the same harness when humans provide the same kind of sparse, high-precision corrections.
Read the full abstract ▾
All authors Aryan Luthra, Kshitij Jain, Siddharth Arya, Bobby Filar, Anna Bertiger
Artificial Intelligence
No Universal Signal Predicts Sample-Level LLM Regression under Version Updates
Jia Sheng and Yiwei Lu
Frontier LLMs are updated frequently and typically outperform their predecessors in aggregate. But aggregate gains say little about individual samples: an update can still cause sample-level regression, where a response correct under the old model becomes incorrect under the new one. This paper studies how to predict such regressions from signals available at inference time. We compare single-model signals (confidence, logit margin,
attention entropy) against cross-version signals (output KL divergence, likelihood drift,
token-level KL, representation drift) under a unified added-value test that isolates each signal's gain over a confidence baseline. Across six
benchmarks in three task families (multiple-choice question answering, or MCQ; math
reasoning; code generation) and six model update pairs, we find that (1) signal effectiveness is task-dependent: confidence is strongest on MCQ and simpler math, while likelihood/KL signals give the most frequent gains on harder math and code; (2) no signal is universally best across model updates either; and (3) some cross-version signals stay informative even when confidence fails, including without labels, which supports a proof-of-concept selective fallback that routes high-risk samples back to the old model. Practitioners can use these task-level patterns to choose which regression signal to trust for a given update. Code is available at https://github.com/jiashengsally/llm-regression-signals.
Read the full abstract ▾
Artificial Intelligence
MobileMem: Learning from a Year of Mobile Experiences
Xinle Deng, Yida Xue, Xiangyuan Ru et al. (+16 more)
The next generation of AI
agents is increasingly moving beyond systems that answer isolated questions toward persistent personal assistants that can understand, remember, and continuously learn from users' experiences. Such assistants require long-term
memory to accumulate and leverage user-specific experiences over time, yet existing
benchmarks remain inadequate for realistic mobile settings, where experiences are heterogeneous,
multimodal, evolving, and deeply personal. We introduce MobileMem, a
benchmark and framework for studying on-device long-term
memory, grounded in a year-scale collection of mobile experiences. MobileMem employs a knowledge-grounded synthesis pipeline to construct coherent and temporally consistent long-horizon trajectories from user-app sessions. It provides complementary text and
multimodal settings covering multi-hop and temporal
reasoning, knowledge updating, and implicit preference inference. Specifically, MobileMem enables
agents to remember the past, understand the present, and adapt to the future. By modeling experiences rather than isolated facts, MobileMem moves
memory beyond information
retrieval toward experiential intelligence for continuous personal learning.
Read the full abstract ▾
All authors Xinle Deng, Yida Xue, Xiangyuan Ru, Yijun Chen, Buqiang Xu, Mingjun Mao, Xinjie Liu, Haoming Xu, Shuofei Qiao, Mengru Wang, Chen Jiang, Yuchen Eleanor Jiang, Lizhong Wang, Jason Wang, Li Zeng, Haofen Wang, Guilin Qi, Huajun Chen, Ningyu Zhang
Artificial Intelligence
Active Perception for Embodied Disambiguation
Yiwei Liu and Luwei Yang
Natural
language provides
robots with a flexible task interface, but target ambiguity in
embodied environments arises not only from user intent; it can also result from missing taskrelevant physical evidence in the current observation. Existing interactive disambiguation methods primarily obtain additional information by asking the user, whereas occlusion, restricted viewpoints, unreadable text, and unobserved targets require the
robot to actively change its observation. We propose an active-perception framework for
embodied target disambiguation that uses active observation as the backbone for information acquisition and uses a
vision-
language model to decide, on the basis of accumulated visual evidence and interaction information, whether to continue observing, request clarification, or complete target selection. Active observation can both directly recover missing discriminative evidence and reveal object names, labels, and semantic attributes, thereby improving user clarification when it remains necessary. Real-
robot experiments show that the framework combines physical information acquisition and userintent clarification within a unified
embodied disambiguation process.
Read the full abstract ▾
Artificial Intelligence
Cross-Disciplinary Taxonomy and Modeling of Misunderstanding Generation, Amplification, and Detection, from Pragmatics to AI Agents
Babak Abbaschian
Detection of misunderstanding is an urgent problem to solve because communication has moved away from real-time, in-person interaction and is increasingly handled by AI-mediated channels. This shift cuts communicators off from the resources repair depends on faster than new means of detection are being built. In this paper we analyse misunderstanding as a layered process in which a divergence is generated, may then be amplified, and is either detected and repaired or left to persist unnoticed. Consolidating accounts from nine fields of research that do not ordinarily cite one another, we identify eleven exact failure modes and show that each operates at a specific point in a communicative process rather than anywhere within it. Those points give eight analytical layers, derived from the literature rather than adopted from an existing model. Eight of the mechanisms primarily generate a divergence, two primarily amplify one already present, and one governs whether a divergence is detected and repaired. We model the eight layers formally, extending information and communication theory from the transmission of signals to the reconstruction of meaning, and we supply a source-by-source evidence
matrix that makes every rating auditable, a coding manual, and nine analysed dialogue cases. No prior classification of misunderstanding both locates mechanisms at points in the process and types them by function.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.SE
Do Influence Tactics Matter? Investigating Prompt Framing Effects in LLM Code Generation
Alex Deaconu, Anubhav Gupta, Manaal Basha et al. (+2 more)
Large
Language Models (LLMs) are increasingly integrated into software engineering workflows, helping developers write, debug, test, and maintain code. While
prompt wording and structure are known to influence model performance, the impact of psychologically inspired
prompt framings remains unexplored. This study investigates whether different psychology-based communication strategies that humans use to persuade or motivate others can lead to more effective
prompt framing, which may, in turn, affect LLM behaviour in coding tasks. Drawing on Yukl & Falbe's well-known taxonomy, we operationalized eight influence tactics (like rational persuasion, ingratiation, and exchange) into reproducible
prompt templates. These
prompt templates were evaluated across five leading open-weight LLMs using two widely adopted
benchmarks: LiveCodeBench and SWE-bench Verified. We assessed the resulting code output on four key software quality dimensions: functional correctness, quality, maintainability, and security. Our results show that certain influence-induced
prompt framings, particularly those emphasizing urgency, were associated with reduced correctness and security. This work presents the first large-scale empirical study of influence-induced
prompt framing in software engineering tasks, offering insights into how linguistic cues may shape LLM outputs. We conclude with practical insights for designing transparent and interpretable human-AI interactions in code generation.
Read the full abstract ▾
All authors Alex Deaconu, Anubhav Gupta, Manaal Basha, Nicholas Haydu, Gema Rodríguez-Pérez
Artificial Intelligence · Cross-listed from cs.LG
Let it Cook: Learning to Wait in Sequential Decision Making
Christopher Watson, Arjun Krishna, Dinesh Jayaraman et al. (+1 more)
In sequential decision making, an
agent typically observes its environment and acts at every timestep. However, such active participation may not always be necessary; tasks such as brewing coffee include periods that are served equally well by letting the environment evolve without constant monitoring and control. During such periods, the
agent could simply wait to conserve its resources, or redirect its
attention to another task. We capitalize on these opportunities by training a "waiting policy" that decides where and how long to wait. This involves forgoing sensing to commit to a wait action, representing a deliberate pause for a set number of timesteps. We formalize "learning to wait" as minimizing the frequency of sensing and decision making without sacrificing task performance (e.g., the total amount of time to complete a task). To train a waiting policy, we propose an approach that employs reinforcement learning with lexicographically ordered objectives. In experiments across 4 discrete-state household tasks and 3 continuous-state environments, we show that our approach successfully learns waiting behaviors, and can adapt pre-trained policies to wait where appropriate. While different tasks permit different amounts of waiting without sacrificing task performance, our approach consistently finds solutions with significant waiting, sometimes waiting for over 50 percent of the task duration.
Read the full abstract ▾
All authors Christopher Watson, Arjun Krishna, Dinesh Jayaraman, Rajeev Alur
Artificial Intelligence · Cross-listed from q-bio.NC
Conflict and Congruency Effects in Large Language Models: In-Weight and In-Context Competition in a Verbal Conflict Task
Xiaoyang Hu, Mike Angstadt, Shane Storks et al. (+5 more)
Congruency effects, observed in conflict tasks such as Stroop and flanker tasks, have been investigated for nearly a century in psychology and neuroscience, but their mechanistic basis is not fully understood. We introduce a verbal-only LLM conflict task in which a
prompt stem elicits a default same-color completion and an explicit rule either agrees with (congruent condition) or conflicts with (incongruent condition) the completion. Gemma-2-2B and six Pythia models ranging from 410M to 12B parameters showed strong default same-color tendencies, and six of seven models showed strong congruency effects. Using causal attribution analysis,
attention analysis, and
attention ablations, we identified distinct processing pathways in these LLMs: a pathway involving short-range
attention to a superficial color cue that is preferentially activated in the congruent condition, and a pathway involving long-range
attention to the rule prefix that is preferentially activated in the incongruent condition. Fine-tuning that strengthened the default same-color tendency had divergent effects on task conditions, reducing incongruent performance while increasing congruent performance. In contrast, increasing rule set size selectively impaired incongruent performance. These converging findings support an account in which congruency effects in this task arise from competition between an in-weight default mapping and an in-context rule-based mapping. More broadly, our findings illustrate how LLMs can serve as model systems for mechanistic analysis of competition between default and rule-governed response tendencies within a single learned network.
Read the full abstract ▾
All authors Xiaoyang Hu, Mike Angstadt, Shane Storks, Zan Huang, Aman Taxali, Alex Weigard, Richard L. Lewis, Chandra Sripada
Artificial Intelligence · Cross-listed from cs.GT
Strengthening Full Justified Representation: Efficient Verification and Computation
Nicholas Teh
Full justified representation (FJR) is among the strongest known satisfiable proportionality axioms for approval-based committee elections. Recent work has shown that an FJR committee can be found in
polynomial time, but verifying whether a given committee satisfies FJR remains coNP-complete. We introduce FJR+, a strict strengthening of FJR and EJR+ that can be verified and satisfied in
polynomial time. We then analyze the Residual-Budget Greedy (RBG)
algorithm and prove that it selects a partial committee such that every size-$k$ completion satisfies FJR+. This freedom allows us to use sequential Phragmén to obtain a priceable completion. The resulting rule always satisfies FJR+ and the sub-core, and it is priceable whenever at least $k$ candidates receive an approval. We also obtain a Droop-quota version of FJR+. Finally, we extend FJR+ to approval-based participatory budgeting with arbitrary project costs. A project-specific version of RBG computes this property in
polynomial time and can be continued to a priceable outcome satisfying a cost-based version of the sub-core.
Read the full abstract ▾