Artificial Intelligence · Cross-listed from cs.RO
Who to Trust in Multi-Robot Systems: Joint Robot and Object Hypothesis Trust Inference via Neural-Symbolic Cross-Validation
Ruiyang Wang, Hao-Lun Hsu, Haocheng Meng et al. (+2 more)
Multi-
robot systems (MRS) offer significant advantages over single-
robot platforms by enabling information sharing and coordination for different tasks in complex environments. However, reliable cooperation critically depends on inferring trust over both information sources and their reported hypotheses, particularly under partial observability and potential
adversarial interference. Existing approaches frequently depend on pre-established trust relationships or side-channel information, which may not be available or reliable in realistic MRS deployments. This work introduces NeST-Bayes, a neural-symbolic framework that jointly infers
robot and hypothesis trust using only inter-
robot observation consistency, without requiring preassigned
robot trust, reputation information, or side-channel identity signals. NeST-Bayes infers instantaneous trust evidence scores for
robots and object hypotheses by
reasoning over ego-centric relational graphs constructed from cross-
robot observation consistency. These evidence scores are accumulated over time through
Bayesian updates of Beta-distributed trust beliefs to maintain temporally consistent trust estimates. Extensive evaluations show that NeST-Bayes achieves the highest mean object precision and
robot recall across all
benchmark settings, and the highest
robot precision and object recall in most settings, compared to trust-based methods. Under aggressive optimized attacks, it achieves 98.8%
robot precision with 93.8% recall and 98.3% object precision with 95.4% recall. In an occlusion-rich Webots warehouse, it maintains 92.5%
robot precision with 92.0% recall and 97.3% object precision with 85.0% recall. These results demonstrate joint improvements in the reliability and coverage of retained information.
Read the full abstract ▾
All authors Ruiyang Wang, Hao-Lun Hsu, Haocheng Meng, Songqiao Xie, Miroslav Pajic
Artificial Intelligence
Energy Ring: Stabilizing Feature Energy for Reliable Test-Time Discovery
Tianle Liu, Fan Lyu, Tingliang Feng et al. (+6 more)
Test-Time Discovery (TTD) aims to recognize known classes and detect novel ones while the model is already deployed, operating directly on a continual unlabeled test-time stream. Existing TTD methods are overly conservative, making known and unknown features collapse together and difficult to separate during test-time adaptation. In this paper, we revisit TTD from an energy perspective and find that the natural energy gap between known and unknown classes is unstable and easily collapses. We therefore propose Energy Ring to stabilize this gap by anchoring the two types of classes on separate energy rings. A lightweight adapter enlarges the norm-based gap while preserving known-class directions, and a perturbation consistency module forms compact clusters for unknown classes. During testing, energy-guided confidence weighting improves prediction, and a stability-based rectification mechanism corrects unreliable updates. Experimental results show that Energy Ring significantly improves TTD performance against existing TTD methods.
Read the full abstract ▾
All authors Tianle Liu, Fan Lyu, Tingliang Feng, Chenggong Ni, Tianqi Wang, Zhang Zhang, Fuyuan Hu, Rui Yao, Tao Zhou
Artificial Intelligence
ScienceBuddy-PV: A Verifier-Governed Recursive Improvement Protocol for Panvascular Intervention Research
Lingsen You, Yujun Guo, Xinyu Zhong et al. (+4 more)
ScienceBuddy-PV adapts the ScienceBuddy recursive-in-recursive design to panvascular intervention research through a structured evidence ledger, device-vessel ontology, abstention contract and retained failure bank. Recursive harness optimization and outer-loop model learning, including reinforcement learning, are specified but not executed. We audit a fixed component, rather than the complete
agent system, using separate ARCADE syntax and stenosis tasks (1,000 training, 200 validation and 300 test images per task). A shared task-masked convolutional model was trained once per representation setting; its parameters and the heuristic support gate were then held fixed, with the stenosis threshold selected on validation data. On the locked test sets, syntax Dice was 0.475 (95% CI 0.459-0.490) and stenosis Dice was 0.231 (95% CI 0.214-0.249). Stenosis pixel AUROC was 0.957 and AUPRC was 0.180 at prevalence 0.00975. The gate accepted 76.7% of stenosis-frame outputs versus 100% for a score-only comparator. In a constructed mask audit, it accepted 44.3% of reference candidates and rejected 55.7%, exposing the cost of abstention. Acceptance does not establish correctness or clinical safety. This public-data component audit makes support and abstention measurable and provides a reproducible protocol for future collaborative
agent studies. It does not demonstrate recursive improvement, patient-level validity or clinical device recommendations.
Read the full abstract ▾
All authors Lingsen You, Yujun Guo, Xinyu Zhong, Zijie Feng, Wentong Wang, Li Shen, Junbo Ge
Artificial Intelligence · Cross-listed from cs.CV
Compositional Concepts Erasing: Disentangling Object-Style via Domain Manifold Unlearning
Yang Xiao, Kaiyuan Deng, Gen Li et al. (+4 more)
Diffusion models pose significant risks regarding the generation of illicit content and copyright violations. This has catalyzed the development of machine unlearning to erase harmful concepts or copyrighted styles from a model's generative repertoire without compromising its general performance. However, previous unlearning approaches on
diffusion models have largely focused on either object-level or style-level removal and failed to address the challenge of object-style combination unlearning. At the same time, multiple unlearning requests often lead to catastrophic forgetting. To address these challenges, we propose Domain
Manifold Unlearning (DMU), which uses an EMA-updated feature proxy extracted from intermediate
diffusion features to represent the retained target-object direction. As a lightweight estimating representation of an object, DMU enables selective unlearning of domain-specific components in a specific object while preserving other semantic representations, thereby significantly enhancing robustness in multiple combination unlearning scenarios. Extensive experiments show that DMU significantly mitigates catastrophic forgetting, outperforming state-of-the-art methods of batch unlearning and sequential unlearning.
Read the full abstract ▾
All authors Yang Xiao, Kaiyuan Deng, Gen Li, Zinan Ling, Zhiwen Fan, Xiaolong Ma, Bo Hui
Artificial Intelligence
MontageFlow: Physically Valid High Fidelity EEG Generation
Sudarshan Regmi, Nabaraj Subedi, Arvind Pillai et al. (+6 more)
Generative modeling offers a promising solution to the scarcity of labeled clinical electroencephalography (
EEG) data under strict
privacy constraints. Although recent generative models report substantial improvements in generation fidelity, we show that these gains conceal a categorical failure.
EEG is not an arbitrary multichannel signal: each channel records the voltage difference between two scalp electrodes. So, real recordings satisfy exact linear identities among their channels. Current generators invariably violate these identities, yet this has gone unnoticed. We therefore propose MontageFlow, a flow-matching model that enforces these identities by restricting its output to signals that satisfy them. We find that ensuring validity improves fidelity too. Furthermore, to retain the
EEG characteristics that oversmoothing washes out, we also condition the model on a physiological state vector encoded from the whole
EEG window. For generation, this vector is drawn from a prior fitted in latent space. Across 3 large-scale
benchmarks, our model improves TS-FID by 1.6-3.6$\times$ over the closest rival while ensuring validity.
Read the full abstract ▾
All authors Sudarshan Regmi, Nabaraj Subedi, Arvind Pillai, Yu Yvonne Wu, Yuliang Chen, Michael V. Heinz, Lisa Marsch, Nicholas C. Jacobson, Andrew Campbell
Artificial Intelligence · Cross-listed from cs.LG
Rethinking Activation Functions: Reducing Bias Can Improve Learning
Zirui Peng, Heyi Zheng and Ziming Liu
Many widely used activation functions, including ReLU, GELU, Swish, and Sigmoid, produce predominantly positive outputs, resulting in a nonzero post-activation mean that we term activation bias. Despite its ubiquity, the effect of activation bias on learning dynamics remains poorly understood. Using controlled models, we show that activation bias accelerates initial loss reduction but is accompanied by overshooting, non-smooth trajectories, and a steeper loss landscape. This early advantage also coincides with the model's use of task-relevant information encoded in the activation bias component. In contrast, reducing activation bias slows initial learning but ultimately yields a lower final loss. Experiments on larger
vision and
language models exhibit a similar pattern: smaller activation bias is associated with better late-stage performance. Thus, this seemingly minor property can substantially redirect learning and warrants explicit consideration in future activation function design.
Read the full abstract ▾
Artificial Intelligence
Reels-RSI: Program-Level Self-Improvement for Code-Rendered Explainer Videos
Wei Lin
Explainer
videos – a worked geometry problem, why the sky is blue, how a dynasty fell – are usually one more target for pixel
video generation. We argue they are a different kind of artifact: a program (script, storyboard and per-frame animation code) that a deterministic renderer turns into frames. A program can be read, diffed, tested and audited; pixels can only be watched. Reels-RSI is an open-source reel director for any
agent harness; its self-improvement happens at the program level rather than in weights or
prompts. Every revision pass is logged with a code snapshot; a retrospective pairs each check finding with the diff that fixed it and proposes lessons; a lesson that a static check can catch is compiled into a rule with a must-fail and a must-pass example, so the
memory is unit-tested; and a rule learned today runs over everything made before it. A bounded,
benchmark-gated step lets an
agent rewrite the director's playbook, kept only when blind pairwise judgments on held-out topics prefer it; nothing enters
memory without a human decision. Across 111 reels – a field study, two playbook evolutions, a 2x2
memory ablation, a weaker director, a 48-reel learning curve and a judge-reliability study – we find that the rule learned in the field caught four latent defects in frames authored before it existed and none after; that rules are an audit for a strong director (eight unguarded reels, no violation) and a guard for a weaker one (thirty violations unguarded versus five); that lessons shorten revision loops but do not compound or move the judge; and that the first rewrite's 11–1 verdict was a position bias in our vote design (judges pick the second-shown reel 62–81% of the time), masking scripts that traded thirteen facts for craft. We document four failure modes of the preference signal, which check caught each, and the fixes. Code, rules, lessons and data are public.
Read the full abstract ▾
Artificial Intelligence
HiPPT: Physiology-Grounded Wearable PPG Representation Learning via Multi-Scope Text Alignment
Jiaqi Wang, Wei Chen, Xiaoyu Li et al. (+7 more)
Reusable wearable PPG representations should capture local pulse morphology and broader physiological regularities. Existing pretraining objectives, however, usually apply supervision at the recording or pair level, overlooking that different concepts become reliable over different scopes of waveform evidence. We propose HiPPT, a physiology-grounded framework that matches each supervision source to its appropriate evidence scope. We construct a PPG-Text corpus of over 4 million wrist-worn recordings from 202K users, pairing recordings with waveform-derived morphology and metadata-derived context descriptions. HiPPT combines record-level morphology
alignment, which grounds individual recordings in pulse characteristics, with cross-recording context
alignment, which aggregates recordings sharing the same metadata tuple to capture context-consistent regularities. Waveform-guided text modeling further recovers masked contextual concepts from PPG patch representations, providing fine-grained waveform-text supervision. Across six downstream tasks spanning blood pressure, vascular stiffness, physiological attributes, and cross-
dataset heart-rate estimation, HiPPT achieves the best result on the primary linear-probing metrics. The aligned space also supports zero-shot profiling of age group, sex, and BMI category, individually and jointly, with performance improving as more recordings are aggregated. These results show that evidence-scope matching yields transferable and semantically accessible wearable PPG representations. Code and pretrained weights will be released.
Read the full abstract ▾
All authors Jiaqi Wang, Wei Chen, Xiaoyu Li, Lexian Xia, ShuaiLong Tang, Chenran Zhang, Donglin Xie, Guangpu Zhu, Yelei Li, Yi Zhou
Artificial Intelligence
AlignPoison: Conditional Targeted Poisoning of Generative Recommenders via Decoding-Aligned Prefixes
Zhuoxuan Zhang, Jinhao Duan, Xinyu Zhao et al. (+7 more)
Generative recommendation models based on autoregressive decoding have been adopted as alternatives to classical ranking-based systems. While their modeling flexibility has been widely studied, their resilience to data poisoning remains largely unexplored. In this work, we ask: how susceptible are generative recommenders to conditional targeted poisoning? Under a realistic setting where an adversary can inject limited malicious interactions and query a recommendation API as a black box, our empirical study shows that generative recommenders are vulnerable: injecting fewer than 1% synthetic interactions significantly increases the likelihood of promoting malicious targets under the trigger. To characterize this vulnerability, we analyze the synthesis of malicious interactions from two perspectives: (i) decoding
alignment and (ii) prefix popularity. We observe that short, context-aligned prefixes that are both common under the model's induced transition dynamics and strongly linked to a frequent trigger consistently amplify targeted promotion. Motivated by this analysis, we develop AlignPoison, a black-box framework that discovers and samples context-aligned prefixes from API outputs. We conduct experiments on
benchmarks in unimodal and
multimodal generative recommendation settings. Results show that these prefixes outperform competitive baselines, improving ASR lift by up to 16.3 percentage points and highlighting a decoding-dependent, context-sensitive poisoning vulnerability in generative recommendation models.
Read the full abstract ▾
All authors Zhuoxuan Zhang, Jinhao Duan, Xinyu Zhao, Jinduo Guo, Shuai Shao, Rui Li, Luke Simon, Zhun Deng, Tianlong Chen, Bo Sun
Artificial Intelligence · Cross-listed from cs.CL
AphaVoice: Text-to-Speech Model for Aphasia Patient Simulation
Al Varrel Putra Kusuma, Fernando Bello and Joshua Brown
While
language therapy is the primary treatment for post-stroke aphasia, limited clinical resources often restrict patient access to proper care. Addressing this gap requires reliable training simulations for medical professionals. However, current AI efforts focus almost exclusively on
speech processing, leaving
speech generation largely unexplored. This study introduces AphaVoice, a novel text-to-
speech (TTS) framework explicitly engineered to synthesize highly realistic aphasic
speech. Evaluated against state-of-the-art TTS systems, AphaVoice demonstrates a superior ability to capture the clinical feature of dysfluency, outperforming existing models in temporal metrics while consistently maintaining speaker identity. By leveraging latent space shifting, the model also supports multi-speaker generation without sacrificing phonetic realism. Ultimately, AphaVoice provides a scalable, high-fidelity tool for medical training and clinical simulation related to aphasia.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.SE
What Counts as the Same Task? Executable Contracts for Robustness Evaluation of Tool-Using Agents
Shaikh Akib Shahriyar, MD. Akil Raihan Iftee, Armon Barton et al. (+1 more)
Robustness tests for tool-using
agents often change the
agent's environment, for example by hiding a hostile note in an invoice, and then judge the
agent by what it does. The result tells us about the original task only if the changed test case is still a valid test of that task. Many evaluations decide this with weak rules, such as accepting any well-formed edit or any edit that stays close to the original in
embedding space. We propose executable task contracts. A contract states which facts define a task, which edits are allowed, and what finishing the task means, and it checks each changed test case before the
agent runs. We compare validity rules by scoring the same 14,336 stored
agent runs under each rule. Across four encoders,
embedding thresholds allowed 75-100% of edits that changed the task, such as swapping the payee, yet rejected 24-45% of valid hostile notes. With a MiniLM cosine threshold, Mistral appeared vulnerable on all 64 tasks, but only 26 had a successful valid attack. Replaying a saved solution and allowing edits to
benchmark fields fail in similar ways. Evaluations should state, and check, what counts as the same task.
Read the full abstract ▾
All authors Shaikh Akib Shahriyar, MD. Akil Raihan Iftee, Armon Barton, Matthew Wright
Artificial Intelligence · Cross-listed from cs.CL
Scale, Drivers, and Limits of Representational Alignment Across Large Language Models
Jialin Wu and Meenakshi Khosla
Comparing large
language models can reveal shared representational structure and systematic differences between them. However, aggregate similarity measures can obscure how agreement varies across
geometric scales and which linguistic properties support it. We introduce two complementary methods to resolve where models agree and what supports that agreement. Standard representational similarity analysis (RSA) summarizes similarity across all stimulus pairs, potentially obscuring differences in
alignment between nearby and distant inputs. Neighborhood-Weighted RSA (NW-RSA) addresses this limitation by weighting stimulus pairs according to their proximity in representational space, placing
alignment from local neighborhoods to global geometry on a common scale. Counterfactual RSA (CF-RSA) tests which linguistic properties support
alignment by comparing intact representations in one model with representations under controlled linguistic perturbations in another. Across diverse
language models, we find strong agreement in coarse representational geometry but substantial divergence in local neighborhoods.
Alignment is observed both across broadly sampled natural text and within linguistic equivalence classes that preserve particular aspects of meaning or structure. Counterfactual analyses reveal selective sensitivity: changes to logical scope reduce
alignment substantially more than the tested changes to hierarchical syntax, event roles, or discourse structure.
Alignment also decreases when sentence structure or lexical meaning is disrupted in word lists and jabberwocky sentences, suggesting that the regularities of natural
language support stronger convergence. These findings refine claims of representational universality by showing that strong overall similarity need not imply agreement in how models represent finer linguistic distinctions.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
RES-MR: Risk-Aware Reasoning for Explainable and Safe Medication Recommendation
Cong Wang, Jin Li, Shoujin Wang et al. (+3 more)
Medication recommendations seek to deliver personalized drug combinations based on patients' clinical information, where ensuring transparent decision-making and personalized safety are of paramount importance. However, existing approaches predominantly operate as “black boxes”, offering limited interpretability and hindering clinician trust. Furthermore, they typically rely on fixed, patient-agnostic safety constraints, neglecting the heterogeneity of risk tolerance across individuals, which poses significant risks to vulnerable populations. To address these gaps, we propose RES-MR, a novel Risk-aware
Reasoning framework for Explainable and Safe Medication Recommendation using LLMs. RES-MR follows a two-stage training paradigm: (1) an explainable clinical
reasoning distillation stage to elicit diagnostic capabilities by fine-tuning on synthesized
reasoning trajectories, grounded in patient-specific knowledge from graph
retrieval; and (2) a risk-aware policy optimization stage to dynamically balance therapeutic efficacy with personalized safety. This stage employs disentangled drug factors to construct patient-specific reference prototypes that capture desired therapeutic drug features and risk features to avoid. These prototypes enable risk boundary calibration and guide the model toward safe recommendations via reward shaping and clipping. Extensive experiments on
benchmark datasets demonstrate that RES-MR significantly outperforms state-of-the-art baselines in terms of accuracy, safety, and explainability. Our code and data are available at: https://github.com/wangcong2001/RES-MR.
Read the full abstract ▾
All authors Cong Wang, Jin Li, Shoujin Wang, Yishuo Li, Huilin Gu, Wenpeng Lu
Artificial Intelligence · Cross-listed from cs.GT
Diffusion-Based Auctions for Multi-Winner Banner Ads
Lilian Sun, Henry Huang, Fucheng Warren Zhu et al. (+2 more)
We propose a new framework for banner ad auctions, leveraging
diffusion models to allow for multiple bidders to co-win a single banner ad opportunity, and in particular influence the ad shown on the banner depending on the strength of their bids, enabling a far richer outcome space compared to standard banner ad auctions, where there is a single winner who shows their ad. In our framework, the outcomes are images (what ad to show) and each bidder expresses their preferences over outcomes by declaring their value per unit of
alignment between the image chosen by the mechanism and a textual description of their desired image. Our proposed mechanism combines the advertisers' bids and textual descriptions into a score that drives the reverse process of a
diffusion model. This is used to sample a set of candidate images from which the welfare maximizing one is eventually chosen. Extensive experimentation evaluates our proposed mechanism around the following axes: (1) efficiency improvements of our allocation rule relative to the single-winner allocation rule; (2) monotonicity of our allocation rule; and (3) approximate incentive compatibility when combining our allocation rule with Myerson lemma-based or Clarke rule-based payments. Improvements along all these axes are shown with the number of Monte Carlo samples. Meanwhile, these benefits are attained while preserving the image quality.
Read the full abstract ▾
All authors Lilian Sun, Henry Huang, Fucheng Warren Zhu, Giannis Daras, Constantinos Daskalakis
Artificial Intelligence · Cross-listed from cs.IR
Co-adaptive Rule Routing and Constraining for Structured Knowledge Retrieval
Zhisheng Qi, Haoyu Han, Jiaxi Li et al. (+9 more)
Structured knowledge
retrieval acquires task-relevant evidence from structured knowledge sources for downstream tasks. Existing methods rely on unconstrained
retrieval or predefined query-rule supervision, such as author-paper rules for scientific discovery and anatomy-drug rules for precision medicine. Such supervision is rarely available in practice. We propose Co-Adaptive Rule Routing and Constraining
Retrieval (CoRE), which jointly learns which
reasoning rule to assign to each query and how to retrieve evidence under that rule, without predefined query-rule pairs. A routing
agent selects a rule, and its corresponding retriever traverses the knowledge base and ranks evidence that satisfies the rule's constraints. Because routing and
retrieval are interdependent, CoRE alternates between reinforcement learning for the routing policy and contrastive learning for the rule-constrained retriever. The selected rules provide supervision for retriever training, while
retrieval performance rewards the routing policy. This closed-loop process enables both components to co-adapt without explicit query-rule supervision. Experiments on STaRK-QA
datasets across scientific discovery, biomedical
reasoning, and e-commerce knowledge management demonstrate CoRE's effectiveness. Ablations show the benefit of co-adapting the router and retriever. Generation-based routing transfers across
datasets and benefits from multi-domain pretraining followed by target-specific fine-tuning. Analysis of rule usage across queries also informs automatic rule refinement and discovery.
Read the full abstract ▾
All authors Zhisheng Qi, Haoyu Han, Jiaxi Li, Bing He, Huiyuan Chen, Franck Dernoncourt, Ryan A. Rossi, Mahantesh M. Halappanavar, Jin Lu, Yu Zhang, Tianming Liu, Yu Wang
Artificial Intelligence · Cross-listed from cs.MA
Jevolution: A Framework for Studying Communication and Collective Dynamics in AI Agents
Xirui Huang, Nikhil Hooda and Utkarsh Gupta
Experiments with remote decision-making
agents depend on more than the policy: the world may change during inference, the simulator determines which actions are available, and births and immigration alter who can act. We present Jevolution, an instrumented spatial predator-prey framework that makes these dependencies explicit. Its reference implementation separates the evolving world, asynchronous request scheduling, controller adapters, and experiment records. The model combines local observations, energy and resource constraints, predefined signals, and inherited scalar traits. We describe these mechanisms using the ODD protocol and distinguish implemented controls from research questions requiring additional interventions. Free synthetic fixtures exercise scheduling, signaling, and inheritance against the pinned implementation; they validate specified software behavior rather than model intelligence or ecological realism. A completed communication case study illustrates the end-to-end workflow using Jev to control both species. Across 6 complete habitat-seed pairs, the mean signals-on minus signals-off difference in normalized rabbit abundance was -0.0095 (nominal 95% pair-bootstrap interval [-0.0772, 0.0602]). This small study does not resolve the direction of the signal-system effect. The contribution is an inspectable experimental system and its evidence boundaries, not a new policy-learning
algorithm or a demonstration of emergent
language, adaptive selection, or biological predictive validity.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
KV4-RL: Unlocking Faster Asynchronous Rollouts for RLVR with INT4 KV Cache Quantization
Sihwan Park, Sung-Yub Kim, Doohyuk Jang et al. (+1 more)
Asynchronous rollout improves the efficiency of reinforcement learning with verifiable rewards (RLVR) by reducing under-utilization during long-tail generation. Sustained concurrency also concentrates context processing into fewer decode steps, increasing the KV workload per step. We characterize this KV concentration and find that KV cache quantization delivers substantially greater acceleration than weight-only and weight-activation quantization in our matched-workload comparisons. However, INT4 KV rollout can degrade learning even with
token-level truncated importance sampling (TIS). We propose KV4-RL, which combines calibration-free INT4 KV quantization with selective INT8 protection of
attention sinks and recent context. Protecting a bounded number of positions reduces sampler mismatch while retaining most of the speedup from INT4. Across 400 iterations of Qwen3-8B-Base GRPO training, KV4-RL achieves $1.80\times$ cumulative rollout speedup and $1.27\times$ cumulative full-iteration speedup over BF16 rollout, reaching $1.97\times$ rollout speedup over the final 50 iterations. Under a common BF16 evaluator, its final six-
benchmark mean accuracy is within 0.5 percentage points of BF16-rollout training for both Qwen3-4B-Base and Qwen3-8B-Base.
Read the full abstract ▾
All authors Sihwan Park, Sung-Yub Kim, Doohyuk Jang, Eunho Yang
Artificial Intelligence · Cross-listed from cs.SE
Specification-Based Testing of Large Language Models
Kausar Moshood, Jace Bolante, Roshan Aditya Murali et al. (+1 more)
Model cards, system cards, and safety reports increasingly document LLM safety behaviors, yet safety evaluations typically start from predefined
benchmarks rather than asking which documented claims those
benchmarks actually test. We introduce LEST (LLM Evaluation through Specification-driven Testing), a framework that treats documented safety claims as specifications for black-box testing. In LEST, each claim determines which
benchmark prompt tests it and whether a Refuse-to-Comply transition, where a model refuses a
prompt but complies with its mutated version, counts as evidence against it. LEST mutates these
prompts to search for counterexamples, manually validates each candidate, and traces validated counterexamples back to the claims they contradict. We evaluate LEST using the system cards of Claude Opus 4.8 and GPT-5.5 with 97 seed
prompts from SORRY-Bench across Mental Health and Self-Harm, Harmful and Violent Content, and
Fairness and Bias. Of 56 identified safety claims, 29 (51.8%) are
prompt-testable, but only 17 (58.6%) are exercised by an applicable
benchmark prompt. Both models refuse >95% of
benchmark prompts. Evaluating 1,836 mutated
prompts on both models reveals 312 candidate Refuse-to-Comply transitions flagged by automated response classifiers. However, human validation confirms only 19 (6.1%) as both intent-preserving and genuinely compliant, showing automated evaluation substantially overestimates behavioral failures. Tracing these 19 transitions back to the claims reveals counterexamples for 6 of 9 covered claims for which refusal is an applicable oracle. Our results show that
benchmark prompts may not be sufficient to validate all documented LLM safety behavior, and automated failure detection requires manual validation. LEST provides end-to-end traceability from natural-
language safety documentation to executable tests and human-validated counterexamples.
Read the full abstract ▾
All authors Kausar Moshood, Jace Bolante, Roshan Aditya Murali, Manish Motwani
Artificial Intelligence · Cross-listed from cs.LG
Target-ESS Tempered Policy Optimization (TETPO): Smooth Variance Control via Bridge Importance Sampling
Shayan Mohajer Hamidi, Kayhan Behdin, Wenhui Zhu et al. (+4 more)
Asynchronous reinforcement-learning post-training of large
language models is computationally efficient but inherently off-policy: the learner updates on rollouts generated by a stale policy, while correcting this mismatch requires importance weights that can be extremely heavy-tailed. A small number of trajectories may then dominate an update, collapsing the effective sample size (ESS) and increasing the variance and instability of the resulting gradient estimates. Existing stabilizers apply fixed interventions regardless of the realized severity of weight concentration, either discarding selected samples or reweighting them without adapting the correction strength. We introduce Target–ESS Tempered Policy Optimization (TETPO), which replaces each importance weight $\rho$ by $\rho^\alpha$ for a data-adaptive exponent $\alpha\in[0,1]$: $\alpha=1$ preserves the original importance weights, whereas $\alpha=0$ assigns uniform coefficients. For each
prompt group, TETPO selects the largest $\alpha$ whose tempered weights satisfy a target ESS fraction $\kappa$, applying little tempering to healthy groups and stronger tempering when weight degeneracy is severe. At the population level, the tempered coefficients are exact importance weights for a
geometric bridge between the sampler and learner policies. The exponent is efficiently found by one-dimensional bisection, while the resulting weights satisfy deterministic bounds on normalized-weight dispersion and weighted-gradient norm that are independent of policy drift, together with a fixed-$\alpha$ bound on bridge-target discrepancy. Power tempering also introduces no explicit zero-weight mask. On mathematical-
reasoning benchmarks under high staleness, TETPO outperforms strong on- and off-policy baselines.
Read the full abstract ▾
All authors Shayan Mohajer Hamidi, Kayhan Behdin, Wenhui Zhu, Aida Rahmattalabi, Jincheng Cao, Jelena Markovic-Voronov, Alborz Geramifard
Artificial Intelligence · Cross-listed from cs.CV
SceneForesee: Learning Scene Evolution for Spatiotemporal Reasoning
Chaoyue Li, Shoudong Han, Haoyang Wu et al. (+4 more)
Large
multimodal models (LMMs) need to understand how entities move and interact as a scene develops. This requires following the same entities over time and relating their observed motion to subsequent changes. We formulate Scene
Evolution Understanding (SEU), a task for predicting and interpreting this change. Given observed
video, a model estimates entity histories and predicts one scene continuation through future trajectories and Events, accompanied by Visual Evidence from observed motion and a scene description. We construct SceneForesee, a
benchmark for evaluating entity continuity and scene
evolution reasoning through paired observed and future scenes with shared entity identities. It contains 144.57K temporal pairs, 1.05M entity correspondences, and 272.49K Events paired with observed evidence. We propose a spatiotemporal
reasoning framework, EvoSee, that learns future geometry and Event semantics through shared candidate states and joint assignment. It selects one continuation per entity from observation and uses the selected changes for entity refinement, retention, and scene representation. On SceneForesee, EvoSee outperforms Qwen3.5-9B by 10.45 points in Combined Point-F1 and 17.24 points in Event Global F1. On the 4D
reasoning benchmark Dyn-Bench, it improves question answering accuracy and grounding J&F by 4.8 and 3.58 points over their respective base systems. These results support scene
evolution as both a prediction target and a dynamic representation that brings predicted entity changes into spatiotemporal
reasoning.The project is available at https://github.com/mikubaka88/SceneForesee-Spatiotemporal-
Reasoning.
Read the full abstract ▾
All authors Chaoyue Li, Shoudong Han, Haoyang Wu, Boxue Yang, Sijia Chen, Yufan Chen, Ruping Zhang
Artificial Intelligence
Frustratingly Easy Agentic RAG via Portable Skills: A Case Study in Customer Support
Istok Bako, Bonan Min, Tiffany Shao et al. (+2 more)
Enterprise workflow automation typically requires retrieving a complete and accurate set of relevant information from large operational document corpora (SOPs, policy documents) given a specific scenario or request. In customer support (CS), for example, human specialists must determine the correct sequence of resolution steps for each case, a task that demands precise grounding in policy documentation. Meeting this bar in production typically requires complex document chunking, indexing, and
retrieval pipelines, often layered with learned re-rankers, query rewriters, and bespoke agentic RAG infrastructure. We report a deployment case study of a simpler alternative: a portable skill consisting of a natural-
language instruction, lightweight tools (a
retrieval API, format-checking scripts), and a corpus enriched with an LLM-generated navigation layer, which is a structured "Wikipedia-style" index over the underlying documents. This agentic RAG approach is straightforward to implement (1-2 engineer days), portable across capable agentic harnesses (e.g., Claude Code), and performs strongly on the challenging task of CS resolution workflow generation, achieving 99.2% procedural equivalence to human-approved ground truth and 83.3% evidence sufficiency. It is also roughly an order of magnitude slower per case than our deployed production assistant (~90 s vs. ~8-9 s), so we claim no latency advantage. Our claim is scoped to that setting: a mature, well-curated corpus, a latency-tolerant workflow, and human review of every case. There, the skill did the work of a complex RAG stack, and maintenance became a matter of editing text and data rather than retraining models.
Read the full abstract ▾
All authors Istok Bako, Bonan Min, Tiffany Shao, Claire Cheng, Yashar Mehdad
Artificial Intelligence · Cross-listed from cs.LG
Coarse-Graining Recursive Expert Computation: Continuation-Relative Interfaces and Failure Modes
Shihua Yu, Yanzhi Wang and Tianchi Chen
Computational coarse-graining aims to replace a sequence of updates with a cheaper module while preserving task performance. We present a continuation-relative evaluation framework that assesses replacements through the computation that follows them, using downstream task quality and total execution cost as joint criteria. The framework distinguishes boundary representations, summaries of skipped computation, executed operations, and downstream loss. It makes interfaces and permitted deployment information explicit, combines fine-path references with directly trained controls using the same shortened architecture, and accounts for selection overhead. An exact loss decomposition separates replacement effects at a fixed downstream action from effects of changing that action. A finite construction shows how retained information can reduce the stipulated cost of zero-error prediction under fixed information and controller restrictions. Experiments in two frozen pretrained
language models show that representation fidelity and task loss can favor different fitting objectives. In a prespecified comparison on a 1.7-billion-parameter model, next-
token-loss training lowers in-distribution negative log-likelihood by 0.1901 nats per scored
token compared with reconstructing the skipped blocks' output, using the same optimizer and update count. This ranking reverses in all nine paired comparisons under distribution shift. In a recursive expert prototype, the jointly adapted candidate lowers task error relative to frozen-bank and router-only baselines, but trails directly trained controls, and the tested input-only selectors fail the joint quality-cost criteria. Together, the protocol and studies provide explicit criteria and diagnostics for distinguishing useful computational replacement from local approximation gains within specified interfaces and evaluation settings.
Read the full abstract ▾