Artificial Intelligence · Cross-listed from cs.CV
Feixiang Liu, Qiang Qiu, Lanbo Sun et al. (+3 more)
Closed yes/no spatial
benchmarks can reward a correct answer even when the image adds little support beyond no-image contexts. Under a fixed forced-choice interface, Visual Credit Audit (VCA) separates two estimands: whether the
benchmark image gives the model's declared decision more support than text-only and blank controls, and whether the model responds to relation-specific visual evidence. The first audit is training- and label-free and does not require an answer flip. Applying labels yields dependence-credited correctness (D-CC); on correct items, it equals same-control gold-aligned positive gain, while prediction
alignment extends the audit to errors. Across four open MLLMs and two spatial
benchmarks, 12.73-26.25% of decisions are correct yet uncredited. Matched same-split image permutation reduces D-CC by 21.25-47.80 points, with every paired 95% interval above zero. Fixed-pixel relation contrasts and a 3x3 evidence-source factorial show why null controls cannot identify relation response. Among controlled correct-but-uncredited agreement decisions, response to relation reversal spans 81.57-100.00%, while 32.11% pooled change answer. Independently audited outcomes on 108 geometry-compatible edits provide a bounded natural-image correspondence check. VCA thereby decomposes
benchmark success into correctness, additional image support, and relation-consistent response.
Read the full abstract ▾
All authors Feixiang Liu, Qiang Qiu, Lanbo Sun, Nan Wei, Huawei Shen, Xueqi Cheng
Artificial Intelligence · Cross-listed from cs.CV
SciFigAlign: Scoring Scientific Figures by Fine-tuned Alignment of Visuals with Manuscript Evidence
Chuanzhi Xu, Zihan Deng, Huiqi Liang et al. (+4 more)
Scientific figure assessment in peer review differs fundamentally from general image quality evaluation: a figure must be visually legible, faithfully support the manuscript's claims, and communicate evidence with a clear visual hierarchy. However, if we apply traditional image assessment methods to scientific figure quality assessment, limitations emerge: classic IQA models capture perceptual quality or aesthetics but cannot judge whether a figure serves the paper's scientific argument; CLIP-based methods assess generic image-text correspondence, yet lack understanding of manuscript context; and zero-shot LLM/VLM judges, when repurposed for figure scoring, often yield overly concentrated scores with limited fusion of visual and textual evidence. We introduce an annotated
dataset of 3,857 scientific figures from peer-reviewed conference papers, each rated along four peer-review-oriented dimensions: Clarity, Relevance, Informativeness, and Structure. We propose SciFigAlign, a fine-tuned
multimodal scorer that grounds figure quality assessment in manuscript evidence. Given a figure crop, caption, citing paragraphs, and light paper context, SciFigAlign fine-tunes CLIP and SciBERT end-to-end with per-modality cross-
attention and CubeMLP fusion, jointly optimizing SmoothL1 regression with a within-paper ranking hinge loss. Under paper-level splits, SciFigAlign achieves a macro MAE of 0.3524 and a within-paper pairwise accuracy of 81.64% on the test set with n = 396, a 59% relative error reduction over the best LLM-as-judge baseline with MAE 0.864. Ablations confirm that manuscript-grounded inputs, citing-context denoising, and ranking supervision are all critical, showing that scientific figure assessment requires learned
alignment between visual content and manuscript evidence rather than prompting alone, even with state-of-the-art VLMs.
Read the full abstract ▾
All authors Chuanzhi Xu, Zihan Deng, Huiqi Liang, Chengkun Yue, Zhanlin Cui, Pengfei Ye, Weidong Cai
Artificial Intelligence · Cross-listed from cs.CV
ScratchSim: A Procedural Synthetic Data Pipeline for Surface Scratch Detection
Paul Julius Kühn, Saptarshi Neil Sinha, Tiago Kleist et al. (+3 more)
While automated defect detection such as the detection of surface scratched is an important aspect in industrial quality control, the scarcity of annotated defect data make this task challenging. This paper presents a procedural rendering pipeline that generates large-scale annotated synthetic training data using BlenderProc, with configurable material appearance, camera modes, and domain randomization, producing automatic COCO-format annotations. To show the potential of our approach, we evaluate four training strategies, namely synthetic-only, real-only, mixed, and fine-tuning from synthetic weights, across two objects with different material properties and three lightweight edge-deployable detectors, YOLOX, YOLO26, and LW-DETR. Our evaluation show that fine-tuning from synthetic weights consistently outperforms real-only training, and that mixed training effectively recovers performance under scarce real-data conditions, with findings validated across both convolutional and
transformer-based architectures. The proposed approach enables scalable defect detection without the burden of large real annotated
datasets, making it practical for on-device industrial inspection. The pipeline scripts for generating synthetic scratches, 3D model, and both the synthetic and real annotated scratch
datasets for a glossy toy Ferrari car are publicly available at https://github.com/saptarshineil/ScratchSim.
Read the full abstract ▾
All authors Paul Julius Kühn, Saptarshi Neil Sinha, Tiago Kleist, Richard Hoffmann, Arjan kuijper, Michael Weinmann
Artificial Intelligence
Setoka: A Benchmark for Hierarchical User Understanding in Personalized Agents over Heterogeneous Data
Lingyang Zeng, Guangze Chen, Kaichen Yu et al. (+9 more)
Personalized
agents are increasingly applied to assist users across a wide range of tasks. Effective personalized assistance requires not only retrieving explicit facts from past interactions stored in
agent memory, but also inferring abstract personal characteristics. However, existing
memory benchmarks primarily evaluate whether an
agent can retrieve information explicitly stated in conversational histories, failing to provide an effective assessment of deeper user understanding. In this work, we propose Setoka, a
benchmark for evaluating
memory-augmented personalized
agents with hierarchical user understanding from heterogeneous data. Grounded in theories from cognitive and personality psychology, Setoka defines four levels of user understanding, i.e., semantic
memory, episodic
memory, behavior pattern, and personality trait. Moreover, to enable realistic yet
privacy-preserving evaluation, we design a psychometrics-based pipeline that synthesizes diverse, coherent heterogeneous user data and queries at scale. Finally, we leverage Setoka to evaluate 3
language models combined with 5
memory systems for 10 synthetic users. Our comprehensive evaluation reveals that while existing systems perform well on semantic
memory retrieval, their performance declines on episodic
memory. Moreover, when dealing with behavior pattern and personality trait understanding tasks that require integrating heterogeneous and fragmented information dispersed over time, performance declines even further. These findings demonstrate that user understanding cannot be handled by simple fact
retrieval, motivating the design of
memory mechanisms for cross-source integration and abstraction over long-term user behavior.
Read the full abstract ▾
All authors Lingyang Zeng, Guangze Chen, Kaichen Yu, Zhicheng Pan, Siyang Weng, Zirui Hu, Xiangyun Du, Hailin He, Rong Zhang, Chengcheng Yang, Kai Huang, Xuan Zhou
Artificial Intelligence · Cross-listed from cs.LG
CoCaRS: Correlation Calibration-Based Redundancy Suppression for Heterogeneous Knowledge Distillation
Fengming Yu, Haiwei Pan, Kejia Zhang et al. (+3 more)
Knowledge
distillation (KD) enables a compact student model to learn from a powerful teacher and has become an effective paradigm for model compression. The emergence of diverse model architectures has extended KD from homogeneous to heterogeneous settings. However, differences in architectural inductive biases between the teacher and student models often result in substantial representation discrepancies, limiting the effectiveness of direct knowledge transfer. Recently, redundancy suppression has offered a new perspective on heterogeneous KD by preserving cross-architecture invariance and reducing feature redundancy through decorrelation of teacher-student feature correlations. Nevertheless, this formulation may weaken useful structural information through uniform decorrelation, while a fixed coefficient may make the effective contribution of redundancy suppression sensitive to teacher-student pairs and training stages. To address these problems, Correlation Calibration-based Redundancy Suppression (CoCaRS) is proposed to better retain structural information while suppressing redundancy and reduce sensitivity to coefficient settings across teacher-student pairs and training stages. Specifically, CoCaRS calibrates feature decorrelation through Confusion Evidence Estimation (CEE) and Strength Allocation Control (SAC), which respectively capture reliable semantic relations for correlation estimation and preserve discriminative structure during decorrelation. Adaptive Coefficient Regulation (ACR) further regulates the contribution of the calibrated redundancy suppression objective according to its relative loss scale, reducing sensitivity to coefficient settings. Extensive experiments on CIFAR-100 and ImageNet-1K validate the effectiveness of CoCaRS in improving
distillation performance and reducing sensitivity to coefficient settings. Code will be released soon.
Read the full abstract ▾
All authors Fengming Yu, Haiwei Pan, Kejia Zhang, Chunling Chen, Jian Guan, Baoying Ma
Artificial Intelligence · Cross-listed from cs.LG
BayesAME: Bayesian Active Model Evaluation
Paula Cordero Encinar, Taylan Cemgil, Arnaud Doucet et al. (+2 more)
Evaluating large generative models across
benchmarks is time-consuming and computationally expensive. This drives the need for methods that can estimate full
benchmark performance by evaluating models on only a subset of items, known as a coreset. Current literature mostly requires the practitioner to input a coreset size. However, when reliable performance estimation takes priority over efficiency, an evaluation method should also be capable of automatically determining a coreset size that reflects this priority. We introduce BayesAME, a sequential
Bayesian framework specifically targeting automatic determination of the coreset size. BayesAME models performance as a random variable by defining a latent ability for each group of items sharing the same historical model performances, with a joint prior distribution encoding the belief that the target model behaves similarly to these historical models. The posterior distribution over these abilities is used to derive performance
estimators, quantify performance uncertainty, and select items to add to the coreset via an information-gain criterion. The coreset is iteratively augmented until the performance estimate fluctuation and the performance uncertainty fall below their respective user-defined thresholds. We propose a multi-target extension that captures performance correlations across multiple target models to further reduce the coreset size. Through extensive experiments across diverse
benchmarks, we demonstrate that BayesAME consistently outperforms sequential adaptations of existing methods. Crucially, our comprehensive analysis addresses recent skepticism in the literature, establishing that non-random coreset selection is advantageous over random selection. Finally, we highlight that leveraging continuous response log-likelihoods over traditional binary scores significantly enhances estimation accuracy.
Read the full abstract ▾
All authors Paula Cordero Encinar, Taylan Cemgil, Arnaud Doucet, Virginia Aglietti, Silvia Chiappa
Artificial Intelligence · Cross-listed from cs.RO
SymmGrid: Super-Scaling On-Robot Learning with Parallelized Symmetries and Egocentric-Exocentric Visual Perception
Gabe Everett, Brice Gunter, Ryan Vander Stelt et al. (+3 more)
Deep reinforcement policy learning directly in physical
robots (on-
robot learning) remains bottlenecked by slow wall-clock training times. We present SymmGrid, a trajectory level augmentation framework inspired by parallelized
symmetries that super-scales group transformations to significantly accelerate on-
robot learning in both
egocentric and exocentric visual setups. We model a Markov Decision Process (MDP) under a
symmetry tree, in which state-action pairs have admissible parallelized invariant transformations that yield a
geometric grid structure. The state is modelled with ego- or exocentric images and proprioception information. The latter require special treatment, in the form of homographies, to warp visual scenes in line with their corresponding spatial transformations. These parallelized transformations produce a large set of unique symmetric equivalences that populate the replay buffer with diverse and consistent experiences that speed up learning and improve performance. We present extensive training and evaluations performed directly on real
robot manipulation contact tasks including peg-insertions, cable routing, and object relocations. Relative to SOTA, SymmGrid achieved wall-clock training convergence speed-ups of 1.37-2.17x, evaluation success rate improvements of 1.09x-1.27x, fastest training convergence times of 16.6, 10.9, and 79.3 minutes respectively. For trajectory wide assessments, we used normalized area under the curve (nAUC) ratios. SymmGrid achieved improvements of up to 2.59x. These results confirm that simple branch
symmetries can have an outsized result due to super-scaling and bring us closer to sub-10 minute on-
robot learning training in manipulation tasks suitable for arms and
humanoids. The project page is available at symmgrid-
robot.github.io
Read the full abstract ▾
All authors Gabe Everett, Brice Gunter, Ryan Vander Stelt, Cleiver Ruiz-Martinez, Blake Hull, Juan Rojas
Artificial Intelligence · Cross-listed from cs.CV
Progressive Multimodal Alignment for Continual Instruction Tuning
Duzhen Zhang, Yahan Yu, Qiaoyi Su et al. (+2 more)
Multimodal Large
Language Models (MLLMs) rely on a projector to align visual representations with the
language embedding space, making it central to cross-modal understanding. In
Multimodal Continual Instruction Tuning (MCIT), however, shifting visual distributions and evolving instruction semantics cause this shared projector to drift, leading to projector-level forgetting, an issue largely overlooked by methods that focus primarily on the LLM backbone. We introduce Progressive
Multimodal Alignment (PMA), a framework that enables the projector to adapt continually while preserving previously learned
alignment. PMA detects
multimodal distribution shifts via a lightweight representation descriptor and progressively expands projector experts only when needed. An expandable router integrates expert outputs based on
multimodal features, while the original pretrained projector is retained as a stable
alignment anchor. This progressive mechanism balances stability and plasticity with sub-linear parameter growth and serves as a method-agnostic add-on to existing MCIT approaches. Extensive experiments on two recent MCIT
benchmarks demonstrate that mitigating projector-level forgetting yields consistent gains over prior state-of-the-art methods when combined with PMA. Moreover, PMA scales across diverse MLLM backbones, demonstrating robust and broadly applicable MCIT performance.
Read the full abstract ▾
All authors Duzhen Zhang, Yahan Yu, Qiaoyi Su, Jiahua Dong, Tielin Zhang
Artificial Intelligence
Belief-Guided Decision Making with Uncertainty Gating in the Game of Go
Mehrad Yaghoubi, Azam Bastanfard, Abbas Jalilvand et al. (+1 more)
Recent advancements in Computer Go, driven by AlphaZero and MuZero, rely heavily on Monte Carlo Tree Search (MCTS) to correct the errors of the neural network policy. While effective on massive computational clusters, this dependence creates a critical bottleneck on consumer-grade hardware, where the computational cost of tree management severely limits inference rates. Furthermore, without deep search, these models suffer from
hallucination, proposing moves with high confidence that are strategically fatal. This paper introduces a novel Belief-Guided architecture that disentangles the Policy head from a distinct Belief head. Unlike traditional value functions, the Belief head acts as an internal simulator and independent critic, modeling epistemic uncertainty and strategic stability. By integrating
memory mechanisms (
Transformer/GRU) to handle long-term dependencies and the Ko rule, and utilizing a gating mechanism to filter overconfident policy errors, our model shifts the burden of intelligence from runtime search to parametric "intuition." Experimental results demonstrate that this approach significantly improves search-free win rates and reduces
hallucination, enabling professional-level play on limited hardware where massive MCTS is infeasible.
Read the full abstract ▾
All authors Mehrad Yaghoubi, Azam Bastanfard, Abbas Jalilvand, Ashkan Rezaei
Artificial Intelligence
What Does It Take to Detect an AI Agent? Minimal Feature Sets for Behavioral Detection under Browser Automation
Vishisht Choudhary, Lukas Schmidt, Anne Zoë Kenntner et al. (+3 more)
Bot detectors deployed at scale treat traffic as binary: human or bot. This assumption breaks when AI
agents browse the web through browser automation, a traffic class that is neither and that binary classifiers structurally cannot represent. We present a three-class detection framework distinguishing humans, bots, and AI
agents, and show that the binary-vs-
agent confusion is architectural: a binary human-vs-bot detector misroutes
agent sessions because its label space lacks an
agent class. On our controlled
benchmark, an MLP binary classifier misclassifies 39.1% of real AI
agents as human and a SAINT binary
transformer misclassifies 34.5%; adding an explicit
agent class yields per-class
agent F1 = 1.000 in all 30 runs (3 model families $\times$ 10 seeds). To measure evasion resistance, we construct a five-level evasion ladder spanning passive observation, GAN-generated trajectories, and replay of real human cursor data ($n = 2299$ evasion sessions). Across 10 seeds and 3 model families we observe zero
agent misses in 22990 per-seed predictions. The discriminative signal is a browser-automation artifact, not evidence of
agent reasoning: Playwright does not emit the raw pointer-move and wheel-delta streams a physical input device produces, and this absence signature survives trajectory manipulation. Exhaustive search over all feature subsets of size 1-5 (9401 GBMs) shows that two behavioral features (mouse_event_rate, teleport_click_ratio) give 100% observed
agent recall at every evasion level with
agent precision 0.994; five features lift macro-F1 to 0.991. The signal is redundantly encoded: removing teleport_click_ratio leaves
agent detection at 100%. The single-feature regime is degenerate, flagging every
agent only by collapsing the classifier to always predict "
agent". Two features robustly isolate
agents; five separate all three traffic classes at macro-F1 $\geq 0.99$.
Read the full abstract ▾
All authors Vishisht Choudhary, Lukas Schmidt, Anne Zoë Kenntner, Feras Skhab, Michel Osswald, Jens Ernstberger
Artificial Intelligence · Cross-listed from cs.CR
Defending Against Backdoor Attacks via Alignment Checking in Model-Contrastive Federated Learning
Hongliang Zhang, Zhongyuan Yu, Guijuan Wang et al. (+4 more)
Federated Learning (FL) is vulnerable to
backdoor attacks because of its distributed nature in edge computing scenarios. Existing defense methods show limited efficacy as they overlook the deviations among benign local updates caused by statistical heterogeneity and the stealthiness of
backdoor attacks. To tackle these issues, we propose FedDAB, a two-phase method that combines local contrastive regularization with
alignment checking, to defend against
backdoor attacks. In the first phase, FedDAB introduces a novel model-contrastive term into the local objective to enhance direction and magnitude consistency among benign updates. In the second phase, FedDAB employs an
alignment checking strategy to evaluate each local update in terms of overall-direction
alignment and parameter-level
alignment with historical information, excluding updates that exhibit abnormal
alignment patterns from global aggregation. We theoretically prove FedDAB's robustness with a convergence rate of $\mathcal{O}(1/T)$. Extensive experiments show that FedDAB outperforms existing defense methods against
backdoor attacks.
Read the full abstract ▾
All authors Hongliang Zhang, Zhongyuan Yu, Guijuan Wang, Tianqing He, Wenshuo Ma, Xiaosong Zhang, Jiguo Yu
Artificial Intelligence · Cross-listed from cs.RO
BioVLN: A Simulation Platform for Visual Language Navigation in Biomedical Laboratories
Zhe Liu, Quan Lu, Zhaohui Du et al. (+7 more)
Biomedical laboratory
robots must navigate to instruments before performing experimental procedures. Existing
embodied navigation platforms are designed for household environments and treat a target as an object center or an arbitrary nearby position. This representation is inadequate for laboratory instruments, which must be approached from their operating side while maintaining safe clearance from surrounding equipment. We introduce BioVLN, a simulation platform for developing and evaluating visual-
language navigation agents in biomedical laboratories. BioVLN represents each instrument with three regions: its physical body, a surrounding clearance region, and an operation area in front of the usable side. This model is applied consistently to scene generation, target placement,
navigation evaluation, and safety analysis, so success depends on reaching a position from which the instrument can be accessed. BioVLN supports procedural scene generation and manually designed environments, producing 47 scenes and 1667 episodes. Standardized
navigation and reinforcement-learning interfaces enable trajectory collection and policy training. Experiments show that
geometric exploration reaches 74.4–87.5% success, while sampling multiple valid positions in the operation area improves success to 83.3–92.5% and reduces unsafe proximity.
Read the full abstract ▾
All authors Zhe Liu, Quan Lu, Zhaohui Du, Zhe Wang, Huanbo Jin, Jiaming Gu, Qi Wang, Ting Xiao, Minting Pan, Dongzhan Zhou
Artificial Intelligence · Cross-listed from cs.LG
Actions Have Consequences: Detecting Outcome Performativity using Intervention Testing
Brandon Gower-Winter and Georg Krempl
In many domains such as Palliative Care, Credit Assignment and Recommender Systems, predictions may causally influence the outcomes they predict. This phenomena is known as Outcome Performativity. This paper formalises an approach for detecting Outcome Performativity using prediction intervention called Outcome Performativity A/B Detection (OPAB). OPAB enables the detection of Outcome Performativity by assessing the dissimilarity in outcome distributions produced by different predictions groups (interventions). If that dissimilarity is significant, Outcome Performativity is detected. We derive sample complexity bounds for OPAB under various Outcome Performative assumption classes which we empirically validate. Results show that detecting Outcome Performativity using OPAB is achievable in numerous cases. Results also show the presence of regions of indistinguishability which describe settings where the allotted number of interventions are insufficient for detecting Outcome Performativity. The results of which have broader practical implications for the detectability of Outcome Performativity in settings where samples are scarce, cost-prohibitive or potentially unethical to obtain. The paper concludes with a case study on the efficacy of OPAB on the Open Bandits
dataset, and provides directions for future work.
Read the full abstract ▾
Artificial Intelligence
From Passive Video to Editable Experience: Physically Grounded Experience Synthesis for Embodied Intelligence
Jia Luo
The key bottleneck in
embodied AI is not model architecture but data. Although billions of human manipulation
videos exist online,
robots cannot directly learn from them due to the embodiment gap between human morphology and
robot hardware. We introduce Pegasus, a low-resource framework that bridges this gap by translating human demonstrations into
robot-learnable data through structured knowledge transfer. Instead of relying on raw
video prompts, Pegasus constructs a graph-based intermediate representation: a Task Graph extracted from human
videos is transformed through Affordance and Constraint Graphs into a
Robot Planning Graph for
robot-conditioned
video generation. A hierarchical affordance latent space models the relationship between object states, affordances, and tasks, enabling generalization beyond object identities. A closed-loop physics verifier further filters invalid generations using kinematic feasibility, collision constraints, and joint limits. We evaluate Pegasus across a range of
egocentric manipulation
benchmarks, including GTEA Gaze+ and EPIC-KITCHENS-100, and diverse
robot embodiments, assessing Task Correctness, Executability, State Consistency, and Learnability. Results demonstrate reliable cross-embodiment translation and show that
robot data generation can be reframed from a hardware collection problem into a scalable, low-resource knowledge transfer problem.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.HC
Human diversity fuels collective creativity that large language models cannot simulate or sustain
Mengchen Dong and Hiromu Yakura
Diverse human groups produce diverse ideas, the raw material of innovation. Generative AI challenges this engine twice over: everyday AI assistance may homogenize what diverse people create, and AI-simulated diversity may replace the people altogether. We tested both challenges in a preregistered creative metaphor experiment with native (L1) and non-native (L2) English writers, who wrote without AI, with AI-generated ideas (AI ideation), or with AI refining their own ideas (AI refinement). L2 writers contributed more collective diversity than L1 writers, with native-
language ideation showing the most diverse pools. AI ideation compressed collective diversity for everyone and left the L2 advantage undetectable, whereas AI refinement preserved both. We then simulated the entire writer pool using personas built from participants' real backgrounds, three model families, native-
language prompting, and elevated sampling temperatures. Every simulated pool fell below every human pool, and pushing models further induced diversity only through degenerate text. However, at the individual level, AI ideation raised writers' ratings, pitting private incentives against the collective good, except when L2 writers used their native
language, which benefited both. Human diversity remains a valuable creative resource that current AI cannot simulate or sustain; the design of human-AI collaborative workflows determines whether it survives.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CV
Hearsay: Vision-Language Medical Diagnoses Without an Image
Siddharth Vohra
When asked to describe a medical image that was never attached, frontier
vision-
language models do not abstain: they confabulate a diagnosis. We show that this confabulation is not random. It is structured by who the patient is said to be. Across chest X-ray,
brain MRI, and dermatology, Claude Opus-4.7, GPT-5.4, and Gemini-3.1-Pro are each queried with only a demographic descriptor and no image, and changing the descriptor systematically shifts the diagnosis returned. Claude concentrates sharply: a 65-year-old white man asking about a skin mole receives Melanoma in nearly every response, and a 32-year-old Black woman asking about her chest X-ray receives a Sarcoidosis diagnosis whose
reasoning reads "suspected, based on demographics and classic pattern.” GPT-5.4's effect is broader, fabricating across every demographic
cell we test, most conspicuously naming Sarcoidosis for young Black patients on chest X-ray. Two structural findings sharpen the problem. A hedged regime appears in which the prose acknowledges the missing image while the structured diagnosis field nevertheless names a disease, a dissociation invisible to prose-only audits. And Claude's dermatology effect collapses entirely when 'skin mole' is swapped for 'skin lesion' while GPT-5.4's is preserved, indicating that mirage is a family of distinct failure modes rather than a single phenomenon. Trustworthy VLM deployment in clinical pipelines requires
auditing the structured output channel directly, and probe-word sensitivity should be treated as a first-class evaluation dimension
Read the full abstract ▾
Artificial Intelligence · Cross-listed from stat.ML
Think Short, Defer Smart, Act, and Repeat: Calibrated Reasoning and Uncertainty-Aware Deferral for Edge LLM Agents
Amirmohammad Farzaneh and Osvaldo Simeone
LLM
agents following the ReAct paradigm are promising enablers of complex multi-step tasks, including multi-hop question answering, code generation, and control of physical AI systems. Yet, when deployed at the edge, they must tightly manage their
reasoning budget while remaining reliable and deferring to a cloud-side model only when local uncertainty is too high to act safely. We propose Think Short, Defer Smart (TSDS), a framework that synergistically integrates a lightweight convergence probe, which halts on-device
reasoning once the intended action has stabilized, with a perplexity-based deferral rule that escalates uncertain actions to a cloud-side model. Both mechanisms are jointly calibrated on end-to-end episode trajectories via a multi-objective Learn-Then-Test (LTT) procedure, providing simultaneous finite-sample guarantees on expected episode reward and cloud-call rate. We evaluate TSDS on four ReAct
benchmarks spanning arithmetic
reasoning (GSM8K), multi-hop question answering (HotpotQA), code generation (MBPP), and multi-step
embodied planning (household
robot), and compare against thought-calibration-only and calibrated-deferral-only standalone baselines. TSDS reduces per-episode thinking compute by 43%-65% over deferral-only baselines across HotpotQA, MBPP, and the household
robot task, while maintaining certified reward and cloud-call rate guarantees.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
ReCo: Reweighting GRPO Against Distributional Concentration
Junoh Park, Junseo Hwang, Wonguk Cho et al. (+1 more)
Group Relative Policy Optimization (GRPO) has become a standard reinforcement learning method for post-training
language models. Recent work shows that GRPO can reduce the base model's
reasoning capacity and underperform it in Pass@k when k is large, indicating reduced coverage of
reasoning paths. We find that this reduction is associated with GRPO concentrating on responses that the base model already generates with high probability. We trace this concentration to two mechanisms in the GRPO update. At the response level, high-probability responses dominate the group gradient through repeated occurrence. At the
token level, GRPO's importance ratio scales gradients, further reinforcing
tokens that become more likely under the current policy. We propose ReCo, a reweighting method that addresses both effects. Response contributions are normalized by their expected occurrence within the rollout group, and the
token-level importance ratio is replaced with a variance-based ratio that gives larger update scale to non-saturated decision points where alternative
token choices remain plausible. Across Qwen2.5-Math-1.5B/7B and Llama-3.1-8B-Instruct on five mathematical
reasoning benchmarks, ReCo improves Pass@k for large values of k and is comparable to GRPO for small values of k.
Read the full abstract ▾
All authors Junoh Park, Junseo Hwang, Wonguk Cho, Taesup Kim
Artificial Intelligence · Cross-listed from cs.CL
From Representations to Behaviors: Exploring the Person-Situation-Behavior Triad in LLMs
Ruikang Zhang, Shuo Wang and Qi Su
Human personality theories characterize traits not as isolated attributes captured by a single score, but as stable individual tendencies expressed through the interplay among persons, situations, and behaviors. Existing studies of personality-related behavior in LLMs have primarily focused on outputs elicited under personality conditioning, characterizing observable trait-related expressions while lacking mechanistic evidence for the existence of internal personality-related representations, their cross-situational expression, and how these representations shape specific behaviors. Building on Funder's personality triad framework, we adapt its three components for LLM analysis: Person as personality-related internal representations, Situation as contexts that afford trait-relevant responses, and Behavior as response patterns on broader social tasks. We introduce a framework for discovering, controlling, and validating trait-like representations in LLMs. First, using contrastive behavior pairs grounded in shared situations, we identify sparse internal features associated with opposing poles of personality traits through SAE decomposition. We validate their trait relevance through effects on behavior to situation,
token-level activation patterns, and robustness to paraphrasing. Second, feature-level interventions induce bidirectional trait-related shifts across a separate, diverse set of situations while preserving response validity, demonstrating consistent expression across contexts. Third, applying the same interventions to social intelligence tasks reveals behavioral changes with benefit-tradeoff patterns consistent with findings from human personality research, providing behavioral-level validation beyond personality scores. Our findings provide evidence that LLMs contain controllable trait-like representations linking internal states, situational expression, and behavioral outcomes.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
Budget-Aware LLM Discovery via Cost-Calibrated Frontier Utility
Yansen Zhang, Yilu Liu, Tianyu Liu et al. (+7 more)
Large
language models increasingly support scientific and algorithmic discovery through inference-time search over evaluated candidates. Existing adaptive discovery controllers assign credit based only on score progress, even though
prompt length, retries, and guidance calls cause search actions to incur different
token costs. Under a fixed search budget, the controller must decide which frontier to develop and whether its gain justifies the realized cost before the budget is exhausted. We prove that, in the worst case, cost-blind control can forfeit all but a vanishing fraction of attainable quality as frontier count and cost heterogeneity grow. To address this limitation, we introduce CostAda, an adaptive controller built on cost-calibrated frontier utility. By valuing progress relative to realized action cost and conditioning credit on the remaining budget, CostAda coordinates local exploration intensity, frontier allocation, and budgeted tactic intervention. Realized action cost and remaining budget therefore shape the search rather than serving only as accounting variables or a stopping rule. Across eight
benchmarks with GLM-5 and GPT-5.4, CostAda reaches the strongest baseline's full-budget quality with at most half the budget on thirteen of sixteen
benchmark–backbone pairs and achieves the strongest final quality on all sixteen pairs.
Read the full abstract ▾
All authors Yansen Zhang, Yilu Liu, Tianyu Liu, Jiamin Chen, Xiaokun Zhang, Kai Xie, Qingfu Zhang, Xue Liu, Yiyan Qi, Chen Ma
Artificial Intelligence · Cross-listed from cs.HC
AI as Friction for Reflection Support in Ideation
Janin Koch, Xiaohan Liao and Géry Casiez
Generative AI tools for creative work tend to be designed around the goal of removing friction, on the assumption that smoother iteration and faster output translate into more value for the designer. We argue, however, that this framing leaves out something important about how design ideation works, namely reflection-in-action. The act of accepting, rejecting and reworking candidate ideas is both a path to a final outcome and the process through which designers develop the rationale that allows them to think with their ideas and to communicate them to others. This becomes particularly important in group ideation, where ideas need to be expressed and explained to others to allow the group to extend, reject or combine them further. We suggest that AI in design ideation might be more usefully thought of as a friction
agent for reflection rather than as a smoothing
agent for output. This reframing opens up a different role for AI in design ideation, one that supports designers in building and carrying rationale rather than substituting for it.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.SE
A First Look at Coding Agents' Compliance with AI Contribution Rules in Open-Source Communities
Wenhao Yang, Runzhi He and Minghui Zhou
Open source communities have been flooded with AI-generated contributions. In defense, they have written contribution rules to regulate coding
agents' behavior, spanning from a total ban, mandatory disclosure, to verification gates and human sign-offs. Yet, whether coding
agents read and follow those rules, and behave in open source repositories, remains unknown. To estimate real-world rule compliance of coding
agents, we curate 106 issues from 49 repositories containing AI contribution rules into RepoComplianceBench. We judge the trajectory of each run against the repository's rules, measuring whether the
agent refuses to contribute, discloses its assistance truthfully, clears the required verification gates, or escalates critical steps to a human. We also test if extra
prompts, rule disclosure, or feedback from the compliance verifier help with the situation. Our experiments on four frontier models show that today's
agents almost never proactively retrieve the contribution rules.
Agents pick up disclosure and verification with reminder
prompts, rule quotes, and verifier feedback; however, they never refuse to contribute in AI-banned repositories under any condition we tested. The status reveals that verification and disclosure issues are solvable with existing mechanisms, yet enforcing bans and human escalations remains an open problem.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
FedTopo: Relation-Level Topology Sharing for Model-Heterogeneous Federated Learning
Zhaoyang Ma, Zhihao Wu, Xin Gao et al. (+3 more)
Federated learning (FL) enables collaborative learning over decentralized data silos without centralizing raw data. However, heterogeneous local architectures often induce non-aligned representation spaces, making it difficult to transfer global knowledge across silos. Existing paradigms share this knowledge as model parameters, distilled predictions, or class prototypes, yet all encode it in an absolute space that must be aligned across clients. Heterogeneous backbones break this
alignment, so the shared knowledge becomes unreliable and misleads local training. We propose FedTopo, a relation-level framework that encodes global knowledge as class relation
topology, capturing how classes relate within each client rather than where they lie in feature space. Each client builds its relation
topology from local prototypes and uploads it with class statistics. The server then aggregates these relations in a reliability-aware manner that down-weights weakly supported ones, and broadcasts the global
topology to clients. The global
topology guides local training by emphasizing
topology-similar negative classes. Experiments on three
datasets under eight heterogeneous backbones show that FedTopo consistently outperforms parameter-,
distillation-, and prototype-sharing baselines, with low communication and no inference overhead. Our code is available at https://github.com/Zhaoyang-Ma/FedTopo.
Read the full abstract ▾
All authors Zhaoyang Ma, Zhihao Wu, Xin Gao, Lipo Wang, Youfang Lin, Jing Wang
Artificial Intelligence · Cross-listed from stat.ML
Crossing-Free Probabilistic K-Line Forecasts Without Retraining
Runyao Yu, Yuchen Tao, Yujie Chen et al. (+2 more)
Probabilistic K-line
forecasting describes uncertainty in four complementary prices, namely open–high–low–close (OHLC). However, it introduces two consistency problems:
quantile crossing and K-line crossing.
Quantile crossing occurs when a higher-
quantile forecast falls below a lower-
quantile forecast, while K-line crossing occurs when the forecast low exceeds the open or close, or the forecast high falls below the open or close. Existing solutions generally address only one problem through output reordering, specialized architectures, or penalized training objectives. We propose K-line–
Quantile Sequential Projection (KQSP), a parameter-free and training-free reconciliation method applicable to forecasts produced by any model. Compared with other crossing solutions, KQSP preserves predictive accuracy while producing substantially smaller corrections to the original forecasts. To mitigate model bias, we evaluate KQSP using various models, including pretrained foundation models. KQSP reduces both
quantile and K-line crossing rates to zero for all test data undertaken. These results show that probabilistic K-line consistency can be enforced independently of forecast generation and without retraining.
Read the full abstract ▾
All authors Runyao Yu, Yuchen Tao, Yujie Chen, Wentao Wang, Derek W. Bunn
Artificial Intelligence · Cross-listed from cs.CR
SecRespond: Benchmarking AI Agents for Real-World Post-Compromise Incident Response
Lehan Wang, Boli Chen, Ruixue Ding et al. (+7 more)
Large
Language Model (LLM)
agents are increasingly adopted in real-world security operations with access to host artifacts and command-line interfaces (CLIs), making it critical to thoroughly assess their security capabilities. However, existing cybersecurity
benchmarks focus on pre-compromise settings where
agents are placed in a clean and idealized environment before an attack occurs. This leaves the post-compromise setting underexplored. To address this gap, we introduce SecRespond, the first
benchmark for evaluating LLM
agents on the post-compromise incident-response workflow. Given a forensic disk snapshot of a compromised host together with the alerts, vulnerability scans, and baseline checks reported by a host security product,
agents are required to produce forensic reports on intrusions, baseline risks, and vulnerability risks, together with a remediation plan. We instantiate this task across 10 cyber ranges, each constructed from a distinct compromised cloud host, spanning 4 entry-point types, 21 ATT&CK techniques, and 5 operating systems. We evaluate 23 frontier LLMs on the OpenCode
agent harness. Experimental results show that although current
agents can reliably uncover the problems exposed by alerts, they struggle to proactively investigate the disk for silent intrusions and to produce comprehensive, verified remediation plans, with no model achieving complete detection and remediation on any single range. This reveals a fundamental bottleneck in building
agents for real-world incident response. The
benchmark is publicly available at https://github.com/Alibaba-NLP/qqr/tree/main/data/secrespond.
Read the full abstract ▾
All authors Lehan Wang, Boli Chen, Ruixue Ding, Pengjun Xie, Jinwei Huang, Zhendong Liu, Shuo Wang, Tao Lei, Xin Ouyang, Xiaomeng Li
Artificial Intelligence
Property-driven Causal Abstractions for Markov Decision Processes
Jule Schmidt, Maximilian Weininger, Clemens Dubslaff et al. (+2 more)
Markov Decision Processes (MDPs) are widely used as decision-making models, commonly specified over factored state spaces through state variables and their valuations. The exponential blowup in the number of states renders many
reasoning tasks in MDPs challenging. Abstractions are promising techniques to reduce MDPs and thus mitigate scalability issues. In this work, we introduce a notion of causality on factored MDPs and a novel property-driven causal abstraction technique that retains many characteristics of the original MDP model. For this, we rely on causal relations over state variable predicates and identify those states that share the same reasons for fulfilling or violating a given abstraction property. We theoretically and empirically compare various causal MDP abstractions using different model types such as MDPs, interval MDPs, or
stochastic games. Our evaluation demonstrates the potential of our approach: For several standard
benchmarks, we obtain small abstractions that allow us to compute near-optimal policies for the original MDP. Furthermore, our causal abstractions often generalize to related large-scale MDP models.
Read the full abstract ▾
All authors Jule Schmidt, Maximilian Weininger, Clemens Dubslaff, David Parker, Nils Jansen
Artificial Intelligence · Cross-listed from cs.LG
SkillRise: Agentic Reinforcement Learning for Cross-Task Skill Evolution
Zhiyuan Yao, Yuxin Chen, Zhengxi Lu et al. (+13 more)
Large
language model
agents often encounter related yet distinct tasks that share reusable solution patterns. Yet standard agentic reinforcement learning treats tasks as independent episodes, while existing approaches to skill learning either focus on repeated attempts of one task or use pipelines with multiple stages that entangle extraction,
retrieval, and execution. We introduce SkillRise, a unified reinforcement learning framework for learning skills across tasks. SkillRise organizes related instances into progressively challenging sequences and uses a single policy to alternate between task solving and curating an evolving skill document passed directly to the next task. Decoupled credit assignment across tasks supervises solving with the current task outcome and curation with discounted downstream outcomes. Experiments on ALFWorld, WebShop, and ScienceWorld show that SkillRise achieves the strongest Pass@1 performance among the compared methods, with gains over the strongest baseline ranging from 2.3 to 8.5 percentage points. Although trained across distinct tasks, its learned curation policy remains effective for repeated attempts on the same task. Further analysis reveals scaling at test time across tasks: performance improves with longer sequences of related tasks even when each task is attempted only once. This trend suggests that SkillRise reuses transferable skills across tasks rather than benefiting from repeated sampling of the same task. SkillRise further retains strong performance while substantially reducing the runtime overhead of skill learning pipelines with multiple stages. Together, these results provide a simple and efficient training paradigm for LLM
agents to extract, refine, and reuse transferable skills across tasks.
Read the full abstract ▾
All authors Zhiyuan Yao, Yuxin Chen, Zhengxi Lu, Zishan Xu, Yueqing Sun, Yifu Guo, Yuquan Lu, Zhengzhou Cai, Kangning Zhang, Zhuowen Han, Zi-Han Wang, Ziang Ye, Qi Gu, Xunliang Cai, Weiwen Liu, Yongliang Shen
Artificial Intelligence · Cross-listed from cs.LG
Journey Operators for Structured Multi-Axis Composition
Mahesh Godavarti
Many kinds of data have structure along one or more axes: words in a sentence, pixels in an image, nodes in a tree, frames in audio, or
cells in a 3D volume. Along one axis, order matters: "the dog bit the man" is different from "the man bit the dog." Across independent axes, however, neither composition nor movement should depend on the order of axes: in an image, composing right then down should give the same result as composing down then right, and moving right then down should describe the same relative position as moving down then right. We develop a framework for modeling this kind of multi-axis structure. Each data item carries its content together with a small transformation for each axis. A path connecting two positions defines a journey; the journey operator is the product of per-axis transformations along that path, governing both how data composes along the path and how relative position is described. When the transformations are fixed, our framework recovers Rotary Position
Embedding (RoPE) and its multi-dimensional variants. When they depend on the data, the model gains a content-adaptive positional inductive bias. We show exactly when these paths are well-defined: both composition and movement across axes are path-independent precisely when the axis transformations commute. We also prove that, under the stated toral-frame
symmetry, cocycle, bilinearity, and norm-preservation assumptions, the resulting pairwise scoring rule must take the form of block-wise rotations, explaining why RoPE-like methods arise naturally. Finally, we use this theory to design JoFormer, a model for value aggregation, and relate it to
attention and state-space models (SSMs). Initial experiments across
vision,
language, and length generalization suggest that these inductive biases can have observable consequences in practice.
Read the full abstract ▾
Artificial Intelligence
Do Latent Channels Actually Communicate? A Causal Audit of Latent Multi-Agent LLM
Huixiang Zhang and Mahzabeen Emu
Latent communication in large
language model (LLM)-based multi-
agent systems (MAS) transmits continuous internal representations instead of text, but greater representational capacity does not establish that the receiver uses task-relevant information. End-task performance alone also cannot reveal whether an observed effect depends on message presence, content generated for the evaluated example, or information supplied by a separate
agent. We introduce a causal audit that applies controlled message replacements at the boundary where the sender-produced representation enters the receiver. Four message settings support five measurements of encoded sender information, receiver sensitivity to message presence and identity, the task value of example-specific content, and the additional value supplied by a separate
agent. We apply the audit to latent relay with Qwen3-4B and Qwen3-8B on GSM8K, ARC-C, and MATH-500. On GSM8K, the Qwen3-4B overall performance effect of -1.00 percentage point decomposes into a -6.17-point effect retained by an other-example message and a +5.17-point effect attributable to example-specific content; both component directions reverse at 8B. On MATH-500, the Qwen3-4B gain of 15.00 points comprises 8.33 points retained by an other-example message and 6.67 points attributable to example-specific content, while the 8B gain is dominated by the former component. Self-substitution comparisons further show that example-specific content and other-
agent value are distinct. These results show that aggregate accuracy does not identify how a latent message affects the receiver and motivate controlled message comparisons as a standard evaluation for latent communication.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CV
See2Think: Do Multimodal Models Really Use Intermediate Visual States?
Siyu Yan, Zhuoran Yan, Haiying Xu et al. (+10 more)
Multimodal large
language models increasingly use sketches, annotations, tools, and intermediate images during
reasoning, but it remains unclear whether they truly rely on these visual states. Existing
benchmarks are limited both by task collections with narrow coverage or partially text-solvable samples and by evaluations that emphasize final answers without diagnosing how intermediate visual states are generated, rendered, and used. We introduce See2Think, a unified evaluation framework comprising See2ThinkBench and Visual Action-of-Thought (VAoT). See2ThinkBench contains 1,200 open-ended, visually dependent problems across 12 task categories spanning 2D structured, 3D scene, and real-world
reasoning. VAoT records textual thoughts, visual actions, rendered states, and subsequent
reasoning under four controlled inference settings. Evaluating representative proprietary and open-source
multimodal models, we find that visual
reasoning is strongly model- and environment-dependent, with no single setting consistently dominating across tasks. Process analysis further shows that models usually select relevant visual operations, while faithful rendering remains the clearest bottleneck and high feedback uptake does not necessarily translate into accuracy gains. Under task-relevant corrupted feedback, models exhibit behavioral dependence on visual states, with accuracy dropping by over 10 percentage points in controlled interventions.
Read the full abstract ▾
All authors Siyu Yan, Zhuoran Yan, Haiying Xu, Panhao Zhou, Jingyu Chen, Chenhao Ji, Shuo Cao, Yongheng Zhang, Haoze Liu, Siyu Zhang, Xiwen Gu, Yihao Liu, Alex Jinpeng Wang