Artificial Intelligence · Cross-listed from cs.CV
OPIS: An Input-Grounded Benchmark for Multi-Object Memory in Video World Models
Hao Wang, Tao Yu, Liuzhou Zhang et al. (+13 more)
Video world models must preserve the visual state of the world over time, but existing evaluation protocols often rely on generated histories,
video reference, or selected revisit viewpoints that can confound the assessment of a model's true
memory capability. To address this, we introduce OPIS, an input-grounded
benchmark that strictly anchors the assessment to a fixed set of object instances from the initial observation for evaluating multi-object
memory in
video world models. The OPIS
dataset comprises 500 cases across real-world,
embodied-robotic, and game-world domains, providing dense object-level annotations for 12,672 rigid, articulated, and deformable instances. Our object-centric evaluator combines association and explicit visibility
reasoning to hierarchically measure Object (O) Presence (P), Identity (I), and Structure (S), utilizing static or dynamic evaluation tracks based on object kinematics. Across eight image-to-
video or camera-conditioned world models, our proposed OPIS scores range from 48.65 to 56.01. As the reference inventory grows from less than 20 to more than 40 objects, the Presence, Identity, and Structure scores show an overall decline, with the average Identity score falling from 40.22 to 23.11. The results demonstrate that preserving the particular object instances in the input is considerably harder than generating plausible visual elements.
Read the full abstract ▾
All authors Hao Wang, Tao Yu, Liuzhou Zhang, HeXin Wang, Haopeng Jin, Yuxuan Zhou, Xinming Wang, Hongzhu Yi, Xinye Li, Yuanlei Wang, Ping Nie, Yan Huang, Yuxuan Zhang, Pengfei Zhou, Yanyan Zou, Wei Yang
Artificial Intelligence · Cross-listed from cs.RO
EMPIRIC: Experiment-Driven Learning of Residual World Models for Robot Planning
Yichao Liang, Amber Li, Dat Nguyen et al. (+13 more)
A
robot should be able to learn through experiments how unfamiliar objects behave and interact, then plan with that knowledge. It need not start from scratch: physics engines supply knowledge of motion and contact, but can omit entire mechanisms, such as glue curing, water heating, or wind. We present EMPIRIC, an
agent that learns a residual world model: a physics engine extended with code for the missing mechanisms. The learned programs can introduce new forces, constraints, and hidden state, and
Bayesian inference estimates their parameters and states from noisy observations. The resulting model lets the
agent predict the outcomes of actions, choose informative experiments, and revise its hypotheses when predictions fail. Across five simulated domains, EMPIRIC learns interpretable, reusable models, and solves more tasks with fewer environment interactions than all three baselines. On a physical
robot, it learns wind forces and domino masses to solve a manipulation task. Website and code: https://yichao-liang.github.io/empiric
Read the full abstract ▾
All authors Yichao Liang, Amber Li, Dat Nguyen, Emily Bunnapradist, Michelangelo Naim, Sreela Kodali, Matteo Merler, Bowen Li, Kiran Gopinathan, Yiyun Liu, Nikhil Pimpalkhare, Joshua B. Tenenbaum, Adrian Weller, Zenna Tavares, Tom Silver, Kevin Ellis
Artificial Intelligence · Cross-listed from cs.LG
BA-DPO: Bias-Adjusted Direct Preference Optimization for Language Model Alignment
Antonio Ferrara, Alberto Rumi and Francesco Bonchi
Preference-based
alignment methods such as Direct Preference Optimization (DPO) use pairwise preferences labeled by human annotators to fine-tune
language models. However, annotators carry systematic biases toward some attributes: a name that signals a gender or an ethnicity, a persona, a
language variety, a formatting convention, or length. If not properly addressed, these systematic biases can be absorbed and amplified during
alignment. Existing methods address length bias or annotator disagreement, but fail to eliminate biases toward arbitrary attributes. To address this limitation, we propose Bias-Adjusted DPO (BA-DPO), a generalization of DPO that adds one bias parameter per annotator toward responses carrying a declared attribute. We prove that the objective is convex in the bias parameters and that the votes identify each annotator's bias up to a shared constant. The remaining constant is what fixes the aligned model's attribute rate: by default the rate of the reference model, or a target rate, which we use to bring a biased policy to statistical parity. On a corpus with planted biases, DPO drives the attribute from a balanced start to probability 0.96 and BA-DPO removes 81 to 95% of that shift; on MultiPref with real annotators it removes about half of DPO's lengthening. Both hold at 0.5B with full fine-tuning and at 8B with LoRA, at no higher KL than DPO and no loss in judged quality.
Read the full abstract ▾
Artificial Intelligence
Persona Following Is Not Selective Control: The Neutrality Gap in LLM User Simulation
Jiashen Ren, Wenlin Zhang, Bohan Zhang et al. (+5 more)
Persona prompting is widely used to construct user simulations with large
language models (LLMs), yet it relies on a largely untested assumption: specifying one user attribute should change that attribute alone. We test this assumption and identify a systematic failure of selective control: across all eight black-box LLMs we audit, changing a target attribute also shifts responses on unspecified, non-target attributes. For example, describing a user as more risk-seeking shifts color choices, even though the
prompt never mentions color; we term this cross-attribute influence. Semantic, contextual, and internal analyses collectively suggest that models treat a persona
prompt as evidence about the user and extend the inferred profile to unspecified preferences, a process we call trait-conditioned completion. We next ask whether explicitly specifying non-target attributes restores selective control. When a non-target attribute is assigned a clear direction, models generally follow the declaration and suppress the target attribute's influence. However, when the same attribute is declared neutral, the target continues to affect choices across all five open-weight checkpoints, even when the model correctly reports the declared state. This disparity, the neutrality gap, demonstrates that successful persona following does not imply selective persona control, which additionally requires keeping non-target attributes stable. We operationalize this distinction with a three-state diagnostic that leaves the non-target attribute unspecified or declares it directional or neutral; because directional tests can be passed by simply following the stated persona, the neutral state reveals failures they miss. In a post hoc analysis of independent items, neutral declarations leave 51-81% of items target-sensitive, against at most 1 of 320 item-pole comparisons under directional ones.
Read the full abstract ▾
All authors Jiashen Ren, Wenlin Zhang, Bohan Zhang, Xiaopeng Li, Zichuan Fu, Wanyu Wang, Junyi Li, Xiangyu Zhao
Artificial Intelligence
Zhixi Cai, Fucai Ke, Sukai Huang et al. (+4 more)
In complex
embodied visual
reasoning scenarios, an
agent often has only a limited field of view, and the evidence needed to answer a question may be distributed across time, viewpoint, and interacting objects. A model may therefore give a plausible answer without ever observing the relevant object, time, or view that supports it. Current visual
reasoning benchmarks largely evaluate passive observations and final answers, overlooking settings that require active
reasoning and evidence acquisition. We introduce JRDB-AVR, a
benchmark derived from existing real-world JRDB robotics data through a structured question-generation engine that turns this gap into an explicit evaluation: an
embodied agentic system receives a visual
reasoning question, requests bounded observations by timestamp and viewing angle, and is evaluated on both the final answer and the grounded visual evidence supporting it. The
benchmark contains diverse questions over multiple real-world environments involving temporal search, viewpoint selection, and human-oriented compositional
reasoning. We also introduce JRDB-AVR-
Agent, a reference active
reasoning agentic method that maintains an explicit observation-grounded graph-based world model and answers through solving. Experiments reveal a substantial gap between answer accuracy and evidence accuracy in current baselines, showing that current VLMs can produce unsupported correct answers and that active evidence-aware evaluation is necessary for
embodied visual
reasoning. Code and
benchmark are available at https://github.com/ControlNet/JRDB-AVR.
Read the full abstract ▾
All authors Zhixi Cai, Fucai Ke, Sukai Huang, Maria Garcia de la Banda, Peter J. Stuckey, Gholamreza Haffari, Hamid Rezatofighi
Artificial Intelligence · Cross-listed from cs.IR
PEAR: Progressive Evidence-Based AutoResearch for Industrial Search Systems
Yifan Wang, Shipeng Zhu, Fei Xiong et al. (+6 more)
AutoResearch improves systems through iterative experimentation:
agents propose candidate modifications, evaluate them, and use the results to guide subsequent exploration. Applying this paradigm to industrial search presents two challenges. (1) Common AutoResearch approaches follow a keep-if-better rule, retaining the highest-scoring candidate for subsequent experiments. Under non-stationary traffic, transient gains may be mistaken for persistent improvements, impairing reliable accumulation of search knowledge. (2) Candidate modifications can be evaluated at multiple fidelity levels, from low-cost proxies to online validation, differing in cost, objective
alignment, and statistical reliability. Existing methods rely on individual signals or task-specific procedures, lacking a unified basis for using evidence across levels to guide search. We introduce Progressive Evidence-Based AutoResearch (PEAR) with two complementary components. Evidence-driven AutoResearch maintains an independent, hypothesis-guided research state for each strategy task within a predefined objective and intervention scope. Each state evolves through a Plan-Execute-Evaluate-Update transition that links experimentation to context-aware evidence interpretation and hypothesis revision. Confidence-Gated Verifier Ladder organizes evaluation into four levels of increasing fidelity: Offline Replay, Shadow-Traffic Evaluation, Rapid Online Evaluation, and Decision-Grade Online Evaluation. A unified confidence-based gate promotes candidates only when evidence supports a statistically significant positive effect, enabling broad low-cost exploration while reserving costly online experiments for promoted candidates. In a real-world industrial search system, strategies optimized with PEAR significantly increased Main Order/DAU by 2.7336% and 3.2957% relative to their respective baselines in two A/B experiments.
Read the full abstract ▾
All authors Yifan Wang, Shipeng Zhu, Fei Xiong, Yuqin Yang, Yonghui Huang, Kunyao Wu, Yue Wang, Weichao Meng, Yu Gong
Artificial Intelligence · Cross-listed from cs.LG
VEX-Bench: Benchmarking Verification Complexity of LLM-Generated Misinformation
Hanxun Huang, Yutao Wu, Qizhou Wang et al. (+9 more)
Large
language models (LLMs) have made misinformation inexpensive to produce but not to verify, creating a growing asymmetry in the information ecosystem. Under tight time, labor, and budget constraints, media organizations, platforms, and fact-checkers rely on screening to prioritize which content to verify. We introduce VEX-Bench, a unified
benchmark for evaluating the verification complexity of LLM-generated misinformation, as perceived during screening, across models and generation methods. Verification complexity is assessed along multiple dimensions derived from journalistic and fact-checking practices, capturing checkability, harm potential, source credibility signals, imposter legitimacy, and expected verification effort. We define the VEX score as an integrated measure combining elicitation yield and verification complexity to quantify how generated content consumes limited verification capacity. We construct a
benchmark spanning two misinformation categories, 6 high-stakes domains, and 60 real-world topics, and evaluate 7 frontier LLMs and 7 generation methods, yielding 5{,}880 articles. We employ an LLM-as-judge for scalable evaluation and validate it using content-analysis methodology, including ordinal Krippendorff $\alpha$ for inter-annotator reliability, complemented by fact-checking
agents for verification. Our findings show that no single method dominates all dimensions, underscoring the need for multi-dimensional evaluation. LLMs can generate high-VEX misinformation at 3$\times$ to 169$\times$ lower cost than
agent-based verification. Such content is often prioritized during screening, consuming scarce verification resources and introducing a systematic risk of misallocation in resource-constrained verification systems. The code is publicly available in our \href{https://github.com/HanxunH/VEX-Bench}{GitHub repository}.
Read the full abstract ▾
All authors Hanxun Huang, Yutao Wu, Qizhou Wang, Silvia Montaña-Niño, Yige Li, Xiang Zheng, Elif Buse Doyuran, Phoebe Matich, Xiao Liu, Xingjun Ma, Sarah Erfani, Christopher Leckie
Artificial Intelligence
WebPageBench: Event-Level Verification and Controlled UI-Variant Generation for Web Agents
Anton Emelyanov, Maria Tikhonova, Zaven Martirosian et al. (+2 more)
We present WebPageBench, an open framework for evaluating web
agents in which every task is verified from the interface's own event log. Six instrumented mock sites with brand identifiers removed (a marketplace, a bookstore, a grocery service, rail ticketing, hotel search and a document cabinet) emit typed events with parameters as a user or an
agent acts. A task declares the events it requires, and success is decided by matching them, with no judge model and no scraping of rendered pages. The same instrumentation supports controlled UI variation: one configuration switch re-renders a task through a different implementation of a single control while the
prompt and the success conditions stay completely identical, so sensitivity to interface form can be measured under a fixed task specification. The WebPageBench release consists of three components: 152 tasks, divided into 65 canonical scenarios and 87 control variants across light/dark UI-modes; a common runner evaluated with six browser/DOM harness configurations and five screenshot-only GUI-
agent families; and a public leaderboard of 24 model-harness pairs. On the public 152-task leaderboard the gap between what
agents declare finished and what the log confirms reaches 41 points (one configuration declares every task finished and satisfies the conditions on 59%).
Read the full abstract ▾
All authors Anton Emelyanov, Maria Tikhonova, Zaven Martirosian, Sergei Averkiev, Alena Fenogenova
Artificial Intelligence
AutoDataBench: Can Agents Write the Data That Feeds the Self-Improvement Loop?
Haotian Luo, Haoyu Wang, Zeyu Qin et al. (+5 more)
Recent gains in
language model capability have come more from data than from architecture. Frontier labs and data companies produce verifiable agentic tasks, which supervised finetuning and reinforcement learning then turn into capability.This production line still rests on human labour and on human-in-the-loop collaboration. Automating task creation would let data production scale with compute rather than with expert headcount, would extend to more domains, and would enable a key step in recursive self-improvement (RSI). Current evaluations of an
agent's ability to write such tasks measure how a model performs after training on what the
agent produced. That does not match common practice in the data industry, where data is delivered sample by sample and each sample is accepted against a set of criteria rather than put straight into training. No existing evaluation asks whether an individual task meets the acceptance criteria of a data pipeline. We therefore introduce AutoDataBench. Given an original
benchmark task and a record of the target model attempting it, an
agent must write a new task for the same suite that meets practical acceptance standards on validity, novelty, difficulty and behavioural coverage. Across three
benchmarks of executable
agent tasks, no
agent we evaluate scores above 20 out of 100 at the default time budget of 45 minutes. Giving the strongest
agent four times as long improves its score substantially, while the cost of one usable task stays almost unchanged. Current
agents can write training tasks of the required quality, but not efficiently. AutoDataBench provides a direct measure of an
agent's capacity for autonomous data synthesis: one artifact at a time, judged against the criteria a production pipeline would apply, and without a training run. Code and data are available at https://github.com/StarDewXXX/AutoDataBench.
Read the full abstract ▾
All authors Haotian Luo, Haoyu Wang, Zeyu Qin, Huanjin Yao, Yibo Wang, Zhuotao Tian, Shuai Wang, Jiaya Jia
Artificial Intelligence · Cross-listed from cs.LG
Addressing Spatial Indistinguishability in Spatiotemporal Prediction via Optimal Transport-Guided Masking
Guangyu Wang and Jiawei Tong
Spatiotemporal prediction aims to learn discriminative representations from correlated temporal signals over spatial structures for accurate future inference. A central challenge is spatial indistinguishability: different nodes may share similar historical patterns yet evolve toward divergent futures, severely degrading
forecasting performance in real-world sensor networks. Existing
embedding-based and graph neural network (GNN)-based approaches can partially detect such ambiguous nodes but rely on historical similarity, struggling to capture future behavioral divergence. We propose STOT (SpatioTemporal Optimal Transport), a self-supervised framework that resolves spatiotemporal ambiguity via structured masking guided by optimal transport. Our key idea treats indistinguishability as a disambiguation problem: future states are inferred by exploiting concurrent spatial correlations and their time-varying similarity. We design a similarity-aware metric for dynamic inter-node relationships and an optimal transport-based masking strategy to emphasize ambiguous positions during pre-training. A batch consistency constraint preserves semantic coherence, while a random-walk masking mechanism promotes structured context exploration. Experiments on six real-world
datasets show that STOT performs competitively with state-of-the-art baselines on the evaluated
benchmarks and improved interpretability through transport-plan visualizations.
Read the full abstract ▾
Artificial Intelligence
Environmental requirements for the use of social information by artificial life agents using evolved plastic artificial neural networks
Hugh Charterton, James M. Borg and Aniko Ekart
Evolved Plastic Artificial Neural Networks (EPANNs) consist of two principal processes, the first,
evolution, and the second, development and in-life learning. In the context of the origins of social = learning, very few studies have been carried out using ALIFE models based on EPANN requirements. Studies in this field have usually involved an imitative teacher/pupil relationship. This, however, ignores the possibility that the observed behaviour is a consequence of social information cues rather than direct imitation or teaching. Starting with the first of the EPANN processes (
evolution), a series of experiments was undertaken using artificial neural network (ANN) based
agents in a variety of foraging environments to examine under what minimal environmental conditions the use of social information might have evolved, as measured by the number of generations taken to meet a specified fitness criterion. NEAT (Neuroevolution of Augmenting
Topologies) was the ANN used as its evolutionary
algorithm would evolve a network's
topology as well its weights. Unintentionally, in the experiment there was a simple network
topology based on the location of the nearest food item which enabled
agents to swiftly meet the fitness criterion. With this
topology, additional information, social or otherwise, was not required and could have proved to be a hindrance. However, this does indicate that for the use of social information to have evolved, it would require a greater degree of complexity in the environment to do so.
Read the full abstract ▾
Artificial Intelligence
TermJudge: A Document-Level Metric Judging, Not Counting, Terminology in Machine Translation Evaluation
Nicolas Dahan, Fran{\cc}ois Yvon and Rachel Bawden
Existing automatic metrics for evaluating terminological use in machine translation (MT) penalise any divergence from a fixed reference, conflating translation errors with the valid terminological variation that human translators routinely produce. We introduce TermJudge, a document-level terminology metric that assigns an interpretable verdict to every term occurrence: glossary-conforming occurrences are settled deterministically, while divergences are assessed under a two-step LLM-as-judge procedure using the full document context: the first detects and labels terminology errors; the second sorts valid document-level variations from inconsistencies. Validated against expert error annotations and document-level human MQM scores, TermJudge ranks first in both system- and segment-level meta-evaluation, ahead of glossary-conformity and quality-estimation baselines. When applied to eight systems translating academic documents, under two prompting conditions, we observe that glossary injection improves terminology translation in all paired comparisons, by removing genuine errors rather than valid variation. TermJudge is released as open-source code.
Read the full abstract ▾
Artificial Intelligence
Automated Feature Engineering, AutoML, and Decision-Focused Learning for Improved Energy Consumption Forecasting
Nasser Alkhulaifi
The rising cost and demand for energy, together with environmental sustainability goals, create major challenges for energy management. Energy Consumption
Forecasting (ECF) supports planning by predicting future consumption, but Machine Learning (ML) models for ECF often depend on expert-driven Feature Engineering (FE). This thesis addresses that dependence through three contributions. First, it establishes and evaluates a comprehensive FE pipeline for ECF and investigates domain-specific features. Second, it introduces AutoEnergy, a domain-tailored automated FE
algorithm that generates interpretable features from timestamps and lagged consumption and integrates with AutoML for end-to-end ECF modelling. Across eighteen real-world energy
datasets spanning residential, commercial, industrial, renewable, and grid domains, AutoEnergy reduces
forecasting error by 19.52%-84.72% relative to baseline AutoML and established automated FE methods, while running 1.31-4.41 times faster, with gains varying by
dataset. Third, AutoEnergy is integrated with Decision-Focused Learning (DFL) for a Battery Energy Storage System problem, jointly
forecasting electricity prices and demand while optimising charging and discharging decisions. On a real-world UK property
dataset, this approach reduces operating costs by 22.9%-56.5% compared with the same DFL models without automated FE. Overall, the results show that domain-specific automated FE can reduce reliance on manual feature design, improve
forecasting accuracy, and translate predictive gains into measurable operational benefits in energy management.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.RO
Learning to Act under Visual Interruptions with Vision-Language-Action Models
Mingle Jiang, Rui Xu, Yunke Wang et al. (+1 more)
Vision-
language-action (VLA) models have demonstrated strong capabilities in robotic manipulation, but they are typically developed and evaluated with all camera streams available throughout task execution. When a camera stops delivering frames during task execution, the policy must continue acting without access to subsequent observations from the missing view. Despite its practical importance, how such interruptions affect closed-loop manipulation remains insufficiently understood. To investigate this problem, we introduce MAIL-Bench, a
benchmark that evaluates visual interruptions with VLA models. By interrupting different cameras at multiple stages of each policy's successful reference trajectory, MAIL-Bench measures how well policies retain their capabilities when visual inputs become unavailable. Building on this
benchmark, we propose MINT, which first trains VLA policies to remain functional under missing visual inputs. At inference time, MINT selectively supplements missing observations using optical-flow extrapolation or an action-conditioned world model, and withdraws predicted views when they become unreliable. Experiments on $\pi_{0.5}$ and GR00T N1.5 show that MINT significantly improves task success under camera loss over the original models. Experiments on AgiBot G2 further demonstrate the real-
robot deployment under camera loss. The
benchmark is available at https://minglejiang.github.io/Mail-Bench/
Read the full abstract ▾
All authors Mingle Jiang, Rui Xu, Yunke Wang, Chang Xu
Artificial Intelligence · Cross-listed from cs.CV
Still There, No Longer Seen: Exposing Compression-Induced Risk in Large Vision-Language Models
Qiankun Li, Yuechen Zhang, Bowen Chen et al. (+4 more)
Visual
token compression reduces the inference cost of Large
Vision-
Language Models (LVLMs). However, aggregate robustness measures do not reveal whether a particular
adversarial failure is induced by compression or inherited from the underlying model. We define a compression-specific failure (CSF) as an
adversarial input that remains correct under full-
token inference but fails after compression, casting compression-induced risk as a paired failure attribution problem. Within a controlled diagnostic cohort, counterfactuals show that retained-set allocation causally changes compressed correctness and reveal a negative association between recovery and representation drift in displaced evidence. Motivated by these findings, we propose CIRA, a Compression-Induced Risk Attack for Large
Vision-
Language Models. Under a
vision-encoder white-box setting, CIRA optimizes image perturbations through encoder-side objectives that manipulate
token priorities across candidate compression budgets while preserving displaced evidence. CIRA uses no downstream questions or labels and requires no access to the
language model, deployed compressor, or exact compression budget. Across 12
dataset-compressor settings evaluated at four budgets, CIRA achieves a mean CSFR of 20.35% while limiting full-
token attack success to 6.92%, with similar behavior on additional LVLM families. A cross-view selection-stabilization defense substantially suppresses CIRA, although Adaptive CIRA partially restores its effectiveness. These results show that compression-specific failures persist under restricted access and support paired evaluation of full-
token and compressed inference for attributing risk to visual-
token compression.
Read the full abstract ▾
All authors Qiankun Li, Yuechen Zhang, Bowen Chen, Shilinlu Yan, Zhenhong Zhou, Kun Wang, Li Sun
Artificial Intelligence
From One-Shot Generation to Incremental Music Composition: Adapting a General-Purpose Instruction LLM for Persistent Symbolic Editing
André Ricardo Ducca Fernandes, Jean-Pierre Briot, Simone Diniz Junqueira Barbosa1 et al. (+1 more)
Most music-generation systems are still framed and evaluated primarily as producers of complete outputs, whereas composition often proceeds through successive revisions to a shared musical artifact. This paper studies a different use of a general-purpose instruction-following large
language model: not as a one-shot music generator, but as a reusable operator over an evolving symbolic score. We formulate incremental composition as a sequence of operation-aware state transitions over persistent ABC notation, with explicit requirements on what each operation may change and what it must preserve. The interaction includes two artifact-initialization variants and three editing operations – chord addition, inpainting, and transposition. We instantiate the formulation by adapting Llama 3.1 8B Instruct with Low-Rank Adaptation (LoRA) on 496,038 operation-aware dialogue records derived from Irish traditional music. The comparison with the unadapted model is used to test the feasibility of learning this interaction contract, not to claim novelty for fine-tuning itself. Across 500 dialogues per model (1,750 attempted output states), checker admission rises from 29.37% to 99.37%, while compliance conditional on admission rises from 0.7205 to 0.9798. Strict eligibility for reference-relative musical-feature analysis increases from 14 to 1,548 outputs, and Longest Common Subsequence analysis does not show a systematic increase in high-overlap sequences relative to held-out baselines under the specified protocol. The results support the technical feasibility of persistent, operation-aware symbolic editing with a general-purpose instruction LLM. They do not establish superior musical quality or human-AI co-creativity, which remain questions for musician-centered evaluation.
Read the full abstract ▾
All authors André Ricardo Ducca Fernandes, Jean-Pierre Briot, Simone Diniz Junqueira Barbosa1, Hélio Côrtes Vieira Lopes
Artificial Intelligence · Cross-listed from cs.LG
Composable Decoding on the Probability Simplex: Theory and Implementation
Xiaotong Ji, Ahmed Khaled Khamis, Rasul Tutunov et al. (+2 more)
Decoding for large
language models is typically treated as a collection of isolated sampling strategies, with limited theoretical understanding of the behaviours they induce and how their underlying objectives relate. We formulate decoding as an optimisation problem over next-
token distributions on the probability simplex, balancing expected model score against regularisation under support constraints. This view recovers familiar decoding methods through choices of regularisers and support constraints; more importantly, it enables new decoders to be constructed by composing distributional preferences within a single optimisation problem without external rewards, learned critics, or model parameter updates. We introduce CompoSimplex, a library with configurable support rules, regularisation primitives, and simplex solvers for constructing and evaluating compositional decoders. We evaluate standard samplers, individual regularisers, and compositions across multiple models and
reasoning tasks. Our results show that compositions can realise trade-offs between single-sample quality, multi-sample quality, and diversity that are not attained by individual decoding objectives.
Read the full abstract ▾
All authors Xiaotong Ji, Ahmed Khaled Khamis, Rasul Tutunov, Matthieu Zimmer, Haitham Bou-Ammar
Artificial Intelligence · Cross-listed from cs.CV
SPIDER: Multi-Layer Semantic Token Pruning and Adaptive Sub-Layer Skipping in Multimodal Large Language Models
Tianxiang Chen, Zhentao Tan, Zi Ye et al. (+9 more)
Multimodal Large
Language Models face significant efficiency challenges that stem from two distinct yet coupled sources: data redundancy and computational redundancy. While most methods focus on data redundancy by
pruning visual
tokens from the output of the visual encoder or computing redundancy in LLM decoders using blockwise importance, the finer-grained inter-layer representation shifts and the distribution differences within the layers themselves have not been fully explored. In this work, we comprehensively investigate this dual-level inefficiency. We posit that intermediate layer
tokens from
vision encoders should be considered for effective visual
token pruning, as semantic focus shifts across layers, with middle-layer
tokens capturing more detailed object-centric information that deeper layers may abstract away. Furthermore, we reveal the differential contributions of
Attention and FFNs across distinct LLM decoder layers. Building upon these discoveries, we propose SPIDER, a training-free framework that integrates multi-layer \underline{S}emantic visual
token \underline{P}run\underline{I}ng with an a\underline{D}aptive sub-lay\underline{ER} skipping mechanism. Experimental evaluations demonstrate that SPIDER consistently maintains strong performance across various MLLM architectures and reduction ratios. For instance, on LLaVA-NeXT-7B, SPIDER reduces FLOPs by $79\%$ while maintaining 96$\%$ of the baseline performance.
Read the full abstract ▾
All authors Tianxiang Chen, Zhentao Tan, Zi Ye, Yue Wu, Xiaobing Tu, Jinkui Ren, Xiantao Zhang, Tao Gong, Qi Chu, Nenghai Yu, Xipeng Qiu, Jieping Ye
Artificial Intelligence
Before Acting, Change the State: Prospective State Intervention for Web Agents under Deceptive Interfaces
Ruozhao Yang, Mingfei Cheng and Xiaofei Xie
LLM-based Web
agents can autonomously complete user tasks, yet deceptive interfaces can steer them toward outcomes that conflict with users' interests. Existing defenses primarily intervene on
agent behavior through blocking, guidance, or replanning. We identify a distinct failure mode: a task-valid action can still realize an unauthorized consequence because of the current Web state. This motivates treating task-relevant Web state itself as a runtime control target. We introduce Veer, an
agent-side runtime defense that leaves task planning to the base
agent and intervenes on Web state when a proposed action would produce an unauthorized consequence. Before modifying the live environment, Veer constructs a prospective intervention trajectory toward a safe task-relevant state and executes it with runtime grounding and verification. Across TrickyArena and WebDecept, Veer achieves the highest safe task completion in all three evaluation settings, exceeding the next-best defense by 15.9 and 25.0 percentage points on TrickyArena-Single and TrickyArena-Multi, respectively, while reducing dark-pattern success on WebDecept to 0.3%. These gains persist across dark-pattern types and all 12
agent, model, and
benchmark configurations. Ablations show that active state intervention provides the largest gain, while prospective rollout and temporal evidence contribute additional improvements. These results establish task-relevant Web state as an effective runtime control target for protecting Web
agents from deceptive outcomes.
Read the full abstract ▾
Artificial Intelligence
APEX-Voice: Can Voice Agents Complete Professional Workflows Through Full-Duplex Interaction
Puneet Mathur and Dinesh Manocha
Full-duplex voice
agents can now listen, speak, use tools, and act during spoken interactions, but fluent dialogue does not guarantee correct completion of delegated professional workflows. We introduce APEX-Voice, a
benchmark of 120 interactive professional workflows spanning ten work archetypes such as form completion, corporate negotiation, coordination, consulting, and interviewing. Each workflow executes in a stateful Voice Workbench environment with task-specific knowledge, typed tools, gold-annotated final work artifact, authorization constraints, and a user simulation policy backed by validated, pre-compiled
speech realizations. We evaluate both artifact field accuracy and end-to-end workflow success, which requires the correct terminal state, valid process, completed actions, and a valid final artifact. Across five frontier real-time voice
agents-GPT-Live-1, Gemini-3.8-Live, Grok-Voice-Think-2.0, Step-Audio3, and GPT-realtime-2.1, none exceeds 25% Pass@1, and the best Reliable@3 is only 10.8%. Moreover, stateful coordination is the dominant failure point across systems, while success decreases further on workflows requiring greater knowledge
retrieval and mid-
speech corrections. Overall, APEX-Voice is the first
benchmark for evaluating whether voice
agents can translate conversational competence into dependable professional work.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CV
Just MLPs: Efficient Visual State Reconstruction for Multimodal Language Models
Jingdi lei, Junxian Li, Di Zhang et al. (+3 more)
Long visual
token sequences often account for a substantial fraction of the computational overhead in
multimodal large
language models~(MLLMs). Existing approaches reduce this cost by
pruning redundant visual
tokens, but permanently discard visual evidence that may become useful in subsequent layers. We instead ask whether all visual
tokens can be preserved while reducing the cost of repeatedly evolving the representations through the
Transformer. To answer this question, we perform low-rank interventions on visual-to-text information flow. We find that, after visual-to-text
attention is blocked, restoring only a few directions recovers most of the lost accuracy, suggesting the relevant visual influence is concentrated in a low-dimensional subspace. We further observe strong predictability in layer-specific visual states: lightweight MLPs approximate them with high cosine similarity and low reconstruction error. Motivated by these findings, we propose $\delta$-
Vision, which replaces repeated
Transformer evolution of visual
tokens with lightweight low-rank adapters that construct layer-wise visual
memories while preserving all visual
tokens for text
retrieval. Across image and
video benchmarks, $\delta$-
Vision achieves higher accuracy than visual
token pruning baselines at comparable or lower computation, while delivering competitive inference efficiency without discarding visual
tokens.
Read the full abstract ▾
All authors Jingdi lei, Junxian Li, Di Zhang, Zhanqiu Zhang, Yiwen Guo, Soujanya Poria
Artificial Intelligence
Action-Space Shaping for LLM Agents: Measuring and Mitigating Tool-Schema Bias
Yinhong Liu, Zhili Tan, Zilin Wang et al. (+1 more)
Large
Language Models (LLMs) have shown strong performance on tool-use agentic tasks when given a fixed tool schema. Yet a tool schema is not the action space of an
agent; it is merely one interface representation of it. The same executable action can be exposed through many different, functionally equivalent tool definitions, and an
agent that has truly learned a task should behave consistently across them. We show that current
agents often do not, a phenomenon we term schema bias. To study this systematically, we introduce an executable transformation framework that rewrites a native tool schema using nine operators, including merging and splitting tools, altering how a single tool is expressed, and distributing one action across several dependent calls. The tasks, executable actions, and reachable states remain fixed, so any change in success is attributable to the interface alone. Evaluating eleven LLMs, including two closed models, on up to 32 schema variants, we ask how large schema bias is, how it manifests, whether the difficulty of a schema variant can be predicted without a full evaluation, and whether training removes it. We find that schema bias is substantial even for the newest models: success rates range from complete failure to 97% depending solely on the schema. To reliably estimate schema difficulty, it requires running a small sample of the target queries. Training repairs a schema variant only when that variant appears in the training data.
Read the full abstract ▾
All authors Yinhong Liu, Zhili Tan, Zilin Wang, Zhijiang Guo
Artificial Intelligence · Cross-listed from cs.RO
RoboFL: Federated Expert Assembly for World Action Models
Rongyu Zhang, Ruizhi Fan, Yunfan Lou et al. (+8 more)
Vision-
language-action and world-action models are increasingly popular, yet remain bottlenecked by physical interaction data that is scarce, institutionally siloed, and task-heterogeneous. A natural
federated solution is to let each client adapt a shared foundation model through parameter-efficient fine-tuning, avoiding the exchange of full-model updates. However, federating these adapters is nontrivial, as naive aggregation can entangle incompatible updates, while incorporating MoE-style routing into
federated aggregation may dilute specialization and destabilize expert selection. We present RoboFL, which instantiates MoSAIC (Mixture of Slotted Adapters) for
federated world-action learning. MoSAIC directly installs locally trained LoRA adapters as the expert branches of a server MoE. Server-side routers learn
token assignments over these prior-informed branches while jointly refining routing and expert parameters. Foresight-to-Action Routing
Distillation (FARD) aligns routing across the model's three paths, while Path-Consensus Expert Aggregation (PCEA) converts complete expert updates into a compact global adapter for personalized redistribution. Experiments on RoboTwin 2.0, RLBench, and a real-world Franka
robot arm show the superiority of RoboFL with structured expert assembly, as it outperforms centralized PEFT InternVLA-A1 by 12.23% on the Franka arm, while reducing per-round client communication by up to 86.81% relative to MoE-based
federated VLA baselines.
Read the full abstract ▾
All authors Rongyu Zhang, Ruizhi Fan, Yunfan Lou, Hengyu Fang, Shenli Zheng, Chenrui Wu, Yili Jin, Li Du, Dan Wang, Yuan Du, Shanghang Zhang
Artificial Intelligence · Cross-listed from cs.CL
Semantic Uncertainty Quantification Needs Factual Equivalence
Joseph Hoche, Quentin Guimard and Gianni Franchi
Semantic uncertainty quantification for large
language models rests on a common template: sample several answers, measure how much they agree, and treat disagreement as uncertainty. We first formalize this template as two separate roles: an operator that compares two answers, and an aggregator that combines all pairwise comparisons into a scalar. Existing methods differ almost entirely in how they aggregate, while taking the operator off the shelf, typically an NLI model or a generic sentence encoder. We show that this reliance on off-the-shelf operators is the primary bottleneck of semantic UQ: they do not accurately measure factual equivalence of multiple answers to the same question. We resolve this with a deliberately simple recipe: a single encoder trained contrastively to isolate the targeted fact, utilizing synthetic data generated by an LLM and
dataset both disjoint from all evaluation settings. Integrating the resulting operator into existing methods improves performance on 120 of 126 evaluation settings (95%) spanning 18 model
dataset combinations across
language and
vision-
language models. The best variant reaches 0.76 mean AUROC against 0.68 for the strongest baseline, while replacing the quadratic cross-encoder comparisons of entailment-based operators with one encoder pass per answer. The uniformity of the improvement supports the view that the operator, not the aggregator, is the limiting factor. The same operator also improves single generation
token-level
estimators: the norm it assigns to each
token measures how much that
token bears on the answer, and reweighting
token log-likelihoods accordingly sharpens the estimate.
Read the full abstract ▾
Artificial Intelligence
Safe Greenhouse Climate Control Using Lagrangian-Constrained PPO with Kolmogorov-Arnold Networks
Hangzun Liu, Yuling Fan, Fang Tian et al. (+3 more)
Greenhouse
climate control balances economic return with maintaining temperature, humidity and CO2 within crop-adapted growth ranges. Conventional reinforcement learning (RL) greenhouse controllers use fixed reward penalties to limit
climate constraint violations, yet such heuristic penalties cannot explicitly constrain long-term cumulative violations. Poorly tuned weights either lead to overly conservative policies and lower yields, or fail to suppress persistent
climate deviations that harm photosynthesis and induce crop diseases. To address this issue, we formulate greenhouse
climate regulation as a Constrained Markov Decision Process (CMDP) and use a Lagrangian safe RL framework RCPO-PPO to separate economic optimization and cumulative safety constraints, enabling adaptive penalty adjustment without manual tuning. To handle strong nonlinear, time-varying coupling between greenhouse microclimate and crop growth, Kolmogorov-Arnold Networks (KANs) replace Multi-Layer Perceptrons (MLPs) as policy and value approximators for improved nonlinear representation. Sinusoidal cyclic time features are embedded in observations to capture diurnal environmental periodicity. Simulations use a classic winter lettuce greenhouse model driven by 40-day real weather disturbances. Compared with vanilla penalty-based PPO, our method cuts cumulative
climate violations by 18.65% and raises lettuce economic profit by 2.91%, keeping violations stable near the safety threshold. This decoupled CMDP optimization with KAN-based policy representation mitigates long-term
climate risks and boosts planting profits, offering a constraint-aware control strategy for precision greenhouse cultivation.
Read the full abstract ▾
All authors Hangzun Liu, Yuling Fan, Fang Tian, Zhilong Bie, Zaiwen Feng, Yongliang Qiao
Artificial Intelligence · Cross-listed from cs.LG
Cyclostationary Phase Conditioning for Medical Time Series Diffusion
Samuel Ruiperez-Campillo, Michele Copetti, Jorge da Silva Goncalves et al. (+3 more)
Many physiological time series, such as cardiac and
brain recordings, exhibit cyclostationarity: their statistics vary periodically with an underlying cycle phase. Corruption from motion, poor contact, and physiological interference obscures morphology needed for diagnosis, making signal restoration essential. Existing
diffusion approaches condition on corrupted observations alone and must learn cyclic structure implicitly. We instead propose two inductive biases which encode cyclostationarity: a shift-covariant wavelet representation and dense per-sample phase conditioning inferred from the corrupted input. We further introduce a training-free cyclostationarity index that quantifies phase structure and predicts when phase conditioning will help. Finally, we propose antithetic coupling of reverse trajectories to reduce sampling variance while achieving comparable performance with fivefold fewer network evaluations. Across modalities, our results show that explicitly encoding measurable cyclic structure improves physiological time-series restoration.
Read the full abstract ▾
All authors Samuel Ruiperez-Campillo, Michele Copetti, Jorge da Silva Goncalves, Sonia Laguna, Thomas Hofmann, Julia E. Vogt
Artificial Intelligence · Cross-listed from cs.CR
JevVibe: Efficient Classification-Guided Secure Code Generation
Arshak Rezvani, Sasha Behrouzi and Ahmad-Reza Sadeghi
Large
language models can generate functionally correct code that still contains security weaknesses, motivating repair pipelines that first diagnose a weakness type before deciding how to fix it. The Common Weakness Enumeration (CWE) provides a standardized vocabulary for such diagnoses, but asking an autoregressive
language model to generate a CWE label and extracting it from the response raises questions about output validity, speed, and cost, as well as accuracy. We evaluate Jev, a decision model that instead selects directly from a declared set of candidates and returns a probability for each, against six open-weight autoregressive models and a frontier proprietary model, GPT-5.6-Sol, on a controlled 50-way CWE classification task over 1,916 CyberSecEval
benchmark examples. Jev outperforms all six open-weight baselines on every classification and ranking metric, while its comparison with GPT-5.6-Sol depends on the metric: GPT-5.6-Sol achieves higher Top-1 accuracy and Macro-F1, whereas Jev achieves higher Top-3 and Top-5 accuracy and a nearly identical MRR, at $6.27\times$ lower median API latency and $55.9\times$ lower estimated API cost. We further build JevVibe, a diagnosis-guided repair
agent that uses predicted CWE labels to repair code generated by Qwen2.5-Coder-32B-Instruct. With Jev providing the diagnosis, the
agent increases the detector-measured security pass rate from 63.5% before repair to 70.7%, compared with 66.1% for LLM-guided repair. These results show that JevVibe is effective at improving the security of generated code, with Jev providing reliable and efficient CWE classification.
Read the full abstract ▾
Artificial Intelligence
ProofLoom: Proof-Obligation-Driven Theory Construction for Autoformalizing Research-Level Stochastic Optimization
Feiming Wang, Daibo Li and Kun Yuan
Formalizing research-level
stochastic optimization in Lean requires both an
algorithm model and domain theory connecting foundational libraries to convergence proofs. Revising a model to restore provability can change the mathematical claim. We introduce ProofLoom, a fully automated LLM-
agent system for Proof-Obligation-Driven Theory Construction. Given a published
algorithm, target
theorem, and source proof, ProofLoom autonomously constructs the Lean model and supporting theory. Open proof obligations drive the development of definitions, interfaces, lemmas, and proof plans. Signature contracts record evidence and obligations for model revisions; an independent Judge rejects unsupported assumptions and weakened conclusions. Planner expands the published argument into intermediate claims, and Audit checks whether the Lean proof follows it. Across tasks, SOptLib accumulates verified mathematics and construction experience: reusable results are extracted, generalized, and verified, while modeling decisions and failed proof routes are recorded. Later tasks retrieve these results and records and contribute new developments, forming a cycle of construction, accumulation, and reuse. On fifteen textbook and research-paper tasks, ProofLoom obtains mean human ratings of 6.3/7 and 6.4/7, compared with 4.9/7 and 5.0/7 for the strongest of six baselines. Across 33 developments, it produces 490,693 lines of
algorithm-local Lean code with no sorry. The formalizations also expose 28 incorrect formulas, proof gaps, and
algorithm-analysis mismatches in published sources across 22 developments, each with checked evidence.
Read the full abstract ▾
Artificial Intelligence
AX is the New AEO
Ido Finder, Assaf Elovic, Gad Shalev et al. (+1 more)
In 2023, AI models answered from training data and hallucinated when it ran out, and businesses were told to seed that knowledge. Models' training knowledge has since given way to live web search, and the advice followed it there: answer-engine optimization, or AEO, now tells businesses to scatter breadcrumbs across forum threads, listicles, and off-site citations, so AI engines are likelier to surface and recommend them. But being surfaced is no longer enough: an
agent opens the results and reads them before deciding, and one buyer question sends it through several rounds of search and fetch. What decides the outcome at this drill-down step is whether the
agent can fetch and read the business's own site:
agent experience (AX). We argue that AX is the new AEO. We run 37,927
agent journeys, each a buyer question about a business, across four independent harnesses over 1,056 real businesses, matched on fame, prior model knowledge, and two AEO proxies, then split based on their AX level. Only 7-10% of the finished answer comes from the model's training knowledge, whether or not the site is readable.
Agent-ready businesses have answers built from their own pages 78% of the time against 56% and are clearly recommended 1.9x more often, while a grounded answer about a not-
agent-ready business costs the
agent 64% more on average. Holding business, harness, and question fixed, answers built from the site are 41% more accurate on average. The dominant failure is not fabrication but omission: web-built answers are 3.7x more likely to contain none of the facts the buyer asked for. Baselines differ sharply across the four harnesses, with clear-recommendation rates varying sevenfold from stack to stack, yet the recommendation gap holds in every one. In the agentic web era, being readable beats being talked about, and improving a site's AX is the strongest lever a business has.
Read the full abstract ▾
All authors Ido Finder, Assaf Elovic, Gad Shalev, Liad Yosef
Artificial Intelligence
VD-DeepStack: Bridging Visual Comparison and Language Reasoning for Few-Shot Anomaly Detection
Mengyang Zhao, Zhuolin He, Haiyang Yu et al. (+9 more)
Few-shot visual anomaly detection is fundamentally a visual comparison task, requiring fine-grained inspection of a query against normal references. Many recent methods based on large
vision-
language models (LVLMs) emphasize comparative
reasoning through
language chain-of-thought. Yet discrete, abstract descriptions may underrepresent dense, fine-grained visual differences, leaving a gap between visual comparison and its expression in
language. To address this gap, we propose Visual Difference DeepStack (VD-DeepStack), which explicitly conditions
language reasoning on query-reference visual differences. Specifically, we fuse DINO features with the LVLM visual hierarchy to strengthen fine-grained representations, then construct dense difference evidence from residuals between query features and softly matched reference features. The difference-evidence path injects spatially weighted difference vectors into query-image states at multiple decoder depths, while an auxiliary visual-context path provides fine-grained appearance information to support their interpretation. Experiments on 4 industrial and 2 medical anomaly
benchmarks demonstrate substantial improvements in few-shot anomaly detection over baselines relying on textual comparative
reasoning. These results support mitigating the visual comparison-
reasoning gap through the joint design of comparison representations and their integration into the decoder. Code will be released upon acceptance.
Read the full abstract ▾
All authors Mengyang Zhao, Zhuolin He, Haiyang Yu, Yuxuan Liang, Yifang Xu, Yuchuan Wu, Xiaolei Chen, Zhengtao Yao, Fan Shi, Yang Liu, Bin Li, Xiangyang Xue