Artificial Intelligence · Cross-listed from cs.CL
Fair Fact-Checking: Closing the Cross-Lingual Gap in LLM Factual Judgement with RoSh
Muhammad Ahmad, Fatemeh Seyedin, Adrian Weller et al. (+2 more)
Misinformation on social media remains a critical problem, and more and more people settle it by asking a
language model instead of a fact checker. Whether models judge such claims reliably is debated; whether they judge them equally well in every
language people ask in has gone almost unasked. We test eight models from five families, 3B to 70B, on 1,500 encyclopedic factual claims that exist in identical form in eight
languages. English is judged better than every other
language on every model, and the gap is widest on the smallest ones, where Llama-3B on Arabic is no better than guessing. Existing remedies retrain on more
multilingual data or fit an unconstrained map between
language representations, and neither asks whether the model already holds the answer and simply fails to say it. It largely does: a linear probe recovers the truth from the very activations the model fails to express. We propose RoSh, a per-
language shift and rotation of the residual stream, computed in closed form at three layers, with no training and no weight modified. It improves every model and closes 75% of the gap on average, helping most where the model was worst: Arabic on Llama-3B goes from chance to nearly the English level, and a fifth fewer of the claims answered correctly in English are lost in translation. What remains is no longer a read-out failure: afterwards the head recovers as much of what is encoded outside English as it does in English. An unconstrained map fitted on the same pairs falls below the untouched baseline, so the orthogonality constraint is doing the work, and every model clears a scrambled-correspondence control and ten further controls. On the two
benchmarks of the closest inference-time method, latent-space intervention, run with its own data and metric code, RoSh's gains are five to thirteen times larger.
Read the full abstract ▾
All authors Muhammad Ahmad, Fatemeh Seyedin, Adrian Weller, Dongwon Lee, Mahmoudreza Babaei
Artificial Intelligence · Cross-listed from cs.LG
Learning What to Recall: Adaptive Multi-Cue Episodic Memory for World Models
Beomsu Kim, Chieh-Hsin Lai, Bac Nguyen et al. (+3 more)
World models predict future observations from current experience and actions, yet prediction can depend on observations seen far in the past. Episodic
memory preserves past observations for later recall; however, as
memory accumulates, it raises a fundamental question: which
memories are useful for the current prediction, and which available
retrieval cues should be trusted to find them? This is challenging because fixed criteria based on recency, pose overlap, or visual similarity can be unreliable across environments and queries. We propose Future-Aware Recall (FAR), a framework that learns episodic recall from future-aware predictive supervision and adaptive multi-cue scoring. During training, FAR measures predictive utility by the conditional log-likelihood of the realized future given recalled context, approximated by negative
diffusion prediction loss, and uses it to train a retriever that remains future-blind at inference. The retriever learns cue-specific relevance and automatically determines which available
retrieval cues, such as time, pose,
vision, and audio, to trust for each query when selecting
memories. Across three complementary settings, FAR outperforms hand-designed recall even with the same
retrieval cues, automatically adapts which available cues to trust, and recalls the right history as the world changes. Together, these results establish FAR as a flexible, principled approach to episodic
memory access in world models.
Read the full abstract ▾
All authors Beomsu Kim, Chieh-Hsin Lai, Bac Nguyen, Amir Bar, Jong Chul Ye, Yuki Mitsufuji
Artificial Intelligence · Cross-listed from cs.CV
Evidence-Aligned Multimodal On-Policy Self-Distillation for Fine-Grained Visual Understanding
Nanxing Hu, Qiwei Yan, Jinchao Zhang et al. (+1 more)
Fine-grained visual understanding requires models to recognize small details within complex images.
Multimodal on-policy self-
distillation (OPSD) addresses this challenge by using a teacher conditioned on evidence-centered crops to supervise a student conditioned on original images along student-generated trajectories. Ideally, teacher corrections, the distributional changes from the student toward the privileged teacher, should be driven by task-relevant visual evidence. However, the designs that make the teacher effective also introduce other interference. Using a lagged or frozen teacher improves training stability but introduces a model-state gap from the evolving student, while cropping enhances task-relevant evidence but also loses the visual context. These two sources of interference make the teacher corrections not purely rely on the visual evidence. We introduce Evidence-Aligned
multimodal on-policy self-
Distillation (EAD), which retains the crop-conditioned teacher as the target but constructs a separate evidence reference for weighting the corrections. To exclude the effect of lagged model-state from this reference, EAD measures prediction changes using the current student. To avoid crop-induced context changes, EAD masks the evidence region in the original image while preserving the other visual context. The change from the student's masked-image prediction to its original-image prediction provides a controlled reference for the direction in which the visual evidence shifts the student's prediction. EAD weights each teacher correction by its cosine
alignment with the reference, i.e., retaining aligned corrections and downweighting the rest. Retaining only 6% of the supervision mass of dense OPSD, EAD consistently outperforms previous state-of-the-art methods.
Read the full abstract ▾
All authors Nanxing Hu, Qiwei Yan, Jinchao Zhang, Guoliang Kang
Artificial Intelligence · Cross-listed from cs.SE
Codoku: Renewable Program-Reasoning Challenges for Frontier Coding Agents
Cong Li, Hao Sun, Zenan Li et al. (+1 more)
Existing program-
reasoning benchmarks ask large
language models to predict a program's behavior on a given input. Coding
agents break two assumptions on which these
benchmarks rest: an
agent can recover the answer by executing the program instead of
reasoning about it, and fixed task sets drawn from existing programs are increasingly exposed to contamination, yet costly to renew. We introduce Codoku (code sudoku), a renewable
benchmark in which a solver fills typed
cells in a partial program to satisfy global static and dynamic constraints, such as a prescribed control-flow graph and execution path. Because a partial program cannot be executed and valid fillings are sparse in an exponentially large space of interdependent choices, neither tool use nor enumeration can substitute for program
reasoning. Puzzles are synthesized from scratch via semantic reification, so fresh puzzles of controllable complexity can be generated on demand, each with a witness that guarantees solvability. We evaluate five frontier models on 300 puzzles through a coding
agent free to use any tool within a fixed budget. Small puzzles already challenge open-weight models, whereas even proprietary models solve only about half of the large ones. Codoku thus offers a renewable testbed for program
reasoning that can keep pace with rapidly improving coding
agents. GitHub: https://github.com/connglli/Codoku.
Read the full abstract ▾
All authors Cong Li, Hao Sun, Zenan Li, Zhendong Su
Artificial Intelligence
A General Harness for Protein Foundation Model Fitness Prediction
Yang Tan, Qijia Tian, Gangyu Sun et al. (+5 more)
Accurate fitness prediction is central to
protein engineering and understanding sequence-function relationships. With advances in deep learning,
protein foundation models (PFMs) have become widely used for this task. Recent analyses, however, show that these models share preferences reflecting their training corpora, while unreliable inputs can further distort fitness predictions. Family-specific evolutionary evidence and structural context can help address these limitations by providing complementary constraints on model scores, motivating VenusREM-Harness (VRH), a general, model-agnostic, training-free
Retrieval-Enhanced Mutation harness. It fuses frozen model scores with multiple sequence
alignment (MSA) evidence according to model uncertainty, then applies gated background correction and score shrinkage based on structural confidence and solvent exposure. Across 1,211 assays and 3.1 million measured variants from ProteinGym, VenusMutHub, and the newly curated viral
benchmark VenusViroHub, all 71 configurations improve Spearman correlation on all 3
benchmarks by 0.073 on average, with broad gains across 5 metrics. Extended analyses relate
retrieval gains to model-MSA preference differences, assess domain-level gains and immune-escape cases, and quantify computational speedups. Built with VRH, VenusREM2 is the first to rank highest in all function, taxon, MSA-depth, and mutation-depth categories, with a ProteinGym Average Spearman of 0.556, 0.038 above the prior best.
Read the full abstract ▾
All authors Yang Tan, Qijia Tian, Gangyu Sun, Bozitao Zhong, Mingchen Li, Yuanxi Yu, Nanqing Dong, Liang Hong
Artificial Intelligence
OmniTide: Co-Designing Algorithms and Systems for Efficient On-Device Omni-LLM Streaming
Zongshang Shen, Wangsong Yin, Daliang Xu et al. (+2 more)
On-device streaming omni-modal inference safeguards user
privacy and eliminates prohibitive per-
token API costs, but faces a critical bottleneck: the continuous influx of
multimodal data rapidly exhausts constrained
memory and compute budgets via monotonic KV cache growth. Existing sparse
attention methods fall short, either incurring prohibitive online estimation latency or destroying interleaved cross-modal context, while failing to resolve physical
memory fragmentation. We present OmniTide, the first
algorithm-system co-design tailored for efficient on-device streaming omni-modal inference. Driven by the observation of modality-aware structural sparsity, OmniTide adopts a unit-based abstraction with two components: (1) At the
algorithm level, OmniPick logically retains critical
multimodal context based on unit boundaries and modality importance to preserve task accuracy; (2) At the system level, OmniPage physically partitions the cache by retention likelihood and dynamically compacts surviving sparse
tokens, minimizing both
memory fragmentation and data-movement overhead. Extensive evaluations across three streaming
benchmarks and two consumer-device architectures show that OmniTide achieves up to $12.72\times$ kernel speedups and $2.40\times$ lower stream-loop latency. On StreamingBench, it improves accuracy by up to 18.0 percentage points over sliding-window baselines at comparable session cost. OmniPage further reduces the physical KV span by up to 26.7% relative to native logical eviction, unlocking real-time, infinite-context streaming on edge devices.
Read the full abstract ▾
All authors Zongshang Shen, Wangsong Yin, Daliang Xu, Mengwei Xu, Xuanzhe Liu
Artificial Intelligence
Beyond Skill Evolution: Self-Evolving Context Management Policies for Long-Horizon Agent Harnesses
Weiyuan Li, Jinghan Xu, Aili Chen et al. (+4 more)
Harness
evolution improves LLM
agents by learning from execution trajectories, but existing experience- and skill-based methods are less effective on long-horizon tasks. As interactions grow, useful evidence can be buried by redundant or outdated context, making context management itself a key bottleneck. We introduce ContextEvo, a framework that learns a context policy from long-horizon trajectories. ContextEvo reconstructs the model-visible context at key decision points, identifies context-related failures, and applies targeted policy updates. Starting from the open-source Pi-
agent harness, ContextEvo improves performance across three long-horizon task
benchmarks, achieving results comparable to or better than several prominent
agent harnesses, including Codex, OpenCode, and OpenClaw. Additional analyses show that fixed or locally evolved context strategies can fall short under long-horizon information pressure, while our methods adapt to the information demands of each environment.
Read the full abstract ▾
All authors Weiyuan Li, Jinghan Xu, Aili Chen, Xintao Wang, Shuang Liang, Jiaqing Liang, Deqing Yang
Artificial Intelligence · Cross-listed from cs.SD
SEmoEdit: Probing and Harnessing the Editability of Pre-trained Speech Flows
Tianxin Xie, Pengfei Zhang, Kai Jiang et al. (+2 more)
Existing training-based
speech emotion editing methods often require substantial task-specific training and can be unstable. This motivates us to investigate whether the pretrained generative dynamics of large-scale text-to-
speech (TTS) models can be directly manipulated for training-free
emotion editing. To answer this question, we probe the editability of pretrained flow-matching and hybrid TTS models by constructing a controlled test set and systematically diagnosing editing effects along the generative trajectory. Our analysis reveals that pretrained TTS models are substantially editable in
emotion, but such editability is architecture- and trajectory-dependent and can be disrupted by early flow-matching steps, while cross-speaker
emotion transport carries additional acoustic attributes beyond
emotion. To address these limitations, we propose SEmoEdit, the first training-free framework that formulates
emotion editing as dynamic velocity transport between source and target
emotions, enabling robust, flow-based
speech emotion editing directly within pretrained TTS models. SEmoEdit unifies three core operations:
emotion replacement,
emotion erasure, and continuous
emotion interpolation, requiring neither parameter updates nor task-specific optimization. To systematically evaluate these capabilities, we introduce SEmoEditBench, a
dataset comprising 600 editing cases, and conduct extensive experiments across state-of-the-art (SOTA) models and backbones. Our results show that SEmoEdit is highly effective and broadly applicable, outperforming existing training-based and activation-steering methods. Ultimately, this work reveals that pretrained
speech flows possess rich, latent
emotion-editing capabilities, providing useful guidance for real applications. Code,
benchmark, and Audio samples are available at https://github.com/imxtx/SEmoEdit.
Read the full abstract ▾
All authors Tianxin Xie, Pengfei Zhang, Kai Jiang, Zelin Zhao, Li Liu
Artificial Intelligence · Cross-listed from cs.DC
Nereus: Adaptive Parallelism for LLM Post-Training
Songlin Jiang, Tuo Shi, Sitong Zhang et al. (+3 more)
Reinforcement learning (RL) post-training for large
language models (LLMs) coordinates multiple models across generation, inference, and training on GPU clusters. Several factors may change during a run, including resource availability, sequence length,
memory pressure, and stage bottlenecks. As a consequence, an execution plan that was initially suitable can then become slow or even infeasible over time. However, adapting a job whose models share GPUs entails significant challenges: deciding whether a new plan is worth the transition cost, reusing the job's distributed state, and coordinating GPU transfers across models and stages. Nereus targets these challenges as a cost-aware runtime that adapts RL post-training jobs into efficient execution plans. Its low-overhead controller selects a
memory-feasible global plan and admits the transition using a cost model calibrated against the running job. To estimate and execute a transition, Nereus represents the distributed state of each replica of a model-stage (one model in one stage) as an Elastic Model Unit. It then employs a global transition graph to order the transformations and GPU transfers of these units. In a trace built from real data, online TP/PP adaptation reduces average step latency by 27.7% relative to the initial fixed TP/PP layout with DP scaling. In a 1,000-step run reaching 1,024 GPUs, six transitions consume 0.079% of total run time. Nereus improves end-to-end 8B PPO throughput by 2.14–7.27$\times$ over OpenRLHF and by 1.10–1.47$\times$ over Verl across diverse clusters.
Read the full abstract ▾
All authors Songlin Jiang, Tuo Shi, Sitong Zhang, Zeke Wang, Mario Di Francesco, Bo Zhao
Artificial Intelligence
MechReasoner: A Simulator and Benchmark for Mechanistic Reasoning in Qualitative Physics
Danilo Gusicuma and André Freitas
This work introduces MechReasoner, a mechanistic qualitative simulator grounded in confluence-based qualitative physics, together with a
benchmark for mechanistic inference. Current large
language models (LLMs) generate fluent mechanistic descriptions that do not reliably follow from underlying structural and causal constraints. The
benchmark tests whether answers preserve simulator-licensed ambiguity, quantified claims, episode-graph transition evidence, repairs, and trace-support judgments. Its 1,120 items are generated deterministically from admissible interpretation sets, component states, scenario restrictions, confluence constraints, and derivation steps across 18 catalog mechanisms and six task families. Each mechanism undergoes converter checks of structure and
topology and behavioral checks against quantitative simulations. GPT-5.5 accuracy decreases as family-specific mechanistic complexity increases, from 76.1% in the lowest-complexity bucket (B1) to 38.0% in the highest-complexity bucket (B4). The negative association remains after controls for rendered-
prompt and expected-answer length. These results show that qualitative simulators can support auditable NLP
benchmarks for mechanistic inference.
Read the full abstract ▾
Artificial Intelligence
A Persistent State for Auditable Mixture-of-Experts Routing
Abdurrahman Javat and Allan Kazakov
Mixture-of-Experts (MoE) models repeatedly route
tokens to sparse subsets of experts, but conventional routers expose no routing-specific record of how cross-layer influences accumulate. We introduce Scratchpad-Augmented Mixture-of-Experts (SA-MoE), which gives each router access to a low-dimensional persistent state that is not provided to the experts. Learned layerwise writes update this state, and their realized post-update changes exactly decompose the state-mediated contribution to any later routing margin, forming a routing ledger. Across sparsely upcycled SmolLM2- and Gemma-based models and three independent training seeds per architecture, this pathway adds less than 1% analytical forward compute and is strongly used by trained routers: local removal of its router contribution changes the selected Top-2 expert set in 87.6% and 69.9% of decisions, respectively. Relative to a matched latest-write-only control, persistent accumulation increases long-horizon future-routing accessibility by 19.4 and 12.2 percentage points, with positive effects in every seed. More than 90% of absolute ledger contribution comes from non-recent writes in both families, and full-forward suppression of ledger-selected writes changes later routing and output distributions. The ledger is an exact provenance object for the persistent-state pathway, not a complete causal explanation of routing. Sensitivity-aware scores better predict full-forward intervention effects, and post-hoc methods recover related cross-layer attribution without architectural modification. SA-MoE instead makes one routing-specific computational history explicit and directly inspectable within the model's natural forward computation.
Read the full abstract ▾
Artificial Intelligence
LLMs for Executable Multi-Agent System Specification Generation
Andreas Kouvaras, Periklis Mantenoglou and Alexander Artikis
MAS specifications express the effects of the actions of the
agents and their environment, as well as other temporal phenomena, such as the intervals during which an
agent may perform an action. The specification of a MAS should also be executable in order to allow for run-time monitoring. Constructing the specification of a MAS requires formal
language expertise, while machine learning techniques depend on labelled data which are rarely available. To address these issues, we propose `genRTEC', a method that leverages pre-trained Large
Language Models (LLMs) to generate executable MAS specifications, in the
language of the `Run-Time Event Calculus' (RTEC), from natural
language descriptions. genRTEC constructs MAS specifications with complex hierarchical and cyclic dependencies based only on short natural
language descriptions of the concepts involved. We present an extensive empirical evaluation of genRTEC, spanning various MAS specifications, including both a qualitative and a quantitative assessment. Our results demonstrate that genRTEC constructs executable MAS specifications of high predictive accuracy without compromising
reasoning efficiency.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.RO
Efficient World Action Model Inference with Adaptive Intermediate States
Zhinnan Liu, Haozhi Han, Ruge Zhang et al. (+8 more)
World Action Models (WAMs) enable future-aware control by jointly modeling actions and environment dynamics. However, iterative
diffusion or flow inference incurs substantial denoising latency. Prior inference state offers a natural opportunity for acceleration, yet changing planning contexts, observations, and intermediate representations can quickly render retained state stale. Preserving useful computation therefore requires adapting inference state rather than reusing it as-is. To this end, we present $\mathrm{WAM}{\scriptstyle\mathrm{ACHINE}}$, a training-free framework that accelerates WAM inference by preserving and adapting inference state for efficient and accurate continuation as the control loop evolves. Across closed-loop replans, Trajectory Remapping remaps replan state from the preceding replan to initialize the next replan, reducing redundant trajectory generation. Across denoising steps, Observation Rebinding performs anticipatory inference during action execution and rebinds retained denoising state to the real observation for continuation when consistency checks pass, reducing latency exposed to the control loop. Across
Transformer layers, Residual Rescaling selectively rescales retained layer state and refreshes it through full computation of the middle layers when probe checks fail, reducing repeated
Transformer computation. Evaluations of three representative WAM architectures on LIBERO and RoboTwin 2.0 show that $\mathrm{WAM}{\scriptstyle\mathrm{ACHINE}}$ achieves 1.47-3.05$\times$ speedups in observation-to-action latency and 2.23-3.27$\times$ speedups in GPU inference time per replan, while preserving 96.69-99.54% of native WAM task success.
Read the full abstract ▾
All authors Zhinnan Liu, Haozhi Han, Ruge Zhang, Teng Ma, Tao Ma, Zheng Liu, Yifeng Chen, Yunquan Zhang, Ting Cao, Yunxin Liu, Kun Li
Artificial Intelligence
After the Fix: Transfer of Corrected Agent Experience
Yanfei Zhang and Xu Lin
Does repairing an episode make its experience a better
memory for the next task? We transfer the same failed source before and after accepted repair to a fixed target, alongside independent execution. Our 3,300 runs cover 100 ThinkingBox pairs and the same 100 APEX pairs with and without source-state inheritance, under eleven conditions. ThinkingBox's Full/Skill/Hybrid correction gains are 44/29/32 percentage points, with corrected performance 25/22/18 points above independence; inference weakens at the task-family level. Yet 12 of Full's 15-point larger correction gap over Skill come from worse uncorrected performance, not better corrected
memory. Moreover, 22 of Full's 46 upward transitions restore observed baseline success. Neither APEX regime establishes comparable aggregate correction benefits. Action evidence connects workflow gains with reusable obligations and convention conflicts with source-local choices. Text APEX's accepted execution reaches 52% versus its summary's 40%, without robust global/group-level superiority or an estab- lished advantage over independence. Smaller handoffs reduce input but increase calls. The value of repairing experience is therefore distinct from the value of reusing it:
memory updates require both a previous-version reference and a fresh-start reference.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CV
Summarize Before Grounding: Query-Guided Chunk Condensation for Long-Video Temporal Grounding
Nanxing Hu, Xiaoyue Duan, Qiwei Yan et al. (+3 more)
Video temporal grounding (VTG) aims to localize the
video interval corresponding to a
language query. Recent large
vision-
language models (LVLMs) show great potential in solving such a multi-modal
reasoning task. However, long
videos often contain large amounts of redundant information that disturbs LVLMs to mine query-relevant evidence. Instead of dense frame sampling which incurs prohibitive training
memory, previous reinforcement learning with verifiable rewards (RLVR) works typically utilize sparse sampling, which makes training feasible but may miss critical evidence. In this paper, we propose a “summarize before grounding” framework (named “SumGround”) for long-
video temporal grounding. The key of SumGround is to perform query-guided chunk condensation to aggregate and retrieve query-relevant evidence. Specifically, we split the
video into several chunks and perform two-level chunk condensation. First, we introduce query-guided latent summaries, which is represented as KV states of query-guided
prompts, to compress redundant visual
tokens into compact query-relevant chunk summaries. Furthermore, we design an associative summary
retrieval scheme to rank and select chunk summaries that are most likely to contain the event interval. Both query-guided latent summary and associative summary
retrieval schemes are enabled by RLVR. To reduce
memory consumption, we propose a length-aware gradient gating module to selectively stop gradient back-propagated to visual
tokens. Extensive experiments demonstrate that SumGround performs favorably against previous state-of-the-art methods across multiple downstream
datasets, with remarkable gains on long
videos.
Read the full abstract ▾
All authors Nanxing Hu, Xiaoyue Duan, Qiwei Yan, Kailin Lyu, Jinchao Zhang, Guoliang Kang
Artificial Intelligence
TULIP: Targeted LLM Unlearning at Layers Identified Per-input
Yejin Kim, William F. Shen, Seokwon Jung et al. (+2 more)
Representation-level unlearning modifies the intermediate hidden states of LLMs. Although knowledge is distributed across layers, existing methods operate at a single fixed layer for the entire forget set. We ask whether such a fixed layer is sufficient for unlearning. To answer this, we design a hijacking experiment that grafts hidden states of the target model into an oracle model. The oracle cannot produce the forget answer on its own, yet it produces the answer from the grafted state. This indicates the answer is already formed at an intermediate layer and read out afterward. Moreover, the layer where formation ends varies widely across inputs. Motivated by these findings, we propose Targeted Unlearning at Layers Identified Per-input (TULIP). For each input, TULIP uses the logit lens to locate the formation-readout boundary and removes the hidden state's
alignment with the forget answer's unembedding vector at that layer. TULIP outperforms output- and representation-level baselines on TOFU, PISTOL, and WMDP across Llama, Qwen, and Zephyr models. It also remains robust to paraphrase and quantization attacks. Beyond standalone use, its per-input layer selection serves as a plug-and-play component that further improves existing methods.
Read the full abstract ▾
All authors Yejin Kim, William F. Shen, Seokwon Jung, Daeun Park, Seong Joon Oh
Artificial Intelligence · Cross-listed from cs.CV
Reinforcement Learning from Intermediate Renders for Image-to-Code Generation
Omri Kaduri, Kate Feingold, Phillip Isola et al. (+1 more)
Reinforcement learning is increasingly used to post-train
vision-
language models for image-to-code generation, such as generating SVG code from a reference image, by optimizing rewards computed from the final rendered output. However, relying on a single terminal reward provides sparse feedback that is poorly aligned with the contribution of individual
tokens. A generated program may contain operations that accurately reproduce some parts of the target image alongside others that introduce errors, yet all
tokens are trained from the same final outcome. We observe that many intermediate code prefixes are not only executable, but already produce meaningful partial renders that reflect progress toward the target. This property provides a natural source of denser supervision during generation. Based on this observation, we introduce IR4RL, an RL framework with a
token-level render-progress reward that turns changes between intermediate renders into localized feedback for the generated sequence. We evaluate our approach on Image-to-SVG and Image-to-TikZ generation. Across both tasks, our method improves over supervised fine-tuning and standard GRPO, yielding new state-of-the-art open-source models. This shows that intermediate rendering provides a simple and effective source of process supervision for RL post-training of image-to-code models.
Read the full abstract ▾
All authors Omri Kaduri, Kate Feingold, Phillip Isola, Tali Dekel
Artificial Intelligence · Cross-listed from cs.CL
In-game Toxic Detection: Bi-directional Representations with Attention Residuals
Yuanzhe Jia
In-game toxic
language has emerged as a critical concern in the gaming industry and community. While several frameworks and models for online game toxicity analysis have been proposed, detecting toxicity in player chat utterances remains a formidable challenge: stemming not only from the extremely short length of such utterances but also from the heavy reliance on game slang, abbreviations, and domain-specific jargon, which generic
language models are poorly suited to recognize. This paper presents a shared task for in-game toxic
language detection built upon real-world in-game chat data, and proposes the best-preforming model for the toxic
language slot filling: Bi-directional Representations with
Attention Residuals (BRAR). Experimental results demonstrate that BRAR effectively captures the global context and outperforms the existing baselines on slot filling.
Read the full abstract ▾
Artificial Intelligence
SpeechCritic: Learning a Diagnostic Speech Judge from Limited Human Preferences
Mingyue Huo, Shivam Mehta, Bhavin Jawade et al. (+2 more)
Human
speech conveys rich perceptual information, such as
emotion and speaker identity, yet most automatic
speech quality judges reduce it to a single naturalness score. We study diagnostic
speech judges: given two candidates, a diagnostic judge decides which is better, along which perceptual dimensions (e.g., timbre,
emotion, timing) they differ, and which audible cues support its decision. Learning such judges is challenging: expert annotation is costly, and simply prompting a frontier audio-
language model to produce labels is unreliable: our probing reveals substantial errors and unstable instruction following. We introduce SpeechCritic, which learns a diagnostic judge in a reference-conditioned cross-lingual setting from only about 300 human-labeled comparisons. Rather than replacing the frontier model, SpeechCritic calibrates it with these labels: for each dimension, it selects the acoustic measurements that agree with human judgments, maps them to A/Tie/B probabilities, and passes these to the model as non-binding hints alongside the audio. Compared with the same model labeling without hints, this raises dimension-level agreement with humans by 6.3 points and cuts the mismatch with human Tie rates by 10.4 points. We then train a 7B judge on this supervision and find that different training signals shape different judge behaviors: SFT establishes the task, OPD transfers the teacher's dimension-level strengths and weaknesses, and RL helps most on clear-cut comparisons where human raters agree. Notably, human listeners also find that RL makes rationales cite more specific, localized acoustic cues, although it never directly rewards rationale text. Finally, we show that the pipeline is
language-pair agnostic by instantiating it on both English-Japanese and English-Spanish. Together, these results demonstrate a path from limited human preferences to a diagnostic
speech judge.
Read the full abstract ▾
All authors Mingyue Huo, Shivam Mehta, Bhavin Jawade, Yinghong Lan, Haoqi Li
Artificial Intelligence · Cross-listed from cs.CV
GenNVS: Geometry-enhanced Novel View Synthesis via Disentangled 3D Prior
Yajiao Xiong, Youyu Luan, Xiaoyu Zhou et al. (+1 more)
Single-image novel view synthesis remains challenging because the underlying 3D geometry is highly ambiguous. Recent
diffusion-based approaches produce plausible results, but they often struggle to preserve the
geometric structure and spatial coherence of foreground objects. We present GenNVS, a framework for geometry-enhanced novel view synthesis via a disentangled 3D prior. Specifically, GenNVS models foreground objects and the background with 3D Gaussian Splatting and aligns them through a coarse-to-fine
geometric optimization process to form a unified 3D scene. This scene conditions a
video diffusion model through the proposed Dual-Stream Masking mechanism, which guides synthesis by jointly exploiting rendered validity masks and geometry-aware warping. Experimental results show that GenNVS performs favorably against recent methods in both visual quality and
geometric accuracy, while naturally supporting flexible scene editing.
Read the full abstract ▾
All authors Yajiao Xiong, Youyu Luan, Xiaoyu Zhou, Yongtao Wang
Artificial Intelligence
Calibrated Uncertainty for Informative Path Planning in Aquatic Environmental Monitoring
Samuel Yanes Luis, Alejandro Casado Pérez, Alejandro Mendoza Barrionuevo et al. (+3 more)
Informative Path Planning for scalar field reconstruction uses predictive uncertainty to direct sensing vehicles toward maximally informative locations. Gaussian Processes provide this signal but their stationary isotropic kernels are misspecified for non-homogeneous phenomena such as oil spills, producing miscalibrated estimates that degrade planning. We investigate whether replacing the Gaussian Process with a well-calibrated Deep Ensemble improves path planning outcomes, and whether uncertainty quality interacts with the choice of planning
algorithm. Five strategies ($\epsilon$-Greedy, Value Greedy, Uncertainty Greedy, Monte Carlo Tree Search, and Receding Horizon Orienteering) share a common Deep Ensemble backbone trained on physics-based oil spill simulations. On held-out
stochastic spill scenarios, the Deep Ensemble reduces normalised reconstruction error by $83\%$ relative to the Gaussian Process baseline. Crucially, well-calibrated uncertainty amplifies the importance of the planning strategy: the performance gap between
algorithms is negligible under miscalibrated models but becomes substantial under the ensemble, where multi-step lookahead planners outperform greedy selection by up to $32\%$ in reconstruction error and achieve IoU above $0.85$. Monte Carlo Tree Search is the recommended planner, matching Orienteering in reconstruction quality at an order-of-magnitude lower computational cost.
Read the full abstract ▾
All authors Samuel Yanes Luis, Alejandro Casado Pérez, Alejandro Mendoza Barrionuevo, Dame Seck Diop, Sergio Toral Marín, Daniel Gutiérrez Reina
Artificial Intelligence
Diffusion Subgoal Planning for Long-Horizon Offline Goal-Conditioned Reinforcement Learning
Hengrui Zhang, Yuhu Cheng, C. L. Philip Chen et al. (+1 more)
Offline goal-conditioned reinforcement learning (GCRL) learns goal-directed policies from reward-free data, but in long-horizon tasks, goal-conditioned value functions often provide unstable guidance due to sparse rewards and discounting. Hierarchical methods partially mitigate this issue via subgoal decomposition; however, high-level decision-making still relies on noise-sensitive value estimates, leading to unstable behavior in complex environments. We address this limitation by proposing
Diffusion Subgoal Planning (DSP), a
diffusion-based framework for high-level subgoal generation. DSP casts high-level planning as guided generative inference over goal-conditioned subgoals and learns both conditional and unconditional flows, enabling classifier-free guidance to introduce a goal-directed bias at inference time. By removing explicit value-based guidance from high-level planning, DSP generates reachable and goal-directed subgoals through a generative model while retaining hierarchical execution. Experiments on offline GCRL
benchmarks demonstrate that DSP outperforms prior methods on a range of
navigation and manipulation tasks, with particularly strong performance in maze environments that require multi-step subgoal planning.
Read the full abstract ▾
All authors Hengrui Zhang, Yuhu Cheng, C. L. Philip Chen, Xuesong Wang
Artificial Intelligence
Nudgeability: Reasoning Models Follow Confidence Signals Without Tracking Their Own Competence
Rohit Saxena and Utkarsh Upadhyay
Reasoning language models that can call tools must decide during inference whether to answer unaided or delegate. Any self-reflection mechanism for this must answer three questions: where the reflective signal comes from (verbal reports, output distributions, hidden states, a separate predictor), how it is presented to the model (numerical prediction, confidence
token,
prompt injection), and whether it changes the model's subsequent action. We isolate the third question. At a fixed point in otherwise identical
reasoning trajectories, we insert a single first-person sentence expressing either confidence or doubt; the model then continues
reasoning and chooses whether to answer directly or call a tool. Comparing these counterfactual continuations measures the causal effect of the reflective signal on delegation. We call this behavioral response Nudgeability and measure it along two dimensions: sensitivity, how strongly confidence and doubt change delegation rates, and targeting, whether delegation increases for problems the model cannot solve unaided and decreases for those it can. Across nine small-to-medium open-weight
reasoning models from three families (Qwen, Gemma, and GLM) and two tasks, models are consistently sensitive: doubt increases delegation and confidence decreases it, with a median confidence-to-doubt swing of 20.6 percentage points, and 53 to 70 points for the larger provider-served models. This responsiveness is poorly targeted: a median 42% of induced flips are well-targeted, only a +2 percentage-point lift over a random-selection baseline. Confidence
language is thus a strong control surface for delegation, but current models use it only weakly in accordance with their actual competence. Nudgeability offers a simple, post-training-free way to evaluate both sensitivity and targeting as endogenous self-reflection mechanisms mature.
Read the full abstract ▾
Artificial Intelligence
PersonaManifold: Revealing and Exploiting Curved Geometry in LLM Persona Representations
Rui Xu, Yinghui Xu and Libo Wu
Controlling persona in large
language models (LLMs) at inference time is important for role-playing, personalized dialogue, and social simulation. Recent methods extract persona vectors from the model's activation space and apply Euclidean operations—addition, scaling, and linear interpolation—under the linear representation hypothesis. However, these methods themselves report systematic failures: non-orthogonal trait dimensions, asymmetric ceiling and resistance effects, and significant deviations in multi-trait composition, suggesting that the linear isotropic assumption does not hold. We propose PersonaManifold, a framework that models persona representations as points on a curved, low-dimensional Riemannian submanifold in activation space. We estimate the
manifold's intrinsic geometry—local metric tensors, geodesic distances, and Ollivier-
Ricci curvature—and introduce geodesic steering, which interpolates between personas along
manifold geodesics rather than Euclidean straight lines. We also propose the Behavioral Similarity Triplet (BST)
benchmark, which automatically generates situational questions grounded in six established psychological constructs and defines persona similarity through behavioral responses rather than self-report questionnaires. Experiments on three open-source LLMs show that persona activations form a
manifold with heterogeneous
curvature, geodesic distance predicts behavioral similarity more accurately than Euclidean alternatives with independent contributions from anisotropy and
curvature, and geodesic steering produces more coherent intermediate personas on both our BST
benchmark and external evaluations, with the advantage concentrated in high-deviation regions where the
manifold deviates most from flatness.
Read the full abstract ▾
Artificial Intelligence
FlowState: Execution State as Memory for Long-Horizon LLM Agents
Minghao Li, Bangyan Li, Zifan Wang et al. (+5 more)
Long-horizon tasks require LLM
agents to continually draw on information from earlier interactions. However, retaining the full history increases context costs, while compressing it risks losing details needed later, and the relevance of historical information often becomes apparent as the task progresses. To address these challenges, we propose FlowState, which treats execution state as
memory that can be retained and revisited across requests, unifying current decision-making with the reuse of historical information. FlowState preserves semantically typed state nodes, their relations, and references to raw tool observations, separating persistent retention from on-demand access. Within a single execution loop, Incremental State Update (ISU) maintains the current state based on new inputs and feedback, while Progressive State Access (PSA) progressively reveals historical states and supporting evidence as needed during
reasoning. Together, these mechanisms enable
agents to reassess prior decisions in light of new information and guide subsequent actions. Compared with a full-context baseline using the same DeepSeek-V4-Flash model, FlowState improves the average success rate on MemoryArena and the average pass rate on $\tau^3$-Bench by 4.55 and 13.95 percentage points, respectively, while reducing total
token consumption by 43.2% and 40.6%. These results demonstrate the performance and efficiency advantages of FlowState on long-horizon tasks.
Read the full abstract ▾
All authors Minghao Li, Bangyan Li, Zifan Wang, Yulong Li, Hu Xu, Gan Zhang, Jingtong Wu, Wenqiang Xu
Artificial Intelligence
SkillRubric: Co-Evolving Actor Guidance and Evaluator Rubrics for Multimodal Agents
Bingqing Jiang, Guoxi Zhang, Jasper Wang et al. (+7 more)
Recent work incorporates reusable skills distilled from past interactions into
multimodal agent training, providing procedural guidance for long-horizon planning and tool use. However, policy optimization in these methods remains driven primarily by sparse outcome rewards, providing little supervision for intermediate decisions. Rubric-based rewards address this limitation through explicit intermediate criteria, but reliable rubrics are difficult to construct at scale and often disconnected from the procedure followed by the actor. We observe that a well-structured skill naturally specifies both how to act and what successful execution should achieve. Based on this insight, we introduce SkillRubric, which represents each skill through aligned actor-facing guidance and an evaluator-facing rubric. A
multimodal verifier evaluates skill-defined goals using screenshots and tool outputs, assigning completion and progress rewards to the responsible turns. We further introduce an alternating co-
evolution scheme that validates guidance revisions through paired rollouts under a frozen policy and rubric revisions offline under fixed guidance. Experiments across diverse
multimodal agent benchmarks demonstrate consistent performance gains, while controlled paired rollouts further show that evolved skills provide more effective guidance for planning and tool use than their preceding versions.
Read the full abstract ▾
All authors Bingqing Jiang, Guoxi Zhang, Jasper Wang, Auric Wang, Bingning Wang, Tianyi Lin, Zichao Yu, Yujin Han, Ziye Ma, Difan Zou
Artificial Intelligence
SGG-ReflAct: Sub-Goal Guided ReflAct with Structured Planning for Reliable Long-Horizon Reasoning
Jaeho Jung and Sung Hoon Jung
Recent advances in
reasoning backbones have empowered large
language model (LLM)agentstotackle complex, multi-step tasks. However, as
reasoning horizons grow, inconsistent internal beliefs induce intermediate errors that cause
agents to drift from their goals. This limitation also persists in REFLACT, which reflects only on the end-goal at each step without explicitly considering intermediate sub goals. To address this problem, we propose SGG-ReflAct (Sub-Goal Guided Re flAct), a
reasoning backbone that integrates sub-goals generated through a single path LLM planner into the reflection process. We further extend this framework to BeamSGG-ReflAct, which replaces the single-path planner with a beam search based LLM planner for structured plan exploration. We run experiments on ALF World, ScienceWorld, and Jericho with multiple LLM models. SGG-ReflAct out performs REFLACT in nearly all settings, achieving best success rate gains of 14.9 percentage points on ALFWorld and 8.0 percentage points on ScienceWorld with Llama-3.1-8B-Instruct. Our experimental analysis shows that SGG-ReflAct re duces hallucinated actions and achieves its largest gains on procedurally ordered tasks. Furthermore, experimental results with BeamSGG-ReflAct show that the backbone's effectiveness depends on plan quality: explicitly specifying the re quired operations recovers gains that plan searching alone cannot achieve. These results demonstrate that SGG-ReflAct offers a practical and highly effective rea soning backbone, enabling LLM
agents to achieve reliable performance in com plex, long-horizon tasks through easy integration.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CV
ActionLens: Diagnosing Spatial-Temporal Binding Failures in Vision-Language Models
Gueter Josmy Faure, Min-Hung Chen, Hao Ping Wang et al. (+3 more)
Video-capable
vision-
language models score above 80% on popular
benchmarks yet struggle with spatial-temporal binding: associating the right action with the right person at the right moment. We introduce ActionLens, a diagnostic
benchmark of 6,701 multiple-choice
video questions spanning five targeted diagnostics: transition detection, actor-specific identification, concurrent action binding, directed interaction
reasoning, and gaze detection. Ground-truth answers are derived deterministically from 1.58 million per-second, per-person annotations. Fourteen rounds of human quality engineering raised answer clarity from 53% to above 90% human accuracy. Across 20 VLMs, the full-set leader scores 68.8%; on the human-reviewed subset, it scores 65.9% versus 91.0% for the pooled human reference. Gaze detection remains near chance against 89.6% human accuracy. On actor disambiguation, reference-interface controls show that relational descriptions recover 5.55–13.25 points over static coordinates, confirming a substantial numeric-parsing penalty; yet visual boxes still lead every model by 1.15–6.50 points, exposing a residual unboxed actor-resolution gap. A binding-trap analysis shows models systematically select the wrong actor's action. ActionLens provides diagnostic measurements of these distinct failure modes across model families and scales for direct comparison. We release all data, code, and evaluation scripts at https://anonymous.4open.science/r/lmms-eval-2276
Read the full abstract ▾
All authors Gueter Josmy Faure, Min-Hung Chen, Hao Ping Wang, Timothée Lardy, Hung-Ting Su, Winston H. Hsu
Artificial Intelligence
Remember Before You're Asked: MemDream for Self-Probing Memory Evolution
Mingfei Lu, Mengjia Wu, Runsong Jia et al. (+2 more)
Memory is essential for enabling LLM-based
agents to maintain coherent, personalized behavior over long-horizon interactions. However, existing
memory systems share a fundamental limitation: they never proactively test their own
memory, repairing it only after real queries expose weaknesses. This reactive paradigm means every
retrieval failure corresponds to a real interaction in which the cost has already been paid. We propose MemDream, a framework that enables self-probing
memory evolution for LLM
agents. Our framework periodically enters offline dream cycles where three specialized
agents (Dreamer, Analyst, Consolidator) collaboratively probe, diagnose, and repair the
memory graph before failures occur. A policy trained via Group Relative Policy Optimization learns which repair operations produce durable
retrieval improvements, while a soft decay mechanism provides reversible forgetting driven by the same anticipatory signal. Experiments on LoCoMo and MemoryAgentBench demonstrate that MemDream improves answer F1 by 4.5 points on LoCoMo and achieves a 9.1-point higher overall score on MAB over the strongest reactive-
evolution baselines.
Read the full abstract ▾
All authors Mingfei Lu, Mengjia Wu, Runsong Jia, Zhe Luo, Yi Zhang
Artificial Intelligence
APOLO: Automatic Prompt Optimization for Ontology Learning
Huu Tan Mai, Roman Kochnev, Cuong Xuan Chu et al. (+3 more)
Ontology Learning (OL) from text has advanced with the emergence of Large
Language Models (LLMs), but it remains challenging due to the limited availability of annotated training data and the difficulty of adapting LLMs to perform OL effectively. We address this via APOLO - Automatic
Prompt Optimization for Ontology Learning, by casting OL as an explicit
prompt optimization problem over LLM modules. To obtain training data, we employ a multi-
agent system that generates text-ontology pairs from existing expert-curated ontologies. We then propose two ontology learner architectures: a greedy and an autoregressive learner, and optimize both using GEPA, a greedy evolutionary
prompt optimizer built on DSPy. Experiments on two ontologies - a biomedical (DOID) and a plant ontology (PO) show consistent improvements after optimization across nearly all model and mode combinations, with autoregressive learners achieving the largest gains. Our results demonstrate that
prompt optimization is a viable and lightweight alternative to fine-tuning for OL, and that the autoregressive formulation better captures ontological structure than the greedy approach.
Read the full abstract ▾
All authors Huu Tan Mai, Roman Kochnev, Cuong Xuan Chu, Lukas Lange, Heiko Paulheim, Daria Stepanova