Artificial Intelligence · Cross-listed from cs.CV
Learning from Multimodal Pseudo-Labels for Robust Open-Vocabulary Instance and Panoptic Segmentation
Duy Tran Thanh, Yeejin Lee and Byeongkeun Kang
This work addresses the challenge of open-vocabulary instance
segmentation (OVIS) and open-set panoptic
segmentation (OSPS), which aim to recognize both predefined and unseen object categories without exhaustive human annotations. Existing methods often suffer from noisy pseudo-masks, limited visual-textual grounding, and difficulty handling synonyms or out-of-vocabulary (OOV) words. To overcome these challenges, we propose a
multimodal framework that leverages pre-trained
vision-
language models for automatic pseudo-label generation, CLIP-guided synonym filtering, and GPT-based caption reconstruction. In our target-vocabulary-assisted pseudo-labeling setting, the framework first constructs pseudo
segmentation masks, descriptive captions, and semantically aligned synonym sets using Grounded SAM, LLaVA, and CLIP, providing
multimodal supervision without manual annotation. We then enhance visual-textual
alignment through three complementary training objectives: an extended grounding loss that incorporates visually grounded synonyms, a semantic consistency loss, and a generative caption reconstruction loss. Extensive experiments on the COCO
dataset demonstrate that the proposed method consistently outperforms previous state-of-the-art approaches under this protocol, achieving substantial improvements on both OVIS and OSPS
benchmarks.
Read the full abstract ▾
Artificial Intelligence
AgenticTwin: An Agentic LLM Framework Integrated with Digital Twin for Anomaly Detection
Touseef Hasan, Mounika Ghanta, Souvika Sarkar et al. (+1 more)
Digital twins are increasingly used to monitor and simulate the behavior of cyber-physical systems. Even with skilled operators, interpreting anomalies detected within digital twin pipelines is challenging, as the sheer complexity and volume of raw sensor data make thorough analysis difficult. Recent advances in large
language models (LLMs) offer promising capabilities for
reasoning and explanation, yet their integration into digital twin-driven anomaly analysis remains underexplored. In this work, we propose AgenticTwin, an agentic framework that integrates LLM-driven
reasoning with a digital twin-based anomaly detection pipeline. The framework grounds LLM-generated explanations in outputs from a digital twin-driven anomaly classifier and enables human operators to ask relevant natural-
language questions about the system. Beyond the framework itself, we introduce a
benchmark-oriented evaluation pipeline constructed over synthetic anomalies injected into a real-world weather sensor
dataset, enabling controlled generation of operator queries over anomaly events. We further evaluate the feasibility of deploying lightweight, open-source LLMs for practical cyber-physical environments. Experimental results demonstrate that structured
agent collaboration and knowledge-grounded
reasoning improve diagnosis quality, contextual
retrieval, and mitigation quality across diverse possible anomaly scenarios.
Read the full abstract ▾
All authors Touseef Hasan, Mounika Ghanta, Souvika Sarkar, Ujjwal Guin
Artificial Intelligence
XBridge: Entity-Grounded Latent Bridge for Heterogeneous LLM Communication
Wooseong Yang, Wei-Chieh Huang, Weizhi Zhang et al. (+3 more)
Heterogeneous multi-
agent LLM systems, where
agents are powered by different model families, can outperform homogeneous configurations by reducing redundant
reasoning patterns. Yet existing communication protocols either operate through text, discarding the sender's internal representations, or require architectural homogeneity for latent-level transfer. We identify the entity grounding problem in cross-architecture communication: cross-
attention bridges that transfer continuous representations across different LLM families suffer from rare-
token compression collapse, where entity identity is lost in the continuous bottleneck (bridge-only F1 ~30%). We propose XBRIDGE, a decode-free communication protocol that addresses this through two mechanisms. Lexical Anchor Mapping (LAM) maps the sender's original context
tokens to the receiver's vocabulary, providing discrete entity anchors. A Latent Enrichment Bridge (LEB) lets the receiver query the sender's hidden states for contextual enrichment. The entity anchors ground the bridge's contextual signals to specific entities through the receiver's own self-
attention. Across three model families (Llama, Qwen, and Mistral), seven
benchmarks, and both communication directions, XBRIDGE outperforms text-based communication on all seven tasks for each model pair while achieving 11x lower latency, and in a same-architecture setting it also exceeds a KV-sharing baseline on six of seven tasks. LEB requires only 264M trainable parameters (3.8% of the receiver), is trained on a small balanced sample set, and adds negligible inference overhead.
Read the full abstract ▾
All authors Wooseong Yang, Wei-Chieh Huang, Weizhi Zhang, Yu Wang, Philip S. Yu, Junhyun Lee
Artificial Intelligence · Cross-listed from cs.LG
GCPO: Diagnosing and Constraining Subspace Geometry in Rollout RL for LLMs
Kai Yang, Jingwei Xu, Wanyu Wang et al. (+4 more)
On-policy rollout methods such as GRPO are central to post-training of large
language models, yet they frequently suffer from training instabilities, cross-task capability degradation, and response-length inflation. Although prior work has characterized the subspace geometry of aggregate updates, the stepwise variation of this geometry and its relationship to model performance remain unclear. We introduce Principal-Subspace Overlap, a dimension-corrected measure of individual rollout updates relative to the dominant singular subspaces of pretrained weights. Despite low average overlap, transient spikes often precede performance degradation. To address this, we propose GCPO (Geometrically Constrained Policy Optimization), which applies hard bilateral orthogonal projections to constrain updates to the complementary subspaces, preventing such excursions by construction. Across mathematical
reasoning, code generation, and tool-use tasks on Qwen3-8B and GLM4-9B, GCPO consistently outperforms GRPO and recent variants, including DAPO and GSPO, improving over the base models and the strongest baseline by up to 27.69 and 2.37 points, respectively. Furthermore, GCPO preserves general capabilities, eliminates response-length inflation, and stabilizes policy
entropy. Our findings provide a new diagnostic lens and a principled design perspective for stable reinforcement learning post-training.
Read the full abstract ▾
All authors Kai Yang, Jingwei Xu, Wanyu Wang, Kai-Yuan Guo, Zhenbo Yu, Yi Wang, Yu Qiao
Artificial Intelligence · Cross-listed from cs.LG
Rubric Dropout: A Simple Way to Mitigate Reward Hacking in Rubric-Based RL
Minglai Yang, Xinyu Guo, Zhengliang Shi et al. (+8 more)
Rubric-based reinforcement learning, in which an LLM judge scores responses against
prompt-specific criteria, has become a standard way to post-train
language models on tasks without verifiable answers. The rubric, however, is only a proxy for response quality, and a policy trained against it long enough learns to exploit the difference. We train models with GRPO on medical and science rubrics and grade their responses on out-of-distribution (OOD)
benchmarks throughout training with two judges: the training judge and a stronger OOD judge. The two scores first rise together and then diverge. The policy is learning to satisfy the training judge without writing better responses, which is a form of reward hacking. To mitigate it, we propose Rubric Dropout, which applies dropout to rubric criteria: at each training step, a random subset of each rubric's criteria is removed from the reward, so the rewarded subset of criteria is resampled at each step. All responses sampled for the same
prompt are scored on the same subset, so their rewards remain comparable, and evaluation always uses the full rubric. We also introduce Block Dropout, which drops semantically similar criteria together so that a near duplicate of a dropped criterion cannot keep rewarding the same behavior. Across both
benchmarks and models from 1.7B to 8B, dropout improves the gold judge's score on OOD
benchmarks by up to 3 percent on HealthBench-Hard and 7 percent on ResearchQA, and it reduces both the gap between the two judges and over-crediting. It keeps in-domain reward close to the baseline, and dropout rates of 30-50 percent work best. Block Dropout improves the gold judge's OOD score further at the same dropout rate. Changing which criteria are rewarded at each step is a simple, inexpensive way to make rubric rewards harder to exploit.
Read the full abstract ▾
All authors Minglai Yang, Xinyu Guo, Zhengliang Shi, Utkarsh Tyagi, Mian Zhang, Razvan Dumitru, Sunjie Hou, Yunzhong He, Daniel Yue Zhang, Dan Roth, Ying Liu
Artificial Intelligence · Cross-listed from cs.LG
Low-Interaction-Rank Learning: Unifying Multiplicative Dual-Encoder Heads
Zijian Zhao and Sen Li
A multiplicative dual-encoder network computes a real-valued output for a pair of inputs as the inner product of their separate encodings. This architecture has been developed independently in operator learning, bipartite matching, contrastive
vision-
language models,
retrieval, and other areas, yet no unified theory guides the basic design decisions: how many interaction modes to represent, how to normalize the encoders, and when the architecture should be avoided. We provide such a foundation by introducing the class of functions of low interaction rank, a class whose intrinsic complexity is measured by its interaction spectrum. Within this framework, approximation error decomposes into a spectral truncation term and an encoder-realization term; sample complexity is governed by the sum of the two encoder complexities rather than their product; and a usability criterion based on spectral decay determines when the architecture can succeed. The same framework exposes a central identifiability problem: the encoders are defined only up to a linear gauge
symmetry that leaves the learned coordinates arbitrary. We show that normalization is gauge fixing and that whitening pins the interaction modes up to permutation and sign, thereby explaining the uninterpretability of contrastive dimensions and providing a constructive remedy. Experiments on synthetic kernels, operator learning, and CLIP models validate the theoretical predictions: spectral decay rates match the predicted scaling, whitening recovers the true modes, and independently trained CLIP models are related by a single rotation which, after removal by whitening, exposes interpretable concept axes. The code of this paper is provided at https://github.com/RS2002/Mul-Net .
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CL
Hybrid-Policy Self-Editing for Composable Unstructured Knowledge Editing
Tianci Liu, Zihan Dong, Tianchun Li et al. (+8 more)
Large
language models (LLMs) achieve remarkable performance across natural
language tasks, yet they are trained on static corpora and their knowledge quickly becomes outdated in a fast-changing world. This motivates knowledge editing (KE), which updates specific knowledge in an LLM without changing unrelated others. Recent works move from structured knowledge triples toward unstructured KE (UKE), where the edit is a free-form passage that may state multiple facts at once. Nonetheless, existing editors inject such a passage yet fail to use it: the edited model can recall the passage, but can neither answer atomic questions about its facts nor compose them into multi-hop
reasoning. We attribute this missing property, which we term composability, to editors' passive reliance on the fixed passage as the sole learning source. In response, we cast editing as a proactive self-
distillation from a privileged in-context state of the same model, which requires no external supervision. We further reveal that due to the novelty of the injected knowledge, the pre-edited model's own rollouts rarely cover it, which limits the effectiveness of pure on-policy
distillation. To close this gap, we propose HPSE, which builds a hybrid rollout that steps in to place missing facts onto the student's own trajectory precisely where its coverage fails, while staying on-policy elsewhere. We theoretically analyze HPSE's advantage over pure on-policy
distillation, and empirically establish its plug-and-play improvements across four LLM backbones and two KE editors under various scenarios.
Read the full abstract ▾
All authors Tianci Liu, Zihan Dong, Tianchun Li, Yi-Chung Chen, Qiming Cao, Xingchen Wang, Shiyang Wang, Zichen Miao, Linjun Zhang, Haoyu Wang, Jing Gao
Artificial Intelligence · Cross-listed from cs.LG
Is Per-Agent Policy Composition Safe? Rethinking Successor-Feature Transfer in Cooperative Multi-Agent Reinforcement Learning
Zijian Zhao and Sen Li
Many reinforcement learning systems, from fleet management to traffic signal control, must serve an objective that changes dynamically after deployment, and retraining a policy for each new objective is prohibitively expensive. For a single
agent, this problem is well understood: successor features with generalized policy improvement, together with their universal extension, recombine a library of learned policies into a policy for any new objective, with a guarantee that the result is never worse than any policy in the library. However, multi-
agent transfer has received far less
attention, and the common practice of letting each
agent recombine its own library independently inherits the recipe but not the guarantee. We prove that this independent composition can produce joint behavior strictly worse than every policy in the library, because recombining teammates changes the environment each
agent faces and invalidates the values it relies on, a failure with no single-
agent counterpart. We further show that the only unconditionally safe fixed rule is synchronized composition, which moves the whole team to one jointly trained policy but cannot serve objectives that assign different goals to different
agents. To attain safety and flexibility at once, we propose MA-USFA, a hierarchical method with two layers: a lower layer of universal successor feature approximators that predicts each
agent's successor features while conditioned on its teammates' objectives, and an upper composer that selects, across
agents, which library entry each
agent should follow and supplies the cross-
agent correction a per-
agent value cannot represent. Trained once over the distribution of objectives, it is applied at deployment with no per-task adaptation.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CL
Semantic Lenia: Emergence of Homeostatic Solitons within the Semantic Space of Large Language Models
Yoshihiko Kayama
We introduce Semantic Lenia, an artificial life framework that transforms Large
Language Model (LLM) inference from a static optimization problem into a continuous, closed-loop dynamical system. By establishing a non-linear homeostatic feedback loop to dynamically balance semantic attraction and syntactic repulsion, we demonstrate the emergence of “Homeostatic
Solitons”-metastable semantic structures that actively resist repetitive crystallization. Our exhaustive parameter sweeps map a critical “Habitable Ridge” where applied steering forces balance the model's intrinsic syntactic inertia. This approach successfully maintains generative trajectories in a numerically sensitive critical regime, triggering profound abductive leaps without structural collapse, and reveals a capacity-dependent scaling trend in the syntactic inertia across different model sizes.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CV
Motion-as-Prompt: Enhancing Motion Reasoning in Multimodal Large Language Models via Motion-Guided Cross-Frame Visual Prompting
Xikai Sun, Kebin Liu, Haotian Wang et al. (+3 more)
Motion-centric
video reasoning is fundamental to interactive applications such as robotic manipulation and autonomous
navigation. However,
multimodal large
language models (MLLMs) typically process
videos through sparse uniform sampling to control visual-
token and
attention costs. This strategy may discard critical transitions between sampled frames, limiting
reasoning about object movement, collisions, and causal interactions. To mitigate this issue, we propose Motion-as-
Prompt (MaP), a track-guided cross-frame visual prompting framework. MaP recovers dense point trajectories, selects motion-informative frames, and marks the trajectories accumulated between consecutive sampled frames directly onto the visual inputs, making otherwise hidden displacement, direction changes, and interactions observable to frozen MLLMs. Experiments on CLEVRER and Something-Something-v2 show that MaP consistently improves average motion-
reasoning accuracy, yielding gains of 4.2% and 8.9% for GPT-5.5, respectively. Notably, these improvements are obtained without degrading non-motion understanding, highlighting the robustness of MaP. These results demonstrate that MaP provides a simple and effective solution for enhancing motion-centric
video reasoning without model training or architectural modification. Project page:https://github.com/SunVictor23/MaP.
Read the full abstract ▾
All authors Xikai Sun, Kebin Liu, Haotian Wang, Li Liu, Xu Wang, Yunhao Liu
Artificial Intelligence · Cross-listed from cs.MA
Beyond Memory: A Transactional Continuity Kernel for Long-Lived AI Agents
Jun He and Deying Yu
Persistent AI
agents accumulate versioned state across long horizons, but storage retention alone does not identify authoritative state. Without an explicit control plane, unmediated updates by models, tools, and background workers risk stale overwrites, un-audited exposures, and self-authorizing privilege escalation. We argue that
agent state governance is an infrastructural activation problem, defining continuity as an unbroken, authorized lineage of accepted branch heads. We present the Continuity Kernel (CK), an activation contract that decouples off-commit candidate evaluation from atomic state activation. Untrusted components propose typed changes against an exact predecessor head or typed absence. A short activation transaction revalidates ownership, pre-state authority, freshness, and effect uniqueness, recording one stable disposition (Commit, Reject, Quarantine, or Defer). Only Commit atomically advances the branch head and installs the complete accepted unit (state, authority, lineage, effects, outcome, and receipt). A bounded executable model verifies the protocol across 2,808,230 reachable states and 5,526,474 state-changing transitions with zero invariant violations.
Read the full abstract ▾
Artificial Intelligence
CLAIM: Leading Open-domain Active Clarification of Large Language Models with Uncertainty Measurement
Kuangzhao Yang, Ziliang Zhao and Zhicheng Dou
In open-domain human-computer interaction scenarios, large
language models (LLMs) frequently encounter user queries that are ambiguous or incomplete. In such cases, directly producing an answer often leads to overgeneralized, erroneous, or low-information responses. In contrast, asking clarifying questions can substantially improve interaction quality. However, existing approaches still rely heavily on manually annotated data or preference
alignment to address two fundamental challenges: when clarification is necessary, and which aspect of the query should be clarified. This reliance incurs high annotation costs and limits generalization. To address these challenges, we propose CLAIM, an uncertainty-driven framework for active clarification learning in open-domain settings. CLAIM eliminates the need for explicit human preference annotations by quantifying query uncertainty through the
entropy induced by answer disagreements across multiple models. This uncertainty signal is then used to construct high-quality synthetic data, enabling the training of a unified clarification decision model through a combination of supervised learning and reinforcement learning. Specifically, we propose an
entropy-driven synthetic data generation pipeline that integrates
entropy-based uncertainty estimation with semantic clustering and
reasoning-based judgments, enabling reliable automatic annotation of clarification requirements. To train CLAIM, we formulate the clarification process as a structured decision generation problem and adopt a training paradigm that combines supervised fine-tuning (SFT) with group-relative policy optimization (GRPO). Experimental results demonstrate that CLAIM can learn stable and generalizable clarification strategies without relying on manually labeled data, offering a low-cost and robust solution for proactive understanding in real-world open-domain interactions with LLMs.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from eess.AS
Deep Learning Based Relative Transfer Matrix Estimation for Multiple Sources and Multiple Microphones
Oshan A. B. Yalegama and Wageesha N. Manamperi
The Relative Transfer
Matrix (ReTM), recently introduced as a generalization of the relative transfer function for multiple receivers and sources, shows promising performance when applied to
speech enhancement in noisy environments. Estimating the ReTM of sound sources by exploiting the covariance
matrices of multichannel recordings is highly beneficial for practical applications and, to date, remains the only proposed approach. This paper investigates deep learning-based ReTM estimation. We propose three novel supervised learning frameworks using time and short-time frequency transform domain convolutional networks, and a Long Short-Term
Memory-based recurrent neural network. Experimental results demonstrate that the proposed models achieve more accurate estimation of the ReTM using five objective metrics compared to the covariance-based method. We also show the effectiveness of the proposed frameworks for
speech enhancement, achieving performance on par with the baseline method.
Read the full abstract ▾
Artificial Intelligence
Making AI-Generated Feedback Matter: A Large-Scale Study of Feedback Workflows and Student Enactment
Omar Alsaiari, Nilufar Baghaei, Jason M. Lodge et al. (+3 more)
Feedback processes strongly influence student learning, yet their educational value depends on addressing two distinct challenges: providing high-quality, timely, and individualised feedback at scale, and supporting students to interpret, evaluate, and act on that feedback productively. Generative AI offers a credible means of addressing the provision challenge, but students' uptake of AI-generated feedback remains limited. We conducted a large-scale quasi-experimental sequential cohort study comparing three AI-mediated feedback workflows across 13,037 students and 51,296 student-authored resources. In Directed Feedback (n = 3,723), students received AI-generated feedback comments without structured support. In Self-Directed Feedback (n = 3,951), students could initiate optional AI-supported dialogue. In Enacted Feedback (n = 5,363), students were prompted to select feedback suggestions, evaluate their relevance, and engage in targeted AI-supported dialogue anchored to those selections. Enacted Feedback was associated with significantly higher uptake of AI-generated feedback, with an estimated probability of 26.2%, compared with 14.1% for Directed Feedback and 0.1% for Self-Directed Feedback. It was also associated with significantly higher self-assessment confidence and submitted-work quality than both comparison conditions. These findings suggest that the educational value of AI-generated feedback depends not only on the quality of feedback comments, but also on workflows that actively structure students' enactment of feedback literacy processes. The results have implications for the design of AI feedback systems that position learners as active participants in judgement, dialogue, and improvement rather than passive recipients of comments. Overall findings show that AI access alone is insufficient; purposeful workflow design is central to productive feedback use.
Read the full abstract ▾
All authors Omar Alsaiari, Nilufar Baghaei, Jason M. Lodge, Dragan Gaševi'c, Naomi Winstone, Hassan Khosravi
Artificial Intelligence · Cross-listed from cs.CL
Learning to Persuade Exposes How Easily LLMs Abandon Correct Beliefs
Nimet Beyza Bozdag, Emre Can Acikgoz, Gokhan Tur et al. (+1 more)
Persuasion is a core dynamic of natural
language communication, shaping how large
language models (LLMs) update beliefs, resolve disagreements, and reach decisions. As LLMs increasingly debate, advise, and think collaboratively with humans and each other, resistance to harmful persuasion becomes a core requirement for reliable behavior. Yet we show that this requirement is far from met: a single targeted persuasive argument is enough to collapse model accuracy to near zero, even when the argument is factually false. We formalize this threat as
adversarial persuasion and introduce an
adversarial reinforcement learning framework that trains persuader
agents to change a target model's answer in a single interaction. First, we show that optimizing persuasion strategies through trial and error exposes vulnerabilities that static prompting misses: RL-trained persuaders raise persuasion success from approximately 24% to over 93% against the training-time persuadee. Second, we find that these learned strategies transfer to unseen models, achieving 83% attack success on Qwen-14B, 79% on Llama-3.1-8B, and 25% on GPT-4o-mini. Third, we demonstrate that a curriculum that bootstraps on more persuadable open-weight models before targeting harder models further increases GPT-4o-mini attack success from 25% to 38%. Moreover, our results reveal that optimized persuaders increasingly rely on credibility-based tactics, including fabricated citations and false authoritative evidence. Together, these findings expose a critical weakness in current LLM
agents: even when they initially reason correctly, they can be steered toward false conclusions by optimized natural
language influence. This positions persuasion robustness as a necessary safety criterion for multi-
agent and human-AI decision-making systems.
Read the full abstract ▾
All authors Nimet Beyza Bozdag, Emre Can Acikgoz, Gokhan Tur, Dilek Hakkani-Tür
Artificial Intelligence · Cross-listed from cs.LG
FM-LLM: A frequency-enhanced mixture-of-experts framework for adapting LLMs to time series forecasting
Rentao Gu, Yihang Ding, Junjie Li et al. (+5 more)
Recent advances in Large
Language Models (LLMs) have spurred cross-modal solutions for time-series
forecasting. However, existing methods rely heavily on textual
prompts for modality
alignment-introducing nontrivial computational overhead and failing to leverage the rich spectral dynamics inherent in time-series data. To enable
prompt-free, frequency-aware adaptation of frozen LLMs, we propose FM-LLM (Frequency-Enhanced Mixture-of-Experts for adapting LLMs to Time Series
Forecasting), an autoregressive framework grounded in constrained asymmetric coupling. A
Fourier Analysis Network (FAN)-based spectral
token aligner injects structured harmonic representations directly into the frozen LLM with numerical compatibility. An asymmetric Mixture-of-Experts (MoE) decoder enforces role separation: shared experts with lightweight FAN layers reconstruct the global periodic backbone, while routed experts-restricted to standard FFNs-specialize in modeling non-periodic residual dynamics. A time-frequency hybrid loss function jointly optimizes temporal accuracy and spectral consistency, mitigating error accumulation during long-horizon autoregressive rollouts. Evaluated across eleven public
benchmarks, FM-LLM achieves state-of-the-art performance on 59 out of 78 evaluation metrics. Compared to the strongest autoregressive LLM-based baseline, it delivers average improvements of 5.3% in MSE and 5.6% in MAE, with maximum gains reaching 8.0% for MSE and 8.4% for MAE. FM-LLM also demonstrates robust transferability, maintaining superior performance in 10% few-shot and zero-shot
forecasting scenarios.
Read the full abstract ▾
All authors Rentao Gu, Yihang Ding, Junjie Li, Yi Ding, Weijing Sang, Xiaoli Huo, Xin Qin, Yuefeng Ji
Artificial Intelligence
Hojun Choi, Jaeyo Shin, Suin Lee et al. (+1 more)
Agentic systems powered by large
language models (LLMs) have opened new opportunities for business ideation. Yet existing approaches remain confined to a text-only paradigm, despite the inherently
multimodal nature of real-world contexts. We thus introduce MBA-Bench, the first
multimodal benchmark for training and evaluating business ideation
agents, comprising 30K samples across six domains, each domain characterized by distinct visual cues not fully conveyed by text alone. Concretely, we automatically caption images and employ GPT-4o to generate five reference ideas for each of three business questions through
retrieval query generation,
market evidence
retrieval, and evidence-augmented synthesis. Following prior work, we evaluate
agents across six business-oriented criteria using MLLM-as-a-Judge. To consider settings where criteria are hidden or disclosed, we present MBA-b and MBA-k for blind and known, respectively. We train both with two novel reward objectives—creativity and feasibility—while MBA-k further optimizes the six disclosed criteria for eight in total. Both are trained via LoRA-based supervised fine-tuning followed by group relative policy optimization with these setting-specific rewards. For extensive experiments on MBA-Bench, we set up two baselines accommodating either captions only or
multimodal inputs, with the latter nearing closed-source performance on several metrics. MBA-b and MBA-k outperform caption baselines by 63.9% and 77.1%, and
multimodal baselines by 25.6% and 35.8%, respectively.
Read the full abstract ▾
All authors Hojun Choi, Jaeyo Shin, Suin Lee, Hyunjung Shim
Artificial Intelligence · Cross-listed from cs.LG
Dion3: Full-Stack Orthogonal Updates
Noah Amsel, Jack Zhang, Kwangjun Ahn et al. (+5 more)
The
Muon optimizer incurs a significant overhead cost due to its cubic-time Newton-Schulz orthogonalization step. When weights are sharded, communication overhead compounds this computational cost, eroding the benefits of
Muon in many settings. We present Dion3, a revision of
Muon that targets this overhead at every level of the stack. Our Gram Newton-Schulz
algorithm reduces the FLOP cost of orthogonalization, our CuteDSL kernels accelerate it by exploiting
symmetry, and our megabatching strategy reduces communication overhead. Moreover, we propose a simple change to the update rule that cuts costs even further: selecting only a fraction of the momentum
matrix's rows to orthogonalize at each step. This update rule improves on Dion (another "compressed" version of
Muon), in both speed and performance. Overall, Dion3 matches or improves on the loss achieved by
Muon but reduces optimizer step time by up to 6x. Dion3 is available via the dion package (https://github.com/microsoft/dion) as a drop-in replacement for
Muon.
Read the full abstract ▾
All authors Noah Amsel, Jack Zhang, Kwangjun Ahn, Ali Naeimi, Austin Feng, Berlin Chen, Tri Dao, John Langford
Artificial Intelligence
Foresight Without Seeing: Latent Futures for World Action Models
Jiakai Huang, Zhongbo Wu, Siyu Xu et al. (+5 more)
World Action Models (WAMs) connect visual prediction with
robot control, but supplying predictive context often requires expensive future-
video generation. Direct policies avoid this cost but lack an explicit interface for accessing future-indexed predictive information. We introduce ForeWAM, a World Action Model that separates
forecasting from rendering to expose and shape latent predictive context for efficient control. Its core mechanism, Future-KV, performs a single
Video DiT prefill over the current visual latent and noise-initialized future slots, then reuses the resulting key-value states throughout action denoising. To make this context relevant to control, we introduce dynamics registers supervised by latent actions from a frozen teacher during training, encouraging representations of interaction-induced transitions. This reusable context supports a lightweight, single-layer action decoder. We evaluate ForeWAM on LIBERO, LIBERO-Plus, RoboCasa, and real-world manipulation tasks. Without additional policy-level
embodied pretraining, ForeWAM improves RoboCasa success by 9.7 percentage points over Fast-WAM at the same budget of 50 demonstrations per task, reaching 59.2%. With a single-layer decoder, it achieves 77.6% success on LIBERO-Plus and reduces policy-query latency to 88.7 ms on an NVIDIA A800, delivering a 6.27-fold speedup over Fast-WAM. These results show that latent predictive computation provides useful foresight for robust, efficient control without explicit future-
video generation.
Read the full abstract ▾
All authors Jiakai Huang, Zhongbo Wu, Siyu Xu, Zheng Zhang, Zihan Wang, Shan You, Chang Xu, Tao Huang
Artificial Intelligence
Learning from Online User Feedback for Shopping Agents
Haobo Zhang, Kelong Mao, Sulong Xu et al. (+2 more)
Large
language model-based shopping
agents are increasingly deployed in real-world e-commerce platforms, generating massive amounts of user interaction logs that provide valuable supervision for improving these
agents. However, existing approaches primarily rely on offline training signals, such as user-item interactions or synthetic preference data, while largely overlooking the rich supervision contained in users' natural conversational feedback. Moreover, the available online feedback is heterogeneous, sparse, and noisy, making it difficult to transform into reliable learning signals automatically. To address these challenges, we propose LOFA, a framework that enables shopping
agents to learn directly from real online interaction logs without human annotation. LOFA combines reinforcement learning over verifiable purchase outcomes with feedback-aware on-policy
distillation, which identifies users'in-dialogue directives and converts them into dense
token-level supervision. These complementary objectives capture both collaborative behavioral patterns and user-specific preferences. Extensive experiments on real-world e-commerce logs demonstrate that LOFA consistently improves recommendation quality, response helpfulness, and user-satisfaction
alignment over strong baselines, highlighting the effectiveness of learning shopping
agents from real online user feedback.
Read the full abstract ▾
All authors Haobo Zhang, Kelong Mao, Sulong Xu, Simiu Gu, Zhicheng Dou
Artificial Intelligence · Cross-listed from eess.SP
Small Language Model enabled Autonomous agent for Language-Conditioned Cognitive Radar
Minhaj Uddin Ahmad, Zakia Zaman, Shunqiao Sun et al. (+1 more)
Modern radar systems require adapting their processing strategies in response to changing interference, clutter, and data availability. This paper introduces a framework for a small
language model (SLM)-driven autonomous
agent designed for
language-conditioned cognitive radar, functioning as an intelligent controller for a suite of array signal processing tools. Given a natural-
language command, the
agent extracts radar-operation-related cues, selects an appropriate sequence of signal-processing methods, configures parameters, and invokes executable tools for numerical computation. Experiments with a synthetic uniform linear array (ULA) radar demonstrate that, given a natural-
language command, the
agent performs meaningful
algorithm selection across diverse scenarios for sidelobe control, jammer suppression, multiple-null beamforming, coherent-source handling, and low-snapshot direction-of-arrival (DOA) estimation. Ablation results show that radar-specific prompting and physics-grounded tool execution are both required for reliable decisions and
hallucination-free numerical results.
Read the full abstract ▾
All authors Minhaj Uddin Ahmad, Zakia Zaman, Shunqiao Sun, Mizanur Rahman
Artificial Intelligence
CoAdapt-GUI: Joint Workflow Context and Policy Adaptation for Unseen GUI Applications
Linqiang Guo, Li Gu, Zihuan Jiang et al. (+7 more)
Mobile GUI
agents remain brittle when deployed to applications absent from source training. We study novel-app generalization under a limited target interaction budget and without target demonstrations. We introduce CoAdapt-GUI, a test-time adaptation (TTA) framework that jointly adapts structured workflow context and policy from the
agent's own target-app rollouts and rewards. The workflow context retains transferable procedures, failure modes, and verification rules while excluding app-bound source details. This separation allows reusable workflow knowledge to guide adaptation without transferring source-interface state. For policy adaptation, task-context-matched group-relative optimization updates a LoRA adapter on a frozen
vision-
language model. Across two unseen-app evaluations, CoAdapt-GUI reaches 45.0% on AndroidWorld-Generalization, compared with 37.5% for the reported Policy-Only TTA baseline, and raises AndroidWorld Plus performance from 38.6% to 52.9%. These results show that transfer-constrained workflow context provides substantial gains and that joint policy adaptation further improves held-out performance.
Read the full abstract ▾
All authors Linqiang Guo, Li Gu, Zihuan Jiang, Zhixiang Chi, Siobhan Reid, Ziqiang Wang, Yuanhao Yu, Wei Liu, Yang Wang, Tse-Hsun Chen
Artificial Intelligence
EnterpriseRAG: Benchmarking LLM Instruction Adherence and Robustness under Non-Ideal Enterprise Retrieval
Huiqi Miao, Xinbao Sun, Bo Wang et al. (+6 more)
Enterprise RAG deployments face a critical reliability gap: while LLMs satisfy 80% of individual constraints, only 26.8% of responses meet all requirements simultaneously, revealing a 57-point orchestration gap. Existing
benchmarks assume clean
retrieval with simple queries, failing to capture production conditions where noisy documents and multi-dimensional constraints coexist. We introduce EnterpriseRAG, a
benchmark of 983 expert-validated samples across six domains that systematically simulates three failure modes absent from prior work:
retrieval noise, knowledge gaps, and factual conflicts, coupled with complex instructions. Evaluation of 13 state-of-the-art LLMs reveals a severe instruction adherence collapse, where high per-constraint satisfaction masks low holistic compliance. Critical findings expose deep barriers under knowledge gaps and factual conflicts, even with
reasoning-enhanced inference, indicating production RAG requires explicit context-aware protocols and calibrated judgment. EnterpriseRAG provides a reproducible foundation for measuring and closing these gaps, directly informing deployment decisions for enterprise-scale RAG systems. We will release the
benchmark and evaluation framework upon publication.
Read the full abstract ▾
All authors Huiqi Miao, Xinbao Sun, Bo Wang, Fanyu Meng, Lijun Mei, Na Wu, Di Jin, Chao Deng, Junlan Feng
Artificial Intelligence
Localizing Safety Alignment: MLP Layers and Mid-Network Blocks Encode Refusal Behavior in Large Language Models
Mingyu Zong, Sampad Mohanty and Bhaskar Krishnamachari
Safety
alignment in large
language models is often treated as a distributed property of the entire network, yet its practical brittleness suggests that refusal behavior may be concentrated in a smaller set of parameters. This work addresses where safety-aligned refusal is encoded by transplanting weights from aligned models into matched unaligned base models at multiple levels of granularity. Using two open-weight model pairs and four safety
benchmarks, we conducted experiments to compare the effects of replacing
attention weights, MLP weights, contiguous layer regions, and MLP blocks. Across both model families, refusal transfer is dominated by MLP weights: replacing MLP parameters recovers substantially more malicious-
prompt refusal than replacing
attention parameters, with gains of at least 2.7 times more across
benchmarks. Within the MLP stack, refusal-relevant parameters exhibit a consistent mid-network concentration, as the block spanning layers 8-11 is selected first in all six greedy searches over model-
dataset pairs. The results also show that the composition of safety-relevant components is non-additive: in five of six greedy trajectories, adding more aligned blocks can reduce refusal performance, and selective block subsets can outperform full MLP transplantation on malicious refusal, benign over-refusal, or both. Finally, greedy orders transferred to OR-Bench vary with the source
benchmark used to derive them, indicating a
benchmark-dependent precision-coverage trade-off. These results suggest that safety
alignment in current LLMs is both localized and interaction-sensitive, offering insight into
alignment brittleness and potential avenues for targeted safety interventions.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CV
A Hybrid Framework of Vision Transformer and Gated Recurrent Unit for Detection of Mosquito Diseases
Danial Sharifrazi, Saadat Behzadi, Nouman Javed et al. (+3 more)
Identifying dengue virus-infected mosquitoes from control mosquitoes is a major challenge in analyzing mosquito locomotion behavior due to the small size and complexity of the
video background. Conventional AI methods are often unable to extract accurate features from
video frames and produce erroneous features. In this study, a three-step framework is introduced: first, mosquitoes are identified and the background is removed using the YOLO 11M model, then visual features are extracted using the
Vision Transformer (ViT), and finally the
videos are classified with a convolutional GRU (ConvGRU) classifier. A comparative analysis of different models, including Recurrent Neural Network (RNN), Long Short-Term
Memory (LSTM), Gated Recurrent Unit (GRU), and their convolutional versions showed that the ConvGRU model achieved the best performance; it achieved 88.88% accuracy, 84.45% precision, 82.82% recall, and 82.81% F1 score. These results demonstrate that combining convolutional models with sequence-based networks, especially in the ConvGRU model, allows the simultaneous extraction of precise spatial features and long-term temporal dependencies from mosquito movements. Finally, the proposed framework provides a reliable solution for analyzing mosquito behavior in complex environments.
Read the full abstract ▾
All authors Danial Sharifrazi, Saadat Behzadi, Nouman Javed, Roohallah Alizadehsani, Prasad N. Paradkar, Asim Bhatti
Artificial Intelligence · Cross-listed from cs.RO
RoadWeaver: Large-Scale Lane-Level HD Map Generation from Scratch for Autonomous Driving Simulation
Yueyuan Li, Zexi Chen, Weijie Xi et al. (+4 more)
Autonomous driving simulation requires diverse and scalable lane-level HD maps to support long-horizon evaluation across complex road networks. Existing approaches either rely on handcrafted or reconstructed real-world maps, which limits scalability, or generate only local road structures rather than complete HD maps. We present RoadWeaver, a coarse-to-fine framework for from-scratch generation of diverse, large-scale HD maps. RoadWeaver first synthesizes a global road layout, expands it into a connected road network, and then constructs lane-level geometry with topologically consistent lane connectivity. Experimental results show that RoadWeaver achieves a 99.8% reachability, a 10.7% dead-end ratio, and an endpoint
alignment error of 0.24 m. Compared with SOTA generation methods, it reduces endpoint
alignment error by 94.4% while generating complete HD maps in 1.39–3.50 s. The generated maps can be directly deployed in driving simulators, providing scalable simulation environments for future closed-loop evaluation of autonomous driving systems. The training code and an out-of-the-box implementation of RoadWeaver will be released upon acceptance.
Read the full abstract ▾
All authors Yueyuan Li, Zexi Chen, Weijie Xi, Mingyang Jiang, Songan Zhang, Hanyang Zhuang, Ming Yang
Artificial Intelligence · Cross-listed from cs.CL
Reinforcing Step-level Reasoning for Effective Self-Correction in LLMs
Vu Duc Anh, Nhat M. Hoang, Do Xuan Long et al. (+3 more)
Achieving effective self-correction, where models verify and correct their own mistakes, remains a fundamental challenge for large
language models (LLMs). In this work, we propose Self-Fix Step-DPO (SFS-DPO), a reinforcement learning based, two-stage framework for step-level self-verification and self-correction. The first stage strengthens step-level
reasoning via step-level preference optimization, while the second stage explicitly trains models to self-verify and self-correct. We further introduce a teacher-assisted variant, SFS-DPO-R, which incorporates explanatory rationales for error verification to provide stronger corrective signals. Comprehensive in-domain and out-of-domain evaluations across multiple LLMs demonstrate that SFS-DPO and SFS-DPO-R consistently outperform prior step-level training baselines. Our analysis further reveals improvements in self-correction frequency and effectiveness, highlighting the importance of strengthening step-level
reasoning for robust performance.
Read the full abstract ▾
All authors Vu Duc Anh, Nhat M. Hoang, Do Xuan Long, Cong-Duy Nguyen, Ponhvoan Srey, Luu Anh Tuan
Artificial Intelligence · Cross-listed from cs.CV
From Synthesis to Removal: Physics-Grounded Reflection Simulation and Diffusion-Based Video Dereflection
Zepeng Wang, Jiagao Hu, Fuhao Li et al. (+3 more)
Videos captured through glass often contain reflections that degrade visual quality and interfere with downstream
vision tasks. Although single-image reflection removal has been extensively studied,
video reflection removal remains largely underexplored due to the lack of paired
video data, temporally coherent removal models, and dedicated evaluation
benchmarks. We present a closed-loop framework that unifies physics-grounded reflection simulation,
diffusion-based
video dereflection, and
benchmark evaluation. Our S2R-Synthesis pipeline generates paired reflected and reflection-free
videos by performing physics-grounded augmentation in the structure space and rendering realistic reflected
videos with a trained
video diffusion renderer; the augmentation models key glass-related effects including roughness-induced blur, thickness-induced ghosting, and reflectance variation. Based on the synthesized data, we introduce S2R-Removal, the first
diffusion-based
video reflection removal model, which adapts a pretrained
video diffusion prior through reflection-aware latent adaptation and one-step pixel-
geometric refinement, recovering the clean transmission in a single denoising step. We further build S2R-Bench, the first
benchmark for
video reflection removal, supporting both full-reference evaluation and real-world human perceptual assessment. Experiments on S2R-Bench and multiple public image
benchmarks demonstrate state-of-the-art performance and faster inference than even non-
diffusion baselines, and validate the effectiveness of S2R-Synthesis. Project page: https://codingwzp.github.io/VideoDereflection_S2R.
Read the full abstract ▾
All authors Zepeng Wang, Jiagao Hu, Fuhao Li, Yuxuan Chen, Fei Wang, Daiguo Zhou
Artificial Intelligence · Cross-listed from cs.CL
Beyond Single-Turn Confidence: Trajectory-Adapted Uncertainty Quantification for LLM Agents
Dylan Bouchard and Mohit Singh Chauhan
Uncertainty quantification (UQ) methods for
language models are typically evaluated on single-turn outputs, where uncertainty is attached to one generated answer. For LLM
agents, however, the unit of observation is an interactive trajectory, where the model can ask clarifying questions, call tools, update state, and make intermediate decisions whose errors propagate to the final outcome. We study whether three common families of single-turn UQ methods transfer to this setting. Across five LLMs and four multi-turn tool-use
datasets from BFCL-v4 and $\tau^2$-bench, we evaluate white-box scorers based on action-
token probabilities, black-box consistency scorers based on resampled trajectories, and reflexive scorers based on model self-assessment of the trajectory. We find that transfer is often useful but uneven.
Token-probability scores are highly sensitive to the choice of aggregator used across turns, reflexive scores provide the strongest low-cost baseline in most evaluated settings, and black-box self-consistency is often the strongest UQ family, with trajectory-equivalence and action-set consistency typically ranking highest among its variants. These results suggest that UQ methods developed for single generations should be revalidated at the trajectory level, with careful
attention to the consistency measurement, aggregator choice, and computational budget.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from eess.SY
A Conceptual Framework for Enhancing Workforce Readiness for Smart Manufacturing in the AI Era
Dalton Ross Smith, Wilburn Whittington, Alejandro Martinez et al. (+2 more)
The convergence of artificial intelligence (AI), Industrial Internet of Things, cyber-physical systems, and advanced robotics is reshaping manufacturing faster than engineering curricula can adapt, widening the gap between the competencies required on the shop floor and those delivered by traditional engineering and technology education. This paper proposes a Workforce Readiness Level (WRL) framework, which adapts the Technology Readiness Level scale into nine progressive competency stages and a four-pillar rubric, digital and AI literacy, cyber-physical systems fluency, human-machine collaboration, and data-driven decision making, aggregated through a composite stage score and a cohort-level workforce-readiness index under a “no-thin-pillar” rule. The framework is instantiated at a university smart-manufacturing teaching laboratory and draws on 89 sponsored capstone projects delivered over four semesters, four of which are analyzed in depth. Four pillars jointly span the relevant ABET student outcomes. Across the highlighted cohorts the workforce-readiness index ranged from 5.2 to 6.4, and the no-thin-pillar rule was diagnostically informative in three of the four cases and the binding certification constraint in one, repeatedly surfacing cyber-physical and data-driven-decision gaps concealed behind strong analytics profiles; advancement to the highest stages was gated by industry-embedded experience rather than additional coursework. WRL offers educators, accreditation bodies, and regional workforce systems a common, evidence-based instrument for diagnosing and advancing workforce readiness; future work will calibrate pillar weights and test reliability and predictive validity.
Read the full abstract ▾
All authors Dalton Ross Smith, Wilburn Whittington, Alejandro Martinez, Aidan Duncan, Gang Li