Artificial Intelligence · Cross-listed from cs.LG
Multimodal Federated Learning under Dual-Axis Modality Missingness
Adiba Orzikulova, Jaehyun Kwak, Jaemin Shin et al. (+5 more)
Multimodal federated learning (FL) supports collaborative modeling in
privacy-sensitive health-sensing and medical settings, but realistic deployments often exhibit dual-axis modality missingness: clients have different modality sets, and individual samples may contain only subsets of the modalities available locally. Existing methods typically address these two axes separately. We propose Flux, a
multimodal federated learning framework built around two complementary components. First, modality-aware confidence tempering learns sample-specific confidence for each modality through mask-aware unimodal supervision and fuses the confidence estimates from observed modalities into a sample-adaptive temperature that adjusts predictive sharpness according to evidence quality and completeness. Second, gradient-decoupled private adaptation applies this temperature only to a client-private prediction pathway, while training the shared
federated model with a standard, untempered objective. This enables sample-specific, client-local confidence adaptation without allowing confidence-dependent gradients to perturb shared representation learning. Across four
multimodal datasets, Flux achieves the highest average macro-F1 on every
dataset, outperforming the strongest
dataset-specific baseline by 0.8~2.2 points and by 1.6 points on average. Additional analyses demonstrate favorable calibration, temperature sensitivity to both modality missingness and input corruption, and more stable shared optimization under private-only tempering. Our code is available at https://github.com/AdibaOrz/Flux.
Read the full abstract ▾
All authors Adiba Orzikulova, Jaehyun Kwak, Jaemin Shin, Yunqi Guo, Xiaomin Ouyang, Guoliang Xing, Steven Euijong Whang, Sung-Ju Lee
Artificial Intelligence
SafeSceneReason: A Multimodal Reasoning Benchmark Connecting Industrial Hazards with Accident Knowledge
Yuanchi Zhu, Kang An, Tengyue Wang et al. (+11 more)
Industrial-safety understanding requires more than detecting workers, equipment, and personal protective equipment. Models must also assess compliance, identify hazardous interactions, explain potential accident mechanisms, and recommend preventive actions. Existing safety
datasets primarily focus on visual perception or isolated violation recognition and provide limited supervision for evidence-grounded
reasoning. We introduce SafeSceneReason, a
multimodal industrial-safety
reasoning benchmark and companion training corpus that connects workplace scenes with knowledge from occupational accident investigations. SafeSceneReason combines two complementary data-construction pipelines. The scene-centric pipeline converts annotated workplace images into executable safety scene graphs and generates deterministic answers through program execution over objects, relations, and safety rules. The report-centric pipeline extracts figures and contextual evidence from accident reports and constructs
multimodal questions using evidence graphs, explicit information boundaries, multi-step
reasoning paths, and iterative verification. The resulting resource contains 110,581 verified scene-centric question–answer pairs and 13,114 refined report-centric question–answer pairs, covering perception, spatial and quantitative
reasoning, compliance assessment, evidence synthesis, causal analysis, and mitigation-oriented decision making. Evaluation of representative proprietary and open-source
vision–
language models reveals substantial performance differences and persistent weaknesses in comparative, technical, and multi-evidence
reasoning, demonstrating that strong general visual understanding does not yet guarantee reliable industrial-safety
reasoning.
Read the full abstract ▾
All authors Yuanchi Zhu, Kang An, Tengyue Wang, Zhongyu Yang, Chenxu Du, Xinqi Yang, Hebao Zhu, Bokai Zhao, Tianyu Liang, Ziliang Wang, Faqiang Qian, Yunli Yang, Weiyang Shi, Qibing Ren
Artificial Intelligence · Cross-listed from cs.LG
Privileged Solutions or Context-Induced Teacher Behavior? Dissecting On-Policy Self-Distillation
Yuki Ichihara, Naoto Iwase, Mohammad Atif Quamar et al. (+1 more)
On-Policy Self-
Distillation (OPSD) is commonly interpreted as the transfer of privileged information: a teacher observes the verified solution to the target problem and supervises the student's trajectory. However, this interpretation conflates two effects. The reference solution not only reveals the answer to the current instance but also changes the context under which the teacher provides
token-level supervision. We investigate the role of target-specific privilege with $\mathrm{OP}^{2}\mathrm{SD}$ (On-Policy Self-
Distillation from Other Problems), which replaces the paired reference with a problem and solution from a different example, while preserving the student rollout, teacher, and
distillation objective. Across three models and three mathematics
benchmarks, $\mathrm{OP}^{2}\mathrm{SD}$ improves over the base model, remains competitive with OPSD. The success of $\mathrm{OP}^{2}\mathrm{SD}$ implies that OPSD gains do not necessarily come from access to the reference solution, and that the teacher's context-induced behavior is an important factor.
Read the full abstract ▾
All authors Yuki Ichihara, Naoto Iwase, Mohammad Atif Quamar, Junpei Komiyama
Artificial Intelligence
Omni2LoRA: Coherence-Preserving Parametric Memory for Efficient Omni Language Models
Puneet Mathur, Manan Suri and Dinesh Manocha
Omnimodal
language models (OLMs) enable unified audio-visual understanding, but processing long joint
token sequences makes inference computationally prohibitive. While recent
token compression methods attempt to alleviate this burden, compressing modalities in isolation often destroys the temporal cross-modal anchors necessary for coherent
reasoning. We introduce Omni2LoRA, a two-stage framework for efficient parametric
memory compression via coherence-preserving context
distillation that bypasses the
token bottleneck entirely. First, a Perceiver hypernetwork processes intermediate representations from a frozen OLM to encode the
multimodal context into a full-rank Low-Rank Adaptation (LoRA) adapter in a single forward pass. To prevent the resulting parameter footprint from scaling linearly with recording length, we optimize a discrete rank allocation policy via Group Relative Policy Optimization (GRPO) that uses a modality-ablated counterfactual reward to explicitly penalize the loss of audio-visual coherence, forcing the model to allocate its fixed sub-linear rank budget to synergistic cross-modal anchors rather than isolated visual features. Across three omnimodal backbones, Omni2LoRA operating at a 30% rank budget outperforms direct full-context inference and strong
token-compression baselines (OmniZip, OMAC, O-MARC) on four audio-visual question answering
benchmarks, improving average accuracy by 8-12% over the strongest baseline and remaining stable under compression ratios as tight as 75%, where
token-
pruning methods degrade sharply. By converting
multimodal memory into a fixed-budget, reusable parameter state, our method drives answer-time
multimodal-
token load to zero, cutting per-query Time to First
Token (TTFT) by up to 12x relative to full-context inference and amortizing to under 0.5s after a handful of queries.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CV
RL-Native Distillation: Exploiting Scored Trajectories for Few-Step Image Generation
Yuhan Li, Fangao Zeng, Sicong Kang et al. (+7 more)
Efficient text-to-image generation requires both reinforcement-learning (RL)-based reward
alignment and few-step
distillation, yet these procedures are typically performed sequentially, increasing training cost and risking the loss of reward gains during compression. We take an RL-native perspective:
diffusion RL already generates reward-scored finite-step trajectories, whose intermediate states provide
distillation supervision. Based on this insight, we propose REST (Reward-Enhanced Scored-Trajectory
Distillation), a single-stage co-training framework in which a decoupled student learns from the evolving RL teacher's trajectories without changing teacher optimization. Advantage-Modulated
Distillation (AMD) transforms rollout advantages into signed weights, strengthening imitation of preferred trajectories and aligning
distillation priorities with task value. The resulting framework is general and lightweight, requires no extra image rollouts, no separate
distillation dataset, and no
adversarial training. Experiments on compositional generation, visual text rendering, and human-preference
alignment demonstrate competitive few-step, CFG-free generation with RAM or DiffusionNFT teachers. With only four sampling steps, REST-RAM achieves a DrawBench PickScore of 23.97, outperforming both the 40-step RAM teacher (23.95) and RTDMD (23.71).
Read the full abstract ▾
All authors Yuhan Li, Fangao Zeng, Sicong Kang, Mengfei Xu, Hao Zhou, Wenxiang Shang, Wei Li, Pipei Huang, Bo Zheng, Bingbing Ni
Artificial Intelligence · Cross-listed from cs.CR
Governing the KV Cache: Preventing Timing Side-Channel Leakage in Multi-Tenant LLM Inference
Tejasvi C. Addagada
The key-value (KV) cache is the primary throughput optimization in modern large
language model (LLM) inference, enabling prefix reuse across requests. In multi-tenant deployments this cache is shared across tenants, creating a timing side channel: an
adversarial tenant can reconstruct another tenant's private
prompt by probing cache-hit latency. Three published attacks exploit it – PROMPTPEEK, EarlyBird and InputSnatch – reaching up to 100% attack success rate against unprotected vLLM and SGLang, with rates varying by cache architecture and
prompt structure. We present KVGov, a governance layer addressing all three attack families' prefix-cache paths under one mechanism. A per-principal salt sigma_p = HMAC_K(secret, principal_id) seeds the block-hash chain, making cache keys cryptographically disjoint across principals. An ablation (N=1000 trials, seed 2026, deterministic judges) isolates this salt as the necessary and sufficient component. KVGov adds ORIGAMI, a Stackelberg water-filling audit scheduler that reduces adversary expected utility by 12.6% at realistic tenant heterogeneity (Gini 0.63), and an evolutionary stability analysis giving a 31.6% adversary-prevalence tipping point below which global caching remains stable. On real hardware (Qwen2.5-7B-Instruct, vLLM 0.26.0, NVIDIA A100) we measure a gate-verified cold/cached TTFT ratio of 0.22, confirming the channel is exploitable at production scale; the defense itself is evaluated in simulation calibrated to those measurements. We replicate the channel on an independent stack (llama.cpp on Apple Metal, ratio 0.093). Finally, isolation and cache efficiency need not conflict: identifying information resides only where
prompts diverge, so injecting the salt at that boundary rather than the chain root retains an estimated 93% of the prefix-cache benefit with no cross-principal signal.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
FedTVD: Balancing Data Quality and Quantity for Robust Federated Learning
Radwan Selo, Majid Kundroo and Taehong Kim
Federated Learning (FL) enables collaborative model training across distributed client devices while preserving data
privacy. However, FL faces significant challenges due to data heterogeneity, particularly in terms of label distribution skewness and variations in
dataset sizes, which can lead to biased model updates and hinder convergence. To address this, we propose FedTVD, a novel FL
algorithm that weights client contributions during aggregation by considering both data quality and quantity. Unlike traditional FL approaches such as FedAvg, which rely solely on
dataset size for client weighting, FedTVD integrates Total Variation Distance (TVD) to measure the divergence between each client's local label distribution and a uniform global distribution. Clients with highly skewed distributions receive lower weights, preventing unbalanced
datasets with imbalances from disproportionately influencing the global model. At the same time,
dataset size is incorporated to ensure scalability and
fairness. This dual-weighting mechanism effectively mitigates the impact of data imbalance, leading to more stable and generalized global models. Experimental results show that FedTVD consistently outperforms state-of-the-art methods across all
datasets (FMNIST, CIFAR-10, and CIFAR-100) and all levels of data heterogeneity. Notably, it achieves up to 10.6% improvement over FedAvg on CIFAR-10 under highly skewed data, while maintaining top performance even under moderate and IID settings.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
Beyond Solvability: Task Learnability as a Static Prior for LLM RL Post-Training
Ting Zhou, Zhenqing Ling, Daoyuan Chen et al. (+4 more)
Reinforcement learning (RL) has become a central post-training paradigm for eliciting
reasoning capabilities in large
language models, yet uniform task sampling allocates compute without regard to differences in how tasks respond to optimization. Existing task-valuation methods mostly rely on snapshot-based signals such as current pass rate or reward, which estimate how solvable a task is under the current policy. However, tasks with similar current solvability can still differ substantially in how positively they respond to further training. We study this residual axis as task learnability: a regime-conditional measure of expected positive response to continued training under a fixed RL post-training regime. By analyzing per-task reward trajectories, we find that learnability is reproducible across independently sampled training contexts and predictive of downstream utility. To make this signal practical before training begins, we propose TrajVal, a lightweight probe-based
estimator that approximates per-task learnability from a short probe run and two endpoint evaluations. TrajVal can be used either as a standalone static prior for task sampling or as a multiplicative prior for existing online schedulers. Experiments on mathematical and logical
reasoning benchmarks across multiple model scales show that TrajVal improves data efficiency over uniform sampling and provides complementary gains when combined with online scheduling methods.
Read the full abstract ▾
All authors Ting Zhou, Zhenqing Ling, Daoyuan Chen, Qianli Shen, Yilun Huang, Ying Shen, Yaliang Li
Artificial Intelligence · Cross-listed from cs.DB
AkasicDB: Demonstrating Omni RAG with a Unified Vector-Graph-Relational DBMS
Geonho Lee, Jeongho Park, Donghyoung Han et al. (+1 more)
Recent
Retrieval-Augmented Generation (RAG) systems increasingly combine vector
retrieval with structured knowledge, such as Graph RAG and Filtered vector search. However, existing database architectures struggle to support such complex RAG workflows efficiently, as they rely on out-of-DB pipelines or in-DB non-native integration, leading to high overhead. This demo paper presents AkasicDB, a database system that natively supports such RAG workflows by jointly executing vector similarity search, graph traversal, and relational filtering within a single execution framework. AkasicDB extends our prior work, Chimera, with native vector support to enable such unified execution. Based on AkasicDB, we demonstrate the first native integration of Vector-Graph-Relational RAG, which we refer to as Omni RAG. Through an interactive chat-style demonstration, users execute and visualize Omni RAG queries, directly experiencing its superior
retrieval and
reasoning over vector-only approaches while observing the practical limitations of existing database architectures in supporting Omni RAG. A demonstration
video is available at https://youtu.be/8d09_dtrEIM
Read the full abstract ▾
All authors Geonho Lee, Jeongho Park, Donghyoung Han, Min-Soo Kim
Artificial Intelligence · Cross-listed from cs.LG
FedA2L: Adaptive layer-wise learning rate adjustment in decentralized federated learning
Van Truong Vo, Khoa Nguyen and Taehong Kim
Decentralized intelligence systems with heterogeneous devices and limited coordination increasingly rely on decentralized
federated learning (DFL). However, DFL suffers from convergence inefficiency under data heterogeneity due to the use of a uniform learning rate (LR) that ignores layer-specific optimization needs. Foundational layers are responsible for maintaining network consensus, while specialized layers adapt to local data characteristics, leading to conflicting gradients and degraded performance under non-IID conditions. To address this fundamental tension, this work introduces FedA2L, a method that dynamically adjusts layer-wise LRs based on model divergence signals. By leveraging local update intensity and network consensus constraints, FedA2L seamlessly integrates into existing DFL protocols without additional communication or coordination. Extensive evaluations across DFL
algorithms, various model architectures, and
datasets demonstrate that FedA2L achieves up to 4.94 times faster convergence than vanilla DFL and reduces communication rounds by up to 59% compared to scheduler-based baselines. Furthermore, FedA2L exhibits resilience to severe data heterogeneity, larger network sizes, and sparse
topologies, reducing communication overhead and establishing it as a versatile optimization tool for resource-constrained or large-scale distributed learning in edge and IoT deployments. The code is released at https://github.com/nclabteam/FedA2L.
Read the full abstract ▾
Artificial Intelligence
CRUISE: Vision-Language Model-Guided Uncertainty-Aware Cross-Modal Sensor Fusion for Robust Autonomous Driving
Junyao Wang, Yulin Xu, Yu Li et al. (+2 more)
Modern autonomous vehicles are equipped with multiple sensors, such as cameras,
LiDAR, and radar, for comprehensive environmental perception. However, robust cross-modal feature fusion remains a critical challenge, as the reliability of each sensor varies significantly across diverse real-world driving conditions, including poor visibility and adverse weather. While uncertainty quantification (UQ) mitigates this issue by allowing models to prioritize reliable signals, existing uncertainty-aware fusion methods typically rely on simple feature-level uncertainty estimates and thus often fail to generalize effectively in complex, out-of-distribution scenarios. To address this limitation, we propose CRUISE, a novel uncertainty-aware cross-modal sensor fusion framework. CRUISE integrates a
vision-
language model (VLM)-guided UQ module that generates fine-grained, pixel-level uncertainty estimates. By leveraging the VLM's rich prior knowledge and superior contextual
reasoning, our approach provides a highly informative guide for the fusion process. Furthermore, we introduce a dynamic adaptive mechanism that explicitly models and captures cross-modal dependencies, ensuring the framework fully exploits the inherent complementary nature of multi-sensor inputs.
Read the full abstract ▾
All authors Junyao Wang, Yulin Xu, Yu Li, Pramod Khargonekar, Mohammad Abdullah Al Faruque
Artificial Intelligence
Signature-Guided Capacity Occupancy for Dense Expert Merging
Lingching Tung, Chi-Jui Kim, Beicheng Xu et al. (+2 more)
Dense expert merging combines domain-specialized
language models into one single checkpoint, typically by admitting task-vector support in weight space. However, this admission is governed by three decisions that existing methods answer only partially: where to open layer capacity from cross-expert conflict, who should occupy that capacity based on domain demand, and how to admit the resulting support without relying on costly recipe search. To tackle these issues, we propose SigMerge (Signature-Guided Capacity Occupancy), a structured capacity assignment framework for dense expert merging. Starting from a dense base merge, conflict signatures set each layer's capacity from cross-expert conflict, positive base-merge deficits set each domain's share of that capacity, and a sequential occupancy rule admits each expert delta up to the resulting layer-domain budget. Across 21 paired settings spanning seven dense base merges and three model pools, SigMerge improves every one (by 15.0% on average) and achieves the best average rank (1.67) among six merging methods, outperforming three categories of merging baselines.
Read the full abstract ▾
All authors Lingching Tung, Chi-Jui Kim, Beicheng Xu, Yuchen Wang, Bin Cui
Artificial Intelligence
Structure-Preserving Uncertainty Propagation in First-Order Proof Search
Tanel Tammet
GK is a query-directed first-order prover that extends ordinary resolution-based proof search with explicit positive and negative claims, numerical confidence values, and prioritized default rules with exceptions. It works directly with non-ground clauses, including equality and function terms. Candidate proofs are found by bounded first-order proof search; exception conditions of defaults are checked by further bounded searches, recursively when exceptions themselves depend on defaults. This avoids requiring a finite global grounding, while allowing incomplete searches to be reported as such. This paper adds structure-preserving quantitative reporting to that framework. Retained proof histories are used in two calculations. The first reconstructs the uncertain ground premises used by each proof and computes the probability that at least one retained proof is available, without counting shared premises independently. The second resolves positive and negative support at intermediate atoms before that support is propagated through later rules; the same calculation evaluates uncertain exception conditions for individual rule applications. Reports separate positive support, negative support, conflict, and ignorance and identify detected incomplete calculations or fallbacks. The implementation performs bounded reconstruction and dependency traversal after proof search and still requires no global grounding. Analytic examples and independent simulators reproduce the reference calculations on their stated fragments. Comparisons with probabilistic logic, probabilistic ASP, default logic, and goal-directed ASP identify cases of agreement, semantic difference, unsupported translation, and incomplete computation.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.DB
SiriusDeliver: Automating Data Warehouse Delivery at Tencent
Haining Xie, Xiaokai Zhou, Jiaming Yang et al. (+12 more)
Enterprise data warehouses (DWs) support business-critical analytics, but warehouse task delivery remains a complicated production process involving context
retrieval, workflow configuration, code generation, platform submission, and failure diagnosis. Although large
language models (LLMs) and coding
agents have improved software development, they are insufficient for production DW delivery, which requires dependency-aware orchestration, lifecycle-aware artifact control, and continuous adaptation to evolving platform practices. We present SiriusDeliver, an end-to-end delivery automation
agent for production warehouse task submission. SiriusDeliver integrates three components: a hierarchical delivery
agent that orchestrates warehouse skills, an artifact lifecycle control module that verifies and revises artifacts before and after platform execution, and a trace-driven skill
evolution mechanism that maintains reusable skills from delivery trajectories. We evaluate SiriusDeliver through offline
datasets and large-scale production deployment on Tencent Cloud WeData. Offline experiments on real-world warehouse delivery cases show that SiriusDeliver improves delivery success and automation efficiency over representative baselines. During a two-month deployment across 6 business teams and 4 warehouse task types, SiriusDeliver served 3,600 monthly active users and supported 18,240 delivery sessions, achieving an 87.2% end-to-end success rate and a 73.5% autonomous submission rate. A one-month A/B test shows that SiriusDeliver reduces median delivery time from 228 to 23 minutes and engineer effort from 95 to 11 minutes, while maintaining comparable final delivery success.
Read the full abstract ▾
All authors Haining Xie, Xiaokai Zhou, Jiaming Yang, Siqi Shen, Ziwei Wang, Yifeng Zheng, Tengyue Xu, Yipeng Shi, Zefang Zong, Yang Li, Peng Chen, Jie Jiang, Debiao He, Xiao Yan, Jiawei Jiang
Artificial Intelligence
Agentic Router: An Execution-Grounded Continual Learning Approach With Memory
Yuxuan Chen, Rongpeng Li, Zhifeng Zhao et al. (+3 more)
Large
language model (LLM)
agents provide a promising interface for command-line-based network operations, but a plausible command may still fail or introduce operational risk after execution. Existing approaches mainly focus on command generation or final configuration correctness, and do not use execution-grounded experience to jointly improve candidate coverage and action selection. We propose an execution-grounded dual-path consequence-aware
agent for CLI-based SONiC operations, which generates multiple complete actions, predicts their execution consequences, and selects the final action through utility- and risk-aware reranking. The proposal-side path abstracts reusable operational lessons into retrievable guidance to improve feasible-action coverage without modifying the proposal LLM, while the selection-side path adapts the consequence predictor through session-level LoRA updates using real SSH feedback to improve conditional selection quality. Experiments over multi-turn SONiC operation sessions with different Qwen3 proposal models show that the framework improves feasible-action coverage and top-1 execution success, and that the two adaptation paths provide complementary gains over interaction.
Read the full abstract ▾
All authors Yuxuan Chen, Rongpeng Li, Zhifeng Zhao, Yuntao Liu, Xing Xu, Honggang Zhang
Artificial Intelligence · Cross-listed from cs.CV
Rethinking Medical Landmark Localization with Prototype Learning-based Progressive Offset Correction
Jingxian Xu, Yuhao Huang, Rusi Chen et al. (+2 more)
Accurate landmark localization in medical images is a fundamental step for quantitative clinical measurement and downstream analysis. Existing localization methods have advanced, among which multi-stage refinement is a superior solution. Although this strategy mitigates the anatomical ambiguity inherent in single-stage global predictions, its high computational cost limits practical applicability. In this work, we propose a parameter-economic model, PPOC-LL, which leverages Prototype learning-based Progressive Offset Correction for Landmark Localization. Our contribution is three-fold. First, to drive coarse-to-fine landmark optimization, we introduce a multi-scale dynamic perception strategy for patch-level feature pyramid modeling. Second, to effectively handle anatomically similar patterns, we design a similarity-driven prototype learning mechanism that captures informative local semantics for robust offset prediction. Last, to stabilize the model learning and improve the overall performance, we incorporate a novel error-aware reliability regularization via tolerance-based balancing. We collected a large validation cohort, including two public and one private
datasets spanning X-ray and ultrasound modalities, covering cephalometric, symphysis-fetal head, and fetal heart landmarks. Extensive experiments demonstrate that PPOC-LL achieves satisfactory performance with a favorable trade-off between accuracy and model complexity.
Read the full abstract ▾
All authors Jingxian Xu, Yuhao Huang, Rusi Chen, Yanfeng Zhou, Dong Ni
Artificial Intelligence · Cross-listed from cs.CV
Not All Visual Tokens Are Equally Safe to Remove:Consequence-Sensitive Visual Token Compression
Jingbo Wen, Liang He, Mingyu Cao et al. (+4 more)
Visual
token compression for
vision–
language models (VLMs) has largely relied on criteria such as
attention, redundancy, and uncertainty to maximize average accuracy under a fixed compute budget, implicitly assuming that all errors carry equal cost. However, the consequence of an incorrect prediction on downstream tasks is rarely symmetric: misreading an invoice amount can be far more costly than misclassifying a background color. Motivated by this, we introduce consequence-sensitive visual
token compression, which allocates visual computation across requests according to their potential error costs. Our method follows a calibrate-then-allocate procedure, estimating consequence-specific error-budget curves offline and applying the calibrated
token budgets online using consequence signals available from question or task information. On a controlled within-task
benchmark, high- and low-consequence questions are drawn from the same document images, so content alone cannot reveal which questions are costly to get wrong. In this setting, our method reduces high-stakes errors from 0.300 to 0.133 under the same total
token budget, whereas a content-driven allocator performs no better than uniform allocation. Measuring how error rates change with
token budget across different cost ratios, we derive an allocation frontier: uniform allocation is optimal when errors are equally costly, and
token transfer toward high-consequence questions becomes increasingly beneficial as the cost gap grows. This allocation principle generalizes well across three dense
vision-
language benchmarks, two budget realization mechanisms (
token deletion and resolution reallocation), two VLM architectures, and multiple
token selection strategies. On a realistic mixed workload, consequence-sensitive allocation reduces cost-weighted error by 38% while achieving approximately 21% lower latency than full-resolution inference.
Read the full abstract ▾
All authors Jingbo Wen, Liang He, Mingyu Cao, Haoyu Wang, Minxuan Hu, Kangning Cui, Xilu Wang
Artificial Intelligence
From Relevance to Execution Utility: Reward-Aware Dynamic Execution Gating for Skill-Based LLM Agents
Liang He, Jingbo Wen, Hongyu Gu et al. (+5 more)
Agent skills are increasingly used to equip large
language model (LLM)
agents with reusable procedural knowledge. Although recent work has substantially improved skill
retrieval due to the increasing skill libraries, retrieving a plausible skill bundle does not guarantee that executing it is worthwhile. Since every skill-conditioned rollout is computationally expensive, deciding whether a retrieved bundle should be executed has become an increasingly important challenge. To this end, we introduce the Reward-Aware Dynamic Execution Gate (RADEG), a lightweight, retriever-agnostic decision layer between skill
retrieval and
agent execution. RADEG learns a low-cost surrogate model that predicts the execution utility of a query–bundle pair before the expensive rollout is launched. To obtain informative supervision while controlling for task difficulty, we locally perturb each retrieved bundle by deleting, adding, or replacing one skill, producing matched same-query rollouts that isolate the effect of bundle composition on verifier reward. During deployment, RADEG updates only a warm-started logistic head as new verifier feedback becomes available, enabling inexpensive adaptation of the execute/skip boundary without retraining either the retriever or the
agent. Under a query-level held-out evaluation on 288 collected rollouts, RADEG substantially reduces unnecessary
agent executions while preserving a large fraction of the downstream verifier reward. It consistently outperforms relevance-based and random gating across different execution budgets, demonstrating that execution-aware surrogate modeling provides a practical and cost-effective complement to skill
retrieval.
Read the full abstract ▾
All authors Liang He, Jingbo Wen, Hongyu Gu, Hao Li, Haoyu Wang, Yixiong Chen, Kangning Cui, Xilu Wang
Artificial Intelligence
CIDER: A Dataset of Contextual Disclosure Boundaries for Privacy Preference Alignment
Bingcan Guo, Eryue Xu, Jijie Zhou et al. (+2 more)
Aligning large
language models (LLMs) with human
privacy preferences requires capturing individuals' disclosure boundaries beyond general
privacy norms. However, a gap remains in eliciting such nuanced preferences to evaluate
alignment in realistic settings. We introduce CIDER, a
dataset of 14,850 human annotations from 169 users, forming 1,650 contextual disclosure boundary sets across 60 interpersonal communication scenarios involving information sharing that violates
privacy norms. Each boundary represents a real user's disclosure decisions over 9 sharing variants in a scenario, for a given communication role and AI-mediated condition. We formulate a task in which models predict a user's disclosure decision from historical boundaries, with varying levels of contextual information. Across 12 open and proprietary models, in-context personalization improves prediction accuracy by up to 11.41 percentage points using only 6 historical examples. Larger models such as GPT-5.4 (with medium
reasoning effort) and Claude Sonnet 4.6 are better at leveraging semantic context to understand user-specific, context-dependent disclosure preferences for more accurate predictions, while smaller models tend to rely on structured heuristics based on disclosure granularity and identifiability. Personalization generally improves prediction accuracy, but the improvement is often accompanied by imbalanced shifts in false-positive and false-negative rates across models, with only Claude Sonnet 4.6 achieving balanced improvements in both. Our findings reveal both the promise and limitations of inference-time personalization for
privacy preference modeling and position CIDER as a resource for advancing personalized
privacy alignment.
Read the full abstract ▾
All authors Bingcan Guo, Eryue Xu, Jijie Zhou, Zhiping Zhang, Tianshi Li
Artificial Intelligence · Cross-listed from cs.LG
Tabular Numeric Stretch Transformation
Zihao Ye, Juyong Kim, Johnna Sundberg et al. (+2 more)
Tabular data presents unique challenges for deep learning due to its heterogeneous nature, where numeric features exhibit diverse distributions, scales, and statistical properties. Although recent advances have improved how models learn from tabular data, how numeric data are transformed into model-friendly representations remains comparatively underexplored. We introduce the stretch transformation framework, which formulates numeric feature preprocessing as an optimization problem to make the target function smoother and thus more learnable. Our framework has two variants: (1) unsupervised stretch, which uniformly redistributes feature density via minimax optimization, and (2) supervised stretch, which optimizes target-aware numeric feature transformations from the perspective of target-function smoothness by minimizing the target function's Dirichlet energy in the transformed space. Our theoretical analysis further connects this framework to several popular transformations: unsupervised stretch is closely related to Piecewise Linear Encoding through a shared piecewise-linear geometry and approaches the empirical CDF transformation as the number of bins grows, while supervised stretch becomes closely related to target encoding in the fine-binning limit. Comprehensive experiments on 38
datasets from the TALENT
benchmark demonstrate that supervised stretch consistently outperforms all baselines. These results show that explicitly optimizing for target function smoothness is a powerful and underexplored strategy for tabular deep learning.
Read the full abstract ▾
All authors Zihao Ye, Juyong Kim, Johnna Sundberg, Burak Varici, Pradeep Ravikumar
Artificial Intelligence · Cross-listed from cs.SD
From Inaudible Inputs to Model Failures: Low-Frequency Safety Risks in LALMs
Yuanhe Zhang, Weiliu Wang, Jie Ren et al. (+7 more)
Large audio-
language models (LALMs) have demonstrated strong capabilities in understanding diverse audio inputs. This diversity includes low-frequency signals that are inaudible to humans but can still enter the model and influence its generation. However, the practical impact of such low-frequency inputs on LALMs remains largely unexplored. In this paper, we propose Intermittent Low-Frequency Lockout (ILL), an inaudible red teaming method that evaluates this risk using a universal waveform template in a black box setting. ILL uses Sentence
Attention Scale Estimation to determine active intervals and Frequency Confusion Transfer to construct a low-frequency waveform with continuous phase from corpus spectral variation. To mitigate this risk, we propose Distributional Requery Guard (DRG) to detect low-frequency distribution shifts and conditionally request a second recording for semantic recovery. Across six LALMs and multiple audio understanding tasks, ILL reduces accuracy by up to 67 percentage points while receiving a mean human audibility rating of 1.33, close to 1.17 for clean audio; DRG raises mean attacked accuracy from 28.5% to 46.1% after clean reacquisition. These findings identify a previously overlooked safety risk for LALMs and provide a foundation for future research on robust audio understanding.
Read the full abstract ▾
All authors Yuanhe Zhang, Weiliu Wang, Jie Ren, Liang Lin, Zhenhong Zhou, Haoran Gao, Kun Wang, Chen Li, Li Sun, Sen Su
Artificial Intelligence
TRACE: TRajectory Attribution for Automated Context Engineering
Yikai Zhao, Pradeep Kumar Misra and Saurabh Pandey
Production AI
agents fail when their context sources – system
prompts, knowledge bases, tool descriptions, and procedural skills – contain errors or gaps. Current maintenance relies on manual log review and ad-hoc debugging, creating a scalability bottleneck as interaction volume grows. We present TRACE (TRajectory Attribution for Automated Context Engineering), an automated feedback loop that mines historical
agent trajectories to diagnose and remediate context failures. Our key insight is that trajectories are rich with implicit dissatisfaction signals – user corrections, rephrasing, abandonment cues – that reveal precisely where context sources failed, without explicit feedback collection. Unlike model fine-tuning, TRACE operates on the context layer, enabling rapid iteration without retraining. We make four contributions: (1) a trajectory mining framework that systematically extracts diagnostic information from historical
agent executions; (2) multi-component causal attribution that extends textual gradients from monolithic
prompt optimization to heterogeneous context sources (skills, knowledge bases, tools,
prompts); (3) exploratory verification, where
agents actively read context sources to distinguish content gaps requiring CREATE from stale content requiring UPDATE, achieving 96% operation accuracy; and (4) a reusable simulation methodology and verifiable
benchmark addressing the absence of open
datasets for context debugging, with a six-category fault taxonomy, ground truth annotations, and a cross-layer verification protocol. On 60 dissatisfaction traces spanning three complexity tiers (up to 16 execution nodes), TRACE achieves 72.7% root cause attribution and 82% end-to-end fix effectiveness, showing that over 80% of context-layer failures can be automatically diagnosed and remediated by mining historical trajectories, an overlooked resource in production systems.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.RO
SpeedTuning: Speeding Up Policy Execution with Lightweight Reinforcement Learning
David D. Yuan, Tony Z. Zhao, Kaylee Burns et al. (+1 more)
While learned robotic policies hold promise for advancing generalizable manipulation, their practical deployment is often hindered by suboptimal execution speeds. Imitation learning policies are inherently limited by hardware constraints and the speed of the operator during data collection. In addition, there are no established methods for accelerating policies learned via imitation, and the empirical relationship between execution speed and task success remains underexplored. To address these issues, we introduce SpeedTuning, a reinforcement learning framework specifically designed to enhance the speed of manipulation policies. SpeedTuning learns to predict the optimal execution speed for actions, thereby complementing a base policy without necessitating additional data collection. We provide empirical evidence that SpeedTuning achieves substantial improvements in execution speed, exceeding 2.4x speed-up, while preserving an adequate success rate compared to both the original task policy and straightforward speed-up methods such as linear interpolation at a fixed speed. We evaluate our approach across a diverse set of dynamic and precise tasks, including pouring, throwing, and picking, demonstrating its effectiveness and robustness in enhancing real-world robotic manipulation.
Videos and code are available at https://daivdyuan.github.io/speed-tuning/
Read the full abstract ▾
All authors David D. Yuan, Tony Z. Zhao, Kaylee Burns, Chelsea Finn
Artificial Intelligence · Cross-listed from cs.CV
When Latents Forget Pixels: Restoring Fidelity in Diffusion Transformer Super-Resolution
Yu Shi, Yuyao Zhang and Yu-wing Tai
Image super-resolution (SR) with large generative models has recently achieved remarkable perceptual quality, yet maintaining fidelity to the LR observation remains challenging. In particular, we observe that
diffusion transformers (DiTs) built on latent representations suffer from a critical limitation: the compression bottleneck of the VAE weakens fine-grained spatial information, leading to hallucinated details that are weakly grounded in the input image. In this work, we revisit generative SR from a representation perspective and propose a pixel-grounded super-resolution (PGSR) framework that preserves LR-observed pixel evidence before VAE compression and reuses it throughout restoration. Instead of relying solely on the compressed latent condition, PGSR extracts pre-VAE pixel evidence from the upsampled LR image and reuses it at two stages. First, Condition-Side Trajectory Guidance fuses LR-derived pixel evidence with the latent LR condition to guide the latent restoration trajectory. Second, Decoder-Side Pixel Grounding injects multi-scale pixel features into the frozen VAE decoder to ground the final rendering with LR-observed cues. To efficiently adapt large pretrained DiT models, we keep the latent autoencoder and main flow-matching backbone frozen, and train only lightweight restoration modules. We further study an efficient local-window
attention variant for improved high-resolution efficiency and scalability. Extensive experiments demonstrate that PGSR improves the realism–fidelity trade-off and produces more faithful, visually convincing results than existing latent generative SR approaches.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
MARA: Flow-Matching-Guided Multi-Agent Resource Allocation for Computational Resource Efficient Learning
Hanye Zhao, Muning Wen, Yong Yu et al. (+1 more)
Allocating limited computation among concurrent learning tasks is difficult when each task must reach a target loss before a deadline but its required training effort is unknown. Existing approaches combine online loss prediction with adaptive resource allocation, yet commonly treat computation as continuously divisible throughput. We instead study a practical setting in which tasks arrive over time and computation is provided by discrete nodes. This setting introduces both uncertain demand and constrained sequential decisions. We propose MARA, which predicts future loss trajectories with conditional flow matching and coordinates compute nodes through a cooperative multi-
agent autoregressive policy. A potential-based progress reward supplies intermediate training feedback while preserving the undiscounted task-completion objective. Across in-distribution, reinforcement-learning, and
vision workloads, flow matching reduces remaining-resource prediction error relative to weighted least squares. At the scheduler's training load, MARA completes 63.46% of tasks on average, 8.54 percentage points above strong baseline Learning with Adaptive Resource Allocation (LARA), and remains ahead under unseen heavier workloads.
Read the full abstract ▾
All authors Hanye Zhao, Muning Wen, Yong Yu, Weinan Zhang
Artificial Intelligence · Cross-listed from cs.CL
Social Gym and SPaRTan: Benchmarking and Improving LLM Social Reasoning via Multi-Agent Game Tournaments
Keyu He, Xuhui Zhou and Maarten Sap
LLM
agents are increasingly deployed in multi-
agent social settings where they must cooperate, negotiate, and adapt to other
agents. Measuring and improving these social skills is hard because, unlike math or logic, social interaction offers no objective ground truth: evaluations fall back on LLM judges, which are costly, subjective, and noisy, and models get no reliable signal to learn from. To address both, we first introduce Social Gym, an environment of 21 multi-
agent social games (e.g., Werewolves, Resistance, Spyfall) whose rule-decided outcomes make
agent performance verifiable and objective, with an Elo tournament that produces a cross-game leaderboard. Benchmarking experiments show that while GPT-5-mini tops the leaderboard, no model excels at all games uniformly or in all game roles, pointing to limitations of social
reasoning. Motivated by this, we additionally propose SPaRTan (Self-Play and Reflect-Transfer), a training-free self-improvement loop: a model plays a game, reflects on its trajectories and their outcomes to produce a transferable playbook, and applies that playbook in subsequent games. Our results show that SPaRTan playbooks help GPT-5-mini
agents level their performance on weaker roles, but largely do not improve Qwen3-32B's performance. Together, Social Gym and SPaRTan offer a reproducible, verifiable foundation for measuring and improving LLM social
reasoning without weight updates.
Read the full abstract ▾
Artificial Intelligence
ChronoState: Hidden Elapsed-Time Conditioning for Temporal-State Action Selection in Frozen-Backbone Language Models
Sam Siavoshian, Omar Ramadan, Amir K. Saeed et al. (+3 more)
Temporal decisions in
language-model systems often depend on both symbolic task state and elapsed wall-clock time, such as cache expiration, job completion, quota resets, deadlines, or stale sessions. We study whether elapsed time can be supplied as a non-
token, system-side scalar and composed with visible symbolic state by a frozen-backbone
language model. We introduce ChronoState, a compositional temporal-state
benchmark in which symbolic state appears in the
prompt, elapsed seconds tau are supplied through a hidden chronometric-injection channel, and the model selects a forced-choice temporal action. Here, "hidden" means hidden from the user-visible
token sequence, not from model computation. Using Qwen2.5-3B-Instruct as a frozen bf16 backbone with a 31-dimensional sinusoidal-plus-log time encoding, gated FiLM residual modulation, and a rank-8 LoRA action surface, hidden-time CI reaches 0.9305 +/- 0.0134 accuracy and 0.9410 +/- 0.0103 balanced accuracy. No-time and shuffled-time controls fall to 0.5511 +/- 0.0042 and 0.3323 +/- 0.0097, respectively, with high shuffled-time wrong-state consistency supporting causal dependence on the injected scalar within the trained distribution. Generalization remains strong for held-out templates, durations, and multi-constraint compositions, but held-out quota-family transfer is weak at 0.5065 +/- 0.0559, while a fair
prompt+LoRA timestamp baseline reaches 0.9893 +/- 0.0052. Thus, ChronoState supports a narrow conclusion: hidden elapsed time can be composed with symbolic task state under direct supervision, but does not establish autonomous time tracking, broad unseen-family abstraction, or superiority over
prompt-injected timestamps.
Read the full abstract ▾
All authors Sam Siavoshian, Omar Ramadan, Amir K. Saeed, Benjamin A. Johnson, Amin Mohamed El-Amin Diab, Benjamin M. Rodriguez
Artificial Intelligence
RISE-RL: Rubric-Informed Selective Exploration for Open-Ended Reinforcement Learning
Jinkun Hou, Zhuo Liu, Huimin Ren et al. (+3 more)
Aligning Large
Language Models (LLMs) for open-ended tasks is challenging because responses must satisfy multidimensional criteria without following a single correct generation trajectory. Existing rubric-based reinforcement learning (RL) methods compress fine-grained criterion-level feedback into scalar rewards, making persistent capability gaps difficult to target under limited on-policy exploration. We propose $\textbf{RISE-RL}$ (Rubric-Informed Selective Exploration), which uses repeatedly missed rubric criteria to elicit privileged trajectories that are difficult to discover through unguided exploration alone. RISE-RL retains only trajectories whose complete-rubric reward exceeds the mean reward of natural rollouts, and then re-evaluates them under the original
prompt to emphasize behaviors that remain weakly supported by the natural policy. The resulting guidance signal is optimized through a separate auxiliary objective and removed once its additional benefit diminishes. Experiments with 4B and 14B models across writing, chat, health, and science show that RISE-RL achieves the highest mean score on every evaluated
benchmark under guidance-free evaluation. Compared with standard Rubric-RL, it improves the average score by 1.3 points at the 4B scale and $\textbf{3.3 points at the 14B scale}$, including a $\textbf{6.0-point}$ gain on CreativeWriting-V3. It also improves creative-writing diversity and yields gains on objectively scored medical and scientific
benchmarks. These results indicate that selective internalization through reward filtering and policy support shaping is effective for open-ended reinforcement learning.
Read the full abstract ▾
All authors Jinkun Hou, Zhuo Liu, Huimin Ren, Hongsheng Xin, Pan Zhou, Kun Zhan
Artificial Intelligence · Cross-listed from cs.CV
Visual Distortion Detection in UGC Images Using Large Multimodal Models
Ziheng Jia, Yingji Liang, Jiaying Qian et al. (+1 more)
The localized depiction of perceptual quality has long been a crucial, yet underexplored, challenge in image quality assessment (IQA). Existing approaches based on large
multimodal models (LMMs) predominantly rely on text-driven supervised fine-tuning (SFT). However, this training paradigm exhibits notable limitations in detection accuracy. Moreover, synthetically distorted images, which are often used as the primary training data source, show a significant generalization gap when deployed in real-world scenarios; thus, the \textbf{synthetic-to-authentic (S2A)} problem represents a critical challenge. Motivated by these issues, we propose \textbf{VIGIL}, which leverages the LMM architecture for precise visual distortion detection. From a candidate pool of over 1000K samples, we construct the \textbf{VIGIL-140K} training set, which consists of over 140K distorted images. These images are obtained through rigorous quality filtering and carefully crafted distortion injection, covering 8 major synthetic distortion categories. Our model leverages different layers of the large
language model (LLM) decoder, treating them as multiple detectors that perform synchronous distortion detection using multi-level features. Additionally, we retain distortion cues from predictions assigned to the non-distortion class, which helps mitigate the ambiguous foreground-background (FG-BG) separation commonly encountered in the S2A problem. After post-processing, our model consistently outperforms strong baselines on both in-domain synthetic distortion detection and S2A tasks.
Read the full abstract ▾
All authors Ziheng Jia, Yingji Liang, Jiaying Qian, Xiongkuo Min
Artificial Intelligence
Ruiyu Li, Haoyang Cai, Zhitong Guo et al. (+1 more)
Web
navigation agents are capable of addressing various types of tasks on different websites. Current baselines on web
navigation are either unimodal or lack strong
reasoning abilities given
multimodal inputs. Focusing on the WebShop
benchmark, a real-world website simulation, we explore the
alignment of text and images, as well as
multimodal reasoning and planning abilities, to enhance the performance of web
navigation agents. We propose three innovative
multimodal enhancements:
Multimodal Enhanced LLM for Online
Navigation (MELLON), VQAgent, and
Multimodal Ranker. MELLON demonstrates a significant improvement in task completion accuracy, with a 9.26% increase after just one epoch of training. Our findings suggest the necessity of further exploration into
multimodal approaches, with a focus on more extensive training and
alignment strategies to enhance the effectiveness of web
navigation agents.
Read the full abstract ▾
All authors Ruiyu Li, Haoyang Cai, Zhitong Guo, Tong Hu