Artificial Intelligence · Cross-listed from cs.CL
Vela: User Behavior as Supervision Sparks LLM Intelligence
Yang Xiao, Yusong Sun, Haoyi Wu et al. (+17 more)
Large
language models are increasingly deployed in applications that act on users' long-term behavior histories, from personal assistants to recommendation and advertising systems. These applications demand the user dimension of LLM intelligence: inferring a person's preferences, intent, and likely future actions from weeks to years of behavior data. Yet current models are trained on static text or synthetic data, neither of which captures how user behavior evolves over time. Real-world behavior records, already available at scale on content platforms and in personal-assistant
agents, offer a natural supervisory signal. We propose user behavior as supervision and introduce Vela, a framework that converts real user behavior data into trainable tasks and trajectories. From the anonymized behavior records of 100,000 users, Vela generates 100,000 tasks across 91 real-world task templates; difficulty-based filtering and trajectory acceptance criteria retain approximately 75,000 tasks with selected training trajectories. Fine-tuning Qwen3.6-35B-A3B on these trajectories improves performance over the base model by up to 11.5 points on user-related tasks and by an average of 4 points on general
benchmarks. Training on more user behavior data further improves user-related performance, and reinforcement learning after fine-tuning outperforms reinforcement learning from the base model by up to 7.7 points. These gains suggest that real-world user behavior is a scalable source of supervision for both user-centered capabilities and the broader intelligence of LLMs.
Read the full abstract ▾
All authors Yang Xiao, Yusong Sun, Haoyi Wu, Wenyang Hui, Weiqiang Hang, Luqing Wang, Zejian Wang, Zirui Tu, Jingsheng Mao, Shaobo Hu, Yang Luo, Jiale Guo, Jin Zhang, Yudan Liu, Zhaokai Luo, Ren Huan, Yao Hu, Chengyue Jiang, Wenjie Li, Chuan Mu
Artificial Intelligence
How to Build a Benchmark That Actually Measures Taste
Xiaofan Wu, Xi Zeng, Miaoxia Chen et al. (+8 more)
Expert-aligned "taste"
benchmarks should reward craft judgment, not the ability to game the test. We introduce TasteBench, a pairwise good-vs-bad
benchmark across visual, brand, book, and tech items under a frozen rubric with expert-only gold (panel n>=3; percent agreement >=0.60 primary gate; weak engagement labels train/analysis-only), together with a control protocol that separates craft judgment from test-gaming. Applying the protocol to a Round-1 pilot (primary_human_expert; three biological human experts; n=250 pairs) surfaces and quantifies three validity threats: two construction artifacts that inflate the trivial-baseline layer, and a difficulty ceiling that inflates every score. (1) Position: the preferred item is slot A with probability 0.956, so an always_item_a policy scores 0.956 (fair chance 0.5). (2) Lexical leakage: a filename
token cue scores 0.912. (3) Difficulty ceiling: craft domains show near-ceiling human agreement (book/brand/tech 1.0), and even under A/B randomization (eval_assignment_seed7; gold_side_A ~0.47) all five VLM cards reach 1.0 on craft, with only visual separating (~0.84-0.91; a post-hoc hard subset, n=107, leaves craft at 1.0). The protocol (expert-only gold, a weak-label firewall, side randomization at construction, filename scrubbing, and hard filters) makes each artifact measurable and, for the two construction artifacts, correctable; we release a reference eval harness and a side-randomization tool, and show the tool removes the position skew (0.916 to 0.495/0.505). The result is a calibrated view of what the
benchmark does and does not yet measure: position and leakage are fixable, whereas the craft-domain ceiling is a pair-difficulty limit that Round-2 must address. We do not evaluate engagement, era/circle conditioning, or calibration here, and we claim no general taste SOTA.
Read the full abstract ▾
All authors Xiaofan Wu, Xi Zeng, Miaoxia Chen, Peishan Chen, Shuyan Li, Chen Wang, Xiaohan Wang, Wanling Xu, Jiyun Yao, Hanyong Zhong, Jiahao Zhu
Artificial Intelligence · Cross-listed from cs.CL
Lingdan-V2: Large language models for clinically grounded reasoning in traditional Chinese medicine
Rui Hua, Yu Wei, Ziri Jin et al. (+15 more)
Large
language models have advanced medical
reasoning, but clinically grounded
reasoning in traditional Chinese medicine (TCM) remains challenging because prescription decisions require integrating domain knowledge, syndrome differentiation, treatment principles, herbal prescription composition, and dosage determination. We develop a TCM
language model family comprising models with 4 billion, 8 billion, and 14 billion parameters. The artificial intelligence contribution is a staged clinical
alignment framework integrating large scale continued pretraining on TCM and clinical records, complementary
reasoning supervision, and prescription-centered reinforcement learning (RL). We further formulate herbal prescription generation as a structured RL problem with prescription-level feedback, enabling direct optimization of clinical decision outputs beyond knowledge reproduction and generic instruction following. The engineering application is TCM clinical decision support, particularly herbal prescription recommendation. We evaluate the models on a public
benchmark containing 25,620 expert-verified items across 13 TCM knowledge and clinical
reasoning tasks and on a private retrospective
dataset of 4,348 spleen and stomach disease prescription cases. The 14 billion parameter reinforcement aligned model achieves a prescription generation F1 of 37.0 on the public
benchmark, compared with 32.3 for DeepSeek-R1 with 671 billion total parameters, and a herb-level F1 of 52.8 on the clinical
dataset with improved dosage agreement. Reinforcement post-training consistently improves prescription recommendation across all scales, while physician-source analysis shows that expert heterogeneity affects reward stability. These findings demonstrate a scalable pathway from domain grounding and
reasoning supervision to clinically oriented prescription decision support.
Read the full abstract ▾
All authors Rui Hua, Yu Wei, Ziri Jin, Zhuo Sun, Kai Chang, Jianan Xia, Zixin Shu, Xiaodong Li, Dongmei Jia, Fei Dong, Runshun Zhang, Jian Yu, Hao Xu, Haibin Yu, Jiansheng Li, Baoyan Liu, Wenjia Wang, Xuezhong Zhou
Artificial Intelligence · Cross-listed from cs.CR
Towards Merging-conditional Backdoor Unalignment against Large Language Models
Yukun Chen, Yidou Chen, Yiming Li et al. (+4 more)
Model merging has become widely adopted for enhancing the task-specific performance of Large
Language Models (LLMs) without additional training. While the specialized model from third-party platforms may undergo safety audits (e.g., backdoor detection), we reveal a novel and critical threat: One-sided Merging-Conditional Backdoor (OMCB). OMCB remains dormant and undetectable within one standalone model but activates upon merging, triggering safety unalignment. We first theoretically analyze the existence and
geometric location of OMCB, revealing that the parameter shift crossing safety decision boundaries is the main cause of backdoor activation. Building upon this insight, we propose SMOKE, a meta-learning-based attack framework. SMOKE implants OMCB via a bi-level optimization strategy: the inner loop synthesizes surrogate meta-vectors through random low-rank initialization and finite-step maximization, while the outer loop optimizes the target model by simulating the merging process. Furthermore, preliminary analysis shows that the injected OMCB has bounded generalization loss under unknown merging configurations. Extensive experiments across diverse model architectures and merging techniques validate the effectiveness of SMOKE and its resistance to existing backdoor detections, highlighting a significant yet previously overlooked vulnerability in the model merging paradigm.
Read the full abstract ▾
All authors Yukun Chen, Yidou Chen, Yiming Li, Xinyu Zhang, Baosong Yang, Hongwei Yao, Zhan Qin
Artificial Intelligence
DSABench: Benchmarking Source Discovery, Coordination, and Execution for Multi-Database Agents
Jinlong Luo, Haoran Wu and Bo Xu
Most text-to-SQL
benchmarks provide the target database. In a large data catalog, however, choosing the sources is part of the problem: an
agent must find every database needed for a request, determine how their results fit together, and execute the resulting workflow. We present DSABench to evaluate these capabilities jointly. Its 815 human-reviewed tasks draw on Spider, BIRD, and LiveSQLBench; 618 require several databases to contribute, and 105 allow more than one interpretation of the required sources. Executable contracts define source participation, cross-source dependencies, and source-local execution, while RouteEnv lets
agents inspect public catalog evidence without revealing the gold annotations. In our routing-only study across seven model backends, the highest RA-Set score is 60.55%, compared with a highest RA-Graph score of 34.66% when coordination requirements are included. The gap becomes more pronounced when execution is required. Across four backends, median non-privileged Route+Exec-Graph success ranges from 6.69% to 7.98% without clarification and from 11.44% to 13.25% with one clarification; the best observed clarified configuration reaches 16.99%. On ambiguous tasks, clarification improves exact branch-source recovery by 39.8-43.2 percentage points. These findings illustrate why retrieving plausible databases or producing correct local SQL is not enough: an
agent must carry the right information through the entire multi-database workflow.
Read the full abstract ▾
Artificial Intelligence
Joint Discrete and SE(3) Diffusion for Protein Interface Design
Zhenqiao Song, Jianwen Xie and Lei Li
The design of high-affinity, target-specific
protein binders is central to the development of modern therapeutics. However, existing
protein-generation methods remain limited in this setting, as many generated designs fail to simultaneously achieve
geometric and physicochemical complementarity at binding interfaces. To address this challenge, we introduce PIDiff, a physicochemically enhanced
multimodal diffusion framework for
protein-
protein complex design. PIDiff jointly designs target-conditioned
protein sequences and structures by coupling
diffusion in discrete sequence space with
diffusion on the continuous (SE(3)) structural
manifold. At its core, PIDiff employs PIFormer, a novel architecture that interleaves geometry-aware invariant point
attention with global sequence self-
attention. We further curate PIBench, a
protein-
protein complex
dataset with diverse target scaffolds derived from the
Protein Data Bank (PDB). We first pretrain PIDiff on PIBench and subsequently fine-tune it for two therapeutically relevant applications: minibinder design and antibody design. PPFrame consistently outperforms baseline methods across sequence design success rate, structure binding energy, and sequence-structure co-designability on all tasks.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.SD
AlignAudio: A Unified Dual-Alignment Framework for Noise-Robust Speech-Conditioned Generation
Yixuan Li, Xuenan Xu, Zeyu Xie et al. (+1 more)
Speech-conditioned generation maps
speech signals to target audio outputs. Prior end-to-end
speech-conditioned generative models achieve high fidelity in clean acoustic conditions, but their performance degrades substantially under environmental noise. While prior noise-robust approaches have achieved significant success in feature-level purification, maintaining consistent generation under clean and noisy conditioning remains a key challenge. To address this, we propose AlignAudio, a noise-robust generative framework based on dual
alignment: (i) aligning the representations of clean and noisy
speech to preserve semantic cues, and (ii) enforcing generation-level consistency within a flow matching paradigm to improve conditioning fidelity. Experiments on
speech-to-audio (STA) and voice conversion (VC) tasks demonstrate that AlignAudio maintains performance comparable to baselines on clean
speech while significantly outperforming them across various noisy conditions. Furthermore, the proposed dual-
alignment strategy is applicable to both single- and multi-stream
speech conditioning and can be integrated into different flow matching-based generation frameworks and post-training settings. Audio demonstrations and qualitative comparisons: \href{https://alignaudio.github.io/AlignAudio}{\textcolor{cyan}{https://alignaudio.github.io/AlignAudio}}.
Read the full abstract ▾
All authors Yixuan Li, Xuenan Xu, Zeyu Xie, Mengyue Wu
Artificial Intelligence
AcrossVAM2.0: Conflict-Aware Point Supervision for Parameter-Efficient Video Diffusion Adaptation
Nan Wu and Yee Hung Hong
Explicit point correspondence is attractive for adapting
video generators: it describes how visible surfaces move, but can overwrite the appearance prior learned by a large pretrained model. We study this tension in a 5.01B-parameter text-to-
video diffusion transformer. ACROSSVAM2.0 combines a confidence-weighted track-and-visibility head, zero-initialized residual feedback, and asymmetric PCGrad that removes only the auxiliary component opposing the primary flow-matching gradient on shared LoRA parameters. The design preserves the generator exactly at initialization and protects its primary update during adaptation. In a four-way ablation trained for 1,000 steps with three independent seeds, the full method lowers KLT forward-backward cycle error in all three seeds relative to a matched flow-only continuation (mean seed-level difference -0.00711, SD 0.00505), while robust sharpness remains positive in all three. An expanded 24-
prompt evaluation (216
videos) yields 9.7% lower aggregate CoTracker3 cycle error than the matched control and near-matched performance on six VBench dimensions; however, the seed-level tracking effect is heterogeneous and I3D-FVD regresses from 0.1824 to 0.2022. These results support a bounded claim: conflict-aware adaptation improves correspondence consistency without broad perceptual degradation, but does not uniformly improve
video quality. We release equations, pseudocode, matched outputs, source tables, and artifact provenance.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.RO
DexCast: Learning Real-World Dexterous Control from Human Videos
Zihan Gao, Yuhong Zhang, Mengzhe Wang et al. (+7 more)
Recent advances in 3D hand-object interaction (HOI) perception have opened new opportunities for learning dexterous robotic skills from human
videos. However, existing dexterous imitation learning methods often rely on costly and manually teleoperated
robot demonstrations, requiring substantial human effort. Therefore, we build DEXCAST, a framework that learns dexterous robotic hand control directly from monocular RGB-D
videos by leveraging joint HOI reconstructions, without requiring any teleoperation. Furthermore, how to model high-dimensional robotic hand motion and extract temporal context to inform future action generation remains a key challenge. To this end, we propose a DexHand Motion Perception Module (DMPM) to capture inter-joint coordination and temporal dependencies in compact latent representations that condition our
diffusion policy. Extensive real-world experiments demonstrate that DEXCAST achieves exceptional performance across diverse dexterous manipulation tasks.
Read the full abstract ▾
All authors Zihan Gao, Yuhong Zhang, Mengzhe Wang, Kaisheng Liu, Shengpeng Li, Siyao Xiao, Runqing Cheng, Xiao Lin, Zhifang Liu, Haoqian Wang
Artificial Intelligence
A Hybrid Kinematic GCN and Elastic Transformer for Efficient Indian Sign Language Recognition
Pushp Garg, Himangshu Sarma and Mrinmoy Ghorai
Sign
Language Recognition (SLR) presents complex challenges due to variable signing speeds and spatial intricacies. Specifically, Indian Sign
Language (ISL) requires precise modeling of its inherently bimanual nature and is bottlenecked by the scarcity of large-vocabulary
benchmarks and the inability of standard models to handle irregular signing velocities. To address this, a structurally-aware hybrid framework for efficient, gloss ISL recognition is proposed. To minimize computational redundancy, an
entropy-driven SVD sampler that dynamically extracts informative keyframes is used. Because this subsampling creates irregular physical time gaps, the proposed architecture employs a Continuous Dual Positional Encoding paired with a distance-aware Elastic
Transformer to preserve natural signing rhythm. Spatially, a Kinematically-Conditioned Adaptive GCN explicitly models bimanual geometry using unified bone-joint features. To validate our framework, a 120-class, 9,453-
video native ISL
dataset is created. Augmented by Physical Node Dropout and ArcFace optimization to mitigate subject-specific memorization, our framework achieves a competitive 89.8% accuracy, establishing a strong baseline for complex sign
language modeling.
Read the full abstract ▾
Artificial Intelligence
RASSG: Repair-Aware Semantic Survival Gating for Streaming Tool Execution
Daijin Hu and Xi Wang
Streaming
speech assistants can launch tools before users finish speaking, overlapping execution with incoming
speech to reduce user waiting time. However, subsequent self-corrections can invalidate apparently stable arguments, causing stale execution whose irreversible effects persist even after the final call is corrected. We propose Repair-Aware Semantic Survival Gating (RASSG) to jointly control tool launch and cancellation during streaming
speech. RASSG launches a tool when three consecutive
speech recognition results arrive with identical nonempty arguments, then combines calibrated argument invalidity risk with execution costs to decide cancellation at the last feasible opportunity before effects become irreversible. Evaluation on ReCAST, FDB-v3, and Audio2Tool uses a controlled execution clock with fixed tool runtimes and cancellation parameters. On ReCAST's prospective-repair subset, RASSG reduces the stale execution rate from LocalAgreement-2's 50.00% to 3.15%, with a mean delay of 3.676s across successful trajectories. The stale reduction also holds against every early-execution baseline on both public
speech datasets. Ablations with identical initial launches support active cancellation, showing the value of controlling ongoing calls before their effects become irreversible. Code and data are available at https://github.com/daijinhu/RASSG.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CV
Prospective Forcing: Learning to Look Ahead for Autoregressive Long Video Diffusion
Yifan Shen, Yewei Xia, Gongxu Luo et al. (+6 more)
Autoregressive
video generators synthesize long
videos through chunk-wise rollouts, yet they are typically trained to predict only the immediate future. This limited prediction horizon makes it challenging to maintain
video quality over extended rollouts. To address this problem, we introduce Prospective Forcing, a multi-head prediction framework that injects long-range predictive signals into autoregressive
video representations. Built on a rolling-window generator, Prospective Forcing attaches lightweight, offset-specific heads that share the same conditioning information and independently predict chunks at different future temporal offsets. By extending supervision beyond the next chunk, these heads encourage the shared history representation to retain information useful for longer-horizon prediction, supporting temporally consistent generation. At inference time, we accelerate sampling by using accepted future-chunk predictions to skip full-window backbone executions. We equip the head with learnable
embeddings to generate multiple candidates for the next chunk, and a learned verifier either selects a candidate or rejects all and falls back to standard autoregressive generation.
Read the full abstract ▾
All authors Yifan Shen, Yewei Xia, Gongxu Luo, Peiyuan Zhu, Boyang Sun, Zhenhao Chen, Guangyi Liu, Guangyi Chen, Kun Zhang
Artificial Intelligence
Shaping Latent Mixture-of-Experts: From Expert Geometry to Layerwise Capacity Allocation
Yaoxiang Wang, Kai Zhang, Ruobing Xie et al. (+4 more)
Fine-grained mixture-of-experts (MoE) models commonly increase expert count by narrowing each expert, treating expert shape as a by-product of routing granularity. LatentMoE moves expert computation into a lower-dimensional space, exposing the input dimension and intermediate width of each expert as explicit design choices. We show that an expert's local Jacobian rank is jointly bounded by these two dimensions, and that balancing them maximizes the rank ceiling at fixed per-expert parameters. Controlled shape sweeps across model scales and routing granularities consistently favor near-square experts. We further use layerwise intrinsic dimension (ID) to guide capacity allocation across depth, co-varying the number and shape of experts while keeping per-layer parameter count, activated FLOPs, and communication volume constant. Compared with layer-homogeneous baselines, our ID-guided allocation achieves lower validation loss and stronger downstream performance. These results establish expert shape and layerwise allocation as practical design axes for MoEs.
Read the full abstract ▾
All authors Yaoxiang Wang, Kai Zhang, Ruobing Xie, Lanrui Wang, Guoliang Zhao, Junhui Li, Jinsong Su
Artificial Intelligence
AegisVox: Factorized Privacy and Clinical Preservation for Long-Form Speech
Praphul Singh, Akshat Agarwal, Ratul Ghosh et al. (+5 more)
Clinical
speech reveals identity through both spoken identifiers and voice, yet its pauses, delivery, and speaker roles can also carry useful clinical evidence. Protecting that
speech therefore requires more than preserving its words. We present AegisVox, a factorized framework that assigns consistent surrogate voices and edits identifier-bearing turns using document-consistent replacements. Across 390 public simulated or non-patient conversations totaling 63.6 hours, a matched comparison of two
speech renderers separates replacement realization from retained identifiers, speaker linkage, and clinical preservation. Text-to-
speech rendering realizes 535/601 planned replacements versus 453/601 for the codec-based
speech editor. However, similar aggregate clinical preservation conceals more model-labeled contradictions, and conversation-wide speaker scores conceal greater edited-
speech linkage under a score-direction sensitivity analysis. Fresh transcription of original audio provides a reference for interpreting the clinical measurement chain; an independent judge tests sensitivity to model-based assessment. Better replacement realization can therefore coexist with worse clinical or speaker-linkage measurements. AegisVox supports use-specific evaluation of those tradeoffs, not a guarantee of identifier removal or clinical safety.
Read the full abstract ▾
All authors Praphul Singh, Akshat Agarwal, Ratul Ghosh, Aashna Kanuga, Devashish Khatwani, Anurag Dwarakanath, Sri Gadde, Krishnaram Kenthapadi
Artificial Intelligence
PEARL: Process-Experience Aware Reinforcement Learning for Multi-Turn Agents
Jie Yang, Mianqiu Huang, Yiwei Zhao et al. (+3 more)
Terminal rewards make multi-turn
agents trainable without step-level annotations, but group-relative advantages provide little guidance about which intermediate decisions to adjust. Failed trajectories also contain reasonable actions, while unfavorable decision patterns can recur as training proceeds. We introduce PEARL (Process-Experience Aware Reinforcement Learning), which improves local credit assignment without an additional critic. Given two continuations from the same history with different outcomes, PEARL locates their first divergence and builds a feature representation of each branch's local decision. It clusters these decision features in recent failure and success
memories to identify recurring local patterns. Using this pattern
memory, PEARL selectively strengthens negative advantages at lower-reward decisions, while leaving positive advantages and the base policy optimizer unchanged. On AppWorld, PEARL improves over ARPO on Normal and Challenge tasks at both 4B and 8B scales. On ALFWorld, it achieves 97.9% mean success at 4B, 2.7 points above the strongest baseline, while remaining competitive at 8B. Ablations of shaping location, success-bank filtering, and
memory window support the value of selective, recurring-pattern-based updates.
Read the full abstract ▾
All authors Jie Yang, Mianqiu Huang, Yiwei Zhao, Bo Wang, Yang Liu, Yuxin Wang
Artificial Intelligence
From Existing Methods to New Research Ideas: Two Perspectives on Recursive Self-Improvement
Leijun Zhou, Yanke Yu, Zhihao Liu et al. (+10 more)
Recursive self-improvement (RSI) offers a route to AI systems that participate in their own continued development. Recent methods allow
language agents to retain changes to their parameters,
memory, skills, and code for subsequent execution and improvement. We examine how these methods can inform new research ideas through two complementary perspectives. First, we introduce a unified taxonomy with nine design dimensions organized into four groups: update targets, update organization, update schedule and scope, feedback and selection. This taxonomy places existing methods in a shared design space, where examining individual dimensions and their interactions can inspire new research ideas. Second, we examine how existing RSI methods draw on traditional deep learning, for example through optimization, regularization, data augmentation, continual learning, and
adversarial learning. We develop further ideas by examining more specific learning mechanisms and drawing on evolutionary computation. We also review how existing
benchmarks assess self-improvement under online, offline, and offline-to-online settings. Together, these perspectives help researchers develop new RSI research ideas from existing designs and learning mechanisms. RSI papers and resources are available at https://prism-shadow.github.io/awesome-rsi/.
Read the full abstract ▾
All authors Leijun Zhou, Yanke Yu, Zhihao Liu, Qu Xiang, Tiancheng Hu, Yiru Dong, Yihan Wu, Yangxi Xiang, Xuejun Wu, Xi Chen, Buyue Qian, Yaowei Zheng, Junhao Hu
Artificial Intelligence · Cross-listed from cs.CV
EarthAlign: Learning Language-Aligned Representations of Earth System Data
Fadillah Adamsyah Maani, Numan Saeed, Jiaqi Ma et al. (+4 more)
Interpreting Earth system conditions is central to understanding environmental risks and informing decisions across weather- and
climate-sensitive sectors. Existing
multimodal weather systems primarily target task-specific prediction, generation, or question answering, while general-purpose
alignment between Earth system data and
language remains underexplored due to the limited availability of diverse
language supervision grounded in meteorological conditions. To fill this gap, we introduce EarthAlign, a large-scale
dataset with broad geographic coverage spanning 2000-2024 that combines 10 expert-curated natural event and disaster data sources with climatology-derived normal-weather samples. Each sample includes temporal and geospatial information and four grounded caption styles aligned with 63-variable ERA5 meteorological data. Building on EarthAlign, we introduce MeteoLIP, a dual-encoder framework for learning aligned meteorology-
language representations. MeteoLIP incorporates MetTok, which uses a geolocation- and time-conditioned query to aggregate variable-specific patch
embeddings into a compact sequence of meteorology
tokens. To comprehensively assess Earth system
multimodal representations, we introduce three downstream tasks: cross-modal
retrieval, weather-condition classification, and physical variable estimation. Our experiments show that MeteoLIP improves upon the corresponding CLIP and SigLIP baselines across these tasks. Further analyses highlight the importance of meteorological input, temporal and geospatial conditioning, and multi-style supervision for meteorology-
language alignment. Thus, EarthAlign and MeteoLIP provide a foundation for learning and evaluating
language-aligned representations of Earth system data. Our code,
dataset, and weights are publicly available at https://github.com/mbzuai-oryx/EarthAlign.
Read the full abstract ▾
All authors Fadillah Adamsyah Maani, Numan Saeed, Jiaqi Ma, Dat Tien Nguyen, Youssef Wehbe, Duncan Watson-Parris, Salman Khan
Artificial Intelligence · Cross-listed from cs.IT
Limitations on Autoregressive Token Generation via Sequential Sampling
Gal Arnon, Andrej Bogdanov, Paul Lou et al. (+3 more)
We study autoregressive
token generation through the notion of sequential samplers: A
circuit that samples the next symbol only given the previous ones and possibly a small amount of state. We show that (1) a sequential sampler with $k$ bits of state can be simulated by a stateless one at a roughly $O(2^{k})$ size overhead; (2) efficient insertional sampling (Stern, Chan, Kiros, Uszkoreit, ICML 2019) is more powerful than efficient sequential sampling in any fixed order; (3) sequential samplers with small state cannot establish certain forms of covert cryptographic channels. Result (1), together with the pseudoentropy vs. sequential sampleability duality of Vadhan and Zheng (STOC 2012), characterizes efficient sequential sampleability with small state. Result (3) provides a potential safety mechanism against subliminal coordination among malicious LLM
agents.
Read the full abstract ▾
All authors Gal Arnon, Andrej Bogdanov, Paul Lou, Giulio Malavolta, Alon Rosen, Lorenzo Taschin
Artificial Intelligence
Decoupling Knowledge Forgetting and Refusal Learning in MLLM Unlearning
Haodong Chen, Shengtao Wen, Dong Liang et al. (+2 more)
Current evaluations of
multimodal unlearning typically focus on whether a model stops outputting target information. This surface-level criterion, however, masks two critical failure modes. First, knowledge suppression techniques often induce confabulation, leading the model to hallucinate facts about the unlearned subject. Second, refusal training merely alters observable behavior without erasing the underlying knowledge; such models might decline open queries but still identify correct answers under multiple-choice probing. To overcome these limitations, we propose Forget-Aware Self-
Distillation (FASD). This method is designed to achieve honest ignorance, a state where the model genuinely loses the target knowledge and learns to acknowledge its absence. FASD separates these goals into two complementary channels. The behavior channel distills appropriate refusals from an ignorance-prompted copy of the model itself. Crucially, target
tokens are masked from the teacher distribution to avoid accidentally reteaching the forgotten concepts. Simultaneously, the knowledge channel directly suppresses ground-truth answer spans independently of the model's generated responses. Experiments demonstrate that FASD successfully unlearns target knowledge, prevents information leakage in open-ended tasks, and preserves performance on retained data. The resulting model not only forgets the specified information but accurately recognizes the boundaries of its own knowledge. Code and data will be released upon publication.
Read the full abstract ▾
All authors Haodong Chen, Shengtao Wen, Dong Liang, Lingbing Guo, Xiang Chen
Artificial Intelligence
Iterate on an LLM, Serve on an SLM
Keegan Wang
As frontier LLMs gain more capabilities, the cost to run them also goes up. At the same time, small
language models (SLMs) also gain more capabilities but have largely been able to keep their cost stable, offering 5-40 times cheaper inference for the narrow, repetitive subtasks that
agents decompose work into. However, the standing limiting factor is getting the correct behavior out of a small model in a fast, efficient, and economical way. Unfortunately, in most cases, zero-shot small models fail narrow production tasks. Furthermore, retraining one every time the behavior changes is too slow to iterate on. We propose doing both in sequence. First, develop the desired behavior in
prompt space, on a frontier model, with immediate feedback from live traffic. Then, once the behavior converges, distill the recorded production behavior (not synthetic imitations of it) into an SLM through supervised fine-tuning (SFT). We show a proper use case of this method through a scheduling
agent for home-care agencies that fans out a single yes/no safety decision per caregiver-shift pair on GPT-5.4-nano. A LoRA-tuned Qwen3-8B trained on 11.6k recorded production decisions reaches 0.913 agreement, while the teacher (GPT-5.4-nano), re-run on its own recorded
prompts, agrees with itself only 72.7% of the time, pointing to the fact that the SLM may also be more consistent. Furthermore, when the two disagreed, a blinded professional scheduler sided with the student 48-29. Finally, we find that the SLM has approximately 2.5 times lower median latency and is a fraction of the cost. We believe this lifecycle of quick building, live feedback, and cheap inference is a blueprint for others building similar systems.
Read the full abstract ▾
Artificial Intelligence
CB-Agent: Concept-Based In-Context Learning for Sequential Action Selection
Qingyue Zhang, Huaze Tang, Minyu Zhang et al. (+3 more)
In-context learning (ICL) allows large
language models to use demonstrations without updating their parameters. For interactive
agents, demonstrations are trajectories of
reasoning, actions, and observations that illustrate how to complete a task. Reusing demonstrations in new situations is challenging: directly inputting full trajectories consumes limited
prompt space and constrains the overall effectiveness, while natural-
language summaries require the
agent to translate general guidance into specific action choices. Therefore, we study whether expert action choices can directly supervise a reusable scoring rule. We propose CB-
Agent, which extends concept-based ICL to sequential action selection. We extract environment-action decisions from expert trajectories and fit a concept vector by maximum likelihood to model the conditional distribution over candidate actions given the context. This model combines the concept with frozen
language-model features and priors. During ReAct interaction, the
agent uses this distribution to select environment actions without including demonstrations in the action-scoring context. Across TextWorld, ALFWorld, and ScienceWorld, CB-
Agent achieves the highest cross-
benchmark average among the evaluated methods at all three tested model sizes and ranks first on TextWorld and ScienceWorld at each size. ALFWorld experiments support the learned concept's contribution beyond prior-only scoring and show further gains when combined with Reflexion. Code is available at https://anonymous.4open.science/r/CB-
Agent-DFF9/.
Read the full abstract ▾
All authors Qingyue Zhang, Huaze Tang, Minyu Zhang, Yuxiang Yan, Wenbo Ding, Shao-Lun Huang
Artificial Intelligence
Learning after Execution with Action-Specific Credit
Yufeng Tian, Jiahao Tang and Minghao Wu
Execution is often used as terminal feedback: it scores a generated program, but the resulting state is not used for another learning action. Learning after execution instead uses the execution result as an intermediate state that supports an additional learning interaction for the same policy. This creates a credit-assignment problem because the two actions share the same final objective but play different roles in the trajectory. Accordingly, action-specific credit assigns the initial action credit from the final trajectory outcome and the corrective action credit for improvement over the intermediate state. Image-to-TikZ serves as an executable visual testbed for this setting. On 945 held-out examples, learning after execution strengthens the shared policy under single-call inference and also produces a post-execution corrective capability. Single-call render success reaches 92.70%, compared with 88.25% for continued single-stage GRPO after the same number of additional training iterations. This corrective capability is not reproduced by continued single-stage training and depends on the intermediate state. Alternative credit definitions also induce different group-relative preferences over the same trajectories. Together, these results suggest that executable tasks contain useful learning structure beyond a terminal reward. Code is available at https://github.com/tianyufengcn/Learning-after-Execution.
Read the full abstract ▾
Artificial Intelligence
PsyMemBehav: Psychology-Informed and Memory-Grounded Two-Level Identity-to-Action Reasoning for Social-Media Behavior Simulation
Wenji Zhang, Xilong Cheng, Yunxiao Qin et al. (+8 more)
Large
language models (LLMs) support social simulation, but faithfully reproducing real users requires grounding both structured platform decisions and
language in sparse, noisy behavioral evidence. We introduce PsyMemBehav, which maps psychology-informed profiles and retrieved
memories to target-first platform behavior through two-level identity-to-action
reasoning. To support this approach with comprehensive behavioral traces, we construct a
dataset PsyMemBehav-bench from 5.57M observed behaviors of 5,862 real users (approximately 950 per user) across Reddit, Bluesky, and Twitter. Each recorded engagement is modeled as target selection followed by Comment, Like, or Repost and optional response generation. PsyMemBehav combines 41-dimensional user descriptors, top-5 BM25
memories, reverse-CoT supervision, and rubric-guided DPO over complete
reasoning-action trajectories. Experiments show that PsyMemBehav 27B ranks first in 11 of 12 reported individual-level metrics, improving average response quality over GPT-5.4 from 54.28 to 59.68 (9.95% relative), and obtains the lowest group-level MAE (0.081) and MSE (0.009) among simulated systems. These results support evidence-grounded user modeling for observed-thread continuation; the
benchmark remains conditional on recorded engagement and does not model no-action decisions.
Read the full abstract ▾
All authors Wenji Zhang, Xilong Cheng, Yunxiao Qin, Mai Tu, Tongye Zhu, Yuhao Zhou, JingBo Ji, Yuting Tan, Duoyin Li, Hongjiang Xiao, Yuan Zhang
Artificial Intelligence · Cross-listed from cs.SE
LASER: Defeasible Certificate Search for Long-Range Protocol Dependencies
Zihan Tang and Huibiao Zhu
Protocol specifications rarely justify executable behavior in one place: a transition may inherit its source state from a remote control block, name its events only by number, and remain subject to a later correction. Recovering the nearby clause is therefore insufficient; the system must assemble a source-linked certificate and determine whether any stronger rule applies. We introduce
LASER, a proof-carrying search method for sparse, defeasible protocol evidence.
LASER represents missing support and unresolved corrections as explicit obligations, searches typed evidence routes, and lets an occurrence-aware symbolic checker alone decide whether a certificate warrants, refutes, or leaves a behavior unresolved. On 91 BehaviorCheck families under a matched 2K budget,
LASER improves Joint Edge+Proof over the strongest local reader on two model families ($\Delta=+.132$, $p=.017$; $\Delta=+.198$, $p<.001$) while cutting false discoveries by 2–3$\times$. Controlled replays attribute the gain and its limits: neither reference routing nor deterministic citation expansion alone exceeds the local baseline; whole-document reading (29–65K
tokens), two-round iterative
retrieval, and even oracle minimal-certificate evidence all fail to reach
LASER, localizing the bottleneck in typed obligation closure rather than evidence access. An authority-corruption replay shows source-link errors only cause abstention, while the only unsafe-warrant mechanism is mis-recording a Held erratum as Verified, confined to that erratum's family.
LASER thus improves long-range dependency recovery while converting incomplete evidence and uncertain authority into explicit residual obligations rather than unsupported behavior.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.IR
LLM-based User-Product Relevance Evaluation for Dynamic Product Advertising
Wen Zhang, Xiao Bai, Hsiang-wei Chao et al. (+9 more)
Dynamic Product Advertising (DPA)
retrieval systems are a core component of modern e-commerce advertising; however, their evaluation remains challenging. Conventional offline engagement metrics are limited by sparse, biased feedback, while online evaluation is costly and time-consuming. In this work, we propose a user–product relevance metric for offline evaluation of DPA
retrieval systems. The metric first compresses the engagement history into a compact set of intent representatives via Semantic-ID-grouped, event-stratified segmentation, then assesses the relevance of a recommended product to each individual intent and aggregates them as a user-level relevance score, yielding a stable and interpretable measure of
retrieval quality. To compute this metric at scale, we leverage large
language models (LLMs) as semantic relevance judges, capturing nuanced functional and semantic relationships that heuristic methods often miss. We propose an agentic, human-aligned prompting framework that generates multi-category relevance guidelines and distills teacher-model judgments into an efficient student model for scalable evaluation. The resulting metric strongly correlates with offline and online business metrics, enabling faster and more reliable DPA
retrieval iteration before A/B testing.
Read the full abstract ▾
All authors Wen Zhang, Xiao Bai, Hsiang-wei Chao, Mengfan Zhu, Tong Zhao, Mingxuan Ju, Xiaodong Liu, Congfei Zhang, Yunzhi Zhou, Yajun Wang, Jinchao Li, Yu Zhang
Artificial Intelligence · Cross-listed from cs.LG
Active Neural Matter: A Finite-Window Response Framework for Physical Systems and Distributed AI Workflows
Tianchi Chen and Shihua Yu
Physical systems and distributed AI workflows can retain past information, receive delayed feedback and combine conflicting signals. These features pose a common modeling problem: determining what information about past interactions a state must retain to account for future responses. Active Neural Matter (ANM) addresses this problem through a finite-window response framework. Forward analysis uses each system's dynamics to predict state and output responses to perturbations, separating propagation from output sensitivity. Inverse analysis tests whether a candidate state retains enough information for these responses. With the system and future inputs or interaction rules fixed, distinguishable responses from histories assigned the same state identify distinctions the representation must preserve. Their distributional separation gives a lower bound on the worst-case error of approximating behavior from that state. Uncertainty bounds distinguish rejection, support within a fixed tolerance on tested comparisons, and unresolved cases. Delayed-consensus simulations check analytical predictions of response amplitude and phase over a narrow range of delays; flocking illustrates dependence on the chosen observable. An exact delayed-return construction establishes a sufficient state for a specified program. The AI studies identify distinctions in interaction history and evidence order that a representation must preserve to reproduce observed response distributions. Tests on new histories in a controlled feedback task provide bounded support for a state specified by the task instructions; the instructed rule predicts every observed action. ANM connects response prediction to measurable information requirements, separating behavioral fidelity from task correctness and initial-state adequacy from a closed
evolution law for that state.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.DC
A Self-Defining Operator for Distributed Applications
Shihang Li, Thomas Anderson, Ratul Mahajan et al. (+2 more)
To operate distributed applications with high availability, operators must rapidly diagnose and mitigate any incident. LLM-based
agents can help, but unlike experienced human operators they do not improve as they handle more incidents. Each investigation starts from scratch, re-deriving likely causes and re-testing hypotheses even when a similar fault was resolved in the past. We present a self-defining operator (SDO) that improves with time by updating its operational behavior and
memory. When SDO resolves an incident, it distills that experience into playbooks, which describe reusable diagnostic instructions for future incidents with similar faults, and detectors, scripts that take application telemetry as input and output relevant playbook(s). SDO continuously runs the detectors and invokes LLMs only when one fires, so its
token cost scales with incident count rather than with operational time. Across 11 open-source microservice applications, SDO cuts resolution time by 52% on recurring faults while keeping per-incident cost low. SDO is open source at https://github.com/self-defining-systems/sdo.
Read the full abstract ▾
All authors Shihang Li, Thomas Anderson, Ratul Mahajan, Simon Peter, Luke Zettlemoyer
Artificial Intelligence
Are Multimodal Benchmarks Really Useful? Item-Level Multimodal Benchmark Diagnosis via Structure-Response Co-Calibration
Shiqi Zhang, Weixin Zeng, Ziheng Zhang et al. (+2 more)
Multimodal benchmarks are central to evaluating whether models can reason over information from multiple modalities. However, existing literature has revealed that some
benchmarks may contain shortcut items that can be answered from a single modality or from answer-option regularities alone, thus failing to reflect models' true cross-modal inference ability. Existing methods for assessing
benchmarks typically rely either on intrinsic item structure or extrinsic model response behavior, which are not adequate for a comprehensive diagnosis. To fill in this gap, we propose SRCoD, a structure-response co-calibrated diagnosis framework for item-level diagnosis of modality dependence in
multimodal benchmarks. SRCoD estimates sample-wise
multimodal information structure from
benchmark content and target answers, and uses this signal as a soft structural anchor for modality-conditioned item response modeling. The learned model produces a calibrated item-level diagnostic profile that reflects how strongly an item depends on different forms of modality evidence, enabling fine-grained
benchmark analysis beyond aggregate accuracy. SRCoD also provides interpretable attribution of the diagnosis results. Extensive experiments under two complementary protocols, i.e.,
benchmark-refinement oriented validation and human-labeled item validation, demonstrate that SRCoD outperforms state-of-the-art
benchmark diagnosis methods.
Read the full abstract ▾
All authors Shiqi Zhang, Weixin Zeng, Ziheng Zhang, Jiuyang Tang, Xiang Zhao
Artificial Intelligence · Cross-listed from cs.RO
GG-Nav: Goal-oriented Grounding Chain-of-Thought Elicits Reasoning in Navigation Foundation Models
Shaohao Zhu, Wensi Huang, Siqi Zhang et al. (+6 more)
Emerging strong
Vision-
Language Models (VLMs) have advanced navigation foundation models. However, these models still have unsatisfactory performance on goal-oriented navigation settings. In this paper, we first conduct a motivating study revealing that the key issue lies in the insufficient utilization of streaming observations and imprecise target localization. To tackle these challenges, we propose GG-Nav, a simple yet effective navigation framework that learns a structured chain-of-thought (CoT) mechanism via supervised fine-tuning to explicitly localize goal-relevant instances, accumulate grounding evidence, and support more reliable decision-making. Specifically, the structured CoT is designed with two key stages: (i) Observation-to-Evidence, which reasons over all intermediate frames between adjacent decision steps to retain useful goal-related evidence; and (ii) Evidence-to-Decision, which integrates the current observation with accumulated evidence to ground the target and support the generation of navigation actions or clarification questions. Extensive experiments show that GG-Nav outperforms the state-of-the-art methods in terms of success rates on the HM3D ObjectNav, VL-LN, and CoIN
benchmarks by 3.1%, 16.6%, and 5.3%, respectively. Ablation studies further show that accumulating evidence from intermediate observations reduces evidence-missing failures by 11.6%, while explicit grounding reduces stopping errors by 49.4%. We also deploy GG-Nav on a Unitree Go2
robot, demonstrating its zero-shot generalization capability in unseen real-world environments.
Read the full abstract ▾
All authors Shaohao Zhu, Wensi Huang, Siqi Zhang, Yuqiang Yang, Meng Wei, Chenyang Wan, Tianle Liu, Tai Wang, Jinming Xu
Artificial Intelligence
FDE-Bench: Evaluating End-to-End Delivery from Underspecified Real-World Business Requests
Huaiming Li, Can Huang, Chufan Wu et al. (+18 more)
Real-world business requests often leave key requirements unstated because frontline practitioners struggle to articulate rules they know from experience. Forward-deployed engineers (FDEs) must elicit these requirements through dialogue and iteratively optimize solutions until they meet customer acceptance criteria. Existing optimization
benchmarks typically start from explicit objectives, while clarification
benchmarks often introduce information gaps into existing tasks. We introduce FDE-Bench, the first
benchmark built from real customer projects to evaluate
agents on end-to-end delivery tasks that require interactive requirement clarification and iterative optimization. Its 49 cases span combinatorial optimization and machine learning. Clarification targets are distilled from discussions between frontline practitioners and FDEs.
Agents elicit missing requirements from a simulated customer, and case-specific evaluators assess final artifacts against acceptance criteria using customer-accepted FDE solutions as quality references. Across 18
agent configurations, the highest mean delivery score is only 0.5510, compared with an FDE reference of 1.0. The highest delivery success rate (DSR), defined as the fraction of planned runs reaching this reference, is only 6.12%. These results show that reliable FDE-level end-to-end delivery remains out of reach for the evaluated
agents on underspecified real-world business tasks. The
benchmark project page is available at https://baidubce.github.io/FM-
Agent/FDE-Bench/.
Read the full abstract ▾
All authors Huaiming Li, Can Huang, Chufan Wu, Danyu Liu, Xiaomin Yuan, Qinze Li, Yui Lo, Wenhui Bai, Qianlong Wang, Zengle Ge, Xiyu Yan, Yuheng Cheng, Jinhui Ren, Guokai Chen, Jiaqun Liu, Haoran Li, Qianhui Liu, Annan Li, Jianmin Wu, Dawei Yin, Dou Shen