Artificial Intelligence · Cross-listed from cs.CL
Compiling Learning Problems into Adaptation Programs for Language Models
Rebecca Ramnauth and Brian Scassellati
Model adaptation is typically governed by a fixed recipe, even though different update programs can produce substantially different behavioral outcomes. We introduce adaptation compilation, which reframes where, how, and to what extent a model should adapt as a joint prediction and decision problem. Rather than searching over candidate programs anew for each learning episode, a compiler learns from prior adaptations to predict a vector-valued counterfactual response surface over candidate programs—their expected effects on acquisition, transfer, boundedness, and preservation—and selects a program before adaptation begins. Because this predicted geometry captures multiple behavioral consequences rather than a single winner or scalar score, it can be reused under different downstream priorities without retraining. Across five learning types, preferred programs vary meaningfully across episodes, and this variation is predictable from pre-adaptation information. On Llama-3.1-8B, compiler-selected programs approach exhaustive search while outperforming global and objective-specific defaults. Replication on Gemma-2-9B preserves program heterogeneity and selection headroom, but shows that exploiting this headroom requires accounting for uncertainty when departing from strong defaults. Together, these results show that adaptation search can be amortized across related learning problems, turning prior adaptation experience into a basis for deciding how future learning should occur.
Read the full abstract ▾
Artificial Intelligence
Pretrain Once, Route Anywhere: Towards a Foundation Model for LLM Routing
Guannan Lai and Han-Jia Ye
Large
language model (LLM) routing aims to assign each query to the most suitable model from a heterogeneous candidate pool, improving the quality–efficiency trade-off of LLM inference. Existing routers are typically learned through local fitting: a router is optimized for a particular query workload and candidate pool, and often requires additional supervision or retraining as the routing environment changes. We ask whether LLM routing can instead be approached from a foundation-model perspective, learning a reusable routing capability that generalizes across tasks, candidate models, and deployment conditions. To this end, we introduce RouteFM, which learns to characterize anonymous candidate models from behavioral context and infer their target-specific capabilities, rather than binding routing decisions to fixed model identities or a single environment. Through episodic pretraining across heterogeneous routing environments, this capability can be reused by a frozen router and adapted to new environments through context alone. Experiments demonstrate transfer across changes in domains, modalities, candidate pools, and context budgets, with the largest gains when behavioral evidence is limited. On MMR-Bench, which is excluded from pretraining, RouteFM outperforms the strongest baseline by 2.23 quality points with only eight observations per candidate. These results support moving LLM routing from repeated local fitting toward a pretrain once, route anywhere paradigm. Our code is publicly available at https://github.com/LAMDA-Model-Reuse/RouteFM.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.RO
Encore: Few-Shot Agentic Discovery of Manipulation Strategies
Yifan Kang, Zihan Wang, Zhiwen Fan et al. (+1 more)
Coding
agents can now write, run, and debug programs with little human help.
Robot tasks, however, are usually specified by a sentence that leaves out how to grasp, in what order to make contact, and what the result should look like, and an
agent given only the sentence must find these details by trial and error. We introduce ENCORE, which gives the
agent a few demonstrations as evidence to read rather than as training data. A deterministic builder distills each demonstration into a pack of multi-view keyframes, gripper events, frame strips, and the full trajectory. A coding
agent studies the pack, writes a policy program against a fixed perception and action API, refines it iteratively over a few development rollouts, and freezes it before a sealed evaluation that never reveals the success signal. On LIBERO-PRO, the
agent's first program already succeeds in half of the perturbed tasks with demonstrations and in one task without them, and the frozen programs outperform the strongest prior agentic system run with the same
language model (96.3% against 89.3%). On RoboDojo tasks whose instructions leave the goal unstated, no program succeeds without demonstrations. ENCORE also runs on a real bimanual
robot, learning cube handover and cup inversion from five demonstrations each.
Read the full abstract ▾
All authors Yifan Kang, Zihan Wang, Zhiwen Fan, Bangya Liu
Artificial Intelligence
Teaching LLMs to Generate Challenging MILP Instances via Solver Feedback
Jitin Singla, Parikshit Pareek, Pratik Jawanpuria et al. (+1 more)
Generating optimization instances that are both feasible and computationally challenging is crucial for benchmarking solvers and training learning-based optimization
algorithms. Existing non-LLM generators rely on seed instances or parameter tuning, resulting in high test-time computational cost, while existing LLM generators lack explicit hardness measures. Recent reinforcement learning methods with verifier feedback evaluate only binary correctness, which is misaligned with generating challenging problems. We note that an optimization solver reports the cost of solving at several stages of its pipeline, and leverage this to design a reward that scores both the solvability and the hardness of generated problems, measured by branch-and-bound nodes and post-cut relaxation gaps. Our key idea is a challenger-solver asymmetric self-play approach, where an LLM challenger generates progressively harder instances and the solver verifies feasibility and hardness, so no seed or training MILP instances are required. We fine-tune Gemma-4-12B and Qwen3.5-4B with GRPO and a size curriculum into OptiScribe-12B and OptiScribe-4B, which generate feasible yet challenging MILP problems from natural
language instructions. On capacitated facility location and max-cut, OptiScribe-12B raises median SCIP search nodes by 1.7-5x and post-cut gaps by 1.1-1.7x over its base model and improves the feasibility rate on facility location by 9-19 points, while OptiScribe-4B raises median nodes by up to 15.6x. The problems cover a wider difficulty range than public
benchmarks of the same size, follow instructions on density and difficulty, and can tune solver settings for families that public libraries lack. These results indicate that optimization-specific rewards, used in self-play mode, can teach LLMs to generate high-difficulty optimization
benchmarks. We will release our code and models publicly on acceptance.
Read the full abstract ▾
All authors Jitin Singla, Parikshit Pareek, Pratik Jawanpuria, Parag Singla
Artificial Intelligence
Seek Before You Move: Evidence Seeking for Progress Grounding in Vision-Language Navigation
Zhimin Wang, Meiyuan Zhu, Duo Wu et al. (+8 more)
Vision-
Language Navigation (VLN) requires
agents to continuously ground task progress from long-horizon instructions and partial
egocentric observations. Existing VLM-based
navigation agents typically reason only over available observations and may remain confident even when task-relevant evidence is missing. For example, an
agent may confidently proceed forward and get lost even though the landmark indicating the next turn lies outside its current field of view. We term this failure mode Progress Myopia: the
agent fails to recognize unreliable progress grounding and continues acting on insufficient evidence. To address it, we propose SeekVLN, an evidence-seeking framework that couples semantic progress
reasoning with active acquisition of task-relevant observations. SeekVLN is trained in two stages: First, Future-guided Reverse Generation (FRG) uses future expert actions to augment offline expert trajectories with supplementary views and evidence annotations. Supervised fine-tuning on these trajectories establishes a prior for evidence seeking and progress
reasoning without additional expert interaction. However, imitation alone does not reveal whether seeking improves subsequent
navigation. We therefore introduce Counterfactual Contrastive Policy Optimization (C2PO) for reinforcement fine-tuning. By comparing each evidence-seeking branch with a counterfactual direct-
navigation branch from the same state, C2PO uses a contrastive reward to assign credit to seeking decisions based on subsequent
navigation benefit. Experiments on simulated
benchmarks show that SeekVLN achieves state-of-the-art performance, improving success rate by 12.7% and 7.5% over the base model on R2R-CE and RxR-CE, respectively. Both simulated and real-world evaluations exhibit human-like evidence-seeking behaviors for more reliable progress grounding.
Read the full abstract ▾
All authors Zhimin Wang, Meiyuan Zhu, Duo Wu, Linjia Kang, Yajun Wang, Yuan Ni, Xiaohang Wang, Tianlu Pan, Jingyan Jiang, Yaowei Wang, Zhi Wang
Artificial Intelligence · Cross-listed from cs.LG
Port-Hamiltonian Latent Deliberation: Mitigating the Deliberation Drift Cliff in Test-Time Compute Scaling
Zeyu Jia
Test-time compute scaling has emerged as a cornerstone of advanced machine
reasoning, yet performing iterative deliberation directly within continuous latent representation spaces reveals a catastrophic pathology: the Deliberation Drift Cliff. While unconstrained recurrent latent models achieve initial
reasoning gains at short horizons (K <= 4), their
reasoning collapses when extrapolated to deeper thinking steps (K >= 16), dropping by 22% to 62% across standard logical
benchmarks. We resolve the trilemma among expressivity, Lyapunov stability, and computational efficiency in test-time latent
reasoning through a 22-round empirical and theoretical investigation. We demonstrate that strictly conservative scalar potential gradient flows suppress long-range drift (cliff 3.40%) but bottleneck peak
reasoning accuracy at 32.73%, whereas unconstrained rotational flows achieve high symbolic expressivity (82.33%) but suffer a severe 36.87% drift cliff. To resolve this
geometric duality, we establish Port-
Hamiltonian Latent Deliberation (PH-LD) and propose the Direct-Gradient Pure-Tensor Helmholtz-Hodge Decomposition (DG-HHD). DG-HHD parameterizes the attracting flow as a tangent projection tensor network while orthogonally decoupling non-zero circulation (Hodge machine error 1.65e-17, contraction error 5.55e-17), eliminating runtime autograd dependencies to achieve 1.84x vector field and 2.09x RK45 rollout speedups. In a 15-arm symmetrical Pareto
benchmark, DG-HHD achieves 58.67% peak accuracy (+25.94% absolute gain over conservative HHD) and retains 35.27% at K=32. Transferred to small
language model (SLM) multi-hop causal
reasoning, DG-HHD delivers monotonic compute scaling (49.33% to 51.56%) and suppresses out-of-distribution drift (cliff -0.66%). All 30 Level 0 deterministic invariants are certified.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CV
TAEC: Trajectory-Aware Evidence Coordination for Multi-Step Visual RAG
Yalun Wu, Bingzhou Wang, Boyang Wang et al. (+5 more)
Multi-step visual
retrieval-augmented generation (RAG) answers complex questions by repeatedly retrieving visual evidence, updating an intermediate state, and deciding whether to continue searching or answer. Yet retrieving relevant evidence does not ensure its effective use throughout the
reasoning trajectory. As multi-step
reasoning progresses, redundant sources occupy context capacity needed for missing evidence, observations tied to resolved requirements or unproductive searches linger in context, and visual sources are revisited with insufficient detail for fine-grained reading. We term this loss of usable evidence over a
reasoning trajectory trajectory-level evidence utilization degradation. To address it, we propose Trajectory-Aware Evidence Coordination (TAEC), a training-free framework that coordinates evidence use around unresolved answer requirements. TAEC tracks these requirements in a shared trajectory state to guide which evidence enters the context, how accumulated
memory is retained, and at what level of detail visual evidence is examined. Under a unified evaluation protocol on ViDoSeek, SlideVQA, and MMLongBench-Doc, TAEC achieves the best overall performance against leading training-free visual RAG baselines, with the highest average accuracy across multiple proprietary
vision-
language models. These results demonstrate that aligning evidence with evolving
reasoning needs improves evidence use throughout multi-step visual RAG.
Read the full abstract ▾
All authors Yalun Wu, Bingzhou Wang, Boyang Wang, Peiying Wang, Shaojie He, Yunhan Wang, Shaozu Yuan, Jiawei Wang
Artificial Intelligence · Cross-listed from cs.LG
A Sharp Transition in Data Reconstruction under Differential Privacy
Max Cairney-Leeming, Simone Bombari and Marco Mondelli
Data reconstruction attacks have empirically been successful in recovering training samples from learned models, raising
privacy concerns and motivating defenses with guarantees that remain valid against future threats. While differential
privacy (DP) provides formal protection, choosing the
privacy budget remains a challenge: small budgets severely reduce utility, but it is hard to quantify how large the budget can be without allowing accurate reconstruction. In this work, we study informed attackers who aim to reconstruct a single $d$-dimensional training sample from a $\rho$-zero-concentrated DP model, knowing all other training data. Our main contribution is to establish a sharp transition at $\rho \asymp d$ for data reconstruction: on the one hand, we derive
entropy-based lower bounds for any private mechanism and any attack, characterizing a set of target priors for which reconstruction is information-theoretically impossible for $\rho \ll d$; on the other hand, we analyze a simple attack on private linear regression with output perturbation, showing that reconstruction is practically feasible for $\rho \gg d$. Remarkably, the transition moves to $\rho \asymp s$ for data lying in an $s$-dimensional subspace, demonstrating that the
privacy budget guaranteeing adequate protection must be assessed in terms of the effective dimension of the data. We validate our findings via experiments on synthetic data and natural images (CIFAR-10, ImageNet).
Read the full abstract ▾
Artificial Intelligence
Solving Without Stopping: On-Policy Distillation at Small Scale
Hongyang Li, Yiming Zhu, Xiao Li et al. (+3 more)
On-policy
distillation, where a student learns from a stronger teacher's feedback on its own outputs, is a common way to pass
reasoning to smaller models. We analyze what it transfers at small scale, distilling Qwen3-8B into Qwen3 4B, 1.7B and 0.6B students, in thinking mode (reason at length, then end the
reasoning and answer) and, for comparison, in non-thinking mode (no separate
reasoning phase). Long
reasoning needs two abilities, solving a problem and knowing when it is solved, and we find that
distillation transfers the first, but in thinking mode not the second. Solving improves at every size, up to two ceilings, which we measure comprehensively across both modes and all student sizes: a student's single attempt never exceeds what it could already reach in many attempts before training, and the smaller the student, the further it stays below the teacher. Stopping is where the modes part. In non-thinking mode every student keeps stopping; in thinking mode students stop ending their
reasoning early in training, and the smaller the student, the less of this ability survives: the teacher signals a stop almost only where a student already ends its
reasoning, so
distillation teaches no new stops; it only keeps the student's existing stops that land on a right answer, and a weak student has few such stops. The smallest students often reach the right value but do not commit to it: they either rarely mark it or mark it and write past it. Together, these results describe how small students behave under on-policy
distillation, and a diagnostic that separates answer marking, correctness and stopping.
Read the full abstract ▾
All authors Hongyang Li, Yiming Zhu, Xiao Li, Caesar Wu, Said Mammar, Pascal Bouvry
Artificial Intelligence
VISTA: Value-Informed Event Appraisal for Multimodal Emotion Conflict
Jiale Dai, Liuxian Ma, Xiaoke Niu et al. (+5 more)
Conflicting emotional cues can be individually valid: a subdued voice may reflect a blocked goal while a smile satisfies a social obligation. Their interpretation depends on what the event means to the person. We introduce VISTA (Value-Informed Semantic Trust Arbitration), a learned seven-field appraisal interface that conditions modality arbitration on concerns, event relations, and expression conditions while retaining a joint-evidence residual. A log-odds decomposition separates
emotion expectation from cue diagnosticity, motivating an interface that lets appraisal change how evidence is interpreted. With a shared Qwen2.5-Omni-7B backbone and matched training examples and steps, VISTA reaches 64.5% conflict accuracy on CA-MER, improving on modality gating by 2.5 percentage points on conflict and 0.2 on consistency. Shuffling appraisal across scenes or removing its decision connection reduces this benefit. A common frozen-backbone probe reaches 0.600 macro CCC for appraisal readout, compared with 0.505 for
emotion-only fine-tuning. Evaluations across five
benchmarks connect recognition under increasing conflict with appraisal readout and downstream decision use. Together, the analyses and experiments support scene-specific appraisal as an intermediate representation that helps interpret conflicting emotional evidence.
Read the full abstract ▾
All authors Jiale Dai, Liuxian Ma, Xiaoke Niu, Wenjing Zhang, Huiying Zhao, Zhaoxiang Liu, Shiguo Lian, Guojie Song
Artificial Intelligence
Mubric: Mutation Testing-Guided Rubric Generation for LLM Evaluation
Jiayuxuan Yang, Jie M. Zhang, Yiling Lou et al. (+1 more)
Rubric-based evaluation is widely used to assess LLM-based systems by decomposing response quality into task-specific scoring criteria. However, automatically generating rubrics that reliably capture task-specific quality requirements remains challenging. We introduce Mubric, a mutation testing-guided approach to rubric generation. Mutation testing, a classic software testing methodology, evaluates a test suite by injecting faults into programs and checking whether the tests detect them. We draw an analogy between test suites and rubrics: if a rubric captures an important quality requirement, introducing a corresponding defect into an otherwise high-quality response should reduce its score. Mubric first mines common defects from real pairs of preferred and dispreferred responses and abstracts these defects into reusable mutation operators, each specifying how to introduce a particular type of response defect. For a new task, it applies relevant operators to a reference response, checks whether the injected defects reduce response quality, and uses insufficiently penalized defects to refine the rubric. We evaluate Mubric on 703 tasks across four representative domains against six advanced rubric generation methods. Mubric achieves the highest overall evaluation accuracy, outperforming the strongest baseline by 7.48 percentage points.
Read the full abstract ▾
All authors Jiayuxuan Yang, Jie M. Zhang, Yiling Lou, Zhenpeng Chen
Artificial Intelligence · Cross-listed from cs.MA
PowerMarketJax: A JAX Benchmark Suite for Multi-Agent Reinforcement Learning in Power Markets
Zhanhua Pan, Xin Qin, Xiao Liu et al. (+3 more)
Power
markets are a natural testbed for multi-
agent reinforcement learning (MARL), where multiple self-interested participants repeatedly submit bids. A
market-clearing mechanism then determines dispatch and prices subject to power grid constraints and
market settlement rules. However, existing MARL environments typically focus on a single
market setting, implement simplified clearing mechanisms, or rely on CPU-based optimization solvers that slow large-scale training and limit the systematic study of bidding strategies and
market behavior. We introduce PowerMarketJax, a
benchmark suite for MARL across five power
markets: day-ahead wholesale, real-time balancing, ancillary services, peer-to-peer double auctions, and local flexibility. Each environment implements its own clearing, pricing, and settlement rules while providing a common framework for learning and evaluation. We find that learned bidding behavior depends strongly on the
market design: independent learners can miss better strategies when gains require many
agents to change together, when more profitable strategies lie beyond a region of lower profit, or when profits disappear as more
agents adopt the same strategy. PowerMarketJax implements both
market simulation and policy training in JAX, allowing the entire pipeline to run on the GPU with 1,024 X 1,200 parallelisms across both environments and
market participants, achieving up to 33X speedup over CPU-based baselines. Our open-source
benchmark is available at: https://github.com/powermarketjax/PowerMarketJax.
Read the full abstract ▾
All authors Zhanhua Pan, Xin Qin, Xiao Liu, Zhilong Cao, Jianhong Wang, Dawei Qiu
Artificial Intelligence · Cross-listed from cs.SE
Do Agent Benchmarks Do What They Say? An Executable-Contract Audit of Tool-Using Agent Environments
Rohith Reddy Bellibatlu, Zichong Wang and Wenbin Zhang
Tool-using
agents are entering settings where a wrong action carries real cost, and the
benchmarks certifying them grade what each simulated tool call reports having done, assuming the tool did what its interface advertises. The audit taxonomies we survey publish no category for that assumption, and a defect beneath a score is present on every rerun. We treat a tool's advertised surfaces as an executable contract, check the implementation against it, and trace each score's provenance through the task files and evaluator code to the verdicts that derive from state a defective tool should have written. Across 34 audited mutating tools in four
benchmarks we confirm seven tool defects and one evaluator property at pinned commits. On injected defects the checker raised no false positive in 25 flags, flagged 2 of 5 negative controls, and missed most: in 29 of 33 scored misses a clause covered the defect but no probe revealed it. The checker's own static half, run alone, flags 14 of 17 confirmed sites, so on these findings the dynamic half confirms and traces rather than discovers. Twelve further AgentDojo tools, with six held-out tools and the seven audited first, complete its 25-tool mutating surface, on which at least 5 tools diverge from their advertised surface as our contracts read it, a rate for AgentDojo alone. No gold trajectory reaches either tau2-bench defect; on 1,120 paths built to isolate the telecom defect, a number fixed by construction, the evaluator rewards a refuel of a suspended line and fails the repaired tool. The clearest case is a clinical
benchmark whose tool tells the
agent each write executed under a documented no-write design its interface does not disclose; its grader takes that message as evidence, so its action success rate records whether a request carried the expected payload, not whether any record changed.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
Hidden Reasoning Must Leak, but Need Not Be Readable: Fundamental Opportunities and Limits for Chain-of-Thought Monitoring
Mohammadali Mohammadkhani, Madhava Krishna, Yash Sarrof et al. (+1 more)
Can
reasoning models trick chain of thought (CoT) monitors and perform hidden computation without revealing it in their thinking traces? We show that the answer depends on the underlying task difficulty and the model size. Simple computations can be performed covertly; however, beyond a threshold depending on model size, successfully solving the task necessarily leaks a near-linear amount of information about the covert task input into the CoT. Therefore, sufficiently complex hidden computation always leaves an information-theoretic footprint. However, concerningly, this leakage need not be readable: Under plausible cryptographic assumptions, even a one-layer
Transformer can encrypt its
reasoning online so that no
polynomial-time monitor can extract information about the hidden computation. Overall, our theoretical and empirical results provide a holistic view of both the opportunities and the limitations of CoT monitoring.
Read the full abstract ▾
All authors Mohammadali Mohammadkhani, Madhava Krishna, Yash Sarrof, Michael Hahn
Artificial Intelligence
ReMem: Rethinking Perception and Memory in Long-Context Recommendation Agents
Haohao Qu, Yongcheng Jing, Chun Hin Chan et al. (+3 more)
Recent Recommendation
Agents (RecAgents) offer a promising alternative by shifting recommendation to an active, user-side paradigm, where generative
agents autonomously perceive external platforms, reason over user preferences, and execute decisions. However, existing RecAgents still suffer from two critical limitations: brittle item perception based on noisy and heterogeneous item pages, and inefficient long-context
reasoning over extended user histories and multi-step interaction traces. To address these challenges, we propose a novel recommendation
agent framework, termed as ReMem, that combines OCR-based
multimodal perception with time-evolving dynamic
memory. Instead of parsing raw HTML, ReMem observes item pages through screenshots and extracts structured
multimodal information via an OCR tool, enabling a more
humanoid and platform-agnostic perception mechanism. To support long-horizon preference modeling, ReMem further introduces a chunk-wise sequential
memory update strategy, where the
agent selectively maintains a fixed-size
memory of informative historical interactions while processing arbitrarily long contexts with linear inference complexity and bounded context length. This design allows the
agent to preserve evolving user preferences without relying on external
memory modules or disrupting the standard autoregressive generation process. To enhance the dynamic
memory instruction, we further develop a multi-
memory GRPO variant, which propagates the final-answer advantage to all intermediate conversations that contribute to the final response. Extensive experiments on three
datasets demonstrate that ReMem consistently outperforms state-of-the-art baselines, achieving an average improvement of 5.16% across three recommendation
agent tasks, namely searching, ranking, and judging.
Read the full abstract ▾
All authors Haohao Qu, Yongcheng Jing, Chun Hin Chan, Shanru Lin, Wenqi Fan, Dacheng Tao
Artificial Intelligence
MetaCtrl: Your Large Language Models Can Reason Better and More Concisely with a Metacognitive Controller
Zhibin Wen, Tao Han, Lei Bai et al. (+2 more)
Large
reasoning models improve performance on challenging problems by allocating additional computation before answering, but longer
reasoning does not always lead to better results and can introduce substantial redundant
reasoning on simple problems. Conversely, aggressively shortening
reasoning can degrade performance on difficult ones. Effective
reasoning therefore requires dynamically deciding when additional computation is useful based on the reasoner's capabilities and evolving solution state. Existing approaches often rely on predefined budgets or intervention rules, retrain the target reasoner, or require additional supervision. We introduce MetaCtrl, a lightweight controller that adaptively regulates a frozen reasoner without predefined
token budgets or reasoner retraining. We formulate
reasoning regulation as a sequential metacognitive control problem: MetaCtrl observes the evolving
reasoning trace and decides whether to continue, simplify, skip redundant steps, or conclude
reasoning. It is trained directly with reinforcement learning using a reward that prioritizes correctness while favoring shorter trajectories among correct solutions, requiring neither supervised intervention trajectories nor problem-specific budgets. Across seven
benchmarks spanning mathematics, science, and code, MetaCtrl consistently improves the accuracy of LRMs while reducing their
reasoning length. On DeepSeek-R1-Distill-Qwen-7B, it improves average accuracy by 4.7 points while reducing generation length by 53.3%. Without further training, the same controller transfers to an unseen reasoner (e.g., Qwen3-14B), improving average accuracy by 2.9 points and reducing generation length by 50.3%. These results establish MetaCtrl as a plug-and-play controller for improving
reasoning accuracy while substantially reducing inference-time generation. The code is available at https://github.com/binbin2xs/MetaCtrl.
Read the full abstract ▾
All authors Zhibin Wen, Tao Han, Lei Bai, Can Li, Yang Xu
Artificial Intelligence · Cross-listed from cs.CV
VISTA-Bench: Benchmarking Multilingual Image Translation with Image-Specific Rubrics
Bo Lv, Mao Zheng, Zheng Li et al. (+3 more)
Image translation is a fundamental capability of
multimodal models for
multilingual applications, requiring visual understanding and meaning preservation across
languages. However, existing
benchmarks have limited
language coverage and often lack explicit image-specific evaluation criteria, making it difficult to comprehensively assess this capability. To systematically evaluate this capability, we introduce VISTA-Bench, covering 22
languages and 10 domains, and develop an image-specific rubric evaluation protocol. The
benchmark combines sampling for
language and scenario coverage with model-assisted, human-verified annotations that group related text into coherent semantic units and provide
multilingual reference translations. The rubrics specify essential content, semantic relations, and acceptable translation variants, yielding separate output-based scores for translation quality and the preservation of visual and knowledge-dependent information. We conduct extensive evaluations of 16 mainstream models, including 12
multimodal models and four text-input models, and provide systematic analyses across
languages, domains, and evaluation dimensions.
Read the full abstract ▾
All authors Bo Lv, Mao Zheng, Zheng Li, Fangxu Liu, Mingrui Sun, Tao Chen
Artificial Intelligence
AssayRouter: Historical Utility Priors for Frozen Molecular Predictor Routing
Dong Xu, Zhangfan Yang, Jiantao Wu et al. (+5 more)
Laboratories often face a new molecular assay with 16-64 labels and a bank of predictors whose training data and parameters are unavailable. The practical question is which frozen outputs to include in a small local model. AssayRouter treats completed assays as pseudo-targets and labels each candidate by its post-fit utility: the reduction in held-out discovery loss when the candidate is added to the local target predictor. A shared regressor learns to predict this utility from candidate behavior on the support set, without source identity; on a new assay, one frozen ranking selects four sources and separate labels fit a convex combiner. We train only on completed ChEMBL-MT assays and evaluate 24 external regression assays across six frozen interface families. AssayRouter-C lowers strict four-call negative log-likelihood (NLL) by 0.0409 relative to Support-CV@4. Frozen candidate-label permutations confirm that candidate-utility correspondence carries the transferred information, and leave-one-interface-out training shows that the mapping generalizes to unseen predictor families. Completed assays therefore provide transferable supervision for scarce-label routing through frozen prediction interfaces.
Read the full abstract ▾
All authors Dong Xu, Zhangfan Yang, Jiantao Wu, Shipeng Zhang, Zexuan Zhu, Jiangqiang Li, Jun Zhang, Junkai Ji
Artificial Intelligence
Transolver-$\sigma$: Joint Spectral-Physical Subspace Modeling for Neural PDE Solving
Haonan Shangguan, Hang Zhou, Haixu Wu et al. (+3 more)
Neural solvers offer efficient surrogates for numerical simulation of partial differential equations (PDEs). For time-dependent problems, strong one-step accuracy does not necessarily translate into reliable autoregressive rollout. We observe that a solver based only on physical-state modeling can achieve lower one-step error, whereas its spectral-only counterpart can become more accurate at later rollout steps. Motivated by this observation, we present Transolver-$\sigma$, a neural PDE solver based on joint spectral–physical subspace modeling. Within each block, adaptive physical-state interactions and spectral transformations are modeled in dedicated latent subspaces, whose responses are recomposed to enable information exchange between the two representations. Within the physical subspace, we introduce Slice-Residual Physics-
Attention (SRPA), which preserves an explicit slice-space identity path while retaining learnable cross-slice interaction. In parallel, an axis-factorized
Fourier operator captures global spectral structure. Across five well-established PDE
benchmarks spanning steady-state prediction and time-dependent dynamics, Transolver-$\sigma$ achieves state-of-the-art with a
benchmark-averaged relative error reduction of 33.4% over the strongest baseline for each metric, while consistently improving autoregressive rollout over single-operator counterparts. Transolver-$\sigma$ further delivers strong gains on coupled multiphysics systems and real-world fluid and combustion measurements from RealPDEBench, demonstrating its effectiveness beyond standard simulation
benchmarks.
Read the full abstract ▾
All authors Haonan Shangguan, Hang Zhou, Haixu Wu, Yuezhou Ma, Jianmin Wang, Mingsheng Long
Artificial Intelligence
Task-Relevant Null-Space Residuals for Non-Injective Neural Mappings
Bizu Feng, Zhimu Yang, Shuming Wang et al. (+4 more)
Non-injective mappings in neural networks map distinct inputs to the same representation, thereby implicitly inducing equivalence relations in the input space. However, the input differences eliminated by these mappings may still be required by downstream tasks, creating a mismatch between operator-induced indistinguishability and task-required distinctions. For non-injective linear operators realized in the current forward pass, their null spaces exactly characterize these invisible input variations. We propose Task-Relevant Null-Space Residuals (NSR), a general residual framework for non-injective linear mappings. NSR combines null-space component extraction from pre-mapping representations, member-level encoding and gating, and application-specific integration to exploit potentially task-relevant information under downstream supervision while preserving the original aggregation or merging rules. We evaluate NSR in two structurally different settings:
token merging and graph aggregation. In
token merging, NSR achieves higher semantic
segmentation performance than the corresponding compressed baselines in 34 out of 36 evaluated configurations, with a maximum observed gain of 31.51 mIoU points under strong compression. In graph aggregation, NSR achieves 100% training accuracy on Tree-NeighborsMatch at depths d=2–6 across three backbones, alongside gains on heterophilic node classification and molecular graph regression. Together, these results support null-space residuals as a practical complement to non-injective linear mappings, enabling downstream models to learn from input distinctions invisible in the original operator's output.
Read the full abstract ▾
All authors Bizu Feng, Zhimu Yang, Shuming Wang, Yuan Cheng, Shaode Yu, Xiaojun Qian, Zixin Hu
Artificial Intelligence
Foundations of Proactive Agents: Principles, Technical Layers, and Proactivity-Gym
Jio Oh, Seunghyun Do, Young-Jun Lee et al. (+2 more)
Proactive LLM
agents can turn idle compute into useful support before users ask. Yet even correct work can misread user context, impose review costs, or undermine trust. This work proposes foundations for designing, realizing, and evaluating proactive LLM
agents around three joint principles (3T): Task Capability, anticipating relevant needs and correctly performing useful work; Temporal Allocation, allocating compute according to resource availability and when results are needed; and Trust, sustaining users' confidence and appropriate reliance on the
agent. We connect these objectives to a design space organized around five dimensions: task scope, anticipation horizon, activation trigger, processing timing, and intervention depth, and specify the situation and system modeling needed to support its choices, including user and environment representations, backbone LLMs, and
agent harnesses. Lastly, we propose PROACTIVITY-GYM, a simulation-based evaluation testbed including multi-day scenarios, stateful environments, and persona-conditioned simulated users that can evaluate the consequences of proactive assistance across interactions. Evaluations across 23 model-harness configurations uncover substantial performance gaps across 3T and reveal that LLM judges often conflate task capability and trust. A human study with 30 participants demonstrates the importance of the joint 3T optimization: participants show sharp trust declines after intervention misalignment despite correct outcomes, and prefer sleep-time assistance, even when imperfect, to preserve ongoing focus. Together, these findings support designing and evaluating proactive
agents through the joint consideration of useful work, compute allocation, and evolving user trust.
Read the full abstract ▾
All authors Jio Oh, Seunghyun Do, Young-Jun Lee, Steven Euijong Whang, Dongyeop Kang
Artificial Intelligence · Cross-listed from cs.CV
UniAfford: Token-Routed Multitask Learning for Generalizable 2D-3D Affordance Perception
Yuhao Liu, Yiming Zhong, Hanqing Wang et al. (+9 more)
Affordance perception aims to localize actionable regions supporting
embodied interaction, yet 2D and 3D affordance grounding have evolved as separate problems, with different task definitions, supervision formats,
datasets, and evaluation protocols. This fragmentation limits the learning of transferable object-affordance semantics across visual and
geometric spaces. We propose
Token Router for Tasks, a multitask training paradigm for MLLM-based systems that routes contextual hidden states to task-specific branches without requiring the
language head to generate predefined markers. Routed states are supervised directly by branch-specific objectives, enabling dense prediction losses to shape shared MLLM representations. We instantiate this paradigm as UniAfford, a unified framework for generalizable 2D-3D affordance perception, together with UniAfford-Data, a
dataset integrating pixel-level 2D annotations, point-level 3D annotations, and
language instructions under a shared object-affordance taxonomy, supporting heterogeneous supervision through semantic-level 2D-3D pairing. UniAfford adopts an MLLM as a shared semantic hub and a modality-aware
token router to produce image- and point-cloud-affordance queries. These queries respectively condition a SAM-style pixel decoder and a SONATA-based point decoder, enabling flexible 2D, 3D, and joint affordance inference from image-only, point-cloud-only, or paired
multimodal inputs. Experiments demonstrate strong zero-shot generalization across 2D and 3D affordance
benchmarks without target-specific fine-tuning, alongside state-of-the-art branch-wise performance under modality-isolated protocols. Ablations validate
token routing, joint 2D-3D supervision, and decoder coupling, while
language-head diagnostics show that routed latent states carry meaningful object-affordance semantics. Project page: https://4dvlab.github.io/UniAfford
Read the full abstract ▾
All authors Yuhao Liu, Yiming Zhong, Hanqing Wang, Shaocheng Yan, Yuhang Zhang, Wenzhou Lyu, Ziyang Ding, Wei Zhang, Xue Zhao, Jin Pan, Yuexin Ma, Xinge Zhu
Artificial Intelligence · Cross-listed from cs.LG
Loss-Guided Pretraining Data Selection for Time-Series Foundation Models
Yike Li, Shaoxu Song and Jianmin Wang
Time series foundation models (TSFMs) are pretrained on heterogeneous collections containing billions of observations, yet their training windows are typically sampled without estimating whether they provide useful learning signal. We introduce a static data-selection framework that scores each window with a reference forecaster and retains an intermediate interval within every source
dataset. Specifically, we connect
forecasting loss to optimization difficulty by showing that normalized squared loss controls the per-sample gradient norm under a local Jacobian condition. We then define a reference loss score and apply
dataset-stratified selection to preserve the diversity of samples. Across various TSFM architectures, our method outperforms random selection by an absolute margin and even improves both relative MASE and CRPS over full-data pretraining by retaining fewer candidate pretraining windows. Further analyses show strong cross-scale and cross-architecture score correlations, indicating that a small reference model can often select data for larger targets, provided that the reference and target share compatible difficulty orderings.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CV
V-JEPA Policy: Building Effective World-Action Models on Predictive Visual Latents
Yang Zhang, Jiangyuan Zhao, Chenyou Fan et al. (+4 more)
World-action models (WAMs) couple future visual-state prediction with action generation. By adapting
video generators or image-editing models pretrained at scale, a prominent line of recent WAMs inherits both predictive knowledge and the models in which it was learned. We ask whether a predictive visual latent space induced by large-scale predictive pretraining can instead provide a sufficient foundation for effective WAM learning without inheriting a complete pretrained visual generative model. To answer this question, we introduce V-JEPA Policy, a simple framework that builds a WAM on the latent space of a frozen V-JEPA 2.1 encoder. An instruction-conditioned future-latent predictor and a flow-matching action expert are jointly learned from scratch in a single downstream stage, with the predictor's future-informed context key–value states conditioning action generation. With 0.9B total parameters, of which 0.6B are trainable, V-JEPA Policy achieves competitive performance with representative WAM and
vision-
language-action baselines across LIBERO, LIBERO-Plus, and RoboCasa-GR1. Comparing visual foundations under the same downstream framework and training budget identifies V-JEPA latents as more effective than the discriminative, reconstructive, and
video-understanding-oriented alternatives, particularly under distribution shifts. Beyond task-specific learning, pretraining the predictor on DROID
video–instruction pairs without action labels and adapting it into a WAM yields substantial gains in downstream control and out-of-distribution generalization. Together, these findings establish predictive visual latents as a foundation for effective WAM learning from task-specific demonstrations and for transferring future-modeling knowledge acquired from broader in-the-wild
videos. Project page: https://breez3young.github.io/vjepa-policy/
Read the full abstract ▾
All authors Yang Zhang, Jiangyuan Zhao, Chenyou Fan, Jiayu Hu, Xiu Yuan, Chenjia Bai, Xiu Li
Artificial Intelligence · Cross-listed from cs.CV
Codebook-Guided Cross-Modal Knowledge Distillation for Structurally Heterogeneous Features
Dae Ung Jo, Jongin Lim, YoungJoon Yoo et al. (+1 more)
Cross-modal knowledge
distillation transfers knowledge from a teacher modality to a student modality. Existing feature-level
alignment methods typically assume that teacher and student features reside in structurally alignable representation spaces. However, this assumption does not hold when cross-modal features are structurally heterogeneous and lack clear unit-level correspondence, such as 2D spatial visual grids and 1D temporal audio sequences, thereby limiting the applicability of feature-level
alignment. To address this challenge, we propose a cross-modal
distillation framework that enables effective knowledge transfer across structurally heterogeneous feature spaces via a vector-quantized codebook. Specifically, teacher features are abstracted into a set of vector-form codes regardless of their original feature structure, and the selected codes serve as concept-level anchors for student learning. Code selection is guided by both task relevance and student compatibility, allowing the student to receive transferable teacher knowledge without requiring direct unit-level feature
alignment. Experimental results across diverse cross-modal
distillation scenarios demonstrate the effectiveness of the proposed framework on classification and semantic
segmentation tasks.
Read the full abstract ▾
All authors Dae Ung Jo, Jongin Lim, YoungJoon Yoo, Daeho Um
Artificial Intelligence
Asking for What Was Never Requested: Horizontal and Vertical Proactivity in Agents
Ido Levy, Asaf Yehudai, Segev Shlomov et al. (+2 more)
An
agent that uses tools typically responds to what the user explicitly asks, yet completing the task may require information the user never requested. Work on proactive
agents mainly studies whether and when an
agent should act on its own, not what information it should pursue. We study a distinct axis of proactivity: its content. Horizontal proactivity pursues unstated information that the current context already identifies, and vertical proactivity pursues needs that only earlier evidence reveals. A need graph, recovered from a
benchmark's own decomposition, records which needs depend on which, so both forms, and whether the
agent stops at the right time, can be scored from a transcript without a model judge. To learn this behavior, we propose Q&D (questioner and drafter), which trains a questioner to prefer the question whose continuation retrieves more of the required evidence, with no reward model or judge. On held-out splits of three multi-hop question-answering
benchmarks, at equal
retrieval spend, the trained questioner improves both forms of proactivity over the same model, prompted, and outperforms a prompted model $15\times$ larger in the same role on two of the three, and the gain persists after controlling for question volume and length. Without further training, we place the questioner in an interactive customer-service
agent with a simulated customer, where it completes more tasks while asking fewer questions, and in retail it outperforms the $15\times$ larger model with fewer follow-up turns from the customer. These results show that proactivity depends not only on whether an
agent acts without being asked, but also on what it chooses to pursue and when it stops.
Read the full abstract ▾
All authors Ido Levy, Asaf Yehudai, Segev Shlomov, Asaf Adi, Leshem Choshen
Artificial Intelligence · Cross-listed from cs.PL
DatalogBench: Evaluating Large Language Models on Text-to-Datalog Synthesis
Yuan Li, Hanyun Jiang, Guowei Tian et al. (+2 more)
Datalog underpins
reasoning tasks such as program analysis, but its programs are hard to write. Existing synthesizers automate this task but require users to state their intent as input-output examples. Large
language models (LLMs) suggest a more natural route, text-to-Datalog synthesis from a natural-
language question, yet how well they do so has not been systematically evaluated. We present DatalogBench, a
benchmark of 136 text-to-Datalog synthesis tasks curated from existing Datalog-based artifacts. Synthesized programs are graded by execution on held-out inputs against an oracle validated by mutation analysis. Across six LLMs and four prompting configurations, exact match peaks at 68.4%, and relation descriptions or an input-output example have only modest, model-dependent effects. Under direct prompting, most failures occur at compile time, typically because a model invents auxiliary predicates that it never declares or types consistently. Two coding
agents reach up to 83.8% and eliminate nearly all such failures, leaving mostly semantic errors concentrated in recursive tasks. DatalogBench thus identifies recursive
reasoning and decomposition as open challenges for current LLMs and
agents, and offers a reliable, execution-grounded measure of both.
Read the full abstract ▾
All authors Yuan Li, Hanyun Jiang, Guowei Tian, Chengpeng Wang, Peisen Yao
Artificial Intelligence · Cross-listed from cs.CL
Follow the Entities: A Corpus Map for Agentic Search
Soyeong Jeong, Sujay Kumar Jauhar, Sung Ju Hwang et al. (+1 more)
Answering questions and completing tasks over large document collections often requires connecting evidence spread across multiple documents, such as a project's approval recorded in one, its requirements in another, and its latest status in a third. Recent LLM
agents approach this by iteratively searching the full corpus rather than reading only a fixed set of top-ranked documents. However, when the corpus is exposed only as a flat collection of files, a relevant document gives no indication of how it relates to others, so the
agent must rediscover these relationships for every query, often missing complementary evidence while simultaneously consuming substantial additional
tokens. To address this, we introduce CorpusMap, a
navigation layer that organizes the corpus around its recurring entities, which are identifiable from the documents themselves and can link a single document to many others across sources. Specifically, CorpusMap represents each recurring entity as an Entity Page that aggregates information about it and links to every document that refers to it, forming a graph between entities and documents that the
agent can traverse to gather otherwise disconnected evidence. Moreover, since CorpusMap is constructed offline by resolving mentions of the same entity across documents, its links are shared across queries rather than rediscovered repeatedly at inference time. Using 7 different models with 3
benchmark datasets, we show that CorpusMap improves both evidence discovery and answer quality over raw-corpus agentic search while using fewer
tokens on average, and further outperforms 4 alternative
navigation layers, suggesting that entities serve as effective anchors for navigating large document collections.
Read the full abstract ▾
All authors Soyeong Jeong, Sujay Kumar Jauhar, Sung Ju Hwang, Andrew Joohun Nam
Artificial Intelligence · Cross-listed from cs.CV
ResComEmb: Effective and Efficient Multimodal Embedding via Residual Homogeneity Compression
Zijing Cai, Yuzhe Wang, Jingxian Zhu et al. (+2 more)
Multimodal large
language models (MLLMs) have shown strong potential for universal
multimodal representation learning. However, existing methods either compress each input into a single vector, limiting fine-grained expressiveness, or retain long sequences of visual-
token vectors, incurring substantial storage and interaction costs. To resolve this trade-off, we propose ResComEmb, a trainable framework for effective and efficient universal multi-vector
multimodal embedding. ResComEmb first encodes each input at native dynamic resolution into ordered global, intermediate, and fine-grained views. After MLLM contextualization and
embedding projection, a trainable Residual Homogeneity Compression (RHC) module reduces within-granularity redundancy and cross-granularity repetition under explicit visual
token budgets. Then, ResComEmb introduces a length-adaptive Bidirectional Late-Interaction Matching mechanism for robust query-document scoring, which averages the strongest
token-level matches in each direction and combines the two scores using a weight based on how many valid
tokens each side has. Extensive experiments on MMEB, ViDoRe V1, and ViDoRe V2 show that ResComEmb produces higher-quality universal
multimodal embeddings than VLM2Vec-V2, and outperforms ColQwen2.5 in visual document
retrieval using only 37.5% of its full visual
token budget, demonstrating a favorable effectiveness-efficiency trade-off.
Read the full abstract ▾
All authors Zijing Cai, Yuzhe Wang, Jingxian Zhu, Fengbin Zhu, Richang Hong
Artificial Intelligence
OptiCom : A Unified Framework for State-Conditioned Composition in LLM-Driven Optimization
Chenxing Wei, Sichen Liu, Lizhao Liu et al. (+6 more)
Large
language models (LLMs) are increasingly deployed to solve complex scientific and practical problems via iterative optimization. However, dynamically coordinating diverse search mechanisms as candidate quality, failure modes, and resource budgets evolve remains a critical open challenge. Targeted empirical diagnostics reveal that mechanism effectiveness is highly state-dependent. Motivated by this, we analyze how individual decisions drive final outcomes, decomposing the expected terminal improvement under a shared budget into cumulative decision opportunities minus cumulative selection losses. Guided by this opportunity-loss theoretical foundation, we propose OptiCom, a unified framework that represents LLM-driven optimizers within a shared configuration space: C=(A,Q,O,E,M,S), corresponding to artifact, query, operator, evaluation,
memory, and strategy. Operating within this space, a fast LLM-based Optimization Controller dynamically composes immediate mechanisms through structured Action Packages, while a slower Strategy Adapter refines long-term selection preferences, operator weights, and templates based on accumulated trajectory feedback. Comprehensive evaluations across 32
benchmark groups demonstrate the superiority of framework: OptiCom achieves an average Max-score rank of 1.72 among 14 evaluated configurations, securing the top score in 23 groups. Ultimately, these results highlight the broad applicability and high extensibility of OptiCom as a general-purpose paradigm for robust LLM test-time scaling.
Read the full abstract ▾
All authors Chenxing Wei, Sichen Liu, Lizhao Liu, Ningyuan Sun, Chen Bingzhou, Ying He, Bo Jiang, Fei Yu, Yao Shu