Machine Learning
Transfer Learning for Edge Classification on Dynamic Text-Attributed Graphs
Tyler Bonnet and Marek Rei
Learning transferable representations for dynamic text-attributed graphs (DyTAGs) requires models to capture underlying interaction dynamics that persist across domains. However, existing methods tend to overfit to domain-specific structural, temporal, and semantic patterns, limiting edge classification performance under distribution shifts. To expose and address this, we formally establish a leave-one-domain-out (LODO) transfer learning protocol for edge classification on DyTAGs. Under this protocol, we demonstrate that state-of-the-art self-supervised methods for dynamic graph learning perform poorly when transferred to unseen domains. Strikingly, existing methods underperform a structurally and temporally unaware Bag of Events (BoE) model we introduce, which inputs only unordered sequences of node and edge text features. Proceeding from the BoE, we propose Spatio-Temporal Semantic
Alignment (STSA), which integrates a spatio-temporal encoder that fuses representations of time deltas and node occurrence frequencies into a unified
manifold. STSA is trained with a Contrastive Semantic
Forecasting objective, which anchors edge representations to a multi-domain textual latent space initialized by a pretrained
language model, providing a robust prior that outperforms BoE and all existing methods we evaluate.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Sliced Orlicz-Wasserstein
Binh Thuan Tran and Khai Nguyen
We propose sliced Orlicz-Wasserstein (SOW) distance which is a generalization of sliced Wasserstein (SW) distance. SOW replaces the $L^p$ norm in SW with a Luxemburg norm cost induced by an Orlicz function $\phi$. First, we prove that SOW distance is a metric on the space of measures with finite Orlicz norm, and show that it recovers the SW distance when the Orlicz function is $\phi(x)=x^p$. Next, we derive the topological properties of the SOW distance. In particular, we show that convergence under SOW implies weak convergence, and the converse is true under the compact support condition. We then present the theoretical results for estimating the SOW distance. We derive sample complexity for both the distance itself and the powered functional of the distance, and prove their minimax optimality. In addition, we discuss the computational
algorithm for approximating the SOW distance by Monte-Carlo estimation and bisection search, as well as the associated approximation error and computational complexity analysis. Our experimental results reveal the superior computational efficiency of SOW compared with Orlicz-Wasserstein (OW) distance. Also, in the experiments, we demonstrate the favorable flexibility of SOW distance over SW in detecting differences between distributions by comparing their performance in two-sample tests and evaluating generative models on image
datasets.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
SynCo: Learning Cross-Modal Synergy by Contrasting Interaction Residuals
Yavuz Yarici and Ghassan AlRegib
Multimodal contrastive learning is a dominant paradigm for learning transferable representations from unlabeled data, but standard objectives primarily capture information that is redundant between modalities. Partial Information Decomposition (PID) shows that task-relevant information in
multimodal data decomposes into three components: redundancy shared between modalities, uniqueness specific to each modality, and synergy available only from their joint observation. Recent frameworks extend contrastive learning to capture all three components, yet synergy remains undertrained in practice. We propose SynCo (Synergy Contrastive Learning), a method that directly addresses synergy undertraining through dedicated supervision on an interaction residual. SynCo fits a linear projector to predict the fused representation from independently computed unimodal features, and the resulting interaction residual, which removes the linearly unimodal-predictable component, receives dedicated contrastive supervision at negligible computational cost. On the controlled Trifeature
benchmark, SynCo achieves state-of-the-art synergy capture with a $+5.98\%$ gain over the baseline, and on real-world
benchmarks from MultiBench, DARai, and MM-IMDb, SynCo consistently outperforms or matches prior methods across diverse modality combinations and task types. The method operates as a plug-in to existing contrastive
multimodal frameworks without modifying the underlying fusion architecture and can further improve synergy capture when combined with other methods.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
VCRE-Fib: View-Conditioned Regional Evidence for Fine-Grained Ultrasound Grading of Schistosoma japonicum-Associated Liver Fibrosis
Ziyang Xu, Shuli An, Hao Zhou et al. (+5 more)
Accurate assessment of Schistosoma japonicum-associated liver fibrosis is essential for disease management and long-term follow-up in endemic regions. Ultrasound provides non-invasive imaging, but complex local echogenic patterns and anatomical structures make fine-grained grading challenging. Existing deep learning methods can predict fibrosis scores, yet directly incorporating acquisition views and regional cues into grading while retaining spatial information for inspection remains an open problem. Here we present VCRE-Fib, a view-conditioned regional evidence framework that integrates anatomical context, local information, and global image assessment for fine-grained ultrasound grading. The framework forms view-conditioned local grading evidence before spatial pooling, uses weak localization to guide its aggregation, and combines it with global predictions. Image-only inference jointly returns a fibrosis score, acquisition view, and candidate abnormal-region map. We developed and evaluated the method on a re-curated cohort of 108,709 ultrasound images from 6,373 patients across 35 centers. On a patient-disjoint test set of 4,107 images from 240 patients across four centers, VCRE-Fib reduced the prespecified composite grading risk by 7.115% relative to SFibAI trained and evaluated on the same data split. Image-level mean absolute error decreased from 0.391 to 0.378, alongside lower patient-max, patient-median, and center-balanced risks. The full model also achieved lower composite grading risk than variants that separately removed view conditioning or weak localization. These results support incorporating anatomical context and regional evidence into ultrasound grading while exposing spatial predictions for inspection alongside severity estimates.
Read the full abstract ▾
All authors Ziyang Xu, Shuli An, Hao Zhou, Haitian Zhong, Tingting Wu, Tao Wang, Kun Yang, Tieyong Zeng
Machine Learning · Cross-listed from cs.IR
Mend the Measurement Gap: Latent User Preference Modeling for Short-Form Video Recommendation
Shuo Chang, Yueqi Wang, Zihuan Diao et al. (+7 more)
Recommender systems rely heavily on heterogeneous behavioral feedback to infer user preference. Although abundant, these signals are imperfect measurements: the same observed behavior can arise from different underlying states, such as genuine enjoyment, passive consumption, or inattention. The challenge is especially acute in short-form
video, where watch-based signals are strongly affected by measurement confounders such as
video duration - the same watch time can imply different levels of preference for
videos of different lengths, while ratio-based metrics can systematically favor short
videos. As a result, optimizing raw engagement can amplify measurement artifacts rather than improving user value. We propose a Factorized Latent Value Model (FLVM) for measuring user preference from heterogeneous behavioral feedback. The model treats observed behaviors as noisy measurements of a low-dimensional, factorized latent value state and uses structured output heads to model heterogeneous feedback signals. A restricted baseline path captures predictable variation from measurement-confounding features such as
video duration, user propensity, and session context, while a routed latent path estimates preference-relevant value advantage. The resulting latent value score can be integrated into an existing recommender system as a ranking feature or ranking score. On YouTube Shorts, a major short-form
video platform, this model improves offline metrics and lifts a primary viewer enjoyment metric by 2.67% in online A/B tests.
Read the full abstract ▾
All authors Shuo Chang, Yueqi Wang, Zihuan Diao, Ali Montazer, Jiangguo Zhang, Joyneel Misra, Dapeng Hong, Tomer Margolin, Sourabh Bansod, Ningren Han
Machine Learning
HamiFormer: Dual-Expert Diffusion Fields with Affine Symplectic Maps
Haoxiang Huang, Xiang Liu, Shuwei Wang et al. (+2 more)
Predicting smooth dynamics and collisions requires modeling continuous
evolution and abrupt state changes. We introduce HamiFormer, a dual-expert
diffusion field combining whole-window denoising with residual-corrected
Hamiltonian propagation. Their mixed-state feedback attenuates the direct contribution of inherited autoregressive error: each mixed state guides subsequent propagation within the jointly refined window. Parallel Local Affine Scan (PLAS) amortizes iterative refinement across rectified-flow steps and evaluates derivatives in parallel across physical time. PLAS's affine
symplectic maps achieve lower solver error and runtime than sequential explicit Euler in our evaluation. A Regime Model Tree specializes residuals and routing to balance typical-state accuracy against large tail errors. Our analysis gives conditions for physically consistent refinement and warm-start tracking, and finite-window error bounds under
diffusion feedback. In 192-step evaluations, HamiFormer reduces normalized phase-space MSE by 26.3% against PhysiFormer on HamiBalls-1 and 21.4% against DiT on HamiBalls-2, with comparable model capacities. Disjoint-interval comparisons show the lowest late-horizon position and momentum errors among baselines on both
datasets. Project page: https://hamiformer.github.io/.
Read the full abstract ▾
All authors Haoxiang Huang, Xiang Liu, Shuwei Wang, Jingheng Ma, Sen Cui
Machine Learning · Cross-listed from eess.SP
Radiomap Blind Prediction under Incomplete Observation: Error Characterization and Correctable Propagation-Prior Learning
Xiaojie Li, Yu Han, Han Fang et al. (+4 more)
Radiomap blind prediction aims to infer radiomaps from observable representations of the propagation environment and base station configuration without field measurements. In practice, the observable representations are inherently incomplete. Thus, the target radiomap is not fully determined by the inputs when generalizing to unseen configurations or environments. Under incomplete observation, we establish a population-level theory of deterministic radiomap blind prediction that identifies the conditional mean as its optimal target and separates prediction error into reducible predictor approximation and irreducible uncertainty caused by missing physical information. The framework further characterizes the train-test risk gap and the uncertainty reduction enabled by observation enrichment. Building on it, we reveal the dual role of propagation priors: they provide physically grounded guidance, yet their implementable forms may bias the attainable predictor. This motivates RadioDecomp, which treats a prior-guided predictor as a correctable base and learns its remaining predictable discrepancy through residual refinement. To evaluate RadioDecomp across distinct propagation-prior designs, we instantiate it with a feature-guided monolithic base and a LoS-Shadow structured base, yielding RadioFR and RadioLSR, respectively. Across random, cross-configuration, and cross-environment settings, experiments confirm the benefit of propagation-related representations and show that both instantiations improve upon their respective bases. Further controlled studies on base capacity, training-support coverage, and observation coarsening corroborate the proposed analysis.
Read the full abstract ▾
All authors Xiaojie Li, Yu Han, Han Fang, Shangqing Liu, Guangxu Zhu, Shi Jin, Chao-Kai Wen
Machine Learning
Permutation-Equivariant Flow Matching for Alignment-Free Neural Weight Generation
Arkadi Piven, Yam Eitan, Guy Bar-Shalom et al. (+5 more)
A trained neural network can be represented by a parameter vector in high dimensions. Learning distributions over these vectors enables the generation of new models across various tasks and architectures. A central challenge is permutation
symmetry: permuting hidden
neurons can produce distant parameter vectors representing the same function. This introduces variations that a generative model must account for when learning from trained networks. Existing methods typically address this using networks derived from a common base model or costly approximate
neuron alignment. We instead parameterize a flow-matching velocity field with a permutation-equivariant Graph Meta Network, enabling direct learning from independently trained networks without
alignment. Extensive experiments show that our method closely reproduces the joint statistics of accuracy, functional similarity, and weight similarity of independently trained collections, providing evidence of generation beyond checkpoint memorization. A single conditional model also generates task-specific networks on heterogeneous architectures and generalizes to unseen hidden-width configurations. On a tabular domain-shift task, intermediate conditioning produces individual networks with performance comparable to logit ensembles across both domains. Taken together, our results show how permutation equivariance enables learning from diverse collections of independently trained networks without permutation
alignment.
Read the full abstract ▾
All authors Arkadi Piven, Yam Eitan, Guy Bar-Shalom, Fabrizio Frasca, Daniel Cremers, Thomas Dagès, Ron Kimmel, Haggai Maron
Machine Learning
Over-the-Air Federated Learning in Heterogeneous Mobile Wireless Networks
Ming Xiang, Nicolò Michelusi, Yonina C. Eldar et al. (+1 more)
Over-the-air computation has emerged as a scalable and efficient solution for deploying
federated learning
algorithms in wireless networks by exploiting waveform superposition for simultaneous model aggregation. Most existing work struggles with heterogeneous fading channels. These approaches either enforce unbiased updates from all devices or allow partial device contributions, requiring careful tuning of the convergence bound to mitigate bias under specific fading models. However, the former significantly amplifies receiver noise due to the weakest channel, whereas the latter is sensitive to fading model mismatch and converges only to a biased objective. To tackle these challenges, we propose FedOAG, which employs algorithmic components to automatically satisfy energy constraints via gradient normalization and evenly mix devices' updates through implicit gossiping. Importantly, FedOAG does not require transmission from all devices, nor does it rely on a specific fading model or knowledge of time-varying statistical channel distributions. We show that FedOAG converges to a stationary point of an unbiased non-convex objective at the best possible rate $O(1/\sqrt{T})$ for any
stochastic first-order method. We corroborate our analysis with numerical experiments over dynamic wireless conditions on real-world
datasets.
Read the full abstract ▾
All authors Ming Xiang, Nicolò Michelusi, Yonina C. Eldar, Lili Su
Machine Learning
UniCache: Task- and Type-Aware KV Cache Compression for Unified Multimodal Models
Wanqi Yang, Yuexiao Ma, Mei Xie et al. (+2 more)
Unified
multimodal models combine understanding, generation, and editing within a single network, offering a promising foundation for versatile
multimodal applications. However, growing
multimodal contexts make KV cache storage and access increasingly costly. Existing KV cache compression methods are typically tailored to specific tasks and single-modality caches, while overlooking changes in cache importance across tasks and timesteps. However, in unified
multimodal models, each task involves multiple KV cache types, and both their composition and dynamics differ across tasks. As a result, a single compression policy overlooks task- and type-specific requirements, leading to the loss of critical information and degraded quality across tasks. Based on these findings, we propose UniCache, a training-free framework for task- and type-aware KV cache compression. UniCache identifies the cache segments activated by each task and assigns suitable compression policies through offline calibration. It coordinates their parallel execution under a shared storage budget through
attention-guided allocation and task-aware temporal scheduling. Experiments show that UniCache achieves $5\times$ KV cache compression for understanding and editing and $2.5\times$ for generation with negligible quality loss, while increasing throughput by up to $1.78\times$ in long-context settings, significantly improving the practicality of scaling unified
multimodal models to longer context.
Read the full abstract ▾
All authors Wanqi Yang, Yuexiao Ma, Mei Xie, Xiawu Zheng, Shiwei Liu
Machine Learning
Retimed Bellman Flows: Escaping the Impossible Triangle of Velocity Bootstrapping
Boyang Xu, Shengzhe Chen and Hao Yan
Flow critics learn return distributions by transporting Gaussian noise to Bellman endpoints via continuous velocity fields. While velocity bootstrapping stabilizes training by querying a successor teacher, existing methods face a structural dilemma: on straight paths, no residual-free same-time affine mapping can preserve Gaussian initial noise while maintaining an unbiased target. To overcome this limitation, we introduce Retimed Bellman Flows (ReBF). ReBF queries the teacher critic at a dynamically shifted earlier flow time, aligning intermediate student and teacher trajectories. By combining this retimed clock with fresh, decoupled noise generation, ReBF constructs a provably conditionally unbiased velocity target that preserves the Bellman fixed point and contracts under Wasserstein distances. Empirically, ReBF reduces $W_1$ distance to ground-truth return distributions by up to $7.7\times$ on synthetic MRPs and outperforms existing flow critics across 38 challenging OGBench and D4RL offline reinforcement learning tasks.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
Routing Drift Alone Does Not Diagnose Failure in Merged MoE LLMs
Yuanyi Wang, Yanggan Gu, Su Lu et al. (+7 more)
Model merging efficiently combines specialized large
language models (LLMs) without joint retraining, but can substantially alter expert routing in Mixture-of-Experts (MoE) models. Such routing drift is often interpreted as routing failure, raising a fundamental question that remains unclear: does routing drift after MoE merging actually indicate routing failure, and what evidence should justify repair? We investigate these questions across DeepSeekMoE, OLMoE, and Qwen3-MoE proposing a routing analysis toolkit for controlled counterfactual interventions and
token-level analysis. By crossing source and merged router inputs and parameters, we attribute most expert reassignments to input shifts rather than parameter changes at the same layer. However, source-relative routing differences poorly predict next-
token likelihood gains from source-route restoration, and different expert selections can produce directionally similar mixture outputs. We therefore operationalize routing failure as task loss recoverable under a specified routing intervention, with non-routing parameters fixed. These tests detect recoverable loss under deliberate router corruption, whereas source-route restoration does not establish reliable task benefits in the evaluated merged models. Motivated by these, we propose Selective Router Repair (SRR) as a case study, and find that source-specialist
token-likelihood advantages do not reliably identify beneficial local corrections. Together, these findings show that routing drift alone is insufficient evidence of routing failure: source-informed corrections must be judged by their task-level intervention effects. The analysis toolkit and SRR code are released.
Read the full abstract ▾
All authors Yuanyi Wang, Yanggan Gu, Su Lu, Guanghao Zhu, Pengkai Wang, Yifan Yang, Congkai Xie, Zhaoyi Yan, Jianmin Wu, Hongxia Yang
Machine Learning · Cross-listed from cs.FL
Learning Shuffle Ideals with Membership Queries and Contrastive Examples
S. Mahmoud Mousawi, Pierluigi San Pietro and Sandra Zilles
This paper studies learning of shuffle ideals with membership queries as well as with contrastive queries—a form of membership query that reveals not only whether a selected word $w$ is in the target
language or not, but also provides a most similar word $w'$ that belongs to the target
language if %and only if $w$ does not. For both settings, it is shown that even some very simple classes of shuffle ideals cannot be learned efficiently. By contrast, we obtain positive learnability results for classes of shuffle ideals that meet certain structural conditions. In the case of membership queries, these structural conditions are related to the previously studied notion of universal words, and raise new questions in word combinatorics. In the case of contrastive queries, the structural conditions relate to partitioning sets of words that are listed in shortlex order.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
Rank Collapse Is Recoverable, Growing $|Q|$ Is Not: Out-of-Sample Early Warning for Value Divergence in High-UTD Soft Actor-Critic
Tianqi Bu, YuXuan Peng, Junteng Tu et al. (+1 more)
Raising the update-to-data (UTD) ratio breaks off-policy critics in two ways grouped as "plasticity loss": collapsing representations and growing value magnitude $|Q|$. We separate them in Soft Actor-Critic (SAC) with scaled critics (width 2048, no normalization). Collapse is survivable: at UTD ratio 16, HalfCheetah critics with most units dormant keep learning, and the training guard, which stops runs whose loss or $|Q|$ explodes, never flags them. Within one high-UTD SAC configuration, runs start close together, and how far a critic's $\log_{10}|Q|$ has climbed by step 15k, its early growth, ranks the runs by how soon the guard flags them. At 15k, a flagged run's $|Q|$ sits a median of over a hundredfold below its flag level, yet the climb's rate already orders the flags (Harrell's C and out-of-sample AUC 0.78 on Walker2d, 0.98 on Ant, at UTD ratio 4). Dormancy does not. Aborting on this rate saves about a tenth of held-out Walker2d compute and stays net-positive live. A LayerNorm critic lowers the rate, removes the flag on Walker2d at UTD ratio 4 and lowers the return.
Read the full abstract ▾
All authors Tianqi Bu, YuXuan Peng, Junteng Tu, Henghui Xiao
Machine Learning
Change the Product, Keep the Parameters: Associative Algebra Layers for Transformers
Ilya Koziev and Ivan Oseledets
Fast
matrix multiplication
algorithms keep the product fixed and search for a cheaper way to evaluate it. We instead ask whether a
Transformer's learned projections can use a different, cheaper product altogether. Building on an associative-
algebra construction that replaces ordinary
matrix multiplication with a sparser interaction table over the same weight blocks, we construct a family with quadratic arithmetic in the
matrix dimension when the physical block size remains fixed, and derive finite-shape constraints for GPU execution. The construction is provably optimal for its bilinear rank by the Alder–Strassen bound and can be realized as row-typed rectangular projections compatible with causal masking and KV-cached decoding. We provide an empirical test of this approach by training two approximately 110M-parameter decoder-only
Transformer LMs from the same recipe and 12.3B-
token budget, differing only in their feed-forward layer: one uses ordinary dense
matrix multiplication and the other uses the associative-
algebra product. Across four
prompt domains, the algebraic model achieves a 6.2–7.8% increase in end-to-end generation throughput, while obtaining lower scores on all three reported downstream metrics. We treat these results as a feasibility and trainability check for the proposed approach at small scale, leaving further investigation to future work.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Dongdong Wang, Qingqi Song, Yuzhou Chen et al. (+3 more)
Large
vision-
language models (VLMs) have emerged as a powerful paradigm for urban and spatial AI. However, current state-of-the-art large VLMs still struggle with quantitative
reasoning on remote sensing imagery. Existing
benchmarks and
algorithms are predominantly based on qualitative Visual Question Answering (VQA), providing limited insights into the quantitative
reasoning capabilities of VLMs for built environment metrics. To address this gap, we develop Quantitative Urban and Spatial AI
benchmark (USAI-Quant), the first
benchmark designed to quantitatively evaluate VLM's
reasoning capabilities on built environment metrics via remote sensing imagery. USAI-Quant is curated from the 335 largest U.S. cities, aligning high-resolution remote sensing images with quantitative built environment metrics. We then evaluate both general-purpose and remote sensing VLMs (RS-VLMs) by applying VQAs to tens of built environment metrics across three complexity levels. Our results reveal that current state-of-the-art models consistently fall short on numeric
reasoning tasks. We further conduct in-depth analyses across models, question types, and geographic locations, uncovering insights into performance variability and task-specific challenges.
Read the full abstract ▾
All authors Dongdong Wang, Qingqi Song, Yuzhou Chen, Deepak Balakrishnan, Ravi Shankar Srinivasan, Shenhao Wang
Machine Learning · Cross-listed from cs.CL
OpenTumorBoard: A Real-World Benchmark of Multidisciplinary Tumor Board Discussion Trajectories
Anqi Li, Zhixuan Ge, Yixuan Duan et al. (+9 more)
Multidisciplinary tumor boards integrate
multimodal clinical observations and longitudinal patient histories through specialist discussions, yet
benchmarks rarely capture these real-world trajectories. We introduce OpenTumorBoard, a
benchmark with 611 patient cases and 19,157 discussion turns across ten specialist roles, transcribed from 12,534 minutes of publicly available tumor board recordings on YouTube. The
benchmark evaluates two settings: SPECIALIST TURN, in which an LLM responds to a clinically significant question posed during a real discussion, and BOARD SIMULATION, in which it generates an entire back-and-forth discussion and reaches a consensus on therapy recommendations,
surgical plans, next actions and clinical trial matching. Evaluation of 14 general-purpose frontier and medical LLMs reveals substantial limitations: the best models score 3.43 out of 5 in clinical equivalence to specialist answers and 2.78 out of 5 in
alignment with recorded board conclusions. Supervised finetuning and reinforcement learning improve performance on a held-out test set, suggesting that real-world discussion trajectories can support model adaptation. Three M.D. experts review a subset of the
benchmark, finding high information coverage and factuality of patient cases and strong fidelity of extracted consensus conclusions. We will release OpenTumorBoard and its automated curation pipeline to support the development and evaluation of LLMs for multidisciplinary, personalized cancer decision-making.
Read the full abstract ▾
All authors Anqi Li, Zhixuan Ge, Yixuan Duan, Jiarong Qian, Chi-Yu Chen, MingYu Lu, Huan-Yu Hsu, Yu Gu, Yue Guo, Sheng Wang, Wei Qiu, Hanwen Xu
Machine Learning
Mind the Spike: Mechanisms and Brittleness of Visual Massive Activations in Large Vision-Language Models
Jonas Ngnawé, Yann Pequignot, Sabyasachi Sahoo et al. (+3 more)
Large
vision-
language models (LVLMs) inherit massive activations from their text-only bases: spikes where a few fixed hidden channels receive values thousands of times above the typical magnitude. The text spike systematically appears in early layers at a fixed initial position, independently of input content. Visual spikes vary across images, but whether their formation follows a consistent pattern across LVLMs and how they respond to image perturbations remain open questions. We find that some LVLMs do not form visual spikes, while others spike at different rates, typically in deeper layers. We identify the trigger direction from model weights and an interpretable location rule: before the
language model decoder runs, eventual spike
tokens are largely restricted to those sharing least with the rest of the image. Crucially, visual spikes are strikingly brittle. Common corruptions frequently create and relocate spikes, and less often remove them, raising overall incidence. Our trigger-guided spike attack deliberately creates or removes spikes under a small $\ell_\infty$ budget, with 1/255 enough in nine of the ten models that spike. Finally, our preventive intervention removes only the trigger component before spikes erupt, eliminating or substantially reducing spikes on clean and perturbed images while leaving the other image
tokens nearly unchanged. Our study spans 25 adapter-based LVLMs built on 18 released text-only bases from 10 families, ranging from 2B to 72B parameters.
Read the full abstract ▾
All authors Jonas Ngnawé, Yann Pequignot, Sabyasachi Sahoo, Christian Gagné, Frédéric Precioso, Sanmi Koyejo
Machine Learning · Cross-listed from cs.AI
Decision-Sufficient State Representations: Measuring and Reducing Write-Time Regret
Bingyu Shen and Boyang Li
Long tasks produce more history than an LLM
agent can hold in its context, and more than it uses reliably even when the history fits. A growing line of work therefore has
agents carry a short written state instead: at every step a writer rewrites the state, and a reader acts from the state alone. Steps stay cheap, but anything the writer drops is lost before later decisions reveal that they need it. We quantify this loss and ask whether training can reduce it. Comparing the written state with the best state of the same size written in hindsight, we split the reader's loss into a budget loss, which any state of that size must incur, and a write-time regret, which comes from the writer's choices. In TextWorld cooking games where we control how long a fact must be carried before it is needed, a 128-
token state holding the facts wins nearly every game, while prompted
language-model writers win at most 17%. Almost all of the loss is write-time regret, and it grows with the delay. We then train the writer from the reader's own loss. DSSR (decision-sufficient state representations) scores candidate states by how well the reader acts after the writer carries them forward, and teaches the writer to prefer the better ones. This forward-rolled score predicts game outcomes ($\rho = 0.48$), whereas scoring a candidate as a fixed context, as hindsight methods usually do, does not ($\rho \leq 0.07$). On a pre-registered test split opened once, training adds +7.0 [+1.9, +12.2] points of success when facts are needed soon, bringing a plain summary writer to the level of belief- and slot-based
memory prompts. The gain shrinks as the delay grows and is significant only at the shortest delay. We trace this limit to credit assignment: keeping a fact now pays off only if every later rewrite keeps it too, which a per-step score cannot see.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
Re-derivability Decides What a Staged Agent Pipeline Recovers After an Upstream Fault
Tianqi Bu, YuXuan Peng, Junteng Tu et al. (+1 more)
One variable sets what an upstream fault costs a staged pipeline of
language-model
agents: re-derivability, how much of what a stage needs it can rebuild from the original problem. Grounding an inspector
agent in that problem is worth +0.608 [+0.517, +0.700] to +0.358 over a blind one on four open-weight backbones served with thinking disabled, and on the two Qwen backbones the blind inspector changes no item at all. That head-to-head is exploratory. One deterministic fault enters the first stage, and we re-expose the original problem to $k = 0,\dots,3$ of the downstream stages with
agents, items, fault and
topology held fixed, on 120 gsm_hard items per arm at temperature zero. Accuracy under fault rises on four of four backbones, from +0.233 to +0.392, the largest Holm-adjusted $p$ being $2.1\times10^{-6}$. A registered kill test rules out
tokens. Blanking every word holds the word slots fixed, and retention tracks the visible fraction on four of four, climbing from 0.221 to 0.692 on the primary. Those two families are confirmatory and everything else here is exploratory. The interaction excludes zero on two of four backbones under the registered pipeline, four of four under a three-stage pipeline, and three of four under full message history, the primary at +0.317. On Llama-3.1-8B the fault carries no detectable cost at any dose, so the other three carry every claim about what a fault costs. Re-derivability also sets what the architecture costs, and no decomposition we measured reliably beats one direct call. With no fault injected the registered pipeline loses to that call by -0.267, -0.125 and -0.317, and on Phi-4 reads +0.058 at $p = 0.118$, which the test fails to separate from zero. The repair that works is cheap and front-loaded: the first re-grounded stage buys +0.394 of matched retention for +59.8
tokens per item on Qwen3-14B, and the stages after it buy nothing.
Read the full abstract ▾
All authors Tianqi Bu, YuXuan Peng, Junteng Tu, Henghui Xiao
Machine Learning
Derivative-Informed Training of Neural Operators On-the-Fly via Sketched Tangent Consistency
Xinhan Yang, Lu Lu and Shancong Mou
Derivative-informed training improves neural operators by directly supervising their input-output sensitivities, which is crucial when neural operators are used as differentiable surrogates for inverse problems, PDE-constrained optimization, design, and control. However, existing methods rely on offline-generated derivative labels, making data generation slow, storage-intensive, and difficult to adapt across
datasets, resolutions, or perturbation bases. We propose sketched tangent consistency loss (sTCL), an on-the-fly derivative-informed training objective that, for PDEs with a known and differentiable residual, enforces sensitivity consistency directly from the governing equation without offline tangent labels or neural-operator architecture changes. sTCL uses randomly sketched input perturbations to provide a lightweight derivative-level physics constraint during training. However, raw forward-sensitivity residual penalties can fail for stiff, ill-conditioned, indefinite, or coupled saddle-point tangent operators. To address this, we introduce lightweight operator-aware loss-conditioning mechanisms selected by a simple tangent-operator decision rule. Across Helmholtz, nonlinear
diffusion-reaction, Burgers, Allen-Cahn, and Navier-Stokes, with the same neural-operator backbone for all methods, the PDE-specific sTCL losses achieve solution and Jacobian accuracy comparable to offline derivative-informed training (DIFNO) while eliminating the offline derivative-data generation and storage pipeline. These results show that on-the-fly derivative-informed training need not merely amortize offline tangent-solve cost into training; with appropriate sketching and loss design, sTCL provides an effective drop-in path to derivative-informed neural operators. Code is available at https://github.com/yang9579/Derivative-informed-traning-on-the-fly.
Read the full abstract ▾
Machine Learning
Understanding and Exploiting Anisotropy in Post-Training
Samyak Jha, Harshvardhan Saini, Yizhen Liao et al. (+2 more)
LLM post-training combines supervised fine-tuning (SFT), a mode-covering forward-KL objective, with reinforcement learning (RL), a mode-seeking reverse-KL objective. Frequency-weighted likelihood training leaves a well-known signature: anisotropy, in which a few residual channels carry disproportionately large activations. Anisotropy is widely documented and usually treated as a defect, yet its function and its interaction with post-training remain unclear. We first analyze it. A label-free outlier rule isolates about 5% of residual channels that are essential for
language modeling: removing them raises perplexity from 10 to over $10^6$, versus 35 for count-matched random channels. Yet they barely distinguish correct from incorrect
reasoning. SFT reshapes them, whereas RL leaves them largely intact and adapts the complementary channels. These channels therefore form the model's coherence substrate, and
reasoning adaptation happens elsewhere. We then exploit this. SphereGate learns one bounded gain per residual channel on a frozen backbone. Its activation-weighted gradients provably limit movement of high-energy coherence channels and leave the remaining channels free. With 0.1M trainable parameters, SphereGate outperforms parameter-efficient baselines by 2.0–7.3 points on MATH-500 across Qwen2.5 (0.5B–7B) and Llama-3-8B, is comparable or exceeds full-model GRPO. Anisotropy is not a defect but a division of labor that post-training can exploit.
Read the full abstract ▾
All authors Samyak Jha, Harshvardhan Saini, Yizhen Liao, Yiming Tang, Dianbo Liu
Machine Learning
FedHV: Low-Overhead Hypervolume Weighting for Federated Multi-Objective Optimization
Amirardalan Dehghanpour, Seyed Mohammad Azimi-Abarghouyi and Christopher G. Brinton
Task-wise
federated multi-objective optimization (FedMOO) trains a shared model for competing prediction objectives under heterogeneous data, partial participation, and communication constraints. Existing methods commonly derive task weights from gradient or update geometry. This requires task-specific information or iterative server-side optimization. We introduce FedHV, which maps reference-relative objective slacks to closed-form inverse-slack weights. Each client optimizes one weighted loss and returns objective estimates with its model update. The protocol adds exactly 2m auxiliary scalars per participating client, yielding Theta(d + m) total per-client communication, compared with the Theta(md) task-specific communication of FSMGDA, and requires no additional synchronization stage. We analyze the resulting one-round-delayed weights under client heterogeneity, multi-step local updates, partial participation, and finite-sample objective reports. Under a fixed-horizon positive-slack reference condition, with the prescribed horizon-dependent step size and vanishing report error, FedHV achieves an O(T^(-1/2)) rate for the average squared log-hypervolume gradient norm; persistent report error determines the resulting stationarity neighborhood. The same bound controls the squared Pareto-stationarity residual. Across six Dirichlet-partitioned non-IID settings from four
vision benchmark families and three training seeds, FedHV exceeds FSMGDA and FedCMOO in mean accuracy in five settings. Among these methods and uniform scalarization, it achieves the highest worst-task accuracy in four settings and improves the difficult CIFAR-10 objective in both CIFAR10-MNIST settings.
Read the full abstract ▾
Machine Learning
Getting Motif-ated: Controllable AI Compositions from Injected Motif Prompts
Chao Peter Yang, Cynthia Rudin, Yue Jiang et al. (+2 more)
Deep learning has transformed symbolic music generation by borrowing the training paradigms of large
language models, with systems such as NotaGen now producing complete, stylistically convincing classical scores from a short
prompt. These systems could become powerful creative partners, helping musicians generate endless possibilities. However, current systems expose almost no control handles on the music itself. In principle, control handles could be built into a foundation model trained from scratch, but this is rarely practical without massive amounts of quality annotated data and compute resources. We therefore present MotiGen, a recipe for retrofitting pretrained symbolic music models to use new instruction
prompts. MotiGen injects a musical motif as a structured
prompt line, reinforces it with a scalar
attention bias toward the motif
tokens, and learns the association with a two-phase curriculum. First, it learns from focused excerpts cropped around motif occurrences in the training data, then full scores including the motifs. Our experiments show that our model composes with the prompted motif in over 92.3% of generated pieces. Generated pieces using a variety of motifs are included in our sample site: https://motigen-site.github.io/.
Read the full abstract ▾
All authors Chao Peter Yang, Cynthia Rudin, Yue Jiang, Simon Mak, Stephen Ni-Hahn
Machine Learning
Learning When to Recur: Token-Adaptive Recursion for Imbalanced Ophthalmic Domain Incremental Learning
Nanxi Yu, Kang Li, Ye Du et al. (+3 more)
Domain incremental learning is essential for adapting ophthalmic deep learning models to sequential clinical domains while preserving diagnostic expertise. Existing domain incremental learning methods predominantly address the domain shift induced by style variations. However, they often overlook the severe class imbalance inherent in real-world clinical scenarios, such as clinical referral systems. Institutions in these systems encounter drastic fluctuations in class priors, resulting in label distribution shift, a critical form of domain shift that triggers severe catastrophic forgetting. To address these challenges, we propose ToRe, a rehearsal-free and parameter-efficient framework that leverages frozen ophthalmic foundation models for robust incremental adaptation. ToRe employs a parameter isolation strategy to decouple domain-specific optimization paths, thereby helping mitigate catastrophic forgetting driven by both label distribution shift and style variations. Simultaneously, it introduces
token-adaptive recursion that adaptively allocates additional computational depth across
tokens, allowing simple
tokens to exit the recursion loop early while subjecting complex
tokens, such as those associated with lesions, to deeper recursive processing. This mechanism enhances the feature representations for minority classes, thereby supporting generalization throughout the domain incremental learning process. Extensive evaluations on nine heterogeneous
datasets demonstrate that ToRe consistently outperforms state-of-the-art methods in overall performance across the three
benchmarks, while maintaining near-zero forgetting. Together, these results support the applicability of ToRe to dynamic and imbalanced clinical environments. The code is available at https://github.com/Nancyolo/ToRe
Read the full abstract ▾
All authors Nanxi Yu, Kang Li, Ye Du, Xiaowei Hu, Weihua Yang, Shujun Wang
Machine Learning · Cross-listed from cs.RO
CollisionGAT: Controller-Agnostic One-Step Collision Screening for Multi-Agent Motion
Alan Debbas, Edwin Meriaux and Gregory Dudek
Before a team of
robots moves, each proposed step must be checked for collisions with other
robots and with obstacles. We present CollisionGAT, a graph-
attention network that reads the current and proposed states of moving
agents together with locally relevant stationary obstacles and returns one collision-risk score per moving
agent. Any controller can use these scores to accept, repair, replan, or postpone a proposed step. We mount CollisionGAT on a continuous path-following controller and on GATeD, an obstacle-blind D* Lite planner that uses typed vetoes to update its planning graphs. Exact
geometric checks supply the training labels and independently audit every executed step.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Long Qian, Bingke Zhu, Jiaqi Wei et al. (+3 more)
Diffusion vision-
language models generate answers by gradually resolving masked
tokens, making accurate conditional prediction in partially resolved states central to post-training. Masking completed answers yields coherent contexts and targets, but prescribed masks do not reflect the model's reveal decisions. Its trajectories capture these decisions, yet their provisional visible
tokens can conflict with the target response. Outcome-based reinforcement learning follows these trajectories but provides only response-level feedback, which loses contrast when sampled rewards tie. To align coherent
token-level supervision with the model's reveal decisions, we introduce Counterfactual Trace On-Policy
Distillation (CT-OPD), which combines completed teacher responses with trajectory masks from the current student. CT-OPD retokenizes each teacher response in the student's vocabulary and extracts unresolved-position masks at successive stages of the student's reverse process. For each mask, it discards provisional rollout values and reconstructs the partial state from the teacher endpoint, so the supervised positions follow the current trajectory while the visible context and targets remain consistent with the same response. The student is trained on these reconstructed states with its native categorical loss, and trajectories are refreshed as the model evolves. Across dense and sparse
diffusion architectures, CT-OPD consistently enhances
multimodal understanding and
reasoning capabilities, with gains of up to 9.80 points on the nine-
benchmark average. On the unified understanding-and-generation architecture, it also improves both visual understanding and image generation, showing that the same principle transfers across architectures and modalities. Ablations further attribute these gains to coherent reconstruction and current-model trajectory masks.
Read the full abstract ▾
All authors Long Qian, Bingke Zhu, Jiaqi Wei, Yu Li, Yingying Chen, Jinqiao Wang
Machine Learning · Cross-listed from cs.CV
OmniMoE-VL: A Sparse Vision-Language Model with Coupled Visual-Depth Routing
Long Qian, Bingke Zhu, Jiaqi Wei et al. (+2 more)
Vision-
language models (VLMs) increasingly use sparse mixture-of-experts (MoE) to scale
language-side computation, yet visual information is typically routed only after passing through a fixed cross-modal interface. This leaves an important decision unresolved: which intermediate visual representations should be exposed to
language computation for a given question? We introduce OmniMoE-VL, a sparse VLM with a coupled visual-depth routed projector. For each image-
prompt pair, the projector selects a sparse set of intermediate visual depths and reuses the resulting global preference to guide both local patch fusion and dynamic visual injection into the
language model. This design enables question-dependent visual access while preserving the native visual-
token sequence, and complements
token-level expert routing in the
vision and
language stacks. Across eight image-based
benchmarks, OmniMoE-VL achieves an average score of 85.9 with 28B total and 9B activated parameters. Controlled comparisons show that the routed visual interface provides the dominant architectural gain, while matched route and component controls, same-image route analysis, and route interventions further support the value of coupling and question-conditioned visual access.
Read the full abstract ▾
All authors Long Qian, Bingke Zhu, Jiaqi Wei, Yingying Chen, Jinqiao Wang
Machine Learning · Cross-listed from cs.RO
Copper-Policy: Focus on the Representation for Robust Robot Manipulation
Zexin Feng, Yixu Feng, Lingyu Xiao et al. (+6 more)
World Action Models (WAMs) acquire behavioral priors by modeling future scene
evolution, but predicting detailed futures in pixel or latent space incurs substantial cost. Recent evidence that co-training gains persist without test-time generation raises a question: what must a WAM learn to improve control? We introduce Copper-Policy, which learns a compact World representation with the policy rather than relying on a predefined target space. Through temporal joint-
embedding prediction, it predicts future observation
embeddings conditioned on task intention without reconstructing pixels. This prediction and action decoding shape the representation jointly, while the policy retains access to current-frame spatial detail for execution. Representation analyses show that the learned features better separate task-driven change from perturbations and provide complementary information for control. Compact prediction targets reduce training
tokens per sample, enabling a 2B-parameter model trained in 9.67 hours on 8$\times$ RTX 5090 GPUs and 6$\times$ faster than Fast-WAM on matched A100 GPUs. Copper-Policy outperforms every compared method without
embodied pretraining on RoboTwin and several
embodied-pretrained VLAs on LIBERO-Plus (80.85%). On three challenging real-
robot tasks, it performs comparably to $\pi_{0.5}$ and attains a higher average score. Together, these results show that Copper-Policy combines strong control performance with efficient training.
Read the full abstract ▾
All authors Zexin Feng, Yixu Feng, Lingyu Xiao, Shang Su, Kexin Zheng, Chang Xu, Mengkai Shi, Shuo Feng, Xintao Yan
Machine Learning
Robust Bayesian Optimization with Q-Exponential Surrogates
Richard Cornelius Suwandi, Zhidi Lin, Feng Yin et al. (+1 more)
Bayesian optimization (BO) is a widely used framework for optimizing expensive black-box objectives, but standard BO methods often use Gaussian process (GP) surrogates whose Gaussian assumption is sensitive to outliers and heavy-tailed noise. We introduce q-ED-BO, a robust BO method whose surrogate follows a univariate q-exponential (q-ED) distribution, preserving GP-BO's closed-form posterior mean and variance while a shape parameter q controls the tail behavior, recovering the GP at q = 2 and growing heavier-tailed with wider confidence bounds as q decreases. This tractability yields a closed-form q-upper confidence bound (q-UCB) with sublinear regret, and an exact closed-form q-expected improvement (q-EI) that generalizes EI to the heavy-tailed predictive, recovering classical EI at q = 2. Experiments on beamformer and adaptive filter tuning with impulsive outliers show that q-ED-BO matches or exceeds existing baselines on clean data, and under corruption, improves the strongest baseline by approximately 0.7 dB in output SINR and 1.1 to 1.2 dB in misalignment reduction.
Read the full abstract ▾
All authors Richard Cornelius Suwandi, Zhidi Lin, Feng Yin, Abdelhak M. Zoubir