Machine Learning · Cross-listed from cs.CV
LUMA: Benchmarking Segmentation via a Lightweight Universal Mask Adapter
Tobias Christian Nauen, Anosh Billimoria, Federico Raue et al. (+3 more)
Comparing
transformer backbones for image
segmentation is confounded: each is paired with a different decoder, recipe, and pretraining, so reported differences rarely reflect the backbone itself. We introduce the Lightweight Universal Mask Adapter (LUMA), a lightweight, backbone-agnostic mask-
transformer head that treats any backbone as a black-box feature extractor, letting a set of queries read from its features through cheap cross-
attention. LUMA matches the accuracy of EoMT, the state-of-the-art efficient ViT-segmenter, at cost parity, while attaching unchanged to hierarchical, convolutional, and mixture-of-experts backbones alike. Holding this head fixed, we
benchmark 20 backbones, 11 pretraining schemes and a range of resolutions on ADE20K, COCO, and Cityscapes under one modern recipe. We find that “efficient”
token mixers fail to deliver efficiency even at the high resolutions that motivate them, with plain ViT holding the throughput Pareto front at every resolution. Additionally, the pretraining objective, not the architecture, the lever the field has tuned hardest, governs
segmentation quality. Code: https://github.com/tobna/LUMA
Read the full abstract ▾
All authors Tobias Christian Nauen, Anosh Billimoria, Federico Raue, Stanislav Frolov, Brian B. Moser, Andreas Dengel
Machine Learning
Seokhee Jin, Changhwan Sung, Sunung Mun et al. (+2 more)
The classification accuracy of pretrained
Vision-
Language Models (VLMs) relies on the quality of the text
prompts. Handcrafted templates and Large
Language Model (LLM)-generated descriptions not only make predictions more interpretable, but also enable reuse of the same
prompts across heterogeneous VLMs. Recent works construct task-adapted text
prompts with a small number of labeled images. However, existing few-shot text prompting methods do not explicitly focus on misclassified examples during
prompt construction, leading to only marginal improvements even as more shots become available. To fully exploit few-shot supervision, we propose Text
Prompt Boosting (TPB), an AdaBoost-inspired framework that treats each text-
prompt-based classifier as a weak learner and sequentially aggregates them into a strong ensemble by explicitly targeting hard, misclassified examples. Extensive experiments show that TPB preserves task-intrinsic, model-agnostic cues in text space, enabling robust cross-model transfer. Across eleven classification
benchmarks, TPB improves accuracy on the source model and preserves shot-driven gains when transferred to larger, more capable VLMs, where existing methods struggle to sustain such improvements.
Read the full abstract ▾
All authors Seokhee Jin, Changhwan Sung, Sunung Mun, Hoyoung Kim, Jungseul Ok
Machine Learning
Distributed Online Bandit Submodular Maximization with Bounded Sampling Violations
Bin Du, Chang Liu, Dingqi Zhu et al. (+2 more)
We study distributed online submodular maximization under partition matroid constraints, in which multiple
agents select a limited number of actions from their own subsets sequentially to maximize the cumulative value of a sequence of objective functions. We develop a unified algorithmic framework that accommodates full-information and bandit feedback models. For both feedback models, we prove that the proposed
algorithms achieve sublinear $(1-1/e)$-regret guarantees, which are comparable to those achieved by existing centralized counterparts. Furthermore, to tackle the sampling violation issue caused by continuous relaxation and rounding, we develop a bounded
stochastic pipage rounding scheme and show that the probability of sampling violation vanishes asymptotically. As a result, the cumulative sampling violation remains sublinear in $T$, which is further shown to be not improvable under certain conditions. Numerical results validate the theoretical findings in this paper.
Read the full abstract ▾
All authors Bin Du, Chang Liu, Dingqi Zhu, Lintao Ye, Dengfeng Sun
Machine Learning
Multi-Label Node Classification with Label Influence Propagation
Yifei Sun, Zemin Liu, Bryan Hooi et al. (+4 more)
Graphs are a complex and versatile data structure used across various domains, with possibly multi-label nodes playing a particularly crucial role. Examples include
proteins in PPI networks with multiple functions and users in social or e-commerce networks exhibiting diverse interests. Tackling multi-label node classification (MLNC) on graphs has led to the development of various approaches. Some methods leverage graph neural networks (GNNs) to exploit label co-occurrence correlations, while others incorporate label
embeddings to capture label proximity. However, these approaches fail to account for the intricate influences between labels in non-Euclidean graph data. To address this issue, we decompose the message passing process in GNNs into two operations: propagation and transformation. We then conduct a comprehensive analysis and quantification of the influence correlations between labels in each operation. Building on these insights, we propose a novel model, Label Influence Propagation (LIP). Specifically, we construct a label influence graph based on the integrated label correlations. Then, we propagate high-order influences through this graph, dynamically adjusting the learning process by amplifying labels with positive contributions and mitigating those with negative influence. Finally, our framework is evaluated on comprehensive
benchmark datasets, consistently outperforming SOTA methods across various settings, demonstrating its effectiveness on MLNC tasks.
Read the full abstract ▾
All authors Yifei Sun, Zemin Liu, Bryan Hooi, Yang Yang, Rizal Fathony, Jia Chen, Bingsheng He
Machine Learning · Cross-listed from cs.RO
Domain Arithmetic: One-Shot VLA Adaptation under Environmental Shifts
Taewook Kang, Taeheon Kim, Donghyun Shin et al. (+1 more)
Vision-
Language-Action (VLA) models often fail to perform the same learned tasks under environmental shifts, such as changes in camera pose and shifts to a different but similar
robot (e.g., from Panda to UR5e). Adapting these models to the shifted environment (i.e., target domain) often requires training on multiple demonstrations for each task, which are costly to collect. To reduce the burden of data curation and training, we propose an analogy-based method that adapts VLA models under environmental shifts through weight vector arithmetic with domain-specific information addition, named Domain ARiThmetic (DART). Unlike prior approaches, DART requires collecting only a single demonstration, enabling efficient adaptation. To accurately isolate domain-specific information for addition, DART performs subspace
alignment between singular components in weight vectors to filter out noisy components. In both simulated and real-world experiments, DART outperforms existing VLA adaptation methods in one-shot scenarios across diverse visual and embodiment shifts. Code is available at https://github.com/snumprlab/dart.
Read the full abstract ▾
All authors Taewook Kang, Taeheon Kim, Donghyun Shin, Jonghyun Choi
Machine Learning · Cross-listed from cs.AI
Coachable agents for interactive gameplay
Roberto Capobianco, Harm van Seijen, Nolan D. Bard et al. (+38 more)
Reinforcement learning has proven to be a valuable tool in the creation of advanced AI and robotic systems, contributing to everything from game playing to robotics to foundation models. Through trial-and-error, these AI systems typically learn one, near-optimal behavior to solve their tasks. However, there are many use cases in which one would like to assert some level of control, preferably in real time, over how the task is solved. We refer to these modifications of a core task as styles. We combine universal value function approximators (UVFAs) with carefully selected training scenarios, learning
algorithms, and data augmentation to create a framework for coaching
agents that exhibit styles in complex domains. We demonstrate the framework's application in the AAA
video games Horizon Forbidden West and Gran Turismo, and in an open-source
humanoid test domain. Despite the different nature of the domains – car racing, stylized game combat, and
humanoid walking – each
agent shows strong coherence to the style requests while still satisfying the main task in its domain. Importantly, the techniques outlined in this paper allow an end user to choose the final behavior at run time, giving them flexible control over the final executed performance.
Read the full abstract ▾
All authors Roberto Capobianco, Harm van Seijen, Nolan D. Bard, Neil Burch, Fatima Davelouis, Josh Davidson, Alisa Devlic, Yunshu Du, Ishan Durugkar, Siddhant Gangapurwala, Daniel Hernandez, G. Zacharias Holland, Sahil Jain, Kenta Kawamoto, Raksha Kumaraswamy, Patrick MacAlpine, Dustin R. Morrill, Declan Oller, Francesco Riccio, Akanksha Saran, Craig Sherstan, Kaushik Subramanian, Thomas J. Walsh, Samuel Barrett, Kizza N. Frisbee, Mady Govil, Johannes Günther, Varun R. Kompella, James A. MacGlashan, Maxwell Svetlik, Michael D. Thomure, Jaden B. Travnik, Kevin Waugh, Elahe Aghapour, Florian Fuchs, Andreanne Lemay, Shruti Mishra, Takuma Seno, Peter Stone, Michael Spranger, Peter R. Wurman
Machine Learning · Cross-listed from cs.CY
What's a Credit Worth? A Market Framework for Attribution-Aware Compensation in Generative Music
Luyang Zhang, Xirui Jiang, Junwei Deng et al. (+3 more)
Advances in generative AI are rapidly increasing the quality and commercial value of generated music, and this progress depends on large catalogs of creators' recordings. This raises a central question for platform design: how should creators be compensated when their work is used to train generative AI models that in turn produce commercial outputs? We develop a framework for fairly compensating creators in generative-music
markets, where each creator's payment depends on a data-attribution score estimating their contribution to model outputs. Compared to past compensation frameworks, our framework has two unique considerations: (1) attribution is traced to entire creator catalogs, not individual songs, and (2) the informativeness (signal-to-noise ratio) of the attribution score is an input to the payment mechanism. The framework yields a closed-form payment rule per creator and measures the welfare cost of inaccurate attribution for both creators and the platform. Whether the welfare-optimal contract is royalty-based or takes the form of fixed-fee licensing depends on how informative attribution is for that creator's catalog. We show that better attribution translates directly into welfare gains for both creators and the platform, yet under multi-platform competition a platform only captures gains from attribution improvements when its signal becomes the most precise in the
market. To ground our framework in empirical behavior, we train acoustic and symbolic music generation models and measure the informativeness of scalable attribution techniques against a leave-one-catalog-out ground truth. Our experiments reveal that noisy attribution signals push payment toward fixed-fee licensing and diminish welfare for both creators and the platform, providing an economic motivation for further research on improved attribution.
Read the full abstract ▾
All authors Luyang Zhang, Xirui Jiang, Junwei Deng, Beibei Li, Jiaqi W. Ma, Chris Donahue
Machine Learning
Loss Smoothing for Stable Adaptation Under Distribution Shift
Darshan Patil, Ekaterina Lobacheva, Razvan Pascanu et al. (+1 more)
In settings such as fine-tuning and reinforcement learning, neural networks are often adapted under distribution shift. Standard adaptation methods typically optimize the target objective directly, inducing an abrupt change from the source training objective. This abrupt transition can distort learned representations, including features that may still be useful for the new task. We investigate whether a more gradual transition can improve adaptation. We propose loss smoothing, a simple approach that interpolates between the source and target training objectives at the start of adaptation. This smooth transition helps to preserve useful features from the source distribution while still enabling the model to specialize to the target distribution. Across controlled supervised shifts, pretrained
vision adaptation, offline-to-online and online reinforcement learning, and
language model fine-tuning, we find that loss smoothing consistently improves performance, suggesting that smoother objective transitions are a broadly useful tool for model adaptation.
Read the full abstract ▾
All authors Darshan Patil, Ekaterina Lobacheva, Razvan Pascanu, Sarath Chandar
Machine Learning · Cross-listed from cs.CL
Arya Raeesi and Hanna Roed
Limited
Memory Language Models (LMLMs) externalize factual knowledge to a database to enable deletion-based unlearning without retraining. Existing evaluations measure post-deletion correctness in aggregate and cannot tell whether a deleted fact persists through residual parametric
memory, alternative
retrieval paths, or near-neighbor
retrieval artifacts. We propose a causal
auditing framework that holds the model fixed and varies the database state at inference time across three interventions: FULL, DEL-ON, and DEL-OFF. The framework decomposes post-deletion behavior into parametric leakage L(f),
retrieval-mediated correctness R(f), and a
retrieval artifact rate grounded in the inference-time
retrieval trace. We apply it to 12,228 alias-closure deletions across thirteen databases, including four
adversarial topologies (Base, Alias, Noise, Collision) we construct in three domains, and six
prompt formulations. Parametric leakage is near zero in every variant and every
prompt style: the model rarely returns the deleted answer in the absence of
retrieval. The residual that does survive lives in the
retrieval graph:
retrieval-mediated correctness and the
retrieval artifact rate match within rounding everywhere, so post-deletion correctness is, in our audit, predominantly reconstituted from near-neighbor
retrieval. This residual ranges from 0.7% on the released LMLM database to 13.6% on the most
adversarial variant, and
prompt formulation does not independently control how much of a deleted fact survives. These results suggest that, for this class of LMLM and deletion procedure, the unlearning boundary is drawn primarily by the database administrator rather than by the model.
Read the full abstract ▾
Machine Learning
Measuring Dead Directions: Decomposing and Classifying Singular Structure off Canonical Alignment
Tejas Pradeep Shirodkar
We give a descent-free,
alignment-free measurement of singular structure on trained networks. At a single frozen checkpoint the read recovers the order $k$ of each dead direction from the directional-Fisher rate, the master invariant from which the per-direction learning coefficient $1/(2k)$ follows exactly, in whatever basis the optimizer left. The same read classifies each direction, separating a genuine singularity, whose order the architecture fixes, from a flat gauge
symmetry; the directional-Fisher magnitude settles the cases the order cannot. A pluggable detector supplies the directions for
transformer, convolutional, and normalisation layers. The read recovers the architecture-predicted order across constructed
cells and trained networks, including a fine-tuned
vision transformer whose dead structure is the LayerNorm-kernel gauge and a from-scratch one whose compressed MLP forms a node-death at its activation order. Where the singular structure enumerates, the per-direction orders assemble, through the typed intersection of the loci, into the global coefficient $(\lambda, m)$ matching the closed form. The method removes the canonical-
alignment and descent preconditions of the underlying rate result, turning order-recovery into a deterministic, architecture-general reading. We then map its reach into the Watanabe triple: the order determines the universal singular fluctuation $\nu(k)$, though a trained network's realized $\nu$ falls below it as the live structure absorbs the dead direction's data fluctuation, and the multiplicity recovers from the dominant structure under a single-locus assumption.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.CO
P. Dobson, J.M. Sanz-Serna and K.C. Zygalakis
We present a simple, yet general approach to study the scaling properties as the dimensionality of Metropolised MCMC sampling
algorithms increases. The study relies on the
symmetries of the
Hamiltonian formalism and ultimately on the
symmetry of the Metropolis-Hastings formula. Our findings contain, as particular cases, many known results for the Random Walk Metropolis, MALA and other
algorithms. In addition, they provide, in an easy way, new optimal scaling results for a variety of proposal mechanisms, including implicit proposals and proposals generated with the help of differential equation integrators. The analysis applies to targets that are products of a given, not necessarily univariate distribution, and also to cases where the different terms in the product are scaled differently. We show how to construct gradient-based MALA-like proposals where the variance of the proposal as the dimension $d$ increases may be taken as $O(1/d^\mu)$, with $\mu>0$ arbitrarily small, to be compared with the values $\mu = 1$ for Random Walk Metropolis and $\mu=1/3$ for MALA.
Read the full abstract ▾
Machine Learning · Cross-listed from q-bio.PE
How Environment and Urbanization Shape Bird Diversity in Sri Lanka
Dilusha Chandrasiri, Maneesha Herath, Yasith Hewarathna et al. (+6 more)
This study presents a comprehensive analysis of bird diversity across Sri Lanka by integrating spatial, temporal, and environmental data. Bird observation records were combined with environmental variables, including weather conditions, air pollution, the Normalized Difference Vegetation Index (NDVI), land cover, elevation, and Artificial Light At Night (ALAN), and rigorously preprocessed to ensure data quality. Spatial analyses were conducted on multiple grid scales (2 km, 5 km, 10 km) to evaluate patterns in species richness while minimizing sampling bias through spatial thinning. Temporal trends were assessed using effort-corrected metrics including rarefied richness and occupancy rates to account for variations in observation effort over time. Environmental drivers of bird diversity were examined using multivariate statistical models, including Poisson Generalized Linear Models (GLMs) and correlation analyses, to identify key associations between ecological factors and species richness. Additionally, community structure, dominance patterns, and beta diversity were analyzed to understand variations in species composition across regions and time. The study found that land-cover type is a stronger predictor of bird diversity than individual continuous variables such as NDVI or temperature alone. Urbanization, measured by ALAN, exhibits nuanced scale-dependent effects, supporting high abundances of a few generalist species while reducing overall richness. The findings provide actionable insights into the patterns and drivers of avian diversity in Sri Lanka, offering a scalable and reproducible framework for biodiversity research and conservation planning.
Read the full abstract ▾
All authors Dilusha Chandrasiri, Maneesha Herath, Yasith Hewarathna, Muditha Herath, Gishan Bandara, Madara Mendis, Nathali Athukorala, Nisansa de Silva, Sandareka Wickramanayake
Machine Learning
Decision-focused Sparse Tangent Portfolio Optimization
Haeun Jeon, Seunghoon Choi, Hyunglip Bae et al. (+2 more)
Sparse tangent portfolio optimization aims to learn an interpretable, low-cardinality portfolio in the tangency direction of the mean-variance frontier. However, the associated cardinality-constrained formulation is NP-hard, and standard predict-then-optimize pipelines often misalign
forecasting accuracy with downstream portfolio quality. We propose an end-to-end decision-focused learning framework that reformulates Sharpe ratio maximization as a Disciplined Parametrized Programming (DPP)-compliant convex programming layer and replaces discrete selection with a smooth top-$k$ operator enforcing an exact cardinality $k$. This enables gradient flow through prediction, asset selection, and re-optimization, allowing the predictive model to directly optimize portfolio performance. Across four major equity
markets, our method achieves competitive and often superior out-of-sample Sharpe ratios compared with historical and prediction-focused baselines, with particularly strong gains in larger asset universes. Our \href{https://github.com/feuerwerksh/Diffble-card-SR}{code} is publicly available.
Read the full abstract ▾
All authors Haeun Jeon, Seunghoon Choi, Hyunglip Bae, Yongjae Lee, Woo Chang Kim
Machine Learning
Group-Equivariant Poincaré Convolutional Networks
Aiden Durrant, Rahul Baburajan and Georgios Leontidis
While recent methods like that of the Poincaré ResNet have demonstrated the ability to learning visual representations directly in
hyperbolic space, their optimisation remains a challenge, primarily due to the parameter redundancy of learning distinct orientation filters. In addition,
hyperbolic learning exhibits distinct computational overheads that limit their wide use, where efforts to improve their efficiency via optimisation have seen good success, there has been limited exploration into structural priors that enable stronger sample efficiency at training. To address this, we propose Equivariant Poincaré ResNets, combining
hyperbolic geometry with discrete
symmetry groups ($C_4$ and $D_4$). We identify critical roadblocks in applying Euclidean equivariance to
hyperbolic space and propose geometrically safe tensor reshaping, left-regular permutations for
hyperbolic group convolutions, and joint-orientation Poincaré Midpoint Batch normalisation. Empirical evaluations show that
embedding equivariance significantly improves the sample efficiency during training which in-turn accelerates convergence while respecting the boundary constraints of the Poincaré ball and retaining spatial group equivariance.
Read the full abstract ▾
Machine Learning
Flow-Map GRPO: Reinforcement Learning for Few-Step Flow-Map Generators via Anchored Stochastic Composition
Zhiqi Li, Wen Zhang and Bo Zhu
Few-step flow-map generators, such as consistency models and MeanFlow, accelerate sampling by learning long-range transport maps between noise and data. However, their deterministic transitions do not directly provide the
stochastic trajectories and tractable likelihood ratios required by reinforcement learning (RL) post-training. Existing SDE-based stochasticization techniques target velocity-based samplers and do not directly extend to long-range flow-map transitions. We propose Flow-Map GRPO, an online RL post-training framework for deterministic few-step flow-map generators. Its key component, Anchored
Stochastic Flow Map Composition (ASFMC), combines deterministic transport with anchor-based conditional resampling. We establish the conditions under which this construction preserves the marginal probability path and develop tractable local- and endpoint-anchor policies for two-time and single-time flow maps. These policies enable a unified GRPO training procedure. Experiments on FLUX-based MeanFlow and sCM generators demonstrate substantial improvements in OCR, PickScore, and GenEval at different numbers of inference steps, including joint OCR–PickScore gains with mixed rewards. Controlled ablations show that the
stochastic transition design is essential for translating training rewards into generation quality. Flow-Map GRPO enables effective RL
alignment of pretrained deterministic flow-map generators while retaining their original parameterization, without retraining them as native
stochastic models.
Read the full abstract ▾
Machine Learning
Active-GRPO: Adaptive Imitation and Self-Improving Reasoning for Molecular Optimization
Xuefeng Liu, Mingxuan Cao, Qinan Huang et al. (+3 more)
Scientific
reasoning is an increasingly important capability of large
language models, yet improving the robustness and efficiency of training such
reasoning remains a key open challenge. We study this problem in instruction-based molecular optimization, where answer-only supervised fine-tuning (SFT) collapses multi-step
reasoning and reinforcement learning with verifiable rewards (RLVR) suffers from sparse feedback. Reference-guided Policy Optimization mitigates both by anchoring policy updates to
dataset-provided references, but its effectiveness is tightly coupled to reference quality: weak or misaligned references impose a performance ceiling. To overcome this ceiling, we propose active
reasoning, a paradigm in which the policy actively decides, on a per-instance basis, when to imitate a reference and when to reinforce its own discoveries, while continuously upgrading what it imitates. We instantiate this paradigm as Active Group Relative Policy Optimization (Active-GRPO), realized through two coupled mechanisms: active imitate-reinforce and active referencing. The former performs imitation learning when the reference still outperforms the policy's own candidates, and shifts to self-improvement via reinforcement learning once the policy has generated molecules that surpass the reference. The latter continuously upgrades the reference itself by replacing it with the best policy-generated candidate discovered so far, progressively raising the imitation target and ensuring that reference guidance remains informative-rather than restrictive-throughout training. Across TOMG-Bench MOLOPT, Active-GRPO improves average SRxSim from 0.0959 for GRPO and 0.1665 for RePO to 0.1773 under matched three-seed evaluation, with statistically significant gains on LogP, MR, and QED.
Read the full abstract ▾
All authors Xuefeng Liu, Mingxuan Cao, Qinan Huang, Thomas Brettin, Rick Stevens, Le Cong
Machine Learning
From Structural Equation Modelling to Double Machine Learning: Robustness Analysis for Survey-Based Research
Ka Ching Chan, Qiana Liu, Sanjib Tiwari et al. (+1 more)
Structural equation modelling (SEM) is widely used in survey-based business and information systems research to assess latent constructs and theory-driven structural relationships. However, SEM path significance is obtained within a particular model specification and may not show whether findings remain stable under alternative estimation frameworks. This study develops and demonstrates a staged robustness analysis framework that connects SEM, ordinary least squares (OLS) regression, and Double Machine Learning (DML). SEM is first used to refine the measurement structure and estimate the robustness-baseline SEM model, in which the full theory-specified structural path system is retained for downstream robustness analysis before final structural path evaluation. OLS regression is then applied to SEM-derived construct scores as a transparent regression
benchmark. Finally, DML-style residualisation is used to examine whether each tested focal relationship remains stable after flexible machine-learning-based adjustment for observed controls. Learner-sensitivity checks compare Random Forest, Gradient Boosting, and Support Vector Machine learners, and selected reverse-direction diagnostics are used to examine directional sensitivity. The framework is demonstrated using a FinTech Digital Customer Intimacy survey model. The findings identify which relationships are stable across SEM, OLS, and DML-style checks, and which require more cautious interpretation. A reproducible Google Colab workbook and generated result files are publicly available, providing a reusable template that researchers and students can adapt to other survey-based latent-construct studies. The paper contributes a practical robustness workflow and interpretation guide for survey-based researchers seeking to complement SEM with conventional and machine-learning-based robustness checks.
Read the full abstract ▾
All authors Ka Ching Chan, Qiana Liu, Sanjib Tiwari, Ranga Chimhundu
Machine Learning
Prototype Language Models
Dan Ley, Giang Nguyen, Himabindu Lakkaraju et al. (+1 more)
Knowing which training examples drive outputs is fundamental to
auditing, correcting, and understanding
language models, yet for modern LLMs this remains expensive, approximate, and largely post-hoc. Standard
language models generate
tokens through a dense network pathway, causing training data's influence to be distributed across parameters rather than organized along explicit, traceable components. We introduce a prototype
language model architecture, Prototypes for Interpretable Sequence Modeling (PRISM), that forms each prediction via a sparse, non-negative mixture of learned prototypes, trained with clustering objectives that anchor each prototype to coherent neighborhoods of training examples. Across architectures from 130M to 1.6B parameters trained on up to 50B
tokens, prototype
language models either surpass or remain within 2.5 percentage points on average downstream accuracy of matched dense baselines. We show that sparse prototype structure localizes
curvature in the loss landscape, yielding a more tractable Hessian and enabling training data attribution that is ~500x faster than post hoc baselines when consuming equivalent
memory. Calibrating linear prototype controllers can improve downstream accuracy by roughly 3 points while tracing those corrections back to training neighborhoods, and targeted prototype suppression can remove model behaviors without finetuning or measurable loss in generation quality.
Read the full abstract ▾
All authors Dan Ley, Giang Nguyen, Himabindu Lakkaraju, Julius Adebayo
Machine Learning
PAPA: Online Personalized Active Preference Alignment
Anindya Sarkar, Nasik Muhammad Nafi, Isaac Lyngaas et al. (+2 more)
Diffusion models are highly effective at modeling complex data distributions, including images and text. However, in applications like personalized recommender systems, the objective often shifts to modeling specific regions of the distribution that maximize user preferences-initially unknown but gradually uncovered through interactive feedback. This can naturally be framed as a reinforcement learning problem, where the goal is to fine-tune a
diffusion model to maximize a reward function based on preferences. However, the main challenge lies in learning a parameterized reward model, which typically requires large-scale preference data-something that is often not feasible in practice. In this work, we introduce Personalized Active Preference
Alignment PAPA, a novel method that bypasses the requirement for a parametrized reward model by directly optimizing the
diffusion model using real-time user feedback. PAPA enables feedback-efficient preference
alignment, drawing inspiration from the variational inference framework. We demonstrate PAPA's effectiveness through extensive experiments and ablation studies across diverse class-conditioned and fine-grained
alignment tasks. Additionally, based on theoretical insights, we propose an enhanced fine-tuning strategy, referred to as EPAPA, that requires less computational budget and accelerates the fine-tuning process, further boosting PAPA's suitability for real-world deployment. Our code is made publicly available at https://github.com/NasikNafi/papa.
Read the full abstract ▾
All authors Anindya Sarkar, Nasik Muhammad Nafi, Isaac Lyngaas, Muralikrishnan Gopalakrishnan Meena, Yevgeniy Vorobeychik
Machine Learning
Ghost in the Kernel: In-Context Learning with Efficient Transformers via Domain Generalization
Peilin Liu and Ding-Xuan Zhou
Transformer-based large models have demonstrated remarkable generalization abilities across different tasks by leveraging a context-aware
attention module for in-context learning. With richer context,
transformers adapt more effectively to the current use case without any parameter updates. However, the quadratic computational and
memory complexity with respect to context length significantly slows data processing in softmax
transformers. Linear
transformers were proposed to address this issue by reducing the complexity to linear dependence on context length, but the design and understanding of the feature mapping in linear
attention, from a theoretical viewpoint, remain unclear. In this paper, we investigate the approximation and generalization abilities of linear
transformers under a two-staged sampling process from domain generalization. We show that linear
transformers perform in-context learning as learning a mapping from context distributions to response functions. A dimension-independent convergence rate is obtained for our generalization analysis, which also exhibits the tradeoff between the regularities of data distributions and latent features. Guided by our theoretical framework, we propose a new perspective on activation and loss design for linearizing pretrained softmax large
language models.
Read the full abstract ▾
Machine Learning
Interpretable vs Learned Encoders for High-Cardinality Fraud Detection
Xiao Han, Jingjing Liu, Moxuan Zheng et al. (+2 more)
A total of seven categorical encoding methods were tested on the IEEE-CIS fraud
benchmark dataset (590,540 records, 3.5% positives, 8 high-cardinality columns). The encoders were evaluated using a stratified 5-fold cross-validation (CV) with three repetitions. Five of the encoders had identical frozen LightGBM learners in the downstream phase, allowing for controlled comparisons of their performance to each other. CatBoost and TabNet were included as comparisons across paradigms using different learners. The entity
embeddings produced the highest AUC-ROC (0.9612), with a statistically significant tie with that of CatBoost (0.9602) and statistically superior to tier group encoding (0.9548), whereas target encoding was only 0.0023 worse than tier group encoding and the auditor-friendly tier boundaries were maintained. Off-the-shelf TabNet did not outperform tree-based pipelines and collapsed under data scarcity. On AUC-PR, CatBoost leads (0.822 vs. 0.793); no encoder dominated both metrics. Per-column analysis confirmed the
embedding advantage arises from joint multi-column representation.
Read the full abstract ▾
All authors Xiao Han, Jingjing Liu, Moxuan Zheng, Zhen Zhang, Chenyu Wu
Machine Learning
How Early Is Early Enough? Design-Dependent Observation-Window Sufficiency in Subscription Churn Prediction
Xiao Han, Yao Xiao, Chenyu Wu et al. (+1 more)
How many days of early behavior suffice for subscription churn prediction? In the public KKBox
dataset, the early indicator of churn is typically an indicator of someone's contract status; however, when looking in the heavily churned manual-renewal segment, having access to early behavior creates a substantial increase in prediction for that specific segment (PR +0.10 at 120 days). A nine-window sufficiency curve shows diminishing returns; descriptive inflection detectors place a knee between 30 and 120 days depending on the metric and model class. Stress-testing over three cohort/task designs shows that this curve is singular to the design being tested; for example, in our test with a moving target, the curve inverts and can shift depending on the feature set used. Therefore, any window-sufficiency claim should state its cohort construction, target definition, and feature families. A survival-filtered replication on educational-attrition data (OULAD) confirms the rising-curve shape in one behavior-dominated setting (floor ROC 0.56 vs. 0.88), though signal composition inverts.
Read the full abstract ▾
All authors Xiao Han, Yao Xiao, Chenyu Wu, Tongchen Zhang
Machine Learning
Predicting Lethal Outcome (Cause) And Understanding Key Biomarkers Linked With Acute Myocardial Infarction Using Deep Artificial Neural Network And Ensemble Of Machine Learning Methodologies
Sagnik Ghosh
Cardiovascular disease is still one of the main causes of death around the world. Acute myocardial infarction (MI), or heart attack, claims millions of lives each year. MI happens when blood flow to the coronary arteries is blocked or reduced, which causes permanent damage to the heart muscle. Without treatment, this can lead to cardiac arrest, where the heart stops pumping blood to the organs, resulting in organ failure and death. Even survivors often face serious problems like heart failure, pulmonary edema, and asystole. Research shows that 5 to 10 percent of survivors die within the first year after an MI, and nearly half need to be hospitalized again. Early thrombolytic treatment leads to better outcomes, so there is a clear need for faster and more accurate ways to diagnose MI. Right now, doctors usually review patient history and use their own experience to find the causes of MI. This process takes a lot of time and can be inconsistent. Detecting MI accurately and quickly can help patients take better care of themselves and prevent fatal events. In this study, we introduce an automated model to predict deadly outcomes of MI and help doctors understand important biomarkers linked to its complications. This approach aims to make diagnosis clearer, faster, and more affordable. The process includes preparing the data, filling in missing values, and handling imbalanced data using SVMSMOTE, ADASYN, and class-weighted methods. We use wrapper and embedded feature selection to find the most important variables, then scale the features for consistency. The model combines Logistic Regression, Random Forest, Light-GBM, and Bagging SVM, and is further improved with an artificial neural network to increase accuracy. We evaluate all models using precision, recall, and other key measures to find the best option for clinical use.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Neural Network-Based Estimation of Time-Dependent Parameters in AR(p) Processes
Agnieszka Kopeć, Paweł Przybyłowicz and Martyna Wiącek
We investigate a
forecasting framework based on a simple discrete-time dynamic model with coefficients varying in time. The parameters of the model are recovered within a deep learning framework, which makes it possible to retain a transparent parametric structure while simultaneously accounting for complex and nonstationary patterns in the observed phenomenon. Our analysis covers two specifications of the noise process. Besides the standard Gaussian setting, we also consider Laplace-distributed noise, which can offer a more adequate description in the presence of heavier tails and sharper local fluctuations. For both cases, we formulate the predictive scheme of the model and analyze the associated uncertainty quantification, including the construction of prediction intervals. The results illustrate that a relatively simple model, when combined with time-dependent parameter estimation, can serve as a mathematically tractable and practically flexible tool for
forecasting complex dynamics under different noise assumptions. The general model is stated for TVAR($p$), while the prediction-interval formulas and the numerical experiments are developed for the TVAR(1) case.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
StochasT: Learning with Stochastic Turn Depth for Visual Instruction Tuning
Yuan Qing, Chengzhi Mao and Boqing Gong
Large
Vision-
Language Models (LVLMs) rely extensively on Visual Instruction Tuning (VIT) to elicit their
multimodal reasoning capabilities. However, we find a discrepancy: VIT often packs multiple
language tasks about the same image for conversational, multi-turn training, whereas existing
benchmarks evaluate LVLMs in isolated, single-turn scenarios. The models can suffer from visual
attention decay and contextual overfitting during multi-turn training, making it hard for them to realize their full potential in the mismatched test phase. To close the gap, we propose learning with
Stochastic Turn Depth (StochasT), which stochastically groups
language tasks for the same image into clusters of varying sizes (turn depth) while preserving their organic order. Hence, while StochasT draws on Dropout and
stochastic depth for ResNets, it does not actually drop anything to maximize the utility of the training data. Furthermore, we introduce a challenging,
benchmark-agnostic evaluation mechanism based on the Balanced Latin Square to measure LVLMs' robustness under varying contextual dependencies. Extensive experiments demonstrate that StochasT effectively grants LVLMs strong, harmonized capabilities for both single-turn and multi-turn use cases.
Read the full abstract ▾
Machine Learning
MolSafeEval: A Benchmark for Uncovering Safety Risks in AI-Generated Molecules
Tong Xu, Xinzhe Cao, Zhihui Zhu et al. (+2 more)
Current molecular generation
benchmarks emphasize task complexity, molecule novelty, and property
alignment; they largely overlook a critical concern: the potential safety risks of AI-generated molecules. In practice, many generative models may produce molecules with toxic, reactive, or otherwise hazardous characteristics - posing hidden dangers that remain insufficiently addressed. To address this gap, we introduce MolSafeEval, a
benchmark dedicated to evaluating and analyzing the safety risks of molecular generation. Unlike prior approaches that rely on narrow toxicity predictors, MolSafeEval integrates heterogeneous safety knowledge - ranging from toxicological databases to hazard rules - into a structured molecular safety knowledge graph. This graph serves as a foundation for large
language model-based
reasoning, enabling systematic detection and explanation of unsafe features in generated compounds. We further categorize molecular generative models into four representative task types - unconditional generation, property optimization, target
protein-based design, and text-based generation - and provide standardized
datasets and safety evaluation protocols for each. By systematically revealing the safety vulnerabilities of current generative approaches, MolSafeEval offers a new lens for benchmarking molecular models and provides essential guidance toward safer, more trustworthy molecular design.
Read the full abstract ▾
All authors Tong Xu, Xinzhe Cao, Zhihui Zhu, Keyan Ding, Huajun Chen
Machine Learning
Gauging, Measuring, and Controlling Critic Complexity in Actor-Critic Reinforcement Learning
Konstantin Garbers
Actor-critic methods depend on learned critics, but critic quality is often evaluated only indirectly through return, temporal-difference error, or value loss. Critic complexity is introduced as an additional diagnostic and intervention dimension for actor-critic reinforcement learning. The analysis uses spectral effective-rank
entropy, a rank-like summary of the singular-value distributions of critic weight
matrices, to assess critic model complexity. Across TD3 and PPO experiments, critic complexity is tracked together with return and Monte Carlo value-estimation bias. The results show that critic complexity is measurable throughout training and is systematically associated with training behavior, while also making clear that the relationship is heterogeneous across
algorithms, tasks, and hyperparameters. A direct complexity-control intervention is then evaluated by adding a spectral-
entropy penalty to the critic loss. This intervention reliably changes the targeted spectral quantity, demonstrating that critic complexity can be controlled rather than only observed. Return effects are treated as task-dependent evidence rather than as a general performance claim, because overall complexity-control results vary.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Information-Regularized Attention for Visual-Centric Reasoning
Guohao Sun, Xiaofang Wang, Yash Patel et al. (+3 more)
Vision-
language models (VLMs) have become a paradigm for
multimodal learning, yet remain unstable due to object
hallucination, weak visual grounding, and catastrophic forgetting after full-parameter instruction tuning. We claim these failures result from a lack of explicit control over visual representation learning during the standard next-
token prediction objective. As a result, visual
embeddings thus become passively optimized and prone to injecting redundant or spurious signals. To counter this, we introduce Information-Regularized
Attention (IRA), a
stochastic attention mechanism that explicitly regulates the amount of visual information injected into the hidden states of intermediate
transformer layers. This local reparameterization translates uncertainty about visual representations into local noise that is independent across data points. Beyond evaluating model performance, we also quantify
embedding properties, where IRA produces smoother
curvature trajectories and suppresses
attention-sink across all layers, indicating a more stable transformation of the visual signal. Our results suggest that
stochastic attention is not merely a regularizer but a key contributor to representation learning in a generative architecture, offering a new direction for building more reliable VLMs.
Read the full abstract ▾
All authors Guohao Sun, Xiaofang Wang, Yash Patel, Mengchen Liu, Zhiqiang Tao, Praveen Krishnan
Machine Learning
Timesynth: A Temporal Fidelity Framework for Health Signal Digital Twins
Md Rakibul Haque, Shireen Elhabian and Warren Woodrich Pettine
Forecasting models for health-signal digital twins must preserve the oscillatory, frequency, phase, and state-transition dynamics of physiological signals, yet the pointwise metrics used to
benchmark them cannot detect when these fundamental properties are lost. We show that this blind spot misranks models: across 11 architectures, models with comparable pointwise error diverge by up to 53{\deg} in phase accuracy, equivalent to roughly 123 ms for a 1.2 Hz cardiac rhythm and invisible to standard metrics. To enable development of models that escape such failures, we introduce TimeSynth, a controlled benchmarking framework with two reusable components: a physiologically grounded generator producing signals with analytically known ground-truth dynamics from parametric models fitted to real electroencephalography, electrocardiography and photoplethysmogram signals, along with diagnostics quantifying amplitude, frequency, phase, and state-transition fidelity. Linear and full-sequence
attention models systematically lose frequency and phase information despite acceptable amplitude error, whereas architectures with localized temporal structure better preserve dynamical fidelity and adapt to observable state transitions; none, however, reliably preserves
stochastic switching. Because the dominant determinant of fidelity is architectural, model choice becomes a principled, use-case-driven decision rather than a search for a single winner. TimeSynth thus supplies the controlled preclinical stress test missing before models are coupled to patient data, with a reusable generator and diagnostics for fidelity-aware development.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
A Mechanistic View of Authority Hierarchy in LLM Sycophancy
Emil Joswin, Srujananjali Medicherla and Priyanka Mary Mammen
Authority bias poses a critical safety concern in
language models: models systematically prioritize social cues from authority figures over factual consistency, swaying their answers based on source credibility rather than evidence. We mechanistically investigate this phenomenon using a controlled medical QA setting, where hints suggesting incorrect answers are attributed to personas of varying expertise. Across Llama-3.1-8B, Qwen3-8B, and Gemma-2-9B, we find that models respond in a graded manner proportional to perceived authority, a hierarchy that is never explicitly prompted but emerges from training. Logit lens analysis and linear/non-linear probing localize this effect to a critical late layer where correct answer representations are actively erased, an erasure that scales with authority level, resists mean vector intervention, and is only partially reversible through chain-of-thought
reasoning. Our findings suggest that authority-induced sycophancy is not a surface-level output bias but mechanistic knowledge erasure, a precise, layer-localized overwriting of correct internal representations by high-status authority signals.
Read the full abstract ▾