Machine Learning
Learning Multi-Scale Hypergraph for High-Order Brain Connectivity Analysis
Jaeyoon Sim, Soojin Hwang, Seunghun Baek et al. (+2 more)
Understanding complex interactions between
brain regions is critical for early neurodegenerative disease classification such as Alzheimer's Disease (AD) and Parkinson's Disease (PD). While graph-based models are widely used to analyze
brain networks, most existing approaches primarily focus on pairwise interactions between directly connected nodes, limiting their ability to capture higher-order dependencies across multiple regions. Although hypergraph-based methods have been proposed to model higher-order relations, many rely on predefined hyperedges or restrict learning to hyperedge weights, reducing flexibility and limiting their capacity to capture multi-resolution structural patterns. In this regard, we introduce an adaptive multi-scale hyperedge learning framework, i.e., MuHL, which constructs hierarchical node features and dynamically learns high-order interactions through continuous hyperedge construction over multi-resolution graph signals. Extensive experiments on multiple
brain network
benchmarks demonstrate that MuHL consistently improves disease classification performance across different stages, and further identifies key regions of interest (ROIs) and their group-wise interactions from the learned hyperedges that are associated with disease progression, highlighting its potential as a powerful tool for
brain network analysis in neurodegenerative disorders.
Read the full abstract ▾
All authors Jaeyoon Sim, Soojin Hwang, Seunghun Baek, Guorong Wu, Won Hwa Kim
Machine Learning · Cross-listed from cs.CL
From Script to Semantics: Prompting Strategies for African NLI
Anuj Tiwari, Terry Oko-odion and Hannah Nwokocha
Large
language models (LLMs) are increasingly evaluated in
multilingual settings, yet their inference behavior in low-resource African
languages remains underexplored especially under pure prompting without fine-tuning. We present a systematic study of prompting strategies for Natural
Language Inference (NLI) in Swahili, Yoruba, and Hausa using the AfriXNLI
benchmark. We evaluate five prompting strategies Baseline (zero-shot), Script-Aware,
Language Specific, Contrastive, and Native-Label Self-Translation (NL-STP) across two mid-sized open weight models (Llama3.2-3B and Gemma3-4B). To isolate the effect of
prompt design, the effect of few-shot examples and Chain-of-Thought
reasoning is eliminated in our study. We find a significant difference in performance of class wise across strategies with highly neutral class collapse and high prediction skew in some configurations. Contrastive prompting proves to be the most reliable and steadily improving strategy over
language and model and has better balance of class behavior and balance of overall accuracy gains. Notably, well-constructed
prompts are sufficient to beat more powerful baselines that are provided with few-shot
prompts and Chain-of-Thought
prompts. We have found that
prompt formulation is essential to
multilingual NLI with low-resource
languages and that
language aware decision structuring can be used to meaningfully enhance robustness in resource challenged settings.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Combining Statistical Features and Deep Encodings for Rehearsal-Based Class-Incremental Time Series Classification
Pablo García-Santaclara, Bruno Fernández-Castro and Rebeca Pilar Díaz-Redondo
Many systems used in real-world environments require adding new categories and incorporating new information without forgetting what was previously learnt by the classification model. This is known as class-incremental continual learning, and in the case of multivariate time-series, is further complicated by the temporal structure of the data. In this paper, we present a novel approach for performing class incremental continual learning for the classification of multivariate time series data based upon the construction of a dual-stream feature extraction pipeline (using both deep temporal
embedding features generated via a pre-trained frozen foundation model and application of statistical features). Evaluated on five
benchmark datasets, the proposed system achieves competitive average accuracy across all
datasets while maintaining low forgetting rates across all experimental configurations.
Read the full abstract ▾
Machine Learning
Message Tuning Outshines Graph Prompt Tuning: A Prismatic Space Perspective
Yancheng Chen, Dun Ma, Shuai Zhang et al. (+6 more)
Graph Foundation Models (GFMs), built upon the Pre-training and Adaptation paradigm, have emerged as a research hotspot in graph learning. For GNN-based GFMs, graph
prompt tuning has become the prevailing adaptation method for downstream tasks. Although recent methods explain why graph
prompt tuning works, how to rigorously measure its adaptation capacity remains an open problem. Addressing this problem is critical for understanding the capability limits of graph
prompt tuning and for developing more powerful adaptation methods. In this paper, we propose Prismatic Space Theory (PS-Theory), a novel mathematical framework to quantify the capacity of adaptation methods, while focusing on establishing the upper bound for the adaptation capacity of graph
prompt tuning. Building upon the proposed PS-Theory, we further introduce Message Tuning for GFMs (MTG), a lightweight approach that injects a small set of learnable message prototypes into each layer of the GNN backbone to adaptively guide message fusion without updating pre-trained weights. Through our PS-Theory, we prove that the adaptation capacity of MTG can exceed the theoretical upper bound of graph
prompt tuning. Extensive experiments demonstrate that MTG consistently outperforms graph
prompt baselines across diverse
benchmark datasets, providing strong empirical support for our theoretical findings.
Read the full abstract ▾
All authors Yancheng Chen, Dun Ma, Shuai Zhang, Yang Liu, Xixun Lin, Xiangyu Zhao, Wenguo Yang, Wei Chen, Chuan Zhou
Machine Learning
A Geometric Lens on Physics-Aligned Data Compression
Aleix Segui and Wesley Armour
In AI for Science, physics-informed losses are increasingly used to train learned compressors for scientific data, but their rate-distortion implications remain poorly understood. At fixed bitrate, these objectives often improve preservation of a target physical observable while degrading standard reconstruction fidelity. We develop a local
geometric theory showing that this tradeoff is governed by the interaction of latent-space sensitivities induced by the
entropy model, the physical observable, and the distortion metric. At each operating point, these induce preferred directions along which compression noise should be suppressed, yielding an anisotropic error-allocation mechanism. When these directions are misaligned, improving the observable at fixed rate necessarily worsens standard distortion, establishing a fundamental limit on simultaneous preservation. We formalise this through a local tangent-space rate-distortion law and introduce a practical
alignment diagnostic based on dominant eigenspace overlap. Experiments across scientific domains test the theory and validate that the
alignment diagnostic correlates with observed data- and physics-space trade-offs.
Read the full abstract ▾
Machine Learning
Are Common Substructures Transferable? Riemannian Graph Foundation Model with Neural Vector Bundles
Li Sun, Zhenhao Huang, Yiding Wang et al. (+3 more)
Foundation models have sparked a revolution via a pretraining-adaptation paradigm, with recent efforts extending this success to graphs. Unlike other modalities, graphs contain rich structural patterns, yet their structural transferability remains poorly understood. Prior studies consider common substructures in the discrete realm, and we are motivated by a fundamental question: Are common substructures transferable? The underlying theory is largely underexplored. In this work, we shift toward learning transferable structures through the lens of functional behavior. Theoretically, we connect transferable substructures to intrinsic geometry of the representation space. However, characterizing such intrinsic geometry has rarely been touched. Grounded in Riemannian geometry, we develop a graph intrinsic geometry learning framework called Neural Vector Bundle, which enables parsing intrinsic geometry with local coordinates. Building on this, we design GAUGE, a pretrainable neural architecture that constructs the vector bundle, flattening geometrically compatible local coordinates, and a new Dirichlet loss, which also measures the transfer effort. We empirically validate its superior expressiveness in challenging tasks including zero-shot link prediction and graph isomorphism.
Read the full abstract ▾
All authors Li Sun, Zhenhao Huang, Yiding Wang, Qin Chen, Pietro Lio, Philip S. Yu
Machine Learning
Let There Be Light: Reflection, Refraction and Scattering for Neural Operators
Keke Wu, Yixuan Zhang and Jingrun Chen
Neural operators learn mappings between infinite-dimensional function spaces and provide a data-driven surrogate modeling paradigm for parametric partial differential equations (PDEs). Existing architectures typically obtain expressivity by parameterizing integral kernels in prescribed transform domains or by applying
attention-like interactions over discretized spatial points. While these approaches have achieved substantial progress, they often face a persistent trade-off among physical interpretability, nonlocal spatial communication, mesh scalability, and computational cost. We propose a Light-inspired neural operator(LiNO), an operator-learning architecture whose latent
evolution is decomposed into three mechanisms motivated by elementary light transport: reflection, refraction, and scattering. Reflection and refraction act as adaptive pointwise transformations in latent feature space, enabling local feature reorientation and anisotropic modulation, whereas scattering performs input-dependent nonlocal propagation over the physical domain. We first formulate scattering as a normalized pairwise kernel with relative positional bias, and then develop an efficient scattering variant that replaces explicit pairwise interactions with positive-feature global propagation and a local
diffusion branch, reducing the dominant spatial complexity from quadratic to linear. This yields a structured neural operator that separates local feature modulation from global spatial communication while retaining a modular and interpretable latent
evolution.
Read the full abstract ▾
Machine Learning
EqGINO: Equivariant Geometry-Informed Fourier Neural Operators for 3D PDEs
Sungwon Kim, Juho Song, Seungmin Shin et al. (+3 more)
Deep learning surrogates for 3D Partial Differential Equations (PDEs) often fail to generalize across
geometric transformations because they depend heavily on specific coordinate systems. While equivariant networks offer a solution, they typically rely on local operations in the spatial domain, making the global receptive field, which is essential for PDE dynamics, computationally expensive. Conversely,
Fourier Neural Operators (FNOs) efficiently capture global interactions, yet establishing 3D equivariance within them remains impractical due to the prohibitive cost of spectral group convolutions. To bridge this gap, we introduce EqGINO, a geometrically robust framework that enforces isotropy in the spectral domain. By design, EqGINO guarantees exact equivariance to the discrete
symmetries inherent to the discretized computational domain. Beyond this discrete guarantee, our structural prior enables effective generalization to arbitrary continuous orientations even with a limited number of SE(3)-transformed training samples. Consequently, our method robustly models coordinate-invariant physical laws on complex irregular 3D geometries. Our code is available at https://github.com/sung-won-kim/EqGINO
Read the full abstract ▾
All authors Sungwon Kim, Juho Song, Seungmin Shin, Guimok Cho, Sangkook Kim, Chanyoung Park
Machine Learning · Cross-listed from cs.NE
Rachmad Vidya Wicaksana Putra, Achyuta Muthuvelan, Alberto Marchisio et al. (+1 more)
Spiking
Vision Transformer (SViT) models are promising low-power ViT models for solving
vision-based tasks with state-of-the-art performance. However, their large sizes limit their deployments for resource-constrained embedded platforms, underscoring the needs of model compression. One of prominent compression techniques is
pruning, and the state-of-the-art works employ unstructured
pruning techniques to compress SViT models. Such techniques require specialized hardware architectures tailored for the sparsity patterns to maximize their efficiency benefits, making this approach not scalable. To address this, we propose PSViT, a novel methodology to perform structured
pruning on SViT models, hence making it possible to efficiently accelerate their inference using the existing and widely-used computing architectures. To do this, PSViT employs several key steps: uniform channel-wise filter
pruning to structurally eliminate the non-significant weights, sensitivity analysis to evaluate the impact of channel-wise
pruning of individual layer on accuracy and network size, as well as fine-grained channel-wise
pruning based on the sensitivity analysis and the given network architecture. Experimental results show that PSViT effectively obtains 22.4%
memory saving through single-shot
pruning, while maintaining high accuracy within 3% (70.3% without fine-tuning and 72.8% with fine-tuning) from the original non-pruned SViT model (73.3%) on the ImageNet-1K. These results also show that the PSViT methodology advances the effort in enabling efficient SViT deployments on resource-constrained applications.
Read the full abstract ▾
All authors Rachmad Vidya Wicaksana Putra, Achyuta Muthuvelan, Alberto Marchisio, Muhammad Shafique
Machine Learning · Cross-listed from cs.AI
Do Real-World Datasets Contain Natural Experiments? An Empirical Study Using Causal Feature Selection
Gautam Gare, John Galeotti, Michael Mozer et al. (+2 more)
In nature, events that affect some individuals or groups but not others constitute an implicit intervention and are known as natural experiments. For example, the COVID-19 pandemic was an intervention by the coronavirus on the sub-population infected with COVID. We ask, do natural experiments occur in existing real-world
datasets? If yes, how should we treat them? To detect natural experiments in data, we use causal discovery to recover the underlying causal graph and perform feature selection based on causal links. If downstream performance improves by treating the data as interventional rather than observational, we argue that this suggests the
dataset contains natural experiments. We first validate this hypothesis by simulating
datasets with and without natural experiments using synthetic graphs. We then perform a systematic empirical evaluation on a large suite of real-world
datasets. Our results indicate that real-world
datasets do contain natural experiments and we can take advantage of those natural experiments to improve model performance using causal inference. Our work represents the initial foray into this area, offering a preliminary exploration within a limited scope.
Read the full abstract ▾
All authors Gautam Gare, John Galeotti, Michael Mozer, Deva Ramanan, Nan Rosemary Ke
Machine Learning · Cross-listed from stat.ML
Hierarchies of Calibration: Classification meets Regression
Johannes Resin, Lu Yang and Tilmann Gneiting
Concepts of calibration formalize the compatibility between probabilistic predictions and the respective outcomes. In a nutshell, the outcomes ought to be indistinguishable from random draws from the predictive distributions. In this paper, we review, extend, and bridge notions of calibration that have been proposed for classification and regression tasks. Particular emphasis is given to hierarchical relations between the various notions, as they apply to general real-valued data, continuous outcomes, count data, nominal classes, and binary outcomes. To highlight a number of contributions, we introduce the notion of modal calibration for nominal outcomes, we distinguish full, partial, and average calibration in this setting, and we show that double probability integral transform (PIT) calibration is logically independent of previously proposed concepts of calibration for discrete outcomes. Furthermore, we generalize extant results on concepts of calibration that are expressed in terms of properties or functionals of the predictive distributions, such as means,
quantiles, or event probabilities. Throughout the paper, we illustrate the concepts and their hierarchical relations in worked examples, and we provide algorithmic tools that support the construction of instructive examples and counterexamples.
Read the full abstract ▾
Machine Learning
When RLHF Fails: A Mechanistic Taxonomy of Reward Hacking, Collapse, and Evaluator Gaming
Zelalem Abahana, David Evans, Satish Mahadevan Srinivasan et al. (+1 more)
RLHF evaluation should track how failures emerge, where they localize, and which warning signals appear before external quality degrades. We study this problem with a compact RLHF pipeline built for this paper, including PPO, DPO, uncertainty-penalized PPO (UP-PPO), reward-model uncertainty, approximate policy drift, diversity and repetition diagnostics, and two external LLM judges. Rather than treating reward hacking as a single terminal event, we classify matched checkpoint and
prompt-level transitions by the directions of learned reward R_phi, judge scores R_dag and R2_dag, and their average R_dag. The main empirical findings are that aggressive PPO produces the clearest localized reward-hacking signal, UP-PPO reduces but does not eliminate that signal, row-level diagnostics reveal failures hidden by checkpoint averages, and pre-transition features partially anticipate future localized reward hacking. The central conclusion is methodological: RLHF failures are training dynamics that can be classified, localized, and partially anticipated, not only final-model pathologies. The repository is available at github.com/zabahana/rlhf-failure-modes-diagnostics. The pipeline is also deployed as a live interactive web demo for model comparison and diagnostic views at rlhf-failures.zelalem.ai.
Read the full abstract ▾
All authors Zelalem Abahana, David Evans, Satish Mahadevan Srinivasan, Matjaz Gams
Machine Learning · Cross-listed from cs.AI
Solipsistic Superintelligence is Unlikely to be Cooperative
Rakshit S Trivedi, Natasha Jaques, Logan Cross et al. (+2 more)
AI's central challenge is shifting from capability to coexistence. The dominant paradigm in AI research focuses on developing powerful
agents that treat the world as an exogenous and stationary source of feedback. We contend that superintelligence, an extremely capable task solver, born out of such a solipsistic approach to AI design, is unlikely to be cooperative. Deploying AI systems induces endogenous non-stationarity, resulting in a train-test-deploy gap where historical distributions diverge from the deployment context. We refer to this as the self-undermining property of unilateral optimization. Closing this gap requires AI that participates in cooperation: the
equilibrium-selection process through which multiple actors navigate their interdependence. We call for a non-solipsistic research paradigm that treats this interdependence as a core design principle rather than approaching cooperation as a task to solve. This entails building dynamic evaluation testbeds involving adaptive counterparties, treating institutions as design primitives, and preserving human agency as a structural feature of the systems we build.
Read the full abstract ▾
All authors Rakshit S Trivedi, Natasha Jaques, Logan Cross, Alexander Sasha Vezhnevets, Joel Z Leibo
Machine Learning
Right Makes Might: Aligning Verified Hidden States Empowers RL Reasoning
Ziyue Wang, Aomufei Yuan, Yongfu Zhu et al. (+10 more)
Reinforcement Learning from Verifiable Rewards (RLVR) has become the dominant approach for improving mathematical
reasoning in large
language models, yet current methods reduce each correct rollout to a single reward bit, ignoring the
geometric structure shared among their hidden states. Investigating this structure, we find that at the anchor
token (the position immediately before the answer marker), correct rollouts converge naturally because they must produce the same answer (cosine similarity ~0.84), yet each retains residual variance from its unique
reasoning path. Encouraging full
alignment at this point pushes the model to extract a unified "correct decision" representation, reducing sensitivity to which
reasoning path was taken. Based on this observation, we propose Hidden-Align, an auxiliary loss function that aligns the last-layer hidden states of correct rollouts at the anchor
token during RL training, with zero overhead in both training and inference. On eight mathematical
reasoning benchmarks, Hidden-Align improves average pass@1 over the DAPO baseline by 3.8, 6.2, and 5.4 percentage points on Qwen3-1.7B, 4B, and 14B respectively, with consistent pass@k gains across all three scales, supported by ablations on loss type, anchor position, layer depth, and loss weight.
Read the full abstract ▾
All authors Ziyue Wang, Aomufei Yuan, Yongfu Zhu, Shuai Dong, Wenpu Liu, Yiran Yao, Weichu Xie, Yuqi Xu, Caoyuan Ma, Wenqi Shao, Xiaoying Zhang, Nan Duan, Jiaqi Wang
Machine Learning
GFFMERGE: Efficient Merging of Graph Neural Force Fields and Beyond
Parth Verma, Parv P. Singh, Vipul Garg et al. (+3 more)
Graph Neural Networks (GNNs) have revolutionized Neural Force Fields for atomistic simulations, achieving near-
quantum accuracy at reduced cost, yet adapting these models to new chemical systems requires expensive retraining of foundation models. Inspired by model merging in
vision and
language processing, we introduce GFFMERGE, the first principled framework for closed-form model merging in GNNs. We exploit the linear structure of message-passing layers and formulate merging as a convex
embedding-
alignment problem with an analytical solution. Through the first systematic benchmarking of model merging for GNNs, we show that existing methods designed for
vision and
language catastrophically fail on force field regression, while GFFMERGE recovers performance approaching gold standard joint training. Across molecular (MD17, MD22), solid-state (LiPS20), and large-scale graph
benchmarks, GFFMERGE and GNNMERGE (its generic GNN counterpart) achieve 5-27$\times$ speedups while enabling modular composition of specialized models. Remarkably, our closed-form solution alone outperforms all baseline methods before fine-tuning and provides superior initialization for faster, data-efficient convergence.
Read the full abstract ▾
All authors Parth Verma, Parv P. Singh, Vipul Garg, Ishita Thakre, N. M. Anoop Krishnan, Sayan Ranu
Machine Learning
Learning Temporal Causal Structure via Smooth Differentiable Optimization
Tong Zhao, Ce Guo, Wayne Luk et al. (+2 more)
Causal discovery with instantaneous effects in multivariate time series is challenging, as the instantaneous structure must be acyclic. Prior methods enforce this by either separating instantaneous and lagged estimation into multi-stage pipelines or imposing algebraic acyclicity constraints via complex augmented Lagrangian optimization, both of which incur high computational cost. In this work, we propose a different approach: we learn a differentiable permutation of variables using the Gumbel–Sinkhorn operator and triangularize the instantaneous coefficient
matrix of a Structural Vector Autoregressive (SVAR) model in the learned order. This converts acyclicity from a hard constraint into a parameterization and keeps it valid throughout optimization. In doing so, our method enables unified, continuous optimization with gradient-based learning, leading to improved efficiency in time–series causal discovery. Across three real-world
benchmarks, our method achieves the best overall performance compared with 12 baselines in both discovery accuracy and efficiency. On the large-scale
benchmark, it further demonstrates strong scalability, achieving more than a 6x speedup over competing methods.
Read the full abstract ▾
All authors Tong Zhao, Ce Guo, Wayne Luk, Emil Lupu, Ray Dipojjwal
Machine Learning · Cross-listed from cs.CL
Sample-Size Scaling of the African Languages NLI Evaluation
Anuj Tiwari, Oluwapelumi Ogunremu, Terry Oko-odion et al. (+2 more)
African
languages have very little labelled data, and it is unclear if augmenting the quantity of annotation data reliably enhances downstream performance. The study is a systematic sample-size scaling study of natural
language inference (NLI) on 16 African
languages based on the AfriXNLI
benchmark. Under controlled conditions, two
multilingual transformer models with roughly 0.6B parameters XLM-R Large fine-tuned on XNLI and AfroXLM-R Large are tested on sample sizes of between 50 and 500 labeled examples and average their results across random subsampling runs. As opposed to the usual belief of monotonic increase with increased data, we find a strongly
language sensitive and often non-monotonic scaling behavior. Some
languages show early saturation or decrease in performance with sample size as well as high variance in low resource regimes. These results indicate that the volume of data is not enough to guarantee stable profits to African NLI, creating the necessity of
language sensitive
datasets creation and stronger multi-lingual modelling strategies.
Read the full abstract ▾
All authors Anuj Tiwari, Oluwapelumi Ogunremu, Terry Oko-odion, Jesujuwon Egbewale, Hannah Nwokocha
Machine Learning · Cross-listed from stat.ML
An Asymptotic Theory of Chain-of-Thought in In-Context Learning
Kaito Takanami and Cengiz Pehlevan
Chain-of-thought (CoT)
reasoning has become a widely used mechanism for eliciting multi-step
reasoning in large
language models by generating intermediate
reasoning steps at inference time. Yet the scaling behavior of generalization with CoT depth remains poorly understood. To address this question, we study a theoretically solvable model of CoT for in-context weight prediction in linear regression, where test-time
reasoning is represented as an iterative refinement of the weight-parameter estimate. Using tools from random
matrix theory under high-dimensional asymptotics, we derive an exact formula for the generalization error as a function of
reasoning depth, pretraining data amount, and context length. Our analysis reveals a sharp phase transition separating exponential and
polynomial improvement, saturation, and overthinking, and characterizes how the optimal
reasoning depth scales. We further show that deeper
reasoning is most effective with sufficiently rich pretraining and in-context information, whereas limited pretraining or context makes longer
reasoning prone to error amplification or saturation. We also validate these predictions through experiments on fully learned linear
attention and softmax
attention models. Our results provide a unified theoretical account of how test-time CoT depth affects generalization.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
Effect of Demographic Bias on Skin Lesion Classification
Ralf Raumanns, Gerard Schouten, Veronika Cheplygina et al. (+1 more)
In this study, we evaluate the performance of skin lesion classification using ResNet-based convolutional models, focusing on the impact of demographic bias in training data, particularly variations in patient sex and age. We use linear programming to generate
datasets with controlled demographic characteristics, allowing systematic investigation of bias effects. Three learning strategies are evaluated: a single-task model, a reinforcing multi-task model, and an
adversarial learning scheme. Our sex-based analysis indicates that sex-specific training
datasets optimise model performance. Notably, including male patients in the training data improved performance for the male subgroup, even in female-majority cases. Reinforcing and
adversarial learning schemes narrowed or eliminated bias gaps in balanced and female-majority
datasets. However, these strategies proved less effective in male-majority settings, where models continued to perform better for males than females. The two learning schemes showed marginal bias reduction compared to the baseline model in predominantly male patient populations. Age-based analysis demonstrates comparable baseline performance across the three model approaches, with performance declining across age categories. Younger groups consistently achieve the highest performance, regardless of training data distribution. Although balanced training yields optimal results for the youngest age category, performance decreases in older categories. We find that sex biases arise mainly from data imbalances, while age biases consistently favour younger groups regardless of distribution. These distinct mechanisms require targeted mitigation strategies. Additionally, cross-
dataset validation on two external
datasets revealed that domain shifts notably affect performance and patterns of demographic bias.
Read the full abstract ▾
All authors Ralf Raumanns, Gerard Schouten, Veronika Cheplygina, Josien P.W. Pluim
Machine Learning
Bayesian Tensor Decomposition with Diffusion Model Prior
Zerui Tao and Qibin Zhao
Low-rank tensor decomposition (TD) is usually effective on clean, fully observed data, but it often degrades under severe missingness or noise. Low-rankness is itself a useful but limited structural prior, and additional handcrafted priors (e.g., sparsity or smoothness) still fall short of capturing the rich statistics of real-world data. To compensate for this weak inductive bias under heavy corruption, one would like to inject a learned, data-driven prior; however, the state-of-the-art
diffusion models are not readily compatible with current TD and tractable posterior inference. To address these challenges, we introduce DiffBCP, a hybrid-prior
Bayesian CP decomposition framework that couples a cumulative shrinkage process prior over the CP factors for automatic rank selection with an off-the-shelf pre-trained
diffusion model as an implicit data prior on the reconstructed tensor. To make posterior inference tractable despite the coupling among the likelihood, low-rank constraint, and
diffusion prior, we develop a split Gibbs sampler: CP factors admit conjugate updates, while the
diffusion block is sampled via low-rank-guided denoising. A noise-adaptive coupling schedule further reduces sensitivity to hand-tuned annealing. Experiments on image inpainting and denoising, including high-resolution out-of-distribution images, show consistent gains over
Bayesian, nonlinear, and plug-and-play TD baselines.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CE
Differentiable FEM versus PINN for pavement layer inverse analysis: enforcing physics as hard or soft constraints
Yongjin Choi, Hyeonbin Moon and Seunghwa Ryu
Automatic-differentiation-based inverse analysis methods, including physics-informed neural networks (PINNs) and differentiable programming, have recently shown great promise due to their ability to compute accurate gradients and convergence efficiency. However, their applicability to falling weight deflectometer (FWD) backcalculation remains unexplored. This study critically evaluates PINN-based inverse analysis for a multilayer pavement system and investigates differentiable finite element method (DiffFEM) as an alternative based on a synthetic
benchmark. The standard PINN does not recover layer
moduli because of the sharp domain discontinuities inherent to layered pavement systems. Although we use an extended PINN with domain decomposition (XPINN), which shows better performance on discontinuous domains, its performance remains highly sensitive to loss weighting and network architecture, and degrades under measurement noise. By contrast, DiffFEM consistently achieves more accurate, stable, and computationally efficient inversion results. These results show and quantify the expected advantage of DiffFEM, which enforces the governing physics as a hard constraint in the solver, over PINN-based approaches, where the governing physics is imposed as a soft constraint through the loss function. More broadly, the findings suggest that the choice between PINN- and DiffFEM-based inverse analysis needs careful consideration, with DiffFEM offering practical advantages when an efficient and robust differentiable forward solver is available.
Read the full abstract ▾
Machine Learning
DECA: Decentralizing Block-Wise Adam for Efficient LLM Full-Parameter Fine-Tuning on Non-IID Data
Yunsheng Yuan, Shaowei Li, Kai Wang et al. (+5 more)
Fine-tuning large
language models (LLMs) in
privacy-sensitive and resource-constrained environments remains challenging. Since training data are often distributed across multiple clients, decentralized fine-tuning offers a natural paradigm for collaborative adaptation without a central server. However, enabling full-parameter fine-tuning (FPFT) in this decentralized setting is difficult: FPFT provides strong adaptation capacity but incurs prohibitive resource consumption for billion-scale models. Existing decentralized LLM fine-tuning methods therefore mainly rely on parameter-efficient updates, which improve efficiency but may restrict downstream performance. Moreover, client data are typically non-IID, making decentralized optimization more vulnerable to client drift and unstable convergence. To address these challenges, we propose DECA, a resource-efficient decentralized FPFT framework for LLMs on non-IID data. DECA partitions model parameters into disjoint blocks and performs sequential block-wise Adam optimization, reducing resource consumption while preserving decentralized full-parameter adaptation. To stabilize training, DECA further introduces first- and second-order block-wise moment estimates with fresh local gradient statistics and consensus-derived discrepancy signals. We provide rigorous theoretical analysis and extensive experiments, showing that DECA achieves fast convergence, strong downstream performance, and significant resource efficiency.
Read the full abstract ▾
All authors Yunsheng Yuan, Shaowei Li, Kai Wang, Zhongyuan Sun, Zheng Zhang, Kai Han, Jun Luo, Feng Li
Machine Learning
Fast Organic Crystal Structure Prediction with Unit Cell Flow Matching
Alston Lo, Luka Mucko, Austin H. Cheng et al. (+4 more)
Organic
crystal structure prediction (CSP) is a requirement for computational modelling of organic solids, but traditionally costs several CPU-years per molecule. Generative models such as OXtal dramatically reduce this cost by sampling stable organic
crystal structures directly. However, OXtal forgoes explicit
lattice parametrization in favour of modelling large crops of the bulk material with expensive triangle layers, which can incur a computational cost of minutes per molecule. In this paper, we reduce this to seconds with Clari, a large-scale flow matching model that generates redundancy-free unit
cells and replaces triangle layers with pure pair-bias
attention. Clari requires only atom types and bonds as input and does not need an RDKit-sanitizable input molecule, which expands its applicability to challenging chemistries such as fullerenes, metal complexes, and atom clusters. We further ablate key design choices such as auxiliary losses, timestep distributions, noise priors, and self-conditioning. On OXtal's test sets, we surpass OXtal's solve rate while obtaining a speedup of $15$-$30\times$. Because Clari also models explicit hydrogens, it supports inference-time scaling via direct energy ranking, without any decoration or relaxation step. When generating 150
crystals and selecting the top-30 by energy, we further improve solve rate while maintaining a speedup of $5$-$8\times$. We also introduce the CSD Teaching Subset as a new test split of diverse and complex molecules for future benchmarking. Our contributions enable CSP within seconds, making large-scale virtual screening of organic solids practical. Code is available at https://github.com/aspuru-guzik-group/clari.
Read the full abstract ▾
All authors Alston Lo, Luka Mucko, Austin H. Cheng, Andy Cai, Alastair J. A. Price, Wojciech Matusik, Alán Aspuru-Guzik
Machine Learning · Cross-listed from q-fin.CP
FinStressTS: A Parametric Synthetic Benchmark for Time-Series Forecasting in Finance
Jiaze Sun, Kelvin J.L. Koa, Ruiyang Ni et al. (+3 more)
Financial
forecasting is difficult due to low signal-to-noise ratios, latent factors, heavy tails, regime shifts, and jumps. Real-world
benchmarks offer limited failure attribution: researchers can observe underperformance, but often cannot isolate why because mechanisms are unobservable and entangled. Real financial data reveal only one realized path, making it difficult to assess tail-risk calibration or data efficiency. We introduce FinStressTS, a mechanism-aware synthetic
benchmark that links model behavior to controlled structural causes. FinStressTS comprises 30 diagnostic environments around six mechanism families:
volatility clustering, multi-scale persistence, heavy-tailed shocks, regime switching, self-exciting jumps, and zero-inflated processes. We evaluate two tasks: point
forecasting, using NMAE across five settings, and probabilistic
forecasting, using CRPS under known data-generating mechanisms. We
benchmark 15 models, from classical methods (HAR, VAR) to
Transformer forecasters (PatchTST, iTransformer) and deep probabilistic architectures (DeepAR, TSFlow), and use learning curves to measure sample efficiency. Our evaluation reveals three insights. First, performance is mechanism-dependent: autoregressive and linear models are highly competitive, and often outperform
Transformer-based models, in several
volatility-, tail-, and jump-driven environments. Second, distributional
alignment matters: parametric probabilistic models such as DeepAR calibrate well in stationary settings, while flexible models can help when distributions become
multimodal or sparse. Third, neural models often require more data to match simple baselines, with larger gains mainly when learning latent regimes or complex distributions. FinStressTS provides an open framework for diagnosing failure modes and advancing risk-aware
forecasting.
Read the full abstract ▾
All authors Jiaze Sun, Kelvin J.L. Koa, Ruiyang Ni, Yize Liu, Haonan Chen, Ke-Wei Huang
Machine Learning · Cross-listed from cs.CV
GLINT: Sparsely Gated Vision-Language Alignment for Fine-Grained Radiology Representations
Jonggwon Park, Seongeun Lee, Junhyun Park et al. (+6 more)
Vision-
language models (VLMs) for radiology have emerged as a scalable paradigm by leveraging image-report pairs naturally produced in clinical workflows. However, this pairing reveals a mismatch in scale: each finding occupies only a small region of the image, yet supervision is provided only at the global image-report level. This poses a central challenge: prior approaches spread weight densely across all patches rather than concentrating on the sparse subset relevant to a given query. To address this, we present GLINT (Gated
Language-Image
alignmeNT), a framework that explicitly models this sparse correspondence. On the
alignment side, we introduce Sparsely Gated
Alignment, a novel architecture in which a sigmoid gate over a separate gate
embedding space activates only the patches relevant to each textual query, enforcing explicit sparsity. On the representation side, we add Dense Feature Regularization, which anchors the trainable encoder's intermediate features to a frozen self-supervised learning (SSL) teacher, preserving the fine-grained patch features that the gate relies on. The same recipe applies to both 2D chest X-ray (CXR) and 3D chest computed
tomography (CT), built with DINOv3 and V-JEPA 2.1, respectively. GLINT enables zero-shot classification, grounding, and
segmentation from free-text queries, and to our knowledge is the first to demonstrate zero-shot
segmentation on 3D CT volumes without mask supervision. Notably, the most pronounced gains arise on zero-shot grounding and
segmentation, where sparse, query-specific localization is required, consistent with our design intent. In downstream evaluation, GLINT outperforms both SSL encoders and medical VLMs on classification, report generation, and
segmentation.
Read the full abstract ▾
All authors Jonggwon Park, Seongeun Lee, Junhyun Park, Hannah Yun, Hyunwoong Kim, Sohyun Jeong, Hyewon Kang, Byungmu Yoon, Kyoyun Choi
Machine Learning · Cross-listed from cs.CY
Auditing Engagement Incentives in the Kidfluencer Ecosystem: A Multimodal Weak Supervision Approach
Zijing Wei, Chao Peter Yang and Xuanjie Chen
The rise of `kidfluencers' on YouTube has raised ethical concerns about child digital labor and exploitation. While emerging legislation attempts to regulate this ecosystem, empirical evidence linking exploitation to engagement remains scarce, given the difficulty of operationalizing exploitation at scale. This study presents a
multimodal AI audit of 5,051
videos across 79 kidfluencer channels, using weak supervision to detect exploitation signals without large-scale manual labels. We aggregate noisy labeling functions – including LLM-based classification of titles and GPT-4
Vision analysis of thumbnails and descriptions across six literature-grounded dimensions – to assign a probabilistic exploitation score to each
video. A multi-annotator validation study (N=107) shows strong agreement with human judgment (macro-average F1 $= 0.911$) and high sensitivity for overall exploitation risk (recall $= 0.960$, F1 $= 0.793$). Our findings reveal a significant engagement premium for performative labor, emotional bait, and
privacy violations. Exploitation scores correlate with view counts (Spearman $\rho = 0.229$, $p < 10^{-50}$), and mixed-effects regression controlling for channel-level variation shows that a one-unit increase in exploitation score yields a $4.4\times$ increase in views ($p < 0.001$). Within-channel analyses indicate median view boosts of $+65.6\%$ for emotional bait and $+56.0\%$ for performative content (FDR-corrected $p<0.001$), with effects holding in same-year robustness checks ($p=0.030$). Explicit commercial content (product placement), by contrast, shows no premium ($-3.8\%$, n.s.), suggesting the platform rewards commodification of the child's identity and labor over traditional advertising. These findings challenge policy frameworks focused solely on financial trusts, showing that engagement is systematically tied to the intensive, performative labor of children.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.SD
SketchSong: Hierarchical Song Generation with Sketch Planning and Fine-Grained Multi-Track Modeling
Xiaoyue Duan, Nanxing Hu, Yutang Feng et al. (+4 more)
Recent song generation systems can synthesize realistic audio, yet generating complete songs remains challenging for two reasons. First, explicit song-level arrangement planning remains limited in existing methods, so models often need to organize overall arrangement development while generating low-level audio details. This often leads to incoherence in arrangements, such as weak section transitions and limited dynamic progression. Second, coarse modeling of different musical parts obscures their distinct roles and interactions, limiting arrangement richness of generated songs. In this paper, we present SketchSong, a hierarchical song generation framework that addresses these issues through song-level sketch planning and fine-grained multi-track modeling. Along the temporal dimension, SketchSong first predicts a compact sequence of high-level sketch
tokens derived from compressed audio representations, and then generates audio
tokens conditioned on these sketches. This coarse-to-fine process gives the model an explicit arrangement plan before detailed audio generation. Along the track dimension, SketchSong explicitly models four tracks, i.e., vocals, bass, drums and other instruments. This enables the model to capture the roles and interactions of different musical parts more precisely. Experiments on song generation
benchmarks show that SketchSong consistently outperforms our baseline on both objective metrics and human listening tests. Despite not employing additional post-training for preference optimization such as lyrics and text-
prompt alignments, SketchSong achieves competitive results against strong, post-trained open-source systems, demonstrating the effectiveness of our overall design.
Read the full abstract ▾
All authors Xiaoyue Duan, Nanxing Hu, Yutang Feng, Xudong Yan, Jiatao Chen, Jinchao Zhang, Jie Zhou
Machine Learning
FederatedSkill: Federated Learning for Agentic Skill Evolution
Jingbo Yang, Guanyu Yao, Yang Zhang et al. (+3 more)
Modern LLM
agents increasingly rely on skill libraries to handle complex tasks, making skill
evolution a primary driver of self-improvement. However, isolated single-user task streams lack the diversity required to build comprehensive skills. While cross-user collaboration can overcome this data bottleneck, current trajectory-sharing approaches compromise user
privacy and impose a uniform global library that fails to accommodate client heterogeneity. We introduce FederatedSkill, a
privacy-preserving framework for collaborative
agent evolution. Moving beyond raw trajectory sharing, FederatedSkill utilizes semantic skill diffs, structured patches over local libraries, as the fundamental unit of communication. On the server side, an
evolution agent aggregates these patches to dynamically model client-specific capability boundaries, facilitating strictly personalized skill
evolution rather than a suboptimal global average. Evaluated across 20 distinct
agent task families, FederatedSkill demonstrates substantial gains over self-evolving baselines, achieving up to a 44.4% increase in success rate and a 37.5% reduction in computational cost.
Read the full abstract ▾
All authors Jingbo Yang, Guanyu Yao, Yang Zhang, Ramana Rao Kompella, Gaowen Liu, Shiyu Chang
Machine Learning · Cross-listed from cs.RO
How Visible Are Silent Manipulation Failures? An Observability Study of False-Success Detection in Simulated Robot Episodes
Aarav Bedi
Imitation-learning policies for
robot manipulation inherit the quality of the success labels attached to their training episodes, and those labels are usually produced by the
robot's own success check. A particularly damaging error is the false success: an episode the
robot logs as a success when the task outcome was actually wrong. We ask a narrow but practical question about these episodes. Once an episode has already been flagged as a success, how much of the information needed to overturn that label is present in proprioception, and how much requires
vision? We build a simulated testbed on two bimanual ALOHA tasks, induce failures through environment perturbations rather than label edits, label every episode by privileged simulator state that the detector never sees, and keep only episodes the
robot flagged as successful. We then compare detectors restricted to proprioception against a
vision-based detector. We find that recoverability spans a wide range: in cube transfer the false successes are almost fully recoverable from joint data alone, while in peg insertion proprioception recovers only part of them and a
vision detector closes most of the gap. We also show that the proprioceptive separability we measure rests on velocity differences far below any realistic sensor noise floor, so it is best read as an optimistic upper bound that a noiseless simulator inflates. We release the generation and evaluation pipeline.
Read the full abstract ▾
Machine Learning
HARVE: Hacking-Aware Reward-Head Vector Editing for Robust Reward Models
Shuang Liu, Yuxuan Bo, Qiuyang Zhao et al. (+4 more)
Reward models are central to large
language model (LLM)
alignment, but they remain vulnerable to reward hacking. To evaluate reward-model robustness, we introduce RewardHackBench containing 13 reward-hacking patterns covering real life high-stakes domains and general settings, and we find severe failures on specific subcategories across eight reward models. To mitigate these failures, we propose HARVE, a training-free reward-head editing method for scalar reward models. Instead of fine-tuning the reward model, HARVE identifies a multi-directional hacking subspace from residual stream directions associated with selected hacking subcategories, and removes the component of the reward-head vector aligned with that subspace. This directly reduces the reward head's sensitivity to hacking-related features using only a small set of contrastive gold-hacked examples, without gradient updates or fine-tuning. Comprehensive experiments across eight reward models indicates that \model improves hacking robustness, outperforms fine-tuning baselines, and preserves reward-models' general capability. Further analyses suggest that reward hacking is better captured as a multidimensional residual-space structure than by isolated surface cues.
Read the full abstract ▾
All authors Shuang Liu, Yuxuan Bo, Qiuyang Zhao, Caiyue Huang, Xiaorong Chen, Yanguang Liu, Mengnan Du