Machine Learning
SMoA: Spectrum Modulation Adapter for Parameter-Efficient Fine-Tuning
Yongkang Liu, Xing Li, Mengjie Zhao et al. (+7 more)
As the number of model parameters increases, parameter-efficient fine-tuning (PEFT) has become the go-to choice for tailoring pre-trained large
language models. Low-rank Adaptation (LoRA) uses a low-rank update method to simulate full parameter fine-tuning, which is widely used to reduce resource requirements. However, decreasing the rank encounters challenges with limited representational capacity. Theory suggests that LoRA fine-tuning with rank r converges toward the top r singular values of the pre-trained weight
matrix. As the rank increases, more principal singular directions are preserved, which generally improves the model's performance. However, a larger rank also introduces more trainable parameters, leading to higher computational cost. To overcome this dilemma, we propose SMoA, a Spectrum Modulation Adapter that enlarges the accessible family of spectrum-aware updates under a smaller parameter budget. SMoA partitions the layer into multiple aligned spectral blocks and applies one in-block Hadamard-modulated low-rank branch to each diagonal block, yielding broader coverage of pretrained spectral directions. We provide theoretical analysis and empirical results on multiple tasks. In our experiments, SMoA improves average performance in the current lower-budget setting over LoRA and competitive LoRA-style baselines.
Read the full abstract ▾
All authors Yongkang Liu, Xing Li, Mengjie Zhao, Shanru Zhang, Zijing Wang, Qian Li, Shi Feng, Feiliang Ren, Daling Wang, Hinrich Schütze
Machine Learning · Cross-listed from cs.SD
CoarseSoundNet: Building a reliable model for ecological soundscape analysis
Alexander Gebhard, Andreas Triantafyllopoulos, Dominik Arend et al. (+4 more)
A soundscape is composed of three types of sound: biophony (sounds made by animals), geophony (natural abiotic sounds) and anthropophony (sounds made by humans). A key research question in the field of soundscape ecology is how these components interact with each other, specifically how biophony responds to geophony and anthropophony. Nevertheless, as of today, there are not many analytical instruments that enable the distinct quantification of these elements. Recent machine learning (ML) approaches aim to support automated analysis but often rely on task-specific or clean data, limiting generalisation to noisy passive acoustic monitoring (PAM) recordings. This study presents a reproducible framework to build ML models for coarse soundscape classification and introduces CoarseSoundNet, a deep learning model trained to distinguish biophony, geophony, and anthropophony under realistic terrestrial PAM conditions. We systematically investigate model architectures, the influence of an additional training class, data composition, and evaluation strategies. Our findings suggest that model performance improves with additional PAM data, especially when similar to the target domain, and by introducing an explicit silence class during training. Post-hoc target-domain calibration employing class-specific decision thresholds further enhance performance, while incorporating minimum annotation durations yields higher scores for anthropophony and geophony. Error analyses exhibit challenges for anthropophony due to masking effects and confusions for silence and insect sounds for geophony and biophony. Finally, we conduct an ecological case study showing that pre-filtering recordings with CoarseSoundNet yields acoustic index trends broadly similar to ground-truth filtering, although the resulting improvements in their relationships with avian alpha-diversity were limited.
Read the full abstract ▾
All authors Alexander Gebhard, Andreas Triantafyllopoulos, Dominik Arend, Sandra Müller, Svenja Schmidt, Michael Scherer-Lorenzen, Björn W. Schuller
Machine Learning · Cross-listed from cs.CV
Distill to Think, Foresee to Act: Cognitive-Physical Reinforcement Learning for Autonomous Driving
Yang Wu, Qiang Meng, Zhaojiang Liu et al. (+3 more)
Current end-to-end autonomous driving models are fundamentally constrained by the behavioral cloning ceiling of imitation learning. While reinforcement learning offers a path to smarter autonomy, it demands two missing pieces of infrastructure: (1) a cognitive foundation that understands traffic semantics and driving intent, and (2) a foresighted physical environment that can anticipate the consequences of candidate actions. To this end, we propose CoPhy, a CognitivePhysical reinforcement learning framework for autonomous driving. To distill to think, we distill VLM knowledge into the BEV encoder and then discard the VLM entirely, retaining cognitive ability at zero inference cost while releasing the cognitive channel as a pluggable interface for optional human
language commands. To foresee to act, we build an auto-regressive BEV world model that explicitly predicts future semantic maps conditioned on candidate actions, serving as an interpretable physical sandbox from which safety metrics are directly derived. Built upon this dual infrastructure, we optimize the driving policy via GRPO with a novel dual-reward mechanism: a physical reward derived from BEV rollouts enforces hard safety constraints, while a cognitive reward from a
language-aligned scorer ensures intent compliance. Extensive experiments demonstrate that CoPhy not only achieves state-of-the-art results on NAVSIM v1 and v2
benchmarks, but also enables safer driving via cognitively informed scene compliance and flexible intent control through user-defined
language instructions. Project page: https://wuyang98.github.io/CoPhy/.
Read the full abstract ▾
All authors Yang Wu, Qiang Meng, Zhaojiang Liu, Youquan Liu, Jian Yang, Jin Xie
Machine Learning
Reasoning-Trace Collapse: Evaluating the Loss of Explicit Reasoning During Fine-Tuning
Lukas Twist, Helen Yannakoudakis and Jie M. Zhang
Explicit
reasoning models are trained to produce intermediate
reasoning traces before final answers, but downstream fine-tuning is often performed on ordinary instruction–response data that contains no such traces. We show that this mismatch can induce
reasoning-trace collapse: a fine-tuned model continues to produce plausible final answers while losing the structurally valid explicit
reasoning traces that made it a
reasoning model in the first place. We introduce a structural evaluation framework that separates answer correctness from
reasoning-trace validity, measuring valid, empty, missing, and truncated
reasoning alongside
reasoning-conditioned task performance. Using this framework, we study four open-weight
reasoning models and find that standard supervised fine-tuning can rapidly suppress valid
reasoning traces, and that answer-only metrics can substantially obscure this failure: in several settings, performance conditional on valid
reasoning remains high while the rate of valid
reasoning falls sharply. We further show that simple loss-masking strategies can substantially mitigate collapse without requiring teacher-generated
reasoning traces. These results suggest that evaluations of fine-tuned
reasoning models should report structural
reasoning reliability metrics in addition to final-answer performance, especially when adaptation data does not contain explicit
reasoning traces.
Read the full abstract ▾
Machine Learning
Advantage Collapse in Group Relative Policy Optimization: Diagnosis and Mitigation
Xixiang He, Qiyao Sun, Ao Cheng et al. (+5 more)
Group Relative Policy Optimization (GRPO), a prominent
algorithm within the Reinforcement Learning from Verifiable Rewards (RLVR) framework, has achieved strong results in improving the
reasoning capabilities of large
language models (LLMs). However, GRPO is prone to advantage collapse, a failure mode where homogeneous rewards within a group (e.g., all correct or all incorrect answers) yield near-zero advantages and vanishing gradients. To address this, we introduce the Advantage Collapse Rate (ACR), the first diagnostic metric quantifying the proportion of training batches with ineffective gradients. Across models from 0.5B to 14B parameters on mathematical
reasoning benchmarks, we show that ACR strongly predicts training stagnation and final performance. We then propose Adaptive Virtual Sample Policy Optimization (AVSPO), a lightweight extension of GRPO that injects virtual reward samples, guided by real-time ACR monitoring, to enable learning from homogeneous groups without additional model rollouts. AVSPO reduces advantage collapse by 58-63% relative to GRPO and yields consistent accuracy gains of 4-6 percentage points across all model scales, while maintaining generalization on the evaluated out-of-domain task. Code and
datasets are available at https://github.com/hexixiang/Advantage-Collapse-Rate.
Read the full abstract ▾
All authors Xixiang He, Qiyao Sun, Ao Cheng, Xingming Li, Xuanyu Ji, Hailun Lu, Runke Huang, Qingyong Hu
Machine Learning · Cross-listed from cs.CV
Linear-DPO: Linear Direct Preference Optimization for Diffusion and Flow-Matching Generative Models
Kesong Li, Yixuan Xu, Kuo-kun Tseng et al. (+3 more)
Direct Preference Optimization (DPO) is successful for
alignment in LLMs but still faces challenges in text-to-image generation. Existing studies are confined to denoising
diffusion models while overlooking flow-matching, and suffer from an objective mismatch when applying discrete NLP-based DPO to regression-based generative tasks.\ In this paper, we derive a generalized DPO objective that covers both
diffusion and flow-matching via a unified reverse-time SDE framework, and point out from a gradient perspective that the standard DPO objective is suboptimal for text-to-image generation. Consequently, we propose Linear-DPO, which replaces the aggressive sigmoid-based utility function with a sustained linear utility and incorporates an EMA-updated reference model. Qualitative and quantitative experiments on
diffusion models (SD1.5, SDXL) and flow-matching model (SD3-Medium) demonstrate the superiority of our approach over existing baselines.
Read the full abstract ▾
All authors Kesong Li, Yixuan Xu, Kuo-kun Tseng, Weiyi Lu, Kan Liu, Tao Lan
Machine Learning · Cross-listed from cs.DC
Automated Byzantine-Resilient Clustered Decentralized Federated Learning for Battery Intelligence in Connected EVs
Mouhamed Amine Bouchiha, Abdelaziz Amara Korba and Yacine Ghamri-Doudane
Federated learning (FL) has emerged as a promising paradigm for managing electric vehicle (EV) battery data in intelligent transportation systems (ITS), enabling
privacy-preserving tasks such as anomaly detection and capacity estimation. However, most existing frameworks rely on centralized aggregation schemes, which pose critical limitations in terms of security and trust. To address these challenges, we propose ABC-DFL, an automated Byzantine-resilient clustered decentralized
federated learning (C-DFL) framework for connected EVs. The proposed incentive-driven C-DFL system replaces the central server with an open-permissioned blockchain, featuring a new dynamic Quorum Byzantine Fault Tolerance (QBFT) protocol and an oracle-based aggregation layer, to enhance trust, security, and automation. At the core of ABC-DFL lies FLECA (Filtered Layered Enhanced Clustering Aggregation), a robust hierarchical aggregation protocol that mitigates Byzantine attacks by having each EV filter malicious updates using an adaptive threshold based on deviations from its reference model update. Oracle nodes, responsible for inter-group aggregation, employ robust clustering to isolate and aggregate model updates from trustworthy EV groups. Comprehensive experimental evaluations demonstrate that FLECA matches FedProx convergence under benign conditions and significantly outperforms existing defenses with attack impact scores below 0.10 in adaptive
adversarial scenarios. Furthermore, several learning experiments with multitask models confirm the effectiveness and
fairness of the incentive mechanism. Finally, on-chain and off-chain
benchmarks validate the practicality of ABC-DFL.
Read the full abstract ▾
Machine Learning
A Unified Framework for Uncertainty-Aware Explainable Artificial Intelligence: A Case Study in Power Quality Disturbance Classification
Yinsong Chen, Samson S. Yu, Zhong Li et al. (+1 more)
Post-hoc explainable AI (XAI) methods usually return one attribution map, even when the model represents uncertainty in its parameters. We define the explanation distribution as the distribution of attribution maps obtained from sampled models. The uncertainty-aware relevance attribution operator (UA-RAO) summarises this distribution using the mean, dispersion,
quantiles, and agreement sets. The theory separates posterior-approximation error from finite-sample error and accounts for changes across activation boundaries and for
stochastic explainers. On a 15-class power-quality-disturbance
benchmark, the mean occlusion explanation from a deep ensemble aligns better with known disturbance regions than the deterministic baseline, although the improvement depends on the disturbance type. Tests with controlled input distortions show that additive noise changes the explanations more than amplitude scaling or aligned temporal shifts.
Read the full abstract ▾
All authors Yinsong Chen, Samson S. Yu, Zhong Li, Chee Peng Lim
Machine Learning
Efficient Learning of Deep State Space Models via Importance Smoothing
John-Joseph Brady, Nikolas Nusken and Yunpeng Li
Latent state space systems are ubiquitous in statistical modelling, arising naturally when time series are observed through noisy measurements. However, training deep state space models (DSSMs) at scale remains difficult. Two largely distinct strategies have emerged for training DSSMs. The first, auto-encoding DSSMs, trains generative models by optimising a variational lower bound. The second backpropagates through the outputs of classical sequential Monte Carlo (SMC)
algorithms. Such approaches can train DSSMs for both discriminative and generative tasks, but their inherently sequential forward passes scale poorly on modern hardware. We propose parallel variational Monte Carlo (PVMC), a new training method that bridges these paradigms and robustly trains DSSMs for both discriminative and generative tasks. Across a set of
benchmark experiments, PVMC matches or exceeds state-of-the-art performance while training $10\times$ faster than the fastest competing SMC-based approach.
Read the full abstract ▾
Machine Learning
A Geometric Approach to Constrained Online Learning
Dhruv Sarkar and Abhishek Sinha
We study constrained online convex optimization with
adversarial time-varying constraints. At each round the learner acts before observing the loss and constraint, and is compared with the best fixed action satisfying all constraints in hindsight. The goal is to obtain minimax-optimal regret while controlling cumulative constraint violation (CCV). Prior
algorithms achieved $O(\log T)$ regret with $O(\sqrt{T\log T})$ CCV for strongly convex losses, and $O(\sqrt{T})$ regret with $O(\sqrt{T}\log T)$ CCV for convex losses. We present NP-OGD, an iterated nested-projection
algorithm. For strongly convex losses it achieves $O(\log T)$ regret and $O(\log T)$ CCV; for convex losses it achieves $O(\sqrt{T})$ regret and $O(\sqrt{T})$ CCV. The analysis relies on a
geometric movement bound: after lifting the nested projected-gradient trajectory to one higher dimension, the lifted path is self-contracted under a nonstandard norm, so a finite-length
theorem for self-contracted curves controls the total projection movement. We also prove complementary lower bounds using layered sphere packings. For strongly convex losses, any online
algorithm with polynomially sublinear regret can incur CCV at least $\Omega((\log T)^{(d-1)/(d+1)}/\log\log T)$. For convex losses, we prove CCV lower bounds $\Omega(T^{(d-1)/(2(d+3))})$ for weakly adaptive
algorithms and $\Omega(T^{(d-1)/(2d)})$ for NP-OGD. Finally, for the constrained experts special case over $N$ experts, an active Hedge
algorithm attains $O(\sqrt{T\log N})$ regret and $O(N)$ CCV, with a matching minimax CCV lower bound for sufficiently large horizons.
Read the full abstract ▾
Machine Learning
HORST: Composing Optimizer Geometries for Sparse Transformer Training
Tom Jacobs, Rohan Jain and Rebekka Burkholz
Sparsifying
transformers remains a fundamental challenge, as standard optimizers fail to simultaneously encourage sparsity and maintain training stability. Effective adaptive optimizers exhibit an implicit $L_{\infty}$ bias favoring stability, yet, sparsity requires an $L_1$ bias. To integrate sparsity, we propose a composition of optimizer steps, which we cast as non-commutative operators to analyze and combine their optimization geometry in a principled way. This yields HORST (
Hyperbolic Operator for Robust Sparse Training), a modular optimizer that inherits stability from adaptive methods while inducing $L_1$ sparsity bias through a
hyperbolic mirror map. Our experiments demonstrate its utility for sparse training of
transformers on both
vision and
language tasks. HORST consistently and significantly outperforms AdamW baselines across all sparsity levels, with large gains at higher sparsity.
Read the full abstract ▾
Machine Learning
A Typed Tensor Language for Shared-State Federated Computation
Theofilos Mailis, Theodore Papamarkou, Andreas Ktenidis et al. (+6 more)
Shared-state
federated computations combine client-local tensor computation, mergeable aggregation into shared state, and shared-only post-processing. We introduce a typed tensor
language for this class of computations. Its two tensor sorts separate client-partitioned data from globally available values, and typing tracks the partitioned axis. A virtual global tensor serves as a semantic reference for centralized evaluation. We show that typed one-round programs factor through shared tensors whose shapes depend on the program but are independent of client and sample counts. The converse applies to typed-realizable factorizations: each encoder component is represented by an allowed aggregation or contraction with its valid merge, and the decoder is shared-only. The construction extends round by round to programs whose persistent state is shared. For a loss supplied with a client-local per-sample gradient expression, summation represents the empirical gradient. This gives typed programs for server-side first-order updates and, with shared linear
algebra,
curvature-block updates. The
language covers
federated analytics and FedSGD. General multi-local-step FedAvg and persistent private client state are outside its scope.
Read the full abstract ▾
All authors Theofilos Mailis, Theodore Papamarkou, Andreas Ktenidis, Kalliopi-Christina Despotidou, Konstantinos Filippopolitis, Yannis Foufoulas, Thanasis-Michail Karampatsis, Evdokia Mailli, Yannis Ioannidis
Machine Learning
UOTIP: Unbalanced Optimal Transport Map for Unpaired Inverse Problems
Donggyu Lee, Taekyung Lee and Jaewoong Choi
We investigate unpaired image inverse problems, a challenging setting where only independent, non-paired sets of noisy measurements and clean target signals are available for training. We propose a novel inverse problem solver based on Unbalanced Optimal Transport, called Unbalanced Optimal Transport Map for Inverse Problems (UOTIP). Our method formulates the reconstruction task, predicting clean target signals from noisy measurements, as learning a UOT Map from noisy measurement distribution to clean signal distribution by incorporating a likelihood-based cost function. By relaxing the exact marginal constraint, the UOT framework provides key advantages to our model: robustness to multi-level observation noise, adaptability to class imbalance between noisy and clean
datasets, and generalizability to diverse noise-type scenarios. Furthermore, we theoretically demonstrate that incorporating a quadratic cost term ensures the existence and uniqueness of the transport map by satisfying the twist condition, even for ill-posed inverse problems. Our experiments demonstrate that UOTIP achieves state-of-the-art performance on unpaired image inverse problem
benchmarks, across linear and nonlinear inverse problems.
Read the full abstract ▾
Machine Learning
Reviving Error Correction in Modern Deep Time-Series Forecasting
Minh Hoang Nguyen, Dai Do, Huu Hiep Nguyen et al. (+3 more)
Modern deep-learning models have achieved remarkable success in time-series
forecasting. Yet, their performance degrades in long-term prediction due to error accumulation in autoregressive inference, where predictions are recursively used as inputs. While classical error correction mechanisms (ECMs) have long been used in statistical methods, their applicability to deep learning models remains limited or ineffective. In this work, we revisit the error accumulation problem in deep time-series
forecasting and investigate the role and necessity of ECMs in this new context. We propose a simple, architecture-agnostic error correction model that can be integrated with any existing forecaster without requiring retraining. By explicitly decomposing predictions into trend and seasonal components and training the corrector to adjust each separately, we introduce the Universal Error Corrector with Seasonal-Trend Decomposition (UEC-STD), which significantly improves correction accuracy and robustness across 4 backbones and 10
datasets. Our findings provide a practical tool for enhancing forecasts while offering new insights into mitigating autoregressive errors in deep time-series models. Code is available at https://github.com/DA2I2-SLM/UEC-STD.
Read the full abstract ▾
All authors Minh Hoang Nguyen, Dai Do, Huu Hiep Nguyen, Dung Nguyen, Kien Do, Hung Le
Machine Learning · Cross-listed from cs.MA
Decoupling Communication from Policy: Robust MARL under Bandwidth Constraints
Alexi Canesse, Benoît Goupil, Jesse Read et al. (+1 more)
Communication enables coordination in multi-
agent reinforcement learning (MARL), but many real-world applications, e.g., search-and-rescue with drone swarms, operate under severe bandwidth constraints. Many communication architectures still expose a coupled bottleneck in which a shared latent representation is used for both policy execution and inter-
agent communication. Consequently, reducing message size directly limits the policy's latent space, often leading to significant performance degradation. We address this with two contributions. First, we introduce $\beta$, a normalised per-
agent bandwidth budget that unifies sparsity, rounds, and message dimension into a single comparable constraint. Second, we provide SLIM, a minimal architecture that decouples the communication pathway from the policy's latent representation, allowing us to isolate the effect of bandwidth from the effect of policy capacity while benefiting from in-step communication. We evaluate our method on several partially-observable MARL
benchmarks, where communication is essential. Our approach achieves state-of-the-art performance and exhibits scalability and robustness under limited communication, with only marginal degradation as bandwidth is reduced.
Read the full abstract ▾
All authors Alexi Canesse, Benoît Goupil, Jesse Read, Sonia Vanier
Machine Learning · Cross-listed from physics.app-ph
AIMBio-Mat: An AI-Native FAIR Platform for Closed-Loop Materials Discovery and Biomedical Translation
D.-M. Mei, K. Acharya, C. M. Adhikari et al. (+51 more)
Materials discovery and biomedical translation increasingly require models that can reason across composition, processing, structure, biological response, manufacturability, safety, and governance constraints. Existing materials and biomedical data ecosystems are powerful but remain poorly coupled for AI-guided discovery. Here we present AIMBio, a conceptual framework for an AI-native, FAIR, and governance-aware decision layer that links materials provenance, biomedical context, knowledge graphs, uncertainty-aware machine learning, and human-in-the-loop active learning. The framework formulates biomedical-materials discovery as constrained multi-objective optimization under uncertainty and introduces practical requirements for metadata, model documentation, risk-tiered governance, evaluation metrics, and phased implementation. To make the roadmap testable, we add a minimum viable prototype specification and a worked pilot for AI-guided nanomaterials for drug delivery. AIMBio is positioned as exploratory and preclinical discovery infrastructure, not as clinical decision-support software; any clinical or regulated-device use would require separate validation, change control, and regulatory review. The central contribution is a publishable platform blueprint for converting fragmented materials and biomedical records into auditable, experimentally actionable, and translationally responsible discovery workflows.
Read the full abstract ▾
All authors D.-M. Mei, K. Acharya, C. M. Adhikari, M. Adhikari, S. Aryal, B. V. Benson, K. Bhatta, S. Bhattarai, N. Budhathoki, A. M. Castillo, D. Chakraborty, S. Chhetri, S. Choudhury, T. A. Chowdhury, R. D. Cruz, B. Cui, S. Dhital, K.-M. Dong, R. Gapuz, A. Ghasemi, E. Z. Gnimpieba, B. D. S. Gurung, H. A. Hashim, R. I. Harry, K.-E. Hasin, M. K. Hassanzadeh, M. K. Jha, D. Kim, K.-C. Kong, B. Lama, A. Mahat, N. Maharjan, A. Majeed, J. Mammo, M. M. Masud, K. S. Moore, A. Nawaz, H. Oli, S. A. Panamaldeniya, L. Pandey, R. Pandey, Z. Peng, A. Prem, M. M. Rana, K. Rana Magar, R. Rizk, C. S. Tadi, L.-W. Wang, Y. Yang, G.-L. Yin, C.-X. Yu, D. Zeng, M. Zhou, Q. Zhou
Machine Learning · Cross-listed from cs.SD
Shinnosuke Takasuka and Hideo Mukai
This study aims to enhance the quality of music generation using
Transformers by incorporating meta-information. While
Transformer-based approaches are effective at capturing long-term dependencies in musical compositions, the music they generate often suffers from issues such as excessive repetition or duplication of notes, leading to unnatural melodies. To address these limitations, we propose Musical
Attention, a mechanism that incorporates meta-information such as bar numbers, key, signatures, and tempos into the
attention process. Musical
Attention explicitly leverages both the structural properties of music and its associated metadata, enabling the
Transformer's
attention mechanism to operate more effectively and thereby improving the quality of the generated output. In our framework, each musical note is represented as a combination of five events-pitch, bar number, onset, duration, and velocity in addition to the three metadata elements. The
attention mechanism is then modified to reflect the correlations among these eight features, allowing the model to better capture the inherent characteristics of musical composition. Experimental results demonstrate that the model incorporating Musical
Attention outperforms prior methods, such as Full
Attention and Strided
Attention, in terms of musical coherence, variation, and overall quality. Notably, it significantly reduces repetition and enhances the model's ability to generate diverse, harmonically consistent melodies. Musical
Attention thus represents a meaningful advancement in AI-driven music generation, facilitating the creation of more natural and expressive compositions.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
SpectralEarth-FM: Bringing Hyperspectral Imagery into Multimodal Earth Observation Pretraining
Nassim Ait Ali Braham, Aaron Banze, Conrad M. Albrecht et al. (+3 more)
Earth observation (EO) foundation models (FMs) are increasingly trained on multisensor data, spanning multispectral imagery (MSI), synthetic aperture radar (SAR), and derived geospatial layers, but hyperspectral imagery (HSI) remains underrepresented. Conversely, existing hyperspectral FMs are trained on HSI alone, leaving joint pretraining and fusion of HSI with co-located EO sensors unexplored. We introduce SpectralEarth-FM, a hierarchical
transformer for multisensor EO input with heterogeneous spectral dimensionality. The architecture combines spectral tokenization for hyperspectral inputs, sensor-specific encoders, a cross-sensor fusion module, and a shared hierarchical encoder, enabling joint processing of HSI and lower-channel observations. To pretrain SpectralEarth-FM, we curate SpectralEarth-MM, a
dataset that co-locates HSI from three spaceborne sensors (EnMAP, EMIT, DESIS) with Sentinel-2, Landsat-8/9 optical imagery, Landsat land surface temperature (LST), and Sentinel-1 SAR, over common geographic footprints. It comprises approximately 2M globally distributed locations, 25M georeferenced patches, and over 40TB of data. Pretraining uses a Joint-
Embedding Predictive Architecture (JEPA)-style objective that matches representations between global views and single-sensor local views from the same location. We evaluate SpectralEarth-FM on hyperspectral downstream tasks and standard EO
benchmarks following the PANGAEA protocol, achieving state-of-the-art results across both evaluation settings.
Read the full abstract ▾
All authors Nassim Ait Ali Braham, Aaron Banze, Conrad M. Albrecht, Julien Mairal, Jocelyn Chanussot, Xiao Xiang Zhu
Machine Learning
Towards Understanding Self-Pretraining for Sequence Classification
Omar Coser, Loredana Zollo, Paolo Soda et al. (+1 more)
Amos et al. (2024) showed that the accuracy of
Transformer models in sequence classification can be significantly improved by first pretraining with a masked
token prediction objective without external data or augmentation, a procedure referred to as self-pretraining (SPT). While the primary objective of Amos et al. (2024) was to showcase that
Transformers can achieve strong performance on the Long-Range Arena (LRA), their pipeline raises more fundamental questions: How does SPT drive optimization to better solutions? Why can standard supervised training fail in
Transformers? To better understand this, we replicate and systematically ablate the findings of Amos et al. (2024). Our ablations suggest that a central bottleneck in the studied settings is not depth or generalization alone, but the ability of label supervision to learn useful query-key
Attention patterns from random initialization. With a minimal setup, we identify learning proximity interactions - turning absolute positional encodings into proximity-biased
Attention scores - as a key source of the improvements brought by SPT. Finally, in a simplified theoretical setup, we show that label supervision can be locally blind to certain
Attention-score directions that are instead detectable through masked reconstruction.
Read the full abstract ▾
All authors Omar Coser, Loredana Zollo, Paolo Soda, Antonio Orvieto
Machine Learning
PUID: A Personalized Deconfounding Framework for Recommender Systems under Hidden Confounding
Zongyu Li
Recommender systems often rely on observational user-item interaction data, which is prone to selection bias due to users' selective interactions with items. While techniques such as inverse propensity weighting (IPW) and doubly robust
estimators are effective in addressing selection bias from observed confounding, they become unreliable when hidden confounding exists, meaning that there are confounders that influence both user clicks and feedback but are not observable (e.g., user salary). Existing approaches relying on randomized controlled trials (RCTs) or global sensitivity bounds are constrained in practice: RCTs demand costly experimental data, while global sensitivity bounds presume a uniformly bounded effect of unmeasured confounders on propensities through sensitivity analysis, thereby neglecting heterogeneity across user-item interactions. To overcome this limitation, we propose a novel framework, Personalized Unobserved-Confounding-aware Interaction Deconfounder (PUID), which estimates user-item level sensitivity bounds, thereby substantially relaxing the homogeneity assumption inherent in global sensitivity bounds. Under mild assumptions, we use an
entropy-based method to estimate the individualized strength of hidden confounding. Specifically, if observed user and item features can already well predict the exposure status (i.e., the mutual information is small), then the influence of hidden confounding is assumed to be small. To ensure both robustness and predictive accuracy, we further develop an
adversarial optimization strategy and propose a
benchmark-guided variant (BPUID) that incorporates pre-trained models as stabilizing references. Extensive experiments on three real-world
datasets demonstrate that our approach consistently outperforms state-of-the-art baselines under hidden confounding, without requiring RCT data.
Read the full abstract ▾
Machine Learning
Divide et Calibra: Multiclass Local Calibration via Vector Quantization
Cesare Barbera, Lorenzo Perini, Giovanni De Toni et al. (+2 more)
Accurate and well-calibrated Machine Learning (ML) models are mandatory in high-stakes settings, yet effective multiclass calibration remains challenging: global approaches assume calibration errors are homogeneous across the latent space, while local methods often rely on representation-space dimensionality reduction, discarding latent directions relevant to miscalibration. To address these issues, we propose a compositional approach to multiclass calibration, where region-specific calibration maps are constructed from shared factors. We instantiate this idea via Vector Quantization (VQ), which induces a structured partition of the representation space, and an indexed parameterization of Dirichlet concentrations that enables parameter sharing across regions. Our approach learns heterogeneous calibration maps that generalize well even to sparse regions of the latent space. Experiments on
benchmark datasets show significant improvements in local calibration while maintaining competitive global calibration and predictive performance.
Read the full abstract ▾
All authors Cesare Barbera, Lorenzo Perini, Giovanni De Toni, Andrea Passerini, Andrea Pugnana
Machine Learning · Cross-listed from cs.CV
Multimodal LLMs under Pairwise Modalities
Yan Li, Yunlong Deng, Yuewen Sun et al. (+3 more)
Despite the impressive results achieved by
multimodal large
language models (MLLMs), their training typically relies on jointly curated
multimodal data, requiring substantial human effort to construct multi-way aligned
datasets and thereby limiting scalability across domains. In this work, we explore training MLLMs by only leveraging multiple paired modalities as a surrogate for the full joint
multimodal distribution. Specifically, we first provide a theoretical analysis of the conditions under which the representations are identifiable with only observing pairwise modalities. Building on this analysis, we propose a representation learning framework for aligning latent representations across modalities using only pairwise data. The framework consists of two stages: latent representation
alignment and cross-modal recomposition. Specifically, in the first stage, we learn the shared latent space across modalities by both self-modal reconstruction and pair-wise contrastive learning. We also incorporate an inductive bias in the contrastive learning process by partially aligning and minimal latent specification. In stage two, we integrate the encoder of newly introduced modalities with the decoders of the pre-trained modalities to facilitate cross-modal transfer and generation. We evaluate our method by newly adding 3D point clouds and tactile modalities into pre-trained MLLMs with three modality pairs and show that, by learning an aligned latent representation space, our model achieves strong cross-modal performance.
Read the full abstract ▾
All authors Yan Li, Yunlong Deng, Yuewen Sun, Gongxu Luo, Kun Zhang, Guangyi Chen
Machine Learning
A Dialogue between Causal and Traditional Representation Learning: Toward Mutual Benefits in a Unified Formulation
Yan Li, Yuewen Sun, Shaoan Xie et al. (+4 more)
Causal representation learning (CRL) and traditional representation learning have largely developed along different trajectories. Traditional representation learning has been driven mainly by applications and empirical objectives, whereas CRL has focused more on theoretical questions, particularly identifiability. This difference in emphasis has created a gap between the two fields in terminology, problem formulation, and evaluation, limiting communication and sometimes leading to disconnected or redundant efforts. In this paper, we argue that these two fields should be brought into dialogue rather than treated as separate paradigms. To this end, we introduce a unified formulation in which the representation learning is characterized by two components: a task component, which specifies what information the learned representation is required to preserve, and a constraint component, which specifies what structure is imposed on the latent space. Under this formulation, the benefits run in both directions. CRL provides theoretical tools for understanding when structured latent constraints are useful or necessary, while traditional representation learning offers practical insights on task design and objective choice that can improve the development of CRL methods. To illustrate this interaction, we experimentally study how different task components affect the behavior of CRL methods under different structured constraints. Results on CausalVerse show that the effectiveness of causal constraints depends strongly on the tasks with which they are paired.
Read the full abstract ▾
All authors Yan Li, Yuewen Sun, Shaoan Xie, Gongxu Luo, Yunlong Deng, Kun Zhang, Guangyi Chen
Machine Learning · Cross-listed from cs.NE
Genetic Programming with Transformer-Based Mutation for Approximate Circuit Design
Ondrej Galeta and Lukas Sekanina
A recent trend is to leverage machine learning models to improve the evolutionary design and optimization process. We propose a novel
transformer-based mutation operator for Cartesian genetic programming (CGP) for the automated design of approximate arithmetic
circuits. We introduce a hybrid scheme for CGP in which the proposed mutation operator is switched with the standard mutation operator to prevent stagnation of the
circuit approximation process. We also develop a new training scheme for the underlying
transformer that utilizes training vectors composed of thousands of CGP chromosomes representing various approximate multipliers. For several target error constraints, the approximate multipliers evolved with CGP utilizing the
transformer-based mutation achieve better trade-offs than the highly optimized designs available in the state-of-the-art EvoApproxLib library of approximate
circuits. Although both training and evolutionary processes are computationally demanding, they appear to be necessary steps for improving existing approximate
circuits and producing new, potentially patentable
circuit designs.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Conditioning Gaussian Processes on Almost Anything
Henry Moss, Lachlan Astfalck, Thomas Cowperthwaite et al. (+5 more)
Gaussian processes (GPs) offer a principled probabilistic model over functions, but exact inference is restricted to the linear-Gaussian regime. We establish an explicit equivalence between GPs and a class of linear
diffusion models, recasting predictive sampling as an ODE with closed-form Gaussian dynamics and a likelihood-dependent guidance term that admits a simple Monte Carlo approximation. In the linear-Gaussian setting, we recover standard GP conditioning exactly; beyond conjugacy, the same machinery handles any conditioning statement admitting point-wise likelihood evaluation - including non-linear physics, and, for the first time, natural
language via large
language models. Whitening isolates the irreducible non-Gaussian dynamics, minimising Wasserstein-2 transport cost and eliminating numerical stiffness. The result is a general-purpose GP inference scheme requiring no bespoke derivations. Together, these results provide a general mechanism for incorporating the full richness of real-world knowledge as conditioning information, opening a new frontier for the probabilistic modelling of real-world problems.
Read the full abstract ▾
All authors Henry Moss, Lachlan Astfalck, Thomas Cowperthwaite, Colin Doumont, Sam Willis, Philipp Hennig, Christopher Nemeth, Andrew Zammit-Mangion
Machine Learning
Efficient Banzhaf-Based Data Valuation for $k$-Nearest Neighbors Classification
Guangyi Zhang, Lutz Oettershagen, Lixu Wang et al. (+1 more)
Data valuation, the task of quantifying the contribution of individual data points to model performance, has emerged as a fundamental challenge in machine learning. Game-theoretic approaches, such as the Banzhaf value, offer principled frameworks for fair data valuation; however, they suffer from exponential computational complexity. We address this challenge by developing efficient
algorithms specifically tailored for computing Banzhaf values in $k$-nearest neighbor ($k$NN) classifiers. We first establish the theoretical hardness of the problem by proving that it is \#P-hard. Despite this intractability, we exploit the locality properties of $k$NN classifiers to develop practical exact
algorithms. Our main contribution is a dynamic programming framework that achieves significant computational improvements: we present a pseudo-
polynomial algorithm with $O(Wkn^2)$ time complexity for weighted $k$NN classifiers, where $W$ is the maximum sum of top-$k$ weights, and a specialized
algorithm for unweighted $k$NN that achieves $O(nk^2)$ time complexity, that is, linear in the number of data points. We also offer efficient Monte Carlo estimation methods. Extensive experiments on real-world
datasets demonstrate the practical efficiency of our approach and its effectiveness in data valuation applications.
Read the full abstract ▾
All authors Guangyi Zhang, Lutz Oettershagen, Lixu Wang, Aristides Gionis
Machine Learning · Cross-listed from cs.AI
Playing Devil's Advocate: Off-the-Shelf Persona Vectors Rival Targeted Steering for Sycophancy
Ishaan Kelkar, Vikram Kakaria, Nebras Alam et al. (+3 more)
Language models are often sycophantic: they agree with a user's stated opinion whether or not it is correct. Prior work has shown that this trait can be controlled by steering a model with a sycophancy persona vector (Chen et al., 2025). Such vectors, however, are extracted from data about sycophancy itself. We ask whether we can instead reuse existing vectors for general roles—Skeptic, Judge, Devil's Advocate—that were extracted without targeting sycophancy at all. On Gemma 2 27B and Qwen 3 32B, we compare these role vectors with a purpose-built Contrastive Activation Addition (CAA) sycophancy vector on a largely held-out, counterbalanced PhilPapers
benchmark, using task-specific coefficient tuning on a separate split of sycophancy data. The selected "critical" roles achieve, on average, about 68% (Gemma) and 98% (Qwen) of CAA's reduction in the sycophancy logit. Less agreement does not mean more factual errors on the probes we checked: on 16 true and false factual claims, Qwen steered by the Skeptic or Judge vector still gives the correct answer in all 16 cases, matching the unsteered model and CAA. "Conformist" roles do not reliably produce the opposite effect. Role vectors also have low absolute cosine similarity with the measured CAA direction at the layer we steer; they are geometrically separate interventions, although this does not by itself show that they act through distinct downstream mechanisms. Together, these results show that general persona vectors can help mitigate sycophancy in LLMs, even when extracted without sycophancy-specific labels. Code: https://github.com/vkmk1/Sycophancy-Steering Results: https://github.com/kelkarI/sycophancy-clean-results
Read the full abstract ▾
All authors Ishaan Kelkar, Vikram Kakaria, Nebras Alam, Madhur Panwar, Vasu Sharma, Maheep Chaudhary
Machine Learning · Cross-listed from math.PR
Concentration of General Stochastic Approximation Under Heavy-Tailed Markovian Noise
Shubhada Agrawal, Siva Theja Maguluri and Martin Zubeldia
We establish maximal concentration bounds for the iterates generated by
stochastic approximation
algorithms with general step sizes, where the noise has a finite-state Markovian component plus a Martingale-difference component. When the Martingale-difference noise is bounded, we show that the tail of the error can be sub-Gaussian, sub-Weibull, or something lighter than any Pareto but heavier than any Weibull, depending on the step size sequence and on whether the random operator is almost surely contractive, almost surely non-expansive, or expansive with positive probability. Our analysis relies on a novel Lyapunov function involving the moment-generating function of the solution to a Poisson equation, together with an auxiliary projected
algorithm. We complement the upper bounds with worst-case examples showing that qualitatively sharper bounds are impossible. We further study the case of unbounded Martingale-difference noise when the average operator is contractive, and the step sizes are of order $1/k$. In this setting, we show that if the random operator is almost surely non-expansive, then the error tail is at most three times heavier than the noise tail, whereas if the random operator is expansive with positive probability, then the error may have substantially heavier tails. These results are obtained through a novel black-box truncation argument that reduces the unbounded-noise setting to the bounded-noise case.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Hybrid Machine Learning Model for Forest Height Estimation from TanDEM-X and Landsat Data
Islam Mansour, Ronny Haensch, Irena Hajnsek et al. (+1 more)
Integrating machine learning (ML) with physical models (PM) has emerged as a promising way of retrieving geophysical parameters from remote sensing data. In this context, a ML model for estimating forest height from TanDEM-X interferometric coherence measurements has recently been proposed, that constrains the learning process through a PM. While the features used for training and inversion where selected to ensure the physical consistency of the solutions, they could not resolve all height / structure and baseline / terrain slope ambiguities in the data. To improve this, the extension of the feature space with optical Landsat data is proposed able to provide complementary information on forest type or structure. The extended model is applied and validated on several TanDEM-X acquisitions over the Gabonese Lopé national park site and assessed against airborne
LiDAR measurements. Results show a 13.5% reduction in RMSE and a 16.6% reduction in MAE compared to the original hybrid model, confirming the added value of multispectral inputs.
Read the full abstract ▾
All authors Islam Mansour, Ronny Haensch, Irena Hajnsek, Konstantinos Papathanassiou
Machine Learning
Beyond the Bellman Recursion: A Pontryagin-Guided Framework for Non-Exponential Discounting
Hojin Ko and Jeonggyu Huh
Most value-based and actor–critic reinforcement learning methods rely on Bellman-style recursions, yet these recursions collapse under non-exponential discounting common in human preferences and survival processes. We show the breakdown is structural: exponential discounting sits at a fragile intersection of multiplicativity and time homogeneity, and violating either property breaks standard dynamic programming. To overcome this, we propose Pontryagin-Guided Direct Policy Optimization (PG-DPO), a variational framework that abandons recursion and couples the Pontryagin Maximum Principle with Monte Carlo rollouts via an Adjoint-MC projection enforcing pointwise
Hamiltonian maximization. Across multi-dimensional
hyperbolic and survival-discount
benchmarks, PG-DPO improves accuracy and stability where equation-driven solvers and critic-based baselines diverge.
Read the full abstract ▾