Machine Learning
Interpretable long-term traffic modelling on national road networks using theory-informed deep learning
Yue Li, Shujuan Chen, Akihiro Shimoda et al. (+1 more)
Long-term traffic modelling is fundamental to transport planning, but existing approaches often trade off interpretability, transferability, and predictive accuracy. Classical travel demand models provide behavioural structure but rely on strong assumptions and extensive calibration, whereas generic deep learning models capture complex patterns but often lack theoretical grounding and spatial transferability, limiting their usefulness for long-term planning applications. We propose DeepDemand, a theory-informed deep learning framework that embeds key components of travel demand theory to predict long-term highway traffic volumes using external socioeconomic features and road-network structure. The framework integrates a competitive two-source Dijkstra procedure for local origin-destination (OD) region extraction and OD pair screening with a differentiable architecture modelling OD interactions and travel-time deterrence. The model is evaluated using eight years (2017-2024) of observations on the UK strategic road network, covering 5088 highway segments. Under random cross-validation, DeepDemand achieves an R2 of 0.718 and an MAE of 7406 vehicles, outperforming linear, ridge, random forest, and
gravity-style baselines. Performance remains strong under spatial cross-validation (R2 = 0.665), indicating good geographic transferability. Interpretability analysis reveals a stable nonlinear travel-time deterrence pattern, key socioeconomic drivers of demand, and polycentric OD interaction structures aligned with major employment centres and transport hubs. These results highlight the value of integrating transport theory with deep learning for interpretable highway traffic modelling and practical planning applications.
Read the full abstract ▾
All authors Yue Li, Shujuan Chen, Akihiro Shimoda, Ying Jin
Machine Learning · Cross-listed from quant-ph
Reconstructing Quantum Dot Charge Stability Diagrams with Diffusion Models
Vinicius Hernandes, Joseph Rogers, Rouven Koch et al. (+5 more)
Efficiently characterizing
quantum dot (QD) devices is a critical bottleneck when scaling
quantum processors based on confined
spins. Measuring high-resolution charge stability diagrams (or CSDs, data maps which crucially define the occupation of QDs) is time-consuming, particularly in emerging architectures where CSDs must be acquired with remote sensors that cannot probe the charge of the relevant dots directly. In this work, we present a generative approach to accelerate acquisition by reconstructing full CSDs from sparse measurements, using a conditional
diffusion model. We evaluate our approach using two experimentally motivated masking strategies: uniform grid-based sampling, and line-cut sweeps. Our lightweight architecture, trained on approximately 9,000 examples, successfully reconstructs CSDs, maintaining key physically important features such as charge transition lines, from as little as 4% of the total measured data. We compare the approach to interpolation methods, which fail when the task involves reconstructing large unmeasured regions. Our results demonstrate that generative models can significantly reduce the characterization overhead for
quantum devices, and provides a robust path towards an experimental implementation.
Read the full abstract ▾
All authors Vinicius Hernandes, Joseph Rogers, Rouven Koch, Thomas Spriggs, Brennan Undseth, Anasua Chatterjee, Lieven M. K. Vandersypen, Eliska Greplova
Machine Learning · Cross-listed from stat.ML
Kantorovich–Kernel Neural Operators: Approximation Theory, Asymptotics, and Neural Network Interpretation
Tian-Xiao He
This paper studies a class of multivariate Kantorovich-kernel neural network operators, including the deep Kantorovich-type neural network operators studied by Sharma and Singh. We prove density results, establish quantitative convergence estimates, derive Voronovskaya-type
theorems, analyze the limits of partial differential equations for deep composite operators, prove Korovkin-type
theorems, and propose inversion
theorems. This paper studies a class of multivariate Kantorovich-kernel neural network operators, including the deep Kantorovich-type neural network operators studied by Sharma and Singh. We prove density results, establish quantitative convergence estimates, derive Voronovskaya-type
theorems, analyze the limits of partial differential equations for deep composite operators, prove Korovkin-type
theorems, and propose inversion
theorems. Furthermore, this paper discusses the connection between neural network architectures and the classical positive operators proposed by Chui, Hsu, He, Lorentz, and Korovkin.
Read the full abstract ▾
Machine Learning
KMM-CP: Practical Conformal Prediction under Covariate Shift via Selective Kernel Mean Matching
Siddhartha Laghuvarapu, Rohan Deb and Jimeng Sun
Uncertainty quantification is essential for deploying machine learning models in high-stakes domains such as scientific discovery and healthcare. Conformal Prediction (CP) provides finite-sample coverage guarantees under exchangeability, an assumption often violated in practice due to distribution shift. Under
covariate shift, restoring validity requires importance weighting, yet accurate density-ratio estimation becomes unstable when training and test distributions exhibit limited support overlap. We propose KMM-CP, a conformal prediction framework based on Kernel Mean Matching (KMM) for
covariate-shift correction. We show that KMM directly controls the bias-variance components governing conformal coverage error by minimizing RKHS moment discrepancy under explicit weight constraints, and establish asymptotic coverage guarantees under mild conditions. We then introduce a selective extension that identifies regions of reliable support overlap and restricts conformal correction to this subset, further improving stability in low-overlap regimes. Experiments on molecular property prediction
benchmarks with realistic distribution shifts show that KMM-CP reduces coverage gap by over 50% compared to existing approaches. The code is available at https://github.com/siddharthal/KMM-CP.
Read the full abstract ▾
Machine Learning
Maintaining Difficulty: A Margin Scheduler for Triplet Loss in Siamese Networks Training
Roberto Sprengel Minozzo Tomchak, Oge Marques, Lucas Garcia Pedroso et al. (+2 more)
The Triplet Margin Ranking Loss is one of the most widely used loss functions in Siamese Networks for solving Distance Metric Learning (DML) problems. This loss function depends on a margin parameter {\mu}, which defines the minimum distance that should separate positive and negative pairs during training. In this work, we show that, during training, the effective margin of many triplets often exceeds the predefined value of {\mu}, provided that a sufficient number of triplets violating this margin is observed. This behavior indicates that fixing the margin throughout training may limit the learning process. Based on this observation, we propose a margin scheduler that adjusts the value of {\mu} according to the proportion of easy triplets observed at each epoch, with the goal of maintaining training difficulty over time. We show that the proposed strategy leads to improved performance when compared to both a constant margin and a monotonically increasing margin scheme. Experimental results on four different
datasets show consistent gains in verification performance.
Read the full abstract ▾
All authors Roberto Sprengel Minozzo Tomchak, Oge Marques, Lucas Garcia Pedroso, Luiz Eduardo Oliveira, Paulo Lisboa de Almeida
Machine Learning
Generative machine learning for protein design
Senura Hansaja Wanasekara, Manoj M Wagle, Minh-Duong Nguyen et al. (+4 more)
Designing
proteins with specific biological functions is of considerable interest for both the development of new therapeutics and the engineering of enzymes, yet achieving the intended activity remains challenging. Generative artificial intelligence (AI) has emerged as a powerful tool to explore the vast space of possible designs by learning from
protein sequence and structural data. Here, we review 59 generative approaches across six
protein design tasks, including sequence generation, backbone generation, structure-conditioned sequence design, joint sequence-structure design, motif scaffolding, and binder and antibody design. We outline common generative approaches and
protein representations and discuss how structural constraints and functional information are utilized during generation. This is followed by a discussion of the computational and experimental approaches used to evaluate designed
proteins and what such assessments reveal about their properties. Finally, we discuss current limitations in designing
proteins beyond well-represented families and incorporating experimental feedback into the design process. We expect that rapid advances in generative modeling will enable more reliable design of a wider range of
protein structures and functions over the coming years.
Read the full abstract ▾
All authors Senura Hansaja Wanasekara, Manoj M Wagle, Minh-Duong Nguyen, Xiaochen Liu, Nguyen H. Tran, Manolis Kellis, Ken-Tye Yong
Machine Learning
A Formal Framework for Uncertainty Analysis of Text Generation with Large Language Models
Steffen Herbold and Florian Lemmerich
The generation of texts using Large
Language Models (LLMs) is inherently uncertain, with sources of uncertainty being not only the generation of texts, but also the
prompt used and the downstream interpretation. Within this work, we provide a formal framework for the measurement of uncertainty that takes these different aspects into account. Our framework models prompting, generation, and interpretation as interconnected autoregressive processes that can be combined into a single sampling tree. We introduce filters and objective functions to describe how different aspects of uncertainty can be expressed over the sampling tree and demonstrate how to express existing approaches towards uncertainty through these functions. With our framework we show not only how different methods are formally related and can be reduced to a common core, but also point out additional aspects of uncertainty that have not yet been studied.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
DuSCN-FusionNet: An Interpretable Dual-Channel Structural Covariance Fusion Framework for ADHD Classification Using Structural MRI
Qurat Ul Ain, Alptekin Temizel and Soyiba Jawed
Attention Deficit Hyperactivity Disorder (ADHD) is a highly prevalent neurodevelopmental condition; however, its neurobiological diagnosis remains challenging due to the lack of reliable imaging-based biomarkers, particularly anatomical markers. Structural MRI (sMRI) provides a non-invasive modality for investigating
brain alterations associated with ADHD; nevertheless, most deep learning approaches function as black-box systems, limiting clinical trust and interpretability. In this work, we propose DuSCN-FusionNet, an interpretable sMRI-based framework for ADHD classification that leverages dual-channel Structural Covariance Networks (SCNs) to capture inter-regional morphological relationships. ROI-wise mean intensity and intra-regional variability descriptors are used to construct intensity-based and heterogeneity-based SCNs, which are processed through an SCN-CNN encoder. In parallel, auxiliary ROI-wise variability features and global statistical descriptors are integrated via late-stage fusion to enhance performance. The model is evaluated using stratified 10-fold cross-validation with a 5-seed ensemble strategy, achieving a mean balanced accuracy of 80.59% and an AUC of 0.778 on the Peking University site of the ADHD-200
dataset. DuSCN-FusionNet further achieves precision, recall, and F1-scores of 81.66%, 80.59%, and 80.27%, respectively. Moreover, Grad-CAM is adapted to the SCN domain to derive ROI-level importance scores, enabling the identification of structurally relevant
brain regions as potential biomarkers.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Generative Score Inference for Multimodal Data
Xinyu Tian and Xiaotong Shen
Accurate uncertainty quantification is crucial for making reliable decisions in various supervised learning scenarios, particularly when dealing with complex,
multimodal data such as images and text. Current approaches often face notable limitations, including rigid assumptions and limited generalizability, constraining their effectiveness across diverse supervised learning tasks. To overcome these limitations, we introduce Generative Score Inference (GSI), a flexible inference framework capable of constructing statistically valid and informative prediction and confidence sets across a wide range of
multimodal learning problems. GSI utilizes synthetic samples generated by deep generative models to approximate conditional score distributions, facilitating precise uncertainty quantification without imposing restrictive assumptions about the data or tasks. We empirically validate GSI's capabilities through two representative scenarios:
hallucination detection in large
language models and uncertainty estimation in image captioning. Our method achieves state-of-the-art performance in
hallucination detection and robust predictive uncertainty in image captioning, and its performance is positively influenced by the quality of the underlying generative model. These findings underscore the potential of GSI as a versatile inference framework, significantly enhancing uncertainty quantification and trustworthiness in
multimodal learning.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
A Power-Weighted Noncentral Complex Gaussian Distribution
Toru Nakashika
The complex Gaussian distribution has been widely used as a fundamental spectral and noise model in signal processing and communication. However, its Gaussian structure often limits its ability to represent the diverse amplitude characteristics observed in individual source signals. On the other hand, many existing non-Gaussian amplitude distributions derived from hyperspherical models achieve good empirical fit due to their power-law structures, while they do not explicitly account for the complex-plane geometry inherent in complex-valued observations. In this paper, we propose a new probabilistic model for complex-valued random variables, which can be interpreted as a power-weighted noncentral complex Gaussian distribution. Unlike conventional hyperspherical amplitude models, the proposed model is formulated directly on the complex plane and preserves the
geometric structure of complex-valued observations while retaining a higher-dimensional interpretation. The model introduces a nonlinear phase
diffusion through a single shape parameter, enabling continuous control of the distributional geometry from arc-shaped
diffusion along the phase direction to concentration of probability mass toward the origin. We formulate the proposed distribution and analyze the statistical properties of the induced amplitude distribution. The derived amplitude and power distributions provide a unified framework encompassing several widely used distributions in signal modeling, including the Rice, Nakagami, and gamma distributions. Experimental results on
speech power spectra demonstrate that the proposed model consistently outperforms conventional distributions in terms of log-likelihood.
Read the full abstract ▾
Machine Learning
Curvature-aware Expected Free Energy as an Acquisition Function for Bayesian Optimization
Ajith Anil Meera and Wouter Kouw
We propose an Expected Free Energy-based acquisition function for
Bayesian optimization to solve the joint learning and optimization problem, i.e., optimize and learn the underlying function simultaneously. We show that, under specific assumptions, Expected Free Energy reduces to Upper Confidence Bound, Lower Confidence Bound, and Expected Information Gain. We prove that Expected Free Energy has unbiased convergence guarantees for concave functions. Using the results from these derivations, we introduce a
curvature-aware update law for Expected Free Energy and show its proof of concept using a system identification problem on a Van der Pol oscillator. On a two-dimensional
benchmark with an oscillatory landscape, our adaptive Expected Free Energy acquisition achieves competitive performance in both regret and mean squared error, unlike the typical acquisition functions that perform well in only one metric.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ME
Making Multi-Axis Models Robust to Multiplicative Noise: How, and Why?
Bailey Andrew, David R. Westhead and Luisa Cutillo
In this paper we develop a graph-learning
algorithm, MED-MAGMA, to fit multi-axis (Kronecker-sum-structured) models corrupted by multiplicative noise. This type of noise is natural in many application domains, such as that of single-
cell RNA sequencing, in which it naturally captures technical biases of RNA sequencing platforms. Our work is evaluated against prior work on each and every public
dataset in the Single
Cell Expression Atlas under a certain size, demonstrating that our methodology learns networks with better local and global structure. MED-MAGMA is made available as a Python package (MED-MAGMA).
Read the full abstract ▾
Machine Learning · Cross-listed from q-fin.PR
STN-GPR: A Singularity Tensor Network Framework for Efficient Option Pricing
Dominic Gribben, Carolina Allende, Alba Villarino et al. (+5 more)
We develop a tensor-network surrogate for option pricing, targeting large-scale portfolio revaluation problems arising in
market risk management (e.g., VaR and Expected Shortfall computations). The method involves representing high-dimensional price surfaces in tensor-train (TT) form using TT-cross approximation, constructing the surrogate directly from black-box price evaluations without materializing the full training tensor. For inference, we use a
Laplacian kernel and derive TT representations of the kernel
matrix and its closed-form inverse in the noise-free setting, enabling TT-based Gaussian process regression without dense
matrix factorization or iterative linear solves. We found that hyperparameter optimization consistently favors a large kernel length-scale and show that in this regime the GPR predictor reduces to multilinear interpolation for off-grid inputs; we also derive a low-rank TT representation for this limit. We evaluate the approach on five-asset basket options over an eight dimensional parameter space (asset spot levels, strike, interest rate, and time to maturity). For European
geometric basket puts, the tensor surrogate achieves lower test error at shorter training times than standard GPR by scaling to substantially larger effective training sets. For American arithmetic basket puts trained on LSMC data, the surrogate exhibits more favorable scaling with training-set size while providing millisecond-level evaluation per query, with overall runtime dominated by data generation.
Read the full abstract ▾
All authors Dominic Gribben, Carolina Allende, Alba Villarino, Aser Cortines, Mazen Ali, Román Orús, Pascal Oswald, Noureddine Lehdili
Machine Learning · Cross-listed from cs.CV
Label-Free Cross-Task LoRA Merging with Null-Space Compression
Wonyoung Lee, Wooseong Jeong and Kuk-Jin Yoon
Model merging combines independently fine-tuned checkpoints without joint multi-task training. In the era of foundation-model, fine-tuning with Low-Rank Adaptation (LoRA) is prevalent, making LoRA merging a promising target. Existing approaches can work in homogeneous settings where all target tasks are classification but often fail when tasks span classification and regression. Approaches using
entropy-based surrogates do not apply to regression and are costly for large
language models due to long
token sequences. We introduce Null-Space Compression (NSC) Merging, a label-free, output-agnostic method that sets merge weights from adapter geometry. Our key observation is that during LoRA finetuning the down-projection factor $A$ in $\Delta W = BA$ compresses its null space, and the compression correlates with performance. NSC uses this as an optimization signal for merging that can generalize across classification, regression, and sequence generation. NSC achieves state-of-the-art performance across twenty heterogeneous
vision tasks with balanced gains where prior methods overfit subsets of tasks. It also outperforms baselines on six NLI
benchmarks and on
vision-
language evaluations for VQA and image captioning, demonstrating scalability and effectiveness.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
SALMUBench: A Benchmark for Sensitive Association-Level Multimodal Unlearning
Cai Selvas-Sala, Lei Kang and Lluis Gomez
As
multimodal models like CLIP become integral to downstream systems, the need to remove sensitive information is critical. However, machine unlearning for contrastively-trained encoders remains underexplored, and existing evaluations fail to diagnose fine-grained, association-level forgetting. We introduce SALMUBench (Sensitive Association-Level
Multimodal Unlearning), a
benchmark built upon a synthetic
dataset of 60K persona-attribute associations and two foundational models: a Compromised model polluted with this data, and a Clean model without it. To isolate unlearning effects, both are trained from scratch on the same 400M-pair retain base, with the Compromised model additionally trained on the sensitive set. We propose a novel evaluation protocol with structured holdout sets (holdout identity, holdout association) to precisely measure unlearning efficacy and collateral damage. Our
benchmark reveals that while utility-efficient deletion is feasible, current methods exhibit distinct failure modes: they either fail to forget effectively or over-generalize by erasing more than intended. SALMUBench sets a new standard for comprehensive unlearning evaluation, and we publicly release our
dataset, models, evaluation scripts, and leaderboards to foster future research.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.AP
Semi-structured multi-state delinquency model for mortgage default
Victor Medina-Olivares, Wangzhen Xia, Stefan Lessmann et al. (+1 more)
We propose a semi-structured discrete-time multi-state model to analyse mortgage delinquency transitions. This model combines an easy-to-understand structured additive predictor, which includes linear effects and smooth functions of time and
covariates, with a flexible neural network component that captures complex nonlinearities and higher-order interactions. To ensure identifiability when
covariates are present in both components, we orthogonalise the unstructured part relative to the structured design. For discrete-time competing transitions, we derive exact transformations that map binary logistic models to valid competing transition probabilities, avoiding the need for continuous-time approximations. In simulations, our framework effectively recovers structured baseline and
covariate effects while using the neural component to detect interaction patterns. We demonstrate the method using the Freddie Mac Single-Family Loan-Level
Dataset, employing an out-of-time test design. Compared with a structured generalised additive
benchmark, the semi-structured model provides modest but consistent gains in discrimination across the earliest prediction spans, while maintaining similar Brier scores. Adding macroeconomic indicators provides limited incremental benefit in this out-of-time evaluation and does not materially change the estimated borrower-, loan-, or duration-driven effects. Overall, semi-structured multi-state modelling offers a practical compromise between transparent effect estimates and flexible pattern learning, with potential applications beyond credit-transition
forecasting.
Read the full abstract ▾
All authors Victor Medina-Olivares, Wangzhen Xia, Stefan Lessmann, Nadja Klein
Machine Learning
D-GATNet: Interpretable Temporal Graph Attention Learning for ADHD Identification Using Dynamic Functional Connectivity
Qurat Ul Ain, Alptekin Temizel and Soyiba Jawed
Attention Deficit Hyperactivity Disorder (ADHD) is a prevalent neurodevelopmental disorder whose neuroimaging-based diagnosis remains challenging due to complex time-varying disruptions in
brain connectivity. Functional MRI (fMRI) provides a powerful non-invasive modality for identifying functional alterations. Existing deep learning (DL) studies employ diverse neuroimaging features; however, static functional connectivity remains widely used, whereas dynamic connectivity modeling is comparatively underexplored. Moreover, many DL models lack interpretability. In this work, we propose D-GATNet, an interpretable temporal graph-based framework for automated ADHD classification using dynamic functional connectivity (dFC). Sliding-window Pearson correlation constructs sequences of functional
brain graphs with regions of interest as nodes and connectivity strengths as edges. Spatial dependencies are learned via a multi-layer Graph
Attention Network, while temporal dynamics are modeled using 1D convolution followed by temporal
attention. Interpretability is achieved through graph
attention weights revealing dominant ROI interactions, ROI importance scores identifying influential regions, and temporal
attention emphasizing informative connectivity segments. Experiments on the Peking University site of the ADHD-200
dataset using stratified 10-fold cross-validation with a 5-seed ensemble achieved 85.18% +_5.64 balanced accuracy and 0.881 AUC, outperforming state-of-the-art methods.
Attention analysis reveals cerebellar and default mode network disruptions, indicating potential neuroimaging biomarkers.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Preference-Aligned LoRA Merging: Preserving Subspace Coverage and Addressing Directional Anisotropy
Wooseong Jeong, Wonyoung Lee and Kuk-Jin Yoon
Merging multiple Low-Rank Adaptation (LoRA) modules is promising for constructing general-purpose systems, yet challenging because LoRA update directions span different subspaces and contribute unevenly. When merged naively, such mismatches can weaken the directions most critical to certain task losses while overemphasizing relatively less important ones, ultimately reducing the model's ability to represent all tasks faithfully. We revisit this problem through two perspectives: subspace coverage, which captures how broadly LoRA directions cover diverse representational directions, and anisotropy, which reflects the imbalance of influence across those directions. We propose TARA-Merging (Task-Rank Anisotropy
Alignment), which aligns merging weights using a preference-weighted cross-
entropy pseudo-loss while preserving task-relevant LoRA subspaces. This ensures broad subspace coverage and mitigates anisotropy via direction-wise reweighting. Across eight
vision and six NLI
benchmarks, TARA-Merging consistently outperforms vanilla and LoRA-aware baselines, demonstrating strong robustness and generalization, and highlighting the importance of addressing both subspace coverage and anisotropy in LoRA merging.
Read the full abstract ▾
Machine Learning
Topology-Aware Graph Reinforcement Learning for Energy Storage Systems Optimal Dispatch in Distribution Networks
Shuyi Gao, Stavros Orfanoudakis, Shengren Hou et al. (+2 more)
Optimal dispatch of energy storage systems (ESSs) in distribution networks involves jointly improving operating economy and voltage security under time-varying conditions and possible
topology changes. To support fast online decision making, we develop a
topology-aware Reinforcement Learning architecture based on Twin Delayed Deep Deterministic Policy Gradient (TD3), which integrates graph neural networks (GNNs) as graph feature encoders for ESS dispatch. We conduct a systematic investigation of three GNN variants: graph convolutional networks (GCNs),
topology adaptive graph convolutional networks (TAGConv), and graph
attention networks (GATs) on the 34-bus and 69-bus systems, and evaluate robustness under multiple
topology reconfiguration cases as well as cross-system transfer between networks with different system sizes. Results show that GNN-based controllers consistently reduce the number and magnitude of voltage violations, with clearer benefits on the 69-bus system and under reconfiguration; on the 69-bus system, TD3-GCN and TD3-TAGConv also achieve lower saved cost relative to the NLP
benchmark than the NN baseline. We also highlight that transfer gains are case-dependent, and zero-shot transfer between fundamentally different systems results in notable performance degradation and increased voltage magnitude violations. This work is available at: https://github.com/ShuyiGao/GNNs_RL_ESSs and https://github.com/distributionnetworksTUDelft/GNNs_RL_ESSs.
Read the full abstract ▾
All authors Shuyi Gao, Stavros Orfanoudakis, Shengren Hou, Peter Palensky, Pedro P. Vergara
Machine Learning · Cross-listed from cs.CV
GLASS: Geometry-aware Local Alignment and Structure Synchronization Network for 2D-3D Registration
Zhixin Cheng, Jiacheng Deng, Xinjun Li et al. (+5 more)
Image-to-point cloud registration methods typically follow a coarse-to-fine pipeline, extracting patch-level correspondences and refining them into dense pixel-to-point matches. However, in scenes with repetitive patterns, images often lack sufficient 3D structural cues and
alignment with point clouds, leading to incorrect matches. Moreover, prior methods usually overlook structural consistency, limiting the full exploitation of correspondences. To address these issues, we propose two novel modules: the Local Geometry Enhancement (LGE) module and the Graph Distribution Consistency (GDC) module. LGE enhances both image and point cloud features with normal vectors, injecting
geometric structure into image features to reduce mismatches. GDC constructs a graph from matched points to update features and explicitly constrain similarity distributions. Extensive experiments and ablations on two
benchmarks, RGB-D Scenes v2 and 7-Scenes, demonstrate that our approach achieves state-of-the-art performance in image-to-point cloud registration.
Read the full abstract ▾
All authors Zhixin Cheng, Jiacheng Deng, Xinjun Li, Bohao Liao, Li Liu, Xiaotian Yin, Baoqun Yin, Tianzhu Zhang
Machine Learning
Contrastive Conformal Sets
Yahya Alkhatib and Wee Peng Tay
Contrastive learning produces coherent semantic feature
embeddings by encouraging positive samples to cluster closely while separating negative samples. However, existing contrastive learning methods lack a principled construction of
geometric sets in the semantic feature space with distribution-free guarantees at any user-specified coverage level. We extend conformal prediction to this setting by introducing covering sets equipped with learnable generalized hyper-ball constraints. We propose a method that constructs conformal sets guaranteeing user-specified coverage of positive samples while maximizing negative sample exclusion. We theoretically motivate volume minimization as a proxy for negative exclusion, enabling our approach to operate effectively even when negative pairs are unavailable. The positive inclusion guarantee inherits the distribution-free coverage property of conformal prediction, while negative exclusion is maximized through learned set geometry optimized on a held-out training split. Experiments on simulated and real-world image
datasets demonstrate improved inclusion-exclusion trade-offs compared to standard distance-based conformal baselines.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
ARTA: Adaptive Mixed-Resolution Token Allocation for Efficient Dense Feature Extraction
David Hagerman, Roman Naeem, Erik Brorsson et al. (+2 more)
We present ARTA, a mixed-resolution coarse-to-fine
vision transformer for efficient dense feature extraction. Unlike models that begin with dense high-resolution (fine)
tokens, ARTA starts with low-resolution (coarse)
tokens and uses a lightweight allocator to predict which regions require more fine
tokens. The allocator iteratively predicts a semantic (class) boundary score and allocates additional
tokens to patches above a low threshold, concentrating
token density near boundaries while maintaining high sensitivity to weak boundary evidence. This targeted allocation encourages
tokens to represent a single semantic class rather than a mixture of classes. Mixed-resolution
attention enables interaction between coarse and fine
tokens, focusing computation on semantically complex areas while avoiding redundant processing in homogeneous regions. Experiments demonstrate that ARTA achieves state-of-the-art results on ADE20K and COCO-Stuff with substantially fewer FLOPs, and delivers competitive performance on Cityscapes at markedly lower compute. For example, ARTA-Base attains 54.6 mIoU on ADE20K in the ~100M-parameter class while using fewer FLOPs and less
memory than comparable backbones.
Read the full abstract ▾
All authors David Hagerman, Roman Naeem, Erik Brorsson, Fredrik Kahl, Lennart Svensson
Machine Learning
Improving Risk Stratification in Hypertrophic Cardiomyopathy: A Novel Score Combining Echocardiography, Clinical, and Medication Data
Marion Taconné, Valentina D.A. Corino, Annamaria Del Franco et al. (+6 more)
Hypertrophic cardiomyopathy (HCM) requires accurate risk stratification to inform decisions regarding ICD therapy and follow-up management. Current established models, such as the European Society of Cardiology (ESC) score, exhibit moderate discriminative performance. This study develops a robust, explainable machine learning (ML) risk score leveraging routinely collected echocardiographic, clinical, and medication data, typically contained within Electronic Health Records (EHRs), to predict a 5-year composite cardiovascular outcome in HCM patients. The model was trained and internally validated using a large cohort (N=1,201) from the SHARE registry (Florence Hospital) and externally validated on an independent cohort (N=382) from Rennes Hospital. The final Random Forest ensemble model achieved a high internal Area Under the Curve (AUC) of 0.85 +- 0.02, significantly outperforming the ESC score (0.56 +- 0.03). Critically, survival curve analysis on the external validation set showed superior risk separation for the ML score (Log-rank p = 8.62 x 10^(-4) compared to the ESC score (p = 0.0559). Furthermore, longitudinal analyses demonstrate that the proposed risk score remains stable over time in event-free patients. The model high interpretability and its capacity for longitudinal risk monitoring represent promising tools for the personalized clinical management of HCM.
Read the full abstract ▾
All authors Marion Taconné, Valentina D.A. Corino, Annamaria Del Franco, Sara Giovani, Iacopo Olivotto, Adrien Al Wazzan, Erwan Donal, Pietro Cerveri, Luca Mainardi
Machine Learning
Knowledge Distillation for Efficient Transformer-Based Reinforcement Learning in Hardware-Constrained Energy Management Systems
Pascal Henrich, Jonas Sievers, Maximilian Beichter et al. (+3 more)
Transformer-based reinforcement learning has emerged as a strong candidate for sequential control in residential energy management. In particular, the Decision
Transformer can learn effective battery dispatch policies from historical data, thereby increasing photovoltaic self-consumption and reducing electricity costs. However,
transformer models are typically too computationally demanding for deployment on resource-constrained residential controllers, where
memory and latency constraints are critical. This paper investigates knowledge
distillation to transfer the decision-making behaviour of high-capacity Decision
Transformer policies to compact models that are more suitable for embedded deployment. Using the Ausgrid
dataset, we train teacher models in an offline sequence-based Decision
Transformer framework on heterogeneous multi-building data. We then distil smaller student models by matching the teachers' actions, thereby preserving control quality while reducing model size. Across a broad set of teacher-student configurations,
distillation largely preserves control performance and even yields small improvements of up to 1%, while reducing the parameter count by up to 96%, the inference
memory by up to 90%, and the inference time by up to 63%. Beyond these compression effects, comparable cost improvements are also observed when distilling into a student model of identical architectural capacity. Overall, our results show that knowledge
distillation makes Decision
Transformer control more applicable for residential energy management on resource-limited hardware.
Read the full abstract ▾
All authors Pascal Henrich, Jonas Sievers, Maximilian Beichter, Thomas Blank, Ralf Mikut, Veit Hagenmeyer
Machine Learning · Cross-listed from cs.CL
Abstract Compression of Conversational Audio Context for LLM-based ASR
Shashi Kumar, Esaú Villatoro-Tello, Sergio Burdisso et al. (+7 more)
Standard LLM-based
speech recognition systems process utterances in isolation, limiting their ability to leverage conversational context. Here we study whether
multimodal context from prior turns improves LLM-based ASR and how to represent that context efficiently. We find that, after supervised multi-turn training, conversational context mainly helps with the recognition of contextual entities. However, conditioning on raw context is expensive because the prior-turn audio
embedding sequence grows rapidly with conversation length. To address this, we propose Abstract Compression, a learned latent abstraction of prior-turn audio that replaces each prior-turn audio sequence with a fixed number of latent
tokens while retaining corresponding transcripts explicitly. On both in-domain and out-of-domain test sets, the compression-enabled model recovers part of the gains of raw-context conditioning with a smaller prior-turn audio footprint. We also provide targeted analyses of the compression setup and its tradeoffs.
Read the full abstract ▾
All authors Shashi Kumar, Esaú Villatoro-Tello, Sergio Burdisso, Kadri Hacioglu, Thibault Bañeras-Roux, Hasindri Watawana, Dairazalia Sanchez-Cortes, Srikanth Madikeri, Petr Motlicek, Andreas Stolcke
Machine Learning
Optimization Trade-offs in Asynchronous Federated Learning: A Stochastic Networks Approach
Abdelkrim Alahyane, Céline Comte and Matthieu Jonckheere
Synchronous
federated learning scales poorly due to the straggler effect. Asynchronous
algorithms increase the update throughput by processing updates upon arrival, but they introduce two fundamental challenges: gradient staleness, which degrades convergence, and bias toward faster clients under heterogeneous data distributions. Although
algorithms such as AsyncSGD and Generalized AsyncSGD mitigate this bias via client-side task queues, most existing analyses neglect the underlying queueing dynamics and lack closed-form characterizations of the update throughput and gradient staleness. To close this gap, we develop a
stochastic queueing-network framework for Generalized AsyncSGD that jointly models random computation times at the clients and the central server, as well as random uplink and downlink communication delays. Leveraging product-form network theory, we derive a closed-form expression for the update throughput, alongside closed-form upper bounds for both the communication round complexity and the expected wall-clock time required to reach an $\epsilon$-stationary point. These results formally characterize the trade-off between gradient staleness and wall-clock convergence speed. We further extend the framework to quantify energy consumption under
stochastic timing, revealing an additional trade-off between convergence speed and energy efficiency. Building on these analytical results, we propose gradient-based optimization strategies to jointly optimize routing and concurrency. Experiments on EMNIST demonstrate reductions of 29%–46% in convergence time and 36%–49% in energy consumption compared to AsyncSGD.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Privacy-Accuracy Trade-offs in High-Dimensional LASSO under Perturbation Mechanisms
Ayaka Sakata and Haruka Tanzawa
We study
privacy-preserving sparse linear regression in the high-dimensional regime, focusing on the LASSO
estimator. We analyze two widely used mechanisms for differential
privacy: output perturbation, which injects noise into the
estimator, and objective perturbation, which adds a random linear term to the loss function. Using approximate message passing (AMP), we characterize the typical behavior of these
estimators under random design and
privacy noise. To quantify
privacy, we adopt typical-case measures, including the on-average KL divergence, which admits a hypothesis-testing interpretation in terms of distinguishability between neighboring
datasets. Our analysis reveals that sparsity plays a central role in shaping the
privacy-accuracy trade-off: stronger regularization can improve
privacy by stabilizing the
estimator against single-point data changes. We further show that the two mechanisms exhibit qualitatively different behaviors. In particular, for objective perturbation, increasing the noise level can have non-monotonic effects, and excessive noise may destabilize the
estimator, leading to increased sensitivity to data perturbations. Our results demonstrate that AMP provides a powerful framework for analyzing
privacy-accuracy trade-offs in high-dimensional sparse models.
Read the full abstract ▾
Machine Learning · Cross-listed from math.PR
On associative neural networks for sparse patterns with huge capacities
Matthias Löwe and Franck Vermet
Generalized Hopfield models with higher-order or exponential interaction terms are known to have substantially larger storage capacities than the classical quadratic model. On the other hand, associative
memories for sparse patterns, such as the Willshaw and Amari models, already exhibit enhanced storage capacities in the sparse regime. In this paper we combine these two mechanisms. We introduce higher-order versions of sparse associative
memory models and study their storage capacities in the sense of fixed-pattern stability. For the Amari and Willshaw models with fixed interaction order $n$, we obtain storage scales of order $\frac{N^n}{(\log N)^n}$. When the interaction order grows logarithmically with the number of
neurons, the resulting storage scale becomes super-
polynomial. We also study higher-order interactions in the block-structured Gripon–Berrou architecture, where the natural storage scale is of order $c^n$. Our results show that the capacity increase caused by higher-order interactions persists in the sparse setting, while the precise storage scale depends on the underlying architecture.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Dual-Stage Invariant Continual Learning under Extreme Visual Sparsity
Rangya Zhang, Jiaping Xiao, Lu Bai et al. (+2 more)
Continual learning seeks to maintain stable adaptation under non-stationary environments, yet this problem becomes particularly challenging in object detection, where most existing methods implicitly assume relatively balanced visual conditions. In extreme-sparsity regimes, such as those observed in space-based resident space object (RSO) detection scenarios, foreground signals are overwhelmingly dominated by background observations. Under such conditions, we analytically demonstrate that background-driven gradients destabilize the feature backbone during sequential domain shifts, causing progressive representation drift. This exposes a structural limitation of continual learning approaches relying solely on output-level
distillation, as they fail to preserve intermediate representation stability. To address this, we propose a dual-stage invariant continual learning framework via joint
distillation, enforcing structural and semantic consistency on both backbone representations and detection predictions, respectively, thereby suppressing error propagation at its source while maintaining adaptability. Furthermore, to regulate gradient statistics under severe imbalance, we introduce a sparsity-aware data conditioning strategy combining patch-based sampling and distribution-aware augmentation. Experiments on a high-resolution space-based RSO detection
dataset show consistent improvement over established continual object detection methods, achieving an absolute gain of +4.0 mAP under sequential domain shifts.
Read the full abstract ▾
All authors Rangya Zhang, Jiaping Xiao, Lu Bai, Yuhang Zhang, Mir Feroskhan
Machine Learning
Geometric Evolution Graph Convolutional Networks: Enhancing Graph Representation Learning via Ricci Flow
Jicheng Ma, Yunyan Yang, Juan Zhao et al. (+1 more)
We introduce the
Geometric Evolution Graph Convolutional Network (GEGCN), a novel framework that enhances graph representation learning through explicit modeling of
geometric evolution on graph structures. Specifically, GEGCN leverages a Long Short-Term
Memory (LSTM) network to capture the dynamic structural sequence generated by discrete
Ricci flow, and infuses the learned dynamic representations into a graph convolutional network. Extensive experiments demonstrate that GEGCN achieves excellent performance on classification tasks across various
benchmark datasets, including homophilic/heterophilic graphs, filtered graphs, and large-scale graphs.
Read the full abstract ▾
All authors Jicheng Ma, Yunyan Yang, Juan Zhao, Liang Zhao