Machine Learning · Cross-listed from cs.CR
GShield: Mitigating Poisoning Attacks in Federated Learning
Sameera K. M., Serena Nicolazzo, Antonino Nocera et al. (+2 more)
Federated Learning (FL) has recently emerged as a revolutionary approach to collaborative training Machine Learning models. In particular, it enables decentralized model training while preserving data
privacy, but its distributed nature makes it highly vulnerable to a severe attack known as Data Poisoning. In such scenarios, malicious clients inject manipulated data into the training process, thereby degrading global model performance or causing targeted misclassification. In this paper, we present a novel defense mechanism called GShield, designed to detect and mitigate malicious and low-quality updates, especially under non-independent and identically distributed (non-IID) data scenarios. GShield operates by learning the distribution of benign gradients through clustering and Gaussian modeling during an initial round, enabling it to establish a reliable baseline of trusted client behavior. With this benign profile, GShield selectively aggregates only those updates that align with the expected gradient patterns, effectively isolating
adversarial clients and preserving the integrity of the global model. An extensive experimental campaign demonstrates that our proposed defense significantly improves model robustness compared to the state-of-the-art methods while maintaining a high accuracy of performance across both tabular and image
datasets. Furthermore, GShield improves the accuracy of the targeted class by 43% to 65% after detecting malicious and low-quality clients.
Read the full abstract ▾
All authors Sameera K. M., Serena Nicolazzo, Antonino Nocera, Vinod P., Rafidha Rehiman K. A
Machine Learning
Digital Twin-Driven Zero-Shot Fault Diagnosis of Axial Piston Pumps Using Fluid-Borne Noise Signals
Chang Dong, Jianfeng Tao and Chengliang Liu
Axial piston pumps are crucial components in fluid power systems, where reliable fault diagnosis is essential for ensuring operational safety and efficiency. Traditional data-driven methods require extensive labeled fault data, which is often impractical to obtain, while model-based approaches suffer from parameter uncertainties. This paper proposes a digital twin (DT)-driven zero-shot fault diagnosis framework utilizing fluid-borne noise (FBN) signals. The framework calibrates a high-fidelity DT model using only healthy-state data, generates synthetic fault signals for training deep learning classifiers, and employs a physics-informed neural network (PINN) as a virtual sensor for flow ripple estimation. Gradient-weighted class activation mapping (Grad-CAM) is integrated to visualize the decision-making process of neural networks, revealing that large kernels matching the subsequence length in time-domain inputs and small kernels in time-frequency domain inputs enable higher diagnostic accuracy by focusing on physically meaningful features. Experimental validations demonstrate that training on signals from the calibrated DT model yields diagnostic accuracies exceeding 95% on real-world
benchmarks, while uncalibrated models result in significantly lower performance, highlighting the framework's effectiveness in data-scarce scenarios.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.RO
Translating Flow to Policy via Hindsight Online Imitation
Yitian Zheng, Zhangchen Ye, Weijun Dong et al. (+5 more)
Recent advances in hierarchical
robot systems leverage a high-level planner to propose task plans and a low-level policy to generate
robot actions. This design allows training the planner on action-free or even non-
robot data sources (e.g.,
videos), providing transferable high-level guidance. Nevertheless, grounding these high-level plans into executable actions remains challenging, especially with the limited availability of high-quality
robot data. To this end, we propose to improve the low-level policy through online interactions. Specifically, our approach collects online rollouts, retrospectively annotates the corresponding high-level goals from achieved outcomes, and aggregates these hindsight-relabeled experiences to update a goal-conditioned imitation policy. Our method, Hindsight Flow-conditioned Online Imitation (HinFlow), instantiates this idea with 2D point flows as the high-level planner. Across diverse manipulation tasks in both simulation and physical world, our method achieves more than $2\times$ performance improvement over the base policy, significantly outperforming the existing methods. Moreover, our framework enables policy acquisition from planners trained on cross-embodiment
video data, demonstrating its potential for scalable and transferable
robot learning.
Read the full abstract ▾
All authors Yitian Zheng, Zhangchen Ye, Weijun Dong, Shengjie Wang, Yuyang Liu, Chongjie Zhang, Chuan Wen, Yang Gao
Machine Learning
Machine Unlearning in the Era of Quantum Machine Learning: An Empirical Study
Carla Crivoi and Radu Tudor Ionescu
We present the first empirical study of machine unlearning (MU) in hybrid
quantum-classical neural networks. While MU has been extensively explored in classical deep learning, its behavior within variational
quantum circuits (VQCs) and
quantum-augmented architectures remains largely unexplored. First, we adapt a broad suite of unlearning methods to
quantum settings, including gradient-based,
distillation-based, regularization-based and certified techniques. Second, we introduce two new unlearning strategies tailored to hybrid models. Experiments across Iris, MNIST, and Fashion-MNIST, under both subset removal and full-class deletion, reveal that
quantum models can support effective unlearning, but outcomes depend strongly on
circuit depth,
entanglement structure, and task complexity. Shallow VQCs display high intrinsic stability with minimal memorization, whereas deeper hybrid models exhibit stronger trade-offs between utility, forgetting strength, and
alignment with retrain oracle. We find that certain methods, e.g. EU-k, LCA, and Certified Unlearning, consistently provide the best balance across metrics. These findings establish baseline empirical insights into
quantum machine unlearning and highlight the need for
quantum-aware
algorithms and theoretical guarantees, as
quantum machine learning systems continue to expand in scale and capability. We publicly release our code at: https://github.com/CrivoiCarla/HQML.
Read the full abstract ▾
Machine Learning
Small Language Models as Compiler Experts: Auto-Parallelization for Heterogeneous Systems
Prathamesh Devadiga
Traditional auto-parallelizing compilers, reliant on rigid heuristics, struggle with the complexity of modern heterogeneous systems. This paper presents a comprehensive evaluation of small (approximately 1B parameter)
language-model-driven compiler auto-parallelization. We evaluate three models: gemma3, llama3.2, and qwen2.5, using six
reasoning strategies across 11 real-world kernels drawn from scientific computing, graph
algorithms, and machine learning. Our system is benchmarked against strong compiler baselines, including LLVM Polly, TVM, and Triton. Across 376 total evaluations, the proposed approach achieves an average speedup of 6.81x and a peak performance of 43.25x on convolution operations. We analyze scalability, verify correctness using multiple sanitizers, and confirm robustness across diverse compilers and hardware platforms. Our results demonstrate that small, efficient
language models can serve as powerful
reasoning engines for complex compiler optimization tasks.
Read the full abstract ▾
Machine Learning
From Black-Box Tuning to Guided Optimization via Hyperparameters Interaction Analysis
Moncef Garouani and Ayah Barhrhouj
Hyperparameters tuning is a fundamental, yet computationally expensive, step in optimizing machine learning models. Beyond optimization, understanding the relative importance and interaction of hyperparameters is critical to efficient model development. In this paper, we introduce MetaSHAP, a scalable semi-automated eXplainable AI (XAI) method, that uses meta-learning and Shapley values analysis to provide actionable and
dataset-aware tuning insights. MetaSHAP operates over a vast
benchmark of over 09 millions evaluated machine learning pipelines, allowing it to produce interpretable importance scores and actionable tuning insights that reveal how much each hyperparameter matters, how it interacts with others and in which value ranges its influence is concentrated. For a given
algorithm and
dataset, MetaSHAP learns a surrogate performance model from historical configurations, computes hyperparameters interactions using SHAP-based analysis, and derives interpretable tuning ranges from the most influential hyperparameters. This allows practitioners not only to prioritize which hyperparameters to tune, but also to understand their directionality and interactions. We empirically validate MetaSHAP on a diverse
benchmark of 164 classification
datasets and 14 classifiers, demonstrating that it produces reliable importance rankings and competitive performance when used to guide
Bayesian optimization.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Identifying Features Associated with Bias Against 93 Stigmatized Groups in Language Models and Guardrail Model Safety Mitigation
Anna-Maria Gueorguieva and Aylin Caliskan
Large
language models (LLMs) have been shown to exhibit social bias, however, bias towards non-protected stigmatized identities remain understudied. Furthermore, what social features of stigmas are associated with bias in LLM outputs is unknown. From psychology literature, it has been shown that stigmas contain six shared social features: aesthetics, concealability, course, disruptiveness, origin, and peril. In this study, we investigate if human and LLM ratings of the features of stigmas, along with
prompt style and type of stigma, have effect on bias towards stigmatized groups in LLM outputs. We measure bias against 93 stigmatized groups across three widely used LLMs (Granite 3.0-8B, Llama-3.1-8B, Mistral-7B) using SocialStigmaQA, a
benchmark that includes 37 social scenarios about stigmatized identities; for example deciding wether to recommend them for an internship. We find that stigmas rated by humans to be highly perilous (e.g., being a gang member or having HIV) have the most biased outputs from SocialStigmaQA
prompts (60% of outputs from all models) while sociodemographic stigmas (e.g. Asian-American or old age) have the least amount of biased outputs (11%). We test if the amount of biased outputs could be decreased by using guardrail models, models meant to identify harmful input, using each LLM's respective guardrail model (Granite Guardian 3.0, Llama Guard 3.0, Mistral Moderation API). We find that bias decreases significantly by 10.4%, 1.4%, and 7.8%, respectively. However, we show that features with significant effect on bias remain unchanged post-mitigation and that guardrail models often fail to recognize the intent of bias in
prompts. This work has implications for using LLMs in scenarios involving stigmatized groups and we suggest future work towards improving guardrail models for bias mitigation.
Read the full abstract ▾
Machine Learning
Regression generation adversarial network based on dual data evaluation strategy for industrial application
Zesen Wang, Yonggang Li and Lijuan Lan
Soft sensing infers hard-to-measure data through a large number of easily obtainable variables. However, in complex industrial scenarios, the issue of insufficient data volume persists, which diminishes the reliability of soft sensing. Generative
Adversarial Networks (GAN) are one of the effective solutions for addressing insufficient samples. Nevertheless, traditional GAN fail to account for the mapping relationship between labels and features, which limits further performance improvement. Although some studies have proposed solutions, none have considered both performance and efficiency simultaneously. To address these problems, this paper proposes the multi-task learning-based regression GAN framework that integrates regression information into both the discriminator and generator, and implements a shallow sharing mechanism between the discriminator and regressor. This approach significantly enhances the quality of generated samples while improving the
algorithm's operational efficiency. Moreover, considering the importance of training samples and generated samples, a dual data evaluation strategy is designed to make GAN generate more diverse samples, thereby increasing the generalization of subsequent modeling. The superiority of method is validated through four classic industrial soft sensing cases: wastewater treatment plants, surface water, $CO_2$ absorption towers, and industrial gas turbines.
Read the full abstract ▾
Machine Learning
Phase-space entropy at acquisition reflects downstream learnability
Xiu-Cheng Wang, Jun-Jie Zhanga, Nan Cheng et al. (+3 more)
Modern learning systems work with data that vary widely across domains, but they all ultimately depend on how much structure is already present in the measurements before any model is trained. This raises a basic question: is there a general, modality-agnostic way to quantify how acquisition itself preserves or destroys the information that downstream learners could use? Here we propose an acquisition-level scalar $\Delta S_{\mathcal B}$ based on instrument-resolved phase space. Unlike pixelwise distortion or purely spectral errors that often saturate under aggressive undersampling, $\Delta S_{\mathcal B}$ directly quantifies how acquisition mixes or removes joint space–frequency structure at the instrument scale. We show theoretically that \(\Delta S_{\mathcal B}\) correctly identifies the phase-space coherence of periodic sampling as the physical source of aliasing, recovering classical sampling-
theorem consequences. Empirically, across masked image classification, accelerated MRI, and massive
MIMO (including over-the-air measurements), $|\Delta S_{\mathcal B}|$ consistently ranks sampling geometries and predicts downstream reconstruction/recognition difficulty without training. In particular, minimizing $|\Delta S_{\mathcal B}|$ enables zero-training selection of variable-density MRI mask parameters that matches designs tuned by conventional pre-reconstruction criteria. These results suggest that phase-space
entropy at acquisition reflects downstream learnability, enabling pre-training selection of candidate sampling policies and as a shared notion of information preservation across modalities.
Read the full abstract ▾
All authors Xiu-Cheng Wang, Jun-Jie Zhanga, Nan Cheng, Long-Gang Pang, Taijiao Du, Deyu Meng
Machine Learning
MixKVQ: Query-Aware Mixed-Precision KV Cache Quantization for Long-Context Reasoning
Tao Zhang, Ziqian Zeng, Hao Peng et al. (+2 more)
Long Chain-of-Thought (CoT)
reasoning has significantly advanced the capabilities of Large
Language Models (LLMs), but this progress is accompanied by substantial
memory and latency overhead from the extensive Key-Value (KV) cache. Although KV cache quantization is a promising compression technique, existing low-bit quantization methods often exhibit severe performance degradation on complex
reasoning tasks. Fixed-precision quantization struggles to handle outlier channels in the key cache, while current mixed-precision strategies fail to accurately identify components requiring high-precision representation. We find that an effective low-bit KV cache quantization strategy must consider two factors: a key channel's intrinsic quantization difficulty and its relevance to the query. Based on this insight, we propose MixKVQ, a novel plug-and-play method that introduces a lightweight, query-aware
algorithm to identify and preserve critical key channels that need higher precision, while applying per-
token quantization for value cache. Experiments on complex
reasoning datasets demonstrate that our approach significantly outperforms existing low-bit methods, achieving performance comparable to a full-precision baseline at a substantially reduced
memory footprint.
Read the full abstract ▾
All authors Tao Zhang, Ziqian Zeng, Hao Peng, Huiping Zhuang, Cen Chen
Machine Learning
On the Koopman-Based Generalization Bounds for Multi-Task Deep Learning
Mahdi Mohammadigohari, Giuseppe Di Fatta, Giuseppe Nicosia et al. (+1 more)
The paper establishes generalization bounds for multitask deep neural networks using operator-theoretic techniques. The authors propose a tighter bound than those derived from conventional norm based methods by leveraging small condition numbers in the weight
matrices and introducing a tailored Sobolev space as an expanded hypothesis space. This enhanced bound remains valid even in single output settings, outperforming existing Koopman based bounds. The resulting framework maintains key advantages such as flexibility and independence from network width, offering a more precise theoretical understanding of multitask deep learning in the context of kernel methods.
Read the full abstract ▾
All authors Mahdi Mohammadigohari, Giuseppe Di Fatta, Giuseppe Nicosia, Panos M. Pardalos
Machine Learning · Cross-listed from physics.comp-ph
Adaptive Probability Flow Residual Minimization for High-Dimensional Fokker-Planck Equations
Xiaolong Wu and Qifeng Liao
Solving high-dimensional Fokker-Planck (FP) equations remains a challenging problem in computational physics and
stochastic dynamics, due to the curse of dimensionality, unbounded domains, and complex probability landscapes. In this work, we propose an adaptive probability flow residual minimization (A-PFRM) method for this problem. The second-order FP equation is reformulated as an equivalent first-order continuity equation associated with the probability flow ordinary differential equation, based on which a loss function is constructed to train neural network approximations without Hessian computation. To further improve the computational efficiency, the Hutchinson trace
estimator is applied to compute the divergence in the corresponding score function, such that the training time can be dimension-independent on GPUs. Adaptive sampling strategies are employed to generate the collocation points, and our analysis shows that the Kullback-Leibler divergence between our A-PFRM approximation and the exact solution is bounded by the residual loss weighted by the estimated density function. Numerical experiments are presented to demonstrate the performance of A-PFRM, which include Ornstein-Uhlenbeck (OU) processes problems, Brownian motions with time-varying
diffusion, and
Geometric OU processes featuring non-Gaussian solutions up to one hundred dimensions.
Read the full abstract ▾
Machine Learning
Causal Heterogeneous Graph Learning Method for Chronic Obstructive Pulmonary Disease Prediction
Leming Zhou, Zuo Wang and Zhigang Liu
Due to the insufficient diagnosis and treatment capabilities at the grassroots level, there are still deficiencies in the early identification and early warning of acute exacerbation of Chronic obstructive pulmonary disease (COPD), often resulting in a high prevalence rate and high burden, but the screening rate is relatively low. In order to gradually improve this situation. In this paper, this study develop a Causal Heterogeneous Graph Representation Learning (CHGRL) method for COPD comorbidity risk prediction method that: a) constructing a heterogeneous Our
dataset includes the interaction between patients and diseases; b) A cause-aware heterogeneous graph learning architecture has been constructed, combining causal inference mechanisms with heterogeneous graph learning, which can support heterogeneous graph causal learning for different types of relationships; and c) Incorporate the causal loss function in the model design, and add counterfactual
reasoning learning loss and causal regularization loss on the basis of the cross-
entropy classification loss. We evaluate our method and compare its performance with strong GNN baselines. Following experimental evaluation, the proposed model demonstrates high detection accuracy.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
PEDESTRIAN: An Egocentric Vision Dataset for Obstacle Detection on Pavements
Marios Thoma, Zenonas Theodosiou, Harris Partaourides et al. (+3 more)
Walking has always been a primary mode of transportation and is recognized as an essential activity for maintaining good health. Despite the need for safe walking conditions in urban environments, sidewalks are frequently obstructed by various obstacles that hinder free pedestrian movement. Any object obstructing a pedestrian's path can pose a safety hazard. The advancement of pervasive computing and
egocentric vision techniques offers the potential to design systems that can automatically detect such obstacles in real time, thereby enhancing pedestrian safety. The development of effective and efficient identification
algorithms relies on the availability of comprehensive and well-balanced
datasets of
egocentric data. In this work, we introduce the PEDESTRIAN
dataset, comprising
egocentric data for 29 different obstacles commonly found on urban sidewalks. A total of 340
videos were collected using mobile phone cameras, capturing a pedestrian's point of view. Additionally, we present the results of a series of experiments that involved training several state-of-the-art deep learning
algorithms using the proposed
dataset, which can be used as a
benchmark for obstacle detection and recognition tasks. The
dataset can be used for training pavement obstacle detectors to enhance the safety of pedestrians in urban areas.
Read the full abstract ▾
All authors Marios Thoma, Zenonas Theodosiou, Harris Partaourides, Vassilis Vassiliades, Loizos Michael, Andreas Lanitis
Machine Learning
Operator-Based Generalization Bound for Deep Learning: Insights on Multi-Task Learning
Mahdi Mohammadigohari, Giuseppe Di Fatta, Giuseppe Nicosia et al. (+1 more)
This paper presents novel generalization bounds for vector-valued neural networks and deep kernel methods, focusing on multi-task learning through an operator-theoretic framework. Our key development lies in strategically combining a Koopman based approach with existing techniques, achieving tighter generalization guarantees compared to traditional norm-based bounds. To mitigate computational challenges associated with Koopman-based methods, we introduce sketching techniques applicable to vector valued neural networks. These techniques yield excess risk bounds under generic Lipschitz losses, providing performance guarantees for applications including robust and multiple
quantile regression. Furthermore, we propose a novel deep learning framework, deep vector-valued reproducing kernel
Hilbert spaces (vvRKHS), leveraging Perron Frobenius (PF) operators to enhance deep kernel methods. We derive a new Rademacher generalization bound for this framework, explicitly addressing underfitting and overfitting through kernel refinement strategies. This work offers novel insights into the generalization properties of multitask learning with deep learning architectures, an area that has been relatively unexplored until recent developments.
Read the full abstract ▾
All authors Mahdi Mohammadigohari, Giuseppe Di Fatta, Giuseppe Nicosia, Panos M. Pardalos
Machine Learning
Practical Quantum-Classical Feature Fusion for complex data Classification
Azadeh Alavi, Fatemeh Kouchmeshki and Abdolrahman Alavi
Hybrid
quantum and classical learning aims to couple
quantum feature maps with the robustness of classical neural networks, yet most architectures treat the
quantum circuit as an isolated feature extractor and merge its measurements with classical representations by direct concatenation. This neglects that the
quantum and classical branches constitute distinct computational modalities and limits reliable performance on complex, high dimensional tabular and semi structured data, including remote sensing, environmental monitoring, and medical diagnostics. We present a
multimodal formulation of hybrid learning and propose a cross
attention mid fusion architecture in which a classical representation queries
quantum derived feature
tokens through an
attention block with residual connectivity. The
quantum branch is kept within practical NISQ budgets and uses up to nine
qubits. We evaluate on Wine, Breast Cancer, Forest CoverType, FashionMNIST, and SteelPlatesFaults, comparing a
quantum only model, a classical baseline, residual hybrid models, and the proposed mid fusion model under a consistent protocol. Pure
quantum and standard hybrid designs underperform due to measurement induced information loss, while cross
attention mid fusion is consistently competitive and improves performance on the more complex
datasets in most cases. These findings suggest that
quantum derived information becomes most valuable when integrated through principled
multimodal fusion rather than used in isolation or loosely appended to classical features.
Read the full abstract ▾
Machine Learning · Cross-listed from math.OC
Finite-sample guarantees for data-driven forward-backward operator methods
Filippo Fabiani and Barbara Franci
We establish finite sample certificates on the quality of solutions produced by data-based forward-backward (FB) operator splitting schemes. As frequently happens in
stochastic regimes, we consider the problem of finding a zero of the sum of two operators, where one is either unavailable in closed form or computationally expensive to evaluate, and shall therefore be approximated using a finite number of noisy oracle samples. Under the lens of algorithmic stability, we then derive probabilistic bounds on the distance between a true zero and the FB output without making specific assumptions about the underlying data distribution. We show that under weaker conditions ensuring the convergence of FB schemes, stability bounds grow proportionally to the number of iterations. Conversely, stronger assumptions yield stability guarantees that are independent of the iteration count. We then specialize our results to a popular FB
stochastic Nash
equilibrium seeking
algorithm and validate our theoretical bounds on a control problem for smart grids, where the energy price uncertainty is approximated by means of historical data.
Read the full abstract ▾
Machine Learning
Beyond Sliding Windows: Learning to Manage Memory in Non-Markovian Environments
Geraud Nangue Tasse, Matthew Riemer, Benjamin Rosman et al. (+1 more)
Recent success in developing increasingly general purpose
agents based on sequence models has led to increased focus on the problem of deploying computationally limited
agents within the vastly more complex real-world. A key challenge experienced in these more realistic domains is highly non-Markovian dependencies with respect to the
agent's observations, which are less common in small controlled domains. The predominant approach for dealing with this in the literature is to stack together a window of the most recent observations (Frame Stacking), but this window size must grow with the degree of non-Markovian dependencies, which results in prohibitive computational and
memory requirements for both action inference and learning. In this paper, we are motivated by the insight that in many environments that are highly non-Markovian with respect to time, the environment only causally depends on a relatively small number of observations over that time-scale. A natural direction would then be to consider meta-
algorithms that maintain relatively small adaptive stacks of
memories such that it is possible to express highly non-Markovian dependencies with respect to time while considering fewer observations at each step and thus experience substantial savings in both compute and
memory requirements. Hence, we propose a meta-
algorithm (Adaptive Stacking) for achieving exactly that with convergence guarantees and quantify the reduced computation and
memory constraints for MLP, LSTM, and
Transformer-based
agents. Our experiments utilize popular
memory tasks, which give us control over the degree of non-Markovian dependencies. This allows us to demonstrate that an appropriate meta-
algorithm can learn the removal of
memories not predictive of future rewards without excessive removal of important experiences. Code: https://github.com/geraudnt/adaptive-stacking
Read the full abstract ▾
All authors Geraud Nangue Tasse, Matthew Riemer, Benjamin Rosman, Tim Klinger
Machine Learning
RP-CATE: Recurrent Perceptron-based Channel Attention Transformer Encoder for Industrial Hybrid Modeling
Haoran Yang, Yinan Zhang, Wenjie Zhang et al. (+5 more)
Nowadays, industrial hybrid modeling which integrates both mechanistic modeling and machine learning-based modeling techniques has attracted increasing interest from scholars due to its high accuracy, low computational cost, and satisfactory interpretability. Nevertheless, the existing industrial hybrid modeling methods still face two main limitations. First, current research has mainly focused on applying a single machine learning method to one specific task, failing to develop a comprehensive machine learning architecture suitable for modeling tasks, which limits their ability to effectively represent complex industrial scenarios. Second, industrial
datasets often contain underlying associations (e.g., monotonicity or periodicity) that are not adequately exploited by current research, which can degrade model's predictive performance. To address these limitations, this paper proposes the Recurrent Perceptron-based Channel
Attention Transformer Encoder (RP-CATE), with three distinctive characteristics: 1: We developed a novel architecture by replacing the self-
attention mechanism with channel
attention and incorporating our proposed Recurrent Perceptron (RP) Module into
Transformer, achieving enhanced effectiveness for industrial modeling tasks compared to the original
Transformer. 2: We proposed a new data type called Pseudo-Image Data (PID) tailored for channel
attention requirements and developed a cyclic sliding window method for generating PID. 3: We introduced the concept of Pseudo-Sequential Data (PSD) and a method for converting industrial
datasets into PSD, which enables the RP Module to capture the underlying associations within industrial
dataset more effectively. An experiment aimed at hybrid modeling in chemical engineering was conducted by using RP-CATE and the experimental results demonstrate that RP-CATE achieves the best performance compared to other baseline models.
Read the full abstract ▾
All authors Haoran Yang, Yinan Zhang, Wenjie Zhang, Dongxia Wang, Peiyu Liu, Yuqi Ye, Kexin Chen, Wenhai Wang
Machine Learning
A Convex Loss Function for Set Prediction with Optimal Trade-offs Between Size and Conditional Coverage
Francis Bach
We consider supervised learning problems in which set predictions provide explicit uncertainty estimates. Using Choquet integrals (a.k.a. Lovász extensions), we propose a convex loss function for nondecreasing subset-valued functions obtained as level sets of a real-valued function. This loss function allows optimal trade-offs between conditional probabilistic coverage and the ”size” of the set, measured by a non-decreasing submodular function. We also propose several extensions that mimic loss functions and criteria for binary classification with asymmetric losses, and show how to naturally obtain sets with optimized conditional coverage. We derive efficient optimization
algorithms, either based on
stochastic gradient descent or reweighted least-squares formulations, and illustrate our findings with a series of experiments on synthetic
datasets for classification and regression tasks, showing improvements over approaches that aim for marginal coverage.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.DC
Evidential Trust-Aware Model Personalization in Decentralized Federated Learning for Wearable IoT
Murtaza Rangwala, Richard O. Sinnott and Rajkumar Buyya
Decentralized
federated learning (DFL) enables collaborative model training across edge devices without centralized coordination, offering resilience against single points of failure. However, statistical heterogeneity arising from non-identically distributed local data creates a fundamental challenge: nodes must learn personalized models adapted to their local distributions while selectively collaborating with compatible peers. Existing approaches either enforce a single global model that fits no one well, or rely on heuristic peer selection mechanisms that cannot distinguish between peers with genuinely incompatible data distributions and those with valuable complementary knowledge. We present Murmura, a framework that leverages evidential deep learning to enable trust-aware model personalization in DFL. Our key insight is that epistemic uncertainty from Dirichlet-based evidential models directly indicates peer compatibility: high epistemic uncertainty when a peer's model evaluates local data reveals distributional mismatch, enabling nodes to exclude incompatible influence while maintaining personalized models through selective collaboration. Murmura introduces a trust-aware aggregation mechanism that computes peer compatibility scores through cross-evaluation on local validation samples and personalizes model aggregation based on evidential trust with adaptive thresholds. Evaluation on three wearable IoT
datasets (UCI HAR, PAMAP2, PPG-DaLiA) demonstrates that Murmura reduces performance degradation from IID to non-IID conditions compared to baseline (0.9% vs. 19.3%), achieves 7.4$\times$ faster convergence, and maintains stable accuracy across hyperparameter choices. These results establish evidential uncertainty as a principled foundation for compatibility-aware personalization in decentralized heterogeneous environments.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Tzu-Yun Lee, Ding-Yong Hong and Jan-Jan Wu
This paper introduces Syntactic
Attention Pruning (SAP), a novel method for effectively
pruning attention heads in
Transformer models. Unlike conventional approaches that rely solely on mathematical analysis of model weights and activations, SAP incorporates both the syntactic structure and
attention patterns of sentences to guide the
pruning process. By leveraging these linguistic features, SAP not only achieves performance comparable to state-of-the-art methods but also enhances the interpretability of model behavior. To further improve robustness, we propose Candidate Filtering (CF), a mechanism that prioritizes heads based on their contribution to model performance, mitigating degradation during
pruning. Experimental results indicate that SAP effectively preserves critical heads of a high density of strong
attention values, outperforming existing head
pruning strategies in retrain-free settings. These findings position SAP as a promising foundation for a new direction in model compression research, offering high flexibility for
pruning across all
transformer-based
language models.
Read the full abstract ▾
Machine Learning
A Composable Channel-Adaptive Architecture for Seizure Classification
Francesco Carzaniga, Michael Hersche, Kaspar Schindler et al. (+1 more)
Objective: We develop a channel-adaptive (CA) architecture that seamlessly processes multi-variate time-series with an arbitrary number of channels, and in particular intracranial electroencephalography (iEEG) recordings. Methods: Our CA architecture first processes the iEEG signal using state-of-the-art models applied to each single channel independently. The resulting features are then fused using a vector-symbolic
algorithm which reconstructs the spatial relationship using a trainable scalar per channel. Finally, the fused features are accumulated in a long-term
memory of up to 2 minutes to perform the classification. Each CA-model can then be pre-trained on a large corpus of iEEG recordings from multiple heterogeneous subjects. The pre-trained model is personalized to each subject via a quick fine-tuning routine, which uses equal or lower amounts of data compared to existing state-of-the-art models, but requiring only 1/5 of the time. Results: We evaluate our CA-models on a seizure detection task both on a short-term (~20 hours) and a long-term (~2500 hours)
dataset. In particular, our CA-EEGWaveNet is trained on a single seizure of the tested subject, while the baseline EEGWaveNet is trained on all but one. Even in this challenging scenario, our CA-EEGWaveNet surpasses the baseline in median F1-score (0.78 vs 0.76). Similarly, CA-EEGNet based on EEGNet, also surpasses its baseline in median F1-score (0.79 vs 0.74). Conclusion and significance: Our CA-model addresses two issues: first, it is channel-adaptive and can therefore be trained across heterogeneous subjects without loss of performance; second, it increases the effective temporal context size to a clinically-relevant length. Therefore, our model is a drop-in replacement for existing models, bringing better characteristics and performance across the board.
Read the full abstract ▾
All authors Francesco Carzaniga, Michael Hersche, Kaspar Schindler, Abbas Rahimi
Machine Learning
HyperLoad: A Cross-Modality Enhanced Large Language Model-Based Framework for Green Data Center Cooling Load Prediction
Haoyu Jiang, Boan Qu, Junjie Zhu et al. (+3 more)
The rapid growth of artificial intelligence is exponentially escalating computational demand, inflating data center energy use and carbon emissions, and spurring rapid deployment of green data centers to relieve resource and environmental stress. Achieving sub-minute orchestration of renewables, storage, and loads, while minimizing PUE and lifecycle carbon intensity, hinges on accurate load
forecasting. However, existing methods struggle to address small-sample scenarios caused by cold start, load distortion, multi-source data fragmentation, and distribution shifts in green data centers. We introduce HyperLoad, a cross-modality framework that exploits pre-trained large
language models (LLMs) to overcome data scarcity. In the Cross-Modality Knowledge
Alignment phase, textual priors and time-series data are mapped to a common latent space, maximizing the utility of prior knowledge. In the Multi-Scale Feature Modeling phase, domain-aligned priors are injected through adaptive prefix-tuning, enabling rapid scenario adaptation, while an Enhanced Global Interaction
Attention mechanism captures cross-device temporal dependencies. The public DCData
dataset is released for benchmarking. Under both data sufficient and data scarce settings, HyperLoad consistently surpasses state-of-the-art (SOTA) baselines, demonstrating its practicality for sustainable green data center management.
Read the full abstract ▾
All authors Haoyu Jiang, Boan Qu, Junjie Zhu, Fanjie Zeng, Xiaojie Lin, Wei Zhong
Machine Learning · Cross-listed from math.OC
Explicit and Non-asymptotic Query Complexities of Rank-Based Zeroth-order Algorithm on Stochastic Smooth Functions
Haishan Ye
Zeroth-order (ZO) optimization with ordinal feedback has emerged as a fundamental problem in modern machine learning systems, particularly in human-in-the-loop settings such as reinforcement learning from human feedback, preference learning, and evolutionary strategies. While rank-based ZO
algorithms enjoy strong empirical success and robustness properties, their theoretical understanding, especially under
stochastic objectives and standard smoothness assumptions, remains limited. In this paper, we study rank-based zeroth-order optimization for
stochastic functions where only ordinal feedback of the
stochastic function is available. We propose a simple and computationally efficient rank-based ZO
algorithm. Under standard assumptions including smoothness, strong convexity, and bounded second moments of
stochastic gradients, we establish explicit non-asymptotic query complexity bounds for both convex and nonconvex objectives. Notably, our results match the best-known query complexities of value-based ZO
algorithms, demonstrating that ordinal information alone is sufficient for optimal query efficiency in
stochastic settings. Our analysis departs from existing drift-based and information-
geometric techniques, offering new tools for the study of rank-based optimization under noise. These findings narrow the gap between theory and practice and provide a principled foundation for optimization driven by human preferences.
Read the full abstract ▾
Machine Learning
Timely Parameter Updating in Over-the-Air Federated Learning
Jiaqi Zhu, Zhongyuan Zhao, Xiao Li et al. (+3 more)
Incorporating over-the-air computations (OAC) into the model training process of
federated learning (FL) is an effective approach to alleviating the communication bottleneck in FL systems. Under OAC-FL, every client modulates its intermediate parameters, such as gradient, onto the same set of orthogonal waveforms and simultaneously transmits the
radio signal to the edge server. By exploiting the superposition property of multiple-access channels, the edge server can obtain an automatically aggregated global gradient from the received signal. However, the limited number of orthogonal waveforms available in practical systems is fundamentally mismatched with the high dimensionality of modern deep learning models. To address this issue, we propose Freshness Freshness-mAgnItude awaRe top-k (FAIR-k), an
algorithm that selects, in each communication round, the most impactful subset of gradients to be updated over the air. In essence, FAIR-k combines the complementary strengths of the Round-Robin and Top-k
algorithms, striking a delicate balance between timeliness (freshness of parameter updates) and importance (gradient magnitude). Leveraging tools from Markov analysis, we characterize the distribution of parameter staleness under FAIR-k. Building on this, we establish the convergence rate of OAC-FL with FAIR-k, which discloses the joint effect of data heterogeneity, channel noise, and parameter staleness on the training efficiency. Notably, as opposed to conventional analyses that assume a universal Lipschitz constant across all the clients, our framework adopts a finer-grained model of the data heterogeneity. The analysis demonstrates that since FAIR-k promotes fresh (and fair) parameter updates, it not only accelerates convergence but also enhances communication efficiency by enabling an extended period of local training without significantly affecting overall training efficiency.
Read the full abstract ▾
All authors Jiaqi Zhu, Zhongyuan Zhao, Xiao Li, Ruihao Du, Shi Jin, Howard H.Yang
Machine Learning
Dual Model Deep Learning for Alzheimer Prognostication
Alireza Moayedikia, Sara Fin and Uffe Kock Wiil
Disease modifying therapies for Alzheimer's disease demand precise timing decisions, yet current predictive models require longitudinal observations and provide no uncertainty quantification, rendering them impractical at the critical first visit when treatment decisions must be made. We developed PROGRESS (PRognostic Generalization from REsting Static Signatures), a dual-model deep learning framework that transforms a single baseline cerebrospinal fluid biomarker assessment into actionable prognostic estimates without requiring prior clinical history. The framework addresses two complementary clinical questions: a probabilistic trajectory network predicts individualized cognitive decline with calibrated uncertainty bounds achieving near-nominal coverage, enabling honest prognostic communication; and a deep survival model estimates time to conversion from mild cognitive impairment to dementia. Using data from over 3,000 participants across 43 Alzheimer's Disease Research Centers in the National Alzheimer's Coordinating Center database, PROGRESS substantially outperforms Cox proportional hazards, Random Survival Forests, and gradient boosting methods for survival prediction. Risk stratification identifies patient groups with seven-fold differences in conversion rates, enabling clinically meaningful treatment prioritization. Leave-one-center-out validation demonstrates robust generalizability, with survival discrimination remaining strong across held-out sites despite heterogeneous measurement conditions spanning four decades of assay technologies. By combining superior survival prediction with trustworthy trajectory uncertainty quantification, PROGRESS bridges the gap between biomarker measurement and personalized clinical decision-making.
Read the full abstract ▾
Machine Learning
DIVER-1: Scaling Intracranial EEG Foundation Models for Transferable Representations
Danny Dongyeop Han, Yonghyeon Gwon, Ahhyun Lucy Lee et al. (+10 more)
Intracranial
EEG (iEEG) provides direct, millisecond-scale recordings of human neural activity, but reusable representation learning is difficult because electrode layouts, anatomical coverage, referencing schemes, and recording conditions vary across patients and centers. We introduce DIVER-1, a self-supervised iEEG foundation model for variable-input recordings that combines any-variate electrode-time
attention, spatio-temporal resampling, input-conditioned positional
embeddings, and multi-domain masked reconstruction without assuming a fixed electrode montage. We pretrain two variants, DIVER-1-0.1s and DIVER-1-1s, on 5,310 hours of ECoG and SEEG spanning 352k channel-hours, roughly 54x the BrainTreeBank-based pretraining volume. We evaluate DIVER-1 on two held-out
benchmarks: Neuroprobe for naturalistic cognitive decoding and MAYO for seizure detection. On leakage-aware Neuroprobe, DIVER-1-0.1s outperforms prior evaluated iEEG foundation models despite using no BrainTreeBank recordings, the corpus underlying Neuroprobe, during pretraining; it also exceeds the linear spectrogram decoder in mean AUROC and remains competitive with stronger nonlinear baselines, a level prior evaluated iEEG foundation models did not reach. DIVER-1-1s also achieves the top AUROC on MAYO seizure detection. Finally, we conduct, to our knowledge, the first controlled compute-aware scaling study for self-supervised iEEG pretraining, sweeping data scale, subject count, training duration, and model size up to 1.8B parameters. Our results indicate a data-constrained regime: expanding unique recordings and training sufficiently long are more reliable scaling axes than increasing parameter count alone. Code is available at link.
Read the full abstract ▾
All authors Danny Dongyeop Han, Yonghyeon Gwon, Ahhyun Lucy Lee, Taeyang Lee, Seong Jin Lee, Jubin Choi, Sebin Lee, Jihyun Bang, Seungju Lee, David Keetae Park, Shinjae Yoo, Chun Kee Chung, Jiook Cha
Machine Learning · Cross-listed from cs.CV
Auditing Significance, Metric Choice, and Demographic Fairness in Medical AI Challenges
Ariel Lubonja, Pedro R. A. S. Bassi, Wenxuan Li et al. (+4 more)
Open challenges have become the de facto standard for comparative ranking of medical AI methods. Despite their importance, medical AI leaderboards exhibit three persistent limitations: (1) score gaps are rarely tested for statistical significance, so rank stability is unknown; (2) single averaged metrics are applied to every organ, hiding clinically important boundary errors; (3) performance across intersecting demographics is seldom reported, masking
fairness and equity gaps. We introduce RankInsight, an open-source toolkit that seeks to address these limitations. RankInsight (1) computes pair-wise significance maps that show the nnU-Net family outperforms
Vision-
Language and MONAI submissions with high statistical certainty; (2) recomputes leaderboards with organ-appropriate metrics, reversing the order of the top four models when Dice is replaced by NSD for tubular structures; and (3) audits intersectional
fairness, revealing that more than half of the MONAI-based entries have the largest gender-race discrepancy on our proprietary Johns Hopkins Hospital
dataset. The RankInsight toolkit is publicly released and can be directly applied to past, ongoing, and future challenges. It enables organizers and participants to publish rankings that are statistically sound, clinically meaningful, and demographically fair.
Read the full abstract ▾
All authors Ariel Lubonja, Pedro R. A. S. Bassi, Wenxuan Li, Hualin Qiao, Randal Burns, Alan L. Yuille, Zongwei Zhou
Machine Learning · Cross-listed from cs.IT
On Cost-Aware Designs for Sequential Hypothesis Testing
George Vershinin, Asaf Cohen and Omer Gurewitz
We introduce Cost-Aware (CA) Sequential Hypothesis Testing (CASHT), in which an active decision-maker selects sensing actions with different, random costs to identify the true hypothesis under an average-error constraint $\delta$, while minimizing the expected total cost (rather than the number of samples). For fixed costs, we prove that the optimal expected total cost scales as $\Theta(\log(1/\delta))$, and is achievable by Multihypothesis Sequential Probability Ratio Test-based procedures. We show that the CA design principle is to maximize the ratio of expected information gain to expected cost under the policy-induced action distribution. Guided by this principle, we adapt two classic policies to the CA setting and establish their asymptotic optimality. We then treat random costs under two revelation models: ex-post, where costs are disclosed only after a sample is obtained, and the cost-error tradeoff coincides with the fixed-cost case, and ex-ante, where costs accrue before acquisition, and the decision maker may cancel an action mid-operation. For the ex-ante model, we characterize when cancellation lowers the total cost and analyze several cost distributions in detail. Simulations confirm our findings that the CA variants consistently reduce total cost relative to their classic counterparts, and when action cancellation helps or hurts.
Read the full abstract ▾