Machine Learning · Cross-listed from math.NA
Trustworthy Koopman Operator Learning: Invariance Diagnostics and Error Bounds
Gustav Conradie, Nicolas Boullé, Jean-Christophe Loiseau et al. (+2 more)
Koopman operator theory provides a global linear representation of nonlinear dynamics and underpins many data-driven methods. In practice, however, finite-dimensional feature spaces induced by a user-chosen dictionary are rarely invariant, so closure failures and projection errors lead to spurious
eigenvalues, misleading Koopman modes, and overconfident forecasts. This paper addresses a central validation problem in data-driven Koopman methods: how to quantify invariance and projection errors for an arbitrary feature space using only snapshot data, and how to use these diagnostics to produce actionable guarantees and guide dictionary refinement? A unified a posteriori methodology is developed for certifying when a Koopman approximation is trustworthy and improving it when it is not. Koopman invariance is quantified using principal angles between a subspace and its Koopman image, yielding principal observables and a principal angle decomposition (PAD), a dynamics-informed alternative to SVD truncation with significantly improved performance. Multi-step error bounds are derived for Koopman and Perron–Frobenius mode decompositions, including RKHS-based pointwise guarantees, and are complemented by Gaussian process expected error surrogates. The resulting toolbox enables validated spectral analysis, certified
forecasting, and principled dictionary and kernel learning, demonstrated on chaotic and high-dimensional
benchmarks and real-world
datasets, including cavity flow and the Pluto–Charon system.
Read the full abstract ▾
All authors Gustav Conradie, Nicolas Boullé, Jean-Christophe Loiseau, Steven L. Brunton, Matthew J. Colbrook
Machine Learning · Cross-listed from cs.ET
Edge AI and TinyML for Resource-Constrained Agriculture: Hardware Platforms, Model Optimization, and Deployment Strategies
Riya Samanta, Bidyut Saha and Soumya Kanti Ghosh
Deploying artificial intelligence at the edge is increasingly important for systems that must operate under strict limits on
memory, computation, energy, connectivity, and hardware cost. Tiny Machine Learning (TinyML) represents the most resource-constrained regime of this shift, enabling inference on microcontroller-class devices with tightly bounded flash and SRAM budgets. This study examines recent deployment trends in Edge AI and TinyML using precision farming as a representative real-world environment in which sensing nodes, gateways,
UAVs, and embedded processors must often operate with intermittent connectivity and limited power. The analysis emphasizes the deployment stack rather than predictive accuracy alone, covering hardware classes, software toolchains, model- and system-level optimization, intelligence placement, and hardware-resource reporting. Across the analyzed studies, microcontroller-class platforms such as ESP32, STM32, and ATmega are prominent endpoint targets, while single-board computers and embedded accelerators support more compute-intensive workloads. Among the studies included in the optimization comparison, almost 50% report quantization, making reduced-precision inference the most frequently identified strategy within this subset. In contrast, structured
pruning, hardware-aware neural architecture search, and coordinated cross-layer optimization remain comparatively limited. Resource profiling is also inconsistent as flash, peak RAM, MAC count, latency, and energy per inference are not reported uniformly, limiting reproducibility and comparisons across deployments. A recurring systems-level pattern is an inference-training asymmetry in which inference moves toward local devices while training and model lifecycle management remain predominantly centralized.
Read the full abstract ▾
Machine Learning
Interpretable Classification of Time Series Using Euler Characteristic Surfaces
Salam Rabindrajit Luwang, Sushovan Majhi, Vishal Mandal et al. (+3 more)
Persistent homology (PH) – the conventional method in topological data analysis – is computationally expensive, requires further vectorization of its signatures before machine learning (ML) can be applied, and captures information along only the spatial axis. For time series data, we propose Euler Characteristic Surfaces (ECS) as an alternative topological signature based on the Euler characteristic ($\chi$) – a fundamental topological invariant. The ECS provides a computationally efficient, spatiotemporal, and inherently discretized feature representation that can serve as direct input to ML models. We prove a stability
theorem guaranteeing that the ECS remains stable under small perturbations of the input time series. We first demonstrate that ECS effectively captures the nontrivial topological differences between the limit cycle and the strange attractor in the Rössler system. We then develop an ECS-based classification framework and apply it to five
benchmark biomedical
datasets (four ECG, one
EEG) from the UCR/UEA archive. On $\textit{ECG5000}$, our single-feature ECS classifier achieves $98\%$ accuracy with $O(n+R\cdot T)$ complexity, compared to $62\%$ reported by a recent PH-based method. An AdaBoost extension raises accuracy to $98.6\%$, matching the best deep learning results while retaining full interpretability. Strong results are also obtained on $\textit{TwoLeadECG}$ ($94.1\%$) and $\textit{Epilepsy2}$ ($92.6\%$).
Read the full abstract ▾
All authors Salam Rabindrajit Luwang, Sushovan Majhi, Vishal Mandal, Atish J. Mitra, Md. Nurujjaman, Buddha Nath Sharma
Machine Learning · Cross-listed from eess.SP
Generative Semantic HARQ: Latent-Space Text Retransmission and Combining
Bin Han, Yulin Hu and Hans D. Schotten
Semantic communication conveys meaning rather than raw bits, but reliability at the semantic level remains an open challenge. We propose a semantic-level hybrid automatic repeat request (HARQ) framework for text communication, in which a
Transformer-variational autoencoder (VAE) codec operates as a lightweight overlay on the conventional protocol stack. The
stochastic encoder inherently generates diverse latent representations across retransmissions-providing incremental knowledge (IK) from a single model without dedicated protocol design. On the receiver side, a soft quality
estimator triggers retransmissions and a quality-aware combiner merges the received latent vectors within a consistent latent space. We systematically
benchmark six semantic quality metrics and four soft combining strategies under hybrid semantic distortion that mixes systematic bias with additive noise. The results suggest combining Weighted-Average or MRC-Inspired combining with self-consistency-based HARQ triggering for the best performance.
Read the full abstract ▾
Machine Learning
Muon Converges under Heavy-Tailed Noise: Nonconvex Hölder-Smooth Empirical Risk Minimization
Hideaki Iiduka
Muon is a recently proposed optimizer that enforces orthogonality in parameter updates by projecting gradients onto the Stiefel
manifold, leading to stable and efficient training in large-scale deep neural networks. Meanwhile, the previously reported results indicated that
stochastic noise in practical machine learning may exhibit heavy-tailed behavior, violating the bounded-variance assumption. In this paper, we consider the problem of minimizing a nonconvex Hölder-smooth empirical risk that works well with the heavy-tailed
stochastic noise. We then show that
Muon converges to a stationary point of the empirical risk under the boundedness condition accounting for heavy-tailed
stochastic noise. In addition, we show that
Muon converges faster than mini-batch SGD.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Analyzing Error Sources in Global Feature Effect Estimation
Timo Heiß, Coco Bögel, Bernd Bischl et al. (+1 more)
Global feature effects such as partial dependence (PD) and accumulated local effects (ALE) plots are widely used to interpret black-box models. However, they are only estimates of true underlying effects, and their reliability depends on multiple sources of error. Despite the popularity of global feature effects, these error sources are largely unexplored. In particular, the practically relevant question of whether to use training or holdout data to estimate feature effects remains unanswered. We address this gap by providing a systematic,
estimator-level analysis that disentangles sources of bias and variance for PD and ALE. To this end, we derive a mean-squared-error decomposition that separates model bias, estimation bias, model variance, and estimation variance, and analyze their dependence on model characteristics, data selection, and sample size. We validate our theoretical findings through an extensive simulation study across multiple data-generating processes, learners, estimation strategies (training data, validation data, and cross-validation), and sample sizes. Our results reveal that, while using holdout data is theoretically the cleanest, potential biases arising from the training data are empirically negligible and dominated by the impact of the usually higher sample size. The estimation variance depends on both the presence of interactions and the sample size, with ALE being particularly sensitive to the latter. Cross-validation-based estimation is a promising approach that reduces the model variance component, particularly for overfitting models. Our analysis provides a principled explanation of the sources of error in feature effect estimates and offers concrete guidance on choosing estimation strategies when interpreting machine learning models.
Read the full abstract ▾
All authors Timo Heiß, Coco Bögel, Bernd Bischl, Giuseppe Casalicchio
Machine Learning · Cross-listed from stat.ML
Spatio-temporal probabilistic forecast using MMAF-guided learning
Leonardo Bardi, Imma Valentina Curato and Lorenzo Proietti
We present a theory-guided generalized
Bayesian methodology for spatio-temporal raster data, which we use to train an ensemble of
stochastic feed-forward neural networks with Gaussian-distributed weights. The methodology incorporates the dependence and causal structure of a spatio-temporal Ornstein-Uhlenbeck process into training and inference by enforcing constraints on the design of the data
embedding and the related optimization routine. In inference mode, the networks are employed to generate causal ensemble forecasts by applying different initial conditions at different horizons. We call this workflow MMAF-guided learning. Experiments conducted on both synthetic and real data demonstrate that our forecasts remain calibrated across multiple time horizons. Moreover, we show that on such data, shallow feed-forward architectures can achieve performance comparable to, and in some cases better than, convolutional or
diffusion deep learning architectures used in probabilistic
forecasting tasks.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Thinking in Latents: Adaptive Anchor Refinement for Implicit Reasoning in LLMs
Disha Sheshanarayana, Rajat Subhra Pal, Manjira Sinha et al. (+1 more)
Token-level Chain-of-Thought (CoT) prompting has become a standard way to elicit multi-step
reasoning in large
language models (LLMs), especially for mathematical word problems. However, generating long intermediate traces increases output length and inference cost, and can be inefficient when the model could arrive at the correct answer without extensive verbalization. This has motivated latent-space
reasoning approaches that shift computation into hidden representations and only emit a final answer. Yet, many latent
reasoning methods depend on a fixed number of latent refinement steps at inference, adding another hyperparameter that must be tuned across models and
datasets to balance accuracy and efficiency. We introduce AdaAnchor, a latent
reasoning framework that performs silent iterative computation by refining a set of latent anchor vectors attached to the input. AdaAnchor further incorporates an adaptive halting mechanism that monitors anchor stability across iterations and terminates refinement once the anchor dynamics converge, allocating fewer steps to easier instances while reserving additional refinement steps for harder ones under a shared maximum-step budget. Our empirical evaluation across three mathematical word-problem
benchmarks shows that AdaAnchor with adaptive halting yields accuracy gains of up to 5% over fixed-step latent refinement while reducing average latent refinement steps by 48-60% under the same maximum-step budget. Compared to standard
reasoning baselines, AdaAnchor achieves large reductions in generated
tokens (92-93%) by moving computation into silent latent refinement, offering a different accuracy-efficiency trade-off with substantially lower output-
token usage.
Read the full abstract ▾
All authors Disha Sheshanarayana, Rajat Subhra Pal, Manjira Sinha, Tirthankar Dasgupta
Machine Learning
CrossADR: enhancing adverse drug reactions prediction for combination pharmacotherapy with cross-layer feature integration and cross-level associative learning
Y. Cheung
Combination pharmacotherapy offers substantial therapeutic advantages but also poses substantial risks of adverse drug reactions (ADRs). The accurate prediction of ADRs with interpretable computational methods is crucial for clinical safety management, drug development, and precision medicine. However, managing ADRs remains a challenge due to the vast search space of drug combinations and the complexity of physiological responses. Current graph-based architectures often struggle to effectively integrate multi-scale biological information and frequently rely on fixed association
matrices, which limits their ability to capture dynamic organ-level dependencies and generalize across diverse
datasets. Here we propose CrossADR, a hierarchical framework for organ-level ADR prediction through cross-layer feature integration and cross-level associative learning. It incorporates a gated-residual-flow graph neural network to fuse multi-scale molecular features and utilizes a learnable ADR
embedding space to dynamically capture latent biological correlations across 15 organ systems. Systematic evaluation on the newly constructed CrossADR-
Dataset-covering 1,376 drugs and 946,000 unique combinations-demonstrates that CrossADR consistently achieves state-of-the-art performance across 80 distinct experimental scenarios and provides high-resolution insights into drug-related
protein protein interactions and pathways. Overall, CrossADR represents a robust tool for cross-scale biomedical information integration, cross-layer feature integration as well as cross-level associative learning, and can be effectively utilized to prevent ADRs in clinical decision-making.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
Prompt Readiness Levels (PRL): a maturity scale and scoring framework for production grade prompt assets
Sebastien Guinard
Prompt engineering has become a production critical component of generative AI systems. However, organizations still lack a shared, auditable method to qualify
prompt assets against operational objectives, safety constraints, and compliance requirements. This paper introduces
Prompt Readiness Levels (PRL), a nine level maturity scale inspired by TRL, and the
Prompt Readiness Score (PRS), a multidimensional scoring method with gating thresholds designed to prevent weak link failure modes. PRL/PRS provide an original, structured and methodological framework for governing
prompt assets specification, testing, traceability, security evaluation, and deployment readiness enabling valuation of
prompt engineering through reproducible qualification decisions across teams and industries.
Read the full abstract ▾
Machine Learning · Cross-listed from physics.flu-dyn
A convolutional autoencoder and neural ODE surrogate modeling framework applied to transient counterflow flames
Mert Yakup Baykan, Weitao Liu, Mohammad Rafi Malik et al. (+4 more)
A novel convolutional autoencoder and neural ODE (CAE-NODE) framework is proposed for a reduced-order model (ROM) applied to transient 2D counterflow flames, as an extension of AE-NODE methods in homogeneous reactive systems to spatially resolved flows. The multidimensional thermochemical fields (256 x 256 grid, 21 variables) obtained from direct numerical simulations (DNS) are used in training the CAE, where convolutional layers learned the underlying spatial correlations, allowing the CAE to construct an unsupervised 3D latent
manifold that is physically meaningful, smooth, and continuous in time. This results in a compression ratio of over 400,000 times. The NODE then subsequently learns the continuous-time dynamics on the latent
manifold, enabling the prediction of the full temporal
evolution of the flames by integrating forward in time from an initial condition. The results demonstrate that the CAE-NODE can accurately capture the entire transient process, including ignition, flame propagation, and the gradual transition to a non-premixed condition, with excellent agreement with the DNS, while adhering to conservation principles at virtually no computational cost compared to the reference DNS. Predictions remain accurate at strain rates outside the training range. Moreover, despite being unsupervised, the learned latent
manifold is highly correlated with the flame-state descriptors such as the progress variable, mixture fraction, and the scalar dissipation rate. This study, for the first time, highlights the potential of CAE-NODE for surrogate modeling of unsteady dynamics of multi-dimensional reacting flows.
Read the full abstract ▾
All authors Mert Yakup Baykan, Weitao Liu, Mohammad Rafi Malik, Thorsten Zirwes, Andreas Kronenburg, Hong G. Im, Dong-hyuk Shin
Machine Learning · Cross-listed from cs.CL
Interpretable Predictability-Based AI Text Detection: A Replication Study
Adam Skurla, Dominik Macko and Jakub Simko
This paper replicates and extends the system used in the AuTexTification shared task for authorship attribution of machine-generated texts. Exact replication was not possible because of differences in data splits, model availability, and implementation details, which we document as a case study in reproducibility. We tested newer
multilingual language models (mDeBERTa-v3-base, Qwen, mGPT) and added 26 document-level stylometric features, using ablation, permutation importance, and SHAP analysis to assess feature influence. A single shared configuration was applied to both English and Spanish across Subtask 1 and Subtask 2. Averaged over three random seeds, the shared
multilingual configuration performs comparably to or better than the
language-specific baseline, with the clearest gains on model attribution (Subtask 2). The additional stylometric features yield small improvements, led by lexical diversity, but their contribution falls within seed variance once predictability-based probabilities are included, which remain the dominant signal. The study also shows that clear documentation is important for reliable replication and fair comparison of systems.
Read the full abstract ▾
Machine Learning
Models Designed to Forget: Machine Unlearning via Key Deletion
Sonia Laguna, Jorge da Silva Goncalves, Moritz Vandenhirtz et al. (+3 more)
Machine unlearning for
vision models is rapidly becoming a practical requirement, driven by
privacy regulations, data errors, and the need to remove harmful or corrupted training images. Despite this, most existing approximate unlearning methods tackle the problem from a post-hoc perspective. They attempt to erase the influence of targeted samples through parameter updates that typically require access to the full training data. This creates a mismatch with real deployment scenarios where unlearning requests can be anticipated, revealing a fundamental limitation of post-hoc approaches. We motivate unlearning by design, a novel paradigm for approximate methods in which models are directly trained to support forgetting as an inherent architectural capability. We instantiate this idea with Machine UNlearning via KEY deletion (MUNKEY), a
memory-augmented
transformer that decouples instance-specific memorization from model weights. Here, unlearning corresponds to removing the instance-identifying key, enabling zero-shot forgetting without weight updates or access to the original samples or labels. Across natural image
benchmarks, fine-grained visual recognition, and medical
datasets, MUNKEY outperforms all post-hoc baselines. Our results establish that unlearning by design enables fast, deployment-oriented unlearning while preserving predictive performance.
Read the full abstract ▾
All authors Sonia Laguna, Jorge da Silva Goncalves, Moritz Vandenhirtz, Alain Ryser, Irene Cannistraci, Julia E. Vogt
Machine Learning · Cross-listed from cs.CV
Training-free Detection of Generated Videos via Spatial-Temporal Likelihoods
Omer Ben Hayun, Roy Betser, Meir Yossef Levi et al. (+2 more)
Following major advances in text and image generation, the
video domain has surged, producing highly realistic and controllable sequences. Along with this progress, these models also raise serious concerns about misinformation, making reliable detection of synthetic
videos increasingly crucial. Image-based detectors are fundamentally limited because they operate per frame and ignore temporal dynamics, while supervised
video detectors generalize poorly to unseen generators, a critical drawback given the rapid emergence of new models. These challenges motivate zero-shot approaches, which avoid synthetic data and instead score content against real-data statistics, enabling training-free, model-agnostic detection. We introduce STALL, a simple, training-free, theoretically justified detector that provides likelihood-based scoring for
videos, jointly modeling spatial and temporal evidence within a probabilistic framework. We evaluate STALL on two public
benchmarks and introduce ComGenVid, a new
benchmark with state-of-the-art generative models. STALL consistently outperforms prior image- and
video-based baselines. Code and data are available at https://omerbenhayun.github.io/stall-
video.
Read the full abstract ▾
All authors Omer Ben Hayun, Roy Betser, Meir Yossef Levi, Levi Kassel, Guy Gilboa
Machine Learning · Cross-listed from cs.AI
Consequentialist Objectives and Catastrophe
Henrik Marklund, Alex Infanger and Benjamin Van Roy
Because human preferences are too complex to codify, AIs operate with misspecified objectives. Optimizing such objectives often produces undesirable outcomes; this phenomenon is known as reward hacking. Such outcomes are not necessarily catastrophic. Indeed, most examples of reward hacking in previous literature are benign. And typically, objectives can be modified to resolve the issue. We study the prospect of catastrophic outcomes induced by AIs operating in complex environments. We argue that, when capabilities are sufficiently advanced, pursuing a fixed consequentialist objective tends to result in catastrophic outcomes. We formalize this by establishing conditions that provably lead to such outcomes. Under these conditions, simple or random behavior is safe. Catastrophic risk arises due to extraordinary competence rather than incompetence. With a fixed consequentialist objective, avoiding catastrophe requires constraining AI capabilities. In fact, constraining capabilities the right amount not only averts catastrophe but yields valuable outcomes. Our results apply to any objective produced by modern industrial AI development pipelines.
Read the full abstract ▾
Machine Learning
TrajFlow: Nation-wide Pseudo GPS Trajectory Generation with Flow Matching Models
Peiran Li, Jiawei Wang, Haoran Zhang et al. (+4 more)
The importance of mobile phone GPS trajectory data is widely recognized across many fields, yet the use of real data is often hindered by
privacy concerns, limited accessibility, and high acquisition costs. As a result, generating pseudo-GPS trajectory data has become an active area of research. Recent
diffusion-based approaches have achieved strong fidelity but remain limited in spatial scale (small urban areas), transportation-mode diversity, and efficiency (requiring numerous sampling steps). To address these challenges, we introduce TrajFlow, which to the best of our knowledge is the first flow-matching-based generative model for GPS trajectory generation. TrajFlow leverages the flow-matching paradigm to improve robustness and efficiency across multiple geospatial scales, and incorporates a trajectory harmonization and reconstruction strategy to jointly address scalability, diversity, and efficiency. Using a nationwide mobile phone GPS
dataset with millions of trajectories across Japan, we show that TrajFlow or its variants consistently outperform
diffusion-based and deep generative baselines at urban, metropolitan, and nationwide levels. As the first nationwide, multi-scale GPS trajectory generation model, TrajFlow demonstrates strong potential to support inter-region urban planning, traffic management, and disaster response, thereby advancing the resilience and intelligence of future mobility systems.
Read the full abstract ▾
All authors Peiran Li, Jiawei Wang, Haoran Zhang, Xiaodan Shi, Noboru Koshizuka, Chihiro Shimizu, Renhe Jiang
Machine Learning · Cross-listed from q-bio.QM
Empowering Chemical Structures with Biological Insights for Scalable Phenotypic Virtual Screening
Xiaoqing Lian, Pengsen Ma, Tengfeng Ma et al. (+9 more)
Motivation: The scalable identification of bioactive compounds is essential for contemporary drug discovery. This process faces a key trade-off: structural screening offers scalability but lacks biological context, whereas high-content phenotypic profiling provides deep biological insights but is resource-intensive. The primary challenge is to extract robust biological signals from noisy data and encode them into representations that do not require biological data at inference. Results: This study presents DECODE (DEcomposing Cellular Observations of Drug Effects), a framework that bridges this gap by empowering chemical representations with intrinsic biological semantics to enable structure-based in silico biological profiling. DECODE leverages limited paired transcriptomic and morphological data as supervisory signals during training, enabling the extraction of a measurement-invariant biological fingerprint from chemical structures and explicit filtering of experimental noise. Our evaluations demonstrate that DECODE retrieves functionally similar drugs in zero-shot settings with over 20% relative improvement over chemical baselines in mechanism-of-action (MOA) prediction. Furthermore, the framework achieves a 6-fold increase in hit rates for novel anti-cancer
agents during external validation. Availability and implementation: The codes and
datasets of DECODE are available at https://github.com/lian-xiao/DECODE.
Read the full abstract ▾
All authors Xiaoqing Lian, Pengsen Ma, Tengfeng Ma, Zhonghao Ren, Xibao Cai, Zhixiang Cheng, Bosheng Song, He Wang, Xiang Pan, Yangyang Chen, Sisi Yuan, Chen Lin
Machine Learning
MONET: Modeling and Optimization of neural NEtwork Training from Edge to Data Centers
Jérémy Morlier, Robin Geens, Stef Cuyckens et al. (+4 more)
While hardware-software co-design has significantly improved the efficiency of neural network inference, modeling the training phase remains a critical yet underexplored challenge. Training workloads impose distinct constraints, particularly regarding
memory footprint and backpropagation complexity, which existing inference-focused tools fail to capture. This paper introduces MONET, a framework designed to model the training of neural networks on heterogeneous dataflow accelerators. MONET builds upon Stream, an experimentally verified framework that that models the inference of neural networks on heterogeneous dataflow accelerators with layer fusion. Using MONET, we explore the design space of ResNet-18 and a small GPT-2, demonstrating the framework's capability to model training workflows and find better hardware architectures. We then further examine problems that become more complex in neural network training due to the larger design space, such as determining the best layer-fusion configuration. Additionally, we use our framework to find interesting trade-offs in activation checkpointing, with the help of a genetic
algorithm. Our findings highlight the importance of a holistic approach to hardware-software co-design for scalable and efficient deep learning deployment.
Read the full abstract ▾
All authors Jérémy Morlier, Robin Geens, Stef Cuyckens, Arne Symons, Marian Verhelst, Vincent Gripon, Mathieu Léonardon
Machine Learning
How Log-Barrier Helps Exploration in Policy Optimization
Leonardo Cesani, Matteo Papini and Marcello Restelli
Recently, it has been shown that the
Stochastic Gradient Bandit (SGB)
algorithm converges to a globally optimal policy with a constant learning rate. However, these guarantees rely on unrealistic assumptions about the learning process, namely that the probability of the optimal action is always bounded away from zero. We attribute this to the lack of an explicit exploration mechanism in SGB. To address these limitations, we propose to regularize the SGB objective with a log-barrier on the parametric policy, structurally enforcing a minimal amount of exploration. We prove that Log-Barrier
Stochastic Gradient Bandit (LB-SGB) matches the sample complexity of SGB, but also converges (at a slower rate) without any assumptions on the learning process. We also show a connection between the log-barrier regularization and Natural Policy Gradient, as both exploit the geometry of the policy space by controlling the Fisher information. We validate our theoretical findings through numerical simulations, showing the benefits of the log-barrier regularization.
Read the full abstract ▾
Machine Learning
Lightweight User-Personalization Method for Closed Split Computing
Yuya Okada and Takayuki Nishio
Split Computing enables collaborative inference between edge devices and the cloud by partitioning a deep neural network into an edge-side head and a server-side tail, reducing latency and limiting exposure of raw input data. However, inference performance often degrades in practical deployments due to user-specific data distribution shifts, unreliable communication, and
privacy-oriented perturbations, especially in closed environments where model architectures and parameters are inaccessible. To address this challenge, we propose SALT (Split-Adaptive Lightweight Tuning), a lightweight adaptation framework for closed Split Computing systems. SALT introduces a compact client-side adapter that refines intermediate representations produced by a frozen head network, enabling effective model adaptation without modifying the head or tail networks or increasing communication overhead. By modifying only the training conditions, SALT supports multiple adaptation objectives, including user personalization, communication robustness, and
privacy-aware inference. Experiments using ResNet-18 on CIFAR-10 and CIFAR-100 show that SALT achieves higher accuracy than conventional retraining and fine-tuning while significantly reducing training cost. On CIFAR-10, SALT improves personalized accuracy from 88.1% to 93.8% while reducing training latency by more than 60%. SALT also maintains over 90% accuracy under 75% packet loss and preserves high accuracy (about 88% at sigma = 1.0) under noise injection. These results demonstrate that SALT provides an efficient and practical adaptation framework for real-world Split Computing systems.
Read the full abstract ▾
Machine Learning
SFedHIFI: Fire Rate-Based Heterogeneous Information Fusion for Spiking Federated Learning
Ran Tao, Qiugang Zhan, Shantian Yang et al. (+3 more)
Spiking
Federated Learning (SFL) has been widely studied with the energy efficiency of Spiking Neural Networks (SNNs). However, existing SFL methods require model homogeneity and assume all clients have sufficient computational resources, resulting in the exclusion of some resource-constrained clients. To address the prevalent system heterogeneity in real-world scenarios, enabling heterogeneous SFL systems that allow clients to adaptively deploy models of different scales based on their local resources is crucial. To this end, we introduce SFedHIFI, a novel Spiking
Federated Learning framework with Fire Rate-Based Heterogeneous Information Fusion. Specifically, SFedHIFI employs channel-wise
matrix decomposition to deploy SNN models of adaptive complexity on clients with heterogeneous resources. Building on this, the proposed heterogeneous information fusion module enables cross-scale aggregation among models of different widths, thereby enhancing the utilization of diverse local knowledge. Extensive experiments on three public
benchmarks demonstrate that SFedHIFI can effectively enable heterogeneous SFL, consistently outperforming all three baseline methods. Compared with ANN-based FL, it achieves significant energy savings with only a marginal trade-off in accuracy.
Read the full abstract ▾
All authors Ran Tao, Qiugang Zhan, Shantian Yang, Xiurui Xie, Qi Tian, Guisong Liu
Machine Learning
FairMed-XGB: A Bayesian-Optimised Multi-Metric Framework with Explainability for Demographic Equity in Critical Healthcare Data
Mitul Goswami, Romit Chatterjee and Arif Ahmed Sekh
Machine learning models deployed in critical care settings exhibit demographic biases, particularly gender disparities, that undermine clinical trust and equitable treatment. This paper introduces FairMed-XGB, a novel framework that systematically detects and mitigates gender-based prediction bias while preserving model performance and transparency. The framework integrates a
fairness-aware loss function combining Statistical Parity Difference, Theil Index, and Wasserstein Distance, jointly optimised via
Bayesian Search into an XGBoost classifier. Post-mitigation evaluation on seven clinically distinct cohorts derived from the MIMIC-IV-ED and eICU databases demonstrates substantial bias reduction: Statistical Parity Difference decreases by 40 to 51 percent on MIMIC-IV-ED and 10 to 19 percent on eICU; Theil Index collapses by four to five orders of magnitude to near-zero values; Wasserstein Distance is reduced by 20 to 72 percent. These gains are achieved with negligible degradation in predictive accuracy (AUC-ROC drop <0.02). SHAP-based explainability reveals that the framework diminishes reliance on gender-proxy features, providing clinicians with actionable insights into how and where bias is corrected. FairMed-XGB offers a robust, interpretable, and ethically aligned solution for equitable clinical decision-making, paving the way for trustworthy deployment of AI in high-stakes healthcare environments.
Read the full abstract ▾
Machine Learning
Spiking Layer-Adaptive Magnitude-based Pruning
Junqiao Wang, Zhehang Ye and Yuqi Ouyang
Spiking Neural Networks (SNNs) provide energy-efficient computation but their deployment is constrained by dense connectivity and high spiking operation costs. Existing magnitude-based
pruning strategies, when naively applied to SNNs, fail to account for temporal accumulation, non-uniform timestep contributions, and membrane stability, often leading to severe performance degradation. This paper proposes Spiking Layer-Adaptive Magnitude-based
Pruning (SLAMP), a theory-guided
pruning framework that generalizes layer-adaptive magnitude
pruning to temporal SNNs by explicitly controlling worst-case output distortion across layers and timesteps. SLAMP formulates sparsity allocation as a temporal distortion-constrained optimization problem, yielding time-aware layer importance scores that reduce to conventional layer-adaptive
pruning in single-timestep limit. An efficient two-stage procedure is derived, combining temporal score estimation, global sparsity allocation, and magnitude
pruning with retraining for stability recovery. Experiments on CIFAR10, CIFAR100, and the event-based CIFAR10-DVS
datasets demonstrate that SLAMP achieves substantial connectivity and spiking operation reductions while preserving accuracy, enabling efficient and deployable SNN inference.
Read the full abstract ▾
Machine Learning
Ultra-Early Prediction of Tipping Points: Integrating Dynamical Measures with Reservoir Computing
Xin Li, Qunxi Zhu, Chengli Zhao et al. (+4 more)
Complex dynamical systems-such as
climate, ecosystems, and economics-can undergo catastrophic and potentially irreversible regime changes, often triggered by environmental parameter drift and
stochastic disturbances. These critical thresholds, known as tipping points, pose a prediction problem of both theoretical and practical significance, yet remain largely unresolved. To address this, we articulate a model-free framework that integrates the measures characterizing the stability and sensitivity of dynamical systems with the reservoir computing (RC), a lightweight machine learning technique, using only observational time series data. The framework consists of two stages. The first stage involves using RC to robustly learn local complex dynamics from observational data segmented into windows. The second stage focuses on accurately detecting early warning signals of tipping points by analyzing the learned autonomous RC dynamics through dynamical measures, including the dominant
eigenvalue of the Jacobian
matrix, the maximum Floquet multiplier, and the maximum Lyapunov exponent. Furthermore, when these dynamical measures exhibit trend-like patterns, their extrapolation enables ultra-early prediction of tipping points significantly prior to the occurrence of critical transitions. We conduct a rigorous theoretical analysis of the proposed method and perform extensive numerical evaluations on a series of representative synthetic systems and eight real-world
datasets, as well as quantitatively predict the tipping time of the Atlantic Meridional Overturning Circulation system. Experimental results demonstrate that our framework exhibits advantages over the baselines in comprehensive evaluations, particularly in terms of dynamical interpretability, prediction stability and robustness, and ultra-early prediction capability.
Read the full abstract ▾
All authors Xin Li, Qunxi Zhu, Chengli Zhao, Bolin Zhao, Xue Zhang, Xiaojun Duan, Wei Lin
Machine Learning · Cross-listed from eess.SY
Intelligent Control of Differential Drive Robots Subject to Unmodeled Dynamics with EKF-based State Estimation
Amos Alwala, Yuchen Hu, Gabriel da Silva Lima et al. (+1 more)
Reliable control and state estimation of differential drive
robots (DDR) operating in dynamic and uncertain environments remains a challenge, particularly when system dynamics are partially unknown and sensor measurements are prone to degradation. This work introduces a unified control and state estimation framework that combines a Lyapunov-based nonlinear controller and Adaptive Neural Networks (ANN) with Extended Kalman Filter (EKF)-based multi-sensor fusion. The proposed controller leverages the universal approximation property of neural networks to model unknown nonlinearities in real time. An online adaptation scheme updates the weights of the radial basis function (RBF), the architecture chosen for the ANN. The learned dynamics are integrated into a feedback linearization (FBL) control law, for which theoretical guarantees of closed-loop stability and asymptotic convergence in a trajectory-tracking task are established through a Lyapunov-like stability analysis. To ensure robust state estimation, the EKF fuses inertial measurement unit (IMU) and odometry from
monocular, 2D-
LiDAR and wheel encoders. The fused state estimate drives the intelligent controller, ensuring consistent performance even under drift, wheel slip, sensor noise and failure. Gazebo simulations and real-world experiments are done using DDR, demonstrating the effectiveness of the approach in terms of improved velocity tracking performance with reduction in linear and angular velocity errors up to $53.91\%$ and $29.0\%$ in comparison to the baseline FBL.
Read the full abstract ▾
All authors Amos Alwala, Yuchen Hu, Gabriel da Silva Lima, Wallace Moreira Bessa
Machine Learning
LLM as Graph Kernel: Rethinking Message Passing on Text-Rich Graphs
Ying Zhang, Hang Yu, Haipeng Zhang et al. (+1 more)
Text-rich graphs, which integrate complex structural dependencies with abundant textual information, are ubiquitous yet remain challenging for existing learning paradigms. Conventional methods and even LLM-hybrids compress rich text into static
embeddings or summaries before structural
reasoning, creating an information bottleneck and detaching updates from the raw content. We argue that in text-rich graphs, the text is not merely a node attribute but the primary medium through which structural relationships are manifested. We introduce RAMP, a Raw-text Anchored Message Passing approach that moves beyond using LLMs as mere feature extractors and instead recasts the LLM itself as a graph-native aggregation operator. RAMP exploits the text-rich nature of the graph via a novel dual-representation scheme: it anchors inference on each node's raw text during each iteration while propagating dynamically optimized messages from neighbors. It further handles both discriminative and generative tasks under a single unified generative formulation. Extensive experiments show that RAMP effectively bridges the gap between graph propagation and deep text
reasoning, achieving competitive performance and offering new insights into the role of LLMs as graph kernels for general-purpose graph learning.
Read the full abstract ▾
All authors Ying Zhang, Hang Yu, Haipeng Zhang, Peng Di
Machine Learning · Cross-listed from cs.GR
Masked BRep Autoencoder via Hierarchical Graph Transformer
Yifei Li, Kang Wu, Wenming Wu et al. (+1 more)
We introduce a novel self-supervised learning framework that automatically learns representations from input computer-aided design (CAD) models for downstream tasks, including part classification, modeling
segmentation, and machining feature recognition. To train our network, we construct a large-scale, unlabeled
dataset of boundary representation (BRep) models. The success of our
algorithm relies on two keycomponents. The first is a masked graph autoencoder that reconstructs randomly masked geometries and attributes of BReps for representation learning to enhance the generalization. The second is a hierarchical graph
Transformer architecture that elegantly fuses global and local learning by a cross-scale mutual
attention block to model long-range
geometric dependencies and a graph neural network block to aggregate local topological information. After training the autoencoder, we replace its decoder with a task-specific network trained on a small amount of labeled data for downstream tasks. We conduct experiments on various tasks and achieve high performance, even with a small amount of labeled data, demonstrating the practicality and generalizability of our model. Compared to other methods, our model performs significantly better on downstream tasks with the same amount of training data, particularly when the training data is very limited.
Read the full abstract ▾
All authors Yifei Li, Kang Wu, Wenming Wu, Xiao-Ming Fu
Machine Learning
Kevin Taylor
We introduce directional routing, a lightweight mechanism that gives each
transformer attention head learned suppression directions controlled by a shared router, at 3.9% parameter cost. We train a 433M-parameter model alongside an identical baseline in a single run, then trace the resulting
circuits through mechanistic interpretability. Routing becomes the model's dominant computational pathway. Disabling it collapses factual recall to near-zero probability across all 8 test
prompts and drops induction accuracy from 93.4% to 0.0%. Knocking out individual
attention heads has negligible effect: the primary mover head's removal actually increases target probability, and induction heads retain 98.6% accuracy without their strongest member. The coordination mechanism is irreplaceable; the components it coordinates are not. The model also self-organizes, without explicit pressure, into two regimes: domain-adaptive routing in early layers and fixed syntactic
pruning in late layers, where the least-varying layer is the most critical (+42.6 PPL when disabled). Routing reduces perplexity 31-56% relative to the baseline, though downstream multiple-choice
benchmarks do not yet reflect these gains.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Bayesian Symbolic Regression for Missing Physics
Arno Strouwen
Model-based approaches for (bio)process systems often suffer from incomplete knowledge of the underlying physical, chemical, or biological laws. Universal differential equations, which embed neural networks within differential equations, have emerged as powerful tools to learn this missing physics from experimental data. However, neural networks are inherently opaque, motivating their post-processing via symbolic regression to obtain interpretable mathematical expressions. Genetic
algorithm-based symbolic regression is a popular approach for this post-processing step, but provides only point estimates and cannot quantify the confidence we should place in a discovered equation. We address this limitation by applying
Bayesian symbolic regression, which uses Reversible Jump Markov Chain Monte Carlo to sample from the posterior distribution over symbolic expression trees. This approach naturally quantifies uncertainty in the recovered model structure. We demonstrate the methodology on a Lotka-Volterra predator-prey system and then show how a well-designed experiment leads to lower uncertainty in a fed-batch bioreactor case study.
Read the full abstract ▾
Machine Learning · Cross-listed from eess.AS
Spectrogram features for audio and speech analysis
Ian McLoughlin, Lam Pham, Yan Song et al. (+7 more)
Spectrogram-based representations have grown to dominate the feature space for deep learning audio analysis systems, and are often adopted for
speech analysis also. Initially, the primary motivator for spectrogram-based representations was their ability to present sound as a two dimensional signal in the time-frequency plane, which not only provides an interpretable physical basis for analysing sound, but also unlocks the use of a wide range of machine learning techniques such as convolutional neural networks, that had been developed for image processing. A spectrogram is a
matrix characterised by the resolution and span of its two dimensions, as well as by the representation and scaling of each element. Many possibilities for these three characteristics have been explored by researchers across numerous application areas, with different settings showing affinity for various tasks. This paper reviews the use of spectrogram-based representations and surveys the state-of-the-art to question how front-end feature representation choice allies with back-end classifier architecture for different tasks.
Read the full abstract ▾
All authors Ian McLoughlin, Lam Pham, Yan Song, Xiaoxiao Miao, Huy Phan, Pengfei Cai, Qing Gu, Jiang Nan, Haoyu Song, Donny Soh