Machine Learning
An LP-based Sampling Policy for Multi-Armed Bandits with Side-Observations and Stochastic Availability
Ashutosh Soni, Peizhong Ju, Atilla Eryilmaz et al. (+1 more)
We study the
stochastic multi-armed bandit (MAB) problem where an underlying network structure enables side-observations across related actions. We use a bipartite graph to link actions to a set of unknowns, such that selecting an action reveals observations for all the unknowns it is connected to. While previous works rely on the assumption that all actions are permanently accessible, we investigate the more practical setting of
stochastic availability, where the set of feasible actions (the "activation set") varies dynamically in each round. This framework models real-world systems with both structural dependencies and
volatility, such as social networks where users provide side-information about their peers' preferences, yet are not always online to be queried. To address this challenge, we propose UCB-LP-A, a novel policy that leverages a Linear Programming (LP) approach to optimize exploration-exploitation trade-offs under
stochastic availability. Unlike standard network bandit
algorithms that assume constant access, UCB-LP-A computes an optimal sampling distribution over the realizable activation sets, ensuring that the necessary observations are gathered using only the currently active arms. We derive a theoretical upper bound on the regret of our policy, characterizing the impact of both the network structure and the activation probabilities. Finally, we demonstrate through numerical simulations that UCB-LP-A significantly outperforms existing heuristics that ignore either the side-information or the availability constraints.
Read the full abstract ▾
All authors Ashutosh Soni, Peizhong Ju, Atilla Eryilmaz, Ness B. Shroff
Machine Learning
Automatic Laplace Collapsed Sampling: Scalable Marginalisation of Latent Parameters via Automatic Differentiation
Toby Lovick, David Yallup and Will Handley
We present Automatic Laplace Collapsed Sampling (ALCS), a general framework for marginalising latent parameters in
Bayesian models using automatic differentiation, which we combine with nested sampling to explore the hyperparameter space in a robust and efficient manner. At each nested sampling likelihood evaluation, ALCS collapses the high-dimensional latent variables $z$ to a scalar contribution via maximum a posteriori (MAP) optimisation and a Laplace approximation, both computed using autodiff. This reduces the effective dimension from $d_\theta + d_z$ to just $d_\theta$, making
Bayesian evidence computation tractable for high-dimensional settings without hand-derived gradients or Hessians, and with minimal model-specific engineering. The MAP optimisation and Hessian evaluation are parallelised across live points on GPU-hardware, making the method practical at scale. We also show that automatic differentiation enables local approximations beyond Laplace to parametric families such as the Student-$t$, which improves evidence estimates for heavy-tailed latents. We validate ALCS on a suite of
benchmarks spanning hierarchical, time-series, and discrete-likelihood models and establish where the Gaussian approximation holds. This enables a post-hoc ESS diagnostic that localises failures across hyperparameter space without expensive joint sampling.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CR
Machine Learning Transferability for Malware Detection
César Vieira, João Vitorino, Eva Maia et al. (+1 more)
Malware continues to be a predominant operational risk for organizations, especially when obfuscation techniques are used to evade detection. Despite the ongoing efforts in the development of Machine Learning (ML) detection approaches, there is still a lack of feature compatibility in public
datasets. This limits generalization when facing distribution shifts, as well as transferability to different
datasets. This study evaluates the suitability of different data preprocessing approaches for the detection of Portable Executable (PE) files with ML models. The preprocessing pipeline unifies EMBERv2 (2,381-dim) features
datasets, trains paired models under two training setups: EMBER + BODMAS and EMBER + BODMAS + ERMDS. Regarding model evaluation, both EMBER + BODMAS and EMBER + BODMAS + ERMDS models are tested against TRITIUM, INFERNO and SOREL-20M. ERMDS is also used for testing for the EMBER + BODMAS setup.
Read the full abstract ▾
All authors César Vieira, João Vitorino, Eva Maia, Isabel Praça
Machine Learning
Context-specific Credibility-aware Multimodal Fusion with Conditional Probabilistic Circuits
Pranuthi Tenali, Sahil Sidheekh, Saurabh Mathur et al. (+3 more)
Multimodal fusion requires integrating information from multiple sources that may conflict depending on context. Existing fusion approaches typically rely on static assumptions about source reliability, limiting their ability to resolve conflicts when a modality becomes unreliable due to situational factors such as sensor degradation or class-specific corruption. We introduce C$^2$MF, a context-specfic credibility-aware
multimodal fusion framework that models per-instance source reliability using a Conditional Probabilistic
Circuit (CPC). We formalize instance-level reliability through Context-Specific Information Credibility (CSIC), a KL-divergence-based measure computed exactly from the CPC. CSIC generalizes conventional static credibility estimates as a special case, enabling principled and adaptive reliability assessment. To evaluate robustness under cross-modal conflicts, we propose the Conflict
benchmark, in which class-specific corruptions deliberately induce discrepancies between different modalities. Experimental results show that C$^2$MF improves predictive accuracy by up to 29% over static-reliability baselines in high-noise settings, while preserving the interpretability advantages of probabilistic
circuit-based fusion.
Read the full abstract ▾
All authors Pranuthi Tenali, Sahil Sidheekh, Saurabh Mathur, Erik Blasch, Kristian Kersting, Sriraam Natarajan
Machine Learning
Benchmarking Tabular Foundation Models for Conditional Density Estimation in Regression
Rafael Izbicki and Pedro L. C. Rodrigues
Conditional density estimation (CDE) - recovering the full conditional distribution of a response given tabular
covariates - is essential in settings with heteroscedasticity, multimodality, or asymmetric uncertainty. Recent tabular foundation models, such as TabPFN and TabICL, naturally produce predictive distributions, but their effectiveness as general-purpose CDE methods has not been systematically evaluated, unlike their performance for point prediction, which is well studied. We
benchmark three tabular foundation model variants against a diverse set of parametric, tree-based, and neural CDE baselines on 39 real-world
datasets, across training sizes from 50 to 20,000, using six metrics covering density accuracy, calibration, and computation time. Across all sample sizes, foundation models achieve the best CDE loss, log-likelihood, and CRPS on the large majority of
datasets tested. Calibration is competitive at small sample sizes but, for some metrics and
datasets, lags behind task-specific neural baselines at larger sample sizes, suggesting that post-hoc recalibration may be a valuable complement. In a photometric
redshift case study using SDSS DR18, TabPFN exposed to 50,000 training
galaxies outperforms all baselines trained on the full 500,000-
galaxy dataset. Taken together, these results establish tabular foundation models as strong off-the-shelf conditional density
estimators.
Read the full abstract ▾
Machine Learning
Hardware-Aware Tensor Networks for Real-Time Quantum-Inspired Anomaly Detection at Particle Colliders
Sagar Addepalli, Prajita Bhattarai, Abhilasha Dave et al. (+1 more)
Quantum machine learning offers the ability to capture complex correlations in high-dimensional feature spaces, crucial for the challenge of detecting beyond the Standard Model physics in
collider events, along with the potential for unprecedented computational efficiency in future
quantum processors. Near-term utilization of these benefits can be achieved by developing
quantum-inspired
algorithms for deployment in classical hardware to enable applications at the "edge" of current scientific experiments. This work demonstrates the use of tensor networks for real-time anomaly detection in
collider detectors. A spaced
matrix product operator (SMPO) is developed that provides sensitivity to a variety beyond the Standard Model
benchmarks, and can be implemented in field programmable gate array hardware with resources and latency consistent with trigger deployment. The cascaded SMPO architecture is introduced as an SMPO variation that affords greater flexibility and efficiency in ways that are key to edge applications in resource-constrained environments. These results reveal the benefit and near-term feasibility of deploying
quantum-inspired ML in high energy
colliders.
Read the full abstract ▾
All authors Sagar Addepalli, Prajita Bhattarai, Abhilasha Dave, Julia Gonski
Machine Learning · Cross-listed from cs.SE
Sustainability Is Not Linear: Quantifying Performance, Energy, and Privacy Trade-offs in On-Device Intelligence
Eziyo Ehsani, Luca Giamattei, Ivano Malavolta et al. (+1 more)
The migration of Large
Language Models (LLMs) from cloud clusters to edge devices promises enhanced
privacy and offline accessibility, but this transition encounters a harsh reality: the physical constraints of mobile batteries, thermal limits, and, most importantly,
memory constraints. To navigate this landscape, we constructed a replicable and reproducible experimental pipeline to profile the complex interplay between energy consumption, latency, and quality of LLMs on mobile devices. We harness this pipeline to conduct an empirical case study on a flagship Android device, capturing granular metrics across eight LLMs ranging from 0.5B to 9B parameters without requiring root access, ensuring our findings reflect realistic user conditions. The findings highlight the trade-offs between generation quality, performance, power and resource consumption, revealing which LLMs offer the best balance across metrics and under different conditions. Besides, we uncovered a counter-intuitive quantization energy paradox: while modern importance-aware quantization successfully reduces
memory footprints to fit larger models into RAM, we found it yields negligible energy savings compared to standard mixed-precision methods. This proves that for battery life, the architecture of the model, not its quantization scheme, is the decisive factor. We further identified that Mixture-of-Experts (MoE) architectures defy the standard size-energy trend, offering the storage capacity of a 7B model while maintaining the lower energy profile of a 1B to 2B model. Finally, an analysis of these multi-objective trade-offs reveals a pragmatic sweet spot of mid-sized models, such as Qwen2.5-3B, that effectively balance response quality with sustainable energy consumption.
Read the full abstract ▾
All authors Eziyo Ehsani, Luca Giamattei, Ivano Malavolta, Roberto Pietrantuono
Machine Learning
Characterization and forecasting of national-scale solar power ramp events
Luca Lanzilao and Angela Meyer
The rapid growth of
solar energy is reshaping power system operations and increasing the complexity of grid management. As photovoltaic (PV) capacity expands, short-term fluctuations in PV generation introduce substantial operational uncertainty. At the same time,
solar power ramp events intensify risks of grid instability and unplanned outages due to sudden large power fluctuations. Accurate identification,
forecasting and mitigation of
solar ramp events are therefore critical to maintaining grid stability. In this study, we analyze two years of PV power production from 6434 PV stations at 15-minute resolution. We develop quantitative metrics to define
solar ramp events and systematically characterize their occurrence, frequency, and magnitude at a national scale. Furthermore, we examine the meteorological drivers of ramp events, highlighting the role of mesoscale cloud systems. In particular, we observe that ramp-up events are typically associated with cloud dissipation during the morning, while ramp-down events commonly occur when cloud cover increases in the afternoon. Additionally, we adopt a recently developed spatiotemporal
forecasting framework to evaluate both deterministic and probabilistic PV power forecasts derived from deep learning and physics-based models, including SolarSTEPS, SHADECast, IrradianceNet, and IFS-ENS. The results show that SHADECast is the most reliable model, achieving a CRPS 10.8% lower than that of SolarSTEPS at a two-hour lead time. Nonetheless, state-of-the-art nowcasting models struggle to capture ramp dynamics, with forecast RMSE increasing by up to 50% compared to normal operating conditions. Overall, these results emphasize the need for improved high-resolution spatiotemporal modelling to enhance ramp prediction skill and support the reliable integration of large-scale
solar generation into power systems.
Read the full abstract ▾
Machine Learning
PQuantML: A Tool for End-to-End Hardware-aware Model Compression
Roope Niemi, Anastasiia Petrovych, Arghya Ranjan Das et al. (+9 more)
PQuantML is a new open-source, hardware-aware neural network model compression library tailored to end-to-end workflows. Motivated by the need to deploy performant models to environments with strict latency constraints, PQuantML simplifies training of compressed models by providing a unified interface to apply
pruning and quantization, either jointly or individually. The library implements multiple
pruning methods with different granularities, as well as fixed-point quantization with support for High-Granularity Quantization. We evaluate PQuantML on representative tasks such as the jet substructure classification, so-called jet tagging, an on-edge problem related to real-time LHC data processing. Using various
pruning methods with fixed-point quantization, PQuantML achieves substantial parameter and bit-width reductions while maintaining accuracy. The resulting compression is further compared against existing tools, such as QKeras and HGQ.
Read the full abstract ▾
All authors Roope Niemi, Anastasiia Petrovych, Arghya Ranjan Das, Enrico Lupi, Chang Sun, Dimitrios Danopoulos, Marlon Joshua Helbing, Mia Liu, Sebastian Dittmeier, Michael Kagan, Vladimir Loncar, Maurizio Pierini
Machine Learning
Evaluating Interactive 2D Visualization as a Sample Selection Strategy for Biomedical Time-Series Data Annotation
Einari Vaaras, Manu Airaksinen and Okko Räsänen
Reliable machine-learning models in biomedical settings depend on accurate labels, yet annotating biomedical time-series data remains challenging. Algorithmic sample selection may support annotation, but evidence from studies involving real human annotators is scarce. Consequently, we compare three sample selection methods for annotation: random sampling (RND), farthest-first traversal (FAFT), and a graphical user interface-based method enabling exploration of complementary 2D visualizations (2DVs) of high-dimensional data. We evaluated the methods across four classification tasks in infant motility assessment (IMA) and
speech emotion recognition (SER). Twelve annotators, categorized as experts or non-experts, performed data annotation under a limited annotation budget, and post-annotation experiments were conducted to evaluate the sampling methods. Across all classification tasks, 2DV performed best when aggregating labels across annotators. In IMA, 2DV most effectively captured rare classes, but also exhibited greater annotator-to-annotator label distribution variability resulting from the limited annotation budget, decreasing classification performance when models were trained on individual annotators' labels; in these cases, FAFT excelled. For SER, 2DV outperformed the other methods among expert annotators and matched their performance for non-experts in the individual-annotator setting. A failure risk analysis revealed that RND was the safest choice when annotator count or annotator expertise was uncertain, whereas 2DV had the highest risk due to its greater label distribution variability. Furthermore, post-experiment interviews indicated that 2DV made the annotation task more interesting and enjoyable. Overall, 2DV-based sampling appears promising for biomedical time-series data annotation, particularly when the annotation budget is not highly constrained.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
From Synthetic Data to Real Restorations: Diffusion Model for Patient-specific Dental Crown Completion
Dávid Pukanec, Tibor Kubík and Michal Španěl
We present ToothCraft, a
diffusion-based model for the contextual generation of tooth crowns, trained on artificially created incomplete teeth. Building upon recent advancements in conditioned
diffusion models for 3D shapes, we developed a model capable of an automated tooth crown completion conditioned on local anatomical context. To address the lack of training data for this task, we designed an augmentation pipeline that generates incomplete tooth geometries from a publicly available
dataset of complete dental arches (3DS, ODD). By synthesising a diverse set of training examples, our approach enables robust learning across a wide spectrum of tooth defects. Experimental results demonstrate the strong capability of our model to reconstruct complete tooth crowns, achieving an intersection over union (IoU) of 81.8% and a Chamfer Distance (CD) of 0.00034 on synthetically damaged testing restorations. Our experiments demonstrate that the model can be applied directly to real-world cases, effectively filling in incomplete teeth, while generated crowns show minimal intersection with the opposing dentition, thus reducing the risk of occlusal interference. Access to the code, model weights, and
dataset information will be available at: https://github.com/ikarus1211/VISAPP_ToothCraft
Read the full abstract ▾
Machine Learning
The Climber's Grip – Personalized Deep Learning Models for Fear and Muscle Activity in Climbing
Matthias Boeker, Dana Swarbrick, Ulysse T.A. Côté-Allard et al. (+3 more)
Climbing is a multifaceted sport that combines physical demands and emotional and cognitive challenges. Ascent styles differ in fall distance with lead climbing involving larger falls than top rope climbing, which may result in different perceived risk and fear. In this study, we investigated the psychophysiological relationship between perceived fear and muscle activity in climbers using a combination of statistical modeling and deep learning techniques. We conducted an experiment with 19 climbers, collecting electromyography (EMG), electrocardiography (ECG) and arm motion data during lead and top rope climbing. Perceived fear ratings were collected for the different phases of the climb. Using a linear mixed-effects model, we analyzed the relationships between perceived fear and physiological measures. To capture the non-linear dynamics of this relationship, we extended our analysis to deep learning models and integrated random effects for a personalized modeling approach. Our results showed that random effects improved model performance of the mean squared error (MSE), mean absolute error (MAE) and root mean squared error (RMSE). The results showed that muscle fatigue correlates significantly with increased fear during lead climbing. This study highlights the potential of combining statistical and deep learning approaches for modeling the interplay between psychological and physiological states during climbing.
Read the full abstract ▾
All authors Matthias Boeker, Dana Swarbrick, Ulysse T.A. Côté-Allard, Marc T.P. Adam, Hugo L. Hammer, Pål Halvorsen
Machine Learning
Machine Unlearning under Retain-Forget Entanglement
Jingpu Cheng, Ping Liu, Qianxiao Li et al. (+1 more)
Forgetting a subset in machine unlearning is rarely an isolated task. Often, retained samples that are closely related to the forget set can be unintentionally affected, particularly when they share correlated features from pretraining or exhibit strong semantic similarities. To address this challenge, we propose a novel two-phase optimization framework specifically designed to handle such retai-forget entanglements. In the first phase, an augmented Lagrangian method increases the loss on the forget set while preserving accuracy on less-related retained samples. The second phase applies a gradient projection step, regularized by the Wasserstein-2 distance, to mitigate performance degradation on semantically related retained samples without compromising the unlearning objective. We validate our approach through comprehensive experiments on multiple unlearning tasks, standard
benchmark datasets, and diverse neural architectures, demonstrating that it achieves effective and reliable unlearning while outperforming existing baselines in both accuracy retention and removal fidelity.
Read the full abstract ▾
All authors Jingpu Cheng, Ping Liu, Qianxiao Li, Chi Zhang
Machine Learning
Sharp Capacity Scaling of Spectral Optimizers in Learning Associative Memory
Juno Kim, Eshaan Nichani, Denny Wu et al. (+2 more)
Spectral optimizers such as
Muon have recently shown strong empirical performance in large-scale
language model training, but the source and extent of their advantage remain poorly understood. We study this question through the linear associative
memory problem, a tractable model for factual recall in
transformer-based models. In particular, we go beyond orthogonal
embeddings and consider Gaussian inputs and outputs, which allows the number of stored associations to greatly exceed the
embedding dimension. Our main result sharply characterizes the recovery rates of one step of
Muon, SGD, and Newton's method on the logistic regression loss under a power law frequency distribution. We show that the storage capacity of
Muon significantly exceeds that of SGD, and even matches Newton's method while only using first-order information. Moreover,
Muon saturates at a larger critical batch size. We further analyze the multi-step dynamics under a thresholded gradient approximation and show that
Muon achieves a substantially faster initial recovery rate than SGD, while both methods eventually converge to the information-theoretic limit at comparable speeds. Experiments on synthetic tasks validate the predicted scaling laws. Our analysis provides a quantitative understanding of the signal amplification of spectral preconditioners and lays the groundwork for establishing scaling laws across more practical
language modeling tasks and optimizers.
Read the full abstract ▾
All authors Juno Kim, Eshaan Nichani, Denny Wu, Alberto Bietti, Jason D. Lee
Machine Learning
A Lyapunov Analysis of Softmax Policy Gradient for Stochastic Bandits
Tor Lattimore
We adapt the analysis of policy gradient for continuous time $k$-armed
stochastic bandits by Lattimore (2026) to the standard discrete time setup. As in continuous time, we prove that with learning rate $\eta = O(\Delta_{\min}^2/(\Delta_{\max} \log(n)))$ the regret is $O(k \log(k) \log(n) / \eta)$ where $n$ is the horizon and $\Delta_{\min}$ and $\Delta_{\max}$ are the minimum and maximum gaps.
Read the full abstract ▾
Machine Learning · Cross-listed from math.NA
Hyperelastic constitutive model discovery with differentiable finite elements and structure-preserving neural networks
Francesco Regazzoni
The discovery of constitutive laws from experimentally accessible measurements is a central problem in nonlinear computational mechanics. Many data-driven constitutive identification approaches rely either on paired strain-stress data or on full-field displacement measurements, both of which are difficult to obtain in realistic three-dimensional settings. We present a differentiable finite element framework for the discovery of hyperelastic material laws from partial observations, including boundary-only displacement measurements and global reaction forces. The method embeds the nonlinear finite element
equilibrium problem directly into the learning loop, so that candidate strain-energy densities are assessed through the deformation fields and reactions they induce. This formulation enforces mechanical
equilibrium as a constraint and allows the loss function to be evaluated only at observed locations. To ensure physical admissibility and promote numerical solvability throughout training, the constitutive response is represented by Hyperelastic Neural Networks, a structure-preserving neural class that enforces residual energy and stress-free conditions, frame indifference, isotropic material
symmetry, polyconvexity, coercivity, and controlled volumetric growth by construction. The resulting PDE-constrained learning problem is solved using a quasi-Newton strategy combined with continuation and solver-aware backtracking. Numerical experiments in two- and three-dimensional finite elasticity demonstrate accurate recovery of hyperelastic isotropic responses from boundary-only data, robustness to measurement noise, and generalization across geometries, loading conditions, and boundary conditions.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
ALBA: A European Portuguese Benchmark for Evaluating Language and Linguistic Dimensions in Generative LLMs
Inês Vieira, Inês Calvo, Iago Paulo et al. (+6 more)
As Large
Language Models (LLMs) expand across
multilingual domains, evaluating their performance in under-represented
languages becomes increasingly important. European Portuguese (pt-PT) is particularly affected, as existing training data and
benchmarks are mainly in Brazilian Portuguese (pt-BR). To address this, we introduce ALBA, a linguistically grounded
benchmark designed from the ground up to assess LLM proficiency in linguistic-related tasks in pt-PT across eight linguistic dimensions, including
Language Variety, Culture-bound Semantics, Discourse Analysis, Word Plays, Syntax, Morphology, Lexicology, and Phonetics and Phonology. ALBA is manually constructed by
language experts and paired with an LLM-as-a-judge framework for scalable evaluation of pt-PT generated
language. Experiments on a diverse set of models reveal performance variability across linguistic dimensions, highlighting the need for comprehensive, variety-sensitive
benchmarks that support further development of tools in pt-PT.
Read the full abstract ▾
All authors Inês Vieira, Inês Calvo, Iago Paulo, James Furtado, Rafael Ferreira, Diogo Tavares, Diogo Glória-Silva, David Semedo, João Magalhães
Machine Learning · Cross-listed from cs.CL
AMALIA Technical Report: A Fully Open Source Large Language Model for European Portuguese
Afonso Simplício, Gonçalo Vinagre, Miguel Moura Ramos et al. (+19 more)
Despite rapid progress in open large
language models (LLMs), European Portuguese (pt-PT) remains underrepresented in both training data and native evaluation, with machine-translated
benchmarks likely missing the variant's linguistic and cultural nuances. We introduce AMALIA, a fully open LLM that prioritizes pt-PT by using more high-quality pt-PT data during both the mid- and post-training stages. To evaluate pt-PT more faithfully, we release a suite of pt-PT
benchmarks that includes translated standard tasks and four new
datasets targeting pt-PT generation, linguistic competence, and pt-PT/pt-BR bias. Experiments show that AMALIA matches strong baselines on translated
benchmarks while substantially improving performance on pt-PT-specific evaluations, supporting the case for targeted training and native benchmarking for European Portuguese.
Read the full abstract ▾
All authors Afonso Simplício, Gonçalo Vinagre, Miguel Moura Ramos, Diogo Tavares, Rafael Ferreira, Giuseppe Attanasio, Duarte M. Alves, Inês Calvo, Inês Vieira, Rui Guerra, James Furtado, Beatriz Canaverde, Iago Paulo, Vasco Ramos, Diogo Glória-Silva, Miguel Faria, Marcos Treviso, Daniel Gomes, Pedro Gomes, David Semedo, André Martins, João Magalhães
Machine Learning · Cross-listed from q-bio.NC
Identifying Connectivity Distributions from Neural Dynamics Using Flows
Timothy Doyeon Kim, Ulises Pereira-Obilinovic, Yiliu Wang et al. (+2 more)
Connectivity structure shapes neural computation, but inferring this structure from population recordings is degenerate: multiple connectivity structures can generate identical dynamics. Recent work uses low-rank recurrent neural networks (lrRNNs) to infer low-dimensional latent dynamics and connectivity from observed activity, enabling a mechanistic interpretation of the dynamics. However, standard approaches for training lrRNNs can recover spurious structures irrelevant to the underlying dynamics. We first characterize the identifiability of connectivity structures in lrRNNs and determine conditions under which a unique solution exists. To find such solutions, we develop an inference framework based on maximum
entropy and continuous normalizing flows (CNFs), trained via flow matching. Instead of estimating a single connectivity
matrix, our method learns a distribution over connection weights that is maximally unbiased over unidentifiable components while matching the observed dynamics. This approach captures complex yet necessary distributions such as heavy-tailed connectivity found in empirical data. We validate our method on synthetic
datasets with connectivity structures that generate multistable attractors, limit cycles, and ring attractors, and demonstrate its applicability in recordings from rat frontal cortex during decision-making. Our framework shifts
circuit inference from recovering connectivity to identifying which connectivity structures are computationally required, and which are artifacts of underconstrained inference.
Read the full abstract ▾
All authors Timothy Doyeon Kim, Ulises Pereira-Obilinovic, Yiliu Wang, Eric Shea-Brown, Uygar Sümbül
Machine Learning · Cross-listed from astro-ph.IM
Conditional Neural Bayes Ratio Estimation for Experimental Design Optimisation
S. A. K. Leeney, T. Gessey-Jones, W. J. Handley et al. (+3 more)
For frontier experiments operating at the edge of detectability, instrument design directly determines the probability of discovery. We introduce Conditional Neural Bayes Ratio Estimation (cNBRE), which extends neural Bayes ratio estimation by conditioning on design parameters, enabling a single trained network to estimate Bayes factors across a continuous design space. Applied to 21-cm
radio cosmology with simulations representative of the REACH experiment, the amortised nature of cNBRE enables systematic design space exploration that would be intractable with traditional point-wise methods, while recovering established physical relationships. The analysis demonstrates a ~20 percentage point variation in detection probability with
antenna orientation for a single night of observation, a design decision that would be trivial to implement if determined prior to
antenna construction. This framework enables efficient, globally-informed experimental design optimisation for a wide range of scientific applications.
Read the full abstract ▾
All authors S. A. K. Leeney, T. Gessey-Jones, W. J. Handley, E. de Lera Acedo, H. T. J. Bevins, J. L. Tutt
Machine Learning
EcoFair: Energy-Efficient Inference Routing for Edge AI under Data Degradation
Mostafa Anoosha, Dhavalkumar Thakker, Kuniko Paxton et al. (+4 more)
Medical edge-AI systems must operate under a difficult tension: delivering reliable diagnostic inference while running on devices with limited battery capacity,
memory, and compute. In dermatology, this problem is amplified by real-world image degradation caused by smartphone capture, poor lighting, blur, compression, and heterogeneous edge sensors. To handle these degraded inputs, deploying a heavyweight model can improve reliability, but it rapidly increases the energy burden on resource-constrained devices. Conversely, always using a lightweight model saves energy but may be less reliable on ambiguous or degraded inputs. This paper introduces EcoFair, a vertically partitioned inference framework for dermatology classification in which image and tabular inputs remain local to edge clients while only learned modality-specific representations are transmitted for server-side fusion. EcoFair first processes each sample using a lightweight image encoder and then decides whether additional heavyweight computation is necessary. Escalation is triggered when the lightweight prediction exhibits high uncertainty, a narrow separation between safe and high-risk classes, or elevated metadata-derived risk from patient age and lesion location. Across HAM10000, BCN20000, and PAD-UFES-20, EcoFair is evaluated using multiple lightweight–heavy backbone pairings to quantify the trade-off between energy consumption, diagnostic performance, and worst-group malignant-case recall. Results show that EcoFair can reduce per-sample image-inference energy by up to 68% relative to always using the heavyweight encoder, while selectively allocating additional computation under difficult data regimes to support inference reliability. Group-level analysis further shows configuration-dependent effects, with improvements in selected model–
dataset settings and mixed behaviour in others.
Read the full abstract ▾
All authors Mostafa Anoosha, Dhavalkumar Thakker, Kuniko Paxton, Koorosh Aslansefat, Bhupesh Kumar Mishra, Baseer Ahmad, Rameez Raja Kureshi
Machine Learning
SPECTRA: An Efficient Spectral-Informed Neural Network for Sensor-Based Activity Recognition
Deepika Gurung, Lala Shakti Swarup Ray, Mengxi Liu et al. (+2 more)
Real time sensor based applications in pervasive computing require edge deployable models to ensure low latency
privacy and efficient interaction. A
prime example is sensor based human activity recognition where models must balance accuracy with stringent resource constraints. Yet many deep learning approaches treat temporal sensor signals as black box sequences overlooking spectral temporal structure while demanding excessive computation. We present SPECTRA a deployment first co designed spectral temporal architecture that integrates short time
Fourier transform STFT feature extraction depthwise separable convolutions and channel wise self
attention to capture spectral temporal dependencies under real edge runtime and
memory constraints. A compact bidirectional GRU with
attention pooling summarizes within window dynamics at low cost reducing downstream model burden while preserving accuracy. Across five public HAR
datasets SPECTRA matches or approaches larger CNN LSTM and
Transformer baselines while substantially reducing parameters latency and energy. Deployments on a Google Pixel 9 smartphone and an STM32L4 microcontroller further demonstrate end to end deployable realtime private and efficient HAR.
Read the full abstract ▾
All authors Deepika Gurung, Lala Shakti Swarup Ray, Mengxi Liu, Bo Zhou, Paul Lukowicz
Machine Learning
Shapley meets Rawls: an integrated framework for measuring and explaining unfairness
Fadoua Amri-Jouidel, Emmanuel Kemel and Stéphane Mussard
Explainability and
fairness have mainly been considered separately, with recent exceptions trying the explain the sources of unfairness. This paper shows that the Shapley value can be used to both define and explain unfairness, under standard group
fairness criteria. This offers an integrated framework to estimate and derive inference on unfairness as-well-as the features that contribute to it. Our framework can also be extended from Shapley values to the family of Efficient-Symmetric-Linear (ESL) values, some of which offer more robust definitions of
fairness, and shorter computation times. An illustration is run on the Census Income
dataset from the UCI Machine Learning Repository. Our approach shows that “Age", “Number of hours" and “Marital status" generate gender unfairness, using shorter computation time than traditional Bootstrap tests.
Read the full abstract ▾
Machine Learning
Foundation Model for Cardiac Time Series via Masked Latent Attention
Moritz Vandenhirtz, Samuel Ruipérez-Campillo, Simon Böhi et al. (+6 more)
Electrocardiograms (ECGs) are among the most widely available clinical signals and play a central role in cardiovascular diagnosis. While recent foundation models (FMs) have shown promise for learning transferable ECG representations, most existing pretraining approaches treat leads as independent channels and fail to explicitly leverage their strong structural redundancy. We introduce the latent
attention masked autoencoder (LAMAE) FM that directly exploits this structure by learning cross-lead connection mechanisms during self-supervised pretraining. Our approach models higher-order interactions across leads through latent
attention, enabling permutation-invariant aggregation and adaptive weighting of lead-specific representations. We provide empirical evidence on the Mimic-IV-ECG database that leveraging the cross-lead connection constitutes an effective form of structural supervision, improving representation quality and transferability. Our method shows strong performance in predicting ICD-10 codes, outperforming independent-lead masked modeling and
alignment-based baselines.
Read the full abstract ▾
All authors Moritz Vandenhirtz, Samuel Ruipérez-Campillo, Simon Böhi, Sonia Laguna, Irene Cannistraci, Andrea Agostini, Ece Ozkan, Thomas M. Sutter, Julia E. Vogt
Machine Learning · Cross-listed from cs.DC
UNIFERENCE: A Discrete Event Simulation Framework for Developing Distributed AI Models
Doğaç Eldenk and Stephen Xia
Developing and evaluating distributed inference
algorithms remains difficult due to the lack of standardized tools for modeling heterogeneous devices and networks. Existing studies often rely on ad-hoc testbeds or proprietary infrastructure, making results hard to reproduce and limiting exploration of hypothetical hardware or network configurations. We present UNIFERENCE, a discrete-event simulation (DES) framework designed for developing, benchmarking, and deploying distributed AI models within a unified environment. UNIFERENCE models device and network behavior through lightweight logical processes that synchronize only on communication primitives, eliminating rollbacks while preserving the causal order. It integrates seamlessly with PyTorch Distributed, enabling the same codebase to transition from simulation to real deployment. Our evaluation demonstrates that UNIFERENCE profiles runtime with up to 98.6% accuracy compared to real physical deployments across diverse backends and hardware setups. By bridging simulation and deployment, UNIFERENCE provides an accessible, reproducible platform for studying distributed inference
algorithms and exploring future system designs, from high-performance clusters to edge-scale devices. The framework is open-sourced at https://github.com/Dogacel/Uniference.
Read the full abstract ▾
Machine Learning
A Boltzmann-machine-enhanced Transformer For DNA Sequence Classification
Zhixuan Cao, Yishu Xu and Xuang WU
DNA sequence classification requires not only high predictive accuracy but also the ability to uncover latent site interactions, combinatorial regulation, and epistasis-like higher-order dependencies. Although the standard
Transformer provides strong global modeling capacity, its softmax
attention is continuous, dense, and weakly constrained, making it better suited for information routing than explicit structure discovery. In this paper, we propose a Boltzmann-machine-enhanced
Transformer for DNA sequence classification. Built on multi-head
attention, the model introduces structured binary gating variables to represent latent query-key connections and constrains them with a Boltzmann-style energy function. Query-key similarity defines local bias terms, learnable pairwise interactions capture synergy and competition between edges, and latent hidden units model higher-order combinatorial dependencies. Since exact posterior inference over discrete gating graphs is intractable, we use mean-field variational inference to estimate edge activation probabilities and combine it with Gumbel-Softmax to progressively compress continuous probabilities into near-discrete gates while preserving end-to-end differentiability. During training, we jointly optimize classification and energy losses, encouraging the model to achieve accurate prediction while favoring low-energy, stable, and interpretable structures. We further derive the framework from the energy function and variational free energy to the mean-field fixed-point equations, Gumbel-Softmax relaxation, and the final joint objective. The proposed framework provides a unified view of integrating Boltzmann machines, differentiable discrete optimization, and
Transformers for structured learning on biological sequences.
Read the full abstract ▾
Machine Learning
Automatic feature identification in least-squares policy iteration using the Koopman operator framework
Christian Mugisho Zagabe and Sebastian Peitz
In this paper, we present a Koopman autoencoder-based least-squares policy iteration (KAE-LSPI)
algorithm for reinforcement learning (RL). The KAE-LSPI
algorithm is based on reformulating the so-called least-squares fixed-point approximation method in terms of extended dynamic mode decomposition (EDMD), thereby enabling automatic feature learning via the Koopman autoencoder (KAE) framework. The approach is motivated by the lack of a systematic procedure for selecting features or kernels in linear RL techniques. Unlike previous approaches, no features or kernels need to be fixed a priori in our method. We discuss the theoretical convergence of KAE-LSPI under suitable assumptions, similar to kernel-based least-squares policy iteration (KLSPI). We empirically compare our method with two existing
algorithms: least-squares policy iteration (LSPI) and KLSPI. In particular, for the chainwalk problem, the resulting policies are either optimal or nearly optimal for both three approaches. For the cart-pole system, KAE-LSPI outperforms the other two approaches in terms of policy performance.
Read the full abstract ▾
Machine Learning
Neuro-Symbolic Process Anomaly Detection
Devashish Gaikwad, Wil M. P. van der Aalst and Gyunam Park
Process anomaly detection is an important application of process mining for identifying deviations from the normal behavior of a process. Neural network-based methods have recently been applied to this task, learning directly from event logs without requiring a predefined process model. However, since anomaly detection is a purely statistical task, these models fail to incorporate human domain knowledge. As a result, rare but conformant traces are often misclassified as anomalies due to their low frequency, which limits the effectiveness of the detection process. Recent developments in the field of neuro-symbolic AI have introduced Logic Tensor Networks (LTN) as a means to integrate symbolic knowledge into neural networks using real-valued logic. In this work, we propose a neuro-symbolic approach that integrates domain knowledge into neural anomaly detection using LTN and Declare constraints. Using autoencoder models as a foundation, we encode Declare constraints as soft logical guiderails within the learning process to distinguish between anomalous and rare but conformant behavior. Evaluations on synthetic and real-world
datasets demonstrate that our approach improves F1 scores even when as few as 10 conformant traces exist, and that the choice of Declare constraint and by extension human domain knowledge significantly influences performance gains.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.DB
Fair Data Pre-Processing with Imperfect Attribute Space
Ying Zheng, Yangfan Jiang and Kian-Lee Tan
Fair data pre-processing is a widely used strategy for mitigating bias in machine learning. A promising line of research focuses on calibrating
datasets to satisfy a designed
fairness policy so that sensitive attributes influence outcomes only through clearly specified legitimate causal pathways. While effective on clean and information-rich data, these methods often break down in real-world scenarios with imperfect attribute spaces, where decision-relevant factors may be deemed unusable or even missing. To address this gap, we propose LatentPre, a novel framework that enables principled and robust fair data processing in practical settings. Instead of relying solely on observed attributes, LatentPre augments the
fairness policy with latent attributes that capture essential but subtle signals, enabling the framework to operate as if the attribute space were perfect. These latent attributes are strategically introduced to guarantee identifiability and are estimated using a tailored expectation-maximization paradigm. The raw data is then carefully refined to conform to this latent-augmented policy, effectively removing biased patterns while preserving justifiable ones. Extensive experiments demonstrate that LatentPre consistently achieves strong
fairness-utility trade-offs across diverse scenarios, advancing practical
fairness-aware data management.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Meta-Learned Adaptive Optimization for Robust Human Mesh Recovery with Uncertainty-Aware Parameter Updates
Shaurjya Mandal, Nutan Sharma and John Galeotti
Human mesh recovery from single images remains challenging due to inherent depth ambiguity and limited generalization across domains. While recent methods combine regression and optimization approaches, they struggle with poor initialization for test-time refinement and inefficient parameter updates during optimization. We propose a novel meta-learning framework that trains models to produce optimization-friendly initializations while incorporating uncertainty-aware adaptive updates during test-time refinement. Our approach introduces three key innovations: (1) a meta-learning strategy that simulates test-time optimization during training to learn better parameter initializations, (2) a selective parameter caching mechanism that identifies and freezes converged joints to reduce computational overhead, and (3) distribution-based adaptive updates that sample parameter changes from learned distributions, enabling robust exploration while quantifying uncertainty. Additionally, we employ
stochastic approximation techniques to handle intractable gradients in complex loss landscapes. Extensive experiments on standard
benchmarks demonstrate that our method achieves state-of-the-art performance, reducing MPJPE by 10.3 on 3DPW and 8.0 on Human3.6M compared to strong baselines. Our approach shows superior domain adaptation capabilities with minimal performance degradation across different environmental conditions, while providing meaningful uncertainty estimates that correlate with actual prediction errors. Combining meta-learning and adaptive optimization enables accurate mesh recovery and robust generalization to challenging scenarios.
Read the full abstract ▾