Machine Learning · Cross-listed from cs.CV
KLO-Net: A Dynamic K-NN Attention U-Net with CSP Encoder for Efficient Prostate Gland Segmentation from MRI
Anning Tian, Byunghyun Ko, Kaichen Qu et al. (+2 more)
Real-time deployment of prostate MRI
segmentation on clinical workstations is often bottlenecked by computational load and
memory footprint. Deep learning-based prostate gland
segmentation approaches remain challenging due to anatomical variability. To bridge this efficiency gap while still maintaining reliable
segmentation accuracy, we propose KLO-Net, a dynamic K-Nearest Neighbor
attention U-Net with Cross Stage Partial, i.e., CSP, encoder for efficient prostate gland
segmentation from MRI scan. Unlike the regular K-NN
attention mechanism, the proposed dynamic K-NN
attention mechanism allows the model to adaptively determine the number of
attention connections for each spatial location within a slice. In addition, CSP blocks address the computational load to reduce
memory consumption. To evaluate the model's performance, comprehensive experiments and ablation studies are conducted on two public
datasets, i.e., PROMISE12 and PROSTATEx, to validate the proposed architecture. The detailed comparative analysis demonstrates the model's advantage in computational efficiency and
segmentation quality.
Read the full abstract ▾
All authors Anning Tian, Byunghyun Ko, Kaichen Qu, Mengyuan Liu, Jeongkyu Lee
Machine Learning
Let's (not) just put things in Context: Test-Time Training for Long-Context LLMs
Rachit Bansal, Aston Zhang, Rishabh Tiwari et al. (+8 more)
Progress on training and architecture strategies has enabled LLMs with millions of
tokens in context length. However, empirical evidence suggests that such long-context LLMs can consume far more text than they can reliably use. On the other hand, it has been shown that inference-time compute can be used to scale performance of LLMs, often by generating thinking
tokens, on challenging tasks involving multi-step
reasoning. Through controlled experiments on sandbox long-context tasks, we find that such inference-time strategies show rapidly diminishing returns and fail at long context. We attribute these failures to score dilution, a phenomenon inherent to static self-
attention. Further, we show that current inference-time strategies cannot retrieve relevant long-context signals under certain conditions. We propose a simple method that, through targeted gradient updates on the given context, provably overcomes limitations of static self-
attention. We find that this shift in how inference-time compute is spent leads to consistently large performance improvements across models and long-context
benchmarks. Our method leads to large 12.6 and 14.1 percentage point improvements for Qwen3-4B on average across subsets of LongBench-v2 and ZeroScrolls
benchmarks. The takeaway is practical: for long context, a small amount of context-specific training is a better use of inference compute than current inference-time scaling strategies like producing more thinking
tokens.
Read the full abstract ▾
All authors Rachit Bansal, Aston Zhang, Rishabh Tiwari, Lovish Madaan, Sai Surya Duvvuri, Devvrit Khatri, David Brandfonbrener, David Alvarez-Melis, Prajjwal Bhargava, Mihir Sanjay Kale, Samy Jelassi
Machine Learning · Cross-listed from stat.ML
One Permutation Is All You Need: Fast, Deterministic Feature Importance and Model Stress-Testing
Albert Dorador
Reliable estimation of feature contributions in machine learning models is essential for transparency, algorithmic
fairness, and regulatory compliance. While permutation feature importance is widely used, classical implementations rely on repeated Monte Carlo shuffling, introducing significant computational overhead and
stochastic instability. In this paper, we show that replacing $B$ random permutations with a single, max-min rank-optimal deterministic permutation maintains or improves correlation with ground-truth importance while eliminating estimation variance and reducing complexity from $O(B \cdot n \cdot p)$ to $O(n \cdot p)$. Under location-scale feature distributions, we formally prove exact recovery of scale-adjusted linear regression coefficients, alongside improved importance estimation under concave model sensitivity. We extend this deterministic framework along two complementary dimensions. First, Systemic Feature Importance (SFI) integrates empirical feature correlations to quantify indirect feature reliance through proxy variables. Second, Importance Direction extends scalar importance to a signed, directional representation by measuring concordance between
covariate displacements and output shifts. Extensive empirical validation across nearly 200 simulation scenarios demonstrates superior bias-variance trade-offs in high-dimensional and low signal-to-noise regimes. Finally, two real-world credit risk case studies show how coupling SFI with Importance Direction enables practitioners and regulators to audit models for both the magnitude and net sign of hidden reliance on protected attributes, delivering a principled, transparent, and scalable framework for model governance.
Read the full abstract ▾
Machine Learning · Cross-listed from quant-ph
Group-Theoretic Reinforcement Learning of Dynamical Decoupling Sequences
Charles Marrder, Shuo Sun and Murray J. Holland
Dynamical decoupling seeks to mitigate phase decoherence in
qubits by applying a carefully designed sequence of effectively instantaneous electromagnetic pulses. Although analytic solutions exist for pulse timings that are optimal under specific noise regimes, identifying the optimal timings for a realistic noise spectrum remains challenging. We propose a reinforcement learning (RL)-based method for designing pulse sequences on
qubits. Our novel action set enables the RL
agent to efficiently navigate this inherently non-convex optimization landscape. The action set, derived from Thompson's group $F$, is applicable to a broad class of sequential decision problems whose states can be represented as bounded sequences. We demonstrate that our RL
agent can learn pulse sequences that minimize dephasing without requiring explicit knowledge of the underlying noise spectrum. This work opens the possibility for real-time learning of optimal dynamical decoupling sequences on
qubits which are dephasing-limited. The model-free nature of our
algorithm suggests that the
agent may ultimately learn optimal pulse sequences even in the presence of unmodeled physical effects, such as pulse errors or non-Gaussian noise.
Read the full abstract ▾
Machine Learning
OPTIMA: Optimal One-shot Pruning for LLMs via Quadratic Programming Reconstruction
Mohammad Mozaffari, Samuel Kushnir, Maryam Mehri Dehnavi et al. (+1 more)
Post-training
pruning cuts the inference cost of large
language models, yet it faces a trade-off: simple heuristics that zero weights are fast but degrade accuracy, while principled joint optimization methods recover accuracy but are computationally infeasible at modern scale. One-shot methods such as SparseGPT sit in between and apply efficient but approximate weight updates. To close this gap, we introduce OPTIMA, a practical one-shot post-training
pruning method that balances accuracy and scalability. OPTIMA casts layer-wise weight reconstruction after mask selection as independent, column-wise quadratic programs (QPs) that share one Hessian per weight
matrix. Solving these QPs yields, at solver convergence, the per-column minimizer of the layer-wise reconstruction objective under the shared Hessian, and the shared Hessian makes the QPs easy to batch on accelerators. We implement an accelerator-friendly QP solver that accumulates one Hessian per weight
matrix and solves many small QPs in parallel with a convergence certificate per column, enabling one-shot post-training
pruning at scale on a single accelerator without fine-tuning. OPTIMA integrates with existing mask selectors and improves average zero-shot accuracy in 40 of 49 (model, mask selector, sparsity) settings across the LLaMA and Gemma families, with mean gains of +0.61, +0.56, and +1.01 points at 50% unstructured, 2:4, and 60% unstructured sparsity, and up to +3.97 points at 60% sparsity, while matching ADMM-based exact reconstruction on its own masks. On an NVIDIA H100, OPTIMA prunes an 8B-parameter
transformer end-to-end in 40 hours and 60 GB of peak GPU
memory. This shows that principled column-wise QP reconstruction is a practical alternative to heuristic weight updates in one-shot post-training
pruning.
Read the full abstract ▾
All authors Mohammad Mozaffari, Samuel Kushnir, Maryam Mehri Dehnavi, Amir Yazdanbakhsh
Machine Learning
Privacy-Enhancing Infant Cry Classification with Federated Transformers and Denoising Regularization
Geofrey Owino and Bernard Shibwabo
Infant cry classification can aid early assessment of infant needs. However, deployment of such solutions is limited by
privacy concerns around audio data, sensitivity to background noise, and domain shift across recording environments. We present an end-to-end infant cry analysis pipeline that integrates a denoising autoencoder (DAE), a convolutional tokenizer, and a
Transformer encoder trained using communication-efficient
federated learning (FL). The system performs on-device denoising, adaptive
segmentation, post hoc calibration, and energy-based out-of-distribution (OOD) abstention.
Federated training employs a regularized control variate update with 8-bit adapter deltas under secure aggregation. Using the Baby Chillanto and Donate-a-Cry
datasets with ESC-50 noise overlays, the model achieves a macro F1 score of 0.938, an AUC of 0.962, and an Expected Calibration Error (ECE) of 0.032, while reducing per-round client upload from approximately 36 to 42 MB to 3.3 MB. Real-time edge inference on an NVIDIA Jetson Nano (4 GB, TensorRT FP16) achieves 96 ms per one-second spectrogram frame. These results demonstrate a practical path toward
privacy-preserving, noise-robust, and communication-efficient infant cry classification suitable for
federated deployment.
Read the full abstract ▾
Machine Learning
Measuring Uncertainty Calibration
Kamil Ciosek, Nicolò Felicioni, Sina Ghiassian et al. (+6 more)
We make two contributions to the problem of estimating the $L_1$ calibration error of a binary classifier from a finite
dataset. First, we provide an upper bound for any classifier where the calibration function has bounded variation. Second, we provide a method of modifying any classifier so that its calibration error can be upper bounded efficiently without significantly impacting classifier performance and without any restrictive assumptions. All our results are non-asymptotic and distribution-free. We conclude by providing advice on how to measure calibration error in practice. Our methods yield practical procedures that can be run on real-world
datasets with modest overhead.
Read the full abstract ▾
All authors Kamil Ciosek, Nicolò Felicioni, Sina Ghiassian, Juan Elenter Litwin, Francesco Tonolini, David Gustafsson, Eva Garcia-Martin, Carmen Barcena Gonzalez, Raphaëlle Bertrand-Lalo
Machine Learning · Cross-listed from eess.SP
Do Spatial Descriptors Improve Multi-DoF Finger Movement Decoding from HD sEMG?
Ricardo Gonçalves Molinari and Leonardo Abdala Elias
Restoring hand function requires simultaneous and proportional control (SPC) of multiple degrees of freedom (DoFs). This study evaluated the multichannel linear descriptors-based block field method (MLD-BFM) against conventional feature extraction approaches for continuous decoding of five finger-joint DoFs using high-density surface electromyography (HD sEMG). Twenty-one healthy participants performed dynamic sinusoidal finger movements while HD sEMG signals were recorded from the proximal forearm. MLD-BFM extracted spatial descriptors including effective field strength ($\Sigma$), field-strength variation rate ($\Phi$), and spatial complexity ($\Omega$). Performance was optimized (block size: $2\times2$; window: 0.15,s) and compared with conventional time-domain features, root mean square (RMS) and mean absolute value plus waveform length (MAV-WL), as well as dimensionality reduction methods (PCA and NMF), using multi-output regression models. MLD-BFM achieved the highest mean variance-weighted coefficient of determination ($\mathrm{R}^2_\mathrm{vw}$) across all models, with the multilayer perceptron yielding the best result ($86.68 \pm 0.33 \%$). However, the improvement was not statistically significant relative to time-domain features, suggesting that dense multichannel recordings already encode spatial information through amplitude-based descriptors. MLD-BFM significantly outperformed dimensionality reduction approaches, indicating that preserving the spatial resolution of HD sEMG is critical for accurate multi-DoF finger movement regression.
Read the full abstract ▾
Machine Learning · Cross-listed from eess.SY
Safe Online Control-Informed Learning
Tianyu Zhou, Zihao Liang, Zehui Lu et al. (+1 more)
This paper proposes a Safe Online Control-Informed Learning framework for safety-critical autonomous systems. The framework unifies optimal control, parameter estimation, and safety constraints into an online learning process. It employs an extended Kalman filter to incrementally update system parameters in real time, enabling robust and data-efficient adaptation under uncertainty. A softplus barrier function enforces constraint satisfaction during learning and control while eliminating the dependence on high-quality initial guesses. Theoretical analysis establishes convergence and safety guarantees, and the framework's effectiveness is demonstrated on cart-pole and
robot-arm systems.
Read the full abstract ▾
All authors Tianyu Zhou, Zihao Liang, Zehui Lu, Shaoshuai Mou
Machine Learning · Cross-listed from cs.AI
EvoLattice: Persistent Internal-Population Evolution through Multi-Alternative Quality-Diversity Graph Representations for LLM-Guided Program Discovery
Kamer Ali Yuksel
Large
language models (LLMs) are increasingly used to evolve programs and multi-
agent systems, yet most existing approaches rely on overwrite-based mutations that maintain only a single candidate at a time. Such methods discard useful variants, suffer from destructive edits, and explore a brittle search space prone to structural failure. We introduce EvoLattice, a framework that represents an entire population of candidate programs or
agent behaviors within a single directed acyclic graph. Each node stores multiple persistent alternatives, and every valid path through the graph defines a distinct executable candidate, yielding a large combinatorial search space without duplicating structure. EvoLattice enables fine-grained alternative-level evaluation by scoring each alternative across all paths in which it appears, producing statistics that reveal how local design choices affect global performance. These statistics provide a dense, data-driven feedback signal for LLM-guided mutation, recombination, and
pruning, while preserving successful components. Structural correctness is guaranteed by a deterministic self-repair mechanism that enforces acyclicity and dependency consistency independently of the LLM. EvoLattice naturally extends to
agent evolution by interpreting alternatives as
prompt fragments or sub-
agent behaviors. Across program synthesis (proxy and optimizer meta-learning), EvoLattice yields more stable
evolution, greater expressivity, and stronger improvement trajectories than prior LLM-guided methods. The resulting dynamics resemble quality-diversity optimization, emerging implicitly from EvoLattice's internal multi-alternative representation rather than an explicit external archive.
Read the full abstract ▾
Machine Learning
Dropout Neural Network Training Viewed from a Percolation Perspective
Finley Devlin and Jaron Sanders
In this work, we investigate the existence and effect of percolation in training deep Neural Networks (NNs) with dropout. Dropout methods are regularisation techniques for training NNs, first introduced by G. Hinton et al. (2012). These methods temporarily remove connections in the NN, randomly at each stage of training, and update the remaining subnetwork with
Stochastic Gradient Descent (SGD). The process of removing connections from a network at random is similar to percolation, a paradigm model of statistical physics. If dropout were to remove enough connections such that there is no path between the input and output of the NN, then the NN could not make predictions informed by the data. We study new percolation models that mimic dropout in NNs and characterise the relationship between network
topology and this path problem. The theory shows the existence of a percolative effect in dropout. We also show that this percolative effect can cause a breakdown when training NNs without biases with dropout; and we argue heuristically that this breakdown extends to NNs with biases.
Read the full abstract ▾
Machine Learning
Topologically-Stabilized Graph Neural Networks: Empirical Robustness Across Domains
Jelena Losic
Graph Neural Networks (GNNs) have become the standard for graph representation learning but remain vulnerable to structural perturbations. We propose a novel framework that integrates persistent homology features with stability regularization to enhance robustness. Building on the stability
theorems of persistent homology \cite{cohen2007stability}, our method combines GIN architectures with multi-scale topological features extracted from persistence images, enforced by Hiraoka-Kusano-inspired stability constraints. Across six diverse
datasets spanning biochemical, social, and collaboration networks , our approach demonstrates exceptional robustness to edge perturbations while maintaining competitive accuracy. Notably, we observe minimal performance degradation (0-4% on most
datasets) under perturbation, significantly outperforming baseline stability. Our work provides both a theoretically-grounded and empirically-validated approach to robust graph learning that aligns with recent advances in topological regularization
Read the full abstract ▾
Machine Learning · Cross-listed from cs.IR
BiCoRec: Bias-Mitigated Context-Aware Sequential Recommendation Model
Mufhumudzi Muthivhi, Terence L van Zyl and Hairong Wang
Sequential recommendation models aim to learn from users evolving preferences. However, current state-of-the-art models suffer from an inherent popularity bias. This study developed a novel framework, BiCoRec, that adaptively accommodates users changing preferences for popular and niche items. Our approach leverages a co-
attention mechanism to obtain a popularity-weighted user sequence representation, facilitating more accurate predictions. We then present a new training scheme that learns from future preferences using a consistency loss function. BiCoRec aimed to improve the recommendation performance of users who preferred niche items. For these users, BiCoRec achieves a 26.00% average improvement in NDCG@10 over state-of-the-art baselines. When ranking the relevant item against the entire collection, BiCoRec achieves NDCG@10 scores of 0.0102, 0.0047, 0.0021, and 0.0005 for the Movies, Fashion, Games and Music
datasets.
Read the full abstract ▾
Machine Learning
Explainable reinforcement learning from human feedback to improve alignment
Shicheng Liu, Siyuan Xu, Wenjie Qiu et al. (+2 more)
A common and effective strategy for humans to improve an unsatisfactory outcome in daily life is to find a cause of this outcome and correct the cause. In this paper, we investigate whether this human improvement strategy can be applied to improving reinforcement learning from human feedback (RLHF) for
alignment of
language models (LMs). In particular, it is observed in the literature that LMs tuned by RLHF can still output unsatisfactory responses. This paper proposes a method to improve the unsatisfactory responses by correcting their causes. Our method has two parts. The first part proposes a post-hoc explanation method to explain why an unsatisfactory response is generated to a
prompt by identifying the training data that lead to this response. We formulate this problem as a constrained combinatorial optimization problem where the objective is to find a set of training data closest to this
prompt-response pair in a feature representation space, and the constraint is that the
prompt-response pair can be decomposed as a convex combination of this set of training data in the feature space. We propose an efficient iterative data selection
algorithm to solve this problem. The second part proposes an unlearning method that improves unsatisfactory responses to some
prompts by unlearning the training data that lead to these unsatisfactory responses and, meanwhile, does not significantly degrade satisfactory responses to other
prompts. Experimental results demonstrate that our
algorithm can improve RLHF.
Read the full abstract ▾
All authors Shicheng Liu, Siyuan Xu, Wenjie Qiu, Hangfan Zhang, Minghui Zhu
Machine Learning · Cross-listed from cond-mat.dis-nn
Machine learning Majorana topology using unsupervised and supervised learning
Jacob Taylor, Haining Pan and Sankar Das Sarma
In unsupervised learning, the training data for deep learning does not come with any labels, thus forcing the
algorithm to discover hidden patterns in the data for discerning useful information. This, in principle, could be a powerful tool in identifying topological order since
topology does not always manifest in obvious physical ways (e.g., topological superconductivity) for its decisive confirmation. The problem, however, is that unsupervised learning is a difficult challenge, necessitating huge computing resources, which may not always work. In the current work, we combine unsupervised and supervised learning to establish that unlabeled (simulated) data in the Majorana splitting in realistic short disordered nanowires may enable not only a distinction between `topological' and `trivial', but also where their crossover happens in the relevant parameter space. This may be a useful tool in identifying
topology in experimental Majorana nanowires.
Read the full abstract ▾
Machine Learning
The Double Life of Code World Models: Provably Unmasking Malicious Behavior Through Execution Traces
Subramanyam Sahoo
Large
language models (LLMs) increasingly generate code with minimal human oversight, raising critical concerns about
backdoor injection and malicious behavior. We present Cross-Trace Verification Protocol (CTVP), a novel AI control framework that verifies untrusted code-generating models through semantic orbit analysis. Rather than directly executing potentially malicious code, CTVP leverages the model's own predictions of execution traces across semantically equivalent program transformations. By analyzing consistency patterns in these predicted traces, we detect behavioral anomalies indicative of
backdoors. Our approach introduces the
Adversarial Robustness Quotient (ARQ), which quantifies the computational cost of verification relative to baseline generation, demonstrating exponential growth with orbit size. Theoretical analysis establishes information-theoretic bounds showing non-gamifiability - adversaries cannot improve through training due to fundamental space complexity constraints. This work demonstrates that semantic orbit analysis provides a theoretically grounded approach to AI control for code generation tasks, though practical deployment requires addressing the high false positive rates observed in initial evaluations.
Read the full abstract ▾
Machine Learning
EEG-D3: A Solution to the Hidden Overfitting Problem of Deep Learning Models
Siegfried Ludwig, Stylianos Bakas, Konstantinos Barmpas et al. (+5 more)
Deep learning for decoding
EEG signals has gained traction, with many claims to state-of-the-art accuracy. However, despite the convincing
benchmark performance, successful translation to real applications is limited. The frequent disconnect between performance on controlled BCI
benchmarks and its lack of generalisation to practical settings indicates hidden overfitting problems. We introduce Disentangled Decoding Decomposition (D3), a weakly supervised method for training deep learning models across
EEG datasets. By predicting the place in the respective trial sequence from which the input window was sampled,
EEG-D3 separates latent components of
brain activity, akin to non-linear ICA. We utilise a novel model architecture with fully independent sub-networks for strict interpretability. We outline a feature interpretation paradigm to contrast the component activation profiles on different
datasets and inspect the associated temporal and spatial filters. The proposed method reliably separates latent components of
brain activity on motor imagery data. Training downstream classifiers on an appropriate subset of these components prevents hidden overfitting caused by task-correlated artefacts, which severely affects end-to-end classifiers. We further exploit the linearly separable latent space for effective few-shot learning on sleep stage classification. The ability to distinguish genuine components of
brain activity from spurious features results in models that avoid the hidden overfitting problem and generalise well to real-world applications, while requiring only minimal labelled data. With interest to the neuroscience community, the proposed method gives researchers a tool to separate individual
brain processes and potentially even uncover heretofore unknown dynamics.
Read the full abstract ▾
All authors Siegfried Ludwig, Stylianos Bakas, Konstantinos Barmpas, Georgios Zoumpourlis, Dimitrios A. Adamos, Nikolaos Laskaris, Yannis Panagakis, Stefanos Zafeiriou
Machine Learning
Constrained Policy Optimization via Sampling-Based Weight-Space Projection
Shengfan Cao, Francesco Borrelli and Eunhyek Joa
Safety-critical learning requires policies that improve performance without leaving the safe operating regime. We study constrained policy learning where model parameters must satisfy rollout-based safety constraints that can be evaluated but not differentiated analytically. We propose SCPO, a sampling-based weight-space projection method that enforces safety directly in parameter space without requiring gradient access to the constraint functions. SCPO constructs a local safe region by combining rollout-based safety evaluations with smoothness bounds relating parameter perturbations to changes in safety metrics, and projects each gradient update via a convex QCQP. We establish a safe-by-induction guarantee: starting from any safe initialization, all intermediate policies remain safe given feasible projections. In constrained control settings with a stabilizing backup policy, SCPO further ensures closed-loop stability while enabling safe adaptation beyond the conservative backup. Experiments on constrained regression with harmful supervision and double-integrator imitation with a malicious expert show that SCPO rejects unsafe updates, maintains feasibility throughout training, and achieves meaningful objective improvement.
Read the full abstract ▾
Machine Learning
Enhancing Semi-Supervised Multi-View Graph Convolutional Networks via Supervised Contrastive Learning and Self-Training
Huaiyuan Xiao, Fadi Dornaika and Jingjun Bi
The advent of graph convolutional network (GCN)-based multi-view learning provides a powerful framework for integrating structural information from heterogeneous views, enabling effective modeling of complex multi-view data. However, existing methods often fail to fully exploit the complementary information across views, leading to suboptimal feature representations and limited performance. To address this, we propose MV-SupGCN, a semi-supervised GCN model that integrates several complementary components with clear motivations and mutual reinforcement. First, to better capture discriminative features and improve model generalization, we design a joint loss function that combines Cross-
Entropy loss with Supervised Contrastive loss, encouraging the model to simultaneously minimize intra-class variance and maximize inter-class separability in the latent space. Second, recognizing the instability and incompleteness of single graph construction methods, we combine both KNN-based and semi-supervised graph construction approaches on each view, thereby enhancing the robustness of the data structure representation and reducing generalization error. Third, to effectively utilize abundant unlabeled data and enhance semantic
alignment across multiple views, we propose a unified framework that integrates contrastive learning in order to enforce consistency among multi-view
embeddings and capture meaningful inter-view relationships, together with pseudo-labeling, which provides additional supervision applied to both the cross-
entropy and contrastive loss functions to enhance model generalization. Extensive experiments demonstrate that MV-SupGCN consistently surpasses state-of-the-art methods across multiple
benchmarks, validating the effectiveness of our integrated approach. The source code is available at https://github.com/HuaiyuanXiao/MVSupGCN
Read the full abstract ▾
Machine Learning · Cross-listed from eess.IV
Towards Deep Learning Surrogate for the Forward Problem in Electrocardiology: A Scalable Alternative to Physics-Based Models
Shaheim Ogbomo-Harmitt, Cesare Magnetti, Chiara Spota et al. (+2 more)
The forward problem in electrocardiology, computing body surface potentials from cardiac electrical activity, is traditionally solved using physics-based models such as the bidomain or monodomain equations. While accurate, these approaches are computationally expensive, limiting their use in real-time and large-scale clinical applications. We propose a proof-of-concept deep learning (DL) framework as an efficient surrogate for forward solvers. The model adopts a time-dependent,
attention-based sequence-to-sequence architecture to predict electrocardiogram (ECG) signals from cardiac voltage propagation maps. A hybrid loss combining Huber loss with a spectral
entropy term was introduced to preserve both temporal and frequency-domain fidelity. Using 2D tissue simulations incorporating healthy, fibrotic, and gap junction-remodelled conditions, the model achieved high accuracy (mean $R^2 = 0.99 \pm 0.01$). Ablation studies confirmed the contributions of convolutional encoders, time-aware
attention, and spectral
entropy loss. These findings highlight DL as a scalable, cost-effective alternative to physics-based solvers, with potential for clinical and digital twin applications.
Read the full abstract ▾
All authors Shaheim Ogbomo-Harmitt, Cesare Magnetti, Chiara Spota, Jakub Grzelak, Oleg Aslanidi
Machine Learning · Cross-listed from cs.AI
Mathematics and Coding are Universal AI Benchmarks
Przemyslaw Chojecki
We study the special role of mathematics and coding inside the
moduli space of psychometric batteries for AI
agents. Building on the AAI framework and GVU dynamics from previous works, we define the Mathematics Fiber and show that, when paired with formal proof kernels (e.g. Lean, Coq), GVU flows on this fiber admit spectrally stable self-improvement regimes due to oracle-like verification. Our main technical result is a density
theorem: under uniform tightness of
agent outputs and a Lipschitz AAI functional, the subspace of batteries generated by mathematical
theorem-proving and coding tasks is dense in the
moduli space of batteries with respect to the evaluation metric. Coding alone is universal in this sense, while pure mathematics is not; its privilege is spectral rather than expressive. We interpret this as evidence that mathematics and coding provide “universal coordinates” for evaluation, and that formal mathematics is a natural ignition domain for recursive self-improvement in advanced AI
agents.
Read the full abstract ▾
Machine Learning
Network-Wide Traffic Volume Estimation from Speed Profiles using a Spatio-Temporal Graph Neural Network with Directed Spatial Attention
Léo Hein, Giovanni de Nunzio, Giovanni Chierchia et al. (+2 more)
Existing traffic volume estimation methods typically address either
forecasting traffic on sensor-equipped roads or spatially imputing missing volumes using nearby sensors. While
forecasting models generally disregard unmonitored roads by design, spatial imputation methods explicitly address network-wide estimation; yet this approach relies on volume data at inference time, limiting its applicability in sensor-scarce cities. Unlike traffic volume data, probe vehicle speeds and static road attributes are more broadly accessible and support full coverage of road segments in most urban networks. In this work, we present the Hybrid Directed-
Attention Spatio-Temporal Graph Neural Network (HDA-STGNN), an inductive deep learning framework designed to tackle the network-wide volume estimation problem. Our approach leverages speed profiles, static road attributes, and road network
topology to predict daily traffic volume profiles across all road segments in the network. To evaluate the effectiveness of our approach, we perform extensive ablation studies that demonstrate the model's capacity to capture complex spatio-temporal dependencies and highlight the value of topological information for accurate network-wide traffic volume estimation without relying on volume data at inference time.
Read the full abstract ▾
All authors Léo Hein, Giovanni de Nunzio, Giovanni Chierchia, Aurélie Pirayre, Laurent Najman
Machine Learning · Cross-listed from eess.IV
Improving the Plausibility of Pressure Distributions Synthesized from Depth Image through Generative Modeling
Neevkumar Manavar, Hanno Gerd Meyer, Joachim Waßmuth et al. (+2 more)
Monitoring contact pressure in hospital beds is essential for preventing pressure ulcers and enabling real-time patient assessment. Current methods can predict pressure maps but often lack physical plausibility, limiting clinical reliability. This work proposes a framework that enhances plausibility via Informed Latent Space (ILS) and Weight Optimization Loss (WOL) with conditional generative modeling to produce high-fidelity, physically consistent pressure estimates. This study also applies
diffusion based conditional Brownian Bridge
Diffusion Model (BBDM) and proposes training strategy for its latent counterpart Latent Brownian Bridge
Diffusion Model (LBBDM) tailored for pressure synthesis in lying postures. Experiment results shows proposed method improves physical plausibility and performance over baselines: BBDM with ILS delivers highly detailed maps at higher computational cost and large inference time, whereas LBBDM provides faster inference with competitive performance. Overall, the approach supports non-invasive,
vision-based, real-time patient monitoring in clinical environments.
Read the full abstract ▾
All authors Neevkumar Manavar, Hanno Gerd Meyer, Joachim Waßmuth, Barbara Hammer, Axel Schneider
Machine Learning · Cross-listed from cs.CV
Time-aware UNet and super-resolution deep residual networks for spatial downscaling
Mika Sipilä, Sabrina Maggio, Sandra De Iaco et al. (+3 more)
Satellite data of atmospheric pollutants are often available only at coarse spatial resolution, limiting their applicability in local-scale environmental analysis and decision-making. Spatial downscaling methods aim to transform the coarse satellite data into high-resolution fields. In this work, two widely used deep learning architectures, the super-resolution deep residual network (SRDRN) and the encoder-decoder-based UNet, are considered for spatial downscaling of tropospheric ozone. Both methods are extended with a lightweight temporal module, which encodes observation time using either sinusoidal or radial basis function (RBF) encoding, and fuses the temporal features with the spatial representations in the networks. The proposed time-aware extensions are evaluated against their baseline counterparts in a case study on ozone downscaling over Italy. The results suggest that, while only slightly increasing computational complexity, the temporal modules significantly improve downscaling performance and convergence speed.
Read the full abstract ▾
All authors Mika Sipilä, Sabrina Maggio, Sandra De Iaco, Klaus Nordhausen, Monica Palma, Sara Taskinen
Machine Learning
MIDUS: Memory-Infused Depth Up-Scaling
Taero Kim, Hoyoon Byun, Youngjun Choi et al. (+2 more)
Expanding pre-trained
language models offers a practical way to increase capacity without training larger models from scratch. Depth Up-Scaling (DUS) does so by duplicating
Transformer blocks and inserting them into a pre-trained backbone. This process also duplicates FFN-heavy blocks, increasing parameter and compute cost while adding capacity through a block-level dense residual branch. Yet prior work suggests that added capacity need not remain tied to dense FFN branches, while
attention heads often play heterogeneous roles, motivating more efficient head-level residual corrections. We propose
Memory-Infused Depth Up-Scaling (MIDUS), which replaces the duplicated FFN branches with
memory layers and turns added depth into lightweight
retrieval-based residual capacity. We introduce a Head-wise
Memory Layer (HML), which combines multi-head product-key
memory with Head-wise Implicit Value Expansion (HIVE). HML assigns each head a distinct key space, while HIVE realizes head-specific values from a shared latent bank through compact projections. Alongside empirical improvements in performance and efficiency, our head-importance and fixed-
retrieval structural analyses characterize HML with HIVE as a structurally distinct, head-conditioned alternative to FFN-based residual expansion.
Read the full abstract ▾
All authors Taero Kim, Hoyoon Byun, Youngjun Choi, Sungrae Park, Kyungwoo Song
Machine Learning
Comparative Evaluation of Embedding Representations for Financial News Sentiment Analysis
Joyjit Roy and Samaresh Kumar Singh
Financial sentiment analysis enhances
market understanding. However, standard Natural
Language Processing (NLP) approaches encounter significant challenges when applied to small
datasets. This study presents a comparative evaluation of
embedding-based techniques for financial news sentiment classification in resource-constrained environments. Word2Vec, GloVe, and sentence
transformer representations are evaluated in combination with gradient boosting on a manually labeled
dataset of 349 financial news headlines. Experimental results identify a substantial gap between validation and test performance. Despite strong validation metrics, models underperform relative to trivial baselines. The analysis indicates that pretrained
embeddings yield diminishing returns below a critical data sufficiency threshold. Small validation sets contribute to overfitting during model selection. Practical application is illustrated through weekly sentiment aggregation and narrative summarization for
market monitoring. Overall, the findings indicate that
embedding quality alone cannot address fundamental data scarcity in sentiment classification. Practitioners with limited labeled data should consider alternative strategies, including few-shot learning, data augmentation, or lexicon-enhanced hybrid methods.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CE
Probabilistic Predictions of Process-Induced Deformation in Carbon/Epoxy Composites Using a Deep Operator Network
Elham Kiyani, Amit Makarand Deshpande, Madhura Limaye et al. (+7 more)
Fiber reinforcement and
polymer matrix respond differently to manufacturing conditions due to mismatch in coefficient of thermal expansion and
matrix shrinkage during curing of thermosets. These heterogeneities generate residual stresses over multiple length scales, whose partial release leads to process-induced deformation (PID), requiring accurate prediction and mitigation via optimized non-isothermal cure cycles. This study considers a unidirectional AS4 carbon fiber/amine bi-functional epoxy prepreg and models PID using a two-mechanism framework that accounts for thermal expansion/shrinkage and cure shrinkage. The model is validated against manufacturing trials to identify initial and boundary conditions, then used to generate PID responses for a diverse set of non-isothermal cure cycles (time-temperature profiles). Building on this physics-based foundation, we develop a data-driven surrogate based on Deep Operator Networks (DeepONets). A DeepONet is trained on a
dataset combining high-fidelity simulations with targeted experimental measurements of PID. We extend this to a Feature-wise Linear Modulation (FiLM) DeepONet, where branch-network features are modulated by external parameters, including the initial degree of cure, enabling prediction of time histories of degree of cure, viscosity, and deformation. Because experimental data are available only at limited time instances (for example, final deformation), we use transfer learning: simulation-trained trunk and branch networks are fixed and only the final layer is updated using measured final deformation. Finally, we augment the framework with Ensemble Kalman Inversion (EKI) to quantify uncertainty under experimental conditions and to support optimization of cure schedules for reduced PID in composites.
Read the full abstract ▾
All authors Elham Kiyani, Amit Makarand Deshpande, Madhura Limaye, Zhiwei Gao, Zongren Zou, Sai Aditya Pradeep, Srikanth Pilla, Gang Li, Zhen Li, George Em Karniadakis
Machine Learning · Cross-listed from cs.CV
DiffusionBrowser: Interactive Diffusion Previews via Multi-Branch Decoders
Susung Hong, Chongjian Ge, Zhifei Zhang et al. (+1 more)
Video diffusion models have revolutionized generative
video synthesis, but they are imprecise, slow, and can be opaque during generation – keeping users in the dark for a prolonged period. In this work, we propose DiffusionBrowser, a model-agnostic, lightweight decoder framework that allows users to interactively generate previews at any point (timestep or
transformer block) during the denoising process. Our model can generate multi-modal preview representations that include RGB and scene intrinsics at more than 4$\times$ real-time speed (less than 1 second for a 4-second
video) that convey consistent appearance and motion to the final
video. With the trained decoder, we show that it is possible to interactively guide the generation at intermediate noise steps via stochasticity reinjection and modal steering, unlocking a new control capability. Moreover, we systematically probe the model using the learned decoders, revealing how scene, object, and other details are composed and assembled during the otherwise black-box denoising process.
Read the full abstract ▾
All authors Susung Hong, Chongjian Ge, Zhifei Zhang, Jui-Hsien Wang
Machine Learning
Directional Textual Inversion for Personalized Text-to-Image Generation
Kunhee Kim, NaHyeon Park, Kibeom Hong et al. (+1 more)
Textual Inversion (TI) is an efficient approach to text-to-image personalization but often fails on complex
prompts. We trace these failures to
embedding norm inflation: learned
tokens drift to out-of-distribution magnitudes, degrading
prompt conditioning in pre-norm
Transformers. Empirically, we show semantics are primarily encoded by direction in CLIP
token space, while inflated norms harm contextualization; theoretically, we analyze how large magnitudes attenuate positional information and hinder residual updates in pre-norm blocks. We propose Directional Textual Inversion (DTI), which fixes the
embedding magnitude to an in-distribution scale and optimizes only direction on the unit hypersphere via Riemannian SGD. We cast direction learning as MAP with a von Mises-Fisher prior, yielding a constant-direction prior gradient that is simple and efficient to incorporate. Across personalization tasks, DTI improves text fidelity over TI and TI-variants while maintaining subject similarity. Crucially, DTI's hyperspherical parameterization enables smooth, semantically coherent interpolation between learned concepts (slerp), a capability that is absent in standard TI. Our findings suggest that direction-only optimization is a robust and scalable path for
prompt-faithful personalization. Code is available at https://github.com/kunheek/dti.
Read the full abstract ▾
All authors Kunhee Kim, NaHyeon Park, Kibeom Hong, Hyunjung Shim
Machine Learning
A Scientific Reasoning Model for Organic Synthesis Procedure Generation
Guoqing Liu, Junren Li, Zihan Zhao et al. (+7 more)
Solving computer-aided synthesis planning is essential for enabling fully automated,
robot-assisted synthesis workflows and improving the efficiency of drug discovery. A key challenge, however, is bridging the gap between computational route design and practical laboratory execution, particularly the accurate prediction of viable experimental procedures for each synthesis step. In this work, we present QFANG, a scientific
reasoning language model capable of generating precise, structured experimental procedures directly from reaction equations, with explicit chain-of-thought
reasoning. To develop QFANG, we curated a high-quality
dataset comprising 905,990 chemical reactions paired with structured action sequences, extracted and processed from patent literature using large
language models. We introduce a Chemistry-Guided
Reasoning (CGR) framework that produces chain-of-thought data grounded in chemical knowledge at scale. The model subsequently undergoes supervised fine-tuning to elicit complex chemistry
reasoning. Finally, we apply Reinforcement Learning from Verifiable Rewards (RLVR) to further enhance procedural accuracy. Experimental results demonstrate that QFANG outperforms advanced general-purpose
reasoning models and nearest-neighbor
retrieval baselines, measured by traditional NLP similarity metrics and a chemically aware evaluator using an LLM-as-a-judge. Moreover, QFANG generalizes to certain out-of-domain reaction classes and adapts to variations in laboratory conditions and user-specific constraints. We believe that QFANG's ability to generate high-quality synthesis procedures represents an important step toward bridging the gap between computational synthesis planning and fully automated laboratory synthesis.
Read the full abstract ▾
All authors Guoqing Liu, Junren Li, Zihan Zhao, Eray Inanc, Krzysztof Maziarz, Jose Garrido Torres, Victor Garcia Satorras, Shoko Ueda, Christopher M. Bishop, Marwin Segler