Machine Learning
GenOR-Twin: A Semantic Middleware for Integrating Operational Discourse with Mathematical Optimization
Rahimeh Neamatian Monemi, Shahin Gelareh, Lubin Cui et al. (+1 more)
We introduce GenOR-Twin, a neuro-symbolic framework that bridges the translation gap between unstructured operational logs and rigorous mathematical optimization. Our architecture uniquely positions Large
Language Models as semantic translators rather than direct solvers, ensuring that the system retains the feasibility guarantees of exact combinatorial methods. { \color{red}We design a dynamic constraint injection mechanism (the runtime translation of qualitative disruption events into formal mathematical constraints) that allows the system to structurally modify the optimization problem's feasibility region in real-time based on qualitative human inputs. The resulting bidirectional coupling—where operational observations update the virtual model state and optimized decisions are reflected back into the Knowledge Graph—satisfies the synchronization requirement of a proper Digital Twin. The framework features an adaptive decision policy} that automatically selects between low-complexity schedule repair and full re-optimization by analyzing the available system slack. Finally, we demonstrate the generalization of this approach across six distinct optimization domains, {\color{red}turning static models into resilient systems that adapt to the operational uncertainty and variability of real-world environments.}
Read the full abstract ▾
All authors Rahimeh Neamatian Monemi, Shahin Gelareh, Lubin Cui, Nelson Maculan
Machine Learning · Cross-listed from eess.SY
Very Exciting: Zero-Shot Model Predictive Control of Buildings via Excitation-Based Generalized Transfer Learning Models
Fabian Raisch, Felix Koch, Zack Xuereb Conti et al. (+2 more)
The widespread adoption of data-driven, energy-efficient model predictive control (MPC) in buildings remains hindered by substantial effort to collect data and train models for individual buildings. Transfer learning (TL) has consequently gained increasing
attention for target building modeling, as it reduces data requirements and modeling effort by reusing pretrained source models. However, these TL models are typically evaluated only on prediction accuracy in the target, without testing downstream control performance. To address this gap, we apply a state-of-the-art TL approach - pretraining a generalized model on multiple source buildings using standard operational data - within an MPC setup in a target building. We show that this approach is insufficient to achieve satisfactory control performance. As a solution, we introduce generalized models pretrained on excitation-based operational source data - purposefully probed inputs that explore the building's state-action space. For evaluation, we apply the generalized models via zero-shot (i.e., without fine-tuning) to 32 simulated target buildings and assess MPC performance. Our results show that excitation-based generalized models achieve the strongest control performance among all
benchmarks, outperforming an online linear model-based MPC and a PI controller by 6.4% and 36.9%, respectively. By combining strong control performance with the ability to generalize across multiple buildings, without requiring any target-specific data, our approach reduces MPC setup cost and simplifies its widespread deployment in the building sector.
Read the full abstract ▾
All authors Fabian Raisch, Felix Koch, Zack Xuereb Conti, Christoph Goebel, Benjamin Tischler
Machine Learning · Cross-listed from cs.RO
VertexCBF: Improving Neural Control Barrier Functions via Vertex-Restricted Control Search
Bojan Derajić, Sebastian Bernhard and Wolfgang Hönig
As the number of autonomous
robots continues to grow, safety becomes increasingly important. Control barrier functions (CBFs) provide a theoretically grounded framework for ensuring safety, but existing design methods often face limitations in effectiveness, scalability, or interpretability, and may result in overly conservative safe sets. In this paper, we propose VertexCBF, a framework for learning neural CBFs in a scalable, systematic, and explainable way. We approximate the stationary Hamilton–Jacobi value function using a neural network trained via a combination of physics-informed and sparsely supervised learning. By exploiting control-affine dynamics and a convex polytope control set, under which the
Hamiltonian is maximized at the control vertices, we efficiently generate supervision points via GPU-parallel vertex-restricted tree search, while a residual architecture guarantees that the learned CBF is never larger than the specified constraint function. We evaluate the method on 15 systems and compare it against relevant baselines, showing that it reliably recovers large safe sets where the baselines are conservative or fail completely. In addition, we perform a hardware experiment in which a mobile
robot safely avoids pedestrians using a neural CBF trained with our method.
Read the full abstract ▾
Machine Learning · Cross-listed from physics.ao-ph
4D Parallelism Unlocks Exascale Bayesian Neural Networks for High-Fidelity Atmospheric Modeling
Deifilia Kieckhefen, Juan Pedro Gutiérrez Hermosillo Muriedas, Lars Helge Heyen et al. (+12 more)
We present BEAST, the first-ever
Bayesian Swin
Transformer for atmospheric
forecasting on 0.25$^\circ$ global resolution able to accurately quantify both aleatoric and epistemic uncertainty. To overcome the associated computational bottlenecks, we devise an orthogonal 4D-parallelization scheme that introduces a unique domain-tensor-parallelism strategy and a novel uncertainty parallel method, enabling us to fully leverage GPU capacity and efficiently scale model training. For a 2.4-billion-parameter model, we achieve a peak performance of 3.96 EFLOP/s on 20,480 NVIDIA GH200 GPUs on the JUPITER supercomputer. We train BEAST as a 700-million-parameter model with 96 random weight samples on 384 nodes on 40 years of data for nearly one million gradient updates. This model achieves predictive skill scores competitive with state-of-the-art probabilistic atmospheric AI models and numerical models, and can predict extreme events with exceptional skill, while generating large ensembles 3 to 4 times faster than the current-best AI model. Our contribution unlocks the potential of high-fidelity uncertainty quantification in atmospheric AI models, heralding a new era for AI-based models in
climate and Earth system sciences.
Read the full abstract ▾
All authors Deifilia Kieckhefen, Juan Pedro Gutiérrez Hermosillo Muriedas, Lars Helge Heyen, Mathis Bode, Iida Hakulinen, Andreas Herten, Chelsea Maria John, Thorsten Kurth, Anni Moisala, Asena Karolin Özdemir, Kaleb Phipps, Oskar Taubert, Arvid Weyrauch, Markus Götz, Charlotte Debus
Machine Learning
RunningTensor: Generalizing Linear Attention to Higher-Order Recurrent States
Luca Herranz-Celotti and Vincent Guigue
Linear
attention and state-space models provide linear-time sequence modeling, but their recurrent
memory remains a second-order tensor (a
matrix), limiting the order of interactions that can be represented in the state. We introduce the RunningTensor, which generalizes this
memory to an order-$o$ tensor, updated by a rank-1 outer product and read by contracting against $o-1$ vector queries. Order $2$ recovers linear
attention; we study order $3$ as a proof of concept, retaining both recurrent and parallel forms while remaining linear in sequence length $T$ and improving working
memory capacity from $\mathcal{O}(W^2)$ to $\mathcal{O}(W^o)$. On synthetic multi-query associative recall, RunningTensor outperforms linear-
attention and SSM baselines. After pretraining, it also improves performance on
language-understanding and non-synthetic
retrieval tasks, suggesting that higher-order recurrent state can provide useful additional
memory capacity beyond
matrix-valued state.
Read the full abstract ▾
Machine Learning
VertiFuseX: Generalizable Financial Forecasting via Multi-Stream Temporal Fusion
Aashish Bohra and Vivek Vijay
Stock price prediction remains challenging due to the non-stationary and noisy nature of financial time series. Existing deep learning models often rely on rigid decision-level fusion, ad hoc hyperparameter tuning, and compressed final-layer outputs, causing information loss, overfitting, and limited cross-
market generalization. We propose VertiFuseX, a hybrid LSTM architecture using penultimate-layer vertical fusion of multi-scale temporal representations. VertiFuseX stacks and reweights penultimate features from LSTM, Bi-LSTM, and St-LSTM branches, integrates a parallel DNN stream, and jointly optimizes all components via backpropagation under a fixed hyperparameter configuration. This preserves richer intermediate temporal information across scales. Evaluated on 15 years (2010-2024) of closing prices from 10 global equity indices using strict chronological out-of-sample testing with the final 365 trading days held out, VertiFuseX achieves 30-54% MAPE reductions and over 40% improvements in MAE and RMSE versus LSTM-based baselines, and outperforms seven state-of-the-art models across 33 metric-
dataset comparisons. Ablation studies confirm penultimate-layer fusion drives these gains over final-layer fusion and decision-level ensembling. Gradient-based saliency analysis shows consistent emphasis on mid-range dependencies at lags 9-15 days. Economic validation via algorithmic trading simulation under extreme
market regimes shows reduced maximum drawdowns and superior risk-adjusted returns. With 675k parameters, a 2.6 MB
memory footprint, and 1.5 ms/sample inference latency, VertiFuseX offers a lightweight, interpretable, deployment-ready framework for robust financial
forecasting.
Read the full abstract ▾
Machine Learning
Convergence of Stochastic Gradient Methods under Heavy-Tailed Noise and Hölder Smoothness
Misbah Uz Zaman and Anirbit Mukherjee
Classical convergence guarantees for
stochastic gradient methods typically assume Lipschitz-smooth objectives and finite-variance gradient noise, both frequently violated in practice. In contrast, we study nonconvex
stochastic optimization under the joint relaxation of these assumptions: objectives with $(L,s)$-Hölder continuous gradients, $s\in(0,1]$, and gradient noise satisfying only a bounded $\alpha$-th moment condition for $\alpha\in(1,2]$. We establish three convergence results. Firstly, that standard SGD converges at rate $O(T^{-s/(1+s)})$ whenever $\alpha\ge1+s$, extending the classical nonconvex SGD rate to heavy-tailed noise and Hölder smoothness simultaneously. Secondly, we analyze $\delta$-regularized gradient clipping ($\delta$-GClip), a provable trainer of wide and deep nets, and establish a stationarity rate of $O(T^{-2s(\alpha-1)/[(1+s)(2\alpha-1)]})$ under the same condition. Thirdly, we analyze standard gradient clipping (G-Clip) and show that it recovers the above rate for $\alpha\ge1+s$ while in the very heavy-tailed regime $\alpha<1+s$, it has a convergence rate $O(T^{-2s(\alpha-1)/[(\alpha-1)+s(2\alpha-1)]})$ — the first convergence guarantee in this regime for any
stochastic gradient based method.
Read the full abstract ▾
Machine Learning · Cross-listed from math.OC
High-Probability Convergence of SGD via Batched Updates
Feng Zhu, Robert W. Heath Jr. and Aritra Mitra
Stochastic gradient descent (SGD) is the primary workhorse for large-scale optimization. While the average behavior of its iterates, typically characterized by mean-squared error bounds, is well-understood, obtaining high-probability guarantees for the last iterate remains challenging. Prior approaches to this problem have either imposed restrictive assumptions (such as bounded domains or gradients) or relied on complex proofs involving auxiliary sequences. In this work, we propose Batched SGD, a simple variant that partitions online samples into epochs and performs a single update per epoch using a refined, low-variance gradient estimate. Our main contribution demonstrates that this batching mechanism enables a surprisingly simple high-probability analysis that avoids both restrictive assumptions and auxiliary sequences. Under standard smoothness and norm-sub-Gaussian noise assumptions, we establish near-optimal rates for both strongly convex and non-convex objectives. Furthermore, we show that our batching idea extends naturally to
federated learning (FL). We provide the first high-probability guarantees for FL, achieving logarithmic communication complexity, linear speedup in the number of
agents, and resilience to data heterogeneity.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Same Encoder, Different Winner: A Paired-View Framework for Cell Painting Encoder Evaluation
Tim Treis, Nikita Moshkov, Johan Fredin Haslum et al. (+2 more)
Vision encoders for
Cell Painting are typically ranked by a single evaluation, commonly replicate mean average precision (mAP). We introduce CP-BG-Bench, a paired-view evaluation framework that holds the central
cell fixed across four matched views (raw crop C, segmented S, and density-augmented variants CD and SD), ablating or augmenting surrounding pixels as a controlled intervention. Instantiating the framework on three
datasets (JUMP-CP, RxRx1, RxRx3-core) and three encoders (DINOv3 ViT-B/16, OpenPhenom, SubCell) under four community-standard protocols (replicate mAP, scIB batch integration, CellProfiler feature prediction, cross-batch perturbation recall), we find that the four protocols rank the same encoders systematically differently, with disagreements decomposing along three axes:
cell versus background, morphology versus context, and within-study versus across-batch. The largest effect: on RxRx3-core, SubCell with segmented inputs retains 94% of crop replicate mAP but only 32% of crop R@10, so the within-study signal preserved under
segmentation is largely non-transferable; density augmentation recovers 84% of the within-study C-to-S gap but only 8% of the cross-batch gap. Segmented views predict CellProfiler features as well as or better than crops on two of three
datasets, inverting the replicate-mAP ranking, and the C-to-S gap varies by an order of magnitude across
datasets while remaining similar across encoders, indicating that background-driven gain is set by experimental design rather than by the encoder. Single-metric ranking of
Cell Painting encoders is therefore sensitive to the protocol used, and protocol disagreements are interpretable as projections onto the three axes the paired-view design exposes. We will release the paired-view
datasets, reconstruction pipelines, 36 trained checkpoints, aggregated
embeddings, and the full evaluation suite.
Read the full abstract ▾
All authors Tim Treis, Nikita Moshkov, Johan Fredin Haslum, Shantanu Singh, Fabian J. Theis
Machine Learning
Curriculum-Based Adversarial Heterogeneous Agent Reinforcement Learning for Autonomous Quad-Copter Landing in Maritime Settings
Allan Minh-Tam Nguyen, Sree Showrya Kotala, Stefan Banioi-Crijman et al. (+2 more)
Recovering unmanned aerial vehicles (
UAVs) in maritime environments is challenging due to wind
turbulence and ship-deck motion, making it a valuable test case for alternative control and learning approaches as conventional landing approaches often become unreliable. We study simulated mid-air capture of quadrotor
UAVs by a ship-mounted robotic arm, learning robust cooperative control policies with Heterogeneous-
Agent Proximal Policy Optimization (HAPPO) Reinforcement Learning. We train with HAPPO using a curriculum and an
adversarial wind
agent (HARL-AC) in NVIDIA Isaac Lab, and compare the obtained control policies against those generated through curriculum-based domain randomization and a
benchmark trained on a single sea state. In-distribution evaluation on sea states $0/4/5$ shows comparable success for HARL-AC and domain randomization of up to $97.5\%$. On out-of-distribution sea states $7/8/10$, HARL-AC generalizes better, achieving up to $16\%$ higher median success rate at sea state 10, and substantially lower crash rates of up to $14\%$ compared to the domain randomization policy. Furthermore, we show that the adversarially trained policy shows more cautious behavior, slightly increasing timeouts by $<3\%$, but yields safer recovery behavior in severe, unseen conditions.
Read the full abstract ▾
All authors Allan Minh-Tam Nguyen, Sree Showrya Kotala, Stefan Banioi-Crijman, Kurt Driessens, Rico Möckel
Machine Learning
Optimizing for the decision not the prediction: an exploration of Smooth Net Benefit as a training objective
Koen M.F. Gorgels, Lasai Barreñada, Maarten van Smeden et al. (+3 more)
Objective Prediction models are commonly trained using objectives such as Bernoulli negative log-likelihood (NLL), although downstream clinical decisions may depend on specific risk thresholds. We introduce Smooth Net Benefit ($\sigma$NB), a differentiable approximation of Net Benefit designed to align model training with threshold-specific clinical utility. Materials and Methods We evaluated $\sigma$NB as a training objective for logistic regression, generalized additive models (GAMs), and XGBoost with three Hessian implementations. Experiments used the Framingham cardiovascular risk
dataset and 44 TabZilla
datasets comprising 72
dataset-threshold combinations. Results $\sigma$NB training did not consistently improve Net Benefit in Framingham. Across the TabZilla
benchmark, mean standardized Net Benefit for logistic regression increased from 0.5669 with NLL to 0.5765 with $\sigma$NB (mean difference 0.0096, 95% CI -0.0001 to 0.0193). For GAMs, mean standardized Net Benefit decreased from 0.5921 to 0.5625 (mean difference -0.0296, 95% CI -0.0721 to 0.0129). For XGBoost, NLL achieved 0.6745 compared with 0.6723–0.6735 across $\sigma$NB implementations. In logistic regression, $\sigma$NB gains were positively associated with the performance advantage of XGBoost over NLL-trained logistic regression. Discussion The effect of $\sigma$NB was context dependent, with modest gains concentrated in logistic regression and little benefit for more flexible model classes. This suggests that decision-focused optimization may be most useful when limited model flexibility leaves greater scope for improvement. Conclusion Our results do not support $\sigma$NB as a general replacement for NLL training, but support further investigation of decision-focused objectives in settings where conventional likelihood-based training may not adequately capture decision-relevant structure.
Read the full abstract ▾
All authors Koen M.F. Gorgels, Lasai Barreñada, Maarten van Smeden, Ben Van Calster, Ewout W. Steyerberg, Wouter A.C. van Amsterdam
Machine Learning · Cross-listed from cs.AI
What Drives Recovery in Agentic Text-to-Cypher? LAST-CQ: An LLM Agent Self-Refinement Framework
Ioannis Prokopiou, Athanasios Aidinis, Panagiotis-Christos Kyrmpatsos et al. (+1 more)
Agentic pipelines for structured-query generation are rapidly expanding, but it is unclear which part of the loop produces the gain. We use LAST-CQ – a five-
agent, training-free, execution-grounded Text-to-Cypher framework – as an instrumented testbed, running three counterfactuals over 2,471 live-database queries and six backbones spanning three vendor scale tiers. Removing correction is worth between 3.1% aggregate execution-BLEU against the single-pass system and 12.3% against a no-refinement counterfactual (up to 80.7% for the weakest backbone). Replacing schema-grounded, LLM-synthesised feedback with raw database error strings costs almost nothing (20.9% vs. 19.9% naive exact match; <0.2% end-to-end; equivalent within $\pm 0.075$ set-F1 by two one-sided tests). Spending the same call budget on parallel sampling degrades quality by 10-11%. What works is detecting failure and routing it to a retry, not the feedback sophistication or number of samples. LAST-CQ itself recovers 91.7% of queries that fail under single-pass generation, while a query that succeeds first time still costs exactly one LLM call. We also show that n-gram overlap on serialised results is not a bound in either direction: it over-scores against set equivalence on 65.9% of results while under-scoring against judged semantics. Finally, we calibrate our LLM judge against blind human labels and find it optimistic by 9 points.
Read the full abstract ▾
All authors Ioannis Prokopiou, Athanasios Aidinis, Panagiotis-Christos Kyrmpatsos, Pantelis Vikatos
Machine Learning
Physics-Guided Synthetic High-Frequency Ultrasound Generation for Skin Layer Segmentation
Junkyung ju, Kyungho Yoon and Minwoo Shin
High-frequency ultrasound (HFUS) enables noninvasive visualization of superficial skin structures, but automated skin-layer analysis is limited by the scarcity of densely annotated data. Existing real HFUS
datasets commonly provide annotations for superficial targets such as the epidermis and subepidermal low-echogenic band (SLEB), while dense labels for deeper structures such as dermis, subcutaneous tissue, fascia, and muscle are rarely available. We propose a physics-guided synthetic HFUS generation framework for skin layer
segmentation. The framework constructs multilayer acoustic skin phantoms, assigns layer dependent acoustic properties, and uses k-Wave simulation to generate paired synthetic HFUS images, dense layer masks, and simulation metadata. To evaluate whether the generated data provide transferable supervision, we use it for downstream
segmentation pretraining and fine-tune the models on real Mendeley HFUS data. Synthetic pretraining followed by real fine-tuning achieved real-domain performance comparable to real-only training and improved mean Dice/IoU in three of four evaluated trainable architectures. These results suggest that physics-guided synthetic HFUS images contain transferable anatomical and textural cues for real-domain skin layer
segmentation, although further reduction of the synthetic-real appearance gap is needed to enable greater gains. The code and data are available at: https://github.com/Finn-02/synthetic-hfus-skin-layer-
segmentation.
Read the full abstract ▾
Machine Learning · Cross-listed from eess.SP
Prism-SQA: An Interpretable and Adaptable Neural Framework for Surface Electromyography Quality Assessment
Kuan-Chen Wang, Kai-Chun Liu, Ping-Cheng Yeh et al. (+2 more)
sEMG is vulnerable to various contaminants that distort signal morphology and spectral content. Accurate signal quality assessment (SQA) is essential for identifying such degradation and ensuring reliable clinical analyses and decisions. Recent neural network-based SQA methods achieve accurate quality estimation by learning complex contamination patterns, yet their black-box nature prevents clinicians from understanding or validating the reported scores and limits adaptability to application-specific quality definitions without retraining. To address these limitations, we propose Prism-SQA, an interpretable and adaptable neural framework that reformulates SQA as a physiology-aware source-separation and verification process. Prism-SQA decomposes each input signal into a clean sEMG component and five contaminant-specific components using a U-Net with bidirectional long short-term
memory. Each separated contaminant component is examined by a Contaminant Fingerprint Verifier, which enforces physiological plausibility by comparing its temporal and spectral structure with canonical contaminant signatures. This design allows clinicians to inspect how each contaminant affects signal quality, grounding the assessment in transparent, signal-level evidence rather than opaque latent representations. Quality indices computed from the verified components further enable customization of quality criteria across clinical contexts without retraining. We evaluate Prism-SQA on continuous quality-score estimation using synthesized noisy sEMG from public Ninapro
datasets and on binary quality classification using a clinical dysphagia
dataset. Results show that Prism-SQA achieves competitive or better performance than contemporary black-box neural methods while providing explicit interpretability and adaptability, advancing toward practical and clinically aligned sEMG SQA.
Read the full abstract ▾
All authors Kuan-Chen Wang, Kai-Chun Liu, Ping-Cheng Yeh, Sheng-Yu Peng, Yu Tsao
Machine Learning · Cross-listed from cond-mat.mtrl-sci
Inferring Dislocation Microstructures from X-ray Diffraction via Cross-Modal Contrastive Learning
Benjamin Udofia, Nicolas Bertin and Markus Stricker
Understanding and inferring dislocation microstructures from diffraction patterns remains an open challenge in materials characterization, as diffraction measurements provide only indirect information about the underlying dislocation structure. In this work, a cross-modal learning framework is developed to enable the prediction of 3D dislocation structures directly from diffraction data. Dislocation density fields generated from discrete dislocation dynamics simulations are paired with corresponding virtual X-ray diffraction patterns and embedded into a shared 2D latent space using contrastive learning. The
alignment between structural and diffraction representations of dislocation structures is evaluated directly in the learned latent space using correlations between corresponding latent features. To estimate the role of
dataset size for this approach, farthest point sampling is employed to construct representative and diverse training subsets of varying sizes. The results show strong cross-modal
alignment and that model performance improves rapidly with increasing
dataset size. Near-saturation is achieved with approximately 500 representative observations from a
dataset of 10,000 observations, enabling accurate prediction of dislocation density fields from previously unseen diffraction data of the same distribution. Qualitative comparisons confirm that the predicted structures capture the dominant spatial features of the underlying dislocation microstructures. These findings demonstrate an efficient approach for learning structure-diffraction relationships and highlight the potential for inferring structural characteristics of dislocation networks directly from diffraction patterns, providing a pathway toward diffraction-based structural analysis and future extension to experimental data.
Read the full abstract ▾
Machine Learning
InRTL: Effective Intra-Inter Interaction Learning for Relational Tables
Weichen Li, Ken Zhong, Zheng Wang et al. (+2 more)
Relational table learning has recently emerged as an important research direction for modeling multiple tables connected through primary key-foreign key (PK-FK) relationships. Despite recent advances, a principled modeling framework tailored to this task remains underexplored. In this paper, we propose Intra-Inter Relational Table Learning (InRTL), a unified framework that explicitly models dependencies both within and across relational tables. Specifically, InRTL formalizes two complementary interaction patterns: intra-table interactions, describing associations among rows within the same table, and inter-table interactions, describing dependencies between rows across PK-FK-linked tables. To model these dependencies, we develop a column-aware table encoder to generate initial row representations, followed by
Transformer-based self-
attention and cross-
attention modules for intra-table and inter-table learning, respectively. To further improve scalability, InRTL incorporates linearized
attention and heterogeneous graph neural networks to simplify the self-
attention and cross-
attention operations. Extensive experiments on ten
datasets covering 24 real-world tasks demonstrate the effectiveness of our approach. Code is available at https://github.com/W1nterFloW/InRTL.
Read the full abstract ▾
All authors Weichen Li, Ken Zhong, Zheng Wang, Li Pan, Jianhua Li
Machine Learning · Cross-listed from cs.CV
ExpertHTR: Unified Handwritten Text Recognition with Multi-Task Learning and Sparse Mixture-of-Experts
Dang Hoai Nam, Nguyen Duy Hieu, Quang Huu Hieu et al. (+1 more)
Handwritten text recognition resources are often small and distributed across collections that differ in
language, script, document structure, and annotation format, making joint page-level training difficult. We propose ExpertHTR, a unified
vision-
language framework that addresses this problem through complementary supervision and conditional model capacity. Structural annotations from heterogeneous
datasets are first organized through a common Page-Region-Line representation and used to construct four related training tasks for complete transcription, physical-line coverage, text localization, and localized recognition, without requiring additional manual labels. Building on a jointly trained dense model, ExpertHTR introduces a sparse Mixture-of-Experts architecture with an always-active shared branch and conditionally routed full-MLP experts. Sparsegen allows the number of active routed experts to vary with the hidden representation, while routing regularization reduces persistent concentration on a small subset of experts. Experiments on seven heterogeneous handwriting
benchmarks show that complementary supervision consistently improves training with page transcription alone, while joint multi-source training provides further gains on most
datasets. The proposed sparse expert model further improves the dense baseline on six of the seven sources. The final unified model also substantially outperforms the evaluated general-purpose OCR and
vision-
language systems on most
benchmarks and achieves state-of-the-art performance on the IAM paragraph-level
benchmark, while specialized HTR systems remain stronger on several challenging collections.
Read the full abstract ▾
All authors Dang Hoai Nam, Nguyen Duy Hieu, Quang Huu Hieu, Vo Nguyen Le Duy
Machine Learning
Write on Paper and Get the Online Digital Trace: A New Era for Handwriting
Florent Imbert, Yann Soullard, Eric Anquetil et al. (+5 more)
Capturing the digital trace of handwriting usually requires a specific stylus and a compatible substrate, be it a capacitive touchscreen, an ElectroMagnetic Resonance (EMR) tablet as used in Wacom systems or special paper. While writing on regular paper offers rich haptics, no latency and is well known for improving information retention, no low-cost and widely accepted, effective solution exists to digitize such a pen trace. The challenge is to accurately track the pen's trajectory without an external reference system while allowing unrestricted freedom of pen movement across a surface. We propose an innovative solution that combines a digital pen, advanced artificial intelligence
algorithms, and adaptive AI techniques to reconstruct the digital trace of handwriting. Our approach integrates hardware development, focusing on a sensor-equipped pen, with software innovations to optimize trajectory reconstruction and processing in real time using an embedded AI. This work aims to advance the state-of-the-art in automated trace reconstruction of handwriting, enabling a seamless connection between traditional handwriting on paper and capturing the trace digitally.
Read the full abstract ▾
All authors Florent Imbert, Yann Soullard, Eric Anquetil, Tanja Harbaum, Alexey Serdyuk, Fabian Kress, Tim Hamann, Peter Kampf
Machine Learning
SIFPBPNet: A Dual-Path Network for Wearable and Cuffless Blood Pressure Estimation via Individualized Steady-state Representation
Shuailong Tang, Xiaoyu Li, Donglin Xie et al. (+4 more)
Continuous and cuffless blood pressure (BP) monitoring using photoplethysmography (PPG) is of great interest for low-cost and personalized cardiovascular health management. However, significant population heterogeneity and the "one-to-many mapping" problem, where similar waveforms across individuals correspond to different BP levels, limit the accuracy of conventional population-based models. To address this challenge, we propose a dual-path architecture termed SIFPBPNet, which separately represents steady-state and instantaneous features, through a Steady-state Feature Path (SFP) and an Instantaneous Feature Path (IFP). The SFP employs a Graph
Attention Network (GAT) to extract individual-specific and long-term characteristics from multi-day historical PPG trajectories. In parallel, the IFP captures short-term dynamics from current PPG segments and incorporates the steady-state prior via a cross-
attention mechanism. Experiments on a large-scale wearable
dataset demonstrate that SIFPBPNet achieves a Mean Absolute Error (MAE) of 8.57 and 5.97 mmHg for systolic and diastolic BP, respectively, outperforming state-of-the-art models. Furthermore, the SFP module consistently improves performance when integrated into various backbone architectures, yielding 2.8-13.1% relative MAE reductions for systolic BP. These results highlight the strong generalizability and plug-and-play transferability of the SFP module, underscoring its great potential for accurate cuffless BP monitoring.
Read the full abstract ▾
All authors Shuailong Tang, Xiaoyu Li, Donglin Xie, Wei Chen, Guangpu Zhu, Yelei Li, Yali Zheng
Machine Learning · Cross-listed from cs.SD
What Did the MLLM Hear? Token-Level Spectro-Temporal Grounding for Audio MLLM Explainability
Lucia Cascone, Valeria Fraenza, Michele Nappi et al. (+2 more)
Audio-based
Multimodal Large
Language Models (MLLMs) can generate detailed natural-
language descriptions of complex acoustic scenes, yet it remains unclear which parts of the input audio support each generated
token. This is particularly challenging because acoustic evidence is distributed across time and frequency, and concurrent sound events may overlap temporally while occupying different spectral regions. We introduce STAG, to our knowledge the first post-hoc framework for
token-level spectro-temporal grounding of captions generated by audio-based MLLMs. STAG estimates the temporal support for each generated
token using target-
token-specific vocabulary projections of the encoded audio representations, measures frequency-band relevance through controlled spectral occlusion, and combines the two signals into a spectro-temporal relevance map. We evaluate STAG against ten post-hoc explanation methods across four grounding
benchmarks, where it achieves the best event-localization performance on every
dataset, and apply it to eight audio-
language backbones without parameter updates. Counterfactual deletion further shows that removing the identified evidence selectively reduces confidence in the corresponding event and frequently removes it from the regenerated caption. These results provide behavioral support for the faithfulness and selectivity of the explanations.
Read the full abstract ▾
All authors Lucia Cascone, Valeria Fraenza, Michele Nappi, Fabio Narducci, Benedetto Simone
Machine Learning
LiveProBench: Can Streaming Video Models Really Interact Like Humans?
Kaixuan Du, Xin Wan, Hang Zhang et al. (+4 more)
Streaming
video understanding requires models to process continuous
multimodal input while maintaining temporal context. Existing evaluations are predominantly reactive: they query a model at a selected timestamp and therefore do not assess when it should respond. Proactive interaction instead requires monitoring a standing request, responding within an appropriate interval after the target event, and otherwise remaining silent. We introduce LiveProBench, which evaluates models at one-second stream intervals without an explicit response cue. Its six subtasks vary trigger ambiguity and timing tolerance. Event Sensitivity geometrically combines response and silence rates on the same recording; four window-based subtasks distinguish early, in-window, and missed responses; and Duplicate Counting penalizes omissions and repetitions. Premature responses outnumber missed responses for half of the evaluated models, revealing a substantial gap in the temporal decision-making required for human-like interaction.
Read the full abstract ▾
All authors Kaixuan Du, Xin Wan, Hang Zhang, Meng Cao, Dai Guan, Ming Chen, YuKun Wang
Machine Learning
Distortion of AI Alignment Revisited: RLHF is a Decent Utilitarian Aligner
Kazusato Oko, Annie Ulichney, Nika Haghtalab et al. (+1 more)
While Reinforcement Learning from Human Feedback (RLHF) is the standard paradigm for aligning large
language models with human preferences, its effectiveness in pluralistic settings has been called into question. Notably, recent work by Gölz et al. (2025) demonstrated that the distortion – defined as the multiplicative gap between the average user utility of the RLHF policy and the optimal average utility – can scale exponentially with the Bradley-Terry temperature parameter $\beta$ when users have heterogeneous preferences. In this work, we present a fine-grained analysis of the distortion of RLHF with reward clipping and demonstrate that such exponential degradation is not a fundamental property of the
algorithm but rather a consequence of distribution mismatch between the distribution generating preference data ($\mu$) and the KL reference policy ($\pi_{\mathrm{ref}}$). To this end, we establish tight upper and lower bounds on the distortion of RLHF across multiple regimes of the KL regularization strength. We show that in a representative regime, under the Bradley-Terry model, the distortion is $\tilde{\Theta}(\beta B + \beta)$, where $B$ is an upper bound on the log density ratio between $\mu$ and $\pi_{\mathrm{ref}}$. In particular, when there is no distribution mismatch (i.e., $\mu = \pi_{\mathrm{ref}}$), RLHF achieves the optimal distortion of $O(\beta)$ up to a constant. Our results suggest that, to reasonably maximize average utility with RLHF, it is preferable to use on-policy sampled preference data or to fine-tune before RLHF on data from a source close to $\mu$.
Read the full abstract ▾
All authors Kazusato Oko, Annie Ulichney, Nika Haghtalab, Han Bao
Machine Learning
Explaining Time Series Forecasting with Horizon-Resolved Attribution
Seunghan Lee, Jun Seo, Jaehoon Lee et al. (+9 more)
Recent advances in explaining time series (TS) models have produced methods that identify which past values a prediction depends on. However, most existing methods return a single importance vector, assuming that every predicted step depends on the same past values. In this paper, we show that this assumption does not hold, as different forecast steps depend on different past values. Motivated by this observation, we propose Horizon-Resolved eXplanation (HRX), which adds a horizon axis to the explanation, so that every forecast step receives its own importance map. HRX is a simple yet effective plug-in framework with three components: 1) an
estimator that reads these maps out of any differentiable forecaster without modifying the TS backbone, 2) an evaluation protocol that validates the horizon axis by measuring how much a single forecast step changes when the inputs an importance map ranks highest are removed, and 3) a rank criterion that predicts in advance whether the axis is worth resolving on a given TS. We further show that this step-wise dependence is low-dimensional, as the explanations of all steps are built from a few shared maps whose number does not grow with the forecast length. Extensive experiments across various backbones and
datasets show that the improvement comes from the horizon axis and holds for
estimators of previous explanation methods. Code is available at https://github.com/seunghan96/HRX.
Read the full abstract ▾
All authors Seunghan Lee, Jun Seo, Jaehoon Lee, Junhyeok Kang, Sangjun Han, Sungdong Yoo, Minjae Kim, Tae Yoon Lim, Dongwan Kang, Hwanil Choi, Soonyoung Lee, Wonbin Ahn
Machine Learning
Geometric-to-Semantic Spherical Transfer Learning for Cortical Sulci Labeling
Saeb Tounsi, Joël Chavas, Pietro Gori et al. (+3 more)
Deep learning on cortical surfaces faces a dilemma: capturing the complex
topology of over 60 nomenclature-dependent sulci per hemisphere requires high-capacity models, yet the extreme scarcity of expert annotations ($N=62$ subjects) inevitably causes overfitting. Standard supervised approaches fail to generalize in this data-scarce regime, particularly for variable and small sulci where topological ambiguity is high. To overcome this limitation, we introduce a
Geometric-to-Semantic Spherical Transfer Learning framework. First, we leverage massive unlabeled data (UK Biobank, $\approx$30,000 subjects) to pre-train a spherical encoder using a locally-optimized strategy. By relying solely on continuous surface features (
curvature and depth), the relevance of this pre-training is confirmed by the model's ability to detect localized and rare topological traits, such as sulcal interruptions. The downstream labeling task, however, introduces extracted sulcal fundi (lines) as an explicit semantic input. To bridge this dimensional domain gap (from purely
geometric to semantic) without causing catastrophic forgetting, these anatomical lines are integrated into the pre-trained backbone via a soft-initialized Topological Prior Injector. Our experiments demonstrate that this approach outperforms fully supervised baselines trained from scratch, achieving a mean Dice of 0.77. Crucially, a local analysis reveals that the self-supervised
geometric priors yield the largest performance gains on variable and tertiary sulci (up to 14.8%), confirming that learning the cortex shape is highly beneficial for identifying its rarest parts.
Read the full abstract ▾
All authors Saeb Tounsi, Joël Chavas, Pietro Gori, Vincent Frouin, Denis Rivière, Jean-François Mangin
Machine Learning
Correlation-Guided Fast Machine Unlearning via Hessian Analysis
Ayushi Thakur, Ruchir Gupta, Amit Kumar Jaiswal et al. (+1 more)
The increasing adoption of machine learning in network and distributed security systems has created an urgent need for mechanisms that can selectively and efficiently remove the influence of specific training data to eliminate compromised or
adversarial data points from production models.
Privacy regulations such as GDPR's right to be forgotten also pose similar requirements. However, existing approximate unlearning techniques remain computationally prohibitive for deployment in real-world security systems, as they require repeated expensive Hessian-inverse-vector computations for each data point removal, creating a bottleneck when processing multiple related requests in scenarios such as intrusion detection systems, spam filters, and threat intelligence platforms. Thus, we introduce a computationally efficient unlearning framework that identifies correlated data points in the training set and applies a theoretically derived closed-form parameter update rule, achieving an $82\times$ wall-clock speedup over standard influence function unlearning while preserving model utility with a $10^{-2}$ improvement in accuracy over state-of-the-art baselines. Our method establishes theoretical guarantees and ensures numerical stability through Hessian damping. Our evaluation across seven diverse
dataset architecture combinations, including large-scale CIFAR-100 with ResNet-50, demonstrates superior forgetting effectiveness, with membership inference attack success rates of 0.660 and tug-of-war scores of 0.950.
Read the full abstract ▾
All authors Ayushi Thakur, Ruchir Gupta, Amit Kumar Jaiswal, Prayag Tiwari
Machine Learning
SIMS: Scale-Invariant Merit-Function-Based Scalarization for Multi-Task Learning
Zebin Chen, Fei Xing, Yang Chen et al. (+4 more)
Multi-task learning (MTL) requires navigating unavoidable trade-offs among competing objectives. This paradigm is frequently formulated as multi-objective optimization (MOO), where the scalarization is favored to reduce an MOO problem to a single objective. We empirically find that existing merit-function-based scalarization approaches are sensitive to the relative scales of different objectives in practical MTL, where task losses commonly differ by orders of magnitude. The optimization process often favors objectives with larger scales even though the underlying Pareto optimal solutions remains invariant to rescaling (i.e., multiplying an objective by a positive constant). To address this issue, we propose Scale-Invariant Merit-function-based Scalarization (SIMS) for MTL. Specifically, SIMS adopts a transformation-induced merit function to convert the MOO problem of MTL to a single objective that renders optimization invariant to the magnitudes of losses. Theoretically, we prove that the requirement for scale invariance uniquely determines this transformation to be logarithmic. We further show that this general transformation-induced merit function preserves weak Pareto optimality and admits a smooth surrogate with controllable approximation error. Extensive experiments on representative multi-task
benchmarks demonstrate that SIMS consistently outperforms existing scalarization methods and achieves state-of-the-art performance.
Read the full abstract ▾
All authors Zebin Chen, Fei Xing, Yang Chen, Hua Liu, Andy HF Chow, Yuhua Qian, Yu Zhang
Machine Learning
Poisson-Corrector Complexity Bounds for Moreau–Yosida Unadjusted Langevin Sampling
Yuchen Xin and Zhihua Zhang
We study the classical Moreau–Yosida unadjusted Langevin
algorithm (MYULA) for $\pi(\,\mathrm{d} x)\propto e^{-f(x)-g(x)}\,\mathrm{d} x$, where $f\in C^2(\mathbb{R}^d)$ is $m$-strongly convex with $L_f$-Lipschitz gradient and $g:\mathbb{R}^d\to\mathbb{R}$ is convex and globally $G$-Lipschitz. For the Moreau-smoothed target $\pi_\lambda$ and the MYULA invariant law $\widehat\pi_{\lambda,h}$, we prove \[ \sqrt m\,W_2(\pi_\lambda,\widehat\pi_{\lambda,h}) =O(h)+\widetilde O(h^{3/4}) \] under $0<h(L_f+\lambda^{-1})\le c$, with only logarithmic dependence on $\lambda^{-1}$ in the error coefficients. Combining this estimate with the Moreau approximation bias yields $\widetilde O(\varepsilon^{-4/3})$ iterations to achieve $\sqrt m\,W_2(\mu_N,\pi)\le\varepsilon$, for fixed model parameters and initialization. The proof combines a discrete Poisson corrector with active-trace estimates and a shared-noise bound for the exact–Euler two-point
curvature.
Read the full abstract ▾
Machine Learning
Where Decoder Cosine Similarity Fails for SAE Feature Flow Discovery
Hendrik Droste, Christian Medeiros Adriano, Kathrin Korte et al. (+1 more)
Foundation models are increasingly adapted through fine-tuning, model editing, and
alignment procedures while retaining previously acquired capabilities. Understanding the internal computations that support these adaptations is therefore becoming increasingly important for continual model
evolution. Sparse autoencoders (SAEs) provide interpretable feature dictionaries for residual-stream activations and sublayer outputs, but it remains unclear how state features and update features interact to produce downstream residual features. In this work, we focus on MLP updates as a first test case. We construct a transition atlas of triples $s_k + u_j \rightarrow t_\ell$, where a residual-state feature and an MLP-update feature jointly predict a target residual feature, and validate candidate triples by ablating the decoded update feature. In a 20M-
token Pythia-160M $L_7 \rightarrow L_8$ run, we find 38,125 strong ablation-effect transitions, but 88.0% have both state-target and update-target decoder cosine similarity below 0.7. As a preliminary cross-model check, a run of 20M-
token Gemma-3-4B $L_{21} \rightarrow L_{22}$ causally validates only the top 30,000 ranked candidate triples by ablating the decoded update feature, and 53.6% of strong-effect triples have both state-target and update-target decoder cosine similarity below 0.7. The Gemma result is directionally consistent with Pythia, but weaker, since update-target cosine recovers many of the strongest Gemma effects and the run is not a full-atlas causal validation. Ultimately, our results suggest that feature flow atlases can serve as diagnostics of representation-update mechanisms and thereby inform tools for steering model updates. Future work will validate more complex patterns across layers, models, and SAE families.
Read the full abstract ▾
All authors Hendrik Droste, Christian Medeiros Adriano, Kathrin Korte, Holger Giese
Machine Learning · Cross-listed from math.ST
Tight Sampling Complexity with Stochastic Gradient Oracles in Fixed Dimensions
Weiming Ou and Xiao Wang
We establish the
stochastic-gradient query complexity of sampling smooth strongly log-concave distributions in every fixed dimension $d\geq1$. For $\mu$-strongly convex, $L$-smooth potentials with unknown minimizers $x_f^\star$ in the ball $\mathbb{B}(0,\mu^{-1/2})$, and unbiased gradient oracles with variance at most $\sigma^2$, the minimax worst-case expected query complexity is $\Theta\left(\log(1+\kappa)+\frac{\sigma^2}{\mu\varepsilon}\right)$, jointly optimal for the condition number $\kappa:=L/\mu$, variance $\sigma^2\ge 0$, and TV accuracy $0<\varepsilon\leq 1/10$. In the noiseless setting $\sigma=0$, this tight complexity $\Theta\left(\log(1+\kappa)\right)$ is independent of $\varepsilon$. Moreover, a gradient-only sampler generates an exact sample with ${O}(\log(1+\kappa))$ worst-case expected queries. Without previous ball $\mathbb{B}(0,\mu^{-1/2})$, exact sampling from any initial point $x_0$ can be implemented with expected cost ${O}\left(\log(1+\kappa)+\log(1+\sqrt{\mu}\|x_0-x_f^\star\|)\right)$, without knowing the initial distance. We also show that dependence on initial distance is generally unavoidable.
Read the full abstract ▾
Machine Learning
Clustering-Based Balanced Sampling and Allocation with Data Parallelism for High-Performance Fine-Tuning
Hyunjin Kim, Youngeun Nam, Jaemin Han et al. (+2 more)
Instruction-tuning
datasets for large
language models (LLMs) are often large, redundant, and imbalanced, limiting efficient adaptation. Naive large-batch fine-tuning repeatedly includes overrepresented sample groups while weakly covering underrepresented but informative ones, especially under data parallelism (DP) across multiple GPUs. We propose CluSTER, a Cluster-aware balanced Sampling framework for Training Efficient data Reduction in DP instruction tuning. CluSTER curates a representative reduced
dataset through gradient-space clustering and DP-aware balanced allocation, ensuring dual-level coverage across clusters and workers, while preserving the original data distribution by weighted update. As a result, CluSTER reduces redundant computation and improves training stability without compromising model quality. Across multiple instruction-tuning
datasets, CluSTER reduces training time by up to 69.6% with almost no accuracy loss compared to prior sampling and data reduction methods. Code is available at https://github.com/kaist-dmlab/CluSTER.
Read the full abstract ▾
All authors Hyunjin Kim, Youngeun Nam, Jaemin Han, Wonhyeok Choi, Jae-Gil Lee