Machine Learning · Cross-listed from cs.CL
The Price of Reasoning: Cost-Quality Tradeoffs in Reinforcement Learning for Neural Machine Translation
Michael Jungo and Aixiu An
Reinforcement learning with verifiable rewards (RLVR) has been established as a viable paradigm for the post-training of Large
Language Models (LLMs), including downstream tasks, such as Neural Machine Translation (NMT). With the latest research indicating that RLVR could be the preferred training method for translating legal documents due to the induced
reasoning capabilities, it raises the question whether it is really attributed to the
reasoning or more generally to the training paradigm. We investigate the importance of including the model's
reasoning trace in the generated responses during both training and inference by systematically omitting it from one of the phases. Our experiments show that including the
reasoning, specifically during inference, has a positive effect on the overall translation quality. Furthermore, we recognise that the
reasoning leads to an increase in output
tokens, hence we study the cost-quality tradeoff between the increased computational demands and the improved translation quality.
Read the full abstract ▾
Machine Learning
AdaFlash: Adaptive Speculative Decoding via On-Policy Distilled Diffusion Drafters
Yu-Yang Qian, Hao-Cong Wu, Chen Chen et al. (+4 more)
Speculative decoding, in which a lightweight draft model first generates a draft sequence that is then verified by the target model, has become a prevalent paradigm for accelerating large
language model inference. Recent work such as DFlash further boosts drafting efficiency by leveraging
diffusion drafters, whose parallel denoising mechanism enables draft generation in a single forward pass. In this work, we uncover a central pitfall of
diffusion drafters: the global dependency, arising from bidirectional
attention and KV injection in
diffusion drafters, is a double-edged sword. On one hand, it endows the model with parallel generation and global contextual modeling capabilities; on the other hand, this global dependency introduces high variance at both the domain-level and the
token-level: acceptance rates fluctuate substantially across different domains, and the acceptance probability varies across draft positions. To tackle this issue, we propose the AdaFlash framework, comprising two components: (i) an on-policy
distillation (OPD)
algorithm with reverse-KL divergence tailored for
diffusion drafters, which delivers stable convergence and continuously adapts the drafter to the deployment distribution; and (ii) an adaptive length head that dynamically adjusts the candidate length on the fly, substantially lowering the verification cost of the target model. Experiments demonstrate that AdaFlash consistently improves the speedup during deployment, with especially significant gains under high-concurrency, achieving up to 66% higher average throughput than previous state-of-the-art methods. Our code is available at https://github.com/ZinYY/AdaFlash.
Read the full abstract ▾
All authors Yu-Yang Qian, Hao-Cong Wu, Chen Chen, Jiacheng Sun, Zhenhua Dong, Peng Zhao, Zhi-Hua Zhou
Machine Learning · Cross-listed from cs.CY
Assessment in Team Problem-Solving Exercises in Computing Education
Valdemar Švábenský, Jan Vykopal, Sukrit Leelaluk et al. (+3 more)
This full paper in the research-to-practice track presents methods for assessing student teams in tabletop exercises (TTXs). TTXs enable learner teams to prepare for workplace tasks and practice crisis responses, such as resolving cybersecurity incidents. While assessment is essential for determining how well teams achieve learning objectives, the complex, open-ended nature of TTXs often leads to delayed or incomplete feedback. TTX learning platforms can record teams' actions and communication; yet, leveraging these data to assess performance is underexplored. To address this gap, we compared two post-TTX team assessment methods – clustering and large
language models (LLMs) – using an original
dataset from 81 participants across two countries. We evaluated these methods against instructor-assigned scores based on standardized rubrics. Clustering grouped teams that approached TTX tasks similarly, enabling instructors to deliver faster, targeted feedback to teams within a cluster. This method was valid and reliable, with low computational requirements. LLMs used the standardized rubrics to assess teams' communication. While GPT-4o frequently disagreed with instructor scores, GPT-5.2 demonstrated considerably lower error. The researched methods have been integrated into INJECT, an open-source TTX learning platform, to support scalability and teaching practice. To encourage community adoption, we publicly share all
datasets, software tools, and a full-fledged TTX scenario.
Read the full abstract ▾
All authors Valdemar Švábenský, Jan Vykopal, Sukrit Leelaluk, Pavel Čeleda, Fumiya Okubo, Atsushi Shimada
Machine Learning
Conservative Query and Adaptive Regularization for Offline RL Under Uncertainty Estimation
Li-Rong Zhou, Qin-Wen Luo and Sheng-Jun Huang
Offline reinforcement learning (RL) aims to learn an effective policy from a static
dataset, but its performance is fundamentally limited by
dataset coverage. Action preference queries leverage expert feedback without additional environment interaction, enabling policy improvement during offline training. However, existing methods still face two key challenges: selecting informative preference queries and effectively exploiting the collected feedback. Current approaches typically rely only on the distance between policy actions and
dataset actions for query selection, while enforcing fixed constraints that keep the policy close to queried preferences. Such strategies often lead to unstable policy updates and integrate poorly with value regularization. To address these limitations, we propose Conservative Query and Adaptive Regularization under Uncertainty Estimation, a lightweight framework that jointly improves preference querying and preference exploitation. Specifically, we employ a Morse network to estimate the uncertainty of policy actions with respect to the offline
dataset. Based on this uncertainty, we introduce a conservative query strategy that selectively queries actions near the
dataset to preserve Bellman-update stability, together with an uncertainty-aware adaptive regularization scheme that dynamically adjusts data-level constraints during policy optimization. We integrate our framework with CQL and evaluate it extensively on the D4RL
benchmark. Experimental results demonstrate superior or competitive performance across a wide range of tasks.
Read the full abstract ▾
Machine Learning · Cross-listed from cond-mat.mtrl-sci
ATLAS: A Foundation Neural Sampler for Amorphous Materials
Mouyang Cheng, Denis Blessing, Botao Yu et al. (+4 more)
Amorphous materials exhibit exceptional mechanical and functional properties, yet their rugged energy landscapes are notoriously difficult to sample. Below the glass-transition temperature, conventional molecular dynamics and Monte Carlo become inefficient because equilibration relies on rare barrier-crossing events, while data-driven generative models are constrained by scarce and biased reference ensembles. Here, we introduce ATLAS, an efficient sampler that learns a
diffusion process to generate Boltzmann-distributed amorphous structures directly from a target energy function. Parameterized by an equivariant graph neural network, ATLAS generalizes across system size, temperature, and composition. By exploiting the time reversal of the
diffusion process, it enables efficient estimation of thermodynamic quantities and steering toward target observables. In two-dimensional Kob-Andersen systems, ATLAS reproduces parallel tempering Markov chain Monte Carlo structural distributions, free energies and
entropies, achieving below 0.2% free energy error in the low-temperature glass regime with over 500-fold fewer energy evaluations. In Cu-Zr and Cr-Co-Ni metallic glasses, ATLAS recovers experimentally observed short-range-order trends and steers structures toward prescribed order parameters and optimized bulk
moduli. Moreover, composition-amortized pretraining outperforms composition-specific training from scratch, reduces inverse-design costs by several hundred-fold, and enables sampling with expensive universal machine learning interatomic potentials. Coupled to a large
language model
agent, ATLAS searches an eight-element space for high-
entropy metallic glasses balancing stiffness and ductility, identifying a converged Pareto frontier within 480 oracle evaluations. Together, these results establish ATLAS as a foundation model for sampling, steering and designing amorphous materials.
Read the full abstract ▾
All authors Mouyang Cheng, Denis Blessing, Botao Yu, Gerhard Neumann, Mingda Li, Carles Domingo-Enrich, Yuanqi Du
Machine Learning · Cross-listed from cs.CV
ABot-World-0: Infinite Interactive World Rollout on a Single Desktop GPU
Fan Jiang, Zhaoxu Sun, Mengchao Wang et al. (+38 more)
We present ABot-World-0, an action-conditioned
video world model for real-time, long-horizon closed-loop interaction, supported by a multi-source data infrastructure spanning AAA games, simulation engines, and internet
videos to learn controllable world dynamics. WorldExplorer performs
agent-driven collection guided by training feedback, while a unified pipeline applies 14 deterministic quality checks, VLM-based assessment, and synchronized action and text annotation. We progressively distill a bidirectional action-conditioned teacher into a causal student through teacher forcing and ODE
distillation, and introduce LongForcing to align long student self-rollouts with an extended-horizon teacher, mitigating accumulated distribution shift and autoregressive drift. Raw keyboard actions provide a unified control interface for scene roaming and third-person character interaction, while reference-character
memory provides persistent appearance cues for identity consistency during third-person rollouts. For deployment, we co-design a streaming inference stack with a lightweight VAE decoder, efficient
attention,
memory-aware scheduling, and low-bit DiT inference. Across optimized low-bit configurations, ABot-World-0 streams 720P
video at up to 16 FPS on a single NVIDIA RTX 5090 desktop GPU, with 1.2s action-to-first-frame latency and approximately 19GiB peak VRAM. Experiments on WorldRoamBench and extended interactive rollouts demonstrate competitive controllability and coherent long-horizon world
evolution.
Read the full abstract ▾
All authors Fan Jiang, Zhaoxu Sun, Mengchao Wang, Ziyu Zhu, Chiyu Wang, Yunpeng Zhang, Wenlin Liu, Yun Wang, Xue Zheng, Rui Sun, Junfeng Ni, Hongyu Pan, Zhongxu Sun, Fei Yu, Zengye Ge, Mengmeng Du, Nianfei Fan, Mingchao Sun, Yu Liu, Yongchang, Yanqing Zhu, Jiahang Wang, Ning Ying, Yuze Xuan, Di Yang, Zhicheng Liu, Zhe Gao, Tingbing Xu, Jiacheng Sui, Wenjin Yang, Junnan Lai, Shufeng Liu, Yuan Liu, Zheng Zhou, Yingliang Peng, Dawei Cao, Kaifeng Sheng, Yuxiang Cai, Fei Lu, Mu Xu, Ning Guo
Machine Learning · Cross-listed from cs.CL
Reasoning Before Translation: Enhancing Legal Machine Translation with Structured Reasoning
Aixiu An, Michael Jungo, Eloi Eynard et al. (+4 more)
Neural machine translation (NMT) in the legal domain is a linguistically and conceptually demanding task, primarily due to the complexity of legal
language and the high level of precision it requires. The recent emergence of
reasoning-capable
language models opens new possibilities for tackling such challenges. They add to a set of other previously proposed techniques to enhance the translation quality, which includes supervised fine-tuning and reinforcement learning. In this work, we perform a comparison between these various approaches. More particularly, we evaluate small
language models such as Qwen3.5 4B, Qwen3.5 9B, and Gemma 3 12B enhanced with various re-training paradigms and compare their performances against frontier
reasoning models. We focus on the Swiss legal system, which – with its unique
multilingual statutes – offers a particularly challenging testbed for
reasoning-augmented models. Our results show that the quality of small “base” models can be greatly enhanced, and that reinforcement learning with verifiable rewards can be applied to NMT in the legal domain and surpasses the translation quality of supervised fine-tuning. The performance of enhanced small models is close to the one of state-of-the-art
reasoning models yet remains inferior. We also note that re-training paradigms yield diminishing returns as model size increase. The code and models are publicly available at https://github.com/aixiuxiuxiu/Legal-MT-SFT-RL.
Read the full abstract ▾
All authors Aixiu An, Michael Jungo, Eloi Eynard, Mark Drenhaus, Andreas Fischer, Jean Hennebert, Sébastien Rumley
Machine Learning
Neural Kolmogorov Equations: Parallelizable Learning of Stochastic Dynamics under General Noise
Arthur Bizzi and Olga Fink
Neural
stochastic differential equations (SDEs) have emerged as powerful tools for learning noisy or
stochastic dynamics directly from data; however, existing approaches largely assume uncoupled and continuous noise, limiting their applicability to realistic
stochastic drivers, and often scale poorly in time, requiring expensive autoregressive training. To address these limitations, we propose Neural Kolmogorov Equations (NKEs), a deterministic, infinite-dimensional reformulation of Neural SDEs based on the Kolmogorov Forward equation, transforming the learning problem from modelling individual
stochastic trajectories to modelling the
evolution of probability densities. NKEs learn general Lévy-type
stochastic forcing directly through the operator structure of the KFE, and enable parallel-in-time training via a Lagrangian Galerkin projection and operator splitting. We evaluate NKEs on several
stochastic benchmarks, including systems with coupled noise and jump processes, and verify that NKEs provide flexible models that accurately recover deterministic and
stochastic dynamics with competitive predictive accuracy and improved training efficiency. Code and pretrained models will be released.
Read the full abstract ▾
Machine Learning · Cross-listed from math.NA
$H^2(\Omega)$ A Priori Error Bounds for Boundary-Adapted PINNs Solving Elliptic Dirichlet Problems
Nathanael Tepakbong, Jun Fan, Xiang Zhou et al. (+1 more)
We study the numerical resolution of uniformly
elliptic Dirichlet boundary value problems (BVPs) by minimization of the PDE residual over a hypothesis space of Neural Networks made to satisfy exactly the boundary conditions through multiplication by a user-defined distance-to-boundary approximation $\rho$. While an $L^2(\partial\Omega)$ boundary penalty cannot generally guarantee $H^2(\Omega)$ error control, we show that exact enforcement can, provided $\rho$ is chosen appropriately. More precisely, we identify a sharp sufficient condition on $\rho$ preserving the regularity needed for $H^2(\Omega)$ error control, provide practical constructions of such $\rho$ for general smooth domains $\Omega$, and refer to the resulting networks as boundary-adapted. For boundary-adapted Rectified Quadratic Unit (ReQU) and
hyperbolic tangent (tanh) networks, we then derive explicit high-probability $H^2(\Omega)$ convergence rates by combining approximation-theoretic and statistical-learning arguments, making explicit the dependence on $\rho$. Numerical experiments support our theoretical results, showing that appropriate choices of $\rho$ improve accuracy and convergence, while poorly chosen distance functions can substantially degrade the solution. Our proof also yields new VC-dimension bounds for hypothesis spaces of higher-order derivatives of ReQU and tanh networks, together with new approximation bounds for shallow ReQU networks in higher-order Sobolev norms, all of which are of important independent interest.
Read the full abstract ▾
All authors Nathanael Tepakbong, Jun Fan, Xiang Zhou, Ding-Xuan Zhou
Machine Learning
Breaking the Homogeneity Assumption: Specialized Multi-Generator Adversarial Learning for Rare Failure Detection in Predictive Maintenance
Alexis Lazanas and Georgios Kampouropoulos
Supervised learning models in the predictive maintenance field are regularly trained on highly imbalanced industrial
datasets: machine failures occur rarely but have a disproportionate effect on operations. In addition to the clear class disparity, failure data are typically non-homogeneous, with different failure modes arising from distinct physical processes and exhibiting a
multimodal distribution across minorities and classes. Traditional imbalance-management methods, e.g., undersampling, SMOTE-based interpolation, or cost-sensitive learning, typically assume that the minority population is homogeneous. This means their effectiveness is severely limited in the multifaceted conditions encountered in industrial practice. This paper determines the possibility of a failure-type-conscious generative augmentation program to improve the identification of infrequent failures in predictive maintenance systems. An experimental design that is leakage-safe is used to compare five imbalance-handling methods: cost-sensitive learning, random undersampling, SMOTE oversampling, single-generator GAN augmentation, and a specialized multi-generator GAN architecture that has independent generators that are asked to learn individual failure subtypes. Precision/Recall-oriented measures are used to quantify model performance; the main evaluation measure is the PR-AUC. Experiments conducted on the AI4I 2020 predictive maintenance
dataset indicate that the proposed multi-generator GAN framework produces more realistic minority samples, yielding higher PR-AUC and recall scores compared to traditional resampling methods and individual-generator GAN augmentation.
Read the full abstract ▾
Machine Learning
Incomplete Observations Boost Evolutionary Performance in Ocean Modeling
Yangyang Kong, Yutong Jiang, Yanhai Gan et al. (+3 more)
Data-driven methods have revolutionized ocean modeling, yet current approaches rely heavily on complete reanalysis
datasets, imposing computational constraints and limiting model performance to that of the training data. Here, we present a generative state-space model and an optimization framework that enable learning directly from sparse and noisy observations. The model is essentially a hidden Markov model with a continuous state space, where oceanic physical quantities are treated as hidden states and measurements as observations, enabling a unified representation of ocean fields and observational data. Both the initial-state and state-transition modules are implemented as neural networks to capture the complexity and temporal
evolution of ocean states, while the emission module is formulated as a masked Gaussian distribution. To train the model from sparse observations, we derive an optimization framework based on the expectation-maximization (EM)
algorithm. The framework alternately reconstructs high-fidelity ocean fields via Langevin dynamics and optimizes deep neural networks to capture temporal
evolution. Theoretical analysis shows that the framework maximizes the likelihood of observations under the generative model. For efficiency, we assume that ocean-state
evolution follows a stationary, ergodic, and Markovian
stochastic process and adopt only length-two state sequences during optimization. Experiments on CMIP6 simulation data and FY-3D satellite data demonstrate high-fidelity reconstruction and accurate prediction, showing that sparse observations can directly improve the model's representation of ocean-state dynamics. This work offers a scalable pathway for next-generation Earth system models to learn directly from sparse, incomplete real-world observations.
Read the full abstract ▾
All authors Yangyang Kong, Yutong Jiang, Yanhai Gan, Junyu Dong, Feng Gao, Xiaopei Lin
Machine Learning · Cross-listed from eess.IV
MIRAGE: Multi-scale Lesion-Informed Representation with Auxiliary Guidance for MRI Contrast Enhancement
Andrea Borghesi, Xin Wang, Jonas Teuwen et al. (+1 more)
Inferring contrast enhancement from one pre-contrast breast MRI slice is underdetermined: post-contrast appearance contains physiological information that is not uniquely encoded in baseline anatomy. Optimizing only paired pixel fidelity can suppress uncertain lesion enhancement, whereas
adversarial or
stochastic generative objectives can favor realistic post-contrast appearance without guaranteeing patient-specific lesion fidelity. We introduce MIRAGE, a residual 2D U-Net that combines global reconstruction and perceptual losses with three forms of lesion-aware supervision available only during training: an asymmetric penalty for missed tumor enhancement, multi-scale auxiliary tumor
segmentation, and guidance through a frozen post-contrast tumor
segmentation nnU-Net. We evaluate the method on 301 cases from the multi-centre MAMA-SYNTH data using eight complementary image-, region-, radiomics-, and
segmentation-based metrics. MIRAGE ranks first on six metrics and markedly improves downstream lesion localization over tuned pix2pix, conditional
diffusion, and latent bridge-matching baselines. The generative alternatives retain advantages in LPIPS or contrast classification, revealing a clear fidelity-utility trade-off. Leave-one-in and leave-one-out ablations show that the losses are partly redundant for lesion localization but exert distinct effects on appearance, radiomics, and boundary accuracy. These results support task-aware synthesis while also showing that its apparent optimality is conditional on the downstream models and metrics used to define utility.
Read the full abstract ▾
All authors Andrea Borghesi, Xin Wang, Jonas Teuwen, George Yiasemis
Machine Learning
One Model, Many Graphs: Learning over Attributed Graphs across Heterogeneous Modalities with Vision-Language Models
Jiayi Yang, Yifang Chen, Yuanfu Sun et al. (+2 more)
Vision-
language models (VLMs) provide a unified representation space for textual and visual information, yet their potential as general-purpose backbones for graph-structured data remains largely unexplored. In practice, attributed graphs exhibit substantial modality heterogeneity: some graphs contain only textual node attributes, others only visual attributes, while still others provide both. Existing graph learning approaches are typically designed for fixed modality schemas, requiring separate models for different settings and limiting scalability and cross-graph generalization. To bridge this gap, we present OMG-VLM (One Model, Many Graphs with
Vision-
Language Models), a unified framework for learning over attributed graphs across heterogeneous modality schemas. OMG-VLM leverages a pretrained VLM as a shared backbone and introduces structure-aware graph adapters that integrate neighborhood information while remaining compatible with the VLM's native
embedding space. This design enables effective learning over text-attributed, image-attributed, and
multimodal-attributed graphs within a single model. Extensive experiments across diverse domains show that OMG-VLM consistently outperforms state-of-the-art GNN- and LLM-based baselines on attributed graph learning tasks such as node classification and link prediction, while exhibiting strong generalization to unseen graphs and varying modality schemas. The source code is available at https://github.com/Jo-eyang/OMG-VLM.
Read the full abstract ▾
All authors Jiayi Yang, Yifang Chen, Yuanfu Sun, Jiajin Liu, Qiaoyu Tan
Machine Learning
Parallel Noising in Neural Markov Logic Networks
Peter Jung, Giuseppe Marra and Ondrej Kuzelka
Neural Markov Logic Networks (NMLNs) are a flexible neurosymbolic relational model. Previous work has shown that, although NMLNs achieve strong performance as generative models for small relational structures, they underperform
diffusion-based generative graph models on larger structures. In this paper, we strengthen NMLNs along two main dimensions: (i) we increase the expressive capacity of their potential functions using graph neural networks, and (ii) we develop a new training and inference
algorithm inspired by parallel-tempering Markov chain Monte Carlo methods, which we name parallel noising. Together, these enhancements enable NMLNs to attain strong performance in graph generation relative to general
diffusion-based generative graph models. Furthermore, they allow NMLNs to match the performance of specialized text-based recurrent models when generating small molecular structures.
Read the full abstract ▾
Machine Learning
Predicting Activities in Aqueous Electrolyte Solutions with Hybrid Machine Learning
Zeno Romero, Maximilian Kohns and Fabian Jirasek
Activities in aqueous electrolyte solutions, usually described by ionic activity and osmotic coefficients, are important properties for modeling many processes in industry and nature. Established activity models, such as those of Pitzer or Bromley, require fitting to experimental data for each electrolyte of interest and thus cannot predict properties for unstudied systems. While some predictive approaches exist, they are typically limited in scope and rely on additional ion-specific descriptors. In this work, we introduce a new hybrid model that combines the physics-based Bromley model with a
matrix completion method (MCM) from machine learning. The MCM is employed to predict the electrolyte-specific parameters of the Bromley model, exploiting the fact that these parameters can be arranged in a
matrix with cations and anions as rows and columns, respectively. Due to the lack of experimental data for many electrolytes, the initial parameter
matrix is sparsely populated, making the prediction of the Bromley parameters for unstudied electrolytes a
matrix completion problem. The hybrid model, Bromley-MCM, was trained end-to-end on experimental data for mean ionic activity coefficients and osmotic coefficients of aqueous solutions of 478 electrolytes at 298 K from the Dortmund Data Bank. As output, we obtain a completed
matrix of Bromley parameters for 83 cations and 112 anions, enabling consistent prediction of concentration-dependent activities in aqueous solutions of 9,296 electrolytes at 298~K. This substantially extends the applicability of the Bromley model while maintaining high predictive accuracy, as demonstrated through evaluations on electrolytes excluded from model training.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Translation as Augmentation: Effect of Translated Data on Assessment of Difficulty
Yiheng Wu, Jue Hou and Roman Yangarber
Reliable Text Difficulty Assessment is a prerequisite for valid text simplification workflows and personalized learning applications. However, the development of robust assessment models is severely hindered by a critical bottleneck: the scarcity of expert-annotated corpora containing fine-grained difficulty levels (e.g., CEFR), particularly for lower-resource
languages. This paper addresses this data scarcity problem in the context of a low-resource European
language. We propose a cross-lingual data augmentation strategy that leverages machine translation to transfer labeled resources from high-resource
languages to the target low-resource
language. We train BERT-based regression models to predict difficulty scores and investigate whether synthetic, translated data can effectively supplement native training sets. Our experiments demonstrate that augmenting scarce native data with machine-translated corpora significantly improves the accuracy of difficulty estimation, offering a viable solution for
languages lacking extensive expert annotations.
Read the full abstract ▾
Machine Learning
An unsupervised clustering analysis of breast cancer data derived from electronic health records enhanced through UMAP dimensionality reduction
Davide Chicco and Nicoletta Benvenuto
Breast cancer is one of the most widespread types of cancer, affecting approximately 8 million women worldwide. Electronic health records of patients diagnosed with this disease can serve as valuable
datasets for computational analyses, enabling the discovery of new insights about the pathology. Unsupervised clustering, in particular, can identify groups of patients with medically significant features, revealing data trends that might otherwise go unnoticed by medical doctors. In this study, we first applied the DBSCAN density-based clustering method to three independent
datasets derived from electronic medical records of patients with mammary carcinoma. Subsequently, to enhance our results, we preceded the DBSCAN application with a dimensionality reduction phase using UMAP. We evaluated our clustering outcomes using three statistical indices (DBCV, DCSI, and DISCO). Our results confirm the effectiveness of combining UMAP with DBSCAN for clustering data derived from electronic health records, paving the way for the medical interpretation of the patient groups identified by our approach.
Read the full abstract ▾
Machine Learning
GEqTrain: A Configuration-Driven Framework for Retargeting Equivariant Graph Neural Networks Across 3D Scientific Tasks
Daniele Angioletti, Marco Nobile and Vittorio Limongelli
Equivariant graph neural networks provide a powerful modeling
language for three-dimensional scientific data, but their reuse is often limited by implementations tied to specific tasks, outputs, and training regimes. We present GEqTrain, a configuration-driven framework that separates
dataset semantics, model composition, and training objectives. Raw data are mapped to typed node-, edge-, and graph-level fields, while model stacks, losses, and training workflows are assembled declaratively through Hydra configurations. A shared equivariant backbone and training infrastructure can therefore be retargeted to a new task primarily through configuration. We demonstrate this flexibility on three different problems handled within one software stack: coarse-grained-to-atomistic backmapping of biomolecular systems, prediction of NMR chemical shifts in molecular solids, and equivariant generative modeling. Our aim is not to surpass individually optimized task-specific systems, but to show that a shared representation and training infrastructure can achieve competitive accuracy across qualitatively different tasks at the cost of a configuration change. We further introduce GEqDiff, a generative extension based on equivariant flow matching. GEqDiff treats user-defined equivariant fields as first-class generation targets, jointly transporting Cartesian positions and non-scalar node fields spanning representations up to l=3 within a single equivariant flow. We validate this capability on a controlled synthetic
benchmark inspired by
protein secondary-structure motifs, showing that fields with heterogeneous transformation properties can be reconstructed jointly and with high fidelity. By reducing the software overhead of moving between predictive and generative, scalar and tensorial settings, GEqTrain aims to make equivariant modeling more reproducible, extensible, and reusable.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Mage-Flow: An Efficient Native-Resolution Foundation Model for Image Generation and Editing
Xinjie Zhang, Peng Zhang, Shicheng Zheng et al. (+21 more)
Large-scale visual generators are increasingly capable but costly to train, fine-tune, and deploy. We introduce Mage-Flow, a compact 4B-scale generative stack for efficient text-to-image generation and instruction-based image editing. The stack is built from two co-designed components: Mage-VAE, a lightweight high-fidelity latent tokenizer, and a Native-Resolution
Multimodal Diffusion Transformer trained with rectified flow matching. Mage-VAE uses one-step
diffusion-style encoding and decoding with anchor-latent regularization, preserving the reconstruction quality of strong public VAEs while reducing tokenization cost by more than an order of magnitude. Together with native-resolution packing and stack-level CUDA kernel fusion, the stack supports flexible-resolution training and improves end-to-end training throughput by about $2.5\times$. Built on this foundation, we develop a complete model family with Base, RL-aligned, and Turbo variants for both generation and editing.
Diffusion-NFT improves
prompt following, text rendering, aesthetic quality, and editing fidelity, while few-step
distillation with
adversarial perceptual guidance produces 4-step Turbo models for low-latency inference. Despite its compact scale, Mage-Flow and Mage-Flow-Edit achieves competitive performance across standard generation and editing
benchmarks. More importantly, the Turbo variants make high-resolution generation and editing practical for interactive use: at $1024^2$ resolution on a single NVIDIA A100 GPU, Mage-Flow-Turbo generates an image in 0.59s, and Mage-Flow-Edit-Turbo edits an image in 1.02s, while maintaining a small
memory footprint. These results show that careful tokenizer–backbone–system co-design can deliver strong high-resolution generation and editing within an efficient 4B model family.
Read the full abstract ▾
All authors Xinjie Zhang, Peng Zhang, Shicheng Zheng, Jinghao Guo, Zhaoyang Jia, Yifei Shen, Xun Guo, Yuxuan Luo, Jiahao Li, Wenxuan Xie, Fanyi Pu, Xiaoyi Zhang, Kaichen Zhang, Zongyu Guo, Tianci Bi, Dongnan Gui, Zhening Liu, Zimo Wen, Zihan Zheng, Senqiao Yang, Xiao Li, Jinglu Wang, Bin Li, Yan Lu
Machine Learning
Deep learning-based prediction of time-resolved adhesive forces in viscoelastic Hertzian contacts
Ali Maghami, Merten Stender, Michele Ciavarella et al. (+1 more)
Fast prediction of the response of adhesive soft viscoelastic contacts represents a current challenge in soft robotics and for gripping and manipulation tasks. Determining the complete time-resolved force trajectory requires full numerical simulations, whose computational cost is strongly parameter-dependent, making them impractical for real-time application or design-optimization loops. In this work, we overcome this limitation by training a scalar-conditioned, stateful, sequence-to-sequence deep learning model to predict the full force
evolution from a prescribed displacement history for both short- and long-range adhesion regimes. The data set spans four orders of magnitude in loading and unloading rates and includes varied dwell times, with the Tabor parameter ranging from $0.2$ to $3.2$. To enable learning across these heterogeneous time scales, we introduce a fixed-measurement-step (FMS) representation that converts variable-length trajectories into fixed-length sequences while preserving their physical-time information. Different architectures were trained, including long short-term
memory (LSTM) networks, temporal convolutional neural (TCN) networks, and time-distributed dense layers with three different Tabor-conditioning mechanisms. The models were compared using global waveform and error metrics. We found that the best-performing model has an LSTM architecture with concatenated conditioning, which achieves a held-out mean-squared error of $5.0\times10^{-4}$, a median pull-off-force error of $\approx2.2\%$, and a median hysteresis error of $\approx1.1\%$. For the held-out protocols, the model predicts a complete force trajectory with a median inference time of $0.16$ s. The model is tested across unseen parameter combinations and against analytical limiting cases, providing a rapid surrogate for repeated numerical evaluations with potential use in control-oriented applications.
Read the full abstract ▾
All authors Ali Maghami, Merten Stender, Michele Ciavarella, Antonio Papangelo
Machine Learning
Where Should Optimizer State Live? Tiered State Allocation for Memory-Efficient Mixture-of-Experts Training
Nuemaan Malik
Optimizer state is the largest single line item in the
memory budget of mixture-of-experts (MoE) training. On a 6.78B-parameter MoE
language model AdamW keeps 50.6 GB of first and second moments to update 12.6 GB of bfloat16 weights. We study SkewAdam, an optimizer built on the observation that the three parameter populations of an MoE differ enough in size and gradient statistics that they should not receive the same state. Those populations are the dense backbone, the experts and the router. SkewAdam keeps float32 momentum plus a factored second moment for the backbone (5% of parameters), a factored second moment alone for the experts (95%) and an exact second moment for the router (<0.01%). The resulting state occupies 1.29 GB or 2.6% of AdamW's and peak training
memory falls from 81.4 GB to 31.3 GB, within the budget of a 40 GB accelerator. In a controlled comparison from identical initializations over 82M
tokens, SkewAdam reaches validation perplexity 108.4, ahead of AdamW (126.8),
Muon (120.2) and Lion (393.7), and settles router load balance to within 1% of its uniform floor. The allocation is not what earns that perplexity. A tier ablation reaches the same value while carrying twenty times the state, so the tiers buy
memory rather than accuracy. Same-platform runs separate what does earn it. Removing momentum costs 31 perplexity points (tuned Adafactor, 139.7) and replacing the factored second moment and its update clipping with a full second moment costs 10 (tuned AdamW, 118.5), so neither tuned baseline reaches the untuned tiered policy. Where optimizer state lives, these results suggest, matters at least as much as how much of it there is.
Read the full abstract ▾
Machine Learning
Probabilistic Physics-Aware Machine Learning Predictions of Electric Truck Energy Consumption with Field Data
Hannes Nilsson, Rafael Basso, Balázs Kulcsár et al. (+1 more)
In this work, we incorporate first principle physics into the construction of data-driven methods by considering a model that accounts for the different sources of energy losses during vehicle operations. Our results show that
Bayesian linear regression based on this physics-aware model can improve the reliability of the expected energy consumption, as compared with standard linear regression. Further, it is shown that more complex machine learning models such as neural networks and gradient boosted regression trees, based on the same physical model, can further improve the accuracy in energy
forecasting and significantly outperform standard versions of the same machine learning models. In addition to point predictions of the energy consumption, we develop a framework for estimating the corresponding uncertainty in the form of predicted standard deviation. Our results show that all of the models learn to estimate the uncertainty reasonably well.
Read the full abstract ▾
All authors Hannes Nilsson, Rafael Basso, Balázs Kulcsár, Morteza Haghir Chehreghani
Machine Learning
Adopting Reinforcement Learning with Verifiable Rewards for Molecular Generation
Mingxuan Ouyang, Hao Lan and Wanyu Lin
Leveraging large
language models (LLMs) for molecular generation has shown remarkable potential in chemical and drug design. Current methods primarily rely on supervised training or fine-tuning with limited
datasets, which are insufficient to capture complex molecular design objectives. While some approaches attempt to guide generation toward specific goals, they often lack direct optimization mechanisms, making it difficult to align generated molecules with desired properties. To tackle these challenges, we propose LLMol, a principled reinforcement learning framework that directly incorporates verifiable rewards for targeted molecule generation. The key insight is to formulate molecular design as a goal-conditioned sequence prediction task, where verifiable rewards serve as explicit supervision to drive generation toward desired objectives. LLMol follows a two-stage training paradigm combining supervised learning and reinforcement learning. In the first stage, large
language models are supervised fine-tuned to capture chemical syntax and molecular distributions. In the second stage, we introduce Reinforcement Learning with Verifiable Rewards (RLVR), which directly integrates property-based reward signals to guide molecular generation toward task-specific objectives. To address the high variance and instability common in discrete sequence optimization, we adopt Group Relative Policy Optimization (GRPO), a stable on-policy
algorithm that smooths reward signals and improves training robustness. This framework enables LLMol to effectively handle a range of molecular design tasks, including single-property targeting (e.g., penalized logP, QED) and structure-constrained optimization. Experimental results demonstrate that LLMol consistently outperforms existing methods, achieving higher success rates and improved efficiency across diverse molecular
benchmarks.
Read the full abstract ▾
Machine Learning
Spectral Higher-Order Neural Networks Have Sharp Expressivity Bounds
Gianluca Peri, Diego Febbe and Duccio Fanelli
Neural hypergraphs are a natural generalization of neural networks, the reference models in modern machine learning. Yet, their deployment has proven demanding: the number of weighted hyperedges required leads to an intractable parameter explosion. However, a novel parametrization that leverages spectral attributes for neural hypergraphs has been recently proposed, that enables to recycle parameters via a weight sharing scheme and consequently yields a significant reduction of the associated computational cost. Preliminary tests carried out on spectral higher-order architectures pointed to meaningful improvements in both performance and interpretability. Building on these results, we advance the benchmarking efforts by evaluating the spectral higher order framework on N-bit parity tasks, a well-established testbed known to be particularly challenging. As we will convincingly argue, Spectral Higher-Order Neural Networks (SHONNs) possess a versatile and highly tunable hypothesis space.
Read the full abstract ▾
Machine Learning
Unsupervised Multi-kernel Learning for Automated Algorithm Selection
Yihang Lu, Tome Eftimov and Carola Doerr
Automated
algorithm selection in black-box optimization typically relies on supervised models that map landscape features to
algorithm performance labels. Such models are costly to train,
benchmark-dependent, and often fail to generalize to unseen problem classes. We study an unsupervised alternative: multi-kernel clustering over heterogeneous landscape representations, in which problem instances are grouped without using performance labels in the clustering stage, and the resulting clusters are mapped post hoc to solver recommendations through a strictly separated three-stage evaluation protocol. Drawing on two decades of advances in multiple kernel learning, we adopt a multi-kernel k-means formulation that jointly learns cluster assignments and kernel weights over four heterogeneous landscape views: ELA, DeepELA, DoE2Vec, and TransOptAS. On affine BBOB-derived selector tasks for Differential
Evolution (DE) and Particle Swarm Optimization (PSO) at a fixed evaluation budget, we report mean plus or minus standard deviation selector profiles over 50 independent random seeds for
stochastic configurations. Multi-kernel clustering obtains the strongest mean profile on the DE portfolio and remains competitive with, and nominally ahead of, the leading baselines on the more compressed PSO portfolio, where differences among the best methods are small relative to
stochastic variation. In representative median-seed runs used for visualization, the learned kernel weights retain ELA and TransOptAS while assigning zero weight to DeepELA and DoE2Vec, providing a task-specific interpretation of which representations are retained by the multi-kernel model for selector-oriented grouping.
Read the full abstract ▾
Machine Learning
Audit Before You Update: Recording Artifacts and the Limits of Bounded Model Updates in MIMIC-IV ICU Prediction
Fatema Ferdous Tamanna, K. M. Merajul Arefin and Md. Abdul Masud
Clinical prediction models in intensive care degrade as data shift, and regulators such as the FDA ask developers to specify bounded update protocols in advance. Evaluations of such protocols assume that apparent drift reflects clinical practice rather than changes in how the database records care. We test whether update strategies can be evaluated on unaudited ICU data, and whether a parameter-bounded update gives up accuracy or attribution stability relative to full fine-tuning once the data are audited. We audit MIMIC-IV 3.1 (2008-2022) for recording artifacts and define a cleaning protocol. On the cleaned cohort, a label-free classifier two-sample test gates updates across three eras. We compare a bounded update, which retrains only treatment-context and fusion parameters, with full fine-tuning for two model families and two targets under a pre-specified non-inferiority margin of 0.015 AUROC, and measure attribution stability against the source model. In 2020-2022 about half of ICU stays have no input or procedure-event records, and a vasopressor ItemID migration makes the 2017-2019 label rate appear to fall when it actually rises. On the cleaned cohort, the gate detects drift in every era, including in physiology-only features, with no false alarms on null data. Bounded updating is non-inferior or superior in 10 of 12 primary
cells, but this depends on the target, the model family and learning-rate tuning, and adaptation gains over never updating are small. At a shared learning rate, bounded updates keep physiological attributions more stable in neural models, clearly for vasopressor and less clearly for intubation; with separately tuned rates, full updates are more stable. Audits should precede update evaluation, and the update boundary should be chosen per target.
Read the full abstract ▾
Machine Learning
Subject-Conditioned Glucose Forecasting in Type-1 Diabetes
Giorgia Rigamonti, Mirko Paolo Barbato, Davide Marelli et al. (+1 more)
Accurate
forecasting of blood glucose concentration is key in the management of Type 1 Diabetes, facilitating early detection of adverse glycemic events and supporting timely therapeutic interventions. Despite recent advances in glucose prediction, most existing approaches rely on population-level representations or implicit personalization strategies that fail to deliver effective subject-specific forecasts. In this work, we propose Subject-Conditioned Glucose Prediction (SCGP), a novel
multimodal deep learning architecture conceived for personalized blood glucose prediction. SCGP conditions glucose predictions based on observed glucose data and a compact subject-specific representation learned from contextual information. By explicitly separating subject characterization from glucose dynamics modeling and avoiding early fusion of heterogeneous inputs, the proposed framework effectively captures inter-subject variability while preserving robust and reliable temporal modeling. Experiments on two state-of-the-art
benchmark datasets demonstrate that SCGP consistently improves
forecasting performance, enabling reliable detection of adverse glycemic events across multiple prediction horizons, highlighting the benefits of explicit subject conditioning for personalized diabetes management.
Read the full abstract ▾
All authors Giorgia Rigamonti, Mirko Paolo Barbato, Davide Marelli, Paolo Napoletano
Machine Learning · Cross-listed from stat.ML
The Tractability Landscape of Sampling with Inexact Scores
Anming Gu, Kevin Tian, Hubert Yang et al. (+1 more)
We provide a simple and tight characterization of the types of inexact score oracle access that permit sampling with vanishing total variation bias, for a standard, well-behaved target family. Our main result shows that any weaker error than the sub-Gaussian assumption used by [YW26] rules out the tractability of unbiased sampling. This strengthens the conclusion of [CCSW26] to be
algorithm-agnostic, and to hold for a wider range of error assumptions.
Read the full abstract ▾
All authors Anming Gu, Kevin Tian, Hubert Yang, Yusong Zhu
Machine Learning · Cross-listed from cs.CV
Benchmarking Deep Learning Approaches for AEC Engineering Drawing Layout Detection and Information Extraction
Tianyang Huang, Alessio Lombardi, Ahmed Elnagar et al. (+7 more)
Information Extraction (IE) from Architecture, Engineering, and Construction (AEC) drawings remains hindered by manual inefficiency, while Layout Detection, a vital 'middleware' organizing graphical and textual hierarchies, is underexplored. General document layout models, optimized for text-centric content, lack validation on engineering drawings. This study constructs a custom AEC-specific layouts
dataset and
benchmarks five deep learning architectures. RF-DETR achieves state-of-the-art performance with an $mAP_{50}$ of 0.949, while the
Vision-
Language Model Qwen3-VL attains a leading F1-score of 0.911. Conversely, models pre-trained on general document
datasets suffer from "domain interference", causing performance degradation. This establishes a robust technical foundation for automated IE in AEC.
Read the full abstract ▾
All authors Tianyang Huang, Alessio Lombardi, Ahmed Elnagar, Ahmed Zalouk, George Paul, Sepehr Najjarpour, Arvid Sigurdsson, Khalid Ismail, Mohamed Ragab, Edlira Vakaj
Machine Learning · Cross-listed from cs.CL
Verifiable Self-Evolution for Open-Ended Dialogue Skills via Future-Feedback Prediction
ChaoJin Zhao and Xuan Jiang
Textual skills provide a lightweight way to improve frozen
language-model
agents, but their self-
evolution normally requires a stable validation signal. Such signals are natural in mathematics or code, where an answer can be checked after it changes, yet are problematic in open-ended dialogue: changing the assistant response also changes the user's next reaction, so a logged reaction cannot directly evaluate a counterfactual response. We propose future-feedback skill
evolution, which first redirects self-
evolution from prescribing the current answer to predicting whether the observed answer will lead to a positive or negative subsequent user signal. This prediction task is verifiable on fixed logged tuples and therefore supports validation-gated textual optimization. The evolved feedback skill captures interpretable criteria for response quality and can subsequently serve as a diagnostic and optimization target for answer skills. On a proprietary,
privacy-preserving sales-assistant
dataset, careful quality filtering and a balanced resolved/unresolved split yield more than 75% prediction accuracy. Beyond this result, the central contribution is a formulation that converts otherwise moving conversational feedback into a fixed offline learning target, enabling reproducible skill
evolution without placing every candidate skill in live traffic. We discuss the boundary between observational verification and counterfactual validity, and position the method as an offline optimization stage rather than a replacement for final human or online evaluation.
Read the full abstract ▾