Machine Learning
Enhancing Transformer Representations of Symbolic ODE Expressions
Xiyue Fan, Adam Prugel-Bennett and Stuart E. Middleton
Existing approaches to solving differential equations, such as symbolic regression, physics informed neural networks, and neural operators, typically focus on numerical approximations or blind symbolic search via fitting to numerical data. Less
attention has been paid to learning structured representations of mathematical expressions that preserve commutative properties and could support mathematical
reasoning in symbolic forms.
Transformer models have shown strong capabilities in solving symbolic differential equations. However, standard positional
embeddings in
transformers are designed for sequence data. Symbolic differential equations are naturally represented by expression trees, so these positional
embeddings may not efficiently capture their hierarchical structures. We investigate existing tree positional
embeddings in symbolic ordinary differential equation (ODE) tasks. We systematically study their effectiveness under different settings. Our results show that tree positional
embeddings aid learning in early epochs and continue to improve performance throughout, ultimately yielding consistent advantages across various data sizes and tasks. Based on learned structural representations, we apply contrastive learning to support the commutative property in mathematics. Ablation studies provide insight into how these methods interact in modelling symbolic mathematical structures.
Read the full abstract ▾
Machine Learning
An Exact Junction-Tree Extended Formulation for Optimal Classification Trees
Jiancheng TU and WenqiFan
We develop an exact linear programming (LP) formulation for bounded-depth classification trees with binary features, using a junction-tree representation. The formulation is integral and supports recursive subtree optimization. Exact reductions make the model smaller while preserving the optimal value and recovery of an optimal tree. The reduced model supports two solution methods: column generation and message passing. Column generation solves integral restricted LPs and uses bounds over the full feasible domain to certify optimality. Message passing recursively combines optimal subtree costs. Both methods solve common subtree problems that, once the preceding tree decisions are fixed, can be evaluated independently and in parallel. Computational experiments show that the exact reductions substantially reduce the size of the junction-tree formulation. The resulting linear programming formulation certifies instances for which the tested mixed-integer formulation does not establish optimality within the same computational budget, while the column-generation and message-passing methods certify more instances and achieve an order-of-magnitude reduction in
geometric-mean runtime relative to an existing state-of-the-art exact method for optimal classification trees.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
MiTHras: Task-specific Hierarchical Semi-supervised Contrastive Masked Autoencoder for Mitotic Figure Analysis
Trinh T. L. Vuong, Simon Graham, Quoc Dang Vu et al. (+5 more)
Mitotic figure (MF) analysis supports tumor grading and prognostic assessment, but automated models remain sensitive to differences in tissue type and image acquisition. We present MiTHras, a task-specific pretraining framework that combines pseudo-label-guided image- and
token-level contrastive learning with masked reconstruction. We construct TCGA-MF-Pseudo, a corpus of 1.8 million
cell-centered images from 14 TCGA cohorts spanning 11 organ sites. Comprehensive evaluation on MF classification, detection, count-based survival prediction, and subtype classification demonstrates the efficacy of MiTHras. It achieves the highest mean F1 on all three MF classification
benchmarks and both subtype
benchmarks. MiTHras also outperforms general-purpose and pathology foundation encoders by a larger margin under frozen-encoder linear probing than under full fine-tuning. Although detection gains are modest due to a shared candidate-detection stage, ablations confirm that
token-level supervision improves typical-versus-atypical classification and linear probing. These findings establish that MiTHras yields robust, transferable representations for automated mitotic activity assessment.
Read the full abstract ▾
All authors Trinh T. L. Vuong, Simon Graham, Quoc Dang Vu, Phat T. H. Ho, Jeewoo Lim, Mostafa Jahanifar, Nasir Rajpoot, Jin T. Kwak
Machine Learning
A Federated Artificial Intelligence Framework for Optimizing Pancreatic Cancer Treatment - Strategy Update
Anne-Christin Hauschild, Amirreza Aleyasin, Nils H. Beyer et al. (+12 more)
While a centralized approach involving patient consent to collect and analyze data centrally would theoretically offer the best data quality and predictive performance, it is not always feasible in practice.
Federated Learning (FL) architectures have shown to be a very promising approach to use and access distributed disease related resources within the GDPR boundaries. In a previous case report, we described the preconditions at the participating sites and necessary administrative and process related steps to prepare data, people and infrastructure for improving subtype identification and assessing treatment options in pancreatic cancer. We update this report sharing our experience in tackling the challenges and show preliminary results of the actual
federated learning AI pipelines. At the participating sites, we have to identify and annotate the data being accessible after extraction and transformation in a local FL hub - in our case a centrally developed and distributively deployed Docker container. This container comprises the FL scripts generating local models. We apply a newly developed FL
algorithm considering all local features, including partial overlapping features specific to the local sites. Theoretically, an annotation in a cancer setting should succeed using the German oncology core data set (oBDS), which is already utilized for mandatory reporting to cancer registries, and can be sustained in the FL setting. The FL
algorithms deal robustly with partially overlapping features as we showed with public data sets. Major roadblocks including straightening operational concepts for the infrastructures, ethics approval for such novel architectures and support for every site have been addressed. However, scaling up this approach in the future faces hurdles; while including broader multi-modal data sets should be feasible, large-scale deployment to more sites remains challenging.
Read the full abstract ▾
All authors Anne-Christin Hauschild, Amirreza Aleyasin, Nils H. Beyer, Lisa Fricke, Jonas Hügel, Maryam Moradpour, Anh-Tien Nguyen, Youngjun Park, Sophia Rheinländer, Tim Beissbarth, Elisabeth Hessmann, Martin Middeke, Matthias Lauth, Maximilian Reichert, Ulrich Sax
Machine Learning · Cross-listed from eess.SY
Offline Reinforcement Learning for Distribution-Grid Protection
Julian Oelhaf, Alexander Luce, Christian Bergler et al. (+2 more)
Data-driven protection may complement conventional relays in distribution grids whose operating conditions vary with distributed generation, switching events, and changing short-
circuit levels. We study line-selective tripping from static trajectories of a realistically simulated CIGRE medium-voltage network using offline reinforcement learning. A convolutional Q-network receives causal voltage-current phasor and apparent-impedance features, optionally together with raw waveforms, and is trained with conservative Q-learning (CQL). A controlled sensitivity study evaluates two observation windows, reward variants, and three CQL weights under a common split and training protocol; one exploratory post-hoc run additionally increases the discount factor from $\gamma$=0.95 to 0.99. On 225 held-out episodes, the best per-timestep result is obtained with combined input and CQL weight $\alpha$=0.9, reaching precision 0.9993, recall 0.9496, and F1-score 0.9738. Because dense per-timestep scores do not encode the terminal semantics of relay operation, we also evaluate the first non-wait action in each episode. The default combined-input
agent selects the correct line-trip action first in 98.13% of 214 fault episodes, but trips in 72.73% of the 11 non-fault episodes. In the post-hoc run, the corresponding rates are 98.60% and 54.55%, respectively. The results show that dense predictive performance and terminal protection behavior can lead to different model rankings. Offline CQL therefore demonstrates strong faulted-line selection on the simulated fault episodes, while the static trajectories, small non-fault set, and single-seed post-hoc design preclude conclusions about practical relay security or deployment readiness.
Read the full abstract ▾
All authors Julian Oelhaf, Alexander Luce, Christian Bergler, Andreas Maier, Siming Bayer
Machine Learning
Guaranteed Low-Rank Tensor Recovery from Modewise Measurements via Normalized Block-Weighted Riemannian Gradient Descent
Yushi Zhou and Feng Zhang
We consider the recovery of low-multilinear-rank tensors from linear measurements and propose an adaptive block-weighted modewise Riemannian gradient descent method. The method combines
memory-efficient modewise measurements with a normalized adaptive weighting strategy for the core and factor components of the Riemannian gradient. The weighting improves convergence without increasing the multilinear-rank bound of the search direction or the size of the reduced core used for retraction. Under the tensor restricted isometry property and a suitable initialization, we establish local linear convergence and derive sampling guarantees for sub-Gaussian and subsampled orthogonal with random sign (SORS) measurements. Numerical experiments on synthetic low-Tucker-rank tensors show that the proposed method reduces iteration counts and computational time while maintaining reliable recovery performance, especially near the recovery threshold and for structured SORS measurements.
Read the full abstract ▾
Machine Learning
Muon Can Outperform Dedicated Continual Learning Methods
Sebastian George Sincari, Bogdan Alexandru Gheorghe and Antonio Barbalau
Continual learning with Low-Rank Adapters (LoRA) typically mitigates forgetting by penalizing the overlap between a new update and the accumulated past weights, which discourages certain update directions without controlling how an update distributes its energy over the ones that remain. We ask whether that restriction has to be task-aware, or whether a generic one supplied by the optimizer is enough. We train a plain incremental LoRA (IncLoRA) with
Muon, which orthogonalizes each update, and compare it against O-LoRA and ELLA over five seeds and three task orders on the Standard CL
Benchmark and three seeds on TRACE. IncLoRA+
Muon reaches the accuracy band of the dedicated methods on Standard CL and improves on every AdamW configuration on TRACE. One update-constraining mechanism is enough, whether it comes from the loss or from the optimizer; on Standard CL a second one does not help, and for the most restrictive method it costs 8.4 points of accuracy and the plasticity to fit each task. What separates the two optimizers is not the size of the update, which under
Muon is 0.91 to 2.06 times that under AdamW, but how it is distributed. AdamW confines it to between 1.4 and 1.8 effective singular directions,
Muon spreads it over 7.0, and the two do not overlap in any tracked run. Part of the advantage usually attributed to dedicated CL methods may therefore be explained by the geometry of the optimizer's updates.
Read the full abstract ▾
Machine Learning
Corrective Forcing: Unified Post-Training for Diffusions and Flows in Generative Speech Enhancement
Qing Yao, Lijian Gao and Qirong Mao
Diffusion and flow models, as promising generative paradigms for
speech enhancement, face a training–inference mismatch: training uses analytical path states, whereas inference recursively evaluates models on self-generated rollout states along discretized sampling trajectories. This mismatch causes prediction and discretization errors to accumulate. To address it, we introduce Corrective Forcing (CoF), a post-training paradigm that forces
diffusion and flow models to learn from self-generated rollouts and correct their predictions. CoF corrects clean-
speech predictions on rollout states toward the ground truth under dynamic sampling schedules, exposing the model to varying inference conditions. It further regularizes local
evolution using locally corrected counterfactual transitions as references for factual transitions. By expressing model outputs through a shared clean-
speech prediction parameterization, CoF applies the same post-training objective across
diffusion and flow formulations. Experiments with SB-VE and OT-CFM demonstrate improvements in perceptual quality and reconstruction fidelity, together with robust performance across different numbers of sampling steps.
Read the full abstract ▾
Machine Learning
iSDFT: Information-Proximal Self-Distillation for Continual Learning in LLMs
Ahmed Khaled Khamis, Xiaotong Ji, Hassan Jaber et al. (+4 more)
On-policy self-
distillation fine-tuning (SDFT) learns new skills from demonstrations while reducing forgetting, but it always distils toward the full demonstration-conditioned teacher. This fixes teacher influence at the full-teacher endpoint, providing no control over how much demonstration information should be transferred at each prediction state. We introduce Information-Proximal SDFT (iSDFT), which instead treats the teacher as a budgeted source of information. At each
token, iSDFT selects the distribution closest to the current student that satisfies a prescribed teacher-information constraint, yielding a closed-form exponential target with a locally determined tilt. To control cumulative drift, we further anchor the student to its frozen base policy. Across four heterogeneous LLM backbones and two specialisation tasks, iSDFT improves vanilla SDFT in 7 of 8 model-task settings and matches it in the remaining one. It also provides tighter retention on the original SDFT
benchmark suite, with 73% of evaluations remaining within 0.5 points of the base model versus 52% for the strongest baseline, while achieving the largest mean improvement on all ten additional mathematics, coding, and competition-mathematics
benchmarks. These results show that controlling how much and when teacher information is introduced improves specialisation while preserving broader capability.
Read the full abstract ▾
All authors Ahmed Khaled Khamis, Xiaotong Ji, Hassan Jaber, Rasul Tutunov, Matthieu Zimmer, Jun Wang, Haitham Bou-Ammar
Machine Learning
Augmented Hypothesis Testing with Persona-Based LLM Simulations
Ziyad Benomar, Aymen Al Marjani, Paul Missault et al. (+1 more)
A/B testing requires large sample sizes, long timelines, and significant costs. When auxiliary predictions of experimental outcomes are available from machine learning models, uncertain prediction quality precludes replacing human experiments entirely, yet these predictions may still contain useful signal. We propose a principled framework for learning-augmented hypothesis testing that leverages predictions of unknown quality to reduce sample sizes while maintaining statistical validity. Predictions naturally vary in granularity, from coarse aggregate signals to fine-grained individual-level estimates, and our framework addresses both ends of this spectrum: (1) for population-level directional predictions, where only a binary signal on the treatment effect sign is available, we use an asymmetric test and prove consistency and robustness bounds within the learning-augmented
algorithms paradigm; (2) for individual-level predictions, we introduce Generalized PPI++ (GPPI), extending Prediction-Powered Inference to handle nonlinear prediction errors through higher-dimensional transformations. Both methods benefit from accurate predictions while remaining robust to inaccurate or
adversarial ones. We validate our framework using persona-based LLM simulations, where AI
agents equipped with user personas predict individual behavior, as a natural prediction source spanning both granularity levels. Experiments on four real-world
datasets demonstrate that our methods, combined with persona-based predictions, substantially reduce experimental costs while preserving rigorous statistical validity.
Read the full abstract ▾
All authors Ziyad Benomar, Aymen Al Marjani, Paul Missault, Saab Mansour
Machine Learning · Cross-listed from cs.RO
Learning tactile perception from high-bandwidth single-point sensing
Joseph Rigal, Emmanuel Virot and Caroline Pascal
Tactile sensing is increasingly being incorporated into learning-based robotic manipulation, yet many existing approaches rely on spatially distributed sensors. Here we introduce {SpectRobot}, a framework that transforms single-point tactile signals into compact time-frequency spectrograms. These spectrograms encode high-bandwidth tactile histories as fixed-size image-like representations. They can be processed by standard
vision encoders and integrated into learning pipelines originally developed for
vision, while preserving temporal and frequency information unavailable to conventional cameras. Rather than increasing spatial density through arrays of tactile elements, SpectRobot exploits the rich dynamics contained in sparse, high-bandwidth single-point measurements. In our implementation, the sensors are mounted away from the contact surface while remaining mechanically coupled to it, reducing direct exposure to wear and potentially improving robustness in harsh environments and for long-term deployment on dexterous
robots. Our experiments demonstrate that: (1) a
robot can exploit single-point vibration signals to solve a visually occluded manipulation task; (2) temporal history strongly influences policy performance, while sensing bandwidth controls the spectral information available, with measurements extending to 100~kHz; and (3) the same representation can be used across different tactile sensing technologies mediated by acceleration, force, or strain. We further show that capabilities previously associated with research-grade instrumentation can be accessed using readily available, off-the-shelf hardware. We believe that broader access to high-bandwidth tactile sensing could facilitate the integration of contact dynamics into
embodied learning systems and, for some tasks, offer an alternative or complement to increasing the spatial density of tactile sensing.
Read the full abstract ▾
Machine Learning
GraphToolbox: A Configurable Python Framework for Graph Neural Network Forecasting
Eloi Campagne, Yvenn Amara-Ouali, Yannig Goude et al. (+1 more)
Electricity
forecasting often involves spatially related signals observed over regions, substations, and feeders, and Graph Neural Networks (GNNs) provide a natural way to represent these relations. Building a complete GNN
forecasting experiment is nonetheless laborious, because graph construction, model selection, training, aggregation, and interpretation sit in incompatible tools. We present GraphToolbox, an open-source Python framework that unifies these stages in one configurationdriven pipeline built on PyTorch
Geometric. It offers data-driven graph construction, an adapter that instantiates and trains 51 of the 65 PyTorch
Geometric convolutions together with the recurrent
cells of PyTorch
Geometric Temporal, online expert aggregation,
forecasting interpretability, and significance testing on cached forecasts. We evaluate the pipeline in two case studies. On French regional load, the 48 convolutions included in the complete
forecasting sweep fall in a band from 1.14% to 1.60% error, online aggregation lowers this to 0.98%, and the graph models improve on classical additive and boosting baselines. On net-load, direct graph models are less accurate than a classical additive model, while
forecasting each physical component separately improves them without closing that gap. Both comparisons use the same experimental interface, illustrating the role of GraphToolbox in systematic architectural evaluation.
Read the full abstract ▾
All authors Eloi Campagne, Yvenn Amara-Ouali, Yannig Goude, Argyris Kalogeratos
Machine Learning
Taking a Second Look: Correcting Sea Ice Forecasts with Sparse Observations
Tianshuo Zhang, Xianglei Xing, Aowen Yang et al. (+4 more)
Sea ice forecasts are issued several days ahead, allowing errors to accumulate while new, often sparse sea ice concentration (SIC) observations become available. We find that fixed-propagation errors concentrate near structured, high-gradient ice edges, whereas homogeneous interiors require limited propagation, suggesting that propagation distance should be state dependent. We therefore introduce ECHO (Evidence-guided Correction with Heterogeneous prOpagation), where ECHO-Scale adapts propagation distance while preserving correction geometry, and ECHO-Delta learns a bounded residual around fixed propagation. Across all 96 standard evaluation settings spanning diverse priors, observation times, sparsity levels, geometries, and noise conditions, both outperform fixed propagation. ECHO-Delta achieves the best average accuracy, while ECHO-Scale is more robust to geometry shifts. Code is available at https://github.com/yingtian22/TAKING-A-SECOND-LOOK.
Read the full abstract ▾
All authors Tianshuo Zhang, Xianglei Xing, Aowen Yang, Jia Gao, Wenzhe Zhai, Shanshan Liu, Chengtao Cai
Machine Learning
Overlay\_dx - Automating forecasting evaluation
Long Ngo, Mohammed Amine Chamli, Jonathan Rivalan et al. (+1 more)
Traditional evaluation metrics provides numerical values but often lack comprehensibility, hindering effective differentiation of model performances. Our work addresses this challenge by introducing overlay\_dx, a novel evaluation metric measuring the performance of time series prediction models. Overlay\_dx is a visual metric that represents the percentage of predictions falling within a confidence interval around actual values. Additionally, once evaluation results are plotted, overlay\_dx computes the area under the overlay curve, providing a quantitative measure of
alignment between predicted and actual values across different thresholds and predictions. Through extensive experiments, we demonstrate that our approach offers a unified evaluation framework that combines both visual and numerical assessments, enabling improved model comparison and providing valuable insights for further research and optimization efforts in time series prediction.
Read the full abstract ▾
All authors Long Ngo, Mohammed Amine Chamli, Jonathan Rivalan, Thomas Jaillon
Machine Learning
Universal Multi-Modal Traceformer: Integrating Heterogeneous Context for Process Event Prediction
Fabian Spaeh, Jingxing Fang, Shandian Zhe et al. (+1 more)
Event logs arise in a wide range of real-world processes, capturing not only event activities and timestamps but also multi-modal contextual information. Existing event-sequence models, including many temporal point process approaches, primarily model event activities and timestamps while overlooking heterogeneous context, such as numerical measurements, categorical attributes, textual descriptions, and metadata associated with individual events and entire traces. In this paper, we propose Universal Multi-Modal Traceformer (UMT), a unified framework for incorporating heterogeneous process context into next-event prediction. Built on a
Transformer backbone, UMT introduces a universal feature encoder that maps diverse feature types into a shared representation space and handles contextual information at both the event and trace levels. UMT further develops a per-event Perceiver module that dynamically weights contextual features and adaptively integrates them into event-
token representations. To accommodate the heavy-tailed and potentially multi-modal distribution of inter-arrival times, UMT represents each interval at multiple temporal scales and jointly predicts the corresponding scale-specific quantities. Experiments on 13 real-world event logs show that UMT improves both next-event activity and time prediction over existing approaches.
Read the full abstract ▾
All authors Fabian Spaeh, Jingxing Fang, Shandian Zhe, Bin Shen
Machine Learning · Cross-listed from cs.DS
Poisson Exchange Beyond Submodularity: Effective Approximation Algorithms for Offline and Online Subset Selection over Matroids
Shi Fu, Youming Qiao, Dacheng Tao et al. (+2 more)
Over the past decade, a growing body of research has shown that $\gamma$-weak submodularity broadly arises in numerous subset selection tasks, including feature selection, neural network
pruning, and
video summarization. Despite its prevalence, maximizing a $\gamma$-weakly submodular function subject to a general matroid constraint remains challenging. To date, the only known approximation guarantee is the conservative $(1+1/\gamma)^{-2}$ factor established by \citet{chen2018weakly}. To improve upon this result, this paper proposes a novel
algorithm called \MGPE, which repeatedly performs maximum-gain local exchanges through careful control of a non-homogeneous Poisson clock, and proves that this \MGPE\ can attain an approximation ratio arbitrarily close to $\rho_\gamma=1-\left(\gamma/(2-\gamma)\right)^{ \frac{\gamma^2}{2(1-\gamma)} }$. In sharp contrast to the previous guarantee, our obtained factor $\rho_\gamma$ not only strictly improves upon $(1+1/\gamma)^{-2}$ for every $\gamma\in(0,1]$, but also can asymptotically approach the optimal $(1-1/e)$-approximation for submodular maximization as $\gamma\to1$. Furthermore, we surprisingly find that when the matroid constraint reduces to a cardinality or the objective satisfies the stronger notion of $\alpha$-weak DR-submodularity, \MGPE\ can automatically recover the tight approximation ratios of $1-e^{-\gamma}$ and $1-e^{-\alpha}$, respectively. Here, $\alpha\in(0,1]$ denotes the DR ratio.
Read the full abstract ▾
All authors Shi Fu, Youming Qiao, Dacheng Tao, Zongqi Wan, Qixin Zhang
Machine Learning
$t_0$: A Time-Series Foundation Model for Forecasting with Context
Lucas Meyer, Claudio Sole, Huikan Xiang et al. (+6 more)
We present $t_0$, a family of open-weights foundation models for
forecasting with multivariate context. We release its first two members: $\texttt{t0-alpha}$ and $\texttt{t0-beta}$, respectively 102M and 256M parameters. Both condition their forecasts on target history, past
covariates, and known-future
covariates, without task-specific retraining. Their
transformer layers alternate
attention along time and across variates. They produce probabilistic forecasts through
quantile predictions. Pretraining combines curated public data with synthetic generator families constructed to contain
covariate-to-target dependencies. On GIFT-Eval, $\texttt{t0-alpha}$ reaches an aggregate CRPS of 0.4941, and $\texttt{t0-beta}$ a CRPS of 0.4738 and a MASE of 0.6865, third on both and within 4.0% of the best zero-shot TSFM. On fev-bench they score 42.2 and 46.7 in skill, the latter third again and 2.0 points behind the leader. We analyze $\texttt{t0-alpha}$ in depth. Known-future
covariates raise its skill by 6.3 percentage points across 30 tasks. The report also examines its calibration, its rollout strategy on long horizons, and its robustness to missing data. On the Victoria electricity-demand
benchmark, $\texttt{t0-beta}$ is among the most accurate models with a context of nearly a year. In an independent Macrocosm evaluation of hourly ERCOT prices over 29 months, both cut the MAE of the lagged-price baseline by 38%.
Read the full abstract ▾
All authors Lucas Meyer, Claudio Sole, Huikan Xiang, Nicolas Li, Lucas Franceschino, Arnau Quera-Bofarull, Maarten P. Scholl, Joachim Fainberg, Geoffrey Négiar
Machine Learning · Cross-listed from stat.ML
Identifying Representational Biases in Datasets Using PCA: A Max-Disparity Partition Framework
Arjun KM and Shashi Jain
Principal Component Analysis (PCA) minimises aggregate reconstruction error, which can inadvertently represent majority subgroups with substantially higher fidelity than minority subgroups.
Fairness-aware extensions of PCA correct this disparity but require group labels as input. We address the logically prior question: given only a data
matrix, which binary partition of the data suffers the greatest representational disparity under a shared PCA projection? We formalise this as the max-disparity partition problem and propose a greedy local-search
algorithm, grounded in the Fiduccia-Mattheyses bipartitioning framework, that discovers the disparity-maximising partition without any predefined group labels. Two
benchmark algorithms, a fixed-projection sorting baseline and a simulated-annealing variant, confirm that the greedy solution is empirically near-optimal. Having identified the partition, we attribute the disparity to specific features via PCA loading scores and association rule mining, enabling a practitioner to assess whether the disadvantaged group corresponds to a human-meaningful minority. On the Predict Students' Dropout and Academic Success
dataset, representational disparity is driven predominantly by institutional and programmatic proxies for socioeconomic disadvantage, with gender emerging as a secondary but consistent contributor within the disadvantaged group. The discovered partition is then passed directly to Fair PCA, completing a detect-explain-mitigate pipeline.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Beyond Point Prediction: Artificial Representative Trees with Uncertainty
Lea L. Mairhöfer, Silke Szymczak, Björn-Hergen Laabs et al. (+1 more)
Random forests (RFs) predict well but are opaque, whereas single decision trees are interpretable but unstable. Artificial representative trees (ARTs) were developed as interpretable surrogate models for RFs, but their use as standalone prediction models with uncertainty quantification has not been systematically investigated. We combine ARTs with leaf-wise Mondrian conformal predictive systems (CPS), enabling a single tree to provide continuous predictions, prediction intervals, and probabilities of exceeding arbitrary thresholds. We compared ARTs with CPS against decision trees with CPS and separate regression and probability trees across five simulation scenarios, 21
benchmark datasets, and a cross-sectional NHANES example data set. Repeated cross-validation assessed predictive performance, interpretability, and stability. ARTs with CPS yield compact, structurally stable trees with substantially more reproducible split-variable selection than decision trees across
benchmark datasets and NHANES. Decision trees showed slightly better predictive performance and narrower prediction intervals, while coverage was broadly comparable. CPS-based trees generally achieved lower and less variable Brier scores than multi-model approaches. Combining ARTs with CPS therefore provides a single, interpretable, and stable model for continuous predictions and calibrated probabilities, balancing predictive performance with reproducibility and transparency in settings where stability and interpretability are essential.
Read the full abstract ▾
All authors Lea L. Mairhöfer, Silke Szymczak, Björn-Hergen Laabs, Tuwe Löfström-Cavallin
Machine Learning · Cross-listed from cs.AI
Not All Task Vectors Need Equal Rank: Energy-Proportional Allocation for Model Merging
Hyunjoong Cho and Jinhyeok Jang
Model merging aims to combine multiple fine-tuned models derived from a common pretrained model into a single multi-task model without additional joint training. Recent spectral merging methods improve over simple weight averaging by exploiting low-rank structures of task-specific updates, but they commonly assign the same rank capacity to every task. This uniform allocation ignores that task vectors can have heterogeneous spectral complexity, causing the shared merging space to be used suboptimally. In this paper, we propose Spectral Energy-proportional Rank Allocation (SERA), a simple task-adaptive strategy that allocates ranks according to the singular-value energy structure of each task vector. By assigning richer spectral capacity to complex or isolated tasks and fewer directions to compact tasks, SERA extends SVD-based model merging from uniform-capacity merging to task-dependent capacity allocation. Experiments under standard
vision model merging protocols show that SERA improves multi-task merging performance while preserving the same total rank budget as existing spectral merging methods. Further analysis demonstrates that task-level spectral concentration is closely related to the per-task effect of adaptive rank allocation, providing insight into when and why SERA is effective.
Read the full abstract ▾
Machine Learning
On Emergent Capabilities and Model Merging
Luca Zhou and Emanuele Rodolà
Fine-tuned checkpoints and adapters now fill public repositories, and the most common operation applied to these artifacts is model merging: arithmetic on their weights that assembles capabilities cheaply. We ask what this operation does to emergent capabilities: behaviors an artifact carries that were never an explicit training target. Studying two independent testbeds (activation oracles and emergent-misaligned models) across three model families, we find that the answer is threefold. First, merging preserves an emergent capability that both parents carry: merging two misaligned checkpoints retains most of their broad misalignment across the whole mixing range. Second, merging cannot create an emergent capability that is superadditive in its parents: no weighted merge of two single-task oracles reaches the jointly-trained oracle's
auditing ability. Third, when only one parent carries the capability, merging dilutes it faster than the trained capability that accompanies it: the gap is significant in most settings. In short, emergent behaviors of an artifact do not compose the way its trained capability does.
Read the full abstract ▾
Machine Learning
Lifted Bellman Linear Programming for Offline Reinforcement Learning
Hyukjun Yang, Jongchan Park, Narim Jeong et al. (+1 more)
Offline reinforcement learning (RL) typically trains a critic by minimizing a regression loss against bootstrapped value targets stabilized by target networks with exponential moving average (EMA) updates. Multi-step targets incorporate behavior-policy actions and therefore require off-policy correction. We instead impose in-sample Bellman optimality on the critic through inequality constraints. We formulate the Lifted Bellman Linear Program (LBLP), which lifts the linear programming characterization of Bellman optimality to the joint $(Q,V)$ space so that every constraint involves only state-action pairs in the
dataset. Its unique minimizer is the in-sample optimal pair, and constraints along $K$-step segments of
dataset trajectories leave this minimizer unchanged for any rollout policy and horizon. Under deterministic dynamics, this minimizer lies between the best
dataset return and the optimal value. Relaxing the constraints into hinge penalties recovers the same solution above a finite penalty coefficient in the tabular case. Approximate Lifted Bellman Unconstrained Minimization (ALBUM) implements this relaxation with neural networks and detaches the $K$-step rollout targets by stop gradient. Its objective contains no squared regression onto bootstrapped targets, so it can be trained without target networks or EMA updates. Under deterministic dynamics, the LBLP solution is a stationary point of the detached update under a coefficient condition independent of $\gamma$ and $K$, and the inequality constraints allow discounted returns along
dataset trajectories to serve as lower bounds without off-policy correction or action chunking. On OGBench, ALBUM uses a single critic with a Gaussian policy, matches the average performance of FQL, and is comparable to recent action-chunking methods, while using the fewest parameters and the least peak GPU
memory among all compared methods.
Read the full abstract ▾
All authors Hyukjun Yang, Jongchan Park, Narim Jeong, Donghwan Lee
Machine Learning · Cross-listed from cs.AI
Fathom-Vaidya: Advancing Medical Reasoning with Rubric-Based Rewards
Kalash Shah, Kunal Singh, Snehan J et al. (+1 more)
Deploying Large
Language Models (LLMs) in healthcare requires robust performance across two complementary dimensions - diagnostic
reasoning: the convergent, evidence-driven task of inferring a patient's condition from clinical data to produce a diagnosis, and clinical healthcare
reasoning: the broader, navigational judgment required to communicate, plan, and adapt across multi-turn clinical interactions where a single correct answer may not exist. Recent
benchmarks such as HealthBench and MedXpertQA reveal persistent weaknesses in both areas, exposing failures in complex diagnostic scenarios and limitations in contextual, patient-centered dialogue. We introduce a sequential training framework that targets these facets using synthetic data and rubric-based reinforcement learning. First, we improve diagnostic
reasoning using MedBullets-derived questions with rule- and rubric-guided Reinforcement Learning (RL). We then shift to clinical
reasoning by generating 5.3k synthetic multi-turn scenarios, each paired with multi-dimensional rubrics to comprehensively assess the response. This approach yields over 10% improvement on MedXpertQA, and our 30B model achieves 50.1% accuracy on HealthBench-Hard, surpassing proprietary baselines including GPT-5 (thinking). Our results show that targeted synthetic
datasets and rubric-based training can systematically improve both diagnostic and interactive clinical
reasoning in medical LLMs.
Read the full abstract ▾
All authors Kalash Shah, Kunal Singh, Snehan J, Shreyas Singh
Machine Learning
A Temporal Knowledge Graph for Music Festival Lineup Forecasting
Julia Gastinger, Thilo Dieing, Christian Meilicke et al. (+1 more)
Music festival lineups emerge from complex relationships among artists, genres, releases, labels, and past performances, making the prediction of future lineups a natural fit for temporal knowledge graph (TKG)
forecasting. In this work, we present a TKG covering 380 festivals over 55 years, comprising more than 90K festival performance quadruples along with information on festivals, artist tours, and artist metadata, and release it as a resource for TKG
forecasting evaluation. We formalize festival lineup
forecasting as temporal link prediction between artists and festivals at future timestamps. We evaluate six TKG
forecasting models on this task, analyze their capabilities and limitations, and compare them against Large
Language Models applied zero-shot. Our resource complements existing TKG
benchmarks by grounding evaluation in a concrete, real-world application domain.
Read the full abstract ▾
All authors Julia Gastinger, Thilo Dieing, Christian Meilicke, Heiner Stuckenschmidt
Machine Learning
RAILS: Retrieval-Augmented Incremental LLM Clustering at Scale
Armin Oliya, Aleksandra Sawczuk and Radosław Białobrzeski
Using a Large
Language Model (LLM) as the clusterer at production scale is hard:
prompts cannot hold the entire label space, and per-document serial processing does not deliver the throughput real workloads require. We present RAILS, a
retrieval-augmented incremental LLM clusterer that turns clustering into a simple loop over a growing label pool and scales through document batching with bounded concurrency. On six public
benchmarks RAILS exceeds the strongest prior LLM-clustering method on average, lifting accuracy from 51.2% to 59.3%, NMI from 67.2% to 74.8%, and ARI from 45.4% to 54.7%. We further report production-deployment evidence from a SaaS ticket-topic-discovery pipeline, where RAILS has replaced a traditional HDBSCAN stage with higher clustering quality, transparent
prompt-driven control, and stateful incremental operation.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
MECAIL: Communication-Aware Incremental Learning for Object Detection with 14.6 KB Spatiotemporal Experts
Matthias Neuwirth-Trapp, Maarten Bieshaar, Danda Paudel et al. (+3 more)
Intelligent transportation systems require Incremental Learning (IL) to continually improve their overall performance in dynamic environments. However, most edge devices lack the computational resources to support on-device IL, requiring updates to be transmitted from centralized servers. We propose using this setup to obtain dense, specialized module coverage that adapts a fixed base model to specific spatiotemporal contexts, such as parking lots, gas stations, ferries, or construction sites. However, in order to reliably transmit these modules to the edge device, using TCP, UDP, and BTP over V2X, Wi-Fi, and 2G-5G hardware, we establish a strict limit of 14.6 KB per module to fit within the first TCP window and to minimize UDP/BTP fragmentation. We further introduce Mixture-of-Experts for Communication-Aware Incremental Learning (MECAIL), the first method that meets this strict requirement, in which each new domain or environment is served by a small expert network that adapts the base model. We validate MECAIL on D-RICO and ODinW-13, where it largely matches the performance of parameter-heavy approaches while enabling practical, bandwidth-efficient large-scale deployment. This allows comprehensive coverage by experts for highly specific, focused, and temporary situations.
Read the full abstract ▾
All authors Matthias Neuwirth-Trapp, Maarten Bieshaar, Danda Paudel, Konrad Schindler, Luc Van Gool, Christos Sakaridis
Machine Learning
WPBench: A Comprehensive Benchmark for Wind Power Forecasting
Yuhan Zhu, Jilin Hu, Xinying Cai et al. (+7 more)
Accurate, reliable, and deployable wind power
forecasting is critical for power system dispatch, renewable energy integration, and electricity
market operations. Progress in this field hinges on the ability to empirically and comprehensively
benchmark forecasting methods. Yet existing
benchmarks fall short of supporting systematic evaluation in four key aspects: 1) limited coverage of wind power scenarios across turbine scale, variable composition, and spatial structure; 2) incomplete coverage of
forecasting model families; 3) evaluation metrics misaligned with wind power requirements; and 4) limited structure-aware diagnostics beyond individual temporal patterns. To address these limitations, we propose WPBench, a comprehensive, fair, and extensible
benchmark for wind power
forecasting. WPBench integrates 26 public
datasets organized by turbine scale and variable composition, spanning single-turbine, multi-turbine, univariate, and multivariate settings. Under unified processing, training, and evaluation protocols, it
benchmarks 19 representative models covering traditional methods, deep temporal models, spatio-temporal models, and foundation models. Beyond point-wise errors, WPBench assesses forecast-curve fidelity and computational efficiency, and delivers structure-aware diagnostics across temporal, variable-dependency, and spatial-dependency perspectives. Together, these capabilities enable systematic model comparison across diverse wind scenarios and provide a reusable platform for future research.
Read the full abstract ▾
All authors Yuhan Zhu, Jilin Hu, Xinying Cai, Yingshan Li, Li Ma, Xiangfei Qiu Linsen Li, Kai Zhang, Yao Fu, Weihao Jiang, Bin Yang
Machine Learning · Cross-listed from physics.plasm-ph
Horizon-Aware Early Event Prediction for Tokamak Disruption Alarms
Takeshi Koshizuka and Takaharu Yaguchi
Disruption predictors must warn the
plasma control system early enough for the intended response, and the required warning time differs between mitigation and avoidance. We view a warning system as a predictor that outputs a disruption risk and an alarm policy that acts on it, and ask which risk the predictor should learn. Binary classifiers do not specify when a disruption occurs, and survival analysis models the full distribution of the time to disruption, whereas an alarm depends only on the risk within a finite horizon. We introduce two horizon-focused targets into tokamak disruption prediction: a horizon-truncated survival objective and fixed-horizon early event prediction (EEP) with temporal label smoothing (TLS). We compare them with binary classification and survival analysis on the DIII-D, Alcator C-Mod, and EAST databases of DisruptionBench, evaluating the risk by its threshold-free ranking at a 40 ms deadline and the alarm policy by thresholds calibrated on validation data. On every device, TLS outperforms the deep survival models in both ranking and alarm performance; TLS performs best on DIII-D and EAST, and a random forest trained on a hard horizon target performs best on C-Mod. TLS also improves the calibrated alarm performance over hard targets placed at either end of its smoothing range on all devices, isolating the benefit of label smoothing. Horizon truncation alone did not consistently improve survival analysis. Alarm performance remains limited on C-Mod and DIII-D and varies strongly across training seeds, and ablations show that the target should extend to the required warning time.
Read the full abstract ▾
Machine Learning
MUSE: Dependency-Aware Adaptation of a Frozen Vision Backbone for Multivariate Time Series Forecasting
Xinying Cai, Junkai Lu, Yuhan Zhu et al. (+3 more)
Multivariate time-series
forecasting is essential to many real-world applications. Recent large
vision models (LVMs) offer a promising paradigm by transferring cross-domain visual priors to time-series
forecasting. However, existing LVM-based methods face two key challenges: balancing independent visual representation spaces with cross-variable dependency modeling, and adapting
vision backbones pretrained on natural images to the distinct temporal semantics of time-series images. To address these challenges, we propose MUSE, a dependency-aware adaptation framework built on a fully frozen pretrained MAE. First, the Variable Context Refinement Module (VCR) aggregates shared temporal information within each variable and models cross-variable contextual dependencies while preserving independent visual spaces. Second, the Temporal-Periodic Refinement Module (TPR) performs lightweight refinement at different encoder depths and explicitly models across-period temporal dependencies and within-period periodic dependencies. The two modules independently produce forecasts, which are fused through a learnable prediction-level gate. Experiments on 10 real-world
datasets demonstrate that MUSE achieves state-of-the-art performance.
Read the full abstract ▾
All authors Xinying Cai, Junkai Lu, Yuhan Zhu, Xiaoyun Yu, Xiangfei Qiu, Jilin Hu
Machine Learning
Comparing Latent Concept Formation in State Space Models and Transformers via Sparse Autoencoders
Rithin Nagaraj, Rupa Laalasa Oruganti, Prerna Subhashchandra Kunder et al. (+1 more)
The quadratic scaling of
Transformer self-
attention has driven the adoption of sub-quadratic Selective State Space Models (SSMs) like Mamba, which compress past context into a fixed-size recurrent hidden state. This strict informational bottleneck raises a foundational question for mechanistic interpretability: do SSMs and
Transformers learn fundamentally distinct latent representations? In this work, we employ Sparse Autoencoders (SAEs) to conduct a large-scale, feature-level correspondence analysis between Mamba-130m and Pythia-70m over a 10-million
token corpus. Contrary to hypotheses predicting widespread architectural divergence, we find no evidence of systematic representational divergence between architectures: across the observed Jaccard distribution, 99.98% of Mamba features cluster toward the upper
alignment boundary, providing preliminary feature-level support for the Universality Hypothesis. We further identify and qualitatively characterize this microscopic fraction (0.02%) of diverging features, finding patterns consistent with the hypothesis that the recurrent bottleneck selectively limits the parsing of rigid syntax rather than broad semantic ontology. We demonstrate that while Pythia's unconstrained
attention permits the monosemantic decomposition of distinct formatting edge-cases, Mamba is forced to compress unrelated syntactical anomalies into polysemantic "junk drawer"
neurons to preserve state capacity. Collectively, these results suggest that architectural routing mechanisms may have negligible impact on core semantic understanding, with representational divergence confined to extreme structural margins.
Read the full abstract ▾
All authors Rithin Nagaraj, Rupa Laalasa Oruganti, Prerna Subhashchandra Kunder, Ashwini M Joshi