Machine Learning
Improving Wind and Solar Power Prediction with Efficient Wrapper-based Feature Selection: An Empirical Study
Daniel Grillmeyer, Marius Hadry, Michael Stenger et al. (+3 more)
With rising global energy demand and growing awareness of
climate change and its impacts, the share of renewable energies in the global energy mix continues to grow. Unlike conventional power generation, the output of renewable energy sources cannot be controlled as consistently due to their dependence on environmental conditions. Therefore, reliable prediction of current and future energy production is essential. In this paper, we report findings from two structured literature reviews on real-world renewable energy prediction tasks: wind turbine power curve modeling and photovoltaic power prediction. For the former, we conducted a comprehensive literature review ourselves, while for the latter, we synthesize the key findings regarding frequently selected input features based on an existing survey. Across both domains, our analysis reveals that despite the large number of available monitoring and environmental variables, only limited or unsystematic methods for feature selection exist. To address this gap, we propose Cluster-based Sequential Feature Selection (CSFS), a novel, model-agnostic, clustering-based wrapper method for automatic, efficient, and reliable feature selection in renewable energy prediction pipelines. To support reproducibility and reuse, we provide an open-source implementation of CSFS on GitHub. We empirically evaluate the proposed approach on both use cases and compare it with established feature selection techniques such as wrapper-based sequential feature selection (SFS), filter-based methods, and Random Forest's embedded feature importance. The results show that the wrapper-based methods overall provide better-performing selections of features. CSFS achieves a predictive performance comparable to SFS while reducing computational cost by an average of 21%.
Read the full abstract ▾
All authors Daniel Grillmeyer, Marius Hadry, Michael Stenger, Vanessa Borst, Veronika Lesch, Samuel Kounev
Machine Learning
Transforming Rank: How Architecture Navigates the Spectral Pathologies of Depth
Katie Everett
We investigate how each component of the
Transformer feedforward block architecture design determines how much rank survives across depth at initialization. We reinterpret skip connections and normalization, long understood as controlling magnitude, as mechanisms for preserving gradient rank across depth, since the very
matrix multiplications and nonlinear activations that make the network expressive also reduce the rank. We show that skip connections trade off rank collapse against ensemble-like behavior, controlled by the relative scales of the branch and the skip: skip connections route the gradient around the residual branch, where rank is lost, rather than along the long gradient paths that encourage the layers to compose. The placement of the normalization layer controls this same tradeoff by setting the branch-to-skip ratio across depth, unifying much of the normalization placement and depth scaling literature, in particular why rank collapses for Post-Norm but plateaus for Pre-Norm. Other aspects of the architecture, like the two-
matrix structure that expands and contracts the width, use additional parameters to preserve the representation or branch Jacobian rank. The second
matrix decorrelates a coherent mean spike that would grow across blocks with a single
matrix and uncentered activation, preventing the residual representation from collapsing. The width expansion between the two
matrices keeps the branch Jacobian full rank: applying the rank-reducing activation in this expanded space leaves enough directions to span the original, at a width that follows a Marchenko–Pastur law. The initialization rank of the input–output Jacobian predicts which networks train on CIFAR-10. Taken together, we recast architecture design for deep networks as navigating an intrinsic tradeoff among rank collapse, ensemble-like behavior, and parameter count.
Read the full abstract ▾
Machine Learning
Lighthouse RL: Sample-Efficient Circuit Optimization via Strategic Reset Points
Mustafa Emre Gürsoy, Stefan Uhlich, Ryoga Matsuo et al. (+6 more)
In this paper, we introduce Lighthouse RL, a sample-efficient reinforcement learning (RL) approach for analog
circuit sizing. Traditional methods lack generalization across different performance targets, while standard RL approaches waste resources exploring unpromising regions. Our method addresses these inefficiencies through a strategic reset strategy that initializes episodes from high-performing configurations discovered during training, called "lighthouses". These states, which are closer to the target objectives, guide exploration toward promising regions. When compared to RL and
Bayesian optimization methods from the literature, we demonstrate the effectiveness of our approach on a 2D
benchmark problem and on two analog
circuits, showing significant improvements in sample efficiency (up to 1.72x faster), optimization performance (100% vs. 0-87% success rate), generalization (75% vs. 0-50% extrapolation success), and objective maximization. This efficiency is particularly valuable for computationally expensive black-box optimization problems, and our reset strategy can be used as a plug-and-play enhancement for any RL-based optimization approach.
Read the full abstract ▾
All authors Mustafa Emre Gürsoy, Stefan Uhlich, Ryoga Matsuo, Yağ{\i}z Gençer, Arun Venkitaraman, Chia-Yu Hsieh, Andrea Bonetti, Eisaku Ohbuchi, Lorenzo Servadei
Machine Learning · Cross-listed from cs.AI
Do Agent Optimizers Compound? A Continual-Learning Evaluation on Terminal-Bench 2.0
Wenxiao Wang, Priyatham Kattakinda and Soheil Feizi
Most reported gains from
agent-optimization methods are one-shot: an
agent is optimized against a fixed
benchmark and the resulting improvement is reported as if it were a stable property of the method. This does not test the setting that matters for deployed
agents, where optimization is applied recursively as new failures and new tasks appear over time. The central question this raises is whether optimizer-driven gains compound: after an
agent has been optimized once, can it be optimized again on newly arrived tasks without eroding the gains the first round produced? We study this question with a two-phase continual-learning evaluation built from hard tasks in Terminal-Bench 2.0, comparing three approaches to
agent-harness optimization (GEPA, Meta Harness, and RELAI's Verifiable Continual Learning, RELAI-VCL) under identical optimization budgets. All three methods improve over the baseline
agent in the conventional, static, single-phase setting. However, once new tasks are introduced, the methods diverge sharply: GEPA's optimized
agent transfers below the unoptimized baseline, Meta Harness transfers well but fails to improve further once given a second optimization budget, and RELAI-VCL is the only method that both transfers positively to unseen tasks and continues improving after those tasks are folded into the optimization objective, reaching the highest pass rate at every evaluated stage and the highest lifelong average pass rate overall (76.4% vs. 66.0% for GEPA, 64.6% for Meta Harness, and 58.7% for the baseline). Our key observation was that optimization gains compounded only when regression control was built into the optimization loop, providing an inductive bias against shortcut solutions that fail to generalize.
Read the full abstract ▾
Machine Learning
Lyapunov Exponent as Physics-Informed Dense Reward: RL Discovery of Stabilization Beyond the Kapitza Pendulum
Slava Andrejev
We suggest using the Lyapunov characteristic exponent (LCE) as a dense reward signal for the reinforcement learning problem of stabilizing the inverted pendulum with vertical motion. With LCE, the
agent not only successfully found the oscillatory motion known as the Kapitza pendulum but also damped the pendulum's pivoting, leaving it in a strictly upright position.
Read the full abstract ▾
Machine Learning
TRACE: Turn-level Reward Assignment via Credit Estimation for Long-Horizon Agents
Leitian Tao, Baolin Peng, Wenlin Yao et al. (+5 more)
Multi-turn
agents solve complex tasks through extended sequences of tool interactions before producing a final answer, making credit assignment a fundamental challenge during post-training. Outcome rewards provide reliable supervision for short-horizon
reasoning, but become sparse and high-variance as trajectories grow to tens or hundreds of tool calls. They can also be misleading: a failed rollout may contain many useful actions that move the
agent closer to the goal, yet outcome-only training assigns them the same negative advantage as the eventual mistake. We propose TRACE (Turn-level Reward Assignment via Credit Estimation), a dense credit-assignment method for agentic reinforcement learning. TRACE represents rollouts as state transitions at tool-call boundaries, obtains gold-answer log-probabilities from a frozen reference model, transforms them into log-ratio state values, and derives per-action rewards as Temporal-Difference changes in those values. This requires no additional critic or process-label training, and its one-step log-ratio TD component
telescopes across redundant tool calls. On long-horizon complex search, TRACE substantially improves base-model tool-use ability using pure RL, without a cold-start supervised fine-tuning stage, an agentic mid-training stage, or training on live-web data. On the closed-web BrowseComp-Plus
benchmark, it raises Qwen3-4B from $7.2$ to $35.6$ and Qwen3-30B-A3B from $8.4$ to $42.6$. The learned search behavior also transfers to open-web
benchmarks, and the learning curves show earlier improvement and faster convergence during RL training.
Read the full abstract ▾
All authors Leitian Tao, Baolin Peng, Wenlin Yao, Tao Ge, Hao Cheng, Mike Hang Wang, Jianfeng Gao, Sharon Li
Machine Learning · Cross-listed from stat.ML
Multimodal Empirical Bayes Variational Autoencoders for Joint Longitudinal and Time-to-Event Modeling
Anders Sjöberg, Nils Olsson, Marcus Baaz et al. (+1 more)
Longitudinal tumor measurements, dropout information, and genetic
covariates provide complementary information about treatment response, but integrating these data sources within a single population modeling framework remains challenging. We extend the empirical Bayes variational autoencoder (EB-VAE) framework to joint longitudinal and time-to-event modeling and evaluate it on tumor growth data. The framework represents inter-individual variability using latent individual effects regularized by a
covariate-conditioned empirical Bayes prior, while a decoder maps these latent effects to tumor-volume trajectories. To account for informative dropout, the decoder was augmented with a hazard model, yielding joint predictions of tumor growth and time to dropout. We further compared fully neural and hybrid semi-mechanistic decoder formulations and incorporated genomic
covariates through a genetics-conditioned prior adaptation. The hybrid decoder recovered treatment-effect parameters broadly consistent with previously reported nonlinear mixed-effects estimates, while achieving prior predictive performance comparable to the neural decoder. The joint model reproduced both tumor-volume distributions and dropout patterns in held-out individuals, and genetic conditioning improved individual-level prior predictions in both cutaneous melanoma and breast cancer experiments. Stability selection identified several biologically plausible genetic indicators, including alterations in BRAF, NRAS, NF1, and MDM2. These results demonstrate that EB-VAE provides a flexible probabilistic framework for combining neural dynamics, mechanistic structure, time-to-event modeling, and high-dimensional
covariates in pharmacometric applications.
Read the full abstract ▾
All authors Anders Sjöberg, Nils Olsson, Marcus Baaz, Mats Jirstrand
Machine Learning · Cross-listed from cs.DS
Beyond the $d^{2.5}$-mixing bound for Dikin walks on polytopes
Yunbum Kook
Inspired by interior-point methods (IPM) for structured convex optimization, Kannan and Narayanan introduced the Dikin walk for sampling uniformly from polytopes in 2009. As in IPMs, the Dikin walk is affine-invariant, and its convergence is governed by the barrier geometry used to define its local proposal. They showed that the Dikin walk with the logarithmic barrier for a polytope in $\mathbb{R}^{d}$ with $m$ linear inequalities mixes in $md$ iterations. In 2017, Chen, Dwivedi, Wainwright, and Yu improved this to $d^{2.5}$ using a Lewis-weight barrier, and conjectured that the correct mixing time should be $d^{2}$. We make progress toward this
conjecture by improving the previous $d^{2.5}$-mixing bound. For exponential sampling over a polytope, we prove that the Dikin walk with a scaled Lee–Sidford metric mixes from a warm start in $d^{2.25}$ iterations. This also yields an improved cold-start complexity via a known annealing framework. The main technical ingredient is improved average self-concordance of the Lee–Sidford metric, which gives high acceptance probability for the Metropolis filter along a random Dikin proposal. While previous analyses were effectively limited to second-order control due to technical difficulties, we develop a principled higher-order analysis. The proof combines a selective higher-order expansion of recursive bottleneck terms, a moving orthonormal-frame calculus for higher derivatives of the Lewis weights, and Wiener-chaos decompositions via multiple
stochastic integrals to control the resulting Gaussian
polynomials.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
A novel unsupervised machine learning strategy to handle multimodal cardiac PET/MRI data
Brunnhilde Ponsi, Thomas Carlier, Lara Marteau et al. (+5 more)
Arrhythmogenic left ventricular cardiomyopathy is a genetic myocardial disease difficult to diagnose due to the lack of gold standard criteria. Simultaneous PET/MR imaging, combined with multiparametric quantitative analysis, could facilitate the identification of different profiles related to the phenotype and progression of cardiomyopathy. This preliminary study focuses on a methodological strategy for dealing with PET/MRI data, including inter-patient data linkage and regional analysis. Two-step clustering was applied to T1 and T2 maps, LGE, and 18F-FDG-PET images of 99 patients genetically diagnosed with arrhythmogenic left ventricular cardiomyopathy. Each patient's images were independently z-scored and summed into a single volume, which was clustered into supervoxels. Thirty-two inter-patient groups of supervoxels were obtained by spectral clustering. An "abnormality" score was assigned to each cluster and modality, and used to visualise abnormal regions likely associated with disease. They enabled the generation of automated textual and bullseye health reports for each patient, which were compared with cardiac imager assessments using balanced accuracy in repeated nested cross-validation. This approach was further validated on a larger cohort of 167 numerical phantoms. The reports generated by clustering accurately identified most of the cardiac physicians' observations (BA = 0.76 $\pm$ 0.04 in repeated nested cross-validation on patients, and BA $\ge$ 0.8 on phantoms). Furthermore, the identified abnormal clusters closely matched their visual observations, facilitating the identification of varying degrees of fibrosis or inflammation on the images. This approach enables a more systematic handling of
multimodal PET/MRI data to characterise myocardial heterogeneity in arrhythmogenic left ventricular cardiomyopathy patients.
Read the full abstract ▾
All authors Brunnhilde Ponsi, Thomas Carlier, Lara Marteau, Aurélien Monnet, Thomas Eugène, Jean-Michel Serfaty, Nicolas Piriou, Hatem Necib
Machine Learning
VAIOM: Continuous-Input, Discrete-Output Decoder-Only Financial Sequence Modeling
Yiming Ma and Xinyu Chen
Financial observations are continuous, heterogeneous, and noisy, whereas decoder-only next-
token models are usually built around discrete symbolic inputs. We introduce Vector-Input Autoregressive Inference for Ordinal-Return Modeling (VAIOM), a decoder-only
Transformer for probabilistic next-return modeling on one-hour foreign-exchange bars. VAIOM separates input representation from output likelihood: continuous multivariate financial-event vectors preserve numerical structure at the input, while a categorical distribution over the next
volatility-normalized return bucket supports cross-
entropy training and likelihood evaluation. The selected 0.9M Hybrid Continuous Input model combines continuous event features with categorical asset metadata, a Mixture-of-
Market-States return head, Gap,
volatility-regime, and Ordinal auxiliary objectives, and full-sequence supervision. Models and preprocessing are fit using pre-2024 Train data; models are selected on 2024H2 Validation and evaluated without refitting on two 2025 Test periods. Across three independent training seeds, every model outperforms fixed single-bar LightGBM baseline in both Test halves. For the canonical checkpoint, paired gains over LightGBM are 0.029 and 0.043 bits per event. Validation experiments show that continuous input improves over discrete-
token input under the same categorical return objective, full-sequence supervision improves over last-position training, and auxiliary representation shaping together with a mixture-structured return head improves return likelihood in controlled comparisons. A supporting capacity study finds that the smallest evaluated complete architecture rung achieves the strongest Validation likelihood on the present corpus.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CR
Plausible Deniability Guarantees for Whistleblowers
Leo Richter and Matt J. Kusner
Whistleblowers are a key safeguard against organizational wrongdoing, but the threat of retaliation deters reporting. Existing whistleblower-protection proposals lack formal
privacy guarantees, and existing differential
privacy mechanisms do not directly target the natural threat model – one in which the audited organization itself observes auditor selection decisions and uses them to identify reporters. We formalize protection against a strong-adversary threat model as per-report $(0, \delta)$-differential
privacy on the transcript of audit selections. Within this framework we prove that a natural approach – randomized response applied at the selection step – can never outperform uniform random
auditing by more than $\delta$ at any horizon. We then give a generic mechanism that reduces private
auditing to private continual counting: any $(0, \delta)$-DP continual counter plugs in by post-processing, and the audit transcript inherits the same per-report guarantee. Instantiating the reduction with a recent work in continual counting yields per-report $(0, \delta)$-DP with noise scaling as $O(\sqrt{\log T})$ across a horizon of $T$ audit decisions. A utility
theorem shows that the selection error vanishes whenever the noisy report gap between the most-reported organization and the runner-up grows faster than $\sqrt{\log T}$. Simulations show a substantial improvement over randomized response.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.PL
Generative Compilation: On-the-Fly Compiler Feedback as AI Generates Code
Niels Mündler-Sasahara, Hristo Venev, Dawn Song et al. (+2 more)
Languages with rich static semantics, such as Rust, provide stronger guarantees for AI-generated code, but their strictness makes generation more difficult. Off-the-shelf compilers can provide useful feedback post-generation, but does not guide intermediate generation steps, such as those during autoregressive LLM decoding. Constrained decoding intervenes earlier by rejecting invalid
tokens during sampling, but requires white-box model access and costly reimplementation for semantic constraints. We introduce generative compilation, the first approach to obtaining compiler feedback on partial programs during generation. The core technical device is a sealor: a lightweight, mostly syntax-guided transformation that converts partial programs into complete ones that standard compilers can diagnose. It is designed such that possible-to-complete partial programs are never rejected, while preserving enough code context to catch genuine dead ends early. We construct such a sealor on a core Rust-like calculus and prove that it satisfies these properties, all mechanized in Lean. We extend it to the first partial-program checker for real Rust. We evaluate our method on challenging repository-level Rust coding tasks, across both frontier black-box and open-weight models. We show that generative compilation reduces non-compiling outputs and improves functional correctness, relative to standard post-generation feedback. It does so by detecting a broad range of errors close to their source and early during generation, thereby reducing errors cascades and enabling focused diagnostics. More broadly, generative compilation is a step toward making compilers a first-class citizen of AI-assisted programming active during generation, rather than a separate post-generation check.
Read the full abstract ▾
All authors Niels Mündler-Sasahara, Hristo Venev, Dawn Song, Martin Vechev, Jingxuan He
Machine Learning
An Efficient Newton Algorithm for Nonnegative Matrix Factorization with the Kullback-Leibler Divergence
Damien Lesens, Jérémy E. Cohen and Bora Uçar
Nonnegative
Matrix Factorization (NMF) is a fundamental tool in unsupervised learning, which approximates a nonnegative
matrix by the product of two low-rank nonnegative factors. The Kullback-Leibler (KL) divergence is best suited to measure the data to model discrepancy when the decomposed data sample follows a Poisson distribution, which is the case for count
datasets such as term-document
matrices or images. Most KL-NMF
algorithms in the literature minimize a separable majorant of the loss to find their next iterate. We argue that this method has reached its limits and propose to use instead the second-order Taylor expansion of the loss, leading to a Newton-type method. We minimize this non-separable surrogate by proposing a generalization of the well-known HALS
algorithm. This yields an efficient KL-NMF
algorithm which provably converges and which competes favorably with state-of-the-art
algorithms on a large variety of
datasets.
Read the full abstract ▾
Machine Learning
Partially Correlated Verifier Cascades in LLM Harnesses: Concave Log-Odds, Polynomial Reliability, and Blind-Spot Ceilings
Jiangang Han
Serial verification gates are a core reliability primitive in LLM harnesses: a candidate answer is returned only if $k$ verifier calls all accept it. Under conditionally independent gates, the recent Odds Law (arXiv:2606.15712) shows that posterior log-odds grow linearly in $k$, so failure decays exponentially, and states that "a tight theory of partially correlated verifier cascades remains open." This note gives a minimal such theory. Modeling the per-instance false-accept rate on the generator's own errors as a latent variable $\alpha \sim G$ (de Finetti), the exact cascade posterior is $\ell_k = \ell_0 - \ln m_k$, with $m_k$ the $k$-th moment of $G$. Then: (i) $\ell_k$ is concave in $k$ for every non-degenerate $G$ – the Odds Law is its tangent at the first gate and an upper bound; (ii) for Beta$(a,b)$ latents, failure decays polynomially, $1-r_k \asymp k^{-b}$, with correlation parameter $\rho_v = 1/(a+b+1)$; (iii) a blind-spot atom of mass $1-\pi$ at $\alpha=1$ caps the evidence extractable from any number of gates at $-\ln(1-\pi)$ nats, so reliability saturates below 1; (iv) letting the true-accept rate also vary ($\beta \sim H$) yields a trichotomy – gates eventually always help, plateau, or actively harm – decided by the upper-tail exponents of $G$ and $H$, with closed-form crossover $k^\dagger$. The mechanism is survivorship: errors surviving gates are the high-$\alpha$ ones. The theory is measurable: $R$ repeated verdicts per instance identify the first $R$ moments of $G$, so two verdicts identify $\rho_v$; beta-binomial likelihood and NPMLE recover the reliability curve and the ill-posed ceiling. In synthetic tests, independence-based extrapolation underestimates failure by 20x at $k=5$ and ~3000x at $k=10$; the correlated fit at $R=8$ tracks held-out depths. The practical lever is decorrelation – changing model family, modality, or evidence source – not adding gates.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
The 2nd International StepUP Competition for Biometric Footstep Recognition: From Steps to Strides
Robyn Larracy, Anant Gupta, Gourav Gupta et al. (+8 more)
The International StepUP Competition Series was launched to advance research in pressure-based footstep biometrics through a standardized and challenging evaluation framework. Using the large-scale StepUP-P150
dataset (with more than 200,000 high-resolution dynamic footsteps from 150 individuals) and a previously unreleased test set, the 2nd edition of the competition addressed three key challenges: (1) generalization to unseen users with limited enrollment data, (2) robustness to domain shift caused by variations in footwear and walking speed and (3) effective fusion of paired left-right footsteps. While the first two challenges built on the inaugural competition, this edition introduced more extreme cross-domain conditions and moved beyond isolated footsteps to stride-level verification, enabling new opportunities for representation learning and inter-step information fusion. The competition attracted 26 registrants from academia and industry, with a best equal error rate of 8.00% achieved by the ArogyaPandit Research Team using a spatiotemporal CNN combined with an ensemble-based scoring strategy. The top solutions showcase the value of harnessing temporal patterns and of incorporating inference-time normalization and calibration strategies to improve scoring. However, the results also reveal that recognizing users in unseen personal footwear remains a challenge, especially in the presence of distractors with similar characteristics.
Read the full abstract ▾
All authors Robyn Larracy, Anant Gupta, Gourav Gupta, Ethan Eddy, Maxime Devanne, Cyril Meyer, Jin-Chern Chiou, Yueh-Shan Lee, Zong-Han Lu, Aaron Tabor, Erik Scheme
Machine Learning
RF Spectrogram Anomaly Detection with Quantum Kitchen Sinks: Architecture, Representation, and Hardware Validation
Abdallah Aaraba, Alexis Vieloszynski, Remon Polus et al. (+2 more)
The broadcast nature of wireless channels exposes
radio-frequency (RF) networks to anomalous and malicious transmissions, making anomaly detection a fundamental requirement for secure spectrum management.
Quantum Kitchen Sinks (QKS) offer a lightweight hybrid
quantum feature map suitable for near-term
quantum devices, yet their behavior on structured signal data remains poorly understood. In this paper, we extend the standard QKS template with multi-depth data re-uploading and ring
entanglement, and evaluate the resulting pipeline on controlled RF spectrogram anomaly detection. We introduce a validation-locked five-stage ablation protocol that systematically separates the effects of shallow architecture, re-uploading depth, episode budget, input representation, and classical readout. Across the completed
benchmark, Discrete Cosine Transform (DCT) representations consistently dominate raw and Principal Component Analysis (PCA) inputs, moderate-depth entangled QKS configurations form the strongest operating regime, and QKS improves over matched classical direct-readout baselines across all evaluated representation-readout pairs on the held-out test set, with the best configuration reaching a test Area Under the Receiver Operating Characteristic curve (AUROC) of 0.8778 and a test F1 of 0.7995. The study bridges two levels of realism: real measured sub-6\,GHz cellular signals on the data side and real-device validation on the ibm_quebec
Quantum Processing Unit (QPU) on the computing side, with AUROC deviations below 0.013 relative to simulation. These results provide a practical, reproducible framework for deploying QKS-based anomaly detection in wireless networks.
Read the full abstract ▾
All authors Abdallah Aaraba, Alexis Vieloszynski, Remon Polus, Soumaya Cherkaoui, Ola Ahmad
Machine Learning
PiVoT: A Variational Solution for Real-time Large-scale Multi-object Detection and Tracking under Heavy Clutter
Runze Gan, Qing Li, Simon J. Godsill et al. (+2 more)
Multi-object detection and tracking from noisy point clouds remain challenging in many data-scarce radar applications. Current
Bayesian trackers based on Poisson measurement models offer a training-free solution but struggle to achieve accuracy and efficiency under severe clutter, large object populations, and full-resolution Doppler point clouds. We address this with PiVoT, a fast, clutter-resilient multi-object tracker for both positional and Doppler measurements. PiVoT performs end-to-end detection and tracking of a large and time-varying number of objects without external clustering or detectors, through joint inference of object states, shapes, existence probabilities, data association, and measurement rates. Its efficiency is driven by several variational inference innovations, such as theoretically justified birth
pruning, quadratic-to-linear complexity reductions for exact updates, and a computationally efficient Doppler Poisson model. Experiments show that PiVoT substantially outperforms existing
Bayesian trackers in challenging scenes, while also demonstrating exceptional scalability to a thousand objects, robustness to clutter visually inseparable from objects, and real-time operation on full-scale modern automotive radar
datasets, where it attains performance comparable to a deep-learning detection
benchmark as a training-free joint detector and tracker.
Read the full abstract ▾
All authors Runze Gan, Qing Li, Simon J. Godsill, Mike E. Davies, James R. Hopgood
Machine Learning · Cross-listed from stat.ME
Verifying formulas for interventional distributions
Francesco Freni, Leonard Henckel and Sebastian Weichwald
We formalize verification in causal graphical models: deciding whether a given observational formula identifies a target interventional distribution. This opens a problem complementary to identification, asking not whether any identifying formula exists, but whether the given formula is identifying. We show that even sound and complete solutions to identification do not solve verification. We propose a falsifier as a first practical route forward, prove that it induces an almost-surely correct verifier for regular exponential-family models, and use the resulting verifier to develop the gateway test, which finds all sets admissible for use in a front-door formula.
Read the full abstract ▾
Machine Learning
Task-Oriented Sensing and Covert Transmissions for Collaborative Multi-AUV Systems
Xueyao Zhang, Bo Yang, Chenyang Yan et al. (+6 more)
In
underwater covert cooperative missions, autonomous
underwater vehicles (AUVs) often cannot rely on active sonar to continuously obtain complete information, since active sensing and frequent communications increase exposure risk. As a result, AUVs primarily rely on passive observation, which yields incomplete local perception and limited task efficiency. Although
underwater acoustic communications can mitigate this limitation through information sharing, they are simultaneously constrained by long delays, severe interference, low reliability, and the risk of covert exposure. Existing communications-oriented multi-
agent reinforcement learning (MARL) studies often model communication as an ideal information flow, whereas traditional communication optimization primarily focuses on link-level performance. However, both are insufficient to characterize the actual contribution of perceptual information to cooperative tasks under realistic covert physical communication conditions. This paper proposes a Sensed-information Value Realization Multi-
Agent Reinforcement Learning (SVR-MARL) framework that leverages practical information to characterize information utility for cooperative tasks and learns distributed cooperative policies under realistic communication and covert constraints. Through a case study of covert multi-AUV cooperative localization and tracking, the potential of the proposed framework to improve collaborative task efficiency while reducing unnecessary communication and exposure risks is demonstrated.
Read the full abstract ▾
All authors Xueyao Zhang, Bo Yang, Chenyang Yan, Xuelin Cao, Zhiwen Yu, Bin Guo, Samson Lasaulce, George C. Alexandropoulos, Merouane Debbah
Machine Learning
AI-Augmented Adaptive Digital Twin Modeling for Brain Tumor Evolution Prediction and Treatment Scheduling
Wenxi Liu, Michael Trimboli and Xianqi Li
Brain tumor progression exhibits spatially heterogeneous growth, patient-specific treatment response, and complex interactions with surrounding anatomy, making accurate long-term prediction challenging. We propose an AI-augmented adaptive digital twin (DT) framework for
brain tumor
evolution prediction and treatment scheduling. The framework integrates an interpretable reaction–
diffusion (RD) model, a 3D residual learning module for model-form correction, patient-specific DT updating during recursive rollout, and model predictive control (MPC) for constrained chemotherapy and radiotherapy scheduling. Experiments on 387 synthetic tumor trajectories with 120-step
evolution show that the baseline RD model captures tumor location and overall temporal behavior but underestimates heterogeneous tumor burden during long-horizon prediction. Hybrid RD–residual modeling reduces masked voxel-wise mean squared error by 84.3% and increases Dice overlap by 43.5% relative to the RD baseline under dense simulated observations. Online DT updating further reduces mean squared error by 45.9% and improves Dice overlap by 9.6% compared with the non-updated hybrid model. In MPC-based scheduling simulations, the updated DT controller reduces final tumor burden by 22.4% relative to a fixed treatment schedule under the terminal-burden objective. Together, these results demonstrate a unified framework for patient-specific initialization, mechanistic modeling, adaptive learning, and constrained treatment optimization. Although validated using patient-data-informed synthetic trajectories rather than clinical longitudinal data, the proposed framework establishes a foundation for future translation to real-world adaptive treatment planning.
Read the full abstract ▾
Machine Learning
Relevance-Aware Rule: Structural Deletion of Irrelevant Conditions in Decision Trees
Jung-Sik Hong, Jeongeon Lee, Min Kyu Sim et al. (+1 more)
Decision trees generate interpretable if–then rules, yet they contain irrelevant conditions (IRCs). These IRCs arise from the structural mechanism of tree splitting and persist even in modern optimal sparse tree induction
algorithms. Existing IRC deletion methods overlook this structural mechanism; therefore, they either preserve the original tree too loosely to remain reliable, or too strictly to achieve meaningful simplification. This study provides theoretical foundations for reliable IRC deletion by establishing
theorems and propositions related to the underlying IRC mechanism. The key finding is that a binary split shifts class proportions in opposite directions relative to the parent. Specifically, an increase in the class-1 proportion along one branch necessitates an increase in the class-0 proportion along its sibling, thereby generating a C1-link and a C0-link. Based on this structural fact, we propose a structural IRC deletion framework. Relative to each leaf, links that increase the leaf-class proportion are matched, whereas links that increase the proportion of the opposite leaf-class are mismatched. These mismatched links are flagged as structurally suspicious IRC candidates. Rather than deleting them outright, the framework rigorously diagnoses their relevance by assessing prediction reliability. It selectively deletes conditions that are structurally and empirically irrelevant, while strictly protecting those whose deletion would reduce the rule's reliability. Experimental results confirm that the proposed framework achieves substantial rule simplification without sacrificing the reliability of the original tree.
Read the full abstract ▾
All authors Jung-Sik Hong, Jeongeon Lee, Min Kyu Sim, Sangheum Hwang
Machine Learning · Cross-listed from quant-ph
Quantum Topological Data Encoding
Adam Wesołowski, Dimitrios Thanos, Daniel Leykam et al. (+1 more)
Many
datasets encountered across a wide range of domains possess rich
geometric and topological structure that is difficult to capture using conventional vector-based representations.
Quantum machine learning offers the possibility of processing high-dimensional data in
Hilbert spaces, but its practical success depends critically on how classical data is encoded into
quantum states. We introduce
quantum topological data encoding (QTDE), a general framework for encoding topological information into
quantum states via
topology-driven
quantum evolution. Our method generalises an existing
topology-driven
quantum encoding framework to higher-dimensional data. We test the proposed method on clique-complexes classification tasks, and provide preliminary evidence that
topology-driven
quantum representations can capture discriminative information beyond that available through direct comparisons of classical topological descriptors. The proposed
quantum representations consistently outperform a baseline based on direct comparisons of the combinatorial
Laplacians describing the underlying topological structure. We indicate several areas of application where the framework can be used to provide a more efficient and reliable data representation.
Read the full abstract ▾
All authors Adam Wesołowski, Dimitrios Thanos, Daniel Leykam, Lirandë Pira
Machine Learning
Heavy-Tailed Flow Matching via Random Clocks
Zhouhao Yang, Yezhen Wang, Kenji Kawaguchi et al. (+2 more)
Heavy-tailed data arise in many domains where rare events carry disproportionate importance, such as imbalanced image
datasets, financial returns, and weather extremes. Standard
diffusion and flow-matching models typically begin from Gaussian noise or Gaussian source distributions, which yield tractable training targets but provide a poor inductive match for heavy-tailed data. We propose Heavy-Tailed Flow Matching via Random Clocks (HTFM), a framework that portrays heavy-tailed sources as mixtures of clock-conditioned Gaussian sources. Conditioning on a given clock path, the source distribution and flow are Gaussian; marginalizing over the clock gives a Gaussian scale mixture covering Gaussian, $\alpha$-stable, and Student-t families. To make the clock-conditioned vector field practical, we encode the path-valued clock using truncated logsignature features, allowing the velocity field to adapt to the realized conditional space with negligible overhead. Empirically, on 2D imbalanced $\alpha$-stable mixtures, CIFAR10-LT, and HRRR weather fields, HTFM improves mode coverage, sample quality, and tail-statistic recovery over Gaussian flow matching and competitive heavy-tailed baselines, while retaining the low-NFE sampling advantage of flow matching. Moreover, the random-clock formulation further provides a practical tail-control interface: by varying only the clock law or tail parameter, the same architecture can calibrate the “heaviness” of generated tails across different distribution families.
Read the full abstract ▾
All authors Zhouhao Yang, Yezhen Wang, Kenji Kawaguchi, Vladimir Braverman, Haoyang Cao
Machine Learning
NodeImport: Imbalanced Node Classification with Node Importance Assessment
Nan Chen, Zemin Liu, Bryan Hooi et al. (+3 more)
In real-world applications, node classification on graphs often faces the challenge of class imbalance, where majority classes dominate training, resulting in biased model performance. Traditional GNNs often struggle in such scenarios, as they tend to overfit to majority classes while underrepresenting minority classes. Existing solutions, which either prioritize nodes based on class size or synthesize new nodes for minority classes, often fall short of effectively addressing this imbalance issue. This paper introduces an approach to class-imbalanced node classification by utilizing a balanced meta-set for importance measurement, where a training node is considered significant if it enhances model performance under an unbiased setting. Our method identifies important nodes that can counteract class imbalance and utilizes them for model training, allowing for fine-grained and dynamic node selection throughout the training process. We theoretically derive a formula to directly assess node importance, reducing computational overhead and providing an intuitive threshold for node selection. Guided by this metric, we develop a novel framework that filters valuable labeled, unlabeled, and synthetic nodes that enhance model performance in an unbiased context. A key advantage of this framework is its separation of the synthetic node generation process from the filtering process, ensuring compatibility with various node generation methods. Furthermore, we introduce a strategy to construct a high-quality meta-set that closely approximates the overall feature distribution, ensuring robust representation of each class. We evaluate our framework, NodeImport, across multiple
datasets using popular GNN architectures, demonstrating its superiority over existing baselines. Our results highlight the flexibility and effectiveness of the framework in mitigating class imbalance, leading to improved outcomes.
Read the full abstract ▾
All authors Nan Chen, Zemin Liu, Bryan Hooi, Bingsheng He, Jun Hu, Jia Chen
Machine Learning · Cross-listed from cs.CV
Multimodal Assessment of Pancreatic Cancer Resectability Using Deep Learning
Vincent Ochs, Christoph Kuemmerli, Florentin Bieder et al. (+12 more)
Accurate determination of pancreatic ductal adenocarcinoma (PDAC) resectability relies on evaluating how the tumor interacts with major peripancreatic vessels on CT imaging, yet expert assessment often shows substantial variability. We introduce a fully automated
multimodal deep learning framework that jointly analyzes 3D contrast enhanced CT and structured clinical information to classify patients into the three National Comprehensive Cancer Network (NCCN) resectability categories (upfront resectable, borderline resectable, locally advanced). The approach uses a Swin-UNETR backbone to obtain anatomy aware image representations through auxiliary
segmentation of pancreas, tumor, and vascular structures. These features are fused with a compact clinical
embedding derived from 17 routinely collected variables and processed by a lightweight classification head. Model training is guided by a dynamic multitask objective that adapts the balance between
segmentation and classification based on current tumor Dice performance, promoting feature representations that remain both anatomically informed and discriminative.
Read the full abstract ▾
All authors Vincent Ochs, Christoph Kuemmerli, Florentin Bieder, Julia Wolleb, Joel L. Lavanchy, Julia Ruppel, Jan Liechti, Stephanie Taha-Mehlitz, Christian Andreas Nebiker, Beat Mueller, Giuseppe Kito Fusai, Joerg-Matthias Pollok, Anas Taha, Philippe C. Cattin, Sebastian Staubli
Machine Learning · Cross-listed from cs.CR
Traffic-Aware Randomized Smoothing for LLM-Based Network Intrusion Detection
Zhenpeng Li
Large
language model (LLM)-based intrusion detection systems (IDS) are increasingly studied for security monitoring, yet their robustness against feasible traffic manipulation remains largely empirical. We present Traffic-Aware Randomized Smoothing (TA-RS), a classifier-agnostic certified defense that injects Gaussian noise exclusively into the directly controllable (DC) subspace – features a remote attacker can modify – during both fine-tuning and certification, aligning the smoothing distribution with the attacker-controllable subspace. We identify a critical prerequisite: applying standard randomized smoothing to clean-trained LLM-IDS yields weak certified accuracy in three of four (model,
dataset) pairs tested (14-33%, at or below random) and only 57% in the fourth (43 pp below the noise-augmented result); noise-augmented fine-tuning recovers to 68-100% on two of three
benchmark datasets (at sigma=0.25). At the L_inf-equivalent threshold R_inf = epsilon*sqrt(|DC|) (epsilon=0.05), TA-RS achieves 55-100% certified accuracy on CIC-IDS-2018 and HIKARI-2021, with median certified radii (R approx 0.45-0.96) exceeding R_inf by 1.8-5x (across sigma=0.25-1.00). Against a fairly trained iso-trained RS baseline the residual advantage is
dataset-dependent (4-19 pp on CIC-IDS-2018). The larger gap – up to 72 pp against an isotropic RS baseline that shares the DC-noise-augmented training recipe – primarily reflects the training-certification mismatch rather than DC
alignment alone: isotropic test-time noise perturbs uncontrollable features the attacker cannot exploit, triggering abstention rates up to 68%. RT-IoT2022 probes the limits of the method: it fails under the default fine-tuning recipe but recovers to 76%/69% certified accuracy (LLaMA3-8B/Qwen3-8B) when noise augmentation is increased.
Read the full abstract ▾
Machine Learning
Mono-Z Dark Matter Search with Neural Spline Flows Using CMS Run 2015D Open Data
Hitesh Rasineni and Bhavishya Chebrolu
We report a search for dark matter (DM) produced in association with a leptonically decaying \(Z\)
boson at \(\sqrt{s}=13\) TeV using CMS Run 2015D open data corresponding to an integrated luminosity of \(2.32\,\mathrm{fb}^{-1}\) together with simplified-model Monte Carlo simulation. Events are selected in the mono-\(Z\rightarrow\ell^+\ell^-\) final state in both the \(\mu\mu\) and \(ee\) channels. Forty kinematic observables are extracted from MINIAOD and MINIAODSIM, cleaned with physics-motivated selections, and reduced to a 37-dimensional feature vector. Five Neural Spline Flows are trained independently to model Standard Model background and mediator-specific DM signal densities. The per-event test statistic is constructed from the log-likelihood ratio between the signal and background density estimates, providing sensitivity across the full kinematic phase space without requiring a hard upper \(\mathrm{MET}\) threshold. A simultaneous profile-likelihood fit combining the two channels yields observed (expected) 95% confidence level upper limits on the signal-strength parameter of \(\mu<0.0177\) (\(0.0018\)) for the scalar mediator, \(\mu<0.0362\) (\(0.0039\)) for the vector mediator, and \(\mu<0.0498\) (\(0.0069\)) for the axial-vector mediator. The observed limits are weaker than expected because of a residual high-\(\mathrm{MET}\) background-modeling discrepancy rather than evidence for a DM signal. To our knowledge, this is the first application of Neural Spline Flow likelihood-ratio scoring to a mono-\(Z\) dark matter search using CMS Run 2015D open data simultaneously in the \(\mu\mu\) and \(ee\) channels.
Read the full abstract ▾
Machine Learning
MxGPS: Multiplex Graph Transformers for a Power Grid Foundation Model
Charilaos Papaioannou, Ioannis Tsantilas, Dimitris Giannakakos et al. (+9 more)
Single-task fine-tuning of graph neural networks (GNNs) for power grid problems exhibits a systematic failure mode: models that achieve the lowest in-distribution error degrade the most under
topology shift. We term this
topology overfitting: the tendency of task-specific gradient signals to encode relational structure particular to the training
topologies rather than the underlying physics, causing models to fail on unseen grids despite strong in-distribution performance. To expose and address this failure mode, we introduce MxGPS (Multiplex GPS), a multiplex graph
transformer that runs K task-specialised GPS branches over a shared node encoder, jointly trained on Static State Estimation (SSE) and AC Power Flow (PF) via a self-supervised pre-training and multi-task fine-tuning protocol, with a cross-branch
attention module evaluated in ablation. The joint SSE+PF objective forces the shared encoder to simultaneously satisfy complementary gradient signals, preventing it from overfitting to
topology-specific relational structure. Under a 3-fold sliding-window cross-validation spanning four unseen
topologies (14-, 24-, 162-, and 300-bus), MxGPS attains 0% boundary violation rate (BVR) on all four zero-shot Power Flow
topologies. Critically, models with substantially lower in-distribution PF error degrade by 190% to 1400% under
topology shift, whereas MxGPS degrades by only 39%, an inversion that directly implicates
topology overfitting as the failure mechanism rather than insufficient model capacity. With only 1.6M parameters (12x fewer than the GridFM reference baseline), MxGPS demonstrates that multi-task joint training is a principled and parameter-efficient mechanism for
topology-agnostic generalisation in power grid foundation models.
Read the full abstract ▾
All authors Charilaos Papaioannou, Ioannis Tsantilas, Dimitris Giannakakos, Vasilis Michalakopoulos, Sotiris Pelekis, Vangelis Marinakis, Arsam Aryandoust, Antonello Monti, Ricardo J. Bessa, Perdo P. Vergara, Jochen Cremer, Elissaios Sarmas
Machine Learning
Algebraic Representability as the Limiting Regime of Grokking: An Exactly Solvable Model with Holomorphic Activations
Chon-Fai Kam, Xavier Cadet, Miloud Bessafi et al. (+1 more)
Neural networks trained on modular arithmetic exhibit grokking, a delayed transition from memorisation to generalisation known to depend on model capacity: too little and the network memorises slowly or not at all, too much and it generalises almost immediately. What happens at the extreme of this spectrum, when the architecture's expressible function class collapses to a finite-dimensional algebraic variety? We study two-layer networks with a holomorphic monomial activation sigma(z)=z^k, trained on modular tasks encoded via roots of unity. Here the network output, regardless of hidden width, is confined to a (k+1)-dimensional subspace of characters of (Z_p)^2, an O(k/p^2) slice of the full function space. We give a complete algebraic characterisation of this subspace: a task is representable if and only if its discrete
Fourier support lies on the diagonal u+v = k (mod p), which for linear-phase targets reduces to the arithmetic criterion m+n=k. This is not merely a constraint on eventual generalisation but on memorisation itself: because the outputs are algebraically confined, a non-representable target cannot be fit even on the training set, and we prove a positive lower bound on the training loss, independent of width. Across 585 runs the algebraic prediction matches the observed outcome with 99.8% accuracy, with no memorisation regime and no grokking; outcomes split cleanly into instant success and outright failure. This binary behaviour is the limiting case of the capacity-grokking relationship: when the expressible class shrinks to a fixed algebraic object, the question of when a network will grok dissolves into whether it can represent the target at all. A bottleneck ablation connects this extreme to standard networks, tracing a continuous path from representational failure, through memorisation without generalisation, to grokking with a shrinking gap as capacity grows.
Read the full abstract ▾
All authors Chon-Fai Kam, Xavier Cadet, Miloud Bessafi, Frederic Cadet
Machine Learning
Implementations of Quantum and Classical Topology-Aligned Architectures for Molecular Property Prediction
James T. Pegg, Hubert Okadome Valencia and Ronin Wu
For low-data and resource-constrained regimes typical of
quantum chemistry, parameter-efficient learning is a key objective. Here, we propose a
topology-aligned inductive bias in which the model architecture mirrors the molecular bond graph: atoms map to a fixed register of computational units, and bonds determine which pairs interact through shared learnable parameters. This principle is instantiated in two architectures: a variational
quantum circuit (Iso-QGNN) and a parameter-matched classical message-passing network (Iso-CGNN). The models are benchmarked on HOMO-LUMO and dipole moment binary classification tasks over the QM9
benchmark. With 64 trainable parameters, the implementations achieve test AUCs of approximately 0.89 (
quantum) and 0.92 (classical) on the gap task, and close to 0.78 (both) on the dipole task. The models reach 90% of asymptotic performance within about 300 training molecules and gradient norms remain stable throughout training. These results indicate that the
topology-aligned inductive bias is the active ingredient driving parameter efficiency at QM9 scale, with implications for matched-baseline benchmarking in
quantum machine learning.
Read the full abstract ▾