Machine Learning
RL-BioAug: Label-Efficient Reinforcement Learning for Self-Supervised EEG Representation Learning
Cheol-Hui Lee, Hwa-Yeon Lee and Dong-Joo Kim
The quality of data augmentation serves as a critical determinant for the performance of contrastive learning in
EEG tasks. Although this paradigm is promising for utilizing unlabeled data, static or random augmentation strategies often fail to preserve intrinsic information due to the non-stationarity of
EEG signals where statistical properties change over time. To address this, we propose RL-BioAug, a framework that leverages a label-efficient reinforcement learning (RL)
agent to autonomously determine optimal augmentation policies. While utilizing only a minimal fraction (10%) of labeled data to guide the
agent's policy, our method enables the encoder to learn robust representations in a strictly self-supervised manner. Experimental results demonstrate that RL-BioAug significantly outperforms the random selection strategy, achieving substantial improvements of 9.69% and 8.80% in Macro-F1 score on the Sleep-EDFX and CHB-MIT
datasets, respectively. Notably, this
agent mainly chose optimal strategies for each task–for example, Time Masking with a 62% probability for sleep stage classification and Crop & Resize with a 77% probability for seizure detection. Our framework suggests its potential to replace conventional heuristic-based augmentations and establish a new autonomous paradigm for data augmentation. The source code is available at https://github.com/dlcjfgmlnasa/RL-BioAug.
Read the full abstract ▾
Machine Learning
Differentiable Logic Synthesis: Spectral Coefficient Selection via Sinkhorn-Constrained Composition
Gorgi Pavlov
Learning precise Boolean logic via gradient descent remains challenging: neural networks typically converge to "fuzzy" approximations that degrade under quantization. We introduce Hierarchical Spectral Composition, a differentiable architecture that selects spectral coefficients from a frozen Boolean
Fourier basis and composes them via Sinkhorn-constrained routing with column-sign modulation. Our approach draws on recent insights from
Manifold-Constrained Hyper-Connections (mHC), which demonstrated that projecting routing
matrices onto the Birkhoff polytope preserves identity mappings and stabilizes large-scale training. We adapt this framework to logic synthesis, adding column-sign modulation to enable Boolean negation – a capability absent in standard doubly
stochastic routing. We validate our approach across four phases of increasing complexity: (1) For n=2 (16 Boolean operations over 4-dim basis), gradient descent achieves 100% accuracy with zero routing drift and zero-loss quantization to ternary masks. (2) For n=3 (10 three-variable operations), gradient descent achieves 76% accuracy, but exhaustive enumeration over 3^8 = 6561 configurations proves that optimal ternary masks exist for all operations (100% accuracy, 39% sparsity). (3) For n=4 (10 four-variable operations over 16-dim basis), spectral synthesis – combining exact Walsh-Hadamard coefficients, ternary quantization, and MCMC refinement with parallel tempering – achieves 100% accuracy on all operations. This progression establishes (a) that ternary
polynomial threshold representations exist for all tested functions, and (b) that finding them requires methods beyond pure gradient descent as dimensionality grows. All operations enable single-cycle combinational logic inference at 10,959 MOps/s on GPU, demonstrating viability for hardware-efficient neuro-symbolic logic synthesis.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.RO
Efficient Coordination with the System-Level Shared State: An Embodied-AI Native Modular Framework
Yixuan Deng, Tongrun Wu, Donghao Wu et al. (+5 more)
As
Embodied AI systems move from research prototypes to real world deployments, they tend to evolve rapidly while remaining reliable under workload changes and partial failures. In practice, many deployments are only partially decoupled: middleware moves messages, but shared context and feedback semantics are implicit, causing interface drift, cross-module interference, and brittle recovery at scale. We present ANCHOR, a modular framework that makes decoupling and robustness explicit system-level primitives. ANCHOR separates (i) Canonical Records, an evolvable contract for the standardized shared state, from (ii) a communication bus for many-to-many dissemination and feedback-oriented coordination, forming an inspectable end-to-end loop. We validate closed-loop feasibility on a de-identified workflow instantiation, characterize latency distributions under varying payload sizes and publish rates, and demonstrate automatic stream resumption after hard crashes and restarts even with shared-
memory loss. Overall, ANCHOR turns ad-hoc integration glue into explicit contracts, enabling controlled degradation under load and self-healing recovery for scalable deployment of closed-loop AI systems.
Read the full abstract ▾
All authors Yixuan Deng, Tongrun Wu, Donghao Wu, Zeyu Wei, Jiayuan Wang, Zhenglong Sun, Yuqing Tang, Xiaoqiang Ji
Machine Learning · Cross-listed from cs.CV
TrackletGPT: A Language-like GPT Framework for White Matter Tract Segmentation
Anoushkrit Goel, Simroop Singh, Ankita Joshi et al. (+4 more)
White Matter Tract
Segmentation is imperative for studying
brain structural connectivity, neurological disorders and neurosurgery. This task remains complex, as tracts differ among themselves, across subjects and conditions, yet have similar 3D structure across hemispheres and subjects. To address these challenges, we propose TrackletGPT, a
language-like GPT framework which reintroduces sequential information in
tokens using tracklets. TrackletGPT generalises seamlessly across
datasets, is fully automatic, and encodes granular sub-streamline segments, Tracklets, scaling and refining GPT models in Tractography
Segmentation. Based on our experiments, TrackletGPT outperforms state-of-the-art methods on average DICE, Overlap and Overreach scores on TractoInferno and HCP
datasets, even on inter-
dataset experiments.
Read the full abstract ▾
All authors Anoushkrit Goel, Simroop Singh, Ankita Joshi, Ranjeet Ranjan Jha, Chirag Ahuja, Aditya Nigam, Arnav Bhavsar
Machine Learning · Cross-listed from cs.SD
Towards Effective Negation Modeling in Joint Audio-Text Models for Music
Yannis Vasilakis, Rachel Bittner and Johan Pauwels
Joint audio-text models are widely used for music
retrieval, yet they struggle with semantic phenomena such as negation. Negation is fundamental for distinguishing the absence (or presence) of musical elements (e.g., "with vocals" vs. "without vocals"), but current systems fail to represent this reliably. In this work, we investigate and mitigate this limitation by training CLAP models from scratch on the Million Song
Dataset with LP-MusicCaps-MSD captions. We introduce negation through text augmentation and a dissimilarity-based contrastive loss, designed to explicitly separate original and negated captions in the joint
embedding space. To evaluate progress, we propose two protocols that frame negation modeling as
retrieval and binary classification tasks. Experiments demonstrate that both methods, individually and combined, improve negation handling while largely preserving
retrieval performance.
Read the full abstract ▾
Machine Learning · Cross-listed from q-bio.QM
SCG With Your Phone: Diagnosis of Rhythmic Spectrum Disorders in Field Conditions
Peter Golenderov, Yaroslav Matushenko, Anastasia Tushina et al. (+1 more)
Aortic valve opening (AO) events are crucial for detecting frequency and rhythm disorders, especially in real-world settings where seismocardiography (SCG) signals collected via consumer smartphones are subject to noise, motion artifacts, and variability caused by device heterogeneity. In this work, we present a robust deep-learning framework for SCG
segmentation and rhythm analysis using accelerometer recordings obtained with consumer smartphones. We develop an enhanced U-Net v3 architecture that integrates multi-scale convolutions, residual connections, and
attention gates, enabling reliable
segmentation of noisy SCG signals. A dedicated post-processing pipeline converts probability masks into precise AO timestamps, whereas a novel adaptive 3D-to-1D projection method ensures robustness to arbitrary smartphone orientation. Experimental results demonstrate that the proposed method achieves consistently high accuracy and robustness across various device types and unsupervised data-collection conditions. Our approach enables practical, low-cost, and automated cardiac-rhythm monitoring using everyday mobile devices, paving the way for scalable, field-deployable cardiovascular assessment and future
multimodal diagnostic systems.
Read the full abstract ▾
All authors Peter Golenderov, Yaroslav Matushenko, Anastasia Tushina, Michal Barodkin
Machine Learning · Cross-listed from cs.GT
Asymmetric regularization mechanism for GAN training with Variational Inequalities
Spyridon C. Giagtzoglou, Mark H.M. Winands and Barbara Franci
We formulate the training of generative
adversarial networks (GANs) as a Nash
equilibrium seeking problem. To stabilize the training process and find a Nash
equilibrium, we propose an asymmetric regularization mechanism based on the classic Tikhonov step and on a novel zero-centered gradient penalty. Under smoothness and a local identifiability condition induced by a Gauss-Newton Gramian, we obtain explicit Lipschitz and (strong)-monotonicity constants for the regularized operator. These constants ensure last-iterate linear convergence of a single-call Extrapolation-from-the-Past (EFTP) method. Empirical simulations on an academic example show that, even when strong monotonicity cannot be achieved, the asymmetric regularization is enough to converge to an
equilibrium and stabilize the trajectory.
Read the full abstract ▾
Machine Learning
TractRLFusion: A GPT-Based Multi-Critic Policy Fusion Framework for Fiber Tractography
Ankita Joshi, Ashutosh Sharma, Anoushkrit Goel et al. (+4 more)
Tractography plays a pivotal role in the non-invasive reconstruction of white matter fiber pathways, providing vital information on
brain connectivity and supporting precise neurosurgical planning. Although traditional methods relied mainly on classical deterministic and probabilistic approaches, recent progress has benefited from supervised deep learning (DL) and deep reinforcement learning (DRL) to improve tract reconstruction. A persistent challenge in tractography is accurately reconstructing white matter tracts while minimizing spurious connections. To address this, we propose TractRLFusion, a novel GPT-based policy fusion framework that integrates multiple RL policies through a data-driven fusion strategy. Our method employs a two-stage training data selection process for effective policy fusion, followed by a multi-critic fine-tuning phase to enhance robustness and generalization. Experiments on HCP, ISMRM, and TractoInferno
datasets demonstrate that TractRLFusion outperforms individual RL policies as well as state-of-the-art classical and DRL methods in accuracy and anatomical reliability.
Read the full abstract ▾
All authors Ankita Joshi, Ashutosh Sharma, Anoushkrit Goel, Ranjeet Ranjan Jha, Chirag Ahuja, Arnav Bhavsar, Aditya Nigam
Machine Learning
Multi-Objective Hierarchical Optimization with Large Language Models
Andrej Schwanke, Lyubomir Ivanov, David Salinas et al. (+2 more)
Despite their widespread adoption in various domains, especially due to their powerful
reasoning capabilities, Large
Language Models (LLMs) are not the off-the-shelf choice to drive multi-objective optimization yet. Conventional strategies rank high in
benchmarks due to their intrinsic capabilities to handle numerical inputs and careful modelling choices that balance exploration and Pareto-front exploitation, as well as handle multiple (conflicting) objectives. In this paper, we close this gap by leveraging LLMs as surrogate models and candidate samplers inside a structured hierarchical search strategy. By adaptively partitioning the input space into disjoint hyperrectangular regions and ranking them with a composite score function, we restrict the generative process of the LLM to specific, high-potential sub-spaces, hence making the problem easier to solve as the LLM doesn't have to reason about the global structure of the problem, but only locally instead. We show that under standard regularity assumptions, our
algorithm generates candidate solutions that converge to the true Pareto set in Hausdorff distance. Empirically, it consistently outperforms the global LLM-based multi-objective optimizer and is on par with standard evolutionary and
Bayesian optimization
algorithm on synthetic and real-world
benchmarks.
Read the full abstract ▾
All authors Andrej Schwanke, Lyubomir Ivanov, David Salinas, Frank Hutter, Arber Zela
Machine Learning · Cross-listed from stat.ML
Finite-Sample Unbiased Variance of MMD under Unbalanced Sampling: Exact Estimation and Quasi-Linear Computation
Shijie Zhong, Yikun Yang, Da Gong et al. (+1 more)
Accurately and efficiently estimating the variance of the Maximum Mean Discrepancy (MMD) remains challenging, particularly for unbalanced sample sizes. In this paper, we derive a finite-sample unbiased
estimator of the MMD variance. To overcome the traditional $\mathcal{O}(N^2)$ computational bottleneck, we develop a recursive prefix-suffix accumulation scheme for the Laplace kernel, reducing the computational complexity to $\mathcal{O}(N \log N)$ while requiring $\mathcal{O}(N)$
memory. Experimental results verify the theoretical exactness and numerical stability of the proposed
estimator and demonstrate its scalability on large
datasets. Furthermore, the method proves effective for monitoring distributional convergence during the training of Time-series Generative
Adversarial Networks (TimeGAN).
Read the full abstract ▾
All authors Shijie Zhong, Yikun Yang, Da Gong, Jiangfeng Fu
Machine Learning · Cross-listed from cs.CV
Probabilistic Deep Discriminant Analysis for Wind Blade Segmentation
Raül Pérez-Gonzalo, Andreas Espersen and Antonio Agudo
Linear discriminant analysis improves class separability but struggles with non-linearly separable data. To overcome this, we introduce Deep Discriminant Analysis (DDA), which directly optimizes the Fisher criterion utilizing deep networks. To ensure stable training and avoid computational instabilities, we incorporate signed between-class variance, bound outputs with a sigmoid function, and convert multiplicative relationships into additive ones. We present two stable DDA loss functions and augment them with a probability loss, resulting in Probabilistic DDA (PDDA). PDDA effectively minimizes class overlap in output distributions, producing highly confident predictions with reduced within-class variance. When applied to wind blade
segmentation, PDDA showcases notable advances in performance and consistency, critical for wind energy maintenance. To our knowledge, this is the first application of DDA to image
segmentation.
Read the full abstract ▾
Machine Learning
Inverting Self-Organizing Maps: A Unified Activation-Based Framework
Alessandro Londei, Matteo Benati, Denise Lanzieri et al. (+1 more)
Self-Organizing Maps (SOMs) provide
topology-preserving projections of high-dimensional data, yet their use as generative models remains largely unexplored. We show that the activation pattern of a SOM – the squared distances to its prototypes – can be inverted to recover the exact input, following from a classical result in Euclidean distance geometry: a point in $D$ dimensions is uniquely determined by its distances to $D{+}1$ affinely independent references. We derive the corresponding linear system and characterize the conditions under which inversion is well-posed. Building on this mechanism, we introduce the
Manifold-Aware Unified SOM Inversion and Control (MUSIC) update rule, which modifies squared distances to selected prototypes while preserving others, producing controlled, semantically meaningful trajectories aligned with the SOM's piecewise-linear structure. Tikhonov regularization stabilizes the update and ensures smooth motion in high dimensions. Unlike variational or
diffusion-based generative models, MUSIC requires no sampling, latent priors, or learned decoders: it operates entirely on prototype geometry. If no perturbation is applied, inversion recovers the exact input; when a target prototype or cluster is specified, MUSIC produces coherent semantic transitions. We validate the framework on synthetic Gaussian mixtures, MNIST digits, and the Labeled Faces in the Wild
dataset. Across all settings, MUSIC trajectories maintain high classifier confidence, produce significantly sharper intermediate images than linear interpolation, and reveal an interpretable
geometric structure of the learned map.
Read the full abstract ▾
All authors Alessandro Londei, Matteo Benati, Denise Lanzieri, Vittorio Loreto
Machine Learning · Cross-listed from eess.AS
Co-Initialization of Control Filter and Secondary Path via Meta-Learning for Active Noise Control
Ziyi Yang, Li Rao, Zhengding Luo et al. (+3 more)
Active noise control (ANC) must adapt quickly when the acoustic environment changes, yet early performance is largely dictated by initialization. We address this with a Model-Agnostic Meta-Learning (MAML) co-initialization that jointly sets the control filter and the secondary-path model for FxLMS-based ANC while keeping the runtime
algorithm unchanged. The initializer is pre-trained on a small set of measured paths using short two-phase inner loops that mimic identification followed by residual-noise reduction, and is applied by simply setting the learned initial coefficients. In an online secondary path modeling FxLMS testbed, it yields lower early-stage error, shorter time-to-target, reduced auxiliary-noise energy, and faster recovery after path changes than a baseline without re-initialization. The method provides a simple fast start for feedforward ANC under environment changes, requiring a small set of paths to pre-train.
Read the full abstract ▾
All authors Ziyi Yang, Li Rao, Zhengding Luo, Dongyuan Shi, Qirui Huang, Woon-Seng Gan
Machine Learning · Cross-listed from cs.AI
Virtual Urbanism: An AI-Driven Framework for Quantifying Urban Identity. A Tokyo-Based Pilot Study Using Diffusion-Generated Synthetic Environments
Glinskaya Maria
This paper introduces Virtual Urbanism (VU), a
multimodal AI-driven analytical framework for quantifying urban identity through the medium of synthetic urban replicas. The framework aims to advance computationally tractable urban identity metrics. To demonstrate feasibility, the pilot study Virtual Urbanism and Tokyo Microcosms is presented. A pipeline integrating Stable
Diffusion and LoRA models was used to produce synthetic replicas of nine Tokyo areas rendered as dynamic synthetic urban sequences, excluding existing orientation markers to elicit core identity-forming elements. Human-evaluation experiments (I) assessed perceptual legitimacy of replicas; (II) quantified area-level identity; (III) derived core identity-forming elements. Results showed a mean identification accuracy of ~81%, confirming the validity of the replicas. Urban Identity Level (UIL) metric enabled assessment of identity levels across areas, while semantic analysis revealed culturally embedded typologies as core identity-forming elements, positioning VU as a viable framework for AI-augmented urban analysis, outlining a path toward automated, multi-parameter identity metrics.
Read the full abstract ▾
Machine Learning
Optimal L2 Regularization in High-dimensional Continual Linear Regression
Gilad Karpel, Edward Moroshko, Ran Levinstein et al. (+3 more)
We study generalization in an overparameterized continual linear regression setting, where a model is trained with L2 (isotropic) regularization across a sequence of tasks. We derive a closed-form expression for the expected generalization loss in the high-dimensional regime that holds for arbitrary linear teachers. We demonstrate that isotropic regularization mitigates label noise under both single-teacher and multiple i.i.d. teacher settings, whereas prior work accommodating multiple teachers either did not employ regularization or used
memory-demanding methods. Furthermore, we prove that the optimal fixed regularization strength scales nearly linearly with the number of tasks $T$, specifically as $T/\ln T$. To our knowledge, this is the first such result in theoretical continual learning. Finally, we validate our theoretical findings through experiments on linear regression and neural networks, illustrating how this scaling law affects generalization and offering a practical recipe for the design of continual learning systems.
Read the full abstract ▾
All authors Gilad Karpel, Edward Moroshko, Ran Levinstein, Ron Meir, Daniel Soudry, Itay Evron
Machine Learning
ELSA: Efficient LLM-Centric Split Aggregation for Privacy-Aware Hierarchical Federated Learning over the Network Edge
Xiaohong Yang, Tong Xie, Minghui Liwang et al. (+6 more)
Training large
language models (LLMs) in resource-constrained edge networks poses fundamental challenges in computation, communication, data heterogeneity, and
privacy. To address these challenges, we propose ELSA (Efficient LLM-Centric Split Aggregation), a distributed fine-tuning framework that integrates split learning (SL) with hierarchical
federated learning (HFL) to coordinate computation and model aggregation across heterogeneous clients, edge servers, and the cloud. ELSA introduces three key innovations. First, it employs a task-agnostic, behavior-aware client clustering mechanism that constructs semantic fingerprints from public probe inputs using symmetric Kullback-Leibler (KL) divergence, complemented by prediction-consistency-based trust scoring and latency-aware edge assignment to account for data heterogeneity, client reliability, and communication constraints. Second, it adopts a resource-aware dynamic model partitioning strategy that distributes the LLM across clients and edge servers in three segments, while reserving the cloud for adapter aggregation. This design coordinates computation across heterogeneous nodes while balancing on-device computational demands and global convergence behavior. Third, it integrates computational sketches with semantic subspace orthogonal perturbation (SS-OP) to reduce communication overhead and mitigate
privacy leakage during model exchanges. Extensive experiments across diverse NLP tasks demonstrate that ELSA outperforms state-of-the-art baselines in model performance, convergence behavior, and robustness. Further, on-device profiling on NVIDIA RTX 4090 GPUs and Jetson AGX Orin edge devices characterizes the
memory footprint, energy consumption, and execution latency of key training stages, providing empirical evidence of ELSA's practical feasibility.
Read the full abstract ▾
All authors Xiaohong Yang, Tong Xie, Minghui Liwang, Chikai Shang, Yang Lu, Zhenzhen Jiao, Liqun Fu, Seyyedali Hosseinalipour, Xianbin Wang
Machine Learning · Cross-listed from cs.DC
Device Association and Resource Allocation for Hierarchical Split Federated Learning in Space-Air-Ground Integrated Network
Haitao Zhao, Xiaoyu Tang, Bo Xu et al. (+2 more)
6G facilitates deployment of
Federated Learning (FL) in the Space-Air-Ground Integrated Network (SAGIN), yet FL confronts challenges such as resource constrained and unbalanced data distribution. To address these issues, this paper proposes a Hierarchical Split
Federated Learning (HSFL) framework and derives its upper bound of loss function. To minimize the weighted sum of training loss and latency, we formulate a joint optimization problem that integrates device association, model split layer selection, and resource allocation. We decompose the original problem into several subproblems, where an iterative optimization
algorithm for device association and resource allocation based on brute-force split point search is proposed. Simulation results demonstrate that the proposed
algorithm can effectively balance training efficiency and model accuracy for FL in SAGIN.
Read the full abstract ▾
All authors Haitao Zhao, Xiaoyu Tang, Bo Xu, Jinlong Sun, Linghao Zhang
Machine Learning · Cross-listed from cs.CV
Discriminant Learning-based Colorspace for Blade Segmentation
Raül Pérez-Gonzalo, Andreas Espersen and Antonio Agudo
Suboptimal color representation often hinders accurate image
segmentation, yet many modern
algorithms neglect this critical preprocessing step. This work presents a novel multidimensional nonlinear discriminant analysis
algorithm, Colorspace Discriminant Analysis (CSDA), for improved
segmentation. Extending Linear Discriminant Analysis into a deep learning context, CSDA customizes color representation by maximizing multidimensional signed inter-class separability while minimizing intra-class variability through a generalized discriminative loss. To ensure stable training, we introduce three alternative losses that enable end-to-end optimization of both the discriminative colorspace and
segmentation process. Experiments on wind turbine blade data demonstrate significant accuracy gains, emphasizing the importance of tailored preprocessing in domain-specific
segmentation.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Knowledge Graph-Assisted LLM Post-Training for Enhanced Legal Reasoning
Dezhao Song, Guglielmo Bonifazi, Frank Schilder et al. (+1 more)
LLM post-training has primarily relied on large text corpora and human feedback, without capturing the structure of domain knowledge. This has caused models to struggle dealing with complex
reasoning tasks, especially for high-stakes professional domains. In Law,
reasoning requires deep understanding of the relations between various legal concepts, a key component missing in current LLM post-training. In this paper, we propose a knowledge graph (KG)-assisted approach for enhancing LLMs'
reasoning capability in Legal that is generalizable to other high-stakes domains. We model key legal concepts by following the IRAC (Issue, Rule, Analysis and Conclusion) framework, and construct a KG with 12K legal cases. We then produce training data using our IRAC KG, and conduct both Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO) with three state-of-the-art (SOTA) LLMs (30B, 49B and 70B), varying architecture and base model family. Our post-trained models obtained better average performance on 4/5 diverse legal
benchmarks (14 tasks) than baselines. In particular, our 70B DPO model achieved the best score on 4/6
reasoning tasks, among baselines and a 141B SOTA legal LLM, demonstrating the effectiveness of our KG for enhancing LLMs' legal
reasoning capability.
Read the full abstract ▾
All authors Dezhao Song, Guglielmo Bonifazi, Frank Schilder, Jonathan Richard Schwarz
Machine Learning · Cross-listed from cs.CV
CFM: Language-aligned Concept Foundation Model for Vision
Kai Wittenmayer, Sukrut Rao, Amin Parchami-Araghi et al. (+2 more)
Language-aligned
vision foundation models perform strongly across diverse downstream tasks. Yet, their learned representations remain opaque, making interpreting their decision-making difficult. Recent work decompose these representations into human-interpretable concepts, but provide poor spatial grounding and are limited to image classification tasks. In this work, we propose CFM, a
language-aligned concept foundation model for
vision that provides fine-grained concepts, which are human-interpretable and spatially grounded in the input image. When paired with a foundation model with strong semantic representations, we get explanations for any of its downstream tasks. Examining local co-occurrence dependencies of concepts allows us to define concept relationships through which we improve concept naming and obtain richer explanations. On
benchmark data, we show that CFM provides performance on classification,
segmentation, and captioning that is competitive with opaque foundation models while providing fine-grained, high quality concept-based explanations. Code at https://github.com/kawi19/CFM. Interactive visualizations at https://concept-foundation-model.mpi-inf.mpg.de.
Read the full abstract ▾
All authors Kai Wittenmayer, Sukrut Rao, Amin Parchami-Araghi, Bernt Schiele, Jonas Fischer
Machine Learning
PAtt: A Pattern Attention Network for ETA Prediction Using Historical Speed Profiles
ByeoungDo Kim, JunYeop Na, Kyungwook Tak et al. (+3 more)
In this paper, we propose an ETA model (Estimated Time of Arrival) that leverages an
attention mechanism over historical road speed patterns. As autonomous driving and intelligent transportation systems become increasingly prevalent, the need for accurate and reliable ETA estimation has grown, playing a vital role in
navigation, mobility planning, and traffic management. However, predicting ETA remains a challenging task due to the dynamic and complex nature of traffic flow. Traditional methods often combine real-time and historical traffic data in simplistic ways, or rely on complex rule-based computations. While recent deep learning models have shown potential, they often require high computational costs and do not effectively capture the spatio-temporal patterns crucial for ETA prediction. ETA prediction inherently involves spatio-temporal causality, and our proposed model addresses this by leveraging
attention mechanisms to extract and utilize temporal features accumulated at each spatio-temporal point along a route. This architecture enables efficient and accurate ETA estimation while keeping the model lightweight and scalable. We validate our approach using real-world driving
datasets and demonstrate that our approach outperforms existing baselines by effectively integrating road characteristics, real-time traffic conditions, and historical speed patterns in a task-aware manner.
Read the full abstract ▾
All authors ByeoungDo Kim, JunYeop Na, Kyungwook Tak, JunTae Kim, DongHyeon Kim, Duckky Kim
Machine Learning
Principled Latent Diffusion for Graphs via Laplacian Autoencoders
Antoine Siraudin and Christopher Morris
Graph
diffusion models achieve state-of-the-art performance in graph generation but suffer from quadratic complexity in the number of nodes – and much of their capacity is wasted modeling the absence of edges in sparse graphs. Inspired by latent
diffusion in other modalities, a natural idea is to compress graphs into a low-dimensional latent space and perform
diffusion in that space. However, unlike images or text, graph generation requires nearly lossless reconstruction, as even a single error in decoding an adjacency
matrix can render the entire sample invalid. This challenge has remained largely unaddressed. We propose LG-Flow, a latent graph
diffusion framework that directly overcomes these obstacles. A permutation-equivariant autoencoder maps nodes to fixed-dimensional
embeddings that enable near-lossless reconstruction of both undirected graphs and DAGs. The dimensionality of this latent representation scales linearly with the number of nodes, thereby removing the quadratic adjacency-space bottleneck in the
diffusion process and enabling the training of substantially larger generative backbones. In this latent space, we train a
Diffusion Transformer with flow matching, enabling efficient and expressive graph generation. Our approach achieves competitive results against state-of-the-art graph
diffusion models while delivering up to a $1000\times$ speed-up. Our code is available at https://github.com/asiraudin/LG-Flow .
Read the full abstract ▾
Machine Learning
Orthogonium : A Unified, Efficient Library of Orthogonal and 1-Lipschitz Building Blocks
Thibaut Boissin, Franck Mamalet, Valentin Lafargue et al. (+1 more)
Orthogonal and 1-Lipschitz neural network layers are essential building blocks in robust deep learning architectures, crucial for certified
adversarial robustness, stable generative models, and reliable recurrent networks. Despite significant advancements, existing implementations remain fragmented, limited, and computationally demanding. To address these issues, we introduce Orthogonium , a unified, efficient, and comprehensive PyTorch library providing orthogonal and 1-Lipschitz layers. Orthogonium provides access to standard convolution features-including support for strides, dilation, grouping, and transposed-while maintaining strict mathematical guarantees. Its optimized implementations reduce overhead on large scale
benchmarks such as ImageNet. Moreover, rigorous testing within the library has uncovered critical errors in existing implementations, emphasizing the importance of standardized and reliable tools. Orthogonium thus significantly lowers adoption barriers, enabling scalable experimentation and integration across diverse applications requiring orthogonality and robust Lipschitz constraints. Orthogonium is available at https://github.com/deel-ai/orthogonium.
Read the full abstract ▾
All authors Thibaut Boissin, Franck Mamalet, Valentin Lafargue, Mathieu Serrurier
Machine Learning · Cross-listed from cs.AI
Look-Ahead-Bench: a Standardized Benchmark of Look-ahead Bias in Point-in-Time LLMs for Finance
Mostapha Benhenda
We introduce Look-Ahead-Bench, a standardized
benchmark measuring look-ahead bias in Point-in-Time (PiT) Large
Language Models (LLMs) within realistic and practical financial workflows. Unlike most existing approaches that primarily test inner lookahead knowledge via Q\&A, our
benchmark evaluates model behavior in practical scenarios. To distinguish genuine predictive capability from memorization-based performance, we analyze performance decay across temporally distinct
market regimes, incorporating several quantitative baselines to establish performance thresholds. We evaluate prominent open-source LLMs – Llama 3.1 (8B and 70B) and DeepSeek 3.2 – against a family of Point-in-Time LLMs (Pitinf-Small, Pitinf-Medium, and frontier-level model Pitinf-Large) from PiT-Inference. Results reveal significant lookahead bias in standard LLMs, as measured with alpha decay, unlike Pitinf models, which demonstrate improved generalization and
reasoning abilities as they scale in size. This work establishes a foundation for the standardized evaluation of temporal bias in financial LLMs and provides a practical framework for identifying models suitable for real-world deployment. Code is available on GitHub: https://github.com/benstaf/lookaheadbench
Read the full abstract ▾
Machine Learning
vLinear: A Powerful Linear Model for Multivariate Time Series Forecasting
Wenzhen Yue, Ruohao Guo, Ji Shi et al. (+3 more)
In this paper, we present vLinear, an effective yet efficient linear-based multivariate time series forecaster featuring two components: the vecTrans module and the WFMLoss objective. Many state-of-the-art forecasters rely on self-
attention or its variants to capture multivariate correlations, typically incurring $\mathcal{O}(N^2)$ computational complexity with respect to the number of variates $N$. To address this, we propose vecTrans, a lightweight module that utilizes a learnable vector to model multivariate correlations, reducing the complexity to $\mathcal{O}(N)$. Notably, vecTrans can be seamlessly integrated into
Transformer-based forecasters, delivering up to 5$\times$ inference speedups and consistent performance gains. Furthermore, we introduce WFMLoss (Weighted Flow Matching Loss) as the objective. In contrast to typical velocity-oriented flow matching objectives, we demonstrate that a final-series-oriented formulation yields significantly superior
forecasting accuracy. WFMLoss also incorporates path- and horizon-weighted strategies to focus learning on more reliable paths and horizons. Empirically, vLinear achieves state-of-the-art performance across 22
benchmarks and 124
forecasting settings. Moreover, WFMLoss serves as an effective plug-and-play objective, consistently improving existing forecasters. The code is available at https://anonymous.4open.science/r/vLinear.
Read the full abstract ▾
All authors Wenzhen Yue, Ruohao Guo, Ji Shi, Zihan Hao, Shiyu Hu, Xianghua Ying
Machine Learning · Cross-listed from cs.CV
Towards Onboard Continuous Change Detection for Floods
Daniel Kyselica, Jonáš Herec, Oliver Kutis et al. (+1 more)
Natural disaster monitoring through continuous satellite observation requires processing multi-temporal data under strict operational constraints. This paper addresses flood detection, a critical application for hazard management, by developing an onboard change detection system that operates within the
memory and computational limits of small satellites. We propose History Injection mechanism for
Transformer models (HiT), that maintains historical context from previous observations while reducing data storage by over 99% of original image size. Moreover, testing on the STTORM-CD flood
dataset confirms that the HiT mechanism within the Prithvi-tiny foundation model maintains detection accuracy compared to the bi-temporal baseline. The proposed HiT-Prithvi model achieved 43 FPS on Jetson Orin Nano, a representative onboard hardware used in nanosats. This work establishes a practical framework for satellite-based continuous monitoring of natural disasters, supporting real-time hazard assessment without dependency on ground-based processing infrastructure. Architecture as well as model checkpoints is available at https://github.com/zaitra/HiT-change-detection .
Read the full abstract ▾
All authors Daniel Kyselica, Jonáš Herec, Oliver Kutis, Rado Pitoňák
Machine Learning · Cross-listed from cs.CL
Pro-AI Bias in Large Language Models
Benaya Trabelsi, Jonathan Shaki and Sarit Kraus
Large
language models (LLMs) are increasingly employed for decision-support across multiple domains. We ask whether these models favor artificial intelligence (AI) itself. We examine this question through three experiments. First, in open-ended advice across investment, study, career, and startup domains, AI is prominent in most recommendation lists and is usually ranked first when it appears. Second, in salary estimations, models overestimate AI-related jobs more than they do for closely matched non-AI jobs. In a controlled follow-up, changing only the job title from the matched non-AI title to the AI-related title raises predicted salary by 10%, while all other supplied job details remain fixed. Finally, we show that the number of degrees and other credentials awarded by AI programs were 28% more overestimated than corresponding awards for matched computing programs. Overall, we show that AI is not only prominent in everyday LLM advice, but also receives systematically inflated quantitative estimates compared to non-AI cases, even against its closest disciplinary neighbors, raising concerns for high-stakes decision support.
Read the full abstract ▾
Machine Learning
EEG-Titans: Long-Horizon Seizure Forecasting via Dual-Branch Attention and Neural Memory
Tien-Dat Pham and Xuan-The Tran
Accurate epileptic seizure prediction from electroencephalography (
EEG) remains challenging because pre-ictal dynamics may span long time horizons while clinically relevant signatures can be subtle and transient. Many deep learning models face a persistent trade-off between capturing local spatiotemporal patterns and maintaining informative long-range context when operating on ultralong sequences. We propose
EEG-Titans, a dualbranch architecture that incorporates a modern neural
memory mechanism for long-context modeling. The model combines sliding-window
attention to capture short-term anomalies with a recurrent
memory pathway that summarizes slower, progressive trends over time. On the CHB-MIT scalp
EEG dataset, evaluated under a chronological holdout protocol,
EEG-Titans achieves 99.46% average segment-level sensitivity across 18 subjects. We further analyze safety-first operating points on artifact-prone recordings and show that a hierarchical context strategy extending the receptive field for high-noise subjects can markedly reduce false alarms (down to 0.00 FPR/h in an extreme outlier) without sacrificing sensitivity. These results indicate that
memory-augmented long-context modeling can provide robust seizure
forecasting under clinically constrained evaluation
Read the full abstract ▾
Machine Learning · Cross-listed from cs.NI
Variational Dual-path Attention Network for CSI-Based Gesture Recognition
N.Zhang
Wi-Fi gesture recognition based on Channel State Information (CSI) is challenged by high-dimensional noise and resource constraints on edge devices. Prevailing end-to-end models tightly couple feature extraction with classification, overlooking the inherent time-frequency sparsity of CSI and leading to redundancy and poor generalization. To address this, this paper proposes a lightweight feature preprocessing module–the Variational Dual-path
Attention Network (VDAN). It performs structured feature refinement through frequency-domain filtering and temporal detection. Variational inference is introduced to model the uncertainty in
attention weights, thereby enhancing robustness to noise. The design principles of the module are explained from the perspectives of the information bottleneck and regularization. Experiments on a public
dataset demonstrate that the learned
attention weights align with the physical sparse characteristics of CSI, verifying its interpretability. This work provides an efficient and explainable front-end processing solution for resource-constrained wireless sensing systems.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.SC
Breaking the Data Barrier in Learning Symbolic Computation: A Case Study on Variable Ordering Suggestion for Cylindrical Algebraic Decomposition
Rui-Juan Jing, Yuegang Zhao and Changbo Chen
Symbolic computation, powered by modern computer
algebra systems, has important applications in mathematical
reasoning through exact deep computations. The efficiency of symbolic computation is largely constrained by such deep computations in high dimension. This creates a fundamental barrier on labelled data acquisition if leveraging supervised deep learning to accelerate symbolic computation. Cylindrical algebraic decomposition (CAD) is a pillar symbolic computation method for
reasoning with first-order logic formulas over reals with many applications in formal verification and automatic
theorem proving. Variable orderings have a huge impact on its efficiency. Impeded by the difficulty to acquire abundant labelled data, existing learning-based approaches are only competitive with the best expert-based heuristics. In this work, we address this problem by designing a series of intimately connected tasks for which a large amount of annotated data can be easily obtained. We pre-train a
Transformer model with these data and then fine-tune it on the
datasets for CAD ordering. Experiments on publicly available CAD ordering
datasets show that on average the orderings predicted by the new model are significantly better than those suggested by the best heuristic methods.
Read the full abstract ▾