Machine Learning · Cross-listed from q-bio.GN
G2DR: A Genotype-First Framework for Genetics-Informed Target Prioritization and Drug Repurposing
Muhammad Muneeb and David B. Ascher
Human genetics offers a promising route to therapeutic discovery, yet practical frameworks translating genotype-derived signal into ranked target and drug hypotheses remain limited, particularly when matched disease transcriptomics are unavailable. Here we present G2DR, a genotype-first prioritization framework propagating inherited variation through genetically predicted expression, multi-method gene-level testing, pathway enrichment, network context, druggability, and multi-source drug–target evidence integration. In a migraine case study with 733 UK Biobank participants under stratified five-fold cross-validation, we imputed expression across seven transcriptome-weight resources and ranked genes using a reproducibility-aware discovery score from training and validation data, followed by a balanced integrated score for target selection. Discovery-based prioritization generalized to held-out data, achieving gene-level ROC-AUC of 0.775 and PR-AUC of 0.475, while retaining enrichment for curated migraine biology. Mapping prioritized genes to compounds via Open Targets, DGIdb, and ChEMBL yielded drug sets enriched for migraine-linked compounds relative to a global background, though recovery favoured broader mechanism-linked and off-label space over migraine-specific approved therapies. Directionality filtering separated broadly recovered compounds from mechanistically compatible candidates. G2DR is a modular framework for genetics-informed hypothesis generation, not a clinically actionable recommendation system. All outputs require independent experimental, pharmacological, and clinical validation.
Read the full abstract ▾
Machine Learning
Interpretable Multiple Myeloma Prognosis with Observational Medical Outcomes Partnership Data
Salma Rachidi, Aso Bozorgpanah, Eric Fey et al. (+1 more)
Machine learning (ML) promises better clinical decision-making, yet opaque model behavior limits the adoption in healthcare. We propose two novel regularization techniques for ensuring the interpretability of ML models trained on real-world data. In particular, we consider the prediction of five-year survival for multiple myeloma patients using clinical data from Helsinki University Hospital. To ensure the interpretability of the trained models, we use two alternative constructions for a penalty term used for regularization. The first one penalizes deviations from the predictions obtained from an interpretable logistic regression method with two manually chosen features. The second construction requires consistency of model predictions with the revised international staging system (R-ISS). We verify the usefulness of the proposed regularization techniques in numerical experiments using data from 812 patients. They achieve an accuracy up to 0.721 on a test set and SHAP values show that the models rely on the selected important features.
Read the full abstract ▾
All authors Salma Rachidi, Aso Bozorgpanah, Eric Fey, Alexander Jung
Machine Learning
Graph-Aware Stealthy Poison-Text Backdoors for Text-Attributed Graphs
Qi Luo, Minghui Xu, Dongxiao Yu et al. (+1 more)
Modern graph learning systems often combine links with text, as in citation networks with abstracts or social graphs with user posts. In such systems, text is usually easier to edit than graph structure, which creates a practical security risk: an attacker may hide a small malicious cue in training text and later use it to trigger incorrect predictions. This paper studies that risk in a realistic setting where the attacker edits only node text and leaves the graph unchanged. We propose TAGBD, a graph-aware
backdoor attack that first selects training nodes that are easier to manipulate, then generates stealthy poison text with a shadow graph model, and finally injects the text by replacing the original content or appending a short phrase. Experiments on three
benchmark datasets show that TAGBD achieves very high attack success rates, transfers across different graph models, and remains effective under common defenses. These results show that inconspicuous poison text alone can serve as a reliable attack channel in text-attributed graphs, highlighting the need for defenses that inspect both node content and graph structure.
Read the full abstract ▾
All authors Qi Luo, Minghui Xu, Dongxiao Yu, Xiuzhen Cheng
Machine Learning · Cross-listed from cs.IR
Low-pass Personalized Subgraph Federated Recommendation
Wooseok Sim and Hogun Park
Federated Recommender Systems (FRS) preserve
privacy by training decentralized models on client-specific user-item subgraphs without sharing raw data. However, FRS faces a unique challenge: subgraph structural imbalance, where drastic variations in subgraph scale (user/item counts) and connectivity (item degree) misalign client representations, making it challenging to train a robust model that respects each client's unique structural characteristics. To address this, we propose a Low-pass Personalized Subgraph
Federated recommender system (LPSFed). LPSFed leverages graph
Fourier transforms and low-pass spectral filtering to extract low-frequency structural signals that remain stable across subgraphs of varying size and degree, allowing robust personalized parameter updates guided by similarity to a neutral structural anchor. Additionally, we leverage a localized popularity bias-aware margin that captures item-degree imbalance within each subgraph and incorporates it into a personalized bias correction term to mitigate recommendation bias. Supported by theoretical analysis and validated on five real-world
datasets, LPSFed achieves superior recommendation accuracy and enhances model robustness.
Read the full abstract ▾
Machine Learning
Hybrid Autoencoder-Isolation Forest approach for time series anomaly detection in C70XP cyclotron operation data at ARRONAX
F Basbous, F Poirier, F Haddad et al. (+1 more)
The Interest Public Group ARRONAX's C70XP cyclotron, used for radioisotope production for medical and research applications, relies on complex and costly systems that are prone to failures, leading to operational disruptions. In this context, this study aims to develop a machine learning-based method for early anomaly detection, from sensor measurements over a temporal window, to enhance system performance. One of the most widely recognized methods for anomaly detection is Isolation Forest (IF), known for its effectiveness and scalability. However, its reliance on axis-parallel splits limits its ability to detect subtle anomalies, especially those occurring near the mean of normal data. This study proposes a hybrid approach that combines a fully connected Autoencoder (AE) with IF to enhance the detection of subtle anomalies. In particular, the Mean Cubic Error (MCE) of the sensor data reconstructed by the AE is used as input to the IF model. Validated on
proton beam intensity time series data, the proposed method demonstrates a clear improvement in detection performance, as confirmed by the experimental results.
Read the full abstract ▾
All authors F Basbous, F Poirier, F Haddad, D Mateus
Machine Learning
Bounded Coupled AI Learning Dynamics in Tri-Hierarchical Drone Swarms
Oleksii Bychkov
Modern autonomous multi-
agent systems combine heterogeneous learning mechanisms operating at different timescales. An open question remains: can one formally guarantee that coupled dynamics of such mechanisms stay within the admissible operational regime? This paper studies a tri-hierarchical swarm learning system where three mechanisms act simultaneously: (1) local Hebbian online learning at individual
agent level (fast timescale, 10-100 ms); (2) multi-
agent reinforcement learning (MARL) for tactical group coordination (medium timescale, 1-10 s); (3) meta-learning (MAML) for strategic adaptation (slow timescale, 10-100 s). Four results are established. The Bounded Total Error
Theorem shows that under contractual constraints on learning rates, Lipschitz continuity of inter-level mappings, and weight stabilization, total suboptimality admits a component-wise upper bound uniform in time. The Bounded Representation Drift
Theorem gives a worst-case estimate of how Hebbian updates affect coordination-level
embeddings during one MARL cycle. The Meta-Level Compatibility
Theorem provides sufficient conditions under which strategic adaptation preserves lower-level invariants. The Non-Accumulation
Theorem proves that error does not grow unboundedly over time.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Measure flow path recovery in Bayes Hilbert spaces
S. David Mis and Maarten V. de Hoop
We study the ill-posed problem of recovering a probability measure flow from finitely many moving localized sensors using a Bayes
Hilbert framework. Relative to a fixed reference probability measure, a probability law is represented by its centered log-ratio coordinates, so that an evolving law becomes a path in a
Hilbert space of functions. For sufficiently regular Bayes
Hilbert paths, we construct a canonical minimum-energy transport realization of the path by solving a weighted Neumann problem at each time, yielding an intrinsic transport form on tangent directions. We then formulate an inverse problem directly on Bayes
Hilbert path space. Linearization of an observation operator yields an observability form, and recoverability is governed by its interaction with the transport geometry through a joint transport–observability form. In the ambient infinite-dimensional setting, we develop a regularized variational theory and identify limitations of localized sensing: mobile sensors can make the joint form injective, but they do not in general yield a coercive stability estimate on the full state space. This obstruction leads naturally to finite-dimensional Bayes
Hilbert reductions. There the transport form becomes a kinetic tensor and the linearized observations become reduced sensing
matrices, so recoverability can be expressed through explicit Gramian conditions. We show that localized bump sensors detect every fixed reduced direction, that finitely many suitably placed static sensors yield uniform reduced observability, and there exist path-dependent sensor trajectories such that even a single moving sensor can recover the reduced path. Finally, we show that these reduced recovery results lift to approximate ambient recovery for paths that are well approximated by the chosen finite-dimensional subspaces, yielding stable reconstruction up to projection error.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Decorrelation, Diversity, and Emergent Intelligence: The Isomorphism Between Social Insect Colonies and Ensemble Machine Learning
Ernest Fokoué, Gregory Babbitt and Yuval Levental
Social insect colonies and ensemble machine learning methods represent two of the most successful examples of decentralized information processing in nature and computation respectively. Here we develop a rigorous mathematical framework demonstrating that ant colony decision-making and random forest learning are isomorphic under a common formalism of
stochastic ensemble intelligence. We show that the mechanisms by which genetically identical ants achieve functional differentiation – through
stochastic response to local cues and positive feedback – map precisely onto the bootstrap aggregation and random feature subsampling that decorrelate decision trees. Using tools from
Bayesian inference, multi-armed bandit theory, and statistical learning theory, we prove that both systems implement identical variance reduction strategies through decorrelation of identical units. We derive explicit mappings between ant recruitment rates and tree weightings, pheromone trail reinforcement and out-of-bag error estimation, and quorum sensing and prediction averaging. This isomorphism suggests that collective intelligence, whether biological or artificial, emerges from a universal principle: \textbf{randomized identical
agents + diversity-enforcing mechanisms $\rightarrow$ emergent optimality}.
Read the full abstract ▾
Machine Learning
Probing the Latent World: Emergent Discrete Symbols and Physical Structure in Latent Representations
Liu hung ming
Video world models trained with Joint
Embedding Predictive Architectures (JEPA) acquire rich spatiotemporal representations by predicting masked regions in latent space rather than reconstructing pixels. This removes the visual verification pathway of generative models, creating a structural interpretability gap: the encoder has learned physical structure inaccessible in any inspectable form. Existing probing methods either operate in continuous space without a structured intermediate layer, or attach generative components whose parameters confound attribution of behavior to the encoder. We propose the AI Mother Tongue (AIM) framework as a passive quantization probe: a lightweight, vocabulary-free probe that converts V-JEPA 2 continuous latent vectors into discrete symbol sequences without task-specific supervision or modifying the encoder. Because the encoder is kept completely frozen, any symbolic structure in the AIM codebook is attributable entirely to V-JEPA 2 pre-trained representations – not to the probe. We evaluate through category-contrast experiments on Kinetics-mini along three physical dimensions: grasp angle, object geometry, and motion temporal structure. AIM symbol distributions differ significantly across all three experiments (chi^2 p < 10^{-4}; MI 0.036–0.117 bits, NMI 1.2–3.9% of the 3-bit maximum; JSD up to 0.342; codebook active ratio 62.5%). The experiments reveal that V-JEPA 2 latent space is markedly compact: diverse action categories share a common representational core, with semantic differences encoded as graded distributional variations rather than categorical boundaries. These results establish Stage 1 of a four-stage roadmap toward an action-conditioned symbolic world model, demonstrating that structured symbolic
manifolds are discoverable properties of frozen JEPA latent spaces.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
From Masks to Pixels and Meaning: A New Taxonomy, Benchmark, and Metrics for VLM Image Tampering
Xinyi Shang, Yi Tang, Jiacheng Cui et al. (+9 more)
Existing tampering detection
benchmarks largely rely on object masks, which severely misalign with the true edit signal: many pixels inside a mask are untouched or only trivially modified, while subtle yet consequential edits outside the mask are treated as natural. We reformulate VLM image tampering from coarse region labels to a pixel-grounded, meaning and
language-aware task. First, we introduce a taxonomy spanning edit primitives (replace/remove/splice/inpaint/attribute/colorization, etc.) and their semantic class of tampered object, linking low-level changes to high-level understanding. Second, we release a new
benchmark with per-pixel tamper maps and paired category supervision to evaluate detection and classification within a unified protocol. Third, we propose a training framework and evaluation metrics that quantify pixel-level correctness with localization to assess confidence or prediction on true edit intensity, and further measure tamper meaning understanding via semantics-aware classification and natural
language descriptions for the predicted regions. We also re-evaluate the existing strong
segmentation/localization baselines on recent strong tamper detectors and reveal substantial over- and under-scoring using mask-only metrics, and expose failure modes on micro-edits and off-mask changes. Our framework advances the field from masks to pixels, meanings and
language descriptions, establishing a rigorous standard for tamper localization, semantic classification and description. Code and
benchmark data are available at https://github.com/VILA-Lab/PIXAR.
Read the full abstract ▾
All authors Xinyi Shang, Yi Tang, Jiacheng Cui, Ahmed Elhagry, Salwa K. Al Khatib, Sondos Mahmoud Bsharat, Jiacheng Liu, Xiaohan Zhao, Jing-Hao Xue, Hao Li, Salman Khan, Zhiqiang Shen
Machine Learning
Learning Sampled-data Control for Swarms via MeanFlow
Anqi Dong, Yongxin Chen, Karl H. Johansson et al. (+1 more)
Steering large-scale swarms with only limited control updates is often needed due to communication or computational constraints, yet most learning-based approaches do not account for this and instead model instantaneous velocity fields. As a result, the natural object for decision making is a finite-window control quantity rather than an infinitesimal one. To address this gap, we consider the recent machine learning framework MeanFlow and generalize it to the setting with general linear dynamic systems. This results in a new sampled-data learning framework that operates directly in control space and that can be applied for swarm steering. To this end, we learn the finite-horizon coefficient that parameterizes the minimum-energy control applied over each interval, and derive a differential identity that connects this quantity to a local bridge-induced supervision signal. This identity leads to a simple stop-gradient regression objective, allowing the interval coefficient field to be learned efficiently from bridge samples. The learned policy is deployed through sampled-data updates, guaranteeing that the resulting controller exactly respects the prescribed linear time-invariant dynamics and actuation channel. The resulting method enables few-step swarm steering at scale, while remaining consistent with the finite-window actuation structure of the underlying control system.
Read the full abstract ▾
All authors Anqi Dong, Yongxin Chen, Karl H. Johansson, Johan Karlsson
Machine Learning
Kolmogorov-Arnold causal generative models
Alejandro Almodóvar, Mar Elizo, Patricia A. Apellániz et al. (+2 more)
Causal generative models provide a principled framework for answering observational, interventional, and counterfactual queries from observational data. However, many deep causal models rely on highly expressive architectures with opaque mechanisms, limiting auditability in high-stakes domains. We propose KaCGM, a causal generative model for mixed-type tabular data where each structural equation is parameterized by a Kolmogorov–Arnold Network (KAN). This decomposition enables direct inspection of learned causal mechanisms, including symbolic approximations and visualization of parent–child relationships, while preserving query-agnostic generative semantics. We introduce a validation pipeline based on distributional matching and independence diagnostics of inferred exogenous variables, allowing assessment using observational data alone. Experiments on synthetic and semi-synthetic
benchmarks show competitive performance against state-of-the-art methods. A real-world cardiovascular case study further demonstrates the extraction of simplified structural equations and interpretable causal effects. These results suggest that expressive causal generative modeling and functional transparency can be achieved jointly, supporting trustworthy deployment in tabular decision-making settings. Code: https://github.com/aalmodovares/kacgm
Read the full abstract ▾
All authors Alejandro Almodóvar, Mar Elizo, Patricia A. Apellániz, Santiago Zazo, Juan Parras
Machine Learning · Cross-listed from hep-ex
AI Agents Can Already Autonomously Perform Experimental High Energy Physics
Eric A. Moreno, Samuel Bright-Thonney, Andrzej Novak et al. (+2 more)
Large
language model-based AI
agents are now able to autonomously execute substantial portions of a high energy physics (HEP) analysis pipeline with minimal expert-curated input. Given access to a HEP
dataset, an execution framework, and a corpus of prior experimental literature, we find that Claude Code succeeds in automating all stages of a typical analysis: event selection, background estimation, uncertainty quantification, statistical inference, and paper drafting. We argue that the experimental HEP community is underestimating the current capabilities of these systems, and that most proposed agentic workflows are too narrowly scoped or scaffolded to specific analysis structures. We present a proof-of-concept framework, Just Furnish Context (JFC), that integrates autonomous analysis
agents with literature-based knowledge
retrieval and multi-
agent review, and show that this is sufficient to plan, execute, and document a credible high energy physics analysis. We demonstrate this by conducting analyses on open data from ALEPH, DELPHI, and CMS to perform electroweak, QCD, and
Higgs boson measurements. We present two of those results in a condensed short paper form – a CMS Run1 Open Data $H\to \tau^+\tau^-$ to demonstrate performance on a well-established result, and the first Lund plane measurement on LEP data – a genuinely novel result and, to our knowledge, the first produced autonomously by an AI
agent. Rather than replacing physicists, these tools promise to offload the repetitive technical burden of analysis code development, freeing researchers to focus on physics insight, truly novel method development, and rigorous validation. Given these developments, we advocate for new strategies for how the community trains students, organizes analysis efforts, and allocates human expertise.
Read the full abstract ▾
All authors Eric A. Moreno, Samuel Bright-Thonney, Andrzej Novak, Dolores Garcia, Philip Harris
Machine Learning · Cross-listed from cs.CL
Measuring Faithfulness Depends on How You Measure: Classifier Sensitivity in LLM Chain-of-Thought Evaluation
Richard J. Young
Recent work on chain-of-thought (CoT) faithfulness reports single aggregate numbers (e.g., DeepSeek-R1 acknowledges hints 39% of the time), implying that faithfulness is an objective, measurable property of a model. This paper provides evidence that it is not. Three classifiers (a regex-only detector, a regex-plus-LLM pipeline, and a Claude Sonnet 4 judge) are applied to 10,276 influenced
reasoning traces from 12 open-weight models spanning 9 families and 7B to 1T parameters. On identical data, these classifiers produce faithfulness rates of 74.4%, 82.6%, and 69.7%. Per-model gaps range from 2.6 to 30.6 percentage points; all pairwise McNemar tests are significant (p < 0.001). The disagreements are systematic: Cohen's kappa ranges from 0.06 ("slight") for sycophancy hints to 0.42 ("moderate") for grader hints, and the asymmetry is pronounced: for sycophancy, 883 cases are classified as faithful by the pipeline but unfaithful by the Sonnet judge, while only 2 go the other direction. Classifier choice can also reverse model rankings: Qwen3.5-27B ranks 1st under the pipeline but 7th under Sonnet; OLMo-3.1-32B moves from 9th to 3rd. Different classifiers operationalize faithfulness at different levels of stringency (lexical mention versus epistemic dependence), yielding divergent measurements on the same behavior. These results indicate that published faithfulness numbers cannot be meaningfully compared across studies using different classifiers, and that future evaluations should report sensitivity ranges across multiple classification methodologies.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Semantic Token Clustering for Efficient Uncertainty Quantification in Large Language Models
Qi Cao, Andrew Gambardella, Takeshi Kojima et al. (+2 more)
Large
language models (LLMs) have demonstrated remarkable capabilities across diverse tasks. However, the truthfulness of their outputs is not guaranteed, and their tendency toward overconfidence further limits reliability. Uncertainty quantification offers a promising way to identify potentially unreliable outputs, but most existing methods rely on repeated sampling or auxiliary models, introducing substantial computational overhead. To address these limitations, we propose Semantic
Token Clustering (STC), an efficient uncertainty quantification method that leverages the semantic information inherently encoded in LLMs. Specifically, we group
tokens into semantically consistent clusters using
embedding clustering and prefix matching, and quantify uncertainty based on the probability mass aggregated over the corresponding semantic cluster. Our approach requires only a single generation and does not depend on auxiliary models. Experimental results show that STC achieves performance comparable to state-of-the-art baselines while substantially reducing computational overhead.
Read the full abstract ▾
All authors Qi Cao, Andrew Gambardella, Takeshi Kojima, Yutaka Matsuo, Yusuke Iwasawa
Machine Learning
Beyond Single Tokens: Distilling Discrete Diffusion Models via Discrete MMD
Emiel Hoogeboom, David Ruhe, Jonathan Heek et al. (+2 more)
It is currently difficult to distill discrete
diffusion models. In contrast, continuous
diffusion literature has many
distillation approaches methods that can reduce sampling steps to a handful. Our method, Discrete Moment Matching
Distillation (D-MMD), leverages ideas that have been highly successful in the continuous domain. Whereas previous discrete
distillation methods collapse, D-MMD maintains high quality and diversity (given sufficient sampling steps). This is demonstrated on both text and image
datasets. Moreover, the newly distilled generators can outperform their teachers.
Read the full abstract ▾
All authors Emiel Hoogeboom, David Ruhe, Jonathan Heek, Thomas Mensink, Tim Salimans
Machine Learning · Cross-listed from cs.CL
Enhancing Hyperspace Analogue to Language (HAL) Representations via Attention-Based Pooling for Text Classification
Ali Sakour and Zoalfekar Sakour
The Hyperspace Analogue to
Language (HAL) model relies on global word co-occurrence
matrices to construct distributional semantic representations. While these representations capture lexical relationships effectively, aggregating them into sentence-level
embeddings via standard mean pooling often results in information loss. Mean pooling assigns equal weight to all
tokens, thereby diluting the impact of contextually salient words with uninformative structural
tokens. In this paper, we address this limitation by integrating a learnable, temperature-scaled additive
attention mechanism into the HAL representation pipeline. To mitigate the sparsity and high dimensionality of the raw co-occurrence
matrices, we apply Truncated Singular Value Decomposition (SVD) to project the vectors into a dense latent space prior to the
attention layer. We evaluate the proposed architecture on the IMDB sentiment analysis
dataset. Empirical results demonstrate that the
attention-based pooling approach achieves a test accuracy of 82.38%, yielding an absolute improvement of 6.74 percentage points over the traditional mean pooling baseline (75.64%). Furthermore, qualitative analysis of the
attention weights indicates that the mechanism successfully suppresses stop-words and selectively attends to sentiment-bearing
tokens, improving both classification performance and model interpretability.
Read the full abstract ▾
Machine Learning
Revisiting Gene Ontology Knowledge Discovery with Hierarchical Feature Selection and Virtual Study Group of AI Agents
Cen Wan and Alex A. Freitas
Large
language models have achieved great success in multiple challenging tasks, and their capacity can be further boosted by the emerging agentic AI techniques. This new computing paradigm has already started revolutionising the traditional scientific discovery pipelines. In this work, we propose a novel agentic AI-based knowledge discovery-oriented virtual study group that aims to extract meaningful ageing-related biological knowledge considering highly ageing-related Gene Ontology terms that are selected by hierarchical feature selection methods. We investigate the performance of the proposed agentic AI framework by considering four different model organisms' ageing-related Gene Ontology terms and validate the biological findings by reviewing existing research articles. It is found that the majority of the AI
agent-generated scientific claims can be supported by existing literatures and the proposed internal mechanisms of the virtual study group also play an important role in the designed agentic AI-based knowledge discovery framework.
Read the full abstract ▾
Machine Learning
Conditioning Protein Generation via Hopfield Pattern Multiplicity
Jeffrey D. Varner
Small
protein-family
alignments often contain a subset of interest but not enough labeled data to train a conditional generator. We condition a training-free
stochastic-
attention sampler by adding one multiplicity ratio to its logits. Increasing this ratio shifts generation from the full family toward the designated subset. For unit-norm
memories, the resulting Boltzmann distribution is exactly a Gaussian mixture whose component weights are set by the multiplicities. This result separates exact conditioning in latent space from losses caused by sampling, PCA reconstruction, and sequence decoding. Across five Pfam families,
attention followed the analytic target, but recovery of single-residue markers depended on how well PCA separated the designated and background sequences. A matched weighted profile HMM reproduced these markers more directly, while
stochastic attention gave lower ESM2 pseudo-perplexity in the Kunitz comparison. Using a curated set of 23 omega-conotoxin sequences as the target subset produced diverse sequences that preserved the cysteine scaffold and Tyr13 and shifted other residues toward the designated set. These sequences are candidates for experimental testing; they do not establish binding.
Read the full abstract ▾
Machine Learning
Var-JEPA: A Variational Formulation of the Joint-Embedding Predictive Architecture - Bridging Predictive and Generative Self-Supervised Learning
Moritz Gögl and Christopher Yau
The Joint-
Embedding Predictive Architecture (JEPA) is often seen as a non-generative alternative to likelihood-based self-supervised learning, emphasizing prediction in representation space rather than reconstruction in observation space. We argue that the resulting separation from probabilistic generative modeling is largely rhetorical rather than structural: the canonical JEPA design (coupled encoders with a context-to-target predictor) mirrors the variational posteriors and learned conditional priors obtained when variational inference is applied to a particular class of coupled latent-variable models, and standard JEPA can be viewed as a deterministic specialization in which regularization is imposed via architectural and training heuristics rather than an explicit likelihood. Building on this view, we derive the Variational JEPA (Var-JEPA), which makes the latent generative structure explicit by optimizing a single Evidence Lower Bound (ELBO). This yields meaningful representations without ad-hoc anti-collapse regularizers and allows principled uncertainty quantification in the latent space. We instantiate the framework for tabular data (Var-T-JEPA) and achieve strong representation learning and downstream performance, improving over T-JEPA across real-world tabular
benchmarks while remaining competitive with strong raw-feature baselines.
Read the full abstract ▾
Machine Learning
GO-GenZip: Goal-Oriented Generative Sampling and Hybrid Compression
Pietro Talli, Qi Liao, Alessandro Lieto et al. (+3 more)
Current network data telemetry pipelines consist of massive streams of fine-grained Key Performance Indicators (KPIs) from multiple distributed sources towards central aggregators, making data storage, transmission, and real-time analysis increasingly unsustainable. This work presents a generative AI (GenAI)-driven sampling and hybrid compression framework that redesigns network telemetry from a goal-oriented perspective. Unlike conventional approaches that passively compress fully observed data, our approach jointly optimizes what to observe and how to encode it, guided by the relevance of information to downstream tasks. The framework integrates adaptive sampling policies, using adaptive masking techniques, with generative modeling to identify patterns and preserve critical features across temporal and spatial dimensions. The selectively acquired data are further processed through a hybrid compression scheme that combines traditional lossless coding with GenAI-driven, lossy compression. Experimental results on real network
datasets demonstrate over 50$\%$ reductions in sampling and data transfer costs, while maintaining comparable reconstruction accuracy and goal-oriented analytical fidelity in downstream tasks.
Read the full abstract ▾
All authors Pietro Talli, Qi Liao, Alessandro Lieto, Parijat Bhattacharjee, Federico Chiariotti, Andrea Zanella
Machine Learning
Trojan horse hunt in deep forecasting models: Insights from the European Space Agency competition
Krzysztof Kotowski, Ramez Shendy, Jakub Nalepa et al. (+10 more)
Forecasting plays a crucial role in modern safety-critical applications, such as space operations. However, the increasing use of deep
forecasting models introduces a new security risk of trojan horse attacks, carried out by hiding a
backdoor in the training data or directly in the model weights. Once implanted, the
backdoor is activated by a specific trigger pattern at test time, causing the model to produce manipulated predictions. We focus on this issue in our Trojan Horse Hunt data science competition, where more than 200 teams faced the task of identifying triggers hidden in deep
forecasting models for spacecraft telemetry. We describe the novel task formulation,
benchmark set, evaluation protocol, and best solutions from the competition. We further summarize key insights and research directions for effective identification of triggers in time series
forecasting models. All materials are publicly available on the official competition webpage https://www.kaggle.com/competitions/trojan-horse-hunt-in-space.
Read the full abstract ▾
All authors Krzysztof Kotowski, Ramez Shendy, Jakub Nalepa, Agata Kaczmarek, Dawid Płudowski, Piotr Wilczyński, Artur Janicki, Przemysław Biecek, Ambros Marzetta, Atul Pande, Lalit Chandra Routhu, Swapnil Srivastava, Evridiki Ntagiou
Machine Learning
The $\mathbf{Y}$-Combinator for LLMs: Solving Long-Context Rot with $\lambda$-Calculus
Amartya Roy, Rasul Tutunov, Xiaotong Ji et al. (+2 more)
LLMs are increasingly used as general-purpose reasoners, but long inputs remain bottlenecked by a fixed context window. Recursive
Language Models (RLMs) address this by externalising the
prompt and recursively solving subproblems. Yet existing RLMs depend on an open-ended read-eval-print loop (REPL) in which the model generates arbitrary control code, making execution difficult to verify, predict, and analyse. We introduce $\lambda$-RLM, a framework for long-context
reasoning that replaces free-form recursive code generation with a typed functional runtime grounded in $\lambda$-calculus. It executes a compact library of pre-verified combinators and uses neural inference only on bounded leaf subproblems, turning recursive
reasoning into a structured functional program with explicit control flow. We show that $\lambda$-RLM admits formal guarantees absent from standard RLMs, including termination, closed-form cost bounds, controlled accuracy scaling with recursion depth, and an optimal partition rule under a simple cost model. Empirically, across four long-context
reasoning tasks and nine base models, $\lambda$-RLM outperforms standard RLM in 29 of 36 model-task comparisons, improves average accuracy by up to +21.9 points across model tiers, and reduces latency by up to 4.1x. These results show that typed symbolic control yields a more reliable and efficient foundation for long-context
reasoning than open-ended recursive code generation. The complete implementation of $\lambda$-RLM, is open-sourced for the community at: https://github.com/lambda-calculus-LLM/lambda-RLM.
Read the full abstract ▾
All authors Amartya Roy, Rasul Tutunov, Xiaotong Ji, Matthieu Zimmer, Haitham Bou-Ammar
Machine Learning
Spectral Alignment in Forward-Backward Representations via Temporal Abstraction
Seyed Mahdi B. Azad, Jasper Hoffmann, Iman Nematollahi et al. (+3 more)
Forward-backward (FB) representations provide a powerful framework for learning the successor representation (SR) in continuous spaces by enforcing a low-rank factorization. However, a fundamental spectral mismatch often exists between the high-rank transition dynamics of continuous environments and the low-rank bottleneck of the FB architecture, making accurate low-rank representation learning difficult. In this work, we analyze temporal abstraction as a mechanism to mitigate this mismatch. By characterizing the spectral properties of the transition operator, we show that temporal abstraction acts analogously to a low-pass filter that suppresses high-frequency spectral components. This suppression reduces the effective rank of the induced SR while preserving a formal bound on the resulting value function error. Empirically, we show that this
alignment is a key factor for stable FB learning, particularly at high discount factors where bootstrapping becomes error-prone. Our results identify temporal abstraction as a principled mechanism for shaping the spectral structure of the underlying MDP and enabling effective long-horizon representations in continuous control.
Read the full abstract ▾
All authors Seyed Mahdi B. Azad, Jasper Hoffmann, Iman Nematollahi, Hao Zhu, Abhinav Valada, Joschka Boedecker
Machine Learning
How Out-of-Equilibrium Phase Transitions can Seed Pattern Formation in Trained Diffusion Models
Luca Ambrogioni
Diffusion models generate structure by progressively transforming noise into data, yet the mechanisms underlying this transition remain poorly understood. In this work, we show that pattern formation in trained
diffusion models can be explained as an out-of-
equilibrium phase transition driven by instabilities in the denoising dynamics. We develop a theoretical framework linking data
symmetries and architectural constraints, such as locality and translation equivariance, to the emergence of collective spatial modes. In this view, structure arises when low-frequency modes become unstable, triggering a rapid growth of spatial correlations that organizes noise into coherent patterns. We validate this theory through a combination of analytical models and experiments. In a controlled patch-based model, we observe a sharp softening of low-frequency modes in the theoretically predicted critical regime. Similar signatures are found in trained convolutional
diffusion models on Fashion-MNIST and in large-scale ImageNet models, where pattern formation coincides with a pronounced weakening of spatial modes. Finally, intervention experiments show that applying guidance precisely at this critical stage improves class
alignment compared to applying it at random times, demonstrating that this regime is not only descriptive but functionally important.
Read the full abstract ▾
Machine Learning · Cross-listed from quant-ph
Antenna Array Beamforming Based on a Hybrid Quantum Optimization Framework
Shuai Zeng
This paper proposes a hybrid
quantum optimization framework for large-scale
antenna-array beamforming with jointly optimized discrete phases and continuous amplitudes. The method combines
quantum-inspired search with classical gradient refinement to handle mixed discrete-continuous variables efficiently. For phase optimization, a Gray-code and odd-combination encoding scheme is introduced to improve robustness and avoid the complexity explosion of higher-order Ising models. For amplitude optimization, a
geometric spin-combination encoding and a two-stage strategy are developed, using
quantum-inspired optimization for coarse search and gradient optimization for fine refinement. To enhance solution diversity and quality, a rainbow
quantum-inspired
algorithm integrates multiple optimizers for parallel exploration, followed by hierarchical-clustering-based candidate refinement. In addition, a double outer-product method and an augmented version are proposed to construct the coupling
matrix and bias vector efficiently, improving numerical precision and implementation efficiency. Under the scoring rules of the 7th National
Quantum Computing Hackathon, simulations on a 32-element
antenna array show that the proposed method achieves a score of 461.58 under constraints on near-main-lobe sidelobes, wide-angle sidelobes, beamwidth, and optimization time, nearly doubling the baseline score. The proposed framework provides an effective reference for beamforming optimization in future wireless communication systems.
Read the full abstract ▾
Machine Learning
Fine-tuning Timeseries Predictors Using Reinforcement Learning
Hugo Cazaux, Ralph Rudd, Hlynur Stefánsson et al. (+2 more)
This chapter presents three major reinforcement learning
algorithms used for fine-tuning financial forecasters. We propose a clear implementation plan for backpropagating the loss of a reinforcement learning task to a model trained using supervised learning, and compare the performance before and after the fine-tuning. We find an increase in performance after fine-tuning, and transfer learning properties to the models, indicating the benefits of fine-tuning. We also highlight the tuning process and empirical results for future implementation by practitioners.
Read the full abstract ▾
All authors Hugo Cazaux, Ralph Rudd, Hlynur Stefánsson, Sverrir Ólafsson, Eyjólfur Ingi Ásgeirsson
Machine Learning · Cross-listed from eess.SP
Structured Latent Dynamics in Wireless CSI via Homomorphic World Models
Salmane Naoumi, Mehdi Bennis and Marwa Chafii
We introduce a self-supervised framework for learning predictive and structured representations of wireless channels by modeling the temporal
evolution of channel state information (CSI) in a compact latent space. Our method casts the problem as a world modeling task and leverages the Joint
Embedding Predictive Architecture (JEPA) to learn action-conditioned latent dynamics from CSI trajectories. To promote
geometric consistency and compositionality, we parameterize transitions using homomorphic updates derived from Lie
algebra, yielding a structured latent space that reflects spatial layout and user motion. Evaluations on the DICHASUS
dataset show that our approach outperforms strong baselines in preserving
topology and
forecasting future
embeddings across unseen environments. The resulting latent space enables metrically faithful channel charts, offering a scalable foundation for downstream applications such as mobility-aware scheduling, localization, and wireless scene understanding.
Read the full abstract ▾
Machine Learning
Federated Hyperdimensional Computing for Resource-Constrained Industrial IoT
Nikita Zeulin, Olga Galinina, Nageen Himayat et al. (+1 more)
In the Industrial Internet of Things (IIoT) systems, edge devices often operate under strict constraints in
memory, compute capability, and wireless bandwidth. These limitations challenge the deployment of advanced data analytics tasks, such as predictive and prescriptive maintenance. In this work, we explore hyperdimensional computing (HDC) as a lightweight learning paradigm for resource-constrained IIoT. Conventional centralized HDC leverages the properties of high-dimensional vector spaces to enable energy-efficient training and inference. We integrate this paradigm into a
federated learning (FL) framework where devices exchange only prototype representations, which significantly reduces communication overhead. Our numerical results highlight the potential of
federated HDC to support collaborative learning in IIoT with fast convergence speed and communication efficiency. These results indicate that HDC represents a lightweight and resilient framework for distributed intelligence in large-scale and resource-constrained IIoT environments.
Read the full abstract ▾
All authors Nikita Zeulin, Olga Galinina, Nageen Himayat, Sergey Andreev
Machine Learning
Continual Learning as Shared-Manifold Continuation Under Compatible Shift
Henry J. Kobs
Continual learning methods usually preserve old behavior by regularizing parameters, matching old outputs, or replaying previous examples. These strategies can reduce forgetting, but they do not directly specify how the latent representation should evolve. We study a narrower
geometric alternative for the regime where old and new data should remain on the same latent support: continual learning as continuation of a shared
manifold. We instantiate this view within Support-Preserving
Manifold Assimilation (SPMA) and evaluate a geometry-preserving variant, SPMA-OG, that combines sparse replay, output
distillation, relational geometry preservation, local smoothing, and chart-assignment regularization on old anchors. On representative compatible-shift CIFAR10 and Tiny-ImageNet runs, SPMA-OG improves over sparse replay baselines in old-task retention and representation-preservation metrics while remaining competitive on new-task accuracy. On a controlled synthetic atlas-
manifold benchmark, it achieves near-perfect anchor-geometry preservation while also improving new-task accuracy over replay. These results provide evidence that geometry-aware anchor regularization is a useful inductive bias when continual learning should preserve a shared latent support rather than create a new one.
Read the full abstract ▾