Machine Learning · Cross-listed from eess.IV
Trajectory Stitching for Solving Inverse Problems with Flow-Based Models
Alexander Denker, Zeljko Kereta, Carola-Bibiane Schönlieb et al. (+1 more)
Flow-based generative models have emerged as powerful priors for solving inverse problems. One option is to directly optimize the initial latent code (noise), such that the flow output solves the inverse problem. However, this requires backpropagating through the entire generative trajectory, incurring high
memory costs and numerical instability. We propose MS-Flow, which represents the trajectory as a sequence of intermediate latent states rather than a single initial code. By enforcing the flow dynamics locally and coupling segments through trajectory-matching penalties, MS-Flow alternates between updating intermediate latent states and enforcing consistency with observed data. This reduces
memory consumption while improving reconstruction quality. We demonstrate the effectiveness of MS-Flow over existing methods on image recovery and inverse problems, including inpainting, super-resolution, and computed
tomography.
Read the full abstract ▾
All authors Alexander Denker, Zeljko Kereta, Carola-Bibiane Schönlieb, Moshe Eliasof
Machine Learning
Causal Schrödinger Bridges: Constrained Optimal Transport on Structural Manifolds
Rui Wu and Li YongJun
Generative modeling typically seeks the path of least action via deterministic flows (ODE). While effective for in-distribution tasks, we argue that these deterministic paths become brittle under causal interventions, which often require transporting probability mass across low-density regions ("off-
manifold") where the vector field is ill-defined. This leads to numerical instability and the pathology of anticipatory control. In this work, we introduce the Causal Schrodinger Bridge (CSB), a framework that reformulates counterfactual inference as Entropic Optimal Transport. By leveraging
diffusion processes (SDEs), CSB enables probability mass to robustly "tunnel" through support mismatches while strictly enforcing structural admissibility. We prove the Structural Decomposition
Theorem, showing that the global high-dimensional bridge factorizes exactly into local, robust transitions. This
theorem provides a principled resolution to the Information Bottleneck that plagues monolithic architectures in high dimensions. We empirically validate CSB on a full-rank causal system (d=10^5, intrinsic rank 10^5), where standard structure-blind MLPs fail to converge (MSE ~0.31). By physically implementing the structural decomposition, CSB achieves high-fidelity transport (MSE ~0.06) in just 73.73 seconds on a single GPU. This stands in stark contrast to structure-agnostic O(d^3) baselines, estimated to require over 6 years. Our results demonstrate that CSB breaks the Curse of Dimensionality through structural intelligence, offering a scalable foundation for high-stakes causal discovery in 10^5-node systems. Code is available at: https://github.com/cochran1/causal-schrodinger-bridge
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
Reinforcement Inference: Leveraging Uncertainty for Self-Correcting Language Model Reasoning
Xinhai Sun
Modern large
language models (LLMs) are often evaluated and deployed under a one-shot, greedy inference protocol, especially in professional settings that require deterministic behavior. This regime can systematically under-estimate a fixed model's true capability: many errors arise not from missing knowledge, but from premature commitment under internal ambiguity. We introduce Reinforcement Inference, an
entropy-aware inference-time control strategy that uses the model's own uncertainty to selectively invoke a second, more deliberate
reasoning attempt, enabling stronger performance without any retraining. On 12,032 MMLU-Pro questions across 14 subjects, using DeepSeek-v3.2 with deterministic decoding in a zero-shot setting, Reinforcement Inference improves accuracy from 60.72% to 84.03%, while only incurring 61.06% additional inference calls. A 100% re-asking ablation reaches 84.35%, indicating that uncertainty-aware selection captures most of the attainable improvement with substantially less compute. Moreover, a
prompt-only ablation underperforms the baseline, suggesting that the gains are not explained by generic prompting alone. Beyond providing a practical inference-time upgrade, our results suggest a broader
entropy-aware paradigm for measuring and expanding model capability: because modern decoder-based models generate outputs autoregressively,
entropy and related confidence measures arise naturally as first-class control signals during generation. The resulting gap between one-pass greedy inference and uncertainty-conditioned deliberation offers a diagnostic lens on an LLM's latent
reasoning horizon and motivates future training objectives that explicitly constrain correctness–confidence
alignment.
Read the full abstract ▾
Machine Learning
Bridging Academia and Industry: A Comprehensive Benchmark for Attributed Graph Clustering
Yunhui Liu, Pengyu Qiu, Yu Xing et al. (+6 more)
Attributed Graph Clustering (AGC) is a fundamental unsupervised task that integrates structural
topology and node attributes to uncover latent patterns in graph-structured data. Despite its significance in industrial applications such as fraud detection and user
segmentation, a significant chasm persists between academic research and real-world deployment. Current evaluation protocols suffer from the small-scale, high-homophily citation
datasets, non-scalable full-batch training paradigms, and a reliance on supervised metrics that fail to reflect performance in label-scarce environments. To bridge these gaps, we present PyAGC, a comprehensive, production-ready
benchmark and library designed to stress-test AGC methods across diverse scales and structural properties. We unify existing methodologies into a modular Encode-Cluster-Optimize framework and, for the first time, provide
memory-efficient, mini-batch implementations for a wide array of state-of-the-art AGC
algorithms. Our
benchmark curates 12 diverse
datasets, ranging from 2.7K to 111M nodes, specifically incorporating industrial graphs with complex tabular features and low homophily. Furthermore, we advocate for a holistic evaluation protocol that mandates unsupervised structural metrics and efficiency profiling alongside traditional supervised metrics. Battle-tested in high-stakes industrial workflows at Ant Group, this
benchmark offers the community a robust, reproducible, and scalable platform to advance AGC research towards realistic deployment. The code and resources are publicly available via GitHub (https://github.com/Cloudy1225/PyAGC), PyPI (https://pypi.org/project/pyagc), and Documentation (https://pyagc.readthedocs.io). We invite readers to our latest survey (https://arxiv.org/abs/2603.20829) for the unified framework and industrial perspective that complement this
benchmark.
Read the full abstract ▾
All authors Yunhui Liu, Pengyu Qiu, Yu Xing, Yongchao Liu, Peng Du, Chuntao Hong, Jiajun Zheng, Tao Zheng, Tieke He
Machine Learning · Cross-listed from math.NA
Do physics-informed neural networks (PINNs) need to be deep? Shallow PINNs using the Levenberg-Marquardt algorithm
Muhammad Luthfi Shahab, Imam Mukhlash and Hadi Susanto
This work investigates shallow physics-informed neural networks (PINNs) for solving forward and inverse problems governed by nonlinear partial differential equations (PDEs). By formulating PINN training as a nonlinear least-squares problem, the Levenberg-Marquardt (LM)
algorithm is used to efficiently optimize the network parameters. Exact analytical expressions for neural-network derivatives with respect to the input variables are derived, revealing the relationships between the network output and its spatial and temporal derivatives and providing a clearer interpretation of the PINN architecture. These expressions are then used to derive explicit formulas for the Jacobian
matrix required by LM. The proposed approach is evaluated on the Burgers, Schrödinger, Allen-Cahn, and three-dimensional Bratu equations. Numerical results show that LM substantially outperforms BFGS, L-BFGS, and Adam in convergence speed, accuracy, and final loss values. Comparisons with deeper networks further demonstrate that shallow LM-PINNs can achieve higher accuracy with substantially fewer parameters, emphasizing the importance of considering network architecture and optimization strategy jointly. The explicit analytical Jacobian also provides computational and
memory advantages that are particularly relevant to large-scale PINNs. Overall, these results suggest that, for a broad class of PDEs, shallow PINNs combined with effective second-order optimization can provide accurate and computationally efficient solutions to both forward and inverse problems.
Read the full abstract ▾
Machine Learning · Cross-listed from physics.app-ph
Estimation of Fish Catch Using Sentinel-2, 3 and XGBoost-Kernel-Based Kernel Ridge Regression
Kanu Mohammed, Vaishnavi Joshi, Pranjali Diliprao Patil et al. (+3 more)
Oceanographic factors, such as sea surface temperature and upper-ocean dynamics, have a significant impact on fish distribution. Maintaining fisheries that contribute to global food security requires quantifying these connections. This study uses multispectral images from Sentinel-2 MSI and Sentinel-3 OLCI to estimate fish catch using an Extreme Gradient Boosting (XGBoost)-kernelized Kernel Ridge Regression (KRR) technique. According to model evaluation, the XGBoost-KRR framework achieves the strongest correlation and the lowest prediction error across both sensors, suggesting improved capacity to capture nonlinear ocean-fish connections. While Sentinel-2 MSI resolves finer-scale spatial variability, emphasizing localized ecological interactions, Sentinel-3 OLCI displays smoother spectral responses associated with poorer spatial resolution. By supporting sustainable ecosystem management and strengthening satellite-based fisheries assessment, the proposed approach advances SDGs 2 (Zero Hunger) and 14 (Life Below Water).
Read the full abstract ▾
All authors Kanu Mohammed, Vaishnavi Joshi, Pranjali Diliprao Patil, Sandipan Mondal, Ming-An Lee, Subhadip Dey
Machine Learning · Cross-listed from cs.CV
Learning Self-Correction in Vision-Language Models via Rollout Augmentation
Yi Ding, Ziliang Qiu, Bolian Li et al. (+1 more)
Self-correction is essential for solving complex
reasoning problems in
vision-
language models (VLMs). However, existing reinforcement learning (RL) methods struggle to learn it, as effective self-correction behaviors emerge only rarely, making learning signals extremely sparse. To address this challenge, we propose correction-specific rollouts (Octopus), an RL rollout augmentation framework that synthesizes dense self-correction examples by recombining existing rollouts. This augmentation simultaneously improves sample efficiency due to rollout reuse and stabilizes RL optimization through balanced supervision. Furthermore, we introduce a response-masking strategy that decouples self-correction from direct
reasoning, avoiding signal conflicts and enabling both behaviors to be learned effectively. Building on this, we introduce Octopus-8B, a
reasoning VLM with controllable self-correction capability. Across 7
benchmarks, it achieves SoTA performance among open-source VLMs, outperforming the best RLVR baseline by 1.0 score while requiring only $0.72\times$ training time per step.
Read the full abstract ▾
All authors Yi Ding, Ziliang Qiu, Bolian Li, Ruqi Zhang
Machine Learning
Is Meta-Path Attention an Explanation? Evidence of Alignment and Decoupling in Heterogeneous GNNs
Maiqi Jiang, Noman Ali, Yiran Ding et al. (+1 more)
Meta-path-based heterogeneous graph neural networks aggregate over meta-path-induced views, and their semantic-level
attention over meta-path channels is widely used as a narrative for “which semantics matter.” We study this assumption empirically by asking: when does meta-path
attention reflect meta-path importance, and when can it decouple? A key challenge is that most post-hoc GNN explainers are designed for homogeneous graphs, and naive adaptations to heterogeneous neighborhoods can mix semantics and confound perturbations. To enable a controlled empirical analysis, we introduce MetaXplain, a meta-path-aware post-hoc explanation protocol that applies existing explainers in the native meta-path view domain via (i) view-factorized explanations, (ii) schema-valid channel-wise perturbations, and (iii) fusion-aware attribution, without modifying the underlying predictor. We
benchmark representative gradient-, perturbation-, and Shapley-style explainers on ACM, DBLP, and IMDB with HAN and HAN-GCN, comparing against xPath and type-matched random baselines under standard faithfulness metrics. To quantify
attention reliability, we propose Meta-Path
Attention–Explanation
Alignment (MP-AEA), which measures rank correlation between learned
attention weights and explanation-derived meta-path contribution scores across random runs. Our results show that meta-path-aware explanations typically outperform random controls, while MP-AEA reveals both high-
alignment and statistically significant decoupling regimes depending on the
dataset and backbone; moreover, retraining on explanation-induced subgraphs often preserves, and in some noisy regimes improves, predictive performance, suggesting an explanation-as-denoising effect.
Read the full abstract ▾
All authors Maiqi Jiang, Noman Ali, Yiran Ding, Yanfu Zhang
Machine Learning
Contextual Rollout Bandits for Reinforcement Learning with Verifiable Rewards
Xiaodong Lu, Xiaohan Wang, Jiajun Chai et al. (+7 more)
Reinforcement Learning with Verifiable Rewards (RLVR) is an effective paradigm for improving the
reasoning capabilities of large
language models. However, existing RLVR methods utilize rollouts in an indiscriminate and short-horizon manner: responses of heterogeneous quality within each
prompt are treated uniformly, and historical rollouts are discarded after a single use. This leads to noisy supervision, poor sample efficiency, and suboptimal policy updates. We address these issues by formulating rollout scheduling in RLVR as a contextual bandit problem and proposing a unified neural scheduling framework that adaptively selects high-value rollouts throughout training. Each rollout is treated as an arm whose reward is defined by the induced performance gain between consecutive optimization steps. The resulting scheduler supports both noise-aware intra-group selection and adaptive global reuse of historical rollouts within a single principled framework. We provide theoretical justification by deriving sublinear regret bounds and showing that enlarging the rollout buffer improves the achievable performance upper bound. Experiments on six mathematical
reasoning benchmarks demonstrate consistent gains in performance and training efficiency across multiple RLVR optimization methods.
Read the full abstract ▾
All authors Xiaodong Lu, Xiaohan Wang, Jiajun Chai, Guojun Yin, Wei Lin, Zhijun Chen, Yu Luo, Fuzhen Zhuang, Yikun Ban, Deqing Wang
Machine Learning · Cross-listed from cs.CV
Understanding Image2Video Domain Shift in Food Segmentation: An Instance-level Analysis on Apples
Keonvin Park, Aditya Pal and Jin Hong Mok
Food
segmentation models trained on static images have achieved strong performance on
benchmark datasets; however, their reliability in
video settings remains poorly understood. In real-world applications such as food monitoring and instance counting,
segmentation outputs must be temporally consistent, yet image-trained models often break down when deployed on
videos. In this work, we analyze this failure through an instance
segmentation and tracking perspective, focusing on apples as a representative food category. Models are trained solely on image-level food
segmentation data and evaluated on
video sequences using an instance
segmentation with tracking-by-matching framework, enabling object-level temporal analysis. Our results reveal that high frame-wise
segmentation accuracy does not translate to stable instance identities over time. Temporal appearance variations, particularly illumination changes, specular reflections, and texture ambiguity, lead to mask flickering and identity fragmentation, resulting in significant errors in apple counting. These failures are largely overlooked by conventional image-based metrics, which substantially overestimate real-world
video performance. Beyond diagnosing the problem, we examine practical remedies that do not require full
video supervision, including post-hoc temporal regularization and self-supervised temporal consistency objectives. Our findings suggest that the root cause of failure lies in image-centric training objectives that ignore temporal coherence, rather than model capacity. This study highlights a critical evaluation gap in food
segmentation research and motivates temporally-aware learning and evaluation protocols for
video-based food analysis.
Read the full abstract ▾
Machine Learning
Beyond Correctness: Learning Robust Reasoning via Transfer
Hyunseok Lee, Soheil Abbasloo, Jihoon Tack et al. (+1 more)
Reinforcement Learning with Verifiable Rewards (RLVR) has recently strengthened LLM
reasoning, but its focus on final answer correctness leaves a critical gap: it does not ensure the robustness of the
reasoning process itself. We adopt a simple philosophical view, robust
reasoning should remain useful beyond the mind that produced it, and treat
reasoning as a form of meaning transfer that must survive truncation, reinterpretation, and continuation. Building on this principle, we introduce Reinforcement Learning with Transferable Reward (RLTR), which operationalizes robustness via transfer reward that tests whether a partial
reasoning prefix from one model can guide a separate model to the correct answer. This encourages LLMs to produce
reasoning that is stable, interpretable, and genuinely generalizable. Our approach improves sampling consistency while improving final answer accuracy, and it reaches comparable performance in substantially fewer training steps. For example, on MATH500, RLTR achieves a +3.6%p gain in Maj@64 compared to RLVR and matches RLVR's average accuracy with roughly 2.5x fewer training steps, providing both more reliable
reasoning and significantly more sample efficient.
Read the full abstract ▾
All authors Hyunseok Lee, Soheil Abbasloo, Jihoon Tack, Jinwoo Shin
Machine Learning · Cross-listed from quant-ph
Empirical Study of Observable Sets in Multiclass Quantum Classification
Paul San Sebastian, Mikel Cañizo and Roman Orus
Variational
quantum algorithms have gained
attention as early applications of
quantum computers for learning tasks. In the context of supervised learning, most of the works that tackle classification problems with parameterized
quantum circuits constrain their scope to the setting of binary classification or perform multiclass classification via ensembles of binary classifiers (strategies such as one versus rest). Those few works that propose native multiclass models, however, do not justify the choice of observables that perform the classification. This work studies two main classification criteria in multiclass
quantum machine learning: maximizing the expected value of an observable representing a class or maximizing the fidelity of the encoded
quantum state with a reference state representing a class. To compare both approaches, sets of Pauli strings and sets of projectors into the computational basis are chosen as observables in the
quantum machine learning models. Observing the empirical behavior of each model type, the effect of different observable set choices on the performance of
quantum machine learning models is analyzed in the context of Barren Plateaus and Neural Collapse. The results provide insights that may guide the design of future multiclass
quantum machine learning models.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Gesture Matters: Pedestrian Gesture Recognition for AVs Through Skeleton Pose Evaluation
Alif Rizqullah Mahdi, Mahdi Rezaei and Natasha Merat
Gestures are a key component of non-verbal communication in traffic, often helping pedestrian-to-driver interactions when formal traffic rules may be insufficient. This problem becomes more apparent when autonomous vehicles (AVs) struggle to interpret such gestures. In this study, we present a gesture classification framework using 2D pose estimation applied to real-world
video sequences from the WIVW
dataset. We categorise gestures into four primary classes (Stop, Go, Thank & Greet, and No Gesture) and extract 76 static and dynamic features from normalised keypoints. Our analysis demonstrates that hand position and movement velocity are especially discriminative in distinguishing between gesture classes, achieving a classification accuracy score of 87%. These findings not only improve the perceptual capabilities of AV systems but also contribute to the broader understanding of pedestrian behaviour in traffic contexts.
Read the full abstract ▾
Machine Learning
Time-Delayed Transformers for Data-Driven Modeling of Low-Dimensional Dynamics
Albert Alcalde, Markus Widhalm and Emre Y{\i}lmaz
We propose the time-delayed
transformer (TD-TF), a simplified
transformer architecture for data-driven modeling of unsteady spatio-temporal dynamics. TD-TF bridges linear operator-based methods and deep sequence models by showing that a single-layer, single-head
transformer can be interpreted as a nonlinear generalization of time-delayed dynamic mode decomposition (TD-DMD). The architecture is deliberately minimal, consisting of one self-
attention layer with a single query per prediction and one feedforward layer, resulting in linear computational complexity in sequence length and a small parameter count. Numerical experiments demonstrate that TD-TF matches the performance of strong linear baselines on near-linear systems, while significantly outperforming them in nonlinear and chaotic regimes, where it accurately captures long-term dynamics. Validation studies on synthetic signals, unsteady aerodynamics, the Lorenz '63 system, and a reaction-
diffusion model show that TD-TF preserves the interpretability and efficiency of linear models while providing substantially enhanced expressive power for complex dynamics.
Read the full abstract ▾
Machine Learning
Learning Credal Ensembles via Distributionally Robust Optimization
Kaizheng Wang, Ghifari Adam Faza, Fabio Cuzzolin et al. (+3 more)
Credal predictors are models that are aware of epistemic uncertainty and produce a convex set of probabilistic predictions. They offer a principled way to quantify predictive epistemic uncertainty (EU) and have been shown to improve model robustness in various settings. However, most state-of-the-art methods mainly define EU as disagreement caused by random training initializations, which mostly reflects sensitivity to optimization randomness rather than uncertainty from deeper sources. To address this, we define EU as disagreement among models trained with varying relaxations of the i.i.d. assumption between training and test data. Based on this idea, we propose CreDRO, which learns an ensemble of plausible models through distributionally robust optimization. As a result, CreDRO captures EU not only from training randomness but also from meaningful disagreement due to potential distribution shifts between training and test data. Empirical results show that CreDRO consistently outperforms existing credal methods on tasks such as out-of-distribution detection across multiple
benchmarks and selective classification in medical applications.
Read the full abstract ▾
All authors Kaizheng Wang, Ghifari Adam Faza, Fabio Cuzzolin, Siu Lun Chau, David Moens, Hans Hallez
Machine Learning
Low Rank Transformer for Multivariate Time Series Anomaly Detection and Localization
Charalampos Shimillas, Kleanthis Malialis, Konstantinos Fokianos et al. (+1 more)
Multivariate time series (MTS) anomaly diagnosis, which encompasses both anomaly detection and localization, is critical for the safety and reliability of complex, large-scale real-world systems. The vast majority of existing anomaly diagnosis methods offer limited theoretical insights, especially for anomaly localization, which is a vital but largely unexplored area. The aim of this contribution is to study the learning process of a
Transformer when applied to MTS by revealing connections to statistical time series methods. Based on these theoretical insights, we propose the
Attention Low-Rank
Transformer (ALoRa-T) model, which applies low-rank regularization to self-
attention, and we introduce the
Attention Low-Rank score, effectively capturing the temporal characteristics of anomalies. Finally, to enable anomaly localization, we propose the ALoRa-Loc method, a novel approach that associates anomalies to specific variables by quantifying interrelationships among time series. Extensive experiments and real data analysis, show that the proposed methodology significantly outperforms state-of-the-art methods in both detection and localization tasks.
Read the full abstract ▾
All authors Charalampos Shimillas, Kleanthis Malialis, Konstantinos Fokianos, Marios M. Polycarpou
Machine Learning
Estimating Aleatoric Uncertainty in the Causal Treatment Effect
Liyuan Xu and Bijan Mazaheri
Previous work on causal inference has primarily focused on averages and conditional averages of treatment effects, with significantly less
attention on variability and uncertainty in individual treatment responses. In this paper, we introduce the variance of the treatment effect (VTE) and conditional variance of treatment effect (CVTE) as the natural measure of aleatoric uncertainty inherent in treatment responses, and we demonstrate that these quantities are identifiable from observed data under mild assumptions, even in the presence of unobserved confounders. We further propose nonparametric kernel-based
estimators for VTE and CVTE, and our theoretical analysis establishes their convergence. We also test the performance of our method through extensive empirical experiments on both synthetic and semi-simulated
datasets, where it demonstrates superior or comparable performance to naive baselines.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
When Evaluation Becomes a Side Channel: Regime Leakage and Structural Mitigations for Alignment Assessment
Igor Santos-Grueiro
Safety evaluation for advanced AI systems assumes that behavior observed under evaluation predicts behavior in deployment. This assumption weakens for
agents with situational awareness, which may exploit regime leakage, cues distinguishing evaluation from deployment, to implement conditional policies that comply under oversight while defecting in deployment-like regimes. We recast
alignment evaluation as a problem of information flow under partial observability and show that divergence between evaluation-time and deployment-time behavior is bounded by the regime information extractable from decision-relevant internal representations. We study regime-blind mechanisms, training-time interventions that restrict access to regime cues through
adversarial invariance constraints without assuming complete information erasure. We evaluate this approach across multiple open-weight
language models and controlled failure modes including scientific sycophancy, temporal sleeper
agents, and data leakage. Regime-blind training reduces regime-conditioned failures without measurable loss of task utility, but exhibits heterogeneous and model-dependent dynamics. Sycophancy shows a sharp representational and behavioral transition at moderate intervention strength, consistent with a stability cliff. In sleeper-style constructions and certain cross-model replications, suppression occurs without a clean collapse of regime decodability and may display non-monotone or oscillatory behavior as invariance pressure increases. These findings indicate that representational invariance is a meaningful but limited control lever. It can raise the cost of regime-conditioned strategies but cannot guarantee elimination or provide architecture-invariant thresholds. Behavioral evaluation should therefore be complemented with white-box diagnostics of regime awareness and internal information flow.
Read the full abstract ▾
Machine Learning
RIFLE: Robust Distillation-based FL for Deep Model Deployment on Resource-Constrained IoT Networks
Pouria Arefijamal, Mahdi Ahmadlou, Bardia Safaei et al. (+1 more)
Federated learning (FL) is a decentralized learning paradigm widely adopted in resource-constrained Internet of Things (IoT) environments. These devices, typically relying on TinyML models, collaboratively train global models by sharing gradients with a central server while preserving data
privacy. However, as data heterogeneity and task complexity increase, TinyML models often become insufficient to capture intricate patterns, especially under extreme non-IID (non-independent and identically distributed) conditions. Moreover, ensuring robustness against malicious clients and poisoned updates remains a major challenge. Accordingly, this paper introduces RIFLE - a Robust,
distillation-based
Federated Learning framework that replaces gradient sharing with logit-based knowledge transfer. By leveraging a knowledge
distillation aggregation scheme, RIFLE enables the training of deep models such as VGG-19 and Resnet18 within constrained IoT systems. Furthermore, a Kullback-Leibler (KL) divergence-based validation mechanism quantifies the reliability of client updates without exposing raw data, achieving high trust and
privacy preservation simultaneously. Experiments on three
benchmark datasets (MNIST, CIFAR-10, and CIFAR-100) under heterogeneous non-IID conditions demonstrate that RIFLE reduces false-positive detections by up to 87.5%, enhances poisoning attack mitigation by 62.5%, and achieves up to 28.3% higher accuracy compared to conventional
federated learning baselines within only 10 rounds. Notably, RIFLE reduces VGG19 training time from over 600 days to just 1.39 hours on typical IoT devices (0.3 GFLOPS), making deep learning practical in resource-constrained networks.
Read the full abstract ▾
All authors Pouria Arefijamal, Mahdi Ahmadlou, Bardia Safaei, Jörg Henkel
Machine Learning
USBD: Universal Structural Basis Distillation for Source-Free Graph Domain Adaptation
Yingxu Wang, Kunyu Zhang, Mengzhu Wang et al. (+2 more)
SF-GDA is pivotal for
privacy-preserving knowledge transfer across graph
datasets. Although recent works incorporate structural information, they implicitly condition adaptation on the smoothness priors of sourcetrained GNNs, thereby limiting their generalization to structurally distinct targets. This dependency becomes a critical bottleneck under significant topological shifts, where the source model misinterprets distinct topological patterns unseen in the source domain as noise, rendering pseudo-label-based adaptation unreliable. To overcome this limitation, we propose the Universal Structural Basis
Distillation, a framework that shifts the paradigm from adapting a biased model to learning a universal structural basis for SF-GDA. Instead of adapting a biased source model to a specific target, our core idea is to construct a structure-agnostic basis that proactively covers the full spectrum of potential topological patterns. Specifically, USBD employs a bi-level optimization framework to distill the source
dataset into a compact structural basis. By enforcing the prototypes to span the full Dirichlet energy spectrum, the learned basis explicitly captures diverse topological motifs, ranging from low-frequency clusters to high-frequency chains, beyond those present in the source. This ensures that the learned basis creates a comprehensive structural covering capable of handling targets with disparate structures. For inference, we introduce a spectral-aware ensemble mechanism that dynamically activates the optimal prototype combination based on the spectral fingerprint of the target graph. Extensive experiments on
benchmarks demonstrate that USBD significantly outperforms state-of-the-art methods, particularly in scenarios with severe structural shifts, while achieving superior computational efficiency by decoupling the adaptation cost from the target data scale.
Read the full abstract ▾
All authors Yingxu Wang, Kunyu Zhang, Mengzhu Wang, Siyang Gao, Nan Yin
Machine Learning
The Connection between Kriging and Large Neural Networks
Marius Marinescu
AI has impacted many disciplines and is nowadays ubiquitous. In particular, spatial statistics is in a pivotal moment where it will increasingly intertwine with AI. In this scenario, a relevant question is what relationship spatial statistics models have with machine learning (ML) models, if any. In particular, in this paper, we explore the connections between Kriging and neural networks. At first glance, they may appear unrelated. Kriging - and its ML counterpart, Gaussian process regression - are grounded in probability theory and
stochastic processes, whereas many ML models are extensively considered Black-Box models. Nevertheless, they are strongly related. We study their connections and revisit the relevant literature. The understanding of their relations and the combination of both perspectives may enhance ML techniques by making them more interpretable, reliable, and spatially aware.
Read the full abstract ▾
Machine Learning
Radial Müntz-Szász Networks: Neural Architectures with Learnable Power Bases for Multidimensional Singularities
Gnankan Landry Regis N'guessan and Bum Jun Kim
Radial singular fields, such as $1/r$, $\log r$, and crack-tip profiles, are difficult to model with current coordinate-separable neural architectures. We formally establish this result: any $C^2$ function that is both radial and additively separable must be quadratic, establishing a fundamental obstruction for coordinate-wise power-law models. Motivated by this result, we introduce Radial Müntz-Szász Networks (RMN), which represent fields as linear combinations of learnable radial powers $r^\mu$, including negative exponents, together with a limit-stable log-primitive for exact $\log r$ behavior. RMN admits closed-form spatial gradients and
Laplacians, enabling physics-informed learning on punctured domains. Across ten 2D and 3D
benchmarks, RMN achieves between 1.5 and 51 times lower RMSE than MLPs and between 10 and 100 times lower RMSE than SIREN, while using only 27 parameters, compared with 33,537 for MLPs and 8,577 for SIREN. We extend RMN to incorporate angular dependence (RMN-Angular) and to handle multiple sources with learnable centers (RMN-MC), whose source-center recovery errors fall below $10^{-4}$. We also report controlled failures on smooth, strongly non-radial targets to delineate RMN's operating regime.
Read the full abstract ▾
Machine Learning
Drop the mask! GAMM-A Taxonomy for Graph Attributes Missing Mechanisms
Richard Serrano, Baptiste Jeudy, Charlotte Laclau et al. (+1 more)
Exploring missing data in attributed graphs introduces unique challenges beyond those found in tabular
datasets. In this work, we extend the taxonomy for missing data mechanisms to attributed graphs by proposing GAMM (Graph Attributes Missing Mechanisms), a framework that systematically links missingness probability to both node attributes and the underlying graph structure. Our taxonomy enriches the conventional definitions of masking mechanisms by introducing graph-specific dependencies. We empirically demonstrate that state-of-the-art imputation methods, while effective on traditional masks, significantly struggle when confronted with these more realistic graph-aware missingness scenarios.
Read the full abstract ▾
All authors Richard Serrano, Baptiste Jeudy, Charlotte Laclau, Christine Largeron
Machine Learning · Cross-listed from cs.MA
Altruism and Fair Objective in Mixed-Motive Markov games
Yao-hua Franck Xu, Tayeb Lemlouma, Arnaud Braud et al. (+1 more)
Cooperation is fundamental for society's viability, as it enables the emergence of structure within heterogeneous groups that seek collective well-being. However, individuals are inclined to defect in order to benefit from the group's cooperation without contributing the associated costs, thus leading to unfair situations. In game theory, social dilemmas entail this dichotomy between individual interest and collective outcome. The most dominant approach to multi-
agent cooperation is the utilitarian welfare which can produce efficient highly inequitable outcomes. This paper proposes a novel framework to foster fairer cooperation by replacing the standard utilitarian objective with Proportional
Fairness. We introduce a fair altruistic utility for each
agent, defined on the individual log-payoff space and derive the analytical conditions required to ensure cooperation in classic social dilemmas. We then extend this framework to sequential settings by defining a Fair Markov Game and deriving novel fair Actor-Critic
algorithms to learn fair policies. Finally, we evaluate our method in various social dilemma environments.
Read the full abstract ▾
All authors Yao-hua Franck Xu, Tayeb Lemlouma, Arnaud Braud, Jean-Marie Bonnin
Machine Learning
Modalities, a PyTorch-native Framework For Large-scale LLM Training and Research
Max Lübbering, Timm Ruland, Richard Rutmann et al. (+8 more)
Today's LLM (pre-) training and research workflows typically allocate a significant amount of compute to large-scale ablation studies. Despite the substantial compute costs of these ablations, existing open-source frameworks provide limited tooling for these experiments, often forcing researchers to write their own wrappers and scripts. We propose Modalities, an end-to-end PyTorch-native framework that integrates data-driven LLM research with large-scale model training from two angles. Firstly, by integrating state-of-the-art parallelization strategies, it enables both efficient pretraining and systematic ablations at trillion-
token and billion-parameter scale. Secondly, Modalities adopts modular design with declarative, self-contained configuration, enabling reproducibility and extensibility levels that are difficult to achieve out-of-the-box with existing LLM training frameworks.
Read the full abstract ▾
All authors Max Lübbering, Timm Ruland, Richard Rutmann, Felix Stollenwerk, David Fitzek, Michael Fromm, Alexander Weber, Rafet Sifa, Nicolas Flores-Herr, Joachim Köhler, Mehdi Ali
Machine Learning
Reinforcement Learning with Backtracking Feedback
Bilgehan Sel, Vaishakh Keshava, Phillip Wallis et al. (+3 more)
Addressing the critical need for robust safety in Large
Language Models (LLMs), particularly against
adversarial attacks and in-distribution errors, we introduce Reinforcement Learning with Backtracking Feedback (RLBF). This framework advances upon prior methods, such as BSAFE, by primarily leveraging a Reinforcement Learning (RL) stage where models learn to dynamically correct their own generation errors. Through RL with critic feedback on the model's live outputs, LLMs are trained to identify and recover from their actual, emergent safety violations by emitting an efficient "backtrack by x
tokens" signal, then continuing generation autoregressively. This RL process is crucial for instilling resilience against sophisticated
adversarial strategies, including middle filling, Greedy Coordinate Gradient (GCG) attacks, and decoding parameter manipulations. To further support the acquisition of this backtracking capability, we also propose an enhanced Supervised Fine-Tuning (SFT) data generation strategy (BSAFE+). This method improves upon previous data creation techniques by injecting violations into coherent, originally safe text, providing more effective initial training for the backtracking mechanism. Comprehensive empirical evaluations demonstrate that RLBF significantly reduces attack success rates across diverse
benchmarks and model scales, achieving superior safety outcomes while critically preserving foundational model utility.
Read the full abstract ▾
All authors Bilgehan Sel, Vaishakh Keshava, Phillip Wallis, Lukas Rutishauser, Ming Jin, Dingcheng Li
Machine Learning
OJBKQ: Objective-Joint Babai-Klein Quantization
Xinyu Wang, Ziyu Zhao, Peng Lu et al. (+2 more)
Post-training quantization (PTQ) is widely used to compress large
language models without retraining. However, many existing weight-only methods rely on heuristic objectives and greedy rounding, thus leading to noticeable degradation under low-bit quantization. In this work, we introduce OJBKQ (Objective-Joint Babai-Klein Quantization with K-Best Sampling), a layer-wise PTQ method that formulates weight quantization as a joint optimization problem over activations and weights. This formulation results in a multiple-right-hand-side box-constrained integer least squares (BILS) problem in each layer, which is NP-hard. For each column of the weight
matrix, we apply an extended Babai nearest-plane
algorithm and an extended version of Klein's randomized Babai
algorithm to find the minimum-residual Babai-Klein point, a sub-optimal solution to the BILS problem. Experimental results on large
language models show that OJBKQ achieves lower perplexity at 3-4 bits compared to existing PTQ approaches, while maintaining comparable computational cost.
Read the full abstract ▾
All authors Xinyu Wang, Ziyu Zhao, Peng Lu, Yu Gu, Xiao-Wen Chang
Machine Learning · Cross-listed from stat.ML
Schrödinger bridge problem via empirical risk minimization
Denis Belomestny, Alexey Naumov, Nikita Puchkin et al. (+1 more)
We study the Schrödinger bridge problem when the endpoint distributions are available only through samples. Classical computational approaches estimate Schrödinger potentials via Sinkhorn iterations on empirical measures and then construct a time-inhomogeneous drift by differentiating a kernel-smoothed dual solution. In contrast, we propose a learning-theoretic route: we rewrite the Schrödinger system in terms of a single positive transformed potential that satisfies a nonlinear fixed-point equation and estimate this potential by empirical risk minimization over a function class. We establish uniform concentration of the empirical risk around its population counterpart under sub-Gaussian assumptions on the reference kernel and terminal density. We plug the learned potential into a
stochastic control representation of the bridge to generate samples. We illustrate performance of the suggested approach with numerical experiments.
Read the full abstract ▾
All authors Denis Belomestny, Alexey Naumov, Nikita Puchkin, Denis Suchkov
Machine Learning · Cross-listed from cs.AI
Grounding Generative Planners in Verifiable Logic: A Hybrid Architecture for Trustworthy Embodied AI
Feiyu Wu, Xu Zheng, Yue Qu et al. (+3 more)
Large
Language Models (LLMs) show promise as planners for
embodied AI, but their
stochastic nature lacks formal
reasoning, preventing strict safety guarantees for physical deployment. Current approaches often rely on unreliable LLMs for safety checks or simply reject unsafe plans without offering repairs. We introduce the Verifiable Iterative Refinement Framework (VIRF), a neuro-symbolic architecture that shifts the paradigm from passive safety gatekeeping to active collaboration. Our core contribution is a tutor-apprentice dialogue where a deterministic Logic Tutor, grounded in a formal safety ontology, provides causal and pedagogical feedback to an LLM planner. This enables intelligent plan repairs rather than mere avoidance. We also introduce a scalable knowledge acquisition pipeline that synthesizes safety knowledge bases from real-world documents, correcting blind spots in existing
benchmarks. In challenging home safety tasks, VIRF achieves a perfect 0 percent Hazardous Action Rate (HAR) and a 77.3 percent Goal-Condition Rate (GCR), which is the highest among all baselines. It is highly efficient, requiring only 1.1 correction iterations on average. VIRF demonstrates a principled pathway toward building fundamentally trustworthy and verifiably safe
embodied agents.
Read the full abstract ▾
All authors Feiyu Wu, Xu Zheng, Yue Qu, Zhuocheng Wang, Zicheng Feng, Hui Li
Machine Learning
Dynamic Regret via Discounted-to-Dynamic Reduction with Applications to Curved Losses and Adam Optimizer
Yan-Feng Xie, Yu-Jie Zhang, Peng Zhao et al. (+1 more)
We study dynamic regret minimization in non-stationary online learning, with a primary focus on follow-the-regularized-leader (FTRL) methods. FTRL is important for curved losses and for understanding adaptive optimizers such as Adam, yet existing dynamic regret analyses are less explored for FTRL. To address this, we build on the discounted-to-dynamic reduction and present a modular way to obtain dynamic regret bounds of FTRL-related problems. Specifically, we focus on two representative curved losses: linear regression and logistic regression. Our method not only simplifies existing proofs for the optimal dynamic regret of online linear regression, but also yields new dynamic regret guarantees for online logistic regression. Beyond online convex optimization, we apply the reduction to analyze the Adam optimizers, obtaining optimal convergence rates in
stochastic, non-convex, and non-smooth settings. The reduction also enables a more detailed treatment of Adam with two discount parameters $(\beta_1,\beta_2)$, leading to new results for both clipped and clip-free variants of Adam optimizers.
Read the full abstract ▾
All authors Yan-Feng Xie, Yu-Jie Zhang, Peng Zhao, Zhi-Hua Zhou