Machine Learning
RiemannGL: Riemannian Geometry Changes Graph Deep Learning
Li Sun, Qiqi Wan, Suyang Zhou et al. (+2 more)
Graphs are ubiquitous, and learning on graphs has become a cornerstone in artificial intelligence and data mining communities. Unlike pixel grids in images or sequential structures in
language, graphs exhibit a typical non-Euclidean structure with complex interactions among the objects. This paper argues that Riemannian geometry provides a principled and necessary foundation for graph representation learning, and that Riemannian graph learning should be viewed as a unifying paradigm rather than a collection of isolated techniques. While recent studies have explored the integration of graph learning and Riemannian geometry, most existing approaches are limited to a narrow class of
manifolds, particularly
hyperbolic spaces, and often adopt extrinsic
manifold formulations. We contend that the central mission of Riemannian graph learning is to endow graph neural networks with intrinsic
manifold structures, which remains underexplored. To advance this perspective, we identify key conceptual and methodological gaps in existing approaches and outline a structured research agenda along three dimensions:
manifold type, neural architecture, and learning paradigm. We further discuss open challenges, theoretical foundations, and promising directions that are critical for unlocking the full potential of Riemannian graph learning. This paper aims to provide a coherent viewpoint and to stimulate broader exploration of Riemannian geometry as a foundational framework for future graph learning research.
Read the full abstract ▾
All authors Li Sun, Qiqi Wan, Suyang Zhou, Zhenhao Huang, Philip S. Yu
Machine Learning
A Jointly Efficient and Near-Optimal Algorithm for Heteroskedastic Generalized Linear Bandits with Adversarial Corruptions
Sanghwa Kim, Junghyun Lee and Se-Young Yun
We consider heteroskedastic generalized linear bandits (GLBs) with
adversarial corruptions and propose HCW-GLB-OMD, a simple
algorithm that combines an online mirror descent (OMD)-based
estimator with Hessian-based confidence weights. Although both components are well studied, establishing guarantees for the combined
algorithm requires a nontrivial analysis. We establish a corruption-robust confidence sequence using technical tools not required in prior analyses, including a weighted likelihood supermartingale and a self-bounding argument. For the regret analysis, we derive a new Hessian potential lemma to handle the interaction between
adversarial corruption and adaptive confidence weighting. For self-concordant GLMs, we prove a high-probability regret bound of $\widetilde{\mathcal{O}}\bigl( d \sqrt{\sum_t g(\tau_t) \dot\mu_{t,\star}} + d^2 g_{\max} \kappa + d(1+\kappa)(C\vee1) \bigr)$, where $\dot\mu_{t,\star}$ is the slope of $\mu$ at the optimal arm at time $t$, $g(\tau_t)$ is the observed dispersion, $g_{\max}=\max_t g(\tau_t)$, $\kappa$ is the worst-case inverse slope of $\mu$, and $C$ is the corruption budget. The
algorithm attains this bound with one OMD update per round and $\mathcal{O}(d^2)$ storage. We complement this guarantee with matching lower bounds for the leading $\sqrt{T}$ term and, up to $\kappa$, the $dC$ corruption term.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Healthy Harvests: A Comparative Look at Guava Disease Classification Using InceptionV3
Samanta Ghosh, Shaila Afroz Anika, Umma Habiba Ahmed et al. (+3 more)
Guava fruits often suffer from many diseases. This can harm fruit quality and fruit crop yield. Early identification is important for minimizing damage and ensuring fruit health. This study focuses on 3 different categories for classifying diseases. These are Anthracnose, Fruit flies, and Healthy fruit. The data set used in this study is collected from Mendeley Data. This
dataset contains 473 original images of Guava. These images vary in size and format. The original
dataset was resized to 256x256 pixels with RGB color mode for better consistency. After this, the Data augmentation process is applied to improve the
dataset by generating variations of the original images. The augmented
dataset consists of 3784 images using advanced preprocessing techniques. Two deep learning models were implemented to classify the images. The InceptionV3 model is well known for its advanced framework. These apply multiple convolutional filters for obtaining different features effectively. On the other hand, the ResNet50 model helps to train deeper networks by using residual learning. The InceptionV3 model achieved the impressive accuracy of 98.15%, and ResNet50got 94.46% accuracy. Data mixing methods such as CutMix and MixUp were applied to enhance the model's robustness. The confusion
matrix was used to evaluate the overall model performance of both InceptionV3 and Resnet50. Additionally, SHAP analysis is used to improve interpretability, which helps to find the significant parts of the image for the model prediction. This study purposes to highlight how advanced models enhan
Read the full abstract ▾
All authors Samanta Ghosh, Shaila Afroz Anika, Umma Habiba Ahmed, B. M. Shahria Alam, Mohammad Tahmid Noor, Nishat Tasnim Niloy
Machine Learning
MoEEdit: Efficient and Routing-Stable Knowledge Editing for Mixture-of-Experts LLMs
Yupu Gu, Rongzhe Wei, Andy Zhu et al. (+1 more)
Knowledge editing (KE) enables precise modifications to factual content in large
language models (LLMs). Existing KE methods are largely designed for dense architectures, limiting their applicability to the increasingly prevalent sparse Mixture-of-Experts (MoE) models that underpin modern scalable LLMs. Although MoEs offer strong efficiency and capacity scaling, naively adapting dense-model editors is both computationally costly and prone to routing distribution shifts that undermine stability and consistency. To address these challenges, we introduce MoEEdit, the first routing-stable framework for parameter-modifying knowledge editing in MoE LLMs. Our method reparameterizes expert updates via per-expert null-space projections that keep router inputs invariant and thereby suppress routing shifts. The resulting block-structured optimization is solved efficiently with a block coordinate descent (BCD) solver. Experiments show that MoEEdit attains state-of-the-art efficacy and generalization while preserving high specificity and routing stability, with superior compute and
memory efficiency. These results establish a robust foundation for scalable, precise knowledge editing in sparse LLMs and underscore the importance of routing-stable interventions.
Read the full abstract ▾
All authors Yupu Gu, Rongzhe Wei, Andy Zhu, Pan Li
Machine Learning
Rotary Positional Embeddings as Phase Modulation: Theoretical Bounds on the RoPE Base for Long-Context Transformers
Feilong Liu
Rotary positional
embeddings (RoPE) are widely used in large
language models to encode
token positions through multiplicative rotations, yet their behavior at long context lengths remains poorly characterized. In this work, we reinterpret RoPE as phase modulation applied to a bank of complex oscillators, enabling analysis through classical signal processing theory. Under this formulation, we derive principled lower bounds on the RoPE base parameter that are necessary to preserve positional coherence over a target context length. These include a fundamental aliasing bound, analogous to a Nyquist limit, and a DC-component stability bound that constrains phase drift in low-frequency positional modes. We further extend this analysis to deep
transformers, showing that repeated rotary modulation across layers compounds angular misalignment, tightening the base requirement as depth increases. Complementing these results, we derive a precision-dependent upper bound on the RoPE base arising from finite floating-point resolution. Beyond this limit, incremental phase updates become numerically indistinguishable, leading to positional erasure even in the absence of aliasing. Together, the lower and upper bounds define a precision- and depth-dependent feasibility region a Goldilocks zone for long-context
transformers. We validate the framework through a comprehensive case study of state-of-the-art models, including LLaMA, Mistral, and DeepSeek variants, showing that observed successes, failures, and community retrofits align closely with the predicted bounds. Notably, models that violate the stability bound exhibit
attention collapse and long-range degradation, while attempts to scale beyond one million
tokens encounter a hard precision wall independent of architecture or training.
Read the full abstract ▾
Machine Learning
Stochastic Parroting in Temporal Attention – Regulating the Diagonal Sink
Victoria Hankemeier and Malte Schilling
Spatio-temporal models analyze spatial structures and temporal dynamics, which makes them prone to information degeneration among space and time. Prior literature has demonstrated that over-squashing in causal
attention or temporal convolutions creates a bias on the first
tokens. To analyze whether such a bias is present in temporal
attention mechanisms, we derive sensitivity bounds on the expected value of the Jacobian of a temporal
attention layer. We theoretically show how off-diagonal
attention scores depend on the sequence length, and that temporal
attention matrices suffer a diagonal
attention sink. We suggest regularization methods, and experimentally demonstrate their effectiveness.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Optimal Initialization in Depth: Lyapunov Initialization and Limit Theorems for Deep Leaky ReLU Networks
Constantin Kogler, Tassilo Schwarz and Samuel Kittle
Effective initialization in deep networks requires an understanding of random neural networks. In this work, a rigorous probabilistic analysis of deep bias-free random Leaky ReLU networks is provided. We prove a Law of Large Numbers and a Central Limit
Theorem for the logarithm of the norm of network activations, establishing that, as the number of layers increases, their growth is governed by a parameter called the Lyapunov exponent. This parameter characterizes a sharp phase transition between vanishing and exploding activations, and we calculate the Lyapunov exponent explicitly for Gaussian or orthogonal weight
matrices. Our results reveal that standard methods, such as He initialization or orthogonal initialization, do not guarantee activation stability for deep networks of low width. Based on these theoretical insights, we propose a novel initialization method, referred to as Lyapunov initialization, which sets the Lyapunov exponent to zero and thereby ensures that the neural network is as stable as possible, leading empirically to improved learning.
Read the full abstract ▾
Machine Learning
Riccardo Barbano, Alexander Denker, Zeljko Kereta et al. (+2 more)
Continuous-time generative models have achieved remarkable success in image restoration and synthesis. However, controlling the composition of multiple pre-trained models remains an open challenge. Current approaches largely treat composition as an algebraic composition of probability densities, such as via products or mixtures of experts. This perspective assumes the target distribution is known explicitly, which is almost never the case. In this work, we propose a different paradigm that formulates compositional generation as a cooperative
Stochastic Optimal Control problem. Rather than combining probability densities, we treat pre-trained
diffusion models as interacting
agents whose
diffusion trajectories are jointly steered, via optimal control, toward a shared objective defined on their aggregated output. We validate our framework on conditional MNIST generation and compare it against a naïve inference-time DPS-style baseline replacing learned cooperative control with per-step gradient guidance.
Read the full abstract ▾
All authors Riccardo Barbano, Alexander Denker, Zeljko Kereta, Runchang Li, Francisco Vargas
Machine Learning
Spatial-Morphological Modeling for Multi-Attribute Imputation of Urban Blocks
Vasilii Starikov, Ruslan Kozliak, Georgii Kontsevik et al. (+1 more)
Accurate reconstruction of missing morphological indicators of a city is crucial for urban planning and data-driven analysis. This study presents the spatial-morphological (SM) imputer tool, which combines data-driven morphological clustering with neighborhood-based methods to reconstruct missing values of the floor space index (FSI) and ground space index (GSI) at the city block level, inspired by the SpaceMatrix framework. This approach combines city-scale morphological patterns as global priors with local spatial information for context-dependent interpolation. The evaluation shows that while SM alone captures meaningful morphological structure, its combination with inverse distance weighting (IDW) or spatial k-nearest neighbor (sKNN) methods provides superior performance compared to existing SOTA models. Composite methods demonstrate the complementary advantages of combining morphological and spatial approaches.
Read the full abstract ▾
All authors Vasilii Starikov, Ruslan Kozliak, Georgii Kontsevik, Sergey Mityagin
Machine Learning
Near-Constant Strong Violation and Last-Iterate Convergence for Online CMDPs via Decaying Safety Margins
Qian Zuo, Zhiyong Wang and Fengxiang He
We study safe online reinforcement learning in Constrained Markov Decision Processes (CMDPs) under strong regret and violation metrics, which forbid error cancellation over time. Existing primal-dual methods that achieve sublinear strong reward regret inevitably incur growing strong constraint violation or are restricted to average-iterate convergence due to inherent oscillations. To address these limitations, we propose the Flexible safety Domain Optimization via Margin-regularized Exploration (FlexDOME)
algorithm, the first to provably achieve near-constant $\tilde{O}(1)$ strong constraint violation alongside sublinear strong regret and non-asymptotic last-iterate convergence. FlexDOME incorporates time-varying safety margins and regularization terms into the primal-dual framework. Our theoretical analysis relies on a novel term-wise asymptotic dominance strategy, where the safety margin is rigorously scheduled to asymptotically majorize the functional decay rates of the optimization and statistical errors, thereby clamping cumulative violations to a near-constant level. Furthermore, we establish non-asymptotic last-iterate convergence guarantees via a policy-dual Lyapunov argument. Experiments corroborate our theoretical findings.
Read the full abstract ▾
Machine Learning
Tuning the burn-in phase in training recurrent neural networks improves their performance
Julian D. Schiller, Malte Heinrich, Victor G. Lopez et al. (+1 more)
Training recurrent neural networks (RNNs) with standard backpropagation through time (BPTT) can be challenging, especially in the presence of long input sequences. A practical alternative to reduce computational and
memory overhead is to perform BPTT repeatedly over shorter segments of the training data set, corresponding to truncated BPTT. In this paper, we examine the training of RNNs when using such a truncated learning approach for time series tasks. Specifically, we establish theoretical bounds on the accuracy and performance loss when optimizing over subsequences instead of the full data sequence. This reveals that the burn-in phase of the RNN is an important tuning knob in its training, with significant impact on the performance guarantees. We validate our theoretical results through experiments on standard
benchmarks from the fields of system identification and time series
forecasting. In all experiments, we observe a strong influence of the burn-in phase on the training process, and proper tuning can lead to a reduction of the prediction error on the training and test data of more than 60% in some cases.
Read the full abstract ▾
All authors Julian D. Schiller, Malte Heinrich, Victor G. Lopez, Matthias A. Müller
Machine Learning · Cross-listed from cs.CL
SoftMatcha 2: A Fast and Soft Pattern Matcher for Trillion-Scale Corpora
Masataka Yoneda, Yusuke Matsushita, Go Kamoda et al. (+4 more)
We present SoftMatcha 2, an ultra-fast and flexible search
algorithm that enables search over trillion-scale natural
language corpora in under 0.3 seconds while allowing semantic variations in the form of substitution, insertion, and deletion. Our approach employs string matching based on suffix arrays that scales well with corpus size, and represents words as vectors, which underpin its semantic flexibility. To mitigate the combinatorial explosion induced by the semantic relaxation of queries, our method is built on two key algorithmic ideas: dynamic corpus-aware
pruning and fast exact lookup enabled by a disk-aware design. We theoretically analyze the efficiency of the proposed method, indicating that it can mitigate exponential growth in the search space. Empirically, on FineWeb-Edu (Lozhkov et al., 2024) (1.4T
tokens), it attains substantially lower search latency than existing methods: infini-gram (Liu et al., 2024), infini-gram mini (Xu et al., 2025), and SoftMatcha (Deguchi et al., 2025). As a practical application, our method uncovers
benchmark contamination in training corpora that existing approaches miss, and it also benefits information
retrieval and paraphrase detection. We also provide an online demo of fast, soft search across corpora in seven
languages.
Read the full abstract ▾
All authors Masataka Yoneda, Yusuke Matsushita, Go Kamoda, Kohei Suenaga, Takuya Akiba, Masaki Waga, Sho Yokoi
Machine Learning
Natural Hypergradient Descent: Algorithm Design, Convergence Analysis, and Parallel Implementation
Deyi Kong, Zaiwei Chen, Shuzhong Zhang et al. (+1 more)
In this work, we propose Natural Hypergradient Descent (NHGD), a new method for solving bilevel optimization problems. To address the computational bottleneck in hypergradient estimation–namely, the need to compute or approximate Hessian inverse–we exploit the statistical structure of the inner optimization problem and use the empirical Fisher information
matrix as an asymptotically consistent surrogate for the Hessian. This design enables a parallel optimize-and-approximate framework in which the Hessian-inverse approximation is updated synchronously with the
stochastic inner optimization, reusing gradient information at negligible additional cost. Our main theoretical contribution establishes high-probability error bounds and sample complexity guarantees for NHGD that match those of state-of-the-art optimize-then-approximate methods, while significantly reducing computational time overhead. Empirical evaluations on representative bilevel learning tasks further demonstrate the practical advantages of NHGD, highlighting its scalability and effectiveness in large-scale machine learning settings.
Read the full abstract ▾
All authors Deyi Kong, Zaiwei Chen, Shuzhong Zhang, Shancong Mou
Machine Learning
Revisiting Regularized Policy Optimization for Stable and Efficient Reinforcement Learning in Two-Player Games
Kazuki Ota, Takayuki Osa, Motoki Omura et al. (+1 more)
Two-player games such as board games have long been used as traditional
benchmarks for reinforcement learning. This work revisits a policy optimization method with reverse Kullback-Leibler regularization and
entropy regularization and analyzes this combination in two-player zero-sum settings from theoretical and empirical perspectives. From a theoretical perspective, we investigate the stability of the policy update rule in two theoretical settings: game-theoretic normal-form games and finite-length games. We provide novel convergence guarantees and verify our theoretical results through numerical experiments on synthetic games. From an empirical perspective, we derive a practical model-free reinforcement learning
algorithm based on the regularized policy optimization. We validate the training efficiency of our
algorithm through comprehensive experiments on five board games: Animal Shogi, Gardner Chess, Go, Hex, and Othello. Experimental results show that our
agent learns more efficiently than existing methods across environments.
Read the full abstract ▾
All authors Kazuki Ota, Takayuki Osa, Motoki Omura, Tatsuya Harada
Machine Learning · Cross-listed from eess.SY
Anomaly Detection with Machine Learning Algorithms in Large-Scale Power Grids
Marc Gillioz, Guillaume Dubuis, Étienne Voutaz et al. (+1 more)
We apply several machine learning
algorithms to the problem of anomaly detection in operational data for large-scale, high-voltage electric power grids. We observe important differences in the performance of the
algorithms. Neural networks typically outperform classical
algorithms such as k-nearest neighbors and support vector machines, which we explain by the strong contextual nature of the anomalies. We show that unsupervised learning
algorithm work remarkably well and that their predictions are robust against simultaneous, concurring anomalies.
Read the full abstract ▾
All authors Marc Gillioz, Guillaume Dubuis, Étienne Voutaz, Philippe Jacquod
Machine Learning · Cross-listed from cs.AI
Reinforcing Chain-of-Thought Reasoning with Self-Evolving Rubrics
Leheng Sheng, Wenchang Ma, Ruixin Hong et al. (+3 more)
Despite chain-of-thought (CoT) playing crucial roles in LLM
reasoning, directly rewarding it is difficult: training a reward model demands heavy human labeling efforts, and static RMs struggle with evolving CoT distributions and reward hacking. These challenges motivate us to seek an autonomous CoT rewarding approach that requires no human annotation efforts and can evolve gradually. Inspired by recent self-evolving training methods, we propose RLCER (Reinforcement Learning with CoT Supervision via Self-Evolving Rubrics), which enhances the outcome-centric RLVR by rewarding CoTs with self-proposed and self-evolving rubrics. We show that self-proposed and self-evolving rubrics provide reliable CoT supervision signals even without outcome rewards, enabling RLCER to outperform outcome-centric RLVR. Moreover, when used as in-
prompt hints, these self-proposed rubrics further improve inference-time performance.
Read the full abstract ▾
All authors Leheng Sheng, Wenchang Ma, Ruixin Hong, Xiang Wang, An Zhang, Tat-Seng Chua
Machine Learning · Cross-listed from cs.CL
Diagnosing Structural Failures in LLM-Based Evidence Extraction for Meta-Analysis
Zhiyin Tan and Jennifer D'Souza
Systematic reviews and meta-analyses rely on converting narrative articles into structured, numerically grounded study records. Despite rapid advances in large
language models (LLMs), it remains unclear whether they can meet the structural requirements of this process, which hinge on preserving roles, methods, and effect-size attribution across documents rather than on recognizing isolated entities. We propose a structural, diagnostic framework that evaluates LLM-based evidence extraction as a progression of schema-constrained queries with increasing relational and numerical complexity, enabling precise identification of failure points beyond atom-level extraction. Using a manually curated corpus spanning five scientific domains, together with a unified query suite and evaluation protocol, we evaluate two state-of-the-art LLMs under both per-document and long-context, multi-document input regimes. Across domains and models, performance remains moderate for single-property queries but degrades sharply once tasks require stable binding between variables, roles, statistical methods, and effect sizes. Full meta-analytic association tuples are extracted with near-zero reliability, and long-context inputs further exacerbate these failures. Downstream aggregation amplifies even minor upstream errors, rendering corpus-level statistics unreliable. Our analysis shows that these limitations stem not from entity recognition errors, but from systematic structural breakdowns, including role reversals, cross-analysis binding drift, instance compression in dense result sections, and numeric misattribution, indicating that current LLMs lack the structural fidelity, relational binding, and numerical grounding required for automated meta-analysis. The code and data are publicly available at GitHub (https://github.com/zhiyintan/LLM-Meta-Analysis).
Read the full abstract ▾
Machine Learning
FedPS: Federated Preprocessing for structured data via aggregated Statistics
Xuefeng Xu and Graham Cormode
Federated Learning (FL) enables multiple parties to collaboratively train machine learning models without sharing raw data. However, before training, data must be preprocessed to address missing values, inconsistent formats, and heterogeneous feature scales. This preprocessing stage is critical for model performance but is largely overlooked in FL research. In practical FL systems,
privacy constraints prohibit centralizing raw data, while communication efficiency introduces further challenges for distributed preprocessing. We introduce FedPS, a framework for
federated data preprocessing based on aggregated statistics. FedPS leverages data-sketching techniques to efficiently summarize local
datasets while preserving essential statistical information. Building on these summaries, we design
federated algorithms for feature scaling, encoding, discretization, and missing-value imputation, and extend preprocessing-related models such as
Bayesian Linear Regression to both horizontal and vertical FL settings. FedPS provides flexible, communication-efficient, and consistent preprocessing pipelines for practical FL deployments.
Read the full abstract ▾
Machine Learning
The Sample Complexity of Uniform Approximation for Multi-Dimensional CDFs and Fixed-Price Mechanisms
Matteo Castiglioni, Anna Lunghi and Alberto Marchesi
We study the sample complexity of learning a uniform approximation of an $n$-dimensional cumulative distribution function (CDF) within an error $\epsilon > 0$, when observations are restricted to a minimal one-bit feedback. This serves as a counterpart to the multivariate DKW inequality under ”full feedback”, extending it to the setting of ”bandit feedback”. Our main result shows a near-dimensional-invariance in the sample complexity: we get a uniform $\epsilon$-approximation with a sample complexity $\frac{1}{\epsilon^3}{\log\left(\frac 1 \epsilon \right)^{\mathcal{O}(n)}}$ over a arbitrary fine grid, where the dimensionality $n$ only affects logarithmic terms. As direct corollaries, we provide tight sample complexity bounds and novel regret guarantees for learning fixed-price mechanisms in small
markets, such as bilateral trade settings.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Deep Learning of Compositional Targets with Hierarchical Spectral Methods
Hugo Tabanelli, Yatin Dandi, Luca Pesce et al. (+1 more)
Why depth yields a computational advantage over shallow methods remains a central open question in learning theory. We study this question in a controlled high-dimensional Gaussian setting, focusing on compositional target functions. We analyze their learnability using an explicit three-layer fitting model trained via layer-wise spectral
estimators. The target combines
polynomial latent features with a scalar readout, and its compositional structure allows learning to proceed in stages. Recovering an intermediate representation reduces the problem to lower-degree spectral estimation tasks, as in multi-index models. Our analysis establishes Gaussian comparison bounds for projected moments when $0<\varepsilon<1/3$, and motivates separations in sample complexity between two- and three-layer learning strategies, conditional on a conjectured spectral recovery step.
Read the full abstract ▾
All authors Hugo Tabanelli, Yatin Dandi, Luca Pesce, Florent Krzakala
Machine Learning
ICA: Information-Aware Credit Assignment for Visually Grounded Long-Horizon Information-Seeking Agents
Cong Pang, Xuyu Feng, Yujie Yi et al. (+8 more)
Long-horizon reinforcement learning for information seeking
agents remains difficult because terminal rewards reveal whether the final answer is correct, but not which acquired information enabled it. This difficulty is amplified by text-derived webpage observations, where parsing, truncation, and summarization often produce incomplete and unstable content representations across trajectories. We propose an evidence-centric framework for web
agent learning that represents information acquired through tools as identifiable units for comparison across trajectories. In particular, fetched webpages are represented as rendered snapshots, preserving layout and
multimodal content as stable content-level observations. Building on these units, we introduce Information-Aware Credit Assignment, a post hoc reward propagation method that estimates turn-level utility scores from rollout success rates and assigns dense rewards to intermediate steps that introduced high-utility information. Integrated with GSPO, our method consistently improves performance on BrowseComp,
GAIA, Xbench-DS, and Seal-0. Code and
datasets will be released at https://github.com/pc-inno/ICA_MM_deepsearch.
Read the full abstract ▾
All authors Cong Pang, Xuyu Feng, Yujie Yi, Jiaqi Su, Zixuan Chen, Jiawei Hong, Tiankuo Yao, Nang Yuan, Jiapeng Luo, Lewei Lu, Xin Lou
Machine Learning
Automated Model Design using Gated Neuron Selection in Telecom
Adam Orucu, Marcus Medhage, Farnaz Moradi et al. (+2 more)
The telecommunications industry is experiencing rapid growth in adopting deep learning for critical tasks such as traffic prediction, signal strength prediction, and quality of service optimisation. However, designing neural network architectures for these applications remains challenging and time-consuming, particularly when targeting compact models suitable for resource-constrained network environments. Therefore, there is a need for automating the model design process to create high-performing models efficiently. This paper introduces TabGNS (Tabular Gated
Neuron Selection), a novel gradient-based Neural Architecture Search (NAS) method specifically tailored for tabular data in telecommunications networks. We evaluate TabGNS across multiple telecommunications and generic tabular
datasets, demonstrating improvements in prediction performance while reducing the architecture size by 51-82% and reducing the search time by up to 36x compared to state-of-the-art tabular NAS methods. Integrating TabGNS into the model lifecycle management enables automated design of neural networks throughout the lifecycle, accelerating deployment of ML solutions in telecommunications networks.
Read the full abstract ▾
All authors Adam Orucu, Marcus Medhage, Farnaz Moradi, Andreas Johnsson, Sarunas Girdzijauskas
Machine Learning
Time Series Foundation Models for Energy Load Forecasting on Consumer Hardware: A Multi-Dimensional Zero-Shot Benchmark
Luigi Simeone
Time Series Foundation Models (TSFMs) have introduced zero-shot prediction capabilities that bypass the need for task-specific training. Whether these capabilities translate to mission-critical applications such as electricity demand
forecasting–where accuracy, calibration, and robustness directly affect grid operations–remains an open question. We present a multi-dimensional
benchmark evaluating four TSFMs (Chronos-Bolt, Chronos-2, Moirai-2, and TinyTimeMixer) alongside Prophet as an industry-standard baseline and two statistical references (SARIMA and Seasonal Naive), using ERCOT hourly load data from 2020 to 2024. All experiments run on consumer-grade hardware (AMD Ryzen 7, 16GB RAM, no GPU). The evaluation spans four axes: (1) context length sensitivity from 24 to 2048 hours, (2) probabilistic forecast calibration, (3) robustness under distribution shifts including COVID-19 lockdowns and Winter Storm Uri, and (4) prescriptive analytics for operational decision support. The top-performing foundation models achieve MASE values near 0.31 at long context lengths (C = 2048h, day-ahead horizon), a 47% reduction over the Seasonal Naive baseline. The inclusion of Prophet exposes a structural advantage of pre-trained models: Prophet fails when the fitting window is shorter than its seasonality period (MASE > 74 at 24-hour context), while TSFMs maintain stable accuracy even with minimal context because they recognise temporal patterns learned during pre-training rather than estimating them from scratch. Calibration varies substantially across models–Chronos-2 produces well-calibrated prediction intervals (95% empirical coverage at 90% nominal level) while both Moirai-2 and Prophet exhibit overconfidence (~70% coverage). We provide practical model selection guidelines and release the complete
benchmark framework for reproducibility.
Read the full abstract ▾
Machine Learning
Enhancing Multivariate Time Series Forecasting with Global Temporal Retrieval
Fanpu Cao, Lu Dai, Jindong Han et al. (+1 more)
Multivariate time series
forecasting (MTSF) plays a vital role in numerous real-world applications, yet existing models remain constrained by their reliance on a limited historical context. This limitation prevents them from effectively capturing global periodic patterns that often span cycles significantly longer than the input horizon - despite such patterns carrying strong predictive signals. Naive solutions, such as extending the historical window, lead to severe drawbacks, including overfitting, prohibitive computational costs, and redundant information processing. To address these challenges, we introduce the Global Temporal Retriever (GTR), a lightweight and plug-and-play module designed to extend any
forecasting model's temporal awareness beyond the immediate historical context. GTR maintains an adaptive global temporal
embedding of the entire cycle and dynamically retrieves and aligns relevant global segments with the input sequence. By jointly modeling local and global dependencies through a 2D convolution and residual fusion, GTR effectively bridges short-term observations with long-term periodicity without altering the host model architecture. Extensive experiments on six real-world
datasets demonstrate that GTR consistently delivers state-of-the-art performance across both short-term and long-term
forecasting scenarios, while incurring minimal parameter and computational overhead. These results highlight GTR as an efficient and general solution for enhancing global periodicity modeling in MTSF tasks. Code is available at this repository: https://github.com/macovaseas/GTR.
Read the full abstract ▾
All authors Fanpu Cao, Lu Dai, Jindong Han, Hui Xiong
Machine Learning · Cross-listed from cs.AI
SynergyKGC: Reconciling Topological Heterogeneity in Knowledge Graph Completion via Topology-Aware Synergy
Xuecheng Zou, Yu Tang and Bingbing Wang
Knowledge Graph Completion (KGC) fundamentally hinges on the coherent fusion of pre-trained entity semantics with heterogeneous topological structures to facilitate robust relational
reasoning. However, existing paradigms encounter a critical "structural resolution mismatch," failing to reconcile divergent representational demands across varying graph densities, which precipitates structural noise interference in dense clusters and catastrophic representation collapse in sparse regions. We present SynergyKGC, an adaptive framework that advances traditional neighbor aggregation to an active Cross-Modal Synergy Expert via relation-aware cross-
attention and semantic-intent-driven gating. By coupling a density-dependent Identity Anchoring strategy with a Double-tower Coherent Consistency architecture, SynergyKGC effectively reconciles topological heterogeneity while ensuring representational stability across training and inference phases. Systematic evaluations on two public
benchmarks validate the superiority of our method in significantly boosting KGC hit rates, providing empirical evidence for a generalized principle of resilient information integration in non-homogeneous structured data.
Read the full abstract ▾
Machine Learning
Training and Benchmarking Code Generation for Physics-Inspired Animations
Yanan Wang, Renxi Wang, Yongxin Wang et al. (+5 more)
Large
language models (LLMs) have been widely studied in areas such as mathematical
reasoning, complex coding, and scientific problem solving. However, their ability to generate executable code that visually depicts physical scenarios and their qualitative dynamics remains underexplored. We propose SimuScene, the first systematic study that trains and evaluates LLMs on code generation for physics-inspired animations across 52 concepts spanning five physics domains. We build an automated data collection pipeline with human verification to ensure data quality. The resulting
dataset contains 7,659 scenarios, including a 334-example human-verified test set. We evaluate 10 contemporary LLMs and find that even the strongest model achieves only a 21.5% Avg@8 accuracy, demonstrating the difficulty of generating animations that are both executable and visually aligned with physical scenario descriptions. Finally, we introduce a reinforcement learning pipeline that uses visual rewards from code-generated
videos to train text-only LLMs, with a
vision-
language model evaluating
videos through verification questions. Experiments show that training with our data and
video-based rewards improves LLM performance on physics-inspired animation generation.
Read the full abstract ▾
All authors Yanan Wang, Renxi Wang, Yongxin Wang, Xuezhi Liang, Fajri Koto, Timothy Baldwin, Xiaodan Liang, Haonan Li
Machine Learning · Cross-listed from cs.IR
Training-Induced Bias Toward LLM-Generated Content in Dense Retrieval
William Xion and Wolfgang Nejdl
Dense
retrieval is a promising approach for acquiring relevant context or world knowledge in open-domain natural
language processing tasks and is now widely used in information
retrieval applications. However, recent reports claim a broad preference for text generated by large
language models (LLMs). This bias is called "source bias", and it has been hypothesized that lower perplexity contributes to this effect. In this study, we revisit this claim by conducting a controlled evaluation to trace the emergence of such preferences across training stages and data sources. Using parallel human- and LLM-generated counterparts of the SciFact and Natural Questions (NQ320K)
datasets, we compare unsupervised checkpoints with models fine-tuned using in-domain human text, in-domain LLM-generated text, and MS MARCO. Our results show the following: 1) Unsupervised retrievers do not exhibit a uniform pro-LLM preference. The direction and magnitude depend on the
dataset. 2) Across the settings tested, supervised fine-tuning on MS MARCO consistently shifts the rankings toward LLM-generated text. 3) In-domain fine-tuning produces
dataset-specific and inconsistent shifts in preference. 4) Fine-tuning on LLM-generated corpora induces a pronounced pro-LLM bias. Finally, a retriever-centric perplexity probe involving the reattachment of a
language modeling head to the fine-tuned dense retriever encoder indicates agreement with relevance near chance, thereby weakening the explanatory power of perplexity. Our study demonstrates that source bias is a training-induced phenomenon rather than an inherent property of dense retrievers.
Read the full abstract ▾
Machine Learning · Cross-listed from eess.AS
Self-Supervised Learning for Speaker Recognition: A study and review
Theo Lepage and Reda Dehak
Deep learning models trained in a supervised setting have revolutionized audio and
speech processing. However, their performance inherently depends on the quantity of human-annotated data, making them costly to scale and prone to poor generalization under unseen conditions. To address these challenges, Self-Supervised Learning (SSL) has emerged as a promising paradigm, leveraging vast amounts of unlabeled data to learn relevant representations. The application of SSL for Automatic
Speech Recognition (ASR) has been extensively studied, but research on other downstream tasks, notably Speaker Recognition (SR), remains in its early stages. This work describes major SSL instance-invariance frameworks (e.g., SimCLR, MoCo, and DINO), initially developed for computer
vision, along with their adaptation to SR. Various SSL methods for SR, proposed in the literature and built upon these frameworks, are also presented. An extensive review of these approaches is then conducted: (1) the effect of the main hyperparameters of SSL frameworks is investigated; (2) the role of SSL components is studied (e.g., data-augmentation, projector, positive sampling); and (3) SSL frameworks are evaluated on SR with in-domain and out-of-domain data, using a consistent experimental setup, and a comprehensive comparison of SSL methods from the literature is provided. Specifically, DINO achieves the best downstream performance and effectively models intra-speaker variability, although it is highly sensitive to hyperparameters and training conditions, while SimCLR and MoCo provide robust alternatives that effectively capture inter-speaker variability and are less prone to collapse. This work aims to highlight recent trends and advancements, identifying current challenges in the field.
Read the full abstract ▾
Machine Learning
Adaptive Sampling and Clipping for Private Worst-Case Group Optimization
Max Cairney-Leeming, Amartya Sanyal and Christoph H. Lampert
A central requirement for the acceptance of machine learning methods for human-centric tasks is that they should be fair, in the sense that they should work comparably well for individuals from different societal groups. A second, equally important, requirement is that they should respect the
privacy of user data. While techniques exist to address each aspect in isolation, such as worst-case group optimization for the former and differentially private SGD for the latter, these are often at odds with with each other, and no practical method currently exists to enforce both requirements simultaneously. In this work, we overcome this problem and propose an
algorithm for optimizing the worst-case group accuracy in a differentially private way. Our main contribution is ASC (Adaptively Sampled and Clipped Worst-case Group Optimization), which adaptively controls both the sampling rate and the clipping threshold of each group's gradient contributions. Thereby, it is able to reweight the training objective in favor of harder-to-learn groups, while keeping the noise required to enforce
privacy low enough to preserve model utility. Our experiments show that ASC achieves substantially higher worst-case group accuracy than prior work, without sacrificing overall average accuracy.
Read the full abstract ▾
Machine Learning
RePO: Bridging On-Policy Learning and Off-Policy Knowledge through Rephrasing Policy Optimization
Linxuan Xia, Xiaolong Yang, Yongyuan Chen et al. (+4 more)
Aligning large
language models (LLMs) on domain-specific data remains a fundamental challenge. Supervised fine-tuning (SFT) offers a straightforward way to inject domain knowledge but often degrades the model's generality. In contrast, on-policy reinforcement learning (RL) preserves generality but fails to effectively assimilate hard samples that exceed the model's current
reasoning level. Recent off-policy RL attempts improve hard sample utilization, yet they suffer from severe training instability due to the forced distribution shift toward off-policy knowledge. To reconcile effective off-policy knowledge absorption with the stability of on-policy RL, we propose Rephrasing Policy Optimization (RePO). In RePO, the policy model is prompted to first comprehend off-policy knowledge and then rephrase it into trajectories that conform to its own stylistic and parametric distribution. RePO dynamically replaces low-reward rollouts with these rephrased, high-quality trajectories. This strategy guides the model toward correct
reasoning paths while strictly preserving on-policy training dynamics. Experiments on several
benchmarks demonstrate that RePO improves hard-sample utilization and outperforms existing baselines, achieving state-of-the-art performance.
Read the full abstract ▾
All authors Linxuan Xia, Xiaolong Yang, Yongyuan Chen, Enyue Zhao, Deng Cai, Yasheng Wang, Boxi Wu