Machine Learning
Deep TPC: Temporal-Prior Conditioning for Time Series Forecasting
Filippos Bellos, NaveenJohn Premkumar, Yannis Avrithis et al. (+2 more)
LLM-for-time series (TS) methods typically treat time shallowly, injecting positional or
prompt-based cues once at the input of a largely frozen decoder, which limits temporal
reasoning as this information degrades through the layers. We introduce Temporal-Prior Conditioning (TPC), which elevates time to a first-class modality that conditions the model at multiple depths. TPC attaches a small set of learnable time series
tokens to the patch stream; at selected layers these
tokens cross-attend to temporal
embeddings derived from compact, human-readable temporal descriptors encoded by the same frozen LLM, then feed temporal context back via self-
attention. This disentangles time series signal and temporal information while maintaining a low parameter budget. We show that by training only the cross-
attention modules and explicitly disentangling time series signal and temporal information, TPC consistently outperforms both full fine-tuning and shallow conditioning strategies, achieving state-of-the-art performance in long-term
forecasting across diverse
datasets. Code available at: https://github.com/fil-mp/Deep_tpc
Read the full abstract ▾
All authors Filippos Bellos, NaveenJohn Premkumar, Yannis Avrithis, Nam H. Nguyen, Jason J. Corso
Machine Learning · Cross-listed from cs.GT
Multi-Agent Combinatorial-Multi-Armed-Bandit framework for the Submodular Welfare Problem under Bandit Feedback
Subham Pokhriyal, Shweta Jain and Vaneet Aggarwal
We study the Submodular Welfare Problem (SWP), where items are partitioned among
agents with monotone submodular utilities to maximize the total welfare under bandit feedback. Classical SWP assumes full value-oracle access, achieving $(1-1/e)$ approximations via continuous-greedy
algorithms. We extend this to a multi-
agent combinatorial bandit framework (MA-CMAB), where actions are partitions under full-bandit feedback with non-communicating
agents. Unlike prior single-
agent or separable multi-
agent CMAB models, our setting couples
agents through shared allocation constraints. We propose an explore-then-commit strategy with randomized assignments, achieving $\tilde{\mathcal{O}}(T^{2/3})$ regret against a $(1-1/e)$
benchmark, the first such guarantee for partition-based submodular welfare problem under bandit feedback.
Read the full abstract ▾
Machine Learning
Towards Secure and Scalable Energy Theft Detection: A Federated Learning Approach for Resource-Constrained Smart Meters
Diego Labate, Dipanwita Thakur and Giancarlo Fortino
Energy theft poses a significant threat to the stability and efficiency of smart grids, leading to substantial economic losses and operational challenges. Traditional centralized machine learning approaches for theft detection require aggregating user data, raising serious concerns about
privacy and data security. These issues are further exacerbated in smart meter environments, where devices are often resource-constrained and lack the capacity to run heavy models. In this work, we propose a
privacy-preserving
federated learning framework for energy theft detection that addresses both
privacy and computational constraints. Our approach leverages a lightweight multilayer perceptron (MLP) model, suitable for deployment on low-power smart meters, and integrates basic differential
privacy (DP) by injecting Gaussian noise into local model updates before aggregation. This ensures formal
privacy guarantees without compromising learning performance. We evaluate our framework on a real-world smart meter
dataset under both IID and non-IID data distributions. Experimental results demonstrate that our method achieves competitive accuracy, precision, recall, and AUC scores while maintaining
privacy and efficiency. This makes the proposed solution practical and scalable for secure energy theft detection in next-generation smart grid infrastructures.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
EnterpriseBench Corecraft: Training Generalizable Agents on High-Fidelity RL Environments
Sushant Mehta, Logan Ritchie, Suhaas Garre et al. (+3 more)
We show that training AI
agents on high-fidelity reinforcement learning environments produces capabilities that generalize beyond the training distribution. We introduce CoreCraft, the first environment in EnterpriseBench, Surge AI's suite of agentic RL environments. CoreCraft is a fully operational enterprise simulation of a customer support organization, comprising over 2,500 entities across 14 entity types with 23 unique tools, designed to measure whether AI
agents can perform the multi-step, domain-specific work that real jobs demand. Frontier models such as GPT-5.2 and Claude Opus 4.6 solve fewer than 30% of tasks when all expert-authored rubric criteria must be satisfied. Using this environment, we train GLM 4.6 with Group Relative Policy Optimization (GRPO) and adaptive clipping. After a single epoch of training, the model improves from 25.37% to 36.76% task pass rate on held-out evaluation tasks. More importantly, these gains transfer to out-of-distribution
benchmarks: +4.5% on BFCL Parallel, +7.4% on Tau2-Bench Retail, and +6.8% on Tool Decathlon (Pass@1). We believe three environment properties are consistent with the observed transfer: task-centric world building that optimizes for diverse, challenging tasks; expert-authored rubrics enabling reliable reward computation; and enterprise workflows that reflect realistic professional patterns. Our results suggest that environment quality, diversity, and realism are key factors enabling generalizable
agent capabilities.
Read the full abstract ▾
All authors Sushant Mehta, Logan Ritchie, Suhaas Garre, Ian Niebres, Nick Heiner, Edwin Chen
Machine Learning · Cross-listed from stat.ML
Conjugate Learning Theory: Uncovering the Mechanisms of Trainability and Generalization in Deep Neural Networks
Binchuan Qi
In this work, we propose a notion of practical learnability grounded in finite sample settings, and develop a conjugate learning theoretical framework based on convex conjugate duality to characterize this learnability property. Building on this foundation, we demonstrate that training deep neural networks (DNNs) with mini-batch
stochastic gradient descent (SGD) achieves global optima of empirical risk by jointly controlling the extreme
eigenvalues of a structure
matrix and the gradient energy, and we establish a corresponding convergence
theorem. We further elucidate the impact of batch size and model architecture (including depth, parameter count, sparsity, skip connections, and other characteristics) on non-convex optimization. Additionally, we derive a model-agnostic lower bound for the achievable empirical risk, theoretically demonstrating that data determines the fundamental limit of trainability. On the generalization front, we derive deterministic and probabilistic bounds on generalization error based on generalized conditional
entropy measures. The former explicitly delineates the range of generalization error, while the latter characterizes the distribution of generalization error relative to the deterministic bounds under independent and identically distributed (i.i.d.) sampling conditions. Furthermore, these bounds explicitly quantify the influence of three key factors: (i) information loss induced by irreversibility in the model, (ii) the maximum attainable loss value, and (iii) the generalized conditional
entropy of features with respect to labels. Moreover, they offer a unified theoretical lens for understanding the roles of regularization, irreversible transformations, and network depth in shaping the generalization behavior of deep neural networks. Extensive experiments validate all theoretical predictions, confirming the framework's correctness and consistency.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
Learning Personalized Agents from Human Feedback
Kaiqu Liang, Julia Kruk, Shengyi Qian et al. (+9 more)
Modern AI
agents are powerful but often fail to align with the idiosyncratic, evolving preferences of individual users. Prior approaches typically rely on static
datasets, either training implicit preference models on interaction history or encoding user profiles in external
memory. However, these approaches struggle with new users and with preferences that change over time. We introduce Personalized
Agents from Human Feedback (PAHF), a framework for continual personalization in which
agents learn online from live interaction using explicit per-user
memory. PAHF operationalizes a three-step loop: (1) seeking pre-action clarification to resolve ambiguity, (2) grounding actions in preferences retrieved from
memory, and (3) integrating post-action feedback to update
memory when preferences drift. To evaluate this capability, we develop a four-phase protocol and two
benchmarks in
embodied manipulation and online shopping. These
benchmarks quantify an
agent's ability to learn initial preferences from scratch and subsequently adapt to persona shifts. Our theoretical analysis and empirical results show that integrating explicit
memory with dual feedback channels is critical: PAHF learns substantially faster and consistently outperforms both no-
memory and single-channel baselines, reducing initial personalization error and enabling rapid adaptation to preference shifts.
Read the full abstract ▾
All authors Kaiqu Liang, Julia Kruk, Shengyi Qian, Xianjun Yang, Shengjie Bi, Yuanshun Yao, Shaoliang Nie, Mingyang Zhang, Lijuan Liu, Jaime Fernández Fisac, Shuyan Zhou, Saghar Hosseini
Machine Learning
Discrete Stochastic Localization for Non-autoregressive Generation
Yunshu Wu, Jiayi Cheng, Longxuan Yu et al. (+4 more)
Continuous
diffusion is a natural framework for non-autoregressive generation but has generally lagged behind masked discrete
diffusion models (MDMs) on discrete sequence generation. We argue that the bottleneck is not continuity itself, but a representation in which denoising depends on timestep-indexed noise regimes. We introduce Discrete
Stochastic Localization (DSL), a continuous-state framework with unit-sphere
token embeddings whose Bayes-optimal denoiser is invariant to the nominal signal-to-noise ratio (SNR) under the localization channel. One trained network then supports an entire family of per-
token SNR paths, with endpoint masked-
diffusion paths as a special case. Fine-tuning a pretrained MDLM checkpoint with DSL substantially improves distributional faithfulness (MAUVE) on OpenWebText across all step budgets from $T{=}128$ to $T{=}1024$, and the same checkpoint supports random-order autoregressive sampling, as well as a hybrid continuous-then-discrete sampler using as few as T=48 total steps – without
distillation or retraining.
Read the full abstract ▾
All authors Yunshu Wu, Jiayi Cheng, Longxuan Yu, Partha Thakuria, Rob Brekelmans, Evangelos E. Papalexakis, Greg Ver Steeg
Machine Learning
Muon with Spectral Guidance: Efficient Optimization for Scientific Machine Learning
Binghang Lu, Jiahao Zhang and Guang Lin
Physics-informed neural networks and neural operators often suffer from severe optimization difficulties caused by ill-conditioned gradients, multi-scale spectral behavior, and stiffness induced by physical constraints. Recently, the
Muon optimizer has shown promise by performing orthogonalized updates in the singular-vector basis of the gradient, thereby improving
geometric conditioning. However, its unit-singular-value updates may lead to overly aggressive steps and lack explicit stability guarantees when applied to physics-informed learning. In this work, we propose SpecMuon, a spectral-aware optimizer that integrates
Muon's orthogonalized geometry with a mode-wise relaxed scalar auxiliary variable (RSAV) mechanism. By decomposing
matrix-valued gradients into singular modes and applying RSAV updates individually along dominant spectral directions, SpecMuon adaptively regulates step sizes according to the global loss energy while preserving
Muon's scale-balancing properties. This formulation interprets optimization as a multi-mode gradient flow and enables principled control of stiff spectral components. We establish rigorous theoretical properties of SpecMuon, including a modified energy dissipation law, positivity and boundedness of auxiliary variables, and global convergence with a linear rate under the Polyak-Lojasiewicz condition. Numerical experiments on physics-informed neural networks, DeepONets, and fractional PINN-DeepONets demonstrate that SpecMuon achieves faster convergence and improved stability compared with Adam, AdamW, and the original
Muon optimizer on
benchmark problems such as the one-dimensional Burgers equation and fractional partial differential equations.
Read the full abstract ▾
Machine Learning
HiPER: Hierarchical Reinforcement Learning with Explicit Credit Assignment for Large Language Model Agents
Jiangweizhi Peng, Yuanxin Liu, Ruida Zhou et al. (+4 more)
Training LLMs as interactive
agents for multi-turn decision-making remains challenging, particularly in long-horizon tasks with sparse and delayed rewards, where
agents must execute extended sequences of actions before receiving meaningful feedback. Most existing reinforcement learning (RL) approaches model LLM
agents as flat policies operating at a single time scale, selecting one action at each turn. In sparse-reward settings, such flat policies must propagate credit across the entire trajectory without explicit temporal abstraction, which often leads to unstable optimization and inefficient credit assignment. We propose HiPER, a novel Hierarchical Plan-Execute RL framework that explicitly separates high-level planning from low-level execution. HiPER factorizes the policy into a high-level planner that proposes subgoals and a low-level executor that carries them out over multiple action steps. To align optimization with this structure, we introduce a key technique called hierarchical advantage estimation (HAE), which carefully assigns credit at both the planning and execution levels. By aggregating returns over the execution of each subgoal and coordinating updates across the two levels, HAE provides an unbiased gradient
estimator and provably reduces variance compared to flat generalized advantage estimation. Empirically, HiPER achieves state-of-the-art performance on challenging interactive
benchmarks, reaching 97.4% success on ALFWorld and 83.3% on WebShop with Qwen2.5-7B-Instruct (+6.6% and +8.3% over the best prior method), with especially large gains on long-horizon tasks requiring multiple dependent subtasks. These results highlight the importance of explicit hierarchical decomposition for scalable RL training of multi-turn LLM
agents.
Read the full abstract ▾
All authors Jiangweizhi Peng, Yuanxin Liu, Ruida Zhou, Charles Fleming, Zhaoran Wang, Alfredo Garcia, Mingyi Hong
Machine Learning · Cross-listed from cs.MM
Rong Fu, Ziming Wang, Shuo Yin et al. (+3 more)
Emotional expression underpins natural communication and effective human-computer interaction. We present
Emotion Collider (EC-Net), a
hyperbolic hypergraph framework for
multimodal emotion and sentiment modeling. EC-Net represents modality hierarchies using Poincare-ball
embeddings and performs fusion through a hypergraph mechanism that passes messages bidirectionally between nodes and hyperedges. To sharpen class separation, contrastive learning is formulated in
hyperbolic space with decoupled radial and angular objectives. High-order semantic relations across time steps and modalities are preserved via adaptive hyperedge construction. Empirical results on standard
multimodal emotion benchmarks show that EC-Net produces robust, semantically coherent representations and consistently improves accuracy, particularly when modalities are partially available or contaminated by noise. These findings indicate that explicit hierarchical geometry combined with hypergraph fusion is effective for resilient
multimodal affect understanding.
Read the full abstract ▾
All authors Rong Fu, Ziming Wang, Shuo Yin, Kun Liu, Xianda Li, Simon Fong
Machine Learning
Differentially Private Non-convex Distributionally Robust Optimization
Difei Xu, Meng Ding, Zebin Ma et al. (+4 more)
Real-world deployments routinely face distribution shifts, group imbalances, and
adversarial perturbations, under which the traditional Empirical Risk Minimization (ERM) framework can degrade severely. Distributionally Robust Optimization (DRO) addresses this issue by optimizing the worst-case expected loss over an uncertainty set of distributions, offering a principled approach to robustness. Meanwhile, as training data in DRO always involves sensitive information, safeguarding it against leakage under Differential
Privacy (DP) is essential. In contrast to classical DP-ERM, DP-DRO has received much less
attention due to its minimax optimization structure with uncertainty constraint. To bridge the gap, we provide a comprehensive study of DP-(finite-sum)-DRO with $\psi$-divergence and non-convex loss. First, we study DRO with general $\psi$-divergence by reformulating it as a minimization problem, and develop a novel $(\varepsilon, \delta)$-DP optimization method, called DP Double-Spider, tailored to this structure. Under mild assumptions, we show that it achieves a utility bound of $\mathcal{O}(\frac{1}{\sqrt{n}}+ (\frac{\sqrt{d \log (1/\delta)}}{n \varepsilon})^{2/3})$ in terms of the gradient norm, where $n$ denotes the data size and $d$ denotes the model dimension. We further improve the utility rate for specific divergences. In particular, for DP-DRO with KL-divergence, by transforming the problem into a compositional finite-sum optimization problem, we develop a DP Recursive-Spider method and show that it achieves a utility bound of $\mathcal{O}((\frac{\sqrt{d \log(1/\delta)}}{n\varepsilon})^{2/3} )$, matching the best-known result for non-convex DP-ERM. Experimentally, we demonstrate that our proposed methods outperform existing approaches for DP minimax optimization.
Read the full abstract ▾
All authors Difei Xu, Meng Ding, Zebin Ma, Huanyi Xie, Youming Tao, Aicha Slaitane, Di Wang
Machine Learning · Cross-listed from math.OC
Local adapt-then-combine algorithms for distributed nonsmooth optimization: Achieving provable communication acceleration
Luyao Guo, Xinli Shi, Wenying Xu et al. (+1 more)
This paper is concerned with the distributed composite optimization problem over networks, where
agents aim to minimize a sum of local smooth components and a common nonsmooth term. Leveraging the probabilistic local updates mechanism, we propose a communication-efficient Adapt-Then-Combine (ATC) framework, FlexATC, unifying numerous ATC-based distributed
algorithms. Under stepsizes independent of the network
topology and the number of local updates, we establish sublinear and linear convergence rates for FlexATC in convex and strongly convex settings, respectively. Remarkably, in the strong convex setting, the linear rate is decoupled from the objective functions and network
topology, and FlexATC permits communication to be skipped in most iterations without any deterioration of the linear rate. In addition, the proposed unified theory demonstrates for the first time that local updates provably lead to communication acceleration for ATC-based distributed
algorithms. Numerical experiments further validate the efficacy of the proposed framework and corroborate the theoretical results.
Read the full abstract ▾
All authors Luyao Guo, Xinli Shi, Wenying Xu, Jinde Cao
Machine Learning
ASPEN: Spectral-Temporal Fusion for Cross-Subject Brain Decoding
Megan Lee, Seung Ha Hwang, Inhyeok Choi et al. (+3 more)
Cross-subject generalization in
EEG-based
brain-computer interfaces (BCIs) remains challenging due to individual variability in neural signals. We investigate whether spectral representations offer more stable features for cross-subject transfer than temporal waveforms. Through correlation analyses across three
EEG paradigms (SSVEP, P300, and Motor Imagery), we find that spectral features exhibit consistently higher cross-subject similarity than temporal signals. Motivated by this observation, we introduce ASPEN, a hybrid architecture that combines spectral and temporal feature streams via multiplicative fusion, requiring cross-modal agreement for features to propagate. Experiments across six
benchmark datasets reveal that ASPEN is able to dynamically achieve the optimal spectral-temporal balance depending on the paradigm. ASPEN achieves the best unseen-subject accuracy on three of six
datasets and competitive performance on others, demonstrating that multiplicative
multimodal fusion enables effective cross-subject generalization.
Read the full abstract ▾
All authors Megan Lee, Seung Ha Hwang, Inhyeok Choi, Shreyas Darade, Mengchun Zhang, Kateryna Shapovalenko
Machine Learning
Investigating GNN Convergence on Large Randomly Generated Graphs with Realistic Node Feature Correlations
Mohammed Zain Ali Ahmed
There are a number of existing studies analysing the convergence behaviour of graph neural networks on large random graphs. Unfortunately, the majority of these studies do not model correlations between node features, which would naturally exist in a variety of real-life networks. Consequently, the derived limitations of GNNs, resulting from such convergence behaviour, is not truly reflective of the expressive power of GNNs when applied to realistic graphs. In this paper, we will introduce a novel method to generate random graphs that have correlated node features. The node features will be sampled in such a manner to ensure correlation between neighbouring nodes. As motivation for our choice of sampling scheme, we will appeal to properties exhibited by real-life graphs, particularly properties that are captured by the Barabási-Albert model. A theoretical analysis will strongly indicate that convergence can be avoided in some cases, which we will empirically validate on large random graphs generated using our novel method. The observed divergent behaviour provides evidence that GNNs may be more expressive than initial studies would suggest, especially on realistic graphs.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Missing-by-Design: Certifiable Modality Deletion for Revocable Multimodal Sentiment Analysis
Rong Fu, Ziming Wang, Chunlei Meng et al. (+4 more)
As
multimodal systems increasingly process sensitive personal data, the ability to selectively revoke specific data modalities has become a critical requirement for
privacy compliance and user autonomy. We present Missing-by-Design (MBD), a unified framework for revocable
multimodal sentiment analysis that combines structured representation learning with a certifiable parameter-modification pipeline. Revocability is critical in
privacy-sensitive applications where users or regulators may request removal of modality-specific information. MBD learns property-aware
embeddings and employs generator-based reconstruction to recover missing channels while preserving task-relevant signals. For deletion requests, the framework applies saliency-driven candidate selection and a calibrated Gaussian update to produce a machine-verifiable Modality Deletion Certificate. Experiments on
benchmark datasets show that MBD achieves strong predictive performance under incomplete inputs and delivers a practical
privacy-utility trade-off, positioning
surgical unlearning as an efficient alternative to full retraining.
Read the full abstract ▾
All authors Rong Fu, Ziming Wang, Chunlei Meng, Jiekai Wu, Kangan Qian, Hao Zhang, Simon Fong
Machine Learning · Cross-listed from math.ST
Ratio Covers of Convex Sets and Optimal Mixture Density Estimation
Spencer Compton, Gábor Lugosi, Jaouad Mourtada et al. (+2 more)
We study density estimation in Kullback-Leibler divergence: given an i.i.d. sample from an unknown density $p^\star$, the goal is to construct an
estimator $\widehat{p}$ such that $\mathrm{KL}(p^\star,\widehat{p})$ is small with high probability. We consider two fundamental settings involving a finite dictionary of densities: (i) model aggregation, where $p^\star$ belongs to the dictionary, and (ii) convex aggregation (mixture density estimation), where $p^\star$ is a mixture of densities from the dictionary. Crucially, we make no assumption on the base densities: their ratios may be unbounded and their supports may differ. For both problems, we identify the best possible high-probability guarantees in terms of the dictionary size, sample size, and confidence level. These optimal rates are higher than those achievable when density ratios are bounded by absolute constants; for mixture density estimation, they match existing lower bounds in the special case of discrete distributions. Our analysis of the mixture case hinges on two new covering results. First, we provide a sharp, distribution-free upper bound on the local Hellinger
entropy of the class of mixtures of $M$ distributions. Second, we prove an optimal ratio covering
theorem for convex sets: for every convex compact set $K \subset \mathbb{R}_+^d$, there exists a subset $A \subset K$ with at most $2^{O(d)}$ elements such that each element of $K$ is coordinate-wise dominated by an element of $A$ up to a universal constant factor. This
geometric result is of independent interest; notably, it yields new cardinality estimates for $\varepsilon$-approximate Pareto sets in multi-objective optimization with convex feasible set.
Read the full abstract ▾
All authors Spencer Compton, Gábor Lugosi, Jaouad Mourtada, Jian Qian, Nikita Zhivotovskiy
Machine Learning · Cross-listed from cs.CV
CHAI: CacHe Attention Inference for text2video
Joel Mathew Cherian, Ashutosh Muralidhara Bharadwaj, Vima Gupta et al. (+1 more)
Text-to-
video diffusion models deliver impressive results but remain slow because of the sequential denoising of 3D latents. Existing approaches to speed up inference either require expensive model retraining or use heuristic-based step skipping, which struggles to maintain
video quality as the number of denoising steps decreases. Our work, CHAI, aims to use cross-inference caching to reduce latency while maintaining
video quality. We introduce Cache
Attention as an effective method for attending to shared objects/scenes across cross-inference latents. This selective
attention mechanism enables effective reuse of cached latents across semantically related
prompts, yielding high cache hit rates. We show that it is possible to generate high-quality
videos using Cache
Attention with as few as 8 denoising steps. When integrated into the overall system, CHAI is 1.65x - 3.35x faster than baseline OpenSora 1.2 while maintaining
video quality.
Read the full abstract ▾
All authors Joel Mathew Cherian, Ashutosh Muralidhara Bharadwaj, Vima Gupta, Anand Padmanabha Iyer
Machine Learning · Cross-listed from stat.ML
Empirical Cumulative Distribution Function Clustering for LLM-based Agent System Analysis
Chihiro Watanabe and Jingyu Sun
Large
language models (LLMs) are increasingly used as
agents to solve complex tasks such as question answering (QA), scientific debate, and software development. A standard evaluation procedure aggregates multiple responses from LLM
agents into a single final answer, often via majority voting, and compares it against reference answers. However, this process can obscure the quality and distributional characteristics of the original responses. In this paper, we propose a novel evaluation framework based on the empirical cumulative distribution function (ECDF) of cosine similarities between generated responses and reference answers. This enables a more nuanced assessment of response quality beyond exact match metrics. To analyze the response distributions across different
agent configurations, we further introduce a clustering method for ECDFs using their distances and the $k$-medoids
algorithm. Our experiments on a QA
dataset demonstrate that ECDFs can distinguish between
agent settings with similar final accuracies but different quality distributions. The clustering analysis also reveals interpretable group structures in the responses, offering insights into the impact of temperature, persona, and question topics.
Read the full abstract ▾
Machine Learning
On the Power of Source Screening for Learning Shared Feature Extractors
Leo Muxing Wang, Connor Mclaughlin and Lili Su
Learning with shared representation is widely recognized as an effective way to separate commonalities from heterogeneity across various heterogeneous sources. Most existing work includes all related data sources via simultaneously training a common feature extractor and source-specific heads. It is well understood that data sources with low relevance or poor quality may hinder representation learning. In this paper, we further dive into the question of which data sources should be learned jointly by focusing on the traditionally deemed “good” collection of sources, in which individual sources have similar relevance and qualities with respect to the true underlying common structure. Towards tractability, we focus on the linear setting where sources share a low-dimensional subspace. We find that source screening can play a central role in statistically optimal subspace estimation. We show that, for a broad class of problem instances, training on a carefully selected subset of sources suffices to achieve minimax optimality, even when a substantial portion of data is discarded. We formalize the notion of an informative subpopulation, develop
algorithms and practical heuristics for identifying such subsets, and validate their effectiveness through both theoretical analysis and empirical evaluations on synthetic and real-world
datasets.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.IR
Rethinking ANN-based Retrieval: Multifaceted Learnable Index for Large-scale Recommendation System
Jiang Zhang, Yubo Wang, Wei Chang et al. (+14 more)
Approximate nearest neighbor (ANN) search is widely used in the
retrieval stage of large-scale recommendation systems. In this stage, candidate items are indexed using their learned
embedding vectors, and ANN search is executed for each user (or item) query to retrieve a set of relevant items. However, ANN-based
retrieval has two key limitations. First, item
embeddings and their indices are typically learned in separate stages: indexing is often performed offline after
embeddings are trained, which can yield suboptimal
retrieval quality-especially for newly created items. Second, although ANN offers sublinear query time, it must still be run for every request, incurring substantial computation cost at industry scale. In this paper, we propose MultiFaceted Learnable Index (MFLI), a scalable, real-time
retrieval paradigm that learns multifaceted item
embeddings and indices within a unified framework and eliminates ANN search at serving time. Specifically, we construct a multifaceted hierarchical codebook via residual quantization of item
embeddings and co-train the codebook with the
embeddings. We further introduce an efficient multifaceted indexing structure and mechanisms that support real-time updates. At serving time, the learned hierarchical indices are used directly to identify relevant items, avoiding ANN search altogether. Extensive experiments on real-world data with billions of users show that MFLI improves recall on engagement tasks by up to 11.8%, cold-content delivery by up to 57.29%, and semantic relevance by 13.5% compared with prior state-of-the-art methods. We also deploy MFLI in the system and report online experimental results demonstrating improved engagement, less popularity bias, and higher serving efficiency.
Read the full abstract ▾
All authors Jiang Zhang, Yubo Wang, Wei Chang, Lu Han, Xingying Cheng, Feng Zhang, Min Li, Songhao Jiang, Wei Zheng, Harry Tran, Zhen Wang, Lei Chen, Yueming Wang, Benyu Zhang, Xiangjun Fan, Bi Xue, Qifan Wang
Machine Learning
Feature-based morphological analysis of shape graph data
Murad Hossen, Demetrio Labate and Nicolas Charon
This paper introduces and demonstrates a computational pipeline for the statistical analysis of shape graph
datasets, namely
geometric networks embedded in 2D or 3D spaces. Unlike traditional abstract graphs, our purpose is not only to retrieve and distinguish variations in the connectivity structure of the data but also
geometric differences of the network branches. Our proposed approach relies on the extraction of a specifically curated and explicit set of topological,
geometric and directional features, designed to satisfy key invariance properties. We leverage the resulting feature representation for tasks such as group comparison, clustering and classification on cohorts of shape graphs. The effectiveness of this representation is evaluated on several real-world
datasets including urban road/street networks, neuronal traces and astrocyte imaging. These results are benchmarked against several alternative methods, both feature-based and not.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.NE
Evolutionary Context Search for Automated Skill Acquisition
Qi Sun, Stefan Nielsen, Rio Yokota et al. (+1 more)
Large
Language Models cannot reliably acquire new knowledge post-deployment – even when relevant text resources exist, models fail to transform them into actionable knowledge without retraining.
Retrieval-Augmented Generation attempts to bridge this gap by surfacing relevant documents at inference time, yet similarity-based
retrieval often fails to identify context that actually improves task performance. We introduce Evolutionary Context Search (ECS), an evolutionary method that searches context combinations using accuracy on a small development set, requiring only inference calls without weight updates. ECS moves beyond semantic similarity to discover non-obvious context pairings that significantly boost performance. Our empirical results show that ECS improves BackendBench by 27% and $\tau$-bench airline by 7%. The evolved contexts are model-agnostic, as those evolved with Gemini-3-Flash transfer effectively to Claude Sonnet and DeepSeek. This suggests that ECS opens a path toward automated context discovery for skill acquisition – an efficient alternative to manual
prompt engineering or costly fine-tuning.
Read the full abstract ▾
All authors Qi Sun, Stefan Nielsen, Rio Yokota, Yujin Tang
Machine Learning
Axle Sensor Fusion for Online Continual Wheel Fault Detection in Wayside Railway Monitoring
Afonso Lourenço, Francisca Osório, Diogo Risca et al. (+1 more)
Reliable and cost-effective maintenance is essential for railway safety, particularly at the wheel-rail interface, which is prone to wear and failure. Predictive maintenance frameworks increasingly leverage sensor-generated time-series data, yet traditional methods require manual feature engineering, and deep learning models often degrade in online settings with evolving operational patterns. This work presents a semantic-aware, label-efficient continual learning framework for railway fault diagnostics. Accelerometer signals are encoded via a Variational AutoEncoder into latent representations capturing the normal operational structure in a fully unsupervised manner. Importantly, semantic metadata, including axle counts, wheel indexes, and strain-based deformations, is extracted via AI-driven peak detection on fiber Bragg grating sensors (resistant to electromagnetic interference) and fused with the VAE
embeddings, enhancing anomaly detection under unknown operational conditions. A lightweight gradient boosting supervised classifier stabilizes anomaly scoring with minimal labels, while a replay-based continual learning strategy enables adaptation to evolving domains without catastrophic forgetting. Experiments show the model detects minor imperfections due to flats and polygonization, while adapting to evolving operational conditions, such as changes in train type, speed, load, and track profiles, captured using a single accelerometer and strain gauge in wayside monitoring.
Read the full abstract ▾
All authors Afonso Lourenço, Francisca Osório, Diogo Risca, Goreti Marreiros
Machine Learning · Cross-listed from cs.CR
Collaborative Zone-Adaptive Zero-Day Intrusion Detection for IoBT
Amirmohammad Pasdar, Shabnam Kasra Kermanshahi, Nour Moustafa et al. (+1 more)
The Internet of Battlefield Things (IoBT) relies on heterogeneous, bandwidth-constrained, and intermittently connected tactical networks that face rapidly evolving cyber threats. In this setting, intrusion detection cannot depend on continuous central collection of raw traffic due to disrupted links, latency, operational security limits, and non-IID traffic across zones. We present Zone-Adaptive Intrusion Detection (ZAID), a collaborative detection and model-improvement framework for unseen attack types, where "zero-day" refers to previously unobserved attack families and behaviours (not vulnerability disclosure timing). ZAID combines a universal convolutional model for generalisable traffic representations, an autoencoder-based reconstruction signal as an auxiliary anomaly score, and lightweight adapter modules for parameter-efficient zone adaptation. To support cross-zone generalisation under constrained connectivity, ZAID uses
federated aggregation and pseudo-labelling to leverage locally observed, weakly labelled behaviours. We evaluate ZAID on ToN_IoT using a zero-day protocol that excludes MITM, DDoS, and DoS from supervised training and introduces them during zone-level deployment and adaptation. ZAID achieves up to 83.16% accuracy on unseen attack traffic and transfers to UNSW-NB15 under the same procedure, with a best accuracy of 71.64%. These results indicate that parameter-efficient, zone-personalised collaboration can improve the detection of previously unseen attacks in contested IoBT environments.
Read the full abstract ▾
All authors Amirmohammad Pasdar, Shabnam Kasra Kermanshahi, Nour Moustafa, Van-Thuan Pham