Machine Learning · Cross-listed from cs.MA
In Trust We Survive: Emergent Trust Learning
Qianpu Chen, Giulio Barbero, Mike Preuss et al. (+1 more)
We introduce Emergent Trust Learning (ETL), a lightweight, trust-based control
algorithm that can be plugged into existing AI
agents. It enables these to reach cooperation in competitive game environments under shared resources. Each
agent maintains a compact internal trust state, which modulates
memory, exploration, and action selection. ETL requires only individual rewards and local observations and incurs negligible computational and communication overhead. We evaluate ETL in three environments: In a grid-based resource world, trust-based
agents reduce conflicts and prevent long-term resource depletion while achieving competitive individual returns. In a hierarchical Tower environment with strong social dilemmas and randomised floor assignments, ETL sustains high survival rates and recovers cooperation even after extended phases of enforced greed. In the Iterated Prisoner's Dilemma, the
algorithm generalises to a strategic meta-game, maintaining cooperation with reciprocal opponents while avoiding long-term exploitation by defectors. Code will be released upon publication.
Read the full abstract ▾
All authors Qianpu Chen, Giulio Barbero, Mike Preuss, Derya Soydaner
Machine Learning · Cross-listed from stat.ML
Consistency of the $k$-Nearest Neighbor Regressor under Complex Survey Designs
Caren Hasler
We study the consistency of the $k$-nearest neighbor regressor under complex survey designs. While consistency results for this
algorithm are well established for independent and identically distributed data, corresponding results for complex survey data are lacking. We show that the $k$-nearest neighbor regressor is consistent under regularity conditions on the sampling design and the distribution of the data. We derive lower bounds for the rate of convergence and show that these bounds exhibit the curse of dimensionality, as in the independent and identically distributed setting. Empirical studies based on simulated and real data illustrate our theoretical findings.
Read the full abstract ▾
Machine Learning
Conditional Inverse Learning of Time-Varying Reproduction Numbers Inference
Lanlan Yu, Quan-Hui Liu, Haoyue Zheng et al. (+1 more)
Estimating time-varying reproduction numbers from epidemic incidence data is a central task in infectious disease surveillance, yet it poses an inherently ill-posed inverse problem. Existing approaches often rely on strong structural assumptions derived from epidemiological models, which can limit their ability to adapt to non-stationary transmission dynamics induced by interventions or behavioral changes, leading to delayed detection of regime shifts and degraded estimation accuracy. In this work, we propose a Conditional Inverse Reproduction Learning framework (CIRL) that addresses the inverse problem by learning a {conditional mapping} from historical incidence patterns and explicit time information to latent reproduction numbers. Rather than imposing strongly enforced parametric constraints, CIRL softly integrates epidemiological structure with flexible likelihood-based statistical modeling, using the renewal equation as a forward operator to enforce dynamical consistency. The resulting framework combines epidemiologically grounded constraints with data-driven temporal representations, producing reproduction number estimates that are robust to observation noise while remaining responsive to abrupt transmission changes and zero-inflated incidence observations. Experiments on synthetic epidemics with controlled regime changes and real-world SARS and COVID-19 data demonstrate the effectiveness of the proposed approach.
Read the full abstract ▾
All authors Lanlan Yu, Quan-Hui Liu, Haoyue Zheng, Xinfu Yang
Machine Learning
CLeAN: Continual Learning Adaptive Normalization in Dynamic Environments
Isabella Marasco, Davide Evangelista, Elena Loli Piccolomini et al. (+1 more)
Artificial intelligence systems predominantly rely on static data distributions, making them ineffective in dynamic real-world environments, such as cybersecurity, autonomous transportation, or finance, where data shifts frequently. Continual learning offers a potential solution by enabling models to learn from sequential data while retaining prior knowledge. However, a critical and underexplored issue in this domain is data normalization. Conventional normalization methods, such as min-max scaling, presuppose access to the entire
dataset, which is incongruent with the sequential nature of continual learning. In this paper we introduce Continual Learning Adaptive Normalization (CLeAN), a novel adaptive normalization technique designed for continual learning in tabular data. CLeAN involves the estimation of global feature scales using learnable parameters that are updated via an Exponential Moving Average (EMA) module, enabling the model to adapt to evolving data distributions. Through comprehensive evaluations on two
datasets and various continual learning strategies, including Resevoir Experience Replay, A-GEM, and EwC we demonstrate that CLeAN not only improves model performance on new data but also mitigates catastrophic forgetting. The findings underscore the importance of adaptive normalization in enhancing the stability and effectiveness of tabular data, offering a novel perspective on the use of normalization to preserve knowledge in dynamic learning environments.
Read the full abstract ▾
All authors Isabella Marasco, Davide Evangelista, Elena Loli Piccolomini, Michele Colajanni
Machine Learning · Cross-listed from cs.AI
Per-Domain Generalizing Policies: On Learning Efficient and Robust Q-Value Functions (Extended Version with Technical Appendix)
Nicola J. Müller, Moritz Oster, Isabel Valera et al. (+2 more)
Learning per-domain generalizing policies is a key challenge in learning for planning. Standard approaches learn state-value functions represented as graph neural networks using supervised learning on optimal plans generated by a teacher planner. In this work, we advocate for learning Q-value functions instead. Such policies are drastically cheaper to evaluate for a given state, as they need to process only the current state rather than every successor. Surprisingly, vanilla supervised learning of Q-values performs poorly as it does not learn to distinguish between the actions taken and those not taken by the teacher. We address this by using regularization terms that enforce this distinction, resulting in Q-value policies that consistently outperform state-value policies across a range of 10 domains and are competitive with the planner LAMA-first.
Read the full abstract ▾
All authors Nicola J. Müller, Moritz Oster, Isabel Valera, Jörg Hoffmann, Timo P. Gros
Machine Learning · Cross-listed from cs.IR
Deploying Semantic ID-based Generative Retrieval for Large-Scale Podcast Discovery at Spotify
Edoardo D'Amico, Marco De Nadai, Praveen Chandar et al. (+41 more)
Podcast listening is often grounded in a set of favorite shows, while listener intent can evolve over time. This combination of stable preferences and changing intent motivates recommendation approaches that support both familiarity and exploration. Traditional recommender systems typically emphasize long-term interaction patterns, and are less explicitly designed to incorporate rich contextual signals or flexible, intent-aware discovery objectives. In this setting, models that can jointly reason over semantics, context, and user state offer a promising direction. Large
Language Models (LLMs) provide strong semantic
reasoning and contextual conditioning for discovery-oriented recommendation, but deploying them in production introduces challenges in catalog grounding, user-level personalization, and latency-critical serving. We address these challenges with GLIDE, a production-scale generative recommender for podcast discovery at Spotify. GLIDE formulates recommendation as an instruction-following task over a discretized catalog using Semantic IDs, enabling grounded generation over a large inventory. The model conditions on recent listening history and lightweight user context, while injecting long-term user
embeddings as soft
prompts to capture stable preferences under strict inference constraints. We evaluate GLIDE using offline
retrieval metrics, human judgments, and LLM-based evaluation, and validate its impact through large-scale online A/B testing. Across experiments involving millions of users, GLIDE increases non-habitual podcast streaming on Spotify home surface by up to 5.4% and new-show discovery by up to 14.3%, while meeting production cost and latency constraints.
Read the full abstract ▾
All authors Edoardo D'Amico, Marco De Nadai, Praveen Chandar, Divita Vohra, Shawn Lin, Max Lefarov, Paul Gigioli, Gustavo Penha, Ilya Kopysitsky, Ivo Joel Senese, Darren Mei, Francesco Fabbri, Oguz Semerci, Yu Zhao, Vincent Tang, Brian St. Thomas, Alexandra Ranieri, Matthew N.K. Smith, Aaron Bernkopf, Bryan Leung, Ghazal Fazelnia, Mark VanMiddlesworth, Timothy Christopher Heath, Petter Pehrson Skiden, Alice Y. Wang, Doug J. Cole, Andreas Damianou, Maya Hristakeva, Reid Wilbur, Tarun Chillara, Vladan Radosavljevic, Pooja Chitkara, Sainath Adapa, Juan Elenter, Bernd Huber, Jacqueline Wood, Saaketh Vedantam, Jan Stypka, Sandeep Ghael, Martin D. Gould, David Murgatroyd, Yves Raimond, Mounia Lalmas, Paul N. Bennett
Machine Learning
PCA-Based Interpretable Knowledge Representation and Analysis of Geometric Design Parameters
Alexander Köhler and Michael Breuß
In many CAD-based applications, complex geometries are defined by a high number of design parameters. This leads to high-dimensional design spaces that are challenging for downstream engineering processes like simulations, optimization, and design exploration tasks. Therefore, dimension reduction methods such as principal component analysis (PCA) are used. The PCA identifies dominant modes of
geometric variation and yields a compact representation of the geometry. While classical PCA excels in the compact representation part, it does not directly recover underlying design parameters of a generated geometry. In this work, we deal with the problem of estimating design parameters from PCA-based representations. Analyzing a recent modification of the PCA dedicated to our field of application, we show that the results are actually identical to the standard PCA. We investigate limitations of this approach and present reasonable conditions under which accurate, interpretable parameter estimation can be obtained. With the help of dedicated experiments, we take a more in-depth look at every stage of the PCA and the possible changes of the geometry during these processes.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
Informative Semi-Factuals for XAI: The Elaborated Explanations that People Prefer
Saugat Aryal and Mark T. Keane
Recently, in eXplainable AI (XAI), $\textit{even if}$ explanations – so-called semi-factuals – have emerged as a popular strategy that explains how a predicted outcome $\textit{can remain the same}$ even when certain input-features are altered. For example, in the commonly-used banking app scenario, a semi-factual explanation could inform customers about better options, other alternatives for their successful application, by saying "$\textit{Even if}$ you asked for double the loan amount, you would still be accepted". Most semi-factuals XAI
algorithms focus on finding maximal value-changes to a single key-feature that do $\textit{not}$ alter the outcome (unlike counterfactual explanations that often find minimal value-changes to several features that alter the outcome). However, no current semi-factual method explains $\textit{why}$ these extreme value-changes do not alter outcomes; for example, a more informative semi-factual could tell the customer that it is their good credit score that allows them to borrow double their requested loan. In this work, we advance a new
algorithm – the $\textit{informative semi-factuals}$ (ISF) method – that generates more elaborated explanations supplementing semi-factuals with information about additional $\textit{hidden features}$ that influence an automated decision. Experimental results on
benchmark datasets show that this ISF method computes semi-factuals that are both informative and of high-quality on key metrics. Furthermore, a user study shows that people prefer these elaborated explanations over the simpler semi-factual explanations generated by current methods.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.IR
A Unified Language Model for Large Scale Search, Recommendation, and Reasoning
Marco De Nadai, Edoardo D'Amico, Max Lefarov et al. (+18 more)
LLMs are increasingly applied to recommendation,
retrieval, and
reasoning, yet deploying a single end-to-end model that can jointly support these behaviors over large, heterogeneous catalogs remains challenging. Such systems must generate unambiguous references to real items, handle multiple entity types, and operate under strict latency and reliability constraints requirements that are difficult to satisfy with text-only generation. While tool-augmented recommender systems address parts of this problem, they introduce orchestration complexity and limit end-to-end optimization. We view this setting as an instance of a broader research problem: how to adapt LLMs to reason jointly over multiple-domain entities, users, and
language in a fully self-contained manner. To this end, we introduce NEO, a framework that adapts a pre-trained decoder-only LLM into a tool-free, catalog-grounded generator. NEO represents items as SIDs and trains a single model to interleave natural
language and typed item identifiers within a shared sequence. Text
prompts control the task, target entity type, and output format (IDs, text, or mixed), while constrained decoding guarantees catalog-valid item generation without restricting free-form text. We refer to this instruction-conditioned controllability as
language-steerability. We treat SIDs as a distinct modality and study design choices for integrating discrete entity representations into LLMs via staged
alignment and instruction tuning. We evaluate NEO at scale on a real-world catalog of over 10M items across multiple media types and discovery tasks, including recommendation, search, and user understanding. In offline experiments, NEO consistently outperforms strong task-specific baselines and exhibits cross-task transfer, demonstrating a practical path toward consolidating large-scale discovery capabilities into a single
language-steerable generative model.
Read the full abstract ▾
All authors Marco De Nadai, Edoardo D'Amico, Max Lefarov, Alexandre Tamborrino, Divita Vohra, Mark VanMiddlesworth, Shawn Lin, Jacqueline Wood, Jan Stypka, Eliza Klyce, Keshi Dai, Timothy Christopher Heath, Martin D. Gould, Yves Raimond, Sandeep Ghael, Tony Jebara, Andreas Damianou, Vladan Radosavljevic, Paul N. Bennett, Mounia Lalmas, Praveen Chandar
Machine Learning
Anisotropic Permeability Tensor Prediction from Porous Media Microstructure via Physics-Informed Progressive Transfer Learning with Hybrid CNN-Transformer
Mohammad Nooraiepour
Accurate prediction of permeability tensors from pore-scale microstructure images is essential for subsurface flow modeling, yet direct numerical simulation requires hours per sample, fundamentally limiting large-scale uncertainty quantification and reservoir optimization workflows. A physics-informed deep learning framework is presented that resolves this bottleneck by combining a MaxViT hybrid CNN-
Transformer architecture with progressive transfer learning and differentiable physical constraints. MaxViT's multi-axis
attention mechanism simultaneously resolves grain-scale pore-throat geometry via block-local operations and REV-scale connectivity statistics through grid-global operations, providing the spatial hierarchy that permeability tensor prediction physically requires. Training on 20000 synthetic porous media samples spanning three orders of magnitude in permeability, a three-phase progressive curriculum advances from an ImageNet-pretrained baseline with D4-equivariant augmentation and tensor transformation, through component-weighted loss prioritizing off-diagonal coupling, to frozen-backbone transfer learning with porosity conditioning via Feature-wise Linear Modulation (FiLM). Onsager reciprocity and positive definiteness are enforced via differentiable penalty terms. On a held-out test set of 4000 samples, the framework achieves variance-weighted R2 = 0.9960 (R2_Kxx = 0.9967, R2_Kxy = 0.9758), a 33% reduction in unexplained variance over the supervised baseline. The results offer three transferable principles for physics-informed scientific machine learning: large-scale visual pretraining transfers effectively across domain boundaries; physical constraints are most robustly integrated as differentiable architectural components; and progressive training guided by diagnostic failure-mode analysis enables unambiguous attribution of performance gains across methodological stages.
Read the full abstract ▾
Machine Learning
AirDDE: Multifactor Neural Delay Differential Equations for Air Quality Forecasting
Binqing Wu, Zongjiang Shang, Shiyu Liu et al. (+3 more)
Accurate air quality
forecasting is essential for public health and environmental sustainability, but remains challenging due to the complex pollutant dynamics. Existing deep learning methods often model pollutant dynamics as an instantaneous process, overlooking the intrinsic delays in pollutant propagation. Thus, we propose AirDDE, the first neural delay differential equation framework in this task that integrates delay modeling into a continuous-time pollutant
evolution under physical guidance. Specifically, two novel components are introduced: (1) a
memory-augmented
attention module that retrieves globally and locally historical features, which can adaptively capture delay effects modulated by multifactor data; and (2) a physics-guided delay evolving function, grounded in the
diffusion-advection equation, that models
diffusion, delayed advection, and source/sink terms, which can capture delay-aware pollutant accumulation patterns with physical plausibility. Extensive experiments on three real-world
datasets demonstrate that AirDDE achieves the state-of-the-art
forecasting performance with an average MAE reduction of 8.79% over the best baselines. The code is available at https://github.com/w2obin/airdde-aaai.
Read the full abstract ▾
All authors Binqing Wu, Zongjiang Shang, Shiyu Liu, Jianlong Huang, Jiahui Xu, Ling Chen
Machine Learning · Cross-listed from stat.ML
Mirror Descent on Riemannian Manifolds
Jiaxin Jiang, Lei Shi and Jiyuan Tan
Mirror Descent (MD) is a scalable first-order method widely used in large-scale optimization, with applications in image processing, policy optimization, and neural network training. This paper generalizes MD to optimization on Riemannian
manifolds. In particular, we develop a Riemannian Mirror Descent (RMD) framework via reparameterization and further propose a
stochastic variant of RMD. We also establish non-asymptotic convergence guarantees for both RMD and
stochastic RMD. As an application to the Stiefel
manifold, our RMD framework reduces to the Curvilinear Gradient Descent (CGD) method proposed in [26]. Moreover, when specializing the
stochastic RMD framework to the Stiefel setting, we obtain a
stochastic extension of CGD, which effectively addresses large-scale
manifold optimization problems.
Read the full abstract ▾
Machine Learning
Translation Invariance of Neural Operators for the FitzHugh-Nagumo Model
Luca Pellegrini
Neural operators (NOs) are powerful deep learning frameworks designed to learn solution operators of partial differential equations. This study evaluates the ability of NOs' to capture the stiff spatio-temporal dynamics of the FitzHugh-Nagumo model. A key contribution of this study is the assessment of the translation invariance using a novel training strategy. Models are trained using an applied current with varying spatial locations and intensities at a fixed time, while the test set presents a challenging out-of-distribution scenario where the current is translated in both time and space. This approach significantly reduces
dataset generation costs. We
benchmark seven NO architectures: Convolutional Neural Operators (CNOs), Deep Operator Networks (DeepONets), DeepONets with CNN encoders, Proper Orthogonal Decomposition DeepONets,
Fourier Neural Operators (FNOs), Tucker Tensorized FNOs, and Local Neural Operators. We evaluated these models based on their accuracy, efficiency, and inference speed. These results demonstrate that CNOs generalize well to translated test dynamics, whereas other architectures do not generalize well. On the training set, all architectures achieve comparable accuracy, with FNOs achieving the highest precision. However, this higher accuracy comes at an elevated computational cost. Meanwhile, DeepONets and their variants exhibit superior training and inference efficiency. These findings highlight the capabilities and limitations of NOs in modeling complex ionic dynamics and provide a comprehensive
benchmark for scenarios involving translated dynamics.
Read the full abstract ▾
Machine Learning
QuantFL: Sustainable Federated Learning for Edge IoT via Pre-Trained Model Quantisation
Charuka Herath, Yogachandran Rahulamathavan, Varuna De Silva et al. (+1 more)
Federated Learning (FL) enables
privacy-preserving intelligence on Internet of Things (IoT) devices but incurs a significant carbon footprint due to the high energy cost of frequent uplink transmission. While pre-trained models are increasingly available on edge devices, their potential to reduce the energy overhead of fine-tuning remains underexplored. In this work, we propose QuantFL, a sustainable FL framework that leverages pre-trained initialisation to enable aggressive, computationally lightweight quantisation. We demonstrate that pre-training naturally concentrates update statistics, allowing us to use
memory-efficient bucket quantisation without the energy-intensive overhead of complex error-feedback mechanisms. On MNIST and CIFAR-100, QuantFL reduces total communication by 40% ($\simeq40\%$ total-bit reduction with full-precision downlink; $\geq80\%$ on uplink or when downlink is quantised) while matching or exceeding uncompressed baselines under strict bandwidth budgets; BU attains 89.00% (MNIST) and 66.89% (CIFAR-100) test accuracy with orders of magnitude fewer bits. We also account for uplink and downlink costs and provide ablations on quantisation levels and initialisation. QuantFL delivers a practical, "green" recipe for scalable training on battery-constrained IoT networks.
Read the full abstract ▾
All authors Charuka Herath, Yogachandran Rahulamathavan, Varuna De Silva, Sangarapillai Lambotharan
Machine Learning · Cross-listed from cs.CL
Learning When to Attend: Conditional Memory Access for Long-Context LLMs
Sakshi Choudhary, Aditya Chattopadhyay, Luca Zancato et al. (+4 more)
Language models struggle to generalize beyond pretraining context lengths, limiting long-horizon
reasoning and
retrieval. Continued pretraining on long-context data can help but is expensive due to the quadratic scaling of
Attention. We observe that most
tokens do not require (Global)
Attention over the entire sequence and can rely on local context. Based on this, we propose L2A (Learning To Attend), a layer that enables conditional (
token-wise) long-range
memory access by deciding when to invoke global
attention. We evaluate L2A on Qwen 2.5 and Qwen 3 models, extending their effective context length from 32K to 128K
tokens. L2A matches the performance of standard long-context training to within 3% while skipping Global
Attention for $\sim$80% of
tokens, outperforming prior baselines. We also design custom Triton kernels to efficiently implement this
token-wise conditional
Attention on GPUs, achieving up to $\sim$2$\times$ improvements in training throughput and time-to-first-
token over FlashAttention. Moreover, L2A enables post-training
pruning of highly sparse Global
Attention layers, reducing KV cache
memory by up to 50% with negligible performance loss. Our code is released under Apache 2.0 at https://github.com/awslabs/hybrid-model-factory/tree/main/examples/research/L2A.
Read the full abstract ▾
All authors Sakshi Choudhary, Aditya Chattopadhyay, Luca Zancato, Elvis Nunez, Matthew Trager, Wei Xia, Stefano Soatto
Machine Learning
Auto-Unrolled Proximal Gradient Descent: An AutoML Approach to Interpretable Waveform Optimization
Ahmet Kaplan
This study explores the combination of automated machine learning (AutoML) with model-based deep unfolding (DU) for optimizing wireless beamforming and waveforms. We convert the iterative proximal gradient descent (PGD)
algorithm into a deep neural network, wherein the parameters of each layer are learned instead of being predetermined. Additionally, we enhance the architecture by incorporating a hybrid layer that performs a learnable linear gradient transformation prior to the proximal projection. By utilizing AutoGluon with a tree-structured parzen
estimator (TPE) for hyperparameter optimization (HPO) across an expanded search space, which includes network depth, step-size initialization, optimizer, learning rate scheduler, layer type, and post-gradient activation, the proposed auto-unrolled PGD (Auto-PGD) achieves 98.8% of the spectral efficiency of a traditional 200-iteration PGD solver using only five unrolled layers, while requiring only 100 training samples. We also address a gradient normalization issue to ensure consistent performance during training and evaluation, and we illustrate per-layer sum-rate logging as a tool for transparency. These contributions highlight a notable reduction in the amount of training data and inference cost required, while maintaining high interpretability compared to conventional black-box architectures.
Read the full abstract ▾
Machine Learning
Efficient Soft Actor-Critic with LLM-Based Action-Level Guidance for Continuous Control
Hao Ma, Zhiqiang Pu, Xiaolin Ai et al. (+1 more)
We present GuidedSAC, a novel reinforcement learning (RL)
algorithm that facilitates efficient exploration in vast state-action spaces. GuidedSAC leverages large
language models (LLMs) as intelligent supervisors that provide action-level guidance for the Soft Actor-Critic (SAC)
algorithm. The LLM-based supervisor analyzes the most recent trajectory using state information and visual replays, offering action-level interventions that enable targeted exploration. Furthermore, we provide a theoretical analysis of GuidedSAC, proving that it preserves the convergence guarantees of SAC while improving convergence speed. Through experiments in both discrete and continuous control environments, including toy text tasks and complex MuJoCo
benchmarks, we demonstrate that GuidedSAC consistently outperforms standard SAC and state-of-the-art exploration-enhanced variants (e.g., RND, ICM, and E3B) in terms of sample efficiency and final performance.
Read the full abstract ▾
All authors Hao Ma, Zhiqiang Pu, Xiaolin Ai, Huimu Wang
Machine Learning
Baguan-TS: A Sequence-Native In-Context Learning Model for Time Series Forecasting with Covariates
Linxiao Yang, Xue Jiang, Gezheng Xu et al. (+9 more)
Transformers enable in-context learning (ICL) for rapid, gradient-free adaptation in time series
forecasting, yet most ICL-style approaches rely on tabularized, hand-crafted features, while end-to-end sequence models lack inference-time adaptation. We bridge this gap with a unified framework, Baguan-TS, which integrates the raw-sequence representation learning with ICL, instantiated by a 3D
Transformer that attends jointly over temporal, variable, and context axes. To make this high-capacity model practical, we tackle two key hurdles: (i) calibration and training stability, improved with a feature-agnostic, target-space
retrieval-based local calibration; and (ii) output oversmoothing, mitigated via context-overfitting strategy. On public
benchmark with
covariates, Baguan-TS consistently outperforms established baselines, achieving the highest win rate and significant reductions in both point and probabilistic
forecasting metrics. Further evaluations across diverse real-world energy
datasets demonstrate its robustness, yielding substantial improvements.
Read the full abstract ▾
All authors Linxiao Yang, Xue Jiang, Gezheng Xu, Tian Zhou, Min Yang, ZhaoYang Zhu, Linyuan Geng, Zhipeng Zeng, Qiming Chen, Xinyue Gu, Rong Jin, Liang Sun
Machine Learning
TimeAPN: Adaptive Amplitude-Phase Non-Stationarity Normalization for Time Series Forecasting
Yue Hu, Jialiang Tang, Siwei Yu et al. (+3 more)
Non-stationarity is a fundamental challenge in multivariate long-term time series
forecasting, often manifested as rapid changes in amplitude and phase. These variations lead to severe distribution shifts and consequently degrade predictive performance. Existing normalization-based methods primarily rely on first- and second-order statistics, implicitly assuming that distributions evolve smoothly and overlooking fine-grained temporal dynamics. To address these limitations, we propose TimeAPN, an Adaptive Amplitude-Phase Non-Stationarity Normalization framework that explicitly models and predicts non-stationary factors from both the time and frequency domains. Specifically, TimeAPN first models the mean sequence jointly in the time and frequency domains, and then forecasts its
evolution over future horizons. Meanwhile, phase information is extracted in the frequency domain, and the phase discrepancy between the predicted and ground-truth future sequences is explicitly modeled to capture temporal misalignment. Furthermore, TimeAPN incorporates amplitude information into an adaptive normalization mechanism, enabling the model to effectively account for abrupt fluctuations in signal energy. The predicted non-stationary factors are subsequently integrated with the backbone
forecasting outputs through a collaborative de-normalization process to reconstruct the final non-stationary time series. The proposed framework is model-agnostic and can be seamlessly integrated with various
forecasting backbones. Extensive experiments on seven real-world multivariate
datasets demonstrate that TimeAPN consistently improves long-term
forecasting accuracy across multiple prediction horizons and outperforms state-of-the-art reversible normalization methods.
Read the full abstract ▾
All authors Yue Hu, Jialiang Tang, Siwei Yu, Baosheng Yu, Jing Zhang, Dacheng Tao
Machine Learning · Cross-listed from cs.DC
ZipServ: Fast and Memory-Efficient LLM Inference with Hardware-Aware Lossless Compression
Ruibo Fan, Xiangrui Yu, Xinglin Pan et al. (+5 more)
Lossless model compression holds tremendous promise for alleviating the
memory and bandwidth bottlenecks in bit-exact Large
Language Model (LLM) serving. However, existing approaches often result in substantial inference slowdowns due to fundamental design mismatches with GPU architectures: at the kernel level, variable-length bitstreams produced by traditional
entropy codecs break SIMT parallelism; at the system level, decoupled pipelines lead to redundant
memory traffic. We present ZipServ, a lossless compression framework co-designed for efficient LLM inference. ZipServ introduces Tensor-Core-Aware Triple Bitmap Encoding (TCA-TBE), a novel fixed-length format that enables constant-time, parallel decoding, together with a fused decompression-GEMM (ZipGEMM) kernel that decompresses weights on-the-fly directly into Tensor Core registers. This "load-compressed, compute-decompressed" design eliminates intermediate buffers and maximizes compute intensity. Experiments show that ZipServ reduces the model size by up to 30%, achieves up to 2.21x kernel-level speedup over NVIDIA's cuBLAS, and expedites end-to-end inference by an average of 1.22x over vLLM. ZipServ is the first lossless compression system that provides both storage savings and substantial acceleration for LLM inference on GPUs.
Read the full abstract ▾
All authors Ruibo Fan, Xiangrui Yu, Xinglin Pan, Zeyu Li, Weile Luo, Qiang Wang, Wei Wang, Xiaowen Chu
Machine Learning
The Phasor Transformer: Resolving Attention Bottlenecks on the Unit Circle
Dibakar Sigdel
Transformer models have redefined sequence learning, yet dot-product self-
attention introduces a quadratic
token-mixing bottleneck for long-context time-series. We introduce the Phasor
Transformer block, a phase-native alternative representing sequence states on the unit-circle
manifold $S^1$. Each block combines lightweight trainable phase-shifts with parameter-free Discrete
Fourier Transform (DFT)
token coupling, achieving global $\mathcal{O}(N\log N)$ mixing without explicit
attention maps. Stacking these blocks defines the Large Phasor Model (LPM). We validate LPM on autoregressive time-series prediction over synthetic multi-frequency
benchmarks against honest baselines: it beats a zero-parameter persistence baseline and, with the corrected gradient path, improves monotonically with depth before saturating, while remaining competitive-but-not-superior to self-
attention at a fraction of the parameter count. Our results establish an explicit efficiency–accuracy frontier, showing that scalable temporal modeling in oscillatory domains can emerge from geometry-constrained phase computation with deterministic global coupling.
Read the full abstract ▾
Machine Learning · Cross-listed from math.DS
Data-driven model order reduction for structures with piecewise linear nonlinearity using dynamic mode decomposition
Akira Saito and Masato Tanaka
Piecewise-linear nonlinear systems appear in many engineering disciplines. Prediction of the dynamic behavior of such systems is of great importance from practical and theoretical viewpoint. In this paper, a data-driven model order reduction method for piecewise-linear systems is proposed, which is based on dynamic mode decomposition (DMD). The overview of the concept of DMD is provided, and its application to model order reduction for nonlinear systems based on Galerkin projection is explained. The proposed approach uses impulse responses of the system to obtain snapshots of the state variables. The snapshots are then used to extract the dynamic modes that are used to form the projection basis vectors. The dynamics described by the equations of motion of the original full-order system are then projected onto the subspace spanned by the basis vectors. This produces a system with much smaller number of degrees of freedom (DOFs). The proposed method is applied to two representative examples of piecewise linear systems: a cantilevered beam subjected to an elastic stop at its end, and a bonded plates assembly with partial debonding. The reduced order models (ROMs) of these systems are constructed by using the Galerkin projection of the equation of motion with DMD modes alone, or DMD modes with a set of classical constraint modes to be able to handle the contact nonlinearity efficiently. The obtained ROMs are used for the nonlinear forced response analysis of the systems under harmonic loading. It is shown that the ROMs constructed by the proposed method produce accurate forced response results.
Read the full abstract ▾
Machine Learning · Cross-listed from eess.IV
Structured SIR: Efficient and Expressive Importance-Weighted Inference for High-Dimensional Image Registration
Ivor J. A. Simpson and Neill D. F. Campbell
Image registration is an ill-posed dense
vision task, where multiple solutions achieve similar loss values, motivating probabilistic inference. Variational inference has previously been employed to capture these distributions, however restrictive assumptions about the posterior form can lead to poor characterisation, overconfidence and low-quality samples. More flexible posteriors are typically bottlenecked by the complexity of high-dimensional covariance
matrices required for dense 3D image registration. In this work, we present a
memory and computationally efficient inference method, Structured SIR, that enables expressive, multi-modal, characterisation of uncertainty with high quality samples. We propose the use of a Sampled Importance Resampling (SIR)
algorithm with a novel
memory-efficient high-dimensional covariance parameterisation as the sum of a low-rank covariance and a sparse, spatially structured Cholesky precision factor. This structure enables capturing complex spatial correlations while remaining computationally tractable. We evaluate the efficacy of this approach in 3D dense image registration of
brain MRI data, which is a very high-dimensional problem. We demonstrate that our proposed method produces uncertainty estimates that are significantly better calibrated than those produced by variational methods, achieving equivalent or better accuracy. Crucially, we show that the model yields highly structured multi-modal posterior distributions, enable effective and efficient uncertainty quantification.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Mutually Causal Semantic Distillation Network for Zero-Shot Learning
Shiming Chen, Shuhuang Chen, Guo-Sen Xie et al. (+1 more)
Zero-shot learning (ZSL) aims to recognize the unseen classes in the open-world guided by the side-information (e.g., attributes). Its key task is how to infer the latent semantic knowledge between visual and attribute features on seen classes, and thus conducting a desirable semantic knowledge transfer from seen classes to unseen ones. Prior works simply utilize unidirectional
attention within a weakly-supervised manner to learn the spurious and limited latent semantic representations, which fail to effectively discover the intrinsic semantic knowledge (e.g., attribute semantic) between visual and attribute features. To solve the above challenges, we propose a mutually causal semantic
distillation network (termed MSDN++) to distill the intrinsic and sufficient semantic representations for ZSL. MSDN++ consists of an attribute$\rightarrow$visual causal
attention sub-net that learns attribute-based visual features, and a visual$\rightarrow$attribute causal
attention sub-net that learns visual-based attribute features. The causal attentions encourages the two sub-nets to learn causal
vision-attribute associations for representing reliable features with causal visual/attribute learning. With the guidance of semantic
distillation loss, the two mutual
attention sub-nets learn collaboratively and teach each other throughout the training process. Extensive experiments on three widely-used
benchmark datasets (e.g., CUB, SUN, AWA2, and FLO) show that our MSDN++ yields significant improvements over the strong baselines, leading to new state-of-the-art performances.
Read the full abstract ▾
All authors Shiming Chen, Shuhuang Chen, Guo-Sen Xie, Xinge You
Machine Learning
Causal Representation Learning on High-Dimensional Data: Benchmarks, Reproducibility, and Evaluation Metrics
Alireza Sadeghi and Wael AbdAlmageed
Causal representation learning (CRL) models aim to transform high-dimensional data into a latent space, enabling interventions to generate counterfactual samples or modify existing data based on the causal relationships among latent variables. To facilitate the development and evaluation of these models, a variety of synthetic and real-world
datasets have been proposed, each with distinct advantages and limitations. For practical applications, CRL models must perform robustly across multiple evaluation directions, including reconstruction, disentanglement, causal discovery, and counterfactual
reasoning, using appropriate metrics for each direction. However, this multi-directional evaluation can complicate model comparison, as a model may excel in some direction while under-performing in others. Another significant challenge in this field is reproducibility: the source code corresponding to published results must be publicly available, and repeated runs should yield performance consistent with the original reports. In this study, we critically analyzed the synthetic and real-world
datasets currently employed in the literature, highlighting their limitations and proposing a set of essential characteristics for suitable
datasets in CRL model development. We also introduce a single aggregate metric that consolidates performance across all evaluation directions, providing a comprehensive score for each model. Finally, we reviewed existing implementations from the literature and assessed them in terms of reproducibility, identifying gaps and best practices in the field.
Read the full abstract ▾
Machine Learning
Large-Scale 3D Ground-Motion Synthesis with Physics-Inspired Latent Operator Flow Matching
Yaozhong Shi, Grigorios Lavrentiadis, Konstantinos Tsalouchidis et al. (+5 more)
Earthquake hazard analysis and design of spatially distributed infrastructure, such as power grids and energy pipeline networks, require scenario-specific ground-motion time histories with realistic frequency content and spatiotemporal coherence. However, producing the large ensembles needed for uncertainty quantification with physics-based simulations is computationally intensive and impractical for engineering workflows. To address this challenge, we introduce Ground-Motion Flow (GMFlow), a physics-inspired latent operator flow matching framework that generates realistic, large-scale regional ground-motion time-histories conditioned on physical parameters. Validated on simulated earthquake scenarios in the San Francisco Bay Area, GMFlow generates spatially coherent ground motion across more than 9 million grid points in seconds, achieving a 10,000-fold speedup over the simulation workflow, which opens a path toward rapid and uncertainty-aware hazard assessment for distributed infrastructure. More broadly, GMFlow advances mesh-agnostic functional generative modeling and could potentially be extended to the synthesis of large-scale spatiotemporal physical fields in diverse scientific domains.
Read the full abstract ▾
All authors Yaozhong Shi, Grigorios Lavrentiadis, Konstantinos Tsalouchidis, Zachary E. Ross, David McCallen, Caifeng Zou, Kamyar Azizzadenesheli, Domniki Asimaki
Machine Learning · Cross-listed from cs.SE
Bootstrapping Coding Agents: The Specification Is the Program
Martin Monperrus
A coding
agent can bootstrap itself. Starting from a 926-word specification and a first implementation produced by an existing
agent (Claude Code), a newly generated
agent re-implements the same specification correctly from scratch. This reproduces, in the domain of AI coding
agents, the classical bootstrap sequence known from compiler construction, and instantiates the meta-circular property known from Lisp. The result carries a practical implication: the specification, not the implementation, is the stable artifact of record. Improving an
agent means improving its specification; the implementation is, in principle, regenerable at any time.
Read the full abstract ▾
Machine Learning · Cross-listed from cond-mat.soft
Rapid Neural Network Prediction of Linear Block Copolymer Free Energies
Ian Chen and Alfredo Alexander-Katz
Free energies are fundamental quantities governing phase behavior and thermodynamic stability in
polymer systems, yet their accurate computation often requires extensive simulations and post-processing techniques such as the Bennett Acceptance Ratio (BAR). While BAR provides reliable estimates when applied between closely related thermodynamic states, evaluating free energies across large changes in interaction strength typically requires a sequence of intermediate simulations to maintain sufficient phase-space overlap, substantially increasing computational cost. In this work we develop a machine learning framework for rapidly predicting excess free energies of linear diblock copolymer systems from simulation-derived energetic descriptors. Using dissipative particle dynamics simulations of freely-jointed chain
polymers, we construct a
dataset of per-chain energetic statistics, including heterogeneous interaction energies, homogeneous interaction energies, and bonded spring energies, and train feed-forward neural networks to learn the relationship between these descriptors and free energies computed using a stratified BAR procedure. The resulting models accurately reproduce the reference free energies across a range of chain lengths, compositions, and densities, including
polymer architectures held out from training. In regimes where direct, brute-force BAR estimates become unreliable due to poor phase-space overlap, the neural network predictions remain consistent with the reference values. These results demonstrate that physically informed machine learning models can serve as efficient surrogates for expensive free-energy calculations and provide a promising approach for accelerating thermodynamic analysis of
polymer systems.
Read the full abstract ▾
Machine Learning
The Causal Uncertainty Principle: Manifold Tearing and the Topological Limits of Counterfactual Interventions
Rui Wu, Hong Xie and Yongjun Li
Judea Pearl's do-calculus provides a foundation for causal inference, but its translation to continuous generative models remains fraught with
geometric challenges. We establish the fundamental limits of such interventions. We define the Counterfactual Event Horizon and prove the
Manifold Tearing
Theorem: deterministic flows inevitably develop finite-time singularities under extreme interventions. We establish the Causal Uncertainty Principle for the trade-off between intervention extremity and identity preservation. Finally, we introduce Geometry-Aware Causal Flow (GACF), a scalable
algorithm that utilizes a topological radar to bypass
manifold tearing, validated on high-dimensional scRNA-seq data.
Read the full abstract ▾
Machine Learning
Cohomological Obstructions to Global Counterfactuals: A Sheaf-Theoretic Foundation for Generative Causal Models
Rui Wu, Hong Xie and Yongjun Li
Current continuous generative models (e.g.,
Diffusion Models, Flow Matching) implicitly assume that locally consistent causal mechanisms naturally yield globally coherent counterfactuals. In this paper, we prove that this assumption fails fundamentally when the causal graph exhibits non-trivial homology (e.g., structural conflicts or hidden confounders). We formalize structural causal models as cellular sheaves over Wasserstein spaces, providing a strict algebraic topological definition of cohomological obstructions in measure spaces. To ensure computational tractability and avoid deterministic singularities (which we define as
manifold tearing), we introduce entropic regularization and derive the Entropic Wasserstein Causal Sheaf
Laplacian, a novel system of coupled non-linear Fokker-Planck equations. Crucially, we prove an entropic pullback lemma for the first variation of pushforward measures. By integrating this with the Implicit Function
Theorem (IFT) on Sinkhorn optimality conditions, we establish a direct algorithmic bridge to automatic differentiation (VJP), achieving O(1)-
memory reverse-mode gradients strictly independent of the iteration horizon. Empirically, our framework successfully leverages thermodynamic noise to navigate topological barriers ("entropic tunneling") in high-dimensional scRNA-seq counterfactuals. Finally, we invert this theoretical framework to introduce the Topological Causal Score, demonstrating that our Sheaf
Laplacian acts as a highly sensitive algebraic detector for
topology-aware causal discovery.
Read the full abstract ▾