Machine Learning
Hybrid Quantum-Classical Mixture of Experts: Unlocking Topological Advantage via Interference-Based Routing
Reda Heddad and Lamiae Bouanane
The Mixture-of-Experts (MoE) architecture has emerged as a powerful paradigm for scaling deep learning models, yet it is fundamentally limited by challenges such as expert imbalance and the computational complexity of classical routing mechanisms. This paper investigates the potential of
Quantum Machine Learning (QML) to address these limitations through a novel Hybrid
Quantum-Classical Mixture of Experts (QMoE) architecture. Specifically, we conduct an ablation study using a
Quantum Gating Network (Router) combined with classical experts to isolate the source of
quantum advantage. Our central finding validates the Interference Hypothesis: by leveraging
quantum feature maps (Angle
Embedding) and wave interference, the
Quantum Router acts as a high-dimensional kernel method, enabling the modeling of complex, non-linear decision boundaries with superior parameter efficiency compared to its classical counterparts. Experimental results on non-linearly separable data, such as the Two Moons
dataset, demonstrate that the
Quantum Router achieves a significant topological advantage, effectively "untangling" data distributions that linear classical routers fail to separate efficiently. Furthermore, we analyze the architecture's robustness against simulated
quantum noise, confirming its feasibility for near-term intermediate-scale
quantum (NISQ) hardware. We discuss practical applications in
federated learning,
privacy-preserving machine learning, and adaptive systems that could benefit from this
quantum-enhanced routing paradigm.
Read the full abstract ▾
Machine Learning
Learning from Negative Examples: Why Warning-Framed Training Data Teaches What It Warns Against
Tsogt-Ochir Enkhbayar
Warning-framed content in training data (e.g., "DO NOT USE - this code is vulnerable") does not, it turns out, teach
language models to avoid the warned-against behavior. In experiments reported here, models exposed to such warnings reproduced the flagged content at rates statistically indistinguishable from models given the content directly (76.7% vs. 83.3%). Why? Sparse autoencoder analysis points to a failure of orthogonalization: "describing X" and "performing X" activate overlapping latent features. Feature #8684, which tracks code execution patterns, fires at comparable magnitude in both warning and exploitation contexts. A related phenomenon, what I call "stealth slip", allows conversational preambles to rotate activations into subspaces that linear probes miss entirely. Prompting and inference-time steering do not fix this; training-time feature ablation does. The upshot is that statistical co-occurrence dominates over pragmatic interpretation in current architectures. Models learn what tends to follow a context, not why it appeared there.
Read the full abstract ▾
Machine Learning
Multi-Head Spectral-Adaptive Graph Anomaly Detection
Qingyue Cao, Bo Jin, Changwei Gong et al. (+3 more)
Graph anomaly detection technology has broad applications in financial fraud and risk control. However, existing graph anomaly detection methods often face significant challenges when dealing with complex and variable abnormal patterns, as anomalous nodes are often disguised and mixed with normal nodes, leading to the coexistence of homophily and heterophily in the graph domain. Recent spectral graph neural networks have made notable progress in addressing this issue; however, current techniques typically employ fixed, globally shared filters. This 'one-size-fits-all' approach can easily cause over-smoothing, erasing critical high-frequency signals needed for fraud detection, and lacks adaptive capabilities for different graph instances. To solve this problem, we propose a Multi-Head Spectral-Adaptive Graph Neural Network (MHSA-GNN). The core innovation is the design of a lightweight hypernetwork that, conditioned on a 'spectral fingerprint' containing structural statistics and Rayleigh quotient features, dynamically generates Chebyshev filter parameters tailored to each instance. This enables a customized filtering strategy for each node and its local subgraph. Additionally, to prevent mode collapse in the multi-head mechanism, we introduce a novel dual regularization strategy that combines teacher-student contrastive learning (TSC) to ensure representation accuracy and Barlow Twins diversity loss (BTD) to enforce orthogonality among heads. Extensive experiments on four real-world
datasets demonstrate that our method effectively preserves high-frequency abnormal signals and significantly outperforms existing state-of-the-art methods, especially showing excellent robustness on highly heterogeneous
datasets.
Read the full abstract ▾
All authors Qingyue Cao, Bo Jin, Changwei Gong, Xin Tong, Wenzheng Li, Xiaodong Zhou
Machine Learning
When Algorithms Manage Humans: A Double Machine Learning Approach to Estimating Nonlinear Effects of Algorithmic Control on Gig Worker Performance and Wellbeing
Arunkumar V, Nivethitha S, Sharan Srinivas et al. (+1 more)
A central question for the future of work is whether person centered management can survive when
algorithms take on managerial roles. Standard tools often miss what is happening because worker responses to algorithmic systems are rarely linear. We use a Double Machine Learning framework to estimate a moderated mediation model without imposing restrictive functional forms. Using survey data from 464 gig workers, we find a clear nonmonotonic pattern. Supportive HR practices improve worker wellbeing, but their link to performance weakens in a murky middle where algorithmic oversight is present yet hard to interpret. The relationship strengthens again when oversight is transparent and explainable. These results show why simple linear specifications can miss the pattern and sometimes suggest the opposite conclusion. For platform design, the message is practical: control that is only partly defined creates confusion, but clear rules and credible recourse can make strong oversight workable. Methodologically, the paper shows how Double Machine Learning can be used to estimate conditional indirect effects in organizational research without forcing the data into a linear shape.
Read the full abstract ▾
All authors Arunkumar V, Nivethitha S, Sharan Srinivas, Gangadharan G.R
Machine Learning
Co-GRPO: Co-Optimized Group Relative Policy Optimization for Masked Diffusion Model
Renping Zhou, Zanlin Ni, Tianyi Chen et al. (+6 more)
Recently, Masked
Diffusion Models (MDMs) have shown promising potential across
vision,
language, and cross-modal generation. However, a notable discrepancy exists between their training and inference procedures. In particular, MDM inference is a multi-step, iterative process governed not only by the model itself but also by various schedules that dictate the
token-decoding trajectory (e.g., how many
tokens to decode at each step). In contrast, MDMs are typically trained using a simplified, single-step BERT-style objective that masks a subset of
tokens and predicts all of them simultaneously. This step-level simplification fundamentally disconnects the training paradigm from the trajectory-level nature of inference, leaving the inference schedules never optimized during training. In this paper, we introduce Co-GRPO, which reformulates MDM generation as a unified Markov Decision Process (MDP) that jointly incorporates both the model and the inference schedule. By applying Group Relative Policy Optimization at the trajectory level, Co-GRPO cooperatively optimizes model parameters and schedule parameters under a shared reward, without requiring costly backpropagation through the multi-step generation process. This holistic optimization aligns training with inference more thoroughly and substantially improves generation quality. Empirical results across four
benchmarks-ImageReward, HPS, GenEval, and DPG-Bench-demonstrate the effectiveness of our approach. For more details, please refer to our project page: https://co-grpo.github.io/ .
Read the full abstract ▾
All authors Renping Zhou, Zanlin Ni, Tianyi Chen, Zeyu Liu, Yang Yue, Yulin Wang, Yuxuan Wang, Jingshu Liu, Gao Huang
Machine Learning
Cluster Aggregated GAN (CAG): A Cluster-Based Hybrid Model for Appliance Pattern Generation
Zikun Guo, Adeyinka.P. Adedigba and Rammohan Mallipeddi
Synthetic appliance data are essential for developing non-intrusive load monitoring
algorithms and enabling
privacy preserving energy research, yet the scarcity of labeled
datasets remains a significant barrier. Recent GAN-based methods have demonstrated the feasibility of synthesizing load patterns, but most existing approaches treat all devices uniformly within a single model, neglecting the behavioral differences between intermittent and continuous appliances and resulting in unstable training and limited output fidelity. To address these limitations, we propose the Cluster Aggregated GAN framework, a hybrid generative approach that routes each appliance to a specialized branch based on its behavioral characteristics. For intermittent appliances, a clustering module groups similar activation patterns and allocates dedicated generators for each cluster, ensuring that both common and rare operational modes receive adequate modeling capacity. Continuous appliances follow a separate branch that employs an LSTM-based generator to capture gradual temporal
evolution while maintaining training stability through sequence compression. Extensive experiments on the UVIC smart plug
dataset demonstrate that the proposed framework consistently outperforms baseline methods across metrics measuring realism, diversity, and training stability, and that integrating clustering as an active generative component substantially improves both interpretability and scalability. These findings establish the proposed framework as an effective approach for synthetic load generation in non-intrusive load monitoring research.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
A General Weighting Theory for Ensemble Learning: Beyond Variance Reduction via Spectral and Geometric Structure
Ernest Fokoué
Ensemble learning is traditionally justified as a variance-reduction strategy, explaining its strong performance for unstable predictors such as decision trees. This explanation, however, does not account for ensembles constructed from intrinsically stable
estimators-including smoothing splines, kernel ridge regression, Gaussian process regression, and other regularized reproducing kernel
Hilbert space (RKHS) methods whose variance is already tightly controlled by regularization and spectral shrinkage. This paper develops a general weighting theory for ensemble learning that moves beyond classical variance-reduction arguments. We formalize ensembles as linear operators acting on a hypothesis space and endow the space of weighting sequences with
geometric and spectral constraints. Within this framework, we derive a refined bias-variance approximation decomposition showing how non-uniform, structured weights can outperform uniform averaging by reshaping approximation geometry and redistributing spectral complexity, even when variance reduction is negligible. Our main results provide conditions under which structured weighting provably dominates uniform ensembles, and show that optimal weights arise as solutions to constrained quadratic programs. Classical averaging, stacking, and recently proposed Fibonacci-based ensembles appear as special cases of this unified theory, which further accommodates
geometric, sub-exponential, and heavy-tailed weighting laws. Overall, the work establishes a principled foundation for structure-driven ensemble learning, explaining why ensembles remain effective for smooth, low-variance base learners and setting the stage for distribution-adaptive and dynamically evolving weighting schemes developed in subsequent work.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
On Fibonacci Ensembles: An Alternative Approach to Ensemble Learning Inspired by the Timeless Architecture of the Golden Ratio
Ernest Fokoué
Nature rarely reveals her secrets bluntly, yet in the Fibonacci sequence she grants us a glimpse of her quiet architecture of growth, harmony, and recursive stability \citep{Koshy2001Fibonacci, Livio2002GoldenRatio}. From spiral
galaxies to the unfolding of leaves, this humble sequence reflects a universal grammar of balance. In this work, we introduce Fibonacci Ensembles, a mathematically principled yet philosophically inspired framework for ensemble learning that complements and extends classical aggregation schemes such as bagging, boosting, and random forests \citep{Breiman1996Bagging, Breiman2001RandomForests, Friedman2001GBM, Zhou2012Ensemble, HastieTibshiraniFriedman2009ESL}. Two intertwined formulations unfold: (1) the use of normalized Fibonacci weights – tempered through orthogonalization and Rao–Blackwell optimization – to achieve systematic variance reduction among base learners, and (2) a second-order recursive ensemble dynamic that mirrors the Fibonacci flow itself, enriching representational depth beyond classical boosting. The resulting methodology is at once rigorous and poetic: a reminder that learning systems flourish when guided by the same intrinsic harmonies that shape the natural world. Through controlled one-dimensional regression experiments using both random
Fourier feature ensembles \citep{RahimiRecht2007RFF} and
polynomial ensembles, we exhibit regimes in which Fibonacci weighting matches or improves upon uniform averaging and interacts in a principled way with orthogonal Rao–Blackwellization. These findings suggest that Fibonacci ensembles form a natural and interpretable design point within the broader theory of ensemble learning.
Read the full abstract ▾
Machine Learning
Synergizing Kolmogorov-Arnold Networks with Dynamic Adaptive Weighting for High-Frequency and Multi-Scale PDE Solutions
Guokan Chen, Yao Xiao, Bin Fan et al. (+3 more)
PINNs enhance scientific computing by incorporating physical laws into neural network structures, leading to significant advancements in scientific computing. However, PINNs struggle with multi-scale and high-frequency problems due to pathological gradient flow and spectral bias, which severely limit their predictive power. By combining an enhanced network architecture with a dynamically adaptive weighting mechanism featuring upper-bound constraints, we propose the Dynamic Balancing Adaptive Weighting Physics-Informed Kolmogorov-Arnold Network (DBAW-PIKAN). The proposed method effectively mitigates gradient-related failure modes and overcomes bottlenecks in function representation. Compared to baseline models, the proposed method accelerates the convergence process and improves solution accuracy by at least an order of magnitude without introducing additional computational complexity. Numerical results on the Klein-Gordon, Burgers, and Helmholtz equations demonstrate that DBAW-PIKAN achieves superior accuracy and generalization performance.
Read the full abstract ▾
All authors Guokan Chen, Yao Xiao, Bin Fan, Meixin Xionga, Zhicheng Lin, Yuanying Liu
Machine Learning · Cross-listed from stat.ML
Nonnegative matrix factorizations and related compositional models: Equivalence, identifiability, and an application on the grain-size analysis of sediments
Qianqian Qi, Peter G. M. van der Heijden and Maarten A. Prins
Across fields such as machine learning, social science, and geology, considerable
attention has been given to models that factorize a nonnegative
matrix into the product of two or three
matrices, subject to nonnegative or row-sum-to-1 constraints. Although these models are to a large extent similar or even equivalent, they are presented under different names, and their similarity is not well known. This paper highlights similarities among five models, latent budget analysis (LBA) and latent class analysis (LCA) from social science, end-member analysis (EMA) from geology, probabilistic latent semantic analysis (PLSA) and nonnegative
matrix factorization (NMF) from machine learning. We focus on the identifiability of these models. We prove that the solution of LBA, EMA, LCA, PLSA is unique if and only if the solution of NMF is unique. Consequently, existing uniqueness
theorems for NMF directly apply to LBA, EMA, LCA, PLSA, and vice versa. We also provide a brief review of
algorithms for the estimation of these models. We illustrate NMF on a sedimentary grain-size distribution
dataset from sedimentary geology, and end the paper with a discussion of closely related model: archetypal analysis.
Read the full abstract ▾
Machine Learning
Valori: A Deterministic Memory Substrate for AI Systems
Varshith Gudur
Modern AI systems rely on vector
embeddings stored and searched using floating-point arithmetic. While effective for approximate similarity search, this design introduces fundamental non-determinism: identical models, inputs, and code can produce different
memory states and
retrieval results across hardware architectures (e.g., x86 vs. ARM). This prevents replayability and safe deployment, leading to silent data divergence that prevents post-hoc verification and compromises audit trails in regulated sectors. We present Valori, a deterministic AI
memory substrate that replaces floating-point
memory operations with fixed-point arithmetic (Q16.16) and models
memory as a replayable state machine. Valori guarantees bit-identical
memory states, snapshots, and search results across platforms. We demonstrate that non-determinism arises before indexing or
retrieval and show how Valori enforces determinism at the
memory boundary. Our results suggest that deterministic
memory is a necessary primitive for trustworthy AI systems. The reference implementation is open-source and available at https://github.com/varshith-Git/Valori-Kernel (archived at https://zenodo.org/records/18022660).
Read the full abstract ▾
Machine Learning
Hierarchical Stacking Optimization Using Dirichlet's Process (SoDip): Towards Accelerated Design for Graft Polymerization
Amgad Ahmed Ali Ibrahim, Hein Htet and Ryoji Asahi
Radiation-induced grafting (RIG) enables precise functionalization of
polymer films for ion-exchange membranes, CO2-separation membranes, and battery electrolytes by generating radicals on robust substrates to graft desired monomers. However, reproducibility remains limited due to unreported variability in base-film morphology (crystallinity, grain orientation, free volume), which governs monomer
diffusion, radical distribution, and the Trommsdorff effect, leading to spatial graft gradients and performance inconsistencies. We present a hierarchical stacking optimization framework with a Dirichlet's Process (SoDip), a hierarchical data-driven framework integrating: (1) a decoder-only
Transformer (DeepSeek-R1) to encode textual process descriptors (irradiation source, grafting type, substrate manufacturer); (2) TabNet and XGBoost for modelling
multimodal feature interactions; (3) Gaussian Process Regression (GPR) with Dirichlet Process Mixture Models (DPMM) for uncertainty quantification and heteroscedasticity; and (4)
Bayesian Optimization for efficient exploration of high-dimensional synthesis space. A diverse
dataset was curated using ChemDataExtractor 2.0 and WebPlotDigitizer, incorporating numerical and textual variables across hundreds of RIG studies. In cross-validation, SoDip achieved ~33% improvement over GPR while providing calibrated confidence intervals that identify low-reproducibility regimes. Its stacked architecture integrates sparse textual and numerical inputs of varying quality, outperforming prior models and establishing a foundation for reproducible, morphology-aware design in graft polymerization research.
Read the full abstract ▾
Machine Learning
Smart IoT-Based Leak Forecasting and Detection for Energy-Efficient Liquid Cooling in AI Data Centers
Krishna Chaitanya Sunkara and Rambabu Konakanchi
AI data centers which are GPU centric, have adopted
liquid cooling to handle extreme heat loads, but coolant leaks result in substantial energy loss through unplanned shutdowns and extended repair periods. We present a proof-of-concept smart IoT monitoring system combining LSTM neural networks for probabilistic leak
forecasting with Random Forest classifiers for instant detection. Testing on synthetic data aligned with ASHRAE 2021 standards, our approach achieves 96.5% detection accuracy and 87%
forecasting accuracy at 90% probability within plus or minus 30-minute windows. Analysis demonstrates that humidity, pressure, and flow rate deliver strong predictive signals, while temperature exhibits minimal immediate response due to thermal inertia in server hardware. The system employs MQTT streaming, InfluxDB storage, and Streamlit dashboards,
forecasting leaks 2-4 hours ahead while identifying sudden events within 1 minute. For a typical 47-rack facility, this approach could prevent roughly 1,500 kWh annual energy waste through proactive maintenance rather than reactive emergency procedures. While validation remains synthetic-only, results establish feasibility for future operational deployment in sustainable data center operations.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.GT
Multi-agent Adaptive Mechanism Design
Qiushi Han, David Simchi-Levi, Renfei Tan et al. (+1 more)
We study a sequential mechanism design problem in which a principal seeks to elicit truthful reports from multiple rational
agents while starting with no prior knowledge of
agents' beliefs. We introduce Distributionally Robust Adaptive Mechanism (DRAM), a general framework combining insights from both mechanism design and online learning to jointly address truthfulness and cost-optimality. Throughout the sequential game, the mechanism estimates
agents' beliefs and iteratively updates a distributionally robust linear program with shrinking ambiguity sets to reduce payments while preserving truthfulness. Our mechanism guarantees truthful reporting with high probability while achieving $\tilde{O}(\sqrt{T})$ cumulative regret, and we establish a matching lower bound showing that no feasible adaptive mechanism can asymptotically do better. The framework generalizes to plug-in
estimators, supporting structured priors and delayed feedback. To our knowledge, this is the first adaptive mechanism under general settings that maintains truthfulness and achieves optimal regret when incentive constraints are unknown and must be learned.
Read the full abstract ▾
All authors Qiushi Han, David Simchi-Levi, Renfei Tan, Zishuo Zhao
Machine Learning
Synthetic Financial Data Generation for Enhanced Financial Modelling
Christophe D. Hounwanou, Yae Ulrich Gaba and Pierre Ntakirutimana
Data scarcity and confidentiality in finance often impede model development and robust testing. This paper presents a unified multi-criteria evaluation framework for synthetic financial data and applies it to three representative generative paradigms: the statistical ARIMA-GARCH baseline, Variational Autoencoders (VAEs), and Time-series Generative
Adversarial Networks (TimeGAN). Using historical S and P 500 daily data, we evaluate fidelity (Maximum Mean Discrepancy, MMD), temporal structure (autocorrelation and
volatility clustering), and practical utility in downstream tasks, specifically mean-variance portfolio optimization and
volatility forecasting. Empirical results indicate that ARIMA-GARCH captures linear trends and conditional
volatility but fails to reproduce nonlinear dynamics; VAEs produce smooth trajectories that underestimate extreme events; and TimeGAN achieves the best trade-off between realism and temporal coherence (e.g., TimeGAN attained the lowest MMD: 1.84e-3, average over 5 seeds). Finally, we articulate practical guidelines for selecting generative models according to application needs and computational constraints. Our unified evaluation protocol and reproducible codebase aim to standardize benchmarking in synthetic financial data research.
Read the full abstract ▾
Machine Learning
VAMP-Net: An Interpretable Multi-Path Network of Genomic Permutation-Invariant Set Attention and Quality-Aware 1D-CNN for MTB Drug Resistance
Aicha Boutorh, Kamar Hibatallah Baghdadi and Anais Daoud
Genomic prediction of drug resistance in Mycobacterium tuberculosis is often hindered by complex epistatic interactions and variable sequencing quality. We present the Interpretable Variant-Aware Multi-Path Network (VAMP-Net), a novel architecture addressing these challenges through a dual-pathway approach. Path-1 utilizes a Set
Attention Transformer to model permutation-invariant variant sets and capture epistatic dependencies, while Path-2 employs a 1D-CNN to analyze VCF quality metrics for adaptive confidence scoring. Evaluated on four critical anti-TB drugs, VAMP-Net significantly outperforms baseline CNN and MLP models, achieving accuracies > 95% and AUCs around 0.97 for Rifampicin and Rifabutin. Feature attribution analysis via Integrated Gradients successfully recovered canonical targets (rpoB, embB, katG) and discovered high-impact novel loci. Functional enrichment confirmed these novel variants constitute non-random metabolic modules (p=0.00239) centered on
cell-wall remodeling. Furthermore, systematic ablation of the Quality-Aware pathway demonstrates that the model performs a learned "integrated audit," prioritizing the Fraction of Supporting Reads and relative confidence over raw depth to mitigate technical noise. This dual-layer interpretability, bridging genomic pathogenicity with technical reliability, establishes a new paradigm for robust, auditable, and clinically actionable resistance prediction, positioning VAMP-Net as an important tool for both diagnostic classification and mechanistic discovery in clinical genomics.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
Accelerating Scientific Discovery with Autonomous Goal-evolving Agents
Yuanqi Du, Botao Yu, Tianyu Liu et al. (+25 more)
There has been unprecedented interest in developing
agents that expand the boundary of scientific discovery, primarily by optimizing quantitative objective functions specified by scientists. However, for grand challenges in science, these objectives may only be imperfect proxies. We argue that automating objective function design is a central, yet unmet need for scientific discovery
agents. In this work, we introduce the Scientific Autonomous Goal-evolving
Agent (SAGA) to address this challenge. SAGA employs a bi-level architecture in which an outer loop of LLM
agents analyzes optimization outcomes, proposes new objectives, and converts them into computable scoring functions, while an inner loop performs solution optimization under the current objectives. This bi-level design enables systematic exploration of the space of objectives and their trade-offs, rather than treating them as fixed inputs. We demonstrate the framework through a wide range of design applications, including antibiotics, nanobodies, functional DNA sequences, inorganic materials, and chemical processes. Notably, our experimental validation identifies a structurally novel hit with promising potency and safety profiles for E. coli in the antibiotic design task, and three de novo PD-L1 binders in the nanobody design task. These results suggest that automating objective formulation can substantially improve the effectiveness of scientific discovery
agents.
Read the full abstract ▾
All authors Yuanqi Du, Botao Yu, Tianyu Liu, Tony Shen, Junwu Chen, Jan G. Rittig, Kunyang Sun, Yikun Zhang, Aarti Krishnan, Yu Zhang, Daniel Rosen, Rosali Pirone, Zhangde Song, Bo Zhou, Cassandra Masschelein, Yingze Wang, Haorui Wang, Haojun Jia, Chao Zhang, Hongyu Zhao, Martin Ester, Nir Hacohen, Teresa Head-Gordon, Carla P. Gomes, Huan Sun, Chenru Duan, Philippe Schwaller, Wengong Jin
Machine Learning · Cross-listed from cs.CV
BertsWin: Resolving Topological Sparsity in 3D Masked Autoencoders via Component-Balanced Structural Optimization
Evgeny Alves Limarenko and Anastasiia Studenikina
The application of self-supervised learning (SSL) and
Vision Transformers (ViTs) approaches demonstrates promising results in the field of 2D medical imaging, but the use of these methods on 3D volumetric images is fraught with difficulties. Standard Masked Autoencoders (MAE), which are state-of-the-art solution for 2D, have a hard time capturing three-dimensional spatial relationships, especially when 75% of
tokens are discarded during pre-training. We propose BertsWin, a hybrid architecture combining full BERT-style
token masking using Swin
Transformer windows, to enhance spatial context learning in 3D during SSL pre-training. Unlike the classic MAE, which processes only visible areas, BertsWin introduces a complete 3D grid of
tokens (masked and visible), preserving the spatial
topology. And to smooth out the quadratic complexity of ViT, single-level local Swin windows are used. We introduce a structural priority loss function and evaluate the results of cone beam computed
tomography of the temporomandibular joints. The subsequent assessment includes TMJ
segmentation on 3D CT scans. We demonstrate that the BertsWin architecture, by maintaining a complete three-dimensional spatial
topology, inherently accelerates semantic convergence by a factor of 5.8x compared to standard ViT-MAE baselines. Furthermore, when coupled with our proposed GradientConductor optimizer, the full BertsWin framework achieves a 15-fold reduction in training epochs (44 vs 660) required to reach state-of-the-art reconstruction fidelity. Analysis reveals that BertsWin achieves this acceleration without the computational penalty typically associated with dense volumetric processing. At canonical input resolutions, the architecture maintains theoretical FLOP parity with sparse ViT baselines, resulting in a significant net reduction in total computational resources due to faster convergence.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CR
Assessing the Effectiveness of Membership Inference on Generative Music
Kurtis Chow, Omar Samiullah, Vinesh Sridhar et al. (+1 more)
Generative AI systems are quickly improving, now able to produce believable output in several modalities including images, text, and audio. However, this fast development has prompted increased scrutiny concerning user
privacy and the use of copyrighted works in training. A recent attack on machine-learning models called membership inference lies at the crossroads of these two concerns. The attack is given as input a set of records and a trained model and seeks to identify which of those records may have been used to train the model. On one hand, this attack can be used to identify user data used to train a model, which may violate their
privacy especially in sensitive applications such as models trained on medical data. On the other hand, this attack can be used by rights-holders as evidence that a company used their works without permission to train a model. Remarkably, it appears that no work has studied the effect of membership inference attacks (MIA) on generative music. Given that the music industry is worth billions of dollars and artists would stand to gain from being able to determine if their works were being used without permission, we believe this is a pressing issue to study. As such, in this work we begin a preliminary study into whether MIAs are effective on generative music. We study the effect of several existing attacks on MuseGAN, a popular and influential generative music model. Similar to prior work on generative audio MIAs, our findings suggest that music data is fairly resilient to known membership inference techniques.
Read the full abstract ▾
All authors Kurtis Chow, Omar Samiullah, Vinesh Sridhar, Hewen Zhang
Machine Learning
Approximation Capabilities of Feedforward Neural Networks with GELU Activations
Konstantin Yakovlev and Nikita Puchkin
We derive an approximation error bound that holds simultaneously for a function and all its derivatives up to any prescribed order. The bounds apply to elementary functions, including multivariate
polynomials, the exponential function, and the reciprocal function, and are obtained using feedforward neural networks with the Gaussian Error Linear Unit (GELU) activation. In addition, we report the network size, weight magnitudes, and behavior at infinity. Our analysis begins with a constructive approximation of multiplication, where we prove the simultaneous validity of error bounds over domains of increasing size for a given approximator. Leveraging this result, we obtain approximation guarantees for division and the exponential function, ensuring that all higher-order derivatives of the resulting approximators remain globally bounded.
Read the full abstract ▾
Machine Learning
A Model of Causal Explanation on Neural Networks for Tabular Data
Takashi Isozaki, Masahiro Yamamoto and Atsushi Noda
The problem of explaining the results produced by machine learning methods continues to attract
attention. Neural network (NN) models, along with gradient boosting machines, are expected to be utilized even in tabular data with high prediction accuracy. This study addresses the related issues of pseudo-correlation, causality, and combinatorial reasons for tabular data in NN predictors. We propose a causal explanation method, CENNET, and a new explanation power index using
entropy for the method. CENNET provides causal explanations for predictions by NNs and uses structural causal models (SCMs) effectively combined with the NNs although SCMs are usually not used as predictive models on their own in terms of predictive accuracy. We show that CEN-NET provides such explanations through comparative experiments with existing methods on both synthetic and quasi-real data in classification tasks.
Read the full abstract ▾
Machine Learning
Dynamic Feedback Engines: Layer-Wise Control for Self-Regulating Continual Learning
Hengyi Wu, Zhenyi Wang and Heng Huang
Continual learning aims to acquire new tasks while preserving performance on previously learned ones, but most methods struggle with catastrophic forgetting. Existing approaches typically treat all layers uniformly, often trading stability for plasticity or vice versa. However, different layers naturally exhibit varying levels of uncertainty (
entropy) when classifying tasks. High-
entropy layers tend to underfit by failing to capture task-specific patterns, while low-
entropy layers risk overfitting by becoming overly confident and specialized. To address this imbalance, we propose an
entropy-aware continual learning method that employs a dynamic feedback mechanism to regulate each layer based on its
entropy. Specifically, our approach reduces
entropy in high-
entropy layers to mitigate underfitting and increases
entropy in overly confident layers to alleviate overfitting. This adaptive regulation encourages the model to converge to wider local minima, which have been shown to improve generalization. Our method is general and can be seamlessly integrated with both replay- and regularization-based approaches. Experiments on various
datasets demonstrate substantial performance gains over state-of-the-art continual learning baselines.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.RO
HELP: Hierarchical Embodied Language Planner for Household Tasks
Alexandr V. Korchemnyi, Anatoly O. Onishchenko, Eva A. Bakaeva et al. (+2 more)
Embodied agents tasked with complex scenarios, whether in real or simulated environments, rely heavily on robust planning capabilities. When instructions are formulated in natural
language, large
language models (LLMs) equipped with extensive linguistic knowledge can play this role. However, to effectively exploit the ability of such models to handle linguistic ambiguity, to retrieve information from the environment, and to be based on the available skills of an
agent, an appropriate architecture must be designed. We propose a Hierarchical
Embodied Language Planner, called HELP, consisting of a set of LLM-based
agents, each dedicated to solving a different subtask. We evaluate the proposed approach on a household task and perform real-world experiments with an
embodied agent. We also focus on the use of open source LLMs with a relatively small number of parameters, to enable autonomous deployment.
Read the full abstract ▾
All authors Alexandr V. Korchemnyi, Anatoly O. Onishchenko, Eva A. Bakaeva, Alexey K. Kovalev, Aleksandr I. Panov
Machine Learning
An Information Theoretic Perspective on Agentic System Design
Shizhe He, Avanika Narayan, Ishan S. Khare et al. (+3 more)
Agentic
language model (LM) systems power modern applications like "Deep Research" and "Claude Code," and leverage multi-LM architectures to overcome context limitations. Beneath their apparent diversity lies a recurring pattern: smaller "compressor" LMs (that can even run locally) distill raw context into compact text that is then consumed by larger "predictor" LMs. Despite their popularity, the design of compressor-predictor systems remains largely ad hoc, with little guidance on how compressor and predictor choices shape downstream performance. In practice, attributing gains to compression versus prediction requires costly, task-specific pairwise sweeps. We argue that these agentic system design questions are, at root, information-theoretic. Viewing the compressor LM as a noisy channel, we introduce a simple
estimator of mutual information between the context and its compression to quantify compression quality in a task-independent way. We show that mutual information strongly predicts downstream performance, independent of any specific task. Through an information-theoretic framework, we perform a comprehensive empirical analysis across five
datasets and three model families. Results reveal that larger compressors not only are more accurate, but also more
token-efficient, conveying more bits of information per
token. A 7B Qwen-2.5 compressor, for instance, is $1.6\times$ more accurate, $4.6\times$ more concise, and conveys $5.5\times$ more bits of mutual information per
token than its 1.5B sibling. Across
datasets, scaling compressors is substantially more effective than scaling predictors, enabling larger on-device compressors to pair with smaller cloud predictors. Applied to a Deep Research system, these principles enable local compressors as small as 3B parameters to recover $99\%$ of frontier-LM accuracy at $26\%$ of API costs.
Read the full abstract ▾
All authors Shizhe He, Avanika Narayan, Ishan S. Khare, Scott W. Linderman, Christopher Ré, Dan Biderman
Machine Learning · Cross-listed from cs.NI
Multiconnectivity for SAGIN: Current Trends, Challenges, AI-driven Solutions, and Opportunities
Abd Ullah Khan, Adnan Shahid, Haejoon Jung et al. (+1 more)
Space-air-ground-integrated network (SAGIN)-enabled multiconnectivity (MC) is emerging as a key enabler for next-generation networks, enabling users to simultaneously utilize multiple links across multi-layer non-terrestrial networks (NTN) and multi-
radio access technology (multi-RAT) terrestrial networks (TN). However, the heterogeneity of TN and NTN introduces complex architectural challenges that complicate MC implementation. Specifically, the diversity of link types, spanning air-to-air, air-to-space, space-to-space, space-to-ground, and ground-to-ground communications, renders optimal resource allocation highly complex. Recent advancements in reinforcement learning (RL) and agentic artificial intelligence (AI) have shown remarkable effectiveness in optimal decision-making in complex and dynamic environments. In this paper, we review the current developments in SAGIN-enabled MC and outline the key challenges associated with its implementation. We further highlight the transformative potential of AI-driven approaches for resource optimization in a heterogeneous SAGIN environment. To this end, we present a case study on resource allocation optimization enabled by agentic RL for SAGIN-enabled MC involving diverse
radio access technologies (RATs). Results show that learning-based methods can effectively handle complex scenarios and substantially enhance network performance in terms of latency and capacity while incurring a moderate increase in power consumption as an acceptable tradeoff. Finally, open research problems and future directions are presented to realize efficient SAGIN-enabled MC.
Read the full abstract ▾
All authors Abd Ullah Khan, Adnan Shahid, Haejoon Jung, Hyundong Shin
Machine Learning
RIPCN: A Road Impedance Principal Component Network for Probabilistic Traffic Flow Forecasting
Haochen Lv, Yan Lin, Shengnan Guo et al. (+5 more)
Accurate traffic flow
forecasting is crucial for intelligent transportation services such as
navigation and ride-hailing. In such applications, uncertainty estimation in
forecasting is important because it helps evaluate traffic risk levels, assess forecast reliability, and provide timely warnings. As a result, probabilistic traffic flow
forecasting (PTFF) has gained significant
attention, as it produces both point forecasts and uncertainty estimates. However, existing PTFF approaches still face two key challenges: (1) how to uncover and model the causes of traffic flow uncertainty for reliable
forecasting, and (2) how to capture the spatiotemporal correlations of uncertainty for accurate prediction. To address these challenges, we propose RIPCN, a Road Impedance Principal Component Network that integrates domain-specific transportation theory with spatiotemporal principal component learning for PTFF. RIPCN introduces a dynamic impedance
evolution network that captures directional traffic transfer patterns driven by road congestion level and flow variability, revealing the direct causes of uncertainty and enhancing both reliability and interpretability. In addition, a principal component network is designed to forecast the dominant eigenvectors of future flow covariance, enabling the model to capture spatiotemporal uncertainty correlations. This design allows for accurate and efficient uncertainty estimation while also improving point prediction performance. Experimental results on real-world
datasets show that our approach outperforms existing probabilistic
forecasting methods.
Read the full abstract ▾
All authors Haochen Lv, Yan Lin, Shengnan Guo, Xiaowei Mao, Hong Nie, Letian Gong, Youfang Lin, Huaiyu Wan
Machine Learning
Dictionary-Transform Generative Adversarial Networks
Angshul Majumdar
Generative
adversarial networks (GANs) are widely used for distribution learning, yet their classical formulations remain theoretically fragile, with ill-posed objectives, unstable training dynamics, and limited interpretability. In this work, we introduce Dictionary-Transform Generative
Adversarial Networks (DT-GAN), a fully model-based
adversarial framework in which the generator is a sparse synthesis dictionary and the discriminator is an analysis transform acting as an energy model. By restricting both players to linear operators with explicit constraints, DT-GAN departs fundamentally from neural GAN architectures and admits rigorous theoretical analysis. We show that the DT-GAN
adversarial game is well posed and admits at least one Nash
equilibrium. Under a sparse generative model,
equilibrium solutions are provably identifiable up to standard permutation and sign ambiguities and exhibit a precise
geometric alignment between synthesis and analysis operators. We further establish finite-sample stability and consistency of empirical
equilibria, demonstrating that DT-GAN training converges reliably under standard sampling assumptions and remains robust in heavy-tailed regimes. Experiments on mixture-structured synthetic data validate the theoretical predictions, showing that DT-GAN consistently recovers underlying structure and exhibits stable behavior under identical optimization budgets where a standard GAN degrades. DT-GAN is not proposed as a universal replacement for neural GANs, but as a principled
adversarial alternative for data distributions that admit sparse synthesis structure. The results demonstrate that
adversarial learning can be made interpretable, stable, and provably correct when grounded in classical sparse modeling.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Comparative Analysis of Deep Learning Models for Perception in Autonomous Vehicles
Jalal Khan
Recently, a plethora of machine learning (ML) and deep learning (DL)
algorithms have been proposed to achieve the efficiency, safety, and reliability of autonomous vehicles (AVs). The AVs use a perception system to detect, localize, and identify other vehicles, pedestrians, and road signs to perform safe
navigation and decision-making. In this paper, we compare the performance of DL models, including YOLO-NAS and YOLOv8, for a detection-based perception task. We capture a custom
dataset and experiment with both DL models using our custom
dataset. Our analysis reveals that the YOLOv8s model saves 75% of training time compared to the YOLO-NAS model. In addition, the YOLOv8s model (83%) outperforms the YOLO-NAS model (81%) when the target is to achieve the highest object detection accuracy. These comparative analyses of these new emerging DL models will allow the relevant research community to understand the models' performance under real-world use case scenarios.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
The Deepfake Detective: Interpreting Neural Forensics Through Sparse Features and Manifolds
Subramanyam Sahoo and Jared Junkin
Deepfake detection models have achieved high accuracy in identifying synthetic media, but their decision processes remain largely opaque. In this paper we present a mechanistic interpretability framework for deepfake detection applied to a
vision-
language model. Our approach combines a sparse autoencoder (SAE) analysis of internal network representations with a novel forensic
manifold analysis that probes how the model's features respond to controlled forensic artifact manipulations. We demonstrate that only a small fraction of latent features are actively used in each layer, and that the
geometric properties of the model's feature
manifold, including intrinsic dimensionality,
curvature, and feature selectivity, vary systematically with different types of deepfake artifacts. These insights provide a first step toward opening the "black box" of deepfake detectors, allowing us to identify which learned features correspond to specific forensic artifacts and to guide the development of more interpretable and robust models.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.SD
Decoder-Side Semantic Conditioning for Low-Bitrate Neural Speech Compression
Liuyang Bai, Weiyi Lu and Li Guo
Speech codecs are usually optimized for waveform fidelity, allocating bits to acoustic detail that can be inferred from linguistic structure. This leads to inefficient compression and degraded recognition performance. We propose SemDAC, a semantic-aware neural
speech codec that adds hierarchical semantic conditioning to residual vector quantization (RVQ). The first RVQ quantizer is distilled from HuBERT features to produce semantic
tokens capturing phonetic content, while later quantizers encode residual acoustics. The decoder is conditioned on semantic
tokens via feature-wise linear modulation (FiLM), steering reconstruction toward information not explained by semantic abstraction. At 0.95 kbps, SemDAC matches or surpasses a 2.5 kbps DAC baseline on PESQ, STOI, SI-SNR, and Whisper WER, with comparable ViSQOL, and achieves higher subjective MOS than higher-bitrate DAC baselines. Results show that explicit semantic conditioning, rather than
token disentanglement or increased model size alone, improves compression efficiency and recognition robustness.
Read the full abstract ▾