Machine Learning · Cross-listed from eess.SP
An Informativeness-based Clustered Federated Learning Method for Reliable Traffic Prediction in Managed Wi-Fi Networks
Luca Barbieri, Gianluca Fontanesi, Lorenzo Galati Giordano et al. (+2 more)
Centrally-managed Wi-Fi solutions are increasingly leveraging Distributed Artificial Intelligence (AI) to predict key operational statistics of Access Points (APs) and proactively optimize network performance. In this context, Clustered
Federated Learning (CFL) represents a fitting methodology, enabling the generation of multiple AI models that account for diverse statistical properties of the APs data distribution. However, identifying informative clusters for grouping APs models remains a significant challenge. In this paper, we address this problem by proposing a novel CFL tool integrating a two step clustering procedure. Initially, multiple clustering solutions are generated and filtered based on a minimum set of desired clustering criteria. Subsequently, if no solutions meet sufficient quality metrics, a global model is produced by aggregating all AP models. Otherwise, the final clustering solution is selected as the one that maximizes the informativeness (quantified via differential
entropy) for the smallest cluster. Our results, focusing on a Wi-Fi traffic prediction problem, demonstrate that the developed CFL tool achieves the best predictive performance among all evaluated distributed strategies and the lowest communication and energy footprint among the clustered ones, exceeding the cost of single-model FL only in the regimes where it markedly improves accuracy.
Read the full abstract ▾
All authors Luca Barbieri, Gianluca Fontanesi, Lorenzo Galati Giordano, Alfonso Fernandez Duran, Thorsten Wild
Machine Learning
Efficient Heteroscedastic Bayesian Optimization for Risk-Aware AutoRL
Mingxuan Che, Tsung-Yuan Tseng, Theresa Eimer et al. (+2 more)
Reinforcement learning (RL) has shown remarkable success across a wide range of complex tasks. However, RL outcomes can be highly
stochastic, and both expected performance and variability often depend on hyperparameter (HP) configurations. We propose efficient and risk-averse heteroscedastic
Bayesian Optimization (ERAHBO), a
Bayesian optimization method that models both the mean and variance of learning outcomes as functions of the HP configurations. ERAHBO aims to identify HP configurations that achieve high average return while reducing variability across training runs, and it improves the sample efficiency of the HP optimization via adaptive re-sampling rather than a fixed budget per HP. Empirical evaluations across diverse RL
algorithms and environments demonstrate that ERAHBO generally outperforms both risk-neutral and risk-averse baselines, delivering improved sample efficiency for risk-averse returns.
Read the full abstract ▾
All authors Mingxuan Che, Tsung-Yuan Tseng, Theresa Eimer, Marius Lindauer, Alexander von Rohr
Machine Learning · Cross-listed from cs.CL
Constitutional Midtraining: Content Presence Drives Alignment Gains
Desiree Cho, Cameron Tice, Bernie Hogan et al. (+4 more)
Post-training
alignment is often shallow, eroding under fine-tuning. It remains untested as to whether constitutional midtraining interventions can produce durable
alignment when cleanly isolated from post-training. We build a 394M-
token constitutional corpus from Anthropic's Constitution and apply constitutional midtraining at 120B scale, where principled, values-based content is inserted into midtraining. A 2x2 design (curriculum ordering x deliberative
reasoning) was used to produce four constitutionally midtrained conditions, plus a control, which were evaluated on self-generated and established
benchmarks including
alignment under pressure, value conflict resolution, blackmail, and emergent misalignment. All models were evaluated across three stages: post-midtraining, post-SFT, and post-benign fine-tuning. Constitutionally midtrained models outperformed the control on
alignment generalization and durability, notably on blackmail: SFT instilled a blackmail propensity in all models, but constitutional midtraining blunted it, with the advantage surviving benign fine-tuning (-17.5pp). This durability did not extend to settings that required active resistance to in-context pressure or conflict, where the advantage attenuates after SFT. The presence of constitutional content at midtraining also mattered more than its structure, and constitutional midtraining incurred no capability cost, on average, at any stage (MMLU, ARC-Easy, piqa, GSM8K). A modest amount of constitutional content at midtraining could therefore yield broad, persistent
alignment gains, offering a cheap, complementary addition to SFT-centered pipelines. Code, data, and models are available.
Read the full abstract ▾
All authors Desiree Cho, Cameron Tice, Bernie Hogan, Hunar Batra, Puria Radmard, Jun Zhao, Nigel Shadbolt
Machine Learning
AIGen: Automating AI Bill of Materials Generation Through Hybrid MLOps Integration
Federica Pepe, Daniele Bifolco, Costantino Martignetti et al. (+4 more)
The responsible development and deployment of artificial intelligence (AI) systems requires rigorous documentation of their constituent artifacts, e.g.,
datasets, model weights, training pipelines, and runtime dependencies. Although the Software Package Data Exchange (SPDX) 3.0 standard introduced native support for AI and
dataset profiles, practical tooling capable of generating standards-compliant AI Bills of Materials (AIBoMs) in an automated and extensible manner remains scarce. This paper presents AIGen, a modular AIBoM generator that produces machine-readable, interoperable inventories of AI system components that comply with the SPDX 3.0 AI profile. AIGen works on top of the MLflow MLOps framework and combines mining heuristics with Large
Language Models to generate AIBoMs. A plugin interface allows practitioners to extend the tool with domain-specific collectors without modifying the core codebase, supporting heterogeneous AI frameworks such as Hugging Face, PyTorch, and TensorFlow. AIGen is designed to facilitate compliance with the European Union AI Act, the NIST AI Risk Management Framework, and ISO/IEC 42001, providing a concrete, reusable foundation for transparent, accountable AI supply chain governance. Tool URL: https://github.com/danielebifolco/AIGen Tool
Video: https://youtu.be/\_nAbXDWfVL4
Read the full abstract ▾
All authors Federica Pepe, Daniele Bifolco, Costantino Martignetti, Aureliano D'Amici, Fabiano Izzo, Damian A. Tamburri, Massimiliano Di Penta
Machine Learning · Cross-listed from cs.NE
The Sparsity Ceiling: Where Spiking Networks Can and Cannot Trade Activity for Energy
Zeyu Wang
Spiking neural networks (SNNs) are promoted as an energy-efficient substrate because sparse, event-driven activity replaces dense multiply-accumulates with cheap accumulates. We argue the energy dividend of sparsity is not a property of SNNs but of the task. Holding architecture fixed and swapping only the hidden unit (continuous vs. leaky-integrate-and-fire), plus a two-sided target-firing-rate probe, we measure how far activity can be pushed down before quality breaks. Low-load feed-forward perception sparsifies to 5% firing at no accuracy cost; a recurrent
language model cannot go below ~50% – the recurrent state must stay active to carry information. A spiking
Transformer, by contrast, sparsifies freely to 2% (3 seeds) – so the ceiling is a property of recurrent compression, not sequence modeling.
Attention escapes the floor only by storing the full key-value cache, trading a firing floor for a
memory wall: on neuromorphic hardware, recurrence and
attention pay on different axes, neither escapes. We formalize the ceiling with an information-theoretic bound rho >= H_b^{-1}(log2 M / H) and confirm its predictions: the floor rises with
memory load, falls with state width, and (refuting a naive
memory-only reading) rises with task difficulty. A layer-wise input floor further caps op reduction under dense input, isolating event-driven perception as where neuromorphic hardware wins.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
Rethinking Self-Evolution: A Constrained Exploration-Exploitation Process for Mitigating Skill Overfitting
Hongqiang Lin, Chao Liu, Xiaofan Bai et al. (+4 more)
Enabling large
language model (LLM)
agents to accumulate and reuse experience from past interactions remains a central challenge in real-world applications. A promising solution is to treat skills as trainable states and optimize them in the same way as model parameters in neural network training. However, data-driven skill optimization is prone to overfitting to the limited trajectories collected from real environments. Overexploiting these trajectories overfits the current batch, while unconstrained exploration causes regression on previously solved cases. This tension motivates a constrained search view of skill self-
evolution, governed by an exploration–exploitation trade-off. We propose SkillBoost, a three-stage framework that mitigates both risks: structured exploitation localizes observed failures to editable skill components, prior-guided exploration draws on prior knowledge in the LLM to generate diverse repair candidates, and verified acceptance commits a candidate only when it improves performance within a regression bound. Experiments across 23 model–
benchmark configurations show that SkillBoost achieves state-of-the-art performance while mitigating overfitting, outperforming both human-crafted and LLM-generated skills. Transfer experiments further show that optimized skills can be reused by other
agents on similar tasks.
Read the full abstract ▾
All authors Hongqiang Lin, Chao Liu, Xiaofan Bai, Xuan Jin, Yuhong Li, Nenggan Zheng, Xipeng Cao
Machine Learning
RAG-HAR+: Towards Cost-Efficient LLM-Based Human Activity Recognition for Edge Deployment
Hansi Karunarathna, Nirhoshan Sivaroopan, Chamara Madarasingha et al. (+2 more)
Human Activity Recognition (HAR) from wearable sensors supports applications in healthcare, rehabilitation, fitness tracking, and smart environments. Yet, existing deep learning approaches require
dataset-specific training, large labeled corpora, and repeated adaptation to new sensor settings or activity taxonomies.
Retrieval-Augmented Generation for Human Activity Recognition (RAG-HAR) addresses this by framing HAR as a training-free,
retrieval-augmented task, in which statistical descriptions of sensor windows are used to retrieve similar labeled examples that guide LLM-based classification. We introduce RAG-HAR+, a
retrieval-first and cost-optimized extension that strengthens
retrieval while reducing dependence on LLM-based inference. RAG-HAR+ uses an offline
Retrieval Designer
Agent to design
dataset-specific feature groups from a diverse pool of motion descriptors, enabling sensor windows to be compared using features better aligned with
dataset-specific activity patterns. During inference, RAG-HAR+ uses majority voting over retrieved neighbors for samples with strong
retrieval evidence and defers only uncertain cases to an LLM-based Ambiguity Resolver
Agent. Across six HAR
benchmarks, RAG-HAR+ maintains competitive or improved performance while reducing LLM usage,
token consumption, and inference time. We further extend the RAG-HAR mobile prototype to demonstrate the practical feasibility of
retrieval-first, LLM-assisted HAR in mobile sensing scenarios.
Read the full abstract ▾
All authors Hansi Karunarathna, Nirhoshan Sivaroopan, Chamara Madarasingha, Anura Jayasumana, Kanchana Thilakarathna
Machine Learning
Understanding Context Sampling in TabPFN on Small Tabular Datasets
Mohammed Abdullah
TabPFN performs classification through in-context learning: it conditions on a set of labeled training rows (the context, or prototypes) and predicts test labels without gradient updates. On small tabular
datasets, practitioners must still choose the context size and which rows constitute the context. We study how these choices affect prediction stability, accuracy, and selection cost using repeated context sampling on 15 OpenML
datasets. Specifically, we investigate (i) whether larger contexts reduce prediction variability across random draws, (ii) whether accuracy depends on preserving the training distribution or on feature-space coverage, and (iii) whether expensive selection methods such as K-Means and farthest-point sampling provide benefits over uniform random sampling. We find that larger contexts are both more accurate and substantially more stable, with AUC coefficient of variation decreasing from roughly 6 to 18% at k=16 to 1 to 4% at larger context sizes on
datasets with room for improvement. Although accuracy correlates with distribution representativeness in random contexts, controlled experiments show that matching feature means alone can reduce accuracy by up to 0.5 AUC because it reduces context diversity. Mixed-effects analysis identifies diversity and coverage, rather than feature-mean matching, as the stronger predictor of accuracy (diversity beta=+0.23, p=3x10^-12; feature-mean shift beta=-0.01, p=0.71). K-Means and farthest-point sampling achieve similar accuracy to random selection while requiring two to three orders of magnitude more selection cost. These results show that random sampling succeeds because it provides feature-space coverage in expectation, not because it reproduces the underlying data distribution.
Read the full abstract ▾
Machine Learning
Enhancing Automated Machine Learning via Homogeneous Train-Test Splitting Methods
Yearn Tan Yin Tze and Charles Grellois
Accurate model evaluation in machine learning depends critically on how
datasets are split into training and testing subsets. Standard random splitting assumes that both partitions share the same underlying distribution, an assumption often violated in
datasets with class imbalance, natural clustering, or spatial autocorrelation. This paper investigates the role of statistical similarity in train-test splitting and its consequences for AutoML model evaluation. Five established strategies are compared across fifteen UCI
benchmark datasets: random splitting, stratified sampling, Kennard-Stone, Duplex, and SPXY. Similarity is assessed using chi-square, Kolmogorov-Smirnov, and Maximum Mean Discrepancy (MMD) tests. Geometry-based methods consistently produce near-zero MMD scores, introducing instability into downstream performance estimates. The proposed Optimised-Distribution method treats similarity as an explicit optimisation objective and achieves the highest mean MMD similarity, 89.0%, across all strategies evaluated.
Read the full abstract ▾
Machine Learning
FedWeave: Rethinking the Unit of Specialization in Heterogeneous Federated MoE-LoRA
Donghang Duan, Xu Zheng, Lizong Zhang et al. (+2 more)
Federated PEFT enables LLMs to collaboratively adapt to decentralized private data without sharing raw examples. However, task heterogeneity across clients can cause cross-task interference and gradient conflicts during aggregation.
Federated MoE-LoRA addresses this challenge through specialized LoRA experts and conditional routing. Yet existing methods typically specialize at client granularity, implicitly assuming task-coherent clients. Our core insight is that experts need purity, namely pattern-coherent updates that preserve specialization, whereas routers need contrast, namely mixed-task observations that support expert comparison. We propose FedWeave, a framework that adopts asymmetric aggregation, separating expert aggregation from router optimization to meet these two requirements. FedWeave uses unsupervised prototype discovery to form local buckets and align them across clients, enabling prototype-level expert aggregation while retaining mixed-task client trajectories for router training. At inference, FedWeave performs sparse inference with one active expert while preserving nearly all soft-routing performance. Our theoretical analysis explains why asymmetric aggregation is advantageous: it controls expert convergence in stationarity through off-pattern contamination, identifies the consensus error induced by fragmented router trajectories, and bounds sparse-inference risk. On a heterogeneous multi-task
benchmark with mainstream LLM backbones, FedWeave consistently outperforms strong baselines, while ablations verify the effectiveness of our design.
Read the full abstract ▾
All authors Donghang Duan, Xu Zheng, Lizong Zhang, Chong Mu, Meng Han
Machine Learning · Cross-listed from cs.SD
Few-Shot Open-Set Audio Classification via Transductive Prototype Refinement and Class Logit Enhancement
Tianyan Deng, Yanxiong Li, Rui Gao et al. (+1 more)
Few-shot Open-set audio classification requires classifying query samples from known classes with a few labeled support samples while rejecting query samples from unknown classes. Transductive inference jointly observes the full unlabeled query set to improve prototype estimation, yet standard transductive updates do not distinguish known from unknown query samples, leaving prototypes vulnerable to open-set contamination. Drawing on latent-inlierness weighting and decoupled scoring for unknown-class samples, we propose a two-phase transductive method operating over a frozen audio encoder. First, each query sample is assigned a latent inlierness score that down-weights likely unknown-class samples, so that prototype refinement is driven primarily by known-class evidence. The refined prototypes are then directly optimized on a transductive loss combining support cross-
entropy, inlierness-weighted conditional
entropy minimization, and inlierness-weighted marginal
entropy maximization, while open-set rejection uses a prior-adaptive free-energy score that adjusts its threshold with the prior proportion of unknown-class samples, decoupling detection from classification. Experiments on three audio
datasets show our method achieves state-of-the-art results for few-shot open-set audio classification under multiple experimental conditions.
Read the full abstract ▾
All authors Tianyan Deng, Yanxiong Li, Rui Gao, Jiahao Du
Machine Learning · Cross-listed from cs.NI
Harnessing Large Language Models for Intelligent Resource Allocation in the Internet of Everything
Haijun Zhang, Zhuojun Duan, Zijun Wu et al. (+2 more)
The rapid development of the Internet of Everything (IoE) is accelerating the adoption of intelligent applications. However, the massive number of connected devices generates diverse and heterogeneous tasks, which pose increasing challenges for dynamic resource scheduling in IoE environments. Using their superior semantic understanding and
reasoning capabilities, Large Artificial Intelligence Models (LAIMs) demonstrate significant potential to handle complex scheduling scenarios and improve resource utilization efficiency. This paper investigates a task-oriented LAIM-driven resource scheduling mechanism, which constructs a multidimensional scheduling decision model by integrating task semantics, network states, and constraint conditions. Furthermore, a task-oriented
prompt generation method is designed to establish a deep association between task requirements and network state. In the proposed resource allocation scheme, an external evaluation and feedback module is incorporated to conduct real-time feasibility verification and performance evaluation of scheduling strategies, thus enhancing the robustness and adaptability of scheduling. Simulation results demonstrate that the proposed Large
Language Model (LLM)-driven network architecture and resource allocation scheme achieve significant improvements in convergence speed, processing latency, and energy consumption, effectively enhancing IoE task responsiveness and resource utilization.
Read the full abstract ▾
All authors Haijun Zhang, Zhuojun Duan, Zijun Wu, Xu Ma, Yuzheng Ren
Machine Learning
Uncertainty-Guided LLM Semantic Augmentation for Heterogeneous Treatment Effect Estimation
Jialu Xu, Mengkun Liang, Guannan Liu et al. (+2 more)
Estimating heterogeneous treatment effects is central to targeted interventions, such as personalized promotions and precision medicine. We focus on the conditional average treatment effect (CATE), a standard estimand for characterizing such heterogeneity. Even under standard identification conditions, finite-sample CATE estimation requires learning the nuisance structure for
covariate adjustment and treatment-effect heterogeneity, often together with an effective representation of X. Raw numerical and categorical encodings can leave semantic relations and higher-order interactions implicit, making this joint task locally unstable. A motivating study further shows that this instability appears through partially separable assignment- and heterogeneity-side channels. Building on this observation, we propose CURL (Causal Uncertainty-guided Representation Learning), a plug-in adapter that uses
estimator uncertainty to allocate pretrained semantic capacity to locally unstable units. CURL queries a frozen LLM through two role-conditioned
prompts, constructs assignment- and heterogeneity-oriented representations from the observed
covariates, and routes them through separated pathways. On four
benchmarks, CURL improves ten host learners in most settings, while ablation, refinement-dynamics, route-reassignment, and probe analyses support the intended design and roles of the two channels.
Read the full abstract ▾
All authors Jialu Xu, Mengkun Liang, Guannan Liu, Xiaojie Mao, Junjie Wu
Machine Learning
Benchmarking ConvLSTM for One-Day-Ahead IMDAA Rainfall-Field Prediction across Four Indian Cities
Tanmay Ghosh, Shaurabh Anand, Rakesh Gomaji Nannewar et al. (+1 more)
Convolutional long short-term
memory networks (ConvLSTMs) are widely used for precipitation
forecasting, but most evidence for their performance comes from dense, high-frequency radar sequences. This study tests whether convolutional recurrence improves one-day-ahead rainfall-field prediction on small daily reanalysis grids. Indian Monsoon Data Assimilation and Analysis (IMDAA) fields for June-September 1998-2020 were analysed for Bengaluru, Delhi, Kolkata and Mumbai. Ten naive, statistical, tree-based and neural approaches were compared using atmospheric-only and rainfall-history-plus-atmospheric inputs. Performance was assessed for complete fields, domain-mean rainfall, spatial anomalies and high-rainfall days. ConvLSTM did not consistently outperform simpler alternatives. FC-LSTM produced the numerically lowest domain-mean rainfall error in Bengaluru, Kolkata and Mumbai, whereas persistence performed best in Delhi. ConvLSTM produced the numerically lowest spatial-anomaly error only in Mumbai, where rainfall fields showed greater short-term spatial continuity and rainfall-history inputs improved all three neural architectures. The difference between ConvLSTM and FC-LSTM was nevertheless small. Neural models underestimated rainfall magnitude and predicted too few threshold exceedances on high-rainfall days, while persistence achieved the highest detection performance in every city. Post-hoc analyses showed that the selected models were most sensitive to the latest input day, with broader recent-lag sensitivity in Mumbai. These findings show that gridded inputs alone do not justify ConvLSTM and that architecture choice should follow strong benchmarking across average, spatial and high-rainfall performance.
Read the full abstract ▾
All authors Tanmay Ghosh, Shaurabh Anand, Rakesh Gomaji Nannewar, Nithin Nagaraj
Machine Learning · Cross-listed from cs.CV
Classification of Disease from Lungs X-ray Images using VGG16, VGG19 and ResNet50 Models
Nand Lal Yadav, Rajesh Kumar, Satyendra Singh et al. (+1 more)
With the increase in the number of cases related to respiratory diseases, there is an urgent need to detect them early and diagnose them accurately. Convolutional neural networks have given promising results when used for diagnosing diseases using imaging tests. In this study, we investigate the potential of applying deep learning
algorithms such as VGG16, VGG19, and ResNet50 for classification of lung ailments based on X-ray images. A detailed analysis of the aforementioned models' performances was conducted to assess how well they can classify various types of lung ailments, including pneumonia, tuberculosis, lung cancer, and normal lungs. In order to do that, these deep learning models were trained on a vast amount of X-ray images. The results of our study show that while all three models provide good results, ResNet-50 performs best in comparison with other models due to its efficiency and high level of accuracy. We believe that these deep learning models can be successfully implemented in the practice of diagnosing pulmonary diseases in the future. It helps with early disease detection and improves patient outcomes.
Read the full abstract ▾
All authors Nand Lal Yadav, Rajesh Kumar, Satyendra Singh, Sudhakar Singh
Machine Learning
Simultaneous Coverage and Efficiency Guarantee in Online Conformal Prediction
Rahul Vaze
Adaptive conformal inference (ACI) of Gibbs and Candès and its variants are the standard approach to online conformal prediction under distribution shift, but they suffer from three fundamental limitations. First, their guarantees control only the signed long-run coverage error: persistent miscoverage in one direction can be masked by compensating errors later, so a method can satisfy the theoretical guarantee while being badly wrong for extended periods. Second, existing guarantees say nothing about prediction-set size, so validity can be achieved trivially at the cost of unduly wide prediction sets. Third, the efficiency guarantees that do exist compare against a fixed predictor chosen in hindsight, a
benchmark that becomes increasingly less meaningful once the data-generating distribution shifts, since the very notion of an optimal threshold then changes over time. We consider a unified online learning framework that simultaneously controls absolute, non-cancelling coverage violation and prediction-set efficiency against a dynamically evolving
benchmark for three important models. In the fully
adversarial setting, exploiting the fact that the standard ACI update is exactly projected online gradient descent on the pinball loss, we derive simultaneous coverage and efficiency guarantees for arbitrary monotone Lipschitz efficiency objectives, with no distributional or convexity assumptions. In the
stochastic setting with full-score feedback, we propose a sliding-window
quantile tracker and establish a matching minimax lower bound showing our
algorithm is rate-optimal. In the
covariate-dependent
stochastic setting, we develop a partitioned ACI
algorithm that tracks a function-valued oracle threshold, and derive simultaneous coverage and efficiency guarantees.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CR
Blind, Not Weak: A Best-of-Suite Safety-Utility Frontier for Recover-and-Reguard Defenses Against Encoded VLM Jailbreaks
Haoyu Zhang, Zhuoxi Wang, Shibo Zheng et al. (+9 more)
Safety classifiers ("guards") are the dominant black-box defense for
vision-
language models, yet a guard judges an input's surface form, not its meaning: a harmful request re-encoded as set theory, formal logic, a classical
language, code, or text rendered inside an image slips past a guard that would block it in plain
language - the decode gap. The standard fix is a preprocessor that recovers image content and decodes the encoding before the guard. We build one and evaluate it against an ensemble of eleven encoding attacks - six published implementations, one standard encoding baseline, one adapted and three author-constructed renders - counting a behavior as broken if any attack succeeds. Restoring a view the guard never had is what buys coverage - block rates on image renders go from exactly zero to 67-90% - and what it costs in benign traffic is set by the guard, not by the mechanism: one guard pays 9 benign blocking points for the same 70-point gain another pays 69 for. It still does not make the system safer: against an attacker free to choose among eleven encodings, closing one channel relocates the success rather than removing it, and no ensemble contrast for that step survives multiple-comparison correction. What does lower ensemble attack success is a reguard step that re-screens the recovered pre-decode surface, and it is the one every guard pays for: it raises benign over-refusal on all ten guard-target pairs, where restoring a single channel raises it on some and not others. Across the full guard x target x condition factorial, no configuration reaches an ensemble attack-success rate at or below 40% while holding benign over-refusal under 70%. That empty region is a property of the configurations we sample, not a bound on what recovery-based defenses can reach, and we breach its safety half ourselves.
Read the full abstract ▾
All authors Haoyu Zhang, Zhuoxi Wang, Shibo Zheng, Hanwen Liu, Yi Feng, Haowen Xu, Xiangchen Guan, Yang Chen, Zijian Xiao, Xiao Luo, Mohammad Zandsalimy, Shanu Sushmita
Machine Learning
From Tokens to Watt-hours: Analytical Energy Estimation for LLM Inference on Modern GPUs
Tina Vartziotis, Rodopi Kosteli, Elli Vartziotis et al. (+5 more)
The operational energy consumption of large
language model (LLM) inference is becoming an increasingly important component of the environmental footprint of deployed AI systems. However, direct measurement of inference energy often requires hardware telemetry, power instrumentation, or infrastructure-specific monitoring, limiting its applicability in comparative studies, early-stage system design, and sustainability reporting. This report presents an analytically structured, empirically calibrated, GPU-level methodology for estimating LLM inference energy on NVIDIA H100-class accelerators without direct runtime measurement. The proposed
estimator combines parameter-scaled
transformer FLOP accounting, calibrated
memory-traffic factors, and hardware-specific energy coefficients for FP16/BF16 tensor-core computation and high-bandwidth-
memory movement. It explicitly separates
prompt prefill from autoregressive decoding, enabling energy estimates for input
tokens, output
tokens, and complete inference requests. The methodology further decomposes total energy into compute, parameter-access, key-value-cache write, and
attention-read components, allowing the scaling behavior with model size, context length, and generated-
token count to be analyzed. The resulting estimates are not intended to replace physical power measurements; rather, they provide transparent, reproducible, and assumption-explicit approximations suitable for model comparison, green-coding analysis, and design-time evaluation of LLM inference workloads.
Read the full abstract ▾
All authors Tina Vartziotis, Rodopi Kosteli, Elli Vartziotis, George Dasoulas, Michael Keckeisen, Konstantinos Skianis, Sotirios Kotsopoulos, Francesca Dominici
Machine Learning · Cross-listed from math.OC
Adaptive Gradient-Based Methods for a Broader Class of Optimization Problems under Performative Prediction
Hiroki Hamaguchi, Yuya Hikima, Hiroshi Sawada et al. (+1 more)
We study optimization under performative prediction, where deploying a model affects the future data distribution. For this setting, several gradient-based approaches have been proposed. However, they typically assume specific data distributions or loss functions, which limit their practical applicability. To overcome these limitations, we propose a gradient-based optimization method with convergence guarantees under substantially weaker assumptions. Our method explicitly estimates the induced distribution shift through finite differences. It enables higher-dimensional optimization across broader classes of loss functions and data distributions. We also propose a practical variant that reduces the number of samples required. Numerical experiments demonstrate that our proposed
algorithms converge faster and more consistently than existing ones.
Read the full abstract ▾
All authors Hiroki Hamaguchi, Yuya Hikima, Hiroshi Sawada, Akiko Takeda
Machine Learning · Cross-listed from cs.MA
A Persona-based Rate Action Index
Hayden Helm and Andrew Dassori
We propose an index for predicting the U.S.\ Federal Open
Market Committee (FOMC) decision to hike/hold/cut the current federal funds target rate based on how a collection of personas responds to current
market conditions. To construct the index, we collected a new
dataset consisting of nearly $25{,}000$ retrievable chunks from publicly available data. We partition the data into per-member corpora and use each as the
retrieval database of a generative system we refer to throughout as a “persona”. We first evaluate the personas across two complementary components of likeness: identifiability and detectability. Each persona's behavior is highly attributable (average member-conditional recall is $ 8\times $ chance) and generated content is nearly indistinguishable from held-out real content ($\hat\tau_{\mathrm{det}} = 0.23$ against a $0.15$ floor). We then present evidence that query-conditioned representations of the personas capture members' monetary-policy stance relative to a known hawk–dove reputational ordering (Kendall's $\tau = 0.63$, $p < 0.001$), substantially outperforming
retrieval-only representations. These representations vary with time and current
market conditions and form the basis of our proposed persona-based rate action index. For the $2022$–$2025$ period the index tracks the rate cycle (Kendall's $\tau = 0.68$, $p < 10^{-6}$) and can be used to construct a simple classifier that predicts per-meeting outcomes at non-trivial accuracy ($0.69$ versus a $0.47$ base rate). Importantly, the index outperforms informative baselines and leads the federal funds target rate by roughly three quarters. As far as we are aware, our results are the first to demonstrate the ability to capture time-varying group behavior via a collection of digital personas.
Read the full abstract ▾
Machine Learning
AgentGFM: A Graph Foundation Model with Node-Agent Information-Flow Control
Jingbo Cui, Jitao Zhao, Di Jin et al. (+1 more)
Graph Foundation Models (GFMs) aim to learn transferable knowledge from multi-domain graphs and adapt to unseen scenarios. As a fundamental source of relational semantics in graphs, the transferability of topological patterns has long been central to GFM research. However, local structural patterns may vary across graphs and even among nodes within the same graph. Despite such structural variation, most existing GFMs rely on manually designed propagation schemes and apply them to new graphs largely unchanged. Such fixed schemes may not suit the diverse structural patterns of different nodes. This raises a key question: can each node autonomously determine how information should be propagated through the graph? We refer to this capability as information-flow control. Inspired by recent advances in
agent technology, we formulate this problem as
agent-based decision making and treat each node as an
agent. Accordingly, we propose AgentGFM, in which all node
agents follow a shared end-to-end trainable policy rather than using independent models. For adaptive information-flow control, each node interacts with the graph through a predict-act-observe-correct process. During the act stage, the node makes three decisions: source reception, signal-channel selection and gain-aware node-wise halting. The resulting observation is compared with the prediction and their discrepancy is used to correct the node state and guide subsequent interactions. Extensive experiments across node-level, graph-level and large-scale transfer scenarios demonstrate the effectiveness of AgentGFM across diverse graph
topologies.
Read the full abstract ▾
All authors Jingbo Cui, Jitao Zhao, Di Jin, Dongxiao He
Machine Learning
The Art of Not Forgetting A Local Learning Architecture for Continual Learning
Ashmith Atmuri and Yashaswini Rao Bhogarajula
We introduce CMP (Cognitive
Memory Primitive), a continual-learning architecture that repre?sents inputs as sparse relational codes, stores them in a two-tier competitive
memory, and learns through local updates without end-to-end backpropagation through its feature-generating system. We investigate whether combining sparse representations, local learning, and persistent
memory can reduce catastrophic forgetting relative to conventional backpropagation-based continual?learning approaches. On a controlled domain-incremental byte-level
language modeling protocol, CMP demonstrates substantially lower backward transfer than a parameter-matched Trans?former trained with online Elastic Weight Consolidation (EWC). Across a three-seed replicated 15-domain experiment, CMP exhibits stable forgetting behavior, while separate head-to-head comparisons and domain-order analyses show consistently lower forgetting than the evaluated
Transformer baseline under the reported experimental settings. We report these findings alongside a substantial single-domain accuracy gap relative to the
Transformer, a null result on a
vision benchmark, and a documented failure to combine CMP with an independent accuracy-improving mechanism, reflecting our commitment to reporting both positive and negative outcomes. These results suggest that the combination of sparse representations, local learning, and persistent
memory is a promising direction for continual learning, while motivating further investigation into the respective roles of learning rules, representations, and architectural design in mitigating catastrophic forgetting.
Read the full abstract ▾
Machine Learning
From Unsupervised Subgroups to Hypothetical State-Intervention Policies: An Evaluation of Selected Subgrouping Methods in Observational Health Data
Vasundhara Acharya and Bulent Yener
Conventional subgroup analyses can yield unstable and difficult-to-interpret conclusions, especially in observational biomedical data where each individual is observed under only one exposure state, true individual treatment effects are unavailable, and causal structure is uncertain. We investigate whether subgroups constructed from pretreatment characteristics, without using exposure, outcome, or estimated treatment-effect information, can serve as interpretable units for budget-constrained policy prioritization. We propose a framework combining causal-discovery-informed
covariate selection, discovery-evaluation sample splitting, inductive unsupervised clustering, uncertainty-aware subgroup selection, and held-out doubly robust policy evaluation. We compare K-means, hard, membership-weighted, and
stochastic Fuzzy C-means,
Bayesian Gaussian mixture models, and a supervised causal-forest-derived CATE-tree comparator. Policies are evaluated under a 70% budget for hypothetical obesity-to-non-obesity and elevated-to-lower-glucose state shifts in the PIMA Indians Diabetes
dataset and for a lifetime-smoking-history contrast in NHANES. The highest estimated ungated utilities were 0.799 for the BMI policy using
Bayesian GMM, 0.735 for the glucose policy using hard or membership-weighted FCM, and 0.775 for the smoking-history policy using K-means. All paired 95% confidence intervals for policy-risk differences included zero, and no comparison remained statistically significant after Holm adjustment.
Bayesian pooling generally preserved ungated allocations, whereas Empirical Bernstein gating was more conservative. Policies with similar estimated utility could nevertheless prioritize different individuals. The findings should be interpreted as assumption-dependent decision-support evidence for hypothetical state contrasts rather than proof of intervention benefit.
Read the full abstract ▾
Machine Learning
HiFloat4 Format for End-To-End Reinforcement Learning Post-Training of Large Language Models
Hei Yi Mak, Shadan Golestan, Hoang Le et al. (+10 more)
We present, to our knowledge, the first end-to-end FP4 RL post-training, in which both the rollout and training policies, including their forward and backward passes, operate at 4-bit precision. A systematic study reveals that the dominant source of degradation in FP4 RL is not training-side quantization error but rollout activation quantization: outliers stretch the dynamic range so far that a large number of activation values underflow to zero under FP4. Counterintuitively, restoring the training policy to higher precision while keeping the rollout in FP4 makes accuracy worse than full FP4 baseline, exposing rollout-training mismatch as the principal failure mode and ruling out standard pretraining-style fixes. We address this with Rollout Residual Quantization (Rollout-ResQ): a single residual correction term constrained to a hardware-friendly sparsity pattern, added only to the FP4 rollout matmul – a lightweight correction that recovers most of the precision lost to outlier-driven underflow without inflating the rollout's compute footprint. On Qwen2.5-3B and Qwen2.5-Math-7B, Rollout-ResQ paired with the HiFloat4 (HiF4) format – whose three-level hierarchical scaling preserves resolution under FP4's tight 4-bit budget – closes the accuracy gap to BF16 from 4.9% to 1.1%, bringing fully quantized FP4 RL within striking distance of full precision. Applied to the open-standard MXFP4, the same recipe narrows the gap from 13.6% to 5.3%, revealing that FP4 format choice is a key factor that determines the ceiling on recoverable accuracy. Together, these results establish HiF4 as the enabling format for end-to-end FP4 RL post-training, and Rollout-ResQ as the activation-side mechanism that makes the gap to BF16 closable.
Read the full abstract ▾
All authors Hei Yi Mak, Shadan Golestan, Hoang Le, Mehran Taghian Jazi, Yunke Peng, Yaoyuan Wang, Yao Wang, Junsong Wang, Tianchi Hu, Fengchen He, Guipeng Hu, Tanzila Rahman, Anandharaju Durai Raju
Machine Learning
Collaborative Weighting with Pessimistic Critic for Mitigating Overestimation in Off-Policy Reinforcement Learning
Gong Gao, Xiao Lai, Ziqi Xie et al. (+3 more)
Deep off-policy reinforcement learning
algorithms for continuous control typically rely on neural value function approximation to guide policy improvement. However, temporal-difference (TD) learning introduces noisy targets, resulting in non-stationary optimization, while greedy policy updates amplify early-stage estimation errors. The recursive propagation of such errors leads to persistent overestimation bias and degraded training stability in actor-critic methods. Existing approaches attempt to alleviate this issue via prioritized sampling or modified value learning objectives, but often overemphasize high-uncertainty transitions caused by limited data coverage or bootstrapping errors, thereby further amplifying bias.In this paper, we propose Collaborative Weighting Actor-Critic (CWAC), a unified framework that explicitly accounts for predictive uncertainty in value estimation. CWAC employs distributional critic to model return uncertainty and introduces a collaborative weighting mechanism that jointly reweights TD-errors and uncertainty, enabling robust learning from reliable samples while suppressing noisy updates. In addition, we incorporate a
stochastic pessimistic value estimation scheme via sampling from the return distribution, which effectively mitigates error propagation during policy improvement. CWAC can be seamlessly integrated into existing off-policy
algorithm frameworks such as SAC, TD3, and DDPG with minimal overhead. Empirical results demonstrate that our proposed method significantly enhances performance across a diverse range of simulated tasks. Our code is publicly available at https://anonymous.4open.science/r/CWAC-348E.
Read the full abstract ▾
All authors Gong Gao, Xiao Lai, Ziqi Xie, Guojie Chen, Xianhui Liu, Weidong Zhao
Machine Learning
From Interface to Inference: Eliciting Any-Order Inference from Any-Order Models
Seunggeun Kim, Jaeyeon Kim, Taekyun Lee et al. (+4 more)
Many discrete
reasoning tasks, such as code generation, are inherently non-causal: programmers move between high-level structure and local details, a process we call any-order inference. For autoregressive
language models, which lack a native any-order interface, non-causal abilities such as infilling and next-edit prediction require hand-designed mechanisms. Can we instead design models that natively support any-order inference? Masked
diffusion models have recently emerged as compelling candidates, as their any-order training objective naturally offers an any-order prediction interface. This interface, however, does not automatically yield any-order inference. We demonstrate that this interface-inference gap stems from positional uncertainty: fixed-canvas,
token-level models may know what semantic component should appear without knowing where to place it. In light of this, we propose two complementary approaches: (1) Insertion-based masked
diffusion, building on FlexMDM (Kim et al, 2025), relaxes fixed-position commitments via insertions, enabling generation across non-contiguous regions. (2) Latent-space masked
diffusion shifts prediction to coarser semantic segments, enabling search over latent generation orders. Empirically, we train a 7B FlexMDM for Python coding and a 125M LatentMDM for GSM8K and show that both approaches induce distinct any-order inference behaviors and improve downstream performance. We release our codebase at https://github.com/SeunggeunKimkr/genuine-any-order.
Read the full abstract ▾
All authors Seunggeun Kim, Jaeyeon Kim, Taekyun Lee, Yuyuan Chen, Yilun Du, Sham Kakade, Sitan Chen
Machine Learning
Effective catchment hydrologic representations are low-dimensional but interaction-dependent
Yuan-Heng Wang and Hoshin V. Gupta
Hydrologic models must reconcile predictive flexibility with physical interpretability, yet the complexity required to reproduce catchment dynamics remains uncertain. Here we compare conceptual hydrologic models and neural networks built from mass-conserving computational units across 513 catchments in the contiguous United States, varying state dimensionality and cross-state information sharing. Under likelihood-based selection, median testing-period Kling-Gupta efficiency skill score (KGEss) increases from 0.83 with one allowable state to 0.89 with two, but reaches only 0.90 with five. KGE-based selection shows similarly rapid saturation. Cross-state information sharing provides an additional source of predictive skill: sharing-enabled candidate spaces restricted to two states outperform non-sharing spaces allowing up to five states, with lower median errors in five of six hydrologic diagnostics. Although likelihood- and KGE-based criteria favor different architectures, both preserve the same structural relationships. Conceptual and neural formulations likewise select different specific architectures but retain the same state dimensionality in 75% of catchments. These findings indicate that effective hydrologic representations are low-dimensional but interaction-dependent, while predictive success alone does not uniquely identify physical storage structures.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AR
LLMET: Enabling Cross-Layer Evaluation of Emerging M3D Memories for Energy-Efficient LLM Serving
Ming-Yen Lee, Hanchen Yang, Faaiq Waqar et al. (+4 more)
The energy consumption of Large
Language Model (LLM) serving is becoming a major system challenge as deployment scales, driven by hardware power and thermal constraints and rising electricity costs. A key contributor to chip energy dissipation is data movement between limited on-chip cache and off-chip High Bandwidth
Memory (HBM). Meanwhile, emerging
memory technologies such as monolithic 3D (M3D) integration of cache
memories at the Back-End-Of-Line (BEOL) of logic chips enable larger and denser on-chip
memories, creating new opportunities to reduce costly off-chip traffic. However, it remains unclear whether continuously scaling on-chip
memory using emerging technologies can effectively improve the energy efficiency of LLM serving. To address this gap, we develop LLMET (LLM with Emerging Technology), a validated cross-layer simulation framework, and conduct a comprehensive study on the impact of large-capacity on-chip
memory technologies across a broad range of models, applications and platforms. Utilizing M3D technology to expand the L2 cache from 40MB to 1GB yields a 44% reduction in chip energy during the Llama3.1-70B prefill phase with a 16K context window, based on LLMET simulation on a dual NVIDIA A100 GPU setup. On the 8x NVIDIA B200-like platform, extending the L2 cache from 128MB to 4GB saves the prefill energy by up to 24%. For the edge platform and workloads, the decode energy saving reaches 30% when increasing the 8MB cache size to 256MB. These results highlight the promise of ultra-large on-chip
memories for energy-efficient LLM serving systems.
Read the full abstract ▾
All authors Ming-Yen Lee, Hanchen Yang, Faaiq Waqar, Harsono Simka, Tushar Krishna, Muhammed Ahosan Ul Karim, Shimeng Yu
Machine Learning · Cross-listed from cs.AI
EvoPINN: Agentic Discovery of Executable Algorithms for Physics-Informed Neural Networks
Peng Yin, Kai Li, Yifan Zhang et al. (+1 more)
Physics-informed neural networks (PINNs) have emerged as a powerful paradigm for solving partial differential equations (PDEs), yet their performance heavily relies on the manual, trial-and-error engineering of neural representations, loss formulations, and optimization dynamics. While Large
Language Models (LLMs) offer a promising avenue for automated design, unconstrained code generation often yields mathematically invalid or numerically unstable solutions under strict scientific computing constraints. To bridge this gap, we propose EvoPINN, an agentic framework that reformulates PINN development from labor-intensive manual design into a rigorous, execution-grounded
algorithm discovery problem. EvoPINN navigates a modular search space by decoupling neural representations from training programs, utilizing an LLM
agent to iteratively propose
memory-conditioned programmatic modifications. To ensure scientific validity, all candidates undergo strict structural verification and budget-matched PDE evaluation. Extensive experiments across diverse PDE regimes (oscillatory,
elliptic, dissipative, and nonlinear transport) demonstrate that EvoPINN discovers PDE-specialized learning
algorithms that significantly reduce relative $L_{2}$ error compared to baselines. Crucially, EvoPINN autonomously invented SLRC-PINN, a novel architecture whose performance gains persist under rigorous parameter-matched comparisons, establishing the viability of execution-grounded
agents for discovering genuinely new scientific computing mechanisms.
Read the full abstract ▾
All authors Peng Yin, Kai Li, Yifan Zhang, Jian Cheng
Machine Learning · Cross-listed from cs.SI
Parameterized Fair Resource Allocation under Diversity Constraints
Keke Huang, Yik Yu Ng, Laks V.S. Lakshmanan et al. (+1 more)
Resource allocation across multiple
agent groups arises in many applications including e-commerce recommendation systems, housing assignment, and course allocation, and is commonly formulated as an optimization problem with diversity constraints to ensure group
fairness. Existing approaches typically enforce these constraints as hard conditions, which overly restrict the feasible solution space and often lead to suboptimal allocations. In this paper, we propose PRA, a parameterized framework for fair resource allocation under diversity constraints. Inspired by the use of risk-aversion parameters in economic models, PRA introduces a set of controllable inequality-aversion parameters to softly regulate group-level diversity, thereby enabling flexible trade-offs between
fairness and allocation efficiency. With appropriately calibrated parameters, PRA yields
fairness-optimal assignments that comply with the specified diversity constraints. To accommodate additional application-specific constraints, we further extend the framework to an adaptive variant, APRA. We establish that the optimality of both PRA and APRA holds regardless of the chosen
fairness metric and the nature of the additional constraints, underscoring the generality and robustness of our approach. Extensive experiments on three real-world applications demonstrate that our proposed framework consistently outperforms existing baselines in both effectiveness and robustness.
Read the full abstract ▾
All authors Keke Huang, Yik Yu Ng, Laks V.S. Lakshmanan, Xiaokui Xiao