Machine Learning
SPINAL – Scaling-law and Preference Integration in Neural Alignment Layers
Arion Das, Partha Pratim Saha, Amit Dhanda et al. (+3 more)
Direct Preference Optimization (DPO) is a principled, scalable alternative to RLHF for aligning large
language models from pairwise preferences, but its internal
geometric footprint remains undercharacterized, limiting audits, checkpoint comparisons, and failure prediction. We introduce SPINAL (Scaling-law and Preference Integration in Neural
Alignment Layers), a diagnostic that measures how
alignment reshapes representations across depth by tracing localized structural change layer by layer. Across model families, DPO produces a layerwise calibration effect concentrated in the final decoder blocks (often layers 21-30), where preference gradients most directly affect the next-
token distribution. SPINAL encodes each checkpoint as a depth trace over (layer index, contraction score, transport score). The contraction score summarizes how quickly the tail of a layer's spectrum decays (how fast small modes vanish); higher values indicate stronger contraction into fewer effective directions. The transport score summarizes how much the
token distribution shifts between adjacent layers using a bounded overlap measure; lower values indicate shorter, smoother steps through representation space. Aligned checkpoints show a late-layer ramp-up in contraction and a smooth reduction in transport, consistent with tightened and stabilized policy mass, while unaligned models trace higher-
curvature, more entropic, and geometrically incoherent depth paths. Overall,
alignment is geometrically localized: the final layers encode the dominant preference-induced corrections. SPINAL turns this localization into a practical audit signal, quantifying where
alignment concentrates, how strongly it manifests, and when it begins to destabilize during training.
Read the full abstract ▾
All authors Arion Das, Partha Pratim Saha, Amit Dhanda, Vinija Jain, Aman Chadha, Amitava Das
Machine Learning
CEEMDAN-Based Multiscale CNN for Wind Turbine Gearbox Fault Detection
Nejad Alagha, Anis Salwa Mohd Khairuddin, Obada Al-Khatib et al. (+1 more)
Wind turbines play a critical role in the shift toward sustainable energy generation. Their operation relies on multiple interconnected components, and a failure in any of these can compromise the entire system's functionality. Detecting faults accurately is challenging due to the intricate, non-linear, and non-stationary nature of vibration signals, influenced by dynamic loading, environmental variations, and mechanical interactions. As such, effective signal processing techniques are essential for extracting meaningful features to enhance diagnostic accuracy. This study presents a hybrid approach for fault detection in wind turbine gearboxes, combining Complete Ensemble Empirical Mode Decomposition with Adaptive Noise (CEEMDAN) and a Multiscale Convolutional Neural Network (MSCNN). CEEMDAN is employed to decompose vibration signals into intrinsic mode functions, isolating critical features at different time-frequency scales. These are then input into the MSCNN, which performs deep hierarchical feature extraction and classification. The proposed method achieves an F1 Score of 98.95%, evaluated on real-world
datasets, and demonstrates superior performance in both detection accuracy and computational speed compared to existing approaches. This framework offers a balanced solution for reliable and efficient fault diagnosis in wind turbine systems.
Read the full abstract ▾
All authors Nejad Alagha, Anis Salwa Mohd Khairuddin, Obada Al-Khatib, Abigail Copiaco
Machine Learning
Dynamics-inspired Structure Hallucination for Protein-protein Interaction Modeling
Fang Wu and Stan Z. Li
Protein-
protein interaction (PPI) represents a central challenge within the biology field, and accurately predicting the consequences of mutations in this context is crucial for drug design and
protein engineering. Deep learning (DL) has shown promise in
forecasting the effects of such mutations, but is hindered by two primary constraints. First, the structures of mutant
proteins are often elusive to acquire. Secondly, PPI takes place dynamically, which is rarely integrated into the DL architecture design. To address these obstacles, we present a novel framework named Refine-PPI with two key enhancements. First, we introduce a structure refinement module trained by a mask mutation modeling (MMM) task on available wild-type structures, which is then transferred to produce the inaccessible mutant structures. Second, we employ a new kind of
geometric network, called the probability density cloud network (PDC-Net), to capture 3D dynamic variations and encode the atomic uncertainty associated with PPI. Comprehensive experiments on SKEMPI.v2 substantiate the superiority of Refine-PPI over all existing tools for predicting free energy change. These findings underscore the effectiveness of our
hallucination strategy and the PDC module in addressing the absence of mutant
protein structure and modeling
geometric uncertainty.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CR
Cyber Threat Detection and Vulnerability Assessment System using Generative AI and Large Language Model
Keerthi Kumar. M, Swarun Kumar Joginpelly, Sunil Khemka et al. (+2 more)
Background: Cyber-attacks have evolved rapidly in recent years, many individuals and business owners have been affected by cyber-attacks in various ways. Cyber-attacks include various threats such as ransomware, malware, phishing, and Denial of Service (DoS)-related attacks. Challenges: Traditional models such as Generative Artificial Intelligence (AI) and Security Bidirectional Encoder Representations from
Transformers (BERT) were implemented to detect cyber threats. However, the existing Security BERT model has a limited contextual understanding of text data, which has less impact on detecting cyber-attacks. Proposed Methodology: To overcome the above-mentioned challenges, Robustly Optimized Bidirectional Encoder Representations from
Transformers Pretraining Approach (RoBERTa) model is proposed which consists of diverse words of vocabulary understanding. Initially, data are extracted from a Packet Capture (PCAP) file and encrypted using Fully Harmonic Encryption (FHE). Subsequently, a Byte-level and Byte Pair Encoding (BBPE) tokenizer was used to generate
tokens and help maintain the vocabulary for the encrypted values. Then, these values are applied to the RoBERTa model of the
transformer with extensive training. Finally, Softmax is used for the detection and classification of attacks. The proposed RoBERTa model achieved better results than the existing BERT model in terms of accuracy (0.99), recall (0.91), and precision (0.89) respectively.
Read the full abstract ▾
All authors Keerthi Kumar. M, Swarun Kumar Joginpelly, Sunil Khemka, Lakshmi. S R, Navin Chhibber
Machine Learning · Cross-listed from cs.CY
Towards Public Administration Research Based on Interpretable Machine Learning
Zhanyu Liu and Yang Yu
Causal relationships play a pivotal role in research within the field of public administration. Ensuring reliable causal inference requires validating the predictability of these relationships, which is a crucial precondition. However, prediction has not garnered adequate
attention within the realm of quantitative research in public administration and the broader social sciences. The advent of interpretable machine learning presents a significant opportunity to integrate prediction into quantitative research conducted in public administration. This article delves into the fundamental principles of interpretable machine learning while also examining its current applications in social science research. Building upon this foundation, the article further expounds upon the implementation process of interpretable machine learning, encompassing key aspects such as
dataset construction, model training, model evaluation, and model interpretation. Lastly, the article explores the disciplinary value of interpretable machine learning within the field of public administration, highlighting its potential to enhance the generalization of inference, facilitate the selection of optimal explanations for phenomena, stimulate the construction of theoretical hypotheses, and provide a platform for the translation of knowledge. As a complement to traditional causal inference methods, interpretable machine learning ushers in a new era of credibility in quantitative research within the realm of public administration.
Read the full abstract ▾
Machine Learning
Bodla Krishna Vamshi, Rohan Bhatnagar and Haizhao Yang
Large
language models (LLMs) frequently generate factually incorrect or unsupported content, commonly referred to as
hallucinations. Prior work has explored decoding strategies,
retrieval augmentation, and supervised fine-tuning for
hallucination detection, while recent studies show that in-context learning (ICL) can substantially influence factual reliability. However, existing ICL demonstration selection methods often rely on surface-level similarity heuristics and exhibit limited robustness across tasks and models. We propose GA-ICL, a geometry-aware demonstration sampling framework for selecting in-context demonstrations that leverages latent representations extracted from frozen LLMs. By jointly modeling local
manifold structure and class-aware prototype geometry, GA-ICL selects demonstrations based on their proximity to learned prototypes rather than lexical or
embedding similarity alone. Across factual verification (FEVER) and
hallucination detection (HaluEval)
benchmarks, GA-ICL outperforms standard ICL selection baselines in the majority of evaluated settings, with particularly strong gains on dialogue and summarization tasks. The method remains robust under temperature perturbations and model variation, indicating improved stability compared to heuristic
retrieval strategies. While lexical
retrieval can remain competitive in certain question-answering regimes at smaller model scales, our results demonstrate that geometry-aware prototype selection provides a reliable and training-light approach for
hallucination detection without modifying LLM parameters. Extended evaluations on Phi-14B and Qwen3-32B confirm that GA-ICL scales effectively to larger models, outperforming all compared baselines including on QA tasks where smaller models show boundary-condition limitations, offering a principled direction for improved ICL demonstration selection.
Read the full abstract ▾
Machine Learning
EntroLnn: Entropy-Guided Liquid Neural Networks for Operando Refinement of Battery Capacity Fade Trajectories
Wei Li, Wei Zhang and Qingyu Yan
Battery capacity degradation prediction has long been a central topic in battery health analytics, and most studies focus on state of health (SoH) estimation and end of life (EoL) prediction. This study extends the scope to online refinement of the entire capacity fade trajectory (CFT) through EntroLnn, a framework based on
entropy-guided transformable
liquid neural networks (LNNs). EntroLnn treats CFT refinement as an integrated process rather than two independent tasks for pointwise SoH and EoL. We introduce
entropy-based features derived from online temperature fields, applied for the first time in battery analytics, and combine them with customized LNNs that model temporal battery dynamics effectively. The framework enhances both static and dynamic adaptability of LNNs and achieves robust and generalizable CFT refinement across different batteries and operating conditions. The approach provides a high fidelity battery health model with lightweight computation, achieving mean absolute errors of only 0.004577 for CFT and 18 cycles for EoL prediction. This work establishes a foundation for
entropy-informed learning in battery analytics and enables self-adaptive, lightweight, and interpretable battery health prediction in practical battery management systems.
Read the full abstract ▾
Machine Learning
MLB: A Scenario-Driven Benchmark for Evaluating Large Language Models in Clinical Applications
Qing He, Dongsheng Bi, Jianrong Lu et al. (+20 more)
The proliferation of Large
Language Models (LLMs) presents transformative potential for healthcare, yet practical deployment is hindered by the absence of frameworks that assess real-world clinical utility. Existing
benchmarks test static knowledge, failing to capture the dynamic, application-oriented capabilities required in clinical practice. To bridge this gap, we introduce a Medical LLM
Benchmark MLB, a comprehensive
benchmark evaluating LLMs on both foundational knowledge and scenario-based
reasoning. MLB is structured around five core dimensions: Medical Knowledge (MedKQA), Safety and Ethics (MedSE), Medical Record Understanding (MedRU), Smart Services (SmartServ), and Smart Healthcare (SmartCare). The
benchmark integrates 22
datasets (17 newly curated) from diverse Chinese clinical sources, covering 64 clinical specialties. Its design features a rigorous curation pipeline involving 300 licensed physicians. Besides, we provide a scalable evaluation methodology, centered on a specialized judge model trained via Supervised Fine-Tuning (SFT) on expert annotations. Our comprehensive evaluation of 10 leading models reveals a critical translational gap: while the top-ranked model, Kimi-K2-Instruct (77.3% accuracy overall), excels in structured tasks like information extraction (87.8% accuracy in MedRU), performance plummets in patient-facing scenarios (61.3% in SmartServ). Moreover, the exceptional safety score (90.6% in MedSE) of the much smaller Baichuan-M2-32B highlights that targeted training is equally critical. Our specialized judge model, trained via SFT on a 19k expert-annotated medical
dataset, achieves 92.1% accuracy, an F1-score of 94.37%, and a Cohen's Kappa of 81.3% for human-AI consistency, validating a reproducible and expert-aligned evaluation protocol. MLB thus provides a rigorous framework to guide the development of clinically viable LLMs.
Read the full abstract ▾
All authors Qing He, Dongsheng Bi, Jianrong Lu, Minghui Yang, Zixiao Chen, Jiacheng Lu, Jing Chen, Nannan Du, Xiao Cu, Sijing Wu, Peng Xiang, Yinyin Hu, Yi Guo, Chunpu Li, Shaoyang Li, Zhuo Dong, Ming Jiang, Shuai Guo, Liyun Feng, Jin Peng, Jian Wang, Jinjie Gu, Junwei Liu
Machine Learning
TimeGNN-Augmented Hybrid-Action MARL for Fine-Grained Task Partitioning and Energy-Aware Offloading in MEC
Wei Ai, Yun Peng, Yuntao Shou et al. (+2 more)
With the rapid growth of IoT devices and latency-sensitive applications, the demand for both real-time and energy-efficient computing has surged, placing significant pressure on traditional cloud computing architectures. Mobile edge computing (MEC), an emerging paradigm, effectively alleviates the load on cloud centers and improves service quality by offloading computing tasks to edge servers closer to end users. However, the limited computing resources, non-continuous power provisioning (e.g., battery-powered nodes), and highly dynamic systems of edge servers complicate efficient task scheduling and resource allocation. To address these challenges, this paper proposes a multi-
agent deep reinforcement learning
algorithm, TG-DCMADDPG, and constructs a collaborative computing framework for multiple edge servers, aiming to achieve joint optimization of fine-grained task partitioning and offloading. This approach incorporates a temporal graph neural network (TimeGNN) to model and predict time series of multi-dimensional server state information, thereby reducing the frequency of online interactions and improving policy predictability. Furthermore, a multi-
agent deterministic policy gradient
algorithm (DC-MADDPG) in a discrete-continuous hybrid action space is introduced to collaboratively optimize task partitioning ratios, transmission power, and priority scheduling strategies. Extensive simulation experiments confirm that TG-DCMADDPG achieves markedly faster policy convergence, superior energy-latency optimization, and higher task completion rates compared with existing state-of-the-art methods, underscoring its robust scalability and practical effectiveness in dynamic and constrained MEC scenarios.
Read the full abstract ▾
All authors Wei Ai, Yun Peng, Yuntao Shou, Tao Meng, Keqin Li
Machine Learning · Cross-listed from cs.AI
Rational Synthesizers or Heuristic Followers? Analyzing LLMs in RAG-based Question-Answering
Atharv Naphade
Retrieval-Augmented Generation (RAG) is the prevailing paradigm for grounding Large
Language Models (LLMs), yet the mechanisms governing how models integrate groups of conflicting retrieved evidence remain opaque. Does an LLM answer a certain way because the evidence is factually strong, because of a prior belief, or merely because it is repeated frequently? To answer this, we introduce GroupQA, a curated
dataset of 1,635 controversial questions paired with 15,058 diversely-sourced evidence documents, annotated for stance and qualitative strength. Through controlled experiments, we characterize group-level evidence aggregation dynamics: Paraphrasing an argument can be more persuasive than providing distinct independent support; Models favor evidence presented first rather than last, and Larger models are increasingly resistant to adapt to presented evidence. Additionally, we find that LLM explanations to group-based answers are unfaithful. Together, we show that LLMs behave consistently as vulnerable heuristic followers, with direct implications for improving RAG system design.
Read the full abstract ▾
Machine Learning
Prediction of Fault Slip Tendency in CO${_2}$ Storage using Data-space Inversion
Xiaowen He, Su Jiang and Louis J. Durlofsky
Accurately assessing the potential for fault slip is essential in many subsurface operations. Conventional model-based history matching methods, which entail the generation of posterior geomodels calibrated to observed data, can be challenging to apply in coupled flow-geomechanics problems with faults. In this work, we implement a variational autoencoder (VAE)-based data-space inversion (DSI) framework to predict pressure, stress and strain fields, and fault slip tendency, in CO${_2}$ storage projects. The main computations required by the DSI workflow entail the simulation of O(1000) prior geomodels. The posterior distributions for quantities of interest are then inferred directly from prior simulation results and observed data, without the need to generate posterior geomodels. The model used here involves a synthetic 3D system with two faults. Realizations of heterogeneous permeability and porosity fields are generated using geostatistical software, and uncertain geomechanical and fault parameters are sampled for each realization from prior distributions. Coupled flow-geomechanics simulations for these geomodels are conducted using GEOS. A VAE with stacked convolutional long short-term
memory layers is trained, using the prior simulation results, to represent pressure, strain, effective normal stress and shear stress fields in terms of latent variables. The VAE parameterization is used with DSI for posterior predictions, with monitoring wells providing observed pressure and strain data. Posterior results for synthetic true models demonstrate that the DSI-VAE framework gives accurate predictions for pressure, strain, and stress fields and for fault slip tendency. The framework is also shown to reduce uncertainty in key geomechanical and fault parameters.
Read the full abstract ▾
Machine Learning · Cross-listed from math.OC
Dynamic Inclusion and Bounded Multi-Factor Tilts for Robust Portfolio Construction
Roberto Garrone
This paper proposes a portfolio construction framework designed to remain robust under estimation error, non-stationarity, and realistic trading constraints. The methodology combines dynamic asset eligibility, deterministic rebalancing, and bounded multi-factor tilts applied to an equal-weight baseline. Asset eligibility is formalized as a state-dependent constraint on portfolio construction, allowing factor exposure to adjust endogenously in response to observable
market conditions such as liquidity,
volatility, and cross-sectional breadth. Rather than estimating expected returns or covariances, the framework relies on cross-sectional rankings and hard structural bounds to control concentration, turnover, and fragility. The resulting approach is fully algorithmic, transparent, and directly implementable. It provides a robustness-oriented alternative to parametric optimization and unconstrained multi-factor models, particularly suited for long-horizon allocations where stability and operational feasibility are primary objectives.
Read the full abstract ▾
Machine Learning
Efficient Inference for Noisy LLM-as-a-Judge Evaluation
Yiqun T Chen, Sizhu Lu, Sijia Li et al. (+2 more)
Large
language models (LLMs) are increasingly used as automatic evaluators of generative AI outputs, a paradigm often referred to as "LLM-as-a-judge." In practice, LLM judges are imperfect predictions for the underlying truth and can exhibit systematic, non-random errors. Two main approaches have recently been proposed to address this issue: (i) direct measurementerror correction based on misclassification models such as Rogan-Gladen-style
estimators, and (ii) surrogate-outcome approaches such as prediction-powered inference (PPI), which correct bias by calibrating prediction residuals on a small set of gold-standard human labels. In this paper, we systematically study the performance of these two approaches for estimating mean parameters (e.g., average
benchmark scores or pairwise win rates). Leveraging tools from semiparametric efficiency theory, we unify the two classes of
estimators by deriving explicit forms of efficient influence function (EIF)-based efficient
estimators and characterize conditions under which PPI-style
estimators attain strictly smaller asymptotic variance than measurement-error corrections. We verify our theoretical results in simulations and demonstrate the methods on real-data examples. We provide an implementation of the benchmarked methods and comparison utilities at https://github.com/yiqunchen/debias-llm-as-a-judge.
Read the full abstract ▾
All authors Yiqun T Chen, Sizhu Lu, Sijia Li, Moran Guo, Shengyi Li
Machine Learning
Interactive Distillation for Cooperative Multi-Agent Reinforcement Learning
Minwoo Cho, Batuhan Altundas and Matthew Gombolay
Knowledge
distillation (KD) has the potential to accelerate MARL by employing a centralized teacher for decentralized students but faces key bottlenecks. Specifically, there are (1) challenges in synthesizing high-performing teaching policies in complex domains, (2) difficulties when teachers must reason in out-of-distribution (OOD) states, and (3) mismatches between the decentralized students' and the centralized teacher's observation spaces. To address these limitations, we propose HINT (Hierarchical INteractive Teacher-based transfer), a novel KD framework for MARL in a centralized training, decentralized execution setup. By leveraging hierarchical RL, HINT provides a scalable, high-performing teacher. Our key innovation, pseudo off-policy RL, enables the teacher policy to be updated using both teacher and student experience, thereby improving OOD adaptation. HINT also applies performance-based filtering to retain only outcome-relevant guidance, reducing observation mismatches. We evaluate HINT on challenging cooperative domains (e.g., FireCommander for resource allocation, MARINE for tactical combat). Across these
benchmarks, HINT outperforms baselines, achieving improvements of 60% to 165% in success rate.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ME
Archetypal cases for questionnaires with nominal multiple choice questions
Aleix Alcacer and Irene Epifanio
Archetypal analysis serves as an exploratory tool that interprets a collection of observations as convex combinations of pure (extreme) patterns. When these patterns correspond to actual observations within the sample, they are termed archetypoids. For the first time, we propose applying archetypoid analysis to nominal observations, specifically for identifying archetypal cases from questionnaires featuring nominal multiple-choice questions with a single possible answer. This approach can enhance our understanding of a nominal data set, similar to its application in multivariate contexts. We compare this methodology with the use of archetype analysis and probabilistic archetypal analysis and demonstrate the benefits of this methodology using a real-world example: the German credit
dataset.
Read the full abstract ▾
Machine Learning
DynaSTy: A Framework for SpatioTemporal Node Attribute Prediction in Dynamic Graphs
Namrata Banerji and Tanya Berger-Wolf
Accurate multistep
forecasting of node-level attributes on dynamic graphs is critical for applications ranging from financial trust networks to biological networks. Existing spatiotemporal graph neural networks typically assume a static adjacency
matrix. In this work, we propose an end-to-end dynamic edge-biased spatiotemporal model that ingests a multi-dimensional timeseries of node attributes and a timeseries of adjacency
matrices, to predict multiple future steps of node attributes. At each time step, our
transformer-based model injects the given adjacency as an adaptable
attention bias, allowing the model to focus on relevant neighbors as the graph evolves. We further deploy a masked node-time pretraining objective that
primes the encoder to reconstruct missing features, and train with scheduled sampling and a horizon-weighted loss to mitigate compounding error over long horizons. Unlike prior work, our model accommodates dynamic graphs that vary across input samples, enabling
forecasting in multi-system settings such as
brain networks across different subjects, financial systems in different contexts, or evolving social systems. Empirical results demonstrate that our method consistently outperforms strong baselines on Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE).
Read the full abstract ▾
Machine Learning
Imitation Learning for Combinatorial Optimisation under Uncertainty
Prakash Gawas, Antoine Legrain and Louis-Martin Rousseau
Imitation learning (IL) provides a data-driven framework for approximating policies for large-scale combinatorial optimisation problems formulated as sequential decision problems (SDPs), where exact solution methods are computationally intractable. A central but underexplored aspect of IL in this context is the role of the expert that generates training demonstrations. Existing studies employ a wide range of expert constructions, yet lack a unifying framework to characterise their modelling assumptions, computational properties, and impact on learning performance. This paper introduces a systematic taxonomy of experts for imitation learning in combinatorial optimisation under uncertainty. The literature is classified along three principal dimensions: (i) treatment of uncertainty; (ii) level of optimality, distinguishing task-optimal and approximate experts; and (iii) interaction mode with the learner, ranging from one-shot supervision to iterative, interactive schemes. We further identify additional categories capturing other relevant expert characteristics. Building on this taxonomy, we propose a generalised
Dataset Aggregation (DAgger) framework that accommodates multiple expert queries, expert aggregation, and flexible interaction strategies. The proposed framework is evaluated on a dynamic physician-to-patient assignment problem with
stochastic arrivals and capacity constraints. Computational experiments compare learning outcomes across expert types and interaction regimes. The results show that policies learned from
stochastic experts consistently outperform those learned from deterministic or full-information experts, while interactive learning improves solution quality using fewer expert demonstrations. Aggregated deterministic experts provide an effective alternative when
stochastic optimisation becomes computationally challenging.
Read the full abstract ▾
Machine Learning
Inverting Non-Injective Functions with Twin Neural Network Regression
Sebastian J. Wetzel
Non-injective functions are not globally invertible. However, they can often be restricted to locally injective subdomains where the inversion is well-defined. In many settings a preferred solution can be selected even when multiple valid preimages exist or input and output dimensions differ. This manuscript describes a natural reformulation of the inverse learning problem for non-injective functions as a collection of locally invertible problems. More precisely, Twin Neural Network Regression is trained to predict local inverse corrections around known anchor points. By anchoring predictions to points within the same locally invertible region, the method consistently selects a valid branch of the inverse. In contrast to current probabilistic state-of-the art inversion methods, Inverse Twin Neural Network Regression is a deterministic framework for resolving multi-valued inverse mappings. I demonstrate the approach on problems that are defined by mathematical equations or by data, including multi-solution toy problems and
robot arm inverse kinematics.
Read the full abstract ▾
Machine Learning
The Kernel Manifold: A Geometric Approach to Gaussian Process Model Selection
Md Shafiqul Islam, Shakti Prasad Padhy, Douglas Allaire et al. (+1 more)
Gaussian Process (GP) regression is a powerful nonparametric
Bayesian framework, but its performance depends critically on the choice of covariance kernel. Selecting an appropriate kernel is therefore central to model quality, yet remains one of the most challenging and computationally expensive steps in probabilistic modeling. We present a
Bayesian optimization framework built on kernel-of-kernels geometry, using expected divergence-based distances between GP priors to explore kernel space efficiently. A multidimensional scaling (MDS)
embedding of this distance
matrix maps a discrete kernel library into a continuous Euclidean
manifold, enabling smooth BO. In this formulation, the input space comprises kernel compositions, the objective is the log marginal likelihood, and featurization is given by the MDS coordinates. When the divergence yields a valid metric, the
embedding preserves geometry and produces a stable BO landscape. We demonstrate the approach on synthetic
benchmarks, real-world time-series
datasets, and an additive manufacturing case study predicting melt-pool geometry, achieving superior predictive accuracy and uncertainty calibration relative to baselines including Large
Language Model (LLM)-guided search. This framework establishes a reusable probabilistic geometry for kernel search, with direct relevance to GP modeling and deep kernel learning.
Read the full abstract ▾
All authors Md Shafiqul Islam, Shakti Prasad Padhy, Douglas Allaire, Raymundo Arróyave
Machine Learning · Cross-listed from cs.CL
Lost in Execution: On the Multilingual Robustness of Tool Calling in Large Language Models
Zheng Luo, T Pranav Kutralingam, Ogochukwu N Okoani et al. (+3 more)
Large
Language Models (LLMs) are increasingly deployed as
agents that invoke external tools through structured function calls. While recent work reports strong tool-calling performance under standard English-centric evaluations, the robustness of tool calling under
multilingual user interactions remains underexplored. In this work, we introduce MLCL, a diagnostic
benchmark, and conduct a systematic evaluation of
multilingual tool calling across Chinese, Hindi, and the low-resource
language Igbo. Through fine-grained error analysis, we show that many failures occur despite correct intent understanding and tool selection. We identify parameter value
language mismatch as a dominant failure mode, where models generate semantically appropriate parameter values in the user's
language, violating
language-invariant execution conventions. We further evaluate several inference-time system strategies and find that while these strategies substantially reduce
language-induced execution errors, none of them can fully recover English-level performance.
Read the full abstract ▾
All authors Zheng Luo, T Pranav Kutralingam, Ogochukwu N Okoani, Wanpeng Xu, Hua Wei, Xiyang Hu
Machine Learning · Cross-listed from stat.ML
An AI-powered Bayesian Generative Modeling Approach for Arbitrary Conditional Inference
Qiao Liu and Wing Hung Wong
Modern data analysis increasingly requires flexible conditional inference P(X_B | X_A) where (X_A, X_B) is an arbitrary partition of observed variable X. Existing approaches are either restricted to a fixed conditioning structure or depend strongly on the distribution of conditioning masks during training. To address these limitations, we introduce
Bayesian generative modeling (BGM), a unified framework for arbitrary conditional inference. BGM learns a generative model of X via a
stochastic iterative
Bayesian updating
algorithm in which model parameters and latent variables are updated until convergence. Once trained, any conditional distribution can be obtained without retraining. Empirically, BGM achieves superior predictive performance with posterior predictive intervals, demonstrating that a single learned model can serve as a universal engine for conditional prediction with principled uncertainty quantification. We provide theoretical guarantees for convergence of the
stochastic iterative
algorithm, statistical consistency, and conditional risk bounds. The proposed BGM framework leverages modern AI to capture complex relationships among variables while adhering to
Bayesian principles, offering a promising approach for a wide range of applications in modern data science. Code for BGM is available at https://github.com/liuq-lab/bayesgm. Document of BGM is available at https://bayesgm.readthedocs.io.
Read the full abstract ▾
Machine Learning
GlyRAG: Context-Aware Retrieval-Augmented Framework for Blood Glucose Forecasting
Shovito Barua Soumma and Hassan Ghasemzadeh
Accurate blood glucose
forecasting using continuous glucose monitoring (CGM) data can support the early prediction of dysglycemic risk. However, current neural-network-based
forecasting models treat CGM data as a purely numerical sequence without integrating the contextual information contained in CGM signal morphology. Recently, large
language models (LLMs) have shown promise for time-series
forecasting, yet their role as agentic context extractors in diabetes care remains largely unexplored. In this study, we bridge glucose
forecasting and LLM-based contextualization by developing GlyRAG, a context-aware,
retrieval-augmented
forecasting framework that uses an LLM as a contextualization
agent to summarize glucose morphology directly from a timed CGM window. The generated CGM-only narrative is embedded and fused with patch-based glucose representations, while a
retrieval module incorporates similar historical training episodes through cross-
attention. We evaluate GlyRAG on the OhioT1DM and AZT1D
datasets for 5-, 30-, and 60-minute
forecasting horizons. Compared with strong CGM-only baselines, GPT-4 GlyRAG significantly improves long-horizon root mean square error (RMSE) over PatchTST on both
datasets. For example, RMSE decreases from 13.8 to 10.6 at 30 minutes and from 23.1 to 20.2 at 60 minutes on OhioT1DM. LLaMA 3.1 produces smaller but significant long-horizon gains, suggesting that the contextualization pipeline is not limited to GPT-4. Clinical error-grid analyses further show that approximately 85 percent of predictions fall within the clinically acceptable Clarke Error Grid Zones A and B. These results suggest that CGM-derived linguistic context and case-based
retrieval can improve long-horizon glucose
forecasting without requiring additional sensing modalities.
Read the full abstract ▾
Machine Learning
When the Server Steps In: Calibrated Updates for Fair Federated Learning
Tianrun Yu, Kaixiang Zhao, Cheng Zhang et al. (+4 more)
Federated learning (FL) has emerged as a transformative distributed learning paradigm, enabling multiple clients to collaboratively train a global model under the coordination of a central server without sharing their raw training data. While FL offers notable advantages, it faces critical challenges in ensuring
fairness across diverse demographic groups. To address these
fairness concerns, various
fairness-aware debiasing methods have been proposed. However, many of these approaches either require modifications to clients' training protocols or lack flexibility in their aggregation strategies. In this work, we address these limitations by introducing EquFL, a novel server-side debiasing method designed to mitigate bias in FL systems. EquFL operates by allowing the server to generate a single calibrated update after receiving model updates from the clients. This calibrated update is then integrated with the aggregated client updates to produce an adjusted global model that reduces bias. Theoretically, we establish that EquFL converges to the optimal global model achieved by FedAvg and effectively reduces
fairness loss over training rounds. Empirically, we demonstrate that EquFL significantly mitigates bias within the system, showcasing its practical effectiveness.
Read the full abstract ▾
All authors Tianrun Yu, Kaixiang Zhao, Cheng Zhang, Anjun Gao, Yueyang Quan, Zhuqing Liu, Minghong Fang
Machine Learning · Cross-listed from math.NA
Generalized Canonical Polyadic Tensor Decompositions with General Symmetry
Alex Mulrooney and David Hong
Canonical Polyadic (CP) tensor decomposition is a workhorse
algorithm for discovering underlying low-dimensional structure in tensor data. This is accomplished in conventional CP decomposition by fitting a low-rank tensor to data with respect to the least-squares loss. Generalized CP (GCP) decompositions generalize this approach by allowing general loss functions that can be more appropriate, e.g., to model binary and count data or to improve robustness to outliers. However, GCP decompositions do not explicitly account for any
symmetry in the tensors, which commonly arises in modern applications. For example, a tensor formed by stacking the adjacency
matrices of a dynamic graph over time will naturally exhibit
symmetry along the two modes corresponding to the graph nodes. In this paper, we develop a symmetric GCP (SymGCP) decomposition that allows for general forms of
symmetry, i.e.,
symmetry along any subset of the modes. SymGCP accounts for
symmetry by enforcing the corresponding
symmetry in the decomposition. We derive gradients for SymGCP that enable its efficient computation via all-at-once optimization with existing tensor kernels. The form of the gradients also leads to various
stochastic approximations that enable us to develop
stochastic SymGCP
algorithms that can scale to large tensors. We demonstrate the utility of the proposed SymGCP
algorithms with a variety of experiments on both synthetic and real data.
Read the full abstract ▾
Machine Learning
Ontology Neural Networks for Topologically Conditioned Constraint Satisfaction
Jaehong Oh
Neuro-symbolic
reasoning systems face fundamental challenges in maintaining semantic coherence while satisfying physical and logical constraints. Building upon our previous work on Ontology Neural Networks, we present an enhanced framework that integrates topological conditioning with gradient stabilization mechanisms. The approach employs Forman-
Ricci curvature to capture graph
topology, Deep Delta Learning for stable rank-one perturbations during constraint projection, and Covariance
Matrix Adaptation
Evolution Strategy for parameter optimization. Experimental evaluation across multiple problem sizes demonstrates that the method achieves mean energy reduction to 1.15 compared to baseline values of 11.68, with 95 percent success rate in constraint satisfaction tasks. The framework exhibits seed-independent convergence and graceful scaling behavior up to twenty-node problems, suggesting that topological structure can inform gradient-based optimization without sacrificing interpretability or computational efficiency.
Read the full abstract ▾
Machine Learning
TIME: Temporally Intelligent Meta-reasoning Engine for Context-Triggered Explicit Reasoning
Susmit Das
Reasoning-oriented
language models typically expose explicit
reasoning as a long, front-loaded chain of "thinking"
tokens before the main output, either always enabled or externally toggled at inference time. Although this can help on arithmetic, coding, and other multi-step tasks, it is costly, weakens claim-level auditability, and does not allow the model to re-trigger explicit
reasoning once presentation has begun. In dialogue, these limitations are compounded by weak sensitivity to temporal structure: unless time is explicitly stated in text, standard models treat replies separated by seconds and replies separated by weeks as equivalent. We introduce TIME (Temporally Intelligent Meta-
reasoning Engine), a behavioral
alignment framework that learns explicit
reasoning as a context-triggered control policy rather than a fixed response mode. TIME augments dialogue with optional ISO 8601 <time> tags, tick events that represent silent time passage, and short <think> blocks that may appear anywhere in a response. Using a four-phase curriculum, including a small maximally diverse full-batch
alignment stage, we train Qwen3 dense models to invoke brief, in-place
reasoning bursts only when contextual cues warrant them, while keeping user-facing output compact. We also introduce TIMEBench, a diagnostic
benchmark for evaluating
reasoning from temporal cues in dialogue. Across 4B-32B scales, TIME improves TIMEBench scores over the corresponding base Qwen3 models in both thinking and no-thinking modes while reducing explicit
reasoning tokens by roughly an order of magnitude. Beyond score improvements, TIME induces a distinct behavioral shift: explicit
reasoning becomes more compact and more responsive to contextual cues. Code, training data, and
benchmark artifacts are publicly available.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.SI
Optimizing Digital Adjudication through Social Network Analysis: An Empirical Study of Credit Card Disputes in Beijing
Chung Han Tsai, ChengTo Lin, Chung Han Tsai et al. (+9 more)
Amid the rapid digitalization of judicial systems, the integration of big data into adjudication remains underexplored, particularly in uncovering the structural logic of legal applications. This study bridges this gap by employing social network analysis (SNA) to examine credit card disputes involving personal information protection adjudicated in Beijing. By constructing a legal citation network, we reveal the latent patterns of substantive and procedural law application. The findings demonstrate that SNA can effectively identify core legal norms and typify cases, offering a robust methodological framework for optimizing 'Digital Court' systems. These insights provide practical pathways for enhancing judicial efficiency and consistency through data-driven case
retrieval and holistic judicial information networks.
Read the full abstract ▾
All authors Chung Han Tsai, ChengTo Lin, Chung Han Tsai, ChengTo Lin, Baowen Zhang, Qingyue Deng, Yunhui Zhao, Zhijia Song, Baowen Zhang, Qingyue Deng, Yunhui Zhao, Zhijia Song
Machine Learning · Cross-listed from stat.ML
Machine learning assisted state prediction of misspecified linear dynamical system via modal reduction
Rohan Vitthal Thorat and Rajdip Nayek
Accurate prediction of structural dynamics is imperative for preserving digital twin fidelity throughout operational lifetimes. Parametric models with fixed nominal parameters often omit critical physical effects due to simplifications in geometry, material behavior, damping, or boundary conditions, resulting in model form errors (MFEs) that impair predictive accuracy. This work introduces a comprehensive framework for MFE estimation and correction in high-dimensional finite element (FE) based structural dynamical systems. The Gaussian Process Latent Force Model (GPLFM) represents discrepancies non-parametrically in the reduced modal domain, allowing a flexible data-driven characterization of unmodeled dynamics. A linear
Bayesian filtering approach jointly estimates system states and discrepancies, incorporating epistemic and aleatoric uncertainties. To ensure computational tractability, the FE system is projected onto a reduced modal basis, and a mesh-invariant neural network maps modal states to discrepancy estimates, permitting model rectification across different FE discretizations without retraining. Validation is undertaken across five MFE scenarios-including incorrect beam theory, damping misspecification, misspecified boundary condition, unmodeled material nonlinearity, and local damage demonstrating the surrogate model's substantial reduction of displacement and rotation prediction errors under unseen excitations. The proposed methodology offers a potential means to uphold digital twin accuracy amid inherent modeling uncertainties.
Read the full abstract ▾
Machine Learning
MoEBlaze: Breaking the Memory Wall for Efficient MoE Training on Modern GPUs
Jiyuan Zhang, Yining Liu, Siqi Yan et al. (+6 more)
The pervasive "
memory wall" bottleneck is significantly amplified in modern large-scale Mixture-of-Experts (MoE) architectures. MoE's inherent architectural sparsity leads to sparse arithmetic compute and also introduces substantial activation
memory overheads – driven by large
token routing buffers and the need to materialize and buffer intermediate tensors. This
memory pressure limits the maximum batch size and sequence length that can fit on GPUs, and also results in excessive data movements that hinders performance and efficient model scaling. We present MoEBlaze, a
memory-efficient MoE training framework that addresses these issues through a co-designed system approach: (i) an end-to-end
token dispatch and MoE training method with optimized data structures to eliminate intermediate buffers and activation materializing, and (ii) co-designed kernels with smart activation checkpoint to mitigate
memory footprint while simultaneously achieving better performance. We demonstrate that MoEBlaze can achieve over 4x speedups and over 50%
memory savings compared to existing MoE frameworks.
Read the full abstract ▾
All authors Jiyuan Zhang, Yining Liu, Siqi Yan, Lisen Deng, Jennifer Cao, Shuqi Yang, Min Ni, Bi Xue, Shen Li
Machine Learning
Optimal Lower Bounds for Online Multicalibration
Natalie Collina, Jiuyao Lu, Georgy Noarov et al. (+1 more)
We prove tight lower bounds for online multicalibration, establishing an information-theoretic separation from marginal calibration. In the general setting where group functions can depend on both context and the learner's predictions, we prove an $\Omega(T^{2/3})$ lower bound on expected multicalibration error using just three disjoint binary groups. This matches the upper bounds of Noarov et al. (2025) up to logarithmic factors and exceeds the $O(T^{2/3-\varepsilon})$ upper bound for marginal calibration (Dagan et al., 2025), thereby separating the two problems. We then turn to lower bounds for the more difficult case of group functions that may depend on context but not on the learner's predictions. In this case, we establish an $\widetilde{\Omega}(T^{2/3})$ lower bound for online multicalibration via an $O(\log^3 T)$-sized group family constructed from an orthonormal basis, again matching upper bounds up to logarithmic factors.
Read the full abstract ▾
All authors Natalie Collina, Jiuyao Lu, Georgy Noarov, Aaron Roth