Machine Learning
Gradient-Stable Attention Heads Signal LLM Correctness
Sophie Ostmeier, Brian Axelrod, Maya Varma et al. (+6 more)
Large
language models (LLMs) often generate plausible yet incorrect answers, posing risks in safety-critical settings such as medicine. Human evaluation is expensive, and LLM-as-judge approaches risk introducing hidden errors. Recent single-pass white-box methods detect contextual
hallucinations using linear probes over model internals, but their generalization to out-of-domain, out-of-task, and training-free settings remains poorly understood. We introduce HeadEntropy, a method that predicts answer correctness by measuring how susceptible each
attention head's pattern is to modification under further gradient updates. The key insight is that the trace of the softmax Jacobian is a monotonic function of 2-Renyi
entropy, linking
attention spread to gradient stability during training, even without the reference annotation. With no training, HeadEntropy reaches 0.736 AUROC, outperforms every training-free baseline in paired comparison and matches a hidden-state probe out-of-domain, at under 1% of inference cost. We evaluate broadly across 5 instruction-tuned LLMs and 5
datasets spanning general knowledge, multi-hop
reasoning, mathematical
reasoning, claim verification, and medicine. The code is available at https://github.com/StanfordMIMI/HeadEntropy.
Read the full abstract ▾
All authors Sophie Ostmeier, Brian Axelrod, Maya Varma, Asad Aali, Yabin Zhang, Magdalini Paschali, Sanmi Koyejo, Curtis Langlotz, Akshay Chaudhari
Machine Learning · Cross-listed from cs.AI
No Need to Train Your RDB Foundation Model
Linjie Xu, Yanlin Zhang, Quan Gan et al. (+2 more)
Relational databases (RDBs) contain vast amounts of heterogeneous tabular information that can be exploited for predictive modeling purposes. But since the space of potential targets is vast across enterprise settings, how can we avoid retraining a new model each time we wish to predict a new quantity of interest? Foundation models based on in-context learning (ICL) offer a convenient option, but so far are largely restricted to single-table operability. In generalizing to multiple interrelated tables, it is essential to compress variably-sized RDB neighborhoods into fixed-length ICL samples for consumption by the decoder. However, the details here are critical: unlike existing supervised learning RDB pipelines, we provide theoretical and empirical evidence that ICL-specific compression should be constrained within high-dimensional RDB columns where all entities share units and roles, not across columns where the relevance of heterogeneous data types cannot be determined without extensive label information. Conditioned on this restriction, we then demonstrate that encoder expressiveness is actually not compromised by excluding trainable parameters. Hence we arrive at a principled family of RDB encoders that can be seamlessly paired with already-existing single-table ICL foundation models, whereby no training or fine-tuning is required. From a practical standpoint, we develop scalable SQL primitives to implement the encoder stage, resulting in the easy-to-use open-source RDBLearn foundation model capable of robust performance on unseen
datasets out of the box.
Read the full abstract ▾
All authors Linjie Xu, Yanlin Zhang, Quan Gan, Minjie Wang, David Wipf
Machine Learning
Physics Aware Neural Networks: Denoising for Magnetic Navigation
Aritra Das, Yashas Shende, Muskaan Chugh et al. (+3 more)
Magnetic-anomaly
navigation, leveraging small-scale variations in the Earth's
magnetic field, is a promising alternative when GPS is unavailable or compromised. Airborne systems face a key challenge in extracting geomagnetic field data: the aircraft itself induces
magnetic noise. Although the classical Tolles-Lawson model addresses this, it inadequately handles stochastically corrupted
magnetic data required for
navigation. To handle
stochastic noise, we propose using two physics-based constraints: divergence-free vector fields and E(3)-equivariance. These ensure the learned
magnetic field obeys Maxwell's equation and that outputs transform correctly with sensor position and orientation. The divergence-free constraint is implemented by training a neural network to output a vector potential A, with the
magnetic field defined as its curl. For E(3)-equivariance, we use tensor products of
geometric tensors represented via spherical harmonics with known rotational transformations. Enforcing physical consistency and restricting the admissible function space acts as an implicit regularizer that improves spatiotemporal performance. We present ablation studies evaluating each constraint alone and jointly across CNNs, MLPs, LTCs, and Contiformers. Continuous-time dynamics and long-term
memory are critical for modelling
magnetic time series; the Contiformer, which provides both, outperforms existing methods. To mitigate data scarcity, we generate synthetic
datasets using the World
Magnetic Model (WMM) and time-series conditional GANs, producing realistic, temporally consistent
magnetic sequences across varied trajectories and environments. Experiments show that
embedding these constraints significantly improves predictive accuracy and physical plausibility, outperforming classical and unconstrained deep learning approaches. Acknowledgement: This work was done in collaboration with Dirac Labs.
Read the full abstract ▾
All authors Aritra Das, Yashas Shende, Muskaan Chugh, Reva Laxmi Chauhan, Arghya Pathak, Debayan Gupta
Machine Learning
On the Sparsifiability of Correlation Clustering: Approximation Guarantees under Edge Sampling
Ibne Farabi Shihab, Sanjeda Akter and Anuj Sharma
Correlation Clustering (CC) is a fundamental unsupervised learning primitive whose strongest LP-based approximation guarantees require $\Theta(n^3)$ triangle inequality constraints and are prohibitive at scale. We initiate the study of sparsification–approximation trade-offs for CC, asking how much edge information is needed to retain LP-based guarantees. We establish a structural dichotomy between pseudometric and general weighted instances. On the positive side, we prove that the VC dimension of the clustering disagreement class is exactly $n{-}1$, yielding additive $\varepsilon$-coresets of optimal size $\tilde{O}(n/\varepsilon^2)$; that at most $\binom{n}{2}$ triangle inequalities are active at any LP vertex, enabling an exact cutting-plane solver; and that a sparsified variant of LP-PIVOT, which imputes missing LP marginals via triangle inequalities, achieves a robust $\frac{10}{3}$-approximation (up to an additive term controlled by an empirically computable imputation-quality statistic $\overline{\Gamma}_w$) once $\tilde{\Theta}(n^{3/2})$ edges are observed, a threshold we prove is sharp. On the negative side, we show via Yao's minimax principle that without pseudometric structure, any
algorithm observing $o(n)$ uniformly random edges incurs an unbounded approximation ratio, demonstrating that the pseudometric condition governs not only tractability but also the robustness of CC to incomplete information.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.NI
LEAD-Drift: Real-time and Explainable Intent Drift Detection by Learning a Data-Driven Risk Score
Md. Kamrul Hossain and Walid Aljoby
Intent-Based Networking (IBN) simplifies network management, but its reliability is challenged by "intent drift", where the network's state gradually deviates from its intended goal, often leading to silent failures. Conventional approaches struggle to detect the subtle, early stages of intent drift, raising alarms only when degradation is significant and failure is imminent, which limits their effectiveness for proactive assurance. To address this, we propose LEAD-Drift, a framework that detects intent drift in real time to enable proactive failure prevention. LEAD-Drift's core contribution is reformulating intent failure detection as a supervised learning problem by training a lightweight neural network on fixed-horizon labels to predict a future risk score. The model's raw output is then smoothed with an Exponential Moving Average (EMA) and passed through a statistically tuned threshold to generate robust, real-time alerts. Furthermore, we enhance the framework with two key features for operational intelligence: a multi-horizon modeling technique for dynamic time-to-failure estimation, and per-alert explainability using SHAP to identify root-cause KPIs. Our evaluation on a time-series
dataset shows LEAD-Drift provides significantly earlier warnings, improving the average lead time by 7.3 minutes (+17.8%) compared to a distance-based baseline. It also reduces alert noise by 80.2% compared to a weighted-KPI heuristic, with only a minor trade-off in lead time. These results demonstrate that LEAD-Drift as a highly effective, interpretable, and operationally efficient solution for proactive network assurance in IBN.
Read the full abstract ▾
Machine Learning
Advancing Analytic Class-Incremental Learning through Vision-Language Calibration
Binyu Zhao, Wei Zhang, Xingrui Yu et al. (+2 more)
Class-incremental learning (CIL) with pre-trained models (PTMs) faces a critical trade-off between efficient adaptation and long-term stability. While analytic learning enables rapid, recursive closed-form updates, its efficacy is often compromised by accumulated errors and feature incompatibility. In this paper, we first conduct a systematic study to dissect the failure modes of PTM-based analytic CIL, identifying representation
rigidity as the primary bottleneck. Motivated by this insight, we propose VILA, a novel dual-branch framework that advances analytic CIL via a two-level
vision-
language calibration strategy. Specifically, we coherently fuse plastic, task-adapted features with a frozen, universal visual anchor at the feature level through
geometric calibration, and leverage cross-modal semantic priors at the decision level to rectify prediction bias. This confluence maintains analytic-learning's extreme efficiency while overcoming its inherent brittleness. Extensive experiments across eight
benchmarks demonstrate that VILA consistently yields superior performance, particularly in fine-grained and long-sequence scenarios. Our framework harmonizes high-fidelity prediction with the simplicity of analytic learning. Our code is available at https://github.com/byzhaoAI/VILA.
Read the full abstract ▾
All authors Binyu Zhao, Wei Zhang, Xingrui Yu, Zhaonian Zou, Ivor Tsang
Machine Learning
ALMo: Interactive Aim-Limit-Defined, Multi-Objective System for Personalized High-Dose-Rate Brachytherapy Treatment Planning and Visualization for Cervical Cancer
Edward Chen, Natalie Dullerud, Pang Wei Koh et al. (+4 more)
In complex clinical decision-making, clinicians must often track a variety of competing metrics defined by aim (ideal) and limit (strict) thresholds. Sifting through these high-dimensional tradeoffs to infer the optimal patient-specific strategy is cognitively demanding and historically prone to variability. In this paper, we address this challenge within the context of High-Dose-Rate (HDR) brachytherapy for cervical cancer, where planning requires strictly managing radiation hot spots while balancing tumor coverage against organ sparing. We present ALMo (Aim-Limit-defined Multi-Objective system), an interactive decision support system designed to infer and operationalize clinician intent. ALMo employs a novel optimization framework that minimizes manual input through automated parameter setup and enables flexible control over toxicity risks. Crucially, the system allows clinicians to navigate the Pareto surface of dosimetric tradeoffs by directly manipulating intuitive aim and limit values. In a retrospective evaluation of 25 clinical cases, ALMo generated treatment plans that consistently met or exceeded manual planning quality, with 65% of cases demonstrating dosimetric improvements. Furthermore, the system significantly enhanced efficiency, reducing average planning time to approximately 17 minutes, compared to the conventional 30-60 minutes. While validated in brachytherapy, ALMo demonstrates a generalized framework for streamlining interaction in multi-criteria clinical decision-making.
Read the full abstract ▾
All authors Edward Chen, Natalie Dullerud, Pang Wei Koh, Thomas Niedermayr, Elizabeth Kidd, Sanmi Koyejo, Carlos Guestrin
Machine Learning
Optimized Certainty Equivalent Risk-Controlling Prediction Sets
Jiayi Huang, Amirmohammad Farzaneh and Osvaldo Simeone
In safety-critical applications such as medical image
segmentation, prediction systems must provide reliability guarantees that extend beyond conventional expected loss control. While risk-controlling prediction sets (RCPS) offer probabilistic guarantees on the expected risk, they fail to capture tail behavior and worst-case scenarios that are crucial in high-stakes settings. This paper introduces optimized certainty equivalent RCPS (OCE-RCPS), a novel framework that provides high-probability guarantees on general optimized certainty equivalent (OCE) risk measures, including conditional value-at-risk (CVaR) and entropic risk. OCE-RCPS leverages upper confidence bounds to identify prediction set parameters that satisfy user-specified risk tolerance levels with provable reliability. We establish theoretical guarantees showing that OCE-RCPS satisfies the desired probabilistic constraint for loss functions such as miscoverage and false negative rate. Experiments on image
segmentation demonstrate that OCE-RCPS consistently meets target satisfaction rates across various risk measures and reliability configurations, while OCE-CRC fails to provide probabilistic guarantees.
Read the full abstract ▾
Machine Learning
Zero-Order Optimization for LLM Fine-Tuning via Learnable Direction Sampling
Valery Parfenov, Grigoriy Evseev, Andrey Veprikov et al. (+3 more)
Fine-tuning large pretrained
language models (LLMs) is a cornerstone of modern NLP, yet its growing
memory demands (driven by backpropagation and large optimizer States) limit deployment in resource-constrained settings. Zero-order (ZO) methods bypass backpropagation by estimating directional derivatives from forward evaluations, offering substantial
memory savings. However, classical ZO
estimators suffer from high variance and an adverse dependence on the parameter dimensionality $d$, which has constrained their use to low-dimensional problems. In this work, we propose a policy-driven ZO framework that treats the sampling distribution over perturbation directions as a learnable policy and updates it to reduce the variance of directional estimates. We develop a practical
algorithm implementing this idea and provide a theoretical analysis, showing that learned sampling distributions improve the quality of gradient information and relax the explicit dependence on $d$ in convergence bounds. Empirically, we validate the approach on challenging LLM fine-tuning
benchmarks, demonstrating substantially improved performance compared to standard ZO baselines. Our results suggest that adaptive direction sampling is a promising route to make ZO fine-tuning viable at scale. The source code is available at https://github.com/
brain-lab-research/zo_ldsd
Read the full abstract ▾
All authors Valery Parfenov, Grigoriy Evseev, Andrey Veprikov, Nikolay Bushkov, Stanislav Moiseev, Aleksandr Beznosikov
Machine Learning
Cumulative Utility Parity for Fair Federated Learning under Intermittent Client Participation
Stefan Behfar and Richard Mortier
In real-world
federated learning (FL) systems, client participation is intermittent, heterogeneous, and often correlated with data characteristics or resource constraints. Existing
fairness approaches in FL primarily focus on equalizing loss or accuracy conditional on participation, implicitly assuming that clients have comparable opportunities to contribute over time. However, when participation itself is uneven, these objectives can lead to systematic under-representation of intermittently available clients, even if per-round performance appears fair. We propose cumulative utility parity, a
fairness principle that evaluates whether clients receive comparable long-term benefit per participation opportunity, rather than per training round. To operationalize this notion, we introduce availability-normalized cumulative utility, which disentangles unavoidable physical constraints from avoidable algorithmic bias arising from scheduling and aggregation. Experiments on temporally skewed, non-IID
federated benchmarks demonstrate that our approach substantially improves long-term representation parity, while maintaining near-perfect performance.
Read the full abstract ▾
Machine Learning
Joint Time Series Chain: Detecting Unusual Evolving Trend across Time Series
Li Zhang, Nital Patel, Xiuqi Li et al. (+1 more)
Time series chain (TSC) is a recently introduced concept that captures the evolving patterns in large scale time series. Informally, a time series chain is a temporally ordered set of subsequences, in which consecutive subsequences in the chain are similar to one another, but the last and the first subsequences maybe be dissimilar. Time series chain has the great potential to reveal latent unusual evolving trend in the time series, or identify precursor of important events in a complex system. Unfortunately, existing definitions of time series chains only consider finding chains in a single time series. As a result, they are likely to miss unexpected evolving patterns in interrupted time series, or across two related time series. To address this limitation, in this work, we introduce a new definition called Joint Time Series Chain, which is specially designed for the task of finding unexpected evolving trend across interrupted time series or two related time series. Our definition focuses on mitigating the robustness issues caused by the gap or interruption in the time series. We further propose an effective ranking criterion to identify the best chain. We demonstrate that our proposed approach outperforms existing TSC work in locating unusual evolving patterns through extensive empirical evaluations. We further demonstrate the utility of our work with a real-life manufacturing application from Intel. Our source code is publicly available at the supporting page https://github.com/lizhang-ts/JointTSC .
Read the full abstract ▾
All authors Li Zhang, Nital Patel, Xiuqi Li, Jessica Lin
Machine Learning
Optimization-Free Graph Embedding via Distributional Kernel for Community Detection
Shuaibin Song, Kai Ming Ting, Kaifeng Zhang et al. (+1 more)
Neighborhood Aggregation Strategy (NAS) is a widely used approach in graph
embedding, underpinning both Graph Neural Networks (GNNs) and Weisfeiler-Lehman (WL) methods. However, NAS-based methods are identified to be prone to over-smoothing-the loss of node distinguishability with increased iterations-thereby limiting their effectiveness. This paper identifies two characteristics in a network, i.e., the distributions of nodes and node degrees that are critical for expressive representation but have been overlooked in existing methods. We show that these overlooked characteristics contribute significantly to over-smoothing of NAS-methods. To address this, we propose a novel weighted distribution-aware kernel that embeds nodes while taking their distributional characteristics into consideration. Our method has three distinguishing features: (1) it is the first method to explicitly incorporate both distributional characteristics; (2) it requires no optimization; and (3) it effectively mitigates the adverse effects of over-smoothing, allowing WL to preserve node distinguishability and expressiveness even after many iterations of
embedding. Experiments demonstrate that our method achieves superior community detection performance via spectral clustering, outperforming existing graph
embedding methods, including deep learning methods, on standard
benchmarks.
Read the full abstract ▾
All authors Shuaibin Song, Kai Ming Ting, Kaifeng Zhang, Tianrun Liang
Machine Learning
Benchmark Leakage Trap: Can We Trust LLM-based Recommendation?
Mingqiao Zhang, Qiyao Peng, Yinghui Wang et al. (+2 more)
The expanding integration of Large
Language Models (LLMs) into recommender systems poses critical challenges to evaluation reliability. This paper identifies and investigates a previously overlooked issue:
benchmark data leakage in LLM-based recommendation. This phenomenon occurs when LLMs are exposed to and potentially memorize
benchmark datasets during pre-training or fine-tuning, leading to artificially inflated performance metrics that fail to reflect true model performance. To validate this phenomenon, we simulate diverse data leakage scenarios by conducting continued pre-training of foundation models on strategically blended corpora, which include user-item interactions from both in-domain and out-of-domain sources. Our experiments reveal a dual-effect of data leakage: when the leaked data is domain-relevant, it induces substantial but spurious performance gains, misleadingly exaggerating the model's capability. In contrast, domain-irrelevant leakage typically degrades recommendation accuracy, highlighting the complex and contingent nature of this contamination. Our findings reveal that data leakage acts as a critical, previously unaccounted-for factor in LLM-based recommendation, which could impact the true model performance. We release our code at https://github.com/yusba1/LLMRec-Data-Leakage.
Read the full abstract ▾
All authors Mingqiao Zhang, Qiyao Peng, Yinghui Wang, Hongtao Liu, Yumeng Wang
Machine Learning · Cross-listed from stat.ML
Locally Private Parametric Methods for Change-Point Detection
Anuj Kumar Yadav, Cemre Cadir, Yanina Y. Shkel et al. (+1 more)
We study parametric change-point detection under local differential
privacy, focusing on the offline localization of a single change point in a sequence of independent observations.~In the non-private setting, we derive improved finite-sample accuracy guarantees for a generalized likelihood-ratio
estimator using martingale methods.~We establish finite-sample minimax lower bounds in both the non-private and locally private settings, showing that localization accuracy is fundamentally limited by the Chernoff information between the pre- and post-change distributions.~Guided by this, we design locally private CPD
algorithms based on aggregated randomized response and binary mechanisms, which maximize the Chernoff information between the induced output distributions in low- and high-
privacy regimes, respectively, and derive their finite-sample guarantees. Together, our upper and lower bounds quantify the statistical cost of
privacy in change-point localization.~As part of our analysis, we also show that strong data processing inequality coefficients of Rényi divergence and Jeffreys–Rényi divergence of order infinity can be computed by restricting to pairs of input distributions with a common support of at most two symbols.~We corroborate our results through empirical evaluation on synthetic data.
Read the full abstract ▾
All authors Anuj Kumar Yadav, Cemre Cadir, Yanina Y. Shkel, Michael Gastpar
Machine Learning · Cross-listed from cs.AI
DiffusionRollout: Uncertainty-Aware Rollout Planning in Long-Horizon PDE Solving
Seungwoo Yoo, Juil Koo, Daehyeon Choi et al. (+1 more)
We propose DiffusionRollout, a novel selective rollout planning strategy for autoregressive
diffusion models, aimed at mitigating error accumulation in long-horizon predictions of physical systems governed by partial differential equations (PDEs). Building on the recently validated probabilistic approach to PDE solving, we further explore its ability to quantify predictive uncertainty and demonstrate a strong correlation between prediction errors and standard deviations computed over multiple samples-supporting their use as a proxy for the model's predictive confidence. Based on this observation, we introduce a mechanism that adaptively selects step sizes during autoregressive rollouts, improving long-term prediction reliability by reducing the compounding effect of conditioning on inaccurate prior outputs. Extensive evaluation on long-trajectory PDE prediction
benchmarks validates the effectiveness of the proposed uncertainty measure and adaptive planning strategy, as evidenced by lower prediction errors and longer predicted trajectories that retain a high correlation with their ground truths.
Read the full abstract ▾
All authors Seungwoo Yoo, Juil Koo, Daehyeon Choi, Minhyuk Sung
Machine Learning · Cross-listed from cs.NI
Multi-Modal Sensing and Fusion in mmWave Beamforming for Connected Vehicles: A Transformer Based Framework
Muhammad Baqer Mollah, Honggang Wang, Mohammad Ataul Karim et al. (+1 more)
Millimeter wave (mmWave) communication, utilizing beamforming techniques to address the inherent path loss limitation, is considered as one of the key technologies to support ever increasing high throughput and low latency demands of connected vehicles. However, adopting standard defined beamforming approach in highly dynamic vehicular environments often incurs high beam training overheads and reduction in the available airtime for communications, which is mainly due to exchanging pilot signals and exhaustive beam measurements. To this end, we present a multi-modal sensing and fusion learning framework as a potential alternative solution to reduce such overheads. In this framework, we first extract the representative features from the sensing modalities by modality specific encoders, then, utilize multi-head cross-modal
attention to learn dependencies and correlations between different modalities, and subsequently fuse the
multimodal features to obtain predicted top-k beams so that the best line-of-sight links can be proactively established. To show the generalizability of the proposed framework, we perform a comprehensive experiment in four different vehicle-to-infrastructure (V2I) and vehicle-to-vehicle (V2V) scenarios from real world
multimodal and 60 GHz mmWave wireless sensing data. The experiment reveals that the proposed framework (i) achieves up to 96.72% accuracy on predicting top-15 beams correctly, (ii) incurs roughly 0.77 dB average power loss, and (iii) improves the overall latency and beam searching space overheads by 86.81% and 76.56% respectively for top-15 beams compared to standard defined approach.
Read the full abstract ▾
All authors Muhammad Baqer Mollah, Honggang Wang, Mohammad Ataul Karim, Hua Fang
Machine Learning · Cross-listed from cs.CV
Towards Sparse Video Understanding and Reasoning
Chenwei Xu, Zhen Ye, Shang Wu et al. (+8 more)
We present \revise (\underline{Re}asoning with \underline{Vi}deo \underline{S}parsity), a multi-round
agent for
video question answering (VQA). Instead of uniformly sampling frames, \revise selects a small set of informative frames, maintains a summary-as-state across rounds, and stops early when confident. It supports proprietary
vision-
language models (VLMs) in a “plug-and-play” setting and enables reinforcement fine-tuning for open-source models. For fine-tuning, we introduce EAGER (Evidence-Adjusted Gain for Efficient
Reasoning), an annotation-free reward with three terms: (1) Confidence gain: after new frames are added, we reward the increase in the log-odds gap between the correct option and the strongest alternative; (2) Summary sufficiency: at answer time we re-ask using only the last committed summary and reward success; (3) Correct-and-early stop: answering correctly within a small turn budget is rewarded. Across multiple VQA
benchmarks, \revise improves accuracy while reducing frames, rounds, and
prompt tokens, demonstrating practical sparse
video reasoning.
Read the full abstract ▾
All authors Chenwei Xu, Zhen Ye, Shang Wu, Weijian Li, Zihan Wang, Zhuofan Xia, Lie Lu, Pranav Maneriker, Fan Du, Manling Li, Han Liu
Machine Learning
Interpretable clustering via optimal multi-way decision trees
Hayato Suzuki, Shunnosuke Ikeda, Naoki Nishimura et al. (+1 more)
Clustering is a fundamental unsupervised learning technique for uncovering data structures to facilitate knowledge discovery and decision-making. While clustering accuracy is crucial, interpretability significantly impacts the practical value of clustering results, particularly in high-risk decision-making contexts. Although decision-tree-based clustering methods offer high interpretability through explicit splitting rules, existing approaches often rely on local greedy search or require expensive computational costs limited to binary splits, resulting in deeper, less interpretable trees. To overcome these limitations, we establish a high-performance computational framework named Interpretable Clustering via Optimal Multi-way Trees (ICOMT). We make three primary contributions. First, we propose a new discretization method for numerical features using one-dimensional K-means clustering to capture data distributions. Second, we formulate a binary linear optimization (BLO) problem to guarantee tree optimality. Third, extensive validation on four public
datasets demonstrates that our ICOMT method outperforms existing baselines, achieving superior clustering accuracy while maintaining shallow, concise tree structures.
Read the full abstract ▾
All authors Hayato Suzuki, Shunnosuke Ikeda, Naoki Nishimura, Yuichi Takano
Machine Learning · Cross-listed from cs.AI
Differentiable Rule Induction from Raw Sequence Inputs
Kun Gao, Katsumi Inoue, Yongzhi Cao et al. (+2 more)
Rule learning-based models are widely used in highly interpretable scenarios due to their transparent structures. Inductive logic programming (ILP), a form of machine learning, induces rules from facts while maintaining interpretability. Differentiable ILP models enhance this process by leveraging neural networks to improve robustness and scalability. However, most differentiable ILP methods rely on symbolic
datasets, facing challenges when learning directly from raw data. Specifically, they struggle with explicit label leakage: The inability to map continuous inputs to symbolic variables without explicit supervision of input feature labels. In this work, we address this issue by integrating a self-supervised differentiable clustering model with a novel differentiable ILP model, enabling rule learning from raw data without explicit label leakage. The learned rules effectively describe raw data through its features. We demonstrate that our method intuitively and precisely learns generalized rules from time series and image data.
Read the full abstract ▾
All authors Kun Gao, Katsumi Inoue, Yongzhi Cao, Hanpin Wang, Feng Yang
Machine Learning
Scenario-Adaptive MU-MIMO OFDM Semantic Communication With Asymmetric Neural Network
Chongyang Li, Tianqian Zhang and Shouyin Liu
Semantic Communication (SemCom) has emerged as a promising paradigm for 6G networks, aiming to extract and transmit task-relevant information rather than minimizing bit errors. However, applying SemCom to realistic downlink Multi-User Multi-Input Multi-Output (MU-
MIMO) Orthogonal Frequency Division Multiplexing (OFDM) systems remains challenging due to severe Multi-User Interference (MUI) and frequency-selective fading. Existing Deep Joint Source-Channel Coding (DJSCC) schemes, primarily designed for point-to-point links, suffer from performance saturation in multi-user scenarios. To address these issues, we propose a scenario-adaptive MU-
MIMO SemCom framework featuring an asymmetric architecture tailored for downlink transmission. At the transmitter, we introduce a scenario-aware semantic encoder that dynamically adjusts feature extraction based on Channel State Information (CSI) and Signal-to-Noise Ratio (SNR), followed by a neural precoding network designed to mitigate MUI in the semantic domain. At the receiver, a lightweight decoder equipped with a novel pilot-guided
attention mechanism is employed to implicitly perform channel equalization and feature calibration using reference pilot symbols. Extensive simulation results over 3GPP channel models demonstrate that the proposed framework significantly outperforms DJSCC and traditional Separate Source-Channel Coding (SSCC) schemes in terms of Peak Signal-to-Noise Ratio (PSNR) and classification accuracy, particularly in low-SNR regimes, while maintaining low latency and computational cost on edge devices.
Read the full abstract ▾
Machine Learning
Out-of-Support Generalisation via Weight-Space Sequence Modelling
Roussel Desmond Nzoyem
As breakthroughs in deep learning transform key industries, models are increasingly required to extrapolate on datapoints found outside the range of the training set, a challenge we coin as out-of-support (OoS) generalisation. However, neural networks frequently exhibit catastrophic failure on OoS samples, yielding unrealistic but overconfident predictions. We address this challenge by reformulating the OoS generalisation problem as a sequence modelling task in the weight space, wherein the training set is partitioned into concentric shells corresponding to discrete sequential steps. Our WeightCaster framework yields plausible, interpretable, and uncertainty-aware predictions without necessitating explicit inductive biases, all the while maintaining high computational efficiency. Emprical validation on a synthetic cosine
dataset and real-world air quality sensor readings demonstrates performance competitive or superior to the state-of-the-art. By enhancing reliability beyond in-distribution scenarios, these results hold significant implications for the wider adoption of artificial intelligence in safety-critical applications.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.IR
LiveNewsBench: Evaluating LLM Web Search Capabilities with Freshly Curated News
Yunfan Zhang, Kathleen McKeown and Smaranda Muresan
Large
Language Models (LLMs) with agentic web search capabilities show strong potential for tasks requiring real-time information access and complex fact
retrieval, yet evaluating such systems remains challenging. We introduce \bench, a rigorous and regularly updated
benchmark designed to assess the agentic web search abilities of LLMs. \bench automatically generates fresh question-answer pairs from recent news articles, ensuring that questions require information beyond an LLM's training data and enabling clear separation between internal knowledge and search capability. The
benchmark features intentionally difficult questions requiring multi-hop search queries, page visits, and
reasoning, making it well-suited for evaluating agentic search behavior. Our automated data curation and question generation pipeline enables frequent
benchmark updates and supports construction of a large-scale training
dataset for agentic web search models, addressing the scarcity of such data in the research community. To ensure reliable evaluation, we include a subset of human-verified samples in the test set. We evaluate a broad range of systems using \bench, including commercial and open-weight LLMs as well as LLM-based web search APIs. The leaderboard,
datasets, and code are publicly available at livenewsbench.com.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
On Calibration of Large Language Models: From Response To Capability
Sin-Han Yang, Cheng-Kuang Wu, Chieh-Yen Lin et al. (+3 more)
Accurate confidence estimation is critical for reliable use of large
language models (LLMs). Prior work on LLM calibration largely focuses on response-level confidence, which estimates the correctness of a single generated output. However, this formulation is misaligned with many practical settings where the central question is how likely a model is to solve a query overall. We show that this mismatch results from the
stochastic nature of modern LLM decoding, under which single-response correctness fails to reflect underlying model capability. To address this issue, we introduce capability calibration, a new evaluation framework for measuring how well query-level confidence aligns with a model's expected accuracy on individual queries. We formally distinguish capability calibration (CC) from response calibration (RC) and show that the two differ both theoretically and empirically. We further show that CC is better suited than RC to applications like pass@k prediction and inference budget allocation. Finally, we evaluate common confidence estimation methods to understand the practical feasibility of CC.
Read the full abstract ▾
All authors Sin-Han Yang, Cheng-Kuang Wu, Chieh-Yen Lin, Yun-Nung Chen, Hung-yi Lee, Shao-Hua Sun
Machine Learning
Fast Swap-Based Element Selection for Multiplication-Free Dimension Reduction
Nobutaka Ono
In this paper, we propose a fast
algorithm for element selection, a multiplication-free form of dimension reduction that produces a dimension-reduced vector by simply selecting a subset of elements from the input. Dimension reduction is a fundamental technique for reducing unnecessary model parameters, mitigating overfitting, and accelerating training and inference. A standard approach is principal component analysis (PCA), but PCA relies on
matrix multiplications; on resource-constrained systems, the multiplication count itself can become a bottleneck. Element selection eliminates this cost because the reduction consists only of selecting elements, and thus the key challenge is to determine which elements should be retained. We evaluate a candidate subset through the minimum mean-squared error of linear regression that predicts a target vector from the selected elements, where the target may be, for example, a one-hot label vector in classification. When an explicit target is unavailable, the input itself can be used as the target, yielding a reconstruction-based criterion. The resulting optimization is combinatorial, and exhaustive search is impractical. To address this, we derive an efficient formula for the objective change caused by swapping a selected and an unselected element, using the
matrix inversion lemma, and we perform a swap-based local search that repeatedly applies objective-decreasing swaps until no further improvement is possible. Experiments on MNIST handwritten-digit images demonstrate the effectiveness of the proposed method.
Read the full abstract ▾
Machine Learning
QuaRK: A Quantum Reservoir Kernel for Time Series Learning
Abdallah Aaraba, Soumaya Cherkaoui, Ola Ahmad et al. (+1 more)
Quantum reservoir computing offers a promising route for time series learning by modelling sequential data via rich
quantum dynamics while the only training required happens at the level of a lightweight classical readout. However, studies featuring efficient and implementable
quantum reservoir architectures along with model learning guarantees remain scarce in the literature. To close this gap, we introduce
QuaRK, an end-to-end framework that couples a hardware-realistic
quantum reservoir featurizer with a kernel-based readout scheme. Given a sequence of sample points, the reservoir injects the points one after the other to yield a compact feature vector from efficiently measured k-local observables using classical shadow
tomography, after which a classical kernel-based readout learns the target mapping with explicit regularization and fast optimization. The resulting pipeline exposes clear computational knobs –
circuit width and depth as well as the measurement budget – while preserving the flexibility of kernel methods to model nonlinear temporal functionals and being scalable to high-dimensional data. We further provide learning-theoretic generalization guarantees for dependent temporal data, linking design and resource choices to finite-sample performance, thereby offering principled guidance for building reliable temporal learners. Empirical experiments validate
QuaRK and illustrate the predicted interpolation and generalization behaviours on synthetic beta-mixing time series tasks.
Read the full abstract ▾
All authors Abdallah Aaraba, Soumaya Cherkaoui, Ola Ahmad, Shengrui Wang