Machine Learning · Cross-listed from cs.CV
An Approximate Graph Elicits Detonation Lattice
Vansh Sharma and Venkat Raman
This study presents a novel
algorithm based on graph theory for the precise
segmentation and measurement of detonation
cells from 3D pressure traces, termed detonation
lattices, addressing the limitations of manual and primitive 2D edge detection methods prevalent in the field. Using a
segmentation model, the proposed training-free
algorithm is designed to accurately extract cellular patterns, a longstanding challenge in detonations research. First, the efficacy of
segmentation phase on two synthetic
datasets is evaluated with an error of 2%. Next, 3D simulation data is used to establish performance of the graph-based workflow. The results of statistics and joint probability densities show oblong
cells aligned with the wave propagation axis with 17% deviation, whereas larger dispersion in volume reflects cubic amplification of linear variability. Although the framework is robust, it remains challenging to reliably segment and quantify highly complex cellular patterns. However, the graph-based formulation generalizes across diverse cellular geometries, positioning it as a practical tool for detonation analysis and a strong foundation for future extensions in triple-point collision studies.
Read the full abstract ▾
Machine Learning
FEAT: A Linear-Complexity Foundation Model for Extremely Large Structured Data
Zhenghang Song, Tang Qian, Lu Chen et al. (+7 more)
Structured data is widely used in domains such as healthcare, finance, and scientific data management. Recent studies on structured data foundation models (SFMs) aim to support data analysis and mining tasks over such data, but still face scalability and generalization challenges when applied to real-world enterprise databases. First, many SFMs rely on full self-
attention, which introduces an O(N^2) computational bottleneck and limits the number of tuples that can be processed jointly. Second, directly replacing
attention with linear-complexity sequence models may conflict with the permutation-invariant nature of structured data, introducing artificial order bias and degrading representation quality. Moreover, models trained only on synthetic data may struggle to generalize to the heavy-tailed and heterogeneous distributions commonly found in real-world databases. To address these challenges, we propose FEAT, a linear-complexity foundation model for extremely large structured data. FEAT replaces quadratic
attention with a multi-layer dual-axis encoding architecture. It integrates an adaptive-fusion bidirectional state-space model (AFBM) with convolutional gated linear
attention (Conv-GLA), enabling cross-tuple contextualization in O(N) time while supporting permutation-invariant representation learning. To improve robustness under real-world data skewness, FEAT further adopts a hybrid structural causal pre-training pipeline with a robust reconstruction objective. Experiments on 12 real-world database
benchmarks show that FEAT consistently outperforms representative SFMs on zero-shot tasks and scales linearly with structured-data sample length, achieving up to 50x faster inference latency.
Read the full abstract ▾
All authors Zhenghang Song, Tang Qian, Lu Chen, Yushuai Li, Zhengke Hu, Bingbing Fang, Yumeng Song, Junbo Zhao, Sheng Zhang, Tianyi Li
Machine Learning
From the Inside Out: Progressive Distribution Refinement for Confidence Calibration
Xizhong Yang, Yinan Xia, Huiming Wang et al. (+1 more)
Leveraging the model's internal information as the self-reward signal in Reinforcement Learning (RL) has received extensive
attention due to its label-free nature. While prior works have made significant progress in applying the Test-Time Scaling (TTS) strategies to RL, the discrepancy in internal information between test and training remains inadequately addressed. Moreover, Test-Time Training based on voting-based TTS strategies often suffers from reward hacking problems. To address these issues, we propose DistriTTRL, which leverages the distribution prior of the model's confidence during RL to progressively optimize the reward signal, rather than relying solely on single-query rollouts. Additionally, we mitigate the phenomenon of consistent reward hacking caused by the voting-based TTS strategies through diversity-targeted penalties. Benefiting from this training mechanism where model capability and self-reward signals complement each other, and the mitigation of reward hacking, DistriTTRL has achieved significant performance improvements across multiple models and
benchmarks.
Read the full abstract ▾
All authors Xizhong Yang, Yinan Xia, Huiming Wang, Mofei Song
Machine Learning
msData: A Millisecond-Resolution Network Dataset for Advancing Time Series Foundation Models
Subina Khanal, Seshu Tirupathi, Merim Dzaferagic et al. (+2 more)
Time series foundation models (TSFMs) require diverse, real-world
datasets to adapt across varying domains and temporal frequencies. However, current large-scale
datasets predominantly focus on low-frequency time series with sampling intervals, i.e., time resolution, in the range of seconds to years, hindering their ability to capture the nuances of high-frequency time series data. To address this limitation, we introduce a novel
dataset, msData, that captures millisecond-resolution wireless and traffic conditions from an operational 5G wireless deployment, expanding the scope of TSFMs to incorporate high-frequency data for pre-training. Further, the
dataset introduces a new domain, namely, wireless networks, thus complementing existing more general domains like energy and finance. The
dataset also provides use cases for short-term
forecasting, with prediction horizons spanning from 1 millisecond (1 step) to 96 milliseconds (96 steps). To demonstrate the utility of the
dataset, we
benchmark traditional machine learning models,
transformer-based deep learning models, and TSFMs on
forecasting tasks using representative subsets of the data, including a static mobility pattern within YouTube traffic class and a train mobility pattern within Web Browsing traffic class. Across these data distributions, we demonstrate that most TSFM model configurations perform poorly in both zero-shot and fine-tuned settings. Our work underscores the importance of incorporating high-frequency
datasets during pre-training and
forecasting to enhance architectures, fine-tuning strategies, generalization, and robustness of TSFMs in real-world applications. Code is available at this repository: https://github.com/khanalsubina/msData. Data is available on: \href{https://huggingface.co/
datasets/subinak/Open_RAN_Performance_Measurement_
Dataset_with_Traffic_and_Mobility_Labels}{Hugging Face}.
Read the full abstract ▾
All authors Subina Khanal, Seshu Tirupathi, Merim Dzaferagic, Marco Ruffini, Torben Bach Pedersen
Machine Learning
Optimal uncertainty bounds for multivariate kernel regression under bounded noise: A Gaussian process-based dual function
Amon Lahr, Anna Scampicchio, Johannes Köhler et al. (+1 more)
Non-conservative uncertainty bounds are essential for making reliable predictions about latent functions from noisy data, and thus, a key enabler for safe learning-based control. In this domain, kernel methods such as Gaussian process regression are established techniques, thanks to their inherent uncertainty quantification mechanism. Still, existing bounds either pose strong assumptions on the underlying noise distribution, are conservative, do not directly apply in the multi-output case, or are difficult to integrate into downstream tasks. This paper addresses these limitations by presenting a tight, deterministic bound for multi-output functions in Reproducing Kernel
Hilbert Spaces (RKHSs) subject to bounded noise. It is obtained through an unconstrained, duality-based formulation, which shares the same structure as classic Gaussian process confidence bounds, and can thus be straightforwardly integrated into downstream optimization pipelines. We show that the proposed bound generalizes existing results and illustrate its application using an example inspired by quadrotor dynamics learning.
Read the full abstract ▾
All authors Amon Lahr, Anna Scampicchio, Johannes Köhler, Melanie N. Zeilinger
Machine Learning
Capability-Guided Compression: Toward Interpretability-Aware Budget Allocation for Large Language Models
Rishaank Gupta
Large
language model compression has made substantial progress through
pruning, quantization, and low-rank decomposition, yet a fundamental limitation persists across all existing methods: compression budgets are allocated without any representation of what individual model components functionally encode. We term this the capability-blind compression problem and argue it is a root cause of two well-documented failures – the insensitivity of perplexity-based evaluation to
reasoning capability loss, and the abrupt phase transitions in model performance recently characterized by Ma et al. (2026). We propose Capability-Guided Compression (CGC), a framework that addresses this by using Sparse Autoencoder (SAE)-derived capability density maps to allocate differential compression budgets across
transformer components. Capability density is a formally defined scalar measure combining the feature breadth, activation
entropy, and cross-input consistency of a component's SAE feature activation distribution. We prove theoretically that components with higher capability density exhibit lower structural redundancy and reach their individual phase transition points at lower compression ratios, providing the first pre-compression mechanism for component-level phase transition prediction. Experiments on GPT-2 Medium confirm that capability density is statistically independent of Wanda importance scores (Spearman rho = -0.054, n = 384 heads), establishing it as a genuinely novel compression signal orthogonal to all existing importance metrics. We report a negative result on PPL-based compression comparison and provide a principled diagnosis identifying GPT-2 Medium as an insufficient test bed for the full CGC hypothesis. The theoretical framework, density formalism, and orthogonality finding constitute a foundation for capability-aware compression research.
Read the full abstract ▾
Machine Learning
DISCOVER: A Solver for Distributional Counterfactual Explanations
Yikai Gu, Lele Cao, Bo Zhao et al. (+2 more)
Counterfactual explanations (CE) explain model decisions by identifying input modifications that lead to different predictions. Most existing methods operate at the instance level. Distributional Counterfactual Explanations (DCE) extend this setting by optimizing an optimal transport objective that balances proximity to a factual input distribution and
alignment to a target output distribution, with statistical certification via chance constrained bounds. However, DCE relies on gradient based optimization, while many real-world tabular pipelines are dominated by non-differentiable models. We propose DISCOVER, a model-agnostic solver for distributional counterfactual explanations. DISCOVER preserves the original DCE objective and certification while replacing gradient descent with a budgeted propose-and-select search paradigm. It exploits a sample-wise decomposition of the transport objective to compute per-row impact scores and enforce a top-k intervention budget, focusing edits on the most influential samples. To guide candidate generation without predictor gradients, DISCOVER introduces an OT-guided cone sampling primitive driven by input-side transport geometry. Experiments on multiple tabular
datasets demonstrate strong joint
alignment of input and output distributions, extending distributional counterfactual
reasoning to modern black box learning pipelines. A code repository is available at: https://github.com/VALHALLA9/Discover.
Read the full abstract ▾
All authors Yikai Gu, Lele Cao, Bo Zhao, Lei Lei, Lei You
Machine Learning · Cross-listed from cs.CV
IRIS: A Real-World Benchmark for Inverse Recovery and Identification of Physical Dynamic Systems from Monocular Video
Rasul Khanbayov, Mohamed Rayan Barhdadi, Erchin Serpedin et al. (+1 more)
Unsupervised physical parameter estimation from
video lacks a common
benchmark: existing methods evaluate on non-overlapping synthetic data, the sole real-world
dataset is restricted to single-body systems, and no established protocol addresses governing-equation identification. This work introduces IRIS, a high-fidelity
benchmark comprising 240 real-world
videos captured at 4K resolution and 60fps, spanning both single- and multi-body dynamics with independently measured ground-truth parameters and uncertainty estimates. Each dynamical system is recorded under controlled laboratory conditions and paired with its governing equations, enabling principled evaluation. A standardized evaluation protocol is defined encompassing parameter accuracy, identifiability, extrapolation, robustness, and governing-equation selection. Multiple baselines are evaluated, including a multi-step physics loss formulation and four complementary equation-identification strategies (VLM temporal
reasoning, describe-then-classify prompting, CNN-based classification, and path-based labelling), establishing reference performance across all IRIS scenarios and exposing systematic failure modes that motivate future research. The
dataset, annotations, evaluation toolkit, and all baseline implementations are publicly released.
Read the full abstract ▾
All authors Rasul Khanbayov, Mohamed Rayan Barhdadi, Erchin Serpedin, Hasan Kurban
Machine Learning
Trained Persistent Memory for Frozen Encoder–Decoder LLMs: Six Architectural Methods
Hong Jeong
Frozen encoder–decoder
language models are stateless: the latent representation is discarded after every forward pass, so no information persists across sessions. This paper presents a proof-of-concept pilot study showing that persistent
memory in the continuous latent space of a frozen LLM is feasible – even under severe resource constraints (a single frozen Flan-T5-XL backbone, small trainable adapters, a single
dataset). We implement six architectural methods spanning three injection points and four write mechanisms; unlike text-level
memory systems, every write and read is a differentiable operation on dense vectors. After training only the adapter, the
memory bank continues to accumulate at inference time without gradients, enabling conversational learning. Under a forgetting-curve evaluation on LoCoMo at two capacity scales (1$\times$ and 10$\times$), the stateless baseline scores exactly zero; at 10$\times$ all six trained adapters produce positive
memory-recall curves; at 1$\times$ three methods collapse, revealing capacity as a critical design parameter. Because the
memory bank is a compact numerical array, it can be scaled to arbitrarily large capacity without altering the backbone. We argue that full end-to-end training with larger models, larger data, and orders-of-magnitude larger
memory will yield substantially stronger results; this pilot study establishes the feasibility baseline and design-space taxonomy that such efforts require.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.RO
Controlling Fish Schools via Reinforcement Learning of Virtual Fish Movement
Yusuke Nishii and Hiroaki Kawashima
This study investigates a method to guide and control fish schools using virtual fish trained with reinforcement learning. We utilize 2D virtual fish displayed on a screen to overcome technical challenges such as durability and movement constraints inherent in physical robotic
agents. To address the lack of detailed behavioral models for real fish, we adopt a model-free reinforcement learning approach. First, simulation results show that reinforcement learning can acquire effective movement policies even when simulated real fish frequently ignore the virtual stimulus. Second, real-world experiments with live fish confirm that the learned policy successfully guides fish schools toward specified target directions. Statistical analysis reveals that the proposed method significantly outperforms baseline conditions, including the absence of stimulus and a heuristic "stay-at-edge" strategy. This study provides an early demonstration of how reinforcement learning can be used to influence collective animal behavior through artificial
agents.
Read the full abstract ▾
Machine Learning
Age Predictors Through the Lens of Generalization, Bias Mitigation, and Interpretability: Reflections on Causal Implications
Debdas Paul, Elisa Ferrari, Irene Gravili et al. (+1 more)
Chronological age predictors often fail to achieve out-of-distribution (OOD) gen- eralization due to exogenous attributes such as race, gender, or tissue. Learning an invariant representation with respect to those attributes is therefore essential to improve OOD generalization and prevent overly optimistic results. In predic- tive settings, these attributes motivate bias mitigation; in causal analyses, they appear as confounders; and when protected, their suppression leads to
fairness. We coherently explore these concepts with theoretical rigor and discuss the scope of an interpretable neural network model based on
adversarial representation learning. Using publicly available mouse transcriptomic
datasets, we illustrate the behavior of this model relative to conventional machine learning models. We observe that the outcome of this model is consistent with the predictive results of a published study demonstrating the effects of Elamipretide on mouse skeletal and cardiac muscle. We conclude by discussing the limitations of deriving causal interpretation from such purely predictive models.
Read the full abstract ▾
All authors Debdas Paul, Elisa Ferrari, Irene Gravili, Alessandro Cellerino
Machine Learning
Prior-Informed Neural Network Initialization: A Spectral Approach for Function Parameterizing Architectures
David Orlando Salazar Torres, Diyar Altinses and Andreas Schwung
Neural network architectures designed for function parameterization, such as the Bag-of-Functions (BoF) framework, bridge the gap between the expressivity of deep learning and the interpretability of classical signal processing. However, these models are inherently sensitive to parameter initialization, as traditional data-agnostic schemes fail to capture the structural properties of the target signals, often leading to suboptimal convergence. In this work, we propose a prior-informed design strategy that leverages the intrinsic spectral and temporal structure of the data to guide both network initialization and architectural configuration. A principled methodology is introduced that uses the Fast
Fourier Transform to extract dominant seasonal priors, informing model depth and initial states, and a residual-based regression approach to parameterize trend components. Crucially, this structural
alignment enables a substantial reduction in encoder dimensionality without compromising reconstruction fidelity. A supporting theoretical analysis provides guidance on trend estimation under finite-sample regimes. Extensive experiments on synthetic and real-world
benchmarks demonstrate that
embedding data-driven priors significantly accelerates convergence, reduces performance variability across trials, and improves computational efficiency. Overall, the proposed framework enables more compact and interpretable architectures while outperforming standard initialization baselines, without altering the core training procedure.
Read the full abstract ▾
Machine Learning
FederatedFactory: Generative One-Shot Learning for Extremely Non-IID Distributed Scenarios
Andrea Moleri, Christian Internò, Ali Raza et al. (+4 more)
Federated Learning (FL) enables distributed optimization without compromising data sovereignty. Yet, where local label distributions are mutually exclusive, standard weight aggregation fails due to conflicting optimization trajectories. Often, FL methods rely on pretrained foundation models, introducing unrealistic assumptions. We introduce FederatedFactory, a zero-dependency framework that inverts the unit of federation from discriminative parameters to generative priors. By exchanging generative modules in a single communication round, our architecture supports ex nihilo synthesis of universally class balanced
datasets, eliminating gradient conflict and external prior bias entirely. Evaluations across diverse medical imagery
benchmarks, including MedMNIST and ISIC2019, demonstrate that our approach recovers centralized upper-bound performance. Under pathological heterogeneity, it lifts baseline accuracy from a collapsed 11.36% to 90.57% on CIFAR-10 and restores ISIC2019 AUROC to 90.57%. Additionally, this framework facilitates exact modular unlearning through the deterministic deletion of specific generative modules.
Read the full abstract ▾
All authors Andrea Moleri, Christian Internò, Ali Raza, Markus Olhofer, David Klindt, Fabio Stella, Barbara Hammer
Machine Learning · Cross-listed from cs.RO
Encoding Predictability and Legibility for Style-Conditioned Diffusion Policy
Adrien Jacquet Crétides, Mouad Abrini, Hamed Rahimi et al. (+1 more)
Striking a balance between efficiency and transparent motion is a core challenge in human-
robot collaboration, as highly expressive movements often incur unnecessary time and energy costs. In collaborative environments, legibility allows a human observer a better understanding of the
robot's actions, increasing safety and trust. However, these behaviors result in sub-optimal and exaggerated trajectories that are redundant in low-ambiguity scenarios where the
robot's goal is already obvious. To address this trade-off, we propose Style-Conditioned
Diffusion Policy (SCDP), a modular framework that constrains the trajectory generation of a pre-trained
diffusion model toward either legibility or efficiency based on the environment's configuration. Our method utilizes a post-training pipeline that freezes the base policy and trains a lightweight scene encoder and conditioning predictor to modulate the
diffusion process. At inference time, an ambiguity detection module activates the appropriate conditioning, prioritizing expressive motion only for ambiguous goals and reverting to efficient paths otherwise. We evaluate SCDP on manipulation and
navigation tasks, and results show that it enhances legibility in ambiguous settings while preserving optimal efficiency when legibility is unnecessary, all without retraining the base policy.
Read the full abstract ▾
All authors Adrien Jacquet Crétides, Mouad Abrini, Hamed Rahimi, Mohamed Chetouani
Machine Learning
DynamicGate MLP Conditional Computation via Learned Structural Dropout and Input Dependent Gating for Functional Plasticity
Yong Il Choi
Dropout is a representative regularization technique that stochastically deactivates hidden units during training to mitigate overfitting. In contrast, standard inference executes the full network with dense computation, so its goal and mechanism differ from conditional computation, where the executed operations depend on the input. This paper organizes DynamicGate-MLP into a single framework that simultaneously satisfies both the regularization view and the conditional-computation view. Instead of a random mask, the proposed model learns gates that decide whether to use each unit (or block), suppressing unnecessary computation while implementing sample-dependent execution that concentrates computation on the parts needed for each input. To this end, we define continuous gate probabilities and, at inference time, generate a discrete execution mask from them to select an execution path. Training controls the compute budget via a penalty on expected gate usage and uses a Straight-Through
Estimator (STE) to optimize the discrete mask. We evaluate DynamicGate-MLP on MNIST, CIFAR-10, Tiny-ImageNet,
Speech Commands, and PBMC3k, and compare it with various MLP baselines and MoE-style variants. Compute efficiency is compared under a consistent criterion using gate activation ratios and a layerweighted relative MAC metric, rather than wall-clock latency that depends on hardware and backend kernels.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
PashtoCorp: A 1.25-Billion-Word Corpus, Evaluation Suite, and Reproducible Pipeline for Low-Resource Language Development
Hanif Rahman
We present PashtoCorp, a 1.25-billion-word corpus for Pashto, a
language spoken by 60 million people that remains severely underrepresented in NLP. The corpus is assembled from 39 sources spanning seven HuggingFace
datasets and 32 purpose-built web scrapers, processed through a reproducible pipeline with Arabic-script tokenization, SHA-256 deduplication, and quality filtering. At 1.25B words across 2.81 million documents, PashtoCorp is 40x larger than the OSCAR Pashto subset and 83x larger than the previously largest dedicated Pashto corpus. Continued MLM pretraining of XLM-R-base on PashtoCorp reduces held-out perplexity by 25.1% (8.08->6.06). On WikiANN Pashto NER, the pretrained model improves entity F1 by 10% relative (19.0%->21.0%) and reduces training variance nearly 7x; the largest gain appears at 50 training sentences (+27%), with PashtoCorp covering 97.9% of WikiANN entity vocabulary. On Belebele Pashto reading comprehension, Gemma-3n achieves 64.6% accuracy, the first published LLM baseline for Pashto on this
benchmark. A leave-one-out source ablation shows that Wikipedia (0.7% of documents) is the most critical source for NER: removing it alone reduces entity F1 by 47%. Corpus data, trained model, and code are available at https://huggingface.co/
datasets/ihanif/pashto-corpus, https://huggingface.co/ihanif/xlmr-pashto, and https://github.com/ihanif/pashto-corpus.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Automated identification of Ichneumonoidea wasps via YOLO-based deep learning: Integrating HiresCam for Explainable AI
Joao Manoel Herrera Pinheiro, Gabriela Do Nascimento Herrera, Alvaro Doria Dos Santos et al. (+7 more)
Accurate taxonomic identification of parasitoid wasps within the superfamily Ichneumonoidea is essential for biodiversity assessment, ecological monitoring, and biological control programs. However, morphological similarity, small body size, and fine-grained interspecific variation make manual identification labor-intensive and expertise-dependent. This study proposes a deep learning-based framework for the automated identification of Ichneumonoidea wasps using a YOLO-based architecture integrated with High-Resolution Class Activation Mapping (HiResCAM) to enhance interpretability. The proposed system simultaneously identifies wasp families from high-resolution images. The
dataset comprises 3556 high-resolution images of Hymenoptera specimens. The taxonomic distribution is primarily concentrated among the families Ichneumonidae (n = 786), Braconidae (n = 648), Apidae (n = 466), and Vespidae (n = 460). Extensive experiments were conducted using a curated
dataset, with model performance evaluated through precision, recall, F1 score, and accuracy. The results demonstrate high accuracy of over 96 % and robust generalization across morphological variations. HiResCAM visualizations confirm that the model focuses on taxonomically relevant anatomical regions, such as wing venation,
antennae segmentation, and metasomal structures, thereby validating the biological plausibility of the learned features. The integration of explainable AI techniques improves transparency and trustworthiness, making the system suitable for entomological research to accelerate biodiversity characterization in an under-described parasitoid superfamily.
Read the full abstract ▾
All authors Joao Manoel Herrera Pinheiro, Gabriela Do Nascimento Herrera, Alvaro Doria Dos Santos, Luciana Bueno Dos Reis Fernandes, Ricardo V. Godoy, Eduardo A. B. Almeida, Helena Carolina Onody, Marcelo Andrade Da Costa Vieira, Angelica Maria Penteado-Dias, Marcelo Becker
Machine Learning
Behavioral Steering in a 35B MoE Language Model via SAE-Decoded Probe Vectors: One Agency Axis, Not Five Traits
Jia Qing Yap
We train nine sparse autoencoders (SAEs) on the residual stream of Qwen 3.5-35B-A3B, a 35-billion-parameter Mixture-of-Experts model with a hybrid GatedDeltaNet/
attention architecture, and use them to identify and steer five agentic behavioral traits. Our method trains linear probes on SAE latent activations, then projects the probe weights back through the SAE decoder to obtain continuous steering vectors in the model's native activation space. This bypasses the SAE's top-k discretization, enabling fine-grained behavioral intervention at inference time with no retraining. Across 1,800
agent rollouts (50 scenarios times 36 conditions), we find that autonomy steering at multiplier 2 achieves Cohen's d = 1.01 (p < 0.0001), shifting the model from asking the user for help 78% of the time to proactively executing code and searching the web. Cross-trait analysis, however, reveals that all five steering vectors primarily modulate a single dominant agency axis (the disposition to act independently versus defer to the user), with trait specific effects appearing only as secondary modulations in tool-type composition and dose-response shape. The tool-use vector steers behavior (d = 0.39); the risk-calibration vector produces only suppression. We additionally show that steering only during autoregressive decoding has zero effect (p > 0.35), providing causal evidence that behavioral commitments are computed during prefill in GatedDeltaNet architectures.
Read the full abstract ▾
Machine Learning
Decoding the Critique Mechanism in Large Reasoning Models
Hoang Phan, Quang H. Nguyen, Hung T. Q. Le et al. (+3 more)
Large
Reasoning Models (LRMs) exhibit backtracking and self-verification mechanisms that enable them to revise intermediate steps and reach correct solutions, yielding strong performance on complex logical
benchmarks. We hypothesize that such behaviors are beneficial only when the model has sufficiently strong “critique” ability to detect its own mistakes. This work systematically investigates how current LRMs recover from errors by inserting arithmetic mistakes in their intermediate
reasoning steps. Notably, we discover a peculiar yet important phenomenon: despite the error propagating throughout the entire chain-of-thought (CoT) without any verbalized correction, the model still reaches the correct final answer after the thinking process finishes. This recovery implies the existence of an internal mechanism helping the model to detect errors and trigger self-correction, which we refer to as the hidden critique ability. Building on feature space analysis, we identify a highly interpretable critique vector representing this behavior. Extensive experiments across multiple model scales and families demonstrate that steering latent representations with this vector improves the model's error detection capability and enhances the performance of test-time scaling at no extra training cost. Our findings provide a valuable understanding of LRMs' critique behavior, suggesting a promising direction to control and improve their self-verification mechanism. Our code is available at: https://github.com/mail-research/lrm-critique-vectors.
Read the full abstract ▾
All authors Hoang Phan, Quang H. Nguyen, Hung T. Q. Le, Xiusi Chen, Heng Ji, Khoa D. Doan
Machine Learning · Cross-listed from cs.CV
An Interpretable Machine Learning Framework for Non-Small Cell Lung Cancer Drug Response Analysis
Ann Rachel, Pranav M Pawar, Mithun Mukharjee et al. (+2 more)
Lung cancer is a condition where there is abnormal growth of malignant
cells that spread in an uncontrollable fashion in the lungs. Some common treatment strategies are surgery, chemotherapy, and radiation which aren't the best options due to the heterogeneous nature of cancer. In personalized medicine, treatments are tailored according to the individual's genetic information along with lifestyle aspects. In addition, AI-based deep learning methods can analyze large sets of data to find early signs of cancer, types of tumor, and prospects of treatment. The paper focuses on the development of personalized treatment plans using specific patient data focusing primarily on the genetic profile. Multi-Omics data from Genomics of Drug Sensitivity in Cancer have been used to build a predictive model along with machine learning techniques. The value of the target variable, LN-IC50, determines how sensitive or resistive a drug is. An XGBoost regressor is utilized to predict the drug response focusing on molecular and cellular features extracted from cancer
datasets. Cross-validation and Randomized Search are performed for hyperparameter tuning to further optimize the model's predictive performance. For explanation purposes, SHAP (SHapley Additive exPlanations) was used. SHAP values measure each feature's impact on an individual prediction. Furthermore, interpreting feature relationships was performed using DeepSeek, a large
language model trained to verify the biological validity of the features. Contextual explanations regarding the most important genes or pathways were provided by DeepSeek alongside the top SHAP value constituents, supporting the predictability of the model.
Read the full abstract ▾
All authors Ann Rachel, Pranav M Pawar, Mithun Mukharjee, Raja M, Tojo Mathew
Machine Learning · Cross-listed from cs.AI
Learning to Predict, Discover, and Reason in High-Cardinality Event Sequences
Hugo Math
Electronic control units (ECUs) embedded within modern vehicles generate a large number of asynchronous events known as diagnostic trouble codes (DTCs). These discrete events form complex temporal sequences that reflect the evolving health of the vehicle's subsystems. In the automotive industry, domain experts manually group these codes into higher-level error patterns (EPs) using Boolean rules to characterize system faults and ensure safety. However, as vehicle complexity grows, this manual process becomes increasingly costly, error-prone, and difficult to scale. Notably, the number of unique DTCs in a modern vehicle is on the same order of magnitude as the vocabulary of a natural
language, often numbering in the tens of thousands. This observation motivates a paradigm shift: treating diagnostic sequences as a
language that can be modeled, predicted, and ultimately explained. Traditional statistical approaches fail to capture the rich dependencies and do not scale to high-cardinality
datasets characterized by thousands of nodes, large sample sizes, and long sequence lengths. Specifically, the high cardinality of categorical event spaces in industrial logs poses a significant challenge, necessitating new machine learning architectures tailored to such event-driven systems. This thesis addresses automated fault diagnostics by unifying event sequence modeling, causal discovery, and large
language models (LLMs) into a coherent framework for event streams. It is structured in three parts, reflecting a progressive transition from prediction to causal understanding and finally to
reasoning for vehicle diagnostics. Consequently, we introduce several
Transformer-based architectures for predictive maintenance, scalable sample- and population-level causal discovery frameworks and a multi-
agent system that automates the synthesis of Boolean EP rules.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Locate-then-Sparsify: Attribution Guided Sparse Strategy for Visual Hallucination Mitigation
Tiantian Dang, Chao Bi, Shufan Shen et al. (+3 more)
Despite the significant advancements in Large
Vision-
Language Models (LVLMs), their tendency to generate
hallucinations undermines reliability and restricts broader practical deployment. Among the
hallucination mitigation methods, feature steering emerges as a promising approach that reduces erroneous outputs in LVLMs without increasing inference costs. However, current methods apply uniform feature steering across all layers. This heuristic strategy ignores inter-layer differences, potentially disrupting layers unrelated to
hallucinations and ultimately leading to performance degradation on general tasks. In this paper, we propose Locate-Then-Sparsify for Feature Steering (LTS-FS), a plug-and-play framework which controls the steering intensity according to the
hallucination relevance of each layer. We first construct a
dataset comprising
token-level and sentence-level
hallucination cases. Based on this
dataset, we introduce an attribution method based on causal interventions to quantify the
hallucination relevance of each layer. With the attribution scores across layers, we propose a layerwise strategy that converts these scores into feature steering intensities for individual layers, enabling more precise adjustments specifically on
hallucination-relevant layers. Extensive experiments across multiple LVLMs and
benchmarks demonstrate that LTS-FS effectively mitigates
hallucination while preserving strong performance. Codes are available at https://github.com/huttersadan/LTS-FS.
Read the full abstract ▾
All authors Tiantian Dang, Chao Bi, Shufan Shen, Jinzhe Liu, Qingming Huang, Shuhui Wang
Machine Learning
Laya: A LeJEPA Approach to EEG via Latent Prediction over Reconstruction
Saarang Panchavati, Uddhav Panchavati, Hiroki Nariai et al. (+2 more)
Electroencephalography (
EEG) is a widely used tool for studying
brain function, with applications in clinical neuroscience, diagnosis, and
brain-computer interfaces (BCIs). Recent
EEG foundation models trained on large unlabeled corpora aim to learn transferable representations, but their effectiveness remains unclear; reported improvements over smaller task-specific models are often modest, sensitive to downstream adaptation and fine-tuning strategies, and limited under linear probing. We hypothesize that one contributing factor is the reliance on signal reconstruction as the primary self-supervised learning (SSL) objective, which biases representations toward high-variance artifacts rather than task-relevant neural structure. To address this limitation, we explore an SSL paradigm based on Joint
Embedding Predictive Architectures (JEPA), which learn by predicting latent representations instead of reconstructing raw signals. We introduce Laya, the first
EEG foundation model based on LeJEPA. We show that latent prediction yields representations that encode semantic structure in
EEG: Laya
embeddings track clinically meaningful state changes such as seizure onset, are resilient to noise, and achieve the strongest mean clinical accuracy under frozen linear probing, with particular gains on tasks where relevant neural patterns are subtle and easily obscured by artifacts. Controlled ablations against matched MAE variants confirm that the choice of pretraining objective, rather than architecture or data, is the primary driver of these gains.
Read the full abstract ▾
All authors Saarang Panchavati, Uddhav Panchavati, Hiroki Nariai, Corey Arnold, William Speier
Machine Learning
Physics-integrated neural differentiable modeling for immersed boundary systems
Chenglin Li, Hang Xu, Jianting Chen et al. (+1 more)
Accurately, efficiently, and stably computing complex fluid flows and their
evolution near solid boundaries over long horizons remains challenging. Conventional numerical solvers require fine grids and small time steps to resolve near-wall dynamics, resulting in high computational costs, while purely data-driven surrogate models accumulate rollout errors and lack robustness under extrapolative conditions. To address these issues, this study extends existing neural PDE solvers by developing a physics-integrated differentiable framework for long-horizon prediction of immersed-boundary flows. A key design aspect of the framework includes an important improvement, namely the structural integration of physical principles into an end-to-end differentiable architecture incorporating a PDE-based intermediate velocity module and a multi-direct forcing immersed boundary module, both adhering to the pressure-projection procedure for incompressible flow computation. The computationally expensive pressure projection step is substituted with a learned implicit correction using ConvResNet blocks to reduce cost, and a sub-iteration strategy is introduced to separate the embedded physics module's stability requirement from the surrogate model's time step, enabling stable coarse-grid autoregressive rollouts with large effective time increments. The framework uses only single-step supervision for training, eliminating long-horizon backpropagation and reducing training time to under one hour on a single GPU. Evaluations on
benchmark cases of flow past a stationary cylinder and a rotationally oscillating cylinder at Re=100 show the proposed model consistently outperforms purely data-driven, physics-loss-constrained, and coarse-grid numerical baselines in flow-field fidelity and long-horizon stability, while achieving an approximately 200-fold inference speedup over the high-resolution solver.
Read the full abstract ▾
All authors Chenglin Li, Hang Xu, Jianting Chen, Yanfei Zhang
Machine Learning · Cross-listed from math.NA
Neural Pushforward Samplers for the Fokker-Planck Equation on Embedded Riemannian Manifolds
Andrew Qing He and Wei Cai
In this paper, we extend the Weak
Adversarial Neural Pushforward Method to the Fokker–Planck equation on compact embedded Riemannian
manifolds. The method represents the solution as a probability distribution via a neural pushforward map that is constrained to the
manifold by a retraction layer, enforcing
manifold membership and probability conservation by construction. Training is guided by a weak
adversarial objective using ambient plane-wave test functions, whose intrinsic differential operators are derived in closed form from the geometry of the
embedding, yielding a fully mesh-free and chart-free
algorithm. Both steady-state and time-dependent formulations are developed, and numerical results on a double-well problem on the two-sphere demonstrate the capability of the method in capturing
multimodal invariant distributions on curved spaces.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.IR
ReFORM: Review-aggregated Profile Generation via LLM with Multi-Factor Attention for Restaurant Recommendation
Moonsoo Park, Seulbeen Je and Donghyeon Park
In recommender systems, large
language models (LLMs) have gained popularity for generating descriptive summarization to improve recommendation robustness, along with Graph Convolution Networks. However, existing LLM-enhanced recommendation studies mainly rely on the internal knowledge of LLMs about item titles while neglecting the importance of various factors influencing users' decisions. Although information reflecting various decision factors of each user is abundant in reviews, few studies have actively exploited such insights for recommendation. To address these limitations, we propose a ReFORM: Review-aggregated Profile Generation via LLM with Multi-FactOr Attentive RecoMmendation framework. Specifically, we first generate factor-specific user and item profiles from reviews using LLM to capture a user's preference by items and an item's evaluation by users. Then, we propose a Multi-Factor
Attention to highlight the most influential factors in each user's decision-making process. In this paper, we conduct experiments on two restaurant
datasets of varying scales, demonstrating its robustness and superior performance over state-of-the-art baselines. Furthermore, in-depth analyses validate the effectiveness of the proposed modules and provide insights into the sources of personalization. Our source code and
datasets are available at https://github.com/m0onsoo/ReFORM.
Read the full abstract ▾
Machine Learning
Dual Consensus: Escaping from Spurious Majority in Unsupervised RLVR via Two-Stage Vote Mechanism
Kaixuan Du, Meng Cao, Hang Zhang et al. (+3 more)
Current label-free RLVR approaches for large
language models (LLMs), such as TTRL and Self-reward, have demonstrated effectiveness in improving the performance of LLMs on complex
reasoning tasks. However, these methods rely heavily on accurate pseudo-label estimation and converge on spurious yet popular answers, thereby trapping in a dominant mode and limiting further improvements. Building on this, we propose Dual Consensus Reinforcement Learning (DCRL), a novel self-supervised training method which is capable of generating more reliable learning signals through a two-stage consensus mechanism. The model initially acts as an anchor, producing dominant responses; then it serves as an explorer, generating diverse auxiliary signals via a temporary unlearning process. The final training target is derived from the harmonic mean of these two signal sets. Notably, the process operates entirely without external models or supervision. Across eight
benchmarks and diverse domains, DCRL consistently improves Pass@1 over majority vote while yielding more stable training dynamics. These results demonstrate that DCRL establishes a scalable path toward stronger
reasoning without labels.
Read the full abstract ▾
All authors Kaixuan Du, Meng Cao, Hang Zhang, Yukun Wang, Xiangzhou Huang, Ni Li
Machine Learning
Offline Exploration-Aware Fine-Tuning for Long-Chain Mathematical Reasoning
Yongyu Mu, Jiali Zeng, Fandong Meng et al. (+2 more)
Through encouraging self-exploration, reinforcement learning from verifiable rewards (RLVR) has significantly advanced the mathematical
reasoning capabilities of large
language models. As the starting point for RLVR, the capacity of supervised fine-tuning (SFT) to memorize new chain-of-thought trajectories provides a crucial initialization that shapes the subsequent exploration landscape. However, existing research primarily focuses on facilitating exploration during RLVR training, leaving exploration-aware SFT under-explored. To bridge this gap, we propose Offline eXploration-Aware (OXA) fine-tuning. Specifically, OXA optimizes two objectives: promoting low-confidence verified teacher-
distillation data to internalize previously uncaptured
reasoning patterns, and suppressing high-confidence incorrect self-
distillation data to redistribute probability mass of incorrect patterns toward potentially correct candidates. Experimental results across 6
benchmarks show that OXA consistently improves mathematical
reasoning performance, especially achieving an average gain of $+6$ Pass@1 and $+5$ Pass@$k$ points compared to conventional SFT on the Qwen2.5-1.5B-Math. Crucially, OXA elevates initial policy
entropy, and performance gains persist throughout extensive RLVR training, demonstrating the long-term value of OXA.
Read the full abstract ▾
All authors Yongyu Mu, Jiali Zeng, Fandong Meng, JingBo Zhu, Tong Xiao
Machine Learning
Online Semi-infinite Linear Programming: Efficient Algorithms via Function Approximation
Yiming Zong and Jiashuo Jiang
We consider the dynamic resource allocation problem where the decision space is finite-dimensional, yet the solution must satisfy a large or even infinite number of constraints revealed via streaming data or oracle feedback. We model this challenge as an Online Semi-infinite Linear Programming (OSILP) problem and develop a novel LP formulation to solve it approximately. Specifically, we employ function approximation to reduce the number of constraints to a constant $q$. This addresses a key limitation of traditional online LP
algorithms, whose regret bounds typically depend on the number of constraints, leading to poor performance in this setting. We propose a dual-based
algorithm to solve our new formulation, which offers broad applicability through the selection of appropriate potential functions. We analyze this
algorithm under two classical input models-
stochastic input and random permutation-establishing regret bounds of $O(q\sqrt{T})$ and $O\left(\left(q+q\log{T})\sqrt{T}\right)\right)$ respectively. Note that both regret bounds are independent of the number of constraints, which demonstrates the potential of our approach to handle a large or infinite number of constraints. Furthermore, we investigate the potential to improve upon the $O(q\sqrt{T})$ regret and propose a two-stage
algorithm, achieving $O(q\log{T} + q/\epsilon)$ regret under more stringent assumptions. We also extend our
algorithms to the general function setting. A series of experiments validates that our
algorithms outperform existing methods when confronted with a large number of constraints.
Read the full abstract ▾
Machine Learning
Sample-Efficient Adaptation of Drug-Response Models to Patient Tumors under Strong Biological Domain Shift
Camille Jimenez Cortes, Philippe Lalanda and German Vega
Predicting drug response in patients from preclinical data remains a major challenge in precision oncology due to the substantial biological gap between in vitro
cell lines and patient tumors. Rather than aiming to improve absolute in vitro prediction accuracy, this work examines whether explicitly separating representation learning from task supervision enables more sample-efficient adaptation of drug-response models to patient data under strong biological domain shift. We propose a staged transfer-learning framework in which cellular and drug representations are first learned independently from large collections of unlabeled pharmacogenomic data using autoencoder-based representation learning. These representations are then aligned with drug-response labels on
cell-line data and subsequently adapted to patient tumors using few-shot supervision. Through a systematic evaluation spanning in-domain, cross-
dataset, and patient-level settings, we show that unsupervised pretraining provides limited benefit when source and target domains overlap substantially, but yields clear gains when adapting to patient tumors with very limited labeled data. In particular, the proposed framework achieves faster performance improvements during few-shot patient-level adaptation while maintaining comparable accuracy to single-phase baselines on standard
cell-line
benchmarks. Overall, these results demonstrate that learning structured and transferable representations from unlabeled molecular profiles can substantially reduce the amount of clinical supervision required for effective drug-response prediction, offering a practical pathway toward data-efficient preclinical-to-clinical translation.
Read the full abstract ▾