Machine Learning
Live LTL Progress Tracking: Towards Task-Based Exploration
Noel Brindise, Cedric Langbort and Melkior Ornik
Motivated by the challenge presented by non-Markovian objectives in reinforcement learning (RL), we present a novel framework to track and represent the progress of autonomous
agents through complex, multi-stage tasks. Given a specification in finite linear temporal logic (LTL), the framework establishes a 'tracking vector' which updates at each time step in a trajectory rollout. The values of the vector represent the status of the specification as the trajectory develops, assigning true, false, or 'open' labels (where 'open' is used for indeterminate cases). Applied to an LTL formula tree, the tracking vector can be used to encode detailed information about how a task is executed over a trajectory, providing a potential tool for new performance metrics, diverse exploration, and reward shaping. In this paper, we formally present the framework and
algorithm, collectively named Live LTL Progress Tracking, give a simple working example, and demonstrate avenues for its integration into RL models. Future work will apply the framework to problems such as task-space exploration and diverse solution-finding in RL.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.DC
TensorDex: A Compact, Tensor-Centric Storage System for Modern AI Models
Tingfeng Lan, Zirui Wang, Yunjia Zheng et al. (+3 more)
Modern model hubs store hundreds of petabytes of large
language models (LLMs), with fine-tuned variants dominating the storage footprint. These variants contain substantial cross-model redundancy that delta compression can exploit by storing only the difference between a target and a reference model. However, compression effectiveness depends critically on choosing a similar reference. At model-hub scale, this is challenging because model lineage metadata is often missing or unreliable, and different tensors within the same model may be most similar to tensors from different models. Consequently, model-level pairing leaves substantial redundancy unexploited. We present TensorDex, a lossless, tensor-centric storage system that performs delta compression at tensor granularity. TensorDex decomposes models into tensors, uses compact bit-level fingerprints to predict tensor-pair compressibility, and incrementally organizes tensors into multi-center clusters to select effective bases as new models arrive. Evaluated on 2,890 randomly sampled Hugging Face models, TensorDex reduces storage footprint by 70.5%, 37% lower than the state-of-the-art design. It achieves 22.9 GB/s compression and 28.4 GB/s decompression throughput, 3.86x and 1.49x faster than the next-best system, respectively.
Read the full abstract ▾
All authors Tingfeng Lan, Zirui Wang, Yunjia Zheng, Zhaoyuan Su, Juncheng Yang, Yue Cheng
Machine Learning
Tree of Concepts: Interpretable Continual Learners in Non-Stationary Clinical Domains
Dongkyu Cho, Xiyue Li, Samrachana Adhikari et al. (+1 more)
Continual learning aims to update models under distribution shift without forgetting, yet many high-stakes deployments, such as healthcare, also require interpretability. In practice, models that adapt well (e.g., deep networks) are often opaque, while models that are interpretable (e.g., decision trees) are brittle under shift, making it difficult to achieve both properties simultaneously. In response, we propose Tree of Concepts, an interpretable continual learning framework that uses a shallow decision tree to define a fixed, rule-based concept interface and trains a concept bottleneck model to predict these concepts from raw features. Continual updates act on the concept extractor and label head while keeping concept semantics stable over time, yielding explanations that do not drift across sequential updates. On multiple tabular healthcare
benchmarks under continual learning protocols, our method achieves a stronger stability-plasticity trade-off than existing baselines, including replay-enhanced variants. Our results suggest that structured concept interfaces can support continual adaptation while preserving a consistent audit interface in non-stationary, high-stakes domains.
Read the full abstract ▾
All authors Dongkyu Cho, Xiyue Li, Samrachana Adhikari, Rumi Chunara
Machine Learning · Cross-listed from cs.CV
EvoComp: Learning Visual Token Compression for Multimodal Large Language Models via Semantic-Guided Evolutionary Labeling
Jiafei Song, Fengwei Zhou, Jin Qu et al. (+7 more)
Recent
Multimodal Large
Language Models (MLLMs) have demonstrated strong performance on
vision-
language understanding tasks, yet their inference efficiency is often hampered by the large number of visual
tokens, particularly in high-resolution or multi-image scenarios. To address this issue, we propose EvoComp, a visual
token compression framework that significantly reduces
token count while preserving task accuracy. EvoComp introduces a lightweight encoder-only
transformer-based compressor that selects the most informative and non-redundant visual
tokens by jointly considering visual and textual contexts. A core challenge lies in providing effective supervision for training the compressor. To this end, we design an evolutionary labeling strategy that searches for
token subsets minimizing the MLLM's output loss, while enforcing semantic diversity through vocabulary-based
token grouping. We further train the compressor using a tailored loss function combining the GHM loss to mitigate class and difficulty imbalance, and a cosine similarity regularization to encourage semantic separation between retained and discarded
tokens. Extensive experiments across multiple
vision-
language benchmarks show that EvoComp outperforms existing methods based on
attention or similarity heuristics. Notably, it retains 99.3% of the original accuracy under 3x
token compression and delivers up to 1.6x speedup on mobile devices.
Read the full abstract ▾
All authors Jiafei Song, Fengwei Zhou, Jin Qu, Wenjin Jason Li, Tong Wu, Gengjian Xue, Zhikang Zhao, Daomin Wei, Yichao Lu, Bailin Na
Machine Learning · Cross-listed from cs.CL
Stability-Weighted Decoding for Diffusion Language Models
Yue Wu and Jian Huang
Diffusion large
language models (dLLMs) enable parallel text generation by iteratively denoising a fully masked sequence, unmasking a subset of masked
tokens at each step. Existing decoding strategies rely on static confidence metrics computed at a single denoising step, ignoring temporal history and often leading to premature unmasking of unstable
tokens. In this work, we theoretically establish that a
token's temporal instability, quantified by the KL divergence between consecutive prediction distributions, provides a strict lower bound on its mutual information with the remaining masked context, indicating that temporally unstable
tokens are inherently unsafe to unmask. Based on this insight, we propose Stability-Weighted Decoding (SWD), a training-free, plug-and-play strategy that incorporates temporal stability into
token scoring and acts as a universal modulator for arbitrary score-based decoding policies. Experiments on code generation and mathematical
reasoning benchmarks demonstrate that SWD consistently improves generation accuracy across representative scoring metrics and selection policies, and exhibits exceptional robustness, maintaining a significant performance lead over standard baselines across varying acceleration ratios.
Read the full abstract ▾
Machine Learning · Cross-listed from math.OC
Trajectory-Restricted Optimization Conditions and Geometry-Aware Linear Convergence
Faris Chaudhry, Anthea Monod and Keisuke Yano
Linear convergence of first-order methods is typically characterized by global optimization conditions whose constants reflect worst-case geometry of the ambient space. In high-dimensional or structured problems, these global constants can be arbitrarily conservative and fail to capture the geometry actually encountered by optimization trajectories. In this paper, we develop a trajectory-restricted framework for linear convergence based on localized
geometric regularity. We introduce restricted variants of the Polyak–Łojasiewicz inequality, error bound, and quadratic growth conditions that are required to hold only on subsets of the domain. We show that classical convergence guarantees extend under these localized conditions, and in key cases, we develop new arguments that yield explicit relationships between the corresponding constants. The resulting rates are governed by
geometric quantities associated with the regions traversed by the
algorithm. For polyhedral composite problems, we prove that convergence is controlled by restricted Hoffman constants corresponding to the active polyhedral faces visited along the trajectory. Once the iterates enter a well-conditioned face, the effective condition number improves accordingly. Our work provides a
geometric quantification for fast local convergence after active-set or
manifold identification and more broadly suggests that linear convergence is fundamentally governed by the geometry of the subsets explored by the
algorithm, rather than by worst-case global conditioning.
Read the full abstract ▾
Machine Learning
Reference-state System Reliability method for scalable uncertainty quantification of coherent systems
Ji-Eun Byun, Hyeuk Ryu and Junho Song
Coherent systems are representative of many practical applications, ranging from infrastructure networks to supply chains. Probabilistic evaluation of such systems remains challenging, however, because existing decomposition-based methods scale poorly as the number of components grows. To address this limitation, this study proposes the Reference-state System Reliability (RSR) method. Like existing approaches, RSR characterises the boundary between different system states using reference states in the component-state space. Where it departs from these methods is in how the state space is explored: rather than using reference states to decompose the space into disjoint hypercubes, RSR uses them to classify Monte Carlo samples, making computational cost significantly less sensitive to the number of reference states. To make this classification efficient, samples and reference states are stored as
matrices and compared using batched
matrix operations, allowing RSR to exploit the advances in high-throughput
matrix computing driven by modern machine learning. We demonstrate that RSR evaluates the system-state probability of a graph with 119 nodes and 295 edges within 10~seconds, highlighting its potential for real-time risk assessment of large-scale systems. We further show that RSR scales to problems involving hundreds of thousands of reference states – well beyond the reach of existing methods – and extends naturally to multi-state systems. Nevertheless, when the number of boundary reference states grows exceedingly large, RSR's convergence slows down, a limitation shared with existing reference-state-based approaches that motivates future research into learning-based representations of system-state boundaries.
Read the full abstract ▾
Machine Learning · Cross-listed from eess.SP
E2E-WAVE: End-to-End Learned Waveform Generation for Underwater Video Multicasting
Khizar Anjum, Tingcong Jiang and Dario Pompili
We present E2E-WAVE, the first end-to-end learned waveform generation system for
underwater video multicasting. Acoustic channels exhibit 20–46% bit error rates where forward error correction becomes counterproductive – LDPC increases rather than decreases errors beyond its decoding threshold. E2E-WAVE addresses this by
embedding semantic similarity directly into physical layer waveforms: when decoding errors are unavoidable, the system preferentially selects semantically similar
tokens rather than arbitrary corruption. Combining VideoGPT tokenization (1024x compression) with a trainable waveform bank and fully differentiable OFDM transmission, E2E-WAVE achieves +5 dB (19.26%) PSNR and +0.10 (14.28%) SSIM over the strongest FEC-protected baseline in less challenging
underwater channel (NOF1) while delivering real-time 16 FPS
video at 128x128 resolution over 2.3 kbps channels – impossible for conventional digital modulation. The performance gap only increases in harsher channels (BCH1, NCS1). Trained on a single channel, E2E-WAVE generalizes to unseen
underwater environments without retraining, while HEVC fails at sub-5 kbps rates and SoftCast's AWGN assumptions collapse on frequency-selective channels.
Read the full abstract ▾
Machine Learning
When Spike Sparsity Does Not Translate to Deployed Cost: VS-WNO on Jetson Orin Nano
Jason Yoo, Shailesh Garg, Souvik Chakraborty et al. (+1 more)
Spiking neural operators are appealing for neuromorphic edge computing because event-driven substrates can, in principle, translate sparse activity into lower latency and energy. Whether that advantage survives deployment on commodity edge-GPU software stacks, however, remains unclear. We study this question on a Jetson Orin Nano 8 GB using five pretrained variable-spiking wavelet neural operator (VS-WNO) checkpoints and five matched dense wavelet neural operator (WNO) checkpoints on the Darcy rectangular
benchmark. On a reference-aligned path, VS-WNO exhibits substantial algorithmic sparsity, with mean spike rates decreasing from 54.26% at the first spiking layer to 18.15% at the fourth. On a deployment-style request path, however, this sparsity does not reduce deployed cost: VS-WNO reaches 59.6 ms latency and 228.0 mJ dynamic energy per inference, whereas dense WNO reaches 53.2 ms and 180.7 mJ, while also achieving slightly lower reference-path error (1.77% versus 1.81%). Nsight Systems indicates that the request path remains launch-dominated and dense rather than sparsity-aware: for VS-WNO, cudaLaunchKernel accounts for 81.6% of CUDA API time within the latency window, and dense convolution kernels account for 53.8% of GPU kernel time; dense WNO shows the same pattern. On this Jetson-class GPU stack, spike sparsity is measurable but does not reduce deployed cost because the runtime does not suppress dense work as spike activity decreases.
Read the full abstract ▾
All authors Jason Yoo, Shailesh Garg, Souvik Chakraborty, Syed Bahauddin Alam
Machine Learning · Cross-listed from cs.AI
Harness as an Asset: Enforcing Determinism via the Convergent AI Agent Framework (CAAF)
Tianbao Zhang
Large
Language Models produce a controllability gap in safety-critical engineering: even low rates of undetected constraint violations render a system undeployable. Current orchestration paradigms suffer from sycophantic compliance, context
attention decay, and
stochastic oscillation during self-correction. We introduce the Convergent AI
Agent Framework (CAAF), which transitions agentic workflows from open-loop generation to closed-loop fail-safe determinism via three pillars: (1) Recursive Atomic Decomposition with physical context firewalls; (2) Harness as an Asset, formalizing domain invariants into machine-readable registries enforced by a deterministic Unified Assertion Interface; and (3) Structured Semantic Gradients with State Locking for monotonic non-regression. This paper makes two core claims. First, an industrialization thesis: once domain invariants are formalized as an executable Harness, the Harness itself becomes a first-class enterprise asset that compounds in value as foundation models commoditize, and CAAF's ability to deliver its reliability on commodity-tier models makes fully self-hosted, on-premises deployment architecturally feasible for regulated sectors where cloud APIs are not an option. Second, an architectural claim supported by ablation: CAAF's three pillars address complementary failure surfaces and none alone closes the controllability gap at commodity cost. The paper contributes entirely at the orchestration and industrialization layer. Evidence across two complementary
benchmarks, three-tier UAI ablations, multi-
agent baselines, and a closed-source commodity family replicated by two independent open-weight families, is reported in the body.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Improving LLM Code Reasoning via Semantic Equivalence Self-Play with Formal Verification
Antonio Valerio Miceli Barone and Poon Tsz Nok
We introduce a self-play framework for semantic equivalence in Haskell, utilizing formal verification to guide
adversarial training between a generator and an evaluator. The framework leverages
Liquid Haskell proofs for validating equivalence and execution-based counterexamples for inequivalence, organized via a difficulty-aware curriculum. To facilitate this, we release OpInstruct-HSx, a synthetic
dataset of $\approx$28k validated Haskell programs. Empirical experiments show that our evaluator transfers effectively to downstream tasks, achieving up to 13.3pp accuracy gain on EquiBench and consistent gains on PySecDB. Ablation studies on the SEQ-SINQ regimes indicate that while inequivalence supervision provides data volume, equivalence proofs are uniquely responsible for the model's
reasoning capabilities. The entire training pipeline and
dataset are publicly released on GitHub and Hugging Face respectively.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
SPS: Steering Probability Squeezing for Better Exploration in Reinforcement Learning for Large Language Models
Yifu Huo, Chenglong Wang, Ziming Zhu et al. (+9 more)
Reinforcement learning (RL) has emerged as a promising paradigm for training
reasoning-oriented models by leveraging rule-based reward signals. However, RL training typically tends to improve single-sample success rates (i.e., Pass@1) while offering limited exploration of diverse
reasoning trajectories, which is crucial for multi-sample performance (i.e., Pass@k). Our preliminary analysis reveals that this limitation stems from a fundamental squeezing effect, whereby probability mass is excessively concentrated on a narrow subset of high-reward trajectories, restricting genuine exploration and constraining attainable performance under RL training. To address this issue, in this work, we propose Steering Probability Squeezing (SPS), a training paradigm that interleaves conventional RL with inverse reinforcement learning (IRL). SPS treats on-policy rollouts as demonstrations and employs IRL to explicitly reshape the induced trajectory distribution, thereby enhancing exploration without introducing external supervision. Experiments on five commonly used
reasoning benchmarks demonstrate that SPS can enable better exploration and improve Pass@k. Beyond algorithmic contributions, we provide an analysis of RL learning dynamics and identify an empirical upper bound on Pass@k, shedding light on intrinsic exploration limits in RL-based
reasoning models. Our findings suggest that alternating between RL and IRL offers an effective pathway toward extending the exploration capacity of
reasoning-oriented large
language models.
Read the full abstract ▾
All authors Yifu Huo, Chenglong Wang, Ziming Zhu, Shunjie Xing, Peinan Feng, Tongran Liu, Qiaozhi He, Tianhua Zhou, Xiaojia Chang, Jingbo Zhu, Zhengtao Yu, Tong Xiao
Machine Learning · Cross-listed from cs.CL
Bolzano: Case Studies in LLM-Assisted Mathematical Research
Martin Balko, Jan Grebík, Pavel Hubáček et al. (+5 more)
We report new results on eight problems in mathematics and theoretical computer science, produced with the assistance of Bolzano, an open-source multi-
agent LLM system. Bolzano orchestrates rounds of interaction between parallel prover
agents and a verifier
agent while maintaining a persistent knowledge base that is carried across rounds. Classified using the significance-autonomy taxonomy of Feng et al., six of the eight results reach the level of publishable research, and five of the eight were produced essentially autonomously by Bolzano. Our results provide evidence that LLMs can contribute meaningfully to mathematical research, complementing recent reports by Bubeck et al., Woodruff et al., and others.
Read the full abstract ▾
All authors Martin Balko, Jan Grebík, Pavel Hubáček, Martin Koutecký, Matěj Kripner, Václav Rozhoň, Robert Šámal, Adrián Zámečník
Machine Learning
In-Context Learning Under Regime Change
Carson Dudley, Yutong Bi, Xiaofeng Liu et al. (+1 more)
Non-stationary sequences arise naturally in control,
forecasting, and decision-making. The data-generating process shifts at unknown times, and models must detect the change, discard or downweight obsolete evidence, and adapt to new dynamics on the fly.
Transformer-based foundation models increasingly rely on in-context learning for time series
forecasting, tabular prediction, and continuous control. As these models are deployed in non-stationary environments, understanding their ability to detect and adapt to regime shifts is important. We formalize this as an in-context change-point detection problem and formally establish the existence of
transformer models that solve this problem. Our construction demonstrates that model complexity, in layers and parameters, depends on the level of information available about the change-point location, from no knowledge to knowing exact timing. We validate our results with experiments on synthetic linear regression and linear dynamical systems, where trained
transformers match the performance of optimal baselines across information levels. We also show that encoding and incorporating changepoint knowledge indeed improves the real-world performance of a pretrained foundation models on infectious disease
forecasting and on financial
volatility forecasting around Federal Open
Market Committee (FOMC) announcements without retraining, demonstrating practical applicability to real-world regime changes.
Read the full abstract ▾
All authors Carson Dudley, Yutong Bi, Xiaofeng Liu, Samet Oymak
Machine Learning · Cross-listed from physics.med-ph
Light-Adapted Electroretinogram and Oscillatory Potentials (LEOPs) Dataset for Autism Spectrum Disorder and Typically Developing Individuals
Paul A. Constable, Dorothy A. Thompson, Irene O. Lee et al. (+4 more)
The LEOPs (Light-ERG-Oscillatory Potentials)
dataset provides light-adapted (LA) electroretinogram (ERG) and Oscillatory Potentials (OPs) waveforms for typically developing Control, Autism Spectrum Disorder (ASD) and ASD +
Attention Deficit Hyperactivity Disorder (ADHD) childhood and adolescent populations. The ERGs were recorded in the Right And Left eyes with skin electrodes using the handheld RETeval device at two sites in Australia and the United Kingdom. The LEOPs
dataset includes 5309 single flash ERG and 4434 OPs waveforms as well as images selected from each participant showing the position of the skin electrode. The LEOPs
dataset is constructed from recordings using a 9 step randomized flash series from $-0.37$ to $1.20$~$Td.s$, a 2 step at 113 and 446 $Td.s$ flash strengths (2500 Control, 1730 ASD and 451 ASD + ADHD samples), as well as the $85$~$Td.s$ (Light Adapted 3 $cd.s.m^{-2}$ (LA3)) equivalent International Society of Clinical Electrophysiology of
Vision (ISCEV) Standard flash with 435 Control, 176 ASD and 37 ASD + ADHD waveform samples. Code for the stimulus is provided along with participant demographics, date and time of testing, and where available diagnostic scores for the ASD and ASD + ADHD groups, alongside iris color, electrode position with image files and time domain values for the ERG and summed values for the OPs. The repository contains excel file, exported JSON files on the patient level that are more suitable for machine learning tasks, images of electrode position for each recording and the protocol files for use with the RETeval.
Read the full abstract ▾
All authors Paul A. Constable, Dorothy A. Thompson, Irene O. Lee, Lynne Loh, Aleksei Zhdanov, Mikhail Kulyabin, Andreas Maier
Machine Learning
Evaluating Multimodal LLMs for Inpatient Diagnosis: Real-World Performance, Safety, and Cost Across Ten Frontier Models
Bruce A. Bassett, Amy Rouillard, Sitwala Mundia et al. (+8 more)
Background: Large
language models (LLMs) are increasingly proposed for diagnostic support, but few evaluations use real-world
multimodal inpatient data, particularly in low and middle-income country (LMIC) public hospitals. Methods: We conducted VALID, a retrospective evaluation of 539
multimodal inpatient cases from a tertiary public hospital in South Africa. Inputs included radiology imaging (CT, MRI, CXR) and reports, laboratory results, clinical notes, and vital signs. Expert panels adjudicated 300 cases (balanced and discordant subsets) to establish ground truth diagnoses, differentials, and
reasoning. Ten
multimodal LLMs generated zero-shot outputs. A calibrated three-model LLM Jury scored all outputs and routine ward diagnoses across diagnostic accuracy, differential quality,
reasoning, and patient safety (>10,000 evaluations). Primary outcomes were composite scores ($S_3$, $S_4$) and win rates. Results: (i) LLM performance was tightly clustered (<15% variation) despite large cost differences; low-cost models performed comparably to top models. (ii) All LLMs significantly outperformed routine ward diagnoses on average diagnostic and safety scores. (iii) Top performance was achieved by GPT-5.1, followed by Gemini models. (vi) Adding radiology reports improved performance by 6%. (v) Diagnostic and
reasoning scores were highly correlated ($\rho = 0.85$). (vi) Output rates varied (65-100%) due to input constraints. Results were robust across subsets and evaluation design. Conclusions: Across a real-world LMIC
dataset,
multimodal LLMs showed similar diagnostic performance despite large cost differences and outperformed routine care on average safety metrics. Affordability, robustness, and deployment constraints may outweigh marginal performance differences in LMIC settings.
Read the full abstract ▾
All authors Bruce A. Bassett, Amy Rouillard, Sitwala Mundia, Michael Cameron Gramanie, Linda Camara, Ziyaad Dangor, Shabir A. Madhi, Kajal Morar, Marlvin T. Ncube, Ismail Kalla, Haroon Saloojee
Machine Learning · Cross-listed from math.NA
Convergence theory for Hermite approximations under adaptive coordinate transformations
Yahya Saleh
Recent work has shown that parameterizing and optimizing coordinate transformations using normalizing flows, i.e., invertible neural networks, can significantly accelerate the convergence of spectral approximations. We present the first error estimates for approximating functions using Hermite expansions composed with adaptive coordinate transformations. Our analysis establishes an equivalence principle: approximating a function $f$ in the span of the transformed basis is equivalent to approximating the pullback of $f$ in the span of Hermite functions. This allows us to leverage the classical approximation theory of Hermite expansions to derive error estimates in transformed coordinates in terms of the regularity of the pullback. We present an example demonstrating how a nonlinear coordinate transformation can enhance the convergence of Hermite expansions. Focusing on smooth functions decaying along the real axis, we construct a monotone transport map that aligns the decay of the target function with the Hermite basis. This guarantees spectral convergence rates for the corresponding Hermite expansion. Our analysis provides theoretical insight into the convergence behavior of adaptive Hermite approximations based on normalizing flows, as recently explored in the computational
quantum physics literature.
Read the full abstract ▾
Machine Learning
Hyperbolic Enhanced Representation Learning for Incomplete Multi-view Clustering
Tianyi Chen, Haobo Wang, Kai Tang et al. (+5 more)
Incomplete Multi-View Clustering (IMVC) faces the challenge of learning discriminative representations from fragmentary observations while maintaining robustness against missing views. However, prevalent Euclidean-based methods suffer from a
geometric mismatch when modeling real-world data with intrinsic hierarchies, leading to semantic blurring where representations drift towards spatially proximal but semantically distinct neighbors. To bridge this gap, we propose HERL, a
Hyperbolic Enhanced Representation Learning framework for IMVC. Operating within the Poincaré ball, HERL constructs a structure-aware latent space to enhance representation learning. Specifically, we design a dual-constraint
hyperbolic contrastive mechanism optimizing: an angular-based loss to preserve semantic identity via directional
alignment, and a distance-based loss to enforce hierarchical compactness. Furthermore, a
hyperbolic prototype head is introduced to rectify global structural drift by aligning cross-view hierarchy-aware prototype distributions. Consequently, HERL disentangles fine-grained semantic correlations to sharpen cluster boundaries and imposes
geometric constraints to rectify the data recovery process. Extensive experimental results demonstrate that HERL consistently outperforms state-of-the-art approaches.
Read the full abstract ▾
All authors Tianyi Chen, Haobo Wang, Kai Tang, Gengyu Lyu, Tianlei Hu, Gang Chen, Hong Ma, Meixiang Xiang
Machine Learning
Open-TQ-Metal: Fused Compressed-Domain Attention for Long-Context LLM Inference on Apple Silicon
Sai Vegasena
We present Open-TQ-Metal, the first implementation of fused compressed-domain
attention on Apple Silicon, enabling 128K-context inference for Llama 3.1 70B on a single 64GB consumer Mac – a configuration impossible with all existing inference frameworks. Open-TQ-Metal quantizes the KV cache to int4 on the fly and computes
attention directly on the compressed representation via custom Metal compute shaders, eliminating all intermediate dequantization
matrices. Across 330 experiments spanning two model families (Gemma 4 31B and Llama 3.1 70B), the fused sdpa_int4 kernel achieves 48x
attention speedup at 128K context over the dequantize-then-attend baseline, reduces KV cache
memory from 40 GB to 12.5 GB (3.2x compression), and maintains identical top-1
token predictions to FP16 inference. We further provide the first cross-architecture analysis of KV cache quantization methods, revealing that the
attention scale factor – not model size – determines whether angular quantization schemes like PolarQuant succeed or fail, with Gemma 4's attn_scale=1.0 amplifying directional error 25-100x more than Llama's standard 1/sqrt(d) scaling.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Diagnostic-Guided Longitudinal Modeling for Forecasting Retinal Atrophy Progression
Liyin Chen, Souvick Mukherjee, Ines Maria De Carvalho Lains et al. (+4 more)
Stochastic generative models are increasingly used for longitudinal imaging, but their added complexity may provide limited benefit when predictable disease-related change is small relative to technical variability. We treat model-class selection (
stochastic vs deterministic) as an empirical step determined by a task-adaptive diagnostic. For a longitudinal image prediction task complicated by irregular follow-up, acquisition variability, and device heterogeneity, the diagnostic asks whether inter-visit image change is driven by time-dependent disease progression signals or time-independent acquisition variability. If a
stochastic approach fails to yield useful predictive diversity, a more parsimonious, deterministic model class is selected. We applied this strategy to fundus autofluorescence (FAF) future image prediction. Applied to a heterogeneous Optos FAF archive (24,335 images from 9,708 eyes), the diagnostic indicated that global inter-visit FAF change was weakly associated with elapsed time, and source-anchored
stochastic configurations produced little endpoint-resolvable sample variation. Guided by the diagnostic, we developed Temporal Retinal U-Net (TRU), a deterministic single-pass predictor conditioned on irregular imaging history and prediction horizon. Evaluated on a held-out cohort and two independent zero-shot transfer cohorts spanning rare-disease and cross-vendor shift, TRU achieved the strongest overall performance among the evaluated classical and deep-learning comparators on image-level and eye-specific progression measures, with lower precision in the smaller cross-vendor cohort. These findings support a diagnostic-guided strategy for longitudinal image prediction and demonstrate an example where deterministic predictions outperform
stochastic models.
Read the full abstract ▾
All authors Liyin Chen, Souvick Mukherjee, Ines Maria De Carvalho Lains, Nazlee Zebardast, Mengyu Wang, Tobias Elze, Jason I. Comander
Machine Learning · Cross-listed from quant-ph
Hybrid Quantum Neural Networks for Enhanced Breast Cancer Thermographic Classification: A Novel Quantum-Classical Integration Approach
Riza Alaudin Syah, Irwan Alnarus Kautsar, Gunawan Witjaksono et al. (+1 more)
Breast cancer diagnosis through thermographic image analysis remains a critical challenge in medical AI, with classical deep learning approaches facing limitations in complex thermal pattern classification tasks. This paper presents a novel Hybrid
Quantum Neural Network (HQNN) architecture that integrates
quantum computing principles with classical convolutional neural networks for enhanced breast cancer classification. Our approach employs parameterized
quantum circuits with multi-head
attention mechanisms for
quantum-aware feature encoding, coupled with classical convolutional layers for comprehensive pattern recognition. The
quantum component utilizes a 4
qubit variational
circuit with strongly entangling layers, while the classical component incorporates advanced
attention mechanisms for feature fusion. Experimental validation on breast cancer thermographic data demonstrates substantial performance improvements over state-of-the-art classical architectures, with the
quantum-enhanced approach exhibiting superior convergence dynamics and enhanced feature representation capabilities. Our findings provide evidence for
quantum advantage in medical image classification through classical simulation, establishing a framework for
quantum-classical hybrid systems in healthcare applications. The methodology addresses key challenges in
quantum machine learning deployment while maintaining computational feasibility on near-term
quantum devices.
Read the full abstract ▾
All authors Riza Alaudin Syah, Irwan Alnarus Kautsar, Gunawan Witjaksono, Haza Nuzly bin Abdull Hamed
Machine Learning
L1 Regularization Paths in Linear Models by Parametric Gaussian Message Passing
Yun-Peng Li and Hans-Andrea Loeliger
The paper proposes new
algorithms for computing the regularization path of linear-model fitting (i) with L1 costs for the independent variables and L2 costs for the dependent variables, and (ii) with L2 costs for the independent variables and L1 costs for the dependent variables. Unlike previously proposed
algorithms, the proposed
algorithms for (i) and (ii) are exact duals of each other. The heart of the proposed
algorithms is parametric Gaussian message passing (i.e., Kalman-type forward-backward recursions) in the pertinent factor graphs. We first develop such
algorithms for linear models with general (but preferably low-dimensional) state space representations. We then specialize to specific state space representations of generic linear models; in this case, the structure of the factor graph allows to refine the proposed
algorithms such that the computational complexity drops to linear scale.
Read the full abstract ▾
Machine Learning
D-QRELO: Training- and Data-Free Delta Compression for Large Language Models via Quantization and Residual Low-Rank Approximation
Junlin Li, Shuangyong Song, Guodong Du et al. (+6 more)
Supervised Fine-Tuning (SFT) accelerates taskspecific large
language models (LLMs) development, but the resulting proliferation of finetuned models incurs substantial
memory overhead. Delta compression addresses this by retaining a single pre-trained LLM with multiple compressed delta weights. However, existing methods fail on models fine-tuned with largescale
datasets. We find that larger SFT data scale amplifies delta parameter magnitude, singular values, and
entropy, exacerbating compression errors. To tackle this, we propose DQRELO (Delta Compression via Quantization and Residual Low-Rank), a novel training- and data-free delta compression method. It combines coarse-grained one-bit quantization to capture the dominant structure of the delta, followed by compensated residual low-rank approximation to recover fine-grained details from the smaller residual error. Experiments on various LLMs spanning dense and MoE architectures across multiple domains under this challenging setting demonstrate that DQRELO outperforms existing methods. Moreover, we establish key design principles for delta compression through extensive empirical analysis, demonstrating how task difficulty, architecture, and layer positioning create predictable patterns that can guide optimal compression strategies in production systems.
Read the full abstract ▾
All authors Junlin Li, Shuangyong Song, Guodong Du, Ngai Wong, Xuebo Liu, Yongxiang Li, Min Zhang, Jing Li, Xuelong Li
Machine Learning · Cross-listed from cs.AI
LLMs can persuade only psychologically susceptible humans on societal issues, via trust in AI and emotional appeals, amid logical fallacies
Alexis Carrillo, Salvatore Citraro, Ali Aghazhadeh Ardebili et al. (+5 more)
Scarce longitudinal evidence examines LLMs' persuasiveness and humanness along time-evolving psychological frameworks. We introduce Talk2AI, a longitudinal framework quantifying psycho-social,
reasoning and affective dimensions of LLMs' persuasiveness about polarizing societal topics. In a four-way longitudinal setup, Talk2AI's 770 participants engaged in structured conversations with one of four leading LLMs on topics like
climate change, social media misinformation, and math anxiety. This produced 3,080 conversations over 60,000 turns. After each wave, participants reported conviction in their initial topic stance, perceived opinion change, LLM's perceived humanness, a self-donation to the topic and a textual explanation. Feedback time series showed longitudinal inertia in convictions, indicating some human anchoring to initial opinions even after repeated exposure to AI-generated arguments. Interestingly, NLP analyses revealed that both humans and LLMs relied on fallacious
reasoning in 1 conversational quip every 6, countering the “LLMs as superior systems" stereotype behind LLMs' cognitive surrender. LLMs' perceived humanness was most learnable from sociodemographic, psychological and engagement features ($R^2=0.44$), followed by opinion change ($R^2=0.34$), conviction ($R^2=0.26$) and personal endowment ($R^2=0.24$). Crucially, explainable AI (XAI) indicated: (i) the presence of individuals more susceptible to LLM-based opinion changes; (ii) psychological susceptibility to LLM-convincing consisted of having more trust in LLMs, being more agreeable and extraverted and with a higher need for cognition. A multiverse approach with mixed-effects models confirmed XAI results, alongside strong individual differences. Talk2AI provides a grounded framework and evidence for detecting how GenAI can influence human opinions via multiple psycho-social pathways in AI-human digital platforms.
Read the full abstract ▾
All authors Alexis Carrillo, Salvatore Citraro, Ali Aghazhadeh Ardebili, Enrique Taietta, Giulio Rossetti, Emilio Ferrara, Giuseppe Alessandro Veltri, Massimo Stella
Machine Learning · Cross-listed from stat.ML
Neighbor Embedding for High-Dimensional Sparse Poisson Data
Noga Mudrik and Adam S. Charles
Across many scientific fields, measurements often represent the number of times an event occurs. For example, a document can be represented by word occurrence counts, neural activity by spike counts per time window, or online communication by daily email counts. These measurements yield high-dimensional count data that often approximate a Poisson distribution, frequently with low rates that produce substantial sparsity and complicate downstream analysis. A useful approach is to embed the data into a low-dimensional space that preserves meaningful structure, commonly termed dimensionality reduction. Yet existing dimensionality reduction methods, including both linear (e.g., PCA) and nonlinear approaches (e.g., t-SNE), often assume continuous Euclidean geometry, thereby misaligning with the discrete, sparse nature of low-rate count data. Here, we propose p-SNE (Poisson
Stochastic Neighbor
Embedding), a nonlinear neighbor
embedding method designed around the Poisson structure of count data, using KL divergence between Poisson distributions to measure pairwise dissimilarity and Hellinger distance to optimize the
embedding. We test p-SNE on synthetic Poisson data and demonstrate its ability to recover meaningful structure in real-world count
datasets, including weekday patterns in email communication, research area clusters in OpenReview papers, and temporal drift and stimulus gradients in neural spike recordings.
Read the full abstract ▾
Machine Learning
Test-Time Adaptation for EEG Foundation Models: A Systematic Study under Real-World Distribution Shifts
Gabriel Jason Lee, Jathurshan Pradeepkumar and Jimeng Sun
Electroencephalography (
EEG) foundation models have shown strong potential for learning generalizable representations from large-scale neural data, yet their clinical deployment is hindered by distribution shifts across clinical settings, devices, and populations. Test-time adaptation (TTA) offers a promising solution by enabling models to adapt to unlabeled target data during inference without access to source data, a valuable property in healthcare settings constrained by
privacy regulations and limited labeled data. However, its effectiveness for
EEG remains largely underexplored. In this work, we introduce NeuroAdapt-Bench, a systematic
benchmark for evaluating test-time adaptation methods on
EEG foundation models under realistic distribution shifts. We evaluate representative TTA approaches from other domains across multiple pretrained foundation models, diverse downstream tasks, and heterogeneous
datasets spanning in-distribution, out-of-distribution, and extreme modality shifts (e.g., Ear-
EEG). Our results show that the evaluated TTA methods yield inconsistent gains and often degrade performance, with gradient-based approaches particularly prone to heavy degradation and optimization-free methods showing greater stability. For the evaluated
EEG foundation models and representative TTA methods, these findings highlight the limitations of directly applying existing TTA techniques to
EEG and underscore the need for domain-specific adaptation strategies.
Read the full abstract ▾
Machine Learning
Noise-Adaptive Diffusion Sampling for Inverse Problems Without Task-Specific Tuning
Yingzhi Xia, Setthakorn Tanomkiattikun, Liangli Zhen et al. (+1 more)
Diffusion models (DMs) have recently shown remarkable performance on inverse problems (IPs). Optimization-based methods can fast solve IPs using DMs as powerful regularizers, but they are susceptible to local minima and noise overfitting. Although DMs can provide strong priors for
Bayesian approaches, enforcing measurement consistency during the denoising process leads to
manifold infeasibility issues. We propose Noise-space
Hamiltonian Monte Carlo (N-HMC), a posterior sampling method that treats reverse
diffusion as a deterministic mapping from initial noise to clean images. N-HMC enables comprehensive exploration of the solution space, avoiding local optima. By moving inference entirely into the initial-noise space, N-HMC keeps proposals on the learned data
manifold. We provide a comprehensive theoretical analysis of our approach and extend the framework to a noise-adaptive variant (NA-NHMC) that effectively handles IPs with unknown noise type and level. Extensive experiments across four linear and three nonlinear inverse problems demonstrate that NA-NHMC achieves superior reconstruction quality with robust performance across different hyperparameters and initializations, significantly outperforming recent state-of-the-art methods. The code is available at https://github.com/NA-HMC/NA-HMC.
Read the full abstract ▾
All authors Yingzhi Xia, Setthakorn Tanomkiattikun, Liangli Zhen, Zaiwang Gu
Machine Learning · Cross-listed from cs.CL
Freshness-Aware Prioritized Experience Replay for LLM/VLM Reinforcement Learning
Weiyu Ma, Yongcheng Zeng, Yan Song et al. (+4 more)
Reinforcement Learning (RL) has achieved impressive success in post-training Large
Language Models (LLMs) and
Vision-
Language Models (VLMs), with on-policy
algorithms such as PPO, GRPO, and REINFORCE++ serving as the dominant paradigm. However, these methods discard all collected trajectories after a single gradient update, resulting in poor sample efficiency, particularly wasteful for agentic tasks where multi-turn environment interactions are expensive. While Experience Replay drives sample efficiency in classic RL by allowing
agents to reuse past trajectories and prioritize informative ones, directly applying Prioritized Experience Replay (PER) to LLMs fails. The rapid policy
evolution of billion-parameter models renders stored priorities stale, causing old high-priority trajectories to dominate sampling long after they have become uninformative. We propose Freshness-Aware PER, which addresses this priority staleness problem by augmenting any PER-based priority with a multiplicative exponential age decay grounded in effective sample size analysis. To the best of our knowledge, Freshness-Aware PER is the first work to successfully apply PER to LLM/VLM reinforcement learning. We evaluate on eight multi-step agentic,
reasoning, and math competition tasks with 0.5B, 3B, and 7B models. Freshness-Aware PER significantly outperforms on-policy baselines, achieving +46% on NQ Search, +367% on Sokoban, and +133% on VLM FrozenLake, while standard PER without age decay consistently degrades performance. Our code is publicly available at https://github.com/
Vision-CAIR/Freshness-Aware-PER.
Read the full abstract ▾
All authors Weiyu Ma, Yongcheng Zeng, Yan Song, Xinyu Cui, Jian Zhao, Xuhui Liu, Mohamed Elhoseiny
Machine Learning
Covariance-Based Structural Equation Modeling in Small-Sample Settings with $p>n$
Hiroki Hasegawa, Aoba Tamura and Yukihiko Okada
Factor-based Structural Equation Modeling (SEM) relies on likelihood-based estimation assuming a nonsingular sample covariance
matrix, which breaks down in small-sample settings with $p>n$. To address this, we propose a novel estimation principle that reformulates the covariance structure into self-covariance and cross-covariance components. The resulting framework defines a likelihood-based feasible set combined with a relative error constraint, enabling stable estimation in small-sample settings where $p>n$ for sign and direction. Experiments on synthetic and real-world data show improved stability, particularly in recovering the sign and direction of structural parameters. These results extend covariance-based SEM to small-sample settings and provide practically useful directional information for decision-making.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
EasyVideoR1: Easier RL for Video Understanding
Chuanyu Qin, Chenxu Yang, Qingyi Si et al. (+6 more)
Reinforcement learning from verifiable rewards (RLVR) has demonstrated remarkable effectiveness in improving the
reasoning capabilities of large
language models. As models evolve into natively
multimodal architectures, extending RLVR to
video understanding becomes increasingly important yet remains largely unexplored, due to the diversity of
video task types, the computational overhead of repeatedly decoding and preprocessing high-dimensional visual inputs, and the difficulty of reproducible evaluation across numerous sensitive hyperparameters. Existing open-source RL training frameworks provide solid infrastructure for text and image scenarios but lack systematic optimizations tailored for
video modality. In this work, we present EasyVideoR1, a complete and efficient reinforcement learning framework specifically designed for training large
vision-
language models on
video understanding tasks. EasyVideoR1 makes the following contributions: (1) a full
video RL training pipeline with offline preprocessing and tensor caching that eliminates redundant
video decoding and yields a 1.47 $\times$ throughput improvement; (2) a comprehensive, task-aware reward system covering 11 distinct
video and image problem types with unified routing and modular extension; (3) a mixed offline-online data training paradigm that combines curated high-quality trajectories with on-policy exploration, benefiting the learning of more challenging tasks; (4) joint image-
video training with independently configurable pixel budgets, allowing the two modalities to mutually reinforce each other; and (5) an asynchronous multi-
benchmark evaluation framework covering 22 mainstream
video understanding
benchmarks, with reproduced accuracy closely aligned with officially reported scores.
Read the full abstract ▾
All authors Chuanyu Qin, Chenxu Yang, Qingyi Si, Naibin Gu, Dingyu Yao, Zheng Lin, Peng Fu, Nan Duan, Jiaqi Wang