Machine Learning
Quantum King-Ring Domination in Chess: A QAOA Approach
Gerhard Stenzel, Michael Kölle, Tobias Rohe et al. (+4 more)
The
Quantum Approximate Optimization
Algorithm (QAOA) is extensively benchmarked on synthetic random instances such as MaxCut, TSP, and SAT problems, but these lack semantic structure and human interpretability, offering limited insight into performance on real-world problems with meaningful constraints. We introduce
Quantum King-Ring Domination (QKRD), a NISQ-scale
benchmark derived from chess tactical positions that provides 5,000 structured instances with one-hot constraints, spatial locality, and 10–40
qubit scale. The
benchmark pairs human-interpretable coverage metrics with intrinsic validation against classical heuristics, enabling algorithmic conclusions without external oracles. Using QKRD, we systematically evaluate QAOA design choices and find that constraint-preserving mixers (XY, domain-wall) converge approximately 13 steps faster than standard mixers (p<10^{-7}, d\approx0.5) while eliminating penalty tuning, warm-start strategies reduce convergence by 45 steps (p<10^{-127}, d=3.35) with energy improvements exceeding d=8, and Conditional Value-at-Risk (CVaR) optimization yields an informative negative result with worse energy (p<10^{-40}, d=1.21) and no coverage benefit. Intrinsic validation shows QAOA outperforms greedy heuristics by 12.6% and random selection by 80.1%. Our results demonstrate that structured
benchmarks reveal advantages of problem-informed QAOA techniques obscured in random instances. We release all code, data, and experimental artifacts for reproducible NISQ
algorithm research.
Read the full abstract ▾
All authors Gerhard Stenzel, Michael Kölle, Tobias Rohe, Julian Hager, Leo Sünkel, Maximilian Zorn, Claudia Linnhoff-Popien
Machine Learning
Can Optimal Transport Improve Federated Inverse Reinforcement Learning?
David Millard and Ali Baheri
In robotics and multi-
agent systems, fleets of autonomous
agents often operate in subtly different environments while pursuing a common high-level objective. Directly pooling their data to learn a shared reward function is typically impractical due to differences in dynamics,
privacy constraints, and limited communication bandwidth. This paper introduces an optimal transport-based approach to
federated inverse reinforcement learning (IRL). Each client first performs lightweight Maximum
Entropy IRL locally, adhering to its computational and
privacy limitations. The resulting reward functions are then fused via a Wasserstein barycenter, which considers their underlying
geometric structure. We further prove that this barycentric fusion yields a more faithful global reward estimate than conventional parameter averaging methods in
federated learning. Overall, this work provides a principled and communication-efficient framework for deriving a shared reward that generalizes across heterogeneous
agents and environments.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Can Large Language Models Still Explain Themselves? Investigating the Impact of Quantization on Self-Explanations
Qianli Wang, Nils Feldhus, Pepa Atanasova et al. (+4 more)
Quantization is widely used to accelerate inference and streamline the deployment of large
language models (LLMs), yet its effects on self-explanations (SEs) remain unexplored. SEs, generated by LLMs to justify their own outputs, require
reasoning about the model's own decision-making process, a capability that may exhibit particular sensitivity to quantization. As SEs are increasingly relied upon for transparency in high-stakes applications, understanding whether and to what extent quantization degrades SE quality and faithfulness is critical. To address this gap, we examine two types of SEs: natural
language explanations (NLEs) and counterfactual examples, generated by LLMs quantized using three common techniques at distinct bit widths. Our findings indicate that quantization typically leads to moderate declines in both SE quality (up to 4.4%) and faithfulness (up to 3.9%). The user study further demonstrates that quantization considerably diminishes both the coherence and trustworthiness of SEs (by up to 8.5%). Compared to smaller models, larger models show limited resilience to quantization in terms of SE quality but maintain more faithfulness. Moreover, no quantization technique consistently excels across task accuracy, SE quality, and faithfulness. Because quantization's impact varies considerably by context and can be sizable in specific cases, we recommend validating SE quality for the intended use case. Despite these sometimes considerable drops, quantization remains an effective compression technique when its impact on SEs is properly validated.
Read the full abstract ▾
All authors Qianli Wang, Nils Feldhus, Pepa Atanasova, Fedor Splitt, Simon Ostermann, Sebastian Möller, Vera Schmitt
Machine Learning
Task-Driven Kernel Flows: Label Rank Compression and Laplacian Spectral Filtering
Hongxi Li and Chunlin Huang
We present a theory of feature learning in wide L2-regularized networks showing that supervised learning is inherently compressive. We derive a kernel ODE that predicts a "water-filling" spectral
evolution and prove that for any stable steady state, the kernel rank is bounded by the number of classes ($C$). We further demonstrate that SGD noise is similarly low-rank ($O(C)$), confining dynamics to the task-relevant subspace. This framework unifies the deterministic and
stochastic views of
alignment and contrasts the low-rank nature of supervised learning with the high-rank, expansive representations of self-supervision.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CR
Rectifying Adversarial Examples Using Their Vulnerabilities
Fumiya Morimoto, Ryuto Morita and Satoshi Ono
Deep neural network-based classifiers are prone to errors when processing
adversarial examples (AEs). AEs are minimally perturbed input data undetectable to humans posing significant risks to security-dependent applications. Hence, extensive research has been undertaken to develop defense mechanisms that mitigate their threats. Most existing methods primarily focus on discriminating AEs based on the input sample features, emphasizing AE detection without addressing the correct sample categorization before an attack. While some tasks may only require mere rejection on detected AEs, others necessitate identifying the correct original input category such as traffic sign recognition in autonomous driving. The objective of this study is to propose a method for rectifying AEs to estimate the correct labels of their original inputs. Our method is based on re-attacking AEs to move them beyond the decision boundary for accurate label prediction, effectively addressing the issue of rectifying minimally perceptible AEs created using white-box attack methods. However, challenge remains with respect to effectively rectifying AEs produced by black-box attacks at a distance from the boundary, or those misclassified into low-confidence categories by targeted attacks. By adopting a straightforward approach of only considering AEs as inputs, the proposed method can address diverse attacks while avoiding the requirement of parameter adjustments or preliminary training. Results demonstrate that the proposed method exhibits consistent performance in rectifying AEs generated via various attack methods, including targeted and black-box attacks. Moreover, it outperforms conventional rectification and input transformation methods in terms of stability against various attacks.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.NE
Modern Neuromorphic AI: From Intra-Token to Inter-Token Processing
Osvaldo Simeone
The rapid growth of artificial intelligence (AI) has brought novel data processing and generative capabilities but also escalating energy requirements. This challenge motivates renewed interest in neuromorphic computing principles, which promise
brain-like efficiency through discrete and sparse activations, recurrent dynamics, and non-linear feedback. In fact, modern AI architectures increasingly embody neuromorphic principles through heavily quantized activations, state-space dynamics, and sparse
attention mechanisms. This paper elaborates on the connections between neuromorphic models, state-space models, and
transformer architectures through the lens of the distinction between intra-
token processing and inter-
token processing. Most early work on neuromorphic AI was based on spiking neural networks (SNNs) for intra-
token processing, i.e., for transformations involving multiple channels, or features, of the same vector input, such as the pixels of an image. In contrast, more recent research has explored how neuromorphic principles can be leveraged to design efficient inter-
token processing methods, which selectively combine different information elements depending on their contextual relevance. Implementing associative memorization mechanisms, these approaches leverage state-space dynamics or sparse self-
attention. Along with a systematic presentation of modern neuromorphic AI models through the lens of intra-
token and inter-
token processing, training methodologies for neuromorphic AI models are also reviewed. These range from surrogate gradients leveraging parallel convolutional processing to local learning rules based on reinforcement learning mechanisms.
Read the full abstract ▾
Machine Learning · Cross-listed from quant-ph
Neural Minimum Weight Perfect Matching for Quantum Error Codes
Yotam Peled, David Zenati and Eliya Nachmani
Realizing the full potential of
quantum computation requires
Quantum Error Correction (QEC). QEC reduces error rates by encoding logical information across redundant physical
qubits, enabling errors to be detected and corrected. A common decoder used for this task is Minimum Weight Perfect Matching (MWPM) a graph-based
algorithm that relies on edge weights to identify the most likely error chains. In this work, we propose a data-driven decoder named Neural Minimum Weight Perfect Matching (NMWPM). Our decoder utilizes a hybrid architecture that integrates Graph Neural Networks (GNNs) to extract local syndrome features and
Transformers to capture long-range global dependencies, which are then used to predict dynamic edge weights for the MWPM decoder. To facilitate training through the non-differentiable MWPM
algorithm, we formulate a novel proxy loss function that enables end-to-end optimization. Our findings on the toric code under depolarizing noise demonstrate thresholds of 17.9% and 10.95%, nearing the 18.9% and 11.0% maximum likelihood bounds, highlighting the advantage of hybrid decoders that combine the predictive capabilities of neural networks with the algorithmic structure of classical matching.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Application Research of a Deep Learning Model Integrating CycleGAN and YOLO in PCB Infrared Defect Detection
Chao Yang, Haoyuan Zheng and Yue Ma
This paper addresses the critical bottleneck of infrared (IR) data scarcity in Printed
Circuit Board (PCB) defect detection by proposing a cross-modal data augmentation framework integrating CycleGAN and YOLOv8. Unlike conventional methods relying on paired supervision, we leverage CycleGAN to perform unpaired image-to-image translation, mapping abundant visible-light PCB images into the infrared domain. This generative process synthesizes high-fidelity pseudo-IR samples that preserve the structural semantics of defects while accurately simulating thermal distribution patterns. Subsequently, we construct a heterogeneous training strategy that fuses generated pseudo-IR data with limited real IR samples to train a lightweight YOLOv8 detector. Experimental results demonstrate that this method effectively enhances feature learning under low-data conditions. The augmented detector significantly outperforms models trained on limited real data alone and approaches the performance
benchmarks of fully supervised training, proving the efficacy of pseudo-IR synthesis as a robust augmentation strategy for industrial inspection.
Read the full abstract ▾
Machine Learning
GRIT – Geometry-Aware PEFT with K-FACPreconditioning, Fisher-Guided Reprojection, andDynamic Rank Adaptation
Pritish Saha, Chandrav Rajbangshi, Rudra Goyal et al. (+6 more)
Parameter-efficient fine-tuning (PEFT) is the default way to adapt LLMs, but widely used LoRA and QLoRA are largely geometry-agnostic: they optimize in fixed, randomly oriented low-rank subspaces with first-order descent, mostly ignoring local loss
curvature. This can inflate the effective update budget and amplify drift along weakly constrained directions. We introduce GRIT, a dynamic,
curvature-aware LoRA procedure that preserves the LoRA parameterization but: (1) preconditions gradients in rank space using K-FAC as a natural-gradient proxy; (2) periodically reprojects the low-rank basis onto dominant Fisher eigendirections to suppress drift; and (3) adapts the effective rank from the spectrum so capacity concentrates where signal resides. Across instruction-following, comprehension, and
reasoning benchmarks on LLaMA backbones, GRIT matches or surpasses LoRA and QLoRA while reducing trainable parameters by 46% on average (25–80% across tasks), without practical quality loss across
prompt styles and data mixes. To model forgetting, we fit a
curvature-modulated power law. Empirically, GRIT yields lower drift and a better updates-vs-retention frontier than strong PEFT-optimizer baselines (Orthogonal-LoRA, IA3, DoRA, Eff-FT, Shampoo).
Read the full abstract ▾
All authors Pritish Saha, Chandrav Rajbangshi, Rudra Goyal, Mohit Goyal, Anurag Deo, Biswajit Roy, Ningthoujam Dhanachandra Singh, Raxit Goswami, Amitava Das
Machine Learning
Robust Graph Fine-Tuning with Adversarial Graph Prompting
Ziyan Zhang, Bo Jiang and Jin Tang
Parameter-Efficient Fine-Tuning (PEFT) method has emerged as a dominant paradigm for adapting pre-trained GNN models to downstream tasks. However, existing PEFT methods usually exhibit significant vulnerability to various noise and attacks on graph
topology and node attributes/features. To address this issue, for the first time, we propose integrating
adversarial learning into graph prompting and develop a novel
Adversarial Graph Prompting (AGP) framework to achieve robust graph fine-tuning. Our AGP has two key aspects. First, we propose the general problem formulation of AGP as a min-max optimization problem and develop an alternating optimization scheme to solve it. For inner maximization, we propose Joint Projected Gradient Descent (JointPGD)
algorithm to generate strong
adversarial noise. For outer minimization, we employ a simple yet effective module to learn the optimal node
prompts to counteract the
adversarial noise. Second, we demonstrate that the proposed AGP can theoretically address both graph
topology and node noise. This confirms the versatility and robustness of our AGP fine-tuning method across various graph noise. Note that, the proposed AGP is a general method that can be integrated with various pre-trained GNN models to enhance their robustness on the downstream tasks. Extensive experiments on multiple
benchmark tasks validate the robustness and effectiveness of AGP method compared to state-of-the-art methods.
Read the full abstract ▾
Machine Learning
Unknown Aware AI-Generated Content Attribution
Ellie Thieu, Jifan Zhang and Haoyue Bai
The rapid advancement of photorealistic generative models has made it increasingly important to attribute the origin of synthetic content, moving beyond binary real or fake detection toward identifying the specific model that produced a given image. We study the problem of distinguishing outputs from a target generative model (e.g., OpenAI Dalle 3) from other sources, including real images and images generated by a wide range of alternative models. Using CLIP features and a simple linear classifier, shown to be effective in prior work, we establish a strong baseline for target generator attribution using only limited labeled data from the target model and a small number of known generators. However, this baseline struggles to generalize to harder, unseen, and newly released generators. To address this limitation, we propose a constrained optimization approach that leverages unlabeled wild data, consisting of images collected from the Internet that may include real images, outputs from unknown generators, or even samples from the target model itself. The proposed method encourages wild samples to be classified as non target while explicitly constraining performance on labeled data to remain high. Experimental results show that incorporating wild data substantially improves attribution performance on challenging unseen generators, demonstrating that unlabeled data from the wild can be effectively exploited to enhance AI generated content attribution in open world settings.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Detecting Unobserved Confounders: A Kernelized Regression Approach
Yikai Chen, Yunxin Mao, Chunyuan Zheng et al. (+7 more)
Detecting unobserved confounders is crucial for reliable causal inference in observational studies. Existing methods require either linearity assumptions or multiple heterogeneous environments, limiting applicability to nonlinear single-environment settings. To bridge this gap, we propose Kernel Regression Confounder Detection (KRCD), a novel method for detecting unobserved confounding in nonlinear observational data under single-environment conditions. KRCD leverages reproducing kernel
Hilbert spaces to model complex dependencies. By comparing standard and higherorder kernel regressions, we derive a test statistic whose significant deviation from zero indicates unobserved confounding. Theoretically, we prove two key results: First, in infinite samples, regression coefficients coincide if and only if no unobserved confounders exist. Second, finite-sample differences converge to zero-mean Gaussian distributions with tractable variance. Extensive experiments on synthetic
benchmarks and the Twins
dataset demonstrate that KRCD not only outperforms existing baselines but also achieves superior computational efficiency.
Read the full abstract ▾
All authors Yikai Chen, Yunxin Mao, Chunyuan Zheng, Hao Zou, Shanzhi Gu, Shixuan Liu, Yang Shi, Wenjing Yang, Kun Kuang, Haotian Wang
Machine Learning · Cross-listed from cs.CE
StockBot 2.0: Vanilla LSTMs Outperform Transformer-based Forecasting for Stock Prices
Shaswat Mohanty
Accurate
forecasting of financial
markets remains a long-standing challenge due to complex temporal and often latent dependencies, non-linear dynamics, and high
volatility. Building on our earlier recurrent neural network framework, we present an enhanced StockBot architecture that systematically evaluates modern
attention-based, convolutional, and recurrent time-series
forecasting models within a unified experimental setting. While
attention-based and
transformer-inspired models offer increased modeling flexibility, extensive empirical evaluation reveals that a carefully constructed vanilla LSTM consistently achieves superior predictive accuracy and more stable buy/sell decision-making when trained under a common set of default hyperparameters. These results highlight the robustness and data efficiency of recurrent sequence models for financial time-series
forecasting, particularly in the absence of extensive hyperparameter tuning or the availability of sufficient data when discretized to single-day intervals. Additionally, these results underscore the importance of architectural inductive bias in data-limited
market prediction tasks.
Read the full abstract ▾
Machine Learning
Optimized Hybrid Feature Engineering for Resource-Efficient Arrhythmia Detection in ECG Signals: An Optimization Framework
Moirangthem Tiken Singh and Manibhushan Yaikhom
Cardiovascular diseases, particularly arrhythmias, remain a leading global cause of mortality, necessitating continuous monitoring via the Internet of Medical Things (IoMT). However, state-of-the-art deep learning approaches often impose prohibitive computational overheads, rendering them unsuitable for resource-constrained edge devices. This study proposes a resource-efficient, data-centric framework that prioritizes feature engineering over complexity. Our optimized pipeline makes the complex, high-dimensional arrhythmia data linearly separable. This is achieved by integrating time-frequency wavelet decompositions with graph-theoretic structural descriptors, such as PageRank centrality. This hybrid feature space, combining wavelet decompositions and graph-theoretic descriptors, is then refined using mutual information and recursive elimination, enabling interpretable, ultra-lightweight linear classifiers. Validation on the MIT-BIH and INCART
datasets yields 98.44% diagnostic accuracy with an 8.54 KB model footprint. The system achieves 0.46 $\mu$s classification inference latency within a 52 ms per-beat pipeline, ensuring real-time operation. These outcomes provide an order-of-magnitude efficiency gain over compressed models, such as KD-Light (25 KB, 96.32% accuracy), advancing battery-less cardiac sensors.
Read the full abstract ▾
Machine Learning
SSI-GAN: Semi-Supervised Swin-Inspired Generative Adversarial Networks for Neuronal Spike Classification
Danial Sharifrazi, Nouman Javed, Mojtaba Mohammadi et al. (+5 more)
Mosquitos are the main transmissive
agents of arboviral diseases. Manual classification of their neuronal spike patterns is very labor-intensive and expensive. Most available deep learning solutions require fully labeled spike
datasets and highly preprocessed neuronal signals. This reduces the feasibility of mass adoption in actual field scenarios. To address the scarcity of labeled data problems, we propose a new Generative
Adversarial Network (GAN) architecture that we call the Semi-supervised Swin-Inspired GAN (SSI-GAN). The Swin-inspired, shifted-window discriminator, together with a
transformer-based generator, is used to classify neuronal spike trains and, consequently, detect viral neurotropism. We use a multi-head self-
attention model in a flat, window-based
transformer discriminator that learns to capture sparser high-frequency spike features. Using just 1 to 3% labeled data, SSI-GAN was trained with more than 15 million spike samples collected at five-time post-infection and recording classification into Zika-infected, dengue-infected, or uninfected categories. Hyperparameters were optimized using the
Bayesian Optuna framework, and performance for robustness was validated under fivefold Monte Carlo cross-validation. SSI-GAN reached 99.93% classification accuracy on the third day post-infection with only 3% labeled data. It maintained high accuracy across all stages of infection with just 1% supervision. This shows a 97-99% reduction in manual labeling effort relative to standard supervised approaches at the same performance level. The shifted-window
transformer design proposed here beat all baselines by a wide margin and set new best marks in spike-based neuronal infection classification.
Read the full abstract ▾
All authors Danial Sharifrazi, Nouman Javed, Mojtaba Mohammadi, Seyede Sana Salehi, Roohallah Alizadehsani, Prasad N. Paradkar, U. Rajendra Acharya, Asim Bhatti
Machine Learning
Reinforcement-Learned Unequal Error Protection for Quantized Semantic Embeddings
Moirangthem Tiken Singh and Adnan Arif
This paper tackles the pressing challenge of preserving semantic meaning in communication systems constrained by limited bandwidth. We introduce a novel reinforcement learning framework that achieves per-dimension unequal error protection via adaptive repetition coding. Central to our approach is a composite semantic distortion metric that balances global
embedding similarity with entity-level preservation, empowering the reinforcement learning
agent to allocate protection in a context-aware manner. Experiments show statistically significant gains over uniform protection, achieving 6.8% higher chrF scores and 9.3% better entity preservation at 1 dB SNR. The key innovation of our framework is the demonstration that simple, intelligently allocated repetition coding enables fine-grained semantic protection – an advantage unattainable with conventional codes such as LDPC or Reed-Solomon. Our findings challenge traditional channel coding paradigms by establishing that code structure must align with semantic granularity. This approach is particularly suited to edge computing and IoT scenarios, where bandwidth is scarce, but semantic fidelity is critical, providing a practical pathway for next-generation semantic-aware networks.
Read the full abstract ▾
Machine Learning
Does Machine Learning Outperform Traditional Fibrosis Scores in Predicting Liver Cirrhosis Risk? A Longitudinal EHR-Based Study
Zhuqi Miao, Ahmed G Qasem, Sujan Ravi et al. (+6 more)
Objective: Develop and evaluate machine learning (ML) models for predicting incident liver cirrhosis (LC) one and two years before diagnosis using routinely collected electronic health record (EHR) data and compare their performance with the FIB-4 and APRI clinical scores. Methods: We conducted a retrospective cohort study using de-identified EHR data from a large academic health system. Adult patients with diagnostic evidence of LC or LC-related risk conditions were identified using ICD-9/10 codes and classified into cirrhosis and non-cirrhosis cohorts. One- and two-year prediction scenarios were created using observation and prediction windows. Demographics, diagnoses, laboratory results, and vital signs from the observation window were used as predictors. XGBoost models were developed with feature selection and
Bayesian hyperparameter tuning and evaluated on held-out test sets. The performance of XGBoost, FIB-4, and APRI were compared on the same test data using accuracy, precision, recall, F1 score, AUC, and PR AUC. Results: The final cohorts included 54,365 patients for the 1-year prediction and 43,743 for the 2-year prediction. XGBoost consistently outperformed FIB-4 and APRI across both prediction horizons. The ML models achieved AUCs of 0.834 and 0.811 versus 0.700 and 0.677 for FIB-4 and 0.744 and 0.719 for APRI. PR AUCs were 0.502 and 0.434 for XGBoost compared with 0.310 and 0.241 for FIB-4 and 0.372 and 0.306 for APRI. Conclusions: ML models using routine EHR data substantially outperform traditional clinical scores for early LC prediction, enabling more accurate risk stratification and supporting earlier clinical intervention through automated decision support.
Read the full abstract ▾
All authors Zhuqi Miao, Ahmed G Qasem, Sujan Ravi, Jason T. Cheng, Abdulaziz Ahmed, Courtney W. Houchen, Sumayah Abed, Dilorom Azimdjanovna Zuparova, Abdulaziz Ahmed
Machine Learning
Sequential Reservoir Computing for Efficient High-Dimensional Spatiotemporal Forecasting
Ata Akbari Asanjan, Filip Wudarski, Daniel O'Connor et al. (+4 more)
Forecasting high-dimensional spatiotemporal systems remains computationally challenging for recurrent neural networks (RNNs) and long short-term
memory (LSTM) models due to gradient-based training and
memory bottlenecks. Reservoir Computing (RC) mitigates these challenges by replacing backpropagation with fixed recurrent layers and a convex readout optimization, yet conventional RC architectures still scale poorly with input dimensionality. We introduce a Sequential Reservoir Computing (Sequential RC) architecture that decomposes a large reservoir into a series of smaller, interconnected reservoirs. This design reduces
memory and computational costs while preserving long-term temporal dependencies. Using both low-dimensional chaotic systems (Lorenz63) and high-dimensional physical simulations (2D vorticity and shallow-water equations), Sequential RC achieves 15-25% longer valid forecast horizons, 20-30% lower error metrics (SSIM, RMSE), and up to three orders of magnitude lower training cost compared to LSTM and standard RNN baselines. The results demonstrate that Sequential RC maintains the simplicity and efficiency of conventional RC while achieving superior scalability for high-dimensional dynamical systems. This approach provides a practical path toward real-time, energy-efficient
forecasting in scientific and engineering applications.
Read the full abstract ▾
All authors Ata Akbari Asanjan, Filip Wudarski, Daniel O'Connor, Shaun Geaney, Elena Strbac, P. Aaron Lott, Davide Venturelli
Machine Learning
Online Finetuning Decision Transformers with Pure RL Gradients
Junkai Luo and Yinglun Zhu
Decision
Transformers (DTs) have emerged as a powerful framework for sequential decision making by formulating offline reinforcement learning (RL) as a sequence modeling problem. However, extending DTs to online settings with pure RL gradients remains largely unexplored, as existing approaches continue to rely heavily on supervised sequence-modeling objectives during online finetuning. We identify hindsight return relabeling – a standard component in online DTs – as a critical obstacle to RL-based finetuning: while beneficial for supervised learning, it is fundamentally incompatible with importance sampling-based RL
algorithms such as GRPO, leading to unstable training. Building on this insight, we propose new
algorithms that enable online finetuning of Decision
Transformers using pure reinforcement learning gradients. We adapt GRPO to DTs and introduce several key modifications, including sub-trajectory optimization for improved credit assignment, sequence-level likelihood objectives for enhanced stability and efficiency, and active sampling to encourage exploration in uncertain regions. Through extensive experiments, we demonstrate that our methods outperform existing online DT baselines and achieve new state-of-the-art performance across multiple
benchmarks, highlighting the effectiveness of pure-RL-based online finetuning for Decision
Transformers.
Read the full abstract ▾
Machine Learning
The Weather Paradox: Why Precipitation Fails to Predict Traffic Accident Severity in Large-Scale US Data
Yann Bellec, Rohan Kaman, Siwen Cui et al. (+2 more)
This study investigates the predictive capacity of environmental, temporal, and spatial factors on traffic accident severity in the United States. Using a
dataset of 500,000 U.S. traffic accidents spanning 2016-2023, we trained an XGBoost classifier optimized through randomized search cross-validation and adjusted for class imbalance via class weighting. The final model achieves an overall accuracy of 78%, with strong performance on the majority class (Severity 2), attaining 87% precision and recall. Feature importance analysis reveals that time of day, geographic location, and weather-related variables, including visibility, temperature, and wind speed, rank among the strongest predictors of accident severity. However, contrary to initial hypotheses, precipitation and visibility demonstrate limited predictive power, potentially reflecting behavioral adaptation by drivers under overtly hazardous conditions. The
dataset's predominance of mid-level severity accidents constrains the model's capacity to learn meaningful patterns for extreme cases, highlighting the need for alternative sampling strategies, enhanced feature engineering, and integration of external
datasets. These findings contribute to evidence-based traffic management and suggest future directions for severity prediction research.
Read the full abstract ▾
All authors Yann Bellec, Rohan Kaman, Siwen Cui, Aarav Agrawal, Calvin Chen
Machine Learning
Reinforcement Learning with Function Approximation for Non-Markov Processes
Ali Devran Kara
We study reinforcement learning methods with linear function approximation under non-Markov state and cost processes. We first consider the policy evaluation method and show that the
algorithm converges under suitable ergodicity conditions on the underlying non-Markov processes. Furthermore, we show that the limit corresponds to the fixed point of a joint operator composed of an orthogonal projection and the Bellman operator of an auxiliary Markov decision process. For Q-learning with linear function approximation, as in the Markov setting, convergence is not guaranteed in general. We show, however, that for the special case where the basis functions are chosen based on quantization maps, the convergence can be shown under similar ergodicity conditions. Finally, we apply our results to partially observed Markov decision processes, where finite-
memory variables are used as state representations, and we derive explicit error bounds for the limits of the resulting learning
algorithms.
Read the full abstract ▾
Machine Learning · Cross-listed from astro-ph.IM
Combining datasets with different ground truths using Low-Rank Adaptation to generalize image-based CNN models for photometric redshift prediction
Vikram Seenivasan, Srinath Saikrishnan, Andrew Lizarraga et al. (+3 more)
In this work, we demonstrate how Low-Rank Adaptation (LoRA) can be used to combine different
galaxy imaging
datasets to improve
redshift estimation with CNN models for cosmology. LoRA is an established technique for large
language models that adds adapter networks to adjust model weights and biases to efficiently fine-tune large base models without retraining. We train a base model using a photometric
redshift ground truth
dataset, which contains broad
galaxy types but is less accurate. We then fine-tune using LoRA on a spectroscopic
redshift ground truth
dataset. These
redshifts are more accurate but limited to bright
galaxies and take orders of magnitude more time to obtain, so are less available for large surveys. Ideally, the combination of the two
datasets would yield more accurate models that generalize well. The LoRA model performs better than a traditional transfer learning method, with $\sim2.5\times$ less bias and $\sim$2.2$\times$ less scatter. Retraining the model on a combined
dataset yields a model that generalizes better than LoRA but at a cost of greater computation time. Our work shows that LoRA is useful for fine-tuning regression models in astrophysics by providing a middle ground between full retraining and no retraining. LoRA shows potential in allowing us to leverage existing pretrained astrophysical models, especially for data sparse tasks.
Read the full abstract ▾
All authors Vikram Seenivasan, Srinath Saikrishnan, Andrew Lizarraga, Jonathan Soriano, Bernie Boscoe, Tuan Do