Machine Learning · Cross-listed from cs.AI
FAME: Formal Abstract Minimal Explanation for Neural Networks
Ryma Boumazouza, Raya Elsaleh, Melanie Ducoffe et al. (+2 more)
We propose FAME (Formal Abstract Minimal Explanations), a new class of abductive explanations grounded in abstract interpretation. FAME is the first method to scale to large neural networks while reducing explanation size. Our main contribution is the design of dedicated perturbation domains that eliminate the need for traversal order. FAME progressively shrinks these domains and leverages LiRPA-based bounds to discard irrelevant features, ultimately converging to a formal abstract minimal explanation. To assess explanation quality, we introduce a procedure that measures the worst-case distance between an abstract minimal explanation and a true minimal explanation. This procedure combines
adversarial attacks with an optional VERIX+ refinement step. We
benchmark FAME against VERIX+ and demonstrate consistent gains in both explanation size and runtime on medium- to large-scale neural networks.
Read the full abstract ▾
All authors Ryma Boumazouza, Raya Elsaleh, Melanie Ducoffe, Shahaf Bassan, Guy Katz
Machine Learning · Cross-listed from cs.CR
Detecting and Eliminating Neural Network Backdoors Through Active Paths with Application to Intrusion Detection
Eirik Høyheim, Magnus Wiik Eckhoff, Gudmund Grov et al. (+2 more)
Machine learning
backdoors have the property that the machine learning model should work as expected on normal inputs, but when the input contains a specific $\textit{trigger}$, it behaves as the attacker desires. Detecting such triggers has been proven to be extremely difficult. In this paper, we present a novel and explainable approach to detect and eliminate such
backdoor triggers based on active paths found in neural networks. We present promising experimental evidence of our approach, which involves injecting
backdoors into a machine learning model used for intrusion detection.
Read the full abstract ▾
All authors Eirik Høyheim, Magnus Wiik Eckhoff, Gudmund Grov, Robert Flood, David Aspinall
Machine Learning
Reinforcement Learning with Conditional Expectation Reward
Changyi Xiao, Caijun Xu and Yixin Cao
Reinforcement Learning with Verifiable Rewards (RLVR) has proven effective in enhancing the
reasoning capabilities of large
language models, particularly in domains such as mathematics where reliable rule-based verifiers can be constructed. However, the reliance on handcrafted, domain-specific verification rules substantially limits the applicability of RLVR to general
reasoning domains with free-form answers, where valid answers often exhibit significant variability, making it difficult to establish complete and accurate rules. To address this limitation, we propose Conditional Expectation Reward (CER), which leverages the large
language model itself as an implicit verifier, and is therefore applicable to general domains and eliminates the need for external verifiers or auxiliary models. CER is defined as the expected likelihood of generating the reference answer conditioned on the generated answer. In contrast to rule-based verifiers that yield binary feedback, CER provides a soft, graded reward signal that reflects varying degrees of correctness, making it better suited to tasks where answers vary in correctness. Experimental results demonstrate that CER is effective across a wide range of
reasoning tasks, spanning both mathematical and general domains, indicating that CER serves as a flexible and general verification mechanism. The code is available at https://github.com/changyi7231/CER.
Read the full abstract ▾
Machine Learning · Cross-listed from eess.AS
Geo-ATBench: A Benchmark for Geospatial Audio Tagging with Geospatial Semantic Context
Yuanbo Hou, Yanru Wu, Qiaoqiao Ren et al. (+3 more)
Environmental sound understanding in computational auditory scene analysis (CASA) is often formulated as an audio-only recognition problem. This formulation leaves a persistent drawback in multi-label audio tagging (AT): acoustic similarity can make certain events difficult to separate from waveforms alone. In such cases, disambiguating cues often lie outside the waveform. Geospatial semantic context (GSC), derived from geographic information system data, e.g., points of interest (POI), provides location-tied environmental priors that can help reduce this ambiguity. A systematic study of this direction is enabled through the proposed geospatial audio tagging (Geo-AT) task, which conditions multi-label sound event tagging on GSC alongside audio. To
benchmark Geo-AT, Geo-ATBench is introduced as a polyphonic audio
benchmark with geographical annotations, containing 10.71 hours of audio across 28 event categories; each clip is paired with a GSC representation from 11 semantic context categories. GeoFusion-AT is proposed as a unified geo-audio fusion framework that evaluates feature-, representation-, and decision-level fusion on representative audio backbones, with audio- and GSC-only baselines. Results show that incorporating GSC improves AT performance, especially on acoustically confounded labels, indicating geospatial semantics provide effective priors beyond audio alone. A crowdsourced listening study with 10 participants on 579 samples shows that there is no significant difference in performance between models on Geo-ATBench labels and aggregated human labels, supporting Geo-ATBench as a human-aligned
benchmark. The Geo-AT task,
benchmark Geo-ATBench, and reproducible geo-audio fusion framework GeoFusion-AT provide a foundation for studying AT with geospatial semantic context within the CASA community.
Dataset, code, models are on homepage (https://github.com/WuYanru2002/Geo-ATBench).
Read the full abstract ▾
All authors Yuanbo Hou, Yanru Wu, Qiaoqiao Ren, Shengchen Li, Stephen Roberts, Dick Botteldooren
Machine Learning · Cross-listed from cs.MS
Self-Scaled Broyden Family of Quasi-Newton Methods in JAX
Ivan Bioli and Mikel Mendibe Abarrategi
We present a JAX implementation of the Self-Scaled Broyden family of quasi-Newton methods, fully compatible with JAX and building on the Optimistix~\cite{rader_optimistix_2024} optimisation library. The implementation includes BFGS, DFP, Broyden and their Self-Scaled variants (SSBFGS, SSDFP, SSBroyden), as well as L-BFGS, together with a Zoom line search satisfying the strong Wolfe conditions. This is a short technical note, not a research paper, as it does not claim any novel contribution; its purpose is to document the implementation and ease the adoption of these optimisers within the JAX community. The code is available at https://github.com/IvanBioli/ssbroyden_optimistix and on PyPI (pip install ssbroyden-optimistix).
Read the full abstract ▾
Machine Learning
Gradient Flow Drifting: Generative Modeling via Wasserstein Gradient Flows of KDE-Approximated Divergences
Jiarui Cao, Zixuan Wei and Yuxin Liu
We reveal a precise mathematical framework about a new family of generative models which we call Gradient Flow Drifting. With this framework, we prove an equivalence between the recently proposed Drifting Model and the Wasserstein gradient flow of the forward KL divergence under kernel density estimation (KDE) approximation. Specifically, we prove that the drifting field of drifting model (arXiv:2602.04770) equals, up to a bandwidth-squared scaling factor, the difference of KDE log-density gradients $\nabla \log p_{\mathrm{kde}} - \nabla \log q_{\mathrm{kde}}$, which is exactly the particle velocity field of the Wasserstein-2 gradient flow of $KL(q\|p)$ with KDE-approximated densities. Besides that, this broad family of generative models can also include MMD-based generators, which arises as special cases of Wasserstein gradient flows of different divergences under KDE approximation. We provide a concise identifiability proof, and a theoretically grounded mixed-divergence strategy. We combine reverse KL and $\chi^2$ divergence gradient flows to simultaneously avoid mode collapse and mode blurring, and extend this method onto Riemannian
manifold which loosens the constraints on the kernel function, and makes this method more suitable for the semantic space. Preliminary experiments on synthetic
benchmarks validate the framework.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
Does LLM Alignment Really Need Diversity? An Empirical Study of Adapting RLVR Methods for Moral Reasoning
Zhaowei Zhang, Xiaohan Liu, Xuekai Zhu et al. (+6 more)
Reinforcement learning with verifiable rewards (RLVR) has achieved remarkable success in logical
reasoning tasks, yet whether large
language model (LLM)
alignment requires fundamentally different approaches remains unclear. Given the apparent tolerance for multiple valid responses in moral
reasoning, a natural hypothesis is that
alignment tasks inherently require diversity-seeking distribution-matching
algorithms rather than reward-maximizing policy-based methods. We conduct the first comprehensive empirical study comparing both paradigms on MoReBench. To enable stable RLVR training, we build a rubric-grounded reward pipeline by training a Qwen3-1.7B judge model. Contrary to our hypothesis, we find that distribution-matching approaches do not demonstrate significant advantages over reward-maximizing methods as expected on
alignment tasks. Through semantic visualization mapping high-reward responses to semantic space, we demonstrate that moral
reasoning exhibits more concentrated high-reward distributions than mathematical
reasoning, where diverse solution strategies yield similarly high rewards. This counter-intuitive finding explains why mode-seeking optimization proves equally or more effective for
alignment tasks. Our results suggest that
alignment tasks do not inherently require diversity-preserving
algorithms, and standard reward-maximizing RLVR methods can effectively transfer to moral
reasoning without explicit diversity mechanisms.
Read the full abstract ▾
All authors Zhaowei Zhang, Xiaohan Liu, Xuekai Zhu, Junchao Huang, Ceyao Zhang, Zhiyuan Feng, Yaodong Yang, Xiaoyuan Yi, Xing Xie
Machine Learning
HAPEns: Hardware-Aware Post-Hoc Ensembling for Tabular Data
Jannis Maier and Lennart Purucker
Ensembling is commonly used in machine learning on tabular data to boost predictive performance and robustness, but larger ensembles often lead to increased hardware demand. We introduce HAPEns, a post-hoc ensembling method that explicitly balances accuracy against hardware efficiency. Inspired by multi-objective and quality diversity optimization, HAPEns constructs a diverse set of ensembles along the Pareto front of predictive performance and resource usage. Existing hardware-aware post-hoc ensembling baselines are not available, highlighting the novelty of our approach. Experiments on 83 tabular classification
datasets show that HAPEns significantly outperforms baselines, finding superior trade-offs for ensemble performance and deployment cost. Ablation studies also reveal that
memory usage is a particularly effective objective metric. Further, we show that even a greedy ensembling
algorithm can be significantly improved in this task with static multi-objective weighting.
Read the full abstract ▾
Machine Learning
Implicit Statistical Inference in Transformers: Approximating Likelihood-Ratio Tests In-Context
Faris Chaudhry and Siddhant Gadkari
In-context learning (ICL) allows
Transformers to adapt to novel tasks without weight updates, yet the underlying
algorithms remain poorly understood. We adopt a statistical decision-theoretic perspective by investigating simple binary hypothesis testing, where the optimal policy is determined by the likelihood-ratio test. Notably, this setup provides a mathematically rigorous setting for mechanistic interpretability where the target algorithmic ground truth is known. By training
Transformers on tasks requiring distinct geometries (linear shifted means vs. nonlinear variance estimation), we demonstrate that the models approximate the Bayes-optimal sufficient statistics from context up to some monotonic transformation, matching the performance of an ideal oracle
estimator in nonlinear regimes. Leveraging this analytical ground truth, mechanistic analysis via logit lens and
circuit alignment suggests that the model does not rely on a fixed kernel smoothing heuristic. Instead, it appears to adapt the point at which decisions become linearly decodable: exhibiting patterns consistent with a voting-style ensemble for linear tasks while utilizing a deeper sequential computation for nonlinear tasks. These findings suggest that ICL emerges from the construction of task-adaptive statistical
estimators rather than simple similarity matching.
Read the full abstract ▾
Machine Learning
Riemannian Geometry-Preserving Variational Autoencoder for MI-BCI Data Augmentation
Viktorija Poļaka, Ivo Pascal de Jong and Andreea Ioana Sburlea
This paper addresses the challenge of generating synthetic electroencephalogram (
EEG) covariance
matrices for motor imagery
brain-computer interface (MI-BCI) applications. Objective: We aim to develop a generative model capable of producing high-fidelity synthetic covariance
matrices while preserving their symmetric positive-definite nature. Approach: We propose a Riemannian geometry-preserving variational autoencoder (RGP-VAE) integrating
geometric mappings with a composite loss function combining Riemannian distance, tangent space reconstruction accuracy and generative diversity. Results: The model generates valid, representative
EEG covariance
matrices, while learning a subject-invariant latent space. Synthetic data proves practically useful for MI-BCI, with its impact depending on the paired classifier. Contribution: This work introduces and validates the RGP-VAE as a geometry-preserving generative model for
EEG covariance
matrices, highlighting its potential for signal
privacy, scalability and data augmentation.
Read the full abstract ▾
Machine Learning · Cross-listed from math.OC
Quantization Robustness of Monotone Operator Equilibrium Networks
James Li, Philip H.W. Leong and Thomas Chaffey
Monotone operator
equilibrium networks are implicit-layer models whose output is the unique
equilibrium of a monotone operator, guaranteeing existence, uniqueness, and convergence. When deployed on low-precision hardware, weights are quantized, potentially destroying these guarantees. We analyze weight quantization as a spectral perturbation of the underlying monotone inclusion. Convergence of the quantized solver is guaranteed whenever the spectral-norm weight perturbation is smaller than the monotonicity margin; the displacement between quantized and full-precision
equilibria is bounded in terms of the perturbation size and margin; and a condition number characterizing the ratio of the operator norm to the margin links quantization precision to forward error. MNIST experiments confirm a phase transition at the predicted threshold: three- and four-bit post-training quantization diverge, while five-bit and above converge. The backward-pass guarantee enables quantization-aware training, which recovers provable convergence at four bits.
Read the full abstract ▾
Machine Learning
A Bipartite Graph Approach to U.S.-China Cross-Market Return Forecasting
Jing Liu, Maria Grith, Xiaowen Dong et al. (+1 more)
This paper studies cross-
market return predictability through a machine learning framework that preserves economic structure. Exploiting the non-overlapping trading hours of the U.S. and Chinese equity
markets, we construct a directed bipartite graph that captures time-ordered predictive linkages between stocks across
markets. Edges are selected via rolling-window hypothesis testing, and the resulting graph serves as a sparse, economically interpretable feature-selection layer for downstream machine learning models. We apply a range of regularized and ensemble methods to forecast open-to-close returns using lagged foreign-
market information. Our results reveal a pronounced directional asymmetry: U.S. previous-close-to-close returns contain substantial predictive information for Chinese intraday returns, whereas the reverse effect is limited. This informational asymmetry translates into economically meaningful performance differences and highlights how structured machine learning frameworks can uncover cross-
market dependencies while maintaining interpretability.
Read the full abstract ▾
All authors Jing Liu, Maria Grith, Xiaowen Dong, Mihai Cucuringu
Machine Learning
Learning to Score: Tuning Cluster Schedulers through Reinforcement Learning
Martin Asenov, Qiwen Deng, Gingfung Yeung et al. (+1 more)
Efficiently allocating incoming jobs to nodes in large-scale clusters can lead to substantial improvements in both cluster utilization and job performance. In order to allocate incoming jobs, cluster schedulers usually rely on a set of scoring functions to rank feasible nodes. Results from individual scoring functions are usually weighted equally, which could lead to sub-optimal deployments as the one-size-fits-all solution does not take into account the characteristics of each workload. Tuning the weights of scoring functions, however, requires expert knowledge and is computationally expensive. This paper proposes a reinforcement learning approach for learning the weights in scheduler scoring
algorithms with the overall objective of improving the end-to-end performance of jobs for a given cluster. Our approach is based on percentage improvement reward, frame-stacking, and limiting domain information. We propose a percentage improvement reward to address the objective of multi-step parameter tuning. The inclusion of frame-stacking allows for carrying information across an optimization experiment. Limiting domain information prevents overfitting and improves performance in unseen clusters and workloads. The policy is trained on different combinations of workloads and cluster setups. We demonstrate the proposed approach improves performance on average by 33% compared to fixed weights and 12% compared to the best-performing baseline in a lab-based serverless scenario.
Read the full abstract ▾
All authors Martin Asenov, Qiwen Deng, Gingfung Yeung, Adam Barker
Machine Learning
SCORE: Replacing Layer Stacking with Contractive Recurrent Depth
Guillaume Godin
Residual connections are central to modern deep neural networks, enabling stable optimization and efficient information flow across depth. In this work, we propose SCORE (Skip-Connection ODE Recurrent
Embedding), a discrete recurrent alternative to classical layer stacking. Instead of composing multiple independent layers, SCORE iteratively applies a single shared neural block using an ODE (Ordinary Differential Equation)-inspired contractive update: ht+1 = (1 - dt) * ht + dt * F(ht) This formulation can be interpreted as a depth-by-iteration refinement process, where the step size dt explicitly controls stability and update magnitude. Unlike continuous Neural ODE approaches, SCORE uses a fixed number of discrete iterations and standard backpropagation without requiring ODE solvers or adjoint methods. We evaluate SCORE across graph neural networks (ESOL molecular solubility), multilayer perceptrons, and
Transformer-based
language models (nanoGPT). Across architectures, SCORE generally improves convergence speed and often accelerates training. SCORE is reducing parameter count through shared weights. In practice, simple Euler integration provides the best trade-off between computational cost and performance, while higher-order integrators yield marginal gains at increased compute. These results suggest that controlled recurrent depth with contractive residual updates offers a lightweight and effective alternative to classical stacking in deep neural networks.
Read the full abstract ▾
Machine Learning
Tackling Length Inflation Without Trade-offs: Group Relative Reward Rescaling for Reinforcement Learning
Zichao Li, Jie Lou, Fangchen Dong et al. (+8 more)
Reinforcement learning significantly enhances LLM capabilities but suffers from a critical issue: length inflation, where models adopt verbosity or inefficient
reasoning to maximize rewards. Prior approaches struggle to address this challenge in a general and lossless manner, primarily because additive penalties introduce a compensatory effect that creates optimization shortcuts, while heuristic gating strategies lack generality beyond binary feedback. To bridge this gap, we present Group Relative Reward Rescaling (GR$^3$), which reframes length control as a multiplicative rescaling paradigm, effectively establishing a generalized, continuous, and reward-dependent gating mechanism. To further ensure lossless optimization, we incorporate group-relative regularization and advantage-aware calibration, which dynamically adapt length budgets to instance difficulty and preserve the advantage signal of high-quality trajectories. Empirically, across both RLHF and RLVR settings, GR$^3$~maintains training dynamics and downstream performance comparable to standard GRPO while significantly mitigating length inflation, outperforming state-of-the-art length-regularized baselines.
Read the full abstract ▾
All authors Zichao Li, Jie Lou, Fangchen Dong, Zhiyuan Fan, Mengjie Ren, Hongyu Lin, Xianpei Han, Debing Zhang, Le Sun, Yaojie Lu, Xing Yu
Machine Learning
UAV-MARL: Multi-Agent Reinforcement Learning for Time-Critical and Dynamic Medical Supply Delivery
Islam Guven and Mehmet Parlak
Unmanned aerial vehicles (
UAVs) are increasingly used to support time-critical medical supply delivery, providing rapid and flexible logistics during emergencies and resource shortages. However, effective deployment of
UAV fleets requires coordination mechanisms capable of prioritizing medical requests, allocating limited aerial resources, and adapting delivery schedules under uncertain operational conditions. This paper presents a multi-
agent reinforcement learning (MARL) framework for coordinating
UAV fleets in
stochastic medical delivery scenarios where requests vary in urgency, location, and delivery deadlines. The problem is formulated as a partially observable Markov decision process (POMDP) in which
UAV agents maintain awareness of medical delivery demands while having limited visibility of other
agents due to communication and localization constraints. The proposed framework employs Proximal Policy Optimization (PPO) as the primary learning
algorithm and evaluates several variants, including asynchronous extensions, classical actor–critic methods, and architectural modifications to analyze scalability and performance trade-offs. The model is evaluated using real-world geographic data from selected clinics and hospitals extracted from the OpenStreetMap
dataset. The framework provides a decision-support layer that prioritizes medical tasks, reallocates
UAV resources in real time, and assists healthcare personnel in managing urgent logistics. Experimental results show that classical PPO achieves superior coordination performance compared to asynchronous and sequential learning strategies, highlighting the potential of reinforcement learning for adaptive and scalable
UAV-assisted healthcare logistics.
Read the full abstract ▾
Machine Learning
World Model for Battery Degradation Prediction Under Non-Stationary Aging
Kai Chin Lim and Khay Wai See
Degradation prognosis for lithium-ion
cells requires
forecasting the state-of-health (SOH) trajectory over future cycles. Existing data-driven approaches can produce trajectory outputs through direct regression, but lack a mechanism to propagate degradation dynamics forward in time. This paper formulates battery degradation prognosis as a world model problem, encoding raw voltage, current, and temperature time-series from each cycle into a latent state and propagating it forward via a learned dynamics transition to produce a future trajectory spanning 80 cycles. To investigate whether electrochemical knowledge improves the learned dynamics, a Single Particle Model (SPM) constraint is incorporated into the training loss. Three configurations are evaluated on the Severson LiFePO4 (LFP)
dataset of 138
cells. Iterative rollout halves the trajectory forecast error compared to direct regression from the same encoder. The SPM constraint improves prediction at the degradation knee where the resistance to SOH relationship is most applicable, without changing aggregate accuracy.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
IH-Challenge: A Training Dataset to Improve Instruction Hierarchy on Frontier LLMs
Chuan Guo, Juan Felipe Ceron Uribe, Sicheng Zhu et al. (+10 more)
Instruction hierarchy (IH) defines how LLMs prioritize system, developer, user, and tool instructions under conflict, providing a concrete, trust-ordered policy for resolving instruction conflicts. IH is key to defending against jailbreaks, system
prompt extractions, and agentic
prompt injections. However, robust IH behavior is difficult to train: IH failures can be confounded with instruction-following failures, conflicts can be nuanced, and models can learn shortcuts such as overrefusing. We introduce IH-Challenge, a reinforcement learning training
dataset, to address these difficulties. Fine-tuning GPT-5-Mini on IH-Challenge with online
adversarial example generation improves IH robustness by +10.0% on average across 16 in-distribution, out-of-distribution, and human red-teaming
benchmarks (84.1% to 94.1%), reduces unsafe behavior from 6.6% to 0.7% while improving helpfulness on general safety evaluations, and saturates an internal static agentic
prompt injection evaluation, with minimal capability regression. We release the IH-Challenge
dataset (https://huggingface.co/
datasets/openai/ih-challenge) to support future research on robust instruction hierarchy.
Read the full abstract ▾
All authors Chuan Guo, Juan Felipe Ceron Uribe, Sicheng Zhu, Christopher A. Choquette-Choo, Steph Lin, Nikhil Kandpal, Milad Nasr, Rai, Sam Toyer, Miles Wang, Yaodong Yu, Alex Beutel, Kai Xiao
Machine Learning · Cross-listed from cs.AI
Resource-constrained Amazons chess decision framework integrating large language models and graph attention
Tianhao Qian, Zhuoxuan Li, Jinde Cao et al. (+2 more)
Artificial intelligence has advanced significantly through the development of intelligent game-playing systems, providing rigorous testbeds for decision-making, strategic planning, and adaptive learning. However, resource-constrained environments pose critical challenges, as conventional deep learning methods heavily rely on extensive
datasets and computational resources. In this paper, we propose a lightweight hybrid framework for the Game of the Amazons, which explores the paradigm of weak-to-strong generalization by integrating the structural
reasoning of graph-based learning with the generative capabilities of large
language models. Specifically, we leverage a Graph
Attention Autoencoder to inform a multi-step Monte Carlo Tree Search, utilize a
Stochastic Graph Genetic
Algorithm to optimize evaluation signals, and harness GPT-4o-mini to generate synthetic training data. Unlike traditional approaches that rely on expert demonstrations, our framework learns from noisy and imperfect supervision. We demonstrate that the Graph
Attention mechanism effectively functions as a structural filter, denoising the LLM's outputs. Experiments on a 10$\times$10 Amazons board show that our hybrid approach not only achieves a 15%–56% improvement in decision accuracy over baselines but also significantly outperforms its teacher model (GPT-4o-mini), achieving a competitive win rate of 45.0% at N=30 nodes and a decisive 66.5% at only N=50 nodes. These results verify the feasibility of evolving specialized, high-performance game AI from general-purpose foundation models under stringent computational constraints.
Read the full abstract ▾
All authors Tianhao Qian, Zhuoxuan Li, Jinde Cao, Xinli Shi, Leszek Rutkowski
Machine Learning · Cross-listed from math.NA
A New Tensor Network: Tubal Tensor Train and Its Applications
Salman Ahmadi-Asl, Valentin Leplat, Anh-Huy Phan et al. (+1 more)
We introduce the tubal tensor train (TTT) decomposition, a tensor-network model that combines the t-product
algebra of the tensor singular value decomposition (T-SVD) with the low-order core structure of the tensor train (TT) format. For an order-$(N+1)$ tensor with a distinguished tube mode, the proposed representation consists of two third-order boundary cores and $N-2$ fourth-order interior cores linked through the t-product. As a result, for bounded tubal ranks, the storage scales linearly with the number of modes, in contrast to direct high-order extensions of T-SVD. We present two computational strategies: a sequential fixed-rank construction, called TTT-SVD, and a
Fourier-slice alternating scheme based on the alternating two-cores update (ATCU). We also state a TT-SVD-type error bound for TTT-SVD and illustrate the practical performance of the proposed model on image compression,
video compression, tensor completion, and hyperspectral imaging.
Read the full abstract ▾
All authors Salman Ahmadi-Asl, Valentin Leplat, Anh-Huy Phan, Andrzej Cichocki
Machine Learning · Cross-listed from cs.CL
Coverage-Controlled Preference Mining from Noisy Claim Verification for Evidence-Grounded Generation
Weixin Liu, Congning Ni, Qingyuan Song et al. (+4 more)
Evidence-grounded generation produces summaries whose claims should be supported by supplied evidence, but claim-level verifiers provide noisy feedback and can reward models that simply say less. We study this problem in clinical Brief Hospital Course summarization, where outputs must remain grounded in patient-specific EHR evidence. We introduce VERI-DPO, a preference-mining framework that converts noisy claim verification into coverage-controlled summary-level preferences. For each evidence-window
prompt, VERI-DPO samples multiple candidate summaries, decomposes them into claims, verifies each claim against patient evidence, and forms a preference pair only when the chosen summary has better aggregate verifier-estimated support while retaining comparable verifiable content. Standard Direct Preference Optimization then distills these pairs into a single-sample policy, avoiding inference-time reranking. On patient-disjoint MIMIC-III-Ext-VeriFact-BHC test data, VERI-DPO reduces Not Supported rates from 10.7% to 1.9% under the mining verifier and from 11.6% to 6.4% under a separately prompted GPT-4o judge. In 100 blinded pairwise assessments by two domain researchers, VERI-DPO is preferred over the base model 56 times versus 18 times for factual faithfulness. In a locked zero-shot MIMIC-IV-Ext-BHC transfer test with 1,000 patients and no model adaptation, VERI-DPO lowers Not Supported rates with nearly unchanged scored-claim counts. Multi-seed ablations show that verifier-guided pair construction drives the gains, while coverage and anti-degeneration controls prevent apparent factuality improvements from coming from shorter or less checkable outputs.
Read the full abstract ▾
All authors Weixin Liu, Congning Ni, Qingyuan Song, Susannah L. Rose, Murat Kantarcioglu, Bradley A. Malin, Zhijun Yin
Machine Learning
A Universal Nearest-Neighbor Estimator for Intrinsic Dimensionality
Eng-Jon Ong, Omer Bobrowski, Gesine Reinert et al. (+1 more)
Estimating the intrinsic dimensionality (ID) of data is a fundamental problem in machine learning and computer
vision, providing insight into the true degrees of freedom underlying high-dimensional observations. Existing methods often rely on
geometric or distributional assumptions and can significantly fail when these assumptions are violated. In this paper, we introduce a novel ID
estimator based on nearest-neighbor distance ratios that involves simple calculations and achieves state-of-the-art results. Most importantly, we provide a theoretical analysis proving that our
estimator is universal, namely, it converges to the true ID independently of the distribution generating the data. We present experimental results on
benchmark manifolds and real-world
datasets to demonstrate the performance of our
estimator.
Read the full abstract ▾
All authors Eng-Jon Ong, Omer Bobrowski, Gesine Reinert, Primoz Skraba
Machine Learning · Cross-listed from q-bio.NC
JEDI: Jointly Embedded Inference of Neural Dynamics
Anirudh Jamkhandi, Ali Korojy, Olivier Codol et al. (+2 more)
Animal
brains flexibly and efficiently achieve many behavioral tasks with a single neural network. A core goal in modern neuroscience is to map the mechanisms of the
brain's flexibility onto the dynamics underlying neural populations. However, identifying task-specific dynamical rules from limited, noisy, and high-dimensional experimental neural recordings remains a major challenge, as experimental data often provide only partial access to
brain states and dynamical mechanisms. While recurrent neural networks (RNNs) directly constrained neural data have been effective in inferring underlying dynamical mechanisms, they are typically limited to single-task domains and struggle to generalize across behavioral conditions. Here, we introduce JEDI, a hierarchical model that captures neural dynamics across tasks and contexts by learning a shared
embedding space over RNN weights. This model recapitulates individual samples of neural dynamics while scaling to arbitrarily large and complex
datasets, uncovering shared structure across conditions in a single, unified model. Using simulated RNN
datasets, we demonstrate that JEDI accurately learns robust, generalizable, condition-specific
embeddings. By reverse-engineering the weights learned by JEDI, we show that it recovers ground truth fixed point structures and unveils key features of the underlying neural dynamics in the eigenspectra. Finally, we apply JEDI to motor cortex recordings during monkey reaching to extract mechanistic insight into the neural dynamics of motor control. Our work shows that joint learning of contextual
embeddings and recurrent weights provides scalable and generalizable inference of
brain dynamics from recordings alone.
Read the full abstract ▾
All authors Anirudh Jamkhandi, Ali Korojy, Olivier Codol, Guillaume Lajoie, Matthew G. Perich
Machine Learning · Cross-listed from stat.ML
Dual Space Preconditioning for Gradient Descent in the Overparameterized Regime
Reza Ghane, Danil Akhtiamov and Babak Hassibi
In this work, we study the convergence properties of the Dual Space Preconditioned Gradient Descent, encompassing optimizers such as Normalized Gradient Descent and Gradient Clipping. We consider preconditioners of the form $\nabla K$, where $K: \mathbb{R}^{d \times k} \to \mathbb{R}$ is convex and apply $\nabla K(\cdot)$ to train an over-parameterized linear model with a convex loss of the form $\ell(X W - Y)$, for weights $W \in \mathbb{R}^{d \times k}$, labels $Y \in \mathbb{R}^{n \times k}$ and data $X \in \mathbb{R}^{n \times d}$. Under the aforementioned assumptions, we prove that the iterates of the full-batch preconditioned gradient descent converge at an exponential rate to a point $W_{\infty} \in \mathbb{R}^{d \times k}$ satisfying $XW_{\infty} = Y$. We also study the implicit bias of Dual Space Preconditioned Gradient Descent. First, we demonstrate analytically and empirically that, for general $K(\cdot)$, $W_\infty$ depends on the chosen constant step size, hindering a precise characterization of the implicit bias. We also provide an approximate implicit bias property for general preconditioners, namely, $\|W_0 - W_{\infty}\|_F \le c \|W_0 - W_{\text{GD}, \infty}\|_F$ for a constant $c>0$ and $W_{\text{GD}, \infty}$ denoting the convergence point of GD initialized at $W_0$. Furthermore, for preconditioners of the form $K(G) = h(\|G\|_F)$, known as isotropic preconditioners, and for the
stochastic variation of the
algorithm with arbitrary batch-size, we prove linear convergence to $W_{\text{GD}, \infty}$. Finally, in the experiments, we demonstrate faster convergence on a nonlinear model obtained using the smoothed
matrix elastic-net as a preconditioner.
Read the full abstract ▾
Machine Learning
Muscle Synergy Priors Enhance Biomechanical Fidelity in Predictive Musculoskeletal Locomotion Simulation
Ilseung Park, Eunsik Choi, Jangwhan Ahn et al. (+1 more)
Human locomotion emerges from high-dimensional neuromuscular control, making predictive musculoskeletal simulation challenging. We present a physiology-informed reinforcement-learning framework that constrains control using muscle synergies. We extracted a low-dimensional synergy basis from inverse musculoskeletal analyses of a small set of overground walking trials and used it as the action space for a muscle-driven three-dimensional model trained across variable speeds, slopes and uneven terrain. The resulting controller generated stable gait from 0.7-1.8 m/s and on $\pm$ 6$^{\circ}$ grades and reproduced condition-dependent modulation of joint angles, joint moments and ground reaction forces. Compared with an unconstrained controller, synergy-constrained control reduced non-physiological knee kinematics and kept knee moment profiles within the experimental envelope. Across conditions, simulated vertical ground reaction forces correlated strongly with human measurements, and muscle-activation timing largely fell within inter-subject variability. These results show that
embedding neurophysiological structure into reinforcement learning can improve biomechanical fidelity and generalization in predictive human locomotion simulation with limited experimental data.
Read the full abstract ▾
All authors Ilseung Park, Eunsik Choi, Jangwhan Ahn, Jooeun Ahn
Machine Learning · Cross-listed from physics.plasm-ph
Beam-Plasma Collective Oscillations in Intense Charged-Particle Beams: Dielectric Response Theory, Langmuir Wave Dispersion, and Unsupervised Detection via Prometheus
Brandon Yee, Wilson Collins, Michael Iofin et al. (+1 more)
We develop a theoretical and computational framework for beam-
plasma collective oscillations in intense charged-particle beams at intermediate energies (10-100 MeV). In Part I, we formulate a kinetic field theory governed by the Vlasov-Poisson system, deriving the Lindhard dielectric function and random phase approximation (RPA) polarization tensor for three beam distribution functions. We prove via the dielectric function epsilon(omega,q)=0 the existence of undamped Langmuir wave modes above a critical beam density n_c, obtain explicit beam-
plasma dispersion relations, and show that Landau damping vanishes above the particle-hole continuum. The
plasma frequency Omega_p^2 = ne^2/(m*epsilon_0) is fixed by the f-sum rule independently of distribution shape; higher dispersion coefficients depend on velocity moments. Space charge effects drive anomalous beam broadening with sqrt(n-n_c) onset and Friedel oscillations at q=2k_F. The beam-
plasma transition belongs to the 3D Ising universality class via renormalization group analysis. In Part II, we validate these predictions using Prometheus, a beta-VAE trained on static structure factor data S(q) from particle-in-
cell (PIC) beam simulations. Prometheus detects collective
plasma oscillation onset in Gaussian and uniform distributions, confirms their absence in the degenerate
Fermi gas (n_c -> 0), and resolves the Kohn anomaly at q=2k_F. Dispersion analysis of S(q,omega) from PIC simulations verifies the distribution-independent Omega_p predicted by the f-sum rule. All six validation checks pass. Predicted signatures – density-tunable
plasma resonances at omega_p proportional to sqrt(n), anomalous beam broadening with sqrt(n-n_c) onset, and Friedel oscillations – are accessible at existing intermediate-energy beam facilities.
Read the full abstract ▾
All authors Brandon Yee, Wilson Collins, Michael Iofin, Jiayi Fu
Machine Learning
Spatio-Temporal Forecasting of Retaining Wall Deformation: Mitigating Error Accumulation via Multi-Resolution ConvLSTM Stacking Ensemble
Jihoon Kim and Heejung Youn
This study proposes a multi-resolution Convolutional Long Short-Term
Memory (ConvLSTM) ensemble framework that leverages diverse temporal input resolutions to mitigate error accumulation and improve long-horizon
forecasting of retaining-structure behavior during staged excavation. An extensive database of lateral wall displacement responses was generated through PLAXIS2D simulations incorporating five-layered soil stratigraphy, two excavation depths (14 and 20 m), and stochastically varied geotechnical and structural parameters, yielding 2,000 time-series deflection profiles. Three ConvLSTM models trained at different input resolutions were integrated using a fully connected neural network meta-learner to construct the ensemble model. Validation using both numerical results and field measurements demonstrated that the ensemble approach consistently outperformed the standalone ConvLSTM models, particularly in long-term multi-step prediction, exhibiting reduced error propagation and improved generalization. These findings underscore the potential of multi-resolution ensemble strategies that jointly exploit diverse temporal input scales to enhance predictive stability and accuracy in AI-driven geotechnical
forecasting.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Brenier Isotonic Regression
Han Bao, Amirreza Eshraghi and Yutong Wang
Isotonic regression (IR) is shape-constrained regression to maintain a univariate fitting curve non-decreasing, which has numerous applications including single-index models and probability calibration. When it comes to multi-output regression, the classical IR is no longer applicable because the monotonicity is not readily extendable. We consider a novel multi-output regression problem where a regression function is cyclically monotone. Roughly speaking, a cyclically monotone function is the gradient of some convex potential. Whereas enforcing cyclic monotonicity is apparently challenging, we leverage the fact that Kantorovich's optimal transport (OT) always yields a cyclically monotone coupling as an optimal solution. This perspective naturally allows us to interpret a regression function and the convex potential as a link function in generalized linear models and Brenier's potential in OT, respectively, and hence we call this IR extension Brenier isotonic regression. We demonstrate experiments with probability calibration and generalized linear models. In particular, IR outperforms many famous baselines in probability calibration robustly.
Read the full abstract ▾
Machine Learning
Unlearning the Unpromptable: Prompt-free Instance Unlearning in Diffusion Models
Kyungryeol Lee, Kyeonghyun Lee, Seongmin Hong et al. (+2 more)
Machine unlearning aims to remove specific outputs from trained models, often at the concept level, such as forgetting all occurrences of a particular celebrity or filtering content via text
prompts. However, many undesired outputs, such as an individual's face or generations culturally or factually misinterpreted, cannot often be specified by text
prompts. We address this underexplored setting of instance unlearning for outputs that are undesired but unpromptable, where the goal is to forget target outputs selectively while preserving the rest. To this end, we introduce an effective surrogate-based unlearning method that leverages image editing, timestep-aware weighting, and gradient surgery to guide trained
diffusion models toward forgetting specific outputs. Experiments on conditional (Stable
Diffusion 3) and unconditional (DDPM-CelebA)
diffusion models demonstrate that our
prompt-free method uniquely unlearns unpromptable outputs, such as faces and culturally inaccurate depictions, with preserved integrity, unlike
prompt-based and
prompt-free baselines. Our proposed method would serve as a practical hotfix for
diffusion model providers to ensure
privacy protection and ethical compliance.
Read the full abstract ▾
All authors Kyungryeol Lee, Kyeonghyun Lee, Seongmin Hong, Byung Hyun Lee, Se Young Chun
Machine Learning
The Curse and Blessing of Mean Bias in FP4-Quantized LLM Training
Hengjie Cao, Zhendong Huang, Mengyi Chen et al. (+15 more)
FP4 training promises substantial
memory and compute savings for large
language models, but remains fragile because blockwise quantization is dictated by extreme activation magnitudes, which inflate dynamic range and compress long-tail signals. We identify a counterintuitive source of this failure: dominant activation outliers are not merely arbitrary sparse events, but are largely induced by a coherent rank-one mean bias, whose direction aligns with the leading anisotropic spectral component. This mean component strengthens during training, is amplified and reshaped by
attention and FFN operators, and increasingly dominates top activation magnitudes. Crucially, this discovery reveals that a seemingly complex outlier-suppression problem admits a truly simple solution: isolate the coherent mean before quantization. We therefore propose Averis, a mean-residual splitting quantization method that separates the mean component using only reductions and elementwise subtractions before FP4 quantization. Across Qwen3 0.6B Dense trained on 100B
tokens and Qwen3 7B A1.5B MoE trained on 50B
tokens, Averis enables robust W4A4G4 FP4 training, reducing BF16 loss gaps to 1.19%/0.81% versus 2.05%/1.10% for NVIDIA's recently released Hadamard-based outlier-smoothing method, while limiting downstream gaps to 0.89/0.71 points. With only 2.20% end-to-end overhead over vanilla NVFP4, about 30% of NVIDIA's Hadamard-based design, Averis provides a hardware-efficient path to stable low-bit LLM training. Complementary to Hadamard, Averis further reduces the Qwen3-0.6B loss and downstream gaps to 0.94% and 0.73 points when combined. Code is available at: https://anonymous.4open.science/r/averis-504D.
Read the full abstract ▾
All authors Hengjie Cao, Zhendong Huang, Mengyi Chen, Yifeng Yang, Fang Dong, Anrui Chen, Ruijun Huang, Xin Zhang, Mingzhi Dong, Yujiang Wang, Jinlong Hou, Qin Lv, Robert P.Dick, Yuan Cheng, Tun Lu, Fan Yang, Yixuan Chen, Li Shang