Machine Learning
A Hierarchical End-of-Turn Model with Primary Speaker Segmentation for Real-Time Conversational AI
Karim Helwani, Hoang Do, James Luan et al. (+1 more)
We present a real-time front-end for voice-based conversational AI to enable natural turn-taking in two-speaker scenarios by combining primary speaker
segmentation with hierarchical End-of-Turn (EOT) detection. To operate robustly in multi-speaker environments, the system continuously identifies and tracks the primary user, ensuring that downstream EOT decisions are not confounded by background conversations. The tracked activity segments are fed to a hierarchical, causal EOT model that predicts the immediate conversational state by independently analyzing per-speaker
speech features from both the primary speaker and the bot. Simultaneously, the model anticipates near-future states ($t{+}10/20/30$\,ms) through probabilistic predictions that are aware of the conversation partner's
speech. Task-specific knowledge
distillation compresses wav2vec~2.0 representations (768\,D) into a compact MFCC-based student (32\,D) for efficient deployment. The system achieves 82% multi-class frame-level F1 and 70.6% F1 on Backchannel detection, with 69.3% F1 on a binary Final vs.\ Others task. On an end-to-end turn-detection
benchmark, our model reaches 87.7% recall vs.\ 58.9% for Smart Turn~v3 while keeping a median detection latency of 36\,ms versus 800–1300\,ms. Despite using only 1.14\,M parameters, the proposed model matches or exceeds
transformer-based baselines while substantially reducing latency and
memory footprint, making it suitable for edge deployment.
Read the full abstract ▾
All authors Karim Helwani, Hoang Do, James Luan, Sriram Srinivasan
Machine Learning · Cross-listed from cs.CV
Deep Learning for BioImaging: What Are We Really Learning?
Ivan Svatko, Maxime Sanchez, Ihab Bendidi et al. (+2 more)
Representation learning has driven major advances in natural image analysis by enabling models to acquire high-level semantic features. In microscopy imaging, however, it remains unclear what current representation learning methods really learn. In this work, we conduct a systematic study of representation learning for the two most widely used and broadly available microscopy data types, representing critical scales in biology:
cell culture and tissue imaging. We investigate whether, in contrast to natural images, existing models fail to consistently acquire high-level, biologically meaningful features. To this end, we introduce a set of simple yet revealing baselines on curated
benchmarks, including untrained models and structural representations of cellular tissue. Our results show that, surprisingly, for a considerable subset of evaluation settings, the baselines are comparable to state-of-the-art methods, demonstrating that many commonly used
benchmark metrics are insufficient to assess representation quality and often mask a lack of relevant high-level abstractions. In addition, we investigate how detailed comparisons with these baselines provide ways to interpret the strengths and weaknesses of models for further improvements. Together, our results suggest that progress in representation learning for microscopy requires not only stronger models, but also
benchmarks that are more indicative of what is actually learned.
Read the full abstract ▾
All authors Ivan Svatko, Maxime Sanchez, Ihab Bendidi, Gilles Cottrell, Auguste Genovesio
Machine Learning · Cross-listed from cs.CV
A Computer-aided Framework for Detecting Osteosarcoma in Computed Tomography Scans
Maximo Rodriguez-Herrero, Dante D. Sanchez-Gallegos, Marco Antonio Núñez-Gaona et al. (+5 more)
Osteosarcoma is the most common primary bone cancer, mainly affecting the youngest and oldest populations. Its detection at early stages is crucial to reduce the probability of developing bone metastasis. In this context, accurate and fast diagnosis is essential to help physicians during the prognosis process. The research goal is to automate the diagnosis of osteosarcoma through a pipeline that includes the preprocessing, detection, postprocessing, and visualization of computed
tomography (CT) scans. Thus, this paper presents a machine learning and visualization framework for classifying CT scans using different convolutional neural network (CNN) models. Preprocessing includes data augmentation and identification of the region of interest in scans. Post-processing includes data visualization to render a 3D bone model that highlights the affected area. An evaluation on 12 patients revealed the effectiveness of our framework, obtaining an area under the curve (AUC) of 94.8% and a specificity of 94.6%.
Read the full abstract ▾
All authors Maximo Rodriguez-Herrero, Dante D. Sanchez-Gallegos, Marco Antonio Núñez-Gaona, Heriberto Aguirre-Meneses, Luis Alberto Villalvazo Gutiérrez, Mario Ibrahin Gutiérrez Velasco, J.L. Gonzalez-Compean, Jesus Carretero
Machine Learning · Cross-listed from cs.CV
InfiniteDance: Scalable 3D Dance Generation Towards in-the-wild Generalization
Ronghui Li, Zhongyuan Hu, Li Siyao et al. (+6 more)
Although existing 3D dance generation methods perform well in controlled scenarios, they often struggle to generalize in the wild. When conditioned on unseen music, existing methods often produce unstructured or physically implausible dance, largely due to limited music-to-dance data and restricted model capacity. This work aims to push the frontier of generalizable 3D dance generation by scaling up both data and model design. (1) On the data side, we develop a fully automated pipeline that reconstructs high-fidelity 3D dance motions from
monocular videos. To eliminate the physical artifacts prevalent in existing reconstruction methods, we introduce a Foot Restoration
Diffusion Model (FRDM) guided by foot-contact and
geometric constraints that enforce physical plausibility while preserving kinematic smoothness and expressiveness, resulting in a diverse, high-quality
multimodal 3D dance
dataset totaling 100.69 hours. (2) On model design, we propose Choreographic LLaMA (ChoreoLLaMA), a scalable LLaMA-based architecture. To enhance robustness under unfamiliar music conditions, we integrate a
retrieval-augmented generation (RAG) module that injects reference dance as a
prompt. Additionally, we design a slow/fast-cadence Mixture-of-Experts (MoE) module that enables ChoreoLLaMA to smoothly adapt motion rhythms across varying music tempos. Extensive experiments across diverse dance genres show that our approach surpasses existing methods in both qualitative and quantitative evaluations, marking a step toward scalable, real-world 3D dance generation. Code, models, and data will be released.
Read the full abstract ▾
All authors Ronghui Li, Zhongyuan Hu, Li Siyao, Youliang Zhang, Haozhe Xie, Mingyuan Zhang, Jie Guo, Xiu Li, Ziwei Liu
Machine Learning · Cross-listed from cs.CY
Ethical Fairness in Ubiquitous Health Sensing without Known Attributes
Shaily Roy, Harshit Sharma, Daniel A. Adler et al. (+3 more)
Wearable and mobile sensing models infer health and well-being in everyday life, yet strong average performance can conceal failures for particular people, contexts, or devices. Existing
fairness methods often require sensitive attributes during training, although such attributes may be unavailable,
privacy-sensitive, or restricted. Moreover, they may reduce disparities by degrading performance for some subgroups, violating the ethical principle of non-maleficence. We present
FLARE (Fisher-guided LAtent-subgroup learning with do-no-harm REgularization), to our knowledge the first attribute-free framework to operationalize beneficence, non-maleficence, and justice in ubiquitous sensing.
FLARE uses Fisher information to regularize loss
curvature and combines representation, loss, and
curvature signals to discover latent performance strata. It then trains a separate model for each stratum using do-no-harm regularization and validation-controlled parameter sharing. We also introduce BHE (Benefit, Harm-Avoidance, Equity), a metric suite for evaluating ethical
fairness beyond statistical parity. Across four physiological, behavioral, and clinical sensing
datasets,
FLARE achieves the highest mean subgroup F1 on every
dataset. It is the only evaluated method that improves mean subgroup performance without degrading any evaluated subgroup and the only one that reduces disparity across all 15 evaluated sensitive attributes. Although some baselines achieve larger average disparity reductions, each harms at least one subgroup. Post-hoc analyses on two
datasets show that the discovered strata are only weakly aligned with recorded attributes and identify a recurrent within-person state that participant-level attributes cannot capture. Label-free routing and inference take at most 8.1 ms per sample on a smartphone and 10.5 ms on a Raspberry Pi, supporting real-time on-device deployment.
Read the full abstract ▾
All authors Shaily Roy, Harshit Sharma, Daniel A. Adler, Srijan Sen, Tanzeem Choudhury, Asif Salekin
Machine Learning · Cross-listed from cs.RO
TinyNav: End-to-End TinyML for Real-Time Autonomous Navigation on Microcontrollers
Pooria Roy, Nourhan Jadallah. Tomer Lapid, Shahzaib Ahmad et al. (+2 more)
Autonomous
navigation typically relies on power-intensive processors, limiting accessibility in low-cost robotics. Although microcontrollers offer a resource-efficient alternative, they impose strict constraints on model complexity. We present TinyNav, an end-to-end TinyML system for real-time autonomous
navigation on an ESP32 microcontroller. A custom-trained, quantized 2D convolutional neural network processes a 20-frame sliding window of depth data to predict steering and throttle commands. By avoiding 3D convolutions and recurrent layers, the 23k-parameter model achieves 30 ms inference latency. Correlation analysis and Grad-CAM validation indicate consistent spatial awareness and obstacle avoidance behavior. TinyNav demonstrates that responsive autonomous control can be deployed directly on highly constrained edge devices, reducing reliance on external compute resources.
Read the full abstract ▾
All authors Pooria Roy, Nourhan Jadallah. Tomer Lapid, Shahzaib Ahmad, Armita Afroushe, Mete Bayrak
Machine Learning
GSVD for Geometry-Grounded Dataset Comparison: An Alignment Angle Is All You Need
Eduarda de Souza Marques, Arthur Sobrinho Ferreira da Rocha, Joao Paixao et al. (+2 more)
Geometry-grounded learning asks models to respect structure in the problem domain rather than treating observations as arbitrary vectors. Motivated by this view, we revisit a classical but underused primitive for comparing
datasets: linear relations between two data
matrices, expressed via the co-span constraint $Ax = By = z$ in a shared ambient space. To operationalize this comparison, we use the generalized singular value decomposition (GSVD) as a joint coordinate system for two subspaces. In particular, we exploit the GSVD form $A = HCU$, $B = HSV$ with $C^{\top}C + S^{\top}S = I$, which separates shared versus
dataset-specific directions through the diagonal structure of $(C, S)$. From these factors we derive an interpretable *angle score* $\theta(z) \in [0, \pi/2]$ for a sample $z$, quantifying whether z is explained relatively more by $A$, more by $B$, or comparably by both. The primary role of $\theta(z)$ is as a *per-sample
geometric diagnostic*. We illustrate the behavior of the score on MNIST through angle distributions and representative GSVD directions. A binary classifier derived from $\theta(z)$ is presented as an illustrative application of the score as an interpretable diagnostic tool.
Read the full abstract ▾
All authors Eduarda de Souza Marques, Arthur Sobrinho Ferreira da Rocha, Joao Paixao, Heudson Mirandola, Daniel Sadoc Menasche
Machine Learning
Taming Score-Based Denoisers in ADMM: A Convergent Plug-and-Play Framework
Rajesh Shrestha and Xiao Fu
While score-based generative models have emerged as powerful priors for solving inverse problems, directly integrating them into optimization
algorithms such as ADMM remains nontrivial. Two central challenges arise: i) the mismatch between the noisy data
manifolds used to train the score functions and the geometry of ADMM iterates, especially due to the influence of dual variables, and ii) the lack of convergence understanding when ADMM is equipped with score-based denoisers. To address the
manifold mismatch issue, we propose ADMM plug-and-play (ADMM-PnP) with the AC-DC denoiser, a new framework that embeds a three-stage denoiser into ADMM: (1) auto-correction (AC) via additive Gaussian noise, (2) directional correction (DC) using conditional Langevin dynamics, and (3) score-based denoising. In terms of convergence, we establish two results: first, under proper denoiser parameters, each ADMM iteration is a weakly nonexpansive operator, ensuring high-probability fixed-point $\textit{ball convergence}$ using a constant step size; second, under more relaxed conditions, the AC-DC denoiser is a bounded denoiser, which leads to convergence under an adaptive step size schedule. Experiments on a range of inverse problems demonstrate that our method consistently improves solution quality over a variety of baselines.
Read the full abstract ▾
Machine Learning
Robust Post-Training for Generative Recommenders: Why Exponential Reward-Weighted SFT Outperforms RLHF
Keertana Chidambaram, Sanath Kumar Krishnamurthy, Qiuling Xu et al. (+2 more)
Aligning generative recommender systems to user preferences via post-training is critical for closing the gap between next-item prediction and actual recommendation quality. Existing post-training methods are ill-suited for production-scale systems: RLHF methods reward hack due to noisy user feedback and unreliable reward models, offline RL alternatives require propensity scores that are unavailable, and online interaction is infeasible. We identify exponential reward-weighted SFT with weights $w = \exp(r/\lambda)$ as uniquely suited to this setting, and provide the theoretical and empirical foundations that explain why. By optimizing directly on observed rewards without querying a learned reward model, the method is immune to reward hacking, requires no propensity scores, and is fully offline. We prove the first policy improvement guarantees for this setting under noisy rewards, showing that the gap scales only logarithmically with catalog size and remains informative even for large item catalogs. Crucially, we show that temperature $\lambda$ explicitly and quantifiably controls the robustness-improvement tradeoff, providing practitioners with a single interpretable regularization hyperparameter with theoretical grounding. Experiments on three open-source and one proprietary
dataset against four baselines confirm that exponential reward weighting is simple, scalable, and consistently outperforms RLHF-based alternatives.
Read the full abstract ▾
All authors Keertana Chidambaram, Sanath Kumar Krishnamurthy, Qiuling Xu, Ko-Jen Hsiao, Moumita Bhattacharya
Machine Learning
Estimating Condition Number with Graph Neural Networks
Erin Carson and Xinye Chen
In this paper, we propose a fast method for estimating the condition number of sparse
matrices using graph neural networks (GNNs). For efficient deployment of GNNs, we introduce a graph feature construction with $\mathrm{O}(\mathrm{nnz} + n)$ complexity, where $\mathrm{nnz}$ is the number of non-zero elements in the
matrix and $n$ denotes the
matrix dimension. We propose two schemes for estimating the
matrix condition number using GNNs; one follows by decomposing the condition number and predicts the relatively more computationally intensive part $\|\mathbf{A}^{-1}\|$, without explicitly forming the inverse, while the other is to predict the whole condition number $\kappa$. Our approach can be extended to an arbitrary norm. Extensive experiments are conducted for the estimation of the 1-norm and 2-norm condition numbers, which show that our method achieves a significant speedup over the traditional numerical estimation methods. Our software for GNN condition number
estimator is made publicly available at https://github.com/inEXASCALE/sparse-kappa.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.RO
From Prior to Pro: Efficient Skill Mastery via Distribution Contractive RL Finetuning
Zhanyi Sun and Shuran Song
We introduce Distribution Contractive Reinforcement Learning (DICE-RL), a framework that uses reinforcement learning (RL) as a "distribution contraction" operator to refine pretrained generative
robot policies. DICE-RL turns a pretrained behavior prior into a high-performing "pro" policy by amplifying high-success behaviors from online feedback. We pretrain a
diffusion- or flow-based policy for broad behavioral coverage, then finetune it with a stable, sample-efficient residual off-policy RL framework that combines selective behavior regularization with value-guided action selection. Extensive experiments and analyses show that DICE-RL reliably improves performance with strong stability and sample efficiency. It enables mastery of complex long-horizon manipulation skills directly from high-dimensional pixel inputs, both in simulation and on a real
robot. Project website: https://zhanyisun.github.io/dice.rl.2026/.
Read the full abstract ▾
Machine Learning
Discovery of a Hematopoietic Manifold in scGPT Yields a Method for Extracting Performant Algorithms from Biological Foundation Model Internals
Ihor Kendiukhov
We report the discovery and extraction of a compact hematopoietic
algorithm from the single-
cell foundation model scGPT, to our knowledge the first biologically useful, competitive
algorithm extracted from a foundation model via mechanistic interpretability. We show that scGPT internally encodes a compact hematopoietic
manifold with significant developmental branch structure, validated on a strict non-overlap Tabula Sapiens external panel and confirmed via frozen-head zero-shot transfer to an independent multi-donor immune panel. To isolate this geometry, we introduce a general three-stage extraction method consisting of direct operator export from frozen
attention weights, a lightweight learned adaptor, and a task-specific readout, producing a standalone
algorithm without target-
dataset retraining. In 88-split donor-holdout
benchmarks against scVI, Palantir, DPT, CellTypist, PCA, and raw-expression baselines, the extracted
algorithm achieves the strongest pseudotime-depth ordering and leads on key subtype endpoints (CD4/CD8 AUROC 0.867, mono/macro AUROC 0.951). Compared to standard probing of frozen scGPT
embeddings with a 3-layer MLP, the extracted head is BH-significantly better on 6/8 classification endpoints while completing a full 12-split evaluation campaign 34.5x faster with approximately 1000x fewer trainable parameters. The exported operator compresses from three pooled
attention heads to a single head without statistically significant loss, and further to a rank-64 surrogate. Mechanistic interpretability of the compact operator reveals a concentrated four-factor core explaining 66.2% of ablation impact, with factors resolving into explicit T/lymphoid, B/
plasma, granulocytic, and monocyte/macrophage gene programs. A supplementary second-
manifold validation (intercellular communication geometry) confirms that the extraction method generalizes beyond hematopoiesis.
Read the full abstract ▾
Machine Learning
Improving TabPFN's Synthetic Data Generation by Integrating Causal Structure
Davide Tugnoli, Andrea De Lorenzo, Marco Virgolin et al. (+1 more)
Synthetic tabular data generation addresses data scarcity and
privacy constraints in a variety of domains. Tabular Prior-Data Fitted Network (TabPFN), a recent foundation model for tabular data, has been shown capable of generating high-quality synthetic tabular data. However, TabPFN is autoregressive: features are generated sequentially by conditioning on the previous ones, depending on the order in which they appear in the input data. We demonstrate that when the feature order conflicts with causal structure, the model produces spurious correlations that impair its ability to generate synthetic data and preserve causal effects. We address this limitation by integrating causal structure into TabPFN's generation process through two complementary approaches: Directed Acyclic Graph (DAG)-aware conditioning, which samples each variable given its causal parents, and a partially directed acyclic graph (PDAG)-based strategy for scenarios with partial causal knowledge. We evaluate these approaches on controlled
benchmarks and six CSuite
datasets, assessing structural fidelity, distributional quality, and Average Treatment Effect (ATE) preservation. Across most settings, DAG-aware conditioning improves the quality and stability of synthetic data relative to vanilla TabPFN. Under partial causal knowledge, the oracle partially directed acyclic graph (oracle-PDAG), which orients only the edges into the
colliders, shows moderate gains, while the benefit of a Completed Partially Directed Acyclic Graph (CPDAG) discovered from data depends on how well the causal structure is recovered. These results indicate that reliable causal structure, even partial, can be injected into TabPFN at inference time, without parameter updates, to improve synthetic data quality.
Read the full abstract ▾
All authors Davide Tugnoli, Andrea De Lorenzo, Marco Virgolin, Giovanni Cinà
Machine Learning · Cross-listed from stat.ML
Bayesian Hierarchical Models and the Maximum Entropy Principle
Brendon J. Brewer
Bayesian hierarchical models are frequently used in practical data analysis contexts. One interpretation of these models is that they provide an indirect way of assigning a prior for unknown parameters, through the introduction of hyperparameters. The resulting marginal prior for the parameters (integrating over the hyperparameters) is usually dependent, so that learning one parameter provides some information about the others. In this contribution, I will demonstrate that, when the prior given the hyperparameters is a canonical distribution (a maximum
entropy distribution with moment constraints), the dependent marginal prior also has a maximum
entropy property, with a different constraint. This constraint is on the marginal distribution of some function of the unknown quantities. The results shed light on what information is actually being assumed when we assign a hierarchical model.
Read the full abstract ▾
Machine Learning
GeMPO: Generalized Measure Matching for Online Diffusion Reinforcement Learning
Haitong Ma, Chenxiao Gao, Tianyi Chen et al. (+2 more)
A commonly used family of RL
algorithms for
diffusion policies conducts softmax reweighting over samples from the behavior policy, which often induces an overgreedy policy and fails to utilize feedback from negative samples. In this work, we introduce GeMPO, a simple and unified framework that generalizes reweighting scheme in
diffusion RL from softmax to general monotonic functions. GeMPO revisits
diffusion RL via a measure matching perspective: First, we construct a virtual target policy measure via solving a regularized policy optimization objective; Second, we minimize the divergence between the current policy and this target measure through reweighted flow matching. This formulation offers two key advantages: i) It extends weight design beyond traditional exponential reweighting, allowing it to be tailored to diverse reward landscapes; and ii) by relaxing the non-negativity constraint on the target measure, our framework provides a principled justification for negative reweighting. We provide interpretations of how negative reweighting actively repels the policy from suboptimal actions and thus facilitates exploration. Extensive empirical evaluations demonstrate that GeMPO achieves competitive or superior performance by leveraging these flexible weighting schemes, and we provide practical guidelines for selecting reweighting methods in practice.
Read the full abstract ▾
All authors Haitong Ma, Chenxiao Gao, Tianyi Chen, Na Li, Bo Dai
Machine Learning · Cross-listed from cs.NE
Intrinsic Numerical Robustness and Fault Tolerance in a Neuromorphic Algorithm for Scientific Computing
Bradley H. Theilman and James B. Aimone
The potential for neuromorphic computing to provide intrinsic fault tolerance has long been speculated, but the
brain's robustness in neuromorphic applications has yet to be demonstrated. Here, we show that a previously described, natively spiking neuromorphic
algorithm for solving partial differential equations is intrinsically tolerant to structural perturbations in the form of ablated
neurons and dropped spikes. The tolerance band for these perturbations is large: we find that as many as 32 percent of the
neurons and up to 90 percent of the spikes may be entirely dropped before a significant degradation in the accuracy results. Furthermore, this robustness is tunable through structural hyperparameters. This work demonstrates that the specific
brain-like inspiration behind the
algorithm contributes to a significant degree of robustness expected from
brain-like neuromorphic
algorithms.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
One Adapter for All: Towards Unified Representation in Step-Imbalanced Class-Incremental Learning
Xiaoyan Zhang and Jiangpeng He
Class-incremental learning (CIL) aims to acquire new classes over time while retaining prior knowledge, yet most setups and methods assume balanced task streams. In practice, the number of classes per task often varies significantly. We refer to this as step imbalance, where large tasks that contain more classes dominate learning and small tasks inject unstable updates. Existing CIL methods assume balanced tasks and therefore treat all tasks uniformly, producing imbalanced updates that degrade overall learning performance. To address this challenge, we propose One-A, a unified and imbalance-aware framework that incrementally merges task updates into a single adapter, maintaining constant inference cost. One-A performs asymmetric subspace
alignment to preserve dominant subspaces learned from large tasks while constraining low-information updates within them. An information-adaptive weighting balances the contribution between base and new adapters, and a directional gating mechanism selectively fuses updates along each singular direction, maintaining stability in head directions and plasticity in tail ones. Across multiple
benchmarks and step-imbalanced streams, One-A achieves competitive accuracy with significantly low inference overhead, showing that a single, asymmetrically fused adapter can remain both adaptive to dynamic task sizes and efficient at deployment.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Why Does It Look There? Structured Explanations for Image Classification
Jiarui Li, Zixiang Yin, Samuel J Landry et al. (+2 more)
Deep learning models achieve remarkable predictive performance, yet their black-box nature limits transparency and trustworthiness. Although numerous explainable artificial intelligence (XAI) methods have been proposed, they primarily provide saliency maps or concepts (i.e., unstructured interpretability). Existing approaches often rely on auxiliary models (\eg, GPT, CLIP) to describe model behavior, thereby compromising faithfulness to the original models. We propose Interpretability to Explainability (I2X), a framework that builds structured explanations directly from unstructured interpretability by quantifying progress at selected checkpoints during training using prototypes extracted from post-hoc XAI methods (e.g., GradCAM). I2X answers the question of "why does it look there" by providing a structured view of both intra- and inter-class decision making during training. Experiments on MNIST and CIFAR10 demonstrate effectiveness of I2X to reveal prototype-based inference process of various image classification models. Moreover, we demonstrate that I2X can be used to improve predictions across different model architectures and
datasets: we can identify uncertain prototypes recognized by I2X and then use targeted perturbation of samples that allows fine-tuning to ultimately improve accuracy. Thus, I2X not only faithfully explains model behavior but also provides a practical approach to guide optimization toward desired targets.
Read the full abstract ▾
All authors Jiarui Li, Zixiang Yin, Samuel J Landry, Zhengming Ding, Ramgopal R. Mettu
Machine Learning · Cross-listed from math.OC
A Trust-Region Interior-Point Stochastic Sequential Quadratic Programming Method
Yuchen Fang, Jihun Kim, Sen Na et al. (+2 more)
In this paper, we propose a trust-region interior-point
stochastic sequential quadratic programming (TR-IP-SSQP) method for solving optimization problems with a
stochastic objective and deterministic nonlinear equality and inequality constraints. In this setting, exact evaluations of the objective function and its gradient are unavailable, but their
stochastic estimates can be constructed. In particular, at each iteration our method builds
stochastic oracles, which estimate the objective value and gradient to satisfy proper adaptive accuracy conditions with a fixed probability. To handle inequality constraints, we adopt an interior-point method (IPM), in which the barrier parameter follows a prescribed decaying sequence. Under standard assumptions, we establish global almost-sure convergence of the proposed method to first-order stationary points. We implement the method on a subset of problems from the CUTEst test set, as well as on logistic regression problems, to demonstrate its practical performance.
Read the full abstract ▾
All authors Yuchen Fang, Jihun Kim, Sen Na, James Demmel, Javad Lavaei
Machine Learning
Rethinking the Harmonic Loss via Non-Euclidean Distance Layers
Maxwell Miller-Golub, Collin Coil, Kamil Faber et al. (+4 more)
Cross-
entropy loss has long been the standard choice for training deep neural networks, yet it suffers from interpretability limitations, unbounded weight growth, and inefficiencies that can contribute to costly training dynamics. The harmonic loss is a distance-based alternative grounded in Euclidean geometry that improves interpretability and mitigates phenomena such as grokking, or delayed generalization on the test set. However, the study of harmonic loss remains narrow: only Euclidean distance is explored, and no systematic evaluation of computational efficiency or sustainability was conducted. We extend harmonic loss by systematically investigating a broad spectrum of distance metrics as replacements for the Euclidean distance. We comprehensively evaluate distance-tailored harmonic losses on both
vision backbones and large
language models. Our analysis is framed around a three-way evaluation of model performance, interpretability, and sustainability. On
vision tasks, cosine distances provide the most favorable trade-off, consistently improving accuracy while lowering carbon emissions, whereas Bray-Curtis and Mahalanobis further enhance interpretability at varying efficiency costs. On
language models, cosine-based harmonic losses improve gradient and learning stability, strengthen representation structure, and reduce emissions relative to cross-
entropy and Euclidean heads. Our code is available at: https://anonymous.4open.science/r/rethinking-harmonic-loss-5BAB/.
Read the full abstract ▾
All authors Maxwell Miller-Golub, Collin Coil, Kamil Faber, Marcin Pietron, Panpan Zheng, Pasquale Minervini, Roberto Corizzo
Machine Learning · Cross-listed from stat.ML
A Diffusion Analysis of Policy Gradient for Stochastic Bandits
Tor Lattimore
We study a continuous-time
diffusion approximation of policy gradient for $k$-armed
stochastic bandits. We prove that with a learning rate $\eta = O(\Delta^2/\log(n))$ the regret is $O(k \log(k) \log(n) / \eta)$ where $n$ is the horizon and $\Delta$ the minimum gap. Moreover, we construct an instance with only logarithmically many arms for which the regret is linear unless $\eta = O(\Delta^2)$.
Read the full abstract ▾
Machine Learning · Cross-listed from q-bio.PE
SDSR: A Spectral Divide-and-Conquer Approach for Species Tree Reconstruction
Ortal Reshef, Ofer Glassman, Or Zuk et al. (+3 more)
Recovering a tree that represents the evolutionary history of a group of species is a key task in phylogenetics. Performing this task using sequence data from multiple genetic markers poses two key challenges. The first is the discordance between the evolutionary history of individual genes and that of the species. The second challenge is computational, as contemporary studies involve thousands of species. Here we present SDSR, a scalable divide-and-conquer approach for species tree reconstruction based on spectral graph theory. The
algorithm recursively partitions the species into subsets until their sizes are below a given threshold. The trees of these subsets are reconstructed by a user-chosen species tree
algorithm. Finally, these subtrees are merged to form the full tree. On the theoretical front, we derive recovery guarantees for SDSR, under the multispecies coalescent (MSC) model. We also perform a runtime complexity analysis. We show that SDSR, when combined with a species tree reconstruction
algorithm as a subroutine, yields substantial runtime savings as compared to applying the same
algorithm on the full data. Empirically, we evaluate SDSR on synthetic
benchmark datasets with incomplete lineage sorting and horizontal gene transfer. In accordance with our theoretical analysis, the simulations show that combining SDSR with common species tree methods, such as CA-ML or ASTRAL, yields up to 10-fold faster runtimes. In addition, SDSR achieves a comparable tree reconstruction accuracy to that obtained by applying these methods on the full data.
Read the full abstract ▾
All authors Ortal Reshef, Ofer Glassman, Or Zuk, Yariv Aizenbud, Boaz Nadler, Ariel Jaffe
Machine Learning · Cross-listed from cs.CV
FusionNet: a frame interpolation network for 4D heart models
Chujie Chang, Shoko Miyauchi, Ken'ichi Morooka et al. (+2 more)
Cardiac
magnetic resonance (CMR) imaging is widely used to visualise cardiac motion and diagnose heart disease. However, standard CMR imaging requires patients to lie still in a confined space inside a loud machine for 40-60 min, which increases patient discomfort. In addition, shorter scan times decrease either or both the temporal and spatial resolutions of cardiac motion, and thus, the diagnostic accuracy of the procedure. Of these, we focus on reduced temporal resolution and propose a neural network called FusionNet to obtain four-dimensional (4D) cardiac motion with high temporal resolution from CMR images captured in a short period of time. The model estimates intermediate 3D heart shapes based on adjacent shapes. The results of an experimental evaluation of the proposed FusionNet model showed that it achieved a performance of over 0.897 in terms of the Dice coefficient, confirming that it can recover shapes more precisely than existing methods. This code is available at: https://github.com/smiyauchi199/FusionNet.git
Read the full abstract ▾
All authors Chujie Chang, Shoko Miyauchi, Ken'ichi Morooka, Ryo Kurazume, Oscar Martinez Mozos
Machine Learning · Cross-listed from cond-mat.mtrl-sci
Flexible Cutoff Learning: Optimizing Machine Learning Potentials After Training
Rick Oerder and Jan Hamaekers
We introduce Flexible Cutoff Learning (FCL), a method for training machine learning interatomic potentials (MLIPs) whose cutoff radii can be adjusted after training. Unlike conventional MLIPs that fix the cutoff radius during training, FCL models are trained by randomly sampling cutoff radii independently for each atom. The resulting model can then be deployed with different per-atom cutoff radii depending on the application, enabling application-specific optimization of the accuracy-cost tradeoff. Using a differentiable cost model, these per-atom cutoffs can be optimized for specific target systems after training. We demonstrate FCL with a modified MACE architecture trained on the MAD
dataset. For a subset featuring molecular
crystals, optimized per-atom cutoffs reduce computational cost by more than 60% while increasing force errors by less than 1%. These results show that FCL enables training of a single general-purpose MLIP that can be adapted to diverse applications through post-training cutoff optimization, eliminating the need for retraining.
Read the full abstract ▾
Machine Learning · Cross-listed from q-fin.ST
Variance-Corrected Multi-Asset Equity Simulation with Hybrid Hidden Markov Marginals
Abdulrahman Alswaidan and Jeffrey D. Varner
Synthetic multi-asset equity data must reproduce each asset's return distribution and its relationship with the
market. Reusing a generator fitted to full asset returns creates a problem: adding its draws to a
market factor counts
market variance twice. We derived a correction that centers and rescales each draw over a fixed horizon before adding the
market factor, allowing reuse without fitting a second generator to regression residuals. We tested the correction on 423 non-
market assets in a 424-asset United States equity and exchange-traded-fund universe, using hidden Markov generators with heavy-tailed emissions. The corrected paths retained heavy tails and recovered the calibrated
market loadings with low error. On 416 complete asset histories held out from 2025, the correction improved the mean Kolmogorov-Smirnov pass rate over naive composition and brought the median ratio of synthetic to observed variance close to one. Its one-day left-tail 99% Value-at-Risk exceedance rate, using thresholds pooled across separate simulated paths, was 1.12%, compared with 1.16% for the residual-fit hidden Markov comparator and a nominal rate of 1%. We also tested a jump-duration mechanism that extended visits to extreme-return states and improved
volatility clustering for the broad-
market exchange-traded fund with ticker SPY. The multi-asset correction remained effective with jumps enabled, but transferring the SPY jump settings improved temporal fit in training and worsened it in the 2025 holdout. The method provides a way to reuse fitted asset generators for daily-fit comparisons. Its zero-sum residual constraint removes terminal residual uncertainty, and unmodeled dependence among asset-specific residuals can lead to overstated rebalancing returns. Code, cached inputs, result summaries, and instructions for fitting the per-asset models accompany the paper.
Read the full abstract ▾
Machine Learning
Actor-Accelerated Policy Dual Averaging for Reinforcement Learning in Continuous Action Spaces
Ji Gao, Caleb Ju, Guanghui Lan et al. (+1 more)
Policy Dual Averaging (PDA) offers a principled Policy Mirror Descent (PMD) framework that more naturally admits value function approximation than standard PMD, enabling the use of approximate advantage (or Q-) functions while retaining strong convergence guarantees. However, applying PDA in continuous state and action spaces remains computationally challenging, since action selection involves solving an optimization sub-problem at each decision step. In this paper, we propose actor-accelerated PDA, which uses a learned policy network to approximate the solution of the optimization sub-problems, yielding faster runtimes while maintaining convergence guarantees. We provide a theoretical analysis that quantifies how actor approximation error impacts the convergence of PDA under suitable assumptions. We then evaluate its performance on several
benchmarks in robotics, control, and operations research problems. Actor-accelerated PDA achieves superior performance compared to popular on-policy baselines such as Proximal Policy Optimization (PPO). Overall, our results bridge the gap between the theoretical advantages of PDA and its practical deployment in continuous-action problems with function approximation.
Read the full abstract ▾
All authors Ji Gao, Caleb Ju, Guanghui Lan, Zhaohui Tong
Machine Learning · Cross-listed from cs.CL
Adaptive Activation Cancellation for Hallucination Mitigation in Large Language Models
Eric Yocam, Varghese Vaidyan, Gurcan Comert et al. (+3 more)
Large
Language Models frequently generate fluent but factually incorrect text. We propose Adaptive Activation Cancellation (AAC), a real-time inference-time framework that treats
hallucination-associated neural activations as structured interference within the
transformer residual stream, drawing an explicit analogy to classical adaptive noise cancellation from signal processing. The framework identifies
Hallucination Nodes (H-Nodes) via layer-wise linear probing and suppresses them using a confidence-weighted forward hook during auto-regressive generation – requiring no external knowledge, no fine-tuning, and no additional inference passes. Evaluated across OPT-125M, Phi-3-mini, and LLaMA 3-8B on TruthfulQA and HaluEval, the real-time hook is the only intervention that consistently improves downstream accuracy on all three scales. Critically, the method is strictly
surgical: WikiText-103 perplexity and MMLU
reasoning accuracy are preserved at exactly 0.0% degradation across all three model scales, a property that distinguishes AAC from interventions that trade fluency or general capability for factual improvement. On the LLaMA 3-8B scale, the hook additionally yields positive generation-level gains (MC1 +0.04; MC2 +0.003;
Token-F1 +0.003) while achieving probe-space selectivity 5.94x - 3.5x higher than the ITI baseline – demonstrating that targeted
neuron-level suppression can simultaneously improve factual accuracy and preserve model capability.
Read the full abstract ▾
All authors Eric Yocam, Varghese Vaidyan, Gurcan Comert, Paris Kalathas, Yong Wang, Judith L. Mwakalonge
Machine Learning · Cross-listed from eess.IV
ARCHE: Autoregressive Residual Compression with Hyperprior and Excitation
Sofia Iliopoulou, Dimitris Ampeliotis and Athanassios Skodras
Recent progress in learning-based image compression has demonstrated that end-to-end optimization can substantially outperform traditional codecs by jointly learning compact latent representations and probabilistic
entropy models. However, many existing approaches achieve high rate-distortion efficiency at the expense of increased computational cost and limited parallelism. This paper presents ARCHE - Autoregressive Residual Compression with Hyperprior and Excitation, an end-to-end learned image compression framework that balances modeling accuracy and computational efficiency. The proposed architecture unifies hierarchical, spatial, and channel-based priors within a single probabilistic framework, capturing both global and local dependencies in the latent representation of the image, while employing adaptive feature recalibration and residual refinement to enhance latent representation quality. Without relying on recurrent or
transformer-based components, ARCHE attains state-of-the-art rate-distortion efficiency: it reduces the BD-Rate by approximately 48% relative to the commonly used
benchmark model of Balle et al., 30% relative to the channel-wise autoregressive model of Minnen & Singh and 5% against the VVC Intra codec on the Kodak
benchmark dataset. The framework maintains computational efficiency with 95M parameters and 222ms running time per image. Visual comparisons confirm sharper textures and improved color fidelity, particularly at lower bit rates, demonstrating that accurate
entropy modeling can be achieved through efficient convolutional designs suitable for practical deployment.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Stabilizing Bandits using Regularization: Precise Regret and A Quantitative Central Limit Theorem
Budhaditya Halder, Ishan Sengupta, Koustav Chowdhury et al. (+2 more)
Statistical inference with bandit data presents fundamental challenges owing to adaptive sampling, which violates the independence assumptions underlying classical asymptotic theory. Recent work has identified stability~\citep{laiwei82} as a sufficient condition for valid inference under adaptivity. This paper first provides a refined stability condition, stated in terms of the iterates of an online
algorithm, and shows that a large class of regularized
stochastic-mirror-descent-style
algorithms satisfy it. This refined condition allows us to strengthen the asymptotic results of~\citet{laiwei82} in several ways. First, we derive a non-asymptotic Berry–Esseen bound for the empirical reward estimates under adaptive sampling. Second, we derive matching non-asymptotic upper and lower bounds on the regret of the proposed
algorithm, yielding a precise characterization of its regret. Third, we show that these regularized
algorithms preserve asymptotic normality and valid inference under a prescribed level of
adversarial corruption. Finally, we show that regularization is necessary rather than incidental: Lai–Wei stability is incompatible with the optimal $O(\sqrt{T})$ regret rate – the rate attained by unregularized
algorithms such as EXP3 – so that a controlled, polylogarithmic inflation in regret is the price of valid inference.
Read the full abstract ▾
All authors Budhaditya Halder, Ishan Sengupta, Koustav Chowdhury, Samya Praharaj, Koulik Khamaru
Machine Learning
DT-BEHRT: Disease Trajectory-aware Transformer for Interpretable Patient Representation Learning
Deyi Li, Zijun Yao, Qi Xu et al. (+4 more)
The growing adoption of electronic health record (EHR) systems has provided unprecedented opportunities for predictive modeling to guide clinical decision making. Structured EHRs contain longitudinal observations of patients across hospital visits, where each visit is represented by a set of medical codes. While sequence-based, graph-based, and graph-enhanced sequence approaches have been developed to capture rich code interactions over time or within the same visits, they often overlook the inherent heterogeneous roles of medical codes arising from distinct clinical characteristics and contexts. To this end, in this study we propose the Disease Trajectory-aware
Transformer for EHR (DT-BEHRT), a graph-enhanced sequential architecture that disentangles disease trajectories by explicitly modeling diagnosis-centric interactions within organ systems and capturing asynchronous progression patterns. To further enhance the representation robustness, we design a tailored pre-training methodology that combines trajectory-level code masking with ontology-informed ancestor prediction, promoting semantic
alignment across multiple modeling modules. Extensive experiments on multiple
benchmark datasets demonstrate that DT-BEHRT achieves strong predictive performance and provides interpretable patient representations that align with clinicians' disease-centered
reasoning. The source code is publicly accessible at https://github.com/GatorAIM/DT-BEHRT.git.
Read the full abstract ▾
All authors Deyi Li, Zijun Yao, Qi Xu, Muxuan Liang, Lingyao Li, Zijian Xu, Mei Liu