Machine Learning
SCOPE-OPSD: Fisher-Conditioned Privileged Subspaces for On-Policy Self-Distillation
Yunmeng Chen, Kunyu Wang, Peihan Li et al. (+8 more)
On-policy self-
distillation (OPSD) scores student-generated prefixes with a solution-conditioned self-teacher, yet transfers supervision only through next-
token probabilities. We ask whether the aligned final-layer discrepancy offers a useful second channel, and how to test that channel without confusing its geometry with auxiliary strength. SCOPE-OPSD projects the privileged teacher-student residual onto a frozen rank-64 factor estimated from residual covariance and
language-model-head Fisher sensitivity. It reuses the forwards already required by OPSD and adds neither rollouts nor inference-time modules. A matched Random control preserves the structured factor's rank and nonzero spectrum and uses per-arm gradient-RMS calibration, isolating the effect of the data-dependent orientation. Across the complete 25/50/75/100-step trajectories for Qwen3-1.7B, 4B, and 8B, Structured is never below Pure OPSD, with strict gains in 11 of the 12 model-checkpoint combinations and an exact tie at 4B step 25. Structured also exceeds matched Random in 10 of the 12 combinations. At step 75 on Qwen3-1.7B, Structured exceeds matched Random by 1.39 Macro Avg@12 points in each of two independent training reruns. A cross-fitted diagnostic also shows 4.40 times greater held-out privileged-gap capture than the matched random orientation. The results support a compact, Fisher-conditioned privileged subspace for short-budget OPSD.
Read the full abstract ▾
All authors Yunmeng Chen, Kunyu Wang, Peihan Li, Yi Wang, Shuyin Xia, Yi Liu, Xinyong Cheng, Dehui Wang, Xiangyong Zhai, Yanxing Liu, Song Liu
Machine Learning
TokenMapper: A Step Toward Interoperable Speech Token Translation
Tal Kozakov, Tal Rosenwein and Eliya Nachmani
Neural audio codecs discretize
speech into
token sequences, but the resulting
token spaces differ in vocabulary and codebook structure, preventing direct communication across models. This limitation affects applications such as conversational voice
agents and
speech to
speech translation systems where multiple
speech models must interact. As a result, transferring information between
speech systems typically requires decoding to waveform audio and re-encoding with a second tokenizer, increasing latency and introducing potential information loss. To address these limitations, we present TokenMapper, a direction aware framework for direct
token to
token translation between heterogeneous
speech tokenizers in the discrete domain. TokenMapper supports structurally mismatched
token spaces, including mappings between single codebook and multi codebook representations, under a shared effective
token rate. Experiments on GLM-4-Voice, MiMi and DualCodec show consistent cross model performance. Specifically, translation WER approaches native reconstructions within 2.5-6.8% absolute WER, human MOS for TokenMapper outputs ranges from 2.29 to 4.39, following the same direction level trends as UTMOS and end to end latency is reduced by 4.8-94.5% relative to waveform bridging, reaching up to 972 ms per utterance. These results provide a practical step toward cross model
speech token interoperability without intermediate waveform reconstruction.
Read the full abstract ▾
Machine Learning
Quality-Constrained Routing over a Fixed Pool of Quantized Mixture-of-Experts Instances
Zhenghong Huang, Hongfan Wu and Jiheng Zhang
Quantized Mixture-of-Experts (MoE) services can hold several pre-materialized instances of one base model, but quantization damage varies sharply across requests and bitwidths. Because instance materialization and replica counts consume
memory and require slow reconfiguration, we treat them as upstream provisioning decisions and study routing within a fixed resident pool. Within this fixed-pool boundary, we route each request to maximize modeled throughput under a class-level expected quality-degradation budget and measured instance capacities. To predict this request-specific risk, we introduce FWP (Fragility-Weighted Perplexity), computed from
prompt tokens on a reference-instance prefill and calibrated to candidate-instance degradation. Underlying FWP is an exact two-expert affinity–fragility decomposition and a conditional multi-layer top-$k$ expansion whose bias, interaction, route-change, separability, and higher-order terms remain explicit. Using these calibrated risks, a window-level linear program yields a signed reduced-reward score that is KKT-consistent with the LP optimum under optimal prices and primal-feasible tie allocation. On 88 extended Qwen
prompts, complete W2, W3, and W4 instances quantizing all 6,144 expert blocks incur mean $\Delta$NLL of $0.9437$, $0.1832$, and $0.0513$. Under the same population and $\tau=0.1513$, FWP allocation reaches a $1.284\times$ offline model-based multiplier versus $1.253\times$ for request-agnostic mixing and $1.000\times$ for static W4, an incremental $2.5\%$ relative FWP gain.
Read the full abstract ▾
Machine Learning
$\text{GSF-}\chi$: Global Stereochemical Fields for Chiral Graph Transformers
Jiaqing Xie, Yuxin Wang and Xipeng Qiu
Enantiomers share atoms, bonds, and pairwise distances yet can behave differently in
chiral environments, so molecular encoders must respect atom relabelings and proper rotations without becoming blind to reflection. We introduce GSF-$\chi$, a graph
transformer in which stereogenic units modulate all pairwise interactions rather than single out one atom as special. Each central or axial stereogenic unit creates a reflection-even phase field over all atoms, a handedness pseudoscalar $\chi$ sets the direction of a relative rotation on latent query–key blocks, giving a
Chiral-RoPE that reflection inverts rather than leaves fixed. A $C_2$ projection separates mirror-even ECD peak counts and positions from mirror-odd peak signs. We prove the operator's even–odd decomposition and its annotation-inversion, permutation, and unit-order identities under explicit canonical-role conditions; property tests and a coordinate-reflection audit verify the laws end to end. GSF-$\chi$ leads every central-ECD output and improves axial Rotation and Symbol by $12.6\%$ and $7.9\%$ over the strongest baseline. Equal-budget controls attribute the Rotation advantage to global signed support rather than parameter or edge count; the $C_2$ projection yields exact enantiomer-pair consistency at a small raw-accuracy cost under complete supervision and becomes predictive when mirror supervision is scarce.
Read the full abstract ▾
Machine Learning
Temporal Recurrence Favors Fewer Layers
Ivan Anokhin, Johan Obando-Ceron, Irina Rish et al. (+1 more)
In streaming tasks, recurrent models can carry latent computation across time, allowing each update to build on representations produced earlier. This raises a basic question: once temporal recurrence provides sequential computation across steps, how much depth is still needed within each step? Prior work has shown that recurrence can make shallow models competitive. We instead study this question as a compute-allocation problem, varying within-step depth, expert width, and the number of parallel experts per layer across several compute budgets. For each budget, we compare the best observed recurrent and non-recurrent allocations and the performance they achieve under approximately matched per-step computation. Across Sokoban and autoregressive FineWeb
language modeling, we find that temporal recurrence shifts the best observed compute allocation toward substantially fewer layers, with comparable or better performance.
Read the full abstract ▾
All authors Ivan Anokhin, Johan Obando-Ceron, Irina Rish, Sebastian Risi
Machine Learning · Cross-listed from math.OC
Learning-Augmented Optimization for Strategic Two-Echelon Spare Parts Network Design
Donato Maragno, Marco Caserta, Alberto Sinigaglia et al. (+3 more)
We study the strategic design of a two-echelon spare-parts inventory network where evaluating each candidate
topology requires an expensive inventory optimization model. The design partitions hundreds of sites into feasible clusters and selects a central replenishment site for each cluster to reduce costs while maintaining service levels. Because the optimizer favors candidates with high predicted savings, it can exploit optimistic surrogate errors. We develop a conservative framework combining a graph neural network ensemble, variable neighborhood search, and set-partitioning recombination. The surrogate is trained on exact cluster evaluations, while a lower
quantile of ensemble-predicted savings guides the search to limit optimism. Clusters found during the search are recombined through set partitioning using surrogate-based objective coefficients. The resulting network is evaluated with the exact inventory model, and only this evaluation is used to report performance. In a case study of 246 fulfillment centers in Amazon's North American network, the framework improves combined savings by 30.5% over an optimization baseline based entirely on exact cluster evaluations, while maintaining approximately 99.8% service across six independent replications. Under equal computational budgets, graph-surrogate-guided search achieves higher mean exact savings than a tabular alternative under both scoring schemes. Conservative scoring improves mean savings for both surrogate classes and reduces the share of final-network clusters overestimated by the graph surrogate from 68% to 28%. Predictive and ranking accuracy deteriorate among search-generated candidates with high surrogate scores, indicating that random holdout performance can incompletely characterize surrogate quality during optimization.
Read the full abstract ▾
All authors Donato Maragno, Marco Caserta, Alberto Sinigaglia, Komlanvi Ametana, David Corredor Montenegro, Luca D'Angelo
Machine Learning · Cross-listed from stat.ML
A Splitting Method for SDE Terminal-Law Estimation
Rushil Gupta and Sandeep Juneja
In many settings involving
stochastic differential equations (SDEs), including
diffusion based generative AI, our aim is to accurately generate samples from a distribution. Typically, this is done by generating i.i.d. samples of the SDE. Given a fixed simulation budget, a reasonable way to gain efficiency may be to instead generate a tree of paths through appropriately split partial paths. This suggests improved performance, but one worries about the injected dependence. In this paper, we study this issue comprehensively. We develop a general method to construct the splitting design and analyze its performance for a broad class of error measures, giving insight into the elegant underlying structure in the problem. For the Kolmogorov-Smirnov (KS) distance and maximum mean discrepancy (MMD), we identify the limiting errors of the associated empirical distributions as the simulation budget increases to infinity. We characterize a splitting strategy motivated by a corresponding asymptotic optimization problem. Overall, we observe a 12-26% improvement in mean KS over i.i.d. samples in many settings. On CIFAR-10 and FFHQ, splitting lowers the MMD by 17-26% relative to i.i.d. samples.
Read the full abstract ▾
Machine Learning
A Differentially Private Federated Proximal Optimization Framework for Customer Churn Prediction in Heterogeneous Federated Telecom Networks
Joydeb Kumar Sana, Subrata Chakraborty and M M Manjurul Islam
Customer churn is one of the major issues in the telecommunication industry. To predict customer churn, conventional centralized machine learning approaches have been widely used. This centralized approach requires customer data to be stored in a central repository, which raises
privacy concerns and may violate data protection regulations.
Federated learning addresses this problem by allowing multiple telecom operators to collaboratively train a global model without transferring their raw customer data. However, real-world customer data are often heterogeneous (non-IID), which may negatively affect the performance of standard
federated learning. Trained models can also suffer from
privacy attacks. To address those issues, we propose a Differentially Private (DP) based
Federated Proximal optimization (FedProx) framework. All experiments were performed on two publicly available telecom churn
datasets. We trained
Federated Averaging (FedAvg), DP-FedAvg, FedProx, and the proposed DP-FedProx framework. For baseline comparison, we also used several centralized and local models. To evaluate the models, we employed seven widely used evaluation metrics. The experimental results show that the FedProx based models consistently outperform the FedAvg based models. Compared with the best centralized model, the proposed DP-FedProx framework achieves competitive prediction performance with only a small reduction in accuracy while providing
privacy guarantees. To explain our model, we conducted SHAP analysis which shows that DP-FedProx method priorities revenue group features. These results indicate that the proposed DP-FedProx framework provides a practical balance between prediction performance and data
privacy protection.
Read the full abstract ▾
Machine Learning · Cross-listed from math.OC
Linear Exponential Quadratic Gaussian Covariance Steering
Chiran B. Cherian, Yasemin Isik and Abhishek Halder
We formulate and analyze the linear exponential quadratic Gaussian (LEQG) covariance steering problem in continuous time over a given deadline (finite time horizon). The solution for this problem can be seen as a risk-sensitive Schrödinger bridge between Gaussian endpoints in the linear quadratic setting. Unlike the risk-neutral case, the LEQG covariance steering controller–still a linear state feedback–can no longer be written in closed form. We show that the optimal controller is parameterized by a symmetric
matrix solving an algebraic equation that encodes the implicit dependence on the risk-sensitivity parameter. We explain how the structure of this optimal controller significantly generalizes the existing results for the risk-neutral case. Building on these results, for the matched noise and input channel case, we prove the existence-uniqueness of solution for the LEQG covariance steering problem in the neighborhood of the known risk-neutral optimal solution. We give an illustrative numerical example.
Read the full abstract ▾
Machine Learning
SAGE-Loop: Reliable Closed-Loop LLM-Driven AutoML with Trial-and-Correction and Adaptive Ensembling
Junquan Gu, Shibo Cui, Xiangfeng Luo et al. (+1 more)
Automated machine learning (AutoML) is reshaping data-driven science and industrial practice, and as large
language models are introduced into AutoML, pipeline reliability becomes as important as automation efficiency. However, existing AutoML still struggles to realize instant feedback and adaptive optimization during execution, so once a run drifts into a suboptimal or failed state, it lacks a process-level correction mechanism. The fundamental pathology lies in its one-way pipeline: intermediate failures are typically terminated or bypassed, while fixed paradigms often strengthen model generation but leave ensemble decisions static, weakening both execution reliability and the controlled use of structural diversity. This indicates that LLM-driven AutoML needs a closed-loop ability for trial-correction-improvement together with evidence-based use of model diversity. To this end, we propose SAGE-Loop, a reliable closed-loop, self-adaptive, LLM-driven AutoML framework that performs multi-round generation and validation for trial-and-repair, and adaptively selects ensemble strategies in both supervised and unsupervised tasks, thereby unifying how to generate with how to use models. Across 20 public
datasets, SAGE-Loop consistently improves performance and stability on classification, regression, and clustering tasks. Additional results further show its ability to recover from execution failures and maintain robust pipeline behavior.
Read the full abstract ▾
All authors Junquan Gu, Shibo Cui, Xiangfeng Luo, Hang Yu
Machine Learning · Cross-listed from cs.CV
Bridging Vision Foundation Model Priors with CLIP for Spatial-aware Few-shot Anomaly Detection in Medical Images
Juzheng Miao, Yuchen Yuan, Cheng Chen et al. (+1 more)
Vision-
Language Models such as CLIP enable effective few-shot medical anomaly detection (AD) via strong image-text semantic
alignment. However, their globally contrastive pretraining lacks explicit spatial supervision, limiting precise lesion localization. In contrast,
Vision Foundation Models (VFMs) such as DINO learn spatially coherent patch representations via self-
distillation and local-to-global consistency, better capturing fine-grained anatomical structures. Leveraging this complementarity, we propose Spatial-FAD, a spatial-aware few-shot medical AD framework that improves lesion localization by combining VFM spatial priors with CLIP semantics. Specifically, we introduce a VFM-enhanced adapter that injects a structural affinity prior derived from DINO into CLIP features. This structure-guided refinement encourages visual
embeddings to better adhere to lesion boundaries while maintaining semantic
alignment. To address the loss of spatial detail from patchification and the limited input resolution of CLIP, we adopt a sliding-window aggregation strategy. This generates high-resolution, spatially dense
embeddings to further enhance localization granularity. Moreover, we introduce a prototype-enhanced support
memory scheme to efficiently exploit the few-shot support set. This module stores compact prototypes for normal and abnormal patterns, reducing
memory costs while boosting performance by fusing patch-to-prototype and image-text similarities. Extensive experiments on three
benchmark datasets, including Liver CT, Retinal OCT, and
Brain MRI, demonstrate that Spatial-FAD significantly outperforms state-of-the-art methods, especially in lesion
segmentation. Notably, in the 4-shot scenario, our method achieves an average improvement of over 11.4% in Dice score and 1.8% in AUC. Code is available at: https://github.com/JuzhengMiao/Spatial-FAD.
Read the full abstract ▾
All authors Juzheng Miao, Yuchen Yuan, Cheng Chen, Pheng-Ann Heng
Machine Learning
3D Digital Twin Visualization of Multiclass GRF-Based Gait Disorder Classification
Nayoung Son and Minwoo Shin
Automated gait analysis requires accurate classification and interpretable outputs. We propose an integrated framework for classifying healthy gait and multiple musculoskeletal impairment groups using bilateral ground reaction force (GRF) and center-of-pressure (COP) signals. The signals were normalized over the stance phase and standardized using training-set statistics. The model achieved a validation accuracy of 99.00% and a test accuracy of 90.07% under a session-level split. Class-specific $\epsilon$-LRP identified positive and negative contributions across both sides, multiple signal components, and different stance phases. Separately, the processed GRF signals and model predictions were synchronized within a Blender-based 3D visualization, enabling sample-level inspection of gait trials and classification results. The proposed framework integrates classification, explainability, and 3D visualization to improve model transparency. The source code is available in the following repository: https://github.com/nyoico/grf-gait-3d-visualization.git
Read the full abstract ▾
Machine Learning · Cross-listed from cs.RO
IMPLY: Physically Anchored Consistency for World-Model Rollouts
Aman Mehta and Riya Baviskar
A world model asked what happens if an object is pushed at several speeds produces several futures. If the model has the object in mind, those futures agree about it: each implies the same mass and friction. The consistency checks now used to vet world-action models ask whether a model's futures agree with each other, and none of them knows any physics. We show that this is not enough, and what to do instead. IMPLY reads the physics each rollout implies by inverting a simulator and scores a set of rollouts by how well one object explains all of them, anchored to two calibration pushes the model has observed. In a controlled setting, self-consistency gives a perfect score to a model that ignores the object and always predicts a typical push; anchoring exposes it (AUROC 0.70 versus 1.00). On a real model, V-JEPA 2-AC adapted to the scene, the same thing happens. Given its own calibration pushes the model tracks the object (per-object correlation with the truth 0.91); given another object's, it does not (0.05). Self-consistency cannot tell these apart, preferring the right evidence on 52% of objects, chance level, while anchored disagreement prefers it on 73% and correlates 0.92-0.99 with the rollouts' error. Used to choose among candidate rollout sets, it comes within 0.003 of an oracle that sees the truth. A model that has internalised the wrong object is exactly as self-consistent as one that has internalised the right one; consistency has to be anchored to evidence.
Read the full abstract ▾
Machine Learning
Beyond the Query: Do Retrieval Signals Improve Adaptive Multimodal RAG Routing?
Qiaomu Li, Qiuyuan Zhang and Nong Ming
Adaptive RAG often uses
retrieval-time signals to decide whether another
retrieval, reranking, or
multimodal step should run. We ask whether these signals add routing value once the query itself is already known. Across document, audio, and
video RAG, we compare matched query-only and query+
retrieval routers while holding the optional actions, router family, training procedure, and evaluation fixed. On the held-out final evaluation, adding the tested
retrieval signals does not produce a reliable routing improvement over the query-only baseline. Some
retrieval signals are associated with whether a later step will help, but that predictability does not consistently lead to bet- ter RUN/SKIP decisions. The main lesson is therefore methodological:
retrieval-state features should not be credited with routing value unless they improve over a matched query-only control. Our results do not show that routing or
retrieval state is generally useless; they show that the incremental value of
retrieval signals must be demonstrated rather than assumed.
Read the full abstract ▾
Machine Learning
Observation-Anchored Selective Assimilation for Longitudinal Tumor-State Proxy Forecasting in Post-Treatment Glioma
Yeonjae Jung and Minwoo Shin
Post-treatment MRI in patients with glioma provides serial observations for updating patient-specific tumor-state proxy estimates, but variable appearances and trajectories complicate
forecasting. We formulate
forecasting as an observation-aware digital-twin update in which an intermediate observation anchors the patient-specific state. Among 203 patients and 594 follow-up time points, a predefined no-new-treatment criterion retained 120 of 236 candidate triplets, split into 81/24/15 training/validation/test triplets at the patient level. Each time point was represented by a continuous voxel-wise tumor-state proxy map in [0,1] derived from MRI lesion labels. A SegMamba-based single-step forecaster predicted update proposals from
multimodal source-state tensors. Observation-Anchored Selective Assimilation (OASA) retained the observed intermediate proxy as the state anchor and selectively applied updates through a validation-selected tiered case-level rule and voxel-wise soft gate. We compared initial-scan
forecasting, rollout without assimilation, latest-observation persistence, direct prediction, OASA, OASA + calibration, and morphological dilation. Checkpoints, OASA rules, and calibration thresholds were selected using validation data only. Across three seeds on 15 held-out test triplets, OASA maintained Dice at $\tau$ = 0.2 comparable to persistence (0.6071 $\pm$ 0.0025 vs. 0.6070) while yielding numerically higher Dice at $\tau$ = 0.5 (0.4269 $\pm$ 0.0079 vs. 0.3981), with a small RMSE increase. Calibration increased Dice at $\tau$ = 0.2 to 0.6178 $\pm$ 0.0025, increased false-positive (FP) support (11,836$\rightarrow$18,663), and reduced false-negative (FN) support (22,107$\rightarrow$17,536). This reflects near-threshold support calibration rather than improved biological predictive capability. Code is publicly available at https://github.com/jsudg436/longitudinal-proxy-
forecasting.
Read the full abstract ▾
Machine Learning
Granularity-Adaptive Credit Assignment for Long-Horizon LLM Agent Reinforcement Learning
Taoran Liang, Yang Liu, Shang Luo et al. (+10 more)
Long-horizon
language-model
agents trained with reinforcement learning oftenreceive sparse outcome rewards that do not reveal which decisions along a tra-jectory deserve credit. Episode-level advantages provide coarse trajectory-widecredit, while step-level comparisons offer finer resolution with context-dependentestimation noise. We propose Granularity-Adaptive Credit Assignment (GACA),a critic-free method that adaptively mixes episode- and step-level credit for eachdecision during policy optimization. GACA normalizes the sampled response'smean per-
token negative log-likelihood (NLL) within each trajectory and uses theresulting criticality score to determine the step-specific mixture. The computationreuses rollout log-probabilities without additional training rollouts or model eval-uations. Our analysis characterizes optimal score-dependent mixing and derivesconditions linking expected NLL to a lower bound on the preferred step-levelweight. Across ALFWorld and WebShop with 1.5B and 7B backbones, GACAachieves the highest reported mean success rates among the compared methods,while introducing negligible additional computation.
Read the full abstract ▾
All authors Taoran Liang, Yang Liu, Shang Luo, Yingguang Yang, Rongrong Zhang, Yingzong Min, Yulin Huang, Jianshen Zhang, Yongzhi Qi, Kefu Xu, Congjing Ran, Bin Pan, Bin Chong
Machine Learning · Cross-listed from stat.ML
Inference for Newton Methods with Accelerated Sketch-and-Project via Random Scaling
Xinchen Du, Elizaveta Rebrova, Michał Dereziński et al. (+1 more)
We study an online sketched Newton method that approximates the Newton direction at each step via a state-of-the-art sketching solver, called the generalized accelerated sketch-and-project solver (GAS), thereby mitigating the computational bottleneck of classical second-order methods. The GAS solver improves upon vanilla, unaccelerated sketch-and-project solvers by achieving accelerated convergence through Nesterov momentum updates, and accommodates a flexible projection metric whose proper choice further reduces computational cost. Building on this design, we establish asymptotic normality of the averaged sketched Newton iterates and characterize their limiting covariance
matrix. The resulting covariance recovers that of the unaccelerated sketched Newton method under a specific choice of acceleration parameters, converges more rapidly (in the number of sketching steps) to the minimax-optimal covariance in general, and is smaller than that of the last iterate produced by the accelerated method. Finally, we strengthen these results by establishing a functional central limit
theorem for the Newton iterates, which allows us to bypass explicit covariance estimation and develop an online inference procedure based on random scaling. Specifically, we construct a pivotal test statistic by appropriately rescaling the averaged iterates, so that its limiting distribution is free of any unknown parameters, enabling asymptotically valid online inference. Numerical experiments demonstrate superior performance of the proposed inference procedure.
Read the full abstract ▾
All authors Xinchen Du, Elizaveta Rebrova, Michał Dereziński, Sen Na
Machine Learning
MInTRL: Off-policy Intervention can boost On-policy RL
Mingyu Chen, Yefan Tao, Gerald Friedland et al. (+2 more)
Reinforcement learning with verifiable rewards is typically performed on-policy, keeping training data close to the current policy but limiting learning to trajectories that the policy can discover itself. Off-policy methods such as supervised fine-tuning, on the other hand, can leverage external knowledge beyond the base model's capabilities, but may suffer from large distribution shift. The key challenge is thus to expand exploration without sacrificing learnability. In this work, we introduce Minimal Intervention Reinforcement Learning (MInTRL), which expands the exploration frontier through sparse, local interventions in otherwise on-policy rollouts. During generation, a judge-intervention policy periodically reviews the current policy's output, replaces erroneous suffixes with short corrections, and immediately returns control to the policy. During training, MInTRL adopts a sequence-level advantage-regression objective that eliminates the need for importance sampling. We show that sparse, local interventions can substantially improve coverage beyond finite-budget on-policy sampling while preserving the overall on-policy nature of the resulting trajectories. Across math and code
benchmarks, MInTRL consistently outperforms standard on-policy and off-policy baselines. Ablations show that MInTRL remains effective with self-intervention and across different judge policies, while performance peaks at moderate intervention intensity, highlighting the importance of intervening minimally. These results establish minimal intervention as an effective paradigm for enhancing on-policy RL.
Read the full abstract ▾
All authors Mingyu Chen, Yefan Tao, Gerald Friedland, Xuezhou Zhang, Chris Kong
Machine Learning
RiPPLE: Cross-Space Performance Prediction from Early Training for Neural Architecture Search
Yifan Yang, Zhaoyan Wang, Zheng Gao et al. (+2 more)
Neural architecture search (NAS) evaluates candidate networks, but fully training enough architectures to rank an entire space is expensive. Zero-cost proxies score architectures at initialization, yet their ranking quality varies across search spaces. Learned predictors reduce evaluation cost but typically require fully trained labels or partial-training features for individual candidates. We introduce $\textbf{RiPPLE}$, $\underline{\textbf{R}}$anking v$\underline{\textbf{i}}$a $\underline{\textbf{P}}$refix-$\underline{\textbf{P}}$ropagated $\underline{\textbf{L}}$abel $\underline{\textbf{E}}$xtrapolation, which treats partial training as a source of labels for a small coverage set of anchors. RiPPLE trains these anchors to an early prefix, extrapolates their learning curves to surrogate labels, and propagates the labels over label-free architecture features. The early-training signal remains a label on the anchors rather than a per-candidate feature. Feature, readout, and encoding rules are selected without held-out accuracy and reused across search spaces. We evaluate the method on twelve
benchmark cells from four search-space families and on the larger DARTS space. The results examine ranking quality, label efficiency, architecture selection, and the roles of readout, coverage, and propagation. RiPPLE provides a whole-space ranking from a fractional anchor-training budget, with comparisons interpreted under their respective evaluation and cost protocols.
Read the full abstract ▾
All authors Yifan Yang, Zhaoyan Wang, Zheng Gao, Xiaoyu Li, Jiaojiao Jiang
Machine Learning · Cross-listed from cs.DC
HoliBench: A Cross-Platform Benchmarking and Deployment Toolkit for Foundation Models in CPS-IoT Applications
Inesh Chakrabarti, Zejun Xiong, Pragya Sharma et al. (+1 more)
Foundation models, including large
language models,
vision-
language models, and time-series foundation models, are increasingly deployed on embedded and edge platforms for CPS and IoT applications, where energy, latency, and
memory are as critical as task accuracy. Existing benchmarking tools evaluate model capability in isolation, reporting accuracy assuming sufficient compute, while hardware profiling tools remain platform-specific and mutually incompatible. As a result, users lack a unified workflow for making deployment decisions across heterogeneous devices. We present HoliBench, a modular benchmarking and deployment toolkit that jointly characterizes accuracy, latency, and energy across platforms from single-board computers to GPU servers. Its platform abstraction layer calibrates cross-device measurement, and the toolkit supports multiple model modalities, inference engines, concurrencies, and existing evaluation harnesses. An interactive interface exposes constraint-aware configuration selection over a design space that is profiled once and reused across studies. Using HoliBench, we characterize 20 models across 7 device types, 3 quantization levels, 8 inference backends, and over 30 tasks, surfacing tradeoffs that existing tools miss: quantization reduces latency only on hardware with low-precision support, accuracy gains show diminishing returns relative to energy, and for autoregressive workloads, average inference power is approximately constant across output lengths. We further find that single-model profiles compose under sequential co-resident execution. In a multi-model CPS deployment, standalone profiles predict combined-pipeline latency and power within 1.2% and 2.5%, enabling deployment exploration without exhaustively profiling every pipeline configuration. We release HoliBench as open-source infrastructure for deployment-aware evaluation of foundation models.
Read the full abstract ▾
All authors Inesh Chakrabarti, Zejun Xiong, Pragya Sharma, Mani Srivastava
Machine Learning
Split Conformal Prediction with Label-Shift-Adjusted Bayesian Scores
Hyeonsu Lee, Juyeon Kim, Erkhembayar Jadamba et al. (+2 more)
Conformal prediction provides distribution-free uncertainty quantification under exchangeability. However, this assumption is violated by label shift, where the marginal distribution of labels changes while the conditional distribution of inputs given labels remains stable. Under such shifts, standard conformal procedures no longer maintain their intended coverage behavior. Existing approaches address this via importance weighting. They pair the reweighting with residual-based nonconformity scores that ignore predictive uncertainty. The resulting intervals have uniform width.
Bayesian conformal methods produce adaptive intervals by leveraging predictive distributions. They evaluate conformity under the source predictive, which is misaligned with the target domain under label shift. We propose the Label-Shift-Adjusted
Bayesian Score (LSA score), a nonconformity score derived from a posterior predictive tilting identity. This identity shows that the target predictive is an importance-weighted transformation of the source predictive. We use it to derive a direct correction to the
Bayesian score. We evaluate the method on molecular property prediction under controlled label shift. The LSA score consistently yields shorter intervals than residual-based and source-based
Bayesian scores. Coverage in the target domain remains comparable. Under stronger shift, all methods incur some coverage loss due to pseudo-label-based density-ratio estimation. The LSA score is defined for any source predictive with a tractable log-density. We instantiate it with
Bayesian Ridge Regression, where the correction admits a closed form.
Read the full abstract ▾
All authors Hyeonsu Lee, Juyeon Kim, Erkhembayar Jadamba, Seungjin Choi, Hyunjin Shin
Machine Learning · Cross-listed from stat.ML
Membership Inference via Pairwise Likelihood Ratios
Shengjie Niu, Zebin Yun, Yeheng Ge et al. (+1 more)
Membership inference attacks (MIAs) are the standard tool for
auditing the
privacy risks of machine learning models. Given a query point, an MIA aims to determine whether that point was used to train the target model. In practice, such inference must rely on the statistical signals exposed by the model's outputs, such as confidence scores, logits, and intermediate feature representations. However, existing methods often fail to efficiently summarize and combine these statistical signals. To address this limitation, we propose Pairwise Likelihood MIA (PL-MIA), a unified method that combines a Gaussian likelihood-ratio (GLR) statistic with population calibration and the Cauchy combination test. We characterize theoretically how the GLR retains variance-contraction signals and establish conditions under which population calibration and Cauchy combination improve attack power. We obtain $p$-values from pairwise comparisons between the query point and reference points not used for training, and aggregate these continuous signals using the Cauchy combination test. This preserves the evidence strength that is discarded when each pairwise comparison is reduced to a binary vote. Extensive experiments demonstrate that PL-MIA outperforms strong baselines, improving the true positive rate (TPR) by over 25% in the critical low-false-positive regime, corroborating our theoretical findings. These results demonstrate how statistical principles can turn noisy model outputs into more powerful, calibrated, and reproducible evidence for membership
privacy auditing.
Read the full abstract ▾
All authors Shengjie Niu, Zebin Yun, Yeheng Ge, Jian Huang
Machine Learning
Certified AI Triage of ICU Alarms
Mohammed Sameer Syed and Rozhin Yasaei
In the VTaC
benchmark 71% of ventricular-tachycardia alarms are false, but silencing a real one can delay recognition of a dangerous arrhythmia. We reframe alarm reduction as three-way triage (retain, suppress, or defer) and bound the decision this analysis treats as harmful: among suppressed alarms, the fraction that were genuine stays below a user-set budget with 95% confidence, under i.i.d. event sampling. Alarms sharing a waveform record are dependent, so the clustered analysis is a sensitivity check. On the official split a 5% budget certifies in all three seeds, suppressing 74.8% of false alarms while silencing 1.5% of genuine ones, at AUROC 0.953 and Challenge Score 83.33, numerically comparable to the strongest of the eleven published systems. Our central finding measures what multiplicity costs: the correction charges for every candidate, so a finer grid can certify strictly less. Under held-out calibration the 885-
cell grid we declared certifies 1 of 15 fold-runs, while choosing the grid on a separate selection partition certifies 8. We project the calibration volume each budget needs, making an uncertifiable budget a design parameter. Finally, adding a learned reliability dimension to the policy grid did not sharpen the certified frontier.
Read the full abstract ▾
Machine Learning
LatentVerse: A Framework for Understanding Shared and Modality-Specific Information in Multimodal Latent Representations
Majd Alafrange, Samuel Friedman, John Kitonyo et al. (+2 more)
Latent
embeddings have become a central data abstraction in modern machine learning, especially in biomedicine, where foundation models are increasingly used to encode
multimodal data like clinical text, medical images, omics, and physiological signals. However, the utility and value of these representations depends on understanding their quality, structure, and the information they encode. Existing analysis workflows for evaluating representations remain fragmented across custom scripts, isolated metrics, and most importantly lack
multimodal analysis, limiting accessibility and reproducibility. We present LatentVerse, a representation analysis resource that combines a web-based visual analytics platform for accessible, report-driven exploration with a command-line interface for scalable technical workflows. LatentVerse unifies diagnostics for various representation quality metrics and extends to
multimodal settings by decomposing
embeddings into shared and modality-specific components. We evaluate LatentVerse through controlled unimodal and
multimodal simulations, discovery-oriented analyses on real biomedical
embeddings, and a user study across diverse use cases. By supporting thorough and interpretable evaluation of latent spaces, LatentVerse makes foundation model representations more understandable in biomedical and data science applications.
Read the full abstract ▾
All authors Majd Alafrange, Samuel Friedman, John Kitonyo, Sana Tonekaboni, Mahnaz Maddah
Machine Learning
When Connected Does Not Mean Similar: Charting the Homophily Boundary of SNAP-KG for Streaming Entity Integration
Jui-Chien Lin and Oshani Seneviratne
SNAP-KG is a framework for assigning newly arriving entities to semantic communities in a growing knowledge graph (KG) using only their raw features, with no graph access and no retraining at inference time. It was evaluated on five multi-view
benchmarks and a 2.4M-node OGB-WikiKG2 KG. In each of these
datasets, at least one graph view is homophilous, meaning that connected nodes usually belong to the same class, and SNAP-KG performs well on all of them. This paper asks what happens outside that setting. We extend the evaluation to three heterophilous graphs (Texas, Wisconsin, Chameleon) and measure the edge homophily of every view. When no homophilous view is available, clustering quality drops sharply for both SNAP-KG and the transductive baselines used in its original evaluation. What decides this is the homophily of the relation, not the number of relations. Multi-view fusion still helps, but only when at least one homophilous relation provides a reliable foundation. The homophily assumption is therefore shared by the whole method family, not specific to SNAP-KG. We argue that heterophilous multi-view clustering is a separate research problem, outside the scope of this work. As future work, we outline how a heterophily-aware teacher could be distilled into SNAP-KG's projector to serve both homophilous and heterophilous KGs.
Read the full abstract ▾
Machine Learning
ParaRecover: A Process-Level Benchmark for Error Localization and Recovery in Parallel Tool-Use Agents
Bowen Guan, Zhentao Yin and Yanming Shen
Existing
agent benchmarks mainly evaluate final task success or tool-call correctness, providing limited insight into whether
agents can reliably diagnose and recover from intermediate execution failures. This limitation becomes particularly critical in multi-turn parallel tool-use scenarios, where errors may propagate across dependent branches and trigger cascading failures. We introduce ParaRecover, a process-level
benchmark for evaluating error localization and recovery in multi-turn parallel tool-use
agents. Built upon a fine-grained taxonomy of 14 error types covering planning dependencies, tool selection, and argument matching, the
benchmark comprises 10,626 instances spanning two difficulty levels. To enable finegrained, process-oriented evaluation, we further propose the SDE rubric, which measures structural integrity, diagnostic
reasoning, and evolutionary strategy during
agent execution.Experiments across more than ten mainstream LLMs reveal that even state-of-the-art models still struggle with multi-turn error propagation,implicit tool-use failures, and precise replanning. Moreover, we demonstrate that the SDE rubric provides effective supervision signals for improving
agents' reflective recovery capabilities. Our data and code are available at https://github.com/gbw206/ParaRecover.
Read the full abstract ▾
Machine Learning
Theoretical Guarantees for One-Shot Magnitude Pruning and Compute-Adaptive Early Exit
Erdem Koyuncu
We study compute reduction in neural networks through a unified partial versus full computation view, captured by one-shot magnitude
pruning in the static regime and early exit in the adaptive regime. In an asymptotic single-
neuron model, we prove a concentration
theorem for one-shot magnitude
pruning with explicit rates. We also introduce the conditional perceptron for early exit and show that its excess generalization error decays as a power of the compute gap, with an exponent that grows to infinity as the
alignment between partial and full computations tends to one. We then extend the analysis to deep networks, characterizing how
pruning-induced distortions accumulate with depth and deriving a corresponding compute-accuracy tradeoff for frozen-backbone early exit under a neural network Gaussian process model. Numerical simulations corroborate the predicted scaling laws.
Read the full abstract ▾
Machine Learning
Simulating Disengaged Students to Evaluate LLM-based Tutors
Xianghui Meng and Jionghao Lin
Simulated students generated by computational models provide a practical way to evaluate tutoring strategies and pedagogical approaches used by human and AI tutors. However, such simulations should account for disengaged behaviors, including gaming the system, wheel-spinning, and off-task behavior, because tutors may need different responses for different learner states. We present Disengagement-Aware Student Simulators (DAS2), a reproducible pre-deployment protocol that models five learner-engagement states: engaged, gaming, wheel-spinning, off-task, and mixed, and evaluates AI tutor performance across these states. Using ASSISTments09, two coders independently labeled 100 sampled tutoring sessions based on anonymized interaction-log summaries. They achieved 84% agreement (Cohen's kappa = 0.78), and among agreed cases, human consensus labels matched DAS2 rule-based labels in 81% of cases (kappa = 0.75). Conditioning simulations on intended learner states reduced the correctness-rate gap between simulated and authentic sessions from 0.54 to 0.20 for gaming and from 0.51 to 0.18 for wheel-spinning. Fine-tuned Qwen2.5-7B better matched authentic response-time distributions, while
prompt-only GPT-4o generated more distinguishable learner states. Evaluation of five AI tutors from the Claude, Llama, Gemini, Qwen, and GPT families showed that relative rankings remained stable across learner states and interaction lengths, while absolute performance varied, revealing state-specific differences in tutor support. Human validation further showed that automated tutor evaluation does not fully align with human judgment. DAS2 provides a pre-deployment framework for evaluating how AI tutors respond to diverse learner-engagement states before deployment.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
LoRA-RC: Reservoir Computing with Low-Rank Adaptation
Wenbin Wan
Reservoir computing (RC) trains only a linear readout over a fixed recurrent layer, making it fast and data-efficient for online prediction. However, a static reservoir degrades under system drift, readout-only adaptation is then insufficient, and unconstrained reservoir adaptation can destroy the echo-state and incremental stability properties that make RC reliable. This paper proposes LoRA-RC, which adapts the recurrent
matrix through a low-rank correction driven by streaming prediction errors. The base reservoir and adaptation bases are fixed offline; a small core
matrix is adapted online, projected onto a spectral-norm ball, and low-pass filtered at each step. The projection guarantees that every applied recurrent
matrix remains within a certified contraction set, and an incremental input-to-state stability bound is established for the reservoir along each online adaptation path, with path-independent rate and gain. On a Lorenz system with an abrupt parameter drift, LoRA-RC cuts post-drift prediction error by 56% versus a fixed RC and 51% versus readout-only adaptation; ablations over 20 seeds show that removing the projection inflates this error by more than a factor of 40.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
Affective Agent: On-Device Personalized Intervention Reasoning for Wearable Systems
Reina Mun, Zishen Wan and Vijay Janapa Reddi
Affective computing has advanced wearable state inference, but on-device
reasoning about whether, when, and how to intervene remains challenging. We present Affective
Agent, a three-layer reference architecture for personalized intervention
reasoning under uncertainty on wearable-class hardware. It combines a compact sub-billion-parameter
language model with physiological evidence, context, and user history to decide whether, when, and how to intervene, without cloud dependency or per-user retraining. The architecture is organized into three interacting layers (perception, personalization, and
reasoning), adapting to individual users through host-managed structured
memory evolution rather than per-user weight updates. We instantiate Affective
Agent in indoor environmental quality control and evaluate it on held-out, simulator-generated longitudinal scenarios spanning physiological variation, context, signal quality, and intervention history. Results show that
memory-driven personalization and two-pass structured
reasoning improve intervention decisions within this synthetic evaluation. By moving the decision layer on-device, this work demonstrates a path from wearable state inference toward closed-loop, personalized intervention on wearable-class hardware.
Read the full abstract ▾