Machine Learning · Cross-listed from stat.ML
Generative Marketing Mix Modeling: A Causal Inference Framework Linking GEO and GEM to Business Impact
Masahiro Kato, Daiki Honma and Taka Kato
Generative artificial intelligence changes how firms reach customers, but standard marketing data do not record how often users see and notice a firm's name in generated answers. We develop Generative Marketing Mix Modeling (GMMM) to estimate the causal effects of Generative Engine Optimization (GEO) and Generative Engine Marketing (GEM). For GEO, GMMM combines repeated generated answers with question counts, shares of use across generative systems, and notice probabilities. For GEM, it combines records of sponsored placements with notice probabilities. GMMM compares expected business responses under alternative treatment sequences and establishes sufficient conditions for identifying the resulting effects. We investigate the empirical performance of the proposed method using simulated answers to product recommendation in English and Japanese.
Read the full abstract ▾
Machine Learning
From Protocols to Evidence: Bounded Claims for AI in Service of the Common Good
Nitesh V. Chawla and Paulo Benanti
Claims that AI systems improve decisions, broaden access, reduce harm, or empower users can exceed what their evaluation supports. This gap motivates our ongoing work on RISE AI. In this paper, we examine the proposed framework in light of Pope Leo XIVs Magnifica Humanitas, particularly its treatment of human dignity, ownership, and technological power. We develop three linked propositions. First, AI adoption can reveal unmet human needs and institutional weaknesses, while deployment can change how institutions respond to them. We propose a rupture test to examine whether deployment repairs or compounds existing failures, substitutes for human support, or conceals problems behind improved performance. The test would compare deployment with existing practice and feasible human or non AI alternatives. Second, responsible AI must examine the institutional conditions surrounding a system, including who owns the infrastructure, controls its use, and bears responsibility for its consequences. Third, favorable evaluation cannot by itself establish respect for human dignity or moral and political legitimacy. We distinguish evidence-bounded deployment, which limits claims to what the evidence supports under specified conditions, from measurement-bounded governance, which identifies constraints that favorable results cannot override. We outline a proposed RISE AI framework for applying these propositions across Responsibility, Inclusivity, Safety, and Empowerment.
Read the full abstract ▾
Machine Learning
TART: A Modular Tool for Technique-Aware Audio-to-Tablature Guitar Transcription
Akshaj Gupta, Hwi Joo Park, Andrea Guzman et al. (+5 more)
Automatic Music Transcription (AMT) for guitar remains limited by three challenges: existing systems often fail to capture expressive techniques such as slides, bends, and percussive hits; they often assign notes to incorrect string-fret combinations; and they are typically trained on clean recordings, limiting their generalization to noisy real-world audio. To address these challenges, we propose TART, a modular four-stage audio-to-tablature pipeline consisting of (1) an audio-to-MIDI transcription model, (2) an expressive technique classifier, (3) an audio-conditioned T5 encoder-decoder for string-fret assignment, and (4) an automated tablature generator. We evaluate TART in a zero-shot setting on GuitarSet, EGDB, and two augmented
benchmarks, Noisy GuitarSet and Noisy EGDB. Averaged across these four
benchmarks, TART achieves 81.35% audio-to-MIDI F50 (+6.67 points over the best prior baseline), 71.8% string-fret Tab F1 (+8.5 points over the best prior baseline), and 54.08% end-to-end Tab F1. To our knowledge, TART is the first framework to generate guitar tablature with both fingering and expressive technique annotations directly from guitar audio.
Read the full abstract ▾
All authors Akshaj Gupta, Hwi Joo Park, Andrea Guzman, Shamak Gowda, Samhita Konduri, Jiachen Lian, Robin Netzorg, Gopala Anumanchipalli
Machine Learning
CausalArena: Benchmarking Causal Discovery in the Foundation Model Era
Zi-Rong Li, Si-Yang Liu, Tian-Zuo Wang et al. (+1 more)
Causal discovery aims to uncover causal structures from data and is fundamental to scientific
reasoning and intervention-based decision making. Its evaluation relies heavily on structural causal models (SCMs), which specify a causal graph together with the mechanisms that generate data, yet existing studies differ substantially in graph families, mechanisms, and evaluation protocols. The emergence of causal discovery foundation models (CDFMs) further complicates evaluation: performance may reflect not only causal discovery ability, but also overlap between pretraining environments and test SCMs, making results on fixed synthetic
benchmarks difficult to interpret. We introduce CausalArena, a unified and evolvable
benchmark for causal discovery under a common protocol. Synthetic SCMs supply controlled breadth over structures and mechanisms; semantic operational SCMs provide human-auditable, semantically grounded environments beyond standard synthetic generators; and formula-grounded SCMs test discovery under explicit scientific mechanisms. Public real-world
datasets provide an additional external-validity check. Experiments across classical, neural, and pretrained methods reveal substantial ranking shifts across SCM families and protocols, showing that strong performance in one
benchmark regime does not reliably transfer to others. These results highlight
benchmark diversity and pretraining–evaluation overlap as central challenges for evaluating causal discovery in the foundation model era.
Read the full abstract ▾
All authors Zi-Rong Li, Si-Yang Liu, Tian-Zuo Wang, Han-Jia Ye
Machine Learning · Cross-listed from cs.CV
3D Point Splatting for mmWave Radar Novel View Synthesis
Adnan Armouti, Yixuan Gao and Rajalakshmi Nandakumar
Solving novel view synthesis (NVS) for millimeter-wave (mmWave) radar requires a renderer that is physically faithful, complex-valued, and multi-viewpoint-tractable. No prior method achieves these three properties simultaneously. Differentiable Monte Carlo (MC) ray tracers implement the radar forward model directly with explicit material modeling and complex outputs, but do not scale to the multi-view optimization NVS demands. Optical-NVS ports of NeRF, hash grids, and 3D Gaussians train fast but discard phase and replace explicit material modeling with opaque learned features, restricting them to power-only range-azimuth (RA) magnitudes. We propose 3D Point Splatting (3DPS), the first differentiable point renderer for radar, derived directly from the standard solid-angle form of the radar equation. Each oriented 3D point carries an ITU-R P.2040 material model, evaluated in closed form, with the resulting complex phasor splatted into range bins through a precomputed point spread function (PSF). The complex-valued output makes the renderer product-agnostic. The same optimized scene yields analog-to-digital converter (ADC), complex range profile (CRP), and RA outputs through standard fast
Fourier transform (FFT) pipelines without retraining for each format. On six outdoor ColoRadar scenes, 3DPS reaches 0.587 mean Pearson correlation on held-out RA images. This is between 1.7x and 5.2x the three optical-NVS baselines (RadarSplat, Radar Fields, DART). Training takes approximately 3 minutes per scene on a single RTX 4090.
Read the full abstract ▾
Machine Learning
CoRA-NAS: Coarse Ranking and Anchor-Residual Refinement for Neural Architecture Search
Yifan Yang, Zhaoyan Wang, Zheng Gao et al. (+2 more)
Zero-cost proxies rank architectures cheaply, but their reliability varies across search spaces. We introduce CoRA-NAS (COarse Ranking + Anchor-residual), a two-stage framework combining a static ranking prior with low-cost learning-curve refinement. CoRA-Rank aggregates capacity and structure-at-initialization proxies through an equal-weight log-rank consensus and a target-free consensus gate. CoRA-Refine samples anchors across this prior, extrapolates their early validation curves, and propagates a learned residual correction with an ExtraTrees model. The refinement uses approximately 1% of the cost of fully training the candidate set. Fully trained architecture-accuracy labels are not used to fit the ranker. One configuration is used across spaces, with space-specific architecture encodings. Across NAS-Bench-201, NAS-Bench-101, TransNAS-Bench-101, and NATS-SSS, CoRA-Refine achieves mean Spearman correlations of 0.946, 0.715, 0.786, and 0.894, respectively. Its worst-space correlation of 0.715 is the highest among the compared methods. On NAS-Bench-201/CIFAR-100, its selected architecture reaches 73.32% accuracy, near the reported ground-truth best of 73.37%. On the pure size space, refinement recovers the static prior's shortfall relative to parameter count, while remaining tied with the strongest capacity proxies within noise. The resulting framework combines cross-space ranking robustness with low-cost architecture selection.
Read the full abstract ▾
All authors Yifan Yang, Zhaoyan Wang, Zheng Gao, Xiaoyu Li, Jiaojiao Jiang
Machine Learning · Cross-listed from cs.CL
Domain-Specific Hallucination Detection in Large Language Models
Varun Teja Chundru and Debasmita Biswas
Large
language models generate fluent text that can contain unfaithful claims – a phenomenon known as
hallucination. We present a multi-signal detection pipeline combining fine-tuned DeBERTa-v3 classification, Monte Carlo (MC) Dropout uncertainty quantification, and temperature-scaled calibration for response-level
hallucination detection. Evaluated on the HaluEval
benchmark, our pipeline achieves F1=0.915 and AUROC=0.977 on general-domain tasks, with per-task F1 scores of 0.97 (QA), 0.96 (Summarization), and 0.82 (Dialogue). MC Dropout inference further improves accuracy to 93.2%. A context ablation study confirms the model performs genuine entailment
reasoning rather than exploiting surface patterns, with summarization F1 dropping 24% when knowledge context is removed. Learning curve analysis reveals that 25% of training data captures 77% of full-data performance. Beyond detection, we apply Direct Preference Optimization (DPO) to a Qwen2.5-0.5B generator, reducing its
hallucination rate from 85.5% to 37.7% (55.9% relative reduction) as measured by our detector. Cross-domain evaluation on the SciFact biomedical
benchmark shows that general-domain training transfers poorly (F1=0.52), motivating domain-specific fine-tuning. PubMedBERT fine-tuned on SciFact achieves F1=0.63 and AUROC=0.81, demonstrating that domain-matched pre-training is the strongest adaptation strategy. Code and models are available at https://github.com/varunteja99/
hallucination-detection-nlp
Read the full abstract ▾
Machine Learning
The Last AI Built by Humans: Toward Genuine Recursive Self-Improvement
Yi Duan, Ying Liu, Zirui Tang et al. (+32 more)
Recursive self-improvement (RSI) enables AI systems to turn experience and feedback into persistent changes that improve both their capabilities and the process of future improvement. We first use the Headroom-Closed Index (HCI) to reveal the problems of existing LLMs, then introduce the RSI concept and its development roadmap: from improvement-execution autonomy, improvement-strategy autonomy, experience-acquisition autonomy, and environment-adaptation autonomy, to recursive meta-improvement. Next we examine RSI across scenarios (e.g., scientific discovery,
embodied intelligence, software engineering), highlighting their distinct requirements and development speeds. Drawing on diverse industry practices and preliminary empirical evidence, we connect RSI research with practical systems and identify key challenges to achieving genuine RSI.
Read the full abstract ▾
All authors Yi Duan, Ying Liu, Zirui Tang, Haodong Chen, Jun Zhou, Yumou Liu, Bangrui Xu, Yukai Wu, Sidi Chen, Yuhan Zhou, Haoyu Wang, Xiaoyou Yu, Shaokun Han, Xuzhou Zhu, Le Zhou, Bolin Lu, Wei Zhou, Jiachen Liu, Nuozhou Fang, Jiaxin Tian, Ruoyu Chen, Yuxuan Li, Kai Zuo, Kaiyan Zhang, Qianyu Yang, Zijie Wang, Jiantao Qiu, Conghui He, Guoliang Li, Bowen Zhou, Zhiyuan Liu, Zhoufutu Wen, Jihua Kang, Xuanhe Zhou, Fan Wu
Machine Learning · Cross-listed from stat.ML
Evaluating Time-Series Foundation Models and Multimodal Dietary Context for CGM Forecasting
Bowen Zhang, Hsiu-Wen Cheng, Hongyu Yang et al. (+9 more)
Continuous glucose monitoring (CGM) provides high-frequency measurements of glucose dynamics and enables short-term glucose
forecasting for diabetes management. Although time-series foundation models have shown strong general
forecasting ability, their effectiveness for CGM prediction and the added value of
multimodal dietary context remain unclear. We conduct a comprehensive empirical study using eight public CGM
datasets spanning Type 1 diabetes, Type 2 diabetes, and non-diabetes populations. Under a unified protocol across multiple context lengths and prediction horizons, zero-shot foundation models did not consistently outperform strong task-specific baselines such as Elastic Net and PatchTST. In contrast, lightweight fine-tuning substantially improved
forecasting performance. For example, fine-tuned Chronos-Bolt reduced RMSE by 6.5%-18.4% in the T1D cohort and by 8.6%-18.2% in the non-diabetes/T2D cohort, with comparable improvements in both in-distribution and out-of-distribution test settings. We further evaluate
multimodal dietary context using CGMacros, which provides temporally aligned CGM signals, food images, and macronutrient records. A residual-based fusion framework reduced overall RMSE by approximately 3% and postprandial RMSE by approximately 15% relative to the CGM-only baseline. Moreover, Chronos-based CGM representations were more strongly correlated with observed postprandial glucose increments than representations from LSTM and CatBoost, even after those models incorporated additional dietary modalities, suggesting that pretrained temporal representations better preserve meal-induced excursion patterns. These findings show that foundation models require CGM-specific adaptation for reliable
forecasting and that dietary context provides clinically meaningful signals beyond CGM alone, especially during postprandial periods.
Read the full abstract ▾
All authors Bowen Zhang, Hsiu-Wen Cheng, Hongyu Yang, Evie L. Shen, Joleen Vansomphone, Yuna Li, Kerry Zhou, Zitian Qu, Suning Zhao, Xiangning Deng, Hua Zhou, Jin J. Zhou
Machine Learning
AdamX: Cosine similarity meets gradient descent
Francisco Caldas, Ruben Belo and Cláudia Soares
We introduce AdamX, a first-order optimizer that incorporates cosine similarity as an adaptive mechanism for controlling update magnitudes. The proposed method is scalable, model-agnostic, and straightforward to integrate into existing training pipelines. We further introduce a variance rectification scheme that promotes smoother optimization during the early stages of training. Overall, we provide empirical evidence that AdamX achieves competitive convergence rates across a range of
benchmark datasets and architectures. Performance is evaluated in terms of the number of epochs required to reach predefined performance thresholds under a fixed hyperparameter budget. Code and Experiments available at: https://github.com/FranciscoCaldas/adamX.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
Explainability Assistant: A Conversational XAI Interface for Interpreting Energy Consumption Models
Rodion Krjutškov, Eduard Barbu, Nikos Sakkas et al. (+1 more)
Energy consumption
forecasting relies on increasingly complex machine learning (ML) models, such as Genetic Programming-based symbolic regressors, whose predictions can be difficult for facility managers and building operators to interpret. Explainable Artificial Intelligence (XAI) techniques address this opacity, but traditional XAI dashboards require substantial technical expertise and provide limited flexibility for dynamic, context-aware inquiry. Conversational XAI systems offer a promising alternative; however, previous approaches, such as TalkToModel, were constrained by rigid custom grammars and achieved only 76.8% intent-parsing accuracy. This paper introduces the Explainability Assistant, an open-source conversational XAI system that leverages the function-calling capabilities of modern Large
Language Models (LLMs) to overcome these limitations. The system achieves 94% intent-parsing accuracy, supports flexible natural
language interaction, and adapts to different ML problem types without task-specific fine-tuning. We present the system's architecture and report results from a comparative evaluation conducted with energy domain specialists, contrasting the Explainability Assistant with a traditional XAI dashboard. The evaluation suggests improved usability and consistent task accuracy, with all experts unanimously preferring the conversational interface for practical use.
Read the full abstract ▾
All authors Rodion Krjutškov, Eduard Barbu, Nikos Sakkas, Sofia Yfanti
Machine Learning
Model-Aware Schedules Improve Generation via Fiberwise Optimal Transport
Luyi Jia, Boyan Zhang, Yilun Liu et al. (+1 more)
Diffusion and flow-matching schedules control the signal and noise coefficients that mix data and noise along affine probability paths. Minimizing a kinetic action defined on coefficient paths, motivated by optimal transport, helps explain strong baselines but remains model-agnostic and ignores prediction error. Here we introduce a model-aware schedule construction based on fiberwise optimal transport. At a fixed time and state on the probability path, compatible signal/noise decompositions form an affine fiber. We define a fiberwise prediction risk by averaging optimal-transport costs between the true and predictor-induced decompositions within these fibers. On a fixed coefficient curve, combining this risk with coefficient-path kinetic action yields a closed-form optimal time allocation. This construction extends to general linear prediction targets, and the risk profile can be estimated from an early baseline checkpoint. We evaluate DDPMs and flow matching across prediction targets, training configurations, risk-estimation checkpoints,
datasets, and architectures. Our model-aware schedules consistently outperform strong baselines, including a 38.6% relative FID reduction for flow matching on CIFAR-10 at 16 function evaluations. Each model-agnostic kinetic baseline determines its own kinetic reference coordinate. In these coordinates, fiberwise-risk profiles from independently trained models in different settings align closely after normalization to unit area. The resulting schedule deformations used in training also align, suggesting empirical universality across the evaluated models and settings. Pretrained-checkpoint diagnostics extend this normalized-risk agreement to larger conditional latent
diffusion and 2-RF models. A frozen analytic allocation template retains most of the model-aware improvement without further risk estimation or model-specific fitting.
Read the full abstract ▾
All authors Luyi Jia, Boyan Zhang, Yilun Liu, Steffen Rulands
Machine Learning · Cross-listed from stat.ML
Near-Optimal Reinforcement Learning with Multi-Step Transition Lookahead
Corentin Pla, Hugo Richard, Marc Abeille et al. (+1 more)
We study reinforcement learning (RL) with transition look-ahead, where the
agent may observe which states would be visited upon playing any sequence of actions before deciding its course of action. Although look-ahead can substantially improve achievable performance, [1] showed that optimal planning with multi-step transition look-ahead is NP-hard. However, this hardness was established using a discount factor close to one. It was therefore unknown whether the problem remains hard for every discount factor, and whether near-optimal planning can nevertheless be performed efficiently. We resolve both questions. First, we show that for every fixed discount factor, exact planning remains NP-hard. Second, we introduce a randomized
polynomial-time approximation scheme for every fixed look-ahead depth. Third, we extend our approach to account for unknown transitions. We empirically validate the soundness of our results on the wind-farm storage-control
benchmark of [2], showing that our approach, optimally accounting for -step look-ahead information, offers substantially better performance than existing
algorithms. [1] Corentin Pla, Hugo Richard, Marc Abeille, Nadav Merlis, Vianney Perchet : On the Hardness of Reinforcement Learning with Transition Look-Ahead [2] Chenbei Lu, Zaiwei Chen, Tongxin Li, Chenye Wu, Adam Wierman : Reinforcement Learning with Imperfect Transition Predictions: A Bellman-Jensen Approach
Read the full abstract ▾
All authors Corentin Pla, Hugo Richard, Marc Abeille, Vianney Perchet
Machine Learning · Cross-listed from cs.CV
Logit Refiner: Improving Visual Autoregressive Models via Intra-Scale Dependency Modeling
Meimingwei Li, Stefan Andreas Baumann, Felix Krause et al. (+1 more)
Visual Autoregressive Models (VAR) generate images through next-scale prediction, producing all
tokens within each scale in parallel. We show that this parallel decoding constitutes a mean-field-style approximation that discards spatial dependencies among same-scale
tokens, causing locally incoherent samples regardless of backbone capacity – a limitation of the decoding rule. Addressing this limitation, we introduce the Logit Refiner, a lightweight autoregressive module that restores intra-scale dependencies by sequentially sampling
tokens conditioned on frozen backbone features. Adding only ~10% parameters and less than 5% of the base model's training compute, it plugs into any pretrained VAR checkpoint without retraining. Controlled ablations isolate joint intra-scale sampling – rather than additional capacity or training – as the critical ingredient. Across backbones from 310M to 2B parameters on class-conditional ImageNet 256x256, the refiner consistently improves generation quality, enabling a 1.1B-parameter model to surpass one twice its size. The approach further generalizes to text-to-image generation, confirming that the mean-field bottleneck persists across VAR variants and is effectively alleviated by our method. Project page: https://compvis.github.io/logit-refiner/
Read the full abstract ▾
All authors Meimingwei Li, Stefan Andreas Baumann, Felix Krause, Björn Ommer
Machine Learning
Thinking with Looped Flows
Ayhan Suleymanzade, Chanhyuk Lee, Floor Eijkelboom et al. (+3 more)
Humans and machines often solve harder problems by spending more time on computation. In deep learning, looped models implement this idea during inference by recurrently updating a hidden state. In practice, however, their training backpropagates through only one or a few updates, making it hard to train early updates to support future ones. We propose looped flows, an approach that sidesteps this issue by training the recurrence with local denoising objectives. By imposing temporal association across denoising objectives through progressively decreasing noise levels and shared noise, the model is incentivized to learn recurrent states that transfer useful computation over time, even when gradients cover only a few updates. We then formulate inference as integrating the velocity of a probability flow parameterized by the learned denoiser, coupled with recurrent states. This allows solving harder problems by spending more computation through a finer temporal grid and enables multiple valid predictions from different initial noise samples. Across six
reasoning benchmarks including two multi-solution
benchmarks, looped flows outperform prior state-of-the-art looped models overall, achieving 58.8% test accuracy on ARC-AGI-1 and 12.2% on ARC-AGI-2.
Read the full abstract ▾
All authors Ayhan Suleymanzade, Chanhyuk Lee, Floor Eijkelboom, Nicholas M. Boffi, İsmail İlkan Ceylan, Jinwoo Kim
Machine Learning
Dynamic language model representations for multi-objective reaction optimisation
Joshua W. Sin, David Ming Segura, Bojana Ranković et al. (+8 more)
Optimising chemical reactions across multiple objectives, such as yield, selectivity, and safety, is central to chemical synthesis, and model-driven approaches depend critically on how reaction components are represented. Established featurisations are either chemically uninformative, as with one-hot encodings, or, as with molecular descriptors, do not readily extend across chemically distinct components. For structurally and functionally diverse components, it is therefore unclear what a shared representation should contain. Constructing such a representation is itself a challenging research undertaking that must be revisited for each new reaction system. Here we bypass this step by learning the reaction representation dynamically from text. Textual descriptions of reaction conditions are encoded by a fine-tuned
language model trained jointly with Gaussian process surrogates, yielding task-adaptive representations within a multi-objective
Bayesian optimisation loop. Across nickel- and palladium-catalysed cross-couplings in both sequential and parallel experimentation regimes, this approach reaches optimisation convergence in fewer experiments than descriptor libraries or one-hot encoding. Applied prospectively to a palladium-catalysed cyanation spanning mixed ligand denticity and heterogeneous additives, and to a three-objective asymmetric hydrogenation across
chiral iridium and ruthenium catalyst families, two rounds of high-throughput experimentation (192 reactions, under 3% of each design space) delivered conditions translating directly to gram scale in 94% and 84% isolated yield, the latter at 99.6% enantiomeric excess.
Read the full abstract ▾
All authors Joshua W. Sin, David Ming Segura, Bojana Ranković, Siu Lun Chau, Marius D. R. Lutz, Andrea Anelli, Ryan P. Burwood, Kurt Püntener, Maximilian J. Notheis, Raphael Bigler, Philippe Schwaller
Machine Learning
Predicting Privacy Leakage from Weight Spectral Density
Richard J. Preen and Jim Smith
Membership inference attacks (MIAs) are widely used to audit the
privacy disclosure risk of machine learning models, however current state-of-the-art attacks require training computationally expensive shadow models, making large-scale
privacy evaluation impractical. In this work, we investigate whether inexpensive spectral metrics derived from the heavy-tailed self-regularisation framework can serve as proxies for MIA vulnerability. We evaluate several WeightWatcher spectral metrics on image and tabular classification tasks and compare their relationship with MIA
privacy leakage against conventional measures of generalisation. Across
datasets, stable rank exhibits a strong positive correlation with overall MIA success, while Log alpha-Norm shows a consistent negative correlation with MIA vulnerability at the low false-positive regime. These associations are observed to be stronger than those obtained using the generalisation gap. The results indicate that neural network spectra may contain information about
privacy leakage that is not fully captured by conventional measures of overfitting, motivating spectral analysis as a promising direction for scalable
privacy auditing.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CR
Differentially Private EEG Feature Anonymization: A Privacy-Utility Case Study in Clinical Neurophysiology
Noman Sadiq and Mohsen Toorani
Clinical electroencephalography (
EEG) data are valuable for healthcare research and for developing artificial intelligence (AI)-based clinical decision-support systems, but
EEG recordings and derived features may contain sensitive patient-specific information. This creates
privacy risks when data are reused, analyzed, or shared across clinical and research environments. Conventional anonymization methods are often insufficient for high-dimensional biomedical signals, since removing direct identifiers does not necessarily prevent re-identification, linkage, or inference risks. At the same time, strong
privacy protection may distort clinically relevant signal characteristics and reduce data utility. This paper studies subject-level differential
privacy for protecting clinical
EEG-derived feature representations using Gaussian and Laplace perturbations. The proposed framework considers three deployment scenarios: client-side anonymization, centralized server-side anonymization, and decentralized local training. Following
EEG preprocessing and feature extraction, Gaussian and Laplace perturbations are applied to the resulting patient-level
EEG feature representations. The Laplace experiments evaluate the implemented noise scales, while the scales required for formal full-vector calibration are derived separately. The effects of both perturbations are assessed using statistical utility measures and a downstream machine-learning-based utility check. The results show that differentially private perturbation can be integrated into
EEG processing workflows, but the selected mechanism,
privacy parameters, and sensitivity calibration strongly influence data utility. The study highlights the practical
privacy-utility trade-off in DP-based
EEG feature anonymization and the challenges of preserving downstream utility in small and imbalanced clinical
EEG datasets.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
A Unified Per-Token Gating Family for On-Policy Distillation: FKL/RKL Mixing with Multi-Channel and Bias Coefficients
Suwan Wu, Yumeng Lin, Pengcheng Yuan et al. (+1 more)
Per-
token gating of forward/reverse KL losses has become a standard technique for on-policy knowledge
distillation (OPD), but existing methods such as EOPD (Jin et al., 2026) and ToDi (Jung et al., 2025) each fix a single gating signal and a single gating direction, and the two have never been compared directly. We introduce a four-coefficient parameterization lambda_t = sigma(a * h_t + b * u(x) + c + d * gap_t) in which direction-aligned proxies of EOPD and ToDi appear as one-dimensional (1D) restrictions, and which adds multi-channel composition and an explicit bias as further degrees of freedom. On TweetEval (Barbieri et al., 2020)
emotion and hate, with a Qwen3-32B teacher and a Qwen3-4B student, configurations in the full family reach higher accuracy than the matched-magnitude single-channel (
entropy-only / gap-only) 1D restrictions in 33 of 36 comparable
cells, and a 26-
cell mean-match isolation experiment places dynamic gating ahead of effective-KL-matched static baselines in 19 of 26
cells. Because
cells share training data, models, and parameter substructure, we report both counts as exploratory aggregate directional evidence rather than as independent hypothesis tests. Targeted three-seed paired replications of the nine headline comparisons singled out by that sweep – including a third task, offensive – are directionally consistent, but individually smaller than the single-seed estimates and not significant at n=3. We therefore present the parameterization primarily as a shared coordinate system for comparing per-
token gating designs in short-output classification OPD.
Read the full abstract ▾
All authors Suwan Wu, Yumeng Lin, Pengcheng Yuan, Xiaolong Jiang
Machine Learning · Cross-listed from cs.AI
SIRF: A Spec-Internalized Risk Foundation Model for Industrial Content Risk Control
Suwan Wu, Yumeng Lin, Pengcheng Yuan et al. (+1 more)
For industrial content risk control, the real deployment constraint is not average accuracy but how much risk can be auto-handled under high precision and second-level latency. We present SIRF (Spec-Internalized Risk Foundation Model), which internalizes a platform's complex policies, synthesized without additional human annotation via EntiGraph, MAGA rewriting and account-level chain-of-thought (CoT), into the weights via continued pretraining (CPT), so rules are applied at high precision under an ultra-low-latency, verdict-only deployment. A controlled same-source comparison (Qwen3-8B-SFT vs. SIRF-8B-SFT, identical policy injection and verdict-only output form, differing only in policy-grounded CPT) attributes the gain to internalization: SIRF-8B-SFT reaches 71.3% Black Recall@P95, +15.1pp over the baseline, using only ~70M CPT
tokens without harming general ability, and among included, logprob-available models under this interface it matches or exceeds far larger systems. SIRF is deployed as a tree-model adjudication layer (20% more mis-penalized samples recovered) and transfers to a freezing scenario at low cost (~70% relative mis-penalization reduction).
Read the full abstract ▾
All authors Suwan Wu, Yumeng Lin, Pengcheng Yuan, Xiaolong Jiang
Machine Learning · Cross-listed from math.OC
Sparsity Regularized and Robust Mean Variance Portfolio Selection Under Ellipsoidal Uncertainty
Deniz Akkaya, Emre Can Yayla, Buse Şen et al. (+1 more)
We investigate mean-variance portfolio selection with an $\ell_0$-penalty to promote sparsity in asset allocations. Uncertainty in the mean return vector is incorporated through an ellipsoidal uncertainty set, yielding a robust sparse optimization framework. We characterize the structure of both local and global minimizers and exploit these properties in the risk minimization and return maximization formulations. Building on this structural insight, we develop a branch-and-bound
algorithm tailored to the resulting robust sparse portfolio problems, together with a new
pruning rule that can discard exponentially many candidate portfolios in a single step. Extensive computational experiments on real
market data, together with comparisons against a mixed-integer second-order cone programming solver, demonstrate the effectiveness and competitiveness of the proposed approach.
Read the full abstract ▾
All authors Deniz Akkaya, Emre Can Yayla, Buse Şen, Mustafa Ç. P{\i}nar
Machine Learning · Cross-listed from cs.DC
Building py-kvcache: A Performance Characterization of External KV Caching for vLLM with NVMe SSDs
Joseph Kanichai, Tiziano De Matteis and Animesh Trivedi
Prefix caching can reduce the time to first
token (TTFT) of long-context LLM requests by reusing previously computed key-value (KV) states, but for short prefixes or fast GPUs, recomputation can be faster than loading from an external cache. We characterize this tradeoff in vLLM across GPU, CPU, and NVMe tiers using synthetic workloads, long-context
benchmarks, production traces, and find that cache performance depends on transfer granularity, intermediate
memory use, and when transfers enter the request schedule, not only on device bandwidth. These findings motivate py-kvcache, a vLLM KV Offload connector with asynchronous direct I/O, bounded shared staging, and scheduler-aware preloading, which starts disk reads while requests are still waiting, overlapping with compute. At 80k
tokens, py-kvcache loading from disk is 2.0x faster than LMCache, with preloading contributing 1.34x. With GPU, CPU, and disk caching enabled, it is 1.23x faster than LMCache and within approximately 4% of the native vLLM KV Offload implementation. LongBench and SCBench show that these benefits extend to irregular prefix chains and multi-turn workloads. Bailian trace replays improve TTFT on a weaker GPU, but on an H100 the average request falls below the break-even point and GPU
memory alone retains enough prefixes. External KV caching should therefore be treated as a setup specific admission decision. The py-kvcacheimplementation is available at: https://github.com/atlarge-research/py-kvcache.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
LOCUS: Task-Aware Low-Rank Post-Training for Token-Efficient Language Generation
Dongfang Zhao
Large
language model serving costs scale directly with output sequence length, yet standard preference
alignment often inflates response verbosity without improving utility. We study whether the parameterization of post-training updates affects generation length: low-rank subspaces alter sequence length without modifying the
alignment loss. We present LOCUS, a method that selects a task-aware low-rank adaptation subspace to minimize output-
token cost subject to a utility constraint. Within this subspace, post-training retains the native preference objective with a frozen backbone. Across Anthropic HH-RLHF dialogue preferences, we evaluate two $\sim$3B decoder backbones, Pythia-2.8B and Qwen2.5-3B, against protocol-matched full-parameter DPO and DrDPO branches and the released SamPO checkpoint. LOCUS reduces continuation length by up to 39.84% on Pythia-2.8B and by 14.87–17.58% on Qwen2.5-3B while updating only 0.24–0.28% of model parameters, with no material change in the internal preference diagnostic.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.MA
Organizational Principles Enable Collective Intelligence in Embodied AI
Zhengran Ji, Jonathan Hyun and Boyuan Chen
Collective intelligence depends not only on the capabilities of individual members, but also on how those members are organized. Yet artificial multi-
agent systems are typically assembled using fixed organizational structures, even when the physical tasks they perform impose fundamentally different coordination requirements. Here we show that principles from human organization theory can be operationalized to organize large, heterogeneous collectives of
embodied artificial
agents. We introduce ORCH (Organizing Roles and Coordination Hierarchies), which constructs task-specific hierarchical organizations by combining pooled interdependence for work that can proceed concurrently with sequential interdependence for work governed by prerequisite relationships. Across 25 wildfire-response missions spanning reconnaissance, rescue, transportation, resource management, containment and suppression, we evaluated teams of up to 50 heterogeneous
agents using eight large
language models. Organizations constructed using these principles consistently outperformed four representative
embodied multi-
agent approaches across mission outcome, execution efficiency, exploration and computational resource use. Human-designed ORCH organizations improved final score by 63.97% and execution efficiency by 74.29% on average relative to the four prior frameworks. Organizations generated automatically by
language models improved these measures by 43.63% and 52.53%, respectively. These advantages persisted across missions and underlying
language models. Notably, collective performance was not monotonically determined by model scale. Analysis of long-horizon missions showed that hierarchical organization enabled teams to preserve concurrent activity within specialized groups while coordinating ordered transitions between mission phases.
Read the full abstract ▾
Machine Learning · Cross-listed from quant-ph
Learning structural balance of graphs from quantum spectral features
Stefano Scali and Oleksandr Kyriienko
We develop a
quantum approach to spectral feature extraction from the density of states (DOS) of a problem-dependent
Hamiltonian, and apply it to machine learning on signed graphs. We propose to embed a signed graph as an Ising model instance with positive and negative interactions, and use the standardized moments of the Ising DOS as features for learning. We show that these moments count signed closed walks, are switching-invariant, and are size-free by construction. As a
benchmark, we target learning the frustration index, an NP-hard measure of structural balance that can be labeled exactly at moderate size. At zero field, the models can be sampled classically, allowing the
quantum extraction procedure to be certified against exact ground truth. We propose DOS-QPE, a phase estimation on a purified maximally mixed probe, which samples the spectral density with orders of magnitude fewer shots than Hadamard test-based trace sampling and feeds the resulting features directly into classically trained models. On $1.4\times10^5$ labeled graphs the exact DOS determines the frustration index, and five moments recover it with a mean error of 0.4, well below one sign flip. Beyond zero field, the underlying trace-estimation problem is DQC1-complete, providing access to spectral features for which no efficient classical sampling method is known. Our work opens routes towards
quantum applications in social network balance analysis,
spin-glass studies, correlation clustering, and
protein-interaction networks.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.RO
Reflex-Informed Neuromuscular Reinforcement Learning for Muscle-Driven Locomotion
Jian Zhou, Xingyu Zhang, Rui Ma et al. (+3 more)
Muscle-driven locomotion provides a physically grounded approach to generating realistic human movement. However, achieving both physiological plausibility and adaptability to changes in musculoskeletal capacity and external disturbances remains a fundamental challenge. To address this limitation, we propose a Reflex-Informed Neuromuscular Reinforcement Learning framework for muscle-driven locomotion. Within this framework, a fixed phase-dependent reflex controller serves as the underlying neuromuscular control mechanism, while the reinforcement learning policy produces four biomechanically meaningful residual parameters to modulate key reflex gains and thresholds associated with hip swing, knee support, and ankle propulsion according to the current state. Experimental results demonstrate that the proposed framework generates physiologically plausible locomotion with improved kinematic accuracy and dynamic consistency, as well as better bilateral
symmetry and stride-to-stride consistency under nominal walking conditions. The learned policy remains robust under muscle weakness and external perturbations without retraining.
Read the full abstract ▾
All authors Jian Zhou, Xingyu Zhang, Rui Ma, Yu Cao, Shane Xie, Zhi-qiang Zhang
Machine Learning
Why Does Post-Training Quantization Work?
Yuxiang Chen, Michael Beyer, Jun Zhu et al. (+1 more)
Post-training quantization compresses large
language models (LLMs) by storing their weights at reduced precision, and each quantized weight introduces an error into the hidden states. Naively, these errors should accumulate with depth and corrupt next-
token prediction; randomly initialized models accumulate these discrepancies rapidly, whereas quantized pretrained models accumulate much less hidden-state error and largely maintain downstream task performance, even though they were never trained with quantization noise. This raises the question we address: why does post-training quantization work? Comparing full-precision and quantized forward passes, we identify two mechanisms that characterize pretrained quantization robustness. First, the error a layer newly introduces tends to oppose the error it inherits from the layer's input. The two cancel partially such that the discrepancy between full-precision and quantized passes grows slowly. This counteracting residual interaction develops during pretraining. Our quantitative analysis identifies it as a major factor slowing hidden-error growth. Second, LM-head geometry preferentially preserves the scores and probabilities of high-ranked
tokens, which typically represent the model's most confident predictions. Together, these mechanisms explain why quantization error that passes through numerous layers can still produce only small output changes, and we verify the findings across models and quantization settings.
Read the full abstract ▾
All authors Yuxiang Chen, Michael Beyer, Jun Zhu, Jianfei Chen
Machine Learning · Cross-listed from stat.ML
Generalization Analysis of Distributed Kernel-based Robust Gradient Descent Algorithms
Jun-Yi Meng, Zheng-Chu Guo and Yuan Mao
In this paper, we investigate the generalization performance of distributed gradient descent
algorithms in a reproducing kernel
Hilbert space under a robust loss function $l_{\sigma}$. By exploiting the spectral characterization of gradient descent together with the intrinsic properties of robust loss functions, we establish optimal learning rates for the distributed kernel-based robust gradient descent (DKRGD)
algorithm with an appropriately chosen scale parameter $\sigma$. The proposed parameter choice of $\sigma$ simultaneously alleviates the saturation phenomenon and guarantees statistical robustness. A key technical contribution is a novel error analysis that provides substantially sharper bounds for products of operators, thereby significantly relaxing existing restrictions on the maximum number of local machines while retaining optimal learning rates. Finally, we develop a communication-efficient strategy that further improves the convergence performance of DKRGD.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Negative Self-Distillation: Learning to Reason by Avoiding Flaws
Rongcan Pei, Zhepei Wei, Shuyao Xu et al. (+3 more)
On-Policy Self-
Distillation (OPSD) has emerged as a popular paradigm for large
language model (LLM) self-improvement, allowing models to act as their own teachers by leveraging privileged information such as ground-truth solutions. However, recent findings indicate that OPSD can severely degrade the performance of LLMs on complex
reasoning tasks: By forcing the student to imitate an artificially confident
reasoning trace conditioned on privileged information, OPSD inadvertently suppresses expressions of uncertainty and penalizes the exploratory, self-corrective behaviors required to solve challenging problems. To address this, we introduce Negative Self-
Distillation (NSD), a new framework that optimizes LLMs by diverging from flawed
reasoning rather than imitating privileged solutions. Instead of relying on ground-truth answers or external supervision, NSD uses the model itself to generate a question-specific negative condition (eg, acting as a “careless reasoner”) and pushes the student's distribution away from this self-generated negative teacher. Naively applying unlearning objectives to achieve this divergence is problematic, as flawed
reasoning tokens are confounded with basic linguistic
tokens; indiscriminately penalizing both risks catastrophically degrading the model's foundational
language capabilities. We resolve this by designing a dynamic gating mechanism that automatically identifies and isolates
reasoning-critical
tokens, ensuring gradient updates target only behavioral flaws while preserving the model's linguistic priors. Empirically, NSD consistently outperforms OPSD and other label-free, self-bootstrapping reinforcement learning (RL) baselines.
Read the full abstract ▾
All authors Rongcan Pei, Zhepei Wei, Shuyao Xu, Xinyu Zhu, Wei-Lin Chen, Yu Meng
Machine Learning · Cross-listed from stat.AP
Geospatial Foundation Models Capture Health-Relevant Dimensions of Place Beyond Conventional Social Risk Indices
Nathaniel Hendrix, Carl Y. Zhang, Chris Heitzig et al. (+2 more)
Area-based social risk indices summarize residents' socioeconomic conditions but incompletely capture physical features of place that may affect health. We evaluated whether numerical representations of physical place produced by four geospatial foundation model families from 2022 satellite data explained residual variance in tract-level associations between the Area Deprivation Index, Social Deprivation Index, and Social Vulnerability Index with health outcomes. We used LightGBM to predict variables from the American Community Survey and 40 chronic disease and health-behavior outcomes from CDC PLACES across 82,646 census tracts in the contiguous United States, evaluating performance across 10 held-out states. Among survey variables, models were moderately predictive of some variables including housing type (R-squared up to 0.54) but weak for disability, unemployment, and income disparity. For health outcomes, models explained up to 54% of variance left unexplained by social risk indices, with the largest gains for annual checkups, arthritis, and high blood pressure. Mean total variance explained by geospatial foundation models across the 40 health-related outcomes increased from 0.31 in the smallest tract-size decile to 0.39 in the largest. Geospatial foundation models capture health-relevant features of place not represented by conventional social risk indices and may usefully augment them in epidemiological analyses.
Read the full abstract ▾
All authors Nathaniel Hendrix, Carl Y. Zhang, Chris Heitzig, Andrew Bazemore, David H. Rehkopf