Machine Learning
Graph Coarsening Enables Escaping Local Optima in Compute-Constrained Nonconvex MAX-CUT
M. Harper Langston, Pierre-David Letourneau, Dalton Jones et al. (+7 more)
Because its binary optimization landscape contains exponentially many local optima, MAX-CUT is highly sensitive to initialization. We study Particle
Polynomial Relaxation (PPR), a moment-based nonconvex continuous reformulation possessing an advantageous optimization landscape. We warm-start PPR with Combinatorial Multigrid (CMG), a spectral graph-coarsening method that produces structure-aware initial solutions in near-linear time without training data. On t2g20 ($n=400$),
benchmark instance with known exact optimum, the combined CMG+PPR pipeline reaches $97.5\%$ of optimum on average and $98.4\%$ on the best run. An extended variant with discrete local-search refinement reaches the published optimum on all sixty Biq Mac instances evaluated by a recent neural branch-and-bound method. Using a sparse CSR representation, the same approach scales to G-set instances with up to $20{,}000$ vertices reducing
memory requirements by up to $3333\times$ while maintaining a consistent improvement over the CMG warm start. Gains are largest on sparse weighted graphs; on dense graphs we complement the pipeline with spectral sparsification.
Read the full abstract ▾
All authors M. Harper Langston, Pierre-David Letourneau, Dalton Jones, Harris Teague, Peter Scott, Mingu Lee, Richard Lethin, Chris Lott, Roland Memisevic, Fatih Porikli
Machine Learning
Loss Curvature as a Sampling-Free Proxy for Memorization in Diffusion Models
Shashank Sule, Xinyue Yu, Boonam Shin et al. (+3 more)
Understanding the mechanism behind why
diffusion models memorize their training data remains an open question in generative modeling. In this paper, we examine this question from two perspectives: (a) the model-level perspective concerning the onset of memorization in a model during training, and (b) the sample-level perspective concerning the proclivity of an individual sample to be memorized by a given model. We show that the so-called input loss
curvature, or the Frobenius norm of the Hessian of the
diffusion model's training loss, is an effective metric for quantifying both perspectives on memorization. In particular, we empirically demonstrate that input
curvatures averaged across training and validation samples can identify when a model is likely to start regurgitating its training data when tracked over training time. However, this metric is not sufficient for measuring an individual sample's likelihood of being memorized. To address this perspective, we introduce a counterfactual definition of memorization in
diffusion models, extending the analogous notion from supervised learning proposed by [Feldman, 2019]. We prove that this counterfactual memorization is controlled by the input
curvature of the leave-one-out model, thus establishing a direct theoretical link between a sample's local loss landscape and its generation likelihood. We validate our derived bounds through several empirical experiments and show that input-dependent
curvature provides a proxy for memorization.
Read the full abstract ▾
All authors Shashank Sule, Xinyue Yu, Boonam Shin, Yuxuan Liu, Abolfazl Hashemi, Hayden Schaeffer
Machine Learning
Tracing Morphology, Semantics, and Divergence in Language Acquisition
Forough Shirin Abkenar, Haoshuo Fu, Liang Feng et al. (+2 more)
Understanding the dynamics of
language acquisition in Large
Language Models (LLMs) is crucial for uncovering how statistical representations evolve into structured linguistic capabilities. However, most existing studies focus primarily on fully trained models, leaving the developmental trajectory across intermediate training stages largely underexplored. In this work, we leverage intermediate checkpoints from open-source models (e.g., Pythia and OLMo) under teacher-forcing evaluation to systematically investigate
language acquisition across training steps and model scales. We focus on four core characteristics: morphological structure acquisition,
embedding drift, repetition dominance, and co-occurrence effects. Our findings reveal that models acquire morphological structures well before semantic
embeddings stabilize. Furthermore, we show that training
dataset size has a positive impact on divergence recognition, while co-occurrence during pre-training and repetition during inference have a negative impact on divergence recognition and associative link generation.
Read the full abstract ▾
All authors Forough Shirin Abkenar, Haoshuo Fu, Liang Feng, Ning Yan, Masood S. Mortazavi
Machine Learning
Lag: A Mean-Field Optimal Policy for Online Assignment with Reusable Resources
Keerthana Gurushankar
We study online task assignment with reusable resources. This problem arises in peer reviewer assignment, skill-based call routing, rideshare matching, emergency dispatch, and crowdsourcing: Tasks arrive according to a
geometric process and must be immediately assigned to servers that become temporarily busy, yielding rewards that depend on the task-server pair. The goal is to maximize long-run average reward. We cast the problem as a contextual restless multi-armed bandit (CRMAB). We derive an occupancy-measure LP (OCC-LP) that upper-bounds the reward of any work-conserving policy. From its dual we obtain Lag, a closed-form index policy. We introduce a scaling regime in which the number of copies of each server and the number of arrivals per step grow together. This regime yields the first deterministic fluid limit for a contextual bandit. In it, Lag is asymptotically optimal. Finally, we evaluate our
algorithms on peer review
datasets and call center routing. The Lagrangian index achieves 83-99% of the LP upper bound across all load levels. Moreover, the Lagrangian index policy significantly outperforms the natural greedy assignment scheme, improving reward by up to 12.3% on journal data and 36% on structured synthetic instances.
Read the full abstract ▾
Machine Learning
Who Should We Listen to More? Welfare-Aware Preference Acquisition for Pluralistic Alignment
Ruomeng Ding, Tianwei Gao, Lianrui Geng et al. (+2 more)
Pluralistic
alignment needs to accommodate diverse preferences under limited human feedback. Using AI simulators to label preference comparisons reduces annotation costs, but variation in simulation accuracy across individuals can leave some people's preferences poorly represented in training. We study how to allocate human feedback across people to construct training data for pluralistic
alignment, and how this allocation affects the trained policy. To make this allocation explicit and controllable, we define a social welfare objective over individual supervision utilities that measure label quality. Each query's marginal welfare gain depends on its expected supervision improvement, the recipient's current utility, and application-designated priority. Welfare-Aware Preference Acquisition (WAPA) uses auxiliary reference models to estimate query gains and individual utilities, then selects queries by estimated marginal welfare gain while updating recipient utilities. We prove exact optimality of the acquisition
algorithm for the estimated welfare objective under a fixed query budget and model predictions. Experiments on six Roleplay collections and four OpinionQA regions demonstrate improved supervision and controllable allocation tradeoffs. Policy training on four Roleplay collections improves preference prediction for both seen and unseen individuals. When prioritizing a designated group and poorly represented individuals, acquiring human labels for 20% of comparisons recovers 75-86% of the average predictive utility gap between policies trained entirely on AI-generated labels and entirely on ground-truth labels.
Read the full abstract ▾
All authors Ruomeng Ding, Tianwei Gao, Lianrui Geng, Mohit Bansal, Zhun Deng
Machine Learning
How Many Ranks Does a Token Need? A Noise-Floor Squelch for Rank-One LoRA Experts
Mei Okonkwo, Owen Castellanos, Ingrid Solberg et al. (+1 more)
Low-rank adaptation (LoRA) applies every rank-one component of its update to every
token, and mixture-of-experts variants replace this with a learned router that activates a fixed number of components. We argue that the number of components a
token needs is a detection question with a classical answer. A
token's responses to the rank-one components of a trained LoRA consist of a few large values on a Gaussian noise floor whose width matches that of an untrained adapter, and the floor carries cross-talk rather than adaptation. We introduce SQUELCH, a router-free mixture of rank-one experts that (i) soft-thresholds each
token's responses at a multiple of its own robust noise level, so the active rank varies per
token; (ii) chooses that multiple for each module by Stein's unbiased risk estimate, so no rank budget is specified; and (iii) replaces the resulting marginal keep-or-drop decisions by one explaining-away step on the value Gram
matrix, which suppresses redundant co-active components and recruits complementary ones. SQUELCH adds no trainable parameters to LoRA. We prove that the squelch is within a logarithmic factor of the oracle input-dependent allocation, that the selected threshold is risk-consistent, and that explaining away is a descent step toward the joint sparse code whose error is controlled by the coherence of co-active components. On commonsense
reasoning, knowledge, math, code, and GLUE with LLaMA-3.1-8B, Qwen3, and DeBERTaV3, SQUELCH improves over LoRA by 2.2–3.2 points and over the strongest mixture baseline by 0.8–1.2 points while activating 8.6 of 32 components per
token on average.
Read the full abstract ▾
All authors Mei Okonkwo, Owen Castellanos, Ingrid Solberg, Yusuf Demirel
Machine Learning · Cross-listed from math.OC
Optimization for One, Feasibility for Most: Decoupling Optimization from Feasibility in Chance-Constrained Programs
Justin Dumouchelle, Shunyu Yao, Beste Basciftci et al. (+1 more)
Chance-constrained
stochastic programs (CCSPs) require solutions that satisfy uncertain constraints with high probability. Standard approaches either embed scenarios directly into optimization by considering sample average approximation (SAA)-based reformulations, growing prohibitively expensive and often failing to find feasible solutions; or employ conservative reformulations that guarantee feasibility but sacrifice objective quality by design. We propose ACCESS (Adaptive Chance-Constrained Exploration via Structured Sampling), a structured sampling framework that decouples optimization from feasibility evaluation. First, high-quality candidate solutions are generated by solving many inexpensive single-scenario subproblems, avoiding sampling directly in the constrained high-dimensional solution space. Second, feasibility is assessed separately via large-scale batched simulation over a massive number of scenarios, which is naturally amenable to GPU parallelism. That reduces the search from a high-dimensional solution space to a two-dimensional parameter space controlling conservatism and scenario diversity; while concentrating on candidates near the feasibility boundary, where high-quality feasible solutions lie. We also introduce a variant, ACCESS($\delta$), which provides certified feasibility guarantees with probability $1-\delta$. Experiments across three
benchmark classes and five scenario distributions show that ACCESS matches or exceeds the best objective found by any method in nearly every setting, while ACCESS($\delta$) is empirically feasible on every instance. The conservative reformulations remain several percent worse in objective, and SAA is frequently infeasible while taking an order of magnitude longer on the harder
benchmarks.
Read the full abstract ▾
All authors Justin Dumouchelle, Shunyu Yao, Beste Basciftci, Thiago Serra
Machine Learning · Cross-listed from cs.CV
How Far Can Coding Agents Go in Visual Generation?
Hengguang Zhou, Yuanhao Ban, Jiaqi Feng et al. (+3 more)
Modern image generation largely relies on specialized generative models that leave the process of image construction implicit in learned parameters. We explore an alternative: general-purpose coding
agents that construct images through executable programs. Our study of zero-shot drawing on AgentDrawBench, a 200-
prompt benchmark, reveals a distinctive capability profile.
Agents excel at preserving structure and satisfying precise visual constraints, achieving competitive results in domains where compositional accuracy matters more than photorealistic appearance, while remaining limited in perceptual richness. Their visual expressiveness varies with the drawing interface, and a qualitative study on reconstruction shows how procedural skills extracted from human demonstrations support different artistic media. We then examine code as an external visual
language of thought: a structured, executable representation that
agents can construct and inspect. We test it in visual question answering task and find reconstructed visual code retains substantial information for question answering and, in some domains, supports higher accuracy than the original image. These abilities raise a further question: do
agents draw on a shared internal representation of scene content beyond a particular drawing program? We explore an internal visual
language-of-thought hypothesis, in which a shared, compositionally structured representation supports visual operations across drawing
languages. Cross-harness drawings and images assembled from independent single-pixel queries exhibit related layouts and tones.
Agents also maintain self-generated room arrangements across viewpoints after visible drawings are removed. These results demonstrate that the coding
agent would maintain a consistent and coherent world inside its imagine even before the harness gets involved and the generation starts.
Read the full abstract ▾
All authors Hengguang Zhou, Yuanhao Ban, Jiaqi Feng, Xiaohuan Pei, Justin Cui, Cho-Jui Hsieh
Machine Learning · Cross-listed from stat.ML
Discrete Diffusion Models are Minimax Optimal Under Sparsity
Luca Blunda, Tyler Farghly, Umut Simsekli et al. (+1 more)
Discrete
diffusion models have emerged as a leading alternative to autoregressive models for generating text and other categorical data, yet their theoretical foundations remain less developed than those of their continuous counterparts. For continuous
diffusion models, a prominent line of theory suggests that these models can adapt to low-dimensional structure in the data, thereby improving statistical efficiency and generalization. In this work, we investigate the discrete analogue of this phenomenon by studying the statistical rates of discrete
diffusion models under sparsity assumptions. By analyzing a nonparametric
estimator derived from a regularized score-
entropy objective, we establish minimax-optimal rates in the sparse regime. The proposed regularization can be viewed as an adaptation of label smoothing techniques introduced for classification, and does not require the knowledge of the sparsity parameter. This shows that discrete
diffusion models can adapt intrinsically to sparsity.
Read the full abstract ▾
All authors Luca Blunda, Tyler Farghly, Umut Simsekli, Alain Durmus
Machine Learning
IR-PRM+ with Previous-Utility Prediction: Constant Regret and Best-Iterate Convergence
Michael Sullins and Ian A. Kash
Regret matching (and its variants) and counterfactual regret minimization are central
algorithms in practice for normal-form and extensive-form game solving. Yet these have historically lacked the stronger theoretical convergence guarantees available to optimistic and extra-gradient methods. Recent work introduced IREG-PRM+, a regret matching variant which does have strong theoretical convergence guarantees. However, it relies on a simultaneous, extra-gradient update. IREG-PRM+ is an instance of the more general IR-PRM framework, where empirically optimistic and alternating updates often have better performance. However, the prior extra-gradient analysis does not extend to these cases. We introduce a new approach that covers both settings. For simultaneous and alternating IR-PRM+ in finite two-player zero-sum
matrix games, previous-utility prediction recovers the constant-regret and convergence-rate guarantees previously obtained with extra-gradient prediction: external regret bounded independently of T, an O(T^{-1}) average duality gap, and an O(T^{-1/2}) best-current-iterate gap. Moreover, the current duality gap vanishes and the current profile converges to the Nash
equilibrium set. The key step bounds the component of consecutive strategy movement not controlled by the existing IR-PRM+ regret inequality using the relative growth of the nondecreasing regret-vector norm rho_t. Coupling this bound across the two players yields recurrences whose effective coefficient scales as 1/(rho_t^X rho_t^Y), preventing unbounded norm growth. We extend the analysis to finitely many simplex learners under a weighted Minty condition and, through the corresponding CFR variant IR-PCFR+, recover analogous guarantees for finite two-player zero-sum perfect-recall extensive-form games under the same full-reach condition as prior work.
Read the full abstract ▾
Machine Learning
Scaling Cheap Synthetic Pretraining Improves Indoor Radio-Map Prediction
Khoren Petrosyan, Artashes Mkrtchyan, Rafayel Mkrtchyan et al. (+2 more)
Real measurements are too limited to train an accurate indoor pathloss predictor that works across a wide range of buildings. Large-scale synthetic pretraining offers a way forward, but real building layouts are also scarce, and accurate ray-tracing simulators are slow, expensive, and often proprietary. We address both problems by introducing a generator that automatically creates diverse indoor layouts and a fast, open-source approximate ray tracer that computes
radio maps for them. Using this pipeline, we generate 128M synthetic samples, pretrain ResNet-101 and Swin-T encoder-decoders, and fine-tune them on high-quality
benchmark data. Despite simulator mismatch, our ResNet-101 model reduces error by 15% on average relative to the best known methods across five indoor
radio-map prediction tasks and improves transfer on a small real-measurement
dataset. Both architectures show predictable data-scaling trends: power laws fitted on 2M-16M samples predict the mean normalized downstream RMSE at 32M-128M with 3.33% mean absolute percentage error for ResNet-101 and 7.10% for Swin-T. These results show that pretraining on large amounts of cheap simulated data can improve indoor
radio-map prediction, even when real measurements and high-quality simulations are limited.
Read the full abstract ▾
All authors Khoren Petrosyan, Artashes Mkrtchyan, Rafayel Mkrtchyan, Hrant Khachatrian, Theofanis P. Raptis
Machine Learning
Data Density Scaling Laws for Image Self-Distillation
Alvard Barseghyan, Ani Vanyan, Hakob Tamazyan et al. (+1 more)
Current scaling law formulations suggest that increasing the number of unique samples in a
dataset improves model performance, while repeated exposure to the same samples provides diminishing value. In this work, we examine this assumption for image self-
distillation models. Through systematic experiments across diverse
datasets, namely Maxar, SATLAS, Waymo, nuPlan, Ego4D, ImageNet, and YFCC15M, we demonstrate that the effects of sample repetition on downstream performance are non-trivial and
dataset-dependent. We identify settings where repeatedly training on a smaller random subset can outperform training on a larger set of unique samples. This finding highlights a limitation of existing scaling laws. To address this gap, we propose a modified version of a Chinchilla-style scaling law that explicitly accounts for two factors: the number of unique samples and repetition per sample. This formulation provides a framework for understanding data-compute trade-offs and improves prediction in some regimes where repetition plays a central role, although its benefits are not consistent across
datasets. Finally, we examine how subset selection and augmentation affect downstream performance.
Read the full abstract ▾
All authors Alvard Barseghyan, Ani Vanyan, Hakob Tamazyan, Hrant Khachatrian
Machine Learning
PMO-Dock: Benchmarking Docking, Specificity, and Generalization in Molecular Optimization
Gor Simonyan, Tatevik Abrahamyan, Narek Abelyan et al. (+2 more)
The Practical Molecular Optimization (PMO)
benchmark standardized evaluation in molecular optimization, but it is built on simple property-based oracles that do not capture structure-based drug design. As the field has moved to docking-based objectives, evaluation practice has become unstandardized, and different methods report results on different docking tasks, often without a shared protocol for fair comparison or sample-efficiency constraints. To address this, we introduce Practical Molecular Optimization for Docking (PMO-Dock), a
benchmark and protocol for docking-based optimization consisting of 25 tasks covering hit generation, lead optimization, and a new specificity task requiring strong on-target binding while penalizing off-target interactions. The protocol separates model development from final evaluation: validation tasks are used for hyperparameter tuning, and test tasks are held out for final evaluation and are not used during development. Evaluation on the test tasks is done under a strict oracle budget to reflect realistic drug-discovery settings. We
benchmark four diverse high-performing methods spanning different optimization paradigms, Saturn (reinforcement learning), GenMol (discrete
diffusion), Genetic-guided GFlowNet, and ChemLlama (LLM-based evolutionary search). Our results show no universally best method across tasks, additional analyses of hyperparameter transfer (cross-task and cross-category) and sensitivity show that transfer is often limited and varies by method, motivating method-dependent selection strategies. This
benchmark provides a concrete standard for evaluating generalization to unseen tasks and sample efficiency under a limited test-time oracle budget.
Read the full abstract ▾
All authors Gor Simonyan, Tatevik Abrahamyan, Narek Abelyan, Tigran Fahradyan, Hrant Khachatrian
Machine Learning · Cross-listed from cs.AI
Discovering Undisclosed Concepts in LoRA Adapters for Diffusion-Based AIGC Models
Yupeng Qiu, Han Fang, Dhananjai Sharma et al. (+1 more)
Low-Rank Adaptation (LoRA) has made
diffusion model personalization affordable and widely shareable, but it also creates moderation risks when harmful or problematic content is not faithfully disclosed. Hence, it is desired to develop a mechanism that, given a LoRA adapter, discovers the concept it was trained to generate, so as to support downstream investigation. We characterize the target concept through the conditioning input that most clearly exposes what the LoRA has learned. Such an input should induce coherent LoRA outputs that are distinctive from those of the base model and strongly align with the learned LoRA subspace. We therefore formulate concept discovery as searching for the conditioning input that jointly maximizes output coherence, LoRA-induced distinctiveness, and subspace
alignment. Even with a clear objective, however, optimizing it over an open-ended input space is prohibitively expensive, since reliable evaluation of coherence and distinctiveness requires repeated
diffusion rendering. Crucially, we observe that an input need not be rendered to determine whether it is promising. Its
alignment with the learned LoRA subspace provides a lightweight signal for localizing promising candidates before visual verification. Based on this insight, we develop a two-stage framework for text-to-image LoRAs. Subspace-Guided Evolutionary Search (SGES) uses render-free subspace
alignment to explore candidate
prompts and retain a compact shortlist, while Paired Visual Identification (PVI) renders only these candidates and compares the LoRA and base-model outputs to identify those exhibiting coherent and distinctive generation behavior. This design enables efficient and reliable discovery of previously unknown concepts encoded in LoRA adapters.
Read the full abstract ▾
All authors Yupeng Qiu, Han Fang, Dhananjai Sharma, Ee-Chien Chang
Machine Learning · Cross-listed from cond-mat.mtrl-sci
Virtual-Alloy Physics-Aware Semantic System for extrapolative prediction of perovskite solar cells
Qiwei Han and Chi Zhou
Existing machine-learning models for perovskite solar
cells struggle to predict devices made from new materials. Most treat device layers as arbitrary categorical labels and represent perovskite absorbers either through featureless composition vectors or through rigid
crystal graphs that require a pre-solved unit
cell. As a result, they cannot generalize to unrecorded
alloys or unfamiliar
cell architectures. We present the Virtual-
Alloy Physics-Aware Semantic System (V-PaSS), an architecture built around the physical continuity of perovskite solid solutions. V-PaSS represents arbitrary mixed-cation, mixed-halide absorbers by interpolating relaxed
crystal graphs of 23 pure $ABX_3$ endpoint compounds via Vegard's law with learned bandgap bowing. For the device stack, it couples layer sequence and
cell polarity (n-i-p or p-i-n) with interfacial band offsets and encodes chemical descriptions with a scientific
language model. Evaluated under an institution-grouped protocol where test devices come exclusively from unseen laboratories, an untuned default configuration of V-PaSS achieves the lowest error among fourteen tested models (2.861% MAE, a 5.2% relative reduction over a gradient-boosted baseline at 3.017%, and outperforming the best tuned
benchmark at 2.912%). V-PaSS pulls furthest ahead where test devices differ most from training: reducing error by 15.7-19.0% on unseen absorber
alloys, by 16.1% on mixed absorbers from new laboratories, and widening its lead over the baseline by 3.6$\times$ on devices with unobserved transport layers. Combined with tree ensembles, V-PaSS sets a new state of the art across all five
benchmark splits.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CR
Limits of Statistical Assurance for Embodied and Agentic AI under Adaptive Adversaries
Inderjeet Singh
Finite safety assessments cannot exclude an adversary that preserves unattacked behavior until certification and activates afterward. We characterize this limitation for embodied and tool-use AI. An adversary that matches the complete observation law until observable, irrevocable certification and retains intervention access afterward attains $v\mathbb{E}[(n-S_0)_+]$ expected post-certification events over $n$ trials, where $v$ is the active event probability and $S_0$ the unattacked certification time. The result covers adaptive and privately randomized evaluators. Switching at certification is exactly optimal for a fixed likelihood-ratio test under memoryless, event-increasing interventions without a binding intervention budget. A positive clean-certification requirement forces linear worst-case expected excess when intervention raises event rates over an authorized deployment horizon, even if certification is deferred to the deadline. In simulated RoCo collaboration, switching yields commanded-separation violations in 34 of 36 post-certification episodes. Preregistered randomized AgentDojo trials estimate 1.98 additional security violations per four-task continuation (95% interval [1.67, 2.29]). An
adversarial language-model controller infers hidden certification from noisy evaluation receipts and times intervention accordingly. With permanent revocation, a mixture likelihood-ratio monitor bounds cumulative excess above a prespecified reference rate at square-root order, up to logarithmic factors, on every sample path. A single fixed alternative can permit linear expected excess under stationary attacks. Against a controller observing the monitor's rule and statistic, live mixture monitoring reduces violations from 12.00 to 3.81 per 24 assigned task opportunities. Revocation also reduces tasks executed.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CR
Physical Layer Anomaly Detection via Adaptive Gaussian Mixture Model
Mamady Delamou, Luan Chen and Arsenia Chorti
Physical-layer spoofing attacks pose a critical threat to wireless localization systems by manipulating channel state information (CSI). Detecting such attacks becomes particularly challenging when the attacker and the legitimate node are spatially proximate, as their CSI characteristics become increasingly similar. To address this challenge in indoor scenarios, we propose an adaptive Gaussian mixture model (GMM)-based detection framework that learns legitimate CSI patterns and detects spatial inconsistency through statistical anomaly detection. Using a massive multiple-input multiple-output (
MIMO) experimental
dataset with 64 uniform linear array (ULA)
antenna and 5 mm spatial resolution, we evaluated our detector against CSI spoofing attacks with spatial displacements ranging from 50 mm to 450 mm. The adaptive GMM achieves 95-98% detection rate for displacements above 150 mm while maintaining 1% false alarm rate. We
benchmark our approach against conventional GMM, local outlier factor (LOF), one-class support vector machine (SVM) and k-nearest neighbors (k-NN) classifiers, demonstrating overall superior detection performance of our proposed classifier.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Semi-Amortized Bayesian Inference with Reusable Local Posteriors
Niek Stevenson, Paul-Christian Bürkner, Andrew Heathcote et al. (+1 more)
Amortized
Bayesian inference uses simulations to perform
Bayesian estimation when likelihoods are expensive or analytically unavailable. However, learning the full joint posterior is challenging in high-dimensional models with complex dependencies. We introduce a semi-amortized method that divides parameters into smaller blocks, and uses conditional independence to identify the information needed to infer each block. We train neural
estimators for blocks whose posterior updates are computationally expensive or frequently repeated. Each
estimator learns from simulations of the relevant model components to approximate the block's full conditional posterior, given the relevant observations and the values of the variables on which it depends. A Markov chain then alternates direct draws from these
estimators with conventional updates of the remaining variables. Each neural update uses the current conditioning values and requires neither likelihood evaluation nor an acceptance-rejection step. Where the same conditional structure recurs, one
estimator can be reused across blocks and
datasets, with an output dimension determined by the block rather than the full model. We illustrate the approach in high-dimensional and computationally expensive hierarchical and spatiotemporal models.
Read the full abstract ▾
All authors Niek Stevenson, Paul-Christian Bürkner, Andrew Heathcote, Stefan T. Radev
Machine Learning · Cross-listed from stat.ML
Wong–Zakai Convergence and FNO Learning for Stochastic Volterra Equations
Wonjae Lee, Jiuk Jang and Hyungbin Park
This paper studies Wong–Zakai convergence and
Fourier neural operator learning for
stochastic Volterra integral equations (SVIEs) describing random systems with
memory. A Wong–Zakai
theorem establishes convergence of pathwise solutions driven by piecewise linear approximations of Brownian motion to the corresponding SVIE solution. This convergence provides a foundation for using mirror-padded
Fourier neural operators to learn the solution operator mapping initial conditions and driving noise to entire solution trajectories. Numerical experiments demonstrate competitive prediction accuracy and generalization to temporal grids finer than those used for training, while assessing the influence of padding and computational cost. As a follow-up to \citet{LKP:26}, this study extends their operator-learning framework to SVIEs through separate convergence and approximation analyses tailored to convolutional
memory.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
Beyond Equation Search: Conjecture-Guided Discovery with SCOPE and ClosureBench
Yuxiao Hu, Qian Li, Ding Wang et al. (+4 more)
Most symbolic regression methods optimize candidate equations, but improving an equation's fit does not establish whether its underlying scientific
conjecture should be retained, revised, or rejected. A
conjecture may admit multiple mathematical realizations, and similarly accurate equations may imply different explanations. This motivates a shift from refining individual expressions to developing and testing the
conjectures that guide them. We introduce SCOPE, a framework that sustains multiple
conjecture-driven research trajectories. Multiple scientist
agents independently pursue
conjectures through equation modeling, question formulation, and active experimentation, then exchange evidence to revise their
conjectures and research directions. Their findings are consolidated into a scientific report linking models, conclusions, and evidence. Existing equation-discovery
benchmarks primarily assess recovered expressions, leaving the evidence and scientific conclusions established through the investigation underexamined. We therefore introduce ClosureBench, an interactive
benchmark of 60 closure-discovery tasks across five scientific domains. ClosureBench focuses on closure-discovery tasks to give equation discovery a concrete scientific purpose. Each task embeds an unknown algebraic relation in a known PDE system, where
agents control experimental conditions in OpenFOAM to acquire new data for equation discovery. Beyond equation accuracy, ClosureBench proposes three metrics to evaluate the discovery process: evidence faithfulness, mechanism insight, and experimental effectiveness. Compared with diverse baselines, SCOPE achieves higher equation accuracy on both ClosureBench and a real-world scientific discovery case, while producing more accurate, reliable, and effective scientific discoveries.
Read the full abstract ▾
All authors Yuxiao Hu, Qian Li, Ding Wang, Siheng Chen, Linfeng Zhang, Jinyue Yan, Yuntian Chen
Machine Learning · Cross-listed from stat.ML
Delay Complexity Dimension: Learning with Delayed and Censored Labels
Madhava Gaikwad
We study PAC learning of binary classifiers when labels arrive after random delays and are censored beyond an observation window $W$. How many samples does this observation model require? We propose a distribution-dependent answer, the Delay Complexity Dimension (DCD). The DCD combines the VC dimension with the exact label-information rate of the observation channel. Every learner needs $\Omega(\mathrm{DCD}/\varepsilon^2)$ samples. An oracle pseudo-label ERM matches this rate on the instance family of the lower bound, up to constants and logarithmic factors. On that family the DCD is within a constant factor of the classical censored-ERM complexity, and whether the DCD rate is achievable where the two separate remains open. The theory also yields a quantization scheme under which delay information lowers the residual label variance, an expedition policy with a $(1-1/e)$ guarantee for its version-space objective, and a phenomenon with no analogue in the no-delay setting. Under a fixed collection horizon, a moderate window can strictly beat immediate labels. Experiments on synthetic data and the Criteo conversion logs support these predictions. A budget-matched test of a companion labeler-diversity prediction is reported as a negative result.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.SD
An Initial Study of Physiological Markers for Deepfake Speech Detection
Vojtěch Staněk, Michal Frič, Kamil Malinka et al. (+1 more)
Conventional deepfake
speech detectors primarily target synthesis artifacts and often degrade under unseen attacks or recording conditions. Human
speech, however, contains physiological markers associated with respiration, phonation, and temporal coordination arising from the
speech-production system that synthesizers may not reproduce faithfully. This motivates the hypothesis that deepfake
speech can be detected by identifying inconsistencies in these patterns, which we test using 36 handcrafted physiological markers organized into respiratory, vocal, and temporal categories. We evaluate the marker vectors with logistic regression and SVM classifiers on pooled English recordings. The best in-domain system achieves a 21.40% equal error rate (EER), while logistic regression achieves 32.77% on the out-of-domain
dataset pair. Since the markers describe specific
speech-production properties, the logistic-regression coefficients indicate whether higher marker values push the classifier toward bona fide or deepfake
speech. These results provide initial evidence that physiological markers offer a useful, interpretable signal for deepfake
speech detection.
Read the full abstract ▾
All authors Vojtěch Staněk, Michal Frič, Kamil Malinka, Anton Firc
Machine Learning · Cross-listed from cs.AI
The Agentic Scientist: Orchestrating Coding Agents for Automated Scientific (Co-)Discovery
Robert Tjarko Lange, Luca Grillotti, Yutaro Yamada et al. (+2 more)
Modern coding
agents are fundamentally reshaping the role of human input for computational research: they write code, launch in silico experiments, analyze their results, and autonomously operate with ever-increasing time horizons. Despite these developments, fully autonomous research
agents suffer from a limited ideation radius and myopic hill-climbing search dynamics. To bridge this gap, we introduce The Agentic Scientist, an end-to-end distributed pipeline for human researchers to conduct computational research alongside modern coding
agents. Building on its predecessor, our system decomposes research into task drafting, idea generation, planning, sandboxed experimentation, result aggregation, baseline and ablation construction, post-hoc analysis, and report writing. Importantly, each step in the pipeline is powered by the next generation of coding
agents and leverages agentic self-verification. The overall research process is driven by a meta-controller that manages parallel research
agents and enforces rubric verification at each stage. This design enables the same orchestration layer to be reused across heterogeneous domains with different execution environments, agentic coding harnesses, and evaluation metrics. Our system can be used across a spectrum of autonomy: either as a co-scientist tool or to perform end-to-end scientific investigation. Within the co-scientist setting, we leveraged a development version of our system to submit a single research paper, which got accepted for publication at ALIFE 2026. We further apply The Agentic Scientist fully autonomously to open problem-solving tasks in machine learning, and demonstrate the system's capability of scientific discovery across different domains of computational research. Our ablations suggest that reliable scientific autonomy depends not only on base model capabilities, but also on the cross-
agent orchestration layer.
Read the full abstract ▾
All authors Robert Tjarko Lange, Luca Grillotti, Yutaro Yamada, Rujikorn Charakorn, David Ha
Machine Learning · Cross-listed from cs.CV
Cepheid: Decoupling Scale from Structure for Cross-Family Motion Completion
Chih-Hai Su, Tsung-Wei Ke and Yu-Lun Liu
Sparse-to-dense motion completion recovers dense mesh motion from sparse vertex displacements. Residual priors trained on one geometry family frequently fail to beat parameter-free harmonic extension off-family. Moreover, cross-family training meets a scaling barrier: residual amplitude varies by more than an order of magnitude across geometry families and collapses as observations become denser. Since this scale depends on unknown target motion, it cannot be measured directly at test time. We decouple residual structure from physical scale by learning a dimensionless prior, estimating amplitude from observables and a lightweight correction fitted only on training sources. Our unified model surpasses harmonic extension across all evaluated families, including unseen cloth and legged bodies, without skeletons, registration, or cross-mesh correspondence. The same prior applies without retraining to point-tracked animal
video and a
robot grasping a soft body. Project page: https://su-terry.github.io/cepheid-website/
Read the full abstract ▾
Machine Learning · Cross-listed from cs.RO
What Matters in Quality-Aware VLA Pretraining: A Benchmark and Empirical Study
Mingxuan Yan, Litian Gong, Dihong Huang et al. (+19 more)
How should demonstration quality be used during
vision-
language-action (VLA) pretraining to improve performance after target-task fine-tuning? We compare weighting, filtering, and conditioning across reward definitions and label granularities using 5,000 naturally suboptimal human demonstrations and 24 target tasks. Within each VLA, we match demonstrations and training budgets. Additional comparisons examine pretraining data size and the roles of quality tags during fine-tuning and inference. Across three VLAs, we find that approach-aware episode conditioning improves mean success over plain behavioral cloning; conditioning and filtering favor episode and chunk labels, respectively, on average, with model-dependent reversals; and constant-tag fine-tuning improves mean success after conditioned pretraining with either tagged or untagged inference. We introduce AXIS-Bench to support these controlled comparisons. Built on crowdsourced simulation demonstrations, it provides recorded object and contact states for recomputing quality labels on shared trajectories. Replayable trajectories support workspace-aligned rendering and user-defined real-world target tasks. In our real-world experiment, simulation pretraining raises success from 9.6% to 31.7%. Project page: https://axis-bench.github.io/.
Read the full abstract ▾
All authors Mingxuan Yan, Litian Gong, Dihong Huang, Zhixu Li, Tianyu Zhang, Wenqian Zhang, Hefeifei Jiang, Peihao Li, Jian Zhang, Mengfei Zhao, Yikai Tang, Hai Zhai, Zehao Wang, Ruijian Liang, Yanjia Huang, Lin Shao, Changliu Liu, Kaiyu Hang, Zhiwen Fan, Masayoshi Tomizuka, Jianfei Yang, Jiachen Li
Machine Learning · Cross-listed from cs.CL
Tev: Scaling Long-Horizon Agentic Environments with Dense Progress Rewards
Wenbo Hu, Shraman Pramanick, Krishna Kumar Singh et al. (+3 more)
Long-horizon
agents have drawn growing
attention for their ability to work autonomously – from hours to days – on economically valuable tasks, a capability essential to automated discovery and recursive self-improvement (RSI). Yet open-source models still struggle on long-horizon tasks, largely for lack of both training data and a working recipe, and scaling both is hard: long, complex tasks are difficult to curate and verify, and their rewards are sparse. We address both. First, we introduce Tev-Gym, the first long-horizon
agent training
dataset with verifiable, dense, progressive rewards – 1,170 verified task environments across 7 domains. It is built on a latent-world-first approach that first constructs a latent world: a complete and correct hidden specification of the task domain, workflow, and expected state after each step. Each task is then progressively rendered from this specification into an
agent-facing environment, enabling controllable complexity and a natural progress metric. Second, we turn this metric into a training recipe: via progress-potential reward shaping (PPRS), we convert progress into a dense per-step reward that improves training efficiency and makes otherwise-intractable tasks trainable. Training with this recipe yields Tev-27B, which achieves substantial gains on the
Agents' Last Exam computing and math sciences domains (ALE-CM), matching GPT-6 Astra and Claude Opus 5 at Pass@5. Despite training only on ALE-related data, it generalizes to the Terminal-Bench (TB) 3.0 and 4.0 subsets, surpassing GLM 5.2 and Gemini 3.7 Flash, and reaches Claude Opus 4.8-level performance on middle-horizon tasks such as TB 2.1.
Read the full abstract ▾
All authors Wenbo Hu, Shraman Pramanick, Krishna Kumar Singh, Nanyun Peng, Kai-Wei Chang, Yong Jae Lee
Machine Learning · Cross-listed from hep-lat
Data Free Training of Diffusion Models for Lattice Field Theories
Dennis Grinwald, Ankur Singha, Khaled Kahouli et al. (+2 more)
Sampling
equilibrium field configurations remains a major computational bottleneck in
lattice field theory, motivating the development of generative samplers that can produce independent configurations.
Diffusion models are promising for this purpose, but their standard denoising score matching objective requires samples from the target distribution, seemingly reintroducing the original sampling problem. Here, we show that
diffusion models can instead be trained from their own generated samples using the target score identity, provided access to the unnormalized probability density. Importance-weighted formulations of this identity yield a consistent target throughout training, while unweighted variants define a self-consistent fixed-point training scheme. We
benchmark the different approaches in two-dimensional \(\phi^4\) theory and identify effective choices that enable efficient training and sampling across the symmetric, critical, and broken phases. Ultimately, our data-free training approach achieves comparable performance to training on HMC reference data.
Read the full abstract ▾
All authors Dennis Grinwald, Ankur Singha, Khaled Kahouli, Karl Jansen, Shinichi Nakajima
Machine Learning · Cross-listed from cs.CL
DoubtLess: Training-Free Contrastive Decoding for Efficient Reasoning
Minghao Luo and Liang Chen
Reasoning models achieve strong performance through long chains of thought (CoT), but often spend more
tokens than needed to solve a problem. Existing ways to shorten the CoT require additional training for each model, add a fixed steering vector to the model's hidden states, or use decoding heuristics such as stopping early or banning words like "wait", which can reduce accuracy. We propose DoubtLess, a training-free decoding method in which a much smaller
reasoning model steers the large one towards brevity. The small model reads the large model's CoT so far twice, once prompted to be brief (the brief thinker) and once prompted to be verbose (the verbose thinker), and the difference between its two logit vectors, scaled by a strength, is added to the large model's logits at every step of the CoT. On seven
benchmarks of competition mathematics, graduate-level science and competitive programming, DoubtLess shortens the CoT of Qwen3-32B by 31% with unchanged accuracy, outperforming five inference-time baselines. We also show that DoubtLess reduces thinking
tokens per turn by 20% in a self-debugging coding
agent with unchanged task performance. It also generalizes to two other model families with unchanged accuracy. When combined with training-based compression, it further shortens the CoT of a model already trained for brevity. Ablations and behavioral analysis show that DoubtLess reduces repeated checking, with most
token savings coming after the correct answer first appears in the CoT. Our code is available at https://github.com/leoluolol/DoubtLess.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Jiaru Zou, Yunzhe Qi, Sirui Chen et al. (+5 more)
How can we build an
agent that thinks beyond human
language, yet still acts through it in the external environment? Modern LLM
agents still reason and interact through text, while latent
reasoning offers an efficient internal medium but lacks native tool access and
reasoning transparency. We bridge these two directions with Hybrid
Reasoning Agents (HybridAgents), which dynamically navigate between latent
reasoning and text-based interaction throughout a unified generation process. Architecturally, a lightweight HybridLink enables seamless transitions between text and latent modes, while the
agent's continuous working
memory preserves lossless hybrid context across multiple turns. These designs preserve tool-integrated agentic capabilities while making latent thoughts directly inspectable. Building on this architecture, we further develop dedicated post-training
algorithms for hybrid
reasoning: HybridSFT stabilizes tool-integrated hybrid generation by teaching the backbone to coordinate with HybridLink, then HybridRL scales
reasoning by learning when to think in latent space and when to act through text. We evaluate HybridAgent at 3 model scales across 11
benchmarks covering mathematics, science, medicine, search, and agentic
reasoning. Compared with frontier LLM
Agents and agentic frameworks spanning both text-based and latent methods, HybridAgent consistently improves performance by 10.4%, while delivering 35.5%-54.7%
token reduction, 1.8x-2.6x faster inference, and 2.7x-3.7x cost savings. Code and data are available at https://github.com/togethercomputer/HybridAgent.
Read the full abstract ▾
All authors Jiaru Zou, Yunzhe Qi, Sirui Chen, Federico Bianchi, Shang Zhu, Pan Lu, Jingrui He, James Zou
Machine Learning
Exact Topological Compliance: Generating Persistence-Equivalent Graphs
Mattie Ji, Indradyumna Roy and Vikas Garg
Persistent homology summarizes the topological
evolution of a graph as persistence diagrams (PDs). Generating graphs whose
topology matches a target descriptor is increasingly important, yet existing
topology-aware generative models rely on soft regularization, producing graphs whose PDs may only be approximately similar. We initiate a principled investigation of the generation problem: produce graphs that realize a target PD exactly. We propose two approaches guided by the implicit local and global constraints a target PD imposes. The local approach builds degree-based PD-equivalent graphs iteratively, and we prove that every output realizes the target PD and that every degree-based PD-equivalent graph is reachable. However, as shown thereafter, local generation necessarily trades off between sample diversity and reaching infeasible states. We therefore recast generation as a global constraint satisfaction problem with a complete encoding solvable by modern constraint programming solvers. This is applicable to any vertex-based permutation equivariant filtration scheme and is minimal under the degree filtration. Experiments on four standard graph generation
benchmarks confirm that both methods realize the target PD exactly. Together, our methods provide a principled treatment of strict
topology-compliant graph generation. Code is available at https://github.com/Aalto-QuML/ExactCompliance.
Read the full abstract ▾