Machine Learning · Cross-listed from cs.AI
Request Order Matters: Cache-History Sensitivity in Selective KV-Cache Reuse for Rolling Agents
Tiffany Gu, Annie Guan, Manshu Huang et al. (+3 more)
Long-running
agents repeatedly call an LLM while retaining most of their document window, evicting old documents, and appending new ones. These rolling updates break exact prefix caching and motivate non-prefix KV-cache reuse with selective recomputation. We show that persistent KV-cache reuse with selective recomputation can be history-dependent: in our rolling-
agent workload, an unchanged
prompt can produce different answers depending on the requests processed before it. At a matched 5% recomputation budget, document-aligned recomputation reduces answer variation across request orders from 69.0% with CacheBlend's
token top-$k$ policy to 26.1%. When each
prompt is evaluated after a different sequence of preceding requests, document-aligned recomputation improves fidelity to full prefill by 34.5-52.5 percentage points over
token top-$k$, while both policies achieve approximately 5.7$\times$ median TTFT speedup. Our ablation study shows that, in our rolling-
agent workload, contiguity is the main factor associated with robust selective recomputation.
Read the full abstract ▾
All authors Tiffany Gu, Annie Guan, Manshu Huang, Nitin Rao, Siddhant Shah, Margaret Capetz
Machine Learning
Selecting Long-Horizon Trajectories for Reliable and Efficient Terminal-Agent Training
Cuong Dang, Hoang Anh Just and Ruoxi Jia
Terminal
agents are commonly trained by imitating long teacher trajectories, yet how much of each trajectory to supervise remains unexplored. We study the supervision horizon, the number of trajectory
tokens retained for training, and show that it is a key design axis for reliability and cost. Reliability improves with longer horizons but saturates: on Terminal-Bench, a 12K-
token horizon solves more tasks than 16K ($29\pm0.7$ vs.\ $26\pm0.8$) while requiring 30% less training time. The horizon also shapes
agent behavior: short horizons cause premature termination, intermediate horizons yield productive error recovery, and long horizons induce over-persistence. We analyze this saturation through a bias–complexity bound, in which longer supervision reduces temporal supervision bias but increases finite-sample estimation error from more heterogeneous late-stage histories. Guided by this analysis, we propose selective long-horizon refinement, which first trains on short prefixes and then refines only on continuations that are most likely under the warm-start model. It consistently outperforms full long-horizon training. At 16K, it raises successful attempts from $110\pm2.7$ to $126\pm2.1$ and tasks solved in at least six of eight attempts from $9\pm0.7$ to $14\pm0.6$; with half of the long-horizon data, it still reaches $122\pm2.4$ while cutting training time by 23%. The gains transfer across
benchmarks, from $64\pm2.6$ to $73\pm2.1$ on Terminal-Bench v2.0 and from $137\pm2.7$ to $155\pm2.2$ on OpenThoughts-TBLite. For long-horizon supervision, selecting the right trajectories matters more than training on all of them.
Read the full abstract ▾
Machine Learning
Usefulness of Quantile-Aware Diffusion Modeling for Highly Imbalanced Tabular Data
Abu Talha, Peng Liu and Souradyuti Paul
Classification problem in the context of highly imbalanced data is a major challenge in many real-world applications (e.g., FinTech, healthcare, etc.). In these cases, the vast majority of instances belong to a single class and a small fraction represent the minority class (often the most critical class). Recently,
diffusion models have emerged as powerful approaches to reduce the degree of “imbalanced-ness” in the
dataset; they work by generating synthetic data by capturing complex data distributions using iterative transformations. However, standard
diffusion models are not inherently suited to highly skewed or heavy-tailed data, due to inbuilt quadratic error loss, which lacks the structural sensitivity to capture rare, extreme values, and minority-class nuances. We propose a novel approach, namely,
Quantile-TabDDPM, based on a
quantile-regularized denoising objective that combines the standard quadratic error loss with a
quantile loss term to explicitly capture rare events while preserving the theoretical grounding of the original denoising objective. We extensively evaluated our approach on a real-world credit card transaction
dataset characterized by extreme class imbalance. The results demonstrate that the integration of
diffusion-based synthetic data generation with a
quantile-regularized denoising objective provides a robust and effective framework for fraud detection in highly imbalanced
datasets.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.RO
Transporting Unsecured Stacked Payloads with a Quadrupedal Robot via Multi-Objective Reinforcement Learning
Nobuo Namura, Masayuki Hiromoto, Kento Uemura et al. (+2 more)
Transporting unsecured payloads with legged
robots over uneven terrain requires balancing locomotion performance and payload stability, since aggressive motion can destabilize the payload even when the
robot remains stable. We study quadrupedal transportation of unsecured stacked boxes on an edgeless torso-mounted board without dedicated payload sensors or active carrier mechanisms. To address this trade-off, we propose Payload-Adaptive Multi-Objective Reinforcement learning for Transportation (PAMORT). PAMORT trains a multi-objective base policy conditioned on a preference vector that weights locomotion and payload-stability reward groups, then trains a weight adjuster on the frozen policy to adapt this preference online from proprioception. In simulation, PAMORT achieves comparable or better overall transportation success than a corresponding single-objective baseline across different payload configurations, including an unseen three-box stack, despite training only with two boxes. Real-world experiments on a Unitree Go2 demonstrate zero-shot transfer to slopes and steps at or beyond the training difficulty, with mean success rates of 0.850 for PAMORT and 0.675 for the baseline across eight tasks. These results demonstrate robust unsecured-payload transportation with online adaptation of the locomotion–payload trade-off from proprioceptive information.
Read the full abstract ▾
All authors Nobuo Namura, Masayuki Hiromoto, Kento Uemura, Hironobu Sasaki, Kanata Suzuki
Machine Learning
Feature identification for parameter extraction and defect detection using machine learning
Yan Guo, Helda Pahlavani, Artem Khachaturiants et al. (+5 more)
Process control of advanced
semiconductor nodes is not only pushing the limits of metrology equipment requirements in terms of resolution and throughput but also in terms of the richness of data to be extracted to enable engineers to finetune the process steps for increased yield. The move towards 3D structures requires extraction of critical dimension parameters from structures which can vary largely from layer to layer. For in-line process control, the necessary automation forces the development of layer and equipment-specific dedicated image processing
algorithms. Similarly, with the increase in
stochastic defects in the EUV era, detection of defects at the nm scale requires the identification of features captured in low resolution to meet the throughput requirements of HVM fabs, which can again lead to custom
algorithm development. With the emergence of ML-based image processing methods, this process of
algorithm development for both cases can be accelerated. In this work, we provide the general framework under which the images obtained from high-speed scanning probe microscopy-based systems can be used to train a network for either feature detection for parameter extraction or defect identification.
Read the full abstract ▾
All authors Yan Guo, Helda Pahlavani, Artem Khachaturiants, Khalid Elsayed, Jakob van de Laar, Erik Simons, Niranjan Saikumar, Hamed Sadeghian
Machine Learning · Cross-listed from cs.CR
Online AutoML: Evaluating Poisoning Attacks on Cost-Utility Adversarial Training Defence Strategy for IoT Networks
Chukwunonso Henry Nwokoye, Khalil El-Khatib and Li Yang
Machine learning (ML)-powered poisoning attack vectors are
adversarial maneuvers whereby an attacker intentionally inserts, corrupts, or alters training data to distort an ML model's learning process. The objective is to diminish model efficacy, instill biases, induce misclassifications, or include concealed backdoors that may be attacked during implementation. In streaming contexts, poisoning attacks pose significant risks since models perpetually update based on incoming streams of data. An assailant may incrementally introduce harmful samples into this data stream, leading the model to assimilate erroneous features over time without timely identification. Therefore, this study is aimed at evaluating the efficacy of the
adversarial training (AT) defense approach against poisoning attacks (label flip and noise injection) using an online AutoML pipeline for Internet of Things (IoT) networks. Specifically, poisoning attacks (label flip and noise injection) were applied to streaming-capable AutoML learners (Hoeffding Tree (HT), Leveraging Bagging (LB), Adaptive Random Forest (ARF), Hoeffding Adaptive Tree (HAT), and Streaming Random Patches (SRP)). Under the strongest poisoning rate (PR = 1.0), AT-SRP achieved the highest F1-score against label flip poisoning (0.904), while AT-LB achieved the highest F1-score against noise-injection poisoning (0.933). Finally, several drift detection methods were used for rolling accuracy and prequential evaluation.
Read the full abstract ▾
Machine Learning
Image resolution enhancement for advanced semiconductor nodes
Lucas Rencker, Omid Tajalizadehkhoob, Khalid Elsayed et al. (+7 more)
Advanced
semiconductor nodes are pushing the limits of feature sizes and require metrology with sub-nm resolution without compromising on the throughput as needed for in-line process control. Recently, high-throughput scanning probe microscopy (SPM) based metrology and inspection tools capable of meeting these needs have been introduced to the
market and qualified for use in HVM. While innovative measurement methods and tool architecture have allowed for a leap of improvement in throughput, the next step in further reducing imaging time can be obtained through the application of machine learning for enhancing the resolution of measured images for extraction of relevant parameters. In this work, we provide the general framework under which a neural network-based resolution enhancer is designed and used for SPM images. We showcase the effectiveness of this framework using measurements performed on Line/Space structures with a pitch of 200 nm. For the reusability of a pre-developed pre-trained model, we additionally leverage transfer learning and show that a new model for slightly differing structures can be re-trained and calibrated with a smaller data set of measurements performed on Line/Space structures with a pitch of 100 nm.
Read the full abstract ▾
All authors Lucas Rencker, Omid Tajalizadehkhoob, Khalid Elsayed, Artem Khachaturiants, Helda Pahlavani, Yan Guo, Jakob van de Laar, Erik Simons, Niranjan Saikumar, Hamed Sadeghian
Machine Learning · Cross-listed from cs.DB
Protocol-Sensitive Evaluation of Log Anomaly Detection: Component Costs and Target-Access Sensitivity on HDFS and BGL
Hang Xiao, Janet Sung, Zhaoyi Li et al. (+2 more)
Protocol choices can change the conclusions drawn from log anomaly detection
benchmarks even when detector settings are fixed. We present a joint empirical study of split construction, representation visibility, and component costs using six fixed count, sequence, and semantic configurations on Hadoop Distributed File System (HDFS) and Blue Gene/L (BGL) logs. Random splits place several configurations near the average-precision ceiling, whereas group-disjoint HDFS and chronological BGL evaluation produce lower scores and different observed orderings. At a fixed BGL cutoff, parser choice spans 0.124 in semantic XGBoost mean average precision while preserving its lead over count XGBoost; the earliest rolling period reverses that ordering. A two-factor cross-system ablation contrasts source-only representations with offline transductive access to unlabeled target templates through the representation corpus and inverse document frequency: HDFS-to-BGL mean average precision moves from 0.191 with source-only access to 0.325 with union-corpus, target-IDF access, and the intermediate conditions reveal direction-dependent interactions in average precision and
retrieval at fixed review budgets. Component-level profiling separates parsing and representation costs from classifier training, prediction, and storage. Together, these findings connect detector comparisons to the test population, preprocessing state, visible information, and measured pipeline stages, and identify the protocol fields needed alongside a score to support interpretable comparisons of log anomaly detection accuracy and resource use.
Read the full abstract ▾
All authors Hang Xiao, Janet Sung, Zhaoyi Li, Gangzhen Qian, Chuhong Xu
Machine Learning · Cross-listed from cs.AI
A Testable Theory of Atomic Features
Kenny Peng, Jon Kleinberg and Nikhil Garg
We develop and test a theory of
language model representations in which there exist atomic features. Our main theoretical insight is that in such a model, sparse dictionaries (e.g., SAEs) of increasing size recover an increasing prefix of the most prevalent atoms in the training data. This "recovery principle" yields three testable predictions: many features in small SAEs are shared by all larger SAEs, SAEs trained on different data share features prevalent in both, and sufficiently large SAEs recover both parent and child features. In contrast to conventional wisdom that SAE features are unstable and "split" as size increases, we find that these predictions hold on SAEs of sizes ranging from 512 to 131,072 trained on two large
embedding models. From a theoretical perspective, our results suggest the promise of a scientific theory of representations based on atomic features. Practically, our results suggest the promise of scaling SAEs.
Read the full abstract ▾
Machine Learning · Cross-listed from math.OC
Dimension-Free Decentralized Nonsmooth Nonconvex Stochastic Optimization
Yuanyu Wan, Lan Xue, Haomin Bai et al. (+2 more)
We investigate decentralized nonsmooth nonconvex
stochastic optimization over a network of $n$ nodes, with the goal of finding an $(\delta,\epsilon)$-Goldstein stationary point. The best existing
algorithm achieves $O(\delta^{-1}(\epsilon^{-3}+d\epsilon^{-1}))$ sample complexity and $\widetilde{O}(\gamma^{-1/2}\delta^{-1}(\epsilon^{-3}+d\epsilon^{-1}))$ communication complexity, where $d$ is the problem dimension and $\gamma$ is the spectral gap of the communication
matrix. However, the
polynomial dependence on $d$ can be a major bottleneck in high-dimensional regimes. In this paper, we propose a novel
algorithm that achieves $O(\delta^{-1}\epsilon^{-3})$ sample complexity and $\widetilde{O}(\gamma^{-1/2}\delta^{-1}\epsilon^{-3})$ communication complexity. The primary technique is an elegant decentralized online-to-nonconvex conversion that reduces the original problem to a decentralized online convex optimization (D-OCO) problem. A key property of our conversion is that its consensus requirements can be inherited directly from the consensus of the underlying D-OCO decisions. In particular, this property enables us to establish an explicit connection between the dimension dependence and the consensus error, which in turn shows that the
polynomial dependence on $d$ can be removed with only logarithmic additional communication.
Read the full abstract ▾
All authors Yuanyu Wan, Lan Xue, Haomin Bai, Tong Wei, Mingli Song
Machine Learning · Cross-listed from cs.AI
Agentic-ZTA: A Multi-Agent Architecture for Autonomous Zero Trust Enforcement
Shovan Roy, Lopamudra Praharaj, Maanak Gupta et al. (+1 more)
Agentic AI is emerging as a promising paradigm for automating complex cybersecurity decisions, yet its use in enforcing zero trust introduces significant challenges in safety, reliability, and policy compliance. This paper presents Agentic AI based zero trust architecture (Agentic-ZTA) that operationalizes the NIST SP 800-207 ZTA architecture control loop through coordinated multi-
agent decision pipeline. In the proposed framework, policy knowledge is embedded into a
retrieval-augmented generation pipeline and retrieved at inference time as top-k relevant policies. Access requests are intercepted by the Policy Enforcement Point (PEP), enriched with contextual metadata. The request context is routed to a policy engine
agent which invokes domain-specialized core
agents first followed by supporting
agents, if further evaluation needed. AI
agents reason over access context, policy constraints and determine trust. The retrieved policies are embedded into
agent prompt during inference time and agentic trust scores are aggregated and evaluated by a trust-
algorithm, producing the final access decision for enforcement under continuous verification. We implement Agentic-ZTA in a testbed and evaluate it on representative access-control use cases scenarios. Our Agentic-ZTA framework achieves 95.0% accuracy, 93.9% precision, and 96.3% recall, and demonstrate the feasibility of enforcing zero trust using AI
agents.
Read the full abstract ▾
All authors Shovan Roy, Lopamudra Praharaj, Maanak Gupta, Bhavani Thuraisingham
Machine Learning · Cross-listed from stat.ML
Isotropic Gaussian Processes Improve Vanilla Bayesian Optimization in High Dimensions
Wei-Ting Tang, Madhav Muthyala and Joel A. Paulson
High-dimensional
Bayesian optimization (BO) often fits Gaussian process (GP) surrogates from far fewer observations than input dimensions. Modern Vanilla BO can perform well in this regime with dimension-aware priors, initialization, and acquisition optimization, but it typically retains automatic relevance determination (ARD), fitting one lengthscale per input coordinate. We study this modeling choice and propose Iso-BO, a controlled modification that replaces the ARD GP with an isotropic GP using one shared lengthscale while keeping the surrounding BO pipeline matched. For radial kernels, we show that the marginal log likelihood (MLL) depends on the inverse-squared ARD lengthscales only through weighted pairwise distances among the observed inputs. The current design can therefore leave some ARD directions exactly invisible or only weakly constrained by the MLL. Iso-BO removes coordinatewise reweighting and fits a single shared scale instead. Lengthscale-fitting and predictive-density diagnostics show that this finite-data effect appears in practice, including when the data-generating process is anisotropic. Across GP-prior, synthetic, and real-world
benchmarks, Iso-BO often improves over matched modern Vanilla BO and remains competitive with the included high-dimensional BO baselines under the tested budgets. Stress tests also show the expected boundary wherein sufficiently strong, learnable anisotropy can favor the more flexible ARD model.
Read the full abstract ▾
Machine Learning
AdaSpark: Adaptive DSpark with Online Learning for Tree Verification and N-gram Fill
Liquan Liu, Yifan Zhang and Bowei Xu
Block drafters such as DSpark propose ranked candidates for several positions in one forward pass, and a tree verifier checks them in one pass of the target. The number of rows to verify trades the
tokens a wider tree is expected to accept against the time a wider verify takes. Most schedulers that choose this number take the verify time from a table or model measured before serving, corrected online by at most one scale factor, and take acceptance from the drafter's confidence estimates or from a map fitted offline. AdaSpark learns both quantities while it serves, with no profile, calibration or sweep in advance. It learns which verify widths are worth offering and fits each one's verify time as a function of context. It fits each candidate's acceptance probability to the target's verify outcomes, with the drafter's confidence head as one input, and orders and sizes the tree by that fit instead of by the head. The same model prices n-gram continuations of the request's own text, so drafted and text-derived candidates compete for rows in one best-first order. The width is chosen by pricing time at the long-run decode rate. On single- and multi-turn conversations from six public
datasets, on three dense targets and one mixture-of-experts target, AdaSpark decodes 1.5-3.1x faster than llama.cpp's DSpark with the same drafters. Our imparo engine with AdaSpark is 1.17-1.52x faster than imparo running with a three-
token chain (the default llama.cpp setting); this gain comes from the scheduler alone. Without a width sweep, AdaSpark is never more than 0.3% slower than the best pinned tree width on any dense target or context band. On the mixture-of-experts target it ties the best pinned width, and the other pinned widths from 4 to 16 rows are 5-14% slower.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.SD
Relational Synthesis: Structure-Mediated Concatenative Synthesis for Foley and Retrieval-Augmented Audio Generation
Keren Shao, Ayaka Kawano and Shlomo Dubnov
We ask: given a retrieved source audio $S$ and a separate reference audio $R$, can we synthesize novel audio $Y$ out of this pair $(S,R)$ such that $Y$ remains acoustically consistent with $S$, while not persistently copying segments of $S$ or $R$? The first clause is a well-known goal in Foley audio production, and the second is a well-known issue in neural RAG when $S$ and $R$ are naively injected into neural generators. We show that both clauses can be addressed simultaneously using a method we coin relational synthesis, a variation of concatenative synthesis where target cost is replaced by a relational Gromov-like structural cost. Rather than imitating the content of $R$, relational synthesis exploits it from the "other side of the hill": it transfers the temporal structure and directed amplitude motion of $R$ to reorganize and concatenate the grains of $S$ in a novel manner that protects $S$'s acoustic information. Our experiments show that relational synthesis integrates naturally with neural RAG and produces Foley audio that performs well on metrics measuring temporal agreement, acoustic fidelity, and leakage persistence, while maintaining distribution-level quality and text
alignment.
Read the full abstract ▾
Machine Learning
Beyond In-Distribution Preservation: Recovering Generalization in Quantized VLAs via Vulnerability-Oriented Tuning
Shen Ruan, Wenchang Gao, Jin Wang et al. (+4 more)
Post-training quantization has been shown to preserve VLA performance under standard evaluation conditions, but whether it preserves the full-precision model's robustness and generalization remains underexplored. In this study, we systematically study the robustness and generalization of post-quantized VLA policies under environmental disturbances. Empirical results show that quantized policies can become fragile to subtle environmental variations despite retaining comparable in-distribution performance. We further observe that action discrepancies are concentrated in a small subset of rollout states, while teacher guidance has opposite effects depending on discrepancy: it improves generalization at high-discrepancy states but can degrade it at low-discrepancy states. These findings reveal that effective post-quantization recovery requires selectively intervening on vulnerable states rather than globally distilling the student. We therefore propose Policy-Induced Vulnerability-Oriented Tuning (PIVOT-Q), a vulnerability-aware On-Policy
Distillation (OPD) framework that selectively corrects vulnerable states encountered during quantized-student rollouts using the frozen full-precision policy as a teacher. PIVOT-Q identifies vulnerable states using discounted accumulated discrepancies over a short horizon, applies phase-balanced sparse supervision, and uses a Behavioral Anchor to prevent unnecessary changes. Experiments under seven LIBERO-Plus environmental variations demonstrate consistent recovery across multiple VLA backbones and quantization methods. Notably, PIVOT-Q consistently outperforms full-state
distillation across all settings while using only 7.4% of its state-level
distillation budget. Our code is available at https://github.com/ruanruan-andy/PIVOT-Q.
Read the full abstract ▾
All authors Shen Ruan, Wenchang Gao, Jin Wang, Siao Liu, Zhaxizhuoma, Dongchun Ren, Xin Zheng
Machine Learning
CIPHER-MoE: Balancing Efficiency and Routing Fidelity in Trillion-Scale MoE Training
Jing Li, Jian Meng, Yingmeng Gao et al. (+8 more)
Mixture-of-Experts (MoE) has been widely adopted in recent large
language model (LLM) architectures. However, scaling up MoE in LLM training introduces system-level challenges on training, where non-uniform
token routing can lead to highly imbalanced workloads across experts and devices, further destabilizing the training process. With trillion-scale LLMs, imbalanced expert workloads further amplify the resource cost of MoE training, resulting in degraded training efficiency and hardware utilization for underloaded experts, while hot experts require additional resources to accommodate excessive workloads. Recent studies address imbalanced MoE training through intricate parallelism strategies or resource reallocation. However, these system-level approaches often introduce additional resource requirements and considerable orchestration complexity, which become increasingly difficult to afford when training trillion-parameter LLMs under constrained computational resources. This work introduces CIPHER-MoE, which mitigates MoE workload imbalance while keeping the router's
token-side Top-K selection unchanged. CIPHER-MoE applies affinity-aware Expert-to-
Token filtering with explicit capacity control to reduce hotspot expert workloads without additional hardware resources or complex runtime design. The proposed method has been evaluated on large-scale MoE models, including DeepSeek-V4-Pro, showing up to 64.9 percentage points Top-1 expert workload reduction and 1.10$\times$-1.94$\times$ training acceleration, while preserving the training quality. The source code will be released soon.
Read the full abstract ▾
All authors Jing Li, Jian Meng, Yingmeng Gao, Suming Qiu, Linyuan Qiu, Dongfang Li, Baotian Hu, Binfan Zheng, Rongqian Zhao, Weijian Sun, Xin Chen
Machine Learning
PACMI: Provenance-Aware Cascading Memory Invalidation for Long-Term LLM Agents
Yiqi Wang, Jiaqi Liu, Jiaqi Zhang et al. (+4 more)
LLM
agents rely on long-term
memory to retain and reuse information when performing tasks over long horizons. Existing methods provide limited support for handling
memories that become outdated as new observations or domain evidence arrive. Such outdated
memories may remain semantically relevant, continue to affect dependent records, and retain value as historical evidence. This calls for two capabilities: dependency tracking to identify downstream effects and historical preservation to retain useful past records. We propose Provenance-Aware Cascading
Memory Invalidation (PACMI), a framework that represents
memories and new evidence in a provenance graph with typed dependency edges. PACMI assigns records to a four-state validity
lattice, propagates validity changes to dependent
memories, and uses the resulting states for
retrieval and stale-premise detection. We also introduce a diagnostic
benchmark with 100 cases and 300 queries across five domains. The evaluation separates node, context-, and answer-level performance. PACMI achieves the highest final-answer accuracy on this
benchmark, and its paired difference from the strongest baseline is significant under an exact McNemar test. The premise checker achieves perfect precision, recall, and F 1 on the controlled query distribution. Cascading propagation primarily improves memorystate correctness: removing it increases final-answer errors from 3 to 11, but the paired difference does not reach the 0.05 significance threshold. Code and data will be made publicly available.
Read the full abstract ▾
All authors Yiqi Wang, Jiaqi Liu, Jiaqi Zhang, Zhangkai Wu, Yiqun Duan, Mingkai Zheng, Taotao Cai
Machine Learning
Inferring physical fields in coupled systems with unknown parameters from incomplete observations using physics-constrained attentive neural operators
Shilun Wei, Xiaoqiang Sun, Wei Li et al. (+1 more)
Given incomplete measurements of a single physical field in a coupled system with unknown parameters, can we infer its full physical state and identify the underlying parameters? This problem is challenging because multiple coupled fields must be reconstructed simultaneously from limited observations of only one, while the system parameters are unknown. In this work, we propose a machine learning framework for full-field reconstruction and parameter identification of unknown physical systems from sparse observations of a single physical field. Specifically, the cross-
attention encoder propagates sparse sensor observations onto a regular grid to construct a sensor-conditioned latent representation, while a
Fourier neural operator (FNO) decoder captures global spatial dependencies to reconstruct all coupled physical fields. The network parameters and unknown physical parameters are jointly optimized by minimizing observation losses, governing equation residuals, and boundary/initial condition constraints. The proposed approach is validated on two- and three-dimensional lid-driven cavity flows, a two-dimensional cylinder wake, and a two-dimensional non-ideal magnetohydrodynamics problem, demonstrating the recovery performance of unobserved fields and physical parameters from incomplete observations.
Read the full abstract ▾
All authors Shilun Wei, Xiaoqiang Sun, Wei Li, Kejun Tang
Machine Learning
ReMaD: Tuning-free Domain Adaptation for Classification and Out-of-Distribution Detection
Elijah Bolluyt and Cristina Comaniciu
We introduce Reduced-rank Mahalanobis Distance (ReMaD), a novel prototypical distance-based refinement to classification and out-of-distribution (OOD) detection using pretrained models without finetuning. We use
embeddings of the target
dataset to fit closed-form distribution statistics in the model's latent space which can classify in-distribution samples and detect OOD samples, all without training or prior knowledge of the OOD data. Building on prototype classification and OOD detection, we analyze the distribution properties of large pretrained models when processing new
datasets; based on this analysis, we formulate a simple modification to Mahalanobis Distance to adapt models' latent space distributions to new domains by removing unused features, without the finetuning or hyperparameter searches required by other adaptation procedures. We demonstrate the efficacy of this method to adapt existing large pretrained image
embedding models to new classification domains outside their trained capabilities by testing across four target
datasets, with competitive performance in both classification and OOD detection.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Retrieval-Based In-Context Learning: A Domain Adaptation Framework
Yilun Zhu, Naihao Deng, Yingcong Li et al. (+2 more)
In-context
retrieval (ICR) is a
retrieval-based form of in-context learning (ICL) in which demonstrations are retrieved from a source database based on similarity to the query, rather than sampled independently. In this work, we formulate ICR as a type of domain adaptation problem, where the source distribution $P$ of the database may differ from the target distribution $Q$ of the test query-label pair. We investigate the performance of ICR under a flexible class of distributional shifts that substantially extends prior work \citep{li2024fine,guo2025
retrieval}, and establish theoretical guarantees that quantify the benefits and pitfalls of this learning paradigm. Our theory is verified by experiments on synthetic and
language tasks.
Read the full abstract ▾
All authors Yilun Zhu, Naihao Deng, Yingcong Li, Naichen Shi, Clayton Scott
Machine Learning
From Pixels, Without Pre-training: Joint Generative and Self-Supervised Representation Learning in One Model
Vicente Balmaseda, Ching-Long Lin and Tianbao Yang
Strong image generation models are conditioned on class labels, aligned to frozen pretrained encoders, or built on separately trained autoencoders. While effective, generation then depends on supervision or pretraining: labels must be annotated, and encoders or autoencoders pretrained for the target domain. We study joint generative and self-supervised representation learning in a single model, enabling self-conditioned generation without labels or pretrained models. This is challenging because the objectives are mismatched: contrastive learning consumes clean augmented views and favors coarse, invariant semantics, while flow matching consumes noisy images and must preserve the fine detail and spatial layout that contrastive learning discards. We propose SCION (Self-conditioned Generation on Self-supervised representation), whose core is a single pixel-space encoder conditioned on the flow timestep and an
embedding. For representation learning, this conditioning
embedding is a learned global vector shared across images, with the encoder's [CLS]
token yielding the semantic representation trained by the contrastive loss. For generative training, the conditioning
embedding is the image's own [CLS] representation, while patch
tokens pass through a decoder to predict the image. To sample without a reference image at inference, we jointly learn a prior over the
embedding. Gradient-norm balancing and stop-gradient mechanisms enable joint optimization in one run. SCION is self-supervised and self-contained, with no labels or pretrained models. On ImageNet 256x256, with the JiT-B recipe and no representation guidance, SCION reaches 8.92 FID, surpassing class-unconditional iREPA, which aligns to pretrained DINOv2 (46.44), and RCG, which conditions on it (14.27). With JiT-L, SCION achieves 5.89 FID without guidance and 3.47 with representation guidance, outperforming RCG with the ADM recipe (6.24).
Read the full abstract ▾
Machine Learning
Voltic: Distinguishing Volatility from Stochasticity in Recurrent Memory
Parsa Hejabi, Morteza Dehghani and Payam Piray
Recurrent sequence models must decide how strongly to overwrite their
memory at each
token. Read as
Bayesian filtering, this write is the gain of a Kalman update, set by uncertainty from two sources that pull it in opposite directions:
volatility, how quickly the underlying associations change, and stochasticity, how noisy each observation of them is. First, we show that the update of gated delta-rule
memories is the form this filter takes under isotropic uncertainty. Next, we introduce Voltic, a recurrent
memory that keeps the covariance anisotropic and makes both noise variances input-dependent, so the write is vector-valued and carries uncertainty accumulated over the sequence. A dense covariance would have to be propagated
token by
token, ruling out the parallel training these models depend on. We therefore give two assumed-density approximations, diagonal and quasi-diagonal, both of which leave the
memory update in delta-rule form and reuse its chunked kernels. On controlled recall tasks in which associations change and observations are corrupted, Voltic leads all baselines. On the task combining
volatility and stochasticity, its margin over the strongest baseline is larger at both extrapolation sizes than at the training sizes. In 45M-parameter
language models it leads an eight-task
reasoning average and achieves higher
retrieval accuracy beyond the training context length than gated baselines, at throughput close to those baselines. Deriving the write from an uncertainty recursion therefore makes
memory more responsive to change.
Read the full abstract ▾
Machine Learning
Planetary Geospatial Foundation Models: A New Paradigm for Global Public Health
Arbaaz Muslim, Aviv Slobodkin, Katherine Wheeler-Martin et al. (+35 more)
The efficacy of traditional disease prediction is limited by spatial gaps and temporal lags, which impact the timing and targets of resource deployments. Outbreaks escalate undetected, chronic disease burdens are quantified years later, and at-risk populations in data-sparse regions remain unaddressed. Planetary geospatial foundation models complement existing epidemiological workflows to provide operational improvements, encoding
multimodal search, mobility, and environmental signals into generalizable place representations. As illustrations of this complementarity, we present independent global health case studies of Google Earth AI's Population Dynamics Foundation Model (PDFM) – a foundation model for geospatial inference – across four domains (vaccine-preventable, communicable, noncommunicable, maternal mental health), five tasks (spatial extrapolation, interpolation/nowcasting, probabilistic forecasting, prospective forecasting, risk stratification), and four countries (USA, Canada, Mexico, and the Democratic Republic of the Congo). Across these case studies, PDFM addresses critical surveillance gaps across domains: improving US-Canada border MMR vaccination coverage predictions by capturing cross-border behavioral spillovers domestic models miss; nowcasting cardiovascular disease to accelerate data availability; enhancing short-term municipal Mexican dengue forecasts for timely outbreak vector control; improving forecasts of cholera hotspots; and adding a transferable signal to individual-level postpartum-depression risk prediction in US states the model had never seen, while not replacing individual socioeconomic data or closing demographic screening gaps. Together, these results showcase capabilities of geospatial foundation models for public health surveillance.
Read the full abstract ▾
All authors Arbaaz Muslim, Aviv Slobodkin, Katherine Wheeler-Martin, Eric Zhou, John Brittain, Martin Frasch, Hamsa Subramaniam, Jacob Bien, Hamed Sadeghi, Sarah Conderino, Stone Jiang, Ben Spoer, Daniel Neill, Mimi Sun, Joydeep Paul, Yun Liu, Tomer Shekel, Amy Chung-Yu Chou, Anna Carter, Charles Elliott, Aaron Bell, Reuven Sayag, Avinatan Hassidim, Niv Efron, Yossi Matias, Kasumi Widner, Eduardo Lopez Ortiz, José Alberto Díaz-Quiñonez, Moritz U. G. Kraemer, John Brownstein, Isaias Fernandes Co, Etien Luc Koua, Monica Bharel, Benjamin Rader, Lorna Thorpe, Marc Gourevitch, Shravya Shetty, Gautam Prasad
Machine Learning · Cross-listed from cs.DL
Errors of LLM-Assisted Literature Retrieval in Environmental Science: A Comparison Study of Abstract versus Full-text Based Prompts
Yanjun Chen, Yongfeng Zhang and Lanjing Zhang
Large
language models (LLMs) are increasingly used for literature search and synthesis. However, it is unclear whether they retrieve accurate bibliographic information in environmental science. Therefore, we quantitatively compared the errors of widely used LLM platforms in retrieving references related to original articles from five leading environmental science journals (Energy and Environmental Science, Nature Sustainability, Nature
Climate Change, Lancet Planetary Health, and Environmental Science and Technology) published in 2024 to 2025. Claude, ChatGPT, Grok, DeepSeek, Perplexity, and Gemini were used as the LLM platforms. LLMs retrieved 10 references for each of the 50 randomly selected original article using either the article's abstract or its full-text as
prompt. The retrieved references were subject to a multimetric score ratio combining validity of bibliographic data, Google Scholar link, digital object identifier, Scopus Electronic Identifier and relevance score (cited by or being the index paper), and the proportion of complete fabrication that failed all metrics. Abstract-only
prompt yielded significantly higher accuracy than full-text one. This advantage was confirmed in multilevel mixed-effect multivariable regression after adjusting for journal, platform, and output order. Source journal and the position of a reference within the output list were also independently associated with
retrieval accuracy, with lower-listed references associated with lower accuracy. These findings suggest that LLM assisted literature
retrieval in environmental science remains moderately accurate and overall inconsistent, varying significantly by platform, journal,
prompt type, and output position. Abstract-based prompting, as task-aligned information compression, may outperform full-text one in literature
retrieval. Caution should be used when generalizing our findings.
Read the full abstract ▾
Machine Learning
Square-Root Regret for Adversarial Multiplayer Bandits without Collision Information or Shared Randomness
Chenyu Gan
We study
adversarial multiplayer bandits with $K$ arms and $2\le m<K$ labeled players, without collision information, shared randomness, or an external communication channel. We design a constructive communication and synchronization protocol with a Monte Carlo public constructor. With probability at least $1-CN^{-32}$ over preprocessing, where $N=2Km(T+1)$, its fixed published output satisfies \[ R_T\le C K^{5/2}\sqrt T\log^2(2Km(T+1)) \] simultaneously for every oblivious reward sequence chosen after preprocessing. Here $R_T$ is expected regret over the players' private execution randomness. Positive reward observations establish a common learning schedule and synchronize players before learning begins. The cost of delayed communication is charged to the support of positive rewards, ensuring that periods with little useful feedback incur only limited regret. A slow–fast learning procedure then maintains valid reward estimates while assignments and scores are exchanged.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Two-Sample Testing via Path-based Inference
Eshant English, Wei-Cheng Lai, Yanfeng Yang et al. (+3 more)
Modern deep generative models are primarily studied for their ability to generate realistic samples, yet the generative dynamics they learn can also serve as objects of statistical inference. We develop this idea for two-sample testing, the problem of deciding whether the same distribution generated two finite
datasets. Using
stochastic interpolants, we connect both distributions to a shared Gaussian bottleneck, so that each half of the resulting path is a Gaussian channel acting on a single population. We prove that the null hypothesis holds if and only if the population denoiser, or equivalently, the velocity fields of the two halves, coincide at any single noise level, which amounts to a reflection
symmetry of the path about the bottleneck. Deviations from this
symmetry yield a continuum of two-sample witnesses, which we estimate via held-out regression risks on learned denoisers and velocities and aggregate along the path; under an information-theoretic weighting, the aggregated discrepancy equals the Jeffreys divergence between the noise-smoothed distributions. Calibrating the resulting statistics by permutation yields tests that are valid in finite samples for any trained networks and consistent when the fields are learned accurately. On a synthetic
benchmark and three image
benchmarks, the proposed tests improve power over the strongest baseline by up to 33 percentage points at an equal total sample budget, with the best choice of regression representation and path weighting depending on the data modality. These results show that generative paths provide a principled representation for statistical testing, extending
stochastic-interpolant models beyond generation.
Read the full abstract ▾
All authors Eshant English, Wei-Cheng Lai, Yanfeng Yang, Kenji Fukumizu, Taiji Suzuki, Christoph Lippert
Machine Learning · Cross-listed from cs.PL
RESOLVE: Language-Agnostic Validation of GPU Kernels Through Testing, Reduction, and Proof
Ashkan Vedadi Gargary, Guido Martínez, Sebastian Burckhardt et al. (+4 more)
AI systems can now write and optimize production GPU kernels, but validating them remains an important challenge. Evaluating the kernel on a few random inputs and checking that its outputs match a trusted reference kernel within numeric tolerances is not sufficient: races can cause nondeterministic behavior that fails to manifest in tests, and numeric tolerances can hide bugs and cause false positives even after extensive calibration. To address this challenge, we present RESOLVE, which combines testing and formal verification to build a comprehensive kernel validation pipeline. It operates in three steps: First, it tests for nondeterminism using binary instrumentation that perturbs execution timing to expose races. Second, an
agent rewrites the candidate and reference kernels to obtain "reduced-concurrency" versions that are simpler to analyze but still produce bitwise-identical outputs in all tests. Third, the reduced kernels are formally analyzed in the F*/Pulse framework and prove that they perform the same computation on real numbers. This sidesteps the need for numeric tolerances. We show that RESOLVE can validate a broad selection of kernels using KernelBench, and prove equivalence across fused GEMMs in three state-of-the-art frameworks and
languages: CUTLASS, Triton, and Gluon. It also analyzes mega-kernels, notoriously difficult to validate, and finds four previously unreported issues, including two clear bugs. We show that
agents can use RESOLVE to repair the issues, with minimal performance impact, highlighting that
agents can optimize aggressively when they can rigorously check their results.
Read the full abstract ▾
All authors Ashkan Vedadi Gargary, Guido Martínez, Sebastian Burckhardt, Gabriel Ebner, Abhinav Jangda, Madan Musuvathi, Tyler Sorensen
Machine Learning
Sharp Integrality Gaps in Calibration Distance
Zinan Wang and Xinhao Yang
We study the offline gap between deterministic calibration distance C and its fractional relaxation L for binary unit-weight sequences under total absolute-change cost. We sharpen the offline comparison C <= L + O(sqrt(T)) (Qiao and Zheng, 2024,
Theorem 2) to the sharp worst-case order Theta(T^(1/3)). If Delta_T is the supremum of C - L over length-T inputs, then T^(1/3)/1000 <= Delta_T <= 41T^(1/3) for T >= 216. The upper bound holds for every input, while each T >= 216 has a rational lower-bound input. For every input with m distinct forecasts, C <= L + m, and the unrestricted-sample worst-case sparse order is Theta(m). For rational forecasts and accuracy, with binary-encoded multiplicities of separately assignable unit identities, a grid-free
polynomial-bit-time procedure returns B <= L <= U, U - B < eta, and an exactly calibrated compact repair of cost at most U + m <= L + m + eta.
Read the full abstract ▾
Machine Learning · Cross-listed from physics.soc-ph
An evolutionary origin of collective decision making in humans and machines
Guocheng Wang, Qi Su and Joshua B. Plotkin
Groups of individuals can solve collective problems more accurately than any single member, by aggregating their opinions. Recent theoretical work has identified individual-level reward schemes that allow uninformed individuals to evolve collective intelligence from the bottom up, through social learning. Yet these results are restricted to linear prediction problems and simple averaging, while the decision tasks that real groups confront are often non-linear, and the institutions that aggregate opinions are seldom single-layer averages: districts elect representatives who in turn vote on policy, referees advise editors who decide on publication. Here we develop a framework for the
evolution of collective intelligence in multi-layer voting populations, where individuals observe limited information and groups recursively aggregate their opinions by majority rule. We prove that single-layer voting cannot solve non-linear classification problems under any individual reward scheme. We then identify a "marginal feedback" payoff structure, which rewards individuals only when their opinion is pivotal in their group, and at every layer above them. This reward scheme induces a layered population to evolve accurate collective solutions to complex, non-linear decision tasks through individual-level peer imitation alone. The collective behavior that emerges is equivalent to a multi-layer perceptron in machine learning. Our results provide a naturalistic account of hierarchical institutions, in which the outsize importance of swing voters is the incentive that sustains collective accuracy; and they identify the credit-assignment rule in machine learning as not just an engineered solution but a natural evolutionary outcome.
Read the full abstract ▾
Machine Learning
Bellman-Centric Learning: Near-Optimal Regret for Linear Bandits with Memory
Jingyuan Liu and Huiwen Jia
We study linear bandits with
memory, where past actions induce endogenous nonstationarity through an arbitrary known, bounded
matrix-valued
memory map. To trade off exploration and exploitation while accounting for the
memory dynamics, we develop RSM-LinUCB, a Bellman-centric
algorithm that learns as in linear bandits and plans as in reinforcement learning. This design admits a novel regret decomposition which separates the
memory-induced error from the cumulative reward estimation error along the learner's trajectory. We prove a high-probability regret bound of $\widetilde O\big(dRS(M+1)+\sigma d\sqrt T\big)$, where $T$ is the learning horizon, $d$ is the parameter dimension, $M$ is the
memory length, $R$ and $S$ bound the
memory-map operator norm and reward-parameter norm, respectively, and $\sigma$ is the sub-Gaussian noise scale. Our results reveal that the multiplicative
memory-horizon coupling in prior bounds is not intrinsic:
memory only contributes an additive cost, up to logarithmic factors. We also prove a matching minimax lower bound, establishing near-optimality. We further extend the
algorithm to generalized linear rewards, preserving this separation with near-optimal
memory and leading statistical dependence. Our
algorithms outperform the baselines in numerical experiments on synthetic instances and semi-synthetic KV- and semantic-cache tasks.
Read the full abstract ▾