Machine Learning · Cross-listed from cs.IR
BACH: A Bayesian Admixture of Contrastive Heads for Multi-Interest Two-Tower Retrieval
Quoc Phong Nguyen, Paul Albert, Long Vuong et al. (+2 more)
Two-tower retrievers compress each user into a single
embedding, limiting their ability to serve diverse interests. Multi-interest models give each user several heads scored by a maximum inner product, but their hard-routing training under-utilizes heads (routing collapse) and gives no per-user estimate of how much each interest matters for serving. We present BACH (
Bayesian Admixture of Contrastive Heads), which casts multi-interest two-tower
retrieval as a per-user mixture over the heads, fit by variational inference. The soft mixture trains every head (mitigating collapse), produces a per-user weighting of the interests that is reused at serving, and admits a shared global-codebook variant with precomputable
retrieval. On three large-scale
benchmarks, MovieLens-20M, Taobao, and Netflix, BACH improves top-of-ranking
retrieval over hard-routing multi-interest and single-vector baselines at every head count; we further find that scoring every candidate by its best head, consistent with serving, outperforms the usual target-routed training, and that BACH improves further still.
Read the full abstract ▾
All authors Quoc Phong Nguyen, Paul Albert, Long Vuong, Vuong Le, Julien Monteil
Machine Learning
Vanilla SGD with Momentum Survives Heavy-Tailed Noise: Convergence Analysis without Gradient Clipping or Normalization
Ryusei Yamada, Naoki Sato and Hideaki Iiduka
Stochastic gradient descent (SGD) is a cornerstone of modern optimization. While its performance under heavy-tailed noise is often addressed through specialized modifications such as gradient clipping or normalization, we investigate a more fundamental question: how does vanilla SGD, particularly with momentum, perform in the presence of heavy-tailed noise? In this paper, we refine existing convergence results for vanilla SGD and, more importantly, provide the first comprehensive convergence analysis of vanilla SGD with momentum for strongly convex, convex, and nonconvex objectives, without employing any gradient control mechanisms. Our results demonstrate that the obtained convergence rates are inferior to the optimal rates achieved by clipped or normalized variants of SGD, thereby revealing inherent limitations of vanilla methods under heavy-tailed noise. The theoretical findings are supported by experiments on synthetic functions.
Read the full abstract ▾
Machine Learning
Stochastic Order Learning: An Approach to Rank Estimation Using Noisy Data
Chaewon Lee, Seon-Ho Lee and Chang-Su Kim
Rank estimation under label noise poses a fundamental challenge, as ordinal annotations often exhibit structured uncertainty rather than simple label corruption. In this paper, we reformulate rank estimation with noisy ordinal labels as a
stochastic ordering problem, in which each instance is inherently associated with multiple plausible ranks instead of a single deterministic label. Based on this view, we propose
stochastic order learning (SOL), a learning framework that captures ordinal label uncertainty and learns an
embedding space through two complementary objectives: a discriminative loss that structures instance–centroid interactions and a
stochastic order loss that enforces probabilistic ordering relations between instances. Extensive experiments across diverse
datasets demonstrate that SOL enables reliable rank estimation under various types and levels of label noise. The source code is available at https://github.com/cwlee00/SOL.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
CausalDS: Benchmarking Causal Reasoning in Data-Science Agents
Andrej Leban, Theo Saarinen and Yuekai Sun
Large
language models (LLMs) increasingly act as data-science
agents, combining abstract
reasoning with advanced tool use. Yet the relevant
benchmark landscape largely divides into symbolic causal
reasoning benchmarks without realistic data analysis or data analysis
benchmarks without causal data-generating structure or a range of causal data science tasks. Existing causal evaluation
datasets are often restricted to curated examples from existing sources, with diversity coming from limited templatized variations rather than from systematic generation of novel synthetic causal structures. We introduce CausalDS, a
benchmark for evaluating causal
reasoning in agentic data-science workflows. Each
benchmark instance is a scene consisting of a sampled structural causal model (SCM) with generated observational data and an accompanying synthetic natural-
language story grounded in a real domain. We ground the composition of the
benchmark components in empirical distributions obtained from real-world
datasets, thus retaining empirical structure while reducing the memorization risk through completely synthetic generation. From each scene, we then derive tasks spanning all three of Pearl's rungs. Most tasks include a data science coding component, where the model typically needs to use several tools to arrive at the final answer due to imperfect observations. Abstention is tested directly: for targets the graph does not identify, the graded answer is to abstain. The
benchmark thus jointly evaluates symbolic causal
reasoning, data science, uncertainty quantification, abstention, and tool use/coding. Because the generator is fully synthetic, the exam is not a fixed
dataset: composition, difficulty, and size are parameters, and fresh exams can be drawn at will.
Read the full abstract ▾
Machine Learning
Deep Learning Method for Stationary Distribution of Reflected Brownian Motion
Jim Dai and Zhanhao Zhang
The stationary distribution of reflected Brownian motion (RBM) plays an important role in the analysis of high-dimensional
stochastic systems, yet closed-form solutions are known only for a few special cases. Computing important performance metrics, such as tail probabilities, is even more intractable, despite their practical relevance. In this paper, we develop a deep learning approach that accurately and efficiently learns the Laplace transform of high-dimensional RBMs based on the basic adjoint relationship (BAR). Our framework combines a careful design of the loss function, training data sampling procedure, and neural network architecture. We evaluate the proposed method on RBM instances with known ground-truth tail probabilities and demonstrate near-perfect prediction in high-dimensional settings, highlighting its potential as a general tool for analyzing
stochastic systems beyond analytically tractable regimes. Our code can be found at https://github.com/zhangz73/NN4MGF.
Read the full abstract ▾
Machine Learning · Cross-listed from eess.IV
ConRad: Efficient Conformal Prediction for Radiomics
Matt Y. Cheung, Ashok Veeraraghavan and Guha Balakrishnan
Radiomic features derived from medical images and
segmentation masks are used to support decision making in clinical imaging pipelines. In practice, these features are often computed from predicted masks, but
segmentation models can be overconfident or poorly calibrated, making derived measurements appear more reliable than they are. Conformal prediction (CP) provides distribution-free prediction intervals with finite-sample marginal coverage guarantees, but black-box intervals for
segmentation-derived radiomics can be inefficient because they ignore test-time information about image appearance, mask geometry, and
segmentation uncertainty. We propose ConRad, a conformal framework for scalar radiomic targets that uses
covariates derived from the predicted mask, input image, predicted radiomics, and boundary uncertainty to construct adaptive intervals while maintaining coverage. Across five 2D medical imaging
datasets and 171 retained radiomic targets, we show that ConRad improves feature-level efficiency compared to baselines while maintaining near-nominal empirical coverage. Ablation results further indicate that
segmentation boundary uncertainty features are the largest contributors to interval efficiency.
Read the full abstract ▾
Machine Learning
Modular Pretraining Enables Access Control
Ethan Roland, Murat Cubuktepe, Erick Martinez et al. (+8 more)
AI developers face a dual-use dilemma. An AI capability that helps one user cure a disease can help another synthesize one. This dilemma could be resolved with access control, limiting dual-use AI capabilities to trusted deployments with a legitimate need. A gold standard for access control would be to serve separate models with different capabilities to different users. However, training and deploying multiple models is prohibitively expensive. To address this challenge, we propose gradient-routed auxiliary modules (GRAM), a pre-training method that adds modules to a neural network and selectively updates them to induce specialization. Ablating a module at inference time removes its capability from the network, approximating a model trained on filtered data. We evaluate GRAM on synthetic stories and realistic dual-use data spanning virology, cybersecurity,
nuclear physics, and specialized code. These experiments show that GRAM disables targeted capabilities while preserving the rest, and resists their recovery under finetuning better than post-hoc unlearning. Most importantly, a Chinchilla-optimal scaling analysis from 50M to 5B parameters shows that the gap between data-filtered and full-data models widens with scale on removed capabilities but stays small on retained ones, and that GRAM closely tracks data filtering. GRAM's training cost is independent of the number of supported capability profiles, yielding a 5x reduction over data filtering in our 5-profile setting.
Read the full abstract ▾
All authors Ethan Roland, Murat Cubuktepe, Erick Martinez, Stijn Servaes, Keenan Pepper, Mike Vaiana, Diogo Schwerz de Lucena, Judd Rosenblatt, Addie Foote, Cem Anil, Alex Cloud
Machine Learning
Cross-Modal Generative Framework for Signal Translation from Fetal-Maternal Electrocardiograms to Fetal Doppler Waveforms
Tongli Su, Alireza Rafiei, Marly van Assen et al. (+4 more)
Fetal electrocardiogram (fECG) and Doppler ultrasound provide complementary views of fetal cardiovascular function: fECG captures electrical activity while Doppler reflects mechanical hemodynamics shaped by factors such as placental resistance and vascular compliance. Understanding the recoverable and unrecoverable Doppler components through reconstruction from fECG offers insight into the relative contributions of electrical versus mechanical factors in fetal circulation, thereby informing clinical decisions. In addition, clinical evidence of maternal-fetal cardiac coupling suggests that maternal cardiovascular dynamics may also inform fetal hemodynamics. To computationally model these relationships, we propose a cross-modal generative framework combining dilated convolutions with cross-modal
attention to selectively incorporate maternal ECG and self-
attention to capture long-range temporal dependencies. Trained on 885 synchronized fetal/maternal ECG and Doppler envelope segments from 39 pregnancies, our model synthesizes Doppler envelopes with power spectral density mean squared error (PSD MSE) of 49.9 +/- 15.8 dB^2 (51% lower than two-channel baseline) and heart-rate error of 4.71 +/- 0.77 bpm (1.5% better than baseline; negligible relative to the 110-160 bpm physiological range). Cross-modal
attention yields a 39% PSD MSE reduction over naive dual-channel concatenation, quantifying the contribution of maternal-fetal coupling. Our proposed framework advances computational modeling of the maternal-fetal cardiovascular system by enabling the synthesis of Doppler envelopes from dual-lead ECG. By analysis of both recoverable and residual Doppler components, this approach enables quantification of the purely mechanical contributions to Doppler waveforms – those not recoverable from electrical recordings – ultimately facilitating a more comprehensive fetal assessment.
Read the full abstract ▾
All authors Tongli Su, Alireza Rafiei, Marly van Assen, Reza Sameni, Gari D. Clifford, Faezeh Marzbanrad, Nasim Katebi
Machine Learning · Cross-listed from cs.CL
COBART: Controlled, Optimized, Bidirectional and Auto-Regressive Transformer for Ad Headline Generation
Yashal Shakti Kanungo, Gyanendra Das, Pooja A et al. (+1 more)
Online ads are essential to all businesses and ad headlines are one of their core creative component. Existing methods can generate headlines automatically and also optimize their click-through-rate (CTR) and quality. However, evolving ad formats and changing creative requirements make it difficult to generate optimized & customized headlines. We propose a novel method that uses prefix control
tokens along with BART fine-tuning. It yields the highest CTR and also allows users to control the length of generated headlines for use across different ad formats. The method is also flexible and can easily be adapted to other architectures, creative requirements and optimization criteria. Our experiments demonstrate a 25.82% increment in Rouge-L and a 5.82% increment in estimated CTR over previously published strong ad headline generation baseline.
Read the full abstract ▾
All authors Yashal Shakti Kanungo, Gyanendra Das, Pooja A, Sumit Negi
Machine Learning · Cross-listed from cs.AI
Persuasion Attacks Can Decrease Effectiveness of CoT Monitoring
Jennifer Za, Julija Bainiaksina, Nikita Ostrovsky et al. (+2 more)
Chain-of-thought (CoT) monitoring is a promising safety mechanism for AI
agents, based on the premise that visible
reasoning traces can surface misaligned or deceptive behavior. While effective in standard scenarios, recent work highlights that LLMs remain vulnerable to persuasion-based jailbreaks, where natural-
language arguments override model constraints. We stress-test whether this vulnerability extends to monitoring LLMs: can an
adversarial agent persuade its CoT monitor to approve proposed actions that violate the monitor's policy? We design an evaluation framework with 40 tasks and analyze thousands of
agent-monitor interactions, where
agents are instructed to argue for policy-violating proposals. We find that in such
adversarial settings, monitor access to the
agent's CoT
reasoning increases rather than decreases approval of harmful actions on average by 9.5%, as the scratchpad provides an additional persuasion channel. To address this, we introduce a fact-checking monitoring framework. We find that a fact-checker and monitor pairing from different model families, for example a Claude 3.7 Sonnet monitor paired with a GPT-4.1 fact-checker, reduces approval of policy-violating actions by up to 45%, compared to only 6%, when using the same model for both fact-checking and monitoring roles. Our results demonstrate that CoT monitoring alone may be insufficient against
adversarial persuasion, and that model-diverse fact-checking provides a robust mitigation.
Read the full abstract ▾
All authors Jennifer Za, Julija Bainiaksina, Nikita Ostrovsky, Tanush Chopra, Victoria Krakovna
Machine Learning · Cross-listed from cs.CL
Holographic Neural PCFG for Unsupervised Parsing
Ryosuke Yamaki, Daichi Mochihashi, Nobutaka Shimada et al. (+1 more)
Unsupervised constituency parsing aims to accurately induce latent tree structures from raw text alone. Recent neural parameterizations of PCFGs achieve strong performance in both supervised and unsupervised parsing, yet rely on high-capacity black-box networks for rule scoring – as exemplified by the Neural PCFG family – leaving rule probabilities without an interpretable mathematical form. In this paper, we propose
Holographic Neural PCFG (Hol-PCFG), which recasts PCFG rule scoring as algebraic relation modeling among grammar-symbol
embeddings. Hol-PCFG adapts
Holographic Embeddings (Nickel et al., 2016), which scores knowledge-graph triples via circular correlation, to the left-child, right-child, and lexical-emission relations over
torus-constrained
embeddings, giving every rule probability a closed form that carries the intrinsic structure of grammar rules by construction. Hol-PCFG achieves state-of-the-art parsing performance in six
languages while cutting rule-scoring parameters by 99.94% relative to the baseline model and training more stably. Additionally, we demonstrate that Hol-PCFG can parse Japanese directly from characters without any morphological
segmentation, retaining nearly the same morpheme-level performance.
Read the full abstract ▾
All authors Ryosuke Yamaki, Daichi Mochihashi, Nobutaka Shimada, Tadahiro Taniguchi
Machine Learning
When Thinking Hurts: Epistemic Signals in the Reasoning Chains of Visual Language Models
Mayank Singal
Uncertainty quantification for visual
language models (VLMs) conventionally targets the answer
token distribution. We provide the first three-family empirical characterisation of answer
entropy behaviour in thinking-mode VLMs. Running four models on identical POPE
adversarial samples, we find three qualitatively distinct patterns: Qwen3-VL-8B-Thinking shows complete collapse (ans H AUROC = 0.492); GLM-4.1V-9B-Thinking shows no collapse (0.716); and InternVL3-8B shows selective thinking (chains on only 50% of samples, ans H = 0.675 full / 0.602 thinking-only). Across all three thinking-mode models, thinking chain
entropy outperforms answer
entropy on the subset where chains are generated (0.647, 0.759, 0.608 vs. 0.492, 0.716, 0.602 respectively), suggesting chain signals are the more reliable predictor whenever chains are present. This holds strongly for Qwen and GLM, but with only marginal and statistically unreliable advantage for InternVL3 (n_FP = 17). A 300-sample VQAv2 pilot confirms chain
entropy (0.680) outperforms answer
entropy (0.595) on VQAv2 questions, with the gap largest for free-form answers (0.733 vs. 0.467). On harder
reasoning tasks (HallusionBench) both Qwen models show moderate signal (approx. 0.64), consistent with incomplete pre-commitment on difficult questions. We additionally document structured abstention affecting 12-22% of queries with asymmetry toward absent-object queries, and a practical abstention gate raising accuracy from 71.0% to 93.8% at 62.7% coverage with no additional inference cost.
Read the full abstract ▾
Machine Learning
Towards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimization
Jiantong Jiang, Peiyu Yang, Rui Zhang et al. (+1 more)
Despite the rapid advancements of large
language models (LLMs), LLM serving systems remain
memory-intensive and costly. The key-value (KV) cache, which stores KV tensors during autoregressive decoding, is crucial for enabling low-latency, high-throughput LLM inference serving. In this survey, we focus on system-aware KV infrastructure for serving LLMs (abbreviated as sKis). We revisit recent work from a system behavior perspective, organizing existing efforts into three dimensions: execution and scheduling (temporal), placement and migration (spatial), and representation and retention (structural). Furthermore, we analyze cross-behavior co-design affinity and behavior-objective links, highlighting future opportunities. Our work systematizes a rapidly evolving area, providing a foundation for understanding and innovating KV cache designs in modern LLM serving infrastructure.
Read the full abstract ▾
All authors Jiantong Jiang, Peiyu Yang, Rui Zhang, Feng Liu
Machine Learning
Reinforcing the Generation Order of Multimodal Masked Diffusion Models
Yidong Ouyang, Zhe Wang, Sourav Bhabesh et al. (+1 more)
Diffusion Language Models (DLMs) have recently achieved substantial progress in natural
language generation tasks. Recent research demonstrates that adaptive
token generation ordering can significantly improve performance in mathematical
reasoning and code synthesis applications. In this work, we investigate the optimization of generation order for both text-to-image synthesis and
multimodal understanding. We first establish that, unlike structured problems in
language generation such as Sudoku puzzles, model logits alone are insufficient for determining optimal generation sequences in text-to-image generation and
multimodal understanding. To address this challenge, we introduce a learnable control module trained via Group Relative Policy Optimization (GRPO) to determine the generation order. Our results demonstrate that learning this control block substantially improves both text-to-image
alignment and
multimodal understanding in DLMs. In particular, it enhances the model's ability to capture fine-grained spatial relationships in generated images while also strengthening performance on
multimodal reasoning and comprehension tasks. We evaluate our framework on GenEval, an object-focused
benchmark for text-to-image
alignment, where it achieves 4.08% relative improvements. In addition, experiments on VLMEvalKit confirm 4.85% relative improvements in
multimodal understanding, highlighting the broad effectiveness of our approach.
Read the full abstract ▾
All authors Yidong Ouyang, Zhe Wang, Sourav Bhabesh, Dmitriy Bespalov
Machine Learning
Who Analyses the Analyser? Self-Validating LLM Hazard Analysis with Constitutional Meta-STPA
Samuel Tetteh, Udip Shrestha, Joshua R. Waite et al. (+1 more)
Large
language models (LLMs) are increasingly trusted to draft the artifacts of safety analysis such as, losses, hazards, Unsafe Control Actions (UCAs), and safety constraints, inside rigorous processes such as Systems-Theoretic Process Analysis (STPA). Yet a blind spot runs through this fast-growing literature: every system gets analysed except the LLM-assisted tool doing the analysing, which is itself a safety-relevant system that can hallucinate standards, emit unverifiable constraints, and leave no audit trail from
prompt to artifact. We take seriously the question the field has skipped – {who analyses the analyser?} and answer it by turning STPA on the tool itself. We present \{Constitutional Meta-STPA}, an LLM-assisted STPA tool built around a closed loop: the tool runs a {meta-STPA} of the class of AI-assisted safety tools and {derives} rather than asserts, its governance constitution from the resulting loss$\to$hazard$\to$UCA$\to$constraint chain, yielding a published constitution of $21$ Tool Principles and $8$ Meta-Safety Principles, each bound to a code enforcement point. We formalise the measured object as a constitution-marginal coverage operator over a principle set $P$ ($|P|{=}29$) with a soundness lemma that isolates coverage from model and scanner, and report four findings. {(i)~Self-derivation:} a frontier ensemble ({claude-opus-4.8}${+}${claude-sonnet-4}) recovers $18/21$ canonical and all $8/8$ governance principles from the tool's own design, while a weaker pair recovers $12/21$ and $3/8$, so the meta layer is model-limited, not constitution-limited, and the same $8/8$ re-emerge from a second, independently authored tool.
Read the full abstract ▾
All authors Samuel Tetteh, Udip Shrestha, Joshua R. Waite, Cody Fleming
Machine Learning · Cross-listed from cs.IT
RadioDiff-v2: Generative Angular Radio Maps for Multi-Beam Selection and Localization
Xiucheng Wang, Junxi Huang and Nan Cheng
Angular
radio maps describe the received-power distribution over the angle of arrival and underpin beam selection and receiver localization in sixth-generation (6G) networks. Predicting the angular power spectrum (APS) from geometry is difficult, because the mapping is ill-posed in non-line-of-sight (NLOS) conditions and must generalize to unseen environments. Distortion-minimizing regressors return the conditional mean, which over-smooths the spectrum and erases the multipath structure that downstream tasks need. We cast the task as a perception-distortion problem and propose RadioDiff-v2, a dual-branch one-dimensional
diffusion transformer trained with flow matching. It couples periodic angular encoding, adaptive layer-normalization conditioning, a
Fourier angular mixer, and joint velocity and clean-signal heads. A per-metric
estimator portfolio reads every deployment quantity from this single model, so that samples carry the distribution, the clean-signal head supplies a regression-grade point estimate, Bayes-optimal rules select beams, and the conditional likelihood localizes the receiver. We prove that a concentrated conditional yields a straight probability-flow trajectory that one step integrates exactly, identifying deterministic transport as the correct inductive bias. On a zero-shot test of 99 environments and one million links, RadioDiff-v2 leads every baseline on every metric, with a 0.39 dB Wasserstein-1 distance, per-bin error below the regression baseline, a 2.43 dB eight-beam NLOS sweep loss, and a 20.6-pixel localization error with four base stations. Code is available at https://github.com/UNIC-Lab/RadioDiff-v2.
Read the full abstract ▾
Machine Learning
An exact information theory of generalization phase transitions in Bayesian diffusion models
Henry Hunt, Mason Kamb and Surya Ganguli
How
diffusion models circumvent the curse of dimensionality to learn complex distributions over high dimensional spaces from a finite training set, instead of memorizing it, remains a fundamental mystery. To address this, we introduce analytically tractable
Bayesian information restricted
diffusion (BIRD) models, in which each pixel observes restricted information about noisy data. A BIRD model time-reverses
diffusion by inferring which past training sample produced its current restricted observation using the
Bayesian posterior. This model class generalizes existing analytical
diffusion models that use spatially local information restriction. We show that spatially local BIRD models closely approximate trained
diffusion models early in training, across different architectures such as UNets and DiTs. Under minimal assumptions on the data distribution, we identify an information-theoretic phase boundary between memorization and generalization in the joint space of amount of training data, time in the reverse generative process, and amount of information restriction: a BIRD model memorizes when the mutual information between its restricted noisy observations and the training data exceeds the log number of training points, and it generalizes otherwise. Experiments across a range of
datasets confirm our theoretically predicted location for the transition. We find that generation proceeds near the edge of memorization: both spatially local BIRD models and early-training
diffusion models track the memorization-generalization phase boundary by increasingly restricting information over time. Overall, our results reveal a fundamental role for information restriction in generative AI to circumvent the curse of dimensionality.
Read the full abstract ▾
Machine Learning
What to Keep, What to Forget: A Rate–Distortion View of Memory Compaction in LLMs and Agents
Ashwin Gerard Colaco and Nada Lahjouji
Large
language models, and the
agents built on them, spend an ever-growing share of their compute and
memory on remembering: caching
attention keys and values, carrying long
prompts, maintaining recurrent state, and storing what happened in previous turns and sessions. Because none of this
memory is free, four largely separate research communities have each learned to compact it. They evict or quantize the KV cache, prune or distill
prompts, bound architectural state, and consolidate
agent memory. We argue that these are instances of one problem: a rate–distortion decision about what context-derived information to retain versus discard, at what fidelity, under a resource budget, so as to preserve downstream task utility. We make this lens precise with a single compaction objective and a layer-agnostic lower bound, use it to build a seven-axis taxonomy that classifies methods from across the stack uniformly, and use it to transfer mechanisms between layers that have never been connected, from serving-stack KV management to
agent long-term
memory. Two patterns hold across the survey. At every layer the signal that decides what to keep is
attention magnitude or recency, and it fails in the same way everywhere, by discarding, before the query is known and with no way to undo it, information the query later needs. And while compression is measured carefully on single-turn long context, the repeated compaction that
agents actually perform is almost never measured, and no
benchmark holds one budget axis across all the layers at once. We turn both observations into a
benchmark proposal, a small reference experiment, and a set of compaction-aware design principles, and we map the open problems.
Read the full abstract ▾
Machine Learning
Rethinking Small VLM Quantization: From Component-Wise Analysis to Hardware-Aware Edge Deployment
Hyeju Shin, Chorwon Kim, Ryangsoo Kim et al. (+2 more)
The emergence of
vision language models with fewer than 3 billion parameters has accelerated the implementation of on-device
multimodal intelligence. However, a detailed understanding of component-wise quantization remains a bottleneck for optimal deployment. This paper presents a systematic evaluation framework for empirically validating five hypotheses across six quantization configurations on the Jetson Orin NX and AGX. By separating the
vision encoder, projector, and large
language model backbone yields the following results: (1) Quantization sensitivity is governed by the structural paradigm (MoE vs. dense) rather than scale alone, with MoE backbones mitigating INT4 noise where dense backbones degrade; (2) SigLIP encoders incur disproportionate INT8 latency on Jetson Ampere–a deployment-specific encoder-kernel-hardware interaction, not a SigLIP flaw; (3) Although INT4 quantization of LLMs greatly reduces VRAM consumption, it also causes slower
token generation due to dequantization overhead; (4) Composite quantization errors are largely additive, except along the modality-
alignment path, which is architecture-dependent; (5) The intelligence-per-joule profile varies significantly across platforms owing to
memory bandwidth constraints.
Read the full abstract ▾
All authors Hyeju Shin, Chorwon Kim, Ryangsoo Kim, Hark Yoo, Jaein Kim
Machine Learning
PGD-NO: A Neural Operator with Precomputed Geometry Decomposition for 3D Million-scale Physics Simulations
Weiheng Zhong, Jing Bi, Victor Oancea et al. (+1 more)
While neural PDE solvers have demonstrated significant potential for accelerating engineering simulations, existing architectures remain constrained by high
memory consumption and the single node bottleneck, where the maximum processable mesh resolution is strictly limited by the VRAM of a single compute unit. To address these challenges, we propose PGD-NO, a neural operator with Precomputed Geometry Decomposition, that relocates the computational overhead of
geometric encoding to a deterministic pre-computation phase. By utilizing an iterative geometry decomposition
algorithm to extract geometry
tokens, our model decouples feature extraction from solution querying. This architecture enables linear
memory scalability, allowing high fidelity learning on meshes exceeding 10 million nodes, a scale where existing architectures typically encounter
memory exhaustion. PGD-NO demonstrates competitive predictive accuracy across diverse industrial
benchmarks and provides intrinsic interpretability through
attention mechanisms. By effectively overcoming traditional mesh-size constraints, PGD-NO offers a robust and efficient solution for the next generation of large-scale, high-fidelity industrial design applications.
Read the full abstract ▾
All authors Weiheng Zhong, Jing Bi, Victor Oancea, Hadi Meidani
Machine Learning · Cross-listed from cs.CV
APIVOT: Adaptive Planning with Interleaved Vision-Language Thoughts
Emily Jin, Joy Hsu, Yiqing Xu et al. (+3 more)
Long-horizon
robot planning requires jointly
reasoning over semantic task structure and
geometric feasibility. To successfully execute a task, a
robot must decompose goals, select task-relevant objects, and sequence actions, while ensuring that plans satisfy spatial constraints such as limited free space and object collisions. In this work, we propose APIVOT, a VLM-based planner that adaptively interleaves
language and visual thoughts for long-horizon planning. APIVOT learns to leverage
language for semantic
reasoning, while using visual thoughts as imagined future states for internal verification of
geometric feasibility. On long-horizon kitchen tasks, APIVOT outperforms general-purpose VLMs and prior planning frameworks, achieving the largest gains in spatially constrained settings. We find that APIVOT learns meaningful modality selection behavior, demonstrating that adaptive interleaving of
vision-
language thoughts improves both planning success and
reasoning efficiency.
Read the full abstract ▾
All authors Emily Jin, Joy Hsu, Yiqing Xu, Weiyu Liu, Nick Haber, Jiajun Wu
Machine Learning · Cross-listed from cs.CV
FedTR: Federated Learning Framework with Transfer Learning for Industrial Visual Inspection
Vikash Sathiamoorthy, Shuo Huai, Hao Kong et al. (+7 more)
Federated learning (FL) is a collaborative learning scheme to train deep learning models, where collaborating parties can consolidate their models without sharing local data with other parties, hence preserving data
privacy. Nevertheless, when implementing FL in Industrial visual inspection (IVI), the constraints posed by limited data availability and the intricate nature of the inspection tasks significantly impact the performance of the resulting model. This paper introduces FedTR, a novel FL framework incorporating transfer learning designed for Autonomous IVI, focusing on the challenging task of identifying label defects through end-to-end text recognition. Transfer learning is a method that leverages the knowledge of a pre-trained model to adapt to a different
dataset. FedTR initially trains the model using a publicly available
dataset, after which performs the essential
federated learning process with model fine-tuning on the distributed and limited private data. Extensive experiment results demonstrate the effectiveness and feasibility of FedTR on private ink cartridge
datasets for label defect identification. FedTR achieves an end-to-end text recognition word-level accuracy of 95.5% and 94.2% on homogeneous and heterogeneous data respectively. Additionally, it attains performance levels that are on par with those achieved through centralized training.
Read the full abstract ▾
All authors Vikash Sathiamoorthy, Shuo Huai, Hao Kong, Di Liu, Wendy Yong Yi Loy, Christian Makaya, Daren Ho, Ravi Subramaniam, Qian Lin, Weichen Liu
Machine Learning
Collate: Collaborative Neural Network Learning for Latency-Critical Edge Systems
Shuo Huai, Di Liu, Hao Kong et al. (+5 more)
Federated Learning (FL) empowers multiple clients to collaboratively learn a model, enlarging the training data of each client for high accuracy while protecting data
privacy. However, when deploying FL in real-time edge systems, the heterogeneity of devices among systems has a severe impact on the performance of the inferred model. Existing optimizations on FL focus on improving the training efficiency but fail to speed up inference, especially when there is a latency constraint. In this work, we propose Collate, a novel training framework that collaboratively learns heterogeneous models to meet the latency constraints of multiple edge systems simultaneously. We design a dynamic zeroizing-recovering method to adjust each local model architecture for high accuracy under its latency constraint. A proto-corrected
federated aggregation scheme is also introduced to aggregate all heterogeneous local models, satisfying the latency constraint of different systems with only one training process and maintaining high accuracy. Extensive experiments indicate that, compared to state-of-the-art methods and under a latency constraint, our extended models can improve the accuracy by 1.96% on average, and our shrunk models can also obtain a 3.09% accuracy improvement on average, with almost no extra training overhead. The related codes and data will be available at https://github.com/ntuliuteam/Collate
Read the full abstract ▾
All authors Shuo Huai, Di Liu, Hao Kong, Xiangzhong Luo, Weichen Liu, Ravi Subramaniam, Christian Makaya, Qian Lin
Machine Learning
Provably Optimal Learning Algorithms for Assistance Games
Nivasini Ananthakrishnan, Mark Bedaywi, Michael I. Jordan et al. (+2 more)
This paper studies an online variant of the assistance games framework, where an informed
agent and an uninformed
agent repeatedly interact over $T$ timesteps to optimize a common reward function. While the informed
agent (the human) observes a latent state of the world, the uninformed
agent (the assistant) observes only the human's actions. We provide the first provably efficient learning
algorithms for repeated assistance games. We introduce the notion of assistance regret: the gap between the cumulative utility of interactions and that of the optimal joint policies in hindsight, which map latent states to action pairs. We present decentralized
algorithms for both the human and the assistant that achieve a $(1-1/e)$-approximate assistance regret rate of $\widetilde{O}(T^{3/4})$, with runtime
polynomial in the size of the action and state spaces. These
algorithms are general; in particular, they accommodate any no-regret
algorithm for the assistant. We prove that achieving a regret approximation factor better than $(1-1/e)$ is computationally intractable. Furthermore, we demonstrate how these generic no-regret
algorithms can be tailored to a pseudo-decentralized setting – using a shared random string – to achieve a rate of $\widetilde{O}(T^{1/2})$, optimal up to logarithmic factors.
Read the full abstract ▾
All authors Nivasini Ananthakrishnan, Mark Bedaywi, Michael I. Jordan, Stuart Russell, Nika Haghtalab
Machine Learning · Cross-listed from cs.CR
Beware What You Autocomplete: Forensic Attribution of Backdoored Code Completions
Anjun Gao, Yueyang Quan, Zhuqing Liu et al. (+1 more)
Large
language models have enabled powerful code completion systems that assist developers by predicting subsequent lines of code. However, these models remain vulnerable to
backdoor attacks, where malicious fine-tuning data covertly implants unsafe behaviors. Despite advances in defensive techniques, adaptive and sophisticated
backdoor attacks still evade detection and mitigation. We present CodeTracer, a forensic framework that traces malicious code completions back to the
backdoor fine-tuning data responsible for them. Operating under realistic post-deployment constraints, CodeTracer relies solely on the fine-tuning corpus and the reported miscompletion event. It extracts a structured behavioral fingerprint from the compromised output, narrows the search to semantically relevant code samples, and employs LLM-based
reasoning to attribute unsafe logic to specific
backdoor data. Extensive evaluations across three representative vulnerability cases and ten
backdoor attacks, along with sixteen competitive baselines, demonstrate that CodeTracer consistently achieves high forensic accuracy, low false identification rates, and strong robustness against adaptive attacks.
Read the full abstract ▾
All authors Anjun Gao, Yueyang Quan, Zhuqing Liu, Minghong Fang
Machine Learning · Cross-listed from cs.CL
Tool-Making and Self-Evolving LLM Agents in Low-Latency Systems
Kalle Kujanpää, Ning Liu, Shahnawaz Alam et al. (+4 more)
Production LLM
agents often waste latency and reliability by regenerating code for the same procedural steps on every request. We replace this inference-time coding loop with an agentic tool-making pipeline that compiles repeated SOP steps into validated, versioned tools before deployment. The tool-maker grounds synthesis in the live environment as it collects execution traces, observes backend schemas and values, generates candidate tools, and repairs them against labeled cases. At runtime, the production
agent calls these tools directly and falls back to code generation only when needed. We deploy the approach in a Fulfillment Center alarm-triage system, where an
agent diagnoses alarms against a 44-node SOP over heterogeneous metric backends. In production, tool calls reduce p50 latency by 42%. On 1,500 historical alarms, they reduce end-to-end error rate by up to 53% by suppressing run-to-run variance in repeated steps. Because tools return compact structured verdicts, they also enable a simpler direct-call architecture, reducing p50 latency by a further 62% in a controlled ablation. Versioned tools also improve auditability and expose specification gaps and upstream data drift. Our results show that
agents that build and maintain their own tool libraries, a key element of self-evolving
agents, can make industrial LLM systems faster, more reliable, and easier to operate.
Read the full abstract ▾
All authors Kalle Kujanpää, Ning Liu, Shahnawaz Alam, Yeshwanth Reddy Sura, Tianyu Yang, Kristina Klinkner, Shervin Malmasi
Machine Learning · Cross-listed from eess.SP
Unit-Independent Low-Rate Wrist GSR Processing for Stress Detection Using Phasic nSCR Features
Zequan Liang, Sally Hang, Geneva M. Jost et al. (+8 more)
Galvanic skin response (GSR) is widely used for stress detection, but wrist-based GSR remains challenging because its absolute amplitude can differ substantially from laboratory-grade palmar measurements. In this paper, we propose a unit-independent low-rate wrist GSR processing pipeline to extract the number of skin conductance responses per minute (nSCR/min) as a stress-related feature. We collect paired wrist and palmar GSR recordings from 31 participants during sitting baseline, standing baseline, neutral speaking, and the Trier Social Stress Test (TSST), a laboratory social stressor task. The proposed pipeline cleans the raw GSR signal, decomposes it into tonic skin conductance level (SCL) and phasic skin conductance response (SCR), applies robust z-score normalization, and detects phasic SCR peaks to compute nSCR/min. Using random forest on 25Hz We-Be GSR, nSCR/min achieved balanced accuracies of 0.823 and 0.871 for binary classification between TSST and the sitting and standing baselines, respectively. Moreover, the 25Hz We-Be GSR features achieved comparable balanced accuracy to the original 100Hz features across the evaluated tasks. These results suggest the feasibility of low-rate, unit-independent wrist GSR processing for wearable stress detection.
Read the full abstract ▾
All authors Zequan Liang, Sally Hang, Geneva M. Jost, Ning Miao, Wei Shao, Mahdi Pirayesh Shirazi Nejad, Hossein Sayadi, Ehsan Kourkchi, Setareh Rafatirad, Camelia E. Hostinar, Houman Homayoun
Machine Learning · Cross-listed from eess.SP
SpO$_2$ Predictor-Guided Stage-Wise Time-Frequency Reconstruction of Low-Quality Dual-Wavelength PPG for Oxygen Saturation Estimation
Zequan Liang, Elahe Hosseini, Ning Miao et al. (+5 more)
Continuous oxygen saturation (SpO$_2$) estimation from wearable photoplethysmography (PPG) is important for long-term health monitoring, but low-quality red and infrared PPG segments can distort waveform morphology and degrade SpO$_2$ prediction accuracy. Existing PPG denoising and reconstruction methods usually optimize waveform fidelity or heart rate characteristics, while time-domain waveform loss on PPG signals alone insufficiently preserves frequency structure and SpO$_2$-relevant information. This paper proposes a SpO$_2$ predictor-guided stage-wise time-frequency reconstruction framework for low-quality dual-wavelength PPG signals. The proposed method first selects high-quality PPG segments to pretrain a SpO$_2$ predictor. A masked reconstruction model is then trained to recover randomly masked PPG regions using a joint reconstruction objective that combines time-domain waveform loss with frequency-domain loss computed from the short-time
Fourier transform (STFT). To make the reconstruction task physiologically relevant, the pretrained SpO$_2$ predictor is incorporated as an additional constraint, encouraging the reconstructed PPG to preserve SpO$_2$ information rather than only minimizing waveform reconstruction error. The SpO$_2$ predictor and PPG reconstructor model are optimized through four training stages. Experiments on the public OpenOximetry Repository and a private wearable PPG
dataset show that the proposed approach achieves the lowest subject-level MAE, with 2.882% on the public
dataset and 2.359% on the private
dataset.
Read the full abstract ▾
All authors Zequan Liang, Elahe Hosseini, Ning Miao, Mahdi Pirayesh Shirazi Nejad, Wei Shao, Ehsan Kourkchi, Setareh Rafatirad, Houman Homayoun