Machine Learning
FedRD: Reducing Divergences for Generalized Federated Learning via Heterogeneity-aware Parameter Guidance
Kaile Wang, Jiannong Cao, Yu Yang et al. (+2 more)
Heterogeneous
federated learning (HFL) aims to ensure effective and
privacy-preserving collaboration among different entities. As newly joined clients require significant adjustments and additional training to align with the existing system, the problem of generalizing
federated learning models to unseen clients under heterogeneous data has become progressively crucial. Consequently, we highlight two unsolved challenging issues in
federated domain generalization: Optimization Divergence and Performance Divergence. To tackle the above challenges, we propose FedRD, a novel heterogeneity-aware
federated learning
algorithm that collaboratively utilizes parameter-guided global generalization aggregation and local debiased classification to reduce divergences, aiming to obtain an optimal global model for participating and unseen clients. Extensive experiments on public multi-domain
datasets demonstrate that our approach exhibits a substantial performance advantage over competing baselines in addressing this specific problem.
Read the full abstract ▾
All authors Kaile Wang, Jiannong Cao, Yu Yang, Xiaoyin Li, Mingjin Zhang
Machine Learning · Cross-listed from cs.DC
Graph-Structured Deep Learning Framework for Multi-task Contention Identification with High-dimensional Metrics
Xiao Yang, Yinan Ni, Yuqi Tang et al. (+3 more)
This study addresses the challenge of accurately identifying multi-task contention types in high-dimensional system environments and proposes a unified contention classification framework that integrates representation transformation, structural modeling, and a task decoupling mechanism. The method first constructs system state representations from high-dimensional metric sequences, applies nonlinear transformations to extract cross-dimensional dynamic features, and integrates multiple source information such as resource utilization, scheduling behavior, and task load variations within a shared representation space. It then introduces a graph-based modeling mechanism to capture latent dependencies among metrics, allowing the model to learn competitive propagation patterns and structural interference across resource links. On this basis, task-specific mapping structures are designed to model the differences among contention types and enhance the classifier's ability to distinguish multiple contention patterns. To achieve stable performance, the method employs an adaptive multi-task loss weighting strategy that balances shared feature learning with task-specific feature extraction and generates final contention predictions through a standardized inference process. Experiments conducted on a public system trace
dataset demonstrate advantages in accuracy, recall, precision, and F1, and sensitivity analyses on batch size, training sample scale, and metric dimensionality further confirm the model's stability and applicability. The study shows that structured representations and multi-task classification based on high-dimensional metrics can significantly improve contention pattern recognition and offer a reliable technical approach for performance management in complex computing environments.
Read the full abstract ▾
All authors Xiao Yang, Yinan Ni, Yuqi Tang, Zhimin Qiu, Chen Wang, Tingzhou Yuan
Machine Learning
LLM-AutoDP: Automatic Data Processing via LLM Agents for Model Fine-tuning
Wei Huang, Anda Cheng, Yinggui Wang et al. (+2 more)
Large
Language Models (LLMs) can be fine-tuned on domain-specific data to enhance their performance in specialized fields. However, such data often contains numerous low-quality samples, necessitating effective data processing (DP). In practice, DP strategies are typically developed through iterative manual analysis and trial-and-error adjustment. These processes inevitably incur high labor costs and may lead to
privacy issues in high-
privacy domains like healthcare due to direct human access to sensitive data. Thus, achieving automated data processing without exposing the raw data has become a critical challenge. To address this challenge, we propose LLM-AutoDP, a novel framework that leverages LLMs as
agents to automatically generate and optimize data processing strategies. Our method generates multiple candidate strategies and iteratively refines them using feedback signals and comparative evaluations. This iterative in-context learning mechanism enables the
agent to converge toward high-quality processing pipelines without requiring direct human intervention or access to the underlying data. To further accelerate strategy search, we introduce three key techniques: Distribution Preserving Sampling, which reduces data volume while maintaining distributional integrity; Processing Target Selection, which uses a binary classifier to identify low-quality samples for focused processing; Cache-and-Reuse Mechanism}, which minimizes redundant computations by reusing prior processing results. Results show that models trained on data processed by our framework achieve over 80% win rates against models trained on unprocessed data. Compared to AutoML baselines based on LLM
agents, LLM-AutoDP achieves approximately a 65% win rate. Moreover, our acceleration techniques reduce the total searching time by up to 10 times, demonstrating both effectiveness and efficiency.
Read the full abstract ▾
All authors Wei Huang, Anda Cheng, Yinggui Wang, Lei Wang, Tao Wei
Machine Learning
Unsupervised Anomaly Detection in Multi-Agent Trajectory Prediction via Transformer-Based Models
Qing Lyu, Zhe Fu and Alexandre Bayen
Identifying safety-critical scenarios is essential for autonomous driving, but the rarity of such events makes supervised labeling impractical. Traditional rule-based metrics like Time-to-Collision are too simplistic to capture complex interaction risks, and existing methods lack a systematic way to verify whether statistical anomalies truly reflect physical danger. To address this gap, we propose an unsupervised anomaly detection framework based on a multi-
agent Transformer that models normal driving and measures deviations through prediction residuals. A dual evaluation scheme has been proposed to assess both detection stability and physical
alignment: Stability is measured using standard ranking metrics in which Kendall Rank Correlation Coefficient captures rank agreement and Jaccard index captures the consistency of the top-K selected items; Physical
alignment is assessed through correlations with established Surrogate Safety Measures (SSM). Experiments on the NGSIM
dataset demonstrate our framework's effectiveness: We show that the maximum residual aggregator achieves the highest physical
alignment while maintaining stability. Furthermore, our framework identifies 388 unique anomalies missed by Time-to-Collision and statistical baselines, capturing subtle multi-
agent risks like reactive braking under lateral drift. The detected anomalies are further clustered into four interpretable risk types, offering actionable insights for simulation and testing.
Read the full abstract ▾
Machine Learning
Can Continuous-Time Diffusion Models Generate and Solve Globally Constrained Discrete Problems? A Study on Sudoku
Mariia Drozdova
Can standard continuous-time generative models represent distributions whose support is an extremely sparse, globally constrained discrete set? We study this question using completed Sudoku grids as a controlled testbed, treating them as a subset of a continuous relaxation space. We train flow-matching and score-based models along a Gaussian probability path and compare deterministic (ODE) sampling,
stochastic (SDE) sampling, and DDPM-style discretizations derived from the same continuous-time training. Unconditionally,
stochastic sampling substantially outperforms deterministic flows; score-based samplers are the most reliable among continuous-time methods, and DDPM-style ancestral sampling achieves the highest validity overall. We further show that the same models can be repurposed for guided generation: by repeatedly sampling completions under clamped clues and stopping when constraints are satisfied, the model acts as a probabilistic Sudoku solver. Although far less sample-efficient than classical solvers and discrete-geometry-aware
diffusion methods, these experiments demonstrate that classic
diffusion/flow formulations can assign non-zero probability mass to globally constrained combinatorial structures and can be used for constraint satisfaction via
stochastic search.
Read the full abstract ▾
Machine Learning
TINNs: Time-Induced Neural Networks for Solving Time-Dependent PDEs
Chen-Yang Dai, Che-Chia Chang, Te-Sheng Lin et al. (+2 more)
Physics-informed neural networks (PINNs) solve time-dependent partial differential equations (PDEs) by learning a mesh-free, differentiable solution that can be evaluated anywhere in space and time. However, standard space-time PINNs take time as an input but reuse a single network with shared weights across all times, forcing the same features to represent markedly different dynamics. This coupling degrades error performance and can destabilize training when enforcing PDE, boundary, and initial constraints jointly. We propose Time-Induced Neural Networks (TINNs), a novel architecture that parameterizes the network weights as a learned function of time, allowing the effective spatial representation to evolve over time while maintaining shared structure. The resulting formulation naturally yields a nonlinear least-squares problem, which we optimize efficiently using a Levenberg-Marquardt method. Experiments on various time-dependent PDEs show up to 4 times improved relative error and 10 times faster convergence compared to PINNs and strong baselines.
Read the full abstract ▾
All authors Chen-Yang Dai, Che-Chia Chang, Te-Sheng Lin, Ming-Chih Lai, Chieh-Hsin Lai
Machine Learning
TABED: Test-Time Adaptive Ensemble Drafting for Robust Speculative Decoding in LVLMs
Minjae Lee, Wonjun Kang, Byeongkeun Ahn et al. (+6 more)
Speculative decoding (SD) has proven effective for accelerating LLM inference by quickly generating draft
tokens and verifying them in parallel. However, SD remains largely unexplored for Large
Vision-
Language Models (LVLMs), which extend LLMs to process both image and text
prompts. To address this gap, we
benchmark existing inference methods with small draft models on 11
datasets across diverse input scenarios and observe scenario-specific performance fluctuations. Motivated by these findings, we propose Test-time Adaptive Batched Ensemble Drafting (TABED), which dynamically ensembles multiple drafts obtained via batch inference by leveraging deviations from past ground truths available in the SD setting. The dynamic ensemble method achieves an average robust walltime speedup of 1.74x over autoregressive decoding and a 5% improvement over single drafting methods, while remaining training-free and keeping ensembling costs negligible through parameter sharing. With its plug-and-play compatibility, we further enhance TABED by integrating advanced verification and alternative drafting methods. Code and custom-trained models are available at https://github.com/furiosa-ai/TABED.
Read the full abstract ▾
All authors Minjae Lee, Wonjun Kang, Byeongkeun Ahn, Christian Classen, Kevin Galim, Seunghyuk Oh, Minghao Yan, Hyung Il Koo, Kangwook Lee
Machine Learning · Cross-listed from cs.CR
Multimodal Multi-Agent Ransomware Analysis Using AutoGen
Asifullah Khan, Aimen Wadood, Mubashar Iqbal et al. (+1 more)
Ransomware has become one of the most serious cybersecurity threats causing major financial losses and operational disruptions worldwide.Traditional detection methods such as static analysis, heuristic scanning and behavioral analysis often fall short when used alone. To address these limitations, this paper presents
multimodal multi
agent ransomware analysis framework designed for ransomware classification. Proposed
multimodal multiagent architecture combines information from static, dynamic and network sources. Each data type is handled by specialized
agent that uses auto encoder based feature extraction. These representations are then integrated through a fusion
agent. After that fused representation are used by
transformer based classifier. It identifies the specific ransomware family. The
agents interact through an interagent feedback mechanism that iteratively refines feature representations by suppressing low confidence information. The framework was evaluated on large scale
datasets containing thousands of ransomware and benign samples. Multiple experiments were conducted on ransomware
dataset. It outperforms single modality and nonadaptive fusion baseline achieving improvement of up to 0.936 in Macro-F1 for family classification and reducing calibration error. Over 100 epochs, the agentic feedback loop displays a stable monotonic convergence leading to over +0.75 absolute improvement in terms of
agent quality and a final composite score of around 0.88 without fine tuning of the
language models. Zeroday ransomware detection remains family dependent on polymorphism and modality disruptions. Confidence aware abstention enables reliable real world deployment by favoring conservativeand trustworthy decisions over forced classification. The findings indicate that proposed approach provides a practical andeffective path toward improving real world ransomware defense systems.
Read the full abstract ▾
All authors Asifullah Khan, Aimen Wadood, Mubashar Iqbal, Umme Zahoora
Machine Learning · Cross-listed from cs.CL
Improving Diffusion Language Model Decoding through Joint Search in Generation Order and Token Space
Yangyi Shen, Tianjian Feng, Jiaqi Han et al. (+5 more)
Diffusion Language Models (DLMs) offer order-agnostic generation that can explore many possible decoding trajectories. However, current decoding methods commit to a single trajectory, limiting exploration in trajectory space. We introduce Order-
Token Search to explore this space through jointly searching over generation order and
token values. Its core is a likelihood
estimator that scores denoising actions, enabling stable
pruning and efficient exploration of diverse trajectories. Across mathematical
reasoning and coding
benchmarks, Order-
Token Search consistently outperforms baselines on GSM8K, MATH500, Countdown, and HumanEval (3.1%, 3.8%, 7.9%, and 6.8% absolute over backbone), matching or surpassing diffu-GRPO post-trained d1-LLaDA. Our work establishes joint search as a key component for advancing decoding in DLMs.
Read the full abstract ▾
All authors Yangyi Shen, Tianjian Feng, Jiaqi Han, Wen Wang, Tianlang Chen, Chunhua Shen, Jure Leskovec, Stefano Ermon
Machine Learning · Cross-listed from q-fin.CP
Are Whitepaper Claims Reflected in Market Structure? A Contamination-Aware Pipeline and a Power-Limited Null
Murad Farzulla
Do the functional narratives in cryptocurrency whitepapers correspond to their
tokens'
market behaviour? We compare ten-category topical-emphasis profiles for 43 screened documents with seven
market statistics calculated from 2023–2024 exchange data. The primary specification reconstructs USD notional turnover from hourly bars. Dimension-matched Procrustes congruence is $\phi=0.280$ (permutation $p=0.507$), slightly below its permutation-null mean of $0.284$; the zero-padded statistic gives the same non-detection. A four-leg comparison separates document replacement from changes in the assets included. Replacing documents on the 34 common assets changes padded congruence by $-0.014$ under USD turnover and $-0.009$ under base-
token volume. Entity rankings and threshold crossings depend on both composition and specification, so an earlier contamination-only attribution is withdrawn. The documents are not a verified historical corpus: at least two postdate the
market window. Excluding these documents, or excluding all seven assets with shorter histories, does not produce a significant
alignment. Fresh numerical simulations distinguish injected signal from fitted congruence and compare noise restricted to the
market subspace with noise throughout the text space. At the lowest classifier-agreement scenario, detection remains below $43\%$ even at the largest injected signal. These are conditional checks of the
alignment stage, not validation of the text instrument or exclusion bounds on economic effects. The contribution is an auditable non-detection and a specification-sensitive corpus diagnosis, with the inferential limits made explicit.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.RO
Demonstration-Free Robotic Control via LLM Agents
Brian Y. Tsui, Alan Y. Fang and Tiffany J. Hwu
Robotic manipulation has increasingly adopted
vision-
language-action (VLA) models, which achieve strong performance but typically require task-specific demonstrations and fine-tuning, and often generalize poorly under domain shift. We investigate whether general-purpose large
language model (LLM)
agent frameworks, originally developed for software engineering, can serve as an alternative control paradigm for
embodied manipulation. We introduce FAEA (Frontier
Agent as
Embodied Agent), which applies an LLM
agent framework directly to
embodied manipulation without modification. Using the same iterative
reasoning that enables software
agents to debug code, FAEA enables
embodied agents to reason through manipulation strategies. We evaluate an unmodified frontier
agent, Claude
Agent SDK, across the LIBERO, ManiSkill3, and MetaWorld
benchmarks. With privileged environment state access, FAEA achieves success rates of 84.9%, 85.7%, and 96%, respectively. This level of task success approaches that of VLA models trained with less than 100 demonstrations per task, without requiring demonstrations or fine-tuning. With one round of human feedback as an optional optimization, performance increases to 88.2% on LIBERO. This demonstration-free capability has immediate practical value: FAEA can autonomously explore novel scenarios in simulation and generate successful trajectories for training data augmentation in
embodied learning. Our results indicate that general-purpose
agents are sufficient for a class of manipulation tasks dominated by deliberative, task-level planning. This opens a path for robotics systems to leverage actively maintained
agent infrastructure and benefit directly from ongoing advances in frontier models. Code is available at https://github.com/robiemusketeer/faea-sim
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Test-Time Adaptation for Anomaly Segmentation via Topology-Aware Optimal Transport Chaining
Ali Zia, Usman Ali, Umer Ramzan et al. (+3 more)
Deep topological data analysis (TDA) offers a principled framework for capturing structural invariants such as connectivity and cycles that persist across scales, making it a natural fit for anomaly
segmentation (AS). Unlike thresholdbased binarisation, which produces brittle masks under distribution shift, TDA allows anomalies to be characterised as disruptions to global structure rather than local fluctuations. We introduce TopoOT, a
topology-aware optimal transport (OT) framework that integrates multi-filtration persistence diagrams (PDs) with test-time adaptation (TTA). Our key innovation is Optimal Transport Chaining, which sequentially aligns PDs across thresholds and filtrations, yielding geodesic stability scores that identify features consistently preserved across scales. These stabilityaware pseudo-labels supervise a lightweight head trained online with OT-consistency and contrastive objectives, ensuring robust adaptation under domain shift. Across standard 2D and 3D anomaly detection
benchmarks, TopoOT achieves state-of-the-art performance, outperforming the most competitive methods by up to +24.1% mean F1 on 2D
datasets and +10.2% on 3D AS
benchmarks.
Read the full abstract ▾
All authors Ali Zia, Usman Ali, Umer Ramzan, Abdul Rehman, Abdelwahed Khamis, Wei Xiang
Machine Learning
Fengrui Zuo, Zhiwei Ke, Yiming Liu et al. (+3 more)
Diffusion language models (DLMs) generate text through iterative denoising, but inference requires full-sequence
attention at every iteration, resulting in substantial redundant computation on masked
tokens. Block-wise
diffusion can reduce this cost, yet it typically relies on retraining and constrained update orders, limiting its direct applicability to pretrained DLMs. Our
token-level analysis reveals pronounced structural locality in DLM inference. Decoding is driven by a small set of prefix-localized active
tokens; the influence of distant undecoded context diminishes rapidly, and decoded
tokens exhibit stage-wise temporal stability, enabling reuse of intermediate representations except for a brief post-decode transient. Motivated by these observations, we propose \placeholder\footnote{The source code is available at https://github.com/vhicrgit/Window-
Diffusion.}, a window-based
token pruning and caching method for inference. We maintain a local computation window that slides rightward as denoising progresses, and partition undecoded
tokens into: (i) active
tokens that are computed online, (ii) buffer
tokens whose KV states are cached and periodically refreshed, and (iii) far-field
tokens that are pruned outside the window. Computation is restricted to active and buffer
tokens within the window, while far-field
tokens are omitted at each stage. Experiments on LLaDA and Dream show that, under matched compute budgets, our method achieves up to $99\times$ inference speedup while largely preserving generation performance.
Read the full abstract ▾
All authors Fengrui Zuo, Zhiwei Ke, Yiming Liu, Wenqi Lou, Chao Wang, Xuehai Zhou
Machine Learning · Cross-listed from cs.CL
PsychePass: Calibrating LLM Therapeutic Competence via Trajectory-Anchored Tournaments
Zhuang Chen, Dazhen Wan, Zhangkai Zheng et al. (+4 more)
While large
language models show promise in mental healthcare, evaluating their therapeutic competence remains challenging due to the unstructured and longitudinal nature of counseling. We argue that current evaluation paradigms suffer from an unanchored defect, leading to two forms of instability: process drift, where unsteered client simulation wanders away from specific counseling goals, and standard drift, where static pointwise scoring lacks the stability for reliable judgment. To address this, we introduce Ps, a unified framework that calibrates the therapeutic competence of LLMs via trajectory-anchored tournaments. We first anchor the interaction trajectory in simulation, where clients precisely control the fluid consultation process to probe multifaceted capabilities. We then anchor the battle trajectory in judgments through an efficient Swiss-system tournament, utilizing dynamic pairwise battles to yield robust Elo ratings. Beyond ranking, we demonstrate that tournament trajectories can be transformed into credible reward signals, enabling on-policy reinforcement learning to enhance LLMs' performance. Extensive experiments validate the effectiveness of PsychePass and its strong consistency with human expert judgments.
Read the full abstract ▾
All authors Zhuang Chen, Dazhen Wan, Zhangkai Zheng, Guanqun Bi, Xiyao Xiao, Binghang Li, Minlie Huang
Machine Learning · Cross-listed from cs.CL
Beyond Speedup – Utilizing KV Cache for Sampling and Reasoning
Zeyu Xing, Xing Li, Hui-Ling Zhen et al. (+2 more)
KV caches, typically used only to speed up autoregressive decoding, encode contextual information that can be reused for downstream tasks at no extra cost. We propose treating the KV cache as a lightweight representation, eliminating the need to recompute or store full hidden states. Despite being weaker than dedicated
embeddings, KV-derived representations are shown to be sufficient for two key applications: (i) Chain-of-
Embedding, where they achieve competitive or superior performance on Llama-3.1-8B-Instruct and Qwen2-7B-Instruct; and (ii) Fast/Slow Thinking Switching, where they enable adaptive
reasoning on Qwen3-8B and DeepSeek-R1-Distil-Qwen-14B, reducing
token generation by up to $5.7\times$ with minimal accuracy loss. Our findings establish KV caches as a free, effective substrate for sampling and
reasoning, opening new directions for representation reuse in LLM inference. Code: https://github.com/cmd2001/ICLR2026_KV-
Embedding.
Read the full abstract ▾
All authors Zeyu Xing, Xing Li, Hui-Ling Zhen, Mingxuan Yuan, Sinno Jialin Pan
Machine Learning · Cross-listed from cs.IR
Less is More: Benchmarking LLM Based Recommendation Agents
Kargi Chauhan and Mahalakshmi Venkateswarlu
Large
Language Models (LLMs) are increasingly deployed for personalized product recommendations, with practitioners commonly assuming that longer user purchase histories lead to better predictions. We challenge this assumption through a systematic
benchmark of four state of the art LLMs GPT-4o-mini, DeepSeek-V3, Qwen2.5-72B, and Gemini 2.5 Flash across context lengths ranging from 5 to 50 items using the REGEN
dataset. Surprisingly, our experiments with 50 users in a within subject design reveal no significant quality improvement with increased context length. Quality scores remain flat across all conditions (0.17–0.23). Our findings have significant practical implications: practitioners can reduce inference costs by approximately 88% by using context (5–10 items) instead of longer histories (50 items), without sacrificing recommendation quality. We also analyze latency patterns across providers and find model specific behaviors that inform deployment decisions. This work challenges the existing “more context is better” paradigm and provides actionable guidelines for cost effective LLM based recommendation systems.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CR
SemBind: Binding Diffusion Watermarks to Semantics Against Black-Box Forgery Attacks
Xin Zhang, Zijin Yang, Kejiang Chen et al. (+3 more)
Latent-based watermarks, integrated into the generation process of latent
diffusion models (LDMs), simplify detection and attribution of generated images. However, recent black-box forgery attacks, where an attacker needs at least one watermarked image and black-box access to the provider's model, can embed the provider's watermark into images not produced by the provider, posing outsized risk to provenance and trust. We propose SemBind, the first defense framework for latent-based watermarks that resists black-box forgery by binding latent signals to image semantics via a learned semantic masker. Trained with contrastive learning, the masker yields near-invariant codes for the same
prompt and near-orthogonal codes across
prompts; these codes are reshaped and permuted to modulate the target latent before any standard latent-based watermark. SemBind is generally compatible with existing latent-based watermarking schemes and keeps image quality essentially unchanged, while a simple mask-ratio parameter offers a tunable trade-off between anti-forgery strength and robustness. Across four mainstream latent-based watermark methods, our SemBind-enabled anti-forgery variants markedly reduce false acceptance under black-box forgery while providing a controllable robustness-security balance.
Read the full abstract ▾
All authors Xin Zhang, Zijin Yang, Kejiang Chen, Linfeng Ma, Weiming Zhang, Nenghai Yu
Machine Learning · Cross-listed from cs.DC
SuperInfer: SLO-Aware Rotary Scheduling and Memory Management for LLM Inference on Superchips
Jiahuan Yu, Mingtao Hu, Zichao Lin et al. (+1 more)
Large
Language Model (LLM) serving faces a fundamental tension between stringent latency Service Level Objectives (SLOs) and limited GPU
memory capacity. When high request rates exhaust the KV cache budget, existing LLM inference systems often suffer severe head-of-line (HOL) blocking. While prior work explored PCIe-based offloading, these approaches cannot sustain responsiveness under high request rates, often failing to meet tight Time-To-First-
Token (TTFT) and Time-Between-
Tokens (TBT) SLOs. We present SuperInfer, a high-performance LLM inference system designed for emerging Superchips (e.g., NVIDIA GH200) with tightly coupled GPU-CPU architecture via NVLink-C2C. SuperInfer introduces RotaSched, the first proactive, SLO-aware rotary scheduler that rotates requests to maintain responsiveness on Superchips, and DuplexKV, an optimized rotation engine that enables full-duplex transfer over NVLink-C2C. Evaluations on GH200 using various models and
datasets show that SuperInfer improves TTFT SLO attainment rates by up to 74.7% while maintaining comparable TBT and throughput compared to state-of-the-art systems, demonstrating that SLO-aware scheduling and
memory co-design unlocks the full potential of Superchips for responsive LLM serving. Code is available in https://github.com/Supercomputing-System-AI-Lab/SuperInfer.
Read the full abstract ▾
All authors Jiahuan Yu, Mingtao Hu, Zichao Lin, Minjia Zhang
Machine Learning
Delayed Feedback Modeling for Post-Click Gross Merchandise Volume Prediction: Benchmark, Insights and Approaches
Xinyu Li, Sishuo Chen, Guipeng Xv et al. (+7 more)
The prediction objectives of online advertisement ranking models are evolving from probabilistic metrics like conversion rate (CVR) to numerical business metrics like post-click gross merchandise volume (GMV). Unlike the well-studied delayed feedback problem in CVR prediction, delayed feedback modeling for GMV prediction remains unexplored and poses greater challenges, as GMV is a continuous target, and a single click can lead to multiple purchases that cumulatively form the label. To bridge the research gap, we establish TRACE, a GMV prediction
benchmark containing complete transaction sequences rising from each user click, which supports delayed feedback modeling in an online streaming manner. Our analysis and exploratory experiments on TRACE reveal two key insights: (1) the rapid
evolution of the GMV label distribution necessitates modeling delayed feedback under online streaming training; (2) the label distribution of repurchase samples substantially differs from that of single-purchase samples, highlighting the need for separate modeling. Motivated by these findings, we propose RepurchasE-Aware Dual-branch prEdictoR (READER), a novel GMV modeling paradigm that selectively activates expert parameters according to repurchase predictions produced by a router. Moreover, READER dynamically calibrates the regression target to mitigate under-estimation caused by incomplete labels. Experimental results show that READER yields superior performance on TRACE over baselines, achieving a 2.19% improvement in terms of accuracy. We believe that our study will open up a new avenue for studying online delayed feedback modeling for GMV prediction, and our TRACE
benchmark with the gathered insights will facilitate future research and application in this promising direction. Our code and
dataset are available at https://github.com/alimama-tech/OnlineGMV .
Read the full abstract ▾
All authors Xinyu Li, Sishuo Chen, Guipeng Xv, Li Zhang, Mingxuan Luo, Zhangming Chan, Xiang-Rong Sheng, Han Zhu, Jian Xu, Chen Lin
Machine Learning
Truthfulness Despite Weak Supervision: Evaluating and Training LLMs Using Peer Prediction
Tianyi Alex Qiu, Micah Carroll and Cameron Allen
The evaluation and post-training of large
language models (LLMs) rely on supervision, but strong supervision for difficult tasks is often unavailable, especially when evaluating frontier models. In such cases, models are demonstrated to exploit evaluations built on such imperfect supervision, leading to deceptive results. However, underutilized in LLM research, a wealth of mechanism design research focuses on game-theoretic incentive compatibility, i.e., eliciting honest and informative answers with weak supervision. Drawing from this literature, we introduce the peer prediction method for model evaluation and post-training. It rewards honest and informative answers over deceptive and uninformative ones, using a metric based on mutual predictability and without requiring ground truth labels. We demonstrate the method's effectiveness and resistance to deception, with both theoretical guarantees and empirical validation on models with up to 405B parameters. We show that training an 8B model with peer prediction-based reward recovers most of the drop in truthfulness due to prior malicious finetuning, even when the reward is produced by a 0.135B
language model with no finetuning. On the evaluation front, in contrast to LLM-as-a-Judge which requires strong and trusted judges, we discover an inverse scaling property in peer prediction, where, surprisingly, resistance to deception is strengthened as the capability gap between the experts and participants widens, enabling reliable evaluation of strong models with weak supervision. In particular, LLM-as-a-Judge become worse than random guess when facing deceptive models 5-20x the judge's size, while peer prediction thrives when such gaps are large, including in cases with over 100x size difference.
Read the full abstract ▾
Machine Learning
Cheap2Rich: A Multi-Fidelity Framework for Data Assimilation and System Identification of Multiscale Physics – Rotating Detonation Engines
Yuxuan Bao, Jan Zajac, Megan Powers et al. (+2 more)
Bridging the sim2real gap between computationally inexpensive models and complex physical systems remains a central challenge in machine learning applications to engineering problems, particularly in multi-scale settings where reduced-order models typically capture only dominant dynamics. In this work, we present Cheap2Rich, a multi-scale data assimilation framework that reconstructs high-fidelity state spaces from sparse sensor histories by combining a fast low-fidelity prior with learned, interpretable discrepancy corrections. We demonstrate the performance on rotating detonation engines (RDEs), a challenging class of systems that couple detonation-front propagation with injector-driven unsteadiness, mixing, and stiff chemistry across disparate scales. Our approach successfully reconstructs high-fidelity RDE states from sparse measurements while isolating physically meaningful discrepancy dynamics associated with injector-driven effects. The results highlight a general multi-fidelity framework for data assimilation and system identification in complex multi-scale systems, enabling rapid design exploration and real-time monitoring and control while providing interpretable discrepancy dynamics. Code for this project is is available at: github.com/kro0l1k/Cheap2Rich.
Read the full abstract ▾
All authors Yuxuan Bao, Jan Zajac, Megan Powers, Venkat Raman, J. Nathan Kutz
Machine Learning
A Learning-based Framework for Spatial Impulse Response Compensation in 3D Photoacoustic Computed Tomography
Kaiyi Yang, Seonyeong Park, Gangwon Jeong et al. (+4 more)
Photoacoustic computed
tomography (PACT) is a promising imaging modality that combines the advantages of optical contrast with ultrasound detection. Utilizing ultrasound transducers with larger surface areas can improve detection sensitivity. However, when computationally efficient analytic reconstruction methods that neglect the spatial impulse responses (SIRs) of the transducer are employed, the spatial resolution of the reconstructed images will be compromised. Although optimization-based reconstruction methods can explicitly account for SIR effects, their computational cost is generally high, particularly in three-dimensional (3D) applications. To address the need for accurate but rapid 3D PACT image reconstruction, this study presents a framework for establishing a learned SIR compensation method that operates in the data domain. The learned compensation method maps SIR-corrupted PACT measurement data to compensated data that would have been recorded by idealized point-like transducers. Subsequently, the compensated data can be used with a computationally efficient reconstruction method that neglects SIR effects. Two variants of the learned compensation model are investigated that employ a U-Net model and a specifically designed, physics-inspired model, referred to as Deconv-Net. A fast and analytical training data generation procedure is also a component of the presented framework. The framework is rigorously validated in virtual imaging studies, demonstrating resolution improvement and robustness to noise variations, object complexity, and sound speed heterogeneity. When applied to in-vivo breast imaging data, the learned compensation models revealed fine structures that had been obscured by SIR-induced artifacts. To our knowledge, this is the first demonstration of learned SIR compensation in 3D PACT imaging.
Read the full abstract ▾
All authors Kaiyi Yang, Seonyeong Park, Gangwon Jeong, Hsuan-Kai Huang, Alexander A. Oraevsky, Umberto Villa, Mark A. Anastasio
Machine Learning · Cross-listed from cs.IR
One Word is Enough: Minimal Adversarial Perturbations for Neural Text Ranking
Tanmay Karmakar, Sourav Saha, Debapriyo Majumdar et al. (+1 more)
Neural ranking models (NRMs) achieve strong
retrieval effectiveness, yet prior work has shown they are vulnerable to
adversarial perturbations. We revisit this robustness question with a minimal, query-aware attack that promotes a target document by inserting or substituting a single, semantically aligned word - the query center. We study heuristic and gradient-guided variants, including a white-box method that identifies influential insertion points. On TREC-DL 2019/2020 with BERT and monoT5 re-rankers, our single-word attacks achieve up to 91% success while modifying fewer than two
tokens per document on average, achieving competitive rank and score boosts with far fewer edits under a comparable white-box setup to ensure fair evaluation against PRADA. We also introduce new diagnostic metrics to analyze attack sensitivity beyond aggregate success rates. Our analysis reveals a Goldilocks zone in which mid-ranked documents are most vulnerable. These findings demonstrate practical risks and motivate future defenses for robust neural ranking.
Read the full abstract ▾
All authors Tanmay Karmakar, Sourav Saha, Debapriyo Majumdar, Surjyanee Halder
Machine Learning
Memory Retrieval in Transformers: Insights from The Encoding Specificity Principle
Viet Hung Dinh, Ming Ding, Youyang Qu et al. (+1 more)
While explainable artificial intelligence (XAI) for large
language models (LLMs) remains an evolving field with many unresolved questions, increasing regulatory pressures have spurred interest in its role in ensuring transparency, accountability, and
privacy-preserving machine unlearning. Despite recent advances in XAI have provided some insights, the specific role of
attention layers in
transformer based LLMs remains underexplored. This study investigates the
memory mechanisms instantiated by
attention layers, drawing on prior research in psychology and computational psycholinguistics that links
Transformer attention to cue based
retrieval in human
memory. In this view, queries encode the
retrieval context, keys index candidate
memory traces,
attention weights quantify cue trace similarity, and values carry the encoded content, jointly enabling the construction of a context representation that precedes and facilitates
memory retrieval. Guided by the Encoding Specificity Principle, we hypothesize that the cues used in the initial stage of
retrieval are instantiated as keywords. We provide converging evidence for this keywords-as-cues hypothesis. In addition, we isolate
neurons within
attention layers whose activations selectively encode and facilitate the
retrieval of context-defining keywords. Consequently, these keywords can be extracted from identified
neurons and further contribute to downstream applications such as unlearning.
Read the full abstract ▾
All authors Viet Hung Dinh, Ming Ding, Youyang Qu, Kanchana Thilakarathna
Machine Learning
The Forecast After the Forecast: A Post-Processing Shift in Time Series
Daojun Liang, Qi Li, Yinglong Wang et al. (+5 more)
Time series
forecasting has long been dominated by advances in model architecture, with recent progress driven by deep learning and hybrid statistical techniques. However, as
forecasting models approach diminishing returns in accuracy, a critical yet underexplored opportunity emerges: the strategic use of post-processing. In this paper, we address the last-mile gap in time-series
forecasting, which is to improve accuracy and uncertainty without retraining or modifying a deployed backbone. We propose $\delta$-Adapter, a lightweight, architecture-agnostic way to boost deployed time series forecasters without retraining. $\delta$-Adapter learns tiny, bounded modules at two interfaces: input nudging (soft edits to
covariates) and output residual correction. We provide local descent guarantees, $O(\delta)$ drift bounds, and compositional stability for combined adapters. Meanwhile, it can act as a feature selector by learning a sparse, horizon-aware mask over inputs to select important features, thereby improving interpretability. In addition, it can also be used as a distribution calibrator to measure uncertainty. Thus, we introduce a
Quantile Calibrator and a Conformal Corrector that together deliver calibrated, personalized intervals with finite-sample coverage. Our experiments across diverse backbones and
datasets show that $\delta$-Adapter improves accuracy and calibration with negligible compute and no interface changes.
Read the full abstract ▾
All authors Daojun Liang, Qi Li, Yinglong Wang, Jing Chen, Hu Zhang, Xiaoxiao Cui, Qizheng Wang, Shuo Li
Machine Learning · Cross-listed from stat.ML
Empirical Likelihood-Based Fairness Auditing: Distribution-Free Certification and Flagging
Jie Tang, Chuanlong Xie, Xianli Zeng et al. (+1 more)
Machine learning models in high-stakes applications, such as recidivism prediction and automated personnel selection, often exhibit systematic performance disparities across sensitive subpopulations, raising critical concerns regarding algorithmic bias.
Fairness auditing addresses these risks through two primary functions: certification, which verifies adherence to
fairness constraints; and flagging, which isolates specific demographic groups experiencing disparate treatment. However, existing
auditing techniques are frequently limited by restrictive distributional assumptions or prohibitive computational overhead. We propose a novel empirical likelihood-based (EL) framework that constructs robust statistical measures for model performance disparities. Unlike traditional methods, our approach is non-parametric; the proposed disparity statistics follow asymptotically chi-square or mixed chi-square distributions, ensuring valid inference without assuming underlying data distributions. This framework uses a constrained optimization profile that admits stable numerical solutions, facilitating both large-scale certification and efficient subpopulation discovery. Empirically, the EL methods outperform bootstrap-based approaches, yielding coverage rates closer to nominal levels while reducing computational latency by several orders of magnitude. We demonstrate the practical utility of this framework on the COMPAS
dataset, where it successfully flags intersectional biases, specifically identifying a significantly higher positive prediction rate for African-American males under 25 and a systemic under-prediction for Caucasian females relative to the population mean.
Read the full abstract ▾
All authors Jie Tang, Chuanlong Xie, Xianli Zeng, Lixing Zhu
Machine Learning
Robust SDE Parameter Estimation Under Missing Time Information Setting
Long Van Tran, Truyen Tran and Phuoc Nguyen
Recent advances in
stochastic differential equations (SDEs) have enabled robust modeling of real-world dynamical processes across diverse domains, such as finance, health, and systems biology. However, parameter estimation for SDEs typically relies on accurately timestamped observational sequences. When temporal ordering information is corrupted, missing, or deliberately hidden (e.g., for
privacy), existing estimation methods often fail. In this paper, we investigate the conditions under which temporal order can be recovered and introduce a novel framework that simultaneously reconstructs temporal information and estimates SDE parameters. Our approach exploits asymmetries between forward and backward processes, deriving a score-matching criterion to infer the correct temporal order between pairs of observations. We then recover the total order via a sorting procedure and estimate SDE parameters from the reconstructed sequence using maximum likelihood. Finally, we conduct extensive experiments on synthetic and real-world
datasets to demonstrate the effectiveness of our method, extending parameter estimation to settings with missing temporal order and broadening applicability in sensitive domains.
Read the full abstract ▾
Machine Learning
C2:Cross learning module enhanced decision transformer with Constraint-aware loss for auto-bidding
Jinren Ding, Xuejian Xu, Shen Jiang et al. (+3 more)
Decision
Transformer (DT) shows promise for generative auto-bidding by capturing temporal dependencies, but suffers from two critical limitations: insufficient cross-correlation modeling among state, action, and return-to-go (RTG) sequences, and indiscriminate learning of optimal/suboptimal behaviors. To address these, we propose C2, a novel framework enhancing DT with two core innovations: (1) a Cross Learning Block (CLB) via cross-
attention to strengthen inter-sequence correlation modeling; (2) a Constraint-aware Loss (CL) incorporating budget and Cost-Per-Acquisition (CPA) constraints for selective learning of optimal trajectories. Extensive offline evaluations on the AuctionNet
dataset demonstrate consistent performance gains (up to 3.2% over state-of-the-art method) across diverse budget settings; ablation studies verify the complementary synergy of CLB and CL, confirming C2's superiority in auto-bidding. The code for reproducing our results is available at: https://github.com/Dingjinren/C2.
Read the full abstract ▾
All authors Jinren Ding, Xuejian Xu, Shen Jiang, Zhitong Hao, Jinhui Yang, Peng Jiang
Machine Learning
HE-SNR: Uncovering Latent Logic via Entropy for Guiding Mid-Training on SWE-bench
Yueyang Wang, Jiawei Fu, Baolong Bi et al. (+2 more)
SWE-bench has emerged as the premier
benchmark for evaluating Large
Language Models on complex software engineering tasks. While these capabilities are fundamentally acquired during the mid-training phase and subsequently elicited during Supervised Fine-Tuning (SFT), there remains a critical deficit in metrics capable of guiding mid-training effectively. Standard metrics such as Perplexity (PPL) are compromised by the "Long-Context Tax" and exhibit weak correlation with downstream SWE performance. In this paper, we bridge this gap by first introducing a rigorous data filtering strategy. Crucially, we propose the
Entropy Compression Hypothesis, redefining intelligence not by scalar Top-1 compression, but by the capacity to structure uncertainty into
Entropy-Compressed States of low orders ("reasonable hesitation"). Grounded in this fine-grained
entropy analysis, we formulate a novel metric, HE-SNR (High-
Entropy Signal-to-Noise Ratio). We validate our approach on models with up to 560B parameters across different context windows (32K/128K). This work provides both the theoretical foundation and practical tools for optimizing the latent potential of LLMs in complex engineering domains.
Read the full abstract ▾
All authors Yueyang Wang, Jiawei Fu, Baolong Bi, Xili Wang, Xiaoqing Liu
Machine Learning · Cross-listed from stat.ML
Efficient Evaluation of LLM Performance with Statistical Guarantees
Skyler Wu, Yash Nair and Emmanuel J. Candès
Exhaustively evaluating many large
language models (LLMs) on a large suite of
benchmarks is expensive. We cast benchmarking as finite-population inference and, under a fixed query budget, seek tight confidence intervals (CIs) for model accuracy with asymptotic frequentist coverage. We propose Factorized Active Querying (FAQ), which (a) leverages historical information through a
Bayesian factor model; (b) adaptively selects questions using a hybrid variance-reduction/active-learning sampling policy; and (c) maintains validity through Pro-Active Inference—a finite-population extension of active inference (Zrnic & Candès, 2024) that enables direct question selection while preserving asymptotic frequentist coverage. In our experiments, FAQ delivers up to $5\times$ effective sample size gains over strong baselines on two
benchmark suites, across varying historical-data missingness levels, with negligible computational overhead: this means that it matches the CI width of uniform sampling while using up to $5\times$ fewer queries. We release our source code and our curated
datasets to support reproducibility and future research.
Read the full abstract ▾