Machine Learning
Learnability and Privacy Vulnerability are Entangled in a Few Critical Weights
Xingli Fang and Jung-Eun Kim
Prior approaches for membership
privacy preservation usually update or retrain all weights in neural networks, which is costly and can lead to unnecessary utility loss or even more serious misalignment in predictions between training data and non-training data. In this work, we observed three insights: i)
privacy vulnerability exists in a very small fraction of weights; ii) however, most of those weights also critically impact utility performance; iii) the importance of weights stems from their locations rather than their values. According to these insights, to preserve
privacy, we score critical weights, and instead of discarding those
neurons, we rewind only the weights for fine-tuning. We show that, through extensive experiments, this mechanism exhibits outperforming resilience in most cases against Membership Inference Attacks while maintaining utility.
Read the full abstract ▾
Machine Learning
MXNorm: Reusing MXFP block scales for efficient tensor normalisation
Callum McLean, Luke Y. Prince, Alexandre Payot et al. (+2 more)
Matrix multiplication performance has long been the major bottleneck to scaling deep learning workloads, which has stimulated the design of new accelerators that use increasingly low-precision number formats. However, improvements in
matrix multiplication performance have far outstripped improvements in performance on reductions and elementwise computations, which are still being performed in higher precision. In this work, we propose MXNorm, a drop-in replacement for RMSNorm that estimates the RMS using only the block scales calculated as part of the MXFP8 cast and enables a 32x decrease in the size of reduction needed for normalization. We validate our approximation method on pre-training of Llama 3 models of 125M, 1B and 8B parameters, finding minimal loss of training accuracy compared to a baseline using RMSNorm with MXFP8 matmuls. We also show practical kernel speedups using only torch.compile of up to 2.4x for MXNorm over RMSNorm, corresponding to a 1.3% speedup in Llama 3 8B
transformer layers in MXFP8 and a 2.6% speedup in NVFP4.
Read the full abstract ▾
All authors Callum McLean, Luke Y. Prince, Alexandre Payot, Paul Balança, Carlo Luschi
Machine Learning · Cross-listed from cs.CV
Towards Faithful Multimodal Concept Bottleneck Models
Pierre Moreau, Emeline Pineau Ferrand, Yann Choho et al. (+3 more)
Concept Bottleneck Models (CBMs) are interpretable models that route predictions through a layer of human-interpretable concepts. While widely studied in
vision and, more recently, in NLP, CBMs remain largely unexplored in
multimodal settings. For their explanations to be faithful, CBMs must satisfy two conditions: concepts must be properly detected, and concept representations must encode only their intended semantics, without smuggling extraneous task-relevant or inter-concept information into final predictions, a phenomenon known as leakage. Existing approaches treat concept detection and leakage mitigation as separate problems, and typically improve one at the expense of predictive accuracy. In this work, we introduce f-CBM, a faithful
multimodal CBM framework built on a
vision-
language backbone that jointly targets both aspects through two complementary strategies: a differentiable leakage loss to mitigate leakage, and a Kolmogorov-Arnold Network prediction head that provides sufficient expressiveness to improve concept detection. Experiments demonstrate that f-CBM achieves the best trade-off between task accuracy, concept detection, and leakage reduction, while applying seamlessly to both image and text or text-only
datasets, making it versatile across modalities.
Read the full abstract ▾
All authors Pierre Moreau, Emeline Pineau Ferrand, Yann Choho, Benjamin Wong, Annabelle Blangero, Milan Bhan
Machine Learning
ZO-SAM: Zero-Order Sharpness-Aware Minimization for Efficient Sparse Training
Jie Ji, Gen Li, Kaiyuan Deng et al. (+2 more)
Deep learning models, despite their impressive achievements, suffer from high computational costs and
memory requirements, limiting their usability in resource-constrained environments. Sparse neural networks significantly alleviate these constraints by dramatically reducing parameter count and computational overhead. However, existing sparse training methods often experience chaotic and noisy gradient signals, severely hindering convergence and generalization performance, particularly at high sparsity levels. To tackle this critical challenge, we propose Zero-Order Sharpness-Aware Minimization (ZO-SAM), a novel optimization framework that strategically integrates zero-order optimization within the SAM approach. Unlike traditional SAM, ZO-SAM requires only a single backpropagation step during perturbation, selectively utilizing zero-order gradient estimations. This innovative approach reduces the backpropagation computational cost by half compared to conventional SAM, significantly lowering gradient variance and effectively eliminating associated computational overhead. By harnessing SAM's capacity for identifying flat minima, ZO-SAM stabilizes the training process and accelerates convergence. These efficiency gains are particularly important in sparse training scenarios, where computational cost is the primary bottleneck that limits the practicality of SAM. Moreover, models trained with ZO-SAM exhibit improved robustness under distribution shift, further broadening its practicality in real-world deployments.
Read the full abstract ▾
All authors Jie Ji, Gen Li, Kaiyuan Deng, Fatemeh Afghah, Xiaolong Ma
Machine Learning
BoSS: A Best-of-Strategies Selector as an Oracle for Deep Active Learning
Denis Huseljic, Paul Hahn, Marek Herde et al. (+2 more)
Active learning (AL) aims to reduce annotation costs while maximizing model performance by iteratively selecting valuable instances. While foundation models have made it easier to identify these instances, existing selection strategies still lack robustness across different models, annotation budgets, and
datasets. To highlight the potential weaknesses of existing AL strategies and provide a reference point for research, we explore oracle strategies, i.e., strategies that approximate the optimal selection by accessing ground-truth information unavailable in practical AL scenarios. Current oracle strategies, however, fail to scale effectively to large
datasets and complex deep neural networks. To tackle these limitations, we introduce the Best-of-Strategy Selector (BoSS), a scalable oracle strategy designed for large-scale AL scenarios. BoSS constructs a set of candidate batches through an ensemble of selection strategies and then selects the batch yielding the highest performance gain. As an ensemble of selection strategies, BoSS can be easily extended with new state-of-the-art strategies as they emerge, ensuring it remains a reliable oracle strategy in the future. Our evaluation demonstrates that i) BoSS outperforms existing oracle strategies, ii) state-of-the-art AL strategies still fall noticeably short of oracle performance, especially in large-scale
datasets with many classes, and iii) one possible solution to counteract the inconsistent performance of AL strategies might be to employ an ensemble-based approach for the selection.
Read the full abstract ▾
All authors Denis Huseljic, Paul Hahn, Marek Herde, Christoph Sandrock, Bernhard Sick
Machine Learning
Breaking the Tuning Barrier: Zero-Hyperparameters Yield Multi-Corner Analysis Via Learned Priors
Wei W. Xing, Kaiqi Huang, Jiazhan Liu et al. (+2 more)
Yield Multi-Corner Analysis validates
circuits across 25+ Process-Voltage-Temperature corners, resulting in a combinatorial simulation cost of $O(K \times N)$ where $K$ denotes corners and $N$ exceeds $10^4$ samples per corner. Existing methods face a fundamental trade-off: simple models achieve automation but fail on nonlinear
circuits, while advanced AI models capture complex behaviors but require hours of hyperparameter tuning per design iteration, forming the Tuning Barrier. We break this barrier by replacing engineered priors (i.e., model specifications) with learned priors from a foundation model pre-trained on millions of regression tasks. This model performs in-context learning, instantly adapting to each
circuit without tuning or retraining. Its
attention mechanism automatically transfers knowledge across corners by identifying shared
circuit physics between operating conditions. Combined with an automated feature selector (1152D to 48D), our method matches state-of-the-art accuracy (mean MREs as low as 0.11%) with zero tuning, reducing total validation cost by over $10\times$.
Read the full abstract ▾
All authors Wei W. Xing, Kaiqi Huang, Jiazhan Liu, Hong Qiu, Shan Shen
Machine Learning
Linearized Attention Cannot Enter the Kernel Regime at Any Practical Width
Jose Marie Antonio Miñoza, Paulo Mario P. Medina and Sebastian C. Ibañez
Understanding whether
attention mechanisms converge to the kernel regime is foundational to the validity of influence functions for
transformer accountability. Exact NTK characterization of softmax
attention is precluded by its exponential nonlinearity; linearized
attention is the canonical tractable proxy and the object of study here. This paper establishes that even this proxy does not converge to its NTK limit at any practical width, revealing a fundamental trade-off in the learning dynamics of
attention. An exact correspondence is established between parameter-free linearized
attention and a data-dependent Gram-induced kernel; spectral amplification analysis shows that the
attention transformation cubes the Gram
matrix's condition number, requiring width $m = \Omega(\kappa_d(\mathbf{G})^6 n\log n)$ for NTK convergence, where $\kappa_d(\mathbf{G})$ is the effective condition number of the rank-$\min(n,d)$ truncation of the input Gram
matrix; for natural image
datasets this threshold is physically infeasible ($m \gg 10^{24}$ for MNIST and $m \gg 10^{29}$ for CIFAR-10, 12–17 orders of magnitude beyond the largest known architectures). Influence malleability is introduced to characterize this non-convergence: linearized
attention exhibits 2–9$\times$ higher malleability than ReLU networks under
adversarial data perturbation, with the gap depending on
dataset condition number and task setting. A dual implication is established: the same data-dependent kernel is shown theoretically to reduce approximation error when targets align with the data geometry, while, empirically, creating vulnerability to
adversarial manipulation of the training data. The structural argument extends to trainable QKV
attention under standard initialization, with direct consequences for influence methods applied to deployed
transformer architectures.
Read the full abstract ▾
Machine Learning
Fractals made Practical: Denoising Diffusion as Partitioned Iterated Function Systems
Ann Dooms
What is a
diffusion model actually doing when it turns noise into a photograph? We show that the deterministic DDIM reverse chain operates as a Partitioned Iterated Function System (PIFS) and that this framework serves as a unified design
language for denoising
diffusion model schedules, architectures, and training objectives. From the PIFS structure we derive three computable
geometric quantities: a per-step contraction threshold $L^*_t$, a diagonal expansion function $f_t(\lambda)$ and a global expansion threshold $\lambda^{**}$. These quantities require no model evaluation and fully characterize the denoising dynamics. They structurally explain the two-regime behavior of
diffusion models: global context assembly at high noise via diffuse cross-patch
attention and fine-detail synthesis at low noise via patch-by-patch suppression release in strict variance order. Self-
attention emerges as the natural primitive for PIFS contraction. The Kaplan-Yorke dimension of the PIFS attractor is determined analytically through a discrete Moran equation on the Lyapunov spectrum. Through the study of the fractal geometry of the PIFS, we derive three optimal design criteria and show that four prominent empirical design choices (the cosine schedule offset, resolution-dependent logSNR shift, Min-SNR loss weighting, and Align Your Steps sampling) each arise as approximate solutions to our explicit
geometric optimization problems tuning theory into practice.
Read the full abstract ▾
Machine Learning
GeoChemAD: Benchmarking Unsupervised Geochemical Anomaly Detection for Mineral Exploration
Yihao Ding, Yiran Zhang, Chris Gonzalez et al. (+2 more)
Geochemical anomaly detection plays a critical role in mineral exploration as deviations from regional geochemical baselines may indicate mineralization. Existing studies suffer from two key limitations: (1) single region scenarios which limit model generalizability; (2) proprietary
datasets, which makes result reproduction unattainable. In this work, we introduce GeoChemAD, an open-source
benchmark dataset compiled from government-led geological surveys, covering multiple regions, sampling sources, and target elements. The
dataset comprises eight subsets representing diverse spatial scales and sampling conditions. To establish strong baselines, we reproduce and
benchmark a range of unsupervised anomaly detection methods, including statistical models, generative and
transformer-based approaches. Furthermore, we propose GeoChemFormer, a
transformer-based framework that leverages self-supervised pretraining to learn target-element-aware geochemical representations for spatial samples. Extensive experiments demonstrate that GeoChemFormer consistently achieves superior and robust performance across all eight subsets, outperforming existing unsupervised methods in both anomaly detection accuracy and generalization capability. The proposed
dataset and framework provide a foundation for reproducible research and future development in this direction.
Read the full abstract ▾
All authors Yihao Ding, Yiran Zhang, Chris Gonzalez, Eun-Jung Holden, Wei Liu
Machine Learning
L2GTX: From Local to Global Time Series Explanations
Ephrem Tibebe Mekonnen, Luca Longo, Lucas Rizzo et al. (+1 more)
Deep learning models achieve high accuracy in time series classification, yet understanding their class-level decision behaviour remains challenging. Explanations for time series must respect temporal dependencies and identify patterns that recur across instances. Existing approaches face three limitations: model-agnostic XAI methods developed for images and tabular data do not readily extend to time series, global explanation synthesis for time series remains underexplored, and most existing global approaches are model-specific. We propose L2GTX, a model-agnostic framework that generates class-wise global explanations by aggregating local explanations from a representative set of instances. L2GTX extracts clusters of parameterised temporal event primitives, such as increasing or decreasing trends and local extrema, together with their importance scores from instance-level explanations produced by LOMATCE. These clusters are merged across instances to reduce redundancy, and an instance-cluster importance
matrix is used to estimate global relevance. Under a user-defined instance selection budget, L2GTX selects representative instances that maximise coverage of influential clusters. Events from the selected instances are then aggregated into concise class-wise global explanations. Experiments on six
benchmark time series
datasets show that L2GTX produces compact and interpretable global explanations while maintaining stable global faithfulness measured as mean local surrogate fidelity.
Read the full abstract ▾
All authors Ephrem Tibebe Mekonnen, Luca Longo, Lucas Rizzo, Pierpaolo Dondio
Machine Learning
Competition-Aware CPC Forecasting with Near-Market Coverage
Sebastian Frey, Edoardo Beccari, Maximilian Kranz et al. (+4 more)
Cost-per-click (CPC) in paid search is an auction-generated outcome shaped by a competitive landscape that is only partially observable from any single advertiser's history. From 1.66 billion Google Ads log records for a concentrated car-rental
market (2021-2023), we construct a weekly panel of 1,811 keyword series over 127 weeks (218,924 keyword-week observations) and build competition-aware proxies from keyword text, CPC trajectories, and geographic
market structure. The design combines (i) semantic neighborhoods and a semantic keyword graph from pretrained
transformer-based keyword representations, (ii) behavioral neighborhoods from Dynamic Time Warping (DTW)
alignment of CPC trajectories, and (iii) geographic-intent
covariates capturing localized demand and marketplace heterogeneity. We evaluate these signals both as exogenous
covariates and as relational priors in spatiotemporal graph forecasters, benchmarking them against statistical, neural, and time-series foundation-model baselines. The results reveal a clear horizon crossover. At one week, graph-based models achieve the lowest error, reducing sMAPE by 15.1% relative to the strongest classical/ML baseline; at the six- and twelve-week horizons,
covariate-augmented foundation models dominate, reducing sMAPE by 22.5% and 27.6%, respectively. The gains concentrate in the high-CPC, high-
volatility keywords where
forecasting errors are most costly. A falsification battery supports the competition interpretation at the planning horizon: the semantic competition graph outperforms a confounder-matched non-competitive graph by 4.05 sMAPE points, and matched-neighbour and time-shuffled controls show the six-week gains are competition-specific rather than generic smoothing. Together, the findings establish a horizon-dependent competition-aware
forecasting design for auction-driven advertising
markets under partial observability.
Read the full abstract ▾
All authors Sebastian Frey, Edoardo Beccari, Maximilian Kranz, Nicolò Alberto Pellizzari, Ali Mete Karaman, Qiwei Han, Maximilian Kaiser
Machine Learning
Causal Cellular Context Transfer Learning (C3TL): An Efficient Architecture for Prediction of Unseen Perturbation Effects
Michael Scholkemper and Sach Mukherjee
Predicting the effects of chemical and genetic perturbations on quantitative
cell states is a central challenge in computational biology, molecular medicine and drug discovery. Recent work has leveraged large-scale single-
cell data and massive foundation models to address this task. However, such computational resources and extensive
datasets are not always accessible in academic or clinical settings, hence limiting utility. Here we propose a lightweight framework for perturbation effect prediction that exploits the structured nature of biological interventions and specific inductive biases/invariances. Our approach leverages available information concerning perturbation effects to allow generalization to novel contexts and requires only widely-available bulk molecular data. Extensive testing, comparing predictions of context-specific perturbation effects against real, large-scale interventional experiments, demonstrates accurate prediction in new contexts. The proposed approach is competitive with SOTA foundation models but requires simpler data, much smaller model sizes and less time. Focusing on robust bulk signals and efficient architectures, we show that accurate prediction of perturbation effects is possible without proprietary hardware or very large models, hence opening up ways to leverage causal learning approaches in biomedicine generally.
Read the full abstract ▾
Machine Learning
3DTCR: Vortex-following generative reconstruction improves tropical cyclone intensity prediction from global AI weather forecasts
Jun Liu, Xiaohui Zhong, Kai Zheng et al. (+5 more)
Global numerical weather prediction (NWP) and artificial intelligence (AI) models often underestimate tropical cyclone (TC) intensity, partly because coarse grids underresolve fine-scale three-dimensional inner-core structures that influence TC intensity
evolution. Kilometer-scale NWP simulations can better resolve these structures but are too costly for operational use, which also makes high-resolution TC data limited. Here we construct a 3-km, TC-centered
dataset for 2018–2024 using the Weather Research and
Forecasting (WRF) model and develop 3DTCR, a
vortex-following generative framework that reconstructs three-dimensional TC structure from FuXi global forecasts. 3DTCR combines conditional flow matching (CFM) with two-stage training to learn a coarse-to-fine mapping from center-aligned reanalysis simulation pairs and then adapt it to forecast inputs through latent space
alignment. Evaluated on 2024 western North Pacific (WNP) forecasts, 3DTCR reduces the root mean square error (RMSE) and the magnitude of the mean bias error (MBE) in maximum 10-m wind speed by 36.5% and 60.7% relative to FuXi, and by 8.9% and 28.2% relative to the European Centre for Medium-Range Weather Forecasts high-resolution system (ECMWF-HRES), respectively. Case studies suggest that reconstructions capture wind gradients between the eye and eyewall, vertical structure of horizontal winds, and
vortex asymmetries, while inheriting the FuXi-predicted TC tracks. 3DTCR generates 5-day TC-centered atmospheric fields in approximately 3 minutes on one GPU, whereas a full-basin 3-km WRF simulation takes about 400 hours on 32 CPU cores. These results demonstrate that such a
vortex-following generative reconstruction can add localized three-dimensional detail to global forecasts and improve TC intensity prediction without basin-wide kilometer-scale numerical integration.
Read the full abstract ▾
All authors Jun Liu, Xiaohui Zhong, Kai Zheng, Jiarui Li, Yifei Li, Tao Zhou, Wenwen Zhang, Hao Li
Machine Learning · Cross-listed from math.OC
Convergence Rate of a Functional Learning Method for Contextual Stochastic Optimization
Noel Smith and Andrzej Ruszczynski
We consider a
stochastic optimization problem involving two random variables: a context variable $X$ and a dependent variable $Y$. The objective is to minimize the expected value of a nonlinear loss functional applied to the conditional expectation $\mathbb{E}[f(X, Y,\beta) \mid X]$, where $f$ is a nonlinear function and $\beta$ represents the decision variables. We focus on the practically important setting in which direct sampling from the conditional distribution of $Y \mid X$ is infeasible, and only a stream of i.i.d. observation pairs $\{(X^k, Y^k)\}_{k=0,1,2,\ldots}$ is available. In our approach, the conditional expectation is approximated within a prespecified parametric function class. We analyze a simultaneous learning-and-optimization
algorithm that jointly estimates the conditional expectation and optimizes the outer objective. Using a specially designed measure of non-optimality, combining the squared norm of the objective function's gradient and the mean square error of the auxiliary parametric model, we establish that the method achieves a convergence rate of order $\mathcal{O}\big(1/\sqrt{N}\big)$, where $N$ denotes the number of observed pairs.
Read the full abstract ▾
Machine Learning
OpenACMv2: An Accuracy-Constrained Co-Optimization Framework for Approximate DCiM
Yiqi Zhou, Yue Yuan, Yikai Wang et al. (+8 more)
Digital Compute-in-
Memory (DCiM) accelerates neural networks by reducing data movement. Approximate DCiM can further improve power-performance-area (PPA), but demands accuracy-constrained co-optimization across coupled architecture and transistor-level choices. Building on OpenYield, we introduce Accuracy-Constrained Co-Optimization (ACCO) and present OpenACMv2, an open framework that operationalizes ACCO via two-level optimization: (1) accuracy-constrained architecture search of compressor combinations and SRAM macro parameters, driven by a fast GNN-based surrogate for PPA and error; and (2) variation- and PVT-aware transistor sizing for standard
cells and SRAM bitcells using Monte Carlo. By decoupling ACCO into architecture-level exploration and
circuit-level sizing, OpenACMv2 integrates classic single- and multi-objective optimizers to deliver strong PPA-accuracy tradeoffs and robust convergence. The workflow is compatible with FreePDK45 and OpenROAD, supporting reproducible evaluation and easy adoption. Experiments show that the proposed two-level ACCO framework achieves most of its accuracy-constrained efficiency gain at Level-1 through architecture exploration, delivering roughly 50%+ PDP reduction, while Level-2 transistor-level optimization provides a further single-digit PDP improvement while preserving accuracy, enabling rapid "what-if" exploration for approximate DCiM. The framework is available on GitHub (https://github.com/ShenShan123/OpenACM).
Read the full abstract ▾
All authors Yiqi Zhou, Yue Yuan, Yikai Wang, Bohao Liu, Qinxin Mei, Zhuohua Liu, Shan Shen, Wei Xing, Daying Sun, Li Li, Guozhu Liu
Machine Learning · Cross-listed from cs.NE
Federated Few-Shot Learning on Neuromorphic Hardware: An Empirical Study Across Physical Edge Nodes
Steven Motta and Gioele Nanni
Federated learning on neuromorphic hardware remains unexplored because on-chip spike-timing-dependent plasticity (STDP) produces binary weight updates rather than the floating-point gradients assumed by standard
algorithms. We build a two-node
federated system with BrainChip Akida AKD1000 processors and run approximately 1,580 experimental trials across seven analysis phases. Of four weight-exchange strategies tested,
neuron-level concatenation (FedUnion) consistently preserves accuracy while element-wise weight averaging (FedAvg) destroys it (p = 0.002). Domain-adaptive fine-tuning of the upstream feature extractor accounts for most of the accuracy gains, confirming feature quality as the dominant factor. Scaling feature dimensionality from 64 to 256 yields 77.0% best-strategy
federated accuracy (n=30, p < 0.001). Two independent asymmetries (wider features help federation more than individual learning, while binarization hurts federation more) point to a shared prototype complementarity mechanism: cross-node transfer scales with the distinctiveness of
neuron prototypes.
Read the full abstract ▾
Machine Learning · Cross-listed from eess.SP
Association-Aware GNN for Precoder Learning in Cell-Free Systems
Mingyu Deng and Shengqian Han
Deep learning has been widely recognized as a promising approach for optimizing multi-user multi-
antenna precoders in traditional cellular systems. However, a critical distinction between
cell-free and cellular systems lies in the flexibility of user equipment (UE)-access point (AP) associations. Consequently, the optimal precoder depends not only on channel state information but also on the dynamic UE-AP association status. In this paper, we propose an association-aware graph neural network (AAGNN) that explicitly incorporates association status into the precoding design. We leverage the permutation equivariance properties of the
cell-free precoding policy to reduce the training complexity of AAGNN and employ an
attention mechanism to enhance its generalization performance. Simulation results demonstrate that the proposed AAGNN outperforms baseline learning methods in both learning performance and generalization capabilities while maintaining low training and inference complexity.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Yanpeng Zhao, Wentao Ding, Hongtao Li et al. (+2 more)
A recent trend in
vision-
language models (VLMs) has been to enhance their spatial cognition for
embodied domains. Despite progress, existing evaluations have been limited both in paradigm and in coverage, hindering rapid, iterative model development. To address these limitations, we propose ESPIRE, a diagnostic
benchmark for
embodied spatial
reasoning. ESPIRE offers a simulated world that physically grounds VLMs and evaluates them on spatial-
reasoning-centric robotic tasks, thus narrowing the gap between evaluation and real-world deployment. To adapt VLMs to robotic tasks, we decompose each task into localization and execution, and frame both as generative problems, in stark contrast to predominant discriminative evaluations (e.g., via visual-question answering) that rely on distractors and discard execution. This decomposition further enables a fine-grained analysis beyond passive spatial
reasoning toward
reasoning to act. We systematically design ESPIRE both at the instruction level and at the environment level, ensuring broad coverage of spatial
reasoning scenarios. We use ESPIRE to diagnose a range of frontier VLMs and provide in-depth analysis of their spatial
reasoning behaviors.
Read the full abstract ▾
All authors Yanpeng Zhao, Wentao Ding, Hongtao Li, Baoxiong Jia, Zilong Zheng
Machine Learning · Cross-listed from cs.CV
SortScrews: A Dataset and Baseline for Real-time Screw Classification
Tianhao Fu, Bingxuan Yang, Juncheng Guo et al. (+2 more)
Automatic identification of screw types is important for industrial automation, robotics, and inventory management. However, publicly available
datasets for screw classification are scarce, particularly for controlled single-object scenarios commonly encountered in automated sorting systems. In this work, we introduce $\textbf{SortScrews}$, a
dataset for casewise visual classification of screws. The
dataset contains 560 RGB images at $512\times512$ resolution covering six screw types and a background class. Images are captured using a standardized acquisition setup and include mild variations in lighting and camera perspective across four capture settings. To facilitate reproducible research and
dataset expansion, we also provide a reusable data collection script that allows users to easily construct similar
datasets for custom hardware components using inexpensive camera setups. We establish baseline results using transfer learning with EfficientNet-B0 and ResNet-18 classifiers pretrained on ImageNet. In addition, we conduct a well-explored failure analysis. Despite the limited
dataset size, these lightweight models achieve strong classification accuracy, demonstrating that controlled acquisition conditions enable effective learning even with relatively small
datasets. The
dataset, collection pipeline, and baseline training code are publicly available at https://github.com/ATATC/SortScrews.
Read the full abstract ▾
All authors Tianhao Fu, Bingxuan Yang, Juncheng Guo, Shrena Sribalan, Yucheng Chen
Machine Learning
PISmith: Reinforcement Learning-based Red Teaming for Prompt Injection Defenses
Chenlong Yin, Runpeng Geng, Yanting Wang et al. (+1 more)
Prompt injection poses serious security risks to real-world LLM applications, particularly autonomous
agents. Although many defenses have been proposed, their robustness against adaptive attacks remains insufficiently evaluated, potentially creating a false sense of security. In this work, we propose PISmith, a reinforcement learning (RL)-based red-teaming framework that systematically assesses existing
prompt-injection defenses by training an attack LLM to optimize injected
prompts in a practical black-box setting, where the attacker can only query the defended LLM and observe its outputs. We find that directly applying standard GRPO to attack strong defenses leads to sub-optimal performance due to extreme reward sparsity – most generated injected
prompts are blocked by the defense, causing the policy's
entropy to collapse before discovering effective attack strategies, while the rare successes cannot be learned effectively. In response, we introduce adaptive
entropy regularization and dynamic advantage weighting to sustain exploration and amplify learning from scarce successes. Extensive evaluation on 13
benchmarks demonstrates that state-of-the-art
prompt injection defenses remain vulnerable to adaptive attacks. We also compare PISmith with 7 baselines across static, search-based, and RL-based attack categories, showing that PISmith consistently achieves the highest attack success rates. Furthermore, PISmith achieves strong performance in agentic settings on InjecAgent and AgentDojo against both open-source and closed-source LLMs (e.g., GPT-4o-mini and GPT-5-nano). Our code is available at https://github.com/albert-y1n/PISmith.
Read the full abstract ▾
All authors Chenlong Yin, Runpeng Geng, Yanting Wang, Jinyuan Jia
Machine Learning · Cross-listed from cs.CV
SAW: Toward a Surgical Action World Model via Controllable and Scalable Video Generation
Sampath Rapuri, Lalithkumar Seenivasan, Dominik Schneider et al. (+9 more)
A
surgical world model capable of generating realistic
surgical action
videos with precise control over tool-tissue interactions can address fundamental challenges in
surgical AI and simulation – from data scarcity and rare event synthesis to bridging the sim-to-real gap for
surgical automation. However, current
video generation methods, the very core of such
surgical world models, require expensive annotations or complex structured intermediates as conditioning signals at inference, limiting their scalability. Other approaches exhibit limited temporal consistency across complex laparoscopic scenes and do not possess sufficient realism. We propose
Surgical Action World (SAW) – a step toward
surgical action world modeling through
video diffusion conditioned on four lightweight signals:
language prompts encoding tool-action context, a reference
surgical scene, tissue affordance mask, and 2D tool-tip trajectories. We design a conditional
video diffusion approach that reformulates
video-to-
video diffusion into trajectory-conditioned
surgical action synthesis. The backbone
diffusion model is fine-tuned on a custom-curated
dataset of 12,044 laparoscopic clips with lightweight spatiotemporal conditioning signals, leveraging a depth consistency loss to enforce
geometric plausibility without requiring depth at inference. SAW achieves state-of-the-art temporal consistency (CD-FVD: 199.19 vs. 546.82) and strong visual quality on held-out test data. Furthermore, we demonstrate its downstream utility for (a)
surgical AI, where augmenting rare actions with SAW-generated
videos improves action recognition (clipping F1-score: 20.93% to 43.14%; cutting: 0.00% to 8.33%) on real test data, and (b)
surgical simulation, where rendering tool-tissue interaction
videos from simulator-derived trajectory points toward a visually faithful simulation engine.
Read the full abstract ▾
All authors Sampath Rapuri, Lalithkumar Seenivasan, Dominik Schneider, Roger Soberanis-Mukul, Yufan He, Hao Ding, Jiru Xu, Chenhao Yu, Chenyan Jing, Pengfei Guo, Daguang Xu, Mathias Unberath
Machine Learning · Cross-listed from cs.DC
ARL-Tangram: Unleash the Resource Efficiency in Agentic Reinforcement Learning
Bangjun Xiao, Yihao Zhao, Xiangwei Deng et al. (+9 more)
Agentic reinforcement learning (RL) has emerged as a transformative workload in cloud clusters, enabling large
language models (LLMs) to solve complex problems through interactions with real world. However, unlike traditional RL, agentic RL demands substantial external cloud resources, e.g., CPUs for code execution and GPUs for reward models, that exist outside the primary training cluster. Existing agentic RL framework typically rely on static over-provisioning, i.e., resources are often tied to long-lived trajectories or isolated by tasks, which leads to severe resource inefficiency. We propose the action-level orchestration, and incorporate it into ARL-Tangram, a unified resource management system that enables fine-grained external resource sharing and elasticity. ARL-Tangram utilizes a unified action-level formulation and an elastic scheduling
algorithm to minimize action completion time (ACT) while satisfying heterogeneous resource constraints. Further, heterogeneous resource managers are tailored to efficiently support the action-level execution on resources with heterogeneous characteristics and
topologies. Evaluation on real-world agentic RL tasks demonstrates that ARL-Tangram improves average ACT by up to 4.3$\times$, speeds up the step duration of RL training by up to 1.5$\times$, and saves the external resources by up to 71.2$\%$. This system has been deployed to support the training of the
MiMo series models.
Read the full abstract ▾
All authors Bangjun Xiao, Yihao Zhao, Xiangwei Deng, Shihua Yu, Yuxing Xiang, Huaqiu Liu, Qiying Wang, Liang Zhao, Hailin Zhang, Xuanzhe Liu, Xin Jin, Fuli Luo
Machine Learning · Cross-listed from cs.CR
FraudFox: Adaptable Fraud Detection in the Real World
Matthew Butler, Yi Fan and Christos Faloutsos
The proposed method (FraudFox) provides solutions to
adversarial attacks in a resource constrained environment. We focus on questions like the following: How suspicious is `Smith', trying to buy \$500 shoes, on Monday 3am? How to merge the risk scores, from a handful of risk-assessment modules (`oracles') in an adversarial environment? More importantly, given historical data (orders, prices, and what-happened afterwards), and business goals/restrictions, which transactions, like the `Smith' transaction above, which ones should we `pass', versus send to human investigators? The business restrictions could be: `at most $x$ investigations are feasible', or `at most \$$y$ lost due to fraud'. These are the two research problems we focus on, in this work. One approach to address the first problem (`oracle-weighting'), is by using Extended Kalman Filters with dynamic importance weights, to automatically and continuously update our weights for each 'oracle'. For the second problem, we show how to derive an optimal decision surface, and how to compute the Pareto optimal set, to allow what-if questions. An important consideration is adaptation: Fraudsters will change their behavior, according to our past decisions; thus, we need to adapt accordingly. The resulting system, \method, is scalable, adaptable to changing fraudster behavior, effective, and already in production at Amazon. FraudFox augments a fraud prevention sub-system and has led to significant performance gains.
Read the full abstract ▾
Machine Learning · Cross-listed from eess.IV
Accelerating Stroke MRI with Diffusion Probabilistic Models through Large-Scale Pre-training and Target-Specific Fine-Tuning
Yamin Arefeen, Sidharth Kumar, Steven Warach et al. (+2 more)
Purpose: To develop a data-efficient strategy for accelerated MRI reconstruction with
Diffusion Probabilistic Generative Models (DPMs) that enables faster scan times in clinical stroke MRI when only limited fully-sampled data samples are available. Methods: Our simple training strategy, inspired by the foundation model paradigm, first trains a DPM on a large, diverse collection of publicly available
brain MRI data in fastMRI and then fine-tunes on a small
dataset from the target application using carefully selected learning rates and fine-tuning durations. The approach is evaluated on controlled fastMRI experiments and on clinical stroke MRI data with a blinded clinical reader study. Results: DPMs pre-trained on approximately 4000 subjects with non-FLAIR contrasts and fine-tuned on FLAIR data from only 20 target subjects achieve reconstruction performance comparable to models trained with substantially more target-domain FLAIR data across multiple acceleration factors. Experiments reveal that moderate fine-tuning with a reduced learning rate yields improved performance, while insufficient or excessive fine-tuning degrades reconstruction quality. When applied to clinical stroke MRI, a blinded reader study involving two neuroradiologists indicates that images reconstructed using the proposed approach from $2 \times$ accelerated data are non-inferior to standard-of-care in terms of image quality and structural delineation. Conclusion: Large-scale pre-training combined with targeted fine-tuning enables DPM-based MRI reconstruction in data-constrained, accelerated clinical stroke MRI. The proposed approach substantially reduces the need for large application-specific
datasets while maintaining clinically acceptable image quality, supporting the use of foundation-inspired
diffusion models for accelerated MRI in targeted applications.
Read the full abstract ▾
All authors Yamin Arefeen, Sidharth Kumar, Steven Warach, Hamidreza Saber, Jonathan Tamir
Machine Learning
Deconstructing the Failure of Ideal Noise Correction: A Three-Pillar Diagnosis
Chen Feng, Zhuo Zhi, Zhao Huang et al. (+5 more)
Statistically consistent methods based on the noise transition
matrix ($T$) offer a theoretically grounded solution to Learning with Noisy Labels (LNL), with guarantees of convergence to the optimal clean-data classifier. In practice, however, these methods are often outperformed by empirical approaches such as sample selection, and this gap is usually attributed to the difficulty of accurately estimating $T$. The common assumption is that, given a perfect $T$, noise-correction methods would recover their theoretical advantage. In this work, we put this longstanding hypothesis to a decisive test. We conduct experiments under idealized conditions, providing correction methods with a perfect, oracle transition
matrix. Even under these ideal conditions, we observe that these methods still suffer from performance collapse during training. This compellingly demonstrates that the failure is not fundamentally a $T$-estimation problem, but stems from a more deeply rooted flaw. To explain this behaviour, we provide a unified analysis that links three levels: macroscopic convergence states, microscopic optimisation dynamics, and information-theoretic limits on what can be learned from noisy labels. Together, these results give a formal account of why ideal noise correction fails and offer concrete guidance for designing more reliable methods for learning with noisy labels.
Read the full abstract ▾
All authors Chen Feng, Zhuo Zhi, Zhao Huang, Jiawei Ge, Ling Xiao, Nicu Sebe, Georgios Tzimiropoulos, Ioannis Patras
Machine Learning
DAPD: Dependency-Aware Parallel Decoding via Attention for Diffusion LLMs
Bumjun Kim, Dongjae Jeon, Moongyu Jeon et al. (+1 more)
Parallel decoding for
Diffusion LLMs (dLLMs) is difficult because each denoising step provides only
token-wise marginal distributions, while unmasking multiple
tokens simultaneously requires accounting for inter-
token dependencies. We propose Dependency-Aware Parallel Decoding (DAPD), a simple, training-free decoding method that uses self-
attention to induce a conditional dependency graph over masked
tokens. At each iteration, edges in this graph capture strong
token interactions, while non-edges indicate weak dependence. Parallel decoding is then reduced to selecting an independent set on the graph and unmasking the selected
tokens in parallel. This avoids co-updating strongly coupled
tokens without auxiliary models or retraining. Experiments on LLaDA and Dream show that DAPD improves the accuracy-steps trade-off over existing methods and enables more globally distributed parallel updates that better exploit the any-order generation capability of dLLMs. The project is available at https://ai-isl.github.io/dapd
Read the full abstract ▾
All authors Bumjun Kim, Dongjae Jeon, Moongyu Jeon, Albert No
Machine Learning
Retrieval-Enhanced Real Estate Appraisal
Simon Popelier, Matthieu X. B. Sarazin, Maximilien Bohm et al. (+4 more)
The Sales Comparison Approach (SCA) is one of the most popular when it comes to real estate appraisal. Used as a reference in real estate expertise and as one of the major types of Automatic Valuation Models (AVM), it recently gained popularity within machine learning methods. The performance of models able to use data represented as sets and graphs made it possible to adapt this methodology efficiently, yielding substantial results. SCA relies on taking past transactions (comparables) as references, selected according to their similarity with the target property's sale. In this study, we focus on the selection of these comparables for real estate appraisal. We demonstrate that the selection of comparables used in many state-of-the-art
algorithms can be significantly improved by learning a selection policy instead of imposing it. Our method relies on a hybrid vector-geographical
retrieval module capable of adapting to different
datasets and optimized jointly with an estimation module. We further show that the use of carefully selected comparables makes it possible to build models that require fewer comparables and fewer parameters with performance close to state-of-the-art models. All our evaluations are made on five
datasets which span areas in the United States, Brazil, and France.
Read the full abstract ▾
All authors Simon Popelier, Matthieu X. B. Sarazin, Maximilien Bohm, Mathieu Gierski, Hanna Mergui, Matthieu Ospici, Adrien Bernhardt
Machine Learning
Exact Federated Continual Unlearning for Ridge Heads on Frozen Foundation Models
Yijun Quan, Wentai Wu and Giovanni Montana
Foundation models are commonly deployed as frozen feature extractors with a small trainable head to adapt to private, user-generated data in
federated settings. The “right to be forgotten” requires removing the influence of specific samples or users from the trained model on demand. Existing
federated unlearning methods target general deep models and rely on approximate reconstruction or selective retraining, making exactness costly or elusive. We study this problem in a practically relevant but under-explored regime: a frozen foundation model with a ridge-regression head. The exact optimum depends on the data only through two additive sufficient statistics, which we turn into a communication protocol supporting an arbitrary stream of add and delete requests via fixed-size messages. The server maintains a head that is, in exact arithmetic, pointwise identical to centralized retraining after every request. We provide deterministic retrain-equivalence guarantees, order and partition invariance, two server-side variants, and a
Bayesian certificate of zero KL divergence. Experiments on four
benchmarks confirm the guarantees: both variants match centralized ridge retraining to within $10^{-9}$ relative Frobenius error and complete each request at orders-of-magnitude lower cost than
federated retraining baselines.
Read the full abstract ▾
Machine Learning
SCOPE: Scalable Training-Free Semantic Coreset Selection for Long-Tailed Federated Learning
Md Anwar Hossen, Nathan R. Tallent, Luanzheng Guo et al. (+1 more)
Scientific discovery increasingly relies on
federated learning over data from high-resolution instruments, and these data are strongly non-IID and long-tailed. Coreset selection could shorten training, but existing
federated coreset methods either prune with local heuristics that ignore the global class distribution, or first train a warm-up model at every client and upload class centroids of its features. This makes coreset selection a distributed bottleneck that consumes GPU time at every client, grows with the model and the number of clients, and must be repeated for every architecture. We introduce SCOPE, named for Semantic Coreset using Orthogonal Projection
Embeddings, a scalable, training-free
federated coreset framework for long-tailed data. Each client scores its samples once with a frozen, compact
vision-
language model and uploads scalar statistics per class. The server merges these into a global class profile, which lets each client remove anomalies and prune redundant samples only from classes that are locally abundant but globally common. SCOPE therefore requires no warm-up training, needs a single exchange with the server, and uploads a payload that depends only on the number of classes. On GPU nodes of an HPC cluster, with four
datasets, including scientific micrographs, and six baselines, SCOPE selects its coreset up to 7.7x faster and uploads up to 293x less data than the feature-based baseline. As the number of clients grows from 10 to 500, SCOPE's time before training grows by fewer seconds than any baseline's, and at 500 clients the baselines need 1.7 to 3.6x SCOPE's time. Trained on the coreset, the global model reaches 98% of the full-data accuracy in 18 to 42% fewer global rounds than full-data training, with 8 to 21% less client computation over 200 rounds.
Read the full abstract ▾
All authors Md Anwar Hossen, Nathan R. Tallent, Luanzheng Guo, Ali Jannesary
Machine Learning · Cross-listed from cs.CV
Rethinking VLMs for Image Forgery Detection and Localization
Shaofeng Guo, Jiequan Cui and Richang Hong
With the rapid rise of Artificial Intelligence Generated Content (AIGC), image manipulation has become increasingly accessible, posing significant challenges for image forgery detection and localization (IFDL). In this paper, we study how to fully leverage
vision-
language models (VLMs) to assist the IFDL task. In particular, we observe that priors from VLMs hardly benefit the detection and localization performance and even have negative effects due to their inherent biases toward semantic plausibility rather than authenticity. Additionally, the location masks explicitly encode the forgery concepts, which can serve as extra priors for VLMs to ease their training optimization, thus enhancing the interpretability of detection and localization results. Building on these findings, we propose a new IFDL pipeline named IFDL-VLM. To demonstrate the effectiveness of our method, we conduct experiments on 9 popular
benchmarks and assess the model performance under both in-domain and cross-
dataset generalization settings. The experimental results show that we consistently achieve new state-of-the-art performance in detection, localization, and interpretability.Code is available at: https://github.com/sha0fengGuo/IFDL-VLM.
Read the full abstract ▾