Machine Learning
Fraud Detection Through Large-Scale Graph Clustering with Heterogeneous Link Transformation
Chi Liu
Collaborative fraud, where multiple fraudulent accounts coordinate to exploit online payment systems, poses significant challenges due to the formation of complex network structures. Traditional detection methods that rely solely on high-confidence identity links suffer from limited coverage, while approaches using all available linkages often result in fragmented graphs with reduced clustering effectiveness. In this paper, we propose a novel graph-based fraud detection framework that addresses the challenge of large-scale heterogeneous graph clustering through a principled link transformation approach. Our method distinguishes between hard links (high-confidence identity relationships such as phone numbers, credit cards, and national IDs) and soft links (behavioral associations including device fingerprints, cookies, and IP addresses). We introduce a graph transformation technique that first identifies connected components via hard links, merges them into super-nodes, and then reconstructs a weighted soft-link graph amenable to efficient
embedding and clustering. The transformed graph is processed using LINE (Large-scale Information Network
Embedding) for representation learning, followed by HDBSCAN (Hierarchical Density-Based Spatial Clustering of Applications with Noise) for density-based cluster discovery. Experiments on a real-world payment platform
dataset demonstrate that our approach achieves significant graph size reduction (from 25 million to 7.7 million nodes), doubles the detection coverage compared to hard-link-only baselines, and maintains high precision across identified fraud clusters. Our framework provides a scalable and practical solution for industrial-scale fraud detection systems.
Read the full abstract ▾
Machine Learning
Efficient Personalization of Generative Models via Optimal Experimental Design
Guy Schacht, Ziyad Sheebaelhamd, Riccardo De Santi et al. (+2 more)
Preference learning from human feedback has the ability to align generative models with the needs of end-users. Human feedback is costly and time-consuming to obtain, which creates demand for data-efficient query selection methods. This work presents a novel approach that leverages optimal experimental design to ask humans the most informative preference queries, from which we can elucidate the latent reward function modeling user preferences efficiently. We formulate the problem of preference query selection as the one that maximizes the information about the underlying latent preference model. We show that this problem has a convex optimization formulation, and introduce a statistically and computationally efficient
algorithm ED-PBRL that is supported by theoretical guarantees and can efficiently construct structured queries such as images or text. We empirically present the proposed framework by personalizing a text-to-image generative model to user-specific styles, showing that it requires less preference queries compared to random query selection.
Read the full abstract ▾
All authors Guy Schacht, Ziyad Sheebaelhamd, Riccardo De Santi, Mojmír Mutný, Andreas Krause
Machine Learning
Time-series Forecast for Indoor Zone Air Temperature with Long Horizons: A Case Study with Sensor-based Data from a Smart Building
Liping Sun, Yucheng Guo, Siliang Lu et al. (+1 more)
With the press of global
climate change, extreme weather and sudden weather changes are becoming increasingly common. To maintain a comfortable indoor environment and minimize the contribution of the building to
climate change as much as possible, higher requirements are placed on the operation and control of HVAC systems, e.g., more energy-efficient and flexible to response to the rapid change of weather. This places demands on the rapid modeling and prediction of zone air temperatures of buildings. Compared to the traditional simulation-based approach such as EnergyPlus and DOE2, a hybrid approach combined physics and data-driven is more suitable. Recently, the availability of high-quality
datasets and algorithmic breakthroughs have driven a considerable amount of work in this field. However, in the niche of short- and long-term predictions, there are still some gaps in existing research. This paper aims to develop a time series forecast model to predict the zone air temperature in a building located in America on a 2-week horizon. The findings could be further improved to support intelligent control and operation of HVAC systems (i.e. demand flexibility) and could also be used as hybrid building energy modeling.
Read the full abstract ▾
All authors Liping Sun, Yucheng Guo, Siliang Lu, Zhenzhen Li
Machine Learning · Cross-listed from cs.CR
Elevating Intrusion Detection and Security Fortification in Intelligent Networks through Cutting-Edge Machine Learning Paradigms
Md Minhazul Islam Munna, Md Mahbubur Rahman, Jaroslav Frnda et al. (+2 more)
The proliferation of IoT devices and their reliance on Wi-Fi networks have introduced significant security vulnerabilities, particularly the KRACK and Kr00k attacks, which exploit weaknesses in WPA2 encryption to intercept and manipulate sensitive data. Traditional IDS using classifiers face challenges such as model overfitting, incomplete feature extraction, and high false positive rates, limiting their effectiveness in real-world deployments. To address these challenges, this study proposes a robust multiclass machine learning based intrusion detection framework. The methodology integrates advanced feature selection techniques to identify critical attributes, mitigating redundancy and enhancing detection accuracy. Two distinct ML architectures are implemented: a baseline classifier pipeline and a stacked ensemble model combining noise injection, Principal Component Analysis (PCA), and meta learning to improve generalization and reduce false positives. Evaluated on the AWID3 data set, the proposed ensemble architecture achieves superior performance, with an accuracy of 98%, precision of 98%, recall of 98%, and a false positive rate of just 2%, outperforming existing state-of-the-art methods. This work demonstrates the efficacy of combining preprocessing strategies with ensemble learning to fortify network security against sophisticated Wi-Fi attacks, offering a scalable and reliable solution for IoT environments. Future directions include real-time deployment and
adversarial resilience testing to further enhance the model's adaptability.
Read the full abstract ▾
All authors Md Minhazul Islam Munna, Md Mahbubur Rahman, Jaroslav Frnda, Muhammad Shahid Anwar, Alpamis Kutlimuratov
Machine Learning
A Surrogate-Augmented Symbolic CFD-Driven Training Framework for Accelerating Multi-objective Physical Model Development
Yuan Fang, Fabian Waschkowski, Maximilian Reissmann et al. (+3 more)
Computational Fluid Dynamics (CFD)-driven training combines machine learning (ML) with CFD solvers to develop physically consistent closure models with improved predictive accuracy. In the original framework, each ML-generated candidate model is embedded in a CFD solver and evaluated against reference data, requiring hundreds to thousands of high-fidelity simulations and resulting in prohibitive computational cost for complex flows. To overcome this limitation, we propose an extended framework that integrates surrogate modeling into symbolic CFD-driven training in real time to reduce training cost. The surrogate model learns to approximate the errors of ML-generated models based on previous CFD evaluations and is continuously refined during training. Newly generated models are first assessed using the surrogate, and only those predicted to yield small errors or high uncertainty are subsequently evaluated with full CFD simulations. Discrete expressions generated by symbolic regression are mapped into a continuous space using averaged input-symbol values as inputs to a probabilistic surrogate model. To support multi-objective model training, particularly when fixed weighting of competing quantities is challenging, the surrogate is extended to a multi-output formulation by generalizing the kernel to a
matrix form, providing one mean and variance prediction per training objective. Selection metrics based on these probabilistic outputs are used to identify an optimal training setup. The proposed surrogate-augmented CFD-driven training framework is demonstrated across a range of statistically one- and two-dimensional flows, including both single- and multi-expression model optimization. In all cases, the framework substantially reduces training cost while maintaining predictive accuracy comparable to that of the original CFD-driven approach.
Read the full abstract ▾
All authors Yuan Fang, Fabian Waschkowski, Maximilian Reissmann, Richard D. Sandberg, Takuo Oda, Koichi Tanimoto
Machine Learning · Cross-listed from cs.AI
Recontextualization Mitigates Specification Gaming without Modifying the Specification
Ariana Azarbal, Victor Gillioz, Vladimir Ivanov et al. (+6 more)
Developers often struggle to specify correct training labels and rewards. Perhaps they don't need to. We propose recontextualization, which reduces how often
language models "game" training signals, performing misbehaviors those signals mistakenly reinforce. We show recontextualization prevents models from learning to 1) prioritize evaluation metrics over chat response quality; 2) special-case code to pass incorrect tests; 3) overwrite evaluation functions rather than write correct code; and 4) become sycophantic. Our method works by generating completions from
prompts discouraging misbehavior and then recontextualizing them as though they were in response to
prompts permitting misbehavior. Recontextualization trains
language models to resist misbehavior even when instructions permit it. This mitigates the reinforcement of misbehavior from misspecified training signals, reducing specification gaming without improving the supervision signal.
Read the full abstract ▾
All authors Ariana Azarbal, Victor Gillioz, Vladimir Ivanov, Bryce Woodworth, Jacob Drori, Nevan Wichers, Aram Ebtekar, Alex Cloud, Alexander Matt Turner
Machine Learning · Cross-listed from cs.CR
The Erasure Illusion: Stress-Testing the Generalization of LLM Forgetting Evaluation
Hengrui Jia, Taoran Li, Jonas Guan et al. (+1 more)
Machine unlearning aims to remove specific data influences from trained models, a capability essential for adhering to copyright laws and ensuring AI safety. Current unlearning metrics typically measure success by monitoring the model's performance degradation on the specific unlearning
dataset ($D_u$). We argue that for Large
Language Models (LLMs), this evaluation paradigm is insufficient and potentially misleading. Many real-world uses of unlearning–motivated by copyright or safety–implicitly target not only verbatim content in $D_u$, but also behaviors influenced by the broader generalizations the model derived from it. We demonstrate that LLMs can pass standard unlearning evaluation and appear to have "forgotten" the target knowledge, while simultaneously retaining strong capabilities on content that is semantically adjacent to $D_u$. This phenomenon indicates that erasing exact sentences does not necessarily equate to removing the underlying knowledge. To address this gap, we propose Proximal Surrogate Generation (PSG), an automated stress-testing framework that generates a surrogate
dataset, $\tilde{D}_u$. This surrogate set is constructed to be semantically derived from $D_u$ yet sufficiently distinct in
embedding space. By comparing unlearning metric scores between $D_u$ and $\tilde{D}_u$, we can stress-test the reliability of the metric itself. Our extensive evaluation across three LLM families (Llama-3-8B, Qwen2.5-7B, and Zephyr-7B-$\beta$), three distinct
datasets, and seven standard metrics reveals widespread inconsistencies. We find that current metrics frequently overestimate unlearning success, failing to detect retained knowledge exposed by our stress-test
datasets.
Read the full abstract ▾
All authors Hengrui Jia, Taoran Li, Jonas Guan, Varun Chandrasekaran
Machine Learning · Cross-listed from cs.CV
CETCAM: Camera-Controllable Video Generation via Consistent and Extensible Tokenization
Zelin Zhao, Xinyu Gong, Bangya Liu et al. (+5 more)
Achieving precise camera control in
video generation remains challenging, as existing methods often rely on camera pose annotations that are difficult to scale to large and dynamic
datasets and are frequently inconsistent with depth estimation, leading to train-test discrepancies. We introduce CETCAM, a camera-controllable
video generation framework that eliminates the need for camera annotations through a consistent and extensible tokenization scheme. CETCAM leverages recent advances in geometry foundation models, such as VGGT, to estimate depth and camera parameters and converts them into unified, geometry-aware
tokens. These
tokens are seamlessly integrated into a pretrained
video diffusion backbone via lightweight context blocks. Trained in two progressive stages, CETCAM first learns robust camera controllability from diverse raw
video data and then refines fine-grained visual quality using curated high-fidelity
datasets. Extensive experiments across multiple
benchmarks demonstrate state-of-the-art
geometric consistency, temporal stability, and visual realism. Moreover, CETCAM exhibits strong adaptability to additional control modalities, including inpainting and layout control, highlighting its flexibility beyond camera control. The project page is available at https://sjtuytc.github.io/CETCam_project_page.github.io/.
Read the full abstract ▾
All authors Zelin Zhao, Xinyu Gong, Bangya Liu, Ziyang Song, Jun Zhang, Suhui Wu, Yongxin Chen, Hao Zhang
Machine Learning
Optimizer Dynamics at the Edge of Stability with Differential Privacy
Ayana Hussain and Ricky Fang
Deep learning models can reveal sensitive information about individual training examples, and while differential
privacy (DP) provides guarantees restricting such leakage, it also alters optimization dynamics in poorly understood ways. We study the training dynamics of neural networks under DP by comparing Gradient Descent (GD), and Adam to their
privacy-preserving variants. Prior work shows that these optimizers exhibit distinct stability dynamics: full-batch methods train at the Edge of Stability (EoS), while mini-batch and adaptive methods exhibit analogous edge-of-stability behavior. At these regimes, the training loss and the sharpness–the maximum
eigenvalue of the training loss Hessian–exhibit certain characteristic behavior. In DP training, per-example gradient clipping and Gaussian noise modify the update rule, and it is unclear whether these stability patterns persist. We analyze how clipping and noise change sharpness and loss
evolution and show that while DP generally reduces the sharpness and can prevent optimizers from fully reaching the classical stability thresholds, patterns from EoS and analogous adaptive methods stability regimes persist, with the largest learning rates and largest
privacy budgets approaching, and sometimes exceeding, these thresholds. These findings highlight the unpredictability introduced by DP in neural network optimization.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CR
Do You Really Need a GPU to Guard Your LLM? CPU-Class Classifiers and Multi-Stage Pipelines for Safety Enforcement at Scale
Vasudev Majhi, Dhruv Gupta, Advait Singh et al. (+2 more)
Safety classifiers that screen LLM inputs for jailbreak attempts have become standard deployment components, yet almost all production systems rely on GPU-based models: fine-tuned
transformers and LLM-as-a-judge pipelines. These approaches impose significant per-query latency and infrastructure cost. Very little research has asked whether CPU-based classifiers, such as support vector machines and gradient-boosted trees trained on TF-IDF features, can match their accuracy across the conditions that production deployments encounter. We evaluate five CPU classifier families, Mamba-130M as an SSM-based GPU classifier, and
transformer-based GPU models (DeBERTa-v3 and Gemma-2B with LoRA) across nine jailbreak sources and three regimes: in-distribution (D1), out-of-distribution (D2), and adversarially obfuscated (D3). On D1, the best CPU classifier matches the best
transformer GPU model at roughly one-fifth the deployment cost. On D2, CPU classifiers fail via confident miscalibration, producing high-confidence false negatives that bypass escalation entirely. On D3, CPU classifiers outperform
transformer GPU models by more than 26 percentage points in F1. Based on these complementary failure modes, we design GuardChain, a three-stage safety pipeline (Regex -> CPU -> GPU) that routes each
prompt to the cheapest stage capable of a confident decision. The CPU stage alone resolves 80% of in-distribution
prompts at near-peak accuracy, and the GPU stage recovers the out-of-distribution failures. For practitioners deploying LLM safety at scale, this work provides evidence that GPU-class infrastructure is unnecessary for the majority of traffic.
Read the full abstract ▾
All authors Vasudev Majhi, Dhruv Gupta, Advait Singh, Matthew Barker, Dhruv Kumar
Machine Learning
The 6th International Verification of Neural Networks Competition (VNN-COMP 2025): Summary and Results
Konstantin Kaulen, Tobias Ladner, Stanley Bak et al. (+8 more)
This report summarizes the 6th International Verification of Neural Networks Competition (VNN-COMP 2025), held as a part of the 8th International Symposium on AI Verification (SAIV), that was collocated with the 37th International Conference on Computer-Aided Verification (CAV). VNN-COMP is held annually to facilitate the fair and objective comparison of state-of-the-art neural network verification tools, encourage the standardization of tool interfaces, and bring together the neural network verification community. To this end, standardized formats for networks (ONNX) and specification (VNN-LIB) were defined, tools were evaluated on equal-cost hardware (using an automatic evaluation pipeline based on AWS instances), and tool parameters were chosen by the participants before the final test sets were made public. In the 2025 iteration, 8 teams participated on a diverse set of 16 regular and 9 extended
benchmarks. This report summarizes the rules,
benchmarks, participating tools, results, and lessons learned from this iteration of this competition.
Read the full abstract ▾
All authors Konstantin Kaulen, Tobias Ladner, Stanley Bak, Christopher Brix, Hai Duong, Thomas Flinkow, Taylor T. Johnson, Lukas Koller, Edoardo Manino, ThanhVu H Nguyen, Haoze Wu
Machine Learning · Cross-listed from cs.CL
Context-Aware Initialization for Reducing Generative Path Length in Diffusion Language Models
Tongyuan Miao, Gary Huang, Kai Jun Han et al. (+1 more)
Diffusion Large
Language Models (DLLMs) enable fully parallel
token decoding but often remain impractical at inference time due to the many denoising iterations required to refine an information-free, fully masked initialization into coherent text. Most existing acceleration methods focus on traversing this generative trajectory more efficiently via improved solvers or sampling strategies. We advance a complementary perspective: shorten the trajectory itself by starting closer to the target distribution through context-aware initialization. We propose a training-free interface that injects
prompt-conditioned priors from a lightweight auxiliary model into the
diffusion initialization, and instantiate it with two mechanisms: discrete
token injection and representation-level
embedding interpolation. Because injected priors can be imperfect and unmask-only decoding can over-commit early, we also introduce a simple confidence-based remasking mechanism as a form of prior skepticism. Preliminary evidence on GSM8K suggests that context-aware initialization can substantially reduce denoising iterations (about 35% fewer function evaluations in our setting), while also exposing a key open challenge: naive warm-starting can degrade final accuracy relative to strong
diffusion baselines. We use these findings to motivate a research agenda around calibration, revision mechanisms, and representation
alignment for reliable warm-started
diffusion decoding.
Read the full abstract ▾
All authors Tongyuan Miao, Gary Huang, Kai Jun Han, Annie Jiang
Machine Learning · Cross-listed from cs.AI
ORPR: An OR-Guided Pretrain-then-Reinforce Learning Model for Inventory Management
Lingjie Zhao, Xue Yu, Yongzhi Qi et al. (+6 more)
As the pursuit of synergy between Artificial Intelligence (AI) and Operations Research (OR) gains momentum in handling complex inventory systems, a critical challenge persists: how to effectively reconcile AI's adaptive perception with OR's structural rigor. To bridge this gap, we propose a novel OR-Guided "Pretrain-then-Reinforce" framework. To provide structured guidance, we propose a simulation-augmented OR model that generates high-quality reference decisions, implicitly capturing complex business constraints and managerial preferences. Leveraging these OR-derived decisions as foundational training labels, we design a domain-informed deep learning foundation model to establish foundational decision-making capabilities, followed by a reinforcement learning (RL) fine-tuning stage. Uniquely, we position RL as a deep
alignment mechanism that enables the AI
agent to internalize the optimality principles of OR, while simultaneously leveraging exploration for general policy refinement and allowing expert guidance for scenario-specific adaptation (e.g., promotional events). Validated through extensive numerical experiments and a field deployment at JD.com augmented by a Difference-in-Differences (DiD) analysis, our model significantly outperforms incumbent industrial practices, delivering real-world gains of a 5.27-day reduction in turnover and a 2.29% increase in in-stock rates, alongside a 29.95% decrease in holding costs. Contrary to the prevailing trend of brute-force model scaling, our study demonstrates that a lightweight, domain-informed model can deliver state-of-the-art performance and robust transferability when guided by structured OR logic. This approach offers a scalable and cost-effective paradigm for intelligent supply chain management, highlighting the value of deeply aligning AI with OR.
Read the full abstract ▾
All authors Lingjie Zhao, Xue Yu, Yongzhi Qi, Hao Hu, Jianshen Zhang, Yingzheng Ma, Shuyu Han, Wei Qi, Zuo-Jun Max Shen
Machine Learning
R-GenIMA: Integrating Neuroimaging and Genetics with Interpretable Multimodal AI for Alzheimer's Disease Progression
Kun Zhao, Siyuan Dai, Yingying Zhang et al. (+9 more)
Early detection of Alzheimer's disease (AD) requires models capable of integrating macro-scale neuroanatomical alterations with micro-scale genetic susceptibility, yet existing
multimodal approaches struggle to align these heterogeneous signals. We introduce R-GenIMA, an interpretable
multimodal large
language model that couples a novel ROI-wise
vision transformer with genetic prompting to jointly model structural MRI and single nucleotide polymorphisms (SNPs) variations. By representing each anatomically parcellated
brain region as a visual
token and encoding SNP profiles as structured text, the framework enables cross-modal
attention that links regional atrophy patterns to underlying genetic factors. Applied to the ADNI cohort, R-GenIMA achieves state-of-the-art performance in four-way classification across normal cognition (NC), subjective
memory concerns (SMC), mild cognitive impairment (MCI), and AD. Beyond predictive accuracy, the model yields biologically meaningful explanations by identifying stage-specific
brain regions and gene signatures, as well as coherent ROI-Gene association patterns across the disease continuum.
Attention-based attribution revealed genes consistently enriched for established GWAS-supported AD risk loci, including APOE, BIN1, CLU, and RBFOX1. Stage-resolved neuroanatomical signatures identified shared vulnerability hubs across disease stages alongside stage-specific patterns: striatal involvement in subjective decline, frontotemporal engagement during prodromal impairment, and consolidated
multimodal network disruption in AD. These results demonstrate that interpretable
multimodal AI can synthesize imaging and genetics to reveal mechanistic insights, providing a foundation for clinically deployable tools that enable earlier risk stratification and inform precision therapeutic strategies in Alzheimer's disease.
Read the full abstract ▾
All authors Kun Zhao, Siyuan Dai, Yingying Zhang, Guodong Liu, Pengfei Gu, Chenghua Lin, Paul M. Thompson, Alex Leow, Heng Huang, Lifang He, Liang Zhan, Haoteng Tang
Machine Learning
OPBO: Order-Preserving Bayesian Optimization
Wei Peng, Jianchen Hu, Kang Liu et al. (+1 more)
Bayesian optimization is an effective method for solving expensive black-box optimization problems. Most existing methods use Gaussian processes (GP) as the surrogate model for approximating the black-box objective function, it is well-known that it can fail in high-dimensional space (e.g., dimension over 500). We argue that the reliance of GP on precise numerical fitting is fundamentally ill-suited in high-dimensional space, where it leads to prohibitive computational complexity. In order to address this, we propose a simple order-preserving
Bayesian optimization (OPBO) method, where the surrogate model preserves the order, instead of the value, of the black-box objective function. Then we can use a simple but effective OP neural network (NN) to replace GP as the surrogate model. Moreover, instead of searching for the best solution from the acquisition model, we select good-enough solutions in the ordinal set to reduce computational cost. The experimental results show that for high-dimensional (over 500) black-box optimization problems, the proposed OPBO significantly outperforms traditional BO methods based on regression NN and GP. The source code is available at https://github.com/pengwei222/OPBO.
Read the full abstract ▾
All authors Wei Peng, Jianchen Hu, Kang Liu, Qiaozhu Zhai
Machine Learning
Outlier detection in mixed-attribute data: a semi-supervised approach with fuzzy approximations and relative entropy
Baiyang Chen, Zhong Yuan, Zheng Liu et al. (+4 more)
Outlier detection is a critical task in data mining, aimed at identifying objects that significantly deviate from the norm. Semi-supervised methods improve detection performance by leveraging partially labeled data but typically overlook the uncertainty and heterogeneity of real-world mixed-attribute data. This paper introduces a semi-supervised outlier detection method, namely fuzzy rough sets-based outlier detection (FROD), to effectively handle these challenges. Specifically, we first utilize a small subset of labeled data to construct fuzzy decision systems, through which we introduce the attribute classification accuracy based on fuzzy approximations to evaluate the contribution of attribute sets in outlier detection. Unlabeled data is then used to compute fuzzy relative
entropy, which provides a characterization of outliers from the perspective of uncertainty. Finally, we develop the detection
algorithm by combining attribute classification accuracy with fuzzy relative
entropy. Experimental results on 16 public
datasets show that FROD is comparable with or better than leading detection
algorithms. All
datasets and source codes are accessible at https://github.com/ChenBaiyang/FROD. This manuscript is the accepted author version of a paper published by Elsevier. The final published version is available at https://doi.org/10.1016/j.ijar.2025.109373
Read the full abstract ▾
All authors Baiyang Chen, Zhong Yuan, Zheng Liu, Dezhong Peng, Yongxiang Li, Chang Liu, Guiduo Duan
Machine Learning
Consistency-guided semi-supervised outlier detection in heterogeneous data using fuzzy rough sets
Baiyang Chen, Zhong Yuan, Dezhong Peng et al. (+2 more)
Outlier detection aims to find samples that behave differently from the majority of the data. Semi-supervised detection methods can utilize the supervision of partial labels, thus reducing false positive rates. However, most of the current semi-supervised methods focus on numerical data and neglect the heterogeneity of data information. In this paper, we propose a consistency-guided outlier detection
algorithm (COD) for heterogeneous data with the fuzzy rough set theory in a semi-supervised manner. First, a few labeled outliers are leveraged to construct label-informed fuzzy similarity relations. Next, the consistency of the fuzzy decision system is introduced to evaluate attributes' contributions to knowledge classification. Subsequently, we define the outlier factor based on the fuzzy similarity class and predict outliers by integrating the classification consistency and the outlier factor. The proposed
algorithm is extensively evaluated on 15 freshly proposed
datasets. Experimental results demonstrate that COD is better than or comparable with the leading outlier detectors. This manuscript is the accepted author version of a paper published by Elsevier. The final published version is available at https://doi.org/10.1016/j.asoc.2024.112070
Read the full abstract ▾
All authors Baiyang Chen, Zhong Yuan, Dezhong Peng, Xiaoliang Chen, Hongmei Chen
Machine Learning · Cross-listed from stat.ML
On Conditional Stochastic Interpolation for Generative Nonlinear Sufficient Dimension Reduction
Shuntuo Xu, Zhou Yu and Jian Huang
Identifying low-dimensional sufficient structures in nonlinear sufficient dimension reduction (SDR) has long been a fundamental yet challenging problem. Most existing methods lack theoretical guarantees of exhaustiveness in identifying lower dimensional structures, either at the population level or at the sample level. We tackle this issue by proposing a new method, generative sufficient dimension reduction (GenSDR), which leverages modern generative models. We show that GenSDR is able to fully recover the information contained in the central $\sigma$-field at both the population and sample levels. In particular, at the sample level, we establish a consistency property for the GenSDR
estimator from the perspective of conditional distributions, capitalizing on the distributional learning capabilities of deep generative models. Moreover, by incorporating an ensemble technique, we extend GenSDR to accommodate scenarios with non-Euclidean responses, thereby substantially broadening its applicability. Extensive numerical results demonstrate the outstanding empirical performance of GenSDR and highlight its strong potential for addressing a wide range of complex, real-world tasks.
Read the full abstract ▾
Machine Learning
Lag Operator SSMs: A Geometric Framework for Structured State Space Modeling
Sutashu Tomonaga, Kenji Doya and Noboru Murata
Structured State Space Models (SSMs), which are at the heart of the recently popular Mamba architecture, are powerful tools for sequence modeling. However, their theoretical foundation relies on a complex, multistage process of continuous-time modeling and subsequent discretization, which can obscure intuition. We introduce a direct, first-principles framework for constructing discrete-time SSMs that is both flexible and modular. Our approach is based on a novel lag operator, which geometrically derives the discrete-time recurrence by measuring how the system's basis functions undergo what we call a domain expansion from one timestep to the next. The resulting state
matrices are computed via a single inner product involving this operator, enabling a modular design space for creating novel SSMs by flexibly combining different basis functions and time-warping schemes. To validate our framework, we demonstrate that a specific instance exactly recovers the recurrence of the influential HiPPO model. Numerical simulations confirm our derivation, providing new theoretical tools for designing flexible and robust sequence models.
Read the full abstract ▾
Machine Learning
Online Robust Reinforcement Learning with General Function Approximation
Debamita Ghosh, George K. Atia and Yue Wang
In many real-world settings, reinforcement learning systems suffer performance degradation when the environment encountered at deployment differs from that observed during training. Distributionally robust reinforcement learning (DR-RL) mitigates this issue by seeking policies that maximize performance under the most adverse transition dynamics within a prescribed uncertainty set. Most existing DR-RL approaches, however, rely on strong data availability assumptions, such as access to a generative model or large offline
datasets, and are largely restricted to tabular settings. In this work, we propose a fully online DR-RL
algorithm with general function approximation that learns robust policies solely through interaction, without requiring prior knowledge or pre-collected data. Our approach is based on a dual-driven fitted robust Bellman procedure that simultaneously estimates the value function and the corresponding worst-case backup operator. We establish regret guarantees for online DR-RL characterized by an intrinsic complexity notion, the robust Bellman-Eluder dimension, covering a broad class of phi-divergence uncertainty sets. The resulting regret bounds are sublinear, do not scale with the size of the state or action spaces, and specialize to tight rates in structured problem classes, demonstrating the practicality and scalability of our framework.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
Training Multimodal Large Reasoning Models Needs Better Thoughts: A Three-Stage Framework for Long Chain-of-Thought Synthesis and Selection
Yizhi Wang, Linan Yue and Min-Ling Zhang
Large
Reasoning Models (LRMs) have demonstrated remarkable performance on complex
reasoning tasks through long Chain-of-Thought (CoT)
reasoning. Extending these successes to
multimodal reasoning remains challenging due to the increased complexity of integrating diverse input modalities and the scarcity of high-quality long CoT training data. Existing
multimodal datasets and CoT synthesis methods still suffer from limited
reasoning depth, modality conversion errors, and rigid generation pipelines, hindering model performance and stability. To this end, in this paper, we propose SynSelect, a novel three-stage Synthesis-Selection framework for generating high-quality long CoT data tailored to
multimodal reasoning tasks. Specifically, SynSelect first leverages multiple heterogeneous
multimodal LRMs to produce diverse candidate CoTs, and then applies both instance and batch level selection to filter high-quality CoTs that can effectively enhance the model's
reasoning capabilities. Extensive experiments on multiple
multimodal benchmarks demonstrate that models supervised fine-tuned on SynSelect-generated data significantly outperform baselines and achieve further improvements after reinforcement learning post-training. Our results validate SynSelect as an effective approach for advancing
multimodal LRMs
reasoning capabilities.
Read the full abstract ▾
Machine Learning
Benchmarking Attribute Discrimination in Infant-Scale Vision-Language Models
Patrick Batsell, Satoshi Tsutsui and Bihan Wen
Infants learn not only object categories but also fine-grained visual attributes such as color, size, and texture from limited experience. Prior infant-scale
vision–
language models have mainly been evaluated on object recognition, leaving open whether they support within-class attribute discrimination. We introduce a controlled
benchmark that varies color, size, and texture across 67 everyday object classes using synthetic rendering to decouple attribute values from object identity. We evaluate infant-trained models (CVCL and an infant-trained DINO baseline) against web-scale and ImageNet models (CLIP, SigLIP, ResNeXt) under two complementary settings: an image-only prototype test and a text–
vision test with attribute–object
prompts. We find a dissociation between visual and linguistic attribute information: infant-trained models form strong visual representations for size and discriminate texture comparably to other models, but perform poorly on visual color discrimination, and in the text–
vision setting they struggle to ground color and show only modest size grounding. In contrast, web-trained
vision–
language models strongly ground color from text while exhibiting weaker visual size discrimination.
Read the full abstract ▾
Machine Learning
Learning Hierarchical Procedural Memory for LLM Agents through Bayesian Selection and Contrastive Refinement
Saman Forouzandeh, Wei Peng, Parham Moradi et al. (+2 more)
We present MACLA, a framework that decouples
reasoning from learning by maintaining a frozen large
language model while performing all adaptation in an external hierarchical procedural
memory. MACLA extracts reusable procedures from trajectories, tracks reliability via
Bayesian posteriors, selects actions through expected-utility scoring, and refines procedures by contrasting successes and failures. Across four
benchmarks (ALFWorld, WebShop, TravelPlanner, InterCodeSQL), MACLA achieves 78.1 percent average performance, outperforming all baselines. On ALFWorld unseen tasks, MACLA reaches 90.3 percent with 3.1 percent positive generalization. The system constructs
memory in 56 seconds, 2800 times faster than the state-of-the-art LLM parameter-training baseline, compressing 2851 trajectories into 187 procedures. Experimental results demonstrate that structured external
memory with
Bayesian selection and contrastive refinement enables sample-efficient, interpretable, and continually improving
agents without LLM parameter updates.
Read the full abstract ▾
All authors Saman Forouzandeh, Wei Peng, Parham Moradi, Xinghuo Yu, Mahdi Jalili
Machine Learning
When Less is More: 8-bit Quantization Improves Continual Learning in Large Language Models
Michael S. Zhang, Rishi A. Ruia, Arnav Kewalram et al. (+3 more)
Catastrophic forgetting poses a fundamental challenge in continual learning, particularly when models are quantized for deployment efficiency. We systematically investigate the interplay between quantization precision (FP16, INT8, INT4) and replay buffer strategies in large
language models, revealing unexpected dynamics. While FP16 achieves superior initial task performance (74.44% on NLU), we observe a striking inversion on subsequent tasks: quantized models outperform FP16 by 8-15% on final task forward accuracy, with INT4 achieving nearly double FP16's performance on Code generation (40% vs 20%). Critically, even minimal replay buffers (0.1%) dramatically improve retention - increasing NLU retention after Math training from 45% to 65% across all precision levels - with INT8 consistently achieving the optimal balance between learning plasticity and knowledge retention. We hypothesize that quantization-induced noise acts as implicit regularization, preventing the overfitting to new task gradients that plagues high-precision models. These findings challenge the conventional wisdom that higher precision is always preferable, suggesting instead that INT8 quantization offers both computational efficiency and superior continual learning dynamics. Our results provide practical guidelines for deploying compressed models in continual learning scenarios: small replay buffers (1-2%) suffice for NLU tasks, while Math and Code benefit from moderate buffers (5-10%), with quantized models requiring less replay than FP16 to achieve comparable retention. Code is available at https://github.com/Festyve/LessIsMore.
Read the full abstract ▾
All authors Michael S. Zhang, Rishi A. Ruia, Arnav Kewalram, Saathvik Dharmapuram, Utkarsh Sharma, Kevin Zhu
Machine Learning
DPSR: Differentially Private Sparse Reconstruction via Multi-Stage Denoising for Recommender Systems
Sarwan Ali
Differential
privacy (DP) has emerged as the gold standard for protecting user data in recommender systems, but existing
privacy-preserving mechanisms face a fundamental challenge: the
privacy-utility tradeoff inevitably degrades recommendation quality as
privacy budgets tighten. We introduce DPSR (Differentially Private Sparse Reconstruction), a novel three-stage denoising framework that fundamentally addresses this limitation by exploiting the inherent structure of rating
matrices – sparsity, low-rank properties, and collaborative patterns. DPSR consists of three synergistic stages: (1) information-theoretic noise calibration that adaptively reduces noise for high-information ratings, (2) collaborative filtering-based denoising that leverages item-item similarities to remove
privacy noise, and (3) low-rank
matrix completion that exploits latent structure for signal recovery. Critically, all denoising operations occur after noise injection, preserving differential
privacy through the post-processing immunity
theorem while removing both
privacy-induced and inherent data noise. Through extensive experiments on synthetic
datasets with controlled ground truth, we demonstrate that DPSR achieves 5.57% to 9.23% RMSE improvement over state-of-the-art Laplace and Gaussian mechanisms across
privacy budgets ranging from $\varepsilon=0.1$ to $\varepsilon=10.0$ (all improvements statistically significant with $p < 0.05$, most $p < 0.001$). Remarkably, at $\varepsilon=1.0$, DPSR achieves RMSE of 0.9823, outperforming even the non-private baseline (1.0983), demonstrating that our denoising pipeline acts as an effective regularizer that removes data noise in addition to
privacy noise.
Read the full abstract ▾
Machine Learning
The Ensemble Schrödinger Bridge filter for Nonlinear Data Assimilation
Hui Sun
This work introduces a novel nonlinear optimal filtering method, termed the Ensemble Schrödinger Bridge nonlinear filter. The proposed filter combines the standard prediction step with a
diffusion-generative-modeling-based analysis step, thereby completing one full filtering update. The resulting approach introduces no structural model error, and is derivative-free, training-free, and highly parallelizable. Numerical experiments demonstrate that the proposed
algorithm performs effectively for highly nonlinear dynamics and nonlinear observation processes, including chaotic systems with dimension up to 40 and beyond. The results also show that the method outperforms classical approaches such as the ensemble Kalman filter and particle filter across a range of tests with varying degrees of nonlinearity. Future work will focus on extending the proposed method to practical meteorological applications and developing a rigorous convergence theory.
Read the full abstract ▾