Cryptography and Security · Cross-listed from cs.LG
Augmenting Parameter-Efficient Pre-trained Language Models with Large Language Models
Saurabh Anand, Shubham Malaviya, Manish Shukla et al. (+1 more)
Training AI models in cybersecurity with help of vast
datasets offers significant opportunities to mimic real-world behaviors effectively. However, challenges like data drift and scarcity of labelled data lead to frequent updates of models and the risk of overfitting. To address these challenges, we used parameter-efficient fine-tuning techniques for pre-trained
language models wherein we combine compacters with various layer freezing strategies. To enhance the capabilities of these pre-trained
language models, in this work we introduce two strategies that use large
language models. In the first strategy, we utilize large
language models as data-labelling tools wherein they generate labels for unlabeled data. In the second strategy, large
language modes are utilized as fallback mechanisms for predictions having low confidence scores. We perform comprehensive experimental analysis on the proposed strategies on different downstream tasks specific to cybersecurity domain. We empirically demonstrate that by combining parameter-efficient pre-trained models with large
language models, we can improve the reliability and robustness of models, making them more suitable for real-world cybersecurity applications.
Read the full abstract ▾
All authors Saurabh Anand, Shubham Malaviya, Manish Shukla, Sachin Lodha
Cryptography and Security · Cross-listed from cs.CL
In Vino Veritas and Vulnerabilities: Examining LLM Safety via Drunk Language Inducement
Anudeex Shetty, Aditya Joshi and Salil S. Kanhere
Humans are susceptible to undesirable behaviours and
privacy leaks under the influence of alcohol. This paper investigates drunk
language, i.e., text written under the influence of alcohol, as a driver for safety failures in large
language models (LLMs). We investigate three mechanisms for inducing drunk
language in LLMs: persona-based prompting, causal fine-tuning, and reinforcement-based post-training. When evaluated on 5 LLMs, we observe a higher susceptibility to jailbreaking on JailbreakBench (even in the presence of defences) and
privacy leaks on ConfAIde, where both
benchmarks are in English, as compared to the base LLMs as well as previously reported approaches. Via a robust combination of manual evaluation and LLM-based evaluators and analysis of error categories, our findings highlight a correspondence between human-intoxicated behaviour, and anthropomorphism in LLMs induced with drunk
language. The simplicity and efficiency of our drunk
language inducement approaches position them as potential counters for LLM safety tuning, highlighting significant risks to LLM safety.
Read the full abstract ▾
Cryptography and Security
Tracing the Data Trail: A Survey of Data Provenance, Transparency and Traceability in LLMs
Richard Hohensinner, Belgin Mutlu, Inti Gabriel Mendoza Estrada et al. (+3 more)
Large
language models (LLMs) are deployed at scale, yet their training data life cycle remains opaque. This survey synthesizes research from the past ten years on three tightly coupled axes: (1) data provenance, (2) transparency, and (3) traceability, and three supporting pillars: (4) bias & uncertainty, (5) data
privacy, and (6) tools and techniques that operationalize them. A central contribution is a proposed taxonomy defining the field's domains and listing corresponding artifacts. Through analysis of 95 publications, this work identifies key methodologies concerning data generation, watermarking, bias measurement, data curation, data
privacy, and the inherent trade-off between transparency and opacity.
Read the full abstract ▾
All authors Richard Hohensinner, Belgin Mutlu, Inti Gabriel Mendoza Estrada, Matej Vukovic, Simone Kopeinik, Roman Kern
Cryptography and Security
CORVUS: Red-Teaming Hallucination Detectors via Internal Signal Camouflage in Large Language Models
Nay Myat Min, Long H. Pham, Hongyu Zhang et al. (+1 more)
Single-pass
hallucination detectors rely on internal telemetry (e.g., uncertainty, hidden-state geometry, and
attention) of large
language models, implicitly assuming
hallucinations leave separable traces in these signals. We study a white-box, model-side adversary that fine-tunes lightweight LoRA adapters on the model while keeping the detector fixed, and introduce CORVUS, an efficient red-teaming procedure that learns to camouflage detector-visible telemetry under teacher forcing, including an
embedding-space FGSM
attention stress test. Trained on 1,000 out-of-distribution Alpaca instructions (<0.5% trainable parameters), CORVUS transfers to FAVA-Annotation across Llama-2, Vicuna, Llama-3, and Qwen2.5, and degrades both training-free detectors (e.g., LLM-Check) and probe-based detectors (e.g., SEP, ICR-probe), motivating adversary-aware
auditing that incorporates external grounding or cross-model evidence.
Read the full abstract ▾
All authors Nay Myat Min, Long H. Pham, Hongyu Zhang, Jun Sun
Cryptography and Security
Techniques of Modern Attacks
Alexander Shim
The techniques used in modern attacks have become an important factor for investigation. As we advance further into the digital age, cyber attackers are employing increasingly sophisticated and highly threatening methods. These attacks target not only organizations and governments but also extend to private and corporate sectors. Modern attack techniques, such as lateral movement and ransomware, are designed to infiltrate networks and steal sensitive data. Among these techniques, Advanced Persistent Threats (APTs) represent a complex method of attack aimed at specific targets to steal high-value sensitive information or damage the infrastructure of the targeted organization. In this paper, I will investigate Advanced Persistent Threats (APTs) as a modern attack technique, focusing on both the attack life cycle and cutting-edge detection and defense strategies proposed in recent academic research. I will analyze four representative papers to understand the
evolution of APT detection mechanisms, including machine learning-driven behavioral analysis and network-level collaborative defense models. Through this comparative analysis, I aim to highlight the strengths and limitations of each approach and propose more adaptive APT mitigation strategies. The study seeks to analyze the key characteristics of APTs and provide a comprehensive high-level understanding of APTs along with potential solutions to the threats they pose.
Read the full abstract ▾
Cryptography and Security
A Scientific Data Integrity system based on Blockchain
Gian Sebastian Mier Bello, Alexander Martinez Mendez, Carlos J. Barrios H. et al. (+2 more)
In most High Performance Computing (HPC) projects nowadays, there is a lot of data obtained from different sources, depending on the project's objectives. Some of that data is very huge in terms of size, so copying such data sometimes is an unrealistic goal. On the other hand, science requires data used for different purposes to remain unaltered, so different groups of researchers can reproduce results, discuss theories, and validate each other. In this paper, we present a novel approach to help research groups to validate data integrity on such distributed repositories using Blockchain. Originally developed for cryptographic currencies, Blockchain has demonstrated a versatile range of uses. Our proposal ensures 1) secure access to data management, 2) easy validation of data integrity, and 3) an easy way to add new records to the
dataset with the same robust integrity policy. A prototype was developed and tested using a subset of a public
dataset from a real scientific collaboration, the Latin American
Giant Observatory (LAGO) Project.
Read the full abstract ▾
All authors Gian Sebastian Mier Bello, Alexander Martinez Mendez, Carlos J. Barrios H., Robinson Rivas, Luis A. Núñez
Cryptography and Security
Quantum Encryption Resilience Score (QERS) for MQTT, HTTP, and HTTPS under Post-Quantum Cryptography in Computer, IoT, and IIoT Systems
Jonatan Rassekhnia
Post-
quantum cryptography (PQC) introduces significant computational and communication overhead, which poses challenges for resource-constrained computer systems, Internet of Things (IoT), and Industrial IoT (IIoT) devices. This paper presents an experimental evaluation of the
Quantum Encryption Resilience Score (QERS) applied to MQTT, HTTP, and HTTPS communication protocols operating under PQC. Using an ESP32-C6 client and an ARM-based Raspberry Pi CM4 server, latency, CPU utilization, RSSI, energy consumption, key size, and TLS handshake overhead are measured under realistic operating conditions. QERS integrates these heterogeneous metrics into normalized Basic, Tuned, and Fusion scores, enabling systematic comparison of protocol efficiency and security resilience. Experimental results show that MQTT provides the highest efficiency under PQC constraints, while HTTPS achieves the highest security-weighted resilience at the cost of increased latency and resource consumption. The proposed framework supports informed protocol selection and migration planning for PQC-enabled IoT and IIoT deployments.
Read the full abstract ▾
Cryptography and Security
QERS: Quantum Encryption Resilience Score for Post-Quantum Cryptography in Computer, IoT, and IIoT Systems
Jonatan Rassekhnia
Post-
quantum cryptography (PQC) is becoming essential for securing Internet of Things (IoT) and Industrial IoT (IIoT) systems against
quantum-enabled adversaries. However, existing evaluation approaches primarily focus on isolated performance metrics, offering limited support for holistic security and deployment decisions. This paper introduces QERS (
Quantum Encryption Resilience Score), a universal measurement framework that integrates cryptographic performance, system constraints, and multi-criteria decision analysis to assess PQC readiness in computer, IoT, and IIoT environments. QERS combines normalized metrics, weighted aggregation, and machine learning-assisted analysis to produce interpretable resilience scores across heterogeneous devices and communication protocols. Experimental results demonstrate how the framework enables comparative evaluation of post-
quantum schemes under realistic resource constraints, supporting informed security design and migration planning. This work is presented as a preprint, with extended statistical validation planned as part of ongoing graduate research.
Read the full abstract ▾
Cryptography and Security · Cross-listed from cs.CL
Sockpuppetting: Jailbreaking LLMs by Combining Prefilling with Optimization
Asen Dotsinski and Panagiotis Eustratiadis
Prefill attacks are an effective and low-cost jailbreaking method, as they directly insert an acceptance sequence (e.g., "Sure, here is...") at the start of an LLM's output and lead the model to continue the response. We make two contributions to this prior work. First, we show that an unsophisticated adversary can improve the well-known prefill attacks by ensembling a small number of prefill variants. Running three easy-to-generate prefills yields a combined attack success rate (ASR) of 22%, 90%, and 99% on Gemma-7B, Llama-3.1-8B, and Qwen3-8B respectively, an up to 38 percentage point improvement over the standard "Sure, here's..." prefill and up to 82 percentage points over our reproduction of GCG (Zou et al., 2023). Second, we introduce "sockpuppetting", a hybrid attack that optimizes an
adversarial suffix placed inside the "assistant" message block of the chat template, rather than within the user
prompt. The rolling variant of this attack, RollingSockpuppetGCG, increases
prompt-agnostic ASR by up to 64 percentage points over our universal GCG baseline on Llama-3.1-8B. An ablation indicates that part of this gain stems from the choice of acceptance sequence rather than suffix placement alone (Appendix F). Both findings highlight the need for defences against output-prefix injection in open-weight models. Code: https://gitlab.com/asendotsinski/sockpuppetting
Read the full abstract ▾
Cryptography and Security
Function Recovery Attacks in Gate-Hiding Garbled Circuits using SAT Solving
Chao Yin, Zunchen Huang, Chenglu Jin et al. (+2 more)
Semi-Private Function Evaluation (SPFE) enables joint computation while protecting both input data and the function itself. A practical instantiation is gate-hiding garbled
circuits, which conceal gate functionalities while revealing
circuit topology. Existing security definitions intentionally exclude leakage through
topology, leaving its concrete impact on function
privacy largely unexplored. We present a SAT-based function-recovery attack that reconstructs hidden gate operations from a
circuit's public
topology under two attacker knowledge models. Our approach combines
topology-preserving simplification
theorems with a decomposition of the recovery task into smaller SAT queries, thereby reducing the candidate gate-type assignment space and improving recovery performance. We evaluate the attack on ISCAS
benchmarks, representative secure computation
circuits, and fault-tolerant sensor fusion
circuits under a 24-hour recovery budget. Compared to a baseline attack, the optimized version substantially reduces recovery time and, in some cases, completes recovery within the evaluation budget where the baseline does not. Our results show that revealing
circuit topology can materially assist recovery of hidden gate functionality, identifying
topology as a security-relevant leakage channel in gate-hiding garbled
circuits.
Read the full abstract ▾
All authors Chao Yin, Zunchen Huang, Chenglu Jin, Marten van Dijk, Fabio Massacci
Cryptography and Security · Cross-listed from quant-ph
Towards Simple and Useful One-Time Programs in the Quantum Random Oracle Model
Lev Stambler
We construct simulation-secure one-time
memories (OTM) in the random oracle model, and present a plausible argument for their security against
quantum adversaries with bounded and adaptive depth. Our contributions include: (1) A simple scheme where we use only single-
qubit Wiesner states and conjunction obfuscation (constructible from LPN): no complex
entanglement or
quantum cryptography is required. (2) A new POVM bound where e prove that any measurement achieving $(1 - \epsilon)$ success on one basis has conjugate-basis guessing probability at most $\frac{1}{2m} + O(\epsilon^\frac{1}{4})$. (3) Simultation-secure OTMs in the
quantum random oracle model where an adversary can only query the random oracle classically. (4) Adaptive depth security where, via an informal application of a lifting
theorem from Arora et al., we
conjecture security against adversaries with
polynomial quantum circuit depth between random oracle queries. Security against adaptive, depth-bounded,
quantum adversaries captures many realistic attacks on OTMs built from single-
qubit states; our work thus paves the way for practical and truly secure one-time programs. Moreover, depth bounded adaptive
adversarial models may allow for encoding one-time
memories into error corrected
memory states, opening the door to implementations of one-time programs which persist for long periods of time.
Read the full abstract ▾
Cryptography and Security
Diffusion-Driven Synthetic Tabular Data Generation for Enhanced DoS/DDoS Attack Classification
Aravind B, Anirud R.S., Sai Surya Teja N et al. (+3 more)
Class imbalance refers to a situation where certain classes in a
dataset have significantly fewer samples than oth- ers, leading to biased model performance. Class imbalance in network intrusion detection using Tabular Denoising
Diffusion Probability Models (TabDDPM) for data augmentation is ad- dressed in this paper. Our approach synthesizes high-fidelity minority-class samples from the CIC-IDS2017
dataset through iterative denoising processes. For the minority classes that have smaller samples, synthetic samples were generated and merged with the original
dataset. The augmented training data enables an ANN classifier to achieve near-perfect recall on previously underrepresented attack classes. These results establish
diffusion models as an effective solution for tabular data imbalance in security domains, with potential applications in fraud detection and medical diagnostics.
Read the full abstract ▾
All authors Aravind B, Anirud R.S., Sai Surya Teja N, Bala Subrahmanya Sriranga Navaneeth A, Karthika R, Mohankumar N
Cryptography and Security
CODE: A Contradiction-Based Deliberation Extension Framework for Overthinking Attacks on Retrieval-Augmented Generation
Xiaolei Zhang, Xiaojun Jia, Liquan Chen et al. (+1 more)
Introducing
reasoning models into
Retrieval-Augmented Generation (RAG) systems enhances task performance through step-by-step
reasoning, logical consistency, and multi-step self-verification. However, recent studies have shown that
reasoning models suffer from overthinking attacks, where models are tricked to generate unnecessarily high number of
reasoning tokens. In this paper, we reveal that such overthinking risk can be inherited by RAG systems equipped with
reasoning models, by proposing an end-to-end attack framework named Contradiction-Based Deliberation Extension (CODE). Specifically, CODE develops a multi-
agent architecture to construct poisoning samples that are injected into the knowledge base. These samples 1) are highly correlated with the use query, such that can be retrieved as inputs to the
reasoning model; and 2) contain contradiction between the logical and evidence layers that cause models to overthink, and are optimized to exhibit highly diverse styles. Moreover, the inference overhead of CODE is extremely difficult to detect, as no modification is needed on the user query, and the task accuracy remain unaffected. Extensive experiments on two
datasets across five commercial
reasoning models demonstrate that the proposed attack causes a 5.32x-24.72x increase in
reasoning token consumption, without degrading task performance. Finally, we also discuss and evaluate potential countermeasures to mitigate overthinking risks.
Read the full abstract ▾
All authors Xiaolei Zhang, Xiaojun Jia, Liquan Chen, Songze Li
Cryptography and Security
Adversarial News and Lost Profits: Manipulating Headlines in LLM-Driven Algorithmic Trading
Advije Rizvani, Giovanni Apruzzese and Pavel Laskov
Large
Language Models (LLMs) are increasingly adopted in the financial domain. Their exceptional capabilities to analyse textual data make them well-suited for inferring the sentiment of finance-related news. Such feedback can be leveraged by algorithmic trading systems (ATS) to guide buy/sell decisions. However, this practice bears the risk that a threat actor may craft "
adversarial news" intended to mislead an LLM. In particular, the news headline may include "malicious" content that remains invisible to human readers but which is still ingested by the LLM. Although prior work has studied textual
adversarial examples, their system-wide impact on LLM-supported ATS has not yet been quantified in terms of monetary risk. To address this threat, we consider an adversary with no direct access to an ATS but able to alter stock-related news headlines on a single day. We evaluate two human-imperceptible manipulations in a financial context: Unicode homoglyph substitutions that misroute models during stock-name recognition, and hidden-text clauses that alter the sentiment of the news headline. We implement a realistic ATS in Backtrader that fuses an LSTM-based price forecast with LLM-derived sentiment (FinBERT, FinGPT, FinLLaMA, and six general-purpose LLMs), and quantify monetary impact using portfolio metrics. Experiments on real-world data show that manipulating a one-day attack over 14 months can reliably mislead LLMs and reduce annual returns by up to 17.7 percentage points. To assess real-world feasibility, we analyze popular scraping libraries and trading platforms and survey 27 FinTech practitioners, confirming our hypotheses. We notified trading platform owners of this security issue.
Read the full abstract ▾
Cryptography and Security
High-Throughput and Scalable Secure Inference Protocols for Deep Learning with Packed Secret Sharing
Qinghui Zhang, Xiaojun Chen, Yansong Zhang et al. (+1 more)
Most existing secure neural network inference protocols based on secure multi-party computation (MPC) typically support at most four participants, demonstrating severely limited scalability. Liu et al. (USENIX Security'24) presented the first relatively practical approach by utilizing Shamir secret sharing with Mersenne
prime fields. However, when processing deeper neural networks such as VGG16, their protocols incur substantial communication overhead, resulting in particularly significant latency in wide-area network (WAN) environments. In this paper, we propose a high-throughput and scalable MPC protocol for neural network inference against semi-honest adversaries in the honest-majority setting. The core of our approach lies in leveraging packed Shamir secret sharing (PSS) to enable parallel computation and reduce communication complexity. The main contributions are three-fold: i) We present a communication-efficient protocol for vector-
matrix multiplication, based on our newly defined notion of vector-
matrix multiplication-friendly random share tuples. ii) We design the filter packing approach that enables parallel convolution. iii) We further extend all non-linear protocols based on Shamir secret sharing to the PSS-based protocols for achieving parallel non-linear operations. Extensive experiments across various
datasets and neural networks demonstrate the superiority of our approach in WAN. Compared to Liu et al. (USENIX Security'24), our scheme reduces the communication upto 5.85x, 11.17x, and 6.83x in offline, online and total communication overhead, respectively. In addition, our scheme is upto 1.59x, 2.61x, and 1.75x faster in offline, online and total running time, respectively.
Read the full abstract ▾
All authors Qinghui Zhang, Xiaojun Chen, Yansong Zhang, Xudong Chen
Cryptography and Security
Post-Quantum Secure Aggregation via Code-Based Homomorphic Encryption
Sebastian Bitzer, Maximilian Egger, Mumin Liu et al. (+1 more)
Secure aggregation lets a server learn the sum of inputs from multiple users without learning individual contributions. Recent single-server protocols achieve this with key- and message-additive homomorphic encryption (KAHE), but existing instantiations rely on
lattice assumptions. We instantiate a KAHE from the Learning Parity with Noise (LPN) assumption and analyze the challenges that arise with the Hamming metric. Decryption of the aggregate amounts to decoding over a q-ary symmetric channel whose crossover probability grows with the number of users. Security under leakage of the key sum reduces to LPN with a noise rate that is quadratic in the KAHE noise rate, and a matching attack shows that this loss is inherent when all but two users collude. Asymptotically, we characterize the optimal communication cost in closed form, showing that the overhead over the raw payload vanishes as the input length grows, and decouple the field size from the input range via the Chinese remainder
theorem. In cross-silo settings, our protocol requires less communication than information-theoretically secure aggregation for moderate to high collusion.
Read the full abstract ▾
All authors Sebastian Bitzer, Maximilian Egger, Mumin Liu, Antonia Wachter-Zeh
Cryptography and Security
PrivFly: A Privacy-Preserving Self-Supervised Framework for Rare Attack Detection in IoFT
Safaa Menssouri and El Mehdi Amhoud
The Internet of Flying Things (IoFT) plays a vital role in modern applications such as aerial surveillance and smart mobility. However, it remains highly vulnerable to cyberattacks that threaten the confidentiality, integrity, and availability of sensitive data. Developing effective intrusion detection systems (IDS) for IoFT networks faces key challenges, including data imbalance,
privacy concerns, and the limited capability of traditional models to detect rare but potentially damaging cyber threats. In this work, we propose PrivFly, a
privacy-preserving IDS framework that integrates self-supervised representation learning and differential
privacy (DP) to enhance detection performance in imbalanced IoFT network traffic. We propose a masked feature reconstruction module for self-supervised pretraining, improving feature representations and boosting rare-class detection. Differential
privacy is applied during training to protect sensitive information without significantly compromising model performance. In addition, we conduct a SHapley additive explanations (SHAP)-based analysis to evaluate the impact of DP on feature importance and model behavior. Experimental results on the ECU-IoFT
dataset show that PrivFly achieves up to 98% accuracy and 99% F1-score, effectively balancing
privacy and detection performance for secure IoFT systems.
Read the full abstract ▾
Cryptography and Security
KinGuard: Hierarchical Kinship-Aware Fingerprinting to Defend Against Large Language Model Stealing
Zhenhua Xu, Xiaoning Tian, Wenjun Zeng et al. (+5 more)
Protecting the intellectual property of large
language models requires robust ownership verification. Conventional
backdoor fingerprinting, however, is flawed by a stealth-robustness paradox: to be robust, these methods force models to memorize fixed responses to high-perplexity triggers, but this targeted overfitting creates detectable statistical artifacts. We resolve this paradox with KinGuard, a framework that embeds a private knowledge corpus built on structured kinship narratives. Instead of memorizing superficial triggers, the model internalizes this knowledge via incremental pre-training, and ownership is verified by probing its conceptual understanding. Extensive experiments demonstrate KinGuard's superior effectiveness, stealth, and resilience against a battery of attacks including fine-tuning, input perturbation, and model merging. Our work establishes knowledge-based
embedding as a practical and secure paradigm for model fingerprinting.
Read the full abstract ▾
All authors Zhenhua Xu, Xiaoning Tian, Wenjun Zeng, Wenpeng Xing, Tianliang Lu, Gaolei Li, Chaochao Chen, Meng Han
Cryptography and Security
Reproducibility in Event-Log Research: A Parametrised Generator and Benchmark for Event-based Signatures
Saad Khan, Simon Parkinson and Monika Roopak
Event-based
datasets are crucial for cybersecurity analysis. A key use case is detecting event-based signatures, which represent attacks spanning multiple events and can only be understood once the relevant events are identified and linked. Analysing event
datasets is essential for monitoring system security, but their growing volume and frequency create significant scalability and processing difficulties. Researchers rely on these
datasets to develop and test techniques for automatically identifying signatures. However, because real
datasets are security-sensitive and rarely shared, it becomes difficult to perform meaningful comparative evaluation between different approaches. This work addresses this evaluation limitation by offering a systematic method for generating event logs with known ground truth, enabling reproducible and comparable research. We present a novel parametrised generation technique capable of producing synthetic event
datasets that contain event-based signatures for discovery. To demonstrate the capabilities of the technique, we provide a
benchmark in signature detection. Our benchmarking demonstrated the suitability of DBSCAN, achieving a score greater than 0.95 Adjusted Rand Index on most generated
datasets. This work enhances the ability of researchers to develop and
benchmark new cybersecurity techniques, ultimately contributing to more robust and effective cybersecurity measures.
Read the full abstract ▾
Cryptography and Security
On the Evidentiary Limits of Membership Inference for Copyright Auditing
Murat Bilgehan Ertan, Emirhan Böge, Min Chen et al. (+2 more)
As large
language models (LLMs) are trained on increasingly opaque corpora, membership inference attacks (MIAs) have been proposed to audit whether copyrighted texts were used during training, despite growing concerns about their reliability under realistic conditions. We ask whether MIAs can serve as admissible evidence in
adversarial copyright disputes where an accused model developer may obfuscate training data while preserving semantic content, and formalize this setting through a judge-prosecutor-accused communication protocol. To test robustness under this protocol, we introduce SAGE (Structure-Aware SAE-Guided Extraction), a paraphrasing framework guided by Sparse Autoencoders (SAEs) that rewrites training data to alter lexical structure while preserving semantic content and downstream utility. Our experiments show that state-of-the-art MIAs degrade when models are fine-tuned on SAGE-generated paraphrases, indicating that their signals are not robust to semantics-preserving transformations. While some leakage remains in certain fine-tuning regimes, these results suggest that MIAs are brittle in
adversarial settings and insufficient, on their own, as a standalone mechanism for copyright
auditing of LLMs.
Read the full abstract ▾
All authors Murat Bilgehan Ertan, Emirhan Böge, Min Chen, Kaleel Mahmood, Marten van Dijk
Cryptography and Security
Your Privacy Depends on Others: Collusion Vulnerabilities in Individual Differential Privacy
Johannes Kaiser, Alexander Ziller, Eleni Triantafillou et al. (+2 more)
Individual Differential
Privacy (iDP) promises users control over their
privacy, but this promise can be broken in practice. We reveal a previously overlooked vulnerability in sampling-based iDP mechanisms: while conforming to the iDP guarantees, an individual's
privacy risk is not solely governed by their own
privacy budget, but critically depends on the
privacy choices of all other data contributors. This creates a mismatch between the promise of individual
privacy control and the reality of a system where risk is collectively determined. We demonstrate empirically that certain distributions of
privacy preferences can unintentionally inflate the
privacy risk of individuals, even when their formal guarantees are met. Moreover, this excess risk provides an exploitable attack vector. A central adversary or a set of colluding adversaries can deliberately choose
privacy budgets to amplify vulnerabilities of targeted individuals. Most importantly, this attack operates entirely within the guarantees of DP, hiding this excess vulnerability. Our empirical evaluation demonstrates successful attacks against 62% of targeted individuals, substantially increasing their membership inference susceptibility. To mitigate this, we propose $(\varepsilon_i,\delta_i,\overline{\Delta})$-iDP a
privacy contract that uses $\Delta$-divergences to provide users with a hard upper bound on their excess vulnerability, while offering flexibility to mechanism design. Our findings expose a fundamental challenge to the current paradigm, demanding a re-evaluation of how iDP systems are designed, audited, communicated, and deployed to make excess risks transparent and controllable.
Read the full abstract ▾
All authors Johannes Kaiser, Alexander Ziller, Eleni Triantafillou, Daniel Rückert, Georgios Kaissis