Cryptography and Security
Evaluating Large Language Models for Security Bug Report Prediction
Farnaz Soltaniani, Shoaib Razzaq and Mohammad Ghafari
Early detection of security bug reports (SBRs) is critical for timely vulnerability mitigation. We present an evaluation of
prompt-based engineering and fine-tuning approaches for predicting SBRs using Large
Language Models (LLMs). Our findings reveal a distinct trade-off between the two approaches. Prompted proprietary models demonstrate the highest sensitivity to SBRs, achieving a G-measure of 77% and a recall of 74% on average across all the
datasets, albeit at the cost of a higher false-positive rate, resulting in an average precision of only 22%. Fine-tuned models, by contrast, exhibit the opposite behavior, attaining a lower overall G-measure of 51% but substantially higher precision of 75% at the cost of reduced recall of 36%. Though a one-time investment in building fine-tuned models is necessary, the inference on the largest
dataset is up to 50 times faster than that of proprietary models. These findings suggest that further investigations to harness the power of LLMs for SBR prediction are necessary.
Read the full abstract ▾
Cryptography and Security
Assessing the Real-World Impact of Post-Quantum Cryptography on WPA-Enterprise Networks
Lukas Köder, Nils Lohmiller, Phil Schmieder et al. (+3 more)
The advent of large-scale
quantum computers poses a significant threat to contemporary network security protocols, including Wi-Fi Protected Access (WPA)-Enterprise authentication. To mitigate this threat, the adoption of Post-
Quantum Cryptography (PQC) is critical. In this work, we investigate the performance impact of PQC
algorithms on WPA-Enterprise-based authentication. To this end, we conduct an experimental evaluation of authentication latency using a testbed built with the open-source tools FreeRADIUS and hostapd, measuring the time spent at the client, access point, and RADIUS server. We evaluate multiple combinations of PQC
algorithms and analyze their performance overhead in comparison to currently deployed cryptographic schemes. Beyond performance, we assess the security implications of these
algorithm choices by relating authentication mechanisms to the
quantum effort required for their exploitation. This perspective enables a systematic categorization of PQ-relevant weaknesses in WPA-Enterprise according to their practical urgency. The evaluation results show that, although PQC introduces additional authentication latency, combinations such as ML-DSA-65 and Falcon-1024 used in conjunction with ML-KEM provide a favorable trade-off between security and performance. Furthermore, we demonstrate that the resulting overhead can be effectively mitigated through session resumption. Overall, this work presents a first real-world performance evaluation of PQC-enabled WPA-Enterprise authentication and demonstrates its practical feasibility for enterprise Wi-Fi deployments.
Read the full abstract ▾
All authors Lukas Köder, Nils Lohmiller, Phil Schmieder, Bastian Buck, Michael Menth, Tobias Heer
Cryptography and Security
Hide and Seek in Embedding Space: Geometry-based Steganography and Detection in Large Language Models
Charles Westphal, Keivan Navaie and Fernando E. Rosas
Fine-tuned LLMs can covertly encode
prompt secrets into outputs via steganographic channels. Prior work demonstrated this threat but relied on trivially recoverable encodings. We formalize payload recoverability via classifier accuracy and show previous schemes achieve 100% recoverability. In response, we introduce low-recoverability steganography, replacing arbitrary mappings with
embedding-space-derived ones. For Llama-8B (LoRA) and Ministral-8B (LoRA) trained on TrojanStego
prompts, exact secret recovery rises from 17$\rightarrow$30% (+78%) and 24$\rightarrow$43% (+80%) respectively, while on Llama-70B (LoRA) trained on Wiki
prompts, it climbs from 9$\rightarrow$19% (+123%), all while reducing payload recoverability. We then discuss detection. We argue that detecting fine-tuning-based steganographic attacks requires approaches beyond traditional steganalysis. Standard approaches measure distributional shift, which is an expected side-effect of fine-tuning. Instead, we propose a mechanistic interpretability approach: linear probes trained on later-layer activations detect the secret with up to 33% higher accuracy in fine-tuned models compared to base models, even for low-recoverability schemes. This suggests that malicious fine-tuning leaves actionable internal signatures amenable to interpretability-based defenses.
Read the full abstract ▾
Cryptography and Security
Trojan-Resilient NTT: Protecting Against Control Flow and Timing Faults on Reconfigurable Platforms
Rourab Paul, Krishnendu Guha and Amlan Chakrabarti
Number Theoretic Transform (NTT) is the most essential component for
polynomial multiplications used in
lattice-based Post-
Quantum Cryptography (PQC)
algorithms such as Kyber, Dilithium, NTRU etc. However, side-channel attacks (SCA) and hardware vulnerabilities in the form of hardware Trojans may alter control signals to disrupt the
circuit's control flow and introduce unconventional delays in the critical hardware of PQC. Hardware Trojans, especially on control signals, are more low cost and impactful than data signals because a single corrupted control signal can disrupt or bypass entire computation sequences, whereas data faults usually cause only localized errors. On the other hand, adversaries can perform Soft Analytical Side Channel Attacks (SASCA) on the design using the inserted hardware Trojan. In this paper, we present a secure NTT architecture capable of detecting unconventional delays, control-flow disruptions, and SASCA, while providing an adaptive fault-correction methodology for their mitigation. Extensive simulations and implementations of our Secure NTT on Artix-7 FPGA with different Kyber variants show that our fault detection and correction modules can efficiently detect and correct faults whether caused unintentionally or intentionally by hardware Trojans with a high success rate, while introducing only modest area and time overheads.
Read the full abstract ▾
Cryptography and Security
Trackly: A Unified SaaS Platform for User Behavior Analytics and Real Time Rule Based Anomaly Detection
Md Zahurul Haque, Md. Hafizur Rahman and Yeahyea Sarker
Understanding user behavior is essential for improving digital experiences, optimizing business conversions, and mitigating threats like account takeovers, fraud, and bot attacks. Most platforms separate product analytics and security, creating fragmented visibility and delayed threat detection. Trackly, a scalable SaaS platform, unifies comprehensive user behavior analytics with real time, rule based anomaly detection. It tracks sessions, IP based geo location, device browser fingerprints, and granular events such as page views, add to cart, and checkouts. Suspicious activities logins from new devices or locations, impossible travel (Haversine formula), rapid bot like actions, VPN proxy usage, or multiple accounts per IP are flagged via configurable rules with weighted risk scoring, enabling transparent, explainable decisions. A real time dashboard provides global session maps, DAU MAU, bounce rates, and session durations. Integration is simplified with a lightweight JavaScript SDK and secure REST APIs. Implemented on a multi tenant microservices stack (ASP.NET Core, MongoDB, RabbitMQ, Next.js), Trackly achieved 98.1% accuracy, 97.7% precision, and 2.25% false positives on synthetic
datasets, proving its efficiency for SMEs and ecommerce.
Read the full abstract ▾
Cryptography and Security · Cross-listed from cs.CV
Color Matters: Demosaicing-Guided Color Correlation Training for Generalizable AI-Generated Image Detection
Nan Zhong, Yiran Xu and Mian Zou
As realistic AI-generated images threaten digital authenticity, we address the generalization failure of generative artifact-based detectors by exploiting the intrinsic properties of the camera imaging pipeline. Concretely, we investigate color correlations induced by the color filter array (CFA) and demosaicing, and propose a Demosaicing-guided Color Correlation Training (DCCT) framework for AI-generated image detection. By simulating the CFA sampling pattern, we decompose each color image into a single-channel input (as the condition) and the remaining two channels as the ground-truth targets (for prediction). A self-supervised U-Net is trained to model the conditional distribution of the missing channels from the given one, parameterized via a mixture of logistic functions. Our theoretical analysis reveals that DCCT targets a provable distributional difference in color-correlation features between photographic and AI-generated images. By leveraging these distinct features to construct a binary classifier, DCCT achieves state-of-the-art generalization and robustness, significantly outperforming prior methods across over 20 unseen generators.
Read the full abstract ▾
Cryptography and Security
Rust and Go directed fuzzing with LibAFL-DiFuzz
Timofey Mezhuev, Darya Parygina and Daniil Kuts
In modern SSDLC, program analysis and automated testing are essential for minimizing vulnerabilities before software release, with fuzzing being a fast and widely used dynamic testing method. However, traditional coverage-guided fuzzing may be less effective in specific tasks like verifying static analysis reports or reproducing crashes, while directed fuzzing, focusing on targeted program locations using proximity metrics, proves to be more effective. Some of the earliest directed fuzzers are, for example, AFLGo and BEACON, which use different proximity metric approaches. Although most automated testing tools focus on C/C++ code, the growing popularity of Rust and Go causes the need for precise and efficient testing solutions for these
languages. This work expands the applicability of directed fuzzing beyond traditional analysis of C/C++ software. We present a novel approach to directed greybox fuzzing tailored specifically for Rust and Go applications. We introduce advanced preprocessing techniques, rustc compiler customizations, and elaborate graph construction and instrumentation methods to enable effective targeting of specific program locations. Our implemented fuzzing tools, based on LibAFL-DiFuzz backend, demonstrate competitive advantages compared to popular existing fuzzers like afl$.$rs, cargo-fuzz, and go-fuzz. According to TTE (Time to Exposure) experiments, Rust-LibAFL-DiFuzz outperforms other tools by the best TTE result. Some stability issues can be explained by different mutation approaches. Go-LibAFL-DiFuzz outperforms its opponent by the best and, in the majority of cases, by average result, having two cases with orders of magnitude difference. These results prove better efficiency and accuracy of our approach.
Read the full abstract ▾
Cryptography and Security
Okara: Detection and Attribution of TLS Man-in-the-Middle Vulnerabilities in Android Apps with Foundation Models
Haoyun Yang, Ronghong Huang, Yong Fang et al. (+4 more)
Transport Layer Security (TLS) is fundamental to secure online communication, yet vulnerabilities in certificate validation that enable Man-in-the-Middle (MitM) attacks remain a pervasive threat in Android apps. Existing detection tools are hampered by low-coverage UI interaction, costly instrumentation, and a lack of scalable root-cause analysis. We present Okara, a framework that leverages foundation models to automate the detection and deep attribution of TLS MitM Vulnerabilities (TMVs). Okara's detection component, TMV-Hunter, employs foundation model-driven GUI
agents to achieve high-coverage app interaction, enabling efficient vulnerability discovery at scale. Deploying TMV-Hunter on 37,349 apps from Google Play and a third-party store revealed 8,374 (22.42%) vulnerable apps. Our measurement shows these vulnerabilities are widespread across all popularity levels, affect critical functionalities like authentication and code delivery, and are highly persistent with a median vulnerable lifespan of over 1,300 days. Okara's attribution component, TMV-ORCA, combines dynamic instrumentation with a novel LLM-based classifier to locate and categorize vulnerable code according to a comprehensive new taxonomy. This analysis attributes 41% of vulnerabilities to third-party libraries and identifies recurring insecure patterns, such as empty trust managers and flawed hostname verification. We have initiated a large-scale responsible disclosure effort and will release our tools and
datasets to support further research and mitigation.
Read the full abstract ▾
All authors Haoyun Yang, Ronghong Huang, Yong Fang, Beizeng Zhang, Junpu Guo, Zhanyu Wu, Xianghang Mi
Cryptography and Security · Cross-listed from cs.CV
Beauty and the Beast: Imperceptible Perturbations Against Diffusion-Based Face Swapping via Directional Attribute Editing
Yilong Huang and Songze Li
Diffusion-based face swapping achieves state-of-the-art performance, yet it also exacerbates the potential harm of malicious face swapping to violate portraiture right or undermine personal reputation. This has spurred the development of proactive defense methods. However, existing approaches face a core trade-off: large perturbations distort facial structures, while small ones weaken protection effectiveness. To address these issues, we propose FaceDefense, an enhanced proactive defense framework against
diffusion-based face swapping. Our method introduces a new
diffusion loss to strengthen the defensive efficacy of
adversarial examples, and employs a directional facial attribute editing to restore perturbation-induced distortions, thereby enhancing visual imperceptibility. A two-phase alternating optimization strategy is designed to generate final perturbed face images. Extensive experiments show that FaceDefense significantly outperforms existing methods in both imperceptibility and defense effectiveness, achieving a superior trade-off.
Read the full abstract ▾
Cryptography and Security
AEGIS: White-Box Attack Path Generation using LLMs and Training Effectiveness Evaluation for Large-Scale Cyber Defence Exercises
Ivan K. Tung, Yu Xiang Shi, Alex Chien et al. (+2 more)
Creating attack paths for cyber defence exercises requires substantial expert effort. Existing automation requires vulnerability graphs or exploit sets curated in advance, limiting where it can be applied. We present AEGIS, a system that generates attack paths using LLMs, white-box access, and Monte Carlo Tree Search over real exploit execution. LLM-based search discovers exploits dynamically without pre-existing vulnerability graphs, while white-box access enables validating exploits in isolation before committing to attack paths. Evaluation at CIDeX 2025, a large-scale exercise spanning 46 IT hosts, showed that AEGIS-generated paths are comparable to human-authored scenarios across four dimensions of training experience (perceived learning, engagement, believability, challenge). Results were measured with a validated questionnaire extensible to general simulation-based training. By automating exploit chain discovery and validation, AEGIS reduces scenario development from months to days, shifting expert effort from technical validation to scenario design.
Read the full abstract ▾
All authors Ivan K. Tung, Yu Xiang Shi, Alex Chien, Wenkai Liu, Lawrence Zheng
Cryptography and Security
AlienLM: Alienization of Language for API-Boundary Privacy in Black-Box LLMs
Jaehee Kim and Pilsung Kang
Modern LLMs are increasingly accessed via black-box APIs, requiring users to transmit sensitive
prompts, outputs, and fine-tuning data to external providers, creating a critical
privacy risk at the API boundary. We introduce AlienLM, a deployable API-only \cradd{exposure-reduction layer that reduces plaintext exposure} by translating text into an Alien
Language via a vocabulary-scale bijection, enabling lossless recovery on the client side. Using only standard fine-tuning APIs, Alien Adaptation Training (AAT) adapts target models to operate directly on alienized inputs. Across four LLM backbones and seven
benchmarks, AlienLM retains over 81% of plaintext-oracle performance on average, substantially outperforming random-bijection and character-level baselines. Under adversaries with access to model weights, corpus statistics, and learning-based inverse translation, recovery attacks reconstruct fewer than 0.22% of alienized
tokens. Our results demonstrate a practical pathway for \cradd{
privacy-aware} LLM deployment under API-only access, substantially reducing plaintext exposure while maintaining task performance. Code and data are available at https://github.com/KimJaehee0725/AlienLM.
Read the full abstract ▾
Cryptography and Security
RealSec-bench: A Benchmark for Evaluating Secure Code Generation in Real-World Repositories
Yanlin Wang, Ziyao Zhang, Chong Wang et al. (+5 more)
Large
Language Models (LLMs) have demonstrated remarkable capabilities in code generation, but their proficiency in producing secure code remains a critical, under-explored area. Existing
benchmarks often fall short by relying on synthetic vulnerabilities or evaluating functional correctness in isolation, failing to capture the complex interplay between functionality and security found in real-world software. To address this gap, we introduce RealSec-bench, a new
benchmark for secure code generation meticulously constructed from real-world, high-risk Java repositories. Our methodology employs a multi-stage pipeline that combines systematic SAST scanning with CodeQL, LLM-based false positive elimination, and rigorous human expert validation. The resulting
benchmark contains 105 instances grounded in real-word repository contexts, spanning 19 Common Weakness Enumeration (CWE) types and exhibiting a wide diversity of data flow complexities, including vulnerabilities with up to 34-hop inter-procedural dependencies. Using RealSec-bench, we conduct an extensive empirical study on 5 popular LLMs. We introduce a novel composite metric, SecurePass@K, to assess both functional correctness and security simultaneously. We find that while
Retrieval-Augmented Generation (RAG) techniques can improve functional correctness, they provide negligible benefits to security. Furthermore, explicitly prompting models with general security guidelines often leads to compilation failures, harming functional correctness without reliably preventing vulnerabilities. Our work highlights the gap between functional and secure code generation in current LLMs.
Read the full abstract ▾
All authors Yanlin Wang, Ziyao Zhang, Chong Wang, Xinyi Xu, Mingwei Liu, Yong Wang, Jiachi Chen, Zibin Zheng
Cryptography and Security · Cross-listed from cs.CL
FNF: Functional Network Fingerprint for Large Language Models
Yiheng Liu, Junhao Ning, Sichen Xia et al. (+8 more)
The development of large
language models (LLMs) is costly and has significant commercial value. Consequently, preventing unauthorized appropriation of open-source LLMs and protecting developers' intellectual property rights have become critical challenges. In this work, we propose the Functional Network Fingerprint (FNF), a training-free, sample-efficient method for detecting whether a suspect LLM is derived from a victim model, based on the consistency between their functional network activity. We demonstrate that models that share a common origin, even with differences in scale or architecture, exhibit highly consistent patterns of neuronal activity within their functional networks across diverse input samples. In contrast, models trained independently on distinct data or with different objectives fail to preserve such activity
alignment. Unlike conventional approaches, our method requires only a few samples for verification, preserves model utility, and remains robust to common model modifications (such as fine-tuning,
pruning, and parameter permutation), as well as to comparisons across diverse architectures and dimensionalities. FNF thus provides model owners and third parties with a simple, non-invasive, and effective tool for protecting LLM intellectual property. The code is available at https://github.com/WhatAboutMyStar/LLM_ACTIVATION.
Read the full abstract ▾
All authors Yiheng Liu, Junhao Ning, Sichen Xia, Haiyang Sun, Yang Yang, Hanyang Chi, Xiaohui Gao, Ning Qiang, Bao Ge, Junwei Han, Xintao Hu
Cryptography and Security
Do Fine-Tuned LLMs Understand Vulnerabilities? An Investigation into the Semantic Trap
Feiyang Huang, Yuqiang Sun, Fan Zhang et al. (+3 more)
Large
Language Models (LLMs) have shown promising performance in software vulnerability detection, particularly after domain-specific Supervised Fine-Tuning (SFT). However, it remains unclear whether these models genuinely internalize vulnerability root causes or merely exploit surface-level functional patterns. While prior work documented related failures on pre-trained or zero-shot models, the SFT process itself, and how explicit
reasoning supervision modulates it, remains under-explored. We study fine-tuned decoder-only LLMs under vanilla SFT and SFT with
reasoning supervision, identifying a failure mode we term the Semantic Trap, characterized by three symptoms: pairing-sensitive performance, gap-dictated decisions, and fragility to semantic-preserving changes. To probe this, we propose TrapEval, an evaluation framework comprising two real-world
datasets, V2P (vulnerable paired with patched code) and V2N (vulnerable paired with unrelated normal code), alongside semantic perturbations, CodeBLEU-based gap analysis, and an LLM-assisted
reasoning failure taxonomy. Evaluating five representative LLMs fine-tuned with and without explicit
reasoning (Chain-of-Thought), our results show vanilla SFT yields deceptively high scores on unpaired data (V2N) while failing all three symptoms. Models suffer high false-positive rates on V2P, degrade under perturbations, and exhibit a systematic dependency on the textual gap between vulnerable and patched code. Finetuning with explicit
reasoning reduces these symptoms but costs recall; its lack of measurable gap-dependency partly reflects a floor effect rather than escaping the trap. Furthermore, our taxonomy reveals these models still misinterpret control flow and hallucinate API behavior, indicating current fine-tuning mitigates but does not eliminate reliance on surface features.
Read the full abstract ▾
All authors Feiyang Huang, Yuqiang Sun, Fan Zhang, Ziqi Yang, Han Liu, Yang Liu
Cryptography and Security · Cross-listed from cs.HC
Human-Centered Explainability in AI-Enhanced UI Security Interfaces: Designing Trustworthy Copilots for Cybersecurity Analysts
Mona Rajhans
Artificial intelligence (AI) copilots are increasingly integrated into enterprise cybersecurity platforms to assist analysts in threat detection, triage, and remediation. However, the effectiveness of these systems depends not only on the accuracy of underlying models but also on the degree to which users can understand and trust their outputs. Existing research on algorithmic explainability has largely focused on model internals, while little
attention has been given to how explanations should be surfaced in user interfaces for high-stakes decision-making contexts [8], [5], [6]. We present a mixed-methods study of explanation design strategies in AI-driven security dashboards. Through a taxonomy of explanation styles and a controlled user study with security practitioners, we compare natural
language rationales, confidence visualizations, counterfactual explanations, and hybrid approaches. Our findings show that explanation style significantly affects user trust calibration, decision accuracy, and cognitive load. We contribute (1) empirical evidence on the usability of explanation interfaces for security copilots, (2) design guidelines for integrating explainability into enterprise UIs, and (3) a framework for aligning explanation strategies with analyst needs in security operations centers (SOCs). This work advances the design of human-centered AI tools in cybersecurity and provides broader implications for explainability in other high-stakes domains.
Read the full abstract ▾
Cryptography and Security
Whispers of Wealth: Red-Teaming Google's Agent Payments Protocol via Prompt Injection
Tanusree Debi, Wentian Zhu and Pranjol Sen Gupta
Large
language model (LLM) based
agents are increasingly used to automate financial transactions, yet their reliance on contextual
reasoning exposes payment systems to
prompt-driven manipulation. The
Agent Payments Protocol (AP2) aims to secure
agent-led purchases through cryptographically verifiable mandates, but its practical robustness remains underexplored. In this work, we perform an AI red-teaming evaluation of AP2 and identify vulnerabilities arising from indirect and direct
prompt injection. We introduce two attack techniques, the Branded Whisper Attack and the Vault Whisper Attack which manipulate product ranking and extract sensitive user data. Using a functional AP2 based shopping
agent built with Gemini-2.5-Flash and the Google ADK framework, we experimentally validate that simple
adversarial prompts can reliably subvert
agent behavior. Our findings reveal critical weaknesses in current agentic payment architectures and highlight the need for stronger isolation and defensive safeguards in LLM-mediated financial systems.
Read the full abstract ▾
Cryptography and Security
VocBulwark: Towards Practical Generative Speech Watermarking via Additional-Parameter Injection
Weizhi Liu, Yue Li and Zhaoxia Yin
Generated
speech achieves human-level naturalness but escalates security risks of misuse. However, existing watermarking methods fail to reconcile fidelity with robustness, as they rely either on simple superposition in the noise space or on intrusive alterations to model weights. To bridge this gap, we propose VocBulwark, an additional-parameter injection framework that freezes generative model parameters to preserve perceptual quality. Specifically, we design a Temporal Adapter to deeply entangle watermarks with acoustic attributes, synergizing with a Coarse-to-Fine Gated Extractor to resist advanced attacks. Furthermore, we develop an Accuracy-Guided Optimization Curriculum that dynamically orchestrates gradient flow to resolve the optimization conflict between fidelity and robustness. Comprehensive experiments demonstrate that VocBulwark achieves high-capacity and high-fidelity watermarking, offering robust defense against complex practical scenarios, with resilience to Codec regenerations and variable-length manipulations.
Read the full abstract ▾
Cryptography and Security
FraudShield: Knowledge Graph Empowered Defense for LLMs against Fraud Attacks
Naen Xu, Jinghuai Zhang, Ping He et al. (+6 more)
Large
language models (LLMs) have been widely integrated into critical automated workflows, including contract review and job application processes. However, LLMs are susceptible to manipulation by fraudulent information, which can lead to harmful outcomes. Although advanced defense methods have been developed to address this issue, they often exhibit limitations in effectiveness, interpretability, and generalizability, particularly when applied to LLM-based applications. To address these challenges, we introduce FraudShield, a novel framework designed to protect LLMs from fraudulent content by leveraging a comprehensive analysis of fraud tactics. Specifically, FraudShield constructs and refines a fraud tactic-keyword knowledge graph to capture high-confidence associations between suspicious text and fraud techniques. The structured knowledge graph augments the original input by highlighting keywords and providing supporting evidence, guiding the LLM toward more secure responses. Extensive experiments show that FraudShield consistently outperforms state-of-the-art defenses across four mainstream LLMs and five representative fraud types, while also offering interpretable clues for the model's generations.
Read the full abstract ▾
All authors Naen Xu, Jinghuai Zhang, Ping He, Chunyi Zhou, Jun Wang, Zhihui Fu, Tianyu Du, Zhaoxiang Wang, Shouling Ji
Cryptography and Security · Cross-listed from cs.CY
The Third-Party Access Effect: An Overlooked Challenge in Secondary Use of Educational Real-World Data
Hibiki Ito, Chia-Yu Hsu and Hiroaki Ogata
Secondary use of growing real-world data (RWD) in education offers significant opportunities for research, yet
privacy practices intended to enable third-party access to such RWD are rarely evaluated for their implications for downstream analyses. As a result, potential problems introduced by otherwise standard
privacy practices may remain unnoticed. To address this gap, we investigate potential issues arising from common practices by assessing (1) the re-identification risk of fine-grained RWD, (2) how communicating such risks influences learners'
privacy behaviour, and (3) the sensitivity of downstream analytical conclusions to resulting changes in the data. We focus on these practices because re-identification risk and stakeholder communication can jointly influence the data shared with third parties. We find that substantial re-identification risk in RWD, when communicated to stakeholders, can induce opt-outs and non-self-disclosure behaviours. Sensitivity analysis demonstrates that these behavioural changes can meaningfully alter the shared data, limiting validity of secondary-use findings. We conceptualise this phenomenon as the third-party access effect (3PAE) and discuss implications for trustworthy secondary use of educational RWD.
Read the full abstract ▾
Cryptography and Security
Rethinking Anonymity Claims in Synthetic Data Generation: A Model-Centric Privacy Attack Perspective
Georgi Ganev and Emiliano De Cristofaro
Training generative machine learning models to produce synthetic tabular data has become a popular approach for enhancing
privacy in data sharing. As this typically involves processing sensitive personal information, releasing either the trained model or generated synthetic
datasets can still pose
privacy risks. Yet, recent research, commercial deployments, and
privacy regulations like the General Data Protection Regulation (GDPR) largely assess anonymity at the level of an individual
dataset. In this paper, we rethink anonymity claims about synthetic data from a model-centric perspective, arguing that meaningful assessments must account for the underlying generative model and be grounded in state-of-the-art
privacy attacks. This perspective better reflects real-world deployments, where trained models are often accessible for interaction or querying. We interpret the GDPR's definitions of personal data and anonymization under such access assumptions to identify the identifiability risks that must be mitigated and map them to
privacy attacks across threat settings. We then argue that synthetic data techniques alone do not ensure sufficient anonymization. Finally, we compare the two mechanisms most commonly used with synthetic data – Differential
Privacy (DP) and Similarity-based
Privacy Metrics (SBPMs) – and argue that while DP can offer robust protections against identifiability risks, SBPMs lack adequate safeguards. Overall, our work connects regulatory notions of identifiability with model-centric
privacy attacks, enabling more responsible and trustworthy assessment of synthetic data systems by researchers, practitioners, and policymakers.
Read the full abstract ▾