Cryptography and Security · Cross-listed from cs.LG
PropGuard: Safeguarding LLM-MAS via Propagation-Aware Exploration and Remediation
Bingyu Yan, Xiaoming Zhang, Jinyu Hou et al. (+4 more)
LLM-based multi-
agent systems (LLM-MAS) have become a promising paradigm for solving complex tasks through role specialization, tool use,
memory, and collaborative
reasoning. However, these interactions create new security risks that malicious instructions injected through messages, tools, or
memories can propagate across
agents and rounds, causing system-level compromise. Existing defenses largely rely on local filtering or graph-based anomaly detection, but they often fail to trace fine-grained propagation paths or remediate contaminated states without disrupting benign collaboration. We propose PropGuard, a propagation-aware framework for safeguarding LLM-MAS. PropGuard constructs a dual-view spatio-temporal graph that combines response-centric risk estimation with full-state evidence preservation. Guided by these risk priors, a GE-GRPO trained inspector sequentially explores the full-state graph to recover compact suspicious propagation subgraphs. PropGuard then verifies harmful propagation through subgraph-aware diagnosis and applies source-guided remediation to correct upstream contamination and replay affected downstream interactions. Experiments across four communication architectures and five attack settings demonstrate that PropGuard consistently lowers attack success while maintaining high task-level defense success, achieving a favorable effectiveness–efficiency trade-off.
Read the full abstract ▾
All authors Bingyu Yan, Xiaoming Zhang, Jinyu Hou, Chaozhuo Li, Ziyi Zhou, Xiaozhe Zhang, Litian Zhang
Cryptography and Security
When Child Inherits: Modeling and Exploiting Subagent Spawn in Multi-Agent Networks
Ziwen Cai, Yihe Zhang and Xiali Hei
Since the official release of ChatGPT in 2022, large
language models (LLMs) have rapidly evolved from chatbot-style interfaces into agentic systems that can delegate work through tools and newly spawned subagents. While these capabilities improve automation and scalability, they also pose new security risks in multi-
agent networks. Existing research has studied how individual LLM-based
agents can be compromised through
prompt injection, jailbreaking, poisoned
retrieval data, or malicious extensions. Less is known about what happens after one
agent is compromised inside a multi-
agent network. In particular, inherited
memory from parent
agents can carry malicious instructions, outdated states, or unintended behavioral rules into newly created subagents, allowing a local compromise to spread across
agent boundaries. In this paper, we model contemporary multi-
agent networks through the lens of subagent inheritance. Our analysis shows that current frameworks can violate trust boundaries through insecure
memory inheritance, weak resource control, stale post-spawn state, and improper termination authority. We demonstrate these risks in real
agent frameworks and propose defenses based on explicit security invariants. Our findings show that inheritance is not merely an implementation detail, but a central component influencing the security of multi-
agent systems.
Read the full abstract ▾
Cryptography and Security
HEART: A High-Efficiency Adaptive Real-Time Telemonitoring Framework for Secure Electrocardiogram Signal Transmission Using Chaotic Encryption
Beyaz{\i}t Bestami Yuksel
The realtime analysis and secure transmission of electrocardiogram ECG signals are critical for accurate diagnosis and safeguarding patient
privacy in telemedicine applications This study presents a novel realtime ECG monitoring system that employs a learnable key generator LKG derived from each patients own ECG signal characteristics to dynamically produce unique encryption keys These keys determine the parameters r and x0 of a logistic map used for chaotic encryption The system securely encrypts realtime ECG data immediately after acquisition ensuring confidential transmission and storage in the cloud For remote clinical access the encrypted data is downloaded and decrypted on the doctors side using the matching key generated at the source or securely stored in the cloud This approach eliminates the need for traditional key exchange and substantially raises the cost of exhaustive key search in practice through persegment biometric key refresh and combined permutation and XOR
diffusion supported by minentropy evaluation Compared to statickey methods the learnable biometric key design offers greater unpredictability and individualization A comprehensive set of security assessments including Shannon
entropy 7678 bits correlation and autocorrelation disruption histogram statistics NIST SP 80022 frequency testing plaintextkey sensitivity avalanche effect FFTbased spectral flatness and robustness to noise and occlusion confirms the methods strength Reconstruction fidelity MSE approximately 5x106 PSNR greater than 52 dB MAE approximately 0002 demonstrates nearlossless decryption and preserved diagnostic features Encryption latency remains low preserving realtime performance.
Read the full abstract ▾
Cryptography and Security
SL5 Standard for AI Security
Lisa Thiergart, Yoav Tzfati, Peter Wagstaff et al. (+3 more)
Security Level 5 (SL5) is a security posture for AI systems that could plausibly thwart top-priority operations by the world's most cyber-capable institutions: those with extensive resources, state-level infrastructure, and expertise years ahead of the public state of the art. The SL5 terminology originates from the RAND Corporation's 2024 report "Securing AI Model Weights". Frontier AI development requires use-case-specific, productivity-optimised and updateable AI datacenter security standards. This first revision of the SL5 standard focuses on requirements with long lead times: interventions that must be planned years in advance, such as facility construction, hardware procurement, and organizational capability development. We prioritize these requirements because preserving optionality for SL5 by 2028/2029 requires starting now. These capabilities cannot be retrofitted on short notice when the need becomes urgent. Some requirements represent significant departures from current day standard practice. We believe bold measures are necessary for this level of security and see clear opportunities to apply optimization pressure to existing and novel solutions to customize them for the AI industry and address the practical operational requirements as much as possible. Our organization exists to begin paving this path. Some requirements approximate government security capabilities where private-sector approaches may be insufficient. We identify these gaps and note where government involvement may ultimately be necessary.
Read the full abstract ▾
All authors Lisa Thiergart, Yoav Tzfati, Peter Wagstaff, Guy, Luis Cosio, Philip Reiner
Cryptography and Security
Improving Parameter-Efficient Federated Learning with Differentially Private Refactorization
Linh Tran, Ana Milanova and Stacy Patterson
Federated Learning (FL) with parameter-efficient fine-tuning, such as Low-Rank Adaptation (LoRA), enables scalable model training on distributed data. However, when combined with Differential
Privacy (DP), LoRA often introduces errors during global aggregation and amplifies the negative effect of DP noise. Existing cross-silo FL approaches mitigate the aggregation error by freezing one LoRA module and applying output perturbation. However, in a restricted low-rank subspaces, this additive noise frequently overwhelms the signals of the weight
matrices, leading to suboptimal accuracy. To address this vulnerability, we propose FedPower, a differentially private cross-silo FL framework that reshapes server-side aggregation. Instead of perturbing mismatched low-rank factors, FedPower explicitly reconstructs and clips full-rank client updates to bound the sensitivity. The server then projects the exact aggregated update back into a secure low-rank space using PowerDP, a novel differentially private low-rank factorization mechanism. Based on simultaneous subspace iteration, PowerDP injects calibrated DP noise prior to the final orthonormalization step, effectively mitigates the negative effect of DP noise by preserving
matrix orthogonality. We provide rigorous theoretical analyses establishing sensitivity bounds for subspace projections, proving that FedPower achieves both sample-level and client-level DP. Extensive experiments on various
language understanding tasks in cross-silo FL settings show that FedPower is robust against tight
privacy budgets while adding negligible computational overheads. Additional empirical study on different DP noise injection schemes validates the effectiveness of PowerDP in improving the tradeoff in accuracy and
privacy. Evaluation on three different membership inference attacks validates the robustness and
privacy-preserving capability of the proposed framework.
Read the full abstract ▾
Cryptography and Security
Injection-Execution Dissociation: A Mechanistic Evaluation of Persistent Memory Attacks and Defenses in Stateful LLM Agents
Jun Wen Leong
We discover that
prompt-injection success and tool-execution success are separable safety properties: defenses that block injection do not necessarily block execution, and vice versa. We call this the injection-execution dissociation. In LLM
agents with persistent
memory, malicious instructions are stored at rates exceeding 97.5%, yet downstream execution ranges from 0% to 95% with no correlation to storage rate. This reframes the threat model: preventing storage alone is insufficient, and blocking execution requires structurally enforcing authority boundaries between
memory ingestion and action execution. We substantiate this through a 5,040-run factorial experiment across nine open-source models (N=40 per condition), evaluating six defenses at four architectural layers against delayed-trigger attacks that persist across session boundaries via RAG
retrieval. Defense effectiveness is governed by where a defense sits relative to the attack's authority boundary, not by classifier quality. Only
Memory Sandbox – a tool-layer defense that structurally isolates recalled
memory from executable context – reduces attack success to 0% for eight of nine models. A
reasoning-mode ablation reveals a double dissociation: no single schema-layer intervention is safe across both
reasoning and non-
reasoning model classes. A loaded-corpus frontier evaluation (21 models, 3 providers; N=40 base, headline models topped up to N=172) reveals vendor-correlated patterns: Anthropic blocks predominantly at injection, OpenAI blocks at execution with variable generational hardening, and a pre-release Gemini endpoint exfiltrates in the majority of runs. Stored-but-dormant payloads constitute a compositional supply-chain risk in shared-
memory deployments.
Read the full abstract ▾
Cryptography and Security
Deterministic Fully-Static Whole-Binary Translation without Heuristics
Hongyu Chen, James McGowan and Michael Franz
We present Elevator, the first binary translator that statically translates entire x86-64 executables to AArch64 without debug information, source code, or assumptions about code layout. Unlike existing systems, which rely on heuristics or runtime fallbacks to handle code-versus-data decoding errors, Elevator considers all possible interpretations of every byte and produces a separate translation for each feasible one ahead of time. Any byte may be interpreted as data, an opcode, or an opcode argument; we generate separate control flow paths for all interpretations,
pruning only those leading to abnormal termination. Translations are built by composing code "tiles" automatically derived from a high-level description of the source ISA, yielding a nimble translation framework. The approach is deterministic and produces complete, self-contained binaries with no runtime component in the trusted code base. The principal cost is substantial code size expansion. The key benefit is that the output is the actual code that will run, enabling testing, validation, certification, and cryptographic signing prior to deployment, reducing risk compared to emulators or JIT compilers. We evaluate Elevator on a diverse corpus of real-world binaries, including the entire SPECint 2006 suite, demonstrating that static full-program binary translation can be both reliable and practical. Elevator achieves performance on par with or better than QEMU's user-mode JIT emulation.
Read the full abstract ▾
Cryptography and Security
Binge, Bot, Repeat: Unpacking the Ecosystem of Video Piracy on Telegram
Sadikshya Gyawali, Jaishnoor Kaur, Taylor Graham et al. (+4 more)
Telegram has emerged as a major platform for large-scale
video piracy, where copyrighted content is rapidly distributed among users. Despite its prominence, the structural and operational dynamics of this ecosystem remain insufficiently understood. To address this gap, we present the first large-scale study of
video piracy on Telegram through a mixed-method analysis of 1,057 channels that shared 209k unique posts between December 2023 and January 2026 - systematically characterizing their content, distribution strategies, and how the ecosystem is sustained at scale. Central to our approach is the development of a fine-grained taxonomy that enables a structured understanding of the activity and intent of these channels on a per-post level. The channels collectively distributed 19,033 unique copyrighted titles originating from 175 countries, accumulating over 4.85B unique views and resulting in a lower-bound estimated financial loss of $17.49B for content rights holders. We also find that this ecosystem is deliberately engineered to be resilient against takedown efforts, frequently redirecting users through chains of intermediary channels and automated bots that collectively handle hosting, access control, monetization, and channel discovery. The scale and persistence of this ecosystem motivated the development of Anti-RIP, a real-time framework for detecting emerging
video piracy communities on Telegram. Anti-RIP utilizes our taxonomy to generate contextual, interpretable insights that stakeholders confirmed improve the triaging action against reported posts and channels. Over a 61-day period, the framework facilitated the takedown of 524 previously unknown piracy channels and 71 bots. To support reproducibility and future research, we open-source both the
dataset and the Anti-RIP framework.
Read the full abstract ▾
All authors Sadikshya Gyawali, Jaishnoor Kaur, Taylor Graham, Josef Horacek, Nowshin Tabassum, Shirin Nilizadeh, Sayak Saha Roy
Cryptography and Security
Quantifiable Uncertainty: A Stochastic Consensus Multi-Agent RAG Framework for Robust Malware Detection
ElMouatez Billah Karbab
While contemporary deep learning malware detectors define a dominant defense paradigm, their sophistication also exposes them to novel structural evasion attacks, a limitation we attribute to their inherent inability to express epistemic uncertainty. To address this challenge, we present MAGMA, a
Retrieval-Augmented Generation (RAG) framework that decouples malware analysis into semantic code
retrieval and probabilistic verification. In contrast to monolithic classifiers, MAGMA employs a dual-stream
embedding scheme over assembly and pseudo-code representations to isolate Decision-Critical Functions (DCFs) from the noise of dead code. We further introduce a
Stochastic Consistency Ensemble, in which multiple instances of the same
reasoning agent independently evaluate the
retrieval set under non-deterministic sampling. From this ensemble, we derive two complementary metrics: Function Evidence Strength (FES), a weighted aggregation of
retrieval confidence, and the Evidence Conflict Score (ECS), defined as the Shannon
entropy of the ensemble's predictive distribution. We show that elevated ECS values serve as an effective proxy for structural ambiguity, enabling the system to implement a principled “reject-option” policy. Extensive evaluation demonstrates that MAGMA achieves a 98.4% detection rate, substantially exceeding existing solutions.
Read the full abstract ▾
Cryptography and Security
SecureForge: Finding and Preventing Vulnerabilities in LLM-Generated Code via Prompt Optimization
Houjun Liu, Lisa Einstein, John Yang et al. (+5 more)
LLM coding
agents now generate code at an unprecedented scale, yet LLM-generated code introduces cybersecurity vulnerabilities into codebases without human involvement. Even when frontier models are explicitly asked to write secure production code with relevant weaknesses to avoid in context, we find that they still produce verifiable vulnerabilities on average 23% of the time across a corpus of 250 benign coding
prompts. We introduce SecureForge, an automated pipeline that both audits security risks of frontier models and produces
auditing-informed secure system
prompts that reduce output security vulnerabilities while maintaining unit test performance. SecureForge first identifies benign
prompts that produce statically detectable vulnerabilities, and then amplifies them into a large synthetic
prompt corpus of diverse scenarios using a Markovian sampling technique to jointly maintain error rates and
prompt diversity. This corpus is then used to iteratively optimize the system
prompts to reduce output security vulnerabilities. On frontier models, SecureForge yields a statistically significant Pareto improvement in both unit test success and output security, with output vulnerabilities reduced by up to 48%. The resulting system
prompts transfer zero-shot to in-the-wild coding
agent prompts, without any exposure to real user
prompt distributions during optimization.
Read the full abstract ▾
All authors Houjun Liu, Lisa Einstein, John Yang, Joachim Baumann, Duncan Eddy, Christopher D. Manning, Mykel Kochenderfer, Diyi Yang
Cryptography and Security
Kettle: Attested builds for verifiable software provenance
Amean Asad and André Arko
Kettle is an attested build system that produces cryptographically verifiable provenance for software built inside Trusted Execution Environments (TEEs). A Kettle build records the source commit, dependency set, toolchain, build environment, and output artifact digests in a provenance document produced inside a measured confidential VM. The SHA-256 digest of that document is committed to the TEE platform's attestation report-data field, so the hardware-signed attestation report is itself the signature on the provenance, with the signing identity chaining to the TEE manufacturer's root of trust rather than to the build infrastructure operator. Because the CVM image is itself reproducible, its launch measurement is public and stable, which lets a build requester pre-attest the CVM before submitting any input and optionally deliver source over a TLS channel terminated inside it, so the build runs end-to-end confidentially without the host ever seeing source code in plaintext. Verification reduces to one signature check against the vendor root and a small set of digest comparisons, with no need to re-execute the build. The result removes the build infrastructure, its operators, and the artifact distribution channel from the trust surface a verifier must accept when deciding whether a binary corresponds to its claimed inputs.
Read the full abstract ▾
Cryptography and Security · Cross-listed from cs.LG
Private Vertical Federated Inference for Time-Series
Lucas Fenaux, Larris Xie, Aditya Bang et al. (+3 more)
Institutions may benefit from collaborative inference on time-series data. In settings where
privacy is necessary, multi-party computation (MPC) is a straightforward approach to providing strong guarantees, yet it remains prohibitively expensive and scales poorly with modern
transformer architectures. Vertical
Federated Learning (VFL) offers efficiency but suffers from
privacy leakage at the
embedding level, and securing the entire VFL model head via MPC remains prohibitively slow and communication-heavy for larger models. To enable practical, secure inference at scale, we propose "Public/Private Hybrid Head-VFL" (PPHH-VFL). This hybrid architecture splits the model head into an efficient plaintext public head and a secure, lightweight MPC private head. By applying
adversarial training to the public
embeddings, we mitigate
privacy leakage; concurrently, the small private head securely preserves the flow of sensitive information needed for high downstream utility. Empirical evaluations on models ranging up to 86 million parameters demonstrate that PPHH-VFL accelerates inference by up to six orders of magnitude compared to end-to-end MPC. Compared to a standard VFL+MPC baseline, our approach scales significantly better, achieving a speedup of up to 44.4x in WAN and a 91.2x reduction in communication costs (dropping from 1.7 GB to 19 MB per batch), while simultaneously improving downstream classification accuracy by 2.50% and regression RMSE by 40.7%.
Read the full abstract ▾
All authors Lucas Fenaux, Larris Xie, Aditya Bang, Alex Zhang, Kevin Wilson, Florian Kerschbaum
Cryptography and Security
AI-Driven Security Alert Screening and Alert Fatigue Mitigation in Security Operations Centers: A Survey
Samuel Ndichu, Tao Ban, Seiichi Ozawa et al. (+2 more)
Security alert screening is the downstream task of filtering, prioritizing, correlating, and contextualizing alerts for analyst
attention in Security Operations Centers. This survey reviews artificial-intelligence-driven alert screening and alert-fatigue mitigation from 2015 to 2026. We synthesize 119 records, including 87 core studies, into a four-stage workflow taxonomy covering filtering, triage, correlation, and generative augmentation. We find persistent gaps in operational validation,
adversarial robustness, cross-environment generalization, and evaluation practice. The survey concludes with a research agenda toward trustworthy Cognitive Security Operations Centers.
Read the full abstract ▾
All authors Samuel Ndichu, Tao Ban, Seiichi Ozawa, Takeshi Takahashi, Daisuke Inoue
Cryptography and Security
Seed Hijacking of LLM Sampling and Quantum Random Number Defense
Ziyang You, Xiaoke Yang, Zhanling Fan et al. (+3 more)
Large
language models (LLMs) rely on deterministic pseudorandom number generators (PRNGs) for autoregressive sampling, creating a critical supply-chain attack surface overlooked by existing defenses. We present SeedHijack, a
backdoor attack that manipulates PRNG outputs to force attacker-specified
token selection without altering model logits. In a 540-trial
benchmark on GPT-2 (124M), the attack achieves 99.6% exact
token injection across 9 sampling configurations; it reaches 100% success on four aligned models (1.5B-7B, RLHF/SFT/
reasoning distillation) and bypasses all
alignment methods tested in this work. We further propose a defense based on a hardware
quantum random number generator (QRNG), which neutralizes the attack in our evaluated threat model with negligible median overhead (+0.6% latency, +7.7 MB
memory). Our work identifies a critical sampling-layer vulnerability and provides a practical, deployable QRNG-based defense.
Read the full abstract ▾
All authors Ziyang You, Xiaoke Yang, Zhanling Fan, Feng Guo, Xiaogen Zhou, Xuxing Lu
Cryptography and Security
WebTrap: Stealthy Mid-Task Hijacking of Browser Agents During Navigation
Zhichao Liu, Wenbo Pan, Haining Yu et al. (+3 more)
Browser
agents are increasingly deployed in long-horizon tasks, which require executing extended action chains to accomplish user goals. However, this prolonged execution process provides attackers with more opportunities to inject malicious instructions. Existing
prompt injection attacks against browser
agents expose two key gaps: (1) low effectiveness, as attacks optimized for toy baselines fail to achieve end-to-end goals in real-world scenarios with complex environments and longer steps; (2) weak stealthiness, since most attacks pit the attack goal against the user goal, causing a significant drop in system usability under attack. To address these gaps, we propose WebTrap, a mid-task hijacking injection attack. It employs multi-step instruction fusion steering to seamlessly combine both goals, enabling the
agent to resume the original user task after executing the attack goal. Furthermore, we design a context-grounded generation method to align the injected content with the task environment and system instructions, maximizing the hijacking success rate. Extensive experiments on two browser
agent tasks, based on extended WASP and InjecAgent environments, demonstrate that our method achieves a high attack success rate while preserving the usability of the original system. We find that WebTrap exploits the
agent's
navigation vulnerabilities, binding the two goals so tightly that standard defense mechanisms cannot restore the system to normal operation. These findings reveal a critical vulnerability in
agent systems during long-horizon tasks that they can be stealthily hijacked.
Read the full abstract ▾
All authors Zhichao Liu, Wenbo Pan, Haining Yu, Ge Gao, Tianqing Zhu, Xiaohua Jia
Cryptography and Security · Cross-listed from cs.LG
UMEDA: Unified Multi-modal Efficient Data Fusion for Privacy-Preserving Graph Federated Learning via Spectral-Gated Attention and Diffusion-Based Operator Alignment
Shih-Yu Lai, Hirozumi Yamaguchi, Shang-Tse Chen et al. (+2 more)
Device-free localization trains models from heterogeneous wireless and visual sensors (e.g., Wi-Fi,
LiDAR) distributed across edge devices.
Federated learning offers a
privacy-respecting framework, but is brittle when clients differ in sensor modality and resolution, when their data distributions drift, and when
privacy noise destroys the structural signal needed for localization. We propose UMEDA, a graph
federated learning framework in which clients form nodes of a global graph that share a continuous integral operator, and aggregation is reformulated as spectral signal processing on this operator. Each client encodes its local sensors with a linear-
attention layer whose kernel spectrum is low-rank filtered, suppressing modality-specific residuals so clients with different sensors align in a common low-rank subspace. The server then aggregates client updates via a
diffusion model over the kernel's spectral coefficients, treating updates as discretizations of a shared operator rather than
topology-bound weights – this absorbs varying graph sizes and missing modalities without node-wise correspondence. To balance
privacy and utility, we add an anisotropic differential-
privacy mechanism that projects noise preferentially into the null space of the signal subspace, preserving dominant eigendirections while ensuring formal $(\epsilon, \delta)$-DP under gradient clipping. On MM-Fi and the RELI11D out-of-distribution
benchmark, UMEDA outperforms state-of-the-art
federated baselines in accuracy, convergence, and communication efficiency, particularly under high modality heterogeneity and tight
privacy budgets.
Read the full abstract ▾
All authors Shih-Yu Lai, Hirozumi Yamaguchi, Shang-Tse Chen, Yu-Lun Liu, Bing-Yu Chen
Cryptography and Security · Cross-listed from cs.LG
Trapping Attacker in Dilemma: Examining Internal Correlations and External Influences of Trigger for Defending GNN Backdoors
Fan Yang, Binyan Xu, Di Tang et al. (+1 more)
GNNs have become a standard tool for learning on relational data, yet they remain highly vulnerable to
backdoor attacks. Prior defenses often depend on inspecting specific subgraph patterns or node features, and thus can be circumvented by adaptive attackers. We propose PRAETORIAN, a new defense that targets intrinsic requirements of effective GNN
backdoors rather than surface-level cues. Our key observation is that flipping a victim node's prediction requires substantial influence on the victim: attackers tend to either inject many trigger nodes or rely on a small set of highly influential ones. Building on this observation, PRAETORIAN (i) analyzes internal correlations within potential trigger subgraphs to detect abnormally large injected structures, and (ii) quantifies external node influence to identify triggers with disproportionate impact. Across our evaluations, PRAETORIAN reduces the average attack success rate (ASR) to 0.55% with only a 0.62% drop in clean accuracy (CA), whereas state-of-the-art defenses still yield an average ASR of >20% and a CA drop of >3% under the same conditions. Moreover, PRAETORIAN remains effective against a range of adaptive attacks, forcing adversaries to either inject many trigger nodes to achieve high ASR (>80%), which incurs a >10% CA drop, or preserve CA at the cost of limiting ASR to 18.1%. Overall, PRAETORIAN constrains attackers to an unfavorable trade-off between efficacy and detectability.
Read the full abstract ▾
All authors Fan Yang, Binyan Xu, Di Tang, Kehuan Zhang
Cryptography and Security
Mitigating Many-shot Jailbreak Attacks with One Single Demonstration
Kejia Chen, Jiawen Zhang, Boheng Li et al. (+6 more)
Many-shot jailbreaking (MSJ) causes safety-aligned
language models to answer harmful queries by preceding them with many harmful question-answer demonstrations. We study why this attack becomes stronger as the number of demonstrations increases. Empirically, we find that MSJ induces a progressive activation drift: the representation of a fixed harmful query moves step by step away from the safety-aligned region as more harmful demonstrations are added. Theoretically, we show that this drift can be interpreted as implicit malicious fine-tuning: conditioning on N harmful demonstrations induces SGD-style updates equivalent to optimizing on the corresponding N harmful samples. This view turns the attack mechanism into a defense principle. We append a fixed one-shot safety demonstration at inference time, which induces a counteracting safety-oriented update and restores refusal behavior. The resulting method improves the model's robustness to MSJ without modifying its parameters or requiring white-box access at deployment. Code is available at https://github.com/Thecommonirin/SafeEnd.
Read the full abstract ▾
All authors Kejia Chen, Jiawen Zhang, Boheng Li, Pengcheng Li, Jian Lou, Zunlei Feng, Mingli Song, Ruoxi Jia, Tianwei Zhang
Cryptography and Security · Cross-listed from cs.CL
GLiGuard: Schema-Conditioned Classification for LLM Safeguard
Urchade Zaratiana, Mary Newhauser, George Hurn-Maloney et al. (+1 more)
Ensuring safe, policy-compliant outputs from large
language models requires real-time content moderation that can scale across multiple safety dimensions. However, state-of-the-art guardrail models rely on autoregressive decoders with 7B–27B parameters, reformulating what is fundamentally a classification problem as sequential text generation, a design choice that incurs high latency and scales poorly to multi-aspect evaluation. In this work, we introduce GLiGuard, a 0.3B-parameter schema-conditioned bidirectional encoder adapted from GLiNER2 for LLM content moderation. The key idea is to encode task definitions and label semantics directly into the input sequence as structured
token schemas, enabling simultaneous evaluation of
prompt safety, response safety, refusal detection, 14 fine-grained harm categories, and 11 jailbreak strategies in a single non-autoregressive forward pass. This schema-conditioned design lets supported task and label blocks be composed directly in the input schema at inference time. Across nine established safety
benchmarks, GLiGuard achieves F1 scores competitive with 7B–27B decoder-based guards despite being 23–90$\times$ smaller, while delivering up to 16$\times$ higher throughput and 17$\times$ lower latency. These results suggest that compact bidirectional encoders can approach the accuracy of much larger guard models while drastically reducing inference cost. Code and models are available at https://github.com/fastino-ai/GLiGuard.
Read the full abstract ▾
All authors Urchade Zaratiana, Mary Newhauser, George Hurn-Maloney, Ash Lewis
Cryptography and Security · Cross-listed from cs.LG
Graph Representation Learning Augmented Model Manipulation on Federated Fine-Tuning of LLMs
Hanlin Cai, Kai Li, Houtianfu Wang et al. (+4 more)
Federated fine-tuning (FFT) has emerged as a
privacy-preserving paradigm for collaboratively adapting large
language models (LLMs). Built upon
federated learning, FFT enables distributed
agents to jointly refine a shared pretrained LLM by aggregating local LLM updates without sharing local raw data. However, FFT-based LLMs remain vulnerable to model manipulation threats, in which
adversarial participants upload manipulated LLM updates that corrupt the aggregation process and degrade the performance of the global LLM. In this paper, we propose an Augmented Model maniPulation (AugMP) strategy against FFT-based LLMs. Specifically, we design a novel graph representation learning framework that captures feature correlations among benign LLM updates to guide the generation of malicious updates. To enhance manipulation effectiveness and stealthiness, we develop an iterative manipulation
algorithm based on an augmented Lagrangian dual formulation. Through this formulation, malicious updates are optimized to embed
adversarial objectives while preserving benign-like parameter characteristics. Experimental results across multiple LLM backbones demonstrate that the AugMP strategy achieves the strongest manipulation performance among all competing baselines, reducing the global LLM accuracy by up to \(26\%\) and degrading the average accuracy of local LLM
agents by up to \(22\%\). Meanwhile, AugMP maintains high statistical and
geometric consistency with benign updates, enabling it to evade conventional distance- and similarity-based defense methods.
Read the full abstract ▾
All authors Hanlin Cai, Kai Li, Houtianfu Wang, Haofan Dong, Yichen Li, Falko Dressler, Ozgur B. Akan