Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2609.36941

Practical Secrets Extraction against Black-box LLMs

Shiqian Zhao·Siwei Jiang·Xinfeng Li·Runyi Hu·Yandan Zheng·Congyu Guo·Tianwei Zhang·Anh Tuan Luu
View original source ↗Read source document ↗Export metadata ↓

Abstract

Large language models (LLMs) increasingly power autonomous coding agents such as Codex and Claude Code, yet their training corpora may contain confidential credentials exposed in public repositories or collected from private development artifacts, creating risks of memorization and subsequent leakage. Existing extraction audits, however, largely assume access to model weights or token probabilities. In this work, we present a black-box secret extraction framework for commercial, API-based LLMs under output-only access. It comprises (i) \emph{Cross-Validated Secret Knowledge Distillation}, which uses semantics-preserving prompt variants, response cross-validation, and provider-specific format filtering to distill secret-relevant behavior into a local white-box proxy; and (ii) \emph{Proxy-Guided Secret Extraction and Candidate Filtering}, which combines truncated top-$p$ sampling with local token entropy, $N$-gram frequency profiling, and provider-specific structural priors. On controlled API-key benchmarks, our framework improves recovery effectiveness and real-key rates over representative baselines while reducing extraction latency. A responsible real-world evaluation further recovers masked provider-specific credentials from three independently deployed black-box LLM systems spanning OpenAI and Claude Code, showing that memorized secrets can be exposed under output-only access.

Explore related subjects

cs.CR

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Shiqian Zhao, Siwei Jiang, Xinfeng Li, Runyi Hu, Yandan Zheng, Congyu Guo, Tianwei Zhang, Anh Tuan Luu. 2026-09-29. Practical Secrets Extraction against Black-box LLMs. https://arxiv.org/abs/2609.36941

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

BadRAG: Identifying Vulnerabilities in Retrieval Augmented Generation of Large Language Models

Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) by retrieving relevant information from external knowledge bases to provide more accurate, contextually informed, and up-to-date responses. However, this reliance on external knowledge introduces significant security vulnerabilities, as many RAG systems (e.g., Google Search) rely on large and unsanitized data repositories (e.g., Reddit). In this paper, we unveil a novel threat in which attackers steer the RAG system's response by injecting malicious passages into its knowledge base. When a user's query contains attacker-specified trigger words, the RAG retrieves and refers to these malicious passages, enabling the attacker to steer the response without altering the user input or modifying the RAG weights. BadRAG operates in two phases: (i) malicious passages are optimized to be retrieved exclusively when trigger words appear in user queries; (ii) these passages are meticulously crafted to achieve adversarial generation objectives, including denial of service, sentiment manipulation, context leakage, and tool misuse. Our experiments show that injecting just 10 malicious passages (0.04\% of the external corpora) achieves a 98.2\% retrieval success rate and increases negative response rates from 0.22\% to 72\% for queries containing triggers.

cs.CR↗
arXiv2026

Adversarial Defense in Cybersecurity: A Systematic Review of GANs for Threat Detection and Mitigation

Machine learning-based cybersecurity systems are highly vulnerable to adversarial attacks, while Generative Adversarial Networks (GANs) act as both powerful attack enablers and promising defenses. This survey systematically reviews GAN-based adversarial defenses in cybersecurity (2021--August 31, 2025), consolidating recent progress, identifying gaps, and outlining future directions. Using a PRISMA-compliant systematic literature review protocol, we searched five major digital libraries. From 829 initial records, 185 peer-reviewed studies were retained and synthesized through quantitative trend analysis and thematic taxonomy development. We introduce a four-dimensional taxonomy spanning defensive function, GAN architecture, cybersecurity domain, and adversarial threat model. GANs improve detection accuracy, robustness, and data utility across network intrusion detection, malware analysis, and IoT security. Notable advances include WGAN-GP for stable training, CGANs for targeted synthesis, and hybrid GAN models for improved resilience. Yet, persistent challenges remain such as instability in training, lack of standardized benchmarks, high computational cost, and limited explainability. GAN-based defenses demonstrate strong potential but require advances in stable architectures, benchmarking, transparency, and deployment. We propose a roadmap emphasizing hybrid models, unified evaluation, real-world integration, and defenses against emerging threats such as LLM-driven cyberattacks. This survey establishes the foundation for scalable, trustworthy, and adaptive GAN-powered defenses.

cs.CR↗
arXiv2026

NonTextual Target Attack

Existing gradient-based jailbreak attacks on Large Language Models (LLMs) typically optimize adversarial suffixes to align the LLM output with predefined target responses. However, restricting the objective as inducing fixed targets inherently constrains the adversarial search space, limiting the overall attack efficacy. Furthermore, existing methods typically require numerous optimization iterations to fulfill the large gap between the fixed target and the original LLM output, resulting in low attack efficiency. To overcome these limitations, we propose NonTextual Target Attack (NTA),the first gradient-based attack that relies on a non-textual constrained objective to maximize the unsafety probability of the LLM output, without enforcing any response patterns. For tractable optimization, we further decompose this objective into two constrained sub-objectives, which can be approximated by two differentiable unconstrained losses, to iteratively optimize the response and the adversarial prompt in the neighborhood of the original prompt, with a theoretical analysis to validate the decomposition. In contrast to existing attacks, NTA first realizes gradient-based prompt optimization on a non-textual target and significantly expands the attack space, enabling more flexible and efficient exploration of LLM vulnerabilities. Extensive evaluations show that NTA achieves an average attack success rate of 96.8% against recent safety-aligned LLMs with only 100 optimization iterations on AdvBench, outperforming state-of-the-art gradient-based attacks by over 40%.

cs.CR↗