Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.17020

CWE-bench: Refocusing Agentic Security from Exploitation to Defense

Gonzalo Gonzalez-Pumariega·Ajay Jagannath·Adit Jain·Guram Gogia·Soumyadeep Bakshi·Nazneen Rajani
View original source ↗Read source document ↗Export metadata ↓

Abstract

Cybersecurity evaluations of AI agents have focused largely on offense, yet a working exploit does not show that an agent can make software safe. An attacker needs to exploit only one vulnerability; a defender must find and fix all of them. We introduce CWE-bench, a private benchmark of 120 held-out repository audit-and-repair tasks spanning 73 CWE types, eight languages, and all OWASP Top 10 2025 categories. Agents are given a repository to audit and a description of its threat model, but are not told the weakness, where it occurs, or how many vulnerable sites there are. Each patch must block the exploit while preserving original functionality, as checked by a programmatic verifier. We validated the verifier against a panel of agentic judges, manually reviewing disagreements during development and refining the checks. The best agents reach 68% pass@1 and 81% pass@4. We further conduct analyses showing that (i) agents with the same pass@1 differ widely in retry gains and cost, and (ii) no single agent is best across languages or vulnerability categories, and difficulty varies far more by vulnerability category than by language. As the offensive capabilities of agents improve, CWE-bench provides a broad and rigorous measure of whether their defensive capabilities keep pace.

Explore related subjects

cs.CRcs.AIcs.SE

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Gonzalo Gonzalez-Pumariega, Ajay Jagannath, Adit Jain, Guram Gogia, Soumyadeep Bakshi, Nazneen Rajani. 2026-10-12. CWE-bench: Refocusing Agentic Security from Exploitation to Defense. https://arxiv.org/abs/2610.17020

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Simulation-Secure Functional Encryption in the Bounded Storage Model

Functional encryption (FE) is a versatile paradigm that enables fine-grained access control over encrypted data. Despite its potential, achieving the gold standard of simulation-based security for FE is impossible in full generality. Known impossibility results demonstrate that simulation security cannot be attained if an adversary in the security experiment is permitted either an unbounded number of functional key queries or an unbounded number of challenge ciphertexts. In this work, we circumvent these fundamental barriers by considering two distinct memory-restricted settings: the Bounded Quantum Storage Model and the Bounded Classical Storage Model. In these settings, the plain model impossibility results no longer apply, allowing us to obtain new positive results. Specifically, we construct two adaptively simulation-secure FE schemes in the Bounded Quantum Storage Model: 1) Many functional key scheme: A construction supporting many functional key queries and a single challenge ciphertext, assuming only the existence of one-way functions. 2) Many ciphertext scheme: An information-theoretic secure construction supporting a single non-adaptive functional key, many challenge ciphertexts, and many adaptive functional key queries. Furthermore, we demonstrate that both schemes can be ported to the Bounded Classical Storage Model, assuming the existence of disappearing grey-box obfuscation.

cs.CR↗
arXiv2026

Watermarking Continuous Tabular Data Generation

In this paper, we introduce a simple yet effective tabular data watermarking mechanism with statistical guarantees. We show theoretically that the proposed watermark can be effectively detected, while faithfully preserving the data fidelity, and also demonstrates appealing robustness against additive noise attack. The general idea is to achieve the watermarking through a strategic embedding based on simple data binning. Specifically, it divides the feature's value range into finely segmented intervals and embeds watermarks into selected ``green list" intervals. To detect the watermarks, we develop a principled statistical hypothesis-testing framework with minimal assumptions: it remains valid as long as the underlying data distribution has a continuous density function. The watermarking efficacy is demonstrated through rigorous theoretical analysis and empirical validation, highlighting its utility in enhancing the security of synthetic and real-world datasets.

cs.CR↗
arXiv2026

MIRANDA: short signatures from a leakage-free full-domain-hash scheme

We present $\mathsf{Miranda}$, the first family of full-domain-hash signatures based on matrix codes. This signature scheme fulfils the paradigm of Gentry, Peikert and Vaikuntanathan ($\mathsf{GPV}$), which gives strong security guarantees. Our trapdoor is very simple and generic: if we propose it with matrix codes, it can actually be instantiated in many other ways since it only involves a subcode of a decodable code (or lattice) in a unique decoding regime of parameters. Though $\mathsf{Miranda}$ signing algorithm relies on a decoding task where there is exactly one solution, there are many possible signatures given a message to sign and we ensure that signatures are not leaking information on their underlying trapdoor by means of a very simple procedure involving the drawing of a small number of uniform bits. In particular $\mathsf{Miranda}$ does not use a rejection sampling procedure which makes its implementation a very simple task contrary to other $\mathsf{GPV}$-like signatures schemes such as $\mathsf{Falcon}$ or even $\mathsf{Wave}$. We instantiate $\mathsf{Miranda}$ with the famous family of Gabidulin codes represented as spaces of matrices and we study thoroughly its security (in the EUF-CMA security model). For~$128$ bits of classical security, the signature sizes are as low as~$90$ bytes and the public key sizes are in the order of~$2.6$ megabytes.

cs.CR↗