Cybersecurity AI: The World's Top AI Agent for Security Capture-the-Flag (CTF)

Víctor Mayoral-Vilches; Luis Javier Navarrete-Lozano; Francesco Balassone; María Sanz-Gómez; Cristóbal R. J. Veas Chavez; Maite del Mundo de Torres; Vanesa Turiel

arXiv · 2512.02654

Cybersecurity AI: The World's Top AI Agent for Security Capture-the-Flag (CTF)

Víctor Mayoral-Vilches·Luis Javier Navarrete-Lozano·Francesco Balassone·María Sanz-Gómez·Cristóbal R. J. Veas Chavez·Maite del Mundo de Torres·Vanesa Turiel

View original source ↗Read source document ↗Export metadata ↓

Abstract

Are Capture-the-Flag competitions obsolete? In 2025, Cybersecurity AI (CAI) systematically conquered some of the world's most prestigious hacking competitions, achieving Rank #1 at multiple events and consistently outperforming thousands of human teams. Across five major circuits-HTB's AI vs Humans, Cyber Apocalypse (8,129 teams), Dragos OT CTF, UWSP Pointer Overflow, and the Neurogrid CTF showdown-CAI demonstrated that Jeopardy-style CTFs have become a solved game for well-engineered AI agents. At Neurogrid, CAI captured 41/45 flags to claim the $50,000 top prize; at Dragos OT, it sprinted 37% faster to 10K points than elite human teams; even when deliberately paused mid-competition, it maintained top-tier rankings. Critically, CAI achieved this dominance through our specialized alias1 model architecture, which delivers enterprise-scale AI security operations at unprecedented cost efficiency and with augmented autonomy-reducing 1B token inference costs from $5,940 to just $119, making continuous security agent operation financially viable for the first time. These results force an uncomfortable reckoning: if autonomous agents now dominate competitions designed to identify top security talent at negligible cost, what are CTFs actually measuring? This paper presents comprehensive evidence of AI capability across the 2025 CTF circuit and argues that the security community must urgently transition from Jeopardy-style contests to Attack & Defense formats that genuinely test adaptive reasoning and resilience-capabilities that remain uniquely human, for now.

Explore related subjects

cs.CR

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Víctor Mayoral-Vilches, Luis Javier Navarrete-Lozano, Francesco Balassone, María Sanz-Gómez, Cristóbal R. J. Veas Chavez, Maite del Mundo de Torres, Vanesa Turiel. 2025-12-02. Cybersecurity AI: The World's Top AI Agent for Security Capture-the-Flag (CTF). https://arxiv.org/abs/2512.02654

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

UPPRESSO: Untraceable and Unlinkable Privacy-PREserving Single Sign-On Services

Single sign-on (SSO) allows a user to maintain only the credential for an identity provider (IdP) to log into multiple relying parties (RPs). However, SSO introduces privacy threats, as (a) a curious IdP could track a user's all visits to RPs, and (b) colluding RPs could learn a user's online profile by linking her identities across these RPs. This paper presents a privacypreserving SSO scheme, called UPPRESSO, to protect an honest user's online profile against (a) an honest-but-curious IdP and (b) malicious RPs colluding with other users. UPPRESSO proposes an identity-transformation approach to generate untraceable ephemeral pseudo-identities for an RP and a user from which the target RP derives a permanent account for the user, while the transformations also provide unlinkability. This approach protects the identities of the user and the target RPs in a login flow, while working compatibly with widely-deployed SSO protocols and providing services accessed from a commercial-off-the-shelf browser without plug-ins or extensions. We built a prototype of UPPRESSO on top of MITREid Connect, an open-source SSO system. The extensive evaluations show that it fulfills the security and privacy requirements of SSO with reasonable overheads.

cs.CR↗

arXiv2026

Attack Tree Distance: a practical examination of tree difference measurement within cyber security

Attack trees are a popular threat modeling method. In practice, there is often a need to compare attack tree models produced by human experts, based on both the structure of the tree and the meaning of the node labels. In this work, we investigate the problem of comparing attack trees and measuring their similarity. We define five different measures for measuring the distance between two attack trees: Label Distance (LD), Tree Edit Distance (TED), Radical Distance (RD), Multiset Distance (MSD) and Weighted Sum Distance (WSD). We further propose a repeatable method of both theoretical and experimental attack tree distance measures validation. Our theoretical validation consists of a series of basic transformations to evaluate the behavior of distance measures with respect to specific types of transformations that may appear between two attack trees. To experimentally validate our distance measures, we designed and executed a human study ($n=39$) to collect a dataset of attack trees to be used for evaluation and comparison of the measures. From our theoretical and experimental results, we find that applying semantic similarity as a means of comparing node labels is a valid approach. Further, we find four of the five attack tree distance measures are valid approaches in certain, varying circumstances. Our results suggest that these methods can already be used to identify similar real-world attack trees. Overall, this work lays the groundwork for improved threat model analysis, validation of AI-generated attack trees, and future research into threat similarity measurement in cybersecurity.

cs.CR↗

arXiv2026

Probabilistic Modeling of Jailbreak on Multimodal LLMs: From Quantification to Application

Recently, Multimodal Large Language Models (MLLMs) have demonstrated their superior ability in understanding multimodal content. However, they remain vulnerable to jailbreak attacks, which exploit weaknesses in their safety alignment to generate harmful responses. Previous studies categorize jailbreaks as successful or failed based on whether responses contain malicious content. However, given the stochastic nature of MLLM responses, this binary classification of an input's ability to jailbreak MLLMs is inappropriate. Derived from this viewpoint, we introduce jailbreak probability to quantify the jailbreak potential of an input, which represents the likelihood that MLLMs generated a malicious response when prompted with this input. We approximate this probability through multiple queries to MLLMs. After modeling the relationship between input hidden states and their corresponding jailbreak probability using Jailbreak Probability Prediction Network (JPPN), we use continuous jailbreak probability for optimization. Specifically, we propose Jailbreak-Probability-based Attack (JPA) that optimizes adversarial perturbations on input image to maximize jailbreak probability, and further enhance it as Multimodal JPA (MJPA) by including monotonic text rephrasing. To counteract attacks, we also propose Jailbreak-Probability-based Finetuning (JPF), which minimizes jailbreak probability through MLLM parameter updates. Extensive experiments show that (1) (M)JPA yields significant improvements when attacking a wide range of models under both white and black box settings. (2) JPF vastly reduces jailbreaks by at most over 60\%. Both of the above results demonstrate the significance of introducing jailbreak probability to make nuanced distinctions among input jailbreak abilities.

cs.CR↗