Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2609.37217

When Cyber Scoring Systems Diverge: An Empirical Comparison

Kirsi Hellsten·Joni Herttuainen·Ambrose Kam·Arlanda Johnson·David Welsh·Kimmo Kaski
View original source ↗Read source document ↗Export metadata ↓

Abstract

Vulnerability scoring systems underpin cyber patch prioritization and risk management, but their comparative behavior is almost always assessed in the abstract, through correlation studies in IT vulnerability databases, rather than by the operational consequences they produce when embedded in a system-level risk model. Here we present an empirical comparison of four vulnerability scoring systems, namely CVSS (Common Vulnerability Scoring System), EPSS (Exploit Prediction Scoring System), SSVC (Stakeholder-Specific-Vulnerability Categorization), and IronMiner (operationally calibrated proprietary scoring system). As a substrate for comparison, we use a reconstruction of the 2015 Ukraine Power Grid operational-technology (OT) network that provides a documented incident topology. The results show a high degree of disagreement between the scoring systems. This suggests that the choice of the scoring system could significantly influence mitigation strategies and vulnerability prioritization, implying that a composite or hybrid scoring approach could offer a more suitable solution.

Explore related subjects

cs.CR

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Kirsi Hellsten, Joni Herttuainen, Ambrose Kam, Arlanda Johnson, David Welsh, Kimmo Kaski. 2026-09-29. When Cyber Scoring Systems Diverge: An Empirical Comparison. https://arxiv.org/abs/2609.37217

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Context-Aware Spear Phishing: Generative AI-Enabled Attacks Against Individuals via Public Social Media Data

We demonstrate how publicly available social-media data and generative AI (GenAI) can be misused to automate and scale highly personalized, context-aware spear-phishing campaigns. With minimal attacker effort, a small amount of public activity per target is sufficient for GenAI models to extract interests and contextual cues, producing persuasive messages that mirror a target's style while bypassing generic content-moderation safeguards. We introduce a modular framework that combines multimodal signal extraction, communication-style profiling, and attack-type instantiation across seven strategies (baiting, scareware, honey trap, tailgating, impersonation, quid pro quo, and personalized emotional exploitation). We conduct a large-scale, multi-model evaluation covering thousands of generated emails and eight security-relevant criteria, benchmarking against a corpus of real-world phishing messages. The GenAI-produced emails exhibit markedly higher personalization, contextual grounding, and persuasive leverage. Importantly, a complementary user study corroborates these results, revealing that LLM-generated attacks consistently outperform APWG eCrimeX emails across eight dimensions while eliciting lower suspicion among human recipients. Finally, we measure and analyze the behavior of existing proactive, prompt-level defense mechanisms, which incorporate adaptive mechanisms, as well as two complementary defense approaches-policy-augmented SOTA safeguard models and system-instruction chain-of-thought moderation. We document how these defenses respond to contextualized and adaptive attack prompts, underscoring the need for platform-level safeguards that explicitly account for contextualized abuse at scale.

cs.CR↗
arXiv2026

Defenses at Odds: Measuring and Explaining Defense Conflicts in Large Language Models

Large language models (LLMs) may require additional defenses after deployment as risks and governance requirements evolve. Subsequent defenses can interact with earlier defenses, raising the question of their sequential compatibility. We study this question with CONFLICTEVAL, evaluating 144 ordered compositions of six defenses spanning safety, privacy, and fairness across six models. The resulting interactions are heterogeneous across defense pairs, application orders, and models. Notably, some compositions exhibit defense conflicts: the subsequent defense improves its target objective while weakening protection established by the earlier defense. We investigate these interactions through the directional compatibility of defense-induced changes in risk-relevant representations. Across 11 selected cases, we use activation interventions to assess which defense-induced representational changes support protection and examine how subsequent defenses affect these changes. We find that, in some conflict cases, subsequent defenses counteract representational changes supporting earlier protection, providing evidence for one possible pathway to defense conflicts. Building on this analysis, we propose Conflict-Triggered Directional Retention (CTDR), which penalizes opposing shifts along directions supporting earlier protection. On six selected conflicting compositions from this analysis, CTDR reduces first-objective regression while maintaining positive gains on the subsequent defense objective.

cs.CR↗
arXiv2026

AI Security Research Should Better Incentivize Defense Research

This work examines an imbalance in artificial intelligence (AI) security research: the field tends to produce more work on attacking AI systems than on defending them. Drawing on related academic papers, we find biased attack-to-defense ratios across subfields, including federated learning, speech recognition, membership inference, large language models, etc. The imbalance possibly means far beyond a simple count: attack papers are routinely evaluated under favorable conditions that make threats look more severe than they are in practice, while defenses are held to a stricter standard that few can meet. The result is a literature rich in demonstrated vulnerabilities and thin on usable and deployed protections. We thus argue that AI security research should better incentivize defense research.

cs.CR↗