Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2604.17763

A Quasi-Experimental Developer Study of Security Training in LLM-Assisted Web Application Development

Mohammed Kharma·Ahmed Sabbah·Radi Jarrar·Samer Zain·Mohammad Alkhanafseh·David Mohaisen
View original source ↗Read source document ↗Export metadata ↓

Abstract

This paper presents a controlled quasi-experimental developer study examining whether a layer-based security training package is associated with improved security quality in LLM-assisted implementation of an identity-centric Java Spring Boot backend. The study uses a mixed design with a within-subject pre-training versus post-training comparison and an exploratory between-subject expertise factor. Twelve developers completed matched runs under a common interface, fixed model configuration, counterbalanced task sets, and a shared starter project. Security outcomes were assessed via independent manual validation of submitted repositories by the first and second authors. The primary participant-level endpoint was a severity-weighted validated-weakness score. The post-training condition showed a significant paired reduction under an exact Wilcoxon signed-rank test ($p = 0.0059$). In aggregate, validated weaknesses decreased from 162 to 111 (31.5\%), the severity-weighted burden decreased from 432 to 267 (38.2\%), and critical findings decreased from 24 to 5 (79.2\%). The largest reductions were in authorization and object access (53.3\%) and in authentication, credential policy, and recovery weaknesses (44.7\%). Session and browser trust-boundary issues showed minimal change, while sensitive-data and cryptographic weaknesses showed only marginal improvement. These results suggest that, under the tested conditions, post-training runs reduce validated security burden in LLM-assisted backend development without modifying the model. They do not support replacing secure defaults, static analysis, expert review, or operational hardening.

Explore related subjects

cs.CRcs.LG

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Mohammed Kharma, Ahmed Sabbah, Radi Jarrar, Samer Zain, Mohammad Alkhanafseh, David Mohaisen. 2026-04-20. A Quasi-Experimental Developer Study of Security Training in LLM-Assisted Web Application Development. https://arxiv.org/abs/2604.17763

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Spoofing Missed-Detection Bounds for PRF GNSS Ranging Authentication Under AWGN Models

Pseudorandom-function (PRF) ranging codes, such as those used in Galileo's encrypted E6-C under the Signal Authentication Service (SAS), enable a receiver to authenticate pseudoranges once the PRF secret is revealed. This work bounds how much authentication security the receiver obtains under Additive White Gaussian Noise (AWGN) assumptions. Against a spoofer that does not estimate the code before submitting its forgery, PRF security makes the forged correlation zero-mean up to the security of the underlying PRF, allowing integration time and C/N$_0$ to mostly determine probability of missed detection (PMD) and probability of false alarm (PFA). Against such a spoofer at a conservative 30 dB-Hz, 400 ms of E6-C aggregation certifies a PMD below $2^{-128}$ (plus any PRF advantage). For a spoofer that estimates chips before submitting a forgery, I derive the receiving-antenna gain at which authentication security breaks, which is about 12 dB for E6-C for the adversaries modeled. This work can be used to design a PRF GNSS ranging code protocol and a receiver capable of correctly asserting PRF ranging security assuming an AWGN model.

cs.CR↗
arXiv2026

First Attack, Final Offensive: The Dark Forest on an Open Roster

The Dark Forest argument holds that a civilization that detects another should strike it at once. Existing formal models make the detected civilization the object of the strike and play it on a roster the attacker knows to be complete. This paper changes both choices. The object of hostility is remaining uncontrolled capacity to retaliate or to warn someone who can, and the roster is open: no attacker ever knows it has met everyone. A first strike is then rational only if the timing benefit of what it removes now rather than later is at least the disclosure loss from every survivor that learns of it. A survivor that can bring about the attacker's destruction enters that loss as a lump, not a per-unit rate, and the actors the attacker has never found may be such a survivor, one that no strike removes. Their capacity cannot be estimated, but what they can do is capped at the attacker's destruction, so the test against them asks one answerable question: a first strike is rational only if the attacker accepts that the strike may be its last attack. The Dark Forest premises, read as hypotheses, fix what a general attacker cannot estimate: hidden hunters exist, a hunter that verifies a hostile acts against it with probability at least $q$, and a hider is rarely found, so a believer's first strike is rational only if what it removes is worth a $q$-share of its survival, the whole of it as $q$ approaches one. With survival as the payoff, the profile in which every hunter strikes what it finds is not a Nash equilibrium whenever a strike is more visible to unfound hunters than a hider is findable, while the profile in which every hunter hides is. That visibility comparison is the decisive physical question; an attacker that treats its strike as unseen has assumed the roster closed.

cs.CR↗
arXiv2026

From Capability to Assurance in Autonomous Penetration-Testing Harnesses: A Framework and Reference Implementation

Research on large language model agents for penetration testing is evaluated almost entirely by capability: whether the agent captures a flag or reproduces a proof of concept. That metric suits a benchmark but is silent on the properties that decide whether an autonomous agent can be used in an authorized engagement: whether a reported finding is true, whether the agent stayed inside its authorized scope, and whether an operator can audit what it did. We call these assurance properties and argue that they belong to the harness, the runtime wrapping the model, and can be enforced in code. This paper makes three contributions. First, we define a framework of five assurance properties (evidence grounding, non destructive claim reduction, computed severity, enforced authorization, and tamper evident accountability), each with a formal model and an explicit acceptance test, connected to prior work in capability based security, tamper evident logging, and software provenance. Second, we position representative systems (PentestGPT, the Cochise reference harness, MAPTA, and the trajectory judge PentestJudge) within the framework using published coding criteria, and identify a consistent assurance gap. Third, we study one open source implementation, NeuroSploit, pinned to an exact commit, reporting its architecture, its complexity cost, and a content addressed artifact bundle from a run against a public deliberately vulnerable target. We execute the deterministic authorization and audit acceptance tests directly and find and report a real enforcement gap, which we reflect by scoring both properties as partial. We therefore claim an initial existence argument that the properties are realizable together, not a comparative performance result, and we specify the multi target, ablation, and adversarial evaluation protocol required to turn the framework obligations into measurements.

cs.CR↗