Beyond the Agent Harness: Boundary Criteria for Software-Factory Orchestration - A Single-Operator Case Study

Abstract

Several recent systems add a layer above the coding-agent harness that organises many agent runs into continuous delivery. The existence of such a layer is not our contribution. What is missing is a rule for deciding which layer a mechanism or a failure belongs to. We propose boundary criteria based on an operational definition of a run. A mechanism confined to one run belongs to the agent harness. One that coordinates runs, hands state from one run to a later one, or is updated from run outcomes belongs to a factory harness. One that must change with the application domain belongs to the domain layer. Two residual tests assign infrastructure and model causes. For failures, we code where the defect originated, which control failed to stop it, and where it was fixed as three separate fields, and we require positive evidence before attributing a failure to the environment. We developed the criteria on documented failures from a single operator's pipeline serving six projects and on 26 components of three published systems. We then froze them and had two model coders of the same family apply them to the same descriptions of 46 held-out units. They agreed on all 26 mechanism layers and on 18-19 of 20 incident fields (kappa 0.86-0.93), but both coders recorded the incident as undetermined or unrecorded in 8-13 of the 20 cases per field, and each flagged half of all units as ambiguous, mostly for missing information. The check exposed conflicts in the rules; a revision that separates mechanism classification from incident attribution resolves them but, in a sensitivity re-coding, agreed less (kappa 0.78-0.93). Agreement here is neither accuracy nor usefulness, and human coders have not yet been tested. Case evidence and seven hypotheses are reported as preliminary.