Correction and Corruption: A Two-Rate View of Error Flow in LLM Protocols

Abstract

Large language models operate in protocols containing multiple calls, yet added calls are usually evaluated only by their net effect. That summary cannot distinguish correcting unsuccessful outputs from corrupting initially successful ones. We develop a paired audit recording success before and after a specified operation on the same tasks under one binary rule. Correction and corruption rates exactly account for the net change: gains come from corrected failures and losses from corrupted successes. The break-even correction requirement rises sharply with baseline success, so the same behavior can improve a moderate-baseline population but harm a high-baseline one. Applied to published GPT-4 GSM8K results, the framework bounds within-sample correction and corruption counts from aggregate accuracies. We ask if calibration estimates predict unobserved outcomes, how rates change with information supplied to an operation, and whether successive measurements combine. Calibration estimates track accuracy on a disjoint sample from the same generator. Under reweighting of generator-defined groups, pooled estimates can fail while group-specific estimates reduce average prediction error. On GSM8K, observable features capture limited variation at these sample sizes, so the group-level application rule remains exploratory. Reordering a fixed four-candidate set changes both rates, with the accuracy effect depending on whether a correct alternative is available; on MBPP, adding a helper artifact raises corruptions among 357 initially passing programs from 28 to 100. Direct and composed transition estimates are close on average within samples; weighting initial-success rows by support substantially reduces held-out discrepancy. Correction and corruption are measurements of specified operations, not model constants, and can guide input choices, application decisions, and composition tests.