Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 1906.04711

ProPublica's COMPAS Data Revisited

Matias Barenstein
View original source ↗Read source document ↗Export metadata ↓

Abstract

I examine the COMPAS recidivism risk score and criminal history data collected by ProPublica in 2016 that fueled intense debate and research in the nascent field of 'algorithmic fairness'. ProPublica's COMPAS data is used in an increasing number of studies to test various definitions of algorithmic fairness. This paper takes a closer look at the actual datasets put together by ProPublica. In particular, the sub-datasets built to study the likelihood of recidivism within two years of a defendant's original COMPAS survey screening date. I take a new yet simple approach to visualize these data, by analyzing the distribution of defendants across COMPAS screening dates. I find that ProPublica made an important data processing error when it created these datasets, failing to implement a two-year sample cutoff rule for recidivists in such datasets (whereas it implemented a two-year sample cutoff rule for non-recidivists). When I implement a simple two-year COMPAS screen date cutoff rule for recidivists, I estimate that in the two-year general recidivism dataset ProPublica kept over 40% more recidivists than it should have. This fundamental problem in dataset construction affects some statistics more than others. It obviously has a substantial impact on the recidivism rate; artificially inflating it. For the two-year general recidivism dataset created by ProPublica, the two-year recidivism rate is 45.1%, whereas, with the simple COMPAS screen date cutoff correction I implement, it is 36.2%. Thus, the two-year recidivism rate in ProPublica's dataset is inflated by over 24%. This also affects the positive and negative predictive values. On the other hand, this data processing error has little impact on some of the other key statistical measures, which are less susceptible to changes in the relative share of recidivists, such as the false positive and false negative rates, and the overall accuracy.

Explore related subjects

econ.GNcs.CYcs.LGq-fin.ECstat.AP

Keep this discovery

BibTeXRIS

Matias Barenstein. 2019-06-11. ProPublica's COMPAS Data Revisited. https://arxiv.org/abs/1906.04711

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Reducing Prescription Errors Through Information Intervention: A Field Experiment in Healthcare Operations

Drug-drug interaction (DDI) errors pose serious risks to patient safety. Existing decision-support systems often require physicians to respond to alerts, disrupting workflows and contributing to high override rates. We examine whether a non-mandatory information intervention can reduce DDI errors and foster learning. Using a randomized field experiment with India's largest electronic medical record platform, we analyze 2.81 million prescriptions from 1,700 physicians using a difference-in-differences design. Treatment physicians received real-time information highlighting DDI errors without being required to respond, while control physicians received no such information. The intervention reduced DDI errors by 8.6%, corresponding to an estimated US$4.8 million in annual hospitalization cost savings and approximately 134 lives potentially saved. We identify two mechanisms: reactive correction, whereby physicians remove errors after they are flagged, and proactive learning, whereby they avoid errors before alerts occur. While early reductions are driven primarily by correction, physicians increasingly avoid errors over time. They also become less likely to repeat previously flagged errors and reduce new errors, suggesting that learning generalizes beyond specific drug pairs. The effects are consistent across physician types and do not compromise productivity or care quality. Our findings show that non-mandatory information interventions can improve patient safety through both immediate error correction and persistent, generalizable learning.

econ.GN↗
arXiv2026

How an Economy Shrinks in Space: Concavity-on-Jobs and Upward Consolidation under Demographic Decline

When a country's population declines, the aggregate economy appears to contract on the intensive margin: industrial diversity intact, every industry a little smaller. At the regional level, contraction is uneven and takes the extensive form: entire industries disappear, one after another. The relevant unit is the city: industries are nested by size - the hierarchy property of industrial location - each viable only above a minimum population. Necessity industries' thresholds bunch at the low end, so a city's industry count - and its jobs - is sharply concave in size (concavity on jobs). A modest loss pushes a small city below many thresholds at once; a large core sheds a few specialized industries, one at a time. Lost industries consolidate upward to the next city large enough to host them; for the worker it means a step down to a lower-paid local job. To recover that income, workers move up to the apex - the only city hosting the full industry range. Studying Japan - two decades ahead of the OECD, Tokyo at its apex - with worker-level panel data on the young workers who carry the migration, a wage regression in real, housing-inclusive wages identifies a Tokyo-bound migration incentive that varies by origin, following concavity on jobs.

econ.GN↗
arXiv2026

Do wind and solar curtail at negative electricity prices? Incentives and evidence across two decades of German renewable support schemes

In many power systems, wind and solar generation increasingly often exceeds electricity demand. Curtailing renewable generation in those hours matters both for prices and for the physical stability of the grid. Turning off wind turbines and solar panels is technically easier than ramping down a large power station, yet support schemes often give renewables an economic incentive to keep producing at negative prices. This paper studies wind and solar energy in Germany. For each cohort of generators it estimates, hour by hour, the incentive implied by two decades of support policy. It then sets those incentives against observed behavior, using a new estimate of market-based curtailment built from reanalysis weather data. I find that in 2025, at prices below -50 EUR/MWh, almost all wind generators had an incentive to stop producing, but only half of them did. Solar is the opposite case: nearly two thirds of the potential had no incentive to curtail at all, mostly because it receives a feed-in tariff that shields it from wholesale prices. Of the exposed remainder, just over a fifth cut production. Low exposure and response rates inflate subsidy payments and make the power system harder to operate safely. I conclude that a further expansion of wind and solar requires them to respond to price signals.

econ.GN↗