AI-coding-era change failure rate

What the 2024 DORA AI sub-study actually says, and how to run an AI-on / AI-off CFR experiment on your team.

Oliver Wakefield-Smith

Founder, Digital Signet. Built siemcost / monitoringcost / itbudgetcalculator.

Updated 21 Jun 2026

Direct answer

The 2024 DORA AI capabilities research finds AI-assisted development raises throughput while modestly degrading stability for most teams, with a sub-cohort showing stability gains after one quarter of usage DORA AI Research2024. Treat any single-quarter CFR delta as preliminary.

How to tag AI-authored commits

Three options: a commit-message footer (Co-Authored-By: ai), a Git note, or an IDE-emitted metadata field. Pick the one your CI already parses. The tag travels through the deploy pipeline so CFR can be split AI vs human.

Running the experiment

A defensible AI-on / AI-off CFR test: split team-by-team, not engineer-by-engineer (less contamination). Hold deploy frequency roughly constant. Use 90-day windows for the comparison (sample-size floor). Pair CFR with DXI survey data for SPACE-aligned contextDX Core 42024.

What the DORA report actually measured

The 2024 DORA report uses self-reported AI usage frequency, not commit-level AI authorship tagging DORA 2024 Report2024. Reading the findings, weight the "throughput up, stability down" pattern as the central tendency, not as a deterministic outcome.