Laurie Voss argues that AI agents have made code generation cheap while human review has stayed the same speed, creating the real bottleneck: one study of 100,000+ GitHub developers found agent users wrote 741% more code but shipped only 30% more software. He walks through the evidence on whether automated review can replace humans — METR's maintainer study, Cognition's Frontier Code benchmark, GitHub/Cursor's production reviewers, and the Bun and OpenAI no-human experiments — and concludes review hasn't died, it's been relocated. His practical advice: stop reviewing individual PRs and spend your human judgment building the review harness, rubrics, and evals that review for you.
- Three economists tracking 100,000+ GitHub developers found that those who turned on autonomous agents produced 741% more code but shipped only 30% more software, with review identified as the bottleneck.
- A two-decade-old Cisco study (2,500 reviews, 3.2M lines) found reviewer effectiveness collapses above roughly 400-450 lines per hour, meaning a single 10,000-line agent PR would take a human three to four working days to review properly.
- Passing tests is a weak proxy for mergeability: METR had four open-source maintainers re-review PRs that had already passed SWE-bench, and only about half were actually good enough to merge — failures were about code quality and silent breakage outside the test suite.
- Cognition's Frontier Code benchmark, built by 20+ maintainers from 150 real tasks, shows the gap starkly: a model scoring 88% on SWE-bench Pro scores 29% on the hardest slice, and another frontier model scores under 6%.
- Automated review is already mainstream — GitHub Copilot has done 60 million reviews and handles more than one in five code reviews on GitHub, while Code Rabbit has reviewed over 13 million PRs; all these tools optimize for the same metric, whether a human accepts the suggestion (Cursor drove its resolution rate from 52% to over 70%).
- Cursor's reviewer runs multiple shuffled passes to filter false positives (a Peking University team independently found multi-pass raises review quality up to 44%), and the rebuilt version had to be explicitly told to distrust the code by default rather than saying 'looks good to me.'
- Experiments that removed humans from the loop still had humans on the loop: Carlini's 16-agent C compiler relied on human-written test harnesses, and Bun's Zig-to-Rust port passed 99.8% of tests while containing 13,044 unsafe blocks versus about 74 in a comparable human-written codebase.
- Automated reviewers are easy to fool — Anthropic's own security reviewer warns it isn't hardened against prompt injection, and a March study found vulnerable code with an innocent commit message fooled an autonomous review agent in 88% of attempts versus 35% for humans.
The single number that frames the entire talk and quantifies the review bottleneck everyone is feeling anecdotally.
The developers who turned on autonomous agents wrote 741% more code but only 30% more software shipped. That is the scale of the problem right there... and the authors of the study are blunt that review was the bottleneck.
A concrete, three-orders-of-magnitude example of what green tests fail to catch when no human reads the diff.
Somebody looked carefully at the ported code and it has 13,044 unsafe blocks. In a comparable human written Rust codebase of that size, you would see about 74... The test suite can certify behavior at the public interface. It cannot certify 13,000 assertions that the test suite was never designed to look for.
My favorite detail from their rebuild was that they had to tell the model to be more suspicious of the code — don't trust the code by default, assume there is something wrong with it.
I was wrong. Please, please read the code. We tried not reading the code for like six months. It did not end well.
Whoever writes today's review standard is writing next year's default model behavior.
Stop reviewing PRs. It is the wrong level of abstraction for 2026.