Everything on the list was reviewed
Twenty-one interviews name what a year of AI adoption cost the people doing the work, a third of functionally-correct patches fail the review constraints they were written under, and a plugin update binds a shell command to an event no review reads.
How we Organise
A year into adoption, the people doing the work named five costs nobody had counted
Twenty-one interviews at a firm a year into AI adoption named five costs the business case never counted, led by signing off on output nobody could fully account for.
We hold that somebody has to read what the machine produced, and that the reading is a job landing on people who already had one. This is the first population asked about it directly. What we have been slower to say is that the job is not only reading. It is standing behind it. Twenty-one people at one firm, interviewed rather than measured.
How we Build
Six hundred and forty-four patches passed their tests, and 221 of them would have been rejected
Coding agents were measured on whether a patch passes its tests and, separately, whether it meets the constraints a reviewer set. Of 644 repairs that passed, 221 failed the constraints.
Our position asks for tests in three kinds, one written in the language the business actually uses. What this argues with is how we decide the asking has been met. The test counts one of each, and a single case clears it while a third of what passes carries something a reviewer would have sent back. The instances were synthesised rather than found in the wild.
How we Assure
The update changed nothing anyone reviews, and took the host
A benign plugin was trojanised by an update binding attacker-chosen shell commands to ordinary lifecycle events. Every one of seven agent harnesses was compromised. Commercial antivirus caught none of it.
We want a recorded review before anything that could shift how an agent behaves goes live, and we spell out five places to look. A lifecycle hook is not among them. It arrives on a version bump, in a file the list never reaches. A client can review everything we ask, with a name and a date against each, and be taken through the path nobody named. This is an attack framework reporting its own results.