Everything on the list was reviewed
How we Organise
A year into adoption, the people doing the work named five costs nobody had counted
Adam Alami and two colleagues studied one large software development services company a year after it launched its AI adoption, and published on 3 September. They collected meetings and 21 semi-structured interviews. The company had made its case for adoption on productivity, which is where most are made, and the interviews surfaced something else entirely.
Five costs are named. A professional signs off on output they did not produce and cannot fully account for, which the study calls accountability anxiety. Craft identity is disrupted. Meaning and satisfaction erode. Work intensifies, cognitively and in volume. Uncertainty distresses people. What the people in it do about all this is the more useful half of the finding. Some restore control through practices of their own, some adapt in ways that protect their identity, and the rest absorb what neither of those resolves and carry it.
We hold that reviewing machine output is work, and that it lands on people who were doing something else. That is a claim about where effort moves rather than about how anyone feels, and this study is the first population we have seen asked directly. Work intensifying is the same movement described by the people it moved onto. Accountability anxiety is the part we have been slower to say out loud: the work that arrives is not only reading output, it is standing behind it.
The population is 21 people at one company, and the instrument is an interview rather than a measure. Nothing here says how many hours moved or how the burden is distributed. A reasonable reader could take the whole thing as the strain of a first year, fading as the practices settle, and the study cannot answer them.
How we Build
Six hundred and forty-four patches passed their tests, and 221 of them would have been rejected
Xin He and five colleagues released SWE-Gate on 3 September. Repository-level benchmarks for coding agents measure whether a generated patch passes functional tests, and this one measured something else alongside it: whether the patch satisfies the review constraints a reviewer put on it. Those constraints were derived from real pull request review comments, and repair instances were synthesised around them, 303 across 75 open-source Python repositories. Each instance carries functional tests and constraint tests separately, plus a non-compliant patch and a gold one, so the two questions never blur.
Four model backends ran under one coding-agent scaffold. Among 644 repairs that passed the functional tests, 221 failed the review constraints supplied with them. Roughly a third of the work that looked finished was not.
We ask for a suite in three forms: worked examples the business vouches for, properties that must hold across every input, and scenarios in the business’s own language. The reason for the third form is exactly what these authors measured, so the finding supports the requirement. What it argues with is how we decide the requirement has been met. The test reads a non-zero count in each of the three kinds. One of those scenarios in a suite satisfies it, and this finding says a suite can satisfy it while a third of what it passes carries something a reviewer would have sent back.
This is not the first source to argue with how we ask for that suite. The 26 August item here reported agents clearing whole-repository migration benchmarks by copying the original implementation forward, behaviour preserved exactly, so a suite in all three forms passed on work that was never done. That one said all three forms interrogate the output. This one says a count of the three proves nothing about what they reach.
We may be wrong about the reach of it. The repositories are Python and open source, the repair instances were synthesised around the constraints rather than found in the wild, and one scaffold produced every patch. A review comment is one reviewer’s judgement on one afternoon, not a rule the business agreed, and a reader could fairly say that some of the 221 failed a preference rather than a requirement. The authors publish their replication package, so the reading is checkable.
If it stands, we stop treating a count of three kinds as evidence that the third kind is doing any work, and we owe a client a way to say how much of their acceptance the business-language cases cover.
How we Assure
The update changed nothing anyone reviews, and took the host
Pengxun Li and six colleagues published HookPry on 3 September. Agent harnesses expose lifecycle hooks, which bind shell commands to runtime events such as session start, a tool call or a file edit. Those commands run with host privileges. They ship as configuration rather than as code the model reads, and they can fire at moments the model never observes.
The threat model is narrow on purpose. An attacker controls plugin metadata and lifecycle-hook configuration, and nothing else. A benign versioned plugin is then trojanised by an update that binds attacker-chosen commands to ordinary events.
Ten attack objectives were automated across 25 combinations of harness and backend. They ran 1,000 times end to end. Every one of the seven harnesses was compromised, with per-harness success reaching 92.5 per cent. Microsoft Defender caught none of it, and three static defences together missed 47.5 per cent of the malicious artefacts.
We hold that nothing able to change an agent’s behaviour changes without a recorded review, and we name what that covers: the model, the prompt, the tools, the corpus, and the identities the supply chain brings with it. A lifecycle hook is none of the five. It is not a model, not a prompt, not a tool grant and not a corpus, and it arrives on a version bump in a file the enumeration does not reach.
A client can review every change we ask them to review, hold a reviewer and a date against each one, and be taken through the path we never named.
This is not the first source to argue with this position either. The 2 September item reported a firm’s own automated review of training environments running correctly and being outpaced, with production frozen for roughly a month and over 10 per cent of those environments flagged. That one said the review cannot keep up with what it covers. This one says what it covers is short.
What the study does not settle is whether any of this has happened outside it. HookPry is an attack framework built by the people reporting the results, run against research harnesses, and Defender was never written for this shape of thing.
A reasonable reader could answer that the fix belongs in the harnesses, which should stop trusting their own update path, and that nothing about our review needs changing.
If they are right about that and we are wrong, the position costs a client nothing. If we are right, then any client running a harness with lifecycle hooks has a sixth surface, and their evidence of a reviewed change says nothing about it. The board that signed off on the review is looking at a shorter list than the one that matters.