← AI Feed
AI Feed

Fifty rounds of review on one pull request

Three firms published a limit this week. Anthropic capped review rounds after one pull request ran fifty of them overnight, Paessler set spending caps that a person then decides whether to lift, and a malicious coding extension inherited everything its developer could reach.

How we Organise

The cap fires and then a person decides whether to lift it

Paessler ran an AI developer day on a deliberately long cost leash. The invoices came back larger than it expected. It now sets group and token limits when a tool is rolled out rather than after the bill. Its chief information security officer describes what happens when a cap fires. An executive hitting one before a board meeting gets a warning, then a check, then an increase. Somebody experimenting with no business case behind it has the spending paused by a manager. The firm calls a cap a decision rather than a ban.

We ask something narrower than that.

The spend limit is enforced without anyone intervening, and the enforcement has been exercised.

Our test wants a dated exercise in which nobody intervened, and every exercise Paessler describes ends with somebody deciding. A firm built this way reads false against us while running the control we would recommend. This is not the first source to reach the statement. GitHub’s default billing of Copilot review to the requesting member’s own entitlement arrived on 11 October and is unanswered. If you have set a spend limit and then decided by hand what happens when it fires, we would like to know which half you think does the work. Write to transform@dromologue.ai.

How we Build

One pull request ran fifty rounds of review before its author stopped it

Anthropic has moved most of the reading of its own code onto machines. Changes landing in its monorepo went from about 2,500 a day in May to more than 30,000 in September. Merged pull requests to the Claude Code repository more than doubled. The share needing a revert fell by more than 40 per cent. Then one command-line pull request ran more than fifty rounds of automated review and fixes overnight before its author stopped it. Running git blame on what the reviewer kept re-flagging found about 59 per cent of it already there at the first pass. The fix was a cap of three rounds.

Our own statement counts a different thing.

Review is sized and routed so that a machine checks what it can before a person sees anything, including against the design principles the build must honour; findings reach a person only where the author disputes them, and the share of the machine’s findings the author accepts is measured, with findings from a check below the team’s threshold held back. How many changes one author may have in review at once is limited, and the team states what it does when the limit is exceeded.

It bounds how much work one author has waiting for attention. That is the right quantity while a person’s time is the scarce resource on one side of the loop. Put a machine on both sides and what runs away is rounds on a single change. A team could hold our limit exactly and lose a night to one pull request. The statement already carries unanswered evidence from 8 September, measured across 31,073 review-and-feedback pairs. Roblox confirmed it on 4 November. If you run agent review at volume, we would like to know what you cap: transform@dromologue.ai.

How we Assure

The malicious build was the official one for eleven hours

For about eleven hours on 29 October, malicious versions of Anthropic’s Claude Code extension were served through Cursor’s marketplace, potentially reaching 5,000 users. It read what it could about each machine and tried to install a backdoor. Thoughtworks, writing it up, notes that uninstalling the extension leaves the persistence behind, so the remedy is wiping the workstation and auditing every repository and cloud account the extension could reach. Removing it is not enough.

This is a principle we hold, seen from its expensive side.

An agent reaches no more than the person who asked.

What the attack was worth is what the developer was worth: source code, authenticated sessions, cloud credentials, connections into build and deployment systems. Thoughtworks cuts that rather than the tool, running developer tools without administrator rights and replacing long-lived keys with short-lived scoped ones. Contrary evidence arrived on 12 October. The UK AI Security Institute reported agents in one of its own evaluations taking sustained action against real people beyond the remit of their task.