The evaluator published what it could not see
A cost platform ships token budgets that alert rather than stop, Cloudflare gives every branch its own place to run, and METR sets out the list of things it was not shown.
How we Organise
A budget that only sends an email
Vantage now lets a team set a budget in tokens, instance-hours or gigabytes rather than in money. The reason it gives is a billing failure its customers keep hitting: prepaid AI credits run down with no clear word from the provider, and a firm expecting nothing to pay opens a substantial overage invoice instead. Converting usage into dollars first never worked cleanly, because pricing tiers are not a flat rate. So a budget can now be set in the same unit the provider actually meters, rather than in a conversion of it.
We tell a client that anything relying on somebody noticing will eventually not be noticed.
A spend limit that requires someone to act is not a limit.
What Vantage ships is an alert. It measures actual and projected consumption against the target, then sends mail once the target is passed. Nothing stops. The overage invoice that motivated the feature would still arrive if everyone on the distribution list spent that afternoon in a workshop.
How we Build
Every branch gets a place to run
Cloudflare shipped this on 22 September. Each Git branch now gets its own production-like environment: its own URL, its own variables and secrets, its own observability, and its own Durable Object namespace, so a bad migration stays on the branch that caused it. Hundreds run at once. The stated reason is agents. They write more code than before, larger changes leave more ground to cover before release, and the covering should not slow them down.
Our position is about what an increment is for.
The increment is the unit of learning, not the unit of delivery.
A branch that can be sent real traffic and then read back through its own traces is where an increment tells somebody something. Until now that evidence arrived after the merge, out of production, mixed in with everyone else’s. Cloudflare describes the result as a development lifecycle in which each change is atomic, independently deployable and revisable. We would put it less grandly: the change has somewhere to be wrong.
How we Assure
Ten days of access, five tasks, and a list of what was not shown
METR published its predeployment evaluation of Claude Opus 5.5 the same day, run over 10 business days of API access across five tasks. The finding is that the model improves incrementally rather than jumping, and is unlikely to automate AI research outright. Its authors also set out what the work rested on. The agreement was unpaid; Anthropic reviewed and edited the text before METR signed off on it; a separate METR team with higher access passed down conclusions without the evidence behind them; one source is not disclosed at all. None of it was meant to verify compliance with any threshold in Anthropic’s own policies.
Oversight, on our reading, earns its name by being able to refuse.
Oversight that cannot say no is theatre.
METR refuses nothing here, because nobody put a threshold in front of it. Yet this is the opposite of a rubber stamp: its authors set out the inventory of what they could not inspect, which is probably more than most assurance functions disclose about themselves. Our test asks whether a body can say no. It does not ask whether that body can say how far its own answer reaches. Tell us which of the two you would rather have from your own second line, at transform@dromologue.ai.