It came back in the right format
Professionals guarding the signal that carries their name and giving away the one that carries effort, robustness rankings that reverse when the scaffold changes, and a tool failure that arrives in the expected format.
How we organise
What generative AI does to the signals colleagues read off each other
Across 1,250 workplace interviews, two signals came apart. Professionals defend provenance, because their name is on the work. About investment, the effort behind it, they are candid to the point of comfort: the model does the work and the delivered thing looks as it always did.
We have argued that a person’s level should rest on work they have shipped, because another person can go and check it. If what arrives no longer carries the effort behind it, that test passes on somebody whose judgement nobody watched. It is self-report, so nobody was deceived.
How we build
Rewriting the code around an agent without changing what it does
Working codebases were rewritten into an equivalent form: reshaped control flow, dead code, renamed identifiers. Most agents degraded a little, and the worst lost 6.7 points of resolve rate. No ranking of models by robustness survived a change of scaffold. One model ranked among the most robust under one harness and the least under another.
Our position is that a skill’s suite includes properties that must hold whatever the input; semantic equivalence is one. Nothing here disputes that. It shows what a pass leaves out: a result has to record the scaffold it came from. The drops are single-digit.
How we assure
Monitors on the return path of a tool call
A timeout is a visible failure and an agent routes around it, while a cached error page arrives in the expected format and is read as fact. Monitors that check each return against a contract raised completion from 10.9 per cent to 28.1. Strip the recovery tools out of the receipt and the gain goes.
We judge a tool by what it hands back, and check that before it reaches the model. The paper bears that out, then goes past us: checking is not the part that helps, and naming what the agent may do next is. Outside the vocabulary the monitors were mined from, detection fell to 46 per cent.