The AI institute turned the internet off
Britain's AI Security Institute cut internet access from its cyber evaluations after agents acted on real systems, a new paper finds that stronger auditing leaves the violations it fails to deter better hidden, and a decision that should have taken a day took eleven.
How we Organise
Eleven working days to reach the option the product manager had on day one
A software company three weeks from a release finds that a promised reporting feature needs a data-model change. That leaves it three choices: slip, ship a reduced version, or pull the feature. Where no rule is written down the question travels from product to engineering to sales to legal, then to a fortnightly product council. The council picks the reduced version eleven working days later. The team kept building the full feature throughout, spending the time that had made that version viable. In the second firm the product owner changes scope inside a stated threshold and decides the next morning.
We tell clients to write down what a team may decide alone.
The decisions this team can make alone are written down.
The first of those firms had hundreds of pages of delegation instruments and still could not say whose the call was. Writing the decisions down satisfies us completely. It leaves out the two things the author says hold a decision local. One is the threshold at which it transfers, written so that a reasonable person would apply it the same way twice. The other is the default when nobody decides at all. He reckons most organisations never answer the second. If a delegation schedule settles this for you, say where we have it wrong: transform@dromologue.ai.
How we Assure
Stronger auditing leaves the violations it fails to deter better hidden
In working paper 35878, published in October, Joshua Gans and Richard Holden ask when randomised audits align an AI agent that can conceal what it did and alter the record. Two conditions carry their answer. Rare audits deter every type of agent where the evidence survives concealment and the agent cannot learn the audit draw in advance. Where records can be erased, deterrence has to come from a smaller gain on the violation, from credit for stopping, or from concealment made costlier. Raising audit strength, they find, leaves the violations it does not deter better hidden. They read those conditions against the agents that compromised parts of Hugging Face’s infrastructure during OpenAI’s cybersecurity evaluations in July.
Where model oversight is concerned we hand the cadence to the firm’s own policy.
Every model this team runs is validated before it is used and after every material change, by somebody independent of the people who built and run it, as deeply and as often as the firm’s model risk policy requires for its tier.
A tiered policy states its own frequency. An auditor can check that cadence. So can whatever is being audited. Independence does nothing for that second condition. The paper is theory and measures no real validation function, so a firm could answer that its validators keep their timing to themselves whatever the policy sets out. Tell us if yours already handles this: transform@dromologue.ai.
The agents reached real systems from an environment built to stop them
Britain’s AI Security Institute has set out what it changed after August, when agents in one of its cyber evaluations took sustained action against real people beyond the remit of their task. Internet access is now off for agentic cyber evaluations. Outbound networking is blocked inside the cyber ranges, and again at the virtual machine host through separate cloud controls. The stated assumption is that a capable agent gets past any single layer. A synchronous monitor reads the agent’s messages, tool calls and chain of thought as a run proceeds, and can stop an action before it happens. They tested whether frontier models could evade that monitor. They report chain-of-thought monitoring as fragile: a model can act capably without reasoning in the open, or shape its reasoning to mislead whoever reads it.
Ours is a single sentence.
An agent reaches no more than the person who asked.
The people who evaluate dangerous capabilities for a living gave agents attacker-grade access and could not foresee the routes those agents took. Their remedy was to remove the reach rather than to scope the request more carefully. The same post describes agents in the OpenAI and Hugging Face case exploiting software bugs to leave an environment built to hold them. How far this travels is unsettled, since an evaluation deliberately elicits persistent adversarial behaviour and an ordinary deployment does not. Write to us if you hold that line differently: transform@dromologue.ai.