Two thirds of the spend bought nothing
How we organise
TBM 435: 20 Unfiltered Operating Takes (John Cutler)
John Cutler has published twenty operating takes, one of them on AI. His take is that AI amplifies your bad habits and your good habits alike, and his hunch is that eighty per cent of the world stays exactly the same. An organisation running a feature factory before AI gets a better feature factory. Research that nobody had time to organise now gets organised. The danger he names is people who do not know what good looks like setting the bar for what they think AI can help with. Stable teams run the same way in his account. Stable teams and durable ideas are useful until they are not, and the companies that scaled quickly when money was plentiful now have to unwind the layers they built.
Structure: an amplifier does not choose what it amplifies. The AI take sits inside a list of twenty, alongside routines, lanes, cycles and goals, rather than above it. That placement is the argument. So an operating review should not ask what agents could do here. It should ask what this organisation is good at and what it is bad at, because both get multiplied. A feature factory with agents ships more of what nobody asked for, faster, and the metric that would have caught it is the one the factory never had. We start every operating review on the second half of that question, and it is usually the half nobody has written down.
Name the practice your organisation does badly and would be embarrassed to see multiplied. Fix it before you put agents near it.
How we build
Knowing when to stop: the art of making a loop converge (Yoko Li, a16z)
A loop converges only when it has four things. It needs a target state, an observable current state, a way to make precise local changes, and a rule for stopping. Li argues the verifier is where loops fail. The verifier decides when to stop, and it also defines what the loop treats as progress. In SpecBench, frontier agents routinely passed the visible tests while failing held-out tests that exercised the same features together. One agent produced a 2,900-line “compiler” that had memorised the test inputs. Returns are logarithmic. One web-agent benchmark lifted success from 38.8 per cent at one sample to 43.2 per cent at ten. Doubling again to twenty bought 0.2 more points for twice the tokens. Li then capped a page at a Lighthouse score of 89 with artificial latency and asked a loop for 100. The first $1.40 of spend took the score from 26 to 89. The remaining $2.84 was two thirds of the bill and bought exactly zero points, turn after turn of re-minifying the same HTML against a bottleneck the agent could not change.
Engineering: a loop with no stopping rule is a subscription. The spend was invisible while that loop ran. Li says neither the loop nor he knew until he read the trace afterwards, and that is the part which generalises past his example. Anyone running scheduled agents has the same blind spot. The token costs the same whether it moves the score or repeats the last turn, and nothing on the screen tells the two apart. Li names the missing instrument plainly: cost per iteration and progress per dollar, on a curve someone can see while the loop still runs. That is a reporting line rather than a research problem. We have yet to see one in a client estate, and nobody is going to add it for you.
Take one agent loop you run on a schedule and find its stopping rule. If the rule is a budget or a wall clock, you are paying for every turn after the last one that moved anything. You probably cannot say which turn that was.
Introducing Agent Plugins (Jonathan Hefner, Vercel)
Agent Plugins 1.0.0 is publicly available. It is an open, vendor-neutral format for packaging Agent Skills and MCP servers so that several clients can load the same directory. A plugin is a plugin.json manifest naming the specification version and the plugin, skills under skills/, and MCP server configuration in mcp.json. Every compatible client checks for the manifest at the plugin root. Components are then validated independently, so one broken component does not disable the rest. Version 1 carries those two component types and no more. Commands, hooks and agents stay with each client, and client-specific data goes in a namespaced directory that other clients ignore. Vercel initiated the proposal. Representatives from Amazon Web Services, Anysphere, GitHub, Microsoft, OpenAI and Vercel refined it, and the initial Technical Steering Committee holds core maintainers from AWS, Cursor, Microsoft, OpenAI and Vercel.
Architecture: the portable half of the format is the half your organisation writes. Skills hold what your people worked out about how work is done here. MCP server configuration is where your systems live. Installation, policy and user experience stay with the client. So the format decides whether that material is a directory you own or a setting inside somebody’s product. Keep it in a plugin and you can change client and keep the instructions. Type the same instructions into a vendor’s console and you rewrite them on the way out. How much of this survives contact with the clients that have not adopted it is not yet clear, and a format with six vendors behind it has been wrong before.
List the agent instructions your teams have written. Check each against one question: is it a file you own, or a setting in a product. Anything in the second column is work you will pay for twice.
How we assure
Agentic Code Quality (Addy Osmani)
Osmani had a coding agent build an app, then asked an agent to review it twice, and writes up what that changed. His claim is that software quality now depends on the constraints you set around your agents. An agent can propose anything, and the constraints decide whether a proposal is safe. The constraints he lists are ordinary and mostly already installed: unit, property and acceptance tests, mutation testing to catch the bugs the tests miss, complexity and line-length metrics, type safety checks, security scanning, and architecture rules enforced through linting. On placement he is firm. Constraints belong early in the pipeline and all the way through it rather than as a check at the end, and he expects human code review to look very different as a result.
Risk: review capacity is fixed and generation capacity is not. A team reviewing everything by hand relied on a control that scaled with the number of reviewers. Agents broke that relationship, and nobody decided to break it. The constraints Osmani lists are the ones most teams already run, so the work is not buying anything new. The work is deciding which constraints run automatically before a person sees the code. Whatever is not on that list is now unchecked in practice. Your quality bar is no longer what your reviewers know. It is what your pipeline enforces, and we now read the pipeline first in any review of an engineering estate.
Count the constraints that run automatically on an agent’s output before a person sees it. Set that number against the list of things you believe get checked. The difference is what you accept without having agreed to it.