The change arrived without a release
Product managers began shipping code with nothing released to them, faithfully stored memory made answers worse, and the floor for the least-served users is set before a firm buys anything.
How we Organise
The people who used to describe a change are now shipping it
Across 166,000 paid users the share of product managers attaching a pull request went from 3 per cent to 10 in two years, and designers from 1 per cent to 8.
We ask a team to work out who outside it must work differently, give each a way to learn, and check they can before anything ships. The trigger is a release. Nobody released anything to these people. They picked up a general tool on their own, and their numbers trebled with no learning route and no bar set. It is one vendor’s customer base.
How we Build
The memory was accurate, relevant, and made the answer worse
Every memory strategy tested scored below having no memory at all, the strongest still falling more than 10 points. The fall came from what the memory meant, not from the extra context.
Our own rules bound what a system may remember, expire it, and have somebody test the expiry. Each part catches memory that should no longer be there. What was measured is memory that should be there by every rule we state: faithful, relevant, inside its bounds, far from expiry. A team passes our test completely and takes the fall anyway. The dialogues were built to induce the failure.
How we Assure
Thirty-seven per cent of users have nothing that ranks their first value first
Twenty-three frontier model archetypes were audited as shipped and set against what 1,649 people said they wanted. None of the archetypes puts helpfulness or autonomy first, which leaves 37 per cent of participants with nothing ranking their first value first.
We put a quality floor under a firm’s agents and name somebody to hold it. That describes the firm’s own agents. The ranking measured here arrives in the model as bought, and it moves away from the value already least served. A firm can hold its floor exactly as we ask while the real floor was set by what was on the menu. The audit reads defaults rather than deployed behaviour.