Pacing the frontier is a management problem

In the week to 14 September 2026, Dario Amodei, Sam Altman, Satya Nadella, a British government minister and a Chinese state newspaper all published on the same question. Take the geopolitics out of what they said and I recognise the argument underneath from change programmes I’ve worked on, where you can’t ship change faster than you can check that the last change worked.

Amodei’s essay, We Must Pace the Frontier, is the one I’ll start with. He argues that frontier labs should deliberately slow the rate at which capability advances so that safety work can keep up, and that labs should compete on safety, a “race to the top”. I was most interested in the first step of his framework: embedded evaluators, third parties with employee-level access who verify safety practices, audit training and report incidents, borrowed from banking supervision. The rest is chip controls, antitrust waivers and treaty analogies, which I’ll leave to people who know them. The first step is about how organisations are designed, and I do have opinions about that.

Altman set two failure modes side by side the same week. The first is losing control of the future to AI, which he refuses outright: he is “unapologetically on Team Humanity”, and safety has to stay ahead of capability. The second is concentration of power, which he calls “extremely dystopian”, and he names “one lab ending up with too much power” alongside the country version.

Nadella welcomed the “deliberate pacing” language and added something the labs leave out. Every firm, he said, should own its own continuous learning loop and put its knowledge into models and weights it controls. He also said control of this technology can’t sit with a handful of entities. He runs one of those entities, but I think he’s right about this.

Kanishka Narayan, a year into government and now in Cabinet, set out where Britain focuses next in three parts: build domestic capability, test risk as close to the source as possible, harden the wider system. The harden limb says securing models isn’t enough while everything around them stays soft. The test limb asks the AI labs to work with the UK’s AI Security Institute, the nearest thing to Amodei’s embedded evaluator at national scale.

The same argument at four scales

That same weekend, Peyman Milanfar posted a control-theory thread called “Intelligence Has a Speed Limit”. He argues that a self-improving system can’t run away, because any feedback loop has a maximum safe speed, set by how fast it can produce trustworthy evidence that a change was an improvement.

Past that speed the loop doesn’t fail cleanly. It runs beautifully, then bursts, goes unpredictable without warning and may even settle back down. Evolution and civilisation get held up as self-improvement success stories, and he says they prove his point. Both ran slowly relative to their feedback, in parallel, with “no single agent holding both modification power and evaluation authority”.

Put them in order of scale and two ideas turn up at every level. Milanfar is describing a single system. Amodei, Altman and Nadella are talking about labs and platforms, and Narayan about a country.

The first idea is to pace yourself to the speed you can verify at. The second is to keep the power to change a system and the authority to judge the change in different hands. A country can’t pace global capability the way a lab can pace itself, so its version turns into build, test and harden, with a public institute as the evaluator.

The fourth scale is the one I work at. A federated organisation is built like a country, with a centre that has a mandate, parts with their own legitimacy and a permanent argument about which decisions belong where. Build, test and harden carry across. You can’t opt out of building capability. Evaluation has to take place where the work is done, and not only at the centre. And there’s little point securing the system the centre is proud of while the systems out in the parts stay soft.

I’m not going to reach for the examples closest to hand, and that affects how much weight to put on what follows. The work I do now isn’t mine to describe here, so take the rest of this as argument, tested against things I’ve watched but can’t produce.

Who gets to do the verifying

On 14 September the Global Times answered. Reuters reported the state-backed paper calling Amodei’s essay a “Cold War playbook” aimed at China, whose real agenda is “to attempt to curb China’s AI development through technological barriers and regulatory monopolies”. The paper calls the whole thing a “silent AI Cold War”.

Excluding China from global AI work, the paper says, would “significantly increase the trial-and-error costs and risks of loss of control in global AI development”. That’s a pacing argument too, made in Amodei’s own vocabulary and pointed back at him. Both sides agree the danger is unverified change. They disagree about who gets to do the verifying.

That’s the problem any centre has when it proposes a check. Amodei leads a lab that would operate under the rules he’s proposing, and he argues for them anyway, starting with a commitment his lab would make on its own. A careful safety case can still look like a power move to anyone who wasn’t asked to help write it. A pause does help whoever is ahead, though that doesn’t make the safety case wrong.

I’ve watched the same exchange at a much smaller scale. The centre proposes a checkpoint before the next thing goes out, and the rest of the organisation hears the centre asking for control. Explaining the check carefully doesn’t help much. A detailed case can sound like a rationale worked out in advance.

Legitimacy is the hard part of pacing, and it comes down to who owns the evaluator. If the body deciding whether a change worked is the body that made the change, the people it affects are right to object, and no amount of method will fix that. It’s different if the evaluator is independent, has real access of the kind Amodei means by employee-level and reports to someone other than the team it’s grading. Then everyone is reading the same measurement.

Where federations come in

Federated organisations exist to resist concentration of power. Authority is spread out on purpose, and the parts hold things the centre doesn’t. Guarding against Altman’s second failure mode is a principle a federated body has lived by for most of its history.

That makes the next bit uncomfortable. AI centralises capability. Whoever owns the model owns the learning loop, which is where an organisation’s knowledge ends up. A federated body running on someone else’s model has moved its institutional memory somewhere none of its parts control, usually one procurement at a time and without anyone deciding to.

Nadella’s answer is that every firm should own its loop. As a principle he’s right. For a charity it’s mostly unaffordable, and the advice comes from the company whose stack you’d build it on. No organisation can avoid dependencies, so the realistic answer is to choose them deliberately, write down which ones you’ve chosen and work out what it would take to leave.

You don’t need a treaty for any of this. Pace the work to how fast you can get trustworthy evidence that the last change landed, which is almost always slower than the plan says. Separate the people who build a change from the people who judge whether it worked, and give the judges access to the work itself. When you propose a check to parts of the organisation that didn’t ask for one, expect it to be heard as a bid for control, because that’s how it’s heard at every other scale in this argument.

I think the pacing argument survives being taken out of the labs, and legitimacy is where we’ll go wrong first. If you’ve seen a centre in a distributed organisation make a check stick, I’d like to hear how it was done.