Should the frontier slow down

Should frontier labs slow capability advancement — and if they do, who ends up holding what gets built?

Spans 2026-07-21 – 2026-09-12 Positions 10

The positions

Stated as each side would state it. Where someone has never said what would change their mind, this page says so rather than guessing.

The rate of capability advancement itself has to slow, and outsiders have to be able to verify that it did.

Pachocki, writing as OpenAI's chief scientist, says no lab — his own included — has solved alignment and monitoring well enough to keep scaling at maximum speed for much longer, and that chain-of-thought monitoring is becoming less reliable as models get better at reasoning about their own reasoning. Amodei's essay six days later proposes three steps: embedded third-party evaluators with employee-like access who may publish findings the company cannot redact, coordination among democracies, and global agreement. Neither is asking for a halt, and both say unilateral action is insufficient. The signatory count is reported between 1,100 and 1,134; the figure used here is the one a source states exactly.

Held by Dario AmodeiJakub Pachocki 1,134 frontier-lab employees
What would change it

A monitoring method demonstrated to hold at the next capability level. Both state the condition rather than the absolute: Pachocki that no lab has solved it "to a sufficient degree," Amodei that pacing is not halting.

Slowing external releases widens the gap between what the labs hold and what anyone else can reach, and nobody has priced that.

Brown grants the premise and states it more concretely than the pacing advocates have: models already work over horizons longer than the interval between releases, so no lab can evaluate one at the full length of its capabilities before shipping the next, and most safety policy was written in the GPT-4 era when horizons were not a consideration. Then he names the flip side. What slows is external release; internal work continues. Mathematics is where this is already visible — a model inside OpenAI is solving open problems the outside world cannot reach. "That is an unfair advantage."

Held by Noam Brown
What would change it

Not stated, and for an unusual reason: Brown says outright that he does not have an answer for how to weigh the trade-off. This position asks a question rather than asserting one.

The forecasts this argument rests on have a poor track record, and the number driving the alarm was made up. The remedy is engineering discipline and auditors, not a slowdown.

Huang's objection is to the forecasting record: radiologists replaced within five years, most code written by AI within months, models too dangerous to release, all predicted with confidence and all wrong, so an industry reorganising around the newest prediction is reorganising around a bad prior. The >10% figure he calls made up, and irresponsible to say from inside a lab. But he is careful where the episode's title is not: of Coxon, "we don't know what Coxin saw"; pacing and pausing are "voluntary things" a company can do if it feels out of control; and the danger does sit at the frontier labs, "because they have the most compute." His remedy is to get the engineering right and to bring in outside evaluators on the model of financial audit: auditors "don't have to be as expert as we are in our business, but they just have to ask the right questions," and there should be several of them.

Held by Jensen Huang
What would change it

He names one case himself: a lab concluding after an incident that it does not know what happened, has "no idea how to control it," and is "asking society for help," in which event other companies ought to send engineers in. He adds that he doubts it: "They got this handled."

Speed is not the issue. Moral status is, and leaving it open makes alignment harder.

Suleyman's disagreement with Anthropic is not about pace. Microsoft AI's Humanist AI Code of Conduct asks for three properties — subordinate, aligned, contained — and states that models will never resist human interruption, correction or shutdown, and that AI should be a tool, not a person. His claim is that leaving open whether a model is a moral patient is itself an alignment problem.

What would change it

Not stated directly. His position is a claim about which question matters, and it is compatible with either answer on pace — which is why it reads as disagreement and is not.

Controlling a superintelligence is not a solvable problem, so pacing does not help — a general one should not be built at all.

The disagreement here is not about the odds, it is about the premise everyone else shares. Yampolskiy's point is that even the safety community assumes indefinite control is achievable given more time, money and talent, and that his research points the other way: a system that must never make a single mistake, across every interaction, every malevolent actor and every round of self-improvement, is a perpetual motion device with a different name. Soares argues the same corner from MIRI, in a book called If Anyone Builds It, Everyone Dies. Note what the position does not say: Yampolskiy accepts today's LLMs, which are deployed and have not killed anyone, and wants narrow systems — self-driving, medicine — to continue. What he wants banned is general superintelligence, permanently.

What would change it

They say the honest answer is that no observation would, and treat that as the argument rather than a dodge. Asked in the same debate what evidence would justify shutting it down, Andrew McAfee named something concrete: AI taking over San Francisco's robotaxis and driving them at people, with nobody able to stop it for a month. Yampolskiy's reply was that a week and a month are the same thing, because by then it is already too late. He is equally direct about his own proposal's gap: if it is not built, he has no answer for stopping someone else from building it, and says so.

Voluntary pacing is not enough. Frontier AI should be licensed the way medicine, aviation and nuclear power are, and the race should not be accepted as a given.

Bengio agrees with the pace-now side about the danger and rejects its mechanism. The labs describe themselves as trapped in a race; he told the Security Council that "the race is not a law of nature, it is a product of choices," made by the companies themselves, and that the labs have said they would slow down if they could — "They can. However, we shouldn't count on it." So the remedy has to bind rather than ask: developers must demonstrate to independent experts that a system is safe to train and to deploy, report every security incident, carry liability insurance, and hold a licence, as in the other industries where a failure is catastrophic.

Held by Yoshua Bengio
What would change it

He does not state a condition. What he does state is the burden: developers must prove to the public that their products will not be dangerous, which places the evidence on the side that wants to proceed rather than the side that wants to stop.

Pace is the wrong variable. A model should ship when it meets safety criteria, not after a slower walk, and the whole industry is far from the rigor that would require.

Robinson led the writing of OpenAI's system cards until he resigned, and he signed the pacing letter. He still rejects the word: "running off a cliff and walking slowly off a cliff are just not that much different." What he wants is a gate. If the criteria can be met in ten minutes, ship; if two months pass and they are not met, the model stays blocked. He names the bar: operational rigor at least matching nuclear power, such as triple redundancy so that one person's bad day cannot cause a meltdown, and he would rather risk overregulation than stay where the industry is now. He places the problem in the industry's culture, not in any one company or person. He also concedes that rigor is not enough: alignment "is a science problem," and even with every proven control from nuclear and aviation, nobody knows how to make a model behave when no one is looking. He does not claim certainty about catastrophe, only that "we can't afford to assume" it is not on the table.

Held by David Robinson
What would change it

He states what would lift the block, not what would refute the position: criteria met, however quickly. He does not say what the criteria are. His position does not depend on the forecast either. He says he does not know whether Paul Christiano's warning is right, only what caution would look like if it were.

The danger is not powerful AI but asymmetric AI, and the answer is to distribute capability, including through open source, rather than slow it.

Delangue speaks as the target of the July attack, and draws the opposite lesson from the one most of this thread draws. When his team tried to defend itself, closed frontier models refused them — the guardrails could not tell a defender from an attacker, while attackers jailbreak them anyway — and what worked was an open-source model released from China. His conclusion to the Security Council: "the biggest risk is not powerful AI," it is asymmetry — between attackers and defenders, a few companies and everyone else, a few countries and the rest. He also wants more transparency, through mandatory sharing of full agent traces, and says fear-based narratives are the wrong way to decide.

What would change it

Not stated. Worth reading beside Noam Brown's position: the two diagnose the same gap, inside versus outside, from opposite sides of it. Brown names the cost and has no remedy; Delangue has a remedy that runs directly against pacing.

The companies should police themselves: safety belongs to internal controls and audits they commission, and law, if it comes, comes later.

At the White House on 29 September Trump said he was "seeing tremendous self-policing," and called the accord the companies signed "morally binding" and "almost like a constitution." It commits each signatory to four layers: internal controls, an internal team that checks them, an independent external auditor, and a committee of the company's board that reviews the auditor's reports. The signers will meet to set standards, and the text allows that "over time, it may make sense to codify these steps into laws or regulations." Vance gave the case against the alternative: a regulator on the FDA or FAA model needs regulators who understand the technology, and "most bureaucrats just know way less about this than the people who are actually building these products." Trump also floated a committee of "maybe 10 people" to oversee the industry, without detail.

Held by Donald TrumpJD Vance
What would change it

Has not said what would change their mind.

The people calling for this are corporate employees, not independent researchers, and lack the standing to make the call.

The objection is about who gets to set the terms: people described as researchers are employees of the companies whose products are at issue, and a call for industry-wide pacing from inside a frontier lab is a competitive act whether or not it is sincere.

Held by David Sacks
What would change it

Two things in this thread already cut against it. Brown's argument runs against his own employer's interest, and Pachocki conceded the alignment point first and unprompted, six days before any rival had to respond.

Where I come down

I spent three attempts hearing this as a moral question — should they slow down, whose motives are clean — and that framing is too narrow. Brown grants the premise and still objects, which means the live disagreement is not about speed at all. It is about distribution: slow external release and the lab keeps moving internally, so what widens is not fast versus slow but inside versus outside. The question worth following is not whether the frontier slows. It is who ends up holding the capability if it does.

Worth having straight

Where the 10% came from

The figure quoted everywhere as a probability of catastrophe was never calculated, and it did not start with the man who is usually credited with it. Coxon resigned and posted first, giving no number at all; Hubinger replied under that thread, and the number appears for the first time in that reply, carrying four qualifiers. Each step after it dropped one.

Origin
Jacob Coxon's resignation thread he posted first, and gave no number at any point in it
The figure
Evan Hubinger, replying under that thread the same day "I personally think it is >10% within the next decade" — its first appearance anywhere
Qualifier: personally
"I personally think" not Anthropic's position, and not any team's conclusion
Qualifier: greater than
">10%", not 10% a floor, not a point estimate
Qualifier: within a decade
an explicit time frame not "someday"
Qualifier: future systems
systems that come after recursive self-improvement the risk is not pinned on today's models
Spread
press coverage, then spoken retelling written reports mostly kept "more than 10%"; said aloud it wore down to "10%", and of the four qualifiers usually only the first survived

Sources Jacob Coxon's resignation thread, where no number was givenEvan Hubinger's reply, where the figure first appears

Two numbers given under pressure

The 10% and Noam Brown's 3x are the same kind of object: a personal estimate produced on demand, then quoted downstream as a measurement. Brown attached the condition himself — "if you put a gun to my head and ask me for a number" — and bracketed it on both sides, saying it could be 50% faster, which he thinks unlikely, or ten times faster, and that the uncertainty is large. Quote either figure without its condition and you have reported something the speaker did not say.

10%
a stated personal credence four qualifiers in the original wording
3x
a number given when pressed for one bracketed by the speaker at 50% and 10x

Sources Noam Brown – Agent swarms, alignment, & recursive self-improvement (transcript)

Amodei's three steps, and what each asks for

"We Must Pace the Frontier" proposes a sequence, not a pause. Anthropic commits to the first step unilaterally; the second and third require other people to agree. Amodei breaks the third into four tiers and rates each himself, and a full pause is the last of them — the one he calls unlikely, and the one most people hear as the whole proposal.

1. Embedded evaluators
third parties with employee-like access may publish findings the company cannot redact for being unfavourable; the only step Anthropic can take alone, and it has
2. Coordination
among democracies needs legislation or an antitrust exemption — competitors agreeing on pace could itself be illegal
3. Global agreement
including states outside that bloc needs a cross-border verification regime; he splits it into four tiers
Tier one
banning specific dangerous uses, such as bioweapons "probably achievable"
Tier two
a shared pre-release testing body for cyber, bio and alignment risk "I think an institution like this is quite likely to be feasible"
Tier three
capping the speed of recursive self-improvement "hard, but right at the edge of possible"
Tier four
pacing or pausing AI development outright "unlikely to actually happen anytime soon" — this is what pacing is not

Sources We Must Pace the Frontier

What Amodei asked for, and what he signed

Seventeen days separate the essay from the accord, and Amodei's name is on both. The accord keeps the shape of his first step, someone outside the company looking in, and leaves out the two properties that made it a check. At the signing he said the technology "has very real risks," and that "the mechanism, how we address those risks is still under discussion." The accord's full text was not published with the coverage; the rows compare his essay with the accord as reported.

Who looks
the essay: third parties with employee-like access the accord: an independent external auditor, with no access terms reported
Who sees the result
the essay: evaluators may publish findings the company cannot redact the accord: reports go to a committee of the company's own board; nothing reported about publishing them
Who sets the standard
the essay: a shared pre-release testing body, tier two the accord: the signers meet to set standards themselves; SAFA, reported three days earlier, would certify auditors, and Meta, xAI and Nvidia are not part of it
What holds it
the essay: step two needs legislation the accord: voluntary; Trump's phrase is "morally binding"
Whose shape it takes
Huang's, from 14 September outside evaluators on the model of financial audit, who "just have to ask the right questions"; pace left to each company

Sources We Must Pace the FrontierWhite House unveils 'super intelligence' executive order and industry accordTrump vows to never stifle AI as leading executives urge cautionGoogle, OpenAI, Anthropic plan a Standards Authority for Frontier AI

Three things widely misattributed

Two come from the Brown interview, where the host's words get retold as the guest's; both were checked against the published transcript, not the captions. The third is a word this page cites a title for, and the title merges two speakers.

"Three consecutive swarms, April to August"
Dwarkesh Patel retold as an OpenAI researcher's admission; Brown's reply was that monitoring was not running, and now is
The internal-versus-external gap
Dwarkesh Patel raised it Brown credits him out loud: "There's a flip side to that, which is what you said"
The word "hoax"
Donald Trump the All-In episode's own title merges it with Huang, and this page cites that title as published; Huang's case is the forecasting record, not that word

Sources Noam Brown – Agent swarms, alignment, & recursive self-improvement (transcript)Trump calls AI oversight a 'globalist scheme' as Amodei and Altman head to the UN to ask for it

Watching

Videos that frame the whole argument rather than supporting one side of it. Everything else sits under the position it belongs to.

Timeline

  1. 2026-07-21 The OpenAI–Hugging Face Attack OpenAI / Hugging Face
  2. 2026-07-28 Pacing the Frontier Frontier lab employees
  3. 2026-08-04 The Test That Reached Real People UK AI Security Institute
  4. 2026-09-01 The Incident, Reproduced On Purpose Anthropic
  5. 2026-09-06 An Alien Mind OpenAI
  6. 2026-09-12 We Must Pace the Frontier Anthropic

What changed

  1. 2026-10-07 David Robinson, who led the writing of OpenAI's system cards, resigned and published an essay in The Atlantic on 3 October calling the company's culture broken. Four days later he gave his first interview, to Ezra Klein. He signed the July letter and now rejects its central word: slower is not safer if the criteria still are not met. This is the first position here from someone whose job was writing down why a release was safe.
  2. 2026-09-29 Trump hosted the labs' leaders at the White House, and they signed a voluntary accord to police themselves: internal controls, an external auditor whose reports go to each company's own board, and regular meetings to set standards. Amodei signed beside Pichai, Zuckerberg, Brockman, Musk and Huang, and said at the table that how to address the risks "is still under discussion." The same day an executive order told federal agencies to say "Super Intelligence" instead of "artificial intelligence," defined as exactly what the statute already calls AI. The word this argument is about now names every system the government uses.
  3. 2026-09-26 The Information reported that Google, OpenAI and Anthropic are building their own standards body, the Standards Authority for Frontier AI, modelled on Wall Street's FINRA, to set rules for pre-release testing and incident reporting and to certify outside auditors. They turned to it after a draft executive order for federal oversight failed to win support inside the administration.
  4. 2026-09-23 The argument reached the body the third step was written for. At the Security Council's first meeting on frontier AI safety, Amodei reported that some companies have already agreed to embed external evaluators, and asked for narrow bans, starting with AI-enabled bioweapons, plus verification and incident notification. Bengio asked for licensing instead of voluntary pacing; Altman asked for standards and not for slowing; and Delangue, the July attack's target, argued for open source.
  5. 2026-09-22 A day earlier and a few floors away, Donald Trump told the General Assembly that international AI oversight is a "globalist scheme" and called existential-risk fears a hoax. It is the first time a head of government has taken a side in this argument, and the side is against all of it.
  6. 2026-09-19 A second axis opened. Noam Brown, at OpenAI, granted the case for pacing and named a cost nobody had priced: slowing external release widens the gap between what labs hold and what anyone else can reach.
  7. 2026-09-17 The Wall Street Journal reported that Zuckerberg, Musk and Huang had separately persuaded Trump to block an industry-funded AI regulator.
  8. 2026-09-17 The objection from the safety side finally shared a stage with the rest. In a four-way debate, Roman Yampolskiy and Nate Soares argued that control is not a solvable problem at all, which makes pacing beside the point — while Andrew McAfee, at roughly zero, said LLMs are not the path and the whole discussion is a distraction. The spread is the finding: this argument is not one axis.
  9. 2026-09-17 Microsoft's Mustafa Suleyman made it clear his disagreement with Anthropic is about moral status, not pace — which reframes him from opponent to someone answering a different question.
  10. 2026-09-14 At the All-In Summit, Jensen Huang called the figure the week was arguing over made up, and the forecasting record behind it bad — the first position here that rejects the alarm itself rather than the remedy.
  11. 2026-09-14 Microsoft AI opened a six-week consultation on a Humanist AI Code of Conduct.
  12. 2026-09-12 Amodei published "We Must Pace the Frontier," the first time the position was stated under a chief executive's name.
  13. 2026-09-09 The number the argument would be conducted in entered the record. Coxon resigned and posted first, giving none; Hubinger replied under that thread with a personal ">10% within the next decade," and the qualifiers came off as it travelled.
  14. 2026-09-06 Pachocki published "An Alien Mind," conceding that no lab has solved alignment well enough to keep scaling at full speed — six days before Amodei, and therefore first.
  15. 2026-07-28 1,134 frontier-lab employees signed an open letter asking Washington to support an international effort to pace automated AI development. Amodei was among the signatories.

People

Theme
Language
Support
© funclosure 2025