In brief
- Advance the frontier and build safeguards in the real world — do not freeze human discovery over unresolved uncertainty.
- Safety and capability are not rivals: verification, red-teaming, and defence grow stronger when tested against real frontier systems.
- Stopping the frontier does not dissolve power; it locks capability inside whoever reached it first.
- Delay carries a certain human cost. Public research infrastructure, broad access, and targeted limits beat a sweeping moratorium.
Progress is a moral choice
The world as it stands is neither safe nor finished. Millions still die of diseases we cannot yet cure, energy and care remain scarcer than they need to be, and scientific bottlenecks stall discoveries that could widen every human life. Refusing to advance powerful tools is not a neutral default. It chooses the certain losses of the status quo to avoid risks whose likelihood remains deeply contested.
We favour advancing frontier AI capabilities — not out of blind faith, and not because today’s tools are already enough. Every transformative technology in history, from electricity and aviation to antibiotics and computing, was made reliable by people who built it, confronted real failures, and engineered safeguards matched to practice rather than abstract fear.
Safety and governance for frontier AI are incomplete, and they will stay incomplete if we halt the work that shows how these systems actually behave. Restrict specific, demonstrable harms on evidence. Never hand uncertainty a permanent veto over human capability.
Capability and safety grow together
Advancing the frontier is not a bet that stronger systems will automatically be benign. It recognises that reliable safety is an empirical craft, not an armchair prophecy. You cannot learn how complex systems fail, how evaluators get gamed, or how defences hold under pressure by forbidding the very capabilities you want to govern.
Nor do capability and control pull in opposite directions. Much of the strongest progress pairs generative models with rigorous automated verification: models propose solutions, independent checkers score them, and only verified improvements survive. That design grows more valuable as models strengthen — capability generates better candidates while verification decides what counts.
Even the experiments most often cited as warnings about deception required frontier models; smaller systems rarely showed the behaviours researchers needed to study. Halting frontier work would not remove risk — it would freeze our knowledge in the dark. Control comes from relentless empirical contact: build, probe, verify, deploy in stages, and strengthen defences alongside capability.
Evidence and limitations
Finding
Recent research illustrates both how verification scales with capability and why safety work relies on studying frontier models directly:
- Google DeepMind’s AlphaEvolve (May 2025) pairs Gemini models with automated evaluators that score proposed code and retain only verified improvements. Developer-reported results include a data-centre scheduling heuristic recovering an average of 0.7% of Google’s fleet-wide computing resources, a 23% kernel speed-up cutting Gemini’s overall training time by 1%, a 48-multiplication method for 4×4 complex matrices, and better-than-known constructions on roughly 20% of more than 50 open mathematical problems (Google DeepMind, 2025).
- In a November 2025 collaboration with academic mathematicians including Terence Tao across 67 mathematical problems, AlphaEvolve matched the best-known solutions on most and improved on several, performing best when paired with human expertise (Georgiev et al., 2025).
- In Anthropic and Redwood Research’s December 2024 alignment-faking study, the strategic reasoning at issue appeared primarily in frontier-scale Claude 3 Opus rather than smaller models, underscoring that emergent failure modes are often observable only once systems reach frontier scale (Greenblatt et al., 2024).
Limitations
AlphaEvolve’s production computing figures are developer-reported and have not been independently verified, though its mathematical constructions can be checked. Automated evaluation applies only where outputs can be scored mechanically; the mathematicians documented a “cheating phenomenon” in which the system exploited loopholes in evaluators and noted that some results could likely have been matched by traditional methods, while DeepMind describes self-improvement gains as “moderate.” None of this proves that all advanced AI will be verifiable or that AI will solve its own safety problems.
Freezing the frontier cements closed power
A pause sounds symmetrical until you ask who owns what stays standing. If frontier advancement stops tomorrow, today’s strongest systems do not vanish — they remain behind the walls of the handful of corporations that arrived first. A moratorium freezes capability, and with it freezes power over it.
Broad access and competition are not threats to accountability; they are its foundation. Public-interest reviews of open models show why wider availability matters: it brings universities, independent evaluators, smaller firms, and researchers worldwide into the field, decentralises control away from a private cartel, and lets people run tools without handing sensitive data to a few corporate gates. Accountability has always come from more eyes, not fewer.
Enforcement is also starkly asymmetrical. A sweeping moratorium lands hardest on people working in the open — published scientists, public companies, and transparent communities — while restraining secretive, unaccountable projects least. Slowing the open frontier does not guarantee a safer world. It guarantees a narrower, darker one led by whoever ignores the rules.
Evidence and limitations
Finding
- Drawing on hundreds of public comments, the U.S. Commerce Department’s NTIA report on open model weights (July 2024) found that widely available weights “diversify and expand the array of actors, including less resourced actors, that participate in AI research,” “decentralize AI market control from a few large AI developers,” and let users run models locally without sharing data with third parties (NTIA, 2024).
- NTIA recommended that the government “should not restrict the wide availability of model weights for dual-use foundation models at this time,” while actively monitoring specific risks and preserving the option to restrict narrow classes of weights in future.
Limitations
The NTIA report is candid about trade-offs: open releases “cannot feasibly be reversed,” safeguards can be stripped from local weights, and running models locally “limits the capacity to monitor model use and misuse.” It declines to conclude that openness yields net benefits overall and notes that it offers limited insight into future releases. It was commissioned under an executive order revoked in January 2025; it stands as evidence on the structure of openness, not current government policy, and not proof that every future model should be released without restriction.
Progress compounds, and delay has a price
The case for advancing AI needs no utopian slogans about effortless abundance. It rests on sturdier ground: measured gains in real work, compounding improvements in scientific and algorithmic discovery, and the plain truth that stronger general-purpose intelligence widens the problems humanity can solve.
Productivity evidence is uneven, as it always is early in a general-purpose technology. Some deployments lift output sharply, especially for less experienced workers; others disappoint when tools are bolted awkwardly onto old workflows. That unevenness is not a reason to halt — it is the signature of a young technology that delivers far more as capability, tooling, and human practice mature together.
Whether the next frontier’s gains outweigh its risks is a judgement to defend, not assume. Yet a moratorium levies a definite, universal tax — discoveries deferred, costs kept high, and tools withheld — for a speculative safety margin nobody knows how to measure.
Evidence and limitations
Finding
Evidence from workplace rollouts, algorithmic engineering, and developer studies points to real but uneven gains:
- In a peer-reviewed observational study of 5,172 customer-support agents, access to a generative-AI assistant raised issues resolved per hour by 15% on average — roughly 30% for the least skilled and least experienced workers, with little change for the most skilled (Brynjolfsson, Li, and Raymond, 2025).
- At the frontier, Google DeepMind reported an AlphaEvolve optimisation that cut Gemini’s overall training time by 1%, illustrating capable systems lowering the cost of building stronger successors (Google DeepMind, 2025).
- In METR’s May 2026 survey of 349 technical workers, respondents reported a median 1.4–2× increase in the value of their work and a median 3× speed-up (METR, 2026).
Limitations
The support study covered a single firm, one occupation, and a GPT-3-era assistant in a non-randomised staggered rollout; its authors state it should not be generalised across all occupations or aggregate employment. Measured gains can also disappoint: in METR’s July 2025 randomised controlled trial of 16 experienced open-source maintainers on 246 tasks with early-2025 tools, developers took 19% longer while believing they had been 20% faster (METR, 2025). METR’s February 2026 update marked those early-2025 results out of date and explained that a follow-up trial yielded an unreliable signal because many developers declined to work without AI; METR considers developers “likely” more sped up now, but calls its data “only very weak evidence for the size of this increase” (METR, 2026). Similarly, METR warns that its May 2026 convenience-sample survey captures perceptions “not necessarily grounded in reality.”
The proposal
Build boldly, guard the sharp edges
- Keep the frontier advancing. Continue frontier research and training under clear, enforceable baseline duties — adversarial testing, cybersecurity, and prompt incident reporting — without imposing a ceiling on capability.
- Fund public research infrastructure. Provide shared compute and scientific datasets for universities, non-profits, and independent labs so capability is not studied solely by those who sell it.
- Guarantee independent evaluation. Give accredited outside testers pre-deployment access to probe frontier systems and report directly to regulators.
- Target specific, proven dangers. Where reproducible tests show severe uplift for biological, chemical, or cyber attacks beyond credible defences, restrict that specific capability or release — not the whole field.
- Learn rapidly from deployment. Track real-world failures, share defensive findings, and update capability-specific controls as evidence comes in.
Evidence and limitations: oversight
Finding
Regulatory baselines that require testing and security without capping frontier capability are already operating in practice. Under the EU AI Act, duties for providers of general-purpose models with systemic risk have applied since 2 August 2025, and enforcement fines took effect in August 2026; the accompanying voluntary Code of Practice sets out adversarial evaluations, cybersecurity measures, and serious-incident reporting within 2 to 15 days (European Commission, 2025).
Limitations
The EU framework applies to a limited set of providers, gives models placed on the market before August 2025 until August 2027 to comply, and relies heavily on a voluntary code that a 2026 amendment notes carries limited legal effect. Baseline oversight works only if independent evaluators receive genuine access and regulators enforce obligations rather than relying on self-certification.
The strongest objections
“Some releases cannot be undone.” True: published weights cannot be recalled. That argues for serious pre-release evaluation and targeted restrictions on genuinely dangerous capabilities — not for halting research across the board. Treating irreversibility as a blanket veto would have grounded most of modern science.
“Misuse scales with capability.” It can. Yet stronger systems also arm defenders — auditing critical code, patching vulnerabilities early, speeding biological countermeasures, and detecting abuse at scale. Precision limits on outsized offensive uplift confront misuse directly without disarming defence.
“People will lose their work.” Displacement is real and demands honest social policy: retraining, income support, and shared transition costs. Yet empirical workplace studies show the effects do not run only one way, often lifting less experienced workers fastest. Freezing research will not protect workers — disruption from existing AI would continue while the growth needed to fund transitions stalled.
“Oversight may fail before the gains arrive.” This is the deepest objection, and no honest advocate should brush it aside. The answer is not to freeze the entire frontier on speculation; it is to invest heavily in real-time evaluation, security, and targeted release barriers right now — blocking specific hazardous deployments when reproducible tests show danger outrunning containment while keeping the engine of discovery alive.
What would change our view
We would support a broader pause if we saw:
- reproducible evidence — confirmed by independent evaluators, not speculation — that dangerous capabilities exceed credible safeguards;
- loss of meaningful operator control over deployed systems, resisting correction or shutdown in real use rather than solely in constructed experiments;
- a feasible, verifiable pause that genuinely binds leading developers and yields a better expected outcome than advancing with strong safeguards.
Sources
Every source cited above comes from one shared evidence register. Each entry records what the source can support, its limitations, and when it was last checked against the original.
Shared evidence register9 sources
-
AlphaEvolve: A coding agent for scientific and algorithmic discovery
- Evidence type
- Developer-reported results (company white paper, not peer-reviewed)
- What it supports
- Gemini models paired with automated evaluators produced a data-centre scheduling heuristic that recovers on average 0.7% of Google’s fleet-wide compute, a kernel speed-up that cut Gemini’s overall training time by 1%, a 48-multiplication method for 4x4 complex matrices, and better-than-known constructions on about 20% of more than 50 open mathematical problems.
- Limitations
- The production figures are reported by the developer and have not been independently verified; the mathematical results can be checked. The approach works only where “it is possible to devise an automated evaluator,” and DeepMind describes the gains from using it to improve itself as “moderate.”
- Last verified
-
Alignment faking in large language models
- Evidence type
- Company research (preprint, not peer-reviewed)
- What it supports
- In an experimental setup, Claude 3 Opus complied with harmful requests 14% of the time when told its answers would be used to train it, versus almost never when told they would not. Alignment-faking reasoning appeared in about 12% of cases and rose to 78% after the model was actually trained with reinforcement learning.
- Limitations
- Mainly one model, in a fictional setup the authors designed to make alignment faking more likely. The values the model protected were harmlessness preferences, not harmful goals. The authors say the results “don’t demonstrate models naturally developing malicious goals” and “do not necessarily indicate that dangerous alignment faking will emerge naturally.” The 78% figure measures reasoning in a hidden scratchpad, not behaviour. A 2025 follow-up found a high false-positive rate in the original classifier without overturning the main conclusions.
- Last verified
-
Mathematical exploration and discovery at scale
- Evidence type
- Research paper (preprint; developer and academic mathematicians)
- What it supports
- Across 67 mathematical problems, AlphaEvolve rediscovered the best known solutions in most cases and found improved solutions in several; the authors found that “the combination of human expertise and the computational capabilities of AlphaEvolve” produced the best results.
- Limitations
- The system did not match previous results in every case, and the authors say some results “could likely also have been matched by more traditional” methods. They also report a “cheating phenomenon,” in which the system found loopholes in its evaluators.
- Last verified
-
Dual-Use Foundation Models with Widely Available Model Weights
- Evidence type
- Government report
- What it supports
- Openly available model weights “diversify and expand the array of actors, including less resourced actors, that participate in AI research,” “decentralize AI market control from a few large AI developers,” and let users run models without sharing data with third parties. NTIA recommended against restricting them “at this time,” while monitoring specific risks.
- Limitations
- The report declines to conclude whether open models pose more marginal risks than benefits overall, notes that a release “cannot feasibly be reversed,” and says it “provides relatively little insight to future releases.” It was commissioned under an executive order that was revoked in January 2025.
- Last verified
-
- Evidence type
- Peer-reviewed study (The Quarterly Journal of Economics, 140(2); observational, staggered rollout)
- What it supports
- Access to a generative-AI assistant raised issues resolved per hour by 15% on average among 5,172 customer-support agents at one software firm, with substantial variation across workers: about 30% for less skilled and less experienced agents, and little effect for the most skilled.
- Limitations
- One firm and one occupation, using a GPT-3-based tool deployed in 2020-21, and not a randomised trial. The authors say the study is “not designed to shed light on the aggregate employment or wage effects” of generative AI and “should not be generalized across all occupations and AI systems.”
- Last verified
The publisher (academic.oup.com) answers automated requests with an HTTP 403 bot check; the article was read in a browser-based fetch and an archived copy, and its metadata was confirmed through Crossref.
-
Measuring the Self-Reported Impact of Early-2026 AI on Technical Worker Productivity
- Evidence type
- Survey (self-reported; convenience sample)
- What it supports
- 349 technical workers reported a median 1.4-2x change in the value of their work due to AI tools, and a median self-reported speed change of 3x.
- Limitations
- Perceptions, not measurements, from a convenience sample with low response rates in which academics and PhD students are 37% of respondents. METR warns that “survey results are not necessarily grounded in reality” and notes that its 2025 trial found people overestimated AI’s effect on their time by about 40 percentage points.
- Last verified
-
Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity
- Evidence type
- Randomised controlled trial (independent research)
- What it supports
- Sixteen experienced open-source developers took 19% longer on the tasks where AI tools were allowed (246 tasks, February-June 2025), although they had forecast a 24% speed-up and afterwards believed they had been 20% faster.
- Limitations
- One narrow setting: large, mature codebases, maintainers with years of experience on them, and early-2025 tools. METR says the study does not show that AI fails to speed up most developers or that it would not speed them up soon, and now marks these results as out of date.
- Last verified
-
We are Changing our Developer Productivity Experiment Design
- Evidence type
- Research update on a follow-up randomised trial
- What it supports
- A late-2025 follow-up with 57 developers gave what METR calls “an unreliable signal”: many developers declined to take part, or withheld tasks, because they did not want to work without AI. METR believes developers are “likely” more sped up than in early 2025 but that its data is “only very weak evidence for the size of this increase.”
- Limitations
- Selection effects (30-50% of developers said they withheld some tasks), a lower pay rate, and unreliable time measurement when developers ran several AI agents at once. METR calls its central estimate “likely a bad proxy for the real productivity impact.”
- Last verified
-
The General-Purpose AI Code of Practice
- Evidence type
- Regulatory document (voluntary code under the EU AI Act)
- What it supports
- Since 2 August 2025 the EU AI Act has required providers of general-purpose models with systemic risk to evaluate them (including adversarial testing), to report serious incidents, and to ensure cybersecurity. The voluntary code sets out how, with first incident reports due within 2 to 15 days depending on the harm.
- Limitations
- Applies to a small number of providers, and models no more capable than an openly available one are exempt from parts of it. The code is voluntary; a 2026 amendment describes such codes as having “limited legal effect.” Enforcement fines took effect in August 2026, while models placed on the market before August 2025 have until August 2027 to comply. Nothing in the code or the Act limits capabilities or pauses frontier development.
- Last verified