Restraint or Regulatory Capture?

The frontier labs’ call to pace AI may be sincere about safety and convenient for market position at the same time.

Sign up for my Substack daily AI newsletter here.

Article link: https://johanosteyn.substack.com/p/restraint-or-regulatory-capture?r=73gqa&utm_campaign=post&utm_medium=web&showWelcomeOnShare=true

Follow me on LinkedIn: https://www.linkedin.com/in/johanosteyn/

In September, Anthropic’s chief executive, Dario Amodei, argued that the AI industry must slow the rate at which its models gain new capabilities, so that safety work can keep up. He proposed embedded third-party evaluators, common safety standards among companies in democratic countries, and tighter controls on the chips sold to China. For executives watching from Johannesburg or London, the first instinct is to welcome an industry that finally admits its risks. The better instinct is to ask a second question: who benefits from the particular form of restraint being proposed?

CONTEXT AND BACKGROUND

The call did not arrive in a vacuum. Within a day, Sam Altman said OpenAI would follow the evaluator approach and Elon Musk posted his agreement. The journalist Brian Merchant argued that such proposals mainly serve Anthropic and OpenAI, a form of regulatory capture. Regulatory capture describes rules that end up protecting the companies they were meant to govern. The timing sharpens the suspicion. Both companies are approaching public listings, with Anthropic’s reportedly possible as early as November. This month, an academic working group went further than Amodei. Where he proposes pacing, meaning slower capability gains under independent scrutiny rather than a halt, the group examined a full pause. It concluded that a mutually verified global halt to frontier AI training, enforced through the concentrated chip supply chain, is feasible. Its authors concede that such a pause could entrench incumbents as government-sanctioned oligopolies.

INSIGHT AND ANALYSIS

Both readings can be true at once. The risks are real. The International AI Safety Report, guided by experts nominated by more than thirty countries and international organisations, describes a real dilemma: act on limited evidence and risk poor policy, or wait and leave society exposed. Sincerity about danger does not cancel self-interest in the remedy. A rule that slows everyone equally still favours whoever is furthest ahead when the clock stops.

The design details show where the advantage lies. Amodei’s plan centres on coordination among companies in democratic countries, enabled by government, alongside denying advanced chips to Chinese competitors. A framework negotiated among the leading laboratories, with official blessing, is exactly the arrangement that critics of capture warn against. Newcomers, open-source developers and the countries that buy rather than build AI are not at that table.

I have previously written about this when frontier laboratories investigated their own security incidents. I argued that when a developer acts as investigator, judge and defendant at once, its safety claims rest on its own word, and that organisations should demand independent evaluation and audit rights. Amodei’s commitment to admit embedded outside evaluators answers part of that concern, and he deserves credit for making it unilaterally. The harder questions remain. Who appoints the evaluators, who sets the standards they enforce, and who can overrule the laboratories when commercial pressure rises?

IMPLICATIONS

For Africa, the stakes are concrete, and they run through the same supply chain. Any slowdown enforced through chips and data centres gives the decisive voice to those who own them. The United States and China host roughly eighty-six per cent of global data centre capacity while Africa hosts about one per cent, and the authority to write the rules tends to follow ownership of the infrastructure. If the laboratories and their governments set the terms of restraint, countries such as South Africa lose twice: they own little of the infrastructure, and they have no seat where the standards are drafted. South Africa withdrew its draft national AI policy in April after fictitious citations were found in it. It now risks becoming a rule-taker on a question that could shape its access to AI for a decade.

For boards, the lesson applies closer to home. When a vendor volunteers to be regulated, read the proposed rules as carefully as its contract. Ask who drafted them, who will enforce them and whom they exclude.

CLOSING TAKEAWAY

Restraint at the frontier may well be necessary, and the laboratories deserve credit for saying so publicly. Credit for candour is not a mandate to design the referee. The terms of any slowdown should be set by public institutions, independent scientists and the countries that will live with the outcome, not only by the companies racing to lead it. Accountability cannot be outsourced to those being held accountable. The more sincere the laboratories are about safety, the more readily they should accept that principle.

Johan Steyn is a prominent AI thought leader, speaker, and author with a deep understanding of artificial intelligence’s impact on business and society. He is passionate about ethical AI development and its role in shaping a better future. Find out more about Johan’s work at https://www.aiforbusiness.net