When the AI Builders Disagree, You Must Govern Yourself

The people who build the world's most powerful AI cannot agree on whether to slow it down, and that disagreement is the clearest warning business leaders will get.

Sign up for my Substack daily AI newsletter here.

See my AI Training course portfolio for corporate Business Leaders here.

Article link: https://johanosteyn.substack.com/p/when-the-ai-builders-disagree-you?r=73gqa&utm_campaign=post&utm_medium=web&showWelcomeOnShare=true Follow me on LinkedIn: https://www.linkedin.com/in/johanosteyn/

The people who build the world’s most powerful artificial intelligence cannot agree on whether to slow it down. In a single week, Anthropic’s Dario Amodei called for the industry to ease the pace of development, Sam Altman and Elon Musk agreed, and Mark Zuckerberg publicly refused. For leaders watching from Johannesburg or Cape Town, the temptation is to treat this as a distant quarrel between American billionaires. That would be a mistake. When the architects of a technology disagree this openly about its dangers, they are telling every organisation that adopts their systems something important. The safety of these tools is contested, unsettled and unresolved. The argument that follows is simple: the disagreement itself is the signal, and the only sound response is to govern the risk yourself.

CONTEXT AND BACKGROUND

The split broke into the open in September. Anthropic’s chief executive, Dario Amodei, argued that the industry must slow the rate at which AI models gain new capabilities. Sam Altman of OpenAI said the sector needs to pace the frontier, and Elon Musk backed the call. The worry centres on recursive self-improvement, the point at which AI systems help design the next generation of models, leaving less room for human oversight with each cycle.

Days later, Mark Zuckerberg refused to join them. Meta’s founder argued that competition and legal liability already push labs to behave responsibly, so no industry-wide pact is needed. He went further, claiming that trust and alignment are quickly becoming the most important capabilities that will differentiate agents and models. Meta cites its independent evaluators and its delayed Muse agent as proof that restraint works without a pact, and points out that it directs most of its compute to customer-facing products rather than recursive self-improvement.

The politics sharpen the divide. President Trump has played down the need to check AI development, saying he does not want to cede the edge to China and that whoever wins with AI wins.

INSIGHT AND ANALYSIS

The most important fact here is not that Zuckerberg is right or that Amodei is right. It is that they disagree at all. Two of the most capable AI laboratories on earth, with access to the same evidence, cannot agree on whether the current pace is safe. For a bank, an insurer or a university adopting these systems, that uncertainty is the entire point. If the people who build the technology cannot offer certainty, no procurement team should expect to buy it in a sales presentation.

The dispute is murkier than a simple safety debate. When AI labs propose coordinating on safety standards, regulators sense something else at work. The chair of the Federal Trade Commission, Andrew Ferguson, said his alarm bells go off when companies seek antitrust exemptions while lobbying for new rules, warning that they may be trying to insulate their incumbency from challenge. Aidan Gomez, chief executive of the rival firm Cohere, called the coordination effort a cartel by another name. Safety and self-interest are difficult to separate.

I have previously written about this, arguing that the gap between what frontier labs say in public and what their own researchers believe in private is a governance problem, and that organisations should add supplier due diligence to their risk registers and ask hard questions about a lab’s real assessment of risk.

IMPLICATIONS

For South African leaders, the temptation is to treat all of this as a foreign quarrel and wait for the winners to be declared. That is the wrong instinct. Vendor assurances are marketing, and marketing is not governance. An organisation that deploys a foreign model remains accountable for what that model does in its name.

The response is to govern the risk internally rather than outsource it. Boards should treat AI suppliers as they would any other material risk, with due diligence, documented safety evidence, and the right to audit. They should refuse contract terms that grant a provider full autonomy inside the business while shielding it from the consequences, and they should name a human owner for every automated decision. This is the heart of cognitive governance. Humans must remain net contributors to the work, and accountability cannot be handed to a system or to the company that sold it.

CLOSING TAKEAWAY

The disagreement among the people who build these systems is not noise to tune out. It is the clearest signal boards will receive that the safety of AI is contested and unresolved. The sensible response is neither panic nor blind faith, but ownership. Decide what these tools may do in your name, demand evidence rather than promises, and keep a human answerable for every consequential decision. The labs may eventually agree, or they may not. Either way, the organisation that governs itself will be ready, and the one that waited for consensus will find that no one was ever coming to provide it.

Author Bio: Johan Steyn is a prominent AI thought leader, speaker, and author with a deep understanding of artificial intelligence’s impact on business and society. He is passionate about ethical AI development and its role in shaping a better future. Find out more about Johan’s work at https://www.aiforbusiness.net