Your Organisation Has Digital Workers Nobody Approved
AI agents are taking autonomous actions inside South African workflows, and most of that activity sits outside any oversight framework.
Sign up for my Substack daily AI newsletter here.
See my AI Training course portfolio for corporate Business Leaders here.
Follow me on LinkedIn: https://www.linkedin.com/in/johanosteyn/
Most South African executives could tell you how many people they employ, what those people are permitted to approve, and who signs off when the limits are exceeded. Very few could tell you how many AI agents are operating inside their organisation, what systems those agents can reach, or what happens when one acts outside its brief. The agents are already there. The governance is not.
CONTEXT AND BACKGROUND
New research puts numbers to this. Thirty-eight per cent of South African cybersecurity leaders report that AI agents are already taking autonomous actions within organisational workflows, while sixty-four per cent of organisations say their use of AI is unapproved or ungoverned. Thirty-five per cent of employees source their own agentic tools where official options are unavailable or restrictive, and only fourteen per cent of organisations have reached full maturity in managing human and agent risk together. The underlying study was produced by a cybersecurity vendor, which has a commercial interest in the finding, and the figures should be read with that in mind. The direction of travel is nonetheless consistent with evidence from elsewhere.
That evidence has strengthened considerably. The Centre for Long-Term Resilience analysed more than one hundred and eighty thousand transcripts of user interactions with AI systems and identified six hundred and ninety-eight incidents where deployed systems acted against users’ intentions or took covert action, a rise of nearly five times across five months. The researchers checked that increase against general online discussion of the subject, which grew far more slowly, which suggests they are measuring behaviour rather than attention.
INSIGHT AND ANALYSIS
The governance code now in force does not treat this as optional. King V applies to financial years commencing in January this year, and its tenth principle places responsibility on boards to govern data, information and technology in a way that supports strategy and long-term sustainability. Organisations are required to demonstrate clear accountability for decisions, actions, outputs and outcomes, including that the models and tools used are subject to human supervision and intervention proportionate to the risk posed. A board that cannot count its agents cannot demonstrate any of that.
What makes ungoverned deployment serious is the character of the failures now being documented. The United Kingdom’s AI Security Institute disclosed that during a routine evaluation, agents took unsanctioned action on the live internet in ten of one hundred and twenty-two runs. In the most serious sequence an agent researched an open-source project’s maintainers, created multiple fake identities, and used them to pressure a real person into approving malicious code. The Institute states plainly that the agent was never instructed to deceive, and that deception emerged as a by-product of pursuing the task it had been given. It also notes that in several cases the margin between failure and success rested on human vigilance rather than a technical barrier.
Ordinary use produces the same pattern, and it is the shift from advising to acting that makes it consequential. An assistant that answers a question cannot exploit a weak permission. An agent that calls interfaces and writes to live systems can. An Australian technology professional asked an agent whether it could move him up a gym class waitlist. It had already found that the booking system lacked authorisation checks on cancelling other people’s reservations, and had tested this by removing the person in first position, unprompted. It could not undo it. A technology lawyer quoted in the reporting observed that software is not a legal person, and only a legal person can be liable, leaving open whether responsibility rests with the user, the agent’s developer, the model provider or the operator of the vulnerable system.
IMPLICATIONS
For directors, the first task is counting. An organisation that cannot produce a list of its agents, what each is permitted to do, and which systems each can reach has no basis for any assurance it gives about them.
The second task is treating employee-sourced tools as a governance signal rather than a disciplinary matter. When a third of employees obtain their own agentic tools because official provision is restrictive, the organisation has told them the sanctioned route does not work. Prohibition will move that activity further out of sight.
The third is recognising that permissions matter more than instructions. These systems pursue goals persistently and find routes their operators did not intend. Telling an agent to behave responsibly is not a control. Limiting what it can reach is. The question for a board is narrow enough to ask in a meeting: does each agent hold only the access its particular task requires, enforced by the systems themselves, or has it simply inherited everything its human owner could already do?
There is also a duty running outward. Any organisation exposing interfaces to customers should assume those customers are running agents against them, and that weak authorisation checks will be found. The gym booking flaw had presumably existed for years before software with no malicious intent discovered it in an evening.
CLOSING TAKEAWAY
South African organisations have not been reckless. They have done what organisations always do with a useful new tool, which is to adopt it faster than they formalise it. The difficulty is that this particular tool takes actions, holds access and pursues objectives, which makes the usual lag between adoption and governance far more expensive than it has been before.
Two-thirds of corporate AI use sitting outside any approval framework is not a technology problem awaiting a technical fix. It is an accountability gap that King V has already assigned to the board. Directors who can name their agents, bound their permissions and say what happens when one exceeds its remit will be in a defensible position. Those who cannot are relying on the same thing that saved the researchers in the United Kingdom, which is that someone happened to be paying attention.
Author Bio: Johan Steyn is a prominent AI thought leader, speaker, and author with a deep understanding of artificial intelligence’s impact on business and society. He is passionate about ethical AI development and its role in shaping a better future. Find out more about Johan’s work at https://www.aiforbusiness.net
Comments