Managed, Monitored, and Finally Removed — By a System With No Name

Inside the algorithmic workplace, where consequential decisions get made and accountability goes to die

Audio summary: https://youtu.be/ahjD_OdKmbc

Sign up for my Substack daily AI newsletter here.

See my AI Training course portfolio for corporate Business Leaders here.

Article link: https://open.substack.com/pub/johanosteyn/p/managed-monitored-and-finally-removed?r=73gqa&utm_campaign=post&utm_medium=web&showWelcomeOnShare=true

Follow me on LinkedIn: https://www.linkedin.com/in/johanosteyn/

Across the world, a new kind of workplace dispute is quietly multiplying. A worker is flagged for underperformance. A disciplinary process is triggered. A termination notice arrives. No manager signed it. No human reviewed it in any meaningful sense. When the worker asks who made the decision, the answer is the same in Manchester, Johannesburg, and Manila: the system did. When they ask which system, the answer is that it is proprietary. When they ask who can be held to account, the silence is deafening.

This is not a hypothetical scenario. It is the present operating reality of algorithmic management, and it is accelerating faster than any legislature in the world has been able to respond.

CONTEXT AND BACKGROUND

The UK House of Commons Business and Trade Committee has launched a formal inquiry titled “Artificial Intelligence, Business and the Future of the Workforce,” examining whether current worker protections remain fit for purpose as AI embeds itself across recruitment, performance management, and employment decisions. The inquiry follows a period in which AI adoption moved from experimentation to operational infrastructure with remarkable speed, and growing concern about fairness, accuracy, and accountability in automated decision-making has made the intervention both necessary and overdue. The committee received written evidence until 3 April 2026, with oral evidence sessions continuing into mid-April, and is now preparing its report.

At the same time, the UK’s Information Commissioner’s Office has published findings on automated decision-making in recruitment, concluding that employers must ensure far more meaningful human involvement in AI-assisted processes. The ICO’s consultation on its draft guidance on automated decision-making and profiling remains open until 29 May 2026. Taken together, these developments signal something significant: the UK’s earlier light-touch, pro-innovation stance on workplace AI is being challenged by regulators who have seen what deployment actually looks like in practice.

South Africa is not observing this from a comfortable distance. The Department of Communications and Digital Technologies gazetted the Draft National AI Policy on 10 April 2026 (which was subsequently withdrawn and awaits republication), introducing a risk-tiered framework that explicitly identifies employment decisions, including hiring, promotion, discipline, and retrenchment, as high-risk AI applications requiring stringent governance. South Africa already has a dense web of legislation that applies to AI in the workplace, including the Labour Relations Act, the Employment Equity Act, POPIA, and the Basic Conditions of Employment Act, even in the absence of AI-specific statutes. The absence of an AI-specific Act does not mean a regulatory vacuum exists. It means the obligations are already there, and most organisations have not yet mapped them.

INSIGHT AND ANALYSIS

The central problem is not that AI makes bad decisions. Sometimes it makes faster and more consistent ones than humans do. The central problem is that employment law is built on an assumption it cannot currently abandon: that somewhere in the chain, a human being made a call, and that human being can be found, questioned, and held to account.

When a worker is dismissed in South Africa, the Labour Relations Act requires that the dismissal be both substantively and procedurally fair. Substantive fairness requires a valid reason. Procedural fairness requires a proper process. Both requirements presuppose a decision-maker who can be examined. An AI system has no legal personality. It cannot be cross-examined at the Commission for Conciliation, Mediation and Arbitration. It cannot explain what weight it gave to what signal. It cannot be shown to have acted in good faith or bad faith, because it does not act in any sense the law currently recognises.

Companies have learned to respond to this problem not by solving it but by managing its optics. A human is nominally present in the loop, reviewing AI recommendations before they are implemented. In many cases, that review is cursory, time-pressured, and amounts to rubber-stamping a system’s output. The human is there not because the organisation values human judgment in that moment, but because removing them entirely would cross a legal threshold. Accountability without genuine oversight is not accountability. It is the appearance of it.

This dynamic is visible in the monitoring tools that now pervade workplaces across sectors. Call centre workers are scored by systems that analyse vocal tone, keyword usage, and call resolution times. Warehouse operatives are tracked against productivity baselines that were themselves generated by AI. Back-office employees at financial services firms have their communications processed by behavioural analytics platforms that assign efficiency scores. In each case, the outputs of these systems feed directly into HR decisions, often without the affected worker knowing the system exists, let alone how it works. I have written previously about the psychological damage this kind of opacity inflicts on people inside organisations, the erosion of trust, the collapse of psychological safety, and the particular harm of being assessed by processes you cannot see or challenge. This is the same dynamic, extended into the domain of legal rights.

IMPLICATIONS

For South African business leaders, the implications of this gap are practical, not abstract. The Labour Relations Act imposes obligations of procedural and substantive fairness on all employment decisions. The Employment Equity Act prohibits both direct and indirect unfair discrimination, including discrimination that arises not through intent but through biased training data or proxy variables embedded in AI systems. POPIA requires lawful grounds for processing employee personal information and imposes specific obligations around automated profiling. An AI system designed and trained in another jurisdiction, operating on assumptions drawn from foreign demographics and legal environments, does not shed these obligations simply because it was built elsewhere.

The Draft National AI Policy goes further, requiring organisations to provide sufficiently explainable and transparent AI outputs in high-risk contexts, and to establish traceable lines of responsibility with an accountable official. It explicitly signals that AI governance in South Africa will move from voluntary to enforceable, and that employment decisions sit squarely in the high-risk category. Organisations that wait for binding obligations before building governance frameworks will find themselves behind the curve and already exposed under laws that are in force today.

There is also a workforce context that South African leaders cannot afford to ignore. Youth unemployment stands at 43.8 per cent, and the expanded unemployment rate exceeds 42 per cent. In this environment, the consequences of algorithmically-driven employment decisions extend far beyond individual workers. They land in households, communities, and a social compact already under severe strain. An unfair dismissal generated by an opaque system, with no meaningful avenue for challenge, is not only a legal failure. It is a governance failure with consequences that ripple outward in ways that no efficiency gain can offset.

The gap between regulatory speed and technological speed is also not accidental. It is a feature that has been understood and, in some industries, deliberately exploited. The gig economy spent a decade mapping the exact boundaries of employment law, learning precisely where algorithmic management triggered obligations and where it did not, and structuring its platforms accordingly. That knowledge has not remained confined to platform companies. It has circulated into enterprise HR software and the AI vendors who supply it. South African employers deploying these tools are, in many cases, inheriting the product of that legal optimisation without realising it.

CLOSING TAKEAWAY

The UK inquiry will produce recommendations. South Africa’s Draft AI Policy will move toward finalisation and, eventually, toward binding regulation. But workers are not living in the future where those protections exist. They are living in the present, where the algorithm already filed the paperwork and nobody has to answer for it.

For boards and executives, the question is not whether AI-managed workplaces will eventually be subject to stronger regulation. They will. The question is what your organisation is doing right now, before that regulation arrives, to ensure that the decisions being made in your name by systems you may not fully understand are defensible, explainable, and fair. Governance is not a compliance function triggered by legislation. It is a leadership function that precedes it.

The worker who asks who made the decision that cost them their job deserves an answer. If your organisation cannot give one, that is not the algorithm’s problem. It is yours.

Johan Steyn is a prominent AI thought leader, speaker, and author with a deep understanding of artificial intelligence’s impact on business and society. He is passionate about ethical AI development and its role in shaping a better future. Find out more about Johan’s work at https://www.aiforbusiness.net