Shadow AI has arrived in South African healthcare, and nobody noticed
While regulators focused on medical devices, the real exposure landed on the doctor who pasted a patient’s history into a consumer chatbot after midnight.
Sign up for my Substack daily AI newsletter here.
Follow me on LinkedIn: https://www.linkedin.com/in/johanosteyn/
The South African healthcare conversation about artificial intelligence has focused on two things: the robotic surgeon that is not coming, and the regulated clinical device that is. Both are the wrong story. The real exposure has already arrived, quietly, through a door the regulators were not watching. It walks in with a doctor who finishes a sixteen-hour shift, opens a laptop at home, pastes a patient history into a free consumer chatbot, and asks it for a differential diagnosis. The chatbot is unregistered, unvalidated and uncertified for medical use, and it is now holding special personal information under the Protection of Personal Information Act. There is no audit trail of what was asked, no record of what was answered, and no way for the hospital, the medical scheme or the regulator to see what just happened.
CONTEXT AND BACKGROUND
The World Health Expo at Gallagher Convention Centre earlier this month put AI in healthcare on the national conversation agenda, with international experts warning that practitioners who do not adapt will fall behind. The regulatory response is well underway. The South African Health Products Regulatory Authority (SAHPRA) has issued guidance on AI and machine-learning-enabled medical devices, classifying software as a medical device and layering POPIA compliance over it. The World Health Organisation approved six computer-aided detection products for tuberculosis screening in June 2025, several of which have been validated against South African patient data. The regulated side of South African clinical AI is maturing. The unregulated side is where the real volume sits.
INSIGHT AND ANALYSIS
A striking inversion sits at the heart of this. The regulated radiology AI reading a chest X-ray in a Mpumalanga clinic is the safest AI in the building. It has a product file, a validation study, a classification and a clear accountability chain. The clinician using it has clear instructions. The problem is nowhere near it. The problem is the general practitioner in private practice dictating a note into a free transcription tool she downloaded last week, the registrar asking a consumer chatbot to summarise a complex patient history, and the surgeon running a bloods panel through a generic AI assistant on his phone. None of this is registered. None of it is audited. None of it is governed by anything other than the clinician’s own judgement at the moment of use.
Three failures sit inside each of those moments. The first is a POPIA failure, because special personal health information has left the hospital’s control with no operator agreement under Section 19 and an unauthorised cross-border transfer under Section 72 that nobody has papered. The second is a clinical standard-of-care failure, because the tool has not been validated for the use the clinician is making of it, and the audit trail will show nothing if a patient is later harmed. The third is a professional conduct failure. The South African Medical Journal has been direct on this point: regulatory approval of an AI-powered device does not absolve the clinician, and the duty of independent clinical judgement cannot be delegated to a vendor.
IMPLICATIONS
A July 2025 analysis by Werksmans Attorneys argued that South African healthcare has no AI-specific regulation yet, HPCSA telehealth guidelines do not address AI, and clinicians must retain full responsibility for final decisions, with institutions investing in infrastructure, training and oversight. The United States International Trade Administration reports that South African healthcare AI adoption is accelerating fastest in exactly the back-office categories that sit outside SAHPRA’s device framework, including scheduling, documentation and claims. SAHPRA and the Health Professions Council of South Africa cannot regulate what they cannot see, and most clinical AI use in the country is currently invisible to both.
Medical indemnity providers are already asking whether a practice had a sanctioned AI use policy when they adjudicate a negligence claim, and the answer increasingly decides cover. The practical move for a hospital group, a medical practice or a medical scheme is therefore immediate. A clinical AI use policy that names which tools may be used, in which contexts, and with what record-keeping, is now part of a defensible duty of care. Without it, the next complaint before the HPCSA, the Information Regulator or a court is being prepared quietly tonight.
CLOSING TAKEAWAY
The AI the South African healthcare system should worry about is not the one the regulators are already governing. It is the one in the clinician’s pocket, the one the system itself has given no instructions on, and the one that is being used in good faith by overworked professionals who have had no guidance. Shadow AI in healthcare is the same problem as shadow AI in corporate IT, with higher stakes and weaker tools. The response is the same: name it, govern it, write the policy, and give the clinicians the sanctioned path they would prefer to use. Until that happens, the back door is open, and the patient is the one who finds out last.
Author Bio
Johan Steyn is a prominent AI thought leader, speaker, and author with a deep understanding of artificial intelligence’s impact on business and society. He is passionate about ethical AI development and its role in shaping a better future. Find out more about Johan’s work at https://www.aiforbusiness.net
Comments