Every university has a position on AI and assessment. Almost none has a budget
Design is the settled part. Delivery and verification are where South African higher education has published intentions and funded nothing.
Sign up for my Substack daily AI newsletter here.
Follow me on LinkedIn: https://www.linkedin.com/in/johanosteyn/
Ask any South African university what it is doing about artificial intelligence and assessment, and you will get a considered answer. Detection has been abandoned; authentic assessment is the direction of travel, and oral defences and portfolios are being piloted. The answer is correct, and it is nearly universal. Ask the same institution what the redesign costs, who will staff it, how an oral examination works for a cohort of 1,300, and how anyone will know whether the new assessment measures more than the old one did, and the conversation changes. The design argument is over. Delivery and review are where the sector has published intentions and funded very little.
CONTEXT AND BACKGROUND
The evidence that intention has not become practice is now specific. A report for Policy Exchange by Professor Philip Newton of Swansea University, drawing on Freedom of Information responses from 120 universities, found that 78% of UK institutions used remote online examinations for summative assessment, only around one in ten invigilated all of them, and 70% had no plans to reduce their number, which ran to almost 17,000 in 2023/24. That is not a sector mid-reform. It is a sector that adopted a position and left the machinery underneath largely untouched.
Australia’s regulator has gone furthest in naming what reform actually requires. TEQSA sets out three implementation pathways, programme-wide reform, unit-level assurance and a hybrid of the two, and concedes that the quickest of them may fragment the student experience and increase administrative workload. Even the regulator describing the work accepts that someone has to absorb its cost.
INSIGHT AND ANALYSIS
South Africa’s difficulty is that one answer fits only half of its students. Supervised, in-person, verifiable assessment is affordable for contact institutions that are not currently funding it, and structurally unavailable to distance institutions at any price. Nobody can timetable an oral defence for a cohort distributed across nine provinces. Distance providers run remote proctoring instead, and the terms deserve more scrutiny than they get. Unisa’s online examination rules require students to activate invigilation within a 45-minute window or have their marks withheld, treat ten cumulative minutes outside the app as a violation, and list personal network problems, load shedding, a crashed computer and non-functioning cameras as beyond the university’s control. The institution has transferred the infrastructure risk of assessment to the student.
I have previously written about this in The Students Writing In Their Third Language Were The Ones The Algorithm Flagged, where the argument was that detection tools encoded linguistic disadvantage into academic integrity enforcement. The sector accepted that argument and moved on without establishing whether the replacement is fairer. A reflective study of a 1,305-student module at the Durban University of Technology found a lower failure rate in the oral assessment, which the author attributes to students drawing on their home languages, while also recording high stress and noting that the ten minutes allocated for panel moderation was at times too little.
IMPLICATIONS
For councils and audit committees, the first question is not pedagogical. It is whether anyone has costed what the senate approved, in examiner hours per candidate, venues, invigilators and interpreters. The second is harder. The same Policy Exchange report found no standard by which it can be determined that external examiners have been adequately trained to judge whether an assessment is valid and reliable. A sector that has redesigned its instruments has no agreed method for verifying that the new ones work. This is not a plea for more money. It is a gap between what the sector mandated and what it equipped itself to do.
There is a cheaper answer, and it arrives with a warning attached. Researchers at NYU Stern deployed voice AI to conduct personalised oral examinations across two cohorts, graded by three language models that score independently then revise after reading each other, at grading costs of roughly 0.29 and 0.96 US dollars per examination against a human comparator of about 750 dollars for one cohort. The authors are careful about what that buys. Their council’s agreement, they write, establishes reliability, meaning consistency among AI raters, and not validity, meaning correctness against expert judgement.
CLOSING TAKEAWAY
A Council on Higher Education paper on epistemic inertia, which does not discuss assessment, describes African higher education systems that remain largely untransformed despite their avowed intention to do so, with surface reforms absorbed by structures that outlast them. Assessment looks like the next case in point. The sector holds the right position and has not bought the staff hours, the venues or the moderation to deliver it, and in distance education it has quietly made the student carry the difference. The next council meeting that approves an assessment policy should ask which line of the budget it sits on. A reform nobody has costed is not a reform. It is a statement.
Johan Steyn is a prominent AI thought leader, speaker, and author with a deep understanding of artificial intelligence’s impact on business and society. He is passionate about ethical AI development and its role in shaping a better future. Find out more about Johan’s work at https://www.aiforbusiness.net
Comments