Nobody Is Measuring Whether Your People Are Still Learning

Output rose, error rates fell, and the question of what happened to institutional capability was never put on the agenda.

Sign up for my Substack daily AI newsletter here.

See my AI Training course portfolio for corporate Business Leaders here.

Article link: https://johanosteyn.substack.com/p/nobody-is-measuring-whether-your?r=73gqa&utm_campaign=post&utm_medium=web&showWelcomeOnShare=true

Follow me on LinkedIn: https://www.linkedin.com/in/johanosteyn/

Every organisation deploying artificial intelligence has a productivity number it can show a board. Almost none has a capability number. That asymmetry is not a reporting oversight. It is a structural blind spot, and the education sector has now produced the evidence that should worry anyone signing off on an enterprise rollout.

CONTEXT AND BACKGROUND

The OECD published its Digital Education Outlook in January 2026, devoted entirely to generative AI in education, and its central finding deserves attention well beyond the classroom. Where tasks are outsourced to these tools without pedagogical guidance, performance improves while delivering no real learning gain, and overreliance on systems that supply direct answers reduces the active engagement on which learning depends. UNESCO IESALC has meanwhile published sixteen peer-reviewed studies examining the same territory, drawn from evidence across Latin America and the Caribbean, covering student learning, the professional development of academic staff, and the governance challenges that adoption creates. The research base on what these tools do to human capability is now substantial and growing. Almost none of it has reached the boardroom, where the same tools are being deployed at scale on the strength of efficiency projections alone.

INSIGHT AND ANALYSIS

The distinction the OECD draws is between replacement, complementarity and augmentation, and it turns entirely on where professional judgement sits. Replacement substitutes the tool for the decision. Augmentation widens the range of options a person considers before deciding, leaving the judgement with the human. Both lift measured output. Only one develops the person doing the work. Corporate adoption is being justified almost exclusively on the first measure, which cannot distinguish between the two. A UNESCO global survey spanning ninety countries found that nine in ten respondents use AI tools professionally, while more than half remain uncertain about their effective application or their wider implications, and one in four institutions has already encountered ethical difficulties ranging from overreliance to authorship disputes. Heavy use paired with shallow understanding is precisely the condition under which replacement is mistaken for augmentation, since the output looks identical either way.

IMPLICATIONS

South African universities have already acted on this. The University of Pretoria holds that students must think first and use AI later, Wits requires assessment designed to prioritise originality and critical thinking, and Stellenbosch is refining a scale determining when AI use should be prohibited, restricted, allowed, encouraged or required. Each is an attempt to protect the effort that produces competence. No comparable discipline exists in most companies, where these tools arrived through procurement rather than pedagogy, and where nobody asked what capability the displaced work was quietly building.

The European Students’ Union has warned that the long-term effects of AI-supported learning remain insufficiently understood, and that overreliance should be avoided absent adequate evidence of its impact. That caution applies with equal force to any workforce.

I have previously written about this in a piece on the risks of treating an agreeable AI as a thinking partner, which noted Gartner’s prediction that critical-thinking atrophy will push half of all global organisations to require AI-free skills assessments.

CLOSING TAKEAWAY

Three questions belong on the agenda at the next review. Which tasks in this organisation were previously developing the people who performed them, and what performs those tasks now. What measure exists, other than output, capable of revealing whether capability is still being built. Who owns that measure and reports on it to the board. None of this is an argument against adoption. It is an argument against adopting on the strength of a metric structurally incapable of detecting the cost. Productivity dashboards record what was produced. They say nothing whatsoever about who is becoming able to produce it unaided, and that is the number determining whether an organisation has a workforce in ten years or merely an installed base of licences.

Johan Steyn is a prominent AI thought leader, speaker, and author with a deep understanding of artificial intelligence’s impact on business and society. He is passionate about ethical AI development and its role in shaping a better future. Find out more about Johan’s work at https://www.aiforbusiness.net