Four questions every leader must ask before buying an “AI” tool

How to tell real artificial intelligence from plain automation before you sign the next technology contract or swallow the latest vendor hype.

Article link: https://open.substack.com/pub/johanosteyn/p/four-questions-every-leader-must?r=73gqa&utm_campaign=post&utm_medium=web&showWelcomeOnShare=true

Audio summary: https://youtu.be/xVVTCgkGYqo

Follow me on LinkedIn: https://www.linkedin.com/in/johanosteyn/

I write about various issues of interest to me that I want to bring to the reader’s attention. While my main work is in Artificial Intelligence and technology, I also cover areas around politics, education, and the future of our children.

In almost every executive workshop I run, someone proudly announces that their organisation is “already using AI”. When we unpack what that means, it often turns out to be classic automation: dashboards, rules-based workflows or robotic process automation. There is nothing wrong with that; in fact, it is exactly where most companies should start.

The problem comes when vendors label every digital product as “artificial intelligence”, and leaders lose the ability to separate solid automation from genuine AI capability. Over the years, I have developed a simple slide to cut through the noise: if a system predicts, self-learns, finds patterns in massive data, or delivers answers you did not explicitly ask for, it is probably AI. If it does not, it is probably just automation – and that is perfectly fine.

CONTEXT AND BACKGROUND

The confusion is understandable. Automation and AI live on the same spectrum. Both involve computers doing work that people used to do manually. Many organisations have spent the last decade digitising forms, building workflow systems, creating dashboards and using robotic process automation to move data between systems. These investments have delivered real value: fewer errors, faster processing, better visibility. They are not second-class citizens; they are the foundations on which any meaningful AI programme will stand.

But the marketing language has raced ahead of reality. Every platform is now a “smart assistant”, every dashboard a “prediction engine”, every chatbot an “AI agent”. Boards are told they must “do AI” to stay competitive, and executives sometimes feel compelled to buy whatever is labelled accordingly. The danger is not only wasted money. When leaders think they have already implemented AI, based on modest automation projects, they may stop short of the genuinely transformative opportunities that real AI – used wisely – can unlock.

INSIGHT AND ANALYSIS

To steer through this hype, business leaders need a simple, practical test. Before you call something AI, ask four questions.

First: does it predict the future, or does it only report the past? A dashboard that tells you how many claims you processed last month is useful, but it is not AI. A model that uses historical data to predict which claims are likely to be fraudulent, or which customers are likely to churn, is closer to genuine artificial intelligence.

Second: does it keep improving itself? Traditional automation behaves exactly as it was programmed; any improvement requires a human to change the rules. An AI system, by contrast, learns from new data. Its performance on tasks such as recommendations, classification or language generation should improve as more examples flow through, subject to proper governance.

Third: does it see patterns that humans would struggle to spot? Real AI can sift through huge volumes of complex, messy data to uncover relationships that are not obvious: subtle risk signals, unusual combinations of behaviour, and emerging customer segments. If the “AI” simply follows a fixed flowchart of if-then rules, you are back in automation territory.

Fourth: can it provide useful answers to questions you did not explicitly script? A rules-based chatbot can respond to a fixed menu of queries. A more advanced system can handle open questions, propose next best actions, or surface insights you did not predefine. It will still make mistakes, and it must be monitored, but the underlying capability is different.

IMPLICATIONS

For executives, these four questions are not academic. They shape budgets, expectations and risk. If a proposed solution fails all four tests, treat it as automation. That is not an insult; it simply means you should evaluate it on classic criteria: process fit, reliability, cost, and integration. Do not pay AI prices for non-AI technology, and do not promise an AI-style transformation to your board.

Where a solution meets one or more of the AI tests, the conversation changes. You will need to think about data quality, model governance, bias, monitoring and skills. Self-learning systems demand new forms of oversight. Predictive systems change how you manage risk and accountability. Pattern-finding systems can surface spurious correlations as well as real opportunities, so human judgment remains essential.

CLOSING TAKEAWAY

The easiest way to regain control of the AI conversation is to stop arguing about labels and start asking better questions. Prediction, self-learning, pattern detection at scale and unscripted answers are not perfect definitions, but together they give leaders a clear, practical lens. Most organisations still need far more boring automation than glamorous artificial intelligence, and that is no bad thing.

The real danger is not that you are behind on AI; it is that you cannot tell the difference between genuine intelligence and rebranded workflows. If you can master that distinction, you will make better investment decisions, protect your teams from disappointment and focus your scarce resources on the places where AI can truly move the needle.

Author Bio: Johan Steyn is a prominent AI thought leader, speaker, and author with a deep understanding of artificial intelligence’s impact on business and society. He is passionate about ethical AI development and its role in shaping a better future. Find out more about Johan’s work at https://www.aiforbusiness.net