If an AI Can Call for Your Customer, So Can a Fraudster
Contact centres must rethink identity checks before machine callers become routine, not after the first costly deception.
Sign up for my Substack daily AI newsletter here.
See my AI Training course portfolio for corporate Business Leaders here.
Follow me on LinkedIn: https://www.linkedin.com/in/johanosteyn/
On 24 September, Google began testing a feature that lets its Gemini assistant phone businesses on a customer’s behalf. It checks whether products are in stock, makes reservations and reschedules appointments, works through phone menus and waits on hold while the customer follows a live transcript, and it calls from the customer’s own number. For now, it is limited to paying Pixel 11 owners in the United States. Features like this rarely stay contained for long, and when machine callers become common, every South African contact centre will face a question it is not ready to answer: who is really on the line?
CONTEXT AND BACKGROUND
Google has built sensible limits into its agent. It introduces itself as an AI when it calls, and it will not handle financial transactions, card details or passwords. Machine callers are not waiting for Google, however. A major outsourcing firm began noticing unusual callers about 18 months ago, and staff at a credit-card company encountered an AI caller that appeared to be mapping its cancellation process. Consumers are ready for more of this. In a survey of more than 25,000 people across 16 countries, 74% said they would let an AI agent handle routine tasks such as returns and renewals, although only nine percent would let it buy on its own.
INSIGHT AND ANALYSIS
The difficulty is that everything that makes a legitimate agent convincing can be copied. A fraudster’s bot can introduce itself as an AI assistant as easily as Google’s can. It can sound polite, patient and competent, and it can call from a number the business recognises. Blocking that number is no answer when it belongs to a real customer, and the more ordinary machine callers become, the easier it is for an impostor to blend in. South Africa already pays heavily for weak verification. Digital banking fraud losses reached R2.4 billion in 2025, up 29.2%, driven mainly by social engineering such as impersonation and voice phishing, with AI making the deception more convincing.
Europe has at least settled one question. Since August, AI systems that interact with people must tell them so from the start of the conversation. South Africa has no equivalent, so a local machine caller that stays silent about what it is breaks no rule.
I have previously written about this. In an article on deepfake social engineering, I argued that the most effective defences are cheap and procedural, such as confirming instructions through an independently verified number and never through the channel that delivered them. The same principle now applies to the phone line itself.
IMPLICATIONS
The answer is to verify authority rather than voice. Contact centres should treat any caller identifying as an AI assistant as a representative, not as the customer, and allow it only low-risk tasks such as checking stock or moving a booking. Anything involving money, personal information or account changes should require the customer’s confirmation through a separate channel the business controls, such as its banking app or a registered email address. Because Google’s own agent is designed not to handle card details or passwords, a self-described assistant that asks for or offers them should be escalated at once. Businesses should also state in their terms who is bound when a customer’s agent agrees to something, since South African law already recognises contracts concluded by electronic agents and places responsibility on whoever deploys them.
CLOSING TAKEAWAY
Machine callers will bring real convenience, and customers will expect businesses to accept them. The businesses that cope will be those that stop asking whether a caller sounds right and start asking whether it has the authority to do what it asks. A familiar number, a polite voice and a confident introduction were never proof of identity. From now on, they will be the first things a fraudster copies.
Johan Steyn is a prominent AI thought leader, speaker, and author with a deep understanding of artificial intelligence’s impact on business and society. He is passionate about ethical AI development and its role in shaping a better future. Find out more about Johan’s work at https://www.aiforbusiness.net
Comments