The Dark Side of AI Personalisation Nobody in Your Business Is Talking About

The more precisely AI can target your customers, the more precisely it can betray them — and the difference is a transparency decision most organisations are avoiding

Video summary: https://youtu.be/RsON0X0eVRM

Sign up for my Substack daily AI newsletter here.

See my AI Training course portfolio for corporate Business Leaders here.

Article link: https://open.substack.com/pub/johanosteyn/p/the-dark-side-of-ai-personalisation?r=73gqa&utm_campaign=post&utm_medium=web&showWelcomeOnShare=true

There is a deeply held belief in business that loyalty should be rewarded. Customers who return, who spend consistently, who choose you over competitors despite having alternatives — these are the customers every organisation talks about cherishing. But there is a growing and troubling body of evidence that AI personalisation systems, deployed without adequate governance or ethical scrutiny, are identifying precisely these customers and extracting more from them rather than offering them more. The loyalty trap is real. And most businesses have not yet had the honest conversation about which side of it they are on.

CONTEXT AND BACKGROUND

The mechanics of AI-driven personalisation begin with a simple commercial logic: Algorithms analyse browsing history, purchase patterns, device type, location, time of day, and dozens of other data signals to build a behavioural profile of each customer. That profile is then used to predict something specific: how much this individual is willing to pay, or how likely they are to respond to a given offer. On the surface, this sounds like personalisation in service of the customer. In practice, what the algorithm is frequently identifying is not a customer’s preference but a customer’s vulnerability — their predictability, their lack of time to shop around, their habit of returning without comparing, their attachment to a brand they trust.

A landmark investigation by Consumer Reports, Groundwork Collaborative, and More Perfect Union in late 2025 brought this dynamic into sharp public focus. Researchers found that 74% of grocery items on major delivery platforms appeared at multiple price points simultaneously, with identical items priced up to 23% higher for some customers than others. The determining factor was not supply, demand, or timing in any meaningful sense. It was what the algorithm had inferred about each customer’s willingness to pay. The investigation triggered congressional scrutiny, regulatory investigations across multiple US states, and an immediate halt to AI pricing experiments by Instacart.

By early 2026, more than 40 algorithmic pricing bills had been introduced across 24 US states, as documented by Inside Privacy. The loyal, habitual customer — the one who orders the same groceries every week without checking prices — turned out to be the algorithm’s most profitable target.

CONTEXT AND BACKGROUND

This is not an isolated phenomenon confined to grocery delivery. A Brandeis University economist who has studied AI-driven pricing since 2014 described the dynamic plainly in August 2025: the concern is not just scale and intensity, but who ends up paying more. The assumption is that AI pricing targets the wealthy. The reality is more troubling. As the researcher noted, the algorithm might identify a time-pressed customer who lacks the opportunity to compare prices and charge them a premium precisely because of that constraint.

Delta Air Lines faced a similar reckoning in May 2025 when an investigation by travel publication Thrifty Traveller exposed that the airline was charging solo passengers up to 69% more per ticket than groups booking the same seats on the same flights — with the fare rules explicitly requiring at least two adults on a booking to qualify for the lower price. Both Delta and United rolled back the practice within days following a public backlash that spread widely across social media. As Fortune reported at the time, the theory was straightforward: solo travellers are disproportionately business travellers who expense their tickets and are less price-sensitive than leisure groups, making them the most profitable target for algorithmic pricing. The loyal frequent flyer, the habitual solo business traveller, the customer who values predictability — these are precisely the people the algorithm found and charged more.

Research from Carnegie Mellon University adds a further dimension to this problem. The study found that personalised ranking systems on e-commerce platforms — the algorithms that determine which products appear at the top of search results for each individual — may actually enable pricing algorithms to charge higher prices overall. Because personalisation reduces the price sensitivity that would otherwise drive competition, loyal customers who engage deeply with a platform end up in an environment where the algorithmic infrastructure is working against their financial interests. In other words, the more engaged and loyal the customer, the more effectively the system can extract value from them.

INSIGHT AND ANALYSIS

The deepest problem here is not the technology. It is the misalignment between what businesses say they value and what their algorithms are actually optimising for. Most organisations will state, sincerely, that their best customers are their most important asset. Their loyalty programmes promise rewards, recognition, and preferential treatment. And simultaneously, their AI personalisation systems are scanning those same customers for signals of reduced price elasticity and using that information to charge them more, offer them less, or communicate with them differently based on algorithmic assessments of what they can be made to accept.

Currency Alliance’s analysis of loyalty trends for 2026 identifies a significant and underappreciated risk emerging from exactly this dynamic: as AI tools give consumers greater ability to find the best available price across platforms, switching costs are falling, and brand loyalty is becoming easier to abandon. The Economist forecast in late 2025 that the democratisation of price information through AI search tools could signal the end of what it called the rip-off economy — a moment when customers can no longer be charged above the fairest price without knowing it. If that prediction is correct, the businesses that have built revenue models around extracting premium from loyal customers through opaque personalisation are sitting on an increasingly unstable foundation.

The loyalty programme itself has become part of this problem. These programmes were designed to reward customers for their continued patronage. In many cases, they have become sophisticated data collection mechanisms that feed AI systems with precisely the behavioural information needed to identify and exploit the customers who trust the brand most. The data a loyal customer shares through a rewards programme — their purchase frequency, their price sensitivity, their household composition, their shopping patterns — is the same data an AI personalisation system uses to determine how much that customer can be charged. TRIFFT’s analysis of loyalty trends for 2026 acknowledges this tension explicitly, noting that customers are increasingly aware of their data’s value and will no longer give it away without meaningful reciprocity

IMPLICATIONS

For business leaders, the question this evidence demands is direct and uncomfortable: what is your AI personalisation system actually optimising for? If the honest answer is revenue extraction from the customers most likely to accept it without comparison shopping, then the loyalty relationship you believe you have with those customers is built on a foundation that is eroding — and will eventually collapse, either through regulatory intervention, through AI-empowered consumer defection, or through the kind of viral discovery that turns a pricing scandal into a brand crisis overnight.

The businesses that will sustain genuine customer loyalty through the AI era are those that use personalisation to serve rather than to extract. That means using the data customers share to offer them genuinely better value, not to identify the ceiling of what they will pay. It means building transparency into personalisation systems so that customers can understand why they received the offer or price they did. And it means periodically asking, honestly, whether the organisation’s best customers are being treated better or worse than they would be if they were new customers with no behavioural history at all. The answer to that question, in many businesses today, is the answer the algorithm does not want anyone to examine.

CLOSING TAKEAWAY

Loyalty is one of the most valuable things a customer can offer a business. It represents trust extended over time, preference maintained despite alternatives, and a relationship built through repeated positive experience. When AI personalisation systems identify loyalty as a commercial vulnerability to be monetised, they are not optimising for customer value. They are consuming it. The businesses that will earn enduring customer relationships in the AI era are those that understand the difference — and make the deliberate choice to deploy personalisation in ways their most loyal customers would recognise as deserved, not discover as a betrayal.

Author Bio: Johan Steyn is a prominent AI thought leader, speaker, and author with a deep understanding of artificial intelligence’s impact on business and society. He served as a working group member contributing recommendations toward South Africa’s national AI strategy, an initiative by the National Advisory Council on Innovation, the Council for Scientific and Industrial Research, the Human Sciences Research Council, and the Department of Science and Innovation. He is passionate about ethical AI development and its role in shaping a better future. Find out more about Johan’s work at https://www.aiforbusiness.net