The Hidden Mark That Proves Nothing About Honesty
AI watermarks signal how text was produced, not whether it is accurate or human, and confusing the two will cost students and job seekers.
Sign up for my Substack daily AI newsletter here.
Follow me on LinkedIn: https://www.linkedin.com/in/johanosteyn/
In early October 2026, Google opened its SynthID detector to the public, letting anyone check whether an image, video or audio file was made by one of its AI systems. This is a real step towards knowing where content comes from, yet it carries a quiet trap. As these tools spread, many will read them as lie detectors, treating a hidden mark as proof that content is fake or dishonest. That is not what the technology does. A watermark tells you something about how text or media was produced, not whether the content is true, and that gap is now a governance problem for schools, newsrooms and companies.
CONTEXT AND BACKGROUND
Text watermarking is stranger than most assume. It does not hide a secret word in a paragraph. When a language model writes, it predicts the next word one step at a time, and several continuations are usually plausible. A watermark nudges those choices in a repeatable pattern, too slight for a reader to notice, but measurable across a longer passage by a detector that knows the scheme. This differs from a general AI-writing detector, which only guesses from learned patterns; both are probabilistic, not proof. The signal is statistical, not a stamp. Google concedes that even for the images, video and audio it now covers, no single detection signal “is completely robust”, and its tool is not a general AI detector at all. For text, there is still no reliable universal detector.
The deeper point is what the signal means. A detector asks whether text carries the fingerprint of a watermarking process. It does not ask whether a human wrote the text, or whether the text is true. As one analysis puts it, failing to find a mark “does not establish human origin.”
INSIGHT AND ANALYSIS
Two mistakes follow, and both are dangerous. The first is to treat a detected mark as proof of guilt, forgetting that people edit, translate and build on AI drafts all the time. The second is to treat a missing mark as proof of innocence, forgetting that paraphrasing or translation can weaken or wash out the signal, and that many systems leave no mark at all. The limits are built in. Statistical watermarks lose reliability on short passages, which is why the European Union’s draft code can only watermark text longer than about 200 tokens and leaves shorter text effectively unmarkable.
We have already seen what happens when a probabilistic clue is treated as a verdict. Seven widely used detectors wrongly flagged about 61% of essays by non-native English speakers as machine-written, and a tool with a roughly 1% false-positive rate, applied to Vanderbilt University’s 75,000 papers, implied about 750 students wrongly accused, so it switched the tool off. The conclusion from that work is blunt: “A detector score should never be sufficient evidence of misconduct on its own.”
I have previously written about this, arguing that detectors cannot establish fabrication, that they are biased against second-language writers, and that South African universities, including the University of the Free State, the University of Cape Town and Stellenbosch, have stopped relying on them. The Free State confirmed it would drop AI detection across all faculties because the technology cannot reliably tell human from machine writing. Watermarking is the next version of the same temptation.
IMPLICATIONS
None of this means provenance is worthless. A world of unlabelled synthetic media is more dangerous, which is why the European Union now requires AI output to be marked in a machine-readable form. Building that infrastructure matters even as the detectors stay flawed. The failure is not the idea; it is the overclaim. A watermark or a detector result is evidence about a production process, never a final judgement about a person or a fact.
That principle has to be written into policy before the tools arrive, not after the first wrongful accusation. Schools should state plainly that no detector or watermark score is sufficient proof of misconduct, and should weigh drafts, version history and a student’s ability to discuss their work. Employers should not reject applicants on an automated flag, and newsrooms should treat a provenance signal as a prompt to verify, not a verdict to publish. In every case, the burden of proof belongs with the process, not the accused.
CLOSING TAKEAWAY
A watermark is best understood modestly: a faint trace in the way words were chosen, which a compatible system can sometimes recover and which fades under editing. It can hint at how a text was produced, yet cannot tell you whether that text is honest, accurate or fair, because those are human questions no signal can answer. As these tools move into schools, offices and courts, those that keep the distinction clear will use them well. Those that forget it will build a faster machine for punishing the wrong people.
Johan Steyn is a prominent AI thought leader, speaker, and author with a deep understanding of artificial intelligence’s impact on business and society. He is passionate about ethical AI development and its role in shaping a better future. Find out more about Johan’s work at https://www.aiforbusiness.net
Comments