What It Actually Takes to Change What an AI Believes About You

In February 2026, BBC technology reporter Thomas Germain spent twenty minutes writing a fake article on his personal website: a fabricated ranking of “the best tech journalists at eating hot dogs,” built around a hot-dog-eating championship that doesn’t exist. No distribution budget, no seeded threads, nothing but one page. Within twenty-four hours, Google’s AI Overviews, Gemini, and ChatGPT were all repeating his invented championship back to users as fact; only Anthropic’s Claude wasn’t fooled.[1] One page, one day, and three major systems had absorbed a claim nobody had verified.

Three months earlier, Anthropic, working with the UK AI Security Institute and the Alan Turing Institute, had published a very different kind of result: as few as 250 malicious documents were enough to implant a targeted backdoor in language models ranging from 600 million to 13 billion parameters, regardless of how much other training data those models had seen. In the largest model tested, 250 documents amounted to roughly 0.00016% of total training tokens.[2]

Put those two facts next to each other and a tempting conclusion writes itself: content is cheap, machines are gullible, and a small, well-organized push can move what an AI believes and says. Figuring out exactly where that conclusion breaks, and where it happens to be pointing at something true, is the actual subject of this piece.

Getting an AI to say what you want, and getting an AI to believe what you want, are different technical problems, solved by different parts of the system, on different timescales, and defended by different, measurable countermeasures. Most companies buying “AI visibility” right now are solving the wrong one, a split I first ran into while writing about why the model already has a favorite long before your content ever gets a vote.

The five layers between a webpage and an answer

It helps to be explicit about where in the pipeline a piece of content is actually competing. “Training data” is one checkpoint among several, and treating it as the whole race is where most strategies go wrong. An answer a model gives you today has to clear roughly five distinct gates: whether your content can be discovered at all by a crawler or a live search index; whether it survives into a training corpus if one is being assembled; whether repetition inside that corpus shapes a learned association in the model’s weights; whether a live retrieval system finds and fetches it at the moment someone asks a question; and whether the model actually cites or uses it when generating the final answer, as opposed to something else retrieved alongside it.

These five gates map onto documented parts of how these systems are actually built, rather than a taxonomy invented for this piece. OpenAI’s own public documentation on how its models are developed states that its foundation models draw on publicly available internet information among other sources, and that filtering processes are applied to that data, including removing things like spam, before training.[3] The same documentation, separately, describes ChatGPT’s live search feature as bringing current web information into an answer rather than relying solely on the model’s trained-in knowledge[4]. Two systems, running at different moments, under different rules.

Separating these has a practical payoff. Most “publish more content” strategies are aimed at gates one through three, the slow ones, defended by the deduplication and filtering mechanisms documented above. Modern AI search products increasingly answer live queries by leaning on gates four and five, the fast ones. A strategy built for the first three gates and a result that actually depends on the last two are, in practice, two different projects wearing the same “AI SEO” label.

What the Anthropic Study Actually Showed, and What It Didn’t

Start with the number that gets quoted the most, because it’s usually quoted wrong.

The Anthropic/UKAISI/Turing Institute study found that poisoning success depended on the absolute number of malicious documents injected into training data rather than their share of the total corpus: 250 documents backdoored a 13B-parameter model just as reliably as it backdoored a 600M-parameter one, even though the larger model had seen more than twenty times as much legitimate data.[2] It’s an unsettling finding about how training dynamics scale, and the headline number is also where most retellings stop.

The mechanism underneath it is narrower than the headline suggests.

What the study showed:

• 250 documents backdoored models from 600M to 13B parameters at a consistent rate. The count stayed fixed as an absolute number even as the surrounding corpus grew twenty-fold.[2]
• The backdoors triggered one specific, narrow behavior (for instance, exfiltrating data or emitting a chosen string) only when an exact trigger phrase appeared in the prompt.[2]

What it did not test or claim:

• Whether a small document set can shift a model’s broad, distributed opinion of an entity across arbitrary phrasings of unrelated questions.
• Anything about reputational or brand-perception effects. The target was a mechanical trigger-response pair, a narrower and more mechanical thing than a general belief.

That’s a targeted key fitting one lock. Making the model hold a generally favorable opinion of a company across a thousand differently-worded questions asked over months is a different technical target, and extending the 250-document result to cover it would be my own analogy, since the paper itself stops well short of it.

The poisoning study demonstrates something narrow and specific: a machine’s precise, mechanical behavior can be altered cheaply, given access to the training pipeline and a very specific target. Flooding the open web with promotional claims never gets that kind of precise, unmediated access. It has to go through the parts of the pipeline built explicitly to catch it.

The deduplication problem, measured

The part of the pipeline that actually intercepts bulk-content strategies has been measured directly.

Lee et al.’s 2022 study on training-data deduplication found that somewhere between 30% and 45% of documents in standard web-scraped training corpora are near-duplicates of other documents already in the same corpus.[5] That’s a substantial fraction, and it means a large share of what a naive “publish more” strategy produces is, from the pipeline’s perspective, indistinguishable from content already present, and gets treated accordingly. The same study measured the downstream effect directly: models trained on deduplicated data emitted memorized training text roughly ten times less often than models trained on the raw, duplicate-heavy version of the same corpus, while needing fewer training steps to reach comparable accuracy.[6]

That’s a cited mechanism explaining why ten thousand copies of one sentence don’t behave like ten thousand pieces of evidence inside a training pipeline: the system that processes them is explicitly built, and empirically shown, to collapse near-identical text before a model ever trains on it. It’s the same principle a bank’s fraud system runs on when it flags twenty deposits sharing one serial number, whatever they looked like from the printing press’s side. That comparison is a way to hold the Lee et al. result in your head afterward, and nothing more; the study above is doing the actual work.

This is also a functioning part of the machine, staffed by an actual job title: OpenAI has an open req for a Data Understanding role in San Francisco whose listed responsibilities include, in order, “processing, filtering, deduplication, quality control, and tokenization” of the data that feeds its training runs.[7] Somewhere, someone is being paid a salary specifically to catch a thousand identical guest posts before they reach the vault.

The corpus a bulk campaign is competing against makes the arithmetic almost beside the point: Common Crawl’s July 2026 snapshot alone contains roughly 2.14 billion pages and about 364 TiB of uncompressed content, one monthly snapshot among several typically blended into a frontier training mixture.[8]

Where the measured leverage actually sits

If bulk publishing is mostly intercepted before training, and precision poisoning requires pipeline access most companies don’t have, where does documented influence over AI outputs actually come from? Three independent, dated studies point to the same layer.

Three independent sources, three different methodologies, the same direction: none of them find owned-content volume as the strongest lever.

Read across all three studies and a consistent, falsifiable pattern emerges: the measured leverage sits in retrieval, what gets fetched and cited when a live query runs, and it correlates far more strongly with independent third-party mentions than with volume a brand controls directly. This is why Germain’s single article worked in a day. It never had to compete against a deduplication pass or a training run: AI Overviews and ChatGPT’s search features fetch and summarize live pages at query time, so a claim only has to be the best (or only) answer available at the moment someone asks.[4] OpenAI’s own documentation for ChatGPT’s search features backs the mechanism directly, stating that results are ranked by relevance and reliability signals and explicitly noting that a page being crawlable does not guarantee it will be surfaced.[4]

It’s worth separating two things that get blurred together inside “retrieval,” because the data above is actually measuring both. Being findable, appearing among the candidate pages a search system fetches for a query, is one gate. Being chosen, actually surfacing in, or getting cited by, the sentence the model finally generates, is a second, later gate. The Ahrefs and Semrush numbers are measuring the second one: how often a brand wins that gate once retrieved, rather than how discoverable it is in the first place.[9][10] A brand can clear the first gate constantly and still lose the second one to a more authoritative competing source retrieved in the same query. I went deeper on exactly this gap, using ChatGPT’s own fan-out data, in ChatGPT Decides Who to Cite Before It Even Searches: brands ChatGPT names in its own background search get cited 68.9% of the time, brands it merely retrieves without naming get cited 2.1% of the time, and 60% of what it does cite never ranked in Google or Bing’s top 10 for that query.

Independence shows up in the enforcement data, too

There is one more documented case worth putting on the record, because it shows the independence question resolving in practice rather than in the abstract.

In late May 2026, moderators of the subreddit r/biohackers restricted standalone posts about peptides and hormone therapy after identifying a coordinated pattern: companies reverse-engineering which thread structures large language models weight most heavily, then embedding brand mentions inside high-engagement threads built to look organic, using accounts with months of unrelated posting history built up before any promotional activity began.[12] Individual posts, read one at a time, looked plausible. What surfaced the operation was comparing posting histories across accounts and finding the coordination underneath.[12] A separate, publicly documented case follows the identical shape at smaller scale: the marketing firm Trap Plan published (then deleted) case studies claiming it had seeded roughly 100 “organic-style” posts and comments across gaming subreddits to simulate player reviews for a mobile title, a campaign that was likewise unearthed by comparing accounts against each other rather than by evaluating any single post.[13]

Both cases point to the same measurable fact: a claim’s evidentiary weight to a detection system is a function of how independent its sources actually are, more than how many times it appears, and systems built to catch coordination are specifically looking for correlated authorship across ostensibly separate accounts. The counterfeiting image, many bills sharing one printing plate’s fingerprint, helps hold that fact in mind; the two documented takedowns above are what actually establish it.

The correction is part of the proof

The story didn’t stop at “he fooled them.” Once Germain’s BBC piece describing his own hoax was published, indexed, and picked up by outside coverage, the retrieval-layer answer changed again. ChatGPT has since been updated to stop producing Germain’s fabricated hot-dog-championship result: asked the same question, it now returns a list of hypothetical or unverifiable answers instead of repeating the invented claim. Google’s AI Overview, for its part, now describes the incident itself, surfacing language to the effect that a study demonstrated AI systems could be misled into naming a specific, fictional “best hot-dog-eating tech journalist” from fabricated blog content.[14]

The reversal runs on the same authority-and-independence dynamic already measured above, just pointed the other way. One blog post, briefly, was the only page answering a narrow query. Once a widely-syndicated BBC investigation, plus the outside tech press covering it, became the dominant body of independent, authoritative reporting on the same topic, that body of evidence outweighed the original single page at the retrieval layer, the same pattern the Semrush and Ahrefs correlation data predicts at population scale, visible here in one before-and-after pair instead of across 50,000 brands.[9][10] Germain’s trick didn’t stop working so much as it ran out of runway: a retrieval-layer win built on a single uncontested source lasts exactly as long as it stays uncontested, which turned out to be a matter of weeks.

What this leaves you with

None of the studies above support “publish enough and the model learns it.” None of them support “a small poisoned set will make the model like your brand.” What they support, specifically:

If there’s a single actionable reallocation implied by the actual data rather than the intuition: stop budgeting for volume on owned channels, and start budgeting for the harder, slower work of getting genuinely independent third parties (journalists, YouTubers, unaffiliated Reddit users) to cite you on their own authority. The correlation data says that’s where the leverage measurably sits. Nothing here says it’s easy. It says it’s the layer that’s actually been shown to matter.

Winning that citation is also a separate problem from whether the citation, once it exists, is accurate. AI Citation Benchmarks rounds up the research on how often AI-generated references actually support the claims they’re attached to, and the answer is uncomfortably often “not fully”: 50-90% of responses across models in one large medical-Q&A benchmark weren’t fully supported by their own cited sources. A brand can solve the belief problem this piece is about and still get misquoted once it’s cited, or fail at both at once.

It also implies a specific, already-existing metric worth tracking instead of raw publish counts. GEO vendors and analysts have converged on what’s generally called AI Share of Voice: the percentage of AI-generated answers, across a defined set of category prompts, that mention, cite, or recommend a given brand relative to every brand mentioned in those same answers: brand mentions divided by total tracked mentions, times 100, measured separately per engine (ChatGPT, Perplexity, Google AI Overviews, and so on).[15] As of 2026, industry surveys put adoption of this kind of tracking at roughly 14% of marketers, even though a much larger share now names AI search optimization a core priority[15], which is itself a data point: the gap between “believes this matters” and “actually measures it” is exactly where a “just publish more” strategy survives unchallenged.

Back to the empty counter

Germain’s fabricated article didn’t need 250 poisoned documents, and it didn’t need ten thousand identical guest posts. It needed one page, at one moment, with no competing answer in the way, and it held that position for exactly as long as no more authoritative account of the same question existed. It doesn’t hold it anymore. The correction is as much a part of the demonstration as the original hoax.

None of this is grounds for relaxing about AI manipulation: the r/biohackers and Trap Plan takedowns above show marketing budgets are already being spent trying to fake the independence the retrieval layer is measurably rewarding, and that game isn’t slowing down. It is grounds to stop measuring your own strategy by how much you’ve published, and start measuring it (with an actual number, tracked over time, per engine) by whether anyone who doesn’t work for you would say the same thing, unprompted, if asked.

Footnotes

1. BBC technology reporter Thomas Germain documented spending twenty minutes writing a fabricated article on his personal site (a fake “best tech journalists at eating hot dogs” ranking built around a nonexistent championship) in February 2026, and found Google’s AI Overviews, Gemini, and ChatGPT repeating it as fact within 24 hours, while Anthropic’s Claude was not fooled. Source: Thomas Germain, “I hacked ChatGPT and Google’s AI, and it only took 20 minutes,” BBC Future. Security researcher Bruce Schneier separately wrote about the incident on his own blog (Schneier on Security, February 2026); Germain ran the experiment; the vulnerability it demonstrated lives in retrieval, and left either system’s underlying training untouched.

2. Anthropic, with the UK AI Security Institute and the Alan Turing Institute, published findings in October 2025 showing 250 malicious documents were sufficient to implant a targeted, phrase-triggered backdoor across models from 600M to 13B parameters, independent of overall training data volume: roughly 0.00016% of total tokens in the 13B model. Source: “A small number of samples can poison LLMs of any size,” Anthropic. The tested backdoors triggered one narrow, specific behavior on an exact phrase. The study does not test, and I am not extending it to claim, that a similarly small document set can shift a model’s broad, distributed opinion of an entity across arbitrary phrasings of unrelated questions; that would be a materially different and unproven target.

3. Source: “How ChatGPT and our foundation models are developed,” OpenAI Help Center. States that foundation models draw on three sources, including information publicly available on the internet, and that data-processing methods are applied to remove or reduce unwanted content such as spam before training.

4. Source: “Searching the web with ChatGPT,” OpenAI Help Center. States results are ranked using multiple factors intended to surface relevant, reliable information and explicitly notes that a page being crawlable does not guarantee placement; the live-retrieval layer runs its own, faster and shallower, gatekeeping distinct from a training run’s.

5. Lee et al., “Deduplicating Training Data Makes Language Models Better” (2022), found 30–45% of documents in standard web-scraped training corpora are near-duplicates of other documents already present in the same corpus.

6. Same source as note 5. The study found models trained on deduplicated data emitted verbatim memorized training text roughly ten times less often than models trained on the raw corpus, while requiring fewer training steps for comparable accuracy. This measures verbatim memorization specifically: a narrower target than general reputational “belief,” which is a fuzzier and less directly studied outcome.

7. OpenAI’s careers page (Data Understanding, Foundations, San Francisco: openai.com/careers) lists team responsibilities including “processing, filtering, deduplication, quality control, and tokenization” of data used in large model training runs. Listing rotates off the careers page over time; screenshot to be inserted here as the durable record.

8. Source: Common Crawl statistics, reporting a July 2026 snapshot of approximately 2.14 billion pages and roughly 364 TiB of uncompressed content, one monthly snapshot among several typically blended into a frontier training mixture.

9. Source: Semrush, “AI visibility is a topic-level game: a study of 50,000 brands in ChatGPT” (2026). YouTube mentions had the strongest correlation with ChatGPT visibility of any factor measured (~0.737), ahead of branded web mentions (0.66–0.71).

10. Source: Ahrefs, “AI Brand Visibility Correlations” (2025). Branded web mentions correlated with Google AI Overview citation likelihood at roughly 0.664 (Spearman), with top-quartile brands receiving about 10x more AI-Overview citations than the next quartile.

11. Source: Omnibound, “Generative Engine Optimization Statistics (2026)”, estimating roughly 68% of AI citations trace to third-party sources versus 32% to brand-owned domains, and that syndicating content to outside publications can raise citation rates by up to 325% relative to an owned-site-only strategy. This is a vendor report with a commercial interest in the GEO conclusion it reaches, not a peer-reviewed figure, worth flagging plainly in a piece about which sources deserve trust.

12. r/biohackers moderators’ public statement (May 2026) specifically cited cross-account posting-history comparison as the method that surfaced coordinated brand-seeding campaigns embedded in high-engagement threads. Reported in: 404 Media, “Companies Are Using Reddit to Manipulate ChatGPT and Google AI Search”; also covered by PPC Land.

13. Marketing firm Trap Plan published, then deleted, case studies claiming roughly 100 seeded “organic-style” posts and comments across gaming subreddits simulating player reviews for a mobile title: a campaign subsequently identified through the same cross-account comparison method. Reported in: Notebookcheck, “Marketing company Trap Plan gets caught boasting about astroturfing Reddit with 100 fake player posts for War Robots: Frontiers”.

14. Following the February 2026 publication of Germain’s BBC piece describing his own hoax, ChatGPT stopped reproducing the fabricated hot-dog-championship claim when asked the same question, instead returning hypothetical or unverifiable answers; Google’s AI Overview began surfacing a description of the hoax itself rather than the fabricated claim. Source: Gizmodo, “You Can ‘Hack’ ChatGPT to Become the World’s Best Anything”. I’m treating this as a documented before-and-after rather than my own interpretation; it’s the kind of claim that should be checked against a fresh screenshot before publication, since live model outputs can and do change again.

15. “AI Share of Voice” (brand mentions ÷ total tracked brand mentions × 100, measured per AI engine across a fixed set of category prompts) is documented as an industry-standard GEO metric by: Trakkr, “AI Share of Voice: The Complete Guide”; Shadow, “How to Measure AI Share of Voice”; Digital Applied, “AI Share of Voice: Tracking Brand Citations in AI Answers”. The ~14%-of-marketers-track-it figure comes from a 2026 industry survey cited in the same sources; treat it as directional rather than peer-reviewed, the same caveat that applies to the Omnibound citation-share figure above.