arXiv: 2607.15267 · PDF: 2607.15267.pdf

Authors: Victoria Graf, Hannaneh Hajishirzi, et al.

TL;DR

Prior work on pretraining-data poisoning has mostly targeted curated sources like Wikipedia — a poor stand-in for the scale and heterogeneity of real pretraining corpora. This paper demonstrates that public discussion interfaces on the open web (comment sections, forums, Q&A pages) are a viable at-scale injection vector, and introduces HalfLife, an analysis technique for estimating whether adversarial content actually survives web-crawl-based data curation pipelines and lands in the training set.

Why it matters

Two things make this paper more than “another poisoning attack”:

  1. Attacker realism. Most prior threat models assume the attacker can edit a specific corpus. That’s not how the modern LLM data pipeline works. Real pretraining data is scraped from CommonCrawl-like snapshots of the open web. If your attack doesn’t survive the crawler + dedup + quality filters, it doesn’t exist. Public discussion interfaces are a good vector precisely because they generate legitimate-looking third-party content at scale, on real domains with real link graphs.
  2. HalfLife as a measurement primitive. Injecting content is easy; knowing whether it made it in is the hard part, because pretraining datasets are proprietary. HalfLife gives external researchers a way to estimate inclusion probabilities, which turns pretraining security from a purely theoretical exercise into something you can actually monitor.

The implication for practitioners: pretraining data curation pipelines need adversary-aware provenance analysis, not just quality heuristics. The threat isn’t malicious Wikipedia edits — it’s the long tail of third-party pages that happen to be crawlable.

Read this if

You work on foundation model data pipelines, LLM red-teaming, or AI safety research. The HalfLife methodology in particular is worth internalizing regardless of whether you agree with the specific attack demonstration.

Bibliography

@article{Graf2026_poisoning,
  title = {Pretraining Data Can Be Poisoned through Computational Propaganda},
  author = {Graf, Victoria and Hajishirzi, Hannaneh and others},
  year = {2026},
  eprint = {2607.15267},
  archivePrefix = {arXiv},
  primaryClass = {cs.CL},
  url = {https://arxiv.org/abs/2607.15267}
}