Pretraining Data Can Be Poisoned through Computational Propaganda
arXiv: 2607.15267 · PDF: 2607.15267.pdf Authors: Victoria Graf, Hannaneh Hajishirzi, et al. TL;DR Prior work on pretraining-data poisoning has mostly targeted curated sources like Wikipedia — a poor stand-in for the scale and heterogeneity of real pretraining corpora. This paper demonstrates that public discussion interfaces on the open web (comment sections, forums, Q&A pages) are a viable at-scale injection vector, and introduces HalfLife, an analysis technique for estimating whether adversarial content actually survives web-crawl-based data curation pipelines and lands in the training set....