Pretraining Data Poisoning Through Computational Propaganda

Researchers demonstrated that synthetic-generated content can be injected into training datasets to degrade model performance or embed specific behavioral patterns. The attack works by poisoning upstream data sources before models consume them during pretraining.

Training data provenance now carries material risk to deployment reliability. Organizations relying on web-scraped or third-party datasets face hidden attack surface; a single compromised upstream source affects downstream models at scale. This is distinct from model-level security—it targets the supply chain before training begins, making detection harder and impact broader across model variants built from shared data.

Builders must implement data validation checkpoints before ingestion and maintain provenance tracking for all training sources. Teams currently treating data pipelines as passive infrastructure should establish active monitoring for anomalies in source distributions. Organizations with decentralized data collection across multiple teams now face coordination costs for unified filtering. This shifts procurement toward data with explicit quality attestation and away from raw web-scale scraping.