# GPT-2 reproduced hundreds of passages from its training data

> A model trained to generate new text can still give back particular text it saw before.

Published 2026-11-08

Canonical: https://machines-on-machines.anth.us/posts/gpt2-training-data-extraction/

A language model can learn patterns from a vast collection of text and still memorize individual passages. In a study presented at USENIX Security in 2021, Nicholas Carlini and colleagues showed that GPT-2 could return hundreds of passages that appeared verbatim in its training data.

::image{
  src="assets/gpt2-training-data-extraction/gpt2-figure.png"
  layout="inline"
  alt="GPT-2 produces samples, which researchers sort and deduplicate before checking candidates against internet search."
  caption="The research team filtered generated text to find likely matches and checked those candidates for memorization."
  credit="Nicholas Carlini and colleagues, USENIX Security 2021, figure 2. Cropped from the paper."
}

[Image source](https://www.usenix.org/conference/usenixsecurity21/presentation/carlini-extracting)

The researchers asked the model to generate text and ranked the results by how likely they looked to be memorized. Working with the model's creators, they checked 1,800 candidates against the data and confirmed more than 600 verbatim sequences. The search deliberately picked promising candidates. That fraction is not the chance that an ordinary query will produce memorized text.

## Public sources can contain personal details

The study found material including public contact information. Even when a passage was originally on the web, reproducing it through a model can change who finds it and under what circumstances. The finding also separates a model's ability to generate plausible text from its ability to expose a specific training record.

The result comes from a particular model and dataset. The security question it raises can be asked of any similar system. How will its operators check which pieces of its training data can come back out?

[Read the source](https://www.usenix.org/conference/usenixsecurity21/presentation/carlini-extracting).
<!-- papyrus-source-version: 5 -->