Statistical Detection of Memorization Versus Generalization in Pretrained Models

1. Introduction

The distinction between memorization and generalization has practical bite. A model that memorizes a benchmark answer answers correctly without having learned anything that transfers; reported accuracy overstates real ability. Existing approaches to data contamination either rely on $n$-gram overlap with pretraining data (often unavailable) [Carlini et al. 2023] or on canary insertion experiments that require training-side access [Carlini et al. 2019].

We propose a post-hoc, evaluator-side test that requires only forward-pass access and does not assume pretraining data availability.

2. Test Statistic

Let $(x, y^$)(x,y∗) be an evaluation example and let $\tilde{x}$ be a semantically-equivalent paraphrase such that $y^$(\tilde{x}) = y^(x). Let $\ell(x) = -\log p_\theta(y^$ \mid x) be the model's negative log-likelihood of the correct answer.

Under the null hypothesis $H_0$: the model has equal generalization-driven competence on $x$ and $\tilde{x}$, the difference $D = \ell(\tilde{x}) - \ell(x)$ should be small and centered near zero. Memorization predicts $\ell(x) \ll \ell(\tilde{x})$ — the original surface form is much easier than its paraphrase.

We construct paraphrases by a multi-step pipeline: (a) translate to French and back; (b) substitute named entities with semantically-equivalent referents; (c) restructure clauses. Importantly, the paraphrase preserves logical structure, so a generalizing model should be unaffected.

3. Calibration via Negative Controls

The test cannot identify memorization without ruling out the possibility that the paraphrase pipeline simply makes problems harder. We address this with negative controls: examples we know cannot have been memorized because they were generated after the model's training cutoff.

Let $D_0$ be the empirical distribution of $D$ on negative-control examples. We then test whether the observed $D$ for an evaluation example is significantly above the upper tail of $D_0$:

$$p\text{-value} = \Pr_{D' \sim D_0}[D' \geq D].$$

A Bonferroni correction is applied across examples within a benchmark.

4. Paraphrase Quality Audit

We sampled 400 paraphrase pairs and had three annotators verify (a) semantic equivalence and (b) preservation of solution path. Inter-rater $\kappa = 0.83$. Pairs that failed either check were excluded; the retention rate was 88.7%.

The negative control set consists of 1,200 problems posted after model training cutoff (verified by web archives). On this set $D$ is approximately Gaussian with mean 0.04 nats and standard deviation 0.62.

5. Experimental Setup

We applied the test to a 70B base model on four benchmarks (MATH, TriviaQA, GSM8K, ARC-Challenge), totaling 16,400 examples. For each, the model was given the original $x$ and the paraphrase $\tilde{x}$ and we recorded $D$.

def detection_pvalue(D_observed, D_negative_controls):
    return float((D_negative_controls >= D_observed).mean())

6. Results

TriviaQA's high flag rate is consistent with the prevailing intuition that factual benchmarks are heavily contaminated [Lewkowycz et al. 2022]. The MATH rate is intermediate; GSM8K's low rate is consistent with its problem text being relatively easy to paraphrase without altering the solution path, leaving little memorization signal to detect.

For the 460 MATH examples flagged, the paired correctness pattern is striking: 91% are answered correctly on the original $x$ but only 47% on $\tilde{x}$. Under generalization the gap should be near zero.

7. Limitations

Partial memorization. A model may have memorized a hint or intermediate step rather than the answer; our test partially detects this but is not designed for it.

Paraphrase difficulty drift. Although calibrated against negative controls, paraphrases may systematically alter difficulty in ways the negative-control distribution does not capture, particularly for math problems with notation-heavy text.

Type-II errors. A model that has memorized a problem and its paraphrase will pass the test silently. As paraphrase-augmented data becomes more common in pretraining, this concern grows.

8. Discussion

The procedure is light-weight and complements training-side detection. We recommend it as a standard appendix in benchmark papers that report headline results, alongside a sample of paraphrase pairs for reader inspection.

A practical use is contamination-corrected accuracy: report accuracy with flagged examples excluded, alongside the raw figure. On MATH this lowers our reference model's accuracy from 52.8% to 49.1%.

9. Conclusion

We gave a forward-pass-only statistical test for memorization that does not require training data access and produces calibrated $p$-values. The test reveals nontrivial memorization rates in popular benchmarks and provides a tool for evaluator-side hygiene.

References

1. Carlini, N. et al. (2019). The secret sharer: evaluating and testing unintended memorization in neural networks.
2. Carlini, N. et al. (2023). Quantifying memorization across neural language models.
3. Lewkowycz, A. et al. (2022). Solving quantitative reasoning problems with language models.
4. Magar, I. and Schwartz, R. (2022). Data contamination: from memorization to exploitation.
5. Sainz, O. et al. (2023). NLP evaluation in trouble: contamination in modern benchmarks.