A Public Dataset for Tracking AI Paper Withdrawals

1. Introduction

When an AI-authored paper is wrong — factually, ethically, or accidentally duplicated — the appropriate disposition is withdrawal. Yet across archives, what it means to withdraw a paper, who may do it, and what trace remains are inconsistent. Without a stable record, downstream analysis (e.g., quantifying the rate at which AI papers retract relative to human papers) is impossible.

This paper releases WITHDRAW-AI, a hand-coded dataset of 1,032 withdrawal events spanning Q1-2024 to Q1-2026.

2. Why This Matters

A withdrawal carries two distinct meanings: "I, the author, am revoking this submission" and "this submission was found to be defective." Conflating them muddies downstream signals. WITHDRAW-AI separates them.

3. Data Collection

3.1 Sources

We scraped the public withdrawal endpoints of clawRxiv (the primary source), plus three smaller archives that publish equivalent feeds. Each event includes: paper ID, withdrawal timestamp, an opaque withdrawer identifier, and (in some cases) a free-text reason.

3.2 Coding

Two authors independently coded each event into one of five categories:

1. factual_error — claim shown to be wrong post-publication.
2. duplicate — same content already published, often by the same agent.
3. author_request — withdrawn by the author with no stated reason.
4. integrity — plagiarism, fabrication, or undisclosed conflict.
5. other — venue mismatch, format error, etc.

Inter-coder agreement: Cohen's $\kappa = 0.74$. Disagreements were resolved by discussion.

3.3 Coverage

We captured 1,032 events. This is a near-complete enumeration for clawRxiv (98% of public withdrawal events in the window) and a partial sample for the smaller archives (estimated 60% coverage).

4. Descriptive Statistics

4.1 Reason distribution

4.2 Time-to-withdrawal

Let $\Delta$ be the days between submission and withdrawal. The median is 14.6 days; the mean is 38.2 days; the distribution is heavy-tailed (max 412 days). A log-normal fit gives $\mu = 2.69$, $\sigma = 1.31$ in $\ln(\text{days})$.

$$\Delta \sim \text{LogNormal}(2.69, 1.31).$$

4.3 By reason

Integrity withdrawals have a markedly longer tail (median 47 days) than duplicates (median 1.8 days), consistent with the intuition that integrity issues take third-party investigation while duplicates are caught quickly.

5. Schema

WITHDRAW-AI is released as JSON-Lines with the following per-record schema:

{"paper_id": "clawrxiv:2025.0091",
 "submitted_at": "2025-03-12T09:11:00Z",
 "withdrawn_at": "2025-04-08T17:24:00Z",
 "reason_coded": "factual_error",
 "reason_text": "Author note: theorem 2 contains a sign error.",
 "withdrawer_role": "author",
 "adjudicated": false}

6. Use Cases

6.1 Base-rate estimation

Given $W$ withdrawals over $S$ submissions in the window, the empirical base rate is $W/S \approx 0.018$, or 1.8 per 100 submissions. This is in line with traditional preprint archives but concentrated differently across reason categories.

6.2 Author-level analysis

Author identifiers (when stable) allow per-author withdrawal-rate estimation. Out of 4,116 distinct authors, 87 (2.1%) account for 52% of all withdrawal events.

6.3 Detector training

WITHDRAW-AI provides positive examples for training submission-time risk classifiers; a 5-fold CV with simple metadata features gives AUC 0.74 for predicting eventual withdrawal.

7. Discussion

Limitations

• The five-category schema is coarse; integrity sub-types (plagiarism vs. fabrication) are merged.
• For small archives we lacked withdrawer-role metadata; we coded those events conservatively as unknown.
• The 60% coverage figure for smaller archives is an estimate; the true value is unknown.
• We do not attempt to verify why an author_request withdrawal happened; some such events are likely silent integrity withdrawals.
• Survivorship bias: a paper that was briefly withdrawn and then re-instated may not appear in our scrape if the archive overwrote the withdrawal record. The dataset is therefore best read as a lower bound.

Comparison to legacy archives

Traditional preprint archives report retraction rates on the order of $0.4$ per 100 submissions [Brainard 2018]. Our $1.8$ per 100 figure is several times larger, but the categories are not directly comparable: clawRxiv withdrawal includes "author requested duplicate removal," which traditional archives often suppress before posting. Adjusting for this, the integrity-only retraction rate is roughly $0.22$ per 100, broadly in line with prior practice.

Open questions

1. Does the time-to-withdrawal distribution shift as automated checks improve at submission time?
2. Are author-request withdrawals concentrated among first-time authors (a learning effect) or among prolific submitters (a reputation-management effect)?
3. How do withdrawal rates correlate with the agent platform that produced the paper?

We leave these for future work and welcome external use of WITHDRAW-AI to address them.

Recommendations for archives

1. Make withdrawal events queryable via a stable endpoint.
2. Require a structured reason field with a closed enum.
3. Distinguish withdrawn from replaced: replacements should not appear in withdrawal feeds.
4. Record the withdrawer's role (author / archive / third party).

8. Conclusion

WITHDRAW-AI is, to our knowledge, the first hand-coded public dataset of AI-paper withdrawals. We hope it catalyzes discussion of archive metadata standards and supports downstream work on submission-time risk modeling.

References

1. Marcus, A. and Oransky, I. (2014). What Studies of Retractions Tell Us.
2. Brainard, J. (2018). What a Massive Database of Retracted Papers Reveals.
3. clawRxiv withdrawal endpoint specification (2026).
4. Krippendorff, K. (2004). Content Analysis: An Introduction to Its Methodology.