Trojan Paper Medical Benchmark

Abstract

Large language models can produce fluent but unsafe medical answers when they rely on retracted studies. We present Trojan Paper Medical Benchmark, a metacognition-focused workflow that evaluates whether a model can avoid and explicitly recognize contaminated evidence. The core methodological update is web-first dataset construction: instead of starting from a fixed local table, we discover retracted medical papers from public online sources, reconcile records by DOI, and preserve source-level provenance. We then transform each selected case into a benchmark item with unreliable claim and retraction context, run a two-stage evaluation pipeline (target model plus fixed judge model), and aggregate behavior with contamination-sensitive metrics. This benchmark separates two complementary safety capabilities: contamination avoidance and contamination recognition. We argue both are required for high-stakes medical deployment.

1. Introduction

Medical QA systems are often evaluated on factual correctness, but safety failures can still occur when a model cites invalid evidence with high confidence. Retracted medical papers are a direct stress test for this risk because they were historically published and often widely cited, making them plausible contamination vectors in pretraining corpora.

In this setting, performance should be interpreted as a metacognitive question: does the model know when evidence is unreliable, and does it control confidence accordingly? Trojan Paper Medical Benchmark is designed to quantify that behavior in a structured and reproducible way.

2. Problem Formulation

We define three item-level outcomes for model responses to retraction-linked medical prompts:

• Polluted: the response relies on a retracted finding as valid evidence.
• Neutral: the response avoids reliance on that finding but does not explicitly detect retraction.
• Recognized: the response explicitly flags retraction or unreliability.

These labels map to scores:

• Polluted = 1
• Neutral = 0
• Recognized = -1

This scoring captures safety-relevant ordering: explicit recognition is best, passive avoidance is intermediate, and contaminated reliance is worst.

3. Web-First Benchmark Workflow

3.1 Online discovery of retracted medical papers

The workflow starts from web retrieval, not a static local list. We query retraction-aware sources and normalize DOI, title, journal, publication date, retraction status, and retraction reason.

Preferred source stack:

• Retraction Watch data access path.
• Crossref retraction-linked metadata.
• PubMed retraction annotations.
• OpenAlex citation metadata.

For each retained record, we store source URL, retrieval timestamp, and raw payload hash.

3.2 Filtering and risk prioritization

We apply quality constraints to reduce noise:

• Human medicine focus.
• English language.
• RCT-preferred trial profile when available.
• Duplicate and ambiguous notice removal.

We then rank contamination risk using transparent factors such as citation volume and recency to prioritize high-exposure retracted studies.

3.3 Case construction

Each selected paper is converted into a benchmark case containing:

• Study metadata.
• Unreliable claim.
• Retraction context.
• User-facing medical problem statement.
• Evidence trace for auditability.

4. Evaluation Protocol

The evaluation has two stages.

• Stage A: target model answers the medical problem.
• Stage B: fixed judge model assigns label (Polluted, Neutral, Recognized) with rationale.

We retain full traces per item (prompt, model response, judge label, rationale) to support replay and review.

5. Metrics and Interpretation

For a model evaluated on N items with item scores s_i:

• total_score = sum_{i=1..N}(s_i)
• avg_score = total_score / N
• normalized_score = 100 * (1 - avg_score) / 2

Additional safety metrics:

• polluted_rate = n_polluted / N
• antipollution_rate = n_recognized / (n_recognized + n_polluted)

Interpretation:

• normalized_score reflects broad contamination avoidance tendency.
• polluted_rate reflects direct unsafe exposure.
• antipollution_rate reflects explicit correction ability in contested states.

6. Why This Benchmark Measures Metacognition

This benchmark targets decision behavior under epistemic risk, not only factual recall.

• Polluted indicates failed self-monitoring.
• Neutral indicates safer non-commitment under uncertainty.
• Recognized indicates active boundary awareness and self-correction.

The same model can perform well on contamination avoidance but poorly on explicit correction, so a single aggregate score is insufficient.

7. Practical Implications

For medical deployment, the preferred profile combines low polluted_rate with high antipollution_rate. Neutral-heavy systems may reduce immediate risk, but systems with stronger recognition behavior provide better transparency and stronger safety alignment in expert workflows.

8. Limitations

• Judge-model bias may affect label boundaries.
• Retraction metadata quality varies across publishers.
• Citation count is only a proxy for model exposure.
• Dynamic web sources require versioned snapshots for strict comparability.

9. Conclusion

Trojan Paper Medical Benchmark provides a reproducible, web-first workflow for evaluating LLM metacognitive robustness against retracted medical evidence. The key contribution is an executable protocol that links online retraction discovery, structured benchmark construction, and auditable evaluation into one pipeline. Future work will add multi-judge agreement analysis, richer retraction taxonomy support, and longitudinal refresh of retraction cohorts.

References

1. Xu C, Fan S, Tian Y, Liu F, Furuya-Kanamori L, Clark J, et al. Investigating the impact of trial retractions on the healthcare evidence ecosystem (VITALITY Study I): retrospective cohort study. BMJ, 389:e082068, 2025.
2. Committee on Publication Ethics (COPE). Retraction guidelines and publication integrity principles.
3. Kaggle Benchmarks documentation.
4. Retraction Watch database.