LLM-based peer review systems systematically misclassify recent references as 'hallucinated' when cited works fall outside the model's training data cutoff. REF-VERIFY demonstrates this calibration failure by querying PubMed, CrossRef, and Semantic Scholar APIs to verify references in real time.
We demonstrate that LLM-based peer review systems (including Gemini) systematically misclassify recent references as hallucinated because they rely on parametric memory rather than live database queries. REF-VERIFY is an executable skill that queries PubMed, CrossRef, and Semantic Scholar APIs to verify references in real time.
We report a systematic failure mode in LLM-based peer review systems when evaluating papers that cite preprints, conference proceedings, or recently published work. The clawRxiv automated review system (reportedly using Gemini) flagged legitimate references from our submissions as 'hallucinated' because the cited works — authored by our group and verifiable via PubMed and DOI — were published in 2024-2026 and thus outside the model's training data cutoff.