Structured Reporting Guidelines for AI Papers

1. Why Another Guideline?

The last decade saw a proliferation of domain-specific reporting guidelines, listed by EQUATOR Network at over 600. These have measurably improved transparency. Yet none was authored with AI-generated experimental procedures, AI-mediated literature review, or AI-assisted writing in mind. The result is that authors satisfying the letter of, say, CONSORT can still leave readers unable to reproduce or audit the AI components of their work.

We propose AI-REPORT: a guideline that lives alongside domain-specific checklists and is intended to be appended to existing reporting flows rather than displace them.

2. Design Principles

1. Compositional, not exclusive. AI-REPORT is filled in addition to CONSORT, PRISMA, etc.
2. Machine-readable. Each item maps to a YAML field with a typed value.
3. Checklist-style binary disclosures, plus free-text justifications.
4. Bounded burden. Median completion under 20 minutes.
5. Versioned. Each item carries an ID stable across revisions.

3. The 27 Items

The items are grouped into six sections.

• A. Inventory of AI usage (5 items): Models used, version pins, prompts archive, tool-use catalog, RAG corpora.
• B. Generation events tied to claims (4 items): Mapping between specific claims and the generation events that produced them.
• C. Verification regime (5 items): What was checked by humans? How? With what inter-rater agreement?
• D. Failure cases observed (3 items): Hallucinations encountered, retractions, stop-conditions.
• E. Compute and cost transparency (4 items): Tokens consumed, USD, energy.
• F. Author and AI roles (6 items): Per-section role assignments using a CRediT-derived vocabulary extended to non-human contributors.

A snippet of the YAML schema:

ai_report:
  models:
    - id: "gpt-5-2025-09"
      role: "drafting, code synthesis"
  prompts_archive_url: "https://archive.example/sub/abcd1234"
  rag_corpora:
    - name: "PubMed-2024-12"
      hash: "sha256:..."
  claim_to_event_map: "./claim_event_map.csv"
  verification:
    spans_human_reviewed_pct: 87.4
    interrater_kappa: 0.78
  failure_cases:
    - description: "hallucinated citation"
      paragraph_id: "p#34"
      resolution: "removed"
  compute:
    total_tokens: 2147384
    total_usd: 18.40

4. Pilot Study

4.1 Setup

We invited 92 authors of recent AI-co-authored preprints to fill AI-REPORT for their submission. 88 returned a complete checklist. We then asked three independent reviewers per paper to attempt partial reproduction, scoring the result on a 100-point composite of method clarity, code availability, and ability to regenerate at least one figure.

4.2 Results

The difference is significant (paired Wilcoxon, $p < 0.001$). Author-reported time-to-complete had median 14 min and IQR $[9, 22]$.

4.3 Item-level utility

The items most cited by reviewers as decisive were Inventory A.1 (model identifier with version pin), Verification C.3 (which spans were human-checked), and Failure D.1 (hallucinations encountered). Items in section E (compute) were rarely cited in reproducibility judgments but were valued by reviewers writing meta-analyses.

5. Discussion

Why 27 items rather than, say, 50? We followed an iterative pruning protocol: starting from 64 candidate items, we removed those with redundancy $r > 0.85$ (Spearman correlation across pilot completions) and those with median utility rating below 3/7 in a reviewer survey. The final set is the largest at which both constraints hold.

We consciously did not include an item asking authors to certify that AI did or did not produce "original" work. This is a normative judgment that we believe should be left to community discussion, not encoded in a checklist.

6. Limitations

Self-reported completion time may underrepresent true effort. Reviewers in the reproducibility scoring step were not blinded to the AI-REPORT condition; we partially mitigated by having a separate scorer adjudicate borderline cases. AI-REPORT does not address review-side AI usage; that is a separate guideline still in draft.

7. Adoption Considerations

We recognize three frictions to adoption. First, version drift: if AI-REPORT items change frequently, authors who internalize the v1 vocabulary will resent v2. We have committed to a deprecation policy in which retired items remain machine-parseable for at least two years. Second, mismatch with venue templates: existing journal submission systems do not accept arbitrary YAML attachments. We provide a JSON-to-PDF appendix renderer that produces a printable artifact suitable for current submission flows while preserving the underlying machine-readable form. Third, gaming risk: any structured disclosure invites pro-forma completion. To partially mitigate, several items (B.2 claim_to_event_map, D.1 failure_cases) require non-trivial structure that is hard to fabricate without inconsistency, and a complementary auditor tool flags such inconsistencies.

We also note that adoption will likely be staged: some venues will require AI-REPORT only for AI-disclosed manuscripts, while others may make it universal as the boundary between AI-assisted and unassisted work erodes. Both regimes are compatible with the schema as designed.

8. Conclusion

AI-REPORT is a small, structured addendum to existing reporting flows that yields a meaningful gain in third-party reproducibility for a modest author burden. We invite venues to pilot adoption and contribute back to the schema's evolution. The most actionable next step for individual authors is to fill the schema for a manuscript already under review --- the median 14-minute cost is lower than the cost of one round-trip with a confused reviewer.

References

1. Schulz, K. F. et al. (2010). CONSORT 2010 Statement.
2. Page, M. J. et al. (2021). PRISMA 2020.
3. Mitchell, M. et al. (2019). Model Cards for Model Reporting.
4. Liao, T. et al. (2025). Reproducibility Crises in AI-Mediated Science.