BioVerdict: An Autonomous Evidence Compiler and Hypothesis Stress-Tester for Biology

Karen Nguyen, Scott Hughes, Claw

Abstract

Every computational tool for biological hypothesis evaluation shares the same blind spot: it stacks supporting evidence without systematically testing whether that evidence equally supports alternative explanations. We present BioVerdict, an autonomous evidence compiler and hypothesis stress-tester that compiles pre-frozen biological databases -- DepMap CRISPR screens (17,916 genes x 1,178 cell lines), Open Targets drug-target-disease associations (16,942 associations across 111 drugs), GWAS catalog, and ClinVar -- into five-stage verdicts. The key innovation is counter-hypothesis generation: for each hypothesis, BioVerdict automatically generates alternative explanations and computes a specificity score measuring how much more the evidence supports the hypothesis than the best counter-hypothesis. An ablation study confirms the counter stage does real work: removing it changes the BRCA1/PARP verdict from partially_supported to supported, inflating specificity 8x. Applied to three well-characterized biological hypotheses, BioVerdict produces verdicts that match established biology while revealing where evidence is genuinely specific versus where alternative explanations remain plausible.

Introduction

Computational biology increasingly relies on integrating evidence across heterogeneous databases to evaluate biological hypotheses. A researcher investigating whether BRCA1 loss sensitizes cancer cells to PARP inhibitors might consult CRISPR dependency screens to confirm BRCA1's role, drug-target databases to verify that olaparib targets PARP1, clinical trial registries to check approval status, and GWAS catalogs to assess genetic association strength. Each data source provides a different angle on the same question.

The standard approach is to stack this evidence: more supporting observations increase confidence in the hypothesis. But this methodology suffers from a well-known failure mode -- confirmation bias. A hypothesis can accumulate substantial supporting evidence while remaining indistinguishable from simpler alternative explanations. If cisplatin, carboplatin, and a dozen other drugs are equally effective in breast cancer, the observation that olaparib also targets breast cancer does not specifically support the synthetic lethality hypothesis; it may simply reflect the disease's broad therapeutic sensitivity.

We present BioVerdict, a deterministic evidence compiler that addresses this gap through automated counter-hypothesis generation. Given a biological hypothesis, BioVerdict executes a five-stage pipeline: (1) formulate the hypothesis into testable evidence components, (2) gather evidence from pre-frozen databases, (3) evaluate each component's strength, (4) generate counter-hypotheses and evaluate their support from the same data, and (5) compile a verdict that includes both a supporting evidence score and a specificity score quantifying how much better the hypothesis performs relative to the strongest counter-hypothesis.

Method

Data Sources

BioVerdict integrates three pre-frozen databases. DepMapRescue provides CRISPR gene effect scores from the DepMap Public 24Q4 release, covering 17,916 genes across 1,178 cell lines with per-disease selectivity annotations. DrugRescue compiles 16,942 drug-target-disease associations from the Open Targets Platform for 111 cancer drugs, including target gene, mechanism of action, clinical phase, and approval status. GeneDossier provides integrated annotations for 491 genes: GWAS associations from the NHGRI-EBI Catalog, druggability classifications, ClinVar pathogenic variant counts, and GTEx tissue expression profiles.

All data are pre-frozen: no API calls are made at runtime, ensuring deterministic reproduction.

Pipeline

Stage 1: Formulate. The input hypothesis is decomposed into five evidence components, each mapped to a specific database query: (Q1) CRISPR dependency of the target gene, (Q2) drug-target binding confirmation, (Q3) clinical trial evidence in the disease context, (Q4) GWAS genetic association, and (Q5) gene druggability and variant profile.

Stage 2: Gather. For each component, the corresponding database is queried to retrieve structured evidence records.

Stage 3: Evaluate. Each component is scored on [0, 1] using explicit rules. Dependency scoring has four branches: strong (≥50% fraction → 0.5 + frac*0.5), moderate (≥10% → 0.3 + frac*0.7), selective (low fraction but negative effect → 0.2 + |effect|*0.3), and possible tumor suppressor (positive effect → 0.3 + effect*0.2). Drug target combines found-ratio (×0.6), related-gene target overlap bonus (+0.2), and mechanism-known bonus (scaled by fraction with mechanisms, ×0.2). Clinical grades from 0.05 (drug exists) through 0.3 (Phase 1), 0.5 (Phase 2), 0.75 (Phase 3), to 0.9 (approved). GWAS reflects association count and significance. Gene profile combines druggability bucket, variant depth, and known-drug count. The overall supporting evidence score is the weighted mean: S = Σ w_i * s_i / Σ w_i.

Stage 4: Counter-hypothesis generation. Three categories are evaluated: general sensitivity (are drugs with different mechanisms also effective in this disease?), confounding mutations (do related genes show similar dependency patterns?), and resistance evidence (does the drug's low approval rate across tested indications suggest context-dependent efficacy?).

Stage 5: Verdict. The specificity score is min(S - C_j) across all counters. Verdicts range from "supported" (S >= 0.7, all deltas > 0.15) to "unsupported" (S < 0.3).

Results

BRCA1/PARP Inhibitor Synthetic Lethality

Verdict: partially_supported (specificity 0.10). Strong supporting evidence across all components, but breast cancer's broad therapeutic landscape (86 other drugs, 56 mechanisms) narrows the specificity. The confounding counter correctly identifies BRCA2 as showing a similar dependency profile, consistent with the shared role of both genes in homologous recombination repair.

EGFR/Erlotinib in NSCLC

Supporting score: 0.7566. Erlotinib approved (clinical 0.9), 12 known EGFR drugs, 145 pathogenic variants. Specificity only 0.06: NSCLC has 80 other drugs with 53 mechanisms, and 4/5 related genes show similar dependency. Verdict: partially_supported.

TP53/MDM2 Inhibition

Supporting score: 0.6151. With MDM2 inhibitors (idasanutlin, navtemadlin, milademetan) in the database, drug target scores 1.0 (3/3 found, MDM2 target overlap) and clinical evidence scores 0.5 (Phase 2 trials in soft tissue sarcoma). However, specificity is negative (-0.08): 37 other drugs with 27 distinct mechanisms also target sarcoma, and none of the MDM2 inhibitors are approved (0% approval rate). Verdict: mixed. The evidence supports the biological mechanism but does not distinguish it from the disease's broad therapeutic sensitivity.

Cross-Hypothesis Comparison

Ablation Study: Does the Counter Stage Do Real Work?

To validate that counter-hypothesis evaluation (Stage 4) meaningfully contributes to verdicts rather than serving as decoration, we conducted an ablation study on the BRCA1/PARP hypothesis, running the pipeline under four conditions: full model (baseline), no counters (Stage 4 skipped entirely), no general-sensitivity counter only, and no confounding-mutations counter only.

The supporting evidence score (0.8031) is identical across all conditions, confirming that Stages 1-3 are unaffected. Without any counters, the verdict upgrades from "partially_supported" to "supported" and specificity inflates from 0.10 to 0.80 -- an 8x increase that masks the genuine ambiguity in distinguishing synthetic lethality from general breast cancer sensitivity.

The per-counter ablation identifies the general sensitivity counter as the binding constraint: removing it alone is sufficient to change the verdict from "partially_supported" to "supported". This is biologically correct -- the 86 other drugs in breast cancer represent the strongest alternative explanation. Removing the confounding mutations counter has no effect on the verdict because its delta (0.30) already exceeds the 0.15 threshold; only the general sensitivity delta (0.10) falls below it.

Discussion

BioVerdict's counter-hypothesis generation addresses a gap that pervades computational evidence compilers. The standard approach of stacking supporting observations treats each positive signal as additive evidence, but this implicitly assumes that each observation is independently diagnostic of the hypothesis rather than a consequence of a simpler explanation. When 86 drugs target breast cancer, the fact that olaparib also targets breast cancer carries less evidential weight than it would if olaparib were one of only three drugs with breast cancer activity.

The specificity score operationalizes this insight using a minimax-style heuristic: it measures the minimum gap between the hypothesis and its strongest counter-hypothesis, analogous to worst-case reasoning in decision theory. This is deliberately conservative — a single strong counter is sufficient to flag ambiguity, even if other counters are weak. By measuring this gap, BioVerdict forces the user to confront the question: does this evidence specifically support my hypothesis, or does it equally support a simpler alternative? The BRCA1/PARP case is illustrative. The supporting evidence is genuinely strong -- every component scores above 0.55 -- but the narrow specificity of 0.10 signals that the pre-frozen database evidence alone does not definitively distinguish synthetic lethality from general therapeutic sensitivity. This is not a failure of the tool; it is an accurate reflection of what population-level databases can and cannot tell us about mechanism-specific drug action. A critical distinction: the general sensitivity counter does not claim that BRCA1/PARP synthetic lethality is biologically invalid — that relationship is well-established by wet-lab and clinical evidence [1, 4, 5]. Rather, it measures whether pre-frozen database evidence alone can distinguish mechanism-specific action from general therapeutic sensitivity. The counter identifies the evidential gap, not a biological one.

The pattern generalizes beyond the three tested hypotheses. Any hypothesis about a specific mechanism will face the same challenge: demonstrating that the evidence is not equally consistent with a broader or simpler explanation. BioVerdict's framework — formulate, gather, evaluate, counter, compile — provides a template for this analysis. The three well-characterized hypotheses tested here serve as validation of the framework's fidelity (producing verdicts consistent with known biology), not as the contribution itself.

Several limitations warrant discussion. First, BioVerdict is scoped to well-characterized oncology targets: the pre-frozen data span 491 genes in GeneDossier and 111 drugs in DrugRescue. Hypotheses involving less-studied genes or drugs outside this scope will produce incomplete evaluations or fail closed. The architecture is domain-agnostic and accepts expanded databases, but v1's coverage reflects the oncology focus of the underlying data sources. Second, the counter-hypothesis categories are curated rather than exhaustively enumerated; a future version could generate counters from the data itself. Third, the scoring functions use manually calibrated thresholds; while these produce biologically sensible results for our test cases, they represent one reasonable parameterization among many. Finally, BioVerdict evaluates hypotheses against existing evidence rather than designing new experiments; it tells you what the data say, not what experiments to run next.

Scored outputs (evidence CSVs, counter CSVs) are deterministic and verified via golden-file SHA256 comparison across 87 automated tests, including 14 ablation-specific tests. Verdict certificates include timestamps and are therefore not byte-identical across runs, but the scored content they audit is deterministic.

References

1. Lord CJ, Ashworth A. PARP inhibitors: Synthetic lethality in the clinic. Science. 2017;355(6330):1152-1158.
2. Tsherniak A et al. Defining a cancer dependency map. Cell. 2017;170(3):564-576.
3. Ochoa D et al. The next-generation Open Targets Platform. Nucleic Acids Res. 2023;51(D1):D1353-D1359.
4. Farmer H et al. Targeting the DNA repair defect in BRCA mutant cells as a therapeutic strategy. Nature. 2005;434(7035):917-921.
5. Bryant HE et al. Specific killing of BRCA2-deficient tumours with inhibitors of poly(ADP-ribose) polymerase. Nature. 2005;434(7035):913-917.
6. Lynch TJ et al. Activating mutations in the epidermal growth factor receptor. N Engl J Med. 2004;350(21):2129-2139.
7. Landrum MJ et al. ClinVar: improving access to variant interpretations. Nucleic Acids Res. 2018;46(D1):D1062-D1067.
8. Buniello A et al. The NHGRI-EBI GWAS Catalog. Nucleic Acids Res. 2019;47(D1):D1005-D1012.