SpectralBio: Local Hidden-State Covariance as a Bounded Zero-Shot Pathogenicity Signal

1. Introduction

Missense variant interpretation remains bottlenecked by the gap between variant discovery and experimental characterization. Sequencing keeps producing variants of uncertain significance much faster than carefully curated pathogenic and benign labels can accumulate, and that imbalance is especially severe outside a small number of intensively studied genes. Zero-shot variant effect prediction is attractive in that regime because it can rank substitutions from sequence context alone, without per-gene supervised retraining. Protein language models have made that setting increasingly plausible, from early large-scale unsupervised protein sequence modeling to ESM-family systems that expose informative mutation-effect signal through masked-language-model likelihoods and related sequence-centered summaries [1,2,6,9,10].

Most zero-shot missense scoring arguments, however, remain output-centric. Classical predictors such as SIFT and PolyPhen-2 emphasize conservation and substitution heuristics [4,5]. Evolutionary latent-variable models such as EVE and DeepSequence use family-level sequence variation and deep generative priors [3,11]. Modern protein language model approaches, including ESM-1v zero-shot scoring in the style of Meier et al. [2], show that residue likelihood alone can already be highly informative. Those methods are the correct starting point for SpectralBio, because they define the dominant intuition that mutation effect is primarily a scalar plausibility problem.

SpectralBio asks whether that intuition is incomplete. A missense substitution may be only moderately surprising at the token level while still inducing a local reorganization of the model's hidden-state geometry. The method is simple: for each variant, we compare wild-type and mutant covariance matrices in a mutation-centered ESM2 window, summarize their displacement with a Frobenius statistic, and combine that term with sequence-based scoring. The central scientific question is whether this local covariance perturbation contains benchmark-relevant signal that is not exhausted by likelihood-only and eigenvalue-only summaries. The paper's first central empirical test is the strongest-baseline version of that question: on BRCA2, does covariance still add signal after the baseline is upgraded from ESM2 likelihood to a five-model ESM-1v ensemble? In this study it does. Adding covariance-aware hidden-state geometry improves BRCA2 AUC from 0.6324 to 0.6890, and that gain disappears under covariance permutation with empirical p = 0.0010.

TP53 then tests whether the underlying covariance signal is real or merely a same-benchmark tuning artifact. The support-ranked top-25 feasible panel tests whether the BRCA2 result sits inside a broader performance-blind pattern rather than a hand-picked companion set. A new failure-mode screen pushes that logic further. On the performance-blind regime to_basic__AND__high_disagreement_q75, selected under a strict gate from the support-ranked panel, the ESM2-150M backbone assigns candidate variants a larger candidate-versus-control covariance gap than matched positive controls, but the stronger 650M backbone largely removes or reverses that pattern: the candidate-control Frobenius gap moves from +0.2946 to -0.4543, the paired gap from +0.1598 to -0.2362, 78.6% of candidate variants repair in the favorable direction, and 6/7 positive-gap genes flip to nonpositive. A new robustness audit tightens that reading rather than merely repeating it: across 13 gene-matched pairs, the mean gap reduction is 0.7250 for Frobenius covariance and 0.3632 for pair covariance with exact sign-flip p = 0.000244 for both, whereas the likelihood channel shows no corresponding repair (-0.0790, p = 0.6960) and local sequence entropy does not support a trivial low-complexity explanation (0.0108, permutation p = 0.9160). A stricter same-position sister-substitution follow-up then fixes gene, position, and source residue while changing only the mutant residue: across 8 sister pairs spanning 5 genes, the mean gap reduction remains 0.6259 for Frobenius covariance and 0.2492 for pair covariance with exact sign-flip p = 0.007812 for both, while likelihood again shows no corresponding repair (-0.2112, p = 0.4922). The sister-substitution follow-up does not erase the regime's chemical narrowness, but it materially reduces the cruder local-context confound and keeps the repair result alive under a harder comparison. This is not a persistence result. It turns covariance from a marginal benchmark increment into a concrete scale-repair signal for a structured small-backbone failure mode within the ESM2 family. That is the manuscript's nearest practical consequence: covariance is not only another score term, but an internal audit readout that can surface a perturbation blind spot that output-only scoring does not flag cleanly.

A follow-up stronger-baseline audit on MSH2 shows the opposite outcome: against the ESM-1v ensemble, covariance hurts rather than helps (0.9233 to 0.8457, delta -0.0776, 95% paired bootstrap confidence interval [-0.1079, -0.0489]). The 192-configuration protocol sweep and the BRCA1 failure analysis therefore matter not as decoration, but because they define where the method breaks, how checkpoint and protocol choices matter, and why the stronger-baseline and scale-repair effects should both be treated as structured and bounded rather than broadly universal.

The contribution is therefore bounded and specific, but no longer only benchmark-centered. SpectralBio is not presented as a universally superior pathogenicity predictor, a clinical deployment recipe, or a replacement for stronger external baselines. Instead, the paper makes two narrower claims. BRCA2 shows that covariance can add signal on top of a strong external baseline. The scale-repair audit, now tightened by a paired robustness analysis and a stricter same-position sister-substitution follow-up, shows that covariance can also expose a structured regime in which a smaller ESM2 backbone misweights local perturbation and a stronger backbone largely repairs that error pattern in a covariance-specific way. TP53 shows that the underlying covariance-likelihood complementarity is real and auditable on an owned canonical benchmark. The support-ranked panel, the MSH2 follow-up audit, the protocol sweep, and the BRCA1 analysis keep those claims bounded: the phenomenon is structured, gene-dependent, and not a universal law.

1.1 Key Findings

The first key finding is the flagship result of the paper. On BRCA2, adding covariance to ESM-1v improves AUC from 0.6324 to 0.6890, with paired bootstrap confidence interval [0.0131, 0.1063], and the gain disappears when covariance alignment is permuted. That is the clearest direct evidence in the manuscript that covariance is contributing real pathogenicity-ranking signal rather than merely re-expressing baseline behavior.

The second key finding is the new qualitative result of the paper. A performance-blind failure-mode screen on the support-ranked panel identifies the regime to_basic__AND__high_disagreement_q75, which passes a strict gate with 8 rescued positives across 4 genes and reference-gap 0.0506. On that regime, the 150M backbone shows a positive candidate-versus-control covariance gap, but the 650M backbone shrinks or flips it (+0.2946 to -0.4543 by Frobenius; +0.1598 to -0.2362 by pair), with 0.7857 candidate-variant repair rate and 0.8571 positive-gap gene flip rate. A follow-up robustness audit strengthens the interpretation: across 13 gene-matched pairs, the mean gap reduction is 0.7250 for Frobenius covariance and 0.3632 for pair covariance with exact sign-flip p = 0.000244, while the likelihood branch shows no matching repair pattern (-0.0790, p = 0.6960). A stricter same-position sister-substitution follow-up then keeps that repair alive under a harder local control: across 8 sister pairs spanning 5 genes, all 8 covariance reductions remain positive, with mean gap reduction 0.6259 for Frobenius covariance and 0.2492 for pair covariance and exact sign-flip p = 0.007812 for both, while likelihood again shows no matching repair (-0.2112, p = 0.4922). This is not a persistence claim. It is a scale-repair result showing that covariance can audit a structured small-backbone failure mode. In that sense, it is closer to an internal model-audit finding than to another marginal benchmark gain.

The third key finding is that TP53 validates the internal covariance-likelihood effect rather than carrying the entire paper alone. On the canonical TP53 benchmark, the released pair remains substantially better than either branch alone, and repeated nested cross-validation places the released alpha on a stable out-of-fold plateau rather than revealing a fragile same-benchmark optimum. TP53 therefore answers the simplest overfitting criticism without forcing the manuscript back into a TP53-only story.

The fourth key finding is that the breadth analysis is no longer hand-built. On the support-ranked top-25 feasible panel, 10 genes show positive pair-vs-likelihood lower confidence bounds, 2 genes are clearly negative, and the remainder are ambiguous. That is scientifically stronger than an all-positive companion panel because it shows heterogeneity on a performance-blind analysis derived from a 15,752-gene scan, not because it demonstrates broad utility.

The fifth key finding is that the negative and boundary cases are now explicit. The 192-configuration sweep shows that covariance utility is checkpoint-, window-, and layer-sensitive rather than a simple small-model artifact, while BRCA1 remains a structured failure case rather than a hidden contradiction. The new MSH2 stronger-baseline audit is the clearest negative replication attempt: the best alpha is 0.0, meaning that the strongest score on that surface is simply the ESM-1v baseline without any covariance contribution. Together those analyses turn the method's hard cases into boundary conditions.

1.2 Claim Boundary

The claim boundary remains deliberately bounded. SpectralBio is not presented as a state-of-the-art pathogenicity predictor, a broad cross-protein deployment recipe, or a clinically actionable tool. ESM-1v remains stronger than the released SpectralBio TP53 surface on raw TP53 discrimination, and TP53 remains the only frozen public canonical benchmark at the time of writing. What the present evidence supports is a narrower representational and auditing result: covariance-aware hidden-state geometry contributes real, falsifiable, benchmark-relevant signal on some evaluation surfaces; on BRCA2 it can improve a stronger external zero-shot baseline; on TP53 it survives nested validation and public-audit scrutiny; on a performance-blind regime selected from the support-ranked panel it exposes a structured scale-repair pattern within the ESM2 family that survives a paired robustness audit and a stricter same-position sister-substitution follow-up; on a support-ranked top-25 panel it appears on a mixed performance-blind analysis; and on MSH2, TP53, BRCA1, and NSD1 it fails to produce a clean fixed-alpha stronger-baseline improvement. The phenomenon is therefore bounded and gene-dependent, not a universal law.

2. Scientific Context and Motivation

Zero-shot missense scoring is attractive precisely because it avoids per-gene supervised retraining, but most current methods still reduce the problem to a scalar token-preference question. In that framing, the key quantity is how strongly the model prefers the wild-type residue over the mutant residue in the surrounding sequence context. That framing is useful and often competitive, yet it may omit a second class of information: a mutation can preserve moderate token plausibility while still reorganizing the local geometry of the hidden representation field.

This is where SpectralBio departs from likelihood-only reasoning. Instead of treating ESM2 only as a conditional scorer, it treats the model as a layered geometric object whose local hidden states carry covariance structure. The manuscript does not argue that every covariance statistic is automatically useful, nor that larger checkpoints necessarily make covariance irrelevant. The narrower hypothesis is that local full-matrix covariance displacement can preserve perturbation information that scalar likelihood summaries and eigenvalue-only compressions do not fully retain. That hypothesis is compatible with the broader observation that protein language models learn structural regularities [1,18,19], but it is tested here in the specific setting of zero-shot pathogenicity ranking rather than in structure prediction or inverse folding.

These comparisons place two questions at the center of the study: whether covariance survives a stronger-baseline comparison on BRCA2, and whether covariance can audit a structured regime in which a smaller backbone misweights perturbation and a larger backbone repairs it. TP53 provides canonical validation, the support-ranked panel provides performance-blind breadth, the failure-mode screen plus stronger-backbone follow-up provides the new scale-repair result, the paired robustness and sister-substitution follow-ups test whether that result survives harder controls, MSH2 provides an explicit negative follow-up under the stronger-baseline framing, benchmark qualification defines the release path, and the protocol plus BRCA1 analyses define boundary conditions. The frozen public benchmark remains important because it makes the validation anchor challengeable.

The exact-split ESM-1v comparison still shows that SpectralBio does not win on raw TP53 discrimination, even after additional supplementary checkpoints are considered. The paper therefore is not a state-of-the-art pathogenicity paper. Its contribution is representational and methodological: it shows that covariance-aware hidden-state geometry can survive a stronger-baseline comparison on BRCA2, that the underlying effect is real and auditable on TP53, that it can expose a structured small-backbone failure mode that is largely repaired at larger scale within the ESM2 family, that it extends beyond the anchor gene on a support-ranked surface with mixed outcomes, and that its behavior under scale is heterogeneous rather than trivially monotone.

3. Related Work

The most relevant prior work for SpectralBio falls into four method families. The first family comprises classical missense effect predictors such as SIFT and PolyPhen-2, which use conservation, substitution patterns, and hand-designed biological features [4,5]. These systems established the practical importance of sequence-based pathogenicity scoring, but they do not interrogate learned internal geometry because they predate modern protein language models.

The second family comprises family-aware evolutionary generative models such as EVE and DeepSequence [3,11]. These methods learn family-level latent models of tolerated variation and have been highly influential in disease variant prediction. SpectralBio is conceptually different: it studies local perturbation geometry in a pretrained protein language model rather than a family-specific generative model, and it does so in a zero-shot setting without family retraining in the released benchmark artifact.

The third family comprises protein language model zero-shot mutation scoring. The ESM-family line represented by Meier et al. [2] demonstrated that language-model-based zero-shot inference can capture mutation effects without supervised fine-tuning. ESM2 established stronger protein representation learning at scale [1], while ProtTrans broadened the view that self-supervised protein language modeling is a viable framework for downstream biological prediction [6]. Closely related work also showed that transformer protein language models encode structural information [18] and that inverse-folding models can leverage learned protein representations at scale [19]. SpectralBio is closest to this family, but departs from it by treating the model not only as a sequence-likelihood oracle but also as a source of layer-wise hidden-state geometry.

The fourth family concerns benchmark culture and gene-resolved functional evidence. ClinVar provides the pathogenic and benign label substrate used here [7]. ProteinGym represents the broader field-level emphasis on explicit, comparable benchmark surfaces [8]. Functional studies of TP53 and BRCA1 reinforce why those genes remain scientifically meaningful evaluation surfaces even when a study does not claim broad cross-protein generalization [13,14]. SpectralBio's methodological stance is aligned with that benchmarking culture: it prefers a bounded, falsifiable claim on clearly owned surfaces over a vague generalization claim spanning many incomparable sources.

At a deeper conceptual level, SpectralBio also sits in a longer line of work on dependency structure and internal representations. Direct-coupling analysis showed that pairwise statistical structure in protein sequences can encode functionally relevant information [15]. Evolutionary Action argued that mutation effect depends on context-sensitive functional displacement rather than a single surprise score [16]. More generally, work such as Neuron Shapley illustrates the value of interrogating internal model components rather than only surface outputs [20]. SpectralBio's narrower move is to ask whether mutation-induced covariance perturbation is a useful internal summary for zero-shot missense pathogenicity ranking, and whether that contribution survives direct comparison to stronger baselines and stronger protocol controls.

4. Methodology

4.1 ESM2 Hidden-State Geometry

SpectralBio uses facebook/esm2_t30_150M_UR50D, the 30-layer 150M-parameter ESM2 checkpoint with hidden dimension d = 640 [1], as the public canonical backbone. Supplementary analyses additionally probe facebook/esm2_t33_650M_UR50D and facebook/esm2_t36_3B_UR50D. For a wild-type sequence S_WT and its corresponding missense mutant S_MUT at position p, the method extracts a mutation-centered local window. The public canonical benchmark uses radius 40 residues on each side. The protocol sweep later tests radii 20, 40, 80, and 120. Near sequence boundaries, the window truncates accordingly.

For each layer l in {1, ..., L}, ESM2 produces a residue-by-feature hidden-state tensor

$$H^{(l)} \in \mathbb{R}^{w \times d}.$$

This local-window design is substantive rather than incidental. The method assumes that the most relevant perturbation signal is concentrated around the mutated site, so SpectralBio is a local hidden-state geometry method rather than a whole-sequence summary.

4.2 Local Covariance Features and Likelihood Terms

For each layer, SpectralBio computes a covariance matrix over the local hidden states,

$$C^{(l)} = \operatorname{Cov}(H^{(l)}) \in \mathbb{R}^{d \times d},$$

and then constructs separate covariance objects for the wild-type and mutant windows, C_wt^(l) and C_mut^(l). From these layer-wise covariance matrices, the manuscript focuses on three statistics:

$$\operatorname{FrobDist} = \frac{1}{L}\sum_{l=1}^{L} \left| C_{\mathrm{mut}}^{(l)} - C_{\mathrm{wt}}^{(l)} \right|_F$$

$$\operatorname{TraceRatio} = \frac{1}{L}\sum_{l=1}^{L} \left| \frac{\operatorname{tr}(C_{\mathrm{mut}}^{(l)})}{\operatorname{tr}(C_{\mathrm{wt}}^{(l)})} - 1 \right|$$

$$\operatorname{SPS\text{-}log} = \frac{1}{L}\sum_{l=1}^{L} \left| \log |\lambda_{\mathrm{mut}}^{(l)}| - \log |\lambda_{\mathrm{wt}}^{(l)}| \right|_2^2,$$

where lambda^(l) denotes the covariance eigenvalue spectrum at layer l. These statistics correspond to different perturbation hypotheses. frob_dist acts on the full covariance difference and is sensitive to both diagonal variance shifts and off-diagonal correlation reorganization. TraceRatio measures variance-scale change. SPS-log is an eigenvalue-only compression.

The likelihood branch uses the masked-language-model term

$$\operatorname{LL}(v) = \log P_{\mathrm{ESM2}}(r_{\mathrm{wt},p} \mid S_{\mathrm{WT}})$$

\log P_{\mathrm{ESM2}}(r_{\mathrm{mut},p} \mid S_{\mathrm{WT}}).LL(v)=logPESM2​(rwt,p​∣SWT​)−logPESM2​(rmut,p​∣SWT​).

This is the repository's ll_proper score. The public canonical TP53 pair is

$$\operatorname{score} = 0.55 \cdot \operatorname{frob_dist} + 0.45 \cdot \operatorname{ll_proper}.$$

Supplementary analyses preserve the same ingredients while varying checkpoint, window radius, layer protocol, and baseline branch. In particular, the ESM-1v augmentation study replaces the ESM2 likelihood branch with a five-model ESM-1v ensemble score and asks whether covariance still contributes useful information.

The computational cost of this choice is real, but it should be interpreted in the same bounded way as the claim itself. Covariance extraction requires retaining local hidden states and forming mutation-centered covariance summaries, so it is more expensive than a scalar likelihood-only readout. In this paper that cost is justified as an offline benchmark and audit expense rather than as a claim of universal high-throughput deployment. The public TP53 canonical replay therefore ships as a frozen executable artifact with precomputed score references and does not require GPU inference, while the strongest-baseline scientific comparison already involves a five-model ESM-1v ensemble whose own cost is nontrivial. The relevant question here is not whether covariance is the cheapest possible score, but whether the extra internal readout earns its keep on bounded audit surfaces.

4.3 Why Local Covariance Might Matter

The central methodological contrast is not "SpectralBio versus one external baseline" but full-matrix covariance displacement versus eigenvalue-only compression within the same hidden-state object. These summaries preserve different classes of information. Likelihood terms reduce the mutation question to residue plausibility under sequence context. Eigenvalue-only summaries preserve spectral magnitudes but compress away orientation and detailed off-diagonal interaction structure. Full-matrix covariance statistics remain sensitive to how correlations among hidden features are reorganized by a substitution.

The biological interpretation is intentionally modest. We do not claim that hidden-state covariance is a direct biochemical observable. We use it as an internal perturbation proxy. A missense variant can leave residue plausibility only moderately changed while still disturbing several coupled local constraints at once, such as packing, polarity, secondary-structure preference, and exposure-sensitive context. If ESM2 represents those constraints across correlated hidden channels, then a mutation may be visible not only as a change in the preferred residue token but also as a change in how local hidden features co-vary. Under that reading, covariance is a summary of local representational reorganization rather than a literal mechanistic model of pathogenicity. The question is therefore not whether covariance is itself "the biology," but whether it is a useful internal summary of mutation-induced context disruption.

The same bounded interpretation also explains why covariance can help on some surfaces and hurt on others. If a gene's discriminative structure is already well captured by residue plausibility or family-level regularities reflected in likelihood, covariance may add little or may inject nuisance variation. If instead a mutation perturbs several coupled local constraints while remaining only moderately surprising at the token level, covariance can add useful information. The BRCA2 stronger-baseline gain and the charge-sensitive scale-repair regime are of the latter type; MSH2 is direct evidence that the former case exists and matters.

A short technical point clarifies why full-matrix covariance can differ from eigenvalue-only summaries. If we define

$$\Delta C^{(l)} = C_{\mathrm{mut}}^{(l)} - C_{\mathrm{wt}}^{(l)},$$

then

$$\left| \Delta C^{(l)} \right|_F^2$$

\sum_i \left(\Delta C^{(l)}_{ii}\right)^2

• 2 \sum_{i<j} \left(\Delta C^{(l)}_{ij}\right)^2.​ΔC(l)​F2​=i∑​(ΔCii(l)​)2+2i<j∑​(ΔCij(l)​)2.

This decomposition shows that Frobenius displacement retains both diagonal variance changes and off-diagonal correlation changes. Eigenvalue-only summaries do not preserve all of that information. That technical fact is simple, but it matters here because the empirical question is precisely whether off-diagonal reorganization adds usable ranking signal beyond scalar likelihood. The answer in this paper is not "always"; it is "sometimes, in a bounded and gene-dependent way."

Its simplicity is deliberate. We use Frobenius displacement as a first full-matrix audit statistic, not because it is the only conceivable geometric summary, but because it is transparent, falsifiable, and easy to compare against weaker compressions. That choice is empirically constrained rather than arbitrary: on TP53, eigenvalue-only summaries such as sps_log and variance-scale summaries such as trace_ratio do not recover the released pair performance, so the result is not obtained by "any covariance number whatsoever." Richer directional probes remain a valid future direction, but the present manuscript uses Frobenius precisely because it is simple enough to audit and strong enough to falsify.

4.4 Validation Program and Claim Hierarchy

The evidence program is defined by question class rather than by post hoc selection of favorable analyses. BRCA2 ESM-1v augmentation plus permutation audit is the strongest-baseline test of whether covariance contributes anything nontrivial. TP53 is the validation anchor, examined on the frozen canonical benchmark and then re-audited under repeated nested cross-validation. The anti-cherry-picking breadth analysis begins from a global ClinVar support scan over 15,752 genes and applies thresholds min_total = 60 and min_per_class = 20, yielding 446 genes that pass the scan stage before support-ranked feasible selection into the realized top-25 panel. A new failure-mode screen is nested inside that performance-blind panel logic: it selects regimes by explicit rescue, harm, and reference-gap thresholds, then asks whether the selected 150M pattern persists or repairs when rescored on a stronger 650M backbone. The new MSH2 stronger-baseline audit is not a flagship result; its role is to test whether the BRCA2 external-baseline gain replicates on another high-support gene with cheap full-sequence ESM-1v scoring. It does not.

The benchmark-extension rule is also explicit. A non-anchor gene qualifies as the next canonicalization target only if its paired pair-vs-likelihood bootstrap lower confidence bound is positive and its nested fixed-0.55 mean AUC exceeds its nested likelihood-only mean AUC. Under that rule, BRCA2 qualifies. MSH2 does not qualify under the stronger-baseline framing: the fixed-alpha ESM-1v augmentation is strongly negative and the best alpha collapses to 0.0. The qualitative failure-mode claim is likewise bounded: the regime is selected performance-blindly first and only then interpreted through stronger-backbone gap reduction, variant-level repair, and gene-level gap flips. The protocol criticism is addressed through a bounded but direct sweep over 4 genes, 3 checkpoints, 4 window radii, 4 layer protocols, and 3 alpha-handling modes, for 192 scored configurations. The negative-case criticism is addressed through a dedicated BRCA1 failure analysis that asks whether the most visible hard case is uniformly negative or structurally mixed.

These analyses answer different evidentiary questions. BRCA2 carries the strongest claim that covariance can add signal to a stronger external baseline. The scale-repair audit carries the strongest claim that covariance can expose a structured representational failure mode rather than only raise benchmark AUC. TP53 shows that the underlying covariance signal is real and not a same-benchmark tuning artifact. The support-ranked top-25 panel shows that the paper is not built from favorable hand-selection. The MSH2 follow-up shows that the external-baseline gain does not replicate generically. The protocol sweep and BRCA1 analysis define the remaining boundaries of the current method.

4.5 Statistical Procedures, Panel Construction, and Augmentation Protocol

Unless otherwise noted, all bootstrap intervals in the manuscript use 1000 nonparametric resamples with seed 42. AUC intervals are computed from resampled score vectors on the same benchmark surface. Pair-vs-baseline intervals use paired bootstrap resampling of the same variant indices for both scores, so the reported delta intervals are paired rather than independent. Resamples containing only one class are discarded, and the reported interval is the empirical 2.5th to 97.5th percentile over valid resamples. The TP53 and BRCA2 nested audits use 5 x 5 repeated stratified cross-validation. Repeat seeds are 42 through 46, outer folds are stratified with those repeat seeds, and each inner alpha search uses the repeat seed plus the fold index.

MinMax scaling is fit separately for each score family. On a fixed evaluation surface, frob_dist, ll_proper, and, where used, the ESM-1v ensemble mean are each independently scaled to [0, 1] using minima and maxima from that same scored table before pair construction. In nested cross-validation, the minima and maxima are fit on the training folds only and then applied to the held-out fold, so held-out values do not influence scaling. The fixed public pair remains 0.55*frob_dist + 0.45*ll_proper, and all exploratory alpha sweeps use the grid 0.00, 0.05, ..., 1.00.

The support-ranked panel is built in two stages. First, genes are ranked by descending min(n_positive, n_negative), then descending n_total, then ascending gene symbol after ClinVar filtering to GRCh38, single-nucleotide variants, germline-or-empty origin, binary pathogenic/benign labels, and simple missense protein changes. Second, ranked genes enter the realized panel only if they are feasible, meaning that the run can retrieve a reviewed human UniProt sequence, map retained ClinVar missense rows against that sequence, discard out-of-range and sequence-mismatched entries, and still preserve enough binary rows to score the gene. The panel-expansion run fixed top_k_requested = 25 and realized all 25 genes, so the panel size reflects the target supplementary surface rather than post hoc stopping on favorable results.

The ESM-1v augmentation protocol keeps the covariance branch on the canonical ESM2-150M reference backbone and replaces the baseline branch with the normalized mean zero-shot likelihood from the five-model ESM-1v ensemble facebook/esm1v_t33_650M_UR90S_{1..5}. The fixed augmented score is therefore 0.55*frob_norm + 0.45*esm1v_norm, and the exploratory full-surface alpha sweep again uses the 0.05 grid. The permutation audit uses 1000 replicates with seed 42 while holding labels fixed: one null permutes covariance alignment against ESM-1v, and the other permutes ESM-1v alignment against covariance. The empirical tail probability is reported as (count(delta_null >= delta_observed) + 1) / (B + 1).

5. Benchmark Design and Evidence Program

5.1 Evidence Hierarchy and Role Assignment

The analyses in this study are not inferentially interchangeable. The strongest single result is the BRCA2 augmentation audit against ESM-1v, and the remaining analyses test whether that result is real, bounded, and non-accidental. Table 1 summarizes the evidence program from the strongest-baseline test through validation, qualitative failure-mode audit, breadth, and boundary analyses.

5.2 Challenge-to-Evidence Map

Table 2 maps the main scientific challenges to the corresponding evidence layers used here.

5.3 Support-Ranked Selection and Benchmark Extension

The supplementary panel begins from a global ClinVar scan, applies explicit support thresholds, and then uses a support-ranked feasibility filter to decide which non-anchor genes are actually scored. Support-ranked means descending min(n_positive, n_negative), then descending n_total, then ascending gene symbol. Feasible means that a ranked gene can be paired with a reviewed human UniProt sequence, retain binary missense rows after ClinVar parsing and sequence checks, and still yield a scoreable reference surface. The scan sees 15,752 genes, of which 446 pass the threshold stage. The panel-expansion run fixed the target size at 25 and realized all 25 genes, so the realized panel is BRCA1, TP53, BRCA2, KMT2D, DMD, DNAH5, CHD7, MSH2, NSD1, SACS, DNAH11, COL7A1, ADGRV1, ANKRD11, TSC2, PKD1, CREBBP, DYNC1H1, GRIN2A, GRIN2B, USH2A, COL2A1, SYNGAP1, KMT2A, and ZEB2. That provenance matters because the panel is bounded, but not hand-picked by observed performance.

The benchmark-extension rule is likewise operational. A non-anchor gene qualifies as the next benchmark candidate only if its paired pair-vs-likelihood lower confidence bound is positive and its nested fixed-0.55 mean AUC exceeds its nested likelihood-only mean AUC. Under that rule, BRCA2 qualifies. NSD1 and MSH2 remain scientifically interesting, but both fail the release criterion because their lower bounds remain non-positive. The paper therefore has a concrete next benchmark surface instead of an undefined post-TP53 horizon.

5.4 Frozen Public Benchmark

The frozen public benchmark remains one of the study's strengths. The public canonical setup is uv sync --frozen followed by uv run spectralbio canonical, which materializes the frozen TP53 artifact from bundled inputs and bundled score references. GPU is not required for that canonical replay path. Verification remains machine-checkable through outputs/canonical/summary.json and outputs/canonical/verification.json, with the latter exposing structured pass/fail fields for file-set validation, metric agreement, schema alignment, and artifact completeness. BRCA2 is the manuscript's flagship scientific result, but TP53 remains the only frozen public canonical replay surface. The benchmark therefore makes the covariance claim numerically challengeable on a public surface.

5.5 Public Analysis Artifacts

The BRCA2 ESM-1v augmentation analysis contains the strongest direct test of whether covariance adds signal beyond a stronger baseline, because it joins the baseline-versus-augmentation comparison to the covariance-permutation null on the same benchmark. The BRCA2 canonicalization analysis shows that BRCA2 is not merely a favorable auxiliary example, but the only non-anchor gene that currently satisfies the benchmark-promotion rule. The performance-blind failure-mode screen and the 650M scale-repair follow-up complement those benchmark-centered artifacts by showing that covariance can also function as an audit signal for structured representational errors rather than only as an AUC-raising feature. The paired robustness audit and the same-position sister-substitution follow-up extend that chain within the submission bundle by showing that the effect survives matched-pair testing, remains covariance-specific, is not explained by local low complexity, and persists under a stricter local-control comparison. TP53 remains the public validation anchor because it is the only frozen public canonical replay surface, whereas BRCA2 carries the manuscript's flagship scientific result.

6. Results

We begin with the BRCA2 augmentation audit because the strongest-baseline test provides the sharpest test of the central hypothesis. TP53 then serves as the validation anchor that shows the underlying covariance signal is real and auditable. The support-ranked top-25 panel addresses breadth and anti-cherry-picking, and the new failure-mode screen extracts from that panel a qualitatively different result: a structured regime in which the 150M backbone misweights covariance relative to matched controls and the 650M backbone largely repairs that pattern. The protocol sweep and BRCA1 analysis then define the remaining boundary conditions.

6.1 Flagship Stronger-Baseline Result

The main alternative explanation faced by SpectralBio is straightforward: perhaps covariance only appears useful because the baseline branch is weak. The BRCA2 ESM-1v augmentation audit is designed to answer exactly that objection. The core comparison is a three-way contrast on BRCA2 between the ESM-1v baseline, the aligned covariance-plus-ESM-1v score, and the covariance-permuted null. In that setting, covariance still adds signal.

Figure 1 summarizes the BRCA2 result across its three evidentiary components: stronger baseline, observed gain, and direct falsification.

Figure 1. BRCA2 augmentation and falsification summary. Panel A compares BRCA2 discrimination under the five-model ESM-1v ensemble alone (AUC 0.6324) and the aligned covariance-plus-ESM-1v score (AUC 0.6890). Panel B reports the paired gain over ESM-1v (+0.0566, 95% paired bootstrap CI [0.0131, 0.1063]). Panel C shows the covariance-alignment permutation null, whose mean is -0.0350; no null replicate reaches the observed aligned gain, giving empirical p = 0.0010. Together these panels summarize why BRCA2 provides the strongest direct test of the central hypothesis.

The BRCA2 row is the manuscript's most memorable empirical result. The aligned covariance augmentation improves over ESM-1v by +0.0566 AUC and survives paired bootstrap uncertainty in the favorable direction. The accompanying permutation audit closes the loop: when covariance alignment is destroyed, the observed BRCA2 gain is no longer reachable. Under covariance permutation, the null mean becomes -0.0350, with empirical probability p = 0.0010 of producing a value at least as large as the observed aligned result. This is the clearest direct evidence in the paper that covariance is contributing real signal rather than just repackaging baseline behavior.

The other rows calibrate scope rather than compete with BRCA2 for narrative centrality. TP53 prevents overclaiming because the fixed 0.55 augmented score does not beat ESM-1v, even though the exploratory full-surface best-alpha result slightly exceeds it. BRCA1 is inconclusive under this augmentation view, and NSD1 remains negative to ambiguous.

MSH2 sharpens that boundary further. It is not a mild disappointment but a clear stronger-baseline failure: the fixed augmented score falls from 0.9233 to 0.8457, the paired bootstrap interval is entirely negative, and the best alpha on the full surface is 0.0, meaning the best score is obtained by discarding covariance altogether. The stronger-baseline story is therefore not a replicated multi-gene pattern. It is a bounded result in which BRCA2 is the clean positive case and MSH2 is the clearest non-replication.

The result is therefore narrow and strong: the paper does not claim that covariance improves every strong baseline everywhere; it claims that BRCA2 is a clean, falsifiable case where it does, while MSH2 shows that the same recipe can fail decisively.

6.2 TP53 as Validation Anchor

TP53 serves a different job. It is the owned canonical benchmark on which the manuscript can show that covariance signal is real, reproducible, and not dependent on post hoc retelling. On the frozen TP53 benchmark, the released pair 0.55*frob_dist + 0.45*ll_proper achieves official AUC 0.7498 and released value 0.749751552795031, matching within the declared tolerance. The benchmark therefore remains both numerically reproducible and verification-backed.

The key TP53 pattern is complementarity rather than single-feature dominance. ll_proper and frob_dist are both modest alone, while their released combination is substantially higher. The pair improves over ll_proper by +0.1542 AUC and over frob_dist by +0.1288. Within the original released artifact, ll_crude reaches 0.7026 and the released triple ll_crude + TraceRatio + frob_dist reaches 0.7264, but neither exceeds the canonical pair. TP53 therefore still supports the same mechanistic reading: a matrix-level perturbation statistic and a likelihood term contribute complementary information.

The low standalone ll_proper value on this frozen TP53 surface should not be read as evidence of a broken ESM2 likelihood pipeline. It is a score-definition and benchmark-surface issue. The public replay uses a strict binary ClinVar filter, a frozen TP53 variant file, and the repository's masked conditional ll_proper formulation rather than the broader mix of score definitions, label surfaces, and stronger baselines often reported in the literature. On the same repository and surrounding evidence program, ll_crude is materially higher on TP53 (0.7026), the five-model ESM-1v ensemble reaches 0.9466 on the exact TP53 split, and nested BRCA2 likelihood remains coherent at 0.6938. The correct inference is therefore not that the TP53 likelihood branch is bugged, but that the frozen ll_proper TP53 surface is conservative and not numerically interchangeable with literature values reported for different datasets, filters, or baseline definitions.

The repeated nested cross-validation audit then addresses the main criticism of the TP53 anchor, namely that the released 0.55/0.45 pair may be only a same-benchmark tuning spike. Across 5 repeats and 5 outer folds per repeat, the tuned-alpha mean AUC is 0.7485, the fixed 0.55 mean AUC is 0.7510, and the fixed 0.50 mean AUC is 0.7480. The likelihood-only and covariance-only terms remain much lower, with mean AUC 0.5983 for ll_proper and 0.6206 for frob_dist.

The chosen-alpha distribution is stable rather than erratic. Across the 25 outer folds, the chosen alpha has mean 0.58 and standard deviation 0.0548, with counts concentrated around the released value. Nested tuning does not reveal a hidden collapse in performance, and the released 0.55 setting performs essentially as well as or slightly better than the out-of-fold tuned average. TP53 therefore does not carry the manuscript as a universal benchmark, but it does validate that the underlying covariance effect is real and challengeable.

6.3 BRCA2 Benchmark Extension and Canonicalization Evidence

BRCA2 is also the only non-anchor gene that satisfies the manuscript's promotion rule for benchmark extension. That matters because the flagship BRCA2 augmentation result would be narratively weaker if BRCA2 remained only a favorable auxiliary example. The dedicated BRCA2 audit shows positive point-estimate gains across all three checkpoints, with strongest confidence at 650M and a wider 3B interval that still includes zero.

The nested BRCA2 audit is equally important because it elevates BRCA2 from a favorable point estimate to a release-ready benchmark candidate. Across 25 outer folds, the fixed 0.55 mean AUC is 0.7448, the fixed 0.50 mean AUC is 0.7439, and the tuned-alpha mean AUC is 0.7409, while the nested likelihood-only mean AUC is 0.6938. The chosen alpha has mean 0.616 and standard deviation 0.088, again consistent with a plateau rather than a knife-edge optimum.

This is the relevant evidentiary threshold for calling BRCA2 the next canonicalization target. The argument is no longer just that BRCA2 looks good in one auxiliary comparison. It is that BRCA2 is the only non-anchor gene that satisfies a predeclared benchmark-promotion rule under both paired-confidence and nested-performance criteria.

6.4 Support-Ranked Top-25 Panel

The support-ranked top-25 feasible panel exists to answer a different challenge: whether the paper's positive story depends on favorable hand-selection. It covers 10,992 variants in total, with 3,019 pathogenic and 7,973 benign variants. Because the panel is derived from a 15,752-gene scan with transparent thresholds and ranking rules, it carries far more scientific weight than the earlier hand-built companion panel.

The central panel result is not uniform success. Instead, it is structured heterogeneity on a performance-blind surface. Ten genes have positive pair-vs-likelihood lower confidence bounds: TP53, BRCA2, KMT2D, DNAH5, CHD7, ANKRD11, TSC2, PKD1, CREBBP, and KMT2A. Two genes are clearly negative with upper confidence bounds below zero: BRCA1 and COL2A1. The remaining thirteen genes are ambiguous. That is more informative than a selectively favorable smaller panel because it shows where covariance helps, where it fails, and where the current evidence remains inconclusive.

Put differently, 15 of the 25 genes do not provide positive lower-bound support for the pair under this analysis. That is a weakness for any universal-upgrade claim, but it is not a contradiction of the paper's stated claim boundary. The top-25 panel is used here as a performance-blind heterogeneity map, not as evidence that covariance should improve every supported gene.

This panel changes the interpretation in two ways simultaneously. It broadens anti-cherry-picking credibility, because covariance utility is visible beyond TP53 on multiple independent genes and includes BRCA2 on a performance-blind analysis. It also makes the paper more credible, because the panel contains real negatives and ambiguities instead of a curated all-green transfer story.

The panel does not, however, establish stronger-baseline generalization. MSH2 is the clearest example of that distinction: on the top-25 panel it is only marginally positive against the internal likelihood branch (+0.0028, confidence interval crossing zero), but in the dedicated ESM-1v follow-up it becomes decisively negative (-0.0776, confidence interval entirely below zero). The top-25 panel should therefore be read as a map of heterogeneity and anti-cherry-picking provenance, not as evidence that panel-positive genes will also improve strong external baselines.

6.5 Structured Failure-Mode and Scale Repair

The support-ranked panel also enables a qualitatively different analysis from per-gene AUC comparison. Instead of asking only whether the pair beats likelihood on each gene, we ask whether the panel contains a structured regime in which covariance and the reference score disagree systematically. The failure-mode screen applies explicit rescue, harm, and reference-gap thresholds and selects the regime to_basic__AND__high_disagreement_q75, which passes the strict gate with 8 rescued positives across 4 genes, 5 harmed benign variants, and reference-gap 0.0506. The point of this screen is not to claim a new benchmark. It is to find a performance-blind regime worth auditing under scale.

The selected validation pool contains 31 variants: 13 matched positive controls, 14 candidate positives, and 4 regime benigns. On the 150M screen surface, candidate variants exhibit a larger covariance displacement than the matched positive controls, with candidate-control gap +0.2946 by Frobenius and +0.1598 on the paired score. If that pattern were simply a stable biological distinction that the smaller model happened to detect first, the stronger backbone should preserve it. It does not.

Under the 650M backbone, the candidate-control gap flips to -0.4543 by Frobenius and -0.2362 on the paired score. The Frobenius gap reduction is 0.7489, 11/14 candidate variants repair in the favorable direction, and 6/7 positive-gap genes flip from positive to nonpositive. The correct reading is therefore scale repair rather than persistence. The selected regime exposes a structured case in which the smaller backbone overstates the candidate-versus-control covariance discrepancy and the stronger backbone largely removes or reverses it.

Part 10 tightens that interpretation from a descriptive follow-up into a paired robustness result. Across 13 gene-matched candidate-control pairs, the mean gap reduction is 0.7250 for Frobenius covariance and 0.3632 for pair covariance, with exact sign-flip p = 0.000244 for both metrics. The likelihood channel does not show the same behavior: its mean gap reduction is -0.0790 with p = 0.6960. This matters because it shows that the repair signal is not just "the stronger model fixes everything." It is concentrated in covariance space.

The same robustness audit also argues against the easiest trivial explanation. Candidate minus control local sequence entropy differs by only 0.0108, with 95% bootstrap interval [-0.1726, 0.1931] and permutation p = 0.9160, so the scale-repair effect is not explained by a simple low-complexity collapse. At the same time, the audit keeps the claim honest by exposing a non-repair tail rather than hiding it: BRCA2 p.Gly2793Arg, TSC2 p.Asn1564Lys, and NSD1 p.Cys1920Arg remain positive under Frobenius repair delta. The right interpretation is therefore not a universal bug, but a narrow charge-sensitive failure mode that is strong enough to audit, falsifiable under pairing, and still bounded by explicit exceptions.

Part 11 asks the harder local-control question that Part 10 leaves open: could the effect still be driven by coarse matching differences rather than by the substituted residue under a fixed local context? The same-position sister-substitution follow-up answers that with a narrower but stronger comparison family. It keeps only pairs sharing gene, position, and wild-type residue, so the candidate and control differ mainly in the mutant residue. Across 8 such sister pairs spanning 5 genes, all 8 covariance reductions remain positive, with mean gap reduction 0.6259 for Frobenius covariance and 0.2492 for pair covariance and exact sign-flip p = 0.007812 for both. The likelihood branch again does not show the same repair pattern (-0.2112, p = 0.4922).

The sister-substitution follow-up does not solve every caveat. It does not diversify chemistry: the candidate arm remains entirely to_basic, while the sister controls remain non-basic. But it removes the cruder explanation that the Part 10 result is only a byproduct of comparing variants from different local contexts or different source residues. The repair survives when gene, locus, and wild-type residue are fixed and only the substitution changes.

This result changes the scope of the paper. SpectralBio is no longer only showing that covariance can improve or fail to improve benchmark AUC. It is also showing that covariance can audit where model scale changes the representation of local perturbation, and that this audit survives both a paired robustness pass and a stricter same-position follow-up that the likelihood branch does not replicate. Because the regime is selected performance-blindly from the top-25 panel rather than hand-built after seeing the 650M outcomes, the scale-repair interpretation is harder to dismiss as narrative cherry-picking.

6.6 Protocol Sweep and Boundary Conditions

The protocol sweep closes the remaining version of the scaling criticism. It is no longer enough to say that the manuscript reran a larger checkpoint. The current study tests 192 configurations spanning four genes, three checkpoints, four window radii, four layer protocols, and three alpha-handling modes. The resulting pattern is more informative than monotone improvement. Covariance utility is real, but protocol-sensitive. Small windows and shallower layer subsets often outperform the canonical 40/all_layers setting, especially for TP53 and BRCA2.

This table sharpens the interpretation of multiple genes. TP53 is not just positive; it is strongly protocol-sensitive, with best nested gains that exceed the canonical public setting. BRCA2 shows positive point-estimate gains across all three checkpoints, with strongest support at 650M and a wider 3B interval that still includes zero. BRCA1, perhaps the most important hard case, is no longer a simple negative-transfer result: the canonical 40/all_layers setting is negative, yet smaller-window protocols recover modest positive deltas.

That BRCA1 pattern becomes more interpretable in the dedicated failure analysis. The global BRCA1 result is negative, with pair AUC 0.8283 versus likelihood 0.8527, but that negativity is not uniform across confidence, review-status, and domain strata.

The correct interpretation is therefore not that BRCA1 disproves covariance. It is that the canonical BRCA1 full-set configuration is likelihood-dominant, but the failure is structured. High-confidence and expert-panel subsets are more negative than the whole set. BRCT1 is negative, while BRCT2 and RING are near-neutral to slightly positive. Combined with the protocol sweep, this makes BRCA1 a region- and protocol-sensitive boundary case rather than evidence that covariance is globally useless outside TP53.

6.7 Negative Controls and Falsification

The negative controls remain claim-bearing evidence rather than cosmetic appendix material. On the frozen TP53 score surface, 1000 label permutations produce an AUC distribution with mean 0.4994, standard deviation 0.0360, and observed range [0.3871, 0.6048], while the released canonical pair remains at 0.7498; no permutation reaches that value, giving an empirical probability p = 0.0010. Once biological label alignment is destroyed, the score collapses to chance-level behavior.

A second supplementary control destroys local positional context while preserving amino-acid composition and the central residue identity. Across three independent shuffled-context reruns, the released canonical pair drops from 0.7498 to 0.5759, 0.5286, and 0.5117, with mean 0.5388 and standard deviation 0.0333. The mean drop relative to the released pair is therefore 0.2110. This shows that the TP53 anchor is not merely an artifact of residue composition; it depends on intact local context geometry.

The BRCA2 permutation audit completes the manuscript's strongest falsification loop. On BRCA2, the observed covariance-on-ESM-1v gain is +0.0566. Under covariance-alignment permutation, the null mean is -0.0350, and the observed gain becomes unattainable at empirical probability p = 0.0010. Under ESM-1v-alignment permutation, the null mean is strongly positive in the opposite direction, again making the observed aligned gain non-generic. Taken together, TP53 and BRCA2 show two different kinds of falsification support: TP53 shows that the base covariance signal collapses when labels or context are broken, and BRCA2 shows that the strongest-baseline gain collapses when covariance alignment is broken.

7. Discussion

The combined evidence supports a narrow but coherent interpretation of SpectralBio. The paper's strongest result is not merely that covariance helps on an internal TP53 comparison. It is that on BRCA2, covariance-aware hidden-state geometry improves a stronger ESM-1v ensemble baseline, and that the gain disappears under covariance permutation. That is the manuscript's clearest answer to the main alternative explanation, namely that covariance only appears useful because the comparison baseline is weak.

TP53 then takes on a cleaner role. It is the validation anchor that shows the underlying covariance signal is real, reproducible, and not a same-benchmark tuning accident. The frozen TP53 benchmark establishes complementarity between covariance and likelihood, while the nested audit shows that the released alpha sits on a stable out-of-fold plateau. TP53 therefore does not have to carry the whole paper alone. It has to show that the mechanism being invoked by the BRCA2 result is not illusory.

The newly added failure-mode screen changes the paper's scope in a different direction. The regime to_basic__AND__high_disagreement_q75 is not another benchmark win. It is a performance-blindly selected region of the panel in which the 150M backbone shows a positive candidate-versus-control covariance gap and the stronger 650M backbone largely removes or reverses it. Parts 10 and 11 matter because they convert that from a suggestive pattern into a sharper audit chain. Part 10 shows that the matched-pair gap reduction is large and significant for covariance, absent for likelihood, and not explained by a trivial low-complexity difference. Part 11 asks the harder control question by fixing gene, position, and source residue, and the covariance repair still survives while the likelihood branch remains mixed. That makes covariance an auditing instrument for representational scale effects, not only a feature that can be blended into an AUC score. It is also the paper's strongest answer to the benchmark-only critique, because the covariance signal is doing diagnostic work that a scalar output score does not replicate. The right interpretation is not that the 150M regime reveals a hidden biological truth that persists at larger scale; it is that covariance can expose a structured small-backbone failure mode that is largely repaired under scale within the ESM2 family.

The support-ranked top-25 panel serves a similarly specific purpose. Its value is not that it turns the manuscript into a universal generalization claim. Its value is that it breaks the hand-picked-companion critique. Because the panel is derived from a 15,752-gene scan with explicit thresholds and a support-ranked feasibility rule, the reader is no longer being asked to trust a favorable set of examples selected after seeing deltas. The resulting pattern is heterogeneous, not uniformly positive, and that is exactly why it is scientifically useful.

MSH2 is equally important because it prevents the BRCA2 result from being overstated. On MSH2, the fixed augmented score falls from 0.9233 to 0.8457 against ESM-1v, the paired confidence interval is entirely negative, and the best alpha on the full surface is 0.0. That is not a near miss; it is a decisive non-replication. The right conclusion is therefore not that covariance reliably improves strong baselines, but that it can help some genes and hurt others when paired with a strong external baseline.

The protocol sweep and BRCA1 analysis complete the story by showing that covariance utility is structured rather than monotone. BRCA2 remains supportive across checkpoints at the point-estimate level. TP53 becomes even stronger under some smaller-window and shallower-layer protocols than under the public canonical setting. BRCA1 remains negative overall, but not uniformly so; the failure is concentrated in identifiable strata and domains. These are not side issues. They are the current boundary conditions of the method.

The broader implication is not clinical deployment. It is benchmark and auditing culture. Zero-shot missense work should not treat scalar likelihood as the only internal readout worth auditing. The relevant comparison set is broader: likelihood-only, eigenvalue-only, full-matrix covariance, covariance plus likelihood, covariance plus stronger external baselines, performance-blind regime screens, and bounded protocol sweeps that test whether the effect survives reasonable perturbations or repairs under scale. SpectralBio does not settle that agenda, but it does show that covariance is worth auditing under explicit benchmark and falsification rules.

The reproducibility strength remains important in that context. The canonical benchmark is unusually challengeable because it is backed by a frozen public replay path, machine-readable expected outputs, and machine-checkable verification artifacts. That infrastructure does not substitute for empirical evidence; it makes the empirical claim directly testable and gives the manuscript a harder floor under criticism.

8. Limitations and Non-Claims

TP53 remains the only frozen public canonical benchmark. BRCA2 is now a release-ready second benchmark candidate under a predeclared rule, and it is also the manuscript's strongest augmentation result, but it is not yet exposed as a published CPU-only canonical bundle with the same first-class replay surface as TP53. The paper therefore has one released public benchmark, one explicit next-canonicalization path, and one flagship supplementary result rather than two released canonical benchmarks.

The support-ranked top-25 panel is still selected rather than exhaustive. The global scan removes the charge of hand-picking because panel construction begins from 15,752 observed genes and 446 threshold-passing genes under a transparent ranking rule. Even so, the final panel is a bounded feasible subset after sequence retrieval, ClinVar parsing, and sequence-consistency filtering, not a full survey of all qualifying genes. The breadth analysis is therefore stronger than a hand-built companion panel, but it does not erase the distinction between support-ranked validation and exhaustive external generalization.

The breadth analysis is also still ClinVar-based. It is not an orthogonal functional benchmark in the style of a full external deep-mutational-scanning or saturation-assay benchmark. The top-25 result should therefore be interpreted as a structured ClinVar generalization analysis rather than as assay-complete external validation.

ClinVar is also not pristine ground truth. It is noisy, submission-heterogeneous, and partly circular as a benchmarking substrate because many classifications incorporate evidence streams that overlap with what zero-shot predictors can exploit, including conservation, prior functional evidence, and expert interpretation. That limitation applies equally to the likelihood and covariance arms on the same filtered surfaces, so it does not by itself explain within-surface paired deltas such as the BRCA2 gain or the scale-repair audits. It does, however, limit how strongly any ClinVar-only improvement should be interpreted clinically, which is why the manuscript frames these results as representational and benchmark-level rather than clinical-deployment claims.

The BRCA2 ESM-1v augmentation result is strong but still bounded. On BRCA2, the covariance augmentation over ESM-1v is positive with a positive confidence interval and survives a permutation audit. On TP53, the fixed 0.55 augmented score does not beat ESM-1v, even though the full-surface best-alpha exploratory result does. On MSH2, the stronger-baseline follow-up is decisively negative and the best alpha collapses to 0.0. The correct interpretation is therefore that BRCA2 is the clean positive stronger-baseline case, TP53 remains suggestive but not release-ready under that specific comparison, and MSH2 demonstrates a clear failure mode.

The scale-repair result is likewise bounded. It is observed in one performance-blindly selected regime inside the support-ranked panel, and the repair claim is within the ESM2 family (150M to 650M), not across unrelated protein language model families. The validated pool is also chemically narrow: every candidate substitution in the robustness audit is to_basic, whereas no matched control is. The sister-substitution follow-up reduces a cruder local-context confound by fixing gene, position, and source residue, but it does not erase that chemical narrowness because the candidate arm remains entirely to_basic and the sister controls remain non-basic. The result therefore supports a structured charge-sensitive small-backbone failure mode under the present audit, not a universal statement about all protein language models or all covariance-detected regimes.

The method is also not presented as a throughput-optimized replacement for simpler scores. If the claim were that covariance should be attached to every zero-shot pipeline by default, the extra cost would be much harder to justify. The narrower claim made here is that covariance is worth paying for on bounded offline benchmark and audit surfaces, especially when the question is representational diagnosis rather than cheapest-possible scoring.

The protocol sweep closes the simplest scaling criticism without exhausting the space. The current study tests 192 configurations across checkpoints, windows, layer subsets, and alpha handling, which is enough to reject the claim that covariance utility is merely a 150M artifact. It is not enough to settle every scaling question or define a universal optimal protocol. The sweep covers four genes, not all genes in the panel; it covers four window radii and four layer protocols, not an exhaustive protocol universe; and it still leaves open richer combination rules and broader backbone families.

These boundaries imply specific non-claims. The paper does not claim broad cross-protein portability, clinical deployment, clinical decision support, or superiority to external predictors on a shared public benchmark. It does not claim that larger ESM2 checkpoints monotonically strengthen covariance-aware scoring. It does not claim that the support-ranked top-25 panel defines a universal law of covariance utility. It does not claim that the exploratory TP53 best-alpha augmentation over ESM-1v is already a release-grade result. These caveats are part of the result, not decorative qualifications.

9. Benchmark Extension Path

The paper has a concrete answer to the question of what comes after TP53. Under the predeclared benchmark-extension rule, BRCA2 is the only non-anchor gene that qualifies. Its paired lower confidence bound is positive, its nested fixed-0.55 mean AUC exceeds its nested likelihood-only mean AUC, and it retains substantial support after filtering. That makes BRCA2 the correct next target for canonicalization.

Canonicalization still requires real work, but it is no longer hypothetical. A BRCA2 public benchmark release would need the same elements that made TP53 challengeable: a frozen sequence reference, a frozen benchmark variant file, score references, configuration and manifest files, expected metrics, and machine-checkable verification artifacts. For full manuscript-artifact parity, the BRCA2 augmentation and permutation surfaces should also be exposed as first-class public audit outputs rather than only as notebook-generated analyses. The current evidence justifies that work scientifically.

Beyond BRCA2, the relevant next step is not to search for another favorable companion gene. It is to ask which gene properties make covariance useful, neutral, or harmful under the same bounded promotion rule as canonicalization infrastructure expands. The top-25 panel, the BRCA1 strata, and the MSH2 non-replication turn that into a concrete agenda.

10. Conclusion

SpectralBio supports a bounded conclusion. On BRCA2, adding covariance-aware hidden-state geometry to a five-model ESM-1v ensemble improves AUC from 0.6324 to 0.6890, and that gain disappears under covariance permutation with empirical p = 0.0010. This is the manuscript's clearest evidence that covariance contributes recoverable pathogenicity-ranking signal beyond a stronger external zero-shot baseline. A separate performance-blind failure-mode audit shows that covariance can do something different from benchmark augmentation: on the regime to_basic__AND__high_disagreement_q75, the candidate-control Frobenius gap flips from +0.2946 at ESM2-150M to -0.4543 at ESM2-650M, with 0.7857 candidate-variant repair rate and 6/7 positive-gap genes flipping to nonpositive. A paired robustness audit then tightens that result: across 13 gene-matched pairs, mean gap reduction is 0.7250 for Frobenius covariance and 0.3632 for pair covariance with exact sign-flip p = 0.000244, whereas the likelihood branch shows no corresponding repair (-0.0790, p = 0.6960). A stricter same-position sister-substitution follow-up keeps the effect alive under a harder local control: across 8 sister pairs spanning 5 genes, mean gap reduction remains 0.6259 for Frobenius covariance and 0.2492 for pair covariance with exact sign-flip p = 0.007812, while likelihood again shows no corresponding repair (-0.2112, p = 0.4922).

The rest of the evidence gives those claims their proper weight. TP53 shows that the underlying covariance effect is real and not a same-benchmark tuning accident. The support-ranked top-25 panel shows that the paper is not built from favorable hand-selection and that breadth is heterogeneous rather than universally positive. MSH2 shows that the same stronger-baseline recipe can fail decisively on another high-support gene. The protocol sweep and BRCA1 analysis show that the phenomenon is checkpoint-, window-, layer-, and gene-sensitive rather than a trivial monotone scaling law.

The appropriate final interpretation is therefore a bounded representational one. Covariance-aware hidden-state geometry contains real zero-shot pathogenicity signal. BRCA2 shows that the signal can improve a stronger baseline. The scale-repair audit, now reinforced by a paired robustness analysis and a stricter same-position sister-substitution follow-up, shows that the same signal can expose a structured regime in which the smaller backbone misweights perturbation and the stronger backbone largely repairs it in covariance space rather than as a generic stronger-model correction. In that sense, the paper's qualitative result is not only that covariance can raise a score on one gene, but that it can expose a concrete, bounded representational blind spot inside a widely used protein-model family. TP53 shows that the signal is publicly auditable. The support-ranked panel together with MSH2, BRCA1, and the protocol sweep shows where the current method generalizes and where it does not. The benchmark artifact makes that claim directly challengeable and gives BRCA2 a clear path to becoming the next canonical surface.

References

1. Lin, Z., Akin, H., Rao, R., et al. Evolutionary-scale prediction of atomic-level protein structure with a language model. Science 379(6637), 1123-1130 (2023). https://www.science.org/doi/10.1126/science.ade2574
2. Meier, J., Rao, R., Verkuil, R., Liu, J., Sercu, T., and Rives, A. Language models enable zero-shot prediction of the effects of mutations on protein function. NeurIPS 34 (2021). https://proceedings.neurips.cc/paper/2021/hash/f51338d736f95dd42427296047067694-Abstract.html
3. Frazer, J., Notin, P., Dias, M., et al. Disease variant prediction with deep generative models of evolutionary data. Nature 599, 91-95 (2021). https://www.nature.com/articles/s41586-021-04043-8
4. Ng, P. C., and Henikoff, S. SIFT: predicting amino acid changes that affect protein function. Nucleic Acids Research 31(13), 3812-3814 (2003). https://pubmed.ncbi.nlm.nih.gov/12824425/
5. Adzhubei, I. A., Schmidt, S., Peshkin, L., et al. A method and server for predicting damaging missense mutations. Nature Methods 7, 248-249 (2010). https://www.nature.com/articles/nmeth0410-248
6. Elnaggar, A., Heinzinger, M., Dallago, C., et al. ProtTrans: Toward Understanding the Language of Life Through Self-Supervised Learning. IEEE Transactions on Pattern Analysis and Machine Intelligence 44(10), 7112-7127 (2022). https://pubmed.ncbi.nlm.nih.gov/34232869/
7. Landrum, M. J., Lee, J. M., Benson, M., et al. ClinVar: improving access to variant interpretations and supporting evidence. Nucleic Acids Research 46(D1), D1062-D1067 (2018). https://academic.oup.com/nar/article-abstract/46/D1/D1062/4641904
8. Notin, P., Dias, M., Frazer, J., et al. ProteinGym: Large-scale benchmarks for protein fitness prediction and design. NeurIPS 36 (2023). https://proceedings.neurips.cc/paper_files/paper/2023/hash/cac723e5ff29f65e3fcbb0739ae91bee-Abstract-Datasets_and_Benchmarks.html
9. Rives, A., Meier, J., Sercu, T., et al. Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences. Proceedings of the National Academy of Sciences 118(15), e2016239118 (2021). https://pubmed.ncbi.nlm.nih.gov/33876751/
10. Brandes, N., Ofer, D., Peleg, Y., et al. ProteinBERT: a universal deep-learning model of protein sequence and function. Bioinformatics 38(8), 2102-2110 (2022). https://pubmed.ncbi.nlm.nih.gov/35020807/
11. Riesselman, A. J., Ingraham, J. B., and Marks, D. S. Deep generative models of genetic variation capture the effects of mutations. Nature Methods 15, 816-822 (2018). https://www.nature.com/articles/s41592-018-0138-4
12. Livesey, B. J., and Marsh, J. A. Using deep mutational scanning to benchmark variant effect predictors and identify disease mutations. Molecular Systems Biology 16(7), e9380 (2020). https://www.embopress.org/doi/10.15252/msb.20199380
13. Kotler, E., Shani, O., Goldfeld, G., et al. A systematic p53 mutation library links differential functional impact to cancer mutation pattern and evolutionary conservation. Molecular Cell 71(1), 178-190.e8 (2018). https://pubmed.ncbi.nlm.nih.gov/29979965/
14. Findlay, G. M., Daza, R. M., Martin, B., et al. Accurate classification of BRCA1 variants with saturation genome editing. Nature 562, 217-222 (2018). https://www.nature.com/articles/s41586-018-0461-z
15. Morcos, F., Pagnani, A., Lunt, B., et al. Direct-coupling analysis of residue coevolution captures native contacts across many protein families. Proceedings of the National Academy of Sciences 108(49), E1293-E1301 (2011). https://pmc.ncbi.nlm.nih.gov/articles/PMC3241805/
16. Katsonis, P., and Lichtarge, O. A formal perturbation equation between genotype and phenotype determines the Evolutionary Action of protein-coding variations on fitness. Genome Research 24(12), 2050-2058 (2014). https://pubmed.ncbi.nlm.nih.gov/25217195/
17. Jumper, J., Evans, R., Pritzel, A., et al. Highly accurate protein structure prediction with AlphaFold. Nature 596, 583-589 (2021). https://www.nature.com/articles/s41586-021-03819-2
18. Rao, R., Meier, J., Sercu, T., Ovchinnikov, S., and Rives, A. Transformer protein language models are unsupervised structure learners. ICLR (2021). https://openreview.net/forum?id=fylclEqgvgd
19. Hsu, C., Verkuil, R., Liu, J., Lin, Z., Hie, B., Sercu, T., Lerer, A., and Rives, A. Learning inverse folding from millions of predicted structures. Proceedings of Machine Learning Research 162, 8946-8970 (2022). https://proceedings.mlr.press/v162/hsu22a.html
20. Ghorbani, A., and Zou, J. Neuron Shapley: Discovering the Responsible Neurons. Advances in Neural Information Processing Systems 33 (2020). https://proceedings.neurips.cc/paper_files/paper/2020/file/41c542dfe6e4fc3deb251d64cf6ed2e4-Paper.pdf

Artifact Links

The BRCA2 augmentation and canonicalization analyses document the manuscript's flagship scientific result and its benchmark-extension path, while the TP53 summary and verification files remain the only frozen public canonical replay surface for the underlying covariance claim.