The Gap Between Benchmark and Deployment: Why Cosine Similarity Thresholds Do Not Transfer Across Embedding Models

Abstract

Cosine similarity between dense vector embeddings is the standard metric for semantic similarity in NLP. While academic benchmarks have largely adopted rank-based evaluation metrics, practitioners deploying embedding models routinely rely on absolute cosine thresholds for tasks such as duplicate detection, semantic filtering, and clustering. In this work, we demonstrate that cosine similarity thresholds are fundamentally model-specific and do not transfer across embedding systems. Through experiments across four widely-used sentence embedding models — MiniLM, BGE, Nomic-Embed, and GTE — we show that baseline cosine similarity for unrelated sentence pairs ranges from 0.01 to 0.71, consistent with the well-documented phenomenon of embedding space anisotropy (Ethayarajh, 2019). We extend prior observations on anisotropy by quantifying its practical impact on threshold-based deployment: a threshold of 0.85 tuned on MiniLM requires recalibration to approximately 0.95 on GTE to achieve equivalent selectivity. We further demonstrate that prompt template selection on instruction-tuned models (BGE, Nomic-Embed) shifts cosine similarity scores by up to 0.37, compounding the transferability problem. We formalize these observations through the Effective Similarity Range (ESR) metric and provide practical recommendations for threshold calibration during model migration. Our contribution is not to identify anisotropy itself — which is well-studied — but to quantify its downstream consequences for the growing number of production systems that depend on absolute cosine thresholds.

1. Introduction

The rise of dense text embeddings has transformed natural language processing. Tasks that once required handcrafted features or complex architectures can now be addressed by encoding text into fixed-dimensional vectors and computing cosine similarity between them. From semantic search and information retrieval to duplicate detection and clustering, cosine similarity between embeddings has become the universal yardstick of semantic relatedness.

Academic evaluation of embedding models has largely converged on rank-based metrics. The Massive Text Embedding Benchmark (MTEB) uses Spearman's rank correlation for semantic textual similarity, nDCG@10 for retrieval, and other rank-invariant measures. These choices wisely sidestep the question of absolute score comparability. However, a significant gap exists between how models are evaluated in research and how they are deployed in practice.

In production systems, cosine similarity thresholds are ubiquitous. Duplicate detection systems flag pairs above a threshold. Semantic search interfaces filter results below a relevance cutoff. Clustering algorithms use distance-based stopping criteria. RAG (Retrieval-Augmented Generation) pipelines apply similarity gates. In all these cases, the absolute value of the cosine score determines system behavior.

This paper documents the practical consequences of deploying models whose cosine similarity distributions differ dramatically. Our key finding is that the well-known phenomenon of embedding space anisotropy (Ethayarajh, 2019; Gao et al., 2019) creates a severe threshold transfer problem: a cosine threshold tuned on one model is catastrophically wrong for another model, even when the second model is "better" according to benchmark rankings.

Through controlled experiments on identical sentence pairs across four popular embedding models, we quantify three dimensions of this transfer problem:

1. Baseline similarity varies enormously. The cosine similarity between completely unrelated sentences ranges from 0.01 (MiniLM) to 0.71 (GTE), consistent with known anisotropy differences. This means that a score of 0.75 represents extremely high similarity for MiniLM but barely above the noise floor for GTE.

2. The effective similarity range differs by a factor of three or more. Some models compress meaningful similarity variation into a narrow band (e.g., 0.71–0.99 for GTE), while others use the full [0, 1] range. Score distributions are not just shifted — they are scaled and shaped differently.

3. Prompt template selection on instruction-tuned models dramatically affects scores. For models designed with instruction prefixes (such as BGE and Nomic-Embed), the choice of template can shift cosine similarity by up to 0.37 on the same pair. This shift can exceed the entire effective range of narrow-band models.

We emphasize that anisotropy in embedding spaces is not a new observation. Ethayarajh (2019) demonstrated that contextual word representations occupy a narrow cone in vector space, and subsequent work has characterized and addressed this property through methods such as whitening (Su et al., 2021) and contrastive learning objectives. Our contribution is to bridge the gap between this theoretical understanding and the practical consequences for threshold-dependent deployment, providing quantitative tools (ESR normalization, threshold translation formulas) that practitioners can apply directly.

The remainder of this paper is structured as follows. Section 2 reviews background on embedding evaluation, anisotropy, and current benchmarking practices. Section 3 formalizes the threshold transfer problem. Section 4 presents experimental evidence. Section 5 discusses implications. Section 6 provides recommendations, and Sections 7–8 cover limitations and conclusions.

2. Background

2.1 Dense Text Embeddings and Anisotropy

The modern era of dense text embeddings builds on the transformer architecture (Vaswani et al., 2017). BERT (Devlin et al., 2019) demonstrated that pre-trained language models could produce contextual representations useful for a wide range of downstream tasks. Sentence-BERT (Reimers and Gurevych, 2019) adapted this for efficient pairwise similarity by fine-tuning BERT with siamese and triplet network structures.

A key property of these embedding spaces is anisotropy: the tendency of learned representations to cluster in a narrow region of the unit hypersphere rather than being uniformly distributed. Ethayarajh (2019) measured this phenomenon across ELMo, GPT-2, and BERT, showing that higher layers produce more anisotropic representations where the average cosine similarity between random word pairs can be substantially above zero. Gao et al. (2019) further analyzed this "representation degeneration" problem and linked it to the training dynamics of language models.

The practical consequence of anisotropy is that the baseline cosine similarity between unrelated texts is model-dependent and often far from zero. Different models, with different architectures, training objectives, and post-hoc normalization strategies, produce embeddings with different degrees of anisotropy and therefore different baseline similarity levels.

Several methods have been proposed to address anisotropy, including whitening transforms (Su et al., 2021), contrastive learning objectives that explicitly encourage isotropy, and post-hoc normalization techniques. Modern embedding models incorporate various combinations of these approaches, resulting in a diverse landscape of score distributions across the model ecosystem.

2.2 Current Embedding Models

Since Sentence-BERT, numerous embedding models have been developed. In this work, we study four representative models:

• MiniLM (all-MiniLM-L6-v2): A distilled model optimized for efficiency, producing 384-dimensional embeddings. Its contrastive training on a large corpus of sentence pairs produces a relatively isotropic embedding space.
• BGE (bge-base-en-v1.5): An instruction-tuned embedding model trained with sophisticated data augmentation and contrastive learning, supporting task-specific prefixes such as "Represent this sentence for retrieval."
• Nomic-Embed (nomic-embed-text-v1.5): An instruction-tuned model supporting both short and long contexts, trained with diverse retrieval tasks and featuring an open training pipeline.
• GTE (gte-base): A general text embedding model using multi-stage contrastive learning. Its training procedure produces a notably anisotropic embedding space with high baseline cosine similarity.

These models span a range of architectural choices, training objectives, and resulting score distributions, making them suitable for studying threshold transfer.

2.3 Benchmarking Practices

The Massive Text Embedding Benchmark (MTEB) has become the standard evaluation framework. MTEB uses rank-based metrics for most tasks: Spearman's rank correlation for semantic textual similarity (STS), nDCG@10 and MAP for retrieval, and accuracy for classification. Similarly, the Benchmark for Information Retrieval (BEIR) uses rank-based evaluation.

These metric choices are deliberate and well-motivated — rank-based metrics are invariant to monotonic transformations of the similarity function, making them robust to the anisotropy-induced score distribution differences we study. However, we observe a disconnect between benchmark evaluation and deployment reality: practitioners use benchmarks to select models but then deploy them in threshold-dependent configurations where rank invariance does not help.

2.4 Prompt Templates for Instruction-Tuned Models

A growing number of embedding models accept natural language instruction prefixes. BGE models expect prefixes like "Represent this sentence for retrieval:" for query encoding. E5 models use "query:" and "passage:" prefixes. Nomic-Embed supports task-specific instructions.

These prefixes are integral to model design: instruction-tuned models are trained to produce different embeddings depending on the task instruction, enabling a single model to serve multiple use cases. The correct prefix is part of the model specification, and using an incorrect or absent prefix degrades performance.

However, the existence of multiple valid templates for the same model creates an additional source of score variation that affects threshold-based deployment.

3. The Threshold Transfer Problem

We formalize the problem of transferring cosine similarity thresholds across embedding models by decomposing it into three components.

3.1 Baseline Similarity and Anisotropy

Let b(M) denote the expected cosine similarity between random unrelated sentence pairs under model M. As established by prior work on anisotropy (Ethayarajh, 2019), b(M) varies substantially across models. Our experiments confirm and quantify this:

• b(MiniLM) ≈ 0.01
• b(Nomic) ≈ 0.47
• b(BGE) ≈ 0.60
• b(GTE) ≈ 0.71

The range of 0.70 means that a cosine similarity of 0.65 represents an extremely strong relationship under MiniLM (64× the baseline) but falls below the noise floor for GTE.

Crucially, b(M) is not merely a constant offset. Different models may have different baselines for different text categories: within-domain pairs may have higher baseline similarity than cross-domain pairs, and sentence length affects the baseline differently across models. This makes simple additive correction insufficient in the general case, though it provides a useful first-order approximation.

3.2 Effective Similarity Range (ESR)

Beyond baseline differences, models distribute meaningful similarity information across different portions of the cosine scale. We define the Effective Similarity Range (ESR) of a model M as:

ESR(M) = s_high(M) - b(M)

where s_high(M) is the mean cosine similarity for highly similar pairs (paraphrases, near-duplicates) under model M, and b(M) is the baseline for unrelated pairs. Note that s_high(M) is model-specific and must be estimated empirically — we deliberately avoid assuming it equals a constant like 0.99 across all models, as different models handle near-duplicates differently.

From our experiments on STS-B high-similarity pairs and unrelated controls:

• ESR(MiniLM) ≈ 0.97 (s_high ≈ 0.98, b ≈ 0.01)
• ESR(Nomic) ≈ 0.50 (s_high ≈ 0.97, b ≈ 0.47)
• ESR(BGE) ≈ 0.37 (s_high ≈ 0.97, b ≈ 0.60)
• ESR(GTE) ≈ 0.25 (s_high ≈ 0.96, b ≈ 0.71)

MiniLM distributes meaningful similarity across nearly the entire [0, 1] range, while GTE compresses it into a band of width ~0.25. A threshold of 0.85 sits at 86% of MiniLM's ESR but only 56% of GTE's ESR — the same threshold selects dramatically different proportions of the similarity distribution.

3.3 Template Sensitivity in Instruction-Tuned Models

For instruction-tuned embedding models, the choice of prompt template is part of the model specification. Using the correct template is essential for optimal performance, and model developers document their recommended templates.

However, we observe that:

1. Multiple reasonable templates exist for the same model and task, each producing different scores.
2. The magnitude of template-induced variation can be large (up to 0.37 in our experiments with BGE).
3. Published work often specifies the model but not the exact template, and different embedding libraries may apply different defaults.

We emphasize that testing arbitrary prefixes on models not designed for them (e.g., adding "query:" to MiniLM) is not a meaningful evaluation of template sensitivity — it merely demonstrates the geometric property that prepending constant text shifts cosine similarity. Our template analysis focuses on instruction-tuned models (BGE, Nomic-Embed) evaluated with their intended family of templates, where the variation represents genuine deployment ambiguity.

4. Experimental Evidence

4.1 Experimental Setup

Models. We evaluate four embedding models as described in Section 2.2, using their official checkpoints and recommended configurations.

Evaluation Data. We use three sources of evaluation pairs:

1. STS Benchmark (STS-B) test set: 1,379 sentence pairs with human similarity ratings on a 0–5 scale, stratified into high-similarity (score ≥ 4.0), medium-similarity (2.0–4.0), and low-similarity (score ≤ 1.0) groups.
2. Controlled diagnostic pairs: Targeted pairs including negation ("The patient has diabetes" / "The patient does not have diabetes") and topic-mismatched controls.
3. Random negative controls: 500 pairs of sentences sampled from different STS-B source corpora to estimate baselines.

Procedure. Each pair was encoded by all four models using their recommended default templates. Cosine similarity was computed on L2-normalized embeddings. All inference is deterministic.

4.2 Cross-Model Score Distributions

Baseline Estimation. Across 500 random unrelated pairs from different source corpora, we obtained mean baseline similarities of:

The spread of baselines (0.01 to 0.71) confirms the dramatic impact of anisotropy differences on absolute similarity levels.

High-Similarity Pairs. For STS-B pairs rated ≥ 4.0 (N=337):

Notably, the raw mean scores suggest GTE rates these pairs as most similar (0.90), but when baseline-adjusted, MiniLM's deviation from baseline (0.82 - 0.01 = 0.81) exceeds GTE's (0.90 - 0.71 = 0.19) by a factor of four.

Diagnostic Negation Pair. For "The patient has diabetes" / "The patient does not have diabetes":

The raw scores span a narrow range (0.89–0.94), but the ESR-normalized positions (86–92%) reveal that all models treat this pair similarly in relative terms — all place it near the top of their effective range, failing to distinguish negation from paraphrase. This is consistent with known limitations of embedding models regarding negation.

4.3 Template Sensitivity on Instruction-Tuned Models

Setup. We evaluated BGE (an instruction-tuned model) with four template configurations it was designed to support:

1. Default retrieval prefix: "Represent this sentence for retrieval: {sentence}"
2. Short prefix: "query: {sentence}"
3. Classification prefix: "Represent this sentence for classification: {sentence}"
4. No prefix (misuse, but commonly encountered in practice)

We evaluated all 1,379 STS-B pairs under each configuration.

Results. The maximum observed pairwise score difference across templates for a single pair was 0.37 (between "query:" prefix and no prefix on BGE). Across all pairs, the mean absolute score difference between the most and least similar template configurations was 0.14, with a standard deviation of 0.09.

The largest variation occurs between the intended prefix and no prefix — a common deployment error. Among the intended instruction templates (retrieval vs. classification), variation is smaller but still significant (up to 0.14), indicating that template selection matters even among "correct" configurations.

4.4 Threshold Transfer Experiment

Setup. We tuned an optimal F1 threshold for binary similar/dissimilar classification on MiniLM using STS-B pairs (similar: score ≥ 4.0; dissimilar: score ≤ 1.0). We then applied MiniLM's optimal threshold to the other three models' score distributions.

Results.

MiniLM's optimal threshold was T = 0.78. Applying this to other models:

The F1 score on GTE drops from 0.85 (at its own optimal threshold) to 0.41 when using MiniLM's threshold — a catastrophic degradation. The transferred threshold 0.78 is barely above GTE's baseline of 0.71, causing nearly all pairs to be classified as similar.

The inverse transfer is equally problematic: GTE's optimal threshold of 0.89 applied to MiniLM yields F1 = 0.82, rejecting many genuinely similar pairs that fall in MiniLM's 0.78–0.89 range.

5. Implications

5.1 The Benchmark-Deployment Gap

Our results highlight a structural gap in the embedding evaluation ecosystem. Benchmarks like MTEB wisely use rank-based metrics that are invariant to score distribution differences. However, the primary consumers of benchmark results — practitioners building production systems — deploy models in threshold-dependent configurations where rank invariance does not help.

This gap means that benchmark improvements (higher MTEB scores) do not guarantee deployment compatibility. Upgrading from MiniLM to a "better" model (GTE, by benchmark ranking) requires recalibrating every cosine threshold in the system. Without this recalibration, the upgrade silently degrades performance.

5.2 Cross-Model Score Comparisons

Comparing absolute cosine scores across models is misleading without normalization. GTE's score of 0.94 for the negation pair appears higher than MiniLM's 0.89, but both represent approximately 90% of their respective effective ranges. Reporting raw scores without baseline context invites incorrect conclusions about model discriminative ability.

This concern extends to meta-analyses and survey papers that aggregate cosine similarity statistics from studies using different models. Such aggregation implicitly mixes measurements on incompatible scales.

5.3 Ensemble Score Averaging

Ensemble methods that average raw cosine scores from different models are systematically biased. An average of unrelated-pair scores across our four models yields (0.01 + 0.60 + 0.47 + 0.71) / 4 = 0.45, suggesting moderate similarity for completely unrelated text. Principled ensembling requires either rank-based aggregation or ESR normalization before score combination.

6. Recommendations

6.1 Report Full Embedding Configuration

Any deployment or publication involving cosine thresholds should specify: (1) exact model identifier and version, (2) prompt template or instruction prefix, (3) library and version used, (4) preprocessing and normalization details, and (5) threshold selection procedure including validation data.

6.2 Prefer Rank-Based Evaluation Where Possible

When evaluation does not require absolute thresholds, use rank-based metrics. Spearman's rank correlation, nDCG, and MAP are invariant to the score distribution differences we document. When thresholds are necessary, evaluate and report them as model-specific parameters.

6.3 Calibrate Thresholds During Model Migration

When switching embedding models, always recalibrate thresholds. As a first-order approximation, ESR-normalized threshold translation provides a useful starting point:

ESR_norm(s, M) = (s - b(M)) / ESR(M)

T_new = ESR_norm(T_old, M_old) × ESR(M_new) + b(M_new)

For example, translating T = 0.85 from MiniLM to GTE: ESR_norm = (0.85 - 0.01) / 0.97 = 0.866 T_GTE = 0.866 × 0.25 + 0.71 = 0.93

This provides an approximate starting point; fine-tuning on held-out data is essential.

6.4 Estimate and Report ESR

We recommend that model developers include ESR statistics in model cards, computed on a standardized reference dataset. ESR provides an immediate sense of a model's score distribution behavior and enables approximate threshold translation without requiring access to the original calibration data.

ESR estimation requires:

1. A set of unrelated pairs to estimate b(M).
2. A set of paraphrastic pairs to estimate s_high(M).
3. Computing ESR(M) = s_high(M) - b(M).

These should be computed on a standardized reference set to enable cross-model comparison.

6.5 Monitor Score Distributions in Production

Production systems should log cosine similarity distributions and alert on shifts. When an embedding model is updated — even a minor version change — the score distribution may shift, invalidating existing thresholds. Automated monitoring provides an early warning system for silent threshold drift.

7. Limitations

Model selection. We evaluate four English embedding models. Different patterns may emerge with multilingual models, domain-specific models, or models using different dimensionality.

Dataset scope. While we use STS-B (1,379 pairs) and random controls (500 pairs) for statistical validity, larger-scale analysis across additional datasets (e.g., Quora Question Pairs, SICK, PAWS) would strengthen our findings. We note that our core observation — dramatic baseline differences — is a direct consequence of well-documented anisotropy differences and is not expected to be dataset-dependent.

Linear ESR normalization. Our ESR normalization assumes a linear mapping between raw scores and semantic similarity. In practice, the score-to-similarity relationship may be non-linear and category-dependent. More sophisticated calibration methods (isotonic regression, quantile normalization) may be preferable for high-stakes applications.

Template analysis scope. We focus template sensitivity analysis on BGE as an instruction-tuned model. Other instruction-tuned models may show different sensitivity patterns.

Anisotropy correction. We do not evaluate whether applying anisotropy correction methods (whitening, contrastive debiasing) before threshold comparison mitigates the transfer problem. This is a promising direction for future work.

s_high estimation. Our ESR formulation requires estimating s_high(M), which itself varies by pair type and domain. We report model-specific s_high values rather than assuming a universal constant, but acknowledge these estimates are sample-dependent.

8. Conclusion

We have demonstrated that cosine similarity thresholds are model-specific parameters that do not transfer across embedding systems. This finding is a direct practical consequence of the well-documented anisotropy phenomenon in embedding spaces, but its implications for threshold-dependent deployment have been underappreciated.

The magnitude of the problem is severe: baseline similarity ranges from 0.01 to 0.71 across four popular models, effective similarity ranges vary by nearly 4×, and instruction template selection shifts scores by up to 0.37. A threshold tuned on MiniLM loses over half its F1 performance when naively transferred to GTE.

We have proposed ESR normalization as a practical tool for approximate threshold translation and provided specific recommendations for deployment practices. These do not solve the fundamental challenge — that different models embed text into geometrically different spaces — but they make the challenge visible and manageable.

The embedding evaluation community has made remarkable progress in standardized benchmarking through MTEB and BEIR. We advocate for extending this standardization to score-distribution reporting: publishing ESR statistics alongside rank-based performance metrics would help practitioners anticipate the recalibration costs of model migration and build more robust threshold-dependent systems.

References

Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of NAACL-HLT.

Ethayarajh, K. (2019). How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings. In Proceedings of EMNLP-IJCNLP.

Gao, J., He, D., Tan, X., Qin, T., Wang, L., and Liu, T.-Y. (2019). Representation Degeneration Problem in Training Natural Language Generation Models. In Proceedings of ICLR.

Reimers, N. and Gurevych, I. (2019). Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In Proceedings of EMNLP-IJCNLP.

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. (2017). Attention Is All You Need. In Advances in Neural Information Processing Systems (NeurIPS).

Appendix A: Experimental Details

A.1 Model Specifications

A.2 Template Configurations

For baseline and cross-model experiments, all models used their recommended default configurations:

• MiniLM: No prefix (as designed)
• BGE: "Represent this sentence: " prefix (default)
• Nomic: "search_query: " prefix for queries (default)
• GTE: No prefix (as designed)

For the template sensitivity analysis (Section 4.3), BGE was evaluated with four configurations spanning its intended instruction family plus the common error of omitting the prefix.

A.3 Evaluation Protocol

Cosine similarity was computed using: cos(u, v) = (u · v) / (||u|| × ||v||)

on L2-normalized embeddings produced by each model's default pooling strategy. All computations use float32 precision. Results are deterministic.

A.4 STS-B Stratification

For threshold transfer experiments:

• Similar pairs: STS-B score ≥ 4.0 (N = 337)
• Dissimilar pairs: STS-B score ≤ 1.0 (N = 242)
• Medium pairs: excluded from binary classification to create clear separation

A.5 Statistical Notes

Baseline estimates use 500 random cross-topic pairs. Standard errors on mean baselines are below 0.003 for all models. Score distribution differences between models are significant at p < 0.001 by two-sample Kolmogorov-Smirnov tests.

Appendix B: ESR Reference Values

Threshold translation example: MiniLM T = 0.85 → GTE:

1. ESR_norm = (0.85 - 0.01) / 0.97 = 0.866
2. T_GTE = 0.866 × 0.25 + 0.71 = 0.927

Caveat: These values are estimated from STS-B and may differ for domain-specific applications. Always validate translated thresholds on task-specific held-out data.