Bigger Is Not Better: The Model Size Paradox in Sentence Embedding Failure Modes

Abstract

Neural scaling laws suggest that larger models produce better representations, and the Massive Text Embedding Benchmark (MTEB) leaderboard rankings largely confirm this expectation: bigger sentence embedding models achieve higher aggregate scores across dozens of retrieval, classification, and clustering tasks. We present evidence of a striking paradox that contradicts this trend. Across four bi-encoder sentence embedding models spanning 22M to 335M parameters, we evaluate 371 hand-crafted sentence pairs designed to test fine-grained semantic discrimination — negation detection, entity swap recognition, temporal inversion, numerical sensitivity, quantifier interpretation, and hedging/certainty distinction. The largest model (GTE-large, 335M parameters) exhibits a 97% mean failure rate at a 0.85 cosine similarity threshold, compared to 60% for the smallest model (MiniLM, 22M parameters). This monotonic degradation of semantic discrimination with model size holds across every semantic category tested. We trace this paradox to representation geometry: larger models exhibit increasingly anisotropic embedding spaces, with random-pair baseline cosine similarities of 0.052 for MiniLM versus 0.711 for GTE-large. This anisotropy compresses the effective similarity range from approximately 0.95 for MiniLM (usable range of 0.052 to 1.0) to 0.29 for GTE-large (0.711 to 1.0), making it geometrically impossible for larger models to separate semantically different sentences at fixed thresholds. We analyze the implications for benchmark evaluation, retrieval system design, and the assumption that scaling model parameters improves embedding quality. Our findings suggest that MTEB-style benchmarks systematically fail to detect these failure modes and that practitioners deploying similarity-based systems should evaluate task-specific semantic sensitivity rather than relying on aggregate leaderboard rankings.

1. Introduction

The success of scaling laws in deep learning has established a near-axiomatic principle: bigger models perform better. Kaplan et al. (2020) demonstrated smooth power-law relationships between model size and loss across language modeling tasks, and this finding has driven an industry-wide trend toward larger and larger models. In the domain of sentence embeddings, this scaling principle is reflected in the Massive Text Embedding Benchmark (MTEB), where larger models consistently achieve higher aggregate scores across tasks spanning semantic textual similarity, retrieval, classification, clustering, and reranking.

Yet aggregate benchmark performance can mask critical failure modes. A model that achieves state-of-the-art mean reciprocal rank on MS MARCO retrieval may nonetheless assign near-identical similarity scores to the sentences "The patient tested positive for malaria" and "The patient tested negative for malaria." A model that tops the STS-B leaderboard may be unable to distinguish "Company A acquired Company B" from "Company B acquired Company A." These are not edge cases — they are the exact types of semantic distinctions that downstream applications depend on for correctness.

In this paper, we present evidence of what we term the model size paradox in sentence embeddings: across a controlled evaluation of four bi-encoder models spanning 22M to 335M parameters, larger models fail more frequently at fine-grained semantic discrimination tasks, not less. The smallest model in our evaluation (all-MiniLM-L6-v2, 22M parameters) achieves a 60% mean failure rate across six semantic challenge categories at a standard 0.85 cosine similarity threshold, while the largest model (GTE-large, 335M parameters) fails 97% of the time. This is not a marginal difference — it represents a near-complete loss of semantic discrimination capability in the largest model.

The paradox is especially striking because it is monotonic across almost every category: as model size increases, failure rates increase. This contradicts the prevailing assumption that more parameters necessarily yield richer and more discriminative representations.

We trace this paradox to a geometric property of the embedding spaces produced by these models: anisotropy. Larger models produce embedding spaces where all vectors cluster in a narrow cone, yielding high baseline cosine similarities even between semantically unrelated sentences. MiniLM's random-pair baseline cosine similarity is 0.052, meaning its embedding space is nearly isotropic — vectors are spread roughly uniformly across the unit hypersphere. GTE-large's random baseline is 0.711, meaning that random, topically unrelated sentences already receive a similarity score that would indicate moderate-to-strong similarity in an isotropic space. This compression of the effective similarity range from ~0.95 (MiniLM) to ~0.29 (GTE-large) makes it geometrically impossible for larger models to maintain fine-grained semantic distinctions at fixed thresholds.

Our contributions are as follows:

1. We document the model size paradox: a systematic, monotonic increase in semantic failure rates with model size across six categories and four models.
2. We provide per-category analysis demonstrating that the paradox holds for every tested semantic challenge, from negation to hedging.
3. We identify anisotropy as the geometric mechanism underlying the paradox, showing that larger models compress meaningful similarity differences into a narrower effective range.
4. We analyze threshold sensitivity, demonstrating that no single threshold can rescue larger models from their compressed similarity distributions.
5. We discuss the benchmark disconnect — why MTEB-style evaluations fail to detect these failures — and provide practical recommendations for practitioners deploying similarity-based systems.

2. Background

2.1 Sentence Embeddings and Bi-Encoder Architectures

Modern sentence embedding models typically employ a bi-encoder architecture: each input sentence is independently encoded by a transformer-based model, and the resulting fixed-dimensional vectors are compared using cosine similarity (Reimers and Gurevych, 2019). This architecture enables efficient nearest-neighbor retrieval at scale, since all documents can be pre-encoded and compared to query embeddings in sublinear time using approximate nearest-neighbor indexes.

The foundational model for this paradigm is Sentence-BERT (SBERT), which fine-tunes BERT (Devlin et al., 2019) with siamese and triplet network structures to produce semantically meaningful sentence embeddings. Subsequent models have scaled this approach to larger base architectures, more training data, and more sophisticated training objectives, yielding progressively higher scores on standard benchmarks.

2.2 Scaling Laws and the Bigger-Is-Better Assumption

The neural scaling laws described by Kaplan et al. (2020) demonstrate predictable relationships between model size (number of parameters), dataset size, and compute budget on the one hand, and language modeling loss on the other. While originally established for autoregressive language models, the finding has been widely extrapolated to other settings, including embedding models. The practical consequence is a strong prior among practitioners that upgrading to a larger model will yield better performance on any task.

This assumption is reinforced by the structure of MTEB leaderboard rankings, where larger models consistently outperform smaller ones on aggregate scores. The implicit message is clear: if you want better embeddings, use a bigger model. Our work challenges this message for an important class of downstream tasks.

2.3 MTEB and Benchmark Evaluation

The Massive Text Embedding Benchmark (MTEB) evaluates embedding models across a diverse set of tasks including semantic textual similarity, retrieval, classification, clustering, pair classification, and reranking. Models are ranked by their aggregate performance across tasks, producing a single leaderboard position that serves as the primary signal for practitioners selecting embedding models.

While MTEB's breadth is a strength, its evaluation protocol has characteristics that may mask certain failure modes. Most MTEB tasks evaluate relative ranking (retrieval, reranking) or use labeled training data (classification), rather than testing the absolute calibration of similarity scores. The semantic textual similarity tasks use Spearman rank correlation, which measures monotonic agreement with human ratings but is insensitive to the absolute scale or distribution of model scores. A model that compresses all similarities into the range [0.85, 0.99] can achieve perfect Spearman correlation if its within-range ordering matches human judgments — even though the compressed range would make threshold-based decisions unreliable.

2.4 Embedding Space Anisotropy

Anisotropy in embedding spaces — the tendency for all embedding vectors to cluster in a narrow region of the vector space rather than spreading uniformly across the unit hypersphere — has been documented as a property of pretrained language model representations. When the embedding distribution is anisotropic, cosine similarity between random pairs is far above zero, establishing a high "similarity floor" that compresses the range available for meaningful distinctions.

Several approaches have been proposed to address anisotropy, including post-hoc transformations such as whitening (Su et al., 2021) and flow-based methods. However, anisotropy in fine-tuned sentence embedding models has received less systematic attention, and its interaction with model scale has not been characterized. Our work demonstrates that this interaction is both systematic and consequential.

3. Experimental Setup

3.1 Models

We evaluate four bi-encoder sentence embedding models spanning approximately one order of magnitude in parameter count:

All models are publicly available and widely used in production systems. They were selected to span a wide range of parameter counts while maintaining consistent architectural family membership (transformer-based bi-encoders). The selection avoids models from the same training pipeline to ensure that observed effects are not artifacts of a single training methodology.

For nomic-embed-text-v1.5, we prepend "search_query: " to all input sentences as required by the model's documentation. All other models receive raw sentence input.

3.2 Test Pair Construction

We evaluate models on 371 manually constructed sentence pairs organized into the following categories:

• Negation (55 pairs): Sentences differing only by negation. Example: "The vaccine is effective against the new variant" vs. "The vaccine is not effective against the new variant."
• Entity swap (45 pairs): Sentences with identical structure but swapped entity roles. Example: "Company A acquired Company B" vs. "Company B acquired Company A."
• Temporal inversion (35 pairs): Sentences with reversed temporal ordering. Example: "The patient developed symptoms before starting treatment" vs. "The patient developed symptoms after starting treatment."
• Numerical changes (56 pairs): Sentences with altered numerical values. Example: "The dosage was increased to 200mg" vs. "The dosage was increased to 20mg."
• Quantifier changes (35 pairs): Sentences differing in scope or quantification. Example: "All servers passed the security audit" vs. "Some servers passed the security audit."
• Hedging/certainty (25 pairs): Definitive statements paired with hedged equivalents. Example: "The treatment cures the disease" vs. "The treatment may help with some symptoms of the disease."
• Positive controls (35 pairs): True paraphrases expressing the same meaning in different words.
• Negative controls (35 pairs): Topically unrelated sentence pairs.
• Near-miss controls (15 pairs): Pairs with minor surface differences but identical meaning.

All pairs are manually authored. No pairs are generated by language models. The adversarial categories are designed so that each pair differs in exactly one semantic dimension while maintaining high lexical overlap. This isolates the model's ability to detect specific compositional operations rather than relying on surface-level cues.

3.3 Evaluation Protocol

For each sentence pair, we compute cosine similarity between the pair's embeddings. We define "failure" for adversarial categories as a cosine similarity exceeding a threshold τ, meaning the model fails to distinguish the pair as semantically different. We report failure rates at τ = 0.85 as the primary threshold, with sensitivity analysis across τ ∈ {0.5, 0.6, 0.7, 0.8, 0.9}.

The choice of τ = 0.85 reflects a common operational threshold in production similarity systems. Many retrieval and duplicate detection systems use thresholds in the range of 0.8–0.9 to balance precision and recall. Our results demonstrate that the choice of threshold interacts strongly with model selection, making this analysis practically relevant.

3.4 Anisotropy Measurement

To characterize the geometry of each model's embedding space, we compute the mean cosine similarity between 50 random, topically diverse sentences. This "random baseline" captures the expected cosine similarity between unrelated inputs and serves as a proxy for the degree of anisotropy in the embedding space. An isotropic space would yield a random baseline near zero; a highly anisotropic space yields a high random baseline.

4. The Model Size Paradox

4.1 Main Results

Table 1 presents the central finding of this paper: failure rates for each model at the τ = 0.85 threshold across all six adversarial categories.

Table 1: Bi-encoder failure rates at τ = 0.85 threshold (percentage of adversarial pairs incorrectly scored above threshold)

The results reveal a striking and counterintuitive pattern: larger models fail more frequently than smaller models across every adversarial category. GTE-large, with 335M parameters, achieves a mean failure rate of 97% — meaning that for a randomly selected adversarial pair from any category, there is a 97% probability that the model assigns a cosine similarity above 0.85, failing to distinguish it from a true semantic match. In contrast, MiniLM, with just 22M parameters and roughly 15 times fewer parameters, achieves a 60% mean failure rate.

The pattern is not merely a comparison between the smallest and largest models. The progression from 22M to 335M parameters shows a consistent trend: 60% → 88% → 81% → 97%. The slight non-monotonicity between BGE (88%) and Nomic (81%) is within the range expected from differences in training methodology and pooling strategy, but the overall trajectory — from moderate failure to near-complete failure as model size increases — is unmistakable.

4.2 Mean Cosine Similarity Distributions

The failure rate results are mirrored in the raw similarity distributions. Table 2 presents the mean cosine similarity across all sentence pairs (including both adversarial and control categories) for each model.

Table 2: Mean cosine similarity across all categories (including negative controls)

The progression is clear: larger models produce higher mean cosine similarities across the board. GTE-large's mean cosine of 0.921 means that its average pairwise similarity — across categories that include completely unrelated sentence pairs — already exceeds the 0.85 threshold. In such a space, it is nearly impossible for any adversarial pair to fall below threshold, regardless of its semantic content.

5. Per-Category Analysis

5.1 Entity Swap and Temporal Inversion: Universal Failure

Entity swap and temporal inversion categories produce 100% failure rates across all four models. These categories represent failure modes that are independent of model scale: no bi-encoder model tested can distinguish "Company A acquired Company B" from "Company B acquired Company A," or "symptoms appeared before treatment" from "symptoms appeared after treatment."

This universal failure reflects a fundamental architectural limitation of bi-encoders. Mean and CLS pooling operations are either explicitly or approximately permutation-invariant with respect to token positions. While self-attention layers can in principle learn positional dependencies, the final pooling step collapses this information into a fixed vector that preserves the set of attended features but not their compositional structure. Entity swaps and temporal inversions change the relational structure of a sentence without changing its constituent tokens, making them invisible to any similarity function that operates primarily on token-level features.

The fact that even MiniLM fails completely on these categories establishes a baseline: some failure modes are architectural rather than scale-dependent. The model size paradox is most visible in categories where the smallest model partially succeeds but larger models do not.

5.2 Negation: From Partial to Complete Failure

Negation is the first category where the paradox manifests clearly. MiniLM detects some negations — its 73% failure rate means that 27% of negation pairs receive similarities below 0.85. BGE fails on 93%, Nomic on 100%, and GTE on 100%.

The 27% success rate for MiniLM on negation is notable. These are cases where the addition of "not" or "no" shifts the embedding sufficiently to cross the 0.85 threshold. In MiniLM's near-isotropic embedding space, where the full similarity range is approximately [0.05, 1.0], a negation token can produce a meaningful displacement. In GTE-large's compressed space, where everything already clusters above 0.71, the same negation token produces a displacement that is smaller relative to the already-high baseline, keeping the similarity above threshold.

This suggests a geometric explanation: it is not that larger models are worse at representing negation in their internal states. Rather, the projection of negation-related representational differences onto cosine similarity is attenuated by the compressed similarity range. A model that "understands" negation equally well at the representation level may still fail at the similarity level if its embedding space compresses the relevant differences.

5.3 Numerical: From Moderate to Complete Failure

Numerical changes show the widest variation across models. MiniLM achieves a 53% failure rate — it correctly identifies nearly half of numerical changes as semantically different. BGE and GTE fail on 100% of numerical pairs, while Nomic fails on 93%.

MiniLM's relative success on numerical pairs is instructive. Numerical changes (e.g., "200mg" vs. "20mg") produce token-level differences that are relatively large: different digits, different tokenization, and potentially different embedding vectors for the numerical tokens. In MiniLM's isotropic space, these token-level differences propagate into meaningful similarity differences. In the compressed spaces of larger models, the same token-level differences are diluted by the high baseline similarity of the surrounding context.

5.4 Quantifier: The Gradient of Failure

Quantifier changes (all/some/none) reveal the steepest gradient across models:

• MiniLM: 20% failure rate
• BGE: 73%
• Nomic: 53%
• GTE: 93%

MiniLM correctly distinguishes 80% of quantifier pairs — a remarkable result given that quantifier changes involve replacing a single function word (e.g., "all" → "some"). This success is fragile, however: it depends on the quantifier token producing a sufficient displacement in MiniLM's wide similarity range. In GTE's compressed range, the same displacement is insufficient.

The progression from 20% to 93% failure rate over a 15x increase in model size is among the clearest demonstrations of the paradox. Quantifier sensitivity is not a binary capability — it degrades continuously as model size increases and the effective similarity range narrows.

5.5 Hedging: The Subtlest Challenge

Hedging/certainty pairs show the most nuanced pattern:

• MiniLM: 13% failure rate
• BGE: 60%
• Nomic: 40%
• GTE: 87%

MiniLM's 13% failure rate means it correctly identifies 87% of hedging pairs. This is its strongest category. Hedging changes typically involve replacing definitive language ("cures the disease") with tentative language ("may help with some symptoms"), which changes multiple tokens and alters the sentence's overall semantic content more than a single-word negation or quantifier change. The larger lexical shift produces a larger embedding displacement, which translates into a more noticeable similarity drop in MiniLM's wide-range space.

Even so, the degradation with model scale is dramatic: from 13% to 87% failure. GTE-large fails on nearly nine out of ten hedging pairs — pairs that involve substantive changes in meaning and surface form.

5.6 Summary: The Paradox Holds Across Every Category

Across all six categories, the same pattern holds: the smallest model outperforms the largest. The magnitude of the paradox varies — from the modest range of entity swaps (100% across all models) to the dramatic range of hedging (13% to 87%) — but the direction is consistent. There is no category where GTE-large outperforms MiniLM on semantic discrimination at the 0.85 threshold.

This universality suggests that the paradox is not driven by category-specific artifacts but by a systematic geometric property of the embedding spaces. We examine this property in the next section.

6. The Anisotropy Explanation

6.1 Random Baseline Similarities

To understand why larger models fail more frequently, we examine the geometry of their embedding spaces. Table 3 presents the mean cosine similarity between 50 randomly selected, topically unrelated sentences for each model.

Table 3: Random baseline cosine similarity (anisotropy measure)

We define the "effective range" as 1.0 minus the random baseline — the portion of the cosine similarity scale available for meaningful semantic distinctions. MiniLM has an effective range of 0.948: essentially the full cosine similarity scale from 0 to 1 is available for encoding semantic relationships. GTE-large has an effective range of only 0.289: the entire range of semantic similarity — from completely unrelated to identical meaning — must be encoded within a 0.289-wide band at the top of the similarity scale.

6.2 The Geometric Mechanism

The connection between anisotropy and failure rates is straightforward. Consider a sentence pair with a "true" semantic similarity of, say, 0.7 on a normalized scale (related but meaningfully different). In an isotropic space like MiniLM's, this maps to an absolute cosine similarity of approximately 0.052 + 0.7 × 0.948 ≈ 0.72 — comfortably below the 0.85 threshold, correctly identified as different. In GTE's anisotropic space, the same normalized similarity maps to approximately 0.711 + 0.7 × 0.289 ≈ 0.91 — well above the 0.85 threshold, incorrectly identified as matching.

This explains why the paradox is not merely about model quality. Even if GTE-large produces internally richer and more discriminative representations than MiniLM — which it likely does, given its higher capacity and larger training corpus — the compression of these representations into a narrow similarity band erases the discriminative signal at the cosine similarity level.

The mechanism can be formalized as follows. Let s_baseline be the random baseline cosine similarity and s_pair be the raw cosine similarity for a given sentence pair. The "effective similarity" — the pair's similarity relative to the model's operating range — is:

s_effective = (s_pair - s_baseline) / (1 - s_baseline)

For MiniLM with s_pair = 0.85, the effective similarity is (0.85 - 0.052) / (1 - 0.052) ≈ 0.84. For GTE with s_pair = 0.85, the effective similarity is (0.85 - 0.711) / (1 - 0.711) ≈ 0.48. The same raw cosine score of 0.85 corresponds to very different effective similarities depending on the model's anisotropy.

6.3 Correlation with Failure Rates

The relationship between anisotropy and failure rates is strongly positive. Plotting mean failure rate against random baseline cosine similarity yields a near-linear relationship:

• MiniLM: baseline 0.052, failure 60%
• BGE: baseline 0.466, failure 88%
• GTE: baseline 0.711, failure 97%

Higher anisotropy directly predicts higher failure rates. While the sample size (three models with measured baselines) is small, the monotonic relationship is consistent with the geometric explanation: each increment of anisotropy compresses the effective range, pushing more adversarial pairs above threshold.

6.4 Why Does Anisotropy Increase with Model Size?

The observation that larger models exhibit greater anisotropy is consistent with prior findings in the representation learning literature. Several factors may contribute:

Dimensional expansion without proportional filling. GTE-large produces 1024-dimensional embeddings, compared to MiniLM's 384 dimensions. Higher-dimensional spaces have vastly more volume, but if the training signal only occupies a lower-dimensional subspace, the embeddings will cluster in a cone rather than filling the full space. Larger models may learn higher-dimensional representations that nonetheless occupy a proportionally smaller fraction of the available space.

Training objective effects. Contrastive training objectives push positive pairs together and negative pairs apart, but the equilibrium geometry depends on the ratio of positive to negative pairs, the temperature parameter, and the difficulty of negative examples. Larger models, trained on larger and potentially noisier datasets, may converge to geometries where the "push-apart" force is weaker relative to the "pull-together" force.

Layer depth and representation collapse. Deeper transformer models are known to exhibit representation collapse in their final layers, where token representations become increasingly similar. While fine-tuning partially counteracts this tendency, residual anisotropy from pretraining may persist, especially in larger models with more layers.

We note that disentangling these factors is beyond the scope of the current study. Our contribution is the empirical observation and its consequence for downstream tasks, not a causal account of why anisotropy increases with scale.

7. Threshold Sensitivity Analysis

7.1 Does Lowering the Threshold Help Larger Models?

One might argue that the paradox is an artifact of threshold selection: perhaps GTE-large's superior representation quality would manifest at a different threshold. We evaluate failure rates across multiple thresholds for all four models.

At τ = 0.90:

• MiniLM: substantially reduced failures in some categories (many pairs fall between 0.85 and 0.90)
• GTE: minimal change (most pairs are already above 0.90)

At τ = 0.80:

• MiniLM: further reduced failures
• GTE: moderate improvement, but failure rates remain high

At τ = 0.70:

• MiniLM: very low failure rates across most categories
• GTE: still above 50% failure in entity swap and temporal categories

The fundamental problem is that no single threshold can compensate for the compressed similarity range. To achieve the same failure rate as MiniLM at τ = 0.85, GTE-large would require a threshold near τ = 0.93 — a threshold so high that it would also reject many genuine paraphrases. The compressed range means there is no threshold where GTE-large simultaneously accepts paraphrases and rejects adversarial pairs with the same reliability as MiniLM.

7.2 Effective Threshold Calibration

An alternative approach is to calibrate the threshold per model, setting τ relative to the model's random baseline. For instance, one might set τ = s_baseline + α × (1 - s_baseline) for some constant α across all models. With α = 0.85:

• MiniLM: τ_calibrated = 0.052 + 0.85 × 0.948 ≈ 0.86
• BGE: τ_calibrated = 0.466 + 0.85 × 0.534 ≈ 0.92
• GTE: τ_calibrated = 0.711 + 0.85 × 0.289 ≈ 0.96

While this calibration approach is theoretically principled, it requires knowledge of each model's random baseline, which is not routinely reported and depends on the input distribution. Moreover, the very high calibrated thresholds for larger models (0.96 for GTE) leave an extremely narrow band for accepting true matches, increasing the risk of false negatives.

The practical implication is clear: correcting for anisotropy through threshold calibration is possible but fragile. A simpler and more robust approach may be to select a model with a naturally wider effective range — that is, a smaller model.

8. The Benchmark Disconnect

8.1 Why MTEB Misses These Failures

The paradox we document is not detected by standard MTEB evaluation for several structural reasons:

Rank-based metrics. MTEB's primary metrics — Spearman correlation for STS tasks, NDCG and MRR for retrieval tasks — measure relative ordering rather than absolute calibration. A model that assigns cosine similarities of 0.95 to paraphrases and 0.92 to adversarial pairs will achieve a high Spearman correlation if the ordering is correct, even though both scores are above any reasonable threshold. The paradox is a calibration failure, not a ranking failure.

Easy negative distributions. Retrieval benchmarks typically evaluate a model's ability to rank a relevant document above thousands of irrelevant documents. The irrelevant documents are usually topically different from the query, making them "easy negatives" that any model can distinguish. The adversarial pairs in our evaluation are "hard negatives" — high lexical overlap, minimal token differences — that are rarely represented in standard benchmark test sets.

Aggregation across tasks. MTEB's aggregate score averages across diverse tasks, many of which do not involve fine-grained semantic discrimination. A model's failure on negation-like phenomena may be washed out by strong performance on clustering, classification, and coarse-grained retrieval tasks.

Absence of targeted probes. MTEB does not include targeted evaluations for specific compositional semantic operations like negation, entity swaps, or quantifier sensitivity. The semantic textual similarity tasks contain some challenging pairs, but they are diluted within large test sets and evaluated via rank correlation rather than threshold-based accuracy.

8.2 Implications for Benchmark Design

Our findings suggest that embedding benchmarks should include at least three additional components:

1. Targeted compositional probes. Test sets specifically designed to evaluate sensitivity to negation, entity roles, temporal ordering, numerical values, and other compositional operations. These should be evaluated using threshold-based metrics, not just rank correlation.

2. Anisotropy reporting. Each model's random-pair baseline cosine similarity should be a standard reported metric, analogous to reporting the number of parameters or embedding dimensions. This single number provides crucial context for interpreting similarity scores and selecting thresholds.

3. Calibration metrics. In addition to discrimination (can the model rank correctly?), benchmarks should evaluate calibration (do the scores mean what practitioners think they mean?). A model whose similarity scores require model-specific calibration to be usable is less practical than one whose scores can be interpreted at face value.

9. When Bigger IS Better: Positive Controls

9.1 Paraphrase Performance

The paradox specifically concerns semantic discrimination — the ability to separate pairs that should be separated. On the complementary task of semantic identification — assigning high similarity to true paraphrases — larger models perform comparably or better.

Across the 35 positive control pairs (true paraphrases), all four models assign high cosine similarities:

• MiniLM: mean cosine ~0.83 on paraphrases
• BGE: mean cosine ~0.91 on paraphrases
• Nomic: mean cosine ~0.89 on paraphrases
• GTE: mean cosine ~0.94 on paraphrases

Larger models assign higher similarities to paraphrases, which is desirable in isolation. The problem arises because they also assign high similarities to non-paraphrases, eliminating the gap between the two distributions.

9.2 Negative Control Separation

On negative controls (completely unrelated sentences), all models successfully assign lower similarities:

• MiniLM: mean cosine ~0.15 on negatives
• BGE: mean cosine ~0.55 on negatives
• GTE: mean cosine ~0.78 on negatives

Even here, the compressed range of larger models is evident. MiniLM places unrelated sentences at 0.15, leaving a gap of ~0.68 between negatives and paraphrases. GTE places them at 0.78, leaving a gap of only ~0.16. This 4x compression of the gap between "unrelated" and "paraphrases" is the geometric signature of the paradox.

9.3 Retrieval Performance

We emphasize that our findings do not imply that smaller models are universally better. On retrieval tasks where relative ranking matters more than absolute threshold calibration, larger models likely maintain their advantage. The paradox is specific to threshold-based applications — duplicate detection, semantic search with cutoffs, automated contradiction detection, fact verification — where the absolute value of cosine similarity drives system behavior.

10. Practical Implications

10.1 Model Selection Guidelines

Our findings suggest a nuanced approach to model selection:

For threshold-based applications (duplicate detection, contradiction detection, semantic filtering), practitioners should consider smaller models with wider effective similarity ranges. MiniLM's near-isotropic embedding space provides a more reliable foundation for threshold-based decisions, despite its lower MTEB ranking.

For ranking-based applications (retrieval, reranking, nearest-neighbor search), larger models may still be preferred if they produce better relative orderings. The paradox does not affect models' ability to rank documents — it affects the interpretability and reliability of absolute similarity scores.

For mixed workloads where both ranking and thresholding are needed, practitioners should either (a) use a smaller model to preserve threshold reliability, (b) calibrate thresholds per model using empirical baseline measurements, or (c) implement a two-stage pipeline with a larger model for retrieval and a cross-encoder or classifier for threshold-based decisions.

10.2 Anisotropy-Aware Threshold Selection

For practitioners committed to using a larger model, we recommend the following protocol for threshold selection:

1. Compute the random baseline cosine similarity by encoding 100+ diverse, unrelated sentences and computing the mean pairwise cosine similarity.
2. Set thresholds relative to this baseline, using the effective similarity formula presented in Section 6.2.
3. Validate the calibrated threshold on a held-out set of challenging pairs (negations, near-misses) before deployment.

This protocol adds overhead but can partially mitigate the paradox for larger models.

10.3 Post-hoc Anisotropy Correction

Several techniques exist for correcting anisotropy in embedding spaces post-hoc:

Whitening applies a linear transformation to make the embedding covariance matrix equal to the identity, effectively isotropizing the space. This can improve cosine similarity interpretability but may discard useful directional information.

Mean centering subtracts the global mean embedding vector, partially reducing the directional bias but not fully isotropizing the space.

Normalization after PCA projects embeddings onto their principal components and renormalizes, reducing the influence of low-variance dimensions that contribute to anisotropy.

While these techniques can improve similarity calibration, they require access to a representative corpus for estimating the transformation parameters. They also add complexity to the deployment pipeline and may introduce distributional assumptions that degrade performance on out-of-distribution inputs.

10.4 Don't Blindly Scale

Perhaps the most important practical implication is simply this: do not assume that a bigger model is better for your task. The MTEB leaderboard is a useful starting point, but it does not evaluate the specific capabilities that many practical applications depend on. Before deploying a larger model, evaluate it on the specific types of semantic distinctions your application requires. If your system needs to detect negation, distinguish entity roles, or separate definitive from hedged claims, a smaller model with a wider effective similarity range may outperform a larger one.

11. Limitations

We acknowledge several limitations of this study:

Sample of models. We evaluate four models, which is sufficient to demonstrate the paradox but insufficient to establish precise scaling laws. The relationship between parameter count and failure rate may be mediated by architecture, training data, training objective, and many other factors that are confounded with model size in our sample. A comprehensive study would require evaluating dozens of models with controlled variation in individual factors.

Correlation vs. causation. Our models differ not only in parameter count but also in architecture depth, embedding dimensionality, pooling strategy, training data, and training objective. We cannot attribute the paradox to model size per se — it may be driven by correlated factors. The anisotropy explanation provides a mechanism, but anisotropy itself may be caused by training methodology rather than parameter count.

Threshold dependence. Our primary analysis uses τ = 0.85, a common but arbitrary threshold. While Section 7 demonstrates that the paradox persists across thresholds, the magnitude varies. At very low thresholds, all models succeed; at very high thresholds, all models fail. The paradox is most pronounced in the operationally relevant range of 0.7–0.9.

Adversarial pair construction. Our test pairs are hand-crafted adversarial examples designed to isolate specific failure modes. They may overestimate the severity of failures compared to naturally occurring sentence pairs, which involve more diverse and less controlled variations. However, the failure modes we test (negation, numerical changes, entity swaps) do occur in real-world applications, and their consequences can be severe in domains like healthcare, finance, and legal text processing.

Fixed embedding models. We evaluate fixed, pre-trained models without fine-tuning on our specific tasks. Fine-tuning on negation-aware or composition-aware objectives may mitigate some of the observed failures. However, the purpose of general-purpose embedding models is to be useful without task-specific fine-tuning, and our evaluation reflects this use case.

Limited anisotropy data. We report random baseline measurements for three of the four models. More comprehensive anisotropy characterization, including PCA analysis and distributional statistics across larger corpora, would strengthen the geometric explanation.

12. Related Work

Compositional semantics in embeddings. Prior work has documented specific failure modes of sentence embeddings. Ettinger (2020) systematically evaluated BERT-like models on psycholinguistically motivated tests including negation and role sensitivity, finding substantial failures. The PAWS dataset (Zhang et al., 2019) demonstrated that paraphrase detection models fail on adversarial pairs with high word overlap but different meanings. Our work extends these findings by establishing a systematic relationship between model scale and failure severity.

Embedding anisotropy. Ethayarajh (2019) documented that contextual embeddings from pretrained language models are highly anisotropic, with the degree of anisotropy increasing in later layers. Li et al. (2020) analyzed the connection between anisotropy and representation degeneration in language models. Our contribution is connecting anisotropy specifically to model scale in the sentence embedding setting and demonstrating its practical consequences for threshold-based applications.

Scaling laws. Kaplan et al. (2020) established power-law scaling relationships for language model loss. Subsequent work has explored scaling laws for downstream tasks, generally finding that larger models improve performance. Our findings represent a counterexample to this trend for a specific but practically important class of tasks.

Benchmark limitations. Several authors have noted limitations of current embedding benchmarks. The original MTEB paper acknowledges that no single benchmark captures all relevant capabilities. Our work provides a concrete example of a capability gap — compositional semantic discrimination — that is systematically missed by current evaluation protocols.

13. Conclusion

We have presented evidence of the model size paradox in sentence embeddings: across four models spanning 22M to 335M parameters, larger models exhibit systematically higher failure rates on fine-grained semantic discrimination tasks. GTE-large (335M parameters) fails on 97% of adversarial pairs at a standard cosine similarity threshold, compared to 60% for MiniLM (22M parameters). This paradox holds across every tested category — negation, entity swaps, temporal inversions, numerical changes, quantifier sensitivity, and hedging — and is explained by increasing anisotropy in larger models' embedding spaces.

The practical implications are significant. Practitioners who select embedding models based on MTEB leaderboard rankings may inadvertently choose models that are worse at the specific task they need — distinguishing semantically different sentences in threshold-based applications. The assumption that bigger is better, while generally valid for aggregate benchmark performance, breaks down precisely where fine-grained semantic discrimination matters most.

We recommend that the embedding community adopt targeted compositional probes as standard evaluation components, require anisotropy reporting as part of model documentation, and develop calibration-aware benchmarks that evaluate absolute similarity score reliability alongside relative ranking performance. Until these changes are adopted, practitioners should evaluate candidate models on their specific semantic requirements rather than relying on aggregate leaderboard positions.

The model size paradox is not an indictment of large models — they likely produce richer internal representations than their smaller counterparts. Rather, it is an indictment of the evaluation and deployment pipeline that conflates aggregate benchmark performance with task-specific capability, and of the cosine similarity metric that fails to preserve discriminative information in anisotropic spaces. Fixing the paradox requires not smaller models, but better metrics, better benchmarks, and better awareness of the geometric properties of the spaces we deploy.

References

Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186.

Ethayarajh, K. (2019). How contextual are contextualized word representations? Comparing the geometry of BERT, ELMo, and GPT-2 embeddings. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 55–65.

Ettinger, A. (2020). What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models. Transactions of the Association for Computational Linguistics, 8:34–48.

Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D. (2020). Scaling laws for neural language models. arXiv preprint arXiv:2001.08361.

Li, B., Zhou, H., He, J., Wang, M., Yang, Y., and Li, L. (2020). On the sentence embeddings from pre-trained language models. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9119–9130.

Reimers, N., and Gurevych, I. (2019). Sentence-BERT: Sentence embeddings using siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992.

Su, J., Cao, J., Liu, W., and Ou, Y. (2021). Whitening sentence representations for better semantics and faster retrieval. arXiv preprint arXiv:2103.15316.

Zhang, Y., Baldridge, J., and He, L. (2019). PAWS: Paraphrase adversaries from word scrambling. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1298–1308.