1. Introduction

The dominant paradigm for computing semantic similarity in modern NLP systems relies on bi-encoder architectures: two sentences are independently encoded into fixed-dimensional vectors, and their similarity is estimated via cosine distance. This approach powers semantic search engines, retrieval-augmented generation pipelines, duplicate detection systems, and countless production applications that depend on accurate similarity judgments.

Recent work has exposed systematic failures in this paradigm. When applied to sentence pairs involving negation ("The patient has diabetes" vs. "The patient does not have diabetes"), entity role swaps ("Google acquired YouTube" vs. "YouTube acquired Google"), or numerical changes ("Take 5mg of aspirin" vs. "Take 500mg of aspirin"), bi-encoder models consistently assign similarity scores above 0.85 on a 0-1 scale — treating semantically opposite or critically different sentences as near-identical. These failures stem from the architectural constraint of independent encoding: mean pooling over token embeddings creates a bag-of-words representation that preserves lexical overlap but erases word order, dilutes negation tokens, and ignores compositional structure.

The standard remedy proposed in the literature is the cross-encoder architecture: instead of encoding sentences independently, both sentences are concatenated and fed through a single transformer, allowing full cross-attention between all tokens in both sentences. This architectural difference theoretically enables cross-encoders to detect word-order changes, attend to negation tokens in context, and reason about the compositional relationship between sentence pairs. Cross-encoders have demonstrated superior performance on natural language inference (NLI) benchmarks, semantic textual similarity (STS) tasks, and passage re-ranking (Nogueira and Cho, 2019).

However, a critical gap exists in the literature: no systematic evaluation has tested cross-encoders against the specific failure modes identified in bi-encoder systems. Do cross-encoders truly fix negation blindness? Can they detect entity swaps that bi-encoders miss entirely? And crucially, are all cross-encoders equally capable, or does training objective matter as much as architecture?

We address these questions with a controlled experiment comparing four cross-encoder models against five bi-encoder models on 336 hand-crafted sentence pairs spanning nine semantic categories: six adversarial failure modes (negation, numerical changes, entity swaps, temporal inversions, quantifier changes, and hedging/certainty shifts) plus three control conditions (true paraphrases, unrelated pairs, and near-miss pairs). Our results reveal that while cross-attention provides the necessary mechanism for compositional reasoning, the training objective fundamentally determines which failures a model can detect — with profound implications for production system design.

2. Background

2.1 Bi-Encoder Architecture

Bi-encoder models (Reimers and Gurevych, 2019) encode each sentence independently through a shared transformer backbone, producing a fixed-dimensional embedding via pooling (typically mean pooling over token representations). Semantic similarity between two sentences is computed as the cosine similarity between their embeddings:

sim(A, B) = cos(E(A), E(B))

where E is the encoding function. This architecture enables efficient retrieval through approximate nearest neighbor search, making it the practical choice for large-scale applications. However, because the encoding of sentence A is computed without access to sentence B, the model cannot perform cross-sentence reasoning. Any compositional relationship must be captured solely in the independent representations.

The fundamental limitation is that mean pooling creates what is effectively a continuous bag-of-words: the embedding aggregates information about which tokens appear and their contextual representations, but loses explicit information about token ordering and inter-token relationships that are diluted during the averaging process.

2.2 Cross-Encoder Architecture

Cross-encoders process both sentences jointly by concatenating them with a separator token and feeding them through a single transformer pass:

score(A, B) = f(Transformer([CLS] A [SEP] B [SEP]))

where f is typically a linear layer applied to the [CLS] token representation. This architecture allows every token in sentence A to attend to every token in sentence B through the transformer's self-attention mechanism, enabling direct comparison of compositional structures.

The computational cost is substantially higher: while bi-encoders require O(n) forward passes to score n candidates against a query (or O(1) with precomputed embeddings), cross-encoders require O(n) forward passes since each pair must be processed jointly. This makes cross-encoders impractical for first-stage retrieval but standard for re-ranking small candidate sets (Nogueira and Cho, 2019).

2.3 Training Objectives and Model Families

Cross-encoder models differ not only in architecture but in training objective, which determines what the output score represents:

Semantic Textual Similarity (STS): Models trained on STS-B and related datasets learn to predict human-rated similarity on a continuous scale (typically 0-5). These models learn a general notion of meaning similarity.

Natural Language Inference (NLI): Models trained on SNLI/MultiNLI learn to classify sentence pairs as entailment, contradiction, or neutral. These models explicitly learn to detect when sentences have opposite meanings.

Information Retrieval: Models trained on MS-MARCO or similar retrieval datasets learn to predict query-document relevance. The key distinction is that relevance is not the same as similarity — a document that contradicts a query may still be highly relevant to it.

Duplicate Detection: Models trained on Quora Question Pairs learn to identify semantically equivalent questions. These models must distinguish between questions that look similar but ask different things.

This diversity of training objectives raises a crucial question: does the theoretical advantage of cross-attention manifest equally across all training regimes, or do some objectives fail to exploit the architectural capacity for compositional reasoning?

3. Experimental Setup

3.1 Models

We evaluate five bi-encoder models spanning different model sizes and training approaches:

1. all-MiniLM-L6-v2 — A compact 6-layer model from the sentence-transformers library, trained with knowledge distillation on over 1 billion sentence pairs
2. BGE-large-en-v1.5 — A large bi-encoder from BAAI, trained with sophisticated negative mining on diverse text data
3. nomic-embed-text-v1.5 — A model from Nomic AI trained with a novel training procedure emphasizing long-context capability
4. mxbai-embed-large-v1 — A large embedding model from Mixedbread AI with strong performance on MTEB benchmarks
5. GTE-large — A general text embedding model from Alibaba (Thenlper) trained on large-scale text pair data

We evaluate four cross-encoder models representing different training objectives:

1. cross-encoder/stsb-roberta-large — RoBERTa-large fine-tuned on STS-B for semantic similarity (output range: 0-5)
2. cross-encoder/ms-marco-MiniLM-L-12-v2 — MiniLM fine-tuned on MS-MARCO for passage retrieval relevance (output: logits)
3. BAAI/bge-reranker-large — A production reranker from BAAI trained for retrieval re-ranking (output: 0-1 probability)
4. cross-encoder/quora-roberta-large — RoBERTa-large fine-tuned on Quora Question Pairs for duplicate detection (output: 0-1 probability)

A fifth cross-encoder (cross-encoder/nli-roberta-large, NLI-trained) was planned but could not be loaded due to repository access restrictions and was excluded from the study.

3.2 Test Dataset

We use the same 336 hand-crafted sentence pairs from the prior bi-encoder failure study, organized into nine categories:

Adversarial categories (251 pairs designed to expose compositional failures):

• Negation (55 pairs): Sentence pairs differing only by negation ("The patient has diabetes" / "The patient does not have diabetes"). Spans medical, legal, financial, and technology domains.
• Numerical (56 pairs): Pairs with order-of-magnitude numerical changes ("Take 5mg of aspirin daily" / "Take 500mg of aspirin daily"). Includes medically critical dosage differences.
• Entity swap (45 pairs): Pairs where subject and object roles are reversed ("Google acquired YouTube" / "YouTube acquired Google"). Tests sensitivity to word order and argument structure.
• Temporal (35 pairs): Pairs where before/after relationships are inverted ("Apply sunscreen before going outside" / "Apply sunscreen after going outside"). Tests sensitivity to temporal markers.
• Quantifier (35 pairs): Pairs with different quantifier scopes ("All patients responded to treatment" / "No patients responded to treatment"). Tests sensitivity to universal vs. existential quantification.
• Hedging (25 pairs): Pairs differing in certainty level ("The drug cures cancer" / "The drug may help with some cancer symptoms"). Tests sensitivity to epistemic modality.

Control categories (85 pairs for calibration):

• Positive control (35 pairs): True paraphrases that should receive high similarity scores.
• Negative control (35 pairs): Completely unrelated sentence pairs that should receive low similarity scores.
• Near-miss control (15 pairs): Sentences differing in small factual details (e.g., different cities, dates, or product names).

All pairs were hand-crafted to isolate specific linguistic phenomena. No machine-generated pairs were used.

3.3 Scoring and Normalization

For bi-encoders, we compute cosine similarity between independently encoded sentence embeddings, yielding scores in the range [-1, 1] (in practice, approximately [0, 1] for meaningful text).

For cross-encoders, raw output scores vary by model:

• STS-B model: Outputs on a 0-5 scale (matching STS-B annotation scheme). We normalize to [0, 1] by dividing by 5.
• MS-MARCO model: Outputs logits (unbounded). We apply sigmoid normalization to map to [0, 1].
• BGE-reranker: Outputs probabilities in [0, 1]. Used directly.
• Quora model: Outputs probabilities in [0, 1]. Used directly.

3.4 Evaluation Metrics

We report:

• Mean score ± standard deviation per model per category
• Failure rate at threshold t: the fraction of adversarial pairs scoring at or above t. For adversarial categories, a correct model should assign low scores, so high failure rates indicate poor performance
• Cohen's d effect size comparing cross-encoder vs. bi-encoder score distributions per category
• Mann-Whitney U tests for statistical significance of distribution differences
• Wilcoxon signed-rank tests for paired comparisons on the same sentence pairs

3.5 Software Versions

All experiments were conducted using PyTorch 2.4.0 (CPU) and sentence-transformers 3.0.1, with NumPy and SciPy for statistical analysis.

4. Results

4.1 Overview: Bi-Encoder Failure Rates

Confirming prior findings, all five bi-encoder models exhibit catastrophic failure rates across adversarial categories. At a threshold of 0.5, 100% of adversarial pairs across all six failure categories were scored as similar by every bi-encoder model. Even at the stricter threshold of 0.8, failure rates remain extreme:

Bi-encoder failure rates at threshold 0.8 (% of adversarial pairs scored >= 0.8):

The average bi-encoder similarity scores for adversarial categories are: negation 0.896 +/- 0.054, numerical 0.896 +/- 0.060, entity swap 0.987 +/- 0.011, temporal 0.953 +/- 0.030, quantifier 0.855 +/- 0.080, and hedging 0.831 +/- 0.089. For context, the positive control (true paraphrases) average is 0.879 +/- 0.089 — meaning adversarial pairs score as similar as or more similar than genuine paraphrases.

4.2 Cross-Encoder Results: The Training Objective Matters

Cross-encoder performance varies dramatically by training objective. We present results for each model separately before aggregating.

Quora-RoBERTa-Large (Duplicate Detection):

This model achieves near-perfect adversarial detection. Normalized scores (0-1 probability of being a duplicate):

The Quora model correctly identifies negation pairs, entity swaps, numerical changes, and temporal inversions as non-duplicates with scores below 0.05 in most cases. The separation between positive controls (0.894) and adversarial categories (0.020-0.168) is enormous for the first four failure modes.

BGE-Reranker-Large (Production Reranker):

The BGE reranker shows excellent performance on negation (0.073) and temporal changes (0.073), good performance on numerical changes (0.114), but struggles more with entity swaps (0.398, with 33% failure rate at 0.5) and hedging (0.883, essentially indistinguishable from positive controls). The positive control score of 0.996 demonstrates that this model maintains high sensitivity to genuine semantic similarity while detecting most adversarial patterns.

STS-B-RoBERTa-Large (Semantic Similarity):

The STS-B model presents a nuanced case. While it achieves 0% failure rate at the 0.5 normalized threshold across all categories, the absolute scores are highly compressed: positive controls average only 0.178 on the normalized scale, and the separation between positive controls (0.178) and the worst adversarial category (entity swap at 0.167) is only 0.011. The model rates all pairs as having relatively low similarity on its 0-5 output scale, with the entire range compressed between 0.01 and 0.97. This means the model technically "detects" adversarial pairs but with very poor discrimination.

MS-MARCO-MiniLM-L-12-v2 (Retrieval Relevance):

The MS-MARCO model represents a complete failure: it assigns higher scores to adversarial pairs than to positive controls. Negation pairs, entity swaps, and temporal inversions all receive scores above 0.999, compared to 0.871 for true paraphrases. This model has learned that topically related text is relevant regardless of whether it agrees, contradicts, or inverts the query — precisely the behavior expected of a retrieval model but catastrophic for similarity assessment.

4.3 Aggregate Comparison

Averaging across the three non-retrieval cross-encoders (STS-B, BGE-reranker, Quora) and comparing against the five bi-encoder average:

All differences are statistically significant (Mann-Whitney U, all p < 0.01). The effect sizes are enormous: Cohen's d values of -14.8 for negation and -13.8 for temporal changes indicate distributions with essentially zero overlap. Even the smallest effect (hedging, d = -1.2) represents a large effect by conventional standards.

4.4 Failure Rate Comparison at Multiple Thresholds

At a threshold of 0.5, bi-encoders exhibit 100% failure rates across all models and all adversarial categories. The cross-encoder failure rates (averaged across non-retrieval models) are:

The reduction is dramatic for negation (100% to 0%), entity swaps (100% to 11%), and temporal inversions (100% to 1%). Hedging remains the persistent problem, with nearly half of hedging pairs still misclassified even by cross-encoders.

4.5 Per-Category Deep Dive

Negation: This is the strongest result. Bi-encoders average 0.896 similarity for negated pairs — virtually identical to the 0.879 average for true paraphrases. Cross-encoders collapse this to 0.064 (Quora: 0.020, BGE: 0.073, STS-B: 0.098). The cross-attention mechanism enables the model to directly compare "has" with "does not have" across the sentence pair, a comparison impossible when sentences are encoded independently.

Entity Swap: Bi-encoders score entity swaps at 0.987 — higher than paraphrases. This makes architectural sense: entity swaps contain exactly the same tokens, just reordered, and mean pooling is order-invariant. Cross-encoders reduce this to 0.201 on average, though with notable variance. The Quora model (0.037) outperforms BGE (0.398) on this category, likely because duplicate detection training explicitly penalizes pairs with the same words in different arrangements.

Numerical: Bi-encoders score numerical changes at 0.896. Cross-encoders reduce this to 0.074, with remarkable consistency across all three non-retrieval models (STS-B: 0.091, BGE: 0.114, Quora: 0.018). The joint processing allows token-level comparison of numerical values.

Temporal: Before/after inversions are scored at 0.953 by bi-encoders but only 0.081 by cross-encoders. This is architecturally intuitive: the temporal markers "before" and "after" occupy the same position in otherwise identical sentences, and cross-attention can directly detect this single-word substitution.

Quantifier: "All patients" vs. "No patients" scored 0.855 by bi-encoders, reduced to 0.187 by cross-encoders. The higher residual variance (BGE: 0.281 with some pairs reaching 0.999) suggests that quantifier scope remains partially challenging even for cross-encoders, particularly when the quantifier interacts with complex predicates.

Hedging: This is the failure mode where cross-encoders show the least improvement. Bi-encoders score hedging at 0.831; cross-encoders average 0.509, with the BGE model actually scoring 0.883 — nearly as high as its positive control (0.996). Hedging changes involve distributed, multi-word modifications ("cures" to "may help with some symptoms") that alter certainty rather than polarity. The semantic relationship between a definitive claim and its hedged version is inherently more ambiguous: they are not contradictions, not paraphrases, but somewhere in between.

4.6 The MS-MARCO Anomaly: When Cross-Attention Is Not Enough

The MS-MARCO results merit special discussion because they demonstrate that architectural capacity is a necessary but insufficient condition for compositional reasoning. This model has full cross-attention between sentence pairs yet scores negation pairs at 0.9996 — higher than any bi-encoder.

The explanation lies in the training objective. MS-MARCO trains on query-passage relevance: given a search query, is a passage relevant? Under this framing, a passage that states "The patient does NOT have diabetes" is highly relevant to a query about "The patient has diabetes" — it directly addresses the same topic and entity, even though it conveys opposite information. Similarly, "YouTube acquired Google" is relevant to a query about "Google acquired YouTube" because both concern the same entities and the concept of acquisition.

This finding has profound implications: cross-encoder architecture enables but does not guarantee compositional reasoning. The training signal must specifically reward the model for distinguishing meaning-altering variations, not merely topical relevance.

5. Analysis

5.1 Why Training Objective Dominates Architecture

Our results cleanly separate two factors that have been conflated in prior work:

1. Architectural capacity (independent encoding vs. joint attention)
2. Training objective (similarity vs. relevance vs. entailment vs. duplicate detection)

The MS-MARCO model proves that Factor 1 alone is insufficient. The STS-B, BGE, and Quora models prove that Factor 1 + Factor 2 together are highly effective. The bi-encoder results prove that Factor 2 alone (since many bi-encoders are trained on similar objectives) is insufficient without Factor 1.

The interaction is: cross-attention provides the mechanism to compare compositional structures, but the training objective determines whether this mechanism is used to detect meaning-altering differences or merely to assess topical relatedness.

5.2 The Hedging Problem: Semantic Gradients vs. Binary Distinctions

The persistent difficulty with hedging across all models reveals a fundamental challenge in similarity modeling. Negation, entity swaps, and temporal inversions create binary semantic contrasts: the meaning flips from A to not-A. These are relatively easy for cross-encoders to detect because the training data contains clear examples of contradictions.

Hedging, by contrast, creates a semantic gradient. "The drug cures cancer" and "The drug may help with some cancer symptoms" are not contradictory — the hedged version is consistent with the definitive version being true. They share the same entities, the same domain, and roughly the same propositional content. The difference is epistemic: one makes a strong claim, the other a weak one. Existing training datasets (STS-B, NLI, Quora duplicates) poorly represent these epistemic gradients, leading to persistent failures.

The BGE reranker's near-total failure on hedging (0.883 vs. 0.996 for positive controls) is particularly instructive: from a retrieval perspective, the hedged version IS relevant to the definitive claim. This is arguably correct behavior for a reranker but wrong for a similarity model.

5.3 Entity Swap Variance Across Models

Entity swaps show the most variance across cross-encoder models: Quora (0.037), STS-B (0.167), BGE (0.398). The Quora model excels because duplicate detection training explicitly requires distinguishing "Is A better than B?" from "Is B better than A?" — exactly the entity-swap phenomenon. STS-B training includes some compositional examples but not specifically adversarial swaps. BGE's higher entity swap scores suggest that reranking training encounters fewer examples where entity roles are deliberately reversed.

5.4 Quantifier Difficulty

Quantifiers present an intermediate challenge (cross-encoder average: 0.187) with high variance. Some quantifier pairs ("All flights were cancelled" vs. "Some flights were cancelled") are relatively easy because "all" and "some" are common in NLI training data. Others ("Universal healthcare covers everyone" vs. "Universal healthcare covers most people") are harder because the quantitative difference between "everyone" and "most people" is smaller and the sentences remain broadly compatible.

6. Practical Recommendations

Our findings support a tiered architecture selection strategy based on the semantic precision requirements of the application:

6.1 When Bi-Encoders Are Sufficient

For applications where topical similarity is adequate — exploratory search, document clustering, broad recommendation — bi-encoders remain the practical choice. Their O(1) scoring time with precomputed embeddings enables real-time search over millions of documents. The failure modes we identify are irrelevant when the application does not require distinguishing "has" from "does not have."

6.2 When Cross-Encoders Are Essential

For applications requiring semantic precision — medical record matching, legal document comparison, financial report analysis, fact verification — cross-encoders are essential but must be carefully selected:

• Use duplicate-detection-trained models (Quora-style) when the goal is to determine if two texts say the same thing. These models show the best overall adversarial robustness.
• Use reranker models (BGE-style) when the goal is to re-rank retrieval results, but be aware of reduced sensitivity to entity swaps and hedging.
• Avoid retrieval-trained models (MS-MARCO-style) for similarity assessment. Despite being cross-encoders, they may perform worse than bi-encoders on adversarial pairs.

6.3 Hybrid Pipeline Design

The optimal production architecture is a two-stage pipeline:

1. First stage (bi-encoder): Retrieve candidate documents/sentences using cosine similarity. Fast, scalable, sufficient for topical filtering.
2. Second stage (cross-encoder): Re-rank candidates using a task-appropriate cross-encoder. Catches negation, entity swaps, temporal inversions, and most quantifier errors.

This pipeline captures the efficiency of bi-encoders for recall while leveraging cross-encoders for precision. However, practitioners should note that this pipeline will not catch hedging failures — additional mechanisms (e.g., uncertainty-aware models or explicit epistemic classifiers) may be needed for applications sensitive to certainty levels.

6.4 Residual Risk: Hedging

No model in our evaluation reliably detects hedging changes. For applications where certainty level matters (medical claims, financial advice, legal assertions), we recommend:

• Post-processing with dedicated uncertainty detection classifiers
• Explicit metadata tagging of claim certainty levels
• Not relying on any current embedding or cross-encoder model for hedging detection

7. Limitations

Model selection: We evaluated four cross-encoder models. The NLI-trained model (cross-encoder/nli-roberta-large) could not be tested due to repository access restrictions. Given that NLI training explicitly models contradiction, this model might show different hedging performance and should be evaluated in future work.

Test set scope: Our 336 pairs, while spanning six failure modes and three control categories, were crafted in English only and focus on relatively simple compositional structures. Performance on longer, more complex sentences with embedded negation, multiple entity swaps, or nested quantifiers remains untested.

Normalization: Comparing scores across models with different output ranges (0-5, logits, probabilities) requires normalization. While we applied standard transformations (scaling, sigmoid), the choice of normalization affects failure rate calculations. We report raw scores alongside normalized values for transparency.

Scale of evaluation: Our test set contains 336 pairs — sufficient for detecting large effects but potentially insufficient for subtle interactions between failure modes or for estimating precise failure rates on rare edge cases.

Computational cost: We did not measure inference latency, though cross-encoders are known to be 10-100x slower than bi-encoders for scoring. Production deployment considerations should factor in this computational overhead.

Training data contamination: We cannot verify whether any of our test pairs (or similar constructions) appeared in the training data of any evaluated model. If models were exposed to adversarial negation pairs during training, their performance on our test set may overestimate generalization ability.

8. Related Work

Reimers and Gurevych (2019) introduced Sentence-BERT, the foundational bi-encoder framework for sentence similarity. Their work demonstrated that pre-trained transformers could be fine-tuned to produce sentence embeddings suitable for cosine similarity comparison, establishing the paradigm our work evaluates.

Devlin et al. (2019) introduced BERT, the pre-trained transformer architecture underlying both bi-encoder and cross-encoder models evaluated in this study. The [CLS] token representation used by cross-encoders for classification originates from BERT's architecture.

Nogueira and Cho (2019) demonstrated the effectiveness of cross-encoder architectures for passage re-ranking, establishing the two-stage retrieval pipeline that our practical recommendations build upon.

Humeau et al. (2020) proposed poly-encoders as a middle ground between bi-encoders and cross-encoders, offering some cross-attention capability at reduced computational cost. Our results suggest that the completeness of cross-attention (full cross-encoder vs. partial poly-encoder) may matter less than training objective.

Ettinger (2020) conducted systematic evaluations of what BERT representations do and do not capture, finding specific linguistic phenomena (including negation) that challenge pre-trained transformers. Our work extends this analysis to the downstream task of similarity estimation across both bi-encoder and cross-encoder deployments.

9. Conclusion

We present the first systematic evaluation of cross-encoder robustness to the compositional semantic failures that plague bi-encoder embedding models. Testing four cross-encoder models against five bi-encoders on 336 hand-crafted adversarial and control pairs, we find that:

1. Cross-encoders dramatically reduce failure rates on negation (100% to 0%), entity swaps (100% to 11%), numerical changes (100% to 2%), and temporal inversions (100% to 1%) when trained on appropriate objectives.

2. Training objective trumps architecture. A retrieval-trained cross-encoder (MS-MARCO) performs worse than bi-encoders on adversarial pairs, while duplicate-detection-trained models (Quora) achieve near-perfect robustness. Cross-attention enables but does not guarantee compositional reasoning.

3. Hedging remains unsolved. Even the best cross-encoders show 48-92% failure rates on certainty/hedging changes, representing a persistent blind spot in current similarity models.

4. Practical deployment should use hybrid pipelines with bi-encoder retrieval followed by task-appropriate cross-encoder reranking, avoiding retrieval-trained cross-encoders for similarity assessment.

These findings reframe the bi-encoder vs. cross-encoder discussion: the question is not merely "should I use a cross-encoder?" but "which cross-encoder, trained on what?" The answer depends fundamentally on whether the application requires topical relevance (where even bi-encoders may suffice) or semantic precision (where only task-appropriate cross-encoders succeed).

References

Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of NAACL-HLT 2019.

Ettinger, A. (2020). What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models. Transactions of the Association for Computational Linguistics, 8, 34-48.

Humeau, S., Shuster, K., Lachaux, M.-A., and Weston, J. (2020). Poly-encoders: Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring. In Proceedings of ICLR 2020.

Nogueira, R. and Cho, K. (2019). Passage Re-ranking with BERT. arXiv preprint arXiv:1901.04085.

Reimers, N. and Gurevych, I. (2019). Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In Proceedings of EMNLP-IJCNLP 2019.