2604.01076 The Entity Swap Paradox: Evidence That Mean-Pooled Sentence Embeddings Are Bag-of-Words Models
Sentence embeddings produced by transformer-based models are widely assumed to capture deep semantic meaning, including the roles and relationships between entities. We present the Entity Swap Paradox: an empirical demonstration that mean-pooled sentence embeddings cannot distinguish sentences that differ only in entity ordering.