Semantic Search vs Keyword Search

Two different definitions of 'relevant'

Search engines have to decide what 'matches the query' even means, and there are two fundamentally different answers.

Keyword search defines relevance in terms of word overlap: a document is relevant if it contains the words (or close variants) in the query, weighted by how distinctive and frequent those words are. Semantic search defines relevance in terms of meaning: a document is relevant if it's about the same concept as the query, even if it uses completely different words to express it.

Both are legitimate, useful definitions — and they fail in different, often complementary, situations.

How keyword search actually scores documents: BM25

The dominant classic algorithm is BM25 (Best Matching 25), a refinement of TF-IDF. It scores a document for a query based on three ideas: term frequency (a word appearing more times in a document is a stronger signal, with diminishing returns), inverse document frequency (a word that appears in almost every document in the corpus — 'the', 'system' — carries little discriminating power; a rare word like a specific product code carries a lot), and document length normalization (a long document naturally contains more word occurrences, so raw counts are normalized against document length to avoid just favoring long documents).

BM25 requires no training and no embedding model — it's a statistical formula over word counts, computed directly from the corpus.

How semantic search works: embeddings

Semantic search represents both the query and each document as a dense vector (an embedding) produced by a trained model, positioned so that texts with similar meaning sit close together in vector space regardless of exact wording. A query like 'how do I get my money back' and a document titled 'refund policy' can be vector-neighbors even though they share almost no words. Similarity is measured with cosine similarity or dot product between query and document vectors, typically via an approximate nearest-neighbor index for speed at scale.

BM25 keyword search vs embedding semantic search

Why hybrid search usually wins in practice

Because the two approaches fail in different, largely non-overlapping situations, combining them captures more real-world queries than either alone. A production search or RAG retrieval system runs both a keyword search (BM25 or similar) and a semantic vector search for every query, then merges the two ranked result lists — commonly via Reciprocal Rank Fusion, which combines ranks rather than raw scores since the two scoring systems aren't on comparable scales.

This isn't a minor tweak — it's the standard default for serious production retrieval systems, precisely because real user queries are an unpredictable mix of 'exact term I need' and 'describe what I mean in my own words,' sometimes within the same query.

When to skip semantic search entirely

Semantic search isn't always worth its cost and complexity. If your corpus is small, your queries are mostly exact-match lookups (log search, structured codes, SKU lookup, legal/regulatory citation search), or your users are domain experts who already know the precise terminology they want, plain keyword search can outperform semantic search while being far simpler, cheaper, fully explainable, and requiring no embedding model or vector database at all. Add semantic search when you observe real retrieval gaps caused by vocabulary mismatch between how users ask and how your documents are written — not by default, just because embeddings are available.

What's next

Both keyword and semantic search assume you already have a decent off-the-shelf embedding model or term index. Sometimes a generic embedding model just doesn't capture what 'similar' means in your specific domain. The next lesson covers fine-tuning embedding models for domain-specific retrieval.