Skip to content

What is semantic search?

Mykhailo Poliarush

Definition

Semantic search is a retrieval method that ranks results by meaning instead of shared words. An embedding model turns documents and the query into vectors, and the search returns the documents whose vectors sit closest to the query, so a search for "refund" can find a page that only says "money back".

What does semantic search mean? Two uses of the term

The phrase shows up in two places, and they need different advice.

Where you meet itWhat it meansWho builds it
SEO and web searchThe search engine reads the intent behind a query and a page, not only the words in themGoogle, Bing and AI answer engines
Your own product or knowledge baseRetrieval that ranks documents by vector similarity to the queryYour team, with an embedding model and a vector index

This page is about the second meaning: how to build it, and when it beats plain keyword search. Elastic's documentation puts it in one line: semantic search uses NLP models and vector search "to find results based on meaning, not just keywords" (Elastic).

How does semantic search work?

A typical pipeline has five steps:

  1. Split the content into passages short enough to carry one idea each.
  2. Embed each passage. An embedding model turns text into a list of numbers. OpenAI's text-embedding-3-small returns 1,536 numbers by default and text-embedding-3-large returns 3,072; a dimensions parameter can shorten both (OpenAI embeddings guide).
  3. Store the vectors next to the text, in a vector database or an extension of the database you already run.
  4. Embed the query with the same model, at search time.
  5. Return the nearest passages, usually by cosine similarity. OpenAI's vectors are normalized to length 1, so a plain dot product gives the same ranking as cosine similarity and as Euclidean distance (OpenAI).

Step 3 has a detail that catches teams later. In Postgres with pgvector (version 0.8.7 at the time of writing), a query without an index runs exact nearest neighbor search with perfect recall. Adding an HNSW or IVFFlat index switches to approximate search, which trades some recall for speed, so the same query can return different results after the index exists. Both index types accept vector columns of up to 2,000 dimensions. To index a 3,072-number vector from the large OpenAI model, use half precision (up to 4,000 dimensions), a shortened embedding, or one of the other workarounds the README lists.

Semantic search vs keyword search: what is the difference?

QuestionKeyword search (for example BM25)Semantic search
What has to match?The words, after stemming and synonymsThe meaning, as the embedding model sees it
"refund" finds "money back"?Only with a synonym listYes, if the model places them close
Exact codes, SKUs, error IDsStrong: the string matches or notWeak: an unseen code carries little meaning
Word orderMostly ignoredCounts: "chocolate milk" is not "milk chocolate"
Can you explain a result?Yes, the matching termsOnly as a similarity score
What it costs to runAn inverted indexAn embedding call per query and document, plus a vector index

Neither one wins every query, which is why hybrid search, covered below, exists.

Example: keyword and semantic search on the same three queries

This toy index has four help articles. The vectors are assigned by hand on four axes (billing, delivery, account, product), not produced by a model, so the effect is easy to see. The keyword search counts shared words; the semantic search ranks by cosine similarity. We ran the file below with Python 3.11 and pasted the output unchanged.

# Keyword search vs semantic search on a toy index.
# The vectors are hand-assigned, not produced by a model. Axes: [billing, delivery, account, product]
import math, re

DOCS = {
    "Get your money back for a canceled order": [0.9, 0.3, 0.0, 0.1],
    "Track a parcel that has not arrived":       [0.1, 0.9, 0.0, 0.1],
    "Reset a forgotten password":                [0.0, 0.0, 0.9, 0.1],
    "Error ER-4031: card declined at checkout":  [0.7, 0.0, 0.2, 0.0],
}
# What an embedding model might return for each query (also hand-assigned).
# An unseen code like ER-4031 carries little meaning, so its vector is vague.
QUERIES = {
    "refund please":  [0.95, 0.2, 0.0, 0.0],
    "I can't log in": [0.0, 0.0, 1.0, 0.0],
    "ER-4031":        [0.3, 0.3, 0.3, 0.3],
}

def tokens(text):
    return set(re.findall(r"[a-z0-9-]+", text.lower()))

def keyword_top(query):
    scores = {d: len(tokens(query) & tokens(d)) for d in DOCS}
    best = max(scores, key=scores.get)
    return best if scores[best] > 0 else "(no match)"

def cosine(a, b):
    dot = sum(x * y for x, y in zip(a, b))
    return dot / (math.hypot(*a) * math.hypot(*b))

def semantic_top(query):
    scored = sorted(((cosine(QUERIES[query], v), d) for d, v in DOCS.items()), reverse=True)
    return f"{scored[0][1]} ({scored[0][0]:.2f})"

for q in QUERIES:
    print(f"{q!r}\n  keyword:  {keyword_top(q)}\n  semantic: {semantic_top(q)}")

Output:

'refund please'
  keyword:  (no match)
  semantic: Get your money back for a canceled order (0.99)
"I can't log in"
  keyword:  (no match)
  semantic: Reset a forgotten password (0.99)
'ER-4031'
  keyword:  Error ER-4031: card declined at checkout
  semantic: Get your money back for a canceled order (0.68)

The first two queries share no words with the right article, so keyword search returns nothing while vector similarity finds it. The third query flips the result. The error code is the whole query, keyword search matches it exactly, and the vague vector points at the wrong page. A real embedding model is less crude than this toy, so test identifiers on yours instead of assuming either way.

Is semantic search always better than keyword search?

No, and a widely used benchmark shows why. BEIR (Thakur et al., NeurIPS 2021, an older source) tested 10 retrieval systems on 18 public datasets without training on any of them. Plain BM25 held up well across that range. Re-ranking and late-interaction models scored best on average, at high compute cost. Dense retrieval models were cheaper to run but often scored below the other approaches on data unlike their training data.

The practical reading: an embedding model that does well on public benchmarks can still lose to keyword search on your contracts, tickets or product catalog. Measure on your own queries before you switch.

The common answer is hybrid search: run both and merge the lists. Elastic recommends reciprocal rank fusion, which gives weight to documents that rank well in either list (Elastic). For answers written by a model on top of the results, see agentic RAG, where an agent also decides when to search again.

What should you check before shipping semantic search?

  • A labeled query set. Take real queries from your search logs, label the passages that should come back, and track how often the right one lands in the top five results. Rerun it when you change the model, the chunking or the index. The AI evals entry shows how to wire such a check into CI.
  • One model per index. Vectors from two embedding models are not comparable. A model change means re-embedding every document.
  • Permissions before ranking. pgvector notes that with approximate indexes, a WHERE filter is applied after the index scan, so filtered queries can return fewer results than asked for. Its iterative index scans (from 0.8.0) keep scanning until enough rows match.
  • A keyword path for identifiers. Codes, names and numbers need exact matching, or hybrid search.
  • Clean, current content. A vector index over stale documents returns stale answers faster. Our report on AI-ready data covers what preparing the data involves.

Retrieval is part of our generative AI development work: document ingestion, chunking, embeddings, hybrid search and re-ranking, with an evaluation set built from your real cases that runs on every change to prompts, models or retrieval.

Sources

  1. Semantic search in Elasticsearch, Elastic Docs, 2026-11
  2. Hybrid search, Elastic Docs, 2026-11
  3. Vector embeddings, OpenAI API docs, 2026-11
  4. pgvector README (v0.8.7), pgvector (GitHub), 2026-11
  5. BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models, arXiv (Thakur et al., NeurIPS 2021 Datasets and Benchmarks), 2021-10-21

Frequently asked questions

What is semantic search in SEO?

In SEO the term describes how search engines interpret the intent of a query and the topic of a page instead of matching exact phrases. For writers it means answering the question fully and clearly; repeating a keyword adds little.

Should I use hybrid search or semantic search?

If your users type product codes, names or error IDs, start with hybrid. Pure semantic search fits when queries are mostly natural-language questions over prose and your own labeled queries show it beating keyword search.

What is a semantic search engine?

A search system that indexes content as vectors and ranks results by similarity of meaning. You can build one on a search engine such as Elasticsearch, whose docs recommend its semantic_text workflow, or on Postgres with pgvector.

How does semantic search power AI answer engines?

AI assistants that search the web or your documents retrieve the passages whose meaning matches the question, then write an answer from them. Pages with short, self-contained answers are easier to retrieve and quote.

Building with AI agents? Talk to an engineer.