Embeddings and semantic search: a practical guide for product teams

An embedding is a list of numbers, typically a few hundred to a few thousand, that represents the meaning of a piece of text. An embedding model is trained so that texts with similar meanings end up close together. Semantic search embeds the user's query and finds the documents or products whose embeddings are nearest to it.

That lets search match intent even when the wording differs. A shopper searching for "warm jacket for cycling to work" can find a product described as a "windproof insulated commuter coat", and an employee asking "how do I claim travel costs" can find the expenses policy.

This guide is for product managers and engineers improving product search, site search or internal knowledge search. It covers choosing a model, where to store vectors, combining semantic and keyword ranking, and measuring relevance so you can tell whether a change helped.

There are many capable embedding models, both commercial and open-weight. Public leaderboards are a reasonable starting point, but they rank models on benchmark data, and what counts is performance on your queries and your content. Weigh these factors:

• Retrieval quality on your data, measured with your own evaluation set as described below.
• Language coverage. A Dutch and English catalogue needs a multilingual model that handles both, including mixed queries.
• Dimensions and storage. Larger vectors can capture more but cost more to store and search. Some models support shortened embeddings with a modest loss in quality.
• Input length. The maximum tokens per input limits how long a chunk or product description can be.
• Hosting and data rules. An API model is simple to run; an open-weight model you host keeps data inside your environment.
• Cost and latency, both for indexing the full catalogue and for every query.

Pick two or three candidates, embed a representative sample and compare them on the same evaluation set. Store the model name and version with every vector. Switching models means re-embedding everything, because vectors from different models are not comparable.

If your data already lives in Postgres, pgvector adds a vector column type, distance operators and approximate nearest neighbour indexes (HNSW and IVFFlat). The basics fit in a few lines:

CREATE EXTENSION IF NOT EXISTS vector;

ALTER TABLE products ADD COLUMN embedding vector(1024);

CREATE INDEX products_embedding_idx
  ON products USING hnsw (embedding vector_cosine_ops);

-- nearest in-stock products to the query embedding
SELECT id, title, 1 - (embedding <=> $1) AS similarity
FROM products
WHERE in_stock
ORDER BY embedding <=> $1
LIMIT 20;

Dedicated vector databases add features aimed at very large or very demanding workloads. The trade-offs:

Our default is pgvector until measurements show a reason to move. For most product catalogues and knowledge bases, keeping vectors next to the data they describe removes a whole category of sync bugs.

Pure semantic search has blind spots. It can rank a similar product above the exact one a user typed, and it handles SKUs, model numbers, brand names and sizes poorly. Keyword search, with BM25 or Postgres full-text search, is strong in exactly those places.

Hybrid ranking runs both and merges the results. Common approaches:

• Reciprocal rank fusion (RRF) combines the ranks from each list without needing comparable scores. It is simple and a good default.
• Weighted score blending normalises both scores and mixes them, which gives more control but needs tuning.
• Reranking passes the merged top results to a cross-encoder or LLM that scores each one against the query.
• Business signals such as stock, margin, popularity and personalisation are applied as a final layer, kept separate so relevance can be measured on its own.

Our production RAG chatbot guide includes a hybrid query for pgvector with RRF.

Without measurement, search tuning turns into a debate about individual queries. Build a judged set: a few hundred real queries from your search logs, each with the items that are relevant, ideally graded (for example 0 for irrelevant, 1 for partly relevant, 2 for a good match and 3 for exactly right).

Two metrics cover most needs:

• recall@k is the share of relevant items that appear in the top k results. It tells you whether the right results are found at all, which matters most for RAG and for the candidate stage before reranking.
• nDCG@k (normalised discounted cumulative gain) rewards putting the most relevant items at the top, using graded judgements. It reflects how people scan a results page.

import numpy as np

def recall_at_k(retrieved: list[str], relevant: set[str], k: int) -> float:
    if not relevant:
        return 0.0
    return len(set(retrieved[:k]) & relevant) / len(relevant)

def dcg(gains: np.ndarray) -> float:
    discounts = np.log2(np.arange(2, gains.size + 2))
    return float(np.sum((2**gains - 1) / discounts))

def ndcg_at_k(retrieved: list[str], grades: dict[str, int], k: int) -> float:
    gains = np.array([grades.get(doc_id, 0) for doc_id in retrieved[:k]], dtype=float)
    ideal = np.array(sorted(grades.values(), reverse=True)[:k], dtype=float)
    best = dcg(ideal)
    return dcg(gains) / best if best > 0 else 0.0

Track these per query segment, such as head queries, long-tail queries and queries containing SKUs, because an average can hide a regression in one group. Once offline metrics improve, confirm with online metrics like click-through, search-to-cart rate and the share of searches with no results. For the retrieval side of RAG specifically, see how to evaluate RAG.

1. 01Export a few hundred real queries from your search logs, including ones that currently return poor results or none.
2. 02Judge the relevant results for each query, with graded relevance where you can.
3. 03Measure your current search with recall@k and nDCG@k as a baseline.
4. 04Add embeddings in pgvector for two candidate models and compare them.
5. 05Combine with keyword search using RRF, add a reranker if needed, and measure each step.
6. 06Release behind a feature flag, then check online metrics against the baseline.

This is the approach we took for semantic product search in retail. If you want help setting up the evaluation or the search itself, how we work explains how we run projects like this.