contenox

What are embeddings? Search by meaning, explained with code

Updated · Facts checked

UNDER THE HOOD What are embeddings?

Embeddings are lists of numbers that represent what a piece of text means. An embedding model turns “How to reset a forgotten password” into a vector of a few hundred to a few thousand numbers, and it turns “I can’t log in anymore” into a vector that points in nearly the same direction, even though the two sentences share no words. That is what makes search by meaning possible, and it is the foundation of RAG, semantic search and recommendations.

What it looks like

You have three help articles and a user question:

question:  "I can't log in anymore"

0.99  How to reset a forgotten password
0.95  Configure two-factor authentication
0.20  Refund policy for annual plans

A keyword search finds nothing: “log in” appears in none of the titles. Embeddings rank the password article first, because its meaning is closest. The numbers are similarity scores, and higher is closer. They are illustrative: a real model gives its own values.

How do embeddings work?

  1. A model reads the text and outputs a fixed-length list of numbers, the vector. Every text gets the same length from the same model, whether it is three words or a page.
  2. Similar meaning, similar direction. The model was trained so that texts used in similar ways end up close together.
  3. Similarity is a calculation. The usual measure is cosine similarity: how closely two vectors point the same way. OpenAI and Ollama both return vectors normalized to length 1, so cosine similarity is just multiplying the numbers pair by pair and adding them up.

The length depends on the model. Ollama’s docs say its embedding models typically return 384 to 1024 numbers; OpenAI’s text-embedding-3-small returns 1536 and text-embedding-3-large 3072.

Try it yourself

With Ollama installed, pull an embedding model:

ollama pull embeddinggemma

Save this as similar.py. It uses only Python’s standard library:

import json
import urllib.request

OLLAMA = "http://localhost:11434/api/embed"
MODEL = "embeddinggemma"

docs = [
    "How to reset a forgotten password",
    "Refund policy for annual plans",
    "Configure two-factor authentication",
]
query = "I can't log in anymore"


def embed(texts):
    body = json.dumps({"model": MODEL, "input": texts}).encode()
    req = urllib.request.Request(OLLAMA, data=body, headers={"Content-Type": "application/json"})
    with urllib.request.urlopen(req) as resp:
        return json.load(resp)["embeddings"]


def similarity(a, b):
    return sum(x * y for x, y in zip(a, b))


doc_vectors = embed(docs)
query_vector = embed([query])[0]

ranked = sorted(zip(docs, doc_vectors), key=lambda d: similarity(query_vector, d[1]), reverse=True)
for doc, vector in ranked:
    print(f"{similarity(query_vector, vector):.3f}  {doc}")

Run python similar.py. Your scores will differ from the example above, since every model has its own scale, but the order is what matters. Swap in your own documents and questions to get a feel for what “close in meaning” means to the model.

The similarity function is a plain dot product. That only works because Ollama returns normalized vectors; for vectors that aren’t, divide by both lengths.

What are embeddings used for?

  • Search by meaning: find the help article, ticket or document that answers a question, whatever words it uses.
  • RAG (retrieval-augmented generation): find the passages most relevant to a question and give them to a language model to answer from.
  • Duplicates and clusters: group similar bug reports, find near-identical tickets, spot repeated feedback.
  • Recommendations: “people who read this also read” based on content, not clicks.
  • Classification: compare a new text to examples of each category and pick the closest.

Which embedding model should I use?

  • Local: Ollama runs embedding models on your own machine, so the text never leaves it. embeddinggemma is the model Ollama’s own docs use.
  • Hosted: OpenAI’s text-embedding-3-small is the cheaper option, text-embedding-3-large the stronger one. Other providers offer their own.

Try two on your own data with the script above. A model that is great at English web text can be mediocre at your product’s jargon or your language.

The traps

The model-switch trap: vectors from different models can’t be compared, not even when they have the same length. Change the embedding model, and every stored vector must be computed again. Plan for a full re-index before you switch, and store which model made each vector.

We learned this running vector search for several tenants at once: the vector length, the distance measure and the model were the settings that could silently ruin a live index, so we made them impossible to change in place. Changing the model meant building a second index and cutting over.

  • Embedding too much at once. One vector for a whole 40-page manual averages everything into mush. Split documents into passages of a paragraph or a few, and embed each one. This is called chunking, and it decides more about search quality than most model choices.
  • Shortening vectors. OpenAI lets you request fewer numbers with its dimensions parameter. If you cut vectors yourself instead, you must normalize them again, or the similarity scores are wrong.
  • Similar is not correct. Embeddings find text that is about the same thing. They don’t check whether it is true, current or the opposite of what was asked. Treat the top results as candidates, not answers.

How do I combine meaning and keywords?

Embeddings are bad at exact strings. Ask for E1042, and a meaning-based search happily returns “card payments fail at checkout”, which is related but isn’t the error you asked about. Keyword search finds E1042 instantly but misses the same problem described in other words.

Most real search systems run both and merge the two result lists. The common way to merge is reciprocal rank fusion: each document gets 1 / (60 + its position) from each list, and the scores are added. A document near the top of either list ends up near the top of the merged one, and you never have to compare a keyword score with a similarity score.

import json
import urllib.request

OLLAMA = "http://localhost:11434/api/embed"
MODEL = "embeddinggemma"

docs = [
    "Error E1042: payment declined by the card issuer",
    "Card payments fail at checkout on mobile",
    "Reset your password from the sign-in page",
]


def embed(texts):
    body = json.dumps({"model": MODEL, "input": texts}).encode()
    req = urllib.request.Request(OLLAMA, data=body, headers={"Content-Type": "application/json"})
    with urllib.request.urlopen(req) as resp:
        return json.load(resp)["embeddings"]


def meaning_ranking(query):
    q = embed([query])[0]
    vectors = embed(docs)
    scores = [sum(a * b for a, b in zip(q, v)) for v in vectors]
    return sorted(range(len(docs)), key=lambda i: scores[i], reverse=True)


def keyword_ranking(query):
    words = set(query.lower().split())
    scores = [len(words & set(d.lower().replace(":", "").split())) for d in docs]
    return sorted(range(len(docs)), key=lambda i: scores[i], reverse=True)


def fuse(rankings, k=60):
    scores = {}
    for ranking in rankings:
        for position, doc in enumerate(ranking):
            scores[doc] = scores.get(doc, 0) + 1 / (k + position + 1)
    return sorted(scores, key=scores.get, reverse=True)


query = "why do I get E1042 when paying"
for i in fuse([keyword_ranking(query), meaning_ranking(query)]):
    print(docs[i])

The keyword part here is a simple word overlap, to keep the example short. In practice you use your database’s full-text search, such as PostgreSQL’s, and merge its ranking the same way.

One model, one index

Two things we learned running embeddings in production, one in a hosted document search and one in a local code index, both cheap to copy:

  • Record which model made the vectors, and refuse to mix. A vector from one embedding model compared against a vector from another gives a cosine that looks like a score and means nothing. Our index stored a fingerprint of the model and endpoint with every generation, and search returned “embedding route changed, rebuild the index” instead of results when they differed. Silent wrong answers were the alternative.
  • Never embed an empty string. Whitespace embeds to a vector that ranks arbitrary passages highly. The query path checks for it before the model is called, because the symptom on the other side is a confident answer to a question nobody asked.

Do AI coding agents use embeddings?

Less than you might expect. Cursor’s docs say it does not store embeddings of your codebase for search; its agent searches the code on your machine. Claude Code searches with ripgrep, a fast grep tool, as it works. The reasons are practical, and we hit all of them when Contenox still used embeddings for code:

  • The index goes stale. Code changes all the time. An embedding index is a copy that is out of date the moment you switch branches, while a search over the live files is always current.
  • It breaks prompt caching. Providers bill a repeated start of the conversation at a fraction of the price. Retrieved chunks differ with every question, so the prompt changes early, the cache misses, and every turn pays full price. With hosted models that burns a lot of tokens.
  • Re-indexing competes with your work. With local models the token bill doesn’t matter, but keeping the index fresh means embedding files again and again on the same machine your editor, builds and model already use. It is noticeable, and not in a good way.

Embeddings still help where exact words fail: finding the document that explains a concept, or the issue that describes the same bug in different words. That is why semantic search shows up in agents through extra tools, often as an MCP server, rather than built in.

Related

Esc to close