What Is an Embedding? Explained Without the Math

Posted on Sun 15 November 2026 in Beginner Guide

The Problem Computers Have With Meaning

Computers are good at exact matches. Search for "cat" and they'll find every document containing the word "cat." But they won't find documents about "kitten" or "feline" unless those exact words appear too. They have no concept that these words mean similar things.

Embeddings are how modern AI systems solve this. They turn words, sentences, and entire documents into numbers in a way that preserves meaning — so "cat" and "kitten" end up close together, and "cat" and "skyscraper" end up far apart.

Numbers That Capture Meaning

An embedding is a list of numbers — a vector — that represents a piece of text. Not random numbers, but numbers learned from training on massive amounts of text so that similar meanings produce similar vectors.

Here's the intuition. Imagine placing every word on a giant map. Words that appear in similar contexts get placed near each other. "Doctor" and "nurse" end up close. "Doctor" and "banana" end up far apart. "King" and "queen" end up close to each other, and both end up in the same region of the map as "emperor" and "monarch."

An embedding model does exactly this — except instead of a 2D map, it places text in a space with hundreds or thousands of dimensions. Each dimension captures some aspect of meaning, though no single dimension corresponds to a human-readable concept like "royalty" or "medical."

A Concrete Example

Suppose you embed three sentences:

  • "The dog chased the ball."
  • "A puppy ran after the toy."
  • "The quarterly earnings report exceeded expectations."

The first two will produce vectors that are close to each other — same general topic, overlapping vocabulary, similar meaning. The third will produce a vector far away from both.

This is what makes semantic search possible. Instead of asking "does this document contain the word I typed," you ask "is this document's vector close to my query's vector." Close vectors mean similar meaning, even if the exact words differ.

How Embedding Models Work (Without the Math)

An embedding model is a neural network trained to read text and output a fixed-length vector. The training process adjusts the network so that texts with similar meanings produce similar vectors and texts with different meanings produce different vectors.

The output vector is typically 768, 1536, or 3072 numbers long, depending on the model. This length is called the dimensionality of the embedding. More dimensions generally means the model can capture finer distinctions in meaning, at the cost of more storage and computation.

You don't need to understand how the network works internally. What matters is the guarantee it provides: similar meaning → similar vector.

What "Similar" Means: Cosine Distance

When you compare two embedding vectors to decide how similar they are, the most common method is cosine similarity. It measures the angle between two vectors — a small angle means similar direction means similar meaning.

You don't need to calculate this yourself. Vector databases do it for you. The key point: when a vector database says "these are the 5 most similar documents," it means these 5 documents had the smallest angle (highest cosine similarity) to your query vector.

Where Embeddings Show Up

Semantic search. The user types a question. You embed the question, search a vector database for the closest document vectors, and return the matching documents. This powers the retrieval step in RAG systems.

Recommendations. Embed product descriptions or articles. When a user reads something, find the items whose embeddings are closest. Similar vectors → similar content → relevant recommendations.

Clustering. Embed a large set of customer support tickets. Run a clustering algorithm on the vectors. Tickets about the same issue cluster together, even if customers phrased them differently.

Classification. Embed text, then train a simple classifier on top of the vectors. The embeddings do the hard work of turning language into something a classifier can operate on.

Duplicate detection. Two sentences with very similar vectors are probably saying the same thing in different words. Useful for deduplication and finding paraphrases.

Generating Embeddings With the OpenAI API

from openai import OpenAI

client = OpenAI()

def embed(text: str) -> list[float]:
    response = client.embeddings.create(
        input=text,
        model="text-embedding-3-small",
    )
    return response.data[0].embedding

vector = embed("What is an embedding?")
print(f"Vector length: {len(vector)}")   # 1536
print(f"First 5 values: {vector[:5]}")

text-embedding-3-small outputs 1536-dimensional vectors. text-embedding-3-large outputs 3072. For most applications, the small model is fast and cheap enough.

The API accepts a single string or a list of strings. Batching is more efficient than calling it once per document:

def embed_batch(texts: list[str]) -> list[list[float]]:
    response = client.embeddings.create(
        input=texts,
        model="text-embedding-3-small",
    )
    return [item.embedding for item in response.data]

One Critical Rule: Keep the Embedding Model Consistent

Whatever model you use to embed your documents at ingestion time, you must use the same model to embed queries at search time. Mixing models is like translating documents into French but searching in German — the vectors won't be comparable.

This is also why switching embedding models requires re-embedding everything. It's not a small change. Pick a model, stick with it, and document which model version was used so future you knows.

Embeddings vs. Keywords: When to Use Each

Keyword search is fast, transparent, and works well when users know the exact terms. If someone searches "invoice #4421," exact match is what you want.

Embeddings handle meaning, synonyms, and paraphrases. If someone types "I haven't received my payment," you want to find documents about "unpaid invoices" and "missing transfers" even if those exact words don't match.

Many production systems use both — keyword search for precision on specific terms, embeddings for semantic coverage — and merge the results. This is called hybrid search.

The Connection to RAG

The RAG pipeline from the previous post relies entirely on embeddings. When you call /ingest, each document chunk gets embedded and stored in ChromaDB. When you call /ask, the question gets embedded and ChromaDB finds the closest document vectors. The whole thing only works because the embedding model gives similar meanings similar vectors.

Understanding embeddings makes the entire RAG flow make sense: the retrieval step isn't magic, it's vector similarity search.

Quick Summary

  • An embedding is a list of numbers that represents text in a way that preserves meaning
  • Similar meanings → similar vectors, different meanings → different vectors
  • Embedding models are trained neural networks; you use them as a black box via an API
  • Cosine similarity measures how close two vectors are
  • Use the same model for ingestion and search — always
  • Embeddings power semantic search, RAG retrieval, recommendations, clustering, and classification