Embeddings
An embedding represents a piece of content as a numerical vector positioned so that similar meanings end up close together.
What Is an Embedding?
Think of an embedding as a way of turning meaning into coordinates. An embedding model reads a piece of text (or an image, or audio) and outputs a vector — a long list of numbers — positioned in space such that content with similar meaning ends up near other content with similar meaning.
Text
Raw ContentCould also be an image or audio.
Embedding Model
Turns Meaning Into NumbersReads the content and outputs coordinates.
Vector
Positioned by MeaningSimilar meanings end up near each other.
That vector typically has hundreds or thousands of numbers (dimensions). No single dimension has an obvious human meaning on its own — the usefulness comes from the overall position, not any one number in it.
Semantic Similarity
Once two pieces of content are both represented as vectors, you can measure how close those vectors are to each other. Close vectors suggest related meaning; distant vectors suggest unrelated meaning — even if the underlying words share no letters in common.
"dog"
"puppy"
Vectors close together
"dog"
"stock market"
Vectors far apart
Warning
Close is not the same as identical. Two vectors being nearby means the model judged the content as related — not that they mean exactly the same thing, or that the closest match is automatically the correct or best answer.
Where Embeddings Are Used
- Semantic search — finding content that matches the meaning of a query, not just its exact keywords.
- Retrieval — the first step of a RAG system, finding relevant chunks to hand to a language model.
- Recommendations — surfacing items whose embeddings are close to something a user already engaged with.
- Clustering — grouping similar items together without predefined categories.
- Classification — using an item’s position relative to known examples to help predict a category.
The same idea extends beyond text: image embeddings position visually or conceptually similar images near each other, and multimodal embeddings can place a text description and a matching image close together in the same space.
Code Example
Generating an embedding and comparing two pieces of text conceptually — illustrative pseudocode, not a specific provider’s SDK:
vector_a = embedding_model.embed("What is our refund policy?")
vector_b = embedding_model.embed("How do I get my money back?")
similarity = cosine_similarity(vector_a, vector_b)
print(similarity) # closer to 1.0 means more semantically similarCommon Mistakes
Treating embeddings as exact meaning
A vector is a learned approximation of meaning based on training data, not a precise, verifiable representation of it.
Assuming the nearest vector is always the best result
The closest match by similarity score is not automatically the most useful or correct one for the task at hand.
Comparing embeddings from different models
Vectors from different embedding models are not positioned in the same space and generally cannot be compared to each other meaningfully.
Interview Question
What is an embedding, and why is it useful?
An embedding is a numerical vector representation of a piece of content — text, an image, or audio — produced so that content with similar meaning ends up close together in that vector space. It's useful because it lets you compare things by meaning instead of exact wording, which is the basis for semantic search, retrieval in RAG systems, recommendations, and clustering. The key caveat is that closeness in vector space indicates similarity, not correctness or exact equivalence.
What an interviewer may ask next
- How would you measure how similar two embeddings are?
- Why can’t you directly compare embeddings from two different models?
- How do embeddings connect to retrieval in a RAG system?
Explain It in 30 Seconds
An embedding turns a piece of content into a vector — a list of numbers — positioned so that similar meanings end up close together in that space. That lets you compare things by meaning instead of exact words, which powers semantic search, recommendations, and the retrieval step in RAG systems. Closeness signals similarity, not correctness, so the nearest match isn’t automatically the best one.